xonecas.io
768
La

learning-ai-lua-code-model / commit 9933a0f

feat: tinyGPT and end of first session

xonecas2026-06-14 9933a0ff8a7d8c6d0afe83fd473efb1a3bb0f486 parent 298e94a

5 files changed

This diff is too big to show in full. The first 1000 lines are below, and every file is listed with its counts.

README.md+18−1
@@ -13,6 +13,7 @@ This repo currently contains:
1313 - a standard-library-only preparation script in `src/lua_code_model/prepare_tiny_corpus.py`
1414 - a PyTorch bigram training script in `src/lua_code_model/train_bigram.py`
1515 - a PyTorch fixed-context MLP training script in `src/lua_code_model/train_mlp.py`
16+- a tiny GPT-style Transformer training script in `src/lua_code_model/train_gpt.py`
1617 - short learning notes in `notes/`
1718
1819 ## Install/sync dependencies
@@ -68,6 +69,20 @@ This trains a model that uses the previous 32 characters to predict the next cha
6869 checkpoints/mlp_tiny.pt
6970 ```
7071
72+## Train the tiny GPT-style Transformer model
73+
74+After preparing the corpus:
75+
76+```bash
77+PYTHONPATH=src uv run python -m lua_code_model.train_gpt
78+```
79+
80+This trains a small character-level GPT-style Transformer with causal self-attention and writes:
81+
82+```text
83+checkpoints/gpt_tiny.pt
84+```
85+
7186 Checkpoints are ignored by git because they are generated output.
7287
7388 ## Learning notes
@@ -75,7 +90,9 @@ Checkpoints are ignored by git because they are generated output.
7590 - `notes/001-first-dataset.md`: first dataset, tokenization, and train/validation split
7691 - `notes/002-bigram-pytorch.md`: tensors, batches, logits, loss, gradients, optimizers, and checkpoints
7792 - `notes/003-fixed-context-mlp.md`: embeddings, fixed context windows, hidden layers, and ReLU
93+- `notes/004-tiny-gpt.md`: token/position embeddings, self-attention, causal masks, Transformer blocks, residuals, and LayerNorm
94+- `notes/005-session-handoff.md`: current project state, learned concepts, run commands, and recommended next steps
7895
7996 ## Next likely step
8097
81-After the fixed-context MLP is clear, the next milestone is a tiny GPT-style Transformer with self-attention.
98+After the tiny GPT architecture is clear, the next milestone is to add a separate sampling script that loads checkpoints without retraining.
notes/004-tiny-gpt.md+213−0
@@ -0,0 +1,213 @@
1+# Tiny GPT-style Transformer
2+
3+This milestone introduces a very small GPT-style model.
4+
5+The model is still character-level and trained on the tiny Lua corpus, but the architecture is now Transformer-based.
6+
7+## Previous models
8+
9+Bigram:
10+
11+```text
12+current character -> next character
13+```
14+
15+Fixed-context MLP:
16+
17+```text
18+previous 32 characters -> flatten everything -> next character
19+```
20+
21+Tiny GPT:
22+
23+```text
24+previous characters -> self-attention -> next-character predictions at every position
25+```
26+
27+## Token embeddings and position embeddings
28+
29+The model starts with token IDs.
30+
31+Each token ID gets a learned token embedding:
32+
33+```text
34+what character is this?
35+```
36+
37+GPT also adds a position embedding:
38+
39+```text
40+where is this character in the block?
41+```
42+
43+The input representation is:
44+
45+```text
46+token embedding + position embedding
47+```
48+
49+This matters because order changes meaning. `local function` and `function local` contain similar characters, but in different positions.
50+
51+## Self-attention
52+
53+Self-attention lets each position gather information from other positions.
54+
55+For every token position, the model computes:
56+
57+- query: what am I looking for?
58+- key: what information do I contain?
59+- value: what information should I pass along?
60+
61+Attention compares queries to keys. Then it uses softmax to create attention weights. Finally, it mixes values using those weights.
62+
63+Conceptually:
64+
65+```text
66+attention scores = query dot key
67+attention weights = softmax(scores)
68+output = weighted sum of values
69+```
70+
71+So self-attention means:
72+
73+```text
74+for this position, choose which earlier positions are useful
75+```
76+
77+## Causal mask
78+
79+GPT predicts future tokens from past tokens.
80+
81+During training, the answer is present later in the sequence, so we must prevent cheating.
82+
83+The causal mask blocks each position from seeing future positions.
84+
85+For example, position 4 may see:
86+
87+```text
88+1, 2, 3, 4
89+```
90+
91+but not:
92+
93+```text
94+5, 6, 7, ...
95+```
96+
97+That is why the attention head uses a lower-triangular matrix called `tril`.
98+
99+## Multi-head attention
100+
101+One attention head can learn one kind of relationship.
102+
103+Multiple heads run in parallel so the model can learn several relationships at once.
104+
105+For code-like text, different heads might learn rough signals such as:
106+
107+- nearby character spelling
108+- indentation/newline patterns
109+- quote/string patterns
110+- names used earlier in the context
111+
112+We do not manually assign those roles. Training discovers useful attention behavior.
113+
114+## Transformer block
115+
116+A GPT block has two main parts:
117+
118+```text
119+causal self-attention
120+feed-forward MLP
121+```
122+
123+In code, one block does roughly:
124+
125+```python
126+x = x + self_attention(layer_norm(x))
127+x = x + feed_forward(layer_norm(x))
128+```
129+
130+The `+` parts are residual connections.
131+
132+## Residual connection
133+
134+A residual connection adds a layer's output back to its input.
135+
136+Instead of:
137+
138+```text
139+x -> layer -> new_x
140+```
141+
142+we do:
143+
144+```text
145+x -> layer -> change
146+x + change -> new_x
147+```
148+
149+This makes optimization easier because each layer can learn a useful adjustment instead of needing to rewrite the whole representation.
150+
151+## LayerNorm
152+
153+Layer normalization keeps each token representation numerically well-behaved.
154+
155+It helps training stay stable, especially as we stack multiple Transformer blocks.
156+
157+## Feed-forward network
158+
159+Transformers still use MLPs.
160+
161+Attention mixes information between positions. The feed-forward network then processes each position independently with nonlinear layers.
162+
163+So the split is:
164+
165+```text
166+attention: communicate across sequence positions
167+MLP: process each position's representation
168+```
169+
170+## Why GPT predicts at every position
171+
172+Given this input:
173+
174+```text
175+local
176+```
177+
178+The targets are shifted by one:
179+
180+```text
181+input: local
182+target: ocal?
183+```
184+
185+More generally, every position predicts the next token.
186+
187+The causal mask makes this valid because each position only uses earlier context.
188+
189+## Tiny model settings
190+
191+The script uses:
192+
193+```python
194+BLOCK_SIZE = 64
195+N_EMBD = 64
196+N_HEAD = 4
197+N_LAYER = 2
198+```
199+
200+Meaning:
201+
202+- the model can see up to 64 previous characters
203+- each token representation has 64 numbers
204+- each Transformer block has 4 attention heads
205+- there are 2 Transformer blocks
206+
207+This is tiny by modern standards, but it contains the same core architectural ideas as larger GPT models.
208+
209+## Expected behavior
210+
211+Because the corpus is very small, this model can overfit.
212+
213+That is okay for this milestone. The purpose is to understand the architecture and verify that training, generation, and checkpointing work with self-attention.
notes/005-session-handoff.md+265−0
@@ -0,0 +1,265 @@
1+# Session handoff: tiny Lua language model
2+
3+This note captures enough context to continue the project in a future session without rereading the full chat.
4+
5+## Project intent
6+
7+This is an educational language-model project for Lua code/documentation continuation experiments.
8+
9+The goal is learning the pieces of a GPT-style language model, not building a useful coding assistant.
10+
11+Current implementation is character-level, local-first, small, and intentionally readable.
12+
13+## Environment and dependency state
14+
15+- Package manager: `uv`
16+- Python project file: `pyproject.toml`
17+- Third-party dependency added so far: `torch`
18+- Do not add new dependencies without explicit user approval.
19+
20+Useful commands:
21+
22+```bash
23+uv sync
24+PYTHONPATH=src uv run python -m lua_code_model.prepare_tiny_corpus
25+PYTHONPATH=src uv run python -m lua_code_model.train_bigram
26+PYTHONPATH=src uv run python -m lua_code_model.train_mlp
27+PYTHONPATH=src uv run python -m lua_code_model.train_gpt
28+```
29+
30+PyTorch may print a warning about NumPy not being installed. The current scripts do not need NumPy, so we intentionally did not add it.
31+
32+## Data pipeline
33+
34+Raw input:
35+
36+```text
37+data/raw/tiny_lua_corpus.txt
38+```
39+
40+Source tracking:
41+
42+```text
43+data/SOURCES.md
44+```
45+
46+Preparation script:
47+
48+```text
49+src/lua_code_model/prepare_tiny_corpus.py
50+```
51+
52+Generated outputs:
53+
54+```text
55+data/processed/tiny_train.txt
56+data/processed/tiny_val.txt
57+data/processed/tiny_train_ids.txt
58+data/processed/tiny_val_ids.txt
59+data/processed/tiny_vocab.json
60+data/processed/tiny_metadata.json
61+```
62+
63+The current tokenizer is character-level:
64+
65+```text
66+text -> characters -> integer token IDs
67+```
68+
69+The current split is 90% train / 10% validation.
70+
71+Last observed prep stats:
72+
73+```text
74+raw characters: 1008
75+vocab size: 51
76+train tokens: 907
77+validation tokens: 101
78+```
79+
80+## Models implemented
81+
82+### 1. Bigram model
83+
84+File:
85+
86+```text
87+src/lua_code_model/train_bigram.py
88+```
89+
90+Checkpoint:
91+
92+```text
93+checkpoints/bigram_tiny.pt
94+```
95+
96+Concept:
97+
98+```text
99+current character -> next character
100+```
101+
102+Purpose: teach tensors, logits, loss, gradients, optimizer steps, checkpointing, and simple generation.
103+
104+### 2. Fixed-context MLP
105+
106+File:
107+
108+```text
109+src/lua_code_model/train_mlp.py
110+```
111+
112+Checkpoint:
113+
114+```text
115+checkpoints/mlp_tiny.pt
116+```
117+
118+Concept:
119+
120+```text
121+previous 32 characters -> embeddings -> flatten -> hidden layer -> logits -> next character
122+```
123+
124+Purpose: teach embeddings, hidden layers, ReLU activation, fixed context windows, temperature during generation, and overfitting.
125+
126+Last observed behavior: it memorized the tiny training corpus very strongly.
127+
128+### 3. Tiny GPT-style Transformer
129+
130+File:
131+
132+```text
133+src/lua_code_model/train_gpt.py
134+```
135+
136+Checkpoint:
137+
138+```text
139+checkpoints/gpt_tiny.pt
140+```
141+
142+Concept:
143+
144+```text
145+token IDs -> token embeddings + position embeddings -> Transformer blocks -> logits
146+```
147+
148+Implemented GPT concepts:
149+
150+- causal self-attention
151+- query/key/value projections
152+- causal mask using a lower-triangular matrix
153+- multi-head attention
154+- feed-forward MLP inside each Transformer block
155+- residual connections
156+- LayerNorm
157+- autoregressive generation
158+
159+Tiny GPT settings at last run:
160+
161+```python
162+BATCH_SIZE = 16
163+BLOCK_SIZE = 64
164+MAX_STEPS = 2_000
165+LEARNING_RATE = 1e-3
166+N_EMBD = 64
167+N_HEAD = 4
168+N_LAYER = 2
169+DROPOUT = 0.0
170+SAMPLE_TEMPERATURE = 0.8
171+```
172+
173+Last observed GPT stats:
174+
175+```text
176+train tokens: 907
177+val tokens: 101
178+vocab size: 51
179+parameters: 110387
180+step 0 train loss: 4.0653
181+step 2000 train loss: 0.0829
182+step 2000 val loss: 6.3499
183+```
184+
185+Interpretation: the model learned/memorized the tiny training data and overfit, which is expected at this stage.
186+
187+## Learning concepts already discussed
188+
189+The user has a working understanding of:
190+
191+- why we track data sources/licenses
192+- character tokenization
193+- train/validation split
194+- tensors as numeric arrays
195+- logits as raw prediction scores
196+- softmax converting logits to probabilities
197+- cross-entropy loss as model surprise/error signal
198+- gradients from `loss.backward()`
199+- optimizers using gradients to update parameters
200+- checkpoints as saved model weights/metadata
201+- training steps vs epochs
202+- activation functions, especially ReLU
203+- hidden layers as internal learned representation spaces
204+- temperature as `softmax(logits / temperature)` during sampling
205+- overfitting as low train loss but poor validation performance
206+- broad Transformer/GPT concepts
207+- GPT vs BERT vs BART-style model families
208+
209+## Important observed behavior
210+
211+Repeated or malformed generated strings from tiny GPT, such as repeated letters, are considered symptoms rather than obvious bugs.
212+
213+Likely causes:
214+
215+- character-level generation is fragile
216+- corpus is extremely small
217+- model is overfitting and partially recalling phrases
218+- probabilistic sampling with temperature can choose imperfect next characters
219+- no top-k/top-p/repetition-penalty decoding yet
220+
221+## Current notes files
222+
223+```text
224+notes/001-first-dataset.md
225+notes/002-bigram-pytorch.md
226+notes/003-fixed-context-mlp.md
227+notes/004-tiny-gpt.md
228+notes/005-session-handoff.md
229+```
230+
231+## Recommended next step
232+
233+Add a separate sampling script that loads checkpoints without retraining.
234+
235+Suggested file:
236+
237+```text
238+src/lua_code_model/sample.py
239+```
240+
241+Possible goals for that script:
242+
243+- choose checkpoint path or model type
244+- load saved vocabulary and model weights
245+- accept a prompt from the command line
246+- expose generation settings such as `--temperature` and `--max-new-tokens`
247+- generate text without running training again
248+
249+This would reinforce the distinction between:
250+
251+```text
252+training: update model weights
253+inference/sampling: use saved weights to generate text
254+```
255+
256+## Later possible milestones
257+
258+After a sampling script:
259+
260+1. Refactor duplicated helper code across training scripts.
261+2. Add top-k sampling to make generation controls more visible.
262+3. Add a small amount of real, legally usable Lua documentation/source data after recording it in `data/SOURCES.md`.
263+4. Compare model behavior before/after adding data.
264+5. Add experiment logs for runs and hyperparameters.
265+6. Eventually consider a byte-level or subword tokenizer, but not yet.
pi-session-2026-06-14T15-38-16-737Z_019ec6c8-9d21-7ec3-a6c8-7b20e72a4bf0.html+4256−0
@@ -0,0 +1,4256 @@
1+<!DOCTYPE html>
2+<html lang="en">
3+<head>
4+ <meta charset="UTF-8">
5+ <meta name="viewport" content="width=device-width, initial-scale=1.0">
6+ <title>Session Export</title>
7+ <style>
8+ :root {
9+ --accent: #8abeb7;
10+ --border: #5f87ff;
11+ --borderAccent: #00d7ff;
12+ --borderMuted: #505050;
13+ --success: #b5bd68;
14+ --error: #cc6666;
15+ --warning: #ffff00;
16+ --muted: #808080;
17+ --dim: #666666;
18+ --text: #d4d4d4;
19+ --thinkingText: #808080;
20+ --selectedBg: #3a3a4a;
21+ --userMessageBg: #343541;
22+ --userMessageText: #d4d4d4;
23+ --customMessageBg: #2d2838;
24+ --customMessageText: #d4d4d4;
25+ --customMessageLabel: #9575cd;
26+ --toolPendingBg: #282832;
27+ --toolSuccessBg: #283228;
28+ --toolErrorBg: #3c2828;
29+ --toolTitle: #d4d4d4;
30+ --toolOutput: #808080;
31+ --mdHeading: #f0c674;
32+ --mdLink: #81a2be;
33+ --mdLinkUrl: #666666;
34+ --mdCode: #8abeb7;
35+ --mdCodeBlock: #b5bd68;
36+ --mdCodeBlockBorder: #808080;
37+ --mdQuote: #808080;
38+ --mdQuoteBorder: #808080;
39+ --mdHr: #808080;
40+ --mdListBullet: #8abeb7;
41+ --toolDiffAdded: #b5bd68;
42+ --toolDiffRemoved: #cc6666;
43+ --toolDiffContext: #808080;
44+ --syntaxComment: #6A9955;
45+ --syntaxKeyword: #569CD6;
46+ --syntaxFunction: #DCDCAA;
47+ --syntaxVariable: #9CDCFE;
48+ --syntaxString: #CE9178;
49+ --syntaxNumber: #B5CEA8;
50+ --syntaxType: #4EC9B0;
51+ --syntaxOperator: #D4D4D4;
52+ --syntaxPunctuation: #D4D4D4;
53+ --thinkingOff: #505050;
54+ --thinkingMinimal: #6e6e6e;
55+ --thinkingLow: #5f87af;
56+ --thinkingMedium: #81a2be;
57+ --thinkingHigh: #b294bb;
58+ --thinkingXhigh: #d183e8;
59+ --bashMode: #b5bd68;
60+ --exportPageBg: #18181e;
61+ --exportCardBg: #1e1e24;
62+ --exportInfoBg: #3c3728;
63+ --body-bg: #18181e;
64+ --container-bg: #1e1e24;
65+ --info-bg: #3c3728;
66+ }
67+
68+ * { margin: 0; padding: 0; box-sizing: border-box; }
69+
70+ :root {
71+ --line-height: 18px; /* 12px font * 1.5 */
72+ --sidebar-width: 400px;
73+ --sidebar-min-width: 240px;
74+ --sidebar-max-width: 840px;
75+ --sidebar-resizer-width: 6px;
76+ }
77+
78+ body {
79+ font-family: ui-monospace, 'Cascadia Code', 'Source Code Pro', Menlo, Consolas, 'DejaVu Sans Mono', monospace;
80+ font-size: 12px;
81+ line-height: var(--line-height);
82+ color: var(--text);
83+ background: var(--body-bg);
84+ }
85+
86+ body.sidebar-resizing {
87+ cursor: col-resize;
88+ user-select: none;
89+ }
90+
91+ #app {
92+ display: flex;
93+ min-height: 100vh;
94+ }
95+
96+ /* Sidebar */
97+ #sidebar {
98+ width: var(--sidebar-width);
99+ min-width: var(--sidebar-width);
100+ max-width: var(--sidebar-width);
101+ background: var(--container-bg);
102+ flex-shrink: 0;
103+ display: flex;
104+ flex-direction: column;
105+ position: sticky;
106+ top: 0;
107+ height: 100vh;
108+ border-right: 1px solid var(--dim);
109+ }
110+
111+ #sidebar-resizer {
112+ width: var(--sidebar-resizer-width);
113+ flex-shrink: 0;
114+ position: sticky;
115+ top: 0;
116+ height: 100vh;
117+ cursor: col-resize;
118+ touch-action: none;
119+ background: transparent;
120+ border-right: 1px solid transparent;
121+ }
122+
123+ #sidebar-resizer:hover,
124+ body.sidebar-resizing #sidebar-resizer {
125+ background: var(--selectedBg);
126+ border-right-color: var(--dim);
127+ }
128+
129+ .sidebar-header {
130+ padding: 8px 12px;
131+ flex-shrink: 0;
132+ }
133+
134+ .sidebar-controls {
135+ padding: 8px 8px 4px 8px;
136+ }
137+
138+ .sidebar-search {
139+ width: 100%;
140+ box-sizing: border-box;
141+ padding: 4px 8px;
142+ font-size: 11px;
143+ font-family: inherit;
144+ background: var(--body-bg);
145+ color: var(--text);
146+ border: 1px solid var(--dim);
147+ border-radius: 3px;
148+ }
149+
150+ .sidebar-filters {
151+ display: flex;
152+ padding: 4px 8px 8px 8px;
153+ gap: 4px;
154+ align-items: center;
155+ flex-wrap: wrap;
156+ }
157+
158+ .sidebar-search:focus {
159+ outline: none;
160+ border-color: var(--accent);
161+ }
162+
163+ .sidebar-search::placeholder {
164+ color: var(--muted);
165+ }
166+
167+ .filter-btn {
168+ padding: 3px 8px;
169+ font-size: 10px;
170+ font-family: inherit;
171+ background: transparent;
172+ color: var(--muted);
173+ border: 1px solid var(--dim);
174+ border-radius: 3px;
175+ cursor: pointer;
176+ }
177+
178+ .filter-btn:hover {
179+ color: var(--text);
180+ border-color: var(--text);
181+ }
182+
183+ .filter-btn.active {
184+ background: var(--accent);
185+ color: var(--body-bg);
186+ border-color: var(--accent);
187+ }
188+
189+ .sidebar-close {
190+ display: none;
191+ padding: 3px 8px;
192+ font-size: 12px;
193+ font-family: inherit;
194+ background: transparent;
195+ color: var(--muted);
196+ border: 1px solid var(--dim);
197+ border-radius: 3px;
198+ cursor: pointer;
199+ margin-left: auto;
200+ }
201+
202+ .sidebar-close:hover {
203+ color: var(--text);
204+ border-color: var(--text);
205+ }
206+
207+ .tree-container {
208+ flex: 1;
209+ overflow: auto;
210+ padding: 4px 0;
211+ }
212+
213+ .tree-node {
214+ padding: 0 8px;
215+ cursor: pointer;
216+ display: flex;
217+ align-items: baseline;
218+ font-size: 11px;
219+ line-height: 13px;
220+ white-space: nowrap;
221+ }
222+
223+ .tree-node:hover {
224+ background: var(--selectedBg);
225+ }
226+
227+ .tree-node.active {
228+ background: var(--selectedBg);
229+ }
230+
231+ .tree-node.active .tree-content {
232+ font-weight: bold;
233+ }
234+
235+ .tree-node.in-path {
236+ background: color-mix(in srgb, var(--accent) 10%, transparent);
237+ }
238+
239+ .tree-node:not(.in-path) {
240+ opacity: 0.5;
241+ }
242+
243+ .tree-node:not(.in-path):hover {
244+ opacity: 1;
245+ }
246+
247+ .tree-prefix {
248+ color: var(--muted);
249+ flex-shrink: 0;
250+ font-family: monospace;
251+ white-space: pre;
252+ }
253+
254+ .tree-marker {
255+ color: var(--accent);
256+ flex-shrink: 0;
257+ }
258+
259+ .tree-content {
260+ color: var(--text);
261+ }
262+
263+ .tree-role-user {
264+ color: var(--accent);
265+ }
266+
267+ .tree-role-skill {
268+ color: var(--customMessageLabel);
269+ }
270+
271+ .tree-role-assistant {
272+ color: var(--success);
273+ }
274+
275+ .tree-role-tool {
276+ color: var(--muted);
277+ }
278+
279+ .tree-muted {
280+ color: var(--muted);
281+ }
282+
283+ .tree-error {
284+ color: var(--error);
285+ }
286+
287+ .tree-compaction {
288+ color: var(--borderAccent);
289+ }
290+
291+ .tree-branch-summary {
292+ color: var(--warning);
293+ }
294+
295+ .tree-custom-message {
296+ color: var(--customMessageLabel);
297+ }
298+
299+ .tree-status {
300+ padding: 4px 12px;
301+ font-size: 10px;
302+ color: var(--muted);
303+ flex-shrink: 0;
304+ }
305+
306+ /* Main content */
307+ #content {
308+ flex: 1;
309+ min-width: 0;
310+ overflow-y: auto;
311+ padding: var(--line-height) calc(var(--line-height) * 2);
312+ display: flex;
313+ flex-direction: column;
314+ align-items: center;
315+ }
316+
317+ #content > * {
318+ width: 100%;
319+ max-width: 800px;
320+ }
321+
322+ /* Help bar */
323+ .help-bar {
324+ font-size: 11px;
325+ color: var(--warning);
326+ margin-bottom: var(--line-height);
327+ display: flex;
328+ align-items: center;
329+ justify-content: space-between;
330+ flex-wrap: wrap;
331+ gap: 12px;
332+ }
333+
334+ .help-hint {
335+ flex: 1 1 240px;
336+ }
337+
338+ .help-actions {
339+ display: flex;
340+ align-items: center;
341+ flex-wrap: wrap;
342+ gap: 8px;
343+ }
344+
345+ .header-toggle-btn,
346+ .download-json-btn {
347+ font-size: 10px;
348+ padding: 2px 8px;
349+ background: var(--container-bg);
350+ border: 1px solid var(--border);
351+ border-radius: 3px;
352+ color: var(--text);
353+ cursor: pointer;
354+ font-family: inherit;
355+ }
356+
357+ .header-toggle-btn:hover,
358+ .download-json-btn:hover {
359+ background: var(--hover);
360+ border-color: var(--borderAccent);
361+ }
362+
363+ /* Header */
364+ .header {
365+ background: var(--container-bg);
366+ border-radius: 4px;
367+ padding: var(--line-height);
368+ margin-bottom: var(--line-height);
369+ }
370+
371+ .header h1 {
372+ font-size: 12px;
373+ font-weight: bold;
374+ color: var(--borderAccent);
375+ margin-bottom: var(--line-height);
376+ }
377+
378+ .header-info {
379+ display: flex;
380+ flex-direction: column;
381+ gap: 0;
382+ font-size: 11px;
383+ }
384+
385+ .info-item {
386+ color: var(--dim);
387+ display: flex;
388+ align-items: baseline;
389+ }
390+
391+ .info-label {
392+ font-weight: 600;
393+ margin-right: 8px;
394+ min-width: 100px;
395+ }
396+
397+ .info-value {
398+ color: var(--text);
399+ flex: 1;
400+ }
401+
402+ /* Messages */
403+ #messages {
404+ display: flex;
405+ flex-direction: column;
406+ gap: var(--line-height);
407+ }
408+
409+ .message-timestamp {
410+ font-size: 10px;
411+ color: var(--dim);
412+ opacity: 0.8;
413+ }
414+
415+ .user-message {
416+ background: var(--userMessageBg);
417+ color: var(--userMessageText);
418+ padding: var(--line-height);
419+ border-radius: 4px;
420+ position: relative;
421+ }
422+
423+ .assistant-message {
424+ padding: 0;
425+ position: relative;
426+ }
427+
428+ /* Copy link button - appears on hover */
429+ .copy-link-btn {
430+ position: absolute;
431+ top: 8px;
432+ right: 8px;
433+ width: 28px;
434+ height: 28px;
435+ padding: 6px;
436+ background: var(--container-bg);
437+ border: 1px solid var(--dim);
438+ border-radius: 4px;
439+ color: var(--muted);
440+ cursor: pointer;
441+ opacity: 0;
442+ transition: opacity 0.15s, background 0.15s, color 0.15s;
443+ display: flex;
444+ align-items: center;
445+ justify-content: center;
446+ z-index: 10;
447+ }
448+
449+ .user-message:hover .copy-link-btn,
450+ .assistant-message:hover .copy-link-btn,
451+ .skill-user-entry:hover .copy-link-btn {
452+ opacity: 1;
453+ }
454+
455+ .copy-link-btn:hover {
456+ background: var(--accent);
457+ color: var(--body-bg);
458+ border-color: var(--accent);
459+ }
460+
461+ .copy-link-btn.copied {
462+ background: var(--success, #22c55e);
463+ color: white;
464+ border-color: var(--success, #22c55e);
465+ }
466+
467+ /* Highlight effect for deep-linked messages */
468+ .user-message.highlight,
469+ .assistant-message.highlight {
470+ animation: highlight-pulse 2s ease-out;
471+ }
472+
473+ @keyframes highlight-pulse {
474+ 0% {
475+ box-shadow: 0 0 0 3px var(--accent);
476+ }
477+ 100% {
478+ box-shadow: 0 0 0 0 transparent;
479+ }
Cut short: this commit's diff is over 1000 lines.
src/lua_code_model/train_gpt.py+479−0
Cut short: this commit's diff is over 1000 lines.