xonecas.io
768
La

learning-ai-lua-code-model / commit 6852eee

chore: AGENTS.md initial commit checkpoit

- Use this commit as a start if you want to follow the same process.

xonecas2026-06-14 6852eee81a25746593eca9c038d1ed68fe32661d

1 file changed

AGENTS.md+120−0
@@ -0,0 +1,120 @@
1+# Lua Code Model Learning Project
2+
3+## Purpose
4+
5+This folder is for a first hands-on language-model project: a tiny GPT-style model focused on Lua code and Lua documentation.
6+
7+The goal is **learning**, not producing a useful coding assistant.
8+
9+We want to build a codebase that can be studied, modified, and experimented with by a software engineer who is new to AI/ML engineering.
10+
11+## High-level objective
12+
13+Train a small autoregressive language model on a focused Lua corpus and use it for simple code/documentation continuation experiments.
14+
15+The project should teach, through code:
16+
17+- dataset collection and cleaning
18+- train/validation splits
19+- tokenization
20+- next-token prediction
21+- transformer/GPT architecture basics
22+- training loops
23+- loss and perplexity
24+- sampling/generation
25+- overfitting and evaluation
26+- practical experiment tracking
27+
28+## Scope
29+
30+Initial scope:
31+
32+- Single language: **Lua**
33+- Small model
34+- Small dataset
35+- Local-first workflow
36+- Clear, readable code over maximum performance
37+- Prefer simple implementations before abstractions
38+
39+Out of scope for the first phase:
40+
41+- Building a generally useful coding assistant
42+- Training a large model
43+- Distributed training
44+- RLHF/RLAIF
45+- Tool use / agent behavior
46+- Multi-language datasets
47+- Massive web-scale scraping
48+
49+## Likely data sources
50+
51+Start with high-quality, legally usable Lua material such as:
52+
53+- Lua reference manual
54+- official Lua documentation
55+- official examples, where available
56+- small curated Lua source files from permissive open-source projects
57+
58+Before adding any dataset, record:
59+
60+- source URL or repository
61+- license
62+- what was included
63+- what was excluded
64+- any cleaning/filtering performed
65+
66+Avoid generated, vendored, minified, obfuscated, or low-quality code.
67+
68+## Preferred implementation approach
69+
70+Start with a minimal educational implementation inspired by `nanoGPT`:
71+
72+1. Build a tiny character-level or byte-level baseline first.
73+2. Train on a very small Lua corpus.
74+3. Verify the full pipeline end to end.
75+4. Add better tokenization later if useful.
76+5. Keep the code readable and heavily commented where concepts are new.
77+
78+Use PyTorch unless we explicitly decide otherwise.
79+
80+## Python and package management
81+
82+Use `uv` for Python environment and package management.
83+
84+Do not install new Python packages, system packages, frameworks, or tooling without explicit user approval first. When proposing a dependency, explain what it is for and why the project needs it.
85+
86+## Working style for future agents
87+
88+When working in this folder:
89+
90+- Read this `AGENTS.md` first.
91+- Keep changes small and explainable.
92+- Prefer incremental milestones over large rewrites.
93+- Do not assume the user already knows ML terminology; explain key concepts briefly.
94+- When adding code, favor clarity over cleverness.
95+- When adding dependencies, explain why they are needed.
96+- Ask before downloading large datasets or model weights.
97+- Ask before introducing Hugging Face, Lightning, WandB, or other larger frameworks.
98+- Preserve this as a learning codebase, not just a training script dump.
99+
100+## Suggested first milestones
101+
102+1. Create a minimal Python project structure.
103+2. Add scripts to download or manually ingest Lua documentation.
104+3. Build a simple text dataset file.
105+4. Implement a character-level tokenizer.
106+5. Implement a tiny GPT model in PyTorch.
107+6. Train until it overfits a tiny sample.
108+7. Train on the initial Lua corpus.
109+8. Add a simple sampling script.
110+9. Write notes explaining what each part does.
111+
112+## Success criteria for the first adventure
113+
114+We should be able to run a local command that:
115+
116+1. prepares a small Lua training corpus,
117+2. trains a tiny model,
118+3. saves a checkpoint,
119+4. generates Lua-like text from a prompt,
120+5. and leaves behind clear notes explaining what happened.