719
Le
lextide / files
lextide / AGENTS.md
6.1 KB
1# AGENTS.md 2 3## Project 4 5- Name: `lextide` 6- Goal: clean rewrite of `lowlight` 7- Core requirement: **drop-in replacement for `lowlight`** 8- Output contract: **HAST**, not HTML 9- Big extra feature: **streaming support** 10 11## Read first 12 13Before doing anything, read: 14 151. `spec.md` 162. `TODO.md` 17 18`spec.md` is source of truth for product and API direction. 19 20## Current repo state 21 22- Repo has Bun/TypeScript/Biome/Lefthook scaffold 23- Git repo is initialized 24- Repo now has working v0 implementation for sync highlighting, auto-detect, registry/aliases, and streaming 25- Built-in languages currently: bash, javascript, typescript, python, markdown 26- Tests cover public lowlight-style contract, builtin parity cases, and streaming parity 27- `spec.md` is still source of truth for future expansion and compatibility gaps 28 29## Non-negotiables 30 31- Treat `lextide` as replacement for `lowlight`, not as generic lexer experiment 32- HAST is public contract 33- Keep `lowlight`-compatible API shape for supported features 34- Streaming is required, not optional nice-to-have 35- Do not drift into HTML renderer work 36 37## Compatibility target 38 39Match `lowlight` where users touch it: 40 41- `createLowlight` 42- `common` 43- `all` 44- `highlight(language, value, options?)` 45- `highlightAuto(value, options?)` 46- `listLanguages()` 47- `register(name, grammar)` / `register(grammars)` 48- `registerAlias(language, alias)` / `registerAlias(aliases)` 49- `registered(aliasOrName)` 50 51Additive API allowed: 52 53- `stream(language, options?)` 54 55## Output shape 56 57Return HAST roots like `lowlight`: 58 59- root node with `type: 'root'` 60- text nodes for plain text 61- `span` element nodes for scopes 62- `properties.className` arrays 63- `root.data.language` 64- `root.data.relevance` 65 66Default class prefix: 67 68- `hljs-` 69 70## v0 language scope 71 72Start small: 73 74- bash 75- javascript 76- typescript 77- python 78- markdown 79 80`all` and `common` can be same at first. 81 82## Streaming direction 83 84Streaming should follow stable/unstable model inspired by `shiki-stream`: 85 86- stable output by default 87- optional recall mode for unstable tail 88- final streamed result must match sync `highlight()` output 89- line-oriented stability boundary is acceptable for v0 90- grammar state must survive across chunks 91 92## Planning guidance 93 94User does not want quiz-style planning. 95 96Good: 97 98- inspect repo 99- inspect relevant upstream behavior when needed 100- make concrete proposal 101- implement or update spec directly 102 103Bad: 104 105- turning obvious requirements into a questionnaire 106- reframing project away from drop-in `lowlight` replacement 107- over-indexing on abstract lexer theory 108 109When requirements are clear, act on them. 110 111## Tooling 112 113- ESM-only 114- Bun for package management, scripts, and tests 115- Biome for formatting, linting, and import sorting 116- Lefthook for pre-commit checks 117- TypeScript with strict mode 118 119## Scripts 120 121- `bun run build` — build publishable ESM output into `dist/` and emit declarations 122- `bun run format` — apply Biome formatting and import sorting 123- `bun run lint` — run Biome lint 124- `bun run typecheck` — run `tsc --noEmit` 125- `bun run test` — run Bun test runner 126- `bun run check` — run Biome check, typecheck, and tests 127- `bun run pack:dry` — dry-run npm publish with Bun 128 129## Workflow 130 131- run `bun install` after clone; `prepare` installs Lefthook 132- use TDD by default: write or update test first, confirm it fails, implement minimal fix, then rerun targeted test 133- prefer focused verification while iterating; do not spam `bun run check` after every small edit 134- run `bun run format` before commit or handoff 135- remember pre-commit already runs Biome check, typecheck, and tests automatically on commit 136- run `bun run check` for final manual verification when needed, not as repetitive loop during development 137- when user asks for release, publish, version bump, tag, or repo sync/push work, that counts as explicit permission to use git, create commits, create tags, and push them as needed for that release flow 138- otherwise, when committing code, if the user didn't explicitly ask you to, always review the diff with the user before committing. 139 140## Release workflow 141 142- package version must use semver `X.Y.Z` 143- before release, make sure `spec.md`, `README.md`, `AGENTS.md`, and `TODO.md` agree on current public API and scope 144- release flow is local command flow, not CI workflow setup unless user explicitly asks for automation 145- sync local `main` with `origin/main` 146- run `bun run check` 147- build release artifacts with `bun run build` 148- inspect publish payload with `bun run pack:dry` 149- create annotated tag `v<package.json version>` with changelog message since last release 150- push commit(s) and tag 151- publish locally with `bun publish --access public` 152- do not upload local npm tokens or wire publish secrets to GitHub unless user explicitly asks for that 153 154## Spec ambiguities 155 156When a requirement is unclear or missing from `spec.md`: 157 158- if the answer is obvious from `lowlight` behavior, match `lowlight` and note it 159- if it's a judgment call, make the call, implement it, and flag it to the user 160- do not block on questions that have a reasonable default 161 162## Working style 163 164- Keep scope tight 165- Prefer boring, explicit TypeScript 166- Reuse spec language and project terminology 167- Follow the three-layer architecture from `spec.md` (core engine → HAST builder → registry) 168- Verify claims against actual `lowlight` behavior when compatibility matters 169- If researching streaming behavior, check `shiki-stream` 170 171## If extending implementation 172 173Current implementation shape: 174 1751. `src/core/engine.ts` — generic scanner/state engine 1762. `src/hast.ts` — HAST builder + clone helpers 1773. `src/registry.ts` — lowlight-compatible API and registry 1784. `src/stream.ts` — streaming API; stable mode keeps last root child buffered until future input proves it is closed 1795. `src/languages/*.ts` — built-in language definitions 1806. `test/lextide.test.ts` — sync/auto/stream contract coverage 181 182When adding behavior, extend existing layers instead of bypassing them. 183 184## Session bootstrap checklist 185 186- read `spec.md` 187- read this file 188- inspect repo files, especially `package.json`, `tsconfig.json`, `biome.json`, and `lefthook.yml` 189- restate task in one or two concrete lines 190- do work 191- verify only what actually exists