@@ -1,178 +0,0 @@
1-# Models info refactor plan
2-
3-## Why this change
4-
5-Current model metadata is hardcoded in `src/llm-api/providers.ts`:
6-
7-- `CONTEXT_WINDOW_TABLE` (regex-based, out of date)
8-- `REASONING_MODELS` (regex-based capability detection)
9-
10-This causes drift and provider-dependent mismatches for the same underlying model.
11-
12-We should move to a runtime-generated, cached model metadata layer using:
13-
14-- provider model lists (what is actually available to this user)
15-- `https://models.dev/api.json` (capability source of truth)
16-
17-## Facts from models.dev (verified)
18-
19-- Top-level shape: object keyed by provider; each provider has `models` map.
20-- Model context limit is at `model.limit.context`.
21-- Reasoning support is at `model.reasoning` (currently boolean across all entries).
22-- `gpt-5.2` and `gpt-5.3-codex` entries show `limit.context = 400000`.
23-
24-## Goals
25-
26-1. Remove hardcoded context/reasoning capability tables.
27-2. Stop provider-specific regex matching for model capability lookup.
28-3. Build a canonical model capability cache in SQLite.
29-4. Refresh from network at app start (configured providers + models.dev).
30-5. Cache lifetime: at least 7 days (stale data still usable if refresh fails).
31-6. Keep runtime lookups fast and synchronous (`getContextWindow`, `supportsThinking`).
32-
33-## Non-goals (for this pass)
34-
35-- Rewriting provider transport/routing (`resolveModel`) behavior.
36-- Perfect aliasing for every third-party model name variant on day 1.
37-- Removing provider-specific payload encoding for thinking options (protocol shape still differs by provider).
38-
39-## Design
40-
41-### 1) New model metadata service
42-
43-Create `src/llm-api/model-info.ts` as the single entrypoint for model capabilities.
44-
45-Responsibilities:
46-
47-- load cached metadata from DB into in-memory maps (startup)
48-- refresh cache from remote sources (startup background task)
49-- expose sync lookups used by hot paths:
50- - `getContextWindow(modelString): number | null`
51- - `supportsThinking(modelString): boolean`
52- - `resolveModelInfo(modelString): ModelInfo | null`
53-
54-### 2) DB schema additions (no destructive migration)
55-
56-**Important:** current DB version mismatch wipes DB, so do **not** bump `DB_VERSION` for this change.
57-Add tables via `CREATE TABLE IF NOT EXISTS` in existing `SCHEMA`.
58-
59-Proposed tables:
60-
61-- `model_capabilities`
62- - `canonical_model_id TEXT PRIMARY KEY`
63- - `context_window INTEGER`
64- - `reasoning INTEGER NOT NULL` (0/1)
65- - `source_provider TEXT` (models.dev provider key)
66- - `raw_json TEXT` (optional, for forward compatibility)
67- - `updated_at INTEGER NOT NULL`
68-
69-- `provider_models`
70- - `provider TEXT NOT NULL`
71- - `provider_model_id TEXT NOT NULL`
72- - `display_name TEXT NOT NULL`
73- - `canonical_model_id TEXT` (nullable if unmatched)
74- - `context_window INTEGER` (provider-advertised fallback)
75- - `free INTEGER` (0/1, optional)
76- - `updated_at INTEGER NOT NULL`
77- - `PRIMARY KEY (provider, provider_model_id)`
78-
79-- `model_info_state`
80- - `key TEXT PRIMARY KEY`
81- - `value TEXT NOT NULL`
82- - (or reuse `settings` with namespaced keys)
83-
84-Store `last_models_dev_sync_at` and per-provider sync timestamps.
85-
86-### 3) Remote fetchers
87-
88-At app start, for configured providers, fetch provider model lists:
89-
90-- `zen`: `GET /zen/v1/models`
91-- `openai`: `GET /v1/models`
92-- `anthropic`: `GET /v1/models`
93-- `google`: `GET /v1beta/models` (normalize `models/<id>`)
94-- `ollama`: `GET /api/tags` (local-only, likely unmatched in models.dev)
95-
96-Fetch models.dev once:
97-
98-- `GET https://models.dev/api.json`
99-
100-Use short timeouts + fail-soft behavior.
101-
102-### 4) Cross-match algorithm (provider-independent capabilities)
103-
104-For each provider model entry:
105-
106-1. Normalize candidate IDs (lowercase, trim, remove known wrappers like `models/`).
107-2. Exact match against models.dev `model.id` index.
108-3. If no exact match, try deterministic alias candidates (e.g. basename after `/`) **only when unique**.
109-4. If ambiguous/unmatched, leave `canonical_model_id = NULL` and keep provider fallback context.
110-
111-Capability resolution order for runtime lookups:
112-
113-1. `provider_models(provider, provider_model_id) -> canonical_model_id`
114-2. `model_capabilities(canonical_model_id)`
115-3. fallback: provider-advertised context (if any)
116-4. else unknown (`null` context, `false` reasoning)
117-
118-This removes regex heuristics and makes capabilities independent from access provider.
119-
120-### 5) Startup behavior
121-
122-In `src/index.ts` main startup flow:
123-
124-- initialize model-info cache from DB synchronously
125-- kick off async refresh if stale (`> 7 days`) or missing
126-- do not block CLI startup on full refresh; reuse cached snapshot immediately
127-
128-Also let `/models` trigger a refresh when cache is stale and no refresh is in-flight.
129-
130-### 6) Refactor existing call sites
131-
132-- Move `getContextWindow` and `supportsThinking` out of regex tables in `providers.ts`.
133-- `providers.ts` should call model-info service for capability lookups.
134-- `fetchAvailableModels()` should read merged `provider_models` cache (and include current freshness status).
135-
136-### 7) Reasoning params handling
137-
138-models.dev currently gives `reasoning` capability (boolean), not effort enum/schema.
139-
140-Plan:
141-
142-- Use models.dev `reasoning` to decide whether thinking is supported.
143-- Keep provider-specific request-shape encoding in `getThinkingProviderOptions`.
144-- Remove model-family regex capability gates; rely on cached capability.
145-- Where per-model effort limits are unknown, clamp conservatively and fail-soft on provider rejection.
146-
147-(If models.dev later adds richer reasoning schema, we can read it from `raw_json` without schema rewrite.)
148-
149-## Testing plan (minimal, targeted)
150-
151-Add tests for:
152-
153-1. models.dev parser/indexing (`limit.context`, `reasoning`).
154-2. matching logic:
155- - exact match
156- - unique alias fallback
157- - ambiguous alias -> no match
158-3. TTL logic (fresh vs stale > 7 days).
159-4. runtime lookup behavior from cached DB rows.
160-5. regression: same model via different providers resolves to same capabilities.
161-
162-## Rollout checklist
163-
164-1. Add DB tables + repo accessors.
165-2. Implement `model-info.ts` cache + refresh pipeline.
166-3. Wire startup init/refresh.
167-4. Replace hardcoded tables in `providers.ts`.
168-5. Update `/models` listing to use cached provider-model rows.
169-6. Run full checks:
170- - `bun run typecheck && bun run format && bun run lint && bun test`
171-
172-## Acceptance criteria
173-
174-- No `CONTEXT_WINDOW_TABLE` or `REASONING_MODELS` hardcoded regex tables remain.
175-- `gpt-5.2`/`gpt-5.3*` context reflects 400k when matched from models.dev.
176-- Same model reached via different providers shows same context/reasoning capability.
177-- Metadata survives restarts and refreshes at most weekly (unless manually forced).
178-- App remains usable offline with last good cache.