xonecas.io
64875
Mc

mini-coder / commit e2a5df6

Refactor model info caching and provider metadata sync

Sean2026-03-05 e2a5df642de60e36651c46bf3c57c9bc7c05a0a3 parent 4051b95

10 files changed

This diff is too big to show in full. The first 1000 lines are below, and every file is listed with its counts.

TODO.md+7−12
@@ -7,17 +7,6 @@
77
88 ---
99
10-# Context window sizes are out of date
11-
12-CONTEXT_WINDOW_TABLE in providers.ts is out of date. Codex 5.3 and 5.2 have 400k tokens context limit not 128.
13-This list looks **VERY** wrong, let's get the facts from model.dev and update our list.
14-
15-`curl https://models.dev/api.json`
16-
17-Let's use the curl command above to update our table.
18-
19----
20-
2110 ## Subagents are not really used that much.
2211
2312 The main agent should be making use of these to ensure a clean and enduring context window. Search results, codebase reading, etc bloat the context.
@@ -52,4 +41,10 @@ We should have a closed loop feedback for LSP diagnostics on edits/reads.
5241
5342 This could potentially be a very big slowdown, waiting for updated diagnostics every edit. This also has bias downsides, stale diagnostics confuse the LLM, and can cause negative distractions.
5443
55-We need to do research first, but maybe we can leverage the hooks feature to achieve a similar result without the performance penalties? Needs brainstorming
44+We need to do research first, but maybe we can leverage the hooks feature to achieve a similar result without the performance penalties? Needs brainstorming
45+
46+---
47+
48+## Deferred fixes
49+
50+- model-info: in `resolveFromProviderRow`, when canonical capability exists but `contextWindow` is null, fall back to provider row context (`capability.contextWindow ?? row.contextWindow`).
models-info-refactor.md+178−0
@@ -0,0 +1,178 @@
1+# Models info refactor plan
2+
3+## Why this change
4+
5+Current model metadata is hardcoded in `src/llm-api/providers.ts`:
6+
7+- `CONTEXT_WINDOW_TABLE` (regex-based, out of date)
8+- `REASONING_MODELS` (regex-based capability detection)
9+
10+This causes drift and provider-dependent mismatches for the same underlying model.
11+
12+We should move to a runtime-generated, cached model metadata layer using:
13+
14+- provider model lists (what is actually available to this user)
15+- `https://models.dev/api.json` (capability source of truth)
16+
17+## Facts from models.dev (verified)
18+
19+- Top-level shape: object keyed by provider; each provider has `models` map.
20+- Model context limit is at `model.limit.context`.
21+- Reasoning support is at `model.reasoning` (currently boolean across all entries).
22+- `gpt-5.2` and `gpt-5.3-codex` entries show `limit.context = 400000`.
23+
24+## Goals
25+
26+1. Remove hardcoded context/reasoning capability tables.
27+2. Stop provider-specific regex matching for model capability lookup.
28+3. Build a canonical model capability cache in SQLite.
29+4. Refresh from network at app start (configured providers + models.dev).
30+5. Cache lifetime: at least 7 days (stale data still usable if refresh fails).
31+6. Keep runtime lookups fast and synchronous (`getContextWindow`, `supportsThinking`).
32+
33+## Non-goals (for this pass)
34+
35+- Rewriting provider transport/routing (`resolveModel`) behavior.
36+- Perfect aliasing for every third-party model name variant on day 1.
37+- Removing provider-specific payload encoding for thinking options (protocol shape still differs by provider).
38+
39+## Design
40+
41+### 1) New model metadata service
42+
43+Create `src/llm-api/model-info.ts` as the single entrypoint for model capabilities.
44+
45+Responsibilities:
46+
47+- load cached metadata from DB into in-memory maps (startup)
48+- refresh cache from remote sources (startup background task)
49+- expose sync lookups used by hot paths:
50+ - `getContextWindow(modelString): number | null`
51+ - `supportsThinking(modelString): boolean`
52+ - `resolveModelInfo(modelString): ModelInfo | null`
53+
54+### 2) DB schema additions (no destructive migration)
55+
56+**Important:** current DB version mismatch wipes DB, so do **not** bump `DB_VERSION` for this change.
57+Add tables via `CREATE TABLE IF NOT EXISTS` in existing `SCHEMA`.
58+
59+Proposed tables:
60+
61+- `model_capabilities`
62+ - `canonical_model_id TEXT PRIMARY KEY`
63+ - `context_window INTEGER`
64+ - `reasoning INTEGER NOT NULL` (0/1)
65+ - `source_provider TEXT` (models.dev provider key)
66+ - `raw_json TEXT` (optional, for forward compatibility)
67+ - `updated_at INTEGER NOT NULL`
68+
69+- `provider_models`
70+ - `provider TEXT NOT NULL`
71+ - `provider_model_id TEXT NOT NULL`
72+ - `display_name TEXT NOT NULL`
73+ - `canonical_model_id TEXT` (nullable if unmatched)
74+ - `context_window INTEGER` (provider-advertised fallback)
75+ - `free INTEGER` (0/1, optional)
76+ - `updated_at INTEGER NOT NULL`
77+ - `PRIMARY KEY (provider, provider_model_id)`
78+
79+- `model_info_state`
80+ - `key TEXT PRIMARY KEY`
81+ - `value TEXT NOT NULL`
82+ - (or reuse `settings` with namespaced keys)
83+
84+Store `last_models_dev_sync_at` and per-provider sync timestamps.
85+
86+### 3) Remote fetchers
87+
88+At app start, for configured providers, fetch provider model lists:
89+
90+- `zen`: `GET /zen/v1/models`
91+- `openai`: `GET /v1/models`
92+- `anthropic`: `GET /v1/models`
93+- `google`: `GET /v1beta/models` (normalize `models/<id>`)
94+- `ollama`: `GET /api/tags` (local-only, likely unmatched in models.dev)
95+
96+Fetch models.dev once:
97+
98+- `GET https://models.dev/api.json`
99+
100+Use short timeouts + fail-soft behavior.
101+
102+### 4) Cross-match algorithm (provider-independent capabilities)
103+
104+For each provider model entry:
105+
106+1. Normalize candidate IDs (lowercase, trim, remove known wrappers like `models/`).
107+2. Exact match against models.dev `model.id` index.
108+3. If no exact match, try deterministic alias candidates (e.g. basename after `/`) **only when unique**.
109+4. If ambiguous/unmatched, leave `canonical_model_id = NULL` and keep provider fallback context.
110+
111+Capability resolution order for runtime lookups:
112+
113+1. `provider_models(provider, provider_model_id) -> canonical_model_id`
114+2. `model_capabilities(canonical_model_id)`
115+3. fallback: provider-advertised context (if any)
116+4. else unknown (`null` context, `false` reasoning)
117+
118+This removes regex heuristics and makes capabilities independent from access provider.
119+
120+### 5) Startup behavior
121+
122+In `src/index.ts` main startup flow:
123+
124+- initialize model-info cache from DB synchronously
125+- kick off async refresh if stale (`> 7 days`) or missing
126+- do not block CLI startup on full refresh; reuse cached snapshot immediately
127+
128+Also let `/models` trigger a refresh when cache is stale and no refresh is in-flight.
129+
130+### 6) Refactor existing call sites
131+
132+- Move `getContextWindow` and `supportsThinking` out of regex tables in `providers.ts`.
133+- `providers.ts` should call model-info service for capability lookups.
134+- `fetchAvailableModels()` should read merged `provider_models` cache (and include current freshness status).
135+
136+### 7) Reasoning params handling
137+
138+models.dev currently gives `reasoning` capability (boolean), not effort enum/schema.
139+
140+Plan:
141+
142+- Use models.dev `reasoning` to decide whether thinking is supported.
143+- Keep provider-specific request-shape encoding in `getThinkingProviderOptions`.
144+- Remove model-family regex capability gates; rely on cached capability.
145+- Where per-model effort limits are unknown, clamp conservatively and fail-soft on provider rejection.
146+
147+(If models.dev later adds richer reasoning schema, we can read it from `raw_json` without schema rewrite.)
148+
149+## Testing plan (minimal, targeted)
150+
151+Add tests for:
152+
153+1. models.dev parser/indexing (`limit.context`, `reasoning`).
154+2. matching logic:
155+ - exact match
156+ - unique alias fallback
157+ - ambiguous alias -> no match
158+3. TTL logic (fresh vs stale > 7 days).
159+4. runtime lookup behavior from cached DB rows.
160+5. regression: same model via different providers resolves to same capabilities.
161+
162+## Rollout checklist
163+
164+1. Add DB tables + repo accessors.
165+2. Implement `model-info.ts` cache + refresh pipeline.
166+3. Wire startup init/refresh.
167+4. Replace hardcoded tables in `providers.ts`.
168+5. Update `/models` listing to use cached provider-model rows.
169+6. Run full checks:
170+ - `bun run typecheck && bun run format && bun run lint && bun test`
171+
172+## Acceptance criteria
173+
174+- No `CONTEXT_WINDOW_TABLE` or `REASONING_MODELS` hardcoded regex tables remain.
175+- `gpt-5.2`/`gpt-5.3*` context reflects 400k when matched from models.dev.
176+- Same model reached via different providers shows same context/reasoning capability.
177+- Metadata survives restarts and refreshes at most weekly (unless manually forced).
178+- App remains usable offline with last good cache.
src/cli/commands.ts+14−3
@@ -78,8 +78,8 @@ async function handleModel(ctx: CommandContext, args: string): Promise<void> {
7878 const idArg = parts[0] ?? "";
7979 let modelId = idArg;
8080 if (!idArg.includes("/")) {
81- const models = await fetchAvailableModels();
82- const match = models.find(
81+ const snapshot = await fetchAvailableModels();
82+ const match = snapshot.models.find(
8383 (m) => m.id.split("/").slice(1).join("/") === idArg || m.id === idArg,
8484 );
8585 if (match) {
@@ -120,7 +120,8 @@ async function handleModel(ctx: CommandContext, args: string): Promise<void> {
120120
121121 writeln(`${c.dim(" fetching models…")}`);
122122
123- const models = await fetchAvailableModels();
123+ const snapshot = await fetchAvailableModels();
124+ const models = snapshot.models;
124125
125126 // Clear the "fetching" line
126127 process.stdout.write("\x1B[1A\r\x1B[2K");
@@ -137,6 +138,16 @@ async function handleModel(ctx: CommandContext, args: string): Promise<void> {
137138 return;
138139 }
139140
141+ if (snapshot.stale) {
142+ const lastSync = snapshot.lastSyncAt
143+ ? new Date(snapshot.lastSyncAt).toLocaleString()
144+ : "never";
145+ const refreshTag = snapshot.refreshing ? " (refreshing in background)" : "";
146+ writeln(
147+ c.dim(` model metadata is stale (last sync: ${lastSync})${refreshTag}`),
148+ );
149+ }
150+
140151 // Group by provider
141152 const byProvider = new Map<string, typeof models>();
142153 for (const m of models) {
src/index.ts+7−0
@@ -10,7 +10,12 @@ import {
1010 writeln,
1111 } from "./cli/output.ts";
1212 import { initApiLog } from "./llm-api/api-log.ts";
13+import {
14+ initModelInfoCache,
15+ refreshModelInfoInBackground,
16+} from "./llm-api/model-info.ts";
1317 import { autoDiscoverModel } from "./llm-api/providers.ts";
18+
1419 import {
1520 getPreferredModel,
1621 getPreferredThinkingEffort,
@@ -22,6 +27,8 @@ import { getMostRecentSession, printSessionList } from "./session/manager.ts";
2227 registerTerminalCleanup();
2328 initErrorLog();
2429 initApiLog();
30+initModelInfoCache();
31+void refreshModelInfoInBackground().catch(() => {});
2532
2633 // ─── CLI argument parsing ─────────────────────────────────────────────────────
2734
src/llm-api/model-info.test.ts+173−0
@@ -0,0 +1,173 @@
1+import { describe, expect, test } from "bun:test";
2+import {
3+ MODEL_INFO_TTL_MS,
4+ buildModelMatchIndex,
5+ getProvidersToRefreshFromEnv,
6+ getRemoteProvidersFromEnv,
7+ isProviderVisibleInSnapshot,
8+ isStaleTimestamp,
9+ matchCanonicalModelId,
10+ parseModelsDevCapabilities,
11+ resolveModelInfoFromRows,
12+} from "./model-info.ts";
13+
14+describe("provider sync policy", () => {
15+ test("freshness providers exclude ollama", () => {
16+ expect(
17+ getRemoteProvidersFromEnv({
18+ OPENAI_API_KEY: "x",
19+ GOOGLE_API_KEY: "y",
20+ }),
21+ ).toEqual(["openai", "google"]);
22+ });
23+
24+ test("refresh providers include ollama", () => {
25+ expect(getProvidersToRefreshFromEnv({ OPENAI_API_KEY: "x" })).toEqual([
26+ "openai",
27+ "ollama",
28+ ]);
29+ });
30+
31+ test("snapshot visibility tracks configured providers", () => {
32+ expect(isProviderVisibleInSnapshot("openai", {})).toBe(false);
33+ expect(isProviderVisibleInSnapshot("openai", { OPENAI_API_KEY: "x" })).toBe(
34+ true,
35+ );
36+ expect(isProviderVisibleInSnapshot("ollama", {})).toBe(true);
37+ });
38+});
39+
40+describe("parseModelsDevCapabilities", () => {
41+ test("parses context and reasoning from models.dev payload", () => {
42+ const rows = parseModelsDevCapabilities(
43+ {
44+ openai: {
45+ models: {
46+ "gpt-5.2": {
47+ id: "gpt-5.2",
48+ limit: { context: 400_000 },
49+ reasoning: true,
50+ },
51+ "gpt-5.3-codex": {
52+ limit: { context: 400_000 },
53+ reasoning: true,
54+ },
55+ },
56+ },
57+ },
58+ 123,
59+ );
60+
61+ const byId = new Map(rows.map((row) => [row.canonical_model_id, row]));
62+ expect(byId.get("gpt-5.2")?.context_window).toBe(400_000);
63+ expect(byId.get("gpt-5.2")?.reasoning).toBe(1);
64+ expect(byId.get("gpt-5.3-codex")?.context_window).toBe(400_000);
65+ expect(byId.get("gpt-5.3-codex")?.reasoning).toBe(1);
66+ });
67+});
68+
69+describe("model match index", () => {
70+ test("matches exact canonical ids", () => {
71+ const index = buildModelMatchIndex(["openai/gpt-5.2"]);
72+ expect(matchCanonicalModelId("OPENAI/GPT-5.2", index)).toBe(
73+ "openai/gpt-5.2",
74+ );
75+ });
76+
77+ test("uses unique basename alias fallback", () => {
78+ const index = buildModelMatchIndex(["openai/gpt-5.2"]);
79+ expect(matchCanonicalModelId("gpt-5.2", index)).toBe("openai/gpt-5.2");
80+ });
81+
82+ test("does not match ambiguous basename aliases", () => {
83+ const index = buildModelMatchIndex(["openai/gpt-5.2", "custom/gpt-5.2"]);
84+ expect(matchCanonicalModelId("gpt-5.2", index)).toBeNull();
85+ });
86+});
87+
88+describe("staleness", () => {
89+ test("uses 7-day ttl", () => {
90+ const now = 1_000_000_000;
91+ expect(isStaleTimestamp(null, now)).toBe(true);
92+ expect(isStaleTimestamp(now - MODEL_INFO_TTL_MS + 1, now)).toBe(false);
93+ expect(isStaleTimestamp(now - MODEL_INFO_TTL_MS - 1, now)).toBe(true);
94+ });
95+});
96+
97+describe("resolveModelInfoFromRows", () => {
98+ const capabilities = [
99+ {
100+ canonical_model_id: "gpt-5.2",
101+ context_window: 400_000,
102+ reasoning: 1,
103+ source_provider: "openai",
104+ raw_json: null,
105+ updated_at: 1,
106+ },
107+ ];
108+
109+ const providers = [
110+ {
111+ provider: "openai",
112+ provider_model_id: "gpt-5.2",
113+ display_name: "GPT-5.2",
114+ canonical_model_id: "gpt-5.2",
115+ context_window: null,
116+ free: 0,
117+ updated_at: 1,
118+ },
119+ {
120+ provider: "zen",
121+ provider_model_id: "gpt-5.2",
122+ display_name: "GPT-5.2",
123+ canonical_model_id: "gpt-5.2",
124+ context_window: 128_000,
125+ free: 0,
126+ updated_at: 1,
127+ },
128+ {
129+ provider: "ollama",
130+ provider_model_id: "qwen2.5-coder",
131+ display_name: "qwen2.5-coder",
132+ canonical_model_id: null,
133+ context_window: 32_768,
134+ free: 0,
135+ updated_at: 1,
136+ },
137+ ];
138+
139+ test("uses canonical capabilities and provider fallback context", () => {
140+ expect(
141+ resolveModelInfoFromRows("openai/gpt-5.2", capabilities, providers),
142+ ).toEqual({
143+ canonicalModelId: "gpt-5.2",
144+ contextWindow: 400_000,
145+ reasoning: true,
146+ });
147+
148+ expect(
149+ resolveModelInfoFromRows("ollama/qwen2.5-coder", capabilities, providers),
150+ ).toEqual({
151+ canonicalModelId: null,
152+ contextWindow: 32_768,
153+ reasoning: false,
154+ });
155+ });
156+
157+ test("returns same capabilities across providers for same canonical model", () => {
158+ const openaiInfo = resolveModelInfoFromRows(
159+ "openai/gpt-5.2",
160+ capabilities,
161+ providers,
162+ );
163+ const zenInfo = resolveModelInfoFromRows(
164+ "zen/gpt-5.2",
165+ capabilities,
166+ providers,
167+ );
168+
169+ expect(openaiInfo).toEqual(zenInfo);
170+ expect(openaiInfo?.contextWindow).toBe(400_000);
171+ expect(openaiInfo?.reasoning).toBe(true);
172+ });
173+});
src/llm-api/model-info.ts+780−0
@@ -0,0 +1,780 @@
1+import {
2+ type ModelCapabilityRow,
3+ type ProviderModelRow,
4+ listModelCapabilities,
5+ listModelInfoState,
6+ listProviderModels,
7+ replaceModelCapabilities,
8+ replaceProviderModels,
9+ setModelInfoState,
10+} from "../session/db/model-info-repo.ts";
11+
12+const ZEN_BASE = "https://opencode.ai/zen/v1";
13+const OPENAI_BASE = "https://api.openai.com";
14+const ANTHROPIC_BASE = "https://api.anthropic.com";
15+const GOOGLE_BASE = "https://generativelanguage.googleapis.com/v1beta";
16+const MODELS_DEV_URL = "https://models.dev/api.json";
17+
18+const MODELS_DEV_SYNC_KEY = "last_models_dev_sync_at";
19+const PROVIDER_SYNC_KEY_PREFIX = "last_provider_sync_at:";
20+
21+export const MODEL_INFO_TTL_MS = 7 * 24 * 60 * 60 * 1000;
22+
23+export interface ModelInfo {
24+ canonicalModelId: string | null;
25+ contextWindow: number | null;
26+ reasoning: boolean;
27+}
28+
29+export interface LiveModel {
30+ id: string;
31+ displayName: string;
32+ provider: string;
33+ context?: number | undefined;
34+ free?: boolean | undefined;
35+}
36+
37+export interface AvailableModelsSnapshot {
38+ models: LiveModel[];
39+ stale: boolean;
40+ refreshing: boolean;
41+ lastSyncAt: number | null;
42+}
43+
44+interface RuntimeCapability {
45+ canonicalModelId: string;
46+ contextWindow: number | null;
47+ reasoning: boolean;
48+ sourceProvider: string | null;
49+}
50+
51+interface RuntimeProviderModel {
52+ provider: string;
53+ providerModelId: string;
54+ displayName: string;
55+ canonicalModelId: string | null;
56+ contextWindow: number | null;
57+ free: boolean;
58+}
59+
60+interface RuntimeCache {
61+ capabilitiesByCanonical: Map<string, RuntimeCapability>;
62+ providerModelsByKey: Map<string, RuntimeProviderModel>;
63+ providerModelUniqIndex: Map<string, string | null>;
64+ matchIndex: ModelMatchIndex;
65+ state: Map<string, string>;
66+}
67+
68+export interface ModelMatchIndex {
69+ exact: Map<string, string>;
70+ alias: Map<string, string | null>;
71+}
72+
73+interface ModelsDevEntry {
74+ canonicalModelId: string;
75+ contextWindow: number | null;
76+ reasoning: boolean;
77+ sourceProvider: string;
78+ rawJson: string | null;
79+}
80+
81+interface ProviderModelCandidate {
82+ providerModelId: string;
83+ displayName: string;
84+ contextWindow: number | null;
85+ free: boolean;
86+}
87+
88+interface ParsedModelString {
89+ provider: string | null;
90+ modelId: string;
91+}
92+
93+let runtimeCache: RuntimeCache = emptyRuntimeCache();
94+let loaded = false;
95+let refreshInFlight: Promise<void> | null = null;
96+
97+function emptyRuntimeCache(): RuntimeCache {
98+ return {
99+ capabilitiesByCanonical: new Map<string, RuntimeCapability>(),
100+ providerModelsByKey: new Map<string, RuntimeProviderModel>(),
101+ providerModelUniqIndex: new Map<string, string | null>(),
102+ matchIndex: {
103+ exact: new Map<string, string>(),
104+ alias: new Map<string, string | null>(),
105+ },
106+ state: new Map<string, string>(),
107+ };
108+}
109+
110+function isRecord(value: unknown): value is Record<string, unknown> {
111+ return typeof value === "object" && value !== null;
112+}
113+
114+function parseModelStringLoose(modelString: string): ParsedModelString {
115+ const slash = modelString.indexOf("/");
116+ if (slash === -1) {
117+ return { provider: null, modelId: modelString };
118+ }
119+ const provider = modelString.slice(0, slash).trim().toLowerCase();
120+ const modelId = modelString.slice(slash + 1);
121+ return { provider: provider || null, modelId };
122+}
123+
124+function providerModelKey(provider: string, modelId: string): string {
125+ return `${provider}/${modelId}`;
126+}
127+
128+function basename(value: string): string {
129+ const idx = value.lastIndexOf("/");
130+ return idx === -1 ? value : value.slice(idx + 1);
131+}
132+
133+export function normalizeModelId(modelId: string): string {
134+ let out = modelId.trim().toLowerCase();
135+ while (out.startsWith("models/")) {
136+ out = out.slice("models/".length);
137+ }
138+ return out;
139+}
140+
141+function parseContextWindow(model: Record<string, unknown>): number | null {
142+ const limit = model.limit;
143+ if (!isRecord(limit)) return null;
144+ const context = limit.context;
145+ if (typeof context !== "number" || !Number.isFinite(context)) return null;
146+ return Math.max(0, Math.trunc(context));
147+}
148+
149+export function parseModelsDevCapabilities(
150+ payload: unknown,
151+ updatedAt: number,
152+): ModelCapabilityRow[] {
153+ if (!isRecord(payload)) return [];
154+
155+ const merged = new Map<string, ModelsDevEntry>();
156+ for (const [provider, providerValue] of Object.entries(payload)) {
157+ if (!isRecord(providerValue)) continue;
158+ const models = providerValue.models;
159+ if (!isRecord(models)) continue;
160+ for (const [modelKey, modelValue] of Object.entries(models)) {
161+ if (!isRecord(modelValue)) continue;
162+ const explicitId =
163+ typeof modelValue.id === "string" && modelValue.id.trim().length > 0
164+ ? modelValue.id
165+ : modelKey;
166+ const canonicalModelId = normalizeModelId(explicitId);
167+ if (!canonicalModelId) continue;
168+ const contextWindow = parseContextWindow(modelValue);
169+ const reasoning = modelValue.reasoning === true;
170+ const rawJson = JSON.stringify(modelValue);
171+ const prev = merged.get(canonicalModelId);
172+ if (!prev) {
173+ merged.set(canonicalModelId, {
174+ canonicalModelId,
175+ contextWindow,
176+ reasoning,
177+ sourceProvider: provider,
178+ rawJson,
179+ });
180+ continue;
181+ }
182+ merged.set(canonicalModelId, {
183+ canonicalModelId,
184+ contextWindow: prev.contextWindow ?? contextWindow,
185+ reasoning: prev.reasoning || reasoning,
186+ sourceProvider: prev.sourceProvider,
187+ rawJson: prev.rawJson ?? rawJson,
188+ });
189+ }
190+ }
191+
192+ return Array.from(merged.values()).map((entry) => ({
193+ canonical_model_id: entry.canonicalModelId,
194+ context_window: entry.contextWindow,
195+ reasoning: entry.reasoning ? 1 : 0,
196+ source_provider: entry.sourceProvider,
197+ raw_json: entry.rawJson,
198+ updated_at: updatedAt,
199+ }));
200+}
201+
202+export function buildModelMatchIndex(
203+ canonicalModelIds: Iterable<string>,
204+): ModelMatchIndex {
205+ const exact = new Map<string, string>();
206+ const aliasCandidates = new Map<string, Set<string>>();
207+
208+ for (const rawCanonical of canonicalModelIds) {
209+ const canonical = normalizeModelId(rawCanonical);
210+ if (!canonical) continue;
211+ exact.set(canonical, canonical);
212+ const short = basename(canonical);
213+ if (!short) continue;
214+ let set = aliasCandidates.get(short);
215+ if (!set) {
216+ set = new Set<string>();
217+ aliasCandidates.set(short, set);
218+ }
219+ set.add(canonical);
220+ }
221+
222+ const alias = new Map<string, string | null>();
223+ for (const [short, candidates] of aliasCandidates) {
224+ if (candidates.size === 1) {
225+ for (const value of candidates) {
226+ alias.set(short, value);
227+ }
228+ } else {
229+ alias.set(short, null);
230+ }
231+ }
232+
233+ return { exact, alias };
234+}
235+
236+export function matchCanonicalModelId(
237+ providerModelId: string,
238+ index: ModelMatchIndex,
239+): string | null {
240+ const normalized = normalizeModelId(providerModelId);
241+ if (!normalized) return null;
242+ const exactMatch = index.exact.get(normalized);
243+ if (exactMatch) return exactMatch;
244+ const short = basename(normalized);
245+ if (!short) return null;
246+ const alias = index.alias.get(short);
247+ return alias ?? null;
248+}
249+
250+export function isStaleTimestamp(
251+ timestamp: number | null,
252+ now = Date.now(),
253+ ttlMs = MODEL_INFO_TTL_MS,
254+): boolean {
255+ if (timestamp === null) return true;
256+ return now - timestamp > ttlMs;
257+}
258+
259+export function buildRuntimeCache(
260+ capabilityRows: ModelCapabilityRow[],
261+ providerRows: ProviderModelRow[],
262+ stateRows: Array<{ key: string; value: string }>,
263+): RuntimeCache {
264+ const capabilitiesByCanonical = new Map<string, RuntimeCapability>();
265+ for (const row of capabilityRows) {
266+ const canonical = normalizeModelId(row.canonical_model_id);
267+ if (!canonical) continue;
268+ capabilitiesByCanonical.set(canonical, {
269+ canonicalModelId: canonical,
270+ contextWindow: row.context_window,
271+ reasoning: row.reasoning === 1,
272+ sourceProvider: row.source_provider,
273+ });
274+ }
275+
276+ const providerModelsByKey = new Map<string, RuntimeProviderModel>();
277+ const providerModelUniqIndex = new Map<string, string | null>();
278+ for (const row of providerRows) {
279+ const provider = row.provider.trim().toLowerCase();
280+ const providerModelId = normalizeModelId(row.provider_model_id);
281+ if (!provider || !providerModelId) continue;
282+ const key = providerModelKey(provider, providerModelId);
283+ providerModelsByKey.set(key, {
284+ provider,
285+ providerModelId,
286+ displayName: row.display_name,
287+ canonicalModelId: row.canonical_model_id
288+ ? normalizeModelId(row.canonical_model_id)
289+ : null,
290+ contextWindow: row.context_window,
291+ free: row.free === 1,
292+ });
293+ const prev = providerModelUniqIndex.get(providerModelId);
294+ if (prev === undefined) {
295+ providerModelUniqIndex.set(providerModelId, key);
296+ } else if (prev !== key) {
297+ providerModelUniqIndex.set(providerModelId, null);
298+ }
299+ }
300+
301+ const matchIndex = buildModelMatchIndex(capabilitiesByCanonical.keys());
302+ const state = new Map<string, string>();
303+ for (const row of stateRows) {
304+ state.set(row.key, row.value);
305+ }
306+
307+ return {
308+ capabilitiesByCanonical,
309+ providerModelsByKey,
310+ providerModelUniqIndex,
311+ matchIndex,
312+ state,
313+ };
314+}
315+
316+function loadCacheFromDb(): void {
317+ runtimeCache = buildRuntimeCache(
318+ listModelCapabilities(),
319+ listProviderModels(),
320+ listModelInfoState(),
321+ );
322+ loaded = true;
323+}
324+
325+function ensureLoaded(): void {
326+ if (!loaded) loadCacheFromDb();
327+}
328+
329+export function initModelInfoCache(): void {
330+ loadCacheFromDb();
331+}
332+
333+function parseStateInt(key: string): number | null {
334+ const raw = runtimeCache.state.get(key);
335+ if (!raw) return null;
336+ const value = Number.parseInt(raw, 10);
337+ if (!Number.isFinite(value)) return null;
338+ return value;
339+}
340+
341+export function getRemoteProvidersFromEnv(
342+ env: Record<string, string | undefined>,
343+): string[] {
344+ const providers: string[] = [];
345+ if (env.OPENCODE_API_KEY) providers.push("zen");
346+ if (env.OPENAI_API_KEY) providers.push("openai");
347+ if (env.ANTHROPIC_API_KEY) providers.push("anthropic");
348+ if (env.GOOGLE_API_KEY ?? env.GEMINI_API_KEY) providers.push("google");
349+ return providers;
350+}
351+
352+export function getProvidersToRefreshFromEnv(
353+ env: Record<string, string | undefined>,
354+): string[] {
355+ return [...getRemoteProvidersFromEnv(env), "ollama"];
356+}
357+
358+export function getVisibleProvidersForSnapshotFromEnv(
359+ env: Record<string, string | undefined>,
360+): ReadonlySet<string> {
361+ return new Set(getProvidersToRefreshFromEnv(env));
362+}
363+
364+export function isProviderVisibleInSnapshot(
365+ provider: string,
366+ env: Record<string, string | undefined>,
367+): boolean {
368+ return getVisibleProvidersForSnapshotFromEnv(env).has(provider);
369+}
370+
371+function getConfiguredProvidersForSync(): string[] {
372+ return getProvidersToRefreshFromEnv(process.env);
373+}
374+
375+function getProvidersRequiredForFreshness(): string[] {
376+ return getRemoteProvidersFromEnv(process.env);
377+}
378+
379+function getProviderSyncKey(provider: string): string {
380+ return `${PROVIDER_SYNC_KEY_PREFIX}${provider}`;
381+}
382+
383+export function isModelInfoStale(now = Date.now()): boolean {
384+ ensureLoaded();
385+ if (isStaleTimestamp(parseStateInt(MODELS_DEV_SYNC_KEY), now)) return true;
386+ for (const provider of getProvidersRequiredForFreshness()) {
387+ const providerSync = parseStateInt(getProviderSyncKey(provider));
388+ if (isStaleTimestamp(providerSync, now)) return true;
389+ }
390+ return false;
391+}
392+
393+function getLastSyncAt(): number | null {
394+ let latest = parseStateInt(MODELS_DEV_SYNC_KEY);
395+ for (const provider of getProvidersRequiredForFreshness()) {
396+ const value = parseStateInt(getProviderSyncKey(provider));
397+ if (value !== null && (latest === null || value > latest)) latest = value;
398+ }
399+ return latest;
400+}
401+
402+async function fetchJson(
403+ url: string,
404+ init: RequestInit,
405+ timeoutMs: number,
406+): Promise<unknown | null> {
407+ try {
408+ const response = await fetch(url, {
409+ ...init,
410+ signal: AbortSignal.timeout(timeoutMs),
411+ });
412+ if (!response.ok) return null;
413+ return (await response.json()) as unknown;
414+ } catch {
415+ return null;
416+ }
417+}
418+
419+async function fetchModelsDevPayload(): Promise<unknown | null> {
420+ return fetchJson(MODELS_DEV_URL, {}, 10_000);
421+}
422+
423+async function fetchZenModels(): Promise<ProviderModelCandidate[] | null> {
424+ const key = process.env.OPENCODE_API_KEY;
425+ if (!key) return null;
426+ const payload = await fetchJson(
427+ `${ZEN_BASE}/models`,
428+ { headers: { Authorization: `Bearer ${key}` } },
429+ 8_000,
430+ );
431+ if (!isRecord(payload)) return null;
432+ const data = payload.data;
433+ if (!Array.isArray(data)) return null;
434+ const out: ProviderModelCandidate[] = [];
435+ for (const item of data) {
436+ if (!isRecord(item) || typeof item.id !== "string") continue;
437+ const modelId = normalizeModelId(item.id);
438+ if (!modelId) continue;
439+ const contextWindow =
440+ typeof item.context_window === "number" &&
441+ Number.isFinite(item.context_window)
442+ ? Math.max(0, Math.trunc(item.context_window))
443+ : null;
444+ out.push({
445+ providerModelId: modelId,
446+ displayName: item.id,
447+ contextWindow,
448+ free:
449+ item.id.endsWith("-free") ||
450+ item.id === "gpt-5-nano" ||
451+ item.id === "big-pickle",
452+ });
453+ }
454+ return out;
455+}
456+
457+async function fetchOpenAIModels(): Promise<ProviderModelCandidate[] | null> {
458+ const key = process.env.OPENAI_API_KEY;
459+ if (!key) return null;
460+ const payload = await fetchJson(
461+ `${OPENAI_BASE}/v1/models`,
462+ { headers: { Authorization: `Bearer ${key}` } },
463+ 6_000,
464+ );
465+ if (!isRecord(payload) || !Array.isArray(payload.data)) return null;
466+ const out: ProviderModelCandidate[] = [];
467+ for (const item of payload.data) {
468+ if (!isRecord(item) || typeof item.id !== "string") continue;
469+ const modelId = normalizeModelId(item.id);
470+ if (!modelId) continue;
471+ out.push({
472+ providerModelId: modelId,
473+ displayName: item.id,
474+ contextWindow: null,
475+ free: false,
476+ });
477+ }
478+ return out;
479+}
480+
481+async function fetchAnthropicModels(): Promise<
482+ ProviderModelCandidate[] | null
483+> {
484+ const key = process.env.ANTHROPIC_API_KEY;
485+ if (!key) return null;
486+ const payload = await fetchJson(
487+ `${ANTHROPIC_BASE}/v1/models`,
488+ {
489+ headers: {
490+ "x-api-key": key,
491+ "anthropic-version": "2023-06-01",
492+ },
493+ },
494+ 6_000,
495+ );
496+ if (!isRecord(payload) || !Array.isArray(payload.data)) return null;
497+ const out: ProviderModelCandidate[] = [];
498+ for (const item of payload.data) {
499+ if (!isRecord(item) || typeof item.id !== "string") continue;
500+ const modelId = normalizeModelId(item.id);
501+ if (!modelId) continue;
502+ const displayName =
503+ typeof item.display_name === "string" &&
504+ item.display_name.trim().length > 0
505+ ? item.display_name
506+ : item.id;
507+ out.push({
508+ providerModelId: modelId,
509+ displayName,
510+ contextWindow: null,
511+ free: false,
512+ });
513+ }
514+ return out;
515+}
516+
517+async function fetchGoogleModels(): Promise<ProviderModelCandidate[] | null> {
518+ const key = process.env.GOOGLE_API_KEY ?? process.env.GEMINI_API_KEY;
519+ if (!key) return null;
520+ const payload = await fetchJson(
521+ `${GOOGLE_BASE}/models?key=${encodeURIComponent(key)}`,
522+ {},
523+ 6_000,
524+ );
525+ if (!isRecord(payload) || !Array.isArray(payload.models)) return null;
526+ const out: ProviderModelCandidate[] = [];
527+ for (const item of payload.models) {
528+ if (!isRecord(item) || typeof item.name !== "string") continue;
529+ const modelId = normalizeModelId(item.name);
530+ if (!modelId) continue;
531+ const displayName =
532+ typeof item.displayName === "string" && item.displayName.trim().length > 0
533+ ? item.displayName
534+ : modelId;
535+ const contextWindow =
536+ typeof item.inputTokenLimit === "number" &&
537+ Number.isFinite(item.inputTokenLimit)
538+ ? Math.max(0, Math.trunc(item.inputTokenLimit))
539+ : null;
540+ out.push({
541+ providerModelId: modelId,
542+ displayName,
543+ contextWindow,
544+ free: false,
545+ });
546+ }
547+ return out;
548+}
549+
550+async function fetchOllamaModels(): Promise<ProviderModelCandidate[] | null> {
551+ const base = process.env.OLLAMA_BASE_URL ?? "http://localhost:11434";
552+ const payload = await fetchJson(`${base}/api/tags`, {}, 3_000);
553+ if (!isRecord(payload) || !Array.isArray(payload.models)) return null;
554+ const out: ProviderModelCandidate[] = [];
555+ for (const item of payload.models) {
556+ if (!isRecord(item) || typeof item.name !== "string") continue;
Cut short: this commit's diff is over 1000 lines.
src/llm-api/providers.ts+14−119
Cut short: this commit's diff is over 1000 lines.
src/session/db/connection.ts+32−0
Cut short: this commit's diff is over 1000 lines.
src/session/db/index.ts+1−0
Cut short: this commit's diff is over 1000 lines.
src/session/db/model-info-repo.ts+128−0
Cut short: this commit's diff is over 1000 lines.