@@ -1,485 +0,0 @@
1-# Zen prompt caching implementation plan
2-
3-## Summary
4-
5-Goal: assume Zen forwards the same provider-specific cache settings that the Vercel AI SDK supports, and update mini-coder so Zen requests actually use prompt caching where possible.
6-
7-The key conclusion from the code review is that we already have most of the plumbing needed for **top-level provider options**, but we do **not** currently have:
8-
9-- any cache-specific settings or request shaping
10-- any per-message provider-option injection for Anthropic cache breakpoints
11-- any Gemini explicit-cache lifecycle support
12-- a stable system prompt prefix suitable for cache reuse
13-
14-The last point is the biggest blocker: `buildSystemPrompt()` currently injects a fresh `Current date/time` string on every turn, which makes the prompt prefix change every turn and substantially defeats provider-side prefix caching.
15-
16-## Findings
17-
18-### 1) Zen already runs through the same provider SDKs that expose cache settings
19-
20-Mini-coder routes Zen models through the same AI SDK providers Zen recommends:
21-
22-- `@ai-sdk/anthropic`, `@ai-sdk/google`, `@ai-sdk/openai` are installed in `package.json:22-24`
23-- Zen Anthropic provider factory: `src/llm-api/providers.ts:70-79`
24-- Zen OpenAI provider factory: `src/llm-api/providers.ts:82-91`
25-- Zen Google provider factory: `src/llm-api/providers.ts:94-103`
26-
27-That means the implementation path is to change **request construction**, not provider selection.
28-
29-### 2) `runTurn()` already has a top-level provider-options merge point
30-
31-The main request path is in `src/llm-api/turn.ts:644-824`.
32-
33-Important pieces:
34-
35-- current request options are assembled in `mergedProviderOptions` at `src/llm-api/turn.ts:749-772`
36-- those options are passed into `streamText()` at `src/llm-api/turn.ts:811-813`
37-- the final call is `streamText(streamOpts)` at `src/llm-api/turn.ts:824`
38-
39-This is already sufficient for providers that accept cache settings at the **function-call level**:
40-
41-- OpenAI prompt caching settings
42-- Google `cachedContent`
43-
44-It is **not** sufficient for Anthropic prompt caching, because Anthropic cache control is attached to **messages / content blocks**, not just the top-level call.
45-
46-### 3) The current system prompt shape is hostile to cache reuse
47-
48-`buildSystemPrompt()` is rebuilt on every turn in `src/agent/session-runner.ts:161-166`.
49-
50-Inside that function, the prompt includes:
51-
52-- working directory: `src/agent/system-prompt.ts:84-91`
53-- **current date/time** via `new Date().toLocaleString(...)`: `src/agent/system-prompt.ts:85`
54-
55-Because that timestamp changes every turn, the repeated prefix is not stable across turns. Even if Zen/OpenAI/Anthropic support provider-side prompt caching, the changing timestamp makes the highest-value prefix differ near the top of the prompt.
56-
57-This should be treated as a required prerequisite for meaningful caching.
58-
59-### 4) Anthropic needs message-level request shaping that we do not have today
60-
61-Right now, the request uses:
62-
63-- `messages: turnMessages` at `src/llm-api/turn.ts:776`
64-- optional top-level `system` at `src/llm-api/turn.ts:810`
65-- optional top-level `providerOptions` at `src/llm-api/turn.ts:811-813`
66-
67-There is currently no helper that walks `CoreMessage[]` and annotates specific messages or parts with provider options.
68-
69-That matters because Anthropic prompt caching needs `cacheControl` attached to stable prefix content rather than only to the whole request.
70-
71-### 5) Message persistence is already compatible with cache metadata
72-
73-Lossless message persistence is already a design invariant:
74-
75-- save verbatim JSON: `src/session/db/message-repo.ts:16-23`
76-- load verbatim JSON: `src/session/db/message-repo.ts:62-78`
77-- runner preserves model-authored message shape: `src/agent/session-runner.ts:198-203`
78-
79-This is good news for caching because provider metadata on messages or content parts can survive round-trips if we need it.
80-
81-### 6) Existing history sanitizers are not a blocker, but they must be respected
82-
83-Before requests are sent, history passes through:
84-
85-- Gemini tool-message sanitization: `src/llm-api/turn.ts:693-707`
86-- GPT commentary stripping: `src/llm-api/turn.ts:709-715`
87-- OpenAI item-id stripping: `src/llm-api/turn.ts:717-723`
88-- context pruning: `src/llm-api/turn.ts:729-736`
89-- tool-result compaction: `src/llm-api/turn.ts:738-747`
90-
91-Any cache annotation helper must run **after** the structural sanitizers that intentionally rewrite history, otherwise the cache annotations can be dropped or attached to messages that later get removed.
92-
93-### 7) Gemini explicit caching is only partially supported by the current stack
94-
95-The AI SDK Google provider supports `providerOptions.google.cachedContent`, which matches our existing top-level provider-options plumbing.
96-
97-However, mini-coder currently has no code that creates or refreshes Gemini cache objects. So for Gemini there are two different scopes:
98-
99-- **MVP**: support passing a pre-created `cachedContent` reference
100-- **follow-up**: create/manage Gemini cache objects automatically
101-
102-The MVP is easy. The full lifecycle is not.
103-
104-## Provider-by-provider implementation impact
105-
106-### OpenAI / Zen GPT models
107-
108-Expected to be the easiest path.
109-
110-What the code already gives us:
111-
112-- Zen GPT models go through `@ai-sdk/openai` Responses API routing: `src/llm-api/providers.ts:49-53`, `src/llm-api/providers.ts:82-91`
113-- request-level provider options already exist: `src/llm-api/turn.ts:749-813`
114-
115-What to add:
116-
117-- `providerOptions.openai.promptCacheRetention`
118-- optional `providerOptions.openai.promptCacheKey`
119-
120-Main caveat:
121-
122-- cache benefit is weak until the system prompt is made stable across turns
123-
124-### Anthropic / Zen Claude models
125-
126-Expected to require the most custom request shaping.
127-
128-What the code already gives us:
129-
130-- Zen Claude models go through `@ai-sdk/anthropic`: `src/llm-api/providers.ts:49-50`, `src/llm-api/providers.ts:70-79`
131-- messages are preserved losslessly in history and DB
132-
133-What is missing:
134-
135-- no message/part-level cache annotations
136-- top-level `system` prompt cannot currently carry Anthropic message-level cache control in our code path
137-
138-Likely implementation shape:
139-
140-- convert the request construction for Anthropic-family models so the stable system prompt is sent as a **system message** in `messages`, not only as the top-level `system` field
141-- annotate that system message and the chosen stable-prefix message(s) with Anthropic `cacheControl`
142-- keep the non-Anthropic path unchanged
143-
144-### Google / Zen Gemini models
145-
146-What the code already gives us:
147-
148-- Zen Gemini models go through `@ai-sdk/google`: `src/llm-api/providers.ts:49-53`, `src/llm-api/providers.ts:94-103`
149-- request-level provider options already exist: `src/llm-api/turn.ts:749-813`
150-
151-What is missing:
152-
153-- no cache-id source in config/settings
154-- no explicit cache object creation lifecycle
155-
156-Practical conclusion:
157-
158-- supporting a manually supplied `cachedContent` reference is straightforward
159-- fully automatic explicit Gemini caching should be treated as phase 2, not phase 1
160-
161-## Proposed implementation approach
162-
163-## Phase 1: make the prompt prefix stable enough to cache
164-
165-**Affected files:**
166-
167-- `src/agent/system-prompt.ts`
168-- `src/agent/session-runner.ts`
169-- possibly `src/session/manager.ts` / `src/session/db/session-repo.ts` if we want a persisted session time anchor
170-
171-### Change
172-
173-Replace the current per-turn timestamp in the system prompt with a stable session-level value.
174-
175-Recommended approach:
176-
177-- compute a `sessionTimeAnchor` once when the runner/session starts
178-- pass it into `buildSystemPrompt()`
179-- render it deterministically, ideally ISO-8601
180-- do **not** call `new Date().toLocaleString()` on every turn
181-
182-Why this matters:
183-
184-- it preserves the user-facing “current date/time” context
185-- it stops blowing up cache-prefix reuse near the top of the prompt
186-- it also makes prompts less locale-dependent and easier to test
187-
188-### Notes
189-
190-If we want resumed sessions to keep the same anchor, extend `ActiveSession` to carry `created_at` from `SessionRow` and use that. Otherwise, the runner can freeze the anchor at resume time.
191-
192-## Phase 2: add a cache-settings helper in the LLM layer
193-
194-**Affected file:** `src/llm-api/providers.ts`
195-
196-Add a dedicated helper alongside `getThinkingProviderOptions()` that returns cache settings based on the current model family, for example:
197-
198-- `getCachingProviderOptions(modelString, settings)` for request-level options
199-- `getCachingMode(modelString)` or equivalent model-family classification
200-
201-This keeps cache logic close to the existing reasoning-option logic and avoids scattering provider-name checks across `turn.ts`.
202-
203-Recommended outputs:
204-
205-- OpenAI / Zen GPT: top-level provider options
206-- Google / Zen Gemini: top-level provider options
207-- Anthropic / Zen Claude: classification only; actual message annotation happens in `turn.ts`
208-
209-## Phase 3: add runtime/user configuration for caching
210-
211-**Affected files:**
212-
213-- `src/session/db/settings-repo.ts`
214-- `src/session/db/index.ts`
215-- `src/agent/agent.ts`
216-- `src/agent/session-runner.ts`
217-- `src/index.ts`
218-- `src/cli/types.ts`
219-- `src/cli/commands.ts`
220-
221-### Suggested config surface
222-
223-Add a minimal cache configuration, not a large matrix.
224-
225-Recommended MVP settings:
226-
227-- `preferred_prompt_caching_enabled` → boolean, default `true`
228-- `preferred_openai_prompt_cache_retention` → `'in_memory' | '24h'`, default `'in_memory'`
229-- `preferred_google_cached_content` → string | null, default `null`
230-
231-I would **not** add Anthropic TTL configuration in v1; default to `ephemeral` and keep the surface small.
232-
233-### CLI surface
234-
235-Add a small `/cache` command family, similar to `/context`:
236-
237-- `/cache` → show current cache settings
238-- `/cache on|off`
239-- `/cache openai <in_memory|24h>`
240-- `/cache gemini <off|cachedContents/...>`
241-
242-Reason to add command support instead of env-only:
243-
244-- it matches how model/reasoning/context preferences already work
245-- settings are already persisted in the SQLite `settings` table: `src/session/db/connection.ts:61-64`
246-
247-## Phase 4: implement request-level cache options in `runTurn()`
248-
249-**Affected file:** `src/llm-api/turn.ts`
250-
251-### Change
252-
253-Extend the current `mergedProviderOptions` build at `src/llm-api/turn.ts:749-772` to also merge request-level cache provider options.
254-
255-For OpenAI-family requests:
256-
257-- set `openai.promptCacheRetention`
258-- optionally set `openai.promptCacheKey`
259-
260-For Google-family requests:
261-
262-- if configured, set `google.cachedContent`
263-
264-### Prompt cache key strategy
265-
266-If we add `promptCacheKey`, it should be derived from **stable prefix inputs**, not the full evolving conversation.
267-
268-Suggested inputs for the key:
269-
270-- provider/model family
271-- stable system prompt content
272-- tool schema signature
273-- active agent prompt
274-- local/global context file contents if present
275-
276-Do **not** include:
277-
278-- current user turn text
279-- latest tool results
280-- volatile timestamps
281-
282-This key should be deterministic and cheap to compute.
283-
284-If we want to keep v1 smaller, we can skip `promptCacheKey` initially and only set `promptCacheRetention`.
285-
286-## Phase 5: implement Anthropic cache breakpoints via message shaping
287-
288-**Affected file:** `src/llm-api/turn.ts`
289-
290-This is the most important structural change.
291-
292-### New helper(s)
293-
294-Add a helper that takes the already-sanitized/pruned/compacted request state and returns cache-annotated request messages, e.g.:
295-
296-- `prepareMessagesForCaching(...)`
297-- `annotateAnthropicCacheBreakpoints(...)`
298-
299-### Recommended sequence inside `runTurn()`
300-
301-Current flow is roughly:
302-
303-1. sanitize / prune / compact messages
304-2. merge top-level provider options
305-3. call `streamText()`
306-
307-New flow should be:
308-
309-1. sanitize / prune / compact messages
310-2. compute stable system prompt once
311-3. annotate message-level Anthropic cache breakpoints on the final request messages
312-4. merge top-level request-level provider options for OpenAI/Google
313-5. call `streamText()`
314-
315-### Anthropic-specific request shape
316-
317-Recommended v1 policy:
318-
319-- for Anthropic-family requests, send the system prompt as a **system message inside `messages`** rather than only top-level `system`
320-- attach `providerOptions.anthropic.cacheControl = { type: 'ephemeral' }` to that system message or its text part
321-- attach a second breakpoint at the last stable prefix message before the live turn, if the SDK/message shape supports it cleanly
322-
323-Why this is the right level:
324-
325-- it matches Anthropic’s message/block-oriented cache model
326-- it avoids trying to pretend Anthropic works like OpenAI request-level caching
327-- it keeps the provider-specific branching local to request construction
328-
329-### Important constraint
330-
331-Do not mutate persisted conversation history in place just to add request-only cache hints.
332-
333-Instead:
334-
335-- derive a request-local `cacheReadyMessages` array from `turnMessages`
336-- pass that to `streamText()`
337-- keep persisted history free of request-only annotations unless the SDK itself returns them on model-authored messages
338-
339-## Phase 6: improve logging so we can verify Zen forwarding behavior
340-
341-**Affected files:**
342-
343-- `src/llm-api/turn.ts`
344-- possibly `src/llm-api/providers.ts`
345-
346-The repo already logs outbound provider requests via the custom fetch wrapper in `src/llm-api/providers.ts:17-44`.
347-
348-Add explicit logs before the request for:
349-
350-- caching enabled/disabled
351-- cache mode selected by provider family
352-- OpenAI retention/key presence
353-- Gemini cachedContent presence
354-- Anthropic breakpoint count / message indexes
355-
356-This will make it easy to inspect `~/.config/mini-coder/api.log` and confirm that mini-coder is emitting the intended fields to Zen.
357-
358-## Detailed implementation steps
359-
360-1. Stabilize the system prompt timestamp.
361- - freeze a session-level time anchor
362- - thread it into `buildSystemPrompt()`
363- - stop recomputing a live locale timestamp every turn
364-
365-2. Add settings repo accessors for cache preferences.
366- - getter/setter for global on/off
367- - getter/setter for OpenAI retention mode
368- - getter/setter for optional Gemini cached-content id
369-
370-3. Thread cache preferences through startup and command context.
371- - load them in `src/index.ts`
372- - store them on `SessionRunner`
373- - expose setters/getters on `CommandContext`
374-
375-4. Add a small `/cache` command in `src/cli/commands.ts`.
376- - show status
377- - allow toggling on/off
378- - allow setting OpenAI retention
379- - allow setting/clearing Gemini cachedContent
380-
381-5. Add provider-family cache helpers in `src/llm-api/providers.ts`.
382- - keep model-family branching centralized
383- - mirror the existing reasoning-option style
384-
385-6. Add request-level cache option merging in `src/llm-api/turn.ts`.
386- - merge OpenAI and Google cache options into `mergedProviderOptions`
387- - preserve existing reasoning-option merging behavior
388-
389-7. Add Anthropic message-annotation helpers in `src/llm-api/turn.ts`.
390- - build request-local cache-ready messages
391- - insert Anthropic cache breakpoints after sanitization/pruning/compaction
392- - keep non-Anthropic request construction unchanged
393-
394-8. Add logging for cache configuration and annotations.
395-
396-9. Verify via focused tests and `api.log` inspection.
397-
398-## Tests to add
399-
400-**Affected files:**
401-
402-- `src/llm-api/turn.test.ts`
403-- likely a new focused settings/commands test if command support is added
404-
405-### Prompt stability tests
406-
407-- system prompt uses a fixed session anchor instead of a fresh per-turn timestamp
408-- prompt output is deterministic for the same anchor
409-
410-### Request-level caching tests
411-
412-- OpenAI-family model adds `promptCacheRetention` when caching is enabled
413-- OpenAI-family model does not add cache settings when caching is disabled
414-- Gemini-family model adds `cachedContent` when configured
415-- non-Google models ignore Gemini cachedContent settings
416-
417-### Anthropic message-shaping tests
418-
419-- Anthropic-family request gets cache annotations on the request-local messages
420-- non-Anthropic models do not get message-level cache annotations
421-- persisted `coreHistory` is not mutated by request-only cache shaping
422-- top-level `system` is not used for Anthropic if we switch that path to system-message form
423-
424-### Regression tests
425-
426-- existing Gemini thought-signature behavior remains intact
427-- existing GPT commentary stripping still works
428-- existing OpenAI item-id stripping still works
429-- cache annotations are applied after pruning/compaction, not before
430-
431-## Risks and caveats
432-
433-### 1) The biggest practical risk is cache misses from volatile prompt material
434-
435-Even after fixing the timestamp, cache hit quality still depends on:
436-
437-- system prompt stability
438-- active agent prompt changes
439-- project-context file changes
440-- tool-set/schema changes
441-- pruning/compaction decisions changing the replayed prefix
442-
443-That is normal, but it is another reason to keep the stable prefix as deterministic as possible.
444-
445-### 2) Anthropic cache-control message shape may require one iteration
446-
447-The repo already preserves provider metadata well, but Anthropic cache control is the least aligned with our current top-level request construction. I expect the first implementation to need tight unit-test coverage around message shape.
448-
449-### 3) Gemini full explicit caching should not block phase 1
450-
451-If we try to also build automatic Gemini cache creation in the same pass, the scope gets much larger. The good boundary is:
452-
453-- phase 1: consume a configured `cachedContent` id if present
454-- phase 2: automatic cache creation/refresh lifecycle
455-
456-## Recommendation on scope
457-
458-For the first implementation, I recommend this exact scope:
459-
460-1. **Fix prompt stability first**
461-2. **Implement OpenAI prompt caching** for Zen GPT models
462-3. **Implement Anthropic cache breakpoints** for Zen Claude models
463-4. **Add optional Gemini `cachedContent` pass-through**, but not automatic cache creation
464-5. **Add minimal CLI/settings support** so the feature can be toggled and inspected
465-
466-That scope gives real cost-saving potential for your actual flagship-heavy usage without dragging Gemini cache lifecycle management into the first pass.
467-
468-## Acceptance criteria
469-
470-- The system prompt no longer contains a fresh per-turn timestamp.
471-- Zen GPT requests include OpenAI cache settings when caching is enabled.
472-- Zen Claude requests include Anthropic cache breakpoints on the request message payload.
473-- Zen Gemini requests can include a configured `cachedContent` id.
474-- Request-only cache annotations do not mutate persisted conversation history.
475-- `api.log` clearly shows cache-related request fields being emitted.
476-- Focused unit tests cover prompt stability, request-level cache options, and Anthropic message shaping.
477-- Existing turn/history behavior remains green.
478-
479-## Sources
480-
481-- OpenAI provider docs: https://sdk.vercel.ai/providers/ai-sdk-providers/openai
482-- Anthropic provider docs: https://sdk.vercel.ai/providers/ai-sdk-providers/anthropic
483-- Google Generative AI provider docs: https://sdk.vercel.ai/providers/ai-sdk-providers/google-generative-ai
484-- AI SDK prompts/provider options docs: https://sdk.vercel.ai/docs/ai-sdk-core/prompts
485-- AI SDK caching docs: https://sdk.vercel.ai/docs/advanced/caching