xonecas.io
64875
Mc

mini-coder / commit e65e10e

feat: Auto compaction at 90% context pressure

xonecas2026-04-21 e65e10ea453022e628576e5edb83043c83005d3a parent 2af109f

13 files changed

This diff is too big to show in full. The first 1000 lines are below, and every file is listed with its counts.

PROGRESS.md+3−77
@@ -1,79 +1,5 @@
11 # CURRENT TASK PROGRESS
22
3-## Baseline
4-
5-- Full baseline completed from `terminal-bench/jobs/benchmark-baseline-full-2026-04-19__12-30-08`.
6-- Trial result: `126/178` passed.
7-- Task split: `53/89` passed both attempts, `20/89` split attempts, `16/89` failed both attempts.
8-
9-## Fast suite for the next iteration
10-
11-Use these as the first local one-attempt loop before any broader rerun:
12-
13-- `configure-git-webserver` — failed both attempts, ~195s avg
14-- `large-scale-text-editing` — failed both attempts, ~205s avg
15-- `gcode-to-text` — failed both attempts, ~322s avg
16-
17-## Completed step: baseline analysis
18-
19-- benchmark symptom:
20- - `configure-git-webserver` did real work but left a reusable script instead of the live configured end state; verifier got HTTP 000.
21- - `large-scale-text-editing` produced the transformed data but still missed the exact artifact contract the verifier checked.
22- - `gcode-to-text` spent the full turn analyzing the geometry and never wrote the required output file before the session died.
23-- general behavior gap:
24- - mini-coder is not end-state / contract driven enough in one-shot tasks. It over-invests in reusable artifacts and open-ended analysis instead of finishing the exact requested state and explicit deliverables first.
25-- why this should help outside Terminal-Bench:
26- - real users also want the current repo/environment fixed now, not a helper script, a near-equivalent artifact, or a long unfinished investigation.
27-- hypothesis:
28- - bias the agent toward the smallest direct completion path, and make it re-check the exact requested deliverables / end state before optional packaging, docs, or extra tooling.
29-- verification method:
30- - make one small product change, then run a one-attempt local fast suite on the three tasks above with `mc --json` logs kept for behavior review.
31-- DECISION:
32- - KEEP — use this fast suite and hypothesis for the next implementation loop.
33-
34-## Completed step: end-state prompt reinforcement
35-
36-- change made:
37- - tightened the core system prompt so the agent explicitly prefers the smallest path that leaves the requested end state already true, and re-checks named deliverables before finishing.
38-- benchmark symptom:
39- - the fast-suite failures were still consistent with “close enough” completion: helper scripts instead of the live state, implied contract satisfaction instead of the exact artifact, and long analysis before writing the requested file.
40-- general behavior gap:
41- - even with strong existing instructions, mini-coder still too easily treats mechanism-building or likely-equivalent output as completion when the task really wants the current environment or explicit deliverable to already be correct.
42-- why this should help outside Terminal-Bench:
43- - this is normal user behavior too: if they ask for `/app/out.txt`, a running service on a port, or an exact file/script shape, they want that state now, not just a way to get there later.
44-- hypothesis:
45- - a small prompt reinforcement around live end state + explicit deliverable re-checks will push one-shot runs toward direct completion without adding task-specific product logic.
46-- verification method:
47- - product checks: `bun test src/prompt.test.ts`, `bun run typecheck`, `bun test`, `bun run check`, `bun run format`
48- - benchmark check: one-attempt local fast suite in `terminal-bench/jobs/fast-end-state-check-2026-04-20__22-30-43`
49-- result summary:
50- - fast suite improved from `0/3` to `1/3`
51- - `large-scale-text-editing` passed; the agent now preserved the exact three-macro artifact contract instead of collapsing it to an equivalent shorter script.
52- - `gcode-to-text` improved materially; it wrote `/app/out.txt` and finished cleanly, but decoded one character wrong (`1` instead of `i`).
53- - `configure-git-webserver` improved partially; it left a live server instead of HTTP 000, but still finished with HTTP 404 because the deployed content was not actually present at verifier time.
54-- DECISION:
55- - KEEP — this looks like a real general behavior improvement, not benchmark-shaped overfitting, and it already fixed one fast-suite task outright while moving the other two closer to completion.
56-
57-## Completed step: contradiction / ambiguity prompt reinforcement
58-
59-- change made:
60- - tightened the core system prompt again so the agent explicitly trusts contradictory tool evidence, avoids guessing between plausible outputs/end states, and verifies again after any later state change.
61-- benchmark symptom:
62- - `configure-git-webserver` was close but still lost the verifier because the agent talked itself into a cleaner final state after successful local verification.
63- - `gcode-to-text` was down to one ambiguous character, so the remaining problem looked like unresolved uncertainty rather than missing effort.
64-- general behavior gap:
65- - mini-coder can still let speculative cleanup or a plausible guess override concrete evidence. When it has multiple believable interpretations, it does not always force a small discriminating check before finalizing.
66-- why this should help outside Terminal-Bench:
67- - this is a normal coding-agent failure mode too: users need the agent to trust the actual test output and settle ambiguous file contents, ports, or command results with one more check instead of narrating past the contradiction.
68-- hypothesis:
69- - a small prompt reinforcement around evidence-over-narrative and ambiguity resolution will make one-shot runs hold onto verified success states and avoid avoidable guesses.
70-- verification method:
71- - product checks: `bun test src/prompt.test.ts`, `bun run typecheck`, `bun test`, `bun run check`, `bun run format`
72- - benchmark check: one-attempt local fast suite in `terminal-bench/jobs/fast-ambiguity-check-2026-04-20__22-45-36`
73-- result summary:
74- - fast suite improved from `1/3` to `2/3`
75- - `configure-git-webserver` now passes end-to-end; the agent kept the verified deployed state instead of reverting to an empty-but-cleaner repo state.
76- - `large-scale-text-editing` stayed green.
77- - `gcode-to-text` did not produce a comparable behavioral result because the run hit a `NonZeroAgentExitCodeError` after several `readImage` calls on generated PNGs; the provider rejected one image as invalid before the agent could finish.
78-- DECISION:
79- - KEEP — this change appears to have produced another real behavior win, and the remaining fast-suite failure now points at a separate product bug in the headless/readImage path rather than a clear rejection of the hypothesis.
3+- Loop errored out and possibly crashed running benchmarks.
4+- Fixes were done to the code, `mc` is now updated to the latest dev.
5+- Benchmark loop re-started
TODO.md+2−10
@@ -7,21 +7,13 @@ The spec and code are the sources of truth; this file is just the verified backl
77
88 ## Open issues:
99
10-- [ ] We are changing the spec and adding auto compaction. When the context pressure gets to 90% compact the oldest ~40% of messages in context. The compaction process does an individual agent call to summarize the messages being compacted the compaction prompt should mention the user request, so compaction stays relevant, this summary replaces the messages removed. With the summary include a <system-message> block informing the agent that he can read these messages again if needed from the db a …
11-
10+- [ ] Add first-class delegation/subagent support instead of relying on shell-level `mc -p` delegation (`src/agent.ts`, `src/tools.ts`, `src/prompt.ts`, `src/headless.ts`)
1211 - [ ] Read tool output should include line numbers to help the llm find it's anchors.
1312 - [ ] If an agent uses readImage on a non-image file, mc crashes
1413 - [ ] If a conversation is truncated away from the db when the UI is still open somewhere, the app crashes.
14+- [ ] Prevent `/session` from switching sessions mid-run (`src/ui/commands.ts`, `src/ui/agent.ts`)
1515
1616 ## Theme config plan
1717
1818 - [ ] Extend the existing `settings.json` schema/loader with `theme` overrides and merge them into the active UI theme (`src/settings.ts`, `src/theme.ts`, `src/index.ts`, `spec.md`)
1919 - [ ] Update tests and user-facing docs/help for the `settings.json`-based theme setup (`src/index.test.ts`, `src/ui/help.ts`, `README.md`, `spec.md`)
20-
21-## Verified UX / behavior debt
22-
23-- [ ] Prevent `/session` from switching sessions mid-run (`src/ui/commands.ts`, `src/ui/agent.ts`)
24-
25-## Product backlog
26-
27-- [ ] Add first-class delegation/subagent support instead of relying on shell-level `mc -p` delegation (`src/agent.ts`, `src/tools.ts`, `src/prompt.ts`, `src/headless.ts`)
spec.md+26−12
@@ -208,7 +208,7 @@ The core runtime. Streaming is the default behavior throughout the turn: user-vi
208208
209209 1. **User submits a message** — the input text (plus any embedded images or skill bodies from `/skill:name`) becomes a pi-ai `UserMessage`. It is appended to the session's message history, rendered in the UI immediately, and persisted to the DB.
210210
211-2. **Build context** — construct a pi-ai `Context`: the system prompt (see [System prompt](#system-prompt)), the full message history, and the registered tool definitions. Before each request, mini-coder may apply a narrow image-compatibility pass to the model-visible history: if the current model does not support image input, or if the session has accumulated more than a small conservative image budget, image blocks are replaced with explicit text placeholders instead of being sent verbatim. …
211+2. **Build context** — construct a pi-ai `Context`: the system prompt (see [System prompt](#system-prompt)), the current model-visible history, and the registered tool definitions. Before each request, mini-coder first rebuilds that model-visible history from any persisted compaction summaries plus the remaining uncompacted raw messages. It estimates context pressure from the same model-bound view of history that will actually be sent, including any image-compatibility replacements for unsuppo …
212212
213213 3. **Stream to LLM** — call `streamSimple(model, context, options)` from pi-ai. Iterate over the event stream:
214214 - `text_delta` / `thinking_delta` / `toolcall_delta` → update the in-progress assistant message and the UI incrementally (stream raw markdown text, show thinking if enabled, accumulate tool call arguments as they arrive).
@@ -241,7 +241,7 @@ The core runtime. Streaming is the default behavior throughout the turn: user-vi
241241
242242 **No step limit** — the loop runs until the model stops (`stopReason: "stop"`) or the user interrupts. There is no maximum number of tool calls per turn.
243243
244-**Context limit** — mini-coder does not implement built-in context compaction. The status bar still estimates current context usage for the next request, but automatic history summarization is not a core feature. If compaction or summarization is added, it should stay optional rather than hardwired into the core, since users disagree about whether automatic compaction is desirable.
244+**Context limit** — mini-coder automatically compacts conversation history when the estimated next-request context usage for the actual model-bound request reaches 90% of the active model window. That estimate reflects any image placeholders the request will send, so unsupported or over-budget images do not cause avoidable lossy compaction. Compaction summarizes the oldest ~40% of still-uncompacted model-visible messages with a separate model call that is anchored to the current user request, …
245245
246246 ## Architecture
247247
@@ -971,6 +971,17 @@ CREATE TABLE messages (
971971
972972 CREATE INDEX idx_messages_session ON messages(session_id, turn);
973973
974+CREATE TABLE session_compactions (
975+ id INTEGER PRIMARY KEY AUTOINCREMENT,
976+ session_id TEXT NOT NULL REFERENCES sessions(id) ON DELETE CASCADE,
977+ message_end_id INTEGER NOT NULL, -- last raw message-row id covered by this summary
978+ summary_data TEXT NOT NULL, -- JSON-serialized synthetic user summary message
979+ usage_data TEXT, -- JSON-serialized assistant usage for the hidden summary call
980+ created_at INTEGER NOT NULL -- unix ms
981+);
982+
983+CREATE INDEX idx_session_compactions_session ON session_compactions(session_id, message_end_id);
984+
974985 CREATE TABLE prompt_history (
975986 id INTEGER PRIMARY KEY AUTOINCREMENT,
976987 text TEXT NOT NULL, -- exact raw submitted prompt text
@@ -986,11 +997,13 @@ CREATE INDEX idx_prompt_history_created_at ON prompt_history(created_at, id);
986997
987998 **Messages as JSON blobs**: pi-ai's `Message` type (UserMessage, AssistantMessage, ToolResultMessage) is already serializable, and we also persist internal UI-only log entries in the same table. We store the full object as JSON in `data` rather than normalizing into columns. Session load returns the complete chronological log; model context construction filters out UI-only messages before replaying the remaining pi-ai messages into a pi-ai `Context`.
988999
1000+**Compaction summaries are stored separately from the raw transcript**: automatic context compaction never rewrites or deletes old conversational rows just to save context. Instead, each compaction stores one synthetic user-summary message in `session_compactions`, the raw message-row boundary it replaces, and the hidden summarization-call usage for stats/accounting. Future model context is rebuilt from those stored summaries followed by the remaining uncompacted raw message tail, while the full …
1001+
9891002 **Todo state lives in message history**: there is no separate todo table. The current todo list is reconstructed from the latest successful persisted todo snapshot found in session messages. Because that snapshot rides along with normal message history, `/undo`, `/fork`, session reload, and session switching all restore the expected todo state automatically.
9901003
9911004 **Turn grouping**: the `turn` column groups only conversational messages. A turn is: one user message + the assistant messages and tool result messages produced after that user message until the next user message is consumed. Persisted UI messages are stored alongside them but have `turn = NULL`, so they remain visible in session history without becoming part of `/undo`. Queued steering messages submitted mid-run still start their own turn when they are consumed; tool results already requested b …
9921005
993-**Cumulative stats are computed, not stored**: the status bar's cumulative `in`, `out`, and `$cost` values are computed by summing `usage` from assistant messages on session load (deserialize all messages, filter for `role: "assistant"`, sum their `usage` fields). The message count per session is small (hundreds), so this is fast. No separate counters to keep in sync. During an active session, a running in-memory accumulator is updated after each assistant message to avoid re-scanning. `context% …
1006+**Cumulative stats are computed, not stored**: the status bar's cumulative `in`, `out`, and `$cost` values are computed on session load by summing `usage` from assistant messages in `messages` plus any persisted hidden-summary usage in `session_compactions`. The message/compaction count per session is small (hundreds), so this is fast. No separate counters to keep in sync. During an active session, a running in-memory accumulator is updated after each assistant message or hidden compaction-summa …
9941007
9951008 **Turn number assignment**: when a user message is appended, whether from an idle submission or a queued steering message that is being consumed, its turn number is `MAX(turn) + 1` for the session (or 1 for the first conversational message). Assistant responses and tool results keep the current active turn number until another queued user message is consumed and becomes the next model input. UI-only messages do not receive a turn number. This is what makes `/undo` atomic for conversation history …
9961009
@@ -1002,12 +1015,13 @@ CREATE INDEX idx_prompt_history_created_at ON prompt_history(created_at, id);
10021015
10031016 ### Operations
10041017
1005-| Operation | SQL |
1006-| -------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
1007-| New session | `INSERT INTO sessions` |
1008-| Append message | `INSERT INTO messages` with the current conversational turn number (or `NULL` for UI-only messages), `UPDATE sessions SET updated_at` |
1009-| Undo | `DELETE FROM messages WHERE session_id = ? AND turn = (SELECT MAX(turn) FROM messages WHERE session_id = ?)` |
1010-| Fork | `INSERT INTO sessions` (new id, `forked_from` set), then `INSERT INTO messages SELECT ... FROM messages WHERE session_id = ?` (copy all, new session_id), then append a UI-only fork notice with `turn = NULL` |
1011-| List sessions | `SELECT * FROM sessions WHERE cwd = ? ORDER BY updated_at DESC` |
1012-| Load session | `SELECT data FROM messages WHERE session_id = ? ORDER BY id` → parse JSON → persisted message log (`pi-ai Message[]` + internal UI messages) |
1013-| Delete session | `DELETE FROM sessions WHERE id = ?` (cascade deletes messages) |
1018+| Operation | SQL |
1019+| --------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
1020+| New session | `INSERT INTO sessions` |
1021+| Append message | `INSERT INTO messages` with the current conversational turn number (or `NULL` for UI-only messages), `UPDATE sessions SET updated_at` |
1022+| Compact context | `INSERT INTO session_compactions` with the synthetic summary message, hidden summary-call usage, and the last raw message-row id it replaces |
1023+| Undo | `DELETE FROM messages WHERE session_id = ? AND turn = (SELECT MAX(turn) FROM messages WHERE session_id = ?)` plus `DELETE FROM session_compactions WHERE session_id = ? AND message_end_id >= <first row id in the deleted turn>` |
1024+| Fork | `INSERT INTO sessions` (new id, `forked_from` set), then `INSERT INTO messages SELECT ... FROM messages WHERE session_id = ?` (copy all, new session_id), then copy `session_compactions` with each `message_end_id` remapped onto the forked message rows, then append a UI-only fork notice with `turn = NULL` |
1025+| List sessions | `SELECT * FROM sessions WHERE cwd = ? ORDER BY updated_at DESC` |
1026+| Load session | `SELECT data FROM messages WHERE session_id = ? ORDER BY id` → parse JSON → persisted message log (`pi-ai Message[]` + internal UI messages); separately load `session_compactions` to rebuild the compacted model-visible history and add hidden summary-call usage to cumulative stats |
1027+| Delete session | `DELETE FROM sessions WHERE id = ?` (cascade deletes messages and compaction summaries) |
src/agent.test.ts+493−0
@@ -26,7 +26,9 @@ import {
2626 import { type AgentEvent, runAgentLoop, type ToolHandler } from "./agent.ts";
2727 import {
2828 appendMessage,
29+ computeSessionStats,
2930 createSession,
31+ loadCompactedModelMessages,
3032 loadMessages,
3133 openDatabase,
3234 } from "./session.ts";
@@ -2078,6 +2080,497 @@ describe("agent loop", () => {
20782080 }
20792081 });
20802082
2083+ test("text-only image sanitization does not trigger unnecessary compaction from stale assistant usage", async () => {
2084+ const api = "test/text-only-image-compaction";
2085+ const sourceId = "test-text-only-image-compaction";
2086+ const model: Model<string> = {
2087+ id: api,
2088+ name: "Text Only Image Compaction Model",
2089+ api,
2090+ provider: "test",
2091+ baseUrl: "http://localhost:0",
2092+ reasoning: false,
2093+ input: ["text"],
2094+ cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
2095+ contextWindow: 1_500,
2096+ maxTokens: 4_096,
2097+ };
2098+ const compactionRequests: Message[][] = [];
2099+ const mainRequests: Message[][] = [];
2100+
2101+ const buildProviderStream = (context: Context) => {
2102+ const isCompaction = (context.systemPrompt ?? "").includes(
2103+ "You summarize earlier conversation context for a coding agent.",
2104+ );
2105+ if (isCompaction) {
2106+ compactionRequests.push(structuredClone(context.messages));
2107+ } else {
2108+ mainRequests.push(structuredClone(context.messages));
2109+ }
2110+
2111+ const stream = createAssistantMessageEventStream();
2112+ const finalMessage = fauxAssistantMessage("done");
2113+ queueMicrotask(() => {
2114+ stream.push({ type: "done", reason: "stop", message: finalMessage });
2115+ stream.end(finalMessage);
2116+ });
2117+ return stream;
2118+ };
2119+
2120+ registerApiProvider(
2121+ {
2122+ api,
2123+ stream: (_model, context) => buildProviderStream(context),
2124+ streamSimple: (_model, context) => buildProviderStream(context),
2125+ },
2126+ sourceId,
2127+ );
2128+
2129+ try {
2130+ const session = createSession(db, { cwd: tmp });
2131+ const firstUser = makeUser("inspect the screenshot");
2132+ const firstTurn = appendMessage(db, session.id, firstUser);
2133+ const imageResult = makeImageToolResultMessage(1);
2134+ appendMessage(db, session.id, imageResult, firstTurn);
2135+ const historicalAssistant: AssistantMessage = {
2136+ ...fauxAssistantMessage("historical reply"),
2137+ api,
2138+ provider: "test",
2139+ model: api,
2140+ usage: {
2141+ ...ZERO_USAGE,
2142+ input: 2_400,
2143+ totalTokens: 2_400,
2144+ cost: { ...ZERO_USAGE.cost },
2145+ },
2146+ };
2147+ appendMessage(db, session.id, historicalAssistant, firstTurn);
2148+ const currentUser = makeUser("continue without using vision");
2149+ const turn = appendMessage(db, session.id, currentUser);
2150+
2151+ const result = await runAgentLoop({
2152+ db,
2153+ sessionId: session.id,
2154+ turn,
2155+ model,
2156+ systemPrompt: "Test",
2157+ tools: [],
2158+ toolHandlers: new Map(),
2159+ messages: [firstUser, imageResult, historicalAssistant, currentUser],
2160+ cwd: tmp,
2161+ });
2162+
2163+ expect(result.stopReason).toBe("stop");
2164+ expect(compactionRequests).toEqual([]);
2165+ expect(mainRequests).toHaveLength(1);
2166+ const sanitizedImageResult = mainRequests[0]?.[1];
2167+ if (
2168+ !sanitizedImageResult ||
2169+ sanitizedImageResult.role !== "toolResult" ||
2170+ !Array.isArray(sanitizedImageResult.content)
2171+ ) {
2172+ throw new Error("Expected the sanitized image tool result in context");
2173+ }
2174+ expect(sanitizedImageResult.content).toEqual([
2175+ {
2176+ type: "text",
2177+ text: "[Image omitted from model context because the active model does not support image input.]",
2178+ },
2179+ ]);
2180+ } finally {
2181+ unregisterApiProviders(sourceId);
2182+ }
2183+ });
2184+
2185+ test("context_compacted events and persisted session stats include hidden summary usage", async () => {
2186+ const api = "test/context-compaction-usage";
2187+ const sourceId = "test-context-compaction-usage";
2188+ const model: Model<string> = {
2189+ id: api,
2190+ name: "Context Compaction Usage Model",
2191+ api,
2192+ provider: "test",
2193+ baseUrl: "http://localhost:0",
2194+ reasoning: false,
2195+ input: ["text"],
2196+ cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
2197+ contextWindow: 250,
2198+ maxTokens: 4096,
2199+ };
2200+ const events: AgentEvent[] = [];
2201+ let compactionCallCount = 0;
2202+ let compactionInput = 0;
2203+ let compactionOutput = 0;
2204+ let compactionCost = 0;
2205+
2206+ const buildProviderStream = (context: Context) => {
2207+ const isCompaction = (context.systemPrompt ?? "").includes(
2208+ "You summarize earlier conversation context for a coding agent.",
2209+ );
2210+ const stream = createAssistantMessageEventStream();
2211+ const finalMessage = isCompaction
2212+ ? {
2213+ ...fauxAssistantMessage(`summary ${++compactionCallCount}`),
2214+ usage: {
2215+ input: 123,
2216+ output: 45,
2217+ cacheRead: 0,
2218+ cacheWrite: 0,
2219+ totalTokens: 168,
2220+ cost: {
2221+ input: 1,
2222+ output: 2,
2223+ cacheRead: 0,
2224+ cacheWrite: 0,
2225+ total: 3,
2226+ },
2227+ },
2228+ }
2229+ : {
2230+ ...fauxAssistantMessage("done"),
2231+ usage: {
2232+ input: 10,
2233+ output: 2,
2234+ cacheRead: 0,
2235+ cacheWrite: 0,
2236+ totalTokens: 12,
2237+ cost: {
2238+ input: 0.1,
2239+ output: 0.2,
2240+ cacheRead: 0,
2241+ cacheWrite: 0,
2242+ total: 0.3,
2243+ },
2244+ },
2245+ };
2246+
2247+ if (isCompaction) {
2248+ compactionInput += finalMessage.usage.input;
2249+ compactionOutput += finalMessage.usage.output;
2250+ compactionCost += finalMessage.usage.cost.total;
2251+ }
2252+
2253+ queueMicrotask(() => {
2254+ stream.push({ type: "done", reason: "stop", message: finalMessage });
2255+ stream.end(finalMessage);
2256+ });
2257+
2258+ return stream;
2259+ };
2260+
2261+ registerApiProvider(
2262+ {
2263+ api,
2264+ stream: (_model, context) => buildProviderStream(context),
2265+ streamSimple: (_model, context) => buildProviderStream(context),
2266+ },
2267+ sourceId,
2268+ );
2269+
2270+ try {
2271+ const session = createSession(db, { cwd: tmp });
2272+ const messages: Message[] = [];
2273+ let turn = 0;
2274+
2275+ for (let index = 0; index < 7; index += 1) {
2276+ const userMsg = makeUser(`request ${index} ${"x".repeat(380)}`);
2277+ turn = appendMessage(db, session.id, userMsg);
2278+ messages.push(userMsg);
2279+ }
2280+
2281+ const result = await runAgentLoop({
2282+ db,
2283+ sessionId: session.id,
2284+ turn,
2285+ model,
2286+ systemPrompt: "Test",
2287+ tools: [],
2288+ toolHandlers: new Map(),
2289+ messages,
2290+ cwd: tmp,
2291+ onEvent: (event) => events.push(event),
2292+ });
2293+
2294+ expect(result.stopReason).toBe("stop");
2295+ const compactionEvents = events.filter(
2296+ (
2297+ event,
2298+ ): event is Extract<AgentEvent, { type: "context_compacted" }> => {
2299+ return event.type === "context_compacted";
2300+ },
2301+ );
2302+ expect(compactionEvents.length).toBe(compactionCallCount);
2303+ expect(compactionEvents.length).toBeGreaterThan(0);
2304+ expect(compactionEvents.at(-1)?.stats).toEqual({
2305+ totalInput: compactionInput,
2306+ totalOutput: compactionOutput,
2307+ totalCost: compactionCost,
2308+ });
2309+ expect(computeSessionStats(db, session.id)).toEqual({
2310+ totalInput: compactionInput + 10,
2311+ totalOutput: compactionOutput + 2,
2312+ totalCost: compactionCost + 0.3,
2313+ });
2314+ } finally {
2315+ unregisterApiProviders(sourceId);
2316+ }
2317+ });
2318+
2319+ test("context compaction summarizes older messages without re-compacting earlier summaries", async () => {
2320+ const api = "test/context-compaction";
2321+ const sourceId = "test-context-compaction";
2322+ const model: Model<string> = {
2323+ id: api,
2324+ name: "Context Compaction Model",
2325+ api,
2326+ provider: "test",
2327+ baseUrl: "http://localhost:0",
2328+ reasoning: false,
2329+ input: ["text"],
2330+ cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
2331+ contextWindow: 250,
2332+ maxTokens: 4096,
2333+ };
2334+ const compactionRequests: Message[][] = [];
2335+ const mainRequests: Message[][] = [];
2336+ let compactionCallCount = 0;
2337+
2338+ const buildProviderStream = (context: Context) => {
2339+ const isCompaction = (context.systemPrompt ?? "").includes(
2340+ "You summarize earlier conversation context for a coding agent.",
2341+ );
2342+ if (isCompaction) {
2343+ compactionRequests.push(structuredClone(context.messages));
2344+ compactionCallCount += 1;
2345+ } else {
2346+ mainRequests.push(structuredClone(context.messages));
2347+ }
2348+
2349+ const stream = createAssistantMessageEventStream();
2350+ const finalMessage = isCompaction
2351+ ? fauxAssistantMessage(`summary ${compactionCallCount}`)
2352+ : fauxAssistantMessage("done");
2353+
2354+ queueMicrotask(() => {
2355+ stream.push({ type: "done", reason: "stop", message: finalMessage });
2356+ stream.end(finalMessage);
2357+ });
2358+
2359+ return stream;
2360+ };
2361+
2362+ registerApiProvider(
2363+ {
2364+ api,
2365+ stream: (_model, context) => buildProviderStream(context),
2366+ streamSimple: (_model, context) => buildProviderStream(context),
2367+ },
2368+ sourceId,
2369+ );
2370+
2371+ try {
2372+ const session = createSession(db, { cwd: tmp });
2373+ const messages: Message[] = [];
2374+ let turn = 0;
2375+
2376+ for (let index = 0; index < 7; index += 1) {
2377+ const label =
2378+ index === 6 ? "current request" : `earlier request ${index}`;
2379+ const userMsg = makeUser(`${label} ${"x".repeat(380)}`);
2380+ turn = appendMessage(db, session.id, userMsg);
2381+ messages.push(userMsg);
2382+ }
2383+
2384+ const result = await runAgentLoop({
2385+ db,
2386+ sessionId: session.id,
2387+ turn,
2388+ model,
2389+ systemPrompt: "Test",
2390+ tools: [],
2391+ toolHandlers: new Map(),
2392+ messages,
2393+ cwd: tmp,
2394+ });
2395+
2396+ expect(result.stopReason).toBe("stop");
2397+ expect(compactionRequests.length).toBeGreaterThanOrEqual(2);
2398+ expect(mainRequests).toHaveLength(1);
2399+
2400+ const firstInstruction = compactionRequests[0]?.at(-1);
2401+ if (!firstInstruction || firstInstruction.role !== "user") {
2402+ throw new Error("Expected the first compaction instruction message");
2403+ }
2404+ if (typeof firstInstruction.content !== "string") {
2405+ throw new Error("Expected the first compaction instruction to be text");
2406+ }
2407+ expect(firstInstruction.content).toContain("Current user request:");
2408+ expect(firstInstruction.content).toContain("current request");
2409+
2410+ const secondCompaction = compactionRequests[1]!;
2411+ expect(
2412+ secondCompaction.some(
2413+ (message) =>
2414+ message.role === "user" &&
2415+ typeof message.content === "string" &&
2416+ message.content.includes("summary 1"),
2417+ ),
2418+ ).toBe(false);
2419+
2420+ const compactedContext = mainRequests[0]!;
2421+ const summaryMessages = compactedContext.filter(
2422+ (message): message is UserMessage =>
2423+ message.role === "user" &&
2424+ typeof message.content === "string" &&
2425+ message.content.includes("<system-message>"),
2426+ );
2427+ expect(summaryMessages.length).toBeGreaterThan(0);
2428+ expect(summaryMessages[0]!.content).toContain("summary 1");
2429+ expect(summaryMessages[0]!.content).toContain(session.id);
2430+ expect(summaryMessages[0]!.content).toContain(":memory:");
2431+ expect(
2432+ compactedContext.some(
2433+ (message) =>
2434+ message.role === "user" &&
2435+ typeof message.content === "string" &&
2436+ message.content.includes("earlier request 0"),
2437+ ),
2438+ ).toBe(false);
2439+ expect(
2440+ loadMessages(db, session.id).some(
2441+ (message) =>
2442+ message.role === "user" &&
2443+ typeof message.content === "string" &&
2444+ message.content.includes("earlier request 0"),
2445+ ),
2446+ ).toBe(true);
2447+ expect(
2448+ loadCompactedModelMessages(db, session.id).some(
2449+ (message) =>
2450+ message.role === "user" &&
2451+ typeof message.content === "string" &&
2452+ message.content.includes("<system-message>"),
2453+ ),
2454+ ).toBe(true);
2455+ } finally {
2456+ unregisterApiProviders(sourceId);
2457+ }
2458+ });
2459+
2460+ test("repeated compactions stay anchored to the active user request after that request is compacted", async () => {
2461+ const api = "test/context-compaction-anchor";
2462+ const sourceId = "test-context-compaction-anchor";
2463+ const model: Model<string> = {
2464+ id: api,
2465+ name: "Context Compaction Anchor Model",
2466+ api,
2467+ provider: "test",
2468+ baseUrl: "http://localhost:0",
2469+ reasoning: false,
2470+ input: ["text"],
2471+ cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
2472+ contextWindow: 250,
2473+ maxTokens: 4096,
2474+ };
2475+ const currentRequestAnchor = "current request anchor";
2476+ const compactionInstructions: string[] = [];
2477+ let compactionCallCount = 0;
2478+
2479+ const buildProviderStream = (context: Context) => {
2480+ const isCompaction = (context.systemPrompt ?? "").includes(
2481+ "You summarize earlier conversation context for a coding agent.",
2482+ );
2483+ if (isCompaction) {
2484+ const instruction = context.messages.at(-1);
2485+ if (
2486+ instruction?.role === "user" &&
2487+ typeof instruction.content === "string"
2488+ ) {
2489+ compactionInstructions.push(instruction.content);
2490+ }
2491+ compactionCallCount += 1;
2492+ }
2493+
2494+ const stream = createAssistantMessageEventStream();
2495+ const finalMessage = isCompaction
2496+ ? fauxAssistantMessage(`summary ${compactionCallCount}`)
2497+ : fauxAssistantMessage("done");
2498+
2499+ queueMicrotask(() => {
2500+ stream.push({ type: "done", reason: "stop", message: finalMessage });
2501+ stream.end(finalMessage);
2502+ });
2503+
2504+ return stream;
2505+ };
2506+
2507+ registerApiProvider(
2508+ {
2509+ api,
2510+ stream: (_model, context) => buildProviderStream(context),
2511+ streamSimple: (_model, context) => buildProviderStream(context),
2512+ },
2513+ sourceId,
2514+ );
2515+
2516+ try {
2517+ const session = createSession(db, { cwd: tmp });
2518+ const currentRequest = `${currentRequestAnchor} ${"x".repeat(380)}`;
2519+ const userMsg = makeUser(currentRequest);
2520+ const turn = appendMessage(db, session.id, userMsg);
2521+ const messages: Message[] = [userMsg];
2522+
2523+ for (let index = 0; index < 6; index += 1) {
2524+ const assistantMessage: AssistantMessage = {
2525+ ...fauxAssistantMessage(`assistant ${index} ${"a".repeat(380)}`),
2526+ api,
2527+ provider: "test",
2528+ model: api,
2529+ usage: {
2530+ ...ZERO_USAGE,
2531+ input: 250,
2532+ totalTokens: 250,
2533+ cost: { ...ZERO_USAGE.cost },
2534+ },
2535+ };
2536+ const toolResultMessage: Message = {
2537+ role: "toolResult",
2538+ toolCallId: `call-${index}`,
2539+ toolName: "shell",
2540+ content: [{ type: "text", text: `tool ${index} ${"t".repeat(380)}` }],
2541+ isError: false,
2542+ timestamp: Date.now() + 100 + index,
2543+ };
2544+
2545+ appendMessage(db, session.id, assistantMessage, turn);
2546+ appendMessage(db, session.id, toolResultMessage, turn);
2547+ messages.push(assistantMessage, toolResultMessage);
2548+ }
2549+
2550+ const result = await runAgentLoop({
2551+ db,
2552+ sessionId: session.id,
2553+ turn,
2554+ model,
2555+ systemPrompt: "Test",
2556+ tools: [],
2557+ toolHandlers: new Map(),
2558+ messages,
2559+ cwd: tmp,
2560+ });
2561+
2562+ expect(result.stopReason).toBe("stop");
2563+ expect(compactionCallCount).toBeGreaterThanOrEqual(2);
2564+ expect(compactionInstructions).toHaveLength(compactionCallCount);
2565+ for (const instruction of compactionInstructions) {
2566+ expect(instruction).toContain("Current user request:");
2567+ expect(instruction).toContain(currentRequestAnchor);
2568+ }
2569+ } finally {
2570+ unregisterApiProviders(sourceId);
2571+ }
2572+ });
2573+
20812574 test("stopReason 'length' returns without looping", async () => {
20822575 faux.setResponses([
20832576 fauxAssistantMessage("Partial response that hit the token limit", {
src/agent.ts+342−14
@@ -21,9 +21,33 @@ import type {
2121 UserMessage,
2222 } from "@mariozechner/pi-ai";
2323 import { streamSimple } from "@mariozechner/pi-ai";
24-import { appendMessage } from "./session.ts";
24+import {
25+ appendMessage,
26+ appendSessionCompaction,
27+ computeContextTokens,
28+ computeSessionStats,
29+ loadCompactedModelMessages,
30+ loadMessages,
31+ loadUncompactedModelMessages,
32+} from "./session.ts";
33+import { collapseWhitespaceToNull, joinTextBlocks } from "./text.ts";
2534 import { getTodoItems, type ToolExecResult } from "./tools.ts";
2635
36+const CONTEXT_COMPACTION_THRESHOLD = 0.9;
37+const CONTEXT_COMPACTION_FRACTION = 0.4;
38+const DEFAULT_CURRENT_USER_REQUEST =
39+ "Continue the current task using the earlier conversation context.";
40+const COMPACTION_RECOVERY_PREFIX =
41+ "These earlier messages were compacted out of the active context.";
42+
43+const COMPACTION_SUMMARY_SYSTEM_PROMPT = [
44+ "You summarize earlier conversation context for a coding agent.",
45+ "Write a concise replacement summary for the provided earlier messages so the agent can continue the current request after those messages are removed from active context.",
46+ "Keep only concrete facts that may still matter: user goals, decisions, constraints, files or paths, commands run, tool results, errors, and unfinished work.",
47+ "Prefer short bullet points.",
48+ "Do not invent details or mention information that is not supported by the provided messages.",
49+].join("\n");
50+
2751 // ---------------------------------------------------------------------------
2852 // Types
2953 // ---------------------------------------------------------------------------
@@ -101,6 +125,11 @@ export type AgentEvent =
101125 result: ToolExecResult;
102126 }
103127 | { type: "tool_result"; message: ToolResultMessage }
128+ | {
129+ type: "context_compacted";
130+ contextTokens: number;
131+ stats: ReturnType<typeof computeSessionStats>;
132+ }
104133 | { type: "done"; message: AssistantMessage }
105134 | { type: "error"; message: AssistantMessage }
106135 | { type: "aborted"; message: AssistantMessage };
@@ -312,21 +341,36 @@ function sanitizeMessageContentForContext(
312341 function sanitizeContextMessages(
313342 messages: Message[],
314343 modelSupportsImages: boolean,
315-): Message[] {
344+): { messages: Message[]; changed: boolean } {
316345 const imageBudget = {
317346 remaining: modelSupportsImages ? MAX_CONTEXT_IMAGE_BLOCKS : 0,
318347 supportsImages: modelSupportsImages,
319348 };
320349 const sanitized = new Array<Message>(messages.length);
350+ let changed = false;
321351
322352 for (let index = messages.length - 1; index >= 0; index -= 1) {
323- sanitized[index] = sanitizeMessageContentForContext(
353+ const message = sanitizeMessageContentForContext(
324354 messages[index]!,
325355 imageBudget,
326356 );
357+ if (message !== messages[index]) {
358+ changed = true;
359+ }
360+ sanitized[index] = message;
327361 }
328362
329- return sanitized;
363+ return { messages: sanitized, changed };
364+}
365+
366+function computeRequestContextTokens(
367+ messages: Message[],
368+ modelSupportsImages: boolean,
369+): number {
370+ const sanitized = sanitizeContextMessages(messages, modelSupportsImages);
371+ return computeContextTokens(sanitized.messages, {
372+ ignoreAssistantUsage: sanitized.changed,
373+ });
330374 }
331375
332376 function buildAgentContext(
@@ -338,7 +382,7 @@ function buildAgentContext(
338382 const contextMessages = sanitizeContextMessages(
339383 messages,
340384 modelSupportsImages,
341- );
385+ ).messages;
342386 return tools.length > 0
343387 ? { systemPrompt, messages: contextMessages, tools }
344388 : { systemPrompt, messages: contextMessages };
@@ -356,6 +400,237 @@ function buildStreamOptions(
356400 };
357401 }
358402
403+function stripSyntheticSystemMessageBlock(text: string): string {
404+ return text.replace(/\n?<system-message>[\s\S]*?<\/system-message>/g, "");
405+}
406+
407+function describeUserContent(content: UserMessage["content"]): string | null {
408+ if (typeof content === "string") {
409+ return collapseWhitespaceToNull(stripSyntheticSystemMessageBlock(content));
410+ }
411+
412+ const text = collapseWhitespaceToNull(joinTextBlocks(content));
413+ const imageCount = content.filter((block) => block.type === "image").length;
414+ const parts = [
415+ text,
416+ imageCount > 0
417+ ? `[${imageCount} image${imageCount === 1 ? "" : "s"} attached]`
418+ : null,
419+ ].filter(
420+ (part): part is string => typeof part === "string" && part.length > 0,
421+ );
422+
423+ return parts.length > 0 ? parts.join("\n") : null;
424+}
425+
426+function resolveCurrentUserRequest(userMessage: UserMessage): string {
427+ return (
428+ describeUserContent(userMessage.content) ?? DEFAULT_CURRENT_USER_REQUEST
429+ );
430+}
431+
432+function isSystemReminderMessage(message: Message): boolean {
433+ return (
434+ message.role === "user" &&
435+ typeof message.content === "string" &&
436+ message.content.startsWith("<system_reminder>")
437+ );
438+}
439+
440+function isCompactionSummaryMessage(message: Message): boolean {
441+ return (
442+ message.role === "user" &&
443+ typeof message.content === "string" &&
444+ message.content.includes("<system-message>") &&
445+ message.content.includes(COMPACTION_RECOVERY_PREFIX)
446+ );
447+}
448+
449+function extractCurrentUserRequest(messages: readonly Message[]): string {
450+ for (let index = messages.length - 1; index >= 0; index -= 1) {
451+ const message = messages[index];
452+ if (
453+ !message ||
454+ message.role !== "user" ||
455+ isSystemReminderMessage(message) ||
456+ isCompactionSummaryMessage(message)
457+ ) {
458+ continue;
459+ }
460+
461+ const content = describeUserContent(message.content);
462+ if (content) {
463+ return content;
464+ }
465+ }
466+
467+ return DEFAULT_CURRENT_USER_REQUEST;
468+}
469+
470+function createCompactionInstructionMessage(
471+ currentUserRequest: string,
472+): UserMessage {
473+ return {
474+ role: "user",
475+ content: [
476+ "Current user request:",
477+ currentUserRequest,
478+ "",
479+ "Summarize the earlier conversation messages above so the coding agent can continue the current request after those messages are removed from active context.",
480+ ].join("\n"),
481+ timestamp: Date.now(),
482+ };
483+}
484+
485+function extractAssistantTextContent(message: AssistantMessage): string | null {
486+ const text = message.content
487+ .flatMap((block) => {
488+ return block.type === "text" ? [block.text] : [];
489+ })
490+ .join("\n")
491+ .trim();
492+ return text.length > 0 ? text : null;
493+}
494+
495+function buildCompactionSummaryMessage(
496+ summaryText: string,
497+ dbPath: string,
498+ sessionId: string,
499+): UserMessage {
500+ return {
501+ role: "user",
502+ content: [
503+ summaryText,
504+ "",
505+ "<system-message>",
506+ `${COMPACTION_RECOVERY_PREFIX} If you need the exact originals again, read them from the SQLite database at ${dbPath} for session ${sessionId}.`,
507+ "</system-message>",
508+ ].join("\n"),
509+ timestamp: Date.now(),
510+ };
511+}
512+
513+function computeCompactionMessageCount(messageCount: number): number {
514+ if (messageCount <= 1) {
515+ return 0;
516+ }
517+
518+ return Math.max(
519+ 1,
520+ Math.min(
521+ messageCount - 1,
522+ Math.floor(messageCount * CONTEXT_COMPACTION_FRACTION),
523+ ),
524+ );
525+}
526+
527+interface CompactionSummaryResult {
528+ /** Replacement summary text for the compacted context prefix. */
529+ text: string;
530+ /** Usage consumed by the hidden summarization request. */
531+ usage: AssistantMessage["usage"];
532+}
533+
534+async function summarizeCompactionChunk(
535+ messages: Message[],
536+ currentUserRequest: string,
537+ opts: Pick<RunAgentOpts, "model" | "apiKey" | "effort" | "signal">,
538+): Promise<CompactionSummaryResult | null> {
539+ const summaryMessage = await streamAssistantMessage({
540+ model: opts.model,
541+ systemPrompt: COMPACTION_SUMMARY_SYSTEM_PROMPT,
542+ tools: [],
543+ messages: [
544+ ...messages,
545+ createCompactionInstructionMessage(currentUserRequest),
546+ ],
547+ apiKey: opts.apiKey,
548+ effort: opts.effort,
549+ signal: opts.signal,
550+ });
551+
552+ if (
553+ summaryMessage.stopReason === "error" ||
554+ summaryMessage.stopReason === "aborted"
555+ ) {
556+ return null;
557+ }
558+
559+ const text = extractAssistantTextContent(summaryMessage);
560+ if (!text) {
561+ return null;
562+ }
563+
564+ return {
565+ text,
566+ usage: summaryMessage.usage,
567+ };
568+}
569+
570+async function maybeCompactContext(
571+ messages: Message[],
572+ pendingContextMessages: readonly Message[],
573+ currentUserRequest: string,
574+ opts: Pick<
575+ RunAgentOpts,
576+ "db" | "sessionId" | "model" | "apiKey" | "effort" | "signal" | "onEvent"
577+ >,
578+): Promise<Message[]> {
579+ let compactedMessages = messages;
580+
581+ while (!opts.signal?.aborted) {
582+ const contextTokens = computeRequestContextTokens(
583+ [...compactedMessages, ...pendingContextMessages],
584+ opts.model.input.includes("image"),
585+ );
586+ if (
587+ contextTokens / opts.model.contextWindow <
588+ CONTEXT_COMPACTION_THRESHOLD
589+ ) {
590+ return compactedMessages;
591+ }
592+
593+ const candidateRows = loadUncompactedModelMessages(opts.db, opts.sessionId);
594+ const compactCount = computeCompactionMessageCount(candidateRows.length);
595+ if (compactCount === 0) {
596+ return compactedMessages;
597+ }
598+
599+ const summaryResult = await summarizeCompactionChunk(
600+ candidateRows.slice(0, compactCount).map((row) => row.message),
601+ currentUserRequest,
602+ opts,
603+ );
604+ if (!summaryResult) {
605+ return compactedMessages;
606+ }
607+
608+ appendSessionCompaction(
609+ opts.db,
610+ opts.sessionId,
611+ candidateRows[compactCount - 1]!.id,
612+ buildCompactionSummaryMessage(
613+ summaryResult.text,
Cut short: this commit's diff is over 1000 lines.
src/session-message.ts+57−65
Cut short: this commit's diff is over 1000 lines.
src/session.test.ts+229−0
Cut short: this commit's diff is over 1000 lines.
src/session.ts+389−42
Cut short: this commit's diff is over 1000 lines.
src/submit.ts+6−2
Cut short: this commit's diff is over 1000 lines.
src/ui.test.ts+130−1
Cut short: this commit's diff is over 1000 lines.
src/ui/agent.ts+3−0
Cut short: this commit's diff is over 1000 lines.
src/ui/commands.test.ts+47−6
Cut short: this commit's diff is over 1000 lines.
src/ui/commands.ts+14−0
Cut short: this commit's diff is over 1000 lines.