@@ -0,0 +1,327 @@
1+# LLM Processing Loop
2+
3+Single-turn conversation processing with multi-round tool call support.
4+
5+## Overview
6+
7+The LLM loop handles one conversation turn, which may involve multiple rounds of LLM calls and tool executions. A **turn** begins when the user sends a message and ends when the LLM provides a final text response without tool calls.
8+
9+**Key principle:** The loop accumulates history during the turn so the LLM sees its own tool calls and results across rounds.
10+
11+## Architecture
12+
13+```
14+User Input → ProcessTurn → [Round 0, Round 1, ..., Round N] → Final Response
15+ ↓
16+ Compress History → LLM Call → Tool Execution
17+ ↑_______________|
18+```
19+
20+## Flow Diagram
21+
22+```
23+processTurn (cli.go:266-282)
24+ │
25+ ├─ Snapshot app.history → historyCopy
26+ │
27+ └─ ProcessTurn (llm/loop.go:32-116)
28+ │
29+ └─ Loop: round = 0 to MaxToolRounds (20)
30+ │
31+ ├─ Compress historyCopy → compressedHistory
32+ │
33+ ├─ LLM Call with compressedHistory + tools
34+ │ └─ Returns: content, tool_calls, reasoning
35+ │
36+ ├─ If no tool_calls:
37+ │ ├─ Display content
38+ │ ├─ OnMessage(assistant_msg)
39+ │ ├─ Append to historyCopy
40+ │ └─ RETURN (turn complete)
41+ │
42+ └─ If tool_calls present:
43+ ├─ OnMessage(assistant_msg_with_tools)
44+ ├─ Append to historyCopy
45+ │
46+ ├─ executeToolCalls:
47+ │ └─ For each tool:
48+ │ ├─ Execute via MCP proxy
49+ │ ├─ OnMessage(tool_result)
50+ │ └─ Collect tool_result
51+ │
52+ ├─ Append all tool_results to historyCopy
53+ └─ Continue to next round
54+```
55+
56+## Implementation Details
57+
58+### Entry Point: `ProcessTurn`
59+
60+**Location:** `internal/llm/loop.go:32-116`
61+
62+**Signature:**
63+
64+```go
65+func ProcessTurn(ctx context.Context, opts ProcessTurnOptions) error
66+```
67+
68+**Options:**
69+
70+- `Provider`: LLM provider interface (Ollama, OpenCode Zen, etc.)
71+- `Proxy`: MCP proxy for tool execution
72+- `Tools`: Available tools for this turn
73+- `History`: Initial conversation history snapshot
74+- `OnMessage`: Callback to save messages to database and `app.history`
75+- `MaxToolRounds`: Maximum iterations (default: 20)
76+- `HistoryKeepLast`: How many recent turns to keep uncompressed (default: 10)
77+
78+### Round Loop
79+
80+Each round performs:
81+
82+1. **History Compression** (line 43)
83+ - Compresses old state query results
84+ - Keeps recent turns full
85+ - See `internal/store/compression.go` for details
86+
87+2. **LLM Call** (line 69)
88+ - Sends compressed history + available tools
89+ - Receives response with optional tool calls
90+
91+3. **Response Handling**
92+ - **No tool calls** (lines 80-93): Display content, save message, update history, exit
93+ - **Tool calls present** (lines 96-110): Save assistant message, execute tools, update history, continue
94+
95+4. **History Update** (Critical Fix - Bug #1)
96+ - After `OnMessage()` callback, message is also appended to `opts.History`
97+ - This ensures next round sees messages from previous rounds
98+ - Without this, LLM would repeat tool calls indefinitely
99+
100+### Tool Execution: `executeToolCalls`
101+
102+**Location:** `internal/llm/loop.go:130-195`
103+
104+**Signature:**
105+
106+```go
107+func executeToolCalls(ctx context.Context, proxy *mcp.Proxy, toolCalls []provider.ToolCall, onMessage MessageCallback) ([]provider.Message, error)
108+```
109+
110+**Flow:**
111+
112+1. For each tool call:
113+ - Display tool name and arguments (truncated)
114+ - Execute via MCP proxy
115+ - Display result (success ✓ or error ✗)
116+ - Create tool result message
117+ - Call `onMessage()` to save to database
118+ - Collect message for return
119+
120+2. Return all tool result messages
121+ - Caller appends to `opts.History`
122+ - Ensures next round sees tool results
123+
124+### Message Callback: `OnMessage`
125+
126+**Implementation:** `internal/cli/cli.go:285-293`
127+
128+```go
129+func (app *App) addMessage(msg provider.Message) {
130+ app.mu.Lock()
131+ app.history = append(app.history, msg)
132+ app.mu.Unlock()
133+
134+ if err := app.sessionMgr.SaveMessage(app.sessionID, msg); err != nil {
135+ log.Warn().Err(err).Msg("Failed to save message to database")
136+ }
137+}
138+```
139+
140+**Purpose:**
141+
142+- Updates canonical `app.history` with mutex protection
143+- Saves message to SQLite database for session persistence
144+- Called for every message: user, assistant, tool results
145+
146+**Note:** The callback updates `app.history` but NOT `opts.History`. The caller (`ProcessTurn`) must also update `opts.History` after calling the callback.
147+
148+## History Management
149+
150+### Snapshot vs Reference
151+
152+**Design Decision:** Pass history snapshot, not reference.
153+
154+**Rationale:**
155+
156+- Prevents concurrent modification during processing
157+- Clean separation between "input to turn" and "output from turn"
158+- Next turn gets fresh snapshot with all previous messages
159+
160+**Implementation:**
161+
162+```go
163+// cli.go:268-271
164+app.mu.Lock()
165+historyCopy := make([]provider.Message, len(app.history))
166+copy(historyCopy, app.history)
167+app.mu.Unlock()
168+```
169+
170+### History Accumulation During Turn
171+
172+**Critical Fix (2026-02-10):** After each message is saved via `OnMessage()`, it's also appended to `opts.History`.
173+
174+**Before Fix:**
175+
176+```go
177+opts.OnMessage(assistantMsg)
178+// opts.History unchanged - next round uses stale history
179+```
180+
181+**After Fix:**
182+
183+```go
184+opts.OnMessage(assistantMsg)
185+opts.History = append(opts.History, assistantMsg)
186+// Next round sees this message
187+```
188+
189+**Why This Matters:**
190+
191+Round 0:
192+
193+- `opts.History = [user: "check status"]`
194+- LLM calls `get_status`
195+- Tool result saved via `OnMessage` to `app.history`
196+- **Before fix:** `opts.History` still `[user: "check status"]`
197+- **After fix:** `opts.History = [user: "check status", assistant: tool_call, tool: result]`
198+
199+Round 1:
200+
201+- **Before fix:** LLM sees compressed `[user: "check status"]` → calls `get_status` again (doesn't see result)
202+- **After fix:** LLM sees compressed `[user: "check status", assistant: tool_call, tool: result]` → processes result
203+
204+## Compression During Turn
205+
206+**Key Insight:** Compression operates on `opts.History`, which now includes messages from previous rounds within the same turn.
207+
208+**Process:**
209+
210+1. `CompressHistory(opts.History, keepFullTurns=10)` compresses old state queries
211+2. Recent turns (last 10) kept full
212+3. Old action tool results truncated to 200 chars
213+4. Authentication tools never compressed
214+
215+**See:** `internal/store/compression.go` for compression logic
216+
217+## Loop Termination
218+
219+**Success Exit:**
220+
221+- LLM returns response without tool calls (line 91)
222+- Turn completes, user regains control
223+
224+**Error Exit:**
225+
226+- LLM call fails (line 70-72)
227+- Tool execution fails (line 107-109)
228+- Max rounds exceeded (line 115)
229+
230+**Max Rounds:** 20 iterations prevents infinite loops if LLM repeatedly calls same tool.
231+
232+## Message Roles
233+
234+### User Messages
235+
236+- **Origin:** User input or autoplay
237+- **Added:** Before `processTurn()` call
238+- **Location:** `cli.go:246` (manual), `autoplay.go:190` (autoplay)
239+
240+### Assistant Messages
241+
242+- **Origin:** LLM response
243+- **Added:** During `ProcessTurn` round
244+- **Variants:**
245+ - Text only: `Role: "assistant", Content: "..."`
246+ - With tool calls: `Role: "assistant", Content: "...", ToolCalls: [...]`
247+
248+### Tool Messages
249+
250+- **Origin:** Tool execution results
251+- **Added:** During `executeToolCalls`
252+- **Format:** `Role: "tool", Content: "result", ToolCallID: "..."`
253+
254+## Thread Safety
255+
256+**Mutex Protection:**
257+
258+- `app.mu` protects `app.history` (cli.go:34)
259+- Lock acquired for snapshot creation (cli.go:268-271)
260+- Lock acquired for message append via `addMessage()` (cli.go:286-288)
261+- `opts.History` is local to `ProcessTurn`, no lock needed
262+
263+**Concurrency Scenarios:**
264+
265+- Manual input and autoplay both call `processTurn()` serially
266+- Each turn operates on its own snapshot
267+- Callbacks serialize access to `app.history` via mutex
268+
269+## Error Handling
270+
271+**LLM Call Failure:**
272+
273+- Returns error immediately
274+- Caller displays error to user
275+- Turn aborted
276+
277+**Tool Call Failure:**
278+
279+- Creates error tool result: `Content: "Error: ..."`
280+- Continues to next tool in list
281+- LLM sees error in next round, can recover
282+
283+**Max Rounds Exceeded:**
284+
285+- Returns error: `"too many tool call rounds (limit: 20)"`
286+- Indicates LLM stuck in loop (bug or model issue)
287+- User sees error, can try different prompt
288+
289+## Performance Characteristics
290+
291+**History Compression:**
292+
293+- Reduces token usage for long conversations
294+- Trade-off: Old details lost vs context window limits
295+- Compression runs every round (negligible overhead)
296+
297+**Round Complexity:**
298+
299+- Each round: 1 LLM call + N tool executions
300+- Typical turn: 1-5 rounds
301+- Worst case: 20 rounds (max limit)
302+
303+**Memory:**
304+
305+- `opts.History` grows during turn (O(messages))
306+- Snapshot copied at turn start (O(messages))
307+- Released at turn end
308+
309+## Configuration
310+
311+**MaxToolRounds:** 20
312+
313+- Prevents infinite loops
314+- Increase if legitimate use cases need more rounds
315+- Decrease to fail fast during debugging
316+
317+**HistoryKeepLast:** 10
318+
319+- Number of recent turns to keep uncompressed
320+- Increase for more context (higher token usage)
321+- Decrease for faster compression (less context)
322+
323+## Future Improvements
324+
325+**Streaming:** Support streaming LLM responses for faster perceived performance
326+
327+**History Window:** Smarter compression based on token count, not just turn count