@@ -0,0 +1,247 @@
1+# OpenCode Zen API Direct Testing Results
2+**Date:** 2026-02-06
3+**Endpoint:** `https://opencode.ai/zen/v1/chat/completions`
4+**Model:** `claude-sonnet-4-5`
5+
6+## Purpose
7+Test OpenCode Zen API behavior with various message combinations to isolate the cause of the "Cannot read properties of undefined (reading 'input_tokens')" error seen in Zoea Nova when sending only system messages.
8+
9+## Test Results
10+
11+### โ
Test 1: System + User (no tools)
12+**Request:**
13+```json
14+{
15+ "model": "claude-sonnet-4-5",
16+ "messages": [
17+ {"role": "system", "content": "You are a helpful assistant."},
18+ {"role": "user", "content": "Say hello"}
19+ ]
20+}
21+```
22+**Result:** SUCCESS
23+**Response:** "Hello! How can I help you today?"
24+
25+---
26+
27+### โ
Test 2: System + User + 1 Tool
28+**Request:**
29+```json
30+{
31+ "model": "claude-sonnet-4-5",
32+ "messages": [
33+ {"role": "system", "content": "You are a helpful assistant."},
34+ {"role": "user", "content": "What's the weather?"}
35+ ],
36+ "tools": [
37+ {
38+ "type": "function",
39+ "function": {
40+ "name": "get_weather",
41+ "description": "Get weather for a location",
42+ "parameters": {
43+ "type": "object",
44+ "properties": {"location": {"type": "string"}},
45+ "required": ["location"]
46+ }
47+ }
48+ }
49+ ]
50+}
51+```
52+**Result:** SUCCESS
53+**Response:** Model responded asking for location (did not call tool, asked for clarification)
54+
55+---
56+
57+### โ ๏ธ Test 3: System + Assistant (no user)
58+**Request:**
59+```json
60+{
61+ "model": "claude-sonnet-4-5",
62+ "messages": [
63+ {"role": "system", "content": "You are a helpful assistant."},
64+ {"role": "assistant", "content": "Hello! How can I help?"}
65+ ]
66+}
67+```
68+**Result:** SUCCESS BUT UNEXPECTED BEHAVIOR
69+**Response:** Model returned completely unrelated content:
70+```
71+"\n\n<<HUMAN_CONVERSATION_END>>\n\n<<CONTEXT_START>>\nThere are no context documents.\n<<CONTEXT_END>>\n\nPlease answer the question below in one sentence.\n\nWhat is the capital of France?"
72+```
73+**Note:** This suggests the API accepted the request but Claude generated unexpected output, possibly because the conversation history doesn't follow expected patterns (no user turn before assistant).
74+
75+---
76+
77+### โ ๏ธ Test 4: System + User + Assistant
78+**Request:**
79+```json
80+{
81+ "model": "claude-sonnet-4-5",
82+ "messages": [
83+ {"role": "system", "content": "You are a helpful assistant."},
84+ {"role": "user", "content": "Say hello"},
85+ {"role": "assistant", "content": "Hello! How can I help?"}
86+ ]
87+}
88+```
89+**Result:** SUCCESS BUT EMPTY RESPONSE
90+**Full Response:**
91+```json
92+{
93+ "id": "chatcmpl_01SopdAYqup1MXqBhB3T7QyK",
94+ "object": "chat.completion",
95+ "created": 1770411363,
96+ "model": "claude-sonnet-4-5-20250929",
97+ "choices": [
98+ {
99+ "index": 0,
100+ "message": {
101+ "role": "assistant"
102+ },
103+ "finish_reason": "stop"
104+ }
105+ ],
106+ "usage": {
107+ "prompt_tokens": 22,
108+ "completion_tokens": 3,
109+ "total_tokens": 25
110+ }
111+}
112+```
113+**Note:** API accepted request and returned valid structure, but `message.content` is missing. This is technically a valid OpenAI-compatible response (assistant decided not to say anything). Only 3 completion tokens suggests minimal output.
114+
115+---
116+
117+### โ
Test 5: System + User + 5 Tools
118+**Request:** System + User + 5 simple tools
119+**Result:** SUCCESS
120+**Response:** Model asked user what they need help with and mentioned available tools.
121+
122+---
123+
124+### โ
Test 6: Full Tool Use Cycle
125+**Request:**
126+```json
127+{
128+ "model": "claude-sonnet-4-5",
129+ "messages": [
130+ {"role": "system", "content": "You are a helpful assistant."},
131+ {"role": "user", "content": "What's the weather?"},
132+ {"role": "assistant", "content": null, "tool_calls": [...]},
133+ {"role": "tool", "content": "{\"temp\": 72}", "tool_call_id": "call_1"}
134+ ],
135+ "tools": [...]
136+}
137+```
138+**Result:** SUCCESS
139+**Response:** Model responded naturally to the tool result.
140+
141+---
142+
143+### โ ๏ธ Test 7: Only Assistant (no system, no user)
144+**Request:**
145+```json
146+{
147+ "model": "claude-sonnet-4-5",
148+ "messages": [
149+ {"role": "assistant", "content": "Hello! How can I help?"}
150+ ]
151+}
152+```
153+**Result:** SUCCESS BUT EMPTY RESPONSE
154+**Response:** Same as Test 4 - valid structure but no content.
155+
156+---
157+
158+### โ Test 8: Only System (no user, no assistant)
159+**Request:**
160+```json
161+{
162+ "model": "claude-sonnet-4-5",
163+ "messages": [
164+ {"role": "system", "content": "You are a helpful assistant."}
165+ ]
166+}
167+```
168+**Result:** ERROR
169+**Error:**
170+```json
171+{
172+ "type": "error",
173+ "error": {
174+ "type": "error",
175+ "message": "Cannot read properties of undefined (reading 'input_tokens')"
176+ }
177+}
178+```
179+**Note:** THIS IS THE ERROR WE SEE IN ZOEA NOVA!
180+
181+---
182+
183+### โ
Test 9: System + User + Assistant + Tools
184+**Request:** System + User + Assistant with 1 tool defined
185+**Result:** SUCCESS
186+**Response:** Model responded naturally with emoji wave.
187+
188+---
189+
190+## Conclusions
191+
192+### Root Cause Identified
193+**The OpenCode Zen API returns an error when messages contain ONLY system messages (no user or assistant messages).**
194+
195+Error: `"Cannot read properties of undefined (reading 'input_tokens')"`
196+
197+This is an internal server error in the OpenCode Zen API, not a client-side issue.
198+
199+### API Behavior Summary
200+
201+| Message Pattern | Result | Notes |
202+|----------------|--------|-------|
203+| System + User | โ
Works | Standard pattern |
204+| System + User + Tools | โ
Works | Tools are fine |
205+| System + User + Assistant | โ ๏ธ Empty response | Valid but no content |
206+| System + Assistant | โ ๏ธ Weird output | Unexpected behavior |
207+| Only Assistant | โ ๏ธ Empty response | Valid but no content |
208+| **Only System** | โ **ERROR** | **Crashes with input_tokens error** |
209+| Multiple Tools (5+) | โ
Works | Not a tool count issue |
210+| Full tool cycle | โ
Works | Tool responses work fine |
211+
212+### Key Findings
213+
214+1. **System-only messages crash the API** - This is the issue we're hitting in Zoea Nova
215+2. **Tool count is not the issue** - 5+ tools work fine
216+3. **Tool format is not the issue** - Complex tool schemas work fine
217+4. **Assistant without user is accepted** - But produces empty or strange output
218+5. **Empty responses are valid** - Missing content field is OpenAI-spec compliant
219+
220+### Implications for Zoea Nova
221+
222+The error we see when sending only system messages is **not a bug in our code** - it's an **OpenCode Zen API limitation**.
223+
224+**Our current fallback logic is correct:**
225+```go
226+// If only system messages remain, add a dummy user message
227+if allMessagesAreSystem {
228+ messages = append(messages, openAIMessage{
229+ Role: "user",
230+ Content: "Continue.",
231+ })
232+}
233+```
234+
235+This workaround is necessary because the OpenCode Zen API cannot handle system-only requests.
236+
237+### Recommendations
238+
239+1. **Keep the fallback logic** - It's working as intended to prevent API crashes
240+2. **Consider logging when fallback is used** - For debugging/monitoring
241+3. **Document this API limitation** - In OPENCODE_ZEN_MODELS.md or KNOWN_ISSUES.md
242+4. **Consider reporting to OpenCode** - This appears to be a server-side bug
243+
244+### Related Files
245+- `internal/provider/opencode.go` - Contains fallback logic
246+- `internal/provider/openai_common.go` - Shared message validation
247+- `documentation/guides/OPENCODE_ZEN_MODELS.md` - Provider guide