@@ -0,0 +1,242 @@
1+# Gemini: Streaming Workaround for the Zen Proxy
2+
3+## Background
4+
5+The opencode zen proxy (`https://opencode.ai/zen/v1`) is a server-side gateway that
6+translates between the native wire formats of different AI providers and a unified
7+request/response model used by the opencode platform. Clients send requests to
8+provider-specific endpoint prefixes and the proxy forwards them to the correct
9+upstream API, handles authentication, tracks usage for billing, and converts
10+responses back.
11+
12+The three main endpoint families are:
13+
14+| Prefix | Provider | Auth header |
15+|---------------------|-----------|---------------------|
16+| `/responses` | OpenAI | `Authorization: Bearer` |
17+| `/messages` | Anthropic | `x-api-key` |
18+| `/models/<model>` | Google | `x-goog-api-key` |
19+
20+Gemini models are accessed through the `/models/<model>` prefix. The proxy route
21+that handles this is a SolidStart dynamic segment (`[model].ts`) which reads the
22+model name from the URL and dispatches to the `googleHelper` provider adapter.
23+
24+---
25+
26+## The Bug
27+
28+### Server-side: `json.usage` vs `json.usageMetadata`
29+
30+The zen proxy's central request handler (`handler.ts`) has a single code path for
31+non-streaming responses:
32+
33+```typescript
34+// handler.ts ~line 190
35+if (!isStream) {
36+ const json = await res.json()
37+ const usageInfo = providerInfo.normalizeUsage(json.usage) // <-- BUG
38+ ...
39+}
40+```
41+
42+`json.usage` is the field name used by OpenAI-style responses. Gemini's upstream
43+API, however, places token counts inside a different field called `usageMetadata`:
44+
45+```json
46+{
47+ "candidates": [...],
48+ "usageMetadata": {
49+ "promptTokenCount": 11,
50+ "candidatesTokenCount": 3,
51+ "totalTokenCount": 14
52+ }
53+}
54+```
55+
56+When the proxy calls `normalizeUsage(json.usage)` for a Gemini response,
57+`json.usage` is `undefined`. The `googleHelper.normalizeUsage` function then
58+immediately tries to read `usage.promptTokenCount`, which throws:
59+
60+```
61+Cannot read properties of undefined (reading 'promptTokenCount')
62+```
63+
64+The proxy returns this as an HTTP 500 to the client.
65+
66+### Client-side: hardcoded `Authorization: Bearer` in streaming
67+
68+A second, independent bug existed in this SDK's own `startStream` function
69+(`stream.go`). It hardcoded the OpenAI-style authorization header for all
70+requests, regardless of provider:
71+
72+```go
73+// stream.go (before fix)
74+req.Header.Set("Authorization", "Bearer "+c.cfg.APIKey)
75+```
76+
77+The non-streaming `doRequest` function in `http.go` already had the correct
78+per-provider routing:
79+
80+```go
81+switch {
82+case strings.HasPrefix(path, "/messages"):
83+ req.Header.Set("x-api-key", c.cfg.APIKey)
84+case strings.HasPrefix(path, "/models/") && strings.Contains(path, "gemini"):
85+ req.Header.Set("x-goog-api-key", c.cfg.APIKey)
86+default:
87+ req.Header.Set("Authorization", "Bearer "+c.cfg.APIKey)
88+}
89+```
90+
91+But `startStream` bypassed `doRequest` entirely and opened its own `http.Request`,
92+so this routing logic was never applied. Any streaming Gemini call would send
93+`Authorization: Bearer <key>` and be rejected by the proxy's authentication layer.
94+
95+---
96+
97+## Why the opencode CLI Was Not Affected
98+
99+The opencode CLI uses `@ai-sdk/google` (the Vercel AI SDK's Google adapter) to
100+talk to Gemini. That adapter **always streams** — it sends requests to
101+`:streamGenerateContent?alt=sse` and processes the SSE response incrementally
102+using `streamText`. It never calls the non-streaming path in the proxy.
103+
104+The proxy's streaming handler reads usage correctly:
105+
106+```typescript
107+// google.ts — createUsageParser
108+parse: (chunk: string) => {
109+ ...
110+ json = JSON.parse(chunk.slice(6)) as { usageMetadata?: Usage }
111+ if (!json.usageMetadata) return
112+ usage = json.usageMetadata // reads the right field
113+},
114+retrieve: () => usage,
115+```
116+
117+And the streaming section of `handler.ts` guards against `usage` being absent:
118+
119+```typescript
120+const usage = usageParser.retrieve()
121+if (usage) { // safe: guarded
122+ const usageInfo = providerInfo.normalizeUsage(usage)
123+ ...
124+}
125+```
126+
127+So the CLI avoids the crash entirely by never taking the broken non-streaming code
128+path.
129+
130+---
131+
132+## URL Format: `:generateContent` and `:streamGenerateContent?alt=sse`
133+
134+Google's Gemini API differentiates between the two generation modes via an
135+**action suffix** appended to the model path:
136+
137+| Mode | URL suffix |
138+|----------------|-----------------------------------|
139+| Non-streaming | `:generateContent` |
140+| Streaming SSE | `:streamGenerateContent?alt=sse` |
141+
142+The zen proxy's model route handler parses this suffix to determine `isStream`:
143+
144+```typescript
145+// [model].ts
146+parseModel: (url, body) => url.split("/").pop()?.split(":")?.[0] ?? "",
147+parseIsStream: (url, body) =>
148+ url.split("/").pop()?.split(":")?.[1]?.startsWith("streamGenerateContent") ?? false,
149+```
150+
151+Without the suffix, `parseIsStream` returns `false` and the request takes the
152+non-streaming path — which is the broken one for Gemini.
153+
154+This SDK initially called `/models/<model>` with no suffix at all. While the proxy
155+still routes the request correctly to `googleHelper`, it falls through to the
156+`!isStream` branch and crashes on `json.usage`.
157+
158+---
159+
160+## The Workaround
161+
162+Since we cannot modify the server, both fixes are implemented client-side.
163+
164+### Fix 1 — `stream.go`: per-provider auth header in `startStream`
165+
166+`startStream` now mirrors the same switch that `doRequest` uses:
167+
168+```go
169+switch {
170+case strings.HasPrefix(path, "/messages"):
171+ req.Header.Set("x-api-key", c.cfg.APIKey)
172+case strings.HasPrefix(path, "/models/") && strings.Contains(path, "gemini"):
173+ req.Header.Set("x-goog-api-key", c.cfg.APIKey)
174+default:
175+ req.Header.Set("Authorization", "Bearer "+c.cfg.APIKey)
176+}
177+```
178+
179+This ensures that streaming Gemini requests are authenticated correctly.
180+
181+### Fix 2 — `gemini.go`: `CreateModelContent` uses streaming internally
182+
183+`CreateModelContent` (the public non-streaming API) now calls
184+`:streamGenerateContent?alt=sse` internally, collects all SSE chunks from the
185+stream, and returns the last non-empty chunk as the response:
186+
187+```go
188+func (c *Client) CreateModelContent(ctx context.Context, model string, body any, raw json.RawMessage) (json.RawMessage, error) {
189+ payload, err := jsonBody(body, raw)
190+ if err != nil {
191+ return nil, err
192+ }
193+
194+ stream, err := c.startStream(ctx, "POST", "/models/"+model+":streamGenerateContent?alt=sse", payload)
195+ if err != nil {
196+ return nil, err
197+ }
198+ defer func() { _ = stream.Close() }()
199+
200+ var last json.RawMessage
201+ for ev := range stream.Events {
202+ if len(ev.Data) > 0 {
203+ last = ev.Data
204+ }
205+ }
206+ if stream.Err != nil {
207+ return nil, stream.Err
208+ }
209+ if last == nil {
210+ return nil, fmt.Errorf("gemini: empty response")
211+ }
212+ return last, nil
213+}
214+```
215+
216+The Gemini streaming SSE format sends one JSON object per chunk. Each chunk
217+represents a turn of candidate content. The final chunk always carries the full
218+`usageMetadata`. By returning the last chunk, callers get both the generated text
219+and accurate token counts.
220+
221+`StreamModelContent` (the public streaming API) also uses the action suffix:
222+
223+```go
224+stream, err := c.startStream(ctx, "POST", "/models/"+model+":streamGenerateContent?alt=sse", payload)
225+```
226+
227+---
228+
229+## Outcome
230+
231+After both fixes, all Gemini models pass the integration test:
232+
233+```
234+✓ gemini-3-pro [models] 3.246s
235+✓ gemini-3-flash [models] 1.351s
236+```
237+
238+The approach matches exactly what `@ai-sdk/google` does: always stream, always
239+use the `:streamGenerateContent?alt=sse` action suffix, read usage from
240+`usageMetadata` in the final SSE chunk. The public `CreateModelContent` API
241+remains synchronous from the caller's perspective — the streaming is an
242+implementation detail.