@@ -0,0 +1,135 @@
1+# Image Support Plan
2+
3+## Overview
4+
5+Three entry points, one unified path: images from `@file`, clipboard paste, or terminal drag-and-drop all resolve to **base64 image content parts** that get inserted into the user `CoreMessage` sent to the LLM.
6+
7+---
8+
9+## 1. Data Model — extend `CoreMessage` content
10+
11+The AI SDK's `CoreUserMessage` already supports multi-part content: `{ role: "user", content: [...parts] }` where a part can be `{ type: "image", image: <base64|URL>, mimeType: "image/png" }`. No new types needed — we just need to build content arrays instead of plain strings.
12+
13+**Change in `src/agent/agent.ts` `processUserInput`:**
14+```ts
15+// Today:
16+const userMsg: CoreMessage = { role: "user", content: coreContent };
17+
18+// With images:
19+const userMsg: CoreMessage = {
20+ role: "user",
21+ content: [
22+ { type: "text", text: coreContent },
23+ ...images, // ImagePart[]
24+ ],
25+};
26+```
27+
28+---
29+
30+## 2. Source A — `@path` image file references (`src/agent/agent.ts`)
31+
32+Extend `resolveFileRefs` (or split it into `resolveFileRefs` + `extractImageRefs`):
33+
34+- Detect image extensions in `@` tokens: `.png`, `.jpg`, `.jpeg`, `.gif`, `.webp`, `.svg`
35+- Instead of inlining as a fenced code block, **remove** the token from the text and return an `ImagePart` array alongside the resolved text
36+- The function signature becomes:
37+ ```ts
38+ async function resolveRefs(
39+ text: string, cwd: string
40+ ): Promise<{ text: string; images: ImagePart[] }>
41+ ```
42+- Update the TAB autocomplete in `src/cli/input.ts` to also surface image files (already works via glob)
43+
44+---
45+
46+## 3. Source B — Clipboard paste (`src/cli/input.ts`)
47+
48+Terminals emit **bracketed paste** sequences when pasting: `\x1B[?2004h` to enable, `\x1B[200~...content...\x1B[201~` wrapping the pasted data.
49+
50+- Enable bracketed paste mode on raw input start: write `\x1B[?2004h`
51+- Disable on exit: write `\x1B[?2004l`
52+- In the input loop, detect `\x1B[200~` … `\x1B[201~` sequences
53+- If bracketed content looks like an **image** (base64 data URI `data:image/...;base64,...`), extract it as an image — don't insert it into the text buffer
54+- Surface it as a new `InputResult` variant: `{ type: "submit"; text: string; images: ImagePart[] }` (images field added, optional/defaults to `[]`)
55+
56+For actual image data pasted from clipboard (e.g. from macOS Screenshot), the image bytes arrive as raw binary, not text. We can detect that with a quick magic-bytes check (`\x89PNG`, `\xff\xd8\xff`, etc.) and base64-encode it ourselves.
57+
58+---
59+
60+## 4. Source C — Terminal drag-and-drop / file drop
61+
62+Most modern terminals (iTerm2, Kitty, WezTerm) emit file paths when files are dropped as either:
63+- Plain text paths (already handled by `@` detection if user types `@`, but auto-drops won't have `@`)
64+- **Escape sequences** like `\x1B]1337;File=...` (iTerm2 inline image protocol) or `\x1B]5113;...` (Kitty file drop)
65+
66+Strategy:
67+- In the input loop, detect these escape sequences and extract the file path or inline image data
68+- If the dropped item is an image path, read it as an `ImagePart`
69+- If it's a non-image file path, auto-insert `@path` into the buffer (existing file ref behavior)
70+- If it's raw inline image data (iTerm2 `\x1B]1337;File=inline=1;...:<base64>`), extract the base64 directly
71+
72+---
73+
74+## 5. `InputResult` type extension (`src/cli/input.ts`)
75+
76+```ts
77+export type InputResult =
78+ | { type: "submit"; text: string; images?: ImagePart[] }
79+ | { type: "interrupt" }
80+ | { type: "eof" }
81+ | { type: "command"; command: string; args: string }
82+ | { type: "shell"; command: string };
83+```
84+
85+The `images` field is optional so nothing breaks at existing call sites.
86+
87+---
88+
89+## 6. UI feedback (`src/cli/output.ts`)
90+
91+When images are attached, render a compact indicator below the prompt echo:
92+```
93+› analyze this screenshot
94+ 📎 image/png 1024×768 (paste)
95+ 📎 src/assets/logo.png (file)
96+```
97+A new `renderImageAttachments(images: ImageMeta[])` helper, where `ImageMeta` carries mime type, source, and optionally dimensions (from magic bytes).
98+
99+---
100+
101+## 7. New module: `src/cli/images.ts`
102+
103+Isolate all image logic here:
104+- `detectMimeType(bytes: Uint8Array): string | null` — magic bytes check
105+- `readImageFile(path: string): Promise<ImagePart | null>` — file → base64 ImagePart
106+- `parsePastedImage(raw: Uint8Array): ImagePart | null` — bracketed paste → ImagePart
107+- `parseITermDrop(seq: string): ImagePart | null` — iTerm2 escape → ImagePart
108+- Constants for supported extensions and magic bytes
109+
110+This keeps `input.ts` and `agent.ts` clean.
111+
112+---
113+
114+## 8. Session persistence
115+
116+Images in messages are already persisted via `saveMessages` — the AI SDK `CoreMessage` with image parts serializes to JSON cleanly in SQLite. No schema changes needed.
117+
118+---
119+
120+## Implementation order
121+
122+1. **`src/cli/images.ts`** — new module, pure functions, fully testable
123+2. **`src/agent/agent.ts`** — extend `resolveRefs` to handle image `@` references
124+3. **`src/cli/input.ts`** — bracketed paste + file drop detection
125+4. **`src/cli/output.ts`** — `renderImageAttachments` helper
126+5. **Wire up** in `agent.ts` `processUserInput` and the REPL switch
127+
128+---
129+
130+## Key constraints
131+
132+- Only models that support vision will accept image parts — the AI SDK will surface a provider error if the model doesn't; we don't need to guard this ourselves
133+- Keep images **out of prompt history display** (don't echo base64 to terminal)
134+- Max image size guard: reject files > ~5 MB before sending (configurable constant)
135+- No new dependencies — Bun's built-in `Bun.file` for reading, `Buffer.from(...).toString("base64")` for encoding