@@ -0,0 +1,420 @@
1+# Symb Tools Documentation
2+
3+This document provides comprehensive details on Symb's tool system, including architecture, implementation, built-in tools, MCP integration, and development guides.
4+
5+## Table of Contents
6+
7+1. [Architecture Overview](#architecture-overview)
8+2. [Tool Types](#tool-types)
9+3. [Built-in Tools](#built-in-tools)
10+4. [MCP Integration](#mcp-integration)
11+5. [Tool Development Guide](#tool-development-guide)
12+6. [Security Model](#security-model)
13+7. [Configuration](#configuration)
14+8. [Testing Tools](#testing-tools)
15+9. [Performance Considerations](#performance-considerations)
16+10. [Troubleshooting](#troubleshooting)
17+
18+---
19+
20+## Architecture Overview
21+
22+### System Design
23+
24+Symb implements a **layered tool architecture** with MCP (Model Context Protocol) at its core:
25+
26+```
27+┌─────────────────────────────────────────────────────────┐
28+│ LLM Providers │
29+│ (Ollama, OpenCode, etc.) │
30+└────────────────┬────────────────────────────────────────┘
31+ │ ChatWithTools()
32+┌────────────────▼────────────────────────────────────────┐
33+│ LLM Loop (internal/llm) │
34+│ - ProcessTurn: orchestrates tool calls │
35+│ - Max 20 rounds per turn │
36+│ - Sequential tool execution │
37+└────────────────┬────────────────────────────────────────┘
38+ │ CallTool()
39+┌────────────────▼────────────────────────────────────────┐
40+│ MCP Proxy (internal/mcp) │
41+│ - Routes to local or upstream tools │
42+│ - Retry logic for rate limits │
43+│ - Error handling & normalization │
44+└────┬───────────────────────────────────────────┬────────┘
45+ │ │
46+ │ Local Tools │ Upstream MCP
47+ ▼ ▼
48+┌────────────────┐ ┌─────────────────┐
49+│ Open Tool │ │ MCP Client │
50+│ Grep Tool │ │ (HTTP/SSE) │
51+└────────────────┘ └─────────────────┘
52+```
53+
54+---
55+
56+## Tool Types
57+
58+### 1. Local Tools
59+
60+**Definition**: Tools implemented directly in Symb's codebase.
61+
62+**Location**: `internal/mcp_tools/`
63+
64+**Registration**: `cmd/symb/main.go`
65+
66+**Current Local Tools**:
67+
68+- `Open` - File viewer with syntax highlighting
69+- `Grep` - File/content search
70+
71+**Characteristics**:
72+
73+- ✓ No network latency
74+- ✓ Full control over implementation
75+- ✓ Direct TUI integration
76+- ✓ Custom security policies
77+
78+### 2. Upstream MCP Tools
79+
80+**Definition**: Tools provided by external MCP servers via HTTP/SSE.
81+
82+**Protocol**: Model Context Protocol (MCP)
83+
84+**Configuration**: `config.toml` → `[mcp]` section
85+
86+**Characteristics**:
87+
88+- ✓ Hot-pluggable (no code changes)
89+- ✓ Leverage existing MCP ecosystem
90+- ✓ Can call external APIs/services
91+- ✗ Network latency
92+- ✗ Requires upstream server availability
93+- ✗ Limited security control
94+
95+### Retry Logic
96+
97+Handles rate limits and transient failures:
98+
99+```go
100+// Retry delays: 2s, 5s, 10s
101+var toolRetryDelays = []time.Duration{
102+ 2 * time.Second,
103+ 5 * time.Second,
104+ 10 * time.Second,
105+}
106+
107+func (p *Proxy) callUpstreamWithRetry(ctx context.Context, name string, args json.RawMessage) (*ToolResult, error) {
108+ var lastErr error
109+
110+ for attempt := 0; attempt <= len(toolRetryDelays); attempt++ {
111+ result, err := p.upstream.CallTool(ctx, name, args)
112+ if err == nil {
113+ return result, nil
114+ }
115+
116+ lastErr = err
117+
118+ // Check if we should retry
119+ if !shouldRetry(err) {
120+ return nil, err
121+ }
122+
123+ // Check for Retry-After header (429 responses)
124+ if retryAfter, ok := parseRetryAfter(err); ok {
125+ delay = retryAfter
126+ } else {
127+ delay = toolRetryDelays[attempt]
128+ }
129+
130+ // Wait with context cancellation support
131+ select {
132+ case <-time.After(delay):
133+ continue
134+ case <-ctx.Done():
135+ return nil, ctx.Err()
136+ }
137+ }
138+
139+ return nil, fmt.Errorf("retry exhausted after %d attempts: %w", len(toolRetryDelays)+1, lastErr)
140+}
141+```
142+
143+**Features**:
144+
145+- Respects `Retry-After` HTTP headers
146+- Exponential backoff (2s → 5s → 10s)
147+- Context-aware (cancellation support)
148+- Logs retry attempts
149+- Returns meaningful error after exhaustion
150+
151+---
152+
153+## Tool Development Guide
154+
155+### Creating a Local Tool
156+
157+**Step 1: Define the tool schema**
158+
159+```go
160+// internal/mcp_tools/mytool.go
161+package mcp_tools
162+
163+import (
164+ "context"
165+ "encoding/json"
166+ "github.com/xonecas/symb/internal/mcp"
167+)
168+
169+func NewMyTool() mcp.Tool {
170+ schema := map[string]interface{}{
171+ "type": "object",
172+ "properties": map[string]interface{}{
173+ "param1": map[string]interface{}{
174+ "type": "string",
175+ "description": "First parameter description",
176+ },
177+ "param2": map[string]interface{}{
178+ "type": "integer",
179+ "description": "Second parameter (optional)",
180+ },
181+ },
182+ "required": []string{"param1"},
183+ }
184+
185+ schemaJSON, _ := json.Marshal(schema)
186+
187+ return mcp.Tool{
188+ Name: "MyTool",
189+ Description: "Does something useful with param1 and optionally param2",
190+ InputSchema: schemaJSON,
191+ }
192+}
193+```
194+
195+**Step 2: Implement the handler**
196+
197+```go
198+type MyToolArgs struct {
199+ Param1 string `json:"param1"`
200+ Param2 int `json:"param2,omitempty"`
201+}
202+
203+func MakeMyToolHandler() mcp.ToolHandler {
204+ return func(ctx context.Context, arguments json.RawMessage) (*mcp.ToolResult, error) {
205+ var args MyToolArgs
206+ if err := json.Unmarshal(arguments, &args); err != nil {
207+ return &mcp.ToolResult{
208+ Content: []mcp.ContentBlock{{
209+ Type: "text",
210+ Text: "Invalid arguments: " + err.Error(),
211+ }},
212+ IsError: true,
213+ }, nil
214+ }
215+
216+ // Validate inputs
217+ if args.Param1 == "" {
218+ return &mcp.ToolResult{
219+ Content: []mcp.ContentBlock{{
220+ Type: "text",
221+ Text: "param1 cannot be empty",
222+ }},
223+ IsError: true,
224+ }, nil
225+ }
226+
227+ // Execute tool logic
228+ result := doSomething(args.Param1, args.Param2)
229+
230+ // Return success
231+ return &mcp.ToolResult{
232+ Content: []mcp.ContentBlock{{
233+ Type: "text",
234+ Text: result,
235+ }},
236+ IsError: false,
237+ }, nil
238+ }
239+}
240+
241+func doSomething(param1 string, param2 int) string {
242+ // Your tool implementation here
243+ return "Result from MyTool"
244+}
245+```
246+
247+**Step 3: Register the tool**
248+
249+```go
250+// cmd/symb/main.go
251+func main() {
252+ // ... existing code ...
253+
254+ // Create MCP proxy
255+ proxy := mcp.NewProxy(mcpClient)
256+ proxy.Initialize(context.Background())
257+
258+ // Register existing tools
259+ proxy.RegisterTool(mcp_tools.NewOpenForUserTool(), mcp_tools.NewOpenForUserHandler().Handle)
260+ proxy.RegisterTool(mcp_tools.NewGrepTool(), mcp_tools.MakeGrepHandler())
261+
262+ // Register your new tool
263+ proxy.RegisterTool(mcp_tools.NewMyTool(), mcp_tools.MakeMyToolHandler())
264+
265+ // ... rest of main ...
266+}
267+```
268+
269+**Step 4: Update system prompts**
270+
271+Add tool description to all prompt files (`internal/llm/*.md`):
272+
273+````markdown
274+### `MyTool`
275+
276+Does something useful with param1 and optionally param2.
277+
278+**Parameters:**
279+
280+```json
281+{
282+ "param1": "string (required)",
283+ "param2": 123 // optional integer
284+}
285+```
286+````
287+
288+**Example:**
289+
290+```json
291+{ "param1": "example", "param2": 42 }
292+```
293+
294+````
295+
296+**Step 5: Test the tool**
297+
298+```go
299+// internal/mcp_tools/mytool_test.go
300+package mcp_tools
301+
302+import (
303+ "context"
304+ "encoding/json"
305+ "testing"
306+)
307+
308+func TestMyTool(t *testing.T) {
309+ handler := MakeMyToolHandler()
310+
311+ tests := []struct {
312+ name string
313+ args map[string]interface{}
314+ wantErr bool
315+ wantText string
316+ }{
317+ {
318+ name: "valid input",
319+ args: map[string]interface{}{
320+ "param1": "test",
321+ "param2": 42,
322+ },
323+ wantErr: false,
324+ wantText: "Result from MyTool",
325+ },
326+ {
327+ name: "missing required param",
328+ args: map[string]interface{}{},
329+ wantErr: true,
330+ },
331+ }
332+
333+ for _, tt := range tests {
334+ t.Run(tt.name, func(t *testing.T) {
335+ argsJSON, _ := json.Marshal(tt.args)
336+
337+ result, err := handler(context.Background(), argsJSON)
338+ if err != nil {
339+ t.Fatalf("unexpected error: %v", err)
340+ }
341+
342+ if result.IsError != tt.wantErr {
343+ t.Errorf("IsError = %v, want %v", result.IsError, tt.wantErr)
344+ }
345+
346+ if !tt.wantErr && result.Content[0].Text != tt.wantText {
347+ t.Errorf("Text = %q, want %q", result.Content[0].Text, tt.wantText)
348+ }
349+ })
350+ }
351+}
352+````
353+
354+### TUI Integration
355+
356+If your tool needs to update the TUI:
357+
358+**Step 1: Define a message type**
359+
360+```go
361+// internal/mcp_tools/mytool.go
362+type MyToolMsg struct {
363+ Data string
364+}
365+```
366+
367+**Step 2: Send message from handler**
368+
369+```go
370+func MakeMyToolHandler(updateChan chan<- interface{}) mcp.ToolHandler {
371+ return func(ctx context.Context, arguments json.RawMessage) (*mcp.ToolResult, error) {
372+ // ... handler logic ...
373+
374+ // Send message to TUI
375+ if updateChan != nil {
376+ updateChan <- MyToolMsg{Data: "something"}
377+ }
378+
379+ return result, nil
380+ }
381+}
382+```
383+
384+**Step 3: Handle message in TUI**
385+
386+```go
387+// internal/tui/tui.go
388+func (m model) Update(msg tea.Msg) (tea.Model, tea.Cmd) {
389+ switch msg := msg.(type) {
390+ case mcp_tools.MyToolMsg:
391+ // Update model state based on message
392+ m.someField = msg.Data
393+ return m, nil
394+ // ... other cases ...
395+ }
396+}
397+```
398+
399+---
400+
401+## References
402+
403+**Standards**:
404+
405+- [Model Context Protocol Specification](https://modelcontextprotocol.io/)
406+- [JSON Schema](https://json-schema.org/)
407+
408+**Inspiration**:
409+
410+- OpenCode MCP implementation
411+- Anthropic Claude tool use
412+- OpenAI function calling
413+
414+**Related Docs**:
415+
416+- `docs/PROMPTS.md` - System prompt documentation
417+- `docs/DESIGN.md` - Architecture overview
418+- `docs/TUI_TESTING.md` - TUI testing guide
419+- `internal/llm/loop.go` - LLM interaction loop
420+- `internal/mcp/proxy.go` - MCP proxy implementation