@@ -1,459 +0,0 @@
1-# LangChain RAG Integration Plan
2-
3-## Overview
4-
5-This plan outlines the steps to integrate LangChain for implementing a Retrieval-Augmented Generation (RAG) system in the personal assistant application. The RAG system will use `data.txt` as the knowledge base.
6-
7-## Architecture Decision
8-
9-**Approach**: Integrate RAG directly into the chat generation flow. For each user message, automatically retrieve relevant context from the knowledge base and inject it into the conversation context before generation. This ensures the LLM always has access to relevant information from `data.txt` without requiring explicit tool calls.
10-
11-## Implementation Steps
12-
13-### 1. Install Dependencies
14-
15-The following LangChain packages are already installed:
16-
17-- `@langchain/core` - Core LangChain library ✅ (already installed)
18-- `@langchain/ollama` - Ollama integration for embeddings and LLM ✅ (already installed)
19-- `@langchain/textsplitters` - Text splitting utilities ✅ (already installed)
20-
21-Additional requirements:
22-
23-- `pgvector` extension for PostgreSQL (requires DB migration)
24-- `tsx` (dev dependency) - For running TypeScript scripts directly ✅ (already installed)
25-
26-**Note**: `@langchain/community` is NOT needed for this implementation since we're using a custom vector store implementation with pgvector directly, rather than LangChain's built-in vector stores.
27-
28-### 2. Create RAG Module
29-
30-**File**: `src/lib/rag.ts`
31-
32-- Define constants: `EMBEDDING_MODEL`, `CHUNK_COUNT`
33-- Create embeddings function
34-- Create functions to store documents and perform similarity search
35-- Create `retrieveRelevantContext` function with caching
36-- All RAG functionality in one file
37-
38-**Code snippet:**
39-
40-```typescript
41-import { db } from "@/src/lib/db";
42-import { documentChunks } from "@/src/lib/db/schema";
43-import { OllamaEmbeddings } from "@langchain/ollama";
44-import { Document } from "@langchain/core/documents";
45-import { cosineDistance, desc, sql } from "drizzle-orm";
46-import { getCache, setCache } from "@/src/lib/cache";
47-
48-const EMBEDDING_MODEL = "nomic-embed-text";
49-const CHUNK_COUNT = 4;
50-const CACHE_TTL_MS = 60 * 60 * 1000; // 1 hour
51-
52-function createEmbeddings(): OllamaEmbeddings {
53- return new OllamaEmbeddings({
54- model: EMBEDDING_MODEL,
55- });
56-}
57-
58-export async function storeDocuments(
59- documents: Document[],
60- embeddings: OllamaEmbeddings
61-): Promise<void> {
62- // Generate embeddings for all chunks
63- const texts = documents.map((doc) => doc.pageContent);
64- const embeddingsList = await embeddings.embedDocuments(texts);
65-
66- // Store in database (Drizzle handles vector type conversion automatically)
67- for (let i = 0; i < documents.length; i++) {
68- await db.insert(documentChunks).values({
69- content: documents[i].pageContent,
70- embedding: embeddingsList[i],
71- metadata: documents[i].metadata,
72- });
73- }
74-}
75-
76-async function similaritySearch(
77- queryEmbedding: number[],
78- limit: number
79-): Promise<Array<{ content: string; metadata: unknown; similarity: number }>> {
80- const similarity = sql<number>`1 - (${cosineDistance(documentChunks.embedding, queryEmbedding)})`;
81-
82- const results = await db
83- .select({
84- content: documentChunks.content,
85- metadata: documentChunks.metadata,
86- similarity,
87- })
88- .from(documentChunks)
89- .orderBy(desc(similarity))
90- .limit(limit);
91-
92- return results;
93-}
94-
95-export async function retrieveRelevantContext(query: string): Promise<string | null> {
96- // Check cache first
97- const cacheKey = `rag:${query.toLowerCase().trim()}`;
98- const cached = await getCache<string>(cacheKey);
99- if (cached !== null) {
100- return cached;
101- }
102-
103- try {
104- const embeddings = createEmbeddings();
105- const queryEmbedding = await embeddings.embedQuery(query);
106-
107- const results = await similaritySearch(queryEmbedding, CHUNK_COUNT);
108-
109- if (results.length === 0) {
110- return null;
111- }
112-
113- // Format chunks into context string
114- const context = results
115- .map((result, index) => `[Chunk ${index + 1}]\n${result.content}`)
116- .join("\n\n");
117-
118- // Cache the result
119- await setCache(cacheKey, context, CACHE_TTL_MS);
120-
121- return context;
122- } catch (error) {
123- console.error("RAG retrieval error:", error);
124- return null;
125- }
126-}
127-```
128-
129-### 3. Integration into Chat Flow
130-
131-**File**: `src/lib/chat.ts`
132-
133-- Modify `generateResponse` function to:
134- - Extract the latest user message from message history (last message with role "user")
135- - Call `retrieveRelevantContext` with the user message content
136- - If context is retrieved, inject it into the message history:
137- - Find the position after all system messages but before user messages
138- - Insert a system message with formatted RAG context at that position
139- - Format: `**Relevant Context from Knowledge Base:**\n\n[formatted chunks]\n\nUse this information to help answer the user's question.`
140- - Pass enhanced message history to `agentLoop`
141-- Ensure RAG context is only injected when relevant chunks are found
142-- Handle errors gracefully - if RAG retrieval fails, continue without context
143-- Note: RAG context is retrieved fresh for each generation, based on the latest user message
144-
145-**Code snippet for `src/lib/chat.ts` modification:**
146-
147-```typescript
148-import { db } from "@/src/lib/db";
149-import { messages, threads } from "@/src/lib/db/schema";
150-import { createMessage } from "@/src/lib/messages";
151-import { retrieveRelevantContext } from "@/src/lib/rag";
152-import { asc, eq } from "drizzle-orm";
153-
154-// ... existing type definitions and functions ...
155-
156-export async function generateResponse(
157- threadId: number,
158- enqueue: (message: MessageHistoryEntry) => void
159-) {
160- if (!threadId) {
161- throw new Error("Thread ID is required");
162- }
163-
164- // Get the thread:
165- const thread = await db.select().from(threads).where(eq(threads.id, threadId)).limit(1);
166- if (thread.length === 0) {
167- throw new Error("Thread not found");
168- }
169-
170- // Get all of the messages for this thread:
171- const messagesList = (await db
172- .select({
173- role: messages.role,
174- content: messages.content,
175- thinking: messages.thinking,
176- })
177- .from(messages)
178- .where(eq(messages.threadId, threadId))
179- .orderBy(asc(messages.createdAt))) as MessageHistory;
180-
181- if (messagesList.length === 0) {
182- throw new Error("No messages found for this thread");
183- }
184-
185- // Extract latest user message
186- const latestUserMessage = messagesList.filter((msg) => msg.role === "user").pop();
187-
188- // Retrieve RAG context if user message exists
189- let ragContext: string | null = null;
190- if (latestUserMessage) {
191- try {
192- ragContext = await retrieveRelevantContext(latestUserMessage.content);
193- } catch (error) {
194- console.error("Failed to retrieve RAG context:", error);
195- // Continue without RAG context
196- }
197- }
198-
199- // Inject RAG context into message history
200- let enhancedMessageHistory = [...messagesList];
201- if (ragContext) {
202- // Find position after all system messages
203- const lastSystemIndex =
204- enhancedMessageHistory
205- .map((msg, idx) => (msg.role === "system" ? idx : -1))
206- .filter((idx) => idx !== -1)
207- .pop() ?? -1;
208-
209- const ragSystemMessage: MessageHistoryEntry = {
210- role: "system",
211- content: `**Relevant Context from Knowledge Base:**\n\n${ragContext}\n\nUse this information to help answer the user's question.`,
212- };
213-
214- enhancedMessageHistory.splice(lastSystemIndex + 1, 0, ragSystemMessage);
215- }
216-
217- const messageHistory = await agentLoop(thread[0].model, enhancedMessageHistory, enqueue);
218-
219- // Update the thread with the new message history:
220- for (const message of messageHistory) {
221- await createMessage(message.content, message.role, threadId, message.thinking);
222- }
223-}
224-```
225-
226-### 4. Vector Store Initialization
227-
228-#### 4.1 Initialization Strategy
229-
230-- Initialization is done via an npm script, not automatically
231-- Chunking, embedding generation, and vector DB updates happen when the script is run
232-- Run the script manually after updating `data.txt` or as part of deployment
233-
234-#### 4.2 Initialization Script
235-
236-**File**: `scripts/update-vector-data.ts`
237-
238-- Load `data.txt` file directly
239-- Split document into chunks using RecursiveCharacterTextSplitter (define `CHUNK_SIZE` and `CHUNK_OVERLAP` constants)
240-- Generate embeddings for all chunks
241-- Store all chunks with embeddings in vector database
242-- Handle errors gracefully
243-- Optionally clear existing data before re-indexing (add `--force` flag)
244-
245-**Code snippet:**
246-
247-```typescript
248-import { readFile } from "fs/promises";
249-import { join } from "path";
250-import { Document } from "@langchain/core/documents";
251-import { RecursiveCharacterTextSplitter } from "@langchain/textsplitters";
252-import { OllamaEmbeddings } from "@langchain/ollama";
253-import { storeDocuments } from "@/src/lib/rag";
254-import { db } from "@/src/lib/db";
255-import { documentChunks } from "@/src/lib/db/schema";
256-import { count } from "drizzle-orm";
257-
258-const DATA_FILE_PATH = "./data.txt";
259-const CHUNK_SIZE = 1000;
260-const CHUNK_OVERLAP = 200;
261-
262-async function updateVectorData(force: boolean = false): Promise<void> {
263- try {
264- // Check if documents are already indexed
265- const existingCount = await db.select({ count: count() }).from(documentChunks);
266- if (existingCount[0].count > 0 && !force) {
267- console.log("RAG vector store already initialized. Use --force to re-index.");
268- return;
269- }
270-
271- if (force && existingCount[0].count > 0) {
272- console.log("Clearing existing vector data...");
273- await db.delete(documentChunks);
274- }
275-
276- console.log("Loading and chunking data.txt...");
277-
278- // Load data.txt file
279- const filePath = join(process.cwd(), DATA_FILE_PATH);
280- const content = await readFile(filePath, "utf-8");
281- const document = new Document({
282- pageContent: content,
283- metadata: { source: DATA_FILE_PATH },
284- });
285-
286- // Split document into chunks
287- const textSplitter = new RecursiveCharacterTextSplitter({
288- chunkSize: CHUNK_SIZE,
289- chunkOverlap: CHUNK_OVERLAP,
290- });
291- const chunks = await textSplitter.splitDocuments([document]);
292-
293- console.log(`Generated ${chunks.length} chunks, generating embeddings...`);
294-
295- // Generate embeddings for all chunks
296- const embeddings = new OllamaEmbeddings({ model: "nomic-embed-text" });
297- await storeDocuments(chunks, embeddings);
298-
299- console.log("Vector data update complete!");
300- } catch (error) {
301- console.error("Failed to update vector data:", error);
302- process.exit(1);
303- }
304-}
305-
306-// Parse command line arguments
307-const force = process.argv.includes("--force");
308-updateVectorData(force);
309-```
310-
311-**NPM Script Entry:**
312-
313-Add to `package.json`:
314-
315-```json
316-{
317- "scripts": {
318- "rag:update": "tsx scripts/update-vector-data.ts",
319- "rag:update:force": "tsx scripts/update-vector-data.ts --force"
320- }
321-}
322-```
323-
324-**Note**: `tsx` is already installed as a dev dependency, so no additional installation is needed.
325-
326-### 5. Database Schema
327-
328-- Add migration to enable pgvector extension
329-- Create `document_chunks` table with:
330- - `id` (serial)
331- - `content` (text)
332- - `embedding` (vector type) - using pgvector
333- - `metadata` (jsonb) - store chunk index, source file, etc.
334- - `created_at` (timestamp)
335-- Add Drizzle schema definition in `src/lib/db/schema.ts`
336-- Create index on embedding column for similarity search performance
337-
338-**Migration Steps:**
339-
340-1. Create the pgvector extension manually (Drizzle doesn't create extensions automatically):
341- - Use `npx drizzle-kit generate --custom` to create an empty migration file
342- - Add: `CREATE EXTENSION IF NOT EXISTS vector;`
343-
344-2. Generate the table migration:
345- - Run `npx drizzle-kit generate` to create migration for the `document_chunks` table
346- - Drizzle will automatically generate the table and HNSW index based on the schema definition
347-
348-**Example migration SQL (auto-generated by drizzle-kit):**
349-
350-```sql
351-CREATE TABLE IF NOT EXISTS "document_chunks" (
352- "id" serial PRIMARY KEY NOT NULL,
353- "content" text NOT NULL,
354- "embedding" vector(768),
355- "metadata" jsonb,
356- "created_at" timestamp with time zone DEFAULT now() NOT NULL
357-);
358---> statement-breakpoint
359-CREATE INDEX IF NOT EXISTS "embeddingIndex" ON "document_chunks" USING hnsw (embedding vector_cosine_ops);
360-```
361-
362-**Code snippet for `src/lib/db/schema.ts`:**
363-
364-```typescript
365-import { pgTable, serial, text, timestamp, jsonb, vector, index } from "drizzle-orm/pg-core";
366-
367-// Note: Verify that nomic-embed-text produces 768-dimensional embeddings
368-// If using a different embedding model, adjust the dimensions accordingly
369-const EMBEDDING_DIMENSIONS = 768;
370-
371-export const documentChunks = pgTable(
372- "document_chunks",
373- {
374- id: serial("id").primaryKey(),
375- content: text("content").notNull(),
376- embedding: vector("embedding", { dimensions: EMBEDDING_DIMENSIONS }),
377- metadata: jsonb("metadata"),
378- createdAt: timestamp("created_at", { withTimezone: true }).notNull().defaultNow(),
379- },
380- (table) => [index("embeddingIndex").using("hnsw", table.embedding.op("vector_cosine_ops"))]
381-);
382-```
383-
384-**Note**: The pgvector extension must be created manually in a migration. Use `npx drizzle-kit generate --custom` to create an empty migration file and add `CREATE EXTENSION IF NOT EXISTS vector;`
385-
386-### 6. Error Handling
387-
388-- Handle Ollama connection errors
389-- Handle embedding generation failures
390-- Provide fallback behavior when RAG is unavailable
391-
392-## File Structure
393-
394-```
395-src/lib/
396-├── rag.ts # All RAG functionality (embeddings, vector store, retrieval)
397-└── chat.ts # (update to integrate RAG)
398-scripts/
399-└── update-vector-data.ts # Script to update vector database (chunking, embedding, storage)
400-```
401-
402-## Implementation Order
403-
404-1. **Phase 1: Setup**
405- - Install dependencies
406- - Create RAG directory structure
407- - Set up database schema and migrations
408-
409-2. **Phase 2: Core RAG Components**
410- - Create unified RAG module (embeddings, vector store, retrieval)
411- - Create initialization script (chunking, embedding, storage)
412- - Add npm script to run initialization
413-
414-3. **Phase 3: Chat Integration**
415- - Modify `generateResponse` to retrieve context
416- - Inject context into message history
417- - Test integration
418-
419-4. **Phase 4: Optimization**
420- - Error handling
421- - Performance tuning
422-
423-5. **Phase 5: Enhancement (Optional)**
424- - Add file watching for `data.txt` updates
425- - Add re-indexing endpoint/functionality
426- - Add monitoring/logging
427-
428-## Considerations
429-
430-### Performance
431-
432-- Embedding generation happens when running the update script (can be slow)
433-- Vector store is populated manually via npm script, not automatically
434-- Cache retrieval results by query to avoid redundant searches
435-- Script checks if vector store is already populated to skip re-indexing (unless `--force` flag is used)
436-
437-### Context Injection Strategy
438-
439-- Inject as additional system message with clear formatting
440- - Format: "**Relevant Context from Knowledge Base:**\n[chunks]"
441- - Pros: Clear separation, easy to identify in logs
442- - Cons: May be treated differently by model
443-
444-### Scalability
445-
446-- pgvector with PostgreSQL handles larger datasets well
447-- Consider chunking strategy for very large files
448-- Monitor query performance and adjust `CHUNK_COUNT` constant as needed
449-
450-### Model Selection
451-
452-- Use an embedding model compatible with Ollama
453-- `nomic-embed-text` is a good default choice
454-
455-### Data Updates
456-
457-- `data.txt` is loaded and indexed via the `npm run rag:update` script
458-- To update the knowledge base: modify `data.txt`, then run `npm run rag:update:force` to re-index
459-- The script can be run manually or as part of deployment/CI pipeline