Giving Your Agents Memory: State, History, and Vector Databases

So far, our agent is intelligent but forgetful. It treats every request as if it's the first time it has ever heard from you. It has no memory of past interactions and no knowledge of the world beyond what's in its training data. To build truly useful agents, we need to give them a memory.

In this chapter, we'll explore two fundamental types of memory you can build for your agents using the Cloudflare ecosystem:

1. Short-Term Memory: The ability to remember the recent history of a conversation. We'll use Cloudflare KV for this.
2. Long-Term Memory: The ability to retrieve knowledge from a large set of documents. We'll use Cloudflare Vectorize and a technique called Retrieval-Augmented Generation (RAG).

By the end, you'll be able to build agents that are not only conversational but also deeply knowledgeable.

Short-Term Memory with Cloudflare KV

Conversational context is what separates a chatbot from a true assistant. If you have to repeat yourself in every message, the illusion of intelligence shatters. The agent needs to remember what you just said.

Cloudflare KV is a global, low-latency, key-value data store. It's perfect for storing small pieces of data, like a conversation history, that need to be accessed quickly from anywhere in the world.

Let's upgrade our Research Agent to be conversational. The plan is simple:

1. The user will include a sessionId in their request.
2. We'll use this sessionId as the key in our KV store.
3. For each request, we'll read the history, add the new user message and the AI's response, and write the updated history back to KV.

Step 1: Configure the KV Binding

First, you need to create a KV namespace. You can do this on the Cloudflare dashboard or via the command line:

```bash

wrangler kv:namespace create "CONVERSATION_HISTORY"

```

This command will output an id. You then add this binding to your wrangler.toml file:

```toml

In wrangler.toml

[[kv_namespaces]]

binding = "CONVERSATION_HISTORY"

id = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" # The ID from the command above

```

Now, a CONVERSATION_HISTORY object will be available on our c.env object in the worker.

Step 2: Update the Agent Logic

Let's modify our src/index.ts to manage the chat history.

```typescript

// src/index.ts

// ... (imports are the same, but add z.string() for sessionId)

const ConversationalResearchRequestSchema = z.object({

topic: z.string().min(1).max(100),

sessionId: z.string().uuid(), // Expect a unique ID for the session

});

// ... Hono setup ...

export type Bindings = {

AI: any;

CONVERSATION_HISTORY: KVNamespace; // Add the KV binding type

};

app.post(

'/conversational-research',

zValidator('json', ConversationalResearchRequestSchema),

async (c) => {

const { topic, sessionId } = c.req.valid('json');

// 1. Get existing chat history from KV

const history = await c.env.CONVERSATION_HISTORY.get(sessionId, { type: 'json' }) || [];

// 2. Add the new user message to the history

history.push({ role: 'user', content: topic });

const template = `

You are a research assistant. You are having a conversation with a user.

Use the provided conversation history to maintain context.

Conversation History:

{history}

User's Latest Message: {topic}

Provide your response as a JSON object... // (rest of the prompt is the same)

`;

// ... (Instantiate prompt, model, chain)

// 3. Run the chain with history and the latest topic

const result = await chain.invoke({

history: JSON.stringify(history, null, 2),

topic: topic,

});

// ... (Validate the LLM output)

const llmResponse = validation.data;

// 4. Add the AI's response to the history

history.push({ role: 'assistant', content: JSON.stringify(llmResponse) });

// 5. Save the updated history back to KV

await c.env.CONVERSATION_HISTORY.put(sessionId, JSON.stringify(history));

return c.json(llmResponse);

}

);

```

Now our agent can hold a conversation! It uses the KV store as its "working memory," allowing it to reference previous messages to provide more contextual and intelligent responses.

Long-Term Memory with Cloudflare Vectorize (RAG)

What if you want your agent to answer questions about a specific set of documents, like your company's internal wiki or a technical manual? You can't fit hundreds of pages into a prompt. This is where Retrieval-Augmented Generation (RAG) comes in.

RAG is a powerful technique that allows an LLM to access knowledge from an external source. Here's how it works at a high level:

1. Indexing (Offline): You take your documents, split them into small chunks, and use an embedding model to convert each chunk into a vector (a list of numbers representing its semantic meaning). You store these vectors in a specialized vector database.
2. Retrieval (Real-time): When a user asks a question, you convert their question into a vector using the same embedding model.
3. Search: You query your vector database to find the document chunks whose vectors are most similar to your question's vector.
4. Augmentation: You take these relevant chunks and "augment" your prompt with them, providing them as context to the LLM.
5. Generation: You ask the LLM to answer the question based only on the provided context.

This approach allows the LLM to answer questions about information it was never trained on, effectively giving it a long-term memory or a library to reference.

Cloudflare Vectorize is a globally distributed vector database built right into the Workers ecosystem, making it incredibly easy to build RAG applications.

A Conceptual RAG Agent

Building a full RAG pipeline involves an indexing step (which we won't cover in detail here), but let's look at the code for the query step within a worker.

First, you'd create a Vectorize index and bind it in wrangler.toml:

```toml

In wrangler.toml

[[vectorize]]

binding = "VECTOR_DB"

index_name = "my-knowledge-base"

```

Then, your agent's code would look something like this:

```typescript

// Conceptual RAG endpoint

app.post('/ask', zValidator('json', QuestionSchema), async (c) => {

const { question, sessionId } = c.req.valid('json');

// 1. Create an embedding of the user's question

const embeddingResponse = await c.env.AI.run('@cf/baai/bge-base-en-v1.5', {

text: [question]

});

const questionVector = embeddingResponse.data[0];

// 2. Query Vectorize to find the most relevant document chunks

const similarChunks = await c.env.VECTOR_DB.query(questionVector, { topK: 3 });

const context = similarChunks.matches.map(match => match.metadata.text).join('\n\n');

// 3. Create a RAG prompt

const ragTemplate = `

You are a helpful assistant. Answer the following question based ONLY on the provided context.

If the answer is not in the context, say "I do not have enough information to answer that."

Context:

---

{context}

---

Question: {question}

`;

const prompt = new PromptTemplate({ template: ragTemplate, inputVariables: ['context', 'question'] });

// ... (Instantiate model, create chain)

// 4. Run the chain with the retrieved context

const response = await chain.invoke({ context, question });

return c.json({ answer: response });

});

```

This pattern is incredibly powerful. You've effectively given your agent access to a library of knowledge that you can control and update, allowing it to become an expert in any domain you choose.

The Two Brains of an Agent

By combining these two forms of memory, you can create truly sophisticated agents.

• Cloudflare KV acts as the agent's short-term, working memory. It's fast, cheap, and perfect for remembering the flow of a conversation.
• Cloudflare Vectorize acts as the agent's long-term, reference memory. It's the library the agent goes to when it needs to look up a specific fact or piece of knowledge.

With a memory in place, our agent is no longer just a clever tool but a capable assistant, ready to learn and engage in meaningful interaction.