Build a RAG app

Use RAG when an agent needs to answer from your documents instead of only from the model’s training data. Start from the docs-agent template, then customize the retrieval hook and document sources.

Terminal

veryfront init my-rag --template docs-agent
cd my-rag
npx veryfront@latest dev

How RAG works

Veryfront splits a RAG app into three flows:

Ingestion: Upload or read documents, extract text, split text into chunks, and store those chunks.
Search: Embed the user’s query and compare it with stored chunk embeddings.
Generation: Add the best matching chunks to the AG-UI request before the agent responds.

The docs-agent template wires these flows with ragStore(), createUploadHandler(), useUploadsRegistry(), and createAgUiHandler().

Create the store

In your project root, create a shared RAG store:

store.ts

import { ragStore } from "veryfront/embedding";

export const store = ragStore({
  storagePath: "data/index.json",
  contentDir: "content",
});

In local development, ragStore() stores chunks and vectors in data/index.json. When Veryfront Cloud bootstrap is present, it uses the Veryfront Cloud RAG backend automatically. Set contentDir: "knowledge" to index knowledge/ instead of content/.

Add upload routes

Create upload routes that share the same store:

lib/upload-auth.ts

import { authorizeSession } from "./session.ts";

// Verify your app's signed, same-origin session cookie. The browser sends that
// cookie for useUploadsRegistry requests without exposing a bearer token to
// client code.
export const authorizeUploads = (request: Request) => authorizeSession(request);

app/api/uploads/route.ts

import { createUploadHandler } from "veryfront/embedding";
import { authorizeUploads } from "../../../lib/upload-auth.ts";
import { store } from "../../../store.ts";

export const { POST, GET, DELETE } = createUploadHandler(store, {
  auth: { authorize: authorizeUploads },
});

POST ingests a file, GET lists ingested documents, and DELETE removes a document. For local-only prototypes, pass auth: { type: "none", allowUnauthenticated: true } to explicitly allow unauthenticated upload routes. The auth policy is required and fails closed. An authorize callback must return the literal value true to permit a request; false or an accidental missing return denies it with 401. Return a Response when the route needs a custom denial or authentication-challenge response. Calling createUploadHandler(store) without an explicit auth policy now fails when the handler is created. The handler bounds each source file to 10 MiB, the complete multipart body to the file limit plus 64 KiB, and extracted UTF-8 text to 5 MiB by default. Set maxFileSize, maxBodySize, or maxExtractedTextBytes only when your deployment has a different, still-bounded budget. Invalid configuration fails when the handler is created. Request cancellation is propagated through multipart reading, extraction, and RAG ingestion. GET /api/uploads?limit=25&offset=0 returns only upload-origin documents, newest first:

{
  "items": [],
  "total": 0,
  "offset": 0,
  "limit": 25,
  "hasMore": false
}

The default and maximum page sizes are controlled by maxListItems (100 by default, at most 1,000). useUploadsRegistry() follows hasMore pages and publishes the new server snapshot only after every bounded page validates. DELETE accepts an id route parameter or query parameter and refuses to delete documents that did not originate from the upload route. In Cloud mode, a 502 means the RAG entry was removed but source blob cleanup is incomplete; retry the same deletion safely.

Understand ingestion

Upload ingestion does three things:

Extracts text: Text, Markdown, and MDX are read directly. CSV files are converted into row text with headers. PDF, DOCX, XLS, XLSX, PPTX, HTML, RTF, EPUB, JSON, and XML use the DocumentExtractor extension backed by @veryfront/ext-document-kreuzberg.
Chunks and embeds: Text is split with chunkOptions before embedding. Defaults are maxChars: 2000, overlap: 200, and separators: ["\n\n", "\n", " ", ""].
Stores data: Cloud mode stores the original uploaded file as a source file blob under .veryfront/rag/uploads/. Local mode stores only the RAG index in data/index.json.

OCR is not a separate step. For scanned PDFs or image-only files, run OCR before calling store.ingest(). Local mode fills embeddings on first search. Cloud mode chunks and embeds during ingestion. When extraction behavior changes or an uploaded document needs reprocessing, call store.refreshDocument(id, text, meta) to keep the document ID while replacing its chunks and embeddings.

Add bundled content ingestion

Use a separate ingestion route for files that ship with the app:

app/api/ingest/route.ts

import { store } from "../../../store.ts";

export async function POST() {
  await store.indexContentDir();
  return Response.json({ ok: true });
}

Run this route after files in content/ change:

Terminal

curl -X POST http://localhost:3000/api/ingest

Keep indexing out of the chat request path. indexContentDir() reads files from contentDir and skips files that are already tracked by source. Uploaded documents do not need this call because the upload route ingests them directly.

Add retrieval to the agent route

Use beforeStream to retrieve context before the agent runs:

app/api/ag-ui/route.ts

import { createAgUiHandler } from "veryfront/agent";
import { store } from "../../../store.ts";

export const POST = createAgUiHandler("rag", {
  beforeStream: async ({ lastUserText }) => {
    const query = lastUserText.trim();
    if (!query) return;

    const results = await store.search(query, { topK: 5 });
    if (results.length === 0) return;

    const contextBlock = results
      .map((result) => `[${result.title}] (score: ${result.score.toFixed(2)})\n${result.text}`)
      .join("\n\n---\n\n");

    return {
      prepend: [
        {
          role: "system",
          parts: [
            {
              type: "text",
              text:
                `Here are relevant documents retrieved for your question:\n\n${contextBlock}\n\n` +
                "Use these documents to answer. Cite the document title when referencing information.",
            },
          ],
        },
      ],
    };
  },
});

Veryfront wraps retrieved context before it reaches the model. Treat retrieved documents as reference data, not instructions. Studio Knowledge Q&A agents use native source citations. When you build inside Studio, keep citation display enabled so answers point back to the retrieved documents that supported them.

Add the chat UI

Use the app-mode Chat component with useUploadsRegistry() and AttachmentsPanel:

app/page.tsx

"use client";

import { AttachmentsPanel, Chat, useUploadsRegistry } from "veryfront/chat";

export default function RagPage() {
  const uploads = useUploadsRegistry({ url: "/api/uploads" });
  const uploadError = uploads.uploadError ??
    uploads.refreshError ??
    uploads.removeError ??
    uploads.storageError;

  return (
    <main className="flex min-h-screen">
      <Chat
        agentId="rag"
        api="/api/ag-ui"
        placeholder="Ask about your documents..."
      />

      <aside>
        {uploadError && <p role="alert">{uploadError.message}</p>}
        <AttachmentsPanel
          uploads={uploads.items}
          loading={uploads.isLoading}
          onAttach={uploads.upload}
          onRemoveUpload={uploads.remove}
        />
      </aside>
    </main>
  );
}

The docs-agent template includes a fuller upload panel. Use this smaller example when you want the minimum wiring. Upload documents through the panel: the RAG ingestion route returns document metadata, not the runtime-fetchable url required by Chat.uploadApi. Composer attachments use a separate createChatUploadHandler route when you want them stored durably.

Use Veryfront Cloud mode

Set Veryfront Cloud bootstrap variables before starting the app:

Terminal

export VERYFRONT_API_TOKEN=<TOKEN>
export VERYFRONT_PROJECT_SLUG=<PROJECT_SLUG>
npx veryfront@latest dev

With cloud bootstrap:

ragStore() uses the Veryfront Cloud RAG backend.
Generation uses Veryfront Cloud model routing.
Embeddings use Veryfront Cloud embedding routing.
veryfront-cloud/openai/... and veryfront-cloud/google/... models route through AI Gateway.

The default cloud embedding model is veryfront-cloud/openai/text-embedding-3-small. Set VERYFRONT_DEFAULT_EMBEDDING_MODEL to provider/model, such as google/text-embedding-004; Cloud bootstrap routes it as veryfront-cloud/google/text-embedding-004:

Terminal

export VERYFRONT_DEFAULT_EMBEDDING_MODEL=google/text-embedding-004

Use raw Cloud APIs

Use raw Cloud APIs when you are building outside a Veryfront app or need direct control over indexing. The manual flow is:

Create or list RAG document records.
Split source content into chunks.
Generate vectors through AI Gateway or another embedding provider.
Store vectors with the embeddings endpoint.
Search with a query vector.

For Veryfront apps, prefer ragStore() unless you need that lower-level control.

Verify it worked

Run veryfront dev, open the app, and check these behaviors:

Upload a document. The upload route returns success and the document appears in the upload list.
Index bundled files with /api/ingest after changing files in content/.
Ask a question that depends on the document. The response cites the document title.
Ask an unrelated question. The response says the retrieved context does not contain a clear answer.
In cloud mode, confirm API requests use VERYFRONT_API_TOKEN and the target project slug.

If retrieval returns no results, check that the uploaded file has extractable text and that the embedding provider is configured.

Build a chat UI: Add or customize the chat surface
Providers: Configure model and embedding routing
CLI knowledge ingestion: Turn files into project knowledge

veryfront/embedding: RAG store and upload helpers
veryfront/agent: AG-UI route helpers
veryfront/chat: Chat components and hooks

Getting Started

Concepts

Guides

API Reference

How RAG works

Create the store

Add upload routes

Understand ingestion

Add bundled content ingestion

Add retrieval to the agent route

Add the chat UI

Use Veryfront Cloud mode

Use raw Cloud APIs

Verify it worked

Next

​How RAG works

​Create the store

​Add upload routes

​Understand ingestion

​Add bundled content ingestion

​Add retrieval to the agent route

​Add the chat UI

​Use Veryfront Cloud mode

​Use raw Cloud APIs

​Verify it worked

​Next

​Related

How RAG works

Create the store

Add upload routes

Understand ingestion

Add bundled content ingestion

Add retrieval to the agent route

Add the chat UI

Use Veryfront Cloud mode

Use raw Cloud APIs

Verify it worked

Next

Related