RAGging

---

RAGging is a local-first document intelligence application for asking questions about PDF files. It combines a Streamlit chat interface, a FastAPI backend, local sentence-transformer embeddings, persistent ChromaDB search and streamed language-model responses from Groq.

The central product decision is traceability. Instead of returning an answer that merely sounds plausible, RAGging keeps the supporting evidence close: every response includes page-level citations and the exact retrieved passages used to generate it.

The starting page keeps document upload and question entry in one focused interface.

The problem

PDFs are convenient to distribute but awkward to explore. Keyword search works when the reader already knows the document's terminology. It is less useful for conceptual questions, summaries or answers assembled from several passages.

Sending an entire document to a language model for every question is slow, expensive and difficult to audit. RAGging uses retrieval-augmented generation to narrow each request to a small set of relevant excerpts before asking the model to respond.

The interaction stays simple: upload one or more text-based PDFs, select which documents to search, ask a question, then inspect the streamed answer and its sources.

System design

The frontend and API are separate so the interface, retrieval pipeline and model provider can evolve independently. Streamlit manages uploads, document selection and conversation state. FastAPI exposes endpoints for service health, document ingestion and streamed chat.

At indexing time, the backend extracts text with pypdf, splits it into overlapping chunks, attaches filenames and page numbers, creates local MiniLM embeddings and stores them in ChromaDB. At question time, it embeds the query, retrieves the three nearest passages from the selected documents and sends a grounded prompt to Groq's openai/gpt-oss-20b model.

Document embeddings stay local. Only the question, recent conversation history and retrieved excerpts are sent to the generation provider, reducing—but not eliminating—the amount of document data shared externally.

Defensive ingestion

The upload path rejects bad input before it reaches the index. It enforces a 20 MB file limit, a 300-page limit and a maximum of 5,000 chunks. It verifies the PDF signature, rejects encrypted or damaged files and reports documents without extractable text. Scanned PDFs therefore need OCR before upload.

Each file receives a SHA-256 identifier derived from its contents. Uploading the same bytes reconnects the existing document instead of creating duplicate vectors. Chunks are written in batches, and failed ingestion removes partial records rather than leaving the index in an ambiguous state.

Embedding inference and Chroma operations are synchronous, CPU-bound tasks. The API moves this work to a worker thread so it does not block FastAPI's event loop, while a lock serialises access to the embedded single-process index.

Retrieval and streaming

Search is filtered by the document identifiers selected in the interface. This prevents passages from unrelated indexed files from entering the answer context. The model receives numbered excerpts with filenames and page numbers and is instructed to answer only from that evidence, cite it as [1], [2] and so on, and say when the material does not contain an answer.

The same instruction treats commands found inside uploaded documents as untrusted content. This provides a first layer of prompt-injection resistance, although it is not a complete security boundary.

Answers stream to the browser through Server-Sent Events. A sources event arrives before generation, token events carry partial text, done marks successful completion and error reports failures after the stream has started. The frontend treats a missing completion event as an interrupted response instead of silently presenting partial output as finished.

Evidence, not confidence

The source inspector shows each retrieved passage, filename, page number and cosine distance. Lower distance means a closer vector match; the interface deliberately does not present it as a confidence score.

Citations are also not described as automatically verified. The prompt encourages grounded answers, but the user can compare every claim with the underlying excerpt. When relevant evidence is absent, the intended behaviour is to say so rather than invent an answer.

Reliability

The automated tests target the boundaries most likely to fail: page-aware extraction, corrupt and oversized PDFs, schema validation, interrupted streams, missing model configuration, empty retrieval results, Chroma persistence, deduplication, document filtering and interface empty states.

Provider calls are replaced by a deterministic service during tests, avoiding API costs and network dependencies. Storage tests use a deterministic encoder while exercising a real Chroma database. The final suite passed all eight automated tests, Ruff reported no lint errors, both services passed health checks and a live smoke test uploaded a PDF, retrieved its passage and streamed a cited response through Groq.

Boundaries

RAGging is intentionally a single-user local MVP. Text extraction works best on PDFs with a conventional text layer; scans require OCR, while complex tables and multi-column layouts may lose structure. Retrieval uses a fixed top-three semantic search without reranking or hybrid keyword search. Follow-up context reaches the generator, but retrieval embeds only the latest question, so self-contained questions remain more reliable.

Document identifiers scope search but are not authorisation credentials. A public, multi-user deployment would need authentication, per-user document permissions, deletion workflows, managed storage, rate limits, audit logging and stronger adversarial testing.

RAGging demonstrates how an AI feature becomes a complete software system: defensive document handling, local embedding inference, persistent vector search, asynchronous APIs, explicit streaming semantics, source-aware interface design, automated tests and honest operational limits. The model makes a document easier to explore; the application keeps the evidence visible enough for the user to judge the result.