Explain retrieval vs. generation

Retrieval-augmented generation supplies selected external information to a model while it answers a request. For example, a support assistant can search a company's current policy documents and include relevant passages in the model's context.

The retrieval component selects evidence. The generation component produces an answer using its instructions and available context. Either component can fail independently. Relevant evidence can be missing, or a generator can misinterpret evidence that was correctly retrieved.

Adding a document to a retrieval index normally does not update the language model's weights. It changes the material available to the application at inference time. This makes document updates possible without retraining the generator, provided the ingestion pipeline actually processes those updates.

Ingestion often runs asynchronously. A successful upload does not necessarily mean the document is searchable. The application should distinguish accepted, processing, failed, and published states.

At query time, the current user and access scope influence which documents are eligible. The model should receive only evidence the application is authorized to use for that request.

Parsing extracts usable content from formats such as HTML, Markdown, PDFs, or office documents. Some PDFs contain selectable text, while scanned pages may require OCR. Extraction quality matters because missing rows or reordered text can change meaning before retrieval begins.

Normalization can remove repeated navigation and headers or standardize whitespace. It should preserve meaningful structure such as headings, table labels, units, and exceptions. Keep a reference to the original source so a generated claim can be checked against it.

For example, extracting a pricing table without its column headers can turn an accurate source into ambiguous text. The retrieval system cannot reliably recover structure that the parser discarded.

A chunk is a searchable portion of a document. Splitting long sources can make retrieval more precise and keep model inputs within limits. Useful boundaries often follow sections, paragraphs, tables, or code blocks.

Very small chunks may lose the subject or exception that gives a sentence meaning. Very large chunks can mix several topics and consume context capacity. Overlap can preserve continuity across boundaries, but increases duplicate content and indexing work.

There is no universal chunk size that works best for every corpus. Compare strategies using the questions the application should answer. A policy exception should remain connected to the rule it qualifies, or be recoverable through linked neighboring content.

Provenance describes where a piece of information came from and how it was derived. A chunk should resolve to a particular source revision, not merely a title that may later refer to different text.

Metadata errors can make a correct passage invisible or expose it to the wrong audience. Metadata is part of the pipeline's correctness, not decoration for the UI.

After retrieval, the application selects passages for the prompt. It may deduplicate overlapping text, include neighboring context, and assign short source labels such as S1 and S2. It must reserve room for instructions, the question, and the response under the model's accounting rules.

The highest-ranked passage alone may not answer every part of a question. A refund question may require both a standard window and an exception clause. Context selection should preserve the facts needed to answer the whole request.

Retrieved text is source data. A document containing “ignore all previous instructions” does not gain authority to change application policy. Security & Privacy covers these attacks in greater depth.

A grounded answer is supported by the evidence used for the task. Citations help readers locate that evidence. A valid-looking citation does not establish that the cited passage supports the claim.

Check three separate properties. Does the citation identify a source actually supplied to the model? Does the source resolve to the right revision and location? Does its content support the associated claim?

The application should build citation links from trusted source mappings rather than trust arbitrary model-generated URLs. Citation access should obey the same permission rules as the source itself.

A source may change after it is indexed. The system needs a way to associate new chunks with the new revision and stop using obsolete chunks for current-policy questions.

Deleting a source also affects derived chunks, embeddings, caches, and citation links. Hiding the upload from a list does not necessarily remove its content from retrieval. The Hard guide develops a lifecycle that handles these relationships explicitly.

If a response is wrong, first inspect whether the necessary evidence was available to the generator. If S2 was never parsed or retrieved, improve the data or retrieval path. If S2 was present but ignored, investigate context construction and generation behavior.

A useful small evaluation set includes directly answerable questions, questions requiring multiple passages, conflicting versions, and questions whose answers are absent. Measure evidence availability and answer support separately.