The value of curated data

Know which sources exist

Classify owners, formats, versions and access. Identify conflicting, obsolete or ownerless documents before indexing them.

Preserve context and permissions

An answer should link to a verifiable version and section. Ingestion must respect source access rules and allow removal or update.

Test where it fails

Build representative questions, including ones with no answer. Review citations, abstention, expired documents and differences between roles.

The advantage is not zero errors: it is the ability to find, explain and correct them.

An inventory that supports a decision

For each document collection, record owner, purpose, review date, format, authorised groups and removal process. If two policies conflict, do not ask the model to settle the contradiction: assign a person to decide which is current. Source review may improve the system more than a model change.

In evaluation, separate questions requiring an exact fact from ones requiring synthesis. Record the expected source for both. This lets you locate the problem in the document, retrieval, permission rule or generated answer.

Turn an archive into a governed source

Before ingestion, choose a bounded collection and map each item to a source system, owner, version, review date, sensitivity and permitted audience. Keep a stable identifier so a correction or withdrawal in the source can propagate to the index. Decide which copy is authoritative when an exported file, a web page and a PDF disagree. A model cannot determine organisational policy from contradictory documents on its own.

For an illustrative policy collection, mark one obsolete procedure as withdrawn and one newer procedure as current. A search result must identify the current version and its section. If a retired version still appears, fix the source or index lifecycle before tuning the model prompt. This is a synthetic example, not a measured deployment.

Test the whole retrieval path

Check extraction, chunk boundaries, metadata, access filtering, ranking and answer composition separately. A correct-looking answer may cite the wrong version; a good retriever may still pass an unauthorised passage to generation. Prepare cases for a precise fact, a question needing two sources, an unsupported question and a conflict between documents. For each, record expected source, permitted role, result and failure category.

Run the same question as two roles whose access differs. Compare retrieved passages before comparing final answers, and include cache, logs and exports in the boundary review. A citation is useful only if the cited passage supports the claim and the reader may open it. A confident answer without that evidence should be treated as a test failure, not hidden behind an overall score.

Keep updates and removal observable

Agree who approves a source update, how often the index is refreshed, how deletion is confirmed and what happens while a source is temporarily unavailable. Log versions and evaluation changes so that a quality regression can be traced to source, retrieval or generation changes. Do not promise that curation eliminates hallucination; it makes failures more diagnosable and gives a team a way to correct them.

For an implementation decision, see private RAG and the local-first RAG guide. The useful question is which information is authorised, current and answerable, not how many documents can be uploaded.