Field note 02
When a business actually needs RAG.
Retrieval is valuable when the hard part is finding the right internal truth—not when the real problem is ownership, process or source quality.
The position
RAG is not a magic layer for messy knowledge. It is an evidence pipeline. It works when the sources are governable, the questions repeat and a grounded answer is more valuable than a plausible one.
Begin with the decision the answer supports.
A useful RAG project starts with a repeated business question: Which policy applies? What did this customer agree to? Which product configuration fits this constraint? The system has a job, an audience and a consequence. 'Chat with all our documents' has none of those boundaries.
Name the decision, frequency, current cost and acceptable failure mode. If the answer only saves a few seconds and mistakes are expensive, conventional navigation may be better. If specialists repeatedly reconstruct context across controlled sources, retrieval can create real leverage.
- Repeated, costly questions
- Known sources of truth
- A review path for consequential answers
Readiness lives upstream of the model.
Most failures begin before generation. Documents are stale, duplicates disagree, permissions are implicit and no one owns the answer. A more capable model cannot decide which obsolete policy the company still honors.
Inventory sources before choosing infrastructure. Record owner, audience, freshness, sensitivity and authority. Decide what should never be retrieved together. Clean a small representative set and test the hardest questions against it. This exposes the operating work hidden inside the AI brief.
Evaluate retrieval and generation separately.
When an answer fails, teams need to know whether the right evidence was missing, retrieved but ignored, or interpreted incorrectly. A single thumbs-up score cannot tell you. Build a small evaluation set with expected sources, required facts and unacceptable claims.
Measure retrieval coverage, citation support, answer completeness, refusal behavior, latency and human correction. Review failures by question type. The goal is not a perfect demo—it is a system whose limits are observable enough to improve.
Design the human boundary.
A trustworthy interface shows where an answer came from, makes source context inspectable and creates an obvious escalation path. Confidence should be earned through evidence, not simulated with polished prose.
The best result may be an assistant, a search layer, a drafting tool or a decision packet. Choose the form that fits the work. RAG is one component inside that operating experience, not the product by itself.
Direct answers
What people ask next.
01What is the simplest sign that we need RAG?
People repeatedly search across a controlled body of changing internal knowledge and need answers grounded in those sources. If the knowledge is tiny, static or already well structured, simpler search may be enough.
02Do we need a vector database first?
No. Start with the questions, sources, permissions and evaluation set. Infrastructure follows the evidence and scale requirements.
03Can RAG eliminate hallucinations?
No. It can improve grounding and make evidence visible, but retrieval and generation can still fail. Consequential workflows need evaluation, citations, permissions and human review boundaries.
Put it to work