“Can I carry over unused vacation days into next year?”
A 20-minute beginner guide
RAG: Giving LLMs the Right Context
How a company handbook assistant finds evidence before it answers.
The running example
One ordinary employee question
v4.2 · 186 pages
2026 edition
effective Jan 2026
The knowledge gap
The model cannot know what it was never given
Your handbook is private, not part of public training data.
Policies change after a model’s training cutoff.
A fluent model may still guess when evidence is missing.
The core move
Retrieve first. Generate second.
RAG adds an evidence-gathering step before the LLM writes.
Employee question
What policy applies to me?
Find relevant text
Search the handbook for likely evidence.
Answer with context
Use the evidence; cite the source.
End-to-end
The full RAG pipeline
1. Question
Capture the user’s need and metadata.
2. Retrieve
Rank handbook chunks by relevance.
3. Augment
Place selected evidence in the prompt.
4. Generate
Ask the LLM to answer from context.
5. Cite
Return the answer with traceable sources.
RAG is a system around the model — not a different kind of model.
05 / 12Before anyone asks
Prepare the handbook in useful pieces
Vacation allowance and annual reset.
Carryover rules and local exceptions.
Manager approval and deadlines.
Leaving the company and payout.
Keep each chunk coherent; preserve its title, section, version, and location metadata.
06 / 12Meaning, made searchable
Embeddings turn text into coordinates
Text with similar meaning lands near other similar text — even when the exact words differ.
“Unused leave may be carried into Q1…”
[ 0.12, −0.48, 0.81, … ]“carry over vacation”
close in meaning
“unused leave next year”
also close
“laptop reimbursement”
far away
At question time
Retrieve the top-k closest chunks
Can I carry over unused vacation?
[ 0.09, −0.43,
0.77, … ]k = 3
filter: country = DE
Germany addendum · carryover exception
0.92Time off · unused leave policy
0.87Time off · manager approval deadline
0.78Benefits · sabbatical eligibility
0.41Expenses · home office equipment
0.13“Top-k” simply means: take the best k results. More is not always better.
08 / 12Augment
Build a prompt with rules, evidence, and the question
Generate + cite
A useful answer stays close to the evidence
Yes. In Germany, you may carry over up to five unused vacation days into the first quarter of the next year Germany §4.2.
You need your manager’s approval by December 15 Time Off §3.1. If you have more than five days, contact People Ops for guidance.
Answers the employee’s actual question.
Uses the Germany policy, not a global guess.
Every key claim points back to evidence.
Watch the seams
RAG can still fail
Bad chunks
A rule is split away from its exception.
FIX → chunk by meaning; keep overlap.Wrong retrieval
The right policy never reaches the prompt.
FIX → test queries, filters, and ranking.Stale sources
An old handbook version outranks the current one.
FIX → version, expire, and re-index.Unsupported answer
The model adds a plausible detail not in context.
FIX → require citations and abstention.Takeaway