← Portfolio

A 20-minute beginner guide

RAG: Giving LLMs the Right Context

How a company handbook assistant finds evidence before it answers.

01 / 12

The running example

One ordinary employee question

Employee · Berlin office

“Can I carry over unused vacation days into next year?”

channel: handbook assistant•needed: policy + location
GLOBAL BENEFITS
v4.2 · 186 pages
TIME OFF POLICY
2026 edition
GERMANY ADDENDUM
effective Jan 2026
02 / 12

The knowledge gap

The model cannot know what it was never given

01

Your handbook is private, not part of public training data.

02

Policies change after a model’s training cutoff.

03

A fluent model may still guess when evidence is missing.

LLM NO ACCESS HANDBOOK
03 / 12

The core move

Retrieve first. Generate second.

RAG adds an evidence-gathering step before the LLM writes.

INPUT / 01

Employee question

What policy applies to me?

RETRIEVAL / 02

Find relevant text

Search the handbook for likely evidence.

GENERATION / 03

Answer with context

Use the evidence; cite the source.

04 / 12

End-to-end

The full RAG pipeline

1. Question

Capture the user’s need and metadata.

2. Retrieve

Rank handbook chunks by relevance.

3. Augment

Place selected evidence in the prompt.

4. Generate

Ask the LLM to answer from context.

5. Cite

Return the answer with traceable sources.

RAG is a system around the model — not a different kind of model.

05 / 12

Before anyone asks

Prepare the handbook in useful pieces

Source document
CHUNK 014

Vacation allowance and annual reset.

CHUNK 015

Carryover rules and local exceptions.

CHUNK 016

Manager approval and deadlines.

CHUNK 017

Leaving the company and payout.

Keep each chunk coherent; preserve its title, section, version, and location metadata.

06 / 12

Meaning, made searchable

Embeddings turn text into coordinates

Text with similar meaning lands near other similar text — even when the exact words differ.

Chunk 015

“Unused leave may be carried into Q1…”

[ 0.12, −0.48, 0.81, … ]

“carry over vacation”
close in meaning

“unused leave next year”
also close

“laptop reimbursement”
far away

07 / 12

At question time

Retrieve the top-k closest chunks

Query embedding

Can I carry over unused vacation?

[ 0.09, −0.43,
0.77, … ]

k = 3
filter: country = DE

#1

Germany addendum · carryover exception

0.92
#2

Time off · unused leave policy

0.87
#3

Time off · manager approval deadline

0.78
#4

Benefits · sabbatical eligibility

0.41
#5

Expenses · home office equipment

0.13

“Top-k” simply means: take the best k results. More is not always better.

08 / 12

Augment

Build a prompt with rules, evidence, and the question

Instruction
Answer only from the provided context. If the context is insufficient, say so.
Context / 1
[Germany Addendum §4.2] Employees may carry up to five unused days into Q1…
Context / 2
[Time Off Policy §3.1] Carryover requires manager approval by December 15…
Question
Can I carry over unused vacation days into next year?
Output
Give a concise answer and cite the relevant section(s).
09 / 12

Generate + cite

A useful answer stays close to the evidence

Handbook assistant

Yes. In Germany, you may carry over up to five unused vacation days into the first quarter of the next year Germany §4.2.

You need your manager’s approval by December 15 Time Off §3.1. If you have more than five days, contact People Ops for guidance.

Specific

Answers the employee’s actual question.

Scoped

Uses the Germany policy, not a global guess.

Traceable

Every key claim points back to evidence.

10 / 12

Watch the seams

RAG can still fail

FAIL / 01

Bad chunks

A rule is split away from its exception.

FIX → chunk by meaning; keep overlap.
FAIL / 02

Wrong retrieval

The right policy never reaches the prompt.

FIX → test queries, filters, and ranking.
FAIL / 03

Stale sources

An old handbook version outranks the current one.

FIX → version, expire, and re-index.
FAIL / 04

Unsupported answer

The model adds a plausible detail not in context.

FIX → require citations and abstention.
11 / 12

Takeaway

RAG checklist

✓Prepare trusted, current sources
✓Chunk text without losing meaning
✓Embed questions and handbook chunks
✓Retrieve a small set of relevant evidence
✓Generate from context, cite, and abstain
12 / 12