Embeddings and indexing

RAG starts with a deceptively simple idea: instead of making the model memorise your documents, you search them and feed the relevant bits into the context. The magic that makes search work on meaning rather than keywords is the embedding.

What an embedding is

An embedding is a vector, a list of a few hundred numbers, that represents a chunk of text. Texts with similar meaning get vectors that are close together in that space. "The cat sat on the mat" and "a feline is on the rug" end up near each other, even though they share no keywords. That is what makes semantic search work.

An embedding model is a small, specialised model that turns text into these vectors. They are cheap, fast, and run easily on a CPU.

How indexing works

1. Split your documents into chunks (next lesson).
2. Run each chunk through the embedding model to get a vector.
3. Store the vector with a pointer back to the original chunk.

At query time, you embed the question, find the nearest vectors, and pull those chunks into the prompt. The model answers with the evidence in front of it.

The two parts

• Embedding model — the quality of the vectors decides whether "close" means "related."
• Vector store — where the vectors live. From a simple in-memory list to a dedicated database; local options are abundant.

The honest guidance

1. The embedding model matters, but the chunking matters more. You will see why next.
2. Start simple. An embedding model plus a small vector store is enough for most personal RAG.
3. Test it on real questions. Semantic search fails silently, and the only way to know it failed is to ask questions you already know the answer to.

Embeddings are the quiet foundation of RAG. Get them right and everything downstream works; get them wrong and no amount of prompting fixes the retrieval.