How RAG actually works, part 2: what an embedding actually is

By · · AI · 5 min

What an embedding actually is: how text becomes coordinates in space, so 'find related text' becomes 'find nearby points.' Includes a real OpenAI embedding call.

Part 2 of 10 in How RAG actually works, a hands-on tour of vectors, embeddings and retrieval.

What an embedding actually is

Forget "vector database" for a minute. Start with a simpler question: how do you get a computer to understand that "How do I reset my password?" and "I forgot my login credentials" mean nearly the same thing, while "How do I reset my router?" means something else, despite sharing the word "reset"?

Keyword search can't do this. It matches strings, not meaning. The word "reset" appears in two of those sentences, but it's the wrong two to group together.

An embedding model solves it by converting a piece of text into a list of numbers, a vector, that represents its meaning as a position in space. Texts with similar meaning land near each other; unrelated texts land far apart. That's the entire idea. The numbers themselves are opaque (nobody reads them), but the distances between them are deeply meaningful.

A toy intuition: imagine describing any word with just two numbers, one for "how royal" and one for "how male."

royalty
                 ↑
        king ●   │   ● queen
                 │
      ───────────┼───────────→ gender
                 │
         man ●   │   ● woman
                 │

"King" and "queen" sit high on the royalty axis; "king" and "man" sit together on the gender side. You could even do arithmetic on these positions: king − man + woman ≈ queen. This famous relationship falls out of real embeddings, and it's the first hint that these vectors capture structure, not just labels.

Real embedding models don't use 2 dimensions, they use hundreds or thousands. OpenAI's current text-embedding-3-small produces 1,536 numbers per text; its larger sibling text-embedding-3-large produces 3,072. Each dimension is some learned, uninterpretable facet of meaning. You'll never know what "dimension 814" represents, and it doesn't matter. What matters is that the model was trained on a colossal amount of text so that semantic similarity reliably becomes spatial proximity.

Here's the mental model to lock in: an embedding turns "meaning" into "coordinates," so that "find related text" becomes "find nearby points." Everything else in RAG is plumbing around that one trick.

Getting a real embedding

Enough theory. In Node with the OpenAI SDK:

import OpenAI from "openai";

const openai = new OpenAI(); // reads OPENAI_API_KEY from env

async function embed(text: string): Promise<number[]> {
  const res = await openai.embeddings.create({
    model: "text-embedding-3-small",
    input: text,
  });
  return res.data[0].embedding; // a 1,536-length array of floats
}

const v = await embed("How do I reset my password?");
console.log(v.length); // 1536
console.log(v.slice(0, 5)); // [-0.013, 0.041, -0.006, 0.028, ...]

That array of floats is the meaning of the sentence, as far as the machine is concerned. Now we need a way to measure how close two of them are.

Related reading