What is a vector embedding?

Let me start with the problem, because the solution makes no sense until you’ve felt the problem.

The problem: computers can’t read

Say you’ve got eight notes saved in an app:

1. The spare set for the front door is in the kitchen drawer.
2. The dog needs his flea treatment on the 3rd.
3. Mum's birthday is in April, she likes tulips.
4. Renew the car insurance before it lapses.
5. The standing desk needs a 4mm hex key to assemble.
6. Pizza dough: 500g flour, 300ml water, rest overnight.
7. The wifi password is taped under the router.
8. Physio said ice the knee for twenty minutes.

Now you search for “where did I put my keys?”

You and I both know the answer is note 1. But look at what a normal search does. It takes your query, splits it into words, and hunts for those words in the text. The word “keys” appears in exactly one note — number 5, the hex key. So LIKE '%key%' hands you back a page about assembling furniture, and note 1 doesn’t even make the list, because it never uses the word “key” at all.

This isn’t a bug you can fix by writing a better query. The search matched letters. You wanted it to match meaning. Those are genuinely different jobs, and string matching can only ever do the first one.

You've hit this before

Every time you’ve searched a help centre for “can’t log in” and got nothing, then tried “sign-in failure” and found the article instantly — that’s this exact problem. The article was always there. Your words just didn’t line up with the author’s words.

The idea: turn meaning into coordinates

Here’s the move. What if every piece of text got a position — actual coordinates — and things that mean similar things landed near each other?

Forget text for a second. Imagine plotting animals on a graph. One axis is size, the other is how domesticated they are:

        domesticated
             ↑
    dog •    │    • cat
             │
   horse •   │
─────────────┼───────────────→ size
             │
             │  • wolf
        wild │       • bear

Nobody had to teach you that dog and cat land near each other. It falls out of the axes. And now a question like “what’s most similar to a dog?” has a boring, mechanical answer: measure the distances, pick the smallest. That’s just Pythagoras. A computer can do that a million times a second without understanding a single thing about animals.

An embedding is exactly this, applied to text. A model reads your sentence and gives you back a position. Sentences that mean similar things get positions close together.

Two honest complications

I’ve simplified two things, and you should know about both.

First: it’s not two axes, it’s hundreds or thousands. OpenAI’s text-embedding-3-small gives you 1,536 numbers per piece of text. Not 2. That’s impossible to draw and impossible to picture, and you should stop trying — your intuition from the 2D graph carries over completely fine. Close is still close. Far is still far. There are just a lot more directions to be close in.

Second: nobody knows what the axes mean. In my animal graph I picked “size” and “domesticated” because they’re easy to explain. In a real embedding, dimension 47 doesn’t stand for anything you can name. It’s not “formality” or “sadness” or “is about food.” The model worked out its own directions while training, and they’re a tangled mess that happens to work brilliantly.

This bothers people, so let me say it plainly: you never need to know what the numbers mean. You need exactly one thing from them — that similar text produces similar numbers. Everything else in this guide is built on that one property. The individual values are none of your business, and treating them as an opaque blob is the correct professional attitude, not a gap in your understanding.

A useful mental model

Think of an embedding like a fingerprint for meaning. You’d never look at a fingerprint and say “ah, this whorl indicates the person is tall.” A fingerprint is only useful for comparison — is this the same as that one? Embeddings are the same. Their value is entirely in comparing them to each other.

So how does the keys search work?

Three steps. This is the whole system, and it doesn’t get more complicated than this:

1. Ahead of time, run each of the eight notes through an embedding model. Store the eight lists of numbers alongside the notes.
2. When someone searches, run their query through the same model. Now you have a ninth list of numbers.
3. Compare the query’s numbers against all eight stored ones. Sort by closeness. Return the winner.

Search for “where did I put my keys?” and note 1 comes back first — because the model has read enough English to know that a spare set for the front door, sitting in a kitchen drawer, is the sort of thing that answers a question about keys. The hex key note scores poorly, despite literally containing the word, because a 4mm hex key for furniture assembly means something quite different.

That last part is worth pausing on. Keyword search got actively misled by the word “key”. Embeddings weren’t, because they were never looking at words in the first place.

What this buys you

Once you can measure “how similar is this text to that text” as a number, a surprising number of features are the same feature underneath:

• Semantic search — the thing we just built.
• Retrieval-augmented generation (RAG) — before asking an LLM a question, find the most relevant chunks of your own documents and paste them into the prompt. The “find the relevant chunks” step is exactly this search.
• Recommendations — you read this article, here are the closest others.
• Deduplication — two support tickets land close together, they’re probably the same complaint written twice.
• Classification — embed some labelled examples, embed the new thing, see which cluster it falls into.
• Clustering — group ten thousand pieces of feedback by theme without deciding the themes in advance.

Every one of those is “turn things into vectors, then compare vectors.” Learn it once.

Where embeddings genuinely fall down

They’re bad at exact matching. Order numbers, SKUs, error codes, surnames, dates — an embedding will happily tell you ERR-4021 and ERR-4012 are nearly identical, because as text they nearly are. If your users search by reference number, keyword search is better and you should keep it. Serious production systems usually run both and merge the results, which people call hybrid search. Alternatives to vector embeddings covers that and the rest of the toolbox.

They also inherit whatever bias was in the training data, and they can’t tell you why two things scored as similar. Both matter if you’re ranking job applicants or moderating content.

Where the idea came from

This isn’t new. The general notion that a word’s meaning can be captured by the company it keeps — “you shall know a word by the company it keeps” — goes back to the linguist J.R. Firth in 1957. It became practical in 2013 with Word2Vec, which famously produced vectors where king - man + woman landed very close to queen.

Word2Vec had one vector per word, which meant “bank” got a single position no matter whether you meant money or a river. The Transformer architecture in 2017 and BERT in 2018 fixed that by reading the whole sentence before deciding, which is why modern embeddings understand context. Today’s models are that lineage, scaled up.

Next

You’ve got the concept. Now go make some real numbers appear on your screen — it takes about five minutes and it makes all of this feel considerably less abstract.

References

• OpenAI — Embeddings guide — the official explanation, plus current model list and dimensions
• Efficient Estimation of Word Representations in Vector Space — the original Word2Vec paper, Mikolov et al., 2013
• Attention Is All You Need — the Transformer paper, Vaswani et al., 2017
• BERT: Pre-training of Deep Bidirectional Transformers — Devlin et al., 2018
• SBERT — Sentence Transformers documentation — the go-to library for running embedding models locally