Alternatives to vector embeddings

Every other page here explains how embeddings work. This one explains when to use something else.

That’s not a disclaimer. Reaching for embeddings by default is the most common and most expensive mistake teams make with them, because the failure is quiet: you get plausible results, they’re 70% right, and nobody can say why the other 30% went wrong. Meanwhile a WHERE clause would have been 100% right, instantly, for free.

So: the rest of the toolbox.

Keyword search (BM25 and friends)

The incumbent, and still the right answer more often than the current excitement suggests.

Keyword search indexes the words in your documents and ranks results by how rare and how frequent each matched term is. BM25 is the standard scoring function, and it has been quietly winning information-retrieval benchmarks since the 1990s.

Where it beats embeddings outright:

• Exact identifiers. Order numbers, SKUs, error codes, version strings, surnames. Search ERR-4021 in an embedding system and it will cheerfully return ERR-4012, because as text those are nearly identical. BM25 returns the one you asked for.
• Rare and technical vocabulary. Embedding models only understand words they saw enough of during training. Your internal project codenames were not in that training data. BM25 doesn’t care what a word means, only that it’s rare and it matched.
• Explaining a result. You can point at the matched terms and say “this is why.” Try that with 1,536 numbers when a user asks why their document didn’t come up.
• Cost and operations. No model, no GPU, no API bill, no re-embedding when you change models. Postgres, SQLite and Elasticsearch all ship it.

Where it fails: the thing this whole site exists for. Someone searching “where did I put my keys?” gets nothing, because the note says “spare set for the front door.”

-- Postgres full-text search. No extension, no model, no API key.
SELECT text
FROM notes
WHERE to_tsvector('english', text) @@ plainto_tsquery('english', 'hex key')
ORDER BY ts_rank(to_tsvector('english', text), plainto_tsquery('english', 'hex key')) DESC;

Hybrid search — usually the honest answer

Run both, merge the results. This is what most strong production search does, and it’s less work than it sounds.

The standard merge is Reciprocal Rank Fusion, which combines two ranked lists using only the positions of results, never their scores. That matters, because BM25 scores and cosine similarities are on completely incomparable scales and averaging them is meaningless.

def reciprocal_rank_fusion(*ranked_lists, k=60):
    """Merge ranked lists by position. Higher output score is better."""
    scores = {}
    for ranking in ranked_lists:
        for position, doc_id in enumerate(ranking):
            scores[doc_id] = scores.get(doc_id, 0) + 1 / (k + position + 1)
    return sorted(scores, key=scores.get, reverse=True)

About fifteen lines, and reliably better than either method alone. If you take one thing from this page, take this one.

Start with keyword search, add vectors when you can prove you need them

Build BM25 first. It’s a day’s work and gives you a baseline you can measure against. Then add embeddings and check whether retrieval actually improved on your queries.

Teams that skip the baseline can never answer “is this better than what we’d have had anyway?” — and quite often the answer would have been no.

Plain SQL and structured filters

If the answer lives in a column, query the column.

“Orders over £100 from last month”, “open tickets assigned to Priya”, “invoices where the total doesn’t match the line items” — these are not search problems. They’re WHERE clauses. They’re exact, instant, auditable, and they don’t hallucinate.

This sounds too obvious to state. It gets ignored constantly, because “we’re building an AI feature” quietly reframes every question as a retrieval problem. A striking number of RAG systems are doing fuzzy semantic matching over data that was structured all along.

The tell: if you can write the answer as a query with no natural language in it, you don’t need embeddings. If users want to ask in natural language, the interesting problem is text-to-SQL, not retrieval.

Fuzzy string matching

For typos, name variants and near-duplicates, string-distance algorithms are cheaper and more predictable than embeddings.

• Levenshtein distance — how many single-character edits separate two strings. “Smith” vs “Smyth” is one edit.
• Trigram similarity — overlap of three-character sequences. Postgres ships this as pg_trgm, and it’s genuinely excellent for name and address matching.
• Phonetic algorithms — Soundex and Metaphone match on how a word sounds, which handles “Catherine” and “Kathryn”.

Embeddings handle these badly, because they’re built to capture meaning and these are questions about spelling. “Catherine” and “Kathryn” mean the same thing to a person, but an embedding model sees two ordinary given names and puts them near every other given name.

Knowledge graphs

Embeddings flatten everything into “how similar are these two things.” Some questions aren’t about similarity at all; they’re about relationships.

“Which suppliers does our biggest customer share with our competitors?” has no meaningful vector answer. It’s a graph traversal — several hops through explicit, typed connections.

Graphs win when:

• The connections between entities matter more than the text describing them
• You need multi-hop reasoning (“A supplies B, B owns C, so…”)
• You need a provable answer rather than a probable one
• Relationships change often and should be updated without re-embedding anything

They cost more up front: someone must define the schema and extract the entities, and that work is ongoing rather than one-off. GraphRAG approaches combine both — use a graph for structure, embeddings for the prose hanging off it.

Train an actual classifier

If you have labelled examples and a fixed set of categories, supervised classification beats embed-and-find-nearest.

“Is this ticket a bug, a billing question, or a feature request?” is classification, not search. With a few thousand labelled examples, a small fine-tuned model — or even logistic regression over TF-IDF features — will be more accurate, faster and far cheaper than embedding every ticket and hunting for neighbours.

Embeddings are attractive here because they need no labels. That’s a real advantage when you have none. Once you have labels, use them.

Embeddings are still a good feature for a classifier

These aren’t fully opposed. A common, effective pattern is to embed your text once and train a small classifier on the vectors rather than on raw text. You get the model’s language understanding plus your labels, and training takes seconds.

That’s different from what people usually mean by “just use embeddings”, which is nearest-neighbour against a handful of examples.

Just put it in the prompt

If your whole corpus fits in a model’s context window, retrieval may be solved already.

Context windows now reach hundreds of thousands of tokens. A 200-page employee handbook is roughly 100,000 tokens. You can paste the lot into the prompt and skip the vector database, the chunking strategy, the embedding pipeline and the entire class of bugs that come with them.

When this is right: small, stable corpora. Product documentation. A policy manual. One long contract.

When it isn’t: cost scales with every token on every request, and prompt caching helps but doesn’t erase it. Quality also degrades as the context fills — models reliably attend better to the start and end of a long prompt than the middle. Above a few hundred thousand tokens, or with content that changes constantly, retrieval wins again.

Worth checking before you build a pipeline, though. “Does this even need RAG?” is a cheap question to ask and an expensive one to skip.

Deterministic rules

Sometimes the right tool is an if statement.

Validation, routing, compliance checks, redaction of things matching a known pattern — if the rule can be written down exactly, write it down exactly. A regex that matches a National Insurance number is correct 100% of the time. An embedding-based approach is correct most of the time and fails in ways nobody can predict or explain.

The temptation is that rules feel unglamorous next to machine learning. In a regulated context “we can show you the rule” is worth considerably more than a few points of accuracy.

The older semantic methods

Worth knowing these exist, mostly so you recognise them in a paper or an older codebase:

• LSA / LSI (Latent Semantic Analysis) — 1988. Runs a matrix factorisation over a term–document matrix to find latent topics. The direct ancestor of what embeddings do, without the neural network.
• LDA (Latent Dirichlet Allocation) — probabilistic topic modelling. Still genuinely useful when you want interpretable topics, since unlike embedding dimensions its topics are lists of words a human can read and name.
• Word2Vec / GloVe — one vector per word, no context. “Bank” gets a single position whether you mean money or a river. Superseded for search, but light and fast where that’s the constraint.

LDA is the one that still earns its place: if a stakeholder needs to see what the clusters are about, “topic 4 is: invoice, payment, overdue, reminder” beats “dimension 847” every time.

Choosing

If your problem looks like…	Reach for
Exact codes, IDs, rare technical terms	Keyword search (BM25)
Real user queries in natural language	Hybrid: BM25 + embeddings, merged with RRF
Filters over structured columns	SQL. Just SQL.
Typos, name and address variants	Trigram or Levenshtein matching
Relationships and multi-hop questions	Knowledge graph
Fixed categories, and you have labels	Train a classifier
A small, stable corpus	Put it in the prompt
A rule you can state exactly	Write the rule
Interpretable themes for humans	Topic modelling (LDA)
“Find me things that mean something like this”	Embeddings

That last row is a real, common, valuable problem. It’s just not every problem.

The question worth asking in the design review

Not “how do we use embeddings here?” but “what would this look like without them?”

If the answer is a two-line SQL query, you have your design. If the answer is “we genuinely cannot express what the user means as a query”, embeddings are earning their place — and now you can say why.

Next

Chunking and vector databasesIf embeddings are right for you, this is what production looks like.

What is a vector embedding?The core idea, if you landed here first.

References

• Okapi BM25 — Wikipedia — the standard keyword-ranking function
• Reciprocal Rank Fusion — Cormack et al., the standard way to merge ranked lists
• Postgres — full-text search and pg_trgm — keyword and fuzzy matching you already have
• SQLite FTS5 — full-text search with no server at all
• Elasticsearch — kNN and hybrid retrieval — running both in one system
• From Local to Global: A Graph RAG Approach — Microsoft Research on combining graphs with retrieval
• Lost in the Middle: How Language Models Use Long Contexts — Liu et al., why long prompts degrade in the middle
• Latent Dirichlet Allocation — Blei, Ng & Jordan, the topic-modelling standard
• Indexing by Latent Semantic Analysis — Deerwester et al., 1990, the ancestor of modern embeddings