Semantic search vs keyword search: see where each one fails

What you're seeing

The query sits in the middle as the node marked Q. On its left are the query's terms, each labeled with how many documents contain it. On its right are the concepts the query maps to, with their weights. Red curves run from each term to the keyword results that contain it, and from each concept to the semantic results that share it. A term that no document contains shows up as a hollow, dashed node, which is the clearest picture of keyword search coming up empty.

In the result cards, a solid red highlight marks an exact term match. A soft red glow marks a concept match, which can light up a word that shares no letters with the query. Hover or tap any card to see its score broken down: per-term contributions on the left, per-concept shares of the cosine on the right.

The caption bar at the top names the lesson for each preset and compares the two engines against the documents we labeled as correct (marked REL). The stats in the top right track keyword hits, the best cosine score, how much the two top-5 lists overlap, and the active lesson.

Five presets, five lessons:

sofa: keyword search returns 0 results. Semantic search ranks Couch buying guide first.
affordable automobile: a double synonym. Keyword search returns 0 results again, and semantic search finds Cheap used cars under $8,000 and Budget car insurance tips.
SKU-4417: keyword search returns exactly one doc, the right one. Semantic search ranks USB-C charger SKU-4471 first and buries the exact part at #6.
puppy food: both engines get it right, in different orders. Their top-5 lists overlap at 0.80.
laptop without fan noise: both engines put loud-fan reviews at the top. Neither one reads "without".

What is keyword search?

Keyword search (also called lexical search or full-text search) ranks documents by the words they share with the query. The standard scoring function is BM25, which has powered search engines like Lucene, Elasticsearch, and OpenSearch for years. Every document gets a score from three ingredients:

Term frequency: a doc that says "laptop" twice scores higher than one that says it once, with diminishing returns.
Inverse document frequency: rare terms count more than common ones. A match on "sku-4417" is worth more than a match on "review".
Length normalization: a match in a short doc counts more than the same match buried in a long one.

score(q, d) = sum over terms t in q of
  idf(t) * tf(t,d) * (k1 + 1) / (tf(t,d) + k1 * (1 - b + b * |d| / avgdl))

idf(t) = ln(1 + (N - df(t) + 0.5) / (df(t) + 0.5))      k1 = 1.2, b = 0.75

BM25 is fast, explainable, and exact. Its blind spot is just as exact: it only sees the strings. A query for "sofa" against a catalog that says "couch" scores zero on every document, and the engine has no way to know it missed anything.

What is semantic search?

Semantic search ranks documents by meaning instead of shared words. An embedding model turns the query and every document into a vector, a list of numbers that places it in a space where related ideas sit close together. Search then becomes geometry: find the document vectors pointing in the same direction as the query vector, usually measured by . are what make "sofa" and "couch" land in the same neighborhood.

The semantic engine in this demo is a toy, and it is labeled as one. Instead of a learned model, it maps about 120 hand-picked words onto 12 named concept axes such as comfort-furniture, vehicles, and price-low. A document's vector is the idf-weighted average of its word vectors. Real embedding models learn hundreds or thousands of unnamed dimensions from huge amounts of text, but the mechanics of ranking by cosine are the same. Named axes just let you see why a match happened. To explore a real embedding space, try embeddings in 3D.

One side effect of a toy this small: "sofa" and "couch" map to the identical vector, so the top cosine reads 1.000. A real model would score them as close but not identical.

Semantic search vs keyword search: where each one wins

The two engines fail in opposite places, which is the whole argument for using both. Keyword search wins when the user types the same string the document contains. Semantic search wins when the user describes the idea in different words.

Synonyms (sofa vs couch): keyword search misses them entirely; semantic search finds them.
Codes, SKUs, and names: keyword search nails them; semantic search blurs them into similar items.
Negation ("without"): both ignore it.
Explaining a result: easy for keyword search (show the matched terms), harder for semantic search (a similarity score).
Infrastructure: keyword search needs an inverted index; semantic search needs an embedding model and a vector index.

Why does semantic search miss exact matches?

Run the SKU-4417 preset. Only one document contains that string, a replacement cooling fan, and keyword search returns it alone at #1. Semantic search ranks USB-C charger SKU-4471 first and Office desk setup second, and the exact part lands at #6.

The reason is that an identifier carries almost no meaning. The toy tokenizer splits "SKU-4417" into "sku" and "4417". It knows "sku" as a vaguely hardware-flavored word and has nothing for the digits, so the query vector just says "some hardware product". SKU-4417 and SKU-4471 differ by two swapped digits, which matters completely to a customer and not at all to a vector. Real embedding models behave the same way in spirit: part numbers, error codes, version strings, and rare proper names are exactly where pure vector search gets fuzzy.

Can search engines understand negation?

Mostly not. Run "laptop without fan noise". The user wants a quiet laptop, and the corpus has one: "Fanless notebooks that run silent". Keyword search ranks the two loud-fan reviews at the top, because they contain "laptop", "fan", and "noise". It even pulls in a headphones review on the word "noise". The fanless doc shares no terms, so it never appears.

Semantic search does rank the fanless notebook at #2, which looks like understanding. It isn't. "Silent" and "noise" both load onto the audio axis, so the embedding scores a silent laptop and a loud laptop as the same topic. The demo proves it with a check you can see in the caption bar: delete "without" from the query, and both rankings come out identical. Neither engine ever used the word.

Antonyms landing close together is a known trait of real embeddings too, since words like "loud" and "quiet" appear in the same contexts. When negation matters, it usually needs a filter (a structured "fanless" attribute) or a reranking model that reads the query and document together.

What is hybrid search?

runs keyword and semantic search together and merges the two result lists. The demo's slider does the simplest version: it scales each engine's scores so its best result is 1, then mixes them with a weight w, where 0 is pure keyword and 1 is pure semantic.

Drag it and watch the blend list. At w = 0, "sofa" fails because keyword search has nothing. Near w = 1, "SKU-4417" fails: at w = 0.9 the exact part drops to #4. In between, from 0.05 to 0.85, both queries keep a correct doc in the top 3. You don't have to predict which kind of query is coming. The blend covers both.

Production systems expose the same idea. As of October 2026, Weaviate's hybrid search takes an alpha parameter where 0 is pure keyword, 1 is pure vector, and 0.5 is the default. Elasticsearch offers both a linear retriever that takes a weighted sum of normalized scores and a reciprocal rank fusion (RRF) retriever. RRF ignores raw scores entirely and combines results by rank position, which sidesteps the problem that BM25 scores and cosine scores live on different scales.

BM25 vs embeddings: which should your site search use?

For most site search boxes, the answer is both, in roughly this order of effort:

Start with BM25 if you have nothing. It handles product names, SKUs, and exact phrases well, needs no model, and every result is explainable.
Add embeddings when users describe things in their own words. Support questions, product discovery, and help-center search are full of paraphrases that keyword search will miss.
Blend them rather than switching. A hybrid query loses almost nothing on exact matches and gains the synonyms.
Rerank the top results if quality still matters. A reranker reads the query and each candidate together, which catches some of what both first-stage engines miss. See how retrieval reranking works.

The same choice shows up inside AI assistants. Retrieval-augmented generation depends on fetching the right passages before the model answers, and a retriever that misses an exact product code will hand the model the wrong context. At scale, the semantic half runs on an approximate nearest-neighbor index; HNSW vector search shows how that index finds close vectors without comparing against every one.

How the demo computes it

The 24 documents were written for this demo, and they were written to contain the traps: couch but never sofa, cheap car but never affordable automobile, two part numbers that differ by two digits, and two reviews about loud laptop fans. Both engines run live in the page on every keystroke. BM25 uses k1 = 1.2 and b = 0.75 over the of each doc, keeping codes like SKU-4417 whole the way a product index would. The semantic side computes idf-weighted word vectors and cosine similarity on the 12 named axes. No ranking is hardcoded.

The REL labels are our relevance judgments for each preset, and the verdict compares the best rank each engine gives a labeled doc. The page checks itself on load: each preset lesson, the blend at w = 0.5, and a set of junk inputs (empty, punctuation, unknown words) that must never throw.

Frequently asked questions

Is semantic search better than keyword search?

Not across the board. Semantic search is better at synonyms and paraphrases. Keyword search is better at exact identifiers, names, and rare terms. Each one fails where the other is strong, which is why most production systems blend them.

What is BM25?

BM25 is the standard ranking function for keyword search. It scores a document by how often it contains each query term, how rare that term is across the collection, and how long the document is. It is the default ranking in Lucene-based engines like Elasticsearch and OpenSearch.

Does semantic search need a vector database?

It needs somewhere to store vectors and find the nearest ones quickly. That can be a dedicated vector database, a vector index inside a search engine you already run, or a plain array for a small collection like this demo's 24 docs.

What is reciprocal rank fusion?

Reciprocal rank fusion (RRF) merges ranked lists by position instead of score. Each document earns 1 / (k + rank) from every list it appears in, and the sums decide the final order. It avoids having to make BM25 scores and cosine scores comparable.

Why did semantic search return results for a made-up word?

It won't in this demo. A query with no recognized concepts gets a zero vector, every cosine is 0, and the semantic column says no concepts were recognized. Real embedding models do return something for nearly any input, because they always produce a vector, which is one reason pure vector search can feel confidently wrong.

Key takeaways

Keyword search fails silently on synonyms. In the demo, "sofa" and "affordable automobile" return 0 keyword hits from 24 docs that are full of couches and cheap cars, because no doc uses those exact words.
Semantic search fails on identifiers. It ranks USB-C charger SKU-4471 first for "SKU-4417" and puts the exact part at #6, because a code is a string, not a meaning.
Neither engine understands negation. Deleting "without" from "laptop without fan noise" leaves both rankings identical, so both put loud-fan reviews at the top.
A plain 50/50 blend fixes both failures at once. In the demo, every weight from 0.05 to 0.85 keeps the couch doc and the exact SKU in the top 3.
Semantic search finding the right answer is not proof it understood the query. The fanless notebook ranks high for "laptop without fan noise" only because "silent" and "noise" sit on the same concept axis.