← back to the archiveCover illustration for “Vector search: finding results by meaning, not keywords”
TERM8mo ago·Published · Updated ·by Andy Padia

Vector search: finding results by meaning, not keywords

In short: Vector search returns the items whose embeddings sit closest to the query's embedding, matching by meaning, usually through an approximate index.

Vector search is a retrieval method that represents queries and items as embeddings and returns the items whose vectors are nearest to the query vector. Because embeddings capture meaning, it can find relevant results that share no words with the query. Large collections rely on approximate nearest-neighbour indexes such as HNSW, published by Malkov and Yashunin in 2016, to stay fast.

Vector search in plain words

You ask for "something to keep my tea hot" and are shown thermos flasks, even though you never typed "thermos".

How it works

  1. Embed every item once with an embedding model.
  2. Build an approximate nearest-neighbour index, commonly HNSW or IVF.
  3. Embed the query with the same model.
  4. Return the top-k closest items by a distance measure such as cosine similarity.

Exact search compares the query against every item, which gets slow as the collection grows. Approximate search checks a subset. The pgvector README shows the trade in one place: exact nearest-neighbour search is its default, and adding an HNSW or IVFFlat index trades some recall for speed. It supports L2 distance, inner product and cosine distance. HNSW itself stacks proximity graphs in layers, so a search starts in a sparse upper layer and descends to denser ones.

Example

As an illustration, take a help centre: the query "app keeps logging me out" retrieves an article titled "Session expiry and token refresh", which a keyword search for "logging out" would miss.

Where you'll meet it

  • RAG retrieval steps
  • Postgres through the pgvector extension, and dedicated vector databases
  • E-commerce and media recommendations
  • Semantic code search in developer tools
  • Image and multimodal search

What it changes for you

For developers, vector search is a standard building block, but tuning it (top-k, filters, distance thresholds) needs evaluation data in place of guesswork. For product teams, it changes what "search works" means: users can describe needs instead of guessing terms.

Vector search is the mechanism. Semantic search is the goal, matching by meaning, and it can be achieved with vector search, rerankers or both.

Common mistake

My view: returning top-k results with no similarity threshold. When nothing relevant exists, vector search still returns the "closest" items, and an LLM will happily answer from them. Set a cutoff and let "no good match" be an answer.

Embeddings · Vector database · HNSW · Semantic search · Hybrid search

From the archive: Why your RAG pipeline returns irrelevant results · Before tuning retrieval, prove the missing item was indexed

Set a similarity cutoff and let "no good match" be a valid answer.

#field-guide#retrieval#vector-search#search#embeddings

Sources

  1. arxiv.org/abs/1603.09320
  2. github.com/pgvector/pgvector
← older drop
Vector database: storage built for similarity search
newer drop →
Time to first token (TTFT): how fast an AI starts replying

related drops

explore all 557 drops →
← back to the archiveday 175