← back to the archiveCover illustration for “Embeddings: turning text into numbers that capture meaning”
TERMday 172·3d ago·Published ·by Andy Padia

Embeddings: turning text into numbers that capture meaning

In short: Embeddings are lists of numbers that represent the meaning of text, images or code, so similar items sit close together for search and RAG.

Embeddings are numeric vectors, lists of hundreds or thousands of numbers, that represent the meaning of a piece of text, an image or code. An embedding model is trained so that items with similar meaning get vectors that are close together. That makes "find things that mean roughly this" a maths problem, which is the basis of semantic search, recommendations, clustering and RAG.

Embeddings in plain words

Giving every sentence a pin on a giant map, where sentences about similar things get pinned near each other, even if they share no words.

How it works

You send text to an embedding model and get back a fixed-length vector. OpenAI's embeddings guide, read in November 2026, lists 1,536 numbers for its small model and 3,072 for its large one. The approach goes back to word2vec in 2013, which learned vectors for single words. Store vectors for all your documents. At query time, embed the query and measure distance, usually cosine similarity, to find the nearest documents. Different embedding models produce incompatible vectors, so you must use the same model for documents and queries.

Example

q = embed("how do I reset my password?")
d1 = embed("Steps to recover account access")
d2 = embed("Pricing for annual plans")
cosine(q, d1)  # high: same meaning, different words
cosine(q, d2)  # low

Where you'll meet it

  • RAG pipelines and semantic search
  • Duplicate detection, clustering and topic discovery
  • Recommendation and "related articles" features
  • Vector databases and search engines with vector fields

What it changes for you

For developers, embeddings make meaning-based search a few lines of code, but choosing the model and chunk size matters more than the database. For data teams, re-embedding a large corpus when you switch models is a real migration with cost and downtime. For multilingual products, pick models tested on your languages.

Keyword search matches exact terms and handles names, codes and IDs well. Embeddings match meaning and handle paraphrase well. Production search usually combines both.

Common mistake

My view: assuming embeddings understand exact identifiers. Searching an error code, SKU or invoice number by embedding often returns "similar-looking" but wrong items. Keep a keyword index for those.

Vector database · Vector search · Semantic search · Hybrid search · Retrieval-augmented generation

From the archive: EmbeddingGemma 2 needs an indexing contract

Use embeddings for meaning and a keyword index for exact identifiers; production search needs both.

#field-guide#embeddings#retrieval#rag#search

Sources

  1. developers.openai.com/api/docs/guides/embeddings
  2. arxiv.org/abs/1301.3781
← older drop
Tool calling: how an LLM uses functions and APIs
newer drop →
Inference: running a trained AI model to get answers

related drops

explore all 557 drops →
← back to the archiveday 175