Embeddings and Semantic Search

How to turn text into lists of numbers so that a program can find passages that mean the same thing, even when they share no words — and how to tell when your search is only matching words after all.

A member of staff types “my IC is missing” into your help portal. The answer is sitting right there in an article titled “Reporting a lost identity card to the registration department”. An ordinary keyword search finds nothing, because the two sentences do not share a single meaningful word. A person sees at once that they mean the same thing. Getting a program to see it too is what embeddings are for — and it is the foundation of the document search behind every “chat with your documents” tool.

Meaning as coordinates

An embedding is a list of numbers that stands for a piece of text (or an image, or a sound) in such a way that things with similar meanings get similar lists. Think of each number as a coordinate. A map gives every town two coordinates, latitude and longitude, and towns that are close in reality are close on the map. An embedding model gives every sentence hundreds or thousands of coordinates, and sentences that are close in meaning end up close in that space.

Nobody hand-writes these coordinates. An embedding model — a neural network related to the large language models you call on Calling LLM APIs — learns them from vast amounts of text, by being trained so that, for example, a question and its answer land near each other and unrelated sentences land far apart. The individual numbers usually have no human meaning on their own; only the distances between points do. The idea has a long history in language research; a widely cited milestone was word2vec (Mikolov and colleagues, 2013), which learned word vectors in which king sat near queen. Today’s models embed whole sentences and paragraphs.

To make the idea concrete, here are five words with hand-made three-number embeddings, where we have decided the three dimensions mean “royalty”, “food” and “animal”. Real embeddings are learned and much longer, but the arithmetic is identical.

Measuring similarity

How do we decide that two lists of numbers are “close”? The usual measure is cosine similarity: imagine each list as an arrow from the origin, and measure the angle between two arrows. Arrows pointing the same way score 1; arrows at right angles score 0; opposite arrows score −1. Because it looks only at direction, not length, a long document and a short sentence about the same topic can still score highly.

import numpy as np

# Hand-made 3-D "embeddings": [royalty, food, animal]
words = {
    "king":   np.array([0.9, 0.0, 0.1]),
    "queen":  np.array([0.8, 0.1, 0.1]),
    "durian": np.array([0.0, 0.9, 0.1]),
    "rambutan": np.array([0.1, 0.8, 0.1]),
    "tiger":  np.array([0.1, 0.1, 0.9]),
}

def cosine(a, b):
    return a @ b / (np.linalg.norm(a) * np.linalg.norm(b))

for other in ["queen", "durian", "rambutan", "tiger"]:
    print(f"king vs {other:<9} {cosine(words['king'], words[other]):.3f}")
print(f"durian vs rambutan {cosine(words['durian'], words['rambutan']):.3f}")
king vs queen     0.992
king vs durian    0.012
king vs rambutan  0.136
king vs tiger     0.218
durian vs rambutan 0.992

a @ b is the dot product: multiply the lists element by element and add up. np.linalg.norm is an arrow’s length. Dividing the dot product by both lengths gives the cosine of the angle. As designed, king and queen point almost the same way (0.992), as do the two fruits, while king and durian are nearly at right angles (0.012). Many embedding APIs return vectors already scaled to length 1, in which case the dot product alone is the cosine similarity — a useful speed-up when you compare millions of vectors.

Semantic search follows directly. Embed every document once and keep the vectors. When a question arrives, embed it with the same model, compute its similarity to every document, and return the top few. That is the whole algorithm; everything else is engineering for speed and scale.

A local stand-in: TF-IDF

Real embedding models need an API key or a sizeable download. To practise the search mechanics with nothing but scikit-learn, we can use an older technique, TF-IDF (term frequency–inverse document frequency). It turns each document into a vector with one position per word in the vocabulary. A word scores highly in a document if it appears there often and is rare across the collection, so “passport” counts for a lot and “the” for nothing.

Be clear what this is

TF-IDF vectors capture shared words, not meaning. They are a fine way to learn the search pipeline, and a strong baseline for keyword search, but they are not embeddings in the modern sense. Watch for the moment they fail.

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics.pairwise import cosine_similarity

docs = [
    "How to renew your passport at an immigration office",
    "Paying your electricity bill online with a bank card",
    "Applying for a driving licence for the first time",
    "Reporting a lost identity card to the registration department",
    "Booking a doctor appointment at a government clinic",
    "Renewing road tax and car insurance before it expires",
    "Opening a savings account for a child",
    "Claiming a refund for a cancelled flight",
    "Registering a newborn baby's birth certificate",
    "Checking your income tax return status online",
]

vectorizer = TfidfVectorizer(stop_words="english")
doc_vectors = vectorizer.fit_transform(docs)
print("Vocabulary size:", len(vectorizer.vocabulary_))

def search(query, k=3):
    scores = cosine_similarity(vectorizer.transform([query]), doc_vectors)[0]
    best = scores.argsort()[::-1][:k]
    print(f"\nQuery: {query}")
    for i in best:
        print(f"  {scores[i]:.3f}  {docs[i]}")

search("renew passport")
search("my IC is missing")
search("lost passport")
search("cara perbaharui pasport")
Vocabulary size: 46

Query: renew passport
  0.707  How to renew your passport at an immigration office
  0.000  Checking your income tax return status online
  0.000  Registering a newborn baby's birth certificate

Query: my IC is missing
  0.000  Checking your income tax return status online
  0.000  Registering a newborn baby's birth certificate
  0.000  Claiming a refund for a cancelled flight

Query: lost passport
  0.354  How to renew your passport at an immigration office
  0.296  Reporting a lost identity card to the registration department
  0.000  Registering a newborn baby's birth certificate

Query: cara perbaharui pasport
  0.000  Checking your income tax return status online
  0.000  Registering a newborn baby's birth certificate
  0.000  Claiming a refund for a cancelled flight

What each part does: TfidfVectorizer(stop_words="english") learns the vocabulary and drops common English words such as “how”, “your” and “to”. fit_transform turns the ten documents into a table of ten rows by 46 word columns. transform turns a query into a row using the same vocabulary, and cosine_similarity compares it with every document. argsort()[::-1][:k] sorts the scores from highest to lowest and keeps the top k.

Now read the results critically, because they teach more than the code:

Notice also that “renew” did not match “Renewing road tax”: TF-IDF treats renew and renewing as unrelated words. And in a real system you would add a minimum score, so that a query scoring 0.000 against everything returns “nothing found” rather than three random documents.

Quick check

Two embedding vectors have a cosine similarity of 0.98. What does that suggest?

Cosine similarity measures direction, not length or word overlap. High similarity means the model places them close in meaning.

Why did TF-IDF fail on “my IC is missing”?

TF-IDF has no idea that “IC” and “identity card”, or “missing” and “lost”, mean the same. A learned embedding model does.

You embed your documents with model A and your queries with model B. What happens?

Each model learns its own space. Comparing vectors from two models is like comparing map coordinates from two different projections.

Real embedding models

A real embedding model replaces the TF-IDF step; the rest of the pipeline is unchanged. Anthropic does not offer its own embedding model, and its documentation points to Voyage AI as one provider As of October 2026; OpenAI, Google, Cohere, Mistral and others offer embedding APIs, and open-weight embedding models can be downloaded and run on your own machine. Here is OpenAI’s embeddings endpoint, following its official guide:

import numpy as np
from openai import OpenAI

MODEL = "text-embedding-3-small"   # check the provider's docs for current model names
client = OpenAI()                  # reads OPENAI_API_KEY

texts = ["How do I renew my passport?",
         "Steps to extend a travel document",
         "Best nasi lemak in Penang"]
response = client.embeddings.create(model=MODEL, input=texts)
vectors = np.array([item.embedding for item in response.data])
print(vectors.shape)

unit = vectors / np.linalg.norm(vectors, axis=1, keepdims=True)
print((unit @ unit.T).round(2))     # cosine similarity of every pair
Example output — your numbers will differ
(3, 1536)
[[1.   0.62 0.08]
 [0.62 1.   0.07]
 [0.08 0.07 1.  ]]

input can be a single string or a list; you get one vector per item in response.data, each as item.embedding, a list of floats. text-embedding-3-small returns 1,536 numbers per text by default As of October 2026. The normalising line divides each vector by its length so that the matrix product gives cosine similarities for every pair at once. The pattern to expect: the two passport sentences, which share almost no words, score much closer to each other than either does to nasi lemak. That is the synonym problem solved. (Exact figures depend on the model; treat the sample output as illustrative.)

Three habits make real embeddings work well. Embed once, store, reuse: embedding costs money and time, so compute document vectors when documents change, not on every query. Use one model for both sides, and re-embed everything if you switch models. And use the provider’s query/document options when it has them — Voyage’s API, for example, takes an input_type of "query" or "document" because the two are phrased differently.

Vector databases

For a few thousand documents, a NumPy array and a loop are all you need — the code above is a perfectly good search engine at that size. A vector database earns its place when you have millions of vectors, many users, or data that changes constantly. What it adds:

ANNFast approximate searchMilliseconds over millions

Comparing a query with every vector gets slow at scale. Vector databases build indexes (such as HNSW graphs) that find the nearest neighbours approximately — almost always the right ones, far faster. You trade a sliver of recall for a large gain in speed.

+metaFilters and metadata“Only HR documents from 2026”

Each vector is stored with its text and labels — source file, date, department, access level — so a search can be limited to what this user is allowed to see. Permission filtering is essential when documents are confidential.

CRUDUpdates and persistenceAdd, change, delete

Documents change. A database lets you insert, update and delete vectors without rebuilding everything, and keeps them safe on disk.

e.g.ExamplesDedicated and built-in

Dedicated vector stores include Chroma, Qdrant, Weaviate, Milvus and Pinecone; libraries such as FAISS give fast search inside your own program; and general databases have added vector search, for example the pgvector extension for PostgreSQL As of 2026. These are examples, not recommendations — if you already run PostgreSQL, adding vectors to it is often the simplest start.

Pitfalls

Chunk size. Embedding models have input limits, and one vector for a 40-page document blurs every topic in it into a single average. So documents are split into smaller pieces first — chunking. Too small (single sentences) and a chunk loses context: “It expires on 31 March” — what does? Too large and the vector is vague and you send the model pages of irrelevant text. A common starting point is a few hundred words per chunk, split at paragraph or heading boundaries, with a little overlap between neighbours, and the document title added to each chunk. Then measure, as on Measuring Models: build a small set of questions with known right passages and check how often the search finds them.

Multilingual text. Malaysian documents often mix Bahasa Melayu, English, Chinese and Tamil — sometimes in one sentence (“Boleh claim MC tak?”). Many embedding models are trained mostly on English. A multilingual model can place “cara perbaharui pasport” near “how to renew a passport”; an English-centred one may not. Test with real queries in every language your users write, including informal Manglish, before you trust the search.

Short words and codes. Abbreviations (IC, MC, KWSP, LHDN), product codes and names can be poorly represented. Many production systems therefore run keyword search and embedding search together and merge the results — so-called hybrid search — getting exact matches and meaning matches.

Consensus Similar is not the same as true or relevant. Semantic search returns the passages closest to the query, even when none actually answers it. It also inherits the biases of the text the embedding model learned from. Always set a minimum score, show users where results came from, and keep testing.

How many dimensions should an embedding have?

Use the model’s default to start. More dimensions can capture finer distinctions but cost more storage; some models let you request shorter vectors with only a small loss in quality.

Can I store my own vectors in a spreadsheet?

You can, but a NumPy file (np.save) or a database is far more practical. A thousand 1,536-number vectors is about six megabytes as 32-bit floats.

Is it safe to send documents to an embedding API?

The text leaves your computer, just as with a chat API. Check your organisation’s rules, and consider a locally run open-weight embedding model for confidential material.

// Pause and reflect

  1. Think of a search box you use at work or on a government site. Where does it fail you in the way TF-IDF failed on “my IC is missing”?
  2. Which languages and abbreviations would your own users mix in their queries?
  3. For a collection you know, what would a sensible chunk be: a paragraph, a section, a page?

Your notes are saved privately in this browser.

Try this

Add a minimum score to search() so that it prints “No good match” when the best score is below 0.2. Then try to fix the IC failure the old-fashioned way: pass TfidfVectorizer(stop_words="english", ngram_range=(1, 2)), and add a synonym line such as “IC identity card MyKad missing lost” to the lost-card document. Notice how much hand-work each synonym costs — that is the work an embedding model does for you. If you have an API key, swap in real embeddings and rerun all four queries.

Key terms on this page

Sources & further reading