What Are Embeddings?

Text as coordinates

Here is the whole idea in one sentence: an embedding gives every piece of text a set of coordinates on a map of meaning — and texts that mean similar things get coordinates that sit close together.

Technically: an embedding is a way to turn text into a list of numbers — a vector — that captures its meaning.

Not keywords, not character codes. Meaning. The sentences “How do I reset my password?” and “I’m locked out of my account” share almost no words, yet their embeddings end up very close. Meanwhile “password” and “passport” — nearly the same string — land in completely different places.

Why should you care? Because this is the trick that lets a computer search by intent instead of by spelling. It’s what makes “chat with your docs”, recommendations, and duplicate detection work — and it’s the foundation of every technique in this track from here on.

Coordinates, from zero

If you have never met a vector, here it is in three steps and no math:

1. One number is a coordinate on a line. “Temperature: 23” puts today somewhere on a line from cold to hot.
2. Two numbers are a coordinate on a sheet of paper. Latitude and longitude put a city on a map. Cities that are close on the map are similar in the only way that map can measure: distance.
3. A list of numbers is a coordinate in that many directions. An embedding is the same trick: 1536 numbers means a location in a space with 1536 directions — a map of meaning, where “far apart” means “different in meaning”.

You cannot picture 1536 directions, and you never need to. The only question you ever ask these coordinates is “which ones are close to this one?” — and that question works exactly the same in 2 dimensions and in 1536.

Three words you’ll meet constantly, with no math attached:

• Vector — a fancy word for “a list of numbers”. That’s all. The embedding of a sentence is a vector: a few hundred numbers that describe where that sentence lives on the map.
• Dimension — how many numbers are in the list. A 1536-dimensional vector is just a list of 1536 numbers. More dimensions means more room to encode nuance; it doesn’t change how you use it.
• Cosine similarity — the “how close are these two?” score that vector search actually computes, usually between 0 and 1. Same idea, score near 1. Unrelated things, score near 0. You never compute it yourself — the database does — but every “these two texts are related” claim is this number.

In practice the vector is long — modern models produce 384 to 3072 numbers per text. Humans can’t picture a 1536-dimensional space, but the math doesn’t care: “distance” between vectors still works, and it works as a proxy for similarity of meaning.

See it for yourself

Below is a mini-map of support tickets, laid out by meaning. Each label is a ticket; the green item is a new incoming question. Hover any point to see its three nearest neighbors — that’s exactly what semantic search computes:

Notice what happened: the query “reset password?” matched forgot password and account locked — not because words match, but because the intent matches. A keyword search would have returned every ticket containing the word “password” and missed everything else.

Hover a billing ticket and a shipping ticket: each one’s neighbors are other tickets from its own group. That clustering is not something anyone programmed — it falls out of the meaning map on its own.

That is the whole product in one interaction: the user asks in their own words, and you return the closest points on the map. Doing it for a million documents instead of twelve is a storage problem, which is what vector databases solve.

What a vector looks like in code

You never read these numbers — you only ever compare them. But seeing one once makes the idea stick:

from openai import OpenAI
client = OpenAI()

text = "How do I reset my password?"
vector = client.embeddings.create(
  model="text-embedding-3-small", input=text
).data[0].embedding

print(len(vector))   # 1536 — one coordinate per dimension
print(vector[:4])    # [0.0231, -0.0412, 0.0167, -0.0084]

# A different sentence, same meaning → nearly the same numbers
other = client.embeddings.create(
  model="text-embedding-3-small",
  input="I can't log in to my account",
).data[0].embedding

Which model produced those numbers matters a lot — that choice is the next article.

Why do similar meanings end up close?

The embedding model is a neural network trained on enormous amounts of text. During training it learns which words and phrases appear in similar contexts — and contexts encode meaning. “Reset my password” and “forgot my password” appear in the same kinds of conversations, so the network learns to place them at nearly the same coordinates.

Two useful properties fall out of this:

• Related texts are near each other. Distance becomes a similarity score you can rank with.
• Direction carries meaning too. The famous example: the vector from “king” to “queen” is roughly the same as from “man” to “woman” — relationships are preserved as geometry.

What embeddings are used for

This one idea powers a surprising amount of modern AI products:

That last one is a big deal: retrieval-augmented generation — the technique behind “chat with your data” — is built on embeddings. It gets its own article, What is RAG?, and everything else in this track leans on them.

A few honest caveats

• Embeddings are produced by a model, and models have biases and blind spots. Rare domains (legal jargon, niche hardware) may be clustered less cleanly.
• The map above is 2D for humans; real embeddings live in hundreds of dimensions, and some distinctions we care about get compressed.
• Different models produce incompatible vectors: you can only compare embeddings from the same model. Picking one is what Embedding Models and Providers is about.