Vector Databases

First: what is a database, anyway?

A database is a program whose whole job is to remember data and answer questions about it. You hand it a million records once; later you ask “which records match this?” and it answers in milliseconds instead of reading everything from scratch.

Two words come with every database:

• A query — the question you ask it. “All orders from March”, “the user with this email”, “the five things most similar to this one”.
• An index — a helper structure the database builds in advance so it can answer without checking every record. Like the index at the back of a book: you jump straight to the right page instead of reading all 400.

A normal database is excellent at exact questions — this value, this date, this user name. It is useless at “find me the five documents that mean something similar to this paragraph”, because nothing about that question is exact.

A vector database is a database built for exactly that question: its records are vectors, and its query is “what is nearest to this one?”

The problem: nearest neighbors at scale

Embeddings solve “what does this text mean” — and whichever model you chose to produce them (Embedding Models) decides the coordinates. The next problem is speed. Say you have a million embedded documents and one query vector. Which 5 documents are closest to it?

In plain language: “closest” means “most similar in meaning”. That’s the whole trick from the embeddings article — nearby vectors are similar texts. We just need to find them fast.

The obvious answer — compute the distance to every single vector and sort — is called brute-force search, and it works fine up to a few thousand vectors. At a million it’s sluggish; at a hundred million it’s hopeless.

A vector database solves exactly this: it stores vectors and answers “give me the K nearest neighbors of this vector” in milliseconds, even at huge scale. It does this with an index — a data structure that trades a little accuracy for a lot of speed.

How indexes work (the one-paragraph version)

The dominant approach is Approximate Nearest Neighbor (ANN) search. The two index families you’ll actually encounter:

• HNSW (Hierarchical Navigable Small World) builds a layered graph where each vector is connected to a few nearby ones. A query starts at the top layer — long jumps across the space — and zooms in layer by layer, like taking a highway, then city streets, then the exact address. Fast and accurate, at the cost of memory.
• IVF (Inverted File) first groups all vectors into buckets — k-means, a standard algorithm, sorts them into a chosen number of neighborhoods. A query then only scans the few buckets closest to it, instead of the whole collection (a database’s word for “one named set of records”). Less memory, a bit less accurate, easy to tune.

The pipeline

Here is the full life of a vector search system — each step is a real component you’ll build:

Which one to pick: just pick ONE

The choices are genuinely good, which makes choosing harder than it should be:

• Chroma — open source, Python-first, zero setup. The default for learning and prototypes.
• FAISS — a library, not a database. Raw speed, battle-tested at Meta, no server, no persistence story. Great when embeddings live inside your app.
• LanceDB — embedded, serverless-style storage on local files. Simple and modern.
• Supabase pgvector — vectors in Postgres. If your data already lives in Postgres and the scale is moderate, this is the lowest-friction option.
• Qdrant — open source, fast, strong filtering. Popular self-hosted choice.
• Weaviate — full-featured open source database with hybrid search built in — old-fashioned keyword search and vector search combined, so you catch exact product codes and fuzzy meanings in one query.
• Pinecone — fully managed, zero ops. The pick when you’d rather pay than run infrastructure.

Implementing it: index and search

The concept fits in ~20 lines. First file: the math underneath — cosine similarity, brute force. Second file: the same thing with a real vector database (Chroma), where the ANN index replaces the loop.

Cosine similarity in one sentence: imagine two arrows drawn from the same point — one for your question, one for a document. The score is how much they point in the same direction: 1 means “same thing”, 0 means “unrelated”. Length doesn’t matter, which is exactly what you want — a two-word question and a long paragraph can still be about the same thing, so they can still be neighbors.

1import numpy as np
2 
3docs = {
4  #98c379">"forgot password": [0.9, 0.1, 0.0],
5  #98c379">"account locked": [0.85, 0.15, 0.0],
6  #98c379">"refund policy": [0.0, 0.9, 0.2],
7  #98c379">"double charge": [0.05, 0.85, 0.1],
8  #98c379">"track my package": [0.1, 0.0, 0.9],
9}
10query = np.array([0.88, 0.12, 0.02])
11 
12for name, vec in docs.items():
13  a = np.array(vec)
14  score = np.dot(a, query) / (np.linalg.norm(a) * np.linalg.norm(query))
15  print(name, round(score, 3))

Run brute_force.py and the two account-related docs score highest, exactly as on the meaning map in the embeddings article. with_chroma.py is the production-shaped version: you hand it raw texts (it embeds for you), and the query returns the two nearest tickets in milliseconds — the index does the heavy lifting. Swap those three documents for your own and you have semantic search; feed the results to an LLM instead of printing them and you have RAG.

Two practical notes for production:

• Always store the original text next to each vector — you’ll need it for the response, and vectors alone are meaningless.
• Re-index when you switch models. Vectors from different models can’t be compared, so a model change means re-embedding and rebuilding the index.