Vector Databases: A Revolution in Search

Introduction

Traditional databases have long been the backbone of information systems, enabling precise queries such as the existence of a specific record. However, vector databases introduce a new dimension by focusing on similarity search. This shift is crucial in a world where a large portion of data—whether documents, images, user behaviors, or audio files—cannot be effectively searched through simple exact matches. Embedding models play a key role here, transforming raw data into vectors, where geometric proximity translates to semantic similarity.

The major challenge lies in scale. Comparing a query vector to every stored vector involves billions of floating-point operations, making real-time search impractical. Vector databases overcome this obstacle through approximate nearest neighbor algorithms, which significantly reduce the number of calculations needed while providing results nearly identical to exhaustive searches, but at a much lower cost.

This article aims to explore this functionality on three levels: the fundamental problem of similarity and the role of vectors, how production systems store and query embeddings using filtering and hybrid search techniques, and finally, the indexing algorithms and architectural choices that enable the management of these systems at scale.

Level 1: Understanding the Similarity Problem

Traditional databases are designed to handle structured data, organized in rows and columns, and to retrieve it using exact searches or range queries. SQL excels in this area. However, a large portion of real-world data, such as texts, images, audio, and user behavior logs, escapes this rigid structuring. For these types of data, exact match searching is inadequate.

The solution lies in representing this data as vectors: fixed-length arrays composed of floating-point numbers. Embedding models, such as OpenAI's text-embedding-3-small for texts or vision models for images, transform raw content into vectors that capture their semantic meaning. Thus, similar content generates close vectors. For example, the terms "dog" and "puppy" are found close together in vector space, just as a photo and a drawing of a cat.

A vector database stores these embeddings and allows for similarity-based searches, such as "find the 10 vectors closest to this query vector." This process is known as nearest neighbor search.

Level 2: Storing and Querying Vectors

Embeddings

Before a vector database can operate, the content must be converted into vectors. This conversion is performed by embedding models, which are neural networks mapping input into a dense vector space, typically with 256 to 4096 dimensions depending on the model used. The specific values in the vector have no direct meaning; what matters is the geometry: close vectors indicate similar content.

To obtain a vector, one can call an embedding API or run a model locally, then store the resulting array of floats along with the associated metadata for the document.

Distance Metrics

The similarity between vectors is measured by their geometric distance. Three metrics are commonly used:

• Cosine similarity: it measures the angle between two vectors, ignoring their magnitude. This metric is often used for textual embeddings, where direction is more significant than length.

• Euclidean distance: it measures the straight-line distance in vector space, useful when magnitude is important.

• Dot product: fast and efficient when vectors are normalized. Many embedding models are designed to use it.

The choice of metric should align with how the embedding model was trained. A poor selection can degrade the quality of results.

The Nearest Neighbor Problem

Finding the exact nearest neighbors is straightforward for small datasets: simply calculate the distance between the query and each vector, sort the results, and return the K best. This method, known as brute-force or flat search, is 100% accurate but scales linearly with the size of the dataset. With 10 million vectors of 1536 dimensions each, a flat search becomes too slow for real-time queries.

The solution lies in approximate nearest neighbor (ANN) algorithms, which trade a small amount of accuracy for significant speed gains. Production vector databases use these ANN algorithms in the background. Specific algorithms, their parameters, and trade-offs will be examined in the next level.

Metadata Filtering

Pure vector search returns the most semantically similar items on a global scale. In practice, one often desires more specific results, such as "find the most similar documents belonging to this user and created after this date." This is known as hybrid search: it combines vector similarity with attribute filters.

Implementations vary. Pre-filtering first applies the attribute filter, then executes the ANN on the remaining subset. Post-filtering first executes the ANN, then applies the filter. Pre-filtering is more accurate but more costly for selective queries. Most production databases use a variant of pre-filtering with a smart index to maintain speed.

Hybrid Search: Dense + Sparse

Pure dense vector search may lack precision at the keyword level. A query for "GPT-5 release date" might semantically drift towards general AI topics rather than the specific document containing the exact phrase. Hybrid search combines dense ANN with sparse retrieval (BM25 or TF-IDF) to achieve both semantic understanding and keyword-level precision.

The standard approach is to run dense and sparse searches in parallel, then combine the scores using reciprocal rank fusion (RRF)—a rank-based fusion algorithm that does not require score normalization. Most production systems now support hybrid search natively.

Level 3: Indexing for Scale

Approximate Nearest Neighbor Algorithms

The three most important approximate nearest neighbor algorithms each occupy a different point on the trade-off surface between speed, memory usage, and recall.

• Hierarchical Navigable Small World (HNSW): this algorithm builds a multi-layer graph where each vector is a node, with edges connecting similar neighbors. The upper layers are sparse, allowing for fast long-distance traversal, while the lower layers are denser for precise local search. During a query, the algorithm navigates through this graph to the nearest neighbors. HNSW is fast, memory-intensive, and offers excellent recall, making it the default choice in many modern systems.

• Inverted File Index (IVF): it groups vectors into clusters using k-means, builds an inverted index that maps each cluster to its members, and then only searches the closest clusters during the query. IVF uses less memory than HNSW but is often slightly slower and requires a training step to build the clusters.

• Product Quantization (PQ): this algorithm compresses vectors by splitting them into subvectors and quantizing each to a codebook. This can reduce memory usage by 4 to 32 times, allowing for the management of datasets at the scale of billions. It is often used in combination with IVF in the form of IVF-PQ in systems like Faiss.

Index Configuration

HNSW has two main parameters: ef_construction and M:

• ef_construction: this parameter controls how many neighbors are considered during index construction. Higher values generally improve recall but take longer to build.

• M: it determines the number of bidirectional links per node. A higher M generally improves recall but increases memory usage.

These parameters must be tuned based on the desired recall, latency, and available memory budget.

During querying, ef_search controls how many candidates are explored. Increasing this parameter improves recall at the expense of latency. It is a runtime parameter that can be adjusted without rebuilding the index.

For IVF, nlist defines the number of clusters, and nprobe determines how many clusters to search during the query. More clusters can improve accuracy but also require more memory. A higher nprobe improves recall but increases latency.

Recall vs. Latency

ANN operates on a trade-off surface. One can always achieve better recall by searching more indexes, but this comes at a cost in latency and computation. It is essential to evaluate the specific dataset and query patterns. A recall@10 of 0.95 might be excellent for a search application; a recommendation system might need 0.99.

Scale and Sharding

A single HNSW index can fit in memory on a machine for about 50 to 100 million vectors, depending on dimensionality and available RAM. Beyond that, sharding is necessary: partitioning the vector space across nodes and distributing queries among the shards, then merging results. This introduces coordination overhead and requires careful selection of the shard key to avoid hot spots.

Storage Backends

Vectors are often stored in RAM for fast ANN search. Metadata is typically stored separately, often in a key-value or column store. Some systems support memory-mapped files to index datasets larger than RAM, spilling to disk when necessary. This trades some latency for scale.

Disk-based ANN indexes like DiskANN (developed by Microsoft) are designed to operate from SSDs with minimal RAM. They achieve good recall and throughput for very large datasets where memory is the primary constraint.

Vector Database Options

Vector search tools generally fall into three categories.

First, you can choose from purpose-built vector databases such as:

• Pinecone: a fully managed, serverless solution.