A vector database is a store specialized in keeping and searching high-dimensional vectors, typically embeddings produced by vectorizing text, images or audio. Unlike a relational database, which looks for an exact match of values, it answers queries of the kind “find the k nearest records” according to a similarity metric – most often cosine distance or the dot product. Searching every vector exactly becomes untenable at millions of records, so approximate nearest neighbour (ANN) search is used instead, built on indexes such as HNSW (a multi-layer navigable graph), IVF (dividing the space into cells) or product quantization, which compresses the vectors. The outcome is always a trade-off between speed, memory and the completeness of the results. A production deployment additionally calls for filtering by metadata – by language or by the user’s permissions, for instance – and for hybrid search combining semantic similarity with the classic BM25 keyword score. A vector database is the load-bearing infrastructure of the RAG architecture and the place where the individual chunks of documents are stored.
An ordinary database is like a phone book: you need the exact surname or you find nothing. A vector database is like a librarian who shelves books not alphabetically but by what they are about. Tell him “something about making bread at home” and he comes back with an armful of books on baking, sourdough and mills – even though your exact wording appears in none of them. To manage it quickly he does not walk the shelves one by one; he keeps his own map of shortcuts between topics and follows that. Which is why a suitable book occasionally slips past him – that is the price of speed.