RAG Series (6): Vector Databases — Storage and Retrieval Infrastructure

Why Do We Need Specialized Vector Databases?

In the first five articles, we figured out how to chunk documents and generate embeddings. Now where do these vectors live, and how are they efficiently retrieved?

You might wonder: "Can't I just store vectors in Redis or PostgreSQL?"

No — traditional databases are designed for exact queries (e.g., WHERE id = 123), while vector retrieval is Approximate Nearest Neighbor (ANN) search: given a query vector, quickly find the Top-K most similar vectors among hundreds of millions of document vectors. Traditional database indexes (B-trees, hash tables) are powerless against this type of "similarity query."

Example:

• Traditional query: Find user with id=42 → O(1) or O(log n)
• Vector query: Find 10 people most similar to user A → requires comparing against all vectors, brute-force O(n) is too slow

Vector databases use specialized ANN indexes (HNSW, IVF, etc.) to reduce O(n) to O(log n), completing similarity searches across billions of vectors in milliseconds.

Three Core Capabilities of Vector Databases

1. Vector Storage

Store massive numbers of vectors (millions to billions), with each vector attached to raw text and metadata. Support incremental writes and deletions.

2. ANN Approximate Nearest Neighbor Search

Core algorithms:

Recommendation: Use Flat for development (simplicity), HNSW for production (best performance).

3. Metadata Filtering

This is the key capability that distinguishes vector databases from pure vector libraries (like FAISS). You can do two things simultaneously:

• Find semantically relevant content using vector similarity
• Apply exact filtering with metadata conditions (e.g., time > 2024-01-01 AND category = "technical")

# Example: Only retrieve technical documents after 2024
results = vectorstore.similarity_search(
    query="microservices monitoring",
    k=5,
    filter={
        "category": "technical",
        "year": {"$gte": 2024}
    }
)

Mainstream Vector Database Comparison

Five Databases at a Glance

Detailed Analysis

Chroma — The Developer's Best Friend

from langchain_chroma import Chroma
 
# Embedded operation, zero configuration
vectorstore = Chroma.from_documents(
    documents=chunks,
    embedding=embeddings,
    persist_directory="./chroma_db"
)

• ✅ Zero config, works after pip install
• ✅ Supports persistence to local disk
• ❌ Single-machine performance limited, not for high concurrency
• ❌ Weak distributed capabilities

Qdrant — The Enterprise Choice for Production

from langchain_qdrant import Qdrant
from qdrant_client import QdrantClient
 
client = QdrantClient(url="http://localhost:6333")
vectorstore = Qdrant(
    client=client,
    collection_name="docs",
    embeddings=embeddings,
)

• ✅ Written in Rust, extremely high performance
• ✅ Very powerful metadata filtering expressions
• ✅ Supports distributed clustering
• ✅ Cloud-hosted version available
• ❌ Requires additional service deployment

Weaviate — The Hybrid Search Specialist

from langchain_weaviate import WeaviateVectorStore
import weaviate
 
client = weaviate.connect_to_local()
vectorstore = WeaviateVectorStore(
    client=client,
    index_name="Docs",
    text_key="text",
    embedding=embeddings,
)

• ✅ Native BM25 + vector hybrid retrieval support
• ✅ Built-in vectorization module (optional)
• ❌ Higher resource consumption
• ❌ Steep learning curve

pgvector — A Blessing for PostgreSQL Users

-- Install extension in PostgreSQL
CREATE EXTENSION vector;
 
-- Create table with vector column
CREATE TABLE documents (
    id SERIAL PRIMARY KEY,
    content TEXT,
    embedding vector(1024),
    category VARCHAR(50)
);
 
-- Create HNSW index
CREATE INDEX ON documents USING hnsw (embedding vector_cosine_ops);

from langchain_community.vectorstores import PGVector
 
vectorstore = PGVector(
    connection_string="postgresql://user:pass@localhost/db",
    embedding_function=embeddings,
    collection_name="docs",
)

• ✅ Seamless integration with existing PG databases
• ✅ Full SQL expressiveness
• ✅ Transaction support (ACID)
• ❌ Vector retrieval performance lower than dedicated databases
• ❌ PostgreSQL itself becomes bottleneck at large scale

Pinecone — For Those Who Want Zero Ops

from langchain_pinecone import PineconeVectorStore
from pinecone import Pinecone
 
pc = Pinecone(api_key="your-key")
index = pc.Index("docs")
vectorstore = PineconeVectorStore(index=index, embedding=embeddings)

• ✅ Fully managed, zero operations
• ✅ Auto-scaling
• ✅ Good metadata filtering support
• ❌ Higher price
• ❌ Data lock-in (high migration cost)

Practical: Chroma (Development) vs Qdrant (Production)

Scenario

Let's build RAG for a technical blog system:

• Development: Use Chroma for quick validation, local execution
• Production: Migrate to Qdrant, supporting multi-tenancy and metadata filtering

Development Phase — Chroma

from langchain_chroma import Chroma
from langchain_openai import OpenAIEmbeddings
 
embeddings = OpenAIEmbeddings(
    model="BAAI/bge-large-zh-v1.5",
    api_key=os.getenv("SILICONFLOW_API_KEY"),
    base_url="https://api.siliconflow.cn/v1",
    chunk_size=32,
)
 
# Create local vector store
vectorstore = Chroma.from_documents(
    documents=chunks,
    embedding=embeddings,
    persist_directory="./chroma_db",
    collection_metadata={"hnsw:space": "cosine"}
)
 
# Search
results = vectorstore.similarity_search("microservices decomposition principles", k=3)
for doc in results:
    print(f"Source: {doc.metadata['source']}")
    print(f"Content: {doc.page_content[:100]}...")
    print()
 
# Persistence (Chroma auto-saves)
# Next time load with:
# vectorstore = Chroma(persist_directory="./chroma_db", embedding_function=embeddings)

Production Phase — Qdrant

from langchain_qdrant import Qdrant
from qdrant_client import QdrantClient, models
 
# Connect to Qdrant service
client = QdrantClient(url="http://localhost:6333")
 
# Create Collection (like a database table)
client.create_collection(
    collection_name="blog_docs",
    vectors_config=models.VectorParams(
        size=1024,  # BGE-large-zh dimensions
        distance=models.Distance.COSINE,
    ),
)
 
# Write data
vectorstore = Qdrant(
    client=client,
    collection_name="blog_docs",
    embeddings=embeddings,
)
 
vectorstore.add_documents(documents=chunks)
 
# Search with metadata filtering
results = vectorstore.similarity_search(
    query="microservices monitoring",
    k=5,
    filter=models.Filter(
        must=[
            models.FieldCondition(
                key="category",
                match=models.MatchValue(value="microservices")
            ),
            models.FieldCondition(
                key="year",
                range=models.Range(gte=2024)
            ),
        ]
    )
)

Migrating from Chroma to Qdrant

# 1. Export from Chroma
chroma_store = Chroma(
    persist_directory="./chroma_db",
    embedding_function=embeddings
)
all_docs = chroma_store.get()
 
# 2. Import to Qdrant
qdrant_store = Qdrant(
    client=qdrant_client,
    collection_name="blog_docs",
    embeddings=embeddings,
)
 
# 3. Batch write (Qdrant supports efficient batch import)
from langchain_core.documents import Document
 
docs = [
    Document(page_content=text, metadata=meta)
    for text, meta in zip(all_docs["documents"], all_docs["metadatas"])
]
qdrant_store.add_documents(docs)

How to Choose a Similarity Algorithm?

When comparing two vectors, the database needs a "distance metric." Three common ones:

Cosine Similarity

cosine(A, B) = (A · B) / (||A|| × ||B||)

• Measures: Cosine of the angle between two vectors
• Characteristics: Only cares about direction, not magnitude
• Best for: Text semantic similarity (most common choice)
• Range: -1 (opposite) to 1 (identical), typically > 0.7 is similar

Dot Product

dot(A, B) = A · B = Σ(Ai × Bi)

• Measures: Sum of element-wise products
• Characteristics: Considers both direction and magnitude
• Best for: Recommendation systems (preference intensity matters)
• Note: If vectors aren't normalized, dot product is affected by vector length

Euclidean Distance

euclidean(A, B) = √Σ(Ai - Bi)²

• Measures: Straight-line distance between two points
• Characteristics: Absolute distance, sensitive to numerical differences
• Best for: Image retrieval, numerical feature scenarios
• Range: 0 (identical) to ∞, smaller is more similar

Selection Guide

⚠️ Important: The embedding model and similarity algorithm must match! BGE models recommend cosine similarity, and OpenAI text-embedding-3 series also recommend cosine similarity.

Metadata Filtering: From "Needle in a Haystack" to "Precise Targeting"

Why Metadata Filtering?

Suppose your knowledge base has 100,000 documents covering tech, product, operations, and sales. A user asks: "What are this year's sales targets?"

Pure vector retrieval might return:

• ✅ Sales department 2024 target document
• ❌ Operations document mentioning "sales" workflows
• ❌ Technical document about "sales system architecture"

Adding metadata filter {"department": "sales", "year": 2024} precisely scopes the search.

Metadata Filtering in LangChain

from langchain_chroma import Chroma
 
# Write with metadata
docs = [
    Document(
        page_content="2024 sales target: 30% revenue growth...",
        metadata={"department": "sales", "year": 2024, "type": "target"}
    ),
    Document(
        page_content="Sales system uses Redis caching...",
        metadata={"department": "tech", "year": 2024, "type": "architecture"}
    ),
]
 
vectorstore = Chroma.from_documents(docs, embeddings)
 
# Search with filter
results = vectorstore.similarity_search(
    "sales targets",
    k=3,
    filter={"department": "sales", "year": 2024}
)

Qdrant Advanced Filtering Expressions

from qdrant_client import models
 
filter = models.Filter(
    must=[  # AND conditions
        models.FieldCondition(key="department", match=models.MatchValue(value="sales")),
        models.FieldCondition(key="year", range=models.Range(gte=2024)),
    ],
    should=[  # OR conditions
        models.FieldCondition(key="type", match=models.MatchValue(value="target")),
        models.FieldCondition(key="type", match=models.MatchValue(value="summary")),
    ],
    must_not=[  # NOT conditions
        models.FieldCondition(key="status", match=models.MatchValue(value="draft")),
    ]
)

Selection Summary

By Scenario

Migration Path from Dev to Production

Phase 1: Development Validation
    └── Chroma (local embedded, zero config)
            ↓
Phase 2: Testing Environment
    └── Qdrant Docker (single node, validate functionality)
            ↓
Phase 3: Production Launch
    └── Qdrant Cluster / Pinecone Managed (high availability)

Summary

This article covered the core knowledge of vector databases:

1. Why vector databases are needed — ANN retrieval is something traditional databases can't do
2. Three core capabilities — storage, ANN retrieval, metadata filtering
3. Five database comparison — Chroma, Qdrant, Weaviate, pgvector, Pinecone
4. Practical code — complete examples for Chroma development and Qdrant production
5. Similarity algorithms — choosing among cosine, dot product, and Euclidean distance
6. Metadata filtering — from "needle in a haystack" to "precise targeting"

Key Insight: The best vector database isn't the most expensive one — it's the one that fits your scenario. Use Chroma for development, Qdrant for production, pgvector if you have PostgreSQL, Pinecone if you want zero ops. No silver bullet, only the right fit.

References

• Qdrant Official Documentation
• Chroma Official Documentation
• pgvector GitHub
• Weaviate Documentation
• Pinecone Documentation
• ANN Algorithm Overview: HNSW vs IVF