Database Topic Archive
HNSW, IVFFlat, recall, embedding search, and production RAG performance notes.
Today's vendor news for database operators: Aurora DSQL gets per-statement CloudWatch Database Insights, ARC Region switch learns to orchestrate RDS read-replica switchovers, AWS walks through multilingual FTS migration to PostgreSQL, AlloyDB's ScaNN index hits 10 billion vectors, and Timestream for…
InnoDB FULLTEXT and PostgreSQL tsvector both promise search without a search engine. Here is how indexing, ranking, and language support actually compare, and when neither is enough.
Vector search observability needs recall, result count, filter selectivity, freshness lag, tenant skew, index health, reranker cost, and answer grounding - not just latency.
A RAG system is unsafe if deleted documents, permission changes, and stale embeddings can still appear in answers. Freshness is part of correctness.
Vector cost is not just storage. Dimensions, top_k, filters, rerankers, embedding refreshes, payload size, index rebuilds, and tenant skew all compound.
The best vector database depends on ownership boundaries, filter semantics, recall targets, migration paths, cost, and operational maturity - not benchmark screenshots.
Embedding model upgrades are migrations. Versioned embeddings, dual indexes, shadow queries, backfills, and rollback decide whether search quality survives.
Rerankers can improve answer quality, but they cannot recover evidence that retrieval never found. Use them after recall, cost, and latency are understood.
A fast RAG system can still be wrong. Production teams need recall@k, MRR, answerability, citation coverage, freshness, no-hit rate, and drift signals.
Chunk size and overlap decide retrieval quality, citation accuracy, freshness, permissions, and cost. Treat chunks as serving data with ownership.