I’ve been benchmarking vector databases for months now, because keeping semantic search realtime at scale is one of those problems that sounds simple until you try to run it on real traffic. In this article I walk through the tests I ran comparing Qdrant, Milvus, and Pinecone focusing on three things teams actually care about: throughput under load, consistency (search result fidelity and staleness), and total cost to serve a production-like workload.
Why these three?
Qdrant and Milvus are the two open-source heavyweights people deploy themselves (or via managed offerings), while Pinecone is the mature SaaS vector DB with lots of production references. Each has different trade-offs: control and cost for self-hosting vs. convenience and operational predictability for SaaS. I wanted to compare them in realistic scenarios, not just toy queries.
Test environment & methodology
I designed the tests to reflect a typical semantic search pipeline: a document store of embeddings, periodic ingestion, and low-latency nearest-neighbor queries from an application. Key points:
Dataset: 5 million 1536-d embeddings (OpenAI/GPT-style), plus metadata for simple filtering (language, doc_type).Query workload: 2000 QPS sustained for 10 minutes with Poisson arrival and 95/5 read/write ratio (writes are incremental document inserts/updates to simulate freshness).Metrics: p50/p95/p99 latency for queries, successful queries per second (throughput), index update visibility latency (consistency), and total cost over a month estimated for a baseline cluster size that can handle the workload with headroom.Filters: we included metadata filters on 30% of queries to test index + metadata performance.Hardware: for self-hosted I used cloud VMs (8 vCPUs, 64 GB RAM) scaled into clusters; for Pinecone I chose comparable production-tier nodes. All nodes were in same region to avoid network variance.Index types: HNSW for approximate NN (default for many), IVF_FLAT for exact/near-exact scenarios where supported.I repeated each test three times and report averages. I intentionally avoided micro-benchmarks (single-query latency) because at scale interference, GC, indexing, and network matter more.
Throughput & latency
Here are the observed latencies and sustained throughput for the 2000 QPS workload with 5M vectors. The numbers are rounded to make them readable; they reflect the cluster sizes I needed to attain stable throughput without losing queries.
| Database | Cluster | Sustained QPS | p50 latency (ms) | p95 latency (ms) | p99 latency (ms) |
| Qdrant (HNSW, 3-node) | 3 × 8vCPU/64GB | 2,000 | 12 | 48 | 120 |
| Milvus (IVF+PQ, 4-node) | 4 × 8vCPU/64GB | 2,000 | 10 | 55 | 180 |
| Pinecone (s1-standard equivalent) | Managed (3 pods) | 2,000 | 9 | 40 | 95 |
Notes:
Pinecone delivered the lowest tail latency and the most stable p99 in the tested configuration. That's the advantage of a managed service with specific optimizations for low-tail latency.Qdrant was competitive in median latency and did well for filtered queries, but saw higher tail spikes under heavy concurrent writes (indexing + search concurrency).Milvus offered great median latency when using efficient compression/indexing (IVF+PQ), but tail latency suffered more. Milvus also needed a larger cluster in my tests to smooth IO during inserts.Consistency & freshness
Consistency is often overlooked. For semantic search, a stale index means users won’t see recently added documents — a problem for news, chat logs, or rapidly-changing catalogs.
Pinecone: strong near-real-time consistency. Document inserts were visible in queries within 200–400 ms on average. Pinecone supports streaming index updates with near-instant visibility in my tests.Qdrant: default behavior showed visibility in ~500 ms to 2s depending on HNSW parameters and indexing threads. With tuned settings (increased num_threads and smaller write batches) I got sub-second visibility but at higher CPU cost.Milvus: consistency varied based on flushing and index build settings. With async flush and batch inserts I saw 2–5s visibility; switching to synchronous flushes brought that down to 400–800 ms but reduced write throughput.In short: if your app requires sub-second visibility for new documents, Pinecone provides this out of the box. Qdrant and Milvus can match that with tuning, but you trade CPU and write throughput.
Operational experience
Operational burden matters a lot for teams. I ran failure tests (node restarts, network partition within the cluster) and observed behavior:
Pinecone: handled failures gracefully; automatic rebalancing and routing kept queries alive with minor latency bumps. No ops required on my side.Qdrant: node restarts required manual rebalancing depending on replication settings. Cloud-managed Qdrant services reduce this pain, but self-hosting requires solid orchestration (Kubernetes + readiness probes).Milvus: recovery is robust but can be slower when indexes need rebuilding; depending on index type (IVF/PQ) rebuild time could be significant. Milvus recommends replica strategies and redundant storage to mitigate.Cost estimates (monthly)
Costs depend heavily on region, workload pattern, and whether you use managed or self-hosted deployments. These are conservative, illustrative numbers for a production cluster matching the throughput above. Prices are approximate and expressed as USD equivalent.
| Option | Estimated monthly infra cost | Other costs | Total est. monthly |
| Qdrant self-hosted (3 nodes) | $1,100 (3 × $370) | Ops (~$800 for 1 full-time engineer fraction or managed service premium) | $1,900 |
| Milvus self-hosted (4 nodes) | $1,400 (4 × $350) | Ops (~$1,000), storage I/O premiums | $2,400 |
| Pinecone managed | $2,200 (managed pods, SLA) | Included ops, support tiers optional | $2,200 |
Interpretation:
Self-hosting can be cheaper on pure infra cost (Qdrant edge), but you must account for engineering time and ops complexity. If you value predictable staffing and 24/7 reliability, Pinecone’s managed price can be justified.Milvus often requires more memory/storage IO in practice, increasing infra costs. Its managed offerings reduce operational costs but add to SaaS bill.When to pick which
I’ll keep this pragmatic — pick based on concrete constraints:
Choose Pinecone if you want a hassle-free managed service with predictable tail latency and sub-second consistency without dedicating ops time. Good for fast product iterations, startups without ops bandwidth, or teams needing guaranteed SLAs.Choose Qdrant if you want more control, prefer open-source, and have ops capacity. Qdrant is great for use cases that need filtering, flexible storage backends, and lower infra costs when managed correctly.Choose Milvus if you need advanced indexing options, compression, and you’re optimizing for storage efficiency across very large corpora. Milvus is a strong choice if you have experienced infra engineers who can tune index types and manage rebuild times.Practical tuning tips I used
For HNSW: increase ef_construction for better recall, but be mindful of index build time and memory footprint. For production queries increase ef_search for better recall at the cost of query latency.Batch inserts and tune flush intervals to balance write throughput and freshness. Small batches and synchronous flushes improve freshness but hurt throughput.Use hybrid strategies: store dense vectors in the vector DB, but use a secondary fast filter service (Redis/Elasticsearch) for heavy boolean filtering if metadata is complex.Monitor p99 and tail latencies — they’re predictive of user experience more than p50.If you want, I can share the exact deployment scripts, index parameters, and the synthetic workload generator I used so you can reproduce these tests on your own region and hardware. I also have the raw results and graphs from the runs if you prefer visualizations over numbers.
