Deploy Qdrant and compare operational characteristics with pgvector

Introduction

Engineers often evaluate vector database options by reading vendor benchmarks that rarely reflect their own workload characteristics. Qdrant and pgvector represent two fundamentally different architectural choices: Qdrant is a purpose-built vector store with dedicated HNSW index management and segment compaction, while pgvector layers vector search onto a relational database with its own index maintenance semantics. By the end of this lesson, you will deploy a production-ready Qdrant instance on Kubernetes and build a benchmark framework that measures real query latency, throughput, and resource consumption side-by-side with pgvector — giving you the data to make an informed architectural decision for vector index maintenance operations.

Key Terminology

• Segment compaction — Qdrant's background process that merges storage segments to reduce read amplification during sustained write workloads; controlled at deploy time via the optimizer_cpu_budget Helm setting.
• HNSW (Hierarchical Navigable Small World) — the graph-based approximate nearest-neighbor index algorithm Qdrant manages natively; Qdrant owns the full index lifecycle (construction, compaction, and eviction) independently of any relational engine.
• WAL-backed index architecture — pgvector's approach to vector index maintenance, where index updates flow through PostgreSQL's write-ahead log; index maintenance semantics are inherited from the relational engine rather than purpose-built for vector workloads.
• Read amplification — the performance penalty that occurs when uncompacted segments force a query to scan multiple storage units instead of one; the optimizer_cpu_budget=2 value in deploy_qdrant.py is calibrated to keep compaction fast enough to avoid this during sustained writes.
• Latency percentile (p50 / p95 / p99) — the per-percentile query latency fields in BenchmarkResult; p95 and p99 are the operationally critical values because compaction stalls and reindex spikes appear there long before they degrade median (p50) latency.
• Idempotent deployment — a deploy pattern where re-running the same command produces the same cluster state without side effects; helm upgrade --install with --wait --timeout 300s achieves this so benchmark runs never start against a partially initialized Qdrant index.

Concepts

Two Architectures, Two Maintenance Models

Qdrant and pgvector are not simply fast versus slow — they represent different design philosophies about who owns vector index maintenance. Qdrant is purpose-built: it controls its own HNSW segment lifecycle, compaction scheduling, and memory-mapped file layout from the ground up. pgvector is additive: it grafts approximate nearest-neighbor search onto PostgreSQL's existing storage engine, which means index maintenance is constrained by WAL semantics, MVCC visibility, and autovacuum scheduling that were designed for row data, not high-dimensional vectors.

This distinction matters operationally because the failure modes are different. Under write pressure, Qdrant's uncompacted segments accumulate until the optimizer merges them — read amplification grows until compaction catches up. pgvector's index degrades in a different shape: bloat accumulates in heap pages alongside dead tuples, and VACUUM contends with live queries for the same I/O budget. Neither system is universally better; the right choice depends on your write pattern, dataset size, and query latency SLO.

Why Vendor Benchmarks Are Not Your Benchmark

Published benchmarks are typically run on purpose-configured hardware with synthetic workloads optimized for the vendor's strengths. They tell you the ceiling, not what you will observe on your actual query distribution, dataset size, or Kubernetes node class. The benchmark framework in this lesson — BenchmarkResult paired with ComparisonReport — is designed specifically to replace vendor numbers with measurements taken against your own cluster, your own dataset, and your own operation mix (see Code Walkthrough).

The recommendation field in ComparisonReport is intentionally left for you to populate after analysis. No framework can encode the right tradeoff between operational complexity, team familiarity, and latency profile — but it can give you the data to make that call on evidence rather than marketing.

Reading p95 and p99 Before p50

Median latency hides the operational story. Compaction events in Qdrant and reindex cycles in pgvector are bursty: they inflate tail latency at p95 and p99 while leaving p50 nearly unchanged. A system that looks equivalent at the median can be meaningfully worse in practice if 5% of queries are 10× slower than the rest — which is exactly the regime that breaks SLOs and causes client-side retries to pile up.

BenchmarkResult captures all three percentiles for this reason. When you compare systems, start with p99 to see compaction and reindex spikes, then work down to p50 to understand baseline query cost. The gap between p50 and p99 is the operational variance you are buying or avoiding with each architectural choice.

Deployment as Reproducible Infrastructure

The deploy_qdrant.py script treats deployment as a reproducible, auditable step rather than a one-time click. Using helm upgrade --install (idempotent) with explicit resource requests and limits means the same command is safe to re-run in CI, staging, and production without manual cleanup. The --wait --timeout 300s flag enforces synchronous readiness — subsequent benchmark code never races against pod initialization (see Code Walkthrough).

The optimizer_cpu_budget=2 value is a concrete example of a deployment decision that has measurable benchmark consequences: too low and compaction lags behind writes, increasing p99 latency; too high and query threads are starved on write-heavy nodes. Benchmarking across different values of this setting is a natural extension of the framework built here.

Code Walkthrough

Now that you understand how Qdrant's segment-based HNSW storage model differs from pgvector's WAL-backed index architecture, you can trace both design decisions directly through the deployment and benchmark code.

The first step is to provision Qdrant into your cluster. The script below uses helm upgrade --install for idempotent deployments, sets memory-mapped file resources appropriate for datasets up to 1M vectors, and enables a Prometheus serviceMonitor for the performance monitoring work covered later in this chapter.

Code snippetpython
1# scripts/deploy_qdrant.py
2import subprocess
3import sys
4
5def deploy_qdrant(namespace: str = "vector-ops") -> None:
6    """Deploy Qdrant with persistent storage."""
7    helm_args = [
8        "helm", "upgrade", "--install", "qdrant",
9        "qdrant/qdrant",
10        "--namespace", namespace,
11        "--create-namespace",
12        "--set", "persistence.size=50Gi",
13        "--set", "resources.requests.memory=2Gi",
14        "--set", "resources.requests.cpu=1000m",
15        "--set", "resources.limits.memory=4Gi",
16        "--set", "resources.limits.cpu=2000m",
17        "--set", "config.storage.performance.optimizer_cpu_budget=2",
18        "--set", "metrics.serviceMonitor.enabled=true",
19        "--wait", "--timeout", "300s",
20    ]
21    result = subprocess.run(helm_args, capture_output=True, text=True)
22    if result.returncode != 0:
23        print(f"Qdrant deploy failed: {result.stderr}", file=sys.stderr)
24        sys.exit(1)
25    print("Qdrant deployed successfully")
26
27if __name__ == "__main__":
28    deploy_qdrant()

The optimizer_cpu_budget setting controls how many cores Qdrant dedicates to background segment compaction. Setting it to 2 keeps compaction fast enough to avoid read amplification during sustained write workloads without starving query threads. The --wait --timeout 300s flags make the deploy step synchronous so that subsequent benchmark runs do not start against a partially initialized index.

With the cluster ready, the benchmark framework captures what actually differs at runtime. BenchmarkResult stores latency percentiles (p50, p95, p99), throughput, and resource consumption for a single run. ComparisonReport pairs results from both systems and carries a recommendation field you populate after analysis.

Code snippetpython
1# src/vector_index_ops/benchmark.py
2from dataclasses import dataclass, field
3from datetime import datetime
4
5@dataclass
6class BenchmarkResult:
7    system: str
8    dataset_size: int
9    operation: str
10    p50_ms: float
11    p95_ms: float
12    p99_ms: float
13    throughput_per_second: float
14    memory_mb: float
15    storage_mb: float
16    timestamp: datetime = field(default_factory=datetime.utcnow)
17
18@dataclass
19class ComparisonReport:
20    pgvector_results: list[BenchmarkResult]
21    qdrant_results: list[BenchmarkResult]
22    recommendation: str
23    details: dict = field(default_factory=dict)

Capturing all three latency percentiles matters because compaction stalls and reindex spikes appear in p95/p99 long before they affect median latency — relying on p50 alone would mask the operational differences between the two systems that this chapter is designed to surface.

Verify by running python scripts/deploy_qdrant.py against a staging namespace and confirming the pod reaches Running state, then instantiate a BenchmarkResult for a sample query operation and confirm all numeric fields populate without type errors.

Do's and Don'ts

Having walked through the material above, the following Do's and Don'ts distill it into practice.

Do's

1. ✓Do include --wait --timeout 300s in the helm upgrade --install command — these flags make the Qdrant deployment synchronous, ensuring the HNSW segment index is fully initialized before deploy_qdrant.py returns; launching benchmark runs against a partially initialized index produces latency numbers that understate steady-state query cost and corrupt the comparison with pgvector.
2. ✓Do record p50, p95, and p99 latency in every BenchmarkResult — Qdrant's background segment compaction and pgvector's WAL-backed index maintenance both generate spikes that appear in p95/p99 long before they move the median; a comparison that reads only p50_ms will show the two systems as nearly identical even when one is causing sustained tail-latency degradation under write load.
3. ✓Do set optimizer_cpu_budget deliberately for your read/write mix — this Helm value controls how many cores Qdrant dedicates to background segment compaction; too low allows read amplification to accumulate during sustained writes, too high starves query threads and inflates latency during compaction windows, and either extreme will skew your ComparisonReport away from steady-state reality.

Don'ts

1. ✗Don't populate ComparisonReport.recommendation based on vendor benchmarks instead of your own BenchmarkResult data — the lesson's core premise is that published benchmarks rarely reflect real workload characteristics; the recommendation field is meaningful only after you have measured p50/p95/p99, throughput_per_second, memory_mb, and storage_mb against your actual dataset size and query patterns.
2. ✗Don't omit storage_mb from the side-by-side comparison — Qdrant's segment-based HNSW layout and pgvector's WAL-backed index have fundamentally different on-disk footprints; a latency-only comparison misses the storage cost differential that drives capacity planning and is exactly what the BenchmarkResult.storage_mb field exists to capture.
3. ✗Don't deploy Qdrant without metrics.serviceMonitor.enabled=true — skipping this Helm flag means Prometheus never scrapes Qdrant's built-in query latency and compaction metrics, breaking the performance monitoring pipeline that the rest of this chapter builds on top of; the absence is silent at deploy time but leaves you with no runtime signal for the monitoring work ahead.