ParadeDB's Postgres COUNT fix hits 5,353 QPS, and it matters for AI retrieval

ParadeDB published the second half of its “Opening a Closed Tin” series on October 13, and the number at the top is the one to remember: 5,353.3 queries per second on a Wikipedia COUNT benchmark, against 2,645.8 for TIN, the Postgres extension it is chasing. That is a clean 2.0x. The engineering post by Ming Ying is nominally about making COUNT(*) fast in Postgres. Read it through an AI lens and it is about something more useful: the retrieval layer underneath every RAG pipeline, and how badly benchmark design can distort which engine looks best.

The headline result is a doubling. The more interesting numbers come later, when ParadeDB moves off the benchmark TIN designed and onto Hacker News data with real filters. Filtered count with a score > 10 clause: 777.3 QPS for ParadeDB versus 77.6 for TIN, a 10.0x gap. Filtered Top K by BM25 relevance: 820.4 versus 70.5, or 11.6x. Top K ordered by timestamp: 377.8 versus 17.4, a 21.7x difference. Those are not small-margin wins. They are the shape of query that production retrieval actually runs.

Three optimizations, one theme

The COUNT work rests on three changes, and each is worth understanding because each addresses a different real-world failure.

First, segment partitioning. ParadeDB stores data in an LSM-like structure of immutable segments merged at size thresholds. The old merge strategy was blind to contents, so a segment’s ctid (Postgres row identifier) range could span most of the table. The new strategy, based on the Spooky LSM compaction paper, partitions segments by column value. Partition by ctid and each segment gets a disjoint range of Postgres page numbers. That lets the engine check the visibility map once per segment and skip per-row visibility checks for whole blocks. Visibility-related page accesses for the benchmark query fell from 1,382 to 30.

Second, inverting the DocId map. The old approach stored a ctid for every document and fetched it per match. Counting n matches meant n lookups. The new representation maps Postgres page numbers to the first DocId on that page, exploiting the fact that segments are already sorted by ctid. After updating 1,000 rows, buffer accesses dropped from over 7,500 to just under 400, a 95% reduction.

Third, bitmaps for dense postings. Tantivy, the search library underneath, encodes posting lists as BP128 blocks. That is efficient for rare terms. For terms matching more than 10% of documents, ParadeDB now switches to a bitmap representation, where bitwise AND/OR processes 64 documents at a time. This single change halved p99 latency on the Wikipedia count benchmark.

The benchmark argument is the real story

ParadeDB is unusually candid about the thing most vendors bury. “The workload you pick often determines the story you tell,” Ying writes. The company credits PlanetScale with designing a benchmark where TIN excelled, and says that benchmark “exposed blind spots in our BM25 and count performance and rigorously drove optimizations.” Then it turns: TIN’s Wikipedia and StackExchange workloads contain a single indexed text column. Real application tables are wider, filtered by metadata, ordered by timestamps.

That is the sentence AI builders should tape to the wall. When you evaluate a retrieval engine for a RAG stack, the benchmark you read is almost never your workload. Your documents carry tenant_id, created_at, source_type, access_level. Your queries filter on those before they rank. The 10x to 21.7x gaps above come from exactly that pattern.

No single benchmark can crown a search engine, especially one designed by the team building the product. ParadeDB says so about its own numbers.

ParadeDB goes further than most vendors would: “we do not believe it establishes a ‘fastest search engine,’ on Postgres or otherwise. No single benchmark can, especially one designed by the same team building the product. That applies to ParadeDB’s Hacker News benchmark” too. That admission is rarer than the performance gain and worth more.

Why an AI newsroom cares about a Postgres extension

The AI angle here is not incidental. Retrieval is the part of a RAG system that quietly sets the ceiling on answer quality and cost. Vector databases get the attention, but a large share of production AI search runs hybrid: BM25 plus filters plus metadata, sitting next to embeddings in the same Postgres instance the application already operates. If COUNT(*) and filtered Top K are slow, the pipeline pays for it on every request, and the bill shows up as latency and dollars, not as a failed demo.

The numbers also expose a benchmarking gap the AI tooling market has not closed. Vendors publish QPS on clean, single-column corpora because those numbers are flattering and reproducible. Buyers need the opposite: numbers on wide tables with the filters they actually run. ParadeDB’s post is a rare case of a vendor publishing its own weakness and then showing the fix.

What to watch

Two things. First, ParadeDB says the segment partitioning feature is “to be announced” and promises a technical deep dive soon. Until that ships and is independently tested, the 5,353 QPS figure is a vendor benchmark on a vendor harness, and the company says as much. Second, the real test is whether TIN responds on filtered and time-sorted queries, which the post concedes “isn’t very fast at these filtered and time-sorted queries yet.” A competitive answer there would be good for everyone running retrieval on Postgres.

For now, the practical takeaway for anyone building on Postgres search: benchmark your own table, with your own filters, before you pick an engine. The 21.7x gap on time-sorted Top K did not appear on the clean corpus. It appeared the moment the query looked like production.