Qdrant 1.19 Per-Tenant IDF: Ranking Changes, Not nDCG

Per-tenant IDF(opens in a new tab) is a Qdrant 1.19 search parameter that scopes BM25 inverse-document-frequency statistics to a payload filter, so term rarity is computed on one tenant’s documents instead of the whole shard.¹ A payload filter on forum_tag keeps tenant A from seeing tenant B’s documents. It does not keep tenant B’s vocabulary out of tenant A’s BM25 scores.

Sparse search with the IDF modifier computes two numbers per term: how many documents exist (N) and how many contain the term (df).² Until 1.19, both were shard-wide. A retrieval filter hid other tenants’ points from the result list. It did not hide them from the rarity table. The request showed up as qdrant#9445(opens in a new tab) before the release: filtered sparse search should recompute IDF on the surviving documents.³

We ran the full official BEIR CQADupStack(opens in a new tab) corpus against qdrant/qdrant:v1.19.0 on a single node: 457,199 documents, 12 StackExchange forums as tenants, 13,145 labeled queries.⁴ Two findings matter, and they answer different questions:

1. Per-tenant IDF reproduces theoretically correct tenant-local BM25. Global IDF does not. Top-1 changes on 19.9% of tenant-filtered queries.
2. On official duplicate-question labels, macro nDCG@10 moves +0.19 percentage points. That is not a leaderboard jump. Report the ranking-change rate, not a fake SOTA delta.

Qdrant per-tenant IDF is not the retrieval filter

Qdrant’s 1.19 release notes(opens in a new tab) describe the feature in one paragraph.⁵ The docs are more precise: params.idf.corpus accepts a payload filter that scopes the population used for N and df. That filter is not the retrieval filter.

The documented pattern is broader IDF, narrower retrieval. Score rarity over a whole tenant. Restrict hits further by year, status, or collection subset:

from qdrant_client import QdrantClient, models

client = QdrantClient(url="http://localhost:6333")

tenant = models.Filter(
    must=[
        models.FieldCondition(
            key="forum_tag",
            match=models.MatchValue(value="tex"),
        )
    ]
)

# q_ind / q_val: FastEmbed Qdrant/bm25 sparse indices and values
client.query_points(
    collection_name="cqadupstack_bm25",
    query=models.SparseVector(indices=q_ind, values=q_val),
    using="bm25",
    query_filter=tenant,  # who can appear in the result list
    search_params=models.SearchParams(
        idf=models.IdfCorpusParams(corpus=tenant),  # who counts in N and df
    ),
    limit=10,
)

Omit search_params.idf and you get global: shard-wide statistics, the pre-1.19 default. Send a retrieval filter without an IDF corpus and you still score against the mixed vocabulary. That is the footgun.

If you already isolate identity at a query gateway, this is a second injection site. Our QQL Gateway covers tenant filters on the retrieval AST. AST filter injection is the same job for the IDF corpus: a second filter, a second place to forget it. Inject one and skip the other, and BM25 still prices terms against the wrong population.

Why CQADupStack is the right corpus

CQADupStack is twelve StackExchange sites sharing one sparse collection. The feature is for that layout: one collection, payload isolation, IDF that should not leak across tenants. Creating twelve collections would have given each forum its own N and df for free, and would have tested nothing.

Query vectors are identical in both modes. We used FastEmbed Qdrant/bm25, the same sparse family as in-process QQL Edge search. Sparse inverted-index retrieval is exact. No HNSW. The only difference is the IDF corpus.

Ground truth for correctness is Okapi BM25 scored in Python over that tenant’s stored sparse vectors, using tenant-local (N, df) and the Okapi formula Qdrant documents:

IDF(q) = ln(1 + (N - df + 0.5) / (df + 0.5))

nDCG@10 in the correctness section is agreement with that ideal ranking, not relevance.

Ground truth for quality is the official duplicate-question qrels, scored the way Thakur et al., 2021(opens in a new tab) report CQADupStack: nDCG@10, MAP, Recall@100, then a macro-average over the twelve forums.⁶

Numeric ids collide across forums. We assigned qrels by walking the HuggingFace dump in alphabetical forum order, which recovers the published 23,703 judgments / 13,145 topics exactly.

Same word, different rarity

BM25 pays for rare terms. “Rare” is a corpus statistic. Mix twelve vocabularies and the statistic lies.

A single-term probe on the tex tenant matches Okapi to six decimals in both modes, so Qdrant is implementing the formula it published:

global : N=457199  df=218876  -> implied 0.736613  predicted 0.736613
tenant : N=68184   df=42213   -> implied 0.479484  predicted 0.479485

The distortion shows up on tenant jargon. The same token is common inside its home forum and rare in the mixed shard, so global IDF overweights it:

Tenant-only stems (natbib, fontspec, skyrim, lagrangian) never appear outside their forum. Global stats still misprice them because N_global is 6.7× larger than the largest tenant. The term is rare in the shard for a boring reason: the other eleven tenants never talk about it. Inside the tenant that does, it may be ordinary.

Correctness: tenant-mode is the dedicated-collection ranking

2,999 queries, 250 per forum, BEIR query texts. Tenant-mode nDCG vs the Python ideal is 1.0000 on 2,996 of 2,999 queries. The three misses are short queries with 1-9 candidate docs (ties or empty overlap), not IDF bugs.

Mean nDCG loss for global IDF is 1.65 percentage points against the ideal. Tenant-mode is better on 2,895 queries. Global-mode is better on zero.

0.9825 still looks high if you only read the mean. The operational number is the ranking shift. The mean hid it.

GIS, webmasters, wordpress, and android flip the first hit on roughly a quarter of queries. Mean top-10 overlap is 0.785, so about two of the ten slots move. The histogram of per-query nDCG gaps is one-sided. Global IDF never wins.

If the question is “does 1.19 implement tenant-local BM25 inside a shared collection?”, the answer is yes.

Labeled quality: do not quote this as a retrieval win

Full official test set: 13,145 queries. Published BM25 nDCG@10 on CQADupStack is 0.299. Our FastEmbed BM25 lands at 0.295 under global IDF, same tokenizer family, so the pipeline is in the public ballpark.

Mean labeled nDCG@10 lift is +0.19 pp macro. Android (+1.15 pp) and english (+1.08 pp) gain about a point. Gaming, physics, and wordpress lose a fraction of a point.

The lift chart is the one to keep. Per-query, tenant-mode nDCG@10 is higher on 920 queries and lower on 1,008. The mean still ticks up because the wins on android and english are larger than the losses. A mean that hides a split like that is how blog posts accidentally become press releases.

Top-1 still changes on 18.5% of the 13,145 queries (2,433 flips). Duplicate-question labels are sparse (mean 1.80 relevants per query). A different first hit often does not move nDCG. The user still sees it.

This is also why retrieval regression checks should assert document IDs, not only a mean metric. A 0.2 point nDCG move can sit on top of a 20% top-1 churn.

Latency stays in the low milliseconds

1,200 queries, 20 timed reps each, after warmup. Client-side wall time, same query, same payload filter, only params.idf changes.

Overhead of scoped statistics: +0.89 ms p50 (+38% relative), +1.02 ms p95. Absolute cost stays small on 457k docs / 1 shard. The relative jump looks large because the baseline is already 2.3 ms. Saving 0.9 ms on a keyword search that currently scores the wrong corpus is not a reason to skip it.

Edge cases match the docs

We hit the documented failure modes, not surprises.

1. idf on a sparse vector without the IDF modifier is rejected with HTTP 400. The parameter is not a no-op on TF-only vectors.
2. A corpus filter that matches zero points does not fall back to shard-wide stats. Every term gets a constant weight. Ranking degenerates to plain TF. Two different empty corpora produce identical scores. A rare-term document that ranked on a valid corpus can drop out of the top-5.

Index the fields you put in the IDF filter. On Qdrant Cloud, strict mode rejects filters on unindexed fields by default.

User-defined sharding is a different story. Routing a search to a tenant’s dedicated shard already scopes IDF to that shard’s data. The idf filter does not reach across shards. If the corpus filter matches points that live elsewhere, Qdrant silently computes statistics from whatever overlap exists locally, which can be empty. We tested one shard, so that path is a docs warning, not a number in this run.

When to turn it on

Turn it on when all of these are true:

• One collection holds many tenants.
• Isolation is a payload filter (with is_tenant=true on the keyword index), not a dedicated shard per tenant.
• You search sparse vectors with modifier=IDF (BM25 or miniCOIL).
• Tenant vocabularies actually differ. Homogeneous tenants will show a smaller ranking shift.

Skip it, or treat it as already solved, when each tenant already lives on its own shard. Shard-local IDF is the default in that layout.

Keep the retrieval filter even after you add the IDF corpus. The two jobs stay separate: IDF decides term weights, the retrieval filter decides which points may appear. If a client, or an agent writing its own retrieval, omits either one, you are back to mixed statistics or mixed documents. Control plane, not scoring.

What this run is, and is not

Limits of the run, stated so nobody has to reverse-engineer them:

• One node, one shard, Docker v1.19.0. Not a cluster, not Qdrant Cloud, not multi-shard routing.
• FastEmbed Qdrant/bm25, not a custom tokenizer. Language-neutral BM25 from the same release is a different pipeline.
• CQADupStack duplicate-question labels. A support-ticket corpus, a product catalog, or a code-search tenant mix can move labeled nDCG more, or less. The correctness result (agreement with tenant-local BM25) should transfer. The BEIR delta should not be treated as a universal quality lift.
• Correctness used 250 queries per forum. BEIR used the full 13,145.

The SDK surface (IdfCorpusParams.corpus, default global scope) matches the 1.19 docs. No client-API update was required beyond passing search_params.

Reproduce it

Every script in this section lives in the public per-tenant-idf demo repo(opens in a new tab). Clone it, download the BEIR dump, and start Qdrant 1.19 before running anything. The indexer is idempotent: if the collection already has 457,199 points, it refuses to rebuild.

docker compose up -d          # qdrant/qdrant:v1.19.0
uv sync
uv run python src/01_download_data.py
# uv run python src/02_index_data.py   # skipped if the collection is full
uv run python src/03_accuracy_benchmark.py
uv run python src/04_latency_benchmark.py
uv run python src/05_edge_cases.py
uv run python src/06_term_case_study.py
uv run python src/08_beir_eval.py
uv run python src/09_plot_results.py

Scripts import common from src/. From the repo root:

PYTHONPATH=src uv run python src/08_beir_eval.py

Frequently asked questions

Does a Qdrant tenant payload filter already scope BM25 IDF?

No. A payload filter (including is_tenant=true) controls which points can be returned. IDF N and df stay shard-wide unless you set params.idf.corpus. The 1.19 parameter exists because the retrieval filter never did that job.

How do I enable per-tenant IDF in Qdrant 1.19?

Create the sparse vector with modifier=IDF, index the tenant field, then pass search_params=SearchParams(idf=IdfCorpusParams(corpus=<tenant filter>)) on query_points. REST uses params.idf.corpus with the same filter body. A QQL WASM playground that projects QQL to REST is the right place to inspect that wire split: retrieval filter on the query, IDF corpus under params.

How much does per-tenant IDF improve nDCG?

Against tenant-local BM25, nDCG@10 goes from 0.9825 to 0.9990 on 2,999 queries. Against BEIR duplicate-question labels, macro nDCG@10 goes from 0.2945 to 0.2964. Use the first number to trust the implementation. Do not use the second number as a quality headline.

Why did top-1 change so often if nDCG barely moved?

CQADupStack labels about 1.8 relevant documents per query. nDCG@10 is muted when relevants are scarce. Rank swaps among unlabeled near-matches do not score. Users still see the new #1.

What is the latency cost of params.idf.corpus?

On this 457k / 1-shard run: +0.89 ms p50, +1.02 ms p95. Measure it on your hardware. The order of magnitude is “a millisecond”, not “a timeout”.

Should I use per-tenant IDF if each tenant has its own shard?

Usually no. A dedicated shard already computes IDF on that shard’s points. The extra filter is for payload-partitioned tenants that share a shard.

Per-tenant IDF does what it claims. BM25 scoring within a tenant becomes the ranking you would get from a dedicated collection, at about 0.9 ms extra. On this corpus, default global statistics change the first search result about every fifth query. Take that number into a design review. The 0.2 point BEIR tick is a footnote.

If you are wiring this into a multi-tenant retrieval path, pair it with an injected retrieval filter, not a client you have to trust. The gateway problem sits next to this one. This post is the scoring side.