pg.ddx.io pgvector/pgvector GitHub issues and pull requests
help / color / mirror / Atom feed[pgvector/pgvector] issue #1003: Support RabitQ in pgvector
12+ messages / 5 participants
[nested] [flat]
* [pgvector/pgvector] issue #1003: Support RabitQ in pgvector
@ 2026-07-28 08:58 "zthhhhh (@zthhhhh)" <noreply+zthhhhh@github.com>
0 siblings, 0 replies; 12+ messages in thread
From: zthhhhh (@zthhhhh) @ 2026-07-28 08:58 UTC (permalink / raw)
To: pgvector/pgvector <noreply+pgvector-pgvector@github.com>
## Summary
I noticed pgvector maintainer mentioned supporting RaBitQ in #989.
We have implemented RaBitQ quantization for both the IVFFlat and HNSW access method
in pgvector. The implementation supports cosine distance and does not change the existing
query flow or SQL interface.
It has seen some internal use in IVF RaBitQ, supporting cosine-distance search
on single heap table containing hundreds of millions of vectors.
We would like to ask whether the maintainers are interested in accepting this work into pgvector.
## Benchmark
Our ann-benchmark results show that RaBitQ reduces index storage and improves
query performance while maintaining competitive recall.
- After each index was built, `pg_prewarm` was used to load it into the PostgreSQL buffer cache before running search benchmarks.
- `shared_buffers` was configured to be large enough to hold the indexes. This minimizes storage I/O during search and makes the benchmark focus on the distance-computation speedup provided by RaBitQ.
### Environment
- Hardware:
- CPU: Intel Xeon Platinum 8269CY @ 2.50GHz
- OS kernel version: Linux 5.10.134
- Software:
- PostgreSQL
- version: v17.9
- shared_buffers = 10GB
- max_parallel_workers = 4
- pgvector base version: v0.8.0
- Benchmark configuration
- TopK = 10
- dataset: dbpedia-openai-1000k-angular (1536D1M)
- IVFFlat configuration:
- lists = 1000
- probes = 1, 2, 5, 10, 20, 50, 100
- quantization_bit = 1
- HNSW configuration:
- m = 16
- ef_construction = 64
- ef_search = 40, 60, 80, 100, 120, 140, 160, 180
- quantization_bit = 7
### IVFFlat
#### Build
| Implementation | Build time | Index size |
| --- | ---: | ---: |
| IVFFLAT baseline | 144.9s | 7820MB |
| IVF RaBitQ | 120s | 281MB |
#### Search performance
| `probes` | Baseline QPS | Baseline Recall@10 | RaBitQ QPS | RaBitQ Recall@10 | Speed up |
| ---: | ---: | ---: | ---: | ---: | ---: |
| 1 | 224.957 | 68.9% | 690.172 | 69.2% | 3.07x |
| 2 | 141.526 | 81.1% | 499.285 | 81.0% | 3.53x |
| 5 | 76.083 | 90.6% | 297.533 | 90.1% | 3.91x |
| 10 | 58.398 | 94.3% | 198.78 | 93.9% | 3.40x |
| 20 | 22.794 | 96.4% | 139.59 | 96.3% | 6.12x |
| 50 | 9.066 | 98.3% | 78.50 | 98.1% | 8.66x |
| 100 | 5.681 | 99.0% | 45.83 | 98.9% | 8.07x |
### HNSW
#### Build
| Implementation | Build time | Index size |
| --- | ---: | ---: |
| HNSW baseline | 359.76s | 7813MB |
| HNSW RaBitQ | 714.22s | 2604MB |
#### Search performance
| `ef_search` | Baseline QPS | Baseline Recall@10 | RaBitQ QPS | RaBitQ Recall@10 | Speed up |
| ---: | ---: | ---: | ---: | ---: | ---: |
| 40 | 395.243 | 99.0% | 447.133 | 98.7% | 1.13x |
| 60 | 295.626 | 99.2% | 353.140 | 99.2% | 1.19x |
| 80 | 277.763 | 99.4% | 300.351 | 99.3% | 1.08x |
| 100 | 225.656 | 99.4% | 245.344 | 99.4% | 1.09x |
| 120 | 187.349 | 99.4% | 213.504 | 99.4% | 1.14x |
| 140 | 185.545 | 99.4% | 208.332 | 99.3% | 1.12x |
| 160 | 167.264 | 99.5% | 193.393 | 99.3% | 1.16x |
| 180 | 151.726 | 99.5% | 175.822 | 99.4% | 1.16x |
## Next steps
If the maintainers are interested, we are willing to prepare the code for an
upstream contribution and do the additional work needed, including:
- Providing more detailed and reproducible benchmarks
- Discussing design details, including the GUCs, index reloptions, operator classes, and extension upgrade objects requires by the design
- Organizing the implemented features into focused, independent commits to make review easier
- Improving the implementation based on maintainer feedback
- Adding support for other distance type, such as L2 distance support
- Adding compatibility and fallback support for different CPU architectures
- Adding documentation
We would appreciate guidance on whether this feature would be of interest and how the maintainers would prefer us to proceed.
^ permalink raw [nested|flat] 12+ messages in thread
* Re: [pgvector/pgvector] issue #1003: Support RabitQ in pgvector
@ 2026-07-28 17:47 ` "ankane (@ankane)" <noreply+ankane@github.com>
10 siblings, 0 replies; 12+ messages in thread
From: ankane (@ankane) @ 2026-07-28 17:47 UTC (permalink / raw)
To: pgvector/pgvector <noreply+pgvector-pgvector@github.com>
Hi @zthhhhh, it'd be good to see how this compares to the existing recommendation for [scaling](https://github.com/pgvector/pgvector#scaling), which is to use the `halfvec` type for the table, binary quantization for indexing, and re-ranking for search.
^ permalink raw [nested|flat] 12+ messages in thread
* Re: [pgvector/pgvector] issue #1003: Support RabitQ in pgvector
@ 2026-07-29 03:18 ` "zthhhhh (@zthhhhh)" <noreply+zthhhhh@github.com>
10 siblings, 0 replies; 12+ messages in thread
From: zthhhhh (@zthhhhh) @ 2026-07-29 03:18 UTC (permalink / raw)
To: pgvector/pgvector <noreply+pgvector-pgvector@github.com>
Hi @ankane , thanks for your suggestion. I'll add a comparision with binary quantization and re-ranking later.
^ permalink raw [nested|flat] 12+ messages in thread
* Re: [pgvector/pgvector] issue #1003: Support RabitQ in pgvector
@ 2026-07-29 09:07 ` "zthhhhh (@zthhhhh)" <noreply+zthhhhh@github.com>
10 siblings, 0 replies; 12+ messages in thread
From: zthhhhh (@zthhhhh) @ 2026-07-29 09:07 UTC (permalink / raw)
To: pgvector/pgvector <noreply+pgvector-pgvector@github.com>
## Query
I used this query to process benchmark with `dbpedia-openai-1000k-angular` dataset.
```sql
SELECT id FROM
(SELECT id, embedding FROM items
ORDER BY binary_quantize(embedding)::bit(1536) <~> binary_quantize(${query_vector}::halfvec(1536))
LIMIT ${rerank_candidates}) candidates
ORDER BY embedding <=> ${query_vector}::halfvec(1536) LIMIT ${topk};
```
Add a benchmark parameter `rerank_candidate` for binary quantization experimental group, and it is swept in a full Cartesian product with the search parameter (probes in IVFFlat, ef_search in HNSW).
## Benchmark Configuration
- IVF configuration
- lists = 1000
- probes = 1, 2, 5, 10, 20, 50, 100, 200, 300, 500
- rerank_candidate = 20, 40, 80, 100, 200, 500, 1000, 2000
- HNSW configuration
- m = 16
- ef_construction = 64
- ef_search = 10, 20, 40, 80, 100, 200
- rerank_candidate = 20, 40, 80, 100, 200
## Results and Analysis
Due to the large number of parameter combinations in the binary quantization experimental group, QPS-recall curves are used below
to visually compare RaBitQ against the binary quantization approach.
### IVF
<img width="1252" height="783" alt="Image" src="https://github.com/user-attachments/assets/e2062f22-87a0-4af8-8158-e65a98cd1b2e"; />
At the same `probes` setting, IVF RaBitQ achieves higher QPS than the binary quantization approach. In the binary quantization
IndexScan executor node, all vectors from the probed lists must be sorted by Hamming distance, which significantly increases query latency.
In contrast, RaBitQ leverages the theoretical error bound on its distance estimates (Formula 16 in the RaBitQ paper).
After computing distances via Algorithm 2, RaBitQ uses this bound to quickly eliminate vectors that cannot appear in the top-K results,
substantially reducing the number of vectors that need to be sorted.
### HNSW
<img width="1249" height="783" alt="Image" src="https://github.com/user-attachments/assets/e8fea191-b38d-47bf-bffe-af7272e560c6"; />
At the same `ef_search` setting, the binary quantization approach achieves lower recall than HNSW RaBitQ. In the HNSW search layer phase, a candidate set `W` is maintained based on computed distances, and these distances determine which vectors the search
visits next. With binary quantization, the inaccurate distance computation affects the selection of vectors in subsequent iterations,
degrading recall. RaBitQ mitigates this with an unbiased distance estimator whose theoretical error bound keeps the search path
closer to that of exact distance, yielding higher recall at the same `ef_search`.
^ permalink raw [nested|flat] 12+ messages in thread
* Re: [pgvector/pgvector] issue #1003: Support RabitQ in pgvector
@ 2026-07-29 23:45 ` "ankane (@ankane)" <noreply+ankane@github.com>
10 siblings, 0 replies; 12+ messages in thread
From: ankane (@ankane) @ 2026-07-29 23:45 UTC (permalink / raw)
To: pgvector/pgvector <noreply+pgvector-pgvector@github.com>
Thanks @zthhhhh, can you share the index sizes and build times as well?
I'm not sure the difference will be worth the complexity. I think it'd also be good to try asymmetric binary quantization (quantize the indexed vectors but not the query vector) to see how it compares.
Also, just fyi, it may be some time before I'm able to focus on this.
^ permalink raw [nested|flat] 12+ messages in thread
* Re: [pgvector/pgvector] issue #1003: Support RabitQ in pgvector
@ 2026-07-30 06:58 ` "zthhhhh (@zthhhhh)" <noreply+zthhhhh@github.com>
10 siblings, 0 replies; 12+ messages in thread
From: zthhhhh (@zthhhhh) @ 2026-07-30 06:58 UTC (permalink / raw)
To: pgvector/pgvector <noreply+pgvector-pgvector@github.com>
Thanks @ankane , I have added the index sizes and build times to the comparison.
## Index build and size
The following results use the same dataset and index parameters as the query benchmarks shared above.
| Access method | Implementation | Build time | Index size |
| --- | --- | ---: | ---: |
| IVFFlat | without quantization | 214.71 s | 7820 MB |
| IVFFlat | RaBitQ | 181.74 s | 256 MB |
| IVFFlat | halfvec + binary quantization | 42.88 s | 210 MB |
| HNSW | without quantization | 644.34 s | 7813 MB |
| HNSW | RaBitQ | 1372.24 s | 2640 MB |
| HNSW | halfvec + binary quantization | 82.82 s | 479 MB |
## RaBitQ integration
> I'm not sure the difference will be worth the complexity.
I understand that minimizing the impact on the existing file is also an important consideration.
For RaBitQ, the current design reuses the existing IVFFlat and HNSW access methods rather than introducing new access methods. The quantization method is reported through the existing type support functions:
- Support function 5 for IVFFlat
- Support function 3 for HNSW
The access methods use this metadata to select the corresponding index representation, distance evaluation, and refinement paths.
This avoids duplicating the existing access methods and requires only limited changes to the existing file.
## Asymmetric binary quantization
> I think it'd also be good to try asymmetric binary quantization (quantize the indexed vectors but not the query vector) to see how it compares.
My understanding of the suggested asymmetric binary quantization approach is that indexed vectors remain binary-quantized while
the query vector remains an unquantized `vector` or `halfvec`.
pgvector currently provides Hamming distance between two `bit` values, but it does not provide a distance operator between a binary-quantized vector and a vector. This could be implemented by adding an asymmetric distance function and registering the corresponding operator and operator class. The index and query would conceptually look like this:
```sql
CREATE INDEX items_embedding_idx ON items USING hnsw (
(binary_quantize(embedding)::bit(1536))
${asymmetric_binary_operator_class}
);
SELECT id
FROM items
ORDER BY binary_quantize(embedding)::bit(1536)
${asymmetric_binary_operator} ${query_vector}
LIMIT 10;
```
Compared with symmetric Hamming distance, this is expected to improve recall before re-ranking because the full values of the query vector are retained in the distance calculation. It provides a tradeoff between symmetric binary quantization and unquantized distance computation.
Since `binary_quantize` already determines the indexed representation, this should mainly require a new distance function, operator, and operator class, without changes to the existing HNSW or IVFFlat access method handlers. I think that this is a worthwhile comparison and would be happy to provide an implementation as a pull request.
Thank you for all your work on maintaining and improving pgvector.
^ permalink raw [nested|flat] 12+ messages in thread
* Re: [pgvector/pgvector] issue #1003: Support RabitQ in pgvector
@ 2026-09-18 09:59 ` "R3gardless (@R3gardless)" <noreply+R3gardless@github.com>
10 siblings, 0 replies; 12+ messages in thread
From: R3gardless (@R3gardless) @ 2026-09-18 09:59 UTC (permalink / raw)
To: pgvector/pgvector <noreply+pgvector-pgvector@github.com>
@zthhhhh Hi, I'm interested in this implementation!
The performance gains look great, but the smaller indexes caught my eye just as much. RAM is expensive, so getting the HNSW index down to about a third of its original size seems pretty valuable on its own, even without a huge speedup. More room in cache and one less reason to ask for a bigger server :)
Would be interesting to see how it performs when the original index doesn't fit in memory but the quantized one does. Thanks for sharing!
^ permalink raw [nested|flat] 12+ messages in thread
* Re: [pgvector/pgvector] issue #1003: Support RabitQ in pgvector
@ 2026-10-23 17:01 ` "gouyt13 (@gouyt13)" <noreply+gouyt13@github.com>
10 siblings, 0 replies; 12+ messages in thread
From: gouyt13 (@gouyt13) @ 2026-10-23 17:01 UTC (permalink / raw)
To: pgvector/pgvector <noreply+pgvector-pgvector@github.com>
Hi @ankane and @zthhhhh,
I’m one of the authors of multi-bit RaBitQ and a maintainer of [RaBitQ-Library](https://github.com/VectorDB-NTU/RaBitQ-Library), our official implementation. Thanks for exploring RaBitQ support in pgvector and sharing these results!
Would you be open to discussing whether parts of the integration could reuse the library’s quantization and distance-estimation components?
We provide a standalone C++17 core with a source-stable API for encoding and scoring, separate from our ANN indexes. It uses caller-owned code buffers and workspace, and can be built without OpenMP. The idea would be to retain pgvector’s storage and index lifecycle management while assessing whether these numerical components could be reused.
This could reduce duplicated algorithm and SIMD maintenance, although the C++ dependency and PostgreSQL integration would add costs that need to be evaluated. We have not yet validated a pgvector integration.
Would an external C++ dependency be an option you would consider? If so, we’d be happy to discuss a minimal IVF-focused integration with @zthhhhh, including comparisons against the recommended halfvec + binary quantization + reranking approach and the asymmetric binary quantization baseline you suggested.
^ permalink raw [nested|flat] 12+ messages in thread
* Re: [pgvector/pgvector] issue #1003: Support RabitQ in pgvector
@ 2026-10-26 08:28 ` "zthhhhh (@zthhhhh)" <noreply+zthhhhh@github.com>
10 siblings, 0 replies; 12+ messages in thread
From: zthhhhh (@zthhhhh) @ 2026-10-26 08:28 UTC (permalink / raw)
To: pgvector/pgvector <noreply+pgvector-pgvector@github.com>
Hi @gouyt13 and @ankane,
Thanks for your attention! `RaBitQ-Library` was an important reference when I developed my initial RaBitQ implementation for pgvector, and it helped me better understand the algorithms described in the papers.
Reusing the library’s standalone numerical components sounds promising. C/C++ interoperability itself should be manageable through an `extern "C"` wrapper, but a few integration details need careful consideration.
PostgreSQL uses MemoryContext for memory management, and its error handling relies on `longjmp`. We would need to catch C++ exceptions within the wrapper and avoid PostgreSQL errors jumping across C++ stack frames that require destructors to run. Ownership and cleanup of any objects that survive individual calls would also need to be clearly defined. pgvector would continue to manage pages, tuples, locking, and WAL. We would need to evaluate how the library’s code layouts fit into pgvector’s index tuple and page formats, whether any conversion or copying is necessary, and the resulting performance impact. The caller-owned buffers and workspace interfaces are helpful for controlling memory usage.
A prototype should help determine which changes belong in the pgvector integration layer and whether any extensions to the library’s interfaces are needed.
> Would you be open to discussing whether parts of the integration could reuse the library’s quantization and distance-estimation components?
I would be happy to work on this. My suggestion is to start with a minimal IVF integration against upstream pgvector, retaining its existing storage and index lifecycle management while reusing selected quantization and distance-estimation components.
I could first validate correctness and evaluate recall, query performance, build time, and memory usage. Those results would help us decide how to proceed with further integration.
^ permalink raw [nested|flat] 12+ messages in thread
* Re: [pgvector/pgvector] issue #1003: Support RabitQ in pgvector
@ 2026-11-03 06:23 ` "ankane (@ankane)" <noreply+ankane@github.com>
10 siblings, 0 replies; 12+ messages in thread
From: ankane (@ankane) @ 2026-11-03 06:23 UTC (permalink / raw)
To: pgvector/pgvector <noreply+pgvector-pgvector@github.com>
Hi @gouyt13, thanks for the interest in pgvector. However, I wouldn't want to add any external dependencies for this.
^ permalink raw [nested|flat] 12+ messages in thread
* Re: [pgvector/pgvector] issue #1003: Support RabitQ in pgvector
@ 2026-11-09 20:12 ` "txfs19260817 (@txfs19260817)" <noreply+txfs19260817@github.com>
10 siblings, 0 replies; 12+ messages in thread
From: txfs19260817 (@txfs19260817) @ 2026-11-09 20:12 UTC (permalink / raw)
To: pgvector/pgvector <noreply+pgvector-pgvector@github.com>
Hi @ankane, I am Zishuo Ding from AWS. I work on PostgreSQL performance. These are my results for asymmetric BQ.
One-commit branches:
- [`asymmetric-bq`](https://github.com/txfs19260817/pgvector/tree/asymmetric-bq): As you suggested above, it adds the `bit_vector_ip_ops` operator class and a support function 4 in about 190 lines of src code.
- [`hnsw-quantization`](https://github.com/txfs19260817/pgvector/tree/hnsw-quantization): Another asymmetric BQ prototype that adds `WITH (quantization = ...)` to the existing operator classes. In this way, queries need no cast. It is about 260 lines of C src.
- Benchmarks across 6 datasets: [`benchmarks/doc`](https://github.com/txfs19260817/pgvector/tree/benchmarks/doc)
**8.8M MS MARCO**
Setup: 8.8M [MS MARCO](https://microsoft.github.io/msmarco/) with 960 dimensions, recall@10 of about 0.99, 8 clients, and r7i.2xlarge (x86) and r7g.2xlarge (ARM) with 64 GB.
| Option | QPS, x86 / ARM | Index size |
|---|---|---|
| HNSW, halfvec | 280 / 368 | 22.5 GiB |
| BQ + rerank, halfvec | 261 / 345 | **3.5 GiB** |
| *Asymmetric BQ + rerank, halfvec* | **625** / **693** | **3.5 GiB** |
| *Asymmetric BQ + rerank, vector* | 441 / 500 | **3.5 GiB** |
In another run, `WITH (quantization = 'binary')` on halfvec with the CTE rerank got 583 / 676 QPS.
**More datasets**
These are recall-QPS curves with 1 client. This is [dbpedia-openai-1000k-angular](https://huggingface.co/datasets/KShivendu/dbpedia-entities-openai-1M) on x86. Each BQ value is the median of 3. The [doc](https://github.com/txfs19260817/pgvector/tree/benchmarks/doc) has 4 more datasets, ARM results, and a dimension sweep.

| Option | QPS at recall 0.90 | 0.95 | 0.98 | Index size | Build time |
|---|---|---|---|---|---|
| HNSW, halfvec | 829 | 438 | 112 | 3,867 MB | 219 s |
| BQ + rerank, halfvec | 466 | 256 | 100 | **474 MB** | 60 s |
| *Asymmetric BQ + rerank, halfvec* | 853 | 435 | **165** | **474 MB** | **59 s** |
| *`WITH (quantization = 'binary')`, halfvec* | **1,021** | **548** | 161 | **474 MB** | 60 s |
A [dimension sweep](https://github.com/txfs19260817/pgvector/tree/benchmarks/doc#dimension-sweep) experiment truncates text-embedding-3-large to 256-3,072 dims. At recall 0.95 on x86, `WITH` is 3.7x as fast as BQ + rerank at 256 dimensions, and 1.2x at 3,072. Below about 1,024 dimensions, HNSW on halfvec is still faster.
**Prototype `WITH (quantization = 'binary')`**
```sql
CREATE INDEX ON items USING hnsw (embedding vector_cosine_ops) WITH (quantization = 'binary');
WITH candidates AS MATERIALIZED (
SELECT id, embedding <=> '[1,-2,3]' AS distance FROM items ORDER BY distance LIMIT 100
) SELECT * FROM candidates ORDER BY distance + 0 LIMIT 10;
```
It works the same as `asymmetric-bq` with no rerank inside the index so a materialized CTE reranks the rows.
**RaBitQ**
I also wrote an experimental RaBitQ patch for HNSW for comparison. It is the [`rabitq-hnsw`](https://github.com/txfs19260817/pgvector/tree/rabitq-hnsw) branch. In my tests it did not do better than asymmetric BQ (see the doc), so I do not plan to continue it, putting my time on asymmetric BQ instead.
@zthhhhh and @gouyt13 know RaBitQ much better than I do. If my RaBitQ results can help your work, feel free to use them. I am also happy to work together on it.
^ permalink raw [nested|flat] 12+ messages in thread
* Re: [pgvector/pgvector] issue #1003: Support RabitQ in pgvector
@ 2026-11-12 00:54 ` "ankane (@ankane)" <noreply+ankane@github.com>
10 siblings, 0 replies; 12+ messages in thread
From: ankane (@ankane) @ 2026-11-12 00:54 UTC (permalink / raw)
To: pgvector/pgvector <noreply+pgvector-pgvector@github.com>
Hi @txfs19260817, thanks for sharing! Will dig into this after the 0.9.0 release.
^ permalink raw [nested|flat] 12+ messages in thread
end of thread, other threads:[~2026-11-12 00:54 UTC | newest]
Thread overview: 12+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2026-07-28 08:58 [pgvector/pgvector] issue #1003: Support RabitQ in pgvector "zthhhhh (@zthhhhh)" <noreply+zthhhhh@github.com>
2026-07-28 17:47 ` "ankane (@ankane)" <noreply+ankane@github.com>
2026-07-29 03:18 ` "zthhhhh (@zthhhhh)" <noreply+zthhhhh@github.com>
2026-07-29 09:07 ` "zthhhhh (@zthhhhh)" <noreply+zthhhhh@github.com>
2026-07-29 23:45 ` "ankane (@ankane)" <noreply+ankane@github.com>
2026-07-30 06:58 ` "zthhhhh (@zthhhhh)" <noreply+zthhhhh@github.com>
2026-09-18 09:59 ` "R3gardless (@R3gardless)" <noreply+R3gardless@github.com>
2026-10-23 17:01 ` "gouyt13 (@gouyt13)" <noreply+gouyt13@github.com>
2026-10-26 08:28 ` "zthhhhh (@zthhhhh)" <noreply+zthhhhh@github.com>
2026-11-03 06:23 ` "ankane (@ankane)" <noreply+ankane@github.com>
2026-11-09 20:12 ` "txfs19260817 (@txfs19260817)" <noreply+txfs19260817@github.com>
2026-11-12 00:54 ` "ankane (@ankane)" <noreply+ankane@github.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox