pg.ddx.io pgvector/pgvector GitHub issues and pull requestshelp / color / mirror / Atom feed
[pgvector/pgvector] issue #1065: Support for Advanced K-means Acceleration Techniques in IVFFlat Index Construction 4+ messages / 3 participants [nested] [flat]
* [pgvector/pgvector] issue #1065: Support for Advanced K-means Acceleration Techniques in IVFFlat Index Construction @ 2026-11-16 13:44 "SunnyBo0302 (@SunnyBo0302)" <noreply+SunnyBo0302@github.com> 0 siblings, 0 replies; 4+ messages in thread From: SunnyBo0302 (@SunnyBo0302) @ 2026-11-16 13:44 UTC (permalink / raw) To: pgvector/pgvector <noreply+pgvector-pgvector@github.com> Hi, thank you for developing and maintaining pgvector! I was wondering whether there are any plans to explore advanced K-means acceleration techniques, such as: - **[[SuperKMeans](https://github.com/cwida/SuperKMeans)](https://github.com/cwida/SuperKMeans)** — [[Paper](https://arxiv.org/abs/2603.20009)](https://arxiv.org/abs/2603.20009) - **[[e-means](https://e-means.github.io/)](https://e-means.github.io/)** — [[Paper](https://arxiv.org/abs/2611.09501)](https://arxiv.org/abs/2611.09501) Both approaches have demonstrated promising improvements in K-means clustering efficiency. Have these approaches been considered, or would there be interest in incorporating similar optimizations to accelerate IVFFlat index construction in pgvector? I'd appreciate any thoughts on their potential applicability. ^ permalink raw [nested|flat] 4+ messages in thread
* Re: [pgvector/pgvector] issue #1065: Support for Advanced K-means Acceleration Techniques in IVFFlat Index Construction @ 2026-11-16 20:50 "ankane (@ankane)" <noreply+ankane@github.com> 2 siblings, 0 replies; 4+ messages in thread From: ankane (@ankane) @ 2026-11-16 20:50 UTC (permalink / raw) To: pgvector/pgvector <noreply+pgvector-pgvector@github.com> Hi @SunnyBo0302, I created [a branch](https://github.com/pgvector/pgvector/compare/e-means) for e-means last week. It doesn't currently outperform the existing code (which uses Elkan and k-means++), but I think it could if cluster assignment is reworked to calculate multiple distances at a time. ^ permalink raw [nested|flat] 4+ messages in thread
* Re: [pgvector/pgvector] issue #1065: Support for Advanced K-means Acceleration Techniques in IVFFlat Index Construction @ 2026-11-17 04:21 "SunnyBo0302 (@SunnyBo0302)" <noreply+SunnyBo0302@github.com> 2 siblings, 0 replies; 4+ messages in thread From: SunnyBo0302 (@SunnyBo0302) @ 2026-11-17 04:21 UTC (permalink / raw) To: pgvector/pgvector <noreply+pgvector-pgvector@github.com> @ankane Thanks for the quick response and for exploring e-means! Looking forward to seeing how it develops. ^ permalink raw [nested|flat] 4+ messages in thread
* Re: [pgvector/pgvector] issue #1065: Support for Advanced K-means Acceleration Techniques in IVFFlat Index Construction @ 2026-11-17 19:12 "saolof (@saolof)" <noreply+saolof@github.com> 2 siblings, 0 replies; 4+ messages in thread From: saolof (@saolof) @ 2026-11-17 19:12 UTC (permalink / raw) To: pgvector/pgvector <noreply+pgvector-pgvector@github.com> Apart from the e-means EMA trick, quantized k-means is another trick to speed up k-means on disk. Rabitq-quantize the vectors first, and run k-means/e-means using the quantized distances to include more points earlier for a given amount of disk traffic. If you combine it with e-means there are a few variations on that idea like doing the initial iteration using the full vectors and combining that with the pass that generates the quantized vectors, then combining the e-means passes through the quantized vectors with the rearrangement into posting lists. ^ permalink raw [nested|flat] 4+ messages in thread
end of thread, other threads:[~2026-11-17 19:12 UTC | newest] Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2026-11-16 13:44 [pgvector/pgvector] issue #1065: Support for Advanced K-means Acceleration Techniques in IVFFlat Index Construction "SunnyBo0302 (@SunnyBo0302)" <noreply+SunnyBo0302@github.com> 2026-11-16 20:50 ` "ankane (@ankane)" <noreply+ankane@github.com> 2026-11-17 04:21 ` "SunnyBo0302 (@SunnyBo0302)" <noreply+SunnyBo0302@github.com> 2026-11-17 19:12 ` "saolof (@saolof)" <noreply+saolof@github.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions for how to clone and mirror all data and code used for this inbox