pg.ddx.io  pgvector/pgvector GitHub issues and pull requests  
help / color / mirror / Atom feed
From: mohakgoel1 (@mohakgoel1) <noreply+mohakgoel1@github.com>
To: pgvector/pgvector <noreply+pgvector-pgvector@github.com>
Subject: Re: [pgvector/pgvector] PR #1067: [Arm CPU] Add SVE2 distance functions for halfvec on AArch64
Date: Thu, 19 Nov 2026 05:33:02 +0000
Message-ID: <gh-pgvector-pgvector-1067-c7000717520@github.com> (raw)
In-Reply-To: <gh-pgvector-pgvector-1067@github.com>
References: <gh-pgvector-pgvector-1067@github.com>

> A few findings from testing:
> 
> * GCC (14 and 16) and Clang (19 and 21) don't seem to generate `FMLALB` / `FMLALT` instructions with auto-vectorization
> * I'm not seeing much benefit with SVE vs Neon on Graviton 5 (likely since they have the same vector width), but if there are platforms that benefit significantly, `target_clones` could be used for SVE
> 
> Will spend some more time on this.

Thanks for testing. I benchmarked on Graviton4 (Neoverse V2, GCC 13.4), comparing five versions of each `halfvec` distance function on identical data: the default loop; the same loop compiled for SVE and for SVE2 (what `target_clones` would produce); this PR's SVE2 kernels; and a hand-written NEON FHM version (`FMLAL`/`FMLAL2`). I measured with data in cache and with data coming from memory (one query vs random vectors from a 1 GB pool), against both an `OPTFLAGS=""` and a `-march=native` baseline.

**Summary: the hand-written SVE2 kernels are faster than the compiler's auto-vectorized code in every case we tested, whether NEON (default and `-march=native`) or SVE/SVE2 (`target_clones`), with one tie (inner product from memory vs `-march=native`).**

| 384–1536 dims | vs `OPTFLAGS=""`, in cache | vs `-march=native`, in cache | vs `-march=native`, from memory |
|---|---|---|---|
| L2 | 1.74–2.19x | 1.67–2.10x | 1.08–1.23x |
| Inner product | 3.32–3.45x | 1.68–1.79x | 0.98–1.03x |
| Cosine | 1.90–2.54x | 1.80–2.41x | 1.11–1.32x |
| L1 | 1.39–1.60x | 1.55–1.78x | 1.02–1.20x |

- **Auto-vectorization for SVE/SVE2 (`target_clones`):** 0.91–1.04x, no benefit, matching your finding.
- **What compilers generate:** with `-march=native`, GCC 13 emits NEON `FMLAL` for the inner-product loop only (hence the smaller inner-product gain vs `-march=native`); L2, cosine and L1 still convert each element with `FCVT`. Even against this best auto-vectorized NEON code, the SVE2 kernels are 1.6–2.4x faster in cache.
- **Hand-written NEON FHM:** about equal to the SVE2 kernels for inner product and cosine, but the SVE2 kernels are about 1.3x faster for L2 (1536 dims: 138 vs 182 ns) and L1 (172 vs 218 ns).

On 128-bit cores like Graviton4/5, SVE and NEON have the same width, so these gains come from the instructions rather than vector width.

**Follow-up SVE work:** this PR also adds the SVE/SVE2 detection and runtime dispatch in `halfvec.h` (`USE_SVE`, `TARGET_SVE`, `SVE_RUNTIME_SUPPORTED()`). I have more SVE optimizations in the pipeline that depend on it, which I'll raise once this PR is merged. They need only base SVE, so they also run on Graviton3, where SVE vectors are 256 bits wide while NEON stays at 128 bits, which should give a larger benefit there. I'll include Graviton3 measurements when I submit them.

view thread (7+ messages)  latest in thread

Message-ID: <gh-pgvector-pgvector-1067-c7000717520@github.com>
Permalink:  ../gh-pgvector-pgvector-1067-c7000717520@github.com/

 · 


This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox