pg.ddx.io pgvector/pgvector GitHub issues and pull requests
help / color / mirror / Atom feed[pgvector/pgvector] issue #1057: Cosine distance returns NaN or incorrect finite values for finite nonzero vectors
3+ messages / 2 participants
[nested] [flat]
* [pgvector/pgvector] issue #1057: Cosine distance returns NaN or incorrect finite values for finite nonzero vectors
@ 2026-11-04 02:38 "yzxxxx888 (@yzxxxx888)" <noreply+yzxxxx888@github.com>
0 siblings, 0 replies; 3+ messages in thread
From: yzxxxx888 (@yzxxxx888) @ 2026-11-04 02:38 UTC (permalink / raw)
To: pgvector/pgvector <noreply+pgvector-pgvector@github.com>
## Summary
The cosine distance operator `<=>` returns `NaN` or an incorrect finite distance for finite, nonzero `vector` inputs with large or small magnitudes.
The issue reproduces with direct SQL expressions, without HNSW or IVFFlat indexes.
## Environment
- PostgreSQL: 17.11
- pgvector: 0.8.7
- Docker image: `pgvector/pgvector:0.8.7-pg17`
- Vector type: `vector` (float32 components)
These results were observed on the versions above. I have not verified the current development branch.
## Steps to reproduce
Enable the extension:
```sql
CREATE EXTENSION IF NOT EXISTS vector;
```
Run the following queries:
```sql
-- Large finite vector compared with itself
SELECT '[1e20,0]'::vector <=> '[1e20,0]'::vector
AS large_self_distance;
-- Two finite nonzero vectors pointing in the same direction
SELECT '[1e20,0]'::vector <=> '[1,0]'::vector
AS same_direction_distance;
-- Small finite nonzero vector compared with itself
SELECT '[1e-30,0]'::vector <=> '[1e-30,0]'::vector
AS small_self_distance;
```
## Actual results
| Query | Actual cosine distance |
|---|---:|
| `[1e20,0]` compared with itself | `NaN` |
| `[1e20,0]` compared with `[1,0]` | `1` |
| `[1e-30,0]` compared with itself | `NaN` |
All input components are finite and representable as nonzero float32 values. None of these vectors is a zero vector.
## Expected results
All three queries should return a cosine distance of `0`, within normal floating-point tolerance.
For nonzero vectors:
```text
cosine_distance(a, b) = 1 - dot(a, b) / (norm(a) * norm(b))
```
A nonzero vector has zero cosine distance from itself. Positive scaling should also preserve cosine distance, so `[1e20,0]` and `[1,0]` should have distance `0`.
The expected result is therefore finite and well-defined in all three cases.
## Normalization control
Normalizing the large vectors first produces the expected result:
```sql
SELECT
l2_normalize('[1e20,0]'::vector)
<=>
l2_normalize('[1e20,0]'::vector)
AS normalized_self_distance;
SELECT
l2_normalize('[1e20,0]'::vector)
<=>
l2_normalize('[1,0]'::vector)
AS normalized_same_direction_distance;
```
Both queries return `0`.
## Cause analysis
In pgvector v0.8.7, `VectorCosineSimilarity` accumulates the dot product and squared norms using `float`:
```c
float similarity = 0.0;
float norma = 0.0;
float normb = 0.0;
similarity += ax[i] * bx[i];
norma += ax[i] * ax[i];
normb += bx[i] * bx[i];
```
Source:
https://github.com/pgvector/pgvector/blob/v0.8.7/src/vector.c#L613
For the large self-comparison, the float32 products overflow, producing an indeterminate infinity/infinity calculation.
For `[1e20,0]` compared with `[1,0]`, the dot product remains finite, but the squared norm of the large vector overflows. This produces a cosine similarity of `0` and a distance of `1`.
For the small self-comparison, the squared components underflow to zero, resulting in a zero/zero calculation.
Converting the accumulated values to double for the final calculation cannot recover information already lost during float32 multiplication and accumulation.
^ permalink raw [nested|flat] 3+ messages in thread
* Re: [pgvector/pgvector] issue #1057: Cosine distance returns NaN or incorrect finite values for finite nonzero vectors
@ 2026-11-04 02:54 "ankane (@ankane)" <noreply+ankane@github.com>
1 sibling, 0 replies; 3+ messages in thread
From: ankane (@ankane) @ 2026-11-04 02:54 UTC (permalink / raw)
To: pgvector/pgvector <noreply+pgvector-pgvector@github.com>
Hi @yzxxxx888, see #1033.
^ permalink raw [nested|flat] 3+ messages in thread
* Re: [pgvector/pgvector] issue #1057: Cosine distance returns NaN or incorrect finite values for finite nonzero vectors
@ 2026-11-04 02:54 "ankane (@ankane)" <noreply+ankane@github.com>
1 sibling, 0 replies; 3+ messages in thread
From: ankane (@ankane) @ 2026-11-04 02:54 UTC (permalink / raw)
To: pgvector/pgvector <noreply+pgvector-pgvector@github.com>
@ankane closed this.
^ permalink raw [nested|flat] 3+ messages in thread
end of thread, other threads:[~2026-11-04 02:54 UTC | newest]
Thread overview: 3+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2026-11-04 02:38 [pgvector/pgvector] issue #1057: Cosine distance returns NaN or incorrect finite values for finite nonzero vectors "yzxxxx888 (@yzxxxx888)" <noreply+yzxxxx888@github.com>
2026-11-04 02:54 ` "ankane (@ankane)" <noreply+ankane@github.com>
2026-11-04 02:54 ` "ankane (@ankane)" <noreply+ankane@github.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox