pg.ddx.io  pgvector/pgvector GitHub issues and pull requests  
help / color / mirror / Atom feed
[pgvector/pgvector] issue #1055: HNSW under SERIALIZABLE: two concurrent check-then-insert transactions can both commit
3+ messages / 2 participants
[nested] [flat]

* [pgvector/pgvector] issue #1055: HNSW under SERIALIZABLE: two concurrent check-then-insert transactions can both commit
@ 2026-11-02 01:49 "anonymous-ai-researcher (@anonymous-ai-researcher)" <noreply+anonymous-ai-researcher@github.com>
  0 siblings, 0 replies; 3+ messages in thread

From: anonymous-ai-researcher (@anonymous-ai-researcher) @ 2026-11-02 01:49 UTC (permalink / raw)
  To: pgvector/pgvector <noreply+pgvector-pgvector@github.com>

Hi, and thanks for pgvector! We use it as a shared memory store for several agents that write at the same time, and while testing isolation we ran into something under `SERIALIZABLE` that looks like a small race between PostgreSQL's conflict check and the HNSW insert. Sharing it here in case it's useful. Happy to be told if we've misread something.

### What we're doing

Each agent runs a simple "check, then insert" transaction: look up the 10 nearest rows, and insert a new row only if none of them is close enough. We run this at `SERIALIZABLE`, expecting that two agents inserting the same thing at the same moment can't both succeed: one of them should get a serialization failure.

Most of the time that's exactly what happens. But occasionally both transactions commit, and we end up with two copies of the same row.

### What we think is going on

HNSW doesn't do its own predicate locking, so PostgreSQL falls back to a lock on the whole index. On insert, `index_insert()` checks for conflicting readers first and only then calls `hnswinsert()`. `hnswinsert()` then searches the graph for neighbors and only at the very end links the new element in.

That leaves a short window. If another serializable transaction starts its scan after the check but before the element is linked:

- the writer's check already happened, so it never sees the reader's lock, and
- the reader can't reach the new element yet, so it doesn't notice it either.

One side of the conflict goes unrecorded, SSI doesn't see a dangerous pattern, and both transactions commit.

### How to reproduce it reliably

With gdb you can hold the writer inside that window and make it happen every time.

Setup:

```sql
CREATE EXTENSION IF NOT EXISTS vector;
CREATE TABLE mem (id bigserial PRIMARY KEY, v vector(3));
INSERT INTO mem (v)
  SELECT ARRAY[random(), random(), random()]::vector FROM generate_series(1, 1000);
CREATE INDEX ON mem USING hnsw (v vector_l2_ops);
```

Both sessions run the same transaction. The query point `[5,5,5]` is far from everything, so neither session finds a close row and both decide to insert:

```sql
BEGIN ISOLATION LEVEL SERIALIZABLE;
SET LOCAL enable_seqscan = off;
SELECT id, v <-> '[5,5,5]' AS d FROM mem ORDER BY v <-> '[5,5,5]' LIMIT 10;
INSERT INTO mem (v) VALUES ('[5,5,5]');
COMMIT;
```

Steps:

1. In session 1, run `SELECT pg_backend_pid();`, then `BEGIN`, `SET LOCAL` and the `SELECT`.
2. Attach gdb to that backend, `break hnswinsert`, `continue`.
3. Run the `INSERT` in session 1. It stops at the start of `hnswinsert()`, just after the conflict check.
4. In session 2, run the whole transaction. It commits.
5. Back in gdb, `continue`, then `COMMIT` in session 1. It also commits.
6. `SELECT count(*) FROM mem WHERE v = '[5,5,5]';` returns 2.

If you do the same thing but start session 2 only after session 1's `INSERT` has finished, one of them gets `could not serialize access`, as expected.

Without a debugger the window is short, but with many tightly timed pairs it still shows up now and then, and more often when inserts are slower (larger `m` or `ef_construction`).

### A small fix that worked for us

Running the same check once more, right after the element is linked, closes the window. The lock is still on the whole index, so nothing else changes. `CheckForSerializableConflictIn()` returns right away when no serializable transaction is running, so other workloads shouldn't notice it.

```diff
inline patch 1: src/hnswinsert.c (+8/-0 lines) download inline-1.patch--- a/src/hnswinsert.c +++ b/src/hnswinsert.c @@ -9,6 +9,7 @@ #include "storage/freespace.h" #include "storage/lmgr.h" #include "storage/lwlock.h" +#include "storage/predicate.h" #include "utils/datum.h" #include "utils/memutils.h" #include "utils/rel.h" @@ -852,6 +853,13 @@ hnswinsert(Relation index, Datum *values, bool *isnull, ItemPointer heap_tid, /* Insert tuple */ HnswInsertTuple(index, values, isnull, heap_tid); + /* + * index_insert() checked for serializable conflicts before the element + * was linked; a scan that started in between missed both the element and + * the check. Repeat the check now that the element is reachable. + */ + CheckForSerializableConflictIn(index, NULL, InvalidBlockNumber); + /* Delete memory context */ MemoryContextSwitchTo(oldCtx); MemoryContextDelete(insertCtx);
``` With this change the gdb steps above end with one transaction aborting, and in our stress runs we no longer saw both rows stored. ### A couple of side notes - This window really comes from PostgreSQL's fallback for index types without predicate locking, so it isn't unique to pgvector: the same gdb trick also gives a double commit on SP-GiST and BRIN (but not on B-tree). A general fix might belong in core, and we're glad to raise it there too if you think that's the better place. - IVFFlat goes through the same fallback path. We haven't checked whether it has the same window. Environment: PostgreSQL 16.15, pgvector 0.8.7, HNSW index with default settings. If it would help, we can turn the reproduction into a test, or open a PR with the patch. Thanks again for the great work on this extension! ^ permalink raw [nested|flat] 3+ messages in thread

* Re: [pgvector/pgvector] issue #1055: HNSW under SERIALIZABLE: two concurrent check-then-insert transactions can both commit
@ 2026-11-02 03:19 ` "ankane (@ankane)" <noreply+ankane@github.com>
  1 sibling, 0 replies; 3+ messages in thread

From: ankane (@ankane) @ 2026-11-02 03:19 UTC (permalink / raw)
  To: pgvector/pgvector <noreply+pgvector-pgvector@github.com>

Hi @anonymous-ai-researcher, nice find. It'd be best to report this to core since it isn't specific to pgvector (HNSW and IVFFlat set `ampredlocks = false` like BRIN and SP-GiST).

^ permalink  raw  [nested|flat] 3+ messages in thread

* Re: [pgvector/pgvector] issue #1055: HNSW under SERIALIZABLE: two concurrent check-then-insert transactions can both commit
@ 2026-11-02 21:46 ` "ankane (@ankane)" <noreply+ankane@github.com>
  1 sibling, 0 replies; 3+ messages in thread

From: ankane (@ankane) @ 2026-11-02 21:46 UTC (permalink / raw)
  To: pgvector/pgvector <noreply+pgvector-pgvector@github.com>

@ankane closed this.

^ permalink  raw  [nested|flat] 3+ messages in thread


end of thread, other threads:[~2026-11-02 21:46 UTC | newest]

Thread overview: 3+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2026-11-02 01:49 [pgvector/pgvector] issue #1055: HNSW under SERIALIZABLE: two concurrent check-then-insert transactions can both commit "anonymous-ai-researcher (@anonymous-ai-researcher)" <noreply+anonymous-ai-researcher@github.com>
2026-11-02 03:19 ` "ankane (@ankane)" <noreply+ankane@github.com>
2026-11-02 21:46 ` "ankane (@ankane)" <noreply+ankane@github.com>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox