From: PG Bug reporting form <noreply@postgresql.org>
To: pgsql-bugs@lists.postgresql.org
Cc: hlinnaka@iki.fi
Subject: BUG #18341: SIGBUS with > 4GB DSM and 'dynamic_shared_memory_type = mmap'
Date: Tue, 13 Feb 2024 17:32:26 +0000
Message-ID: <18341-ce16599e7fd6228c@postgresql.org> (raw)
The following bug has been logged on the website:
Bug reference: 18341
Logged by: Heikki Linnakangas
Email address: hlinnaka@iki.fi
PostgreSQL version: 16.2
Operating system: Linux
Description:
There's a bug in the mmap DSM implementation: If you create a DSM segment
larger than 4 GB, the file that's created is too small. As a result, at
least on Linux, you get a SIGBUS when you try to use it.
I bumped into this with pgvector v0.6.0, which creates a one giant DSM
segment with size roughly equal to `maintenance_work_mem`, when you create
an HNSW index:
postgres=# SET maintenance_work_mem='5 GB';
SET
postgres=# create index on items_small using hnsw (embedding
vector_cosine_ops) WITH (m=16, ef_construction=100);
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.
The culprit is this 'uint32' local variable that's used to store the size:
diff --git a/src/backend/storage/ipc/dsm_impl.c
b/src/backend/storage/ipc/dsm_impl.c
index 03aa47a104..8dd669e0ce 100644inline patch 1: src/backend/storage/ipc/dsm_impl.c (+0/-0 lines) download inline-1.patch--- a/src/backend/storage/ipc/dsm_impl.c+++ b/src/backend/storage/ipc/dsm_impl.c@@ -873,7 +873,7 @@ dsm_impl_mmap(dsm_op op, dsm_handle handle, Sizerequest_size,
* transferring data to the kernel.
*/
char *zbuffer = (char *) palloc0(ZBUFFER_SIZE);
- uint32 remaining = request_size;
+ Size remaining = request_size;
bool success = true;
/*
Barring objections, I'll commit and backpatch that shortly.
- Heikki
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-bugs@postgresql.org
Cc: noreply@postgresql.org, pgsql-bugs@lists.postgresql.org, hlinnaka@iki.fi
Subject: Re: BUG #18341: SIGBUS with > 4GB DSM and 'dynamic_shared_memory_type = mmap'
In-Reply-To: <18341-ce16599e7fd6228c@postgresql.org>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox