Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1xRYdu-000000014up-1KdM for pgsql-bugs@arkaria.postgresql.org; Thu, 12 Nov 2026 17:31:22 +0000 Received: from localhost ([127.0.0.1]:47716 helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xRYds-0000000450q-0xfh for pgsql-bugs@arkaria.postgresql.org; Thu, 12 Nov 2026 17:31:20 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]:41676) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1xRALr-0000000BmoE-1ADH for pgsql-bugs@lists.postgresql.org; Wed, 11 Nov 2026 15:35:07 +0000 Received: from mahout.postgresql.org ([2001:4800:3e1:1::227]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1xRALq-00000000OWU-0USN for pgsql-bugs@lists.postgresql.org; Wed, 11 Nov 2026 15:35:06 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=postgresql.org; s=20171124; h=Message-ID:Date:Reply-To:Cc:From:To:Subject: Content-Transfer-Encoding:MIME-Version:Content-Type:Sender:Content-ID: Content-Description:In-Reply-To:References; bh=jnpT0shK0ivN38gNx4jNRCn0e8nk4OQVvHhm1FP6qgU=; b=qPbN14H8YDNMgAj0P38bsxv8AE VzS5lVU64szdrHjMfSVxd49rm9Y510k2UCcRwWooupJKAS04n4SuuSdOoXut4ZHt2IoqdWr57wWyk ssepeMARVnvhERe7W2odZVLy6Or6ZwdSSXHq25owYNKFxjx1wT2h+we7ywABpXAt8bCd1aRYvryDH nix9tt+v2KR836rcHk7NOKLdkb9w6GZzw2b0InGVFu4Rx017lsXbkwhCjkjZXN46y/h7G03gT8+Xg rcI6kVN2Un2etq7eIPnVI8+01Nqv0/jqPz9hKHrl5dlf5/o4FgUtzHuxCNVZRxt8KtYqi0oRiMvFK fiA6gDRQ==; Received: from wrigleys.postgresql.org ([2a02:16a8:dc51::60]) by mahout.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1xRALo-000000015Fz-3jP1 for pgsql-bugs@lists.postgresql.org; Wed, 11 Nov 2026 15:35:05 +0000 Received: from localhost ([127.0.0.1] helo=wrigleys.postgresql.org) by wrigleys.postgresql.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xRALn-00000002n3E-267d for pgsql-bugs@lists.postgresql.org; Wed, 11 Nov 2026 15:35:03 +0000 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Subject: BUG #19889: ShmemRequestHash underestimates storage for highly partitioned hashes To: pgsql-bugs@lists.postgresql.org From: PG Bug reporting form Cc: imchifan@mail.com Reply-To: imchifan@mail.com, pgsql-bugs@lists.postgresql.org Date: Wed, 11 Nov 2026 15:34:26 +0000 Message-ID: <19889-7b1b4e5720653305@postgresql.org> X-Auto-Response-Suppress: All Auto-Submitted: auto-generated List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk The following bug has been logged on the website: Bug reference: 19889 Logged by: Faye Email address: imchifan@mail.com PostgreSQL version: Unsupported/Unknown Operating system: Linux/amd64 Description: =20 ShmemRequestHash() reserves too little contiguous memory when a small nelems value is combined with HASH_PARTITION and a larger partition count. A request for one element with 512 partitions fails during initialization with "failed to initialize hash table", although the named shared-memory allocation is present. The request uses supported hash options and should initialize successfully. The apparent cause is that hash_estimate_size(nelems, entrysize) does not account for num_partitions, while init_htab() raises the bucket count to at least num_partitions. Here the estimate covers one 256-bucket segment, but initialization requires two. How to repeat: Save these four files in an empty directory. Set PG_CONFIG to pg_config from the PostgreSQL installation under test if necessary, then run ./run.sh as a non-root user permitted to install a module into that installation. Makefile: MODULES =3D partition_hash_probe PGFILEDESC =3D "partition_hash_probe - shared hash sizing probe" PG_CONFIG ?=3D pg_config PGXS :=3D $(shell $(PG_CONFIG) --pgxs) include $(PGXS) partition_hash_probe.c: #include "postgres.h" #include "fmgr.h" #include "storage/shmem.h" #include "utils/hsearch.h" PG_MODULE_MAGIC; typedef struct PartitionHashEntry { uint32 key; uint32 value; } PartitionHashEntry; static HTAB *partition_hash; static void request_partition_hash(void *arg) { ShmemRequestHash(.name =3D "partition_hash_probe", .nelems =3D 1, .ptr =3D &partition_hash, .hash_info.keysize =3D sizeof(uint32), .hash_info.entrysize =3D sizeof(PartitionHashEntry= ), .hash_info.num_partitions =3D 512, .hash_flags =3D HASH_ELEM | HASH_BLOBS | HASH_PARTITION, ); } static const ShmemCallbacks callbacks =3D { .request_fn =3D request_partition_hash, .flags =3D SHMEM_CALLBACKS_ALLOW_AFTER_STARTUP, }; void _PG_init(void) { RegisterShmemCallbacks(&callbacks); } test.sql: \set ON_ERROR_STOP on LOAD 'partition_hash_probe'; SELECT name, size > 0 AS allocated FROM pg_shmem_allocations WHERE name =3D 'partition_hash_probe'; run.sh: #!/bin/sh set -eu : "${PG_CONFIG:=3Dpg_config}" bindir=3D$($PG_CONFIG --bindir) work=3D$(mktemp -d "${TMPDIR:-/tmp}/partition-hash.XXXXXX") trap '"$bindir/pg_ctl" -D "$work/data" -m immediate stop >/dev/null 2>&1 || true' EXIT HUP INT TERM make PG_CONFIG=3D"$PG_CONFIG" make PG_CONFIG=3D"$PG_CONFIG" install "$bindir/initdb" -D "$work/data" --no-locale --auth=3Dtrust >/dev/null "$bindir/pg_ctl" -D "$work/data" -l "$work/server.log" -o "-k $work -c listen_addresses=3D''" start >/dev/null if "$bindir/psql" -h "$work" -d postgres -X -v ON_ERROR_STOP=3D1 -f test.sql >"$work/result.out" 2>&1; then echo "NOT REPRODUCED: LOAD succeeded" exit 1 fi cat "$work/result.out" grep -q 'failed to initialize hash table.*partition_hash_probe' "$work/result.out" echo "BUG REPRODUCED: estimator-sized partitioned hash failed initialization" The confirmed execution used a standalone backend. With the identifiers above, its relevant output is: ERROR: failed to initialize hash table "partition_hash_probe" The subsequent pg_shmem_allocations query showed the allocation left by the failed initialization: name | allocated ----------------------+----------- partition_hash_probe | t The supplied run.sh checks for the same initialization error and, when it finds it, prints: BUG REPRODUCED: estimator-sized partitioned hash failed initialization That line is the runner's oracle; it was not part of the standalone-backend output. On a corrected build, LOAD should succeed and the query should return allocated =3D t. A failed after-startup initialization can leave an unusable ShmemIndex entry and its static allocation behind, so the disposable server should be restarted or removed after the test. The issue was reproduced on PostgreSQL 20devel on Linux/amd64, compiled with GCC 11.4.0. PostgreSQL 18.6 predates the target shared-memory API and was not used to exercise this module. This occurs when a very small shared hash is requested with enough partitions to raise its initialization-time bucket minimum above the storage estimate.