MIME-Version: 1.0
References: 
 <CAExHW5vM1bneLYfg0wGeAa=52UiJ3z4vKd3AJ72X8Fw6k3KKrg@mail.gmail.com>
 <CAExHW5uEK+eeG7e2g6uWh7POrFpfp+dqfaa=_3miMN17zgeaJw@mail.gmail.com>
 <CAExHW5vz+PUHHUuzGRwtyx-mPLQk3nCZXxrFqnruRadEFrO5Xg@mail.gmail.com>
 <CAExHW5so6VSxBC-1V=35229Z1+dw5vhw8HxHg9ry7UzceKcXzA@mail.gmail.com>
 <26c766d6-db0f-43d3-a618-44f8d40a3121@iki.fi>
 <CA+TgmoaoSha9KnSdrA7DDMYD9+Uor4OPEaQ88bbEC3QuVE5Mcg@mail.gmail.com>
 <62b8dc23-8f6a-4cac-91ff-f74bb5bc159a@iki.fi>
 <CA+TgmoboxAaS6MCRf__LRhf8Ji3B0O9_=d7KzRS12QhOuocnTQ@mail.gmail.com>
 <c7b4dafc-7a6c-43e4-8b9e-20807396590b@iki.fi>
 <CA+Tgmob7E3CGDoEBB3mwfk+v1CgcOG8yL0VHje2gy9Ts=e1Zxw@mail.gmail.com>
 <f8fc6f77-292d-4225-9f54-382c39da5ebc@iki.fi>
 <CA+TgmobqwCHp1w6xkTJmZiq4C8Pr0O3tNUvSFZcAgYLkvF45bg@mail.gmail.com>
 <8a6799be-bd42-49fb-8914-856c97bb1977@iki.fi>
 <CA+Tgmob8ot7qam7a650nx_4jyEoFLs8s_kPMc1+EB7-dr56=WA@mail.gmail.com>
 <cc3ff976-c303-45cd-9a8d-a90483170e37@iki.fi>
 <CA+TgmoauqYXm8iA3FGRAVKxYShUxWBiS_MSLmQpTrmO7wNHamw@mail.gmail.com>
 <113724ab-0028-493f-9605-6e8570f0939f@iki.fi>
In-Reply-To: <113724ab-0028-493f-9605-6e8570f0939f@iki.fi>
From: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Date: Tue, 24 Mar 2026 21:02:00 +0530
Message-ID: 
 <CAExHW5uTNWOSxJDWQAUnS0tZawob2_J3dRAtc67NHNZ98X4_xA@mail.gmail.com>
Subject: Re: Better shared data structure management and resizable shared data
 structures
To: Heikki Linnakangas <hlinnaka@iki.fi>
Cc: Robert Haas <robertmhaas@gmail.com>, Andres Freund <andres@anarazel.de>,
	pgsql-hackers <pgsql-hackers@postgresql.org>, chaturvedipalak1911@gmail.com
Content-Type: multipart/mixed; boundary="000000000000098aaa064dc6db70"
Archived-At: 
 <https://www.postgresql.org/message-id/CAExHW5uTNWOSxJDWQAUnS0tZawob2_J3dRAtc67NHNZ98X4_xA%40mail.gmail.com>
Precedence: bulk

--000000000000098aaa064dc6db70
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Sun, Mar 22, 2026 at 5:44=E2=80=AFAM Heikki Linnakangas <hlinnaka@iki.fi=
> wrote:
>
>
> I split this into more incremental patches. The first few are just
> refactorings that are probably useful on their own.

Here are some comments on the patches

0001
+
+# Enable pg_stat_statements to test restart of shared_preload_libraries.
+$node->append_conf(
+ 'postgresql.conf',
+ qq{shared_preload_libraries =3D 'pg_stat_statements'
+pg_stat_statements.max =3D 50000
+compute_query_id =3D 'regress'
+});
+

In order to make sure that the shared memory and LWLocks for
pg_stat_statements are initialized after crash restart, we need to at
least query pg_stat_statements after the restart or do something so
that it's shared memory is used. Also, we can check whether the shared
memory structures are created by querying pg_shmem_allocations after
the restart.

0002
void
InitShmemAllocator(PGShmemHeader *seghdr)
{

The new ShmemIndex initialization code is cleaner and more
straightforward. It avoids the recursive nature of ShmemInitHash.
However with this change it's hard to keep track of all the
initialization steps and their dependencies. Attached is a patch that
makes small adjustments to the code to make it more clear.

Use of variable hash_size is actually misleading since it's not the
size of the hash table but the expected/max number of entries in it.
Removing it makes code more readable.

0003, 0005, 0006 is straight forward, no comments. Usually these
patches make the code more readable. I will review it more when I see
the patches that use this refactoring.

0004 I traced back the placement of CreateLWLock in history. It feels
like the patch is moving it to its intended place since the function
was introduced. It needs a comment to explain why the function is not
called from CreateOrAttachShmemStructs where other in-memory
structures are allocated.

0007 without using new APIs is not necessarily a win. So I would
suggest committing it along with the refactoring patch.

If we are going to commit these patches separately, I would suggest
squashing all predicate.c patches into one commit.

I will continue from 0008 tomorrow.

--=20
Best Wishes,
Ashutosh Bapat

--000000000000098aaa064dc6db70
Content-Type: application/octet-stream;
	name="0002_adjustments.patch.no_cibot"
Content-Disposition: attachment; filename="0002_adjustments.patch.no_cibot"
Content-Transfer-Encoding: base64
Content-ID: <f_mn4f3pxv0>
X-Attachment-Id: f_mn4f3pxv0

ZGlmZiAtLWdpdCBhL3NyYy9iYWNrZW5kL3N0b3JhZ2UvaXBjL3NobWVtLmMgYi9zcmMvYmFja2Vu
ZC9zdG9yYWdlL2lwYy9zaG1lbS5jCmluZGV4IGRjZTM1NWU2NjgzLi43YTlkNWM2MTk4YiAxMDA2
NDQKLS0tIGEvc3JjL2JhY2tlbmQvc3RvcmFnZS9pcGMvc2htZW0uYworKysgYi9zcmMvYmFja2Vu
ZC9zdG9yYWdlL2lwYy9zaG1lbS5jCkBAIC0xMjUsMTEgKzEyNSwxNCBAQCB2b2lkCiBJbml0U2ht
ZW1BbGxvY2F0b3IoUEdTaG1lbUhlYWRlciAqc2VnaGRyKQogewogCVNpemUJCW9mZnNldDsKLQlp
bnQ2NAkJaGFzaF9zaXplOwogCUhBU0hDVEwJCWluZm87CiAJaW50CQkJaGFzaF9mbGFnczsKIAlz
aXplX3QJCXNpemU7CiAKKyNpZm5kZWYgRVhFQ19CQUNLRU5ECisJQXNzZXJ0KCFJc1VuZGVyUG9z
dG1hc3Rlcik7CisjZW5kaWYKKwogCUFzc2VydChzZWdoZHIgIT0gTlVMTCk7CiAKIAkvKgpAQCAt
MTM5LDEwICsxNDIsNiBAQCBJbml0U2htZW1BbGxvY2F0b3IoUEdTaG1lbUhlYWRlciAqc2VnaGRy
KQogCUFzc2VydChzZWdoZHIgPT0gKHZvaWQgKikgTUFYQUxJR04oc2VnaGRyKSk7CiAJQXNzZXJ0
KHNlZ2hkci0+Y29udGVudF9vZmZzZXQgPT0gTUFYQUxJR04oc2VnaGRyLT5jb250ZW50X29mZnNl
dCkpOwogCi0JU2htZW1TZWdIZHIgPSBzZWdoZHI7Ci0JU2htZW1CYXNlID0gc2VnaGRyOwotCVNo
bWVtRW5kID0gKGNoYXIgKikgU2htZW1CYXNlICsgc2VnaGRyLT50b3RhbHNpemU7Ci0KIAkvKgog
CSAqIEFsbG9jYXRpb25zIGFmdGVyIHRoaXMgcG9pbnQgc2hvdWxkIGdvIHRocm91Z2ggU2htZW1B
bGxvYywgd2hpY2gKIAkgKiBleHBlY3RzIHRvIGFsbG9jYXRlIGV2ZXJ5dGhpbmcgb24gY2FjaGUg
bGluZSBib3VuZGFyaWVzLiAgTWFrZSBzdXJlIHRoZQpAQCAtMTU1LDE4ICsxNTQsMjUgQEAgSW5p
dFNobWVtQWxsb2NhdG9yKFBHU2htZW1IZWFkZXIgKnNlZ2hkcikKIAkJCQkgZXJybXNnKCJvdXQg
b2Ygc2hhcmVkIG1lbW9yeSAoJXp1IGJ5dGVzIHJlcXVlc3RlZCkiLAogCQkJCQkJb2Zmc2V0KSkp
OwogCisJLyoKKwkgKiBJbiBwb3N0bWFzdGVyIG9yIHN0YW5kLWFsb25lIGJhY2tlbmQsIGluaXRp
YWxpemUgdGhlIHNoYXJlZCBtZW1vcnkKKwkgKiBhbGxvY2F0b3IgYW5kIHRoZSBzcGlubG9jayBz
byB0aGF0IHdlIGNhbiBhbGxvY2F0ZSBzaGFyZWQgbWVtb3J5IGZvcgorCSAqIFNobWVtSW5kZXgg
dXNpbmcgU2htZW1BbGxvYygpLiBJbiBhIHJlZ3VsYXIgYmFja2VuZCBqdXN0IHNldCB1cCB0aGUK
KwkgKiBwb2ludGVycyByZXF1aXJlZCBieSBTaG1lbUFsbG9jKCkuCisJICovCiAJU2htZW1BbGxv
Y2F0b3IgPSAoU2htZW1BbGxvY2F0b3JEYXRhICopICgoY2hhciAqKSBzZWdoZHIgKyBzZWdoZHIt
PmNvbnRlbnRfb2Zmc2V0KTsKLQlTaG1lbUxvY2sgPSAmU2htZW1BbGxvY2F0b3ItPnNobWVtX2xv
Y2s7CiAKLSNpZm5kZWYgRVhFQ19CQUNLRU5ECi0JQXNzZXJ0KCFJc1VuZGVyUG9zdG1hc3Rlcik7
Ci0jZW5kaWYKIAlpZiAoIUlzVW5kZXJQb3N0bWFzdGVyKQogCXsKIAkJU3BpbkxvY2tJbml0KCZT
aG1lbUFsbG9jYXRvci0+c2htZW1fbG9jayk7CiAJCVNobWVtQWxsb2NhdG9yLT5mcmVlX29mZnNl
dCA9IG9mZnNldDsKIAl9CiAKKwlTaG1lbUxvY2sgPSAmU2htZW1BbGxvY2F0b3ItPnNobWVtX2xv
Y2s7CisJU2htZW1TZWdIZHIgPSBzZWdoZHI7CisJU2htZW1CYXNlID0gc2VnaGRyOworCVNobWVt
RW5kID0gKGNoYXIgKikgU2htZW1CYXNlICsgc2VnaGRyLT50b3RhbHNpemU7CisKIAkvKgogCSAq
IENyZWF0ZSAob3IgYXR0YWNoIHRvKSB0aGUgc2hhcmVkIG1lbW9yeSBpbmRleCBvZiBzaG1lbSBh
cmVhcy4KIAkgKgpAQCAtMTc0LDExICsxODAsOSBAQCBJbml0U2htZW1BbGxvY2F0b3IoUEdTaG1l
bUhlYWRlciAqc2VnaGRyKQogCSAqIHVzZSBTaG1lbUluaXRIYXNoKCkgaGVyZSBiZWNhdXNlIGl0
IHJlbGllcyBvbiBTaG1lbUluZGV4IGJlaW5nIGFscmVhZHkKIAkgKiBpbml0aWFsaXplZC4KIAkg
Ki8KLQloYXNoX3NpemUgPSBTSE1FTV9JTkRFWF9TSVpFOwotCiAJaW5mby5rZXlzaXplID0gU0hN
RU1fSU5ERVhfS0VZU0laRTsKIAlpbmZvLmVudHJ5c2l6ZSA9IHNpemVvZihTaG1lbUluZGV4RW50
KTsKLQlpbmZvLmRzaXplID0gaW5mby5tYXhfZHNpemUgPSBoYXNoX3NlbGVjdF9kaXJzaXplKGhh
c2hfc2l6ZSk7CisJaW5mby5kc2l6ZSA9IGluZm8ubWF4X2RzaXplID0gaGFzaF9zZWxlY3RfZGly
c2l6ZShTSE1FTV9JTkRFWF9TSVpFKTsKIAlpbmZvLmFsbG9jID0gU2htZW1BbGxvY05vRXJyb3I7
CiAJaGFzaF9mbGFncyA9IEhBU0hfRUxFTSB8IEhBU0hfU1RSSU5HUyB8IEhBU0hfU0hBUkVEX01F
TSB8IEhBU0hfQUxMT0MgfCBIQVNIX0RJUlNJWkU7CiAJaWYgKCFJc1VuZGVyUG9zdG1hc3RlcikK
QEAgLTE4OSw3ICsxOTMsNyBAQCBJbml0U2htZW1BbGxvY2F0b3IoUEdTaG1lbUhlYWRlciAqc2Vn
aGRyKQogCWVsc2UKIAkJaGFzaF9mbGFncyB8PSBIQVNIX0FUVEFDSDsKIAlpbmZvLmhjdGwgPSBT
aG1lbUFsbG9jYXRvci0+aW5kZXg7Ci0JU2htZW1JbmRleCA9IGhhc2hfY3JlYXRlKCJTaG1lbUlu
ZGV4IiwgaGFzaF9zaXplLCAmaW5mbywgaGFzaF9mbGFncyk7CisJU2htZW1JbmRleCA9IGhhc2hf
Y3JlYXRlKCJTaG1lbUluZGV4IiwgU0hNRU1fSU5ERVhfU0laRSwgJmluZm8sIGhhc2hfZmxhZ3Mp
OwogCUFzc2VydChTaG1lbUluZGV4ICE9IE5VTEwpOwogfQogCg==
--000000000000098aaa064dc6db70--