Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1rOll2-006vsz-9z for pgsql-hackers@arkaria.postgresql.org; Sat, 13 Jan 2024 21:41:37 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1rOlkz-009hEE-OM for pgsql-hackers@arkaria.postgresql.org; Sat, 13 Jan 2024 21:41:33 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1rOlkz-009hE6-5U for pgsql-hackers@lists.postgresql.org; Sat, 13 Jan 2024 21:41:33 +0000 Received: from mail-io1-xd31.google.com ([2607:f8b0:4864:20::d31]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94.2) (envelope-from ) id 1rOlku-001OYK-Rr for pgsql-hackers@postgresql.org; Sat, 13 Jan 2024 21:41:31 +0000 Received: by mail-io1-xd31.google.com with SMTP id ca18e2360f4ac-7bed9c7d33fso184120539f.1 for ; Sat, 13 Jan 2024 13:41:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1705182087; x=1705786887; darn=postgresql.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=d3JggrJsZaPqI5lhNmpHHkL1V0E3Pbf1BRZS98OnGPo=; b=MRFJu4IfvLfhrB+4tLrI0IfhGVaWV/hJuo2da/AivJeQkFVB/eYsGNLswAk0+tJox+ yHnGnTXnBpdywDGaRx3nfYIj9RIGtVVuwp69lqB1hmVrQxzDsh3ZY19pzDuzoarv3QU9 RaavZtO7WFz77cU1+AdCtrsZ8gZtbDkOdf9KT9saB+j2dQJIefzahG1cv0+KrUsoxVMr QyDGgwy/1pdQphXrHHhaptmpw/uY2mgxX5K2DDT2nHEkc3FAezZyWWWS2B6AshbMFFpE q7Ia1SJrJPG4jBDWF+dhRu9v2ykUq5xrJECtIUiDfcCmsFnnKoW8YBnfHEPEQDyUXKqA EOMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705182087; x=1705786887; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=d3JggrJsZaPqI5lhNmpHHkL1V0E3Pbf1BRZS98OnGPo=; b=sQ8Os2SxCzCjEyphIQZXzdBdQ+FpAXmbwNHQQifKJ6L6MysvHevsdhmdcwPZig6NHw JAoiHDDJQw44F2MZl8w9jvzt2ndPh+2WwpM+EExugm4DX6Yvonjo+W2lxLA9qoMZGBZR kdN9NLnxwiHDdSGhUD8T8zh6vkTnTR6ANqQ2znGY8CNo63uZ3UeJebBw66ct2OaNK+WU 09ajmawKVnSUFm/i0kqCjRLM1PG6F4bGuQ1oglMFU8PcE5ku8sP0SrwRkXq7diGcriYw jjdaVhgyErFhYbRtNRd7vdxPgJhdRXAzu59mTLfLAS67WeRGC9dDsVIm5cCpjcVTZLec 0ZBg== X-Gm-Message-State: AOJu0Ywzs8qJD5+8Hzzs2iOKukmFdL2CrXDVQ5Zpl52OOnm8JYu8ewYh rAZJHmH2gRjudzPp7iDein373n09IX8= X-Google-Smtp-Source: AGHT+IHXL+u+URdndaIkwGAKIFEy3OQmrJi6m6SYFOQ26OveGfBNfiZ38CPLCjuYq9TYTUbNLqUbcg== X-Received: by 2002:a6b:5908:0:b0:7bb:b125:49a1 with SMTP id n8-20020a6b5908000000b007bbb12549a1mr3533175iob.41.1705182087164; Sat, 13 Jan 2024 13:41:27 -0800 (PST) Received: from nathanxps13 (162-195-168-172.lightspeed.stlsmo.sbcglobal.net. [162.195.168.172]) by smtp.gmail.com with ESMTPSA id z15-20020a6bc90f000000b007bf05f618f3sm1585548iof.55.2024.01.13.13.41.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 13 Jan 2024 13:41:26 -0800 (PST) Date: Sat, 13 Jan 2024 15:41:24 -0600 From: Nathan Bossart To: Abhijit Menon-Sen Cc: Bharath Rupireddy , Michael Paquier , Robert Haas , pgsql-hackers@postgresql.org Subject: Re: introduce dynamic shared memory registry Message-ID: <20240113214124.GA4162765@nathanxps13> References: <20240102224907.GA1246933@nathanxps13> <20240106163516.GB2435448@nathanxps13> <20240108171627.GA2611898@nathanxps13> <20240112172152.GA3803561@nathanxps13> <20240112194555.GA3804945@nathanxps13> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="huq684BweRXVnRxX" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20240112194555.GA3804945@nathanxps13> List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --huq684BweRXVnRxX Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit On Fri, Jan 12, 2024 at 01:45:55PM -0600, Nathan Bossart wrote: > On Fri, Jan 12, 2024 at 11:13:46PM +0530, Abhijit Menon-Sen wrote: >> At 2024-01-12 11:21:52 -0600, nathandbossart@gmail.com wrote: >>> + Each backend sould obtain a pointer to the reserved shared memory by >> >> sould → should > > D'oh. Thanks. > >>> + Add-ins can reserve LWLocks on server startup. Like with shared memory, >> >> (Would "As with shared memory" read better? Maybe, but then again maybe >> it should be left alone because you also write "Unlike with" elsewhere.) > > I think "As with shared memory..." sounds better here. Here is a new version of the patch set with these changes. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com --huq684BweRXVnRxX Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v7-0001-reorganize-shared-memory-and-lwlocks-documentatio.patch" From b56931edc4488a7376b27ba0e5519cc3a61b4899 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Thu, 11 Jan 2024 21:55:25 -0600 Subject: [PATCH v7 1/3] reorganize shared memory and lwlocks documentation --- doc/src/sgml/xfunc.sgml | 182 +++++++++++++++++++++++++--------------- 1 file changed, 114 insertions(+), 68 deletions(-) diff --git a/doc/src/sgml/xfunc.sgml b/doc/src/sgml/xfunc.sgml index 89116ae74c..82e1dadcca 100644 --- a/doc/src/sgml/xfunc.sgml +++ b/doc/src/sgml/xfunc.sgml @@ -3397,90 +3397,136 @@ CREATE FUNCTION make_array(anyelement) RETURNS anyarray - Shared Memory and LWLocks + Shared Memory - - Add-ins can reserve LWLocks and an allocation of shared memory on server - startup. The add-in's shared library must be preloaded by specifying - it in - shared_preload_libraries. - The shared library should register a shmem_request_hook - in its _PG_init function. This - shmem_request_hook can reserve LWLocks or shared memory. - Shared memory is reserved by calling: + + Requesting Shared Memory at Startup + + + Add-ins can reserve shared memory on server startup. To do so, the + add-in's shared library must be preloaded by specifying it in + shared_preload_libraries. + The shared library should also register a + shmem_request_hook in its + _PG_init function. This + shmem_request_hook can reserve shared memory by + calling: void RequestAddinShmemSpace(int size) - from your shmem_request_hook. - - - LWLocks are reserved by calling: + Each backend should obtain a pointer to the reserved shared memory by + calling: + +void *ShmemInitStruct(const char *name, Size size, bool *foundPtr) + + If this function sets foundPtr to + false, the caller should proceed to initialize the + contents of the reserved shared memory. If foundPtr + is set to true, the shared memory was already + initialized by another backend, and the caller need not initialize + further. + + + + To avoid race conditions, each backend should use the LWLock + AddinShmemInitLock when initializing its allocation + of shared memory, as shown here: + +static mystruct *ptr = NULL; +bool found; + +LWLockAcquire(AddinShmemInitLock, LW_EXCLUSIVE); +ptr = ShmemInitStruct("my struct name", size, &found); +if (!found) +{ + ... initialize contents of shared memory ... + ptr->locks = GetNamedLWLockTranche("my tranche name"); +} +LWLockRelease(AddinShmemInitLock); + + shmem_startup_hook provides a convenient place for the + initialization code, but it is not strictly required that all such code + be placed in this hook. Each backend will execute the registered + shmem_startup_hook shortly after it attaches to shared + memory. Note that add-ins should still acquire + AddinShmemInitLock within this hook, as shown in the + example above. + + + + An example of a shmem_request_hook and + shmem_startup_hook can be found in + contrib/pg_stat_statements/pg_stat_statements.c in + the PostgreSQL source tree. + + + + + + LWLocks + + + Requesting LWLocks at Startup + + + Add-ins can reserve LWLocks on server startup. As with shared memory, + the add-in's shared library must be preloaded by specifying it in + shared_preload_libraries, + and the shared library should register a + shmem_request_hook in its + _PG_init function. This + shmem_request_hook can reserve LWLocks by calling: void RequestNamedLWLockTranche(const char *tranche_name, int num_lwlocks) - from your shmem_request_hook. This will ensure that an array of - num_lwlocks LWLocks is available under the name - tranche_name. Use GetNamedLWLockTranche - to get a pointer to this array. - - - An example of a shmem_request_hook can be found in - contrib/pg_stat_statements/pg_stat_statements.c in the - PostgreSQL source tree. - - - There is another, more flexible method of obtaining LWLocks. First, - allocate a tranche_id from a shared counter by - calling: + This ensures that an array of num_lwlocks LWLocks is + available under the name tranche_name. A pointer to + this array can be obtained by calling: -int LWLockNewTrancheId(void) +LWLockPadded *GetNamedLWLockTranche(const char *tranche_name) - Next, each individual process using the tranche_id - should associate it with a tranche_name by calling: + + + + + Requesting LWLocks After Startup + + + There is another, more flexible method of obtaining LWLocks that can be + done after server startup and outside a + shmem_request_hook. To do so, first allocate a + tranche_id by calling: -void LWLockRegisterTranche(int tranche_id, const char *tranche_name) +int LWLockNewTrancheId(void) - It is also required to call LWLockInitialize once - per LWLock, passing the tranche_id as argument: + Next, initialize each LWLock, passing the new + tranche_id as an argument: void LWLockInitialize(LWLock *lock, int tranche_id) - A complete usage example of LWLockNewTrancheId, - LWLockInitialize and - LWLockRegisterTranche can be found in - contrib/pg_prewarm/autoprewarm.c in the - PostgreSQL source tree. - - - To avoid possible race-conditions, each backend should use the LWLock - AddinShmemInitLock when connecting to and initializing - its allocation of shared memory, as shown here: - -static mystruct *ptr = NULL; + Similar to shared memory, each backend should ensure that only one + process allocates a new tranche_id and initializes + each new LWLock. One way to do this is to only call these functions in + your shared memory initialization code with the + AddinShmemInitLock held exclusively. + -if (!ptr) -{ - bool found; - - LWLockAcquire(AddinShmemInitLock, LW_EXCLUSIVE); - ptr = ShmemInitStruct("my struct name", size, &found); - if (!found) - { - initialize contents of shmem area; - acquire any requested LWLocks using: - ptr->locks = GetNamedLWLockTranche("my tranche name"); - } - LWLockRelease(AddinShmemInitLock); -} + + Finally, each backend using the tranche_id should + associate it with a tranche_name by calling: + +void LWLockRegisterTranche(int tranche_id, const char *tranche_name) - - - It is convenient to use shmem_startup_hook which allows - placing all the code responsible for initializing shared memory in one - place. When using shmem_startup_hook the extension - still needs to acquire AddinShmemInitLock in order to - work properly on all the supported platforms. - + + + + A complete usage example of LWLockNewTrancheId, + LWLockInitialize, and + LWLockRegisterTranche can be found in + contrib/pg_prewarm/autoprewarm.c in the + PostgreSQL source tree. + + -- 2.25.1 --huq684BweRXVnRxX Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v7-0002-add-dsm-registry.patch" From bb4941207d5b64e9e5641a6f0ad234e0d199e498 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Wed, 11 Oct 2023 22:07:26 -0500 Subject: [PATCH v7 2/3] add dsm registry --- doc/src/sgml/xfunc.sgml | 49 ++++- src/backend/storage/ipc/Makefile | 1 + src/backend/storage/ipc/dsm_registry.c | 174 ++++++++++++++++++ src/backend/storage/ipc/ipci.c | 3 + src/backend/storage/ipc/meson.build | 1 + src/backend/storage/lmgr/lwlock.c | 4 + src/backend/storage/lmgr/lwlocknames.txt | 1 + .../utils/activity/wait_event_names.txt | 3 + src/include/storage/dsm_registry.h | 23 +++ src/include/storage/lwlock.h | 2 + src/test/modules/Makefile | 1 + src/test/modules/meson.build | 1 + src/test/modules/test_dsm_registry/.gitignore | 4 + src/test/modules/test_dsm_registry/Makefile | 23 +++ .../expected/test_dsm_registry.out | 14 ++ .../modules/test_dsm_registry/meson.build | 33 ++++ .../sql/test_dsm_registry.sql | 4 + .../test_dsm_registry--1.0.sql | 10 + .../test_dsm_registry/test_dsm_registry.c | 76 ++++++++ .../test_dsm_registry.control | 4 + src/tools/pgindent/typedefs.list | 3 + 21 files changed, 431 insertions(+), 3 deletions(-) create mode 100644 src/backend/storage/ipc/dsm_registry.c create mode 100644 src/include/storage/dsm_registry.h create mode 100644 src/test/modules/test_dsm_registry/.gitignore create mode 100644 src/test/modules/test_dsm_registry/Makefile create mode 100644 src/test/modules/test_dsm_registry/expected/test_dsm_registry.out create mode 100644 src/test/modules/test_dsm_registry/meson.build create mode 100644 src/test/modules/test_dsm_registry/sql/test_dsm_registry.sql create mode 100644 src/test/modules/test_dsm_registry/test_dsm_registry--1.0.sql create mode 100644 src/test/modules/test_dsm_registry/test_dsm_registry.c create mode 100644 src/test/modules/test_dsm_registry/test_dsm_registry.control diff --git a/doc/src/sgml/xfunc.sgml b/doc/src/sgml/xfunc.sgml index 82e1dadcca..fbeac2fd1d 100644 --- a/doc/src/sgml/xfunc.sgml +++ b/doc/src/sgml/xfunc.sgml @@ -3460,6 +3460,45 @@ LWLockRelease(AddinShmemInitLock); the PostgreSQL source tree. + + + Requesting Shared Memory After Startup + + + There is another, more flexible method of reserving shared memory that + can be done after server startup and outside a + shmem_request_hook. To do so, each backend that will + use the shared memory should obtain a pointer to it by calling: + +void *GetNamedDSMSegment(const char *name, size_t size, + void (*init_callback) (void *ptr), + bool *found) + + If a dynamic shared memory segment with the given name does not yet + exist, this function will allocate it and initialize it with the provided + init_callback callback function. If the segment has + already been allocated and initialized by another backend, this function + simply attaches the existing dynamic shared memory segment to the current + backend. + + + + Unlike shared memory reserved at server startup, there is no need to + acquire AddinShmemInitLock or otherwise take action + to avoid race conditions when reserving shared memory with + GetNamedDSMSegment. This function ensures that only + one backend allocates and initializes the segment and that all other + backends receive a pointer to the fully allocated and initialized + segment. + + + + A complete usage example of GetNamedDSMSegment can + be found in + src/test/modules/test_dsm_registry/test_dsm_registry.c + in the PostgreSQL source tree. + + @@ -3469,8 +3508,9 @@ LWLockRelease(AddinShmemInitLock); Requesting LWLocks at Startup - Add-ins can reserve LWLocks on server startup. As with shared memory, - the add-in's shared library must be preloaded by specifying it in + Add-ins can reserve LWLocks on server startup. As with shared memory + reserved at server startup, the add-in's shared library must be preloaded + by specifying it in shared_preload_libraries, and the shared library should register a shmem_request_hook in its @@ -3508,7 +3548,10 @@ void LWLockInitialize(LWLock *lock, int tranche_id) process allocates a new tranche_id and initializes each new LWLock. One way to do this is to only call these functions in your shared memory initialization code with the - AddinShmemInitLock held exclusively. + AddinShmemInitLock held exclusively. If using + GetNamedDSMSegment, calling these functions in the + init_callback callback function is sufficient to + avoid race conditions. diff --git a/src/backend/storage/ipc/Makefile b/src/backend/storage/ipc/Makefile index 6d5b921038..d8a1653eb6 100644 --- a/src/backend/storage/ipc/Makefile +++ b/src/backend/storage/ipc/Makefile @@ -12,6 +12,7 @@ OBJS = \ barrier.o \ dsm.o \ dsm_impl.o \ + dsm_registry.o \ ipc.o \ ipci.o \ latch.o \ diff --git a/src/backend/storage/ipc/dsm_registry.c b/src/backend/storage/ipc/dsm_registry.c new file mode 100644 index 0000000000..7deaa0d6c5 --- /dev/null +++ b/src/backend/storage/ipc/dsm_registry.c @@ -0,0 +1,174 @@ +/*------------------------------------------------------------------------- + * + * dsm_registry.c + * + * Functions for interfacing with the dynamic shared memory registry. This + * provides a way for libraries to use shared memory without needing to + * request it at startup time via a shmem_request_hook. + * + * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group + * Portions Copyright (c) 1994, Regents of the University of California + * + * IDENTIFICATION + * src/backend/storage/ipc/dsm_registry.c + * + *------------------------------------------------------------------------- + */ + +#include "postgres.h" + +#include "lib/dshash.h" +#include "storage/dsm_registry.h" +#include "storage/lwlock.h" +#include "storage/shmem.h" +#include "utils/memutils.h" + +typedef struct DSMRegistryCtxStruct +{ + dsa_handle dsah; + dshash_table_handle dshh; +} DSMRegistryCtxStruct; + +static DSMRegistryCtxStruct *DSMRegistryCtx; + +typedef struct DSMRegistryEntry +{ + char name[64]; + dsm_handle handle; +} DSMRegistryEntry; + +static const dshash_parameters dsh_params = { + offsetof(DSMRegistryEntry, handle), + sizeof(DSMRegistryEntry), + dshash_memcmp, + dshash_memhash, + LWTRANCHE_DSM_REGISTRY_HASH +}; + +static dsa_area *dsm_registry_dsa; +static dshash_table *dsm_registry_table; + +Size +DSMRegistryShmemSize(void) +{ + return MAXALIGN(sizeof(DSMRegistryCtxStruct)); +} + +void +DSMRegistryShmemInit(void) +{ + bool found; + + DSMRegistryCtx = (DSMRegistryCtxStruct *) + ShmemInitStruct("DSM Registry Data", + DSMRegistryShmemSize(), + &found); + + if (!found) + { + DSMRegistryCtx->dsah = DSA_HANDLE_INVALID; + DSMRegistryCtx->dshh = DSHASH_HANDLE_INVALID; + } +} + +/* + * Initialize or attach to the dynamic shared hash table that stores the DSM + * registry entries, if not already done. This must be called before accessing + * the table. + */ +static void +init_dsm_registry(void) +{ + /* Quick exit if we already did this. */ + if (dsm_registry_table) + return; + + /* Otherwise, use a lock to ensure only one process creates the table. */ + LWLockAcquire(DSMRegistryLock, LW_EXCLUSIVE); + + if (DSMRegistryCtx->dshh == DSHASH_HANDLE_INVALID) + { + /* Initialize dynamic shared hash table for registry. */ + dsm_registry_dsa = dsa_create(LWTRANCHE_DSM_REGISTRY_DSA); + dsa_pin(dsm_registry_dsa); + dsa_pin_mapping(dsm_registry_dsa); + dsm_registry_table = dshash_create(dsm_registry_dsa, &dsh_params, 0); + + /* Store handles in shared memory for other backends to use. */ + DSMRegistryCtx->dsah = dsa_get_handle(dsm_registry_dsa); + DSMRegistryCtx->dshh = dshash_get_hash_table_handle(dsm_registry_table); + } + else + { + /* Attach to existing dynamic shared hash table. */ + dsm_registry_dsa = dsa_attach(DSMRegistryCtx->dsah); + dsa_pin_mapping(dsm_registry_dsa); + dsm_registry_table = dshash_attach(dsm_registry_dsa, &dsh_params, + DSMRegistryCtx->dshh, 0); + } + + LWLockRelease(DSMRegistryLock); +} + +/* + * Initialize or attach a named DSM segment. + * + * This routine returns the address of the segment. init_callback is called to + * initialize the segment when it is first created. + */ +void * +GetNamedDSMSegment(const char *name, size_t size, + void (*init_callback) (void *ptr), bool *found) +{ + DSMRegistryEntry *entry; + MemoryContext oldcontext; + char name_padded[offsetof(DSMRegistryEntry, handle)] = {0}; + void *ret; + + Assert(name); + Assert(size); + Assert(found); + + if (strlen(name) >= offsetof(DSMRegistryEntry, handle)) + elog(ERROR, "DSM segment name too long"); + + /* Be sure any local memory allocated by DSM/DSA routines is persistent. */ + oldcontext = MemoryContextSwitchTo(TopMemoryContext); + + /* Connect to the registry. */ + init_dsm_registry(); + + strcpy(name_padded, name); + entry = dshash_find_or_insert(dsm_registry_table, name_padded, found); + if (!(*found)) + { + /* Initialize the segment. */ + dsm_segment *seg = dsm_create(size, 0); + + dsm_pin_segment(seg); + dsm_pin_mapping(seg); + entry->handle = dsm_segment_handle(seg); + ret = dsm_segment_address(seg); + + if (init_callback) + (*init_callback) (ret); + } + else if (!dsm_find_mapping(entry->handle)) + { + /* Attach to existing segment. */ + dsm_segment *seg = dsm_attach(entry->handle); + + dsm_pin_mapping(seg); + ret = dsm_segment_address(seg); + } + else + { + /* Return address of an already-attached segment. */ + ret = dsm_segment_address(dsm_find_mapping(entry->handle)); + } + + dshash_release_lock(dsm_registry_table, entry); + MemoryContextSwitchTo(oldcontext); + + return ret; +} diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c index e5119ed55d..fbc62b1563 100644 --- a/src/backend/storage/ipc/ipci.c +++ b/src/backend/storage/ipc/ipci.c @@ -40,6 +40,7 @@ #include "replication/walsender.h" #include "storage/bufmgr.h" #include "storage/dsm.h" +#include "storage/dsm_registry.h" #include "storage/ipc.h" #include "storage/pg_shmem.h" #include "storage/pmsignal.h" @@ -115,6 +116,7 @@ CalculateShmemSize(int *num_semaphores) size = add_size(size, hash_estimate_size(SHMEM_INDEX_SIZE, sizeof(ShmemIndexEnt))); size = add_size(size, dsm_estimate_size()); + size = add_size(size, DSMRegistryShmemSize()); size = add_size(size, BufferShmemSize()); size = add_size(size, LockShmemSize()); size = add_size(size, PredicateLockShmemSize()); @@ -289,6 +291,7 @@ CreateOrAttachShmemStructs(void) InitShmemIndex(); dsm_shmem_init(); + DSMRegistryShmemInit(); /* * Set up xlog, clog, and buffers diff --git a/src/backend/storage/ipc/meson.build b/src/backend/storage/ipc/meson.build index 08bdc718b8..5a936171f7 100644 --- a/src/backend/storage/ipc/meson.build +++ b/src/backend/storage/ipc/meson.build @@ -4,6 +4,7 @@ backend_sources += files( 'barrier.c', 'dsm.c', 'dsm_impl.c', + 'dsm_registry.c', 'ipc.c', 'ipci.c', 'latch.c', diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c index b4b989ac56..2f2de5a562 100644 --- a/src/backend/storage/lmgr/lwlock.c +++ b/src/backend/storage/lmgr/lwlock.c @@ -190,6 +190,10 @@ static const char *const BuiltinTrancheNames[] = { "LogicalRepLauncherDSA", /* LWTRANCHE_LAUNCHER_HASH: */ "LogicalRepLauncherHash", + /* LWTRANCHE_DSM_REGISTRY_DSA: */ + "DSMRegistryDSA", + /* LWTRANCHE_DSM_REGISTRY_HASH: */ + "DSMRegistryHash", }; StaticAssertDecl(lengthof(BuiltinTrancheNames) == diff --git a/src/backend/storage/lmgr/lwlocknames.txt b/src/backend/storage/lmgr/lwlocknames.txt index d621f5507f..ef8542de46 100644 --- a/src/backend/storage/lmgr/lwlocknames.txt +++ b/src/backend/storage/lmgr/lwlocknames.txt @@ -55,3 +55,4 @@ WrapLimitsVacuumLock 46 NotifyQueueTailLock 47 WaitEventExtensionLock 48 WALSummarizerLock 49 +DSMRegistryLock 50 diff --git a/src/backend/utils/activity/wait_event_names.txt b/src/backend/utils/activity/wait_event_names.txt index f625473ad4..6bcb1cca0c 100644 --- a/src/backend/utils/activity/wait_event_names.txt +++ b/src/backend/utils/activity/wait_event_names.txt @@ -329,6 +329,7 @@ WrapLimitsVacuum "Waiting to update limits on transaction id and multixact consu NotifyQueueTail "Waiting to update limit on NOTIFY message storage." WaitEventExtension "Waiting to read or update custom wait events information for extensions." WALSummarizer "Waiting to read or update WAL summarization state." +DSMRegistry "Waiting to read or update the dynamic shared memory registry." # # END OF PREDEFINED LWLOCKS (DO NOT CHANGE THIS LINE) @@ -367,6 +368,8 @@ PgStatsHash "Waiting for stats shared memory hash table access." PgStatsData "Waiting for shared memory stats data access." LogicalRepLauncherDSA "Waiting to access logical replication launcher's dynamic shared memory allocator." LogicalRepLauncherHash "Waiting to access logical replication launcher's shared hash table." +DSMRegistryDSA "Waiting to access dynamic shared memory registry's dynamic shared memory allocator." +DSMRegistryHash "Waiting to access dynamic shared memory registry's shared hash table." # # Wait Events - Lock diff --git a/src/include/storage/dsm_registry.h b/src/include/storage/dsm_registry.h new file mode 100644 index 0000000000..8e8a23ba60 --- /dev/null +++ b/src/include/storage/dsm_registry.h @@ -0,0 +1,23 @@ +/*------------------------------------------------------------------------- + * + * dsm_registry.h + * + * + * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group + * Portions Copyright (c) 1994, Regents of the University of California + * + * src/include/storage/dsm_registry.h + * + *------------------------------------------------------------------------- + */ +#ifndef DSM_REGISTRY_H +#define DSM_REGISTRY_H + +extern void *GetNamedDSMSegment(const char *name, size_t size, + void (*init_callback) (void *ptr), + bool *found); + +extern Size DSMRegistryShmemSize(void); +extern void DSMRegistryShmemInit(void); + +#endif /* DSM_REGISTRY_H */ diff --git a/src/include/storage/lwlock.h b/src/include/storage/lwlock.h index 167ae34208..50a65e046d 100644 --- a/src/include/storage/lwlock.h +++ b/src/include/storage/lwlock.h @@ -207,6 +207,8 @@ typedef enum BuiltinTrancheIds LWTRANCHE_PGSTATS_DATA, LWTRANCHE_LAUNCHER_DSA, LWTRANCHE_LAUNCHER_HASH, + LWTRANCHE_DSM_REGISTRY_DSA, + LWTRANCHE_DSM_REGISTRY_HASH, LWTRANCHE_FIRST_USER_DEFINED, } BuiltinTrancheIds; diff --git a/src/test/modules/Makefile b/src/test/modules/Makefile index 5d33fa6a9a..f656032589 100644 --- a/src/test/modules/Makefile +++ b/src/test/modules/Makefile @@ -18,6 +18,7 @@ SUBDIRS = \ test_custom_rmgrs \ test_ddl_deparse \ test_dsa \ + test_dsm_registry \ test_extensions \ test_ginpostinglist \ test_integerset \ diff --git a/src/test/modules/meson.build b/src/test/modules/meson.build index 00ff1d77d1..2c3b8d73bc 100644 --- a/src/test/modules/meson.build +++ b/src/test/modules/meson.build @@ -15,6 +15,7 @@ subdir('test_copy_callbacks') subdir('test_custom_rmgrs') subdir('test_ddl_deparse') subdir('test_dsa') +subdir('test_dsm_registry') subdir('test_extensions') subdir('test_ginpostinglist') subdir('test_integerset') diff --git a/src/test/modules/test_dsm_registry/.gitignore b/src/test/modules/test_dsm_registry/.gitignore new file mode 100644 index 0000000000..5dcb3ff972 --- /dev/null +++ b/src/test/modules/test_dsm_registry/.gitignore @@ -0,0 +1,4 @@ +# Generated subdirectories +/log/ +/results/ +/tmp_check/ diff --git a/src/test/modules/test_dsm_registry/Makefile b/src/test/modules/test_dsm_registry/Makefile new file mode 100644 index 0000000000..b13e99a354 --- /dev/null +++ b/src/test/modules/test_dsm_registry/Makefile @@ -0,0 +1,23 @@ +# src/test/modules/test_dsm_registry/Makefile + +MODULE_big = test_dsm_registry +OBJS = \ + $(WIN32RES) \ + test_dsm_registry.o +PGFILEDESC = "test_dsm_registry - test code for the DSM registry" + +EXTENSION = test_dsm_registry +DATA = test_dsm_registry--1.0.sql + +REGRESS = test_dsm_registry + +ifdef USE_PGXS +PG_CONFIG = pg_config +PGXS := $(shell $(PG_CONFIG) --pgxs) +include $(PGXS) +else +subdir = src/test/modules/test_dsm_registry +top_builddir = ../../../.. +include $(top_builddir)/src/Makefile.global +include $(top_srcdir)/contrib/contrib-global.mk +endif diff --git a/src/test/modules/test_dsm_registry/expected/test_dsm_registry.out b/src/test/modules/test_dsm_registry/expected/test_dsm_registry.out new file mode 100644 index 0000000000..8ffbd343a0 --- /dev/null +++ b/src/test/modules/test_dsm_registry/expected/test_dsm_registry.out @@ -0,0 +1,14 @@ +CREATE EXTENSION test_dsm_registry; +SELECT set_val_in_shmem(1236); + set_val_in_shmem +------------------ + +(1 row) + +\c +SELECT get_val_in_shmem(); + get_val_in_shmem +------------------ + 1236 +(1 row) + diff --git a/src/test/modules/test_dsm_registry/meson.build b/src/test/modules/test_dsm_registry/meson.build new file mode 100644 index 0000000000..a4045fea37 --- /dev/null +++ b/src/test/modules/test_dsm_registry/meson.build @@ -0,0 +1,33 @@ +# Copyright (c) 2024, PostgreSQL Global Development Group + +test_dsm_registry_sources = files( + 'test_dsm_registry.c', +) + +if host_system == 'windows' + test_dsm_registry_sources += rc_lib_gen.process(win32ver_rc, extra_args: [ + '--NAME', 'test_dsm_registry', + '--FILEDESC', 'test_dsm_registry - test code for the DSM registry',]) +endif + +test_dsm_registry = shared_module('test_dsm_registry', + test_dsm_registry_sources, + kwargs: pg_test_mod_args, +) +test_install_libs += test_dsm_registry + +test_install_data += files( + 'test_dsm_registry.control', + 'test_dsm_registry--1.0.sql', +) + +tests += { + 'name': 'test_dsm_registry', + 'sd': meson.current_source_dir(), + 'bd': meson.current_build_dir(), + 'regress': { + 'sql': [ + 'test_dsm_registry', + ], + }, +} diff --git a/src/test/modules/test_dsm_registry/sql/test_dsm_registry.sql b/src/test/modules/test_dsm_registry/sql/test_dsm_registry.sql new file mode 100644 index 0000000000..b3351be0a1 --- /dev/null +++ b/src/test/modules/test_dsm_registry/sql/test_dsm_registry.sql @@ -0,0 +1,4 @@ +CREATE EXTENSION test_dsm_registry; +SELECT set_val_in_shmem(1236); +\c +SELECT get_val_in_shmem(); diff --git a/src/test/modules/test_dsm_registry/test_dsm_registry--1.0.sql b/src/test/modules/test_dsm_registry/test_dsm_registry--1.0.sql new file mode 100644 index 0000000000..8c55b0919b --- /dev/null +++ b/src/test/modules/test_dsm_registry/test_dsm_registry--1.0.sql @@ -0,0 +1,10 @@ +/* src/test/modules/test_dsm_registry/test_dsm_registry--1.0.sql */ + +-- complain if script is sourced in psql, rather than via CREATE EXTENSION +\echo Use "CREATE EXTENSION test_dsm_registry" to load this file. \quit + +CREATE FUNCTION set_val_in_shmem(val INT) RETURNS VOID + AS 'MODULE_PATHNAME' LANGUAGE C; + +CREATE FUNCTION get_val_in_shmem() RETURNS INT + AS 'MODULE_PATHNAME' LANGUAGE C; diff --git a/src/test/modules/test_dsm_registry/test_dsm_registry.c b/src/test/modules/test_dsm_registry/test_dsm_registry.c new file mode 100644 index 0000000000..b46cd6c620 --- /dev/null +++ b/src/test/modules/test_dsm_registry/test_dsm_registry.c @@ -0,0 +1,76 @@ +/*-------------------------------------------------------------------------- + * + * test_dsm_registry.c + * Test the DSM registry + * + * Copyright (c) 2024, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/modules/test_dsm_registry/test_dsm_registry.c + * + * ------------------------------------------------------------------------- + */ +#include "postgres.h" + +#include "fmgr.h" +#include "storage/dsm_registry.h" +#include "storage/lwlock.h" + +PG_MODULE_MAGIC; + +typedef struct TestDSMRegistryStruct +{ + int val; + LWLock lck; +} TestDSMRegistryStruct; + +static TestDSMRegistryStruct *tdr_state; + +static void +tdr_init_shmem(void *ptr) +{ + TestDSMRegistryStruct *state = (TestDSMRegistryStruct *) ptr; + + LWLockInitialize(&state->lck, LWLockNewTrancheId()); + state->val = 0; +} + +static void +tdr_attach_shmem(void) +{ + bool found; + + tdr_state = GetNamedDSMSegment("test_dsm_registry", + sizeof(TestDSMRegistryStruct), + tdr_init_shmem, + &found); + LWLockRegisterTranche(tdr_state->lck.tranche, "test_dsm_registry"); +} + +PG_FUNCTION_INFO_V1(set_val_in_shmem); +Datum +set_val_in_shmem(PG_FUNCTION_ARGS) +{ + tdr_attach_shmem(); + + LWLockAcquire(&tdr_state->lck, LW_EXCLUSIVE); + tdr_state->val = PG_GETARG_UINT32(0); + LWLockRelease(&tdr_state->lck); + + PG_RETURN_VOID(); +} + +PG_FUNCTION_INFO_V1(get_val_in_shmem); +Datum +get_val_in_shmem(PG_FUNCTION_ARGS) +{ + int ret; + + tdr_attach_shmem(); + + LWLockAcquire(&tdr_state->lck, LW_SHARED); + ret = tdr_state->val; + LWLockRelease(&tdr_state->lck); + + PG_RETURN_UINT32(ret); +} diff --git a/src/test/modules/test_dsm_registry/test_dsm_registry.control b/src/test/modules/test_dsm_registry/test_dsm_registry.control new file mode 100644 index 0000000000..813f099889 --- /dev/null +++ b/src/test/modules/test_dsm_registry/test_dsm_registry.control @@ -0,0 +1,4 @@ +comment = 'Test code for the DSM registry' +default_version = '1.0' +module_pathname = '$libdir/test_dsm_registry' +relocatable = true diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index f582eb59e7..b7e73736e9 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -610,6 +610,8 @@ DropSubscriptionStmt DropTableSpaceStmt DropUserMappingStmt DropdbStmt +DSMRegistryCtxStruct +DSMRegistryEntry DumpComponents DumpId DumpOptions @@ -2799,6 +2801,7 @@ Tcl_NotifierProcs Tcl_Obj Tcl_Time TempNamespaceStatus +TestDSMRegistryStruct TestDecodingData TestDecodingTxnData TestSpec -- 2.25.1 --huq684BweRXVnRxX Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v7-0003-use-dsm-registry-for-pg_prewarm.patch" From c914c3850c5ffed043fa360150cdb312947bfa7a Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Tue, 26 Dec 2023 22:25:45 -0600 Subject: [PATCH v7 3/3] use dsm registry for pg_prewarm --- contrib/pg_prewarm/autoprewarm.c | 46 +++++++++++--------------------- 1 file changed, 15 insertions(+), 31 deletions(-) diff --git a/contrib/pg_prewarm/autoprewarm.c b/contrib/pg_prewarm/autoprewarm.c index 9085a409db..9ea6c2252a 100644 --- a/contrib/pg_prewarm/autoprewarm.c +++ b/contrib/pg_prewarm/autoprewarm.c @@ -32,12 +32,12 @@ #include "access/xact.h" #include "catalog/pg_class.h" #include "catalog/pg_type.h" -#include "miscadmin.h" #include "pgstat.h" #include "postmaster/bgworker.h" #include "postmaster/interrupt.h" #include "storage/buf_internals.h" #include "storage/dsm.h" +#include "storage/dsm_registry.h" #include "storage/fd.h" #include "storage/ipc.h" #include "storage/latch.h" @@ -95,8 +95,6 @@ static void apw_start_database_worker(void); static bool apw_init_shmem(void); static void apw_detach_shmem(int code, Datum arg); static int apw_compare_blockinfo(const void *p, const void *q); -static void autoprewarm_shmem_request(void); -static shmem_request_hook_type prev_shmem_request_hook = NULL; /* Pointer to shared-memory state. */ static AutoPrewarmSharedState *apw_state = NULL; @@ -140,26 +138,11 @@ _PG_init(void) MarkGUCPrefixReserved("pg_prewarm"); - prev_shmem_request_hook = shmem_request_hook; - shmem_request_hook = autoprewarm_shmem_request; - /* Register autoprewarm worker, if enabled. */ if (autoprewarm) apw_start_leader_worker(); } -/* - * Requests any additional shared memory required for autoprewarm. - */ -static void -autoprewarm_shmem_request(void) -{ - if (prev_shmem_request_hook) - prev_shmem_request_hook(); - - RequestAddinShmemSpace(MAXALIGN(sizeof(AutoPrewarmSharedState))); -} - /* * Main entry point for the leader autoprewarm process. Per-database workers * have a separate entry point. @@ -767,6 +750,16 @@ autoprewarm_dump_now(PG_FUNCTION_ARGS) PG_RETURN_INT64((int64) num_blocks); } +static void +apw_init_state(void *ptr) +{ + AutoPrewarmSharedState *state = (AutoPrewarmSharedState *) ptr; + + LWLockInitialize(&state->lock, LWLockNewTrancheId()); + state->bgworker_pid = InvalidPid; + state->pid_using_dumpfile = InvalidPid; +} + /* * Allocate and initialize autoprewarm related shared memory, if not already * done, and set up backend-local pointer to that state. Returns true if an @@ -777,19 +770,10 @@ apw_init_shmem(void) { bool found; - LWLockAcquire(AddinShmemInitLock, LW_EXCLUSIVE); - apw_state = ShmemInitStruct("autoprewarm", - sizeof(AutoPrewarmSharedState), - &found); - if (!found) - { - /* First time through ... */ - LWLockInitialize(&apw_state->lock, LWLockNewTrancheId()); - apw_state->bgworker_pid = InvalidPid; - apw_state->pid_using_dumpfile = InvalidPid; - } - LWLockRelease(AddinShmemInitLock); - + apw_state = GetNamedDSMSegment("autoprewarm", + sizeof(AutoPrewarmSharedState), + apw_init_state, + &found); LWLockRegisterTranche(apw_state->lock.tranche, "autoprewarm"); return found; -- 2.25.1 --huq684BweRXVnRxX--