Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1rO8wJ-001mAJ-JW for pgsql-hackers@arkaria.postgresql.org; Fri, 12 Jan 2024 04:14:40 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1rO8wI-00HIJ4-LG for pgsql-hackers@arkaria.postgresql.org; Fri, 12 Jan 2024 04:14:38 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1rO8wI-00HIIH-5G for pgsql-hackers@lists.postgresql.org; Fri, 12 Jan 2024 04:14:38 +0000 Received: from mail-io1-xd29.google.com ([2607:f8b0:4864:20::d29]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94.2) (envelope-from ) id 1rO8wE-0017Tg-Vt for pgsql-hackers@postgresql.org; Fri, 12 Jan 2024 04:14:36 +0000 Received: by mail-io1-xd29.google.com with SMTP id ca18e2360f4ac-7bed9c7d33fso132084239f.1 for ; Thu, 11 Jan 2024 20:14:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1705032873; x=1705637673; darn=postgresql.org; h=content-disposition:mime-version:message-id:subject:to:from:date :from:to:cc:subject:date:message-id:reply-to; bh=F8Y5sx6Qp4PSWT14Y+xG7NQ8jgQx8fKlaluTxObz9ck=; b=Bcydjf9M9nfgT8rhQj9O0UpBIl13RA0OBxas8wsRQxBXJMJHJPjwNNT059+eyQY/NL AeYu3McUVb2h7nVANkHosOgJupZ9pB/iOLVCRozjb3Oh4lGrsIw+O+cL9XalnIfZ4yxP TGGI/FiHNMP36MKJu+9Tg1RxtOLxMZSOY0U6y3XL23y2g3c1V27O3RO4JtyWZSmM++JX wh6t1MU8XnPaWhJ5LaS4uQQEBxwH2/5LT0LkqZbULwU290OdzeWTgdWARyOlP0oKm3oe 5TatKl5Eg6LDEpvgkes5GHGuUCWxylvR4qBv7fzPkxKAJ/oafvhdfZunFk7rxi6Jc6GZ ahPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705032873; x=1705637673; h=content-disposition:mime-version:message-id:subject:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=F8Y5sx6Qp4PSWT14Y+xG7NQ8jgQx8fKlaluTxObz9ck=; b=Gf+8KeAwTPBRU7apzLDJWLe2/MvMSQArlVeJQ3dEfvQBQEyuB2kXBTelOYtRBQ52cl Pk8Q86AGxnO9Ejs+cEEtVfgz9Mojsea66HAix6AxM7unKRUYpSy5cJr7NMCZ6xzSrsAb CExvX0v8sInu7KyeXbAujSOEzAWwYROaDxxzi7auA5RV26/w7I4A0HJYUrBvQE+QJt8k 39/xBkmbwmoccR3UOsz/8srZ1SAS1MnWomb/HpHrsWPUkrwb1+Eu87PxBEJmByd3BOXC eeubvfTWf9BpBrdTAunIDazadJKewWYe9qaWcCWU04JvebgtDEfvDzv+yUbLvUR4jP3/ 0kzA== X-Gm-Message-State: AOJu0Yzs+lnmJSuTMbUVyPBGep6pVmiH0rxuqzQ+G0a2wpEB97HhEhYk 1vvaW5HWOZPCKLa1tcIVPT9TXrIiXuM= X-Google-Smtp-Source: AGHT+IFtxf1Thbd9AVfg9QMQNkb/EL57FUXxwfZuKw7gXFH8TJvH/ZT3BaFQvV5C7ZRckleM+S+wdg== X-Received: by 2002:a5e:d614:0:b0:7bc:775e:6a85 with SMTP id w20-20020a5ed614000000b007bc775e6a85mr770443iom.42.1705032872696; Thu, 11 Jan 2024 20:14:32 -0800 (PST) Received: from nathanxps13 (162-195-168-172.lightspeed.stlsmo.sbcglobal.net. [162.195.168.172]) by smtp.gmail.com with ESMTPSA id n26-20020a056602341a00b007bef3151723sm610089ioz.16.2024.01.11.20.14.31 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Jan 2024 20:14:32 -0800 (PST) Date: Thu, 11 Jan 2024 22:14:30 -0600 From: Nathan Bossart To: pgsql-hackers@postgresql.org Subject: reorganize "Shared Memory and LWLocks" section of docs Message-ID: <20240112041430.GA3557928@nathanxps13> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="Nq2Wo0NMKNjxTN9z" Content-Disposition: inline List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --Nq2Wo0NMKNjxTN9z Content-Type: text/plain; charset=us-ascii Content-Disposition: inline I recently began trying to write documentation for the dynamic shared memory registry feature [0], and I noticed that the "Shared Memory and LWLocks" section of the documentation might need some improvement. At least, I felt that it would be hard to add any new content to this section without making it very difficult to follow. Concretely, I am proposing breaking it into two sections: one for shared memory and one for LWLocks. Furthermore, the LWLocks section will be split into two: one for requesting locks at server startup and one for requesting locks after server startup. I intend to also split the shared memory section into at-startup and after-startup sections if/when the dynamic shared memory registry feature is committed. Besides this restructuring, I felt that certain parts of this documentation could benefit from rephrasing and/or additional detail. Thoughts? [0] https://postgr.es/m/20231205034647.GA2705267%40nathanxps13 -- Nathan Bossart Amazon Web Services: https://aws.amazon.com --Nq2Wo0NMKNjxTN9z Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v1-0001-reorganize-shared-memory-and-lwlocks-documentatio.patch" From dc5c1b37ced883021f4a92a17aa50b9d80d73fc6 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Thu, 11 Jan 2024 21:55:25 -0600 Subject: [PATCH v1 1/1] reorganize shared memory and lwlocks documentation --- doc/src/sgml/xfunc.sgml | 180 +++++++++++++++++++++++++--------------- 1 file changed, 112 insertions(+), 68 deletions(-) diff --git a/doc/src/sgml/xfunc.sgml b/doc/src/sgml/xfunc.sgml index 89116ae74c..a758384a9a 100644 --- a/doc/src/sgml/xfunc.sgml +++ b/doc/src/sgml/xfunc.sgml @@ -3397,90 +3397,134 @@ CREATE FUNCTION make_array(anyelement) RETURNS anyarray - Shared Memory and LWLocks + Shared Memory - - Add-ins can reserve LWLocks and an allocation of shared memory on server - startup. The add-in's shared library must be preloaded by specifying - it in - shared_preload_libraries. - The shared library should register a shmem_request_hook - in its _PG_init function. This - shmem_request_hook can reserve LWLocks or shared memory. - Shared memory is reserved by calling: + + Requesting Shared Memory at Startup + + + Add-ins can reserve shared memory on server startup. To do so, the + add-in's shared library must be preloaded by specifying it in + shared_preload_libraries. + The shared library should also register a + shmem_request_hook in its + _PG_init function. This + shmem_request_hook can reserve shared memory by + calling: void RequestAddinShmemSpace(int size) - from your shmem_request_hook. - - - LWLocks are reserved by calling: + Each backend sould obtain a pointer to the reserved shared memory by + calling: + +void *ShmemInitStruct(const char *name, Size size, bool *foundPtr) + + If this function sets foundPtr to + false, the caller should proceed to initialize the + contents of the reserved shared memory. If foundPtr + is set to true, the shared memory was already + initialized by another backend, and the caller need not initialize + further. + + + + To avoid race conditions, each backend should use the LWLock + AddinShmemInitLock when initializing its allocation + of shared memory, as shown here: + +static mystruct *ptr = NULL; +bool found; + +LWLockAcquire(AddinShmemInitLock, LW_EXCLUSIVE); +ptr = ShmemInitStruct("my struct name", size, &found); +if (!found) +{ + ... initialize contents of shared memory ... + ptr->locks = GetNamedLWLockTranche("my tranche name"); +} +LWLockRelease(AddinShmemInitLock); + + shmem_startup_hook provides a convenient place for the + initialization code, but it is not strictly required that all such code + be placed in this hook. Any registered + shmem_startup_hook will be executed shortly after each + backend attaches to shared memory. Note that add-ins should still + acquire AddinShmemInitLock within this hook, as + shown in the example above. + + + + An example of a shmem_request_hook and + shmem_startup_hook can be found in + contrib/pg_stat_statements/pg_stat_statements.c in + the PostgreSQL source tree. + + + + + + LWLocks + + + Requesting LWLocks at Startup + + + Add-ins can reserve LWLocks on server startup. Like with shared memory, + the add-in's shared library must be preloaded by specifying it in + shared_preload_libraries, + and the shared library should register a + shmem_request_hook in its + _PG_init function. This + shmem_request_hook can reserve LWLocks by calling: void RequestNamedLWLockTranche(const char *tranche_name, int num_lwlocks) - from your shmem_request_hook. This will ensure that an array of - num_lwlocks LWLocks is available under the name - tranche_name. Use GetNamedLWLockTranche - to get a pointer to this array. - - - An example of a shmem_request_hook can be found in - contrib/pg_stat_statements/pg_stat_statements.c in the - PostgreSQL source tree. - - - There is another, more flexible method of obtaining LWLocks. First, - allocate a tranche_id from a shared counter by - calling: + This ensures that an array of num_lwlocks LWLocks is + available under the name tranche_name. A pointer to + this array can be obtained by calling: -int LWLockNewTrancheId(void) +LWLockPadded *GetNamedLWLockTranche(const char *tranche_name) - Next, each individual process using the tranche_id - should associate it with a tranche_name by calling: + + + + + Requesting LWLocks After Startup + + + There is another, more flexible method of obtaining LWLocks that can be + done after server startup and outside a + shmem_request_hook. To do so, first allocate a + tranche_id by calling: -void LWLockRegisterTranche(int tranche_id, const char *tranche_name) +int LWLockNewTrancheId(void) - It is also required to call LWLockInitialize once - per LWLock, passing the tranche_id as argument: + Next, initialize each LWLock, passing the new + tranche_id as an argument: void LWLockInitialize(LWLock *lock, int tranche_id) - A complete usage example of LWLockNewTrancheId, - LWLockInitialize and - LWLockRegisterTranche can be found in - contrib/pg_prewarm/autoprewarm.c in the - PostgreSQL source tree. - - - To avoid possible race-conditions, each backend should use the LWLock - AddinShmemInitLock when connecting to and initializing - its allocation of shared memory, as shown here: + Finally, each backend using the tranche_id should + associate it with a tranche_name by calling: -static mystruct *ptr = NULL; - -if (!ptr) -{ - bool found; - - LWLockAcquire(AddinShmemInitLock, LW_EXCLUSIVE); - ptr = ShmemInitStruct("my struct name", size, &found); - if (!found) - { - initialize contents of shmem area; - acquire any requested LWLocks using: - ptr->locks = GetNamedLWLockTranche("my tranche name"); - } - LWLockRelease(AddinShmemInitLock); -} +void LWLockRegisterTranche(int tranche_id, const char *tranche_name) - - - It is convenient to use shmem_startup_hook which allows - placing all the code responsible for initializing shared memory in one - place. When using shmem_startup_hook the extension - still needs to acquire AddinShmemInitLock in order to - work properly on all the supported platforms. - + Similar to shared memory, each backend should ensure that only one + process allocates a new tranche_id + (LWLockNewTrancheId) and initializes each new LWLock + (LWLockInitialize). One way to do this is to only + call these functions in your shared memory initialization code with the + AddinShmemInitLock held exclusively. + + + + A complete usage example of LWLockNewTrancheId, + LWLockInitialize, and + LWLockRegisterTranche can be found in + contrib/pg_prewarm/autoprewarm.c in the + PostgreSQL source tree. + + -- 2.25.1 --Nq2Wo0NMKNjxTN9z--