Date: Fri, 17 May 2024 11:44:52 -0500
From: Nathan Bossart <nathandbossart@gmail.com>
To: "Imseih (AWS), Sami" <simseih@amazon.com>
Cc: Justin Pryzby <pryzby@telsasoft.com>,
	"pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Subject: problems with "Shared Memory and Semaphores" section of docs
Message-ID: <20240517164452.GA1914161@nathanxps13>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Archived-At: 
 <https://www.postgresql.org/message-id/20240517164452.GA1914161%40nathanxps13>
Precedence: bulk

(moving to a new thread)

On Thu, May 16, 2024 at 09:16:46PM -0500, Nathan Bossart wrote:
> On Thu, May 16, 2024 at 04:37:10PM +0000, Imseih (AWS), Sami wrote:
>> Also, Not sure if I am mistaken here, but the "+ 5" in the existing docs
>> seems wrong.
>>  
>> If it refers to NUM_AUXILIARY_PROCS defined in 
>> include/storage/proc.h, it should a "6"
>> 
>> #define NUM_AUXILIARY_PROCS             6
>> 
>> This is not a consequence of this patch, and can be dealt with
>> In a separate thread if my understanding is correct.
> 
> Ha, I think it should actually be "+ 7"!  The value is calculated as
> 
> 	MaxConnections + autovacuum_max_workers + 1 + max_worker_processes + max_wal_senders + 6
> 
> Looking at the history, this documentation tends to be wrong quite often.
> In v9.2, the checkpointer was introduced, and these formulas were not
> updated.  In v9.3, background worker processes were introduced, and the
> formulas were still not updated.  Finally, in v9.6, it was fixed in commit
> 597f7e3.  Then, in v14, the archiver process was made an auxiliary process
> (commit d75288f), making the formulas out-of-date again.  And in v17, the
> WAL summarizer was added.
> 
> On top of this, IIUC you actually need even more semaphores if your system
> doesn't support atomics, and from a quick skim this doesn't seem to be
> covered in this documentation.

A couple of other problems I noticed:

* max_wal_senders is missing from this sentence:

    When using System V semaphores,
    <productname>PostgreSQL</productname> uses one semaphore per allowed connection
    (<xref linkend="guc-max-connections"/>), allowed autovacuum worker process
    (<xref linkend="guc-autovacuum-max-workers"/>) and allowed background
    process (<xref linkend="guc-max-worker-processes"/>), in sets of 16.

* AFAICT the discussion about the formulas in the paragraphs following the
  table doesn't explain the reason for the constant.

* IMHO the following sentence is difficult to decipher, and I can't tell if
  it actually matches the formula in the table:

    The maximum number of semaphores in the system
    is set by <varname>SEMMNS</varname>, which consequently must be at least
    as high as <varname>max_connections</varname> plus
    <varname>autovacuum_max_workers</varname> plus <varname>max_wal_senders</varname>,
    plus <varname>max_worker_processes</varname>, plus one extra for each 16
    allowed connections plus workers (see the formula in <xref
    linkend="sysvipc-parameters"/>).

At a bare minimum, we should probably fix the obvious problems, but I
wonder if we could simplify this section a bit, too.  If the exact values
are important, maybe we could introduce more GUCs like
shared_memory_size_in_huge_pages that can be consulted (instead of
requiring users to break out their calculators).

-- 
Nathan Bossart
Amazon Web Services: https://aws.amazon.com