From: Alexander Korotkov <akorotkov@postgresql.org>
To: pgsql-committers@lists.postgresql.org
Subject: pgsql: Process sync requests incrementally in AbsorbSyncRequests
Date: Sun, 27 Jul 2025 12:23:12 +0000
Message-ID: <E1ug0PH-000yp4-30@gemulon.postgresql.org> (raw)
Process sync requests incrementally in AbsorbSyncRequests
If the number of sync requests is big enough, the palloc() call in
AbsorbSyncRequests() will attempt to allocate more than 1 GB of memory,
resulting in failure. This can lead to an infinite loop in the checkpointer
process, as it repeatedly fails to absorb the pending requests.
This commit introduces the following changes to cope with this problem:
1. Turn pending checkpointer requests array in shared memory into a bounded
ring buffer.
2. Limit maximum ring buffer size to 10M items.
3. Make AbsorbSyncRequests() process requests incrementally in 10K batches.
Even #2 makes the whole queue size fit the maximum palloc() size of 1 GB.
of continuous lock holding.
This commit is for master only. Simpler fix, which just limits a request
queue size to 10M, will be backpatched.
Reported-by: Ekaterina Sokolova <e.sokolova@postgrespro.ru>
Discussion: https://postgr.es/m/db4534f83a22a29ab5ee2566ad86ca92%40postgrespro.ru
Author: Maxim Orlov <orlovmg@gmail.com>
Co-authored-by: Xuneng Zhou <xunengzhou@gmail.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-by: Alexander Korotkov <aekorotkov@gmail.com>
Branch
------
master
Details
-------
https://git.postgresql.org/pg/commitdiff/258bf0a2ea8ff86257f750018bfd44397ce7e554
Modified Files
--------------
src/backend/postmaster/checkpointer.c | 155 +++++++++++++++++++++++++---------
1 file changed, 114 insertions(+), 41 deletions(-)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-committers@postgresql.org
Cc: akorotkov@postgresql.org, pgsql-committers@lists.postgresql.org
Subject: Re: pgsql: Process sync requests incrementally in AbsorbSyncRequests
In-Reply-To: <E1ug0PH-000yp4-30@gemulon.postgresql.org>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox