From: Nathan Bossart <nathandbossart@gmail.com>
To: Andres Freund <andres@anarazel.de>
Cc: Bossart, Nathan <bossartn@amazon.com>
Cc: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Cc: Maxim Orlov <orlovmg@gmail.com>
Cc: Amul Sul <sulamul@gmail.com>
Cc: Bruce Momjian <bruce@momjian.us>
Cc: Robert Haas <robertmhaas@gmail.com>
Cc: pgsql-hackers@postgresql.org <pgsql-hackers@postgresql.org>
Subject: Re: O(n) tasks cause lengthy startups and checkpoints
Date: Fri, 18 Feb 2022 08:44:54 -0800
Message-ID: <20220218164454.GA3403811@nathanxps13> (raw)
In-Reply-To: <20220217231247.llfqjvvuk6rkawel@alap3.anarazel.de>
References: <20220217005057.GA3201038@nathanxps13>
<20220217015052.2y3wxibeommk37ey@alap3.anarazel.de>
<20220217041404.GA3243546@nathanxps13>
<20220217065938.x2esfdppzypegn5j@alap3.anarazel.de>
<20220217182337.GA3247866@nathanxps13>
<20220217192709.eyf2cnymhxqsals7@alap3.anarazel.de>
<20220217210022.GA3248793@nathanxps13>
<20220217222829.lndzwq72fh4v34c2@alap3.anarazel.de>
<20220217225838.GA3292179@nathanxps13>
<20220217231247.llfqjvvuk6rkawel@alap3.anarazel.de>
On Thu, Feb 17, 2022 at 03:12:47PM -0800, Andres Freund wrote:
>> > The improvements around deleting temporary files and serialized snapshots
>> > afaict don't require a dedicated process - they're only relevant during
>> > startup. We could use the approach of renaming the directory out of the way as
>> > done in this patchset but perform the cleanup in the startup process after
>> > we're up.
>>
>> Perhaps this is a good place to start. As I mentioned above, IME the
>> temporary file cleanup is the most common problem, so I think even getting
>> that one fixed would be a huge improvement.
>
> Cool.
Hm. How should this work for standbys? I can think of the following
options:
1. Do temporary file cleanup in the postmaster (as it does today).
2. Pause after allowing connections to clean up temporary files.
3. Do small amounts of temporary file cleanup whenever there is an
opportunity during recovery.
4. Wait until recovery completes before cleaning up temporary files.
I'm not too thrilled about any of these options.
--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: nathandbossart@gmail.com, andres@anarazel.de, bossartn@amazon.com, bharath.rupireddyforpostgres@gmail.com, orlovmg@gmail.com, sulamul@gmail.com, bruce@momjian.us, robertmhaas@gmail.com
Subject: Re: O(n) tasks cause lengthy startups and checkpoints
In-Reply-To: <20220218164454.GA3403811@nathanxps13>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox