From: Justin Pryzby <pryzby@telsasoft.com>
To: Michael Paquier <michael@paquier.xyz>
Cc: Thomas Munro <thomas.munro@gmail.com>
Cc: David Steele <david@pgmasters.net>
Cc: Fujii Masao <masao.fujii@oss.nttdata.com>
Cc: Bruce Momjian <bruce@momjian.us>
Cc: Paul Guo <guopa@vmware.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Michael Brown <michael.brown@discourse.org>
Cc: pgsql-hackers@postgresql.org
Subject: Re: fdatasync performance problem with large number of DB files
Date: Fri, 4 Jun 2021 09:39:04 -0500
Message-ID: <20210604143904.GQ14099@telsasoft.com> (raw)
In-Reply-To: <YLnVEvpJ4+ObuUIv@paquier.xyz>
References: <CA+hUKGJyjpuqYq60=zhoMWL5cCJKR_V9FaJtoZUv6nJCdtkPVA@mail.gmail.com>
<CA+hUKG+jOK-0ZbCAiTwPx8_ajfRs9cAgkFf4BodfQiE9G+3n1w@mail.gmail.com>
<ca3983c4-0fb3-a64f-c533-ddba3e36b177@oss.nttdata.com>
<cb8dc397-3376-3d2e-4be0-b986a1c17e82@oss.nttdata.com>
<CA+hUKGKgj+SN6z91nVmOmTv2KYrG7VnAGdTkWdSjbOPghdtooQ@mail.gmail.com>
<7716fafe-fcab-3faf-c337-d1e8c6ae6191@pgmasters.net>
<CA+hUKG+z+ET2-o8cX5XBqPTNzNUXzMAZKP2YU8cgz53cdC65OA@mail.gmail.com>
<20210526001359.GE3676@telsasoft.com>
<20210529192321.GM2082@telsasoft.com>
<YLnVEvpJ4+ObuUIv@paquier.xyz>
On Fri, Jun 04, 2021 at 04:24:02PM +0900, Michael Paquier wrote:
> On Sat, May 29, 2021 at 02:23:21PM -0500, Justin Pryzby wrote:
> > On Tue, May 25, 2021 at 07:13:59PM -0500, Justin Pryzby wrote:
> >> On Sat, Mar 20, 2021 at 12:16:27PM +1300, Thomas Munro wrote:
> >> > > > + {
> >> > > > + {"recovery_init_sync_method", PGC_POSTMASTER, ERROR_HANDLING_OPTIONS,
> >> > > > + gettext_noop("Sets the method for synchronizing the data directory before crash recovery."),
> >> > > > + },
> >>
> >> Is there any reason why this can't be PGC_SIGHUP ?
> >
> > I can't see any reason why this is nontrivial.
>
> I think that we had better let recovery_init_sync_method as
> PGC_POSTMASTER, to stay on the safe side. SyncDataDirectory() only
> gets called now in the backend code by the startup process after a
> crash at the beginning of recovery, so switching to PGC_SIGHUP would
> have zero effect to begin with. Now, let's not forget that
> SyncDataDirectory() is a published API, and if anything exterior were
> to call that, it does not seem right to me to make that its behavior
> reloadable at will.
You said switching to SIGHUP "would have zero effect"; but, actually it allows
an admin who's DB took a long time in recovery/startup to change the parameter
without shutting down the service. This mitigates the downtime if it crashes
again. I think that's at least 50% of how this feature might end up being
used.
It might be "safer" if fsync were PGC_POSTMASTER, but it's allowed to change at
runtime that parameter, which is much more widely applicable. I've already
mentioned restart_after_crash, and remove_temp_files_after_crash.
--
Justin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: pryzby@telsasoft.com, michael@paquier.xyz, thomas.munro@gmail.com, david@pgmasters.net, masao.fujii@oss.nttdata.com, bruce@momjian.us, guopa@vmware.com, tgl@sss.pgh.pa.us, michael.brown@discourse.org
Subject: Re: fdatasync performance problem with large number of DB files
In-Reply-To: <20210604143904.GQ14099@telsasoft.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox