pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Justin Pryzby <pryzby@telsasoft.com>
To: Michael Paquier <michael@paquier.xyz>
Cc: Thomas Munro <thomas.munro@gmail.com>
Cc: David Steele <david@pgmasters.net>
Cc: Fujii Masao <masao.fujii@oss.nttdata.com>
Cc: Bruce Momjian <bruce@momjian.us>
Cc: Paul Guo <guopa@vmware.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Michael Brown <michael.brown@discourse.org>
Cc: pgsql-hackers@postgresql.org
Subject: Re: fdatasync performance problem with large number of DB files
Date: Fri, 4 Jun 2021 09:39:04 -0500
Message-ID: <20210604143904.GQ14099@telsasoft.com> (raw)
In-Reply-To: <YLnVEvpJ4+ObuUIv@paquier.xyz>
References: <CA+hUKGJyjpuqYq60=zhoMWL5cCJKR_V9FaJtoZUv6nJCdtkPVA@mail.gmail.com>
	<CA+hUKG+jOK-0ZbCAiTwPx8_ajfRs9cAgkFf4BodfQiE9G+3n1w@mail.gmail.com>
	<ca3983c4-0fb3-a64f-c533-ddba3e36b177@oss.nttdata.com>
	<cb8dc397-3376-3d2e-4be0-b986a1c17e82@oss.nttdata.com>
	<CA+hUKGKgj+SN6z91nVmOmTv2KYrG7VnAGdTkWdSjbOPghdtooQ@mail.gmail.com>
	<7716fafe-fcab-3faf-c337-d1e8c6ae6191@pgmasters.net>
	<CA+hUKG+z+ET2-o8cX5XBqPTNzNUXzMAZKP2YU8cgz53cdC65OA@mail.gmail.com>
	<20210526001359.GE3676@telsasoft.com>
	<20210529192321.GM2082@telsasoft.com>
	<YLnVEvpJ4+ObuUIv@paquier.xyz>

On Fri, Jun 04, 2021 at 04:24:02PM +0900, Michael Paquier wrote:
> On Sat, May 29, 2021 at 02:23:21PM -0500, Justin Pryzby wrote:
> > On Tue, May 25, 2021 at 07:13:59PM -0500, Justin Pryzby wrote:
> >> On Sat, Mar 20, 2021 at 12:16:27PM +1300, Thomas Munro wrote:
> >> > > > +     {
> >> > > > +             {"recovery_init_sync_method", PGC_POSTMASTER, ERROR_HANDLING_OPTIONS,
> >> > > > +                     gettext_noop("Sets the method for synchronizing the data directory before crash recovery."),
> >> > > > +             },
> >> 
> >> Is there any reason why this can't be PGC_SIGHUP ?
> > 
> > I can't see any reason why this is nontrivial.
> 
> I think that we had better let recovery_init_sync_method as
> PGC_POSTMASTER, to stay on the safe side.  SyncDataDirectory() only
> gets called now in the backend code by the startup process after a
> crash at the beginning of recovery, so switching to PGC_SIGHUP would
> have zero effect to begin with.  Now, let's not forget that
> SyncDataDirectory() is a published API, and if anything exterior were
> to call that, it does not seem right to me to make that its behavior
> reloadable at will.

You said switching to SIGHUP "would have zero effect"; but, actually it allows
an admin who's DB took a long time in recovery/startup to change the parameter
without shutting down the service.  This mitigates the downtime if it crashes
again.  I think that's at least 50% of how this feature might end up being
used.

It might be "safer" if fsync were PGC_POSTMASTER, but it's allowed to change at
runtime that parameter, which is much more widely applicable.  I've already
mentioned restart_after_crash, and remove_temp_files_after_crash.

-- 
Justin





view thread (49+ messages)  latest in thread

Message-ID: <20210604143904.GQ14099@telsasoft.com>
Permalink:  ../20210604143904.GQ14099@telsasoft.com/
Also on:    postgresql.org/message-id/20210604143904.GQ14099@telsasoft.com

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-hackers@postgresql.org
  Cc: pryzby@telsasoft.com, michael@paquier.xyz, thomas.munro@gmail.com, david@pgmasters.net, masao.fujii@oss.nttdata.com, bruce@momjian.us, guopa@vmware.com, tgl@sss.pgh.pa.us, michael.brown@discourse.org
  Subject: Re: fdatasync performance problem with large number of DB files
  In-Reply-To: <20210604143904.GQ14099@telsasoft.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox