pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Fujii Masao <masao.fujii@oss.nttdata.com>
To: Thomas Munro <thomas.munro@gmail.com>
To: Michael Brown <michael.brown@discourse.org>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: Re: fdatasync performance problem with large number of DB files
Date: Thu, 11 Mar 2021 10:00:37 +0900
Message-ID: <cfc60fae-dfac-eb49-4bab-9e90bc633588@oss.nttdata.com> (raw)
In-Reply-To: <CA+hUKGLgb4ZMV6xQrZb8N5ax0g+EZR=b0dtg9pPJmsBtaun1-A@mail.gmail.com>
References: <11bc2bb7-ecb5-3ad0-b39f-df632734cd81@discourse.org>
	<CA+hUKGLt7zoFcJemmXzL=ZB3Rdse6Ds_Mm05G5Ogai_yZRp2=A@mail.gmail.com>
	<CA+hUKGLgb4ZMV6xQrZb8N5ax0g+EZR=b0dtg9pPJmsBtaun1-A@mail.gmail.com>



On 2021/03/11 8:30, Thomas Munro wrote:
> On Thu, Mar 11, 2021 at 11:38 AM Thomas Munro <thomas.munro@gmail.com> wrote:
>> On Thu, Mar 11, 2021 at 11:01 AM Michael Brown
>> <michael.brown@discourse.org> wrote:
>>> * is there a knob missing we can configure?
>>> * can we get an opt-in knob to use a single sync() call instead of a
>>> recursive fsync()?
>>> * would you be open to merging a patch providing said knob?
>>> * is there something else we missed?
>>
>> As discussed on that other thread, I don't think sync() is an option
>> (it doesn't wait on all OSes or in the standard and it doesn't report
>> errors).  syncfs() on Linux 5.8+ looks like a good candidate though,
>> and I think we'd consider a patch like that.  I mean, I even posted
>> one[1] in that other thread.  There will of course be cases where
>> that's slower (small database sharing filesystem with other software
>> that has a lot of dirty data to write back).
> 
> Thinking about this some more, if you were to propose a patch like
> that syncfs() one but make it a configurable option, I'd personally be
> in favour of trying to squeeze it into v14.  Others might object on
> commitfest procedural grounds, I dunno, but I think this is a real
> operational issue and that's a fairly simple and localised change.

+1 to push this kind of change into v14!!

> I've run into a couple of users who have just commented that recursive
> fsync() code out!

BTW, we can skip that recursive fsync() by disabling fsync GUC even without
commenting out the code?

Regards,

-- 
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION





view thread (49+ messages)  latest in thread

Message-ID: <cfc60fae-dfac-eb49-4bab-9e90bc633588@oss.nttdata.com>
Permalink:  ../cfc60fae-dfac-eb49-4bab-9e90bc633588@oss.nttdata.com/
Also on:    postgresql.org/message-id/cfc60fae-dfac-eb49-4bab-9e90bc633588@oss.nttdata.com

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-hackers@postgresql.org
  Cc: masao.fujii@oss.nttdata.com, thomas.munro@gmail.com, michael.brown@discourse.org
  Subject: Re: fdatasync performance problem with large number of DB files
  In-Reply-To: <cfc60fae-dfac-eb49-4bab-9e90bc633588@oss.nttdata.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox