Date: Tue, 22 Jan 2019 13:01:52 -0800
From: Andres Freund <andres@anarazel.de>
To: Kevin Grittner <kgrittn@gmail.com>
Cc: Thomas Munro <thomas.munro@enterprisedb.com>,
	Dmitry Dolgov <9erthalion6@gmail.com>,
	Robert Haas <robertmhaas@gmail.com>,
	PostgreSQL-development <pgsql-hackers@postgresql.org>,
	Shawn Debnath <sdn@amazon.com>
Subject: Re: Refactoring the checkpointer's fsync request queue
Message-ID: <20190122210152.gfeak6basvbvfkon@alap3.anarazel.de>
References: 
 <CAEepm=1TDor1xHkYtpqEpuu5kpoU-DbLGX49scCY_zY-t3H-ug@mail.gmail.com>
 <CA+q6zcV=p=1G2Z8itJ9=tAn=b8ha3pO4RQCMt9YXOEutRV33YQ@mail.gmail.com>
 <CAEepm=2qBbQXGeekhCY+W8m=fy-+eU_FAUzRCSPFKuZ2rKG1LQ@mail.gmail.com>
 <CAEepm=1uqZLcTqLz_+egZRYotnTvzDK-jOGZdN8QhL=7nixY5Q@mail.gmail.com>
 <CAEepm=1mp+7PPY+sKJ4cvs6hba0NnWeed00oJ=EWry0gUgnJ_A@mail.gmail.com>
 <CACjxUsNAtKw-ARB=OEPde5j3e18Xza5xsBBsBrZxJw5nK1LLgQ@mail.gmail.com>
 <20190122181702.jueacsyffjwla3bb@alap3.anarazel.de>
 <CACjxUsMVJAKtq8UU+xCRVVpeSGb-+u6jyGaOXCx1Qw+qz3d=Nw@mail.gmail.com>
 <20190122203832.hckrvue3q5s75luv@alap3.anarazel.de>
 <CACjxUsOfv2-AyiRESKjiwzGjwOPjjWUsrNsZmkHrqHw97-+w2A@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: 
 <CACjxUsOfv2-AyiRESKjiwzGjwOPjjWUsrNsZmkHrqHw97-+w2A@mail.gmail.com>
Precedence: bulk

Hi,

On 2019-01-22 14:53:11 -0600, Kevin Grittner wrote:
> On Tue, Jan 22, 2019 at 2:38 PM Andres Freund <andres@anarazel.de> wrote:
> 
> > close() doesn't trigger an fsync() in general
> 
> What sort of a performance hit was observed when testing the addition
> of fsync or fdatasync before any PostgreSQL close() of a writable
> file, or have we not yet checked that?

I briefly played with it, and it was so atrocious (as in, less than
something like 0.2x the throughput) that I didn't continue far down that
path.  Two ways I IIRC (and it's really just memory) I tried were:

a) Short lived connections that do a bunch of writes to files each. That
   turns each disconnect into an fsync of most files.
b) Workload with > max_files_per_process files (IIRC I just used a bunch
   of larger tables with a few indexes each) in a read/write workload
   that's a bit larger than shared buffers. That lead to most file
   closes being integrity writes, with obvious downsides.

Greetings,

Andres Freund