pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  

help / color / mirror / Atom feed
From: Andres Freund <andres@anarazel.de>
To: Kevin Grittner <kgrittn@gmail.com>
Cc: Thomas Munro <thomas.munro@enterprisedb.com>
Cc: Dmitry Dolgov <9erthalion6@gmail.com>
Cc: Robert Haas <robertmhaas@gmail.com>
Cc: PostgreSQL-development <pgsql-hackers@postgresql.org>
Cc: Shawn Debnath <sdn@amazon.com>
Subject: Re: Refactoring the checkpointer's fsync request queue
Date: Tue, 22 Jan 2019 13:01:52 -0800
Message-ID: <20190122210152.gfeak6basvbvfkon@alap3.anarazel.de> (raw)
In-Reply-To: <CACjxUsOfv2-AyiRESKjiwzGjwOPjjWUsrNsZmkHrqHw97-+w2A@mail.gmail.com>
References: <CAEepm=1TDor1xHkYtpqEpuu5kpoU-DbLGX49scCY_zY-t3H-ug@mail.gmail.com>
	<CA+q6zcV=p=1G2Z8itJ9=tAn=b8ha3pO4RQCMt9YXOEutRV33YQ@mail.gmail.com>
	<CAEepm=2qBbQXGeekhCY+W8m=fy-+eU_FAUzRCSPFKuZ2rKG1LQ@mail.gmail.com>
	<CAEepm=1uqZLcTqLz_+egZRYotnTvzDK-jOGZdN8QhL=7nixY5Q@mail.gmail.com>
	<CAEepm=1mp+7PPY+sKJ4cvs6hba0NnWeed00oJ=EWry0gUgnJ_A@mail.gmail.com>
	<CACjxUsNAtKw-ARB=OEPde5j3e18Xza5xsBBsBrZxJw5nK1LLgQ@mail.gmail.com>
	<20190122181702.jueacsyffjwla3bb@alap3.anarazel.de>
	<CACjxUsMVJAKtq8UU+xCRVVpeSGb-+u6jyGaOXCx1Qw+qz3d=Nw@mail.gmail.com>
	<20190122203832.hckrvue3q5s75luv@alap3.anarazel.de>
	<CACjxUsOfv2-AyiRESKjiwzGjwOPjjWUsrNsZmkHrqHw97-+w2A@mail.gmail.com>

Hi,

On 2019-01-22 14:53:11 -0600, Kevin Grittner wrote:
> On Tue, Jan 22, 2019 at 2:38 PM Andres Freund <andres@anarazel.de> wrote:
> 
> > close() doesn't trigger an fsync() in general
> 
> What sort of a performance hit was observed when testing the addition
> of fsync or fdatasync before any PostgreSQL close() of a writable
> file, or have we not yet checked that?

I briefly played with it, and it was so atrocious (as in, less than
something like 0.2x the throughput) that I didn't continue far down that
path.  Two ways I IIRC (and it's really just memory) I tried were:

a) Short lived connections that do a bunch of writes to files each. That
   turns each disconnect into an fsync of most files.
b) Workload with > max_files_per_process files (IIRC I just used a bunch
   of larger tables with a few indexes each) in a read/write workload
   that's a bit larger than shared buffers. That lead to most file
   closes being integrity writes, with obvious downsides.

Greetings,

Andres Freund




view thread (87+ messages)  latest in thread

Message-ID: <20190122210152.gfeak6basvbvfkon@alap3.anarazel.de>
Permalink:  ../20190122210152.gfeak6basvbvfkon@alap3.anarazel.de/
Also on:    postgresql.org/message-id/20190122210152.gfeak6basvbvfkon@alap3.anarazel.de

 · 

reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-hackers@postgresql.org
  Cc: andres@anarazel.de, kgrittn@gmail.com, thomas.munro@enterprisedb.com, 9erthalion6@gmail.com, robertmhaas@gmail.com, sdn@amazon.com
  Subject: Re: Refactoring the checkpointer's fsync request queue
  In-Reply-To: <20190122210152.gfeak6basvbvfkon@alap3.anarazel.de>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox