agora inbox for pgsql-hackers@postgresql.org
help / color / mirror / Atom feedFrom: Heikki Linnakangas <hlinnakangas@vmware.com>
To: Mitsumasa KONDO <kondo.mitsumasa@gmail.com>
To: Fujii Masao <masao.fujii@gmail.com>
Cc: PostgreSQL-development <pgsql-hackers@postgresql.org>
Subject: Re: posix_fadvise() and pg_receivexlog
Date: Thu, 7 Aug 2014 11:02:31 +0300
Message-ID: <53E33297.2020706@vmware.com> (raw)
In-Reply-To: <CADupcHVhOfpiCQVAfT3PAEcPV8iPe8Z61QHZrcnbHCGShHGr4g@mail.gmail.com>
References: <CAHGQGwF4fG8mRB_Ly6toVgVnhNVx+dzsuDyk_j7sPUrLge5k_g@mail.gmail.com>
<53E27B13.50506@vmware.com>
<CAHGQGwEBO0F1LnMLyv5exap508cKwk6hBU-g8X2YmOKPaNUNMA@mail.gmail.com>
<CADupcHVhOfpiCQVAfT3PAEcPV8iPe8Z61QHZrcnbHCGShHGr4g@mail.gmail.com>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-hackers>
On 08/07/2014 10:10 AM, Mitsumasa KONDO wrote:
> 2014-08-07 13:47 GMT+09:00 Fujii Masao <masao.fujii@gmail.com>:
>
>> On Thu, Aug 7, 2014 at 3:59 AM, Heikki Linnakangas
>> <hlinnakangas@vmware.com> wrote:
>>> On 08/06/2014 08:39 PM, Fujii Masao wrote:
>>>> The WAL files that pg_receivexlog writes will not be re-read soon
>>>> basically,
>>>> so we can advise the OS to release any cached pages when WAL file is
>>>> closed. I feel inclined to change pg_receivexlog that way. Thought?
>>>
>>>
>>> -1. The OS should be smart enough to not thrash the cache by files that
>> are
>>> written sequentially and never read.
>>
> OS's buffer strategy is optimized for general situation. Do you forget OS
> hackers discussion last a half of year?
>
>> Yep, the OS should be so smart, but I'm not sure if it actually is. Maybe
>> not,
>> so I was thinking that posix_fadvise is called when the server closes WAL
>> file.
>
> That's right.
Well, I'd like to hear someone from the field complaining that
pg_receivexlog is thrashing the cache and thus reducing the performance
of some other process. Or a least a synthetic test case that
demonstrates that happening.
> By the way, does pg_receivexlog process have fsync() in every WAL commit?
It fsync's each file after finishing to write it. Ie. each WAL file is
fsync'd once.
> If yes, I think that we need no or less fsync() option for the better
> performance. It is general in NOSQL storages.
> If no, we need fsync() option for more getting reliability and data
> integrarity.
Hmm. An fsync=off style option might make sense, although I doubt the
one fsync at end of file is causing a performance problem for anyone in
practice. Haven't heard any complaints, anyway.
An option to fsync after every commit record might make sense if you use
pg_receivexlog with synchronous replication. Doing that would require
parsing the WAL, though, to see where the commit records are. But then
again, the fsync's wouldn't need to correspond to commit records. We
could fsync just before we go to sleep to wait for more WAL to be received.
- Heikki
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
view thread (607+ messages) latest in thread
Message-ID: <53E33297.2020706@vmware.com>
Permalink: ../53E33297.2020706@vmware.com/
Also on: postgresql.org/message-id/53E33297.2020706@vmware.com
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: hlinnakangas@vmware.com, kondo.mitsumasa@gmail.com, masao.fujii@gmail.com
Subject: Re: posix_fadvise() and pg_receivexlog
In-Reply-To: <53E33297.2020706@vmware.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox