pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  

help / color / mirror / Atom feed
From: Andrew Gierth <andrew@tao11.riddles.org.uk>
To: Thomas Munro <thomas.munro@enterprisedb.com>
Cc: Andres Freund <andres@anarazel.de>
Cc: Robert Haas <robertmhaas@gmail.com>
Cc: Tobias Oberstein <tobias.oberstein@gmail.com>
Cc: pgsql-hackers\@postgresql.org <pgsql-hackers@postgresql.org>
Subject: Re: [HACKERS] lseek/read/write overhead becomes visible at scale ..
Date: Mon, 16 Apr 2018 07:13:30 +0100
Message-ID: <87604rg0n0.fsf@news-spur.riddles.org.uk> (raw)
In-Reply-To: <CAEepm=3BGNbWaznawm3LVk69ucKqXdAcb-aboG8cK375GcsMdQ@mail.gmail.com>
References: <20170124171841.2n2nzqkuetbivzta@alap3.anarazel.de>
	<f94252f1-c860-573e-332a-b367e6f64ba1@gmail.com>
	<20170124174158.nk6htg7jjmqia7hr@alap3.anarazel.de>
	<422b4e6c-b7f0-90e0-6f70-389b2d50a848@gmail.com>
	<20170124181121.pgk7kqfkq4dd3hpo@alap3.anarazel.de>
	<a55b21d1-7c99-2c66-d661-ef5288f29e30@gmail.com>
	<20170124190705.2ioaocx6wuhvvegn@alap3.anarazel.de>
	<bc5e13d1-88e4-5394-dfb7-a23a0b8c3f2d@gmail.com>
	<20170125195238.2sjh477x2zvmjm5a@alap3.anarazel.de>
	<CA+TgmobfAxi5g-UyHTJqkFq28WB-sasx+oPv0Po286M1UQHSng@mail.gmail.com>
	<20170622165048.lwuje4f5wqw57mke@alap3.anarazel.de>
	<CAEepm=3BGNbWaznawm3LVk69ucKqXdAcb-aboG8cK375GcsMdQ@mail.gmail.com>

>>>>> "Thomas" == Thomas Munro <thomas.munro@enterprisedb.com> writes:

 Thomas> * it's also been claimed that readahead heuristics are not
 Thomas> defeated on Linux or FreeBSD, which isn't too surprising
 Thomas> because you'd expect it to be about blocks being faulted in,
 Thomas> not syscalls

I don't know about linux, but on FreeBSD, readahead/writebehind is
tracked at the level of open files but implemented at the level of
read/write clustering. I have patched kernels in the past to improve the
performance in mixed read/write cases; pg would benefit on unpatched
kernels from using separate file opens for backend reads and writes.
(The typical bad scenario is doing a create index, or other seqscan that
updates hint bits, on a freshly-restored table; the alternation of
reading block N and writing block N-x destroys the readahead/writebehind
since they use a common offset.)

The code that detects sequential behavior can not distinguish between
pread() and lseek+read, it looks only at the actual offset of the
current request compared to the previous one for the same fp.

 Thomas> +1 for adopting pread()/pwrite() in PG12.

ditto

-- 
Andrew (irc:RhodiumToad)




view thread (21+ messages)  latest in thread

Message-ID: <87604rg0n0.fsf@news-spur.riddles.org.uk>
Permalink:  ../87604rg0n0.fsf@news-spur.riddles.org.uk/
Also on:    postgresql.org/message-id/87604rg0n0.fsf@news-spur.riddles.org.uk

 · 

reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-hackers@postgresql.org
  Cc: andrew@tao11.riddles.org.uk, thomas.munro@enterprisedb.com, andres@anarazel.de, robertmhaas@gmail.com, tobias.oberstein@gmail.com
  Subject: Re: [HACKERS] lseek/read/write overhead becomes visible at scale ..
  In-Reply-To: <87604rg0n0.fsf@news-spur.riddles.org.uk>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox