From: Arthur Zakirov <a.zakirov@postgrespro.ru>
To: Michael Paquier <michael@paquier.xyz>
Cc: Grigory Smolkin <g.smolkin@postgrespro.ru>
Cc: pgsql-hackers@lists.postgresql.org
Subject: Re: [PATCH] xlogreader: do not read a file block twice
Date: Thu, 14 Feb 2019 11:20:56 +0300
Message-ID: <7eaccd53-ea89-e0cf-b718-80e3f3fc3abf@postgrespro.ru> (raw)
In-Reply-To: <20190214065156.GE2366@paquier.xyz>
References: <2ddf4a32-517e-d6f4-d992-4a63b6035bfd@postgrespro.ru>
<87e30af6-f85a-dc95-4574-95cded028786@postgrespro.ru>
<20190212042300.GL1475@paquier.xyz>
<1fa0c9fe-32a5-3ebd-e9c2-b5e2c9abc03e@postgrespro.ru>
<20190214065156.GE2366@paquier.xyz>
On 14.02.2019 09:51, Michael Paquier wrote:
> Now I don't actually agree that this qualifies as a bug fix. As
> things stand, a page may finish by being more than once if what has
> been read previously equals what is requested, however this does not
> prevent the code to work correctly. The performance gain is also
> heavily dependent on the callback reading a page and the way the WAL
> reader is used. How do you actually read WAL pages in your own
> plugin with compressed data? It begins by reading a full page once,
> then it moves on to a per-record read after making sure that the page
> has been read?
Yes, an application reads WAL pages wholly at a time. It is done within
SimpleXLogPageRead() (it is a read_page callback passed to
XLogReaderAllocate()). It returns XLOG_BLCKSZ.
Here is the part of the code, not sure that it will be useful though:
SimpleXLogPageRead(...)
{
...
targetPageOff = targetPagePtr % private_data->xlog_seg_size;
...
if (gzseek(private_data->gz_xlogfile, (z_off_t) targetPageOff,
SEEK_SET) == -1)
...
if (gzread(private_data->gz_xlogfile, readBuf, XLOG_BLCKSZ) !=
XLOG_BLCKSZ)
...
return XLOG_BLCKSZ;
}
So we read whole page with size XLOG_BLCKSZ. The full code:
https://github.com/postgrespro/pg_probackup/blob/c052651b8c8864733bcabbc2660c387b792229d8/src/parsex...
Here is the little optimization I made. Mainly I just add a buffer to
store previous read page:
https://github.com/postgrespro/pg_probackup/blob/c052651b8c8864733bcabbc2660c387b792229d8/src/parsex...
--
Arthur Zakirov
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: a.zakirov@postgrespro.ru, michael@paquier.xyz, g.smolkin@postgrespro.ru, pgsql-hackers@lists.postgresql.org
Subject: Re: [PATCH] xlogreader: do not read a file block twice
In-Reply-To: <7eaccd53-ea89-e0cf-b718-80e3f3fc3abf@postgrespro.ru>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox