Subject: Re: [PATCH] xlogreader: do not read a file block twice
To: Michael Paquier <michael@paquier.xyz>
Cc: Grigory Smolkin <g.smolkin@postgrespro.ru>,
 pgsql-hackers@lists.postgresql.org
References: <2ddf4a32-517e-d6f4-d992-4a63b6035bfd@postgrespro.ru>
 <87e30af6-f85a-dc95-4574-95cded028786@postgrespro.ru>
 <20190212042300.GL1475@paquier.xyz>
 <1fa0c9fe-32a5-3ebd-e9c2-b5e2c9abc03e@postgrespro.ru>
 <20190214065156.GE2366@paquier.xyz>
From: Arthur Zakirov <a.zakirov@postgrespro.ru>
Message-ID: <7eaccd53-ea89-e0cf-b718-80e3f3fc3abf@postgrespro.ru>
Date: Thu, 14 Feb 2019 11:20:56 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101
 Thunderbird/60.5.0
MIME-Version: 1.0
In-Reply-To: <20190214065156.GE2366@paquier.xyz>
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 14.02.2019 09:51, Michael Paquier wrote:
> Now I don't actually agree that this qualifies as a bug fix.  As
> things stand, a page may finish by being more than once if what has
> been read previously equals what is requested, however this does not
> prevent the code to work correctly.  The performance gain is also
> heavily dependent on the callback reading a page and the way the WAL
> reader is used.  How do you actually read WAL pages in your own
> plugin with compressed data?  It begins by reading a full page once,
> then it moves on to a per-record read after making sure that the page
> has been read?

Yes, an application reads WAL pages wholly at a time. It is done within 
SimpleXLogPageRead() (it is a read_page callback passed to 
XLogReaderAllocate()). It returns XLOG_BLCKSZ.

Here is the part of the code, not sure that it will be useful though:

SimpleXLogPageRead(...)
{
     ...
     targetPageOff = targetPagePtr % private_data->xlog_seg_size;
     ...
     if (gzseek(private_data->gz_xlogfile, (z_off_t) targetPageOff,
                SEEK_SET) == -1)
         ...
     if (gzread(private_data->gz_xlogfile, readBuf, XLOG_BLCKSZ) !=
                XLOG_BLCKSZ)
     ...
     return XLOG_BLCKSZ;
}

So we read whole page with size XLOG_BLCKSZ. The full code:
https://github.com/postgrespro/pg_probackup/blob/c052651b8c8864733bcabbc2660c387b792229d8/src/parsexlog.c#L1074

Here is the little optimization I made. Mainly I just add a buffer to 
store previous read page:
https://github.com/postgrespro/pg_probackup/blob/c052651b8c8864733bcabbc2660c387b792229d8/src/parsexlog.c#L1046

-- 
Arthur Zakirov
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company