From: Tels <nospam-pg-abuse@bloodgate.com>
To: Andrey Borodin <x4mmm@yandex-team.ru>
Cc: Tomas Vondra <tomas.vondra@2ndquadrant.com>
Cc: Alvaro Herrera <alvherre@2ndquadrant.com>
Cc: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Cc: Gasper Zejn <zejn@owca.info>
Cc: pgsql-hackers@lists.postgresql.org
Subject: Re: pglz performance
Date: Sun, 03 Nov 2019 10:24:43 +0100
Message-ID: <d56c85b989a3bd8c0a98d79553276b0e@bloodgate.com> (raw)
In-Reply-To: <27F232D7-25C5-4D12-AFA0-50EEBAB98E9C@yandex-team.ru>
References: <3c2d92d0-e715-f9ed-e1a7-3c510ed65940@2ndquadrant.com>
<20191101154828.GA15829@alvherre.pgsql>
<20191101165950.b5azncggaqfan5fi@development>
<27F232D7-25C5-4D12-AFA0-50EEBAB98E9C@yandex-team.ru>
Hello Andrey,
On 2019-11-02 12:30, Andrey Borodin wrote:
>> 1 нояб. 2019 г., в 18:48, Alvaro Herrera <alvherre@2ndquadrant.com>
>> написал(а):
> PFA two patches:
> v4-0001-Use-memcpy-in-pglz-decompression.patch (known as 'hacked' in
> test_pglz extension)
> v4-0001-Use-memcpy-in-pglz-decompression-for-long-matches.patch (known
> as 'hacked8')
Looking at the patches, it seems only the case of a match is changed.
But when we observe a literal byte, this is copied byte-by-byte with:
else
{
* An unset control bit means LITERAL BYTE. So we just
* copy one from INPUT to OUTPUT.
*/
*dp++ = *sp++;
}
Maybe we can optimize this, too. For instance, you could just increase a
counter:
else
{
/*
* An unset control bit means LITERAL BYTE. We count
* these and copy them later.
*/
literal_bytes ++;
}
and in the case of:
if (ctrl & 1)
{
/* First copy all the literal bytes */
if (literal_bytes > 0)
{
memcpy( sp, dp, literal_bytes);
sp += literal_bytes;
dp += literal_bytes;
literal_bytes = 0;
}
(Code untested!)
The same would need to be done at the very end, if the input ends
without any new CTRL-byte.
Wether that gains us anything depends on how common literal bytes are.
It might be that highly compressible input has almost none, while input
that is a mix of incompressible strings and compressible ones might have
longer stretches. One example would be something like an SHA-256, that
is repeated twice. The first instance would be incompressible, the
second one would be just a copy. This might not happens that often in
practical inputs, though.
I wonder if you agree and what would happen if you try this variant on
your corpus tests.
Best regards,
Tels
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: nospam-pg-abuse@bloodgate.com, x4mmm@yandex-team.ru, tomas.vondra@2ndquadrant.com, alvherre@2ndquadrant.com, peter.eisentraut@2ndquadrant.com, zejn@owca.info, pgsql-hackers@lists.postgresql.org
Subject: Re: pglz performance
In-Reply-To: <d56c85b989a3bd8c0a98d79553276b0e@bloodgate.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox