Message-ID: <0ecaec9d-861b-c74f-846b-b64d929507ef@enterprisedb.com>
Date: Mon, 6 Feb 2023 20:57:04 +0100
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.7.1
Subject: Re: pglz compression performance, take two
Content-Language: en-US
To: Andrey Borodin <amborodin86@gmail.com>
Cc: Ian Lawrence Barwick <barwick@gmail.com>,
 Andrey Borodin <x4mmm@yandex-team.ru>,
 Mark Dilger <mark.dilger@enterprisedb.com>,
 Justin Pryzby <pryzby@telsasoft.com>, pgsql-hackers@postgresql.org
References: <FEF3DC5E-4BC4-44E1-8DEB-DADC67046FE3@yandex-team.ru>
 <20201230043930.GG25152@telsasoft.com>
 <BF0A7E6D-AB21-47F8-A5FA-328CAA32DB12@yandex-team.ru>
 <20210122024811.GE27167@telsasoft.com>
 <EB24F56E-901B-46D6-BFFC-71D51603ABDE@enterprisedb.com>
 <88C049C6-1B86-4EF7-940D-585B07FEB81B@yandex-team.ru>
 <90649B7F-EF71-40F7-93D0-1029AD0D90F3@enterprisedb.com>
 <B1D3105C-D68B-445F-B3DF-E3F41FE79C46@enterprisedb.com>
 <84aa2da0-004d-c8f8-48c8-aa2861d1ff73@enterprisedb.com>
 <120DD821-48F4-446C-865A-093E560A2143@yandex-team.ru>
 <CAB8KJ=h5q3Cn8YPezQCQHGCgA5r=XOEfeZb_tW+8YY1JQfK1xQ@mail.gmail.com>
 <bcf3e6ee-44a1-5615-bb3d-0eb9531b6e8c@enterprisedb.com>
 <CAAhFRxhFvuO-yUu8VuT3k6jKeJUjzA0ff3rJS6qtPLHdq3Epzg@mail.gmail.com>
 <CAAhFRxg4cinQ=Wy9QCaFuojFM9qPpWpwvUSdqkPcsDcwqAiMog@mail.gmail.com>
 <CAAhFRxhCMG4b17SdheURdEF2AdByUXnrokqLV9Cas9qmKq7aJQ@mail.gmail.com>
 <0f54129f-255d-2d8f-94a0-d18b6f388745@enterprisedb.com>
 <CAAhFRxgE6fOGL0WYv+pNFbRq0MKTsmuRVsSfL5iymhV_L39aMg@mail.gmail.com>
From: Tomas Vondra <tomas.vondra@enterprisedb.com>
In-Reply-To: 
 <CAAhFRxgE6fOGL0WYv+pNFbRq0MKTsmuRVsSfL5iymhV_L39aMg@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Archived-At: 
 <https://www.postgresql.org/message-id/0ecaec9d-861b-c74f-846b-b64d929507ef%40enterprisedb.com>
Precedence: bulk


On 2/6/23 03:00, Andrey Borodin wrote:
> On Sun, Feb 5, 2023 at 5:51 PM Tomas Vondra
> <tomas.vondra@enterprisedb.com> wrote:
>>
>> On 2/5/23 19:36, Andrey Borodin wrote:
>>> On Fri, Jan 6, 2023 at 10:02 PM Andrey Borodin <amborodin86@gmail.com> wrote:
>>>>
>>>> Hello! Please find attached v8.
>>>
>>> I got some interesting feedback from some patch users.
>>> There was an oversight that frequently yielded results that are 1,2 or
>>> 3 bytes longer than expected.
>>> Looking closer I found that the correctness of the last 3-byte tail is
>>> checked in two places. PFA fix for this. Previously compressed data
>>> was correct, however in some cases few bytes longer than the result of
>>> current pglz implementation.
>>>
>>
>> Thanks. What were the consequences of the issue? Lower compression
>> ratio, or did we then fail to decompress the data (or would current pglz
>> implementation fail to decompress it)?
>>
> The data was decompressed fine. But extension tests (Citus's columnar
> engine) hard-coded a lot of compression ratio stuff.

OK. Not sure I'd blame the patch for these failures, as long as long as
the result is still correct and can be decompressed. I'm not aware of a
specification of what the compression must (not) produce.

> And there is still 1 more test where optimized version produces 1 byte
> longer output. I'm trying to find it, but with no success yet.
> 
> There are known and documented cases when optimized pglz version would
> do so. good_match without 10-division and memcmp by 4 bytes. But even
> disabling this, still observing 1-byte longer compression results
> persists... The problem is the length is changed after deleting some
> data, so compression of that particular sequence seems to be somewhere
> far away.
> It was funny at the beginning - to hunt for 1 byte. But the weekend is
> ending, and it seems that byte slipped from me again...
> 

I wonder what that means for the patch. I haven't investigated this at
all, but it seems as if the optimization means we fail to find a match,
producing a tad larger output. That may be still be a good tradeoff, as
long as the output is correct (assuming it doesn't break some promise
regarding expected output).

regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company