Message-ID: <e4d4fef2-165c-4d27-8366-18cfef5d6484@tantorlabs.com>
Date: Tue, 24 Mar 2026 22:28:24 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Compress prune/freeze records with Delta Frame of Reference
 algorithm
To: Andres Freund <andres@anarazel.de>,
 PostgreSQL Hackers <pgsql-hackers@postgresql.org>,
 Andrey Borodin <x4mmm@yandex-team.ru>
References: <acf7e923-7ad6-42ed-b16a-df15e876d249@tantorlabs.com>
 <wps75qdtwpykdt4zatfh2u2hi3zju4drdzqi2zh7uy3x4ooivv@kc2fxfz7lx3e>
Content-Language: en-US
From: Evgeny Voropaev <evgeny.voropaev@tantorlabs.com>
In-Reply-To: <wps75qdtwpykdt4zatfh2u2hi3zju4drdzqi2zh7uy3x4ooivv@kc2fxfz7lx3e>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Archived-At: 
 <https://www.postgresql.org/message-id/e4d4fef2-165c-4d27-8366-18cfef5d6484%40tantorlabs.com>
Precedence: bulk

Hello Andres,

> I'm unconvinced that this is a serious problem - typically the overhead of WAL
> volume due to pruning / freezing is due to the full page images emitted, not
> the raw size of the records. Once an FPI is emitted, this doesn't matter.
> 
> What gains have you measured in somewhat realistic workloads?

So far, we have had no tests in any real production environment. 
Moreover, the load in the new test 
(recovery/t/052_prune_dfor_compression.pl) is quite contrived. However, 
it demonstrates a compression ratio of more than 5, and it is measured 
for an overall size of all prune/freeze records with no filtering.

Further development is the implementation of compression of unsorted 
sequences. This is going to allow PostgreSQL to compress also the 
'frozen' and the 'redirected' offset sequences, which should result in a 
greater compression ratio.

But I agree with you, Andres, we need practical results to estimate a 
profit. I wish we would test it on some real load soon.

Also I hope, independently of its usage in prune/freeze records, the 
DFoR itself might be used for compression sequences in other places of PG.

Best regards,
Evgeny Voropaev.