Message-ID: <1ea99cb0-c39c-ff1b-37ca-5816ed08f8d7@enterprisedb.com>
Date: Tue, 14 Mar 2023 16:52:20 +0100
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.8.0
Subject: Re: Add LZ4 compression in pg_dump
Content-Language: en-US
To: gkokolatos@pm.me
Cc: Justin Pryzby <pryzby@telsasoft.com>, shiy.fnst@fujitsu.com,
 Michael Paquier <michael@paquier.xyz>, pgsql-hackers@lists.postgresql.org,
 Rachel Heaton <rachelmheaton@gmail.com>
References: <AcPUJxzDanXfXdqYOrrZ0L9J9qcnCblkc5z5x9IgKUm7d89pa6B7y8fhUOb2JGDef8h3VP6Ts7j-BYSMmtn50jVuHAT2nBleGosIdVyhG7Q=@pm.me>
 <mYBKRYonRy5OiDDYxcxOOJ5xM9iLnuDsGP92zC7mjw8ew1wiccYZIpTvQEmtgRoAXRhaj91SAKwtFluUtRDZ3r6pRdGYLncWK7rXIQrHhFY=@pm.me>
 <22cb80ed-ec87-54c7-fd5b-cdcf79766bb1@enterprisedb.com>
 <8421f53d-c9f7-c02e-3278-9f9522a8e8f3@enterprisedb.com>
 <20230228235834.GC30529@telsasoft.com>
 <lsZgBfZRB5w5slcXnKwoL9qgpzdlAC_UwYHXfbj1oioWsNkckwr2BcLvvCi9-x7eV261s8AP5OHwROZ-QYtpeixxt73DQ_d6-LNfghYOaIQ=@pm.me>
 <b0ced9ea-e92d-f9d1-7a2f-075881b755b2@enterprisedb.com>
 <ZAfosKXCewHVENaZ@telsasoft.com>
 <c1493de3-7d97-c3c9-dd39-5a781e10a7b8@enterprisedb.com>
 <LtyfSKFLLHwZB8M9HDPXJ3yqzyiWwGRe5gADfUVZo0MNCl8eWV43tDpZHlGhgiObPaFWZmIpZxgB714UFh8k8X-9fcADJP5IReT75a1Jhco=@pm.me>
From: Tomas Vondra <tomas.vondra@enterprisedb.com>
In-Reply-To: <LtyfSKFLLHwZB8M9HDPXJ3yqzyiWwGRe5gADfUVZo0MNCl8eWV43tDpZHlGhgiObPaFWZmIpZxgB714UFh8k8X-9fcADJP5IReT75a1Jhco=@pm.me>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Archived-At: <https://www.postgresql.org/message-id/1ea99cb0-c39c-ff1b-37ca-5816ed08f8d7%40enterprisedb.com>
Precedence: bulk

On 3/14/23 12:07, gkokolatos@pm.me wrote:
> 
> 
> ------- Original Message -------
> On Monday, March 13th, 2023 at 10:47 PM, Tomas Vondra <tomas.vondra@enterprisedb.com> wrote:
> 
> 
> 
>>
>>> Change pg_fatal() to an assertion+comment;
>>
>>
>> Yeah, that's reasonable. I'd even ditch the assert/comment, TBH. We
>> could add such protections against "impossible" stuff to a zillion other
>> places and the confusion likely outweighs the benefits.
>>
> 
> A minor note to add is to not ignore the lessons learned from a7885c9bb.
> 
> For example, as the testing framework stands, one can not test that the
> contents of the custom format are indeed compressed. One can infer it by
> examining the header of the produced dump and searching for the
> compression flag. The code responsible for writing the header and the
> code responsible for actually dealing with data, is not the same. Also,
> the compression library itself will happily read and write uncompressed
> data.
> 
> A pg_fatal, assertion, or similar, is the only guard rail against this
> kind of error. Without it, the tests will continue passing even after
> e.g. a wrong initialization of the API. It was such a case that lead to
> a7885c9bb in the first place. I do think that we wish it to be an
> "impossible" case. Also it will be an untested case with some history
> without such a guard rail.
> 

So is the pg_fatal() a dead code or not? My understanding was it's not
really reachable, and the main purpose is to remind people this is not
possible. Or am I mistaken/confused?

If it's reachable, can we test it? AFAICS we don't, per the coverage
reports.

If it's just a protection against incorrect API initialization, then an
assert is the right solution, I think. With proper comment. But can't we
actually verify that *during* the initialization?

Also, how come WriteDataToArchiveLZ4() doesn't need this protection too?
Or is that due to gzip being the default compression method?

> Of course I will not object to removing it, if you think that is more
> confusing than useful.
> 

Not sure, I have a feeling I don't quite understand in what situation
this actually helps.


regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company