From: Tom Lane <tgl@sss.pgh.pa.us>
To: Justin Pryzby <pryzby@telsasoft.com>
cc: Dilip Kumar <dilipbalaut@gmail.com>, Robert Haas <robertmhaas@gmail.com>,
        Andres Freund <andres@anarazel.de>,
        Tomas Vondra <tomas.vondra@2ndquadrant.com>,
        Alexander Korotkov <a.korotkov@postgrespro.ru>,
        David Steele <david@pgmasters.net>,
        Ildus Kurbangaliev <i.kurbangaliev@gmail.com>,
        Dmitry Dolgov <9erthalion6@gmail.com>,
        Andrey Borodin <x4mmm@yandex-team.ru>,
        PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Subject: Re: [HACKERS] Custom compression methods (mac+lz4.h)
In-reply-to: <456616.1616360715@sss.pgh.pa.us>
References: <20210319174425.GM11765@telsasoft.com>
 <CA+TgmobXnzCRf-UOWnjk+OoCT-mzFR867HvoxM2N9uA7+_V-+g@mail.gmail.com>
 <20210320212302.GW11765@telsasoft.com> <301730.1616276227@sss.pgh.pa.us>
 <20210320231957.GX11765@telsasoft.com> <309337.1616287493@sss.pgh.pa.us>
 <311370.1616290403@sss.pgh.pa.us>
 <CAFiTN-tN-fi=9VEatyiZyhW28yhhQCmn-egYzbdxBTLkOnVQvg@mail.gmail.com>
 <CAFiTN-t0w+Rc2U3S+y=7KWcLuOYNB5MfWeGdNa7+pg0UovVdcQ@mail.gmail.com>
 <414943.1616358751@sss.pgh.pa.us> <20210321204148.GB4203@telsasoft.com>
 <456616.1616360715@sss.pgh.pa.us>
Comments: In-reply-to Tom Lane <tgl@sss.pgh.pa.us>
	message dated "Sun, 21 Mar 2021 17:05:15 -0400"
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-ID: <504395.1616368310.1@sss.pgh.pa.us>
Date: Sun, 21 Mar 2021 19:11:50 -0400
Message-ID: <504396.1616368310@sss.pgh.pa.us>
Archived-At: 
 <https://www.postgresql.org/message-id/504396.1616368310%40sss.pgh.pa.us>
Precedence: bulk

I wrote:
> Justin Pryzby <pryzby@telsasoft.com> writes:
>> On Sun, Mar 21, 2021 at 04:32:31PM -0400, Tom Lane wrote:
>>> This seems somewhat repeatable (three identical failures in three
>>> attempts).  Not sure why I did not see it yesterday; but anyway,
>>> there is something wrong with partial detoasting for LZ4.

>> With what version of LZ4 ?

> RHEL8's, which is
> lz4-1.8.3-2.el8.x86_64

I hate to be the bearer of bad news, but this suggests that
LZ4_decompress_safe_partial is seriously broken in 1.9.2
as well:

https://github.com/lz4/lz4/issues/783

Maybe we cannot rely on that function for a few more years yet.

Also, I don't really understand why this code:

	/* slice decompression not supported prior to 1.8.3 */
	if (LZ4_versionNumber() < 10803)
		return lz4_decompress_datum(value);

It seems likely to me that we'd get a flat out build failure
from library versions lacking LZ4_decompress_safe_partial,
and thus that this run-time test is dead code and we should
better be using a configure probe if we intend to allow old
liblz4 versions.  Though that might be moot.

			regards, tom lane