Date: Sat, 27 Feb 2021 22:18:21 -0600
From: Justin Pryzby <pryzby@telsasoft.com>
To: Dilip Kumar <dilipbalaut@gmail.com>
Cc: Robert Haas <robertmhaas@gmail.com>,
	Tomas Vondra <tomas.vondra@2ndquadrant.com>,
	Alexander Korotkov <a.korotkov@postgrespro.ru>,
	David Steele <david@pgmasters.net>,
	Ildus Kurbangaliev <i.kurbangaliev@gmail.com>,
	Dmitry Dolgov <9erthalion6@gmail.com>,
	pgsql-hackers@lists.postgresql.org
Subject: Re: [HACKERS] Custom compression methods
Message-ID: <20210228041821.GB20769@telsasoft.com>
References: <20210214184940.GL1793@telsasoft.com>
 <20210214202831.GM1793@telsasoft.com>
 <20210218211314.GC14772@telsasoft.com>
 <CAFiTN-u2pyXDDDwZXJ-fVUwbLhJSe9TbrVR6rfW_rhdyL1A5bg@mail.gmail.com>
 <CA+TgmoaXs3_uoyOeJhJaS+oY6BvpGmHor70kaS6_03kssM7bVw@mail.gmail.com>
 <CAFiTN-t=nNd6-ZyWP=5TQtqvWW0_VhdzQfcMsQYRdEa7eAX0gA@mail.gmail.com>
 <CAFiTN-vcVGZrK_T=Oi9vVf=-N=xc0WoK7HiywHf8F9Jap5-hKw@mail.gmail.com>
 <CAFiTN-u3jrRDiuyGxGvNSGDXLhG4=U2orHnjk4B6WBy9Eo9kMQ@mail.gmail.com>
 <CAFiTN-uaZ8X=sXu6V_EYt8cNE2mWrehKMH3qGoAxzjepPuZGkA@mail.gmail.com>
 <CAFiTN-sTcQq2iszeTxZum4x+0jK1x-PHfVDJqUAQCsCTBEeDQQ@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: 
 <CAFiTN-sTcQq2iszeTxZum4x+0jK1x-PHfVDJqUAQCsCTBEeDQQ@mail.gmail.com>
User-Agent: Mutt/1.9.4 (2018-02-28)
Precedence: bulk

On my PC, this new change is causing a test failure:

 SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
-                       substr                       
-----------------------------------------------------
- 01234567890123456789012345678901234567890123456789
-(1 row)
-
+ERROR:  compressed lz4 data is corrupt

@@ -119,15 +119,15 @@ lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
        int32           rawsize;
        struct varlena *result;
 
-       /* allocate memory for holding the uncompressed data */
-       result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+       /* allocate memory for the uncompressed data */
+       result = (struct varlena *) palloc(slicelength + VARHDRSZ);
 
-       /* decompress partial data using lz4 routine */
+       /* decompress the data */
        rawsize = LZ4_decompress_safe_partial((char *) value + VARHDRSZ_COMPRESS,
                                                                                  VARDATA(result),
                                                                                  VARSIZE(value) - VARHDRSZ_COMPRESS,
                                                                                  slicelength,
-                                                                                 VARRAWSIZE_4B_C(value));
+                                                                                 slicelength);

Also, in the tests, you have this at both the top and bottom of the file:

src/test/regress/sql/compression.sql:\set HIDE_COMPRESSAM false
src/test/regress/sql/compression.sql:\set HIDE_COMPRESSAM false

Whereas the patch I sent had at the end:

+\set HIDE_COMPRESSAM on

("on" is the default when run under pg_regress)