pg.ddx.io pgsql-hackers@postgresql.org mailing list archive help / color / mirror / Atom feed From: Justin Pryzby <pryzby@telsasoft.com>
To: Dilip Kumar <dilipbalaut@gmail.com>
Cc: Robert Haas <robertmhaas@gmail.com>
Cc: Tomas Vondra <tomas.vondra@2ndquadrant.com>
Cc: Alexander Korotkov <a.korotkov@postgrespro.ru>
Cc: David Steele <david@pgmasters.net>
Cc: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Cc: Dmitry Dolgov <9erthalion6@gmail.com>
Cc: pgsql-hackers@lists.postgresql.org
Subject: Re: [HACKERS] Custom compression methods
Date: Sat, 27 Feb 2021 22:18:21 -0600
Message-ID: <20210228041821.GB20769@telsasoft.com> (raw )
In-Reply-To: <CAFiTN-sTcQq2iszeTxZum4x+0jK1x-PHfVDJqUAQCsCTBEeDQQ@mail.gmail.com >
References: <20210214184940.GL1793@telsasoft.com >
<20210214202831.GM1793@telsasoft.com >
<20210218211314.GC14772@telsasoft.com >
<CAFiTN-u2pyXDDDwZXJ-fVUwbLhJSe9TbrVR6rfW_rhdyL1A5bg@mail.gmail.com >
<CA+TgmoaXs3_uoyOeJhJaS+oY6BvpGmHor70kaS6_03kssM7bVw@mail.gmail.com >
<CAFiTN-t=nNd6-ZyWP=5TQtqvWW0_VhdzQfcMsQYRdEa7eAX0gA@mail.gmail.com >
<CAFiTN-vcVGZrK_T=Oi9vVf=-N=xc0WoK7HiywHf8F9Jap5-hKw@mail.gmail.com >
<CAFiTN-u3jrRDiuyGxGvNSGDXLhG4=U2orHnjk4B6WBy9Eo9kMQ@mail.gmail.com >
<CAFiTN-uaZ8X=sXu6V_EYt8cNE2mWrehKMH3qGoAxzjepPuZGkA@mail.gmail.com >
<CAFiTN-sTcQq2iszeTxZum4x+0jK1x-PHfVDJqUAQCsCTBEeDQQ@mail.gmail.com >
On my PC, this new change is causing a test failure:
SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
- substr
-----------------------------------------------------
- 01234567890123456789012345678901234567890123456789
-(1 row)
-
+ERROR: compressed lz4 data is corrupt
@@ -119,15 +119,15 @@ lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
int32 rawsize;
struct varlena *result;
- /* allocate memory for holding the uncompressed data */
- result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
- /* decompress partial data using lz4 routine */
+ /* decompress the data */
rawsize = LZ4_decompress_safe_partial((char *) value + VARHDRSZ_COMPRESS,
VARDATA(result),
VARSIZE(value) - VARHDRSZ_COMPRESS,
slicelength,
- VARRAWSIZE_4B_C(value));
+ slicelength);
Also, in the tests, you have this at both the top and bottom of the file:
src/test/regress/sql/compression.sql:\set HIDE_COMPRESSAM false
src/test/regress/sql/compression.sql:\set HIDE_COMPRESSAM false
Whereas the patch I sent had at the end:
+\set HIDE_COMPRESSAM on
("on" is the default when run under pg_regress)
view thread (430+ messages) latest in thread
Message-ID: <20210228041821.GB20769@telsasoft.com>
Permalink: ../20210228041821.GB20769@telsasoft.com/
Also on: postgresql.org/message-id/20210228041821.GB20769@telsasoft.com
copy link · copy postgr.es
reply Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: pryzby@telsasoft.com, dilipbalaut@gmail.com, robertmhaas@gmail.com, tomas.vondra@2ndquadrant.com, a.korotkov@postgrespro.ru, david@pgmasters.net, i.kurbangaliev@gmail.com, 9erthalion6@gmail.com, pgsql-hackers@lists.postgresql.org
Subject: Re: [HACKERS] Custom compression methods
In-Reply-To: <20210228041821.GB20769@telsasoft.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox