pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Alexander Lakhin <exclusion@gmail.com>
To: Tomas Vondra <tomas.vondra@enterprisedb.com>
To: gkokolatos@pm.me
Cc: shiy.fnst@fujitsu.com <shiy.fnst@fujitsu.com>
Cc: Michael Paquier <michael@paquier.xyz>
Cc: Justin Pryzby <pryzby@telsasoft.com>
Cc: pgsql-hackers@lists.postgresql.org <pgsql-hackers@lists.postgresql.org>
Cc: Rachel Heaton <rachelmheaton@gmail.com>
Subject: Re: Add LZ4 compression in pg_dump
Date: Sat, 11 Mar 2023 09:00:00 +0300
Message-ID: <33496f7c-3449-1426-d568-63f6bca2ac1f@gmail.com> (raw)
In-Reply-To: <8421f53d-c9f7-c02e-3278-9f9522a8e8f3@enterprisedb.com>
References: <7zu6GYFi72jByVk2SEe_nd8r1DHhwA35vSntAE9sAfiMX9Lfk6bwimkwmu7GDgYaEvL27SE4qF1uK4_LHWQM4H-SXOwkMAI-GV8PKGmED_E=@pm.me>
	<Y9IUZzQsusY3VuyJ@paquier.xyz>
	<20230126062846.GO22427@telsasoft.com>
	<6YZqN2HEE4yJxxxXWiMl4QWwKzLon0SdKETTGVkT9zjWnXMftGZjLKHHjGFuwC_5I6xL2s8nMXdLQGA4vEFveYXSHbfD9K9pDQvEJF2y99o=@pm.me>
	<Y9JpuLGng5zLO4Mx@paquier.xyz>
	<AcPUJxzDanXfXdqYOrrZ0L9J9qcnCblkc5z5x9IgKUm7d89pa6B7y8fhUOb2JGDef8h3VP6Ts7j-BYSMmtn50jVuHAT2nBleGosIdVyhG7Q=@pm.me>
	<OSZPR01MB631087FC3B53AD8B1E3439F7FDA39@OSZPR01MB6310.jpnprd01.prod.outlook.com>
	<SnTBJOQtJQf_mnXh6icTznqC05gnGqXKWDwjEA15uIx3zf0cJBwY1aK7ULztKoqlXrGM39lHtEpLGmS48A7585FXo1sO6W2YR5RtiDoWpDU=@pm.me>
	<535565cd-b69f-c064-a845-15c0fc44db4a@enterprisedb.com>
	<mYBKRYonRy5OiDDYxcxOOJ5xM9iLnuDsGP92zC7mjw8ew1wiccYZIpTvQEmtgRoAXRhaj91SAKwtFluUtRDZ3r6pRdGYLncWK7rXIQrHhFY=@pm.me>
	<22cb80ed-ec87-54c7-fd5b-cdcf79766bb1@enterprisedb.com>
	<8421f53d-c9f7-c02e-3278-9f9522a8e8f3@enterprisedb.com>

Hello,
23.02.2023 23:24, Tomas Vondra wrote:
> On 2/23/23 16:26, Tomas Vondra wrote:
>> Thanks for v30 with the updated commit messages. I've pushed 0001 after
>> fixing a comment typo and removing (I think) an unnecessary change in an
>> error message.
>>
>> I'll give the buildfarm a bit of time before pushing 0002 and 0003.
>>
> I've now pushed 0002 and 0003, after minor tweaks (a couple typos etc.),
> and marked the CF entry as committed. Thanks for the patch!
>
> I wonder how difficult would it be to add the zstd compression, so that
> we don't have the annoying "unsupported" cases.

With the patch 0003 committed, a single warning -Wtype-limits appeared in the
master branch:
$ CPPFLAGS="-Og -Wtype-limits" ./configure --with-lz4 -q && make -s -j8
compress_lz4.c: In function ‘LZ4File_gets’:
compress_lz4.c:492:19: warning: comparison of unsigned expression in ‘< 0’ is 
always false [-Wtype-limits]
   492 |         if (dsize < 0)
       |
(I wonder, is it accidental that there no other places that triggers
the warning, or some buildfarm animals had this check enabled before?)

It is not a false positive as can be proved by the 002_pg_dump.pl modified as
follows:
-                       program => $ENV{'LZ4'},
+                       program => 'mv',
                         args    => [
-                               '-z', '-f', '--rm',
"$tempdir/compression_lz4_dir/blobs.toc",
"$tempdir/compression_lz4_dir/blobs.toc.lz4",
                         ],
                 },
A diagnostic logging added shows:
LZ4File_gets() after LZ4File_read_internal; dsize: 18446744073709551615

and pg_restore fails with:
error: invalid line in large object TOC file 
".../src/bin/pg_dump/tmp_check/tmp_test_22ri/compression_lz4_dir/blobs.toc": "????"

Best regards,
Alexander





view thread (205+ messages)  latest in thread

Message-ID: <33496f7c-3449-1426-d568-63f6bca2ac1f@gmail.com>
Permalink:  ../33496f7c-3449-1426-d568-63f6bca2ac1f@gmail.com/
Also on:    postgresql.org/message-id/33496f7c-3449-1426-d568-63f6bca2ac1f@gmail.com

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-hackers@postgresql.org
  Cc: exclusion@gmail.com, tomas.vondra@enterprisedb.com, gkokolatos@pm.me, shiy.fnst@fujitsu.com, michael@paquier.xyz, pryzby@telsasoft.com, pgsql-hackers@lists.postgresql.org, rachelmheaton@gmail.com
  Subject: Re: Add LZ4 compression in pg_dump
  In-Reply-To: <33496f7c-3449-1426-d568-63f6bca2ac1f@gmail.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox