pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  

help / color / mirror / Atom feed
From: Michael Paquier <michael@paquier.xyz>
To: Tomas Vondra <tomas.vondra@enterprisedb.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: gkokolatos@pm.me, Andrew Dunstan <andrew@dunslane.net>
Cc: Alexander Lakhin <exclusion@gmail.com>
Cc: Justin Pryzby <pryzby@telsasoft.com>
Cc: shiy.fnst@fujitsu.com, pgsql-hackers@lists.postgresql.org, Rachel Heaton <rachelmheaton@gmail.com>
Subject: Re: Add LZ4 compression in pg_dump
Date: Tue, 9 May 2023 07:10:55 +0900
Message-ID: <ZFlzb5KTJs9m0Fsn@paquier.xyz> (raw)
In-Reply-To: <f735df01-0bb4-2fbc-1297-73a520cfc534@enterprisedb.com>
References: <ZBjg4D/X05X+JbcJ@telsasoft.com>
	<3beb0dd7-c927-3eca-689f-716e255d5b27@enterprisedb.com>
	<94ae9bca-5ebb-1e68-bb7b-4f32e89fefbe@gmail.com>
	<MmuCeC4k017VCWzr-f8MEQ9hx8DRak2ZGbb5SVwKvBBHqCt8tLKzVVdMcPMsi3Zj314OJbs1UPEzwc5RwOv3IMp1lAkC7rg-yDOaDBJGww8=@pm.me>
	<71868915-0345-1b7b-f3d6-ec66547c662a@dunslane.net>
	<P-Dl7-QRVPmjv35H2n-Wnj9amDG_MUEb0-jtlzs05n5z03lq0IZNTsI5h4DjTmLekfmKexgFBQhmVXtUO402vAIFx0e002lIzsb8O8Hi1wA=@pm.me>
	<ZFhCyn4Gm2eu60rB@paquier.xyz>
	<219662.1683508165@sss.pgh.pa.us>
	<220484.1683508611@sss.pgh.pa.us>
	<f735df01-0bb4-2fbc-1297-73a520cfc534@enterprisedb.com>

On Mon, May 08, 2023 at 08:00:39PM +0200, Tomas Vondra wrote:
> The LZ4Stream_write() forgot to move the pointer to the next chunk, so
> it was happily decompressing the initial chunk over and over. A bit
> embarrassing oversight :-(
> 
> The custom format calls WriteDataToArchiveLZ4(), which was correct.
> 
> The attached patch fixes this for me.

Ouch.  So this was corrupting the dumps and the compression when
trying to write more than two chunks at once, not the decompression
steps.  That addresses the issue here as well, thanks!
--
Michael

Attachments:

  [application/pgp-signature] signature.asc (832B, ../ZFlzb5KTJs9m0Fsn@paquier.xyz/2-signature.asc)
  download

view thread (205+ messages)  latest in thread

Message-ID: <ZFlzb5KTJs9m0Fsn@paquier.xyz>
Permalink:  ../ZFlzb5KTJs9m0Fsn@paquier.xyz/
Also on:    postgresql.org/message-id/ZFlzb5KTJs9m0Fsn@paquier.xyz

 · 

reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-hackers@postgresql.org
  Cc: michael@paquier.xyz, tomas.vondra@enterprisedb.com, tgl@sss.pgh.pa.us, andrew@dunslane.net, exclusion@gmail.com, pryzby@telsasoft.com, rachelmheaton@gmail.com
  Subject: Re: Add LZ4 compression in pg_dump
  In-Reply-To: <ZFlzb5KTJs9m0Fsn@paquier.xyz>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox