From: Justin Pryzby <pryzby@telsasoft.com>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Andrew Dunstan <andrew@dunslane.net>
Cc: Dilip Kumar <dilipbalaut@gmail.com>
Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>
Cc: Andres Freund <andres@anarazel.de>
Cc: Tomas Vondra <tomas.vondra@2ndquadrant.com>
Cc: Alexander Korotkov <a.korotkov@postgrespro.ru>
Cc: David Steele <david@pgmasters.net>
Cc: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Cc: Dmitry Dolgov <9erthalion6@gmail.com>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Subject: Re: [HACKERS] Custom compression methods
Date: Wed, 24 Mar 2021 12:24:50 -0500
Message-ID: <20210324172450.GD15100@telsasoft.com> (raw)
In-Reply-To: <CA+TgmoZh+1LrD3SnvNrpx-poxoMOA6C0QVAN_++ZAUxsrkDM4Q@mail.gmail.com>
References: <4158590.1616185209@sss.pgh.pa.us>
<20210320000625.GP11765@telsasoft.com>
<20210320074420.GR11765@telsasoft.com>
<CAFiTN-uX=hLscjL=0raA6dYcoiZB=9YqL3DmX15CMvZiu+WB+w@mail.gmail.com>
<20210320153933.GT11765@telsasoft.com>
<229250.1616261775@sss.pgh.pa.us>
<261449.1616267030@sss.pgh.pa.us>
<fe462d41-7063-72a1-2e70-0300c53ee2c4@dunslane.net>
<1099280.1616600518@sss.pgh.pa.us>
<CA+TgmoZh+1LrD3SnvNrpx-poxoMOA6C0QVAN_++ZAUxsrkDM4Q@mail.gmail.com>
On Wed, Mar 24, 2021 at 12:24:38PM -0400, Robert Haas wrote:
> On Wed, Mar 24, 2021 at 11:42 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > On reflection, though, I wonder if we've made pg_dump do the right
> > thing anyway. There is a strong case to be made for the idea that
> > when dumping from a pre-14 server, it should emit
> > SET default_toast_compression = 'pglz';
> > rather than omitting any mention of the variable, which is what
> > I made it do in aa25d1089. If we changed that, I think all these
> > diffs would go away. Am I right in thinking that what's being
> > compared here is new pg_dump's dump from old server versus new
> > pg_dump's dump from new server?
> >
> > The "strong case" goes like this: initdb a v14 cluster, change
> > default_toast_compression to lz4 in its postgresql.conf, then
> > try to pg_upgrade from an old server. If the dump script doesn't
> > set default_toast_compression = 'pglz' then the upgrade will
> > do the wrong thing because all the tables will be recreated with
> > a different behavior than they had before. IIUC, this wouldn't
> > result in broken data, but it still seems to me to be undesirable.
> > dump/restore ought to do its best to preserve the old DB state,
> > unless you explicitly tell it --no-toast-compression or the like.
>
> This feels a bit like letting the tail wag the dog, because one might
> reasonably guess that the user's intention in such a case was to
> switch to using LZ4, and we've subverted that intention by deciding
> that we know better. I wouldn't blame someone for thinking that using
> --no-toast-compression with a pre-v14 server ought to have no effect,
> but with your proposal here, it would. Furthermore, IIUC, the user has
> no way of passing --no-toast-compression through to pg_upgrade, so
> they're just going to have to do the upgrade and then fix everything
> manually afterward to the state that they intended to have all along.
> Now, on the other hand, if they wanted to make practically any other
> kind of change while upgrading, they'd have to do something like that
> anyway, so I guess this is no worse.
I think it's not specific to pg_upgrade, but any pg_dump |pg_restore.
The analogy with tablespaces is restoring from a cluster where the tablespace
is named "vast" to one where it's named "huge". I do this by running
PGOPTIONS=-cdefault_tablespace=huge pg_restore --no-tablespaces
So I thinks as long as --no-toast-compression does the corresponding thing, the
"restore with alternate compression" case is handled fine.
--
Justin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: pryzby@telsasoft.com, robertmhaas@gmail.com, tgl@sss.pgh.pa.us, andrew@dunslane.net, dilipbalaut@gmail.com, alvherre@alvh.no-ip.org, andres@anarazel.de, tomas.vondra@2ndquadrant.com, a.korotkov@postgrespro.ru, david@pgmasters.net, i.kurbangaliev@gmail.com, 9erthalion6@gmail.com, pgsql-hackers@lists.postgresql.org
Subject: Re: [HACKERS] Custom compression methods
In-Reply-To: <20210324172450.GD15100@telsasoft.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox