Date: Wed, 16 Oct 2019 12:58:38 -0500
From: Justin Pryzby <pryzby@telsasoft.com>
To: Stephen Frost <sfrost@snowman.net>
Cc: "imai.yoshikazu@fujitsu.com" <imai.yoshikazu@fujitsu.com>,
	Euler Taveira <euler@timbira.com.br>, Tom Lane <tgl@sss.pgh.pa.us>,
	Andrew Gierth <andrew@tao11.riddles.org.uk>,
	pgsql-hackers@lists.postgresql.org
Subject: Re: v12 and pg_restore -f-
Message-ID: <20191016175838.GL3599@telsasoft.com>
References: <20191006190839.GE18030@telsasoft.com>
 <28770.1570394593@sss.pgh.pa.us>
 <875zl11tla.fsf@news-spur.riddles.org.uk>
 <30069.1570396524@sss.pgh.pa.us>
 <20191008180840.GQ6962@tamriel.snowman.net>
 <CAHE3wgi=VU0G9=S60=gQH+yyk8-Ke99dkt8Dp4OpZXkSu=67fQ@mail.gmail.com>
 <20191009124505.GT6962@tamriel.snowman.net>
 <OSBPR01MB4616298336E32F8EAEDA0FF394920@OSBPR01MB4616.jpnprd01.prod.outlook.com>
 <20191016172148.GH6962@tamriel.snowman.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20191016172148.GH6962@tamriel.snowman.net>
User-Agent: Mutt/1.5.24 (2015-08-30)
Precedence: bulk

On Sun, Oct 06, 2019 at 04:43:13PM -0400, Tom Lane wrote:
> Nobody is going to wish that to mean "write to a file named '-'", so

Probably right, but it occurs to me that someone could make a named pipe called
that, possibly to make "dump to stdout" work with scripts that don't support
dumping to stdout, but with what's arguably a self-documenting syntax.

On Wed, Oct 09, 2019 at 09:07:40AM -0300, Euler Taveira wrote:
> > Maintenance scripts break across major versions.
...
> Yeah, if you check pg_restore version, you could use new syntax for
> 12+. We break scripts every release (mainly with catalog changes) and
> I don't know why this change is different than the other ones. The
> pg_restore changes is more user-friendly and less error-prone.

The issue isn't that scripts broke, but that after I fixed scripts to work with
v12, it's impossible (within pg_restore and without relying on shell or linux
conventions) to write something that works on both v11 and v12, without
conditionalizing on pg_restore --version.

On Wed, Oct 16, 2019 at 01:21:48PM -0400, Stephen Frost wrote:
> [...] if they actually need to work with both concurrently (which strikes me
> as already at least relatively uncommon...).

I doubt it's uncommon.  In our case, we have customers running centos6 and 7.
There's no postgis RPMs provided for centos6 which allow an upgrade path to
v12, so we'll end up supporting at least (centos6, pg11) and (centos7,pg12) for
months, at least.

I have a half dozen maintenance scripts to do things like reindex, vacuum,
cluster, alter tblspace.  In the immediate case, our backup script runs
pg_restore to check if an existing pg_dump backup of an old table is empty when
the table is not itself empty - which has happened before due to logic errors
and mishandled DST...  (We're taking advantage of timeseries partitioning so
daily backups exclude tables older than a certain thershold, which are assumed
to be unchanged, or at least not have data updated).

I'd *like* to be able to deploy our most recent maint scripts during the
interval of time our customers are running different major versions.  The
alternative being to try to remember to avoid deploying updated v12 scripts at
customers still running v11.  I went to the effort to make our vacuum/analyze
script support both versions following the OID change.

I worked around the pg_restore change using /dev/stdout ; possibly the
documentation should mention that workaround for portability to earlier
versions: that would work for maybe 85% of cases.  If need be, one could check
pg_restore --version.  But it's nicer not to need to.

Tom's proposed in February to backpatch the -f- behavior, so ISTM that we're
right now exactly where we (or at least he) planned to be, except that the
backpatch ideally should've been included in the minor releases in August,
before v12 was released.

https://www.postgresql.org/message-id/24868.1550106683%40sss.pgh.pa.us

Justin