public inbox for [email protected]  

help / color / mirror / Atom feed

From: Jan Wieck <[email protected]>
To: Tom Lane <[email protected]>
Cc: Bruce Momjian <[email protected]>
Cc: Zhihong Yu <[email protected]>
Cc: Andrew Dunstan <[email protected]>
Cc: Magnus Hagander <[email protected]>
Cc: Robins Tharakan <[email protected]>
Cc: Peter Eisentraut <[email protected]>
Cc: [email protected] <[email protected]>
Subject: Re: pg_upgrade failing for 200+ million Large Objects
Date: Tue, 23 Mar 2021 15:59:48 -0400
Message-ID: <[email protected]> (raw)
In-Reply-To: <[email protected]>
References: <[email protected]>
	<[email protected]>
	<[email protected]>
	<[email protected]>
	<[email protected]>
	<CALNJ-vSPnudYcS8YowHeq7VAf7PBRyu0vpsbmb_VvTJgguVyHA@mail.gmail.com>
	<[email protected]>
	<[email protected]>
	<[email protected]>
	<[email protected]>
	<[email protected]>
	<[email protected]>
	<[email protected]>
	<[email protected]>
	<[email protected]>
	<[email protected]>
	<[email protected]>

On 3/23/21 3:35 PM, Tom Lane wrote:
> Jan Wieck <[email protected]> writes:
>> The problem here is that pg_upgrade itself is invoking a shell again. It 
>> is not assembling an array of arguments to pass into exec*(). I'd be a 
>> happy camper if it did the latter. But as things are we'd have to add 
>> full shell escapeing for arbitrary strings.
> 
> Surely we need that (and have it already) anyway?

There are functions to shell escape a single string, like

     appendShellString()

but that is hardly enough when a single optarg for --restore-option 
could look like any of

     --jobs 8
     --jobs=8
     --jobs='8'
     --jobs '8'
     --jobs "8"
     --jobs="8"
     --dont-bother-about-jobs

When placed into a shell string, those things have very different 
effects on your args[].

I also want to say that we are overengineering this whole thing. Yes, 
there is the problem of shell quoting possibly going wrong as it passes 
from one shell to another. But for now this is all about passing a few 
numbers down from pg_upgrade to pg_restore (and eventually pg_dump).

Have we even reached a consensus yet on that doing it the way, my patch 
is proposing, is the right way to go? Like that emitting BLOB TOC 
entries into SECTION_DATA when in binary upgrade mode is a good thing? 
Or that bunching all the SQL statements for creating the blob, changing 
the ACL and COMMENT and SECLABEL all in one multi-statement-query is.

Maybe we should focus on those details before getting into all the 
parameter naming stuff.


Regards, Jan

-- 
Jan Wieck
Principle Database Engineer
Amazon Web Services

view thread (59+ messages)  latest in thread

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]
  Subject: Re: pg_upgrade failing for 200+ million Large Objects
  In-Reply-To: <[email protected]>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox