Subject: Re: gitlab post-mortem: pg_basebackup waiting for
 checkpoint
To: Jim Nasby <Jim.Nasby@BlueTreble.com>, Robert Haas
 <robertmhaas@gmail.com>, Alvaro Herrera <alvherre@2ndquadrant.com>
References: 
 <CA+TgmoZ2Ru__LmeNQJm3AjGPHBHjfhU0D+X5pTw7UdZtvK2qhQ@mail.gmail.com>
 <20170214210651.ldwzk6frxxclbomw@alvherre.pgsql>
 <CA+TgmoYeTytkZdSQqX8Ck+ZHSQkW5dyY8-0=rCdofxUbjdya_A@mail.gmail.com>
 <c48f49ef-a3e1-5315-447b-84f8fa7ccab0@BlueTreble.com>
Cc: Magnus Hagander <magnus@hagander.net>,
 Michael Banck <michael.banck@credativ.de>,
 Pg Hackers <pgsql-hackers@postgresql.org>
From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
Message-ID: <c869c5ba-0aa2-06df-4e1d-60169cf7e230@2ndquadrant.com>
Date: Sat, 18 Feb 2017 00:22:20 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101
 Thunderbird/45.4.0
MIME-Version: 1.0
In-Reply-To: <c48f49ef-a3e1-5315-447b-84f8fa7ccab0@BlueTreble.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org

On 02/17/2017 08:17 PM, Jim Nasby wrote:
 > On 2/14/17 5:18 PM, Robert Haas wrote:
 >> On Tue, Feb 14, 2017 at 4:06 PM, Alvaro Herrera
 >> <alvherre@2ndquadrant.com> wrote:
 >>> I'd rather have a --quiet mode instead.  If you're running it by hand,
 >>> you're likely to omit the switch, whereas when writing the cron job
 >>> you're going to notice lack of switch even before you let the job run
 >>> once.
 >>
 >> Well, that might've been a better way to design it, but changing it
 >> now would break backward compatibility and I'm not really sure that's
 >
 > Meh... it's really only going to affect cronjobs or scripts, which are
 > easy enough to fix, and you're not going to have that many of them (or
 > if you do you certainly have an automated way to push the update).
 >

I think you're underestimating the breakage and overestimating how easy 
it's going to be to it. It's true we'd only change this in a major 
version, so people should assume possible breakage and test.

 >> a good idea.  Even if it is, it's a separate concern from whether or
 >> not in the less-quiet mode we should point out that we're waiting for
 >> a checkpoint on the server side.
 >
 > Well, --quite was suggested because of confusion from pg_basebackup
 > twiddling it's thumbs...

I'm in favor of the '--verbose' route. People are used to that when 
investigating issues, and it does not break existing cron jobs. I can 
live with --quiet though, as long as we don't resort to some craziness 
along the lines "if there's tty be verbose, otherwise be quiet".

I have my doubts about this actually addressing gitlab-like mistakes, 
though, because it's a helluva jump from "It's waiting and not doing 
anything," to "We need to remove the datadir." (One of the reasons being 
that non-empty directory is a local issue, and there's no reason why the 
tool should wait instead of just reporting an error.)

FWIW before messing with the pg_basebackup code, perhaps we should 
improve the documentation and explain clearly the meaning of 'fast' and 
'spread' checkpoint modes. Right now, pg_basebackup docs only say this:

   Sets checkpoint mode to fast or spread (default) (see Section 24.3.3).

which is pretty damn useless, when you're investigating an issue. And 
the referenced section (Making a Base Backup Using the Low Level API) 
does not clearly explain how this maps to pg_start_backup(_,?).

What about adding a paragraph into pg_basebackup docs, explaining that 
with 'fast' it does immediate checkpoint, while with 'spread' it'll wait 
for a spread checkpoint.

regards

-- Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers