Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1ptUBd-0001uq-1z for pgsql-admin@arkaria.postgresql.org; Mon, 01 May 2023 14:07:29 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1ptUBb-00037E-Dw for pgsql-admin@arkaria.postgresql.org; Mon, 01 May 2023 14:07:27 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1ptUBa-000375-SJ for pgsql-admin@lists.postgresql.org; Mon, 01 May 2023 14:07:27 +0000 Received: from mail-yb1-xb32.google.com ([2607:f8b0:4864:20::b32]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94.2) (envelope-from ) id 1ptUBT-002sjo-RE for pgsql-admin@lists.postgresql.org; Mon, 01 May 2023 14:07:25 +0000 Received: by mail-yb1-xb32.google.com with SMTP id 3f1490d57ef6-b9d87dffadfso1973182276.3 for ; Mon, 01 May 2023 07:07:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1682950039; x=1685542039; h=in-reply-to:from:references:to:content-language:subject:user-agent :mime-version:date:message-id:from:to:cc:subject:date:message-id :reply-to; bh=ZUE1/5Oou7k7UPof7WAcPbW4E7wDVWVwEBezkKMJ2zc=; b=iXOHLQdoljJyVo04xpmLMxLl/JNi5v2sFjHmQe8V64GIySukQumg2n5L9c+9kz0R9Z Qux/hflTbr0ozWS0JXZfk/A10Yon4SBCa2KLziJTOqE+Lg1RbVhjHnZsUP0pFzSLggM6 ApawZKWIXzT9EpQxkKahHnTxQxAGXIRcHk45xorHy+lWQcqZ+UZEBkZD2IDr9LYhHAiN W6MB/yxSbrh6i5kTHYUVX5U6eWn2HXRy1Z9kxPv8+UK3+VU78ZNo48IxETUtg3UHK04N PDE8fkVYvRZOosyNjvOS3HlpSl4j0QaZWvtkle8Whmkxe2brKxNrhVYsMd64wy5ciP0k QFRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682950039; x=1685542039; h=in-reply-to:from:references:to:content-language:subject:user-agent :mime-version:date:message-id:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=ZUE1/5Oou7k7UPof7WAcPbW4E7wDVWVwEBezkKMJ2zc=; b=blZAdO/uRKiPFPgbwbKVjA4guFR2oI90EYEwwaQkDjdwYT6A4rGAPsOJ0ULyluiL1o vAq5U3np1m/2dhWaZZH7VNP8QEOEw1mLWEq0uYYDiiqm0hidiUzyBZb1INTheTMbiomX pXCxSSrEfSsXaIrrwOeHUnmX5HbXVyFdnDJqkLdbtIJifn80PDg1xRN2fqXH3pLa/cbg ep+CmvC04s0zp8Sov1/c/orPxEXPjicAddQ50i3JVkmAyO6oyyAiLJ4DVKA0dRiRL2uC ef/61qSMubGuY/vDViTR8fjtGe3bcgTmY+hhkm4i0nbSjCrmg6b1E9i7MXnU4b9T0SC9 ukzg== X-Gm-Message-State: AC+VfDwzwjuN5xdmey/aeF5EAwm/AmTF3UYaJSqO1n7VdOfQw6vfdb7d B146MtRfJCNaaXEIAUsIQK1xjdW6/E0= X-Google-Smtp-Source: ACHHUZ4MVKj0yuMMpjxeYQSkiZITjjp4n8UYjcZSs17Ts9BtPuTFkPqwh6EqzU25wYWI5NODVv03Tw== X-Received: by 2002:a25:1045:0:b0:b8f:5302:b86c with SMTP id 66-20020a251045000000b00b8f5302b86cmr13190256ybq.52.1682950038782; Mon, 01 May 2023 07:07:18 -0700 (PDT) Received: from ?IPV6:2600:1700:bd2c:b810:edd6:19bb:e3ec:a4c2? ([2600:1700:bd2c:b810:edd6:19bb:e3ec:a4c2]) by smtp.googlemail.com with ESMTPSA id z82-20020a256555000000b00b9a82c1b070sm2367922ybb.27.2023.05.01.07.07.18 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 01 May 2023 07:07:18 -0700 (PDT) Content-Type: multipart/alternative; boundary="------------g1PzUYXl4XqclyhqjpdQzJx0" Message-ID: <294e2735-73e5-3935-17e4-a00a35635a31@gmail.com> Date: Mon, 1 May 2023 09:07:17 -0500 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Subject: Re: pg_dump why no indicator of completion Content-Language: en-US To: pgsql-admin@lists.postgresql.org References: From: Ron In-Reply-To: List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk This is a multi-part message in MIME format. --------------g1PzUYXl4XqclyhqjpdQzJx0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit That's like complaining that a Toyota Camry isn't fit for hauling 10 cubic yards of sand. On 5/1/23 08:55, richard coleman wrote: > Ron, > > Are you writing that pg_dump is unfit for purpose and that I should be > using a commercial backup solution instead? > > rik. > > On Mon, May 1, 2023 at 9:43 AM Ron wrote: > > > > On 5/1/23 07:56, richard coleman wrote: >> Ron, >> So what you are saying is that I have to write a shell script around >> pg_dump to catch the exit code of pg_dump since the developers of >> pg_dump didn't have pg_dump simply write an exit message? >> >> Since multiple pg_dump commands are often run at the same command >> prompt, and they can take hours, if not days to run, > > Running a days-long pg_dump at any time except special occasions isn't > very wise.  Especially from a command prompt. > > Binary backup/restore programs are /much/ faster (if for no other > reason than they let you do incremental and differential backups). > >> and there are a myriad of other commands the will be run in the >> interim, any exit code generated by pg_dump with be lost in the >> flotsam and jetsam of the multitude of exit codes created by every >> other command run between the calling of pg_dump and it's eventual >> termination. It could have completed successfully, crashed, been >> killed, etc. >> >> Or am I mistaken? >> >> thanks, >> rik. >> >> On Sun, Apr 30, 2023 at 11:28 PM Ron wrote: >> >> On 4/30/23 19:18, richard coleman wrote: >>> Hi all, >>> >>> I've been working with pg_dump and one thing that's always >>> struck me as strange is that there never seems to be an >>> indication in the log that the pg_dump process completed >>> successfully. >>> >>> For example, I've been running a bunch of backups with pg_dump >>> in directory mode with multiple jobs.  When I come back several >>> hours later there are no pg_dump processes running. The log is >>> filled with: >>> >>> dumping contents of foo >>> finished item 123456 of TABLE DATA bar >>> etc. >>> >>> >>> Other than trying to restore the multi terabyte database >>> somewhere else is there any way to know that it actually >>> finished successfully? >>> >>> Why doesn't pg_dump add a line in the output like: >>> >>> pg_backup finished >>> >>> when it's completed successfully? >>> >>> This seems like a terrible oversight. >>> >>> Am I missing something obvious? >> >> "They" expect you to do the Unix Thing and check $?. Thus, that's >> what I do; any non-zero value generates an email with a scary >> subject line, the specifics of which are based on the exact code >> value.  I'm not near my work computer; otherwise I'd share them >> with you. >> >> -- >> Born in Arizona, moved to Babylonia. >> > > -- > Born in Arizona, moved to Babylonia. > -- Born in Arizona, moved to Babylonia. --------------g1PzUYXl4XqclyhqjpdQzJx0 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit That's like complaining that a Toyota Camry isn't fit for hauling 10 cubic yards of sand.


On 5/1/23 08:55, richard coleman wrote:
Ron, 

Are you writing that pg_dump is unfit for purpose and that I should be using a commercial backup solution instead?

rik.

On Mon, May 1, 2023 at 9:43 AM Ron <ronljohnsonjr@gmail.com> wrote:


On 5/1/23 07:56, richard coleman wrote:
Ron, 
So what you are saying is that I have to write a shell script around pg_dump to catch the exit code of pg_dump since the developers of pg_dump didn't have pg_dump simply write an exit message?

Since multiple pg_dump commands are often run at the same command prompt, and they can take hours, if not days to run,

Running a days-long pg_dump at any time except special occasions isn't very wise.  Especially from a command prompt.

Binary backup/restore programs are much faster (if for no other reason than they let you do incremental and differential backups).

and there are a myriad of other commands the will be run in the interim, any exit code generated by pg_dump with be lost in the flotsam and jetsam of the multitude of exit codes created by every other command run between the calling of pg_dump and it's eventual termination. It could have completed successfully, crashed, been killed, etc.

Or am I mistaken?

thanks, 
rik. 

On Sun, Apr 30, 2023 at 11:28 PM Ron <ronljohnsonjr@gmail.com> wrote:
On 4/30/23 19:18, richard coleman wrote:
Hi all, 

I've been working with pg_dump and one thing that's always struck me as strange is that there never seems to be an indication in the log that the pg_dump process completed successfully.

For example, I've been running a bunch of backups with pg_dump in directory mode with multiple jobs.  When I come back several hours later there are no pg_dump processes running.  The log is filled with:
dumping contents of foo
finished item 123456 of TABLE DATA bar
etc.

Other than trying to restore the multi terabyte database somewhere else is there any way to know that it actually finished successfully?

Why doesn't pg_dump add a line in the output like:
pg_backup finished
when it's completed successfully? 

This seems like a terrible oversight.

Am I missing something obvious?

"They" expect you to do the Unix Thing and check $?. Thus, that's what I do; any non-zero value generates an email with a scary subject line, the specifics of which are based on the exact code value.  I'm not near my work computer; otherwise I'd share them with you.

--
Born in Arizona, moved to Babylonia.

--
Born in Arizona, moved to Babylonia.

--
Born in Arizona, moved to Babylonia.
--------------g1PzUYXl4XqclyhqjpdQzJx0--