From: Ron <ronljohnsonjr@gmail.com>
To: pgsql-admin@lists.postgresql.org
Subject: Re: pg_dump why no indicator of completion
Date: Mon, 1 May 2023 09:40:32 -0500
Message-ID: <c26a3199-40d1-f779-73ba-9b9af7901ce4@gmail.com> (raw)
In-Reply-To: <CAGA3vBvnwnX41+3GKcDPbjcSpuW_D71Vfiahrw0y_=gpNbDv5A@mail.gmail.com>
References: <CAGA3vBugp4Ce8ZbOfntb8HjCvo5QtX1hhAHj3iuCMZdzSTo4dg@mail.gmail.com>
<f68e9f75-4036-30c7-6d10-72dce6521bb3@gmail.com>
<CAGA3vBsq49mS47iH9CpCjipxu-BP_85x9doEHrvP5jyaPuNN4Q@mail.gmail.com>
<ac61c17e-1816-716e-9cb9-6d41682bc7c1@gmail.com>
<CAGA3vBsUcbcofa6F6nDBvqFHRtrBUFxu4270dnWwT3RjDP7QRQ@mail.gmail.com>
<294e2735-73e5-3935-17e4-a00a35635a31@gmail.com>
<CAGA3vBvnwnX41+3GKcDPbjcSpuW_D71Vfiahrw0y_=gpNbDv5A@mail.gmail.com>
I backup some databases every night using pg_dump. From a bash script
initiated by cron, the way God intended them to be run. And I check the
return code, like every good production shell script should.
I back up other databases using pgBackRest (also from a bash script
initiated by cron, the way God intended them to be run). And I check the
return code, like every good production shell script should (even though it
prints a message saying "Finished").
Because who in their right mind manually checks log files every morning,
when a shell script will gladly email you if something went wrong? *Make the
computer work for you!!!*
On 5/1/23 09:31, richard coleman wrote:
> Ron,
>
> That seems to be a very odd stance. PostgreSQL and it's various utilities
> are and have been used in very large commercial operations for decades.
> pg_dump successfully backs up databases from 1MB to 10TB and beyond. The
> only issue I am having is for some inexplicable reason the devs haven't
> included a status message upon completion.
>
> If you don't think that pg_dump is fit for purpose, then what do you
> propose using in its stead?
>
> thanks,
> rik.
>
> On Mon, May 1, 2023 at 10:07 AM Ron <ronljohnsonjr@gmail.com> wrote:
>
> That's like complaining that a Toyota Camry isn't fit for hauling 10
> cubic yards of sand.
>
>
> On 5/1/23 08:55, richard coleman wrote:
>> Ron,
>>
>> Are you writing that pg_dump is unfit for purpose and that I should
>> be using a commercial backup solution instead?
>>
>> rik.
>>
>> On Mon, May 1, 2023 at 9:43 AM Ron <ronljohnsonjr@gmail.com> wrote:
>>
>>
>>
>> On 5/1/23 07:56, richard coleman wrote:
>>> Ron,
>>> So what you are saying is that I have to write a shell script
>>> around pg_dump to catch the exit code of pg_dump since the
>>> developers of pg_dump didn't have pg_dump simply write an exit
>>> message?
>>>
>>> Since multiple pg_dump commands are often run at the same
>>> command prompt, and they can take hours, if not days to run,
>>
>> Running a days-long pg_dump at any time except special occasions
>> isn't very wise. Especially from a command prompt.
>>
>> Binary backup/restore programs are /much/ faster (if for no other
>> reason than they let you do incremental and differential backups).
>>
>>> and there are a myriad of other commands the will be run in the
>>> interim, any exit code generated by pg_dump with be lost in the
>>> flotsam and jetsam of the multitude of exit codes created by
>>> every other command run between the calling of pg_dump and it's
>>> eventual termination. It could have completed successfully,
>>> crashed, been killed, etc.
>>>
>>> Or am I mistaken?
>>>
>>> thanks,
>>> rik.
>>>
>>> On Sun, Apr 30, 2023 at 11:28 PM Ron <ronljohnsonjr@gmail.com>
>>> wrote:
>>>
>>> On 4/30/23 19:18, richard coleman wrote:
>>>> Hi all,
>>>>
>>>> I've been working with pg_dump and one thing that's always
>>>> struck me as strange is that there never seems to be an
>>>> indication in the log that the pg_dump process completed
>>>> successfully.
>>>>
>>>> For example, I've been running a bunch of backups with
>>>> pg_dump in directory mode with multiple jobs. When I come
>>>> back several hours later there are no pg_dump processes
>>>> running. The log is filled with:
>>>>
>>>> dumping contents of foo
>>>> finished item 123456 of TABLE DATA bar
>>>> etc.
>>>>
>>>>
>>>> Other than trying to restore the multi terabyte database
>>>> somewhere else is there any way to know that it actually
>>>> finished successfully?
>>>>
>>>> Why doesn't pg_dump add a line in the output like:
>>>>
>>>> pg_backup finished
>>>>
>>>> when it's completed successfully?
>>>>
>>>> This seems like a terrible oversight.
>>>>
>>>> Am I missing something obvious?
>>>
>>> "They" expect you to do the Unix Thing and check $?. Thus,
>>> that's what I do; any non-zero value generates an email with
>>> a scary subject line, the specifics of which are based on
>>> the exact code value. I'm not near my work computer;
>>> otherwise I'd share them with you.
>>>
>>> --
>>> Born in Arizona, moved to Babylonia.
>>>
>>
>> --
>> Born in Arizona, moved to Babylonia.
>>
>
> --
> Born in Arizona, moved to Babylonia.
>
--
Born in Arizona, moved to Babylonia.
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-admin@postgresql.org
Cc: ronljohnsonjr@gmail.com, pgsql-admin@lists.postgresql.org
Subject: Re: pg_dump why no indicator of completion
In-Reply-To: <c26a3199-40d1-f779-73ba-9b9af7901ce4@gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox