Subject: Re: Bad recovery: no pg_xlog/RECOVERYXLOG
To: pgsql-admin@postgresql.org
References: <d90474b1-6913-f3ff-d224-73fd931cdd98@cahoots.pl>
 <1070071508952614@web46j.yandex.ru>
 <ae7f2ee9-97ff-267e-b6dc-9399250a9314@cahoots.pl>
 <20171030154706.GE4628@tamriel.snowman.net>
From: Mark Kirkwood <mark.kirkwood@catalyst.net.nz>
Message-ID: <7966ee9e-2241-eced-4639-e32c157b915a@catalyst.net.nz>
Date: Tue, 31 Oct 2017 09:30:48 +1300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
 Thunderbird/52.4.0
MIME-Version: 1.0
In-Reply-To: <20171030154706.GE4628@tamriel.snowman.net>
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding: 7bit
Content-Language: en-US
Precedence: bulk
Sender: pgsql-admin-owner@postgresql.org


On 31/10/17 04:47, Stephen Frost wrote:
> Greetings,
>
> * Marcin Koziej (marcin@cahoots.pl) wrote:
>> Now it's fixed, but if anyone needs I'm attaching all scripts to 1)
>> backup and restore wal's and 2) backup and restore base backup from
>> OpenStack SWIFT
> Interesting, but these scripts seem to be seriously lacking in error
> checking (what happens if the copy to swift fails..?  or pg_basebackup
> fails?) and it's unclear how you can be sure that the WAL file has been
> sync'd to disk which is important or you might end up having holes in
> your WAL stream if the swift system fails.  There's also no checking to
> make sure that the WAL needed for a given pg_basebackup ever actually
> made it to the swift system, which is required to ensure you have a
> consistent backup.
>
> Generally speaking, these kinds of scripts really aren't a good choice
> for doing backups of PG.  I'd strongly suggest you look at one of the
> existing tools which are developed specifically for doing backups of PG
> and are well tested, supported, and maintained.  If you'd like support
> for a new storage system, I know that at least pgBackRest's storage
> layer is pluggable and adding a new storage option is pretty straight
> forward.
>
>
I'm not convinced that his approach is bad.

The script checks the result of the 'swift upload' for the base backup, 
it is the wal backup one that does not explicitly check the 'swift 
upload' result (this should really be added). To be fair, anything wrong 
with the swift system will likely be discovered immediately beforehand 
where he does a 'swift stat'!

I'd guess his original problem was an improperly setup recovery.conf, 
rather than the overall design.

regards

Mark


-- 
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin