Subject: Re: OK to delete old .history files from archive?
To: Don Seiler <don@seiler.us>, Pgsql-admin <pgsql-admin@lists.postgresql.org>
References: 
 <CAHJZqBDLLeHk7=R9kdH0w+p16ujQ+EUf7TcX-2c82ks_n=9+YQ@mail.gmail.com>
From: David Steele <david@pgmasters.net>
Message-ID: <8d7ed172-a84f-613d-9a1c-3c98b22993dd@pgmasters.net>
Date: Fri, 10 Jul 2020 16:05:24 -0400
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:68.0)
 Gecko/20100101 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: 
 <CAHJZqBDLLeHk7=R9kdH0w+p16ujQ+EUf7TcX-2c82ks_n=9+YQ@mail.gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit
Precedence: bulk

Hi Don,

On 7/3/20 2:02 PM, Don Seiler wrote:
> TL;DR: Is it OK to delete .history files from WAL archive (wal-g/S3) 
> that are older than the oldest full backup?
> 
> Today we had restores fail complaining that they can't find timeline 21 
> when they should only be on timeline 1. The logs show them recovering 
> 12+ .history files from the WAL archives. Looking at the archives, these 
> history files are all timestamped between last July and October. The 
> restore is from a full backup taken last night.
> 
> This happens to be the first restore after adding 
> "recovery_target_timeline='latest'" to our recovery.conf template. So 
> I'm guessing that PG just didn't bother with those old .history files 
> before.
> 
> I believe these old .history files were just not properly cleaned up 
> from some exercises done last year on previous/other incarnations and 
> are now just polluting the WAL archive. I don't see why I would need 
> .history files that are nearly a year older than the backup I'm 
> restoring from, but just want to confirm it is safe to blow them all away.

It should be fine to remove the history files as they do not refer to 
any valid history in the WAL.

Of course, it would be best to keep a copy of them until you test your 
restore again. Just in case.

Regards,
-- 
-David
david@pgmasters.net