Message-ID: <ed20d3de-2ef1-4019-3d0e-7e7348d315ba@pgmasters.net>
Date: Sat, 23 Jul 2022 09:20:32 -0400
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:91.0)
 Gecko/20100101 Thunderbird/91.11.0
Subject: Re: could not link file in wal restore lines
Content-Language: en-US
To: Michael Paquier <michael@paquier.xyz>, Zsolt Ero <zsolt.ero@gmail.com>
Cc: pgsql-bugs@lists.postgresql.org
References: 
 <CAKw-smBhLOGtRJTC5c=qKTPz8gz5+WPoVAXrHB6mY-1U4_N7-w@mail.gmail.com>
 <Yttsz99SeUpyFBD/@paquier.xyz>
From: David Steele <david@pgmasters.net>
In-Reply-To: <Yttsz99SeUpyFBD/@paquier.xyz>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Archived-At: 
 <https://www.postgresql.org/message-id/ed20d3de-2ef1-4019-3d0e-7e7348d315ba%40pgmasters.net>
Precedence: bulk

On 7/22/22 23:36, Michael Paquier wrote:
> On Fri, Jul 22, 2022 at 04:24:54AM -0500, Zsolt Ero wrote:
>> I've opened an issue on pgbackrest, where the developer confirmed that
>> those files are not touched by pgbackrest, they are basically copied back
>> exactly as they were. The backup/restore is on the same partition as the
>> data folder, so no chance of file system changes, etc. Single master, no
>> replication.
>>
>> I'm always doing clean data dir + init_db so there is nothing there before
>> the restore. I can replicate this behaviour 100%.
>>
>> PostgreSQL version: psql (PostgreSQL) 12.11 (Ubuntu 12.11-1.pgdg18.04+1)
>>
>> OS is Ubuntu 18.04
>>
>> pgbackrest issue: https://github.com/pgbackrest/pgbackrest/issues/1815
>>
>> log details:
>>
>> could not link file "pg_wal/000000010000015600000098" to
>> "pg_wal/00000001000001570000006E": File exists
> 
> FWIW, the backend code has protections to prevent *exactly* this kind
> of problems when recycling WAL segment files at checkpoints with a set
> of LWLocks taken on the control file, for one.  Perhaps you have
> messed up things and you have finished in such a state that backrest
> writes to pg_wal/ concurrently with a cluster running and running a
> checkpoint, which would explain those link() calls to be failing?

During recovery pgbackrest only writes into pg_wal when invoked by 
restore_command. That generally means pgbackrest should only be writing 
to RECOVERYXLOG.

Zsolt, can you add --log-level-console=info to your archive_command so 
we can see what parameters are being passed to pgbackrest? This logging 
will appear in the postgres log.

Regards,
-David