public inbox for [email protected]
help / color / mirror / Atom feedFrom: Ron Johnson <[email protected]>
To: pgsql-general <[email protected]>
Subject: Re: Questions about the continuity of WAL archiving
Date: Tue, 12 Aug 2025 13:51:37 -0400
Message-ID: <CANzqJaBUxFRzV-YTL_2tWGW+JhOgU2i0E3sPrKXL0sZpmhVnoA@mail.gmail.com> (raw)
In-Reply-To: <CAAccyYLLKu6znfc4-tmm++KE1zb4mQCU6J6g_dj16dQ9ycB4Pw@mail.gmail.com>
References: <CAAccyYKpNsQMD+S-A7a8YtDevFN0uRXkzg4tYWWBOFsv_jASNg@mail.gmail.com>
<CAKAnmmJPGgEp2kKPomhq7P=GUA5HD1eOaP5X+7c68gm5f4YVqQ@mail.gmail.com>
<CANzqJaBdhehezNtnaC_=08abyozUTo=YwW-Ma1PAzx-mg56Rgg@mail.gmail.com>
<CAAccyYLLKu6znfc4-tmm++KE1zb4mQCU6J6g_dj16dQ9ycB4Pw@mail.gmail.com>
On Tue, Aug 12, 2025 at 4:37 AM px shi <[email protected]> wrote:
> Bog-standard PgBackRest retains all WAL files required for a full backup
>> set and its associated differential/incremental backups.
>
> Yes, WAL files are continuous under normal circumstances. However, if the
> primary node crashes under high load, the archived WAL logs on S3 may be
> discontinuous.
>
1) PG does not purge WAL files that are needed for immediate crash recovery.
2) PgBackRest can archive (compressed and encrypted) WAL files to S3.
https://pgbackrest.org/user-guide-rhel.html#s3-support
>
> Ron Johnson <[email protected]> 于2025年8月9日周六 02:45写道：
>
>> On Fri, Aug 8, 2025 at 2:26 PM Greg Sabino Mullane <[email protected]>
>> wrote:
>>
>>> There is a scenario: the current timeline of the PostgreSQL primary node
>>>> is 1, and the latest WAL file is 100. The standby node has also received up
>>>> to WAL file 100. However, the latest WAL file archived is only file 80. If
>>>> the primary node crashes at this point and the standby is promoted to the
>>>> new primary, archiving will resume from file 100 on timeline 2. As a
>>>> result, WAL files from 81 to 100 on timeline 1 will be missing from the
>>>> archive.
>>>> Is there a good solution to prevent this situation?
>>>>
>>>
>>> I'm still not clear on what the problem here is, other than your
>>> archiving not keeping up. The best solution to that is:
>>>
>>>
>>> https://pgbackrest.org/1/configuration.html#section-archive/option-archive-async
>>>
>>> Yes, you would lost some ability for easy PITR for 80-100, but could
>>> still be done by resurrecting your crashed primary, or carefully grabbing
>>> from the replica before they get recycled. You can set archive_mode=always
>>> on the replicas to help with this.
>>>
>>
>> Bog-standard PgBackRest retains all WAL files required for a full backup
>> set and its associated differential/incremental backups, no? I've
>> certainly done more than one --type=time --target="${RestoreUntil}" restore
>> without giving a second thought to timelines or whether the WAL exists.
>>
>> Maybe I've just ignored the problem, since it (seemingly) does everything
>> for PITR backups.
>>
>
--
Death to <Redacted>, and butter sauce.
Don't boil me, I'm still alive.
<Redacted> lobster!
view thread (5+ messages)
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected]
Subject: Re: Questions about the continuity of WAL archiving
In-Reply-To: <CANzqJaBUxFRzV-YTL_2tWGW+JhOgU2i0E3sPrKXL0sZpmhVnoA@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox