agora inbox for pgsql-admin@postgresql.org  
help / color / mirror / Atom feed
From: Johannes Truschnigg <johannes@truschnigg.info>
To: Mohan NBSPS <mohan.nbs.ont@gmail.com>
Cc: pgsql-admin@lists.postgresql.org
Subject: Re: Postgresql 9.5: Streaming Replication: Secondaries Fail To Start Post WAL Error
Date: Tue, 28 May 2024 20:47:12 +0200
Message-ID: <ZlYmsG4NX7WjzFs-@vault.lan> (raw)
In-Reply-To: <CAPCvfWcm0JDC+q54MSW7N90PYvh+PefaP6SxfonbkGcUwpS1+g@mail.gmail.com>
References: <CAPCvfWcm0JDC+q54MSW7N90PYvh+PefaP6SxfonbkGcUwpS1+g@mail.gmail.com>

Hi Mohan,

On Tue, May 28, 2024 at 02:26:41PM -0400, Mohan NBSPS wrote:
> Dear Community,
> [...]
> ```
> FATAL:  could not receive data from WAL stream: ERROR:  requested WAL
> segment 000000010000004100000049 has already been removed
>  FATAL:  the database system is starting up
> ```
> 
> from my understanding, the WAL file is streamed over the network (secondary
> pulls from primary) and creates a WAL file in the secondary.
> then it replays the copied WAL file using a different process.
> 
> in order for the local WAL file to go out of sync,
> 
> 1. the primary removed the WAL file, the secondary was streaming
> 2. the WAL file on the secondary got corrupted
> 3 ....
> 
> Questions
> 
> - what do those error messages mean ?
> - how can I prevent this from happening ?

It means that, unless you have archived the required WAL segments somewhere
and can recover them from there, your replica is now broken, and you will have
to re-create it anew.

You can prevent this by correctly configuring streaming replication either by
using replication slots (not sure if that's already implemented in 9.5,
actually - you should prioritize upgrading to a supported release while you
are working this problem!), or by introducing a WAL archive[0] for replicas to
retrieve WAL from that the primary has already evicted from its kept segments.

Hth!

[0]:
https://www.postgresql.org/docs/9.5/runtime-config-wal.html#RUNTIME-CONFIG-WAL-ARCHIVING

-- 
with best regards:
- Johannes Truschnigg ( johannes@truschnigg.info )

www:   https://johannes.truschnigg.info/

Attachments:

  [application/pgp-signature] signature.asc (832B, ../ZlYmsG4NX7WjzFs-@vault.lan/2-signature.asc)
  download

view thread (8+ messages)  latest in thread

Message-ID: <ZlYmsG4NX7WjzFs-@vault.lan>
Permalink:  ../ZlYmsG4NX7WjzFs-@vault.lan/
Also on:    postgresql.org/message-id/ZlYmsG4NX7WjzFs-@vault.lan

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-admin@postgresql.org
  Cc: johannes@truschnigg.info, mohan.nbs.ont@gmail.com, pgsql-admin@lists.postgresql.org
  Subject: Re: Postgresql 9.5: Streaming Replication: Secondaries Fail To Start Post WAL Error
  In-Reply-To: <ZlYmsG4NX7WjzFs-@vault.lan>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox