agora inbox for pgsql-admin@postgresql.org
help / color / mirror / Atom feed PostgreSQL Replication Lag – WALs Streaming but Not Being Applied
5+ messages / 4 participants
[nested] [flat]
* PostgreSQL Replication Lag – WALs Streaming but Not Being Applied
@ 2026-08-18 18:03 Simon Musila - Localhost <smusila1@gmail.com>
2026-08-18 18:06 ` Re: PostgreSQL Replication Lag – WALs Streaming but Not Being Applied Gyanuba Gyanuba <gyanubagyanuba@gmail.com>
2026-08-19 12:13 ` Re: PostgreSQL Replication Lag – WALs Streaming but Not Being Applied hubert depesz lubaczewski <depesz@depesz.com>
0 siblings, 2 replies; 5+ messages in thread
From: Simon Musila - Localhost @ 2026-08-18 18:03 UTC (permalink / raw)
To: pgsql-admin@lists.postgresql.org
Hello,
I would appreciate some guidance on an issue we are experiencing with
PostgreSQL streaming replication.
The standby appears to be *receiving/streaming WALs and is up to date with
the primary*, but the WAL records are not being applied to the standby,
resulting in an increasing replication/replay lag.
We have confirmed that:
- WALs are being streamed from the primary to the standby.
- The WAL receiver appears to be active.
- The standby is receiving the WALs, but the replay/apply process is not
keeping up.
- pg_stat_database_conflicts shows no conflicts.
- pg_is_wal_replay_paused() returns false.
- recovery_min_apply_delay is 0.
- hot_standby_feedback is disabled.
[image: Screenshot 2026-08-18 at 20.52.58.png]
*My questions are:*
1. What could cause WALs to be successfully streamed to the standby but
not applied/replayed?
2. What are the best steps to identify what the WAL replay process is
waiting on?
3. Is restarting the replication/standby the only practical solution, or
is there a way to safely resume WAL replay without restarting?
4. Are there specific PostgreSQL views, wait events, or system metrics
that you would recommend checking to identify the bottleneck?
I would appreciate any guidance on troubleshooting this and identifying the
underlying cause rather than simply restarting the replication whenever
this occurs.
*NB: This happens on a daily basis!*
Thank you in advance.
Regards,
Musila
Attachments:
[image/png] Screenshot 2026-08-18 at 20.52.58.png (40.3K, ../../CAEcumTPW5fDOFiv5yF8qDZCX2wVTUiR7KgNXu3Sc_dPE7sMRWA@mail.gmail.com/3-Screenshot%202026-08-18%20at%2020.52.58.png)
download | view image
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: PostgreSQL Replication Lag – WALs Streaming but Not Being Applied
2026-08-18 18:03 PostgreSQL Replication Lag – WALs Streaming but Not Being Applied Simon Musila - Localhost <smusila1@gmail.com>
@ 2026-08-18 18:06 ` Gyanuba Gyanuba <gyanubagyanuba@gmail.com>
2026-08-18 18:14 ` Re: PostgreSQL Replication Lag – WALs Streaming but Not Being Applied Avinash Vallarapu <avinash.vallarapu@gmail.com>
1 sibling, 1 reply; 5+ messages in thread
From: Gyanuba Gyanuba @ 2026-08-18 18:06 UTC (permalink / raw)
To: Simon Musila - Localhost <smusila1@gmail.com>; +Cc: pgsql-admin@lists.postgresql.org
Wait pending
On Tue, Aug 18, 2026, 11:34 PM Simon Musila - Localhost <smusila1@gmail.com>
wrote:
> Hello,
>
> I would appreciate some guidance on an issue we are experiencing with
> PostgreSQL streaming replication.
>
> The standby appears to be *receiving/streaming WALs and is up to date
> with the primary*, but the WAL records are not being applied to the
> standby, resulting in an increasing replication/replay lag.
>
> We have confirmed that:
>
> - WALs are being streamed from the primary to the standby.
> - The WAL receiver appears to be active.
> - The standby is receiving the WALs, but the replay/apply process is
> not keeping up.
> - pg_stat_database_conflicts shows no conflicts.
> - pg_is_wal_replay_paused() returns false.
> - recovery_min_apply_delay is 0.
> - hot_standby_feedback is disabled.
>
> [image: Screenshot 2026-08-18 at 20.52.58.png]
>
> *My questions are:*
>
> 1. What could cause WALs to be successfully streamed to the standby
> but not applied/replayed?
> 2. What are the best steps to identify what the WAL replay process is
> waiting on?
> 3. Is restarting the replication/standby the only practical solution,
> or is there a way to safely resume WAL replay without restarting?
> 4. Are there specific PostgreSQL views, wait events, or system metrics
> that you would recommend checking to identify the bottleneck?
>
> I would appreciate any guidance on troubleshooting this and identifying
> the underlying cause rather than simply restarting the replication whenever
> this occurs.
>
> *NB: This happens on a daily basis!*
>
> Thank you in advance.
>
> Regards,
> Musila
>
Attachments:
[image/png] Screenshot 2026-08-18 at 20.52.58.png (40.3K, ../../CAP7qGJ6CMQ7tNoL9kcA+FXK-RYfMSk67oQaOBb3FbX3pSEHBUg@mail.gmail.com/3-Screenshot%202026-08-18%20at%2020.52.58.png)
download | view image
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: PostgreSQL Replication Lag – WALs Streaming but Not Being Applied
2026-08-18 18:03 PostgreSQL Replication Lag – WALs Streaming but Not Being Applied Simon Musila - Localhost <smusila1@gmail.com>
2026-08-18 18:06 ` Re: PostgreSQL Replication Lag – WALs Streaming but Not Being Applied Gyanuba Gyanuba <gyanubagyanuba@gmail.com>
@ 2026-08-18 18:14 ` Avinash Vallarapu <avinash.vallarapu@gmail.com>
2026-08-18 18:25 ` Re: PostgreSQL Replication Lag – WALs Streaming but Not Being Applied Simon Musila - Localhost <smusila1@gmail.com>
0 siblings, 1 reply; 5+ messages in thread
From: Avinash Vallarapu @ 2026-08-18 18:14 UTC (permalink / raw)
To: Gyanuba Gyanuba <gyanubagyanuba@gmail.com>; +Cc: Simon Musila - Localhost <smusila1@gmail.com>; pgsql-admin@lists.postgresql.org
Hi,
Please see what the below parameters are set to (On Standby).
max_standby_streaming_delay
max_standby_archive_delay
Do you have any activity recorded in pg_stat_activity on the standby ?
On Tue, Aug 18, 2026 at 2:06 PM Gyanuba Gyanuba <gyanubagyanuba@gmail.com>
wrote:
> Wait pending
>
> On Tue, Aug 18, 2026, 11:34 PM Simon Musila - Localhost <
> smusila1@gmail.com> wrote:
>
>> Hello,
>>
>> I would appreciate some guidance on an issue we are experiencing with
>> PostgreSQL streaming replication.
>>
>> The standby appears to be *receiving/streaming WALs and is up to date
>> with the primary*, but the WAL records are not being applied to the
>> standby, resulting in an increasing replication/replay lag.
>>
>> We have confirmed that:
>>
>> - WALs are being streamed from the primary to the standby.
>> - The WAL receiver appears to be active.
>> - The standby is receiving the WALs, but the replay/apply process is
>> not keeping up.
>> - pg_stat_database_conflicts shows no conflicts.
>> - pg_is_wal_replay_paused() returns false.
>> - recovery_min_apply_delay is 0.
>> - hot_standby_feedback is disabled.
>>
>> [image: Screenshot 2026-08-18 at 20.52.58.png]
>>
>> *My questions are:*
>>
>> 1. What could cause WALs to be successfully streamed to the standby
>> but not applied/replayed?
>> 2. What are the best steps to identify what the WAL replay process is
>> waiting on?
>> 3. Is restarting the replication/standby the only practical solution,
>> or is there a way to safely resume WAL replay without restarting?
>> 4. Are there specific PostgreSQL views, wait events, or system
>> metrics that you would recommend checking to identify the bottleneck?
>>
>> I would appreciate any guidance on troubleshooting this and identifying
>> the underlying cause rather than simply restarting the replication whenever
>> this occurs.
>>
>> *NB: This happens on a daily basis!*
>>
>> Thank you in advance.
>>
>> Regards,
>> Musila
>>
>
--
Regards,
Avinash Vallarapu
HexaCluster Corp.
www.hexacluster.ai
Attachments:
[image/png] Screenshot 2026-08-18 at 20.52.58.png (40.3K, ../../CAN0Tujdd40MsLGUWQkw320GgcyhYcq2m5G+6+V97GHKkk6ChCQ@mail.gmail.com/3-Screenshot%202026-08-18%20at%2020.52.58.png)
download | view image
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: PostgreSQL Replication Lag – WALs Streaming but Not Being Applied
2026-08-18 18:03 PostgreSQL Replication Lag – WALs Streaming but Not Being Applied Simon Musila - Localhost <smusila1@gmail.com>
2026-08-18 18:06 ` Re: PostgreSQL Replication Lag – WALs Streaming but Not Being Applied Gyanuba Gyanuba <gyanubagyanuba@gmail.com>
2026-08-18 18:14 ` Re: PostgreSQL Replication Lag – WALs Streaming but Not Being Applied Avinash Vallarapu <avinash.vallarapu@gmail.com>
@ 2026-08-18 18:25 ` Simon Musila - Localhost <smusila1@gmail.com>
0 siblings, 0 replies; 5+ messages in thread
From: Simon Musila - Localhost @ 2026-08-18 18:25 UTC (permalink / raw)
To: Avinash Vallarapu <avinash.vallarapu@gmail.com>; +Cc: Gyanuba Gyanuba <gyanubagyanuba@gmail.com>; pgsql-admin@lists.postgresql.org
Hello,
max_standby_streaming_delay=30000ms
max_standby_archive_delay=30000ms
I had changed at some point but still failed.
Do you have any activity recorded in pg_stat_activity on the standby ? *Yes*
For the replicas displayed: only one doesn't fail.
Regards,
Musila
On Tue, Aug 18, 2026 at 9:15 PM Avinash Vallarapu <
avinash.vallarapu@gmail.com> wrote:
> Hi,
>
> Please see what the below parameters are set to (On Standby).
>
> max_standby_streaming_delay
> max_standby_archive_delay
>
> Do you have any activity recorded in pg_stat_activity on the standby ?
>
>
>
> On Tue, Aug 18, 2026 at 2:06 PM Gyanuba Gyanuba <gyanubagyanuba@gmail.com>
> wrote:
>
>> Wait pending
>>
>> On Tue, Aug 18, 2026, 11:34 PM Simon Musila - Localhost <
>> smusila1@gmail.com> wrote:
>>
>>> Hello,
>>>
>>> I would appreciate some guidance on an issue we are experiencing with
>>> PostgreSQL streaming replication.
>>>
>>> The standby appears to be *receiving/streaming WALs and is up to date
>>> with the primary*, but the WAL records are not being applied to the
>>> standby, resulting in an increasing replication/replay lag.
>>>
>>> We have confirmed that:
>>>
>>> - WALs are being streamed from the primary to the standby.
>>> - The WAL receiver appears to be active.
>>> - The standby is receiving the WALs, but the replay/apply process is
>>> not keeping up.
>>> - pg_stat_database_conflicts shows no conflicts.
>>> - pg_is_wal_replay_paused() returns false.
>>> - recovery_min_apply_delay is 0.
>>> - hot_standby_feedback is disabled.
>>>
>>> [image: Screenshot 2026-08-18 at 20.52.58.png]
>>>
>>> *My questions are:*
>>>
>>> 1. What could cause WALs to be successfully streamed to the standby
>>> but not applied/replayed?
>>> 2. What are the best steps to identify what the WAL replay process
>>> is waiting on?
>>> 3. Is restarting the replication/standby the only practical
>>> solution, or is there a way to safely resume WAL replay without restarting?
>>> 4. Are there specific PostgreSQL views, wait events, or system
>>> metrics that you would recommend checking to identify the bottleneck?
>>>
>>> I would appreciate any guidance on troubleshooting this and identifying
>>> the underlying cause rather than simply restarting the replication whenever
>>> this occurs.
>>>
>>> *NB: This happens on a daily basis!*
>>>
>>> Thank you in advance.
>>>
>>> Regards,
>>> Musila
>>>
>>
>
> --
> Regards,
> Avinash Vallarapu
> HexaCluster Corp.
> www.hexacluster.ai
>
Attachments:
[image/png] Screenshot 2026-08-18 at 20.52.58.png (40.3K, ../../CAEcumTMFkB6uk-6pZ+hVYhzCHTYsPMr2K21sJ=fBixrW7B5dHg@mail.gmail.com/3-Screenshot%202026-08-18%20at%2020.52.58.png)
download | view image
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: PostgreSQL Replication Lag – WALs Streaming but Not Being Applied
2026-08-18 18:03 PostgreSQL Replication Lag – WALs Streaming but Not Being Applied Simon Musila - Localhost <smusila1@gmail.com>
@ 2026-08-19 12:13 ` hubert depesz lubaczewski <depesz@depesz.com>
1 sibling, 0 replies; 5+ messages in thread
From: hubert depesz lubaczewski @ 2026-08-19 12:13 UTC (permalink / raw)
To: Simon Musila - Localhost <smusila1@gmail.com>; +Cc: pgsql-admin@lists.postgresql.org
On Tue, Aug 18, 2026 at 09:03:39PM +0300, Simon Musila - Localhost wrote:
> *My questions are:*
> 1. What could cause WALs to be successfully streamed to the standby but
> not applied/replayed?
Concurrent "traffic" (queries). Or special operations in wal that take
"a while" to apply.
> 2. What are the best steps to identify what the WAL replay process is
> waiting on?
I'd start with:
a. `ps uwf -u postgres` shell command (as long as the underlying os is
linux-ish.
b. `select * from pg_stat_activity`
c. `sudo strace -f -tt -T -s 256 -p PID-OF-PG-STARTUP-PROCESS` - again,
on linux hosts
d. pg_waldump of the wal file that takes long time, and check what is
going on there
Best regards,
depesz
^ permalink raw reply [nested|flat] 5+ messages in thread
end of thread, other threads:[~2026-08-19 12:13 UTC | newest]
Thread overview: 5+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2026-08-18 18:03 PostgreSQL Replication Lag – WALs Streaming but Not Being Applied Simon Musila - Localhost <smusila1@gmail.com>
2026-08-18 18:06 ` Gyanuba Gyanuba <gyanubagyanuba@gmail.com>
2026-08-18 18:14 ` Avinash Vallarapu <avinash.vallarapu@gmail.com>
2026-08-18 18:25 ` Simon Musila - Localhost <smusila1@gmail.com>
2026-08-19 12:13 ` hubert depesz lubaczewski <depesz@depesz.com>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox