public inbox for [email protected]
help / color / mirror / Atom feedOut of Memory error triggering replica to transition into recovery mode
2+ messages / 2 participants
[nested] [flat]
* Out of Memory error triggering replica to transition into recovery mode
@ 2024-11-28 13:34 Siraj G <[email protected]>
2024-11-28 16:31 ` Re: Out of Memory error triggering replica to transition into recovery mode Tom Lane <[email protected]>
0 siblings, 1 reply; 2+ messages in thread
From: Siraj G @ 2024-11-28 13:34 UTC (permalink / raw)
To: Pgsql-admin <[email protected]>
Hello Experts!
As the subject says, today very frequently our replica DB is going into the
recovery mode causing an outage in the application side.
Here are the server & details:
Server type: Compute engine
OS: Ubuntu 20
Pgsql: 12.2
CPUs: 64
Memory: 128GB
Shared_buffers: 32GB
Work_mem: 256MB
maintenance_work_mem = 3GB
shared_buffers = 32GB
max_connections = 4000
Total size of the DBs: 3TB
The application is designed in such a way that it consumes data
primarily from SECONDARY. And, there are several applications of such type.
I can see tons of messages in the postgres log being written as:
"IP, 2024-11-28 ,<db name>, <user>,1, FATAL: the database system is in
recovery mode"
This indicates that the app services are trying to connect to the DB
constantly and there are tons of them.
Any advice on how we can improvise the situation.
Regards
Siraj
^ permalink raw reply [nested|flat] 2+ messages in thread
* Re: Out of Memory error triggering replica to transition into recovery mode
2024-11-28 13:34 Out of Memory error triggering replica to transition into recovery mode Siraj G <[email protected]>
@ 2024-11-28 16:31 ` Tom Lane <[email protected]>
0 siblings, 0 replies; 2+ messages in thread
From: Tom Lane @ 2024-11-28 16:31 UTC (permalink / raw)
To: Siraj G <[email protected]>; +Cc: Pgsql-admin <[email protected]>
Siraj G <[email protected]> writes:
> As the subject says, today very frequently our replica DB is going into the
> recovery mode causing an outage in the application side.
If you're not on this month's minor releases, perhaps you should be:
Reduce memory consumption of logical decoding (Masahiko Sawada)
Use a smaller default block size to store tuple data received
during logical replication. This reduces memory wastage, which
has been reported to be severe while processing long-running
transactions, even leading to out-of-memory failures.
I recall past updates that fixed other memory leaks in logical
replication, too.
> Pgsql: 12.2
Egad. Your version-updating strategy seriously needs a rethink.
regards, tom lane
^ permalink raw reply [nested|flat] 2+ messages in thread
end of thread, other threads:[~2024-11-28 16:31 UTC | newest]
Thread overview: 2+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2024-11-28 13:34 Out of Memory error triggering replica to transition into recovery mode Siraj G <[email protected]>
2024-11-28 16:31 ` Tom Lane <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox