public inbox for [email protected]
help / color / mirror / Atom feedFrom: by Yang <[email protected]>
To: [email protected] <[email protected]>
Subject: memory leak in pgoutput
Date: Sat, 16 Nov 2024 08:37:07 +0000
Message-ID: <DM3PR84MB3442E14B340E553313B5C816E3252@DM3PR84MB3442.NAMPRD84.PROD.OUTLOOK.COM> (raw)
Hello,
I recently noticed a unusually large memory consumption in pgoutput where
"RelationSyncCache" is maintaining approximately 3 GB of memory while
handling 15,000 tables.
Upon investigating the cause, I found that "tts_tupleDescriptor" in both
"old_slot" and "new_slot" wouldn't be freed before the reusing the entry.
The refcount of the tuple descriptor is initialized as -1 in init_tuple_slot(),
which means it will not be refcounted. So, when cleaning the outdated slots,
ExecDropSingleTupleTableSlot() would omit tuple descriptors.
To address this issue, calling FreeTupleDesc() to release the tuple descriptor
before ExecDropSingleTupleTableSlot() might be a suitable solution. However,
I am uncertain whether this approach is appropriate.
Reproduct
=========
It's easy to reproduce the issue with sysbench:
1. Create emtpy tables on primary.
```
sysbench oltp_read_only --db-driver=pgsql \
--pgsql-port=5432 \
--pgsql-db=postgres \
--pgsql-user=postgres \
--tables=15000 --table_size=0 \
--report-interval=1 \
--threads=10 prepare
```
2. Create a standby and promote it.
3. Create publication on primary and subscription on standby.
```
CREATE PUBLICATION pub1 FOR ALL TABLES;
CREATE SUBSCRIPTION sub1
CONNECTION 'port=5432 user=postgres dbname=postgres'
PUBLICATION pub1
WITH (enabled, create_slot, slot_name='pub1_to_sub1', copy_data=false);
```
4. Bench on primary.
```
sysbench oltp_write_only --db-driver=pgsql \
--pgsql-port=5432 \
--pgsql-db=postgres \
--pgsql-user=postgres \
--tables=15000 --table_size=100 \
--report-interval=1 \
--threads=10 run \
--time=180
```
Tested in PostgreSQL 17, the memory consumption of walsender is 804MB after the
benchmark, and it decreases to 441MB after applying the patch.
Thanks for your feedback.
Kind Regards,
Boyu Yang
Attachments:
[application/octet-stream] 0001-Fix-memory-leak-in-pgoutput.patch (1.2K, ../DM3PR84MB3442E14B340E553313B5C816E3252@DM3PR84MB3442.NAMPRD84.PROD.OUTLOOK.COM/3-0001-Fix-memory-leak-in-pgoutput.patch)
download | inline diff:
From a53c9c5a8d9e44998d0d7f1ca7b293ae793bb051 Mon Sep 17 00:00:00 2001
From: "yangboyu.yby" <[email protected]>
Date: Sat, 16 Nov 2024 16:26:02 +0800
Subject: [PATCH] Fix memory leak in pgoutput
---
src/backend/replication/pgoutput/pgoutput.c | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index 00e7024563..b0a5884dd4 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -2055,9 +2055,19 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
* Tuple slots cleanups. (Will be rebuilt later if needed).
*/
if (entry->old_slot)
+ {
+ Assert(entry->old_slot->tts_tupleDescriptor->tdrefcount == -1);
+
+ FreeTupleDesc(entry->old_slot->tts_tupleDescriptor);
ExecDropSingleTupleTableSlot(entry->old_slot);
+ }
if (entry->new_slot)
+ {
+ Assert(entry->new_slot->tts_tupleDescriptor->tdrefcount == -1);
+
+ FreeTupleDesc(entry->new_slot->tts_tupleDescriptor);
ExecDropSingleTupleTableSlot(entry->new_slot);
+ }
entry->old_slot = NULL;
entry->new_slot = NULL;
--
2.32.0.3.g01195cf9f
view thread (2+ messages)
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected]
Subject: Re: memory leak in pgoutput
In-Reply-To: <DM3PR84MB3442E14B340E553313B5C816E3252@DM3PR84MB3442.NAMPRD84.PROD.OUTLOOK.COM>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox