From: Justin Pryzby <pryzby@telsasoft.com>
To: Tomas Vondra <tomas.vondra@enterprisedb.com>
Cc: pgsql-hackers@postgresql.org
Subject: mem context is not reset between extended stats
Date: Wed, 15 Sep 2021 15:09:28 -0500
Message-ID: <20210915200928.GP831@telsasoft.com> (raw)
Memory allocation appeared be O(1) WRT the number of statistics objects, which
was not expected to me. This is true in v13 (and probably back to v10).
It seems to work fine to reset the memory context within the loop, so long as
the statslist is allocated in the parent context.
|DROP TABLE t; CREATE TABLE t AS SELECT i, i+1 AS a, i+2 AS b, i+3 AS c, i+4 AS d, i+5 AS e FROM generate_series(1,99999)i;
|SELECT format('CREATE STATISTICS sta%s (ndistinct) ON a,(1+b),(2+c),(3+d),(4+e) FROM t', a) FROM generate_series(1,9)a\gexec
|SET log_statement_stats=on; SET client_min_messages=debug; ANALYZE t;
|=> 369432 kB max resident size
|SELECT format('CREATE STATISTICS sta%s (ndistinct) ON a,b,c,d,e FROM t', a) FROM generate_series(1,33)a\gexec
|SET log_statement_stats=on; SET client_min_messages=debug; ANALYZE t;
|=> 1284368 kB max resident size
Attachments:
[text/x-diff] 0001-stx-do-not-leak-memory-for-each-stats-obj.patch (1.6K, ../20210915200928.GP831@telsasoft.com/2-0001-stx-do-not-leak-memory-for-each-stats-obj.patch)
download | inline diff:
From 6ae4d059f2ed8baf2af92ec0847458055147383c Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Wed, 15 Sep 2021 14:38:49 -0500
Subject: [PATCH] stx: do not leak memory for each stats obj
---
src/backend/statistics/extended_stats.c | 12 +++++-------
1 file changed, 5 insertions(+), 7 deletions(-)
diff --git a/src/backend/statistics/extended_stats.c b/src/backend/statistics/extended_stats.cindex 63549757ec..d9387ceeb6 100644--- a/src/backend/statistics/extended_stats.c+++ b/src/backend/statistics/extended_stats.c@@ -136,14 +136,14 @@ BuildRelationExtStatistics(Relation onerel, double totalrows,
if (!natts)
return;
+ pg_stext = table_open(StatisticExtRelationId, RowExclusiveLock);+ statslist = fetch_statentries_for_relation(pg_stext, RelationGetRelid(onerel));+
cxt = AllocSetContextCreate(CurrentMemoryContext,
"BuildRelationExtStatistics",
ALLOCSET_DEFAULT_SIZES);
oldcxt = MemoryContextSwitchTo(cxt);
- pg_stext = table_open(StatisticExtRelationId, RowExclusiveLock);- statslist = fetch_statentries_for_relation(pg_stext, RelationGetRelid(onerel));-
/* report this phase */
if (statslist != NIL)
{
@@ -245,14 +245,12 @@ BuildRelationExtStatistics(Relation onerel, double totalrows,
pgstat_progress_update_param(PROGRESS_ANALYZE_EXT_STATS_COMPUTED,
++ext_cnt);
- /* free the build data (allocated as a single chunk) */- pfree(data);+ MemoryContextReset(cxt);
}
- table_close(pg_stext, RowExclusiveLock);-
MemoryContextSwitchTo(oldcxt);
MemoryContextDelete(cxt);
+ table_close(pg_stext, RowExclusiveLock);
}
/*
--
2.17.0
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: pryzby@telsasoft.com, tomas.vondra@enterprisedb.com
Subject: Re: mem context is not reset between extended stats
In-Reply-To: <20210915200928.GP831@telsasoft.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox