From: Justin Pryzby <pryzby@telsasoft.com>
To: Tomas Vondra <tomas.vondra@enterprisedb.com>
Cc: pgsql-hackers@postgresql.org
Subject: Re: mem context is not reset between extended stats
Date: Mon, 20 Sep 2021 20:37:45 -0500
Message-ID: <20210921013745.GM831@telsasoft.com> (raw)
In-Reply-To: <cd3af9ee-fc08-c9b0-e7c9-d3eb6fb74df6@enterprisedb.com>
References: <20210915200928.GP831@telsasoft.com>
<cd3af9ee-fc08-c9b0-e7c9-d3eb6fb74df6@enterprisedb.com>
On Tue, Sep 21, 2021 at 02:15:45AM +0200, Tomas Vondra wrote:
> On 9/15/21 10:09 PM, Justin Pryzby wrote:
> > Memory allocation appeared be O(1) WRT the number of statistics objects, which
> > was not expected to me. This is true in v13 (and probably back to v10).
Of course I meant to say that it's O(N) and not O(1) :)
> In principle we don't expect too many extended statistics on a single table,
Yes, but note that expression statistics make it more reasonable to have
multiple extended stats objects. I noticed this while testing a patch to build
(I think) 7 stats objects on each of our current month's partitions.
autovacuum was repeatedly killed on this vm after using using 2+GB RAM,
probably in part because there were multiple autovacuum workers handling the
most recent batch of inserted tables.
First, I tried to determine what specifically was leaking so badly, and
eventually converged to this patch. Maybe there's additional subcontexts which
would be useful, but the minimum is to reset between objects.
> These issues exist pretty much since PG10, which is where extended stats
> were introduced, so we'll have to backpatch it. But there's no rush and I
> don't want to interfere with rc1 at the moment.
Ack that. It'd be *nice* if if the fix were included in v14.0, but I don't
know the rules about what can change after rc1.
> Attached are two patches - 0001 is your patch (seems fine, but I looked only
> very briefly) and 0002 is the context reset I proposed.
I noticed there seems to be a 3rd patch available, which might either be junk
for testing or a cool new feature I'll hear about later ;)
> From 204f4602b218ec13ac1e3fa501a7f94adc8a4ea1 Mon Sep 17 00:00:00 2001
> From: Tomas Vondra <tomas.vondra@postgresql.org>
> Date: Tue, 21 Sep 2021 01:14:11 +0200
> Subject: [PATCH 1/3] reset context
cheers,
--
Justin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: pryzby@telsasoft.com, tomas.vondra@enterprisedb.com
Subject: Re: mem context is not reset between extended stats
In-Reply-To: <20210921013745.GM831@telsasoft.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox