Date: Mon, 20 Sep 2021 20:37:45 -0500
From: Justin Pryzby <pryzby@telsasoft.com>
To: Tomas Vondra <tomas.vondra@enterprisedb.com>
Cc: pgsql-hackers@postgresql.org
Subject: Re: mem context is not reset between extended stats
Message-ID: <20210921013745.GM831@telsasoft.com>
References: <20210915200928.GP831@telsasoft.com>
 <cd3af9ee-fc08-c9b0-e7c9-d3eb6fb74df6@enterprisedb.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <cd3af9ee-fc08-c9b0-e7c9-d3eb6fb74df6@enterprisedb.com>
User-Agent: Mutt/1.9.4 (2018-02-28)
Archived-At: 
 <https://www.postgresql.org/message-id/20210921013745.GM831%40telsasoft.com>
Precedence: bulk

On Tue, Sep 21, 2021 at 02:15:45AM +0200, Tomas Vondra wrote:
> On 9/15/21 10:09 PM, Justin Pryzby wrote:
> > Memory allocation appeared be O(1) WRT the number of statistics objects, which
> > was not expected to me.  This is true in v13 (and probably back to v10).

Of course I meant to say that it's O(N) and not O(1) :)

> In principle we don't expect too many extended statistics on a single table,

Yes, but note that expression statistics make it more reasonable to have
multiple extended stats objects.  I noticed this while testing a patch to build
(I think) 7 stats objects on each of our current month's partitions.
autovacuum was repeatedly killed on this vm after using using 2+GB RAM,
probably in part because there were multiple autovacuum workers handling the
most recent batch of inserted tables.

First, I tried to determine what specifically was leaking so badly, and
eventually converged to this patch.  Maybe there's additional subcontexts which
would be useful, but the minimum is to reset between objects.

> These issues exist pretty much since PG10, which is where extended stats
> were introduced, so we'll have to backpatch it. But there's no rush and I
> don't want to interfere with rc1 at the moment.

Ack that.  It'd be *nice* if if the fix were included in v14.0, but I don't
know the rules about what can change after rc1.

> Attached are two patches - 0001 is your patch (seems fine, but I looked only
> very briefly) and 0002 is the context reset I proposed.

I noticed there seems to be a 3rd patch available, which might either be junk
for testing or a cool new feature I'll hear about later ;)

> From 204f4602b218ec13ac1e3fa501a7f94adc8a4ea1 Mon Sep 17 00:00:00 2001
> From: Tomas Vondra <tomas.vondra@postgresql.org>
> Date: Tue, 21 Sep 2021 01:14:11 +0200
> Subject: [PATCH 1/3] reset context

cheers,
-- 
Justin