Date: Tue, 4 Oct 2022 17:15:31 -0700
From: Nathan Bossart <nathandbossart@gmail.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: David Rowley <dgrowleyml@gmail.com>,
	Dmitriy Kuzmin <kuzmin.db4@gmail.com>,
	pgsql-hackers@lists.postgresql.org
Subject: Re: Startup process on a hot standby crashes with an error "invalid
 memory alloc request size 1073741824" while replaying "Standby/LOCK" records
Message-ID: <20221005001531.GC779730@nathanxps13>
References: 
 <CAHLDt=_ts0A7Agn=hCpUh+RCFkxd+G6uuT=kcTfqFtGur0dp=A@mail.gmail.com>
 <CAApHDvrDg2rJ-sqa7c=wPoHeEGrox46sQ=CFj=FkXqBx26dr0A@mail.gmail.com>
 <2138765.1664924048@sss.pgh.pa.us>
 <2208668.1664927591@sss.pgh.pa.us>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <2208668.1664927591@sss.pgh.pa.us>
Archived-At: 
 <https://www.postgresql.org/message-id/20221005001531.GC779730%40nathanxps13>
Precedence: bulk

On Tue, Oct 04, 2022 at 07:53:11PM -0400, Tom Lane wrote:
> I wrote:
>> PFA a quick-hack fix that solves this issue by making per-transaction
>> subsidiary hash tables.  That's overkill perhaps; I'm a little worried
>> about whether this slows down normal cases more than it's worth.
>> But we ought to do something about this, because aside from the
>> duplication aspect the current storage of these lists seems mighty
>> space-inefficient.
> 
> After further thought, maybe it'd be better to do it as attached,
> with one long-lived hash table for all the locks.  This is a shade
> less space-efficient than the current code once you account for
> dynahash overhead, but the per-transaction overhead should be lower
> than the previous patch since we only need to create/destroy a hash
> table entry not a whole hash table.

This feels like a natural way to solve this problem.  I saw several cases
of the issue that was fixed with 6301c3a, so I'm inclined to believe this
usage pattern is actually somewhat common.

-- 
Nathan Bossart
Amazon Web Services: https://aws.amazon.com