pg.ddx.io  pgsql-committers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: David Rowley <drowley@postgresql.org>
To: pgsql-committers@lists.postgresql.org
Subject: pgsql: Overload index_form_tuple to allow the memory context to be supp
Date: Wed, 06 Jul 2022 20:14:20 +0000
Message-ID: <E1o9BPf-001Wu1-AI@gemulon.postgresql.org> (raw)

Overload index_form_tuple to allow the memory context to be supplied

40af10b57 changed things so we make use of a generation memory context for
storing tuples to be sorted by tuplesort.c. That change does not play
nicely with the changes made in 9f03ca915 (back in 2014). That commit
changed things so that index_form_tuple() is called while switched into
the tuplestore's tuplecontext. In order to fetch the tuple from the index,
index_form_tuple() must do various memory allocations which are unrelated
to the storage of the final returned tuple. Although all of these
allocations are pfree'd, the fact that we now use a generation context
means that the memory for these pfree'd allocations won't be used again by
any other allocation due to generation.c's lack of freelists.  This could
result in sorts used for building indexes exceeding maintenance_work_mem
by a very large amount.

Here we fix it so we no longer allocate anything apart from the tuple
itself into the generation context by adding a new version of
index_form_tuple named index_form_tuple_context, which can be called to
specify the MemoryContext to allocate the tuple into.

Discussion: https://postgr.es/m/CAApHDvrHQkiFRHiGiAS-LMOvJN-eK-s762=tVzBz8ZqUea-a_A@mail.gmail.com
Backpatch-through: 15, where 40af10b57 was added.

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/0229106afab93e9868e758f4b555daafcfac4b75

Modified Files
--------------
src/backend/access/common/indextuple.c | 29 ++++++++++++++++++++++++-----
src/backend/utils/sort/tuplesort.c     |  7 ++++---
src/include/access/itup.h              |  3 +++
3 files changed, 31 insertions(+), 8 deletions(-)



view thread (2+ messages)  latest in thread

Message-ID: <E1o9BPf-001Wu1-AI@gemulon.postgresql.org>
Permalink:  ../E1o9BPf-001Wu1-AI@gemulon.postgresql.org/
Also on:    postgresql.org/message-id/E1o9BPf-001Wu1-AI@gemulon.postgresql.org

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-committers@postgresql.org
  Cc: drowley@postgresql.org
  Subject: Re: pgsql: Overload index_form_tuple to allow the memory context to be supp
  In-Reply-To: <E1o9BPf-001Wu1-AI@gemulon.postgresql.org>
  References: <E1o9BPf-001Wu1-AI@gemulon.postgresql.org>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox