pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Tomas Vondra <tomas.vondra@enterprisedb.com>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: Dilip Kumar <dilipbalaut@gmail.com>
Cc: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Cc: Andres Freund <andres@anarazel.de>
Cc: Petr Jelinek <petr.jelinek@enterprisedb.com>
Cc: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Subject: Re: Decoding speculative insert with toast leaks memory
Date: Sat, 29 May 2021 14:15:10 +0200
Message-ID: <0d35243e-cf09-81a8-1dc3-0e52800732e9@enterprisedb.com> (raw)
In-Reply-To: <CAA4eK1Kmj9HzdDPOW3HFaq0A0=d7Ng_qyH3T=7yk0=MYcPZ6Qg@mail.gmail.com>
References: <CAExHW5sPKF-Oovx_qZe4p5oM6Dvof7_P+XgsNAViug15Fm99jA@mail.gmail.com>
	<CAA4eK1LSxfzn1qGNS3n7Dy9MAmyCsgmhnxG=51N50mJVMPqxrg@mail.gmail.com>
	<CAA4eK1Kr9j1fr-=BZBeP=3-N+D-NpapOiBw37f7ceXi4Kho0Ew@mail.gmail.com>
	<CAFiTN-vkOsxSbXX=54nCD2cKpEzJ8tR8LSmwTQADEwOBqoYqig@mail.gmail.com>
	<CAA4eK1K41Kg01k5AkU-uhzkike5G4VsGNTd7arpt8kG32GZb=A@mail.gmail.com>
	<CAFiTN-vHmMm=Mq8VBhOO5NgKe1As914rdo0uMNxQrwELkHRFPw@mail.gmail.com>
	<f1342d07-a91c-f2e8-ef15-18a8ebfdb910@enterprisedb.com>
	<CAA4eK1Kmj9HzdDPOW3HFaq0A0=d7Ng_qyH3T=7yk0=MYcPZ6Qg@mail.gmail.com>

On 5/29/21 6:29 AM, Amit Kapila wrote:
> On Fri, May 28, 2021 at 5:16 PM Tomas Vondra
> <tomas.vondra@enterprisedb.com> wrote:
>>
>> I wonder if there's a way to free the TOASTed data earlier, instead of
>> waiting until the end of the transaction (as this patch does).
>>
> 
> IIUC we are anyway freeing the toasted data at the next
> insert/update/delete. We can try to free at other change message types
> like REORDER_BUFFER_CHANGE_MESSAGE but as you said that may make the
> patch more complex, so it seems better to do the fix on the lines of
> what is proposed in the patch.
> 

+1

Even if we started doing what you mention (freeing the hash for other
change types), we'd still need to do what the patch proposes because the
speculative insert may be the last change in the transaction. For the
other cases it works as a mitigation, so that we don't leak the memory
forever.

So let's get this committed, perhaps with a comment explaining that it
might be possible to reset earlier if needed.

regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company





view thread (66+ messages)  latest in thread

Message-ID: <0d35243e-cf09-81a8-1dc3-0e52800732e9@enterprisedb.com>
Permalink:  ../0d35243e-cf09-81a8-1dc3-0e52800732e9@enterprisedb.com/
Also on:    postgresql.org/message-id/0d35243e-cf09-81a8-1dc3-0e52800732e9@enterprisedb.com

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-hackers@postgresql.org
  Cc: tomas.vondra@enterprisedb.com, amit.kapila16@gmail.com, dilipbalaut@gmail.com, ashutosh.bapat.oss@gmail.com, andres@anarazel.de, petr.jelinek@enterprisedb.com, peter.eisentraut@2ndquadrant.com
  Subject: Re: Decoding speculative insert with toast leaks memory
  In-Reply-To: <0d35243e-cf09-81a8-1dc3-0e52800732e9@enterprisedb.com>
  References: <CAExHW5sPKF-Oovx_qZe4p5oM6Dvof7_P+XgsNAViug15Fm99jA@mail.gmail.com> <CAA4eK1LSxfzn1qGNS3n7Dy9MAmyCsgmhnxG=51N50mJVMPqxrg@mail.gmail.com> <CAA4eK1Kr9j1fr-=BZBeP=3-N+D-NpapOiBw37f7ceXi4Kho0Ew@mail.gmail.com> <CAFiTN-vkOsxSbXX=54nCD2cKpEzJ8tR8LSmwTQADEwOBqoYqig@mail.gmail.com> <CAA4eK1K41Kg01k5AkU-uhzkike5G4VsGNTd7arpt8kG32GZb=A@mail.gmail.com> <CAFiTN-vHmMm=Mq8VBhOO5NgKe1As914rdo0uMNxQrwELkHRFPw@mail.gmail.com> <f1342d07-a91c-f2e8-ef15-18a8ebfdb910@enterprisedb.com> <CAA4eK1Kmj9HzdDPOW3HFaq0A0=d7Ng_qyH3T=7yk0=MYcPZ6Qg@mail.gmail.com> <0d35243e-cf09-81a8-1dc3-0e52800732e9@enterprisedb.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox