Feedback-ID: id4a34324:Fastmail
Date: Mon, 23 Oct 2023 19:44:35 -0700
From: Andres Freund <andres@anarazel.de>
To: Andrei Lepikhov <a.lepikhov@postgrespro.ru>
Cc: Stephen Frost <sfrost@snowman.net>, reid.thompson@crunchydata.com,
	Arne Roland <A.Roland@index.de>,
	"pgsql-hackers@lists.postgresql.org" <pgsql-hackers@lists.postgresql.org>,
	vignesh C <vignesh21@gmail.com>,
	Justin Pryzby <pryzby@telsasoft.com>,
	Ibrar Ahmed <ibrar.ahmad@gmail.com>,
	"stephen.frost" <stephen.frost@crunchydata.com>
Subject: Re: Add the ability to limit the amount of memory that can be
 allocated to backends.
Message-ID: <20231024024435.yaqrajcchcliwhjl@awork3.anarazel.de>
References: <7912c911af51d5cf28c611190bf3d463b9209343.camel@crunchydata.com>
 <4edafedc0f8acb12a2979088ac1317bd7dd42145.camel@crunchydata.com>
 <f62b4107-227d-49e7-a9be-104d1ceb03d9@postgrespro.ru>
 <268e0ac7-8a81-4d65-8b40-b62c4b3f1bf9@postgrespro.ru>
 <ZTArWsctGn5fEVPR@tamriel.snowman.net>
 <48548d40-634b-4943-a737-3c9d95eacf06@postgrespro.ru>
 <ZTGoUt+pH8uPucfm@tamriel.snowman.net>
 <c6520d47-e584-4287-833c-82779cc166e0@postgrespro.ru>
 <ZTJ1BfHC99LSzoHw@tamriel.snowman.net>
 <8fcb4406-49f5-4069-b8e9-197a38004ddd@postgrespro.ru>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <8fcb4406-49f5-4069-b8e9-197a38004ddd@postgrespro.ru>
Archived-At: <https://www.postgresql.org/message-id/20231024024435.yaqrajcchcliwhjl%40awork3.anarazel.de>
Precedence: bulk

Hi,

On 2023-10-24 09:39:42 +0700, Andrei Lepikhov wrote:
> On 20/10/2023 19:39, Stephen Frost wrote:
> Greetings,
> > * Andrei Lepikhov (a.lepikhov@postgrespro.ru) wrote:
> > > The only issue I worry about is the uncertainty and clutter that can be
> > > created by this feature. In the worst case, when we have a complex error
> > > stack (including the extension's CATCH sections, exceptions in stored
> > > procedures, etc.), the backend will throw the memory limit error repeatedly.
> > 
> > I'm not seeing what additional uncertainty or clutter there is- this is,
> > again, exactly the same as what happens today on a system with
> > overcommit disabled and I don't feel like we get a lot of complaints
> > about this today.
> 
> Maybe I missed something or see this feature from an alternate point of view
> (as an extension developer), but overcommit is more useful so far: it kills
> a process.

In case of postgres it doesn't just kill one postgres, it leads to *all*
connections being terminated.


> It means that after restart, the backend/background worker will have an
> initial internal state. With this limit enabled, we need to remember that
> each function call can cause an error, and we have to remember it using
> static PG_CATCH sections where we must rearrange local variables to the
> initial (?) state. So, it complicates development.

You need to be aware of errors being thrown regardless this feature, as
out-of-memory errors can be encountered today already. There also are many
other kinds of errors that can be thrown.

Greetings,

Andres Freund