pg.ddx.io  pgsql-bugs@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: Josef Machytka <josef.machytka@gmail.com>
Cc: pgsql-bugs@lists.postgresql.org
Subject: Re: memory problems and crash of db when deleting data from table with thousands of partitions
Date: Mon, 28 Oct 2019 17:24:46 +0100
Message-ID: <20191028162446.wlktskknjpsrj4rt@development> (raw)
In-Reply-To: <CAGvVEFsTp8cqALqPaOVxXaXMXx9V6TXtCAGobvWBb-ezjDbRtA@mail.gmail.com>
References: <CAGvVEFueqXeYL0z2fTQMWYSz7Gc9czSrOvCSNubb9rHHVL-2OA@mail.gmail.com>
	<20191023124751.pnzcos7i3ei2vjxc@development>
	<CAGvVEFsTp8cqALqPaOVxXaXMXx9V6TXtCAGobvWBb-ezjDbRtA@mail.gmail.com>

On Mon, Oct 28, 2019 at 04:18:59PM +0100, Josef Machytka wrote:
>Thank you for your email, FYI - we now did tests with PostgreSQL 12 and
>unfortunately it is also not able to handle to our case. Only difference is
>that PG 12 is not killed by OOM killer and even does not crash - which is
>good. But it reports error "out of memory" and stops the statement. So at
>least it looks like much more stable then PG 11.
>

Hmmm, this seems a bit weird to me:

  MessageContext: 52197329840 total in 8274 blocks; 75904 free (9 chunks);
52197253936 used

That context is generally meant for parse trees and other long-lived
stuff, and I wouldn't expect it to grow to 52GB of data, even if there
are many many partitions.

I wonder if this might be just another manifestation of the memory leak
from [1]. Can you provide a self-contained reproducer, i.e. a script I
could use to reproduce the issue?


[1] https://www.postgresql.org/message-id/20191024221758.vfv2enubnwmy3deu@development


regards


-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services 





view thread (6+ messages)  latest in thread

Message-ID: <20191028162446.wlktskknjpsrj4rt@development>
Permalink:  ../20191028162446.wlktskknjpsrj4rt@development/
Also on:    postgresql.org/message-id/20191028162446.wlktskknjpsrj4rt@development

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-bugs@postgresql.org
  Cc: tomas.vondra@2ndquadrant.com, josef.machytka@gmail.com
  Subject: Re: memory problems and crash of db when deleting data from table with thousands of partitions
  In-Reply-To: <20191028162446.wlktskknjpsrj4rt@development>
  References: <CAGvVEFueqXeYL0z2fTQMWYSz7Gc9czSrOvCSNubb9rHHVL-2OA@mail.gmail.com> <20191023124751.pnzcos7i3ei2vjxc@development> <CAGvVEFsTp8cqALqPaOVxXaXMXx9V6TXtCAGobvWBb-ezjDbRtA@mail.gmail.com> <20191028162446.wlktskknjpsrj4rt@development>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox