From: Tom Lane <tgl@sss.pgh.pa.us>
To: Haribabu Kommi <kommi.haribabu@gmail.com>
Cc: iserbin@bostonsd.ru
Cc: pgsql-bugs@lists.postgresql.org
Subject: Re: BUG #15660: pg_dump memory leaks when dumping LOBs
Date: Thu, 28 Feb 2019 12:30:48 -0500
Message-ID: <11676.1551375048@sss.pgh.pa.us> (raw)
In-Reply-To: <CAJrrPGc8O8KC39nFB7VOVKvwK9kNnnB_86Uxpu04yhg-8CFdsA@mail.gmail.com>
References: <15660-db101adab620a17a@postgresql.org>
<CAJrrPGc8O8KC39nFB7VOVKvwK9kNnnB_86Uxpu04yhg-8CFdsA@mail.gmail.com>
Haribabu Kommi <kommi.haribabu@gmail.com> writes:
> On Wed, Feb 27, 2019 at 10:58 PM PG Bug reporting form <
> noreply@postgresql.org> wrote:
>> One of our customers faced an unexpected behaviour when using pg_dump to
>> export one of DBs - pg_dump consumed all the server memory and got
>> terminated with following error:
> pg_dump process collects all the database objects first (tables, indexes
> and etc) and then it write them into the file/archive.
> In your scenario, there are many large objects that are present which leads
> to allocate memory for the each object before it gets dumped leads to out of
> memory.
Yeah. We've heard about this repeatedly since large objects were promoted
to be full-fledged objects with individual owners and ACL info. It was
argued at the time that large objects would be, well, large, and therefore
there wouldn't be *that* many of them, and so it'd be okay for pg_dump to
treat them as full-fledged objects with their own dump-table-of-contents
entries. Evidently, not everybody uses them that way :-(.
At some point I think we're going to have to revisit that decision in
pg_dump. Maybe we could merge things so that we have one TOC entry
for all blobs that share the same owner and ACL? But it comes up
seldom enough that nobody's gotten motivated to do the work.
In the short run, the only answer is to run pg_dump on beefier iron,
or else split up the dump as Haribabu suggests (although TBH I'm
afraid that might not work either --- if memory serves, pg_dump tends
to pull in all the per-object info even if it's not going to dump it
all ...)
regards, tom lane
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-bugs@postgresql.org
Cc: tgl@sss.pgh.pa.us, kommi.haribabu@gmail.com, iserbin@bostonsd.ru, pgsql-bugs@lists.postgresql.org
Subject: Re: BUG #15660: pg_dump memory leaks when dumping LOBs
In-Reply-To: <11676.1551375048@sss.pgh.pa.us>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox