pg.ddx.io  pgsql-general@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Wyatt Tellis <wyatt.tellis@gmail.com>
Cc: Ron Johnson <ronljohnsonjr@gmail.com>
Cc: pgsql-generallists.postgresql.org <pgsql-general@lists.postgresql.org>
Subject: Re: Improving pg_dump performance when handling large numbers of LOBs
Date: Mon, 05 Feb 2024 15:38:05 -0500
Message-ID: <268824.1707165485@sss.pgh.pa.us> (raw)
In-Reply-To: <CANCMp8wRi=v9vrf4HyvX9XHjyoxuu-gM8WRoUNnF+fBdmWGevw@mail.gmail.com>
References: <CANCMp8yTnRrVtEQnMMfxw_pxRSXsv40TWFUbnaR4rvDnBzJ2eg@mail.gmail.com>
	<CANzqJaBE39Kx3e5TUa-Fxkrkeu4EQMg_K-rzdNPFzjccq=gO=Q@mail.gmail.com>
	<CANCMp8wRi=v9vrf4HyvX9XHjyoxuu-gM8WRoUNnF+fBdmWGevw@mail.gmail.com>

Wyatt Tellis <wyatt.tellis@gmail.com> writes:
> Yes, the LOBs themselves are tiny, but there are a lot of them (~150
> million) which seem to be slowing down pg_dump.  Note, we did not
> design/build this system and agree that use of LOBs for this purpose was
> not necessary.

I don't know of anything much you can do at the user level.  But there
is a patchset under development to improve pg_dump's behavior with
tons of blobs:

https://www.postgresql.org/message-id/842242.1706287466@sss.pgh.pa.us

That's intended for v17, and I'm not sure how tough it might be to
adapt to v15 pg_dump, but maybe you could make that happen.  In any
case, more eyeballs reviewing that patchset would be welcome.

			regards, tom lane





view thread (12+ messages)  latest in thread

Message-ID: <268824.1707165485@sss.pgh.pa.us>
Permalink:  ../268824.1707165485@sss.pgh.pa.us/
Also on:    postgresql.org/message-id/268824.1707165485@sss.pgh.pa.us

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-general@postgresql.org
  Cc: tgl@sss.pgh.pa.us, wyatt.tellis@gmail.com, ronljohnsonjr@gmail.com
  Subject: Re: Improving pg_dump performance when handling large numbers of LOBs
  In-Reply-To: <268824.1707165485@sss.pgh.pa.us>
  References: <CANCMp8yTnRrVtEQnMMfxw_pxRSXsv40TWFUbnaR4rvDnBzJ2eg@mail.gmail.com> <CANzqJaBE39Kx3e5TUa-Fxkrkeu4EQMg_K-rzdNPFzjccq=gO=Q@mail.gmail.com> <CANCMp8wRi=v9vrf4HyvX9XHjyoxuu-gM8WRoUNnF+fBdmWGevw@mail.gmail.com> <268824.1707165485@sss.pgh.pa.us>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox