pg.ddx.io  pgsql-general@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Bruce Momjian <bruce@momjian.us>
To: David Rowley <dgrowleyml@gmail.com>
Cc: Ron <ronljohnsonjr@gmail.com>
Cc: pgsql-general@lists.postgresql.org, Andres Freund <andres@anarazel.de>
Subject: Re: Pg 16: will pg_dump & pg_restore be faster?
Date: Tue, 30 May 2023 21:13:08 -0400
Message-ID: <ZHafJF3FadgFr/5A@momjian.us> (raw)
In-Reply-To: <CAApHDvoD-QsDzqKEMpG07dWU=5g1ZWSK_tjR5mC=a+79ZudddA@mail.gmail.com>
References: <7e1a8736-d74c-8de1-679f-f47b250a13cc@gmail.com>
	<CAApHDvoD-QsDzqKEMpG07dWU=5g1ZWSK_tjR5mC=a+79ZudddA@mail.gmail.com>

On Wed, May 31, 2023 at 09:14:20AM +1200, David Rowley wrote:
> On Wed, 31 May 2023 at 08:54, Ron <ronljohnsonjr@gmail.com> wrote:
> > https://www.postgresql.org/about/news/postgresql-16-beta-1-released-2643/
> > says "PostgreSQL 16 can also improve the performance of concurrent bulk
> > loading of data using COPY up to 300%."
> >
> > Since pg_dump & pg_restore use COPY (or something very similar), will the
> > speed increase translate to higher speeds for those utilities?
> 
> I think the improvements to relation extension only help when multiple
> backends need to extend the relation at the same time.  pg_restore can
> have multiple workers, but the tasks that each worker performs are
> only divided as far as an entire table, i.e. 2 workers will never be
> working on the same table at the same time. So there is no concurrency
> in terms of 2 or more workers working on loading data into the same
> table at the same time.
> 
> It might be an interesting project now that we have TidRange scans, to
> have pg_dump split larger tables into chunks so that they can be
> restored in parallel.

Uh, the release notes say:

	<!--
	Author: Andres Freund <andres@anarazel.de>
	2023-04-06 [00d1e02be] hio: Use ExtendBufferedRelBy() to extend tables more eff
	Author: Andres Freund <andres@anarazel.de>
	2023-04-06 [26158b852] Use ExtendBufferedRelTo() in XLogReadBufferExtended()
	-->
	
	<listitem>
	<para>
	Allow more efficient addition of heap and index pages (Andres Freund)
	</para>
	</listitem>

There is no mention of concurrency being a requirement.  Is it wrong?  I
think there was a question of whether you had to add _multiple_ blocks
ot get a benefit, not if concurrency was needed.  This email about the
release notes didn't mention the concurrent requirement:

	https://www.postgresql.org/message-id/20230521171341.jjxykfsefsek4kzj%40awork3.anarazel.de


	While the case of extending by multiple pages improved the most, even
	extending by a single page at a time got a good bit more scalable. Maybe
	just "Improve efficiency of extending relations"?

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.





view thread (11+ messages)  latest in thread

Message-ID: <ZHafJF3FadgFr/5A@momjian.us>
Permalink:  ../ZHafJF3FadgFr%2F5A@momjian.us/
Also on:    postgresql.org/message-id/ZHafJF3FadgFr/5A@momjian.us

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-general@postgresql.org
  Cc: bruce@momjian.us, dgrowleyml@gmail.com, ronljohnsonjr@gmail.com, andres@anarazel.de
  Subject: Re: Pg 16: will pg_dump & pg_restore be faster?
  In-Reply-To: <ZHafJF3FadgFr/5A@momjian.us>
  References: <7e1a8736-d74c-8de1-679f-f47b250a13cc@gmail.com> <CAApHDvoD-QsDzqKEMpG07dWU=5g1ZWSK_tjR5mC=a+79ZudddA@mail.gmail.com> <ZHafJF3FadgFr/5A@momjian.us>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox