pg.ddx.io  pgsql-general@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Jonathan S. Katz <jkatz@postgresql.org>
To: David Rowley <dgrowleyml@gmail.com>
To: Bruce Momjian <bruce@momjian.us>
Cc: Ron <ronljohnsonjr@gmail.com>
Cc: pgsql-general@lists.postgresql.org, Andres Freund <andres@anarazel.de>
Subject: Re: Pg 16: will pg_dump & pg_restore be faster?
Date: Fri, 2 Jun 2023 08:14:45 -0400
Message-ID: <d966a173-d66c-c873-a154-50ab822ea933@postgresql.org> (raw)
In-Reply-To: <CAApHDvrQ1bpzyS8k=nHsL0rNFNOkiGmiJRQGO4jUb4yPS4EysQ@mail.gmail.com>
References: <7e1a8736-d74c-8de1-679f-f47b250a13cc@gmail.com>
	<CAApHDvoD-QsDzqKEMpG07dWU=5g1ZWSK_tjR5mC=a+79ZudddA@mail.gmail.com>
	<ZHafJF3FadgFr/5A@momjian.us>
	<CAApHDvrQ1bpzyS8k=nHsL0rNFNOkiGmiJRQGO4jUb4yPS4EysQ@mail.gmail.com>

On 5/30/23 10:05 PM, David Rowley wrote:

> My understanding had been that concurrency was required, but I see the
> commit message for 00d1e02be mentions:
> 
>> Even single threaded
>> COPY is measurably faster, primarily due to not dirtying pages while
>> extending, if supported by the operating system (see commit 4d330a61bb1).
> 
> If that's the case then maybe the beta release notes could be edited
> slightly to reflect this. Maybe something like:
> 
> "Relation extensions have been improved allowing faster bulk loading
> of data using COPY. These improvements are more significant when
> multiple processes are concurrently loading data into the same table."
> 
> The current text of "PostgreSQL 16 can also improve the performance of
> concurrent bulk loading of data using COPY up to 300%." does lead me
> to believe that nothing has been done to improve things when only a
> single backend is involved.

Typically once a release announcement is out, we'll only edit it if it's 
inaccurate. I don't think the statement in the release announcement is 
inaccurate, as it specifies that concurrent bulk loading is faster.

I had based the description on what Andres described in the original 
discussion and through reading[1], which showed a "measurable" 
improvement as the commit message said, but it was not to the same 
degree as concurrently loading. It does still seem impactful -- the 
results show up to 20% improvement on a single backend -- but the bigger 
story was around the concurrency.

I'm -0.5 for revising the announcement, but I also don't want people to 
miss out on testing this. I'd be OK with this:

"PostgreSQL 16 can also improve the performance of bulk loading of data, 
with some tests showing using up to 300% improvement when concurrently 
executing `COPY` commands."

Thanks,

Jonathan

[1] 
https://www.postgresql.org/message-id/20221029025420.eplyow6k7tgu6he3@awork3.anarazel.de



Attachments:

  OpenPGP_signature (application/pgp-signature, 839B) download

view thread (11+ messages)  latest in thread

Message-ID: <d966a173-d66c-c873-a154-50ab822ea933@postgresql.org>
Permalink:  ../d966a173-d66c-c873-a154-50ab822ea933@postgresql.org/
Also on:    postgresql.org/message-id/d966a173-d66c-c873-a154-50ab822ea933@postgresql.org

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-general@postgresql.org
  Cc: jkatz@postgresql.org, dgrowleyml@gmail.com, bruce@momjian.us, ronljohnsonjr@gmail.com, andres@anarazel.de
  Subject: Re: Pg 16: will pg_dump & pg_restore be faster?
  In-Reply-To: <d966a173-d66c-c873-a154-50ab822ea933@postgresql.org>
  References: <7e1a8736-d74c-8de1-679f-f47b250a13cc@gmail.com> <CAApHDvoD-QsDzqKEMpG07dWU=5g1ZWSK_tjR5mC=a+79ZudddA@mail.gmail.com> <ZHafJF3FadgFr/5A@momjian.us> <CAApHDvrQ1bpzyS8k=nHsL0rNFNOkiGmiJRQGO4jUb4yPS4EysQ@mail.gmail.com> <d966a173-d66c-c873-a154-50ab822ea933@postgresql.org>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox