Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1q4AOp-0003cW-UC for pgsql-general@arkaria.postgresql.org; Wed, 31 May 2023 01:13:15 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1q4AOo-0004lF-HX for pgsql-general@arkaria.postgresql.org; Wed, 31 May 2023 01:13:14 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1q4AOo-0004l6-6u for pgsql-general@lists.postgresql.org; Wed, 31 May 2023 01:13:14 +0000 Received: from momjian.us ([72.94.173.45]) by magus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1q4AOj-003Ani-4y for pgsql-general@lists.postgresql.org; Wed, 31 May 2023 01:13:13 +0000 Received: from bruce by momjian.us with local (Exim 4.94.2) (envelope-from ) id 1q4AOi-0038jk-4E; Tue, 30 May 2023 21:13:08 -0400 Date: Tue, 30 May 2023 21:13:08 -0400 From: Bruce Momjian To: David Rowley Cc: Ron , pgsql-general@lists.postgresql.org, Andres Freund Subject: Re: Pg 16: will pg_dump & pg_restore be faster? Message-ID: References: <7e1a8736-d74c-8de1-679f-f47b250a13cc@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk On Wed, May 31, 2023 at 09:14:20AM +1200, David Rowley wrote: > On Wed, 31 May 2023 at 08:54, Ron wrote: > > https://www.postgresql.org/about/news/postgresql-16-beta-1-released-2643/ > > says "PostgreSQL 16 can also improve the performance of concurrent bulk > > loading of data using COPY up to 300%." > > > > Since pg_dump & pg_restore use COPY (or something very similar), will the > > speed increase translate to higher speeds for those utilities? > > I think the improvements to relation extension only help when multiple > backends need to extend the relation at the same time. pg_restore can > have multiple workers, but the tasks that each worker performs are > only divided as far as an entire table, i.e. 2 workers will never be > working on the same table at the same time. So there is no concurrency > in terms of 2 or more workers working on loading data into the same > table at the same time. > > It might be an interesting project now that we have TidRange scans, to > have pg_dump split larger tables into chunks so that they can be > restored in parallel. Uh, the release notes say: Allow more efficient addition of heap and index pages (Andres Freund) There is no mention of concurrency being a requirement. Is it wrong? I think there was a question of whether you had to add _multiple_ blocks ot get a benefit, not if concurrency was needed. This email about the release notes didn't mention the concurrent requirement: https://www.postgresql.org/message-id/20230521171341.jjxykfsefsek4kzj%40awork3.anarazel.de While the case of extending by multiple pages improved the most, even extending by a single page at a time got a good bit more scalable. Maybe just "Improve efficiency of extending relations"? -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com Only you can decide what is important to you.