From: Nathan Bossart <nathandbossart@gmail.com>
To: Matthias van de Meent <boekewurm+postgres@gmail.com>
Cc: Dilip Kumar <dilipbalaut@gmail.com>
Cc: Ranier Vilela <ranier.vf@gmail.com>
Cc: pgsql-hackers@postgresql.org
Subject: Re: use CREATE DATABASE STRATEGY = FILE_COPY in pg_upgrade
Date: Tue, 11 Jun 2024 11:03:41 -0500
Message-ID: <Zmh1XZrMLGfyYdG-@nathan> (raw)
In-Reply-To: <CAEze2WhfkYQbyRaBptT2BQ5Sv-DN0gEWCuSMWbQzCwyb8qjkBA@mail.gmail.com>
References: <Zl9ta3FtgdjizkJ5@nathan>
<CAEudQAptbRCeh08gaQKFdazOXPBqZh7UrjRcZ8fixyqkwrkUrA@mail.gmail.com>
<CAEze2WgjAoU30Kr5y8U4nTNi1F_RMW4HGo1L9ALaVTJYJbDjww@mail.gmail.com>
<CAFiTN-sj5C__1QV8WSJL3H0jQqvs5BEb14GWeb2VBEYaSApCPg@mail.gmail.com>
<CAEze2WiHAX1PYtcOpoi82G=ruTqJ9fu5PpYV4TCDd7S_W=_1OA@mail.gmail.com>
<CAFiTN-uaaaFeZq1ubckuTozWQj+A=dTUaEgdtDN5F-LNyjRfrA@mail.gmail.com>
<CAEze2WicbrOx6JWy0hK9yySm3gHFqbYE4qcDB+vaYzDCkpnL1Q@mail.gmail.com>
<ZmewBFgfGQ-Jka_n@nathan>
<CAEze2WhfkYQbyRaBptT2BQ5Sv-DN0gEWCuSMWbQzCwyb8qjkBA@mail.gmail.com>
On Tue, Jun 11, 2024 at 10:39:51AM +0200, Matthias van de Meent wrote:
> On Tue, 11 Jun 2024 at 04:01, Nathan Bossart <nathandbossart@gmail.com> wrote:
>> I did a handful of benchmarks on an r5.24xlarge that seem to prove your
>> point. The following are the durations of the pg_restore step of
>> pg_upgrade:
>>
>> * 10k empty databases, 128MB shared_buffers
>> WAL_LOG: 1m 01s
>> FILE_COPY: 0m 22s
>>
>> * 10k empty databases, 100GB shared_buffers
>> WAL_LOG: 2m 03s
>> FILE_COPY: 5m 08s
>>
>> * 2.5k databases with 10k tables each, 128MB shared_buffers
>> WAL_LOG: 17m 20s
>> FILE_COPY: 16m 44s
>>
>> * 2.5k databases with 10k tables each, 100GB shared_buffers
>> WAL_LOG: 16m 39s
>> FILE_COPY: 15m 21s
>>
>> I was surprised with the last result, but there's enough other stuff
>> happening during such a test that I hesitate to conclude much.
>
> If you still have the test data set up, could you test the attached
> patch (which does skip the checkpoints in FILE_COPY mode during binary
> upgrades)?
With your patch, I see the following:
* 10k empty databases, 128MB shared_buffers: 0m 27s
* 10k empty databases, 100GB shared_buffers: 1m 44s
I believe the reason the large buffer cache test is still quite a bit
slower is due to the truncation of pg_largeobject (specifically its call to
DropRelationsAllBuffers()). This TRUNCATE command was added in commit
bbe08b8.
--
nathan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: nathandbossart@gmail.com, boekewurm+postgres@gmail.com, dilipbalaut@gmail.com, ranier.vf@gmail.com
Subject: Re: use CREATE DATABASE STRATEGY = FILE_COPY in pg_upgrade
In-Reply-To: <Zmh1XZrMLGfyYdG-@nathan>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox