Date: Tue, 11 Jun 2024 11:03:41 -0500
From: Nathan Bossart <nathandbossart@gmail.com>
To: Matthias van de Meent <boekewurm+postgres@gmail.com>
Cc: Dilip Kumar <dilipbalaut@gmail.com>,
	Ranier Vilela <ranier.vf@gmail.com>, pgsql-hackers@postgresql.org
Subject: Re: use CREATE DATABASE STRATEGY = FILE_COPY in pg_upgrade
Message-ID: <Zmh1XZrMLGfyYdG-@nathan>
References: <Zl9ta3FtgdjizkJ5@nathan>
 <CAEudQAptbRCeh08gaQKFdazOXPBqZh7UrjRcZ8fixyqkwrkUrA@mail.gmail.com>
 <CAEze2WgjAoU30Kr5y8U4nTNi1F_RMW4HGo1L9ALaVTJYJbDjww@mail.gmail.com>
 <CAFiTN-sj5C__1QV8WSJL3H0jQqvs5BEb14GWeb2VBEYaSApCPg@mail.gmail.com>
 <CAEze2WiHAX1PYtcOpoi82G=ruTqJ9fu5PpYV4TCDd7S_W=_1OA@mail.gmail.com>
 <CAFiTN-uaaaFeZq1ubckuTozWQj+A=dTUaEgdtDN5F-LNyjRfrA@mail.gmail.com>
 <CAEze2WicbrOx6JWy0hK9yySm3gHFqbYE4qcDB+vaYzDCkpnL1Q@mail.gmail.com>
 <ZmewBFgfGQ-Jka_n@nathan>
 <CAEze2WhfkYQbyRaBptT2BQ5Sv-DN0gEWCuSMWbQzCwyb8qjkBA@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: 
 <CAEze2WhfkYQbyRaBptT2BQ5Sv-DN0gEWCuSMWbQzCwyb8qjkBA@mail.gmail.com>
Archived-At: <https://www.postgresql.org/message-id/Zmh1XZrMLGfyYdG-%40nathan>
Precedence: bulk

On Tue, Jun 11, 2024 at 10:39:51AM +0200, Matthias van de Meent wrote:
> On Tue, 11 Jun 2024 at 04:01, Nathan Bossart <nathandbossart@gmail.com> wrote:
>> I did a handful of benchmarks on an r5.24xlarge that seem to prove your
>> point.  The following are the durations of the pg_restore step of
>> pg_upgrade:
>>
>> * 10k empty databases, 128MB shared_buffers
>>   WAL_LOG:   1m 01s
>>   FILE_COPY: 0m 22s
>>
>> * 10k empty databases, 100GB shared_buffers
>>   WAL_LOG:   2m 03s
>>   FILE_COPY: 5m 08s
>>
>> * 2.5k databases with 10k tables each, 128MB shared_buffers
>>   WAL_LOG:   17m 20s
>>   FILE_COPY: 16m 44s
>>
>> * 2.5k databases with 10k tables each, 100GB shared_buffers
>>   WAL_LOG:   16m 39s
>>   FILE_COPY: 15m 21s
>>
>> I was surprised with the last result, but there's enough other stuff
>> happening during such a test that I hesitate to conclude much.
> 
> If you still have the test data set up, could you test the attached
> patch (which does skip the checkpoints in FILE_COPY mode during binary
> upgrades)?

With your patch, I see the following:

* 10k empty databases, 128MB shared_buffers: 0m 27s
* 10k empty databases, 100GB shared_buffers: 1m 44s

I believe the reason the large buffer cache test is still quite a bit
slower is due to the truncation of pg_largeobject (specifically its call to
DropRelationsAllBuffers()).  This TRUNCATE command was added in commit
bbe08b8.

-- 
nathan