Subject: Re: file cloning in pg_upgrade and CREATE DATABASE
To: Michael Paquier <michael@paquier.xyz>
References: <bc9ca382-b98d-0446-f699-8c5de2307ca7@2ndquadrant.com>
 <20180319070636.GA4633@paquier.xyz> <20180319071415.GB4633@paquier.xyz>
 <20180320025849.GB4963@paquier.xyz>
 <678999c6-0967-d22a-e7d1-2a3aff83a77b@2ndquadrant.com>
 <20180322023831.GG2490@paquier.xyz>
 <9380bebc-3c16-42f0-d0e1-da7c235f8a58@2ndquadrant.com>
 <20180326061516.GC2759@paquier.xyz>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
From: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Organization: 2ndQuadrant
Message-ID: <c9c905e4-dae6-6c77-2ad4-b4dc497b6b86@2ndquadrant.com>
Date: Fri, 30 Mar 2018 16:15:39 -0400
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:52.0)
 Gecko/20100101 Thunderbird/52.6.0
MIME-Version: 1.0
In-Reply-To: <20180326061516.GC2759@paquier.xyz>
Content-Type: text/plain; charset=windows-1252
Content-Language: en-US
Content-Transfer-Encoding: 8bit
Precedence: bulk

On 3/26/18 02:15, Michael Paquier wrote:
> f8c183a has introduced the optimization that your patch is removing,
> which was discussed on this thread:
> https://www.postgresql.org/message-id/flat/4B78906A.7020309%40mark.mielke.cc

Note that that thread is from 2010 and talks about creation of a
database from the standard template being too slow on spinning rust,
because we fsync too often.  I think we have moved well past that
problem size.

I have run some more tests on both macOS and Linux with ext4, and my
results are that the bigger the flush distance, the better.  Before we
made the adjustments for APFS, we had a flush size of 64kB, now it's 1MB
and 32MB on macOS.  In my tests, I see 256MB as the best across both
platforms, and not flushing early at all is only minimally worse.

You can measure this to death, and this obviously doesn't apply equally
on all systems and configurations, but clearly some of the old
assumptions from 8 years ago are no longer applicable.

> I am not much into the internals of copy_file_range, but isn't there a
> risk to have a large range of blocks copied to discard potentially
> useful blocks from the OS cache?  That's what this patch makes me worry
> about.  Performance is good, but on a system where the OS cache is
> heavily used for a set of hot blocks this could cause performance side
> effects that I think we canot neglect.

How would we go about assessing that?  It's possible, but if
copy_file_range() really blows away all your in-use cache, that would be
surprising.

> Another thing is that 71d6d07 allowed a couple of database commands to
> be more sensitive to interruptions.  With large databases used as a base
> template it seems to me that this would cause the interruptions to be
> less responsive.

The maximum file size that we copy is 1GB and that nowadays takes maybe
10 seconds.  I think that would be an acceptable response time.

-- 
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services