Subject: Re: refactoring relation extension and BufferAlloc(), faster COPY
Message-ID: <2f2b4022-d39d-5e32-b738-1672a593ba83@amazon.com>
Date: Tue, 21 Feb 2023 16:49:37 -0600
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0)
 Gecko/20100101 Thunderbird/102.7.2
Content-Language: en-US
To: Andres Freund <andres@anarazel.de>
CC: <pgsql-hackers@postgresql.org>, Thomas Munro <thomas.munro@gmail.com>,
	Melanie Plageman <melanieplageman@gmail.com>, Yura Sokolov
	<y.sokolov@postgrespro.ru>, Robert Haas <robertmhaas@gmail.com>
References: <20221029025420.eplyow6k7tgu6he3@awork3.anarazel.de>
 <a1efd2f9-290e-b860-b490-a8bf6530b288@amazon.com>
 <20230221211256.ibxhwtdrvytr2hpt@awork3.anarazel.de>
From: Jim Nasby <nasbyj@amazon.com>
In-Reply-To: <20230221211256.ibxhwtdrvytr2hpt@awork3.anarazel.de>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
Archived-At: 
 <https://www.postgresql.org/message-id/2f2b4022-d39d-5e32-b738-1672a593ba83%40amazon.com>
Precedence: bulk


On 2/21/23 3:12 PM, Andres Freund wrote:
> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
>
>
>
> Hi,
>
> On 2023-02-21 15:00:15 -0600, Jim Nasby wrote:
>> Some food for thought: I think it's also completely fine to extend any
>> relation over a certain size by multiple blocks, regardless of concurrency.
>> E.g. 10 extra blocks on an 80MB relation is 0.1%. I don't have a good feel
>> for what algorithm would make sense here; maybe something along the lines of
>> extend = max(relpages / 2048, 128); if extend < 8 extend = 1; (presumably
>> extending by just a couple extra pages doesn't help much without
>> concurrency).
> I previously implemented just that. It's not easy to get right. You can easily
> end up with several backends each extending the relation by quite a bit, at
> the same time (or you re-introduce contention). Which can end up with a
> relation being larger by a bunch if data loading stops at some point.
>
> We might want that as well at some point, but the approach implemented in the
> patchset is precise and thus always a win, and thus should be the baseline.
Yeah, what I was suggesting would only make sense when there *wasn't* 
contention.