Message-ID: <18200c04-84a1-46ad-b3a7-c1617f6f39ae@enterprisedb.com>
Date: Sat, 30 Mar 2024 00:40:01 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: BitmapHeapScan streaming read user and prelim refactoring
Content-Language: en-US
To: Thomas Munro <thomas.munro@gmail.com>
Cc: Melanie Plageman <melanieplageman@gmail.com>,
 Heikki Linnakangas <hlinnaka@iki.fi>, Andres Freund <andres@anarazel.de>,
 Pg Hackers <pgsql-hackers@postgresql.org>,
 Nazir Bilal Yavuz <byavuz81@gmail.com>
References: <5a172d1e-d69c-409a-b1fa-6521214c81c2@iki.fi>
 <20240323002607.abegehmmjyxrnmm2@liskov>
 <e472f0cb-bdcd-4fe9-9bf6-23a041dedc5d@enterprisedb.com>
 <20240324173833.rdtw42aur2q5hhb2@liskov>
 <58fa8dd3-a49b-4774-a9c4-7422ba8bc698@enterprisedb.com>
 <CAAKRu_Yzm6ae-Njj9DHwk0bbFC6QpzbjB=aY39civtpsUYgORQ@mail.gmail.com>
 <358c737e-772c-40dd-9ecf-35b0c05828cd@enterprisedb.com>
 <CAAKRu_ZS016J_qhs=BnCsteYYC0HgYpAbZB8mDZBvFdy_VD1MQ@mail.gmail.com>
 <20240325160709.qjo6txzu6zjxzkqy@liskov>
 <20240327193750.3mlcmzqondpj27xe@liskov>
 <7e98781c-34e0-4c02-8e5b-0500aa074271@enterprisedb.com>
 <CA+hUKGJtm_gkmW_h_02-Q9ZRcG3yOx2uzVqbCTfz7YPnTfs+DA@mail.gmail.com>
 <8330278b-9615-4030-8626-51ce714a7f54@enterprisedb.com>
 <CA+hUKGKVT1DpFG_k7xAm_6Mepeg0U76XwRPh+gx9nAGvUzZOwA@mail.gmail.com>
 <CA+hUKG+QcN9AfW+v5isNMEBJr34VPOWifyC-LX=tLN4Ddh_Srw@mail.gmail.com>
 <3ac67b68-47ff-4edb-bb9e-d86504acd573@enterprisedb.com>
 <CA+hUKGKxPEh4oUtz_9fij=DdrJRWejbf1r0qWyqpSxUe2tu3VA@mail.gmail.com>
From: Tomas Vondra <tomas.vondra@enterprisedb.com>
In-Reply-To: <CA+hUKGKxPEh4oUtz_9fij=DdrJRWejbf1r0qWyqpSxUe2tu3VA@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Archived-At: <https://www.postgresql.org/message-id/18200c04-84a1-46ad-b3a7-c1617f6f39ae%40enterprisedb.com>
Precedence: bulk

On 3/29/24 22:39, Thomas Munro wrote:
> ...
> 
>> I don't recall seeing a system with disabled readahead, but I'm
>> sure there are cases where it may not really work - it clearly can't
>> work with direct I/O, ...
> 
> Right, for direct I/O everything is slow right now including seq scan.
> We need to start asynchronous reads in the background (imagine
> literally just a bunch of background "I/O workers" running preadv() on
> your behalf to get your future buffers ready for you, or equivalently
> Linux io_uring).  That's the real goal of this project: restructuring
> so we have the information we need to do that, ie teach every part of
> PostgreSQL to predict the future in a standard and centralised way.
> Should work out better than RA heuristics, because we're not just
> driving in a straight line, we can turn corners too.
> 
>> ... but I've also not been very successful with
>> prefetching on ZFS.
> 
> posix_favise() did not do anything in OpenZFS before 2.2, maybe you
> have an older version?
> 

Sorry, I meant the prefetch (readahead) built into ZFS. I may be wrong
but I don't think the regular RA (in linux kernel) works for ZFS, right?

I was wondering if we could use this (posix_fadvise) to improve that,
essentially by issuing fadvise even for sequential patterns. But now
that I think about that, if posix_fadvise works since 2.2, maybe RA
works too now?)


regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company