public inbox for [email protected]
help / color / mirror / Atom feedFrom: Tomas Vondra <[email protected]>
To: pgsql-hackers <[email protected]>
Subject: Re: linux cachestat in file Readv and Prefetch
Date: Sun, 18 Feb 2024 00:10:44 +0100
Message-ID: <[email protected]> (raw)
In-Reply-To: <[email protected]>
References: <[email protected]>
On 1/18/24 01:25, Cedric Villemain wrote:
> Hi,
>
> I was testing the index prefetch and streamIO patches and I added
> cachestat() syscall to get a better view of the prefetching.
>
> It's a new linux syscall, it requires 6.5, it provides numerous
> interesting information from the VM for the range of pages examined.
> It's way way faster than the old mincore() and provides much more
> valuable information:
>
> uint64 nr_cache; /* Number of cached pages */
> uint64 nr_dirty; /* Number of dirty pages */
> uint64 nr_writeback; /* Number of pages marked for writeback. */
> uint64 nr_evicted; /* Number of pages evicted from the cache. */
> /*
> * Number of recently evicted pages. A page is recently evicted if its
> * last eviction was recent enough that its reentry to the cache would
> * indicate that it is actively being used by the system, and that
there
> * is memory pressure on the system.
> */
> uint64 nr_recently_evicted;
>
>
> While here I also added some quick tweaks to suspend prefetching on
> memory pressure.
I may be missing some important bit behind this idea, but this does not
seem like a great idea to me. The comment added to FilePrefetch says this:
/*
* last time we visit this file (somewhere), nr_recently_evicted pages
* of the range were just removed from vm cache, it's a sign a memory
* pressure. so do not prefetch further.
* it is hard to guess if it is always the right choice in absence of
* more information like:
* - prefetching distance expected overall
* - access pattern/backend maybe
*/
Firstly, is this even a good way to detect memory pressure? It's clearly
limited to a single 1GB segment, so what's the chance we'll even see the
"pressure" on a big database with many files?
If we close/reopen the file (which on large databases we tend to do very
often) how does that affect the data reported for the file descriptor?
I'm not sure I even agree with the idea that we should stop prefetching
when there is memory pressure. IMHO it's perfectly fine to keep
prefetching stuff even if it triggers eviction of unnecessary pages from
page cache. That's kinda why the eviction exists.
> It's working but I have absolutely not checked the performance impact of
> my additions.
>
Well ... I'd argue at least some basic evaluation of performance is a
rather important / expected part of a proposal for a patch that aims to
improve a performance-focused feature. It's impossible to have any sort
of discussion about such patch without that.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
view thread (3+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected]
Subject: Re: linux cachestat in file Readv and Prefetch
In-Reply-To: <[email protected]>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox