public inbox for [email protected]
help / color / mirror / Atom feedFrom: Matthias van de Meent <[email protected]>
To: Andres Freund <[email protected]>
Cc: Ants Aasma <[email protected]>
Cc: Dmitry Koterov <[email protected]>
Cc: PostgreSQL-development <[email protected]>
Subject: Re: hash_search_with_hash_value is high in "perf top" on a replica
Date: Tue, 4 Feb 2025 20:03:10 +0100
Message-ID: <CAEze2WjTHdo4F-NQ3nK8hdUw0xk6bBaHW58D3YYEezz1EdRodA@mail.gmail.com> (raw)
In-Reply-To: <ii7jyq47owdsce5bwelxusaknc7fqsnhuljnx4vfafu5ob7sy6@akljdxwtnhiw>
References: <CA+CZih5R_ZMjcuMHM3Ub10vTNDLEBjW8VsO3h-y2WKb=oK4fWw@mail.gmail.com>
<lmaidb3bb7aw2ls5sc7zaahqtmp755ykuhsfu6brzmzddlgbqi@cic3lptr5kgx>
<CANwKhkO0WJpp147eSmdiHU9KobqhkHzMi0+DR-TWz8FM7jMHVw@mail.gmail.com>
<ii7jyq47owdsce5bwelxusaknc7fqsnhuljnx4vfafu5ob7sy6@akljdxwtnhiw>
On Sat, 1 Feb 2025 at 16:55, Andres Freund <[email protected]> wrote:
>
> Hi,
>
> On 2025-02-01 15:43:41 +0100, Ants Aasma wrote:
> > On Fri, Jan 31, 2025, 15:43 Andres Freund <[email protected]> wrote:
> >
> > > > Maybe it's a red herring though, but it looks pretty suspicious.
> > >
> > > It's unfortunately not too surprising - our buffer mapping table is a
> > > pretty
> > > big bottleneck. Both because a hash table is just not a good fit for the
> > > buffer mapping table due to the lack of locality and because dynahash is
> > > really poor hash table implementation.
> > >
> >
> > I measured similar things when looking at apply throughput recently. For
> > in-cache workloads buffer lookup and locking was about half of the load.
> >
> > One other direction is to extract more memory concurrency. Prefetcher could
> > batch multiple lookups together so CPU OoO execution has a chance to fire
> > off multiple memory accesses at the same time.
>
> I think at the moment we have a *hilariously* cache-inefficient buffer lookup,
> that's the first thing to address. A hash table for buffer mapping lookups imo
> is a bad idea, due to loosing all locality in a workload that exhibits a *lot*
> of locality. But furthermore, dynahash.c is very far from a cache efficient
> hashtable implementation.
In case you might be interested, I've sent a new patch with a new
approach to reducing the buffer lookup table's memory in [0], which
attempts to create a more cache-efficient hash table implementation.
Kind regards,
Matthias van de Meent
Neon (https://neon.tech)
[0] https://www.postgresql.org/message-id/CAEze2WiRo4Zu71jwxYmqjq6XK814Avf2-kytaL6n%3DBreZR2ZbA%40mail.g...
view thread (10+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected]
Subject: Re: hash_search_with_hash_value is high in "perf top" on a replica
In-Reply-To: <CAEze2WjTHdo4F-NQ3nK8hdUw0xk6bBaHW58D3YYEezz1EdRodA@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox