public inbox for [email protected]
help / color / mirror / Atom feedFrom: Bagga, Rishu <[email protected]>
To: Li, Yong <[email protected]>
To: Robert Haas <[email protected]>
To: Debnath, Shawn <[email protected]>
Cc: Andrey Borodin <[email protected]>
Cc: PostgreSQL Hackers <[email protected]>
Cc: Aleksander Alekseev <[email protected]>
Cc: Shyrabokau, Anton <[email protected]>
Subject: Re: Proposal to add page headers to SLRU pages
Date: Tue, 19 Dec 2023 02:23:24 +0000
Message-ID: <[email protected]> (raw)
In-Reply-To: <DM4PR07MB9812DA12DA729E8C709DEDD9B98AA@DM4PR07MB9812.namprd07.prod.outlook.com>
References: <CAJ7c6TMUY=H2Hn=YbdYimk_NfVGq1QR9UhaPsE2Zd+8rgzdZMg@mail.gmail.com>
<IA1PR07MB982925731C4C0B38DC8F2F7FB98BA@IA1PR07MB9829.namprd07.prod.outlook.com>
<[email protected]>
<[email protected]>
<[email protected]>
<CA+TgmoZ2i5smJuGDRqsUbEKHga_xOc27f6=59ci9aLjA0c2_KA@mail.gmail.com>
<DM4PR07MB9812DA12DA729E8C709DEDD9B98AA@DM4PR07MB9812.namprd07.prod.outlook.com>
On Thu, Dec 8, 2023 at 1:36 AM Li, Yong <[email protected]> wrote:
>Given so many different approaches were discussed, I have started a
>wiki to record and collaborate all efforts towards SLRU
>improvements. The wiki provides a concise overview of all the ideas
>discussed and can serve as a portal for all historical
>discussions. Currently, the wiki summarizes four recent threads
>ranging from identifier format change to page header change, to moving
>SLRU into the main buffer pool, to reduce lock contention on SLRU
>latches. We can keep the patch related discussions in this thread and
>use the wiki as a live document for larger scale collaborations.
>The wiki page is
>here: https://wiki.postgresql.org/wiki/SLRU_improvements
>Regarding the benefits of this patch, here is a detailed explanation:
1. Checksum is added to each page, allowing us to verify if a page has
been corrupted when read from the disk.
2. The ad-hoc LSN group structure is removed from the SLRU cache
control data and is replaced by the page LSN in the page header.
This allows us to use the same WAL protocol as used by pages in the
main buffer pool: flush all redo logs up to the page LSN before
flushing the page itself. If we move SLRU caches into the main
buffer pool, this change fits naturally.
3. It leaves further optimizations open. We can continue to pursue the
goal of moving SLRU into the main buffer pool, or we can follow the
lock partition idea. This change by itself does not conflict with
either proposal.
>Also, the patch is now complete and is ready for review. All check-
>world tests including tap tests passed with this patch.
Hi Yong,
I agree we should break the effort for the SLRU optimization into
smaller chunks after having worked on some of the bigger patches and
facing difficulty in making progress that way.
The patch looks mostly good to me; though one thing that I thought about
differently with the upgrade portion is where we should keep the logic
of re-writing the CLOG files.
There is a precedent introduced back in Postgres v9.6 in making on disk
page format changes across different in visibility map: [1]
code comment:
* In versions of PostgreSQL prior to catversion 201603011, PostgreSQL's
* visibility map included one bit per heap page; it now includes two.
* When upgrading a cluster from before that time to a current PostgreSQL
* version, we could refuse to copy visibility maps from the old cluster
* to the new cluster; the next VACUUM would recreate them, but at the
* price of scanning the entire table. So, instead, we rewrite the old
* visibility maps in the new format.
This work is being done in file.c – it seems to me the proper way to
proceed would be to continue writing on-disk upgrade logic here.
Besides that this looks good to me, would like to hear what others have to say.
Thanks,
Rishu Bagga
Amazon Web Services (AWS)
[1] https://github.com/postgres/postgres/commit/7087166a88fe0c04fc6636d0d6d6bea1737fc1fb
view thread (22+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]
Subject: Re: Proposal to add page headers to SLRU pages
In-Reply-To: <[email protected]>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox