From: Bruce Momjian <bruce@momjian.us>
To: Pavan Deolasee <pavan.deolasee@gmail.com>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: Re: Patch: Write Amplification Reduction Method (WARM)
Date: Thu, 1 Sep 2016 12:14:46 -0400
Message-ID: <20160901161446.GA25168@momjian.us> (raw)
In-Reply-To: <CABOikdN7yozYYPoYaJcz8=p50qHeOJZdJkhOR9KvQkh7vBW4VA@mail.gmail.com>
References: <CABOikdMNy6yowA+wTGK9RVd8iw+CzqHeQSGpW7Yka_4RSZ_LOQ@mail.gmail.com>
<20160831200329.GC31352@momjian.us>
<CABOikdN7yozYYPoYaJcz8=p50qHeOJZdJkhOR9KvQkh7vBW4VA@mail.gmail.com>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-hackers>
On Thu, Sep 1, 2016 at 02:37:40PM +0530, Pavan Deolasee wrote:
> I like the simplified approach, as long as it doesn't block further
> improvements.
>
>
>
> Yes, the proposed approach is simple yet does not stop us from improving things
> further. Moreover it has shown good performance characteristics and I believe
> it's a good first step.
Agreed. This is BIG. Do you think it can be done for PG 10?
> Thanks. What's also interesting and something that headline numbers don't show
> is that WARM TPS is as much as 3 times of master TPS when the percentage of
> WARM updates is very high. Notice the spike in TPS in the comparison graph.
>
> Results with non-default heap fill factor are even better. In both cases, the
> improvement in TPS stays constant over long periods.
Yes, I expect the benefits of this to show up in better long-term
performance.
> > During first heap scan of VACUUM, we look for tuples with HEAP_WARM_TUPLE
> set.
> > If all live tuples in the chain are either marked with Blue flag or Red
> flag
> > (but no mix of Red and Blue), then the chain is a candidate for HOT
> conversion.
>
> Uh, if the chain is all blue, then there is are WARM entries so it
> already a HOT chain, so there is nothing to do, right?
>
>
> For aborted WARM updates, the heap chain may be all blue, but there may still
> be a red index pointer which must be cleared before we allow further WARM
> updates to the chain.
Ah, understood now. Thanks.
> Why not just remember the tid of chains converted from WARM to HOT, then
> use "amrecheck" on an index entry matching that tid to see if the index
> matches one of the entries in the chain.
>
>
> That will require random access to heap during index vacuum phase, something I
> would like to avoid. But we can have that as a fall back solution for handling
> aborted vacuums.
Yes, that is true. So the challenge is figuring how which of the index
entries pointing to the same tid is valid, and coloring helps with that?
> (It will match all of them or
> none of them, because they are all red.) I don't see a point in
> coloring the index entries as reds as later you would have to convert to
> blue in the WARM-to-HOT conversion, and a vacuum crash could lead to
> inconsistencies.
>
>
> Yes, that's a concern since the conversion of red to blue will also need to WAL
> logged to ensure that a crash doesn't leave us in inconsistent state. I still
> think that this will be an overall improvement as compared to allowing one WARM
> update per chain.
OK. I will think some more on this to see if I can come with another
approach.
>
>
> Consider that you can just call "amrecheck" on the few
> chains that have converted from WARM to HOT. I believe this is more
> crash-safe too. However, if you have converted WARM to HOT in the heap,
> but crash durin the index entry removal, you could potentially have
> duplicates in the index later, which is bad.
>
> As you probably already noted, we clear heap flags only after all indexes are
> cleared of duplicate entries and hence a crash in between should not cause any
> correctness issue. As long as heap tuples are marked as warm, amrecheck will
> ensure that only valid tuples are returned to the caller.
OK, got it.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: bruce@momjian.us, pavan.deolasee@gmail.com
Subject: Re: Patch: Write Amplification Reduction Method (WARM)
In-Reply-To: <20160901161446.GA25168@momjian.us>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox