From: Tom Lane <tgl@sss.pgh.pa.us>
To: Kevin Grittner <kgrittn@gmail.com>
cc: Tomas Vondra <tomas.vondra@2ndquadrant.com>,
        Andres Freund <andres@anarazel.de>,
        "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Subject: Re: what to revert
In-reply-to: 
 <CACjxUsOL0s2DWRT5kNgc_498rsrrGXVNWHGiqBiAA2BGv9kxZw@mail.gmail.com>
References: 
 <CA+TgmoYOWTtBQEL+Bv=w93bvUjbXSUw3uGnp+R29dduZ==8K0Q@mail.gmail.com>
 <CACjxUsOuf_a6KMsFBNtt8Tn8y7qNLE2Wcb9BQeRFdHD6oAKp3g@mail.gmail.com>
 <20160503162209.ufuezlkovbrdrayp@alap3.anarazel.de>
 <CACjxUsPb63TqU1uBc6zW5fsFynUB=BZ8NbFp6E=jK9qA7twibg@mail.gmail.com>
 <20160503174104.tk7oz4akwbeqsmuq@alap3.anarazel.de>
 <c7ad0f06-52c6-6284-c3a0-251807be9d3f@2ndquadrant.com>
 <20160503224241.rwtxrhgmprzqxhlv@alap3.anarazel.de>
 <eb3d6105-7165-fe71-f753-0a95281c3c19@2ndquadrant.com>
 <e17a7535-09b8-2273-0f7c-2d368390b0ed@2ndquadrant.com>
 <CACjxUsPgmm+LLG1+3d56EhCD8yEKP_b14zHGFOUpJp0Qx-J2pw@mail.gmail.com>
 <CACjxUsOL0s2DWRT5kNgc_498rsrrGXVNWHGiqBiAA2BGv9kxZw@mail.gmail.com>
Comments: In-reply-to Kevin Grittner <kgrittn@gmail.com>
	message dated "Tue, 10 May 2016 08:04:52 -0500"
Date: Tue, 10 May 2016 10:02:50 -0400
Message-ID: <2289.1462888970@sss.pgh.pa.us>
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org

Kevin Grittner <kgrittn@gmail.com> writes:
> There were 75 samples each of "disabled" and "reverted" in the
> spreadsheet.  Averaging them all, I see this:

> reverted:  290,660 TPS
> disabled:  292,014 TPS

> That's a 0.46% overall increase in performance with the patch,
> disabled, compared to reverting it.  I'm surprised that you
> consider that to be a "clearly measurable difference".  I mean, it
> was measured and it is a difference, but it seems to be well within
> the noise.  Even though it is based on 150 samples, I'm not sure we
> should consider it statistically significant.

You don't have to guess about that --- compare it to the standard
deviation within each group.

			regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers