Re: Speed up Clog Access by increasing CLOG buffers

pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  
help / color / mirror / Atom feed

From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: Robert Haas <robertmhaas@gmail.com>
Cc: Jim Nasby <Jim.Nasby@bluetreble.com>
Cc: Dilip Kumar <dilipbalaut@gmail.com>
Cc: Andres Freund <andres@anarazel.de>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: Re: Speed up Clog Access by increasing CLOG buffers
Date: Wed, 28 Dec 2016 00:13:12 +0100
Message-ID: <91d57161-d3ea-0cc2-6066-80713e4f90d7@2ndquadrant.com> (raw)
In-Reply-To: <CAA4eK1KAysCmBaYGbFXyu0wGTZuMWeNgYBHNd3goNrfD0-G2CA@mail.gmail.com>
References: <CAA4eK1+8=X9mSNeVeHg_NqMsOR-XKsjuqrYzQf=iCsdh3U4EOA@mail.gmail.com>
	<CAFiTN-t15PjFTFQH0fBfM5jVSv6rfm5J_8vj=RGuKeFpdLgSoQ@mail.gmail.com>
	<CAA4eK1LyR2A+m=RBSZ6rcPEwJ=rVi1ADPSndXHZdjn56yqO6Vg@mail.gmail.com>
	<4a52a34f-57fa-7bcf-d34c-c15db40f0361@2ndquadrant.com>
	<CAA4eK1KTbNbZSDo=6k4YgaJh_FM20zJCKu2Yt0bxaFMv9QcSXQ@mail.gmail.com>
	<b3586234-6c80-5b64-1261-871e0e852bbb@2ndquadrant.com>
	<956b695d-f6c5-d652-bf0c-cea95981547c@BlueTreble.com>
	<f769dd83-6288-2c37-4958-b7ddad0bc974@2ndquadrant.com>
	<5275bf49-545e-e189-48c5-17b5defc45a2@2ndquadrant.com>
	<CA+TgmobmDLM1dzFvN_tJoBunarMpz+cehx1cVfv-QMKk4rZLPQ@mail.gmail.com>
	<ca714e28-81a2-440c-bd92-79fe1fa0fc15@2ndquadrant.com>
	<CAA4eK1JjatUZu0+HCi=5VM1q-hFgN_OhegPAwEUJqxf-7pESbg@mail.gmail.com>
	<04e39ac9-af28-0fda-8f72-7268197d281f@2ndquadrant.com>
	<84c22fbb-b9c4-a02f-384b-b4feb2c67193@2ndquadrant.com>
	<CAA4eK1KAysCmBaYGbFXyu0wGTZuMWeNgYBHNd3goNrfD0-G2CA@mail.gmail.com>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-hackers>

On 12/23/2016 03:58 AM, Amit Kapila wrote:
> On Thu, Dec 22, 2016 at 6:59 PM, Tomas Vondra
> <tomas.vondra@2ndquadrant.com> wrote:
>> Hi,
>>
>> But as discussed with Amit in Tokyo at pgconf.asia, I got access to a
>> Power8e machine (IBM 8247-22L to be precise). It's a much smaller machine
>> compared to the x86 one, though - it only has 24 cores in 2 sockets, 128GB
>> of RAM and less powerful storage, for example.
>>
>> I've repeated a subset of x86 tests and pushed them to
>>
>>     https://bitbucket.org/tvondra/power8-results-2
>>
>> The new results are prefixed with "power-" and I've tried to put them right
>> next to the "same" x86 tests.
>>
>> In all cases the patches significantly reduce the contention on
>> CLogControlLock, just like on x86. Which is good and expected.
>>
>
> The results look positive.  Do you think we can conclude based on all
> the tests you and Dilip have done, that we can move forward with this
> patch (in particular group-update) or do you still want to do more
> tests?   I am aware that in one of the tests we have observed that
> reducing contention on CLOGControlLock has increased the contention on
> WALWriteLock, but I feel we can leave that point as a note to
> committer and let him take a final call.  From the code perspective
> already Robert and Andres have taken one pass of review and I have
> addressed all their comments, so surely more review of code can help,
> but I think that is not a big deal considering patch size is
> relatively small.
>

Yes, I believe that seems like a reasonable conclusion. I've done a few 
more tests on the Power machine with data placed on a tmpfs filesystem 
(to minimize all the I/O overhead), but the results are the same.

I don't think more testing is needed at this point, at lest not with the 
synthetic test cases we've been using for the testing. The patch already 
received way more benchmarking than most other patches.

regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

view thread (248+ messages)  latest in thread

Message-ID: <91d57161-d3ea-0cc2-6066-80713e4f90d7@2ndquadrant.com>
Permalink:  ../91d57161-d3ea-0cc2-6066-80713e4f90d7@2ndquadrant.com/
Also on:    postgresql.org/message-id/91d57161-d3ea-0cc2-6066-80713e4f90d7@2ndquadrant.com

 ·

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-hackers@postgresql.org
  Cc: tomas.vondra@2ndquadrant.com, amit.kapila16@gmail.com, robertmhaas@gmail.com, Jim.Nasby@bluetreble.com, dilipbalaut@gmail.com, andres@anarazel.de
  Subject: Re: Speed up Clog Access by increasing CLOG buffers
  In-Reply-To: <91d57161-d3ea-0cc2-6066-80713e4f90d7@2ndquadrant.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox