pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  

help / color / mirror / Atom feed
From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: Jim Nasby <Jim.Nasby@BlueTreble.com>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: Dilip Kumar <dilipbalaut@gmail.com>
Cc: Robert Haas <robertmhaas@gmail.com>
Cc: Andres Freund <andres@anarazel.de>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: Re: Speed up Clog Access by increasing CLOG buffers
Date: Mon, 31 Oct 2016 22:48:01 +0100
Message-ID: <5275bf49-545e-e189-48c5-17b5defc45a2@2ndquadrant.com> (raw)
In-Reply-To: <f769dd83-6288-2c37-4958-b7ddad0bc974@2ndquadrant.com>
References: <CAA4eK1+8=X9mSNeVeHg_NqMsOR-XKsjuqrYzQf=iCsdh3U4EOA@mail.gmail.com>
	<CAA4eK1JPVwPW0X8Ss+Rz+VQcPTYxCMGQuHEHfOcCTOtGqE_=ZA@mail.gmail.com>
	<09275f4d-f78d-dd69-22de-57c784dc410e@2ndquadrant.com>
	<CAFiTN-uQ+Jbd31cXvRbj48Ba6TqDUDpLKSPnsUCCYRju0Y0U8Q@mail.gmail.com>
	<CA+TgmobjO01bJ2caVWOzU=PuoKfx1X6epGokhHD8fc9hVwkhKw@mail.gmail.com>
	<CAFiTN-taV4iVkPHrxg=YCicKjBS6=QZm_cM4hbS_2q2ryLhUUw@mail.gmail.com>
	<acf46406-cf6d-41fe-4118-4e1c960b4790@2ndquadrant.com>
	<CAFiTN-tCqDucbikxue0x0dwNOAa6yzjMxJNgXGQm90jVVRSj0Q@mail.gmail.com>
	<CAFiTN-t15PjFTFQH0fBfM5jVSv6rfm5J_8vj=RGuKeFpdLgSoQ@mail.gmail.com>
	<CAA4eK1LyR2A+m=RBSZ6rcPEwJ=rVi1ADPSndXHZdjn56yqO6Vg@mail.gmail.com>
	<4a52a34f-57fa-7bcf-d34c-c15db40f0361@2ndquadrant.com>
	<CAA4eK1KTbNbZSDo=6k4YgaJh_FM20zJCKu2Yt0bxaFMv9QcSXQ@mail.gmail.com>
	<b3586234-6c80-5b64-1261-871e0e852bbb@2ndquadrant.com>
	<956b695d-f6c5-d652-bf0c-cea95981547c@BlueTreble.com>
	<f769dd83-6288-2c37-4958-b7ddad0bc974@2ndquadrant.com>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-hackers>

On 10/31/2016 02:24 PM, Tomas Vondra wrote:
> On 10/31/2016 05:01 AM, Jim Nasby wrote:
>> On 10/30/16 1:32 PM, Tomas Vondra wrote:
>>>
>>> Now, maybe this has nothing to do with PostgreSQL itself, but maybe it's
>>> some sort of CPU / OS scheduling artifact. For example, the system has
>>> 36 physical cores, 72 virtual ones (thanks to HT). I find it strange
>>> that the "good" client counts are always multiples of 72, while the
>>> "bad" ones fall in between.
>>>
>>>   72 = 72 * 1   (good)
>>>  108 = 72 * 1.5 (bad)
>>>  144 = 72 * 2   (good)
>>>  180 = 72 * 2.5 (bad)
>>>  216 = 72 * 3   (good)
>>>  252 = 72 * 3.5 (bad)
>>>  288 = 72 * 4   (good)
>>>
>>> So maybe this has something to do with how OS schedules the tasks, or
>>> maybe some internal heuristics in the CPU, or something like that.
>>
>> It might be enlightening to run a series of tests that are 72*.1 or *.2
>> apart (say, 72, 79, 86, ..., 137, 144).
>
> Yeah, I've started a benchmark with client a step of 6 clients
>
>     36 42 48 54 60 66 72 78 ... 252 258 264 270 276 282 288
>
> instead of just
>
>     36 72 108 144 180 216 252 288
>
> which did a test every 36 clients. To compensate for the 6x longer runs,
> I'm only running tests for "group-update" and "master", so I should have
> the results in ~36h.
>

So I've been curious and looked at results of the runs executed so far, 
and for the group_update patch it looks like this:

   clients  tps
  -----------------
        36  117663
        42  139791
        48  129331
        54  144970
        60  124174
        66  137227
        72  146064
        78  100267
        84  141538
        90   96607
        96  139290
       102   93976
       108  136421
       114   91848
       120  133563
       126   89801
       132  132607
       138   87912
       144  129688
       150   87221
       156  129608
       162   85403
       168  130193
       174   83863
       180  129337
       186   81968
       192  128571
       198   82053
       204  128020
       210   80768
       216  124153
       222   80493
       228  125503
       234   78950
       240  125670
       246   78418
       252  123532
       258   77623
       264  124366
       270   76726
       276  119054
       282   76960
       288  121819

So, similar saw-like behavior, perfectly periodic. But the really 
strange thing is the peaks/valleys don't match those observed before!

That is, during the previous runs, 72, 144, 216 and 288 were "good" 
while 108, 180 and 252 were "bad". But in those runs, all those client 
counts are "good" ...

Honestly, I have no idea what to think about this ...

regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers



view thread (248+ messages)  latest in thread

Message-ID: <5275bf49-545e-e189-48c5-17b5defc45a2@2ndquadrant.com>
Permalink:  ../5275bf49-545e-e189-48c5-17b5defc45a2@2ndquadrant.com/
Also on:    postgresql.org/message-id/5275bf49-545e-e189-48c5-17b5defc45a2@2ndquadrant.com

 · 

reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-hackers@postgresql.org
  Cc: tomas.vondra@2ndquadrant.com, Jim.Nasby@BlueTreble.com, amit.kapila16@gmail.com, dilipbalaut@gmail.com, robertmhaas@gmail.com, andres@anarazel.de
  Subject: Re: Speed up Clog Access by increasing CLOG buffers
  In-Reply-To: <5275bf49-545e-e189-48c5-17b5defc45a2@2ndquadrant.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox