From: Fabien COELHO <coelho@cri.ensmp.fr>
To: Andres Freund <andres@2ndquadrant.com>
Cc: Tomas Vondra <tomas.vondra@2ndquadrant.com>
Cc: Robert Haas <robertmhaas@gmail.com>
Cc: PostgreSQL Developers <pgsql-hackers@postgresql.org>
Subject: Re: PATCH: pgbench - merging transaction logs
Date: Sat, 21 Mar 2015 20:35:18 +0100 (CET)
Message-ID: <alpine.DEB.2.10.1503211958280.14445@sto> (raw)
In-Reply-To: <20150321095006.GA3075@awork2.anarazel.de>
References: <54FCB11B.5050907@2ndquadrant.com>
<alpine.DEB.2.10.1503150945070.4015@sto>
<5505CF30.7030308@2ndquadrant.com>
<alpine.DEB.2.10.1503151935160.4015@sto>
<CA+TgmoZAuB+ZfMhS-LSjqsyv=gA7kBjADWf_A34cUeVTrNUmVA@mail.gmail.com>
<alpine.DEB.2.10.1503171614580.31431@sto>
<CA+TgmoZ8eFnFeg14bhWYstRdODLHa+f+zDtPjAweOgmPkLcUww@mail.gmail.com>
<alpine.DEB.2.10.1503201327170.12124@sto>
<550CCBE5.8060009@2ndquadrant.com>
<alpine.DEB.2.10.1503210907380.14445@sto>
<20150321095006.GA3075@awork2.anarazel.de>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-hackers>
Hello Andres,
>> With your worst-case figure and some rounding, it seems to look like:
>>
>> #threads collision probability performance impact
>> 2 1/40 1/3200
>> 4 1/7 1/533
>> 8 0.7 < 0.01 (about 1%)
>>
>> This suggest that for a pessimistic (ro load) fprintf overhead ratio there
>> would be a small impact even with 8 thread doing 20000 tps each.
>
> I think math like this mostly disregards hardware realities.
Hmmm. In my mind, doing the maths helps understand what may be going on.
Note that it does not preclude to check afterwards that it does indeed
correspond to reality:-)
The key suggestion of the maths is that if p*t << 1 all is (seems) well.
> You don't actually need to have actual lock contention to notice
> overhead.
The overhead assumed is 1/40 of the transaction time from Tomas' measures.
Given the ~ 18000 tps (we are talking of an in-memory read-only load
probably on the same host), transaction time for pgbench seems to be about
0.06 ms, and fprintf seems to be about 0.0015 ms (1.5 µs).
> - frequently acquiring an *uncontended* lock that resides in another
> socket's cache and where the cacheline is dirty requires relatively
> expensive cross cpu transfers. That's all besides the overhead of doing
> a lock operation itself. A lock; xaddl;, or whatever you end up using,
> has a significant cost in itself. It implies a bus lock and cache flush,
> which is far from free.
Ok, I did not assume an additional "lock cost". Do you have a figure? A
quick googling suggested figure for "lightweight mutexes" around 100 ns,
but the test conditions were unclear. If it is oky, then it is does not
change much the above maths to add that overhead.
> Additionally we're quite possibly talking about more than 8 threads.
> I've frequently used pgbench with hundreds of threads; for imo good
> reasons.
Good for you. I do not have access to a host on which this would make
sense:-)
> That all said, it's far from guaranteed that there's an actual problem
> here. If done right, i.e. the expensive formatting of the string is
> separated from the locked output to the kernel, it might end up being
> acceptable.
That is what I would like to assess. Indeed, probably snprinf (to avoid
mallocing anything) and then fputs/write/whatever would indeed help reduce
the "contention" probability, if not the actual overhead.
--
Fabien.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: coelho@cri.ensmp.fr, andres@2ndquadrant.com, tomas.vondra@2ndquadrant.com, robertmhaas@gmail.com
Subject: Re: PATCH: pgbench - merging transaction logs
In-Reply-To: <alpine.DEB.2.10.1503211958280.14445@sto>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox