From: Fabien COELHO <coelho@cri.ensmp.fr>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Tomas Vondra <tomas.vondra@2ndquadrant.com>
Cc: PostgreSQL Developers <pgsql-hackers@postgresql.org>
Subject: Re: PATCH: pgbench - merging transaction logs
Date: Fri, 20 Mar 2015 13:43:14 +0100 (CET)
Message-ID: <alpine.DEB.2.10.1503201327170.12124@sto> (raw)
In-Reply-To: <CA+TgmoZ8eFnFeg14bhWYstRdODLHa+f+zDtPjAweOgmPkLcUww@mail.gmail.com>
References: <54FCB11B.5050907@2ndquadrant.com>
<alpine.DEB.2.10.1503150945070.4015@sto>
<5505CF30.7030308@2ndquadrant.com>
<alpine.DEB.2.10.1503151935160.4015@sto>
<CA+TgmoZAuB+ZfMhS-LSjqsyv=gA7kBjADWf_A34cUeVTrNUmVA@mail.gmail.com>
<alpine.DEB.2.10.1503171614580.31431@sto>
<CA+TgmoZ8eFnFeg14bhWYstRdODLHa+f+zDtPjAweOgmPkLcUww@mail.gmail.com>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-hackers>
Hello Robert,
>> The fprintf we are talking about occurs at most once per pgbench
>> transaction, possibly much less when aggregation is activated, and this
>> transaction involves networks exchanges and possibly disk writes on the
>> server.
>
> random() was occurring four times per transaction rather than once,
> but OTOH I think fprintf() is probably a much heavier-weight
> operation.
Yes, sure.
My point is that if there are many threads and tremendous TPS, the
*detailed* per-transaction log (aka simple log) is probably a bad choice
anyway, and the aggregated version is the way to go.
Note that even without mutex fprintf may be considered a "heavy function"
which is going to slow down the transaction rate significantly. That could
be tested as well.
It is possible to reduce the lock time by preparing the string (which
would mean introducing buffers) and just do a "fputs" under mutex. That
would not reduce the print time anyway, and that may add malloc/free
operations, though.
> The way to know if there's a real problem here is to test it, but I'd be
> pretty surprised if there isn't.
Indeed, I think I can contrive a simple example where it is, basically a
more or less empty or read only transaction (eg SELECT 1).
My opinion is that there is a tradeoff between code simplicity and later
maintenance vs feature benefit.
If threads are assumed and fprintf is used, the feature is much simpler to
implement, and the maintenance is lighter. The alternative implementation
means reparsing the generated files over and over for merging their
contents.
Also, I do not think that the detailed log provides much benefit with very
fast transactions, where probably the aggregate is a much better choice
anyway. If the user persists, she may generate a per-thread log and merge
it later, in which case a merge script is needed, but I do not think that
would be a bad thing.
Obviously, all that is only my opinion and is quite debatable.
--
Fabien.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: coelho@cri.ensmp.fr, robertmhaas@gmail.com, tomas.vondra@2ndquadrant.com
Subject: Re: PATCH: pgbench - merging transaction logs
In-Reply-To: <alpine.DEB.2.10.1503201327170.12124@sto>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox