Date: Sat, 2 May 2015 16:30:08 -0400 (AST)
From: Fabien COELHO <coelho@cri.ensmp.fr>
To: Tomas Vondra <tomas.vondra@2ndquadrant.com>
cc: PostgreSQL Developers <pgsql-hackers@postgresql.org>
Subject: Re: PATCH: pgbench - merging transaction logs
In-Reply-To: <5544F8EC.6030908@2ndquadrant.com>
Message-ID: <alpine.DEB.2.10.1505021604170.31061@sto>
References: <54FCB11B.5050907@2ndquadrant.com>
 <alpine.DEB.2.10.1503150945070.4015@sto> <5505CF30.7030308@2ndquadrant.com>
 <alpine.DEB.2.10.1503151935160.4015@sto>
 <CA+TgmoZAuB+ZfMhS-LSjqsyv=gA7kBjADWf_A34cUeVTrNUmVA@mail.gmail.com>
 <alpine.DEB.2.10.1503171614580.31431@sto>
 <CA+TgmoZ8eFnFeg14bhWYstRdODLHa+f+zDtPjAweOgmPkLcUww@mail.gmail.com>
 <alpine.DEB.2.10.1503201327170.12124@sto> <550CCBE5.8060009@2ndquadrant.com>
 <alpine.DEB.2.10.1503281016080.17117@sto> <551730E9.108@2ndquadrant.com>
 <alpine.DEB.2.10.1503291003160.17117@sto>
 <5519F65F.7040005@2ndquadrant.com> <alpine.DEB.2.10.1505020916130.8161@sto>
 <5544F8EC.6030908@2ndquadrant.com>
User-Agent: Alpine 2.10 (DEB 1266 2009-07-14)
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org


> It might be done in the other direction, though - the "writer" thread 
> might collect current results at the end of the interval.

Yep, you can indeed accumulate per thread and sum on the end of the 
interval, but a lock is still needed if you want exact figures.

ISTM that it is what is done for --progress, without even bothering to 
lock anything (even if transactions are fast and some inconsistency arise, 
this is just for showing how things are faring on stderr, so no big deal).

>> "fprintf" does a lot of processing, so it is the main issue.
>
> The question is whether the processing happens while holding the lock, 
> though. I don't think that's the case.

Alas I'm pretty nearly sure it is necessarily the case, there is no 
intermediate buffer in fprintf, the buffering is done trough the file 
handler so a lock must be acquire throughout format processing.

https://www.gnu.org/software/libc/manual/html_node/Streams-and-Threads.html#Streams-and-Threads

"The POSIX standard requires that by default the stream operations are 
atomic.  I.e., issuing two stream operations for the same stream in two 
threads at the same time will cause the operations to be executed as if 
they were issued sequentially. The buffer operations performed while 
reading or writing are protected from other uses of the same stream. To do 
this each stream has an internal lock object which has to be (implicitly) 
acquired before any work can be done. "

-- 
Fabien.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers