Received: from malur.postgresql.org ([217.196.149.56])
	by arkaria.postgresql.org with esmtp (Exim 4.72) (envelope-from
	<pgsql-performance-owner+M49456=pgsql-performance-ar@postgresql.org>)
	id 1UFQ3y-0002Ha-Ly for pgsql-performance@arkaria.postgresql.org;
	Tue, 12 Mar 2013 14:20:22 +0000
Received: from localhost ([127.0.0.1] helo=postgresql.org)
	by malur.postgresql.org with smtp (Exim 4.72) (envelope-from
	<pgsql-performance-owner+M49456=pgsql-performance-ar@postgresql.org>)
	id 1UFQ3x-00080l-Uh for pgsql-performance@arkaria.postgresql.org;
	Tue, 12 Mar 2013 14:20:21 +0000
Received: from makus.postgresql.org ([2001:4800:7903:4::125])
	by malur.postgresql.org with esmtp (Exim 4.72)
	(envelope-from <scrawford@pinpointresearch.com>) id 1UFQ3x-00080g-04
	for pgsql-performance@postgresql.org; Tue, 12 Mar 2013 14:20:21 +0000
Received: from cerberus.pinpointresearch.com ([66.7.238.130]
	helo=polaris.pinpointresearch.com)
	by makus.postgresql.org with esmtp (Exim 4.72)
	(envelope-from <scrawford@pinpointresearch.com>) id 1UFQ3v-0000Cz-B2
	for pgsql-performance@postgresql.org; Tue, 12 Mar 2013 14:20:20 +0000
Received: from [192.168.1.179] (betelgeuse.pinpointresearch.com
	[192.168.1.179])
	by polaris.pinpointresearch.com (Postfix) with ESMTP id 4C9F3E00EC84;
	Tue, 12 Mar 2013 07:20:18 -0700 (PDT)
Message-ID: <513F39A2.7020902@pinpointresearch.com>
Date: Tue, 12 Mar 2013 07:20:18 -0700
From: Steve Crawford <scrawford@pinpointresearch.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:17.0) Gecko/20130221 Thunderbird/17.0.3
MIME-Version: 1.0
To: Misa Simic <misa.simic@gmail.com>
CC: "pgsql-performance@postgresql.org" <pgsql-performance@postgresql.org>
Subject: Re: Slow concurrent processing
References: 
 <CAH3i69mZ-e1rQshbimu24GVKHV5pKx-oPHaELqnZYo6Ed_y5TA@mail.gmail.com>
In-Reply-To: 
 <CAH3i69mZ-e1rQshbimu24GVKHV5pKx-oPHaELqnZYo6Ed_y5TA@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-Pg-Spam-Score: -4.5 (----)
List-Archive: <http://archives.postgresql.org/pgsql-performance>
List-Help: <mailto:majordomo@postgresql.org?body=help>
List-ID: <pgsql-performance.postgresql.org>
List-Owner: <mailto:pgsql-performance-owner@postgresql.org>
List-Post: <mailto:pgsql-performance@postgresql.org>
List-Subscribe: <mailto:majordomo@postgresql.org?body=sub%20pgsql-performance>
List-Unsubscribe: 
 <mailto:majordomo@postgresql.org?body=unsub%20pgsql-performance>
X-Mailing-List: pgsql-performance
Precedence: bulk
Sender: pgsql-performance-owner@postgresql.org

On 03/11/2013 08:55 PM, Misa Simic wrote:
> Hi all,
>
> We have one table with list of "records for processing"...
>
> We loop trough that table and call one long runing function:
>
>  do_the_math_for_record(record_id)...<snip>...
>
> but - if replace do_the_math_and_save_results with pg_sleep(1); To 
> simulate long running function so processing each row - last 1 sec...
>
> Sequential processing last as expected 300 seconds!
>
> Concurrent processing last faster with higher number of 
> concurrent connections - about 30 seconds with 30 connections! (much 
> faster - and expected...)
>
> however, if we return our: do_the_math_and_save_results - we can't get 
> better results in concurrent processing...

Sleep will not have any significant impact on CPU, memory or disk use 
and thus is not a simulation of actual processing.

All you have really shown us so far is a black box. Please provide an 
overview of your schemas and the type of processing you are attempting 
on them.

Cheers,
Steve



-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance