From: Steve Crawford <scrawford@pinpointresearch.com>
To: Misa Simic <misa.simic@gmail.com>
Cc: pgsql-performance@postgresql.org <pgsql-performance@postgresql.org>
Subject: Re: Slow concurrent processing
Date: Tue, 12 Mar 2013 07:20:18 -0700
Message-ID: <513F39A2.7020902@pinpointresearch.com> (raw)
In-Reply-To: <CAH3i69mZ-e1rQshbimu24GVKHV5pKx-oPHaELqnZYo6Ed_y5TA@mail.gmail.com>
References: <CAH3i69mZ-e1rQshbimu24GVKHV5pKx-oPHaELqnZYo6Ed_y5TA@mail.gmail.com>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-performance>
On 03/11/2013 08:55 PM, Misa Simic wrote:
> Hi all,
>
> We have one table with list of "records for processing"...
>
> We loop trough that table and call one long runing function:
>
> do_the_math_for_record(record_id)...<snip>...
>
> but - if replace do_the_math_and_save_results with pg_sleep(1); To
> simulate long running function so processing each row - last 1 sec...
>
> Sequential processing last as expected 300 seconds!
>
> Concurrent processing last faster with higher number of
> concurrent connections - about 30 seconds with 30 connections! (much
> faster - and expected...)
>
> however, if we return our: do_the_math_and_save_results - we can't get
> better results in concurrent processing...
Sleep will not have any significant impact on CPU, memory or disk use
and thus is not a simulation of actual processing.
All you have really shown us so far is a black box. Please provide an
overview of your schemas and the type of processing you are attempting
on them.
Cheers,
Steve
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: scrawford@pinpointresearch.com, misa.simic@gmail.com
Subject: Re: Slow concurrent processing
In-Reply-To: <513F39A2.7020902@pinpointresearch.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox