pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  

help / color / mirror / Atom feed
From: Yeb Havinga <yebhavinga@gmail.com>
To: Venki Ramachandran <venki_ramachandran@yahoo.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Pavel Stehule <pavel.stehule@gmail.com>
Cc: Samuel Gendler <sgendler@ideasculptor.com>
Cc: pgsql-performance@postgresql.org <pgsql-performance@postgresql.org>
Subject: Re: Parallel Scaling of a pgplsql problem
Date: Thu, 26 Apr 2012 08:49:12 +0200
Message-ID: <4F98EFE8.5090609@gmail.com> (raw)
In-Reply-To: <1335408018.29771.YahooMailNeo@web184514.mail.ne1.yahoo.com>
References: <1335379923.47084.YahooMailNeo@web184514.mail.ne1.yahoo.com>
	<CAEV0TzAVQXf42n9vXu26JAe0-eQ9AgwwAErYFn0HMm8ScQaodA@mail.gmail.com>
	<1335388323.84458.YahooMailNeo@web184517.mail.ne1.yahoo.com>
	<CAFj8pRB7gvUocfW5Uv5-NVFtRPSE97AS6Sbk5gzrdMweCpNS_Q@mail.gmail.com>
	<1335390338.10325.YahooMailNeo@web184505.mail.ne1.yahoo.com>
	<5452.1335390728@sss.pgh.pa.us>
	<1335408018.29771.YahooMailNeo@web184514.mail.ne1.yahoo.com>

On 2012-04-26 04:40, Venki Ramachandran wrote:
> Thanks Tom, clock_timestamp() worked. Appreciate it!!! and Sorry was 
> hurrying to get this done at work and hence did not read through.
>
> Can you comment on how you would solve the original problem? Even if I 
> can get  the 11 seconds down to 500 ms for one pair, running it for 
> 300k pairs will take multiple hours. How can one write a combination 
> of a bash script/pgplsql code so as to use all 8 cores of a server. I 
> am seeing that this is just executing in one session/process.

You want to compare a calculation on the cross product 'employee x 
employee'. If employee is partitioned into emp1, emp2, ... emp8, the 
cross product is equal to the union of emp1 x employee, emp2 x employee, 
.. emp8 x employee. Each of these 8 cross products on partitions can be 
executed in parallel. I'd look into dblink to execute each of the 8 
cross products in parallel, and then union all of those results.

http://www.postgresql.org/docs/9.1/static/contrib-dblink-connect.html

regards,
Yeb

view thread (13+ messages)  latest in thread

Message-ID: <4F98EFE8.5090609@gmail.com>
Permalink:  ../4F98EFE8.5090609@gmail.com/
Also on:    postgresql.org/message-id/4F98EFE8.5090609@gmail.com

 · 

reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-performance@postgresql.org
  Cc: yebhavinga@gmail.com, venki_ramachandran@yahoo.com, tgl@sss.pgh.pa.us, pavel.stehule@gmail.com, sgendler@ideasculptor.com
  Subject: Re: Parallel Scaling of a pgplsql problem
  In-Reply-To: <4F98EFE8.5090609@gmail.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox