Received: from magus.postgresql.org (magus.postgresql.org [87.238.57.229]) by mail.postgresql.org (Postfix) with ESMTP id 78ED61E62E85 for ; Wed, 25 Apr 2012 15:52:22 -0300 (ADT) Received: from nm11-vm0.bullet.mail.sp2.yahoo.com ([98.139.91.240]) by magus.postgresql.org with smtp (Exim 4.72) (envelope-from ) id 1SN7K5-0004eJ-DK for pgsql-performance@postgresql.org; Wed, 25 Apr 2012 18:52:21 +0000 Received: from [98.139.91.66] by nm11.bullet.mail.sp2.yahoo.com with NNFMP; 25 Apr 2012 18:52:03 -0000 Received: from [98.139.44.78] by tm6.bullet.mail.sp2.yahoo.com with NNFMP; 25 Apr 2012 18:52:03 -0000 Received: from [127.0.0.1] by omp1015.access.mail.sp2.yahoo.com with NNFMP; 25 Apr 2012 18:52:03 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 936897.86782.bm@omp1015.access.mail.sp2.yahoo.com Received: (qmail 53255 invoked by uid 60001); 25 Apr 2012 18:52:03 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1335379923; bh=b15jTkyczJVCidGE1w8aGFH500eUUg2dBSMP/gUAC20=; h=X-YMail-OSG:Received:X-Mailer:Message-ID:Date:From:Reply-To:Subject:To:MIME-Version:Content-Type; b=2UQxdxliqmZIT5/rFD5p3Dexs+V8/4qA0aCcr5ecpvii1v2ANXngiq+Mu360PPU4SQBK+i7ydmErStqf/+FbI7/onNwpTzLJiDxi3JApxyL3SAW8MYG0QcPWZ9/TJ1UigzjaDNZsY2EL3heBVEfRt50MtNQ8Yq9UWpzFzLlhudY= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Mailer:Message-ID:Date:From:Reply-To:Subject:To:MIME-Version:Content-Type; b=AjVL5l5AEGIUq8L01T3+n4/KBSiNZdpY2r0laFjp4CudLGMMAHq/K5cj4PuwYMwNQfR70hzECjgQuLfjAW4DPccKA+DenGsdVLVcnC+wHZAlkKad9woye1C6f7e6nwebOBXWFyS6Kv4eXbLp8zSR1/ubw9IpZKA0vr1IkpjZIi4=; X-YMail-OSG: 1bPaNf4VM1l2L82zAZS4ZCJ7i9s8JYesqLJVhBd7pdcCVjz CUJAFPgVNUePqZSsIAoaJFI0vRv34sGgScGVU1Z6nptzjxajn6fB.iPVN.LB gr9bChxWrfUwi2Vbp8xfgSfbYBxJaB5MV_3fT2II6nbT8bWg7o8m0QsdVKct FGw3ntZI.E9VxcHRwIc4qvyVQzMJ.MCHy7uidzzmihvUvr_jM0qYbwW9DY73 mw2mElfibPOG9axSvQsZYp2TeBX6y4Q6IXwEUddjhUyC0_pK9v5Ks6tw.apd XdZOUMNWy6TrYSV7zCymtmAHJE3ztTLJD7iy.r0NhB1C63kBxHd3v2acdFgX UWaKBSiBs2X40arkaaCRN.GtvGgSsTclycfcmi7KqNlXpfoWtkdkSnwTe2ao 43Y_EetoDJqceasKSew5Xvf5dxA4WepYCotv5f2LHjYXATM.YF7C3Lj3CF1C VDxVUNbQIZDek Received: from [24.6.62.191] by web184514.mail.ne1.yahoo.com via HTTP; Wed, 25 Apr 2012 11:52:03 PDT X-Mailer: YahooMailWebService/0.8.117.340979 Message-ID: <1335379923.47084.YahooMailNeo@web184514.mail.ne1.yahoo.com> Date: Wed, 25 Apr 2012 11:52:03 -0700 (PDT) From: Venki Ramachandran Reply-To: Venki Ramachandran Subject: Parallel Scaling of a pgplsql problem To: "pgsql-performance@postgresql.org" MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="-986170028-289403806-1335379923=:47084" X-Pg-Spam-Score: -0.6 (/) X-Archive-Number: 201204/142 X-Sequence-Number: 46788 ---986170028-289403806-1335379923=:47084 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Hi all:=0ACan someone please guide me as to how to solve this problem? If t= his is the wrong forum, please let me know which one to post this one in. I= am new to Postgres (about 3 months into it)=0A=0AI have PostGres 9.0 datab= ase in a AWS server (x-large) and a pgplsql program that does some computat= ion. It takes in a date range and for one pair of personnel (two employees = in a company) it calculates some values over the time period. It takes abou= t 40ms (milli seconds) to complete and give me the answer. All good so far.= =0A=0ANow I have to run the same pgplsql on all possible combinations of em= ployees and with 542 employees that is about say 300,000 unique pairs.=0A= =0ASo (300000 * 40)/(1000 * 60 * 60) =3D 3.33 hours and I have to rank them= and show it on a screen. No user wants to wait for 3 hours, =A0they can pr= obably wait for 10 minutes (even that is too much for a UI application). Ho= w do I solve this scaling problem? Can I have multiple parellel sessions an= d each session have multiple/processes that do a pair each at 40 ms and the= n collate the results. Does PostGres or pgplsql have any parallel computing= capability.=0A=0AThanks, Venki ---986170028-289403806-1335379923=:47084 Content-Type: text/html; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable
Hi all:
Can som= eone please guide me as to how to solve this problem? If this is the wrong = forum, please let me know which one to post this one in. I am new to Postgr= es (about 3 months into it)

I have PostGres 9.0 da= tabase in a AWS server (x-large) and a pgplsql program that does some compu= tation. It takes in a date range and for one pair of personnel (two employe= es in a company) it calculates some values over the time period. It takes a= bout 40ms (milli seconds) to complete and give me the answer. All good so f= ar.

Now I have to run the same pgplsql on all poss= ible combinations of employees and with 542 employees that is about say 300= ,000 unique pairs.

So (300000 * 40)/(1000 * 60 * 6= 0) =3D 3.33 hours and I have to rank them and show it on a screen. No user wants to wait for 3 hours,  they can probably wait for 10 minute= s (even that is too much for a UI application). How do I solve this scaling= problem? Can I have multiple parellel sessions and each session have multi= ple/processes that do a pair each at 40 ms and then collate the results. Do= es PostGres or pgplsql have any parallel computing capability.
Thanks, Venki
---986170028-289403806-1335379923=:47084--