From spotluri@ismartpanache.com Mon Mar 31 06:26:54 2008 Received: from localhost (unknown [200.46.204.184]) by developer.postgresql.org (Postfix) with ESMTP id 53CDF2E002D for ; Mon, 31 Mar 2008 03:26:54 -0300 (ADT) Received: from developer.postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.204.184]) (amavisd-maia, port 10024) with ESMTP id 79211-06 for ; Mon, 31 Mar 2008 03:26:51 -0300 (ADT) X-Greylist: from auto-whitelisted by SQLgrey-1.7.5 Received: from app01.sm.onr.chicago.hostway (fw.mfg.onr.siteprotect.com [64.26.60.152]) by developer.postgresql.org (Postfix) with ESMTP id EBAB72E002C for ; Mon, 31 Mar 2008 03:26:53 -0300 (ADT) Received: from localhost (localhost [127.0.0.1]) by app01.sm.onr.chicago.hostway (Postfix) with ESMTP id 9B5AE8DC0E4 for ; Mon, 31 Mar 2008 01:26:52 -0500 (CDT) MIME-Version: 1.0 X-Mailer: AtMail PHP 5.04 Message-ID: <57026.1206944812@ismartpanache.com> To: Reply-To: spotluri@ismartpanache.com Content-Type: text/html; charset="utf-8" X-Origin: 203.84.191.122 Date: Mon, 31 Mar 2008 01:26:52 -0500 Subject: how to improve COPY timings or reduce data loading time using copy From: "Potluri Srikanth" Content-Transfer-Encoding: quoted-printable X-Archive-Number: 200803/266 X-Sequence-Number: 28749 Hi team,

I am using COPY to load data in my tables from CSV file.
File size : 16 gb
time taken to drop index + truncate data + load data + apply index is : 1.5= hrs.

how can i reduce the time ?

OS : SOLARIS 10
Machine : THUMPER (x4500, SUN )
Datatbase : greenplum (based on postgresql)
postgres version : 8.4
greenplum version : 3.0.0.2

Your suggestion will be of great help to me .


Thanks & Regards,
Srikanth k Potluri
+63 9177444783(philippines)

From depesz@depesz.com Mon Mar 31 07:51:28 2008 Received: from localhost (unknown [200.46.204.183]) by developer.postgresql.org (Postfix) with ESMTP id 42D112E00A3 for ; Mon, 31 Mar 2008 04:51:28 -0300 (ADT) Received: from developer.postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.204.183]) (amavisd-maia, port 10024) with ESMTP id 80769-04 for ; Mon, 31 Mar 2008 04:51:27 -0300 (ADT) X-Greylist: from auto-whitelisted by SQLgrey-1.7.5 Received: from depesz.com (depesz.com [88.198.37.40]) by developer.postgresql.org (Postfix) with ESMTP id B0C542E00A0 for ; Mon, 31 Mar 2008 04:51:27 -0300 (ADT) Received: from depesz by depesz.com with local (Exim 4.63) (envelope-from ) id 1JgEna-0001Yj-K4; Mon, 31 Mar 2008 09:51:22 +0200 Date: Mon, 31 Mar 2008 09:51:22 +0200 From: hubert depesz lubaczewski To: Potluri Srikanth Cc: pgsql-admin@postgresql.org Subject: Re: how to improve COPY timings or reduce data loading time using copy Message-ID: <20080331075122.GA5620@depesz.com> Reply-To: depesz@depesz.com References: <57026.1206944812@ismartpanache.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <57026.1206944812@ismartpanache.com> User-Agent: Mutt/1.5.13 (2006-08-11) X-Archive-Number: 200803/268 X-Sequence-Number: 28751 On Mon, Mar 31, 2008 at 01:26:52AM -0500, Potluri Srikanth wrote: > how can i reduce the time ? try this: http://pgbulkload.projects.postgresql.org/ depesz -- quicksil1er: "postgres is excellent, but like any DB it requires a highly paid DBA. here's my CV!" :) http://www.depesz.com/ - blog dla ciebie (i moje CV)