Received: from localhost (unknown [200.46.204.183]) by developer.postgresql.org (Postfix) with ESMTP id CA2212E0079 for ; Tue, 8 Apr 2008 04:40:49 -0300 (ADT) Received: from developer.postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.204.183]) (amavisd-maia, port 10024) with ESMTP id 93497-02-9 for ; Tue, 8 Apr 2008 04:40:39 -0300 (ADT) X-Greylist: from auto-whitelisted by SQLgrey-1.7.5 Received: from svr2.hagander.net (svr2.hagander.net [88.198.128.226]) by developer.postgresql.org (Postfix) with ESMTP id 7337C2E0035 for ; Tue, 8 Apr 2008 04:40:25 -0300 (ADT) Received: from dynamic.hagander.net ([127.0.0.1]) (encrypted and authenticated) by svr2.hagander.net (Postfix) with ESMTP id 108FFDCCA3A; Tue, 8 Apr 2008 09:40:22 +0200 (CEST) Received: from mha-laptop (localhost [127.0.0.1]) by mha-laptop.hagander.net (Postfix) with ESMTP id 61D93FF09D; Tue, 8 Apr 2008 09:40:21 +0200 (CEST) Date: Tue, 8 Apr 2008 09:40:17 +0200 From: Magnus Hagander To: spotluri@ismartpanache.com Cc: Subject: Re: bulk data loading Message-ID: <20080408094017.349949af@mha-laptop> In-Reply-To: <41583.1207626107@ismartpanache.com> References: <41583.1207626107@ismartpanache.com> X-Mailer: Claws Mail 3.3.1 (GTK+ 2.12.0; i486-pc-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200804/83 X-Sequence-Number: 29576 Potluri Srikanth wrote: > Hi all, > > I need to do a bulk data loading around 704GB (log file size) at > present in 8 hrs (1 am - 9am). The data file size may increase 3 to > 5 times in future. > > Using COPY it takes 96 hrs to finish the task. > What is the best way to do it ? > > HARDWARE: SUN THUMPER/ RAID10 > OS : SOLARIS 10. > DB: Greenplum/Postgres If you're using Greenplum, you should probably be talking to the Greenplum folks. IIRC, they have made some fairly large changes to the load process, so they'll be the ones with the proper answers for you. //Magnus