Received: from localhost (unknown [200.46.204.184]) by developer.postgresql.org (Postfix) with ESMTP id A5E962E0030 for ; Mon, 28 Apr 2008 04:49:47 -0300 (ADT) Received: from developer.postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.204.184]) (amavisd-maia, port 10024) with ESMTP id 86190-04 for ; Mon, 28 Apr 2008 04:49:40 -0300 (ADT) X-Greylist: from auto-whitelisted by SQLgrey-1.7.5 Received: from hi-media.com (9.140-14-84.ripe.coltfrance.com [84.14.140.9]) by developer.postgresql.org (Postfix) with ESMTP id 43CA52E002D for ; Mon, 28 Apr 2008 04:49:43 -0300 (ADT) Received: from dim (gut75-1-81-57-27-121.fbx.proxad.net [81.57.27.121]) by hi-media.com (Postfix) with ESMTP id 9F21B188263; Mon, 28 Apr 2008 09:44:54 +0200 (CEST) From: Dimitri Fontaine Organization: Hi-Media To: pgsql-performance@postgresql.org Subject: Re: Best practice to load a huge table from ORACLE to PG Date: Mon, 28 Apr 2008 09:49:37 +0200 User-Agent: KMail/1.9.9 Cc: Greg Smith , Adonias Malosso References: <8a5d3c890804260625o7ad0700fp79f951c58c242720@mail.gmail.com> In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart1767675.qCJLfGrkED"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Message-Id: <200804280949.40101.dfontaine@hi-media.com> X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200804/386 X-Sequence-Number: 29879 --nextPart1767675.qCJLfGrkED Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Hi, Le dimanche 27 avril 2008, Greg Smith a =E9crit=A0: > than SQL*PLUS. Then on the PostgreSQL side, you could run multiple COPY > sessions importing at once to read this data all back in, because COPY > will bottleneck at the CPU level before the disks will if you've got > reasonable storage hardware. Latest pgloader version has been made to handle this exact case, so if you= =20 want to take this route, please consider pgloader 2.3.0: http://pgloader.projects.postgresql.org/#_parallel_loading http://pgfoundry.org/projects/pgloader/ Another good reason to consider using pgloader is when the datafile contain= s=20 erroneous input lines and you don't want the COPY transaction to abort. Tho= se=20 error lines will get rejected out by pgloader while the correct ones will g= et=20 COPYied in. Regards, =2D-=20 dim --nextPart1767675.qCJLfGrkED Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQBIFYGUlBXRlnbh1bkRAlcuAJ9X8NCdCyz61H8LwRR17L/m3vBAmACgrtg9 oxPM0SLlDbJC/cld9F4x7l8= =qIhZ -----END PGP SIGNATURE----- --nextPart1767675.qCJLfGrkED--