X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.2]) by svr1.postgresql.org (Postfix) with ESMTP id A7CFBD1C9AB for ; Tue, 3 Feb 2004 15:10:32 +0000 (GMT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (neptune.hub.org [200.46.204.2]) (amavisd-new, port 10024) with ESMTP id 64191-04 for ; Tue, 3 Feb 2004 11:10:27 -0400 (AST) Received: from sss.pgh.pa.us (unknown [192.204.191.242]) by svr1.postgresql.org (Postfix) with ESMTP id 051E1D1B83D for ; Tue, 3 Feb 2004 11:10:25 -0400 (AST) Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1]) by sss.pgh.pa.us (8.12.11/8.12.11) with ESMTP id i13FATtL017868; Tue, 3 Feb 2004 10:10:29 -0500 (EST) To: Kevin Brown Cc: pgsql-hackers@postgresql.org Subject: Re: COPY from question In-reply-to: <20040203105746.GE2608@filer> References: <1070584326.18838.235.camel@cletus.lyris.com> <20040203105746.GE2608@filer> Comments: In-reply-to Kevin Brown message dated "Tue, 03 Feb 2004 02:57:46 -0800" Date: Tue, 03 Feb 2004 10:10:29 -0500 Message-ID: <17867.1075821029@sss.pgh.pa.us> From: Tom Lane X-Virus-Scanned: by amavisd-new at postgresql.org X-Archive-Number: 200402/91 X-Sequence-Number: 49680 Kevin Brown writes: > Slavisa Garic wrote: >> Using pg module in python I am trying to run the COPY command to populate >> the large table. I am using this to replace the INSERT which takes about >> few hours to add 70000 entries where copy takes minute and a half. > That difference in speed seems quite large. Too large. Are you batching > your INSERTs into transactions (you should be in order to get good > performance)? Do you have a ton of indexes on the table? Does it have > triggers on it or some other thing (if so then COPY may well wind up doing > the wrong thing since the triggers won't fire for the rows it > inserts)? COPY *does* fire triggers, and has done so for quite a few releases. My bet is that the issue is failing to batch individual INSERTs into transactions. On a properly-set-up machine you can't get more than one transaction commit per client per disk revolution, so the penalty for trivial transactions like single inserts is pretty steep. regards, tom lane