Received: from localhost (unknown [200.46.204.187]) by postgresql.org (Postfix) with ESMTP id A594F2E0167 for ; Sun, 16 Dec 2007 13:27:46 -0400 (AST) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.204.187]) (amavisd-maia, port 10024) with ESMTP id 99206-02 for ; Sun, 16 Dec 2007 13:27:41 -0400 (AST) Received: from sss.pgh.pa.us (sss.pgh.pa.us [66.207.139.130]) by postgresql.org (Postfix) with ESMTP id B0CEB2E00F2 for ; Sun, 16 Dec 2007 13:27:41 -0400 (AST) Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1]) by sss.pgh.pa.us (8.14.1/8.14.1) with ESMTP id lBGHRaBa026962; Sun, 16 Dec 2007 12:27:36 -0500 (EST) To: Hannu Krosing cc: Josh Berkus , Neil Conway , pgsql-hackers@postgresql.org, Simon Riggs Subject: Re: VLDB Features In-reply-to: <1197717031.7974.23.camel@hannu-laptop> References: <1197371508.4255.1327.camel@ebony.site> <1197677930.1536.18.camel@dell.linuxdev.us.dell.com> <16876.1197680636@sss.pgh.pa.us> <200712142034.57808.josh@agliodbs.com> <6430.1197699178@sss.pgh.pa.us> <1197717031.7974.23.camel@hannu-laptop> Comments: In-reply-to Hannu Krosing message dated "Sat, 15 Dec 2007 13:10:30 +0200" Date: Sun, 16 Dec 2007 12:27:36 -0500 Message-ID: <26961.1197826056@sss.pgh.pa.us> From: Tom Lane X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200712/614 X-Sequence-Number: 111903 Hannu Krosing writes: > But can't we _define_ such a subset, where we can do a transactionless > load ? Sure ... but you'll find that it's not large enough to be useful. Once you remove all the interesting consistency checks such as unique indexes and foreign keys, the COPY will tend to go through just fine, and then you're still stuck trying to weed out bad data without very good tools for it. The only errors we could really separate out without subtransaction fencing are extremely trivial ones like too many or too few fields on a line ... which can be caught with a sed script. regards, tom lane