Received: from localhost (unknown [200.46.204.191])
	by postgresql.org (Postfix) with ESMTP id CB21E2E00C4
	for <pgsql-hackers-postgresql.org@postgresql.org>;
	Fri, 14 Dec 2007 15:26:34 -0400 (AST)
Received: from postgresql.org ([200.46.204.71])
	by localhost (mx1.hub.org [200.46.204.191]) (amavisd-maia, port 10024)
	with ESMTP id 79482-06 for
	<pgsql-hackers-postgresql.org@postgresql.org>;
	Fri, 14 Dec 2007 15:26:31 -0400 (AST)
Received: from mailbox.samurai.com (mailbox.samurai.com [205.207.28.82])
	by postgresql.org (Postfix) with ESMTP id 686232E0070
	for <pgsql-hackers@postgresql.org>;
	Fri, 14 Dec 2007 15:26:31 -0400 (AST)
Received: from localhost (mailbox.samurai.com [205.207.28.82])
	by mailbox.samurai.com (Postfix) with ESMTP id C870661D52;
	Fri, 14 Dec 2007 14:26:30 -0500 (EST)
X-Virus-Scanned: by amavisd-new at mailbox.samurai.com
Received: from mailbox.samurai.com ([205.207.28.82])
	by localhost (mailbox.samurai.com [205.207.28.82]) (amavisd-new,
	port 10024)
	with LMTP id P+B4nV0yY1QV; Fri, 14 Dec 2007 14:26:27 -0500 (EST)
Received: from [10.10.30.141] (204.11.227.33.static.etheric.net
	[204.11.227.33])
	(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
	(No client certificate requested)
	by mailbox.samurai.com (Postfix) with ESMTP id 35CB261D51;
	Fri, 14 Dec 2007 14:26:27 -0500 (EST)
Subject: Re: VLDB Features
From: Neil Conway <neilc@samurai.com>
To: Hannu Krosing <hannu@skype.net>
Cc: Josh Berkus <josh@agliodbs.com>, pgsql-hackers@postgresql.org,
	Simon Riggs <simon@2ndquadrant.com>
In-Reply-To: <1197636530.7974.16.camel@hannu-laptop>
References: <1197371508.4255.1327.camel@ebony.site>
	<200712111053.11178.josh@agliodbs.com>
	<1197416461.29432.10.camel@dell.linuxdev.us.dell.com>
	<1197636530.7974.16.camel@hannu-laptop>
Content-Type: text/plain
Date: Fri, 14 Dec 2007 11:27:03 -0800
Message-Id: <1197660423.1536.8.camel@dell.linuxdev.us.dell.com>
Mime-Version: 1.0
X-Mailer: Evolution 2.12.1 
Content-Transfer-Encoding: 7bit
X-Virus-Scanned: Maia Mailguard 1.0.1
X-Archive-Number: 200712/573
X-Sequence-Number: 111862

On Fri, 2007-12-14 at 14:48 +0200, Hannu Krosing wrote:
> How did you do it ?
> 
> Did you enchance COPY command or was it something completely new ?

By modifying COPY: COPY IGNORE ERRORS or some such would instruct COPY
to drop (and log) rows that contain malformed data. That is, rows with
too many or too few columns, rows that result in constraint violations,
and rows containing columns where the data type's input function raises
an error. The last case is the only thing that would be a bit tricky to
implement, I think: you could use PG_TRY() around the InputFunctionCall,
but I guess you'd need a subtransaction to ensure that you reset your
state correctly after catching an error.

-Neil