Received: from localhost (maia-3.hub.org [200.46.204.184]) by postgresql.org (Postfix) with ESMTP id 9E1AE9F93A2 for ; Sat, 23 Dec 2006 23:09:02 -0400 (AST) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.204.184]) (amavisd-new, port 10024) with ESMTP id 11210-02 for ; Sat, 23 Dec 2006 23:08:52 -0400 (AST) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.4 Received: from nf-out-0910.google.com (nf-out-0910.google.com [64.233.182.184]) by postgresql.org (Postfix) with ESMTP id E5AAE9F939F for ; Sat, 23 Dec 2006 23:08:57 -0400 (AST) Received: by nf-out-0910.google.com with SMTP id x4so4134606nfb for ; Sat, 23 Dec 2006 19:08:56 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=hfGgBasEjv05oSZJ/D1SSJhBVsSuxQsbXineldAwEBC+AzQY8lIef6d6aDFHh/iUaU8jFLOwJQKhKdgYkrBhjxNMKVFSvl2TWo1LJqzqti4aic3LCJo7B8T8yTcSg8Y0QSFC1jxTm1bTdlz4bCdr96cWAXWxd2vfEKFL6I26Ffg= Received: by 10.78.185.7 with SMTP id i7mr2137698huf.1166929736131; Sat, 23 Dec 2006 19:08:56 -0800 (PST) Received: by 10.78.171.4 with HTTP; Sat, 23 Dec 2006 19:08:56 -0800 (PST) Message-ID: <523e84030612231908i5c272b5atabe95643d482e157@mail.gmail.com> Date: Sat, 23 Dec 2006 19:08:56 -0800 From: Mason To: pgsql-patches@postgresql.org Subject: Fwd: I would like to alter the COPY command In-Reply-To: <523e84030612231908s32ef5145s4e18582b4a579d07@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <523e84030612231908s32ef5145s4e18582b4a579d07@mail.gmail.com> X-Virus-Scanned: Maia Mailguard 1.0.1 X-Spam-Status: No, hits=0 tagged_above=0 required=5 tests=BAYES_50, SPF_PASS X-Spam-Level: X-Archive-Number: 200612/193 X-Sequence-Number: 21869 What I have is data with two different characters for "start quote" and "end quote". In my case it's '[' and ']', but it could be anything from "smart quotes", to parentheses, to brackets, braces, ^/$ in regexps, etc. I think this isn't too unreasonable a feature to have to make copy more functional when importing data that is difficult to transform properly beforehand (in my case is about half a terabyte of log files, which takes hours and hours, just to cat, let alone reparse and dump into COPY). Now, in my case I can just say "cat file | tr '[]' '""' | psql -f import.sql", but then I lose the ability for psql to do anything smart like using mmap (I'm making assumptions that it does anything smart like that, but even if it doesn't now, it could some day). So, I'm a passable c/c++ programmer, when I have to be, so theoretically I can do the work myself, but I have never touched postgres before, so I don't know where to begin. Any ideas how to add this?