X-Original-To: pgsql-general-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [64.117.224.130]) by svr1.postgresql.org (Postfix) with ESMTP id 1A221D1B512 for ; Fri, 26 Sep 2003 11:58:34 +0000 (GMT) Received: from svr1.postgresql.org ([64.117.224.193]) by localhost (neptune.hub.org [64.117.224.130]) (amavisd-new, port 10024) with ESMTP id 89389-09 for ; Fri, 26 Sep 2003 08:58:26 -0300 (ADT) Received: from web1.nexusinternetsolutions.net (web1.nexusinternetsolutions.net [206.47.131.12]) by svr1.postgresql.org (Postfix) with SMTP id 8728FD1B50C for ; Fri, 26 Sep 2003 08:58:19 -0300 (ADT) Received: (qmail 96168 invoked from network); 26 Sep 2003 11:58:17 -0000 Received: from unknown (HELO ws1) (65.49.236.97) by web1.nexusinternetsolutions.net with SMTP; 26 Sep 2003 11:58:17 -0000 From: "Dave [Hawk-Systems]" To: Subject: Re: populate table with large csv file Date: Fri, 26 Sep 2003 07:58:16 -0400 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.6604 (9.0.2911.0) Importance: Normal In-Reply-To: <3F731CF0.2090800@sonalysts.com> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 X-Virus-Scanned: by amavisd-new at postgresql.org X-Spam-Status: No, hits=1.9 tagged_above=0.0 required=5.0 tests=BAYES_30, FORGED_MUA_OIMO, IN_REP_TO, MSGID_GOOD_EXCHANGE, QUOTED_EMAIL_TEXT, RCVD_IN_OSIRUSOFT_COM X-Spam-Level: * X-Archive-Number: 200309/1668 X-Sequence-Number: 49887 >> aside from parsing the csv file through a PHP interface, what isthe >easiest way >> to get that csv data importted into the postgres database. thoughts? > >Assuming the CSV file data is well formed, use psql and >the COPY command. > >In psql, create the table. Then issue command: > >copy from 'filename' using delimiters ','; perfect solution that was overlooked. Unfortunately processing the 143mb file which would result in a database size of approx 500mb takes an eternity. As luck would have it we can get away with just dropping to an exec and doing a cat/grep for any data we need... takes 2-3 seconds. the copy command is definately a keeper as I am not looking at replacing code elsewhere with a simpler model using that. Thanks Dave