From: Lou Picciano <loupicciano@comcast.net>
To: pgsql-admin <pgsql-admin@postgresql.org>
Subject: How To: LARGE html text or csv file COPY FROM?
Date: Wed, 15 Sep 2010 13:03:16 +0000 (UTC)
Message-ID: <594168409.400001.1284555796001.JavaMail.root@sz0093a.westchester.pa.mail.comcast.net> (raw)
In-Reply-To: <203410795.399786.1284555612401.JavaMail.root@sz0093a.westchester.pa.mail.comcast.net>
Hello Friends,
We're trying something for the first time: A COPY into a database, from a TEXT (or CSV) file containing one really, really, big field of html.
The field happens to be content of complete webpages, which we then need to later analyze, slice, dice, etc. - so it's verbatim html, with all the carriage returns, spaces, linefeeds(?) and double quotes included!
Problem is: With the very first record, the COPY commands hiccups with: missing data from column error.
in CSV mode, it's 'extra data after last expected column' (yes, using different input files for test).
Both errors above make sense to me; COPY is running into either a cr or a tab character in each case.
Q: Is there way to handle this directly, as a PG import?
Meanwhile, we're off into using grep/gawk to remove all carriage returns in the field?
TIA for any help, inspiration, recipes (or time in the stocks). Lou
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-admin@postgresql.org
Cc: loupicciano@comcast.net
Subject: Re: How To: LARGE html text or csv file COPY FROM?
In-Reply-To: <594168409.400001.1284555796001.JavaMail.root@sz0093a.westchester.pa.mail.comcast.net>
References: <203410795.399786.1284555612401.JavaMail.root@sz0093a.westchester.pa.mail.comcast.net> <594168409.400001.1284555796001.JavaMail.root@sz0093a.westchester.pa.mail.comcast.net>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox