From: Greg Williamson <gwilliamson39@yahoo.com>
To: Wells Oliver <wellsoliver@gmail.com>
To: pgsql-admin@postgresql.org <pgsql-admin@postgresql.org>
Subject: Re: Trouble w/ COPY command
Date: Sat, 14 Apr 2012 16:17:07 -0700 (PDT)
Message-ID: <1334445427.92228.YahooMailNeo@web46106.mail.sp1.yahoo.com> (raw)
In-Reply-To: <CAOC+FBUmbk9RhTZTFTULBhKMGQcXfhVEqjEeqJ7PChgk3kdLRQ@mail.gmail.com>
References: <CAOC+FBUmbk9RhTZTFTULBhKMGQcXfhVEqjEeqJ7PChgk3kdLRQ@mail.gmail.com>
Wells Oliver asked:
>
>I have CSV files blowing up because they have double quotes around numeric values: " 1.34" yields ERROR: invalid input syntax for type numeric: " 1.34"
>>
>And occasionally these are empty, and you see invalid syntax for type numeric: " "
>
>Are there flags I can pass to COPY to avoid this? There are properly quoted string values elsewhere in the final, so I'm reticent to strip out double quote characters or anything.
>
There's no easy command line method that I know of. Perhaps if you can detect patterns some sed / awk might help.
I'd probably load the CSV file into a temporary work table with the columns that are giving you issues defined as type "text." Then use SQL to remove the offending quotation marks from the afflicted columns. Create new columns of type numeric in the table, and copy the values from the text variant to the numeric ones with the proper cast one at a time; if it blows up you go back to resolving issues. Then copy the table using the numeric columns to the desired destination.
Hopefully others might have better ideas ... best o' luck!
Greg Williamson
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-admin@postgresql.org
Cc: gwilliamson39@yahoo.com, wellsoliver@gmail.com
Subject: Re: Trouble w/ COPY command
In-Reply-To: <1334445427.92228.YahooMailNeo@web46106.mail.sp1.yahoo.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox