X-Original-To: pgsql-admin-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id A77C33A4771 for ; Mon, 15 Nov 2004 17:46:30 +0000 (GMT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 88181-07 for ; Mon, 15 Nov 2004 17:46:23 +0000 (GMT) Received: from leda.steelsun.com (dsl093-240-204.ral1.dsl.speakeasy.net [66.93.240.204]) by svr1.postgresql.org (Postfix) with ESMTP id 400693A4779 for ; Mon, 15 Nov 2004 17:46:22 +0000 (GMT) Received: from leda.steelsun.com (localhost [127.0.0.1]) by leda.steelsun.com (8.12.11/8.12.8) with ESMTP id iAFHkKCf012213 for ; Mon, 15 Nov 2004 12:46:20 -0500 Received: from localhost (cfowler@localhost) by leda.steelsun.com (8.12.11/8.12.11/Submit) with ESMTP id iAFHkJGD012210 for ; Mon, 15 Nov 2004 12:46:20 -0500 X-Authentication-Warning: leda.steelsun.com: cfowler owned process doing -bs Date: Mon, 15 Nov 2004 12:46:19 -0500 (EST) From: Christian Fowler X-X-Sender: cfowler@leda.steelsun.com To: pgsql-admin list Subject: evil characters #bfef cause dump failure Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.0 tagged_above=0.0 required=5.0 tests= X-Spam-Level: X-Archive-Number: 200411/117 X-Sequence-Number: 15501 I have been trying to track down the source of why my 7.4.5 database won't reimport it's own dump ( http://archives.postgresql.org/pgsql-admin/2004-10/msg00213.php ) After much wrestling, it appears the hex byte sequence #bfef in a VARCHAR column causes a truncated COPY line to be written (and thus the *entire* COPY block fails). Exporting as inserts did not fix the problem either. Any thoughts on why this might be so or how it can be avoided? Evil thought of the day is if someone were to go around and paste this multi-byte character in various websites' html forms it could cause a lot of trouble. Also, the behavior of the restore / psql import to complete the COPY fields from the *following* line seems not good. It would be nice if the missing columns could just be written as NULL's. 6 bad rows makes a 6 gig dump worthless. Or perhaps an option to import each copy row in it's own transaction so 5+ million copied rows don't fail for 6 bogus ones. Perhaps a --this_is_an_emergency_so_please_do_everything_you_can_to_restore_as_much_as_possible option. If any of the core dev's want some small debug dumps I created, I'd be happy to pass them on. [ \ / [ >X< Christian Fowler | spider AT viovio.com [ / \ http://www.viovio.com | http://www.tikipro.org