X-Original-To: pgsql-admin-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id A00B73A5622 for ; Tue, 16 Nov 2004 01:17:11 +0000 (GMT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 24056-05 for ; Tue, 16 Nov 2004 01:17:06 +0000 (GMT) Received: from millenium.mst.co.jp (unknown [210.230.185.241]) by svr1.postgresql.org (Postfix) with ESMTP id 165E63A563C for ; Tue, 16 Nov 2004 01:17:07 +0000 (GMT) Received: from mst1x5r347kymb (lc12114 [192.168.1.114]) by millenium.mst.co.jp (8.11.6p2/3.7W) with SMTP id iAG1GuZ19223; Tue, 16 Nov 2004 10:16:56 +0900 Message-ID: <007901c4cb7a$1cbddcb0$7201a8c0@mst1x5r347kymb> From: "Iain" To: "Christian Fowler" , "Tom Lane" Cc: "pgsql-admin list" References: <9996.1100547889@sss.pgh.pa.us> <10478.1100550946@sss.pgh.pa.us> <10656.1100552422@sss.pgh.pa.us> Subject: Re: evil characters #bfef cause dump failure Date: Tue, 16 Nov 2004 10:17:54 +0900 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-2022-jp"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.2180 X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2900.2180 X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.0 tagged_above=0.0 required=5.0 tests= X-Spam-Level: X-Archive-Number: 200411/124 X-Sequence-Number: 15508 It seems that this kind of thing pops up from time to time. I don't have v8 available right now to check, but is SQL_ASCII still the default DB encoding? I'm wondering is unicode wouldn't be a better choice these days. regards Iain ----- Original Message ----- From: "Tom Lane" To: "Christian Fowler" Cc: "pgsql-admin list" Sent: Tuesday, November 16, 2004 6:00 AM Subject: Re: [ADMIN] evil characters #bfef cause dump failure > Christian Fowler writes: >> server_encoding >> ----------------- >> SQL_ASCII > >> whoa! yikes, I bet this has a lot to do with it? I really wanted to keep >> everything UNICODE end-to-end. I must have forgotten --encoding on my >> initdb? Anything I can do at this point? > > Hmm ... the safe way would be dump-n-reload but that's not working for > you. What you can try is to alter the pg_database.encoding value for > that database, then start fresh backends (any existing ones won't notice > the change). Worst case if that doesn't make life good is to change it > back. > > The real problem is that you've got invalid unicode data in the database > (I'm not an expert, but I think that #bf is a 1-byte UTF8 sequence and > then #ef starts a 3-byte sequence, so if this comes within 2 characters > of end-of-line that would explain your dump problem). You had better > fix the data first before trying to lock down the encoding. Once you > change the encoding, backend internal operations will start spitting up > on any stored bad data, whereas right now it's just passing it through > unchanged. > > The safest way might be a dump-n-reload in any case, since reloading > into a fresh UNICODE database will catch bad data. If you try manual > repairs you're likely to miss some places :-( > > regards, tom lane > > ---------------------------(end of broadcast)--------------------------- > TIP 5: Have you checked our extensive FAQ? > > http://www.postgresql.org/docs/faqs/FAQ.html