From: Iain <iain@mst.co.jp>
To: Christian Fowler <spider@viovio.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: pgsql-admin list <pgsql-admin@postgresql.org>
Subject: Re: evil characters #bfef cause dump failure
Date: Tue, 16 Nov 2004 10:17:54 +0900
Message-ID: <007901c4cb7a$1cbddcb0$7201a8c0@mst1x5r347kymb> (raw)
References: <Pine.LNX.4.61.0411150335350.10091@leda.steelsun.com>
<9996.1100547889@sss.pgh.pa.us>
<Pine.LNX.4.61.0411151527550.12980@leda.steelsun.com>
<10478.1100550946@sss.pgh.pa.us>
<Pine.LNX.4.61.0411151539540.12980@leda.steelsun.com>
<10656.1100552422@sss.pgh.pa.us>
It seems that this kind of thing pops up from time to time. I don't have v8
available right now to check, but is SQL_ASCII still the default DB
encoding? I'm wondering is unicode wouldn't be a better choice these days.
regards
Iain
----- Original Message -----
From: "Tom Lane" <tgl@sss.pgh.pa.us>
To: "Christian Fowler" <spider@viovio.com>
Cc: "pgsql-admin list" <pgsql-admin@postgresql.org>
Sent: Tuesday, November 16, 2004 6:00 AM
Subject: Re: [ADMIN] evil characters #bfef cause dump failure
> Christian Fowler <spider@viovio.com> writes:
>> server_encoding
>> -----------------
>> SQL_ASCII
>
>> whoa! yikes, I bet this has a lot to do with it? I really wanted to keep
>> everything UNICODE end-to-end. I must have forgotten --encoding on my
>> initdb? Anything I can do at this point?
>
> Hmm ... the safe way would be dump-n-reload but that's not working for
> you. What you can try is to alter the pg_database.encoding value for
> that database, then start fresh backends (any existing ones won't notice
> the change). Worst case if that doesn't make life good is to change it
> back.
>
> The real problem is that you've got invalid unicode data in the database
> (I'm not an expert, but I think that #bf is a 1-byte UTF8 sequence and
> then #ef starts a 3-byte sequence, so if this comes within 2 characters
> of end-of-line that would explain your dump problem). You had better
> fix the data first before trying to lock down the encoding. Once you
> change the encoding, backend internal operations will start spitting up
> on any stored bad data, whereas right now it's just passing it through
> unchanged.
>
> The safest way might be a dump-n-reload in any case, since reloading
> into a fresh UNICODE database will catch bad data. If you try manual
> repairs you're likely to miss some places :-(
>
> regards, tom lane
>
> ---------------------------(end of broadcast)---------------------------
> TIP 5: Have you checked our extensive FAQ?
>
> http://www.postgresql.org/docs/faqs/FAQ.html
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-admin@postgresql.org
Cc: iain@mst.co.jp, spider@viovio.com, tgl@sss.pgh.pa.us
Subject: Re: evil characters #bfef cause dump failure
In-Reply-To: <007901c4cb7a$1cbddcb0$7201a8c0@mst1x5r347kymb>
References: <Pine.LNX.4.61.0411150335350.10091@leda.steelsun.com> <9996.1100547889@sss.pgh.pa.us> <Pine.LNX.4.61.0411151527550.12980@leda.steelsun.com> <10478.1100550946@sss.pgh.pa.us> <Pine.LNX.4.61.0411151539540.12980@leda.steelsun.com> <10656.1100552422@sss.pgh.pa.us> <007901c4cb7a$1cbddcb0$7201a8c0@mst1x5r347kymb>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox