pg.ddx.io  pgsql-admin@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Christian Fowler <spider@viovio.com>
Cc: pgsql-admin list <pgsql-admin@postgresql.org>
Subject: Re: evil characters #bfef cause dump failure
Date: Mon, 15 Nov 2004 16:00:22 -0500
Message-ID: <10656.1100552422@sss.pgh.pa.us> (raw)
In-Reply-To: <Pine.LNX.4.61.0411151539540.12980@leda.steelsun.com>
References: <Pine.LNX.4.61.0411150335350.10091@leda.steelsun.com>
	<9996.1100547889@sss.pgh.pa.us>
	<Pine.LNX.4.61.0411151527550.12980@leda.steelsun.com>
	<10478.1100550946@sss.pgh.pa.us>
	<Pine.LNX.4.61.0411151539540.12980@leda.steelsun.com>

Christian Fowler <spider@viovio.com> writes:
>   server_encoding
> -----------------
>   SQL_ASCII

> whoa! yikes, I bet this has a lot to do with it? I really wanted to keep 
> everything UNICODE end-to-end.  I must have forgotten --encoding on my 
> initdb? Anything I can do at this point?

Hmm ... the safe way would be dump-n-reload but that's not working for
you.  What you can try is to alter the pg_database.encoding value for
that database, then start fresh backends (any existing ones won't notice
the change).  Worst case if that doesn't make life good is to change it
back.

The real problem is that you've got invalid unicode data in the database
(I'm not an expert, but I think that #bf is a 1-byte UTF8 sequence and
then #ef starts a 3-byte sequence, so if this comes within 2 characters
of end-of-line that would explain your dump problem).  You had better
fix the data first before trying to lock down the encoding.  Once you
change the encoding, backend internal operations will start spitting up
on any stored bad data, whereas right now it's just passing it through
unchanged.

The safest way might be a dump-n-reload in any case, since reloading
into a fresh UNICODE database will catch bad data.  If you try manual
repairs you're likely to miss some places :-(

			regards, tom lane



view thread (16+ messages)  latest in thread

Message-ID: <10656.1100552422@sss.pgh.pa.us>
Permalink:  ../10656.1100552422@sss.pgh.pa.us/
Also on:    postgresql.org/message-id/10656.1100552422@sss.pgh.pa.us

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-admin@postgresql.org
  Cc: tgl@sss.pgh.pa.us, spider@viovio.com
  Subject: Re: evil characters #bfef cause dump failure
  In-Reply-To: <10656.1100552422@sss.pgh.pa.us>
  References: <Pine.LNX.4.61.0411150335350.10091@leda.steelsun.com> <9996.1100547889@sss.pgh.pa.us> <Pine.LNX.4.61.0411151527550.12980@leda.steelsun.com> <10478.1100550946@sss.pgh.pa.us> <Pine.LNX.4.61.0411151539540.12980@leda.steelsun.com> <10656.1100552422@sss.pgh.pa.us>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox