From: Bruce Momjian <pgman@candle.pha.pa.us>
Message-Id: <200512082244.jB8MiYT02161@candle.pha.pa.us>
Subject: Re: Upcoming PG re-releases
In-Reply-To: <Pine.LNX.4.58.0512090932030.12041@linuxworld.com.au>
To: Gavin Sherry <swm@linuxworld.com.au>
Date: Thu, 8 Dec 2005 17:44:34 -0500 (EST)
CC: Peter Eisentraut <peter_e@gmx.net>, pgsql-hackers@postgresql.org
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=US-ASCII

Gavin Sherry wrote:
> On Tue, 6 Dec 2005, Bruce Momjian wrote:
> 
> >
> > Exactly what does vim do that iconv does not?  Fuzzy encoding sounds
> > scary to me.
> >
> 
> Right. It actually makes assumptions about the source encoding. People who
> care about their data need, unfortunately, to spend a bit of time on this
> problem. I've been discussing the same issue on the slony1 mailing list,
> because the issue can affect people's ability upgrade using slony1.
> 
> http://gborg.postgresql.org/pipermail/slony1-general/2005-December/003430.html
> 
> It would be good if had the script I suggest in the email:
> 
> 	A script which identifies non-utf-8 characters and provides some
> 	context, line numbers, etc, will greatly speed up the process of
> 	remedying the situation.

I think the best we can do is the "iconv -c with the diff" idea, which
is already in the release notes.  I suppose we could merge the iconv and
diff into a single command, but I don't see a portable way to output the
iconv output to stdout., /dev/stdin not being portable.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073