Date: Fri, 9 Dec 2005 19:28:12 +0100
From: Martijn van Oosterhout <kleptog@svana.org>
To: Bruce Momjian <pgman@candle.pha.pa.us>
Cc: Gregory Maxwell <gmaxwell@gmail.com>,
 Gavin Sherry <swm@linuxworld.com.au>,
	Peter Eisentraut <peter_e@gmx.net>, pgsql-hackers@postgresql.org
Subject: Re: Upcoming PG re-releases
Message-ID: <20051209182811.GH20352@svana.org>
Reply-To: Martijn van Oosterhout <kleptog@svana.org>
References: <20051209164409.GG20352@svana.org>
	<200512091738.jB9HcLI18465@candle.pha.pa.us>
Mime-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha1;
	protocol="application/pgp-signature"; boundary="/GPgYEyhnw15BExa"
Content-Disposition: inline
In-Reply-To: <200512091738.jB9HcLI18465@candle.pha.pa.us>
User-Agent: Mutt/1.3.28i


--/GPgYEyhnw15BExa
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Fri, Dec 09, 2005 at 12:38:21PM -0500, Bruce Momjian wrote:
> > This means someone who is planning on upgrading to 8.1 in two months
> > can use this function now to weed out the bad data before the upgrade
> > even starts.
>=20
> Oh, so you back-load it into the old database.  Interesting.  I assume
> to be useful you would have to write something that checked every column
> values in every table and database.

Umm, yeah. I was thinking about how to do that. pl/pgsql is not the
best language to do that in. In any case I found a bug in the version I
posted and also added a function that does:

test=3D# select * from db_utf8_verify();
 tab  | fld | location=20
------+-----+----------
 tbl1 | foo | (12,3)
(1 row)

It gives the table, field and ctid of any values that failed. It skips
pg_catalog. It's also *really* slow for long strings. Just executing it
on the pg_rewrite in the default installation takes forever. If someone
really wanted this for a large database maybe they should recode it in
C.

http://svana.org/kleptog/pgsql/utf8_verify.sql

Have a nice day,
--=20
Martijn van Oosterhout   <kleptog@svana.org>   http://svana.org/kleptog/
> Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a
> tool for doing 5% of the work and then sitting around waiting for someone
> else to do the other 95% so you can sue them.

--/GPgYEyhnw15BExa
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQFDmcy7IB7bNG8LQkwRAlB+AJwL1Aa+cPEblMipOsf8+fT9DsRvRwCbBcQ4
LXQJV6a5dx+GnxY6cv5LIjY=
=kybc
-----END PGP SIGNATURE-----

--/GPgYEyhnw15BExa--