pg.ddx.io  pgsql-admin@postgresql.org mailing list archive  
help / color / mirror / Atom feed
Converting from LATIN1 to UNICODE encoding?
3+ messages / 3 participants
[nested] [flat]

* Converting from LATIN1 to UNICODE encoding?
@ 2005-09-21 08:16 Scott Eade <seade@backstagetech.com.au>
  2005-09-21 14:15 ` Re: Converting from LATIN1 to UNICODE encoding? Peter Eisentraut <peter_e@gmx.net>
  2005-09-21 14:40 ` Re: Converting from LATIN1 to UNICODE encoding? Alvaro Herrera <alvherre@alvh.no-ip.org>
  0 siblings, 2 replies; 3+ messages in thread

From: Scott Eade @ 2005-09-21 08:16 UTC (permalink / raw)
  To: pgsql-admin

I have a database that was created using the LATIN1 encoding.

Is it necessary for me to convert the database to some other encoding
(e.g. UNICODE) before I can store non-LATIN1 characters or does
PostgreSQL catch these and encode them somehow?

I have actually been attempting to convert a database by doing a pg_dump
(from the LATIN1 database) followed by a pg_restore (to one created with
the UNICODE encoding).  Seemed to work with a sparsely populated 8.0.3
database, but I am running into all sorts of problems with 7.3.10 (e.g.
corrupted database followed by corrupted pg_clog).

What about JDBC, how does it know the encoding of the data I throw at
it?  I guess I should ask this last question on the JDBC list.

Thanks in advance for any comments.

Scott



^ permalink  raw  reply  [nested|flat] 3+ messages in thread

* Re: Converting from LATIN1 to UNICODE encoding?
  2005-09-21 08:16 Converting from LATIN1 to UNICODE encoding? Scott Eade <seade@backstagetech.com.au>
@ 2005-09-21 14:15 ` Peter Eisentraut <peter_e@gmx.net>
  1 sibling, 0 replies; 3+ messages in thread

From: Peter Eisentraut @ 2005-09-21 14:15 UTC (permalink / raw)
  To: Scott Eade <seade@backstagetech.com.au>; +Cc: pgsql-admin

Am Mittwoch, 21. September 2005 10:16 schrieb Scott Eade:
> Is it necessary for me to convert the database to some other encoding
> (e.g. UNICODE) before I can store non-LATIN1 characters or does
> PostgreSQL catch these and encode them somehow?

Yes and no.

> I have actually been attempting to convert a database by doing a pg_dump
> (from the LATIN1 database) followed by a pg_restore (to one created with
> the UNICODE encoding).

That is the right method.

> Seemed to work with a sparsely populated 8.0.3 
> database, but I am running into all sorts of problems with 7.3.10 (e.g.
> corrupted database followed by corrupted pg_clog).

That is the symptom of a different, much bigger problem.  Encoding problems 
certainly never corrupt the clog.

> What about JDBC, how does it know the encoding of the data I throw at
> it?

That depends on where your throws come from.  As you surely know, Java mostly 
uses Unicode internally, so the data sent by the JDBC driver to the database 
is in Unicode, but as to what encoding the data that you input into your Java 
program has, you need to sort that out with the Java library functions that 
you use to read that data.



^ permalink  raw  reply  [nested|flat] 3+ messages in thread

* Re: Converting from LATIN1 to UNICODE encoding?
  2005-09-21 08:16 Converting from LATIN1 to UNICODE encoding? Scott Eade <seade@backstagetech.com.au>
@ 2005-09-21 14:40 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
  1 sibling, 0 replies; 3+ messages in thread

From: Alvaro Herrera @ 2005-09-21 14:40 UTC (permalink / raw)
  To: Scott Eade <seade@backstagetech.com.au>; +Cc: pgsql-admin

On Wed, Sep 21, 2005 at 06:16:31PM +1000, Scott Eade wrote:
> I have a database that was created using the LATIN1 encoding.
> 
> Is it necessary for me to convert the database to some other encoding
> (e.g. UNICODE) before I can store non-LATIN1 characters or does
> PostgreSQL catch these and encode them somehow?
> 
> I have actually been attempting to convert a database by doing a pg_dump
> (from the LATIN1 database) followed by a pg_restore (to one created with
> the UNICODE encoding).  Seemed to work with a sparsely populated 8.0.3
> database, but I am running into all sorts of problems with 7.3.10 (e.g.
> corrupted database followed by corrupted pg_clog).

Hmm.  Do tell, what database did you use as template?  Was it correctly
frozen?  If not, has it suffered from Xid wraparound?

Try dumping the template database.  If it succeeds, then it's OK.  If it
complains in the same way, with a missing pg_clog file, there lies the
problem.

-- 
Alvaro Herrera                                http://www.planetpostgresql.org
"En las profundidades de nuestro inconsciente hay una obsesiva necesidad
de un universo lógico y coherente. Pero el universo real se halla siempre
un paso más allá de la lógica" (Irulan)



^ permalink  raw  reply  [nested|flat] 3+ messages in thread


end of thread, other threads:[~2005-09-21 14:40 UTC | newest]

Thread overview: 3+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2005-09-21 08:16 Converting from LATIN1 to UNICODE encoding? Scott Eade <seade@backstagetech.com.au>
2005-09-21 14:15 ` Peter Eisentraut <peter_e@gmx.net>
2005-09-21 14:40 ` Alvaro Herrera <alvherre@alvh.no-ip.org>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox