X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org Received: from localhost (av.hub.org [200.46.204.144]) by postgresql.org (Postfix) with ESMTP id EE3B49DCC3B for ; Wed, 7 Dec 2005 00:38:45 -0400 (AST) Received: from postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 31687-02-2 for ; Wed, 7 Dec 2005 00:38:44 -0400 (AST) X-Greylist: from auto-whitelisted by SQLgrey- Received: from candle.pha.pa.us (candle.pha.pa.us [64.139.89.126]) by postgresql.org (Postfix) with ESMTP id 4363C9DCB84 for ; Wed, 7 Dec 2005 00:38:05 -0400 (AST) Received: (from pgman@localhost) by candle.pha.pa.us (8.11.6/8.11.6) id jB74buw20492; Tue, 6 Dec 2005 23:37:56 -0500 (EST) From: Bruce Momjian Message-Id: <200512070437.jB74buw20492@candle.pha.pa.us> Subject: Re: Upcoming PG re-releases In-Reply-To: To: Gavin Sherry Date: Tue, 6 Dec 2005 23:37:56 -0500 (EST) CC: Peter Eisentraut , pgsql-hackers@postgresql.org X-Mailer: ELM [version 2.4ME+ PL121 (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, score=0.006 required=5 tests=[AWL=0.006] X-Spam-Score: 0.006 X-Spam-Level: X-Archive-Number: 200512/367 X-Sequence-Number: 77215 Exactly what does vim do that iconv does not? Fuzzy encoding sounds scary to me. --------------------------------------------------------------------------- Gavin Sherry wrote: > Hi, > > On Tue, 6 Dec 2005, Bruce Momjian wrote: > > > > > Nice, updated. > > > > --------------------------------------------------------------------------- > > > > I think my suggestion from the other day is useful also. > > --- > > Omar Kilani and I have spent a few hours looking at the problem. For > situations where there is a lot of invalid encoding, manual fixing is just > not viable. The vim project has a kind of fuzzy encoding conversion which > accounts for a lot of the non-UTF8 sequences in UTF8 data. You can use vim > to modify your text dump as follows: > > vim -c ":wq! ++enc=utf8 fixed.dump" original.dump > > --- > > I think this is a viable option for people with a non-trivial amount of > data and don't see manual fixing or potentially losing data as a viable > option. > > Thanks, > > Gavin > > ---------------------------(end of broadcast)--------------------------- > TIP 5: don't forget to increase your free space map settings > -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073