Subject: Re: List of encodings

From: Igor Korot <[email-masked]>
Date: Sun, 19 Apr 2026 19:19:15 -0500

Hi, Adrian,

On Sun, Apr 19, 2026 at 4:21 PM Adrian Klaver <[email-masked]> wrote:
>
> On 4/19/26 1:27 PM, Igor Korot wrote:
> > Hi, David,
> >
> > On Sat, Apr 18, 2026 at 2:19 AM David G. Johnston
> > <[email-masked]> wrote:
> >>
> >> On Friday, April 17, 2026, Igor Korot <[email-masked]> wrote:
> >>>
> >>> Hi, ALL,
> >>> Does the list shown in
> >>> https://www.postgresql.org/docs/current/multibyte.html#MULTIBYTE-CHARSET-SUPPORTED
> >>> stored somewhere in INFORMATION_SCHEMA?
> >>
> >>
> >> This wouldn’t be under the purview of information schema.  You can find pg-specific pieces though:
> >>
> >> https://www.postgresql.org/docs/current/catalog-pg-conversion.html
> >>
> >> Note the function used to convert ids to names.
> >
> > Tried the following query:
> >
> > SELECT conname AS name, pg_encoding_to_char( conforencoding ) AS
> > encoding, condefault AS default FROM pg_conversion ORDER BY encoding;
> >
> > and got following results (for simplicity I will post only couple of rows):
> >
> >   big5_to_utf8                   | BIG5           | t
> >   big5_to_euc_tw                 | BIG5           | t
> >   big5_to_mic                    | BIG5           | t
> >   euc_cn_to_mic                  | EUC_CN         | t
> >   euc_cn_to_utf8                 | EUC_CN         | t
> >   euc_jis_2004_to_shift_jis_2004 | EUC_JIS_2004   | t
> >   euc_jis_2004_to_utf8           | EUC_JIS_2004   | t
> >   euc_jp_to_mic                  | EUC_JP         | t
> >   euc_jp_to_sjis                 | EUC_JP         | t
> >   euc_jp_to_utf8                 | EUC_JP         | t
> >   euc_kr_to_utf8                 | EUC_KR         | t
> >   euc_kr_to_mic                  | EUC_KR         | t
> >   euc_tw_to_big5                 | EUC_TW         | t
> >   euc_tw_to_utf8                 | EUC_TW         | t
> >   euc_tw_to_mic                  | EUC_TW         | t
> >
> > What I noticed is that all encodings are default, as they all have 't'
> > in the last column.
> >
> > It's a little confusing...
>
> Not if you read the docs:
>
> https://www.postgresql.org/docs/current/catalog-pg-conversion.html
>
> "The catalog pg_conversion describes encoding conversion functions. See
> CREATE CONVERSION for more information."
>
> https://www.postgresql.org/docs/current/sql-createconversion.html
>
> "Conversions that are marked DEFAULT can be used for automatic encoding
> conversion between client and server. To support that usage, two
> conversions, from encoding A to B and from encoding B to A, must be
> defined."

>From the https://www.postgresql.org/docs/current/catalog-pg-conversion.html:

[quote]

condefault bool

True if this is the default conversion
[/quote]

So, what info do I trust?

Thank you.

>
>
> >
> > Thx for the help.
> >
> >>
> >>>
> >>>
> >>> Or is it hard coded inside the PostgreSQL codebase?
> >>
> >>
> >>   Yes.  Doesn’t preclude exposing it via SQL but we don’t do so directly.
> >>
> >> David J.
> >
> >
>
>
> --
> Adrian Klaver
> [email-masked]

---

From: "David G. Johnston" <[email-masked]>
Date: Sun, 19 Apr 2026 17:41:22 -0700

> True if this is the default conversion
In theory a given pair of encodings could have more than one converter
installed.  Only one of those many could be marked default.

David J.

---

From: Adrian Klaver <[email-masked]>
Date: Sun, 19 Apr 2026 17:53:21 -0700

> So, what info do I trust?

Both.

In your setup all the installed encoding conversion functions are also 
the default for those conversions. It is possible to create/install a 
conversion function that is not the default.

---

From: Igor Korot <[email-masked]>
Date: Sun, 19 Apr 2026 19:55:37 -0500

Hi, David,

On Sun, Apr 19, 2026 at 7:42 PM David G. Johnston
<[email-masked]> wrote:
> In theory a given pair of encodings could have more than one converter installed.  Only one of those many could be marked default.

I think you meant to stay "should", right?

"Only one should be marked default."

because if two of them are - and the user did not supply any - which
one the engine will use?

Thank you.

---

From: Igor Korot <[email-masked]>
Date: Sun, 19 Apr 2026 20:13:29 -0500

Adrian,

> conversion function that is not the default.

So, let's say I chose "BIG5"".

As stated the table contains:

 big5_to_utf8                   | BIG5           | t

Since all 3 are default character sets, which one would be chosen?
(in the context of CREATE DATABASE)

Thank you.

---

From: Ron Johnson <[email-masked]>
Date: Sun, 19 Apr 2026 21:31:43 -0400

On Sun, Apr 19, 2026 at 9:13 PM Igor Korot <[email-masked]> wrote:

> [snip]

Does CREATE DATABASE *convert text*?  (I think you might be
misunderstanding the purpose of the pg_conversion table.)

Wouldn't it only *convert* text when a client is inserting text of encoding
X into a table with encoding Y?

ISTM that pg_conversion says whether PG knows how to convert from X to Y,
not the encoding scheme you defined when creating the db.

-- 
Death to <Redacted>, and butter sauce.
Don't boil me, I'm still alive.
<Redacted> lobster!

---

From: Tom Lane <[email-masked]>
Date: Sun, 19 Apr 2026 21:49:30 -0400

> misunderstanding the purpose of the pg_conversion table.)

Indeed.  I doubt that CREATE DATABASE references this catalog at all.

> X into a table with encoding Y?

In the current system structure, where conversion actually happens
is at the client interface, when sending/receiving data.  All text
that's running around inside a backend process is expected to be
in the database's encoding, and we convert if the client has
declared that it wants to work in some other encoding.  So the
pg_conversion catalog is actually consulted during connection
startup, to see if we can support the requested client_encoding
with the database encoding.

There is also the convert() function, which allows you to convert a
blob of text from one encoding to another --- but the input and output
are both declared as bytea, so that they don't have to be valid in the
current database encoding.

IIRC, the client conversion lookups will only choose "condefault"
conversions, so that a non-default conversion is only reachable via
convert().  So that feature is really pretty vestigial.

			regards, tom lane

---

From: "Peter J. Holzer" <[email-masked]>
Date: Mon, 20 Apr 2026 07:53:52 +0200

> (in the context of CREATE DATABASE)

Note that the table contains *two* encodings (conforencoding and
contoencoding) for each conversion. If you look at both it becomes
clear:

hjp=> select conname,
        pg_encoding_to_char(conforencoding) as for_enc,
        pg_encoding_to_char(contoencoding) to_enc,
        condefault
      from pg_conversion
      where conname like 'big5%';
╔════════════════╤═════════╤═══════════════╤════════════╗
║    conname     │ for_enc │    to_enc     │ condefault ║
╟────────────────┼─────────┼───────────────┼────────────╢
║ big5_to_euc_tw │ BIG5    │ EUC_TW        │ t          ║
║ big5_to_mic    │ BIG5    │ MULE_INTERNAL │ t          ║
║ big5_to_utf8   │ BIG5    │ UTF8          │ t          ║
╚════════════════╧═════════╧═══════════════╧════════════╝
(3 rows)

If you need to convert from BIG5 to UTF8, big5_to_utf8 is the default
(and indeed only) conversion. If you need to convert from BIG5 to
EUC_TW, it's big5_to_euc_tw, etc.

        hjp

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | [email-masked]         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"

---

From: Igor Korot <[email-masked]>
Date: Mon, 20 Apr 2026 19:42:18 -0700

Hi, ALL,

On Sun, Apr 19, 2026 at 8:32 PM Ron Johnson <[email-masked]> wrote:
> Does CREATE DATABASE convert text?  (I think you might be misunderstanding the purpose of the pg_conversion table.)

No it does not.

But it has an option that can be chosen and supplied to the command...

So when I write "CREATE DATABASE mydb ECODING = BIG5", what will happen?

Moreover, I'm curious - if I chose "BIG5", there are only number of
available collate/ctype pairs.
How do I choose which one to present to the user.
Because there is not one default "BIG5" - there are 3 default "BIG5"s.

Thank you.

>
> Wouldn't it only convert text when a client is inserting text of encoding X into a table with encoding Y?
>
> ISTM that pg_conversion says whether PG knows how to convert from X to Y, not the encoding scheme you defined when creating the db.
>
> --

---

From: Igor Korot <[email-masked]>
Date: Mon, 20 Apr 2026 19:47:12 -0700

My understanding is that if I have 3 "BIG5" encodings, only one can be
a default.

And if you want you can choose the other 2, but selecting "BIG5" will
make only one
to be selected by default.
That is why it is called "default" ;-)

Thank you.

---

From: Ron Johnson <[email-masked]>
Date: Mon, 20 Apr 2026 22:55:07 -0400

Igor,

pg_*conversion* is for when you *convert* text.  Why does it matter if you
never convert text?

On Mon, Apr 20, 2026 at 10:47 PM Igor Korot <[email-masked]> wrote:

> > > Does CREATE DATABASE convert text?  (I think you might be
> > > ISTM that pg_conversion says whether PG knows how to convert from X to
> Y, not the encoding scheme you defined when creating the db.
> > >
> > > --

---

From: "David G. Johnston" <[email-masked]>
Date: Mon, 20 Apr 2026 20:29:11 -0700

> My understanding is that if I have 3 "BIG5" encodings, only one can be

That would be a misunderstanding of what a conversion table is about.

David J.

---

From: Igor Korot <[email-masked]>
Date: Mon, 20 Apr 2026 21:22:00 -0700

Hi, everybody,

> That would be a misunderstanding of what a conversion table is about.

What I did:

1. Google "PostgreSQL create database"
2. Click the first link - to PostgreSQL documentation.
3. The command have many options. One of them is "Encoding".
4, Scrolled down for an explanation. The explanation had a link.
5. Clicked the link. Received a page with the list of encodings.

At this point I asked the original question
Does the list on that page stored somewhere? Or it is hardcoded inside
the sources?

That's when I started receiving a references to that table.

Did I ask the wrong question?

Thank you.

---

From: "David G. Johnston" <[email-masked]>
Date: Mon, 20 Apr 2026 21:32:06 -0700

> Did I ask the wrong question?

And the answer you got was “no, it’s not (i.e., it’s hardcoded inside), but
you can get to it indirectly”.  In this case if you involve the
pgconversion table you should ignore the conversion is default field as it
has nothing to do with the question - what encodings does the system
recognize.  You also got an answer involving generate_series.

David J.

---

From: Igor Korot <[email-masked]>
Date: Mon, 20 Apr 2026 21:58:24 -0700

David,

On Mon, Apr 20, 2026 at 9:32 PM David G. Johnston
<[email-masked]> wrote:
> And the answer you got was “no, it’s not (i.e., it’s hardcoded inside), but you can get to it indirectly”.  In this case if you involve the pgconversion table you should ignore the conversion is default field as it has nothing to do with the question - what encodings does the system recognize.  You also got an answer involving generate_series.

Understood, thx. This clears it up.

Sorry for the confusion.

And yes - I will query the pg_conversion table.

Thx once again.