From: Tom Lane <tgl@sss.pgh.pa.us>
To: Noah Misch <noah@leadboat.com>
cc: Andrew Dunstan <andrew.dunstan@2ndquadrant.com>,
        PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Subject: Re: Augment every test postgresql.conf
In-reply-to: <20190512015615.GD1124997@rfd.leadboat.com>
References: <20181229021950.GA3302966@rfd.leadboat.com>
 <20181230024014.GA181668@gust.leadboat.com> <16623.1546141591@sss.pgh.pa.us>
 <20181230055346.GB156585@gust.leadboat.com>
 <f8938ff6-4dc8-2beb-636b-1d524db78e2f@2ndQuadrant.com>
 <20190407064156.GC1914190@rfd.leadboat.com>
 <CAA8=A786VMBGfUbBswhP6oP_6YpNy+2KrcP1rB-Kj1yX05_H5Q@mail.gmail.com>
 <20190512015615.GD1124997@rfd.leadboat.com>
Comments: In-reply-to Noah Misch <noah@leadboat.com>
	message dated "Sat, 11 May 2019 18:56:15 -0700"
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-ID: <25382.1557629039.1@sss.pgh.pa.us>
Content-Transfer-Encoding: quoted-printable
Date: Sat, 11 May 2019 22:43:59 -0400
Message-ID: <25383.1557629039@sss.pgh.pa.us>
Precedence: bulk

Noah Misch <noah@leadboat.com> writes:
> Pushed.  This broke 010_dump_connstr.pl on bowerbird, introducing 'inval=
id
> byte sequence for encoding "UTF8"' errors.  That's because log_connectio=
ns
> renders this 010_dump_connstr.pl solution insufficient:

Ugh.

> 4. If GetMessageEncoding()=3D=3DPG_SQL_ASCII, make pgwin32_message_to_UT=
F16()
>    return NULL.  The caller will always send untranslated bytes to write=
() or
>    ReportEventA().  This seems consistent with the SQL_ASCII concept and=
 with
>    pg_do_encoding_conversion()'s interpretation of SQL_ASCII.

> 5. When including a datname or rolname value in a message, hex-escape
>    non-ASCII bytes.  They are byte sequences, not text of known encoding=
.
>    This preserves the most information, but it's overkill and ugly in th=
e
>    probably-common case of one encoding across all databases of a cluste=
r.

> I'm inclined to do (1) in back branches and (4) in HEAD only.  (If start=
ing
> fresh today, I would store the encoding of each rolname and dbname or ju=
st use
> UTF8 for those particular fields.)  Other preferences?

I agree that (4) is a fairly reasonable thing to do, and wouldn't mind
back-patching that.  Taking a wider view, this seems closely related
to something I've been thinking about in connection with the recent
pg_stat_activity contretemps: that mechanism is also shoving strings
across database boundaries without a lot of worry about encodings.
Maybe we should try to develop a common solution.

One difference from the datname/rolname situation is that for
pg_stat_activity we can know the source encoding --- we aren't storing
it now, but we easily could.  If we're thinking of a future solution
only, adding a "name encoding" field to relevant shared catalogs makes
sense perhaps.  Alternatively, requiring names in shared catalogs to be
UTF8 might be a reasonable answer too.

In all these cases, throwing an error when we can't translate a character
into the destination encoding is not very pleasant.  For pg_stat_activity,
I was imagining that translating such characters to '?' might be the best
answer.  I don't know if we can get away with that for the datname/rolname
case --- at the very least, it opens problems with apparent duplication of
names that should be unique.  I don't much like your hex-encoding answer,
though; that has its own uniqueness-violation hazards, plus it's ugly.

I don't have a strong feeling about what's best.

			regards, tom lane