From: Ronan Dunklau <ronan.dunklau@aiven.io>
To: pgsql-hackers@lists.postgresql.org,
 Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Subject: Re: Proposal: More structured logging
Date: Wed, 01 Sep 2021 10:00:28 +0200
Message-ID: <163276811.1OrvvjSa7g@aivenronan>
Organization: aiven
In-Reply-To: <e9b5a975-503d-61c7-c60e-d0ad89f003ad@enterprisedb.com>
References: <3207907.AWbSqkKDnR@aivenronan>
 <e9b5a975-503d-61c7-c60e-d0ad89f003ad@enterprisedb.com>
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset="iso-8859-1"
Archived-At: 
 <https://www.postgresql.org/message-id/163276811.1OrvvjSa7g%40aivenronan>
Precedence: bulk

Le mercredi 1 septembre 2021, 09:36:50 CEST Peter Eisentraut a =E9crit :
> On 13.08.21 15:23, Ronan Dunklau wrote:
> > The logging system already captures a lot of information in the ErrorDa=
ta.
> > But at present there is no way for a log message authors to include more
> > metadata about the log outside of the log message itself. For example,
> > including the extension name which can be useful for filtering /
> > dispatching log messages. This can be done in the log message itself, b=
ut
> > that requires specialized parsing in the log output.
> >=20
> > Even though among the available loggers in core, only the csv logger wo=
uld
> > be able to make sense of it, I feel it would be beneficial to add a
> > tagging system to logs, by adding different (tag, value) combinations to
> > a log entry, with an API similar to what we do for other ErrorData
> > fields:
> >=20
> > ereport(NOTICE,
> >=20
> >   (errmsg("My log message")),
> >   (errtag("EMITTER", "MYEXTENSION")),
> >   (errtag("MSG-ID", "%d", error_message_id))
> >=20
> > );
>=20
> What are some more examples of where this could be used?  The extension
> name could be added more or less automatically to ereport() calls.  I
> don't know what the MSG-ID is supposed to be.  Are there other use cases?

Adding it automatically would be nice, but how would you go about that ?

In-core it would open up the possibility to split log messages into differe=
nt=20
fields, for example the different statistics reported in the logs by VACUUM=
 /=20
ANALYZE VERBOSE and make it easier to consume the output without having to=
=20
parse the message. Parsing the message also means that any tool parsing it=
=20
needs to either be aware of the locale, or to force the user to use a speci=
fic=20
one.

Out-of-core, the same things could be done for extensions like pg_audit whi=
ch=20
logs structured information into the message itself, leaving the message=20
format to be parsed at a later time.

=2D-=20
Ronan Dunklau