pg.ddx.io pgsql-admin@postgresql.org mailing list archive
help / color / mirror / Atom feed9.5 new setting "cluster name" and logging
20+ messages / 10 participants
[nested] [flat]
* 9.5 new setting "cluster name" and logging
@ 2016-01-30 06:19 Evan Rempel <erempel@uvic.ca>
2016-02-08 14:24 ` Re: 9.5 new setting "cluster name" and logging Andres Freund <andres@anarazel.de>
2016-02-18 21:42 ` Re: 9.5 new setting "cluster name" and logging Bruce Momjian <bruce@momjian.us>
0 siblings, 2 replies; 20+ messages in thread
From: Evan Rempel @ 2016-01-30 06:19 UTC (permalink / raw)
To: pgsql-admin
Now that there is a setting to give a cluster a "name", it would be nice
to have an escape sequence in the log_line_prefix setting that could
reference the cluster_name.
Comments?
Can we get this on the "to do list"?
Evan.
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: 9.5 new setting "cluster name" and logging
2016-01-30 06:19 9.5 new setting "cluster name" and logging Evan Rempel <erempel@uvic.ca>
@ 2016-02-08 14:24 ` Andres Freund <andres@anarazel.de>
2016-02-08 16:30 ` Re: [HACKERS] 9.5 new setting "cluster name" and logging Joe Conway <mail@joeconway.com>
2016-10-17 21:56 ` Re: [ADMIN] 9.5 new setting "cluster name" and logging Thomas Munro <thomas.munro@enterprisedb.com>
1 sibling, 2 replies; 20+ messages in thread
From: Andres Freund @ 2016-02-08 14:24 UTC (permalink / raw)
To: Evan Rempel <erempel@uvic.ca>; +Cc: pgsql-admin; pgsql-hackers@postgresql.org
Hi,
(x-posting to -hackers, more relevant audience)
On 2016-01-29 22:19:45 -0800, Evan Rempel wrote:
> Now that there is a setting to give a cluster a "name", it would be nice to
> have an escape sequence in the log_line_prefix setting that could reference
> the cluster_name.
I've argued[1][2] for this when cluster_name was introduced, but back
then I seemed to have been the only one arguing for it. Josh later
jumped that train.
Given that we now had a number of people wishing for this, can we maybe
reconsider?
Greetings,
Andres Freund
[1] http://archives.postgresql.org/message-id/CADLWmXUm%3D7Y3UORZnGMdSC6p1eymO0k0JkH4NKr4KstdWk0P7g%40ma...
[2] http://archives.postgresql.org/message-id/20140505101007.GU12715@awork2.anarazel.de
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: [HACKERS] 9.5 new setting "cluster name" and logging
2016-01-30 06:19 9.5 new setting "cluster name" and logging Evan Rempel <erempel@uvic.ca>
2016-02-08 14:24 ` Re: 9.5 new setting "cluster name" and logging Andres Freund <andres@anarazel.de>
@ 2016-02-08 16:30 ` Joe Conway <mail@joeconway.com>
2016-02-08 20:36 ` Re: [ADMIN] 9.5 new setting "cluster name" and logging Thomas Munro <thomas.munro@enterprisedb.com>
1 sibling, 1 reply; 20+ messages in thread
From: Joe Conway @ 2016-02-08 16:30 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; Evan Rempel <erempel@uvic.ca>; +Cc: pgsql-admin; pgsql-hackers@postgresql.org
On 02/08/2016 06:24 AM, Andres Freund wrote:
> On 2016-01-29 22:19:45 -0800, Evan Rempel wrote:
>> Now that there is a setting to give a cluster a "name", it would be nice to
>> have an escape sequence in the log_line_prefix setting that could reference
>> the cluster_name.
>
> I've argued[1][2] for this when cluster_name was introduced, but back
> then I seemed to have been the only one arguing for it. Josh later
> jumped that train.
>
> Given that we now had a number of people wishing for this, can we maybe
> reconsider?
Seems like a reasonable idea to me. But if we add a log_line_prefix
setting, shouldn't we also add it to csvlog output too?
Joe
--
Crunchy Data - http://crunchydata.com
PostgreSQL Support for Secure Enterprises
Consulting, Training, & Open Source Development
Attachments:
signature.asc (application/pgp-signature, 835B) download
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: [ADMIN] 9.5 new setting "cluster name" and logging
2016-01-30 06:19 9.5 new setting "cluster name" and logging Evan Rempel <erempel@uvic.ca>
2016-02-08 14:24 ` Re: 9.5 new setting "cluster name" and logging Andres Freund <andres@anarazel.de>
2016-02-08 16:30 ` Re: [HACKERS] 9.5 new setting "cluster name" and logging Joe Conway <mail@joeconway.com>
@ 2016-02-08 20:36 ` Thomas Munro <thomas.munro@enterprisedb.com>
2016-02-08 20:42 ` Re: [ADMIN] 9.5 new setting "cluster name" and logging Stephen Frost <sfrost@snowman.net>
0 siblings, 1 reply; 20+ messages in thread
From: Thomas Munro @ 2016-02-08 20:36 UTC (permalink / raw)
To: Joe Conway <mail@joeconway.com>; +Cc: Andres Freund <andres@anarazel.de>; Evan Rempel <erempel@uvic.ca>; pgsql-admin@postgresql.org, Pg Hackers <pgsql-hackers@postgresql.org>
On Tue, Feb 9, 2016 at 5:30 AM, Joe Conway <mail@joeconway.com> wrote:
> On 02/08/2016 06:24 AM, Andres Freund wrote:
>> On 2016-01-29 22:19:45 -0800, Evan Rempel wrote:
>>> Now that there is a setting to give a cluster a "name", it would be nice to
>>> have an escape sequence in the log_line_prefix setting that could reference
>>> the cluster_name.
>>
>> I've argued[1][2] for this when cluster_name was introduced, but back
>> then I seemed to have been the only one arguing for it. Josh later
>> jumped that train.
>>
>> Given that we now had a number of people wishing for this, can we maybe
>> reconsider?
>
> Seems like a reasonable idea to me. But if we add a log_line_prefix
> setting, shouldn't we also add it to csvlog output too?
Here's a tiny patch adding support for %C to log_line_prefix (this was
part of the cluster_name patch that didn't go it).
Given that csvlog's output format is hardcoded in write_csvlog, how is
it supposed to evolve without upsetting consumers of this data?
Wouldn't we first need to add a GUC that lets you control the columns
it outputs?
--
Thomas Munro
http://www.enterprisedb.com
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Attachments:
cluster_name_log_line_prefix.patch (application/octet-stream, 2.1K) download
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index de84b77..e8a9fee 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -4671,6 +4671,11 @@ local0.* /var/log/postgresql
</thead>
<tbody>
<row>
+ <entry><literal>%C</literal></entry>
+ <entry>Cluster name</entry>
+ <entry>no</entry>
+ </row>
+ <row>
<entry><literal>%a</literal></entry>
<entry>Application name</entry>
<entry>yes</entry>
@@ -5075,6 +5080,10 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
The process title is typically viewed using programs like
<application>ps</> or, on Windows, <application>Process Explorer</>.
</para>
+ <para>
+ The cluster name can also be included in log files. See
+ <varname>log_line_prefix</varname> for details.
+ </para>
</listitem>
</varlistentry>
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 9005b26..9fe40f8 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -2413,6 +2413,12 @@ log_line_prefix(StringInfo buf, ErrorData *edata)
else
appendStringInfo(buf, "%lx.%x", (long) (MyStartTime), MyProcPid);
break;
+ case 'C':
+ if (padding != 0)
+ appendStringInfo(buf, "%*s", padding, cluster_name);
+ else
+ appendStringInfoString(buf, cluster_name);
+ break;
case 'p':
if (padding != 0)
appendStringInfo(buf, "%*d", padding, MyProcPid);
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 09b2003..748e6cc 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -421,6 +421,7 @@
#log_error_verbosity = default # terse, default, or verbose messages
#log_hostname = off
#log_line_prefix = '' # special values:
+ # %C = cluster name
# %a = application name
# %u = user name
# %d = database name
* Re: [ADMIN] 9.5 new setting "cluster name" and logging
2016-01-30 06:19 9.5 new setting "cluster name" and logging Evan Rempel <erempel@uvic.ca>
2016-02-08 14:24 ` Re: 9.5 new setting "cluster name" and logging Andres Freund <andres@anarazel.de>
2016-02-08 16:30 ` Re: [HACKERS] 9.5 new setting "cluster name" and logging Joe Conway <mail@joeconway.com>
2016-02-08 20:36 ` Re: [ADMIN] 9.5 new setting "cluster name" and logging Thomas Munro <thomas.munro@enterprisedb.com>
@ 2016-02-08 20:42 ` Stephen Frost <sfrost@snowman.net>
0 siblings, 0 replies; 20+ messages in thread
From: Stephen Frost @ 2016-02-08 20:42 UTC (permalink / raw)
To: Thomas Munro <thomas.munro@enterprisedb.com>; +Cc: Joe Conway <mail@joeconway.com>; Andres Freund <andres@anarazel.de>; Evan Rempel <erempel@uvic.ca>; pgsql-admin@postgresql.org, Pg Hackers <pgsql-hackers@postgresql.org>
Thomas,
* Thomas Munro (thomas.munro@enterprisedb.com) wrote:
> On Tue, Feb 9, 2016 at 5:30 AM, Joe Conway <mail@joeconway.com> wrote:
> > On 02/08/2016 06:24 AM, Andres Freund wrote:
> >> On 2016-01-29 22:19:45 -0800, Evan Rempel wrote:
> >>> Now that there is a setting to give a cluster a "name", it would be nice to
> >>> have an escape sequence in the log_line_prefix setting that could reference
> >>> the cluster_name.
> >>
> >> I've argued[1][2] for this when cluster_name was introduced, but back
> >> then I seemed to have been the only one arguing for it. Josh later
> >> jumped that train.
> >>
> >> Given that we now had a number of people wishing for this, can we maybe
> >> reconsider?
> >
> > Seems like a reasonable idea to me. But if we add a log_line_prefix
> > setting, shouldn't we also add it to csvlog output too?
>
> Here's a tiny patch adding support for %C to log_line_prefix (this was
> part of the cluster_name patch that didn't go it).
>
> Given that csvlog's output format is hardcoded in write_csvlog, how is
> it supposed to evolve without upsetting consumers of this data?
> Wouldn't we first need to add a GUC that lets you control the columns
> it outputs?
Not sure if you really want to go there, but I do agree with you and
there's a thread from a few years back about something similar:
http://www.postgresql.org/message-id/flat/20110112142345.GA4933@tamriel.snowman.net
Included in that thread is a patch, which likely requires some dusting
off, to add exactly that ability.
Thanks!
Stephen
Attachments:
signature.asc (application/pgp-signature, 818B) download
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: [ADMIN] 9.5 new setting "cluster name" and logging
2016-01-30 06:19 9.5 new setting "cluster name" and logging Evan Rempel <erempel@uvic.ca>
2016-02-08 14:24 ` Re: 9.5 new setting "cluster name" and logging Andres Freund <andres@anarazel.de>
@ 2016-10-17 21:56 ` Thomas Munro <thomas.munro@enterprisedb.com>
2016-10-17 22:24 ` Re: [ADMIN] 9.5 new setting "cluster name" and logging Stephen Frost <sfrost@snowman.net>
1 sibling, 1 reply; 20+ messages in thread
From: Thomas Munro @ 2016-10-17 21:56 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Evan Rempel <erempel@uvic.ca>; pgsql-admin@postgresql.org, Pg Hackers <pgsql-hackers@postgresql.org>
On Tue, Feb 9, 2016 at 3:24 AM, Andres Freund <andres@anarazel.de> wrote:
> Hi,
>
> (x-posting to -hackers, more relevant audience)
>
> On 2016-01-29 22:19:45 -0800, Evan Rempel wrote:
>> Now that there is a setting to give a cluster a "name", it would be nice to
>> have an escape sequence in the log_line_prefix setting that could reference
>> the cluster_name.
>
> I've argued[1][2] for this when cluster_name was introduced, but back
> then I seemed to have been the only one arguing for it. Josh later
> jumped that train.
>
> Given that we now had a number of people wishing for this, can we maybe
> reconsider?
+1
I missed the part of this conversation that took place on -admin. I
agree with Evan Rempel's post over there[1] that it's useful for
syslog users, and that it's not ideal to have to hijack syslog_ident
or explicitly copy the name into log_line_prefix.
[1] https://www.postgresql.org/message-id/56C64017.7050303%40uvic.ca
--
Thomas Munro
http://www.enterprisedb.com
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: [ADMIN] 9.5 new setting "cluster name" and logging
2016-01-30 06:19 9.5 new setting "cluster name" and logging Evan Rempel <erempel@uvic.ca>
2016-02-08 14:24 ` Re: 9.5 new setting "cluster name" and logging Andres Freund <andres@anarazel.de>
2016-10-17 21:56 ` Re: [ADMIN] 9.5 new setting "cluster name" and logging Thomas Munro <thomas.munro@enterprisedb.com>
@ 2016-10-17 22:24 ` Stephen Frost <sfrost@snowman.net>
2016-10-18 18:25 ` Re: [HACKERS] 9.5 new setting "cluster name" and logging Robert Haas <robertmhaas@gmail.com>
0 siblings, 1 reply; 20+ messages in thread
From: Stephen Frost @ 2016-10-17 22:24 UTC (permalink / raw)
To: Thomas Munro <thomas.munro@enterprisedb.com>; +Cc: Andres Freund <andres@anarazel.de>; Evan Rempel <erempel@uvic.ca>; pgsql-admin@postgresql.org, Pg Hackers <pgsql-hackers@postgresql.org>
* Thomas Munro (thomas.munro@enterprisedb.com) wrote:
> On Tue, Feb 9, 2016 at 3:24 AM, Andres Freund <andres@anarazel.de> wrote:
> > (x-posting to -hackers, more relevant audience)
> >
> > On 2016-01-29 22:19:45 -0800, Evan Rempel wrote:
> >> Now that there is a setting to give a cluster a "name", it would be nice to
> >> have an escape sequence in the log_line_prefix setting that could reference
> >> the cluster_name.
> >
> > I've argued[1][2] for this when cluster_name was introduced, but back
> > then I seemed to have been the only one arguing for it. Josh later
> > jumped that train.
> >
> > Given that we now had a number of people wishing for this, can we maybe
> > reconsider?
>
> +1
>
> I missed the part of this conversation that took place on -admin. I
> agree with Evan Rempel's post over there[1] that it's useful for
> syslog users, and that it's not ideal to have to hijack syslog_ident
> or explicitly copy the name into log_line_prefix.
>
> [1] https://www.postgresql.org/message-id/56C64017.7050303%40uvic.ca
I'm with Thomas on this and I disagree that the "csvlog bloat" argument
has merit. If we're worried about bloat in csv then we should provide a
way for users to control what goes into the csvlog, not argue that
something which is clearly useful be excluded.
There's already a patch out there for adding a way to control what goes
into csvlog and it probably wouldn't be too hard to update it. I know,
because I wrote it. I continue to feel that it would be useful to add
and that tools which need to deal with csvlogs should be updated to
handle being told what's in it, just like pgBadger does for our much
more complicated and painful regular log files. That would remove these
bars to moving forward with more flexibility in log_line_prefix and
csvlog output.
Thanks!
Stephen
Attachments:
signature.asc (application/pgp-signature, 818B) download
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: [HACKERS] 9.5 new setting "cluster name" and logging
2016-01-30 06:19 9.5 new setting "cluster name" and logging Evan Rempel <erempel@uvic.ca>
2016-02-08 14:24 ` Re: 9.5 new setting "cluster name" and logging Andres Freund <andres@anarazel.de>
2016-10-17 21:56 ` Re: [ADMIN] 9.5 new setting "cluster name" and logging Thomas Munro <thomas.munro@enterprisedb.com>
2016-10-17 22:24 ` Re: [ADMIN] 9.5 new setting "cluster name" and logging Stephen Frost <sfrost@snowman.net>
@ 2016-10-18 18:25 ` Robert Haas <robertmhaas@gmail.com>
2016-10-18 18:38 ` Re: [HACKERS] 9.5 new setting "cluster name" and logging Alvaro Herrera <alvherre@2ndquadrant.com>
2016-10-18 18:47 ` Re: [ADMIN] 9.5 new setting "cluster name" and logging Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 2 replies; 20+ messages in thread
From: Robert Haas @ 2016-10-18 18:25 UTC (permalink / raw)
To: Stephen Frost <sfrost@snowman.net>; +Cc: Thomas Munro <thomas.munro@enterprisedb.com>; Andres Freund <andres@anarazel.de>; Evan Rempel <erempel@uvic.ca>; pgsql-admin; Pg Hackers <pgsql-hackers@postgresql.org>
On Mon, Oct 17, 2016 at 6:24 PM, Stephen Frost <sfrost@snowman.net> wrote:
> I'm with Thomas on this and I disagree that the "csvlog bloat" argument
> has merit. If we're worried about bloat in csv then we should provide a
> way for users to control what goes into the csvlog, not argue that
> something which is clearly useful be excluded.
I agree, but I think if we do that it would be highly desirable do
something to make the format discoverable. For example, at the
beginning of each file and whenever the format changes, we write some
kind of unmistakable header line into the file identifying what fields
will be present on each line thereafter. It's undesirable for log
analysis tools to need to contain logic that tries (imperfectly, no
doubt) to reverse-engineer the field list.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: [HACKERS] 9.5 new setting "cluster name" and logging
2016-01-30 06:19 9.5 new setting "cluster name" and logging Evan Rempel <erempel@uvic.ca>
2016-02-08 14:24 ` Re: 9.5 new setting "cluster name" and logging Andres Freund <andres@anarazel.de>
2016-10-17 21:56 ` Re: [ADMIN] 9.5 new setting "cluster name" and logging Thomas Munro <thomas.munro@enterprisedb.com>
2016-10-17 22:24 ` Re: [ADMIN] 9.5 new setting "cluster name" and logging Stephen Frost <sfrost@snowman.net>
2016-10-18 18:25 ` Re: [HACKERS] 9.5 new setting "cluster name" and logging Robert Haas <robertmhaas@gmail.com>
@ 2016-10-18 18:38 ` Alvaro Herrera <alvherre@2ndquadrant.com>
1 sibling, 0 replies; 20+ messages in thread
From: Alvaro Herrera @ 2016-10-18 18:38 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: Stephen Frost <sfrost@snowman.net>; Thomas Munro <thomas.munro@enterprisedb.com>; Andres Freund <andres@anarazel.de>; Evan Rempel <erempel@uvic.ca>; pgsql-admin; Pg Hackers <pgsql-hackers@postgresql.org>
Robert Haas wrote:
> On Mon, Oct 17, 2016 at 6:24 PM, Stephen Frost <sfrost@snowman.net> wrote:
> > I'm with Thomas on this and I disagree that the "csvlog bloat" argument
> > has merit. If we're worried about bloat in csv then we should provide a
> > way for users to control what goes into the csvlog, not argue that
> > something which is clearly useful be excluded.
>
> I agree, but I think if we do that it would be highly desirable do
> something to make the format discoverable. For example, at the
> beginning of each file and whenever the format changes, we write some
> kind of unmistakable header line into the file identifying what fields
> will be present on each line thereafter. It's undesirable for log
> analysis tools to need to contain logic that tries (imperfectly, no
> doubt) to reverse-engineer the field list.
Hmm, I wouldn't use the same file for different formats. If the format
is changed, a different file should be used.
--
Álvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: [ADMIN] 9.5 new setting "cluster name" and logging
2016-01-30 06:19 9.5 new setting "cluster name" and logging Evan Rempel <erempel@uvic.ca>
2016-02-08 14:24 ` Re: 9.5 new setting "cluster name" and logging Andres Freund <andres@anarazel.de>
2016-10-17 21:56 ` Re: [ADMIN] 9.5 new setting "cluster name" and logging Thomas Munro <thomas.munro@enterprisedb.com>
2016-10-17 22:24 ` Re: [ADMIN] 9.5 new setting "cluster name" and logging Stephen Frost <sfrost@snowman.net>
2016-10-18 18:25 ` Re: [HACKERS] 9.5 new setting "cluster name" and logging Robert Haas <robertmhaas@gmail.com>
@ 2016-10-18 18:47 ` Tom Lane <tgl@sss.pgh.pa.us>
1 sibling, 0 replies; 20+ messages in thread
From: Tom Lane @ 2016-10-18 18:47 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: Stephen Frost <sfrost@snowman.net>; Thomas Munro <thomas.munro@enterprisedb.com>; Andres Freund <andres@anarazel.de>; Evan Rempel <erempel@uvic.ca>; pgsql-admin; Pg Hackers <pgsql-hackers@postgresql.org>
Robert Haas <robertmhaas@gmail.com> writes:
> On Mon, Oct 17, 2016 at 6:24 PM, Stephen Frost <sfrost@snowman.net> wrote:
>> I'm with Thomas on this and I disagree that the "csvlog bloat" argument
>> has merit. If we're worried about bloat in csv then we should provide a
>> way for users to control what goes into the csvlog, not argue that
>> something which is clearly useful be excluded.
> I agree, but I think if we do that it would be highly desirable do
> something to make the format discoverable. For example, at the
> beginning of each file and whenever the format changes, we write some
> kind of unmistakable header line into the file identifying what fields
> will be present on each line thereafter. It's undesirable for log
> analysis tools to need to contain logic that tries (imperfectly, no
> doubt) to reverse-engineer the field list.
Automatic "csv header" at the start of each logfile, perhaps? And
force a logfile switch if it changes.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: 9.5 new setting "cluster name" and logging
2016-01-30 06:19 9.5 new setting "cluster name" and logging Evan Rempel <erempel@uvic.ca>
@ 2016-02-18 21:42 ` Bruce Momjian <bruce@momjian.us>
2016-02-18 21:51 ` Re: 9.5 new setting "cluster name" and logging Evan Rempel <erempel@uvic.ca>
2016-02-18 21:53 ` Re: 9.5 new setting "cluster name" and logging Tom Lane <tgl@sss.pgh.pa.us>
1 sibling, 2 replies; 20+ messages in thread
From: Bruce Momjian @ 2016-02-18 21:42 UTC (permalink / raw)
To: Evan Rempel <erempel@uvic.ca>; +Cc: pgsql-admin
On Fri, Jan 29, 2016 at 10:19:45PM -0800, Evan Rempel wrote:
>
> Now that there is a setting to give a cluster a "name", it would be
> nice to have an escape sequence in the log_line_prefix setting that
> could reference the cluster_name.
>
> Comments?
>
> Can we get this on the "to do list"?
Uh, wouldn't the cluster name be the same on every line? Is that
useful?
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ As you are, so once was I. As I am, so you will be. +
+ Roman grave inscription +
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: 9.5 new setting "cluster name" and logging
2016-01-30 06:19 9.5 new setting "cluster name" and logging Evan Rempel <erempel@uvic.ca>
2016-02-18 21:42 ` Re: 9.5 new setting "cluster name" and logging Bruce Momjian <bruce@momjian.us>
@ 2016-02-18 21:51 ` Evan Rempel <erempel@uvic.ca>
1 sibling, 0 replies; 20+ messages in thread
From: Evan Rempel @ 2016-02-18 21:51 UTC (permalink / raw)
To: pgsql-admin
On 02/18/2016 01:42 PM, Bruce Momjian wrote:
> On Fri, Jan 29, 2016 at 10:19:45PM -0800, Evan Rempel wrote:
>> Now that there is a setting to give a cluster a "name", it would be
>> nice to have an escape sequence in the log_line_prefix setting that
>> could reference the cluster_name.
>>
>> Comments?
>>
>> Can we get this on the "to do list"?
> Uh, wouldn't the cluster name be the same on every line? Is that
> useful?
>
That is correct. It does not make sense when looking into local log files, but you would not use the log prefix in that environment.
When this feature would be useful is when you are sending the logs to syslog from many postgresql clusters, and on a central server, You want to collect logs together that use the same cluster name. It just provides more flexibility to add contextual information to the log lines.
--
Evan Rempel
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: 9.5 new setting "cluster name" and logging
2016-01-30 06:19 9.5 new setting "cluster name" and logging Evan Rempel <erempel@uvic.ca>
2016-02-18 21:42 ` Re: 9.5 new setting "cluster name" and logging Bruce Momjian <bruce@momjian.us>
@ 2016-02-18 21:53 ` Tom Lane <tgl@sss.pgh.pa.us>
2016-02-18 22:05 ` Re: 9.5 new setting "cluster name" and logging Evan Rempel <erempel@uvic.ca>
1 sibling, 1 reply; 20+ messages in thread
From: Tom Lane @ 2016-02-18 21:53 UTC (permalink / raw)
To: Bruce Momjian <bruce@momjian.us>; +Cc: Evan Rempel <erempel@uvic.ca>; pgsql-admin
Bruce Momjian <bruce@momjian.us> writes:
> On Fri, Jan 29, 2016 at 10:19:45PM -0800, Evan Rempel wrote:
>> Now that there is a setting to give a cluster a "name", it would be
>> nice to have an escape sequence in the log_line_prefix setting that
>> could reference the cluster_name.
> Uh, wouldn't the cluster name be the same on every line? Is that
> useful?
I guess if you were running multiple clusters and having them all log
to the same syslog daemon, there might be multiple clusters' output
in the same log ... but we already have ways to disambiguate that case,
ie, syslog_ident. Likewise for Windows' event log. I don't think we
should consider it supported for multiple clusters to be logging into
the same plain files. So yeah, the use case for this seems unclear.
regards, tom lane
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: 9.5 new setting "cluster name" and logging
2016-01-30 06:19 9.5 new setting "cluster name" and logging Evan Rempel <erempel@uvic.ca>
2016-02-18 21:42 ` Re: 9.5 new setting "cluster name" and logging Bruce Momjian <bruce@momjian.us>
2016-02-18 21:53 ` Re: 9.5 new setting "cluster name" and logging Tom Lane <tgl@sss.pgh.pa.us>
@ 2016-02-18 22:05 ` Evan Rempel <erempel@uvic.ca>
2016-02-19 13:56 ` Re: 9.5 new setting "cluster name" and logging Fujii Masao <masao.fujii@gmail.com>
0 siblings, 1 reply; 20+ messages in thread
From: Evan Rempel @ 2016-02-18 22:05 UTC (permalink / raw)
To: pgsql-admin
On 02/18/2016 01:53 PM, Tom Lane wrote:
> Bruce Momjian <bruce@momjian.us> writes:
>> On Fri, Jan 29, 2016 at 10:19:45PM -0800, Evan Rempel wrote:
>>> Now that there is a setting to give a cluster a "name", it would be
>>> nice to have an escape sequence in the log_line_prefix setting that
>>> could reference the cluster_name.
>> Uh, wouldn't the cluster name be the same on every line? Is that
>> useful?
> I guess if you were running multiple clusters and having them all log
> to the same syslog daemon, there might be multiple clusters' output
> in the same log ... but we already have ways to disambiguate that case,
> ie, syslog_ident. Likewise for Windows' event log. I don't think we
> should consider it supported for multiple clusters to be logging into
> the same plain files. So yeah, the use case for this seems unclear.
>
> regards, tom lane
The use case is for logging to syslog.
Using the syslog_ident effectively changes the "program name" to something other than postgres. This would break any type of SIEM system that would be looking for events from the program postgres. This means that if you want to have a cluster specific contextual identifier, it needs to be in the log message content, which for the purposes of postgresql means placing it into the log_line_prefix.
It could be hard coded as part of the log_line_prefix such as
log_line_prefix = 'cluster-db1:%d:%u:%h '
That would work, but that means that my cluster name needs to be in the configuration file twice and that opens up the error scenario of not making them exactly the same.
Allowing for the cluster_name to be referenced by an escape sequence would make manual and automated configuration management easier.
Thanks for listening.
--
Evan Rempel
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: 9.5 new setting "cluster name" and logging
2016-01-30 06:19 9.5 new setting "cluster name" and logging Evan Rempel <erempel@uvic.ca>
2016-02-18 21:42 ` Re: 9.5 new setting "cluster name" and logging Bruce Momjian <bruce@momjian.us>
2016-02-18 21:53 ` Re: 9.5 new setting "cluster name" and logging Tom Lane <tgl@sss.pgh.pa.us>
2016-02-18 22:05 ` Re: 9.5 new setting "cluster name" and logging Evan Rempel <erempel@uvic.ca>
@ 2016-02-19 13:56 ` Fujii Masao <masao.fujii@gmail.com>
2016-02-19 15:52 ` Re: 9.5 new setting "cluster name" and logging Evan Rempel <erempel@uvic.ca>
0 siblings, 1 reply; 20+ messages in thread
From: Fujii Masao @ 2016-02-19 13:56 UTC (permalink / raw)
To: Evan Rempel <erempel@uvic.ca>; +Cc: pgsql-admin
On Fri, Feb 19, 2016 at 7:05 AM, Evan Rempel <erempel@uvic.ca> wrote:
> On 02/18/2016 01:53 PM, Tom Lane wrote:
>>
>> Bruce Momjian <bruce@momjian.us> writes:
>>>
>>> On Fri, Jan 29, 2016 at 10:19:45PM -0800, Evan Rempel wrote:
>>>>
>>>> Now that there is a setting to give a cluster a "name", it would be
>>>> nice to have an escape sequence in the log_line_prefix setting that
>>>> could reference the cluster_name.
>>>
>>> Uh, wouldn't the cluster name be the same on every line? Is that
>>> useful?
>>
>> I guess if you were running multiple clusters and having them all log
>> to the same syslog daemon, there might be multiple clusters' output
>> in the same log ... but we already have ways to disambiguate that case,
>> ie, syslog_ident. Likewise for Windows' event log. I don't think we
>> should consider it supported for multiple clusters to be logging into
>> the same plain files. So yeah, the use case for this seems unclear.
>>
>> regards, tom lane
>
>
> The use case is for logging to syslog.
>
> Using the syslog_ident effectively changes the "program name" to something
> other than postgres. This would break any type of SIEM system that would be
> looking for events from the program postgres. This means that if you want to
> have a cluster specific contextual identifier, it needs to be in the log
> message content, which for the purposes of postgresql means placing it into
> the log_line_prefix.
>
> It could be hard coded as part of the log_line_prefix such as
>
> log_line_prefix = 'cluster-db1:%d:%u:%h '
>
> That would work, but that means that my cluster name needs to be in the
> configuration file twice and that opens up the error scenario of not making
> them exactly the same.
>
> Allowing for the cluster_name to be referenced by an escape sequence would
> make manual and automated configuration management easier.
What if log_destination is set to csvlog? The same cluster_name is
output in every lines, and there is no way to disable it because
something like csv_line_prefix has not been supported yet?
Regards,
--
Fujii Masao
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: 9.5 new setting "cluster name" and logging
2016-01-30 06:19 9.5 new setting "cluster name" and logging Evan Rempel <erempel@uvic.ca>
2016-02-18 21:42 ` Re: 9.5 new setting "cluster name" and logging Bruce Momjian <bruce@momjian.us>
2016-02-18 21:53 ` Re: 9.5 new setting "cluster name" and logging Tom Lane <tgl@sss.pgh.pa.us>
2016-02-18 22:05 ` Re: 9.5 new setting "cluster name" and logging Evan Rempel <erempel@uvic.ca>
2016-02-19 13:56 ` Re: 9.5 new setting "cluster name" and logging Fujii Masao <masao.fujii@gmail.com>
@ 2016-02-19 15:52 ` Evan Rempel <erempel@uvic.ca>
2016-02-19 16:10 ` Re: 9.5 new setting "cluster name" and logging Fujii Masao <masao.fujii@gmail.com>
2016-02-19 16:12 ` Re: 9.5 new setting "cluster name" and logging Bruce Momjian <bruce@momjian.us>
0 siblings, 2 replies; 20+ messages in thread
From: Evan Rempel @ 2016-02-19 15:52 UTC (permalink / raw)
To: Fujii Masao <masao.fujii@gmail.com>; +Cc: pgsql-admin
On 02/19/2016 05:56 AM, Fujii Masao wrote:
> On Fri, Feb 19, 2016 at 7:05 AM, Evan Rempel <erempel@uvic.ca> wrote:
>> On 02/18/2016 01:53 PM, Tom Lane wrote:
>>> Bruce Momjian <bruce@momjian.us> writes:
>>>> On Fri, Jan 29, 2016 at 10:19:45PM -0800, Evan Rempel wrote:
>>>>> Now that there is a setting to give a cluster a "name", it would be
>>>>> nice to have an escape sequence in the log_line_prefix setting that
>>>>> could reference the cluster_name.
>>>> Uh, wouldn't the cluster name be the same on every line? Is that
>>>> useful?
>>> I guess if you were running multiple clusters and having them all log
>>> to the same syslog daemon, there might be multiple clusters' output
>>> in the same log ... but we already have ways to disambiguate that case,
>>> ie, syslog_ident. Likewise for Windows' event log. I don't think we
>>> should consider it supported for multiple clusters to be logging into
>>> the same plain files. So yeah, the use case for this seems unclear.
>>>
>>> regards, tom lane
>>
>> The use case is for logging to syslog.
>>
>> Using the syslog_ident effectively changes the "program name" to something
>> other than postgres. This would break any type of SIEM system that would be
>> looking for events from the program postgres. This means that if you want to
>> have a cluster specific contextual identifier, it needs to be in the log
>> message content, which for the purposes of postgresql means placing it into
>> the log_line_prefix.
>>
>> It could be hard coded as part of the log_line_prefix such as
>>
>> log_line_prefix = 'cluster-db1:%d:%u:%h '
>>
>> That would work, but that means that my cluster name needs to be in the
>> configuration file twice and that opens up the error scenario of not making
>> them exactly the same.
>>
>> Allowing for the cluster_name to be referenced by an escape sequence would
>> make manual and automated configuration management easier.
> What if log_destination is set to csvlog? The same cluster_name is
> output in every lines, and there is no way to disable it because
> something like csv_line_prefix has not been supported yet?
>
> Regards,
>
I would think that in the case of using a csvlog that you would configure the log_line_prefix to not include the escape sequence of the cluster_name. The administrator would tune all of the logging components to work together, or am I missing something here?
--
Evan Rempel
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: 9.5 new setting "cluster name" and logging
2016-01-30 06:19 9.5 new setting "cluster name" and logging Evan Rempel <erempel@uvic.ca>
2016-02-18 21:42 ` Re: 9.5 new setting "cluster name" and logging Bruce Momjian <bruce@momjian.us>
2016-02-18 21:53 ` Re: 9.5 new setting "cluster name" and logging Tom Lane <tgl@sss.pgh.pa.us>
2016-02-18 22:05 ` Re: 9.5 new setting "cluster name" and logging Evan Rempel <erempel@uvic.ca>
2016-02-19 13:56 ` Re: 9.5 new setting "cluster name" and logging Fujii Masao <masao.fujii@gmail.com>
2016-02-19 15:52 ` Re: 9.5 new setting "cluster name" and logging Evan Rempel <erempel@uvic.ca>
@ 2016-02-19 16:10 ` Fujii Masao <masao.fujii@gmail.com>
2016-02-19 16:19 ` Re: 9.5 new setting "cluster name" and logging Evan Rempel <erempel@uvic.ca>
1 sibling, 1 reply; 20+ messages in thread
From: Fujii Masao @ 2016-02-19 16:10 UTC (permalink / raw)
To: Evan Rempel <erempel@uvic.ca>; +Cc: pgsql-admin
On Sat, Feb 20, 2016 at 12:52 AM, Evan Rempel <erempel@uvic.ca> wrote:
> On 02/19/2016 05:56 AM, Fujii Masao wrote:
>>
>> On Fri, Feb 19, 2016 at 7:05 AM, Evan Rempel <erempel@uvic.ca> wrote:
>>>
>>> On 02/18/2016 01:53 PM, Tom Lane wrote:
>>>>
>>>> Bruce Momjian <bruce@momjian.us> writes:
>>>>>
>>>>> On Fri, Jan 29, 2016 at 10:19:45PM -0800, Evan Rempel wrote:
>>>>>>
>>>>>> Now that there is a setting to give a cluster a "name", it would be
>>>>>> nice to have an escape sequence in the log_line_prefix setting that
>>>>>> could reference the cluster_name.
>>>>>
>>>>> Uh, wouldn't the cluster name be the same on every line? Is that
>>>>> useful?
>>>>
>>>> I guess if you were running multiple clusters and having them all log
>>>> to the same syslog daemon, there might be multiple clusters' output
>>>> in the same log ... but we already have ways to disambiguate that case,
>>>> ie, syslog_ident. Likewise for Windows' event log. I don't think we
>>>> should consider it supported for multiple clusters to be logging into
>>>> the same plain files. So yeah, the use case for this seems unclear.
>>>>
>>>> regards, tom lane
>>>
>>>
>>> The use case is for logging to syslog.
>>>
>>> Using the syslog_ident effectively changes the "program name" to
>>> something
>>> other than postgres. This would break any type of SIEM system that would
>>> be
>>> looking for events from the program postgres. This means that if you want
>>> to
>>> have a cluster specific contextual identifier, it needs to be in the log
>>> message content, which for the purposes of postgresql means placing it
>>> into
>>> the log_line_prefix.
>>>
>>> It could be hard coded as part of the log_line_prefix such as
>>>
>>> log_line_prefix = 'cluster-db1:%d:%u:%h '
>>>
>>> That would work, but that means that my cluster name needs to be in the
>>> configuration file twice and that opens up the error scenario of not
>>> making
>>> them exactly the same.
>>>
>>> Allowing for the cluster_name to be referenced by an escape sequence
>>> would
>>> make manual and automated configuration management easier.
>>
>> What if log_destination is set to csvlog? The same cluster_name is
>> output in every lines, and there is no way to disable it because
>> something like csv_line_prefix has not been supported yet?
>>
>> Regards,
>>
>
> I would think that in the case of using a csvlog that you would configure
> the log_line_prefix to not include the escape sequence of the cluster_name.
> The administrator would tune all of the logging components to work together,
> or am I missing something here?
log_line_prefix is only for stderr and syslog, not for csvlog.
It cannot control what to output in csvlog, for now.
Regards,
--
Fujii Masao
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: 9.5 new setting "cluster name" and logging
2016-01-30 06:19 9.5 new setting "cluster name" and logging Evan Rempel <erempel@uvic.ca>
2016-02-18 21:42 ` Re: 9.5 new setting "cluster name" and logging Bruce Momjian <bruce@momjian.us>
2016-02-18 21:53 ` Re: 9.5 new setting "cluster name" and logging Tom Lane <tgl@sss.pgh.pa.us>
2016-02-18 22:05 ` Re: 9.5 new setting "cluster name" and logging Evan Rempel <erempel@uvic.ca>
2016-02-19 13:56 ` Re: 9.5 new setting "cluster name" and logging Fujii Masao <masao.fujii@gmail.com>
2016-02-19 15:52 ` Re: 9.5 new setting "cluster name" and logging Evan Rempel <erempel@uvic.ca>
2016-02-19 16:10 ` Re: 9.5 new setting "cluster name" and logging Fujii Masao <masao.fujii@gmail.com>
@ 2016-02-19 16:19 ` Evan Rempel <erempel@uvic.ca>
2016-02-19 16:50 ` Re: 9.5 new setting "cluster name" and logging Fujii Masao <masao.fujii@gmail.com>
0 siblings, 1 reply; 20+ messages in thread
From: Evan Rempel @ 2016-02-19 16:19 UTC (permalink / raw)
To: ; +Cc: pgsql-admin
On 02/19/2016 08:10 AM, Fujii Masao wrote:
> On Sat, Feb 20, 2016 at 12:52 AM, Evan Rempel <erempel@uvic.ca> wrote:
>> On 02/19/2016 05:56 AM, Fujii Masao wrote:
>>> On Fri, Feb 19, 2016 at 7:05 AM, Evan Rempel <erempel@uvic.ca> wrote:
>>>> On 02/18/2016 01:53 PM, Tom Lane wrote:
>>>>> Bruce Momjian <bruce@momjian.us> writes:
>>>>>> On Fri, Jan 29, 2016 at 10:19:45PM -0800, Evan Rempel wrote:
>>>>>>> Now that there is a setting to give a cluster a "name", it would be
>>>>>>> nice to have an escape sequence in the log_line_prefix setting that
>>>>>>> could reference the cluster_name.
>>>>>> Uh, wouldn't the cluster name be the same on every line? Is that
>>>>>> useful?
>>>>> I guess if you were running multiple clusters and having them all log
>>>>> to the same syslog daemon, there might be multiple clusters' output
>>>>> in the same log ... but we already have ways to disambiguate that case,
>>>>> ie, syslog_ident. Likewise for Windows' event log. I don't think we
>>>>> should consider it supported for multiple clusters to be logging into
>>>>> the same plain files. So yeah, the use case for this seems unclear.
>>>>>
>>>>> regards, tom lane
>>>>
>>>> The use case is for logging to syslog.
>>>>
>>>> Using the syslog_ident effectively changes the "program name" to
>>>> something
>>>> other than postgres. This would break any type of SIEM system that would
>>>> be
>>>> looking for events from the program postgres. This means that if you want
>>>> to
>>>> have a cluster specific contextual identifier, it needs to be in the log
>>>> message content, which for the purposes of postgresql means placing it
>>>> into
>>>> the log_line_prefix.
>>>>
>>>> It could be hard coded as part of the log_line_prefix such as
>>>>
>>>> log_line_prefix = 'cluster-db1:%d:%u:%h '
>>>>
>>>> That would work, but that means that my cluster name needs to be in the
>>>> configuration file twice and that opens up the error scenario of not
>>>> making
>>>> them exactly the same.
>>>>
>>>> Allowing for the cluster_name to be referenced by an escape sequence
>>>> would
>>>> make manual and automated configuration management easier.
>>> What if log_destination is set to csvlog? The same cluster_name is
>>> output in every lines, and there is no way to disable it because
>>> something like csv_line_prefix has not been supported yet?
>>>
>>> Regards,
>>>
>> I would think that in the case of using a csvlog that you would configure
>> the log_line_prefix to not include the escape sequence of the cluster_name.
>> The administrator would tune all of the logging components to work together,
>> or am I missing something here?
> log_line_prefix is only for stderr and syslog, not for csvlog.
> It cannot control what to output in csvlog, for now.
>
> Regards,
In that case, my request for a new escape sequence for the log_line_prefix will have no impact on csvlog output because it would
only affect uses cases where log_line_prefix is used AND where the administrator has added the cluster name into the log_line_prefix.
--
Evan
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: 9.5 new setting "cluster name" and logging
2016-01-30 06:19 9.5 new setting "cluster name" and logging Evan Rempel <erempel@uvic.ca>
2016-02-18 21:42 ` Re: 9.5 new setting "cluster name" and logging Bruce Momjian <bruce@momjian.us>
2016-02-18 21:53 ` Re: 9.5 new setting "cluster name" and logging Tom Lane <tgl@sss.pgh.pa.us>
2016-02-18 22:05 ` Re: 9.5 new setting "cluster name" and logging Evan Rempel <erempel@uvic.ca>
2016-02-19 13:56 ` Re: 9.5 new setting "cluster name" and logging Fujii Masao <masao.fujii@gmail.com>
2016-02-19 15:52 ` Re: 9.5 new setting "cluster name" and logging Evan Rempel <erempel@uvic.ca>
2016-02-19 16:10 ` Re: 9.5 new setting "cluster name" and logging Fujii Masao <masao.fujii@gmail.com>
2016-02-19 16:19 ` Re: 9.5 new setting "cluster name" and logging Evan Rempel <erempel@uvic.ca>
@ 2016-02-19 16:50 ` Fujii Masao <masao.fujii@gmail.com>
0 siblings, 0 replies; 20+ messages in thread
From: Fujii Masao @ 2016-02-19 16:50 UTC (permalink / raw)
To: Evan Rempel <erempel@uvic.ca>; +Cc: pgsql-admin
On Sat, Feb 20, 2016 at 1:19 AM, Evan Rempel <erempel@uvic.ca> wrote:
> On 02/19/2016 08:10 AM, Fujii Masao wrote:
>>
>> On Sat, Feb 20, 2016 at 12:52 AM, Evan Rempel <erempel@uvic.ca> wrote:
>>>
>>> On 02/19/2016 05:56 AM, Fujii Masao wrote:
>>>>
>>>> On Fri, Feb 19, 2016 at 7:05 AM, Evan Rempel <erempel@uvic.ca> wrote:
>>>>>
>>>>> On 02/18/2016 01:53 PM, Tom Lane wrote:
>>>>>>
>>>>>> Bruce Momjian <bruce@momjian.us> writes:
>>>>>>>
>>>>>>> On Fri, Jan 29, 2016 at 10:19:45PM -0800, Evan Rempel wrote:
>>>>>>>>
>>>>>>>> Now that there is a setting to give a cluster a "name", it would be
>>>>>>>> nice to have an escape sequence in the log_line_prefix setting that
>>>>>>>> could reference the cluster_name.
>>>>>>>
>>>>>>> Uh, wouldn't the cluster name be the same on every line? Is that
>>>>>>> useful?
>>>>>>
>>>>>> I guess if you were running multiple clusters and having them all log
>>>>>> to the same syslog daemon, there might be multiple clusters' output
>>>>>> in the same log ... but we already have ways to disambiguate that
>>>>>> case,
>>>>>> ie, syslog_ident. Likewise for Windows' event log. I don't think we
>>>>>> should consider it supported for multiple clusters to be logging into
>>>>>> the same plain files. So yeah, the use case for this seems unclear.
>>>>>>
>>>>>> regards, tom lane
>>>>>
>>>>>
>>>>> The use case is for logging to syslog.
>>>>>
>>>>> Using the syslog_ident effectively changes the "program name" to
>>>>> something
>>>>> other than postgres. This would break any type of SIEM system that
>>>>> would
>>>>> be
>>>>> looking for events from the program postgres. This means that if you
>>>>> want
>>>>> to
>>>>> have a cluster specific contextual identifier, it needs to be in the
>>>>> log
>>>>> message content, which for the purposes of postgresql means placing it
>>>>> into
>>>>> the log_line_prefix.
>>>>>
>>>>> It could be hard coded as part of the log_line_prefix such as
>>>>>
>>>>> log_line_prefix = 'cluster-db1:%d:%u:%h '
>>>>>
>>>>> That would work, but that means that my cluster name needs to be in the
>>>>> configuration file twice and that opens up the error scenario of not
>>>>> making
>>>>> them exactly the same.
>>>>>
>>>>> Allowing for the cluster_name to be referenced by an escape sequence
>>>>> would
>>>>> make manual and automated configuration management easier.
>>>>
>>>> What if log_destination is set to csvlog? The same cluster_name is
>>>> output in every lines, and there is no way to disable it because
>>>> something like csv_line_prefix has not been supported yet?
>>>>
>>>> Regards,
>>>>
>>> I would think that in the case of using a csvlog that you would configure
>>> the log_line_prefix to not include the escape sequence of the
>>> cluster_name.
>>> The administrator would tune all of the logging components to work
>>> together,
>>> or am I missing something here?
>>
>> log_line_prefix is only for stderr and syslog, not for csvlog.
>> It cannot control what to output in csvlog, for now.
>>
>> Regards,
>
>
> In that case, my request for a new escape sequence for the log_line_prefix
> will have no impact on csvlog output because it would
> only affect uses cases where log_line_prefix is used AND where the
> administrator has added the cluster name into the log_line_prefix.
So cluster_name will never appear in csvlog. Maybe I'm OK with that.
OTOH, there are many "escape sequences" available in log_line_prefix,
and their corresponding information is basically output in csvlog,
but only cluster_name is not. That design looks a bit odd.
Regards,
--
Fujii Masao
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: 9.5 new setting "cluster name" and logging
2016-01-30 06:19 9.5 new setting "cluster name" and logging Evan Rempel <erempel@uvic.ca>
2016-02-18 21:42 ` Re: 9.5 new setting "cluster name" and logging Bruce Momjian <bruce@momjian.us>
2016-02-18 21:53 ` Re: 9.5 new setting "cluster name" and logging Tom Lane <tgl@sss.pgh.pa.us>
2016-02-18 22:05 ` Re: 9.5 new setting "cluster name" and logging Evan Rempel <erempel@uvic.ca>
2016-02-19 13:56 ` Re: 9.5 new setting "cluster name" and logging Fujii Masao <masao.fujii@gmail.com>
2016-02-19 15:52 ` Re: 9.5 new setting "cluster name" and logging Evan Rempel <erempel@uvic.ca>
@ 2016-02-19 16:12 ` Bruce Momjian <bruce@momjian.us>
1 sibling, 0 replies; 20+ messages in thread
From: Bruce Momjian @ 2016-02-19 16:12 UTC (permalink / raw)
To: Evan Rempel <erempel@uvic.ca>; +Cc: Fujii Masao <masao.fujii@gmail.com>; pgsql-admin
On Fri, Feb 19, 2016 at 07:52:25AM -0800, Evan Rempel wrote:
> >>Allowing for the cluster_name to be referenced by an escape sequence
> >>would make manual and automated configuration management easier.
> >What if log_destination is set to csvlog? The same cluster_name is
> >output in every lines, and there is no way to disable it because
> >something like csv_line_prefix has not been supported yet?
> >
> >Regards,
> >
>
> I would think that in the case of using a csvlog that you would
> configure the log_line_prefix to not include the escape sequence of
> the cluster_name. The administrator would tune all of the logging
> components to work together, or am I missing something here?
csvlog isn't affected by log_line_prefix --- it outputs all fields every
time:
http://www.postgresql.org/docs/9.5/static/runtime-config-logging.html#RUNTIME-CONFIG-LOGGING-CSVLOG
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ As you are, so once was I. As I am, so you will be. +
+ Roman grave inscription +
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
^ permalink raw reply [nested|flat] 20+ messages in thread
end of thread, other threads:[~2016-10-18 18:47 UTC | newest]
Thread overview: 20+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2016-01-30 06:19 9.5 new setting "cluster name" and logging Evan Rempel <erempel@uvic.ca>
2016-02-08 14:24 ` Andres Freund <andres@anarazel.de>
2016-02-08 16:30 ` Joe Conway <mail@joeconway.com>
2016-02-08 20:36 ` Thomas Munro <thomas.munro@enterprisedb.com>
2016-02-08 20:42 ` Stephen Frost <sfrost@snowman.net>
2016-10-17 21:56 ` Thomas Munro <thomas.munro@enterprisedb.com>
2016-10-17 22:24 ` Stephen Frost <sfrost@snowman.net>
2016-10-18 18:25 ` Robert Haas <robertmhaas@gmail.com>
2016-10-18 18:38 ` Alvaro Herrera <alvherre@2ndquadrant.com>
2016-10-18 18:47 ` Tom Lane <tgl@sss.pgh.pa.us>
2016-02-18 21:42 ` Bruce Momjian <bruce@momjian.us>
2016-02-18 21:51 ` Evan Rempel <erempel@uvic.ca>
2016-02-18 21:53 ` Tom Lane <tgl@sss.pgh.pa.us>
2016-02-18 22:05 ` Evan Rempel <erempel@uvic.ca>
2016-02-19 13:56 ` Fujii Masao <masao.fujii@gmail.com>
2016-02-19 15:52 ` Evan Rempel <erempel@uvic.ca>
2016-02-19 16:10 ` Fujii Masao <masao.fujii@gmail.com>
2016-02-19 16:19 ` Evan Rempel <erempel@uvic.ca>
2016-02-19 16:50 ` Fujii Masao <masao.fujii@gmail.com>
2016-02-19 16:12 ` Bruce Momjian <bruce@momjian.us>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox