From: Evan Rempel <erempel@uvic.ca>
Cc: pgsql-admin@postgresql.org
Subject: Re: Logging: stderr vs syslog?
Date: Sat, 5 Aug 2017 17:31:15 -0700
Message-ID: <edc8a91a-381a-dd3d-670f-23e984100d81@uvic.ca> (raw)
In-Reply-To: <CAHJZqBDMVf90xb5M410=fnFSgMXW6nH25+BAE9P7P1_HxV3UzQ@mail.gmail.com>
References: <CAHJZqBDe51uWLTiRpHJE_Ebj6mbGH8QhmK7j4hSTiLLbr6G8hA@mail.gmail.com>
<da1673fa-df23-937b-69ad-c6290ff0a607@uvic.ca>
<CAHJZqBDMVf90xb5M410=fnFSgMXW6nH25+BAE9P7P1_HxV3UzQ@mail.gmail.com>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-admin>
Tou are now asking questions that are more appropriate for the specific
syslog product you are using.
rsyslog, syslogd, syslog-ng etc.
In general you need to ask yourself what the ramifications of loosing
the most recent locally stored syslog messages. If you are forwarding
you syslog message to a central syslog server, or perhaps event two
syslog servers then the loss of some local syslog messages in the event
of a local host crash is not very significant.
If you can live with the local loss then adding a "no sync" option to
your syslog configuration is a good idea.
For further details please follow up with your syslog specific support
channel.
Evan.
On 08/04/2017 10:22 PM, Don Seiler wrote:
> Thanks everyone for your feedback. I've got a test setup running with
> syslog and logrotate and it's working well.
>
> Is there any concern with the syslog file sync where it's suggested to
> put a dash before the file name to remove the sync requirement?
>
> Don.
>
> On Aug 4, 2017 2:51 PM, "Evan Rempel" <erempel@uvic.ca
> <mailto:erempel@uvic.ca>> wrote:
>
> I have been a systems administrator of PostgreSQL since version
> 7.0. I am also a primary logging architect for a 4000+ device
> network infrastructure and SEIM alerting infrastructure architect.
>
> Syslog is by far the best approach to logging. With the right
> product it is way FASTER than stderr and there are a ton of tools
> to parse, analyze, view and report on syslog streams.
>
> There is one caveat with PostgreSQL version <= 9.5 and that is
> that syslog messages are wrapped at approx 80 characters which
> makes parsing and error detection problematic. pgBadger may
> address this limitation by unwrapping such logs messages whereas
> generic log parsing engines do not have any specialized knowledge
> about how PostgreSQL lines might be wrapped. PostgreSQL version <=
> 9.5 this is a compile time only option but starting in 9.6 this is
> a runtime configuration directive.
>
> There are two strong reasons for using syslog.
>
> 1. In a well architected logging solution the syslog process on
> the host will also send the log messages to a central log server.
> This means that if the database server is compromised leading to
> an untrusted set of log files, there is a trusted copy of the logs
> on another server.
>
> 2. When running a high availability or clustered database all of
> the logs can be aggregated to a central log server which places
> all of the logs from all of the database servers into one easy to
> read/parse/process location.
>
> I hope this provides some rationale for using syslog.
>
> Evan.
>
> On 08/04/2017 09:26 AM, Don Seiler wrote:
>> I've just inherited a few PostgreSQL DBs, having come from Oracle
>> land. I'm looking to shore up the logging situation. Right now we
>> use stderr logging and they get rotated based on size threshold.
>> I'd like for those old logs to be gzipped so we can keep more on
>> disk rather than current method of just deleting old logs to free
>> up space. This is mostly on pgsql 9.2 with a couple of 9.3, but
>> I'm planning to upgrade everything to 9.6.3 when I get my feet on
>> solid ground.
>>
>> Couple of question around this:
>>
>> 1. I thought logrotate would be a no-brainer here, but it sounds
>> like I should then change to use syslog rather than stderr.
>> I've read some caveats around syslog needing to sync files
>> and potentially slow things down. I'm wondering if any
>> grizzled production postgres veterans could offer up their
>> experience.
>> 2. Alternatively I could just keep it going with stderr and have
>> a separate script run find/gzip on log files beyond a certain
>> mtime threshold. This would probably be the quickest to
>> implement, but I'd much rather use logrotate facilities if
>> there are no strong opinions against using syslog.
>>
>> Thanks in advance for your time, I'm sure I'll be making a lot of
>> us of these mailing lists in the not-too-distant future.
>>
>> Don.
>>
>> --
>> Don Seiler
>> www.seiler.us <http://www.seiler.us;
>
>
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-admin@postgresql.org
Cc: erempel@uvic.ca
Subject: Re: Logging: stderr vs syslog?
In-Reply-To: <edc8a91a-381a-dd3d-670f-23e984100d81@uvic.ca>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox