public inbox for [email protected]  

help / color / mirror / Atom feed

From: Christophe Pettus <[email protected]>
To: Dominique Devienne <[email protected]>
Cc: [email protected]
Subject: Re: Advice on efficiently logging outputs to PostgreSQL
Date: Tue, 15 Oct 2024 10:56:25 -0700
Message-ID: <[email protected]> (raw)
In-Reply-To: <CAFCRh-9nu9LXb6+p_pp2iXRxp4mrfJz-2S_cormOo7X-ZkBv=g@mail.gmail.com>
References: <CAFCRh-9nu9LXb6+p_pp2iXRxp4mrfJz-2S_cormOo7X-ZkBv=g@mail.gmail.com>



> On Oct 15, 2024, at 07:17, Dominique Devienne <[email protected]> wrote:
> Am I worrying too much? :)

Probably. :-)  The main things I'd worry about is:

1. What's the ratio of log lines to database updates?  You want this to be as high as usefully possible, since in effect you are doing write amplification by writing to the logs as well as to the "real" database.

2. One of the things to watch out for if you are writing the log lines to the database is that if you write a log line in a transaction, and that transaction rolls back, you lose the log line.  That may or may not be what you want: if it's reporting an error (such as the reason that the transaction rolled back), you want to preserve that data.  One way of handling this is to have the application have a separate session for logging, although you are now doing 2x the number of connections.

If the write volume is very high, you might consider using a dedicate log-ingestion service (there are tons) rather than PostgreSQL, so that you aren't overburdening the database with log activity.

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected], [email protected]
  Subject: Re: Advice on efficiently logging outputs to PostgreSQL
  In-Reply-To: <[email protected]>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox