From: Justin Pryzby <pryzby@telsasoft.com>
To: pgsql-general@lists.postgresql.org
Cc: Christoph Berg <myon@debian.org>
Cc: Devrim Gündüz <devrim@gunduz.org>
Subject: Re: Properly handle OOM death?
Date: Mon, 13 Nov 2023 08:42:31 -0600
Message-ID: <ZVI112aVNCHOQgfF@pryzbyj2023> (raw)
In-Reply-To: <20230313174301.23n6crb5dpttozup@hjp.at>
References: <E1525E23-A751-41C6-AD65-B3BF6C982B06@alaska.edu>
<20230313174301.23n6crb5dpttozup@hjp.at>
On Mon, Mar 13, 2023 at 06:43:01PM +0100, Peter J. Holzer wrote:
> On 2023-03-13 09:21:18 -0800, Israel Brewster wrote:
> > I’m running a postgresql 13 database on an Ubuntu 20.04 VM that is a bit more
> > memory constrained than I would like, such that every week or so the various
> > processes running on the machine will align badly and the OOM killer will kick
> > in, killing off postgresql, as per the following journalctl output:
> >
> > Mar 12 04:04:23 novarupta systemd[1]: postgresql@13-main.service: A process of this unit has been killed by the OOM killer.
> > Mar 12 04:04:32 novarupta systemd[1]: postgresql@13-main.service: Failed with result 'oom-kill'.
> > Mar 12 04:04:32 novarupta systemd[1]: postgresql@13-main.service: Consumed 5d 17h 48min 24.509s CPU time.
> >
> > And the service is no longer running.
>
> I might be misreading this, but it looks to me that systemd detects that
> *some* process in the group was killed by the oom killer and stops the
> service.
Yeah.
I found this old message on google. I'm surprised there aren't more,
similar complaints about this. It's as Peter said: it (sometimes)
causes systemd to actively *stop* the cluster after OOM, when it
would've come back online on its own if the init (supervisor) process
didn't interfere.
My solution was to set:
/usr/lib/systemd/system/postgresql@.service OOMPolicy=continue
I suggest that the default unit files should do likewise.
--
Justin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-general@postgresql.org
Cc: pryzby@telsasoft.com, myon@debian.org, devrim@gunduz.org
Subject: Re: Properly handle OOM death?
In-Reply-To: <ZVI112aVNCHOQgfF@pryzbyj2023>
References: <E1525E23-A751-41C6-AD65-B3BF6C982B06@alaska.edu> <20230313174301.23n6crb5dpttozup@hjp.at> <ZVI112aVNCHOQgfF@pryzbyj2023>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox