Message-ID: <1975c07f-82a0-3503-e221-211b43878467@joeconway.com>
Date: Tue, 14 Feb 2023 10:59:40 -0500
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.7.1
Subject: Re: BUG #17793: Query with large number of joins crashes PostgreSQL
Content-Language: en-US
To: Tom Lane <tgl@sss.pgh.pa.us>, Francisco Olarte <folarte@peoplecall.com>
Cc: tom@quist.de, pgsql-bugs@lists.postgresql.org
References: <17793-21fa598adbc21118@postgresql.org>
 <CA+bJJbzUKMzkYjGxbQpaRo_evErh+XsnMUUJC+q+edGCM3un8A@mail.gmail.com>
 <1682145.1676386040@sss.pgh.pa.us>
From: Joe Conway <mail@joeconway.com>
In-Reply-To: <1682145.1676386040@sss.pgh.pa.us>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Archived-At: 
 <https://www.postgresql.org/message-id/1975c07f-82a0-3503-e221-211b43878467%40joeconway.com>
Precedence: bulk

On 2/14/23 09:47, Tom Lane wrote:
> Francisco Olarte <folarte@peoplecall.com> writes:
>> On Tue, 14 Feb 2023 at 11:29, PG Bug reporting form
>> <noreply@postgresql.org> wrote:
>>> It looks like the OS is killing the process due to running OOM, which is not
>>> very surprising when looking at the query. Is this expected, or should PG
>>> have guards in place to prevent this from happening?
> 
>> When you run postgres in an environment where someone ( OOM killer )
>> can K9 it, protection is hard. IIRC OOM can kill you because ANOTHER
>> process touches memory, among other things.
> 
> Yeah.  We have no visibility into what the OOM killer will choose to
> allow or not allow at any instant.
> 
>> ( I do run DBs in machines w/ overcommit disabled, this prevents it
>> from happening, but it is not Pg who prevents it ).
> 
> If overcommit-disabled doesn't seem practical, another idea that's
> been recommended is to start the postmaster under a "ulimit -v"
> setting chosen to cause plain ENOMEM failures before the OOM
> killer kicks in.

Given:
> It looks like the OS is killing the process due to running OOM ...
> I could reproduce it using a fresh docker container of the image
> "postgres:13.10".

I gather the OP is running Postgres in a container. If so, and if they 
have a memory.limit set (also assuming cgroup v1, otherwise memory.max 
for cgroup v2), disabling overcommit at the host level will not prevent 
an OOM kill when the cgroup exceeds memory.limit


-- 
Joe Conway
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com