Subject: Re: PG 11.8 > PG 12.3 : Unable to allocate section memory
To: mjoigny@neteven.com
Cc: pgsql-admin@postgresql.org
References: <64eae88b-8ac5-366c-b6a2-e7b59b3d191b@neteven.com>
From: MichaelDBA <MichaelDBA@sqlexec.com>
Message-ID: <544598c0-8d92-b66b-e015-4fc7e5735c35@sqlexec.com>
Date: Wed, 12 Aug 2020 10:26:47 -0400
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101
 PostboxApp/7.0.25
MIME-Version: 1.0
In-Reply-To: <64eae88b-8ac5-366c-b6a2-e7b59b3d191b@neteven.com>
Content-Type: multipart/alternative;
 boundary="------------EB5C4E4BD1C384431B48256D"
Content-Language: en-US
Precedence: bulk

This is a multi-part message in MIME format.
--------------EB5C4E4BD1C384431B48256D
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit

Please specify some other stuff:

1. PG onprem or cloud? If cloud, which one and what instance type.
2. Is it a dedicated PG host?  How many hyper threads and memory do you 
have?
3. Does PGBouncer run on another host or on the same host as PG?

JOIGNY Michael @Neteven wrote on 8/12/2020 10:09 AM:
>
> Hi Community,
>
> We have migrated from postgresql 11.8 to 12.3,unfortunately we have 
> sometimes the following errors in the logs, causing postgres process 
> to be killed.
>
> 2020-08-12 14:20:29.043 CEST [134644]: [71-1] 
> user=xx,db=xx,app=[unknown],client=localhost FATAL:  out of memory
> 2020-08-12 14:20:29.049 CEST [30022]: [6126-1] 
> user=xx,db=xx,app=[unknown],client=localhost FATAL:  fatal llvm error: 
> Unable to allocate section memory!
> 2020-08-12 14:20:29.197 CEST [78792]: [1-1] user=,db=,app=,client= 
> FATAL:  terminating connection due to administrator command
> 2020-08-12 14:20:29.197 CEST [78791]: [1-1] user=,db=,app=,client= 
> FATAL:  terminating connection due to administrator command
> 2020-08-12 14:20:29.197 CEST [78793]: [1-1] user=,db=,app=,client= 
> FATAL:  terminating connection due to administrator command
> 2020-08-12 14:20:29.197 CEST [78790]: [1-1] user=,db=,app=,client= 
> FATAL:  terminating connection due to administrator command
> 2020-08-12 14:21:02.621 CEST [86289]: [2-1] 
> user=postgres,db=template1,app=[unknown],client=[local] FATAL: out of 
> memory
> 2020-08-12 14:21:04.632 CEST [29597]: [26704-1] 
> user=xx,db=xx,app=[unknown],client=localhost FATAL:  fatal llvm error: 
> Unable to allocate section memory!
>
> Here are our settings :
>
> _PG :_
>
> max_connections = 3600
>
> shared_buffers = 32GB
> temp_buffers = 16MB
> work_mem = 96MB
> maintenance_work_mem = 8GB
> max_stack_depth = 6MB
> dynamic_shared_memory_type = posix
>
> We use postgresql with pgbouncer as a pooler.
>
>
> _PGbouncer :
> _
>
> pooler mode : session
> max_client_conn = 6000
> default_pool_size = 3000
>
> We have ~ 2000 connections at the same time with ~ 20/30 are active.
>
> Note that we had the same configuration under PG11 without any problem.
>
> Do you have an idea ?
>
> Regards.
>


--------------EB5C4E4BD1C384431B48256D
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 8bit

<html theme="default-light"><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head><body text="#000000">Please specify some other stuff:<br>
<br>
1. PG onprem or cloud? If cloud, which one and what instance type.<br>
2. Is it a dedicated PG host?  How many hyper threads and memory do you 
have?<br>
3. Does PGBouncer run on another host or on the same host as PG?<br>
<br>
<span>JOIGNY Michael @Neteven wrote on 8/12/2020 10:09 AM:</span><br>
<blockquote type="cite" 
cite="mid:64eae88b-8ac5-366c-b6a2-e7b59b3d191b@neteven.com">
  <meta http-equiv="content-type" content="text/html; charset=utf-8">

  
  <p>Hi Community,</p>

    We have migrated from postgresql 11.8 to 12.3,<span lang="en">
      unfortunately </span>we have sometimes the following errors in
    the logs, causing postgres process to be killed. <br>

    <br>

    <font size="-1">2020-08-12 14:20:29.043 CEST [134644]: [71-1]
      user=xx,db=xx,app=[unknown],client=localhost FATAL:  out of memory<br>
      2020-08-12 14:20:29.049 CEST [30022]: [6126-1]
      user=xx,db=xx,app=[unknown],client=localhost FATAL:  fatal llvm
      error: Unable to allocate section memory!<br>
      2020-08-12 14:20:29.197 CEST [78792]: [1-1] user=,db=,app=,client=
      FATAL:  terminating connection due to administrator command<br>
      2020-08-12 14:20:29.197 CEST [78791]: [1-1] user=,db=,app=,client=
      FATAL:  terminating connection due to administrator command<br>
      2020-08-12 14:20:29.197 CEST [78793]: [1-1] user=,db=,app=,client=
      FATAL:  terminating connection due to administrator command<br>
      2020-08-12 14:20:29.197 CEST [78790]: [1-1] user=,db=,app=,client=
      FATAL:  terminating connection due to administrator command<br>
      2020-08-12 14:21:02.621 CEST [86289]: [2-1]
      user=postgres,db=template1,app=[unknown],client=[local] FATAL: 
      out of memory<br>
      2020-08-12 14:21:04.632 CEST [29597]: [26704-1]
      user=xx,db=xx,app=[unknown],client=localhost FATAL:  fatal llvm
      error: Unable to allocate section memory!</font><br>

    
  <p>Here are our settings : <br>
    </p>

    
  <p><u>PG :</u><br>
    </p>

    
  <p><font size="-1">max_connections = 3600     <br>
      </font></p>

    
  <p>
    </p>

    <font size="-1">shared_buffers = 32GB            <br>
      temp_buffers = 16MB                 <br>
      work_mem = 96MB                     <br>
      maintenance_work_mem = 8GB            <br>
      max_stack_depth = 6MB                   <br>
      dynamic_shared_memory_type = posix  </font><br>

    
  <p>We use postgresql with pgbouncer as a pooler.</p>

    <font size="-1"> </font><br>

    
  <p><u>PGbouncer : <br>
      </u></p>

    
  <p><font size="-1">pooler mode : session<br>
        max_client_conn = 6000<br>
        default_pool_size = 3000</font></p>

    <span lang="en">We have ~ 2000 connections at the same time with ~
      20/30 are active.
      <br>
      <br>
      Note that we had the same configuration under PG11 without any
      problem.</span><br>

    
  <p>Do you have an idea ?</p>

    
  <p>Regards.<br>
    </p>
</blockquote>
<br>
</body></html>

--------------EB5C4E4BD1C384431B48256D--