Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1ougcs-0004Gl-1C for pgsql-general@arkaria.postgresql.org; Mon, 14 Nov 2022 21:04:18 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1ougcq-0004Cz-Vi for pgsql-general@arkaria.postgresql.org; Mon, 14 Nov 2022 21:04:16 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1ougcq-0004Cq-LQ for pgsql-general@lists.postgresql.org; Mon, 14 Nov 2022 21:04:16 +0000 Received: from mail.pernau.at ([2a01:4f8:151:4086::2]) by makus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1ougcn-00070C-Tz for pgsql-general@postgresql.org; Mon, 14 Nov 2022 21:04:15 +0000 Received: from pernau.at (localhost.localdomain [127.0.0.1]) by mail.pernau.at (Postfix) with ESMTPSA id C42B52EA5F76 for ; Mon, 14 Nov 2022 22:04:09 +0100 (CET) MIME-Version: 1.0 Date: Mon, 14 Nov 2022 22:04:09 +0100 From: klaus.mailinglists@pernau.at To: pgsql-general@postgresql.org Subject: PANIC: could not flush dirty data: Cannot allocate memory Message-ID: X-Sender: klaus.mailinglists@pernau.at Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk Hi all! We have a setup with a master and plenty of logical replication slaves. Master and slaves are 12.12-1.pgdg22.04+1 runnning on Ubuntu 22.04. SELECT pg_size_pretty( pg_database_size('regdns') ); is from 25GB (fresh installed slave) to 42GB (probably bloat) Replication slaves VMs have between 22G and 48G RAM, most have 48G RAM. We are using: maintenance_work_mem = 128MB work_mem = 64MB and VMs with 48G RAM: effective_cache_size = 8192MB shared_buffers = 6144MB and VMs with 22G RAM: effective_cache_size = 4096MB shared_buffers = 2048MB On several servers we see the error message: PANIC: could not flush dirty data: Cannot allocate memory Unfortunately I do not find any reference to this kind of error. Can you please describe what happens here in detail? Is it related to server memory? Or our memory settings? I am not so surprised that it happens with the 22G RAM VM. It is not happening on our 32G RAM VMs. But it also happens on some of the 48G RAM VMs which should have plenty of RAM available: # free -h total used free shared buff/cache available Mem: 47Gi 9Gi 1.2Gi 6.1Gi 35Gi 30Gi Swap: 7.8Gi 3.0Gi 4.9Gi Of course I could upgrade all our VMs and then wait and see if it solved the problem. But I would like to understand what is happening here before spending $$$. Thanks Klaus