Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jQcFD-0000WR-S3 for pgsql-admin@arkaria.postgresql.org; Mon, 20 Apr 2020 19:38:15 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1jQcFC-0001kN-Np for pgsql-admin@arkaria.postgresql.org; Mon, 20 Apr 2020 19:38:14 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jQcFC-0001kG-GR for pgsql-admin@lists.postgresql.org; Mon, 20 Apr 2020 19:38:14 +0000 Received: from mout.perfora.net ([74.208.4.194]) by magus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1jQcFA-0000SX-3X for pgsql-admin@postgresql.org; Mon, 20 Apr 2020 19:38:14 +0000 Received: from [192.168.1.143] ([98.169.208.7]) by mrelay.perfora.net (mreueus004 [74.208.5.2]) with ESMTPSA (Nemesis) id 1MnqT8-1ity7W3T9m-00pQ5A; Mon, 20 Apr 2020 21:37:52 +0200 Subject: Re: checkpoint process use too much memory To: Scott Ribe Cc: jian xu , Laurenz Albe , "pgsql-admin@postgresql.org" References: <5193b00d175f2e06a171ba375f1ef723e1700de4.camel@cybertec.at> From: MichaelDBA Message-ID: Date: Mon, 20 Apr 2020 15:37:48 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 PostboxApp/7.0.12 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/alternative; boundary="------------DAD4202D0C6B1543EEC89FAE" Content-Language: en-US X-Provags-ID: V03:K1:+x99f33UzFR9yXccxKPK3czC2VDErUTo61fVd3Th4RnG1qz+9Uy As6Q9qcUnFm+VshCEvtnzqmerVlkT6gFhM79k1FvduyHg7Zddrslwki0E7wtZIyPugu4+wc zmDBUgT3wgsxngBm8dkRY0w9Ldg5Q1FbIGTp+PMlrv10Qpgn9cRQAGwG+gnPfB6HZCOcXV1 KEbBmISd2SBWTJxoohdDQ== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:q0qpcQ+zSq4=:ezuZY9q6inWmHU1sWbD7/v eModE0SAVyWD6tluhH9yjGZzjTwtI3+UhH6pGEdswiGCDO+CBTj1JOScBrOkal/2QhmWZSNqZ wvsyvoDrf92m1do5zxcRfbOQpDuzr9NL9OsRuv9DhQYpNlepnmRyU2x9AJ68EeXrHdBH9hJaK 3tmjX9Cm0s/fZIMeNwSjPzOqR3IlLWeq/xLBHuNfRVl+Bq3Tyo0W0UXELPbdOTZzkGf9kj+EX Rv8b2E3cZI8qu7e32AGpem4ks/zKLf4fRz9Xe8QPgLpQLHSomRRvv+exB/f9EA70yhYd5jr0u m4CWlKretvGZ/ssa/uENAaC7+GnRn6rWljBGjrFVa7YP/j86e/wcixi+bPMozI3ThY+v1Wdt3 1fLTNcihIGdgzu5tCZL+DpYW1hH8bO8p2bi1IvSbvlEkQYBSCk8iyrI0/Ab0wsblNZssFPBvR 98IYSwfqY1a+HgMKjIzDNyiE2U8rWft/dpV3mkQrGgGGW5gq7MPM6TXng6Wfx5Sj5klPQNmCv vd5gJaAPXbm7CvA1MqtNSQJg+jMBZqo1k+7+qig9ZLsc44VGif+l92nAeM4hXVRfS1iwOekBb gYqxxgdpLtADRB1dqvhK8DViGjx2UZl7zrIN9DQ+Wp4DUoQphIbVQ56YoNtWZlJi3caAxr3hV 0qR/uVUJF3uUb7YR0MNmP4k3NXZImtF1Z2nGeYdASszIAIgP+plZ+BSc58rdYAp1/oLkRsery vbLLxMFHnIKp3EBXCvj5DC9BWNloXdBDXAnijsrxDbQdwmbBBJRjTpVNErbzgAdwe3OVLmU0o TPq0yXxbL/S8+ZG8Rby4Yd+si58WtXk4trNhaf3Vz3nSkuTjTOdokeqGsDYd+Bh72YkG3p3 List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk This is a multi-part message in MIME format. --------------DAD4202D0C6B1543EEC89FAE Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit You said: "the work_mem is 128MB, so they could use mostly 25GB memory," But work_mem is sometimes very deceiving based on types of queries executed.  You may need many work_mem buffers for a single query. So 8 buffers used = 1GB.  So it could conceivable surpass 25GB easily.  I would do some explaining on your SQL workload. Regards, Michael Vitale Scott Ribe wrote on 4/20/2020 3:23 PM: >> On Apr 20, 2020, at 1:13 PM, jian xu wrote: >> >> Do you mean the file cache will be counted in postgres checkpointer memory usage? > No, it won't be counted as PG memory. I just mean that free memory low is not a sign of a problem, when it's file cache. > > --------------DAD4202D0C6B1543EEC89FAE Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit You said: "the work_mem is 128MB, so they could use mostly 25GB memory,"

But work_mem is sometimes very deceiving based on types of queries executed.  You may need many work_mem buffers for a single query. So 8 buffers used = 1GB.  So it could conceivable surpass 25GB easily.  I would do some explaining on your SQL workload.

Regards,
Michael Vitale



Scott Ribe wrote on 4/20/2020 3:23 PM:
On Apr 20, 2020, at 1:13 PM, jian xu <jamesxu@outlook.com> wrote:

Do you mean the file cache will be counted in postgres checkpointer memory usage?

No, it won't be counted as PG memory. I just mean that free memory low is not a sign of a problem, when it's file cache.

--------------DAD4202D0C6B1543EEC89FAE--