Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lSJFO-0004mH-Gl for pgsql-admin@arkaria.postgresql.org; Fri, 02 Apr 2021 12:49:58 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1lSJFN-0000fO-62 for pgsql-admin@arkaria.postgresql.org; Fri, 02 Apr 2021 12:49:57 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lSJFM-0000fH-SZ for pgsql-admin@lists.postgresql.org; Fri, 02 Apr 2021 12:49:56 +0000 Received: from mout.perfora.net ([74.208.4.196]) by magus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lSJFJ-0000vk-St for pgsql-admin@lists.postgresql.org; Fri, 02 Apr 2021 12:49:56 +0000 Received: from [192.168.1.165] ([98.169.214.254]) by mrelay.perfora.net (mreueus003 [74.208.5.2]) with ESMTPSA (Nemesis) id 0MWD5b-1l8KZv3fj7-00XNBd; Fri, 02 Apr 2021 14:49:47 +0200 Subject: Re: work_mem To: SASIKUMAR Devaraj Cc: holger@jakobs.com, pgsql-admin@lists.postgresql.org References: <04D84259-1B75-4648-949F-78677916194E@jakobs.com> <66290583.2329958.1617367063840@mail.yahoo.com> <1654783699.2333997.1617367552583@mail.yahoo.com> From: MichaelDBA Message-ID: <592e5895-8dbd-1a4f-a87c-af72ac8eea02@sqlexec.com> Date: Fri, 2 Apr 2021 08:49:43 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 PostboxApp/7.0.47 MIME-Version: 1.0 In-Reply-To: <1654783699.2333997.1617367552583@mail.yahoo.com> Content-Type: multipart/alternative; boundary="------------1E1942ACF9EB12DA1D821943" Content-Language: en-US X-Provags-ID: V03:K1:VrYSG5EyGkAn7NI6r/hEiAuNgBYYQMZ9YX4gbfdaR4eJhDAOcM2 7r7viO0btmnxlwgNrCiJDlCMoPNQr1zAWdjsNQZQpZJ2QwZVYGLU35KK8byzJ8Oz/B1/9mO GWuFI7aC03yrkCadQEX1GCfUtYHSNfQBdhGrz6e0YR+Tc3ekXey7x158Eqaw475fOjNePVP mOkX6tMeZzB2SbOfaayRw== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:3jY/0EVhnI4=:b7I9V0T9G/OWDZDT2QDAN3 RahubLaeDSZZsiU7rF9OFfKL1P62faeUdWyzXMZ4+Shc/V/UTw+CbqSS8Op/k9G1fz7NqsxX+ mcY4ZizpBZoZILm7XM9cEPIlXG5klYmOS/KAQDpoNs4OAT5ILTjUQI/8/QkzJt4dYqlu7HiP/ c2Nn5KLqw90NJcHqG4M/pb6QXG90rUsOx4HFiQR8+h9cHEmKpK3YBwNDuS6gaeSZX0kTDGEyG KuBsk5sAaPBTpXhkJvpjGzHWYoR+8ii1sd9YHVA6hRlvrACk4F1t5hEmm4V2134whQNP21i8B UbYWumPNVdtnf32Q+Pxw2H6NuFx6PHYSEld1UQKU/+yf+ugWn+6426rfYpgUx+wwyY6+vtSJN 98POZP0BeTs/K6BPcmUlVyymmXmu3GE1sHf2eaXeW+xGhJDF9dYwOs6fsDuL8FvqyIn5bDen4 JAVLzu7hrKbbc459708EYdmuH24B1LIOmNw0QiHiK3TPHf8C9AyK List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk This is a multi-part message in MIME format. --------------1E1942ACF9EB12DA1D821943 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit That is a common misconception.  It is not one work_mem buffer per SQL, but one work_mem buffer per required operation within that SQL.  So you can have manner work_mem buffers per SQL statement! Right from the official docs: work_mem sets the maximum amount of memory to be used by a query operation (such as a sort or hash table) before writing to temporary disk files. If this value is specified without units, it is taken as kilobytes...Note that for a complex query, several sort or hash operations might be running in parallel; each operation will be allowed to use as much memory as this value specifies before it starts to write data into temporary files. Also, several running sessions could be doing such operations concurrently. Therefore, the total memory used could be many times the value of|work_mem|; it is necessary to keep this fact in mind when choosing the value. Sort operations are used for|ORDER BY|,|DISTINCT|, and merge joins. Hash tables are used in hash joins, hash-based aggregation, and hash-based processing of|IN|subqueries. Regards, Michael Vitale SASIKUMAR Devaraj wrote on 4/2/2021 8:45 AM: > Thanks Michael > > For example work_mem is 4M and if I had 300 connections connected to > dB the total memory requirement is 1.2 Gb. >  So as per my understanding this 1.2 gb is not allocated as soon as > 300 connections established, but it may vary from 0 to 1.2gb as per > operations from client. Please confirm > > Regards > > Sasi > > Sent from Yahoo Mail on Android > > > On Fri, Apr 2, 2021 at 6:09 PM, MichaelDBA > wrote: > Memory is allocated dynamically per internal work_mem buffer > requests. > > SASIKUMAR Devaraj wrote on 4/2/2021 8:37 AM: > Hi All > > As soon as client session is established work_mem will be > allocated or only when sort happens for that particular session? > Please advise how the internal behavior? This will help me to > configure my database memory with high connections > > Regards > > Sasi > > Sent from Yahoo Mail on Android > > > On Wed, Mar 31, 2021 at 8:52 PM, Holger Jakobs > wrote: > You can show any or all settings with the command SHOW. > > show work_mem; > show all; > > How many times the amount of work_mem has been allocated > currently cannot be shown this way. Any sort operation running > currently allocates it. So it is not identical with the number > of current queries, because queries can allocate it several times. > > > Am 31. März 2021 16:04:07 MESZ schrieb "Campbell, Lance" > : > > PostgreSQL 12 > > Is there a query that will tell us at any given time what > amount of PostgreSQL memory is being used for work_mem? > > Thanks, > > *LANCE CAMPBELL * > > Web Services > > Public Affairs > > /var/folders/wp/1f6l7hw95y718z976kgnl5f9kr5rtc/T/com.microsoft.Outlook/WebArchiveCopyPasteTempFiles/signature_logo.png > > > /Under the Illinois Freedom of Information Act any written > communication to or from university employees regarding > university business is a public record and may be subject > to public disclosure./ > > > -- > Holger Jakobs, Bergisch Gladbach > +49 178 9759012 > - sent from mobile, therefore short - > > --------------1E1942ACF9EB12DA1D821943 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit That is a common misconception.  It is not one work_mem buffer per SQL, but one work_mem buffer per required operation within that SQL.  So you can have manner work_mem buffers per SQL statement!

Right from the official docs:
work_mem sets the maximum amount of memory to be used by a query operation (such as a sort or hash table) before writing to temporary disk files. If this value is specified without units, it is taken as kilobytes... Note that for a complex query, several sort or hash operations might be running in parallel; each operation will be allowed to use as much memory as this value specifies before it starts to write data into temporary files. Also, several running sessions could be doing such operations concurrently. Therefore, the total memory used could be many times the value of work_mem; it is necessary to keep this fact in mind when choosing the value. Sort operations are used for ORDER BY, DISTINCT, and merge joins. Hash tables are used in hash joins, hash-based aggregation, and hash-based processing of IN subqueries.

Regards,
Michael Vitale

SASIKUMAR Devaraj wrote on 4/2/2021 8:45 AM:
Thanks Michael 

For example work_mem is 4M and if I had 300 connections connected to dB the total memory requirement is 1.2 Gb.
 So as per my understanding this 1.2 gb is not allocated as soon as 300 connections established, but it may vary from 0 to 1.2gb as per operations from client. Please confirm 

Regards

Sasi


On Fri, Apr 2, 2021 at 6:09 PM, MichaelDBA
Memory is allocated dynamically per internal work_mem buffer requests.

SASIKUMAR Devaraj wrote on 4/2/2021 8:37 AM:
Hi All

As soon as client session is established work_mem will be allocated or only when sort happens for that particular session? Please advise how the internal behavior? This will help me to configure my database memory with high connections

Regards

Sasi


On Wed, Mar 31, 2021 at 8:52 PM, Holger Jakobs
You can show any or all settings with the command SHOW.

show work_mem;
show all;

How many times the amount of work_mem has been allocated currently cannot be shown this way. Any sort operation running currently allocates it. So it is not identical with the number of current queries, because queries can allocate it several times.


Am 31. März 2021 16:04:07 MESZ schrieb "Campbell, Lance" <lance@illinois.edu>:

PostgreSQL 12

 

Is there a query that will tell us at any given time what amount of PostgreSQL memory is being used for work_mem?  

 

Thanks,

 

LANCE CAMPBELL

 

Web Services

Public Affairs

 

 

/var/folders/wp/1f6l7hw95y718z976kgnl5f9kr5rtc/T/com.microsoft.Outlook/WebArchiveCopyPasteTempFiles/signature_logo.png

 

Under the Illinois Freedom of Information Act any written communication to or from university employees regarding university business is a public record and may be subject to public disclosure.

 


--
Holger Jakobs, Bergisch Gladbach
+49 178 9759012
- sent from mobile, therefore short -


--------------1E1942ACF9EB12DA1D821943--