Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lSSNn-0003ED-Rx for pgsql-admin@arkaria.postgresql.org; Fri, 02 Apr 2021 22:35:16 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1lSSNl-0005qS-5a for pgsql-admin@arkaria.postgresql.org; Fri, 02 Apr 2021 22:35:13 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lSSNk-0005qI-Po for pgsql-admin@lists.postgresql.org; Fri, 02 Apr 2021 22:35:12 +0000 Received: from mail-ot1-x32f.google.com ([2607:f8b0:4864:20::32f]) by magus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1lSSNd-0005u9-8K for pgsql-admin@lists.postgresql.org; Fri, 02 Apr 2021 22:35:11 +0000 Received: by mail-ot1-x32f.google.com with SMTP id t23-20020a0568301e37b02901b65ab30024so6106509otr.4 for ; Fri, 02 Apr 2021 15:35:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-language; bh=BjKYq0dzK3yxubhLKwM465Kn+s4qy1mYGwEHBub2XtU=; b=YyVvBZvGnY3cpTuIgkK+MBaRmS1+rQfni1nRAwDn1otJNkHWX2xXI1y5g/+CeYhIzo NAFkhePO1hU+4M6OieczLUst0/Wpf1pLzGLiKQVz56LosNs8MqHkV4sKkT5O58i2M8Ed VIUYAbqYnpo09XUYk9qjbh59TZvf2dXfuQoCdlF9v7+WGeMEfgTF1xh1ANcq8oWYE/DG cSXxFgxVApwai1fqYQybdwFmu5ZX0BE8cHRwottwl429EZgyB0nrXCvEcSvG4sMwcnls FK21mti9li3XD5yn12KGh6RPI/STnkS5KJICarU4nf+MVCwURG2kTuVXuM5RuAY0xJ65 SxJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language; bh=BjKYq0dzK3yxubhLKwM465Kn+s4qy1mYGwEHBub2XtU=; b=WocnzIYKtuAOep8az72O/XsYnmrxaFLpYKPQ/6fVv49mVGDRqfUAV5zls8Gd30Hr0i B5QCOoBgMEz1PdhNk0kKckX1+09F2b0BBeDhr6nJkoOBki8+1FZqVWqP3o/tj3nGv3mu MWd0s+XV7i14JyO4mZDEyzOdEmkzcMddr7KMmU1eE8/35cnRqwfQj6nWPoabg6ivycj4 DNxUNDhL0wOaXX5WHLpousAPDeHn7ljTEQj9NmOTLB84dm/D7gEuULd4kJi5ysIJXpWJ VCLRfOT7MyiITfgBxFPZdSqc9246ux0RMi8J37Du5Uky8q7Ti7KLTDJo1d/iHG4yLxBO NqTw== X-Gm-Message-State: AOAM532n3093W8tQG/S64/B03SMBH0CIV7Rvxgdq3YidId9wqS8B20+d ZthJGEiPZ578nH3eQ8pShpFv8oImL5c= X-Google-Smtp-Source: ABdhPJy9mw57wSSPQjXQBiUZ3zVLItLxpaNsGDmXfSgRuNQ/v4ZMVAVGPoEjrrt263wCmyHlwwqwvA== X-Received: by 2002:a05:6830:2469:: with SMTP id x41mr12931664otr.167.1617402903168; Fri, 02 Apr 2021 15:35:03 -0700 (PDT) Received: from [192.168.88.10] (ip174-67-141-204.no.no.cox.net. [174.67.141.204]) by smtp.googlemail.com with ESMTPSA id u7sm2014697ooq.13.2021.04.02.15.35.02 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 02 Apr 2021 15:35:02 -0700 (PDT) Subject: Global memory or local memory? (was Re: work_mem) To: pgsql-admin@lists.postgresql.org References: <04D84259-1B75-4648-949F-78677916194E@jakobs.com> <66290583.2329958.1617367063840@mail.yahoo.com> <1654783699.2333997.1617367552583@mail.yahoo.com> <592e5895-8dbd-1a4f-a87c-af72ac8eea02@sqlexec.com> From: Ron Message-ID: <852027d8-bc23-692e-8330-dc09d0f03af9@gmail.com> Date: Fri, 2 Apr 2021 17:35:01 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: <592e5895-8dbd-1a4f-a87c-af72ac8eea02@sqlexec.com> Content-Type: multipart/alternative; boundary="------------C7ADA165D93361D863A6BB8D" Content-Language: en-US List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk This is a multi-part message in MIME format. --------------C7ADA165D93361D863A6BB8D Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit On 4/2/21 7:49 AM, MichaelDBA wrote: > That is a common misconception.  It is not one work_mem buffer per SQL, > but one work_mem buffer per required operation within that SQL.  So you > can have manner work_mem buffers per SQL statement! > > Right from the official docs: > work_mem sets the maximum amount of memory to be used by a query operation > (such as a sort or hash table) before writing to temporary disk files. If > this value is specified without units, it is taken as kilobytes...Note > that for a complex query, several sort or hash operations might be running > in parallel; each operation will be allowed to use as much memory as this > value specifies before it starts to write data into temporary files. Also, > several running sessions could be doing such operations concurrently. > Therefore, the total memory used could be many times the value > of|work_mem|; it is necessary to keep this fact in mind when choosing the > value. Sort operations are used for|ORDER BY|,|DISTINCT|, and merge joins. > Hash tables are used in hash joins, hash-based aggregation, and hash-based > processing of|IN|subqueries. > > Regards, > Michael Vitale If multiple users are querying the same regions of the same tables, does each process have to read the same blocks, or can they share ("oh look, some other process has already read into memory some of the data blocks I need, so I'll just share those buffers")? -- Angular momentum makes the world go 'round. --------------C7ADA165D93361D863A6BB8D Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit On 4/2/21 7:49 AM, MichaelDBA wrote:
That is a common misconception.  It is not one work_mem buffer per SQL, but one work_mem buffer per required operation within that SQL.  So you can have manner work_mem buffers per SQL statement!

Right from the official docs:
work_mem sets the maximum amount of memory to be used by a query operation (such as a sort or hash table) before writing to temporary disk files. If this value is specified without units, it is taken as kilobytes... Note that for a complex query, several sort or hash operations might be running in parallel; each operation will be allowed to use as much memory as this value specifies before it starts to write data into temporary files. Also, several running sessions could be doing such operations concurrently. Therefore, the total memory used could be many times the value of work_mem; it is necessary to keep this fact in mind when choosing the value. Sort operations are used for ORDER BY, DISTINCT, and merge joins. Hash tables are used in hash joins, hash-based aggregation, and hash-based processing of IN subqueries.

Regards,
Michael Vitale

If multiple users are querying the same regions of the same tables, does each process have to read the same blocks, or can they share ("oh look, some other process has already read into memory some of the data blocks I need, so I'll just share those buffers")?


--
Angular momentum makes the world go 'round.
--------------C7ADA165D93361D863A6BB8D--