Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1hycxe-0004XL-Nw for pgsql-hackers@arkaria.postgresql.org; Fri, 16 Aug 2019 14:12:11 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1hycxd-0001pQ-KC for pgsql-hackers@arkaria.postgresql.org; Fri, 16 Aug 2019 14:12:09 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1hycxc-0001cS-UO for pgsql-hackers@lists.postgresql.org; Fri, 16 Aug 2019 14:12:09 +0000 Received: from cyclops.postgrespro.ru ([93.174.131.138] helo=mail.postgrespro.ru) by makus.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1hycxZ-0000Md-Bs for pgsql-hackers@postgresql.org; Fri, 16 Aug 2019 14:12:07 +0000 Received: from localhost (localhost [127.0.0.1]) by mail.postgrespro.ru (Postfix) with ESMTP id BDBDF21C409B; Fri, 16 Aug 2019 17:12:02 +0300 (MSK) X-Virus-Scanned: Debian amavisd-new at postgrespro.ru X-Spam-Flag: NO X-Spam-Score: 0 X-Spam-Level: X-Spam-Status: No, score=x tagged_above=-99 required=4 WHITELISTED tests=[] autolearn=unavailable Received: from [192.168.27.200] (gw.postgrespro.ru [93.174.131.141]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mail.postgrespro.ru (Postfix) with ESMTPSA id 94C0021C4099; Fri, 16 Aug 2019 17:12:02 +0300 (MSK) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=postgrespro.ru; s=mail; t=1565964722; bh=YhZoswM5MCW4G7S5IsweCzrsrrumJkpUpSFjxoxUGDU=; h=Subject:From:To:Cc:References:Date:In-Reply-To; b=LMNu344MKxJJD+JFr+h07soSc3vhRlllWSB+mpDW2HdM7KxB1pApBUsBTJD40rYu+ oaUFX0letzIaCjLUSUD+nqguU0WqbhnQFycfGO/BFw+ISutY2Zk7JIxGNkUjOthkW+ KsZBWR/P6LTENwINW5EWI1mrhD9WpcuavCb+Mr4Q= Subject: Re: Global temporary tables From: Konstantin Knizhnik To: Craig Ringer Cc: Pavel Stehule , PostgreSQL Hackers References: <73954ab7-44d3-b37b-81a3-69bdcbb446f7@postgrespro.ru> <94a19e7b-d99a-110c-8e5b-f5068682b474@postgrespro.ru> <614c1e9f-c0e6-332c-ba2b-85a7e1efb956@postgrespro.ru> <5a711f75-e999-a809-60e1-c74c8c9e7915@postgrespro.ru> <21f047d6-bb2d-950c-2aff-2fee301d5851@postgrespro.ru> <91de2094-165a-04f1-9b3d-348473394345@postgrespro.ru> <758e5293-3ccd-3a6a-8464-6373bb441994@postgrespro.ru> Message-ID: Date: Fri, 16 Aug 2019 17:12:02 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.2.1 MIME-Version: 1.0 In-Reply-To: <758e5293-3ccd-3a6a-8464-6373bb441994@postgrespro.ru> Content-Type: multipart/alternative; boundary="------------5B6A8F90C7F344AE32A9A31C" Content-Language: en-US List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk This is a multi-part message in MIME format. --------------5B6A8F90C7F344AE32A9A31C Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit I did more investigations of performance of global temp tables with shared buffers vs. vanilla (local) temp tables. 1. Combination of persistent and temporary tables in the same query. Preparation: create table big(pk bigint primary key, val bigint); insert into big values (generate_series(1,100000000),generate_series(1,100000000)); create temp table lt(key bigint, count bigint); create global temp table gt(key bigint, count bigint); Size of table is about 6Gb, I run this test on desktop with 16GB of RAM and postgres with 1Gb shared buffers. I run two queries: insert into T (select count(*),pk/P as key from big group by key); select sum(count) from T; where P is (100,10,1) and T is name of temp table (lt or gt). The table below contains times of both queries in msec: Percent of selected data 1% 10% 100% Local temp table 44610 90 47920 891 63414 21612 Global temp table 44669 35 47939 298 59159 26015 As you can see, time of insertion in temporary table is almost the same and time of traversal of temporary table is about twice smaller for global temp table when it fits in RAM together with persistent table and slightly worser when it doesn't fit. 2. Temporary table only access. The same system, but Postgres is configured with shared_buffers=10GB, max_parallel_workers = 4, max_parallel_workers_per_gather = 4 Local temp tables: create temp table local_temp(x1 bigint, x2 bigint, x3 bigint, x4 bigint, x5 bigint, x6 bigint, x7 bigint, x8 bigint, x9 bigint); insert into local_temp values (generate_series(1,100000000),0,0,0,0,0,0,0,0); select sum(x1) from local_temp; Global temp tables: create global temporary table global_temp(x1 bigint, x2 bigint, x3 bigint, x4 bigint, x5 bigint, x6 bigint, x7 bigint, x8 bigint, x9 bigint); insert into global_temp values (generate_series(1,100000000),0,0,0,0,0,0,0,0); select sum(x1) from global_temp; Results (msec): Insert Select Local temp table 37489 48322 Global temp table 44358 3003 So insertion in local temp table is performed slightly faster but select is 16 times slower! Conclusion: In the assumption then temp table fits in memory, global temp tables with shared buffers provides better performance than local temp table. I didn't consider here global temp tables with local buffers because for them results should be similar with local temp tables. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company --------------5B6A8F90C7F344AE32A9A31C Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 7bit I did more investigations of performance of global temp tables with shared buffers vs. vanilla (local) temp tables.

1. Combination of persistent and temporary tables in the same query.

Preparation:
create table big(pk bigint primary key, val bigint);
insert into big values (generate_series(1,100000000),generate_series(1,100000000));
create temp table lt(key bigint, count bigint);
create global temp table gt(key bigint, count bigint);

Size of table is about 6Gb, I run this test on desktop with 16GB of RAM and postgres with 1Gb shared buffers.
I run two queries:

insert into T (select count(*),pk/P as key from big group by key);
select sum(count) from T;

where P is (100,10,1) and T is name of temp table (lt or gt).
The table below contains times of both queries in msec:

Percent of selected data
 1%
 10%
 100%
Local temp table
 44610
90
 47920
891
 63414
21612
Global temp table
 44669
35
 47939
298
 59159
26015

As you can see, time of insertion in temporary table is almost the same
and time of traversal of temporary table is about twice smaller for global temp table
when it fits in RAM together with persistent table and slightly worser when it doesn't fit.



2. Temporary table only access.
The same system, but Postgres is configured with shared_buffers=10GB, max_parallel_workers = 4, max_parallel_workers_per_gather = 4

Local temp tables:
create temp table local_temp(x1 bigint, x2 bigint, x3 bigint, x4 bigint, x5 bigint, x6 bigint, x7 bigint, x8 bigint, x9 bigint);
insert into local_temp values (generate_series(1,100000000),0,0,0,0,0,0,0,0);
select sum(x1) from local_temp;

Global temp tables:
create global temporary table global_temp(x1 bigint, x2 bigint, x3 bigint, x4 bigint, x5 bigint, x6 bigint, x7 bigint, x8 bigint, x9 bigint);
insert into global_temp values (generate_series(1,100000000),0,0,0,0,0,0,0,0);
select sum(x1) from global_temp;

Results (msec):

 Insert
 Select
Local temp table 37489
 48322
Global temp table 44358
 3003

So insertion in local temp table is performed slightly faster but select is 16 times slower!

Conclusion:
In the assumption then temp table fits in memory, global temp tables with shared buffers provides better performance than local temp table.
I didn't consider here global temp tables with local buffers because for them results should be similar with local temp tables.


-- 
Konstantin Knizhnik
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
--------------5B6A8F90C7F344AE32A9A31C--