Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1puIBq-0002Bb-Sp for pgsql-hackers@arkaria.postgresql.org; Wed, 03 May 2023 19:31:03 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1puIAq-0000j3-IP for pgsql-hackers@arkaria.postgresql.org; Wed, 03 May 2023 19:30:00 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1puIAp-0000hg-DQ for pgsql-hackers@lists.postgresql.org; Wed, 03 May 2023 19:30:00 +0000 Received: from mail-co1nam11olkn20818.outbound.protection.outlook.com ([2a01:111:f400:7eab::818] helo=NAM11-CO1-obe.outbound.protection.outlook.com) by makus.postgresql.org with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1puIAg-000MG0-Bc for pgsql-hackers@postgresql.org; Wed, 03 May 2023 19:29:55 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=k6PQS4jgYZDIYgDV/kCPHH3qV3TGUYAODoaA9ivt2Nbh3qVGKCdNA61nLKAl/ctCC1LrDdF87+KZ8jcZ3iQgf2/AFCnamMfjMevWYlCz+caOqVHrCoVtr+tDDRkrgb9EJOFHEWtXm8GmkDd36SvUgI/CYklXS8OFfHqdLtv8I/9P9r6bSfCPC3rzFC5OE2xPas1xkh8jAn2Ozc9CS5hnufAH0dXHjbOe3y7RMitCuL4WKbzOVUJqehXTPFaa7OU3mSBX4TtkFwLIYlVTYxpVebDPUN/6eOPOEzbYwzFMwpNS/2gFFxq9QFfhcoS8HFC4zJQOfJpY4VKUR9afFTyAEQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=L5xyeisN0JJxbkO4dlw54gSQYMlmfbZehmiUIIht/KU=; b=WwZ2vUCKv4tJ6w/GI2fZ7OB4N69vM3JbhLtCSOAbBAvXad5OCdksVbrWwI6YXMbV+oEx65JAf4Jp4Afo8lTuWIMcM6eUB3EBkZYHZLKU6sEl55naYVZfez7PeLHwwqB+5obFjumr0QFEXgRfgM3rM08v9bf9jDfNZYsBnY5HqLgTGmn7EIiU8OrJetDVwN0SxSc8ZyQDirMAsOi31wUdiiqHNlawHa08bq5NABrj0+3ajWOsqmWVeTXZwfhNlwOPHlVrdNWLz23rYznux6PAkiDwoTAsKH7TObnijGR98O/CRHLcqwL9nn4ifK29QSO0u5s4C2CD5Ta4BzDbgKcr/g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=L5xyeisN0JJxbkO4dlw54gSQYMlmfbZehmiUIIht/KU=; b=s9L+J0pAVfnvkcgmnQVuT6Zy212xyCpRRDI1HhuTPBqFWT8FEp0CaKq64rK4nyDcekXNuMVkwwFgovwO7hGjKBV6y600PGEc7JLX9ITIirCe/rpqld2Ok4tzrztb2EvS2Ct3vw7muDROk/Y+hhOjChTg4WA1tEkOh+yrlOt/o450a0hzXNhmQW+J945eNnKnXuv70EjPDsfI3ev9HxMBFTAVsBlp8U+g9kozRWE3svLlLFlPiGdBLiH+yjo49JWNvh3Mnt4JXAaPgCRQB+cVfOb2hoFQNCSv9emwtkzs4GpsiQFWgIqpKloXYfTYlvopRIFWxaMY2YFGlHOKdwa7NA== Received: from SJ0PR17MB5841.namprd17.prod.outlook.com (2603:10b6:a03:40c::19) by BY5PR17MB3892.namprd17.prod.outlook.com (2603:10b6:a03:21e::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6363.22; Wed, 3 May 2023 19:29:46 +0000 Received: from SJ0PR17MB5841.namprd17.prod.outlook.com ([fe80::1604:a8bf:3b24:36f6]) by SJ0PR17MB5841.namprd17.prod.outlook.com ([fe80::1604:a8bf:3b24:36f6%6]) with mapi id 15.20.6319.032; Wed, 3 May 2023 19:29:46 +0000 From: Muhammad Malik To: Andres Freund CC: "pgsql-hackers@postgresql.org" , Thomas Munro , Melanie Plageman , Yura Sokolov , Robert Haas Subject: Re: refactoring relation extension and BufferAlloc(), faster COPY Thread-Topic: refactoring relation extension and BufferAlloc(), faster COPY Thread-Index: AQHZfeqr38lsd7fnFke17hp1/TlM8K9I21dsgAAHXYCAAAkGUA== Date: Wed, 3 May 2023 19:29:46 +0000 Message-ID: References: <20221029025420.eplyow6k7tgu6he3@awork3.anarazel.de> <20230503184419.vxwtcabkqweef2wi@awork3.anarazel.de> In-Reply-To: <20230503184419.vxwtcabkqweef2wi@awork3.anarazel.de> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: x-tmn: [AYKWtvQoyyELq/T9z/FeTs6DnwF/1hm4h9S8PZmizRI=] x-ms-publictraffictype: Email x-ms-traffictypediagnostic: SJ0PR17MB5841:EE_|BY5PR17MB3892:EE_ x-ms-office365-filtering-correlation-id: b1308878-098f-459b-0db5-08db4c0cc0d6 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: nA+fq6eMlDfd1FKsfbjBOl9FJCIKgexzT+JuIx82NRB7SY6pb4+dqV/OLwrOlXmBF+6H8r5FYVtr0hLTO1JCPOaD7wIK2CWpEDQxqDSIbluOAIo+K51qLXe9LnUSW6G1zfHE9ezk8La9Fkvrotlf6C3q8PVPgamMfQlu/bYjyayNIiG6EZ76kTZOtRc6Eip4zbR5WlniwGKVrcocnfuCYbLoggGv8VoncXMHgkKnUpNgnezLlcOmpbbDWZw2HtjWwUYX4l0yGgEb3ltp2wWuN/2DJKkyYk0WY9uM4hfsBU5aCSEaUtNxCRDNviBVuONUE3/9EkEkm+tE+/PND3mGbt0Q9uU5hldVWef5IvqC+hGX7ERnSLXwcycw+CuxWhFl64cD4wUA/lJsfO2OHabMZEe8DHYHqrHBRm3Lj8yBuN4XUi4u1y1Y5N3MCqTX0104Yw0JNpBpORfIRn9KWcVuIs3dYiIvB/YKUiYCaw7nXeQwdGArRjA9EP279D0tRpMFEfz8d7eM1BKNgLz2zOGxsAq6Q6zfvefetp1hEbp6fAAx1RKbVXfEttvbmX8gwTdXD8Jx43hKrrsUv+Je13dYi0SY7H5x6Wr3UgIHEjr1ag8zBVrZ3S1OsRK+obb5kvB1 x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?Q?EVbvKVeeVWWtaP0KH2rUHxIp81JlPXAVwtCovEyvAJR0uXARPXXk9K8i0y?= =?iso-8859-1?Q?Ip4moN8DZ4ja/20It/xcZk+SY1RK+RenebycJdwyaFreY1c/RvBjf78IG3?= =?iso-8859-1?Q?MXzAa0yAsbNz6Jhwt8/yp5D0jX1YzeBywSigfjxaF8tg3TNL6voWFeAj1J?= =?iso-8859-1?Q?/hYChljx0vfdZWOHbQFiobeKij7+3ZlvpmbP/53r0bhPKhOHoWD4DTi3Zu?= =?iso-8859-1?Q?j06OIffdNQpR8clG66eAkonQb/7Q4LE4Mf7TvYDbvExfAIm481qfBKbfBw?= =?iso-8859-1?Q?Mxn78RmpGkYVTBGPd+dq9Bl3TDgnSgvDjCVc+g05EgBZT949x03Na5Pzu6?= =?iso-8859-1?Q?uB9S4DvA9DFxL0hQLABJTxtUHBmChL50xp9CdGDx2yGfJNG0kQzlncbLe7?= =?iso-8859-1?Q?qpCFN4LjbzF6URuzAgj/HRb9lkztI8RP2sbENpzc3jCmkRg96V3Q8KlLyz?= =?iso-8859-1?Q?muJXS+zyOazQUQZgpm087vUb+PWssWTdsZWuRXdC/MqDGaIx2rWREA+2EH?= =?iso-8859-1?Q?XY7N7XOFWpe/ZS41mKAAmxloIYdcDzYlWxFx7KdDY8Kl5LOyZgG1xoBP4o?= =?iso-8859-1?Q?omUcdTh4A9CW64qXeUCUSQo6UnN42vqja8e6uRRGERsC54ykBGgckkgBbm?= =?iso-8859-1?Q?iyjQvEqGt4oG80lYT6dQuZd3rUwn/bj+P2d19K/UfLOSEEsTqnkTNe3vD+?= =?iso-8859-1?Q?ac2I7Qrx1CITm15aM8FXDhUELAsWImZorhUtfZTgvHJIpI5UMPs13rBT2Z?= =?iso-8859-1?Q?NSgFFqT9IN/a6oOa7jzJFCbG6kMtUpMfRlFSp7CKdzR7XcCIdTv5oiMEdv?= =?iso-8859-1?Q?eWhtlJ8qchK0hArl6XFwJYJ5uwbqSRddDMb3mDglQC+dZJ9QmnjXnMxiBR?= =?iso-8859-1?Q?pfL+ob3ktgI8cPCpsCMwrF/ThMQmu4Uc6EBtm13ZYjNnipNQ+axnhWIU4m?= =?iso-8859-1?Q?SgBFYTnmU16nQvQnoAgl1VurPEg0m/wzF4ZWpvwav5OUaMOenE5+lEhGkg?= =?iso-8859-1?Q?mXHM7nLjeI4+NKG0j2WHnT2WoW7YUwuX8uM1G/sETWVb26jItAS7yfe9U5?= =?iso-8859-1?Q?hhIyU1bTim7KrhY08jQ2j94ouMe62ytjUQtVjhwk3NdTQrywxHYT0U82kK?= =?iso-8859-1?Q?7770BPcIlIKJMG5CQbaBrFVzb+SKyjFy9ilFqvx2++L4a3uLfXnMIrIzVe?= =?iso-8859-1?Q?EWirPLsxxcsM9+IohwGDZmVRPFqeZnzy6ohHWiAVL9Y8BRxtVIbdX13BhD?= =?iso-8859-1?Q?7m7yM4GFSTWHkIbGAo+qnp2Bcv/GWMEJu+4oq+gsM=3D?= Content-Type: multipart/alternative; boundary="_000_SJ0PR17MB58412FDEE3DCAC0C23C062ABA66C9SJ0PR17MB5841namp_" MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-4755-11-msonline-outlook-c704e.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: SJ0PR17MB5841.namprd17.prod.outlook.com X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: b1308878-098f-459b-0db5-08db4c0cc0d6 X-MS-Exchange-CrossTenant-originalarrivaltime: 03 May 2023 19:29:46.1476 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR17MB3892 List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --_000_SJ0PR17MB58412FDEE3DCAC0C23C062ABA66C9SJ0PR17MB5841namp_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable > I use a script like: > c=3D16;psql -c 'DROP TABLE IF EXISTS copytest_0; CREATE TABLE copytest_0(= data text not null);' && time /srv/dev/build/m-opt/src/bin/pgbench/pgbench = -n -P1 -c$c -j$c -t$((1024/$c)) -f ~/tmp/copy.sql && psql -c 'TRUNCATE copy= test_0' > >[1] COPY (SELECT repeat(random()::text, 5) FROM generate_series(1, 10000= 0)) TO '/tmp/copytest_data_text.copy' WITH (FORMAT test); > >[2] COPY (SELECT repeat(random()::text, 5) FROM generate_series(1, 6*100= 000)) TO '/tmp/copytest_data_text.copy' WITH (FORMAT text); When I ran this script it did not insert anything into the copytest_0 table= . It only generated a single copytest_data_text.copy file of size 9.236MB. Please help me understand how is this 'pgbench running COPY into a single t= able'. Also what are the 'seconds' and 'tbl-MBs' metrics that were reported= . Thank you, Muhammad --_000_SJ0PR17MB58412FDEE3DCAC0C23C062ABA66C9SJ0PR17MB5841namp_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

= > I use a script like:

> c=3D16;psql -c 'DROP TABLE IF EXISTS copytest_0; CREATE TABLE cop= ytest_0(data text not null);' && time /srv/dev/build/m-opt/src/bin/= pgbench/pgbench -n -P1 -c$c -j$c -t$((1024/$c)) -f ~/tmp/copy.sql &&= ; psql -c 'TRUNCATE copytest_0'

> >[1] COPY (SELECT repeat(random()::text, 5) FROM generate_series(1,= 100000)) TO '/tmp/copytest_data_text.copy' WITH (FORMAT test);
> >[2] COPY (SELECT repeat(random()::text, 5) FROM generate_series(1,= 6*100000)) TO '/tmp/copytest_data_text.copy' WITH (FORMAT text);

When I ran this script it did not insert anything into the copytest_0 table. It only generated a single copytest_data_text.copy file of size 9.236MB. 
Please help me understand how is this 'pgbench running COPY i= nto a single table'. Also what are the 'seconds' and 'tbl-MBs' metrics that were reported.

Thank you,
Muhammad


--_000_SJ0PR17MB58412FDEE3DCAC0C23C062ABA66C9SJ0PR17MB5841namp_--