Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1qfal0-00CCqU-O8 for pgsql-hackers@arkaria.postgresql.org; Mon, 11 Sep 2023 06:50:51 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1qfakz-00E7D2-1i for pgsql-hackers@arkaria.postgresql.org; Mon, 11 Sep 2023 06:50:48 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1qfakx-00E7Cl-Vq for pgsql-hackers@lists.postgresql.org; Mon, 11 Sep 2023 06:50:48 +0000 Received: from mail-os0jpn01olkn2071.outbound.protection.outlook.com ([40.92.98.71] helo=JPN01-OS0-obe.outbound.protection.outlook.com) by makus.postgresql.org with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1qfaku-003v7s-2d for pgsql-hackers@postgresql.org; Mon, 11 Sep 2023 06:50:46 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WFgNvYrqR5UJyShmCHF8vJ3i0Ic/Oef/Q2+lsn/flQX/M2/TfDk4BNCTu8lhWXSQDc9WDvhnKgpLzkT7jIekZfg2Btz4QjebLRl3pveBSk6KQVOAs5ZXI7ricq0v02B2ej2wWRWJaUgVcuAHSAnWvyI7IhwMtoQvK6Bgh1G+yfBfC3ACZ4N+TTdmWov5jIssz2jVLqHkXrlfiMZqEcvES7NIQMAGoGb9xrNrR0Wgg3tgb98yVPpQG30qR8eum8xSim04Nr9emSmSceFR5mwM7ZQzlTuq71huOSFDMWLQnBL/y3Z5gZqjMyPdBPzR1Bym2+Eh7roQokCcJebsGWd0vw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=t5wyaoilkiVJhZbfF15S5hBiJWGdGEz43FIFDrICPAM=; b=Yg4LrtW3oZWspg6jc5axprMSMfuf2QGbsocJ2sEdmggpunxOh7cNaLM/uHMM20Vn4sgSL4re4dGHkV+G3Di3uNMX34SQRjHfA1tukpjjGFYQtlbXERuFtlus03fxy7iAFpjE72/0SEDmS41M/CiAFZlI7hf0xGW97EHQLCjPGe1NPscqEmgJBJ5g/8NQfsqB7AZ9tF4UrUePebzswuRhCExpnGYgdXZQPKzuPXiIG+6poAinSD09v4nvsGFTVIkWLIfptUoQ0OaHcKaHa7VOPIPRNeY5DgjmI/QIvxuwuT5Nnf96ryxMUWanPPwA9Y8JZ6qXZTslLxTeFMoXJ+Da8A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=t5wyaoilkiVJhZbfF15S5hBiJWGdGEz43FIFDrICPAM=; b=dbbZiSegQj4PUoxGNB1WD3JyHijbo9sG0PMGE+mvPFJoP64+LNTXi+RyxKAuiMsLM7xstp6Nw34xAlDL5wIbq+Hu/blCHwg21C2Mu8YMfTGIHZDDm+Yg41Ku7ss1rJnOWh31V+5L2aYs0ZC+MoCMkJJnEplnZDk8K8QK7nD/Swwzq8LDdO0Hetej6kXG1PP4NkYfLN0sH42X2qify448NWdugwkO/hvU2k9SYeA1ymANySr9drNNDSQVkXWmxYmVpbnJBt1Hrg8ixthNovatxTmi0Pu+2nOkXAPZbLJslI+JjP/aXhCsvhLraKr3V4xJliR6cWQ3MnTmBSIdTaondQ== Received: from TYBP286MB0351.JPNP286.PROD.OUTLOOK.COM (2603:1096:404:8026::12) by OSZP286MB2393.JPNP286.PROD.OUTLOOK.COM (2603:1096:604:15b::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6792.16; Mon, 11 Sep 2023 06:50:39 +0000 Received: from TYBP286MB0351.JPNP286.PROD.OUTLOOK.COM ([fe80::1242:7aa6:fa74:b568]) by TYBP286MB0351.JPNP286.PROD.OUTLOOK.COM ([fe80::1242:7aa6:fa74:b568%3]) with mapi id 15.20.6792.016; Mon, 11 Sep 2023 06:50:39 +0000 From: Hongxu Ma To: "David G. Johnston" CC: PostgreSQL Hackers Subject: Re: PSQL error: total cell count of XXX exceeded Thread-Topic: PSQL error: total cell count of XXX exceeded Thread-Index: AQHZ18uXSycftmOWqUSMUNXSYnmSXa/79p8AgABzd+2AGN2xFQ== Date: Mon, 11 Sep 2023 06:50:39 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US, zh-HK, zh-CN Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: msip_labels: x-tmn: [f3XrhsQ7zmPbkfgFxytLEATeImm7/z0D] x-ms-publictraffictype: Email x-ms-traffictypediagnostic: TYBP286MB0351:EE_|OSZP286MB2393:EE_ x-ms-office365-filtering-correlation-id: 485e2b02-ec42-4563-3d98-08dbb2936949 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: y3eb6GE5YbbTVG5LGrg1fi8dE4smXam4Wfn7CT01mF9/T7ihtbP2mF7rVIYI2NIkrhyeqPcuVFabDitlPd6lwJGXSFFlob76VYBmviPwNtBGfbA6aIW0izua4PKD23Cru7E/9Zi3533N53Wr20GzFzKDcy91VeBKb3+cc4T2Qo/0waMaN8RzgOU3WXsIz+WKQYyVavxEoEvytsF3qDtylCHjpoJ+FFIe7rbMRYyQBn+RTg6zPHovEBaZdZACmH54mXq+kCMNqWeBEqyF7WRfeH2j7cbrfFuSJ8e+6lEGGKscnzR3+cNthDYaPDjHSm4GmfpcoZhBox9ESqvM0thr8qfLoQPLj/c/ygd11qoanrLtX9FdufftER+cFeCsV27CyV8oqc82p8UYAmT9aIJRd4Stlc9FNohwo0ZnRA6hlDzzUc3I5p0ZXNwXL7tA9p4A2sz9lVHA2Q6z6cyqIN41mR3H+RmUA1OGeo+a+0RWzKtPXAEDZvGks/wJXP45OBgHB/wGA5zN0g0I0CDJCMytJXQl6WFQH/7MNILzY1TP98zibfab2Ta2/omI16oceyGA x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?Windows-1252?Q?XSSJk78S9CziRU3Y011S8v4LwpaRSUWt6vhgPVQUfuPzSfA3IIwDetmo?= =?Windows-1252?Q?vIgXo1qhNcAakyshJJInDFzkuVWk/evQKe0ZbXuX6e6BgNOK8C2E/94o?= =?Windows-1252?Q?32sQBkEI+rKTf8GRQspnzdGuRdNx7aCb8oSKWA6rFStpBY88NwVsuQ4D?= =?Windows-1252?Q?w1LTvqk7gUci6mbWFOyLCC6a04Z5R/mS3v3n6dsmcukYoE50EeyftqcF?= =?Windows-1252?Q?U3WfPb0D1i39ZMPz6mQO9Toxqv4vQwm24x3zfkinB2Rt5x/LsQzT9XiP?= =?Windows-1252?Q?lsfMXBf1csuSVWuBadYKbrbbmTqvuTF2gjlPlX1xVj5cqAkRwUf0fLM8?= =?Windows-1252?Q?I/pa8SSoQy8NaNvkaTb8/xTnrszKZRV0vAn3s/3FdaEeZLIor4fuk+ff?= =?Windows-1252?Q?RReieBNutrNkI6zbIrMYRPaKMa3QtyMcIopz1qEtDPM1GA6ZtbWXoCJf?= =?Windows-1252?Q?+JZuDa72RhlaM9VrpB58xP6r5oSI5i/WWnBFtqLyC0pJqbv4VQnKC9GD?= =?Windows-1252?Q?CS1SPPWr3qbJpqDpAmRRYWOLRetCNtx2lbGgYdnJi9Y6pKQuPxBQeRtd?= =?Windows-1252?Q?4K37u3QJEu5nebnFOFC62ElugwYYkCOoIdBhOxqB252GOlwnJlR9I+eZ?= =?Windows-1252?Q?LkemeXp1vhw3h3g23UcS6lJ+QWP7Mmk9THv5NREUK7JZFRsHLeROGntj?= =?Windows-1252?Q?yze31LYJIuzgwPDHh56hBGv/XD3NzEqr8AETJMbnN45qaWB59npwHnmG?= =?Windows-1252?Q?AZGO8maB6XEPNJ3ZgdqV9OPwx72/awfdoR88LGVm4MwUTtwlJHEpvFSs?= =?Windows-1252?Q?aww35bFW17MxbKsxlQJrx0WW+68IYv1q8zvDX1ickuu9GtX4q2ywQKM5?= =?Windows-1252?Q?blGLxrHSsNaLH1e69MudOq81qWxWQBqOFShMgqNHJNwCN7+t194Xc+mu?= =?Windows-1252?Q?7DrTPdwsZj3cmyKST/FFgoyhhFiKGtR+9WBo4/DaCCcDnDQw3Poqw80k?= =?Windows-1252?Q?6asz4ixY/8FnvYKmgiQPAvpCYKvfr7JtXiem68DlYWX4w+nKa8Nv2I/C?= =?Windows-1252?Q?l7G9VFnHbITVKvhiiFEXtvLdiBkgHQ1ac3herbiBbSQobE+lHtSiKIvH?= =?Windows-1252?Q?K+XWJmfjA/Qr0kGv3jEm0Lt/orFyBZGU6ymBKEd45frDg72foIagMyTJ?= =?Windows-1252?Q?eZKW6Xbx4wbWoyefO7gGqLWb3e53UpRmXgGrPkIv30iVq6iGKF872Ymn?= =?Windows-1252?Q?4g87X69pyE4gFGk4NamY5UiuwZ0XUL3TmJvvpShW?= Content-Type: multipart/mixed; boundary="_004_TYBP286MB03510C21D207F77D80CF6235B4F2ATYBP286MB0351JPNP_" MIME-Version: 1.0 X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: TYBP286MB0351.JPNP286.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: 485e2b02-ec42-4563-3d98-08dbb2936949 X-MS-Exchange-CrossTenant-originalarrivaltime: 11 Sep 2023 06:50:39.9029 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: OSZP286MB2393 List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --_004_TYBP286MB03510C21D207F77D80CF6235B4F2ATYBP286MB0351JPNP_ Content-Type: multipart/alternative; boundary="_000_TYBP286MB03510C21D207F77D80CF6235B4F2ATYBP286MB0351JPNP_" --_000_TYBP286MB03510C21D207F77D80CF6235B4F2ATYBP286MB0351JPNP_ Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable I created a patch to fix it. Really appreciate to anyone can help to review it. Thanks. ________________________________ From: Hongxu Ma Sent: Saturday, August 26, 2023 19:19 To: David G. Johnston Cc: PostgreSQL Hackers Subject: Re: PSQL error: total cell count of XXX exceeded Thank you David. From the code logic, I don't think this check is meant to check the limit: If it enters the double-loop (cont.nrows * cont.ncolumns) in printQuery(), = the check should be always false (except overflow happened). So, if want to= check the limit, we could have done this check before the double-loop: jus= t checking PGresult and reports error earlier. > I wouldn=92t be adverse to an improved error message, and possibly docume= nting said limit. Agreed with you, current error message may even report a negative value, it= 's very confusing for user. It's better to introduce a limit here. Or using= a bigger integer type (e.g. long) for them, but it's also have the theoret= ical upbound. Thanks. ________________________________ From: David G. Johnston Sent: Saturday, August 26, 2023 12:09 To: Hongxu Ma Cc: PostgreSQL Hackers Subject: Re: PSQL error: total cell count of XXX exceeded On Friday, August 25, 2023, Hongxu Ma > wrote: When I tried to select a big amount of rows, psql complains a error "Cannot= add cell to table content: total cell count of 905032704 exceeded." We should use long for ncolumns and nrows and give a more obvious error me= ssage here. Any thoughts? or some other hidden reasons? 9 millions cells seems more than realistic a limit for a psql query result = output. In any case it isn=92t a bug, the code demonstrates that fact by p= roducing an explicit error. I wouldn=92t be adverse to an improved error message, and possibly document= ing said limit. David J. --_000_TYBP286MB03510C21D207F77D80CF6235B4F2ATYBP286MB0351JPNP_ Content-Type: text/html; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable
I created a patch to fix it.
Really appreciate to anyone can help to review it.
Thanks.

From: Hongxu Ma <in= terma@outlook.com>
Sent: Saturday, August 26, 2023 19:19
To: David G. Johnston <david.g.johnston@gmail.com>
Cc: PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Subject: Re: PSQL error: total cell count of XXX exceeded
 
Thank you David.

From the code logic, I don't think this check is meant to check the limit:<= /div>
So, if want to check the limit, we could have done this check before= the double-loop: just checking PGresult and reports error earlier.

> I would= n=92t be adverse to an improved error message, and possibly documenting sai= d limit.

Agreed with you, current error message may even report a negative value, it= 's very confusing for user. It's better to introduce a limit here. Or using= a bigger integer type (e.g. long) for them, but it's also have the theoretical upbound.

Thanks.

From: David G. Johns= ton <david.g.johnston@gmail.com>
Sent: Saturday, August 26, 2023 12:09
To: Hongxu Ma <interma@outlook.com>
Cc: PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Subject: Re: PSQL error: total cell count of XXX exceeded
 
On Friday, August 25, 2023, Hongxu Ma <interma@outlook.com> wrote:

When I tried to select a big amount of rows, psql complains a error "<= span style=3D"display: inline !important; background-color: rgb(255, 255, 2= 55);">Cannot add cell to table content: total cell count of 905032704 excee= ded."

 = We should use long for ncolumns = and nrows and give a more obviou= s error message here.

Any thoughts? or some other hidden reasons?<= /span>

9 millions cells seems more than realistic a limit for a psql query re= sult output.  In any case it isn=92t a bug, the code demonstrates that= fact by producing an explicit error.

I wouldn=92t be adverse to an improved error message, and possibly doc= umenting said limit.

David J.

--_000_TYBP286MB03510C21D207F77D80CF6235B4F2ATYBP286MB0351JPNP_-- --_004_TYBP286MB03510C21D207F77D80CF6235B4F2ATYBP286MB0351JPNP_ Content-Type: application/octet-stream; name="0001-Using-long-type-in-printTableAddCell.patch" Content-Description: 0001-Using-long-type-in-printTableAddCell.patch Content-Disposition: attachment; filename="0001-Using-long-type-in-printTableAddCell.patch"; size=1592; creation-date="Mon, 11 Sep 2023 06:47:41 GMT"; modification-date="Mon, 11 Sep 2023 06:48:04 GMT" Content-Transfer-Encoding: base64 RnJvbSAxZTMzNjg4OTVlNzJjYzFhYjU5ZWZlY2M2YjVlOTNhNzc1MjYwOGM3IE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiBpbnRlcm1hIDxpbnRlcm1hQG91dGxvb2suY29tPgpEYXRlOiBN b24sIDExIFNlcCAyMDIzIDE0OjQyOjE0ICswODAwClN1YmplY3Q6IFtQQVRDSF0gVXNpbmcgbG9u ZyB0eXBlIGluIHByaW50VGFibGVBZGRDZWxsKCkgdG8gcHJldmVudCBpbnQKIG92ZXJmbG93Cgot LS0KIHNyYy9mZV91dGlscy9wcmludC5jIHwgMTIgKysrKysrKy0tLS0tCiAxIGZpbGUgY2hhbmdl ZCwgNyBpbnNlcnRpb25zKCspLCA1IGRlbGV0aW9ucygtKQoKZGlmZiAtLWdpdCBhL3NyYy9mZV91 dGlscy9wcmludC5jIGIvc3JjL2ZlX3V0aWxzL3ByaW50LmMKaW5kZXggN2FmMWNjYjZiNS4uNGQ2 YTllMWQ0ZiAxMDA2NDQKLS0tIGEvc3JjL2ZlX3V0aWxzL3ByaW50LmMKKysrIGIvc3JjL2ZlX3V0 aWxzL3ByaW50LmMKQEAgLTMyNDksMTUgKzMyNDksMTcgQEAgdm9pZAogcHJpbnRUYWJsZUFkZENl bGwocHJpbnRUYWJsZUNvbnRlbnQgKmNvbnN0IGNvbnRlbnQsIGNoYXIgKmNlbGwsCiAJCQkJICBj b25zdCBib29sIHRyYW5zbGF0ZSwgY29uc3QgYm9vbCBtdXN0ZnJlZSkKIHsKKwlsb25nIHByb2R1 Y3Q7CiAjaWZuZGVmIEVOQUJMRV9OTFMKIAkodm9pZCkgdHJhbnNsYXRlOwkJCS8qIHVudXNlZCBw YXJhbWV0ZXIgKi8KICNlbmRpZgotCi0JaWYgKGNvbnRlbnQtPmNlbGxzYWRkZWQgPj0gY29udGVu dC0+bmNvbHVtbnMgKiBjb250ZW50LT5ucm93cykKKwkvKiBwcm9kdWN0IG9mIGNvbHMgYW5kIHJv d3MsIHVzaW5nIGxvbmcgdHlwZSB0byBwcmV2ZW50IGludCBvdmVyZmxvdyAqLworCXByb2R1Y3Qg PSAobG9uZyljb250ZW50LT5uY29sdW1ucyAqIChsb25nKWNvbnRlbnQtPm5yb3dzOworCWlmIChj b250ZW50LT5jZWxsc2FkZGVkID49IHByb2R1Y3QpCiAJewogCQlmcHJpbnRmKHN0ZGVyciwgXygi Q2Fubm90IGFkZCBjZWxsIHRvIHRhYmxlIGNvbnRlbnQ6ICIKLQkJCQkJCSAgInRvdGFsIGNlbGwg Y291bnQgb2YgJWQgZXhjZWVkZWQuXG4iKSwKLQkJCQljb250ZW50LT5uY29sdW1ucyAqIGNvbnRl bnQtPm5yb3dzKTsKKwkJCQkJCSAgInRvdGFsIGNlbGwgY291bnQgb2YgJWxkIGV4Y2VlZGVkLCBj ZWxscyBhZGRlZDogJWxkLlxuIiksCisJCQkJcHJvZHVjdCwgY29udGVudC0+Y2VsbHNhZGRlZCk7 CiAJCWV4aXQoRVhJVF9GQUlMVVJFKTsKIAl9CiAKQEAgLTMyNzMsNyArMzI3NSw3IEBAIHByaW50 VGFibGVBZGRDZWxsKHByaW50VGFibGVDb250ZW50ICpjb25zdCBjb250ZW50LCBjaGFyICpjZWxs LAogCXsKIAkJaWYgKGNvbnRlbnQtPmNlbGxtdXN0ZnJlZSA9PSBOVUxMKQogCQkJY29udGVudC0+ Y2VsbG11c3RmcmVlID0KLQkJCQlwZ19tYWxsb2MwKChjb250ZW50LT5uY29sdW1ucyAqIGNvbnRl bnQtPm5yb3dzICsgMSkgKiBzaXplb2YoYm9vbCkpOworCQkJCXBnX21hbGxvYzAoKHByb2R1Y3Qg KyAxKSAqIHNpemVvZihib29sKSk7CiAKIAkJY29udGVudC0+Y2VsbG11c3RmcmVlW2NvbnRlbnQt PmNlbGxzYWRkZWRdID0gdHJ1ZTsKIAl9Ci0tIAoyLjM5LjIgKEFwcGxlIEdpdC0xNDMpCgo= --_004_TYBP286MB03510C21D207F77D80CF6235B4F2ATYBP286MB0351JPNP_--