public inbox for [email protected]
help / color / mirror / Atom feedFrom: Hongxu Ma <[email protected]>
To: David G. Johnston <[email protected]>
Cc: PostgreSQL Hackers <[email protected]>
Subject: Re: PSQL error: total cell count of XXX exceeded
Date: Mon, 11 Sep 2023 06:50:39 +0000
Message-ID: <TYBP286MB03510C21D207F77D80CF6235B4F2A@TYBP286MB0351.JPNP286.PROD.OUTLOOK.COM> (raw)
In-Reply-To: <TYBP286MB03514675D5D9AA3F533DC987B4E2A@TYBP286MB0351.JPNP286.PROD.OUTLOOK.COM>
References: <TYBP286MB0351B057B101C90D7C1239E6B4E2A@TYBP286MB0351.JPNP286.PROD.OUTLOOK.COM>
<CAKFQuwYfypo7hMXs8a7Ccx4x2bv1XKCMu=pOoQAfq_0Fpu7YQQ@mail.gmail.com>
<TYBP286MB03514675D5D9AA3F533DC987B4E2A@TYBP286MB0351.JPNP286.PROD.OUTLOOK.COM>
I created a patch to fix it.
Really appreciate to anyone can help to review it.
Thanks.
________________________________
From: Hongxu Ma <[email protected]>
Sent: Saturday, August 26, 2023 19:19
To: David G. Johnston <[email protected]>
Cc: PostgreSQL Hackers <[email protected]>
Subject: Re: PSQL error: total cell count of XXX exceeded
Thank you David.
From the code logic, I don't think this check is meant to check the limit:
If it enters the double-loop (cont.nrows * cont.ncolumns) in printQuery(), the check should be always false (except overflow happened). So, if want to check the limit, we could have done this check before the double-loop: just checking PGresult and reports error earlier.
> I wouldn’t be adverse to an improved error message, and possibly documenting said limit.
Agreed with you, current error message may even report a negative value, it's very confusing for user. It's better to introduce a limit here. Or using a bigger integer type (e.g. long) for them, but it's also have the theoretical upbound.
Thanks.
________________________________
From: David G. Johnston <[email protected]>
Sent: Saturday, August 26, 2023 12:09
To: Hongxu Ma <[email protected]>
Cc: PostgreSQL Hackers <[email protected]>
Subject: Re: PSQL error: total cell count of XXX exceeded
On Friday, August 25, 2023, Hongxu Ma <[email protected]<mailto:[email protected]>> wrote:
When I tried to select a big amount of rows, psql complains a error "Cannot add cell to table content: total cell count of 905032704 exceeded."
We should use long for ncolumns and nrows and give a more obvious error message here.
Any thoughts? or some other hidden reasons?
9 millions cells seems more than realistic a limit for a psql query result output. In any case it isn’t a bug, the code demonstrates that fact by producing an explicit error.
I wouldn’t be adverse to an improved error message, and possibly documenting said limit.
David J.
Attachments:
[application/octet-stream] 0001-Using-long-type-in-printTableAddCell.patch (1.6K, ../TYBP286MB03510C21D207F77D80CF6235B4F2A@TYBP286MB0351.JPNP286.PROD.OUTLOOK.COM/3-0001-Using-long-type-in-printTableAddCell.patch)
download | inline diff:
From 1e3368895e72cc1ab59efecc6b5e93a7752608c7 Mon Sep 17 00:00:00 2001
From: interma <[email protected]>
Date: Mon, 11 Sep 2023 14:42:14 +0800
Subject: [PATCH] Using long type in printTableAddCell() to prevent int
overflow
---
src/fe_utils/print.c | 12 +++++++-----
1 file changed, 7 insertions(+), 5 deletions(-)
diff --git a/src/fe_utils/print.c b/src/fe_utils/print.c
index 7af1ccb6b5..4d6a9e1d4f 100644
--- a/src/fe_utils/print.c
+++ b/src/fe_utils/print.c
@@ -3249,15 +3249,17 @@ void
printTableAddCell(printTableContent *const content, char *cell,
const bool translate, const bool mustfree)
{
+ long product;
#ifndef ENABLE_NLS
(void) translate; /* unused parameter */
#endif
-
- if (content->cellsadded >= content->ncolumns * content->nrows)
+ /* product of cols and rows, using long type to prevent int overflow */
+ product = (long)content->ncolumns * (long)content->nrows;
+ if (content->cellsadded >= product)
{
fprintf(stderr, _("Cannot add cell to table content: "
- "total cell count of %d exceeded.\n"),
- content->ncolumns * content->nrows);
+ "total cell count of %ld exceeded, cells added: %ld.\n"),
+ product, content->cellsadded);
exit(EXIT_FAILURE);
}
@@ -3273,7 +3275,7 @@ printTableAddCell(printTableContent *const content, char *cell,
{
if (content->cellmustfree == NULL)
content->cellmustfree =
- pg_malloc0((content->ncolumns * content->nrows + 1) * sizeof(bool));
+ pg_malloc0((product + 1) * sizeof(bool));
content->cellmustfree[content->cellsadded] = true;
}
--
2.39.2 (Apple Git-143)
view thread (5+ messages)
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected]
Subject: Re: PSQL error: total cell count of XXX exceeded
In-Reply-To: <TYBP286MB03510C21D207F77D80CF6235B4F2A@TYBP286MB0351.JPNP286.PROD.OUTLOOK.COM>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox