public inbox for [email protected]
help / color / mirror / Atom feedFrom: Hongxu Ma <[email protected]>
To: Tom Lane <[email protected]>
To: Michael Paquier <[email protected]>
Cc: PostgreSQL Hackers <[email protected]>
Subject: Re: PSQL error: total cell count of XXX exceeded
Date: Wed, 13 Sep 2023 02:22:48 +0000
Message-ID: <TYBP286MB0351C97FB2A217D927E9C332B4F0A@TYBP286MB0351.JPNP286.PROD.OUTLOOK.COM> (raw)
In-Reply-To: <[email protected]>
References: <TYBP286MB0351B057B101C90D7C1239E6B4E2A@TYBP286MB0351.JPNP286.PROD.OUTLOOK.COM>
<CAKFQuwYfypo7hMXs8a7Ccx4x2bv1XKCMu=pOoQAfq_0Fpu7YQQ@mail.gmail.com>
<TYBP286MB03514675D5D9AA3F533DC987B4E2A@TYBP286MB0351.JPNP286.PROD.OUTLOOK.COM>
<TYBP286MB03510C21D207F77D80CF6235B4F2A@TYBP286MB0351.JPNP286.PROD.OUTLOOK.COM>
<CAGECzQSo0qk+QwzVgGhjx+-gGmj41w+JPmUHmdeObFCp1h7pjQ@mail.gmail.com>
<TYBP286MB03515815935D9F6FA90C5921B4F1A@TYBP286MB0351.JPNP286.PROD.OUTLOOK.COM>
<ZP/[email protected]>
<[email protected]>
Thanks for pointing that, I did miss some other "ncols * nrows" places. Uploaded v3 patch to fix them.
As for the Windows, I didn't test it before but I think it should also have the issue (and happens more possible since `cellsadded` is also a long type).
My fix idea is simple: define a common long64 type for it.
I referred MSDN: only `LONGLONG` and `LONG64` are 64 bytes. And I assume Postgres should already have a similar type, but only found `typedef long int int64` in src/include/c.h, looks it's not a proper choose.
@Michael Paquier<mailto:[email protected]>, could you help to give some advices here (which type should be used? or should define a new one?). Thank you very much.
________________________________
From: Tom Lane <[email protected]>
Sent: Tuesday, September 12, 2023 12:19
To: Michael Paquier <[email protected]>
Cc: Hongxu Ma <[email protected]>; Jelte Fennema <[email protected]>; David G. Johnston <[email protected]>; PostgreSQL Hackers <[email protected]>
Subject: Re: PSQL error: total cell count of XXX exceeded
Michael Paquier <[email protected]> writes:
> On Tue, Sep 12, 2023 at 02:39:55AM +0000, Hongxu Ma wrote:
> + long total_cells;
> long is 4 bytes on Windows, and 8 bytes basically elsewhere. So you
> would still have the same problem on Windows, no?
More to the point: what about the multiplication in printTableInit?
The cat's been out of the bag for quite some time before we get to
printTableAddCell.
I'm more than a bit skeptical about trying to do something about this,
simply because this range of query result sizes is far past what is
practical. The OP clearly hasn't tested his patch on actually
overflowing query results, and I don't care to either.
regards, tom lane
Attachments:
[application/octet-stream] v3-0001-Using-long-type-in-printTableAddCell.patch (2.7K, ../TYBP286MB0351C97FB2A217D927E9C332B4F0A@TYBP286MB0351.JPNP286.PROD.OUTLOOK.COM/3-v3-0001-Using-long-type-in-printTableAddCell.patch)
download | inline diff:
From 19c8c53949959a0bac2408268d8709c8930e042d Mon Sep 17 00:00:00 2001
From: interma <[email protected]>
Date: Mon, 11 Sep 2023 14:42:14 +0800
Subject: [PATCH] Using long type in printTableAddCell() to prevent int
overflow
---
src/fe_utils/print.c | 26 ++++++++++++++++++--------
1 file changed, 18 insertions(+), 8 deletions(-)
diff --git a/src/fe_utils/print.c b/src/fe_utils/print.c
index 7af1ccb6b5..5c0748df10 100644
--- a/src/fe_utils/print.c
+++ b/src/fe_utils/print.c
@@ -3172,6 +3172,8 @@ void
printTableInit(printTableContent *const content, const printTableOpt *opt,
const char *title, const int ncolumns, const int nrows)
{
+ long total_cells;
+
content->opt = opt;
content->title = title;
content->ncolumns = ncolumns;
@@ -3179,7 +3181,8 @@ printTableInit(printTableContent *const content, const printTableOpt *opt,
content->headers = pg_malloc0((ncolumns + 1) * sizeof(*content->headers));
- content->cells = pg_malloc0((ncolumns * nrows + 1) * sizeof(*content->cells));
+ total_cells = (long)ncolumns * (long)nrows;
+ content->cells = pg_malloc0((total_cells + 1) * sizeof(*content->cells));
content->cellmustfree = NULL;
content->footers = NULL;
@@ -3249,15 +3252,21 @@ void
printTableAddCell(printTableContent *const content, char *cell,
const bool translate, const bool mustfree)
{
+ long total_cells;
#ifndef ENABLE_NLS
(void) translate; /* unused parameter */
#endif
- if (content->cellsadded >= content->ncolumns * content->nrows)
+ /*
+ * total_cells is the product of ncolumns and nrows
+ * Using long type here to prevent int overflow
+ */
+ total_cells = (long)content->ncolumns * (long)content->nrows;
+ if (content->cellsadded >= total_cells)
{
fprintf(stderr, _("Cannot add cell to table content: "
- "total cell count of %d exceeded.\n"),
- content->ncolumns * content->nrows);
+ "total cell count of %ld exceeded, cells added: %ld.\n"),
+ total_cells, content->cellsadded);
exit(EXIT_FAILURE);
}
@@ -3273,7 +3282,7 @@ printTableAddCell(printTableContent *const content, char *cell,
{
if (content->cellmustfree == NULL)
content->cellmustfree =
- pg_malloc0((content->ncolumns * content->nrows + 1) * sizeof(bool));
+ pg_malloc0((total_cells + 1) * sizeof(bool));
content->cellmustfree[content->cellsadded] = true;
}
@@ -3341,9 +3350,10 @@ printTableCleanup(printTableContent *const content)
{
if (content->cellmustfree)
{
- int i;
-
- for (i = 0; i < content->nrows * content->ncolumns; i++)
+ long i;
+ long total_cells;
+ total_cells = (long)content->ncolumns * (long)content->nrows;
+ for (i = 0; i < total_cells; i++)
{
if (content->cellmustfree[i])
free(unconstify(char *, content->cells[i]));
--
2.39.2 (Apple Git-143)
view thread (6+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected]
Subject: Re: PSQL error: total cell count of XXX exceeded
In-Reply-To: <TYBP286MB0351C97FB2A217D927E9C332B4F0A@TYBP286MB0351.JPNP286.PROD.OUTLOOK.COM>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox