public inbox for [email protected]
help / color / mirror / Atom feedFrom: Hongxu Ma <[email protected]>
To: Michael Paquier <[email protected]>
Cc: PostgreSQL Hackers <[email protected]>
Subject: Re: PSQL error: total cell count of XXX exceeded
Date: Wed, 13 Sep 2023 07:31:54 +0000
Message-ID: <TYBP286MB0351CA77974AFE5A689E3369B4F0A@TYBP286MB0351.JPNP286.PROD.OUTLOOK.COM> (raw)
In-Reply-To: <TYBP286MB0351C97FB2A217D927E9C332B4F0A@TYBP286MB0351.JPNP286.PROD.OUTLOOK.COM>
References: <TYBP286MB0351B057B101C90D7C1239E6B4E2A@TYBP286MB0351.JPNP286.PROD.OUTLOOK.COM>
<CAKFQuwYfypo7hMXs8a7Ccx4x2bv1XKCMu=pOoQAfq_0Fpu7YQQ@mail.gmail.com>
<TYBP286MB03514675D5D9AA3F533DC987B4E2A@TYBP286MB0351.JPNP286.PROD.OUTLOOK.COM>
<TYBP286MB03510C21D207F77D80CF6235B4F2A@TYBP286MB0351.JPNP286.PROD.OUTLOOK.COM>
<CAGECzQSo0qk+QwzVgGhjx+-gGmj41w+JPmUHmdeObFCp1h7pjQ@mail.gmail.com>
<TYBP286MB03515815935D9F6FA90C5921B4F1A@TYBP286MB0351.JPNP286.PROD.OUTLOOK.COM>
<ZP/[email protected]>
<[email protected]>
<TYBP286MB0351C97FB2A217D927E9C332B4F0A@TYBP286MB0351.JPNP286.PROD.OUTLOOK.COM>
After double check, looks `int64` of src/include/c.h is the proper type for it.
Uploaded the v4 patch to fix it.
Thanks.
________________________________
From: Hongxu Ma <[email protected]>
Sent: Wednesday, September 13, 2023 10:22
To: Tom Lane <[email protected]>; Michael Paquier <[email protected]>
Cc: PostgreSQL Hackers <[email protected]>
Subject: Re: PSQL error: total cell count of XXX exceeded
Thanks for pointing that, I did miss some other "ncols * nrows" places. Uploaded v3 patch to fix them.
As for the Windows, I didn't test it before but I think it should also have the issue (and happens more possible since `cellsadded` is also a long type).
My fix idea is simple: define a common long64 type for it.
I referred MSDN: only `LONGLONG` and `LONG64` are 64 bytes. And I assume Postgres should already have a similar type, but only found `typedef long int int64` in src/include/c.h, looks it's not a proper choose.
@Michael Paquier<mailto:[email protected]>, could you help to give some advices here (which type should be used? or should define a new one?). Thank you very much.
________________________________
From: Tom Lane <[email protected]>
Sent: Tuesday, September 12, 2023 12:19
To: Michael Paquier <[email protected]>
Cc: Hongxu Ma <[email protected]>; Jelte Fennema <[email protected]>; David G. Johnston <[email protected]>; PostgreSQL Hackers <[email protected]>
Subject: Re: PSQL error: total cell count of XXX exceeded
Michael Paquier <[email protected]> writes:
> On Tue, Sep 12, 2023 at 02:39:55AM +0000, Hongxu Ma wrote:
> + long total_cells;
> long is 4 bytes on Windows, and 8 bytes basically elsewhere. So you
> would still have the same problem on Windows, no?
More to the point: what about the multiplication in printTableInit?
The cat's been out of the bag for quite some time before we get to
printTableAddCell.
I'm more than a bit skeptical about trying to do something about this,
simply because this range of query result sizes is far past what is
practical. The OP clearly hasn't tested his patch on actually
overflowing query results, and I don't care to either.
regards, tom lane
Attachments:
[application/octet-stream] v4-0001-Using-int64-type-for-the-number-of-total-cells.patch (3.4K, ../TYBP286MB0351CA77974AFE5A689E3369B4F0A@TYBP286MB0351.JPNP286.PROD.OUTLOOK.COM/3-v4-0001-Using-int64-type-for-the-number-of-total-cells.patch)
download | inline diff:
From f322951005315f55beaae332b72359615342340b Mon Sep 17 00:00:00 2001
From: interma <[email protected]>
Date: Mon, 11 Sep 2023 14:42:14 +0800
Subject: [PATCH] Using int64 type for the number of total cells in
printTableAddCell() to prevent int overflow
---
src/fe_utils/print.c | 23 +++++++++++++++--------
src/include/fe_utils/print.h | 2 +-
2 files changed, 16 insertions(+), 9 deletions(-)
diff --git a/src/fe_utils/print.c b/src/fe_utils/print.c
index 7af1ccb6b5..d2ceafcdc4 100644
--- a/src/fe_utils/print.c
+++ b/src/fe_utils/print.c
@@ -3172,6 +3172,8 @@ void
printTableInit(printTableContent *const content, const printTableOpt *opt,
const char *title, const int ncolumns, const int nrows)
{
+ int64 total_cells;
+
content->opt = opt;
content->title = title;
content->ncolumns = ncolumns;
@@ -3179,7 +3181,8 @@ printTableInit(printTableContent *const content, const printTableOpt *opt,
content->headers = pg_malloc0((ncolumns + 1) * sizeof(*content->headers));
- content->cells = pg_malloc0((ncolumns * nrows + 1) * sizeof(*content->cells));
+ total_cells = (int64)ncolumns * (int64)nrows;
+ content->cells = pg_malloc0((total_cells + 1) * sizeof(*content->cells));
content->cellmustfree = NULL;
content->footers = NULL;
@@ -3249,15 +3252,18 @@ void
printTableAddCell(printTableContent *const content, char *cell,
const bool translate, const bool mustfree)
{
+ int64 total_cells;
#ifndef ENABLE_NLS
(void) translate; /* unused parameter */
#endif
- if (content->cellsadded >= content->ncolumns * content->nrows)
+ total_cells = (int64)content->ncolumns * (int64)content->nrows;
+ if (content->cellsadded >= total_cells)
{
fprintf(stderr, _("Cannot add cell to table content: "
- "total cell count of %d exceeded.\n"),
- content->ncolumns * content->nrows);
+ "total cell count of " INT64_FORMAT " "
+ "exceeded, cells added: " INT64_FORMAT ".\n"),
+ total_cells, content->cellsadded);
exit(EXIT_FAILURE);
}
@@ -3273,7 +3279,7 @@ printTableAddCell(printTableContent *const content, char *cell,
{
if (content->cellmustfree == NULL)
content->cellmustfree =
- pg_malloc0((content->ncolumns * content->nrows + 1) * sizeof(bool));
+ pg_malloc0((total_cells + 1) * sizeof(bool));
content->cellmustfree[content->cellsadded] = true;
}
@@ -3341,9 +3347,10 @@ printTableCleanup(printTableContent *const content)
{
if (content->cellmustfree)
{
- int i;
-
- for (i = 0; i < content->nrows * content->ncolumns; i++)
+ int64 i;
+ int64 total_cells;
+ total_cells = (int64)content->ncolumns * (int64)content->nrows;
+ for (i = 0; i < total_cells; i++)
{
if (content->cellmustfree[i])
free(unconstify(char *, content->cells[i]));
diff --git a/src/include/fe_utils/print.h b/src/include/fe_utils/print.h
index cc6652def9..6e96f1c26e 100644
--- a/src/include/fe_utils/print.h
+++ b/src/include/fe_utils/print.h
@@ -171,7 +171,7 @@ typedef struct printTableContent
const char **cells; /* NULL-terminated array of cell content
* strings */
const char **cell; /* Pointer to the last added cell */
- long cellsadded; /* Number of cells added this far */
+ int64 cellsadded; /* Number of cells added this far */
bool *cellmustfree; /* true for cells that need to be free()d */
printTableFooter *footers; /* Pointer to the first footer */
printTableFooter *footer; /* Pointer to the last added footer */
--
2.39.2 (Apple Git-143)
view thread (6+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected]
Subject: Re: PSQL error: total cell count of XXX exceeded
In-Reply-To: <TYBP286MB0351CA77974AFE5A689E3369B4F0A@TYBP286MB0351.JPNP286.PROD.OUTLOOK.COM>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox