Subject: PSQL error: total cell count of XXX exceeded

From: Hongxu Ma <[email-masked]>
Date: Sat, 26 Aug 2023 03:29:44 +0000

Hi Hackers,

When I tried to select a big amount of rows, psql complains a error "Cannot add cell to table content: total cell count of 905032704 exceeded."

Here are the reproduce steps:
```
interma=# select version();
                                         version
-----------------------------------------------------------------------------------------
 PostgreSQL 12.13 on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
(1 row)

interma=# create table t26(a int,b int,c int,d int,e int,f int,g int,h int,i int,j int,k int,l int,m int,n int,o int,p int,q int,r int,s int,t int ,u int,v int,w int,x int,y int,z int);
CREATE TABLE
interma=# insert into t26 select generate_series(1,200000000);
INSERT 0 200000000
interma=# select * from t26;
Cannot add cell to table content: total cell count of 905032704 exceeded.
```

I checked the related code, and root cause is clear:
```
// in printTableAddCell()
if (content->cellsadded >= content->ncolumns * content->nrows)
      report this error and exit

// cellsadded is long type, but ncolumns and nrows are int
// so, it's possible overflow the int value here.

// using a test program to verify:
int rows = 200000000;
int cols = 26;
printf("%d*%d = %d\n", rows,cols, rows*cols);

output:
      2,0000,0000*26 = 9,0503,2704 // overflow and be truncated into int value here
```

Based on it, I think it's a bug. We should use long for ncolumns and nrows and give a more obvious error message here.

My version is 12.13, and I think the latest code also exists this issue: issue: https://github.com/postgres/postgres/blob/1a4fd77db85abac63e178506335aee74625f6499/src/fe_utils/print.c#L3259

Any thoughts? or some other hidden reasons?
Thanks.

---

From: "David G. Johnston" <[email-masked]>
Date: Fri, 25 Aug 2023 21:09:28 -0700

On Friday, August 25, 2023, Hongxu Ma <[email-masked]> wrote:
>
>
> When I tried to select a big amount of rows, psql complains a error "Cannot
> add cell to table content: total cell count of 905032704 exceeded."
>
>  We should use long for ncolumns and nrows and give a more obvious error
> message here.
>

9 millions cells seems more than realistic a limit for a psql query result
output.  In any case it isn’t a bug, the code demonstrates that fact by
producing an explicit error.

I wouldn’t be adverse to an improved error message, and possibly
documenting said limit.

David J.

---

From: Hongxu Ma <[email-masked]>
Date: Sat, 26 Aug 2023 11:19:03 +0000

Thank you David.

>From the code logic, I don't think this check is meant to check the limit:
If it enters the double-loop (cont.nrows * cont.ncolumns) in printQuery(), the check should be always false (except overflow happened). So, if want to check the limit, we could have done this check before the double-loop: just checking PGresult and reports error earlier.

> I wouldn’t be adverse to an improved error message, and possibly documenting said limit.

Agreed with you, current error message may even report a negative value, it's very confusing for user. It's better to introduce a limit here. Or using a bigger integer type (e.g. long) for them, but it's also have the theoretical upbound.

Thanks.

________________________________
From: David G. Johnston <[email-masked]>
Sent: Saturday, August 26, 2023 12:09
To: Hongxu Ma <[email-masked]>
Cc: PostgreSQL Hackers <[email-masked]>
Subject: Re: PSQL error: total cell count of XXX exceeded

On Friday, August 25, 2023, Hongxu Ma <[email-masked]<mailto:[email-masked]>> wrote:

 We should use long for ncolumns and nrows and give a more obvious error message here.

9 millions cells seems more than realistic a limit for a psql query result output.  In any case it isn’t a bug, the code demonstrates that fact by producing an explicit error.

I wouldn’t be adverse to an improved error message, and possibly documenting said limit.

David J.

---

From: Hongxu Ma <[email-masked]>
Date: Mon, 11 Sep 2023 06:50:39 +0000

I created a patch to fix it.
Really appreciate to anyone can help to review it.
Thanks.

________________________________
From: Hongxu Ma <[email-masked]>
Sent: Saturday, August 26, 2023 19:19
To: David G. Johnston <[email-masked]>
Cc: PostgreSQL Hackers <[email-masked]>
Subject: Re: PSQL error: total cell count of XXX exceeded

Thank you David.

Thanks.

David J.