Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1laJfq-0001yi-N9 for pgsql-bugs@arkaria.postgresql.org; Sat, 24 Apr 2021 14:54:22 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1laJfo-0006tp-3I for pgsql-bugs@arkaria.postgresql.org; Sat, 24 Apr 2021 14:54:20 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1laJfn-0006ti-TL for pgsql-bugs@lists.postgresql.org; Sat, 24 Apr 2021 14:54:19 +0000 Received: from sss.pgh.pa.us ([66.207.139.130]) by magus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1laJfh-0001pS-04 for pgsql-bugs@lists.postgresql.org; Sat, 24 Apr 2021 14:54:19 +0000 Received: from sss1.sss.pgh.pa.us (localhost [127.0.0.1]) by sss.pgh.pa.us (8.15.2/8.15.2) with ESMTP id 13OEs8G61661418; Sat, 24 Apr 2021 10:54:08 -0400 From: Tom Lane To: Michael Paquier cc: konstbozhko@gmail.com, pgsql-bugs@lists.postgresql.org Subject: Re: BUG #16980: invalid memory alloc request size In-reply-to: References: <16980-0d2064db94c17318@postgresql.org> Comments: In-reply-to Michael Paquier message dated "Sat, 24 Apr 2021 14:59:06 +0900" MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <1661416.1619276048.1@sss.pgh.pa.us> Content-Transfer-Encoding: quoted-printable Date: Sat, 24 Apr 2021 10:54:08 -0400 Message-ID: <1661417.1619276048@sss.pgh.pa.us> List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk Michael Paquier writes: > On Fri, Apr 23, 2021 at 10:10:39AM +0000, PG Bug reporting form wrote: >> I have a table attachment_contents in my PostgreSQL database with bytea= type >> column. One cell of this bytea column is too big (appr. 900Mb). >> I receive error when try to select this cell: >> taxmon=3D> select content from attachment_contents where attachment_id = =3D 3293; >> ERROR: invalid memory alloc request size 1850079891 >> How can i avoid this problem? > No idea without more context. Some of your data may be corrupted, > leading to this error. It may also be possible that you have > discovered a new bug, but it is going to be hard for someone to figure > out what's happening without a self-contained test case. I doubt there's any corruption or unknown bug. If the bytea itself is 900MB, its hex-string representation will be twice that, exceeding the 1GB palloc chunk-size limit. So we can't represent the value as text, and even if we could, it wouldn't fit into a DataRow message. You can imagine various hacks that would move the threshold for trouble, but the long and the short of it is that field values exceeding maybe 100MB are likely to cause problems somewhere. If you have to store such things, you'd be best off to keep them in files outside the database. regards, tom lane