Received: from malur.postgresql.org ([217.196.149.56])
	by arkaria.postgresql.org with esmtps  (TLS1.3) tls
 TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
	(Exim 4.94.2)
	(envelope-from <pgsql-bugs-owner+archive@lists.postgresql.org>)
	id 1tYVNd-00BSZC-0r
	for pgsql-bugs@arkaria.postgresql.org; Thu, 16 Jan 2025 19:18:13 +0000
Received: from localhost ([127.0.0.1] helo=malur.postgresql.org)
	by malur.postgresql.org with esmtp (Exim 4.94.2)
	(envelope-from <pgsql-bugs-owner+archive@lists.postgresql.org>)
	id 1tYVNa-000RDW-IV
	for pgsql-bugs@arkaria.postgresql.org; Thu, 16 Jan 2025 19:18:11 +0000
Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29])
	by malur.postgresql.org with esmtps  (TLS1.3) tls
 TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
	(Exim 4.94.2)
	(envelope-from <tgl@sss.pgh.pa.us>)
	id 1tYVNa-000RDN-9T
	for pgsql-bugs@lists.postgresql.org; Thu, 16 Jan 2025 19:18:10 +0000
Received: from sss.pgh.pa.us ([68.162.161.243])
	by magus.postgresql.org with esmtps  (TLS1.3) tls
 TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
	(Exim 4.96)
	(envelope-from <tgl@sss.pgh.pa.us>)
	id 1tYVNY-000om3-1x
	for pgsql-bugs@lists.postgresql.org;
	Thu, 16 Jan 2025 19:18:10 +0000
Received: from sss1.sss.pgh.pa.us (localhost [127.0.0.1])
	by sss.pgh.pa.us (8.15.2/8.15.2) with ESMTP id 50GJI6t31849377;
	Thu, 16 Jan 2025 14:18:06 -0500
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Ilya Knyazev <knuazev@gmail.com>
cc: pgsql-bugs@lists.postgresql.org
Subject: Re: BUG #18775: PQgetCopyData always has an out-of-memory error if
 the table field stores bytea ~700 MB
In-reply-to: 
 <CAFqQ4ZWyUwwwgHA+BBGiLKbmm+SaRPJZQSgOHajDa+AB+igfRA@mail.gmail.com>
References: <18775-e6af405ca08393cc@postgresql.org>
 <1623899.1737043379@sss.pgh.pa.us>
 <CAFqQ4ZWyUwwwgHA+BBGiLKbmm+SaRPJZQSgOHajDa+AB+igfRA@mail.gmail.com>
Comments: In-reply-to Ilya Knyazev <knuazev@gmail.com>
	message dated "Thu, 16 Jan 2025 20:02:03 +0300"
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-ID: <1849375.1737055086.1@sss.pgh.pa.us>
Date: Thu, 16 Jan 2025 14:18:06 -0500
Message-ID: <1849376.1737055086@sss.pgh.pa.us>
List-Id: <pgsql-bugs.lists.postgresql.org>
List-Help: <https://lists.postgresql.org/manage/>
List-Subscribe: <https://lists.postgresql.org/manage/>
List-Post: <mailto:pgsql-bugs@lists.postgresql.org>
List-Owner: <mailto:pgsql-bugs-owner@lists.postgresql.org>
List-Archive: <https://www.postgresql.org/list/pgsql-bugs>
Archived-At: 
 <https://www.postgresql.org/message-id/1849376.1737055086%40sss.pgh.pa.us>
Precedence: bulk

Ilya Knyazev <knuazev@gmail.com> writes:
> But I know that there may not be enough memory, so I use the "copy" keyword
> in the query and the PQgetCopyData function. I thought that this function
> was designed for portioned work. By analogy with the PQputCopyData
> function, which works fine.

Its documentation is fairly clear, I thought:

       Attempts to obtain another row of data from the server during a
       <command>COPY</command>.  Data is always returned one data row at
       a time; if only a partial row is available, it is not returned.

If you need to work with data values that are large enough to risk
memory problems, I think "large objects" are the best answer.  Their
interface is a bit clunky, but it's at least designed to let you
both read and write by chunks.

			regards, tom lane