Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mwTpJ-00012x-Cy for pgsql-sql@arkaria.postgresql.org; Sun, 12 Dec 2021 18:44:01 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1mwTpH-0006A5-Ub for pgsql-sql@arkaria.postgresql.org; Sun, 12 Dec 2021 18:43:59 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mwTpH-00069u-M3 for pgsql-sql@lists.postgresql.org; Sun, 12 Dec 2021 18:43:59 +0000 Received: from sss.pgh.pa.us ([66.207.139.130]) by magus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mwTpB-0005eW-KR for pgsql-sql@postgresql.org; Sun, 12 Dec 2021 18:43:59 +0000 Received: from sss1.sss.pgh.pa.us (localhost [127.0.0.1]) by sss.pgh.pa.us (8.15.2/8.15.2) with ESMTP id 1BCIhpDQ106301; Sun, 12 Dec 2021 13:43:51 -0500 From: Tom Lane To: =?UTF-8?B?QnJpY2UgQW5kcsOp?= cc: pgsql-sql@postgresql.org Subject: Re: Memory exhaustion on large query In-reply-to: References: Comments: In-reply-to =?UTF-8?B?QnJpY2UgQW5kcsOp?= message dated "Sun, 12 Dec 2021 19:25:49 +0100" MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <106299.1639334631.1@sss.pgh.pa.us> Date: Sun, 12 Dec 2021 13:43:51 -0500 Message-ID: <106300.1639334631@sss.pgh.pa.us> List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk =?UTF-8?B?QnJpY2UgQW5kcsOp?= writes: > The content of the table is so huge that, if PQsendQueryPrepared retrieves > all data, or if postgresql engine is creating temp file with all data, the > query cannot succeed (I have no enough RAM or disk space to copy the whole > data). But I was expecting that using PQsendQueryPrepared and PQgetResult > would avoid this by returning one result at a time. You're confusing asynchronous mode with single-row mode. Async mode, per se, doesn't change memory consumption; it just lets you do something else while waiting for the query result. You are (I suppose) missing a call to PQsetSingleRowMode --- see https://www.postgresql.org/docs/current/libpq-single-row-mode.html regards, tom lane