Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1rX5jH-008jcW-5P for pgsql-general@arkaria.postgresql.org; Mon, 05 Feb 2024 20:38:11 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1rX5jG-00FtrN-5p for pgsql-general@arkaria.postgresql.org; Mon, 05 Feb 2024 20:38:10 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1rX5jF-00FtrF-Rh for pgsql-general@lists.postgresql.org; Mon, 05 Feb 2024 20:38:09 +0000 Received: from sss.pgh.pa.us ([68.162.161.243]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1rX5jD-005FFn-An for pgsql-general@lists.postgresql.org; Mon, 05 Feb 2024 20:38:08 +0000 Received: from sss1.sss.pgh.pa.us (localhost [127.0.0.1]) by sss.pgh.pa.us (8.15.2/8.15.2) with ESMTP id 415Kc5C0268825; Mon, 5 Feb 2024 15:38:05 -0500 From: Tom Lane To: Wyatt Tellis cc: Ron Johnson , "pgsql-generallists.postgresql.org" Subject: Re: Improving pg_dump performance when handling large numbers of LOBs In-reply-to: References: Comments: In-reply-to Wyatt Tellis message dated "Mon, 05 Feb 2024 11:55:43 -0800" MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <268823.1707165485.1@sss.pgh.pa.us> Date: Mon, 05 Feb 2024 15:38:05 -0500 Message-ID: <268824.1707165485@sss.pgh.pa.us> List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk Wyatt Tellis writes: > Yes, the LOBs themselves are tiny, but there are a lot of them (~150 > million) which seem to be slowing down pg_dump. Note, we did not > design/build this system and agree that use of LOBs for this purpose was > not necessary. I don't know of anything much you can do at the user level. But there is a patchset under development to improve pg_dump's behavior with tons of blobs: https://www.postgresql.org/message-id/842242.1706287466@sss.pgh.pa.us That's intended for v17, and I'm not sure how tough it might be to adapt to v15 pg_dump, but maybe you could make that happen. In any case, more eyeballs reviewing that patchset would be welcome. regards, tom lane