Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1ugNpM-002FGa-Mf for pgsql-admin@arkaria.postgresql.org; Mon, 28 Jul 2025 13:23:41 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1ugNpL-007kWe-S8 for pgsql-admin@arkaria.postgresql.org; Mon, 28 Jul 2025 13:23:40 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1ugNpL-007kWV-66 for pgsql-admin@lists.postgresql.org; Mon, 28 Jul 2025 13:23:39 +0000 Received: from fout-b7-smtp.messagingengine.com ([202.12.124.150]) by makus.postgresql.org with smtp (Exim 4.96) (envelope-from ) id 1ugNpJ-001DA9-0C for pgsql-admin@lists.postgresql.org; Mon, 28 Jul 2025 13:23:38 +0000 Received: from phl-compute-02.internal (phl-compute-02.phl.internal [10.202.2.42]) by mailfout.stl.internal (Postfix) with ESMTP id 709501D00289; Mon, 28 Jul 2025 09:23:36 -0400 (EDT) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-02.internal (MEProxy); Mon, 28 Jul 2025 09:23:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kurilemu.de; h= cc:cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :reply-to:subject:subject:to:to; s=fm1; t=1753709016; x= 1753795416; bh=RQT7iH1/Cujphv2/pfdWQ3h3NjTzhDw0ubUrPvMkdwY=; b=C Fn7vWtLcDsTSiXszpFx6jeeee72mbJSo8LwKmAqVNgY6suIymo74KrsTmO7EKeyu Sw6qrPyob3xPGRe6aJyiQgZd5HVp+vXxImb149/ITnsELyDogTjvI0QW2SLYJMoD fh/UoFGvlWrbnvdpaaVzyD8f5TA0Ijfxpr3yyYifUG9qzR1u8aZnnLtA4McYkgjd gA18mP1URk+Pe+JWPFaDX/jdXVLs5QzIiWE7mBSTMt5TJs8sFP5DDzf/mLmlGapf XntpZLXAqBbN322f+CYxqu6Y8RRJZ3uLsqCSjmNntmY42QBioB90fiT8wEyoZxwI yN6oEitqQZrUh4K7Vd/5w== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :reply-to:subject:subject:to:to:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm3; t=1753709016; x=1753795416; bh=R QT7iH1/Cujphv2/pfdWQ3h3NjTzhDw0ubUrPvMkdwY=; b=SQTB8B9t2gGl5fngT ymlCA+VkkF5Ia3LDcD6KBmM9Zrg0mcdypbu6hjcxcsEbmZKQ7vpyv4pK6HO7gWbS Oy0k+4g0yjHYpN4qF6AHl3wtnSy2kVuKSV2bv4pWciZjCJgd8nxPl7lV1t9eWlSt LyVJoStDEW5XmGWBlHUNG1pCfBRjUu1iQnsQmCOf+OyA/3L64U9ylSWOmaXTPhxA gQeFsNYr2lfh/V5EtRMj5xyRTyMirDrvgIFjFAb/8hYgriZUcCaToNszxR6mWQXk DloFYUchZCWW8EPeHL3dRsgcsstoAOxAk2pWs5tq0vw1mZH9xymOePTlcz9/+Y+r q2gPA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeffedrtdefgdelvddvkecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpuffrtefokffrpgfnqfghnecuuegr ihhlohhuthemuceftddtnecunecujfgurhepfffhvfevuffkgggtugfgjgesthekredttd dtjeenucfhrhhomheplmhlvhgrrhhoucfjvghrrhgvrhgruceorghlvhhhvghrrhgvsehk uhhrihhlvghmuhdruggvqeenucggtffrrghtthgvrhhnpeetuedvheffkeevgfeuheevte evkefggedttdeufeeuheduuddthfeffffhjeefffenucffohhmrghinhepvghnthgvrhhp rhhishgvuggsrdgtohhmnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrg hilhhfrhhomheprghlvhhhvghrrhgvsehkuhhrihhlvghmuhdruggvpdhnsggprhgtphht thhopedvpdhmohguvgepshhmthhpohhuthdprhgtphhtthhopehpghhsqhhlqdgrughmih hnsehlihhsthhsrdhpohhsthhgrhgvshhqlhdrohhrghdprhgtphhtthhopehssghosges qhhurggurhgrthhumhdqsghrrggttggrshdrtghomh X-ME-Proxy: Feedback-ID: ie3de48e3:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 28 Jul 2025 09:23:35 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kurilemu.de; s=schmee; t=1753709013; bh=Fqhz6kVVD8MJ2z0WeoQ0UnUY4gGVNPfbOSCjeLFVon8=; h=Date:From:To:Cc:Subject:In-Reply-To:From; b=E2A5bTomlX75fY+YLVtHYeyTvmZ3FLit0EWaPJsCceSUOkcjVzcK+NcigMoa4m2T4 Nh6NCa+p3sgl+a8rK07jWReytZ4cH/GcCgulQdV+Gkkt8vIatoIy1jRFy5GukRFdH7 t06jGIlS/7eTZP0pSrYE78ueAjUvUbdyy5juRdUmrrzsy2EM7SHiG8/CIqZWmcqHxP ih8zw6LMEnzuKdp5MNULNH9GwMcKv3TSWBvLVgt+QJIS6OOnZNtdijgm/qtp4b6r45 pXPed9QyjqPTVi0SJise/3D/MTLXnCZusGaf0BlWgScQLnby1Vub84VXQYP67of5wR smm+oVwZJpfoA== Received: by schmee.kurilemu.internal (Postfix, from userid 1000) id B1E0290; Mon, 28 Jul 2025 15:23:33 +0200 (CEST) Date: Mon, 28 Jul 2025 15:23:33 +0200 From: =?utf-8?Q?=C3=81lvaro?= Herrera To: Sbob Cc: pgsql-admin@lists.postgresql.org Subject: Re: pg_bulkload slowing down Message-ID: <202507281323.55loakn7tfyx@alvherre.pgsql> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <2846ad9c-e1d7-4780-b4e0-c2bceee64b64@quadratum-braccas.com> List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk On 2025-Jul-28, Sbob wrote: > All; > > > We are loading a series of files into a single table, each file has approx > 500million rows, the first file took ~ 1.25 hours, now on file number 8 it > has been running for more than 10 hours, there is only a PK index on the > table. I bet it's trying to find unused OIDs for toast pointers -- there are only 2^32-1 of those available in any one toast table, and by the eighth file of 500 million rows, you'd run out. You could workaround this by partitioning the table. -- Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/