Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1ugPWQ-002ijf-ED for pgsql-admin@arkaria.postgresql.org; Mon, 28 Jul 2025 15:12:15 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1ugPWP-008gNq-Ih for pgsql-admin@arkaria.postgresql.org; Mon, 28 Jul 2025 15:12:13 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1ugPWP-008gNi-5z for pgsql-admin@lists.postgresql.org; Mon, 28 Jul 2025 15:12:13 +0000 Received: from mail-wr1-x431.google.com ([2a00:1450:4864:20::431]) by magus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.96) (envelope-from ) id 1ugPWM-001JrP-1V for pgsql-admin@lists.postgresql.org; Mon, 28 Jul 2025 15:12:13 +0000 Received: by mail-wr1-x431.google.com with SMTP id ffacd0b85a97d-3b7823559a5so961453f8f.0 for ; Mon, 28 Jul 2025 08:12:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cybertec.at; s=google; t=1753715530; x=1754320330; darn=lists.postgresql.org; h=mime-version:user-agent:content-transfer-encoding:references :in-reply-to:date:to:from:subject:message-id:from:to:cc:subject:date :message-id:reply-to; bh=ozreYhjeG89jiqOf4A8SjdumyONQwRFwewubK71+qz0=; b=l1IsliMdrw4JzN0L8e1aDCL59oui1TzJ3nQ6wmZq+lbAX502JW4o4/Us2Fygoumpdn Sfi6jI7xq6ULA2xzi4mIHwULO7wo8pV7OtJ1jWANfZU/Shwoq2sXHQyJIrpSBb3SoYdY 37mzfcpPGvx94H3TnX95Y2gQO4o4ULfaMP2q1WlXsx31YDCDrg0t6YOWMM1A7Doap/MW edx4K/xbFEwlSoyNfafN0PRnoKg/KbXWaU03pfuQqRl3rFvwjXfZHInrM4N8F85h2w37 Uj2FOKmSX/WtP43yRBxSfHsQtngiyzhLNAxOEUh5PFE2IgkAm68EKsI5G/KMjOXmcW+c M5hw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753715530; x=1754320330; h=mime-version:user-agent:content-transfer-encoding:references :in-reply-to:date:to:from:subject:message-id:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=ozreYhjeG89jiqOf4A8SjdumyONQwRFwewubK71+qz0=; b=Fpo6SynqHmY0G7O+sI7uLEPGNGl5haWWWyC8FkRSqM4zgXAXvfqO5WDiZMBMhDQ/YL ImDfGLWPeG02iXFPxV6Dv+MagU46DHs6bWdFDFOQbiMXW4pdwI/1XfUbDsmClp7bbP6y X9ybA6WC6Ruz53YxvREauKOnv5GboOdMROc+lwNesEMtYpJKHWE4vEF7pOBa96L0AaGB d7v7P1OUJZWu92YqGlP/PXsfwQwyNWG+yWIc4vyr8L2+lL2nfSARk68UjHrwGpkemlST leJ0G7d2MkU0q7e0PKPobKDfPDRJv6U9XltW7KG9vS3Oky64aNA1yKesdP+e+QYOvFLY ykkg== X-Forwarded-Encrypted: i=1; AJvYcCXgMP4yR88GwVLp2s5ZX4am9dTUnxgP0PB/YSACS02gK08QVHUaMpDZARVqiNlF0m67J5A2awXZJEHtwA==@lists.postgresql.org X-Gm-Message-State: AOJu0YzbkeeVXPjFCgtAwZJIXSDsiXuFrDHk9NpfeO5n1vBVH0X0xtOV emTYaPyHMletU7RnmzccRaj4b2vCduRH+3gLWGPq8Qok+O5lbfu/IXtA2AH/VYnelDgPfiyzeXn ALF1m X-Gm-Gg: ASbGncv+xiQudKuUis5d7GfnmjrDtXjWt7pJE9xTqqiLwy1fPTY0hKO7UCBYtq1alnr ncbfsIZ+GB/g2ykq7frprLkNaMRNWCtbGQ5uBwqHAy8biYIgQvzSqHzkXYox2/qk+5uKo2QQfJC PTiAaheuPp3pNLqLTj83d88mHR8Tea7B3KYXm5m7ud+iPCzSXd+M1JSjXyq7+Ert2WuhJG958X8 kq9wZVoaJQ2vC5H6S7l56huhPOSYCYFiniERwiyXMQlXvMSMqD8cxDJupzlnU/eumrEfvPd9DQY wD+c9lgZVDdPPtpcf6yFuxZCn3ZANrQdXSu8eowqm1jGm6uK+ZX7A1UEztRQy3MVOX1flBD6S9B veoQKeScCdrn1Tq/LZP98Z0qBq2DmoiTQi1836/2KwhuHYxTQSB/O X-Google-Smtp-Source: AGHT+IET/k6cpKmZ1oDqWlJkZqQcMehjdFJP2gaacg0GjDNV13sFVGryQ4DZk7h26DjVkydqD/972Q== X-Received: by 2002:a05:6000:2508:b0:3b7:8410:22b6 with SMTP id ffacd0b85a97d-3b78410258dmr3445281f8f.6.1753715529643; Mon, 28 Jul 2025 08:12:09 -0700 (PDT) Received: from laurenz.albe-K4N0CV00F97414D ([2001:871:260:9b8a:5191:4127:e9f1:710b]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4587ac661acsm105746995e9.23.2025.07.28.08.12.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Jul 2025 08:12:09 -0700 (PDT) Message-ID: <6871c96b81c4a6c3f9029821fa48205e707dca19.camel@cybertec.at> Subject: Re: pg_bulkload slowing down From: Laurenz Albe To: Ron Johnson , Pgsql-admin Date: Mon, 28 Jul 2025 17:12:08 +0200 In-Reply-To: References: <2846ad9c-e1d7-4780-b4e0-c2bceee64b64@quadratum-braccas.com> <202507281323.55loakn7tfyx@alvherre.pgsql> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable User-Agent: Evolution 3.56.2 (3.56.2-1.fc42) MIME-Version: 1.0 List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk On Mon, 2025-07-28 at 09:52 -0400, Ron Johnson wrote: > That seems to imply you can only insert 4Bn rows into a table with > attached TOAST.=C2=A0 Am=C2=A0I misunderstanding something? Only if those rows actually contain column values big enough to warrant storing them out of line. And if each row had *two* such columns, you could only store 2^31 such rows. Yours, Laurenz Albe