Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1qyI3A-007PHh-FJ for pgsql-general@arkaria.postgresql.org; Wed, 01 Nov 2023 20:42:52 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1qyI38-008pbZ-RQ for pgsql-general@arkaria.postgresql.org; Wed, 01 Nov 2023 20:42:50 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1qyI38-008pbQ-Gn for pgsql-general@lists.postgresql.org; Wed, 01 Nov 2023 20:42:50 +0000 Received: from mail-wm1-x330.google.com ([2a00:1450:4864:20::330]) by magus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94.2) (envelope-from ) id 1qyI35-004DNA-Lh for pgsql-general@lists.postgresql.org; Wed, 01 Nov 2023 20:42:49 +0000 Received: by mail-wm1-x330.google.com with SMTP id 5b1f17b1804b1-40859c46447so1435035e9.1 for ; Wed, 01 Nov 2023 13:42:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cybertec-at.20230601.gappssmtp.com; s=20230601; t=1698871366; x=1699476166; darn=lists.postgresql.org; h=mime-version:user-agent:content-transfer-encoding:references :in-reply-to:date:to:from:subject:message-id:from:to:cc:subject:date :message-id:reply-to; bh=F8L+Msybw9kbpi0slnm1jve+p5Ldg7tAh2tRwrD2768=; b=jkLTvy0UkOmtYynSpPZfdmKWKqeI2t7B1pIeojRCwNRAl4JLoAhCZg9WywLzyOONMA Or0d0qlGuKGq6fGxBdbPRr5VaWVCGRHEx7a2wXTFIPelH2mr9QNZ3YJNOWdVORpSWGVx miTG70LYDmSIfqV/JsPPsl/9BFsnkRf24qx5iBY2Tqwsbjy93NhVEy9UaimgJt//biA2 LLT8ToZ0bvtn+S4ia3H5F4wfu4H/MO89la5z5/JpOixTXG4M2q7Z04JQDIboxb6R1bJK jetsrBYpKxe5sj9cZV2OOtLG9qHxTEP6booOwAUiCdNa7n5FEx+WkNGwLNitW/Z9vFXO oy8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698871366; x=1699476166; h=mime-version:user-agent:content-transfer-encoding:references :in-reply-to:date:to:from:subject:message-id:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=F8L+Msybw9kbpi0slnm1jve+p5Ldg7tAh2tRwrD2768=; b=Gd+8YBt4R51og1psWewF/th80YexHzfrLYAQ/MuSBodnWDcazV/LJuFbPJhpVhQ6TQ qE8UeDtlo7AXIPEzs0rG8jU8ltRMzt3dmjPn7+aRbbeW5MNF/PESBsfVQVlvQ2V3YlxN J4QMxwERAcNnaIWIZR4aFi50TNUVnF1rZFvFfdGgaKyIN7yZckhSJKwmz2gbBLYXkFew hnPErTE+Dhq0KwTxj6ur3KrdRmRhpB9Q8JenH0Z/ORBNOsQL79t4PX8Yq6gotj6wAH24 0chhJfFzCowpkqZOQZWCOonyZI2ZWNUyPwedFOxiQ5FOhCQoGluM3Icjb5OoO/g7Rmb1 dGnA== X-Gm-Message-State: AOJu0YyZedJWQ47bn1Z7KGAS8Q5AjbZ31pKAm7GOoz1dwkkxYbwgN3ic 2In22wOiLnF2AhNftRd9i0x/qQ== X-Google-Smtp-Source: AGHT+IF6UhUWGZYDPQ3AuC3Mn0FtagVaH90IFxje7IUTb0rutVzqlZVhHB5Q1P6mLHYZim8wzkt3dA== X-Received: by 2002:a05:600c:154a:b0:401:73b2:f043 with SMTP id f10-20020a05600c154a00b0040173b2f043mr15187966wmg.1.1698871365900; Wed, 01 Nov 2023 13:42:45 -0700 (PDT) Received: from localhost.localdomain ([2001:871:260:b539:1410:61d3:4510:7322]) by smtp.gmail.com with ESMTPSA id f9-20020a05600c154900b00407460234f9sm659108wmg.21.2023.11.01.13.42.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Nov 2023 13:42:45 -0700 (PDT) Message-ID: <0c82a8d32181fca3b979347f34ffbb2f7fdf0e90.camel@cybertec.at> Subject: Re: pg_dump/pg_restore --jobs practical limit? From: Laurenz Albe To: Ron , pgsql-general@lists.postgresql.org Date: Wed, 01 Nov 2023 21:42:45 +0100 In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable User-Agent: Evolution 3.48.4 (3.48.4-1.fc38) MIME-Version: 1.0 List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk On Wed, 2023-11-01 at 13:09 -0500, Ron wrote: > I will "soon" migrate some 1+ TB database from Pg 9.6.24 on RHEL 6 VMs to= Pg=20 > 14.latest on RHEL 8 VMs.=C2=A0 The VMs have 10Gbps "NICs", SAN-based LUNs= managed=20 > by LVM, and are all on ESX blades.=C2=A0 nproc count on some is 16 and on= others=20 > is 32. >=20 > Does anyone have experience as to the point of diminishing returns? >=20 > IOW, can I crank them processes up to --jobs=3D30, will I see no gain -- = or=20 > even degradation -- after, for example, --jobs=3D24? >=20 > This would be for both pg_dump and pg_restore (which would be run on the= =20 > RHEL 8 VM). Test, test, test. Theoretical considerations are pretty worthless, and it is easy to measure that. Yours, Laurenz Albe