Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1sGio2-00GNUz-JM for pgsql-hackers@arkaria.postgresql.org; Mon, 10 Jun 2024 17:27:43 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1sGinz-00EOUY-MP for pgsql-hackers@arkaria.postgresql.org; Mon, 10 Jun 2024 17:27:40 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1sGinz-00EOU2-91 for pgsql-hackers@lists.postgresql.org; Mon, 10 Jun 2024 17:27:40 +0000 Received: from mail-yb1-xb34.google.com ([2607:f8b0:4864:20::b34]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94.2) (envelope-from ) id 1sGinq-000fAS-Vs for pgsql-hackers@postgresql.org; Mon, 10 Jun 2024 17:27:38 +0000 Received: by mail-yb1-xb34.google.com with SMTP id 3f1490d57ef6-df771b6cc9cso145370276.3 for ; Mon, 10 Jun 2024 10:27:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dunslane-net.20230601.gappssmtp.com; s=20230601; t=1718040449; x=1718645249; darn=postgresql.org; h=content-transfer-encoding:in-reply-to:autocrypt:content-language :from:references:cc:to:subject:user-agent:mime-version:date :message-id:from:to:cc:subject:date:message-id:reply-to; bh=KV77rNQ6KrAwDlFwA+7Lfv9kXWMzTEE/NTppkrAyVTU=; b=BOv6KyeaIo0UnuJofKVytIaIuMDpf4sIY3tIGJJT/P3vuYmwR0ouejxy1kTsNKVOe0 JW/hgQHdI7dhzehBz2Kqpa49po+Ia4j+XrjP0veyVVISxFXpu0n9JMRXDUvYzWvqoOJj rQtxgjgCKntbdAI08UE8pZO32AOBjNDjWS9UfCi4u2ed35UVZvJ4JmourvL0dRdmXpe8 FeTRivqD9ljB/s2Rwx6DIWJD8rwYqTvnu105hs4B3/QJAk3F0XEEq+JMUVpAdFChuUEf h0/LUcylg+cmKOU17uacX9+llbqI9549hct8XrNiGtyCpz7dRx6gHpb9pgiAxjzj9Fwr Xeqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718040449; x=1718645249; h=content-transfer-encoding:in-reply-to:autocrypt:content-language :from:references:cc:to:subject:user-agent:mime-version:date :message-id:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=KV77rNQ6KrAwDlFwA+7Lfv9kXWMzTEE/NTppkrAyVTU=; b=V/pXPWFirn6Rsh+vcE/bmv/T0u4qy5UrFOTgH5RMWVf5huHhRT62TyqS9w3uC092u2 iOk8F2xgbuKEwUIlDG9a35KCDQzU3TxkI30/ScevYthQZlXXXgC+cSXVin2dTMt2gjnS WdsLbnymexAN2429Gd1/UcEdlEQ8smQpZWKXRgKcd4BdhUu03wZVDvRv6fa9uGnDVnSt vUWlcKzLz7xIsfvxKG+iL0wJL5DflBhbrMlQU4c9QTGDdiVWVD88q9VHUNWvZuPS5G5G HTMQHVZAXhT6Rnu9h1QZAk/hE9YN/lQ/o98lZYpz0LK3vJdum77D0jjdY8fMk20CHQUe P+cw== X-Forwarded-Encrypted: i=1; AJvYcCVAJU073V9DaVuwdj7uH3X23ng9CtGzYAF1pW1OSXD990DHTjivATDW2iwYoHs9AimfFFt+rYZyAuaENf5Zmit8ZW18fQ3M8DcjcZCA X-Gm-Message-State: AOJu0YyWrX2zKR6cHNeCgJf0CjI6yQc9Y1A2ZQqZDZGtS/7795aWTeQS 8tFZ/rq4/u4sNwSku7yQfqriUapE5RLbwjDByC7oXRiNyyRZet524sdTTO2WKqs= X-Google-Smtp-Source: AGHT+IFs5XouZAaGMoJb8+KYmkAPvF1cVIKEm+dydIH2yUuVmLNPcLYWF7CjH7MqoapvzWcFOHNnqA== X-Received: by 2002:a25:bcc:0:b0:dfa:72cc:f88b with SMTP id 3f1490d57ef6-dfaf6624a07mr9957859276.52.1718040448667; Mon, 10 Jun 2024 10:27:28 -0700 (PDT) Received: from ?IPV6:2605:a601:9180:9800::2bb? ([2605:a601:9180:9800::2bb]) by smtp.googlemail.com with ESMTPSA id d75a77b69052e-440530ac873sm27080201cf.67.2024.06.10.10.27.27 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 10 Jun 2024 10:27:28 -0700 (PDT) Message-ID: <9f559106-f655-469d-8f66-457761398e88@dunslane.net> Date: Mon, 10 Jun 2024 13:27:27 -0400 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Non-text mode for pg_dumpall To: Tom Lane , Magnus Hagander Cc: Nathan Bossart , PostgreSQL-development References: <740696.1718036463@sss.pgh.pa.us> From: Andrew Dunstan Content-Language: en-US Autocrypt: addr=andrew@dunslane.net; keydata= xsBNBE7KWFkBCAClridxur2AIc7eW2AR7izbfp3EnNefie2HbLF0izW5Ik5UjX2HBXBx4syI gY6b0ugohXrr274+baoAlvSbq6cAoQuEVrk5IZFzt20b1Xkx65FwGSEj526yiKLocqkJceSq Xr9xcA5SGY+FZv441chh5SU92v4q6z+6LPpoHOh97ptAVXZYNTtU0LevyvD5lja0TzbvJm6C eFXitJfnm1pLEr0DGJCR/iUOl/N62Kh4855zZC7NHIjQHPOvV5Stz/l5ilDhvGVk+xkXFPys SjZoUr1rXhYLpiyi5sR0X9FHXT0KnGuz1F5ERO7ZTLSSQ6fJwPj6gOk9K+vvoKvoeql5ABEB AAHNJEFuZHJldyBEdW5zdGFuIDxhbmRyZXdAZHVuc2xhbmUubmV0PsLAmwQTAQgARQIbAwIX gAIZAQULCQgHAgMiAgEGFQoJCAsCBBYCAwECHgcWIQTkPlhGHfx8v0RpFaWZ+n/LWfw7gQUC ZFlxxwUJGVGAbgAKCRCZ+n/LWfw7gXikB/9ZdcUy6CTBFIIuL/bVsc1eLEW/gJBjJBF6HxNY xgEkAgXAp4Lg4A5U+QB9GouFr7+GYxF0BU4hzoGhNPUWltxnHdMWP8nC/38LAqgMi8L/bbsm HW5YPBdWYaAZAPJQVfOAgjTbRUb26KSprpyrrJKW0ZmrZfjhNPcQ72jpWzoPLQqx2X6B0fru 1jq+cBh8lb6r1mJTim1T3JIn+F/v5VpdQS+EL8xqsHkfzKjIPsW3CIXpkypSk6saA55Rkkbl 26AW8ftPVB0Q6Lnn6FLt9CP0MGNixBQ55yq8r1K+nCBvCCjvQjM8RDm0UUum0WNl+ifQgTLO E8TWEnwVtkBf+3QWzsBNBE7KWFkBCADRnOM0FCzsYW6jtncg+dWIagjUZpvaClmqn/sJluLa Q3v1VXMQJzYs3eC1gh386W+XBwLRpDj3jzH81lX+p73Re3d3oJW7X+ffsxuzu5ZVdMUkqBYo nkAbKxr6gyJ12F/+JkUVzLcoTN+d/7YsQvUVi7NaKH8mJgjz112O4fUe3p9wfAaFa0RXHc5S GPzRTYRRlv/XZBIho4J2tkZOnteZJZ+GbxQVlINt6fd8P6al3MWOvpP/ExJPguEfjOsO6Njy xjo3WfpD4lHMOR/Oc3/8mScEF84rF2jXbsFgelWnbPWAvXY+pD0dXOFRkagGmC/viwBDqq5b 5tk76kKmUbZxABEBAAHCwHwEGAEIACYCGwwWIQTkPlhGHfx8v0RpFaWZ+n/LWfw7gQUCZFlx 5wUJGVGAjgAKCRCZ+n/LWfw7gf+iB/4g8CPY5jihf5r/8EsoIGe2H+dpVmpPF8YGBzTIvCz/ fQoOq8AX/pE76QEuFnFZWfjw+wgBXgCVmkox2Eflkk6z4ND3pcwGZ6CfCxTQCDk/dij+2DQ4 6bmDCy/sBgcbz9mTpoLC11HLoPae6YN9nBNQRZDcEFEu54OaVOqlIdbA6m+POIBCXZdHOFc0 WoDTgxHRzC1jgQNidyd6tKqcsVJs0dzF0oKTmFFmUAqTdJO12LBuNA1rlqrR3EtpYk8B/wtS 5dIMD7Q8hwQpL+4C6GNpb6ZKnPkLi47pDOLhz2qBrqN+rqUEsT3YnExYpzj5yOBi+FlmV1Hw 49QYe1sn2ZPs In-Reply-To: <740696.1718036463@sss.pgh.pa.us> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk On 2024-06-10 Mo 12:21, Tom Lane wrote: > Magnus Hagander writes: >> On Mon, Jun 10, 2024 at 5:03 PM Nathan Bossart >> wrote: >>> Is there a particular advantage to that approach as opposed to just using >>> "directory" mode for everything? >> A gazillion files to deal with? Much easier to work with individual custom >> files if you're moving databases around and things like that. >> Much easier to monitor eg sizes/dates if you're using it for backups. > You can always tar up the directory tree after-the-fact if you want > one file. Sure, that step's not parallelized, but I think we'd need > some non-parallelized copying to create such a file anyway. > > Yeah. I think I can probably allow for Magnus' suggestion fairly easily, but if I have to choose I'm going to go for the format that can be produced with the maximum parallelism. cheers andrew -- Andrew Dunstan EDB: https://www.enterprisedb.com