Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1sGfmp-00FmPb-Ga for pgsql-hackers@arkaria.postgresql.org; Mon, 10 Jun 2024 14:14:16 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1sGfmn-00Cfvp-5U for pgsql-hackers@arkaria.postgresql.org; Mon, 10 Jun 2024 14:14:14 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1sGfmm-00Cfvh-SN for pgsql-hackers@lists.postgresql.org; Mon, 10 Jun 2024 14:14:13 +0000 Received: from mail-io1-xd2a.google.com ([2607:f8b0:4864:20::d2a]) by magus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94.2) (envelope-from ) id 1sGfml-000wff-Ln for pgsql-hackers@postgresql.org; Mon, 10 Jun 2024 14:14:13 +0000 Received: by mail-io1-xd2a.google.com with SMTP id ca18e2360f4ac-7ea7bcc72caso182339f.0 for ; Mon, 10 Jun 2024 07:14:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1718028850; x=1718633650; darn=postgresql.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=FJ607GZBVW797HqBoHeDKvlMdNPTpFuC9yKpdhBViRE=; b=JCTEPQJoFGKUIOtbDsT0kfEvNmILfp1yxyC78MWNpoczctzT6Vl/0JcvBQX6V3x9JH hGnPVt2FxzDt32QSPe+WL7r8dchGpidj+oBq/jghvUhmXOTxn/1Ok5uqjKgYgdnKYQ98 CIqyjjtndxtQTuZMvtADprGIMUyLQjqljFrRGq4bfr2BLEbgp7KvSQ4ukYBekAd6IStw S22ln4KCFjk4vT9D5Q+e1b3UwRdAWImD5e3rhbSUo9FAVcQka9SepmBSfdz9JZlNG4CL J/NAES4ZixanEiL+iqIr0S41cF8jLVPcWNFt9GD70RmYBww+4VUe3vcjJ/pIw88iGMpD sMAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718028850; x=1718633650; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=FJ607GZBVW797HqBoHeDKvlMdNPTpFuC9yKpdhBViRE=; b=bewzz9rV2GIN1Q92rF10cqJdTJu0cryFeHwoINw1l/MqvtXQ0heHQSp9C24mNLv759 dZKnabC3CGHDF5Mrv7WiaoLgFngG5lfgSLI4CoeOO0ED34qvp3avSAeoUhpTsvs1krA1 qI3M8hIUScD7arsArwi53lS73mD2Pf1VZ4XIijoWVOz+9aNEjxgn8Zby6ROGnF135b4s njnj0I42x80z1Q7xG0TP30eQlsIjn04nXPThwHPrBzOliCzD9ZjEXV3eKQlcwxmzuQ6w 8vhTwbqVCY5g3LQp7M91q9z2mXNsnaoFz7tfuYifaBso5VSd3oIbkaSM0qtrwDRha7sT Ptog== X-Gm-Message-State: AOJu0YwFxi/I94PT9PgR5pf9zqr/CXC5oum983wKTBE+G/eY2Bqs7jE9 OiXtfBGDR/TirNnYnmK4Xv0ZmHUK6SB32ZSeyuCYYHDfXMNRo+sRLHGldw== X-Google-Smtp-Source: AGHT+IEQQs9IOtE3yxwya9+wv2ZA9/NZhZNhUJjDpH2vAOpD5vr3TSG5iQ5fBeO9YKmwoTDt6w8RFA== X-Received: by 2002:a05:6602:341e:b0:7eb:7bc9:7fcb with SMTP id ca18e2360f4ac-7eb7bc982a7mr592452439f.3.1718028850205; Mon, 10 Jun 2024 07:14:10 -0700 (PDT) Received: from nathan (162-195-168-172.lightspeed.stlsmo.sbcglobal.net. [162.195.168.172]) by smtp.gmail.com with ESMTPSA id ca18e2360f4ac-7ebb1882a22sm1005639f.55.2024.06.10.07.14.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Jun 2024 07:14:09 -0700 (PDT) Date: Mon, 10 Jun 2024 09:14:08 -0500 From: Nathan Bossart To: Andrew Dunstan Cc: PostgreSQL-development Subject: Re: Non-text mode for pg_dumpall Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk On Mon, Jun 10, 2024 at 08:58:49AM -0400, Andrew Dunstan wrote: > Tom and Nathan opined recently that providing for non-text mode for > pg_dumpall would be a Good Thing (TM). Not having it has been a > long-standing complaint, so I've decided to give it a go. Thank you! > I think we would need to restrict it to directory mode, at least to begin > with. I would have a toc.dat with a different magic block (say "PGGLO" > instead of "PGDMP") containing the global entries (roles, tablespaces, > databases). Then for each database there would be a subdirectory (named for > its toc entry) with a standard directory mode dump for that database. These > could be generated in parallel (possibly by pg_dumpall calling pg_dump for > each database). pg_restore on detecting a global type toc.data would restore > the globals and then each of the databases (again possibly in parallel). I'm curious why we couldn't also support the "custom" format. > Following that I would turn my attention to using it in pg_upgrade. +1 -- nathan