Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1t95Te-007hBB-Dk for pgsql-admin@arkaria.postgresql.org; Thu, 07 Nov 2024 16:35:22 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1t95Tb-00HGO6-SG for pgsql-admin@arkaria.postgresql.org; Thu, 07 Nov 2024 16:35:20 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1t95Ta-00HGNx-SG for pgsql-admin@lists.postgresql.org; Thu, 07 Nov 2024 16:35:20 +0000 Received: from mail-southcentralusazon11010036.outbound.protection.outlook.com ([40.93.193.36] helo=SA9PR09CU002.outbound.protection.outlook.com) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1t95TY-000f27-8z for pgsql-admin@postgresql.org; Thu, 07 Nov 2024 16:35:18 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=pwxWLMVuMSIbXsP1b3voxRiyY7h4TExxUL26hj8Dnk96dj5vfUoNYE5oQVca+Y9xgSmJ3cOYi+/f49RKSvmijSH8m8N8aVXrHzQKYIUm12bMl8hfdR7na1vtaB3SFhFTHeRkN66/GHaxNfAlrnOSUhcvzZMnNQuLWkOl0npfrDDzxyKzynLY/AKq5iwU0y3yeCBnTrPXls/GSsIbtx5n3sqf+JHVJ7/GSrxgwSfTSSYeS4/8YwBR4cMStAZ/M2D01pUqHVYMr8N+1YsvPrRHfvjAVcx9Dpmd61n4qZz3zJ79X0TUj0cTm1lW8qj6fAYFndZX3YIqx8+9hvV+YgB6jg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=trxsPG1sm+mSZ5xwUL2W4Xv1avK2J9eU7KKIkQ4wZHU=; b=OWnRF+FyckV89rI6kGfdRTL78kWLlE+pdpSVN2aJ15pgY4REpCmBQnaEj0ID+BPtiTlDhRKSZML28eERmmAsym0LDbWL9kMhiuhaJafZD8unVr4es3v2+FvGOIc15eb0AOFTX1vMgJbKpSAieFVAvzssCXEaiaMI5h3c85SF8J0UBqVCv9PnVN4fFykArxp073JixBVwXxFI6loUev9OAlCCZLUyOo49uRFJ81+D/AwcC1ndrGqdojkK4lPk0VpdxATM/3us7ZFOkuLIj35+3+uRjedIDI990elhd6RiGycXNB3GMdGNu8g87BqCq0gJ+Bte2yAvCAU5q1dq87F98g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=fnal.gov; dmarc=pass action=none header.from=fnal.gov; dkim=pass header.d=fnal.gov; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fnal.gov; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=trxsPG1sm+mSZ5xwUL2W4Xv1avK2J9eU7KKIkQ4wZHU=; b=UJgOjWnuxaY0IoZo4bRvJ1WD1odkegY7dVT3W6q4aH9E9rK3wgE7bxu9jaTzqnagPgvqMrmm2HXkQ7POVoma17Z4Nq9iALDZSEb3NDx8H3UNtDAMxCN/9PhaoJJMdYBgmACRPJFjlkdJH7seGAYOCvAGJ7WokYpfGzT4fJQiTi9X/nj4XyzPRc4zVaVVK6dvERd22QlKldYLJoxLgFKSU/VAvVfnHQdsTsEbrMGdYAC+CJ9ixUM6sQfH8cHd5xftnmBrOp0lDVSLWkDz3UwyHT2E6RyaQdBH5s2lMDGAgYi9/FdwD4haiRjVXL0gwpY89UIH/38wHLAT7zeGO0We4A== Received: from DM8PR09MB6677.namprd09.prod.outlook.com (2603:10b6:5:2ee::12) by BLAPR09MB7426.namprd09.prod.outlook.com (2603:10b6:208:28b::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8137.21; Thu, 7 Nov 2024 16:35:10 +0000 Received: from DM8PR09MB6677.namprd09.prod.outlook.com ([fe80::cdab:8e31:245:e366]) by DM8PR09MB6677.namprd09.prod.outlook.com ([fe80::cdab:8e31:245:e366%4]) with mapi id 15.20.8114.028; Thu, 7 Nov 2024 16:35:10 +0000 From: Murthy Nunna To: "pgsql-admin@postgresql.org" Subject: Running rsync backups in pg15 Thread-Topic: Running rsync backups in pg15 Thread-Index: AdsxMMB3rygxFbtfQHGrPyr/13wYDQ== Date: Thu, 7 Nov 2024 16:35:10 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=fnal.gov; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: DM8PR09MB6677:EE_|BLAPR09MB7426:EE_ x-ms-office365-filtering-correlation-id: 3cee1b84-e6c0-477d-03aa-08dcff4a25c8 x-fermilab-ob: 1 x-fermilab-sender-location: inside x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0;ARA:13230040|1800799024|366016|38070700018|8096899003; x-microsoft-antispam-message-info: =?us-ascii?Q?Gts7uplU7Si0xesxQARG8CLUk1/mby4v1069ctW8BJoHWmLKr81szGd/MWDj?= =?us-ascii?Q?CXnJcXW2PTE4ynjUhx00kgpcH+u4h5OPrQ962icwf+Wdgd9vSD51b8Ar1QBV?= =?us-ascii?Q?Y/MAK+mXMGbGOCkd5wRmFoBUI/U43s41WH70S2v45XNP5M1YpjxQecohDFzb?= =?us-ascii?Q?r1KtSe868fSyjj9VzAf+YwlaiLTSyswJbl8+MFBOtnxTHis0YGTpHEEBX8qg?= =?us-ascii?Q?XMxE/B+iLaK4Ja7ODDMsJ763EDb/U/ex61Kk/QORaiAWQIXdWVDFby0auWNB?= =?us-ascii?Q?mk30qklQvhwgBj6eTAuPOUtbQX4OcjRFTDgxXKh2b6HCjy4UioNgYOf7GHLl?= =?us-ascii?Q?KUCa/cUXDZd7+UqJD8C9cBWW9B6tGy2/dPrufc9N76dqqVg747Udqvjoju25?= =?us-ascii?Q?td7Jn0cy3mnvJFRf3YKcO4T3oYZt/uMiqcaCQjbir9aW+2PXudWFiiWdVv2s?= =?us-ascii?Q?45zfSXbmnZlaAlb7JYFePXLckqZgErLybFxbdXldVsroNMSLkQ1r89/HHv2Y?= =?us-ascii?Q?MJ8NUsvNCJ/0jDJflY2f+P5cpn8vmYo21l+i9JwE804pbeyuQAyfzlNGljqF?= =?us-ascii?Q?eEBCx3RYgUerB4m4BIMP1EyytKydx1T8Yx6t6S29SglpdcID6LYAjfSDC4OL?= =?us-ascii?Q?NQ+gyTm7o7p18gpRYjPb2tYoV9FZSA3HZ8MU71416Iu9eGu9Pa8qoSo+IQDc?= =?us-ascii?Q?uToAsMO1FdNRR8o4jMwUXt+rc8o67IFbOzRLSYRjz/DcXQEmAxsti6b7J7dk?= =?us-ascii?Q?rpcQE3BPPxMG1gJv5N/4O0APt/yihV0K9xl6+XmfBewJ/47o+9RQZiCtILhw?= =?us-ascii?Q?pHPKmoaMChKRs0YTvEVEv8N7+PjLdy5/+x1KZcr4ShgdeJ0aNugmeV92ICZ8?= =?us-ascii?Q?YQBBBTJS1SsIna+4CmtMnPNOzKYRxbXv1BNzGtppFA8qSlg7uqaqz2TO5aUY?= =?us-ascii?Q?K5QPKWsG6tibzLCSiMWZcZZjjVFWLLwya4v/958wQ4DBex5WKF31fRp3Kk4c?= =?us-ascii?Q?rQ0bl57/AIFZNwqz6xCzRKk52UyJ/yu02PkRjAsBFVuKsWdh0/fnru2Qtebq?= =?us-ascii?Q?8lp90XGOEXXNcjiUserSIVcyHL1EU4jKTDOsUv4unTTiZimOKlH44JJlz4zk?= =?us-ascii?Q?QJ+LsYAf5Hpun24LTWAp0zuEQOSo439yPoQP8o4G3VeBsqfEcCyxZvgXf1r6?= =?us-ascii?Q?EDTFWvOn6DnlQuXMZJ7uhJTj+0V1Uqwb7I+HsLg4PhuugSuEYUUDVXjWjzlH?= =?us-ascii?Q?dhTwrTrWzFhkYjm8bP/d6iGwCzT6VGWWDS8nwgz636voaiqSV3PdBMz/jINI?= =?us-ascii?Q?XcBd0e833cL8kvFM9OVrxnwD+2uQmoKDTxiMgDvoGRdDbXadUn6884Q7xi3n?= =?us-ascii?Q?13pc0Hs=3D?= x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM8PR09MB6677.namprd09.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(366016)(38070700018)(8096899003);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?ykfVHsTVejLBhVOTMR4orEoZzDQgjzQeBHY3uAOxya56S/hWZ62AC5UiF/AM?= =?us-ascii?Q?WjcWt4UW/qwh7buJmGGOI6AGsUCCnPJRqY8v/smigxmf13tO0d++wqRPJ8HF?= =?us-ascii?Q?+jRsFM89mNWevM+MLiyxDWBkhVvXBiJLB4IqOX1gUtdAxmyzLX/RMGDpOP43?= =?us-ascii?Q?E8po9mwRadIWVnb7g22r3ArmpErBet603yJH1Qk4Wq9kvqFXDPBkHnCBjMpx?= =?us-ascii?Q?DcHc6FWgH729ho+L0pQMAeQch1XUsK1RslBAEjXih+y1Wtyr+cHsoM/73jd+?= =?us-ascii?Q?jbYYEPDHWmSBkI2zfHWvEiobFGr7IqLDs54NPTKJevwWECzyo1wlwZ/c0uPU?= =?us-ascii?Q?j+mztSYGNmjNTTdU2N72p5Yqyg49X56UvB+MNom9orQKmzZM6UOWcLjiZTAR?= =?us-ascii?Q?pfsi17XugqljJEPTFOfwef8A/nOaOIksy3OQO71QCj6z2EfBZr3Kax1SVUvK?= =?us-ascii?Q?BN8zcBskcG3sneQE4U5wN42aDSyQXZO+P/98JFBjw6hnTkTCG7lIsWM0W78s?= =?us-ascii?Q?8wxpFDMh7PR4nfo6hrv1S2GA1yKRAp/CJVAMDSjYVa8kZ+KIcYtMVXIvw7YC?= =?us-ascii?Q?4BonebTC+GeFzjr/7jO2p4nU7JPBM77y4Q8s+V+82zcA2RltJNj6CtR5oCFA?= =?us-ascii?Q?gVPI0tEBZKrAWn5NDebK2ylrdQ5yjiADkUfYliLsfYGJlA3Ti575ZUaJ4PiA?= =?us-ascii?Q?G9BiiGnJUDle13d645q0Pe5YSrjZ3ZHK6CSWZyyZVz94rK1Lb6X9X+w48FMm?= =?us-ascii?Q?lNz0U9zG0uYRsjIMewz28xEbW2FN78sY5UYWuRyquE5yVj0kBnEhuyVC+x5l?= =?us-ascii?Q?33zYQ/cc0s52BwBCRQoipsSovlzLmSOR5C/pptDX6Yn9a+HMdmzKt/FW6wUQ?= =?us-ascii?Q?fVhwspP4zULL0Gg+nJhQAu1nDB2bOFwoRE41fLFUQc0/8B7c9WJpeY+iSCrE?= =?us-ascii?Q?WhCq6/4jorxqmHRlRxgmWovqZyQHgSMgp0/6TsQoYJzSE4yFj7S8DWlwxQd/?= =?us-ascii?Q?hAJxLe8cFjNjhSqsvRYdA/K81+2WKPPOHu18AI9ximHAS+wx9cA+dIvkM0X5?= =?us-ascii?Q?+PtjZy/8CUjmA/iN1i8avy4lJUoJz6rUxYNfr7QrGfpQaw6BHe+AFbOu4Njy?= =?us-ascii?Q?3IIu12HPwITOpLSaZelspeSj7r6jwFuw78yiff1yVx6Ry0KDF5OhhAU6SelR?= =?us-ascii?Q?DGG6+iRVDiSINwrXdduD1WxF/Ysh4VT4Hsm7VkmpEmCDwyZTPaM9UNB4rA9W?= =?us-ascii?Q?NjpqxQ1T1wBGmaY5UrrCu/4neD6E65saq4XqUoPvO2gJtoSHvQi/z3ydusu9?= =?us-ascii?Q?a8NFuy21opOq5iRMXa0Ysgq9WOGkQS9UInRayxJImT2Hy0FQvrJOGtLNSMmO?= =?us-ascii?Q?kd1O0YKzgrTEFoAIzRh2gEwIovlhVAyPA1NeuIp3IZfKFhhx2BXj6Cocm8OP?= =?us-ascii?Q?Mn8lYp4B0Nfhij27Er2MXvkZMyBoptfTO4LqiLm2KLxqhLg6jgMdnhwArPvS?= =?us-ascii?Q?h4WEevtuN+fSQKROirT1VfkZ7voUdKvl5fvX1gXGJ9f5dpdDmjMbTIdnmaDv?= =?us-ascii?Q?AfPUtRMOBdY3U8qLapH0Tpb2EB/qiXMN5EvmmDx7?= Content-Type: multipart/alternative; boundary="_000_DM8PR09MB667714AB7E04AB70B4B5A7B1B85C2DM8PR09MB6677namp_" MIME-Version: 1.0 X-OriginatorOrg: fnal.gov X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM8PR09MB6677.namprd09.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3cee1b84-e6c0-477d-03aa-08dcff4a25c8 X-MS-Exchange-CrossTenant-originalarrivaltime: 07 Nov 2024 16:35:10.5948 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 9d5f83d3-d338-4fd3-b1c9-b7d94d70255a X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLAPR09MB7426 List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --_000_DM8PR09MB667714AB7E04AB70B4B5A7B1B85C2DM8PR09MB6677namp_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi, In PG14 and earlier, there is no requirement to keep database connection wh= ile rsync is in progress. However, there is a change in PG15+ that requires= rsync to be while we have the same database session open that executes SEL= ECT pg_backup_start('label'). This change requires a rewrite of existing sc= ripts we have. Currently (pg14): In bash script (run from cron) 1. psql Select pg_start_backup 2. rsync 3. psql Select pg_stop_backup In pg15 and later: In bash script (run from cron) psql Select pg_start_backup ! run-rsync-script Select pg_stop_backup It can be done, but it makes it ugly to check errors and so forth that occu= r in the rsync script. Anybody found an elegant way of doing this? Thank you in advance for your ideas. --_000_DM8PR09MB667714AB7E04AB70B4B5A7B1B85C2DM8PR09MB6677namp_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi,

 

In PG14 and earlier= , there is no requirement to keep database connection while rsync is in pro= gress. However, there is a change in PG15+ that requires rsync to be while = we have the same database session open that executes SELECT pg_backup_start('label'). This change requires= a rewrite of existing scripts we have.

 

Currently (pg14):

 

        &nbs= p;       In bash script (run from cron)<= /o:p>

  1. psql Select pg_start_backup
  2. rsync=
  3. psql Select pg_stop_backup

 =

In pg15 and later:

 

In bash script (r= un from cron)

 =

psql

Select pg_start_b= ackup

! run-rsync-scrip= t

Select pg_stop_ba= ckup

 

It can be done, but it makes it ugly to check errors= and so forth that occur in the rsync script.

 

Anybody found an elegant way of doing this?

 

Thank you in advance for your ideas.

 

 

--_000_DM8PR09MB667714AB7E04AB70B4B5A7B1B85C2DM8PR09MB6677namp_--