From: "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>
To: 'Kyotaro Horiguchi' <horikyota.ntt@gmail.com>, "amit.kapila16@gmail.com"
	<amit.kapila16@gmail.com>
CC: "dilipbalaut@gmail.com" <dilipbalaut@gmail.com>,
	"pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Subject: RE: Exit walsender before confirming remote flush in logical
 replication
Thread-Topic: Exit walsender before confirming remote flush in logical
 replication
Thread-Index:
 AdkVx8aq4GMGg1SCRf2FTgyqlfeu/AD/qjmAAAEe/IAAAmV+AAAj99zAAzbW2gAAgqTqgAAJEDlg
Date: Mon, 16 Jan 2023 11:08:30 +0000
Message-ID:
 <TYAPR01MB5866027B7A1FA4D07E6DC813F5C19@TYAPR01MB5866.jpnprd01.prod.outlook.com>
References:
 <CAA4eK1+pD654+XnrPugYueh7Oh22EBGTr6dA_fS0+gPiHayG9A@mail.gmail.com>
	<TYAPR01MB5866E7B0583504FBDE6B60FAF5F29@TYAPR01MB5866.jpnprd01.prod.outlook.com>
	<CAA4eK1LmCfLcu2hV_NCL9m0iQaE9=g7f8ePj79da373haFdZ-A@mail.gmail.com>
 <20230116.103153.304248162331209247.horikyota.ntt@gmail.com>
In-Reply-To: <20230116.103153.304248162331209247.horikyota.ntt@gmail.com>
Accept-Language: ja-JP, en-US
Content-Language: ja-JP
msip_labels: MSIP_Label_a7295cc1-d279-42ac-ab4d-3b0f4fece050_Enabled=true;
 MSIP_Label_a7295cc1-d279-42ac-ab4d-3b0f4fece050_SetDate=2023-01-16T05:55:38Z;
 MSIP_Label_a7295cc1-d279-42ac-ab4d-3b0f4fece050_Method=Standard;
 MSIP_Label_a7295cc1-d279-42ac-ab4d-3b0f4fece050_Name=FUJITSU-RESTRICTED?;
 MSIP_Label_a7295cc1-d279-42ac-ab4d-3b0f4fece050_SiteId=a19f121d-81e1-4858-a9d8-736e267fd4c7;
 MSIP_Label_a7295cc1-d279-42ac-ab4d-3b0f4fece050_ActionId=29d68c8c-d877-4865-b49f-6b64847c3860;
 MSIP_Label_a7295cc1-d279-42ac-ab4d-3b0f4fece050_ContentBits=0
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=fujitsu.com;
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: TYAPR01MB5866:EE_|TYWPR01MB8742:EE_
x-ms-office365-filtering-correlation-id: 8f952fe6-7c28-4b68-7c0f-08daf7b20042
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 xSjQE0pmaD25f7Ssqpljx+p4R4SRwjRCxQWEhdgLa3u7IL4mpMkbgfjBiX0cqI6XupU0/ibOWIrvpkMhScWK6ucKyOagsh4DEtFmO5eWYNQA7pEya/i3RVT94LGagRwhMKYmACSs646Wdx4ZNU+FpKIggauuI+wpto44rG7OOdOwL/v5i482loNA+HfBq/6v/N7z3hC7BstExkR1GgZ0bDBAwMDOn7ZwHkBXRu0mlfzSX8k571oHEXdTgiJDndpjwarQHJThfS2sMlCEBrPxpB9NL5j2IzTs6jTx6Tt+F7OH3DQmA6ZO0Jkk2qZvCNfdTHWGWhTHxFAgK6TW78nEy9bqYEPuNlWqPNW5eB9l9vCEgVunB8W5TzigzYG38laMFSpnMRZqA7gy6kZWrYhybAqBP24NjA4HkqvFwkNnskg4oAOhrlpEflfHQOTNs/7QPLJuRyyNDRwDRzZcjj55y4PU7JPTuSwwp1IiaTzewAT+jn11j2bG1hhKaZUrX6XNDtEMc8TJA6kjXtFZsLSfxBDut22L+F6SOMzEDPW3tJ/kP9N1/R7NVbhcunb/9V32/sq6IlQyyxjZxCtZAeFJaNDsD9N3LHKBeWkSgjB/vbBO7ZwozrEyBKRpxStxJ29hqGQjtgjlhbGb7sw48U0Jh7GF1Muopjr4/QcDlLBKGTmevqGhaBL8laRUc6AnkNa8rbyNwO3EXVonyBz7tL6gTw1ayykGfVizs09b9zVE9vP6TGLmiKClSvXFvTfJeJtilYvv0+8YphRCvzXH62SUww==
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:TYAPR01MB5866.jpnprd01.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230022)(4636009)(396003)(39860400002)(376002)(366004)(136003)(346002)(451199015)(1590799012)(38100700002)(33656002)(1580799009)(38070700005)(82960400001)(55016003)(85182001)(86362001)(966005)(66446008)(7696005)(66556008)(8676002)(4326008)(26005)(9686003)(64756008)(66476007)(186003)(478600001)(66946007)(76116006)(71200400001)(54906003)(110136005)(122000001)(6506007)(316002)(99936003)(52536014)(41300700001)(8936002)(2906002)(83380400001)(5660300002);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?iso-2022-jp?B?b2lDYlYvUzZ0cHVNd1laL0FkNThzOFdHQWxiekF4T3VTUEkrSjRKNjNS?=
 =?iso-2022-jp?B?YVBpVHUzWVFrQXBVK2tTM29YY2RTSnlIYkdBZWppMjRIVzQvZjFGOFFC?=
 =?iso-2022-jp?B?WUNrczNraXE2OWU4OXdFRWxMa2VNU2ExL2JHcFZCVzNkRVZQUm8zTTJh?=
 =?iso-2022-jp?B?emZ0NDA1cU9qVnNRQjNWY2dFUlBXRU02a3VHRDlwaW1rYUF3WDZtV2xS?=
 =?iso-2022-jp?B?Ri9lbXlJSFJqUTBTcW4xQ0o1NG9FYTBGWm5vYzUvSjV1Z2tXUjIwQmUy?=
 =?iso-2022-jp?B?cjc0UEorNjcyQXJ6N1d0emt6dWZjM0J5RGV4bDI0TmVRa0VubFMwWkJS?=
 =?iso-2022-jp?B?NTlJTlMwMzZvMTdXM21DYlMyM2F3TjgzczlCVDJpZ3ZJUzVPVWY0ME43?=
 =?iso-2022-jp?B?VytPdGE3ZC9SaEh2V2E2cHBDd250cnF0QTNEaWZGWEJTOXpGU09mZndP?=
 =?iso-2022-jp?B?YmdEdE1BU0k1TS9hZ2M1aEpEMXY3QkdGbFpqQ2xpVjd5WFJiV216aUZp?=
 =?iso-2022-jp?B?STA3WmJGSmFjbmwxSk1WQnZGNGViSEdyYS8xbmFHcWZLRTVucHZGcFBG?=
 =?iso-2022-jp?B?RWxiSFJ1RTFjUTNCODdHdTBWL1R4VWl0YStWejlmN2h1Z1RGbUFFOTU4?=
 =?iso-2022-jp?B?QkU1eXVVb0xacUcvUjl1WHdVS3dVV2ZvUVR3S29vR21Lb29JeFRwY0Jr?=
 =?iso-2022-jp?B?a0wwY0lvNDE1bytMaTZSZWxpNUEzMysvSzR1S1VWN0dSS01kZFRJRnBF?=
 =?iso-2022-jp?B?U0I3NEo3Z0tyTE9zbFFZNHhOMzJJMksybzE4RitlVU4rU1d2eVlFdTRJ?=
 =?iso-2022-jp?B?bXJLUzdTV2RBbnp2UHNzakpLZXJ3d1ZPbTlTVmtVZTJFb0NvOXJRK3V6?=
 =?iso-2022-jp?B?Q2ZVRHNIU0tPVWpOVmZSdm0zRXgzR0NiSzBkZG1TVTkrWFoyMXB0UTNq?=
 =?iso-2022-jp?B?bTRST0dobnNtQXcvdU9mS3puWjZWV2J6UHlHSmJTS3JGYkMydHVqcTd4?=
 =?iso-2022-jp?B?bUE0bnZPQ1R6SEM0QURFKzFaYVlWUmJpdHNNUUhNZTZ2T2ZUSnF5NE9q?=
 =?iso-2022-jp?B?K3Q0bzd2cTY3a3QxSWhzSkZIdTNxSWwrZUl3bGxkRzZYZnRPQnlXMUhJ?=
 =?iso-2022-jp?B?Rm1EeERxT3YrWEc3d2FHZk1WSGVXelpQeml3WEFFaytUcFM4Rzdaak1q?=
 =?iso-2022-jp?B?WEdScmYycmdySGdWcmk4WVpDN2gvL3BZWFZNRDNCQnNKb09tWHdEbHJw?=
 =?iso-2022-jp?B?M2hWM2lzKzlCdW4xZWdHOXV6NGM1N2xNcWZ3NDZqc21xZlR3SitMendu?=
 =?iso-2022-jp?B?R2FxU3BRRDFYVnUvY2ZPWUwyek1EcXNQOXVULzdLR1htWEg0NTBwYkl1?=
 =?iso-2022-jp?B?cjJMYTZvclRMTENCMEp6N2RMdVBXWStwbStNanNQTmMxSy93OE9kaVZk?=
 =?iso-2022-jp?B?S3JwdTh6RnBjdGFDcFdyUFFmWlR4QTJtZE04NmhiRHgwUjhvendSbDM5?=
 =?iso-2022-jp?B?anROTVM1NkdrL0l0MGpwNlNaalZwMDBKd1Fsb2Jma0ZNM0tldVpTaTha?=
 =?iso-2022-jp?B?aDBXa3U4b01ralJ5VUx0aFh1UTBHcjN0SHRjMW5rTFB2TS9naW02SlJM?=
 =?iso-2022-jp?B?cFUyekd4ZW5pdTNIaHYwMGFzOXhjTVlsRG5RK1JNNFFIdlVCS2U0bkJP?=
 =?iso-2022-jp?B?NVVPcUkzRUxFYzVsSXc0TTBWa2RRNm5KVTJUZ3dlcUh6R1hhNFhlMWVI?=
 =?iso-2022-jp?B?TWlRQWRyYVhRNXVHMWdlOWw1cDROREx2KzYyeXZEMXNrNFVhcnVBWUVw?=
 =?iso-2022-jp?B?bWR3NkE0dmYrb0IxOXpUVTFVUWhGQTZQTHFMdWRFNElpekVlOWdwNE5O?=
 =?iso-2022-jp?B?TGhKZ2xKclBZd1JTWEJ6K0NkTWRCUHM0SWkzWkdqRUViUWlYV2FsMTk1?=
 =?iso-2022-jp?B?UVhZWFBhcERCeU83VGoyU0o1TVlqWXVSeUhUcHlhWjhQaElneFNIZnkz?=
 =?iso-2022-jp?B?OHMrczhseUpPb25Pc1d2TjdlMmF1eGZ1YlBTOXNORzJuOHNwWDJTOVNi?=
 =?iso-2022-jp?B?MW1yTjBIclBtTTZ0eFN1ejE4N3ZGbXR1RlZtbVBFVUY5WG95WTdSYkln?=
 =?iso-2022-jp?B?cTl3TGVlMzhBbERjNXQwd2plVVBpTXJLaGNtK1BzZ1ZQUGFnOGNSalF2?=
 =?iso-2022-jp?B?VE52U2ZIOEt1OGtJT01XT211WCtzeHpnVHVIWFdSYkVmSG9OWndLSk91?=
 =?iso-2022-jp?B?WkhlZFBQRkErUkh2UG54S0hCRjVMMTkzeGlMQ2tkTVRJSDJ6dFlvRjMz?=
 =?iso-2022-jp?B?aFdtb1RzOHFIWFpydWw5L1U1VFN1clBISVE9PQ==?=
Content-Type: multipart/mixed;
	boundary="_002_TYAPR01MB5866027B7A1FA4D07E6DC813F5C19TYAPR01MB5866jpnp_"
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0:
	EEkxtHAgQHysjm+Bjw76BX851/SQzGmUgJpnLg9EDW+05UaJB+7HCKOkTw9l0EIHP77qbhuALf7FIMploKzsj5Kb9/sHgLAyOIUxCHcuSMvGzylh0lFOIZPzqJ3P+04QUckA1H4ZSONQH7z1zYXD9eml/pbXy+YXlu0uItyCAzVYYFzC3AZc/VHQD0ExMGqHhgBHDTokLlU52IcxOl1ycslzaCcMkOj6NhkhlYCk6rALA3kdsWNzwhlalk9Z1sv2PJi9J8ohLxmZcZzjtDmreskeC6g0u2GqfdfLy0UsnthFrXqZ8OwHLVrxaXfxEpVJiMe29JQbEDOHgN8YgUtscBDh3e2m4ICwAOKpt+orhO3UDUbVZ52p2xOnx5j1w3qnV1ZSxpnfbf6c3yfOH6CKtBvLl1yik/7eYonHFMKp+qsQGTPKxz7yvzMCgboNgFImHNX1S9n/pjMXbFxdJ2gWafKmcup04p6Xo7kCP83jOHObw/NWPpryOv90JZjoi2GY+de7hvKqk2vPmYATB5A0gVeu2tRNfL8BfV2+wj3YqvOLHSJ3uQVaoGKQTp+XPKhD8Gflpgw07xjYMJmpUDFLR0aJ7BDmSw8PYkvXNpaRCn3oTLHB4EEs96txtR8/SeCjNweNXoYw9gTnWtHd6vVmZx1VCoABvwmjD+Ptrcj+lsXU1L3dpZkSqI0U1r7jDPy0HxWfkNDODdjDLPRZAv0wYtbiFq1/0wIVFJ/xYoFhvLQt0ICn/bZ5pjeMTTaobtXTktaof7g9xBHsYSJkIUse0l78DbiLsF2HyF4UhfY1UhA=
X-OriginatorOrg: fujitsu.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: TYAPR01MB5866.jpnprd01.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 8f952fe6-7c28-4b68-7c0f-08daf7b20042
X-MS-Exchange-CrossTenant-originalarrivaltime: 16 Jan 2023 11:08:30.6786
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: a19f121d-81e1-4858-a9d8-736e267fd4c7
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: uPUzxCrlwOaMTn21n0DFDPyAVXNUYsBK4R9Lmg43fJZ+uxJhxeNFJ4AeiaXpDcmyXD0U5ZuRny6jPdiM+mENBBYQsyo2nt8zaNLdAlJMqOM=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: TYWPR01MB8742
List-Id: <pgsql-hackers.lists.postgresql.org>
List-Help: <https://lists.postgresql.org/manage/>
List-Subscribe: <https://lists.postgresql.org/manage/>
List-Post: <mailto:pgsql-hackers@lists.postgresql.org>
List-Owner: <mailto:pgsql-hackers-owner@lists.postgresql.org>
List-Archive: <https://www.postgresql.org/list/pgsql-hackers>
Archived-At: <https://www.postgresql.org/message-id/TYAPR01MB5866027B7A1FA4D07E6DC813F5C19%40TYAPR01MB5866.jpnprd01.prod.outlook.com>
Precedence: bulk

--_002_TYAPR01MB5866027B7A1FA4D07E6DC813F5C19TYAPR01MB5866jpnp_
Content-Type: text/plain; charset="iso-2022-jp"
Content-Transfer-Encoding: quoted-printable

Dear Horiguchi-san, Amit,

> At Fri, 13 Jan 2023 16:41:08 +0530, Amit Kapila <amit.kapila16@gmail.com>
> wrote in
> > Okay, but what happens in the case of physical replication when
> > synchronous_commit =3D remote_apply? In that case, won't it ensure that
> > apply has also happened? If so, then shouldn't the time delay feature
> > also cause a similar problem for physical replication as well?
>=20
> As written in another mail, WalSndDone doesn't honor
> synchronous_commit. In other words, AFAIS walsender finishes not
> waiting remote_apply.  The unapplied recods will be applied at the
> next startup.
>=20
> I didn't confirmed that behavior for myself, though..

If Amit wanted to say about the case that sending data is pending in physic=
al
replication, the walsender cannot stop. But this is not related with the
synchronous_commit: it is caused because it must sweep all pending data bef=
ore
shutting down. We can reproduce the situation with:

1. build streaming replication system
2. kill -STOP $walreceiver
3. insert data to primary server
4. try to stop the primary server

If what you said was not related with pending data, walsender can be stoppe=
d even
if the synchronous_commit =3D remote_apply. As Horiguchi-san said, such a c=
ondition
is not written in WalSndDone() [1]. I think the parameter synchronous_commi=
t does
not affect walsender process so well. It just define when backend returns t=
he
result to client.

I could check by following steps:

1. built streaming replication system. PSA the script to follow that.

Primary config.

```
synchronous_commit =3D 'remote_apply'
synchronous_standby_names =3D 'secondary'
```

Secondary config.

```
recovery_min_apply_delay =3D 1d
primary_conninfo =3D 'user=3Dpostgres port=3D$port_N1 application_name=3Dse=
condary'
hot_standby =3D on
```

2. inserted data to primary. This waited the remote apply

psql -U postgres -p $port_primary -c "INSERT INTO tbl SELECT generate_serie=
s(1, 5000)"

3. Stopped the primary server from another terminal. It could be done.
The terminal on step2 said like:

```
WARNING:  canceling the wait for synchronous replication and terminating co=
nnection due to administrator command
DETAIL:  The transaction has already committed locally, but might not have =
been replicated to the standby.
server closed the connection unexpectedly
        This probably means the server terminated abnormally
        before or while processing the request.
connection to server was lost
```

[1]: https://github.com/postgres/postgres/blob/master/src/backend/replicati=
on/walsender.c#L3121

Best Regards,
Hayato Kuroda
FUJITSU LIMITED


--_002_TYAPR01MB5866027B7A1FA4D07E6DC813F5C19TYAPR01MB5866jpnp_
Content-Type: application/octet-stream; name="test_phy.sh"
Content-Description: test_phy.sh
Content-Disposition: attachment; filename="test_phy.sh"; size=1263;
	creation-date="Mon, 16 Jan 2023 07:08:08 GMT";
	modification-date="Mon, 16 Jan 2023 11:08:29 GMT"
Content-Transfer-Encoding: base64

IyEvYmluL2Jhc2gKCnBvcnRfTjE9NTQzMQpwb3J0X04yPTU0MzIKCmVjaG8gJz09PT09PT09PT0n
CmVjaG8gJz1DbGVhbiB1cD0nCmVjaG8gJz09PT09PT09PT0nCgpwZ19jdGwgc3RvcCAtRCBkYXRh
X3ByaW1hcnkKcGdfY3RsIHN0b3AgLUQgZGF0YV9zZWNvbmRhcnkKCnJtIC1yIGRhdGFfcHJpbWFy
eSBkYXRhX3NlY29uZGFyeSBwcmltYXJ5LmxvZyBzZWNvbmRhcnkubG9nCgplY2hvICc9PT09PT09
PT09PT09PT09PT09PT09PScKZWNobyAnPVNldCB1cCBwcmltYXJ5IHNlcnZlcj0nCmVjaG8gJz09
PT09PT09PT09PT09PT09PT09PT09JwoKaW5pdGRiIC1EIGRhdGFfcHJpbWFyeSAtVSBwb3N0Z3Jl
cwoKY2F0IDw8IEVPRiA+PiBkYXRhX3ByaW1hcnkvcG9zdGdyZXNxbC5jb25mCmxpc3Rlbl9hZGRy
ZXNzZXMgPSAnKicKcG9ydCA9ICRwb3J0X04xCnN5bmNocm9ub3VzX2NvbW1pdCA9ICdyZW1vdGVf
YXBwbHknCkVPRgoKY2F0IDw8IEVPRiA+PiBkYXRhX3ByaW1hcnkvcGdfaGJhLmNvbmYKaG9zdCBh
bGwscmVwbGljYXRpb24gYWxsIDAuMC4wLjAvMCB0cnVzdApFT0YKCnBnX2N0bCAtRCBkYXRhX3By
aW1hcnkgc3RhcnQgLXcgLWwgcHJpbWFyeS5sb2cKcHNxbCAtVSBwb3N0Z3JlcyAtcCAkcG9ydF9O
MSAtYyAiQ1JFQVRFIFRBQkxFIHRibCAoaWQgaW50IHByaW1hcnkga2V5KTsiCgplY2hvICc9PT09
PT09PT09PT09PT09PT09PT09PT09JwplY2hvICc9U2V0IHVwIHNlY29uZGFyeSBzZXJ2ZXI9Jwpl
Y2hvICc9PT09PT09PT09PT09PT09PT09PT09PT09JwoKCnBnX2Jhc2ViYWNrdXAgLUQgZGF0YV9z
ZWNvbmRhcnkgLVUgcG9zdGdyZXMgLXAgJHBvcnRfTjEKCmNhdCA8PCBFT0YgPj4gZGF0YV9zZWNv
bmRhcnkvcG9zdGdyZXNxbC5jb25mCnBvcnQgPSAkcG9ydF9OMgpyZWNvdmVyeV9taW5fYXBwbHlf
ZGVsYXkgPSAxZApwcmltYXJ5X2Nvbm5pbmZvID0gJ3VzZXI9cG9zdGdyZXMgcG9ydD0kcG9ydF9O
MSBhcHBsaWNhdGlvbl9uYW1lPXNlY29uZGFyeScKaG90X3N0YW5kYnkgPSBvbgpFT0YKCmNhdCA8
PCBFT0YgPj4gZGF0YV9zZWNvbmRhcnkvc3RhbmRieS5zaWduYWwKRU9GCgpjYXQgPDwgRU9GID4+
IGRhdGFfcHJpbWFyeS9wb3N0Z3Jlc3FsLmNvbmYKc3luY2hyb25vdXNfc3RhbmRieV9uYW1lcyA9
ICdzZWNvbmRhcnknCkVPRgoKCnBnX2N0bCAtRCBkYXRhX3ByaW1hcnkgICByZXN0YXJ0IC13IC1s
IHByaW1hcnkubG9nCnBnX2N0bCAtRCBkYXRhX3NlY29uZGFyeSBzdGFydCAgIC13IC1sIHNlY29u
ZGFyeS5sb2cK

--_002_TYAPR01MB5866027B7A1FA4D07E6DC813F5C19TYAPR01MB5866jpnp_--