Content-Language: ru-RU
Content-Type: text/plain; charset="koi8-r"
Content-Transfer-Encoding: quoted-printable
From: =?koi8-r?B?9czBxdcg4czFy9PBzsTSIPPF0sfFxdfJ3g==?=
	<alexander.ulaev@rtlabs.ru>
To: Sergei Kornilov <sk@zsrv.org>
CC: "pgsql-bugs@lists.postgresql.org" <pgsql-bugs@lists.postgresql.org>, "PG
 Bug reporting form" <noreply@postgresql.org>
Subject: RE: Re:BUG #17392: archiver process exited with exit code 2 was
 unexpectedly cause for immediate shutdown request
Thread-Topic: Re:BUG #17392: archiver process exited with exit code 2 was
 unexpectedly cause for immediate shutdown request
Thread-Index: AQHYGJqfkx/xgV+GkkKotwEX2bSY56yBUx8AgAA6ZaD//9pSgIAANqOA
Date: Thu, 3 Feb 2022 10:29:18 +0000
Message-ID: <ca8f25eb70214fc5b320540375877c2a@rtlabs.ru>
References: <17392-ae1e272049dfec87@postgresql.org>
	 <22132351643878322@sas1-941a8864505d.qloud-c.yandex.net>
 <779598bff1e7430f9e032d1b62b8006b@rtlabs.ru>
 <17309001643882771@iva1-b50b8ed859e3.qloud-c.yandex.net>
In-Reply-To: <17309001643882771@iva1-b50b8ed859e3.qloud-c.yandex.net>
Accept-Language: ru-RU, en-US
MIME-Version: 1.0
Archived-At: 
 <https://www.postgresql.org/message-id/ca8f25eb70214fc5b320540375877c2a%40rtlabs.ru>
Precedence: bulk

We found on ETCD1 such errors in the syslog:

Feb  1 16:12:01 etcd1 etcd: got unexpected response error (etcdserver: requ=
est timed out)
Feb  1 16:12:02 etcd1 etcd: got unexpected response error (etcdserver: requ=
est timed out) [merged 1 repeated lines in 1.21s]
Feb  1 16:12:03 etcd1 etcd: got unexpected response error (etcdserver: requ=
est timed out) [merged 1 repeated lines in 1s]
Feb  1 16:12:20 etcd1 etcd: sync duration of 29.69857369s, expected less th=
an 1s
Feb  1 16:26:55 etcd1 etcd: got unexpected response error (etcdserver: requ=
est timed out)
Feb  1 16:27:03 etcd1 etcd: got unexpected response error (etcdserver: requ=
est timed out)
Feb  1 16:27:17 etcd1 etcd: sync duration of 1m0.745329542s, expected less =
than 1s

So, this problem related to SAN =3D> i\o freeze many VMs including DB's, ET=
CD nodes =3D> etcd i\o long delay affect =3D>  patroni reaction to demoted =
self postgresql instance
Sergey, Thank you much for your support!


Best regards,
Ulaev Alexander

-----Original Message-----
From: Sergei Kornilov [mailto:sk@zsrv.org]=20
Sent: Thursday, February 3, 2022 1:06 PM
To: =F5=CC=C1=C5=D7 =E1=CC=C5=CB=D3=C1=CE=C4=D2 =F3=C5=D2=C7=C5=C5=D7=C9=DE=
 <alexander.ulaev@rtlabs.ru>
Cc: pgsql-bugs@lists.postgresql.org; PG Bug reporting form <noreply@postgre=
sql.org>
Subject: Re:BUG #17392: archiver process exited with exit code 2 was unexpe=
ctedly cause for immediate shutdown request

Hello
> 2022-02-01 16:12:24,928 ERROR: failed to update leader lock
> 2022-02-01 16:12:27,063 INFO: demoted self because failed to update leade=
r lock in DCS

Between these two messages, an immediate shutdown is called: https://github=
.com/zalando/patroni/blob/v2.1.2/patroni/ha.py#L1045

Regards, Sergei