Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1gPLcS-0004ak-HM for pgsql-admin@arkaria.postgresql.org; Wed, 21 Nov 2018 06:04:12 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1gPLcQ-0001PR-7y for pgsql-admin@arkaria.postgresql.org; Wed, 21 Nov 2018 06:04:10 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1gPLcP-0001PK-Sy for pgsql-admin@lists.postgresql.org; Wed, 21 Nov 2018 06:04:10 +0000 Received: from hermes.gatewaynet.com ([193.92.121.98]) by makus.postgresql.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1gPLcM-0000Zk-0v for pgsql-admin@lists.postgresql.org; Wed, 21 Nov 2018 06:04:08 +0000 Received: from hercules.internal.net (hercules.internal.net [10.9.200.8]) by hermes.gatewaynet.com (8.14.3/8.14.3/Debian-9.4) with ESMTP id wAL641II015294 for ; Wed, 21 Nov 2018 08:04:01 +0200 Received: from [10.9.251.8] (dcmchgw.internal.net [10.9.0.1]) by hercules.internal.net (Postfix) with ESMTPS id 67806135074D for ; Wed, 21 Nov 2018 08:04:00 +0200 (EET) Subject: Re: PostgreSQL 10.5 : Logical replication timeout results in PANIC in pg_wal "No space left on device" To: pgsql-admin@lists.postgresql.org References: <838a3914-a078-88eb-ee5e-4059b7d14620@matrix.gatewaynet.com> <0BBC4BB3-4A59-4040-B065-82D09268A4F7@crazybean.net> <48415b2a-ea0f-cf5e-1145-17e8797a6e79@matrix.gatewaynet.com> <739822AC-E20F-493B-8B4F-8AF78A8276D0@crazybean.net> <946FBD41-7BAF-4414-8FB9-C0F16A0680F6@crazybean.net> <14A9158A-B487-4DB6-84FC-0B04F08D69A2@crazybean.net> From: Achilleas Mantzios Message-ID: Date: Wed, 21 Nov 2018 08:03:57 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.2.1 MIME-Version: 1.0 In-Reply-To: <14A9158A-B487-4DB6-84FC-0B04F08D69A2@crazybean.net> Content-Type: multipart/alternative; boundary="------------03A61DF90617CE1E116E2E4E" Content-Language: en-US X-Virus-Scanned: clamav-milter 0.98.7 at hermes X-Virus-Status: Clean X-Spam-Status: No, score=-26.0 required=5.0 tests=ALL_TRUSTED,DYN_HAM_IT, DYN_HAM_IT_PARTNERS,HTML_MESSAGE autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on hermes.gatewaynet.com List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk This is a multi-part message in MIME format. --------------03A61DF90617CE1E116E2E4E Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit On 20/11/18 10:48 μ.μ., Rui DeSousa wrote: > > >> On Nov 20, 2018, at 3:34 PM, Achilleas Mantzios >> > >> wrote: >> >> Hey, I was reading the docs, it seems it means : >> >> net.ipv4.tcp_keepalive_time + net.ipv4.tcp_keepalive_intvl * >> net.ipv4.tcp_keepalive_probes = 2hrs 11 Mins 15 Secs, rather than 18 Hrs > > Yeah, that’s correct.  I wonder why it didn’t terminate. Most probably because there was another created clone, cloud migration magic, that's my theory, albeit not confirmed by the provider. Logical worker (walreceiver) was still alive and happy even after the primary crushed. I have the logs from the other standby and it immediately detected the problem (PANIC on the primary) and retried. No firewall dropping packets, in every test I did, the logical bgworker detects any problems *instantly*, and retries after 5 secs by default. --------------03A61DF90617CE1E116E2E4E Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit


On 20/11/18 10:48 μ.μ., Rui DeSousa wrote:


On Nov 20, 2018, at 3:34 PM, Achilleas Mantzios <achill@matrix.gatewaynet.com> wrote:

Hey, I was reading the docs, it seems it means :

net.ipv4.tcp_keepalive_time + net.ipv4.tcp_keepalive_intvl * net.ipv4.tcp_keepalive_probes = 2hrs 11 Mins 15 Secs, rather than 18 Hrs

Yeah, that’s correct.  I wonder why it didn’t terminate.


Most probably because there was another created clone, cloud migration magic, that's my theory, albeit not confirmed by the provider. Logical worker (walreceiver) was still alive and happy even after the primary crushed. I have the logs from the other standby and it immediately detected the problem (PANIC on the primary) and retried. No firewall dropping packets, in every test I did, the logical bgworker detects any problems *instantly*, and retries after 5 secs by default.
--------------03A61DF90617CE1E116E2E4E--