Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kqEMO-0004HK-Bo for pgsql-admin@arkaria.postgresql.org; Fri, 18 Dec 2020 11:55:48 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1kqEMK-0008Br-AJ for pgsql-admin@arkaria.postgresql.org; Fri, 18 Dec 2020 11:55:44 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kqEMK-0008Bj-32 for pgsql-admin@lists.postgresql.org; Fri, 18 Dec 2020 11:55:44 +0000 Received: from mail-ed1-x52a.google.com ([2a00:1450:4864:20::52a]) by magus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1kqEMD-0006Ll-4p for pgsql-admin@lists.postgresql.org; Fri, 18 Dec 2020 11:55:43 +0000 Received: by mail-ed1-x52a.google.com with SMTP id r5so2034159eda.12 for ; Fri, 18 Dec 2020 03:55:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cybertec-at.20150623.gappssmtp.com; s=20150623; h=message-id:subject:from:to:date:in-reply-to:references:user-agent :mime-version:content-transfer-encoding; bh=RxZ+wKu2otohoBKcmhFg9KBcx0SYy5w9VnW6j9fmIL0=; b=mRRHcPCMGTOeFS9zg2DuNCeFC2Iapo298JJUuCKiX0DOgnx9zzFKZg0zv9wV/TBEF8 FSnJ4Ci6CGlRMwWELDPpt7/TgQW00KzjJQd3G5Qw528BFai0gWE/3kWRSwoJ8ALwY1I/ iXJN7s7YBcVdb0e1a6tu14+c2vILZ3jM6U1zgEQBOhbGC5LygkGin2YkD26CVadv3cg7 uoJ11qj/LW1S1AcIv1A70UAe7etVGjDhASl8ZsQ4F24U/zJZnoJZ09TEPToeekeEEe8R 0WCuJxQlvUkRPfPzTc23OKmgDP2EzBhNH0BHPmwM2I+9qF//iDPedR3ppyuMoHT54jr6 +DOA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:subject:from:to:date:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=RxZ+wKu2otohoBKcmhFg9KBcx0SYy5w9VnW6j9fmIL0=; b=N2+R87L8wHb2dvBXVK9uGOTV/Wf5RCri/ai5SiWhwO2ZczRV9Xz+fdnRT2jOUPJKWs BqayohDkXnBvr5xDP8mo+JMLt8xhc2TK/sbfvgkHb8qxMLN+bgN+a4fjSbH3l12ux1c0 shLrClCgdn/TAJa7D9euaG0TNKVgyft67pNzHxC7s61ve2WIjTxPpqo7Kci2t89T3fCY RUjnjbfMoOvWhvOimwZrLRlslVZPbcxdi1+Jp5ThIl3tWjFrxTfoSLb1Rp9HsegvTfcM fLECLYKInBLvKtVnjjOKXVizf3/xkjVJcZMaxlA+ZfMQjPWHLOt6wg4vMhHj5SjXDzLj X7yg== X-Gm-Message-State: AOAM530P0XPp7CZaT5/lwcPqStBzc7ho5RjIEFny2iiid5K2p/3eEXwd XWPs5gEvrVea7Qpc1w9oDAQZcA== X-Google-Smtp-Source: ABdhPJw8ND5ahmIFDIkqM0VvXT2lF/nrsDzQVyzdBBFIRl2ceuPgTfy8MCsevRnIwPMOvrYKVp5oKw== X-Received: by 2002:a50:e84d:: with SMTP id k13mr4003240edn.154.1608292535497; Fri, 18 Dec 2020 03:55:35 -0800 (PST) Received: from localhost.localdomain (217-149-175-126.nat.highway.telekom.at. [217.149.175.126]) by smtp.gmail.com with ESMTPSA id f17sm11325299edu.25.2020.12.18.03.55.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Dec 2020 03:55:35 -0800 (PST) Message-ID: Subject: Re: Replication failed From: Laurenz Albe To: Sreejith P , "pgsql-admin@lists.postgresql.org" Date: Fri, 18 Dec 2020 12:55:34 +0100 In-Reply-To: References: <9492f3e4fff86b6fcb1d064d680920e8579d9c37.camel@cybertec.at> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.36.5 (3.36.5-2.fc32) MIME-Version: 1.0 Content-Transfer-Encoding: 7bit List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk > > On Fri, 2020-12-18 at 00:56 +0530, Sreejith P wrote: > > > We had 1 M x 4 slave server streaming replication using Postgres 10. Was working successfully for long time. > > > > > > Suddenly all replication servers got failed and getting following message. > > > > > > 2020-12-17 22:24:32 +04 [1587]: [357-1] db=,user=LOG: invalid contrecord length 2722 at AEA/A1FFF9E0. > > > > > > Requesting help for identifying root cause. > > > > Looks like this problem: > > https://postgr.es/m/77734732-44A4-4209-8C2F-3AF36C9D4D18%40amazon.com > > > > Was there a crash on the primary sever? > > > > There is a patch under development at this thread: > > https://postgr.es/m/CBDDFA01-6E40-46BB-9F98-9340F4379505%40amazon.com > > There was a crash. > > Master recovered automatically .. But replication failed. Then I am pretty sure that this is the same problem. Yours, Laurenz Albe -- Cybertec | https://www.cybertec-postgresql.com