Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1rPgc6-00DO2Z-8I for pgsql-hackers@arkaria.postgresql.org; Tue, 16 Jan 2024 10:24:10 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1rPgc4-002oB2-ML for pgsql-hackers@arkaria.postgresql.org; Tue, 16 Jan 2024 10:24:08 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1rPgc4-002oAu-Cu for pgsql-hackers@lists.postgresql.org; Tue, 16 Jan 2024 10:24:08 +0000 Received: from mail-lf1-x12d.google.com ([2a00:1450:4864:20::12d]) by magus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94.2) (envelope-from ) id 1rPgc2-0026EE-1l for pgsql-hackers@lists.postgresql.org; Tue, 16 Jan 2024 10:24:07 +0000 Received: by mail-lf1-x12d.google.com with SMTP id 2adb3069b0e04-50eaaf2c7deso10820960e87.2 for ; Tue, 16 Jan 2024 02:24:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1705400644; x=1706005444; darn=lists.postgresql.org; h=in-reply-to:from:references:cc:to:content-language:subject :user-agent:mime-version:date:message-id:from:to:cc:subject:date :message-id:reply-to; bh=BfZb+PLV4k/ki9ff3WJK2QGQlr9/e5J73L8ofO5Ch5g=; b=UGv0NCcWVPcvSL+EKJ1IoOxxtpSP9zl33nZkxsmDMoi0iDjK6CFZh+vbPKhhN1Ot+L hPfnDs8uF3P2c8JefS3phMkmYB6nb24A60HyVl2AeuKFTrdFmK3MDA1EjVrYtffdJ4pI 1DPVWLHKtEa2H+yDyleksM/eFmVpBagrTa55HJuz1oKsMzQjzmpIu0CVxDVL4wo4b7wR AO+zw/wdMpFcBN+syhSfFGqqpX4hVRG32SjD9V0yuKs7zbmFgry7FAu3OYy5BiCLDhEq jbFdC24hrooAb7NX9xm/6MXel/X5/QeIDdrAh5YHh/kJzjeQSmI4hfO/l78Ry2XjKB0f slvQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705400644; x=1706005444; h=in-reply-to:from:references:cc:to:content-language:subject :user-agent:mime-version:date:message-id:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=BfZb+PLV4k/ki9ff3WJK2QGQlr9/e5J73L8ofO5Ch5g=; b=qqWWgmKRtPbzWUrQkbpcOdA9cPCyOXt4UmKs5y+421joLOWHORNn7L9AAlDII70o6X MhKvYHJpE9oSoaqIR9ItGHjTChYv/msmWEI7zcF3Y1PXVFgInX16lwjBt9cRF/jSBEgL uOlknL9BYNZe8OVXpbK5GKa/4fpKo2qMCb5sQmzbyArAbZHLQiqZLcpvNLJur6MEWYeK YxwwThni+E+BF0c3pdYFGD+y5+iuEtd/AO9AVJWb81SIAXHyjG2Eh0cFH3tcN4BLQTxC YFKyB7ZYD0jpREy+4fmjxQ41q6Sc6aryEPPAGCSDayzGPEEKpeRMWxzL6StEV8Dd2uzX AMAw== X-Gm-Message-State: AOJu0Yx8xc860u4nPlciS96R0WlNIfHr4CUjSJmZ4/2IQgI/hLPaUu9D 3h9Pb1pbbplTwzVdPjUpICq3u95ugDP+QA== X-Google-Smtp-Source: AGHT+IFwSBBDgnyFGErZispzDqStzngQn1SYK9iVooyq2/aeGT2iu/JdF8z8p3IQKxis36LhifU3YA== X-Received: by 2002:a05:6512:689:b0:50e:558e:125b with SMTP id t9-20020a056512068900b0050e558e125bmr3307471lfe.125.1705400643799; Tue, 16 Jan 2024 02:24:03 -0800 (PST) Received: from [172.30.2.50] (cyclops.postgrespro.ru. [93.174.131.138]) by smtp.gmail.com with ESMTPSA id d24-20020ac25458000000b0050e69d7b5cbsm1727560lfn.187.2024.01.16.02.24.03 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 16 Jan 2024 02:24:03 -0800 (PST) Content-Type: multipart/mixed; boundary="------------RHGLEoAGn5S4Q5IRthGDJOR1" Message-ID: Date: Tue, 16 Jan 2024 17:24:02 +0700 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: long-standing data loss bug in initial sync of logical replication Content-Language: en-US, ru-RU To: Andres Freund , Tomas Vondra Cc: PostgreSQL Hackers References: <20231118015443.sfxfbr4eqolxbmjc@awork3.anarazel.de> <20231118025445.crhaeeuvoe2g5dv6@awork3.anarazel.de> <49958b83-009a-9360-2969-ea42cd15fff7@enterprisedb.com> <20231118181257.qtebsnkrzj3icg36@awork3.anarazel.de> <3c5ad91c-330a-0c5a-974a-da3801ffc18b@enterprisedb.com> <20231118210519.yborkczu2aln6zdd@awork3.anarazel.de> <5cc4bbc8-b2ad-3e6e-9878-cef8ddde8ffe@enterprisedb.com> <20231119021830.d6t6aaxtrkpn743y@awork3.anarazel.de> From: Vadim Lakt In-Reply-To: <20231119021830.d6t6aaxtrkpn743y@awork3.anarazel.de> List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk This is a multi-part message in MIME format. --------------RHGLEoAGn5S4Q5IRthGDJOR1 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Hi, On 19.11.2023 09:18, Andres Freund wrote: > Yea, the situation is much simpler during logical decoding than "originally" - > there's no concurrency. > > Greetings, > > Andres Freund > We've encountered a similar error on our industrial server. The case: After adding a table to logical replication, table initialization proceeds normally, but new data from the publisher's table does not appear on the subscriber server. After we added the table, we checked and saw that the data was present on the subscriber and everything was normal, we discovered the error after some time. I have attached scripts to the email. The patch from the first message also solves this problem. -- Best regards, Vadim Lakt --------------RHGLEoAGn5S4Q5IRthGDJOR1 Content-Type: application/zip; name="err_logical_replication.zip" Content-Disposition: attachment; filename="err_logical_replication.zip" Content-Transfer-Encoding: base64 UEsDBBQAAAAAAKiJMFgAAAAAAAAAAAAAAAAYACAAZXJyX2xvZ2ljYWxfcmVwbGljYXRpb24v VVQNAAe8VqZlxVamZbxWpmV1eAsAAQToAwAABOgDAABQSwMEFAAIAAgAh3AwWAAAAAAAAAAA FAQAACEAIABlcnJfbG9naWNhbF9yZXBsaWNhdGlvbi80X3Rlc3Quc2hVVA0AB28qpmXLVqZl vFamZXV4CwABBOgDAAAE6AMAAK2STW+CQBCG7/srptaINhJdre2BcGiaHnowJq3empiFHWUT YLfsUv9+dxG0UhMvXAjz9b4Pw9zfTSKRT3RCcslxuk2kNuHEZKqOlSxMuHicz+q41FiE/c3n 20ed4MywiGkMlZ3cF6hJVaAtJXpSmtdxS4leU1L6OwV/A/2zOfi8CZsB8FWTcibgJ03oGMCP oUci3Is8ICK3EgZEbiQYOhR8DD8sHblHiRqGdAwe80a3G2e2MXKNscwyYYLGgGOKBmFXyMzO wSHBAkFwCIEGRNtibEDttzpFVMOn6VmgN4AOv7ftRK0TsNRYPVVGqYiZETJ378A4B6ubouUN ejBoUdBLCvqfgl5S0BOF16aYWQqvqnxpNPCyWa9eV8vl+xpkfswfEXUZ6bgQqmK0ARS4s/eQ /GW3Qm3ULhf27BZ26wgW9gh29giqrR2YMIRU00C7J3tobsqZdfWHroqTX1BLBwhT+AchXAEA ABQEAABQSwMEFAAIAAgAvYwvWAAAAAAAAAAAyQAAACgAIABlcnJfbG9naWNhbF9yZXBsaWNh dGlvbi8xX2luaXRfbm9kZTAuc3FsVVQNAAcHC6Vly1amZZRWpmV1eAsAAQToAwAABOgDAABV jjEPgjAQhff7FTdC4sLMVJsOjQik1IHJFFqThkZIOQ0/3xKN0e29+17ePRvnBZfHEPxoyM93 9Dd0m19p3a8l2J2TGYL7IVSUAGN0htyHUZEBorfYCSVZdUjmaQKS22jXvKk7rZisdUpelwlb Jc9M9XgSPWbe5pCnRq4E0wLby7GSnGnZ1O8NYAK5+LcyaTTWfr+X8AJQSwcItKIgAZYAAADJ AAAAUEsDBBQACAAIALGML1gAAAAAAAAAAAABAAAoACAAZXJyX2xvZ2ljYWxfcmVwbGljYXRp b24vMl9pbml0X25vZGUxLnNxbFVUDQAH7gqlZctWpmWbVqZldXgLAAEE6AMAAAToAwAATU89 b8IwEN39K24DpC4UOqEMxvJgFULkhIGpcuIrtQix5btW/PzGqEO393VP73yOCei7pyGHxCFO ED4BH4GYirwTvgTY9SP+c3i9E2LI6Bj/PF4vBUDw0Gpr5OFlJj9uBMYHF6xOddtZaepuTn6k GzTWHKW9wLu+wDL4lVjNjUJZLTsN7XnfKmuazpzqsqKc11o96cL3k7tjlSLxNSNBipmrt+3m Fb5mqRrj4MaC4EqE03CPHisfqKxcQHPeH4ySz6ZU3vsFUEsHCFT+vaXGAAAAAAEAAFBLAwQU AAgACACUjC9YAAAAAAAAAAAkAgAAIQAgAGVycl9sb2dpY2FsX3JlcGxpY2F0aW9uLzNfaW5p dC5zaFVUDQAHuQqlZatWpmWjVqZldXgLAAEE6AMAAAToAwAArZAxT8MwFIR3/4pHqAgMUTCF KcqAEANDVQmaDalykkdjycTG70X8fZw0KW3KyOY76+47+/IiLXWbUiNaW+PttrHEecqfbtTO es4f7pd3o+4Ifb4o3p5fR6NWrEpFmLuQ3HkkMVzIWZM8NC1HPWuSfzU5+jKQFLD4hUNST3IK QOImq4dA0kyy3wBJBZEyHKKuK42uFGvb9meovXUQOgwCyyya8eQpT57z5ClPHnjxOyHDY7FZ P61Xq5cN2DYe/P0O6kqqvHbDkCDA40d4b3M8MNsH2HdtMIaB8dU//oggNFgxuN2WDKK7ljeZ OKaJKOC+lWbxA1BLBwiA7LdI9AAAACQCAABQSwECFAMUAAAAAACoiTBYAAAAAAAAAAAAAAAA GAAgAAAAAAAAAAAA/UEAAAAAZXJyX2xvZ2ljYWxfcmVwbGljYXRpb24vVVQNAAe8VqZlxVam ZbxWpmV1eAsAAQToAwAABOgDAABQSwECFAMUAAgACACHcDBYU/gHIVwBAAAUBAAAIQAgAAAA AAAAAAAAtIFWAAAAZXJyX2xvZ2ljYWxfcmVwbGljYXRpb24vNF90ZXN0LnNoVVQNAAdvKqZl y1amZbxWpmV1eAsAAQToAwAABOgDAABQSwECFAMUAAgACAC9jC9YtKIgAZYAAADJAAAAKAAg AAAAAAAAAAAAtIEhAgAAZXJyX2xvZ2ljYWxfcmVwbGljYXRpb24vMV9pbml0X25vZGUwLnNx bFVUDQAHBwulZctWpmWUVqZldXgLAAEE6AMAAAToAwAAUEsBAhQDFAAIAAgAsYwvWFT+vaXG AAAAAAEAACgAIAAAAAAAAAAAALSBLQMAAGVycl9sb2dpY2FsX3JlcGxpY2F0aW9uLzJfaW5p dF9ub2RlMS5zcWxVVA0AB+4KpWXLVqZlm1amZXV4CwABBOgDAAAE6AMAAFBLAQIUAxQACAAI AJSML1iA7LdI9AAAACQCAAAhACAAAAAAAAAAAAC0gWkEAABlcnJfbG9naWNhbF9yZXBsaWNh dGlvbi8zX2luaXQuc2hVVA0AB7kKpWWrVqZlo1amZXV4CwABBOgDAAAE6AMAAFBLBQYAAAAA BQAFADACAADMBQAAAAA= --------------RHGLEoAGn5S4Q5IRthGDJOR1--