Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1whpS5-000RT7-0K for pgsql-hackers@arkaria.postgresql.org; Thu, 09 Jul 2026 14:10:09 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.96) (envelope-from ) id 1whpS3-00DL5r-17 for pgsql-hackers@arkaria.postgresql.org; Thu, 09 Jul 2026 14:10:08 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1whpS2-00DL5j-2e for pgsql-hackers@lists.postgresql.org; Thu, 09 Jul 2026 14:10:07 +0000 Received: from mail-ej1-x62e.google.com ([2a00:1450:4864:20::62e]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.98.2) (envelope-from ) id 1whpS1-00000000KPw-1Y2t for pgsql-hackers@lists.postgresql.org; Thu, 09 Jul 2026 14:10:06 +0000 Received: by mail-ej1-x62e.google.com with SMTP id a640c23a62f3a-c15c7c07a57so304381366b.1 for ; Thu, 09 Jul 2026 07:10:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1783606204; cv=none; d=google.com; s=arc-20260327; b=atl58zlsgHQyMiR2U/JF8TsUeo45Wq7aTFVnV8Duxbi884VdET98nwAlGEC8znAut3 pfTLId/C+fc7o9DNa6JUEaeo68gjCtmtZDvPtDqOeclEUUcxLKCTqsuKdNLIykB3doyB jzaxOt4vNPCb9HXjWTj+waoASaY1LKTnnstx/J2DVCNUY92StFUK0k6rpOHk0EtQjBfA D3/E5I+iqyuaOIYLo7blZSuakERFcflGItMYz/jfpAyVrMix6t85SzIeE7YBkLf+P9// jROY0l8qbNuqGGlGkcg8Jxv/5iEclSvRTTPArzh8LjiWd6cUdpDNjQQAAKFvlyF+aVbZ UN2A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:dkim-signature; bh=IcNM1c7a47MLCihvs/QxK2f9Pp4Eq/KrcoH90rkCbOs=; fh=rHYI8usACfgK76uchn2yGyzUGwGiMQG2XHk7W5BA2Nc=; b=Q6x626qcmgi5LT183LS79blPZKGo9SSymnYGhWLirf1XNOzQlOKZXYnIEUiasdQRD5 qVQKYGnAvcO7dJBCxNufUfk6q899sTtomellzVoPNZ9oyIoKvPupGeCAsi9eFDZckKEQ mx4b+HqhaT6hVqK24DbCXYyWsnoh2R1UIfSX6wIMLYc3jwwYBWey9v0EP7N8EgprMTLZ 7DTYIEhH9neq9ovOJZnmjkjud9WEVznt77WJw5EYT8sAlyyFAMJ+CdSH3MlwLVOmBP1O UWeUaxWcNtVcH73yqcADjPtszg0AmMO4lkTHtsiyAbIRl2omBIa7G28Ke/T2XJFKgfrp vBEw==; darn=lists.postgresql.org ARC-Authentication-Results: i=1; mx.google.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1783606204; x=1784211004; darn=lists.postgresql.org; h=content-transfer-encoding:content-type:cc:to:subject:message-id :date:from:in-reply-to:references:mime-version:from:to:cc:subject :date:message-id:reply-to:content-type; bh=IcNM1c7a47MLCihvs/QxK2f9Pp4Eq/KrcoH90rkCbOs=; b=E0mPas+Uw0oYnlXsKr7IoFjIUo844ucrzQoxVwIRG2SIS6+tcyNCohRCr8mD/h/P8z nkDQo/UGB6Y8RosbJoa/tS6PJBL6goSJrJDoAuTmLJbgrx6Bku3T8cAYTZI2y9NMeqFi Nsqf6WmYRCOfftiXOlBfVXMipBkd+5TXxax0d7OwSAC1HpuM4FPs/C43k64vFrxhZF0u iXQv/5fltA86/HCZys42Lkgzu+JdkwzvXmqUJaG6ZvmEXC+ISdGCTJd4zjU0SmtEWyPe 263MLOcanNwxVSp16wK7J3WTJlPOImzt9GYZlqxt9/T17mACg7lgt3lQ55vTtYpoNCWH 4sgQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1783606204; x=1784211004; h=content-transfer-encoding:content-type:cc:to:subject:message-id :date:from:in-reply-to:references:mime-version:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=IcNM1c7a47MLCihvs/QxK2f9Pp4Eq/KrcoH90rkCbOs=; b=ik19Hbg/YhMSaFoymzGoYZUW3ZDSkkOvwXzD5ujOIjXuGxoZ7hIFCPKbg+9Z8+U7cV XNZcKfHE2rElBCtBjWkH+qMeW2/69hfs2G7wcdXNtB1ULeFhewbrTbFNkT9xaIETT67p 9kYvqY/8z/tbrU8ByuxOQ51k2K6U+NoKggvL42U/m1bp4Ttbtvqjd2+4PvP0wARzMYT7 L+qICBUq5zX9HSjy8T458b7WHdzyMq0T0cwAer3wZX2d+uJ2a2zwQriEXJ8KdQHCPiaj zr+8fNLNsXLgG1Y92c++N6+d4RIQkol3uWLQWlzmUiEHKVS5j8aLh21fueP5Nij0tlu/ x4xQ== X-Forwarded-Encrypted: i=1; AHgh+RrkEOUWUphE3sXBF+M7u01Kih9j6sA6jcZMAFO5YLWlXEpWI9juPkdQz+zxlpt2nD5mWZVWtHN4HmJgArZm@lists.postgresql.org X-Gm-Message-State: AOJu0YzUzVovk+cTa8/T7ixyMbyA6b0K/AGDqd0FmSoDRG5o/abw4dcR JQhX8Ns4WFMRwkxTM2bSvqD9W4FjacIqoYnvBOO2Gj/FBHHUJDIY+CglEVK/LF2Ob+k8GvkT4xt RJ3RdCRTKwhnFjjQtRwyB6q47L3KjcwY= X-Gm-Gg: AfdE7cni0enYSL4yDWSkj1SDnMXdAQRgjCqLZNzq/IO1CmELTNq0eUhVf/ohxEm36De Gy/HYqNVjJ4U6sXOkUdH6KY4NAmdlGsGtQ4DVOJpH+wTUPirlx1cog39jQIqerTgJzkZ6aJlp47 D0HAtAJPflZrbraBk/sAUS3d/O6fiP/6oxu5sj1nu3GJzRYO8IIkvXCtEEoiecOH1he73fmmuA9 c2cChc5wEgnPwD8wJVqJg0eJWAPorvcEHeVpssTt6UQAWgQwSrFU3B6rRAyE/Xb0Rlpne2Bkga5 Q3F9bD965sgwBAzYFdQFBOMMP0Co X-Received: by 2002:a17:906:3b42:b0:bf0:b39:dcc8 with SMTP id a640c23a62f3a-c15ce0e0ccfmr250920066b.26.1783606203966; Thu, 09 Jul 2026 07:10:03 -0700 (PDT) MIME-Version: 1.0 References: From: Robert Haas Date: Thu, 9 Jul 2026 10:09:51 -0400 X-Gm-Features: AVVi8CcmhQTk84Ll379MnEcX2FvYk_0emUFWIa0dVj-ncFQnKTdkiAIvwUVrPq8 Message-ID: Subject: Re: Proposal: Conflict log history table for Logical Replication To: Amit Kapila Cc: Dilip Kumar , shveta malik , Peter Smith , vignesh C , Nisha Moond , Masahiko Sawada , Bharath Rupireddy , PostgreSQL Hackers Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk On Tue, Jul 7, 2026 at 4:24=E2=80=AFAM Amit Kapila wrote: > If I understand you correctly, the concern is that > AbortOutOfAnyTransaction() tears down the entire transaction stack, so > if it runs from a nested PG_CATCH, any outer catch block still queued > on the stack that expects a live (sub)transaction would then execute > against already-aborted state. The concern is about this code: + PG_TRY(); + { + LogicalParallelApplyLoop(mqh); + } + PG_CATCH(); + { + MemoryContext oldcontext; + ErrorData *edata; + + /* + * Reset the origin state to prevent the advancement of origin + * progress if we fail to apply. Otherwise, this will result in + * transaction loss as that transaction won't be sent again by the + * server. + */ + replorigin_xact_clear(true); + + /* + * Copy the error and recover to an idle state so we can insert the + * deferred conflict log tuple (if any) before re-throwing. Copy the + * error into a longer-lived context first, as it may have been raised + * under ErrorContext. Also reset the error context stack: the + * callbacks in effect when the error was thrown belong to unwound + * stack frames, and the deferred insert installs its own. + */ + oldcontext =3D MemoryContextSwitchTo(TopMemoryContext); + edata =3D CopyErrorData(); + MemoryContextSwitchTo(oldcontext); + + FlushErrorState(); + error_context_stack =3D NULL; + + /* + * Tell the leader we failed and are about to report the error and log + * the conflict. This must be set before AbortOutOfAnyTransaction() + * below releases the transaction lock that the leader waits on in + * pa_wait_for_xact_finish(); otherwise the leader would see a + * non-finished state, assume the connection was lost, and tear this + * worker down while it is still writing the conflict log tuple. + */ + pa_set_xact_state(MyParallelShared, PARALLEL_TRANS_ERROR); + + AbortOutOfAnyTransaction(); + + /* + * Insert the deferred conflict log tuple before re-throwing. + * Re-throwing is what reports the error to the leader (via the error + * queue set up above), so the insertion must happen first: otherwise + * the leader could start tearing down this worker while it is still + * writing the conflict log tuple. If the insertion itself fails, + * that error (annotated with the conflict context, see + * InsertConflictLogTuple) propagates to the leader instead of the + * original. + */ + ProcessPendingConflictLogTuple(); + + /* Re-throw the original error, which reports it to the leader. */ + ReThrowError(edata); + } + PG_END_TRY(); I see your point about this being the outermost try/catch, but this still doesn't look good to me. It seems different than the way other workers do error recovery, and I am really doubtful that is is safe. > > 2. If inserting into the conflict table itself errors out, then the > > conflict log table will be permanently out-of-sync with the status of > > the replicated table. > > The conflict log table can't permanently diverge from the replicated > table. On the error path we call replorigin_xact_clear() before > inserting the conflict row, so the CLT-insert transaction commits > without advancing the replication origin. The origin therefore stays > at the last-confirmed LSN, and the errored transaction is re-streamed > and re-applied on the next attempt; which also re-attempts the > conflict logging. If the CLT insert itself fails, nothing is committed > and the same replay re-does both. Is there a specific path where you > see the origin advancing without the row being applied that could lead > to divergence? I'm glad we agree on the goal. If an error inserting into the conflict table doesn't advance the replication position, then we should eventually either (a) succeed in replicating the data or (b) succeed in inserting the error, even if it takes many retries before one or the other happens. But why are we trying to send the error to the leader here? It seems to me that a replication apply worker should be directly sending the errors to either the log or the conflict table, and the leader shouldn't need to know about it. --=20 Robert Haas EDB: http://www.enterprisedb.com