Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1wV49J-001ewD-2m for pgsql-hackers@arkaria.postgresql.org; Thu, 04 Jun 2026 09:14:02 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.96) (envelope-from ) id 1wV49I-005hKm-2A for pgsql-hackers@arkaria.postgresql.org; Thu, 04 Jun 2026 09:14:00 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1wV49I-005hKe-0S for pgsql-hackers@lists.postgresql.org; Thu, 04 Jun 2026 09:14:00 +0000 Received: from mail-lf1-x12d.google.com ([2a00:1450:4864:20::12d]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.98.2) (envelope-from ) id 1wV49G-0000000131Y-0SQ0 for pgsql-hackers@lists.postgresql.org; Thu, 04 Jun 2026 09:13:59 +0000 Received: by mail-lf1-x12d.google.com with SMTP id 2adb3069b0e04-5aa610cbd9bso414778e87.1 for ; Thu, 04 Jun 2026 02:13:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1780564436; cv=none; d=google.com; s=arc-20240605; b=ZmIG0Tgx58VleFXcUyLhLnpdlCEHam/tbeAvuBM6Gm0L8YykmmZW2arb/UjXq0fGuz 1KI4YrgH7MTypmbtjTI2/+b+8tFrGVK8+Votao3WK4Dpg5+ZF0C2bhX+9IcqHAAxZR5J da//bUbsPTIJh/tBY/CxXbKvW1GY/WGdwpx02Wx/4QciqUJMa6p2e6sX3O8qD3tFqjRW WoVf849S4s4LMTN9be/pSC/xrSVObnF1c3FlM4e9H2YnEY7vB3Np0OFrgtublWQkeLBF TMF9L8cJrSsxKE/a5mhPJYAFdDEYRMRLKzsJ+wZwpDooKz+YNLKzRN0csdW0d8TonYLF 1cZA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:dkim-signature; bh=36xSLZksS4zKH4spUhePlLO78SlutspBGzhW9cN4mtA=; fh=N4J24sd7LRqmOfeaLBtblBPFIo0liViDoC0Zb9v8ubc=; b=gV0DnO39hBM0Et3bNdgADYloROrsyC1Dg6QduT/yIsEndvp9AObgRBR1vct0l2yUpF vW07oMtC3j1nmNFxS55lfB/1b8l2HHxbM411GCvC07/ZIyVVbGel/uUvrx2vU3EFCM8i UoMw9JZhvWtQ7WjEdrRkYelVe9uBWQMUSdTo7j3ynwb85d4+dRfln8BIg3immLiN6/UY Pd6PSGq5hjOMe3GtKww+D5l2Cd3+xcl1ziP8tjog/g+g1adT2BUNzjLRD9wr5MZKRfdg 5DWOlQ/MFUXsDQCG5a3Oao7ILWcdMbwiZJFMSNR6S7Xl0wKp8KJrmkEEcj3u0xI4ix1D e9fg==; darn=lists.postgresql.org ARC-Authentication-Results: i=1; mx.google.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1780564436; x=1781169236; darn=lists.postgresql.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=36xSLZksS4zKH4spUhePlLO78SlutspBGzhW9cN4mtA=; b=RgITBOTnqJyYCL2nR1lXuYCn6y7la2G+A7jsvMdjrM5f8mqSQ+WPNVo/ClDPCl4zjF d+NLabb395YFmbgovnazbKHUSuNNlZPZ4O6TE8MI0oBkt1wJU099bKO9KokeRgVEgHXc H0irFeP2v3s12pZ7zAiiZ1+Dj8Kgu3qGuZ7DCSQATBdjPpeOHQIqtRBgaCLtqIt/BmMS MVJLYljz//ySsZz7om7kQMao3Hgp2mQSqPSFfJDC5ASw8lRFqNlIMV9l1P6LUxNlU8gu mrn4bThp+hEsGa1mRnPTRRc1/jolEep6PmLM0n2Ha41YYDYw1g5r3WgZITBlk83ylH1T S03w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780564436; x=1781169236; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=36xSLZksS4zKH4spUhePlLO78SlutspBGzhW9cN4mtA=; b=Gy+vsPksBDpWpnrE1UprvLmV4FSylKpPDVsROdQavN2nW61T+OaAbvtXIZap0r9/vz K90oc93ZWBdTQx/lXZUQj6qf6Nu89nIB9NpdD4gsEAPxTZADLZBcqU2fmYZRG6rqGwZo Rfr5JD9VxcrpDdueMpmSd/AgCj4ASEAlMXT3oKfzXTVaibOh+5iCAD/xVT5VCRAD9pii tpGpf9FjYqGIFzKBkXGGVGQ8TKkzQ08XeIzm37q2f9agP8KB9MadjlOo2KDXbfOS7zfR fYYhZZOQTMP4OLT50Tl/g2tNvAWq535zubX1Mp51oht0aC+3AO+gOk6YjBSTwuuNmd1d Fmaw== X-Forwarded-Encrypted: i=1; AFNElJ/d40RTjW+8V1Sfub+M+BqBGWa4gemb9K6CQHGCG3+lXgf1e2fxf2BFJKZTRjad4znB/NI4IPl3dxY6MqVH@lists.postgresql.org X-Gm-Message-State: AOJu0YxZwe0TPzW279CicPubqzLeXrXeMK0witcUzELc153RqmetSIdi Ki5XlYZL2Q8ZYNqeiTcU0loju+y/gJMdtri5F7s6YrkrfN24T1wjIFY/NZf94EKCgz0LXiR5p5v ng4b4oM+KZaRy9sHE9FmSnVBI0cx3jQ== X-Gm-Gg: Acq92OF0vU/DuBxeHXX77iDI0H7lRAbOzEq/6Ak8z//FGL0L4Pn2ELDCj0Y1VSG2M1q mat3YXcHzUnWA/k59QhD6+/TgqLxueKy/2FUyaMVr/cOjiR83zTJw5n5qDG5e1q+A3BXisTNtGN kL9U2bv03q119+mYM5NWy9f14QXW5a4v0gHl0ZD4aVY9R1N5Dp77lfbw4HsXtNpH/EcZ13gUWyl cFAad1Kl7tgsCstVJCSO5JoCKeiubdfNyMBLbkXrlbcT4stuWDElZxQQilXWsjhIipgEcRNqXOX XhaU+4ysF2o2SKwkxeNxDWih X-Received: by 2002:a05:6512:61a2:b0:5aa:7005:125c with SMTP id 2adb3069b0e04-5aa80c7a164mr675682e87.8.1780564436216; Thu, 04 Jun 2026 02:13:56 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Nisha Moond Date: Thu, 4 Jun 2026 14:43:43 +0530 X-Gm-Features: AVHnY4LQCfO3SgL4bCG9tIo4Vfwuhz9o1BUNN6UlTz3SxrGMbZiDOj2FW2rXg7c Message-ID: Subject: Re: Proposal: Conflict log history table for Logical Replication To: Dilip Kumar Cc: vignesh C , Amit Kapila , Peter Smith , shveta malik , Masahiko Sawada , Bharath Rupireddy , PostgreSQL Hackers , shveta malik Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk On Thu, Jun 4, 2026 at 12:55=E2=80=AFPM Dilip Kumar = wrote: > > On Wed, Jun 3, 2026 at 3:25=E2=80=AFPM Nisha Moond wrote: > > > > > > 2) As ProcessPendingConflictLogTuple() may perform a heap_insert, it > > could fail for various reasons, such as table access errors, write > > failures, WAL write failures, or ownership-related issues. > > I looked into the error handling when CLT insert fails: > > > > 2a) When disable_on_error =3D false > > > > /* Try to allocate a worker for the streaming transaction. */ > > @@ -5666,6 +5674,7 @@ start_apply(XLogRecPtr origin_startpos) > > */ > > AbortOutOfAnyTransaction(); > > pgstat_report_subscription_error(MySubscription->oid); > > + ProcessPendingConflictLogTuple(); > > > > PG_RE_THROW(); > > } > > > > If ProcessPendingConflictLogTuple() fails, PG_RE_THROW() is never > > reached, so the original conflict error is lost and only the secondary > > CLT-related error is reported, e.g.: > > > > ERROR: could not open relation with OID 16421 > > LOG: background worker "logical replication apply worker" exited > > with exit code 1 > > > > If the failure persists, the apply worker will keep retrying and users > > may never see the actual conflict error. The same concern applies to > > the parallel worker case too. > > > > Is there a clean way to ensure the original apply/conflict error is > > still logged even if ProcessPendingConflictLogTuple() itself fails? > > Does this happening when destination is table or with all as well? > It can happen with both conflict_log_destination =3D 'table' and 'all', since for conflicts such as insert_exists (ERROR), we call ProcessPendingConflictLogTuple() to insert the conflict into the CLT in both cases. I used the following steps to reproduce the error: - Attach gdb to the apply worker and set a breakpoint in ProcessPendingConflictLogTuple(). - Trigger an insert_exists conflict. - After execution passes StartTransactionCommand(), change conflict_log_destination to log from another session. - The conflict log table is dropped, causing table_open() to fail and resulting in the above error. I think Shveta's suggestion #4 in [1] should address this issue. > > ~~ > > > > 2c) When "conflict_log_destination =3D table" the log reports: > > > > ERROR: conflict detected on relation "public.t1": conflict=3Dinsert_= exists > > DETAIL: Conflict details are logged to the conflict log table: > > pg_conflict_log_16403 > > > > However, the subsequent CLT write can still fail: > > ... > > ERROR: could not open relation with OID 16426 > > LOG: background worker "logical replication apply worker" exited > > with exit code 1 > > > > Since the CLT write has not yet occurred when this message is emitted, > > would it make sense to change it to: > > "Conflict details will be logged to the conflict log table: ..." > > > > to avoid implying that the logging has already succeeded? > > IMHO "Conflict details are logged to the conflict log table" doesn't > mean a conflict has been logged to the table; what I interpret from > this message is that conflict logging to the table is enabled. > I agree it can be read either way, but I felt the intent might be ambiguous from a user's perspective, so I raised it. That said, I'm fine with it if others also don't see any ambiguity. > > ~~~ > > > > 3) The v45-002 commit message still refers to the old conflict log > > table name, pg_conflict.pg_conflict_log_for_subid_16396. It should be > > updated to pg_conflict_log_16396. > > ~~~ > > > > 4) 035_conflicts.pl: I see that v45-002 has corrected the CLT name in > > below test - > > +# Verify the contents of the Conflict Log Table (CLT) > > +# This section ensures that the CLT contains the expected > > +# type and specific key data. > > +my $subid =3D $node_subscriber->safe_psql('postgres', > > + "SELECT oid FROM pg_subscription WHERE subname =3D 'sub_tab';"); > > +my $clt =3D "pg_conflict.pg_conflict_log_$subid"; > > + > > +# Wait for the conflict to be logged in the CLT > > +my $log_check =3D $node_subscriber->poll_query_until( > > + 'postgres', > > + "SELECT count(*) > 0 FROM $clt;" > > +); > > > > I wonder why this wasn't caught in the v44 tests. It seems the check > > below is silently broken because poll_query_until() keeps retrying > > until timeout when the table does not exist. > > Should we replace it with an ok() test instead? something like - > > > > # Wait for the conflict to be logged in the CLT > > ok($node_subscriber->poll_query_until( > > 'postgres', > > "SELECT count(*) > 0 FROM $clt;"), > > 'conflict appeared in CLT after insert'); > > Why would CLT not exist, IIUC this is positive test where we are > trying to validate the conflict data from CLT, no? > Ideally, the CLT should exist. I initially assumed the tests were clean in v44 despite the incorrect table name being used. I re-ran the tests and confirmed that poll_query_until() does report a failure if it times out because the CLT is unavailable. Sorry for the noise. [1] https://www.postgresql.org/message-id/CAJpy0uAA_XszCAcoBuCUM3VobD39DbMD= wCPUT%2BXW7wFfE%2B_E8w%40mail.gmail.com -- Thanks, Nisha