Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1wZTjq-0014Vx-10 for pgsql-hackers@arkaria.postgresql.org; Tue, 16 Jun 2026 13:21:58 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.96) (envelope-from ) id 1wZTjp-0004IF-0I for pgsql-hackers@arkaria.postgresql.org; Tue, 16 Jun 2026 13:21:57 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1wZTjo-0004I4-1e for pgsql-hackers@lists.postgresql.org; Tue, 16 Jun 2026 13:21:56 +0000 Received: from mail-lf1-x135.google.com ([2a00:1450:4864:20::135]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.98.2) (envelope-from ) id 1wZTjm-00000000e8m-3uaa for pgsql-hackers@lists.postgresql.org; Tue, 16 Jun 2026 13:21:55 +0000 Received: by mail-lf1-x135.google.com with SMTP id 2adb3069b0e04-5aa68dbd44fso4492556e87.2 for ; Tue, 16 Jun 2026 06:21:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1781616113; cv=none; d=google.com; s=arc-20240605; b=CDwbB05aDUfZssEB6DBtu0Y6FKEMCwD64mbFOWVpaDT8V1PpZ2Sw13g4prwYdSjJzm gS4j1jppRAeBo97xbY+bjfESx2ZXbFkenbsNvmOPXyf7UFFSM9RJmSI13859vAORMh+O K1rxOIyQIzWmNSABqQFQSgGl68sKwbVF6QUWD01Ca2GZ7DhN28kuI4hue/92vdxB6CmQ GFuxpZNlAM5GRb18Le0gXpd/KrSuhM9diVQZOdrhjcjs1DRtHVbrTDFWZjMca8EW2vLY KUAalw7XJc+sVuG5/i5pggkE/h5dm/WTg6AqeKJj9387L4/Jq/eneUcuQ2T/qKIyS+BR EpLg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:dkim-signature; bh=ItBgBjaJOtDqYR0+02zQmZhRFNERqJXjf0isj1GdQ6Q=; fh=1s2sEbiXDT4W5XiRAau4dLpD4vdxXJ3MA8Gxa6HSNGw=; b=lXahPhMTCtaZqBHyvd5JwWIrVdzwC+0Tzimamq1Gjc8ZFFZiQLjeW3tNhYjCi9zMpm b1ZAwGJBIG9MNPeAIHuAzcoi2/6wDsgH5v0wMBBCp70DMFY9aYL1DR4xn0DoOGo2RbNQ ajxM5Q3MGUvp25MuUui41XcTntrOH892ppNuaX18xEjvGsh590WGSZXo7aySo0uq0/Fi HEoogd+wi8ACGSnwOFZrJzG3vR2tAmUI7UeDZqGzKx33seaETlsROqu3xDoEHyqwQ8mx QVauQFfLv6rRKzY2i2EeQyxQVT6L2lfAsSAI2FygwiwcwndqQIokNa2UqFh/WzT0uUqD o0Vw==; darn=lists.postgresql.org ARC-Authentication-Results: i=1; mx.google.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1781616113; x=1782220913; darn=lists.postgresql.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=ItBgBjaJOtDqYR0+02zQmZhRFNERqJXjf0isj1GdQ6Q=; b=oaIewVUX78YR2fhWhzUAXJ2k/6+YovfYeHbsWJtzm9z3XH8IvdLcHi+A1+bvnINr2P e8YnisvGUyGo7TBx5Kg5pdsFXAbDnBnCgiL0LMW7gqC8GFYGRxxQDccj75asuPXodJBN qI64daRnKmxXZbxM93AXtkCjUgkgCEUPuN511Y+TGfXaACPhHcra9JzcAF8jfwhRGDvG n5W+gsluBxyYDtjGXaE+U/dvRH+E7pGa1x14GJcJXdWYG9ryh+9613YBTYoDAnHOucpK +i1C+L2H0EoTI/47MljbfSynmMdXVw1mAUwLfzMitoErwizQZRdcmJfh0ky3jSG0WUzr D9Ig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1781616113; x=1782220913; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=ItBgBjaJOtDqYR0+02zQmZhRFNERqJXjf0isj1GdQ6Q=; b=nzRnGMr3USq2ucmF2PTu2vO+Glc/sevwWrOmydjjLsxt9WpUSlIq9jsvj68T9DUB1Y 4CmSaDY1/px/LbSbMk3WI1q4rigsDmhZU8jJRXXolPAVUuwFWLW9MW5UmDN5QW61o/k4 36DB06+/5Krg9fsVlHvlMRe5Bt5/9FCF72TfQUQ4UOrCwYY2sjOSMHwVeVe+8hm0VzjK SEkakA1evN1gsDSx51NRYzVOIPrwBk1c3uJuPSbke1QI4TTS8rPVdpXMQ4td07ipfKgk X1yJOCj+YyH0EmYBoNESO1AdYyL6lQm+N7Sje6qyRORMpO7m57LdGdZt4vIsp65Oi4PV hRRg== X-Forwarded-Encrypted: i=1; AFNElJ8Cjwqo79upVSua4jXY2h/nPDdY7ha3spuVZCKip71k/glon2hGkF3odPzIBQn7y+r0yqeTRFW5Fcb00iZI@lists.postgresql.org X-Gm-Message-State: AOJu0YxT9OO5e//qZAe5DVZa060iB/v3W+6p0+iS//D0NsG9YThDKS7t L1uyn1jaoN5Or0jceWxMNsAtBmQmw875WznAGbbADk1RIuGyRilTL2o+c9PKuWefJMgii1zWDWo NRGN2XGUJSRZC6E0O2bHNuR9Lesmi3KI6Y8K3e+o= X-Gm-Gg: Acq92OGFNceQR7TyehAyxLTxYEJ3eGLYbPYuW/8kZtZhuHNP7E8XVZZjUImGk3HK8YI 74CPoXSJBF2tmYd6dSX0VqtospTJlibcjpHLlLW2yfgljQAbii73BQTxhRLjuaOUftqlhz4rzAs K+zZl+yQNvMirzhJgUZb3ZdGHh6I7300FwfUhPL5u0JB4Hic1gXWIRVEO/RB4ikbbPg1eOMIKEA CpBR/Dc6hC1h2+Du6EiuQkIO/dyTPBtydolx1rOOqQJPCOjx0iVajYeVhoQOMYPMUWT+3aqMSZA XsYaDqYKuGt+VUZ6tKaW35lHbjBkhuLkj8Jz3ZiI/AzFDkiyww== X-Received: by 2002:a05:6512:a93:b0:5aa:62c4:e25a with SMTP id 2adb3069b0e04-5ad30dcdf72mr4017767e87.31.1781616112154; Tue, 16 Jun 2026 06:21:52 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Dilip Kumar Date: Tue, 16 Jun 2026 18:51:33 +0530 X-Gm-Features: AVVi8Cf6WZxSRkDNHfnSHYzAXejAn056sDY1qlm5KfT9h0_NAmB0QDeWSwRO5lo Message-ID: Subject: Re: Proposal: Conflict log history table for Logical Replication To: Nisha Moond Cc: shveta malik , vignesh C , Amit Kapila , Peter Smith , Masahiko Sawada , Bharath Rupireddy , PostgreSQL Hackers Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk On Mon, Jun 8, 2026 at 2:52=E2=80=AFPM Nisha Moond wrote: > > On Fri, Jun 5, 2026 at 4:22=E2=80=AFPM Dilip Kumar wrote: > > > > Here is the updated patch which fixes all open issues except Peter > > reported on 0004 patch, Vignesh would you take care of that? > > > > Thanks Dilip for the updated patches. > > Patch 001 + 002 > 1) Issue #5 from [1] still appears to be present. I still see NULL > values in remote_commit_ts. From patch, it looks like remote_commit_ts > is updated in v46, but too late in the flow. I tested by moving: > /* Set remote_commit_ts for conflict logging. */ > remote_commit_ts =3D commit_data.committime; > before the apply_spooled_messages() call, and remote_commit_ts was > logged correctly. It seems the assignment needs to happen earlier. Yeah right, I have fixed it, will be available in next version > 2) CLT reports NULL or stale remote_xid for two-phase transactions > Test case: > setup : pub-sub nodes with max_prepared_transactions =3D 10 to enable two= _phase > -- Publisher setup: > CREATE TABLE clt_test (id int PRIMARY KEY); > CREATE PUBLICATION clt_pub FOR TABLE clt_test; > > -- Subscriber setup: > CREATE TABLE clt_test (id int PRIMARY KEY); > INSERT INTO clt_test VALUES (1); -- pre-existing row to conflict > CREATE SUBSCRIPTION clt_sub > CONNECTION 'port=3D9933 dbname=3Dpostgres' > PUBLICATION clt_pub > WITH (two_phase =3D true, conflict_log_destination =3D 'all'); > > -- Trigger a conflict via a prepared transaction on the publisher: > -- Publisher: > BEGIN; > select txid_current(); > INSERT INTO clt_test VALUES (1); -- will conflict on subscriber > PREPARE TRANSACTION 'clt_bug_test'; > COMMIT PREPARED 'clt_bug_test'; > > -- Check the CLT on the subscriber. The publisher transaction was 699, > but the CLT contains: > postgres=3D# SELECT conflict_type, remote_xid, remote_commit_ts FROM > pg_conflict.pg_conflict_log_16398 ; > conflict_type | remote_xid | remote_commit_ts > ---------------+------------+---------------------------------- > insert_exists | 698 | 2026-06-08 14:04:34.903284+05:30 > insert_exists | | > insert_exists | | > ... > It looks like remote_xid is either stale values from a previous > transaction or remain unset when apply worker tries again. > The change below fixes it. > @@ -1300,6 +1300,8 @@ apply_handle_begin_prepare(StringInfo s) > set_apply_error_context_xact(begin_data.xid, begin_data.prepare_l= sn); > > remote_final_lsn =3D begin_data.prepare_lsn; > + remote_xid =3D begin_data.xid; > + remote_commit_ts =3D 0; I couldn't reproduce this, but I think we should reset remote_xid and remote_commit_ts, but I will check on this. > patch-003 > 3) alter_sub_conflictlogdestination(): > + relid =3D get_relname_relid(relname, PG_CONFLICT_NAMESPACE); > + if (OidIsValid(relid)) > + { > + /* Existing table from upgrade */ > + Assert(IsBinaryUpgrade); > + } > + else > + { > + relid =3D create_conflict_log_table(sub->oid, sub->name, > + sub->owner); > + } > + > > I think Assert() above should be replaced with an ERROR similar to the > one in create_conflict_log_table(). > One possible scenario is that a user manually creates > pg_conflict.pg_conflict_log_ with allow_system_table_mods =3D on. > In that case: > -- Assert builds will hit the assertion. > -- Non-assert builds will silently skip table creation, and later CLT > operations may fail in unexpected ways because the user-created table > definition may not match the expected schema. > > Before patch-003, create_conflict_log_table() correctly reported an > error if a table with the same name already existed, which seems > safer. > > 4) 002_pg_dump.pl > - WITH (connect =3D false, origin =3D any, streaming =3D on);', > + WITH (connect =3D false, origin =3D any, streaming =3D on, > conflict_log_destination=3D table);', > > typo: space before "=3D". > /conflict_log_destination=3D table/conflict_log_destination =3D table/ > ~~~ Vignesh can you look into this? > patch-005 > 5) doc/src/sgml/ddl.sgml > + direct user manipulation. Unlike pg_catalog, the > + pg_catalog schema is not implicitly included in t= he > + search path, so objects within it must be referenced explicitly or b= y > > typo : repeated "pg_catalog" > It should be: Unlike pg_catalog, the > pg_conflict schema is not implicitly... > ~~~ > > [1] https://www.postgresql.org/message-id/CABdArM5N0LgkHc%2BJOKcbDjpx%3D0= hdWDx%2BJ7y%3DUS2apEEmi07oyw%40mail.gmail.com > I have noted the doc changes, will fix after we agree on initial patches. --=20 Regards, Dilip Kumar Google