Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1wbZzJ-002g9v-1O for pgsql-hackers@arkaria.postgresql.org; Mon, 22 Jun 2026 08:26:37 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.96) (envelope-from ) id 1wbZzI-005Ksu-0e for pgsql-hackers@arkaria.postgresql.org; Mon, 22 Jun 2026 08:26:36 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1wbZzH-005KqP-27 for pgsql-hackers@lists.postgresql.org; Mon, 22 Jun 2026 08:26:35 +0000 Received: from mail-pj1-x102a.google.com ([2607:f8b0:4864:20::102a]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.98.2) (envelope-from ) id 1wbZzG-00000001YkI-0NUo for pgsql-hackers@lists.postgresql.org; Mon, 22 Jun 2026 08:26:34 +0000 Received: by mail-pj1-x102a.google.com with SMTP id 98e67ed59e1d1-36d630c0e35so4045742a91.3 for ; Mon, 22 Jun 2026 01:26:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1782116792; cv=none; d=google.com; s=arc-20240605; b=cmoDw9C9iD7ffoflKziCmFjoST9cIagmK9Y0RVYjbJyF71hpzFcaur5Tow5dbrX7do vuQaPCdLbCsRqqtexa9FM4j+/Ap+dpEmA9fLRcR+a5RlvQ7EeW3e33c4Iu500C5CXcV1 OWnivFL0DIgmW0BeCUem+/JBFdWyi32bPjAairnThWksfjHqI3QGqE0i3OJKG8yFhQIO 0x8dqOdEl7ukr8ijFv2maAwLy7p56Ny9ymw5jYSxBnucScTqF/SQ7/O/z/Bpjf8UUq1R OCIjuYNbf9pOL2TQro7zC1k4ut5BrphK6gWDc5H1m7Y2k+Y0lSwJ9PRucTDVnnYfM6el BsXg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:dkim-signature; bh=g/IyG2WC/eg91MDfcKiHOOyPJZc9NSeWrWKeP1t+DpM=; fh=9+Zt8d+c8AETve3MPmu2qs+bCQSioyFtYWuNbP2ftVs=; b=RLP1Zotx8EAVPzU+fQkUPkIWETllrLjr569jdUN4zLkjA0x6ogM9Xh1L5abTnozahx RRERM09NGOL5VLmQuc97YNBtVJN580Fp5KRBnvb1tApjNHobpapdZhulqcERmKW6cghg tiyAXmUiiM0Ln6wM6dr8ohwf5beBIlwYrMDRbN5AUWx2jz36BMgzcNlWrmsiK0XvBdlI 2EMlvL1jY31vlNVJF2j9vm01D89QKEMns7x2NKzWQcAdxbRbRkSkDMxw4zLfwIL60ppE Qp1Gtv1TisBTjezoaiUqalm3QUUmK/Qbrgp2/KNahkNn5gF+0ITBDy7VORdcs+9C9ssC Qqiw==; darn=lists.postgresql.org ARC-Authentication-Results: i=1; mx.google.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1782116792; x=1782721592; darn=lists.postgresql.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=g/IyG2WC/eg91MDfcKiHOOyPJZc9NSeWrWKeP1t+DpM=; b=OMbXXBN4A7jdJ06BfU7Un++BfyeioPzRB5v+mWdfACZtiK1Ect1yEjyrpYYRNYS7zm I7OuNllz4ZLPuTs0h52lRH2IyqSJkUpEQD1JyjJK6x2bSfDwNT/eQqF9Os9BeAVDcIc7 5fS0J2b+2L4otJpUo+6HWVX0wSMndM1ArgabqpeEZveFdSN9fTI2+2HP/FmwNUU+NVVb 4YjjeZUqNoTnbMfeJ35CkhM11yI9hfCOYIJF3Y9kkZazQLakzRFweVU7fTXifnP8IKk+ MwAgD6wFpXxp2Web9ISvtjA4S+8AiS1b593ccIdI6VPkLq2umlBarZTWtX5PB3yroHVz +KAw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1782116792; x=1782721592; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=g/IyG2WC/eg91MDfcKiHOOyPJZc9NSeWrWKeP1t+DpM=; b=QiwEY4ZxYKoqlUhjf00cxCpClXt7YqnxxM0LT1JBaekrLrz08TK1gSRZbeTmbHK+0Z 0rFc9fN5qEzS4A4XgA+Dbn+YMXfcUKgG+lNeDHJSS7Va8whCl386xGcD+UDu5eQxw2Fu nZAleMV++kkhVx8pt59MVnX04S4d16Nfl8RRlXzycF5A9tpwNbpkZ/pwTv8OVbp++4jB rGhAf3FGZTRlFJ6PEGb0KJ/3Y9tG5CS5qb08hdn3nqVaCWGvcwlOL2ibA7UWaig6sVfK 2fqbydRt4lokETVHQ6+KDHglBKkY2FxGVCeHf4obuoyiIr0kfGx9SGSrT1oFTGwtX76G ezVQ== X-Forwarded-Encrypted: i=1; AFNElJ/glvgH3lP2Z+/U075LxeQEGTzFXqgVPzDMHBu4NgIl7kolpnCN5jpNIkmutfInUeziMoacoLhui5xoiq/M@lists.postgresql.org X-Gm-Message-State: AOJu0YylyJZJXV7JnJZCNLtRgoNqXWvj9OAFYQQ/6I2oo837fdxOFhNP xGOucZ5Ud21y1Dd2SNCS517UcazJxCZ6sJoWYZLyQJZ0IDf3L1IrnL+G5W4oOqGh4bYP/pFoj8K qyTt4TZkI5FHHMM0cq+OGSuWesdGB6zk= X-Gm-Gg: AfdE7cm7yFnxFyTbRXu85rRTuzAbn2znAHCSJhTBmJffvp0ZUZkt3F6kvY9DRpS8eJ4 1NShvldSaGI4lMk/HpCtm5g+ivMZKKQq8j9nhBZmlLBP4Zg/oRc3w826Btv1NjdsvlTtc89xUcc 61SZqFObbUQVMwN8Yd053vDDcq4aEG7C1ILEiiiO/4BQ4PDCwC0ssSp9pZis2FCqtpHJ5BooYxv VVDhlcgxKPhO1lH0dAL4MXw6Cir4BY+q4g+YGtm9PUY0bokzI0m22dB3cAcg95SfzZRWHVgsY76 i/wWUnhVHiMJPpkKDhuow2jIBgZxC9rr7bdC47ZD X-Received: by 2002:a05:6a20:1609:b0:3a2:dabf:fef9 with SMTP id adf61e73a8af0-3bb34048493mr14487393637.27.1782116792511; Mon, 22 Jun 2026 01:26:32 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: shveta malik Date: Mon, 22 Jun 2026 13:56:19 +0530 X-Gm-Features: AVVi8CfpiKBSfOAc_bEdlAD8JFvNuiw8dGo9lVH6XzMM8o0Vnr-6eyFUHdP_bME Message-ID: Subject: Re: Proposal: Conflict log history table for Logical Replication To: vignesh C Cc: Amit Kapila , Dilip Kumar , Shlok Kyal , Peter Smith , Nisha Moond , Masahiko Sawada , Bharath Rupireddy , PostgreSQL Hackers , shveta malik Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk On Mon, Jun 22, 2026 at 9:33=E2=80=AFAM vignesh C wro= te: > > On Mon, 22 Jun 2026 at 08:41, Amit Kapila wrote= : > > > > On Sun, Jun 21, 2026 at 7:53=E2=80=AFPM vignesh C = wrote: > > > > > > While attempting to log a conflict, a concurrent ALTER SUBSCRIPTION > > > can change the conflict logging destination from all to log. In this > > > scenario, the apply worker may already have cached the conflictlogdes= t > > > information, including the OID of the current conflict log table. > > > However, the concurrent ALTER SUBSCRIPTION drops the conflict log > > > table as part of the destination change: > > > +Relation > > > +GetConflictLogDestAndTable(ConflictLogDest *log_dest) > > > +{ > > > + Oid conflictlogrelid; > > > + > > > + /* > > > + * Convert the text log destination to the internal enum. > > > MySubscription > > > + * already contains the data from pg_subscription. > > > + */ > > > + *log_dest =3D GetConflictLogDest(MySubscription->conflictlogd= est); > > > + > > > + /* Quick exit if a conflict log table was not requested. */ > > > + if (!CONFLICTS_LOGGED_TO_TABLE(*log_dest)) > > > + return NULL; > > > + > > > + conflictlogrelid =3D MySubscription->conflictlogrelid; > > > + > > > + Assert(OidIsValid(conflictlogrelid)); > > > + > > > + return table_open(conflictlogrelid, RowExclusiveLock); > > > +} > > > > > > As a result, when the apply worker later attempts to open the cached > > > conflict log table, table_open() fails because the relation has > > > already been dropped. This causes the error handling path itself to > > > fail before the conflict record can be written to either the conflict > > > log table or the server log. > > > > > > In such cases, the conflict record is effectively lost and is not > > > logged anywhere. For example: > > > 2026-06-21 19:31:13.592 IST [263598] LOG: logical replication apply > > > worker for subscription "sub1" has started > > > 2026-06-21 19:32:26.731 IST [263598] ERROR: could not open relation > > > with OID 16405 > > > 2026-06-21 19:32:26.731 IST [263598] CONTEXT: processing remote data > > > for replication origin "pg_16404" during message type "INSERT" for > > > replication target relation "public.t1" in transaction 698, finished > > > at 0/017D39A0 > > > 2026-06-21 19:32:26.735 IST [263471] LOG: background worker "logical > > > replication apply worker" (PID 263598) exited with exit code 1 > > > > > > Ideally, failure to access the conflict log table should not prevent > > > the conflict from being reported in the server log. This issue is > > > present with the v52 version. I have not yet checked if Amit's recent > > > patch posted a few minutes ago at [1] handles this issue. > > > > > > > There are two places in the patch from where we LOG/Insert the > > conflict data. First is ReportApplyConflict() where we LOG if the > > conflict arises from a non-ERROR path (aka conflicts other > > INSERT/UPDATE_EXISTS). In that case, the conflict data will be logged > > even when we fail to insert into CLT. Second is the place for > > conflicts that arose as ERRORs (aka INSERT/UPDATE_EXISTS), where the > > conflict information will be logged along with insert failure as > > CONTEXT. Can you please verify your test based on this input and share > > your findings and thoughts? > > The scenario I am testing is an insert_exists conflict. > On the publisher: > CREATE TABLE t1 (c1 int); > > On the subscriber: > CREATE TABLE t1 (c1 int PRIMARY KEY); > > Then execute the following on the publisher: > INSERT INTO t1 VALUES (10); > INSERT INTO t1 VALUES (10); > > The second insert generates an insert_exists conflict on the > subscriber. The conflict is reported and logged through the following > call chain: > apply_handle_insert > -> apply_handle_insert_internal > -> ExecSimpleRelationInsert > -> CheckAndReportConflict > -> ReportApplyConflict > > Pause execution in ReportApplyConflict() at > GetConflictLogDestAndTable(), immediately before opening the conflict > log table: > ... > return table_open(conflictlogrelid, RowExclusiveLock); > ... > > While the apply worker is paused, execute the following command concurren= tly: > ALTER SUBSCRIPTION sub1 > SET (conflict_log_destination =3D 'log'); > > This succeeds and drops the conflict log table: > NOTICE: dropped conflict log table "pg_conflict.pg_conflict_log_16404" > for subscription "sub1" > ALTER SUBSCRIPTION > > At this point, GetConflictLogDestAndTable() has already determined > that the conflict should be logged to a table and has cached the > corresponding relation OID. However, the concurrent ALTER SUBSCRIPTION > has removed that table. > > When execution resumes, the subsequent table_open() call fails with: > 2026-06-22 09:24:53.072 IST [304864] ERROR: could not open relation > with OID 16405 > > As a result, conflict processing itself fails before the conflict > details can be recorded. The conflict is therefore not logged to the > conflict log table and is also not emitted to the server log. > I understand this case, but I feel it isn't critical because the table is going to be dropped in parallel, so ultimately, all data is lost. At-max, we can provide a LOG when table-open fails, indicating that the CLT table is dropped concurrently and thus conflict-data cannott be logged to table. thanks Shveta