Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1wd3Vj-003r1d-3B for pgsql-hackers@arkaria.postgresql.org; Fri, 26 Jun 2026 10:10:12 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.96) (envelope-from ) id 1wd3Vi-00AKGx-27 for pgsql-hackers@arkaria.postgresql.org; Fri, 26 Jun 2026 10:10:10 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1wd3Vi-00AKGo-0J for pgsql-hackers@lists.postgresql.org; Fri, 26 Jun 2026 10:10:10 +0000 Received: from mail-pj1-x1030.google.com ([2607:f8b0:4864:20::1030]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.98.2) (envelope-from ) id 1wd3Vg-00000000JuA-0GMA for pgsql-hackers@lists.postgresql.org; Fri, 26 Jun 2026 10:10:09 +0000 Received: by mail-pj1-x1030.google.com with SMTP id 98e67ed59e1d1-37d82f3a244so357719a91.2 for ; Fri, 26 Jun 2026 03:10:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1782468607; cv=none; d=google.com; s=arc-20260327; b=aGZviQJrmIDARiFXnVzyV4BSYKqpLMEhQZUmpIlfatOU0PdNcz05VLAYRXr/B8UOp0 fRTEOUMgbdm+u26weM9Zd8QH7Kh+ySx6exZEEaP6/UuQQdd9+RE/NvZC/1UAfmDLDfpA ugkVztKRnm2aRlI3+qiwms6uPudZtVu8P4dRsZJVypEJ7efQaL0eur6Ax3nanHj9zta8 fyPb3nnEss2mXdmpjdwhgxLe0ygmXBe5jWG35F0f7aJMJgjJUAaFm7fNFNsU9KcIox76 QteFTdqpZcuAX6KP6bcQS9mLojKqs9JUsAC69VnRAtTzPOHLjgphNIf6obAe5BFPvT1O T9lg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:dkim-signature; bh=4nHEq32ofnSf+gMeUzM0Hx+iLsV3yEuQU2hPZcXcwXU=; fh=enmTdOyxfPe6QkOXyGur2qVePNGcycU+2oun5a4iazg=; b=L+et28qtRCPyyJ51iP+jP9v5oBYxCjoMmVSma7os6I8UNiF314aYsTJK1JA/265ZEP BVU2TosdkEMxSGTstlvxVtAGTOHZ0HOCTkbxngcB8/zGaF0dEcaOVcyxNiuBUP4aRRah fMJIfJGgDy6jt7f0Low/uqRs1FpaykeyUbayd5qwqL5JDXGqojvUF6j+TdCu2fjGPxo9 5yJ+zQmu1GhOn+aCzZo/V5HFQoQIh5SsulpHMfQ80VkhiB4DwHWh/1vsQL2F59yLzvez cA95X0zlLGioQYA4VsgqNZzz7mAy3RwSXJliSL429JCUHEpcK77G00TWeQeMUI9+KufI gk4g==; darn=lists.postgresql.org ARC-Authentication-Results: i=1; mx.google.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1782468607; x=1783073407; darn=lists.postgresql.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=4nHEq32ofnSf+gMeUzM0Hx+iLsV3yEuQU2hPZcXcwXU=; b=MbSG7h9qG8Ybnoqyk3ZGloFU3+Sb+TdeI4ECYGyI+zD/2yNXswUjp5qdk+1oY75g69 MMYXH2WByJrBNaU0Uw+5nD4XPKbvRBucolAaMHxM/pTSAx4oovD3A4QpDhAEY9TmiL87 RVycJIfus65SnWyOAaizjWKVUEBG1UOk8/LMKqUbKcZdZtQO8M/pXx87Zfk9ib8DxICO xePTD6Nr/EUu9Cf88X4waaE7z8zt49Sj4E4FUh7RhBxHuawABxJ1Qu+BuTZ7x8bB1pno fdDOg+Tq5+e3ySB1JYzzKgqgHLr9E9k4mhKypzL7uX1YnjhowGLk5YoR+wEO9kfbM3zG IWXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1782468607; x=1783073407; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=4nHEq32ofnSf+gMeUzM0Hx+iLsV3yEuQU2hPZcXcwXU=; b=CBMKN9DXrxYUv3Q5jDInUnG45BstrO0rTJJYXMIqe4eH6eEhp6v9LEm55VlFKzooL+ nNYKfGbqFElqnC4OuHHqBXP0BeZadwHbjF5fN8ddZLznk5NEgeFtQzQTm8zvuafYKRbz Sjhz/AQg6y1oDnlS0QLHgp2SFUsiAJfoJgIC3bppv3PwlU4To6Mo9GH5150nfAhpCcfD ZWqqciEE4VYj++ItIMIRIN5hCK104NkIlkOwQ8iKGLcKAleRSxETm/SgJ64Zisl+fdLf Q1wPfET65XoQ3h2a4MSmfc5mGsI9RxL+t5BwOm2NoZZCUG3twUhEko968iXzp5Hy14eK IWhA== X-Forwarded-Encrypted: i=1; AHgh+RpIUEGZVcyUn4Cj/uRk0iaa+PYTRxI0a7uXL2PP8enzgH6UZ6HGnbmkLiEA+yxCNT1ImlyG6p6UoViJ5b81@lists.postgresql.org X-Gm-Message-State: AOJu0YyYuVF4E77pPXs+ojtUOotS/LqrzuB3sPgqvqtJqbzBSdLoYPeu QmSkXz6w6aBCJQEdgd9p/UzTdGRmvCxPeMD1hDdVOEf9OWpEDgYPTJxwrF4hda/Zyz8hxF7A2xy 447xwyL40/Q7xnKtUIEngWcEyeF6TYwA= X-Gm-Gg: AfdE7cnzg3x2dRt1AhaOEyukbMrOJmgmgP3paT9IizAgiM7n4KEMzqds8+0nhCo3YvS ghMHa5ZfjYlWV4AxAJRXB+tj172seP4Hqdop69VEshkTSE2j5vxkhMuUYy8SoAWV5EWeS5/fsv0 yH237l06KnizFSzr4doqjc9uCnQwYRRkZdxBKjCP02bagEw+6jLExjhlXq+hlVQehC8VzEqpYrG R886tt1rrgJ0qMc1EL06ce+AKthdyuq07UqXkWHtd9wvSpPO63Hus15Q5zjSzpBkX7Ce6B0qclc sMGvxRBHUD6nHfOzpDukcUIh97oma4qK9QdeKw4Bh+Hos849aRQha9uqvY29a5qM62uepdHVIdU gY/4J1uFOsA== X-Received: by 2002:a17:902:a583:b0:2c7:d603:117f with SMTP id d9443c01a7336-2c7fc75a75amr48465055ad.26.1782468606826; Fri, 26 Jun 2026 03:10:06 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Amit Kapila Date: Fri, 26 Jun 2026 15:39:54 +0530 X-Gm-Features: AVVi8CfvK1OMWV71UOWXoav-spDMcmZbnaVFZTzCFHZ75irvyCmiT956jOi9A7M Message-ID: Subject: Re: Proposal: Conflict log history table for Logical Replication To: vignesh C Cc: Dilip Kumar , Nisha Moond , shveta malik , Peter Smith , Masahiko Sawada , Bharath Rupireddy , PostgreSQL Hackers Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk On Thu, Jun 25, 2026 at 4:57=E2=80=AFPM Amit Kapila wrote: > > On Thu, Jun 25, 2026 at 10:34=E2=80=AFAM vignesh C = wrote: > > > > On Wed, 24 Jun 2026 at 19:27, Dilip Kumar wrote= : > > > > > > > 2) jsonb supports indexing, whereas json does not. Was json chosen > > here because inserts are faster due to the lack of parsing and binary > > conversion? > > > > I think it is more because the local/remote tuples are usually data > which we want users to see as it was in original tables, using jsonb > can change the ordering of columns or remove some space etc. We can > add a comment on the lines of: "preserve the exact tuple > representation (column order, formatting) for the audit record; the > value is natively json so this avoids a per-conflict conversion". Now, > this is true for tuple data but I am not so sure if the same thing > applies to the replica_identity column in CLT. Would users ever want > to query using ORDER/GROUP BY on replica_identity or use DISTINCT on > it, because if that is possible then the current schema would result > into ERROR as follows: > postgres=3D# select * from pg_conflict.pg_conflict_log_16394 order by > replica_identity; > ERROR: could not identify an ordering operator for type json > > This needs more thoughts from the perspective of how users would like > to fetch the data. > The values stored for replica_identity are heterogeneous across the table (different key shapes per relid), and for the RI-FULL case it isn't even a stable identifier, two conflicts on the "same row" can carry different whole-tuple JSON. So, I don't see a case for creating an index on the RI column as well and rather users would like to see the exact value used to find the local tuple. So, we can add a comment on the lines: "The tuple/key columns (remote_tuple, replica_identity, local_conflicts) are typed json rather than jsonb on purpose: they hold an exact audit snapshot of the applied tuples and replica identity, and json preserves the verbatim representation whereas jsonb would normalize it. Indexing them (jsonb's main advantage) wouldn't help anyway, as the conflict log is looked up by its scalar columns (relid, conflict_type, commit timestamp) while these JSON columns are per-conflict payload to inspect, not search keys." One more thing that bothers me is that it will be inconvenient for users to identify whether logged values for replica_identity are for an index or a complete tuple via replica identity full, so I propose to add a boolean column replica_identity_full. I also considered to add replica_identity_index but preferred a boolean instead because this is a historical/audit row, and the index may be dropped or replaced later, leaving a meaningless OID which is opposite of what an audit record should do. The other question is whether we create an internal index on one of the columns? Currently, there's no established query pattern to index for. The plausible candidates are relid ("conflicts for table X"), or remote_commit_ts (time-based cleanup) but those are guesses. Indexing the JSON columns we've already ruled out (payload, not keys). Picking an index before we know how people actually query/prune the table risks indexing the wrong thing. We can cosider adding the index in later versions or even in PG20 itself if we get some user feedback. --=20 With Regards, Amit Kapila.