Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1taIdp-000a3s-LN for pgsql-general@arkaria.postgresql.org; Tue, 21 Jan 2025 18:06:22 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1taIdo-0035Nn-HL for pgsql-general@arkaria.postgresql.org; Tue, 21 Jan 2025 18:06:20 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1taIdo-0035Nf-2G for pgsql-general@lists.postgresql.org; Tue, 21 Jan 2025 18:06:20 +0000 Received: from mail-ed1-x535.google.com ([2a00:1450:4864:20::535]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.96) (envelope-from ) id 1taIdl-000luK-2Z for pgsql-general@lists.postgresql.org; Tue, 21 Jan 2025 18:06:19 +0000 Received: by mail-ed1-x535.google.com with SMTP id 4fb4d7f45d1cf-5d982de9547so11675828a12.2 for ; Tue, 21 Jan 2025 10:06:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1737482777; x=1738087577; darn=lists.postgresql.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=tlEO2jJSgIX7PJNaRmKjD7GX1ubkA9u4fMvqz+0D4w8=; b=MNm+pOZhKBe8zGWUgMw27RTthIrjfMsqn6aHVGyOOuRHpcbmHbza47RKAorbbH0C6f m2aMzNHBP5eeC28DiLiT2VqhNlcx+AxxyjMbhY2//GXX5/yQEHvEn0s6r7Lbe5br5CBb 4L6W7Zz+oYJ/44v/dn72Rx6rNJHuqzUoWIvlKe2zHlpCovSuT2tofAiVdgOsz2lNL35F yCnK4wJLtaLb8sHV7q7kQscpkcFBc8fZ7y4LPyT4scq005B9L6CKltSKTWxsKcpRciaK z38khcRYpZsBeGA6juzY1bSHPfW2uHW+LwgtNkLafibU0z/XYGomKn61EppvYjxbG2+a A2Og== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737482777; x=1738087577; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=tlEO2jJSgIX7PJNaRmKjD7GX1ubkA9u4fMvqz+0D4w8=; b=Cpzxma8KRvKKZG58ZMkWrSLpAjiJhw4E6KVgO2bcNiHozomucQoYcU/jD1PVPheUv4 K0TANJIDRO6YzmBQEUmwre6gAOR2q8As+TP85c7F0YtzXXQE9XoTPDh8bt9DLRnn0Tth HG/tlGZjEpPB2A1eJApAr/YtR7d7uiHDpnCARMj/u8PpJBSnxY4ZYQ8FVj+tYhH920TK TH95fOgsFy3dkp8nzBqOAMiyJcuEL6+ea9vI0WYjUf0q0ENjyyzyVpZt9qOTCZ/04jlL sMeB64OqE1R7uhvhQ17ieuSQeKisqChVwVIYaWIIFkUhfa1CGupOmseTnNaEpTcBHD6V GdEw== X-Gm-Message-State: AOJu0YyDRbFi3upiqJGoAL0sPVlqvfwQAtK91qYlhPfJ7gFmAwMnDOpV DFPr9tbIYID9Te9lsRN3aMCEDJunLwFS1bidY2ddw0V8QhdqWNRvStNB1oBajpIBKilJC8nDotj bidhlZrPgd5zBTIitvG/AgZ4Kxhf9Xe49mhY= X-Gm-Gg: ASbGncsLdH6OiEhxFU+3ekvk+GEh1ZAV3ipr351uripxGbu6sVK3REn117iBB+eoZvm xQgZPteEZnH7G4KgoS5MvfPHRvf5IH2I33iTYHm7ZTj5aHnDMSLA= X-Google-Smtp-Source: AGHT+IFych1oqNCbWJ/afn1SItuHXbhcdX/xqJL4AYpQfzwsgbmHYmDxmWc89filZWK0A6MiLnmqcPi5cEGm7S6Oj24= X-Received: by 2002:a17:907:6d09:b0:aa6:8935:ae71 with SMTP id a640c23a62f3a-ab38b0b7f21mr1692222466b.12.1737482776544; Tue, 21 Jan 2025 10:06:16 -0800 (PST) MIME-Version: 1.0 References: <79b7d5dc-3a1d-4ef0-9a3d-055268dc7d09@aklaver.com> In-Reply-To: <79b7d5dc-3a1d-4ef0-9a3d-055268dc7d09@aklaver.com> From: Durgamahesh Manne Date: Tue, 21 Jan 2025 23:36:50 +0530 X-Gm-Features: AbW1kvapkFA417H8N9p3vJ5wd0L_ZMlRR622wyjMDtgXXRSLyMoZalV3v83UdPA Message-ID: Subject: Re: Records count mismatch with logical replication To: Adrian Klaver Cc: pgsql-general Content-Type: multipart/alternative; boundary="000000000000b30c16062c3b3b4a" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --000000000000b30c16062c3b3b4a Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Tue, Jan 21, 2025 at 11:26=E2=80=AFPM Adrian Klaver wrote: > On 1/21/25 09:38, Durgamahesh Manne wrote: > > > > > > > > > Hi Adrian Klaver > > > > Really Thanks for your quick response > > > > This happened during repack lag went to more than 350Gb then gradually > > decreased to minimal lag after running pg_repack > > I don't use pg_repack so I don't know what effect it would have on the > process. > > > > > 3) Define how lag is being calculated and what 'minimal' is. > > > > postgres=3D> select > > slot_name,pg_wal_lsn_diff(pg_current_wal_lsn(),restart_lsn) as > bytes_behind, > > pg_size_pretty(pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn)) as > > behind_size, active from pg_replication_slots; > > slot_name | bytes_behind | behind_size | active > > -------------+--------------+-------------+-------- > > cls_eva_msa | 22906216 | 22 MB | t > > > > 4) Define how the record counts are being derived. > > Source : archiving=3D> select count(*) from archiving.events_archive= ; > > count > > --------- > > 1262908 > > (1 row) > > > > Destination : archiving=3D> select count(*) from archiving.events_archi= ve ; > > count > > --------- > > 1252062 > > (1 row) > > 22,906,216 bytes/10,846 rows works out to 2112 bytes per row. > > Is that a reasonable per row estimate? > > > > > 5) The network distance between the servers. > > > > Both are under same vpc security groups > > > > Regards > > Durga Mahesh > > -- > Adrian Klaver > adrian.klaver@aklaver.com > > Hi Adrian Klaver 22,906,216 bytes/10,846 rows works out to 2112 bytes per row. Is that a reasonable per row estimate? Yes sometimes would be vary Regards, Durga Mahesh --000000000000b30c16062c3b3b4a Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Tue, Jan 21,= 2025 at 11:26=E2=80=AFPM Adrian Klaver <adrian.klaver@aklaver.com> wrote:
On 1/21/25 09:38, Durgamahesh Manne = wrote:
>
>

>
> Hi Adrian Klaver
>
> Really Thanks for your quick response
>
> This happened during repack lag went to more than 350Gb then gradually=
> decreased=C2=A0to minimal=C2=A0lag after running pg_repack

I don't use pg_repack so I don't know what effect it would have on = the
process.
>

> 3) Define how lag is being calculated and what 'minimal' is. >
> postgres=3D> select
> slot_name,pg_wal_lsn_diff(pg_current_wal_lsn(),restart_lsn) as bytes_b= ehind,
> pg_size_pretty(pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn)) as =
> behind_size, active from pg_replication_slots;
>=C2=A0 =C2=A0 slot_name =C2=A0| bytes_behind | behind_size | active
> -------------+--------------+-------------+--------
>=C2=A0 =C2=A0cls_eva_msa | =C2=A0 =C2=A0 22906216 | 22 MB =C2=A0 =C2=A0= =C2=A0 | t
>
> 4) Define how the record counts are being derived.
>=C2=A0 =C2=A0 Source : archiving=3D> select count(*) from archiving.= events_archive ;
>=C2=A0 =C2=A0 count
> ---------
>=C2=A0 =C2=A01262908
> (1 row)
>
> Destination : archiving=3D> select count(*) from archiving.events_a= rchive ;
>=C2=A0 =C2=A0 count
> ---------
>=C2=A0 =C2=A01252062
> (1 row)

22,906,216 bytes/10,846 rows=C2=A0 works out to 2112 bytes per row.

Is that a reasonable per row estimate?

>
> 5) The network distance between the servers.
>
>=C2=A0 =C2=A0Both are under same vpc security groups
>
> Regards
> Durga Mahesh

--
Adrian Klaver
adrian.klave= r@aklaver.com

=C2=A0
Hi Adrian Klaver=C2=A0

22,906,216 bytes/10,846 rows=C2=A0 works out= to 2112 bytes per row.

Is that a reasonable per row estimate?
=

Yes=C2=A0 sometimes would be vary=C2=A0

Regards,
Durga Mahesh

= =C2=A0
--000000000000b30c16062c3b3b4a--