MIME-Version: 1.0
From: Durgamahesh Manne <maheshpostgres9@gmail.com>
Date: Fri, 13 Sep 2024 01:21:55 +0530
Message-ID: <CAJCZkoKbfzFSfTemPiZN-mBrgy7ob=9gFq-5BnNEbHsH3Fe-dA@mail.gmail.com>
Subject: Recommendations on Improving the debezium performance even on medium workload
To: pgsql-general@lists.postgresql.org, pgsql-in-general@postgresql.org, 
	Greg Sabino Mullane <htamfids@gmail.com>, "kdg.dev@gmail.com" <kdg.dev@gmail.com>
Content-Type: multipart/alternative; boundary="0000000000003405560621f1671d"
Archived-At: <https://www.postgresql.org/message-id/CAJCZkoKbfzFSfTemPiZN-mBrgy7ob%3D9gFq-5BnNEbHsH3Fe-dA%40mail.gmail.com>
Precedence: bulk

--0000000000003405560621f1671d
Content-Type: text/plain; charset="UTF-8"

Hi Team

How do we improve the debezium performance?
Recommendations at kafka configuration side
Agenda is to minimize the lag during moderate or high work load on db

default > poll.interval.ms = 500ms
Recommended value for balanced performance > 1000ms
Recommended value for high throughput > 1000ms to 5000ms


default > max.batch.size = 2048 allocated > 2560
Recommended value for balanced performance > 500 to 1000
Recommended value for high throughput > 2000 or 5000


default > max.queue.size = 8192 allocated > 10000
Recommended value for balanced performance > 7500 (1000 to 10000)
Recommended value for high throughput > 10000

based on kafka load we can optimize debezium connector performance


default > heartbeat.interval.ms = 0 allocated > 5000ms
Recommended value for this > 5000ms to 10000ms


Are these helpful  ?

Need auto connectivity even on connector(slot) failure

please let me know your recommendations please


Regards,

Durga Mahesh

--0000000000003405560621f1671d
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi Team=C2=A0<div><br></div><div>How do we improve=C2=A0th=
e debezium performance?</div><div>Recommendations at kafka configuration si=
de</div><div>Agenda is to minimize the lag during moderate or high work loa=
d on db=C2=A0</div><div><br></div><div><span class=3D"gmail-ui-provider gma=
il-a gmail-b gmail-c gmail-d gmail-e gmail-f gmail-g gmail-h gmail-i gmail-=
j gmail-k gmail-l gmail-m gmail-n gmail-o gmail-p gmail-q gmail-r gmail-s g=
mail-t gmail-u gmail-v gmail-w gmail-x gmail-y gmail-z gmail-ab gmail-ac gm=
ail-ae gmail-af gmail-ag gmail-ah gmail-ai gmail-aj gmail-ak" dir=3D"ltr"><=
p>default &gt; <a href=3D"http://poll.interval.ms">poll.interval.ms</a> =3D=
 500ms=C2=A0<br>
Recommended value for balanced performance &gt; 1000ms<br>
Recommended value for high throughput &gt; 1000ms to 5000ms</p><p>=C2=A0</p=
><p>=C2=A0</p><p>default &gt; max.batch.size =3D 2048 allocated &gt; 2560<b=
r>
Recommended value for balanced performance &gt; 500 to 1000<br>
Recommended value for high throughput &gt; 2000 or 5000</p><p>=C2=A0</p><p>=
default &gt; max.queue.size =3D 8192 allocated &gt; 10000<br>
Recommended value for balanced performance &gt; 7500 (1000 to 10000)<br>
Recommended value for high throughput &gt; 10000</p><p>based on kafka load =
we can optimize debezium connector performance=C2=A0</p><p>=C2=A0</p><p>def=
ault &gt; <a href=3D"http://heartbeat.interval.ms">heartbeat.interval.ms</a=
> =3D 0 allocated &gt; 5000ms<br>
Recommended value for this &gt; 5000ms to 10000ms</p><p><br></p><p>Are thes=
e helpful=C2=A0 ?</p><p>Need auto connectivity even on connector(slot) fail=
ure=C2=A0</p><p>please=C2=A0let=C2=A0me know your recommendations please=C2=
=A0</p><p><br></p><p>Regards,</p><p>Durga Mahesh</p></span></div></div>

--0000000000003405560621f1671d--