MIME-Version: 1.0
References: <Z_1Cq7JvabsFYjQo@ark> <CAKAnmmJxGHcQmzXqEQ-nppZjXyH7i5W_7Cz8JNMb2+6c5+D+og@mail.gmail.com>
 <CAM527d9+eGvdih1GxoPDJ2WYs6eitf39oNTxGxnCK1PgF=A8wg@mail.gmail.com>
 <1798838.1744759182@sss.pgh.pa.us> <20FB597F-641F-48F8-8428-D8DDBA802D58@yandex-team.ru>
 <CAExHW5udz081Unx5qcLZ89DnR4cp6OhW0TCoco3+AEkGw480uQ@mail.gmail.com>
 <212D5973-FDD0-4CF5-BCD0-2760EC319DF3@yandex-team.ru> <CAExHW5tq8ShEfboFa52wDMWmyaW+6kX_6x8MZ-0TP65tOhJ1wQ@mail.gmail.com>
In-Reply-To: <CAExHW5tq8ShEfboFa52wDMWmyaW+6kX_6x8MZ-0TP65tOhJ1wQ@mail.gmail.com>
From: Alastair Turner <minion@decodable.me>
Date: Wed, 16 Apr 2025 13:53:23 +0100
Message-ID: <CAC0Gmyw=-=+wTZdbyEzH2cxtdtR6RjMeLNEdH8HLdygt3k36uQ@mail.gmail.com>
Subject: Re: Built-in Raft replication
To: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Cc: Andrey Borodin <x4mmm@yandex-team.ru>, Tom Lane <tgl@sss.pgh.pa.us>, 
	Konstantin Osipov <kostja.osipov@gmail.com>, Greg Sabino Mullane <htamfids@gmail.com>, 
	Nikolay Samokhvalov <nik@postgres.ai>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Content-Type: multipart/alternative; boundary="000000000000ec68460632e4c585"
Archived-At: <https://www.postgresql.org/message-id/CAC0Gmyw%3D-%3D%2BwTZdbyEzH2cxtdtR6RjMeLNEdH8HLdygt3k36uQ%40mail.gmail.com>
Precedence: bulk

--000000000000ec68460632e4c585
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Wed, 16 Apr 2025 at 07:18, Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
wrote:

> On Wed, Apr 16, 2025 at 10:29=E2=80=AFAM Andrey Borodin <x4mmm@yandex-tea=
m.ru>
> wrote:
> >
> > If you use build-in failover you have to resort to 3 big Postgres
> machines because you need 2/3 majority. Of course, you can install
> MySQL-stype arbiter - host that had no real PGDATA, only participates in
> voting. But this is a solution to problem induced by built-in autofailove=
r.
>
> Users find it a waste of resources to deploy 3 big PostgreSQL
> instances just for HA where 2 suffice even if they deploy 3
> lightweight DCS instances. Having only some of the nodes act as DCS
> and others purely PostgreSQL nodes will reduce waste of resources.
>
> The experience of other projects/products with automated failover based o=
n
quorum shows that this is a critical issue for adoption. In the In-memory
Data Grid space (Coherence, Geode/GemFire) the question of how to ensure
that some nodes didn't carry any data comes up early in many architecture
discussions. When RabbitMQ shipped their Quorum Queues feature, the first
and hardest area of pushback was around all nodes hosting message content.

It's not just about the requirement for compute resources, it's also about
bandwidth and latency. Many large organisations have, for historical
reasons, pairs of data centres with very good point-to-point connectivity.
As the requirement for quorum witnesses has come up for all sorts of
things, including storage arrays, they have built arbiter/witness sites at
branches, colocation providers or even on the public cloud. More than not
holding user data or processing queries, the arbiter can't even be sent the
replication stream for the user data in the database, it just won't fit
down the pipe.

Which feels like a very difficult requirement to meet if the replication
model for all data is being changed to a quorum model.

Regards
Alastair

--000000000000ec68460632e4c585
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr">On Wed, 16 Apr 2025 at 07:18, Ashutosh Ba=
pat &lt;<a href=3D"mailto:ashutosh.bapat.oss@gmail.com">ashutosh.bapat.oss@=
gmail.com</a>&gt; wrote:</div><div class=3D"gmail_quote gmail_quote_contain=
er"><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;bor=
der-left:1px solid rgb(204,204,204);padding-left:1ex">On Wed, Apr 16, 2025 =
at 10:29=E2=80=AFAM Andrey Borodin &lt;<a href=3D"mailto:x4mmm@yandex-team.=
ru" target=3D"_blank">x4mmm@yandex-team.ru</a>&gt; wrote:<br>&gt;<br>
&gt; If you use build-in failover you have to resort to 3 big Postgres mach=
ines because you need 2/3 majority. Of course, you can install MySQL-stype =
arbiter - host that had no real PGDATA, only participates in voting. But th=
is is a solution to problem induced by built-in autofailover.<br>
<br>
Users find it a waste of resources to deploy 3 big PostgreSQL<br>
instances just for HA where 2 suffice even if they deploy 3<br>
lightweight DCS instances. Having only some of the nodes act as DCS<br>
and others purely PostgreSQL nodes will reduce waste of resources.<br>
<br></blockquote><div>The experience of other projects/products with automa=
ted failover based on quorum shows that this is a critical issue for adopti=
on. In the In-memory Data Grid space (Coherence, Geode/GemFire) the questio=
n of how to ensure that some nodes didn&#39;t carry any data comes up early=
 in many architecture discussions. When RabbitMQ shipped their Quorum Queue=
s feature, the first and hardest area of pushback was around all nodes host=
ing message content.<br><br>It&#39;s not just about the requirement for com=
pute resources, it&#39;s also about bandwidth and latency. Many large organ=
isations have, for historical reasons, pairs of data centres with very good=
 point-to-point connectivity. As the requirement for quorum witnesses has c=
ome up for all sorts of things, including storage arrays, they have built a=
rbiter/witness sites at branches, colocation providers or even on the publi=
c cloud. More than not holding user data or processing queries, the arbiter=
 can&#39;t even be sent the replication stream for the user data in the dat=
abase, it just won&#39;t fit down the pipe.</div><div><br></div><div>Which =
feels like a very difficult requirement to meet if the replication model fo=
r all data is being changed to a quorum model.</div><div><br>Regards<br>Ala=
stair <br></div></div></div>

--000000000000ec68460632e4c585--