MIME-Version: 1.0
References: <Z_1Cq7JvabsFYjQo@ark> <CAKAnmmJxGHcQmzXqEQ-nppZjXyH7i5W_7Cz8JNMb2+6c5+D+og@mail.gmail.com>
 <CAM527d9+eGvdih1GxoPDJ2WYs6eitf39oNTxGxnCK1PgF=A8wg@mail.gmail.com>
 <1798838.1744759182@sss.pgh.pa.us> <20FB597F-641F-48F8-8428-D8DDBA802D58@yandex-team.ru>
In-Reply-To: <20FB597F-641F-48F8-8428-D8DDBA802D58@yandex-team.ru>
From: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Date: Wed, 16 Apr 2025 10:03:15 +0530
Message-ID: <CAExHW5udz081Unx5qcLZ89DnR4cp6OhW0TCoco3+AEkGw480uQ@mail.gmail.com>
Subject: Re: Built-in Raft replication
To: Andrey Borodin <x4mmm@yandex-team.ru>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, Konstantin Osipov <kostja.osipov@gmail.com>, 
	Greg Sabino Mullane <htamfids@gmail.com>, Nikolay Samokhvalov <nik@postgres.ai>, pgsql-hackers@lists.postgresql.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Archived-At: <https://www.postgresql.org/message-id/CAExHW5udz081Unx5qcLZ89DnR4cp6OhW0TCoco3%2BAEkGw480uQ%40mail.gmail.com>
Precedence: bulk

On Wed, Apr 16, 2025 at 9:37=E2=80=AFAM Andrey Borodin <x4mmm@yandex-team.r=
u> wrote:
>
> My view is what Konstantin wants is automatic replication topology manage=
ment. For some reason this technology is called HA, DCS, Raft, Paxos and ma=
ny other scary words. But basically it manages primary_conn_info of some no=
des to provide some fault-tolerance properties. I'd start to design from he=
re, not from Raft paper.
>

In my experience, the load of managing hundreds of replicas which all
participate in RAFT protocol becomes more than regular transaction
load. So making every replica a RAFT participant will affect the
ability to deploy hundreds of replica. We may build an extension which
has a similar role in PostgreSQL world as zookeeper in Hadoop. It can
be then used for other distributed systems as well - like shared
nothing clusters based on FDW. There's already a proposal to bring
CREATE SERVER to the world of logical replication - so I see these two
worlds uniting in future. The way I imagine it is some PostgreSQL
instances, which have this extension installed, will act as a RAFT
cluster (similar to Zookeeper ensemble or etcd cluster). The
distributed system based on logical replication or FDW or both will
use this ensemble to manage its shared state. The same ensemble can be
shared across multiple distributed clusters if it has scaling
capabilities.

--=20
Best Wishes,
Ashutosh Bapat