Message-ID: <67ff6156.050a0220.18223a.416b@mx.google.com>
Sender: Michael Banck <mbanck@gmail.com>
Date: Wed, 16 Apr 2025 09:50:45 +0200
From: Michael Banck <mbanck@gmx.net>
To: Andrey Borodin <x4mmm@yandex-team.ru>
Cc: Tom Lane <tgl@sss.pgh.pa.us>,
	Konstantin Osipov <kostja.osipov@gmail.com>,
	Greg Sabino Mullane <htamfids@gmail.com>,
	Nikolay Samokhvalov <nik@postgres.ai>,
	PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Subject: Re: Built-in Raft replication
References: <Z_1Cq7JvabsFYjQo@ark>
 <CAKAnmmJxGHcQmzXqEQ-nppZjXyH7i5W_7Cz8JNMb2+6c5+D+og@mail.gmail.com>
 <CAM527d9+eGvdih1GxoPDJ2WYs6eitf39oNTxGxnCK1PgF=A8wg@mail.gmail.com>
 <1798838.1744759182@sss.pgh.pa.us>
 <20FB597F-641F-48F8-8428-D8DDBA802D58@yandex-team.ru>
 <1832513.1744777613@sss.pgh.pa.us>
 <0ED408AD-2209-4D60-9142-B6D01DD0C46F@yandex-team.ru>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <0ED408AD-2209-4D60-9142-B6D01DD0C46F@yandex-team.ru>
User-Agent: Mutt/1.10.1 (2018-07-13)
Archived-At: <https://www.postgresql.org/message-id/67ff6156.050a0220.18223a.416b%40mx.google.com>
Precedence: bulk

Hi,

On Wed, Apr 16, 2025 at 10:24:48AM +0500, Andrey Borodin wrote:
> I think I can provide some reasons why it cannot be neither extension,
> nor any part running within postmaster reign.
> 
> 1. When joining cluster, there’s not PGDATA to run postmaster on top
> of it.
> 
> 2. After failover, old Primary node must rejoin cluster by running
> pg_rewind and following timeline switch.
> 
> The system in hand must be able to manipulate with PGDATA without
> starting Postgres.

Yeah, while you could maybe implement some/all of the RAFT protocol in
an extension, actually building something useful on top with regards to
high availability or distributed whatever does not look feasible.
 
> My question to Konstantin is Why wouldn’t you just add Raft to
> Patroni?

Patroni can use pysyncobj, which is a Python implementation of RAFT, so
then you do not need an external RAFT provider like etcd, consul or
zookeeper. However, it is deemed deprecated by the Patroni authors due
to being difficult to debug when it breaks.

I guess a better Python implementation of RAFT for Patroni to use or
Patroni to implement it itself would help, but I believe nobody is
working on the latter right now, nor has any plans to do so. And there
also does not seem to be anybody working on a better pysyncobj.

> Is there a reason why something like Patroni is not in core and noone
> rushes to get it in?  Everyone is using it, or system like it.

Well, Patroni is written in Python, for starters.  It also does a lot
more than just leader election / cluster config. So I think nobody
seriously thought about proposing to put Patroni into core so far.

I guess the current proposal tries to be a step into the "something like
Patroni in core" if you tilt your head a little. It's just that the
whole thing would be a really big step for Postgres, maybe similar to
deciding we want in-core replication way back when.


Michael