Subject: Re: PostgreSQL Active-Active Clustering

From: Christophe Pettus <[email-masked]>
Date: Mon, 15 Jul 2024 12:28:14 -0700

> On Jul 15, 2024, at 12:06, Sarkar, Subhadeep <[email-masked]> wrote:
> 
>   • Does the Community edition of PostgreSQL provide NATIVE active-active high availability clustering with objectives of scalability, load balancing and high availability without using any extensions or external components or usage of Kubernetes/Dockers. 
> 
>   • In the Community edition of PostgreSQL is it possible to setup a database cluster in load balancing mode and provide vertical and horizontal scalability without repartitioning or changes to the database objects or 3rd party transaction routing mechanisms using NATIVE features only (i.e. without using any extensions or external components or usage of Kubernetes/Dockers).
> 
>   • In the Community edition of PostgreSQL is it possible to setup  a cluster where all the nodes are able to concurrently read-write the underlying database image using NATIVE features (i.e. without using any extensions or external components or usage of Kubernetes/Dockers).

Short answer: No.

The community version of PostgreSQL, without any extensions beyond what is available in the core distribution, supports none of these.  I will offer that your client is not being realistic if these are their requirements, and they expect them to be fulfilled by the core distribution of any open-source database.

There are commercial extensions to PostgreSQL that provide the first, but PostgreSQL does not do so out of the box.  It is possible, now, to build this on top of community PostgreSQL with logical replication, but there is notable development work involved, and you cannot just drop an existing database into PostgreSQL and expect this to work.  In any event, you will need to make sure the schema is compatible with an active-active model.

For the second, you can explore open-source projects such as Citus, but some attention to the schema and queries will be required. 

No product, either commercial or open-source, provides the last one (read-write shared storage), although there are commercial products that provide for a shared-storage model single-writer, multiple-reader model (for example, Amazon Aurora).

---

From: Ron Johnson <[email-masked]>
Date: Mon, 15 Jul 2024 15:55:19 -0400

> > On Jul 15, 2024, at 12:06, Sarkar, Subhadeep <[email-masked]>
[snip]

> >   • In the Community edition of PostgreSQL is it possible to setup  a
> cluster where all the nodes are able to concurrently read-write the
> underlying database image using NATIVE features (i.e. without using any
> extensions or external components or usage of Kubernetes/Dockers).
>
[snip]

> No product, either commercial or open-source, provides the last one
> (read-write shared storage), although there are commercial products that
> provide for a shared-storage model single-writer, multiple-reader model
> (for example, Amazon Aurora).
>

This "lack of products" puzzles me, because DEC was doing this with VAX
(then Alpha and Itanium) clusters 40 years ago via a Distributed Lock
Manager integrated deep into VMS.  Their Rdb and (CODASYL) DBMS products
used those functions extensively.

(In the late 1990s, they sold the DLM code to Oracle, which is where RAC
comes from.)

It was shared-disk, multiple-writer, because the DLM allowed for locking at
the row level.  Thus, a half dozen cluster nodes could hold write locks on
different rows on the same data page.

---

From: Christoph Moench-Tegeder <[email-masked]>
Date: Mon, 15 Jul 2024 23:54:13 +0200

> Manager integrated deep into VMS.  Their Rdb and (CODASYL) DBMS products

Tech and trade-offs have changed over the last 40 years :)
These days you can so many cores in one package, while "more than one
processor" was quite a feat in the 80ies ("A dual processor VAX 11/780",
1982 https://dl.acm.org/doi/10.5555/800048.801738; also the 11/782 and
11/784), and you get so much RAM and storage (even fast storage, if
you keep it local) with that package. Response Latency really jumps
if you have to communicate with anything outside your box.
While latency matters, the number of problems where you absolutely
need that distributed lock manager has not really grown that much,
I think.

Regards,
Christoph

-- 
Spare Space

---

From: Ron Johnson <[email-masked]>
Date: Mon, 15 Jul 2024 22:26:26 -0400

> need that distributed lock manager has not really grown that much,

 Customers still want High Availability, and VMS Clusters were great for
HA.

---

From: Achilleas Mantzios <[email-masked]>
Date: Tue, 16 Jul 2024 08:26:55 +0300

Στις 15/7/24 22:55, ο/η Ron Johnson έγραψε:
>     >   • In the Community edition of PostgreSQL is it possible to
>     setup  a cluster where all the nodes are able to concurrently
>     read-write the underlying database image using NATIVE features
>     (i.e. without using any extensions or external components or usage
>     of Kubernetes/Dockers).
>
> [snip]
>
>     No product, either commercial or open-source, provides the last
>     one (read-write shared storage), although there are commercial
>     products that provide for a shared-storage model single-writer,
>     multiple-reader model (for example, Amazon Aurora).
>
>
> This "lack of products" puzzles me, because DEC was doing this with 
> VAX (then Alpha and Itanium) clusters 40 years ago via a Distributed 
> Lock Manager integrated deep into VMS. Their Rdb and (CODASYL) DBMS 
> products used those functions extensively.
IMHO IBM did something similar with their shared DASDi back in 70s+. 
There was serialization mechanism enabling concurrent writes to data 
sets (meaning files). Not to mention IBM had great VM technology back in 
the day.
>
> (In the late 1990s, they sold the DLM code to Oracle, which is where 
> RAC comes from.)
>
> It was shared-disk, multiple-writer, because the DLM allowed for 
> locking at the row level.  Thus, a half dozen cluster nodes could hold 
> write locks on different rows on the same data page.
>
-- 
Achilleas Mantzios
  IT DEV - HEAD
  IT DEPT
  Dynacom Tankers Mgmt (as agents only)