Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1uciYP-00G89c-Gs for pgsql-general@arkaria.postgresql.org; Fri, 18 Jul 2025 10:43:01 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1uciYN-00Ag68-8y for pgsql-general@arkaria.postgresql.org; Fri, 18 Jul 2025 10:42:59 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1uciYM-00Ag5y-S2 for pgsql-general@lists.postgresql.org; Fri, 18 Jul 2025 10:42:59 +0000 Received: from mail-pl1-x634.google.com ([2607:f8b0:4864:20::634]) by magus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.96) (envelope-from ) id 1uciYL-008QMW-0D for pgsql-general@lists.postgresql.org; Fri, 18 Jul 2025 10:42:59 +0000 Received: by mail-pl1-x634.google.com with SMTP id d9443c01a7336-234b9dfb842so16427385ad.1 for ; Fri, 18 Jul 2025 03:42:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1752835375; x=1753440175; darn=lists.postgresql.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=uElEj9OJugYbhoWnZrqOmr+lb+wPevaIBcOmFh6KSGI=; b=AvRZ//0Q3oTJoo7iKMyV+GF+g8jH0c8Gg1qgLyBFBM9wBVQ8Y81tWmT2/NsIq/0lXv JXkjPW3H/rx6p63jXAAAotQFeZ2nHtRPqjqsMnLWCDK72iClRlhh061Tca7kSjJ8iXdA GYkUYk1T5GWXY9pujV1CKWup8TNEI+lwvd4kobzzO9KLtf+mdYTscUrwd10khJBewWVu f+uxlmcARDI3aOOtcvFOf/AEpgWQ+rvMKbGf9kWMUZhww301CP5aA0AL+ym35cVIXFFi qZzWeR75U6gjIt8FVEWCUMjHwJk9NtCGfNgGIfecb9aOinkA3qo9ekG9RYXqk+VmKmje fdKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1752835375; x=1753440175; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=uElEj9OJugYbhoWnZrqOmr+lb+wPevaIBcOmFh6KSGI=; b=aCaYxiNrGjDsmN9RbtokZ6GSGvJGZFb1hM9SNZ+Wjtd5gla4wSZbcnwEGpWTh4mfFc ezwwUvEzo/l1xPNPY2EyFjSmbQN5HIw0/RMWrdEfT15gt5zBw0Oz0U4M77VyXfcokLdr 3ER9P7tGxlEZItqX/47yyqzaW+21o366fjUAN4RYM8yJf9Z/0MCPJJPHse5gL/4ovtDg vgHg4k+yjw+7nsGoHXRPu7TuTFuC5PKGJa7kyguOzyd8FBK5R36ed0IXmPZI4y/H6Gpm fPD6jjxSF0UBi8UDe766kUIUYr74z/AvB0zE2cX9lwp52w+QyqTPDgpOn7L+Wyy3bTcz L4Xg== X-Gm-Message-State: AOJu0Ywtx74GqnUPYE71g1Q8hAaicVfMRDKFjCkaRFia9Y/Lii9070Cj qAhQwIoQkL2ZRC++16MmSD47jN9rbe1qGivpuPHrh6Rupa54RR+uLIXKglcekutr5XqQxn+TV66 MYj0nN+liXIH3/za/diGtvnmDrrfpmJI= X-Gm-Gg: ASbGncutkB3OF0ZFROAQIMNFyQRL/aJOe2wU2RdvLmQUX/1YQ3oZqHZ1srVQNC45M3f ehXgoBggk8gshRGCldwM6RZp+6jF9UHqHPCgXFzPpo1pxVGoNGITEe1TtOGkvMMyFHaNHCrFnHZ BZYX0H+caSGb+N3m+fnnXobeN5acMhepgdLW0UEqDm55mxNUjHmz3jELfa+FrmC/mI9KQUIAoYr TzQ X-Google-Smtp-Source: AGHT+IENS40UEbsl+tK8J3+GwtOtqdLAP/KXcfcNUtXcr9qmtlH+UG9U+bP/udTjc1hriP50KZFR/B9JS3v+kK8K7+s= X-Received: by 2002:a17:903:1106:b0:234:98eb:8eda with SMTP id d9443c01a7336-23e2572aad3mr161947125ad.28.1752835374300; Fri, 18 Jul 2025 03:42:54 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Seref Arikan Date: Fri, 18 Jul 2025 11:42:42 +0100 X-Gm-Features: Ac12FXy1ERq3E_BqIZfKW3N2Kvcukntom44GmGiRt-PPoku6DRr411z4r4jiJRE Message-ID: Subject: Re: PostgreSQL on S3-backed Block Storage with Near-Local Performance To: Pierre Barre Cc: pgsql-general@lists.postgresql.org Content-Type: multipart/alternative; boundary="000000000000d5c387063a31c979" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --000000000000d5c387063a31c979 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Sorry, this was meant to go to the whole group: Very interesting!. Great work. Can you clarify how exactly you're running postgres in your tests? A specific AWS service? What's the test infrastructure that sits above the file system? On Thu, Jul 17, 2025 at 11:59=E2=80=AFPM Pierre Barre wro= te: > Hi everyone, > > I wanted to share a project I've been working on that enables PostgreSQL > to run on S3 storage while maintaining performance comparable to local > NVMe. The approach uses block-level access rather than trying to map > filesystem operations to S3 objects. > > ZeroFS: https://github.com/Barre/ZeroFS > > # The Architecture > > ZeroFS provides NBD (Network Block Device) servers that expose S3 storage > as raw block devices. PostgreSQL runs unmodified on ZFS pools built on > these block devices: > > PostgreSQL -> ZFS -> NBD -> ZeroFS -> S3 > > By providing block-level access and leveraging ZFS's caching capabilities > (L2ARC), we can achieve microsecond latencies despite the underlying > storage being in S3. > > ## Performance Results > > Here are pgbench results from PostgreSQL running on this setup: > > ### Read/Write Workload > > ``` > postgres@ubuntu-16gb-fsn1-1:/root$ pgbench -c 50 -j 15 -t 100000 example > pgbench (16.9 (Ubuntu 16.9-0ubuntu0.24.04.1)) > starting vacuum...end. > transaction type: > scaling factor: 50 > query mode: simple > number of clients: 50 > number of threads: 15 > maximum number of tries: 1 > number of transactions per client: 100000 > number of transactions actually processed: 5000000/5000000 > number of failed transactions: 0 (0.000%) > latency average =3D 0.943 ms > initial connection time =3D 48.043 ms > tps =3D 53041.006947 (without initial connection time) > ``` > > ### Read-Only Workload > > ``` > postgres@ubuntu-16gb-fsn1-1:/root$ pgbench -c 50 -j 15 -t 100000 -S > example > pgbench (16.9 (Ubuntu 16.9-0ubuntu0.24.04.1)) > starting vacuum...end. > transaction type: > scaling factor: 50 > query mode: simple > number of clients: 50 > number of threads: 15 > maximum number of tries: 1 > number of transactions per client: 100000 > number of transactions actually processed: 5000000/5000000 > number of failed transactions: 0 (0.000%) > latency average =3D 0.121 ms > initial connection time =3D 53.358 ms > tps =3D 413436.248089 (without initial connection time) > ``` > > These numbers are with 50 concurrent clients and the actual data stored i= n > S3. Hot data is served from ZFS L2ARC and ZeroFS's memory caches, while > cold data comes from S3. > > ## How It Works > > 1. ZeroFS exposes NBD devices (e.g., /dev/nbd0) that PostgreSQL/ZFS can > use like any other block device > 2. Multiple cache layers hide S3 latency: > a. ZFS ARC/L2ARC for frequently accessed blocks > b. ZeroFS memory cache for metadata and hot dataZeroFS exposes NBD > devices (e.g., /dev/nbd0) that PostgreSQL/ZFS can use like any other bloc= k > device > c. Optional local disk cache > 3. All data is encrypted (ChaCha20-Poly1305) before hitting S3 > 4. Files are split into 128KB chunks for insertion into ZeroFS' LSM-tree > > ## Geo-Distributed PostgreSQL > > Since each region can run its own ZeroFS instance, you can create > geographically distributed PostgreSQL setups. > > Example architectures: > > Architecture 1 > > > PostgreSQL Client > | > | SQL queries > | > +--------------+ > | PG Proxy | > | (HAProxy/ | > | PgBouncer) | > +--------------+ > / \ > / \ > Synchronous Synchronous > Replication Replication > / \ > / \ > +---------------+ +---------------+ > | PostgreSQL 1 | | PostgreSQL 2 | > | (Primary) |=E2=97=84------=E2=96=BA| (Standby) | > +---------------+ +---------------+ > | | > | POSIX filesystem ops | > | | > +---------------+ +---------------+ > | ZFS Pool 1 | | ZFS Pool 2 | > | (3-way mirror)| | (3-way mirror)| > +---------------+ +---------------+ > / | \ / | \ > / | \ / | \ > NBD:10809 NBD:10810 NBD:10811 NBD:10812 NBD:10813 NBD:10814 > | | | | | | > +--------++--------++--------++--------++--------++--------+ > |ZeroFS 1||ZeroFS 2||ZeroFS 3||ZeroFS 4||ZeroFS 5||ZeroFS 6| > +--------++--------++--------++--------++--------++--------+ > | | | | | | > | | | | | | > S3-Region1 S3-Region2 S3-Region3 S3-Region4 S3-Region5 S3-Region6 > (us-east) (eu-west) (ap-south) (us-west) (eu-north) (ap-east) > > Architecture 2: > > PostgreSQL Primary (Region 1) =E2=86=90=E2=86=92 PostgreSQL Standby (Regi= on 2) > \ / > \ / > Same ZFS Pool (NBD) > | > 6 Global ZeroFS > | > S3 Regions > > > The main advantages I see are: > 1. Dramatic cost reduction for large datasets > 2. Simplified geo-distribution > 3. Infinite storage capacity > 4. Built-in encryption and compression > > Looking forward to your feedback and questions! > > Best, > Pierre > > P.S. The full project includes a custom NFS filesystem too. > > > --000000000000d5c387063a31c979 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Sorry, this was meant to go to the whole group:

Very interesting!. Great work. Can you clarify how exactly you'= ;re running postgres in your tests? A specific=C2=A0AWS service? What's= the test infrastructure that sits above the file system?

On Thu, Jul 17, 2025 at 11:59=E2=80=AFPM Pierre Barre <pierre@barre.sh> wrote:
Hi everyone,

I wanted to share a project I've been working on that enables PostgreSQ= L to run on S3 storage while maintaining performance comparable to local NV= Me. The approach uses block-level access rather than trying to map filesyst= em operations to S3 objects.

ZeroFS: https://github.com/Barre/ZeroFS

# The Architecture

ZeroFS provides NBD (Network Block Device) servers that expose S3 storage a= s raw block devices. PostgreSQL runs unmodified on ZFS pools built on these= block devices:

PostgreSQL -> ZFS -> NBD -> ZeroFS -> S3

By providing block-level access and leveraging ZFS's caching capabiliti= es (L2ARC), we can achieve microsecond latencies despite the underlying sto= rage being in S3.

## Performance Results

Here are pgbench results from PostgreSQL running on this setup:

### Read/Write Workload

```
postgres@ubuntu-16gb-fsn1-1:/root$ pgbench -c 50 -j 15 -t 100000 example pgbench (16.9 (Ubuntu 16.9-0ubuntu0.24.04.1))
starting vacuum...end.
transaction type: <builtin: TPC-B (sort of)>
scaling factor: 50
query mode: simple
number of clients: 50
number of threads: 15
maximum number of tries: 1
number of transactions per client: 100000
number of transactions actually processed: 5000000/5000000
number of failed transactions: 0 (0.000%)
latency average =3D 0.943 ms
initial connection time =3D 48.043 ms
tps =3D 53041.006947 (without initial connection time)
```

### Read-Only Workload

```
postgres@ubuntu-16gb-fsn1-1:/root$ pgbench -c 50 -j 15 -t 100000 -S example=
pgbench (16.9 (Ubuntu 16.9-0ubuntu0.24.04.1))
starting vacuum...end.
transaction type: <builtin: select only>
scaling factor: 50
query mode: simple
number of clients: 50
number of threads: 15
maximum number of tries: 1
number of transactions per client: 100000
number of transactions actually processed: 5000000/5000000
number of failed transactions: 0 (0.000%)
latency average =3D 0.121 ms
initial connection time =3D 53.358 ms
tps =3D 413436.248089 (without initial connection time)
```

These numbers are with 50 concurrent clients and the actual data stored in = S3. Hot data is served from ZFS L2ARC and ZeroFS's memory caches, while= cold data comes from S3.

## How It Works

1. ZeroFS exposes NBD devices (e.g., /dev/nbd0) that PostgreSQL/ZFS can use= like any other block device
2. Multiple cache layers hide S3 latency:
=C2=A0 =C2=A0a. ZFS ARC/L2ARC for frequently accessed blocks
=C2=A0 =C2=A0b. ZeroFS memory cache for metadata and hot dataZeroFS exposes= NBD devices (e.g., /dev/nbd0) that PostgreSQL/ZFS can use like any other b= lock device
=C2=A0 =C2=A0c. Optional local disk cache
3. All data is encrypted (ChaCha20-Poly1305) before hitting S3
4. Files are split into 128KB chunks for insertion into ZeroFS' LSM-tre= e

## Geo-Distributed PostgreSQL

Since each region can run its own ZeroFS instance, you can create geographi= cally distributed PostgreSQL setups.

Example architectures:

Architecture 1


=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0PostgreSQL Client
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0| SQL queries
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 +--------------+
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 |=C2=A0 PG Proxy=C2=A0 =C2=A0 |
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 | (HAProxy/=C2=A0 =C2=A0 |
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 |=C2=A0 PgBouncer)=C2=A0 |
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 +--------------+
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0/=C2=A0 =C2=A0 =C2=A0 =C2=A0 \
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 /=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 \
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Synchr= onous=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Synchronous
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Replic= ation=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Replication
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 /=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = \
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0/=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 \
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 +---------------+=C2=A0 = =C2=A0 =C2=A0 =C2=A0 +---------------+
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 | PostgreSQL 1=C2=A0 |=C2= =A0 =C2=A0 =C2=A0 =C2=A0 | PostgreSQL 2=C2=A0 |
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 | (Primary)=C2=A0 =C2=A0 = =C2=A0|=E2=97=84------=E2=96=BA| (Standby)=C2=A0 =C2=A0 =C2=A0|
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 +---------------+=C2=A0 = =C2=A0 =C2=A0 =C2=A0 +---------------+
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 |=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 |
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 |=C2=A0 POSIX filesystem ops=C2=A0 |
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 |=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 |
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 +---------------+=C2=A0 = =C2=A0 =C2=A0 =C2=A0 +---------------+
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 |=C2=A0 =C2=A0ZFS Pool 1= =C2=A0 |=C2=A0 =C2=A0 =C2=A0 =C2=A0 |=C2=A0 =C2=A0ZFS Pool 2=C2=A0 |
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 | (3-way mirror)|=C2=A0 = =C2=A0 =C2=A0 =C2=A0 | (3-way mirror)|
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 +---------------+=C2=A0 = =C2=A0 =C2=A0 =C2=A0 +---------------+
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0/=C2=A0 =C2=A0 =C2= =A0 |=C2=A0 =C2=A0 =C2=A0 \=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 /=C2=A0 =C2= =A0 =C2=A0 |=C2=A0 =C2=A0 =C2=A0 \
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 /=C2=A0 =C2=A0 =C2=A0 =C2= =A0|=C2=A0 =C2=A0 =C2=A0 =C2=A0\=C2=A0 =C2=A0 =C2=A0 =C2=A0 /=C2=A0 =C2=A0 = =C2=A0 =C2=A0|=C2=A0 =C2=A0 =C2=A0 =C2=A0\
=C2=A0 =C2=A0 =C2=A0 =C2=A0 NBD:10809 NBD:10810 NBD:10811=C2=A0 NBD:10812 N= BD:10813 NBD:10814
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|=C2=A0 =C2=A0 =C2=A0 =C2= =A0 |=C2=A0 =C2=A0 =C2=A0 =C2=A0 |=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= |=C2=A0 =C2=A0 =C2=A0 =C2=A0 |=C2=A0 =C2=A0 =C2=A0 =C2=A0 |
=C2=A0 =C2=A0 =C2=A0 =C2=A0 +--------++--------++--------++--------++------= --++--------+
=C2=A0 =C2=A0 =C2=A0 =C2=A0 |ZeroFS 1||ZeroFS 2||ZeroFS 3||ZeroFS 4||ZeroFS= 5||ZeroFS 6|
=C2=A0 =C2=A0 =C2=A0 =C2=A0 +--------++--------++--------++--------++------= --++--------+
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|=C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0|=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0|=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0|
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|=C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0|=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0|=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0|
=C2=A0 =C2=A0 =C2=A0 =C2=A0 S3-Region1 S3-Region2 S3-Region3 S3-Region4 S3-= Region5 S3-Region6
=C2=A0 =C2=A0 =C2=A0 =C2=A0 (us-east) (eu-west) (ap-south) (us-west) (eu-no= rth) (ap-east)

Architecture 2:

PostgreSQL Primary (Region 1) =E2=86=90=E2=86=92 PostgreSQL Standby (Region= 2)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 \=C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 /
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0\=C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 /
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Same ZFS Poo= l (NBD)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0|
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 6 Global Zer= oFS
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0|
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 S3 Regions


The main advantages I see are:
1. Dramatic cost reduction for large datasets
2. Simplified geo-distribution
3. Infinite storage capacity
4. Built-in encryption and compression

Looking forward to your feedback and questions!

Best,
Pierre

P.S. The full project includes a custom NFS filesystem too.


--000000000000d5c387063a31c979--