Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1uf4K5-001jPh-AM for pgsql-general@arkaria.postgresql.org; Thu, 24 Jul 2025 22:21:58 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1uf4K4-00Bwre-BM for pgsql-general@arkaria.postgresql.org; Thu, 24 Jul 2025 22:21:56 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1uf4K3-00BwrW-P7 for pgsql-general@lists.postgresql.org; Thu, 24 Jul 2025 22:21:56 +0000 Received: from mail-ej1-x632.google.com ([2a00:1450:4864:20::632]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.96) (envelope-from ) id 1uf4K2-000ca4-0J for pgsql-general@lists.postgresql.org; Thu, 24 Jul 2025 22:21:55 +0000 Received: by mail-ej1-x632.google.com with SMTP id a640c23a62f3a-af0dc229478so221562366b.1 for ; Thu, 24 Jul 2025 15:21:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1753395711; x=1754000511; darn=lists.postgresql.org; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :from:to:cc:subject:date:message-id:reply-to; bh=kYuZ0Culm307z2cB+WlNJfN0Qy882LyFdRQFXWFciw4=; b=K87sohIcGmvy1xTpdU1lYqe3KIzd/6qNvqaaOn3IJ/BR1VOVQSLQ71lEA5XGs34HXc 1zduN59p2lFhLm4Tk1GPXmjJwwGFCmM7PNsyX+7660Ien+SkeZ7G0oRnsOzcJJhyRseP yRLwl6Ot7x5UhZUDx3tsrng4ktTmESAvfTXRUzlUzjAeZGWAlISb8fI9V5ULBtMyMyZW JvX0WREt2LYl4jOwD8uI/WP1Y2toaFOV/GBtzv7f1sgXQds+M5lL2R6LFXQGgVpMiseZ 6nzsH+i3LXxwTTFfsSlAFVAWGZOgFMyqOI3s+9V31Gfy1s1Rnv2eTqkK5kLWt8f9nn35 LE5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753395711; x=1754000511; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=kYuZ0Culm307z2cB+WlNJfN0Qy882LyFdRQFXWFciw4=; b=iCOpzu36sh8nXorppXakKhxtPOO7jcloQkK3DGAlnpIS/nU8VnfWrnCOMphyMshuiT HoZPPRcR5ou/DH4gHnSYcJyZpTEAP4ttxhsaBbRViU34o+Whuana27jNkUDTQC9mgPJL 3AgYHiQPx0PGXbRZP2Ng3lXySncvuq+c2SY7kHxhcMrSWql3+jeNoh49bAyyDHLgoQTg LuXwm/v9IR73MjF8rniZ0fO+QpB9vdZcLFnQ3KNBCztwaK5kRua/gF0qrigBszZIHkaT kaZGKcg5xGfuUqbXtUlkK2WC+O5FxKtw9BWmKL9rsLPigWghEUDKDULNfh7QqPy81hjo jaAw== X-Gm-Message-State: AOJu0YzrpfLymBNeFytRn32APIftotKRfLPgMOKTu30o1FFDOb4AWKO8 G4Xj/NpL5IVtr8WNiZpuWY/jZjNxrFyhIN5ZDZM0ls5bPyTfKdShbEgVjCp1Nxr5Wsz9gGIf7OQ vSPnCSixDDuh4ztgKhCATFhZumZKUk5VVpw== X-Gm-Gg: ASbGncu7VkoFkuTWAEAdOzt17y9ITqN7T8JmmTePiR/z2aD0WPx+zHgwZHKpb3SduvN IVlXuOkxBEbd6BnrkVwgquXwhJ+5GYqOVWJwPLn45jufIATt2Uc0MT134c+Vj+0aMYG3GXUckuF PEz/P3QmKB8V/d2ggNe4BN22rs+NrjngK2fCj+3AMVzPa6UOqZ/muJqcYTLHmFzA2nwrl5SRiEg SJ19eMN X-Google-Smtp-Source: AGHT+IG0ZgG02vqdHM2tIpuJHrFxSRSAE5Ymh9s1+zm88NiRXiuu9Il8RTQsRRjAaYCRDS42WDDjhEZiZWYS6bqtskw= X-Received: by 2002:a17:907:3fa0:b0:ad8:a935:b8e8 with SMTP id a640c23a62f3a-af2f64bf8f4mr868175966b.5.1753395710920; Thu, 24 Jul 2025 15:21:50 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Marco Torres Date: Thu, 24 Jul 2025 16:21:39 -0600 X-Gm-Features: Ac12FXyxAFP76-8hQPpjw2HkWLgYZLw7w8caiaaKvjDc4_rwMwkGISeJ3Qq-qIA Message-ID: Subject: Re: PostgreSQL on S3-backed Block Storage with Near-Local Performance To: pgsql-general@lists.postgresql.org Content-Type: multipart/alternative; boundary="000000000000800bf2063ab4409f" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --000000000000800bf2063ab4409f Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable My humble take on this project: well done! You are opening the doors to work on a much-needed endeavor, decouple compute from storage, and potentially elaborate on other projects for an active/active cluster! I applaud you. On Thu, Jul 17, 2025, 4:59=E2=80=AFPM Pierre Barre wrote: > Hi everyone, > > I wanted to share a project I've been working on that enables PostgreSQL > to run on S3 storage while maintaining performance comparable to local > NVMe. The approach uses block-level access rather than trying to map > filesystem operations to S3 objects. > > ZeroFS: https://github.com/Barre/ZeroFS > > # The Architecture > > ZeroFS provides NBD (Network Block Device) servers that expose S3 storage > as raw block devices. PostgreSQL runs unmodified on ZFS pools built on > these block devices: > > PostgreSQL -> ZFS -> NBD -> ZeroFS -> S3 > > By providing block-level access and leveraging ZFS's caching capabilities > (L2ARC), we can achieve microsecond latencies despite the underlying > storage being in S3. > > ## Performance Results > > Here are pgbench results from PostgreSQL running on this setup: > > ### Read/Write Workload > > ``` > postgres@ubuntu-16gb-fsn1-1:/root$ pgbench -c 50 -j 15 -t 100000 example > pgbench (16.9 (Ubuntu 16.9-0ubuntu0.24.04.1)) > starting vacuum...end. > transaction type: > scaling factor: 50 > query mode: simple > number of clients: 50 > number of threads: 15 > maximum number of tries: 1 > number of transactions per client: 100000 > number of transactions actually processed: 5000000/5000000 > number of failed transactions: 0 (0.000%) > latency average =3D 0.943 ms > initial connection time =3D 48.043 ms > tps =3D 53041.006947 (without initial connection time) > ``` > > ### Read-Only Workload > > ``` > postgres@ubuntu-16gb-fsn1-1:/root$ pgbench -c 50 -j 15 -t 100000 -S > example > pgbench (16.9 (Ubuntu 16.9-0ubuntu0.24.04.1)) > starting vacuum...end. > transaction type: > scaling factor: 50 > query mode: simple > number of clients: 50 > number of threads: 15 > maximum number of tries: 1 > number of transactions per client: 100000 > number of transactions actually processed: 5000000/5000000 > number of failed transactions: 0 (0.000%) > latency average =3D 0.121 ms > initial connection time =3D 53.358 ms > tps =3D 413436.248089 (without initial connection time) > ``` > > These numbers are with 50 concurrent clients and the actual data stored i= n > S3. Hot data is served from ZFS L2ARC and ZeroFS's memory caches, while > cold data comes from S3. > > ## How It Works > > 1. ZeroFS exposes NBD devices (e.g., /dev/nbd0) that PostgreSQL/ZFS can > use like any other block device > 2. Multiple cache layers hide S3 latency: > a. ZFS ARC/L2ARC for frequently accessed blocks > b. ZeroFS memory cache for metadata and hot dataZeroFS exposes NBD > devices (e.g., /dev/nbd0) that PostgreSQL/ZFS can use like any other bloc= k > device > c. Optional local disk cache > 3. All data is encrypted (ChaCha20-Poly1305) before hitting S3 > 4. Files are split into 128KB chunks for insertion into ZeroFS' LSM-tree > > ## Geo-Distributed PostgreSQL > > Since each region can run its own ZeroFS instance, you can create > geographically distributed PostgreSQL setups. > > Example architectures: > > Architecture 1 > > > PostgreSQL Client > | > | SQL queries > | > +--------------+ > | PG Proxy | > | (HAProxy/ | > | PgBouncer) | > +--------------+ > / \ > / \ > Synchronous Synchronous > Replication Replication > / \ > / \ > +---------------+ +---------------+ > | PostgreSQL 1 | | PostgreSQL 2 | > | (Primary) |=E2=97=84------=E2=96=BA| (Standby) | > +---------------+ +---------------+ > | | > | POSIX filesystem ops | > | | > +---------------+ +---------------+ > | ZFS Pool 1 | | ZFS Pool 2 | > | (3-way mirror)| | (3-way mirror)| > +---------------+ +---------------+ > / | \ / | \ > / | \ / | \ > NBD:10809 NBD:10810 NBD:10811 NBD:10812 NBD:10813 NBD:10814 > | | | | | | > +--------++--------++--------++--------++--------++--------+ > |ZeroFS 1||ZeroFS 2||ZeroFS 3||ZeroFS 4||ZeroFS 5||ZeroFS 6| > +--------++--------++--------++--------++--------++--------+ > | | | | | | > | | | | | | > S3-Region1 S3-Region2 S3-Region3 S3-Region4 S3-Region5 S3-Region6 > (us-east) (eu-west) (ap-south) (us-west) (eu-north) (ap-east) > > Architecture 2: > > PostgreSQL Primary (Region 1) =E2=86=90=E2=86=92 PostgreSQL Standby (Regi= on 2) > \ / > \ / > Same ZFS Pool (NBD) > | > 6 Global ZeroFS > | > S3 Regions > > > The main advantages I see are: > 1. Dramatic cost reduction for large datasets > 2. Simplified geo-distribution > 3. Infinite storage capacity > 4. Built-in encryption and compression > > Looking forward to your feedback and questions! > > Best, > Pierre > > P.S. The full project includes a custom NFS filesystem too. > > > --000000000000800bf2063ab4409f Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
My humble take on this p= roject: well done! You are opening the doors to work on a much-needed endea= vor, decouple compute from storage, and potentially elaborate on other proj= ects for an active/active cluster! I applaud you.

On Thu, Jul 17, 2025, 4:59=E2=80=AFPM Pierre Barre <pierre@barre.sh> wrote:
Hi everyone,

I wanted to share a project I've been working on that enables PostgreSQ= L to run on S3 storage while maintaining performance comparable to local NV= Me. The approach uses block-level access rather than trying to map filesyst= em operations to S3 objects.

ZeroFS: https://github.com/Barre/ZeroFS

# The Architecture

ZeroFS provides NBD (Network Block Device) servers that expose S3 storage a= s raw block devices. PostgreSQL runs unmodified on ZFS pools built on these= block devices:

PostgreSQL -> ZFS -> NBD -> ZeroFS -> S3

By providing block-level access and leveraging ZFS's caching capabiliti= es (L2ARC), we can achieve microsecond latencies despite the underlying sto= rage being in S3.

## Performance Results

Here are pgbench results from PostgreSQL running on this setup:

### Read/Write Workload

```
postgres@ubuntu-16gb-fsn1-1:/root$ pgbench -c 50 -j 15 -t 100000 example pgbench (16.9 (Ubuntu 16.9-0ubuntu0.24.04.1))
starting vacuum...end.
transaction type: <builtin: TPC-B (sort of)>
scaling factor: 50
query mode: simple
number of clients: 50
number of threads: 15
maximum number of tries: 1
number of transactions per client: 100000
number of transactions actually processed: 5000000/5000000
number of failed transactions: 0 (0.000%)
latency average =3D 0.943 ms
initial connection time =3D 48.043 ms
tps =3D 53041.006947 (without initial connection time)
```

### Read-Only Workload

```
postgres@ubuntu-16gb-fsn1-1:/root$ pgbench -c 50 -j 15 -t 100000 -S example=
pgbench (16.9 (Ubuntu 16.9-0ubuntu0.24.04.1))
starting vacuum...end.
transaction type: <builtin: select only>
scaling factor: 50
query mode: simple
number of clients: 50
number of threads: 15
maximum number of tries: 1
number of transactions per client: 100000
number of transactions actually processed: 5000000/5000000
number of failed transactions: 0 (0.000%)
latency average =3D 0.121 ms
initial connection time =3D 53.358 ms
tps =3D 413436.248089 (without initial connection time)
```

These numbers are with 50 concurrent clients and the actual data stored in = S3. Hot data is served from ZFS L2ARC and ZeroFS's memory caches, while= cold data comes from S3.

## How It Works

1. ZeroFS exposes NBD devices (e.g., /dev/nbd0) that PostgreSQL/ZFS can use= like any other block device
2. Multiple cache layers hide S3 latency:
=C2=A0 =C2=A0a. ZFS ARC/L2ARC for frequently accessed blocks
=C2=A0 =C2=A0b. ZeroFS memory cache for metadata and hot dataZeroFS exposes= NBD devices (e.g., /dev/nbd0) that PostgreSQL/ZFS can use like any other b= lock device
=C2=A0 =C2=A0c. Optional local disk cache
3. All data is encrypted (ChaCha20-Poly1305) before hitting S3
4. Files are split into 128KB chunks for insertion into ZeroFS' LSM-tre= e

## Geo-Distributed PostgreSQL

Since each region can run its own ZeroFS instance, you can create geographi= cally distributed PostgreSQL setups.

Example architectures:

Architecture 1


=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0PostgreSQL Client
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0| SQL queries
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 +--------------+
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 |=C2=A0 PG Proxy=C2=A0 =C2=A0 |
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 | (HAProxy/=C2=A0 =C2=A0 |
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 |=C2=A0 PgBouncer)=C2=A0 |
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 +--------------+
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0/=C2=A0 =C2=A0 =C2=A0 =C2=A0 \
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 /=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 \
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Synchr= onous=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Synchronous
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Replic= ation=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Replication
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 /=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = \
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0/=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 \
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 +---------------+=C2=A0 = =C2=A0 =C2=A0 =C2=A0 +---------------+
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 | PostgreSQL 1=C2=A0 |=C2= =A0 =C2=A0 =C2=A0 =C2=A0 | PostgreSQL 2=C2=A0 |
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 | (Primary)=C2=A0 =C2=A0 = =C2=A0|=E2=97=84------=E2=96=BA| (Standby)=C2=A0 =C2=A0 =C2=A0|
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 +---------------+=C2=A0 = =C2=A0 =C2=A0 =C2=A0 +---------------+
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 |=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 |
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 |=C2=A0 POSIX filesystem ops=C2=A0 |
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 |=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 |
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 +---------------+=C2=A0 = =C2=A0 =C2=A0 =C2=A0 +---------------+
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 |=C2=A0 =C2=A0ZFS Pool 1= =C2=A0 |=C2=A0 =C2=A0 =C2=A0 =C2=A0 |=C2=A0 =C2=A0ZFS Pool 2=C2=A0 |
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 | (3-way mirror)|=C2=A0 = =C2=A0 =C2=A0 =C2=A0 | (3-way mirror)|
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 +---------------+=C2=A0 = =C2=A0 =C2=A0 =C2=A0 +---------------+
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0/=C2=A0 =C2=A0 =C2= =A0 |=C2=A0 =C2=A0 =C2=A0 \=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 /=C2=A0 =C2= =A0 =C2=A0 |=C2=A0 =C2=A0 =C2=A0 \
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 /=C2=A0 =C2=A0 =C2=A0 =C2= =A0|=C2=A0 =C2=A0 =C2=A0 =C2=A0\=C2=A0 =C2=A0 =C2=A0 =C2=A0 /=C2=A0 =C2=A0 = =C2=A0 =C2=A0|=C2=A0 =C2=A0 =C2=A0 =C2=A0\
=C2=A0 =C2=A0 =C2=A0 =C2=A0 NBD:10809 NBD:10810 NBD:10811=C2=A0 NBD:10812 N= BD:10813 NBD:10814
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|=C2=A0 =C2=A0 =C2=A0 =C2= =A0 |=C2=A0 =C2=A0 =C2=A0 =C2=A0 |=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= |=C2=A0 =C2=A0 =C2=A0 =C2=A0 |=C2=A0 =C2=A0 =C2=A0 =C2=A0 |
=C2=A0 =C2=A0 =C2=A0 =C2=A0 +--------++--------++--------++--------++------= --++--------+
=C2=A0 =C2=A0 =C2=A0 =C2=A0 |ZeroFS 1||ZeroFS 2||ZeroFS 3||ZeroFS 4||ZeroFS= 5||ZeroFS 6|
=C2=A0 =C2=A0 =C2=A0 =C2=A0 +--------++--------++--------++--------++------= --++--------+
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|=C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0|=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0|=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0|
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|=C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0|=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0|=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0|
=C2=A0 =C2=A0 =C2=A0 =C2=A0 S3-Region1 S3-Region2 S3-Region3 S3-Region4 S3-= Region5 S3-Region6
=C2=A0 =C2=A0 =C2=A0 =C2=A0 (us-east) (eu-west) (ap-south) (us-west) (eu-no= rth) (ap-east)

Architecture 2:

PostgreSQL Primary (Region 1) =E2=86=90=E2=86=92 PostgreSQL Standby (Region= 2)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 \=C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 /
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0\=C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 /
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Same ZFS Poo= l (NBD)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0|
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 6 Global Zer= oFS
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0|
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 S3 Regions


The main advantages I see are:
1. Dramatic cost reduction for large datasets
2. Simplified geo-distribution
3. Infinite storage capacity
4. Built-in encryption and compression

Looking forward to your feedback and questions!

Best,
Pierre

P.S. The full project includes a custom NFS filesystem too.


--000000000000800bf2063ab4409f--