public inbox for [email protected]  

help / color / mirror / Atom feed

From: Pierre Barre <[email protected]>
To: Laurenz Albe <[email protected]>
To: [email protected]
Subject: Re: PostgreSQL on S3-backed Block Storage with Near-Local Performance
Date: Fri, 18 Jul 2025 11:59:18 +0200
Message-ID: <[email protected]> (raw)
In-Reply-To: <[email protected]>
References: <[email protected]>
	<[email protected]>

Hi Laurenz,

> I think the biggest hurdle you will have to overcome is to
> convince notoriously paranoid DBAs that this tall stack
> provides reliable service, honors fsync() etc.

Indeed, but that doesn't have to be "sudden." I think we need to gain confidence in the whole system gradually by starting with throwable workloads (e.g., persistent volumes in CI), then moving to data we can afford to lose, then backups, and finally to production data.

>> P.S. The full project includes a custom NFS filesystem too.
>
> "NFS" is a key word that does not inspire confidence in
> PostgreSQL circles...

I've had my fair share of major annoyances with NFS too!

I think bad experiences with NFS are basically due to the fact that when the hardware is bad, the NFS server implementation is bad, and the kernel treats it mostly like a "local" filesystem (in terms of failure behavior). 

So when it doesn't work well, everything goes down.

But the protocols themselves are not inherently bad—they are actually quite elegant. NFSv3 is just what you need to reach (very close to) POSIX compliance. The NFS server implementation in ZeroFS passes all 8,662 tests in https://github.com/Barre/pjdfstest_nfs.

https://github.com/Barre/ZeroFS/actions/runs/16367571315/job/46248240251#step:11:9376

For database workloads specifically, users will probably prefer running something like ZFS on top of the NBD server rather than using NFS directly.

Best,
Pierre

On Fri, Jul 18, 2025, at 06:40, Laurenz Albe wrote:
> On Fri, 2025-07-18 at 00:57 +0200, Pierre Barre wrote:
>> Looking forward to your feedback and questions!
>
> I think the biggest hurdle you will have to overcome is to
> convince notoriously paranoid DBAs that this tall stack
> provides reliable service, honors fsync() etc.
>
> Performance is great, but it is not everything.  If things
> perform surprisingly well, people become suspicious.
>
>> P.S. The full project includes a custom NFS filesystem too.
>
> "NFS" is a key word that does not inspire confidence in
> PostgreSQL circles...
>
> Yours,
> Laurenz Albe

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected], [email protected]
  Subject: Re: PostgreSQL on S3-backed Block Storage with Near-Local Performance
  In-Reply-To: <[email protected]>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox