Message-ID: <68dd1b79.170a0220.3c4175.198f@mx.google.com>
Sender: Michael Banck <mbanck@gmail.com>
Date: Wed, 1 Oct 2025 14:15:52 +0200
From: Michael Banck <mbanck@gmx.net>
To: Jakub Wartak <jakub.wartak@enterprisedb.com>
Cc: Aleksander Alekseev <aleksander@tigerdata.com>,
	pgsql-hackers@lists.postgresql.org, Tom Lane <tgl@sss.pgh.pa.us>,
	Frits Hoogland <frits.hoogland@gmail.com>
Subject: Re: The ability of postgres to determine loss of files of the main
 fork
References: <013D63E2-5D75-492E-85FF-1D5CC0148C82@gmail.com>
 <CAJ7c6TMYvYBQpgv8k+4QnajqPcroOzrj7d3snBBTKX4tHQfVYw@mail.gmail.com>
 <499686.1759250489@sss.pgh.pa.us>
 <68dcd1f2.df0a0220.3300c0.f7af@mx.google.com>
 <CAKZiRmwHKY=KTjBEL3S2cVQpo1OjHyky4BdgJm4Hkv1-ig9PfQ@mail.gmail.com>
 <CAJ7c6TMCyhX4Z7FrS=ORCcKS1F_iZzfUuQM3shx64WKynoxjsA@mail.gmail.com>
 <CAKZiRmy0CK3m0-raCdTDELg0JjY7qAqzEN9P5n4N4wGw6ys4tw@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: 
 <CAKZiRmy0CK3m0-raCdTDELg0JjY7qAqzEN9P5n4N4wGw6ys4tw@mail.gmail.com>
User-Agent: Mutt/1.10.1 (2018-07-13)
Archived-At: 
 <https://www.postgresql.org/message-id/68dd1b79.170a0220.3c4175.198f%40mx.google.com>
Precedence: bulk

Hi,

On Wed, Oct 01, 2025 at 02:05:53PM +0200, Jakub Wartak wrote:
> On Wed, Oct 1, 2025 at 1:46 PM Aleksander Alekseev
> <aleksander@tigerdata.com> wrote:
> > > IMHO all files should be opened at least on startup to check
> > > integrity,

I would say s/startup/crash recovery/, if any.

> > That might be a lot of files to open.
> 
> I was afraid of that, but let's say modern high-end is 200TB big DB,
> that's like 200*1024 1GB files, but I'm getting such time(1) timings
> for 204k files on ext4:
> 
> $ time ./createfiles                      # real    0m2.157s, it's
> open(O_CREAT)+close()
> $ time ls -l many_files_dir/ > /dev/null # real    0m0.734s
> $ time ./openfiles                          # real    0m0.297s , for
> already existing ones (hot)
> $ time ./openfiles                          # real    0m1.456s , for
> already existing ones (cold, echo 3 > drop_caches sysctl)
>
> Not bad in my book as a one time activity. It could pose a problem
> potentially with some high latency open() calls, maybe NFS or
> something remote I guess.

Yeah, did you try on SAN as well? I am doubtful that will be performant.


Michael