Date: Fri, 2 Dec 2005 02:49:53 -0500 (EST)
From: Qingqing Zhou <zhouqq@cs.toronto.edu>
To: David Lang <dlang@invendra.net>
Cc: pgsql-performance@postgresql.org
Subject: Re: filesystem performance with lots of files
In-Reply-To: <Pine.LNX.4.62.0512012258430.2807@qnivq.ynat.uz>
Message-ID: <Pine.LNX.4.58.0512020231470.3867@eon.cs>
References: <Pine.LNX.4.62.0512010616200.2807@qnivq.ynat.uz>
	<dmndri$c39$1@news.hub.org>
	<Pine.LNX.4.62.0512012258430.2807@qnivq.ynat.uz>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII


On Fri, 2 Dec 2005, David Lang wrote:
>
> I don't have all the numbers readily available (and I didn't do all the
> tests on every filesystem), but I found that even with only 1000
> files/directory ext3 had some problems, and if you enabled dir_hash some
> functions would speed up, but writing lots of files would just collapse
> (that was the 80 min run)
>

Interesting. I would suggest test small number but bigger file would be
better if the target is for database performance comparison. By small
number, I mean 10^2 - 10^3; By bigger, I mean file size from 8k to 1G
(PostgreSQL data file is at most this size under normal installation).

Let's take TPCC as an example, if we get a TPCC database of 500 files,
each one is at most 1G (PostgreSQL has this feature/limit in ordinary
installation), then this will give us a 500G database, which is big enough
for your current configuration.

Regards,
Qingqing