From: Qingqing Zhou <zhouqq@cs.toronto.edu>
To: David Lang <dlang@invendra.net>
Cc: pgsql-performance@postgresql.org
Subject: Re: filesystem performance with lots of files
Date: Fri, 2 Dec 2005 02:49:53 -0500 (EST)
Message-ID: <Pine.LNX.4.58.0512020231470.3867@eon.cs> (raw)
In-Reply-To: <Pine.LNX.4.62.0512012258430.2807@qnivq.ynat.uz>
References: <Pine.LNX.4.62.0512010616200.2807@qnivq.ynat.uz>
<dmndri$c39$1@news.hub.org>
<Pine.LNX.4.62.0512012258430.2807@qnivq.ynat.uz>
On Fri, 2 Dec 2005, David Lang wrote:
>
> I don't have all the numbers readily available (and I didn't do all the
> tests on every filesystem), but I found that even with only 1000
> files/directory ext3 had some problems, and if you enabled dir_hash some
> functions would speed up, but writing lots of files would just collapse
> (that was the 80 min run)
>
Interesting. I would suggest test small number but bigger file would be
better if the target is for database performance comparison. By small
number, I mean 10^2 - 10^3; By bigger, I mean file size from 8k to 1G
(PostgreSQL data file is at most this size under normal installation).
Let's take TPCC as an example, if we get a TPCC database of 500 files,
each one is at most 1G (PostgreSQL has this feature/limit in ordinary
installation), then this will give us a 500G database, which is big enough
for your current configuration.
Regards,
Qingqing
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: zhouqq@cs.toronto.edu, dlang@invendra.net
Subject: Re: filesystem performance with lots of files
In-Reply-To: <Pine.LNX.4.58.0512020231470.3867@eon.cs>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox