Date: Mon, 02 May 2005 01:22:49 +0200
To: apoc9009@yahoo.de, pgsql-performance@postgresql.org
Subject: Re: BLOB's bypassing the OS Filesystem for better Image
References: <426DFA46.8090402@yahoo.de> <426E0B35.9080509@yahoo.de>
	<426E045C.7010601@archonet.com>
From: PFC <lists@boutiquenumerique.com>
Organization: =?iso-8859-15?Q?La_Boutique_Num=E9rique?=
Content-Type: text/plain; format=flowed; delsp=yes; charset=iso-8859-15
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Message-ID: <op.sp4dwbd2th1vuj@localhost>
In-Reply-To: <426E0B35.9080509@yahoo.de>
User-Agent: Opera M2(BETA2)/8.0 (Linux, build 987)


	My laptop reads an entire compiled linux kernel (23000 files totalling  
250 MBytes) in about 1.5 seconds if they're in cache. It's about 15.000  
files/second. You think it's slow ? If you want to read them in random  
order, you'll probably use something else than a laptop drive, but you get  
the idea.

	Filesystem is reiser4.

	If you use ext2, you'll have a problem with many files in the same  
directory because I believe it uses a linear search, hence time  
proportional to the number of files (ouch). I once tried to put a million  
1-kbyte files in a directory ; it was with reiserfs 3, and it didn't seem  
to feel anything close to molested. I believe it took some 10 minutes, but  
it was two years ago so I don't remember very well. NTFS took a day, that  
I do remember ! By curiosity I tried to stuff 1 million 1KB files in a  
directory on my laptop right now, It took a bit less than two minutes.

On Tue, 26 Apr 2005 11:34:45 +0200, apoc9009@yahoo.de <apoc9009@yahoo.de>  
wrote:

>
>> Which filesystems? I know ext2 used to have issues with many-thousands  
>> of files in one directory, but that was a directory scanning issue  
>> rather than file reading.
>
>  From my Point of view i think it is better to let one Process do the  
> operation to an Postgres Cluster Filestructure as
> if i bypass it with a second process.
>
> For example:
> A User loads up some JPEG Images over HTTP.
>
> a) (Filesystem)
> On Filesystem it would be written in a File with a random generated  
> Filename (timestamp or what ever)
> (the Directory Expands and over a Million Fileobjects with will be  
> archived, written, replaced, e.t.c)
>
> b) (Database)
> The JPEG Image Information will be stored into a BLOB as Part of a  
> special Table, where is linked
> wit the custid of the primary Usertable.
>
>  From my Point of view is any outside Process (must be created, forked,  
> Memory allocated, e.t.c)
> a bad choice. I think it is generall better to Support the Postmaster in  
> all Ways and do some
> Hardware RAID Configurations.
>
>>> My Question:
>>> Can i speedup my Webapplication if i store my JPEG Images with small
>>> sizes inside my PostgreSQL Database (on verry large Databasis over 1  
>>> GByte
>>> and above without Images at this time!)
>>
>>
>> No. Otherwise the filesystem people would build their filesystems on  
>> top of PostgreSQL not the other way around. Of course, if you want  
>> image updates to be part of a database transaction, then it might be  
>> worth storing them in the database.
>
> Hmm, ORACLE is going the other Way. All File Objects can be stored into  
> the Database if the DB
> has the IFS Option (Database Filesystem and Fileserver insinde the  
> Database).
>
>
>>
>>> I hope some Peoples can give me a Tip or Hint where in can
>>> some usefull Information about it!
>>
>> Look into having a separate server (process or actual hardware) to  
>> handle requests for static text and images. Keep the Java server for  
>> actually processing
>
>
> Thanks
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 6: Have you searched our list archives?
>
>                http://archives.postgresql.org
>