From: Rod Taylor <pg@rbt.ca>
To: Suchandra Thapa <s-thapa-11@alumni.uchicago.edu>
Cc: Postgresql Performance <pgsql-performance@postgresql.org>
Subject: Re: performance optimzations
Date: Wed, 12 Nov 2003 23:35:33 -0500
Message-ID: <1068698132.31569.46.camel@jester> (raw)
In-Reply-To: <1068663070.5193.21.camel@hepcat>
References: <1068658430.1071.42.camel@hepcat>
<1068661380.30452.17.camel@jester>
<1068663070.5193.21.camel@hepcat>
> > How much in total storage? If you have (or will have) > ~6 disks, go
> > for RAID 5 otherwise 10 is probably appropriate.
>
> I'm not sure but I believe there are about 6-8 10K scsi drives on the
> system. There is quite a bit of storage to spare currently so I think
I see.. With 8 drives, you'll probably want to go with RAID 5. It grows
beyond that point fairly well with a decent controller card. Be sure to
have some battery backed write cache on the raid card (128MB goes a long
way).
> > > The database gets fairly heavy activity (the system handles about 500MB
> > > of incoming and about 750MB of outgoing emails daily). I have a fairly
> No, it's more like hotmail. Some users may keep mail for a longer term
> but a lot of the mail probably gets deleted fairly quickly. The
> database load will be mixed with a insertions due to deliveries, queries
> by the webmail system, and deletions from pop and webmail.
You might consider having the mailserver gzip the emails prior to
injection into the database (turn off compression in PostgreSQL) and
decompress the data on the webserver for display to the client. Now you
have about 7 times the number of emails in memory.
It's easier to toss a webserver at the problem than make the database
bigger in size. Take the savings in CPU on the DB and add it to ram.
1200MB of compressed mail is about 200MB? Assume email descriptive
material (subject, from, etc.), account structure, indexes... so about
400MB for one days worth of information?
You may want to consider keeping the compressed email in a separate
table than the information describing it. It would mean descriptive
information is more likely to be in RAM, where the body probably doesn't
matter as much (you view them 1 at a time, subjects tend to be listed
all at once).
Most clients will be interested in say the last 7 days worth of data?
Great.. Start out with 4GB ram on a good Dual CPU -- Opterons seem to
work quite well -- and make sure the motherboard can hold double that in
memory for an upgrade sometime next year when you've become popular.
I firmly believe lots of RAM is the answer to most IO issues until you
start getting into large sets of active data (>50GB). 64GB ram is fairly
cheap compared to ongoing maintenance of the 30+ drive system required
to get decent throughput.
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: pg@rbt.ca, s-thapa-11@alumni.uchicago.edu
Subject: Re: performance optimzations
In-Reply-To: <1068698132.31569.46.camel@jester>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox