pg.ddx.io pgsql-performance@postgresql.org mailing list archive
help / color / mirror / Atom feedHow to handle a large DB and simultaneous accesses?
2+ messages / 2 participants
[nested] [flat]
* How to handle a large DB and simultaneous accesses?
@ 2006-01-10 14:41 Charles A. Landemaine <landemaine@gmail.com>
0 siblings, 1 reply; 2+ messages in thread
From: Charles A. Landemaine @ 2006-01-10 14:41 UTC (permalink / raw)
To: pgsql-performance
Hello,
I have to develop a companies search engine (looks like the Yellow
pages). We're using PostgreSQL at the company, and the initial DB is
2GB large, as it
has companies from the entire world, with a fair amount of information.
What reading do you suggest so that we can develop the search engine
core, in order that the result pages show up instantly, no matter the
heavy load and
the DB size. The DB is 2GB but should grow to up to 10GB in 2 years,
and there should be 250,000 unique visitors per month by the end of
the year.
Are there special techniques? Maybe there's a way to sort of cache
search results? We're using PHP5 + phpAccelerator.
Thanks,
--
Charles A. Landemaine.
^ permalink raw reply [nested|flat] 2+ messages in thread
* Re: How to handle a large DB and simultaneous accesses?
@ 2006-01-11 03:50 David Lang <dlang@invendra.net>
parent: Charles A. Landemaine <landemaine@gmail.com>
0 siblings, 0 replies; 2+ messages in thread
From: David Lang @ 2006-01-11 03:50 UTC (permalink / raw)
To: Charles A. Landemaine <landemaine@gmail.com>; +Cc: pgsql-performance
On Tue, 10 Jan 2006, Charles A. Landemaine wrote:
> Hello,
>
> I have to develop a companies search engine (looks like the Yellow
> pages). We're using PostgreSQL at the company, and the initial DB is
> 2GB large, as it
> has companies from the entire world, with a fair amount of information.
>
> What reading do you suggest so that we can develop the search engine
> core, in order that the result pages show up instantly, no matter the
> heavy load and
> the DB size. The DB is 2GB but should grow to up to 10GB in 2 years,
> and there should be 250,000 unique visitors per month by the end of
> the year.
>
> Are there special techniques? Maybe there's a way to sort of cache
> search results? We're using PHP5 + phpAccelerator.
> Thanks,
frankly that is a small enough chunk of data compared to available memory
sizes that I think your best bet is to plan to have enough ram that you
only do disk I/O to write and on boot.
a dual socket Opteron system can hold 16G with 2G memory modules (32G as
4G modules become readily available over the next couple of years). this
should be enough to keep your data and indexes in ram at all times. if you
find that other system processes push the data out of ram consider loading
the data from disk to a ramfs filesystem, just make sure you don't update
the ram-only copy (or if you do that you have replication setup to
replicate from the ram copy to a copy on real disks somewhere). depending
on your load you could go with single core or dual core chips (and the
cpu's are a small enough cost compared to this much ram that you may as
well go with the dual core cpu's)
now even with your data in ram you can slow down if your queries, indexes,
and other settings are wrong, but if performance is important you should
be able to essentially eliminate disks for databases of this size.
David Lang
^ permalink raw reply [nested|flat] 2+ messages in thread
end of thread, other threads:[~2006-01-11 03:50 UTC | newest]
Thread overview: 2+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2006-01-10 14:41 How to handle a large DB and simultaneous accesses? Charles A. Landemaine <landemaine@gmail.com>
2006-01-11 03:50 ` David Lang <dlang@invendra.net>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox