From: Mark Stosberg <mark@summersault.com>
To: pgsql-performance@postgresql.org
Subject: Re: getting the most of out multi-core systems for repeated complex SELECT statements
Date: Fri, 04 Feb 2011 16:18:18 -0500
Message-ID: <4D4C6D1A.2000305@summersault.com> (raw)
In-Reply-To: <20110203105704.BAX82905@ms14.lnh.mail.rcn.net>
References: <20110203105704.BAX82905@ms14.lnh.mail.rcn.net>
On 02/03/2011 10:57 AM, gnuoytr@rcn.com wrote:
> For PG to prosper in the future, it has to embrace the multi-core/processor/SSD machine at the query level
As the person who brought up the original concern, I'll add that
"multi-core at the query level" really isn't important for us. Most of
our PostgreSQL usage is through a web application which fairly
automatically takes advantage of multiple cores, because there are
several parallel connections.
A smaller but important piece of what we do is run this cron script
needs to run hundreds of thousands of variations of the same complex
SELECT as fast it can.
What honestly would have helped most is not technical at all-- it would
have been some documentation on how to take advantage of multiple cores
for this case.
It looks like it's going to be trivial-- Divide up the data with a
modulo, and run multiple parallel cron scripts that each processes a
slice of the data. A benchmark showed that this approach sped up our
processing 3x when splitting the application 4 ways across 4 processors.
(I think we failed to achieve a 4x improvement because the server was
already busy handling some other tasks).
Part of our case is likely fairly common *today*: many servers are
multi-core now, but people don't necessarily understand how to take
advantage of that if it doesn't happen automatically.
Mark
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: mark@summersault.com
Subject: Re: getting the most of out multi-core systems for repeated complex SELECT statements
In-Reply-To: <4D4C6D1A.2000305@summersault.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox