From: Craig Ringer <craig@postnewspapers.com.au>
To: Aren Cambre <aren@arencambre.com>
Cc: pgsql-performance@postgresql.org
Subject: Re: Postgres refusing to use >1 core
Date: Mon, 23 May 2011 13:22:51 +0800
Message-ID: <4DD9EF2B.9030104@postnewspapers.com.au> (raw)
In-Reply-To: <BANLkTikz3mYua4e1kmTfKkfOxay06oBJjg@mail.gmail.com>
References: <BANLkTimohiUa6rOYdxYTwqmvSUg2CRYMHg@mail.gmail.com>
<BANLkTinYJFLSmZ8G8kaTxifapZDRAKBjNg@mail.gmail.com>
<BANLkTikz3mYua4e1kmTfKkfOxay06oBJjg@mail.gmail.com>
On 23/05/11 12:09, Aren Cambre wrote:
> Also, thanks for the advice on batching my queries. I am now using a
> very efficient bulk data read and write methods for Postgres.
>
> My program bulk reads 100,000 rows, processes those rows (during which
> it does a few SELECTs), and then writes 100,000 rows at a time.
>
> It cycles through this until it has processed all 12,000,000 rows.
>
> This, plus the parallelism fix, will probably convert this 30 hour
> program to a <2 hour program.
It's always good to hear when these things work out. Thanks for
reporting back.
Using the set-based nature of relational databases to your advantage,
writing smarter queries that do more work server-side with fewer
round-trips, and effective batching can make a huge difference.
--
Craig Ringer
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: craig@postnewspapers.com.au, aren@arencambre.com
Subject: Re: Postgres refusing to use >1 core
In-Reply-To: <4DD9EF2B.9030104@postnewspapers.com.au>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox