DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta;
	h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references;
	b=CiE7mWkTRVNoHGo5DGUNPaftkSYz2RWgumrNKv9juxLKcOtTPeJB+ccFoWX37DGsU6LXuwmtah3dNctLhpByE5L/C1m0AFKlAZD1pTCT3egHZ9qc6ptMGa2Utg4Dk27GRh3ALpYTRIi/nvwLopE4gdZa7+GFKjYDHLExND0SJZU=
Message-ID: <36e682920710210944m4ba6b073yde7ef284aa7e1d74@mail.gmail.com>
Date: Sun, 21 Oct 2007 12:44:01 -0400
From: "Jonah H. Harris" <jonah.harris@gmail.com>
To: "Pavel Velikhov" <pvelikhov@yahoo.com>
Subject: Re: need help with a query
Cc: "Kenneth Marshall" <ktm@rice.edu>, pgsql-performance@postgresql.org
In-Reply-To: <639622.45761.qm@web56412.mail.re3.yahoo.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
References: <639622.45761.qm@web56412.mail.re3.yahoo.com>

On 10/20/07, Pavel Velikhov <pvelikhov@yahoo.com> wrote:
> Left the query running for 10+ hours and had to kill it. I guess there
> really was no need to have lots of
> shared buffers (the hope was that postgresql will cache the whole table). I
> ended up doing this step inside
> the application as a pre-processing step. Can't have postgres running with
> different fsych options since this
> will be part of an "easy to install and run" app, that should just require a
> typical PosgreSQL installation.

Is the size always different?  If not, you could limit the updates:

UPDATE links
  SET target_size = size
 FROM articles
 WHERE articles.article_id = links.article_to
        AND links.target_size != articles.size;

Since this is a huge operation, what about trying:

CREATE TABLE links_new AS SELECT l.col1, l.col2, a.size as
target_size, l.col4, ... FROM links l, articles a WHERE a.article_id =
l.article_to;

Then truncate links, copy the data from links_new.  Alternatively, you
could drop links, rename links_new to links, and recreate the
constraints.

I guess the real question is application design.  Why doesn't this app
store size at runtime instead of having to batch this huge update?

-- 
Jonah H. Harris, Sr. Software Architect | phone: 732.331.1324
EnterpriseDB Corporation                | fax: 732.331.1301
499 Thornall Street, 2nd Floor          | jonah.harris@enterprisedb.com
Edison, NJ 08837                        | http://www.enterprisedb.com/