X-Original-To: pgsql-sql-postgresql.org@localhost.postgresql.org
Received: from localhost (av.hub.org [200.46.204.144])
	by postgresql.org (Postfix) with ESMTP id 898F39DC842
	for <pgsql-sql-postgresql.org@localhost.postgresql.org>;
	Tue, 14 Feb 2006 12:35:01 -0400 (AST)
Received: from postgresql.org ([200.46.204.71])
	by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024)
	with ESMTP id 62932-01
	for <pgsql-sql-postgresql.org@localhost.postgresql.org>;
	Tue, 14 Feb 2006 12:34:59 -0400 (AST)
X-Greylist: from auto-whitelisted by SQLgrey-
Received: from rwcrmhc13.comcast.net (rwcrmhc13.comcast.net [216.148.227.153])
	by postgresql.org (Postfix) with ESMTP id 8FBAF9DC81F
	for <pgsql-sql@postgresql.org>; Tue, 14 Feb 2006 12:34:58 -0400 (AST)
Received: from [10.0.1.5] (c-24-20-199-254.hsd1.or.comcast.net[24.20.199.254])
	by comcast.net (rwcrmhc13) with SMTP
	id <20060214163441m1300fkos5e>; Tue, 14 Feb 2006 16:34:57 +0000
Subject: Re: Slow update SQL
From: Ken Hill <ken@scottshill.com>
Reply-To: ken@scottshill.com
To: Michael Fuhr <mike@fuhr.org>
Cc: pgsql-sql@postgresql.org
In-Reply-To: <20060214051721.GA76364@winnie.fuhr.org>
References: <1139881725.3183.2.camel@localhost.localdomain>
	<20060214021434.GA75196@winnie.fuhr.org>
	<1139885286.3183.10.camel@localhost.localdomain>
	<20060214051721.GA76364@winnie.fuhr.org>
Content-Type: multipart/alternative; boundary="=-iY4l5K5D8UT+SG56auYX"
Organization: Scotts Hill Associates, Inc.
Date: Tue, 14 Feb 2006 08:35:16 -0800
Message-Id: <1139934917.3083.2.camel@localhost.localdomain>
Mime-Version: 1.0
X-Mailer: Evolution 2.4.1 
X-Virus-Scanned: by amavisd-new at hub.org
X-Spam-Status: No, score=0.081 required=5 tests=[AWL=0.080,
 HTML_MESSAGE=0.001]
X-Spam-Score: 0.081
X-Spam-Level: 
X-Archive-Number: 200602/149
X-Sequence-Number: 24159


--=-iY4l5K5D8UT+SG56auYX
Content-Type: text/plain
Content-Transfer-Encoding: 7bit

On Mon, 2006-02-13 at 22:17 -0700, Michael Fuhr wrote:

> [Please copy the mailing list on replies.]
> 
> On Mon, Feb 13, 2006 at 06:48:06PM -0800, Ken Hill wrote:
> > On Mon, 2006-02-13 at 19:14 -0700, Michael Fuhr wrote:
> > > How many rows does the condition match?
> >
> > csalgorithm=# SELECT count(*) FROM ncccr10 WHERE
> > date_part('year',dxdate) > '2000';
> >  count
> > --------
> >  199209
> > (1 row)
> 
> You're updating about a third of the table; an expression index on
> date_part probably wouldn't help because the planner is likely to
> stick with a sequential scan for such a large update.  Even if it
> did help it's likely to be a small fraction of the total time.
> 
> The table definition you sent me showed nine indexes.  You might
> see a substantial performance improvement by dropping all the
> indexes, doing the update, then creating the indexes again (don't
> forget to vacuum and analyze the table after the update).  However,
> dropping the indexes has obvious implications for other queries so
> you might need to do the update at a time when that doesn't matter.
> 
> > > Have you queried pg_locks
> > > to see if the update is blocked on an ungranted lock?
> > 
> > I don't know what that is. How do I query pg_locks?
> 
> SELECT * FROM pg_locks;
> 
> http://www.postgresql.org/docs/7.4/static/monitoring-locks.html
> 
> > > What version of PostgreSQL are you running?
> > 
> > 7.4.8. Thank you for your help.
> 
> Newer versions generally perform better; consider upgrading to 8.0
> or 8.1 if possible.
> 

Removing the indexes, running the update SQL, and then adding back the
indexes worked much faster. Thank you for you help.

--=-iY4l5K5D8UT+SG56auYX
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 7bit

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 TRANSITIONAL//EN">
<HTML>
<HEAD>
  <META HTTP-EQUIV="Content-Type" CONTENT="text/html; CHARSET=UTF-8">
  <META NAME="GENERATOR" CONTENT="GtkHTML/3.8.1">
</HEAD>
<BODY>
On Mon, 2006-02-13 at 22:17 -0700, Michael Fuhr wrote:
<BLOCKQUOTE TYPE=CITE>
<PRE>
<FONT COLOR="#000000">[Please copy the mailing list on replies.]</FONT>

<FONT COLOR="#000000">On Mon, Feb 13, 2006 at 06:48:06PM -0800, Ken Hill wrote:</FONT>
<FONT COLOR="#000000">&gt; On Mon, 2006-02-13 at 19:14 -0700, Michael Fuhr wrote:</FONT>
<FONT COLOR="#000000">&gt; &gt; How many rows does the condition match?</FONT>
<FONT COLOR="#000000">&gt;</FONT>
<FONT COLOR="#000000">&gt; csalgorithm=# SELECT count(*) FROM ncccr10 WHERE</FONT>
<FONT COLOR="#000000">&gt; date_part('year',dxdate) &gt; '2000';</FONT>
<FONT COLOR="#000000">&gt;  count</FONT>
<FONT COLOR="#000000">&gt; --------</FONT>
<FONT COLOR="#000000">&gt;  199209</FONT>
<FONT COLOR="#000000">&gt; (1 row)</FONT>

<FONT COLOR="#000000">You're updating about a third of the table; an expression index on</FONT>
<FONT COLOR="#000000">date_part probably wouldn't help because the planner is likely to</FONT>
<FONT COLOR="#000000">stick with a sequential scan for such a large update.  Even if it</FONT>
<FONT COLOR="#000000">did help it's likely to be a small fraction of the total time.</FONT>

<FONT COLOR="#000000">The table definition you sent me showed nine indexes.  You might</FONT>
<FONT COLOR="#000000">see a substantial performance improvement by dropping all the</FONT>
<FONT COLOR="#000000">indexes, doing the update, then creating the indexes again (don't</FONT>
<FONT COLOR="#000000">forget to vacuum and analyze the table after the update).  However,</FONT>
<FONT COLOR="#000000">dropping the indexes has obvious implications for other queries so</FONT>
<FONT COLOR="#000000">you might need to do the update at a time when that doesn't matter.</FONT>

<FONT COLOR="#000000">&gt; &gt; Have you queried pg_locks</FONT>
<FONT COLOR="#000000">&gt; &gt; to see if the update is blocked on an ungranted lock?</FONT>
<FONT COLOR="#000000">&gt; </FONT>
<FONT COLOR="#000000">&gt; I don't know what that is. How do I query pg_locks?</FONT>

<FONT COLOR="#000000">SELECT * FROM pg_locks;</FONT>

<FONT COLOR="#000000"><A HREF="http://www.postgresql.org/docs/7.4/static/monitoring-locks.html">http://www.postgresql.org/docs/7.4/static/monitoring-locks.html</A></FONT>

<FONT COLOR="#000000">&gt; &gt; What version of PostgreSQL are you running?</FONT>
<FONT COLOR="#000000">&gt; </FONT>
<FONT COLOR="#000000">&gt; 7.4.8. Thank you for your help.</FONT>

<FONT COLOR="#000000">Newer versions generally perform better; consider upgrading to 8.0</FONT>
<FONT COLOR="#000000">or 8.1 if possible.</FONT>

</PRE>
</BLOCKQUOTE>
Removing the indexes, running the update SQL, and then adding back the indexes worked much faster. Thank you for you help.
</BODY>
</HTML>

--=-iY4l5K5D8UT+SG56auYX--