pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Martin Foster <martin@ethereal-realms.org>
To: pgsql-performance@postgresql.org
Subject: Re: Performance Bottleneck
Date: Fri, 06 Aug 2004 23:18:49 GMT
Message-ID: <tlUQc.32079$cd2.29211@clgrps12> (raw)
In-Reply-To: <1091812011.7149.25.camel@ipso.snappymail.ca>
References: <ktQPc.23297$yT2.16983@clgrps13>
	<410FDC53.5080509@bigfoot.com>
	<X0ZPc.23543$yT2.23053@clgrps13>
	<4110FFF6.1030001@bigfoot.com>
	<1091812011.7149.25.camel@ipso.snappymail.ca>

Mike Benoit wrote:

> On Wed, 2004-08-04 at 17:25 +0200, Gaetano Mendola wrote:
> 
> 
>>>The queries themselves are simple, normally drawing information from one 
>>>table with few conditions or in the most complex cases using joins on 
>>>two table or sub queries.   These behave very well and always have, the 
>>>problem is that these queries take place in rather large amounts due to 
>>>the dumb nature of the scripts themselves.
>>
>>Show us the explain analyze on that queries, how many rows the tables are
>>containing, the table schema could be also usefull.
>>
> 
> 
> If the queries themselves are optimized as much as they can be, and as
> you say, its just the sheer amount of similar queries hitting the
> database, you could try using prepared queries for ones that are most
> often executed to eliminate some of the overhead. 
> 
> I've had relatively good success with this in the past, and it doesn't
> take very much code modification.
> 

One of the biggest problems is most probably related to the indexes. 
Since the performance penalty of logging the information needed to see 
which queries are used and which are not is a slight problem, then I 
cannot really make use of it for now.

However, I am curious how one would go about preparing query?   Is this 
similar to the DBI::Prepare statement with placeholders and simply 
changing the values passed on execute?  Or is this something database 
level such as a view et cetera?

SELECT
  Post.PostIDNumber,
  Post.$format,
  Post.PuppeteerLogin,
  Post.PuppetName,
  Post.PostCmd,
  Post.PostClass
FROM Post
WHERE Post.PostIDNumber > ?::INT
   AND (Post.PostTo='all' OR Post.PostTo=?)
   AND (NOT EXISTS (SELECT PuppetIgnore.PuppetLogin
	FROM PuppetIgnore
	WHERE PuppetIgnore.PuppetIgnore='global'
	  AND PuppetIgnore.PuppeteerLogin=?
	  AND PuppetIgnore.PuppetLogin=Post.PuppeteerLogin)
    OR Post.PuppeteerLogin IS NULL)
   AND (NOT EXISTS (SELECT PuppetIgnore.PuppetName
	FROM PuppetIgnore
	WHERE PuppetIgnore.PuppetIgnore='single'
	  AND PuppetIgnore.PuppeteerLogin=?
	  AND PuppetIgnore.PuppetName=Post.PuppetName)
    OR Post.PuppetName IS NULL)
ORDER BY Post.PostIDNumber LIMIT 100

The range is determined from the previous run or through a query listed 
below.   It was determined that using INT was far faster then limiting 
by timestamp.

SELECT MIN(PostIDNumber)
FROM Post
WHERE RealmName=?
   AND PostClass IN ('general','play')
   AND PostTo='all'

The above simply provides a starting point, nothing more.   Once posts 
are pulled the script will throw in the last pulled number as to start 
from a fresh point.

Under MySQL time was an stored as an INT which may have helped it handle 
timestamps more efficiently.    It also made use of three or more 
queries, where two were done to generate an IN statement for the query 
actually running at the time.

	Martin Foster
	Creator/Designer Ethereal Realms
	martin@ethereal-realms.org



view thread (38+ messages)  latest in thread

Message-ID: <tlUQc.32079$cd2.29211@clgrps12>
Permalink:  ../tlUQc.32079$cd2.29211@clgrps12/
Also on:    postgresql.org/message-id/tlUQc.32079$cd2.29211@clgrps12

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-performance@postgresql.org
  Cc: martin@ethereal-realms.org
  Subject: Re: Performance Bottleneck
  In-Reply-To: <tlUQc.32079$cd2.29211@clgrps12>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox