pg.ddx.io  pgsql-admin@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Jim C. Nasby <jnasby@pervasive.com>
To: Sriram Dandapani <sdandapani@counterpane.com>
Cc: pgsql-admin@postgresql.org
Subject: Re: slow cursor
Date: Mon, 24 Apr 2006 12:42:29 -0500
Message-ID: <20060424174229.GE48010@pervasive.com> (raw)
In-Reply-To: <6992E470F12A444BB787B5C937B9D4DF0406AD88@ca-mail1.cis.local>
References: <6992E470F12A444BB787B5C937B9D4DF0406AD88@ca-mail1.cis.local>

Ok, so why does that require row-by-row processing?

On Thu, Apr 20, 2006 at 08:45:57PM -0700, Sriram Dandapani wrote:
> Yes..all of it is in one transaction as there is a window of record ids
> that need to be processed in 1 transaction. Data inflow is very
> voluminous appx 1 million every 15 minutes and the goal is to create
> aggregate tables on the fly (the alternative is to use nightly
> aggregates). 
> 
> -----Original Message-----
> From: Jim C. Nasby [mailto:jnasby@pervasive.com] 
> Sent: Thursday, April 20, 2006 7:36 PM
> To: Sriram Dandapani
> Cc: pgsql-admin@postgresql.org
> Subject: Re: [ADMIN] slow cursor
> 
> On Mon, Apr 17, 2006 at 07:07:54AM -0700, Sriram Dandapani wrote:
> > I have a cursor that fetches 150K rows and updates or inserts a table
> > with 150K rows.
> > 
> > It takes several minutes for the process to complete (about 15
> minutes).
> > The select by itself (without cursor) gets all rows in 15 seconds.
> > 
> > Is there a way to optimize the cursor to fetch all records and speed
> up
> > the process. I still need to do the record by record processing
> 
> Not likely. Are you at least doing all this inside a transaction?
> -- 
> Jim C. Nasby, Sr. Engineering Consultant      jnasby@pervasive.com
> Pervasive Software      http://pervasive.com    work: 512-231-6117
> vcard: http://jim.nasby.net/pervasive.vcf       cell: 512-569-9461
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Have you searched our list archives?
> 
>                http://archives.postgresql.org
> 

-- 
Jim C. Nasby, Sr. Engineering Consultant      jnasby@pervasive.com
Pervasive Software      http://pervasive.com    work: 512-231-6117
vcard: http://jim.nasby.net/pervasive.vcf       cell: 512-569-9461



view thread (6+ messages)

Message-ID: <20060424174229.GE48010@pervasive.com>
Permalink:  ../20060424174229.GE48010@pervasive.com/
Also on:    postgresql.org/message-id/20060424174229.GE48010@pervasive.com

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-admin@postgresql.org
  Cc: jnasby@pervasive.com, sdandapani@counterpane.com
  Subject: Re: slow cursor
  In-Reply-To: <20060424174229.GE48010@pervasive.com>
  References: <6992E470F12A444BB787B5C937B9D4DF0406AD88@ca-mail1.cis.local> <20060424174229.GE48010@pervasive.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox