agora inbox for pgsql-sql@postgresql.org  

help / color / mirror / Atom feed

From: gmb <gmbouwer@gmail.com>
To: pgsql-sql@postgresql.org
Subject: Re: High memory usage / performance issue ( temp tables ? )
Date: Sun, 17 Aug 2014 03:40:21 -0700 (PDT)
Message-ID: <1408272021571-5815111.post@n5.nabble.com> (raw)
In-Reply-To: <B6F6FD62F2624C4C9916AC0175D56D8828AC64E4@jenmbs01.ad.intershop.net>
References: <1408260946918-5815108.post@n5.nabble.com>
	<B6F6FD62F2624C4C9916AC0175D56D8828AC64E4@jenmbs01.ad.intershop.net>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-sql>

>> Are you using the same temp tables for the whole batch or do you generate
a few 100K
>> of them ? 

The process re-creates the 10 temp table for each instance of the function
being called.
I.e. this will equate to 500k temp tables for 50k xml files. 
The "ON COMMIT DROP" part was added at some stage as an attempt to solve
some performance issues. THe argument was that , since a COMMIT is done
after each of the 50k xml files , the number of temp tables will not build
up and cause any problems. 

I can understand the performance issue due to load on the catalog, but I
would not have expected this to have the impact I'm experiencing.

>> It may help to call analyze  explicitly on the touched tables 
>> a few times during your process. Here a look at the monitoring statistics
>> may give some clue. 
>> (http://blog.pgaddict.com/posts/the-two-kinds-of-stats-in-postgresql) 

Thanks, I'll try this and see of this makes any difference.

THanks for the input.

Regards

gmb



--
View this message in context: http://postgresql.1045698.n5.nabble.com/High-memory-usage-performance-issue-temp-tables-tp5815108p58...
Sent from the PostgreSQL - sql mailing list archive at Nabble.com.


-- 
Sent via pgsql-sql mailing list (pgsql-sql@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-sql

view thread (10+ messages)  latest in thread

Message-ID: <1408272021571-5815111.post@n5.nabble.com>
Permalink:  ../1408272021571-5815111.post@n5.nabble.com/
Also on:    postgresql.org/message-id/1408272021571-5815111.post@n5.nabble.com

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-sql@postgresql.org
  Cc: gmbouwer@gmail.com
  Subject: Re: High memory usage / performance issue ( temp tables ? )
  In-Reply-To: <1408272021571-5815111.post@n5.nabble.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox