agora inbox for pgsql-sql@postgresql.org  

help / color / mirror / Atom feed

From: gmb <gmbouwer@gmail.com>
To: pgsql-sql@postgresql.org
Subject: Re: High memory usage / performance issue ( temp tables ? )
Date: Sun, 17 Aug 2014 22:33:18 -0700 (PDT)
Message-ID: <1408339998171-5815175.post@n5.nabble.com> (raw)
In-Reply-To: <CAFj8pRB44Hq8kdiwfQZRJVNLGz1Q2NAd_7RLrLJyi9Xgy3pwoA@mail.gmail.com>
References: <1408260946918-5815108.post@n5.nabble.com>
	<B6F6FD62F2624C4C9916AC0175D56D8828AC64E4@jenmbs01.ad.intershop.net>
	<1408272021571-5815111.post@n5.nabble.com>
	<1408304846514-5815126.post@n5.nabble.com>
	<CAFj8pRB44Hq8kdiwfQZRJVNLGz1Q2NAd_7RLrLJyi9Xgy3pwoA@mail.gmail.com>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-sql>

Thanks for the replies.

I tried a couple of alternative methods over the weekend in the hope of
improving performance, but unfortunately to no avail.
One of these was to have the processing of the 500K xml files shared between
multiple threads ( multiple connections ). 
In an attempt to "force" the dropping of the temp tables , each thread
creates its own connection, run the function with XML payload and the
disconnects.
The impression I got was that the avg time per transaction still increases
as the process progresses.

My one concern with this method was locking  , which I'm unfortunately quite
unfamiliar with. 

Is it possible that locking could be a key problem when following this
multi-thread approach ?

Regards

gmb



--
View this message in context: http://postgresql.1045698.n5.nabble.com/High-memory-usage-performance-issue-temp-tables-tp5815108p58...
Sent from the PostgreSQL - sql mailing list archive at Nabble.com.


-- 
Sent via pgsql-sql mailing list (pgsql-sql@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-sql

view thread (10+ messages)  latest in thread

Message-ID: <1408339998171-5815175.post@n5.nabble.com>
Permalink:  ../1408339998171-5815175.post@n5.nabble.com/
Also on:    postgresql.org/message-id/1408339998171-5815175.post@n5.nabble.com

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-sql@postgresql.org
  Cc: gmbouwer@gmail.com
  Subject: Re: High memory usage / performance issue ( temp tables ? )
  In-Reply-To: <1408339998171-5815175.post@n5.nabble.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox