Date: Sun, 17 Aug 2014 22:33:18 -0700 (PDT)
From: gmb <gmbouwer@gmail.com>
To: pgsql-sql@postgresql.org
Message-ID: <1408339998171-5815175.post@n5.nabble.com>
In-Reply-To: 
 <CAFj8pRB44Hq8kdiwfQZRJVNLGz1Q2NAd_7RLrLJyi9Xgy3pwoA@mail.gmail.com>
References: <1408260946918-5815108.post@n5.nabble.com>
 <B6F6FD62F2624C4C9916AC0175D56D8828AC64E4@jenmbs01.ad.intershop.net>
 <1408272021571-5815111.post@n5.nabble.com>
 <1408304846514-5815126.post@n5.nabble.com>
 <CAFj8pRB44Hq8kdiwfQZRJVNLGz1Q2NAd_7RLrLJyi9Xgy3pwoA@mail.gmail.com>
Subject: Re: High memory usage / performance issue ( temp tables ? )
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Precedence: bulk
Sender: pgsql-sql-owner@postgresql.org

Thanks for the replies.

I tried a couple of alternative methods over the weekend in the hope of
improving performance, but unfortunately to no avail.
One of these was to have the processing of the 500K xml files shared between
multiple threads ( multiple connections ). 
In an attempt to "force" the dropping of the temp tables , each thread
creates its own connection, run the function with XML payload and the
disconnects.
The impression I got was that the avg time per transaction still increases
as the process progresses.

My one concern with this method was locking  , which I'm unfortunately quite
unfamiliar with. 

Is it possible that locking could be a key problem when following this
multi-thread approach ?

Regards

gmb


--
View this message in context: http://postgresql.1045698.n5.nabble.com/High-memory-usage-performance-issue-temp-tables-tp5815108p5815175.html
Sent from the PostgreSQL - sql mailing list archive at Nabble.com.


-- 
Sent via pgsql-sql mailing list (pgsql-sql@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-sql