X-Original-To: pgsql-interfaces-postgresql.org@localhost.postgresql.org Received: from localhost (av.hub.org [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id 8BCC1D7D42 for ; Wed, 9 Nov 2005 12:52:38 -0400 (AST) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 98212-02 for ; Wed, 9 Nov 2005 16:52:35 +0000 (GMT) X-Greylist: from auto-whitelisted by SQLgrey- Received: from sss.pgh.pa.us (sss.pgh.pa.us [66.207.139.130]) by svr1.postgresql.org (Postfix) with ESMTP id A1D81D6FCC for ; Wed, 9 Nov 2005 12:52:35 -0400 (AST) Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1]) by sss.pgh.pa.us (8.13.1/8.13.1) with ESMTP id jA9GqZ0n016567; Wed, 9 Nov 2005 11:52:35 -0500 (EST) To: Jon Clements Cc: "pgsql-interfaces@postgresql.org" Subject: Re: PLPythonU & Out of Memory - Importing Query In-reply-to: <4c05f1b439ce248766e7cd5375a83ddc@readgroup.co.uk> References: <4c05f1b439ce248766e7cd5375a83ddc@readgroup.co.uk> Comments: In-reply-to Jon Clements message dated "Tue, 08 Nov 2005 14:21:01 +0000" Date: Wed, 09 Nov 2005 11:52:35 -0500 Message-ID: <16566.1131555155@sss.pgh.pa.us> From: Tom Lane X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, score=0.006 required=5 tests=[AWL=0.006] X-Spam-Score: 0.006 X-Spam-Level: X-Archive-Number: 200511/9 X-Sequence-Number: 5805 Jon Clements writes: > I am currently experimenting using plpythonu with postgresql 8.0 for > Win32. It's basically a quick script that imports data from CSV files, > but does some quite complicated data lookups and selections. The area > in which I'm somewhat confunded is memory usage. The process > successfully runs, but keeps climbing in memory usage relentlessly, > successfully importing about 200k records, before the memory usage of > postgres soars to 2gb and of course, shortly after that, grinds to a > halt with a "Out of Memory" error. You may have stumbled across a memory leak in plpythonu, but there's not enough info here for anyone to investigate. Can you put together a self-contained test case for people to try? regards, tom lane