X-Original-To: pgsql-admin-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id 158393A434D for ; Thu, 11 Nov 2004 19:50:45 +0000 (GMT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 17287-05 for ; Thu, 11 Nov 2004 19:50:34 +0000 (GMT) Received: from mail710.megamailservers.com (mail710.megamailservers.com [69.49.98.20]) by svr1.postgresql.org (Postfix) with ESMTP id B65CA3A4171 for ; Thu, 11 Nov 2004 19:50:32 +0000 (GMT) X-Authenticated-User: paul.pnlassociates.com Received: from [69.3.2.106] (h-69-3-2-106.mclnva23.covad.net [69.3.2.106]) (authenticated bits=0) by mail710.megamailservers.com (8.12.10/8.12.9) with ESMTP id iABJoPMj007028 (version=TLSv1/SSLv3 cipher=RC4-MD5 bits=128 verify=NO) for ; Thu, 11 Nov 2004 14:50:31 -0500 Message-ID: <4193C282.2040904@pnlassociates.com> Date: Thu, 11 Nov 2004 14:50:26 -0500 From: "Paul B. Anderson" User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.7.3) Gecko/20040910 X-Accept-Language: en-us, en MIME-Version: 1.0 To: pgsql-admin@postgresql.org Subject: Re: Large transaction problem Content-Type: multipart/alternative; boundary="------------070603020305080602090307" X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=1.0 tagged_above=0.0 required=5.0 tests=HTML_20_30, HTML_MESSAGE, HTML_TITLE_EMPTY X-Spam-Level: * X-Archive-Number: 200411/98 X-Sequence-Number: 15483 This is a multi-part message in MIME format. --------------070603020305080602090307 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit I think I've solved this problem. It involved the vacuumdb command rather than the large insert. I found this out when a nightly vacuumdb script reported the problem even though no records had been loaded at all. My postgresql.conf file had a 10 second timeout and the large database required more than 10 seconds for the vacuum. It seems that this left postmaster and/or the particular database in a state where any SQL against that database gave the same error response about being canceled by the user. I'm guessing postgresql had left some internal state set for the database and was trying to complete the vacuum first. Anyway, I set the statement_timeout = 0 and the problem isn't recurring. Paul "Paul B. Anderson" writes: >> Ulimit says > > >> # su - postgres >> -bash-2.05b$ ulimit -a >> core file size (blocks, -c) 0 >> data seg size (kbytes, -d) unlimited >> file size (blocks, -f) unlimited >> max locked memory (kbytes, -l) 4 >> max memory size (kbytes, -m) unlimited >> open files (-n) 1024 >> pipe size (512 bytes, -p) 8 >> stack size (kbytes, -s) 10240 >> cpu time (seconds, -t) unlimited >> max user processes (-u) 3072 >> virtual memory (kbytes, -v) unlimited > > >> None of the actual limits seems to be a problem. I can't seem to change >> max locked memory using ulimit but I'll try increasing stack size. >> Could that be involved? > > No, the 'file size' and 'cpu time' limits are the ones I'd expect to cause this sort of behavior. One possible gotcha is that depending on how you start the postmaster, it might be living under different ulimit settings than what you see interactively. regards, tom lane --------------070603020305080602090307 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit I think I've solved this problem.  It involved the vacuumdb command rather than the large insert.  I found this out when a nightly vacuumdb script reported the problem even though no records had been loaded at all.

My postgresql.conf file had a 10 second timeout and the large database required more than 10 seconds for the vacuum.  It seems that this left postmaster and/or the particular database in a state where any SQL against that database gave the same error response about being canceled by the user.

I'm guessing postgresql had left some internal state set for the database and was trying to complete the vacuum first.   Anyway, I set the statement_timeout = 0 and the problem isn't recurring.

Paul


"Paul B. Anderson" <paul@pnlassociates.com> writes:
> Ulimit says
  

> # su - postgres
> -bash-2.05b$ ulimit -a
> core file size        (blocks, -c) 0
> data seg size         (kbytes, -d) unlimited
> file size             (blocks, -f) unlimited
> max locked memory     (kbytes, -l) 4
> max memory size       (kbytes, -m) unlimited
> open files                    (-n) 1024
> pipe size          (512 bytes, -p) 8
> stack size            (kbytes, -s) 10240
> cpu time             (seconds, -t) unlimited
> max user processes            (-u) 3072
> virtual memory        (kbytes, -v) unlimited
  

> None of the actual limits seems to be a problem.  I can't seem to change 
> max locked memory using ulimit but I'll try increasing stack size.  
> Could that be involved?
  

No, the 'file size' and 'cpu time' limits are the ones I'd expect to
cause this sort of behavior.

One possible gotcha is that depending on how you start the postmaster,
it might be living under different ulimit settings than what you see
interactively.

			regards, tom lane


--------------070603020305080602090307--