Received: from localhost (maia-2.hub.org [200.46.204.187]) by postgresql.org (Postfix) with ESMTP id 35F5A9FA334 for ; Fri, 15 Dec 2006 08:42:56 -0400 (AST) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.204.187]) (amavisd-new, port 10024) with ESMTP id 30926-07 for ; Fri, 15 Dec 2006 08:42:35 -0400 (AST) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.4 Received: from ug-out-1314.google.com (ug-out-1314.google.com [66.249.92.172]) by postgresql.org (Postfix) with ESMTP id 79A0E9FA0FB for ; Fri, 15 Dec 2006 08:42:44 -0400 (AST) Received: by ug-out-1314.google.com with SMTP id k3so802043ugf for ; Fri, 15 Dec 2006 04:42:42 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:references; b=XLgnzL1PEExga/jghFSIirsAB3VTpXsfN5cSEHnHFqWVxYgDaVe6nmT3lnnt9P8Gp9GzZJa9HPgxKlmrKbjheUYAhjwWirY5XTd/B7zc0q1mmmBYJVRg0uNO0I/mFWWamzjFjVkNGSmZ8E6uUkkpsfOB+ob49NNhRfGrGQD0ajs= Received: by 10.67.121.15 with SMTP id y15mr1078657ugm.1166186562369; Fri, 15 Dec 2006 04:42:42 -0800 (PST) Received: by 10.66.252.11 with HTTP; Fri, 15 Dec 2006 04:42:42 -0800 (PST) Message-ID: Date: Fri, 15 Dec 2006 09:42:42 -0300 From: "Marcelo Costa" To: "Thomas Markus" Subject: Re: out of memory in backup and restore Cc: pgsql-admin@postgresql.org In-Reply-To: <45829417.3030109@proventis.net> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_198305_10894583.1166186562272" References: <45826AD7.7050005@proventis.net> <45828ED6.9020501@proventis.net> <45829417.3030109@proventis.net> X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200612/217 X-Sequence-Number: 24009 ------=_Part_198305_10894583.1166186562272 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline To decrease shared buffers you need restart your pgsql. If do you make on df -h command what is the result, please send. 2006/12/15, Thomas Markus : > > Hi, > > free diskspace is 34gb (underlying xfs) (complete db dump is 9gb). free > -tm says 6gb free ram and 6gb unused swap space. > can i decrease shared buffers without pg restart? > > thx > Thomas > > Shoaib Mir schrieb: > > Looks like with 1.8 GB usage not much left for dump to get the > > required chunk from memory. Not sure if that will help but try > > increasing the swap space... > > > > ------------- > > Shoaib Mir > > EnterpriseDB ( www.enterprisedb.com ) > > > > On 12/15/06, *Thomas Markus* > > wrote: > > > > Hi, > > > > logfile content see http://www.rafb.net/paste/results/cvD7uk33.html > > - cat /proc/sys/kernel/shmmax is 2013265920 > > - ulimit is unlimited > > kernel is 2.6.15-1-em64t-p4-smp, pg version is 8.1.0 32bit > > postmaster process usage is 1.8gb ram atm > > > > thx > > Thomas > > > > > > Shoaib Mir schrieb: > > > Can you please show the dbserver logs and syslog at the same > > time when > > > it goes out of memory... > > > > > > Also how much is available RAM you have and the SHMMAX set? > > > > > > ------------ > > > Shoaib Mir > > > EnterpriseDB ( www.enterprisedb.com > > ) > > > > > > On 12/15/06, *Thomas Markus* > > > > > >> wrote: > > > > > > Hi, > > > > > > i'm running pg 8.1.0 on a debian linux (64bit) box (dual > > xeon 8gb ram) > > > pg_dump creates an error when exporting a large table with > > blobs > > > (largest blob is 180mb) > > > > > > error is: > > > pg_dump: ERROR: out of memory > > > DETAIL: Failed on request of size 1073741823. > > > pg_dump: SQL command to dump the contents of table > > "downloads" failed: > > > PQendcopy() failed. > > > pg_dump: Error message from server: ERROR: out of memory > > > DETAIL: Failed on request of size 1073741823. > > > pg_dump: The command was: COPY public.downloads ... TO > stdout; > > > > > > if i try pg_dump with -d dump runs with all types (c,t,p), > > but i cant > > > restore (out of memory error or corrupt tar header at ...) > > > > > > how can i backup (and restore) such a db? > > > > > > kr > > > Thomas > > > > > > ---------------------------(end of > > > broadcast)--------------------------- > > > TIP 4: Have you searched our list archives? > > > > > > http://archives.postgresql.org > > > > > > > > > > > > ---------------------------(end of broadcast)--------------------------- > TIP 4: Have you searched our list archives? > > http://archives.postgresql.org > -- Marcelo Costa ------=_Part_198305_10894583.1166186562272 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline To decrease shared buffers you need restart your pgsql.

If do you make on df -h command what is the result, please send.

2006/12/15, Thomas Markus < t.markus@proventis.net>:
Hi,

free diskspace is 34gb (underlying xfs) (complete db dump is 9gb). free
-tm says 6gb free ram and 6gb unused swap space.
can i decrease shared buffers without pg restart?

thx
Thomas

Shoaib Mir schrieb:
> Looks like with 1.8 GB usage not much left for dump to get the
> required chunk from memory. Not sure if that will help but try
> increasing the swap space...
>
> -------------
> Shoaib Mir
> EnterpriseDB ( www.enterprisedb.com <http://www.enterprisedb.com>)
>
> On 12/15/06, *Thomas Markus* <t.markus@proventis.net
> <mailto: t.markus@proventis.net>> wrote:
>
>     Hi,
>
>     logfile content see http://www.rafb.net/paste/results/cvD7uk33.html
>     - cat /proc/sys/kernel/shmmax is 2013265920
>     - ulimit is unlimited
>     kernel is 2.6.15-1-em64t-p4-smp, pg version is 8.1.0 32bit
>     postmaster process usage is 1.8gb ram atm
>
>     thx
>     Thomas
>
>
>     Shoaib Mir schrieb:
>     > Can you please show the dbserver logs and syslog at the same
>     time when
>     > it goes out of memory...
>     >
>     > Also how much is available RAM you have and the SHMMAX set?
>     >
>     > ------------
>     > Shoaib Mir
>     > EnterpriseDB ( www.enterprisedb.com
>     < http://www.enterprisedb.com> <http://www.enterprisedb.com>)
>     >
>     > On 12/15/06, *Thomas Markus* < t.markus@proventis.net
>     <mailto:t.markus@proventis.net>
>     > <mailto: t.markus@proventis.net
>     <mailto: t.markus@proventis.net>>> wrote:
>     >
>     >     Hi,
>     >
>     >     i'm running pg 8.1.0 on a debian linux (64bit) box (dual
>     xeon 8gb ram)
>     >     pg_dump creates an error when exporting a large table with
>     blobs
>     >     (largest blob is 180mb)
>     >
>     >     error is:
>     >     pg_dump: ERROR:  out of memory
>     >     DETAIL:  Failed on request of size 1073741823.
>     >     pg_dump: SQL command to dump the contents of table
>     "downloads" failed:
>     >     PQendcopy() failed.
>     >     pg_dump: Error message from server: ERROR:  out of memory
>     >     DETAIL:  Failed on request of size 1073741823.
>     >     pg_dump: The command was: COPY public.downloads ...  TO stdout;
>     >
>     >     if i try pg_dump with -d dump runs with all types (c,t,p),
>     but i cant
>     >     restore (out of memory error or corrupt tar header at ...)
>     >
>     >     how can i backup (and restore) such a db?
>     >
>     >     kr
>     >     Thomas
>     >
>     >     ---------------------------(end of
>     >     broadcast)---------------------------
>     >     TIP 4: Have you searched our list archives?
>     >
>     >                     http://archives.postgresql.org
>     >
>     >
>
>

---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?

               http://archives.postgresql.org



--
Marcelo Costa ------=_Part_198305_10894583.1166186562272--