Received: from localhost (maia-3.hub.org [200.46.204.184]) by postgresql.org (Postfix) with ESMTP id F12279FA0B7 for ; Fri, 15 Dec 2006 08:17:23 -0400 (AST) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.204.184]) (amavisd-new, port 10024) with ESMTP id 23434-06 for ; Fri, 15 Dec 2006 08:17:10 -0400 (AST) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.4 Received: from nz-out-0506.google.com (nz-out-0506.google.com [64.233.162.228]) by postgresql.org (Postfix) with ESMTP id B0E169FA3A7 for ; Fri, 15 Dec 2006 08:17:14 -0400 (AST) Received: by nz-out-0506.google.com with SMTP id i11so398823nzh for ; Fri, 15 Dec 2006 04:17:13 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:references; b=E16fQGKt9LoSOYYvuH5WAfB+FR37Q0egJQ1WCN+atELVZW27y7Y0GQLkBVjReQ7x1DFZPBvuSWcTl0+AscF+uhV73RcRZivkpbd6+Ew+isTuv6/Y0nVeBJ1I1k3B/7nMoA7mOBi/HC/wv1HcG9PgpRMjNkAWMuIpuNA0KgBphSE= Received: by 10.65.254.13 with SMTP id g13mr1115100qbs.1166185031560; Fri, 15 Dec 2006 04:17:11 -0800 (PST) Received: by 10.65.206.17 with HTTP; Fri, 15 Dec 2006 04:17:11 -0800 (PST) Message-ID: Date: Fri, 15 Dec 2006 17:17:11 +0500 From: "Shoaib Mir" To: "Thomas Markus" Subject: Re: out of memory in backup and restore Cc: pgsql-admin@postgresql.org In-Reply-To: <45828ED6.9020501@proventis.net> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_117577_27281510.1166185031505" References: <45826AD7.7050005@proventis.net> <45828ED6.9020501@proventis.net> X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200612/213 X-Sequence-Number: 24005 ------=_Part_117577_27281510.1166185031505 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Looks like with 1.8 GB usage not much left for dump to get the required chunk from memory. Not sure if that will help but try increasing the swap space... ------------- Shoaib Mir EnterpriseDB (www.enterprisedb.com) On 12/15/06, Thomas Markus wrote: > > Hi, > > logfile content see http://www.rafb.net/paste/results/cvD7uk33.html > - cat /proc/sys/kernel/shmmax is 2013265920 > - ulimit is unlimited > kernel is 2.6.15-1-em64t-p4-smp, pg version is 8.1.0 32bit > postmaster process usage is 1.8gb ram atm > > thx > Thomas > > > Shoaib Mir schrieb: > > Can you please show the dbserver logs and syslog at the same time when > > it goes out of memory... > > > > Also how much is available RAM you have and the SHMMAX set? > > > > ------------ > > Shoaib Mir > > EnterpriseDB ( www.enterprisedb.com ) > > > > On 12/15/06, *Thomas Markus* > > wrote: > > > > Hi, > > > > i'm running pg 8.1.0 on a debian linux (64bit) box (dual xeon 8gb > ram) > > pg_dump creates an error when exporting a large table with blobs > > (largest blob is 180mb) > > > > error is: > > pg_dump: ERROR: out of memory > > DETAIL: Failed on request of size 1073741823. > > pg_dump: SQL command to dump the contents of table "downloads" > failed: > > PQendcopy() failed. > > pg_dump: Error message from server: ERROR: out of memory > > DETAIL: Failed on request of size 1073741823. > > pg_dump: The command was: COPY public.downloads ... TO stdout; > > > > if i try pg_dump with -d dump runs with all types (c,t,p), but i > cant > > restore (out of memory error or corrupt tar header at ...) > > > > how can i backup (and restore) such a db? > > > > kr > > Thomas > > > > ---------------------------(end of > > broadcast)--------------------------- > > TIP 4: Have you searched our list archives? > > > > http://archives.postgresql.org > > > > > > -- > Thomas Markus > > Tel: +49 30 29 36 399 - 22 > Fax: +49 30 29 36 399 - 50 > Mail: t.markus@proventis.net > Web: www.proventis.net > Web: www.blue-ant.de > > proventis GmbH > Zimmerstra=DFe 79-80 > 10117 Berlin > > "proventis: Wir bewegen Organisationen." > > > > > ---------------------------(end of broadcast)--------------------------- > TIP 4: Have you searched our list archives? > > http://archives.postgresql.org > > > > ------=_Part_117577_27281510.1166185031505 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Looks like with 1.8 GB usage not much left for dump to get the required chu= nk from memory. Not sure if that will help but try increasing the swap spac= e...

-------------
Shoaib Mir
EnterpriseDB ( www.enterprisedb.com)

On 12/15/= 06, Thomas Markus <t.markus@proventis.net> wrote: Hi,

logfile content see http://www.rafb.net/paste/results/cvD7uk33.html
- ca= t /proc/sys/kernel/shmmax is 2013265920
- ulimit is unlimited
kernel = is=20 2.6.15-1-em64t-p4-smp, pg version is 8.1.0 32bit
postmaster process usag= e is 1.8gb ram atm

thx
Thomas


Shoaib Mir schrieb:
&= gt; Can you please show the dbserver logs and syslog at the same time when
> it goes out of memory...
>
> Also how much is availabl= e RAM you have and the SHMMAX set?
>
> ------------
> Sho= aib Mir
> EnterpriseDB ( www.= enterprisedb.com <http://www.enterprisedb.co= m>)
>
> On 12/15/06, *Thomas Markus* <t.markus@proventis.net
> <mailto: t.markus@proventis.net>> wrote:
>
>   =   Hi,
>
>     i'm running pg 8.1.0 on = a debian linux (64bit) box (dual xeon 8gb ram)
>   &nb= sp; pg_dump creates an error when exporting a large table with blobs
>     (largest blob is 180mb)
>
>&nb= sp;    error is:
>     pg_dump: ER= ROR:  out of memory
>     DETAIL: =  Failed on request of size 1073741823.
>    = pg_dump: SQL command to dump the contents of table "downloads" f= ailed:
>     PQendcopy() failed.
>  &nb= sp;  pg_dump: Error message from server: ERROR:  out of memo= ry
>     DETAIL:  Failed on request of = size 1073741823.
>     pg_dump: The command was: = COPY public.downloads ...  TO stdout;
>
>     if i try= pg_dump with -d dump runs with all types (c,t,p), but i cant
> =     restore (out of memory error or corrupt tar header at ..= .)
>
>     how can i backup (and restore) s= uch a db?
>
>     kr
>    = Thomas
>
>     ---------------------------= (end of
>     broadcast)-------------------------= --
>     TIP 4: Have you searched our list archiv= es?
>
>         &n= bsp;           http://archives.postgresql.org
>
>

--
Thomas Markus

Tel:   =  +49 30 29 36 399 - 22
Fax:    +49 30 29 36 399= - 50
Mail:  
t.markus@proventis.net
Web:    www.proventis.net
Web:    www.blue-ant.de

proventis GmbH<= br>Zimmerstra=DFe 79-80
10117 Berlin

"proventis: Wir bewegen Organisationen."




-= --------------------------(end of broadcast)---------------------------
= TIP 4: Have you searched our list archives?

    =            http://archives.postgresql.org




------=_Part_117577_27281510.1166185031505--