pg.ddx.io  pgsql-admin@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Marty Scholes <marty@outputservices.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: pgsql-admin@postgresql.org
Subject: Re: 7.4.6 pg_dump failed
Date: Wed, 26 Jan 2005 09:32:06 -0700
Message-ID: <41F7C606.8050007@outputservices.com> (raw)
References: <41F7B44F.7030706@outputservices.com>
	<15489.1106756375@sss.pgh.pa.us>

The table contains postscript documents.  My company prints and mails 
about 50,000 pages of invoices each day for a customer. We are 
experimenting with putting up a 7 year web based document archive for 
their customers to cut mailing costs and make it easier for their 
customers to take advantage of early pay discounts.  The archive might 
also be used for the customer service reps at the call centers.

I was able to truncate the table since this is experimental and most of 
the data can be reconstructed after the upgrade to 8.0.  The TEXT field 
contains the postscript of each document.  Some invoices are 10,000+ 
pages.  The largest one I recall was about 200 MB in size.

If we do get this project rolling under Pg, I will still have the issue 
with dumping.  Perhaps it was the -d option that caused it.  I 
appreciate the feedback.

Marty

Tom Lane wrote:
> Marty Scholes <marty@outputservices.com> writes:
> 
>>A pg_dump of one table ran for 28:53:29.50 and produced a 30 GB dump 
>>before it aborted with:
> 
> 
>>pg_dump: dumpClasses(): SQL command failed
>>pg_dump: Error message from server: out of memory for query result
>>pg_dump: The command was: FETCH 100 FROM _pg_dump_cursor
> 
> 
> Even though it says "from server", this is actually an out-of-memory
> problem inside pg_dump, or more specifically inside libpq.
> 
> 
>>The table contains a text field that could contain several hundred MB of 
>>data, although always less than 2GB.
> 
> 
> "Could contain"?  What's the actual maximum field width, and how often
> do very wide values occur?  I don't recall the exact space allocation
> algorithms inside libpq, but I'm wondering if it could choke on such a
> wide row.
> 
> You might have better luck if you didn't use -d.
> 
> 			regards, tom lane






view thread (4+ messages)  latest in thread

Message-ID: <41F7C606.8050007@outputservices.com>
Permalink:  ../41F7C606.8050007@outputservices.com/
Also on:    postgresql.org/message-id/41F7C606.8050007@outputservices.com

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-admin@postgresql.org
  Cc: marty@outputservices.com, tgl@sss.pgh.pa.us
  Subject: Re: 7.4.6 pg_dump failed
  In-Reply-To: <41F7C606.8050007@outputservices.com>
  References: <41F7B44F.7030706@outputservices.com> <15489.1106756375@sss.pgh.pa.us> <41F7C606.8050007@outputservices.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox