pg.ddx.io pgsql-performance@postgresql.org mailing list archive
help / color / mirror / Atom feedpgsql BLOB issues
4+ messages / 3 participants
[nested] [flat]
* pgsql BLOB issues
@ 2003-04-28 02:30 Jeremy Andrus <jeremy@jeremya.com>
0 siblings, 1 reply; 4+ messages in thread
From: Jeremy Andrus @ 2003-04-28 02:30 UTC (permalink / raw)
To: pgsql-performance
Hello,
I have a database that contains a large amount of Large Objects
(>500MB). I am using this database to store images for an e-commerce
website, so I have a simple accessor script written in perl to dump out
a blob based on a virtual 'path' stored in a table (and associated with
the large object's OID). This system seemed to work wonderfully until I
put more than ~500MB of binary data into the database.
Now, every time I run the accessor script (via the web OR the command
line), the postmaster process gobbles up my CPU resources (usually >30%
for a single process - and it's a 1GHz processor with 1GB of RAM!), and
the script takes a very long time to completely dump out the data.
I have the same issue with an import script that reads files from the
hard drive and puts them into Large Objects in the database. It takes a
very long time to import whereas before, it would run extremely fast.
Are there any known issues in PostgreSQL involving databases with a
lot of binary data? I am using PostgreSQL v7.2.3 on a linux system.
Thanks,
-Jeremy
--
------------------------
Jeremy C. Andrus
http://www.jeremya.com/
------------------------
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: pgsql BLOB issues
@ 2003-04-28 05:00 Tom Lane <tgl@sss.pgh.pa.us>
parent: Jeremy Andrus <jeremy@jeremya.com>
0 siblings, 1 reply; 4+ messages in thread
From: Tom Lane @ 2003-04-28 05:00 UTC (permalink / raw)
To: jeremy@jeremya.com; +Cc: pgsql-performance
Jeremy Andrus <jeremy@jeremya.com> writes:
> I have a database that contains a large amount of Large Objects
> (>500MB). I am using this database to store images for an e-commerce
> website, so I have a simple accessor script written in perl to dump out
> a blob based on a virtual 'path' stored in a table (and associated with
> the large object's OID). This system seemed to work wonderfully until I
> put more than ~500MB of binary data into the database.
Are you talking about 500MB in one BLOB, or 500MB total?
If the former, I can well imagine swap thrashing being a problem when
you try to access such a large blob.
If the latter, I can't think of any reason for total blob storage to
cause any big performance issue. Perhaps you just haven't vacuumed
pg_largeobject in a long time?
regards, tom lane
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: pgsql BLOB issues
@ 2003-04-28 05:33 Jeremy Andrus <jeremy@jeremya.com>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 4+ messages in thread
From: Jeremy Andrus @ 2003-04-28 05:33 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-performance
On Monday 28 April 2003 01:00 am, Tom Lane wrote:
> Are you talking about 500MB in one BLOB, or 500MB total?
I meant 500MB _total_. There are over 5000 separate BLOBs.
I'll ask my friendly sys-admin to vacuum pg_largeobject, and I'll let
you know what happens :-) In general though, how much performance is
really gained through regular vacuuming?
Thanks for your help,
-Jeremy
--
------------------------
Jeremy C. Andrus
http://www.jeremya.com/
------------------------
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: pgsql BLOB issues
@ 2003-04-28 08:19 Christopher Kings-Lynne <chriskl@familyhealth.com.au>
parent: Jeremy Andrus <jeremy@jeremya.com>
0 siblings, 0 replies; 4+ messages in thread
From: Christopher Kings-Lynne @ 2003-04-28 08:19 UTC (permalink / raw)
To: jeremy@jeremya.com; Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-performance
> I'll ask my friendly sys-admin to vacuum pg_largeobject, and I'll let
> you know what happens :-) In general though, how much performance is
> really gained through regular vacuuming?
Significant. It's essential to vacuum regularly.
Chris
^ permalink raw reply [nested|flat] 4+ messages in thread
end of thread, other threads:[~2003-04-28 08:19 UTC | newest]
Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2003-04-28 02:30 pgsql BLOB issues Jeremy Andrus <jeremy@jeremya.com>
2003-04-28 05:00 ` Tom Lane <tgl@sss.pgh.pa.us>
2003-04-28 05:33 ` Jeremy Andrus <jeremy@jeremya.com>
2003-04-28 08:19 ` Christopher Kings-Lynne <chriskl@familyhealth.com.au>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox