pg.ddx.io pgsql-performance@postgresql.org mailing list archive
help / color / mirror / Atom feedsize of pg_dump files containing bytea values
5+ messages / 4 participants
[nested] [flat]
* size of pg_dump files containing bytea values
@ 2006-07-12 18:36 Steve McWilliams <smcwilliams@EmprisaNetworks.com>
2006-07-13 02:53 ` Re: size of pg_dump files containing bytea values Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 5+ messages in thread
From: Steve McWilliams @ 2006-07-12 18:36 UTC (permalink / raw)
To: pgsql-performance
I notice that non-printables in bytea values are being spit out by pg_dump
using escaped octet sequences even when the "-Fc" option is present
specifying use of the custom binary output format rather than plain text
format. This bloats the size of bytea values in the dump file by a factor
of 3+ typically. When you have alot of large bytea values in your db this
can add up very quickly.
Shouldn't the custom format be smart and just write the raw bytes to the
output file rather than trying to make them ascii readable?
Thanks.
Steve McWilliams
Software Engineer
Emprisa Networks
703-691-0433x21
smcwilliams@emprisanetworks.com
The information contained in this communication is intended only for the
use of the recipient named above, and may be legally privileged,
confidential and exempt from disclosure under applicable law. If the
reader of this communication is not the intended recipient, you are hereby
notified that any dissemination, distribution or copying of this
communication, or any of its contents, is strictly prohibited. If you have
received this communication in error, please resend this communication to
the sender and delete the original communication and any copy of it from
your computer system. Thank you.
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: size of pg_dump files containing bytea values
2006-07-12 18:36 size of pg_dump files containing bytea values Steve McWilliams <smcwilliams@EmprisaNetworks.com>
@ 2006-07-13 02:53 ` Tom Lane <tgl@sss.pgh.pa.us>
2006-07-13 16:30 ` Re: size of pg_dump files containing bytea values Greg Stark <gsstark@mit.edu>
0 siblings, 1 reply; 5+ messages in thread
From: Tom Lane @ 2006-07-13 02:53 UTC (permalink / raw)
To: Steve McWilliams <smcwilliams@EmprisaNetworks.com>; +Cc: pgsql-performance
"Steve McWilliams" <smcwilliams@EmprisaNetworks.com> writes:
> I notice that non-printables in bytea values are being spit out by pg_dump
> using escaped octet sequences even when the "-Fc" option is present
> specifying use of the custom binary output format rather than plain text
> format. This bloats the size of bytea values in the dump file by a factor
> of 3+ typically.
No, because the subsequent compression step should buy back most of
that.
regards, tom lane
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: size of pg_dump files containing bytea values
2006-07-12 18:36 size of pg_dump files containing bytea values Steve McWilliams <smcwilliams@EmprisaNetworks.com>
2006-07-13 02:53 ` Re: size of pg_dump files containing bytea values Tom Lane <tgl@sss.pgh.pa.us>
@ 2006-07-13 16:30 ` Greg Stark <gsstark@mit.edu>
2006-07-13 17:42 ` Re: size of pg_dump files containing bytea values Tom Lane <tgl@sss.pgh.pa.us>
2006-07-14 07:05 ` Re: size of pg_dump files containing bytea values Florian Weimer <fweimer@bfk.de>
0 siblings, 2 replies; 5+ messages in thread
From: Greg Stark @ 2006-07-13 16:30 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Steve McWilliams <smcwilliams@EmprisaNetworks.com>; pgsql-performance
Tom Lane <tgl@sss.pgh.pa.us> writes:
> "Steve McWilliams" <smcwilliams@EmprisaNetworks.com> writes:
> > I notice that non-printables in bytea values are being spit out by pg_dump
> > using escaped octet sequences even when the "-Fc" option is present
> > specifying use of the custom binary output format rather than plain text
> > format. This bloats the size of bytea values in the dump file by a factor
> > of 3+ typically.
>
> No, because the subsequent compression step should buy back most of
> that.
Didn't byteas used to get printed as hex? Even in psql they're now being
printed in the escaped octet sequence. When did this change?
--
greg
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: size of pg_dump files containing bytea values
2006-07-12 18:36 size of pg_dump files containing bytea values Steve McWilliams <smcwilliams@EmprisaNetworks.com>
2006-07-13 02:53 ` Re: size of pg_dump files containing bytea values Tom Lane <tgl@sss.pgh.pa.us>
2006-07-13 16:30 ` Re: size of pg_dump files containing bytea values Greg Stark <gsstark@mit.edu>
@ 2006-07-13 17:42 ` Tom Lane <tgl@sss.pgh.pa.us>
1 sibling, 0 replies; 5+ messages in thread
From: Tom Lane @ 2006-07-13 17:42 UTC (permalink / raw)
To: Greg Stark <gsstark@mit.edu>; +Cc: Steve McWilliams <smcwilliams@EmprisaNetworks.com>; pgsql-performance
Greg Stark <gsstark@mit.edu> writes:
> Didn't byteas used to get printed as hex?
No, not that I recall. I don't have anything older than 7.0 running,
but it behaves the same as now:
play=> select 'xyz\\001'::bytea;
?column?
----------
xyz\001
(1 row)
play=>
regards, tom lane
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: size of pg_dump files containing bytea values
2006-07-12 18:36 size of pg_dump files containing bytea values Steve McWilliams <smcwilliams@EmprisaNetworks.com>
2006-07-13 02:53 ` Re: size of pg_dump files containing bytea values Tom Lane <tgl@sss.pgh.pa.us>
2006-07-13 16:30 ` Re: size of pg_dump files containing bytea values Greg Stark <gsstark@mit.edu>
@ 2006-07-14 07:05 ` Florian Weimer <fweimer@bfk.de>
1 sibling, 0 replies; 5+ messages in thread
From: Florian Weimer @ 2006-07-14 07:05 UTC (permalink / raw)
To: Greg Stark <gsstark@mit.edu>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Steve McWilliams <smcwilliams@EmprisaNetworks.com>; pgsql-performance
* Greg Stark:
> Didn't byteas used to get printed as hex?
No, they didn't. It would be useful to support hexadecimal BYTEA
literals, though. Unfortunately, X'DEADBEEF' has already been taken
by bit strings.
--
Florian Weimer <fweimer@bfk.de>
BFK edv-consulting GmbH http://www.bfk.de/
Durlacher Allee 47 tel: +49-721-96201-1
D-76131 Karlsruhe fax: +49-721-96201-99
^ permalink raw reply [nested|flat] 5+ messages in thread
end of thread, other threads:[~2006-07-14 07:05 UTC | newest]
Thread overview: 5+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2006-07-12 18:36 size of pg_dump files containing bytea values Steve McWilliams <smcwilliams@EmprisaNetworks.com>
2006-07-13 02:53 ` Tom Lane <tgl@sss.pgh.pa.us>
2006-07-13 16:30 ` Greg Stark <gsstark@mit.edu>
2006-07-13 17:42 ` Tom Lane <tgl@sss.pgh.pa.us>
2006-07-14 07:05 ` Florian Weimer <fweimer@bfk.de>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox