pg.ddx.io pgsql-admin@postgresql.org mailing list archivehelp / color / mirror / Atom feed
Compression of text columns 3+ messages / 2 participants [nested] [flat]
* Compression of text columns @ 2005-10-10 12:18 Stef <svb@ucs.co.za> 2005-10-10 16:31 ` Re: Compression of text columns Jim C. Nasby <jnasby@pervasive.com> 0 siblings, 1 reply; 3+ messages in thread From: Stef @ 2005-10-10 12:18 UTC (permalink / raw) To: pgsql-admin I have a table in the databases I work with, that contains two text columns with XML data stored inside them. This table is by far the biggest table in the databases, and the text columns use up the most space. I saw that the default storage type for text columns is "EXTENDED" which, according to the documentation, uses up extra space to make possible substring functioning faster. Suppose that the data in those columns are only really ever _used_ once, but may be needed in future for viewing purposes mostly, and I cannot really change the underlying structure of the table, what can I possibly do to maximally reduce the amount of disk space used by the table on disk. (There are no indexes on these two columns.) I've thought about compression using something like : ztext http://www.mahalito.net/~harley/sw/postgres/ but I have to change the table structure a lot and I've already encountered problems unzipping the data again. The other problem with this solution, is that database dumps almost double in size, because of double compression. Any suggestions much appreciated TIA Stefan ^ permalink raw reply [nested|flat] 3+ messages in thread
* Re: Compression of text columns 2005-10-10 12:18 Compression of text columns Stef <svb@ucs.co.za> @ 2005-10-10 16:31 ` Jim C. Nasby <jnasby@pervasive.com> 2005-10-10 17:48 ` Re: Compression of text columns Stef <svb@ucs.co.za> 0 siblings, 1 reply; 3+ messages in thread From: Jim C. Nasby @ 2005-10-10 16:31 UTC (permalink / raw) To: Stef <svb@ucs.co.za>; +Cc: pgsql-admin On Mon, Oct 10, 2005 at 02:18:39PM +0200, Stef wrote: > I have a table in the databases I work with, > that contains two text columns with XML data > stored inside them. > > This table is by far the biggest table in the databases, > and the text columns use up the most space. > I saw that the default storage type for text columns is > "EXTENDED" which, according to the documentation, uses up extra > space to make possible substring functioning faster. You're thinking of EXTERNAL. From http://lnk.nu/postgresql.org/4ff.html: EXTERNAL is for external, uncompressed data, and EXTENDED is for external, compressed data. EXTENDED is the default for most data types that support non-PLAIN storage. Use of EXTERNAL will make substring operations on text and bytea columns faster, at the penalty of increased storage space. > Suppose that the data in those columns are only really ever > _used_ once, but may be needed in future for viewing purposes mostly, > and I cannot really change the underlying structure of the table, > what can I possibly do to maximally reduce the amount of disk space > used by the table on disk. (There are no indexes on these two columns.) > I've thought about compression using something like : > ztext http://www.mahalito.net/~harley/sw/postgres/ > > but I have to change the table structure a lot and I've already > encountered problems unzipping the data again. > The other problem with this solution, is that database dumps almost double > in size, because of double compression. Are you seeing much gain using ztext over using EXTENDED? -- Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com Pervasive Software http://pervasive.com work: 512-231-6117 vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461 ^ permalink raw reply [nested|flat] 3+ messages in thread
* Re: Compression of text columns 2005-10-10 12:18 Compression of text columns Stef <svb@ucs.co.za> 2005-10-10 16:31 ` Re: Compression of text columns Jim C. Nasby <jnasby@pervasive.com> @ 2005-10-10 17:48 ` Stef <svb@ucs.co.za> 0 siblings, 0 replies; 3+ messages in thread From: Stef @ 2005-10-10 17:48 UTC (permalink / raw) To: Jim C. Nasby <jnasby@pervasive.com>; +Cc: pgsql-admin Jim C. Nasby mentioned : => Are you seeing much gain using ztext over using EXTENDED? When storage is defined external, there is a visible gain, but not really significant enough to make me want to use it. And I had instances where not all the rows would uncompress again afterwards. I want to try and get the bzip2 compression working, and see if that does anything for me. But at this stage, I think it's as compressed as I will be able to get it. I'm not getting the best results, because a lot of the data rows don't contain enough data to compress properly, and I think with ztext the column sizes sometimes end up bigger than they were to start off with. Kind Regards Stefan ^ permalink raw reply [nested|flat] 3+ messages in thread
end of thread, other threads:[~2005-10-10 17:48 UTC | newest] Thread overview: 3+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2005-10-10 12:18 Compression of text columns Stef <svb@ucs.co.za> 2005-10-10 16:31 ` Jim C. Nasby <jnasby@pervasive.com> 2005-10-10 17:48 ` Stef <svb@ucs.co.za>
This inbox is served by DDX for PostgreSQL; see mirroring instructions for how to clone and mirror all data and code used for this inbox