X-Original-To: pgsql-admin-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.2]) by svr1.postgresql.org (Postfix) with ESMTP id 70A55D1D06C for ; Sun, 11 Apr 2004 18:11:06 -0300 (ADT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (neptune.hub.org [200.46.204.2]) (amavisd-new, port 10024) with ESMTP id 36256-03 for ; Sun, 11 Apr 2004 18:11:09 -0300 (ADT) Received: from news.hub.org (news.hub.org [200.46.204.72]) by svr1.postgresql.org (Postfix) with ESMTP id 905C3D1B9EB for ; Sun, 11 Apr 2004 18:11:03 -0300 (ADT) Received: from news.hub.org (news.hub.org [200.46.204.72]) by news.hub.org (8.12.9/8.12.9) with ESMTP id i3BLB8T1012942 for ; Sun, 11 Apr 2004 21:11:08 GMT (envelope-from news@news.hub.org) Received: (from news@localhost) by news.hub.org (8.12.9/8.12.9/Submit) id i3BKvdfZ010743 for pgsql-admin@postgresql.org; Sun, 11 Apr 2004 20:57:39 GMT From: reina_ga@hotmail.com (Tony Reina) X-Newsgroups: comp.databases.postgresql.admin Subject: Re: Database storage bloat -- FOLLOWUP Date: 11 Apr 2004 13:57:40 -0700 Organization: http://groups.google.com Lines: 46 Message-ID: <272e4be7.0404111257.555ad4a7@posting.google.com> References: <272e4be7.0404080115.39f1cb36@posting.google.com> <20040408063838.Q93611@megazone.bigpanda.com> <15732.1081446563@sss.pgh.pa.us> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-Complaints-To: groups-abuse@google.com To: pgsql-admin@postgresql.org X-Virus-Scanned: by amavisd-new at postgresql.org X-Archive-Number: 200404/188 X-Sequence-Number: 13161 I thought I'd make a followup to the question of storage bloat. I tried approaching the problem by living a little loosely with database normalization and use arrays instead of flattening everything out in the tables. So my original table, CREATE TABLE original ( dbIndex integer, index1 smallint, index2 smallint, index3 smallint, dvalue real ): becomes CREATE TABLE newtable ( dbIndex integer, dvalue real[][][] ); Now I'd expect to get better storage here just because I'm dropping 3 smallint fields (so 2 bytes x 3 for each value, or a 6 fold decrease). However, in actuality I'm seeing a 30 fold decrease: original table = 48 MB newtable = 1.6 MB !!!!! If I'm understanding the data directory correctly, the array tuples are being toasted (which I think must compress them). The actual table in the newtable format is only 8K and the pg_toast that goes with this table is 1.6MB. So I'm very pleased with the storage here. Plus, reads are faster than the original design. All in all, I think I've found a good solution for my kind of data. Now I know that the use of arrays is considered taboo in the books, but in my case the data being stored actually do fall into an array (matrix) structure naturally (it's a timebased recording so there's a sample for each time point and each X,Y position -- a natural 3D matrix). What are the possible downsides that I may be missing?