Message-ID: <4B460C5D.7080307@emolecules.com>
Date: Thu, 07 Jan 2010 08:31:25 -0800
From: Craig James <craig_james@emolecules.com>
User-Agent: Thunderbird 2.0.0.23 (X11/20090817)
MIME-Version: 1.0
To: Alvaro Herrera <alvherre@commandprompt.com>, Lefteris <lsidir@gmail.com>,
	Jochen Erwied <jochen@pgsql-performance.erwied.eu>,
	pgsql-performance@postgresql.org
Subject: Re: Air-traffic benchmark
References: <852badbc1001070438j36eb10d3qa22398ffcbef405f@mail.gmail.com>
	<4B45E36E.4040706@tweakers.net>
	<852badbc1001070547p48b47175vf87efc473e3bd41b@mail.gmail.com>
	<26014999.20100107150535@erwied.eu>
	<852badbc1001070610p698dc3cfr4628b1e7768990e6@mail.gmail.com>
	<20100107141423.GC4315@alvh.no-ip.org>
In-Reply-To: <20100107141423.GC4315@alvh.no-ip.org>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Alvaro Herrera wrote:
> No amount of tinkering is going to change the fact that a seqscan is the
> fastest way to execute these queries.  Even if you got it to be all in
> memory, it would still be much slower than the other systems which, I
> gather, are using columnar storage and thus are perfectly suited to this
> problem (unlike Postgres).  The talk about "compression ratios" caught
> me by surprise until I realized it was columnar stuff.  There's no way
> you can get such high ratios on a regular, row-oriented storage.

One of the "good tricks" with Postgres is to convert a very wide table into a set of narrow tables, then use a view to create something that looks like the original table.  It requires you to modify the write portions of your app, but the read portions can stay the same.

A seq scan on one column will *much* faster when you rearrange your database this way since it's only scanning relevant data.  You pay the price of an extra join on primary keys, though.

If you have just a few columns in a very wide table that are seq-scanned a lot, you can pull out just those columns and leave the rest in the wide table.

The same trick is also useful if you have one or a few columns that are updated frequently: pull them out, and use a view to recreate the original appearance.  It saves a lot on garbage collection.

Craig