X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (developer.postgresql.org [64.117.224.193]) by svr1.postgresql.org (Postfix) with ESMTP id E3413B439E1 for ; Wed, 9 Jul 2003 17:43:12 +0000 (GMT) Received: from svr1.postgresql.org ([64.117.224.193]) by localhost (svr1.postgresql.org [64.117.224.193]) (amavisd-new, port 10024) with ESMTP id 05878-08 for ; Wed, 9 Jul 2003 14:42:40 -0300 (ADT) Received: from mailav.tor1.inquent.com (mail.inquent.com [216.208.117.106]) by svr1.postgresql.org (Postfix) with ESMTP id 6F76BB439FF for ; Wed, 9 Jul 2003 14:42:06 -0300 (ADT) Received: from localhost (localhost [127.0.0.1]) by mailav.tor1.inquent.com (Postfix) with ESMTP id CA22D1025A; Wed, 9 Jul 2003 13:35:23 -0400 (EDT) Received: from [10.0.2.7] (unknown [10.0.2.7]) by mailav.tor1.inquent.com (Postfix) with ESMTP id B7DA510248; Wed, 9 Jul 2003 13:35:21 -0400 (EDT) Subject: Re: Some very weird behaviour.... From: Rod Taylor To: excalibur@hub.org Cc: Postgresql Performance In-Reply-To: <1057771777.33712.98.camel@freebsd47> References: <1057771777.33712.98.camel@freebsd47> Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="=-2WPcwRj2P0eDr0K6ZKFz" Message-Id: <1057772554.46100.103.camel@jester> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.0 Date: 09 Jul 2003 17:42:35 +0000 X-Virus-Scanned: by AMaViS perl-11 X-Virus-Scanned: by amavisd-new at postgresql.org X-Archive-Number: 200307/154 X-Sequence-Number: 2447 --=-2WPcwRj2P0eDr0K6ZKFz Content-Type: text/plain Content-Transfer-Encoding: quoted-printable > To give you some perspective on the size of the dataset and the > performance level we are hitting, here are some "good" results based on > some explains: Before Tom jumps in taking all the fun out of trying to solve it... The estimates in the slow queries seem perfectly reasonable. In fact, the cost estimates of both the slow and fast queries are the same which is what would be expected if all of the data was distributed evenly amongst the table. Given it's a date, I would guess that the data is generally inserted into the table in an order following the date but for some reason those 'high' dates have their data distributed more evenly amongst the table.=20 Clustered data will have fewer disk seeks and deal with fewer pages of information in general which makes for a much faster query. Distributed data will have to pull out significantly more information from the disk, throwing most of it away. I would guess that sometime on 2002-05-25 someone did a bit of data cleaning (deleting records). Next day the free space map had entries available in various locations within the table, and used them rather than appending to the end. With 89 Million records with date being significant, I'm guessing there aren't very many modifications or deletes on it. So.. How to solve the problem? If this is the type of query that occurs most often, you do primarily inserts, and the inserts are generally created following date, cluster the table by index "some_table_ix_0".=20 The clustering won't degrade very much since that is how you naturally insert the data. --=-2WPcwRj2P0eDr0K6ZKFz Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.2 (FreeBSD) iD8DBQA/DFQK6DETLow6vwwRAgZKAJ9HI3RaznYivCB+4IXKnXa5SbzqLwCfRZ4V hz7nN0f7Ps1DRNwWtZ0XR9I= =XEZh -----END PGP SIGNATURE----- --=-2WPcwRj2P0eDr0K6ZKFz--