Content-class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Subject: Re: Decreasing BLKSZ
Date: Mon, 25 Sep 2006 17:54:10 -0400
Message-ID: <2BCEB9A37A4D354AA276774EE13FB8C20113068D@mailserver.sandvine.com>
In-Reply-To: <451845EC.3020506@logix-tt.com>
Thread-Topic: [PERFORM] Decreasing BLKSZ
Thread-Index: Acbg5yAAmluj1qAhT+ilYQEeCFKP1AABPBDw
From: "Marc Morin" <marc@sandvine.com>
To: "Markus Schaber" <schabi@logix-tt.com>
Cc: <pgsql-performance@postgresql.org>

> Would it be possible to change the primary key to=20
> (logtime,key)? This could help keeping the "working window" small.

No, the application accessing the data wants all the rows between start
and end time for a particular key value. =20

>=20
> Secondly, the real working set is smaller, as the rows are=20
> all inserted at the end of the table, filling each page until=20
> it's full, so only the last pages are accessed. There's no=20
> relation between the index order, and the order of data on=20
> disk, unless you CLUSTER.

I'd theorizing that my problem is in updating the index itself and not
the heap.  Insert order
Refers to the order by which the applications are inserting the rows and
as such, the order by
Which the index is being updated.  This in turn, is causing the b-tree
to be traverse.  Problem
Is the working set of blocks at the bottom of the btree is too big for
my cache.

>=20
> > Any comment on other affects or gotchas with lowering the size of=20
> > BLKSZ?  Currently, our database is thrashing its cache of blocks we=20
> > we're getting only ~100 inserts/second, every insert results in a=20
> > evict-read-modify operation.
>=20
> I'm not shure that's the correct diagnosis.
>=20
> Do you have one transaction per insert? Every transaction=20
> means a forced sync to the disk, so you won't get more than=20
> about 100-200 commits per second, depending on your actual=20
> disk rotation speed.

No, an insert consists of roughly 10,000+ rows per transaction block. =20

>=20
> To improve concurrency of the "numer of inserters" running in=20
> parallel, try to tweak the config variables commit_delay and=20
> commit_sibling, so you get a higher overall throughput at=20
> cost of an increased delay per connection, and increase the=20
> number of inserters. Using sensible tweaking, the throughput=20
> should scale nearly linear with the number of backens. :-)

I don't think this will help us here due to large transactions already.

>=20
> If feasible for your application, you can also bundle several=20
> log entries into a single transaction. If you're CPU bound,=20
> you can use COPY instead of INSERT or (if you can wait for=20
> 8.2) the new multi-row INSERT to further improve performance,=20
> but I doubt that you're CPU bound.

>=20
> The only way to "really" get over the sync limit is to have=20
> (at least) the WAL on a battery backed ram / SSD media that=20
> has no "spinning disk"
> physical limit, or abandon crash safety by turning fsync off.

Again, problem is not with WAL writing, already on it's own raid1 disk
pair.  The=20
I/O pattern we see is about 1-2% load on WAL and 100% load on the array
holding the indexes and tables. Throughput is very low, something like
150k-200K bytes/second of real  rows being deposited on the disk.

The disks are busy seeking all over the disk platter to fetch a block,
add a single row, then seek to another spot and write back a previously
dirty buffer....

>=20
> Thanks,
> Markus.
> --
> Markus Schaber | Logical Tracking&Tracing International AG
> Dipl. Inf.     | Software Development GIS
>=20
> Fight against software patents in Europe! www.ffii.org=20
> www.nosoftwarepatents.org
>=20