pg.ddx.io pgsql-performance@postgresql.org mailing list archivehelp / color / mirror / Atom feed
From: Guy Thornley <guy@esphion.com>
To: Jeffrey W. Baker <jwbaker@acm.org>
Cc: Josh Berkus <josh@agliodbs.com>
Cc: pgsql-performance@postgresql.org, Steve Poe <spoe@sfnet.cc>
Cc: Chris Browne <cbbrowne@acm.org>
Subject: Re: Read/Write block sizes
Date: Wed, 24 Aug 2005 17:20:23 +1200
Message-ID: <20050824052023.GB31806@conker.esphion.com> (raw)
In-Reply-To: <1124852854.11147.5.camel@noodles>
References: <d4b67d7c2b.d7c2bd4b67@bur-mail1.east.sun.com>
<60k6icpapm.fsf@dba2.int.libertyrms.com>
<1124846743.12045.94.camel@amd64-laptop-spoe>
<200508231931.29682.josh@agliodbs.com>
<1124852854.11147.5.camel@noodles>
> Unfortunately I'm really afraid that this conversation is about trees
> when the forest is the problem. PostgreSQL doesn't even have an async
> reader, which is the sort of thing that could double or triple its
> performance. You're talking about block sizes and such, but the kinds
> of improvements you can get there are in the tens of percents at most.
Not 100% sure, but I'm fairly cirtain we were seeing significant performance
degradation by too much _scheduled_ I/O activity
ie: too much work being submitted to the kernel, due to excessive
parallelism already!!
The classic example of this is a seqscan being interleved by a index scan,
and the disks end up doing nothing but seek activity
Out of all the stuff talked about on this thread so far, only tweaking the
block size (and the madvise() stuff) makes any real-world sense, as its the
only thing talked about that increases the _work_per_seek_.
As for the async IO, sure you might think 'oh async IO would be so cool!!'
and I did, once, too. But then I sat down and _thought_ about it, and
decided well, no, actually, theres _very_ few areas it could actually help,
and in most cases it just make it easier to drive your box into lseek()
induced IO collapse.
Dont forget that already in postgres, you have a process per connection, and
all the processes take care of their own I/O.
Somebody mentioned having threaded backends too, but the only benefit would
be reduced memory footprint (a backend consumes 1-2MB of RAM, which is
almost enough to be a concern for largish systems with a lot of backends)
but personally I _know_ the complixities introduced through threading are
usually not worth it.
IMVVHO (naive experience) what is needed is a complete architecture change
(probably infeasible and only useful as a thought experiment), where:
* a network I/O process deals with client connections
* a limited pool of worker processes deal with statements (perhaps related
to number of spindles somehow)
so when a client issues a statement, the net-IO process simply forwards the
connection state to a worker process and says 'deal with this'.
(Clearly the state object needs to contain all user and transaction state
the connection is involved in).
- Guy Thornley
view thread (24+ messages) latest in thread
Message-ID: <20050824052023.GB31806@conker.esphion.com> Permalink: ../20050824052023.GB31806@conker.esphion.com/ Also on: postgresql.org/message-id/20050824052023.GB31806@conker.esphion.com ·
reply
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Reply to all the recipients using the --to and --cc options: reply via email To: pgsql-performance@postgresql.org Cc: guy@esphion.com, jwbaker@acm.org, josh@agliodbs.com, spoe@sfnet.cc, cbbrowne@acm.org Subject: Re: Read/Write block sizes In-Reply-To: <20050824052023.GB31806@conker.esphion.com> * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions for how to clone and mirror all data and code used for this inbox