public inbox for [email protected]  

help / color / mirror / Atom feed

From: Markus Schiltknecht <[email protected]>
To: Gregory Stark <[email protected]>
Cc: Josh Berkus <[email protected]>
Cc: [email protected]
Subject: Re: VLDB Features
Date: Thu, 13 Dec 2007 16:21:04 +0100
Message-ID: <[email protected]> (raw)
In-Reply-To: <[email protected]>
References: <[email protected]>
	<[email protected]>
	<[email protected]>
	<[email protected]>
	<[email protected]>

Hello Gregory,

Gregory Stark wrote:
> Oracle is using Direct I/O so they need the reader and writer threads to avoid
> blocking on i/o all the time. We count on the OS doing readahead and buffering
> our writes so we don't have to. Direct I/O and needing some way to do
> asynchronous writes and reads are directly tied.

Yeah, except in cases where we can tell ahead non-sequential reads. 
Which admittedly doesn't come up too frequently and can probably be 
handled with posix_fadvice - as you are currently testing.

> Where Parallel query is useful is when you have queries that involve a
> substantial amount of cpu resources, especially if you have a very fast I/O
> system which can saturate the bandwidth to a single cpu.

Full ACK, the very same applies to parallel querying on shared-nothing 
clusters. Those can help if the bandwidth to all processing cores 
together becomes the bottleneck (and the resulting data is relatively 
small compared to the input data).

For example, Sun's UltraSparc T2 features only 8 PCIe lanes for those 8 
cores, so you end up with 250 MiB/sec per core or about 32 MiB/sec per 
thread on average. To be fair: their 10 Gig Ethernet ports don't go via 
PCIe, so you get an additional 2x 1 GiB/sec for the complete chip. And 
memory bandwidth looks a lot better: Sun claims 60+ GiB/sec, leaving 
almost 8 GiB/sec per core or 1 GiB/sec per thread.

If my calculations for Intel are correct, a Quad Xeon with a 1.33 GHz 
FSB has around 21 GiB/sec throughput to main memory, giving 5 GiB/sec 
per core. (Why are these numbers so hard to find? It looks like Intel 
deliberately obfuscates them with FSB MHz or Giga-transactions per sec 
and the like.)

Regards

Markus

view thread (39+ messages)  latest in thread

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected], [email protected]
  Subject: Re: VLDB Features
  In-Reply-To: <[email protected]>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox