Date: Sat, 8 Oct 2005 09:31:06 -0400
From: mark@mark.mielke.cc
To: Luke Lonergan <llonergan@greenplum.com>
Cc: "Steinar H. Gunderson" <sgunderson@bigfoot.com>,
	pgsql-performance@postgresql.org
Subject: Re: [HACKERS] A Better External Sort?
Message-ID: <20051008133106.GB23913@mark.mielke.cc>
References: <20051008001704.GA11677@mark.mielke.cc>
	<BF6C973B.10E2B%llonergan@greenplum.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <BF6C973B.10E2B%llonergan@greenplum.com>
User-Agent: Mutt/1.4.2.1i

On Fri, Oct 07, 2005 at 09:20:59PM -0700, Luke Lonergan wrote:
> On 10/7/05 5:17 PM, "mark@mark.mielke.cc" <mark@mark.mielke.cc> wrote:
> > On Fri, Oct 07, 2005 at 04:55:28PM -0700, Luke Lonergan wrote:
> >> On 10/5/05 5:12 PM, "Steinar H. Gunderson" <sgunderson@bigfoot.com> wrote:
> >>> What? strlen is definitely not in the kernel, and thus won't count as
> >>> system time.
> >> System time on Linux includes time spent in glibc routines.
> > Do you have a reference for this?
> > I believe this statement to be 100% false.
> How about 99%? OK, you're right, I had this confused with the profiling
> problem where glibc routines aren't included in dynamic linked profiles.

Sorry to emphasize the 100%. It wasn't meant to judge you. It was meant
to indicate that I believe 100% of system time is accounted for, while
the system call is actually active, which is not possible while glibc
is active.

I believe the way it works, is that a periodic timer interrupt
increments a specific integer every time it wakes up. If it finds
itself within the kernel, it increments the system time for the active
process, if it finds itself outside the kernel, it incremenets the
user time for the active process.

> Back to the statements earlier - the output of time had much of time for a
> dd spent in system, which means kernel, so where in the kernel would that be
> exactly?

Not really an expert here. I only play around. At a minimum, their is a
cost to switching from user context to system context and back, and then
filling in the zero bits. There may be other inefficiencies, however.
Perhaps /dev/zero always fill in a whole block (8192 usually), before
allowing the standard file system code to read only one byte.

I dunno.

But, I see this oddity too:

$ time dd if=/dev/zero of=/dev/zero bs=1 count=10000000
10000000+0 records in
10000000+0 records out
dd if=/dev/zero of=/dev/zero bs=1 count=10000000  4.05s user 11.13s system 94% cpu 16.061 total

$ time dd if=/dev/zero of=/dev/zero bs=10 count=1000000
1000000+0 records in
1000000+0 records out
dd if=/dev/zero of=/dev/zero bs=10 count=1000000  0.37s user 1.37s system 100% cpu 1.738 total