From: Josh Berkus <josh@agliodbs.com>
To: Dan Ports <drkp@csail.mit.edu>
Cc: pgsql-performance@postgresql.org
Subject: Re: [HACKERS] MIT benchmarks pgsql multicore (up to 48)performance
Date: Mon, 04 Oct 2010 11:49:43 -0700
Message-ID: <4CAA21C7.1030701@agliodbs.com> (raw)
In-Reply-To: <20101004175545.GA2690@csail.mit.edu>
References: <AANLkTimW2UgVPGw6MRUBj9HabvwmsmkZasL3hKv-nUJ2@mail.gmail.com>
<20101004175545.GA2690@csail.mit.edu>
Dan,
(btw, OpenSQL Confererence is going to be at MIT in 2 weeks. Think
anyone from the MOSBENCH team could attend?
http://www.opensqlcamp.org/Main_Page)
> The big takeaway for -hackers, I think, is that lock manager
> performance is going to be an issue for large multicore systems, and
> the uncontended cases need to be lock-free. That includes cases where
> multiple threads are trying to acquire the same lock in compatible
> modes.
Yes; we were aware of this due to work Jignesh did at Sun on TPC-E.
> Currently even acquiring a shared heavyweight lock requires taking out
> an exclusive LWLock on the partition, and acquiring shared LWLocks
> requires acquiring a spinlock. All of this gets more expensive on
> multicores, where even acquiring spinlocks can take longer than the
> work being done in the critical section.
Certainly, the question has always been how to fix it without breaking
major features and endangering data integrity.
> Note that their implementation of the lock manager omits some features
> for simplicity, like deadlock detection, 2PC, and probably any
> semblance of portability. (These are the sort of things we're allowed
> to do in the research world! :-)
Well, nice that you did! We'd never have that much time to experiment
with non-production stuff as a group in the project. So, now we have a
theoretical solution which we can look at maybe implementing parts of in
some watered-down form.
> The other major bottleneck they ran into was a kernel one: reading from
> the heap file requires a couple lseek operations, and Linux acquires a
> mutex on the inode to do that. The proper place to fix this is
> certainly in the kernel but it may be possible to work around in
> Postgres.
Or we could complain to Kernel.org. They've been fairly responsive in
the past. Too bad this didn't get posted earlier; I just got back from
LinuxCon.
So you know someone who can speak technically to this issue? I can put
them in touch with the Linux geeks in charge of that part of the kernel
code.
--
-- Josh Berkus
PostgreSQL Experts Inc.
http://www.pgexperts.com
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: josh@agliodbs.com, drkp@csail.mit.edu
Subject: Re: [HACKERS] MIT benchmarks pgsql multicore (up to 48)performance
In-Reply-To: <4CAA21C7.1030701@agliodbs.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox