pg.ddx.io pgsql-performance@postgresql.org mailing list archive
help / color / mirror / Atom feedRe: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ...
12+ messages / 4 participants
[nested] [flat]
* Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ...
@ 2004-10-15 07:55 Simon Riggs <simon@2ndquadrant.com>
2004-10-15 16:48 ` Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Tom Lane <tgl@sss.pgh.pa.us>
2004-10-18 21:00 ` Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Josh Berkus <josh@agliodbs.com>
0 siblings, 2 replies; 12+ messages in thread
From: Simon Riggs @ 2004-10-15 07:55 UTC (permalink / raw)
To: Timothy D. Witham <wookie@osdl.org>; josh@agliodbs.com; +Cc: pgsql-performance; testperf-general@pgfoundry.org
>Timothy D. Witham
> On Thu, 2004-10-14 at 16:57 -0700, Josh Berkus wrote:
> > Simon,
> >
> > <lots of good stuff clipped>
> >
> > > If you draw a graph of speedup (y) against cache size as a
> > > % of total database size, the graph looks like an upside-down
> "L" - i.e.
> > > the graph rises steeply as you give it more memory, then
> turns sharply at a
> > > particular point, after which it flattens out. The "turning
> point" is the
> > > "sweet spot" we all seek - the optimum amount of cache memory
> to allocate -
> > > but this spot depends upon the worklaod and database size,
> not on available
> > > RAM on the system under test.
> >
> > Hmmm ... how do you explain, then the "camel hump" nature of the real
> > performance? That is, when we allocated even a few MB more than the
> > "optimum" ~190MB, overall performance stated to drop quickly.
> The result is
> > that allocating 2x optimum RAM is nearly as bad as allocating
> too little
> > (e.g. 8MB).
Two ways of explaining this:
1. Once you've hit the optimum size of shared_buffers, you may not yet have
hit the optimum size of the OS cache. If that is true, every extra block
given to shared_buffers is wasted, yet detracts from the beneficial effect
of the OS cache. I don't see how the small drop in size of the OS cache
could have the effect you have measured, so I suggest that this possible
explanation doesn't fit the results well.
2. There is some algorithmic effect within PostgreSQL that makes larger
shared_buffers much worse than smaller ones. Imagine that each extra block
we hold in cache has the positive benefit from caching, minus a postulated
negative drag effect. With that model we would get: Once the optimal size of
the cache has been reached the positive benefit tails off to almost zero and
we are just left with the situation that each new block added to
shared_buffers acts as a further drag on performance. That model would fit
the results, so we can begin to look at what the drag effect might be.
Speculating wildly because I don't know that portion of the code this might
be:
CONJECTURE 1: the act of searching for a block in cache is an O(n)
operation, not an O(1) or O(log n) operation - so searching a larger cache
has an additional slowing effect on the application, via a buffer cache lock
that is held while the cache is searched - larger caches are locked for
longer than smaller caches, so this causes additional contention in the
system, which then slows down performance.
The effect might show up by examining the oprofile results for the test
cases. What we would be looking for is something that is being called more
frequently with larger shared_buffers - this could be anything....but my
guess is the oprofile results won't be similar and could lead us to a better
understanding.
> >
> > The only explanation I've heard of this so far is that there is
> a significant
> > loss of efficiency with larger caches. Or do you see the loss
> of 200MB out
> > of 3500MB would actually affect the Kernel cache that much?
> >
> In a past life there seemed to be a sweet spot around the
> applications
> working set. Performance went up until you got just a little larger
> than
> the cache needed to hold the working set and then went down. Most of
> the time a nice looking hump. It seems to have to do with the
> additional pages
> not increasing your hit ratio but increasing the amount of work to get a
> hit in cache. This seemed to be independent of the actual database
> software being used. (I observed this running Oracle, Informix, Sybase
> and Ingres.)
Good, our experiences seems to be similar.
>
> > Anyway, one test of your theory that I can run immediately is
> to run the exact
> > same workload on a bigger, faster server and see if the desired
> quantity of
> > shared_buffers is roughly the same.
I agree that you could test this by running on a bigger or smaller server,
i.e. one with more or less RAM. Running on a faster/slower server at the
same time might alter the results and confuse the situation.
> I'm hoping that you're wrong -- not
> because I don't find your argument persuasive, but because if
> you're right it
> > leaves us without any reasonable ability to recommend
> shared_buffer settings.
>
For the record, what I think we need is dynamically resizable
shared_buffers, not a-priori knowledge of what you should set shared_buffers
to. I've been thinking about implementing a scheme that helps you decide how
big the shared_buffers SHOULD BE, by making the LRU list bigger than the
cache itself, so you'd be able to see whether there is beneficial effect in
increasing shared_buffers.
...remember that this applies to other databases too, and with those we find
that they have dynamically resizable memory.
Having said all that, there are still a great many other performance tests
to run so that we CAN recommend other settings, such as the optimizer cost
parameters, bg writer defaults etc.
Best Regards,
Simon Riggs
2nd Quadrant
^ permalink raw reply [nested|flat] 12+ messages in thread
* Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ...
2004-10-15 07:55 Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Simon Riggs <simon@2ndquadrant.com>
@ 2004-10-15 16:48 ` Tom Lane <tgl@sss.pgh.pa.us>
2004-10-15 20:13 ` Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Josh Berkus <josh@agliodbs.com>
1 sibling, 1 reply; 12+ messages in thread
From: Tom Lane @ 2004-10-15 16:48 UTC (permalink / raw)
To: Simon Riggs <simon@2ndquadrant.com>; +Cc: Timothy D. Witham <wookie@osdl.org>; josh@agliodbs.com; pgsql-performance; testperf-general@pgfoundry.org
"Simon Riggs" <simon@2ndquadrant.com> writes:
> Speculating wildly because I don't know that portion of the code this might
> be:
> CONJECTURE 1: the act of searching for a block in cache is an O(n)
> operation, not an O(1) or O(log n) operation
I'm not sure how this meme got into circulation, but I've seen a couple
of people recently either conjecturing or asserting that. Let me remind
people of the actual facts:
1. We use a hashtable to keep track of which blocks are currently in
shared buffers. Either a cache hit or a cache miss should be O(1),
because the hashtable size is scaled proportionally to shared_buffers,
and so the number of hash entries examined should remain constant.
2. There are some allegedly-not-performance-critical operations that do
scan through all the buffers, and therefore are O(N) in shared_buffers.
I just eyeballed all the latter, and came up with this list of O(N)
operations and their call points:
AtEOXact_Buffers
transaction commit or abort
UnlockBuffers
transaction abort, backend exit
StrategyDirtyBufferList
background writer's idle loop
FlushRelationBuffers
VACUUM
DROP TABLE, DROP INDEX
TRUNCATE, CLUSTER, REINDEX
ALTER TABLE SET TABLESPACE
DropRelFileNodeBuffers
TRUNCATE (only for ON COMMIT TRUNC temp tables)
REINDEX (inplace case only)
smgr_internal_unlink (ie, the tail end of DROP TABLE/INDEX)
DropBuffers
DROP DATABASE
The fact that the first two are called during transaction commit/abort
is mildly alarming. The constant factors are going to be very tiny
though, because what these routines actually do is scan backend-local
status arrays looking for locked buffers, which they're not going to
find very many of. For instance AtEOXact_Buffers looks like
int i;
for (i = 0; i < NBuffers; i++)
{
if (PrivateRefCount[i] != 0)
{
// some code that should never be executed at all in the commit
// case, and not that much in the abort case either
}
}
I suppose with hundreds of thousands of shared buffers this might get to
the point of being noticeable, but I've never seen it show up at all in
profiling with more-normal buffer counts. Not sure if it's worth
devising a more complex data structure to aid in finding locked buffers.
(To some extent this code is intended to be belt-and-suspenders stuff
for catching omissions elsewhere, and so a more complex data structure
that could have its own bugs is not especially attractive.)
The one that's bothering me at the moment is StrategyDirtyBufferList,
which is a new overhead in 8.0. It wouldn't directly affect foreground
query performance, but indirectly it would hurt by causing the bgwriter
to suck more CPU cycles than one would like (and it holds the BufMgrLock
while it's doing it, too :-(). One easy way you could see whether this
is an issue in the OSDL test is to see what happens if you double all
three bgwriter parameters (delay, percent, maxpages). This should
result in about the same net I/O demand from the bgwriter, but
StrategyDirtyBufferList will be executed half as often.
I doubt that the other ones are issues. We could improve them by
devising a way to quickly find all buffers for a given relation, but
I am just about sure that complicating the buffer management to do so
would be a net loss for normal workloads.
> For the record, what I think we need is dynamically resizable
> shared_buffers, not a-priori knowledge of what you should set
> shared_buffers to.
This isn't likely to happen because the SysV shared memory API isn't
conducive to it. Absent some amazingly convincing demonstration that
we have to have it, the effort of making it happen in a portable way
isn't going to get spent.
> I've been thinking about implementing a scheme that helps you decide how
> big the shared_buffers SHOULD BE, by making the LRU list bigger than the
> cache itself, so you'd be able to see whether there is beneficial effect in
> increasing shared_buffers.
ARC already keeps such a list --- couldn't you learn what you want to
know from the existing data structure? It'd be fairly cool if we could
put out warnings "you ought to increase shared_buffers" analogous to the
existing facility for noting excessive checkpointing.
regards, tom lane
^ permalink raw reply [nested|flat] 12+ messages in thread
* Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ...
2004-10-15 07:55 Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Simon Riggs <simon@2ndquadrant.com>
2004-10-15 16:48 ` Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Tom Lane <tgl@sss.pgh.pa.us>
@ 2004-10-15 20:13 ` Josh Berkus <josh@agliodbs.com>
2004-10-15 20:34 ` Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 12+ messages in thread
From: Josh Berkus @ 2004-10-15 20:13 UTC (permalink / raw)
To: pgsql-performance; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Simon Riggs <simon@2ndquadrant.com>; Timothy D. Witham <wookie@osdl.org>; testperf-general@pgfoundry.org
Tom, Simon:
First off, two test runs with OProfile are available at:
http://khack.osdl.org/stp/298124/
http://khack.osdl.org/stp/298121/
> AtEOXact_Buffers
> transaction commit or abort
> UnlockBuffers
> transaction abort, backend exit
Actually, this might explain the "hump" shape of the curve for this test.
DBT2 is an OLTP test, which means that (at this scale level) it's attempting
to do approximately 30 COMMITs per second as well as one ROLLBACK every 3
seconds. When I get the tests on DBT3 running, if we see a more gentle
dropoff on overallocated memory, it would indicate that the above may be a
factor.
--
--Josh
Josh Berkus
Aglio Database Solutions
San Francisco
^ permalink raw reply [nested|flat] 12+ messages in thread
* Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ...
2004-10-15 07:55 Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Simon Riggs <simon@2ndquadrant.com>
2004-10-15 16:48 ` Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Tom Lane <tgl@sss.pgh.pa.us>
2004-10-15 20:13 ` Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Josh Berkus <josh@agliodbs.com>
@ 2004-10-15 20:34 ` Tom Lane <tgl@sss.pgh.pa.us>
2004-10-15 20:38 ` Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Josh Berkus <josh@agliodbs.com>
0 siblings, 1 reply; 12+ messages in thread
From: Tom Lane @ 2004-10-15 20:34 UTC (permalink / raw)
To: josh@agliodbs.com; +Cc: pgsql-performance@postgresql.org, "Simon Riggs" <simon@2ndquadrant.com>; Timothy D. Witham <wookie@osdl.org>; testperf-general@pgfoundry.org
Josh Berkus <josh@agliodbs.com> writes:
> First off, two test runs with OProfile are available at:
> http://khack.osdl.org/stp/298124/
> http://khack.osdl.org/stp/298121/
Hmm. The stuff above 1% in the first of these is
Counted CPU_CLK_UNHALTED events (clocks processor is not halted) with a unit mask of 0x00 (No unit mask) count 100000
samples % app name symbol name
8522858 19.7539 vmlinux default_idle
3510225 8.1359 vmlinux recalc_sigpending_tsk
1874601 4.3449 vmlinux .text.lock.signal
1653816 3.8331 postgres SearchCatCache
1080908 2.5053 postgres AllocSetAlloc
920369 2.1332 postgres AtEOXact_Buffers
806218 1.8686 postgres OpernameGetCandidates
803125 1.8614 postgres StrategyDirtyBufferList
746123 1.7293 vmlinux __copy_from_user_ll
651978 1.5111 vmlinux __copy_to_user_ll
640511 1.4845 postgres XLogInsert
630797 1.4620 vmlinux rm_from_queue
607833 1.4088 vmlinux next_thread
436682 1.0121 postgres LWLockAcquire
419672 0.9727 postgres yyparse
In the second test AtEOXact_Buffers is much lower (down around 0.57
percent) but the other suspects are similar. Since the only difference
in parameters is shared_buffers (36000 vs 9000), it does look like we
are approaching the point where AtEOXact_Buffers is a problem, but so
far it's only a 2% drag.
I suspect the reason recalc_sigpending_tsk is so high is that the
original coding of PG_TRY involved saving and restoring the signal mask,
which led to a whole lot of sigsetmask-type kernel calls. Is this test
with beta3, or something older?
Another interesting item here is the costs of __copy_from_user_ll/
__copy_to_user_ll:
36000 buffers:
746123 1.7293 vmlinux __copy_from_user_ll
651978 1.5111 vmlinux __copy_to_user_ll
9000 buffers:
866414 2.0810 vmlinux __copy_from_user_ll
852620 2.0479 vmlinux __copy_to_user_ll
Presumably the higher costs for 9000 buffers reflect an increased amount
of shuffling of data between kernel and user space. So 36000 is not
enough to make the working set totally memory-resident, but even if we
drove this cost to zero we'd only be buying a couple percent.
regards, tom lane
^ permalink raw reply [nested|flat] 12+ messages in thread
* Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ...
2004-10-15 07:55 Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Simon Riggs <simon@2ndquadrant.com>
2004-10-15 16:48 ` Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Tom Lane <tgl@sss.pgh.pa.us>
2004-10-15 20:13 ` Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Josh Berkus <josh@agliodbs.com>
2004-10-15 20:34 ` Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Tom Lane <tgl@sss.pgh.pa.us>
@ 2004-10-15 20:38 ` Josh Berkus <josh@agliodbs.com>
2004-10-15 21:27 ` Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 12+ messages in thread
From: Josh Berkus @ 2004-10-15 20:38 UTC (permalink / raw)
To: pgsql-performance; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Simon Riggs <simon@2ndquadrant.com>; Timothy D. Witham <wookie@osdl.org>; testperf-general@pgfoundry.org
Tom,
> I suspect the reason recalc_sigpending_tsk is so high is that the
> original coding of PG_TRY involved saving and restoring the signal mask,
> which led to a whole lot of sigsetmask-type kernel calls. Is this test
> with beta3, or something older?
Beta3, *without* Gavin or Neil's Futex patch.
--
--Josh
Josh Berkus
Aglio Database Solutions
San Francisco
^ permalink raw reply [nested|flat] 12+ messages in thread
* Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ...
2004-10-15 07:55 Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Simon Riggs <simon@2ndquadrant.com>
2004-10-15 16:48 ` Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Tom Lane <tgl@sss.pgh.pa.us>
2004-10-15 20:13 ` Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Josh Berkus <josh@agliodbs.com>
2004-10-15 20:34 ` Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Tom Lane <tgl@sss.pgh.pa.us>
2004-10-15 20:38 ` Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Josh Berkus <josh@agliodbs.com>
@ 2004-10-15 21:27 ` Tom Lane <tgl@sss.pgh.pa.us>
2004-10-15 21:32 ` Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Mark Wong <markw@osdl.org>
0 siblings, 1 reply; 12+ messages in thread
From: Tom Lane @ 2004-10-15 21:27 UTC (permalink / raw)
To: josh@agliodbs.com; +Cc: pgsql-performance@postgresql.org, "Simon Riggs" <simon@2ndquadrant.com>; Timothy D. Witham <wookie@osdl.org>; testperf-general@pgfoundry.org
Josh Berkus <josh@agliodbs.com> writes:
>> I suspect the reason recalc_sigpending_tsk is so high is that the
>> original coding of PG_TRY involved saving and restoring the signal mask,
>> which led to a whole lot of sigsetmask-type kernel calls. Is this test
>> with beta3, or something older?
> Beta3, *without* Gavin or Neil's Futex patch.
Hmm, in that case the cost deserves some further investigation. Can we
find out just what that routine does and where it's being called from?
regards, tom lane
^ permalink raw reply [nested|flat] 12+ messages in thread
* Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ...
2004-10-15 07:55 Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Simon Riggs <simon@2ndquadrant.com>
2004-10-15 16:48 ` Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Tom Lane <tgl@sss.pgh.pa.us>
2004-10-15 20:13 ` Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Josh Berkus <josh@agliodbs.com>
2004-10-15 20:34 ` Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Tom Lane <tgl@sss.pgh.pa.us>
2004-10-15 20:38 ` Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Josh Berkus <josh@agliodbs.com>
2004-10-15 21:27 ` Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Tom Lane <tgl@sss.pgh.pa.us>
@ 2004-10-15 21:32 ` Mark Wong <markw@osdl.org>
2004-10-15 21:44 ` Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 12+ messages in thread
From: Mark Wong @ 2004-10-15 21:32 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: josh@agliodbs.com, Simon Riggs <simon@2ndquadrant.com>; testperf-general@pgfoundry.org; pgsql-performance
On Fri, Oct 15, 2004 at 05:27:29PM -0400, Tom Lane wrote:
> Josh Berkus <josh@agliodbs.com> writes:
> >> I suspect the reason recalc_sigpending_tsk is so high is that the
> >> original coding of PG_TRY involved saving and restoring the signal mask,
> >> which led to a whole lot of sigsetmask-type kernel calls. Is this test
> >> with beta3, or something older?
>
> > Beta3, *without* Gavin or Neil's Futex patch.
>
> Hmm, in that case the cost deserves some further investigation. Can we
> find out just what that routine does and where it's being called from?
>
There's a call-graph feature with oprofile as of version 0.8 with
the opstack tool, but I'm having a terrible time figuring out why the
output isn't doing the graphing part. Otherwise, I'd have that
available already...
Mark
^ permalink raw reply [nested|flat] 12+ messages in thread
* Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ...
2004-10-15 07:55 Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Simon Riggs <simon@2ndquadrant.com>
2004-10-15 16:48 ` Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Tom Lane <tgl@sss.pgh.pa.us>
2004-10-15 20:13 ` Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Josh Berkus <josh@agliodbs.com>
2004-10-15 20:34 ` Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Tom Lane <tgl@sss.pgh.pa.us>
2004-10-15 20:38 ` Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Josh Berkus <josh@agliodbs.com>
2004-10-15 21:27 ` Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Tom Lane <tgl@sss.pgh.pa.us>
2004-10-15 21:32 ` Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Mark Wong <markw@osdl.org>
@ 2004-10-15 21:44 ` Tom Lane <tgl@sss.pgh.pa.us>
2004-10-15 22:10 ` Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Mark Wong <markw@osdl.org>
0 siblings, 1 reply; 12+ messages in thread
From: Tom Lane @ 2004-10-15 21:44 UTC (permalink / raw)
To: Mark Wong <markw@osdl.org>; +Cc: josh@agliodbs.com, Simon Riggs <simon@2ndquadrant.com>; testperf-general@pgfoundry.org; pgsql-performance
Mark Wong <markw@osdl.org> writes:
> On Fri, Oct 15, 2004 at 05:27:29PM -0400, Tom Lane wrote:
>> Hmm, in that case the cost deserves some further investigation. Can we
>> find out just what that routine does and where it's being called from?
> There's a call-graph feature with oprofile as of version 0.8 with
> the opstack tool, but I'm having a terrible time figuring out why the
> output isn't doing the graphing part. Otherwise, I'd have that
> available already...
I was wondering if this might be associated with do_sigaction.
do_sigaction is only 0.23 percent of the runtime according to the
oprofile results:
http://khack.osdl.org/stp/298124/oprofile/DBT_2_Profile-all.oprofile.txt
but the profile results for the same run:
http://khack.osdl.org/stp/298124/profile/DBT_2_Profile-tick.sort
show do_sigaction very high and recalc_sigpending_tsk nowhere at all.
Something funny there.
regards, tom lane
^ permalink raw reply [nested|flat] 12+ messages in thread
* Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ...
2004-10-15 07:55 Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Simon Riggs <simon@2ndquadrant.com>
2004-10-15 16:48 ` Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Tom Lane <tgl@sss.pgh.pa.us>
2004-10-15 20:13 ` Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Josh Berkus <josh@agliodbs.com>
2004-10-15 20:34 ` Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Tom Lane <tgl@sss.pgh.pa.us>
2004-10-15 20:38 ` Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Josh Berkus <josh@agliodbs.com>
2004-10-15 21:27 ` Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Tom Lane <tgl@sss.pgh.pa.us>
2004-10-15 21:32 ` Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Mark Wong <markw@osdl.org>
2004-10-15 21:44 ` Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Tom Lane <tgl@sss.pgh.pa.us>
@ 2004-10-15 22:10 ` Mark Wong <markw@osdl.org>
0 siblings, 0 replies; 12+ messages in thread
From: Mark Wong @ 2004-10-15 22:10 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: josh@agliodbs.com, Simon Riggs <simon@2ndquadrant.com>; testperf-general@pgfoundry.org; pgsql-performance
On Fri, Oct 15, 2004 at 05:44:34PM -0400, Tom Lane wrote:
> Mark Wong <markw@osdl.org> writes:
> > On Fri, Oct 15, 2004 at 05:27:29PM -0400, Tom Lane wrote:
> >> Hmm, in that case the cost deserves some further investigation. Can we
> >> find out just what that routine does and where it's being called from?
>
> > There's a call-graph feature with oprofile as of version 0.8 with
> > the opstack tool, but I'm having a terrible time figuring out why the
> > output isn't doing the graphing part. Otherwise, I'd have that
> > available already...
>
> I was wondering if this might be associated with do_sigaction.
> do_sigaction is only 0.23 percent of the runtime according to the
> oprofile results:
> http://khack.osdl.org/stp/298124/oprofile/DBT_2_Profile-all.oprofile.txt
> but the profile results for the same run:
> http://khack.osdl.org/stp/298124/profile/DBT_2_Profile-tick.sort
> show do_sigaction very high and recalc_sigpending_tsk nowhere at all.
> Something funny there.
>
I have always attributed those kind of differences based on how
readprofile and oprofile collect their data. Granted I don't exactly
understand it. Anyone familiar with the two differences?
Mark
^ permalink raw reply [nested|flat] 12+ messages in thread
* Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ...
2004-10-15 07:55 Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Simon Riggs <simon@2ndquadrant.com>
@ 2004-10-18 21:00 ` Josh Berkus <josh@agliodbs.com>
1 sibling, 0 replies; 12+ messages in thread
From: Josh Berkus @ 2004-10-18 21:00 UTC (permalink / raw)
To: pgsql-performance
Simon,
> I agree that you could test this by running on a bigger or smaller server,
> i.e. one with more or less RAM. Running on a faster/slower server at the
> same time might alter the results and confuse the situation.
Unfortunately, a faster server is the only option I have that also has more
RAM. If I double the RAM and double the processors at the same time, what
would you expect to happen to the shared_buffers curve?
--
--Josh
Josh Berkus
Aglio Database Solutions
San Francisco
^ permalink raw reply [nested|flat] 12+ messages in thread
* Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ...
@ 2004-10-15 19:42 Simon Riggs <simon@2ndquadrant.com>
2004-10-15 19:44 ` Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Josh Berkus <josh@agliodbs.com>
0 siblings, 1 reply; 12+ messages in thread
From: Simon Riggs @ 2004-10-15 19:42 UTC (permalink / raw)
To: Bruce Momjian <pgman@candle.pha.pa.us>; Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Timothy D. Witham <wookie@osdl.org>; josh@agliodbs.com; pgsql-performance; testperf-general@pgfoundry.org
> Bruce Momjian
> Tom Lane wrote:
> > > I've been thinking about implementing a scheme that helps you
> decide how
> > > big the shared_buffers SHOULD BE, by making the LRU list
> bigger than the
> > > cache itself, so you'd be able to see whether there is
> beneficial effect in
> > > increasing shared_buffers.
> >
> > ARC already keeps such a list --- couldn't you learn what you want to
> > know from the existing data structure? It'd be fairly cool if we could
> > put out warnings "you ought to increase shared_buffers" analogous to the
> > existing facility for noting excessive checkpointing.
First off, many thanks for taking the time to provide the real detail on the
code.
That gives us some much needed direction in interpreting the oprofile
output.
>
> Agreed. ARC already keeps a list of buffers it had to push out recently
> so if it needs them again soon it knows its sizing of recent/frequent
> might be off (I think). Anyway, such a log report would be super-cool,
> say if you pushed out a buffer and needed it very soon, and the ARC
> buffers are already at their maximum for that buffer pool.
>
OK, I guess I hadn't realised we were half-way there.
The "increase shared_buffers" warning would be useful, but it would be much
cooler to have some guidance as to how big to set it, especially since this
requires a restart of the server.
What I had in mind was a way of keeping track of how the buffer cache hit
ratio would look at various sizes of shared_buffers, for example 50%, 80%,
120%, 150%, 200% and 400% say. That way you'd stand a chance of plotting the
curve and thereby assessing how much memory could be allocated. I've got a
few ideas, but I need to check out the code first.
I'll investigate both simple/complex options as an 8.1 feature.
Best Regards, Simon Riggs
^ permalink raw reply [nested|flat] 12+ messages in thread
* Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ...
2004-10-15 19:42 Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Simon Riggs <simon@2ndquadrant.com>
@ 2004-10-15 19:44 ` Josh Berkus <josh@agliodbs.com>
0 siblings, 0 replies; 12+ messages in thread
From: Josh Berkus @ 2004-10-15 19:44 UTC (permalink / raw)
To: pgsql-performance; +Cc: Simon Riggs <simon@2ndquadrant.com>; Bruce Momjian <pgman@candle.pha.pa.us>; Tom Lane <tgl@sss.pgh.pa.us>; Timothy D. Witham <wookie@osdl.org>; testperf-general@pgfoundry.org
People:
> First off, many thanks for taking the time to provide the real detail on
> the code.
>
> That gives us some much needed direction in interpreting the oprofile
> output.
I have some oProfile output; however, it's in 2 out of 20 tests I ran recently
and I need to get them sorted out.
--Josh
--
--Josh
Josh Berkus
Aglio Database Solutions
San Francisco
^ permalink raw reply [nested|flat] 12+ messages in thread
end of thread, other threads:[~2004-10-18 21:00 UTC | newest]
Thread overview: 12+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2004-10-15 07:55 Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Simon Riggs <simon@2ndquadrant.com>
2004-10-15 16:48 ` Tom Lane <tgl@sss.pgh.pa.us>
2004-10-15 20:13 ` Josh Berkus <josh@agliodbs.com>
2004-10-15 20:34 ` Tom Lane <tgl@sss.pgh.pa.us>
2004-10-15 20:38 ` Josh Berkus <josh@agliodbs.com>
2004-10-15 21:27 ` Tom Lane <tgl@sss.pgh.pa.us>
2004-10-15 21:32 ` Mark Wong <markw@osdl.org>
2004-10-15 21:44 ` Tom Lane <tgl@sss.pgh.pa.us>
2004-10-15 22:10 ` Mark Wong <markw@osdl.org>
2004-10-18 21:00 ` Josh Berkus <josh@agliodbs.com>
2004-10-15 19:42 Re: [Testperf-general] Re: First set of OSDL Shared Memscalability results, some wierdness ... Simon Riggs <simon@2ndquadrant.com>
2004-10-15 19:44 ` Josh Berkus <josh@agliodbs.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox