pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  
help / color / mirror / Atom feed
Re: [osdldbt-general] Re: [GENERAL] how to get accurate
2+ messages / 2 participants
[nested] [flat]

* Re: [osdldbt-general] Re: [GENERAL] how to get accurate
@ 2003-09-11 00:17 Mary Edie Meredith <maryedie@osdl.org>
  2003-09-11 03:07 ` Re: [osdldbt-general] Re: [GENERAL] how to get accurate Christopher Browne <cbbrowne@acm.org>
  0 siblings, 1 reply; 2+ messages in thread

From: Mary Edie Meredith @ 2003-09-11 00:17 UTC (permalink / raw)
  To: Bruce Momjian <pgman@candle.pha.pa.us>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; pgsql-performance; osdldbt-general <osdldbt-general@lists.sourceforge.net>

We tried 1000 as the default and found that the plans were good plans
and were consistent, but the pg_statistics was not exactly the same.

We took Tom's' advice and tried SET SEED=0 (actually select setseed (0)
).

We did runs last night on our project machine which produced consistent
pg_statistics data and (of course) the same plans.

We will next try runs where we vary the default buckets.  Other than 10
and 1000, what numbers would you like us to try besides.  Previously the
number 100 was mentioned.  Are there others?

On Wed, 2003-09-10 at 12:44, Bruce Momjian wrote:
> Tom Lane wrote:
> > Mary Edie Meredith <maryedie@osdl.org> writes:
> > > Stephan Szabo kindly responded to our earlier queries suggesting we look
> > > at default_statistics_target and ALTER TABLE ALTER COLUMN SET
> > > STATISTICS.  
> > 
> > > These determine the number of bins in the histogram for a given column. 
> > > But for a large number of rows (for example 6 million) the maximum value
> > > (1000) does not guarantee that ANALYZE will do a full scan of the table.
> > > We do not see a way to guarantee the same statistics run to run without
> > > forcing ANALYZE to examine every row of every table.  
> > 
> > Do you actually still have a problem with the plans changing when the
> > stats target is above 100 or so?  I think the notion of "force ANALYZE
> > to do a full scan" is inherently wrongheaded ... it certainly would not
> > produce numbers that have anything to do with ordinary practice.
> > 
> > If you have data statistics that are so bizarre that the planner still
> > gets things wrong with a target of 1000, then I'd like to know more
> > about why.
> 
> Has there been any progress in determining if the number of default
> buckets (10) is the best value?
-- 
Mary Edie Meredith <maryedie@osdl.org>
Open Source Development Lab




^ permalink  raw  reply  [nested|flat] 2+ messages in thread

* Re: [osdldbt-general] Re: [GENERAL] how to get accurate
  2003-09-11 00:17 Re: [osdldbt-general] Re: [GENERAL] how to get accurate Mary Edie Meredith <maryedie@osdl.org>
@ 2003-09-11 03:07 ` Christopher Browne <cbbrowne@acm.org>
  0 siblings, 0 replies; 2+ messages in thread

From: Christopher Browne @ 2003-09-11 03:07 UTC (permalink / raw)
  To: pgsql-performance

The world rejoiced as maryedie@osdl.org (Mary Edie Meredith) wrote:
> We tried 1000 as the default and found that the plans were good
> plans and were consistent, but the pg_statistics was not exactly the
> same.
>
> We took Tom's' advice and tried SET SEED=0 (actually select setseed
> (0) ).

When you're trying to get strict replicability of results, setting the
seed to some specific value is necessary.

Some useful results could be attained by varying the seed, and seeing
how the plans change.

> We did runs last night on our project machine which produced
> consistent pg_statistics data and (of course) the same plans.

> We will next try runs where we vary the default buckets.  Other than
> 10 and 1000, what numbers would you like us to try besides.
> Previously the number 100 was mentioned.  Are there others?

That presumably depends on what your goal is.

A useful experiment would be to see at what point (e.g. - at what
bucket size) plans tend to "settle down" to the right values.

It might well be that defaulting to 23 buckets (I'm picking that out
of thin air) would cause the plans to typically be stable whatever
seed got used.

A test for this would be to, for each bucket size value, repeatedly
ANALYZE and check query plans.

At bucket size 10, you have seen the query plans vary quite a bit.

At 1000, they seem to stabilize very well.

The geometric centre, between 10 and 1000, is 100, so it would surely
be useful to see if query plans are stable at that bucket size.

The most interesting number to know would be the lowest number of
buckets at which query plans are nearly always stable.  Supposing that
number was 23 (the number I earlier pulled out of the air), then that
can be used as evidence that the default value for SET STATISTICS
should be changed from 10 to 23.
-- 
wm(X,Y):-write(X),write('@'),write(Y). wm('aa454','freenet.carleton.ca').
http://www3.sympatico.ca/cbbrowne/nonrdbms.html
Sturgeon's Law: 90% of *EVERYTHING* is crud.



^ permalink  raw  reply  [nested|flat] 2+ messages in thread


end of thread, other threads:[~2003-09-11 03:07 UTC | newest]

Thread overview: 2+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2003-09-11 00:17 Re: [osdldbt-general] Re: [GENERAL] how to get accurate Mary Edie Meredith <maryedie@osdl.org>
2003-09-11 03:07 ` Christopher Browne <cbbrowne@acm.org>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox