pg.ddx.io pgsql-performance@postgresql.org mailing list archive
help / color / mirror / Atom feedPerformance of ORDER BY
8+ messages / 6 participants
[nested] [flat]
* Performance of ORDER BY
@ 2006-12-05 17:12 Glenn Sullivan <glenn.sullivan@varianinc.com>
2006-12-05 17:36 ` Re: Performance of ORDER BY Luke Lonergan <llonergan@greenplum.com>
2006-12-05 18:02 ` Re: Performance of ORDER BY Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 2 replies; 8+ messages in thread
From: Glenn Sullivan @ 2006-12-05 17:12 UTC (permalink / raw)
To: pgsql-performance
I am wanting some ideas about improving the performance of ORDER BY in
our use. I have a DB on the order of 500,000 rows and 50 columns.
The results are always sorted with ORDER BY. Sometimes, the users end up
with a search that matches most of the rows. In that case, I have a
LIMIT 5000 to keep the returned results under control. However, the
sorting seems to take 10-60 sec. If I do the same search without the
ORDER BY, it takes about a second.
I am currently on version 8.0.1 on Windows XP using a Dell Optiplex 280
with 1Gb of ram. I have set sort_mem=100000 set.
Any ideas?
Thanks,
Glenn
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Performance of ORDER BY
2006-12-05 17:12 Performance of ORDER BY Glenn Sullivan <glenn.sullivan@varianinc.com>
@ 2006-12-05 17:36 ` Luke Lonergan <llonergan@greenplum.com>
1 sibling, 0 replies; 8+ messages in thread
From: Luke Lonergan @ 2006-12-05 17:36 UTC (permalink / raw)
To: Glenn Sullivan <glenn.sullivan@varianinc.com>; pgsql-performance
Glenn,
On 12/5/06 9:12 AM, "Glenn Sullivan" <glenn.sullivan@varianinc.com> wrote:
> I am wanting some ideas about improving the performance of ORDER BY in
> our use. I have a DB on the order of 500,000 rows and 50 columns.
> The results are always sorted with ORDER BY. Sometimes, the users end up
> with a search that matches most of the rows. In that case, I have a
> LIMIT 5000 to keep the returned results under control. However, the
> sorting seems to take 10-60 sec. If I do the same search without the
> ORDER BY, it takes about a second.
>
> I am currently on version 8.0.1 on Windows XP using a Dell Optiplex 280
> with 1Gb of ram. I have set sort_mem=100000 set.
>
> Any ideas?
Upgrade to 8.1 or 8.2, there were very large performance improvements to the
sort code made for 8.1+. Also, when you've upgraded, you can experiment
with increasing work_mem to get better performance. At some value of
work_mem (probably > 32MB) you will reach a plateau of performance, probably
4-5 times faster than what you see with 8.0.
- Luke
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Performance of ORDER BY
2006-12-05 17:12 Performance of ORDER BY Glenn Sullivan <glenn.sullivan@varianinc.com>
@ 2006-12-05 18:02 ` Tom Lane <tgl@sss.pgh.pa.us>
2006-12-05 18:17 ` Re: Performance of ORDER BY Steinar H. Gunderson <sgunderson@bigfoot.com>
2006-12-05 18:20 ` Re: Performance of ORDER BY A. Kretschmer <andreas.kretschmer@schollglas.com>
2006-12-05 20:42 ` Re: Performance of ORDER BY Glenn Sullivan <glenn.sullivan@varianinc.com>
1 sibling, 3 replies; 8+ messages in thread
From: Tom Lane @ 2006-12-05 18:02 UTC (permalink / raw)
To: Glenn Sullivan <glenn.sullivan@varianinc.com>; +Cc: pgsql-performance
Glenn Sullivan <glenn.sullivan@varianinc.com> writes:
> I am wanting some ideas about improving the performance of ORDER BY in
> our use. I have a DB on the order of 500,000 rows and 50 columns.
> The results are always sorted with ORDER BY. Sometimes, the users end up
> with a search that matches most of the rows. In that case, I have a
> LIMIT 5000 to keep the returned results under control. However, the
> sorting seems to take 10-60 sec. If I do the same search without the
> ORDER BY, it takes about a second.
Does the ORDER BY match an index? If so, is it using the index?
(See EXPLAIN.)
> I am currently on version 8.0.1 on Windows XP using a Dell Optiplex 280
> with 1Gb of ram. I have set sort_mem=100000 set.
In 8.0 that might be counterproductively high --- we have seen cases
where more sort_mem = slower with the older sorting code. I concur
with Luke's advice that you should update to 8.2 (not 8.1) to get the
improved sorting code.
regards, tom lane
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Performance of ORDER BY
2006-12-05 17:12 Performance of ORDER BY Glenn Sullivan <glenn.sullivan@varianinc.com>
2006-12-05 18:02 ` Re: Performance of ORDER BY Tom Lane <tgl@sss.pgh.pa.us>
@ 2006-12-05 18:17 ` Steinar H. Gunderson <sgunderson@bigfoot.com>
2006-12-05 18:39 ` Re: Performance of ORDER BY Tom Lane <tgl@sss.pgh.pa.us>
2006-12-05 19:55 ` Re: Performance of ORDER BY Stefan Kaltenbrunner <stefan@kaltenbrunner.cc>
2 siblings, 2 replies; 8+ messages in thread
From: Steinar H. Gunderson @ 2006-12-05 18:17 UTC (permalink / raw)
To: pgsql-performance
On Tue, Dec 05, 2006 at 01:02:06PM -0500, Tom Lane wrote:
> In 8.0 that might be counterproductively high --- we have seen cases
> where more sort_mem = slower with the older sorting code. I concur
> with Luke's advice that you should update to 8.2 (not 8.1) to get the
> improved sorting code.
By the way, is the new sorting code any better for platforms that already
have a decent qsort() (like Linux)?
/* Steinar */
--
Homepage: http://www.sesse.net/
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Performance of ORDER BY
2006-12-05 17:12 Performance of ORDER BY Glenn Sullivan <glenn.sullivan@varianinc.com>
2006-12-05 18:02 ` Re: Performance of ORDER BY Tom Lane <tgl@sss.pgh.pa.us>
2006-12-05 18:17 ` Re: Performance of ORDER BY Steinar H. Gunderson <sgunderson@bigfoot.com>
@ 2006-12-05 18:39 ` Tom Lane <tgl@sss.pgh.pa.us>
1 sibling, 0 replies; 8+ messages in thread
From: Tom Lane @ 2006-12-05 18:39 UTC (permalink / raw)
To: Steinar H. Gunderson <sgunderson@bigfoot.com>; +Cc: pgsql-performance
"Steinar H. Gunderson" <sgunderson@bigfoot.com> writes:
> By the way, is the new sorting code any better for platforms that already
> have a decent qsort() (like Linux)?
It seemed better to us. Linux' qsort() is really mergesort, which is
better sometimes but often worse --- mergesort tends to have a less
CPU-cache-friendly memory access distribution. Another big problem with
the Linux version is that it pays no attention to sort_mem, but will
enthusiastically allocate lots of additional memory, thereby blowing
whatever cross-backend memory budgeting you might have been doing.
If you care there is quite a lot of discussion in the -hackers and
-performance archives from last spring or so.
regards, tom lane
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Performance of ORDER BY
2006-12-05 17:12 Performance of ORDER BY Glenn Sullivan <glenn.sullivan@varianinc.com>
2006-12-05 18:02 ` Re: Performance of ORDER BY Tom Lane <tgl@sss.pgh.pa.us>
2006-12-05 18:17 ` Re: Performance of ORDER BY Steinar H. Gunderson <sgunderson@bigfoot.com>
@ 2006-12-05 19:55 ` Stefan Kaltenbrunner <stefan@kaltenbrunner.cc>
1 sibling, 0 replies; 8+ messages in thread
From: Stefan Kaltenbrunner @ 2006-12-05 19:55 UTC (permalink / raw)
To: sgunderson@bigfoot.com; +Cc: pgsql-performance
Steinar H. Gunderson wrote:
> On Tue, Dec 05, 2006 at 01:02:06PM -0500, Tom Lane wrote:
>> In 8.0 that might be counterproductively high --- we have seen cases
>> where more sort_mem = slower with the older sorting code. I concur
>> with Luke's advice that you should update to 8.2 (not 8.1) to get the
>> improved sorting code.
>
> By the way, is the new sorting code any better for platforms that already
> have a decent qsort() (like Linux)?
yes - especially on-disk sorts will get some tremendous speedups in 8.2.
Stefan
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Performance of ORDER BY
2006-12-05 17:12 Performance of ORDER BY Glenn Sullivan <glenn.sullivan@varianinc.com>
2006-12-05 18:02 ` Re: Performance of ORDER BY Tom Lane <tgl@sss.pgh.pa.us>
@ 2006-12-05 18:20 ` A. Kretschmer <andreas.kretschmer@schollglas.com>
2 siblings, 0 replies; 8+ messages in thread
From: A. Kretschmer @ 2006-12-05 18:20 UTC (permalink / raw)
To: pgsql-performance
am Tue, dem 05.12.2006, um 13:02:06 -0500 mailte Tom Lane folgendes:
> In 8.0 that might be counterproductively high --- we have seen cases
> where more sort_mem = slower with the older sorting code. I concur
> with Luke's advice that you should update to 8.2 (not 8.1) to get the
> improved sorting code.
Hey, yes, 8.2 is released! Great, and thanks to all the people, which
help to develop this great software.
Andreas
--
Andreas Kretschmer
Kontakt: Heynitz: 035242/47215, D1: 0160/7141639 (mehr: -> Header)
GnuPG-ID: 0x3FFF606C, privat 0x7F4584DA http://wwwkeys.de.pgp.net
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Performance of ORDER BY
2006-12-05 17:12 Performance of ORDER BY Glenn Sullivan <glenn.sullivan@varianinc.com>
2006-12-05 18:02 ` Re: Performance of ORDER BY Tom Lane <tgl@sss.pgh.pa.us>
@ 2006-12-05 20:42 ` Glenn Sullivan <glenn.sullivan@varianinc.com>
2 siblings, 0 replies; 8+ messages in thread
From: Glenn Sullivan @ 2006-12-05 20:42 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-performance
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
<title></title>
</head>
<body bgcolor="#ffffff" text="#000000">
Thanks to Luke and Tom for the input. I guess this was good timing
given that it looks like<br>
8.2 was just released today. I will upgade to that before doing
anything else.<br>
<br>
Glenn<br>
<br>
Tom Lane wrote:
<blockquote cite="mid7090.1165341726@sss.pgh.pa.us" type="cite">
<pre wrap="">Glenn Sullivan <a class="moz-txt-link-rfc2396E" href="mailto:glenn.sullivan@varianinc.com"><glenn.sullivan@varianinc.com></a> writes:
</pre>
<blockquote type="cite">
<pre wrap="">I am wanting some ideas about improving the performance of ORDER BY in
our use. I have a DB on the order of 500,000 rows and 50 columns.
The results are always sorted with ORDER BY. Sometimes, the users end up
with a search that matches most of the rows. In that case, I have a
LIMIT 5000 to keep the returned results under control. However, the
sorting seems to take 10-60 sec. If I do the same search without the
ORDER BY, it takes about a second.
</pre>
</blockquote>
<pre wrap=""><!---->
Does the ORDER BY match an index? If so, is it using the index?
(See EXPLAIN.)
</pre>
<blockquote type="cite">
<pre wrap="">I am currently on version 8.0.1 on Windows XP using a Dell Optiplex 280
with 1Gb of ram. I have set sort_mem=100000 set.
</pre>
</blockquote>
<pre wrap=""><!---->
In 8.0 that might be counterproductively high --- we have seen cases
where more sort_mem = slower with the older sorting code. I concur
with Luke's advice that you should update to 8.2 (not 8.1) to get the
improved sorting code.
regards, tom lane
</pre>
</blockquote>
</body>
</html>
^ permalink raw reply [nested|flat] 8+ messages in thread
end of thread, other threads:[~2006-12-05 20:42 UTC | newest]
Thread overview: 8+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2006-12-05 17:12 Performance of ORDER BY Glenn Sullivan <glenn.sullivan@varianinc.com>
2006-12-05 17:36 ` Luke Lonergan <llonergan@greenplum.com>
2006-12-05 18:02 ` Tom Lane <tgl@sss.pgh.pa.us>
2006-12-05 18:17 ` Steinar H. Gunderson <sgunderson@bigfoot.com>
2006-12-05 18:39 ` Tom Lane <tgl@sss.pgh.pa.us>
2006-12-05 19:55 ` Stefan Kaltenbrunner <stefan@kaltenbrunner.cc>
2006-12-05 18:20 ` A. Kretschmer <andreas.kretschmer@schollglas.com>
2006-12-05 20:42 ` Glenn Sullivan <glenn.sullivan@varianinc.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox