From: Kouber Saparev <kouber@saparev.com>
To: pgsql-performance@postgresql.org
Subject: LIMIT confuses the planner
Date: Mon, 23 Feb 2009 14:26:24 +0200
Message-ID: <49A295F0.5010405@saparev.com> (raw)
Hello,
I'm experiencing a strange issue. I have a table with around 11 million
records (11471762 to be exact), storing login attempts to a web site.
Thanks to the index I have created on username, looking into that table
by username is very fast:
db=# EXPLAIN ANALYZE
SELECT
*
FROM
login_attempt
WHERE
username='kouber'
ORDER BY
login_attempt_sid DESC;
QUERY PLAN
------------------------------------------------------------------------------------------------------------------------------------------------------
Sort (cost=1415.15..1434.93 rows=7914 width=38) (actual
time=0.103..0.104 rows=2 loops=1)
Sort Key: login_attempt_sid
Sort Method: quicksort Memory: 25kB
-> Index Scan using login_attempt_username_idx on login_attempt
(cost=0.00..902.71 rows=7914 width=38) (actual time=0.090..0.091 rows=2
loops=1)
Index Cond: ((username)::text = 'kouber'::text)
Total runtime: 0.140 ms
(6 rows)
As you can see, there are only 2 records for that particular username.
However when I add a LIMIT clause to the same query the planner no
longer uses the right index, hence the query becomes very slow:
db=# EXPLAIN ANALYZE
SELECT
*
FROM
login_attempt
WHERE
username='kouber'
ORDER BY
login_attempt_sid DESC LIMIT 20;
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------------------------
Limit (cost=0.00..770.45 rows=20 width=38) (actual
time=0.064..3797.660 rows=2 loops=1)
-> Index Scan Backward using login_attempt_pkey on login_attempt
(cost=0.00..304866.46 rows=7914 width=38) (actual time=0.062..3797.657
rows=2 loops=1)
Filter: ((username)::text = 'kouber'::text)
Total runtime: 3797.691 ms
(4 rows)
Now, recently I have altered some of the default parameters in order to
get as much as possible out of the hardware - 12 GB of RAM, 8
processors. So, I guess I have done something wrong, thus the planner is
taking that wrong decision. Here's what I have changed in
postgresql.conf (from the default one):
max_connections = 200
shared_buffers = 256MB
work_mem = 64MB
maintenance_work_mem = 128MB
max_stack_depth = 6MB
max_fsm_pages = 100000
synchronous_commit = off
wal_buffers = 1MB
commit_delay = 100
commit_siblings = 5
checkpoint_segments = 10
checkpoint_timeout = 10min
random_page_cost = 0.1
effective_cache_size = 2048MB
Any idea what's wrong here?
Regards,
--
Kouber Saparev
http://kouber.saparev.com
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: kouber@saparev.com
Subject: Re: LIMIT confuses the planner
In-Reply-To: <49A295F0.5010405@saparev.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox