pg.ddx.io pgsql-admin@postgresql.org mailing list archive
help / color / mirror / Atom feedSimple Query Very Slow
5+ messages / 4 participants
[nested] [flat]
* Simple Query Very Slow
@ 2012-12-21 21:23 Jose Martinez <jmartinez@opencrowd.com>
2012-12-22 12:06 ` Re: Simple Query Very Slow Alban Hertroys <haramrae@gmail.com>
0 siblings, 1 reply; 5+ messages in thread
From: Jose Martinez @ 2012-12-21 21:23 UTC (permalink / raw)
To: pgsql-general@postgresql.org; pgsql-admin
Hello,
I have the following table and indices defined:
CREATE TABLE ticket
(
wid bigint NOT NULL DEFAULT nextval('tickets_id_seq'::regclass),
eid bigint,
created timestamp with time zone NOT NULL DEFAULT now(),
status integer NOT NULL DEFAULT 0,
argsxml text,
moduleid character varying(255),
source_id bigint,
file_type_id bigint,
file_name character varying(255),
status_reason character varying(255),
...
)
I created an index on the 'created' timestamp as fallows:
CREATE INDEX ticket_1_idx
ON ticket
USING btree
(created );
and here's my query
select * from ticket
where created between '2012-12-19 00:00:00' and '2012-12-20 00:00:00'
This was working fine until the number of records started to grow (about 5
million) and now it's taking forever to return.
Explain analyze reveals this:
"Index Scan using ticket_1_idx on ticket (cost=0.00..10202.64 rows=52543
width=1297) (actual time=0.109..125.704 rows=53340 loops=1)"
" Index Cond: ((created >= '2012-12-19 00:00:00+00'::timestamp with time
zone) AND (created <= '2012-12-20 00:00:00+00'::timestamp with time zone))"
"Total runtime: 175.853 ms"
So far I've tried setting
random_page_cost = 1.75
effective_cache_size = 3
Also created
create CLUSTER ticket USING ticket_1_idx;
Nothing works. What am I doing wrong? why is it selecting sequential scan?
the indexes are supposed to make the query fast. Anything that can be done
to optimize it?
Help is appreciated! Thx.
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: Simple Query Very Slow
2012-12-21 21:23 Simple Query Very Slow Jose Martinez <jmartinez@opencrowd.com>
@ 2012-12-22 12:06 ` Alban Hertroys <haramrae@gmail.com>
2012-12-22 12:26 ` Re: Simple Query Very Slow Andres Freund <andres@2ndquadrant.com>
0 siblings, 1 reply; 5+ messages in thread
From: Alban Hertroys @ 2012-12-22 12:06 UTC (permalink / raw)
To: Jose Martinez <jmartinez@opencrowd.com>; +Cc: pgsql-general@postgresql.org; pgsql-admin
> and here's my query
>
> select * from ticket
> where created between '2012-12-19 00:00:00' and '2012-12-20 00:00:00'
>
> This was working fine until the number of records started to grow (about 5 million) and now it's taking forever to return.
>
> Explain analyze reveals this:
>
> "Index Scan using ticket_1_idx on ticket (cost=0.00..10202.64 rows=52543 width=1297) (actual time=0.109..125.704 rows=53340 loops=1)"
> " Index Cond: ((created >= '2012-12-19 00:00:00+00'::timestamp with time zone) AND (created <= '2012-12-20 00:00:00+00'::timestamp with time zone))"
> "Total runtime: 175.853 ms"
> Nothing works. What am I doing wrong? why is it selecting sequential scan? the indexes are supposed to make the query fast. Anything that can be done to optimize it?
It is not selecting sequential scan, you're looking at an index scan. That should be pretty fast, and it isn't that slow - that's still sub-second performance (0.176s).
Is that explain from the correct table? According to the results there are but 53 thousand rows in it, not anywhere near 5 million.
Perhaps your index got bloated? Do you have any long-running transactions still active that prevent cleaning up deprecated index entries? If that's the case, close the application that keeps that connection open and run a vacuum.
Are you vacuuming that table often enough?
If none of that helps, perhaps a REINDEX does.
Is that a dedicated database machine or is it also doing other stuff that's eating up resources?
You didn't mention what version of Postgres you're on or what OS you're using.
Alban Hertroys
--
Screwing up is an excellent way to attach something to the ceiling.
--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: Simple Query Very Slow
2012-12-21 21:23 Simple Query Very Slow Jose Martinez <jmartinez@opencrowd.com>
2012-12-22 12:06 ` Re: Simple Query Very Slow Alban Hertroys <haramrae@gmail.com>
@ 2012-12-22 12:26 ` Andres Freund <andres@2ndquadrant.com>
2012-12-24 17:10 ` Re: Simple Query Very Slow Jose Martinez <jmartinez@opencrowd.com>
0 siblings, 1 reply; 5+ messages in thread
From: Andres Freund @ 2012-12-22 12:26 UTC (permalink / raw)
To: Alban Hertroys <haramrae@gmail.com>; +Cc: Jose Martinez <jmartinez@opencrowd.com>; pgsql-general@postgresql.org; pgsql-admin
On 2012-12-22 13:06:21 +0100, Alban Hertroys wrote:
> > and here's my query
> >
> > select * from ticket
> > where created between '2012-12-19 00:00:00' and '2012-12-20 00:00:00'
> >
> > This was working fine until the number of records started to grow (about 5 million) and now it's taking forever to return.
> >
> > Explain analyze reveals this:
> >
> > "Index Scan using ticket_1_idx on ticket (cost=0.00..10202.64 rows=52543 width=1297) (actual time=0.109..125.704 rows=53340 loops=1)"
> > " Index Cond: ((created >= '2012-12-19 00:00:00+00'::timestamp with time zone) AND (created <= '2012-12-20 00:00:00+00'::timestamp with time zone))"
> > "Total runtime: 175.853 ms"
>
> > Nothing works. What am I doing wrong? why is it selecting sequential scan? the indexes are supposed to make the query fast. Anything that can be done to optimize it?
Whats the time you would need? Beause the above isn't that slow. Perhaps
the timing youre seing from your application includes transferring the
data over a not too fast link?
It would be interesting to see EXPLAIN (ANALYZE, BUFFERS) $query
> It is not selecting sequential scan, you're looking at an index scan. That should be pretty fast, and it isn't that slow - that's still sub-second performance (0.176s).
> Is that explain from the correct table? According to the results there are but 53 thousand rows in it, not anywhere near 5 million.
Well, thats the estimate *after* applying the restriction, so that seems
sensible.
Greetings,
Andres Freund
--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: Simple Query Very Slow
2012-12-21 21:23 Simple Query Very Slow Jose Martinez <jmartinez@opencrowd.com>
2012-12-22 12:06 ` Re: Simple Query Very Slow Alban Hertroys <haramrae@gmail.com>
2012-12-22 12:26 ` Re: Simple Query Very Slow Andres Freund <andres@2ndquadrant.com>
@ 2012-12-24 17:10 ` Jose Martinez <jmartinez@opencrowd.com>
2012-12-26 13:34 ` Re: [ADMIN] Simple Query Very Slow Spike Grobstein <spike@ticketevolution.com>
0 siblings, 1 reply; 5+ messages in thread
From: Jose Martinez @ 2012-12-24 17:10 UTC (permalink / raw)
To: Andres Freund <andres@2ndquadrant.com>; +Cc: Alban Hertroys <haramrae@gmail.com>; pgsql-general@postgresql.org; pgsql-admin
Thanks for your responses. Sorry, I forgot to mention that the query
actually takes 46 seconds despite what analyze (I dont quite understand the
output of explain). We did perform a vacuum last Friday and it seems to
help but not too much. We'll also try to recreate the indices.
Here's the output of
EXPLAIN (ANALYZE, BUFFERS) SELECT * FROM TICKET
WHERE CREATED BETWEEN '2012-12-19 00:00:00' AND '2012-12-20 00:00:00'
"Index Scan using t_created_idx on ticket (cost=0.00..127638.47
rows=206383 width=183) (actual time=0.065..46104.557 rows=212126 loops=1)"
" Index Cond: ((created >= '2012-12-19 00:00:00+00'::timestamp with time
zone) AND (created <= '2012-12-20 00:00:00+00'::timestamp with time zone))"
" Buffers: shared hit=44141 read=157167"
"Total runtime: 46293.384 ms"
Thanks.
On Sat, Dec 22, 2012 at 7:26 AM, Andres Freund <andres@2ndquadrant.com>wrote:
> On 2012-12-22 13:06:21 +0100, Alban Hertroys wrote:
> > > and here's my query
> > >
> > > select * from ticket
> > > where created between '2012-12-19 00:00:00' and '2012-12-20 00:00:00'
> > >
> > > This was working fine until the number of records started to grow
> (about 5 million) and now it's taking forever to return.
> > >
> > > Explain analyze reveals this:
> > >
> > > "Index Scan using ticket_1_idx on ticket (cost=0.00..10202.64
> rows=52543 width=1297) (actual time=0.109..125.704 rows=53340 loops=1)"
> > > " Index Cond: ((created >= '2012-12-19 00:00:00+00'::timestamp with
> time zone) AND (created <= '2012-12-20 00:00:00+00'::timestamp with time
> zone))"
> > > "Total runtime: 175.853 ms"
> >
> > > Nothing works. What am I doing wrong? why is it selecting sequential
> scan? the indexes are supposed to make the query fast. Anything that can be
> done to optimize it?
>
> Whats the time you would need? Beause the above isn't that slow. Perhaps
> the timing youre seing from your application includes transferring the
> data over a not too fast link?
>
> It would be interesting to see EXPLAIN (ANALYZE, BUFFERS) $query
>
> > It is not selecting sequential scan, you're looking at an index scan.
> That should be pretty fast, and it isn't that slow - that's still
> sub-second performance (0.176s).
> > Is that explain from the correct table? According to the results there
> are but 53 thousand rows in it, not anywhere near 5 million.
>
> Well, thats the estimate *after* applying the restriction, so that seems
> sensible.
>
> Greetings,
>
> Andres Freund
>
> --
> Andres Freund http://www.2ndQuadrant.com/
> PostgreSQL Development, 24x7 Support, Training & Services
>
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: [ADMIN] Simple Query Very Slow
2012-12-21 21:23 Simple Query Very Slow Jose Martinez <jmartinez@opencrowd.com>
2012-12-22 12:06 ` Re: Simple Query Very Slow Alban Hertroys <haramrae@gmail.com>
2012-12-22 12:26 ` Re: Simple Query Very Slow Andres Freund <andres@2ndquadrant.com>
2012-12-24 17:10 ` Re: Simple Query Very Slow Jose Martinez <jmartinez@opencrowd.com>
@ 2012-12-26 13:34 ` Spike Grobstein <spike@ticketevolution.com>
0 siblings, 0 replies; 5+ messages in thread
From: Spike Grobstein @ 2012-12-26 13:34 UTC (permalink / raw)
To: pgsql-admin; pgsql-general@postgresql.org <pgsql-general@postgresql.org>
Hi Jose,
How much ram is in the server? Can you also post the following values from postgres config:
work_mem
shared_buffers
Frequently when those are incorrectly configured, it can lead to significant performance issues (like 40 second queries that should be sub-second). Out of the box, Postgres is configured incredibly conservatively as to run on even the smallest of servers.
Also, what operating system and version of potgres?
Thanks.
...spike
(Sent via handheld, please pardon spelling errors)
On Dec 24, 2012, at 12:10 PM, Jose Martinez <jmartinez@opencrowd.com> wrote:
> Thanks for your responses. Sorry, I forgot to mention that the query actually takes 46 seconds despite what analyze (I dont quite understand the output of explain). We did perform a vacuum last Friday and it seems to help but not too much. We'll also try to recreate the indices.
>
> Here's the output of
> EXPLAIN (ANALYZE, BUFFERS) SELECT * FROM TICKET
> WHERE CREATED BETWEEN '2012-12-19 00:00:00' AND '2012-12-20 00:00:00'
>
> "Index Scan using t_created_idx on ticket (cost=0.00..127638.47 rows=206383 width=183) (actual time=0.065..46104.557 rows=212126 loops=1)"
> " Index Cond: ((created >= '2012-12-19 00:00:00+00'::timestamp with time zone) AND (created <= '2012-12-20 00:00:00+00'::timestamp with time zone))"
> " Buffers: shared hit=44141 read=157167"
> "Total runtime: 46293.384 ms"
>
>
> Thanks.
>
>
> On Sat, Dec 22, 2012 at 7:26 AM, Andres Freund <andres@2ndquadrant.com> wrote:
>> On 2012-12-22 13:06:21 +0100, Alban Hertroys wrote:
>> > > and here's my query
>> > >
>> > > select * from ticket
>> > > where created between '2012-12-19 00:00:00' and '2012-12-20 00:00:00'
>> > >
>> > > This was working fine until the number of records started to grow (about 5 million) and now it's taking forever to return.
>> > >
>> > > Explain analyze reveals this:
>> > >
>> > > "Index Scan using ticket_1_idx on ticket (cost=0.00..10202.64 rows=52543 width=1297) (actual time=0.109..125.704 rows=53340 loops=1)"
>> > > " Index Cond: ((created >= '2012-12-19 00:00:00+00'::timestamp with time zone) AND (created <= '2012-12-20 00:00:00+00'::timestamp with time zone))"
>> > > "Total runtime: 175.853 ms"
>> >
>> > > Nothing works. What am I doing wrong? why is it selecting sequential scan? the indexes are supposed to make the query fast. Anything that can be done to optimize it?
>>
>> Whats the time you would need? Beause the above isn't that slow. Perhaps
>> the timing youre seing from your application includes transferring the
>> data over a not too fast link?
>>
>> It would be interesting to see EXPLAIN (ANALYZE, BUFFERS) $query
>>
>> > It is not selecting sequential scan, you're looking at an index scan. That should be pretty fast, and it isn't that slow - that's still sub-second performance (0.176s).
>> > Is that explain from the correct table? According to the results there are but 53 thousand rows in it, not anywhere near 5 million.
>>
>> Well, thats the estimate *after* applying the restriction, so that seems
>> sensible.
>>
>> Greetings,
>>
>> Andres Freund
>>
>> --
>> Andres Freund http://www.2ndQuadrant.com/
>> PostgreSQL Development, 24x7 Support, Training & Services
>
^ permalink raw reply [nested|flat] 5+ messages in thread
end of thread, other threads:[~2012-12-26 13:34 UTC | newest]
Thread overview: 5+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2012-12-21 21:23 Simple Query Very Slow Jose Martinez <jmartinez@opencrowd.com>
2012-12-22 12:06 ` Alban Hertroys <haramrae@gmail.com>
2012-12-22 12:26 ` Andres Freund <andres@2ndquadrant.com>
2012-12-24 17:10 ` Jose Martinez <jmartinez@opencrowd.com>
2012-12-26 13:34 ` Spike Grobstein <spike@ticketevolution.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox