pg.ddx.io pgsql-sql@postgresql.org mailing list archive
help / color / mirror / Atom feeddatabase with 1000000 rows is very slow
6+ messages / 3 participants
[nested] [flat]
* database with 1000000 rows is very slow
@ 2000-03-05 20:11 David Celjuska <dcsoft@dcsoft.sk>
0 siblings, 0 replies; 6+ messages in thread
From: David Celjuska @ 2000-03-05 20:11 UTC (permalink / raw)
To: pgsql-novice@postgresql.org; pgsql-sql
Hallo All!
I have database with follow structure:
CREATE TABLE "article" (
"id" character varying(15) NOT NULL,
"obj_kod" character varying(15),
"popis" character varying(80),
"net_price" float4,
"our_price" float4,
"quantity" int2,
"group1" character varying(40) DEFAULT 'ine',
"group2" character varying(40),
"pic1" character varying(10) DEFAULT 'noname.jpg',
"pic2" character varying(10) DEFAULT 'noname.jpg',
"alt1" character varying(15),
"alt2" character varying(15),
"zisk" int2);
CREATE UNIQUE INDEX "article_pkey" on "article" using btree ( "id"
"varchar_ops" );
and with 1000000 rows. Postgres deamon run on 2xPentiumII 330Mhz with
SCSI disk where is
this database store. But I think that select * from article where id
like 'something%' is very slow
(some minutes) and query as: select * from article where id='something'
is very slow too.
I don't know where is a problem a I would like optimalise this, but how
can I do it?
When I use hash except btree, query as: select * from article where
id='something' is fast but
select * from article where id='something%' is very slow.
Can I index some columns externaly? For example: psql index database
table col.
Or postgresql make indexes automaticly?
How can I see that postgres use/or no use index on some query? It is
possible?
Thank you every reply,
Davy!
^ permalink raw reply [nested|flat] 6+ messages in thread
* database with 1000000 rows is very slow
@ 2000-03-05 20:21 David Celjuska <dcsoft@dcsoft.sk>
2000-03-06 07:22 ` Re: [SQL] database with 1000000 rows is very slow Peter Eisentraut <e99re41@DoCS.UU.SE>
0 siblings, 1 reply; 6+ messages in thread
From: David Celjuska @ 2000-03-05 20:21 UTC (permalink / raw)
To: pgsql-sql
Hallo All!
I have database with follow structure:
CREATE TABLE "article" (
"id" character varying(15) NOT NULL,
"obj_kod" character varying(15),
"popis" character varying(80),
"net_price" float4,
"our_price" float4,
"quantity" int2,
"group1" character varying(40) DEFAULT 'ine',
"group2" character varying(40),
"pic1" character varying(10) DEFAULT 'noname.jpg',
"pic2" character varying(10) DEFAULT 'noname.jpg',
"alt1" character varying(15),
"alt2" character varying(15),
"zisk" int2);
CREATE UNIQUE INDEX "article_pkey" on "article" using btree ( "id"
"varchar_ops" );
and with 1000000 rows. Postgres deamon run on 2xPentiumII 330Mhz with
SCSI disk where is
this database store. But I think that select * from article where id
like 'something%' is very slow
(some minutes) and query as: select * from article where id='something'
is very slow too.
I don't know where is a problem a I would like optimalise this, but how
can I do it?
When I use hash except btree, query as: select * from article where
id='something' is fast but
select * from article where id='something%' is very slow.
Can I index some columns externaly? For example: psql index database
table col.
Or postgresql make indexes automaticly?
How can I see that postgres use/or no use index on some query? It is
possible?
Thank you every reply,
Davy!
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: [SQL] database with 1000000 rows is very slow
2000-03-05 20:21 database with 1000000 rows is very slow David Celjuska <dcsoft@dcsoft.sk>
@ 2000-03-06 07:22 ` Peter Eisentraut <e99re41@DoCS.UU.SE>
0 siblings, 0 replies; 6+ messages in thread
From: Peter Eisentraut @ 2000-03-06 07:22 UTC (permalink / raw)
To: David Celjuska <dcsoft@dcsoft.sk>; +Cc: pgsql-sql
On Sun, 5 Mar 2000, David Celjuska wrote:
> CREATE UNIQUE INDEX "article_pkey" on "article" using btree ( "id"
> "varchar_ops" );
> this database store. But I think that select * from article where id
> like 'something%' is very slow (some minutes) and query as: select *
> from article where id='something' is very slow too. I don't know where
> is a problem a I would like optimalise this, but how can I do it?
If you haven't run vacuum analyze lately then you should do that.
> When I use hash except btree, query as: select * from article where
> id='something' is fast but select * from article where id='something%'
> is very slow.
Yup. That's because hashes only work on exact matches and btrees can do
ordering (like 'somethink' is surely "larger" than 'something%').
> Or postgresql make indexes automaticly?
No, you have to make them, but you did that right.
> How can I see that postgres use/or no use index on some query? It is
> possible?
Yup. EXPLAIN SELECT ....
In fact, if you can't make any progress you should always accompany any
optimizer issues with the EXPLAIN output. That will help our optimizer
gurus. :)
--
Peter Eisentraut Sernanders väg 10:115
peter_e@gmx.net 75262 Uppsala
http://yi.org/peter-e/ Sweden
^ permalink raw reply [nested|flat] 6+ messages in thread
* RE: [SQL] database with 1000000 rows is very slow
@ 2000-03-06 15:27 Justin Long <justinlong@strategicnetwork.org>
0 siblings, 0 replies; 6+ messages in thread
From: Justin Long @ 2000-03-06 15:27 UTC (permalink / raw)
To: 'Peter Eisentraut' <peter_e@gmx.net>; +Cc: pgsql-sql
Who do we send optimizer questions to?
For example, I have a logbook which is ordered by a logtime stamp. Here's
my explain output (and by the way, that was very helpful, I hadn't noticed
that command before):
EXPLAIN select * from logbook order by logtime desc limit 25;
NOTICE: QUERY PLAN:
Sort (cost=3671.28 rows=76766 width=80)
-> Seq Scan on logbook (cost=3671.28 rows=76766 width=80)
Now I wonder why it's doing a sequential scan. If it's ordered by logtime
and a limit is specified can't it determine that it should just take the
bottom 25 entries off the list?
_____
Justin Long
Network for Strategic Missions
977 Centerville Turnpike, Va Beach, VA 23463
Ofc 757-226-5011, Fax 757-226-5006, Email justinlong@strategicnetwork.org
Never retreat. Never surrender. Never cut a deal with a dragon.
http://www.strategicnetwork.org
-----Original Message-----
From: Peter Eisentraut [SMTP:e99re41@DoCS.UU.SE]
Sent: Monday, March 06, 2000 2:22 AM
To: David Celjuska
Cc: pgsql-sql@postgreSQL.org
Subject: Re: [SQL] database with 1000000 rows is very slow
On Sun, 5 Mar 2000, David Celjuska wrote:
> CREATE UNIQUE INDEX "article_pkey" on "article" using btree ( "id"
> "varchar_ops" );
> this database store. But I think that select * from article where id
> like 'something%' is very slow (some minutes) and query as: select *
> from article where id='something' is very slow too. I don't know where
> is a problem a I would like optimalise this, but how can I do it?
If you haven't run vacuum analyze lately then you should do that.
> When I use hash except btree, query as: select * from article where
> id='something' is fast but select * from article where id='something%'
> is very slow.
Yup. That's because hashes only work on exact matches and btrees can do
ordering (like 'somethink' is surely "larger" than 'something%').
> Or postgresql make indexes automaticly?
No, you have to make them, but you did that right.
> How can I see that postgres use/or no use index on some query? It is
> possible?
Yup. EXPLAIN SELECT ....
In fact, if you can't make any progress you should always accompany any
optimizer issues with the EXPLAIN output. That will help our optimizer
gurus. :)
--
Peter Eisentraut Sernanders vag 10:115
peter_e@gmx.net 75262 Uppsala
http://yi.org/peter-e/ Sweden
************
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: [SQL] database with 1000000 rows is very slow
@ 2000-03-08 11:11 David Celjuska <dcsoft@dcsoft.sk>
0 siblings, 0 replies; 6+ messages in thread
From: David Celjuska @ 2000-03-08 11:11 UTC (permalink / raw)
To: pgsql-sql
Hello Peter Eisentraut!
>> CREATE UNIQUE INDEX "article_pkey" on "article" using btree ( "id"
>> "varchar_ops" );
>> this database store. But I think that select * from article where id
>> like 'something%' is very slow (some minutes) and query as: select *
>> from article where id='something' is very slow too. I don't know where
>> is a problem a I would like optimalise this, but how can I do it?
>If you haven't run vacuum analyze lately then you should do that.
Ok, I was run it and here is result, but I understand what it means.
Can you explain it to me, please?
[dcsoft@bigmax dcsoft]$ vacuumdb --analyze --verbose --table article evid
NOTICE: --Relation article--
NOTICE: Pages 24341: Changed 0, Reapped 8, Empty 0, New 0; Tup 1376029: Vac 0, Keep/VTL 0/0, Crash 0, UnUsed 29, MinLen 112, MaxLen 166; Re-using: Free/Avail. Space 1924/0; EndEmpty/Avail. Pages 0/0. Elapsed 3/246 sec.
NOTICE: Index article_index: Pages 6465; Tuples 1376029: Deleted 0. Elapsed 1/5 sec.
NOTICE: Index hash_obj_kod: Pages 10841; Tuples 1376029: Deleted 0. Elapsed 1/25 sec.
NOTICE: Index article_obj_kod_key: Pages 6860; Tuples 1376029: Deleted 0. Elapsed 1/5 sec.
NOTICE: Index article_pkey: Pages 6858; Tuples 1376029: Deleted 0. Elapsed 1/5 sec.
>> When I use hash except btree, query as: select * from article where
>> id='something' is fast but select * from article where id='something%'
>> is very slow.
>Yup. That's because hashes only work on exact matches and btrees can do
>ordering (like 'somethink' is surely "larger" than 'something%').
Ok, I understand.
>> How can I see that postgres use/or no use index on some query? It is
>> possible?
>Yup. EXPLAIN SELECT ....
>In fact, if you can't make any EXPLAIN progress you should always accompany any
>optimizer issues with the EXPLAIN output. That will help our optimizer
>gurus. :)
OK, My table structure is:
CREATE TABLE "article" (
"id" character varying(15) NOT NULL,
"obj_kod" character varying(15),
"popis" character varying(80),
"net_price" float4,
"our_price" float4,
"quantity" int2,
"group1" character varying(40) DEFAULT 'ine',CREATE TABLE "article" (
"id" character varying(15) NOT NULL,
"obj_kod" character varying(15),
"popis" character varying(80),
"net_price" float4,
"our_price" float4,
"quantity" int2,
"group1" character varying(40) DEFAULT 'ine',
"group2" character varying(40),
"pic1" character varying(10) DEFAULT 'noname.jpg',
"pic2" character varying(10) DEFAULT 'noname.jpg',
"alt1" character varying(15),
"alt2" character varying(15),
"zisk" int2);
REVOKE ALL on "article" from PUBLIC;
GRANT INSERT,UPDATE,DELETE,SELECT on "article" to GROUP "evid";
CREATE UNIQUE INDEX "article_pkey" on "article" using btree ( "id" "varchar_ops" );
CREATE INDEX "article_obj_kod_key" on "article" using btree ( "obj_kod" "varchar_ops" );
CREATE INDEX "hash_obj_kod" on "article" using hash ( "obj_kod" "varchar_ops" );
CREATE INDEX "article_index" on "article" using btree ( "obj_kod" "varchar_ops" );
with 1376029 rows.
and here is result:
evid=> explain select * from article where id = 'something';
NOTICE: QUERY PLAN:
Index Scan using article_pkey on article (cost=2.05 rows=1 width=120)
EXPLAIN
I think that it is ok (fast), but next result is no.
evid=> explain select * from article where id like '186892%';
NOTICE: QUERY PLAN:
Index Scan using article_pkey on article (cost=33333.82 rows=1 width=120)
EXPLAIN
I think that price is very big.
If you can help me or If you have some sugestion please writte me.
With regards,
.~.
/V\ Davy
// \\ [dcsoft@dcsoft.sk]
/( )\
^`~'^
... online ANG<->SK slovnik: http://www.dcsoft.sk/slovnik ...
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: [SQL] database with 1000000 rows is very slow
@ 2000-03-08 13:06 David Celjuska <dcsoft@dcsoft.sk>
0 siblings, 0 replies; 6+ messages in thread
From: David Celjuska @ 2000-03-08 13:06 UTC (permalink / raw)
To: pgsql-sql
Hello Peter Eisentraut!
>> CREATE UNIQUE INDEX "article_pkey" on "article" using btree ( "id"
>> "varchar_ops" );
>> this database store. But I think that select * from article where id
>> like 'something%' is very slow (some minutes) and query as: select *
>> from article where id='something' is very slow too. I don't know where
>> is a problem a I would like optimalise this, but how can I do it?
>If you haven't run vacuum analyze lately then you should do that.
Ok, I was run it and here is result, but I understand what it means.
Can you explain it to me, please?
[dcsoft@bigmax dcsoft]$ vacuumdb --analyze --verbose --table article
evid
NOTICE: --Relation article--
NOTICE: Pages 24341: Changed 0, Reapped 8, Empty 0, New 0; Tup 1376029:
Vac 0, Keep/VTL 0/0, Crash 0, UnUsed 29, MinLen 112, MaxLen 166;
Re-using: Free/Avail. Space 1924/0; EndEmpty/Avail. Pages 0/0. Elapsed
3/246 sec.
NOTICE: Index article_index: Pages 6465; Tuples 1376029: Deleted 0.
Elapsed 1/5 sec.
NOTICE: Index hash_obj_kod: Pages 10841; Tuples 1376029: Deleted 0.
Elapsed 1/25 sec.
NOTICE: Index article_obj_kod_key: Pages 6860; Tuples 1376029: Deleted
0. Elapsed 1/5 sec.
NOTICE: Index article_pkey: Pages 6858; Tuples 1376029: Deleted 0.
Elapsed 1/5 sec.
>> When I use hash except btree, query as: select * from article where
>> id='something' is fast but select * from article where id='something%'
>> is very slow.
>Yup. That's because hashes only work on exact matches and btrees can do
>ordering (like 'somethink' is surely "larger" than 'something%').
Ok, I understand.
>> How can I see that postgres use/or no use index on some query? It is
>> possible?
>Yup. EXPLAIN SELECT ....
>In fact, if you can't make any EXPLAIN progress you should always accompany any
>optimizer issues with the EXPLAIN output. That will help our optimizer
>gurus. :)
OK, My table structure is:
CREATE TABLE "article" (
"id" character varying(15) NOT NULL,
"obj_kod" character varying(15),
"popis" character varying(80),
"net_price" float4,
"our_price" float4,
"quantity" int2,
"group1" character varying(40) DEFAULT 'ine',CREATE TABLE
"article" (
"id" character varying(15) NOT NULL,
"obj_kod" character varying(15),
"popis" character varying(80),
"net_price" float4,
"our_price" float4,
"quantity" int2,
"group1" character varying(40) DEFAULT 'ine',
"group2" character varying(40),
"pic1" character varying(10) DEFAULT 'noname.jpg',
"pic2" character varying(10) DEFAULT 'noname.jpg',
"alt1" character varying(15),
"alt2" character varying(15),
"zisk" int2);
REVOKE ALL on "article" from PUBLIC;
GRANT INSERT,UPDATE,DELETE,SELECT on "article" to GROUP "evid";
CREATE UNIQUE INDEX "article_pkey" on "article" using btree ( "id"
"varchar_ops" );
CREATE INDEX "article_obj_kod_key" on "article" using btree ( "obj_kod"
"varchar_ops" );
CREATE INDEX "hash_obj_kod" on "article" using hash ( "obj_kod"
"varchar_ops" );
CREATE INDEX "article_index" on "article" using btree ( "obj_kod"
"varchar_ops" );
with 1376029 rows.
and here is result:
evid=> explain select * from article where id = 'something';
NOTICE: QUERY PLAN:
Index Scan using article_pkey on article (cost=2.05 rows=1 width=120)
EXPLAIN
I think that it is ok (fast), but next result is no.
evid=> explain select * from article where id like '186892%';
NOTICE: QUERY PLAN:
Index Scan using article_pkey on article (cost=33333.82 rows=1
width=120)
EXPLAIN
I think that price is very big.
If you can help me or If you have some sugestion please writte me.
With regards,
.~.
/V\ Davy
// \\ [dcsoft@dcsoft.sk]
/( )\
^`~'^
... online ANG<->SK slovnik: http://www.dcsoft.sk/slovnik ...
^ permalink raw reply [nested|flat] 6+ messages in thread
end of thread, other threads:[~2000-03-08 13:06 UTC | newest]
Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2000-03-05 20:11 database with 1000000 rows is very slow David Celjuska <dcsoft@dcsoft.sk>
2000-03-05 20:21 database with 1000000 rows is very slow David Celjuska <dcsoft@dcsoft.sk>
2000-03-06 07:22 ` Peter Eisentraut <e99re41@DoCS.UU.SE>
2000-03-06 15:27 RE: [SQL] database with 1000000 rows is very slow Justin Long <justinlong@strategicnetwork.org>
2000-03-08 11:11 Re: [SQL] database with 1000000 rows is very slow David Celjuska <dcsoft@dcsoft.sk>
2000-03-08 13:06 Re: [SQL] database with 1000000 rows is very slow David Celjuska <dcsoft@dcsoft.sk>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox