pg.ddx.io  pgsql-sql@postgresql.org mailing list archive  
help / color / mirror / Atom feed
database with 1000000 rows is very slow
6+ messages / 3 participants
[nested] [flat]

* database with 1000000 rows is very slow
@ 2000-03-05 20:11  David Celjuska <dcsoft@dcsoft.sk>
  0 siblings, 0 replies; 6+ messages in thread

From: David Celjuska @ 2000-03-05 20:11 UTC (permalink / raw)
  To: pgsql-novice@postgresql.org; pgsql-sql

Hallo All!

I have database with follow structure:

CREATE TABLE "article" (
        "id" character varying(15) NOT NULL,
        "obj_kod" character varying(15),
        "popis" character varying(80),
        "net_price" float4,
        "our_price" float4,
        "quantity" int2,
        "group1" character varying(40) DEFAULT 'ine',
        "group2" character varying(40),
        "pic1" character varying(10) DEFAULT 'noname.jpg',
        "pic2" character varying(10) DEFAULT 'noname.jpg',
        "alt1" character varying(15),
        "alt2" character varying(15),
        "zisk" int2);

CREATE UNIQUE INDEX "article_pkey" on "article" using btree ( "id"
"varchar_ops" );

and with 1000000 rows. Postgres deamon run on 2xPentiumII 330Mhz with
SCSI disk where is
this database store. But I think that select * from article where id
like 'something%' is very slow
(some minutes) and query as: select * from article where id='something'
is very slow too.
I don't know where is a problem a I would like optimalise this, but how
can I do it?

When I use hash except btree, query as: select * from article where
id='something' is fast but
select * from article where id='something%' is very slow.

Can I index some columns externaly? For example: psql index database
table col.
Or postgresql make indexes automaticly?

How can I see that postgres use/or no use index on some query? It is
possible?

Thank you every reply,
    Davy!



^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* database with 1000000 rows is very slow
@ 2000-03-05 20:21  David Celjuska <dcsoft@dcsoft.sk>
  0 siblings, 1 reply; 6+ messages in thread

From: David Celjuska @ 2000-03-05 20:21 UTC (permalink / raw)
  To: pgsql-sql

Hallo All!

I have database with follow structure:

CREATE TABLE "article" (
        "id" character varying(15) NOT NULL,
        "obj_kod" character varying(15),
        "popis" character varying(80),
        "net_price" float4,
        "our_price" float4,
        "quantity" int2,
        "group1" character varying(40) DEFAULT 'ine',
        "group2" character varying(40),
        "pic1" character varying(10) DEFAULT 'noname.jpg',
        "pic2" character varying(10) DEFAULT 'noname.jpg',
        "alt1" character varying(15),
        "alt2" character varying(15),
        "zisk" int2);

CREATE UNIQUE INDEX "article_pkey" on "article" using btree ( "id"
"varchar_ops" );

and with 1000000 rows. Postgres deamon run on 2xPentiumII 330Mhz with
SCSI disk where is
this database store. But I think that select * from article where id
like 'something%' is very slow
(some minutes) and query as: select * from article where id='something'
is very slow too.
I don't know where is a problem a I would like optimalise this, but how
can I do it?

When I use hash except btree, query as: select * from article where
id='something' is fast but
select * from article where id='something%' is very slow.

Can I index some columns externaly? For example: psql index database
table col.
Or postgresql make indexes automaticly?

How can I see that postgres use/or no use index on some query? It is
possible?

Thank you every reply,
    Davy!



^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* Re: [SQL] database with 1000000 rows is very slow
@ 2000-03-06 07:22  Peter Eisentraut <e99re41@DoCS.UU.SE>
  parent: David Celjuska <dcsoft@dcsoft.sk>
  0 siblings, 0 replies; 6+ messages in thread

From: Peter Eisentraut @ 2000-03-06 07:22 UTC (permalink / raw)
  To: David Celjuska <dcsoft@dcsoft.sk>; +Cc: pgsql-sql

On Sun, 5 Mar 2000, David Celjuska wrote:

> CREATE UNIQUE INDEX "article_pkey" on "article" using btree ( "id"
> "varchar_ops" );

> this database store. But I think that select * from article where id
> like 'something%' is very slow (some minutes) and query as: select *
> from article where id='something' is very slow too. I don't know where
> is a problem a I would like optimalise this, but how can I do it?

If you haven't run vacuum analyze lately then you should do that.

> When I use hash except btree, query as: select * from article where
> id='something' is fast but select * from article where id='something%'
> is very slow.

Yup. That's because hashes only work on exact matches and btrees can do
ordering (like 'somethink' is surely "larger" than 'something%').

> Or postgresql make indexes automaticly?

No, you have to make them, but you did that right.

> How can I see that postgres use/or no use index on some query? It is
> possible?

Yup. EXPLAIN SELECT ....

In fact, if you can't make any progress you should always accompany any
optimizer issues with the EXPLAIN output. That will help our optimizer
gurus. :)


-- 
Peter Eisentraut                  Sernanders väg 10:115
peter_e@gmx.net                   75262 Uppsala
http://yi.org/peter-e/            Sweden




^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* RE: [SQL] database with 1000000 rows is very slow
@ 2000-03-06 15:27  Justin Long <justinlong@strategicnetwork.org>
  0 siblings, 0 replies; 6+ messages in thread

From: Justin Long @ 2000-03-06 15:27 UTC (permalink / raw)
  To: 'Peter Eisentraut' <peter_e@gmx.net>; +Cc: pgsql-sql

Who do we send optimizer questions to?

For example, I have a logbook which is ordered by a logtime stamp. Here's 
my explain output (and by the way, that was very helpful, I hadn't noticed 
that command before):

EXPLAIN select * from logbook order by logtime desc limit 25;
NOTICE:  QUERY PLAN:

Sort  (cost=3671.28 rows=76766 width=80)
  ->  Seq Scan on logbook  (cost=3671.28 rows=76766 width=80)

Now I wonder why it's doing a sequential scan. If it's ordered by logtime 
and a limit is specified can't it determine that it should just take the 
bottom 25 entries off the list?

_____
Justin Long
Network for Strategic Missions
977 Centerville Turnpike, Va Beach, VA 23463
Ofc 757-226-5011, Fax 757-226-5006, Email justinlong@strategicnetwork.org
Never retreat. Never surrender. Never cut a deal with a dragon.
http://www.strategicnetwork.org


-----Original Message-----
From:	Peter Eisentraut [SMTP:e99re41@DoCS.UU.SE]
Sent:	Monday, March 06, 2000 2:22 AM
To:	David Celjuska
Cc:	pgsql-sql@postgreSQL.org
Subject:	Re: [SQL] database with 1000000 rows is very slow

On Sun, 5 Mar 2000, David Celjuska wrote:

> CREATE UNIQUE INDEX "article_pkey" on "article" using btree ( "id"
> "varchar_ops" );

> this database store. But I think that select * from article where id
> like 'something%' is very slow (some minutes) and query as: select *
> from article where id='something' is very slow too. I don't know where
> is a problem a I would like optimalise this, but how can I do it?

If you haven't run vacuum analyze lately then you should do that.

> When I use hash except btree, query as: select * from article where
> id='something' is fast but select * from article where id='something%'
> is very slow.

Yup. That's because hashes only work on exact matches and btrees can do
ordering (like 'somethink' is surely "larger" than 'something%').

> Or postgresql make indexes automaticly?

No, you have to make them, but you did that right.

> How can I see that postgres use/or no use index on some query? It is
> possible?

Yup. EXPLAIN SELECT ....

In fact, if you can't make any progress you should always accompany any
optimizer issues with the EXPLAIN output. That will help our optimizer
gurus. :)


--
Peter Eisentraut                  Sernanders vag 10:115
peter_e@gmx.net                   75262 Uppsala
http://yi.org/peter-e/            Sweden


************



^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* Re: [SQL] database with 1000000 rows is very slow
@ 2000-03-08 11:11  David Celjuska <dcsoft@dcsoft.sk>
  0 siblings, 0 replies; 6+ messages in thread

From: David Celjuska @ 2000-03-08 11:11 UTC (permalink / raw)
  To: pgsql-sql

Hello Peter Eisentraut!

>> CREATE UNIQUE INDEX "article_pkey" on "article" using btree ( "id"
>> "varchar_ops" );

>> this database store. But I think that select * from article where id
>> like 'something%' is very slow (some minutes) and query as: select *
>> from article where id='something' is very slow too. I don't know where
>> is a problem a I would like optimalise this, but how can I do it?

>If you haven't run vacuum analyze lately then you should do that.

Ok, I was run it and here is result, but I understand what it means.

Can you explain it to me, please?

[dcsoft@bigmax dcsoft]$ vacuumdb --analyze --verbose --table article evid
NOTICE:  --Relation article--
NOTICE:  Pages 24341: Changed 0, Reapped 8, Empty 0, New 0; Tup 1376029: Vac 0, Keep/VTL 0/0, Crash 0, UnUsed 29, MinLen 112, MaxLen 166; Re-using: Free/Avail. Space 1924/0; EndEmpty/Avail. Pages 0/0. Elapsed 3/246 sec.
NOTICE:  Index article_index: Pages 6465; Tuples 1376029: Deleted 0. Elapsed 1/5 sec.
NOTICE:  Index hash_obj_kod: Pages 10841; Tuples 1376029: Deleted 0. Elapsed 1/25 sec.
NOTICE:  Index article_obj_kod_key: Pages 6860; Tuples 1376029: Deleted 0. Elapsed 1/5 sec.
NOTICE:  Index article_pkey: Pages 6858; Tuples 1376029: Deleted 0. Elapsed 1/5 sec.

>> When I use hash except btree, query as: select * from article where
>> id='something' is fast but select * from article where id='something%'
>> is very slow.

>Yup. That's because hashes only work on exact matches and btrees can do
>ordering (like 'somethink' is surely "larger" than 'something%').

Ok, I understand.

>> How can I see that postgres use/or no use index on some query? It is
>> possible?

>Yup. EXPLAIN SELECT ....

>In fact, if you can't make any EXPLAIN progress you should always accompany any
>optimizer issues with the EXPLAIN output. That will help our optimizer
>gurus. :)

OK, My table structure is:

CREATE TABLE "article" (
        "id" character varying(15) NOT NULL,
        "obj_kod" character varying(15),
        "popis" character varying(80),
        "net_price" float4,
        "our_price" float4,
        "quantity" int2,
        "group1" character varying(40) DEFAULT 'ine',CREATE TABLE "article" (
        "id" character varying(15) NOT NULL,
        "obj_kod" character varying(15),
        "popis" character varying(80),
        "net_price" float4,
        "our_price" float4,
        "quantity" int2,
        "group1" character varying(40) DEFAULT 'ine',

        "group2" character varying(40),
        "pic1" character varying(10) DEFAULT 'noname.jpg',
        "pic2" character varying(10) DEFAULT 'noname.jpg',
        "alt1" character varying(15),
        "alt2" character varying(15),
        "zisk" int2);

REVOKE ALL on "article" from PUBLIC;
GRANT INSERT,UPDATE,DELETE,SELECT on "article" to GROUP "evid";

CREATE UNIQUE INDEX "article_pkey" on "article" using btree ( "id" "varchar_ops" );
CREATE  INDEX "article_obj_kod_key" on "article" using btree ( "obj_kod" "varchar_ops" );
CREATE  INDEX "hash_obj_kod" on "article" using hash ( "obj_kod" "varchar_ops" );
CREATE  INDEX "article_index" on "article" using btree ( "obj_kod" "varchar_ops" );

with 1376029 rows.

and here is result:

evid=> explain select * from article where id = 'something';
NOTICE:  QUERY PLAN:

Index Scan using article_pkey on article  (cost=2.05 rows=1 width=120)

EXPLAIN

I think that it is ok (fast), but next result is no.

evid=> explain select * from article where id like '186892%';
NOTICE:  QUERY PLAN:

Index Scan using article_pkey on article  (cost=33333.82 rows=1 width=120)

EXPLAIN

I think that price is very big.

If you can help me or If you have some sugestion please writte me.

With regards,

    .~.
    /V\         Davy
   // \\  [dcsoft@dcsoft.sk]
  /(   )\
   ^`~'^
... online ANG<->SK slovnik: http://www.dcsoft.sk/slovnik ...




^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* Re: [SQL] database with 1000000 rows is very slow
@ 2000-03-08 13:06  David Celjuska <dcsoft@dcsoft.sk>
  0 siblings, 0 replies; 6+ messages in thread

From: David Celjuska @ 2000-03-08 13:06 UTC (permalink / raw)
  To: pgsql-sql

Hello Peter Eisentraut!

>> CREATE UNIQUE INDEX "article_pkey" on "article" using btree ( "id"
>> "varchar_ops" );

>> this database store. But I think that select * from article where id
>> like 'something%' is very slow (some minutes) and query as: select *
>> from article where id='something' is very slow too. I don't know where
>> is a problem a I would like optimalise this, but how can I do it?

>If you haven't run vacuum analyze lately then you should do that.

Ok, I was run it and here is result, but I understand what it means.

Can you explain it to me, please?

[dcsoft@bigmax dcsoft]$ vacuumdb --analyze --verbose --table article
evid
NOTICE:  --Relation article--
NOTICE:  Pages 24341: Changed 0, Reapped 8, Empty 0, New 0; Tup 1376029:
Vac 0, Keep/VTL 0/0, Crash 0, UnUsed 29, MinLen 112, MaxLen 166;
Re-using: Free/Avail. Space 1924/0; EndEmpty/Avail. Pages 0/0. Elapsed
3/246 sec.
NOTICE:  Index article_index: Pages 6465; Tuples 1376029: Deleted 0.
Elapsed 1/5 sec.
NOTICE:  Index hash_obj_kod: Pages 10841; Tuples 1376029: Deleted 0.
Elapsed 1/25 sec.
NOTICE:  Index article_obj_kod_key: Pages 6860; Tuples 1376029: Deleted
0. Elapsed 1/5 sec.
NOTICE:  Index article_pkey: Pages 6858; Tuples 1376029: Deleted 0.
Elapsed 1/5 sec.

>> When I use hash except btree, query as: select * from article where
>> id='something' is fast but select * from article where id='something%'
>> is very slow.

>Yup. That's because hashes only work on exact matches and btrees can do
>ordering (like 'somethink' is surely "larger" than 'something%').

Ok, I understand.

>> How can I see that postgres use/or no use index on some query? It is
>> possible?

>Yup. EXPLAIN SELECT ....

>In fact, if you can't make any EXPLAIN progress you should always accompany any
>optimizer issues with the EXPLAIN output. That will help our optimizer
>gurus. :)

OK, My table structure is:

CREATE TABLE "article" (
        "id" character varying(15) NOT NULL,
        "obj_kod" character varying(15),
        "popis" character varying(80),
        "net_price" float4,
        "our_price" float4,
        "quantity" int2,
        "group1" character varying(40) DEFAULT 'ine',CREATE TABLE
"article" (
        "id" character varying(15) NOT NULL,
        "obj_kod" character varying(15),
        "popis" character varying(80),
        "net_price" float4,
        "our_price" float4,
        "quantity" int2,
        "group1" character varying(40) DEFAULT 'ine',
        "group2" character varying(40),
        "pic1" character varying(10) DEFAULT 'noname.jpg',
        "pic2" character varying(10) DEFAULT 'noname.jpg',
        "alt1" character varying(15),
        "alt2" character varying(15),
        "zisk" int2);
REVOKE ALL on "article" from PUBLIC;
GRANT INSERT,UPDATE,DELETE,SELECT on "article" to GROUP "evid";
CREATE UNIQUE INDEX "article_pkey" on "article" using btree ( "id"
"varchar_ops" );
CREATE  INDEX "article_obj_kod_key" on "article" using btree ( "obj_kod"
"varchar_ops" );
CREATE  INDEX "hash_obj_kod" on "article" using hash ( "obj_kod"
"varchar_ops" );
CREATE  INDEX "article_index" on "article" using btree ( "obj_kod"
"varchar_ops" );

with 1376029 rows.

and here is result:

evid=> explain select * from article where id = 'something';
NOTICE:  QUERY PLAN:

Index Scan using article_pkey on article  (cost=2.05 rows=1 width=120)

EXPLAIN

I think that it is ok (fast), but next result is no.

evid=> explain select * from article where id like '186892%';
NOTICE:  QUERY PLAN:

Index Scan using article_pkey on article  (cost=33333.82 rows=1
width=120)

EXPLAIN

I think that price is very big.

If you can help me or If you have some sugestion please writte me.

With regards,

    .~.
    /V\         Davy
   // \\  [dcsoft@dcsoft.sk]
  /(   )\
   ^`~'^
... online ANG<->SK slovnik: http://www.dcsoft.sk/slovnik ...



^ permalink  raw  reply  [nested|flat] 6+ messages in thread


end of thread, other threads:[~2000-03-08 13:06 UTC | newest]

Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2000-03-05 20:11 database with 1000000 rows is very slow David Celjuska <dcsoft@dcsoft.sk>
2000-03-05 20:21 database with 1000000 rows is very slow David Celjuska <dcsoft@dcsoft.sk>
2000-03-06 07:22 ` Peter Eisentraut <e99re41@DoCS.UU.SE>
2000-03-06 15:27 RE: [SQL] database with 1000000 rows is very slow Justin Long <justinlong@strategicnetwork.org>
2000-03-08 11:11 Re: [SQL] database with 1000000 rows is very slow David Celjuska <dcsoft@dcsoft.sk>
2000-03-08 13:06 Re: [SQL] database with 1000000 rows is very slow David Celjuska <dcsoft@dcsoft.sk>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox