pg.ddx.io  pgsql-bugs@postgresql.org mailing list archive  
help / color / mirror / Atom feed
BUG #16089: Index only scan does not happen but expected
6+ messages / 3 participants
[nested] [flat]

* BUG #16089: Index only scan does not happen but expected
@ 2019-10-30 12:54  PG Bug reporting form <noreply@postgresql.org>
  0 siblings, 1 reply; 6+ messages in thread

From: PG Bug reporting form @ 2019-10-30 12:54 UTC (permalink / raw)
  To: pgsql-bugs@lists.postgresql.org; +Cc: stepya@ukr.net

The following bug has been logged on the website:

Bug reference:      16089
Logged by:          Stepan Yankevych
Email address:      stepya@ukr.net
PostgreSQL version: 11.5
Operating system:   CentOS Linux release 7.3.1611 (Core)
Description:        

Not a real issue but rather performance leak.
The issue is reproducible on the version 11.5 and 12.0 as well. 
See script of partitioned table. Please notice PK and index contains
start_date_id as second field.
 
CREATE TABLE if not exists dwh.l1_snapshot_tail2 (
	l1_snapshot_id int8 NOT NULL,
	start_date_id int4 NOT NULL,
	transaction_id int8 NOT NULL,
	exchange_id varchar(6) NULL,
	instrument_id int4 NULL,
	bid_price numeric(12,4) NULL,
	ask_price numeric(12,4) NULL,
	bid_quantity int8 NULL,
	ask_quantity int8 NULL,
	dataset_id int4 NULL,
	transaction_time timestamp NULL,
	CONSTRAINT pk_l1_snapshot PRIMARY KEY (l1_snapshot_id, start_date_id)
)
PARTITION BY RANGE (start_date_id);
CREATE INDEX if not exists l1_snapshot_transact_date_idx ON 
dwh.l1_snapshot_tail2 USING btree (transaction_id, start_date_id);


CREATE TABLE partitions.l1_snapshot_201805 PARTITION OF
dwh.l1_snapshot_tail2 FOR VALUES FROM (20180501) TO (20180601);
CREATE TABLE partitions.l1_snapshot_201806 PARTITION OF
dwh.l1_snapshot_tail2 FOR VALUES FROM (20180601) TO (20180701);
CREATE TABLE partitions.l1_snapshot_201807 PARTITION OF
dwh.l1_snapshot_tail2 FOR VALUES FROM (20180701) TO (20180801);
CREATE TABLE partitions.l1_snapshot_201808 PARTITION OF
dwh.l1_snapshot_tail2 FOR VALUES FROM (20180801) TO (20180901);
CREATE TABLE partitions.l1_snapshot_201809 PARTITION OF
dwh.l1_snapshot_tail2 FOR VALUES FROM (20180901) TO (20181001);
CREATE TABLE partitions.l1_snapshot_201810 PARTITION OF
dwh.l1_snapshot_tail2 FOR VALUES FROM (20181001) TO (20181101);
CREATE TABLE partitions.l1_snapshot_201811 PARTITION OF
dwh.l1_snapshot_tail2 FOR VALUES FROM (20181101) TO (20181201);
CREATE TABLE partitions.l1_snapshot_201812 PARTITION OF
dwh.l1_snapshot_tail2 FOR VALUES FROM (20181201) TO (20190101);

i have loaded 500000 rows with date_id = 20181112
Real table has 20-40 millions records a day.

Examine query 
select start_date_id, count(start_date_id) as cn
from dwh.l1_snapshot_tail2
where start_date_id between 20181112 and 20181112
group by start_date_id

The execution plan shows reading full partitions.l1_snapshot_201811
Why do we need to read data from table. 
We have all needed information in the index that is smaller. 
I would expect index only scan (something like Oracle version of index fast
full scan )



^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* Re: BUG #16089: Index only scan does not happen but expected
@ 2019-10-30 14:13  Dmitry Dolgov <9erthalion6@gmail.com>
  parent: PG Bug reporting form <noreply@postgresql.org>
  0 siblings, 1 reply; 6+ messages in thread

From: Dmitry Dolgov @ 2019-10-30 14:13 UTC (permalink / raw)
  To: stepya@ukr.net; pgsql-bugs@lists.postgresql.org

Thank you for the report.

> On Wed, Oct 30, 2019 at 12:54:31PM +0000, PG Bug reporting form wrote:
> The following bug has been logged on the website:
>
> Bug reference:      16089
> Logged by:          Stepan Yankevych
> Email address:      stepya@ukr.net
> PostgreSQL version: 11.5
> Operating system:   CentOS Linux release 7.3.1611 (Core)
> Description:
>
> Not a real issue but rather performance leak. The issue is
> reproducible on the version 11.5 and 12.0 as well.

Does it mean, that on the previous versions you observed different
behaviour?

> The execution plan shows reading full partitions.l1_snapshot_201811
> Why do we need to read data from table. We have all needed information
> in the index that is smaller. I would expect index only scan
> (something like Oracle version of index fast full scan )

After a few experiments with this schema it looks like planner sometimes
prefers seq scan (parallel seq scan) instead of using the index due to
random read being more costly than sequential reads, even if it's
necessary to read more pages. And in fact at least in my tests this was
indeed faster. If you want to try out, it's possible to set
random_page_cost lower and seq_page_cost higher, and planner will most
likely choose a different plan, but whether it would be better or not
is not clear.





^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* RE: BUG #16089: Index only scan does not happen but expected
@ 2019-10-30 14:18  Stepan Yankevych <Stepan_Yankevych@epam.com>
  parent: Dmitry Dolgov <9erthalion6@gmail.com>
  0 siblings, 1 reply; 6+ messages in thread

From: Stepan Yankevych @ 2019-10-30 14:18 UTC (permalink / raw)
  To: Dmitry Dolgov <9erthalion6@gmail.com>; stepya@ukr.net <stepya@ukr.net>; pgsql-bugs@lists.postgresql.org <pgsql-bugs@lists.postgresql.org>

>> And in fact at least in my tests this was indeed faster
Can you show execution plan when index is used?
Does it uses Index only scan or index scan and table scan ? 

STEPAN YANKEVYCH 
Software Engineering Team Leader 
Software Engineering Manager
OCA 
 
Office: +380 322 424 642 x 58840   Cell: +380 96 915 9551   Email: stepan_yankevych@epam.com 
Lviv, Ukraine   epam.com 
 
 
CONFIDENTIALITY CAUTION AND DISCLAIMER
This message is intended only for the use of the individual(s) or entity(ies) to which it is addressed and contains information that is legally privileged and confidential. If you are not the intended recipient, or the person responsible for delivering the message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. All unintended recipients are obliged to delete this message and destroy any printed copies. 
 


-----Original Message-----
From: Dmitry Dolgov <9erthalion6@gmail.com> 
Sent: Wednesday, October 30, 2019 16:13
To: stepya@ukr.net; pgsql-bugs@lists.postgresql.org
Subject: Re: BUG #16089: Index only scan does not happen but expected

Thank you for the report.

> On Wed, Oct 30, 2019 at 12:54:31PM +0000, PG Bug reporting form wrote:
> The following bug has been logged on the website:
>
> Bug reference:      16089
> Logged by:          Stepan Yankevych
> Email address:      stepya@ukr.net
> PostgreSQL version: 11.5
> Operating system:   CentOS Linux release 7.3.1611 (Core)
> Description:
>
> Not a real issue but rather performance leak. The issue is 
> reproducible on the version 11.5 and 12.0 as well.

Does it mean, that on the previous versions you observed different behaviour?

> The execution plan shows reading full partitions.l1_snapshot_201811 
> Why do we need to read data from table. We have all needed information 
> in the index that is smaller. I would expect index only scan 
> (something like Oracle version of index fast full scan )

After a few experiments with this schema it looks like planner sometimes prefers seq scan (parallel seq scan) instead of using the index due to random read being more costly than sequential reads, even if it's necessary to read more pages. And in fact at least in my tests this was indeed faster. If you want to try out, it's possible to set random_page_cost lower and seq_page_cost higher, and planner will most likely choose a different plan, but whether it would be better or not is not clear.





^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* Re: BUG #16089: Index only scan does not happen but expected
@ 2019-10-30 15:10  Dmitry Dolgov <9erthalion6@gmail.com>
  parent: Stepan Yankevych <Stepan_Yankevych@epam.com>
  0 siblings, 1 reply; 6+ messages in thread

From: Dmitry Dolgov @ 2019-10-30 15:10 UTC (permalink / raw)
  To: Stepan Yankevych <Stepan_Yankevych@epam.com>; +Cc: stepya@ukr.net <stepya@ukr.net>; pgsql-bugs@lists.postgresql.org <pgsql-bugs@lists.postgresql.org>

> On Wed, Oct 30, 2019 at 02:18:09PM +0000, Stepan Yankevych wrote:
>
> >> And in fact at least in my tests this was indeed faster
> Can you show execution plan when index is used?
> Does it uses Index only scan or index scan and table scan?

Yes, at the bottom of the plan there was an Index Only Scan going
through l1_snapshot_201811 (although while populating the table I've set
values only for not nullable columns).





^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* Re: BUG #16089: Index only scan does not happen but expected
@ 2019-10-30 15:14  Dmitry Dolgov <9erthalion6@gmail.com>
  parent: Dmitry Dolgov <9erthalion6@gmail.com>
  0 siblings, 1 reply; 6+ messages in thread

From: Dmitry Dolgov @ 2019-10-30 15:14 UTC (permalink / raw)
  To: Stepan Yankevych <Stepan_Yankevych@epam.com>; +Cc: stepya@ukr.net <stepya@ukr.net>; pgsql-bugs@lists.postgresql.org <pgsql-bugs@lists.postgresql.org>

> On Wed, Oct 30, 2019 at 04:10:06PM +0100, Dmitry Dolgov wrote:
> > On Wed, Oct 30, 2019 at 02:18:09PM +0000, Stepan Yankevych wrote:
> >
> > >> And in fact at least in my tests this was indeed faster
> > Can you show execution plan when index is used?
> > Does it uses Index only scan or index scan and table scan?
> 
> Yes, at the bottom of the plan there was an Index Only Scan going
> through l1_snapshot_201811 (although while populating the table I've set
> values only for not nullable columns).

To clarify, Index Only Scan was there only after changing
random_page_cost/seq_page_cost. As I mentioned in the previous email,
original plan was also parallel seq scan.





^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* RE: BUG #16089: Index only scan does not happen but expected
@ 2019-10-30 15:27  Stepan Yankevych <Stepan_Yankevych@epam.com>
  parent: Dmitry Dolgov <9erthalion6@gmail.com>
  0 siblings, 0 replies; 6+ messages in thread

From: Stepan Yankevych @ 2019-10-30 15:27 UTC (permalink / raw)
  To: Dmitry Dolgov <9erthalion6@gmail.com>; +Cc: stepya@ukr.net <stepya@ukr.net>; pgsql-bugs@lists.postgresql.org <pgsql-bugs@lists.postgresql.org>


 It would be great to have sequential scan of index in Postgres like we have Index Fast Full Scan in Oracle.
 Oracle Index FFS reads whole index and only index in the same way as full table scan (by several sequential blocks (defined by parameter) ) 


STEPAN YANKEVYCH 
Software Engineering Team Leader 
Software Engineering Manager
OCA 
 
Office: +380 322 424 642 x 58840   Cell: +380 96 915 9551   Email: stepan_yankevych@epam.com 
Lviv, Ukraine   epam.com 
 
 
CONFIDENTIALITY CAUTION AND DISCLAIMER
This message is intended only for the use of the individual(s) or entity(ies) to which it is addressed and contains information that is legally privileged and confidential. If you are not the intended recipient, or the person responsible for delivering the message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. All unintended recipients are obliged to delete this message and destroy any printed copies. 
 


-----Original Message-----
From: Dmitry Dolgov <9erthalion6@gmail.com> 
Sent: Wednesday, October 30, 2019 17:15
To: Stepan Yankevych <Stepan_Yankevych@epam.com>
Cc: stepya@ukr.net; pgsql-bugs@lists.postgresql.org
Subject: Re: BUG #16089: Index only scan does not happen but expected

> On Wed, Oct 30, 2019 at 04:10:06PM +0100, Dmitry Dolgov wrote:
> > On Wed, Oct 30, 2019 at 02:18:09PM +0000, Stepan Yankevych wrote:
> >
> > >> And in fact at least in my tests this was indeed faster
> > Can you show execution plan when index is used?
> > Does it uses Index only scan or index scan and table scan?
> 
> Yes, at the bottom of the plan there was an Index Only Scan going 
> through l1_snapshot_201811 (although while populating the table I've 
> set values only for not nullable columns).

To clarify, Index Only Scan was there only after changing random_page_cost/seq_page_cost. As I mentioned in the previous email, original plan was also parallel seq scan.





^ permalink  raw  reply  [nested|flat] 6+ messages in thread


end of thread, other threads:[~2019-10-30 15:27 UTC | newest]

Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2019-10-30 12:54 BUG #16089: Index only scan does not happen but expected PG Bug reporting form <noreply@postgresql.org>
2019-10-30 14:13 ` Dmitry Dolgov <9erthalion6@gmail.com>
2019-10-30 14:18   ` Stepan Yankevych <Stepan_Yankevych@epam.com>
2019-10-30 15:10     ` Dmitry Dolgov <9erthalion6@gmail.com>
2019-10-30 15:14       ` Dmitry Dolgov <9erthalion6@gmail.com>
2019-10-30 15:27         ` Stepan Yankevych <Stepan_Yankevych@epam.com>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox