pg.ddx.io pgsql-hackers@postgresql.org mailing list archive help / color / mirror / Atom feed From: Alvaro Herrera <alvherre@2ndquadrant.com>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Simon Riggs <simon@2ndquadrant.com>
Cc: Pg Hackers <pgsql-hackers@lists.postgresql.org>
Cc: David Fetter <david@fetter.org>
Cc: Rahila Syed <rahila.syed@2ndquadrant.com>
Cc: Pavan Deolasee <pavan.deolasee@gmail.com>
Cc: Amit Langote <Langote_Amit_f8@lab.ntt.co.jp>
Subject: Re: monitoring CREATE INDEX [CONCURRENTLY]
Date: Fri, 29 Mar 2019 12:12:44 -0300
Message-ID: <20190329151244.GA28626@alvherre.pgsql> (raw )
In-Reply-To: <20190329150218.GA25010@alvherre.pgsql >
On 2019-Mar-29, Alvaro Herrera wrote:
> I just noticed that the CLUSTER calls index_build, which my patch
> modifies to include additional progress metrics; this means that during
> the index build phase, the metrics set by CLUSTER will be trashed by the
> ones my patch introduces.
Indeed:
pid | datid | datname | relid | command | phase | cluster_index_relid | heap_tuples_scanned | heap_tuples_written | heap_blks_total | heap_blks_scanned | index_rebuild_count
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 162402 | 162402 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 460362 | 460362 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 754004 | 754004 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 1047058 | 1047058 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 1356296 | 1356296 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 1645321 | 1645321 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 1939920 | 1939920 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 2227450 | 2227450 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 2526116 | 2526116 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 2828468 | 2828468 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 3142982 | 3142982 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 3451494 | 3451494 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 3769799 | 3769799 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 4077513 | 4077513 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 4383255 | 4383255 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 4700286 | 4700286 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 5015468 | 5015468 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 5324951 | 5324951 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 5628172 | 5628172 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 5940862 | 5940862 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 6253778 | 6253778 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 6560474 | 6560474 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 6881248 | 6881248 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 7186555 | 7186555 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 7503888 | 7503888 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 7821706 | 7821706 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 8137334 | 8137334 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 8453280 | 8453280 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 8764097 | 8764097 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 9079908 | 9079908 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 9389465 | 9389465 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 9706288 | 9706288 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 10024875 | 10024875 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 10344875 | 10344875 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 10658531 | 10658531 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 10974097 | 10974097 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 11289868 | 11289868 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 11602965 | 11602965 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 11917624 | 11917624 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 12237998 | 12237998 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 12559406 | 12559406 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 12876413 | 12876413 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 13192793 | 13192793 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 13506188 | 13506188 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 13822722 | 13822722 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 14138624 | 14138624 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 14432335 | 14432335 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 16387 | 14753617 | 14753617 | 0 | 0 | 0
-->
pid | datid | datname | relid | command | phase | cluster_index_relid | heap_tuples_scanned | heap_tuples_written | heap_blks_total | heap_blks_scanned | index_rebuild_count
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 2 | 15000000 | 15000000 | 0 | 66372 | 743
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 2 | 15000000 | 15000000 | 0 | 66372 | 4639
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 2 | 15000000 | 15000000 | 0 | 66372 | 8938
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 2 | 15000000 | 15000000 | 0 | 66372 | 13457
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 2 | 15000000 | 15000000 | 0 | 66372 | 17806
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 2 | 15000000 | 15000000 | 0 | 66372 | 22237
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 2 | 15000000 | 15000000 | 0 | 66372 | 26594
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 2 | 15000000 | 15000000 | 0 | 66372 | 30935
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 2 | 15000000 | 15000000 | 0 | 66372 | 35362
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 2 | 15000000 | 15000000 | 0 | 66372 | 39705
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 2 | 15000000 | 15000000 | 0 | 66372 | 44126
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 2 | 15000000 | 15000000 | 0 | 66372 | 48546
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 2 | 15000000 | 15000000 | 0 | 66372 | 52966
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 2 | 15000000 | 15000000 | 0 | 66372 | 57174
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 2 | 15000000 | 15000000 | 0 | 66372 | 61536
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 2 | 15000000 | 15000000 | 0 | 66372 | 65868
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 2 | 15000000 | 15000000 | 0 | 66372 | 66193
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 2 | 15000000 | 15000000 | 0 | 66372 | 66193
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 2 | 15000000 | 15000000 | 0 | 66372 | 66193
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 2 | 15000000 | 15000000 | 0 | 66372 | 66193
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 2 | 15000000 | 15000000 | 0 | 66372 | 66193
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 5 | 15000000 | 15000000 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 5 | 15000000 | 15000000 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 5 | 15000000 | 15000000 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 5 | 15000000 | 15000000 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 5 | 15000000 | 15000000 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 5 | 15000000 | 15000000 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 5 | 15000000 | 15000000 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 5 | 15000000 | 15000000 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 5 | 15000000 | 15000000 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 5 | 15000000 | 15000000 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 5 | 15000000 | 15000000 | 0 | 0 | 0
28457 | 12750 | alvherre | 16384 | CLUSTER | index scanning heap | 5 | 15000000 | 15000000 | 0 | 0 | 0
I suppose I can just pick non-overlapping numbers for those columns.
--
Álvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
view thread (60+ messages) latest in thread
Message-ID: <20190329151244.GA28626@alvherre.pgsql>
Permalink: ../20190329151244.GA28626@alvherre.pgsql/
Also on: postgresql.org/message-id/20190329151244.GA28626@alvherre.pgsql
copy link · copy postgr.es
reply Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: alvherre@2ndquadrant.com, robertmhaas@gmail.com, simon@2ndquadrant.com, pgsql-hackers@lists.postgresql.org, david@fetter.org, rahila.syed@2ndquadrant.com, pavan.deolasee@gmail.com, Langote_Amit_f8@lab.ntt.co.jp
Subject: Re: monitoring CREATE INDEX [CONCURRENTLY]
In-Reply-To: <20190329151244.GA28626@alvherre.pgsql>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox