Content-Type: multipart/alternative;
 boundary="------------t3iQk71I4TiklzNgzFlIt6Ra"
Message-ID: <52ca6a89-d19d-4df9-ac4b-95548500bf8b@tantorlabs.com>
Date: Thu, 7 Nov 2024 17:49:23 +0300
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Vacuum statistics
To: Alena Rybakina <a.rybakina@postgrespro.ru>,
 pgsql-hackers <pgsql-hackers@postgresql.org>
Cc: jian he <jian.universality@gmail.com>,
 Alexander Korotkov <aekorotkov@gmail.com>, Andrei Zubkov
 <zubkov@moonset.ru>, Alena Rybakina <lena.ribackina@yandex.ru>,
 a.lepikhov@postgrespro.ru
References: <bee34abe-fe87-49b3-8523-8ae19e641ccf@yandex.ru>
 <53c47c2d-72a5-44f2-900c-9973b2af1808@tantorlabs.com>
 <c82e7776-7841-42a0-9247-878584174750@tantorlabs.com>
 <4a902cea-54fb-41b5-b208-b84731a5f577@postgrespro.ru>
 <CACJufxHb_YGCp=pVH6DZcpk9yML+SueffPeaRbX2LzXZVahd_w@mail.gmail.com>
 <CACJufxE6yAP+jUm4_GyKp7gUCkzbuJ9HGB-rR=92_hcLLZ9KTg@mail.gmail.com>
 <092adec6-4eae-4bd4-bd0d-473a9df1282b@tantorlabs.com>
 <3deae1bd-ad84-4459-a26e-04c9136b84e9@postgrespro.ru>
 <CAPpHfdug0s2MD7bBf-5nDQGn1WBxCKiTmZyGfxHz_7P0CDOjbg@mail.gmail.com>
 <c4e4e305-7119-4183-b49a-d7092f4efba3@postgrespro.ru>
 <f05a9031-f923-41b1-9bf9-81204cb56561@postgrespro.ru>
 <CACJufxFU4ej3iGtGg3GDqmGzRmTVq3d9RGq+ibLfQoS8E3hJEQ@mail.gmail.com>
 <9b10c6d3-52c4-4eef-b67c-c33442667729@postgrespro.ru>
 <30e2e941-b9f6-4ea1-aa3b-f0c406b8797d@postgrespro.ru>
 <1a960398-d12d-419a-af48-4acfe58d0c3a@tantorlabs.com>
 <8b2f3c20-02ac-4658-b78f-e9d7590f893b@postgrespro.ru>
 <995657bc-9966-47c0-b085-4c5e8886d249@postgrespro.ru>
Content-Language: en-US
From: Ilia Evdokimov <ilya.evdokimov@tantorlabs.com>
In-Reply-To: <995657bc-9966-47c0-b085-4c5e8886d249@postgrespro.ru>
Archived-At: <https://www.postgresql.org/message-id/52ca6a89-d19d-4df9-ac4b-95548500bf8b%40tantorlabs.com>
Precedence: bulk

This is a multi-part message in MIME format.
--------------t3iQk71I4TiklzNgzFlIt6Ra
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit


On 22.10.2024 22:30, Alena Rybakina wrote:
>
> Hi!
>
> On 16.10.2024 14:01, Alena Rybakina wrote:
>>>
>>> Thank you for rebasing.
>>>
>>> I have noticed that when I create a table or an index on this table, 
>>> there is no information about the table or index in 
>>> pg_stat_vacuum_tables and pg_stat_vacuum_indexes until we perform a 
>>> VACUUM.
>>>
>>> Example:
>>>
>>> CREATE TABLE t (i INT, j INT);
>>> INSERT INTO t SELECT i/10, i/100 FROM GENERATE_SERIES(1,1000000) i;
>>> SELECT * FROM pg_stat_vacuum_tables WHERE relname = 't';
>>> ....
>>> (0 rows)
>>> CREATE INDEX ON t (i);
>>> SELECT * FROM pg_stat_vacuum_indexes WHERE relname = 't_i_idx';
>>> ...
>>> (0 rows)
>>>
>>> I can see the entries after running VACUUM or executing 
>>> autovacuum. or when autovacuum is executed. I would suggest adding a 
>>> line about the relation even if it has not yet been processed by 
>>> vacuum. Interestingly, this issue does not occur with 
>>> pg_stat_vacuum_database:
>>>
>>> CREATE DATABASE example_db;
>>> SELECT * FROM pg_stat_vacuum_database WHERE dbname = 'example_db';
>>> dboid |       dbname | ...
>>>  ...      | example_db | ...
>>> (1 row)
>>>
>>> BTW, I recommend renaming the view pg_stat_vacuum_database to 
>>> pg_stat_vacuum_database_S_  for consistency with 
>>> pg_stat_vacuum_tables and pg_stat_vacuum_indexes
>>>
>> Thanks for the review. I'm investigating this. I agree with the 
>> renaming, I will do it in the next version of the patch.
>>
> I fixed it. I added the left outer join to the vacuum views and for 
> converting the coalesce function from NULL to null values.
>
> I also fixed the code in getting database statistics - we can get it 
> through the existing pgstat_fetch_stat_dbentry function and fixed 
> couple of comments.
>
> I attached a diff file, as well as new versions of patches.
>
> -- 
> Regards,
> Alena Rybakina
> Postgres Professional

Thank you for fixing it.

1) I have found some typos in the test output files (out-files) when 
running 'make check' and 'make check-world'. These typos might cause 
minor discrepancies in test results. You may already be aware of them, 
but I wanted to bring them to your attention in case they haven't been 
noticed. I believe these can be fixed quickly.

2) Additionally, I observed that when we create a table and insert some 
rows, executing the VACUUM FULL command does not update the information 
in the 'pg_stat_get_vacuum_tables' However, running the VACUUM command 
does update this information as expected. This seems inconsistent, and 
it might be a bug.

Example:
CREATE TABLE t (i INT, j INT) WITH (autovacuum_enabled = false);
INSERT INTO t SELECT i/10, i/100 FROM  GENERATE_SERIES(1,1000000) i;
SELECT * FROM pg_stat_get_vacuum_tables WHERE relname = 't';
schema | relname |    relid | total_blks_read | .........
-----------+------------+---------+----------------------+---------
    public | t            | 21416 |                       0 | ......
(1 row)

VACUUM FULL;
SELECT * FROM pg_stat_get_vacuum_tables WHERE relname = 't';
schema | relname |    relid | total_blks_read | .........
-----------+------------+---------+----------------------+---------
    public | t            | 21416 |                       0 | ......
(1 row)

VACUUM;
SELECT * FROM pg_stat_get_vacuum_tables WHERE relname = 't';
schema | relname |    relid | total_blks_read | .........
-----------+------------+---------+----------------------+---------
    public | t            | 21416 |                 4425 | ......
(1 row)

Regards,
Ilia Evdokimov,
Tantor Labs LLC.

--------------t3iQk71I4TiklzNgzFlIt6Ra
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 8bit

<!DOCTYPE html>
<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  </head>
  <body>
    <p><br>
    </p>
    <div class="moz-cite-prefix">On 22.10.2024 22:30, Alena Rybakina
      wrote:<br>
    </div>
    <blockquote type="cite"
      cite="mid:995657bc-9966-47c0-b085-4c5e8886d249@postgrespro.ru">
      <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
      <p>Hi!<br>
      </p>
      <div class="moz-cite-prefix">On 16.10.2024 14:01, Alena Rybakina
        wrote:<br>
      </div>
      <blockquote type="cite"
        cite="mid:8b2f3c20-02ac-4658-b78f-e9d7590f893b@postgrespro.ru">
        <blockquote type="cite"
          cite="mid:1a960398-d12d-419a-af48-4acfe58d0c3a@tantorlabs.com">
          <p>Thank you for rebasing.</p>
          <p>I have noticed that when I create a table or an index on
            this table, there is no information about the table or index
            in pg_stat_vacuum_tables and pg_stat_vacuum_indexes until we
            perform a VACUUM.</p>
          <p>Example:</p>
          <p>CREATE TABLE t (i INT, j INT);<br>
            INSERT INTO t SELECT i/10, i/100 FROM 
            GENERATE_SERIES(1,1000000) i;<br>
            SELECT * FROM pg_stat_vacuum_tables WHERE relname = 't';<br>
            ....<br>
            (0 rows)<br>
            CREATE INDEX ON t (i);<br>
            SELECT * FROM pg_stat_vacuum_indexes WHERE relname =
            't_i_idx';<br>
            ...<br>
            (0 rows)</p>
          <p>I can see the entries after running VACUUM or executing
            autovacuum. or when autovacuum is executed. I would suggest
            adding a line about the relation even if it has not yet been
            processed by vacuum. Interestingly, this issue does not
            occur with pg_stat_vacuum_database:</p>
          <p>CREATE DATABASE example_db;<br>
            SELECT * FROM pg_stat_vacuum_database WHERE dbname =
            'example_db';<br>
            dboid |       dbname | ...<br>
             ...      | example_db | ...<br>
            (1 row)<br>
            <br>
            BTW, I recommend renaming the view pg_stat_vacuum_database
            to pg_stat_vacuum_database<u>S</u>  for consistency with
            pg_stat_vacuum_tables and pg_stat_vacuum_indexes</p>
        </blockquote>
        <p>Thanks for the review. I'm investigating this. I agree with
          the renaming, I will do it in the next version of the patch.<br>
        </p>
      </blockquote>
      <p>I fixed it. I added the left outer join to the vacuum views and
        for converting the coalesce function from NULL to null values.</p>
      <p>I also fixed the code in getting database statistics - we can
        get it through the existing pgstat_fetch_stat_dbentry function
        and fixed couple of comments. </p>
      <p>I attached a diff file, as well as new versions of patches.<br>
      </p>
      <pre class="moz-signature" cols="72">-- 
Regards,
Alena Rybakina
Postgres Professional</pre>
    </blockquote>
    <p>Thank you for fixing it.</p>
    <p>1) I have found some typos in the test output files (out-files)
      when running 'make check' and 'make check-world'. These typos
      might cause minor discrepancies in test results. You may already
      be aware of them, but I wanted to bring them to your attention in
      case they haven't been noticed. I believe these can be fixed
      quickly.</p>
    <p>2) Additionally, I observed that when we create a table and
      insert some rows, executing the VACUUM FULL command does not
      update the information in the 'pg_stat_get_vacuum_tables' However,
      running the VACUUM command does update this information as
      expected. This seems inconsistent, and it might be a bug.</p>
    <p>Example:<br>
      CREATE TABLE t (i INT, j INT) WITH (autovacuum_enabled = false);<br>
      INSERT INTO t SELECT i/10, i/100 FROM  GENERATE_SERIES(1,1000000)
      i;<br>
      SELECT * FROM pg_stat_get_vacuum_tables WHERE relname = 't';<br>
      schema | relname |    relid | total_blks_read | ......... <br>
-----------+------------+---------+----------------------+---------<br>
         public | t            | 21416 |                       0 |
      ......<br>
      (1 row)<br>
      <br>
      VACUUM FULL;<br>
      SELECT * FROM pg_stat_get_vacuum_tables WHERE relname = 't';<br>
      schema | relname |    relid | total_blks_read | ......... <br>
-----------+------------+---------+----------------------+---------<br>
         public | t            | 21416 |                       0 |
      ......<br>
      (1 row)<br>
      <br>
      VACUUM;<br>
      SELECT * FROM pg_stat_get_vacuum_tables WHERE relname = 't';<br>
      schema | relname |    relid | total_blks_read | ......... <br>
-----------+------------+---------+----------------------+---------<br>
         public | t            | 21416 |                 4425 | ......<br>
      (1 row)</p>
    <p>Regards,<br>
      Ilia Evdokimov,<br>
      Tantor Labs LLC.<br>
    </p>
  </body>
</html>

--------------t3iQk71I4TiklzNgzFlIt6Ra--