Subject: Re: more detailed description of tup_returned and tup_fetched
To: Masahiro Ikeda <ikedamsh@oss.nttdata.com>, pgsql-docs@lists.postgresql.org
References: <9eeeccdb-5dd7-90f9-2807-a4b5d2b76ca3@oss.nttdata.com>
 <19f98fd2-c81b-f77f-d940-2816c565f850@oss.nttdata.com>
 <ca021203-2c9b-1063-c5fb-503384d781b8@oss.nttdata.com>
 <03d26cbe-cce3-5cdf-89e6-df3769ff7728@oss.nttdata.com>
 <14c6b74b-bb73-f8eb-3ce7-3e903e651bd3@oss.nttdata.com>
 <733bdf4a-e888-41df-0437-6dfb922d4883@oss.nttdata.com>
 <ebef239a-b44f-1a5e-4548-dc055070a5dc@oss.nttdata.com>
 <dc21f9fd-52de-ba5d-24c2-c87069986f6f@oss.nttdata.com>
 <a19d3810-03e7-4cb8-2298-f2c95505b608@oss.nttdata.com>
From: Fujii Masao <masao.fujii@oss.nttdata.com>
Message-ID: <4a7fe815-7d79-aff9-19d8-ede4d0f1c10d@oss.nttdata.com>
Date: Thu, 20 May 2021 17:00:13 +0900
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:78.0)
 Gecko/20100101 Thunderbird/78.10.2
MIME-Version: 1.0
In-Reply-To: <a19d3810-03e7-4cb8-2298-f2c95505b608@oss.nttdata.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit
Archived-At: 
 <https://www.postgresql.org/message-id/4a7fe815-7d79-aff9-19d8-ede4d0f1c10d%40oss.nttdata.com>
Precedence: bulk


On 2021/05/20 9:46, Masahiro Ikeda wrote:
> 
> 
> On 2021/05/18 20:10, Fujii Masao wrote:
>>
>>
>> On 2021/05/18 18:23, Masahiro Ikeda wrote:
>>>
>>>
>>> On 2021/05/18 16:01, Fujii Masao wrote:
>>>> On 2021/05/18 13:20, Masahiro Ikeda wrote:
>>>>> Tid Range Scan increments the tup_returned, and
>>>>> pg_stat_all_tables.seq_tup_read is also incremented. I thought it's ok
>>>>> because
>>>>> Tid Range Scan is like sequential scan.
>>>>
>>>> Yes, you're right. One interesting thing I found is;
>>>> when Tid Range Scan happens, seq_tup_read is incremented
>>>> but seq_scan is not. I'm not sure if this is expected behavior or not.
>>>
>>> The following comment says that this behavior is expected. But, I agree it's
>>> odd and it's natural both seq_tup_read and seq_scan are incremented at the
>>> same time or not...
>>>
>>> /*
>>>    * Currently, we only have a stats counter for sequential heap scans (but
>>>    * e.g for bitmap scans the underlying bitmap index scans will be counted,
>>>    * and for sample scans we update stats for tuple fetches).
>>>    */
>>> if (scan->rs_base.rs_flags & SO_TYPE_SEQSCAN)
>>>      pgstat_count_heap_scan(scan->rs_base.rs_rd);
>>>
>>>
>>>>> That's the reason why the document of
>>>>> pg_stat_all_tables.seq_tup_read says "Number of live rows fetched by
>>>>> sequential scans"
>>>>
>>>> Regarding the original issue, as far as I understand correctly,
>>>>
>>>> * pg_stat_database.tup_returned = sum(pg_stat_all_tables.seq_tup_read) +
>>>> sum(pg_stat_all_indexes.idx_tup_read)
>>>> * pg_stat_database.tup_fetched = sum(pg_stat_all_tables.idx_tup_fetch)
>>>>
>>>> But the counters for some system catalogs like pg_database shared
>>>> across all databases of a cluster are excluded from that calculation.
>>>> Is this my understanding right? If right, probably we can reuse
>>>> the existing descriptions for those counters to document
>>>> pg_stat_database counters. For example,
>>>
>>> Yes, my understanding is same now.
>>>
>>>
>>>> pg_stat_database.tup_returned:> Number of live rows fetched by sequential
>>>> and index scans in this database
>>>
>>> I wonder "live rows fetched by index scans" may mislead. I think "live" means
>>> it's not dead tuple and "rows" mean the tuple user want to get.
>>>
>>> But, pg_stat_all_indexes.idx_tup_read says that "index entires returned by
>>> scans on this index". There is no meaning of "live" and "rows", so I thought
>>> it's better to distinguish them.
>>>
>>> So, why don't you change to "Number of live rows fetched by sequential scans
>>> and index entries returned by index scans in this database"?
>>
>> Yes, LGTM.
>>
>>
>>>> pg_stat_database.tup_fetched:
>>>> Number of index entries returned by scans on indexes in this database
>>> Is this the sum of pg_stat_all_indexes.idx_tup_read? This is accounted to
>>> pg_stat_database.tup_returned.
>>
>> I was thinking that pg_stat_database.tup_fetched is the same as
>> the sum of pg_stat_all_tables.idx_tup_fetch. Because they both
>> are incremented by bitmap index scans, but pg_stat_all_indexes.idx_tup_read
>> is not.
> 
> Yes. So, "Number of index entries returned by scans on indexes in this
> database" is incorrect, and "Number of live rows fetched by index scans in
> this database" is correct?

Yes, I think so!

Regards,

-- 
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION