Subject: Re: shared-memory based stats collector
From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp>
Cc: alvherre@2ndquadrant.com, andres@anarazel.de, ah@cybertec.at,
 magnus@hagander.net, robertmhaas@gmail.com, tgl@sss.pgh.pa.us,
 pgsql-hackers@postgresql.org
References: <20181112.201042.147595779.horiguchi.kyotaro@lab.ntt.co.jp>
 <8e041a34a70ae5d44cb10a7e516308e2f6d1b651.camel@2ndquadrant.com>
 <6c079a69-feba-e47c-7b85-8a9ff31adef3@2ndquadrant.com>
 <20181127.175949.06807946.horiguchi.kyotaro@lab.ntt.co.jp>
 <06f751c9-e400-3742-83f1-3f2292297622@2ndquadrant.com>
Message-ID: <b760035b-1941-38bb-5e84-c2fbc63fef6b@2ndquadrant.com>
Date: Sun, 20 Jan 2019 18:13:04 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101
 Thunderbird/60.4.0
MIME-Version: 1.0
In-Reply-To: <06f751c9-e400-3742-83f1-3f2292297622@2ndquadrant.com>
Content-Type: text/plain; charset=windows-1252
Content-Language: en-US
Content-Transfer-Encoding: 8bit
Precedence: bulk


Hi,

The patch needs rebasing, as it got broken by 285d8e1205, and there's
some other minor bitrot.

On 11/27/18 4:40 PM, Tomas Vondra wrote:
> On 11/27/18 9:59 AM, Kyotaro HORIGUCHI wrote:
>>>
>>> ...>>
>>> For the main workload there's pretty much no difference, but for
>>> selects from the stats catalogs there's ~20% drop in throughput.
>>> In absolute numbers this means drop from ~670tps to ~550tps. I
>>> haven't investigated this, but I suppose this is due to dshash
>>> seqscan being more expensive than reading the data from file.
>>
>> Thanks for finding that. The three seqscan loops in 
>> pgstat_vacuum_stat cannot take such a long time, I think. I'll 
>> investigate it.
>>
> 
> OK. I'm not sure this is related to pgstat_vacuum_stat - the
> slowdown happens while querying the catalogs, so why would that
> trigger vacuum of the stats? I may be missing something, of course.
> 
> FWIW, the "query statistics" test simply does this:
> 
>   SELECT * FROM pg_stat_all_tables;
>   SELECT * FROM pg_stat_all_indexes;
>   SELECT * FROM pg_stat_user_indexes;
>   SELECT * FROM pg_stat_user_tables;
>   SELECT * FROM pg_stat_sys_tables;
>   SELECT * FROM pg_stat_sys_indexes;
> 
> and the slowdown happened even it was running on it's own (nothing
> else running on the instance). Which mostly rules out concurrency
> issues with the hash table locking etc.
> 

Did you have time to investigate the slowdown?

regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services