Subject: Re: Protect syscache from bloating with negative cache entries
To: "Tsunakawa, Takayuki" <tsunakawa.takay@jp.fujitsu.com>,
 'Kyotaro HORIGUCHI' <horiguchi.kyotaro@lab.ntt.co.jp>
Cc: "alvherre@2ndquadrant.com" <alvherre@2ndquadrant.com>,
 "bruce@momjian.us" <bruce@momjian.us>,
 "andres@anarazel.de" <andres@anarazel.de>,
 "robertmhaas@gmail.com" <robertmhaas@gmail.com>,
 "tgl@sss.pgh.pa.us" <tgl@sss.pgh.pa.us>,
 "Ideriha, Takeshi" <ideriha.takeshi@jp.fujitsu.com>,
 "pgsql-hackers@lists.postgresql.org" <pgsql-hackers@lists.postgresql.org>,
 "michael.paquier@gmail.com" <michael.paquier@gmail.com>,
 "david@pgmasters.net" <david@pgmasters.net>,
 "craig@2ndquadrant.com" <craig@2ndquadrant.com>
References: <20190207.211845.263152487.horiguchi.kyotaro@lab.ntt.co.jp>
 <0A3221C70F24FB45833433255569204D1FB972A6@G01JPEXMBYT05>
 <20190212.203628.118792892.horiguchi.kyotaro@lab.ntt.co.jp>
 <20190213.153114.239737674.horiguchi.kyotaro@lab.ntt.co.jp>
 <0A3221C70F24FB45833433255569204D1FB99888@G01JPEXMBYT05>
 <0A3221C70F24FB45833433255569204D1FB9D37E@G01JPEXMBYT05>
From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
Message-ID: <d34689a2-12fd-5a56-2805-22f5b20d6c63@2ndquadrant.com>
Date: Tue, 19 Feb 2019 12:53:25 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101
 Thunderbird/60.5.0
MIME-Version: 1.0
In-Reply-To: <0A3221C70F24FB45833433255569204D1FB9D37E@G01JPEXMBYT05>
Content-Type: text/plain; charset=iso-2022-jp
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Precedence: bulk


On 2/19/19 12:43 AM, Tsunakawa, Takayuki wrote:
> Hi Horiguchi-san,
> 
> I've looked through your patches.  This is the first part of my review results.  Let me post the rest after another work today.
> 
> BTW, how about merging 0003 and 0005, and separating and deferring 0004 in another thread?  That may help to relieve other community members by making this patch set not so large and complex.
> 
> 
> 
> [Bottleneck investigation]
> Ideriha-san and I are trying to find the bottleneck.  My first try shows there's little overhead.  Here's what I did:
> 
> <postgresql.conf>
> shared_buffers = 1GB
> catalog_cache_prune_min_age = -1
> catalog_cache_max_size = 10MB
> 
> <benchmark>
> $ pgbench -i -s 10
> $ pg_ctl stop and then start
> $ cache all data in shared buffers by running pg_prewarm on branches, tellers, accounts, and their indexes
> $ pgbench --select-only -c 1 -T 60
> 
> <result>
> master : 8612 tps
> patched: 8553 tps (-0.7%)
> 
> There's little (0.7%) performance overhead with:
> * one additional dlist_move_tail() in every catcache access
> * memory usage accounting in operations other than catcache access (relevant catcache entries should be cached in the first pgbench transaction)
> 
> I'll check other patterns to find out how big overhead there is.
> 

0.7% may easily be just a noise, possibly due to differences in layout
of the binary. How many runs? What was the variability of the results
between runs? What hardware was this tested on?

FWIW I doubt tests with such small small schema are proving anything -
the cache/lists are likely tiny. That's why I tested with much larger
number of relations.

regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services