Subject: Re: [PROPOSAL] Shared Ispell dictionaries
To: Robert Haas <robertmhaas@gmail.com>
Cc: Tomas Vondra <tomas.vondra@2ndquadrant.com>,
 Andres Freund <andres@anarazel.de>, Tom Lane <tgl@sss.pgh.pa.us>,
 Pavel Stehule <pavel.stehule@gmail.com>,
 pgsql-hackers <pgsql-hackers@postgresql.org>
References: <27296.1522078068@sss.pgh.pa.us>
 <CA+TgmoaR_fVd_u60mA-PHO=J1u0PmdVGOt9WaxZ1UYuGq276oQ@mail.gmail.com>
 <20180516113631.GA29544@zakirov.localdomain>
 <20180614084015.GA12451@zakirov.localdomain>
 <20181001092204.GA5071@zakirov.localdomain>
 <68aaaff6-0efe-c14b-7aee-fb110bb97f69@2ndquadrant.com>
 <337f7a55-58b8-4fcc-a933-5e7799fa6882@postgrespro.ru>
 <e0a25563-cbb1-7089-8c25-44668cfb031b@postgrespro.ru>
 <42b1ded6-fd44-5975-77e4-72ada1f134a2@2ndquadrant.com>
 <20190120222121.ehistvcr4wsu2su5@alap3.anarazel.de>
 <95aea959-97a1-2ded-c881-eede4e5a1f0c@2ndquadrant.com>
 <cad38cdb-2bb1-da75-235f-2e6684826f10@postgrespro.ru>
 <b9e470f7-01b5-bf8c-78fc-5633ebc6fa2e@2ndquadrant.com>
 <5113daa6-b6e7-59f6-c4a8-96b5b81474fb@postgrespro.ru>
 <CAKNkYnyPxoByX0W7fUy=2NPvn880+DW3YZ5tBrFiV4cOaBX1xQ@mail.gmail.com>
 <1932084f-167a-8893-be24-c4c06afe113b@2ndquadrant.com>
 <26e59c3b-3598-cc2c-6b8d-81d24d6d0930@postgrespro.ru>
 <CA+TgmoYiWVvgUrDBJXVG9Crpg=s=Y1BLhMwVd1RZQOq5aF4Ctw@mail.gmail.com>
From: Arthur Zakirov <a.zakirov@postgrespro.ru>
Message-ID: <4c321cb7-899f-0548-ecf4-5e965dd71335@postgrespro.ru>
Date: Thu, 21 Feb 2019 16:28:29 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101
 Thunderbird/60.5.0
MIME-Version: 1.0
In-Reply-To: 
 <CA+TgmoYiWVvgUrDBJXVG9Crpg=s=Y1BLhMwVd1RZQOq5aF4Ctw@mail.gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 21.02.2019 15:45, Robert Haas wrote:
> On Wed, Feb 20, 2019 at 9:33 AM Arthur Zakirov <a.zakirov@postgrespro.ru> wrote:
>> I'm working on the (b) approach. I thought about a priority queue
>> structure. There no such ready structure within PostgreSQL sources
>> except binaryheap.c, but it isn't for concurrent algorithms.
> 
> I don't see why you need a priority queue or, really, any other fancy
> data structure.  It seems like all you need to do is somehow set it up
> so that a backend which doesn't use a dictionary for a while will
> dsm_detach() the segment.  Eventually an unused dictionary will have
> no remaining references and will go away.

Hm, I didn't think in this way. Agree that using a new data structure is 
overengineering.

Now in the current patch all DSM segments are pinned (and therefore 
dsm_pin_segment() is called). So a dictionary lives in shared memory 
even if nobody have the reference to it.

I thought about periodically scanning the shared hash table and 
unpinning old and unused dictionaries. But this approach needs 
sequential scan facility for dshash. Happily there is the patch from 
Kyotaro-san (the v16-0001-sequential-scan-for-dshash.patch part):

https://www.postgresql.org/message-id/20190221.160555.191280262.horiguchi.kyotaro@lab.ntt.co.jp

Your approach looks simpler. It is necessary just to periodically scan 
dictionaries' cache hash table and not call dsm_pin_segment() when a DSM 
segment initialized. It also means that a dictionary is loaded into DSM 
only while there is a backend which attached the dictionary's DSM.

-- 
Arthur Zakirov
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company