Subject: Re: [PROPOSAL] Shared Ispell dictionaries
To: Tomas Vondra <tomas.vondra@2ndquadrant.com>,
 Andres Freund <andres@anarazel.de>
Cc: Robert Haas <robertmhaas@gmail.com>, Tom Lane <tgl@sss.pgh.pa.us>,
 Pavel Stehule <pavel.stehule@gmail.com>,
 pgsql-hackers <pgsql-hackers@postgresql.org>
References: <27296.1522078068@sss.pgh.pa.us>
 <20180327121954.GA12726@zakirov.localdomain>
 <CA+TgmoaR_fVd_u60mA-PHO=J1u0PmdVGOt9WaxZ1UYuGq276oQ@mail.gmail.com>
 <20180516113631.GA29544@zakirov.localdomain>
 <20180614084015.GA12451@zakirov.localdomain>
 <20181001092204.GA5071@zakirov.localdomain>
 <68aaaff6-0efe-c14b-7aee-fb110bb97f69@2ndquadrant.com>
 <337f7a55-58b8-4fcc-a933-5e7799fa6882@postgrespro.ru>
 <e0a25563-cbb1-7089-8c25-44668cfb031b@postgrespro.ru>
 <42b1ded6-fd44-5975-77e4-72ada1f134a2@2ndquadrant.com>
 <20190120222121.ehistvcr4wsu2su5@alap3.anarazel.de>
 <95aea959-97a1-2ded-c881-eede4e5a1f0c@2ndquadrant.com>
 <cad38cdb-2bb1-da75-235f-2e6684826f10@postgrespro.ru>
 <b9e470f7-01b5-bf8c-78fc-5633ebc6fa2e@2ndquadrant.com>
From: Arthur Zakirov <a.zakirov@postgrespro.ru>
Message-ID: <5113daa6-b6e7-59f6-c4a8-96b5b81474fb@postgrespro.ru>
Date: Mon, 21 Jan 2019 19:42:45 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101
 Thunderbird/60.4.0
MIME-Version: 1.0
In-Reply-To: <b9e470f7-01b5-bf8c-78fc-5633ebc6fa2e@2ndquadrant.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 21.01.2019 17:56, Tomas Vondra wrote:
> On 1/21/19 12:51 PM, Arthur Zakirov wrote:
>> I'll try to implement the syntax, you suggested earlier:
>>
>> ALTER TEXT SEARCH DICTIONARY x UNLOAD/RELOAD
>>
>> The main point here is that UNLOAD/RELOAD can't release the memory
>> immediately, because some other backend may pin a DSM.
>>
>> The second point we should consider (I think) - how do we know which
>> dictionary should be unloaded. There was such function earlier, which
>> was removed. But what about adding an information in the "\dFd" psql's
>> command output? It could be a column which shows is a dictionary loaded.
>>
> ...The only thing we have is "unload" capability by closing the
> connection...
BTW, even if the connection was closed and there are no other 
connections a dictionary still remains "loaded". It is because 
dsm_pin_segment() is called during loading the dictionary into DSM.

> ...
> I wonder if we could devise some simple cache eviction policy. We don't
> have any memory limit GUC anymore, but maybe we could use unload
> dictionaries that were unused for sufficient amount of time (a couple of
> minutes or so). Of course, the question is when exactly would it happen
> (it seems far too expensive to invoke on each dict access, and it should
> happen even when the dicts are not accessed at all).

Yes, I thought about such feature too. Agree, it could be expensive 
since we need to scan pg_ts_dict table to get list of dictionaries (we 
can't scan dshash_table).

I haven't a good solution yet. I just had a thought to return 
max_shared_dictionaries_size. Then we can unload dictionaries (and scan 
the pg_ts_dict table) that were accessed a lot time ago if we reached 
the size limit.
We can't set exact size limit since we can't release the memory 
immediately. So max_shared_dictionaries_size can be renamed to 
shared_dictionaries_threshold. If it is equal to "0" then PostgreSQL has 
unlimited space for dictionaries.

-- 
Arthur Zakirov
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company