From: Arthur Zakirov <a.zakirov@postgrespro.ru>
To: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: Andres Freund <andres@anarazel.de>
Cc: Robert Haas <robertmhaas@gmail.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Pavel Stehule <pavel.stehule@gmail.com>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: Re: [PROPOSAL] Shared Ispell dictionaries
Date: Mon, 21 Jan 2019 19:42:45 +0300
Message-ID: <5113daa6-b6e7-59f6-c4a8-96b5b81474fb@postgrespro.ru> (raw)
In-Reply-To: <b9e470f7-01b5-bf8c-78fc-5633ebc6fa2e@2ndquadrant.com>
References: <27296.1522078068@sss.pgh.pa.us>
<20180327121954.GA12726@zakirov.localdomain>
<CA+TgmoaR_fVd_u60mA-PHO=J1u0PmdVGOt9WaxZ1UYuGq276oQ@mail.gmail.com>
<20180516113631.GA29544@zakirov.localdomain>
<20180614084015.GA12451@zakirov.localdomain>
<20181001092204.GA5071@zakirov.localdomain>
<68aaaff6-0efe-c14b-7aee-fb110bb97f69@2ndquadrant.com>
<337f7a55-58b8-4fcc-a933-5e7799fa6882@postgrespro.ru>
<e0a25563-cbb1-7089-8c25-44668cfb031b@postgrespro.ru>
<42b1ded6-fd44-5975-77e4-72ada1f134a2@2ndquadrant.com>
<20190120222121.ehistvcr4wsu2su5@alap3.anarazel.de>
<95aea959-97a1-2ded-c881-eede4e5a1f0c@2ndquadrant.com>
<cad38cdb-2bb1-da75-235f-2e6684826f10@postgrespro.ru>
<b9e470f7-01b5-bf8c-78fc-5633ebc6fa2e@2ndquadrant.com>
On 21.01.2019 17:56, Tomas Vondra wrote:
> On 1/21/19 12:51 PM, Arthur Zakirov wrote:
>> I'll try to implement the syntax, you suggested earlier:
>>
>> ALTER TEXT SEARCH DICTIONARY x UNLOAD/RELOAD
>>
>> The main point here is that UNLOAD/RELOAD can't release the memory
>> immediately, because some other backend may pin a DSM.
>>
>> The second point we should consider (I think) - how do we know which
>> dictionary should be unloaded. There was such function earlier, which
>> was removed. But what about adding an information in the "\dFd" psql's
>> command output? It could be a column which shows is a dictionary loaded.
>>
> ...The only thing we have is "unload" capability by closing the
> connection...
BTW, even if the connection was closed and there are no other
connections a dictionary still remains "loaded". It is because
dsm_pin_segment() is called during loading the dictionary into DSM.
> ...
> I wonder if we could devise some simple cache eviction policy. We don't
> have any memory limit GUC anymore, but maybe we could use unload
> dictionaries that were unused for sufficient amount of time (a couple of
> minutes or so). Of course, the question is when exactly would it happen
> (it seems far too expensive to invoke on each dict access, and it should
> happen even when the dicts are not accessed at all).
Yes, I thought about such feature too. Agree, it could be expensive
since we need to scan pg_ts_dict table to get list of dictionaries (we
can't scan dshash_table).
I haven't a good solution yet. I just had a thought to return
max_shared_dictionaries_size. Then we can unload dictionaries (and scan
the pg_ts_dict table) that were accessed a lot time ago if we reached
the size limit.
We can't set exact size limit since we can't release the memory
immediately. So max_shared_dictionaries_size can be renamed to
shared_dictionaries_threshold. If it is equal to "0" then PostgreSQL has
unlimited space for dictionaries.
--
Arthur Zakirov
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: a.zakirov@postgrespro.ru, tomas.vondra@2ndquadrant.com, andres@anarazel.de, robertmhaas@gmail.com, tgl@sss.pgh.pa.us, pavel.stehule@gmail.com
Subject: Re: [PROPOSAL] Shared Ispell dictionaries
In-Reply-To: <5113daa6-b6e7-59f6-c4a8-96b5b81474fb@postgrespro.ru>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox