pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  

help / color / mirror / Atom feed
From: Andrey Lepikhov <a.lepikhov@postgrespro.ru>
To: Peter Geoghegan <pg@bowt.ie>
Cc: PostgreSQL-Dev <pgsql-hackers@postgresql.org>
Subject: Re: [WIP] [B-Tree] Retail IndexTuple deletion
Date: Thu, 9 Aug 2018 08:43:37 +0500
Message-ID: <41f4184f-6e49-b2ef-b3f0-a2ac5dd665f0@postgrespro.ru> (raw)
In-Reply-To: <CAH2-Wz=dOU0_jXQM4G4M=Kzqom_DgmPfatUHFzpQq3DX6DpDKQ@mail.gmail.com>
References: <425db134-8bba-005c-b59d-56e50de3b41e@postgrespro.ru>
	<CAH2-WzmpjdeU77aJf1gtte3QhWrnwMffx3pPJQpEDHA1zbx3gw@mail.gmail.com>
	<e4cec53a-ab12-4858-d9d8-5fac9948a4c7@postgrespro.ru>
	<CAH2-Wz=mkrkQJJxHjDgiTv25yk0yqrx0EOn5exAHvPyQRR6iNQ@mail.gmail.com>
	<9a008ab3-f591-e5e9-8e03-c0dd1dc13579@postgrespro.ru>
	<CAL-rCA0BkRJgEfkQdDkii6kt0L-_8xcYqraaSmrrn_jzTFx-Pg@mail.gmail.com>
	<2ada5790-dffa-a41a-ef84-10f60a525ba6@postgrespro.ru>
	<CAH2-Wz=mnW+8+oZ9vh45+WEzt4dinYrz5f5acqn8mRjVY=YXqg@mail.gmail.com>
	<CAH2-WzmPf0pmnS4go9+MmqQMA0MijsZ9XR+zdMy9J2gkEzadxA@mail.gmail.com>
	<7598602c-f329-a6aa-00e6-e3335ed06048@postgrespro.ru>
	<CAH2-Wzm2SG40Hp2nYMzqkKXz3-B9bxSM1N02bGNkft4djwYm7A@mail.gmail.com>
	<CAD21AoApVGFf3q7WV5FuFzHDRtew9+fHHdCyOkk1uG+XG_6OKw@mail.gmail.com>
	<CAH2-Wz=MkEWanC4WPTAwOdiHB7VpetgsY8i1qbYG9m-XY-PuTg@mail.gmail.com>
	<f49bb262-d246-829d-f835-3950ddac503c@postgrespro.ru>
	<CAH2-Wz=dOU0_jXQM4G4M=Kzqom_DgmPfatUHFzpQq3DX6DpDKQ@mail.gmail.com>


09.08.2018 05:19, Peter Geoghegan пишет:
> On Tue, Aug 7, 2018 at 12:19 AM, Andrey Lepikhov
> <a.lepikhov@postgrespro.ru> wrote:
>> I wrote a background worker (hcleaner) to demonstrate application of Retail
>> IndexTuple deletion (see patch at attachment).
>> Like Autovacuum it utilizes concept of one launcher and many workers. But
>> one worker correspond to one database.
>>
>> Short description:
>> Backend collects dirty block numbers by a hash table at the point in code
>> immediately after heap_page_prune() call. Backend send a package of dirty
>> block numbers (not one-by-one!) by socket at the end of transaction or if
>> hash table is full.
>> Launcher transfers block numbers to correspond workers.
>> Worker collects dead tuples from a block, clean index relations, clean heap
>> block. It uses conditional locking with waiting list approach if heap block
>> are busy.
>>
>> hcleaner has experimental status, but make check-world passed
> 
> How does this affect ordinary opportunistic pruning?
> 
As far as I understand, background worker not affect on ordinary 
opportunistic pruning. For a dirty block it made cleaning work like 
vacuum (if acquire conditional lock).

I tried two ways:
1. In executor: tid of deleted/updated tuples collects directly in 
executor and send to background worker. Background worker make 
heap_page_prune() himself.
But this is no good variant, because each backend perform opportunistic 
pruning and it is sufficient work to detect dead tuples. Moreover, in 
this case the worker compete with backends for pruning and has high load 
when many backends existed.
2. In heap_page_prune_opt(): backend collects ids of dirty blocks 
immediately after pruning and send it to the worker. In this case 
backend perform all work for detection of dead tuples. It let the worker 
to miss block cleaning during periods of heavy load: later it can clean 
block.
Since locks are conditional we do not hamper backends work during high 
load periods and utilize idle time for relation cleaning in soft manner.

Variant No.2 look better better and now i use it.

-- 
Andrey Lepikhov
Postgres Professional
https://postgrespro.com
The Russian Postgres Company




view thread (47+ messages)  latest in thread

Message-ID: <41f4184f-6e49-b2ef-b3f0-a2ac5dd665f0@postgrespro.ru>
Permalink:  ../41f4184f-6e49-b2ef-b3f0-a2ac5dd665f0@postgrespro.ru/
Also on:    postgresql.org/message-id/41f4184f-6e49-b2ef-b3f0-a2ac5dd665f0@postgrespro.ru

 · 

reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-hackers@postgresql.org
  Cc: a.lepikhov@postgrespro.ru, pg@bowt.ie
  Subject: Re: [WIP] [B-Tree] Retail IndexTuple deletion
  In-Reply-To: <41f4184f-6e49-b2ef-b3f0-a2ac5dd665f0@postgrespro.ru>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox