Subject: Re: Rename max_parallel_degree?
To: Robert Haas <robertmhaas@gmail.com>
References: <20160426152217.GA548975@alvherre.pgsql>
 <57509894.9080507@agliodbs.com>
 <CAKFQuwaBbH53x+i0MbaFzBDh3DhAXPS7EzjpEGKVvPRW-6HzNg@mail.gmail.com>
 <CA+TgmobYx11xuXy_GtuT=1ejh27K5t-jrc=gOk+XWhxUbpLbkQ@mail.gmail.com>
 <11330.1464961183@sss.pgh.pa.us>
 <CA+TgmoZF+-ckDHethxzznM-b-vraFvUXz-jXTDA8-cCUodpzGg@mail.gmail.com>
 <WM!a97c0cfedc9cd65ee73bc40bafb7ef019892cb9797432fc9c5209f5ce29a85ceab4031e102c0b11649cc454d31683eb4!@mailstronghold-1.zmailcloud.com>
 <57581603.3040309@agliodbs.com> <18837.1465395492@sss.pgh.pa.us>
 <CA+TgmoZn4E_6Yw-irgqaxsjPxf=z=OrkB0sOWcj5J7a7V2oSog@mail.gmail.com>
 <dabc5b37-b51e-169a-15f1-8441d5bc4c0b@dalibo.com>
 <5856af3e-3c20-89d1-3a9c-af3c2cd3a92b@dalibo.com>
 <CA+TgmobUZZV6sMj-ZXP71vV5zaHXNtUu5uL+oaeEw54_SySE2Q@mail.gmail.com>
 <9b599d4c-cb80-4f1f-48ad-169469579efe@dalibo.com>
 <CA+TgmoZWASaz5_wwF8RifWh95AmzzJUcLqgW5aVZJdRMO=m0Rw@mail.gmail.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, Josh Berkus <josh@agliodbs.com>,
 "David G. Johnston" <david.g.johnston@gmail.com>,
 Amit Kapila <amit.kapila16@gmail.com>, Peter Geoghegan <pg@heroku.com>,
 Noah Misch <noah@leadboat.com>, David Rowley <david.rowley@2ndquadrant.com>,
 Alvaro Herrera <alvherre@2ndquadrant.com>,
 Magnus Hagander <magnus@hagander.net>, Bruce Momjian <bruce@momjian.us>,
 PostgreSQL-development <pgsql-hackers@postgresql.org>
From: Julien Rouhaud <julien.rouhaud@dalibo.com>
Message-ID: <f5aadd54-6ea0-c8a2-a414-612acdbdab46@dalibo.com>
Date: Tue, 14 Jun 2016 13:10:01 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101
 Thunderbird/45.1.0
MIME-Version: 1.0
In-Reply-To: 
 <CA+TgmoZWASaz5_wwF8RifWh95AmzzJUcLqgW5aVZJdRMO=m0Rw@mail.gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org

On 14/06/2016 04:09, Robert Haas wrote:
> On Mon, Jun 13, 2016 at 5:42 AM, Julien Rouhaud
> <julien.rouhaud@dalibo.com> wrote:
>> Agreed, and fixed in attached v3.
> 
> I don't entirely like the new logic in
> RegisterDynamicBackgroundWorker.

I'm not that happy with it too. We can avoid iterating over every slots
if the feature isn't activated though (max_parallel_workers >=
max_worker_processes).

> I wonder if we can't drive this off
> of a couple of counters, instead of having the process registering the
> background worker iterate over every slot.  Suppose we add two
> counters to BackgroundWorkerArray, parallel_register_count and
> parallel_terminate_count.  Whenever a backend successfully registers a
> parallel worker, it increments parallel_register_count.  Whenever the
> postmaster marks a parallel wokrer slot as no longer in use, it
> increments parallel_terminate_count.  Then, the number of active
> parallel workers is just parallel_register_count -
> parallel_terminate_count.  (We can't have the postmaster and the
> backends share the same counter, because then it would need locking,
> and the postmaster can't try to take spinlocks - can't even use
> atomics, because those might be emulated using spinlocks.)
> 

I wanted to maintain counters at first, but it seemed more invasive, and
I thought that the max_parallel_worker would be ueful in environnements
where there're lots of parallel workers and dynamic workers used, so
finding a free slot would require iterating over most of the slots most
of the time anyway.  I'm of course also ok with maintaining counters.

> If we want to allow the number of parallel workers started to be available
> for statistical purposes, we can keep to uint32 values for that
> (parallel_register_count_lo and parallel_register_count_hi, for
> example), and increment the second one whenever the first one rolls
> over to zero.
> 

I didn't think about monitoring. I'm not sure if this counter would be
really helpful without also having the number of time a parallel worker
couldn't be launched (and I'd really like to have this one).

-- 
Julien Rouhaud
http://dalibo.com - http://dalibo.org


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers