Subject: Re: Logical replication and multimaster
Mime-Version: 1.0 (Apple Message framework v1081)
Content-Type: text/plain; charset=us-ascii
From: konstantin knizhnik <k.knizhnik@postgrespro.ru>
In-Reply-To: 
 <CAMsr+YHMe=xs_E4fd2q1j7JPLpwvdU6+vY__rJTy2L4zkSSgTg@mail.gmail.com>
Date: Thu, 3 Dec 2015 10:27:01 +0300
Cc: Robert Haas <robertmhaas@gmail.com>, Craig Ringer <craig@2ndquadrant.com>
Content-Transfer-Encoding: quoted-printable
Message-Id: <B58749EF-69B0-44F6-A2D3-0708BC92CFF7@postgrespro.ru>
References: <565C7756.9070604@postgrespro.ru>
 <CA+TgmoY1o3G0B-21zv2Pw5iEkpR8=J42GdsUOs4m0inKka3FEA@mail.gmail.com>
 <565F5208.3070100@postgrespro.ru>
 <CAMsr+YHMe=xs_E4fd2q1j7JPLpwvdU6+vY__rJTy2L4zkSSgTg@mail.gmail.com>
To: pgsql-hackers Hackers <pgsql-hackers@postgresql.org>
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org


On Dec 3, 2015, at 4:18 AM, Craig Ringer wrote:

> Excellent.
>=20
> It should be possible to make that a separate extension. You can use C fu=
nctions from other extensions by exposing a single pg_proc function with 'i=
nternal' return type that populates a struct of function pointers for the A=
PI. A single DirectFunctionCall lets you get the API struct. That's how pgl=
ogical_output handles hooks. The main downside is that you can't do that wi=
thout a connection to a database with the extension installed so the pg_pro=
c entry is exposed.


Actually, working under cluster and columnar storage extension I got severa=
l questions about PostgreSQL infrastructure.
I always found some workarounds, but may it is better to ask community abou=
t it:)

1. Why there is no "conditional event" synchronization primitive in Postgre=
SQL. There is latch, but it is implemented using sockets and I afraid that =
it is not very fast.
It will be nice to have some fast primitive like pthread condition variable=
s.

2. PostgreSQL semaphores seems to be not intended for external use outside =
PostgreSQL core  (for example in extensions).
There is no way to request additional amount of semaphores. Right now semap=
hores are allocated based on maximal number of backends and spinlocks.
And a semaphore as well as event is very popular and convenient synchroniza=
tion primitive required in many cases.

3. What is the right way of creation of background worker requiring access =
to shared memory, i.e. having control structure in main memory?
As far as I understand background workers have to be registered either PG_i=
nit, either outside Postmaster environment.
If extension requires access to shared memory, then it should be registered=
 in shared_preload_libraries list and should be initialized using shmem_sta=
rtup hook.
Something like this:

void _PG_init(void)
{
	if (!process_shared_preload_libraries_in_progress)
		return;
        ...
	prev_shmem_startup_hook =3D shmem_startup_hook;
	shmem_startup_hook =3D My_shmem_startup;
}

My_shmem_startup is needed because in _PG_init it is not possible to alloca=
te shared memory.
So if I need to allocate some control structure for background workers  in =
shared memory, then I should do it in My_shmem_startup.
But I can not register background workers in My_shmem_startup! I will get "=
must be registered in shared_preload_libraries" error:

void
RegisterBackgroundWorker(BackgroundWorker *worker)
{
	if (!process_shared_preload_libraries_in_progress)
	{
		if (!IsUnderPostmaster)
			ereport(LOG,
					(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
					 errmsg("background worker \"%s\": must be registered in shared_preloa=
d_libraries",
							worker->bgw_name)));
		return;
	}
}

So I have to register background workers in PG_init while control structure=
 for them is not yet ready.
When I have implemented pool of background workers, I solved this problem b=
y proving function which return address of control structure later - when i=
t will be actually allocated.
But it seems to be some design flaw in BGW, isn' it?


--=20
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers