pg.ddx.io pgsql-committers@postgresql.org mailing list archive
help / color / mirror / Atom feedpgsql: Default to hidden visibility for extension libraries where possi
27+ messages / 8 participants
[nested] [flat]
Commits from this thread:
2022-07-20 1caf915ff3 Add PGDLLEXPORTS to some plpgsql function declarations
* pgsql: Default to hidden visibility for extension libraries where possi
@ 2022-07-18 01:05 Andres Freund <andres@anarazel.de>
0 siblings, 2 replies; 27+ messages in thread
From: Andres Freund @ 2022-07-18 01:05 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Default to hidden visibility for extension libraries where possible
Until now postgres built extension libraries with global visibility, i.e.
exporting all symbols. On the one platform where that behavior is not
natively available, namely windows, we emulate it by analyzing the input files
to the shared library and exporting all the symbols therein.
Not exporting all symbols is actually desirable, as it can improve loading
speed, reduces the likelihood of symbol conflicts and can improve intra
extension library function call performance. It also makes the non-windows
builds more similar to windows builds.
Additionally, with meson implementing the export-all-symbols behavior for
windows, turns out to be more verbose than desirable.
This patch adds support for hiding symbols by default and, to counteract that,
explicit symbol visibility annotation for compilers that support
__attribute__((visibility("default"))) and -fvisibility=hidden. That is
expected to be most, if not all, compilers except msvc (for which we already
support explicit symbol export annotations).
Now that extension library symbols are explicitly exported, we don't need to
export all symbols on windows anymore, hence remove that behavior from
src/tools/msvc. The supporting code can't be removed, as we still need to
export all symbols from the main postgres binary.
Author: Andres Freund <andres@anarazel.de>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/20211101020311.av6hphdl6xbjbuif@alap3.anarazel.de
Branch
------
master
Details
-------
https://git.postgresql.org/pg/commitdiff/089480c077056fc20fa8d8f5a3032a9dcf5ed812
Modified Files
--------------
configure | 152 +++++++++++++++++++++++++++++++++++++++++++++
configure.ac | 13 ++++
src/Makefile.global.in | 3 +
src/Makefile.shlib | 13 ++++
src/include/c.h | 13 ++--
src/include/pg_config.h.in | 3 +
src/makefiles/pgxs.mk | 5 +-
src/tools/msvc/Project.pm | 7 ---
src/tools/msvc/Solution.pm | 1 +
9 files changed, 198 insertions(+), 12 deletions(-)
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: pgsql: Default to hidden visibility for extension libraries where possi
@ 2022-07-18 01:39 Andres Freund <andres@anarazel.de>
parent: Andres Freund <andres@anarazel.de>
1 sibling, 1 reply; 27+ messages in thread
From: Andres Freund @ 2022-07-18 01:39 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Hi,
On 2022-07-18 01:05:56 +0000, Andres Freund wrote:
> Default to hidden visibility for extension libraries where possible
Looking at the odd failures, not sure what went wrong.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: pgsql: Default to hidden visibility for extension libraries where possi
@ 2022-07-18 02:01 Andres Freund <andres@anarazel.de>
parent: Andres Freund <andres@anarazel.de>
0 siblings, 2 replies; 27+ messages in thread
From: Andres Freund @ 2022-07-18 02:01 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
On 2022-07-17 18:39:35 -0700, Andres Freund wrote:
> On 2022-07-18 01:05:56 +0000, Andres Freund wrote:
> > Default to hidden visibility for extension libraries where possible
>
> Looking at the odd failures, not sure what went wrong.
I don't know how the configure exec bit got removed, shell history doesn't
show anything odd in that regard. I do see a mistake in locally merging the
symbol-visibility branch (leading to a crucial commit being missed). I do see
in shell history that I ran check-world without a failure just before pushing,
which should have shown the failure, but didn't.
Definitely a brown paper bag moment.
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: pgsql: Default to hidden visibility for extension libraries where possi
@ 2022-07-18 02:34 Tom Lane <tgl@sss.pgh.pa.us>
parent: Andres Freund <andres@anarazel.de>
1 sibling, 0 replies; 27+ messages in thread
From: Tom Lane @ 2022-07-18 02:34 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: pgsql-committers@lists.postgresql.org
Andres Freund <andres@anarazel.de> writes:
> I don't know how the configure exec bit got removed, shell history doesn't
> show anything odd in that regard. I do see a mistake in locally merging the
> symbol-visibility branch (leading to a crucial commit being missed). I do see
> in shell history that I ran check-world without a failure just before pushing,
> which should have shown the failure, but didn't.
check-world would not have re-run configure even if it was out of date
(only config.status), so that part of it's not so surprising.
> Definitely a brown paper bag moment.
We've all been there :-)
regards, tom lane
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: pgsql: Default to hidden visibility for extension libraries where possi
@ 2022-07-18 23:36 Justin Pryzby <pryzby@telsasoft.com>
parent: Andres Freund <andres@anarazel.de>
1 sibling, 0 replies; 27+ messages in thread
From: Justin Pryzby @ 2022-07-18 23:36 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: pgsql-committers@lists.postgresql.org
On Sun, Jul 17, 2022 at 07:01:55PM -0700, Andres Freund wrote:
> On 2022-07-17 18:39:35 -0700, Andres Freund wrote:
> > On 2022-07-18 01:05:56 +0000, Andres Freund wrote:
> > > Default to hidden visibility for extension libraries where possible
> >
> > Looking at the odd failures, not sure what went wrong.
>
> I don't know how the configure exec bit got removed, shell history doesn't
I think git reflog -p would help to answer that, but I suppose you already
looked.
--
Justin
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: pgsql: Default to hidden visibility for extension libraries where possi
@ 2022-07-19 03:26 Pavel Stehule <pavel.stehule@gmail.com>
parent: Andres Freund <andres@anarazel.de>
1 sibling, 1 reply; 27+ messages in thread
From: Pavel Stehule @ 2022-07-19 03:26 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: pgsql-committers@lists.postgresql.org
Hi
po 18. 7. 2022 v 3:06 odesílatel Andres Freund <andres@anarazel.de> napsal:
> Default to hidden visibility for extension libraries where possible
>
> Until now postgres built extension libraries with global visibility, i.e.
> exporting all symbols. On the one platform where that behavior is not
> natively available, namely windows, we emulate it by analyzing the input
> files
> to the shared library and exporting all the symbols therein.
>
> Not exporting all symbols is actually desirable, as it can improve loading
> speed, reduces the likelihood of symbol conflicts and can improve intra
> extension library function call performance. It also makes the non-windows
> builds more similar to windows builds.
>
> Additionally, with meson implementing the export-all-symbols behavior for
> windows, turns out to be more verbose than desirable.
>
> This patch adds support for hiding symbols by default and, to counteract
> that,
> explicit symbol visibility annotation for compilers that support
> __attribute__((visibility("default"))) and -fvisibility=hidden. That is
> expected to be most, if not all, compilers except msvc (for which we
> already
> support explicit symbol export annotations).
>
> Now that extension library symbols are explicitly exported, we don't need
> to
> export all symbols on windows anymore, hence remove that behavior from
> src/tools/msvc. The supporting code can't be removed, as we still need to
> export all symbols from the main postgres binary.
>
> Author: Andres Freund <andres@anarazel.de>
> Author: Tom Lane <tgl@sss.pgh.pa.us>
> Discussion:
> https://postgr.es/m/20211101020311.av6hphdl6xbjbuif@alap3.anarazel.de
>
>
Unfortunately, this commit definitely breaks plpgsql_check. Can the
following routines be visible?
AssertVariableIsOfType(&plpgsql_build_datatype,
plpgsql_check__build_datatype_t);
plpgsql_check__build_datatype_p = (plpgsql_check__build_datatype_t)
LOAD_EXTERNAL_FUNCTION("$libdir/plpgsql", "plpgsql_build_datatype");
AssertVariableIsOfType(&plpgsql_compile, plpgsql_check__compile_t);
plpgsql_check__compile_p = (plpgsql_check__compile_t)
LOAD_EXTERNAL_FUNCTION("$libdir/plpgsql", "plpgsql_compile");
AssertVariableIsOfType(&plpgsql_parser_setup,
plpgsql_check__parser_setup_t);
plpgsql_check__parser_setup_p = (plpgsql_check__parser_setup_t)
LOAD_EXTERNAL_FUNCTION("$libdir/plpgsql", "plpgsql_parser_setup");
AssertVariableIsOfType(&plpgsql_stmt_typename,
plpgsql_check__stmt_typename_t);
plpgsql_check__stmt_typename_p = (plpgsql_check__stmt_typename_t)
LOAD_EXTERNAL_FUNCTION("$libdir/plpgsql", "plpgsql_stmt_typename");
AssertVariableIsOfType(&plpgsql_exec_get_datum_type,
plpgsql_check__exec_get_datum_type_t);
plpgsql_check__exec_get_datum_type_p =
(plpgsql_check__exec_get_datum_type_t)
LOAD_EXTERNAL_FUNCTION("$libdir/plpgsql",
"plpgsql_exec_get_datum_type");
AssertVariableIsOfType(&plpgsql_recognize_err_condition,
plpgsql_check__recognize_err_condition_t);
plpgsql_check__recognize_err_condition_p =
(plpgsql_check__recognize_err_condition_t)
LOAD_EXTERNAL_FUNCTION("$libdir/plpgsql",
"plpgsql_recognize_err_condition");
AssertVariableIsOfType(&plpgsql_ns_lookup, plpgsql_check__ns_lookup_t);
plpgsql_check__ns_lookup_p = (plpgsql_check__ns_lookup_t)
LOAD_EXTERNAL_FUNCTION("$libdir/plpgsql", "plpgsql_ns_lookup");
Regards
Pavel
> Branch
> ------
> master
>
> Details
> -------
>
> https://git.postgresql.org/pg/commitdiff/089480c077056fc20fa8d8f5a3032a9dcf5ed812
>
> Modified Files
> --------------
> configure | 152
> +++++++++++++++++++++++++++++++++++++++++++++
> configure.ac | 13 ++++
> src/Makefile.global.in | 3 +
> src/Makefile.shlib | 13 ++++
> src/include/c.h | 13 ++--
> src/include/pg_config.h.in | 3 +
> src/makefiles/pgxs.mk | 5 +-
> src/tools/msvc/Project.pm | 7 ---
> src/tools/msvc/Solution.pm | 1 +
> 9 files changed, 198 insertions(+), 12 deletions(-)
>
>
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: pgsql: Default to hidden visibility for extension libraries where possi
@ 2022-07-19 13:18 Alvaro Herrera <alvherre@alvh.no-ip.org>
parent: Pavel Stehule <pavel.stehule@gmail.com>
0 siblings, 1 reply; 27+ messages in thread
From: Alvaro Herrera @ 2022-07-19 13:18 UTC (permalink / raw)
To: Pavel Stehule <pavel.stehule@gmail.com>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-committers@lists.postgresql.org
On 2022-Jul-19, Pavel Stehule wrote:
> po 18. 7. 2022 v 3:06 odesílatel Andres Freund <andres@anarazel.de> napsal:
>
> > Default to hidden visibility for extension libraries where possible
> Unfortunately, this commit definitely breaks plpgsql_check. Can the
> following routines be visible?
Do you just need to send a patch to add an exports.txt file to
src/pl/plpgsql/src/ for these functions?
plpgsql_build_datatype
plpgsql_compile
plpgsql_parser_setup
plpgsql_stmt_typename
plpgsql_exec_get_datum_type
plpgsql_recognize_err_condition
plpgsql_ns_lookup
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: pgsql: Default to hidden visibility for extension libraries where possi
@ 2022-07-19 13:45 Tom Lane <tgl@sss.pgh.pa.us>
parent: Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 1 reply; 27+ messages in thread
From: Tom Lane @ 2022-07-19 13:45 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Pavel Stehule <pavel.stehule@gmail.com>; Andres Freund <andres@anarazel.de>; pgsql-committers@lists.postgresql.org
Alvaro Herrera <alvherre@alvh.no-ip.org> writes:
> On 2022-Jul-19, Pavel Stehule wrote:
>> Unfortunately, this commit definitely breaks plpgsql_check. Can the
>> following routines be visible?
> Do you just need to send a patch to add an exports.txt file to
> src/pl/plpgsql/src/ for these functions?
The precedent of plpython says that PGDLLEXPORT markers are sufficient.
But yeah, we need a list of exactly which functions need to be
re-exposed. I imagine pldebugger has its own needs.
regards, tom lane
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: pgsql: Default to hidden visibility for extension libraries where possi
@ 2022-07-19 14:28 Alvaro Herrera <alvherre@alvh.no-ip.org>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 4 replies; 27+ messages in thread
From: Alvaro Herrera @ 2022-07-19 14:28 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Pavel Stehule <pavel.stehule@gmail.com>; Andres Freund <andres@anarazel.de>; pgsql-hackers@lists.postgresql.org
[ Redirecting thread to -hackers from -committers ]
On 2022-Jul-19, Tom Lane wrote:
> Alvaro Herrera <alvherre@alvh.no-ip.org> writes:
> > Do you just need to send a patch to add an exports.txt file to
> > src/pl/plpgsql/src/ for these functions?
>
> The precedent of plpython says that PGDLLEXPORT markers are sufficient.
> But yeah, we need a list of exactly which functions need to be
> re-exposed. I imagine pldebugger has its own needs.
A reasonable guess. I went as far as downloading pldebugger and
compiling it, but it doesn't have a test suite of its own, so I couldn't
verify anything about it. I did notice that plpgsql_check is calling
function load_external_function(), and that doesn't appear in
pldebugger. I wonder if the find_rendezvous_variable business is at
play.
Anyway, the minimal patch that makes plpgsql_check tests pass is
attached. This seems a bit random. Maybe it'd be better to have a
plpgsql_internal.h with functions that are exported only for plpgsql
itself, and keep plpgsql.h with a set of functions, all marked
PGDLLEXPORT, that are for external use.
... oh, and:
$ postmaster -c shared_preload_libraries=plugin_debugger
2022-07-19 16:27:24.006 CEST [742142] FATAL: cannot request additional shared memory outside shmem_request_hook
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
Attachments:
0001-add-PGDLLEXPORTS-to-plpgsql.patch (text/x-diff, 0B) download
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: pgsql: Default to hidden visibility for extension libraries where possi
@ 2022-07-19 14:45 Julien Rouhaud <rjuju123@gmail.com>
parent: Alvaro Herrera <alvherre@alvh.no-ip.org>
3 siblings, 0 replies; 27+ messages in thread
From: Julien Rouhaud @ 2022-07-19 14:45 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Pavel Stehule <pavel.stehule@gmail.com>; Andres Freund <andres@anarazel.de>; pgsql-hackers@lists.postgresql.org
On Tue, Jul 19, 2022 at 04:28:07PM +0200, Alvaro Herrera wrote:
> ... oh, and:
>
> $ postmaster -c shared_preload_libraries=plugin_debugger
> 2022-07-19 16:27:24.006 CEST [742142] FATAL: cannot request additional shared memory outside shmem_request_hook
This has been reported multiple times (including on one of my own projects),
see
https://www.postgresql.org/message-id/flat/81f82c00-8818-91f3-96fa-47976f94662b%40pm.me
for the last report.
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: pgsql: Default to hidden visibility for extension libraries where possi
@ 2022-07-19 14:47 Pavel Stehule <pavel.stehule@gmail.com>
parent: Alvaro Herrera <alvherre@alvh.no-ip.org>
3 siblings, 0 replies; 27+ messages in thread
From: Pavel Stehule @ 2022-07-19 14:47 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Andres Freund <andres@anarazel.de>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Hi
út 19. 7. 2022 v 16:28 odesílatel Alvaro Herrera <alvherre@alvh.no-ip.org>
napsal:
> [ Redirecting thread to -hackers from -committers ]
>
> On 2022-Jul-19, Tom Lane wrote:
>
> > Alvaro Herrera <alvherre@alvh.no-ip.org> writes:
>
> > > Do you just need to send a patch to add an exports.txt file to
> > > src/pl/plpgsql/src/ for these functions?
> >
> > The precedent of plpython says that PGDLLEXPORT markers are sufficient.
> > But yeah, we need a list of exactly which functions need to be
> > re-exposed. I imagine pldebugger has its own needs.
>
> A reasonable guess. I went as far as downloading pldebugger and
> compiling it, but it doesn't have a test suite of its own, so I couldn't
> verify anything about it. I did notice that plpgsql_check is calling
> function load_external_function(), and that doesn't appear in
> pldebugger. I wonder if the find_rendezvous_variable business is at
> play.
>
> Anyway, the minimal patch that makes plpgsql_check tests pass is
> attached. This seems a bit random. Maybe it'd be better to have a
> plpgsql_internal.h with functions that are exported only for plpgsql
> itself, and keep plpgsql.h with a set of functions, all marked
> PGDLLEXPORT, that are for external use.
>
>
I can confirm that the attached patch fixes plpgsql_check.
Thank you
Pavel
>
> ... oh, and:
>
> $ postmaster -c shared_preload_libraries=plugin_debugger
> 2022-07-19 16:27:24.006 CEST [742142] FATAL: cannot request additional
> shared memory outside shmem_request_hook
>
> --
> Álvaro Herrera Breisgau, Deutschland —
> https://www.EnterpriseDB.com/
>
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: pgsql: Default to hidden visibility for extension libraries where possi
@ 2022-07-19 15:31 Andres Freund <andres@anarazel.de>
parent: Alvaro Herrera <alvherre@alvh.no-ip.org>
3 siblings, 2 replies; 27+ messages in thread
From: Andres Freund @ 2022-07-19 15:31 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Pavel Stehule <pavel.stehule@gmail.com>; pgsql-hackers@lists.postgresql.org
Hi,
On 2022-07-19 16:28:07 +0200, Alvaro Herrera wrote:
> Anyway, the minimal patch that makes plpgsql_check tests pass is
> attached. This seems a bit random. Maybe it'd be better to have a
> plpgsql_internal.h with functions that are exported only for plpgsql
> itself, and keep plpgsql.h with a set of functions, all marked
> PGDLLEXPORT, that are for external use.
It does seem a bit random. But I think we probably should err on the side of
adding more declarations, rather than the oposite.
I like the plpgsql_internal.h idea, but probably done separately?
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: pgsql: Default to hidden visibility for extension libraries where possi
@ 2022-07-19 15:37 Pavel Stehule <pavel.stehule@gmail.com>
parent: Andres Freund <andres@anarazel.de>
1 sibling, 1 reply; 27+ messages in thread
From: Pavel Stehule @ 2022-07-19 15:37 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Tom Lane <tgl@sss.pgh.pa.us>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Hi
út 19. 7. 2022 v 17:31 odesílatel Andres Freund <andres@anarazel.de> napsal:
> Hi,
>
> On 2022-07-19 16:28:07 +0200, Alvaro Herrera wrote:
> > Anyway, the minimal patch that makes plpgsql_check tests pass is
> > attached. This seems a bit random. Maybe it'd be better to have a
> > plpgsql_internal.h with functions that are exported only for plpgsql
> > itself, and keep plpgsql.h with a set of functions, all marked
> > PGDLLEXPORT, that are for external use.
>
> It does seem a bit random. But I think we probably should err on the side
> of
> adding more declarations, rather than the oposite.
>
This list can be extended. I think plpgsql_check is maybe one extension
that uses code from another extension directly. This is really not common
usage.
>
> I like the plpgsql_internal.h idea, but probably done separately?
>
can be
I have not any problem with it or with exports.txt file.
> Greetings,
>
> Andres Freund
>
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: pgsql: Default to hidden visibility for extension libraries where possi
@ 2022-07-19 15:40 Andres Freund <andres@anarazel.de>
parent: Pavel Stehule <pavel.stehule@gmail.com>
0 siblings, 0 replies; 27+ messages in thread
From: Andres Freund @ 2022-07-19 15:40 UTC (permalink / raw)
To: Pavel Stehule <pavel.stehule@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Tom Lane <tgl@sss.pgh.pa.us>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Hi,
On 2022-07-19 17:37:11 +0200, Pavel Stehule wrote:
> út 19. 7. 2022 v 17:31 odesílatel Andres Freund <andres@anarazel.de> napsal:
>
> > Hi,
> >
> > On 2022-07-19 16:28:07 +0200, Alvaro Herrera wrote:
> > > Anyway, the minimal patch that makes plpgsql_check tests pass is
> > > attached. This seems a bit random. Maybe it'd be better to have a
> > > plpgsql_internal.h with functions that are exported only for plpgsql
> > > itself, and keep plpgsql.h with a set of functions, all marked
> > > PGDLLEXPORT, that are for external use.
> >
> > It does seem a bit random. But I think we probably should err on the side
> > of
> > adding more declarations, rather than the oposite.
> >
>
> This list can be extended. I think plpgsql_check is maybe one extension
> that uses code from another extension directly. This is really not common
> usage.
There's a few more use cases, e.g. transform modules. Hence exposing e.g. many
plpython helpers.
> I have not any problem with it or with exports.txt file.
Just to be clear, there shouldn't be any use exports.txt here, just a few
PGDLLEXPORTs.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: pgsql: Default to hidden visibility for extension libraries where possi
@ 2022-07-19 22:39 Andres Freund <andres@anarazel.de>
parent: Alvaro Herrera <alvherre@alvh.no-ip.org>
3 siblings, 1 reply; 27+ messages in thread
From: Andres Freund @ 2022-07-19 22:39 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Pavel Stehule <pavel.stehule@gmail.com>; pgsql-hackers@lists.postgresql.org
Hi,
On 2022-07-19 16:28:07 +0200, Alvaro Herrera wrote:
> Anyway, the minimal patch that makes plpgsql_check tests pass is
> attached.
Do you want to commit that or should I?
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: pgsql: Default to hidden visibility for extension libraries where possi
@ 2022-07-19 22:44 Andres Freund <andres@anarazel.de>
parent: Andres Freund <andres@anarazel.de>
1 sibling, 0 replies; 27+ messages in thread
From: Andres Freund @ 2022-07-19 22:44 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Pavel Stehule <pavel.stehule@gmail.com>; pgsql-hackers@lists.postgresql.org
Hi,
On 2022-07-19 08:31:49 -0700, Andres Freund wrote:
> But I think we probably should err on the side of adding more
> declarations, rather than the oposite.
To see if there's other declarations that should be added, I used
https://codesearch.debian.net/search?q=load_external_function&literal=1&perpkg=1
which shows plpgsql_check and hstore_pllua. All the hstore symbols for
the latter are exported already.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: pgsql: Default to hidden visibility for extension libraries where possi
@ 2022-07-20 09:02 Alvaro Herrera <alvherre@alvh.no-ip.org>
parent: Andres Freund <andres@anarazel.de>
0 siblings, 1 reply; 27+ messages in thread
From: Alvaro Herrera @ 2022-07-20 09:02 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Pavel Stehule <pavel.stehule@gmail.com>; pgsql-hackers@lists.postgresql.org
Hello
On 2022-Jul-19, Andres Freund wrote:
> On 2022-07-19 16:28:07 +0200, Alvaro Herrera wrote:
> > Anyway, the minimal patch that makes plpgsql_check tests pass is
> > attached.
>
> Do you want to commit that or should I?
Done.
No immediate plans for splitting plpgsql.h, so if anyone wants to take a
stab at that, be my guest.
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"Aprender sin pensar es inútil; pensar sin aprender, peligroso" (Confucio)
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: pgsql: Default to hidden visibility for extension libraries where possi
@ 2022-07-20 13:38 Tom Lane <tgl@sss.pgh.pa.us>
parent: Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 2 replies; 27+ messages in thread
From: Tom Lane @ 2022-07-20 13:38 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Andres Freund <andres@anarazel.de>; Pavel Stehule <pavel.stehule@gmail.com>; pgsql-hackers@lists.postgresql.org
Alvaro Herrera <alvherre@alvh.no-ip.org> writes:
> No immediate plans for splitting plpgsql.h, so if anyone wants to take a
> stab at that, be my guest.
ISTM that a comment pointing out that the functions marked PGDLLEXPORT
are meant to be externally accessible should be sufficient.
I'll try to do some research later today to identify anything else
we need to mark in plpgsql. I recall doing some work specifically
creating functions for pldebugger's use, but I'll need to dig.
regards, tom lane
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: pgsql: Default to hidden visibility for extension libraries where possi
@ 2022-07-20 13:54 Robert Haas <robertmhaas@gmail.com>
parent: Tom Lane <tgl@sss.pgh.pa.us>
1 sibling, 2 replies; 27+ messages in thread
From: Robert Haas @ 2022-07-20 13:54 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Andres Freund <andres@anarazel.de>; Pavel Stehule <pavel.stehule@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Wed, Jul 20, 2022 at 9:39 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> ISTM that a comment pointing out that the functions marked PGDLLEXPORT
> are meant to be externally accessible should be sufficient.
The name PGDLLEXPORT is actually slightly misleading, now, because
there's no longer anything about it that is specific to DLLs.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: pgsql: Default to hidden visibility for extension libraries where possi
@ 2022-07-20 14:04 Alvaro Herrera <alvherre@alvh.no-ip.org>
parent: Tom Lane <tgl@sss.pgh.pa.us>
1 sibling, 2 replies; 27+ messages in thread
From: Alvaro Herrera @ 2022-07-20 14:04 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Andres Freund <andres@anarazel.de>; Pavel Stehule <pavel.stehule@gmail.com>; pgsql-hackers@lists.postgresql.org
On 2022-Jul-20, Tom Lane wrote:
> I'll try to do some research later today to identify anything else
> we need to mark in plpgsql. I recall doing some work specifically
> creating functions for pldebugger's use, but I'll need to dig.
I suppose you're probably thinking of commit 53ef6c40f1e7; that didn't
expose functions directly, but through plpgsql_plugin_ptr. Maybe that
one does need to be made PGDLLEXPORT, since currently it isn't.
That was also reported by Pavel. He was concerned about plpgsql_check,
though, not pldebugger.
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
Bob [Floyd] used to say that he was planning to get a Ph.D. by the "green
stamp method," namely by saving envelopes addressed to him as 'Dr. Floyd'.
After collecting 500 such letters, he mused, a university somewhere in
Arizona would probably grant him a degree. (Don Knuth)
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: pgsql: Default to hidden visibility for extension libraries where possi
@ 2022-07-20 14:09 Tom Lane <tgl@sss.pgh.pa.us>
parent: Robert Haas <robertmhaas@gmail.com>
1 sibling, 0 replies; 27+ messages in thread
From: Tom Lane @ 2022-07-20 14:09 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Andres Freund <andres@anarazel.de>; Pavel Stehule <pavel.stehule@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Robert Haas <robertmhaas@gmail.com> writes:
> On Wed, Jul 20, 2022 at 9:39 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> ISTM that a comment pointing out that the functions marked PGDLLEXPORT
>> are meant to be externally accessible should be sufficient.
> The name PGDLLEXPORT is actually slightly misleading, now, because
> there's no longer anything about it that is specific to DLLs.
True, but the mess of changing it seems to outweigh any likely clarity
gain. As long as there's adequate commentary about what it means,
I'm okay with the existing naming.
regards, tom lane
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: pgsql: Default to hidden visibility for extension libraries where possi
@ 2022-07-20 14:12 Andres Freund <andres@anarazel.de>
parent: Robert Haas <robertmhaas@gmail.com>
1 sibling, 1 reply; 27+ messages in thread
From: Andres Freund @ 2022-07-20 14:12 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pavel Stehule <pavel.stehule@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Hi,
On July 20, 2022 3:54:03 PM GMT+02:00, Robert Haas <robertmhaas@gmail.com> wrote:
>On Wed, Jul 20, 2022 at 9:39 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> ISTM that a comment pointing out that the functions marked PGDLLEXPORT
>> are meant to be externally accessible should be sufficient.
>
>The name PGDLLEXPORT is actually slightly misleading, now, because
>there's no longer anything about it that is specific to DLLs.
How so? Right now it's solely used for making symbols in DLLs as exported?
Andres
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: Re: pgsql: Default to hidden visibility for extension libraries where possi
@ 2022-07-20 14:15 Tom Lane <tgl@sss.pgh.pa.us>
parent: Andres Freund <andres@anarazel.de>
0 siblings, 0 replies; 27+ messages in thread
From: Tom Lane @ 2022-07-20 14:15 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Robert Haas <robertmhaas@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Pavel Stehule <pavel.stehule@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Andres Freund <andres@anarazel.de> writes:
> On July 20, 2022 3:54:03 PM GMT+02:00, Robert Haas <robertmhaas@gmail.com> wrote:
>> The name PGDLLEXPORT is actually slightly misleading, now, because
>> there's no longer anything about it that is specific to DLLs.
> How so? Right now it's solely used for making symbols in DLLs as exported?
I suspect Robert is reading "DLL" as meaning only a Windows thing.
You're right, if you read it as a generic term for loadable libraries,
it's more or less applicable everywhere.
regards, tom lane
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: pgsql: Default to hidden visibility for extension libraries where possi
@ 2022-07-20 14:20 Tom Lane <tgl@sss.pgh.pa.us>
parent: Alvaro Herrera <alvherre@alvh.no-ip.org>
1 sibling, 1 reply; 27+ messages in thread
From: Tom Lane @ 2022-07-20 14:20 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Andres Freund <andres@anarazel.de>; Pavel Stehule <pavel.stehule@gmail.com>; pgsql-hackers@lists.postgresql.org
Alvaro Herrera <alvherre@alvh.no-ip.org> writes:
> I suppose you're probably thinking of commit 53ef6c40f1e7; that didn't
> expose functions directly, but through plpgsql_plugin_ptr. Maybe that
> one does need to be made PGDLLEXPORT, since currently it isn't.
Hm. Not sure if the rules are the same for global variables as
they are for functions, but if so, yeah ...
regards, tom lane
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: pgsql: Default to hidden visibility for extension libraries where possi
@ 2022-07-20 15:05 Andres Freund <andres@anarazel.de>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 0 replies; 27+ messages in thread
From: Andres Freund @ 2022-07-20 15:05 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Pavel Stehule <pavel.stehule@gmail.com>; pgsql-hackers@lists.postgresql.org
Hi,
On July 20, 2022 4:20:04 PM GMT+02:00, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>Alvaro Herrera <alvherre@alvh.no-ip.org> writes:
>> I suppose you're probably thinking of commit 53ef6c40f1e7; that didn't
>> expose functions directly, but through plpgsql_plugin_ptr. Maybe that
>> one does need to be made PGDLLEXPORT, since currently it isn't.
>
>Hm. Not sure if the rules are the same for global variables as
>they are for functions, but if so, yeah ...
They're the same on the export side. On windows the rules for linking to variables are stricter (they need declspec dllimport), but that doesn't matter for dlsym style stuff.
Andres
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: pgsql: Default to hidden visibility for extension libraries where possi
@ 2022-07-20 15:11 Tom Lane <tgl@sss.pgh.pa.us>
parent: Alvaro Herrera <alvherre@alvh.no-ip.org>
1 sibling, 1 reply; 27+ messages in thread
From: Tom Lane @ 2022-07-20 15:11 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Dave Page <dpage@postgresql.org>; Andres Freund <andres@anarazel.de>; Pavel Stehule <pavel.stehule@gmail.com>; pgsql-hackers@lists.postgresql.org
Alvaro Herrera <alvherre@alvh.no-ip.org> writes:
> On 2022-Jul-20, Tom Lane wrote:
>> I'll try to do some research later today to identify anything else
>> we need to mark in plpgsql. I recall doing some work specifically
>> creating functions for pldebugger's use, but I'll need to dig.
> I suppose you're probably thinking of commit 53ef6c40f1e7; that didn't
> expose functions directly, but through plpgsql_plugin_ptr. Maybe that
> one does need to be made PGDLLEXPORT, since currently it isn't.
After some experimentation, it does not need to be marked: pldebugger
gets at that via find_rendezvous_variable(), so there is no need for
any explicit linkage at all between plpgsql.so and plugin_debugger.so.
Along the way, I made a quick hack to get pldebugger to load into
v15/HEAD. It lacks #ifdef's which'd be needed so that it'd still
compile against older branches, but perhaps this'll save someone
some time.
regards, tom lane
Attachments:
pldebugger-shmem.patch (text/x-diff, 1.3K) download
diff --git a/plugin_debugger.c b/plugin_debugger.c
index 6620021..1bd2057 100644
--- a/plugin_debugger.c
+++ b/plugin_debugger.c
@@ -114,6 +114,8 @@ static debugger_language_t *debugger_languages[] = {
NULL
};
+static shmem_request_hook_type prev_shmem_request_hook = NULL;
+
/**********************************************************************
* Function declarations
**********************************************************************/
@@ -124,6 +126,7 @@ void _PG_init( void ); /* initialize this module when we are dynamically load
* Local (hidden) function prototypes
**********************************************************************/
+static void pldebugger_shmem_request( void );
//static char ** fetchArgNames( PLpgSQL_function * func, int * nameCount );
static void * writen( int peer, void * src, size_t len );
static bool connectAsServer( void );
@@ -154,6 +157,15 @@ void _PG_init( void )
for (i = 0; debugger_languages[i] != NULL; i++)
debugger_languages[i]->initialize();
+ prev_shmem_request_hook = shmem_request_hook;
+ shmem_request_hook = pldebugger_shmem_request;
+}
+
+static void pldebugger_shmem_request( void )
+{
+ if (prev_shmem_request_hook)
+ prev_shmem_request_hook();
+
reserveBreakpoints();
dbgcomm_reserve();
}
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: pgsql: Default to hidden visibility for extension libraries where possi
@ 2022-07-20 15:58 Dave Page <dpage@postgresql.org>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 0 replies; 27+ messages in thread
From: Dave Page @ 2022-07-20 15:58 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Andres Freund <andres@anarazel.de>; Pavel Stehule <pavel.stehule@gmail.com>; pgsql-hackers@lists.postgresql.org
On Wed, 20 Jul 2022 at 16:12, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Alvaro Herrera <alvherre@alvh.no-ip.org> writes:
> > On 2022-Jul-20, Tom Lane wrote:
> >> I'll try to do some research later today to identify anything else
> >> we need to mark in plpgsql. I recall doing some work specifically
> >> creating functions for pldebugger's use, but I'll need to dig.
>
> > I suppose you're probably thinking of commit 53ef6c40f1e7; that didn't
> > expose functions directly, but through plpgsql_plugin_ptr. Maybe that
> > one does need to be made PGDLLEXPORT, since currently it isn't.
>
> After some experimentation, it does not need to be marked: pldebugger
> gets at that via find_rendezvous_variable(), so there is no need for
> any explicit linkage at all between plpgsql.so and plugin_debugger.so.
>
> Along the way, I made a quick hack to get pldebugger to load into
> v15/HEAD. It lacks #ifdef's which'd be needed so that it'd still
> compile against older branches, but perhaps this'll save someone
> some time.
>
Thanks Tom - I've pushed that patch with the relevant #ifdefs added.
--
Dave Page
PostgreSQL Core Team
http://www.postgresql.org/
^ permalink raw reply [nested|flat] 27+ messages in thread
end of thread, other threads:[~2022-07-20 15:58 UTC | newest]
Thread overview: 27+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2022-07-18 01:05 pgsql: Default to hidden visibility for extension libraries where possi Andres Freund <andres@anarazel.de>
2022-07-18 01:39 ` Andres Freund <andres@anarazel.de>
2022-07-18 02:01 ` Andres Freund <andres@anarazel.de>
2022-07-18 02:34 ` Tom Lane <tgl@sss.pgh.pa.us>
2022-07-18 23:36 ` Justin Pryzby <pryzby@telsasoft.com>
2022-07-19 03:26 ` Pavel Stehule <pavel.stehule@gmail.com>
2022-07-19 13:18 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2022-07-19 13:45 ` Tom Lane <tgl@sss.pgh.pa.us>
2022-07-19 14:28 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2022-07-19 14:45 ` Julien Rouhaud <rjuju123@gmail.com>
2022-07-19 14:47 ` Pavel Stehule <pavel.stehule@gmail.com>
2022-07-19 15:31 ` Andres Freund <andres@anarazel.de>
2022-07-19 15:37 ` Pavel Stehule <pavel.stehule@gmail.com>
2022-07-19 15:40 ` Andres Freund <andres@anarazel.de>
2022-07-19 22:44 ` Andres Freund <andres@anarazel.de>
2022-07-19 22:39 ` Andres Freund <andres@anarazel.de>
2022-07-20 09:02 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2022-07-20 13:38 ` Tom Lane <tgl@sss.pgh.pa.us>
2022-07-20 13:54 ` Robert Haas <robertmhaas@gmail.com>
2022-07-20 14:09 ` Tom Lane <tgl@sss.pgh.pa.us>
2022-07-20 14:12 ` Andres Freund <andres@anarazel.de>
2022-07-20 14:15 ` Tom Lane <tgl@sss.pgh.pa.us>
2022-07-20 14:04 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2022-07-20 14:20 ` Tom Lane <tgl@sss.pgh.pa.us>
2022-07-20 15:05 ` Andres Freund <andres@anarazel.de>
2022-07-20 15:11 ` Tom Lane <tgl@sss.pgh.pa.us>
2022-07-20 15:58 ` Dave Page <dpage@postgresql.org>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox