From: Andrei Lepikhov <lepihov@gmail.com>
To: Michael Paquier <michael@paquier.xyz>
To: Alexander Korotkov <aekorotkov@gmail.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Yurii Rashkovskii <yrashk@omnigres.com>
Cc: PostgreSQL-development <pgsql-hackers@postgresql.org>
Subject: Re: Add Postgres module info
Date: Fri, 7 Mar 2025 16:56:41 +0100
Message-ID: <992ec366-5c0d-4f23-a02e-30e663613f44@gmail.com> (raw)
In-Reply-To: <29d29c3e-ac9e-48cc-87b4-b1cfd8c2ede8@gmail.com>
References: <dd4d1b59-d0fe-49d5-b28f-1e463b68fa32@gmail.com>
<506788.1733941263@sss.pgh.pa.us>
<2c215c98-f713-4c5b-b21a-a4b8fa41890b@gmail.com>
<556341.1733967371@sss.pgh.pa.us>
<0844622e-9967-409a-8038-470a36f1039c@gmail.com>
<CAG=VW14ZmKDcxYs7irSvhVS+JeDg2W_gyR++L2GXEpzVVYQ+Yw@mail.gmail.com>
<8b81a3c0-761f-4cfb-97c6-68819fe4d282@gmail.com>
<982748.1734059835@sss.pgh.pa.us>
<0e82bf8c-ae70-498d-861e-dba2bb154cad@gmail.com>
<CAPpHfduL6=LdHco_+wCk=VQyZzEd5hjLi46jtV2gQbDrpXf5Gw@mail.gmail.com>
<Z7Kmann0SyoP7kDr@paquier.xyz>
<29d29c3e-ac9e-48cc-87b4-b1cfd8c2ede8@gmail.com>
On 2/3/2025 20:35, Andrei Lepikhov wrote:
> On 17/2/2025 04:00, Michael Paquier wrote:
>> No documentation provided.
> Ok, I haven't been sure this idea has a chance to be committed. I will
> introduce the docs in the next version.
This is a new version with bug fixes. Also, use TAP tests instead of
regression tests. Still, no documentation is included.
--
regards, Andrei Lepikhov
Attachments:v4-0001-Introduce-PG_MODULE_MAGIC_EXT-macro.patch (text/plain, 11.9K) download
From 3ef2579bfa3dd0306d63e355ceae65f9f476dea8 Mon Sep 17 00:00:00 2001
From: "Andrei V. Lepikhov" <lepihov@gmail.com>
Date: Tue, 19 Nov 2024 18:45:36 +0700
Subject: [PATCH v4] Introduce PG_MODULE_MAGIC_EXT macro.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
This macro provides dynamically loaded shared libraries (modules) with standard
way to incorporate version (supposedly, defined according to semantic versioning
specification) and name data. The introduced catalogue routine pg_get_modules≈ can
be used to find this module by name and check the version. It makes users
independent from file naming conventions.
With a growing number of Postgres core hooks and the introduction of named DSM
segments, the number of modules that don't need to be loaded on startup may
grow fast. Moreover, in many cases related to query tree transformation or
extra path recommendation, such modules might not need database objects except
GUCs - see auto_explain as an example. That means they don't need to execute
the 'CREATE EXTENSION' statement at all and don't have a record in
the pg_extension table. Such a trick provides much flexibility, including
an online upgrade and may become widespread.
In addition, it is also convenient in support to be sure that the installation
(or at least the backend) includes a specific version of the module. Even if
a module has an installation script, it is not rare that it provides
an implementation for a range of UI routine versions. It makes sense to ensure
which specific version of the code is used.
Discussions [1,2] already mentioned module-info stuff, but at that time,
extensibility techniques and extension popularity were low, and it wasn't
necessary to provide that data.
[1] https://www.postgresql.org/message-id/flat/20060507211705.GB3808%40svana.org
[2] https://www.postgresql.org/message-id/flat/20051106162658.34c31d57%40thunder.logicalchaos.org
---
src/backend/utils/fmgr/dfmgr.c | 66 ++++++++++++++++++-
src/include/catalog/pg_proc.dat | 6 ++
src/include/fmgr.h | 28 +++++++-
src/test/modules/test_misc/Makefile | 5 +-
.../test_misc/t/008_check_pg_get_modules.pl | 65 ++++++++++++++++++
src/test/modules/test_shm_mq/test.c | 5 +-
src/test/modules/test_slru/test_slru.c | 5 +-
7 files changed, 172 insertions(+), 8 deletions(-)
create mode 100644 src/test/modules/test_misc/t/008_check_pg_get_modules.pl
diff --git a/src/backend/utils/fmgr/dfmgr.c b/src/backend/utils/fmgr/dfmgr.cindex 87b233cb887..70d6ced4157 100644--- a/src/backend/utils/fmgr/dfmgr.c+++ b/src/backend/utils/fmgr/dfmgr.c@@ -21,10 +21,12 @@
#endif /* !WIN32 */
#include "fmgr.h"
+#include "funcapi.h"
#include "lib/stringinfo.h"
#include "miscadmin.h"
#include "storage/fd.h"
#include "storage/shmem.h"
+#include "utils/builtins.h"
#include "utils/hsearch.h"
@@ -51,6 +53,7 @@ typedef struct df_files
ino_t inode; /* Inode number of file */
#endif
void *handle; /* a handle for pg_dl* functions */
+ const pg_module_info *minfo;
char filename[FLEXIBLE_ARRAY_MEMBER]; /* Full pathname of file */
} DynamicFileList;
@@ -75,7 +78,7 @@ static char *substitute_libpath_macro(const char *name);
static char *find_in_dynamic_libpath(const char *basename);
/* Magic structure that module needs to match to be accepted */
-static const Pg_magic_struct magic_data = PG_MODULE_MAGIC_DATA;+static const Pg_magic_struct magic_data = PG_MODULE_MAGIC_DATA(0);
/*
@@ -245,8 +248,14 @@ internal_load_library(const char *libname)
{
const Pg_magic_struct *magic_data_ptr = (*magic_func) ();
+ /*+ * Check magic field from loading library to be sure it compiled+ * for the same Postgres code. Skip maintainer fields at the end+ * of the struct.+ */
if (magic_data_ptr->len != magic_data.len ||
- memcmp(magic_data_ptr, &magic_data, magic_data.len) != 0)+ memcmp(magic_data_ptr, &magic_data,+ offsetof(Pg_magic_struct, module_extra)) != 0)
{
/* copy data block before unlinking library */
Pg_magic_struct module_magic_data = *magic_data_ptr;
@@ -258,6 +267,9 @@ internal_load_library(const char *libname)
/* issue suitable complaint */
incompatible_module_error(libname, &module_magic_data);
}
++ /* Save link to the maintainer-provided info */+ file_scanner->minfo = &magic_data_ptr->module_extra;
}
else
{
@@ -675,3 +687,53 @@ RestoreLibraryState(char *start_address)
start_address += strlen(start_address) + 1;
}
}
++Datum+pg_get_modules(PG_FUNCTION_ARGS)+{+ FuncCallContext *funcctx;+ MemoryContext oldcontext;+ DynamicFileList *file_scanner;+ Datum result;+ Datum values[3];+ bool isnull[3] = {0};++ if (SRF_IS_FIRSTCALL())+ {+ TupleDesc tupdesc;++ funcctx = SRF_FIRSTCALL_INIT();+ oldcontext = MemoryContextSwitchTo(funcctx->multi_call_memory_ctx);++ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)+ elog(ERROR, "return type must be a row type");++ funcctx->tuple_desc = BlessTupleDesc(tupdesc);+ funcctx->user_fctx = (void *) file_list;++ MemoryContextSwitchTo(oldcontext);+ }++ funcctx = SRF_PERCALL_SETUP();+ file_scanner = (DynamicFileList *) funcctx->user_fctx;++ if (file_scanner != NULL)+ {+ if (file_scanner->minfo->name == NULL)+ isnull[0] = true;+ else+ values[0] = CStringGetTextDatum(file_scanner->minfo->name);+ if (file_scanner->minfo->version == NULL)+ isnull[1] = true;+ else+ values[1] = CStringGetTextDatum(file_scanner->minfo->version);++ values[2] = CStringGetTextDatum(file_scanner->filename);+ result = HeapTupleGetDatum(heap_form_tuple(funcctx->tuple_desc,+ values, isnull));+ funcctx->user_fctx = file_scanner->next;+ SRF_RETURN_NEXT(funcctx, result);+ }+ else+ SRF_RETURN_DONE(funcctx);+}diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.datindex 134b3dd8689..254f16e0496 100644--- a/src/include/catalog/pg_proc.dat+++ b/src/include/catalog/pg_proc.dat@@ -311,6 +311,12 @@
proname => 'scalargtjoinsel', provolatile => 's', prorettype => 'float8',
proargtypes => 'internal oid internal int2 internal',
prosrc => 'scalargtjoinsel' },
+{ oid => '111', descr => 'Module Info',+ proname => 'pg_get_modules', provolatile => 's', prorettype => 'record',+ proretset => 't', proargtypes => '', proallargtypes => '{text,text,text}',+ proargmodes => '{o,o,o}', prorows => 10,+ proargnames => '{module_name,version,libname}',+ prosrc => 'pg_get_modules' },
{ oid => '336',
descr => 'restriction selectivity of <= and related operators on scalar datatypes',
diff --git a/src/include/fmgr.h b/src/include/fmgr.hindex e609ea875a7..5018abf3a18 100644--- a/src/include/fmgr.h+++ b/src/include/fmgr.h@@ -459,6 +459,12 @@ extern PGDLLEXPORT void _PG_init(void);
*-------------------------------------------------------------------------
*/
+typedef struct pg_module_info+{+ const char *name;+ const char *version;+} pg_module_info;+
/* Definition of the magic block structure */
typedef struct
{
@@ -469,10 +475,15 @@ typedef struct
int namedatalen; /* NAMEDATALEN */
int float8byval; /* FLOAT8PASSBYVAL */
char abi_extra[32]; /* see pg_config_manual.h */
+ pg_module_info module_extra;
} Pg_magic_struct;
-/* The actual data block contents */-#define PG_MODULE_MAGIC_DATA \+/* The actual data block contents+ *+ * Fill the module info part with zeros by default. Zero-length module name+ * indicates that it is not initialised.+ */+#define PG_MODULE_MAGIC_DATA(...) \
{ \
sizeof(Pg_magic_struct), \
PG_VERSION_NUM / 100, \
@@ -481,6 +492,7 @@ typedef struct
NAMEDATALEN, \
FLOAT8PASSBYVAL, \
FMGR_ABI_EXTRA, \
+ {__VA_ARGS__}, \
}
StaticAssertDecl(sizeof(FMGR_ABI_EXTRA) <= sizeof(((Pg_magic_struct *) 0)->abi_extra),
@@ -500,11 +512,21 @@ extern PGDLLEXPORT const Pg_magic_struct *PG_MAGIC_FUNCTION_NAME(void); \
const Pg_magic_struct * \
PG_MAGIC_FUNCTION_NAME(void) \
{ \
- static const Pg_magic_struct Pg_magic_data = PG_MODULE_MAGIC_DATA; \+ static const Pg_magic_struct Pg_magic_data = PG_MODULE_MAGIC_DATA(0); \
return &Pg_magic_data; \
} \
extern int no_such_variable
+#define PG_MODULE_MAGIC_EXT(...) \+extern PGDLLEXPORT const Pg_magic_struct *PG_MAGIC_FUNCTION_NAME(void); \+const Pg_magic_struct * \+PG_MAGIC_FUNCTION_NAME(void) \+{ \+ static const Pg_magic_struct Pg_magic_data = \+ PG_MODULE_MAGIC_DATA(__VA_ARGS__); \+ return &Pg_magic_data; \+} \+extern int no_such_variable
/*-------------------------------------------------------------------------
* Support routines and macros for callers of fmgr-compatible functions
diff --git a/src/test/modules/test_misc/Makefile b/src/test/modules/test_misc/Makefileindex 919a25fc67f..c576e6d61de 100644--- a/src/test/modules/test_misc/Makefile+++ b/src/test/modules/test_misc/Makefile@@ -2,7 +2,10 @@
TAP_TESTS = 1
-EXTRA_INSTALL=src/test/modules/injection_points+EXTRA_INSTALL=src/test/modules/injection_points \+ src/test/modules/test_shm_mq \+ src/test/modules/test_dsa \+ src/test/modules/test_slru
export enable_injection_points
diff --git a/src/test/modules/test_misc/t/008_check_pg_get_modules.pl b/src/test/modules/test_misc/t/008_check_pg_get_modules.plnew file mode 100644index 00000000000..57196eb3864--- /dev/null+++ b/src/test/modules/test_misc/t/008_check_pg_get_modules.pl@@ -0,0 +1,65 @@+#+# Copyright (c) 2025, PostgreSQL Global Development Group+#+# Tests of the pg_get_modules() function+#++use strict;+use warnings FATAL => 'all';+use PostgreSQL::Test::Cluster;+use PostgreSQL::Test::Utils;+use Test::More;++my $node = PostgreSQL::Test::Cluster->new('main');+$node->init;+$node->start;++# Empty set of loaded libraries+my $result = $node->safe_psql('postgres',+ "SELECT module_name,version FROM pg_get_modules()");+is($result, "", "No modules in the list");++$result = $node->safe_psql('postgres',+ qq{+ LOAD 'test_dsa';+ SELECT module_name,version FROM pg_get_modules();+ });+is($result, "|", "One library without version info");++$result = $node->safe_psql('postgres',+ qq{+ LOAD 'test_shm_mq';+ SELECT module_name,version FROM pg_get_modules();+ });+is($result, "test_shm_mq|1.0.0", "One library with version num");++$result = $node->safe_psql('postgres',+ qq{+ LOAD 'test_dsa';+ LOAD 'test_shm_mq';+ SELECT module_name,version FROM pg_get_modules();+ });+is($result, "|\ntest_shm_mq|1.0.0", "Couple of libraries, only one with not-null version number and name");++# Combine loaded on startup and dynamically loaded libraries++$node->append_conf('postgresql.conf',+ "shared_preload_libraries = 'test_slru'");+$node->restart;++$result = $node->safe_psql('postgres',+ qq{+ SELECT module_name,version FROM pg_get_modules();+ });+is($result, "test_slru|1.2.3", "One library with version info");++$result = $node->safe_psql('postgres',+ qq{+ LOAD 'test_dsa';+ LOAD 'test_shm_mq';+ SELECT module_name,version FROM pg_get_modules();+ });+is($result, "test_slru|1.2.3\n|\ntest_shm_mq|1.0.0", "Three libraries, two with not-null version number and name");+++done_testing();diff --git a/src/test/modules/test_shm_mq/test.c b/src/test/modules/test_shm_mq/test.cindex 443281addd0..130d1f81300 100644--- a/src/test/modules/test_shm_mq/test.c+++ b/src/test/modules/test_shm_mq/test.c@@ -20,7 +20,10 @@
#include "test_shm_mq.h"
-PG_MODULE_MAGIC;+PG_MODULE_MAGIC_EXT(+ .name = "test_shm_mq",+ .version = "1.0.0"+);
PG_FUNCTION_INFO_V1(test_shm_mq);
PG_FUNCTION_INFO_V1(test_shm_mq_pipelined);
diff --git a/src/test/modules/test_slru/test_slru.c b/src/test/modules/test_slru/test_slru.cindex 3ea5ceb8552..ed19ee1e719 100644--- a/src/test/modules/test_slru/test_slru.c+++ b/src/test/modules/test_slru/test_slru.c@@ -22,7 +22,10 @@
#include "storage/shmem.h"
#include "utils/builtins.h"
-PG_MODULE_MAGIC;+PG_MODULE_MAGIC_EXT(+ .name = "test_slru",+ .version = "1.2.3"+);
/*
* SQL-callable entry points
--
2.48.1
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-hackers@postgresql.org
Cc: lepihov@gmail.com, michael@paquier.xyz, aekorotkov@gmail.com, tgl@sss.pgh.pa.us, yrashk@omnigres.com
Subject: Re: Add Postgres module info
In-Reply-To: <992ec366-5c0d-4f23-a02e-30e663613f44@gmail.com>
References: <dd4d1b59-d0fe-49d5-b28f-1e463b68fa32@gmail.com> <506788.1733941263@sss.pgh.pa.us> <2c215c98-f713-4c5b-b21a-a4b8fa41890b@gmail.com> <556341.1733967371@sss.pgh.pa.us> <0844622e-9967-409a-8038-470a36f1039c@gmail.com> <CAG=VW14ZmKDcxYs7irSvhVS+JeDg2W_gyR++L2GXEpzVVYQ+Yw@mail.gmail.com> <8b81a3c0-761f-4cfb-97c6-68819fe4d282@gmail.com> <982748.1734059835@sss.pgh.pa.us> <0e82bf8c-ae70-498d-861e-dba2bb154cad@gmail.com> <CAPpHfduL6=LdHco_+wCk=VQyZzEd5hjLi46jtV2gQbDrpXf5Gw@mail.gmail.com> <Z7Kmann0SyoP7kDr@paquier.xyz> <29d29c3e-ac9e-48cc-87b4-b1cfd8c2ede8@gmail.com> <992ec366-5c0d-4f23-a02e-30e663613f44@gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox