public inbox for [email protected]  
help / color / mirror / Atom feed
[PATCH 01/17] doc: than than
12+ messages / 6 participants
[nested] [flat]

* [PATCH 01/17] doc: than than
@ 2022-01-25 01:40 Justin Pryzby <[email protected]>
  0 siblings, 0 replies; 12+ messages in thread

From: Justin Pryzby @ 2022-01-25 01:40 UTC (permalink / raw)

My mistake at 410aa248e5a883fde4832999cc9b23c7ace0f2ff
---
 doc/src/sgml/postgres-fdw.sgml | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/doc/src/sgml/postgres-fdw.sgml b/doc/src/sgml/postgres-fdw.sgml
index 8ebf0dc3a05..88f564d7fd8 100644
--- a/doc/src/sgml/postgres-fdw.sgml
+++ b/doc/src/sgml/postgres-fdw.sgml
@@ -1003,8 +1003,8 @@ postgres=# SELECT postgres_fdw_disconnect_all();
       it's passed to and used as <varname>application_name</varname>
       in a foreign server, note that it will be truncated to less than
       <symbol>NAMEDATALEN</symbol> characters and anything other than
-      than printable ASCII characters will be replaced with question
-      marks (<literal>?</literal>).
+      printable ASCII characters will be replaced with question marks
+      (<literal>?</literal>).
       See <xref linkend="guc-application-name"/> for details.
      </para>
 
-- 
2.17.1


--juZjCTNxrMaZdGZC
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
 filename="0002-doc-duplicate-words-THE-since-0aa8a01d04c8fe200b7a10.patch"



^ permalink  raw  reply  [nested|flat] 12+ messages in thread

* Re: RFC: Additional Directory for Extensions
@ 2024-06-25 14:43 Robert Haas <[email protected]>
  2024-06-25 22:31 ` Re: RFC: Additional Directory for Extensions David E. Wheeler <[email protected]>
  0 siblings, 1 reply; 12+ messages in thread

From: Robert Haas @ 2024-06-25 14:43 UTC (permalink / raw)
  To: Alvaro Herrera <[email protected]>; +Cc: David E. Wheeler <[email protected]>; pgsql-hackers

On Tue, Jun 25, 2024 at 6:12 AM Alvaro Herrera <[email protected]> wrote:
> Now, I'm not saying that this is an easy journey.  But if we don't
> start, we're not going to get there.

I actually kind of agree with you. I think I've said something similar
in a previous email to the list somewhere. But I don't agree that this
patch should be burdened with taking the first step. We seem to often
find reasons why patches that packagers for prominent distributions
are carrying shouldn't be put into core, and I think that's a bad
habit. They're not going to stop applying those packages because we
refuse to put suitable functionality in core; we're just creating a
situation where lots of people are running slightly patched versions
of PostgreSQL instead of straight-up PostgreSQL. That's not improving
anything. If we want to work on making the sorts of changes that
you're proposing, let's do it on a separate thread. It's not going to
be meaningfully harder to move in that direction after some patch like
this than it is today.

-- 
Robert Haas
EDB: http://www.enterprisedb.com






^ permalink  raw  reply  [nested|flat] 12+ messages in thread

* Re: RFC: Additional Directory for Extensions
  2024-06-25 14:43 Re: RFC: Additional Directory for Extensions Robert Haas <[email protected]>
@ 2024-06-25 22:31 ` David E. Wheeler <[email protected]>
  2024-06-26 09:30   ` Re: RFC: Additional Directory for Extensions Jelte Fennema-Nio <[email protected]>
  2024-07-08 16:01   ` Re: RFC: Additional Directory for Extensions David E. Wheeler <[email protected]>
  0 siblings, 2 replies; 12+ messages in thread

From: David E. Wheeler @ 2024-06-25 22:31 UTC (permalink / raw)
  To: Robert Haas <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; pgsql-hackers

On Jun 25, 2024, at 10:43 AM, Robert Haas <[email protected]> wrote:

> If we want to work on making the sorts of changes that
> you're proposing, let's do it on a separate thread. It's not going to
> be meaningfully harder to move in that direction after some patch like
> this than it is today.

I appreciate this separation of concerns, Robert.

In other news, here’s an updated patch that expands the documentation to record that the destination directory is a prefix, and full paths should be used under it. Also take the opportunity to document the PGXS DESTDIR variable as the thing to use to install files under the destination directory.

It still requires a server restart; I can change it back to superuser-only if that’s the consensus.

For those who prefer a GitHub patch review experience, see this PR:

  https://github.com/theory/postgres/pull/3/files

Best,

David






Attachments:

  [application/octet-stream] v4-0001-Add-extension_destdir-GUC.patch (11.5K, ../../[email protected]/2-v4-0001-Add-extension_destdir-GUC.patch)
  download | inline diff:
From 698bdd970c18585320b222ecc2b0906ace3177c6 Mon Sep 17 00:00:00 2001
From: "David E. Wheeler" <[email protected]>
Date: Tue, 25 Jun 2024 18:28:27 -0400
Subject: [PATCH v4] Add extension_destdir GUC

Based on a [patch] by Christophe Berg in the Debian Project, add a new
GUC, `extension_destdir`, that prepends a directory prefix for extension
loading. This directory is prepended to the `SHAREDIR` paths when
loading extensions (control and SQL files), and to the `$libdir`
directive when loading modules that back functions. Changing the
configuration requires a server restart, and is visible only to super
users.

Also document the PGXS `DESTDIR` variable, which should be used to
install extensions into the proper destination directory.

  [patch]: https://salsa.debian.org/postgresql/postgresql/-/blob/17/debian/patches/extension_destdir?ref_type=heads
---
 doc/src/sgml/config.sgml                      | 35 ++++++++
 doc/src/sgml/extend.sgml                      | 12 ++-
 src/backend/commands/extension.c              | 90 +++++++++++++++++++
 src/backend/utils/fmgr/dfmgr.c                | 29 +++++-
 src/backend/utils/misc/guc_tables.c           | 12 +++
 src/backend/utils/misc/postgresql.conf.sample |  2 +
 src/include/utils/guc.h                       |  1 +
 7 files changed, 179 insertions(+), 2 deletions(-)

diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 0c7a9082c5..350ed5b9c9 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -10389,6 +10389,41 @@ dynamic_library_path = 'C:\tools\postgresql;H:\my_project\lib;$libdir'
       </listitem>
      </varlistentry>
 
+     <varlistentry id="guc-extension-destdir" xreflabel="extension_destdir">
+      <term><varname>extension_destdir</varname> (<type>string</type>)
+      <indexterm>
+       <primary><varname>extension_destdir</varname> configuration parameter</primary>
+      </indexterm>
+      </term>
+      <listitem>
+       <para>
+        Specifies a directory prefix into which extensions should be
+        installed. When set, the postmaster will search this directory for an
+        extension before searching the default paths.
+       </para>
+
+       <para>
+        For example, this configuration:
+<programlisting>
+extension_destdir = '/mnt/extensions'
+</programlisting>
+        will allow <productname>PostgreSQL</productname> to first look for
+        extension control files, SQL files, and loadable modules installed in
+        <literal>/mnt/extensions</literal> and fall back on the
+        default directories if they're not found there.
+       </para>
+
+       <para>
+        Note that the files should be installed in their full paths under the
+        <varname>extension_destdir</varname> prefix. When using
+        <link linkend="extend-pgxs">PGXS</link> to install an extension, pass
+        the destination directory via the <varname>DESTDIR</varname> variable
+        to install the files in the proper location. For more information see
+        <xref linkend="extend-extensions-files-directory"/>.
+       </para>
+      </listitem>
+     </varlistentry>
+
      </variablelist>
     </sect2>
    </sect1>
diff --git a/doc/src/sgml/extend.sgml b/doc/src/sgml/extend.sgml
index 218940ee5c..6653955d53 100644
--- a/doc/src/sgml/extend.sgml
+++ b/doc/src/sgml/extend.sgml
@@ -669,7 +669,8 @@ RETURNS anycompatible AS ...
        <para>
         The directory containing the extension's <acronym>SQL</acronym> script
         file(s).  Unless an absolute path is given, the name is relative to
-        the installation's <literal>SHAREDIR</literal> directory.  The
+        the <literal>SHAREDIR</literal> under the <xref linkend="guc-extension-destdir"/>
+        prefix and to the installation's <literal>SHAREDIR</literal> directory.  The
         default behavior is equivalent to specifying
         <literal>directory = 'extension'</literal>.
        </para>
@@ -1710,6 +1711,15 @@ include $(PGXS)
       </listitem>
      </varlistentry>
 
+     <varlistentry id="extend-pgxs-destdir">
+      <term><varname>DESTDIR</varname></term>
+      <listitem>
+       <para>
+        install all files under this directory prefix
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry id="extend-pgxs-no-installcheck">
       <term><varname>NO_INSTALLCHECK</varname></term>
       <listitem>
diff --git a/src/backend/commands/extension.c b/src/backend/commands/extension.c
index 1643c8c69a..f3b7735c5b 100644
--- a/src/backend/commands/extension.c
+++ b/src/backend/commands/extension.c
@@ -393,6 +393,16 @@ get_extension_control_filename(const char *extname)
 
 	get_share_path(my_exec_path, sharepath);
 	result = (char *) palloc(MAXPGPATH);
+	/*
+	 * If extension_destdir is set, try to find the file there first
+	 */
+	if (*extension_destdir != '\0')
+	{
+		snprintf(result, MAXPGPATH, "%s%s/extension/%s.control",
+				 extension_destdir, sharepath, extname);
+		if (pg_file_exists(result))
+			return result;
+	}
 	snprintf(result, MAXPGPATH, "%s/extension/%s.control",
 			 sharepath, extname);
 
@@ -432,6 +442,16 @@ get_extension_aux_control_filename(ExtensionControlFile *control,
 	scriptdir = get_extension_script_directory(control);
 
 	result = (char *) palloc(MAXPGPATH);
+	/*
+	 * If extension_destdir is set, try to find the file there first
+	 */
+	if (*extension_destdir != '\0')
+	{
+		snprintf(result, MAXPGPATH, "%s%s/%s--%s.control",
+				 extension_destdir, scriptdir, control->name, version);
+		if (pg_file_exists(result))
+			return result;
+	}
 	snprintf(result, MAXPGPATH, "%s/%s--%s.control",
 			 scriptdir, control->name, version);
 
@@ -450,6 +470,23 @@ get_extension_script_filename(ExtensionControlFile *control,
 	scriptdir = get_extension_script_directory(control);
 
 	result = (char *) palloc(MAXPGPATH);
+	/*
+	 * If extension_destdir is set, try to find the file there first
+	 */
+	if (*extension_destdir != '\0')
+	{
+		if (from_version)
+			snprintf(result, MAXPGPATH, "%s%s/%s--%s--%s.sql",
+					 extension_destdir, scriptdir, control->name, from_version, version);
+		else
+			snprintf(result, MAXPGPATH, "%s%s/%s--%s.sql",
+					 extension_destdir, scriptdir, control->name, version);
+		if (pg_file_exists(result))
+		{
+			pfree(scriptdir);
+			return result;
+		}
+	}
 	if (from_version)
 		snprintf(result, MAXPGPATH, "%s/%s--%s--%s.sql",
 				 scriptdir, control->name, from_version, version);
@@ -1209,6 +1246,59 @@ get_ext_ver_list(ExtensionControlFile *control)
 	DIR		   *dir;
 	struct dirent *de;
 
+	/*
+	 * If extension_destdir is set, try to find the files there first
+	 */
+	if (*extension_destdir != '\0')
+	{
+		char		location[MAXPGPATH];
+
+		snprintf(location, MAXPGPATH, "%s%s", extension_destdir,
+				get_extension_script_directory(control));
+		dir = AllocateDir(location);
+		while ((de = ReadDir(dir, location)) != NULL)
+		{
+			char	   *vername;
+			char	   *vername2;
+			ExtensionVersionInfo *evi;
+			ExtensionVersionInfo *evi2;
+
+			/* must be a .sql file ... */
+			if (!is_extension_script_filename(de->d_name))
+				continue;
+
+			/* ... matching extension name followed by separator */
+			if (strncmp(de->d_name, control->name, extnamelen) != 0 ||
+				de->d_name[extnamelen] != '-' ||
+				de->d_name[extnamelen + 1] != '-')
+				continue;
+
+			/* extract version name(s) from 'extname--something.sql' filename */
+			vername = pstrdup(de->d_name + extnamelen + 2);
+			*strrchr(vername, '.') = '\0';
+			vername2 = strstr(vername, "--");
+			if (!vername2)
+			{
+				/* It's an install, not update, script; record its version name */
+				evi = get_ext_ver_info(vername, &evi_list);
+				evi->installable = true;
+				continue;
+			}
+			*vername2 = '\0';		/* terminate first version */
+			vername2 += 2;			/* and point to second */
+
+			/* if there's a third --, it's bogus, ignore it */
+			if (strstr(vername2, "--"))
+				continue;
+
+			/* Create ExtensionVersionInfos and link them together */
+			evi = get_ext_ver_info(vername, &evi_list);
+			evi2 = get_ext_ver_info(vername2, &evi_list);
+			evi->reachable = lappend(evi->reachable, evi2);
+		}
+		FreeDir(dir);
+	}
+
 	location = get_extension_script_directory(control);
 	dir = AllocateDir(location);
 	while ((de = ReadDir(dir, location)) != NULL)
diff --git a/src/backend/utils/fmgr/dfmgr.c b/src/backend/utils/fmgr/dfmgr.c
index 092004dcf3..25971b25b6 100644
--- a/src/backend/utils/fmgr/dfmgr.c
+++ b/src/backend/utils/fmgr/dfmgr.c
@@ -35,6 +35,7 @@
 #include "miscadmin.h"
 #include "storage/fd.h"
 #include "storage/shmem.h"
+#include "utils/guc.h"
 #include "utils/hsearch.h"
 
 
@@ -415,7 +416,7 @@ expand_dynamic_library_name(const char *name)
 {
 	bool		have_slash;
 	char	   *new;
-	char	   *full;
+	char	   *full, *full2;
 
 	Assert(name);
 
@@ -430,6 +431,19 @@ expand_dynamic_library_name(const char *name)
 	else
 	{
 		full = substitute_libpath_macro(name);
+		/*
+		 * If extension_destdir is set, try to find the file there first
+		 */
+		if (*extension_destdir != '\0')
+		{
+			full2 = psprintf("%s%s", extension_destdir, full);
+			if (pg_file_exists(full2))
+			{
+				pfree(full);
+				return full2;
+			}
+			pfree(full2);
+		}
 		if (pg_file_exists(full))
 			return full;
 		pfree(full);
@@ -448,6 +462,19 @@ expand_dynamic_library_name(const char *name)
 	{
 		full = substitute_libpath_macro(new);
 		pfree(new);
+		/*
+		 * If extension_destdir is set, try to find the file there first
+		 */
+		if (*extension_destdir != '\0')
+		{
+			full2 = psprintf("%s%s", extension_destdir, full);
+			if (pg_file_exists(full2))
+			{
+				pfree(full);
+				return full2;
+			}
+			pfree(full2);
+		}
 		if (pg_file_exists(full))
 			return full;
 		pfree(full);
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 46c258be28..8a3c6e2968 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -541,6 +541,7 @@ char	   *ConfigFileName;
 char	   *HbaFileName;
 char	   *IdentFileName;
 char	   *external_pid_file;
+char	   *extension_destdir;
 
 char	   *application_name;
 
@@ -4487,6 +4488,17 @@ struct config_string ConfigureNamesString[] =
 		check_canonical_path, NULL, NULL
 	},
 
+	{
+		{"extension_destdir", PGC_POSTMASTER, FILE_LOCATIONS,
+			gettext_noop("Path to prepend for extension loading."),
+			gettext_noop("This directory is prepended to paths when loading extensions (control and SQL files), and to the '$libdir' directive when loading modules that back functions. The location is made configurable to allow build-time testing of extensions that do not have been installed to their proper location yet."),
+			GUC_SUPERUSER_ONLY
+		},
+		&extension_destdir,
+		"",
+		NULL, NULL, NULL
+	},
+
 	{
 		{"ssl_library", PGC_INTERNAL, PRESET_OPTIONS,
 			gettext_noop("Shows the name of the SSL library."),
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index e0567de219..93a70487b6 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -771,6 +771,8 @@
 # - Other Defaults -
 
 #dynamic_library_path = '$libdir'
+#extension_destdir = ''			# prepend path when loading extensions
+					# and shared objects (added by Debian)
 #gin_fuzzy_search_limit = 0
 
 
diff --git a/src/include/utils/guc.h b/src/include/utils/guc.h
index ff506bf48d..eaf0b4f337 100644
--- a/src/include/utils/guc.h
+++ b/src/include/utils/guc.h
@@ -277,6 +277,7 @@ extern PGDLLIMPORT char *ConfigFileName;
 extern PGDLLIMPORT char *HbaFileName;
 extern PGDLLIMPORT char *IdentFileName;
 extern PGDLLIMPORT char *external_pid_file;
+extern PGDLLIMPORT char *extension_destdir;
 
 extern PGDLLIMPORT char *application_name;
 
-- 
2.45.2



^ permalink  raw  reply  [nested|flat] 12+ messages in thread

* Re: RFC: Additional Directory for Extensions
  2024-06-25 14:43 Re: RFC: Additional Directory for Extensions Robert Haas <[email protected]>
  2024-06-25 22:31 ` Re: RFC: Additional Directory for Extensions David E. Wheeler <[email protected]>
@ 2024-06-26 09:30   ` Jelte Fennema-Nio <[email protected]>
  1 sibling, 0 replies; 12+ messages in thread

From: Jelte Fennema-Nio @ 2024-06-26 09:30 UTC (permalink / raw)
  To: David E. Wheeler <[email protected]>; +Cc: Robert Haas <[email protected]>; Alvaro Herrera <[email protected]>; pgsql-hackers

On Wed, 26 Jun 2024 at 00:32, David E. Wheeler <[email protected]> wrote:
> In other news, here’s an updated patch that expands the documentation to record that the destination directory is a prefix, and full paths should be used under it. Also take the opportunity to document the PGXS DESTDIR variable as the thing to use to install files under the destination directory.

Docs are much clearer now thanks.

        full = substitute_libpath_macro(name);
+       /*
+        * If extension_destdir is set, try to find the file there first
+        */
+       if (*extension_destdir != '\0')
+       {
+           full2 = psprintf("%s%s", extension_destdir, full);
+           if (pg_file_exists(full2))
+           {
+               pfree(full);
+               return full2;
+           }
+           pfree(full2);
+       }

I think this should be done differently. For two reasons:
1. I don't think extension_destdir should be searched when $libdir is
not part of the name.
2. find_in_dynamic_libpath currently doesn't use extension_destdir at
all, so if there is no slash in the filename we do not search
extension_destdir.

I feel like changing the substitute_libpath_macro function a bit (or
adding a new similar function) is probably the best way to achieve
that.

We should also check somewhere (probably GUC check hook) that
extension_destdir is an absolute path.

> It still requires a server restart;

When reading the code I see no reason why this cannot be PGC_SIGHUP.
Even though it's probably not needed to change on a running server, I
think it's better to allow that. Even just so people can disable it if
necessary for some reason without restarting the process.

> I can change it back to superuser-only if that’s the consensus.

It still is GUC_SUPERUSER_ONLY, right?

> For those who prefer a GitHub patch review experience, see this PR:
>
>   https://github.com/theory/postgres/pull/3/files

Sidenote: The "D" link for each patch on cfbot[1] now gives a similar
link for all commitfest entries[2].

[1]: http://cfbot.cputube.org/
[2]: https://github.com/postgresql-cfbot/postgresql/compare/cf/4913~1...cf/4913






^ permalink  raw  reply  [nested|flat] 12+ messages in thread

* Re: RFC: Additional Directory for Extensions
  2024-06-25 14:43 Re: RFC: Additional Directory for Extensions Robert Haas <[email protected]>
  2024-06-25 22:31 ` Re: RFC: Additional Directory for Extensions David E. Wheeler <[email protected]>
@ 2024-07-08 16:01   ` David E. Wheeler <[email protected]>
  2024-08-21 19:59     ` Re: RFC: Additional Directory for Extensions Gabriele Bartolini <[email protected]>
  1 sibling, 1 reply; 12+ messages in thread

From: David E. Wheeler @ 2024-07-08 16:01 UTC (permalink / raw)
  To: Robert Haas <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; pgsql-hackers

On Jun 25, 2024, at 18:31, David E. Wheeler <[email protected]> wrote:

> For those who prefer a GitHub patch review experience, see this PR:
> 
>  https://github.com/theory/postgres/pull/3/files

Rebased and restored PGC_SUSET in the attached v5 patch, plus noted the required privileges in the docs.

Best,

David






Attachments:

  [application/octet-stream] v5-0001-Add-extension_destdir-GUC.patch (11.6K, ../../[email protected]/2-v5-0001-Add-extension_destdir-GUC.patch)
  download | inline diff:
From c1a146b18187fa1ba96000815aab0a574e0110f2 Mon Sep 17 00:00:00 2001
From: "David E. Wheeler" <[email protected]>
Date: Mon, 8 Jul 2024 11:58:45 -0400
Subject: [PATCH v5] Add extension_destdir GUC

Based on a [patch] by Christophe Berg in the Debian Project, add a new
GUC, `extension_destdir`, that prepends a directory prefix for extension
loading. This directory is prepended to the `SHAREDIR` paths when
loading extensions (control and SQL files), and to the `$libdir`
directive when loading modules that back functions. Requires a superuser
or user with the appropriate SET privilege.

Also document the PGXS `DESTDIR` variable, which should be used to
install extensions into the proper destination directory.

  [patch]: https://salsa.debian.org/postgresql/postgresql/-/blob/17/debian/patches/extension_destdir?ref_type=heads
---
 doc/src/sgml/config.sgml                      | 37 ++++++++
 doc/src/sgml/extend.sgml                      | 12 ++-
 src/backend/commands/extension.c              | 90 +++++++++++++++++++
 src/backend/utils/fmgr/dfmgr.c                | 29 +++++-
 src/backend/utils/misc/guc_tables.c           | 12 +++
 src/backend/utils/misc/postgresql.conf.sample |  2 +
 src/include/utils/guc.h                       |  1 +
 7 files changed, 181 insertions(+), 2 deletions(-)

diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index f627a3e63c..defe218007 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -10389,6 +10389,43 @@ dynamic_library_path = 'C:\tools\postgresql;H:\my_project\lib;$libdir'
       </listitem>
      </varlistentry>
 
+     <varlistentry id="guc-extension-destdir" xreflabel="extension_destdir">
+      <term><varname>extension_destdir</varname> (<type>string</type>)
+      <indexterm>
+       <primary><varname>extension_destdir</varname> configuration parameter</primary>
+      </indexterm>
+      </term>
+      <listitem>
+       <para>
+        Specifies a directory prefix into which extensions should be
+        installed. Only superusers and users with the appropriate
+        <literal>SET</literal> privilege can change this setting. When set,
+        the postmaster will search this directory for an extension before
+        searching the default paths.
+       </para>
+
+       <para>
+        For example, this configuration:
+<programlisting>
+extension_destdir = '/mnt/extensions'
+</programlisting>
+        will allow <productname>PostgreSQL</productname> to first look for
+        extension control files, SQL files, and loadable modules installed in
+        <literal>/mnt/extensions</literal> and fall back on the
+        default directories if they're not found there.
+       </para>
+
+       <para>
+        Note that the files should be installed in their full paths under the
+        <varname>extension_destdir</varname> prefix. When using
+        <link linkend="extend-pgxs">PGXS</link> to install an extension, pass
+        the destination directory via the <varname>DESTDIR</varname> variable
+        to install the files in the proper location. For more information see
+        <xref linkend="extend-extensions-files-directory"/>.
+       </para>
+      </listitem>
+     </varlistentry>
+
      </variablelist>
     </sect2>
    </sect1>
diff --git a/doc/src/sgml/extend.sgml b/doc/src/sgml/extend.sgml
index 218940ee5c..6653955d53 100644
--- a/doc/src/sgml/extend.sgml
+++ b/doc/src/sgml/extend.sgml
@@ -669,7 +669,8 @@ RETURNS anycompatible AS ...
        <para>
         The directory containing the extension's <acronym>SQL</acronym> script
         file(s).  Unless an absolute path is given, the name is relative to
-        the installation's <literal>SHAREDIR</literal> directory.  The
+        the <literal>SHAREDIR</literal> under the <xref linkend="guc-extension-destdir"/>
+        prefix and to the installation's <literal>SHAREDIR</literal> directory.  The
         default behavior is equivalent to specifying
         <literal>directory = 'extension'</literal>.
        </para>
@@ -1710,6 +1711,15 @@ include $(PGXS)
       </listitem>
      </varlistentry>
 
+     <varlistentry id="extend-pgxs-destdir">
+      <term><varname>DESTDIR</varname></term>
+      <listitem>
+       <para>
+        install all files under this directory prefix
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry id="extend-pgxs-no-installcheck">
       <term><varname>NO_INSTALLCHECK</varname></term>
       <listitem>
diff --git a/src/backend/commands/extension.c b/src/backend/commands/extension.c
index 1643c8c69a..f3b7735c5b 100644
--- a/src/backend/commands/extension.c
+++ b/src/backend/commands/extension.c
@@ -393,6 +393,16 @@ get_extension_control_filename(const char *extname)
 
 	get_share_path(my_exec_path, sharepath);
 	result = (char *) palloc(MAXPGPATH);
+	/*
+	 * If extension_destdir is set, try to find the file there first
+	 */
+	if (*extension_destdir != '\0')
+	{
+		snprintf(result, MAXPGPATH, "%s%s/extension/%s.control",
+				 extension_destdir, sharepath, extname);
+		if (pg_file_exists(result))
+			return result;
+	}
 	snprintf(result, MAXPGPATH, "%s/extension/%s.control",
 			 sharepath, extname);
 
@@ -432,6 +442,16 @@ get_extension_aux_control_filename(ExtensionControlFile *control,
 	scriptdir = get_extension_script_directory(control);
 
 	result = (char *) palloc(MAXPGPATH);
+	/*
+	 * If extension_destdir is set, try to find the file there first
+	 */
+	if (*extension_destdir != '\0')
+	{
+		snprintf(result, MAXPGPATH, "%s%s/%s--%s.control",
+				 extension_destdir, scriptdir, control->name, version);
+		if (pg_file_exists(result))
+			return result;
+	}
 	snprintf(result, MAXPGPATH, "%s/%s--%s.control",
 			 scriptdir, control->name, version);
 
@@ -450,6 +470,23 @@ get_extension_script_filename(ExtensionControlFile *control,
 	scriptdir = get_extension_script_directory(control);
 
 	result = (char *) palloc(MAXPGPATH);
+	/*
+	 * If extension_destdir is set, try to find the file there first
+	 */
+	if (*extension_destdir != '\0')
+	{
+		if (from_version)
+			snprintf(result, MAXPGPATH, "%s%s/%s--%s--%s.sql",
+					 extension_destdir, scriptdir, control->name, from_version, version);
+		else
+			snprintf(result, MAXPGPATH, "%s%s/%s--%s.sql",
+					 extension_destdir, scriptdir, control->name, version);
+		if (pg_file_exists(result))
+		{
+			pfree(scriptdir);
+			return result;
+		}
+	}
 	if (from_version)
 		snprintf(result, MAXPGPATH, "%s/%s--%s--%s.sql",
 				 scriptdir, control->name, from_version, version);
@@ -1209,6 +1246,59 @@ get_ext_ver_list(ExtensionControlFile *control)
 	DIR		   *dir;
 	struct dirent *de;
 
+	/*
+	 * If extension_destdir is set, try to find the files there first
+	 */
+	if (*extension_destdir != '\0')
+	{
+		char		location[MAXPGPATH];
+
+		snprintf(location, MAXPGPATH, "%s%s", extension_destdir,
+				get_extension_script_directory(control));
+		dir = AllocateDir(location);
+		while ((de = ReadDir(dir, location)) != NULL)
+		{
+			char	   *vername;
+			char	   *vername2;
+			ExtensionVersionInfo *evi;
+			ExtensionVersionInfo *evi2;
+
+			/* must be a .sql file ... */
+			if (!is_extension_script_filename(de->d_name))
+				continue;
+
+			/* ... matching extension name followed by separator */
+			if (strncmp(de->d_name, control->name, extnamelen) != 0 ||
+				de->d_name[extnamelen] != '-' ||
+				de->d_name[extnamelen + 1] != '-')
+				continue;
+
+			/* extract version name(s) from 'extname--something.sql' filename */
+			vername = pstrdup(de->d_name + extnamelen + 2);
+			*strrchr(vername, '.') = '\0';
+			vername2 = strstr(vername, "--");
+			if (!vername2)
+			{
+				/* It's an install, not update, script; record its version name */
+				evi = get_ext_ver_info(vername, &evi_list);
+				evi->installable = true;
+				continue;
+			}
+			*vername2 = '\0';		/* terminate first version */
+			vername2 += 2;			/* and point to second */
+
+			/* if there's a third --, it's bogus, ignore it */
+			if (strstr(vername2, "--"))
+				continue;
+
+			/* Create ExtensionVersionInfos and link them together */
+			evi = get_ext_ver_info(vername, &evi_list);
+			evi2 = get_ext_ver_info(vername2, &evi_list);
+			evi->reachable = lappend(evi->reachable, evi2);
+		}
+		FreeDir(dir);
+	}
+
 	location = get_extension_script_directory(control);
 	dir = AllocateDir(location);
 	while ((de = ReadDir(dir, location)) != NULL)
diff --git a/src/backend/utils/fmgr/dfmgr.c b/src/backend/utils/fmgr/dfmgr.c
index 092004dcf3..25971b25b6 100644
--- a/src/backend/utils/fmgr/dfmgr.c
+++ b/src/backend/utils/fmgr/dfmgr.c
@@ -35,6 +35,7 @@
 #include "miscadmin.h"
 #include "storage/fd.h"
 #include "storage/shmem.h"
+#include "utils/guc.h"
 #include "utils/hsearch.h"
 
 
@@ -415,7 +416,7 @@ expand_dynamic_library_name(const char *name)
 {
 	bool		have_slash;
 	char	   *new;
-	char	   *full;
+	char	   *full, *full2;
 
 	Assert(name);
 
@@ -430,6 +431,19 @@ expand_dynamic_library_name(const char *name)
 	else
 	{
 		full = substitute_libpath_macro(name);
+		/*
+		 * If extension_destdir is set, try to find the file there first
+		 */
+		if (*extension_destdir != '\0')
+		{
+			full2 = psprintf("%s%s", extension_destdir, full);
+			if (pg_file_exists(full2))
+			{
+				pfree(full);
+				return full2;
+			}
+			pfree(full2);
+		}
 		if (pg_file_exists(full))
 			return full;
 		pfree(full);
@@ -448,6 +462,19 @@ expand_dynamic_library_name(const char *name)
 	{
 		full = substitute_libpath_macro(new);
 		pfree(new);
+		/*
+		 * If extension_destdir is set, try to find the file there first
+		 */
+		if (*extension_destdir != '\0')
+		{
+			full2 = psprintf("%s%s", extension_destdir, full);
+			if (pg_file_exists(full2))
+			{
+				pfree(full);
+				return full2;
+			}
+			pfree(full2);
+		}
 		if (pg_file_exists(full))
 			return full;
 		pfree(full);
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 630ed0f162..1f5430fd76 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -543,6 +543,7 @@ char	   *ConfigFileName;
 char	   *HbaFileName;
 char	   *IdentFileName;
 char	   *external_pid_file;
+char	   *extension_destdir;
 
 char	   *application_name;
 
@@ -4489,6 +4490,17 @@ struct config_string ConfigureNamesString[] =
 		check_canonical_path, NULL, NULL
 	},
 
+	{
+		{"extension_destdir", PGC_SUSET, FILE_LOCATIONS,
+			gettext_noop("Path to prepend for extension loading."),
+			gettext_noop("This directory is prepended to paths when loading extensions (control and SQL files), and to the '$libdir' directive when loading modules that back functions. The location is made configurable to allow build-time testing of extensions that do not have been installed to their proper location yet."),
+			GUC_SUPERUSER_ONLY
+		},
+		&extension_destdir,
+		"",
+		NULL, NULL, NULL
+	},
+
 	{
 		{"ssl_library", PGC_INTERNAL, PRESET_OPTIONS,
 			gettext_noop("Shows the name of the SSL library."),
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 9ec9f97e92..152997e67e 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -771,6 +771,8 @@
 # - Other Defaults -
 
 #dynamic_library_path = '$libdir'
+#extension_destdir = ''			# prepend path when loading extensions
+					# and shared objects (added by Debian)
 #gin_fuzzy_search_limit = 0
 
 
diff --git a/src/include/utils/guc.h b/src/include/utils/guc.h
index ff506bf48d..eaf0b4f337 100644
--- a/src/include/utils/guc.h
+++ b/src/include/utils/guc.h
@@ -277,6 +277,7 @@ extern PGDLLIMPORT char *ConfigFileName;
 extern PGDLLIMPORT char *HbaFileName;
 extern PGDLLIMPORT char *IdentFileName;
 extern PGDLLIMPORT char *external_pid_file;
+extern PGDLLIMPORT char *extension_destdir;
 
 extern PGDLLIMPORT char *application_name;
 
-- 
2.45.2



^ permalink  raw  reply  [nested|flat] 12+ messages in thread

* Re: RFC: Additional Directory for Extensions
  2024-06-25 14:43 Re: RFC: Additional Directory for Extensions Robert Haas <[email protected]>
  2024-06-25 22:31 ` Re: RFC: Additional Directory for Extensions David E. Wheeler <[email protected]>
  2024-07-08 16:01   ` Re: RFC: Additional Directory for Extensions David E. Wheeler <[email protected]>
@ 2024-08-21 19:59     ` Gabriele Bartolini <[email protected]>
  2024-08-21 23:07       ` Re: RFC: Additional Directory for Extensions Craig Ringer <[email protected]>
  0 siblings, 1 reply; 12+ messages in thread

From: Gabriele Bartolini @ 2024-08-21 19:59 UTC (permalink / raw)
  To: David E. Wheeler <[email protected]>; +Cc: Robert Haas <[email protected]>; Alvaro Herrera <[email protected]>; pgsql-hackers

Hi everyone,

Apologies for only starting to look into this now. Thanks, David, for
pushing this forward.

I want to emphasize the importance of this patch for the broader adoption
of extensions in immutable container environments, such as those used by
the CloudNativePG operator in Kubernetes.

To provide some context, one of the key principles of CloudNativePG is that
containers, once started, cannot be modified—this includes the installation
of Postgres extensions and their libraries. This restriction prevents us
from adding extensions on the fly, requiring them to be included in the
main PostgreSQL operand image. As a result, users who need specific
extensions must build custom images through automated pipelines (see:
https://cloudnative-pg.io/blog/creating-container-images/).

We’ve been considering ways to improve this process for some time. The
direction we're exploring involves mounting an ephemeral volume that
contains the necessary extensions (namely $sharedir and $pkglibdir from
pg_config). These volumes would be created and populated with the required
extensions when the container starts and destroyed when it shuts down. To
make this work, each extension must be independently packaged as a
container image containing the appropriate files for a specific extension
version, tailored to the architecture, distribution, OS version, and
Postgres version.

I’m committed to thoroughly reviewing this patch, testing it with
CloudNativePG and a few extensions, and providing feedback as soon as
possible.

Best,
Gabriele

On Mon, 8 Jul 2024 at 18:02, David E. Wheeler <[email protected]> wrote:

> On Jun 25, 2024, at 18:31, David E. Wheeler <[email protected]> wrote:
>
> > For those who prefer a GitHub patch review experience, see this PR:
> >
> >  https://github.com/theory/postgres/pull/3/files
>
> Rebased and restored PGC_SUSET in the attached v5 patch, plus noted the
> required privileges in the docs.
>
> Best,
>
> David
>
>
>
>

-- 
Gabriele Bartolini
Vice President, Cloud Native at EDB
enterprisedb.com


^ permalink  raw  reply  [nested|flat] 12+ messages in thread

* Re: RFC: Additional Directory for Extensions
  2024-06-25 14:43 Re: RFC: Additional Directory for Extensions Robert Haas <[email protected]>
  2024-06-25 22:31 ` Re: RFC: Additional Directory for Extensions David E. Wheeler <[email protected]>
  2024-07-08 16:01   ` Re: RFC: Additional Directory for Extensions David E. Wheeler <[email protected]>
  2024-08-21 19:59     ` Re: RFC: Additional Directory for Extensions Gabriele Bartolini <[email protected]>
@ 2024-08-21 23:07       ` Craig Ringer <[email protected]>
  2024-08-22 07:31         ` Re: RFC: Additional Directory for Extensions Jelte Fennema-Nio <[email protected]>
  2024-08-22 08:58         ` Re: RFC: Additional Directory for Extensions Gabriele Bartolini <[email protected]>
  0 siblings, 2 replies; 12+ messages in thread

From: Craig Ringer @ 2024-08-21 23:07 UTC (permalink / raw)
  To: Gabriele Bartolini <[email protected]>; +Cc: David E. Wheeler <[email protected]>; Robert Haas <[email protected]>; Alvaro Herrera <[email protected]>; pgsql-hackers

On Thu, 22 Aug 2024 at 08:00, Gabriele Bartolini
<[email protected]> wrote:
>
> Hi everyone,
>
> Apologies for only starting to look into this now. Thanks, David, for pushing this forward.


 100%. I've wanted this for some time but never had time to cook up a patch.

> I want to emphasize the importance of this patch for the broader adoption of extensions in immutable container environments, such as those used by the CloudNativePG operator in Kubernetes.


It's also very relevant for local development and testing.

Right now postgres makes it impossible to locally compile and install
an extension for a distro-packaged postgres (whether from upstream
repos or PGDG repos) without dirtying the distro-managed filesystem
subtrees with local changes under /usr etc, because it cannot be
configured to look for locally installed extensions on non-default
paths.

> To provide some context, one of the key principles of CloudNativePG is that containers, once started, cannot be modified—this includes the installation of Postgres extensions and their libraries. This restriction prevents us from adding extensions on the fly, requiring them to be included in the main PostgreSQL operand image. As a result, users who need specific extensions must build custom images through automated pipelines (see: https://cloudnative-pg.io/blog/creating-container-images/).


It may be possible to weaken this restriction somewhat thanks to the
upcoming https://kubernetes.io/blog/2024/08/16/kubernetes-1-31-image-volume-source/
feature that permits additional OCI images to be mounted as read-only
volumes on a workload. This would still only permit mounting at
Pod-creation time, not runtime mounting and unmonuting, but means the
base postgres image could be supplemented by mounting additional
images for extensions.

For example, one might mount image "postgis-vX.Y.Z" image onto base
image "postgresql-16" if support for PostGIS is desired, without then
having to bake every possible extension anyone might ever want into
the base image. This solves all sorts of messy issues with upgrades
and new version releases.

But for it to work, it must be possible to tell postgres to look in
_multiple places_ for extension .sql scripts and control files. This
is presently possible for modules (dynamic libraries, .so / .dylib /
.dll) but without a way to also configure the path for extensions it's
of very limited utility.

> We’ve been considering ways to improve this process for some time. The direction we're exploring involves mounting an ephemeral volume that contains the necessary extensions (namely $sharedir and $pkglibdir from pg_config). These volumes would be created and populated with the required extensions when the container starts and destroyed when it shuts down. To make this work, each extension must be independently packaged as a container image containing the appropriate files for a specific extension version, tailored to the architecture, distribution, OS version, and Postgres version.


Right. And there might be more than one of them.

So IMO this should be a _path_ to search for extension control files
and SQL scripts.

If the current built-in default extension dir was exposed as a var
$extdir like we do for $libdir, this might look something like this
for local development and testing while working with a packaged
postgres build:

    SET extension_search_path = $extsdir, /opt/myapp/extensions,
/usr/local/postgres/my-custom-extension/extensions;
    SET dynamic_library_path = $libdir, /opt/myapp/lib,
/usr/local/postgres/my-custom-extension/lib

or in the container extensions case, something like:

    SET extension_search_path = $extsdir,
/mnt/extensions/pg16/postgis-vX.Y/extensions,
/mnt/extensions/pg16/gosuperfast/extensions;
    SET dynamic_library_path = $libdir,
/mnt/extensions/pg16/postgis-vX.Y/lib,
/mnt/extensions/pg16/gosuperfast/lib;

For safety, it might make sense to impose the restriction that if an
extension control file is found in a given directory, SQL scripts will
also only be looked for in that same directory. That way there's no
chance of accidentally mixing and matching SQL scripts from different
versions of an extension if it appears twice on the extension search
path in different places. The rule for loading SQL scripts would be:

* locate first directory on path contianing matching extension control file
* use this directory as the extension directory for all subsequent SQL
script loading and running actions

--
Craig Ringer
EnterpriseDB






^ permalink  raw  reply  [nested|flat] 12+ messages in thread

* Re: RFC: Additional Directory for Extensions
  2024-06-25 14:43 Re: RFC: Additional Directory for Extensions Robert Haas <[email protected]>
  2024-06-25 22:31 ` Re: RFC: Additional Directory for Extensions David E. Wheeler <[email protected]>
  2024-07-08 16:01   ` Re: RFC: Additional Directory for Extensions David E. Wheeler <[email protected]>
  2024-08-21 19:59     ` Re: RFC: Additional Directory for Extensions Gabriele Bartolini <[email protected]>
  2024-08-21 23:07       ` Re: RFC: Additional Directory for Extensions Craig Ringer <[email protected]>
@ 2024-08-22 07:31         ` Jelte Fennema-Nio <[email protected]>
  2024-08-22 08:59           ` Re: RFC: Additional Directory for Extensions Gabriele Bartolini <[email protected]>
  1 sibling, 1 reply; 12+ messages in thread

From: Jelte Fennema-Nio @ 2024-08-22 07:31 UTC (permalink / raw)
  To: Craig Ringer <[email protected]>; +Cc: Gabriele Bartolini <[email protected]>; David E. Wheeler <[email protected]>; Robert Haas <[email protected]>; Alvaro Herrera <[email protected]>; pgsql-hackers

On Thu, 22 Aug 2024 at 01:08, Craig Ringer
<[email protected]> wrote:
>     SET extension_search_path = $extsdir,
> /mnt/extensions/pg16/postgis-vX.Y/extensions,
> /mnt/extensions/pg16/gosuperfast/extensions;

It looks like you want one directory per extension, so that list would
get pretty long if you have multiple extensions. Maybe (as a follow up
change), we should start to support a * as a wildcard in both of these
GUCs. So you could say:

SET extension_search_path = /mnt/extensions/pg16/*

To mean effectively the same as you're describing above.






^ permalink  raw  reply  [nested|flat] 12+ messages in thread

* Re: RFC: Additional Directory for Extensions
  2024-06-25 14:43 Re: RFC: Additional Directory for Extensions Robert Haas <[email protected]>
  2024-06-25 22:31 ` Re: RFC: Additional Directory for Extensions David E. Wheeler <[email protected]>
  2024-07-08 16:01   ` Re: RFC: Additional Directory for Extensions David E. Wheeler <[email protected]>
  2024-08-21 19:59     ` Re: RFC: Additional Directory for Extensions Gabriele Bartolini <[email protected]>
  2024-08-21 23:07       ` Re: RFC: Additional Directory for Extensions Craig Ringer <[email protected]>
  2024-08-22 07:31         ` Re: RFC: Additional Directory for Extensions Jelte Fennema-Nio <[email protected]>
@ 2024-08-22 08:59           ` Gabriele Bartolini <[email protected]>
  2024-08-22 22:14             ` Re: RFC: Additional Directory for Extensions Craig Ringer <[email protected]>
  0 siblings, 1 reply; 12+ messages in thread

From: Gabriele Bartolini @ 2024-08-22 08:59 UTC (permalink / raw)
  To: Jelte Fennema-Nio <[email protected]>; +Cc: Craig Ringer <[email protected]>; David E. Wheeler <[email protected]>; Robert Haas <[email protected]>; Alvaro Herrera <[email protected]>; pgsql-hackers

Hi Jelte,

On Thu, 22 Aug 2024 at 09:32, Jelte Fennema-Nio <[email protected]> wrote:

> It looks like you want one directory per extension, so that list would
> get pretty long if you have multiple extensions. Maybe (as a follow up
> change), we should start to support a * as a wildcard in both of these
> GUCs. So you could say:
>
> SET extension_search_path = /mnt/extensions/pg16/*
>
> To mean effectively the same as you're describing above.
>

That'd be great. +1.

-- 
Gabriele Bartolini
Vice President, Cloud Native at EDB
enterprisedb.com


^ permalink  raw  reply  [nested|flat] 12+ messages in thread

* Re: RFC: Additional Directory for Extensions
  2024-06-25 14:43 Re: RFC: Additional Directory for Extensions Robert Haas <[email protected]>
  2024-06-25 22:31 ` Re: RFC: Additional Directory for Extensions David E. Wheeler <[email protected]>
  2024-07-08 16:01   ` Re: RFC: Additional Directory for Extensions David E. Wheeler <[email protected]>
  2024-08-21 19:59     ` Re: RFC: Additional Directory for Extensions Gabriele Bartolini <[email protected]>
  2024-08-21 23:07       ` Re: RFC: Additional Directory for Extensions Craig Ringer <[email protected]>
  2024-08-22 07:31         ` Re: RFC: Additional Directory for Extensions Jelte Fennema-Nio <[email protected]>
  2024-08-22 08:59           ` Re: RFC: Additional Directory for Extensions Gabriele Bartolini <[email protected]>
@ 2024-08-22 22:14             ` Craig Ringer <[email protected]>
  2024-08-22 22:16               ` Re: RFC: Additional Directory for Extensions Craig Ringer <[email protected]>
  0 siblings, 1 reply; 12+ messages in thread

From: Craig Ringer @ 2024-08-22 22:14 UTC (permalink / raw)
  To: Gabriele Bartolini <[email protected]>; +Cc: Jelte Fennema-Nio <[email protected]>; David E. Wheeler <[email protected]>; Robert Haas <[email protected]>; Alvaro Herrera <[email protected]>; pgsql-hackers

On Thu, 22 Aug 2024 at 21:00, Gabriele Bartolini
<[email protected]> wrote:
> On Thu, 22 Aug 2024 at 09:32, Jelte Fennema-Nio <[email protected]> wrote:
>> SET extension_search_path = /mnt/extensions/pg16/*
>
> That'd be great. +1.

Agreed, that'd be handy, but not worth blocking the underlying capability for.

Except possibly to the degree that the feature should reserve wildcard
characters and require them to be escaped if they appear on a path, so
there's no BC break if it's added later.

On Thu, 22 Aug 2024 at 21:00, Gabriele Bartolini
<[email protected]> wrote:
>
> Hi Jelte,
>
> On Thu, 22 Aug 2024 at 09:32, Jelte Fennema-Nio <[email protected]> wrote:
>>
>> It looks like you want one directory per extension, so that list would
>> get pretty long if you have multiple extensions. Maybe (as a follow up
>> change), we should start to support a * as a wildcard in both of these
>> GUCs. So you could say:
>>
>> SET extension_search_path = /mnt/extensions/pg16/*
>>
>> To mean effectively the same as you're describing above.
>
>
> That'd be great. +1.
>
> --
> Gabriele Bartolini
> Vice President, Cloud Native at EDB
> enterprisedb.com






^ permalink  raw  reply  [nested|flat] 12+ messages in thread

* Re: RFC: Additional Directory for Extensions
  2024-06-25 14:43 Re: RFC: Additional Directory for Extensions Robert Haas <[email protected]>
  2024-06-25 22:31 ` Re: RFC: Additional Directory for Extensions David E. Wheeler <[email protected]>
  2024-07-08 16:01   ` Re: RFC: Additional Directory for Extensions David E. Wheeler <[email protected]>
  2024-08-21 19:59     ` Re: RFC: Additional Directory for Extensions Gabriele Bartolini <[email protected]>
  2024-08-21 23:07       ` Re: RFC: Additional Directory for Extensions Craig Ringer <[email protected]>
  2024-08-22 07:31         ` Re: RFC: Additional Directory for Extensions Jelte Fennema-Nio <[email protected]>
  2024-08-22 08:59           ` Re: RFC: Additional Directory for Extensions Gabriele Bartolini <[email protected]>
  2024-08-22 22:14             ` Re: RFC: Additional Directory for Extensions Craig Ringer <[email protected]>
@ 2024-08-22 22:16               ` Craig Ringer <[email protected]>
  0 siblings, 0 replies; 12+ messages in thread

From: Craig Ringer @ 2024-08-22 22:16 UTC (permalink / raw)
  To: Gabriele Bartolini <[email protected]>; +Cc: Jelte Fennema-Nio <[email protected]>; David E. Wheeler <[email protected]>; Robert Haas <[email protected]>; Alvaro Herrera <[email protected]>; pgsql-hackers

On Fri, 23 Aug 2024 at 10:14, Craig Ringer
<[email protected]> wrote:
> On Thu, 22 Aug 2024 at 21:00, Gabriele Bartolini
> <[email protected]> wrote:
> > On Thu, 22 Aug 2024 at 09:32, Jelte Fennema-Nio <[email protected]> wrote:
> >> SET extension_search_path = /mnt/extensions/pg16/*
> >
> > That'd be great. +1.
>
> Agreed, that'd be handy, but not worth blocking the underlying capability for.
>
> Except possibly to the degree that the feature should reserve wildcard
> characters and require them to be escaped if they appear on a path, so
> there's no BC break if it's added later.

... though on second thoughts, it might make more sense to just
recursively search directories found under each path entry. Rules like
'search if a redundant trailing / is present' can be an option.

That way there's no icky path escaping needed for normal configuration.






^ permalink  raw  reply  [nested|flat] 12+ messages in thread

* Re: RFC: Additional Directory for Extensions
  2024-06-25 14:43 Re: RFC: Additional Directory for Extensions Robert Haas <[email protected]>
  2024-06-25 22:31 ` Re: RFC: Additional Directory for Extensions David E. Wheeler <[email protected]>
  2024-07-08 16:01   ` Re: RFC: Additional Directory for Extensions David E. Wheeler <[email protected]>
  2024-08-21 19:59     ` Re: RFC: Additional Directory for Extensions Gabriele Bartolini <[email protected]>
  2024-08-21 23:07       ` Re: RFC: Additional Directory for Extensions Craig Ringer <[email protected]>
@ 2024-08-22 08:58         ` Gabriele Bartolini <[email protected]>
  1 sibling, 0 replies; 12+ messages in thread

From: Gabriele Bartolini @ 2024-08-22 08:58 UTC (permalink / raw)
  To: Craig Ringer <[email protected]>; +Cc: David E. Wheeler <[email protected]>; Robert Haas <[email protected]>; Alvaro Herrera <[email protected]>; pgsql-hackers

Hi Craig,

On Thu, 22 Aug 2024 at 01:07, Craig Ringer <[email protected]>
wrote:

> It's also very relevant for local development and testing.
>

Yep, which is the original goal of Christoph IIRC.


> It may be possible to weaken this restriction somewhat thanks to the
> upcoming
> https://kubernetes.io/blog/2024/08/16/kubernetes-1-31-image-volume-source/
> feature that permits additional OCI images to be mounted as read-only
> volumes on a workload. This would still only permit mounting at
> Pod-creation time, not runtime mounting and unmonuting, but means the
> base postgres image could be supplemented by mounting additional
> images for extensions.
>

I'm really excited about that feature, but it's still in the alpha stage.
However, I don't anticipate any issues for the future general availability
(GA) release. Regardless, we may need to consider a temporary solution that
is compatible with existing Kubernetes and possibly Postgres versions (but
that's beyond the purpose of this thread).

For example, one might mount image "postgis-vX.Y.Z" image onto base
> image "postgresql-16" if support for PostGIS is desired, without then
> having to bake every possible extension anyone might ever want into
> the base image. This solves all sorts of messy issues with upgrades
> and new version releases.
>

Yep.


> But for it to work, it must be possible to tell postgres to look in
> _multiple places_ for extension .sql scripts and control files. This
> is presently possible for modules (dynamic libraries, .so / .dylib /
> .dll) but without a way to also configure the path for extensions it's
> of very limited utility.
>

Agree.


> So IMO this should be a _path_ to search for extension control files
> and SQL scripts.
>

I like this. I also prefer the name `extension_search_path`.

For safety, it might make sense to impose the restriction that if an
> extension control file is found in a given directory, SQL scripts will
> also only be looked for in that same directory. That way there's no
> chance of accidentally mixing and matching SQL scripts from different
> versions of an extension if it appears twice on the extension search
> path in different places. The rule for loading SQL scripts would be:
>
> * locate first directory on path contianing matching extension control file
> * use this directory as the extension directory for all subsequent SQL
> script loading and running actions
>

It could work, but it requires some prototyping and exploration. I'm
willing to participate and use CloudNativePG as a test bed.

Cheers,
Gabriele
-- 
Gabriele Bartolini
Vice President, Cloud Native at EDB
enterprisedb.com


^ permalink  raw  reply  [nested|flat] 12+ messages in thread


end of thread, other threads:[~2024-08-22 22:16 UTC | newest]

Thread overview: 12+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2022-01-25 01:40 [PATCH 01/17] doc: than than Justin Pryzby <[email protected]>
2024-06-25 14:43 Re: RFC: Additional Directory for Extensions Robert Haas <[email protected]>
2024-06-25 22:31 ` Re: RFC: Additional Directory for Extensions David E. Wheeler <[email protected]>
2024-06-26 09:30   ` Re: RFC: Additional Directory for Extensions Jelte Fennema-Nio <[email protected]>
2024-07-08 16:01   ` Re: RFC: Additional Directory for Extensions David E. Wheeler <[email protected]>
2024-08-21 19:59     ` Re: RFC: Additional Directory for Extensions Gabriele Bartolini <[email protected]>
2024-08-21 23:07       ` Re: RFC: Additional Directory for Extensions Craig Ringer <[email protected]>
2024-08-22 07:31         ` Re: RFC: Additional Directory for Extensions Jelte Fennema-Nio <[email protected]>
2024-08-22 08:59           ` Re: RFC: Additional Directory for Extensions Gabriele Bartolini <[email protected]>
2024-08-22 22:14             ` Re: RFC: Additional Directory for Extensions Craig Ringer <[email protected]>
2024-08-22 22:16               ` Re: RFC: Additional Directory for Extensions Craig Ringer <[email protected]>
2024-08-22 08:58         ` Re: RFC: Additional Directory for Extensions Gabriele Bartolini <[email protected]>

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox