public inbox for [email protected]
help / color / mirror / Atom feed[PATCH v20 4/5] restructure archive modules docs in preparation for restore modules
24+ messages / 7 participants
[nested] [flat]
* [PATCH v20 4/5] restructure archive modules docs in preparation for restore modules
@ 2023-02-15 22:48 Nathan Bossart <[email protected]>
0 siblings, 0 replies; 24+ messages in thread
From: Nathan Bossart @ 2023-02-15 22:48 UTC (permalink / raw)
---
doc/src/sgml/archive-and-restore-modules.sgml | 226 ++++++++++--------
1 file changed, 120 insertions(+), 106 deletions(-)
diff --git a/doc/src/sgml/archive-and-restore-modules.sgml b/doc/src/sgml/archive-and-restore-modules.sgml
index cf7438a759..a52d15082d 100644
--- a/doc/src/sgml/archive-and-restore-modules.sgml
+++ b/doc/src/sgml/archive-and-restore-modules.sgml
@@ -1,9 +1,9 @@
-<!-- doc/src/sgml/archive-modules.sgml -->
+<!-- doc/src/sgml/archive-and-restore-modules.sgml -->
-<chapter id="archive-modules">
- <title>Archive Modules</title>
- <indexterm zone="archive-modules">
- <primary>Archive Modules</primary>
+<chapter id="archive-and-restore-modules">
+ <title>Archive and Restore Modules</title>
+ <indexterm zone="archive-and-restore-modules">
+ <primary>Archive and Restore Modules</primary>
</indexterm>
<para>
@@ -14,45 +14,52 @@
performant.
</para>
- <para>
- When a custom <xref linkend="guc-archive-library"/> is configured, PostgreSQL
- will submit completed WAL files to the module, and the server will avoid
- recycling or removing these WAL files until the module indicates that the files
- were successfully archived. It is ultimately up to the module to decide what
- to do with each WAL file, but many recommendations are listed at
- <xref linkend="backup-archiving-wal"/>.
- </para>
-
- <para>
- Archiving modules must at least consist of an initialization function (see
- <xref linkend="archive-module-init"/>) and the required callbacks (see
- <xref linkend="archive-module-callbacks"/>). However, archive modules are
- also permitted to do much more (e.g., declare GUCs and register background
- workers).
- </para>
-
<para>
The <filename>contrib/basic_archive</filename> module contains a working
example, which demonstrates some useful techniques.
</para>
- <sect1 id="archive-module-init">
- <title>Initialization Functions</title>
- <indexterm zone="archive-module-init">
- <primary>_PG_archive_module_init</primary>
+ <sect1 id="archive-modules">
+ <title>Archive Modules</title>
+ <indexterm zone="archive-modules">
+ <primary>Archive Modules</primary>
</indexterm>
+
+ <para>
+ When a custom <xref linkend="guc-archive-library"/> is configured,
+ PostgreSQL will submit completed WAL files to the module, and the server
+ will avoid recycling or removing these WAL files until the module indicates
+ that the files were successfully archived. It is ultimately up to the
+ module to decide what to do with each WAL file, but many recommendations are
+ listed at <xref linkend="backup-archiving-wal"/>.
+ </para>
+
<para>
- An archive library is loaded by dynamically loading a shared library with the
- <xref linkend="guc-archive-library"/>'s name as the library base name. The
- normal library search path is used to locate the library. To provide the
- required archive module callbacks and to indicate that the library is
- actually an archive module, it needs to provide a function named
- <function>_PG_archive_module_init</function>. The result of the function
- must be a pointer to a struct of type
- <structname>ArchiveModuleCallbacks</structname>, which contains everything
- that the core code needs to know to make use of the archive module. The
- return value needs to be of server lifetime, which is typically achieved by
- defining it as a <literal>static const</literal> variable in global scope.
+ Archiving modules must at least consist of an initialization function (see
+ <xref linkend="archive-module-init"/>) and the required callbacks (see
+ <xref linkend="archive-module-callbacks"/>). However, archive modules are
+ also permitted to do much more (e.g., declare GUCs and register background
+ workers).
+ </para>
+
+ <sect2 id="archive-module-init">
+ <title>Initialization Functions</title>
+ <indexterm zone="archive-module-init">
+ <primary>_PG_archive_module_init</primary>
+ </indexterm>
+
+ <para>
+ An archive library is loaded by dynamically loading a shared library with
+ the <xref linkend="guc-archive-library"/>'s name as the library base name.
+ The normal library search path is used to locate the library. To provide
+ the required archive module callbacks and to indicate that the library is
+ actually an archive module, it needs to provide a function named
+ <function>_PG_archive_module_init</function>. The result of the function
+ must be a pointer to a struct of type
+ <structname>ArchiveModuleCallbacks</structname>, which contains everything
+ that the core code needs to know to make use of the archive module. The
+ return value needs to be of server lifetime, which is typically achieved by
+ defining it as a <literal>static const</literal> variable in global scope.
<programlisting>
typedef struct ArchiveModuleCallbacks
@@ -65,103 +72,110 @@ typedef struct ArchiveModuleCallbacks
typedef const ArchiveModuleCallbacks *(*ArchiveModuleInit) (void);
</programlisting>
- Only the <function>archive_file_cb</function> callback is required. The
- others are optional.
- </para>
- </sect1>
+ Only the <function>archive_file_cb</function> callback is required. The
+ others are optional.
+ </para>
+ </sect2>
- <sect1 id="archive-module-callbacks">
- <title>Archive Module Callbacks</title>
- <para>
- The archive callbacks define the actual archiving behavior of the module.
- The server will call them as required to process each individual WAL file.
- </para>
+ <sect2 id="archive-module-callbacks">
+ <title>Archive Module Callbacks</title>
- <sect2 id="archive-module-startup">
- <title>Startup Callback</title>
<para>
- The <function>startup_cb</function> callback is called shortly after the
- module is loaded. This callback can be used to perform any additional
- initialization required. If the archive module has any state, it can use
- <structfield>state->private_data</structfield> to store it.
+ The archive callbacks define the actual archiving behavior of the module.
+ The server will call them as required to process each individual WAL file.
+ </para>
+
+ <sect3 id="archive-module-startup">
+ <title>Startup Callback</title>
+
+ <para>
+ The <function>startup_cb</function> callback is called shortly after the
+ module is loaded. This callback can be used to perform any additional
+ initialization required. If the archive module has any state, it can use
+ <structfield>state->private_data</structfield> to store it.
<programlisting>
typedef void (*ArchiveStartupCB) (ArchiveModuleState *state);
</programlisting>
- </para>
- </sect2>
+ </para>
+ </sect3>
- <sect2 id="archive-module-check">
- <title>Check Callback</title>
- <para>
- The <function>check_configured_cb</function> callback is called to determine
- whether the module is fully configured and ready to accept WAL files (e.g.,
- its configuration parameters are set to valid values). If no
- <function>check_configured_cb</function> is defined, the server always
- assumes the module is configured.
+ <sect3 id="archive-module-check">
+ <title>Check Callback</title>
+
+ <para>
+ The <function>check_configured_cb</function> callback is called to
+ determine whether the module is fully configured and ready to accept WAL
+ files (e.g., its configuration parameters are set to valid values). If no
+ <function>check_configured_cb</function> is defined, the server always
+ assumes the module is configured.
<programlisting>
typedef bool (*ArchiveCheckConfiguredCB) (ArchiveModuleState *state);
</programlisting>
- If <literal>true</literal> is returned, the server will proceed with
- archiving the file by calling the <function>archive_file_cb</function>
- callback. If <literal>false</literal> is returned, archiving will not
- proceed, and the archiver will emit the following message to the server log:
+ If <literal>true</literal> is returned, the server will proceed with
+ archiving the file by calling the <function>archive_file_cb</function>
+ callback. If <literal>false</literal> is returned, archiving will not
+ proceed, and the archiver will emit the following message to the server
+ log:
<screen>
WARNING: archive_mode enabled, yet archiving is not configured
</screen>
- In the latter case, the server will periodically call this function, and
- archiving will proceed only when it returns <literal>true</literal>.
- </para>
-
- <note>
- <para>
- When returning <literal>false</literal>, it may be useful to append some
- additional information to the generic warning message. To do that, provide
- a message to the <function>arch_module_check_errdetail</function> macro
- before returning <literal>false</literal>. Like
- <function>errdetail()</function>, this macro accepts a format string
- followed by an optional list of arguments. The resulting string will be
- emitted as the <literal>DETAIL</literal> line of the warning message.
+ In the latter case, the server will periodically call this function, and
+ archiving will proceed only when it returns <literal>true</literal>.
</para>
- </note>
- </sect2>
- <sect2 id="archive-module-archive">
- <title>Archive Callback</title>
- <para>
- The <function>archive_file_cb</function> callback is called to archive a
- single WAL file.
+ <note>
+ <para>
+ When returning <literal>false</literal>, it may be useful to append some
+ additional information to the generic warning message. To do that,
+ provide a message to the <function>arch_module_check_errdetail</function>
+ macro before returning <literal>false</literal>. Like
+ <function>errdetail()</function>, this macro accepts a format string
+ followed by an optional list of arguments. The resulting string will be
+ emitted as the <literal>DETAIL</literal> line of the warning message.
+ </para>
+ </note>
+ </sect3>
+
+ <sect3 id="archive-module-archive">
+ <title>Archive Callback</title>
+
+ <para>
+ The <function>archive_file_cb</function> callback is called to archive a
+ single WAL file.
<programlisting>
typedef bool (*ArchiveFileCB) (ArchiveModuleState *state, const char *file, const char *path);
</programlisting>
- If <literal>true</literal> is returned, the server proceeds as if the file
- was successfully archived, which may include recycling or removing the
- original WAL file. If <literal>false</literal> is returned, the server will
- keep the original WAL file and retry archiving later.
- <replaceable>file</replaceable> will contain just the file name of the WAL
- file to archive, while <replaceable>path</replaceable> contains the full
- path of the WAL file (including the file name).
- </para>
- </sect2>
+ If <literal>true</literal> is returned, the server proceeds as if the file
+ was successfully archived, which may include recycling or removing the
+ original WAL file. If <literal>false</literal> is returned, the server
+ will keep the original WAL file and retry archiving later.
+ <replaceable>file</replaceable> will contain just the file name of the WAL
+ file to archive, while <replaceable>path</replaceable> contains the full
+ path of the WAL file (including the file name).
+ </para>
+ </sect3>
- <sect2 id="archive-module-shutdown">
- <title>Shutdown Callback</title>
- <para>
- The <function>shutdown_cb</function> callback is called when the archiver
- process exits (e.g., after an error) or the value of
- <xref linkend="guc-archive-library"/> changes. If no
- <function>shutdown_cb</function> is defined, no special action is taken in
- these situations. If the archive module has any state, this callback should
- free it to avoid leaks.
+ <sect3 id="archive-module-shutdown">
+ <title>Shutdown Callback</title>
+
+ <para>
+ The <function>shutdown_cb</function> callback is called when the archiver
+ process exits (e.g., after an error) or the value of
+ <xref linkend="guc-archive-library"/> changes. If no
+ <function>shutdown_cb</function> is defined, no special action is taken in
+ these situations. If the archive module has any state, this callback
+ should free it to avoid leaks.
<programlisting>
typedef void (*ArchiveShutdownCB) (ArchiveModuleState *state);
</programlisting>
- </para>
+ </para>
+ </sect3>
</sect2>
</sect1>
</chapter>
--
2.25.1
--3V7upXqbjpZ4EhLz
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v20-0005-introduce-restore_library.patch"
^ permalink raw reply [nested|flat] 24+ messages in thread
* [PATCH v18 4/5] restructure archive modules docs in preparation for restore modules
@ 2023-02-15 22:48 Nathan Bossart <[email protected]>
0 siblings, 0 replies; 24+ messages in thread
From: Nathan Bossart @ 2023-02-15 22:48 UTC (permalink / raw)
---
doc/src/sgml/archive-and-restore-modules.sgml | 226 ++++++++++--------
1 file changed, 120 insertions(+), 106 deletions(-)
diff --git a/doc/src/sgml/archive-and-restore-modules.sgml b/doc/src/sgml/archive-and-restore-modules.sgml
index cf7438a759..a52d15082d 100644
--- a/doc/src/sgml/archive-and-restore-modules.sgml
+++ b/doc/src/sgml/archive-and-restore-modules.sgml
@@ -1,9 +1,9 @@
-<!-- doc/src/sgml/archive-modules.sgml -->
+<!-- doc/src/sgml/archive-and-restore-modules.sgml -->
-<chapter id="archive-modules">
- <title>Archive Modules</title>
- <indexterm zone="archive-modules">
- <primary>Archive Modules</primary>
+<chapter id="archive-and-restore-modules">
+ <title>Archive and Restore Modules</title>
+ <indexterm zone="archive-and-restore-modules">
+ <primary>Archive and Restore Modules</primary>
</indexterm>
<para>
@@ -14,45 +14,52 @@
performant.
</para>
- <para>
- When a custom <xref linkend="guc-archive-library"/> is configured, PostgreSQL
- will submit completed WAL files to the module, and the server will avoid
- recycling or removing these WAL files until the module indicates that the files
- were successfully archived. It is ultimately up to the module to decide what
- to do with each WAL file, but many recommendations are listed at
- <xref linkend="backup-archiving-wal"/>.
- </para>
-
- <para>
- Archiving modules must at least consist of an initialization function (see
- <xref linkend="archive-module-init"/>) and the required callbacks (see
- <xref linkend="archive-module-callbacks"/>). However, archive modules are
- also permitted to do much more (e.g., declare GUCs and register background
- workers).
- </para>
-
<para>
The <filename>contrib/basic_archive</filename> module contains a working
example, which demonstrates some useful techniques.
</para>
- <sect1 id="archive-module-init">
- <title>Initialization Functions</title>
- <indexterm zone="archive-module-init">
- <primary>_PG_archive_module_init</primary>
+ <sect1 id="archive-modules">
+ <title>Archive Modules</title>
+ <indexterm zone="archive-modules">
+ <primary>Archive Modules</primary>
</indexterm>
+
+ <para>
+ When a custom <xref linkend="guc-archive-library"/> is configured,
+ PostgreSQL will submit completed WAL files to the module, and the server
+ will avoid recycling or removing these WAL files until the module indicates
+ that the files were successfully archived. It is ultimately up to the
+ module to decide what to do with each WAL file, but many recommendations are
+ listed at <xref linkend="backup-archiving-wal"/>.
+ </para>
+
<para>
- An archive library is loaded by dynamically loading a shared library with the
- <xref linkend="guc-archive-library"/>'s name as the library base name. The
- normal library search path is used to locate the library. To provide the
- required archive module callbacks and to indicate that the library is
- actually an archive module, it needs to provide a function named
- <function>_PG_archive_module_init</function>. The result of the function
- must be a pointer to a struct of type
- <structname>ArchiveModuleCallbacks</structname>, which contains everything
- that the core code needs to know to make use of the archive module. The
- return value needs to be of server lifetime, which is typically achieved by
- defining it as a <literal>static const</literal> variable in global scope.
+ Archiving modules must at least consist of an initialization function (see
+ <xref linkend="archive-module-init"/>) and the required callbacks (see
+ <xref linkend="archive-module-callbacks"/>). However, archive modules are
+ also permitted to do much more (e.g., declare GUCs and register background
+ workers).
+ </para>
+
+ <sect2 id="archive-module-init">
+ <title>Initialization Functions</title>
+ <indexterm zone="archive-module-init">
+ <primary>_PG_archive_module_init</primary>
+ </indexterm>
+
+ <para>
+ An archive library is loaded by dynamically loading a shared library with
+ the <xref linkend="guc-archive-library"/>'s name as the library base name.
+ The normal library search path is used to locate the library. To provide
+ the required archive module callbacks and to indicate that the library is
+ actually an archive module, it needs to provide a function named
+ <function>_PG_archive_module_init</function>. The result of the function
+ must be a pointer to a struct of type
+ <structname>ArchiveModuleCallbacks</structname>, which contains everything
+ that the core code needs to know to make use of the archive module. The
+ return value needs to be of server lifetime, which is typically achieved by
+ defining it as a <literal>static const</literal> variable in global scope.
<programlisting>
typedef struct ArchiveModuleCallbacks
@@ -65,103 +72,110 @@ typedef struct ArchiveModuleCallbacks
typedef const ArchiveModuleCallbacks *(*ArchiveModuleInit) (void);
</programlisting>
- Only the <function>archive_file_cb</function> callback is required. The
- others are optional.
- </para>
- </sect1>
+ Only the <function>archive_file_cb</function> callback is required. The
+ others are optional.
+ </para>
+ </sect2>
- <sect1 id="archive-module-callbacks">
- <title>Archive Module Callbacks</title>
- <para>
- The archive callbacks define the actual archiving behavior of the module.
- The server will call them as required to process each individual WAL file.
- </para>
+ <sect2 id="archive-module-callbacks">
+ <title>Archive Module Callbacks</title>
- <sect2 id="archive-module-startup">
- <title>Startup Callback</title>
<para>
- The <function>startup_cb</function> callback is called shortly after the
- module is loaded. This callback can be used to perform any additional
- initialization required. If the archive module has any state, it can use
- <structfield>state->private_data</structfield> to store it.
+ The archive callbacks define the actual archiving behavior of the module.
+ The server will call them as required to process each individual WAL file.
+ </para>
+
+ <sect3 id="archive-module-startup">
+ <title>Startup Callback</title>
+
+ <para>
+ The <function>startup_cb</function> callback is called shortly after the
+ module is loaded. This callback can be used to perform any additional
+ initialization required. If the archive module has any state, it can use
+ <structfield>state->private_data</structfield> to store it.
<programlisting>
typedef void (*ArchiveStartupCB) (ArchiveModuleState *state);
</programlisting>
- </para>
- </sect2>
+ </para>
+ </sect3>
- <sect2 id="archive-module-check">
- <title>Check Callback</title>
- <para>
- The <function>check_configured_cb</function> callback is called to determine
- whether the module is fully configured and ready to accept WAL files (e.g.,
- its configuration parameters are set to valid values). If no
- <function>check_configured_cb</function> is defined, the server always
- assumes the module is configured.
+ <sect3 id="archive-module-check">
+ <title>Check Callback</title>
+
+ <para>
+ The <function>check_configured_cb</function> callback is called to
+ determine whether the module is fully configured and ready to accept WAL
+ files (e.g., its configuration parameters are set to valid values). If no
+ <function>check_configured_cb</function> is defined, the server always
+ assumes the module is configured.
<programlisting>
typedef bool (*ArchiveCheckConfiguredCB) (ArchiveModuleState *state);
</programlisting>
- If <literal>true</literal> is returned, the server will proceed with
- archiving the file by calling the <function>archive_file_cb</function>
- callback. If <literal>false</literal> is returned, archiving will not
- proceed, and the archiver will emit the following message to the server log:
+ If <literal>true</literal> is returned, the server will proceed with
+ archiving the file by calling the <function>archive_file_cb</function>
+ callback. If <literal>false</literal> is returned, archiving will not
+ proceed, and the archiver will emit the following message to the server
+ log:
<screen>
WARNING: archive_mode enabled, yet archiving is not configured
</screen>
- In the latter case, the server will periodically call this function, and
- archiving will proceed only when it returns <literal>true</literal>.
- </para>
-
- <note>
- <para>
- When returning <literal>false</literal>, it may be useful to append some
- additional information to the generic warning message. To do that, provide
- a message to the <function>arch_module_check_errdetail</function> macro
- before returning <literal>false</literal>. Like
- <function>errdetail()</function>, this macro accepts a format string
- followed by an optional list of arguments. The resulting string will be
- emitted as the <literal>DETAIL</literal> line of the warning message.
+ In the latter case, the server will periodically call this function, and
+ archiving will proceed only when it returns <literal>true</literal>.
</para>
- </note>
- </sect2>
- <sect2 id="archive-module-archive">
- <title>Archive Callback</title>
- <para>
- The <function>archive_file_cb</function> callback is called to archive a
- single WAL file.
+ <note>
+ <para>
+ When returning <literal>false</literal>, it may be useful to append some
+ additional information to the generic warning message. To do that,
+ provide a message to the <function>arch_module_check_errdetail</function>
+ macro before returning <literal>false</literal>. Like
+ <function>errdetail()</function>, this macro accepts a format string
+ followed by an optional list of arguments. The resulting string will be
+ emitted as the <literal>DETAIL</literal> line of the warning message.
+ </para>
+ </note>
+ </sect3>
+
+ <sect3 id="archive-module-archive">
+ <title>Archive Callback</title>
+
+ <para>
+ The <function>archive_file_cb</function> callback is called to archive a
+ single WAL file.
<programlisting>
typedef bool (*ArchiveFileCB) (ArchiveModuleState *state, const char *file, const char *path);
</programlisting>
- If <literal>true</literal> is returned, the server proceeds as if the file
- was successfully archived, which may include recycling or removing the
- original WAL file. If <literal>false</literal> is returned, the server will
- keep the original WAL file and retry archiving later.
- <replaceable>file</replaceable> will contain just the file name of the WAL
- file to archive, while <replaceable>path</replaceable> contains the full
- path of the WAL file (including the file name).
- </para>
- </sect2>
+ If <literal>true</literal> is returned, the server proceeds as if the file
+ was successfully archived, which may include recycling or removing the
+ original WAL file. If <literal>false</literal> is returned, the server
+ will keep the original WAL file and retry archiving later.
+ <replaceable>file</replaceable> will contain just the file name of the WAL
+ file to archive, while <replaceable>path</replaceable> contains the full
+ path of the WAL file (including the file name).
+ </para>
+ </sect3>
- <sect2 id="archive-module-shutdown">
- <title>Shutdown Callback</title>
- <para>
- The <function>shutdown_cb</function> callback is called when the archiver
- process exits (e.g., after an error) or the value of
- <xref linkend="guc-archive-library"/> changes. If no
- <function>shutdown_cb</function> is defined, no special action is taken in
- these situations. If the archive module has any state, this callback should
- free it to avoid leaks.
+ <sect3 id="archive-module-shutdown">
+ <title>Shutdown Callback</title>
+
+ <para>
+ The <function>shutdown_cb</function> callback is called when the archiver
+ process exits (e.g., after an error) or the value of
+ <xref linkend="guc-archive-library"/> changes. If no
+ <function>shutdown_cb</function> is defined, no special action is taken in
+ these situations. If the archive module has any state, this callback
+ should free it to avoid leaks.
<programlisting>
typedef void (*ArchiveShutdownCB) (ArchiveModuleState *state);
</programlisting>
- </para>
+ </para>
+ </sect3>
</sect2>
</sect1>
</chapter>
--
2.25.1
--IJpNTDwzlM2Ie8A6
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v18-0005-introduce-restore_library.patch"
^ permalink raw reply [nested|flat] 24+ messages in thread
* [PATCH v19 4/5] restructure archive modules docs in preparation for restore modules
@ 2023-02-15 22:48 Nathan Bossart <[email protected]>
0 siblings, 0 replies; 24+ messages in thread
From: Nathan Bossart @ 2023-02-15 22:48 UTC (permalink / raw)
---
doc/src/sgml/archive-and-restore-modules.sgml | 226 ++++++++++--------
1 file changed, 120 insertions(+), 106 deletions(-)
diff --git a/doc/src/sgml/archive-and-restore-modules.sgml b/doc/src/sgml/archive-and-restore-modules.sgml
index cf7438a759..a52d15082d 100644
--- a/doc/src/sgml/archive-and-restore-modules.sgml
+++ b/doc/src/sgml/archive-and-restore-modules.sgml
@@ -1,9 +1,9 @@
-<!-- doc/src/sgml/archive-modules.sgml -->
+<!-- doc/src/sgml/archive-and-restore-modules.sgml -->
-<chapter id="archive-modules">
- <title>Archive Modules</title>
- <indexterm zone="archive-modules">
- <primary>Archive Modules</primary>
+<chapter id="archive-and-restore-modules">
+ <title>Archive and Restore Modules</title>
+ <indexterm zone="archive-and-restore-modules">
+ <primary>Archive and Restore Modules</primary>
</indexterm>
<para>
@@ -14,45 +14,52 @@
performant.
</para>
- <para>
- When a custom <xref linkend="guc-archive-library"/> is configured, PostgreSQL
- will submit completed WAL files to the module, and the server will avoid
- recycling or removing these WAL files until the module indicates that the files
- were successfully archived. It is ultimately up to the module to decide what
- to do with each WAL file, but many recommendations are listed at
- <xref linkend="backup-archiving-wal"/>.
- </para>
-
- <para>
- Archiving modules must at least consist of an initialization function (see
- <xref linkend="archive-module-init"/>) and the required callbacks (see
- <xref linkend="archive-module-callbacks"/>). However, archive modules are
- also permitted to do much more (e.g., declare GUCs and register background
- workers).
- </para>
-
<para>
The <filename>contrib/basic_archive</filename> module contains a working
example, which demonstrates some useful techniques.
</para>
- <sect1 id="archive-module-init">
- <title>Initialization Functions</title>
- <indexterm zone="archive-module-init">
- <primary>_PG_archive_module_init</primary>
+ <sect1 id="archive-modules">
+ <title>Archive Modules</title>
+ <indexterm zone="archive-modules">
+ <primary>Archive Modules</primary>
</indexterm>
+
+ <para>
+ When a custom <xref linkend="guc-archive-library"/> is configured,
+ PostgreSQL will submit completed WAL files to the module, and the server
+ will avoid recycling or removing these WAL files until the module indicates
+ that the files were successfully archived. It is ultimately up to the
+ module to decide what to do with each WAL file, but many recommendations are
+ listed at <xref linkend="backup-archiving-wal"/>.
+ </para>
+
<para>
- An archive library is loaded by dynamically loading a shared library with the
- <xref linkend="guc-archive-library"/>'s name as the library base name. The
- normal library search path is used to locate the library. To provide the
- required archive module callbacks and to indicate that the library is
- actually an archive module, it needs to provide a function named
- <function>_PG_archive_module_init</function>. The result of the function
- must be a pointer to a struct of type
- <structname>ArchiveModuleCallbacks</structname>, which contains everything
- that the core code needs to know to make use of the archive module. The
- return value needs to be of server lifetime, which is typically achieved by
- defining it as a <literal>static const</literal> variable in global scope.
+ Archiving modules must at least consist of an initialization function (see
+ <xref linkend="archive-module-init"/>) and the required callbacks (see
+ <xref linkend="archive-module-callbacks"/>). However, archive modules are
+ also permitted to do much more (e.g., declare GUCs and register background
+ workers).
+ </para>
+
+ <sect2 id="archive-module-init">
+ <title>Initialization Functions</title>
+ <indexterm zone="archive-module-init">
+ <primary>_PG_archive_module_init</primary>
+ </indexterm>
+
+ <para>
+ An archive library is loaded by dynamically loading a shared library with
+ the <xref linkend="guc-archive-library"/>'s name as the library base name.
+ The normal library search path is used to locate the library. To provide
+ the required archive module callbacks and to indicate that the library is
+ actually an archive module, it needs to provide a function named
+ <function>_PG_archive_module_init</function>. The result of the function
+ must be a pointer to a struct of type
+ <structname>ArchiveModuleCallbacks</structname>, which contains everything
+ that the core code needs to know to make use of the archive module. The
+ return value needs to be of server lifetime, which is typically achieved by
+ defining it as a <literal>static const</literal> variable in global scope.
<programlisting>
typedef struct ArchiveModuleCallbacks
@@ -65,103 +72,110 @@ typedef struct ArchiveModuleCallbacks
typedef const ArchiveModuleCallbacks *(*ArchiveModuleInit) (void);
</programlisting>
- Only the <function>archive_file_cb</function> callback is required. The
- others are optional.
- </para>
- </sect1>
+ Only the <function>archive_file_cb</function> callback is required. The
+ others are optional.
+ </para>
+ </sect2>
- <sect1 id="archive-module-callbacks">
- <title>Archive Module Callbacks</title>
- <para>
- The archive callbacks define the actual archiving behavior of the module.
- The server will call them as required to process each individual WAL file.
- </para>
+ <sect2 id="archive-module-callbacks">
+ <title>Archive Module Callbacks</title>
- <sect2 id="archive-module-startup">
- <title>Startup Callback</title>
<para>
- The <function>startup_cb</function> callback is called shortly after the
- module is loaded. This callback can be used to perform any additional
- initialization required. If the archive module has any state, it can use
- <structfield>state->private_data</structfield> to store it.
+ The archive callbacks define the actual archiving behavior of the module.
+ The server will call them as required to process each individual WAL file.
+ </para>
+
+ <sect3 id="archive-module-startup">
+ <title>Startup Callback</title>
+
+ <para>
+ The <function>startup_cb</function> callback is called shortly after the
+ module is loaded. This callback can be used to perform any additional
+ initialization required. If the archive module has any state, it can use
+ <structfield>state->private_data</structfield> to store it.
<programlisting>
typedef void (*ArchiveStartupCB) (ArchiveModuleState *state);
</programlisting>
- </para>
- </sect2>
+ </para>
+ </sect3>
- <sect2 id="archive-module-check">
- <title>Check Callback</title>
- <para>
- The <function>check_configured_cb</function> callback is called to determine
- whether the module is fully configured and ready to accept WAL files (e.g.,
- its configuration parameters are set to valid values). If no
- <function>check_configured_cb</function> is defined, the server always
- assumes the module is configured.
+ <sect3 id="archive-module-check">
+ <title>Check Callback</title>
+
+ <para>
+ The <function>check_configured_cb</function> callback is called to
+ determine whether the module is fully configured and ready to accept WAL
+ files (e.g., its configuration parameters are set to valid values). If no
+ <function>check_configured_cb</function> is defined, the server always
+ assumes the module is configured.
<programlisting>
typedef bool (*ArchiveCheckConfiguredCB) (ArchiveModuleState *state);
</programlisting>
- If <literal>true</literal> is returned, the server will proceed with
- archiving the file by calling the <function>archive_file_cb</function>
- callback. If <literal>false</literal> is returned, archiving will not
- proceed, and the archiver will emit the following message to the server log:
+ If <literal>true</literal> is returned, the server will proceed with
+ archiving the file by calling the <function>archive_file_cb</function>
+ callback. If <literal>false</literal> is returned, archiving will not
+ proceed, and the archiver will emit the following message to the server
+ log:
<screen>
WARNING: archive_mode enabled, yet archiving is not configured
</screen>
- In the latter case, the server will periodically call this function, and
- archiving will proceed only when it returns <literal>true</literal>.
- </para>
-
- <note>
- <para>
- When returning <literal>false</literal>, it may be useful to append some
- additional information to the generic warning message. To do that, provide
- a message to the <function>arch_module_check_errdetail</function> macro
- before returning <literal>false</literal>. Like
- <function>errdetail()</function>, this macro accepts a format string
- followed by an optional list of arguments. The resulting string will be
- emitted as the <literal>DETAIL</literal> line of the warning message.
+ In the latter case, the server will periodically call this function, and
+ archiving will proceed only when it returns <literal>true</literal>.
</para>
- </note>
- </sect2>
- <sect2 id="archive-module-archive">
- <title>Archive Callback</title>
- <para>
- The <function>archive_file_cb</function> callback is called to archive a
- single WAL file.
+ <note>
+ <para>
+ When returning <literal>false</literal>, it may be useful to append some
+ additional information to the generic warning message. To do that,
+ provide a message to the <function>arch_module_check_errdetail</function>
+ macro before returning <literal>false</literal>. Like
+ <function>errdetail()</function>, this macro accepts a format string
+ followed by an optional list of arguments. The resulting string will be
+ emitted as the <literal>DETAIL</literal> line of the warning message.
+ </para>
+ </note>
+ </sect3>
+
+ <sect3 id="archive-module-archive">
+ <title>Archive Callback</title>
+
+ <para>
+ The <function>archive_file_cb</function> callback is called to archive a
+ single WAL file.
<programlisting>
typedef bool (*ArchiveFileCB) (ArchiveModuleState *state, const char *file, const char *path);
</programlisting>
- If <literal>true</literal> is returned, the server proceeds as if the file
- was successfully archived, which may include recycling or removing the
- original WAL file. If <literal>false</literal> is returned, the server will
- keep the original WAL file and retry archiving later.
- <replaceable>file</replaceable> will contain just the file name of the WAL
- file to archive, while <replaceable>path</replaceable> contains the full
- path of the WAL file (including the file name).
- </para>
- </sect2>
+ If <literal>true</literal> is returned, the server proceeds as if the file
+ was successfully archived, which may include recycling or removing the
+ original WAL file. If <literal>false</literal> is returned, the server
+ will keep the original WAL file and retry archiving later.
+ <replaceable>file</replaceable> will contain just the file name of the WAL
+ file to archive, while <replaceable>path</replaceable> contains the full
+ path of the WAL file (including the file name).
+ </para>
+ </sect3>
- <sect2 id="archive-module-shutdown">
- <title>Shutdown Callback</title>
- <para>
- The <function>shutdown_cb</function> callback is called when the archiver
- process exits (e.g., after an error) or the value of
- <xref linkend="guc-archive-library"/> changes. If no
- <function>shutdown_cb</function> is defined, no special action is taken in
- these situations. If the archive module has any state, this callback should
- free it to avoid leaks.
+ <sect3 id="archive-module-shutdown">
+ <title>Shutdown Callback</title>
+
+ <para>
+ The <function>shutdown_cb</function> callback is called when the archiver
+ process exits (e.g., after an error) or the value of
+ <xref linkend="guc-archive-library"/> changes. If no
+ <function>shutdown_cb</function> is defined, no special action is taken in
+ these situations. If the archive module has any state, this callback
+ should free it to avoid leaks.
<programlisting>
typedef void (*ArchiveShutdownCB) (ArchiveModuleState *state);
</programlisting>
- </para>
+ </para>
+ </sect3>
</sect2>
</sect1>
</chapter>
--
2.25.1
--oyUTqETQ0mS9luUI
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v19-0005-introduce-restore_library.patch"
^ permalink raw reply [nested|flat] 24+ messages in thread
* [PATCH v20 4/5] restructure archive modules docs in preparation for restore modules
@ 2023-02-15 22:48 Nathan Bossart <[email protected]>
0 siblings, 0 replies; 24+ messages in thread
From: Nathan Bossart @ 2023-02-15 22:48 UTC (permalink / raw)
---
doc/src/sgml/archive-and-restore-modules.sgml | 226 ++++++++++--------
1 file changed, 120 insertions(+), 106 deletions(-)
diff --git a/doc/src/sgml/archive-and-restore-modules.sgml b/doc/src/sgml/archive-and-restore-modules.sgml
index cf7438a759..a52d15082d 100644
--- a/doc/src/sgml/archive-and-restore-modules.sgml
+++ b/doc/src/sgml/archive-and-restore-modules.sgml
@@ -1,9 +1,9 @@
-<!-- doc/src/sgml/archive-modules.sgml -->
+<!-- doc/src/sgml/archive-and-restore-modules.sgml -->
-<chapter id="archive-modules">
- <title>Archive Modules</title>
- <indexterm zone="archive-modules">
- <primary>Archive Modules</primary>
+<chapter id="archive-and-restore-modules">
+ <title>Archive and Restore Modules</title>
+ <indexterm zone="archive-and-restore-modules">
+ <primary>Archive and Restore Modules</primary>
</indexterm>
<para>
@@ -14,45 +14,52 @@
performant.
</para>
- <para>
- When a custom <xref linkend="guc-archive-library"/> is configured, PostgreSQL
- will submit completed WAL files to the module, and the server will avoid
- recycling or removing these WAL files until the module indicates that the files
- were successfully archived. It is ultimately up to the module to decide what
- to do with each WAL file, but many recommendations are listed at
- <xref linkend="backup-archiving-wal"/>.
- </para>
-
- <para>
- Archiving modules must at least consist of an initialization function (see
- <xref linkend="archive-module-init"/>) and the required callbacks (see
- <xref linkend="archive-module-callbacks"/>). However, archive modules are
- also permitted to do much more (e.g., declare GUCs and register background
- workers).
- </para>
-
<para>
The <filename>contrib/basic_archive</filename> module contains a working
example, which demonstrates some useful techniques.
</para>
- <sect1 id="archive-module-init">
- <title>Initialization Functions</title>
- <indexterm zone="archive-module-init">
- <primary>_PG_archive_module_init</primary>
+ <sect1 id="archive-modules">
+ <title>Archive Modules</title>
+ <indexterm zone="archive-modules">
+ <primary>Archive Modules</primary>
</indexterm>
+
+ <para>
+ When a custom <xref linkend="guc-archive-library"/> is configured,
+ PostgreSQL will submit completed WAL files to the module, and the server
+ will avoid recycling or removing these WAL files until the module indicates
+ that the files were successfully archived. It is ultimately up to the
+ module to decide what to do with each WAL file, but many recommendations are
+ listed at <xref linkend="backup-archiving-wal"/>.
+ </para>
+
<para>
- An archive library is loaded by dynamically loading a shared library with the
- <xref linkend="guc-archive-library"/>'s name as the library base name. The
- normal library search path is used to locate the library. To provide the
- required archive module callbacks and to indicate that the library is
- actually an archive module, it needs to provide a function named
- <function>_PG_archive_module_init</function>. The result of the function
- must be a pointer to a struct of type
- <structname>ArchiveModuleCallbacks</structname>, which contains everything
- that the core code needs to know to make use of the archive module. The
- return value needs to be of server lifetime, which is typically achieved by
- defining it as a <literal>static const</literal> variable in global scope.
+ Archiving modules must at least consist of an initialization function (see
+ <xref linkend="archive-module-init"/>) and the required callbacks (see
+ <xref linkend="archive-module-callbacks"/>). However, archive modules are
+ also permitted to do much more (e.g., declare GUCs and register background
+ workers).
+ </para>
+
+ <sect2 id="archive-module-init">
+ <title>Initialization Functions</title>
+ <indexterm zone="archive-module-init">
+ <primary>_PG_archive_module_init</primary>
+ </indexterm>
+
+ <para>
+ An archive library is loaded by dynamically loading a shared library with
+ the <xref linkend="guc-archive-library"/>'s name as the library base name.
+ The normal library search path is used to locate the library. To provide
+ the required archive module callbacks and to indicate that the library is
+ actually an archive module, it needs to provide a function named
+ <function>_PG_archive_module_init</function>. The result of the function
+ must be a pointer to a struct of type
+ <structname>ArchiveModuleCallbacks</structname>, which contains everything
+ that the core code needs to know to make use of the archive module. The
+ return value needs to be of server lifetime, which is typically achieved by
+ defining it as a <literal>static const</literal> variable in global scope.
<programlisting>
typedef struct ArchiveModuleCallbacks
@@ -65,103 +72,110 @@ typedef struct ArchiveModuleCallbacks
typedef const ArchiveModuleCallbacks *(*ArchiveModuleInit) (void);
</programlisting>
- Only the <function>archive_file_cb</function> callback is required. The
- others are optional.
- </para>
- </sect1>
+ Only the <function>archive_file_cb</function> callback is required. The
+ others are optional.
+ </para>
+ </sect2>
- <sect1 id="archive-module-callbacks">
- <title>Archive Module Callbacks</title>
- <para>
- The archive callbacks define the actual archiving behavior of the module.
- The server will call them as required to process each individual WAL file.
- </para>
+ <sect2 id="archive-module-callbacks">
+ <title>Archive Module Callbacks</title>
- <sect2 id="archive-module-startup">
- <title>Startup Callback</title>
<para>
- The <function>startup_cb</function> callback is called shortly after the
- module is loaded. This callback can be used to perform any additional
- initialization required. If the archive module has any state, it can use
- <structfield>state->private_data</structfield> to store it.
+ The archive callbacks define the actual archiving behavior of the module.
+ The server will call them as required to process each individual WAL file.
+ </para>
+
+ <sect3 id="archive-module-startup">
+ <title>Startup Callback</title>
+
+ <para>
+ The <function>startup_cb</function> callback is called shortly after the
+ module is loaded. This callback can be used to perform any additional
+ initialization required. If the archive module has any state, it can use
+ <structfield>state->private_data</structfield> to store it.
<programlisting>
typedef void (*ArchiveStartupCB) (ArchiveModuleState *state);
</programlisting>
- </para>
- </sect2>
+ </para>
+ </sect3>
- <sect2 id="archive-module-check">
- <title>Check Callback</title>
- <para>
- The <function>check_configured_cb</function> callback is called to determine
- whether the module is fully configured and ready to accept WAL files (e.g.,
- its configuration parameters are set to valid values). If no
- <function>check_configured_cb</function> is defined, the server always
- assumes the module is configured.
+ <sect3 id="archive-module-check">
+ <title>Check Callback</title>
+
+ <para>
+ The <function>check_configured_cb</function> callback is called to
+ determine whether the module is fully configured and ready to accept WAL
+ files (e.g., its configuration parameters are set to valid values). If no
+ <function>check_configured_cb</function> is defined, the server always
+ assumes the module is configured.
<programlisting>
typedef bool (*ArchiveCheckConfiguredCB) (ArchiveModuleState *state);
</programlisting>
- If <literal>true</literal> is returned, the server will proceed with
- archiving the file by calling the <function>archive_file_cb</function>
- callback. If <literal>false</literal> is returned, archiving will not
- proceed, and the archiver will emit the following message to the server log:
+ If <literal>true</literal> is returned, the server will proceed with
+ archiving the file by calling the <function>archive_file_cb</function>
+ callback. If <literal>false</literal> is returned, archiving will not
+ proceed, and the archiver will emit the following message to the server
+ log:
<screen>
WARNING: archive_mode enabled, yet archiving is not configured
</screen>
- In the latter case, the server will periodically call this function, and
- archiving will proceed only when it returns <literal>true</literal>.
- </para>
-
- <note>
- <para>
- When returning <literal>false</literal>, it may be useful to append some
- additional information to the generic warning message. To do that, provide
- a message to the <function>arch_module_check_errdetail</function> macro
- before returning <literal>false</literal>. Like
- <function>errdetail()</function>, this macro accepts a format string
- followed by an optional list of arguments. The resulting string will be
- emitted as the <literal>DETAIL</literal> line of the warning message.
+ In the latter case, the server will periodically call this function, and
+ archiving will proceed only when it returns <literal>true</literal>.
</para>
- </note>
- </sect2>
- <sect2 id="archive-module-archive">
- <title>Archive Callback</title>
- <para>
- The <function>archive_file_cb</function> callback is called to archive a
- single WAL file.
+ <note>
+ <para>
+ When returning <literal>false</literal>, it may be useful to append some
+ additional information to the generic warning message. To do that,
+ provide a message to the <function>arch_module_check_errdetail</function>
+ macro before returning <literal>false</literal>. Like
+ <function>errdetail()</function>, this macro accepts a format string
+ followed by an optional list of arguments. The resulting string will be
+ emitted as the <literal>DETAIL</literal> line of the warning message.
+ </para>
+ </note>
+ </sect3>
+
+ <sect3 id="archive-module-archive">
+ <title>Archive Callback</title>
+
+ <para>
+ The <function>archive_file_cb</function> callback is called to archive a
+ single WAL file.
<programlisting>
typedef bool (*ArchiveFileCB) (ArchiveModuleState *state, const char *file, const char *path);
</programlisting>
- If <literal>true</literal> is returned, the server proceeds as if the file
- was successfully archived, which may include recycling or removing the
- original WAL file. If <literal>false</literal> is returned, the server will
- keep the original WAL file and retry archiving later.
- <replaceable>file</replaceable> will contain just the file name of the WAL
- file to archive, while <replaceable>path</replaceable> contains the full
- path of the WAL file (including the file name).
- </para>
- </sect2>
+ If <literal>true</literal> is returned, the server proceeds as if the file
+ was successfully archived, which may include recycling or removing the
+ original WAL file. If <literal>false</literal> is returned, the server
+ will keep the original WAL file and retry archiving later.
+ <replaceable>file</replaceable> will contain just the file name of the WAL
+ file to archive, while <replaceable>path</replaceable> contains the full
+ path of the WAL file (including the file name).
+ </para>
+ </sect3>
- <sect2 id="archive-module-shutdown">
- <title>Shutdown Callback</title>
- <para>
- The <function>shutdown_cb</function> callback is called when the archiver
- process exits (e.g., after an error) or the value of
- <xref linkend="guc-archive-library"/> changes. If no
- <function>shutdown_cb</function> is defined, no special action is taken in
- these situations. If the archive module has any state, this callback should
- free it to avoid leaks.
+ <sect3 id="archive-module-shutdown">
+ <title>Shutdown Callback</title>
+
+ <para>
+ The <function>shutdown_cb</function> callback is called when the archiver
+ process exits (e.g., after an error) or the value of
+ <xref linkend="guc-archive-library"/> changes. If no
+ <function>shutdown_cb</function> is defined, no special action is taken in
+ these situations. If the archive module has any state, this callback
+ should free it to avoid leaks.
<programlisting>
typedef void (*ArchiveShutdownCB) (ArchiveModuleState *state);
</programlisting>
- </para>
+ </para>
+ </sect3>
</sect2>
</sect1>
</chapter>
--
2.25.1
--3V7upXqbjpZ4EhLz
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v20-0005-introduce-restore_library.patch"
^ permalink raw reply [nested|flat] 24+ messages in thread
* [PATCH v15 6/7] restructure archive modules docs in preparation for restore modules
@ 2023-02-15 22:48 Nathan Bossart <[email protected]>
0 siblings, 0 replies; 24+ messages in thread
From: Nathan Bossart @ 2023-02-15 22:48 UTC (permalink / raw)
---
doc/src/sgml/archive-and-restore-modules.sgml | 206 ++++++++++--------
1 file changed, 110 insertions(+), 96 deletions(-)
diff --git a/doc/src/sgml/archive-and-restore-modules.sgml b/doc/src/sgml/archive-and-restore-modules.sgml
index 7064307d9e..7ed50a3b52 100644
--- a/doc/src/sgml/archive-and-restore-modules.sgml
+++ b/doc/src/sgml/archive-and-restore-modules.sgml
@@ -1,9 +1,9 @@
-<!-- doc/src/sgml/archive-modules.sgml -->
+<!-- doc/src/sgml/archive-and-restore-modules.sgml -->
-<chapter id="archive-modules">
- <title>Archive Modules</title>
- <indexterm zone="archive-modules">
- <primary>Archive Modules</primary>
+<chapter id="archive-and-restore-modules">
+ <title>Archive and Restore Modules</title>
+ <indexterm zone="archive-and-restore-modules">
+ <primary>Archive and Restore Modules</primary>
</indexterm>
<para>
@@ -14,45 +14,52 @@
performant.
</para>
- <para>
- When a custom <xref linkend="guc-archive-library"/> is configured, PostgreSQL
- will submit completed WAL files to the module, and the server will avoid
- recycling or removing these WAL files until the module indicates that the files
- were successfully archived. It is ultimately up to the module to decide what
- to do with each WAL file, but many recommendations are listed at
- <xref linkend="backup-archiving-wal"/>.
- </para>
-
- <para>
- Archiving modules must at least consist of an initialization function (see
- <xref linkend="archive-module-init"/>) and the required callbacks (see
- <xref linkend="archive-module-callbacks"/>). However, archive modules are
- also permitted to do much more (e.g., declare GUCs and register background
- workers).
- </para>
-
<para>
The <filename>contrib/basic_archive</filename> module contains a working
example, which demonstrates some useful techniques.
</para>
- <sect1 id="archive-module-init">
- <title>Initialization Functions</title>
- <indexterm zone="archive-module-init">
- <primary>_PG_archive_module_init</primary>
+ <sect1 id="archive-modules">
+ <title>Archive Modules</title>
+ <indexterm zone="archive-modules">
+ <primary>Archive Modules</primary>
</indexterm>
+
+ <para>
+ When a custom <xref linkend="guc-archive-library"/> is configured,
+ PostgreSQL will submit completed WAL files to the module, and the server
+ will avoid recycling or removing these WAL files until the module indicates
+ that the files were successfully archived. It is ultimately up to the
+ module to decide what to do with each WAL file, but many recommendations are
+ listed at <xref linkend="backup-archiving-wal"/>.
+ </para>
+
<para>
- An archive library is loaded by dynamically loading a shared library with the
- <xref linkend="guc-archive-library"/>'s name as the library base name. The
- normal library search path is used to locate the library. To provide the
- required archive module callbacks and to indicate that the library is
- actually an archive module, it needs to provide a function named
- <function>_PG_archive_module_init</function>. The result of the function
- must be a pointer to a struct of type
- <structname>ArchiveModuleCallbacks</structname>, which contains everything
- that the core code needs to know to make use of the archive module. The
- return value needs to be of server lifetime, which is typically achieved by
- defining it as a <literal>static const</literal> variable in global scope.
+ Archiving modules must at least consist of an initialization function (see
+ <xref linkend="archive-module-init"/>) and the required callbacks (see
+ <xref linkend="archive-module-callbacks"/>). However, archive modules are
+ also permitted to do much more (e.g., declare GUCs and register background
+ workers).
+ </para>
+
+ <sect2 id="archive-module-init">
+ <title>Initialization Functions</title>
+ <indexterm zone="archive-module-init">
+ <primary>_PG_archive_module_init</primary>
+ </indexterm>
+
+ <para>
+ An archive library is loaded by dynamically loading a shared library with
+ the <xref linkend="guc-archive-library"/>'s name as the library base name.
+ The normal library search path is used to locate the library. To provide
+ the required archive module callbacks and to indicate that the library is
+ actually an archive module, it needs to provide a function named
+ <function>_PG_archive_module_init</function>. The result of the function
+ must be a pointer to a struct of type
+ <structname>ArchiveModuleCallbacks</structname>, which contains everything
+ that the core code needs to know to make use of the archive module. The
+ return value needs to be of server lifetime, which is typically achieved by
+ defining it as a <literal>static const</literal> variable in global scope.
<programlisting>
typedef struct ArchiveModuleCallbacks
@@ -65,91 +72,98 @@ typedef struct ArchiveModuleCallbacks
typedef const ArchiveModuleCallbacks *(*ArchiveModuleInit) (void);
</programlisting>
- Only the <function>archive_file_cb</function> callback is required. The
- others are optional.
- </para>
- </sect1>
+ Only the <function>archive_file_cb</function> callback is required. The
+ others are optional.
+ </para>
+ </sect2>
- <sect1 id="archive-module-callbacks">
- <title>Archive Module Callbacks</title>
- <para>
- The archive callbacks define the actual archiving behavior of the module.
- The server will call them as required to process each individual WAL file.
- </para>
+ <sect2 id="archive-module-callbacks">
+ <title>Archive Module Callbacks</title>
- <sect2 id="archive-module-startup">
- <title>Startup Callback</title>
<para>
- The <function>startup_cb</function> callback is called shortly after the
- module is loaded. This callback can be used to perform any additional
- initialization required. If the archive module has any state, it can use
- <structfield>state->private_data</structfield> to store it.
+ The archive callbacks define the actual archiving behavior of the module.
+ The server will call them as required to process each individual WAL file.
+ </para>
+
+ <sect3 id="archive-module-startup">
+ <title>Startup Callback</title>
+
+ <para>
+ The <function>startup_cb</function> callback is called shortly after the
+ module is loaded. This callback can be used to perform any additional
+ initialization required. If the archive module has any state, it can use
+ <structfield>state->private_data</structfield> to store it.
<programlisting>
typedef void (*ArchiveStartupCB) (ArchiveModuleState *state);
</programlisting>
- </para>
- </sect2>
+ </para>
+ </sect3>
- <sect2 id="archive-module-check">
- <title>Check Callback</title>
- <para>
- The <function>check_configured_cb</function> callback is called to determine
- whether the module is fully configured and ready to accept WAL files (e.g.,
- its configuration parameters are set to valid values). If no
- <function>check_configured_cb</function> is defined, the server always
- assumes the module is configured.
+ <sect3 id="archive-module-check">
+ <title>Check Callback</title>
+
+ <para>
+ The <function>check_configured_cb</function> callback is called to
+ determine whether the module is fully configured and ready to accept WAL
+ files (e.g., its configuration parameters are set to valid values). If no
+ <function>check_configured_cb</function> is defined, the server always
+ assumes the module is configured.
<programlisting>
typedef bool (*ArchiveCheckConfiguredCB) (ArchiveModuleState *state);
</programlisting>
- If <literal>true</literal> is returned, the server will proceed with
- archiving the file by calling the <function>archive_file_cb</function>
- callback. If <literal>false</literal> is returned, archiving will not
- proceed, and the archiver will emit the following message to the server log:
+ If <literal>true</literal> is returned, the server will proceed with
+ archiving the file by calling the <function>archive_file_cb</function>
+ callback. If <literal>false</literal> is returned, archiving will not
+ proceed, and the archiver will emit the following message to the server
+ log:
<screen>
WARNING: archive_mode enabled, yet archiving is not configured
</screen>
- In the latter case, the server will periodically call this function, and
- archiving will proceed only when it returns <literal>true</literal>.
- </para>
- </sect2>
+ In the latter case, the server will periodically call this function, and
+ archiving will proceed only when it returns <literal>true</literal>.
+ </para>
+ </sect3>
- <sect2 id="archive-module-archive">
- <title>Archive Callback</title>
- <para>
- The <function>archive_file_cb</function> callback is called to archive a
- single WAL file.
+ <sect3 id="archive-module-archive">
+ <title>Archive Callback</title>
+
+ <para>
+ The <function>archive_file_cb</function> callback is called to archive a
+ single WAL file.
<programlisting>
typedef bool (*ArchiveFileCB) (ArchiveModuleState *state, const char *file, const char *path);
</programlisting>
- If <literal>true</literal> is returned, the server proceeds as if the file
- was successfully archived, which may include recycling or removing the
- original WAL file. If <literal>false</literal> is returned, the server will
- keep the original WAL file and retry archiving later.
- <replaceable>file</replaceable> will contain just the file name of the WAL
- file to archive, while <replaceable>path</replaceable> contains the full
- path of the WAL file (including the file name).
- </para>
- </sect2>
-
- <sect2 id="archive-module-shutdown">
- <title>Shutdown Callback</title>
- <para>
- The <function>shutdown_cb</function> callback is called when the archiver
- process exits (e.g., after an error) or the value of
- <xref linkend="guc-archive-library"/> changes. If no
- <function>shutdown_cb</function> is defined, no special action is taken in
- these situations. If the archive module has any state, this callback should
- free it to avoid leaks.
+ If <literal>true</literal> is returned, the server proceeds as if the file
+ was successfully archived, which may include recycling or removing the
+ original WAL file. If <literal>false</literal> is returned, the server
+ will keep the original WAL file and retry archiving later.
+ <replaceable>file</replaceable> will contain just the file name of the WAL
+ file to archive, while <replaceable>path</replaceable> contains the full
+ path of the WAL file (including the file name).
+ </para>
+ </sect3>
+
+ <sect3 id="archive-module-shutdown">
+ <title>Shutdown Callback</title>
+
+ <para>
+ The <function>shutdown_cb</function> callback is called when the archiver
+ process exits (e.g., after an error) or the value of
+ <xref linkend="guc-archive-library"/> changes. If no
+ <function>shutdown_cb</function> is defined, no special action is taken in
+ these situations. If the archive module has any state, this callback
+ should free it to avoid leaks.
<programlisting>
typedef void (*ArchiveShutdownCB) (ArchiveModuleState *state);
</programlisting>
- </para>
+ </para>
+ </sect3>
</sect2>
</sect1>
</chapter>
--
2.25.1
--k+w/mQv8wyuph6w0
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v15-0007-introduce-restore_library.patch"
^ permalink raw reply [nested|flat] 24+ messages in thread
* [PATCH v12 5/6] restructure archive modules docs in preparation for restore modules
@ 2023-02-15 22:48 Nathan Bossart <[email protected]>
0 siblings, 0 replies; 24+ messages in thread
From: Nathan Bossart @ 2023-02-15 22:48 UTC (permalink / raw)
---
doc/src/sgml/archive-and-restore-modules.sgml | 207 ++++++++++--------
1 file changed, 111 insertions(+), 96 deletions(-)
diff --git a/doc/src/sgml/archive-and-restore-modules.sgml b/doc/src/sgml/archive-and-restore-modules.sgml
index 7cf44e82e2..f30ee591e7 100644
--- a/doc/src/sgml/archive-and-restore-modules.sgml
+++ b/doc/src/sgml/archive-and-restore-modules.sgml
@@ -1,9 +1,9 @@
-<!-- doc/src/sgml/archive-modules.sgml -->
+<!-- doc/src/sgml/archive-and-restore-modules.sgml -->
-<chapter id="archive-modules">
- <title>Archive Modules</title>
- <indexterm zone="archive-modules">
- <primary>Archive Modules</primary>
+<chapter id="archive-and-restore-modules">
+ <title>Archive and Restore Modules</title>
+ <indexterm zone="archive-and-restore-modules">
+ <primary>Archive and Restore Modules</primary>
</indexterm>
<para>
@@ -14,45 +14,53 @@
performant.
</para>
- <para>
- When a custom <xref linkend="guc-archive-library"/> is configured, PostgreSQL
- will submit completed WAL files to the module, and the server will avoid
- recycling or removing these WAL files until the module indicates that the files
- were successfully archived. It is ultimately up to the module to decide what
- to do with each WAL file, but many recommendations are listed at
- <xref linkend="backup-archiving-wal"/>.
- </para>
-
- <para>
- Archiving modules must at least consist of an initialization function (see
- <xref linkend="archive-module-init"/>) and the required callbacks (see
- <xref linkend="archive-module-callbacks"/>). However, archive modules are
- also permitted to do much more (e.g., declare GUCs and register background
- workers).
- </para>
-
<para>
The <filename>contrib/basic_archive</filename> module contains a working
example, which demonstrates some useful techniques.
</para>
- <sect1 id="archive-module-init">
- <title>Initialization Functions</title>
- <indexterm zone="archive-module-init">
- <primary>_PG_archive_module_init</primary>
+ <sect1 id="archive-modules">
+ <title>Archive Modules</title>
+ <indexterm zone="archive-modules">
+ <primary>Archive Modules</primary>
</indexterm>
+
+ <para>
+ When a custom <xref linkend="guc-archive-library"/> is configured,
+ PostgreSQL will submit completed WAL files to the module, and the server
+ will avoid recycling or removing these WAL files until the module indicates
+ that the files were successfully archived. It is ultimately up to the
+ module to decide what to do with each WAL file, but many recommendations are
+ listed at <xref linkend="backup-archiving-wal"/>.
+ </para>
+
<para>
- An archive library is loaded by dynamically loading a shared library with the
- <xref linkend="guc-archive-library"/>'s name as the library base name. The
- normal library search path is used to locate the library. To provide the
- required archive module callbacks and to indicate that the library is
- actually an archive module, it needs to provide a function named
- <function>_PG_archive_module_init</function>. The result of the function
- must be a pointer to a struct of type
- <structname>ArchiveModuleCallbacks</structname>, which contains everything
- that the core code needs to know how to make use of the archive module. The
- return value needs to be of server lifetime, which is typically achieved by
- defining it as a <literal>static const</literal> variable in global scope.
+ Archiving modules must at least consist of an initialization function (see
+ <xref linkend="archive-module-init"/>) and the required callbacks (see
+ <xref linkend="archive-module-callbacks"/>). However, archive modules are
+ also permitted to do much more (e.g., declare GUCs and register background
+ workers).
+ </para>
+
+ <sect2 id="archive-module-init">
+ <title>Initialization Functions</title>
+ <indexterm zone="archive-module-init">
+ <primary>_PG_archive_module_init</primary>
+ </indexterm>
+
+ <para>
+ An archive library is loaded by dynamically loading a shared library with
+ the <xref linkend="guc-archive-library"/>'s name as the library base name.
+ The normal library search path is used to locate the library. To provide
+ the required archive module callbacks and to indicate that the library is
+ actually an archive module, it needs to provide a function named
+ <function>_PG_archive_module_init</function>. The result of the function
+ must be a pointer to a struct of type
+ <structname>ArchiveModuleCallbacks</structname>, which contains everything
+ that the core code needs to know how to make use of the archive module.
+ The return value needs to be of server lifetime, which is typically
+ achieved by defining it as a <literal>static const</literal> variable in
+ global scope.
<programlisting>
typedef struct ArchiveModuleCallbacks
@@ -65,91 +73,98 @@ typedef struct ArchiveModuleCallbacks
typedef const ArchiveModuleCallbacks *(*ArchiveModuleInit) (void);
</programlisting>
- Only the <function>archive_file_cb</function> callback is required. The
- others are optional.
- </para>
- </sect1>
+ Only the <function>archive_file_cb</function> callback is required. The
+ others are optional.
+ </para>
+ </sect2>
- <sect1 id="archive-module-callbacks">
- <title>Archive Module Callbacks</title>
- <para>
- The archive callbacks define the actual archiving behavior of the module.
- The server will call them as required to process each individual WAL file.
- </para>
+ <sect2 id="archive-module-callbacks">
+ <title>Archive Module Callbacks</title>
- <sect2 id="archive-module-startup">
- <title>Startup Callback</title>
<para>
- The <function>startup_cb</function> callback is called shortly after the
- module is loaded. This callback can be used to perform any additional
- initialization required. If the archive module has a state, it can use
- <structfield>state->private_data</structfield> to store it.
+ The archive callbacks define the actual archiving behavior of the module.
+ The server will call them as required to process each individual WAL file.
+ </para>
+
+ <sect3 id="archive-module-startup">
+ <title>Startup Callback</title>
+
+ <para>
+ The <function>startup_cb</function> callback is called shortly after the
+ module is loaded. This callback can be used to perform any additional
+ initialization required. If the archive module has state, it can use
+ <structfield>state->private_data</structfield> to store it.
<programlisting>
typedef void (*ArchiveStartupCB) (ArchiveModuleState *state);
</programlisting>
- </para>
- </sect2>
+ </para>
+ </sect3>
- <sect2 id="archive-module-check">
- <title>Check Callback</title>
- <para>
- The <function>check_configured_cb</function> callback is called to determine
- whether the module is fully configured and ready to accept WAL files (e.g.,
- its configuration parameters are set to valid values). If no
- <function>check_configured_cb</function> is defined, the server always
- assumes the module is configured.
+ <sect3 id="archive-module-check">
+ <title>Check Callback</title>
+
+ <para>
+ The <function>check_configured_cb</function> callback is called to
+ determine whether the module is fully configured and ready to accept WAL
+ files (e.g., its configuration parameters are set to valid values). If no
+ <function>check_configured_cb</function> is defined, the server always
+ assumes the module is configured.
<programlisting>
typedef bool (*ArchiveCheckConfiguredCB) (ArchiveModuleState *state);
</programlisting>
- If <literal>true</literal> is returned, the server will proceed with
- archiving the file by calling the <function>archive_file_cb</function>
- callback. If <literal>false</literal> is returned, archiving will not
- proceed, and the archiver will emit the following message to the server log:
+ If <literal>true</literal> is returned, the server will proceed with
+ archiving the file by calling the <function>archive_file_cb</function>
+ callback. If <literal>false</literal> is returned, archiving will not
+ proceed, and the archiver will emit the following message to the server
+ log:
<screen>
WARNING: archive_mode enabled, yet archiving is not configured
</screen>
- In the latter case, the server will periodically call this function, and
- archiving will proceed only when it returns <literal>true</literal>.
- </para>
- </sect2>
+ In the latter case, the server will periodically call this function, and
+ archiving will proceed only when it returns <literal>true</literal>.
+ </para>
+ </sect3>
- <sect2 id="archive-module-archive">
- <title>Archive Callback</title>
- <para>
- The <function>archive_file_cb</function> callback is called to archive a
- single WAL file.
+ <sect3 id="archive-module-archive">
+ <title>Archive Callback</title>
+
+ <para>
+ The <function>archive_file_cb</function> callback is called to archive a
+ single WAL file.
<programlisting>
typedef bool (*ArchiveFileCB) (ArchiveModuleState *state, const char *file, const char *path);
</programlisting>
- If <literal>true</literal> is returned, the server proceeds as if the file
- was successfully archived, which may include recycling or removing the
- original WAL file. If <literal>false</literal> is returned, the server will
- keep the original WAL file and retry archiving later.
- <replaceable>file</replaceable> will contain just the file name of the WAL
- file to archive, while <replaceable>path</replaceable> contains the full
- path of the WAL file (including the file name).
- </para>
- </sect2>
-
- <sect2 id="archive-module-shutdown">
- <title>Shutdown Callback</title>
- <para>
- The <function>shutdown_cb</function> callback is called when the archiver
- process exits (e.g., after an error) or the value of
- <xref linkend="guc-archive-library"/> changes. If no
- <function>shutdown_cb</function> is defined, no special action is taken in
- these situations. If the archive module has a state, this callback should
- free it to avoid leaks.
+ If <literal>true</literal> is returned, the server proceeds as if the file
+ was successfully archived, which may include recycling or removing the
+ original WAL file. If <literal>false</literal> is returned, the server
+ will keep the original WAL file and retry archiving later.
+ <replaceable>file</replaceable> will contain just the file name of the WAL
+ file to archive, while <replaceable>path</replaceable> contains the full
+ path of the WAL file (including the file name).
+ </para>
+ </sect3>
+
+ <sect3 id="archive-module-shutdown">
+ <title>Shutdown Callback</title>
+
+ <para>
+ The <function>shutdown_cb</function> callback is called when the archiver
+ process exits (e.g., after an error) or the value of
+ <xref linkend="guc-archive-library"/> changes. If no
+ <function>shutdown_cb</function> is defined, no special action is taken in
+ these situations. If the archive module has state, this callback should
+ free it to avoid leaks.
<programlisting>
typedef void (*ArchiveShutdownCB) (ArchiveModuleState *state);
</programlisting>
- </para>
+ </para>
+ </sect3>
</sect2>
</sect1>
</chapter>
--
2.25.1
--YiEDa0DAkWCtVeE4
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v12-0006-introduce-restore_library.patch"
^ permalink raw reply [nested|flat] 24+ messages in thread
* [PATCH v13 6/7] restructure archive modules docs in preparation for restore modules
@ 2023-02-15 22:48 Nathan Bossart <[email protected]>
0 siblings, 0 replies; 24+ messages in thread
From: Nathan Bossart @ 2023-02-15 22:48 UTC (permalink / raw)
---
doc/src/sgml/archive-and-restore-modules.sgml | 207 ++++++++++--------
1 file changed, 111 insertions(+), 96 deletions(-)
diff --git a/doc/src/sgml/archive-and-restore-modules.sgml b/doc/src/sgml/archive-and-restore-modules.sgml
index 7cf44e82e2..f30ee591e7 100644
--- a/doc/src/sgml/archive-and-restore-modules.sgml
+++ b/doc/src/sgml/archive-and-restore-modules.sgml
@@ -1,9 +1,9 @@
-<!-- doc/src/sgml/archive-modules.sgml -->
+<!-- doc/src/sgml/archive-and-restore-modules.sgml -->
-<chapter id="archive-modules">
- <title>Archive Modules</title>
- <indexterm zone="archive-modules">
- <primary>Archive Modules</primary>
+<chapter id="archive-and-restore-modules">
+ <title>Archive and Restore Modules</title>
+ <indexterm zone="archive-and-restore-modules">
+ <primary>Archive and Restore Modules</primary>
</indexterm>
<para>
@@ -14,45 +14,53 @@
performant.
</para>
- <para>
- When a custom <xref linkend="guc-archive-library"/> is configured, PostgreSQL
- will submit completed WAL files to the module, and the server will avoid
- recycling or removing these WAL files until the module indicates that the files
- were successfully archived. It is ultimately up to the module to decide what
- to do with each WAL file, but many recommendations are listed at
- <xref linkend="backup-archiving-wal"/>.
- </para>
-
- <para>
- Archiving modules must at least consist of an initialization function (see
- <xref linkend="archive-module-init"/>) and the required callbacks (see
- <xref linkend="archive-module-callbacks"/>). However, archive modules are
- also permitted to do much more (e.g., declare GUCs and register background
- workers).
- </para>
-
<para>
The <filename>contrib/basic_archive</filename> module contains a working
example, which demonstrates some useful techniques.
</para>
- <sect1 id="archive-module-init">
- <title>Initialization Functions</title>
- <indexterm zone="archive-module-init">
- <primary>_PG_archive_module_init</primary>
+ <sect1 id="archive-modules">
+ <title>Archive Modules</title>
+ <indexterm zone="archive-modules">
+ <primary>Archive Modules</primary>
</indexterm>
+
+ <para>
+ When a custom <xref linkend="guc-archive-library"/> is configured,
+ PostgreSQL will submit completed WAL files to the module, and the server
+ will avoid recycling or removing these WAL files until the module indicates
+ that the files were successfully archived. It is ultimately up to the
+ module to decide what to do with each WAL file, but many recommendations are
+ listed at <xref linkend="backup-archiving-wal"/>.
+ </para>
+
<para>
- An archive library is loaded by dynamically loading a shared library with the
- <xref linkend="guc-archive-library"/>'s name as the library base name. The
- normal library search path is used to locate the library. To provide the
- required archive module callbacks and to indicate that the library is
- actually an archive module, it needs to provide a function named
- <function>_PG_archive_module_init</function>. The result of the function
- must be a pointer to a struct of type
- <structname>ArchiveModuleCallbacks</structname>, which contains everything
- that the core code needs to know how to make use of the archive module. The
- return value needs to be of server lifetime, which is typically achieved by
- defining it as a <literal>static const</literal> variable in global scope.
+ Archiving modules must at least consist of an initialization function (see
+ <xref linkend="archive-module-init"/>) and the required callbacks (see
+ <xref linkend="archive-module-callbacks"/>). However, archive modules are
+ also permitted to do much more (e.g., declare GUCs and register background
+ workers).
+ </para>
+
+ <sect2 id="archive-module-init">
+ <title>Initialization Functions</title>
+ <indexterm zone="archive-module-init">
+ <primary>_PG_archive_module_init</primary>
+ </indexterm>
+
+ <para>
+ An archive library is loaded by dynamically loading a shared library with
+ the <xref linkend="guc-archive-library"/>'s name as the library base name.
+ The normal library search path is used to locate the library. To provide
+ the required archive module callbacks and to indicate that the library is
+ actually an archive module, it needs to provide a function named
+ <function>_PG_archive_module_init</function>. The result of the function
+ must be a pointer to a struct of type
+ <structname>ArchiveModuleCallbacks</structname>, which contains everything
+ that the core code needs to know how to make use of the archive module.
+ The return value needs to be of server lifetime, which is typically
+ achieved by defining it as a <literal>static const</literal> variable in
+ global scope.
<programlisting>
typedef struct ArchiveModuleCallbacks
@@ -65,91 +73,98 @@ typedef struct ArchiveModuleCallbacks
typedef const ArchiveModuleCallbacks *(*ArchiveModuleInit) (void);
</programlisting>
- Only the <function>archive_file_cb</function> callback is required. The
- others are optional.
- </para>
- </sect1>
+ Only the <function>archive_file_cb</function> callback is required. The
+ others are optional.
+ </para>
+ </sect2>
- <sect1 id="archive-module-callbacks">
- <title>Archive Module Callbacks</title>
- <para>
- The archive callbacks define the actual archiving behavior of the module.
- The server will call them as required to process each individual WAL file.
- </para>
+ <sect2 id="archive-module-callbacks">
+ <title>Archive Module Callbacks</title>
- <sect2 id="archive-module-startup">
- <title>Startup Callback</title>
<para>
- The <function>startup_cb</function> callback is called shortly after the
- module is loaded. This callback can be used to perform any additional
- initialization required. If the archive module has a state, it can use
- <structfield>state->private_data</structfield> to store it.
+ The archive callbacks define the actual archiving behavior of the module.
+ The server will call them as required to process each individual WAL file.
+ </para>
+
+ <sect3 id="archive-module-startup">
+ <title>Startup Callback</title>
+
+ <para>
+ The <function>startup_cb</function> callback is called shortly after the
+ module is loaded. This callback can be used to perform any additional
+ initialization required. If the archive module has state, it can use
+ <structfield>state->private_data</structfield> to store it.
<programlisting>
typedef void (*ArchiveStartupCB) (ArchiveModuleState *state);
</programlisting>
- </para>
- </sect2>
+ </para>
+ </sect3>
- <sect2 id="archive-module-check">
- <title>Check Callback</title>
- <para>
- The <function>check_configured_cb</function> callback is called to determine
- whether the module is fully configured and ready to accept WAL files (e.g.,
- its configuration parameters are set to valid values). If no
- <function>check_configured_cb</function> is defined, the server always
- assumes the module is configured.
+ <sect3 id="archive-module-check">
+ <title>Check Callback</title>
+
+ <para>
+ The <function>check_configured_cb</function> callback is called to
+ determine whether the module is fully configured and ready to accept WAL
+ files (e.g., its configuration parameters are set to valid values). If no
+ <function>check_configured_cb</function> is defined, the server always
+ assumes the module is configured.
<programlisting>
typedef bool (*ArchiveCheckConfiguredCB) (ArchiveModuleState *state);
</programlisting>
- If <literal>true</literal> is returned, the server will proceed with
- archiving the file by calling the <function>archive_file_cb</function>
- callback. If <literal>false</literal> is returned, archiving will not
- proceed, and the archiver will emit the following message to the server log:
+ If <literal>true</literal> is returned, the server will proceed with
+ archiving the file by calling the <function>archive_file_cb</function>
+ callback. If <literal>false</literal> is returned, archiving will not
+ proceed, and the archiver will emit the following message to the server
+ log:
<screen>
WARNING: archive_mode enabled, yet archiving is not configured
</screen>
- In the latter case, the server will periodically call this function, and
- archiving will proceed only when it returns <literal>true</literal>.
- </para>
- </sect2>
+ In the latter case, the server will periodically call this function, and
+ archiving will proceed only when it returns <literal>true</literal>.
+ </para>
+ </sect3>
- <sect2 id="archive-module-archive">
- <title>Archive Callback</title>
- <para>
- The <function>archive_file_cb</function> callback is called to archive a
- single WAL file.
+ <sect3 id="archive-module-archive">
+ <title>Archive Callback</title>
+
+ <para>
+ The <function>archive_file_cb</function> callback is called to archive a
+ single WAL file.
<programlisting>
typedef bool (*ArchiveFileCB) (ArchiveModuleState *state, const char *file, const char *path);
</programlisting>
- If <literal>true</literal> is returned, the server proceeds as if the file
- was successfully archived, which may include recycling or removing the
- original WAL file. If <literal>false</literal> is returned, the server will
- keep the original WAL file and retry archiving later.
- <replaceable>file</replaceable> will contain just the file name of the WAL
- file to archive, while <replaceable>path</replaceable> contains the full
- path of the WAL file (including the file name).
- </para>
- </sect2>
-
- <sect2 id="archive-module-shutdown">
- <title>Shutdown Callback</title>
- <para>
- The <function>shutdown_cb</function> callback is called when the archiver
- process exits (e.g., after an error) or the value of
- <xref linkend="guc-archive-library"/> changes. If no
- <function>shutdown_cb</function> is defined, no special action is taken in
- these situations. If the archive module has a state, this callback should
- free it to avoid leaks.
+ If <literal>true</literal> is returned, the server proceeds as if the file
+ was successfully archived, which may include recycling or removing the
+ original WAL file. If <literal>false</literal> is returned, the server
+ will keep the original WAL file and retry archiving later.
+ <replaceable>file</replaceable> will contain just the file name of the WAL
+ file to archive, while <replaceable>path</replaceable> contains the full
+ path of the WAL file (including the file name).
+ </para>
+ </sect3>
+
+ <sect3 id="archive-module-shutdown">
+ <title>Shutdown Callback</title>
+
+ <para>
+ The <function>shutdown_cb</function> callback is called when the archiver
+ process exits (e.g., after an error) or the value of
+ <xref linkend="guc-archive-library"/> changes. If no
+ <function>shutdown_cb</function> is defined, no special action is taken in
+ these situations. If the archive module has state, this callback should
+ free it to avoid leaks.
<programlisting>
typedef void (*ArchiveShutdownCB) (ArchiveModuleState *state);
</programlisting>
- </para>
+ </para>
+ </sect3>
</sect2>
</sect1>
</chapter>
--
2.25.1
--5vNYLRcllDrimb99
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v13-0007-introduce-restore_library.patch"
^ permalink raw reply [nested|flat] 24+ messages in thread
* [PATCH v14 6/7] restructure archive modules docs in preparation for restore modules
@ 2023-02-15 22:48 Nathan Bossart <[email protected]>
0 siblings, 0 replies; 24+ messages in thread
From: Nathan Bossart @ 2023-02-15 22:48 UTC (permalink / raw)
---
doc/src/sgml/archive-and-restore-modules.sgml | 207 ++++++++++--------
1 file changed, 111 insertions(+), 96 deletions(-)
diff --git a/doc/src/sgml/archive-and-restore-modules.sgml b/doc/src/sgml/archive-and-restore-modules.sgml
index 7cf44e82e2..f30ee591e7 100644
--- a/doc/src/sgml/archive-and-restore-modules.sgml
+++ b/doc/src/sgml/archive-and-restore-modules.sgml
@@ -1,9 +1,9 @@
-<!-- doc/src/sgml/archive-modules.sgml -->
+<!-- doc/src/sgml/archive-and-restore-modules.sgml -->
-<chapter id="archive-modules">
- <title>Archive Modules</title>
- <indexterm zone="archive-modules">
- <primary>Archive Modules</primary>
+<chapter id="archive-and-restore-modules">
+ <title>Archive and Restore Modules</title>
+ <indexterm zone="archive-and-restore-modules">
+ <primary>Archive and Restore Modules</primary>
</indexterm>
<para>
@@ -14,45 +14,53 @@
performant.
</para>
- <para>
- When a custom <xref linkend="guc-archive-library"/> is configured, PostgreSQL
- will submit completed WAL files to the module, and the server will avoid
- recycling or removing these WAL files until the module indicates that the files
- were successfully archived. It is ultimately up to the module to decide what
- to do with each WAL file, but many recommendations are listed at
- <xref linkend="backup-archiving-wal"/>.
- </para>
-
- <para>
- Archiving modules must at least consist of an initialization function (see
- <xref linkend="archive-module-init"/>) and the required callbacks (see
- <xref linkend="archive-module-callbacks"/>). However, archive modules are
- also permitted to do much more (e.g., declare GUCs and register background
- workers).
- </para>
-
<para>
The <filename>contrib/basic_archive</filename> module contains a working
example, which demonstrates some useful techniques.
</para>
- <sect1 id="archive-module-init">
- <title>Initialization Functions</title>
- <indexterm zone="archive-module-init">
- <primary>_PG_archive_module_init</primary>
+ <sect1 id="archive-modules">
+ <title>Archive Modules</title>
+ <indexterm zone="archive-modules">
+ <primary>Archive Modules</primary>
</indexterm>
+
+ <para>
+ When a custom <xref linkend="guc-archive-library"/> is configured,
+ PostgreSQL will submit completed WAL files to the module, and the server
+ will avoid recycling or removing these WAL files until the module indicates
+ that the files were successfully archived. It is ultimately up to the
+ module to decide what to do with each WAL file, but many recommendations are
+ listed at <xref linkend="backup-archiving-wal"/>.
+ </para>
+
<para>
- An archive library is loaded by dynamically loading a shared library with the
- <xref linkend="guc-archive-library"/>'s name as the library base name. The
- normal library search path is used to locate the library. To provide the
- required archive module callbacks and to indicate that the library is
- actually an archive module, it needs to provide a function named
- <function>_PG_archive_module_init</function>. The result of the function
- must be a pointer to a struct of type
- <structname>ArchiveModuleCallbacks</structname>, which contains everything
- that the core code needs to know how to make use of the archive module. The
- return value needs to be of server lifetime, which is typically achieved by
- defining it as a <literal>static const</literal> variable in global scope.
+ Archiving modules must at least consist of an initialization function (see
+ <xref linkend="archive-module-init"/>) and the required callbacks (see
+ <xref linkend="archive-module-callbacks"/>). However, archive modules are
+ also permitted to do much more (e.g., declare GUCs and register background
+ workers).
+ </para>
+
+ <sect2 id="archive-module-init">
+ <title>Initialization Functions</title>
+ <indexterm zone="archive-module-init">
+ <primary>_PG_archive_module_init</primary>
+ </indexterm>
+
+ <para>
+ An archive library is loaded by dynamically loading a shared library with
+ the <xref linkend="guc-archive-library"/>'s name as the library base name.
+ The normal library search path is used to locate the library. To provide
+ the required archive module callbacks and to indicate that the library is
+ actually an archive module, it needs to provide a function named
+ <function>_PG_archive_module_init</function>. The result of the function
+ must be a pointer to a struct of type
+ <structname>ArchiveModuleCallbacks</structname>, which contains everything
+ that the core code needs to know how to make use of the archive module.
+ The return value needs to be of server lifetime, which is typically
+ achieved by defining it as a <literal>static const</literal> variable in
+ global scope.
<programlisting>
typedef struct ArchiveModuleCallbacks
@@ -65,91 +73,98 @@ typedef struct ArchiveModuleCallbacks
typedef const ArchiveModuleCallbacks *(*ArchiveModuleInit) (void);
</programlisting>
- Only the <function>archive_file_cb</function> callback is required. The
- others are optional.
- </para>
- </sect1>
+ Only the <function>archive_file_cb</function> callback is required. The
+ others are optional.
+ </para>
+ </sect2>
- <sect1 id="archive-module-callbacks">
- <title>Archive Module Callbacks</title>
- <para>
- The archive callbacks define the actual archiving behavior of the module.
- The server will call them as required to process each individual WAL file.
- </para>
+ <sect2 id="archive-module-callbacks">
+ <title>Archive Module Callbacks</title>
- <sect2 id="archive-module-startup">
- <title>Startup Callback</title>
<para>
- The <function>startup_cb</function> callback is called shortly after the
- module is loaded. This callback can be used to perform any additional
- initialization required. If the archive module has a state, it can use
- <structfield>state->private_data</structfield> to store it.
+ The archive callbacks define the actual archiving behavior of the module.
+ The server will call them as required to process each individual WAL file.
+ </para>
+
+ <sect3 id="archive-module-startup">
+ <title>Startup Callback</title>
+
+ <para>
+ The <function>startup_cb</function> callback is called shortly after the
+ module is loaded. This callback can be used to perform any additional
+ initialization required. If the archive module has state, it can use
+ <structfield>state->private_data</structfield> to store it.
<programlisting>
typedef void (*ArchiveStartupCB) (ArchiveModuleState *state);
</programlisting>
- </para>
- </sect2>
+ </para>
+ </sect3>
- <sect2 id="archive-module-check">
- <title>Check Callback</title>
- <para>
- The <function>check_configured_cb</function> callback is called to determine
- whether the module is fully configured and ready to accept WAL files (e.g.,
- its configuration parameters are set to valid values). If no
- <function>check_configured_cb</function> is defined, the server always
- assumes the module is configured.
+ <sect3 id="archive-module-check">
+ <title>Check Callback</title>
+
+ <para>
+ The <function>check_configured_cb</function> callback is called to
+ determine whether the module is fully configured and ready to accept WAL
+ files (e.g., its configuration parameters are set to valid values). If no
+ <function>check_configured_cb</function> is defined, the server always
+ assumes the module is configured.
<programlisting>
typedef bool (*ArchiveCheckConfiguredCB) (ArchiveModuleState *state);
</programlisting>
- If <literal>true</literal> is returned, the server will proceed with
- archiving the file by calling the <function>archive_file_cb</function>
- callback. If <literal>false</literal> is returned, archiving will not
- proceed, and the archiver will emit the following message to the server log:
+ If <literal>true</literal> is returned, the server will proceed with
+ archiving the file by calling the <function>archive_file_cb</function>
+ callback. If <literal>false</literal> is returned, archiving will not
+ proceed, and the archiver will emit the following message to the server
+ log:
<screen>
WARNING: archive_mode enabled, yet archiving is not configured
</screen>
- In the latter case, the server will periodically call this function, and
- archiving will proceed only when it returns <literal>true</literal>.
- </para>
- </sect2>
+ In the latter case, the server will periodically call this function, and
+ archiving will proceed only when it returns <literal>true</literal>.
+ </para>
+ </sect3>
- <sect2 id="archive-module-archive">
- <title>Archive Callback</title>
- <para>
- The <function>archive_file_cb</function> callback is called to archive a
- single WAL file.
+ <sect3 id="archive-module-archive">
+ <title>Archive Callback</title>
+
+ <para>
+ The <function>archive_file_cb</function> callback is called to archive a
+ single WAL file.
<programlisting>
typedef bool (*ArchiveFileCB) (ArchiveModuleState *state, const char *file, const char *path);
</programlisting>
- If <literal>true</literal> is returned, the server proceeds as if the file
- was successfully archived, which may include recycling or removing the
- original WAL file. If <literal>false</literal> is returned, the server will
- keep the original WAL file and retry archiving later.
- <replaceable>file</replaceable> will contain just the file name of the WAL
- file to archive, while <replaceable>path</replaceable> contains the full
- path of the WAL file (including the file name).
- </para>
- </sect2>
-
- <sect2 id="archive-module-shutdown">
- <title>Shutdown Callback</title>
- <para>
- The <function>shutdown_cb</function> callback is called when the archiver
- process exits (e.g., after an error) or the value of
- <xref linkend="guc-archive-library"/> changes. If no
- <function>shutdown_cb</function> is defined, no special action is taken in
- these situations. If the archive module has a state, this callback should
- free it to avoid leaks.
+ If <literal>true</literal> is returned, the server proceeds as if the file
+ was successfully archived, which may include recycling or removing the
+ original WAL file. If <literal>false</literal> is returned, the server
+ will keep the original WAL file and retry archiving later.
+ <replaceable>file</replaceable> will contain just the file name of the WAL
+ file to archive, while <replaceable>path</replaceable> contains the full
+ path of the WAL file (including the file name).
+ </para>
+ </sect3>
+
+ <sect3 id="archive-module-shutdown">
+ <title>Shutdown Callback</title>
+
+ <para>
+ The <function>shutdown_cb</function> callback is called when the archiver
+ process exits (e.g., after an error) or the value of
+ <xref linkend="guc-archive-library"/> changes. If no
+ <function>shutdown_cb</function> is defined, no special action is taken in
+ these situations. If the archive module has state, this callback should
+ free it to avoid leaks.
<programlisting>
typedef void (*ArchiveShutdownCB) (ArchiveModuleState *state);
</programlisting>
- </para>
+ </para>
+ </sect3>
</sect2>
</sect1>
</chapter>
--
2.25.1
--IS0zKkzwUGydFO0o
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v14-0007-introduce-restore_library.patch"
^ permalink raw reply [nested|flat] 24+ messages in thread
* [PATCH v18 4/5] restructure archive modules docs in preparation for restore modules
@ 2023-02-15 22:48 Nathan Bossart <[email protected]>
0 siblings, 0 replies; 24+ messages in thread
From: Nathan Bossart @ 2023-02-15 22:48 UTC (permalink / raw)
---
doc/src/sgml/archive-and-restore-modules.sgml | 226 ++++++++++--------
1 file changed, 120 insertions(+), 106 deletions(-)
diff --git a/doc/src/sgml/archive-and-restore-modules.sgml b/doc/src/sgml/archive-and-restore-modules.sgml
index cf7438a759..a52d15082d 100644
--- a/doc/src/sgml/archive-and-restore-modules.sgml
+++ b/doc/src/sgml/archive-and-restore-modules.sgml
@@ -1,9 +1,9 @@
-<!-- doc/src/sgml/archive-modules.sgml -->
+<!-- doc/src/sgml/archive-and-restore-modules.sgml -->
-<chapter id="archive-modules">
- <title>Archive Modules</title>
- <indexterm zone="archive-modules">
- <primary>Archive Modules</primary>
+<chapter id="archive-and-restore-modules">
+ <title>Archive and Restore Modules</title>
+ <indexterm zone="archive-and-restore-modules">
+ <primary>Archive and Restore Modules</primary>
</indexterm>
<para>
@@ -14,45 +14,52 @@
performant.
</para>
- <para>
- When a custom <xref linkend="guc-archive-library"/> is configured, PostgreSQL
- will submit completed WAL files to the module, and the server will avoid
- recycling or removing these WAL files until the module indicates that the files
- were successfully archived. It is ultimately up to the module to decide what
- to do with each WAL file, but many recommendations are listed at
- <xref linkend="backup-archiving-wal"/>.
- </para>
-
- <para>
- Archiving modules must at least consist of an initialization function (see
- <xref linkend="archive-module-init"/>) and the required callbacks (see
- <xref linkend="archive-module-callbacks"/>). However, archive modules are
- also permitted to do much more (e.g., declare GUCs and register background
- workers).
- </para>
-
<para>
The <filename>contrib/basic_archive</filename> module contains a working
example, which demonstrates some useful techniques.
</para>
- <sect1 id="archive-module-init">
- <title>Initialization Functions</title>
- <indexterm zone="archive-module-init">
- <primary>_PG_archive_module_init</primary>
+ <sect1 id="archive-modules">
+ <title>Archive Modules</title>
+ <indexterm zone="archive-modules">
+ <primary>Archive Modules</primary>
</indexterm>
+
+ <para>
+ When a custom <xref linkend="guc-archive-library"/> is configured,
+ PostgreSQL will submit completed WAL files to the module, and the server
+ will avoid recycling or removing these WAL files until the module indicates
+ that the files were successfully archived. It is ultimately up to the
+ module to decide what to do with each WAL file, but many recommendations are
+ listed at <xref linkend="backup-archiving-wal"/>.
+ </para>
+
<para>
- An archive library is loaded by dynamically loading a shared library with the
- <xref linkend="guc-archive-library"/>'s name as the library base name. The
- normal library search path is used to locate the library. To provide the
- required archive module callbacks and to indicate that the library is
- actually an archive module, it needs to provide a function named
- <function>_PG_archive_module_init</function>. The result of the function
- must be a pointer to a struct of type
- <structname>ArchiveModuleCallbacks</structname>, which contains everything
- that the core code needs to know to make use of the archive module. The
- return value needs to be of server lifetime, which is typically achieved by
- defining it as a <literal>static const</literal> variable in global scope.
+ Archiving modules must at least consist of an initialization function (see
+ <xref linkend="archive-module-init"/>) and the required callbacks (see
+ <xref linkend="archive-module-callbacks"/>). However, archive modules are
+ also permitted to do much more (e.g., declare GUCs and register background
+ workers).
+ </para>
+
+ <sect2 id="archive-module-init">
+ <title>Initialization Functions</title>
+ <indexterm zone="archive-module-init">
+ <primary>_PG_archive_module_init</primary>
+ </indexterm>
+
+ <para>
+ An archive library is loaded by dynamically loading a shared library with
+ the <xref linkend="guc-archive-library"/>'s name as the library base name.
+ The normal library search path is used to locate the library. To provide
+ the required archive module callbacks and to indicate that the library is
+ actually an archive module, it needs to provide a function named
+ <function>_PG_archive_module_init</function>. The result of the function
+ must be a pointer to a struct of type
+ <structname>ArchiveModuleCallbacks</structname>, which contains everything
+ that the core code needs to know to make use of the archive module. The
+ return value needs to be of server lifetime, which is typically achieved by
+ defining it as a <literal>static const</literal> variable in global scope.
<programlisting>
typedef struct ArchiveModuleCallbacks
@@ -65,103 +72,110 @@ typedef struct ArchiveModuleCallbacks
typedef const ArchiveModuleCallbacks *(*ArchiveModuleInit) (void);
</programlisting>
- Only the <function>archive_file_cb</function> callback is required. The
- others are optional.
- </para>
- </sect1>
+ Only the <function>archive_file_cb</function> callback is required. The
+ others are optional.
+ </para>
+ </sect2>
- <sect1 id="archive-module-callbacks">
- <title>Archive Module Callbacks</title>
- <para>
- The archive callbacks define the actual archiving behavior of the module.
- The server will call them as required to process each individual WAL file.
- </para>
+ <sect2 id="archive-module-callbacks">
+ <title>Archive Module Callbacks</title>
- <sect2 id="archive-module-startup">
- <title>Startup Callback</title>
<para>
- The <function>startup_cb</function> callback is called shortly after the
- module is loaded. This callback can be used to perform any additional
- initialization required. If the archive module has any state, it can use
- <structfield>state->private_data</structfield> to store it.
+ The archive callbacks define the actual archiving behavior of the module.
+ The server will call them as required to process each individual WAL file.
+ </para>
+
+ <sect3 id="archive-module-startup">
+ <title>Startup Callback</title>
+
+ <para>
+ The <function>startup_cb</function> callback is called shortly after the
+ module is loaded. This callback can be used to perform any additional
+ initialization required. If the archive module has any state, it can use
+ <structfield>state->private_data</structfield> to store it.
<programlisting>
typedef void (*ArchiveStartupCB) (ArchiveModuleState *state);
</programlisting>
- </para>
- </sect2>
+ </para>
+ </sect3>
- <sect2 id="archive-module-check">
- <title>Check Callback</title>
- <para>
- The <function>check_configured_cb</function> callback is called to determine
- whether the module is fully configured and ready to accept WAL files (e.g.,
- its configuration parameters are set to valid values). If no
- <function>check_configured_cb</function> is defined, the server always
- assumes the module is configured.
+ <sect3 id="archive-module-check">
+ <title>Check Callback</title>
+
+ <para>
+ The <function>check_configured_cb</function> callback is called to
+ determine whether the module is fully configured and ready to accept WAL
+ files (e.g., its configuration parameters are set to valid values). If no
+ <function>check_configured_cb</function> is defined, the server always
+ assumes the module is configured.
<programlisting>
typedef bool (*ArchiveCheckConfiguredCB) (ArchiveModuleState *state);
</programlisting>
- If <literal>true</literal> is returned, the server will proceed with
- archiving the file by calling the <function>archive_file_cb</function>
- callback. If <literal>false</literal> is returned, archiving will not
- proceed, and the archiver will emit the following message to the server log:
+ If <literal>true</literal> is returned, the server will proceed with
+ archiving the file by calling the <function>archive_file_cb</function>
+ callback. If <literal>false</literal> is returned, archiving will not
+ proceed, and the archiver will emit the following message to the server
+ log:
<screen>
WARNING: archive_mode enabled, yet archiving is not configured
</screen>
- In the latter case, the server will periodically call this function, and
- archiving will proceed only when it returns <literal>true</literal>.
- </para>
-
- <note>
- <para>
- When returning <literal>false</literal>, it may be useful to append some
- additional information to the generic warning message. To do that, provide
- a message to the <function>arch_module_check_errdetail</function> macro
- before returning <literal>false</literal>. Like
- <function>errdetail()</function>, this macro accepts a format string
- followed by an optional list of arguments. The resulting string will be
- emitted as the <literal>DETAIL</literal> line of the warning message.
+ In the latter case, the server will periodically call this function, and
+ archiving will proceed only when it returns <literal>true</literal>.
</para>
- </note>
- </sect2>
- <sect2 id="archive-module-archive">
- <title>Archive Callback</title>
- <para>
- The <function>archive_file_cb</function> callback is called to archive a
- single WAL file.
+ <note>
+ <para>
+ When returning <literal>false</literal>, it may be useful to append some
+ additional information to the generic warning message. To do that,
+ provide a message to the <function>arch_module_check_errdetail</function>
+ macro before returning <literal>false</literal>. Like
+ <function>errdetail()</function>, this macro accepts a format string
+ followed by an optional list of arguments. The resulting string will be
+ emitted as the <literal>DETAIL</literal> line of the warning message.
+ </para>
+ </note>
+ </sect3>
+
+ <sect3 id="archive-module-archive">
+ <title>Archive Callback</title>
+
+ <para>
+ The <function>archive_file_cb</function> callback is called to archive a
+ single WAL file.
<programlisting>
typedef bool (*ArchiveFileCB) (ArchiveModuleState *state, const char *file, const char *path);
</programlisting>
- If <literal>true</literal> is returned, the server proceeds as if the file
- was successfully archived, which may include recycling or removing the
- original WAL file. If <literal>false</literal> is returned, the server will
- keep the original WAL file and retry archiving later.
- <replaceable>file</replaceable> will contain just the file name of the WAL
- file to archive, while <replaceable>path</replaceable> contains the full
- path of the WAL file (including the file name).
- </para>
- </sect2>
+ If <literal>true</literal> is returned, the server proceeds as if the file
+ was successfully archived, which may include recycling or removing the
+ original WAL file. If <literal>false</literal> is returned, the server
+ will keep the original WAL file and retry archiving later.
+ <replaceable>file</replaceable> will contain just the file name of the WAL
+ file to archive, while <replaceable>path</replaceable> contains the full
+ path of the WAL file (including the file name).
+ </para>
+ </sect3>
- <sect2 id="archive-module-shutdown">
- <title>Shutdown Callback</title>
- <para>
- The <function>shutdown_cb</function> callback is called when the archiver
- process exits (e.g., after an error) or the value of
- <xref linkend="guc-archive-library"/> changes. If no
- <function>shutdown_cb</function> is defined, no special action is taken in
- these situations. If the archive module has any state, this callback should
- free it to avoid leaks.
+ <sect3 id="archive-module-shutdown">
+ <title>Shutdown Callback</title>
+
+ <para>
+ The <function>shutdown_cb</function> callback is called when the archiver
+ process exits (e.g., after an error) or the value of
+ <xref linkend="guc-archive-library"/> changes. If no
+ <function>shutdown_cb</function> is defined, no special action is taken in
+ these situations. If the archive module has any state, this callback
+ should free it to avoid leaks.
<programlisting>
typedef void (*ArchiveShutdownCB) (ArchiveModuleState *state);
</programlisting>
- </para>
+ </para>
+ </sect3>
</sect2>
</sect1>
</chapter>
--
2.25.1
--IJpNTDwzlM2Ie8A6
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v18-0005-introduce-restore_library.patch"
^ permalink raw reply [nested|flat] 24+ messages in thread
* [PATCH v19 4/5] restructure archive modules docs in preparation for restore modules
@ 2023-02-15 22:48 Nathan Bossart <[email protected]>
0 siblings, 0 replies; 24+ messages in thread
From: Nathan Bossart @ 2023-02-15 22:48 UTC (permalink / raw)
---
doc/src/sgml/archive-and-restore-modules.sgml | 226 ++++++++++--------
1 file changed, 120 insertions(+), 106 deletions(-)
diff --git a/doc/src/sgml/archive-and-restore-modules.sgml b/doc/src/sgml/archive-and-restore-modules.sgml
index cf7438a759..a52d15082d 100644
--- a/doc/src/sgml/archive-and-restore-modules.sgml
+++ b/doc/src/sgml/archive-and-restore-modules.sgml
@@ -1,9 +1,9 @@
-<!-- doc/src/sgml/archive-modules.sgml -->
+<!-- doc/src/sgml/archive-and-restore-modules.sgml -->
-<chapter id="archive-modules">
- <title>Archive Modules</title>
- <indexterm zone="archive-modules">
- <primary>Archive Modules</primary>
+<chapter id="archive-and-restore-modules">
+ <title>Archive and Restore Modules</title>
+ <indexterm zone="archive-and-restore-modules">
+ <primary>Archive and Restore Modules</primary>
</indexterm>
<para>
@@ -14,45 +14,52 @@
performant.
</para>
- <para>
- When a custom <xref linkend="guc-archive-library"/> is configured, PostgreSQL
- will submit completed WAL files to the module, and the server will avoid
- recycling or removing these WAL files until the module indicates that the files
- were successfully archived. It is ultimately up to the module to decide what
- to do with each WAL file, but many recommendations are listed at
- <xref linkend="backup-archiving-wal"/>.
- </para>
-
- <para>
- Archiving modules must at least consist of an initialization function (see
- <xref linkend="archive-module-init"/>) and the required callbacks (see
- <xref linkend="archive-module-callbacks"/>). However, archive modules are
- also permitted to do much more (e.g., declare GUCs and register background
- workers).
- </para>
-
<para>
The <filename>contrib/basic_archive</filename> module contains a working
example, which demonstrates some useful techniques.
</para>
- <sect1 id="archive-module-init">
- <title>Initialization Functions</title>
- <indexterm zone="archive-module-init">
- <primary>_PG_archive_module_init</primary>
+ <sect1 id="archive-modules">
+ <title>Archive Modules</title>
+ <indexterm zone="archive-modules">
+ <primary>Archive Modules</primary>
</indexterm>
+
+ <para>
+ When a custom <xref linkend="guc-archive-library"/> is configured,
+ PostgreSQL will submit completed WAL files to the module, and the server
+ will avoid recycling or removing these WAL files until the module indicates
+ that the files were successfully archived. It is ultimately up to the
+ module to decide what to do with each WAL file, but many recommendations are
+ listed at <xref linkend="backup-archiving-wal"/>.
+ </para>
+
<para>
- An archive library is loaded by dynamically loading a shared library with the
- <xref linkend="guc-archive-library"/>'s name as the library base name. The
- normal library search path is used to locate the library. To provide the
- required archive module callbacks and to indicate that the library is
- actually an archive module, it needs to provide a function named
- <function>_PG_archive_module_init</function>. The result of the function
- must be a pointer to a struct of type
- <structname>ArchiveModuleCallbacks</structname>, which contains everything
- that the core code needs to know to make use of the archive module. The
- return value needs to be of server lifetime, which is typically achieved by
- defining it as a <literal>static const</literal> variable in global scope.
+ Archiving modules must at least consist of an initialization function (see
+ <xref linkend="archive-module-init"/>) and the required callbacks (see
+ <xref linkend="archive-module-callbacks"/>). However, archive modules are
+ also permitted to do much more (e.g., declare GUCs and register background
+ workers).
+ </para>
+
+ <sect2 id="archive-module-init">
+ <title>Initialization Functions</title>
+ <indexterm zone="archive-module-init">
+ <primary>_PG_archive_module_init</primary>
+ </indexterm>
+
+ <para>
+ An archive library is loaded by dynamically loading a shared library with
+ the <xref linkend="guc-archive-library"/>'s name as the library base name.
+ The normal library search path is used to locate the library. To provide
+ the required archive module callbacks and to indicate that the library is
+ actually an archive module, it needs to provide a function named
+ <function>_PG_archive_module_init</function>. The result of the function
+ must be a pointer to a struct of type
+ <structname>ArchiveModuleCallbacks</structname>, which contains everything
+ that the core code needs to know to make use of the archive module. The
+ return value needs to be of server lifetime, which is typically achieved by
+ defining it as a <literal>static const</literal> variable in global scope.
<programlisting>
typedef struct ArchiveModuleCallbacks
@@ -65,103 +72,110 @@ typedef struct ArchiveModuleCallbacks
typedef const ArchiveModuleCallbacks *(*ArchiveModuleInit) (void);
</programlisting>
- Only the <function>archive_file_cb</function> callback is required. The
- others are optional.
- </para>
- </sect1>
+ Only the <function>archive_file_cb</function> callback is required. The
+ others are optional.
+ </para>
+ </sect2>
- <sect1 id="archive-module-callbacks">
- <title>Archive Module Callbacks</title>
- <para>
- The archive callbacks define the actual archiving behavior of the module.
- The server will call them as required to process each individual WAL file.
- </para>
+ <sect2 id="archive-module-callbacks">
+ <title>Archive Module Callbacks</title>
- <sect2 id="archive-module-startup">
- <title>Startup Callback</title>
<para>
- The <function>startup_cb</function> callback is called shortly after the
- module is loaded. This callback can be used to perform any additional
- initialization required. If the archive module has any state, it can use
- <structfield>state->private_data</structfield> to store it.
+ The archive callbacks define the actual archiving behavior of the module.
+ The server will call them as required to process each individual WAL file.
+ </para>
+
+ <sect3 id="archive-module-startup">
+ <title>Startup Callback</title>
+
+ <para>
+ The <function>startup_cb</function> callback is called shortly after the
+ module is loaded. This callback can be used to perform any additional
+ initialization required. If the archive module has any state, it can use
+ <structfield>state->private_data</structfield> to store it.
<programlisting>
typedef void (*ArchiveStartupCB) (ArchiveModuleState *state);
</programlisting>
- </para>
- </sect2>
+ </para>
+ </sect3>
- <sect2 id="archive-module-check">
- <title>Check Callback</title>
- <para>
- The <function>check_configured_cb</function> callback is called to determine
- whether the module is fully configured and ready to accept WAL files (e.g.,
- its configuration parameters are set to valid values). If no
- <function>check_configured_cb</function> is defined, the server always
- assumes the module is configured.
+ <sect3 id="archive-module-check">
+ <title>Check Callback</title>
+
+ <para>
+ The <function>check_configured_cb</function> callback is called to
+ determine whether the module is fully configured and ready to accept WAL
+ files (e.g., its configuration parameters are set to valid values). If no
+ <function>check_configured_cb</function> is defined, the server always
+ assumes the module is configured.
<programlisting>
typedef bool (*ArchiveCheckConfiguredCB) (ArchiveModuleState *state);
</programlisting>
- If <literal>true</literal> is returned, the server will proceed with
- archiving the file by calling the <function>archive_file_cb</function>
- callback. If <literal>false</literal> is returned, archiving will not
- proceed, and the archiver will emit the following message to the server log:
+ If <literal>true</literal> is returned, the server will proceed with
+ archiving the file by calling the <function>archive_file_cb</function>
+ callback. If <literal>false</literal> is returned, archiving will not
+ proceed, and the archiver will emit the following message to the server
+ log:
<screen>
WARNING: archive_mode enabled, yet archiving is not configured
</screen>
- In the latter case, the server will periodically call this function, and
- archiving will proceed only when it returns <literal>true</literal>.
- </para>
-
- <note>
- <para>
- When returning <literal>false</literal>, it may be useful to append some
- additional information to the generic warning message. To do that, provide
- a message to the <function>arch_module_check_errdetail</function> macro
- before returning <literal>false</literal>. Like
- <function>errdetail()</function>, this macro accepts a format string
- followed by an optional list of arguments. The resulting string will be
- emitted as the <literal>DETAIL</literal> line of the warning message.
+ In the latter case, the server will periodically call this function, and
+ archiving will proceed only when it returns <literal>true</literal>.
</para>
- </note>
- </sect2>
- <sect2 id="archive-module-archive">
- <title>Archive Callback</title>
- <para>
- The <function>archive_file_cb</function> callback is called to archive a
- single WAL file.
+ <note>
+ <para>
+ When returning <literal>false</literal>, it may be useful to append some
+ additional information to the generic warning message. To do that,
+ provide a message to the <function>arch_module_check_errdetail</function>
+ macro before returning <literal>false</literal>. Like
+ <function>errdetail()</function>, this macro accepts a format string
+ followed by an optional list of arguments. The resulting string will be
+ emitted as the <literal>DETAIL</literal> line of the warning message.
+ </para>
+ </note>
+ </sect3>
+
+ <sect3 id="archive-module-archive">
+ <title>Archive Callback</title>
+
+ <para>
+ The <function>archive_file_cb</function> callback is called to archive a
+ single WAL file.
<programlisting>
typedef bool (*ArchiveFileCB) (ArchiveModuleState *state, const char *file, const char *path);
</programlisting>
- If <literal>true</literal> is returned, the server proceeds as if the file
- was successfully archived, which may include recycling or removing the
- original WAL file. If <literal>false</literal> is returned, the server will
- keep the original WAL file and retry archiving later.
- <replaceable>file</replaceable> will contain just the file name of the WAL
- file to archive, while <replaceable>path</replaceable> contains the full
- path of the WAL file (including the file name).
- </para>
- </sect2>
+ If <literal>true</literal> is returned, the server proceeds as if the file
+ was successfully archived, which may include recycling or removing the
+ original WAL file. If <literal>false</literal> is returned, the server
+ will keep the original WAL file and retry archiving later.
+ <replaceable>file</replaceable> will contain just the file name of the WAL
+ file to archive, while <replaceable>path</replaceable> contains the full
+ path of the WAL file (including the file name).
+ </para>
+ </sect3>
- <sect2 id="archive-module-shutdown">
- <title>Shutdown Callback</title>
- <para>
- The <function>shutdown_cb</function> callback is called when the archiver
- process exits (e.g., after an error) or the value of
- <xref linkend="guc-archive-library"/> changes. If no
- <function>shutdown_cb</function> is defined, no special action is taken in
- these situations. If the archive module has any state, this callback should
- free it to avoid leaks.
+ <sect3 id="archive-module-shutdown">
+ <title>Shutdown Callback</title>
+
+ <para>
+ The <function>shutdown_cb</function> callback is called when the archiver
+ process exits (e.g., after an error) or the value of
+ <xref linkend="guc-archive-library"/> changes. If no
+ <function>shutdown_cb</function> is defined, no special action is taken in
+ these situations. If the archive module has any state, this callback
+ should free it to avoid leaks.
<programlisting>
typedef void (*ArchiveShutdownCB) (ArchiveModuleState *state);
</programlisting>
- </para>
+ </para>
+ </sect3>
</sect2>
</sect1>
</chapter>
--
2.25.1
--oyUTqETQ0mS9luUI
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v19-0005-introduce-restore_library.patch"
^ permalink raw reply [nested|flat] 24+ messages in thread
* [PATCH v18 4/5] restructure archive modules docs in preparation for restore modules
@ 2023-02-15 22:48 Nathan Bossart <[email protected]>
0 siblings, 0 replies; 24+ messages in thread
From: Nathan Bossart @ 2023-02-15 22:48 UTC (permalink / raw)
---
doc/src/sgml/archive-and-restore-modules.sgml | 226 ++++++++++--------
1 file changed, 120 insertions(+), 106 deletions(-)
diff --git a/doc/src/sgml/archive-and-restore-modules.sgml b/doc/src/sgml/archive-and-restore-modules.sgml
index cf7438a759..a52d15082d 100644
--- a/doc/src/sgml/archive-and-restore-modules.sgml
+++ b/doc/src/sgml/archive-and-restore-modules.sgml
@@ -1,9 +1,9 @@
-<!-- doc/src/sgml/archive-modules.sgml -->
+<!-- doc/src/sgml/archive-and-restore-modules.sgml -->
-<chapter id="archive-modules">
- <title>Archive Modules</title>
- <indexterm zone="archive-modules">
- <primary>Archive Modules</primary>
+<chapter id="archive-and-restore-modules">
+ <title>Archive and Restore Modules</title>
+ <indexterm zone="archive-and-restore-modules">
+ <primary>Archive and Restore Modules</primary>
</indexterm>
<para>
@@ -14,45 +14,52 @@
performant.
</para>
- <para>
- When a custom <xref linkend="guc-archive-library"/> is configured, PostgreSQL
- will submit completed WAL files to the module, and the server will avoid
- recycling or removing these WAL files until the module indicates that the files
- were successfully archived. It is ultimately up to the module to decide what
- to do with each WAL file, but many recommendations are listed at
- <xref linkend="backup-archiving-wal"/>.
- </para>
-
- <para>
- Archiving modules must at least consist of an initialization function (see
- <xref linkend="archive-module-init"/>) and the required callbacks (see
- <xref linkend="archive-module-callbacks"/>). However, archive modules are
- also permitted to do much more (e.g., declare GUCs and register background
- workers).
- </para>
-
<para>
The <filename>contrib/basic_archive</filename> module contains a working
example, which demonstrates some useful techniques.
</para>
- <sect1 id="archive-module-init">
- <title>Initialization Functions</title>
- <indexterm zone="archive-module-init">
- <primary>_PG_archive_module_init</primary>
+ <sect1 id="archive-modules">
+ <title>Archive Modules</title>
+ <indexterm zone="archive-modules">
+ <primary>Archive Modules</primary>
</indexterm>
+
+ <para>
+ When a custom <xref linkend="guc-archive-library"/> is configured,
+ PostgreSQL will submit completed WAL files to the module, and the server
+ will avoid recycling or removing these WAL files until the module indicates
+ that the files were successfully archived. It is ultimately up to the
+ module to decide what to do with each WAL file, but many recommendations are
+ listed at <xref linkend="backup-archiving-wal"/>.
+ </para>
+
<para>
- An archive library is loaded by dynamically loading a shared library with the
- <xref linkend="guc-archive-library"/>'s name as the library base name. The
- normal library search path is used to locate the library. To provide the
- required archive module callbacks and to indicate that the library is
- actually an archive module, it needs to provide a function named
- <function>_PG_archive_module_init</function>. The result of the function
- must be a pointer to a struct of type
- <structname>ArchiveModuleCallbacks</structname>, which contains everything
- that the core code needs to know to make use of the archive module. The
- return value needs to be of server lifetime, which is typically achieved by
- defining it as a <literal>static const</literal> variable in global scope.
+ Archiving modules must at least consist of an initialization function (see
+ <xref linkend="archive-module-init"/>) and the required callbacks (see
+ <xref linkend="archive-module-callbacks"/>). However, archive modules are
+ also permitted to do much more (e.g., declare GUCs and register background
+ workers).
+ </para>
+
+ <sect2 id="archive-module-init">
+ <title>Initialization Functions</title>
+ <indexterm zone="archive-module-init">
+ <primary>_PG_archive_module_init</primary>
+ </indexterm>
+
+ <para>
+ An archive library is loaded by dynamically loading a shared library with
+ the <xref linkend="guc-archive-library"/>'s name as the library base name.
+ The normal library search path is used to locate the library. To provide
+ the required archive module callbacks and to indicate that the library is
+ actually an archive module, it needs to provide a function named
+ <function>_PG_archive_module_init</function>. The result of the function
+ must be a pointer to a struct of type
+ <structname>ArchiveModuleCallbacks</structname>, which contains everything
+ that the core code needs to know to make use of the archive module. The
+ return value needs to be of server lifetime, which is typically achieved by
+ defining it as a <literal>static const</literal> variable in global scope.
<programlisting>
typedef struct ArchiveModuleCallbacks
@@ -65,103 +72,110 @@ typedef struct ArchiveModuleCallbacks
typedef const ArchiveModuleCallbacks *(*ArchiveModuleInit) (void);
</programlisting>
- Only the <function>archive_file_cb</function> callback is required. The
- others are optional.
- </para>
- </sect1>
+ Only the <function>archive_file_cb</function> callback is required. The
+ others are optional.
+ </para>
+ </sect2>
- <sect1 id="archive-module-callbacks">
- <title>Archive Module Callbacks</title>
- <para>
- The archive callbacks define the actual archiving behavior of the module.
- The server will call them as required to process each individual WAL file.
- </para>
+ <sect2 id="archive-module-callbacks">
+ <title>Archive Module Callbacks</title>
- <sect2 id="archive-module-startup">
- <title>Startup Callback</title>
<para>
- The <function>startup_cb</function> callback is called shortly after the
- module is loaded. This callback can be used to perform any additional
- initialization required. If the archive module has any state, it can use
- <structfield>state->private_data</structfield> to store it.
+ The archive callbacks define the actual archiving behavior of the module.
+ The server will call them as required to process each individual WAL file.
+ </para>
+
+ <sect3 id="archive-module-startup">
+ <title>Startup Callback</title>
+
+ <para>
+ The <function>startup_cb</function> callback is called shortly after the
+ module is loaded. This callback can be used to perform any additional
+ initialization required. If the archive module has any state, it can use
+ <structfield>state->private_data</structfield> to store it.
<programlisting>
typedef void (*ArchiveStartupCB) (ArchiveModuleState *state);
</programlisting>
- </para>
- </sect2>
+ </para>
+ </sect3>
- <sect2 id="archive-module-check">
- <title>Check Callback</title>
- <para>
- The <function>check_configured_cb</function> callback is called to determine
- whether the module is fully configured and ready to accept WAL files (e.g.,
- its configuration parameters are set to valid values). If no
- <function>check_configured_cb</function> is defined, the server always
- assumes the module is configured.
+ <sect3 id="archive-module-check">
+ <title>Check Callback</title>
+
+ <para>
+ The <function>check_configured_cb</function> callback is called to
+ determine whether the module is fully configured and ready to accept WAL
+ files (e.g., its configuration parameters are set to valid values). If no
+ <function>check_configured_cb</function> is defined, the server always
+ assumes the module is configured.
<programlisting>
typedef bool (*ArchiveCheckConfiguredCB) (ArchiveModuleState *state);
</programlisting>
- If <literal>true</literal> is returned, the server will proceed with
- archiving the file by calling the <function>archive_file_cb</function>
- callback. If <literal>false</literal> is returned, archiving will not
- proceed, and the archiver will emit the following message to the server log:
+ If <literal>true</literal> is returned, the server will proceed with
+ archiving the file by calling the <function>archive_file_cb</function>
+ callback. If <literal>false</literal> is returned, archiving will not
+ proceed, and the archiver will emit the following message to the server
+ log:
<screen>
WARNING: archive_mode enabled, yet archiving is not configured
</screen>
- In the latter case, the server will periodically call this function, and
- archiving will proceed only when it returns <literal>true</literal>.
- </para>
-
- <note>
- <para>
- When returning <literal>false</literal>, it may be useful to append some
- additional information to the generic warning message. To do that, provide
- a message to the <function>arch_module_check_errdetail</function> macro
- before returning <literal>false</literal>. Like
- <function>errdetail()</function>, this macro accepts a format string
- followed by an optional list of arguments. The resulting string will be
- emitted as the <literal>DETAIL</literal> line of the warning message.
+ In the latter case, the server will periodically call this function, and
+ archiving will proceed only when it returns <literal>true</literal>.
</para>
- </note>
- </sect2>
- <sect2 id="archive-module-archive">
- <title>Archive Callback</title>
- <para>
- The <function>archive_file_cb</function> callback is called to archive a
- single WAL file.
+ <note>
+ <para>
+ When returning <literal>false</literal>, it may be useful to append some
+ additional information to the generic warning message. To do that,
+ provide a message to the <function>arch_module_check_errdetail</function>
+ macro before returning <literal>false</literal>. Like
+ <function>errdetail()</function>, this macro accepts a format string
+ followed by an optional list of arguments. The resulting string will be
+ emitted as the <literal>DETAIL</literal> line of the warning message.
+ </para>
+ </note>
+ </sect3>
+
+ <sect3 id="archive-module-archive">
+ <title>Archive Callback</title>
+
+ <para>
+ The <function>archive_file_cb</function> callback is called to archive a
+ single WAL file.
<programlisting>
typedef bool (*ArchiveFileCB) (ArchiveModuleState *state, const char *file, const char *path);
</programlisting>
- If <literal>true</literal> is returned, the server proceeds as if the file
- was successfully archived, which may include recycling or removing the
- original WAL file. If <literal>false</literal> is returned, the server will
- keep the original WAL file and retry archiving later.
- <replaceable>file</replaceable> will contain just the file name of the WAL
- file to archive, while <replaceable>path</replaceable> contains the full
- path of the WAL file (including the file name).
- </para>
- </sect2>
+ If <literal>true</literal> is returned, the server proceeds as if the file
+ was successfully archived, which may include recycling or removing the
+ original WAL file. If <literal>false</literal> is returned, the server
+ will keep the original WAL file and retry archiving later.
+ <replaceable>file</replaceable> will contain just the file name of the WAL
+ file to archive, while <replaceable>path</replaceable> contains the full
+ path of the WAL file (including the file name).
+ </para>
+ </sect3>
- <sect2 id="archive-module-shutdown">
- <title>Shutdown Callback</title>
- <para>
- The <function>shutdown_cb</function> callback is called when the archiver
- process exits (e.g., after an error) or the value of
- <xref linkend="guc-archive-library"/> changes. If no
- <function>shutdown_cb</function> is defined, no special action is taken in
- these situations. If the archive module has any state, this callback should
- free it to avoid leaks.
+ <sect3 id="archive-module-shutdown">
+ <title>Shutdown Callback</title>
+
+ <para>
+ The <function>shutdown_cb</function> callback is called when the archiver
+ process exits (e.g., after an error) or the value of
+ <xref linkend="guc-archive-library"/> changes. If no
+ <function>shutdown_cb</function> is defined, no special action is taken in
+ these situations. If the archive module has any state, this callback
+ should free it to avoid leaks.
<programlisting>
typedef void (*ArchiveShutdownCB) (ArchiveModuleState *state);
</programlisting>
- </para>
+ </para>
+ </sect3>
</sect2>
</sect1>
</chapter>
--
2.25.1
--IJpNTDwzlM2Ie8A6
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v18-0005-introduce-restore_library.patch"
^ permalink raw reply [nested|flat] 24+ messages in thread
* [PATCH v19 4/5] restructure archive modules docs in preparation for restore modules
@ 2023-02-15 22:48 Nathan Bossart <[email protected]>
0 siblings, 0 replies; 24+ messages in thread
From: Nathan Bossart @ 2023-02-15 22:48 UTC (permalink / raw)
---
doc/src/sgml/archive-and-restore-modules.sgml | 226 ++++++++++--------
1 file changed, 120 insertions(+), 106 deletions(-)
diff --git a/doc/src/sgml/archive-and-restore-modules.sgml b/doc/src/sgml/archive-and-restore-modules.sgml
index cf7438a759..a52d15082d 100644
--- a/doc/src/sgml/archive-and-restore-modules.sgml
+++ b/doc/src/sgml/archive-and-restore-modules.sgml
@@ -1,9 +1,9 @@
-<!-- doc/src/sgml/archive-modules.sgml -->
+<!-- doc/src/sgml/archive-and-restore-modules.sgml -->
-<chapter id="archive-modules">
- <title>Archive Modules</title>
- <indexterm zone="archive-modules">
- <primary>Archive Modules</primary>
+<chapter id="archive-and-restore-modules">
+ <title>Archive and Restore Modules</title>
+ <indexterm zone="archive-and-restore-modules">
+ <primary>Archive and Restore Modules</primary>
</indexterm>
<para>
@@ -14,45 +14,52 @@
performant.
</para>
- <para>
- When a custom <xref linkend="guc-archive-library"/> is configured, PostgreSQL
- will submit completed WAL files to the module, and the server will avoid
- recycling or removing these WAL files until the module indicates that the files
- were successfully archived. It is ultimately up to the module to decide what
- to do with each WAL file, but many recommendations are listed at
- <xref linkend="backup-archiving-wal"/>.
- </para>
-
- <para>
- Archiving modules must at least consist of an initialization function (see
- <xref linkend="archive-module-init"/>) and the required callbacks (see
- <xref linkend="archive-module-callbacks"/>). However, archive modules are
- also permitted to do much more (e.g., declare GUCs and register background
- workers).
- </para>
-
<para>
The <filename>contrib/basic_archive</filename> module contains a working
example, which demonstrates some useful techniques.
</para>
- <sect1 id="archive-module-init">
- <title>Initialization Functions</title>
- <indexterm zone="archive-module-init">
- <primary>_PG_archive_module_init</primary>
+ <sect1 id="archive-modules">
+ <title>Archive Modules</title>
+ <indexterm zone="archive-modules">
+ <primary>Archive Modules</primary>
</indexterm>
+
+ <para>
+ When a custom <xref linkend="guc-archive-library"/> is configured,
+ PostgreSQL will submit completed WAL files to the module, and the server
+ will avoid recycling or removing these WAL files until the module indicates
+ that the files were successfully archived. It is ultimately up to the
+ module to decide what to do with each WAL file, but many recommendations are
+ listed at <xref linkend="backup-archiving-wal"/>.
+ </para>
+
<para>
- An archive library is loaded by dynamically loading a shared library with the
- <xref linkend="guc-archive-library"/>'s name as the library base name. The
- normal library search path is used to locate the library. To provide the
- required archive module callbacks and to indicate that the library is
- actually an archive module, it needs to provide a function named
- <function>_PG_archive_module_init</function>. The result of the function
- must be a pointer to a struct of type
- <structname>ArchiveModuleCallbacks</structname>, which contains everything
- that the core code needs to know to make use of the archive module. The
- return value needs to be of server lifetime, which is typically achieved by
- defining it as a <literal>static const</literal> variable in global scope.
+ Archiving modules must at least consist of an initialization function (see
+ <xref linkend="archive-module-init"/>) and the required callbacks (see
+ <xref linkend="archive-module-callbacks"/>). However, archive modules are
+ also permitted to do much more (e.g., declare GUCs and register background
+ workers).
+ </para>
+
+ <sect2 id="archive-module-init">
+ <title>Initialization Functions</title>
+ <indexterm zone="archive-module-init">
+ <primary>_PG_archive_module_init</primary>
+ </indexterm>
+
+ <para>
+ An archive library is loaded by dynamically loading a shared library with
+ the <xref linkend="guc-archive-library"/>'s name as the library base name.
+ The normal library search path is used to locate the library. To provide
+ the required archive module callbacks and to indicate that the library is
+ actually an archive module, it needs to provide a function named
+ <function>_PG_archive_module_init</function>. The result of the function
+ must be a pointer to a struct of type
+ <structname>ArchiveModuleCallbacks</structname>, which contains everything
+ that the core code needs to know to make use of the archive module. The
+ return value needs to be of server lifetime, which is typically achieved by
+ defining it as a <literal>static const</literal> variable in global scope.
<programlisting>
typedef struct ArchiveModuleCallbacks
@@ -65,103 +72,110 @@ typedef struct ArchiveModuleCallbacks
typedef const ArchiveModuleCallbacks *(*ArchiveModuleInit) (void);
</programlisting>
- Only the <function>archive_file_cb</function> callback is required. The
- others are optional.
- </para>
- </sect1>
+ Only the <function>archive_file_cb</function> callback is required. The
+ others are optional.
+ </para>
+ </sect2>
- <sect1 id="archive-module-callbacks">
- <title>Archive Module Callbacks</title>
- <para>
- The archive callbacks define the actual archiving behavior of the module.
- The server will call them as required to process each individual WAL file.
- </para>
+ <sect2 id="archive-module-callbacks">
+ <title>Archive Module Callbacks</title>
- <sect2 id="archive-module-startup">
- <title>Startup Callback</title>
<para>
- The <function>startup_cb</function> callback is called shortly after the
- module is loaded. This callback can be used to perform any additional
- initialization required. If the archive module has any state, it can use
- <structfield>state->private_data</structfield> to store it.
+ The archive callbacks define the actual archiving behavior of the module.
+ The server will call them as required to process each individual WAL file.
+ </para>
+
+ <sect3 id="archive-module-startup">
+ <title>Startup Callback</title>
+
+ <para>
+ The <function>startup_cb</function> callback is called shortly after the
+ module is loaded. This callback can be used to perform any additional
+ initialization required. If the archive module has any state, it can use
+ <structfield>state->private_data</structfield> to store it.
<programlisting>
typedef void (*ArchiveStartupCB) (ArchiveModuleState *state);
</programlisting>
- </para>
- </sect2>
+ </para>
+ </sect3>
- <sect2 id="archive-module-check">
- <title>Check Callback</title>
- <para>
- The <function>check_configured_cb</function> callback is called to determine
- whether the module is fully configured and ready to accept WAL files (e.g.,
- its configuration parameters are set to valid values). If no
- <function>check_configured_cb</function> is defined, the server always
- assumes the module is configured.
+ <sect3 id="archive-module-check">
+ <title>Check Callback</title>
+
+ <para>
+ The <function>check_configured_cb</function> callback is called to
+ determine whether the module is fully configured and ready to accept WAL
+ files (e.g., its configuration parameters are set to valid values). If no
+ <function>check_configured_cb</function> is defined, the server always
+ assumes the module is configured.
<programlisting>
typedef bool (*ArchiveCheckConfiguredCB) (ArchiveModuleState *state);
</programlisting>
- If <literal>true</literal> is returned, the server will proceed with
- archiving the file by calling the <function>archive_file_cb</function>
- callback. If <literal>false</literal> is returned, archiving will not
- proceed, and the archiver will emit the following message to the server log:
+ If <literal>true</literal> is returned, the server will proceed with
+ archiving the file by calling the <function>archive_file_cb</function>
+ callback. If <literal>false</literal> is returned, archiving will not
+ proceed, and the archiver will emit the following message to the server
+ log:
<screen>
WARNING: archive_mode enabled, yet archiving is not configured
</screen>
- In the latter case, the server will periodically call this function, and
- archiving will proceed only when it returns <literal>true</literal>.
- </para>
-
- <note>
- <para>
- When returning <literal>false</literal>, it may be useful to append some
- additional information to the generic warning message. To do that, provide
- a message to the <function>arch_module_check_errdetail</function> macro
- before returning <literal>false</literal>. Like
- <function>errdetail()</function>, this macro accepts a format string
- followed by an optional list of arguments. The resulting string will be
- emitted as the <literal>DETAIL</literal> line of the warning message.
+ In the latter case, the server will periodically call this function, and
+ archiving will proceed only when it returns <literal>true</literal>.
</para>
- </note>
- </sect2>
- <sect2 id="archive-module-archive">
- <title>Archive Callback</title>
- <para>
- The <function>archive_file_cb</function> callback is called to archive a
- single WAL file.
+ <note>
+ <para>
+ When returning <literal>false</literal>, it may be useful to append some
+ additional information to the generic warning message. To do that,
+ provide a message to the <function>arch_module_check_errdetail</function>
+ macro before returning <literal>false</literal>. Like
+ <function>errdetail()</function>, this macro accepts a format string
+ followed by an optional list of arguments. The resulting string will be
+ emitted as the <literal>DETAIL</literal> line of the warning message.
+ </para>
+ </note>
+ </sect3>
+
+ <sect3 id="archive-module-archive">
+ <title>Archive Callback</title>
+
+ <para>
+ The <function>archive_file_cb</function> callback is called to archive a
+ single WAL file.
<programlisting>
typedef bool (*ArchiveFileCB) (ArchiveModuleState *state, const char *file, const char *path);
</programlisting>
- If <literal>true</literal> is returned, the server proceeds as if the file
- was successfully archived, which may include recycling or removing the
- original WAL file. If <literal>false</literal> is returned, the server will
- keep the original WAL file and retry archiving later.
- <replaceable>file</replaceable> will contain just the file name of the WAL
- file to archive, while <replaceable>path</replaceable> contains the full
- path of the WAL file (including the file name).
- </para>
- </sect2>
+ If <literal>true</literal> is returned, the server proceeds as if the file
+ was successfully archived, which may include recycling or removing the
+ original WAL file. If <literal>false</literal> is returned, the server
+ will keep the original WAL file and retry archiving later.
+ <replaceable>file</replaceable> will contain just the file name of the WAL
+ file to archive, while <replaceable>path</replaceable> contains the full
+ path of the WAL file (including the file name).
+ </para>
+ </sect3>
- <sect2 id="archive-module-shutdown">
- <title>Shutdown Callback</title>
- <para>
- The <function>shutdown_cb</function> callback is called when the archiver
- process exits (e.g., after an error) or the value of
- <xref linkend="guc-archive-library"/> changes. If no
- <function>shutdown_cb</function> is defined, no special action is taken in
- these situations. If the archive module has any state, this callback should
- free it to avoid leaks.
+ <sect3 id="archive-module-shutdown">
+ <title>Shutdown Callback</title>
+
+ <para>
+ The <function>shutdown_cb</function> callback is called when the archiver
+ process exits (e.g., after an error) or the value of
+ <xref linkend="guc-archive-library"/> changes. If no
+ <function>shutdown_cb</function> is defined, no special action is taken in
+ these situations. If the archive module has any state, this callback
+ should free it to avoid leaks.
<programlisting>
typedef void (*ArchiveShutdownCB) (ArchiveModuleState *state);
</programlisting>
- </para>
+ </para>
+ </sect3>
</sect2>
</sect1>
</chapter>
--
2.25.1
--oyUTqETQ0mS9luUI
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v19-0005-introduce-restore_library.patch"
^ permalink raw reply [nested|flat] 24+ messages in thread
* [PATCH v16 4/5] restructure archive modules docs in preparation for restore modules
@ 2023-02-15 22:48 Nathan Bossart <[email protected]>
0 siblings, 0 replies; 24+ messages in thread
From: Nathan Bossart @ 2023-02-15 22:48 UTC (permalink / raw)
---
doc/src/sgml/archive-and-restore-modules.sgml | 206 ++++++++++--------
1 file changed, 110 insertions(+), 96 deletions(-)
diff --git a/doc/src/sgml/archive-and-restore-modules.sgml b/doc/src/sgml/archive-and-restore-modules.sgml
index 7064307d9e..7ed50a3b52 100644
--- a/doc/src/sgml/archive-and-restore-modules.sgml
+++ b/doc/src/sgml/archive-and-restore-modules.sgml
@@ -1,9 +1,9 @@
-<!-- doc/src/sgml/archive-modules.sgml -->
+<!-- doc/src/sgml/archive-and-restore-modules.sgml -->
-<chapter id="archive-modules">
- <title>Archive Modules</title>
- <indexterm zone="archive-modules">
- <primary>Archive Modules</primary>
+<chapter id="archive-and-restore-modules">
+ <title>Archive and Restore Modules</title>
+ <indexterm zone="archive-and-restore-modules">
+ <primary>Archive and Restore Modules</primary>
</indexterm>
<para>
@@ -14,45 +14,52 @@
performant.
</para>
- <para>
- When a custom <xref linkend="guc-archive-library"/> is configured, PostgreSQL
- will submit completed WAL files to the module, and the server will avoid
- recycling or removing these WAL files until the module indicates that the files
- were successfully archived. It is ultimately up to the module to decide what
- to do with each WAL file, but many recommendations are listed at
- <xref linkend="backup-archiving-wal"/>.
- </para>
-
- <para>
- Archiving modules must at least consist of an initialization function (see
- <xref linkend="archive-module-init"/>) and the required callbacks (see
- <xref linkend="archive-module-callbacks"/>). However, archive modules are
- also permitted to do much more (e.g., declare GUCs and register background
- workers).
- </para>
-
<para>
The <filename>contrib/basic_archive</filename> module contains a working
example, which demonstrates some useful techniques.
</para>
- <sect1 id="archive-module-init">
- <title>Initialization Functions</title>
- <indexterm zone="archive-module-init">
- <primary>_PG_archive_module_init</primary>
+ <sect1 id="archive-modules">
+ <title>Archive Modules</title>
+ <indexterm zone="archive-modules">
+ <primary>Archive Modules</primary>
</indexterm>
+
+ <para>
+ When a custom <xref linkend="guc-archive-library"/> is configured,
+ PostgreSQL will submit completed WAL files to the module, and the server
+ will avoid recycling or removing these WAL files until the module indicates
+ that the files were successfully archived. It is ultimately up to the
+ module to decide what to do with each WAL file, but many recommendations are
+ listed at <xref linkend="backup-archiving-wal"/>.
+ </para>
+
<para>
- An archive library is loaded by dynamically loading a shared library with the
- <xref linkend="guc-archive-library"/>'s name as the library base name. The
- normal library search path is used to locate the library. To provide the
- required archive module callbacks and to indicate that the library is
- actually an archive module, it needs to provide a function named
- <function>_PG_archive_module_init</function>. The result of the function
- must be a pointer to a struct of type
- <structname>ArchiveModuleCallbacks</structname>, which contains everything
- that the core code needs to know to make use of the archive module. The
- return value needs to be of server lifetime, which is typically achieved by
- defining it as a <literal>static const</literal> variable in global scope.
+ Archiving modules must at least consist of an initialization function (see
+ <xref linkend="archive-module-init"/>) and the required callbacks (see
+ <xref linkend="archive-module-callbacks"/>). However, archive modules are
+ also permitted to do much more (e.g., declare GUCs and register background
+ workers).
+ </para>
+
+ <sect2 id="archive-module-init">
+ <title>Initialization Functions</title>
+ <indexterm zone="archive-module-init">
+ <primary>_PG_archive_module_init</primary>
+ </indexterm>
+
+ <para>
+ An archive library is loaded by dynamically loading a shared library with
+ the <xref linkend="guc-archive-library"/>'s name as the library base name.
+ The normal library search path is used to locate the library. To provide
+ the required archive module callbacks and to indicate that the library is
+ actually an archive module, it needs to provide a function named
+ <function>_PG_archive_module_init</function>. The result of the function
+ must be a pointer to a struct of type
+ <structname>ArchiveModuleCallbacks</structname>, which contains everything
+ that the core code needs to know to make use of the archive module. The
+ return value needs to be of server lifetime, which is typically achieved by
+ defining it as a <literal>static const</literal> variable in global scope.
<programlisting>
typedef struct ArchiveModuleCallbacks
@@ -65,91 +72,98 @@ typedef struct ArchiveModuleCallbacks
typedef const ArchiveModuleCallbacks *(*ArchiveModuleInit) (void);
</programlisting>
- Only the <function>archive_file_cb</function> callback is required. The
- others are optional.
- </para>
- </sect1>
+ Only the <function>archive_file_cb</function> callback is required. The
+ others are optional.
+ </para>
+ </sect2>
- <sect1 id="archive-module-callbacks">
- <title>Archive Module Callbacks</title>
- <para>
- The archive callbacks define the actual archiving behavior of the module.
- The server will call them as required to process each individual WAL file.
- </para>
+ <sect2 id="archive-module-callbacks">
+ <title>Archive Module Callbacks</title>
- <sect2 id="archive-module-startup">
- <title>Startup Callback</title>
<para>
- The <function>startup_cb</function> callback is called shortly after the
- module is loaded. This callback can be used to perform any additional
- initialization required. If the archive module has any state, it can use
- <structfield>state->private_data</structfield> to store it.
+ The archive callbacks define the actual archiving behavior of the module.
+ The server will call them as required to process each individual WAL file.
+ </para>
+
+ <sect3 id="archive-module-startup">
+ <title>Startup Callback</title>
+
+ <para>
+ The <function>startup_cb</function> callback is called shortly after the
+ module is loaded. This callback can be used to perform any additional
+ initialization required. If the archive module has any state, it can use
+ <structfield>state->private_data</structfield> to store it.
<programlisting>
typedef void (*ArchiveStartupCB) (ArchiveModuleState *state);
</programlisting>
- </para>
- </sect2>
+ </para>
+ </sect3>
- <sect2 id="archive-module-check">
- <title>Check Callback</title>
- <para>
- The <function>check_configured_cb</function> callback is called to determine
- whether the module is fully configured and ready to accept WAL files (e.g.,
- its configuration parameters are set to valid values). If no
- <function>check_configured_cb</function> is defined, the server always
- assumes the module is configured.
+ <sect3 id="archive-module-check">
+ <title>Check Callback</title>
+
+ <para>
+ The <function>check_configured_cb</function> callback is called to
+ determine whether the module is fully configured and ready to accept WAL
+ files (e.g., its configuration parameters are set to valid values). If no
+ <function>check_configured_cb</function> is defined, the server always
+ assumes the module is configured.
<programlisting>
typedef bool (*ArchiveCheckConfiguredCB) (ArchiveModuleState *state);
</programlisting>
- If <literal>true</literal> is returned, the server will proceed with
- archiving the file by calling the <function>archive_file_cb</function>
- callback. If <literal>false</literal> is returned, archiving will not
- proceed, and the archiver will emit the following message to the server log:
+ If <literal>true</literal> is returned, the server will proceed with
+ archiving the file by calling the <function>archive_file_cb</function>
+ callback. If <literal>false</literal> is returned, archiving will not
+ proceed, and the archiver will emit the following message to the server
+ log:
<screen>
WARNING: archive_mode enabled, yet archiving is not configured
</screen>
- In the latter case, the server will periodically call this function, and
- archiving will proceed only when it returns <literal>true</literal>.
- </para>
- </sect2>
+ In the latter case, the server will periodically call this function, and
+ archiving will proceed only when it returns <literal>true</literal>.
+ </para>
+ </sect3>
- <sect2 id="archive-module-archive">
- <title>Archive Callback</title>
- <para>
- The <function>archive_file_cb</function> callback is called to archive a
- single WAL file.
+ <sect3 id="archive-module-archive">
+ <title>Archive Callback</title>
+
+ <para>
+ The <function>archive_file_cb</function> callback is called to archive a
+ single WAL file.
<programlisting>
typedef bool (*ArchiveFileCB) (ArchiveModuleState *state, const char *file, const char *path);
</programlisting>
- If <literal>true</literal> is returned, the server proceeds as if the file
- was successfully archived, which may include recycling or removing the
- original WAL file. If <literal>false</literal> is returned, the server will
- keep the original WAL file and retry archiving later.
- <replaceable>file</replaceable> will contain just the file name of the WAL
- file to archive, while <replaceable>path</replaceable> contains the full
- path of the WAL file (including the file name).
- </para>
- </sect2>
-
- <sect2 id="archive-module-shutdown">
- <title>Shutdown Callback</title>
- <para>
- The <function>shutdown_cb</function> callback is called when the archiver
- process exits (e.g., after an error) or the value of
- <xref linkend="guc-archive-library"/> changes. If no
- <function>shutdown_cb</function> is defined, no special action is taken in
- these situations. If the archive module has any state, this callback should
- free it to avoid leaks.
+ If <literal>true</literal> is returned, the server proceeds as if the file
+ was successfully archived, which may include recycling or removing the
+ original WAL file. If <literal>false</literal> is returned, the server
+ will keep the original WAL file and retry archiving later.
+ <replaceable>file</replaceable> will contain just the file name of the WAL
+ file to archive, while <replaceable>path</replaceable> contains the full
+ path of the WAL file (including the file name).
+ </para>
+ </sect3>
+
+ <sect3 id="archive-module-shutdown">
+ <title>Shutdown Callback</title>
+
+ <para>
+ The <function>shutdown_cb</function> callback is called when the archiver
+ process exits (e.g., after an error) or the value of
+ <xref linkend="guc-archive-library"/> changes. If no
+ <function>shutdown_cb</function> is defined, no special action is taken in
+ these situations. If the archive module has any state, this callback
+ should free it to avoid leaks.
<programlisting>
typedef void (*ArchiveShutdownCB) (ArchiveModuleState *state);
</programlisting>
- </para>
+ </para>
+ </sect3>
</sect2>
</sect1>
</chapter>
--
2.25.1
--SLDf9lqlvOQaIe6s
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0005-introduce-restore_library.patch"
^ permalink raw reply [nested|flat] 24+ messages in thread
* pgsql: Add LZ4 compression to pg_dump
@ 2023-02-23 20:21 Tomas Vondra <[email protected]>
0 siblings, 1 reply; 24+ messages in thread
From: Tomas Vondra @ 2023-02-23 20:21 UTC (permalink / raw)
To: [email protected]
Add LZ4 compression to pg_dump
Expand pg_dump's compression streaming and file APIs to support the lz4
algorithm. The newly added compress_lz4.{c,h} files cover all the
functionality of the aforementioned APIs. Minor changes were necessary
in various pg_backup_* files, where code for the 'lz4' file suffix has
been added, as well as pg_dump's compression option parsing.
Author: Georgios Kokolatos
Reviewed-by: Michael Paquier, Rachel Heaton, Justin Pryzby, Shi Yu, Tomas Vondra
Discussion: https://postgr.es/m/faUNEOpts9vunEaLnmxmG-DldLSg_ql137OC3JYDmgrOMHm1RvvWY2IdBkv_CRxm5spCCb_OmKNk2T03...
Branch
------
master
Details
-------
https://git.postgresql.org/pg/commitdiff/0da243fed0875932f781aff08df782b56af58d02
Modified Files
--------------
doc/src/sgml/ref/pg_dump.sgml | 13 +-
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/compress_io.c | 26 +-
src/bin/pg_dump/compress_lz4.c | 626 ++++++++++++++++++++++++++++++++++
src/bin/pg_dump/compress_lz4.h | 24 ++
src/bin/pg_dump/meson.build | 8 +-
src/bin/pg_dump/pg_backup_archiver.c | 6 +-
src/bin/pg_dump/pg_backup_directory.c | 9 +-
src/bin/pg_dump/pg_dump.c | 5 +-
src/bin/pg_dump/t/002_pg_dump.pl | 82 ++++-
src/tools/pginclude/cpluspluscheck | 1 +
src/tools/pgindent/typedefs.list | 2 +
12 files changed, 782 insertions(+), 22 deletions(-)
^ permalink raw reply [nested|flat] 24+ messages in thread
* lz4 --rm on Ubuntu 18.04 (Add LZ4 compression to pg_dump)
@ 2023-03-08 17:55 Christoph Berg <[email protected]>
parent: Tomas Vondra <[email protected]>
0 siblings, 1 reply; 24+ messages in thread
From: Christoph Berg @ 2023-03-08 17:55 UTC (permalink / raw)
To: Tomas Vondra <[email protected]>; +Cc: PostgreSQL Hackers <[email protected]>
Re: Tomas Vondra
> Add LZ4 compression to pg_dump
This broke the TAP tests on Ubuntu 18.04 (bionic):
[17:06:45.513](0.000s) ok 1927 - compression_lz4_custom: should not dump test_table with 4-row INSERTs
# Running: pg_dump --jobs=2 --format=directory --compress=lz4:1 --file=/home/myon/projects/postgresql/pg/master/build/src/bin/pg_dump/tmp_check/tmp_test__aAO/compression_lz4_dir postgres
[17:06:46.651](1.137s) ok 1928 - compression_lz4_dir: pg_dump runs
# Running: /usr/bin/lz4 -z -f --rm /home/myon/projects/postgresql/pg/master/build/src/bin/pg_dump/tmp_check/tmp_test__aAO/compression_lz4_dir/blobs.toc /home/myon/projects/postgresql/pg/master/build/src/bin/pg_dump/tmp_check/tmp_test__aAO/compression_lz4_dir/blobs.toc.lz4
Incorrect parameters
Usage :
/usr/bin/lz4 [arg] [input] [output]
input : a filename
with no FILE, or when FILE is - or stdin, read standard input
Arguments :
-1 : Fast compression (default)
-9 : High compression
-d : decompression (default for .lz4 extension)
-z : force compression
-f : overwrite output without prompting
-h/-H : display help/long help and exit
[17:06:46.667](0.016s) not ok 1929 - compression_lz4_dir: compression commands
[17:06:46.668](0.001s)
[17:06:46.668](0.001s) # Failed test 'compression_lz4_dir: compression commands'
[17:06:46.669](0.000s) # at t/002_pg_dump.pl line 4274.
[17:06:46.670](0.001s) ok 1930 - compression_lz4_dir: glob check for /home/myon/projects/postgresql/pg/master/build/src/bin/pg_dump/tmp_check/tmp_test__aAO/compression_lz4_dir/toc.dat
The lz4 binary there doesn't have the --rm option yet.
liblz4-tool 0.0~r131-2ubuntu3
--rm appears in a single place only:
# Give coverage for manually compressed blob.toc files during
# restore.
compress_cmd => {
program => $ENV{'LZ4'},
args => [
'-z', '-f', '--rm',
"$tempdir/compression_lz4_dir/blobs.toc",
"$tempdir/compression_lz4_dir/blobs.toc.lz4",
],
},
18.04 will be EOL in a few weeks so it might be ok to just say it's
not supported, but removing the input file manually after calling lz4
would be an easy fix.
Christoph
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: lz4 --rm on Ubuntu 18.04 (Add LZ4 compression to pg_dump)
@ 2023-03-08 19:20 Daniel Gustafsson <[email protected]>
parent: Christoph Berg <[email protected]>
0 siblings, 1 reply; 24+ messages in thread
From: Daniel Gustafsson @ 2023-03-08 19:20 UTC (permalink / raw)
To: Christoph Berg <[email protected]>; +Cc: Tomas Vondra <[email protected]>; PostgreSQL Hackers <[email protected]>
> On 8 Mar 2023, at 18:55, Christoph Berg <[email protected]> wrote:
> 18.04 will be EOL in a few weeks so it might be ok to just say it's
> not supported, but removing the input file manually after calling lz4
> would be an easy fix.
Is it reasonable to expect that this version of LZ4 can/will appear on any
other platform outside of archeology? Removing the file manually would be a
trivial way to stabilize but if it's only expected to happen on platforms which
are long since EOL by the time 16 ships then the added complication could be
hard to justify.
--
Daniel Gustafsson
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: lz4 --rm on Ubuntu 18.04 (Add LZ4 compression to pg_dump)
@ 2023-03-08 23:39 Tomas Vondra <[email protected]>
parent: Daniel Gustafsson <[email protected]>
0 siblings, 1 reply; 24+ messages in thread
From: Tomas Vondra @ 2023-03-08 23:39 UTC (permalink / raw)
To: Daniel Gustafsson <[email protected]>; Christoph Berg <[email protected]>; +Cc: Tomas Vondra <[email protected]>; PostgreSQL Hackers <[email protected]>
On 3/8/23 20:20, Daniel Gustafsson wrote:
>> On 8 Mar 2023, at 18:55, Christoph Berg <[email protected]> wrote:
>
>> 18.04 will be EOL in a few weeks so it might be ok to just say it's
>> not supported, but removing the input file manually after calling lz4
>> would be an easy fix.
>
> Is it reasonable to expect that this version of LZ4 can/will appear on any
> other platform outside of archeology? Removing the file manually would be a
> trivial way to stabilize but if it's only expected to happen on platforms which
> are long since EOL by the time 16 ships then the added complication could be
> hard to justify.
>
IMO we should fix that. We have a bunch of buildfarm members running on
Ubuntu 18.04 (or older) - it's true none of them seems to be running TAP
tests. But considering how trivial the fix is ...
Barring objections, I'll push a fix early next week.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: lz4 --rm on Ubuntu 18.04 (Add LZ4 compression to pg_dump)
@ 2023-03-09 00:30 Michael Paquier <[email protected]>
parent: Tomas Vondra <[email protected]>
0 siblings, 1 reply; 24+ messages in thread
From: Michael Paquier @ 2023-03-09 00:30 UTC (permalink / raw)
To: Tomas Vondra <[email protected]>; +Cc: Daniel Gustafsson <[email protected]>; Christoph Berg <[email protected]>; Tomas Vondra <[email protected]>; PostgreSQL Hackers <[email protected]>
On Thu, Mar 09, 2023 at 12:39:08AM +0100, Tomas Vondra wrote:
> IMO we should fix that. We have a bunch of buildfarm members running on
> Ubuntu 18.04 (or older) - it's true none of them seems to be running TAP
> tests. But considering how trivial the fix is ...
>
> Barring objections, I'll push a fix early next week.
+1, better to change that, thanks. Actually, would --rm be OK even on
Windows? As far as I can see, the CI detects a LZ4 command for the
VS2019 environment but not the liblz4 libraries that would be needed
to trigger the set of tests.
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: lz4 --rm on Ubuntu 18.04 (Add LZ4 compression to pg_dump)
@ 2023-03-09 18:00 Tomas Vondra <[email protected]>
parent: Michael Paquier <[email protected]>
0 siblings, 1 reply; 24+ messages in thread
From: Tomas Vondra @ 2023-03-09 18:00 UTC (permalink / raw)
To: Michael Paquier <[email protected]>; +Cc: Daniel Gustafsson <[email protected]>; Christoph Berg <[email protected]>; Tomas Vondra <[email protected]>; PostgreSQL Hackers <[email protected]>
On 3/9/23 01:30, Michael Paquier wrote:
> On Thu, Mar 09, 2023 at 12:39:08AM +0100, Tomas Vondra wrote:
>> IMO we should fix that. We have a bunch of buildfarm members running on
>> Ubuntu 18.04 (or older) - it's true none of them seems to be running TAP
>> tests. But considering how trivial the fix is ...
>>
>> Barring objections, I'll push a fix early next week.
>
> +1, better to change that, thanks. Actually, would --rm be OK even on
> Windows? As far as I can see, the CI detects a LZ4 command for the
> VS2019 environment but not the liblz4 libraries that would be needed
> to trigger the set of tests.
Thanks for noticing that. I'll investigate next week.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: lz4 --rm on Ubuntu 18.04 (Add LZ4 compression to pg_dump)
@ 2023-03-13 23:16 Tomas Vondra <[email protected]>
parent: Tomas Vondra <[email protected]>
0 siblings, 2 replies; 24+ messages in thread
From: Tomas Vondra @ 2023-03-13 23:16 UTC (permalink / raw)
To: Michael Paquier <[email protected]>; +Cc: Daniel Gustafsson <[email protected]>; Christoph Berg <[email protected]>; Tomas Vondra <[email protected]>; PostgreSQL Hackers <[email protected]>
On 3/9/23 19:00, Tomas Vondra wrote:
>
>
> On 3/9/23 01:30, Michael Paquier wrote:
>> On Thu, Mar 09, 2023 at 12:39:08AM +0100, Tomas Vondra wrote:
>>> IMO we should fix that. We have a bunch of buildfarm members running on
>>> Ubuntu 18.04 (or older) - it's true none of them seems to be running TAP
>>> tests. But considering how trivial the fix is ...
>>>
>>> Barring objections, I'll push a fix early next week.
>>
>> +1, better to change that, thanks. Actually, would --rm be OK even on
>> Windows? As far as I can see, the CI detects a LZ4 command for the
>> VS2019 environment but not the liblz4 libraries that would be needed
>> to trigger the set of tests.
>
> Thanks for noticing that. I'll investigate next week.
>
So, here's a fix that should (I think) replace the 'lz4 --rm' with a
simple 'rm'. I have two doubts about this, though:
1) I haven't found a simple way to inject additional command into the
test. The pg_dump runs have a predefined list of "steps" to run:
-- compress_cmd
-- glob_patterns
-- command_like
-- restore_cmd
and I don't think there's a good place to inject the 'rm' so I ended up
adding a 'cleanup_cmd' right after 'compress_cmd'. But it seems a bit
strange / hacky. Maybe there's a better way?
2) I wonder if Windows will know what 'rm' means. I haven't found any
TAP test doing 'rm' and don't see 'rm' in any $ENV either.
That being said, I have no idea how to make this work on our Windows CI.
As mentioned, the environment is missing the lz4 library - there's a
setup_additional_packages_script: |
REM choco install -y --no-progress ...
in the .yml file, but AFAICS the chocolatey does not have lz4 :-/
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Attachments:
[text/x-patch] lz4-rm-tap-test.patch (1.4K, ../../[email protected]/2-lz4-rm-tap-test.patch)
download | inline diff:
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 9c354213ce..b3dcf6ff6d 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -178,11 +178,18 @@ my %pgdump_runs = (
compress_cmd => {
program => $ENV{'LZ4'},
args => [
- '-z', '-f', '--rm',
+ '-z', '-f',
"$tempdir/compression_lz4_dir/blobs.toc",
"$tempdir/compression_lz4_dir/blobs.toc.lz4",
],
},
+ # remove the source (uncompressed) .toc file
+ cleanup_cmd => {
+ program => 'rm',
+ args => [
+ "$tempdir/compression_lz4_dir/blobs.toc",
+ ],
+ },
# Verify that data files were compressed
glob_patterns => [
"$tempdir/compression_lz4_dir/toc.dat",
@@ -4274,6 +4281,20 @@ foreach my $run (sort keys %pgdump_runs)
command_ok(\@full_compress_cmd, "$run: compression commands");
}
+ if ($pgdump_runs{$run}->{cleanup_cmd})
+ {
+ my ($cleanup_cmd) = $pgdump_runs{$run}->{cleanup_cmd};
+ my $cleanup_program = $cleanup_cmd->{program};
+
+ # Skip the rest of the test if the compression program is
+ # not defined.
+ next if (!defined($cleanup_cmd) || $cleanup_cmd eq '');
+
+ my @full_cleanup_cmd =
+ ($cleanup_cmd->{program}, @{ $cleanup_cmd->{args} });
+ command_ok(\@full_cleanup_cmd, "$run: cleanup commands");
+ }
+
if ($pgdump_runs{$run}->{glob_patterns})
{
my $glob_patterns = $pgdump_runs{$run}->{glob_patterns};
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: lz4 --rm on Ubuntu 18.04 (Add LZ4 compression to pg_dump)
@ 2023-03-14 10:34 Christoph Berg <[email protected]>
parent: Tomas Vondra <[email protected]>
1 sibling, 1 reply; 24+ messages in thread
From: Christoph Berg @ 2023-03-14 10:34 UTC (permalink / raw)
To: Tomas Vondra <[email protected]>; +Cc: Michael Paquier <[email protected]>; Daniel Gustafsson <[email protected]>; Tomas Vondra <[email protected]>; PostgreSQL Hackers <[email protected]>
Re: Tomas Vondra
> and I don't think there's a good place to inject the 'rm' so I ended up
> adding a 'cleanup_cmd' right after 'compress_cmd'. But it seems a bit
> strange / hacky. Maybe there's a better way?
Does the file need to be removed at all? Could we leave it there and
have "make clean" remove it?
Christoph
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: lz4 --rm on Ubuntu 18.04 (Add LZ4 compression to pg_dump)
@ 2023-03-14 14:24 Tomas Vondra <[email protected]>
parent: Christoph Berg <[email protected]>
0 siblings, 0 replies; 24+ messages in thread
From: Tomas Vondra @ 2023-03-14 14:24 UTC (permalink / raw)
To: Christoph Berg <[email protected]>; +Cc: Michael Paquier <[email protected]>; Daniel Gustafsson <[email protected]>; Tomas Vondra <[email protected]>; PostgreSQL Hackers <[email protected]>
On 3/14/23 11:34, Christoph Berg wrote:
> Re: Tomas Vondra
>> and I don't think there's a good place to inject the 'rm' so I ended up
>> adding a 'cleanup_cmd' right after 'compress_cmd'. But it seems a bit
>> strange / hacky. Maybe there's a better way?
>
> Does the file need to be removed at all? Could we leave it there and
> have "make clean" remove it?
>
I don't think that'd work, because of the automatic "discovery" where we
check if a file exists, and if not we try to append .gz and .lz4. So if
you leave the .toc, we'd not find the .lz4, making the test useless ...
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: lz4 --rm on Ubuntu 18.04 (Add LZ4 compression to pg_dump)
@ 2023-04-06 16:39 Justin Pryzby <[email protected]>
parent: Tomas Vondra <[email protected]>
1 sibling, 1 reply; 24+ messages in thread
From: Justin Pryzby @ 2023-04-06 16:39 UTC (permalink / raw)
To: Tomas Vondra <[email protected]>; +Cc: Michael Paquier <[email protected]>; Daniel Gustafsson <[email protected]>; Christoph Berg <[email protected]>; Tomas Vondra <[email protected]>; Jacob Champion <[email protected]>; Andres Freund <[email protected]>; [email protected]
On Tue, Mar 14, 2023 at 12:16:16AM +0100, Tomas Vondra wrote:
> On 3/9/23 19:00, Tomas Vondra wrote:
> > On 3/9/23 01:30, Michael Paquier wrote:
> >> On Thu, Mar 09, 2023 at 12:39:08AM +0100, Tomas Vondra wrote:
> >>> IMO we should fix that. We have a bunch of buildfarm members running on
> >>> Ubuntu 18.04 (or older) - it's true none of them seems to be running TAP
> >>> tests. But considering how trivial the fix is ...
> >>>
> >>> Barring objections, I'll push a fix early next week.
> >>
> >> +1, better to change that, thanks. Actually, would --rm be OK even on
> >> Windows? As far as I can see, the CI detects a LZ4 command for the
> >> VS2019 environment but not the liblz4 libraries that would be needed
> >> to trigger the set of tests.
> >
> > Thanks for noticing that. I'll investigate next week.
>
> So, here's a fix that should (I think) replace the 'lz4 --rm' with a
> simple 'rm'. I have two doubts about this, though:
>
> 1) I haven't found a simple way to inject additional command into the
> test. The pg_dump runs have a predefined list of "steps" to run:
>
> -- compress_cmd
> -- glob_patterns
> -- command_like
> -- restore_cmd
>
> and I don't think there's a good place to inject the 'rm' so I ended up
> adding a 'cleanup_cmd' right after 'compress_cmd'. But it seems a bit
> strange / hacky. Maybe there's a better way?
I don't know if there's a better way, and I don't think it's worth
complicating the tests by more than about 2 lines to handle this.
> 2) I wonder if Windows will know what 'rm' means. I haven't found any
> TAP test doing 'rm' and don't see 'rm' in any $ENV either.
CI probably will, since it's Andres' image built with git-tools and
other helpful stuff installed. But in general I think it won't; perl is
being used for all the portable stuff.
*If* you wanted to do something to fix this, you could create a key
called files_to_remove_after_loading, and run unlink on those files
rather than running a shell command. Or maybe just remove the file
unconditionally at the start of the script ?
> That being said, I have no idea how to make this work on our Windows CI.
> As mentioned, the environment is missing the lz4 library - there's a
>
> setup_additional_packages_script: |
> REM choco install -y --no-progress ...
>
> in the .yml file, but AFAICS the chocolatey does not have lz4 :-/
I updated what I'd done in the zstd patch to also run with LZ4.
This won't apply directly due to other patches, but you get the idea...
Maybe it'd be good to have a commented-out "wraps" hint like there is
for choco. The downloaded files could be cached, too.
diff --git a/.cirrus.yml b/.cirrus.yml
index a3977a4036e..b4387a739f3 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -644,9 +644,11 @@ task:
vcvarsall x64
mkdir subprojects
meson wrap install zstd
- meson configure -D zstd:multithread=enabled --force-fallback-for=zstd
+ meson wrap install lz4
+ meson subprojects download
+ meson configure -D zstd:multithread=enabled --force-fallback-for=zstd --force-fallback-for=lz4
set CC=c:\ProgramData\chocolatey\lib\ccache\tools\ccache-4.8-windows-x86_64\ccache.exe cl.exe
- meson setup --backend ninja --buildtype debug -Dc_link_args=/DEBUG:FASTLINK -Dcassert=true -Db_pch=false -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% -DPG_TEST_EXTRA="%PG_TEST_EXTRA%" -D zstd=enabled -Dc_args="/Z7 /MDd" build
+ meson setup --backend ninja --buildtype debug -Dc_link_args=/DEBUG:FASTLINK -Dcassert=true -Db_pch=false -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% -DPG_TEST_EXTRA="%PG_TEST_EXTRA%" -D zstd=enabled -D lz4=enabled -Dc_args="/Z7 /MDd" build
build_script: |
vcvarsall x64
--
Justin
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: lz4 --rm on Ubuntu 18.04 (Add LZ4 compression to pg_dump)
@ 2023-04-06 18:40 Daniel Gustafsson <[email protected]>
parent: Justin Pryzby <[email protected]>
0 siblings, 0 replies; 24+ messages in thread
From: Daniel Gustafsson @ 2023-04-06 18:40 UTC (permalink / raw)
To: Justin Pryzby <[email protected]>; +Cc: Tomas Vondra <[email protected]>; Michael Paquier <[email protected]>; Christoph Berg <[email protected]>; Tomas Vondra <[email protected]>; Jacob Champion <[email protected]>; Andres Freund <[email protected]>; [email protected]
> On 6 Apr 2023, at 18:39, Justin Pryzby <[email protected]> wrote:
> *If* you wanted to do something to fix this, you could create a key
> called files_to_remove_after_loading, and run unlink on those files
> rather than running a shell command. Or maybe just remove the file
> unconditionally at the start of the script ?
Since the test is written in Perl, and Perl has a function for deleting files
which abstracts the platform differences, using it seems like a logical choice?
{cleanup_cmd} can be replaced with {cleanup_files} with an unlink called on
that?
--
Daniel Gustafsson
^ permalink raw reply [nested|flat] 24+ messages in thread
end of thread, other threads:[~2023-04-06 18:40 UTC | newest]
Thread overview: 24+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2023-02-15 22:48 [PATCH v18 4/5] restructure archive modules docs in preparation for restore modules Nathan Bossart <[email protected]>
2023-02-15 22:48 [PATCH v12 5/6] restructure archive modules docs in preparation for restore modules Nathan Bossart <[email protected]>
2023-02-15 22:48 [PATCH v20 4/5] restructure archive modules docs in preparation for restore modules Nathan Bossart <[email protected]>
2023-02-15 22:48 [PATCH v15 6/7] restructure archive modules docs in preparation for restore modules Nathan Bossart <[email protected]>
2023-02-15 22:48 [PATCH v19 4/5] restructure archive modules docs in preparation for restore modules Nathan Bossart <[email protected]>
2023-02-15 22:48 [PATCH v16 4/5] restructure archive modules docs in preparation for restore modules Nathan Bossart <[email protected]>
2023-02-15 22:48 [PATCH v14 6/7] restructure archive modules docs in preparation for restore modules Nathan Bossart <[email protected]>
2023-02-15 22:48 [PATCH v19 4/5] restructure archive modules docs in preparation for restore modules Nathan Bossart <[email protected]>
2023-02-15 22:48 [PATCH v18 4/5] restructure archive modules docs in preparation for restore modules Nathan Bossart <[email protected]>
2023-02-15 22:48 [PATCH v19 4/5] restructure archive modules docs in preparation for restore modules Nathan Bossart <[email protected]>
2023-02-15 22:48 [PATCH v18 4/5] restructure archive modules docs in preparation for restore modules Nathan Bossart <[email protected]>
2023-02-15 22:48 [PATCH v20 4/5] restructure archive modules docs in preparation for restore modules Nathan Bossart <[email protected]>
2023-02-15 22:48 [PATCH v13 6/7] restructure archive modules docs in preparation for restore modules Nathan Bossart <[email protected]>
2023-02-23 20:21 pgsql: Add LZ4 compression to pg_dump Tomas Vondra <[email protected]>
2023-03-08 17:55 ` lz4 --rm on Ubuntu 18.04 (Add LZ4 compression to pg_dump) Christoph Berg <[email protected]>
2023-03-08 19:20 ` Re: lz4 --rm on Ubuntu 18.04 (Add LZ4 compression to pg_dump) Daniel Gustafsson <[email protected]>
2023-03-08 23:39 ` Re: lz4 --rm on Ubuntu 18.04 (Add LZ4 compression to pg_dump) Tomas Vondra <[email protected]>
2023-03-09 00:30 ` Re: lz4 --rm on Ubuntu 18.04 (Add LZ4 compression to pg_dump) Michael Paquier <[email protected]>
2023-03-09 18:00 ` Re: lz4 --rm on Ubuntu 18.04 (Add LZ4 compression to pg_dump) Tomas Vondra <[email protected]>
2023-03-13 23:16 ` Re: lz4 --rm on Ubuntu 18.04 (Add LZ4 compression to pg_dump) Tomas Vondra <[email protected]>
2023-03-14 10:34 ` Re: lz4 --rm on Ubuntu 18.04 (Add LZ4 compression to pg_dump) Christoph Berg <[email protected]>
2023-03-14 14:24 ` Re: lz4 --rm on Ubuntu 18.04 (Add LZ4 compression to pg_dump) Tomas Vondra <[email protected]>
2023-04-06 16:39 ` Re: lz4 --rm on Ubuntu 18.04 (Add LZ4 compression to pg_dump) Justin Pryzby <[email protected]>
2023-04-06 18:40 ` Re: lz4 --rm on Ubuntu 18.04 (Add LZ4 compression to pg_dump) Daniel Gustafsson <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox