public inbox for [email protected]
help / color / mirror / Atom feedExtensible storage manager API - smgr hooks
7+ messages / 7 participants
[nested] [flat]
* Extensible storage manager API - smgr hooks
@ 2021-06-29 21:49 Anastasia Lubennikova <[email protected]>
0 siblings, 1 reply; 7+ messages in thread
From: Anastasia Lubennikova @ 2021-06-29 21:49 UTC (permalink / raw)
To: [email protected] <[email protected]>
Hi, hackers!
Many recently discussed features can make use of an extensible storage
manager API. Namely, storage level compression and encryption [1], [2], [3],
disk quota feature [4], SLRU storage changes [5], and any other features
that may want to substitute PostgreSQL storage layer with their
implementation (i.e. lazy_restore [6]).
Attached is a proposal to change smgr API to make it extensible. The idea
is to add a hook for plugins to get control in smgr and define custom
storage managers. The patch replaces smgrsw[] array and smgr_sw selector
with smgr() function that loads f_smgr implementation.
As before it has only one implementation - smgr_md, which is wrapped into
smgr_standard().
To create custom implementation, a developer needs to implement smgr API
functions
static const struct f_smgr smgr_custom =
{
.smgr_init = custominit,
...
}
create a hook function
const f_smgr * smgr_custom(BackendId backend, RelFileNode rnode)
{
//Here we can also add some logic and chose which smgr to use based
on rnode and backend
return &smgr_custom;
}
and finally set the hook:
smgr_hook = smgr_custom;
[1]
https://www.postgresql.org/message-id/flat/[email protected]
[2]
https://www.postgresql.org/message-id/flat/272dd2d9.e52a.17235f2c050.Coremail.chjischj%40163.com
[3] https://postgrespro.com/docs/enterprise/9.6/cfs
[4]
https://www.postgresql.org/message-id/flat/CAB0yre%3DRP_ho6Bq4cV23ELKxRcfhV2Yqrb1zHp0RfUPEWCnBRw%40m...
[5]
https://www.postgresql.org/message-id/flat/20180814213500.GA74618%4060f81dc409fc.ant.amazon.com
[6] https://wiki.postgresql.org/wiki/PGCon_2021_Fun_With_WAL#Lazy_Restore
--
Best regards,
Lubennikova Anastasia
Attachments:
[text/x-patch] 0001-smgr_api.patch (12.9K, ../../CAP4vRV6JKXyFfEOf=n+v5RGsZywAQ3CTM8ESWvgq+S87Tmgx_g@mail.gmail.com/3-0001-smgr_api.patch)
download | inline diff:
From 90085398f5ecc90d6b7caa318bd3d5f2867ef95c Mon Sep 17 00:00:00 2001
From: anastasia <[email protected]>
Date: Tue, 29 Jun 2021 22:16:26 +0300
Subject: [PATCH] smgr_api.patch
Make smgr API pluggable. Add smgr_hook that can be used to define custom storage managers.
Remove smgrsw[] array and smgr_sw selector. Instead, smgropen() uses smgr() function to load
f_smgr implementation using smgr_hook.
Also add smgr_init_hook and smgr_shutdown_hook.
And a lot of mechanical changes in smgr.c functions.
---
src/backend/storage/smgr/smgr.c | 136 ++++++++++++++------------------
src/include/storage/smgr.h | 56 ++++++++++++-
2 files changed, 116 insertions(+), 76 deletions(-)
diff --git a/src/backend/storage/smgr/smgr.c b/src/backend/storage/smgr/smgr.c
index 4dc24649df..5f1981a353 100644
--- a/src/backend/storage/smgr/smgr.c
+++ b/src/backend/storage/smgr/smgr.c
@@ -26,47 +26,8 @@
#include "utils/hsearch.h"
#include "utils/inval.h"
-
-/*
- * This struct of function pointers defines the API between smgr.c and
- * any individual storage manager module. Note that smgr subfunctions are
- * generally expected to report problems via elog(ERROR). An exception is
- * that smgr_unlink should use elog(WARNING), rather than erroring out,
- * because we normally unlink relations during post-commit/abort cleanup,
- * and so it's too late to raise an error. Also, various conditions that
- * would normally be errors should be allowed during bootstrap and/or WAL
- * recovery --- see comments in md.c for details.
- */
-typedef struct f_smgr
-{
- void (*smgr_init) (void); /* may be NULL */
- void (*smgr_shutdown) (void); /* may be NULL */
- void (*smgr_open) (SMgrRelation reln);
- void (*smgr_close) (SMgrRelation reln, ForkNumber forknum);
- void (*smgr_create) (SMgrRelation reln, ForkNumber forknum,
- bool isRedo);
- bool (*smgr_exists) (SMgrRelation reln, ForkNumber forknum);
- void (*smgr_unlink) (RelFileNodeBackend rnode, ForkNumber forknum,
- bool isRedo);
- void (*smgr_extend) (SMgrRelation reln, ForkNumber forknum,
- BlockNumber blocknum, char *buffer, bool skipFsync);
- bool (*smgr_prefetch) (SMgrRelation reln, ForkNumber forknum,
- BlockNumber blocknum);
- void (*smgr_read) (SMgrRelation reln, ForkNumber forknum,
- BlockNumber blocknum, char *buffer);
- void (*smgr_write) (SMgrRelation reln, ForkNumber forknum,
- BlockNumber blocknum, char *buffer, bool skipFsync);
- void (*smgr_writeback) (SMgrRelation reln, ForkNumber forknum,
- BlockNumber blocknum, BlockNumber nblocks);
- BlockNumber (*smgr_nblocks) (SMgrRelation reln, ForkNumber forknum);
- void (*smgr_truncate) (SMgrRelation reln, ForkNumber forknum,
- BlockNumber nblocks);
- void (*smgr_immedsync) (SMgrRelation reln, ForkNumber forknum);
-} f_smgr;
-
-static const f_smgr smgrsw[] = {
+static const f_smgr smgr_md = {
/* magnetic disk */
- {
.smgr_init = mdinit,
.smgr_shutdown = NULL,
.smgr_open = mdopen,
@@ -82,11 +43,8 @@ static const f_smgr smgrsw[] = {
.smgr_nblocks = mdnblocks,
.smgr_truncate = mdtruncate,
.smgr_immedsync = mdimmedsync,
- }
};
-static const int NSmgr = lengthof(smgrsw);
-
/*
* Each backend has a hashtable that stores all extant SMgrRelation objects.
* In addition, "unowned" SMgrRelation objects are chained together in a list.
@@ -110,13 +68,10 @@ static void smgrshutdown(int code, Datum arg);
void
smgrinit(void)
{
- int i;
+ if (smgr_init_hook)
+ (*smgr_init_hook)();
- for (i = 0; i < NSmgr; i++)
- {
- if (smgrsw[i].smgr_init)
- smgrsw[i].smgr_init();
- }
+ smgr_init_standard();
/* register the shutdown proc */
on_proc_exit(smgrshutdown, 0);
@@ -128,15 +83,50 @@ smgrinit(void)
static void
smgrshutdown(int code, Datum arg)
{
- int i;
+ if (smgr_shutdown_hook)
+ (*smgr_shutdown_hook)();
+
+ smgr_shutdown_standard();
+}
+
+/* Hooks for plugins to get control in smgr */
+smgr_hook_type smgr_hook = NULL;
+smgr_init_hook_type smgr_init_hook = NULL;
+smgr_shutdown_hook_type smgr_shutdown_hook = NULL;
+
+const f_smgr *
+smgr_standard(BackendId backend, RelFileNode rnode)
+{
+ return &smgr_md;
+}
- for (i = 0; i < NSmgr; i++)
+void
+smgr_init_standard(void)
+{
+ mdinit();
+}
+
+void
+smgr_shutdown_standard(void)
+{
+}
+
+const f_smgr *
+smgr(BackendId backend, RelFileNode rnode)
+{
+ const f_smgr *result;
+
+ if (smgr_hook)
{
- if (smgrsw[i].smgr_shutdown)
- smgrsw[i].smgr_shutdown();
+ result = (*smgr_hook)(backend, rnode);
}
+ else
+ result = smgr_standard(backend, rnode);
+
+ return result;
}
+
/*
* smgropen() -- Return an SMgrRelation object, creating it if need be.
*
@@ -176,10 +166,11 @@ smgropen(RelFileNode rnode, BackendId backend)
reln->smgr_targblock = InvalidBlockNumber;
for (int i = 0; i <= MAX_FORKNUM; ++i)
reln->smgr_cached_nblocks[i] = InvalidBlockNumber;
- reln->smgr_which = 0; /* we only have md.c at present */
+
+ reln->smgr = smgr(backend, rnode);
/* implementation-specific initialization */
- smgrsw[reln->smgr_which].smgr_open(reln);
+ (*reln->smgr).smgr_open(reln);
/* it has no owner yet */
dlist_push_tail(&unowned_relns, &reln->node);
@@ -246,7 +237,7 @@ smgrclearowner(SMgrRelation *owner, SMgrRelation reln)
bool
smgrexists(SMgrRelation reln, ForkNumber forknum)
{
- return smgrsw[reln->smgr_which].smgr_exists(reln, forknum);
+ return (*reln->smgr).smgr_exists(reln, forknum);
}
/*
@@ -259,7 +250,7 @@ smgrclose(SMgrRelation reln)
ForkNumber forknum;
for (forknum = 0; forknum <= MAX_FORKNUM; forknum++)
- smgrsw[reln->smgr_which].smgr_close(reln, forknum);
+ (*reln->smgr).smgr_close(reln, forknum);
owner = reln->smgr_owner;
@@ -332,7 +323,7 @@ smgrclosenode(RelFileNodeBackend rnode)
void
smgrcreate(SMgrRelation reln, ForkNumber forknum, bool isRedo)
{
- smgrsw[reln->smgr_which].smgr_create(reln, forknum, isRedo);
+ (*reln->smgr).smgr_create(reln, forknum, isRedo);
}
/*
@@ -360,12 +351,10 @@ smgrdosyncall(SMgrRelation *rels, int nrels)
*/
for (i = 0; i < nrels; i++)
{
- int which = rels[i]->smgr_which;
-
for (forknum = 0; forknum <= MAX_FORKNUM; forknum++)
{
- if (smgrsw[which].smgr_exists(rels[i], forknum))
- smgrsw[which].smgr_immedsync(rels[i], forknum);
+ if ((*rels[i]->smgr).smgr_exists(rels[i], forknum))
+ (*rels[i]->smgr).smgr_immedsync(rels[i], forknum);
}
}
}
@@ -404,13 +393,12 @@ smgrdounlinkall(SMgrRelation *rels, int nrels, bool isRedo)
for (i = 0; i < nrels; i++)
{
RelFileNodeBackend rnode = rels[i]->smgr_rnode;
- int which = rels[i]->smgr_which;
rnodes[i] = rnode;
/* Close the forks at smgr level */
for (forknum = 0; forknum <= MAX_FORKNUM; forknum++)
- smgrsw[which].smgr_close(rels[i], forknum);
+ (*rels[i]->smgr).smgr_close(rels[i], forknum);
}
/*
@@ -439,10 +427,8 @@ smgrdounlinkall(SMgrRelation *rels, int nrels, bool isRedo)
for (i = 0; i < nrels; i++)
{
- int which = rels[i]->smgr_which;
-
for (forknum = 0; forknum <= MAX_FORKNUM; forknum++)
- smgrsw[which].smgr_unlink(rnodes[i], forknum, isRedo);
+ (*rels[i]->smgr).smgr_unlink(rnodes[i], forknum, isRedo);
}
pfree(rnodes);
@@ -462,7 +448,7 @@ void
smgrextend(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
char *buffer, bool skipFsync)
{
- smgrsw[reln->smgr_which].smgr_extend(reln, forknum, blocknum,
+ (*reln->smgr).smgr_extend(reln, forknum, blocknum,
buffer, skipFsync);
/*
@@ -486,7 +472,7 @@ smgrextend(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
bool
smgrprefetch(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum)
{
- return smgrsw[reln->smgr_which].smgr_prefetch(reln, forknum, blocknum);
+ return (*reln->smgr).smgr_prefetch(reln, forknum, blocknum);
}
/*
@@ -501,7 +487,7 @@ void
smgrread(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
char *buffer)
{
- smgrsw[reln->smgr_which].smgr_read(reln, forknum, blocknum, buffer);
+ (*reln->smgr).smgr_read(reln, forknum, blocknum, buffer);
}
/*
@@ -523,7 +509,7 @@ void
smgrwrite(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
char *buffer, bool skipFsync)
{
- smgrsw[reln->smgr_which].smgr_write(reln, forknum, blocknum,
+ (*reln->smgr).smgr_write(reln, forknum, blocknum,
buffer, skipFsync);
}
@@ -536,7 +522,7 @@ void
smgrwriteback(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
BlockNumber nblocks)
{
- smgrsw[reln->smgr_which].smgr_writeback(reln, forknum, blocknum,
+ (*reln->smgr).smgr_writeback(reln, forknum, blocknum,
nblocks);
}
@@ -554,7 +540,7 @@ smgrnblocks(SMgrRelation reln, ForkNumber forknum)
if (result != InvalidBlockNumber)
return result;
- result = smgrsw[reln->smgr_which].smgr_nblocks(reln, forknum);
+ result = (*reln->smgr).smgr_nblocks(reln, forknum);
reln->smgr_cached_nblocks[forknum] = result;
@@ -620,7 +606,7 @@ smgrtruncate(SMgrRelation reln, ForkNumber *forknum, int nforks, BlockNumber *nb
/* Make the cached size is invalid if we encounter an error. */
reln->smgr_cached_nblocks[forknum[i]] = InvalidBlockNumber;
- smgrsw[reln->smgr_which].smgr_truncate(reln, forknum[i], nblocks[i]);
+ (*reln->smgr).smgr_truncate(reln, forknum[i], nblocks[i]);
/*
* We might as well update the local smgr_cached_nblocks values. The
@@ -659,7 +645,7 @@ smgrtruncate(SMgrRelation reln, ForkNumber *forknum, int nforks, BlockNumber *nb
void
smgrimmedsync(SMgrRelation reln, ForkNumber forknum)
{
- smgrsw[reln->smgr_which].smgr_immedsync(reln, forknum);
+ (*reln->smgr).smgr_immedsync(reln, forknum);
}
/*
diff --git a/src/include/storage/smgr.h b/src/include/storage/smgr.h
index a6fbf7b6a6..19c804de57 100644
--- a/src/include/storage/smgr.h
+++ b/src/include/storage/smgr.h
@@ -18,6 +18,8 @@
#include "storage/block.h"
#include "storage/relfilenode.h"
+struct f_smgr;
+
/*
* smgr.c maintains a table of SMgrRelation objects, which are essentially
* cached file handles. An SMgrRelation is created (if not already present)
@@ -59,7 +61,7 @@ typedef struct SMgrRelationData
* Fields below here are intended to be private to smgr.c and its
* submodules. Do not touch them from elsewhere.
*/
- int smgr_which; /* storage manager selector */
+ const struct f_smgr *smgr; /* storage manager selector */
/*
* for md.c; per-fork arrays of the number of open segments
@@ -77,6 +79,58 @@ typedef SMgrRelationData *SMgrRelation;
#define SmgrIsTemp(smgr) \
RelFileNodeBackendIsTemp((smgr)->smgr_rnode)
+
+/*
+ * This struct of function pointers defines the API between smgr.c and
+ * any individual storage manager module. Note that smgr subfunctions are
+ * generally expected to report problems via elog(ERROR). An exception is
+ * that smgr_unlink should use elog(WARNING), rather than erroring out,
+ * because we normally unlink relations during post-commit/abort cleanup,
+ * and so it's too late to raise an error. Also, various conditions that
+ * would normally be errors should be allowed during bootstrap and/or WAL
+ * recovery --- see comments in md.c for details.
+ */
+typedef struct f_smgr
+{
+ void (*smgr_init) (void); /* may be NULL */
+ void (*smgr_shutdown) (void); /* may be NULL */
+ void (*smgr_open) (SMgrRelation reln);
+ void (*smgr_close) (SMgrRelation reln, ForkNumber forknum);
+ void (*smgr_create) (SMgrRelation reln, ForkNumber forknum,
+ bool isRedo);
+ bool (*smgr_exists) (SMgrRelation reln, ForkNumber forknum);
+ void (*smgr_unlink) (RelFileNodeBackend rnode, ForkNumber forknum,
+ bool isRedo);
+ void (*smgr_extend) (SMgrRelation reln, ForkNumber forknum,
+ BlockNumber blocknum, char *buffer, bool skipFsync);
+ bool (*smgr_prefetch) (SMgrRelation reln, ForkNumber forknum,
+ BlockNumber blocknum);
+ void (*smgr_read) (SMgrRelation reln, ForkNumber forknum,
+ BlockNumber blocknum, char *buffer);
+ void (*smgr_write) (SMgrRelation reln, ForkNumber forknum,
+ BlockNumber blocknum, char *buffer, bool skipFsync);
+ void (*smgr_writeback) (SMgrRelation reln, ForkNumber forknum,
+ BlockNumber blocknum, BlockNumber nblocks);
+ BlockNumber (*smgr_nblocks) (SMgrRelation reln, ForkNumber forknum);
+ void (*smgr_truncate) (SMgrRelation reln, ForkNumber forknum,
+ BlockNumber nblocks);
+ void (*smgr_immedsync) (SMgrRelation reln, ForkNumber forknum);
+} f_smgr;
+
+typedef void (*smgr_init_hook_type) (void);
+typedef void (*smgr_shutdown_hook_type) (void);
+extern PGDLLIMPORT smgr_init_hook_type smgr_init_hook;
+extern PGDLLIMPORT smgr_shutdown_hook_type smgr_shutdown_hook;
+extern void smgr_init_standard(void);
+extern void smgr_shutdown_standard(void);
+
+
+typedef const f_smgr *(*smgr_hook_type) (BackendId backend, RelFileNode rnode);
+extern PGDLLIMPORT smgr_hook_type smgr_hook;
+extern const f_smgr *smgr_standard(BackendId backend, RelFileNode rnode);
+
+extern const f_smgr *smgr(BackendId backend, RelFileNode rnode);
+
extern void smgrinit(void);
extern SMgrRelation smgropen(RelFileNode rnode, BackendId backend);
extern bool smgrexists(SMgrRelation reln, ForkNumber forknum);
--
2.25.1
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Extensible storage manager API - smgr hooks
@ 2021-06-30 02:36 Yura Sokolov <[email protected]>
parent: Anastasia Lubennikova <[email protected]>
0 siblings, 1 reply; 7+ messages in thread
From: Yura Sokolov @ 2021-06-30 02:36 UTC (permalink / raw)
To: Anastasia Lubennikova <[email protected]>; +Cc: [email protected]
Anastasia Lubennikova писал 2021-06-30 00:49:
> Hi, hackers!
>
> Many recently discussed features can make use of an extensible storage
> manager API. Namely, storage level compression and encryption [1],
> [2], [3], disk quota feature [4], SLRU storage changes [5], and any
> other features that may want to substitute PostgreSQL storage layer
> with their implementation (i.e. lazy_restore [6]).
>
> Attached is a proposal to change smgr API to make it extensible. The
> idea is to add a hook for plugins to get control in smgr and define
> custom storage managers. The patch replaces smgrsw[] array and smgr_sw
> selector with smgr() function that loads f_smgr implementation.
>
> As before it has only one implementation - smgr_md, which is wrapped
> into smgr_standard().
>
> To create custom implementation, a developer needs to implement smgr
> API functions
> static const struct f_smgr smgr_custom =
> {
> .smgr_init = custominit,
> ...
> }
>
> create a hook function
>
> const f_smgr * smgr_custom(BackendId backend, RelFileNode rnode)
> {
> //Here we can also add some logic and chose which smgr to use
> based on rnode and backend
> return &smgr_custom;
> }
>
> and finally set the hook:
> smgr_hook = smgr_custom;
>
> [1]
> https://www.postgresql.org/message-id/flat/[email protected]
> [2]
> https://www.postgresql.org/message-id/flat/272dd2d9.e52a.17235f2c050.Coremail.chjischj%40163.com
> [3] https://postgrespro.com/docs/enterprise/9.6/cfs
> [4]
> https://www.postgresql.org/message-id/flat/CAB0yre%3DRP_ho6Bq4cV23ELKxRcfhV2Yqrb1zHp0RfUPEWCnBRw%40m...
> [5]
> https://www.postgresql.org/message-id/flat/20180814213500.GA74618%4060f81dc409fc.ant.amazon.com
> [6]
> https://wiki.postgresql.org/wiki/PGCon_2021_Fun_With_WAL#Lazy_Restore
>
> --
>
> Best regards,
> Lubennikova Anastasia
Good day, Anastasia.
I also think smgr should be extended with different implementations
aside of md.
But which way concrete implementation will be chosen for particular
relation?
I believe it should be (immutable!) property of tablespace, and should
be passed
to smgropen. Patch in current state doesn't show clear way to distinct
different
implementations per relation.
I don't think patch should be that invasive. smgrsw could pointer to
array instead of static array as it is of now, and then reln->smgr_which
will remain with same meaning. Yep it then will need a way to select
specific
implementation, but something like `char smgr_name[NAMEDATALEN]` field
with
linear search in (i believe) small smgrsw array should be enough.
Maybe I'm missing something?
regards,
Sokolov Yura.
Attachments:
[text/x-patch] 0001-smgr_api.patch (12.9K, ../../[email protected]/2-0001-smgr_api.patch)
download | inline diff:
From 90085398f5ecc90d6b7caa318bd3d5f2867ef95c Mon Sep 17 00:00:00 2001
From: anastasia <[email protected]>
Date: Tue, 29 Jun 2021 22:16:26 +0300
Subject: [PATCH] smgr_api.patch
Make smgr API pluggable. Add smgr_hook that can be used to define custom storage managers.
Remove smgrsw[] array and smgr_sw selector. Instead, smgropen() uses smgr() function to load
f_smgr implementation using smgr_hook.
Also add smgr_init_hook and smgr_shutdown_hook.
And a lot of mechanical changes in smgr.c functions.
---
src/backend/storage/smgr/smgr.c | 136 ++++++++++++++------------------
src/include/storage/smgr.h | 56 ++++++++++++-
2 files changed, 116 insertions(+), 76 deletions(-)
diff --git a/src/backend/storage/smgr/smgr.c b/src/backend/storage/smgr/smgr.c
index 4dc24649df..5f1981a353 100644
--- a/src/backend/storage/smgr/smgr.c
+++ b/src/backend/storage/smgr/smgr.c
@@ -26,47 +26,8 @@
#include "utils/hsearch.h"
#include "utils/inval.h"
-
-/*
- * This struct of function pointers defines the API between smgr.c and
- * any individual storage manager module. Note that smgr subfunctions are
- * generally expected to report problems via elog(ERROR). An exception is
- * that smgr_unlink should use elog(WARNING), rather than erroring out,
- * because we normally unlink relations during post-commit/abort cleanup,
- * and so it's too late to raise an error. Also, various conditions that
- * would normally be errors should be allowed during bootstrap and/or WAL
- * recovery --- see comments in md.c for details.
- */
-typedef struct f_smgr
-{
- void (*smgr_init) (void); /* may be NULL */
- void (*smgr_shutdown) (void); /* may be NULL */
- void (*smgr_open) (SMgrRelation reln);
- void (*smgr_close) (SMgrRelation reln, ForkNumber forknum);
- void (*smgr_create) (SMgrRelation reln, ForkNumber forknum,
- bool isRedo);
- bool (*smgr_exists) (SMgrRelation reln, ForkNumber forknum);
- void (*smgr_unlink) (RelFileNodeBackend rnode, ForkNumber forknum,
- bool isRedo);
- void (*smgr_extend) (SMgrRelation reln, ForkNumber forknum,
- BlockNumber blocknum, char *buffer, bool skipFsync);
- bool (*smgr_prefetch) (SMgrRelation reln, ForkNumber forknum,
- BlockNumber blocknum);
- void (*smgr_read) (SMgrRelation reln, ForkNumber forknum,
- BlockNumber blocknum, char *buffer);
- void (*smgr_write) (SMgrRelation reln, ForkNumber forknum,
- BlockNumber blocknum, char *buffer, bool skipFsync);
- void (*smgr_writeback) (SMgrRelation reln, ForkNumber forknum,
- BlockNumber blocknum, BlockNumber nblocks);
- BlockNumber (*smgr_nblocks) (SMgrRelation reln, ForkNumber forknum);
- void (*smgr_truncate) (SMgrRelation reln, ForkNumber forknum,
- BlockNumber nblocks);
- void (*smgr_immedsync) (SMgrRelation reln, ForkNumber forknum);
-} f_smgr;
-
-static const f_smgr smgrsw[] = {
+static const f_smgr smgr_md = {
/* magnetic disk */
- {
.smgr_init = mdinit,
.smgr_shutdown = NULL,
.smgr_open = mdopen,
@@ -82,11 +43,8 @@ static const f_smgr smgrsw[] = {
.smgr_nblocks = mdnblocks,
.smgr_truncate = mdtruncate,
.smgr_immedsync = mdimmedsync,
- }
};
-static const int NSmgr = lengthof(smgrsw);
-
/*
* Each backend has a hashtable that stores all extant SMgrRelation objects.
* In addition, "unowned" SMgrRelation objects are chained together in a list.
@@ -110,13 +68,10 @@ static void smgrshutdown(int code, Datum arg);
void
smgrinit(void)
{
- int i;
+ if (smgr_init_hook)
+ (*smgr_init_hook)();
- for (i = 0; i < NSmgr; i++)
- {
- if (smgrsw[i].smgr_init)
- smgrsw[i].smgr_init();
- }
+ smgr_init_standard();
/* register the shutdown proc */
on_proc_exit(smgrshutdown, 0);
@@ -128,15 +83,50 @@ smgrinit(void)
static void
smgrshutdown(int code, Datum arg)
{
- int i;
+ if (smgr_shutdown_hook)
+ (*smgr_shutdown_hook)();
+
+ smgr_shutdown_standard();
+}
+
+/* Hooks for plugins to get control in smgr */
+smgr_hook_type smgr_hook = NULL;
+smgr_init_hook_type smgr_init_hook = NULL;
+smgr_shutdown_hook_type smgr_shutdown_hook = NULL;
+
+const f_smgr *
+smgr_standard(BackendId backend, RelFileNode rnode)
+{
+ return &smgr_md;
+}
- for (i = 0; i < NSmgr; i++)
+void
+smgr_init_standard(void)
+{
+ mdinit();
+}
+
+void
+smgr_shutdown_standard(void)
+{
+}
+
+const f_smgr *
+smgr(BackendId backend, RelFileNode rnode)
+{
+ const f_smgr *result;
+
+ if (smgr_hook)
{
- if (smgrsw[i].smgr_shutdown)
- smgrsw[i].smgr_shutdown();
+ result = (*smgr_hook)(backend, rnode);
}
+ else
+ result = smgr_standard(backend, rnode);
+
+ return result;
}
+
/*
* smgropen() -- Return an SMgrRelation object, creating it if need be.
*
@@ -176,10 +166,11 @@ smgropen(RelFileNode rnode, BackendId backend)
reln->smgr_targblock = InvalidBlockNumber;
for (int i = 0; i <= MAX_FORKNUM; ++i)
reln->smgr_cached_nblocks[i] = InvalidBlockNumber;
- reln->smgr_which = 0; /* we only have md.c at present */
+
+ reln->smgr = smgr(backend, rnode);
/* implementation-specific initialization */
- smgrsw[reln->smgr_which].smgr_open(reln);
+ (*reln->smgr).smgr_open(reln);
/* it has no owner yet */
dlist_push_tail(&unowned_relns, &reln->node);
@@ -246,7 +237,7 @@ smgrclearowner(SMgrRelation *owner, SMgrRelation reln)
bool
smgrexists(SMgrRelation reln, ForkNumber forknum)
{
- return smgrsw[reln->smgr_which].smgr_exists(reln, forknum);
+ return (*reln->smgr).smgr_exists(reln, forknum);
}
/*
@@ -259,7 +250,7 @@ smgrclose(SMgrRelation reln)
ForkNumber forknum;
for (forknum = 0; forknum <= MAX_FORKNUM; forknum++)
- smgrsw[reln->smgr_which].smgr_close(reln, forknum);
+ (*reln->smgr).smgr_close(reln, forknum);
owner = reln->smgr_owner;
@@ -332,7 +323,7 @@ smgrclosenode(RelFileNodeBackend rnode)
void
smgrcreate(SMgrRelation reln, ForkNumber forknum, bool isRedo)
{
- smgrsw[reln->smgr_which].smgr_create(reln, forknum, isRedo);
+ (*reln->smgr).smgr_create(reln, forknum, isRedo);
}
/*
@@ -360,12 +351,10 @@ smgrdosyncall(SMgrRelation *rels, int nrels)
*/
for (i = 0; i < nrels; i++)
{
- int which = rels[i]->smgr_which;
-
for (forknum = 0; forknum <= MAX_FORKNUM; forknum++)
{
- if (smgrsw[which].smgr_exists(rels[i], forknum))
- smgrsw[which].smgr_immedsync(rels[i], forknum);
+ if ((*rels[i]->smgr).smgr_exists(rels[i], forknum))
+ (*rels[i]->smgr).smgr_immedsync(rels[i], forknum);
}
}
}
@@ -404,13 +393,12 @@ smgrdounlinkall(SMgrRelation *rels, int nrels, bool isRedo)
for (i = 0; i < nrels; i++)
{
RelFileNodeBackend rnode = rels[i]->smgr_rnode;
- int which = rels[i]->smgr_which;
rnodes[i] = rnode;
/* Close the forks at smgr level */
for (forknum = 0; forknum <= MAX_FORKNUM; forknum++)
- smgrsw[which].smgr_close(rels[i], forknum);
+ (*rels[i]->smgr).smgr_close(rels[i], forknum);
}
/*
@@ -439,10 +427,8 @@ smgrdounlinkall(SMgrRelation *rels, int nrels, bool isRedo)
for (i = 0; i < nrels; i++)
{
- int which = rels[i]->smgr_which;
-
for (forknum = 0; forknum <= MAX_FORKNUM; forknum++)
- smgrsw[which].smgr_unlink(rnodes[i], forknum, isRedo);
+ (*rels[i]->smgr).smgr_unlink(rnodes[i], forknum, isRedo);
}
pfree(rnodes);
@@ -462,7 +448,7 @@ void
smgrextend(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
char *buffer, bool skipFsync)
{
- smgrsw[reln->smgr_which].smgr_extend(reln, forknum, blocknum,
+ (*reln->smgr).smgr_extend(reln, forknum, blocknum,
buffer, skipFsync);
/*
@@ -486,7 +472,7 @@ smgrextend(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
bool
smgrprefetch(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum)
{
- return smgrsw[reln->smgr_which].smgr_prefetch(reln, forknum, blocknum);
+ return (*reln->smgr).smgr_prefetch(reln, forknum, blocknum);
}
/*
@@ -501,7 +487,7 @@ void
smgrread(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
char *buffer)
{
- smgrsw[reln->smgr_which].smgr_read(reln, forknum, blocknum, buffer);
+ (*reln->smgr).smgr_read(reln, forknum, blocknum, buffer);
}
/*
@@ -523,7 +509,7 @@ void
smgrwrite(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
char *buffer, bool skipFsync)
{
- smgrsw[reln->smgr_which].smgr_write(reln, forknum, blocknum,
+ (*reln->smgr).smgr_write(reln, forknum, blocknum,
buffer, skipFsync);
}
@@ -536,7 +522,7 @@ void
smgrwriteback(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
BlockNumber nblocks)
{
- smgrsw[reln->smgr_which].smgr_writeback(reln, forknum, blocknum,
+ (*reln->smgr).smgr_writeback(reln, forknum, blocknum,
nblocks);
}
@@ -554,7 +540,7 @@ smgrnblocks(SMgrRelation reln, ForkNumber forknum)
if (result != InvalidBlockNumber)
return result;
- result = smgrsw[reln->smgr_which].smgr_nblocks(reln, forknum);
+ result = (*reln->smgr).smgr_nblocks(reln, forknum);
reln->smgr_cached_nblocks[forknum] = result;
@@ -620,7 +606,7 @@ smgrtruncate(SMgrRelation reln, ForkNumber *forknum, int nforks, BlockNumber *nb
/* Make the cached size is invalid if we encounter an error. */
reln->smgr_cached_nblocks[forknum[i]] = InvalidBlockNumber;
- smgrsw[reln->smgr_which].smgr_truncate(reln, forknum[i], nblocks[i]);
+ (*reln->smgr).smgr_truncate(reln, forknum[i], nblocks[i]);
/*
* We might as well update the local smgr_cached_nblocks values. The
@@ -659,7 +645,7 @@ smgrtruncate(SMgrRelation reln, ForkNumber *forknum, int nforks, BlockNumber *nb
void
smgrimmedsync(SMgrRelation reln, ForkNumber forknum)
{
- smgrsw[reln->smgr_which].smgr_immedsync(reln, forknum);
+ (*reln->smgr).smgr_immedsync(reln, forknum);
}
/*
diff --git a/src/include/storage/smgr.h b/src/include/storage/smgr.h
index a6fbf7b6a6..19c804de57 100644
--- a/src/include/storage/smgr.h
+++ b/src/include/storage/smgr.h
@@ -18,6 +18,8 @@
#include "storage/block.h"
#include "storage/relfilenode.h"
+struct f_smgr;
+
/*
* smgr.c maintains a table of SMgrRelation objects, which are essentially
* cached file handles. An SMgrRelation is created (if not already present)
@@ -59,7 +61,7 @@ typedef struct SMgrRelationData
* Fields below here are intended to be private to smgr.c and its
* submodules. Do not touch them from elsewhere.
*/
- int smgr_which; /* storage manager selector */
+ const struct f_smgr *smgr; /* storage manager selector */
/*
* for md.c; per-fork arrays of the number of open segments
@@ -77,6 +79,58 @@ typedef SMgrRelationData *SMgrRelation;
#define SmgrIsTemp(smgr) \
RelFileNodeBackendIsTemp((smgr)->smgr_rnode)
+
+/*
+ * This struct of function pointers defines the API between smgr.c and
+ * any individual storage manager module. Note that smgr subfunctions are
+ * generally expected to report problems via elog(ERROR). An exception is
+ * that smgr_unlink should use elog(WARNING), rather than erroring out,
+ * because we normally unlink relations during post-commit/abort cleanup,
+ * and so it's too late to raise an error. Also, various conditions that
+ * would normally be errors should be allowed during bootstrap and/or WAL
+ * recovery --- see comments in md.c for details.
+ */
+typedef struct f_smgr
+{
+ void (*smgr_init) (void); /* may be NULL */
+ void (*smgr_shutdown) (void); /* may be NULL */
+ void (*smgr_open) (SMgrRelation reln);
+ void (*smgr_close) (SMgrRelation reln, ForkNumber forknum);
+ void (*smgr_create) (SMgrRelation reln, ForkNumber forknum,
+ bool isRedo);
+ bool (*smgr_exists) (SMgrRelation reln, ForkNumber forknum);
+ void (*smgr_unlink) (RelFileNodeBackend rnode, ForkNumber forknum,
+ bool isRedo);
+ void (*smgr_extend) (SMgrRelation reln, ForkNumber forknum,
+ BlockNumber blocknum, char *buffer, bool skipFsync);
+ bool (*smgr_prefetch) (SMgrRelation reln, ForkNumber forknum,
+ BlockNumber blocknum);
+ void (*smgr_read) (SMgrRelation reln, ForkNumber forknum,
+ BlockNumber blocknum, char *buffer);
+ void (*smgr_write) (SMgrRelation reln, ForkNumber forknum,
+ BlockNumber blocknum, char *buffer, bool skipFsync);
+ void (*smgr_writeback) (SMgrRelation reln, ForkNumber forknum,
+ BlockNumber blocknum, BlockNumber nblocks);
+ BlockNumber (*smgr_nblocks) (SMgrRelation reln, ForkNumber forknum);
+ void (*smgr_truncate) (SMgrRelation reln, ForkNumber forknum,
+ BlockNumber nblocks);
+ void (*smgr_immedsync) (SMgrRelation reln, ForkNumber forknum);
+} f_smgr;
+
+typedef void (*smgr_init_hook_type) (void);
+typedef void (*smgr_shutdown_hook_type) (void);
+extern PGDLLIMPORT smgr_init_hook_type smgr_init_hook;
+extern PGDLLIMPORT smgr_shutdown_hook_type smgr_shutdown_hook;
+extern void smgr_init_standard(void);
+extern void smgr_shutdown_standard(void);
+
+
+typedef const f_smgr *(*smgr_hook_type) (BackendId backend, RelFileNode rnode);
+extern PGDLLIMPORT smgr_hook_type smgr_hook;
+extern const f_smgr *smgr_standard(BackendId backend, RelFileNode rnode);
+
+extern const f_smgr *smgr(BackendId backend, RelFileNode rnode);
+
extern void smgrinit(void);
extern SMgrRelation smgropen(RelFileNode rnode, BackendId backend);
extern bool smgrexists(SMgrRelation reln, ForkNumber forknum);
--
2.25.1
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Extensible storage manager API - smgr hooks
@ 2022-06-16 08:41 Kirill Reshke <[email protected]>
parent: Yura Sokolov <[email protected]>
0 siblings, 1 reply; 7+ messages in thread
From: Kirill Reshke @ 2022-06-16 08:41 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Yura Sokolov <[email protected]>; Anastasia Lubennikova <[email protected]>; [email protected]
Hello Yura and Anastasia.
I have tried to implement per-relation SMGR approach, and faced with a
serious problem with redo.
So, to implement per-relation SMGR feature i have tried to do things
similar to custom table AM apporach: that is, we can define our custom SMGR
in an extention (which defines smgr handle) and then use this SMGR in
relation definition. like this:
```postgres=# create extension proxy_smgr ;
CREATE EXTENSION
postgres=# select * from pg_smgr ;
oid | smgrname | smgrhandler
-------+------------+--------------------
4646 | md | smgr_md_handler
16386 | proxy_smgr | proxy_smgr_handler
(2 rows)
postgres=# create table tt(i int) storage manager proxy_smgr_handler;
ERROR: storage manager "proxy_smgr_handler" does not exist
postgres=# create table tt(i int) storage manager proxy_smgr;
INFO: proxy open 1663 5 16391
INFO: proxy create 16391
INFO: proxy close, 16391
INFO: proxy close, 16391
INFO: proxy close, 16391
INFO: proxy close, 16391
CREATE TABLE
postgres=# select * from tt;
INFO: proxy open 1663 5 16391
INFO: proxy nblocks 16391
INFO: proxy nblocks 16391
i
---
(0 rows)
postgres=# insert into tt values(1);
INFO: proxy exists 16391
INFO: proxy nblocks 16391
INFO: proxy nblocks 16391
INFO: proxcy extend 16391
INSERT 0 1
postgres=# select * from tt;
INFO: proxy nblocks 16391
INFO: proxy nblocks 16391
i
---
1
(1 row)
```
extention sql files looks like this:
```
CREATE FUNCTION proxy_smgr_handler(internal)
RETURNS table_smgr_handler
AS 'MODULE_PATHNAME'
LANGUAGE C;
-- Storage manager
CREATE STORAGE MANAGER proxy_smgr HANDLER proxy_smgr_handler;
```
To do this i have defined catalog relation pg_smgr where i store smgr`s
handlers and use this relation when we need to open some other(non-catalog)
relations in smgropen function. The patch almost passes regression tests(8
of 214 tests failed.) but it fails on first checkpoint or in crash
recorvery. Also, i have changed WAL format, added SMGR oid to each WAL
record with RelFileNode structure. Why do we need WAL changes? well, i
tried to solve folowing issue.
As i mentioned, there is a problem with redo, with is: we cannot do
syscache search to get relation`s SMGR to apply wal, because syscache is
not initialized during redo (crash recovery). As i understand, syscache is
not initialised because system catalogs are not consistent until crash
recovery is done.
So, thants it, I decided to write to this thread to get feedback and
understand how best to solve the problem with redo.
What do you think?
On Thu, Jun 16, 2022 at 1:38 PM Andres Freund <[email protected]> wrote:
> Hi,
>
> On 2021-06-30 05:36:11 +0300, Yura Sokolov wrote:
> > Anastasia Lubennikova писал 2021-06-30 00:49:
> > > Hi, hackers!
> > >
> > > Many recently discussed features can make use of an extensible storage
> > > manager API. Namely, storage level compression and encryption [1],
> > > [2], [3], disk quota feature [4], SLRU storage changes [5], and any
> > > other features that may want to substitute PostgreSQL storage layer
> > > with their implementation (i.e. lazy_restore [6]).
> > >
> > > Attached is a proposal to change smgr API to make it extensible. The
> > > idea is to add a hook for plugins to get control in smgr and define
> > > custom storage managers. The patch replaces smgrsw[] array and smgr_sw
> > > selector with smgr() function that loads f_smgr implementation.
> > >
> > > As before it has only one implementation - smgr_md, which is wrapped
> > > into smgr_standard().
> > >
> > > To create custom implementation, a developer needs to implement smgr
> > > API functions
> > > static const struct f_smgr smgr_custom =
> > > {
> > > .smgr_init = custominit,
> > > ...
> > > }
> > >
> > > create a hook function
> > >
> > > const f_smgr * smgr_custom(BackendId backend, RelFileNode rnode)
> > > {
> > > //Here we can also add some logic and chose which smgr to use
> > > based on rnode and backend
> > > return &smgr_custom;
> > > }
> > >
> > > and finally set the hook:
> > > smgr_hook = smgr_custom;
> > >
> > > [1]
> > >
> https://www.postgresql.org/message-id/flat/[email protected]
> > > [2]
> > >
> https://www.postgresql.org/message-id/flat/272dd2d9.e52a.17235f2c050.Coremail.chjischj%40163.com
> > > [3] https://postgrespro.com/docs/enterprise/9.6/cfs
> > > [4]
> > >
> https://www.postgresql.org/message-id/flat/CAB0yre%3DRP_ho6Bq4cV23ELKxRcfhV2Yqrb1zHp0RfUPEWCnBRw%40m...
> > > [5]
> > >
> https://www.postgresql.org/message-id/flat/20180814213500.GA74618%4060f81dc409fc.ant.amazon.com
> > > [6]
> > > https://wiki.postgresql.org/wiki/PGCon_2021_Fun_With_WAL#Lazy_Restore
> > >
> > > --
> > >
> > > Best regards,
> > > Lubennikova Anastasia
> >
> > Good day, Anastasia.
> >
> > I also think smgr should be extended with different implementations
> aside of
> > md.
> > But which way concrete implementation will be chosen for particular
> > relation?
> > I believe it should be (immutable!) property of tablespace, and should be
> > passed
> > to smgropen. Patch in current state doesn't show clear way to distinct
> > different
> > implementations per relation.
> >
> > I don't think patch should be that invasive. smgrsw could pointer to
> > array instead of static array as it is of now, and then reln->smgr_which
> > will remain with same meaning. Yep it then will need a way to select
> > specific
> > implementation, but something like `char smgr_name[NAMEDATALEN]` field
> with
> > linear search in (i believe) small smgrsw array should be enough.
> >
> > Maybe I'm missing something?
>
> There has been no activity on this thread for > 6 months. Therefore I'm
> marking it as returned with feedback. Anastasia, if you want to work on
> this,
> please do, but there's obviously no way it can be merged into 15...
>
> Greetings,
>
> Andres
>
>
>
>
>
Attachments:
[application/octet-stream] v1-0001-Add-create-storage-manager-ddl-and-routines.patch (80.5K, ../../CADVKa1VvBzSg=07Hw13QoKpd4b=A1nqcvHPKR=UVmSt0KKdbRw@mail.gmail.com/3-v1-0001-Add-create-storage-manager-ddl-and-routines.patch)
download | inline diff:
From acd602598192b1a0b6c7cd7c77c2a16597f7af99 Mon Sep 17 00:00:00 2001
From: reshke kirill <[email protected]>
Date: Sat, 11 Jun 2022 11:54:10 +0300
Subject: [PATCH v1] Add create storage manager ddl and routines
add create storage manager ddl and support
pass smgr routines to smgropen
boostrap now work
post boostrap ok
initdb ok
fixes
regression passed with 8 failed tests
add smgr extension and fixes
---
contrib/rsmgr/Makefile | 9 +
contrib/rsmgr/proxy_smgr--1.0.sql | 12 ++
contrib/rsmgr/proxy_smgr.c | 123 +++++++++++
contrib/rsmgr/proxy_smgr.control | 5 +
src/backend/access/heap/heapam_handler.c | 2 +
src/backend/access/rmgrdesc/heapdesc.c | 3 +-
src/backend/access/rmgrdesc/xlogdesc.c | 4 +-
src/backend/access/transam/xlogrecovery.c | 4 +-
src/backend/bootstrap/bootparse.y | 3 +
src/backend/catalog/Makefile | 4 +-
src/backend/catalog/aclchk.c | 6 +-
src/backend/catalog/catalog.c | 4 +-
src/backend/catalog/dependency.c | 6 +
src/backend/catalog/heap.c | 10 +-
src/backend/catalog/index.c | 7 +-
src/backend/catalog/objectaddress.c | 66 ++++++
src/backend/catalog/toasting.c | 2 +
src/backend/commands/Makefile | 1 +
src/backend/commands/alter.c | 1 +
src/backend/commands/cluster.c | 6 +-
src/backend/commands/dbcommands.c | 3 +
src/backend/commands/event_trigger.c | 4 +
src/backend/commands/matview.c | 2 +-
src/backend/commands/smgrcmds.c | 197 ++++++++++++++++++
src/backend/commands/tablecmds.c | 29 ++-
src/backend/nodes/equalfuncs.c | 13 ++
src/backend/parser/gram.y | 51 ++++-
src/backend/storage/smgr/Makefile | 4 +-
src/backend/storage/smgr/smgr.c | 84 ++------
src/backend/storage/smgr/smgr_handler.c | 75 +++++++
src/backend/storage/smgr/smgrapi.c | 96 +++++++++
src/backend/tcop/utility.c | 15 ++
src/backend/utils/adt/pseudotypes.c | 1 +
src/backend/utils/cache/relcache.c | 59 +++++-
src/backend/utils/cache/syscache.c | 23 ++
src/bin/psql/describe.c | 1 +
src/include/catalog/catalog.h | 2 +-
src/include/catalog/dependency.h | 1 +
src/include/catalog/heap.h | 2 +
src/include/catalog/pg_class.h | 3 +
src/include/catalog/pg_proc.dat | 13 ++
src/include/catalog/pg_smgr.dat | 17 ++
src/include/catalog/pg_smgr.h | 52 +++++
src/include/catalog/pg_type.dat | 5 +
src/include/commands/cluster.h | 2 +-
src/include/commands/defrem.h | 5 +
src/include/nodes/nodes.h | 2 +
src/include/nodes/parsenodes.h | 14 ++
src/include/parser/kwlist.h | 1 +
src/include/storage/md.h | 3 +
src/include/storage/relfilenode.h | 1 +
src/include/storage/smgr.h | 35 ++--
src/include/tcop/cmdtaglist.h | 2 +
src/include/utils/rel.h | 7 +
src/include/utils/relcache.h | 2 +
src/include/utils/syscache.h | 2 +
.../modules/test_oat_hooks/test_oat_hooks.c | 6 +
57 files changed, 998 insertions(+), 114 deletions(-)
create mode 100644 contrib/rsmgr/Makefile
create mode 100644 contrib/rsmgr/proxy_smgr--1.0.sql
create mode 100644 contrib/rsmgr/proxy_smgr.c
create mode 100644 contrib/rsmgr/proxy_smgr.control
create mode 100644 src/backend/commands/smgrcmds.c
create mode 100644 src/backend/storage/smgr/smgr_handler.c
create mode 100644 src/backend/storage/smgr/smgrapi.c
create mode 100644 src/include/catalog/pg_smgr.dat
create mode 100644 src/include/catalog/pg_smgr.h
diff --git a/contrib/rsmgr/Makefile b/contrib/rsmgr/Makefile
new file mode 100644
index 0000000000..d065209bdf
--- /dev/null
+++ b/contrib/rsmgr/Makefile
@@ -0,0 +1,9 @@
+MODULES = proxy_smgr
+
+EXTENSION = proxy_smgr
+DATA = proxy_smgr--1.0.sql
+PGFILEDESC = "proxy_smgr - template table smgra"
+
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
diff --git a/contrib/rsmgr/proxy_smgr--1.0.sql b/contrib/rsmgr/proxy_smgr--1.0.sql
new file mode 100644
index 0000000000..11e743c3dd
--- /dev/null
+++ b/contrib/rsmgr/proxy_smgr--1.0.sql
@@ -0,0 +1,12 @@
+/* proxy_smgr/proxy_smgr--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION proxy_smgr" to load this file. \quit
+
+CREATE FUNCTION proxy_smgr_handler(internal)
+RETURNS table_smgr_handler
+AS 'MODULE_PATHNAME'
+LANGUAGE C;
+
+-- Storage manager
+CREATE STORAGE MANAGER proxy_smgr HANDLER proxy_smgr_handler;
diff --git a/contrib/rsmgr/proxy_smgr.c b/contrib/rsmgr/proxy_smgr.c
new file mode 100644
index 0000000000..955dbd1aea
--- /dev/null
+++ b/contrib/rsmgr/proxy_smgr.c
@@ -0,0 +1,123 @@
+
+#include "postgres.h"
+
+#include <math.h>
+
+#include "miscadmin.h"
+
+#include "access/tableam.h"
+#include "access/heapam.h"
+#include "access/amapi.h"
+#include "catalog/index.h"
+#include "commands/vacuum.h"
+#include "storage/smgr.h"
+#include "storage/md.h"
+#include "utils/elog.h"
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(proxy_smgr_handler);
+
+
+/* ------------------------------------------------------------------------
+ * Definition of the blackhole table access method.
+ * ------------------------------------------------------------------------
+ */
+
+void proxyinit(void) {
+ elog(INFO, "proxy init");
+ mdinit();
+}
+
+void proxyopen(SMgrRelation reln) {
+ elog(INFO, "proxy open %d %d %d", reln->smgr_rnode.node.spcNode, reln->smgr_rnode.node.dbNode, reln->smgr_rnode.node.relNode);
+ mdopen(reln);
+}
+
+void proxyclose(SMgrRelation reln, ForkNumber forknum) {
+ elog(INFO, "proxy close, %d", reln->smgr_rnode.node.relNode);
+ mdclose(reln, forknum);
+}
+
+void proxycreate(SMgrRelation reln, ForkNumber forknum, bool isRedo) {
+ elog(INFO, "proxy create %d", reln->smgr_rnode.node.relNode);
+
+ mdcreate(reln, forknum, isRedo);
+}
+
+bool proxyexists(SMgrRelation reln, ForkNumber forknum) {
+ elog(INFO, "proxy exists %d", reln->smgr_rnode.node.relNode);
+ return mdexists(reln, forknum);
+}
+
+void proxyunlink(RelFileNodeBackend rnode, ForkNumber forknum, bool isRedo) {
+ elog(INFO, "proxy unlink %d", rnode.node.relNode);
+ mdunlink(rnode, forknum, isRedo);
+}
+
+void proxyextend(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum, char *buffer, bool skipFsync) {
+ elog(INFO, "proxcy extend %d", reln->smgr_rnode.node.relNode);
+ mdextend(reln, forknum, blocknum, buffer, skipFsync);
+}
+
+bool proxyprefetch(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum) {
+ elog(INFO, "proxy prefecth %d", reln->smgr_rnode.node.relNode);
+ mdprefetch(reln, forknum, blocknum);
+}
+
+void proxyread(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum, char *buffer) {
+ elog(INFO, "proxy read %d", reln->smgr_rnode.node.relSmgr);
+ mdread(reln, forknum, blocknum, buffer);
+}
+
+void proxywrite(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum, char *buffer, bool skipFsync) {
+ elog(INFO, "proxy write %d", reln->smgr_rnode.node.relNode);
+ mdwrite(reln, forknum, blocknum, buffer, skipFsync);
+}
+
+void proxywriteback(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum, BlockNumber nblocks) {
+ elog(INFO, "proxy writeback %d", reln->smgr_rnode.node.relNode);
+ mdwriteback(reln, forknum, blocknum, nblocks);
+}
+
+BlockNumber proxynblocks(SMgrRelation reln, ForkNumber forknum) {
+ elog(INFO, "proxy nblocks %d", reln->smgr_rnode.node.relNode);
+ return mdnblocks(reln, forknum);
+}
+
+void proxytruncate(SMgrRelation reln, ForkNumber forknum, BlockNumber nblocks) {
+ elog(INFO, "proxy truncate %d", reln->smgr_rnode.node.relNode);
+ return mdtruncate(reln, forknum, nblocks);
+}
+
+void proxyimmedsync(SMgrRelation reln, ForkNumber forknum) {
+ elog(INFO, "proxy immedsync %d", reln->smgr_rnode.node.relNode);
+ return mdnblocks(reln, forknum);
+}
+
+static const TableSmgr proxy_smgr_methods = {
+ .type = T_TableSmgr,
+
+ .smgr_init = proxyinit,
+ .smgr_shutdown = NULL,
+ .smgr_open = proxyopen,
+ .smgr_close = proxyclose,
+ .smgr_create = proxycreate,
+ .smgr_exists = proxyexists,
+ .smgr_unlink = proxyunlink,
+ .smgr_extend = proxyextend,
+ .smgr_prefetch = proxyprefetch,
+ .smgr_read = proxyread,
+ .smgr_write = proxywrite,
+ .smgr_writeback = proxywriteback,
+ .smgr_nblocks = proxynblocks,
+ .smgr_truncate = proxytruncate,
+ .smgr_immedsync = proxyimmedsync,
+};
+
+
+Datum
+proxy_smgr_handler(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_POINTER(&proxy_smgr_methods);
+}
diff --git a/contrib/rsmgr/proxy_smgr.control b/contrib/rsmgr/proxy_smgr.control
new file mode 100644
index 0000000000..053d67d74a
--- /dev/null
+++ b/contrib/rsmgr/proxy_smgr.control
@@ -0,0 +1,5 @@
+# proxy_smgr extension
+comment = 'template table SMGR'
+default_version = '1.0'
+module_pathname = '$libdir/proxy_smgr'
+relocatable = true
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 444f027149..bad035f39e 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -1190,6 +1190,7 @@ heapam_index_build_range_scan(Relation heapRelation,
* sanity checks
*/
Assert(OidIsValid(indexRelation->rd_rel->relam));
+ Assert(OidIsValid(indexRelation->rd_rel->relsmgr));
/* Remember if it's a system catalog */
is_system_catalog = IsSystemRelation(heapRelation);
@@ -1760,6 +1761,7 @@ heapam_index_validate_scan(Relation heapRelation,
* sanity checks
*/
Assert(OidIsValid(indexRelation->rd_rel->relam));
+ Assert(OidIsValid(indexRelation->rd_rel->relsmgr));
/*
* Need an EState for evaluation of index expressions and partial-index
diff --git a/src/backend/access/rmgrdesc/heapdesc.c b/src/backend/access/rmgrdesc/heapdesc.c
index 6238085d65..2af6d3e544 100644
--- a/src/backend/access/rmgrdesc/heapdesc.c
+++ b/src/backend/access/rmgrdesc/heapdesc.c
@@ -169,10 +169,11 @@ heap2_desc(StringInfo buf, XLogReaderState *record)
{
xl_heap_new_cid *xlrec = (xl_heap_new_cid *) rec;
- appendStringInfo(buf, "rel %u/%u/%u; tid %u/%u",
+ appendStringInfo(buf, "rel %u/%u/%u/%u; tid %u/%u",
xlrec->target_node.spcNode,
xlrec->target_node.dbNode,
xlrec->target_node.relNode,
+ xlrec->target_node.relSmgr,
ItemPointerGetBlockNumber(&(xlrec->target_tid)),
ItemPointerGetOffsetNumber(&(xlrec->target_tid)));
appendStringInfo(buf, "; cmin: %u, cmax: %u, combo: %u",
diff --git a/src/backend/access/rmgrdesc/xlogdesc.c b/src/backend/access/rmgrdesc/xlogdesc.c
index fefc563323..18f0da9aa1 100644
--- a/src/backend/access/rmgrdesc/xlogdesc.c
+++ b/src/backend/access/rmgrdesc/xlogdesc.c
@@ -306,9 +306,9 @@ XLogRecGetBlockRefInfo(XLogReaderState *record, bool pretty,
else
{
appendStringInfo(buf,
- ", blkref #%d: rel %u/%u/%u blk %u",
+ ", blkref #%d: rel %u/%u/%u/%u blk %u",
block_id,
- rnode.spcNode, rnode.dbNode, rnode.relNode,
+ rnode.spcNode, rnode.dbNode, rnode.relNode, rnode.relSmgr,
blk);
}
diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c
index 6eba626420..11a6adda17 100644
--- a/src/backend/access/transam/xlogrecovery.c
+++ b/src/backend/access/transam/xlogrecovery.c
@@ -2181,9 +2181,9 @@ xlog_block_info(StringInfo buf, XLogReaderState *record)
forknum,
blk);
else
- appendStringInfo(buf, "; blkref #%d: rel %u/%u/%u, blk %u",
+ appendStringInfo(buf, "; blkref #%d: rel %u/%u/%u/%u, blk %u",
block_id,
- rnode.spcNode, rnode.dbNode, rnode.relNode,
+ rnode.spcNode, rnode.dbNode, rnode.relNode, rnode.relSmgr,
blk);
if (XLogRecHasBlockImage(record, block_id))
appendStringInfoString(buf, " FPW");
diff --git a/src/backend/bootstrap/bootparse.y b/src/backend/bootstrap/bootparse.y
index e5cf1b3d43..3a49595fdd 100644
--- a/src/backend/bootstrap/bootparse.y
+++ b/src/backend/bootstrap/bootparse.y
@@ -25,6 +25,7 @@
#include "catalog/pg_authid.h"
#include "catalog/pg_class.h"
#include "catalog/pg_namespace.h"
+#include "catalog/pg_smgr.h"
#include "catalog/pg_tablespace.h"
#include "catalog/toasting.h"
#include "commands/defrem.h"
@@ -205,6 +206,7 @@ Boot_CreateStmt:
$3,
InvalidOid,
HEAP_TABLE_AM_OID,
+ SMGR_MD_OID,
tupdesc,
RELKIND_RELATION,
RELPERSISTENCE_PERMANENT,
@@ -228,6 +230,7 @@ Boot_CreateStmt:
InvalidOid,
BOOTSTRAP_SUPERUSERID,
HEAP_TABLE_AM_OID,
+ SMGR_MD_OID,
tupdesc,
NIL,
RELKIND_RELATION,
diff --git a/src/backend/catalog/Makefile b/src/backend/catalog/Makefile
index 89a0221ec9..8869bc9961 100644
--- a/src/backend/catalog/Makefile
+++ b/src/backend/catalog/Makefile
@@ -72,7 +72,7 @@ CATALOG_HEADERS := \
pg_collation.h pg_parameter_acl.h pg_partitioned_table.h \
pg_range.h pg_transform.h \
pg_sequence.h pg_publication.h pg_publication_namespace.h \
- pg_publication_rel.h pg_subscription.h pg_subscription_rel.h
+ pg_publication_rel.h pg_subscription.h pg_subscription_rel.h pg_smgr.h
GENERATED_HEADERS := $(CATALOG_HEADERS:%.h=%_d.h) schemapg.h system_fk_info.h
@@ -86,7 +86,7 @@ POSTGRES_BKI_DATA = $(addprefix $(top_srcdir)/src/include/catalog/,\
pg_namespace.dat pg_opclass.dat pg_operator.dat pg_opfamily.dat \
pg_proc.dat pg_range.dat pg_tablespace.dat \
pg_ts_config.dat pg_ts_config_map.dat pg_ts_dict.dat pg_ts_parser.dat \
- pg_ts_template.dat pg_type.dat \
+ pg_ts_template.dat pg_type.dat pg_smgr.dat \
)
all: distprep generated-header-symlinks
diff --git a/src/backend/catalog/aclchk.c b/src/backend/catalog/aclchk.c
index 5f1726c095..ba727d0b2d 100644
--- a/src/backend/catalog/aclchk.c
+++ b/src/backend/catalog/aclchk.c
@@ -51,6 +51,7 @@
#include "catalog/pg_parameter_acl.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
+#include "catalog/pg_smgr.h"
#include "catalog/pg_subscription.h"
#include "catalog/pg_tablespace.h"
#include "catalog/pg_transform.h"
@@ -3638,6 +3639,7 @@ aclcheck_error(AclResult aclerr, ObjectType objtype,
case OBJECT_PUBLICATION_REL:
case OBJECT_ROLE:
case OBJECT_RULE:
+ case OBJECT_STORAGE_MANAGER:
case OBJECT_TABCONSTRAINT:
case OBJECT_TRANSFORM:
case OBJECT_TRIGGER:
@@ -3778,6 +3780,7 @@ aclcheck_error(AclResult aclerr, ObjectType objtype,
case OBJECT_PUBLICATION_NAMESPACE:
case OBJECT_PUBLICATION_REL:
case OBJECT_ROLE:
+ case OBJECT_STORAGE_MANAGER:
case OBJECT_TRANSFORM:
case OBJECT_TSPARSER:
case OBJECT_TSTEMPLATE:
@@ -6254,7 +6257,8 @@ recordExtObjInitPriv(Oid objoid, Oid classoid)
classoid == TSDictionaryRelationId ||
classoid == TSParserRelationId ||
classoid == TSTemplateRelationId ||
- classoid == TransformRelationId
+ classoid == TransformRelationId ||
+ classoid == StorageManagerRelationId
)
{
/* no ACL for these object types, so do nothing. */
diff --git a/src/backend/catalog/catalog.c b/src/backend/catalog/catalog.c
index e784538aae..9029ecf756 100644
--- a/src/backend/catalog/catalog.c
+++ b/src/backend/catalog/catalog.c
@@ -497,7 +497,7 @@ GetNewOidWithIndex(Relation relation, Oid indexId, AttrNumber oidcolumn)
* created by bootstrap have preassigned OIDs, so there's no need.
*/
Oid
-GetNewRelFileNode(Oid reltablespace, Relation pg_class, char relpersistence)
+GetNewRelFileNode(Oid reltablespace, Relation pg_class, char relpersistence, Oid relsmgr)
{
RelFileNodeBackend rnode;
char *rpath;
@@ -536,6 +536,8 @@ GetNewRelFileNode(Oid reltablespace, Relation pg_class, char relpersistence)
*/
rnode.backend = backend;
+ rnode.node.relSmgr = relsmgr;
+
do
{
CHECK_FOR_INTERRUPTS();
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index de10923391..285a728ad0 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -54,6 +54,7 @@
#include "catalog/pg_publication_rel.h"
#include "catalog/pg_rewrite.h"
#include "catalog/pg_statistic_ext.h"
+#include "catalog/pg_smgr.h"
#include "catalog/pg_subscription.h"
#include "catalog/pg_tablespace.h"
#include "catalog/pg_transform.h"
@@ -185,6 +186,7 @@ static const Oid object_classes[] = {
PublicationRelationId, /* OCLASS_PUBLICATION */
PublicationRelRelationId, /* OCLASS_PUBLICATION_REL */
SubscriptionRelationId, /* OCLASS_SUBSCRIPTION */
+ StorageManagerRelationId, /* OCLASS_SMGR */
TransformRelationId /* OCLASS_TRANSFORM */
};
@@ -1490,6 +1492,7 @@ doDeletion(const ObjectAddress *object, int flags)
case OCLASS_AMOP:
case OCLASS_AMPROC:
case OCLASS_SCHEMA:
+ case OCLASS_SMGR:
case OCLASS_TSPARSER:
case OCLASS_TSDICT:
case OCLASS_TSTEMPLATE:
@@ -2882,6 +2885,9 @@ getObjectClass(const ObjectAddress *object)
case SubscriptionRelationId:
return OCLASS_SUBSCRIPTION;
+ case StorageManagerRelationId:
+ return OCLASS_SMGR;
+
case TransformRelationId:
return OCLASS_TRANSFORM;
}
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 1803194db9..0ad8f31f4f 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -50,6 +50,7 @@
#include "catalog/pg_opclass.h"
#include "catalog/pg_partitioned_table.h"
#include "catalog/pg_statistic.h"
+#include "catalog/pg_smgr.h"
#include "catalog/pg_subscription_rel.h"
#include "catalog/pg_tablespace.h"
#include "catalog/pg_type.h"
@@ -291,6 +292,7 @@ heap_create(const char *relname,
Oid relid,
Oid relfilenode,
Oid accessmtd,
+ Oid smgrid,
TupleDesc tupDesc,
char relkind,
char relpersistence,
@@ -368,6 +370,7 @@ heap_create(const char *relname,
tupDesc,
relid,
accessmtd,
+ smgrid,
relfilenode,
reltablespace,
shared_relation,
@@ -898,6 +901,7 @@ InsertPgClassTuple(Relation pg_class_desc,
values[Anum_pg_class_reloftype - 1] = ObjectIdGetDatum(rd_rel->reloftype);
values[Anum_pg_class_relowner - 1] = ObjectIdGetDatum(rd_rel->relowner);
values[Anum_pg_class_relam - 1] = ObjectIdGetDatum(rd_rel->relam);
+ values[Anum_pg_class_relsmgr - 1] = ObjectIdGetDatum(rd_rel->relsmgr);
values[Anum_pg_class_relfilenode - 1] = ObjectIdGetDatum(rd_rel->relfilenode);
values[Anum_pg_class_reltablespace - 1] = ObjectIdGetDatum(rd_rel->reltablespace);
values[Anum_pg_class_relpages - 1] = Int32GetDatum(rd_rel->relpages);
@@ -1093,6 +1097,7 @@ heap_create_with_catalog(const char *relname,
Oid reloftypeid,
Oid ownerid,
Oid accessmtd,
+ Oid smgrid,
TupleDesc tupdesc,
List *cooked_constraints,
char relkind,
@@ -1230,7 +1235,7 @@ heap_create_with_catalog(const char *relname,
if (!OidIsValid(relid))
relid = GetNewRelFileNode(reltablespace, pg_class_desc,
- relpersistence);
+ relpersistence, smgrid);
}
/*
@@ -1275,6 +1280,7 @@ heap_create_with_catalog(const char *relname,
relid,
relfilenode,
accessmtd,
+ smgrid,
tupdesc,
relkind,
relpersistence,
@@ -1455,6 +1461,8 @@ heap_create_with_catalog(const char *relname,
add_exact_object_address(&referenced, addrs);
}
+ ObjectAddressSet(referenced, StorageManagerRelationId, smgrid);
+
record_object_address_dependencies(&myself, addrs, DEPENDENCY_NORMAL);
free_object_addresses(addrs);
}
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index bdd3c34841..45459d8fe7 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -50,6 +50,7 @@
#include "catalog/pg_inherits.h"
#include "catalog/pg_opclass.h"
#include "catalog/pg_operator.h"
+#include "catalog/pg_smgr.h"
#include "catalog/pg_tablespace.h"
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
@@ -728,6 +729,7 @@ index_create(Relation heapRelation,
Oid namespaceId;
int i;
char relpersistence;
+ Oid relsmgr;
bool isprimary = (flags & INDEX_CREATE_IS_PRIMARY) != 0;
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
@@ -758,6 +760,7 @@ index_create(Relation heapRelation,
shared_relation = heapRelation->rd_rel->relisshared;
mapped_relation = RelationIsMapped(heapRelation);
relpersistence = heapRelation->rd_rel->relpersistence;
+ relsmgr = heapRelation->rd_rel->relsmgr;
/*
* check parameters
@@ -937,7 +940,7 @@ index_create(Relation heapRelation,
else
{
indexRelationId =
- GetNewRelFileNode(tableSpaceId, pg_class, relpersistence);
+ GetNewRelFileNode(tableSpaceId, pg_class, relpersistence, relsmgr);
}
}
@@ -952,6 +955,7 @@ index_create(Relation heapRelation,
indexRelationId,
relFileNode,
accessMethodObjectId,
+ SMGR_MD_OID,
indexTupDesc,
relkind,
relpersistence,
@@ -981,6 +985,7 @@ index_create(Relation heapRelation,
*/
indexRelation->rd_rel->relowner = heapRelation->rd_rel->relowner;
indexRelation->rd_rel->relam = accessMethodObjectId;
+ indexRelation->rd_rel->relsmgr = relsmgr;
indexRelation->rd_rel->relispartition = OidIsValid(parentIndexRelid);
/*
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index 8377b4f7d4..5c39309aa2 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -53,6 +53,7 @@
#include "catalog/pg_publication_rel.h"
#include "catalog/pg_rewrite.h"
#include "catalog/pg_statistic_ext.h"
+#include "catalog/pg_smgr.h"
#include "catalog/pg_subscription.h"
#include "catalog/pg_tablespace.h"
#include "catalog/pg_transform.h"
@@ -414,6 +415,20 @@ static const ObjectPropertyType ObjectProperty[] =
OBJECT_SCHEMA,
true
},
+ {
+ "storage manager",
+ StorageManagerRelationId,
+ SmgrOidIndexId,
+ SMGROID,
+ SMGRNAME,
+ Anum_pg_smgr_oid,
+ Anum_pg_smgr_smgrname,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ -1,
+ true
+ },
{
"relation",
RelationRelationId,
@@ -843,6 +858,10 @@ static const struct object_type_map
{
"subscription", OBJECT_SUBSCRIPTION
},
+ /* OCLASS_SMGR */
+ {
+ "storage manager", OBJECT_STORAGE_MANAGER
+ },
/* OCLASS_TRANSFORM */
{
"transform", OBJECT_TRANSFORM
@@ -1335,6 +1354,11 @@ get_object_address_unqualified(ObjectType objtype,
address.objectId = get_subscription_oid(name, missing_ok);
address.objectSubId = 0;
break;
+ case OBJECT_STORAGE_MANAGER:
+ address.classId = StorageManagerRelationId;
+ address.objectId = get_smgr_oid(name, missing_ok);
+ address.objectSubId = 0;
+ break;
default:
elog(ERROR, "unrecognized objtype: %d", (int) objtype);
/* placate compiler, which doesn't know elog won't return */
@@ -2318,6 +2342,7 @@ pg_get_object_address(PG_FUNCTION_ARGS)
case OBJECT_ROLE:
case OBJECT_SCHEMA:
case OBJECT_SUBSCRIPTION:
+ case OBJECT_STORAGE_MANAGER:
case OBJECT_TABLESPACE:
if (list_length(name) != 1)
ereport(ERROR,
@@ -3261,6 +3286,26 @@ getObjectDescription(const ObjectAddress *object, bool missing_ok)
break;
}
+ case OCLASS_SMGR:
+ {
+ HeapTuple tup;
+
+ tup = SearchSysCache1(SMGROID,
+ ObjectIdGetDatum(object->objectId));
+ if (!HeapTupleIsValid(tup))
+ {
+ if (!missing_ok)
+ elog(ERROR, "cache lookup failed for storage manager %u",
+ object->objectId);
+ break;
+ }
+
+ appendStringInfo(&buffer, _("storage manager %s"),
+ NameStr(((Form_pg_smgr) GETSTRUCT(tup))->smgrname));
+ ReleaseSysCache(tup);
+ break;
+ }
+
case OCLASS_AMOP:
{
Relation amopDesc;
@@ -4493,6 +4538,10 @@ getObjectTypeDescription(const ObjectAddress *object, bool missing_ok)
appendStringInfoString(&buffer, "access method");
break;
+ case OCLASS_SMGR:
+ appendStringInfoString(&buffer, "storage manager");
+ break;
+
case OCLASS_AMOP:
appendStringInfoString(&buffer, "operator of access method");
break;
@@ -5121,6 +5170,23 @@ getObjectIdentityParts(const ObjectAddress *object,
*objname = list_make1(amname);
}
break;
+ case OCLASS_SMGR:
+ {
+ char *smgrname;
+
+ smgrname = get_smgr_name(object->objectId);
+ if (!smgrname)
+ {
+ if (!missing_ok)
+ elog(ERROR, "cache lookup failed for storage mamnager %u",
+ object->objectId);
+ break;
+ }
+ appendStringInfoString(&buffer, quote_identifier(smgrname));
+ if (objname)
+ *objname = list_make1(smgrname);
+ }
+ break;
case OCLASS_AMOP:
{
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 9bc10729b0..99763185ac 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -27,6 +27,7 @@
#include "catalog/pg_namespace.h"
#include "catalog/pg_opclass.h"
#include "catalog/pg_type.h"
+#include "catalog/pg_smgr.h"
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
@@ -254,6 +255,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
InvalidOid,
rel->rd_rel->relowner,
table_relation_toast_am(rel),
+ SMGR_MD_OID,
tupdesc,
NIL,
RELKIND_TOASTVALUE,
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index 48f7348f91..8f0c71f8d7 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -51,6 +51,7 @@ OBJS = \
seclabel.o \
sequence.o \
statscmds.o \
+ smgrcmds.o \
subscriptioncmds.o \
tablecmds.o \
tablespace.o \
diff --git a/src/backend/commands/alter.c b/src/backend/commands/alter.c
index 5456b8222b..e29a3bd20e 100644
--- a/src/backend/commands/alter.c
+++ b/src/backend/commands/alter.c
@@ -663,6 +663,7 @@ AlterObjectNamespace_oid(Oid classId, Oid objid, Oid nspOid,
case OCLASS_PUBLICATION:
case OCLASS_PUBLICATION_NAMESPACE:
case OCLASS_PUBLICATION_REL:
+ case OCLASS_SMGR:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
/* ignore object types that don't have schema-qualified names */
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index cea2c8be80..7ab7160e06 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -632,6 +632,7 @@ rebuild_relation(Relation OldHeap, Oid indexOid, bool verbose)
{
Oid tableOid = RelationGetRelid(OldHeap);
Oid accessMethod = OldHeap->rd_rel->relam;
+ Oid storageMethod = OldHeap->rd_rel->relsmgr;
Oid tableSpace = OldHeap->rd_rel->reltablespace;
Oid OIDNewHeap;
char relpersistence;
@@ -654,7 +655,7 @@ rebuild_relation(Relation OldHeap, Oid indexOid, bool verbose)
/* Create the transient table that will receive the re-ordered data */
OIDNewHeap = make_new_heap(tableOid, tableSpace,
accessMethod,
- relpersistence,
+ relpersistence, storageMethod,
AccessExclusiveLock);
/* Copy the heap data into the new table in the desired order */
@@ -683,7 +684,7 @@ rebuild_relation(Relation OldHeap, Oid indexOid, bool verbose)
* data, then call finish_heap_swap to complete the operation.
*/
Oid
-make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod,
+make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, Oid NewStorageMethod,
char relpersistence, LOCKMODE lockmode)
{
TupleDesc OldHeapDesc;
@@ -744,6 +745,7 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod,
InvalidOid,
OldHeap->rd_rel->relowner,
NewAccessMethod,
+ NewStorageMethod,
OldHeapDesc,
NIL,
RELKIND_RELATION,
diff --git a/src/backend/commands/dbcommands.c b/src/backend/commands/dbcommands.c
index f269168401..81e5bbe5b9 100644
--- a/src/backend/commands/dbcommands.c
+++ b/src/backend/commands/dbcommands.c
@@ -195,6 +195,7 @@ CreateDatabaseUsingWalLog(Oid src_dboid, Oid dst_dboid,
dstrnode.dbNode = dst_dboid;
dstrnode.relNode = srcrnode.relNode;
+ dstrnode.relSmgr = srcrnode.relSmgr;
/*
* Acquire locks on source and target relations before copying.
@@ -271,6 +272,7 @@ ScanSourceDatabasePgClass(Oid tbid, Oid dbid, char *srcpath)
rnode.spcNode = tbid;
rnode.dbNode = dbid;
rnode.relNode = relfilenode;
+ rnode.relSmgr = F_SMGR_MD_HANDLER;
/*
* We can't use a real relcache entry for a relation in some other
@@ -441,6 +443,7 @@ ScanSourceDatabasePgClassTuple(HeapTupleData *tuple, Oid tbid, Oid dbid,
relinfo->rnode.dbNode = dbid;
relinfo->rnode.relNode = relfilenode;
+ relinfo->rnode.relSmgr = F_SMGR_MD_HANDLER;
relinfo->reloid = classForm->oid;
/* Temporary relations were rejected above. */
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index 4642527881..6bfed2d6dd 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -982,6 +982,7 @@ EventTriggerSupportsObjectType(ObjectType obtype)
case OBJECT_SEQUENCE:
case OBJECT_SUBSCRIPTION:
case OBJECT_STATISTIC_EXT:
+ case OBJECT_STORAGE_MANAGER:
case OBJECT_TABCONSTRAINT:
case OBJECT_TABLE:
case OBJECT_TRANSFORM:
@@ -1042,6 +1043,7 @@ EventTriggerSupportsObjectClass(ObjectClass objclass)
case OCLASS_TRIGGER:
case OCLASS_SCHEMA:
case OCLASS_STATISTIC_EXT:
+ case OCLASS_SMGR:
case OCLASS_TSPARSER:
case OCLASS_TSDICT:
case OCLASS_TSTEMPLATE:
@@ -2080,6 +2082,7 @@ stringify_grant_objtype(ObjectType objtype)
case OBJECT_PUBLICATION_REL:
case OBJECT_ROLE:
case OBJECT_RULE:
+ case OBJECT_STORAGE_MANAGER:
case OBJECT_STATISTIC_EXT:
case OBJECT_SUBSCRIPTION:
case OBJECT_TABCONSTRAINT:
@@ -2164,6 +2167,7 @@ stringify_adefprivs_objtype(ObjectType objtype)
case OBJECT_PUBLICATION_REL:
case OBJECT_ROLE:
case OBJECT_RULE:
+ case OBJECT_STORAGE_MANAGER:
case OBJECT_STATISTIC_EXT:
case OBJECT_SUBSCRIPTION:
case OBJECT_TABCONSTRAINT:
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index d1ee106465..d3ae12c835 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -298,7 +298,7 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
* will be gone).
*/
OIDNewHeap = make_new_heap(matviewOid, tableSpace,
- matviewRel->rd_rel->relam,
+ matviewRel->rd_rel->relam, matviewRel->rd_rel->relsmgr,
relpersistence, ExclusiveLock);
LockRelationOid(OIDNewHeap, AccessExclusiveLock);
dest = CreateTransientRelDestReceiver(OIDNewHeap);
diff --git a/src/backend/commands/smgrcmds.c b/src/backend/commands/smgrcmds.c
new file mode 100644
index 0000000000..becfab4cfa
--- /dev/null
+++ b/src/backend/commands/smgrcmds.c
@@ -0,0 +1,197 @@
+
+/*-------------------------------------------------------------------------
+ *
+ * smgrcmds.c
+ * Routines for SQL commands that manipulate storage managers.
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/smgrcmds.c
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_smgr.h"
+#include "catalog/pg_proc.h"
+#include "catalog/pg_type.h"
+#include "commands/defrem.h"
+#include "miscadmin.h"
+#include "parser/parse_func.h"
+#include "utils/builtins.h"
+#include "utils/lsyscache.h"
+#include "utils/rel.h"
+#include "utils/syscache.h"
+#include "catalog/objectaccess.h"
+
+
+
+/*
+ * Convert a handler function name to an Oid.
+ *
+ * This function either return valid function Oid or throw an error.
+ */
+static Oid
+lookup_smgr_handler_func(List *handler_name)
+{
+ Oid handlerOid;
+ Oid funcargtypes[1] = {INTERNALOID};
+ Oid expectedType = TABLE_SMGR_HANDLEROID;
+
+ if (handler_name == NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_FUNCTION),
+ errmsg("handler function is not specified")));
+
+ /* handlers have one argument of type internal */
+ handlerOid = LookupFuncName(handler_name, 1, funcargtypes, false);
+
+ if (get_func_rettype(handlerOid) != expectedType)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("function %s must return type %s",
+ get_func_name(handlerOid),
+ format_type_extended(expectedType, -1, 0))));
+
+ return handlerOid;
+}
+
+/*
+ * CreateStorageManager
+ * Registers a new storage manager.
+ */
+ObjectAddress
+CreateStorageManager(CreateSmgrStmt *stmt) {
+
+ Relation rel;
+ ObjectAddress retval;
+ ObjectAddress referenced;
+ Oid smgroid;
+ Oid smgrhandler;
+ bool nulls[Natts_pg_smgr];
+ Datum values[Natts_pg_smgr];
+ HeapTuple tup;
+
+ rel = table_open(StorageManagerRelationId, RowExclusiveLock);
+
+ /* Must be superuser */
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("permission denied to create storage manager \"%s\"",
+ stmt->smgrname),
+ errhint("Must be superuser to create an storage manager.")));
+
+ /* Check if name is used */
+ smgroid = GetSysCacheOid1(SMGRNAME, Anum_pg_smgr_oid,
+ CStringGetDatum(stmt->smgrname));
+ if (OidIsValid(smgroid))
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_DUPLICATE_OBJECT),
+ errmsg("storage manager \"%s\" already exists",
+ stmt->smgrname)));
+ }
+
+ /*
+ * Get the handler function oid.
+ */
+ smgrhandler = lookup_smgr_handler_func(stmt->handler_name);
+
+ /*
+ * Insert tuple into pg_am.
+ */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+
+ smgroid = GetNewOidWithIndex(rel, SmgrOidIndexId, Anum_pg_smgr_oid);
+ values[Anum_pg_smgr_oid - 1] = ObjectIdGetDatum(smgroid);
+ values[Anum_pg_smgr_smgrname - 1] =
+ DirectFunctionCall1(namein, CStringGetDatum(stmt->smgrname));
+ values[Anum_pg_smgr_smgrhandler - 1] = ObjectIdGetDatum(smgrhandler);
+ tup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
+
+ CatalogTupleInsert(rel, tup);
+ heap_freetuple(tup);
+
+ retval.classId = StorageManagerRelationId;
+ retval.objectId = smgroid;
+ retval.objectSubId = 0;
+
+ /* Record dependency on handler function */
+ referenced.classId = ProcedureRelationId;
+ referenced.objectId = smgrhandler;
+ referenced.objectSubId = 0;
+
+ recordDependencyOn(&retval, &referenced, DEPENDENCY_NORMAL);
+
+ recordDependencyOnCurrentExtension(&retval, false);
+
+ InvokeObjectPostCreateHook(StorageManagerRelationId, smgroid, 0);
+
+ table_close(rel, RowExclusiveLock);
+
+ return retval;
+}
+
+
+
+/*
+ * get_am_type_oid
+ * Worker for various get_am_*_oid variants
+ *
+ * If missing_ok is false, throw an error if access method not found. If
+ * true, just return InvalidOid.
+ *
+ * If amtype is not '\0', an error is raised if the AM found is not of the
+ * given type.
+ */
+Oid
+get_smgr_oid(const char *smgrname, bool missing_ok)
+{
+ HeapTuple tup;
+ Oid oid = InvalidOid;
+
+ tup = SearchSysCache1(SMGRNAME, CStringGetDatum(smgrname));
+ if (HeapTupleIsValid(tup))
+ {
+ Form_pg_smgr smgrform = (Form_pg_smgr) GETSTRUCT(tup);
+ oid = smgrform->oid;
+ ReleaseSysCache(tup);
+ }
+
+ if (!OidIsValid(oid) && !missing_ok)
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("storage manager \"%s\" does not exist", smgrname)));
+ return oid;
+}
+
+
+
+/*
+ * get_smgr_name - given an storage manager OID, look up its name.
+ */
+char *
+get_smgr_name(Oid amOid)
+{
+ HeapTuple tup;
+ char *result = NULL;
+
+ tup = SearchSysCache1(SMGROID, ObjectIdGetDatum(amOid));
+ if (HeapTupleIsValid(tup))
+ {
+ Form_pg_smgr smgrform = (Form_pg_smgr) GETSTRUCT(tup);
+
+ result = pstrdup(NameStr(smgrform->smgrname));
+ ReleaseSysCache(tup);
+ }
+ return result;
+}
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 2de0ebacec..70712e5308 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -43,6 +43,7 @@
#include "catalog/pg_namespace.h"
#include "catalog/pg_opclass.h"
#include "catalog/pg_statistic_ext.h"
+#include "catalog/pg_smgr.h"
#include "catalog/pg_tablespace.h"
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
@@ -179,6 +180,7 @@ typedef struct AlteredTableInfo
bool verify_new_notnull; /* T if we should recheck NOT NULL */
int rewrite; /* Reason for forced rewrite, if any */
Oid newAccessMethod; /* new access method; 0 means no change */
+ Oid newStorageManager; /* new storage manager; 0 means no change */
Oid newTableSpace; /* new tablespace; 0 means no change */
bool chgPersistence; /* T if SET LOGGED/UNLOGGED is used */
char newrelpersistence; /* if above is true */
@@ -678,6 +680,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
LOCKMODE parentLockmode;
const char *accessMethod = NULL;
Oid accessMethodId = InvalidOid;
+ const char *storageManager = NULL;
+ Oid smgrOid = SMGR_MD_OID;
/*
* Truncate relname to appropriate length (probably a waste of time, as
@@ -953,6 +957,14 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (accessMethod != NULL)
accessMethodId = get_table_am_oid(accessMethod, false);
+ if (stmt->storageManager != NULL) {
+ storageManager = stmt->storageManager;
+ }
+
+ if (storageManager) {
+ smgrOid = get_smgr_oid(storageManager, false);
+ }
+
/*
* Create the relation. Inherited defaults and constraints are passed in
* for immediate handling --- since they don't need parsing, they can be
@@ -966,6 +978,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ofTypeId,
ownerId,
accessMethodId,
+ smgrOid,
descriptor,
list_concat(cookedDefaults,
old_constraints),
@@ -5429,6 +5442,7 @@ ATRewriteTables(AlterTableStmt *parsetree, List **wqueue, LOCKMODE lockmode,
Relation OldHeap;
Oid OIDNewHeap;
Oid NewAccessMethod;
+ Oid NewStorageManager;
Oid NewTableSpace;
char persistence;
@@ -5478,6 +5492,16 @@ ATRewriteTables(AlterTableStmt *parsetree, List **wqueue, LOCKMODE lockmode,
else
NewAccessMethod = OldHeap->rd_rel->relam;
+ /*
+ * Select destination storage manager (same as original unless user
+ * requested a change)
+ */
+ if (OidIsValid(tab->newStorageManager))
+ NewStorageManager = tab->newStorageManager;
+ else
+ NewStorageManager = OldHeap->rd_rel->relsmgr;
+
+
/*
* Select persistence of transient table (same as original unless
* user requested a change)
@@ -5517,7 +5541,7 @@ ATRewriteTables(AlterTableStmt *parsetree, List **wqueue, LOCKMODE lockmode,
* persistence. That wouldn't work for pg_class, but that can't be
* unlogged anyway.
*/
- OIDNewHeap = make_new_heap(tab->relid, NewTableSpace, NewAccessMethod,
+ OIDNewHeap = make_new_heap(tab->relid, NewTableSpace, NewAccessMethod, NewStorageManager,
persistence, lockmode);
/*
@@ -12683,6 +12707,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
case OCLASS_PUBLICATION:
case OCLASS_PUBLICATION_NAMESPACE:
case OCLASS_PUBLICATION_REL:
+ case OCLASS_SMGR:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
@@ -14374,7 +14399,7 @@ ATExecSetTableSpace(Oid tableOid, Oid newTableSpace, LOCKMODE lockmode)
* to allocate a new one in the new tablespace.
*/
newrelfilenode = GetNewRelFileNode(newTableSpace, NULL,
- rel->rd_rel->relpersistence);
+ rel->rd_rel->relpersistence, rel->rd_rel->relsmgr);
/* Open old and new relation */
newrnode = rel->rd_node;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index e747e1667d..88ff3c8fc5 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2433,6 +2433,16 @@ _equalCreateAmStmt(const CreateAmStmt *a, const CreateAmStmt *b)
return true;
}
+
+static bool
+_equalCreateSmgrStmt(const CreateSmgrStmt *a, const CreateSmgrStmt *b)
+{
+ COMPARE_STRING_FIELD(smgrname);
+ COMPARE_NODE_FIELD(handler_name);
+
+ return true;
+}
+
static bool
_equalCreateTrigStmt(const CreateTrigStmt *a, const CreateTrigStmt *b)
{
@@ -4120,6 +4130,9 @@ equal(const void *a, const void *b)
case T_CreateAmStmt:
retval = _equalCreateAmStmt(a, b);
break;
+ case T_CreateSmgrStmt:
+ retval = _equalCreateSmgrStmt(a, b);
+ break;
case T_CreateTrigStmt:
retval = _equalCreateTrigStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 969c9c158f..77038e33d4 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -328,7 +328,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
AlterTSConfigurationStmt AlterTSDictionaryStmt
CreateMatViewStmt RefreshMatViewStmt CreateAmStmt
CreatePublicationStmt AlterPublicationStmt
- CreateSubscriptionStmt AlterSubscriptionStmt DropSubscriptionStmt
+ CreateSubscriptionStmt AlterSubscriptionStmt DropSubscriptionStmt CreateSmgrStmt
%type <node> select_no_parens select_with_parens select_clause
simple_select values_clause
@@ -606,7 +606,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> constraints_set_list
%type <boolean> constraints_set_mode
-%type <str> OptTableSpace OptConsTableSpace
+%type <str> OptTableSpace OptConsTableSpace OptStorageManager
%type <rolespec> OptTableSpaceOwner
%type <ival> opt_check_option
@@ -814,7 +814,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
LEADING LEAKPROOF LEAST LEFT LEVEL LIKE LIMIT LISTEN LOAD LOCAL
LOCALTIME LOCALTIMESTAMP LOCATION LOCK_P LOCKED LOGGED
- MAPPING MATCH MATCHED MATERIALIZED MAXVALUE MERGE METHOD
+ MANAGER MAPPING MATCH MATCHED MATERIALIZED MAXVALUE MERGE METHOD
MINUTE_P MINVALUE MODE MONTH_P MOVE
NAME_P NAMES NATIONAL NATURAL NCHAR NESTED NEW NEXT NFC NFD NFKC NFKD NO
@@ -1112,6 +1112,7 @@ stmt:
| CreatePLangStmt
| CreateSchemaStmt
| CreateSeqStmt
+ | CreateSmgrStmt
| CreateStmt
| CreateSubscriptionStmt
| CreateStatsStmt
@@ -3578,7 +3579,7 @@ copy_generic_opt_arg_list_item:
CreateStmt: CREATE OptTemp TABLE qualified_name '(' OptTableElementList ')'
OptInherit OptPartitionSpec table_access_method_clause OptWith
- OnCommitOption OptTableSpace
+ OnCommitOption OptTableSpace OptStorageManager
{
CreateStmt *n = makeNode(CreateStmt);
@@ -3593,12 +3594,13 @@ CreateStmt: CREATE OptTemp TABLE qualified_name '(' OptTableElementList ')'
n->options = $11;
n->oncommit = $12;
n->tablespacename = $13;
+ n->storageManager = $14;
n->if_not_exists = false;
$$ = (Node *) n;
}
| CREATE OptTemp TABLE IF_P NOT EXISTS qualified_name '('
OptTableElementList ')' OptInherit OptPartitionSpec table_access_method_clause
- OptWith OnCommitOption OptTableSpace
+ OptWith OnCommitOption OptTableSpace OptStorageManager
{
CreateStmt *n = makeNode(CreateStmt);
@@ -3613,12 +3615,13 @@ CreateStmt: CREATE OptTemp TABLE qualified_name '(' OptTableElementList ')'
n->options = $14;
n->oncommit = $15;
n->tablespacename = $16;
+ n->storageManager = $17;
n->if_not_exists = true;
$$ = (Node *) n;
}
| CREATE OptTemp TABLE qualified_name OF any_name
OptTypedTableElementList OptPartitionSpec table_access_method_clause
- OptWith OnCommitOption OptTableSpace
+ OptWith OnCommitOption OptTableSpace OptStorageManager
{
CreateStmt *n = makeNode(CreateStmt);
@@ -3634,12 +3637,13 @@ CreateStmt: CREATE OptTemp TABLE qualified_name '(' OptTableElementList ')'
n->options = $10;
n->oncommit = $11;
n->tablespacename = $12;
+ n->storageManager = $13;
n->if_not_exists = false;
$$ = (Node *) n;
}
| CREATE OptTemp TABLE IF_P NOT EXISTS qualified_name OF any_name
OptTypedTableElementList OptPartitionSpec table_access_method_clause
- OptWith OnCommitOption OptTableSpace
+ OptWith OnCommitOption OptTableSpace OptStorageManager
{
CreateStmt *n = makeNode(CreateStmt);
@@ -3655,12 +3659,13 @@ CreateStmt: CREATE OptTemp TABLE qualified_name '(' OptTableElementList ')'
n->options = $13;
n->oncommit = $14;
n->tablespacename = $15;
+ n->storageManager = $16;
n->if_not_exists = true;
$$ = (Node *) n;
}
| CREATE OptTemp TABLE qualified_name PARTITION OF qualified_name
OptTypedTableElementList PartitionBoundSpec OptPartitionSpec
- table_access_method_clause OptWith OnCommitOption OptTableSpace
+ table_access_method_clause OptWith OnCommitOption OptTableSpace OptStorageManager
{
CreateStmt *n = makeNode(CreateStmt);
@@ -3676,12 +3681,13 @@ CreateStmt: CREATE OptTemp TABLE qualified_name '(' OptTableElementList ')'
n->options = $12;
n->oncommit = $13;
n->tablespacename = $14;
+ n->storageManager = $15;
n->if_not_exists = false;
$$ = (Node *) n;
}
| CREATE OptTemp TABLE IF_P NOT EXISTS qualified_name PARTITION OF
qualified_name OptTypedTableElementList PartitionBoundSpec OptPartitionSpec
- table_access_method_clause OptWith OnCommitOption OptTableSpace
+ table_access_method_clause OptWith OnCommitOption OptTableSpace OptStorageManager
{
CreateStmt *n = makeNode(CreateStmt);
@@ -3697,6 +3703,7 @@ CreateStmt: CREATE OptTemp TABLE qualified_name '(' OptTableElementList ')'
n->options = $15;
n->oncommit = $16;
n->tablespacename = $17;
+ n->storageManager = $18;
n->if_not_exists = true;
$$ = (Node *) n;
}
@@ -4513,6 +4520,11 @@ OptTableSpace: TABLESPACE name { $$ = $2; }
| /*EMPTY*/ { $$ = NULL; }
;
+OptStorageManager: STORAGE MANAGER name { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+
OptConsTableSpace: USING INDEX TABLESPACE name { $$ = $4; }
| /*EMPTY*/ { $$ = NULL; }
;
@@ -5827,6 +5839,24 @@ am_type:
| TABLE { $$ = AMTYPE_TABLE; }
;
+
+/*****************************************************************************
+ *
+ * QUERY:
+ * CREATE STORAGE MANAGER name HANDLER handler_name
+ *
+ *****************************************************************************/
+
+CreateSmgrStmt: CREATE STORAGE MANAGER name HANDLER handler_name
+ {
+ CreateSmgrStmt *n = makeNode(CreateSmgrStmt);
+
+ n->smgrname = $4;
+ n->handler_name = $6;
+ $$ = (Node *) n;
+ }
+ ;
+
/*****************************************************************************
*
* QUERIES :
@@ -6896,6 +6926,7 @@ drop_type_name:
| PUBLICATION { $$ = OBJECT_PUBLICATION; }
| SCHEMA { $$ = OBJECT_SCHEMA; }
| SERVER { $$ = OBJECT_FOREIGN_SERVER; }
+ | STORAGE MANAGER { $$ = OBJECT_STORAGE_MANAGER; }
;
/* object types attached to a table */
@@ -17815,6 +17846,7 @@ unreserved_keyword:
| LOCK_P
| LOCKED
| LOGGED
+ | MANAGER
| MAPPING
| MATCH
| MATCHED
@@ -18422,6 +18454,7 @@ bare_label_keyword:
| LOCK_P
| LOCKED
| LOGGED
+ | MANAGER
| MAPPING
| MATCH
| MATCHED
diff --git a/src/backend/storage/smgr/Makefile b/src/backend/storage/smgr/Makefile
index 596b564656..929ee45435 100644
--- a/src/backend/storage/smgr/Makefile
+++ b/src/backend/storage/smgr/Makefile
@@ -14,6 +14,8 @@ include $(top_builddir)/src/Makefile.global
OBJS = \
md.o \
- smgr.o
+ smgr.o \
+ smgrapi.o \
+ smgr_handler.o
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/storage/smgr/smgr.c b/src/backend/storage/smgr/smgr.c
index b3cf942d51..5f870e0309 100644
--- a/src/backend/storage/smgr/smgr.c
+++ b/src/backend/storage/smgr/smgr.c
@@ -23,27 +23,10 @@
#include "storage/ipc.h"
#include "storage/md.h"
#include "storage/smgr.h"
+#include "catalog/pg_smgr.h"
#include "utils/hsearch.h"
#include "utils/inval.h"
-static const f_smgr smgr_md = {
- /* magnetic disk */
- .smgr_init = mdinit,
- .smgr_shutdown = NULL,
- .smgr_open = mdopen,
- .smgr_close = mdclose,
- .smgr_create = mdcreate,
- .smgr_exists = mdexists,
- .smgr_unlink = mdunlink,
- .smgr_extend = mdextend,
- .smgr_prefetch = mdprefetch,
- .smgr_read = mdread,
- .smgr_write = mdwrite,
- .smgr_writeback = mdwriteback,
- .smgr_nblocks = mdnblocks,
- .smgr_truncate = mdtruncate,
- .smgr_immedsync = mdimmedsync,
-};
/*
* Each backend has a hashtable that stores all extant SMgrRelation objects.
@@ -68,11 +51,8 @@ static void smgrshutdown(int code, Datum arg);
void
smgrinit(void)
{
- if (smgr_init_hook)
- (*smgr_init_hook)();
-
- smgr_init_standard();
-
+ // init other smgrs in pg_init
+ mdinit();
/* register the shutdown proc */
on_proc_exit(smgrshutdown, 0);
}
@@ -82,51 +62,8 @@ smgrinit(void)
*/
static void
smgrshutdown(int code, Datum arg)
-{
- if (smgr_shutdown_hook)
- (*smgr_shutdown_hook)();
-
- smgr_shutdown_standard();
-}
-
-/* Hooks for plugins to get control in smgr */
-smgr_hook_type smgr_hook = NULL;
-smgr_init_hook_type smgr_init_hook = NULL;
-smgr_shutdown_hook_type smgr_shutdown_hook = NULL;
-
-const f_smgr *
-smgr_standard(BackendId backend, RelFileNode rnode)
-{
- return &smgr_md;
-}
-
-void
-smgr_init_standard(void)
-{
- mdinit();
-}
-
-void
-smgr_shutdown_standard(void)
{
}
-
-const f_smgr *
-smgr(BackendId backend, RelFileNode rnode)
-{
- const f_smgr *result;
-
- if (smgr_hook)
- {
- result = (*smgr_hook)(backend, rnode);
- }
- else
- result = smgr_standard(backend, rnode);
-
- return result;
-}
-
-
/*
* smgropen() -- Return an SMgrRelation object, creating it if need be.
*
@@ -167,14 +104,25 @@ smgropen(RelFileNode rnode, BackendId backend)
for (int i = 0; i <= MAX_FORKNUM; ++i)
reln->smgr_cached_nblocks[i] = InvalidBlockNumber;
- reln->smgr = smgr(backend, rnode);
+ if (rnode.relSmgr == SMGR_MD_OID)
+ {
+ /*
+ * Avoid doing a syscache lookup for catalog tables.
+ */
+ reln->smgr = &smgr_md;
+ } else {
+ Assert(rnode.relSmgr != 0);
+ reln->smgr = GetTableSmgr(rnode.relSmgr);
+ }
+ Assert(reln->smgr != NULL);
/* implementation-specific initialization */
- (*reln->smgr).smgr_open(reln);
+ reln->smgr->smgr_open(reln);
/* it has no owner yet */
dlist_push_tail(&unowned_relns, &reln->node);
}
+ Assert(reln->smgr != NULL);
return reln;
}
diff --git a/src/backend/storage/smgr/smgr_handler.c b/src/backend/storage/smgr/smgr_handler.c
new file mode 100644
index 0000000000..4b99d5744f
--- /dev/null
+++ b/src/backend/storage/smgr/smgr_handler.c
@@ -0,0 +1,75 @@
+/*-------------------------------------------------------------------------
+ *
+ * smgr_md_handler.c
+ * md table access method code
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/storage/smgr/smgr_md_handler.c
+ *
+ *
+ * NOTES
+ * This files wires up the lower level heapam.c et al routines with the
+ * tableam abstraction.
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+#include "storage/smgr.h"
+#include "storage/md.h"
+
+#include "access/genam.h"
+#include "access/heapam.h"
+#include "access/heaptoast.h"
+#include "access/multixact.h"
+#include "access/rewriteheap.h"
+#include "access/syncscan.h"
+#include "access/tableam.h"
+#include "access/tsmapi.h"
+#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
+#include "catalog/storage.h"
+#include "catalog/storage_xlog.h"
+#include "commands/progress.h"
+#include "executor/executor.h"
+#include "miscadmin.h"
+#include "pgstat.h"
+#include "storage/bufmgr.h"
+#include "storage/bufpage.h"
+#include "storage/lmgr.h"
+#include "storage/predicate.h"
+#include "storage/procarray.h"
+#include "storage/smgr.h"
+#include "utils/builtins.h"
+#include "utils/rel.h"
+
+const TableSmgr smgr_md = {
+ /* magnetic disk */
+ .type = T_TableSmgr,
+ .smgr_init = mdinit,
+ .smgr_shutdown = NULL,
+ .smgr_open = mdopen,
+ .smgr_close = mdclose,
+ .smgr_create = mdcreate,
+ .smgr_exists = mdexists,
+ .smgr_unlink = mdunlink,
+ .smgr_extend = mdextend,
+ .smgr_prefetch = mdprefetch,
+ .smgr_read = mdread,
+ .smgr_write = mdwrite,
+ .smgr_writeback = mdwriteback,
+ .smgr_nblocks = mdnblocks,
+ .smgr_truncate = mdtruncate,
+ .smgr_immedsync = mdimmedsync,
+};
+
+Datum
+smgr_md_handler(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_POINTER(&smgr_md);
+}
diff --git a/src/backend/storage/smgr/smgrapi.c b/src/backend/storage/smgr/smgrapi.c
new file mode 100644
index 0000000000..dde65dc773
--- /dev/null
+++ b/src/backend/storage/smgr/smgrapi.c
@@ -0,0 +1,96 @@
+
+/*
+ * GetTableAmRoutine
+ * Call the specified access method handler routine to get its
+ * TableAmRoutine struct, which will be palloc'd in the caller's
+ * memory context.
+ */
+
+#include "postgres.h"
+
+
+#include "storage/smgr.h"
+#include "catalog/pg_smgr.h"
+#include "catalog/pg_proc.h"
+#include "commands/defrem.h"
+#include "miscadmin.h"
+#include "utils/fmgroids.h"
+#include "utils/elog.h"
+#include "utils/memutils.h"
+#include "utils/syscache.h"
+
+TableSmgr *
+GetTableSmgr(Oid smgrhandler)
+{
+ Datum datum;
+ TableSmgr *routine;
+
+ datum = OidFunctionCall0(smgrhandler);
+ routine = (TableSmgr *) DatumGetPointer(datum);
+
+ if (routine == NULL || !IsA(routine, TableSmgr))
+ elog(ERROR, "table storage manage handler %u did not return a f_smgr struct",
+ smgrhandler);
+
+ /*
+ * Assert that all required callbacks are present. That makes it a bit
+ * easier to keep AMs up to date, e.g. when forward porting them to a new
+ * major version.
+ */
+ // Assert(routine->scan_begin != NULL);
+ // Assert(routine->scan_end != NULL);
+ // Assert(routine->scan_rescan != NULL);
+ // Assert(routine->scan_getnextslot != NULL);
+
+ // Assert(routine->parallelscan_estimate != NULL);
+ // Assert(routine->parallelscan_initialize != NULL);
+ // Assert(routine->parallelscan_reinitialize != NULL);
+
+ // Assert(routine->index_fetch_begin != NULL);
+ // Assert(routine->index_fetch_reset != NULL);
+ // Assert(routine->index_fetch_end != NULL);
+ // Assert(routine->index_fetch_tuple != NULL);
+
+ // Assert(routine->tuple_fetch_row_version != NULL);
+ // Assert(routine->tuple_tid_valid != NULL);
+ // Assert(routine->tuple_get_latest_tid != NULL);
+ // Assert(routine->tuple_satisfies_snapshot != NULL);
+ // Assert(routine->index_delete_tuples != NULL);
+
+ // Assert(routine->tuple_insert != NULL);
+
+ /*
+ * Could be made optional, but would require throwing error during
+ * parse-analysis.
+ // */
+ // Assert(routine->tuple_insert_speculative != NULL);
+ // Assert(routine->tuple_complete_speculative != NULL);
+
+ // Assert(routine->multi_insert != NULL);
+ // Assert(routine->tuple_delete != NULL);
+ // Assert(routine->tuple_update != NULL);
+ // Assert(routine->tuple_lock != NULL);
+
+ // Assert(routine->relation_set_new_filenode != NULL);
+ // Assert(routine->relation_nontransactional_truncate != NULL);
+ // Assert(routine->relation_copy_data != NULL);
+ // Assert(routine->relation_copy_for_cluster != NULL);
+ // Assert(routine->relation_vacuum != NULL);
+ // Assert(routine->scan_analyze_next_block != NULL);
+ // Assert(routine->scan_analyze_next_tuple != NULL);
+ // Assert(routine->index_build_range_scan != NULL);
+ // Assert(routine->index_validate_scan != NULL);
+
+ // Assert(routine->relation_size != NULL);
+ // Assert(routine->relation_needs_toast_table != NULL);
+
+ // Assert(routine->relation_estimate_size != NULL);
+
+ // /* optional, but one callback implies presence of the other */
+ // Assert((routine->scan_bitmap_next_block == NULL) ==
+ // (routine->scan_bitmap_next_tuple == NULL));
+ // Assert(routine->scan_sample_next_block != NULL);
+ // Assert(routine->scan_sample_next_tuple != NULL);
+
+ return routine;
+}
\ No newline at end of file
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 6a5bcded55..bb91383a03 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -192,6 +192,7 @@ ClassifyUtilityCommandAsReadOnly(Node *parsetree)
case T_CreateSeqStmt:
case T_CreateStatsStmt:
case T_CreateStmt:
+ case T_CreateSmgrStmt:
case T_CreateSubscriptionStmt:
case T_CreateTableAsStmt:
case T_CreateTableSpaceStmt:
@@ -1903,6 +1904,10 @@ ProcessUtilitySlow(ParseState *pstate,
address = AlterCollation((AlterCollationStmt *) parsetree);
break;
+
+ case T_CreateSmgrStmt:
+ address = CreateStorageManager((CreateSmgrStmt *) parsetree);
+ break;
default:
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -2640,6 +2645,9 @@ CreateCommandTag(Node *parsetree)
case OBJECT_STATISTIC_EXT:
tag = CMDTAG_DROP_STATISTICS;
break;
+ case OBJECT_STORAGE_MANAGER:
+ tag = CMDTAG_DROP_STORAGE_MANAGER;
+ break;
default:
tag = CMDTAG_UNKNOWN;
}
@@ -3075,6 +3083,10 @@ CreateCommandTag(Node *parsetree)
tag = CMDTAG_CREATE_STATISTICS;
break;
+ case T_CreateSmgrStmt:
+ tag = CMDTAG_CREATE_STORAGE_MANAGER;
+ break;
+
case T_AlterStatsStmt:
tag = CMDTAG_ALTER_STATISTICS;
break;
@@ -3656,6 +3668,9 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
+ case T_CreateSmgrStmt:
+ lev = LOGSTMT_DDL;
+ break;
case T_CreatePublicationStmt:
lev = LOGSTMT_DDL;
break;
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index c8202502c9..01ef244a80 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -389,3 +389,4 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
PSEUDOTYPE_DUMMY_IO_FUNCS(anynonarray);
PSEUDOTYPE_DUMMY_IO_FUNCS(anycompatible);
PSEUDOTYPE_DUMMY_IO_FUNCS(anycompatiblenonarray);
+PSEUDOTYPE_DUMMY_IO_FUNCS(table_smgr_handler);
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 60e72f9e8b..6038a975ef 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -59,6 +59,7 @@
#include "catalog/pg_rewrite.h"
#include "catalog/pg_shseclabel.h"
#include "catalog/pg_statistic_ext.h"
+#include "catalog/pg_smgr.h"
#include "catalog/pg_subscription.h"
#include "catalog/pg_tablespace.h"
#include "catalog/pg_trigger.h"
@@ -77,6 +78,7 @@
#include "rewrite/rowsecurity.h"
#include "storage/lmgr.h"
#include "storage/smgr.h"
+#include "storage/md.h"
#include "utils/array.h"
#include "utils/builtins.h"
#include "utils/datum.h"
@@ -1209,6 +1211,8 @@ retry:
else
Assert(relation->rd_rel->relam == InvalidOid);
+ RelationInitSmgrHandler(relation);
+
/* extract reloptions if any */
RelationParseRelOptions(relation, pg_class_tuple);
@@ -1843,6 +1847,51 @@ RelationInitTableAccessMethod(Relation relation)
InitTableAmRoutine(relation);
}
+
+static void
+InitTableSmgr(Relation relation)
+{
+ if (IsCatalogRelation(relation))
+ {
+ /*
+ * Avoid doing a syscache lookup for catalog tables.
+ */
+ Assert(relation->rd_rel->relsmgr == SMGR_MD_OID);
+ relation->rd_f_smgr = &smgr_md;
+ } else {
+ relation->rd_f_smgr = GetTableSmgr(relation->rd_smgrhandler);
+ }
+}
+
+void RelationInitSmgrHandler(Relation relation) {
+ HeapTuple tuple;
+ Form_pg_smgr smgrform;
+
+ /*
+ * Look up the table storage manager, save the OID of its handler
+ * function.
+ */
+
+ if (IsCatalogRelation(relation)) {
+ relation->rd_smgrhandler = F_SMGR_MD_HANDLER;
+ relation->rd_rel->relsmgr = SMGR_MD_OID;
+ } else {
+ tuple = SearchSysCache1(SMGROID,
+ ObjectIdGetDatum(relation->rd_rel->relsmgr));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for storage manager %u" ,
+ relation->rd_rel->relsmgr);
+ smgrform = (Form_pg_smgr) GETSTRUCT(tuple);
+ relation->rd_smgrhandler = smgrform->smgrhandler;
+ ReleaseSysCache(tuple);
+ }
+
+ Assert(relation->rd_rel->relsmgr != InvalidOid);
+ relation->rd_node.relSmgr = relation->rd_smgrhandler;
+ InitTableSmgr(relation);
+}
+
+
/*
* formrdesc
*
@@ -1931,6 +1980,8 @@ formrdesc(const char *relationName, Oid relationReltype,
relation->rd_rel->relkind = RELKIND_RELATION;
relation->rd_rel->relnatts = (int16) natts;
relation->rd_rel->relam = HEAP_TABLE_AM_OID;
+ relation->rd_rel->relsmgr = SMGR_MD_OID;
+ relation->rd_node.relSmgr = F_SMGR_MD_HANDLER;
/*
* initialize attribute tuple form
@@ -1983,6 +2034,7 @@ formrdesc(const char *relationName, Oid relationReltype,
* specifying that the initial filenode is the same as the OID.
*/
relation->rd_rel->relfilenode = InvalidOid;
+ relation->rd_node.relSmgr = F_SMGR_MD_HANDLER;
if (IsBootstrapProcessingMode())
RelationMapUpdateMap(RelationGetRelid(relation),
RelationGetRelid(relation),
@@ -3459,6 +3511,7 @@ RelationBuildLocalRelation(const char *relname,
TupleDesc tupDesc,
Oid relid,
Oid accessmtd,
+ Oid smgrid,
Oid relfilenode,
Oid reltablespace,
bool shared_relation,
@@ -3642,6 +3695,7 @@ RelationBuildLocalRelation(const char *relname,
RelationInitPhysicalAddr(rel);
rel->rd_rel->relam = accessmtd;
+ rel->rd_rel->relsmgr = smgrid;
/*
* RelationInitTableAccessMethod will do syscache lookups, so we mustn't
@@ -3653,6 +3707,9 @@ RelationBuildLocalRelation(const char *relname,
if (RELKIND_HAS_TABLE_AM(relkind) || relkind == RELKIND_SEQUENCE)
RelationInitTableAccessMethod(rel);
+ rel->rd_rel->relsmgr = smgrid;
+ RelationInitSmgrHandler(rel);
+
/*
* Okay to insert into the relcache hash table.
*
@@ -3709,7 +3766,7 @@ RelationSetNewRelfilenode(Relation relation, char persistence)
/* Allocate a new relfilenode */
newrelfilenode = GetNewRelFileNode(relation->rd_rel->reltablespace, NULL,
- persistence);
+ persistence, relation->rd_rel->relsmgr);
/*
* Get a writable copy of the pg_class tuple for the given relation.
diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c
index 1912b12146..7c8de8e2db 100644
--- a/src/backend/utils/cache/syscache.c
+++ b/src/backend/utils/cache/syscache.c
@@ -64,6 +64,7 @@
#include "catalog/pg_statistic.h"
#include "catalog/pg_statistic_ext.h"
#include "catalog/pg_statistic_ext_data.h"
+#include "catalog/pg_smgr.h"
#include "catalog/pg_subscription.h"
#include "catalog/pg_subscription_rel.h"
#include "catalog/pg_tablespace.h"
@@ -829,6 +830,28 @@ static const struct cachedesc cacheinfo[] = {
},
128
},
+ {StorageManagerRelationId, /* STORANGE MANAGER OID */
+ SmgrOidIndexId,
+ 1,
+ {
+ Anum_pg_smgr_oid,
+ 0,
+ 0,
+ 0,
+ },
+ 4
+ },
+ {StorageManagerRelationId, /* STORANGE MANAGER NAME*/
+ SmgrNameIndexId,
+ 1,
+ {
+ Anum_pg_smgr_smgrname,
+ 0,
+ 0,
+ 0,
+ },
+ 4
+ },
{SubscriptionRelationId, /* SUBSCRIPTIONNAME */
SubscriptionNameIndexId,
2,
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 1a5d924a23..5e5cb9f1f7 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1534,6 +1534,7 @@ describeOneTableDetails(const char *schemaname,
char relpersistence;
char relreplident;
char *relam;
+ char *relsmgr;
} tableinfo;
bool show_column_details = false;
diff --git a/src/include/catalog/catalog.h b/src/include/catalog/catalog.h
index 60c1215362..afdc7fe0c7 100644
--- a/src/include/catalog/catalog.h
+++ b/src/include/catalog/catalog.h
@@ -39,6 +39,6 @@ extern bool IsPinnedObject(Oid classId, Oid objectId);
extern Oid GetNewOidWithIndex(Relation relation, Oid indexId,
AttrNumber oidcolumn);
extern Oid GetNewRelFileNode(Oid reltablespace, Relation pg_class,
- char relpersistence);
+ char relpersistence, Oid relsmgr);
#endif /* CATALOG_H */
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index d027075a4c..d57459226a 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -126,6 +126,7 @@ typedef enum ObjectClass
OCLASS_PUBLICATION_NAMESPACE, /* pg_publication_namespace */
OCLASS_PUBLICATION_REL, /* pg_publication_rel */
OCLASS_SUBSCRIPTION, /* pg_subscription */
+ OCLASS_SMGR, /* pg_smgr */
OCLASS_TRANSFORM /* pg_transform */
} ObjectClass;
diff --git a/src/include/catalog/heap.h b/src/include/catalog/heap.h
index 07c5b88f0e..41e7e84fda 100644
--- a/src/include/catalog/heap.h
+++ b/src/include/catalog/heap.h
@@ -52,6 +52,7 @@ extern Relation heap_create(const char *relname,
Oid relid,
Oid relfilenode,
Oid accessmtd,
+ Oid smgrid,
TupleDesc tupDesc,
char relkind,
char relpersistence,
@@ -70,6 +71,7 @@ extern Oid heap_create_with_catalog(const char *relname,
Oid reloftypeid,
Oid ownerid,
Oid accessmtd,
+ Oid smgrid,
TupleDesc tupdesc,
List *cooked_constraints,
char relkind,
diff --git a/src/include/catalog/pg_class.h b/src/include/catalog/pg_class.h
index e1f4eefa22..a6a669e655 100644
--- a/src/include/catalog/pg_class.h
+++ b/src/include/catalog/pg_class.h
@@ -52,6 +52,9 @@ CATALOG(pg_class,1259,RelationRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(83,Relat
/* access method; 0 if not a table / index */
Oid relam BKI_DEFAULT(heap) BKI_LOOKUP_OPT(pg_am);
+ /* stogare manager; */
+ Oid relsmgr BKI_DEFAULT(4646);
+
/* identifier of physical storage file */
/* relfilenode == 0 means it is a "mapped" relation, see relmapper.c */
Oid relfilenode BKI_DEFAULT(0);
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 87aa571a33..c2469e6fc2 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -7443,6 +7443,13 @@
{ oid => '268', descr => 'I/O',
proname => 'table_am_handler_out', prorettype => 'cstring',
proargtypes => 'table_am_handler', prosrc => 'table_am_handler_out' },
+{ oid => '4648', descr => 'I/O',
+ proname => 'table_smgr_handler_in', proisstrict => 'f',
+ prorettype => 'table_smgr_handler', proargtypes => 'cstring',
+ prosrc => 'table_smgr_handler_in' },
+{ oid => '4649', descr => 'I/O',
+ proname => 'table_smgr_handler_out', prorettype => 'cstring',
+ proargtypes => 'table_smgr_handler', prosrc => 'table_smgr_handler_out' },
{ oid => '5086', descr => 'I/O',
proname => 'anycompatible_in', prorettype => 'anycompatible',
proargtypes => 'cstring', prosrc => 'anycompatible_in' },
@@ -11885,4 +11892,10 @@
prorettype => 'bytea', proargtypes => 'pg_brin_minmax_multi_summary',
prosrc => 'brin_minmax_multi_summary_send' },
+# Storage access method handlers
+{ oid => '4642', descr => 'md storage manager handler',
+ proname => 'smgr_md_handler', provolatile => 'v',
+ prorettype => 'table_smgr_handler', proargtypes => 'internal',
+ prosrc => 'smgr_md_handler' },
+
]
diff --git a/src/include/catalog/pg_smgr.dat b/src/include/catalog/pg_smgr.dat
new file mode 100644
index 0000000000..ee7dea1087
--- /dev/null
+++ b/src/include/catalog/pg_smgr.dat
@@ -0,0 +1,17 @@
+#----------------------------------------------------------------------
+#
+# pg_smgr.dat
+# Initial contents of the pg_smgr system catalog.
+#
+# Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+# Portions Copyright (c) 1994, Regents of the University of California
+#
+# src/include/catalog/pg_smgr.dat
+#
+#----------------------------------------------------------------------
+
+[
+{ oid => '4646', oid_symbol => 'SMGR_MD_OID',
+ descr => 'md storage manager',
+ smgrname => 'md', smgrhandler => 'smgr_md_handler' },
+]
diff --git a/src/include/catalog/pg_smgr.h b/src/include/catalog/pg_smgr.h
new file mode 100644
index 0000000000..c2152cd844
--- /dev/null
+++ b/src/include/catalog/pg_smgr.h
@@ -0,0 +1,52 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_smgr.h
+ * definition of the "storage manager" system catalog (pg_smgr)
+ *
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_smgr.h
+ *
+ * NOTES
+ * The Catalog.pm module reads this file and derives schema
+ * information.
+ *
+ *-------------------------------------------------------------------------
+ */
+/* ----------------
+ * pg_smgr definition. cpp turns this into
+ * typedef struct FormData_pg_smgr
+ * ----------------
+ */
+
+
+#ifndef PG_SMGR_H
+#define PG_SMGR_H
+
+#include "catalog/genbki.h"
+#include "catalog/pg_smgr_d.h"
+
+CATALOG(pg_smgr,4643,StorageManagerRelationId)
+{
+ Oid oid; /* oid */
+
+ /* storage manager name */
+ NameData smgrname;
+
+ /* handler function */
+ regproc smgrhandler BKI_LOOKUP(pg_proc);
+} FormData_pg_smgr;
+
+/* ----------------
+ * Form_pg_smgr corresponds to a pointer to a tuple with
+ * the format of pg_smgr relation.
+ * ----------------
+ */
+typedef FormData_pg_smgr *Form_pg_smgr;
+
+DECLARE_UNIQUE_INDEX(pg_smgr_name_index, 4644, SmgrNameIndexId, on pg_smgr using btree(smgrname name_ops));
+DECLARE_UNIQUE_INDEX_PKEY(pg_smgr_oid_index, 4645, SmgrOidIndexId, on pg_smgr using btree(oid oid_ops));
+
+#endif /* PG_SMGR_H */
\ No newline at end of file
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index df45879463..446c3f1c55 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -637,6 +637,11 @@
typcategory => 'P', typinput => 'table_am_handler_in',
typoutput => 'table_am_handler_out', typreceive => '-', typsend => '-',
typalign => 'i' },
+{ oid => '4647',
+ typname => 'table_smgr_handler', typlen => '4', typbyval => 't', typtype => 'p',
+ typcategory => 'P', typinput => 'table_smgr_handler_in',
+ typoutput => 'table_smgr_handler_out', typreceive => '-', typsend => '-',
+ typalign => 'i' },
{ oid => '3831',
descr => 'pseudo-type representing a range over a polymorphic base type',
typname => 'anyrange', typlen => '-1', typbyval => 'f', typtype => 'p',
diff --git a/src/include/commands/cluster.h b/src/include/commands/cluster.h
index df8e73af40..64535486a2 100644
--- a/src/include/commands/cluster.h
+++ b/src/include/commands/cluster.h
@@ -37,7 +37,7 @@ extern void check_index_is_clusterable(Relation OldHeap, Oid indexOid,
LOCKMODE lockmode);
extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal);
-extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod,
+extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, Oid NewStorageMethod,
char relpersistence, LOCKMODE lockmode);
extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
bool is_system_catalog,
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 56d2bb6616..99b27164dd 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -143,6 +143,11 @@ extern Oid get_table_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+/* commands/smgrcmds.c */
+extern Oid get_smgr_oid(const char *smgrname, bool missing_ok);
+extern ObjectAddress CreateStorageManager(CreateSmgrStmt *stmt);
+extern char *get_smgr_name(Oid amOid);
+
/* support routines in commands/define.c */
extern char *defGetString(DefElem *def);
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index b3b407579b..0622739a79 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -437,6 +437,7 @@ typedef enum NodeTag
T_AlterPolicyStmt,
T_CreateTransformStmt,
T_CreateAmStmt,
+ T_CreateSmgrStmt,
T_CreatePublicationStmt,
T_AlterPublicationStmt,
T_CreateSubscriptionStmt,
@@ -553,6 +554,7 @@ typedef enum NodeTag
T_FdwRoutine, /* in foreign/fdwapi.h */
T_IndexAmRoutine, /* in access/amapi.h */
T_TableAmRoutine, /* in access/tableam.h */
+ T_TableSmgr, /* in storage/smgr.h */
T_TsmRoutine, /* in access/tsmapi.h */
T_ForeignKeyCacheInfo, /* in utils/rel.h */
T_CallContext, /* in nodes/parsenodes.h */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 73f635b455..ac1f49422f 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -2172,6 +2172,7 @@ typedef enum ObjectType
OBJECT_SEQUENCE,
OBJECT_SUBSCRIPTION,
OBJECT_STATISTIC_EXT,
+ OBJECT_STORAGE_MANAGER,
OBJECT_TABCONSTRAINT,
OBJECT_TABLE,
OBJECT_TABLESPACE,
@@ -2540,6 +2541,7 @@ typedef struct CreateStmt
OnCommitAction oncommit; /* what do we do at COMMIT? */
char *tablespacename; /* table space to use, or NULL */
char *accessMethod; /* table access method */
+ char *storageManager; /* table access method */
bool if_not_exists; /* just do nothing if it already exists? */
} CreateStmt;
@@ -2881,6 +2883,18 @@ typedef struct CreateAmStmt
char amtype; /* type of access method */
} CreateAmStmt;
+
+/*----------------------
+ * Create STORAGE MANAGER Statement
+ *----------------------
+ */
+typedef struct CreateSmgrStmt
+{
+ NodeTag type;
+ char *smgrname; /* access method name */
+ List *handler_name; /* handler function name */
+} CreateSmgrStmt;
+
/* ----------------------
* Create TRIGGER Statement
* ----------------------
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index ae35f03251..0fa6186f90 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -267,6 +267,7 @@ PG_KEYWORD("location", LOCATION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("lock", LOCK_P, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("locked", LOCKED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("logged", LOGGED, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("manager", MANAGER, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("mapping", MAPPING, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("match", MATCH, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("matched", MATCHED, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/storage/md.h b/src/include/storage/md.h
index ffffa40db7..f3b6f5942c 100644
--- a/src/include/storage/md.h
+++ b/src/include/storage/md.h
@@ -49,4 +49,7 @@ extern int mdsyncfiletag(const FileTag *ftag, char *path);
extern int mdunlinkfiletag(const FileTag *ftag, char *path);
extern bool mdfiletagmatches(const FileTag *ftag, const FileTag *candidate);
+
+const TableSmgr smgr_md;
+
#endif /* MD_H */
diff --git a/src/include/storage/relfilenode.h b/src/include/storage/relfilenode.h
index 4fdc606cc3..e1d51ebc6e 100644
--- a/src/include/storage/relfilenode.h
+++ b/src/include/storage/relfilenode.h
@@ -59,6 +59,7 @@ typedef struct RelFileNode
Oid spcNode; /* tablespace */
Oid dbNode; /* database */
Oid relNode; /* relation */
+ Oid relSmgr; /* smgr handler */
} RelFileNode;
/*
diff --git a/src/include/storage/smgr.h b/src/include/storage/smgr.h
index 7aca0d5ca5..30778c23df 100644
--- a/src/include/storage/smgr.h
+++ b/src/include/storage/smgr.h
@@ -14,9 +14,13 @@
#ifndef SMGR_H
#define SMGR_H
+#include "postgres.h"
+
#include "lib/ilist.h"
#include "storage/block.h"
#include "storage/relfilenode.h"
+#include "utils/snapshot.h"
+#include "nodes/nodes.h"
struct f_smgr;
@@ -61,8 +65,11 @@ typedef struct SMgrRelationData
* Fields below here are intended to be private to smgr.c and its
* submodules. Do not touch them from elsewhere.
*/
- const struct f_smgr *smgr; /* storage manager selector */
+ /*
+ * Storage manager.
+ */
+ const struct TableSmgr *smgr;
/*
* for md.c; per-fork arrays of the number of open segments
* (md_num_open_segs) and the segments themselves (md_seg_fds).
@@ -90,8 +97,12 @@ typedef SMgrRelationData *SMgrRelation;
* would normally be errors should be allowed during bootstrap and/or WAL
* recovery --- see comments in md.c for details.
*/
-typedef struct f_smgr
+typedef struct TableSmgr
{
+ /* this must be set to T_TableSmgr */
+ NodeTag type;
+
+
void (*smgr_init) (void); /* may be NULL */
void (*smgr_shutdown) (void); /* may be NULL */
void (*smgr_open) (SMgrRelation reln);
@@ -115,21 +126,7 @@ typedef struct f_smgr
void (*smgr_truncate) (SMgrRelation reln, ForkNumber forknum,
BlockNumber nblocks);
void (*smgr_immedsync) (SMgrRelation reln, ForkNumber forknum);
-} f_smgr;
-
-typedef void (*smgr_init_hook_type) (void);
-typedef void (*smgr_shutdown_hook_type) (void);
-extern PGDLLIMPORT smgr_init_hook_type smgr_init_hook;
-extern PGDLLIMPORT smgr_shutdown_hook_type smgr_shutdown_hook;
-extern void smgr_init_standard(void);
-extern void smgr_shutdown_standard(void);
-
-
-typedef const f_smgr *(*smgr_hook_type) (BackendId backend, RelFileNode rnode);
-extern PGDLLIMPORT smgr_hook_type smgr_hook;
-extern const f_smgr *smgr_standard(BackendId backend, RelFileNode rnode);
-
-extern const f_smgr *smgr(BackendId backend, RelFileNode rnode);
+} TableSmgr;
extern void smgrinit(void);
extern SMgrRelation smgropen(RelFileNode rnode, BackendId backend);
@@ -162,4 +159,8 @@ extern void smgrimmedsync(SMgrRelation reln, ForkNumber forknum);
extern void AtEOXact_SMgr(void);
extern bool ProcessBarrierSmgrRelease(void);
+
+TableSmgr *
+GetTableSmgr(Oid smgrhandler);
+
#endif /* SMGR_H */
diff --git a/src/include/tcop/cmdtaglist.h b/src/include/tcop/cmdtaglist.h
index 2b1163ce33..28dff1f8ad 100644
--- a/src/include/tcop/cmdtaglist.h
+++ b/src/include/tcop/cmdtaglist.h
@@ -111,6 +111,7 @@ PG_CMDTAG(CMDTAG_CREATE_SCHEMA, "CREATE SCHEMA", true, false, false)
PG_CMDTAG(CMDTAG_CREATE_SEQUENCE, "CREATE SEQUENCE", true, false, false)
PG_CMDTAG(CMDTAG_CREATE_SERVER, "CREATE SERVER", true, false, false)
PG_CMDTAG(CMDTAG_CREATE_STATISTICS, "CREATE STATISTICS", true, false, false)
+PG_CMDTAG(CMDTAG_CREATE_STORAGE_MANAGER, "CREATE STORAGE MANAGER", true, false, false)
PG_CMDTAG(CMDTAG_CREATE_SUBSCRIPTION, "CREATE SUBSCRIPTION", true, false, false)
PG_CMDTAG(CMDTAG_CREATE_TABLE, "CREATE TABLE", true, false, false)
PG_CMDTAG(CMDTAG_CREATE_TABLE_AS, "CREATE TABLE AS", true, false, false)
@@ -164,6 +165,7 @@ PG_CMDTAG(CMDTAG_DROP_SCHEMA, "DROP SCHEMA", true, false, false)
PG_CMDTAG(CMDTAG_DROP_SEQUENCE, "DROP SEQUENCE", true, false, false)
PG_CMDTAG(CMDTAG_DROP_SERVER, "DROP SERVER", true, false, false)
PG_CMDTAG(CMDTAG_DROP_STATISTICS, "DROP STATISTICS", true, false, false)
+PG_CMDTAG(CMDTAG_DROP_STORAGE_MANAGER, "DROP STORAGE MANAGER", false, false, false)
PG_CMDTAG(CMDTAG_DROP_SUBSCRIPTION, "DROP SUBSCRIPTION", true, false, false)
PG_CMDTAG(CMDTAG_DROP_TABLE, "DROP TABLE", true, false, false)
PG_CMDTAG(CMDTAG_DROP_TABLESPACE, "DROP TABLESPACE", false, false, false)
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 90b3c49bc1..7fa646fe0a 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -184,6 +184,13 @@ typedef struct RelationData
*/
const struct TableAmRoutine *rd_tableam;
+
+ Oid rd_smgrhandler; /* OID of index smgr handler function */
+ /*
+ * Storage manager.
+ */
+ const struct TableSmgr *rd_f_smgr;
+
/* These are non-NULL only for an index relation: */
Form_pg_index rd_index; /* pg_index tuple describing this index */
/* use "struct" here to avoid needing to include htup.h: */
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 86dddbd975..2c95debeb7 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -79,6 +79,7 @@ extern void RelationBuildPublicationDesc(Relation relation,
struct PublicationDesc *pubdesc);
extern void RelationInitTableAccessMethod(Relation relation);
+extern void RelationInitSmgrHandler(Relation relation);
/*
* Routines to support ereport() reports of relation-related errors
@@ -103,6 +104,7 @@ extern Relation RelationBuildLocalRelation(const char *relname,
TupleDesc tupDesc,
Oid relid,
Oid accessmtd,
+ Oid smgrid,
Oid relfilenode,
Oid reltablespace,
bool shared_relation,
diff --git a/src/include/utils/syscache.h b/src/include/utils/syscache.h
index 4463ea66be..8ac7e10760 100644
--- a/src/include/utils/syscache.h
+++ b/src/include/utils/syscache.h
@@ -95,6 +95,8 @@ enum SysCacheIdentifier
STATEXTNAMENSP,
STATEXTOID,
STATRELATTINH,
+ SMGROID,
+ SMGRNAME,
SUBSCRIPTIONNAME,
SUBSCRIPTIONOID,
SUBSCRIPTIONRELMAP,
diff --git a/src/test/modules/test_oat_hooks/test_oat_hooks.c b/src/test/modules/test_oat_hooks/test_oat_hooks.c
index 7ef272cc7a..44f8fd3131 100644
--- a/src/test/modules/test_oat_hooks/test_oat_hooks.c
+++ b/src/test/modules/test_oat_hooks/test_oat_hooks.c
@@ -1485,6 +1485,9 @@ nodetag_to_string(NodeTag tag)
case T_CreateStatsStmt:
return "CreateStatsStmt";
break;
+ case T_CreateSmgrStmt:
+ return "CreateSmgrStmt";
+ break;
case T_AlterCollationStmt:
return "AlterCollationStmt";
break;
@@ -1713,6 +1716,9 @@ nodetag_to_string(NodeTag tag)
case T_TableAmRoutine:
return "TableAmRoutine";
break;
+ case T_TableSmgr:
+ return "TableSmgr";
+ break;
case T_TsmRoutine:
return "TsmRoutine";
break;
--
2.25.1
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Extensible storage manager API - smgr hooks
@ 2022-08-26 05:25 Andrey Borodin <[email protected]>
parent: Kirill Reshke <[email protected]>
0 siblings, 0 replies; 7+ messages in thread
From: Andrey Borodin @ 2022-08-26 05:25 UTC (permalink / raw)
To: Kirill Reshke <[email protected]>; +Cc: Andres Freund <[email protected]>; Yura Sokolov <[email protected]>; Anastasia Lubennikova <[email protected]>; [email protected]
> On 16 Jun 2022, at 13:41, Kirill Reshke <[email protected]> wrote:
>
> Hello Yura and Anastasia.
FWIW this technology is now a part of Greenplum [0]. We are building GP extension that automatically offloads cold data to S3 - a very simplified version of Neon for analytical workloads.
When a segment of a table is not used for a long period of time, extension will sync files with backup storage in the Cloud.
When the user touches data, extension's smgr will bring table segments back from backup or latest synced version.
Our #1 goal is to provide a tool useful for the community. We easily can provide same extension for Postgres if this technology (extensible smgr) is in core. Does such an extension seem useful for Postgres? Or does this data access pattern seems unusual for Postgres? By pattern I mean vast amounts of cold data only ever appended and never touched.
Best regards, Andrey Borodin.
[0] https://github.com/greenplum-db/gpdb/pull/13601
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Extensible storage manager API - SMGR hook Redux
@ 2025-03-07 11:52 Andreas Karlsson <[email protected]>
0 siblings, 2 replies; 7+ messages in thread
From: Andreas Karlsson @ 2025-03-07 11:52 UTC (permalink / raw)
To: Nitin Jadhav <[email protected]>; Tristan Partin <[email protected]>; +Cc: pgsql-hackers; Matthias van de Meent <[email protected]>; Heikki Linnakangas <[email protected]>; Andres Freund <[email protected]>
Hi,
Here is a rebased version of it to make the CI happy. I plan to work
more on this next week but am happy with any feedback on what is already
there.
Andreas
Attachments:
[text/x-patch] v4-0006-SMGR-GUC-variable-and-chaining.patch (47.2K, ../../[email protected]/2-v4-0006-SMGR-GUC-variable-and-chaining.patch)
download | inline diff:
From 1dd58c606c5e026c96ed93a71d85d3605078a429 Mon Sep 17 00:00:00 2001
From: Zsolt Parragi <[email protected]>
Date: Mon, 2 Dec 2024 07:47:07 +0000
Subject: [PATCH v4 6/6] SMGR GUC variable and chaining
The overall goal of this commit is to introduce a user interface to
the previous SMGR patch.
The idea is to allow a simple configuration for multiple "modificator"
SMGRs similar to the fsync_checker in the original proposal.
* Extensions should be able to declare a named lists of SMGR implementations,
also specifying if the given SMGR is an "end" implementation for
actual storage, or if it is a modifier implementation for some other
purpose.
* Users should be able to specify a list of SMGRs: possibly multiple modifiers,
and one storage implementation at the end to configure how the
storage manager is constructed.
This commit introduces a new GUC variable, `smgr_chain`, which allows
users to configure multiple SMGR implementations: it is a comma separated list,
where the last entry most be a storage implementation, the others must be
modifiers. The default value of this variable is "md".
The internal storage manager API is also refactored to include an easy
way for SMGR implementations to support proper chaining. Modifier SMGR
implementations also only have to implement the functions they actually
change, and can leave everything else as empty (NULL). And with this
change we can make the functions of the md smgr static.
The fsync example extension is also modified to match the new API.
---
contrib/fsync_checker/fsync_checker_smgr.c | 57 ++--
src/backend/postmaster/postmaster.c | 2 +
src/backend/storage/smgr/md.c | 102 +++++---
src/backend/storage/smgr/smgr.c | 247 ++++++++++++++----
src/backend/tcop/postgres.c | 2 +
src/backend/utils/init/miscinit.c | 63 ++++-
src/backend/utils/misc/guc_tables.c | 11 +
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/include/miscadmin.h | 2 +
src/include/storage/md.h | 28 --
src/include/storage/smgr.h | 93 +++++--
src/tools/pgindent/typedefs.list | 1 +
12 files changed, 448 insertions(+), 161 deletions(-)
diff --git a/contrib/fsync_checker/fsync_checker_smgr.c b/contrib/fsync_checker/fsync_checker_smgr.c
index 97ad0f78da8..626ff058764 100644
--- a/contrib/fsync_checker/fsync_checker_smgr.c
+++ b/contrib/fsync_checker/fsync_checker_smgr.c
@@ -27,15 +27,15 @@ typedef struct
void _PG_init(void);
static void fsync_checker_extend(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
- const void *buffer, bool skipFsync);
-static void fsync_checker_immedsync(SMgrRelation reln, ForkNumber forknum);
+ const void *buffer, bool skipFsync, SmgrChainIndex chain_index);
+static void fsync_checker_immedsync(SMgrRelation reln, ForkNumber forknum, SmgrChainIndex chain_index);
static void fsync_checker_writev(SMgrRelation reln, ForkNumber forknum,
BlockNumber blocknum, const void **buffers,
- BlockNumber nblocks, bool skipFsync);
+ BlockNumber nblocks, bool skipFsync, SmgrChainIndex chain_index);
static void fsync_checker_writeback(SMgrRelation reln, ForkNumber forknum,
- BlockNumber blocknum, BlockNumber nblocks);
+ BlockNumber blocknum, BlockNumber nblocks, SmgrChainIndex chain_index);
static void fsync_checker_zeroextend(SMgrRelation reln, ForkNumber forknum,
- BlockNumber blocknum, int nblocks, bool skipFsync);
+ BlockNumber blocknum, int nblocks, bool skipFsync, SmgrChainIndex chain_index);
static void fsync_checker_checkpoint_create(const CheckPoint *checkPoint);
static void fsync_checker_shmem_request(void);
@@ -47,24 +47,25 @@ static void remove_reln(SMgrRelation reln, ForkNumber forknum);
static SMgrId fsync_checker_smgr_id;
static const struct f_smgr fsync_checker_smgr = {
.name = "fsync_checker",
- .smgr_init = mdinit,
+ .chain_position = SMGR_CHAIN_MODIFIER,
+ .smgr_init = NULL,
.smgr_shutdown = NULL,
- .smgr_open = mdopen,
- .smgr_close = mdclose,
- .smgr_create = mdcreate,
- .smgr_exists = mdexists,
- .smgr_unlink = mdunlink,
+ .smgr_open = NULL,
+ .smgr_close = NULL,
+ .smgr_create = NULL,
+ .smgr_exists = NULL,
+ .smgr_unlink = NULL,
.smgr_extend = fsync_checker_extend,
.smgr_zeroextend = fsync_checker_zeroextend,
- .smgr_prefetch = mdprefetch,
- .smgr_maxcombine = mdmaxcombine,
- .smgr_readv = mdreadv,
+ .smgr_prefetch = NULL,
+ .smgr_maxcombine = NULL,
+ .smgr_readv = NULL,
.smgr_writev = fsync_checker_writev,
.smgr_writeback = fsync_checker_writeback,
- .smgr_nblocks = mdnblocks,
- .smgr_truncate = mdtruncate,
+ .smgr_nblocks = NULL,
+ .smgr_truncate = NULL,
.smgr_immedsync = fsync_checker_immedsync,
- .smgr_registersync = mdregistersync,
+ .smgr_registersync = NULL,
};
static HTAB *volatile_relns;
@@ -91,8 +92,6 @@ _PG_init(void)
* could use MdSmgrRelation as the parent.
*/
fsync_checker_smgr_id = smgr_register(&fsync_checker_smgr, 0);
-
- storage_manager_id = fsync_checker_smgr_id;
}
static void
@@ -200,50 +199,50 @@ remove_reln(SMgrRelation reln, ForkNumber forknum)
static void
fsync_checker_extend(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
- const void *buffer, bool skipFsync)
+ const void *buffer, bool skipFsync, SmgrChainIndex chain_index)
{
if (!SmgrIsTemp(reln) && !skipFsync)
add_reln(reln, forknum);
- mdextend(reln, forknum, blocknum, buffer, skipFsync);
+ smgr_extend_next(reln, forknum, blocknum, buffer, skipFsync, chain_index + 1);
}
static void
-fsync_checker_immedsync(SMgrRelation reln, ForkNumber forknum)
+fsync_checker_immedsync(SMgrRelation reln, ForkNumber forknum, SmgrChainIndex chain_index)
{
if (!SmgrIsTemp(reln))
remove_reln(reln, forknum);
- mdimmedsync(reln, forknum);
+ smgr_immedsync_next(reln, forknum, chain_index + 1);
}
static void
fsync_checker_writev(SMgrRelation reln, ForkNumber forknum,
BlockNumber blocknum, const void **buffers,
- BlockNumber nblocks, bool skipFsync)
+ BlockNumber nblocks, bool skipFsync, SmgrChainIndex chain_index)
{
if (!SmgrIsTemp(reln) && !skipFsync)
add_reln(reln, forknum);
- mdwritev(reln, forknum, blocknum, buffers, nblocks, skipFsync);
+ smgr_writev_next(reln, forknum, blocknum, buffers, nblocks, skipFsync, chain_index + 1);
}
static void
fsync_checker_writeback(SMgrRelation reln, ForkNumber forknum,
- BlockNumber blocknum, BlockNumber nblocks)
+ BlockNumber blocknum, BlockNumber nblocks, SmgrChainIndex chain_index)
{
if (!SmgrIsTemp(reln))
remove_reln(reln, forknum);
- mdwriteback(reln, forknum, blocknum, nblocks);
+ smgr_writeback_next(reln, forknum, blocknum, nblocks, chain_index + 1);
}
static void
fsync_checker_zeroextend(SMgrRelation reln, ForkNumber forknum,
- BlockNumber blocknum, int nblocks, bool skipFsync)
+ BlockNumber blocknum, int nblocks, bool skipFsync, SmgrChainIndex chain_index)
{
if (!SmgrIsTemp(reln) && !skipFsync)
add_reln(reln, forknum);
- mdzeroextend(reln, forknum, blocknum, nblocks, skipFsync);
+ smgr_zeroextend_next(reln, forknum, blocknum, nblocks, skipFsync, chain_index + 1);
}
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 88ea821573d..3a9fba2fbc5 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -926,6 +926,8 @@ PostmasterMain(int argc, char *argv[])
*/
process_shared_preload_libraries();
+ process_smgr_chain();
+
/*
* Initialize SSL library, if specified.
*/
diff --git a/src/backend/storage/smgr/md.c b/src/backend/storage/smgr/md.c
index 1766bbe1e57..963bd0e9cde 100644
--- a/src/backend/storage/smgr/md.c
+++ b/src/backend/storage/smgr/md.c
@@ -124,6 +124,33 @@ typedef MdSMgrRelationData *MdSMgrRelation;
/* don't try to open a segment, if not already open */
#define EXTENSION_DONT_OPEN (1 << 5)
+/* md storage manager functionality */
+static void mdinit(void);
+static void mdopen(SMgrRelation reln, SmgrChainIndex chain_index);
+static void mdclose(SMgrRelation reln, ForkNumber forknum, SmgrChainIndex chain_index);
+static void mdcreate(RelFileLocator relold, SMgrRelation reln, ForkNumber forknum, bool isRedo, SmgrChainIndex chain_index);
+static bool mdexists(SMgrRelation reln, ForkNumber forknum, SmgrChainIndex chain_index);
+static void mdunlink(RelFileLocatorBackend rlocator, ForkNumber forknum, bool isRedo, SmgrChainIndex chain_index);
+static void mdextend(SMgrRelation reln, ForkNumber forknum,
+ BlockNumber blocknum, const void *buffer, bool skipFsync, SmgrChainIndex chain_index);
+static void mdzeroextend(SMgrRelation reln, ForkNumber forknum,
+ BlockNumber blocknum, int nblocks, bool skipFsync, SmgrChainIndex chain_index);
+static bool mdprefetch(SMgrRelation reln, ForkNumber forknum,
+ BlockNumber blocknum, int nblocks, SmgrChainIndex chain_index);
+static uint32 mdmaxcombine(SMgrRelation reln, ForkNumber forknum,
+ BlockNumber blocknum, SmgrChainIndex chain_index);
+static void mdreadv(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
+ void **buffers, BlockNumber nblocks, SmgrChainIndex chain_index);
+static void mdwritev(SMgrRelation reln, ForkNumber forknum,
+ BlockNumber blocknum,
+ const void **buffers, BlockNumber nblocks, bool skipFsync, SmgrChainIndex chain_index);
+static void mdwriteback(SMgrRelation reln, ForkNumber forknum,
+ BlockNumber blocknum, BlockNumber nblocks, SmgrChainIndex chain_index);
+static BlockNumber mdnblocks(SMgrRelation reln, ForkNumber forknum, SmgrChainIndex chain_index);
+static void mdtruncate(SMgrRelation reln, ForkNumber forknum,
+ BlockNumber old_blocks, BlockNumber nblocks, SmgrChainIndex chain_index);
+static void mdimmedsync(SMgrRelation reln, ForkNumber forknum, SmgrChainIndex chain_index);
+static void mdregistersync(SMgrRelation reln, ForkNumber forknum, SmgrChainIndex chain_index);
/*
* Fixed-length string to represent paths to files that need to be built by
@@ -151,6 +178,7 @@ mdsmgr_register(void)
/* magnetic disk */
f_smgr md_smgr = (f_smgr) {
.name = "md",
+ .chain_position = SMGR_CHAIN_TAIL,
.smgr_init = mdinit,
.smgr_shutdown = NULL,
.smgr_open = mdopen,
@@ -210,7 +238,7 @@ _mdfd_open_flags(void)
/*
* mdinit() -- Initialize private state for magnetic disk storage manager.
*/
-void
+static void
mdinit(void)
{
MdCxt = AllocSetContextCreate(TopMemoryContext,
@@ -223,8 +251,8 @@ mdinit(void)
*
* Note: this will return true for lingering files, with pending deletions
*/
-bool
-mdexists(SMgrRelation reln, ForkNumber forknum)
+static bool
+mdexists(SMgrRelation reln, ForkNumber forknum, SmgrChainIndex chain_index)
{
MdSMgrRelation mdreln = (MdSMgrRelation) reln;
@@ -234,7 +262,7 @@ mdexists(SMgrRelation reln, ForkNumber forknum)
* which already closes relations when dropping them.
*/
if (!InRecovery)
- mdclose(reln, forknum);
+ mdclose(reln, forknum, 0);
return (mdopenfork(mdreln, forknum, EXTENSION_RETURN_NULL) != NULL);
}
@@ -244,8 +272,8 @@ mdexists(SMgrRelation reln, ForkNumber forknum)
*
* If isRedo is true, it's okay for the relation to exist already.
*/
-void
-mdcreate(RelFileLocator /* reln */, SMgrRelation reln, ForkNumber forknum, bool isRedo)
+static void
+mdcreate(RelFileLocator relold, SMgrRelation reln, ForkNumber forknum, bool isRedo, SmgrChainIndex chain_index)
{
MdSMgrRelation mdreln = (MdSMgrRelation) reln;
MdfdVec *mdfd;
@@ -360,8 +388,8 @@ mdcreate(RelFileLocator /* reln */, SMgrRelation reln, ForkNumber forknum, bool
* Note: any failure should be reported as WARNING not ERROR, because
* we are usually not in a transaction anymore when this is called.
*/
-void
-mdunlink(RelFileLocatorBackend rlocator, ForkNumber forknum, bool isRedo)
+static void
+mdunlink(RelFileLocatorBackend rlocator, ForkNumber forknum, bool isRedo, SmgrChainIndex chain_index)
{
/* Now do the per-fork work */
if (forknum == InvalidForkNumber)
@@ -510,9 +538,9 @@ mdunlinkfork(RelFileLocatorBackend rlocator, ForkNumber forknum, bool isRedo)
* EOF). Note that we assume writing a block beyond current EOF
* causes intervening file space to become filled with zeroes.
*/
-void
+static void
mdextend(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
- const void *buffer, bool skipFsync)
+ const void *buffer, bool skipFsync, SmgrChainIndex chain_index)
{
MdSMgrRelation mdreln = (MdSMgrRelation) reln;
off_t seekpos;
@@ -576,9 +604,9 @@ mdextend(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
* Similar to mdextend(), except the relation can be extended by multiple
* blocks at once and the added blocks will be filled with zeroes.
*/
-void
+static void
mdzeroextend(SMgrRelation reln, ForkNumber forknum,
- BlockNumber blocknum, int nblocks, bool skipFsync)
+ BlockNumber blocknum, int nblocks, bool skipFsync, SmgrChainIndex chain_index)
{
MdSMgrRelation mdreln = (MdSMgrRelation) reln;
MdfdVec *v;
@@ -727,8 +755,8 @@ mdopenfork(MdSMgrRelation reln, ForkNumber forknum, int behavior)
/*
* mdopen() -- Initialize newly-opened relation.
*/
-void
-mdopen(SMgrRelation reln)
+static void
+mdopen(SMgrRelation reln, SmgrChainIndex chain_index)
{
MdSMgrRelation mdreln = (MdSMgrRelation) reln;
@@ -740,8 +768,8 @@ mdopen(SMgrRelation reln)
/*
* mdclose() -- Close the specified relation, if it isn't closed already.
*/
-void
-mdclose(SMgrRelation reln, ForkNumber forknum)
+static void
+mdclose(SMgrRelation reln, ForkNumber forknum, SmgrChainIndex chain_index)
{
MdSMgrRelation mdreln = (MdSMgrRelation) reln;
int nopensegs = mdreln->md_num_open_segs[forknum];
@@ -764,9 +792,9 @@ mdclose(SMgrRelation reln, ForkNumber forknum)
/*
* mdprefetch() -- Initiate asynchronous read of the specified blocks of a relation
*/
-bool
+static bool
mdprefetch(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
- int nblocks)
+ int nblocks, SmgrChainIndex chain_index)
{
#ifdef USE_PREFETCH
MdSMgrRelation mdreln = (MdSMgrRelation) reln;
@@ -862,9 +890,9 @@ buffers_to_iovec(struct iovec *iov, void **buffers, int nblocks)
* mdmaxcombine() -- Return the maximum number of total blocks that can be
* combined with an IO starting at blocknum.
*/
-uint32
+static uint32
mdmaxcombine(SMgrRelation reln, ForkNumber forknum,
- BlockNumber blocknum)
+ BlockNumber blocknum, SmgrChainIndex index)
{
BlockNumber segoff;
@@ -876,9 +904,9 @@ mdmaxcombine(SMgrRelation reln, ForkNumber forknum,
/*
* mdreadv() -- Read the specified blocks from a relation.
*/
-void
+static void
mdreadv(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
- void **buffers, BlockNumber nblocks)
+ void **buffers, BlockNumber nblocks, SmgrChainIndex chain_index)
{
MdSMgrRelation mdreln = (MdSMgrRelation) reln;
@@ -999,9 +1027,9 @@ mdreadv(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
* relation (ie, those before the current EOF). To extend a relation,
* use mdextend().
*/
-void
+static void
mdwritev(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
- const void **buffers, BlockNumber nblocks, bool skipFsync)
+ const void **buffers, BlockNumber nblocks, bool skipFsync, SmgrChainIndex chain_index)
{
MdSMgrRelation mdreln = (MdSMgrRelation) reln;
@@ -1106,9 +1134,9 @@ mdwritev(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
* This accepts a range of blocks because flushing several pages at once is
* considerably more efficient than doing so individually.
*/
-void
+static void
mdwriteback(SMgrRelation reln, ForkNumber forknum,
- BlockNumber blocknum, BlockNumber nblocks)
+ BlockNumber blocknum, BlockNumber nblocks, SmgrChainIndex chain_index)
{
MdSMgrRelation mdreln = (MdSMgrRelation) reln;
@@ -1167,8 +1195,8 @@ mdwriteback(SMgrRelation reln, ForkNumber forknum,
* called, then only segments up to the last one actually touched
* are present in the array.
*/
-BlockNumber
-mdnblocks(SMgrRelation reln, ForkNumber forknum)
+static BlockNumber
+mdnblocks(SMgrRelation reln, ForkNumber forknum, SmgrChainIndex chain_index)
{
MdSMgrRelation mdreln = (MdSMgrRelation) reln;
MdfdVec *v;
@@ -1232,9 +1260,9 @@ mdnblocks(SMgrRelation reln, ForkNumber forknum)
* sure we have opened all active segments, so that truncate loop will get
* them all!
*/
-void
+static void
mdtruncate(SMgrRelation reln, ForkNumber forknum,
- BlockNumber curnblk, BlockNumber nblocks)
+ BlockNumber curnblk, BlockNumber nblocks, SmgrChainIndex chain_index)
{
MdSMgrRelation mdreln = (MdSMgrRelation) reln;
BlockNumber priorblocks;
@@ -1322,8 +1350,8 @@ mdtruncate(SMgrRelation reln, ForkNumber forknum,
/*
* mdregistersync() -- Mark whole relation as needing fsync
*/
-void
-mdregistersync(SMgrRelation reln, ForkNumber forknum)
+static void
+mdregistersync(SMgrRelation reln, ForkNumber forknum, SmgrChainIndex chain_index)
{
MdSMgrRelation mdreln = (MdSMgrRelation) reln;
int segno;
@@ -1333,7 +1361,7 @@ mdregistersync(SMgrRelation reln, ForkNumber forknum)
* NOTE: mdnblocks makes sure we have opened all active segments, so that
* the loop below will get them all!
*/
- mdnblocks(reln, forknum);
+ mdnblocks(reln, forknum, 0);
min_inactive_seg = segno = mdreln->md_num_open_segs[forknum];
@@ -1374,8 +1402,8 @@ mdregistersync(SMgrRelation reln, ForkNumber forknum)
* crash before the next checkpoint syncs the newly-inactive segment, that
* segment may survive recovery, reintroducing unwanted data into the table.
*/
-void
-mdimmedsync(SMgrRelation reln, ForkNumber forknum)
+static void
+mdimmedsync(SMgrRelation reln, ForkNumber forknum, SmgrChainIndex chain_index)
{
MdSMgrRelation mdreln = (MdSMgrRelation) reln;
int segno;
@@ -1385,7 +1413,7 @@ mdimmedsync(SMgrRelation reln, ForkNumber forknum)
* NOTE: mdnblocks makes sure we have opened all active segments, so that
* the loop below will get them all!
*/
- mdnblocks(reln, forknum);
+ mdnblocks(reln, forknum, 0);
min_inactive_seg = segno = mdreln->md_num_open_segs[forknum];
@@ -1750,7 +1778,7 @@ _mdfd_getseg(MdSMgrRelation reln, ForkNumber forknum, BlockNumber blkno,
mdextend((SMgrRelation) reln, forknum,
nextsegno * ((BlockNumber) RELSEG_SIZE) - 1,
- zerobuf, skipFsync);
+ zerobuf, skipFsync, 0);
pfree(zerobuf);
}
flags = O_CREAT;
diff --git a/src/backend/storage/smgr/smgr.c b/src/backend/storage/smgr/smgr.c
index 0498fd6c317..08892563768 100644
--- a/src/backend/storage/smgr/smgr.c
+++ b/src/backend/storage/smgr/smgr.c
@@ -63,13 +63,13 @@
#include "utils/inval.h"
#include "utils/memutils.h"
-static f_smgr *smgrsw;
+f_smgr *smgrsw;
static int NSmgr = 0;
static Size LargestSMgrRelationSize = 0;
-SMgrId storage_manager_id;
+SMgrChain storage_manager_chain;
/*
* Each backend has a hashtable that stores all extant SMgrRelation objects.
@@ -98,20 +98,23 @@ smgr_register(const f_smgr *smgr, Size smgrrelation_size)
if (smgr->name == NULL || *smgr->name == 0)
elog(FATAL, "smgr registered with invalid name");
- Assert(smgr->smgr_open != NULL);
- Assert(smgr->smgr_close != NULL);
- Assert(smgr->smgr_create != NULL);
- Assert(smgr->smgr_exists != NULL);
- Assert(smgr->smgr_unlink != NULL);
- Assert(smgr->smgr_extend != NULL);
- Assert(smgr->smgr_zeroextend != NULL);
- Assert(smgr->smgr_prefetch != NULL);
- Assert(smgr->smgr_readv != NULL);
- Assert(smgr->smgr_writev != NULL);
- Assert(smgr->smgr_writeback != NULL);
- Assert(smgr->smgr_nblocks != NULL);
- Assert(smgr->smgr_truncate != NULL);
- Assert(smgr->smgr_immedsync != NULL);
+ if (smgr->chain_position == SMGR_CHAIN_TAIL)
+ {
+ Assert(smgr->smgr_open != NULL);
+ Assert(smgr->smgr_close != NULL);
+ Assert(smgr->smgr_create != NULL);
+ Assert(smgr->smgr_exists != NULL);
+ Assert(smgr->smgr_unlink != NULL);
+ Assert(smgr->smgr_extend != NULL);
+ Assert(smgr->smgr_zeroextend != NULL);
+ Assert(smgr->smgr_prefetch != NULL);
+ Assert(smgr->smgr_readv != NULL);
+ Assert(smgr->smgr_writev != NULL);
+ Assert(smgr->smgr_writeback != NULL);
+ Assert(smgr->smgr_nblocks != NULL);
+ Assert(smgr->smgr_truncate != NULL);
+ Assert(smgr->smgr_immedsync != NULL);
+ }
old = MemoryContextSwitchTo(TopMemoryContext);
@@ -138,6 +141,17 @@ smgr_register(const f_smgr *smgr, Size smgrrelation_size)
return my_id;
}
+SMgrId
+smgr_lookup(const char *name)
+{
+ for (int i = 0; i < NSmgr; i++)
+ {
+ if (strcmp(smgrsw[i].name, name) == 0)
+ return i;
+ }
+ elog(FATAL, "Storage manager not found with name: %s", name);
+}
+
/*
* smgrinit(), smgrshutdown() -- Initialize or shut down storage
* managers.
@@ -176,6 +190,22 @@ smgrshutdown(int code, Datum arg)
}
}
+#define SMGR_CHAIN_LOOKUP(SMGR_METHOD) \
+ do \
+ { \
+ while (chain_index < reln->smgr_chain.size && smgrsw[reln->smgr_chain.chain[chain_index]].SMGR_METHOD == NULL) \
+ chain_index++; \
+ Assert(chain_index < reln->smgr_chain.size); \
+ } while (0)
+
+void
+smgr_open_next(SMgrRelation reln, SmgrChainIndex chain_index)
+{
+ SMGR_CHAIN_LOOKUP(smgr_open);
+
+ smgrsw[reln->smgr_chain.chain[chain_index]].smgr_open(reln, chain_index);
+}
+
/*
* smgropen() -- Return an SMgrRelation object, creating it if need be.
*
@@ -229,10 +259,10 @@ smgropen(RelFileLocator rlocator, ProcNumber backend)
for (int i = 0; i <= MAX_FORKNUM; ++i)
reln->smgr_cached_nblocks[i] = InvalidBlockNumber;
- reln->smgr_which = storage_manager_id;
+ memcpy(&reln->smgr_chain, &storage_manager_chain, sizeof(SMgrChain));
/* implementation-specific initialization */
- smgrsw[reln->smgr_which].smgr_open(reln);
+ smgr_open_next(reln, 0);
/* it is not pinned yet */
reln->pincount = 0;
@@ -270,6 +300,14 @@ smgrunpin(SMgrRelation reln)
dlist_push_tail(&unpinned_relns, &reln->node);
}
+void
+smgr_close_next(SMgrRelation reln, ForkNumber forknum, SmgrChainIndex chain_index)
+{
+ SMGR_CHAIN_LOOKUP(smgr_close);
+
+ smgrsw[reln->smgr_chain.chain[chain_index]].smgr_close(reln, forknum, chain_index);
+}
+
/*
* smgrdestroy() -- Delete an SMgrRelation object.
*/
@@ -281,7 +319,7 @@ smgrdestroy(SMgrRelation reln)
Assert(reln->pincount == 0);
for (forknum = 0; forknum <= MAX_FORKNUM; forknum++)
- smgrsw[reln->smgr_which].smgr_close(reln, forknum);
+ smgr_close_next(reln, forknum, 0);
dlist_delete(&reln->node);
@@ -301,7 +339,7 @@ smgrrelease(SMgrRelation reln)
{
for (ForkNumber forknum = 0; forknum <= MAX_FORKNUM; forknum++)
{
- smgrsw[reln->smgr_which].smgr_close(reln, forknum);
+ smgr_close_next(reln, forknum, 0);
reln->smgr_cached_nblocks[forknum] = InvalidBlockNumber;
}
reln->smgr_targblock = InvalidBlockNumber;
@@ -391,13 +429,29 @@ smgrreleaserellocator(RelFileLocatorBackend rlocator)
smgrrelease(reln);
}
+bool
+smgr_exists_next(SMgrRelation reln, ForkNumber forknum, SmgrChainIndex chain_index)
+{
+ SMGR_CHAIN_LOOKUP(smgr_exists);
+
+ return smgrsw[reln->smgr_chain.chain[chain_index]].smgr_exists(reln, forknum, chain_index);
+}
+
/*
* smgrexists() -- Does the underlying file for a fork exist?
*/
bool
smgrexists(SMgrRelation reln, ForkNumber forknum)
{
- return smgrsw[reln->smgr_which].smgr_exists(reln, forknum);
+ return smgr_exists_next(reln, forknum, 0);
+}
+
+void
+smgr_create_next(RelFileLocator relold, SMgrRelation reln, ForkNumber forknum, bool isRedo, SmgrChainIndex chain_index)
+{
+ SMGR_CHAIN_LOOKUP(smgr_create);
+
+ smgrsw[reln->smgr_chain.chain[chain_index]].smgr_create(relold, reln, forknum, isRedo, chain_index);
}
/*
@@ -410,7 +464,15 @@ smgrexists(SMgrRelation reln, ForkNumber forknum)
void
smgrcreate(RelFileLocator relold, SMgrRelation reln, ForkNumber forknum, bool isRedo)
{
- smgrsw[reln->smgr_which].smgr_create(relold, reln, forknum, isRedo);
+ smgr_create_next(relold, reln, forknum, isRedo, 0);
+}
+
+void
+smgr_immedsync_next(SMgrRelation reln, ForkNumber forknum, SmgrChainIndex chain_index)
+{
+ SMGR_CHAIN_LOOKUP(smgr_immedsync);
+
+ smgrsw[reln->smgr_chain.chain[chain_index]].smgr_immedsync(reln, forknum, chain_index);
}
/*
@@ -438,16 +500,22 @@ smgrdosyncall(SMgrRelation *rels, int nrels)
*/
for (i = 0; i < nrels; i++)
{
- int which = rels[i]->smgr_which;
-
for (forknum = 0; forknum <= MAX_FORKNUM; forknum++)
{
- if (smgrsw[which].smgr_exists(rels[i], forknum))
- smgrsw[which].smgr_immedsync(rels[i], forknum);
+ if (smgr_exists_next(rels[i], forknum, 0))
+ smgr_immedsync_next(rels[i], forknum, 0);
}
}
}
+void
+smgr_unlink_next(SMgrRelation reln, RelFileLocatorBackend rlocator, ForkNumber forknum, bool isRedo, SmgrChainIndex chain_index)
+{
+ SMGR_CHAIN_LOOKUP(smgr_unlink);
+
+ smgrsw[reln->smgr_chain.chain[chain_index]].smgr_unlink(rlocator, forknum, isRedo, chain_index);
+}
+
/*
* smgrdounlinkall() -- Immediately unlink all forks of all given relations
*
@@ -482,13 +550,12 @@ smgrdounlinkall(SMgrRelation *rels, int nrels, bool isRedo)
for (i = 0; i < nrels; i++)
{
RelFileLocatorBackend rlocator = rels[i]->smgr_rlocator;
- int which = rels[i]->smgr_which;
rlocators[i] = rlocator;
/* Close the forks at smgr level */
for (forknum = 0; forknum <= MAX_FORKNUM; forknum++)
- smgrsw[which].smgr_close(rels[i], forknum);
+ smgr_close_next(rels[i], forknum, 0);
}
/*
@@ -512,15 +579,22 @@ smgrdounlinkall(SMgrRelation *rels, int nrels, bool isRedo)
for (i = 0; i < nrels; i++)
{
- int which = rels[i]->smgr_which;
-
for (forknum = 0; forknum <= MAX_FORKNUM; forknum++)
- smgrsw[which].smgr_unlink(rlocators[i], forknum, isRedo);
+ smgr_unlink_next(rels[i], rlocators[i], forknum, isRedo, 0);
}
pfree(rlocators);
}
+void
+smgr_extend_next(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
+ const void *buffer, bool skipFsync, SmgrChainIndex chain_index)
+{
+ SMGR_CHAIN_LOOKUP(smgr_extend);
+
+ smgrsw[reln->smgr_chain.chain[chain_index]].smgr_extend(reln, forknum, blocknum,
+ buffer, skipFsync, chain_index);
+}
/*
* smgrextend() -- Add a new block to a file.
@@ -535,8 +609,7 @@ void
smgrextend(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
const void *buffer, bool skipFsync)
{
- smgrsw[reln->smgr_which].smgr_extend(reln, forknum, blocknum,
- buffer, skipFsync);
+ smgr_extend_next(reln, forknum, blocknum, buffer, skipFsync, 0);
/*
* Normally we expect this to increase nblocks by one, but if the cached
@@ -549,6 +622,16 @@ smgrextend(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
reln->smgr_cached_nblocks[forknum] = InvalidBlockNumber;
}
+void
+smgr_zeroextend_next(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
+ int nblocks, bool skipFsync, SmgrChainIndex chain_index)
+{
+ SMGR_CHAIN_LOOKUP(smgr_zeroextend);
+
+ smgrsw[reln->smgr_chain.chain[chain_index]].smgr_zeroextend(reln, forknum, blocknum,
+ nblocks, skipFsync, chain_index);
+}
+
/*
* smgrzeroextend() -- Add new zeroed out blocks to a file.
*
@@ -560,8 +643,7 @@ void
smgrzeroextend(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
int nblocks, bool skipFsync)
{
- smgrsw[reln->smgr_which].smgr_zeroextend(reln, forknum, blocknum,
- nblocks, skipFsync);
+ smgr_zeroextend_next(reln, forknum, blocknum, nblocks, skipFsync, 0);
/*
* Normally we expect this to increase the fork size by nblocks, but if
@@ -574,6 +656,16 @@ smgrzeroextend(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
reln->smgr_cached_nblocks[forknum] = InvalidBlockNumber;
}
+bool
+smgr_prefetch_next(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
+ int nblocks, SmgrChainIndex chain_index)
+{
+ SMGR_CHAIN_LOOKUP(smgr_prefetch);
+
+ return smgrsw[reln->smgr_chain.chain[chain_index]].smgr_prefetch(reln, forknum, blocknum,
+ nblocks, chain_index);
+}
+
/*
* smgrprefetch() -- Initiate asynchronous read of the specified block of a relation.
*
@@ -585,7 +677,16 @@ bool
smgrprefetch(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
int nblocks)
{
- return smgrsw[reln->smgr_which].smgr_prefetch(reln, forknum, blocknum, nblocks);
+ return smgr_prefetch_next(reln, forknum, blocknum, nblocks, 0);
+}
+
+uint32
+smgr_maxcombine_next(SMgrRelation reln, ForkNumber forknum,
+ BlockNumber blocknum, SmgrChainIndex chain_index)
+{
+ SMGR_CHAIN_LOOKUP(smgr_maxcombine);
+
+ return smgrsw[reln->smgr_chain.chain[chain_index]].smgr_maxcombine(reln, forknum, blocknum, chain_index);
}
/*
@@ -598,7 +699,17 @@ uint32
smgrmaxcombine(SMgrRelation reln, ForkNumber forknum,
BlockNumber blocknum)
{
- return smgrsw[reln->smgr_which].smgr_maxcombine(reln, forknum, blocknum);
+ return smgr_maxcombine_next(reln, forknum, blocknum, 0);
+}
+
+void
+smgr_readv_next(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
+ void **buffers, BlockNumber nblocks, SmgrChainIndex chain_index)
+{
+ SMGR_CHAIN_LOOKUP(smgr_readv);
+
+ smgrsw[reln->smgr_chain.chain[chain_index]].smgr_readv(reln, forknum, blocknum,
+ buffers, nblocks, chain_index);
}
/*
@@ -616,8 +727,17 @@ void
smgrreadv(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
void **buffers, BlockNumber nblocks)
{
- smgrsw[reln->smgr_which].smgr_readv(reln, forknum, blocknum, buffers,
- nblocks);
+ smgr_readv_next(reln, forknum, blocknum, buffers, nblocks, 0);
+}
+
+void
+smgr_writev_next(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
+ const void **buffers, BlockNumber nblocks, bool skipFsync, SmgrChainIndex chain_index)
+{
+ SMGR_CHAIN_LOOKUP(smgr_writev);
+
+ smgrsw[reln->smgr_chain.chain[chain_index]].smgr_writev(reln, forknum, blocknum,
+ buffers, nblocks, skipFsync, chain_index);
}
/*
@@ -650,8 +770,17 @@ void
smgrwritev(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
const void **buffers, BlockNumber nblocks, bool skipFsync)
{
- smgrsw[reln->smgr_which].smgr_writev(reln, forknum, blocknum,
- buffers, nblocks, skipFsync);
+ smgr_writev_next(reln, forknum, blocknum,
+ buffers, nblocks, skipFsync, 0);
+}
+
+void
+smgr_writeback_next(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
+ BlockNumber nblocks, SmgrChainIndex chain_index)
+{
+ SMGR_CHAIN_LOOKUP(smgr_writeback);
+
+ smgrsw[reln->smgr_chain.chain[chain_index]].smgr_writeback(reln, forknum, blocknum, nblocks, chain_index);
}
/*
@@ -662,8 +791,15 @@ void
smgrwriteback(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
BlockNumber nblocks)
{
- smgrsw[reln->smgr_which].smgr_writeback(reln, forknum, blocknum,
- nblocks);
+ smgr_writeback_next(reln, forknum, blocknum, nblocks, 0);
+}
+
+extern BlockNumber
+smgr_nblocks_next(SMgrRelation reln, ForkNumber forknum, SmgrChainIndex chain_index)
+{
+ SMGR_CHAIN_LOOKUP(smgr_nblocks);
+
+ return smgrsw[reln->smgr_chain.chain[chain_index]].smgr_nblocks(reln, forknum, chain_index);
}
/*
@@ -680,7 +816,7 @@ smgrnblocks(SMgrRelation reln, ForkNumber forknum)
if (result != InvalidBlockNumber)
return result;
- result = smgrsw[reln->smgr_which].smgr_nblocks(reln, forknum);
+ result = smgr_nblocks_next(reln, forknum, 0);
reln->smgr_cached_nblocks[forknum] = result;
@@ -708,6 +844,14 @@ smgrnblocks_cached(SMgrRelation reln, ForkNumber forknum)
return InvalidBlockNumber;
}
+void
+smgr_truncate_next(SMgrRelation reln, ForkNumber forknum, BlockNumber curnblk, BlockNumber nblocks, SmgrChainIndex chain_index)
+{
+ SMGR_CHAIN_LOOKUP(smgr_truncate);
+
+ smgrsw[reln->smgr_chain.chain[chain_index]].smgr_truncate(reln, forknum, curnblk, nblocks, chain_index);
+}
+
/*
* smgrtruncate() -- Truncate the given forks of supplied relation to
* each specified numbers of blocks
@@ -752,8 +896,7 @@ smgrtruncate(SMgrRelation reln, ForkNumber *forknum, int nforks,
/* Make the cached size is invalid if we encounter an error. */
reln->smgr_cached_nblocks[forknum[i]] = InvalidBlockNumber;
- smgrsw[reln->smgr_which].smgr_truncate(reln, forknum[i],
- old_nblocks[i], nblocks[i]);
+ smgr_truncate_next(reln, forknum[i], old_nblocks[i], nblocks[i], 0);
/*
* We might as well update the local smgr_cached_nblocks values. The
@@ -766,6 +909,14 @@ smgrtruncate(SMgrRelation reln, ForkNumber *forknum, int nforks,
}
}
+void
+smgr_registersync_next(SMgrRelation reln, ForkNumber forknum, SmgrChainIndex chain_index)
+{
+ SMGR_CHAIN_LOOKUP(smgr_registersync);
+
+ smgrsw[reln->smgr_chain.chain[chain_index]].smgr_registersync(reln, forknum, chain_index);
+}
+
/*
* smgrregistersync() -- Request a relation to be sync'd at next checkpoint
*
@@ -781,7 +932,7 @@ smgrtruncate(SMgrRelation reln, ForkNumber *forknum, int nforks,
void
smgrregistersync(SMgrRelation reln, ForkNumber forknum)
{
- smgrsw[reln->smgr_which].smgr_registersync(reln, forknum);
+ smgr_registersync_next(reln, forknum, 0);
}
/*
@@ -813,7 +964,7 @@ smgrregistersync(SMgrRelation reln, ForkNumber forknum)
void
smgrimmedsync(SMgrRelation reln, ForkNumber forknum)
{
- smgrsw[reln->smgr_which].smgr_immedsync(reln, forknum);
+ smgr_immedsync_next(reln, forknum, 0);
}
/*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 947ffb40421..d523f306ab8 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -4071,6 +4071,8 @@ PostgresSingleUserMain(int argc, char *argv[],
*/
process_shared_preload_libraries();
+ process_smgr_chain();
+
/* Initialize MaxBackends */
InitializeMaxBackends();
diff --git a/src/backend/utils/init/miscinit.c b/src/backend/utils/init/miscinit.c
index 1b3ce51cfce..32d99e1244a 100644
--- a/src/backend/utils/init/miscinit.c
+++ b/src/backend/utils/init/miscinit.c
@@ -56,6 +56,7 @@
#include "utils/pidfile.h"
#include "utils/syscache.h"
#include "utils/varlena.h"
+#include "storage/smgr.h"
#define DIRECTORY_LOCK_FILE "postmaster.pid"
@@ -1834,6 +1835,8 @@ char *session_preload_libraries_string = NULL;
char *shared_preload_libraries_string = NULL;
char *local_preload_libraries_string = NULL;
+char *smgr_chain_string = NULL;
+
/* Flag telling that we are loading shared_preload_libraries */
bool process_shared_preload_libraries_in_progress = false;
bool process_shared_preload_libraries_done = false;
@@ -1910,6 +1913,62 @@ process_shared_preload_libraries(void)
process_shared_preload_libraries_done = true;
}
+void
+process_smgr_chain(void)
+{
+ char *rawstring;
+ List *elemlist;
+ ListCell *l;
+ uint8 idx = 0;
+
+ if (smgr_chain_string == NULL || smgr_chain_string[0] == '\0')
+ return; /* nothing to do */
+
+ /* Need a modifiable copy of string */
+ rawstring = pstrdup(smgr_chain_string);
+
+ /* Parse string into list of filename paths */
+ if (!SplitIdentifierString(rawstring, ',', &elemlist))
+ {
+ /* syntax error in list */
+ pfree(rawstring);
+ ereport(LOG,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("invalid list syntax in parameter \"%s\"",
+ "smgr_chain")));
+ return;
+ }
+
+ foreach(l, elemlist)
+ {
+ char *smgrname = (char *) lfirst(l);
+ SMgrId id = smgr_lookup(smgrname);
+
+ storage_manager_chain.chain[idx++] = id;
+
+ ereport(DEBUG1,
+ (errmsg_internal("using storage manager in chain \"%s\"", smgrname)));
+ }
+
+ for (int i = 0; i < idx; ++i)
+ {
+ int chain_position = smgrsw[storage_manager_chain.chain[i]].chain_position;
+
+ if (i == idx - 1 && chain_position != SMGR_CHAIN_TAIL)
+ ereport(FATAL,
+ (errmsg_internal("smgr_chain: the last element should be a `tail` implementation, not a modifier.")));
+
+ if (i != idx - 1 && chain_position != SMGR_CHAIN_MODIFIER)
+ ereport(FATAL,
+ (errmsg_internal("smgr_chain: element %i/%i %s is not a modifier.", i, idx, smgrsw[storage_manager_chain.chain[i]].name)));
+ }
+
+ storage_manager_chain.size = idx;
+
+ list_free(elemlist);
+ pfree(rawstring);
+}
+
/*
* process any libraries that should be preloaded at backend start
*/
@@ -1932,7 +1991,9 @@ register_builtin_dynamic_managers(void)
{
mdsmgr_register();
- storage_manager_id = MdSMgrId;
+ /* setup a dummy chain with md, for tools */
+ storage_manager_chain.chain[0] = MdSMgrId;
+ storage_manager_chain.size = 1;
}
/*
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index ad25cbb39c5..ea43aadc96a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -4408,6 +4408,17 @@ struct config_string ConfigureNamesString[] =
NULL, NULL, NULL
},
+ {
+ {"smgr_chain", PGC_POSTMASTER, CLIENT_CONN_PRELOAD,
+ gettext_noop("Lists storage managers used by the server, in order."),
+ NULL,
+ GUC_LIST_INPUT | GUC_LIST_QUOTE | GUC_SUPERUSER_ONLY
+ },
+ &smgr_chain_string,
+ "md",
+ NULL, NULL, NULL
+ },
+
{
{"search_path", PGC_USERSET, CLIENT_CONN_STATEMENT,
gettext_noop("Sets the schema search order for names that are not schema-qualified."),
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 2d1de9c37bd..84d1159c4d7 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -787,6 +787,7 @@ autovacuum_worker_slots = 16 # autovacuum worker slots to allocate
#session_preload_libraries = ''
#shared_preload_libraries = '' # (change requires restart)
#jit_provider = 'llvmjit' # JIT library to use
+#smgr_chain = 'md' # SMGR implementations to use
# - Other Defaults -
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index ff4ef578a1f..4e218941a4b 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -505,6 +505,7 @@ extern PGDLLIMPORT bool process_shmem_requests_in_progress;
extern PGDLLIMPORT char *session_preload_libraries_string;
extern PGDLLIMPORT char *shared_preload_libraries_string;
extern PGDLLIMPORT char *local_preload_libraries_string;
+extern PGDLLIMPORT char *smgr_chain_string;
extern void CreateDataDirLockFile(bool amPostmaster);
extern void CreateSocketLockFile(const char *socketfile, bool amPostmaster,
@@ -515,6 +516,7 @@ extern bool RecheckDataDirLockFile(void);
extern void ValidatePgVersion(const char *path);
extern void register_builtin_dynamic_managers(void);
extern void process_shared_preload_libraries(void);
+extern void process_smgr_chain(void);
extern void process_session_preload_libraries(void);
extern void process_shmem_requests(void);
extern void pg_bindtextdomain(const char *domain);
diff --git a/src/include/storage/md.h b/src/include/storage/md.h
index 61c0e85dd74..5b4992c0855 100644
--- a/src/include/storage/md.h
+++ b/src/include/storage/md.h
@@ -23,34 +23,6 @@
extern void mdsmgr_register(void);
extern SMgrId MdSMgrId;
-/* md storage manager functionality */
-extern void mdinit(void);
-extern void mdopen(SMgrRelation reln);
-extern void mdclose(SMgrRelation reln, ForkNumber forknum);
-extern void mdcreate(RelFileLocator relold, SMgrRelation reln, ForkNumber forknum, bool isRedo);
-extern bool mdexists(SMgrRelation reln, ForkNumber forknum);
-extern void mdunlink(RelFileLocatorBackend rlocator, ForkNumber forknum, bool isRedo);
-extern void mdextend(SMgrRelation reln, ForkNumber forknum,
- BlockNumber blocknum, const void *buffer, bool skipFsync);
-extern void mdzeroextend(SMgrRelation reln, ForkNumber forknum,
- BlockNumber blocknum, int nblocks, bool skipFsync);
-extern bool mdprefetch(SMgrRelation reln, ForkNumber forknum,
- BlockNumber blocknum, int nblocks);
-extern uint32 mdmaxcombine(SMgrRelation reln, ForkNumber forknum,
- BlockNumber blocknum);
-extern void mdreadv(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
- void **buffers, BlockNumber nblocks);
-extern void mdwritev(SMgrRelation reln, ForkNumber forknum,
- BlockNumber blocknum,
- const void **buffers, BlockNumber nblocks, bool skipFsync);
-extern void mdwriteback(SMgrRelation reln, ForkNumber forknum,
- BlockNumber blocknum, BlockNumber nblocks);
-extern BlockNumber mdnblocks(SMgrRelation reln, ForkNumber forknum);
-extern void mdtruncate(SMgrRelation reln, ForkNumber forknum,
- BlockNumber old_blocks, BlockNumber nblocks);
-extern void mdimmedsync(SMgrRelation reln, ForkNumber forknum);
-extern void mdregistersync(SMgrRelation reln, ForkNumber forknum);
-
extern void ForgetDatabaseSyncRequests(Oid dbid);
extern void DropRelationFiles(RelFileLocator *delrels, int ndelrels, bool isRedo);
diff --git a/src/include/storage/smgr.h b/src/include/storage/smgr.h
index 5b2b6de91c4..8f789cb7f80 100644
--- a/src/include/storage/smgr.h
+++ b/src/include/storage/smgr.h
@@ -20,7 +20,17 @@
typedef uint8 SMgrId;
-extern PGDLLIMPORT SMgrId storage_manager_id;
+typedef uint8 SmgrChainIndex;
+
+#define MAX_SMGR_CHAIN 15
+
+typedef struct
+{
+ SMgrId chain[MAX_SMGR_CHAIN]; /* storage manager selector */
+ uint8 size;
+} SMgrChain;
+
+extern PGDLLIMPORT SMgrChain storage_manager_chain;
/*
* smgr.c maintains a table of SMgrRelation objects, which are essentially
@@ -55,7 +65,7 @@ typedef struct SMgrRelationData
* Fields below here are intended to be private to smgr.c and its
* submodules. Do not touch them from elsewhere.
*/
- SMgrId smgr_which; /* storage manager selector */
+ SMgrChain smgr_chain; /* selected storage manager chain */
/*
* Pinning support. If unpinned (ie. pincount == 0), 'node' is a list
@@ -70,6 +80,9 @@ typedef SMgrRelationData *SMgrRelation;
#define SmgrIsTemp(smgr) \
RelFileLocatorBackendIsTemp((smgr)->smgr_rlocator)
+#define SMGR_CHAIN_TAIL 1
+#define SMGR_CHAIN_MODIFIER 2
+
/*
* This struct of function pointers defines the API between smgr.c and
* any individual storage manager module. Note that smgr subfunctions are
@@ -83,40 +96,44 @@ typedef SMgrRelationData *SMgrRelation;
typedef struct f_smgr
{
const char *name;
+ int chain_position;
void (*smgr_init) (void); /* may be NULL */
void (*smgr_shutdown) (void); /* may be NULL */
- void (*smgr_open) (SMgrRelation reln);
- void (*smgr_close) (SMgrRelation reln, ForkNumber forknum);
+ void (*smgr_open) (SMgrRelation reln, SmgrChainIndex chain_index);
+ void (*smgr_close) (SMgrRelation reln, ForkNumber forknum, SmgrChainIndex chain_index);
void (*smgr_create) (RelFileLocator relold, SMgrRelation reln, ForkNumber forknum,
- bool isRedo);
- bool (*smgr_exists) (SMgrRelation reln, ForkNumber forknum);
+ bool isRedo, SmgrChainIndex chain_index);
+ bool (*smgr_exists) (SMgrRelation reln, ForkNumber forknum, SmgrChainIndex chain_index);
void (*smgr_unlink) (RelFileLocatorBackend rlocator, ForkNumber forknum,
- bool isRedo);
+ bool isRedo, SmgrChainIndex chain_index);
void (*smgr_extend) (SMgrRelation reln, ForkNumber forknum,
- BlockNumber blocknum, const void *buffer, bool skipFsync);
+ BlockNumber blocknum, const void *buffer, bool skipFsync, SmgrChainIndex chain_index);
void (*smgr_zeroextend) (SMgrRelation reln, ForkNumber forknum,
- BlockNumber blocknum, int nblocks, bool skipFsync);
+ BlockNumber blocknum, int nblocks, bool skipFsync, SmgrChainIndex chain_index);
bool (*smgr_prefetch) (SMgrRelation reln, ForkNumber forknum,
- BlockNumber blocknum, int nblocks);
+ BlockNumber blocknum, int nblocks, SmgrChainIndex chain_index);
uint32 (*smgr_maxcombine) (SMgrRelation reln, ForkNumber forknum,
- BlockNumber blocknum);
+ BlockNumber blocknum, SmgrChainIndex chain_index);
void (*smgr_readv) (SMgrRelation reln, ForkNumber forknum,
BlockNumber blocknum,
- void **buffers, BlockNumber nblocks);
+ void **buffers, BlockNumber nblocks, SmgrChainIndex chain_index);
void (*smgr_writev) (SMgrRelation reln, ForkNumber forknum,
BlockNumber blocknum,
const void **buffers, BlockNumber nblocks,
- bool skipFsync);
+ bool skipFsync, SmgrChainIndex chain_index);
void (*smgr_writeback) (SMgrRelation reln, ForkNumber forknum,
- BlockNumber blocknum, BlockNumber nblocks);
- BlockNumber (*smgr_nblocks) (SMgrRelation reln, ForkNumber forknum);
+ BlockNumber blocknum, BlockNumber nblocks, SmgrChainIndex chain_index);
+ BlockNumber (*smgr_nblocks) (SMgrRelation reln, ForkNumber forknum, SmgrChainIndex chain_index);
void (*smgr_truncate) (SMgrRelation reln, ForkNumber forknum,
- BlockNumber old_blocks, BlockNumber nblocks);
- void (*smgr_immedsync) (SMgrRelation reln, ForkNumber forknum);
- void (*smgr_registersync) (SMgrRelation reln, ForkNumber forknum);
+ BlockNumber old_blocks, BlockNumber nblocks, SmgrChainIndex chain_index);
+ void (*smgr_immedsync) (SMgrRelation reln, ForkNumber forknum, SmgrChainIndex chain_index);
+ void (*smgr_registersync) (SMgrRelation reln, ForkNumber forknum, SmgrChainIndex chain_index);
} f_smgr;
extern SMgrId smgr_register(const f_smgr *smgr, Size smgrrelation_size);
+extern SMgrId smgr_lookup(const char *name);
+
+extern f_smgr *smgrsw;
extern void smgrinit(void);
extern SMgrRelation smgropen(RelFileLocator rlocator, ProcNumber backend);
@@ -158,6 +175,46 @@ extern void smgrregistersync(SMgrRelation reln, ForkNumber forknum);
extern void AtEOXact_SMgr(void);
extern bool ProcessBarrierSmgrRelease(void);
+extern void
+ smgr_open_next(SMgrRelation reln, SmgrChainIndex chain_index);
+extern void
+ smgr_close_next(SMgrRelation reln, ForkNumber forknum, SmgrChainIndex chain_index);
+extern bool
+ smgr_exists_next(SMgrRelation reln, ForkNumber forknum, SmgrChainIndex chain_index);
+extern void
+ smgr_create_next(RelFileLocator relold, SMgrRelation reln, ForkNumber forknum, bool isRedo, SmgrChainIndex chain_index);
+extern void
+ smgr_immedsync_next(SMgrRelation reln, ForkNumber forknum, SmgrChainIndex chain_index);
+extern void
+ smgr_unlink_next(SMgrRelation reln, RelFileLocatorBackend rlocator, ForkNumber forknum, bool isRedo, SmgrChainIndex chain_index);
+extern void
+ smgr_extend_next(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
+ const void *buffer, bool skipFsync, SmgrChainIndex chain_index);
+extern void
+ smgr_zeroextend_next(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
+ int nblocks, bool skipFsync, SmgrChainIndex chain_index);
+extern bool
+ smgr_prefetch_next(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
+ int nblocks, SmgrChainIndex chain_index);
+extern uint32
+ smgr_maxcombine_next(SMgrRelation reln, ForkNumber forknum,
+ BlockNumber blocknum, SmgrChainIndex chain_index);
+extern void
+ smgr_readv_next(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
+ void **buffers, BlockNumber nblocks, SmgrChainIndex chain_index);
+extern void
+ smgr_writev_next(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
+ const void **buffers, BlockNumber nblocks, bool skipFsync, SmgrChainIndex chain_index);
+extern void
+ smgr_writeback_next(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
+ BlockNumber nblocks, SmgrChainIndex chain_index);
+extern BlockNumber
+ smgr_nblocks_next(SMgrRelation reln, ForkNumber forknum, SmgrChainIndex chain_index);
+extern void
+ smgr_truncate_next(SMgrRelation reln, ForkNumber forknum, BlockNumber curnblk, BlockNumber nblocks, SmgrChainIndex chain_index);
+extern void
+ smgr_registersync_next(SMgrRelation reln, ForkNumber forknum, SmgrChainIndex chain_index);
+
static inline void
smgrread(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
void *buffer)
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index bc260e713ae..b1b485d5445 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2553,6 +2553,7 @@ SID_IDENTIFIER_AUTHORITY
SID_NAME_USE
SISeg
SIZE_T
+SMgrChain
SMgrRelation
SMgrRelationData
SMgrSortArray
--
2.47.2
[text/x-patch] v4-0005-Refactor-smgr-API-mdcreate-needs-the-old-relfilel.patch (13.3K, ../../[email protected]/3-v4-0005-Refactor-smgr-API-mdcreate-needs-the-old-relfilel.patch)
download | inline diff:
From 96523c547e90de64b116acf470c07307541795b1 Mon Sep 17 00:00:00 2001
From: Zsolt Parragi <[email protected]>
Date: Sat, 12 Oct 2024 22:01:28 +0100
Subject: [PATCH v4 5/6] Refactor smgr API: mdcreate needs the old
relfilelocator
With this change, mdcreate receives the old relfilelocator along
with the new for operations that create a new file for an existing
relation.
This is required for tde_heap in pg_tde.
---
src/backend/access/heap/heapam_handler.c | 10 ++++++----
src/backend/access/transam/xlogutils.c | 2 +-
src/backend/catalog/heap.c | 2 +-
src/backend/catalog/index.c | 2 +-
src/backend/catalog/storage.c | 8 ++++----
src/backend/commands/sequence.c | 2 +-
src/backend/commands/tablecmds.c | 4 ++--
src/backend/storage/buffer/bufmgr.c | 7 ++++---
src/backend/storage/smgr/md.c | 2 +-
src/backend/storage/smgr/smgr.c | 4 ++--
src/backend/utils/cache/relcache.c | 2 +-
src/include/catalog/storage.h | 3 ++-
src/include/storage/md.h | 2 +-
src/include/storage/smgr.h | 4 ++--
14 files changed, 29 insertions(+), 25 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index e78682c3cef..96463d1bb14 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -584,6 +584,8 @@ heapam_relation_set_new_filelocator(Relation rel,
{
SMgrRelation srel;
+ RelFileLocator oldlocator = rel->rd_locator;
+
/*
* Initialize to the minimum XID that could put tuples in the table. We
* know that no xacts older than RecentXmin are still running, so that
@@ -601,7 +603,7 @@ heapam_relation_set_new_filelocator(Relation rel,
*/
*minmulti = GetOldestMultiXactId();
- srel = RelationCreateStorage(*newrlocator, persistence, true);
+ srel = RelationCreateStorage(oldlocator, *newrlocator, persistence, true);
/*
* If required, set up an init fork for an unlogged table so that it can
@@ -611,7 +613,7 @@ heapam_relation_set_new_filelocator(Relation rel,
{
Assert(rel->rd_rel->relkind == RELKIND_RELATION ||
rel->rd_rel->relkind == RELKIND_TOASTVALUE);
- smgrcreate(srel, INIT_FORKNUM, false);
+ smgrcreate(oldlocator, srel, INIT_FORKNUM, false);
log_smgrcreate(newrlocator, INIT_FORKNUM);
}
@@ -644,7 +646,7 @@ heapam_relation_copy_data(Relation rel, const RelFileLocator *newrlocator)
* NOTE: any conflict in relfilenumber value will be caught in
* RelationCreateStorage().
*/
- dstrel = RelationCreateStorage(*newrlocator, rel->rd_rel->relpersistence, true);
+ dstrel = RelationCreateStorage(rel->rd_locator, *newrlocator, rel->rd_rel->relpersistence, true);
/* copy main fork */
RelationCopyStorage(RelationGetSmgr(rel), dstrel, MAIN_FORKNUM,
@@ -656,7 +658,7 @@ heapam_relation_copy_data(Relation rel, const RelFileLocator *newrlocator)
{
if (smgrexists(RelationGetSmgr(rel), forkNum))
{
- smgrcreate(dstrel, forkNum, false);
+ smgrcreate(rel->rd_locator, dstrel, forkNum, false);
/*
* WAL log creation if the relation is persistent, or this is the
diff --git a/src/backend/access/transam/xlogutils.c b/src/backend/access/transam/xlogutils.c
index c389b27f77d..2179d2f73fa 100644
--- a/src/backend/access/transam/xlogutils.c
+++ b/src/backend/access/transam/xlogutils.c
@@ -487,7 +487,7 @@ XLogReadBufferExtended(RelFileLocator rlocator, ForkNumber forknum,
* filesystem loses an inode during a crash. Better to write the data
* until we are actually told to delete the file.)
*/
- smgrcreate(smgr, forknum, true);
+ smgrcreate(rlocator, smgr, forknum, true);
lastblock = smgrnblocks(smgr, forknum);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index bd3554c0bfd..251d22f50b2 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -386,7 +386,7 @@ heap_create(const char *relname,
relpersistence,
relfrozenxid, relminmxid);
else if (RELKIND_HAS_STORAGE(rel->rd_rel->relkind))
- RelationCreateStorage(rel->rd_locator, relpersistence, true);
+ RelationCreateStorage(rel->rd_locator, rel->rd_locator, relpersistence, true);
else
Assert(false);
}
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 022b9b99b13..cc99d45f2ff 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -3089,7 +3089,7 @@ index_build(Relation heapRelation,
if (indexRelation->rd_rel->relpersistence == RELPERSISTENCE_UNLOGGED &&
!smgrexists(RelationGetSmgr(indexRelation), INIT_FORKNUM))
{
- smgrcreate(RelationGetSmgr(indexRelation), INIT_FORKNUM, false);
+ smgrcreate(indexRelation->rd_locator, RelationGetSmgr(indexRelation), INIT_FORKNUM, false);
log_smgrcreate(&indexRelation->rd_locator, INIT_FORKNUM);
indexRelation->rd_indam->ambuildempty(indexRelation);
}
diff --git a/src/backend/catalog/storage.c b/src/backend/catalog/storage.c
index 624ed41bbf3..59fa01decc5 100644
--- a/src/backend/catalog/storage.c
+++ b/src/backend/catalog/storage.c
@@ -118,7 +118,7 @@ AddPendingSync(const RelFileLocator *rlocator)
* pass register_delete = false.
*/
SMgrRelation
-RelationCreateStorage(RelFileLocator rlocator, char relpersistence,
+RelationCreateStorage(RelFileLocator oldlocator, RelFileLocator rlocator, char relpersistence,
bool register_delete)
{
SMgrRelation srel;
@@ -147,7 +147,7 @@ RelationCreateStorage(RelFileLocator rlocator, char relpersistence,
}
srel = smgropen(rlocator, procNumber);
- smgrcreate(srel, MAIN_FORKNUM, false);
+ smgrcreate(oldlocator, srel, MAIN_FORKNUM, false);
if (needs_wal)
log_smgrcreate(&srel->smgr_rlocator.locator, MAIN_FORKNUM);
@@ -976,7 +976,7 @@ smgr_redo(XLogReaderState *record)
SMgrRelation reln;
reln = smgropen(xlrec->rlocator, INVALID_PROC_NUMBER);
- smgrcreate(reln, xlrec->forkNum, true);
+ smgrcreate(xlrec->rlocator, reln, xlrec->forkNum, true);
}
else if (info == XLOG_SMGR_TRUNCATE)
{
@@ -997,7 +997,7 @@ smgr_redo(XLogReaderState *record)
* XLogReadBufferForRedo, we prefer to recreate the rel and replay the
* log as best we can until the drop is seen.
*/
- smgrcreate(reln, MAIN_FORKNUM, true);
+ smgrcreate(xlrec->rlocator, reln, MAIN_FORKNUM, true);
/*
* Before we perform the truncation, update minimum recovery point to
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index 4b7c5113aab..d8c560a11b2 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -344,7 +344,7 @@ fill_seq_with_data(Relation rel, HeapTuple tuple)
SMgrRelation srel;
srel = smgropen(rel->rd_locator, INVALID_PROC_NUMBER);
- smgrcreate(srel, INIT_FORKNUM, false);
+ smgrcreate(rel->rd_locator, srel, INIT_FORKNUM, false);
log_smgrcreate(&rel->rd_locator, INIT_FORKNUM);
fill_seq_fork_with_data(rel, tuple, INIT_FORKNUM);
FlushRelationBuffers(rel);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 59156a1c1f6..7d0d9d3efa9 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -16313,7 +16313,7 @@ index_copy_data(Relation rel, RelFileLocator newrlocator)
* NOTE: any conflict in relfilenumber value will be caught in
* RelationCreateStorage().
*/
- dstrel = RelationCreateStorage(newrlocator, rel->rd_rel->relpersistence, true);
+ dstrel = RelationCreateStorage(rel->rd_locator, newrlocator, rel->rd_rel->relpersistence, true);
/* copy main fork */
RelationCopyStorage(RelationGetSmgr(rel), dstrel, MAIN_FORKNUM,
@@ -16325,7 +16325,7 @@ index_copy_data(Relation rel, RelFileLocator newrlocator)
{
if (smgrexists(RelationGetSmgr(rel), forkNum))
{
- smgrcreate(dstrel, forkNum, false);
+ smgrcreate(rel->rd_locator, dstrel, forkNum, false);
/*
* WAL log creation if the relation is persistent, or this is the
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 7915ed624c1..ecacb5fb50a 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -943,7 +943,7 @@ ExtendBufferedRelTo(BufferManagerRelation bmr,
/* recheck, fork might have been created concurrently */
if (!smgrexists(bmr.smgr, fork))
- smgrcreate(bmr.smgr, fork, flags & EB_PERFORMING_RECOVERY);
+ smgrcreate(bmr.rel->rd_locator, bmr.smgr, fork, flags & EB_PERFORMING_RECOVERY);
UnlockRelationForExtension(bmr.rel, ExclusiveLock);
}
@@ -4754,7 +4754,7 @@ CreateAndCopyRelationData(RelFileLocator src_rlocator,
* directory. Therefore, each individual relation doesn't need to be
* registered for cleanup.
*/
- RelationCreateStorage(dst_rlocator, relpersistence, false);
+ RelationCreateStorage(src_rlocator, dst_rlocator, relpersistence, false);
/* copy main fork. */
RelationCopyStorageUsingBuffer(src_rlocator, dst_rlocator, MAIN_FORKNUM,
@@ -4766,7 +4766,8 @@ CreateAndCopyRelationData(RelFileLocator src_rlocator,
{
if (smgrexists(src_rel, forkNum))
{
- smgrcreate(dst_rel, forkNum, false);
+ /* TODO: for sure? */
+ smgrcreate(src_rel->smgr_rlocator.locator, dst_rel, forkNum, false);
/*
* WAL log creation if the relation is persistent, or this is the
diff --git a/src/backend/storage/smgr/md.c b/src/backend/storage/smgr/md.c
index 5a2072e0816..1766bbe1e57 100644
--- a/src/backend/storage/smgr/md.c
+++ b/src/backend/storage/smgr/md.c
@@ -245,7 +245,7 @@ mdexists(SMgrRelation reln, ForkNumber forknum)
* If isRedo is true, it's okay for the relation to exist already.
*/
void
-mdcreate(SMgrRelation reln, ForkNumber forknum, bool isRedo)
+mdcreate(RelFileLocator /* reln */, SMgrRelation reln, ForkNumber forknum, bool isRedo)
{
MdSMgrRelation mdreln = (MdSMgrRelation) reln;
MdfdVec *mdfd;
diff --git a/src/backend/storage/smgr/smgr.c b/src/backend/storage/smgr/smgr.c
index 9b3e63aff55..0498fd6c317 100644
--- a/src/backend/storage/smgr/smgr.c
+++ b/src/backend/storage/smgr/smgr.c
@@ -408,9 +408,9 @@ smgrexists(SMgrRelation reln, ForkNumber forknum)
* to be created.
*/
void
-smgrcreate(SMgrRelation reln, ForkNumber forknum, bool isRedo)
+smgrcreate(RelFileLocator relold, SMgrRelation reln, ForkNumber forknum, bool isRedo)
{
- smgrsw[reln->smgr_which].smgr_create(reln, forknum, isRedo);
+ smgrsw[reln->smgr_which].smgr_create(relold, reln, forknum, isRedo);
}
/*
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index d1ae761b3f6..db3e241404d 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -3829,7 +3829,7 @@ RelationSetNewRelfilenumber(Relation relation, char persistence)
/* handle these directly, at least for now */
SMgrRelation srel;
- srel = RelationCreateStorage(newrlocator, persistence, true);
+ srel = RelationCreateStorage(relation->rd_locator, newrlocator, persistence, true);
smgrclose(srel);
}
else
diff --git a/src/include/catalog/storage.h b/src/include/catalog/storage.h
index ba99225b0a3..ecc3b792f4f 100644
--- a/src/include/catalog/storage.h
+++ b/src/include/catalog/storage.h
@@ -22,7 +22,8 @@
/* GUC variables */
extern PGDLLIMPORT int wal_skip_threshold;
-extern SMgrRelation RelationCreateStorage(RelFileLocator rlocator,
+extern SMgrRelation RelationCreateStorage(RelFileLocator oldlocator,
+ RelFileLocator rlocator,
char relpersistence,
bool register_delete);
extern void RelationDropStorage(Relation rel);
diff --git a/src/include/storage/md.h b/src/include/storage/md.h
index da1d1d339be..61c0e85dd74 100644
--- a/src/include/storage/md.h
+++ b/src/include/storage/md.h
@@ -27,7 +27,7 @@ extern SMgrId MdSMgrId;
extern void mdinit(void);
extern void mdopen(SMgrRelation reln);
extern void mdclose(SMgrRelation reln, ForkNumber forknum);
-extern void mdcreate(SMgrRelation reln, ForkNumber forknum, bool isRedo);
+extern void mdcreate(RelFileLocator relold, SMgrRelation reln, ForkNumber forknum, bool isRedo);
extern bool mdexists(SMgrRelation reln, ForkNumber forknum);
extern void mdunlink(RelFileLocatorBackend rlocator, ForkNumber forknum, bool isRedo);
extern void mdextend(SMgrRelation reln, ForkNumber forknum,
diff --git a/src/include/storage/smgr.h b/src/include/storage/smgr.h
index 629c78cfdde..5b2b6de91c4 100644
--- a/src/include/storage/smgr.h
+++ b/src/include/storage/smgr.h
@@ -87,7 +87,7 @@ typedef struct f_smgr
void (*smgr_shutdown) (void); /* may be NULL */
void (*smgr_open) (SMgrRelation reln);
void (*smgr_close) (SMgrRelation reln, ForkNumber forknum);
- void (*smgr_create) (SMgrRelation reln, ForkNumber forknum,
+ void (*smgr_create) (RelFileLocator relold, SMgrRelation reln, ForkNumber forknum,
bool isRedo);
bool (*smgr_exists) (SMgrRelation reln, ForkNumber forknum);
void (*smgr_unlink) (RelFileLocatorBackend rlocator, ForkNumber forknum,
@@ -128,7 +128,7 @@ extern void smgrdestroyall(void);
extern void smgrrelease(SMgrRelation reln);
extern void smgrreleaseall(void);
extern void smgrreleaserellocator(RelFileLocatorBackend rlocator);
-extern void smgrcreate(SMgrRelation reln, ForkNumber forknum, bool isRedo);
+extern void smgrcreate(RelFileLocator relold, SMgrRelation reln, ForkNumber forknum, bool isRedo);
extern void smgrdosyncall(SMgrRelation *rels, int nrels);
extern void smgrdounlinkall(SMgrRelation *rels, int nrels, bool isRedo);
extern void smgrextend(SMgrRelation reln, ForkNumber forknum,
--
2.47.2
[text/x-patch] v4-0004-Add-contrib-fsync_checker.patch (9.9K, ../../[email protected]/4-v4-0004-Add-contrib-fsync_checker.patch)
download | inline diff:
From 608fc0377f7823d4f21ed8af60586f0d7c301cd0 Mon Sep 17 00:00:00 2001
From: Tristan Partin <[email protected]>
Date: Wed, 20 Sep 2023 14:23:38 -0500
Subject: [PATCH v4 4/6] Add contrib/fsync_checker
fsync_checker is an extension which overrides the global storage manager
to check for volatile relations, those which have been written but not
synced to disk.
---
contrib/Makefile | 1 +
contrib/fsync_checker/fsync_checker.control | 5 +
contrib/fsync_checker/fsync_checker_smgr.c | 249 ++++++++++++++++++++
contrib/fsync_checker/meson.build | 22 ++
contrib/meson.build | 1 +
src/tools/pgindent/typedefs.list | 2 +
6 files changed, 280 insertions(+)
create mode 100644 contrib/fsync_checker/fsync_checker.control
create mode 100644 contrib/fsync_checker/fsync_checker_smgr.c
create mode 100644 contrib/fsync_checker/meson.build
diff --git a/contrib/Makefile b/contrib/Makefile
index 952855d9b61..1c9f22b1c86 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -19,6 +19,7 @@ SUBDIRS = \
dict_int \
dict_xsyn \
earthdistance \
+ fsync_checker \
file_fdw \
fuzzystrmatch \
hstore \
diff --git a/contrib/fsync_checker/fsync_checker.control b/contrib/fsync_checker/fsync_checker.control
new file mode 100644
index 00000000000..7d0e36434bf
--- /dev/null
+++ b/contrib/fsync_checker/fsync_checker.control
@@ -0,0 +1,5 @@
+# fsync_checker extension
+comment = 'SMGR extension for checking volatile writes'
+default_version = '1.0'
+module_pathname = '$libdir/fsync_checker'
+relocatable = true
diff --git a/contrib/fsync_checker/fsync_checker_smgr.c b/contrib/fsync_checker/fsync_checker_smgr.c
new file mode 100644
index 00000000000..97ad0f78da8
--- /dev/null
+++ b/contrib/fsync_checker/fsync_checker_smgr.c
@@ -0,0 +1,249 @@
+#include "postgres.h"
+
+#include "access/xlog.h"
+#include "fmgr.h"
+#include "miscadmin.h"
+#include "storage/ipc.h"
+#include "storage/lwlock.h"
+#include "storage/shmem.h"
+#include "storage/smgr.h"
+#include "storage/md.h"
+#include "utils/hsearch.h"
+
+PG_MODULE_MAGIC;
+
+typedef struct
+{
+ RelFileLocator locator;
+ ForkNumber forknum;
+} VolatileRelnKey;
+
+typedef struct
+{
+ VolatileRelnKey key;
+ XLogRecPtr lsn;
+} VolatileRelnEntry;
+
+void _PG_init(void);
+
+static void fsync_checker_extend(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
+ const void *buffer, bool skipFsync);
+static void fsync_checker_immedsync(SMgrRelation reln, ForkNumber forknum);
+static void fsync_checker_writev(SMgrRelation reln, ForkNumber forknum,
+ BlockNumber blocknum, const void **buffers,
+ BlockNumber nblocks, bool skipFsync);
+static void fsync_checker_writeback(SMgrRelation reln, ForkNumber forknum,
+ BlockNumber blocknum, BlockNumber nblocks);
+static void fsync_checker_zeroextend(SMgrRelation reln, ForkNumber forknum,
+ BlockNumber blocknum, int nblocks, bool skipFsync);
+
+static void fsync_checker_checkpoint_create(const CheckPoint *checkPoint);
+static void fsync_checker_shmem_request(void);
+static void fsync_checker_shmem_startup(void);
+
+static void add_reln(SMgrRelation reln, ForkNumber forknum);
+static void remove_reln(SMgrRelation reln, ForkNumber forknum);
+
+static SMgrId fsync_checker_smgr_id;
+static const struct f_smgr fsync_checker_smgr = {
+ .name = "fsync_checker",
+ .smgr_init = mdinit,
+ .smgr_shutdown = NULL,
+ .smgr_open = mdopen,
+ .smgr_close = mdclose,
+ .smgr_create = mdcreate,
+ .smgr_exists = mdexists,
+ .smgr_unlink = mdunlink,
+ .smgr_extend = fsync_checker_extend,
+ .smgr_zeroextend = fsync_checker_zeroextend,
+ .smgr_prefetch = mdprefetch,
+ .smgr_maxcombine = mdmaxcombine,
+ .smgr_readv = mdreadv,
+ .smgr_writev = fsync_checker_writev,
+ .smgr_writeback = fsync_checker_writeback,
+ .smgr_nblocks = mdnblocks,
+ .smgr_truncate = mdtruncate,
+ .smgr_immedsync = fsync_checker_immedsync,
+ .smgr_registersync = mdregistersync,
+};
+
+static HTAB *volatile_relns;
+static LWLock *volatile_relns_lock;
+static shmem_request_hook_type prev_shmem_request_hook;
+static shmem_startup_hook_type prev_shmem_startup_hook;
+static checkpoint_create_hook_type prev_checkpoint_create_hook;
+
+void
+_PG_init(void)
+{
+ prev_checkpoint_create_hook = checkpoint_create_hook;
+ checkpoint_create_hook = fsync_checker_checkpoint_create;
+
+ prev_shmem_request_hook = shmem_request_hook;
+ shmem_request_hook = fsync_checker_shmem_request;
+
+ prev_shmem_startup_hook = shmem_startup_hook;
+ shmem_startup_hook = fsync_checker_shmem_startup;
+
+ /*
+ * Relation size of 0 means we can just defer to md, but it would be nice
+ * to just expose this functionality, so if I needed my own relation, I
+ * could use MdSmgrRelation as the parent.
+ */
+ fsync_checker_smgr_id = smgr_register(&fsync_checker_smgr, 0);
+
+ storage_manager_id = fsync_checker_smgr_id;
+}
+
+static void
+fsync_checker_checkpoint_create(const CheckPoint *checkPoint)
+{
+ long num_entries;
+ HASH_SEQ_STATUS status;
+ VolatileRelnEntry *entry;
+
+ if (prev_checkpoint_create_hook)
+ prev_checkpoint_create_hook(checkPoint);
+
+ LWLockAcquire(volatile_relns_lock, LW_EXCLUSIVE);
+
+ hash_seq_init(&status, volatile_relns);
+
+ num_entries = hash_get_num_entries(volatile_relns);
+ elog(INFO, "Analyzing %ld volatile relations", num_entries);
+ while ((entry = hash_seq_search(&status)))
+ {
+ if (entry->lsn < checkPoint->redo)
+ {
+ RelPathStr path;
+
+ path = relpathperm(entry->key.locator, entry->key.forknum);
+
+ elog(WARNING, "Relation not previously synced: %s", path.str);
+ }
+ }
+
+ LWLockRelease(volatile_relns_lock);
+}
+
+static void
+fsync_checker_shmem_request(void)
+{
+ if (prev_shmem_request_hook)
+ prev_shmem_request_hook();
+
+ RequestAddinShmemSpace(hash_estimate_size(1024, sizeof(VolatileRelnEntry)));
+ RequestNamedLWLockTranche("fsync_checker volatile relns lock", 1);
+}
+
+static void
+fsync_checker_shmem_startup(void)
+{
+ HASHCTL ctl;
+
+ if (prev_shmem_startup_hook)
+ prev_shmem_startup_hook();
+
+ ctl.keysize = sizeof(VolatileRelnKey);
+ ctl.entrysize = sizeof(VolatileRelnEntry);
+ volatile_relns = NULL;
+ volatile_relns_lock = NULL;
+
+ /*
+ * Create or attach to the shared memory state, including hash table
+ */
+ LWLockAcquire(AddinShmemInitLock, LW_EXCLUSIVE);
+
+ volatile_relns = ShmemInitHash("fsync_checker volatile relns",
+ 1024, 1024, &ctl, HASH_BLOBS | HASH_ELEM);
+ volatile_relns_lock = &GetNamedLWLockTranche("fsync_checker volatile relns lock")->lock;
+
+ LWLockRelease(AddinShmemInitLock);
+}
+
+static void
+add_reln(SMgrRelation reln, ForkNumber forknum)
+{
+ bool found;
+ XLogRecPtr lsn;
+ VolatileRelnKey key;
+ VolatileRelnEntry *entry;
+
+ key.locator = reln->smgr_rlocator.locator;
+ key.forknum = forknum;
+
+ lsn = GetXLogWriteRecPtr();
+
+ LWLockAcquire(volatile_relns_lock, LW_EXCLUSIVE);
+
+ entry = hash_search(volatile_relns, &key, HASH_ENTER, &found);
+ if (!found)
+ entry->lsn = lsn;
+
+ LWLockRelease(volatile_relns_lock);
+}
+
+static void
+remove_reln(SMgrRelation reln, ForkNumber forknum)
+{
+ VolatileRelnKey key;
+
+ key.locator = reln->smgr_rlocator.locator;
+ key.forknum = forknum;
+
+ LWLockAcquire(volatile_relns_lock, LW_EXCLUSIVE);
+
+ hash_search(volatile_relns, &key, HASH_REMOVE, NULL);
+
+ LWLockRelease(volatile_relns_lock);
+}
+
+static void
+fsync_checker_extend(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
+ const void *buffer, bool skipFsync)
+{
+ if (!SmgrIsTemp(reln) && !skipFsync)
+ add_reln(reln, forknum);
+
+ mdextend(reln, forknum, blocknum, buffer, skipFsync);
+}
+
+static void
+fsync_checker_immedsync(SMgrRelation reln, ForkNumber forknum)
+{
+ if (!SmgrIsTemp(reln))
+ remove_reln(reln, forknum);
+
+ mdimmedsync(reln, forknum);
+}
+
+static void
+fsync_checker_writev(SMgrRelation reln, ForkNumber forknum,
+ BlockNumber blocknum, const void **buffers,
+ BlockNumber nblocks, bool skipFsync)
+{
+ if (!SmgrIsTemp(reln) && !skipFsync)
+ add_reln(reln, forknum);
+
+ mdwritev(reln, forknum, blocknum, buffers, nblocks, skipFsync);
+}
+
+static void
+fsync_checker_writeback(SMgrRelation reln, ForkNumber forknum,
+ BlockNumber blocknum, BlockNumber nblocks)
+{
+ if (!SmgrIsTemp(reln))
+ remove_reln(reln, forknum);
+
+ mdwriteback(reln, forknum, blocknum, nblocks);
+}
+
+static void
+fsync_checker_zeroextend(SMgrRelation reln, ForkNumber forknum,
+ BlockNumber blocknum, int nblocks, bool skipFsync)
+{
+ if (!SmgrIsTemp(reln) && !skipFsync)
+ add_reln(reln, forknum);
+
+ mdzeroextend(reln, forknum, blocknum, nblocks, skipFsync);
+}
diff --git a/contrib/fsync_checker/meson.build b/contrib/fsync_checker/meson.build
new file mode 100644
index 00000000000..ce6ed7fe90b
--- /dev/null
+++ b/contrib/fsync_checker/meson.build
@@ -0,0 +1,22 @@
+# Copyright (c) 2023, PostgreSQL Global Development Group
+
+fsync_checker_sources = files(
+ 'fsync_checker_smgr.c',
+)
+
+if host_system == 'windows'
+ fsync_checker_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+ '--NAME', 'fsync_checker',
+ '--FILEDESC', 'fsync_checker - SMGR extension for checking volatile relations',])
+endif
+
+fsync_checker = shared_module('fsync_checker',
+ fsync_checker_sources,
+ kwargs: contrib_mod_args,
+)
+contrib_targets += fsync_checker
+
+install_data(
+ 'fsync_checker.control',
+ kwargs: contrib_data_args,
+)
diff --git a/contrib/meson.build b/contrib/meson.build
index 1ba73ebd67a..c48fb138751 100644
--- a/contrib/meson.build
+++ b/contrib/meson.build
@@ -28,6 +28,7 @@ subdir('dict_int')
subdir('dict_xsyn')
subdir('earthdistance')
subdir('file_fdw')
+subdir('fsync_checker')
subdir('fuzzystrmatch')
subdir('hstore')
subdir('hstore_plperl')
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 4b971b81ae5..bc260e713ae 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -3145,6 +3145,8 @@ ViewStmt
VirtualTransactionId
VirtualTupleTableSlot
VolatileFunctionStatus
+VolatileRelnEntry
+VolatileRelnKey
Vsrt
WAIT_ORDER
WALAvailability
--
2.47.2
[text/x-patch] v4-0003-Add-checkpoint_create_hook.patch (1.9K, ../../[email protected]/5-v4-0003-Add-checkpoint_create_hook.patch)
download | inline diff:
From 01d76e69d1e0b2018f72160ef8873c4997f42030 Mon Sep 17 00:00:00 2001
From: Tristan Partin <[email protected]>
Date: Fri, 13 Oct 2023 13:57:18 -0500
Subject: [PATCH v4 3/6] Add checkpoint_create_hook
Allows an extension to hook into CheckPointCreate().
---
src/backend/access/transam/xlog.c | 5 +++++
src/include/access/xlog.h | 4 ++++
2 files changed, 9 insertions(+)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 799fc739e18..cdd6eff2fd5 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -208,6 +208,8 @@ const struct config_enum_entry archive_mode_options[] = {
*/
CheckpointStatsData CheckpointStats;
+checkpoint_create_hook_type checkpoint_create_hook = NULL;
+
/*
* During recovery, lastFullPageWrites keeps track of full_page_writes that
* the replayed WAL records indicate. It's initialized with full_page_writes
@@ -7173,6 +7175,9 @@ CreateCheckPoint(int flags)
*/
END_CRIT_SECTION();
+ if (checkpoint_create_hook != NULL)
+ checkpoint_create_hook(&checkPoint);
+
/*
* In some cases there are groups of actions that must all occur on one
* side or the other of a checkpoint record. Before flushing the
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index d313099c027..8aab37ef52d 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -13,6 +13,7 @@
#include "access/xlogbackup.h"
#include "access/xlogdefs.h"
+#include "catalog/pg_control.h"
#include "datatype/timestamp.h"
#include "lib/stringinfo.h"
#include "nodes/pg_list.h"
@@ -59,6 +60,9 @@ extern PGDLLIMPORT int wal_decode_buffer_size;
extern PGDLLIMPORT int CheckPointSegments;
+typedef void (*checkpoint_create_hook_type) (const CheckPoint *);
+extern PGDLLIMPORT checkpoint_create_hook_type checkpoint_create_hook;
+
/* Archive modes */
typedef enum ArchiveMode
{
--
2.47.2
[text/x-patch] v4-0002-Allow-extensions-to-override-the-global-storage-m.patch (2.0K, ../../[email protected]/6-v4-0002-Allow-extensions-to-override-the-global-storage-m.patch)
download | inline diff:
From f1a1dc0f8146c724589ae2578aa470faf68fdb9d Mon Sep 17 00:00:00 2001
From: Tristan Partin <[email protected]>
Date: Fri, 13 Oct 2023 14:00:44 -0500
Subject: [PATCH v4 2/6] Allow extensions to override the global storage
manager
---
src/backend/storage/smgr/smgr.c | 4 +++-
src/backend/utils/init/miscinit.c | 2 ++
src/include/storage/smgr.h | 2 ++
3 files changed, 7 insertions(+), 1 deletion(-)
diff --git a/src/backend/storage/smgr/smgr.c b/src/backend/storage/smgr/smgr.c
index 7635c231ea0..9b3e63aff55 100644
--- a/src/backend/storage/smgr/smgr.c
+++ b/src/backend/storage/smgr/smgr.c
@@ -69,6 +69,8 @@ static int NSmgr = 0;
static Size LargestSMgrRelationSize = 0;
+SMgrId storage_manager_id;
+
/*
* Each backend has a hashtable that stores all extant SMgrRelation objects.
* In addition, "unpinned" SMgrRelation objects are chained together in a list.
@@ -227,7 +229,7 @@ smgropen(RelFileLocator rlocator, ProcNumber backend)
for (int i = 0; i <= MAX_FORKNUM; ++i)
reln->smgr_cached_nblocks[i] = InvalidBlockNumber;
- reln->smgr_which = MdSMgrId; /* we only have md.c at present */
+ reln->smgr_which = storage_manager_id;
/* implementation-specific initialization */
smgrsw[reln->smgr_which].smgr_open(reln);
diff --git a/src/backend/utils/init/miscinit.c b/src/backend/utils/init/miscinit.c
index 3176cdce6d7..1b3ce51cfce 100644
--- a/src/backend/utils/init/miscinit.c
+++ b/src/backend/utils/init/miscinit.c
@@ -1931,6 +1931,8 @@ void
register_builtin_dynamic_managers(void)
{
mdsmgr_register();
+
+ storage_manager_id = MdSMgrId;
}
/*
diff --git a/src/include/storage/smgr.h b/src/include/storage/smgr.h
index 52f74f917b2..629c78cfdde 100644
--- a/src/include/storage/smgr.h
+++ b/src/include/storage/smgr.h
@@ -20,6 +20,8 @@
typedef uint8 SMgrId;
+extern PGDLLIMPORT SMgrId storage_manager_id;
+
/*
* smgr.c maintains a table of SMgrRelation objects, which are essentially
* cached file handles. An SMgrRelation is created (if not already present)
--
2.47.2
[text/x-patch] v4-0001-Expose-f_smgr-to-extensions-for-manual-implementa.patch (33.7K, ../../[email protected]/7-v4-0001-Expose-f_smgr-to-extensions-for-manual-implementa.patch)
download | inline diff:
From 772790b5a5b4ab215d1243722f1b31303dc976f5 Mon Sep 17 00:00:00 2001
From: Matthias van de Meent <[email protected]>
Date: Tue, 27 Jun 2023 15:59:23 +0200
Subject: [PATCH v4 1/6] Expose f_smgr to extensions for manual implementation
There are various reasons why one would want to create their own
implementation of a storage manager, among which are block-level compression,
encryption and offloading to cold storage. This patch is a first patch that
allows extensions to register their own SMgr.
Note, however, that this SMgr is not yet used - only the first SMgr to register
is used, and this is currently the md.c smgr. Future commits will include
facilities to select an SMgr for each tablespace.
---
src/backend/postmaster/postmaster.c | 5 +
src/backend/storage/smgr/md.c | 187 +++++++++++++++++++---------
src/backend/storage/smgr/smgr.c | 137 ++++++++++----------
src/backend/utils/init/miscinit.c | 13 ++
src/include/miscadmin.h | 1 +
src/include/storage/md.h | 4 +
src/include/storage/smgr.h | 59 +++++++--
src/tools/pgindent/typedefs.list | 1 +
8 files changed, 266 insertions(+), 141 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index d2a7a7add6f..88ea821573d 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -916,6 +916,11 @@ PostmasterMain(int argc, char *argv[])
*/
ApplyLauncherRegister();
+ /*
+ * Register built-in managers that are not part of static arrays
+ */
+ register_builtin_dynamic_managers();
+
/*
* process any libraries that should be preloaded at postmaster start
*/
diff --git a/src/backend/storage/smgr/md.c b/src/backend/storage/smgr/md.c
index f3220f98dc4..5a2072e0816 100644
--- a/src/backend/storage/smgr/md.c
+++ b/src/backend/storage/smgr/md.c
@@ -84,6 +84,21 @@ typedef struct _MdfdVec
} MdfdVec;
static MemoryContext MdCxt; /* context for all MdfdVec objects */
+SMgrId MdSMgrId;
+
+typedef struct
+{
+ SMgrRelationData reln; /* parent data */
+
+ /*
+ * for md.c; per-fork arrays of the number of open segments
+ * (md_num_open_segs) and the segments themselves (md_seg_fds).
+ */
+ int md_num_open_segs[MAX_FORKNUM + 1];
+ MdfdVec *md_seg_fds[MAX_FORKNUM + 1];
+} MdSMgrRelationData;
+
+typedef MdSMgrRelationData *MdSMgrRelation;
/* Populate a file tag describing an md.c segment file. */
@@ -130,26 +145,55 @@ typedef struct MdPathStr
} MdPathStr;
+void
+mdsmgr_register(void)
+{
+ /* magnetic disk */
+ f_smgr md_smgr = (f_smgr) {
+ .name = "md",
+ .smgr_init = mdinit,
+ .smgr_shutdown = NULL,
+ .smgr_open = mdopen,
+ .smgr_close = mdclose,
+ .smgr_create = mdcreate,
+ .smgr_exists = mdexists,
+ .smgr_unlink = mdunlink,
+ .smgr_extend = mdextend,
+ .smgr_zeroextend = mdzeroextend,
+ .smgr_prefetch = mdprefetch,
+ .smgr_maxcombine = mdmaxcombine,
+ .smgr_readv = mdreadv,
+ .smgr_writev = mdwritev,
+ .smgr_writeback = mdwriteback,
+ .smgr_nblocks = mdnblocks,
+ .smgr_truncate = mdtruncate,
+ .smgr_immedsync = mdimmedsync,
+ .smgr_registersync = mdregistersync,
+ };
+
+ MdSMgrId = smgr_register(&md_smgr, sizeof(MdSMgrRelationData));
+}
+
/* local routines */
static void mdunlinkfork(RelFileLocatorBackend rlocator, ForkNumber forknum,
bool isRedo);
-static MdfdVec *mdopenfork(SMgrRelation reln, ForkNumber forknum, int behavior);
-static void register_dirty_segment(SMgrRelation reln, ForkNumber forknum,
+static MdfdVec *mdopenfork(MdSMgrRelation reln, ForkNumber forknum, int behavior);
+static void register_dirty_segment(MdSMgrRelation reln, ForkNumber forknum,
MdfdVec *seg);
static void register_unlink_segment(RelFileLocatorBackend rlocator, ForkNumber forknum,
BlockNumber segno);
static void register_forget_request(RelFileLocatorBackend rlocator, ForkNumber forknum,
BlockNumber segno);
-static void _fdvec_resize(SMgrRelation reln,
+static void _fdvec_resize(MdSMgrRelation reln,
ForkNumber forknum,
int nseg);
-static MdPathStr _mdfd_segpath(SMgrRelation reln, ForkNumber forknum,
+static MdPathStr _mdfd_segpath(MdSMgrRelation reln, ForkNumber forknum,
BlockNumber segno);
-static MdfdVec *_mdfd_openseg(SMgrRelation reln, ForkNumber forknum,
+static MdfdVec *_mdfd_openseg(MdSMgrRelation reln, ForkNumber forknum,
BlockNumber segno, int oflags);
-static MdfdVec *_mdfd_getseg(SMgrRelation reln, ForkNumber forknum,
+static MdfdVec *_mdfd_getseg(MdSMgrRelation reln, ForkNumber forknum,
BlockNumber blkno, bool skipFsync, int behavior);
-static BlockNumber _mdnblocks(SMgrRelation reln, ForkNumber forknum,
+static BlockNumber _mdnblocks(MdSMgrRelation reln, ForkNumber forknum,
MdfdVec *seg);
static inline int
@@ -182,6 +226,8 @@ mdinit(void)
bool
mdexists(SMgrRelation reln, ForkNumber forknum)
{
+ MdSMgrRelation mdreln = (MdSMgrRelation) reln;
+
/*
* Close it first, to ensure that we notice if the fork has been unlinked
* since we opened it. As an optimization, we can skip that in recovery,
@@ -190,7 +236,7 @@ mdexists(SMgrRelation reln, ForkNumber forknum)
if (!InRecovery)
mdclose(reln, forknum);
- return (mdopenfork(reln, forknum, EXTENSION_RETURN_NULL) != NULL);
+ return (mdopenfork(mdreln, forknum, EXTENSION_RETURN_NULL) != NULL);
}
/*
@@ -201,14 +247,15 @@ mdexists(SMgrRelation reln, ForkNumber forknum)
void
mdcreate(SMgrRelation reln, ForkNumber forknum, bool isRedo)
{
+ MdSMgrRelation mdreln = (MdSMgrRelation) reln;
MdfdVec *mdfd;
RelPathStr path;
File fd;
- if (isRedo && reln->md_num_open_segs[forknum] > 0)
+ if (isRedo && mdreln->md_num_open_segs[forknum] > 0)
return; /* created and opened already... */
- Assert(reln->md_num_open_segs[forknum] == 0);
+ Assert(mdreln->md_num_open_segs[forknum] == 0);
/*
* We may be using the target table space for the first time in this
@@ -243,13 +290,13 @@ mdcreate(SMgrRelation reln, ForkNumber forknum, bool isRedo)
}
}
- _fdvec_resize(reln, forknum, 1);
- mdfd = &reln->md_seg_fds[forknum][0];
+ _fdvec_resize(mdreln, forknum, 1);
+ mdfd = &mdreln->md_seg_fds[forknum][0];
mdfd->mdfd_vfd = fd;
mdfd->mdfd_segno = 0;
if (!SmgrIsTemp(reln))
- register_dirty_segment(reln, forknum, mdfd);
+ register_dirty_segment(mdreln, forknum, mdfd);
}
/*
@@ -467,6 +514,7 @@ void
mdextend(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
const void *buffer, bool skipFsync)
{
+ MdSMgrRelation mdreln = (MdSMgrRelation) reln;
off_t seekpos;
int nbytes;
MdfdVec *v;
@@ -493,7 +541,7 @@ mdextend(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
relpath(reln->smgr_rlocator, forknum).str,
InvalidBlockNumber)));
- v = _mdfd_getseg(reln, forknum, blocknum, skipFsync, EXTENSION_CREATE);
+ v = _mdfd_getseg(mdreln, forknum, blocknum, skipFsync, EXTENSION_CREATE);
seekpos = (off_t) BLCKSZ * (blocknum % ((BlockNumber) RELSEG_SIZE));
@@ -517,9 +565,9 @@ mdextend(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
}
if (!skipFsync && !SmgrIsTemp(reln))
- register_dirty_segment(reln, forknum, v);
+ register_dirty_segment(mdreln, forknum, v);
- Assert(_mdnblocks(reln, forknum, v) <= ((BlockNumber) RELSEG_SIZE));
+ Assert(_mdnblocks(mdreln, forknum, v) <= ((BlockNumber) RELSEG_SIZE));
}
/*
@@ -532,6 +580,7 @@ void
mdzeroextend(SMgrRelation reln, ForkNumber forknum,
BlockNumber blocknum, int nblocks, bool skipFsync)
{
+ MdSMgrRelation mdreln = (MdSMgrRelation) reln;
MdfdVec *v;
BlockNumber curblocknum = blocknum;
int remblocks = nblocks;
@@ -566,7 +615,7 @@ mdzeroextend(SMgrRelation reln, ForkNumber forknum,
else
numblocks = remblocks;
- v = _mdfd_getseg(reln, forknum, curblocknum, skipFsync, EXTENSION_CREATE);
+ v = _mdfd_getseg(mdreln, forknum, curblocknum, skipFsync, EXTENSION_CREATE);
Assert(segstartblock < RELSEG_SIZE);
Assert(segstartblock + numblocks <= RELSEG_SIZE);
@@ -621,9 +670,9 @@ mdzeroextend(SMgrRelation reln, ForkNumber forknum,
}
if (!skipFsync && !SmgrIsTemp(reln))
- register_dirty_segment(reln, forknum, v);
+ register_dirty_segment(mdreln, forknum, v);
- Assert(_mdnblocks(reln, forknum, v) <= ((BlockNumber) RELSEG_SIZE));
+ Assert(_mdnblocks(mdreln, forknum, v) <= ((BlockNumber) RELSEG_SIZE));
remblocks -= numblocks;
curblocknum += numblocks;
@@ -641,7 +690,7 @@ mdzeroextend(SMgrRelation reln, ForkNumber forknum,
* invent one out of whole cloth.
*/
static MdfdVec *
-mdopenfork(SMgrRelation reln, ForkNumber forknum, int behavior)
+mdopenfork(MdSMgrRelation reln, ForkNumber forknum, int behavior)
{
MdfdVec *mdfd;
RelPathStr path;
@@ -651,7 +700,7 @@ mdopenfork(SMgrRelation reln, ForkNumber forknum, int behavior)
if (reln->md_num_open_segs[forknum] > 0)
return &reln->md_seg_fds[forknum][0];
- path = relpath(reln->smgr_rlocator, forknum);
+ path = relpath(reln->reln.smgr_rlocator, forknum);
fd = PathNameOpenFile(path.str, _mdfd_open_flags());
@@ -681,9 +730,11 @@ mdopenfork(SMgrRelation reln, ForkNumber forknum, int behavior)
void
mdopen(SMgrRelation reln)
{
+ MdSMgrRelation mdreln = (MdSMgrRelation) reln;
+
/* mark it not open */
for (int forknum = 0; forknum <= MAX_FORKNUM; forknum++)
- reln->md_num_open_segs[forknum] = 0;
+ mdreln->md_num_open_segs[forknum] = 0;
}
/*
@@ -692,7 +743,8 @@ mdopen(SMgrRelation reln)
void
mdclose(SMgrRelation reln, ForkNumber forknum)
{
- int nopensegs = reln->md_num_open_segs[forknum];
+ MdSMgrRelation mdreln = (MdSMgrRelation) reln;
+ int nopensegs = mdreln->md_num_open_segs[forknum];
/* No work if already closed */
if (nopensegs == 0)
@@ -701,10 +753,10 @@ mdclose(SMgrRelation reln, ForkNumber forknum)
/* close segments starting from the end */
while (nopensegs > 0)
{
- MdfdVec *v = &reln->md_seg_fds[forknum][nopensegs - 1];
+ MdfdVec *v = &mdreln->md_seg_fds[forknum][nopensegs - 1];
FileClose(v->mdfd_vfd);
- _fdvec_resize(reln, forknum, nopensegs - 1);
+ _fdvec_resize(mdreln, forknum, nopensegs - 1);
nopensegs--;
}
}
@@ -717,6 +769,7 @@ mdprefetch(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
int nblocks)
{
#ifdef USE_PREFETCH
+ MdSMgrRelation mdreln = (MdSMgrRelation) reln;
Assert((io_direct_flags & IO_DIRECT_DATA) == 0);
@@ -729,7 +782,7 @@ mdprefetch(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
MdfdVec *v;
int nblocks_this_segment;
- v = _mdfd_getseg(reln, forknum, blocknum, false,
+ v = _mdfd_getseg(mdreln, forknum, blocknum, false,
InRecovery ? EXTENSION_RETURN_NULL : EXTENSION_FAIL);
if (v == NULL)
return false;
@@ -827,6 +880,8 @@ void
mdreadv(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
void **buffers, BlockNumber nblocks)
{
+ MdSMgrRelation mdreln = (MdSMgrRelation) reln;
+
while (nblocks > 0)
{
struct iovec iov[PG_IOV_MAX];
@@ -838,7 +893,7 @@ mdreadv(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
size_t transferred_this_segment;
size_t size_this_segment;
- v = _mdfd_getseg(reln, forknum, blocknum, false,
+ v = _mdfd_getseg(mdreln, forknum, blocknum, false,
EXTENSION_FAIL | EXTENSION_CREATE_RECOVERY);
seekpos = (off_t) BLCKSZ * (blocknum % ((BlockNumber) RELSEG_SIZE));
@@ -948,6 +1003,8 @@ void
mdwritev(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
const void **buffers, BlockNumber nblocks, bool skipFsync)
{
+ MdSMgrRelation mdreln = (MdSMgrRelation) reln;
+
/* This assert is too expensive to have on normally ... */
#ifdef CHECK_WRITE_VS_EXTEND
Assert((uint64) blocknum + (uint64) nblocks <= (uint64) mdnblocks(reln, forknum));
@@ -964,7 +1021,7 @@ mdwritev(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
size_t transferred_this_segment;
size_t size_this_segment;
- v = _mdfd_getseg(reln, forknum, blocknum, skipFsync,
+ v = _mdfd_getseg(mdreln, forknum, blocknum, skipFsync,
EXTENSION_FAIL | EXTENSION_CREATE_RECOVERY);
seekpos = (off_t) BLCKSZ * (blocknum % ((BlockNumber) RELSEG_SIZE));
@@ -1034,7 +1091,7 @@ mdwritev(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
}
if (!skipFsync && !SmgrIsTemp(reln))
- register_dirty_segment(reln, forknum, v);
+ register_dirty_segment(mdreln, forknum, v);
nblocks -= nblocks_this_segment;
buffers += nblocks_this_segment;
@@ -1053,6 +1110,8 @@ void
mdwriteback(SMgrRelation reln, ForkNumber forknum,
BlockNumber blocknum, BlockNumber nblocks)
{
+ MdSMgrRelation mdreln = (MdSMgrRelation) reln;
+
Assert((io_direct_flags & IO_DIRECT_DATA) == 0);
/*
@@ -1067,7 +1126,7 @@ mdwriteback(SMgrRelation reln, ForkNumber forknum,
int segnum_start,
segnum_end;
- v = _mdfd_getseg(reln, forknum, blocknum, true /* not used */ ,
+ v = _mdfd_getseg(mdreln, forknum, blocknum, true /* not used */ ,
EXTENSION_DONT_OPEN);
/*
@@ -1111,14 +1170,15 @@ mdwriteback(SMgrRelation reln, ForkNumber forknum,
BlockNumber
mdnblocks(SMgrRelation reln, ForkNumber forknum)
{
+ MdSMgrRelation mdreln = (MdSMgrRelation) reln;
MdfdVec *v;
BlockNumber nblocks;
BlockNumber segno;
- mdopenfork(reln, forknum, EXTENSION_FAIL);
+ mdopenfork(mdreln, forknum, EXTENSION_FAIL);
/* mdopen has opened the first segment */
- Assert(reln->md_num_open_segs[forknum] > 0);
+ Assert(mdreln->md_num_open_segs[forknum] > 0);
/*
* Start from the last open segments, to avoid redundant seeks. We have
@@ -1133,12 +1193,12 @@ mdnblocks(SMgrRelation reln, ForkNumber forknum)
* that's OK because the checkpointer never needs to compute relation
* size.)
*/
- segno = reln->md_num_open_segs[forknum] - 1;
- v = &reln->md_seg_fds[forknum][segno];
+ segno = mdreln->md_num_open_segs[forknum] - 1;
+ v = &mdreln->md_seg_fds[forknum][segno];
for (;;)
{
- nblocks = _mdnblocks(reln, forknum, v);
+ nblocks = _mdnblocks(mdreln, forknum, v);
if (nblocks > ((BlockNumber) RELSEG_SIZE))
elog(FATAL, "segment too big");
if (nblocks < ((BlockNumber) RELSEG_SIZE))
@@ -1156,7 +1216,7 @@ mdnblocks(SMgrRelation reln, ForkNumber forknum)
* undermines _mdfd_getseg's attempts to notice and report an error
* upon access to a missing segment.
*/
- v = _mdfd_openseg(reln, forknum, segno, 0);
+ v = _mdfd_openseg(mdreln, forknum, segno, 0);
if (v == NULL)
return segno * ((BlockNumber) RELSEG_SIZE);
}
@@ -1176,6 +1236,7 @@ void
mdtruncate(SMgrRelation reln, ForkNumber forknum,
BlockNumber curnblk, BlockNumber nblocks)
{
+ MdSMgrRelation mdreln = (MdSMgrRelation) reln;
BlockNumber priorblocks;
int curopensegs;
@@ -1196,14 +1257,14 @@ mdtruncate(SMgrRelation reln, ForkNumber forknum,
* Truncate segments, starting at the last one. Starting at the end makes
* managing the memory for the fd array easier, should there be errors.
*/
- curopensegs = reln->md_num_open_segs[forknum];
+ curopensegs = mdreln->md_num_open_segs[forknum];
while (curopensegs > 0)
{
MdfdVec *v;
priorblocks = (curopensegs - 1) * RELSEG_SIZE;
- v = &reln->md_seg_fds[forknum][curopensegs - 1];
+ v = &mdreln->md_seg_fds[forknum][curopensegs - 1];
if (priorblocks > nblocks)
{
@@ -1218,13 +1279,13 @@ mdtruncate(SMgrRelation reln, ForkNumber forknum,
FilePathName(v->mdfd_vfd))));
if (!SmgrIsTemp(reln))
- register_dirty_segment(reln, forknum, v);
+ register_dirty_segment(mdreln, forknum, v);
/* we never drop the 1st segment */
- Assert(v != &reln->md_seg_fds[forknum][0]);
+ Assert(v != &mdreln->md_seg_fds[forknum][0]);
FileClose(v->mdfd_vfd);
- _fdvec_resize(reln, forknum, curopensegs - 1);
+ _fdvec_resize(mdreln, forknum, curopensegs - 1);
}
else if (priorblocks + ((BlockNumber) RELSEG_SIZE) > nblocks)
{
@@ -1244,7 +1305,7 @@ mdtruncate(SMgrRelation reln, ForkNumber forknum,
FilePathName(v->mdfd_vfd),
nblocks)));
if (!SmgrIsTemp(reln))
- register_dirty_segment(reln, forknum, v);
+ register_dirty_segment(mdreln, forknum, v);
}
else
{
@@ -1264,6 +1325,7 @@ mdtruncate(SMgrRelation reln, ForkNumber forknum,
void
mdregistersync(SMgrRelation reln, ForkNumber forknum)
{
+ MdSMgrRelation mdreln = (MdSMgrRelation) reln;
int segno;
int min_inactive_seg;
@@ -1273,7 +1335,7 @@ mdregistersync(SMgrRelation reln, ForkNumber forknum)
*/
mdnblocks(reln, forknum);
- min_inactive_seg = segno = reln->md_num_open_segs[forknum];
+ min_inactive_seg = segno = mdreln->md_num_open_segs[forknum];
/*
* Temporarily open inactive segments, then close them after sync. There
@@ -1281,20 +1343,20 @@ mdregistersync(SMgrRelation reln, ForkNumber forknum)
* harmless. We don't bother to clean them up and take a risk of further
* trouble. The next mdclose() will soon close them.
*/
- while (_mdfd_openseg(reln, forknum, segno, 0) != NULL)
+ while (_mdfd_openseg(mdreln, forknum, segno, 0) != NULL)
segno++;
while (segno > 0)
{
- MdfdVec *v = &reln->md_seg_fds[forknum][segno - 1];
+ MdfdVec *v = &mdreln->md_seg_fds[forknum][segno - 1];
- register_dirty_segment(reln, forknum, v);
+ register_dirty_segment(mdreln, forknum, v);
/* Close inactive segments immediately */
if (segno > min_inactive_seg)
{
FileClose(v->mdfd_vfd);
- _fdvec_resize(reln, forknum, segno - 1);
+ _fdvec_resize(mdreln, forknum, segno - 1);
}
segno--;
@@ -1315,6 +1377,7 @@ mdregistersync(SMgrRelation reln, ForkNumber forknum)
void
mdimmedsync(SMgrRelation reln, ForkNumber forknum)
{
+ MdSMgrRelation mdreln = (MdSMgrRelation) reln;
int segno;
int min_inactive_seg;
@@ -1324,7 +1387,7 @@ mdimmedsync(SMgrRelation reln, ForkNumber forknum)
*/
mdnblocks(reln, forknum);
- min_inactive_seg = segno = reln->md_num_open_segs[forknum];
+ min_inactive_seg = segno = mdreln->md_num_open_segs[forknum];
/*
* Temporarily open inactive segments, then close them after sync. There
@@ -1332,12 +1395,12 @@ mdimmedsync(SMgrRelation reln, ForkNumber forknum)
* is harmless. We don't bother to clean them up and take a risk of
* further trouble. The next mdclose() will soon close them.
*/
- while (_mdfd_openseg(reln, forknum, segno, 0) != NULL)
+ while (_mdfd_openseg(mdreln, forknum, segno, 0) != NULL)
segno++;
while (segno > 0)
{
- MdfdVec *v = &reln->md_seg_fds[forknum][segno - 1];
+ MdfdVec *v = &mdreln->md_seg_fds[forknum][segno - 1];
/*
* fsyncs done through mdimmedsync() should be tracked in a separate
@@ -1358,7 +1421,7 @@ mdimmedsync(SMgrRelation reln, ForkNumber forknum)
if (segno > min_inactive_seg)
{
FileClose(v->mdfd_vfd);
- _fdvec_resize(reln, forknum, segno - 1);
+ _fdvec_resize(mdreln, forknum, segno - 1);
}
segno--;
@@ -1375,14 +1438,14 @@ mdimmedsync(SMgrRelation reln, ForkNumber forknum)
* enough to be a performance problem).
*/
static void
-register_dirty_segment(SMgrRelation reln, ForkNumber forknum, MdfdVec *seg)
+register_dirty_segment(MdSMgrRelation reln, ForkNumber forknum, MdfdVec *seg)
{
FileTag tag;
- INIT_MD_FILETAG(tag, reln->smgr_rlocator.locator, forknum, seg->mdfd_segno);
+ INIT_MD_FILETAG(tag, reln->reln.smgr_rlocator.locator, forknum, seg->mdfd_segno);
/* Temp relations should never be fsync'd */
- Assert(!SmgrIsTemp(reln));
+ Assert(!SmgrIsTemp(&reln->reln));
if (!RegisterSyncRequest(&tag, SYNC_REQUEST, false /* retryOnError */ ))
{
@@ -1500,7 +1563,7 @@ DropRelationFiles(RelFileLocator *delrels, int ndelrels, bool isRedo)
* _fdvec_resize() -- Resize the fork's open segments array
*/
static void
-_fdvec_resize(SMgrRelation reln,
+_fdvec_resize(MdSMgrRelation reln,
ForkNumber forknum,
int nseg)
{
@@ -1548,12 +1611,12 @@ _fdvec_resize(SMgrRelation reln,
* returned string is palloc'd.
*/
static MdPathStr
-_mdfd_segpath(SMgrRelation reln, ForkNumber forknum, BlockNumber segno)
+_mdfd_segpath(MdSMgrRelation reln, ForkNumber forknum, BlockNumber segno)
{
RelPathStr path;
MdPathStr fullpath;
- path = relpath(reln->smgr_rlocator, forknum);
+ path = relpath(reln->reln.smgr_rlocator, forknum);
if (segno > 0)
sprintf(fullpath.str, "%s.%u", path.str, segno);
@@ -1568,7 +1631,7 @@ _mdfd_segpath(SMgrRelation reln, ForkNumber forknum, BlockNumber segno)
* and make a MdfdVec object for it. Returns NULL on failure.
*/
static MdfdVec *
-_mdfd_openseg(SMgrRelation reln, ForkNumber forknum, BlockNumber segno,
+_mdfd_openseg(MdSMgrRelation reln, ForkNumber forknum, BlockNumber segno,
int oflags)
{
MdfdVec *v;
@@ -1611,7 +1674,7 @@ _mdfd_openseg(SMgrRelation reln, ForkNumber forknum, BlockNumber segno,
* EXTENSION_CREATE case.
*/
static MdfdVec *
-_mdfd_getseg(SMgrRelation reln, ForkNumber forknum, BlockNumber blkno,
+_mdfd_getseg(MdSMgrRelation reln, ForkNumber forknum, BlockNumber blkno,
bool skipFsync, int behavior)
{
MdfdVec *v;
@@ -1685,7 +1748,7 @@ _mdfd_getseg(SMgrRelation reln, ForkNumber forknum, BlockNumber blkno,
char *zerobuf = palloc_aligned(BLCKSZ, PG_IO_ALIGN_SIZE,
MCXT_ALLOC_ZERO);
- mdextend(reln, forknum,
+ mdextend((SMgrRelation) reln, forknum,
nextsegno * ((BlockNumber) RELSEG_SIZE) - 1,
zerobuf, skipFsync);
pfree(zerobuf);
@@ -1740,7 +1803,7 @@ _mdfd_getseg(SMgrRelation reln, ForkNumber forknum, BlockNumber blkno,
* Get number of blocks present in a single disk file
*/
static BlockNumber
-_mdnblocks(SMgrRelation reln, ForkNumber forknum, MdfdVec *seg)
+_mdnblocks(MdSMgrRelation reln, ForkNumber forknum, MdfdVec *seg)
{
off_t len;
@@ -1763,7 +1826,7 @@ _mdnblocks(SMgrRelation reln, ForkNumber forknum, MdfdVec *seg)
int
mdsyncfiletag(const FileTag *ftag, char *path)
{
- SMgrRelation reln = smgropen(ftag->rlocator, INVALID_PROC_NUMBER);
+ MdSMgrRelation reln = (MdSMgrRelation) smgropen(ftag->rlocator, INVALID_PROC_NUMBER);
File file;
instr_time io_start;
bool need_to_close;
diff --git a/src/backend/storage/smgr/smgr.c b/src/backend/storage/smgr/smgr.c
index ebe35c04de5..7635c231ea0 100644
--- a/src/backend/storage/smgr/smgr.c
+++ b/src/backend/storage/smgr/smgr.c
@@ -53,84 +53,21 @@
#include "access/xlogutils.h"
#include "lib/ilist.h"
+#include "miscadmin.h"
#include "storage/bufmgr.h"
#include "storage/ipc.h"
#include "storage/md.h"
#include "storage/smgr.h"
+#include "port/atomics.h"
#include "utils/hsearch.h"
#include "utils/inval.h"
+#include "utils/memutils.h"
+static f_smgr *smgrsw;
-/*
- * This struct of function pointers defines the API between smgr.c and
- * any individual storage manager module. Note that smgr subfunctions are
- * generally expected to report problems via elog(ERROR). An exception is
- * that smgr_unlink should use elog(WARNING), rather than erroring out,
- * because we normally unlink relations during post-commit/abort cleanup,
- * and so it's too late to raise an error. Also, various conditions that
- * would normally be errors should be allowed during bootstrap and/or WAL
- * recovery --- see comments in md.c for details.
- */
-typedef struct f_smgr
-{
- void (*smgr_init) (void); /* may be NULL */
- void (*smgr_shutdown) (void); /* may be NULL */
- void (*smgr_open) (SMgrRelation reln);
- void (*smgr_close) (SMgrRelation reln, ForkNumber forknum);
- void (*smgr_create) (SMgrRelation reln, ForkNumber forknum,
- bool isRedo);
- bool (*smgr_exists) (SMgrRelation reln, ForkNumber forknum);
- void (*smgr_unlink) (RelFileLocatorBackend rlocator, ForkNumber forknum,
- bool isRedo);
- void (*smgr_extend) (SMgrRelation reln, ForkNumber forknum,
- BlockNumber blocknum, const void *buffer, bool skipFsync);
- void (*smgr_zeroextend) (SMgrRelation reln, ForkNumber forknum,
- BlockNumber blocknum, int nblocks, bool skipFsync);
- bool (*smgr_prefetch) (SMgrRelation reln, ForkNumber forknum,
- BlockNumber blocknum, int nblocks);
- uint32 (*smgr_maxcombine) (SMgrRelation reln, ForkNumber forknum,
- BlockNumber blocknum);
- void (*smgr_readv) (SMgrRelation reln, ForkNumber forknum,
- BlockNumber blocknum,
- void **buffers, BlockNumber nblocks);
- void (*smgr_writev) (SMgrRelation reln, ForkNumber forknum,
- BlockNumber blocknum,
- const void **buffers, BlockNumber nblocks,
- bool skipFsync);
- void (*smgr_writeback) (SMgrRelation reln, ForkNumber forknum,
- BlockNumber blocknum, BlockNumber nblocks);
- BlockNumber (*smgr_nblocks) (SMgrRelation reln, ForkNumber forknum);
- void (*smgr_truncate) (SMgrRelation reln, ForkNumber forknum,
- BlockNumber old_blocks, BlockNumber nblocks);
- void (*smgr_immedsync) (SMgrRelation reln, ForkNumber forknum);
- void (*smgr_registersync) (SMgrRelation reln, ForkNumber forknum);
-} f_smgr;
-
-static const f_smgr smgrsw[] = {
- /* magnetic disk */
- {
- .smgr_init = mdinit,
- .smgr_shutdown = NULL,
- .smgr_open = mdopen,
- .smgr_close = mdclose,
- .smgr_create = mdcreate,
- .smgr_exists = mdexists,
- .smgr_unlink = mdunlink,
- .smgr_extend = mdextend,
- .smgr_zeroextend = mdzeroextend,
- .smgr_prefetch = mdprefetch,
- .smgr_maxcombine = mdmaxcombine,
- .smgr_readv = mdreadv,
- .smgr_writev = mdwritev,
- .smgr_writeback = mdwriteback,
- .smgr_nblocks = mdnblocks,
- .smgr_truncate = mdtruncate,
- .smgr_immedsync = mdimmedsync,
- .smgr_registersync = mdregistersync,
- }
-};
+static int NSmgr = 0;
-static const int NSmgr = lengthof(smgrsw);
+static Size LargestSMgrRelationSize = 0;
/*
* Each backend has a hashtable that stores all extant SMgrRelation objects.
@@ -144,6 +81,60 @@ static dlist_head unpinned_relns;
static void smgrshutdown(int code, Datum arg);
static void smgrdestroy(SMgrRelation reln);
+#define MaxSMgrId UINT8_MAX
+
+SMgrId
+smgr_register(const f_smgr *smgr, Size smgrrelation_size)
+{
+ SMgrId my_id;
+ MemoryContext old;
+
+ if (process_shared_preload_libraries_done)
+ elog(FATAL, "SMgrs must be registered in the shared_preload_libraries phase");
+ if (NSmgr == MaxSMgrId)
+ elog(FATAL, "Too many smgrs registered");
+ if (smgr->name == NULL || *smgr->name == 0)
+ elog(FATAL, "smgr registered with invalid name");
+
+ Assert(smgr->smgr_open != NULL);
+ Assert(smgr->smgr_close != NULL);
+ Assert(smgr->smgr_create != NULL);
+ Assert(smgr->smgr_exists != NULL);
+ Assert(smgr->smgr_unlink != NULL);
+ Assert(smgr->smgr_extend != NULL);
+ Assert(smgr->smgr_zeroextend != NULL);
+ Assert(smgr->smgr_prefetch != NULL);
+ Assert(smgr->smgr_readv != NULL);
+ Assert(smgr->smgr_writev != NULL);
+ Assert(smgr->smgr_writeback != NULL);
+ Assert(smgr->smgr_nblocks != NULL);
+ Assert(smgr->smgr_truncate != NULL);
+ Assert(smgr->smgr_immedsync != NULL);
+
+ old = MemoryContextSwitchTo(TopMemoryContext);
+
+ my_id = NSmgr++;
+ if (my_id == 0)
+ smgrsw = palloc_array(f_smgr, 1);
+ else
+ smgrsw = repalloc_array(smgrsw, f_smgr, NSmgr);
+
+ MemoryContextSwitchTo(old);
+
+ pg_compiler_barrier();
+
+ if (!smgrsw)
+ {
+ NSmgr--;
+ elog(FATAL, "Failed to extend smgr array");
+ }
+
+ smgrsw[my_id] = *smgr;
+
+ LargestSMgrRelationSize = Max(LargestSMgrRelationSize, smgrrelation_size);
+
+ return my_id;
+}
/*
* smgrinit(), smgrshutdown() -- Initialize or shut down storage
@@ -211,8 +202,11 @@ smgropen(RelFileLocator rlocator, ProcNumber backend)
/* First time through: initialize the hash table */
HASHCTL ctl;
+ LargestSMgrRelationSize = MAXALIGN(LargestSMgrRelationSize);
+ Assert(NSmgr > 0);
+
ctl.keysize = sizeof(RelFileLocatorBackend);
- ctl.entrysize = sizeof(SMgrRelationData);
+ ctl.entrysize = LargestSMgrRelationSize;
SMgrRelationHash = hash_create("smgr relation table", 400,
&ctl, HASH_ELEM | HASH_BLOBS);
dlist_init(&unpinned_relns);
@@ -232,7 +226,8 @@ smgropen(RelFileLocator rlocator, ProcNumber backend)
reln->smgr_targblock = InvalidBlockNumber;
for (int i = 0; i <= MAX_FORKNUM; ++i)
reln->smgr_cached_nblocks[i] = InvalidBlockNumber;
- reln->smgr_which = 0; /* we only have md.c at present */
+
+ reln->smgr_which = MdSMgrId; /* we only have md.c at present */
/* implementation-specific initialization */
smgrsw[reln->smgr_which].smgr_open(reln);
diff --git a/src/backend/utils/init/miscinit.c b/src/backend/utils/init/miscinit.c
index dc3521457c7..3176cdce6d7 100644
--- a/src/backend/utils/init/miscinit.c
+++ b/src/backend/utils/init/miscinit.c
@@ -43,6 +43,7 @@
#include "replication/slotsync.h"
#include "storage/fd.h"
#include "storage/ipc.h"
+#include "storage/md.h"
#include "storage/latch.h"
#include "storage/pg_shmem.h"
#include "storage/pmsignal.h"
@@ -192,6 +193,9 @@ InitStandaloneProcess(const char *argv0)
InitProcessLocalLatch();
InitializeLatchWaitSet();
+ /* Initialize smgrs */
+ register_builtin_dynamic_managers();
+
/*
* For consistency with InitPostmasterChild, initialize signal mask here.
* But we don't unblock SIGQUIT or provide a default handler for it.
@@ -1920,6 +1924,15 @@ process_session_preload_libraries(void)
true);
}
+/*
+ * Register any internal managers.
+ */
+void
+register_builtin_dynamic_managers(void)
+{
+ mdsmgr_register();
+}
+
/*
* process any shared memory requests from preloaded libraries
*/
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index a2b63495eec..ff4ef578a1f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -513,6 +513,7 @@ extern void TouchSocketLockFiles(void);
extern void AddToDataDirLockFile(int target_line, const char *str);
extern bool RecheckDataDirLockFile(void);
extern void ValidatePgVersion(const char *path);
+extern void register_builtin_dynamic_managers(void);
extern void process_shared_preload_libraries(void);
extern void process_session_preload_libraries(void);
extern void process_shmem_requests(void);
diff --git a/src/include/storage/md.h b/src/include/storage/md.h
index 05bf537066e..da1d1d339be 100644
--- a/src/include/storage/md.h
+++ b/src/include/storage/md.h
@@ -19,6 +19,10 @@
#include "storage/smgr.h"
#include "storage/sync.h"
+/* registration function for md storage manager */
+extern void mdsmgr_register(void);
+extern SMgrId MdSMgrId;
+
/* md storage manager functionality */
extern void mdinit(void);
extern void mdopen(SMgrRelation reln);
diff --git a/src/include/storage/smgr.h b/src/include/storage/smgr.h
index 4016b206ad6..52f74f917b2 100644
--- a/src/include/storage/smgr.h
+++ b/src/include/storage/smgr.h
@@ -18,6 +18,8 @@
#include "storage/block.h"
#include "storage/relfilelocator.h"
+typedef uint8 SMgrId;
+
/*
* smgr.c maintains a table of SMgrRelation objects, which are essentially
* cached file handles. An SMgrRelation is created (if not already present)
@@ -51,14 +53,7 @@ typedef struct SMgrRelationData
* Fields below here are intended to be private to smgr.c and its
* submodules. Do not touch them from elsewhere.
*/
- int smgr_which; /* storage manager selector */
-
- /*
- * for md.c; per-fork arrays of the number of open segments
- * (md_num_open_segs) and the segments themselves (md_seg_fds).
- */
- int md_num_open_segs[MAX_FORKNUM + 1];
- struct _MdfdVec *md_seg_fds[MAX_FORKNUM + 1];
+ SMgrId smgr_which; /* storage manager selector */
/*
* Pinning support. If unpinned (ie. pincount == 0), 'node' is a list
@@ -73,6 +68,54 @@ typedef SMgrRelationData *SMgrRelation;
#define SmgrIsTemp(smgr) \
RelFileLocatorBackendIsTemp((smgr)->smgr_rlocator)
+/*
+ * This struct of function pointers defines the API between smgr.c and
+ * any individual storage manager module. Note that smgr subfunctions are
+ * generally expected to report problems via elog(ERROR). An exception is
+ * that smgr_unlink should use elog(WARNING), rather than erroring out,
+ * because we normally unlink relations during post-commit/abort cleanup,
+ * and so it's too late to raise an error. Also, various conditions that
+ * would normally be errors should be allowed during bootstrap and/or WAL
+ * recovery --- see comments in md.c for details.
+ */
+typedef struct f_smgr
+{
+ const char *name;
+ void (*smgr_init) (void); /* may be NULL */
+ void (*smgr_shutdown) (void); /* may be NULL */
+ void (*smgr_open) (SMgrRelation reln);
+ void (*smgr_close) (SMgrRelation reln, ForkNumber forknum);
+ void (*smgr_create) (SMgrRelation reln, ForkNumber forknum,
+ bool isRedo);
+ bool (*smgr_exists) (SMgrRelation reln, ForkNumber forknum);
+ void (*smgr_unlink) (RelFileLocatorBackend rlocator, ForkNumber forknum,
+ bool isRedo);
+ void (*smgr_extend) (SMgrRelation reln, ForkNumber forknum,
+ BlockNumber blocknum, const void *buffer, bool skipFsync);
+ void (*smgr_zeroextend) (SMgrRelation reln, ForkNumber forknum,
+ BlockNumber blocknum, int nblocks, bool skipFsync);
+ bool (*smgr_prefetch) (SMgrRelation reln, ForkNumber forknum,
+ BlockNumber blocknum, int nblocks);
+ uint32 (*smgr_maxcombine) (SMgrRelation reln, ForkNumber forknum,
+ BlockNumber blocknum);
+ void (*smgr_readv) (SMgrRelation reln, ForkNumber forknum,
+ BlockNumber blocknum,
+ void **buffers, BlockNumber nblocks);
+ void (*smgr_writev) (SMgrRelation reln, ForkNumber forknum,
+ BlockNumber blocknum,
+ const void **buffers, BlockNumber nblocks,
+ bool skipFsync);
+ void (*smgr_writeback) (SMgrRelation reln, ForkNumber forknum,
+ BlockNumber blocknum, BlockNumber nblocks);
+ BlockNumber (*smgr_nblocks) (SMgrRelation reln, ForkNumber forknum);
+ void (*smgr_truncate) (SMgrRelation reln, ForkNumber forknum,
+ BlockNumber old_blocks, BlockNumber nblocks);
+ void (*smgr_immedsync) (SMgrRelation reln, ForkNumber forknum);
+ void (*smgr_registersync) (SMgrRelation reln, ForkNumber forknum);
+} f_smgr;
+
+extern SMgrId smgr_register(const f_smgr *smgr, Size smgrrelation_size);
+
extern void smgrinit(void);
extern SMgrRelation smgropen(RelFileLocator rlocator, ProcNumber backend);
extern bool smgrexists(SMgrRelation reln, ForkNumber forknum);
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 9840060997f..4b971b81ae5 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1629,6 +1629,7 @@ ManyTestResourceKind
Material
MaterialPath
MaterialState
+MdSMgrRelationData
MdfdVec
MdPathStr
Memoize
--
2.47.2
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Extensible storage manager API - SMGR hook Redux
@ 2025-03-10 12:30 Kirill Reshke <[email protected]>
parent: Andreas Karlsson <[email protected]>
1 sibling, 0 replies; 7+ messages in thread
From: Kirill Reshke @ 2025-03-10 12:30 UTC (permalink / raw)
To: Andreas Karlsson <[email protected]>; +Cc: Nitin Jadhav <[email protected]>; Tristan Partin <[email protected]>; pgsql-hackers; Matthias van de Meent <[email protected]>; Heikki Linnakangas <[email protected]>; Andres Freund <[email protected]>
On Fri, 7 Mar 2025 at 16:52, Andreas Karlsson <[email protected]> wrote:
>
> Hi,
Hi!
> Here is a rebased version of it to make the CI happy.
Looks like CI is still unhappy with this change[0]
0001:
>+
>+SMgrId
>+smgr_register(const f_smgr *smgr, Size smgrrelation_size)
...
> + Assert(smgr->smgr_open != NULL);
> + Assert(smgr->smgr_close != NULL);
> + Assert(smgr->smgr_create != NULL);
> + Assert(smgr->smgr_exists != NULL);
> + Assert(smgr->smgr_unlink != NULL);
> + Assert(smgr->smgr_extend != NULL);
> + Assert(smgr->smgr_zeroextend != NULL);
> + Assert(smgr->smgr_prefetch != NULL);
> + Assert(smgr->smgr_readv != NULL);
> + Assert(smgr->smgr_writev != NULL);
> + Assert(smgr->smgr_writeback != NULL);
> + Assert(smgr->smgr_nblocks != NULL);
> + Assert(smgr->smgr_truncate != NULL);
> + Assert(smgr->smgr_immedsync != NULL);
Are we sure we need to force extension authors to implement prefetch?
Also, do we intentionally skip Assert on smgr_registersync and
smgr_init here? I am not questioning smgr_shutdown here, as I can see
it is NULL for md implementation.
0002:
should we merge this with 0001?
0003: Looks mature, no comments.
0004:
It's a bit strange to place fsync_checker under contrib, huh? Like,
you will never use it in production. Maybe src/test/modules is a
better place?
0005:
We are missing rationale for this change in the commit message.
I didn't look at the 0006 modifications. Later, I'll try to take another look.
[0] https://cirrus-ci.com/task/6466113875214336
--
Best regards,
Kirill Reshke
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Extensible storage manager API - SMGR hook Redux
@ 2025-03-17 06:52 vignesh C <[email protected]>
parent: Andreas Karlsson <[email protected]>
1 sibling, 0 replies; 7+ messages in thread
From: vignesh C @ 2025-03-17 06:52 UTC (permalink / raw)
To: Andreas Karlsson <[email protected]>; +Cc: Nitin Jadhav <[email protected]>; Tristan Partin <[email protected]>; pgsql-hackers; Matthias van de Meent <[email protected]>; Heikki Linnakangas <[email protected]>; Andres Freund <[email protected]>
On Fri, 7 Mar 2025 at 17:22, Andreas Karlsson <[email protected]> wrote:
>
> Hi,
>
> Here is a rebased version of it to make the CI happy. I plan to work
> more on this next week but am happy with any feedback on what is already
> there.
I noticed that Kirill's comments from [1] are not yet addressed, I
have changed the commitfest entry status to "Waiting on Author. Please
address them and change the status to "Needs Review".
[1] - https://www.postgresql.org/message-id/[email protected]...
Regards,
Vignesh
^ permalink raw reply [nested|flat] 7+ messages in thread
end of thread, other threads:[~2025-03-17 06:52 UTC | newest]
Thread overview: 7+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2021-06-29 21:49 Extensible storage manager API - smgr hooks Anastasia Lubennikova <[email protected]>
2021-06-30 02:36 ` Yura Sokolov <[email protected]>
2022-06-16 08:41 ` Kirill Reshke <[email protected]>
2022-08-26 05:25 ` Andrey Borodin <[email protected]>
2025-03-07 11:52 Re: Extensible storage manager API - SMGR hook Redux Andreas Karlsson <[email protected]>
2025-03-10 12:30 ` Re: Extensible storage manager API - SMGR hook Redux Kirill Reshke <[email protected]>
2025-03-17 06:52 ` Re: Extensible storage manager API - SMGR hook Redux vignesh C <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox