public inbox for [email protected]
help / color / mirror / Atom feedFrom: Sadhuprasad Patro <[email protected]>
To: Jelte Fennema <[email protected]>
Cc: PostgreSQL Hackers <[email protected]>
Subject: Re: Per-table storage parameters for TableAM/IndexAM extensions
Date: Tue, 18 Jan 2022 22:44:10 +0530
Message-ID: <CAFF0-CFSy0wJCTfjJ6QLsHZGfFLAVKiHzgdTJyNnzqJfWpwh7w@mail.gmail.com> (raw)
In-Reply-To: <CAGECzQRDj2UrOvg-rJfWdTAPjh0WEjt7xBWV61bGTOH7jONcPw@mail.gmail.com>
References: <CAFF0-CG4KZHdtYHMsonWiXNzj16gWZpduXAn8yF7pDDub+GQMg@mail.gmail.com>
<CAGECzQRDj2UrOvg-rJfWdTAPjh0WEjt7xBWV61bGTOH7jONcPw@mail.gmail.com>
On Mon, Jan 17, 2022 at 4:47 PM Jelte Fennema <[email protected]> wrote:
>
> Big +1, this is a great addition!
>
> I think it would be very useful if there were some tests for this new
> feature. Something similar to the tests for storage parameters for
> index AMs in src/test/modules/dummy_index_am.
>
Sure, I will refer to the index AM test and add the test cases needed.
> Apart from that I think the documentation for table storage parameters
> needs to be updated in doc/src/sgml/ref/create_table.sgml. It now
> needs to indicate that these parameters are different for each table
> access method. Similar to this paragraph in the create index storage
> parameter section of the docs:
Sure, I will add the documentation part for this.
As of now, I have fixed the comments from Dilip & Rushabh and have
done some more changes after internal testing and review. Please find
the latest patch attached.
Thanks & Regards
SadhuPrasad
www.EnterpriseDB.com
Attachments:
[application/octet-stream] v2-0001-PATCH-v2-Per-table-storage-parameters-for-TableAM.patch (14.1K, ../CAFF0-CFSy0wJCTfjJ6QLsHZGfFLAVKiHzgdTJyNnzqJfWpwh7w@mail.gmail.com/2-v2-0001-PATCH-v2-Per-table-storage-parameters-for-TableAM.patch)
download | inline diff:
From 6e09f9e5c95f3c88c0240437f99b854a92355e50 Mon Sep 17 00:00:00 2001
From: B Sadhu Prasad Patro <[email protected]>
Date: Tue, 18 Jan 2022 08:36:28 -0800
Subject: [PATCH v2] [PATCH v2] Per-table storage parameters for
TableAM/IndexAM extensions
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Currently all the storage options for a table are very much specific
to the heap but a different AM might need some user defined AM
specific parameters to help tune the AM. So here is a patch which
provides an AM level routine so that instead of getting parameters
validated using “heap_reloptions” it will call the registered AM
routine.
---
src/backend/access/common/reloptions.c | 30 +++++++++++++--
src/backend/access/heap/heapam_handler.c | 1 +
src/backend/commands/tablecmds.c | 66 +++++++++++++++++++++++---------
src/backend/postmaster/autovacuum.c | 18 ++++++---
src/backend/utils/cache/relcache.c | 11 +++++-
src/include/access/reloptions.h | 6 ++-
src/include/access/tableam.h | 8 +++-
7 files changed, 108 insertions(+), 32 deletions(-)
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index d592655..bcb08d7 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -1372,7 +1372,8 @@ untransformRelOptions(Datum options)
*/
bytea *
extractRelOptions(HeapTuple tuple, TupleDesc tupdesc,
- amoptions_function amoptions)
+ amoptions_function amoptions,
+ reloptions_function reloptions)
{
bytea *options;
bool isnull;
@@ -1394,7 +1395,9 @@ extractRelOptions(HeapTuple tuple, TupleDesc tupdesc,
case RELKIND_RELATION:
case RELKIND_TOASTVALUE:
case RELKIND_MATVIEW:
- options = heap_reloptions(classForm->relkind, datum, false);
+ options = table_reloptions(reloptions,
+ classForm->relkind,
+ datum, false);
break;
case RELKIND_PARTITIONED_TABLE:
options = partitioned_table_reloptions(datum, false);
@@ -2007,7 +2010,8 @@ view_reloptions(Datum reloptions, bool validate)
}
/*
- * Parse options for heaps, views and toast tables.
+ * Parse options for heaps, views and toast tables. This is
+ * implementation of relOptions for access method heap.
*/
bytea *
heap_reloptions(char relkind, Datum reloptions, bool validate)
@@ -2038,6 +2042,26 @@ heap_reloptions(char relkind, Datum reloptions, bool validate)
/*
+ * Parse options for tables.
+ *
+ * reloptions tables AM's option parser function
+ * reloptions options as text[] datum
+ * validate error flag
+ */
+bytea *
+table_reloptions(reloptions_function reloptsfun, char relkind,
+ Datum reloptions, bool validate)
+{
+ Assert(reloptsfun != NULL);
+
+ /* Assume function is strict */
+ if (!PointerIsValid(DatumGetPointer(reloptions)))
+ return NULL;
+
+ return reloptsfun(relkind, reloptions, validate);
+}
+
+/*
* Parse options for indexes.
*
* amoptions index AM's option parser function
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 39ef8a0..4f7f110 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2581,6 +2581,7 @@ static const TableAmRoutine heapam_methods = {
.index_build_range_scan = heapam_index_build_range_scan,
.index_validate_scan = heapam_index_validate_scan,
+ .relation_options = heap_reloptions,
.relation_size = table_block_relation_size,
.relation_needs_toast_table = heapam_relation_needs_toast_table,
.relation_toast_am = heapam_relation_toast_am,
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 1f0654c..2d4bb5d 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -808,24 +808,6 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (!OidIsValid(ownerId))
ownerId = GetUserId();
- /*
- * Parse and validate reloptions, if any.
- */
- reloptions = transformRelOptions((Datum) 0, stmt->options, NULL, validnsps,
- true, false);
-
- switch (relkind)
- {
- case RELKIND_VIEW:
- (void) view_reloptions(reloptions, true);
- break;
- case RELKIND_PARTITIONED_TABLE:
- (void) partitioned_table_reloptions(reloptions, true);
- break;
- default:
- (void) heap_reloptions(relkind, reloptions, true);
- }
-
if (stmt->ofTypename)
{
AclResult aclresult;
@@ -946,6 +928,52 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
accessMethodId = get_table_am_oid(accessMethod, false);
/*
+ * Parse and validate reloptions, if any.
+ */
+ reloptions = transformRelOptions((Datum) 0, stmt->options, NULL, validnsps,
+ true, false);
+ switch (relkind)
+ {
+ case RELKIND_VIEW:
+ (void) view_reloptions(reloptions, true);
+ break;
+ case RELKIND_PARTITIONED_TABLE:
+ (void) partitioned_table_reloptions(reloptions, true);
+ break;
+ case RELKIND_RELATION:
+ case RELKIND_TOASTVALUE:
+ case RELKIND_MATVIEW:
+ {
+ const TableAmRoutine *routine;
+ HeapTuple tuple;
+ Form_pg_am aform;
+
+ tuple = SearchSysCache1(AMOID, ObjectIdGetDatum(accessMethodId));
+ if (!HeapTupleIsValid(tuple))
+ {
+ elog(ERROR, "cache lookup failed for access method %u",
+ accessMethodId);
+ }
+
+ aform = (Form_pg_am) GETSTRUCT(tuple);
+ routine = GetTableAmRoutine(aform->amhandler);
+ if (routine->relation_options == NULL)
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("specifying a table access method is not supported")));
+ }
+
+ (void) routine->relation_options(relkind, reloptions, true);
+ ReleaseSysCache(tuple);
+ break;
+ }
+
+ default:
+ (void) heap_reloptions(relkind, reloptions, true);
+ }
+
+ /*
* Create the relation. Inherited defaults and constraints are passed in
* for immediate handling --- since they don't need parsing, they can be
* stored immediately.
@@ -14136,7 +14164,7 @@ ATExecSetRelOptions(Relation rel, List *defList, AlterTableType operation,
case RELKIND_RELATION:
case RELKIND_TOASTVALUE:
case RELKIND_MATVIEW:
- (void) heap_reloptions(rel->rd_rel->relkind, newOptions, true);
+ rel->rd_tableam->relation_options(rel->rd_rel->relkind, newOptions, true);
break;
case RELKIND_PARTITIONED_TABLE:
(void) partitioned_table_reloptions(newOptions, true);
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 681ef91..cd9c0bb 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -327,6 +327,7 @@ static void FreeWorkerInfo(int code, Datum arg);
static autovac_table *table_recheck_autovac(Oid relid, HTAB *table_toast_map,
TupleDesc pg_class_desc,
+ reloptions_function reloptions,
int effective_multixact_freeze_max_age);
static void recheck_relation_needs_vacanalyze(Oid relid, AutoVacOpts *avopts,
Form_pg_class classForm,
@@ -341,7 +342,7 @@ static void relation_needs_vacanalyze(Oid relid, AutoVacOpts *relopts,
static void autovacuum_do_vac_analyze(autovac_table *tab,
BufferAccessStrategy bstrategy);
static AutoVacOpts *extract_autovac_opts(HeapTuple tup,
- TupleDesc pg_class_desc);
+ TupleDesc pg_class_desc, reloptions_function reloptions);
static PgStat_StatTabEntry *get_pgstat_tabentry_relid(Oid relid, bool isshared,
PgStat_StatDBEntry *shared,
PgStat_StatDBEntry *dbentry);
@@ -2118,7 +2119,8 @@ do_autovacuum(void)
}
/* Fetch reloptions and the pgstat entry for this table */
- relopts = extract_autovac_opts(tuple, pg_class_desc);
+ relopts = extract_autovac_opts(tuple, pg_class_desc,
+ classRel->rd_tableam->relation_options);
tabentry = get_pgstat_tabentry_relid(relid, classForm->relisshared,
shared, dbentry);
@@ -2191,7 +2193,8 @@ do_autovacuum(void)
* fetch reloptions -- if this toast table does not have them, try the
* main rel
*/
- relopts = extract_autovac_opts(tuple, pg_class_desc);
+ relopts = extract_autovac_opts(tuple, pg_class_desc,
+ classRel->rd_tableam->relation_options);
if (relopts == NULL)
{
av_relation *hentry;
@@ -2427,6 +2430,7 @@ do_autovacuum(void)
*/
MemoryContextSwitchTo(AutovacMemCxt);
tab = table_recheck_autovac(relid, table_toast_map, pg_class_desc,
+ classRel->rd_tableam->relation_options,
effective_multixact_freeze_max_age);
if (tab == NULL)
{
@@ -2748,7 +2752,8 @@ deleted2:
* be a risk; fortunately, it doesn't.
*/
static AutoVacOpts *
-extract_autovac_opts(HeapTuple tup, TupleDesc pg_class_desc)
+extract_autovac_opts(HeapTuple tup, TupleDesc pg_class_desc,
+ reloptions_function reloptions)
{
bytea *relopts;
AutoVacOpts *av;
@@ -2757,7 +2762,7 @@ extract_autovac_opts(HeapTuple tup, TupleDesc pg_class_desc)
((Form_pg_class) GETSTRUCT(tup))->relkind == RELKIND_MATVIEW ||
((Form_pg_class) GETSTRUCT(tup))->relkind == RELKIND_TOASTVALUE);
- relopts = extractRelOptions(tup, pg_class_desc, NULL);
+ relopts = extractRelOptions(tup, pg_class_desc, NULL, reloptions);
if (relopts == NULL)
return NULL;
@@ -2803,6 +2808,7 @@ get_pgstat_tabentry_relid(Oid relid, bool isshared, PgStat_StatDBEntry *shared,
static autovac_table *
table_recheck_autovac(Oid relid, HTAB *table_toast_map,
TupleDesc pg_class_desc,
+ reloptions_function reloptions,
int effective_multixact_freeze_max_age)
{
Form_pg_class classForm;
@@ -2824,7 +2830,7 @@ table_recheck_autovac(Oid relid, HTAB *table_toast_map,
* Get the applicable reloptions. If it is a TOAST table, try to get the
* main table reloptions if the toast table itself doesn't have.
*/
- avopts = extract_autovac_opts(classTup, pg_class_desc);
+ avopts = extract_autovac_opts(classTup, pg_class_desc, reloptions);
if (classForm->relkind == RELKIND_TOASTVALUE &&
avopts == NULL && table_toast_map != NULL)
{
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 2e760e8..17bdfa3 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -460,6 +460,7 @@ RelationParseRelOptions(Relation relation, HeapTuple tuple)
{
bytea *options;
amoptions_function amoptsfn;
+ reloptions_function reloptsfn;
relation->rd_options = NULL;
@@ -471,13 +472,18 @@ RelationParseRelOptions(Relation relation, HeapTuple tuple)
{
case RELKIND_RELATION:
case RELKIND_TOASTVALUE:
- case RELKIND_VIEW:
case RELKIND_MATVIEW:
+ reloptsfn = relation->rd_tableam->relation_options;
+ amoptsfn = NULL;
+ break;
+ case RELKIND_VIEW:
case RELKIND_PARTITIONED_TABLE:
+ reloptsfn = NULL;
amoptsfn = NULL;
break;
case RELKIND_INDEX:
case RELKIND_PARTITIONED_INDEX:
+ reloptsfn = NULL;
amoptsfn = relation->rd_indam->amoptions;
break;
default:
@@ -489,7 +495,8 @@ RelationParseRelOptions(Relation relation, HeapTuple tuple)
* we might not have any other for pg_class yet (consider executing this
* code for pg_class itself)
*/
- options = extractRelOptions(tuple, GetPgClassDescriptor(), amoptsfn);
+ options = extractRelOptions(tuple, GetPgClassDescriptor(),
+ amoptsfn, reloptsfn);
/*
* Copy parsed data into CacheMemoryContext. To guard against the
diff --git a/src/include/access/reloptions.h b/src/include/access/reloptions.h
index f740513..c42d5e9 100644
--- a/src/include/access/reloptions.h
+++ b/src/include/access/reloptions.h
@@ -21,6 +21,7 @@
#include "access/amapi.h"
#include "access/htup.h"
+#include "access/tableam.h"
#include "access/tupdesc.h"
#include "nodes/pg_list.h"
#include "storage/lock.h"
@@ -224,7 +225,8 @@ extern Datum transformRelOptions(Datum oldOptions, List *defList,
bool acceptOidsOff, bool isReset);
extern List *untransformRelOptions(Datum options);
extern bytea *extractRelOptions(HeapTuple tuple, TupleDesc tupdesc,
- amoptions_function amoptions);
+ amoptions_function amoptions,
+ reloptions_function reloptions);
extern void *build_reloptions(Datum reloptions, bool validate,
relopt_kind kind,
Size relopt_struct_size,
@@ -238,6 +240,8 @@ extern bytea *default_reloptions(Datum reloptions, bool validate,
extern bytea *heap_reloptions(char relkind, Datum reloptions, bool validate);
extern bytea *view_reloptions(Datum reloptions, bool validate);
extern bytea *partitioned_table_reloptions(Datum reloptions, bool validate);
+extern bytea *table_reloptions(reloptions_function reloptsfun, char relkind,
+ Datum reloptions, bool validate);
extern bytea *index_reloptions(amoptions_function amoptions, Datum reloptions,
bool validate);
extern bytea *attribute_reloptions(Datum reloptions, bool validate);
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index bb36573..e110085 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -252,6 +252,10 @@ typedef void (*IndexBuildCallback) (Relation index,
bool tupleIsAlive,
void *state);
+/* This callback parse the table reloptions and returns in bytea format */
+typedef bytea *(*reloptions_function) (char relkind,
+ Datum reloptions, bool validate);
+
/*
* API struct for a table AM. Note this must be allocated in a
* server-lifetime manner, typically as a static const struct, which then gets
@@ -692,6 +696,8 @@ typedef struct TableAmRoutine
* ------------------------------------------------------------------------
*/
+ reloptions_function relation_options;
+
/*
* See table_relation_size().
*
@@ -702,7 +708,6 @@ typedef struct TableAmRoutine
*/
uint64 (*relation_size) (Relation rel, ForkNumber forkNumber);
-
/*
* This callback should return true if the relation requires a TOAST table
* and false if it does not. It may wish to examine the relation's tuple
@@ -2073,5 +2078,6 @@ extern const TableAmRoutine *GetTableAmRoutine(Oid amhandler);
extern const TableAmRoutine *GetHeapamTableAmRoutine(void);
extern bool check_default_table_access_method(char **newval, void **extra,
GucSource source);
+extern bytea *heap_reloptions(char relkind, Datum reloptions, bool validate);
#endif /* TABLEAM_H */
--
1.8.3.1
view thread (2+ messages)
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected]
Subject: Re: Per-table storage parameters for TableAM/IndexAM extensions
In-Reply-To: <CAFF0-CFSy0wJCTfjJ6QLsHZGfFLAVKiHzgdTJyNnzqJfWpwh7w@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox