public inbox for [email protected]
help / color / mirror / Atom feed[PATCH v33 1/5] Base implementation of subscripting mechanism
3+ messages / 3 participants
[nested] [flat]
* [PATCH v33 1/5] Base implementation of subscripting mechanism
@ 2019-01-31 21:37 erthalion <[email protected]>
0 siblings, 0 replies; 3+ messages in thread
From: erthalion @ 2019-01-31 21:37 UTC (permalink / raw)
Introduce all the required machinery for generalizing subscripting
operation for a different data types:
* subscripting handler procedure, to set up a relation between data type
and corresponding subscripting logic.
* subscripting routines, that help generalize a subscripting logic,
since it involves few stages, namely preparation (e.g. to figure out
required types), validation (to check the input and return meaningful
error message), fetch (executed when we extract a value using
subscripting), assign (executed when we update a data type with a new
value using subscripting). Without this it would be neccessary to
introduce more new fields to pg_type, which would be too invasive.
All ArrayRef related logic was removed and landed as a separate
subscripting implementation in the following patch from this series. The
rest of the code was rearranged, to e.g. store a type of assigned value
for an assign operation.
Reviewed-by: Tom Lane, Arthur Zakirov
---
.../pg_stat_statements/pg_stat_statements.c | 1 +
src/backend/catalog/heap.c | 6 +-
src/backend/catalog/pg_type.c | 15 +-
src/backend/commands/typecmds.c | 77 +++++++++-
src/backend/executor/execExpr.c | 25 +---
src/backend/executor/execExprInterp.c | 124 +++------------
src/backend/nodes/copyfuncs.c | 2 +
src/backend/nodes/equalfuncs.c | 2 +
src/backend/nodes/outfuncs.c | 2 +
src/backend/nodes/readfuncs.c | 2 +
src/backend/parser/parse_expr.c | 54 ++++---
src/backend/parser/parse_node.c | 141 ++++--------------
src/backend/parser/parse_target.c | 88 +++++------
src/backend/utils/adt/ruleutils.c | 21 +--
src/backend/utils/cache/lsyscache.c | 23 +++
src/include/c.h | 2 +
src/include/catalog/pg_type.h | 9 +-
src/include/executor/execExpr.h | 13 +-
src/include/nodes/primnodes.h | 6 +
src/include/nodes/subscripting.h | 42 ++++++
src/include/parser/parse_node.h | 6 +-
src/include/utils/lsyscache.h | 1 +
22 files changed, 336 insertions(+), 326 deletions(-)
create mode 100644 src/include/nodes/subscripting.h
diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c
index 14cad19afb..bf19507d32 100644
--- a/contrib/pg_stat_statements/pg_stat_statements.c
+++ b/contrib/pg_stat_statements/pg_stat_statements.c
@@ -2793,6 +2793,7 @@ JumbleExpr(pgssJumbleState *jstate, Node *node)
JumbleExpr(jstate, (Node *) sbsref->reflowerindexpr);
JumbleExpr(jstate, (Node *) sbsref->refexpr);
JumbleExpr(jstate, (Node *) sbsref->refassgnexpr);
+ APP_JUMB(sbsref->refnestedfunc);
}
break;
case T_FuncExpr:
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 3985326df6..911e2a1ffe 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -1056,7 +1056,8 @@ AddNewRelationType(const char *typeName,
-1, /* typmod */
0, /* array dimensions for typBaseType */
false, /* Type NOT NULL */
- InvalidOid); /* rowtypes never have a collation */
+ InvalidOid, /* rowtypes never have a collation */
+ InvalidOid); /* typsubshandler - none */
}
/* --------------------------------
@@ -1335,7 +1336,8 @@ heap_create_with_catalog(const char *relname,
-1, /* typmod */
0, /* array dimensions for typBaseType */
false, /* Type NOT NULL */
- InvalidOid); /* rowtypes never have a collation */
+ InvalidOid, /* rowtypes never have a collation */
+ 0); /* array implementation */
pfree(relarrayname);
}
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..bf353878f5 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -118,6 +118,7 @@ TypeShellMake(const char *typeName, Oid typeNamespace, Oid ownerId)
values[Anum_pg_type_typtypmod - 1] = Int32GetDatum(-1);
values[Anum_pg_type_typndims - 1] = Int32GetDatum(0);
values[Anum_pg_type_typcollation - 1] = ObjectIdGetDatum(InvalidOid);
+ values[Anum_pg_type_typsubshandler - 1] = ObjectIdGetDatum(InvalidOid);
nulls[Anum_pg_type_typdefaultbin - 1] = true;
nulls[Anum_pg_type_typdefault - 1] = true;
nulls[Anum_pg_type_typacl - 1] = true;
@@ -158,10 +159,10 @@ TypeShellMake(const char *typeName, Oid typeNamespace, Oid ownerId)
GenerateTypeDependencies(tup,
pg_type_desc,
NULL,
- NULL,
0,
false,
false,
+ InvalidOid,
false);
/* Post creation hook for new shell type */
@@ -219,7 +220,8 @@ TypeCreate(Oid newTypeOid,
int32 typeMod,
int32 typNDims, /* Array dimensions for baseType */
bool typeNotNull,
- Oid typeCollation)
+ Oid typeCollation,
+ Oid subscriptingHandlerProcedure)
{
Relation pg_type_desc;
Oid typeObjectId;
@@ -372,6 +374,7 @@ TypeCreate(Oid newTypeOid,
values[Anum_pg_type_typtypmod - 1] = Int32GetDatum(typeMod);
values[Anum_pg_type_typndims - 1] = Int32GetDatum(typNDims);
values[Anum_pg_type_typcollation - 1] = ObjectIdGetDatum(typeCollation);
+ values[Anum_pg_type_typsubshandler - 1] = ObjectIdGetDatum(subscriptingHandlerProcedure);
/*
* initialize the default binary value for this type. Check for nulls of
@@ -695,6 +698,14 @@ GenerateTypeDependencies(HeapTuple typeTuple,
/* Normal dependency on the default expression. */
if (defaultExpr)
recordDependencyOnExpr(&myself, defaultExpr, NIL, DEPENDENCY_NORMAL);
+
+ if (OidIsValid(typeForm->typsubshandler))
+ {
+ referenced.classId = ProcedureRelationId;
+ referenced.objectId = typeForm->typsubshandler;
+ referenced.objectSubId = 0;
+ recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ }
}
/*
diff --git a/src/backend/commands/typecmds.c b/src/backend/commands/typecmds.c
index 9e5938b10e..bf1e6fdc5c 100644
--- a/src/backend/commands/typecmds.c
+++ b/src/backend/commands/typecmds.c
@@ -115,6 +115,7 @@ static Oid findTypeSendFunction(List *procname, Oid typeOid);
static Oid findTypeTypmodinFunction(List *procname);
static Oid findTypeTypmodoutFunction(List *procname);
static Oid findTypeAnalyzeFunction(List *procname, Oid typeOid);
+static Oid findTypeSubscriptingFunction(List *procname, Oid typeOid, bool parseFunc);
static Oid findRangeSubOpclass(List *opcname, Oid subtype);
static Oid findRangeCanonicalFunction(List *procname, Oid typeOid);
static Oid findRangeSubtypeDiffFunction(List *procname, Oid subtype);
@@ -148,6 +149,7 @@ DefineType(ParseState *pstate, List *names, List *parameters)
List *typmodinName = NIL;
List *typmodoutName = NIL;
List *analyzeName = NIL;
+ List *subscriptingParseName = NIL;
char category = TYPCATEGORY_USER;
bool preferred = false;
char delimiter = DEFAULT_TYPDELIM;
@@ -166,6 +168,7 @@ DefineType(ParseState *pstate, List *names, List *parameters)
DefElem *typmodinNameEl = NULL;
DefElem *typmodoutNameEl = NULL;
DefElem *analyzeNameEl = NULL;
+ DefElem *subscriptingParseNameEl = NULL;
DefElem *categoryEl = NULL;
DefElem *preferredEl = NULL;
DefElem *delimiterEl = NULL;
@@ -187,6 +190,7 @@ DefineType(ParseState *pstate, List *names, List *parameters)
Oid typoid;
ListCell *pl;
ObjectAddress address;
+ Oid subscriptingParseOid = InvalidOid;
/*
* As of Postgres 8.4, we require superuser privilege to create a base
@@ -287,6 +291,8 @@ DefineType(ParseState *pstate, List *names, List *parameters)
else if (strcmp(defel->defname, "analyze") == 0 ||
strcmp(defel->defname, "analyse") == 0)
defelp = &analyzeNameEl;
+ else if (strcmp(defel->defname, "subscripting_handler") == 0)
+ defelp = &subscriptingParseNameEl;
else if (strcmp(defel->defname, "category") == 0)
defelp = &categoryEl;
else if (strcmp(defel->defname, "preferred") == 0)
@@ -357,6 +363,8 @@ DefineType(ParseState *pstate, List *names, List *parameters)
typmodoutName = defGetQualifiedName(typmodoutNameEl);
if (analyzeNameEl)
analyzeName = defGetQualifiedName(analyzeNameEl);
+ if (subscriptingParseNameEl)
+ subscriptingParseName = defGetQualifiedName(subscriptingParseNameEl);
if (categoryEl)
{
char *p = defGetString(categoryEl);
@@ -481,6 +489,10 @@ DefineType(ParseState *pstate, List *names, List *parameters)
if (analyzeName)
analyzeOid = findTypeAnalyzeFunction(analyzeName, typoid);
+ if (subscriptingParseName)
+ subscriptingParseOid = findTypeSubscriptingFunction(subscriptingParseName,
+ typoid, true);
+
/*
* Check permissions on functions. We choose to require the creator/owner
* of a type to also own the underlying functions. Since creating a type
@@ -562,7 +574,8 @@ DefineType(ParseState *pstate, List *names, List *parameters)
-1, /* typMod (Domains only) */
0, /* Array Dimensions of typbasetype */
false, /* Type NOT NULL */
- collation); /* type's collation */
+ collation, /* type's collation */
+ subscriptingParseOid); /* subscripting procedure */
Assert(typoid == address.objectId);
/*
@@ -603,7 +616,8 @@ DefineType(ParseState *pstate, List *names, List *parameters)
-1, /* typMod (Domains only) */
0, /* Array dimensions of typbasetype */
false, /* Type NOT NULL */
- collation); /* type's collation */
+ collation, /* type's collation */
+ 0);
pfree(array_type);
@@ -666,6 +680,7 @@ DefineDomain(CreateDomainStmt *stmt)
Oid receiveProcedure;
Oid sendProcedure;
Oid analyzeProcedure;
+ Oid subscriptingHandlerProcedure;
bool byValue;
char category;
char delimiter;
@@ -799,6 +814,9 @@ DefineDomain(CreateDomainStmt *stmt)
/* Analysis function */
analyzeProcedure = baseType->typanalyze;
+ /* Subscripting functions */
+ subscriptingHandlerProcedure = baseType->typsubshandler;
+
/* Inherited default value */
datum = SysCacheGetAttr(TYPEOID, typeTup,
Anum_pg_type_typdefault, &isnull);
@@ -1004,7 +1022,8 @@ DefineDomain(CreateDomainStmt *stmt)
basetypeMod, /* typeMod value */
typNDims, /* Array dimensions for base type */
typNotNull, /* Type NOT NULL */
- domaincoll); /* type's collation */
+ domaincoll, /* type's collation */
+ subscriptingHandlerProcedure); /* subscripting procedure */
/*
* Create the array type that goes with it.
@@ -1044,7 +1063,8 @@ DefineDomain(CreateDomainStmt *stmt)
-1, /* typMod (Domains only) */
0, /* Array dimensions of typbasetype */
false, /* Type NOT NULL */
- domaincoll); /* type's collation */
+ domaincoll, /* type's collation */
+ 0); /* array subscripting implementation */
pfree(domainArrayName);
@@ -1159,7 +1179,8 @@ DefineEnum(CreateEnumStmt *stmt)
-1, /* typMod (Domains only) */
0, /* Array dimensions of typbasetype */
false, /* Type NOT NULL */
- InvalidOid); /* type's collation */
+ InvalidOid, /* type's collation */
+ InvalidOid); /* typsubshandler - none */
/* Enter the enum's values into pg_enum */
EnumValuesCreate(enumTypeAddr.objectId, stmt->vals);
@@ -1199,7 +1220,8 @@ DefineEnum(CreateEnumStmt *stmt)
-1, /* typMod (Domains only) */
0, /* Array dimensions of typbasetype */
false, /* Type NOT NULL */
- InvalidOid); /* type's collation */
+ InvalidOid, /* type's collation */
+ 0); /* array subscripting implementation */
pfree(enumArrayName);
@@ -1487,7 +1509,8 @@ DefineRange(CreateRangeStmt *stmt)
-1, /* typMod (Domains only) */
0, /* Array dimensions of typbasetype */
false, /* Type NOT NULL */
- InvalidOid); /* type's collation (ranges never have one) */
+ InvalidOid, /* type's collation (ranges never have one) */
+ InvalidOid); /* typsubshandler - none */
Assert(typoid == InvalidOid || typoid == address.objectId);
typoid = address.objectId;
@@ -1530,7 +1553,8 @@ DefineRange(CreateRangeStmt *stmt)
-1, /* typMod (Domains only) */
0, /* Array dimensions of typbasetype */
false, /* Type NOT NULL */
- InvalidOid); /* typcollation */
+ InvalidOid, /* typcollation */
+ 0); /* array subscripting implementation */
pfree(rangeArrayName);
@@ -1881,6 +1905,43 @@ findTypeAnalyzeFunction(List *procname, Oid typeOid)
return procOid;
}
+static Oid
+findTypeSubscriptingFunction(List *procname, Oid typeOid, bool parseFunc)
+{
+ Oid argList[2];
+ Oid procOid;
+ int nargs;
+
+ if (parseFunc)
+ {
+ /*
+ * Subscripting parse functions always take two INTERNAL arguments and
+ * return INTERNAL.
+ */
+ argList[0] = INTERNALOID;
+ nargs = 1;
+ }
+ else
+ {
+ /*
+ * Subscripting fetch/assign functions always take one typeOid
+ * argument, one INTERNAL argument and return typeOid.
+ */
+ argList[0] = typeOid;
+ argList[1] = INTERNALOID;
+ nargs = 2;
+ }
+
+ procOid = LookupFuncName(procname, nargs, argList, true);
+ if (!OidIsValid(procOid))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_FUNCTION),
+ errmsg("function %s does not exist",
+ func_signature_string(procname, nargs, NIL, argList))));
+
+ return procOid;
+}
+
/*
* Find suitable support functions and opclasses for a range type.
*/
diff --git a/src/backend/executor/execExpr.c b/src/backend/executor/execExpr.c
index 236413f62a..ee1077ebe9 100644
--- a/src/backend/executor/execExpr.c
+++ b/src/backend/executor/execExpr.c
@@ -2545,18 +2545,16 @@ ExecInitSubscriptingRef(ExprEvalStep *scratch, SubscriptingRef *sbsref,
{
bool isAssignment = (sbsref->refassgnexpr != NULL);
SubscriptingRefState *sbsrefstate = palloc0(sizeof(SubscriptingRefState));
- List *adjust_jumps = NIL;
- ListCell *lc;
- int i;
+ List *adjust_jumps = NIL;
+ ListCell *lc;
+ int i;
+ RegProcedure typsubshandler = get_typsubsprocs(sbsref->refcontainertype);
/* Fill constant fields of SubscriptingRefState */
sbsrefstate->isassignment = isAssignment;
sbsrefstate->refelemtype = sbsref->refelemtype;
sbsrefstate->refattrlength = get_typlen(sbsref->refcontainertype);
- get_typlenbyvalalign(sbsref->refelemtype,
- &sbsrefstate->refelemlength,
- &sbsrefstate->refelembyval,
- &sbsrefstate->refelemalign);
+ sbsrefstate->sbsroutines = (SubscriptRoutines *) OidFunctionCall0(typsubshandler);
/*
* Evaluate array input. It's safe to do so into resv/resnull, because we
@@ -2580,19 +2578,6 @@ ExecInitSubscriptingRef(ExprEvalStep *scratch, SubscriptingRef *sbsref,
state->steps_len - 1);
}
- /* Verify subscript list lengths are within limit */
- if (list_length(sbsref->refupperindexpr) > MAXDIM)
- ereport(ERROR,
- (errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
- errmsg("number of array dimensions (%d) exceeds the maximum allowed (%d)",
- list_length(sbsref->refupperindexpr), MAXDIM)));
-
- if (list_length(sbsref->reflowerindexpr) > MAXDIM)
- ereport(ERROR,
- (errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
- errmsg("number of array dimensions (%d) exceeds the maximum allowed (%d)",
- list_length(sbsref->reflowerindexpr), MAXDIM)));
-
/* Evaluate upper subscripts */
i = 0;
foreach(lc, sbsref->refupperindexpr)
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index b812bbacee..838bb4d005 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -3146,8 +3146,8 @@ bool
ExecEvalSubscriptingRef(ExprState *state, ExprEvalStep *op)
{
SubscriptingRefState *sbsrefstate = op->d.sbsref_subscript.state;
- int *indexes;
- int off;
+ Datum *indexes;
+ int off;
/* If any index expr yields NULL, result is NULL or error */
if (sbsrefstate->subscriptnull)
@@ -3167,7 +3167,7 @@ ExecEvalSubscriptingRef(ExprState *state, ExprEvalStep *op)
indexes = sbsrefstate->lowerindex;
off = op->d.sbsref_subscript.off;
- indexes[off] = DatumGetInt32(sbsrefstate->subscriptvalue);
+ indexes[off] = sbsrefstate->subscriptvalue;
return true;
}
@@ -3181,36 +3181,14 @@ void
ExecEvalSubscriptingRefFetch(ExprState *state, ExprEvalStep *op)
{
SubscriptingRefState *sbsrefstate = op->d.sbsref.state;
+ SubscriptRoutines *sbsroutines = sbsrefstate->sbsroutines;
/* Should not get here if source container (or any subscript) is null */
Assert(!(*op->resnull));
- if (sbsrefstate->numlower == 0)
- {
- /* Scalar case */
- *op->resvalue = array_get_element(*op->resvalue,
- sbsrefstate->numupper,
- sbsrefstate->upperindex,
- sbsrefstate->refattrlength,
- sbsrefstate->refelemlength,
- sbsrefstate->refelembyval,
- sbsrefstate->refelemalign,
- op->resnull);
- }
- else
- {
- /* Slice case */
- *op->resvalue = array_get_slice(*op->resvalue,
- sbsrefstate->numupper,
- sbsrefstate->upperindex,
- sbsrefstate->lowerindex,
- sbsrefstate->upperprovided,
- sbsrefstate->lowerprovided,
- sbsrefstate->refattrlength,
- sbsrefstate->refelemlength,
- sbsrefstate->refelembyval,
- sbsrefstate->refelemalign);
- }
+
+ *op->resvalue = sbsroutines->fetch(*op->resvalue, sbsrefstate);
+ *op->resnull = sbsrefstate->resnull;
}
/*
@@ -3223,40 +3201,20 @@ void
ExecEvalSubscriptingRefOld(ExprState *state, ExprEvalStep *op)
{
SubscriptingRefState *sbsrefstate = op->d.sbsref.state;
+ SubscriptRoutines *sbsroutines = sbsrefstate->sbsroutines;
if (*op->resnull)
{
- /* whole array is null, so any element or slice is too */
+ /* whole container is null, so any element or slice is too */
sbsrefstate->prevvalue = (Datum) 0;
sbsrefstate->prevnull = true;
}
- else if (sbsrefstate->numlower == 0)
- {
- /* Scalar case */
- sbsrefstate->prevvalue = array_get_element(*op->resvalue,
- sbsrefstate->numupper,
- sbsrefstate->upperindex,
- sbsrefstate->refattrlength,
- sbsrefstate->refelemlength,
- sbsrefstate->refelembyval,
- sbsrefstate->refelemalign,
- &sbsrefstate->prevnull);
- }
else
{
- /* Slice case */
- /* this is currently unreachable */
- sbsrefstate->prevvalue = array_get_slice(*op->resvalue,
- sbsrefstate->numupper,
- sbsrefstate->upperindex,
- sbsrefstate->lowerindex,
- sbsrefstate->upperprovided,
- sbsrefstate->lowerprovided,
- sbsrefstate->refattrlength,
- sbsrefstate->refelemlength,
- sbsrefstate->refelembyval,
- sbsrefstate->refelemalign);
- sbsrefstate->prevnull = false;
+ sbsrefstate->prevvalue = sbsroutines->fetch(*op->resvalue, sbsrefstate);
+
+ if (sbsrefstate->numlower != 0)
+ sbsrefstate->prevnull = false;
}
}
@@ -3270,59 +3228,11 @@ void
ExecEvalSubscriptingRefAssign(ExprState *state, ExprEvalStep *op)
{
SubscriptingRefState *sbsrefstate = op->d.sbsref_subscript.state;
+ SubscriptRoutines *sbsroutines = sbsrefstate->sbsroutines;
- /*
- * For an assignment to a fixed-length container type, both the original
- * container and the value to be assigned into it must be non-NULL, else
- * we punt and return the original container.
- */
- if (sbsrefstate->refattrlength > 0)
- {
- if (*op->resnull || sbsrefstate->replacenull)
- return;
- }
-
- /*
- * For assignment to varlena arrays, we handle a NULL original array by
- * substituting an empty (zero-dimensional) array; insertion of the new
- * element will result in a singleton array value. It does not matter
- * whether the new element is NULL.
- */
- if (*op->resnull)
- {
- *op->resvalue = PointerGetDatum(construct_empty_array(sbsrefstate->refelemtype));
- *op->resnull = false;
- }
-
- if (sbsrefstate->numlower == 0)
- {
- /* Scalar case */
- *op->resvalue = array_set_element(*op->resvalue,
- sbsrefstate->numupper,
- sbsrefstate->upperindex,
- sbsrefstate->replacevalue,
- sbsrefstate->replacenull,
- sbsrefstate->refattrlength,
- sbsrefstate->refelemlength,
- sbsrefstate->refelembyval,
- sbsrefstate->refelemalign);
- }
- else
- {
- /* Slice case */
- *op->resvalue = array_set_slice(*op->resvalue,
- sbsrefstate->numupper,
- sbsrefstate->upperindex,
- sbsrefstate->lowerindex,
- sbsrefstate->upperprovided,
- sbsrefstate->lowerprovided,
- sbsrefstate->replacevalue,
- sbsrefstate->replacenull,
- sbsrefstate->refattrlength,
- sbsrefstate->refelemlength,
- sbsrefstate->refelembyval,
- sbsrefstate->refelemalign);
- }
+ sbsrefstate->resnull = *op->resnull;
+ *op->resvalue = sbsroutines->assign(*op->resvalue, sbsrefstate);
+ *op->resnull = sbsrefstate->resnull;
}
/*
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 89c409de66..ec9b0dd97f 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -1548,8 +1548,10 @@ _copySubscriptingRef(const SubscriptingRef *from)
COPY_SCALAR_FIELD(refcontainertype);
COPY_SCALAR_FIELD(refelemtype);
+ COPY_SCALAR_FIELD(refassgntype);
COPY_SCALAR_FIELD(reftypmod);
COPY_SCALAR_FIELD(refcollid);
+ COPY_SCALAR_FIELD(refnestedfunc);
COPY_NODE_FIELD(refupperindexpr);
COPY_NODE_FIELD(reflowerindexpr);
COPY_NODE_FIELD(refexpr);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index e3f33c40be..48b436f7f7 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -274,8 +274,10 @@ _equalSubscriptingRef(const SubscriptingRef *a, const SubscriptingRef *b)
{
COMPARE_SCALAR_FIELD(refcontainertype);
COMPARE_SCALAR_FIELD(refelemtype);
+ COMPARE_SCALAR_FIELD(refassgntype);
COMPARE_SCALAR_FIELD(reftypmod);
COMPARE_SCALAR_FIELD(refcollid);
+ COMPARE_SCALAR_FIELD(refnestedfunc);
COMPARE_NODE_FIELD(refupperindexpr);
COMPARE_NODE_FIELD(reflowerindexpr);
COMPARE_NODE_FIELD(refexpr);
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index e2f177515d..70c9736c46 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -1194,8 +1194,10 @@ _outSubscriptingRef(StringInfo str, const SubscriptingRef *node)
WRITE_OID_FIELD(refcontainertype);
WRITE_OID_FIELD(refelemtype);
+ WRITE_OID_FIELD(refassgntype);
WRITE_INT_FIELD(reftypmod);
WRITE_OID_FIELD(refcollid);
+ WRITE_OID_FIELD(refnestedfunc);
WRITE_NODE_FIELD(refupperindexpr);
WRITE_NODE_FIELD(reflowerindexpr);
WRITE_NODE_FIELD(refexpr);
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index 42050ab719..1c9752c771 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -669,8 +669,10 @@ _readSubscriptingRef(void)
READ_OID_FIELD(refcontainertype);
READ_OID_FIELD(refelemtype);
+ READ_OID_FIELD(refassgntype);
READ_INT_FIELD(reftypmod);
READ_OID_FIELD(refcollid);
+ READ_OID_FIELD(refnestedfunc);
READ_NODE_FIELD(refupperindexpr);
READ_NODE_FIELD(reflowerindexpr);
READ_NODE_FIELD(refexpr);
diff --git a/src/backend/parser/parse_expr.c b/src/backend/parser/parse_expr.c
index f69976cc8c..ea9d35429c 100644
--- a/src/backend/parser/parse_expr.c
+++ b/src/backend/parser/parse_expr.c
@@ -433,11 +433,13 @@ unknown_attribute(ParseState *pstate, Node *relref, const char *attname,
static Node *
transformIndirection(ParseState *pstate, A_Indirection *ind)
{
- Node *last_srf = pstate->p_last_srf;
- Node *result = transformExprRecurse(pstate, ind->arg);
- List *subscripts = NIL;
- int location = exprLocation(result);
- ListCell *i;
+ Node *last_srf = pstate->p_last_srf;
+ Node *result = transformExprRecurse(pstate, ind->arg);
+ SubscriptRoutines *sbsroutines;
+ SubscriptingRef *sbsref;
+ List *subscripts = NIL;
+ int location = exprLocation(result);
+ ListCell *i;
/*
* We have to split any field-selection operations apart from
@@ -465,13 +467,20 @@ transformIndirection(ParseState *pstate, A_Indirection *ind)
/* process subscripts before this field selection */
if (subscripts)
- result = (Node *) transformContainerSubscripts(pstate,
- result,
- exprType(result),
- InvalidOid,
- exprTypmod(result),
- subscripts,
- NULL);
+ {
+ sbsref = transformContainerSubscripts(pstate,
+ result,
+ exprType(result),
+ InvalidOid,
+ exprTypmod(result),
+ subscripts,
+ NULL);
+
+ sbsroutines = getSubscriptingRoutines(sbsref->refcontainertype);
+ sbsref = sbsroutines->prepare(false, sbsref);
+ sbsroutines->validate(false, sbsref, pstate);
+ result = (Node *) sbsref;
+ }
subscripts = NIL;
newresult = ParseFuncOrColumn(pstate,
@@ -488,13 +497,20 @@ transformIndirection(ParseState *pstate, A_Indirection *ind)
}
/* process trailing subscripts, if any */
if (subscripts)
- result = (Node *) transformContainerSubscripts(pstate,
- result,
- exprType(result),
- InvalidOid,
- exprTypmod(result),
- subscripts,
- NULL);
+ {
+ sbsref = transformContainerSubscripts(pstate,
+ result,
+ exprType(result),
+ InvalidOid,
+ exprTypmod(result),
+ subscripts,
+ NULL);
+
+ sbsroutines = getSubscriptingRoutines(sbsref->refcontainertype);
+ sbsref = sbsroutines->prepare(false, sbsref);
+ sbsroutines->validate(false, sbsref, pstate);
+ result = (Node *) sbsref;
+ }
return result;
}
diff --git a/src/backend/parser/parse_node.c b/src/backend/parser/parse_node.c
index 6e98fe55fc..4f46d6310a 100644
--- a/src/backend/parser/parse_node.c
+++ b/src/backend/parser/parse_node.c
@@ -184,21 +184,12 @@ pcb_error_callback(void *arg)
* transformContainerType()
* Identify the types involved in a subscripting operation for container
*
- *
- * On entry, containerType/containerTypmod identify the type of the input value
- * to be subscripted (which could be a domain type). These are modified if
- * necessary to identify the actual container type and typmod, and the
- * container's element type is returned. An error is thrown if the input isn't
- * an array type.
+ * On entry, containerType/containerTypmod are modified if necessary to
+ * identify the actual container type and typmod.
*/
-Oid
+void
transformContainerType(Oid *containerType, int32 *containerTypmod)
{
- Oid origContainerType = *containerType;
- Oid elementType;
- HeapTuple type_tuple_container;
- Form_pg_type type_struct_container;
-
/*
* If the input is a domain, smash to base type, and extract the actual
* typmod to be applied to the base type. Subscripting a domain is an
@@ -219,25 +210,6 @@ transformContainerType(Oid *containerType, int32 *containerTypmod)
*containerType = INT2ARRAYOID;
else if (*containerType == OIDVECTOROID)
*containerType = OIDARRAYOID;
-
- /* Get the type tuple for the container */
- type_tuple_container = SearchSysCache1(TYPEOID, ObjectIdGetDatum(*containerType));
- if (!HeapTupleIsValid(type_tuple_container))
- elog(ERROR, "cache lookup failed for type %u", *containerType);
- type_struct_container = (Form_pg_type) GETSTRUCT(type_tuple_container);
-
- /* needn't check typisdefined since this will fail anyway */
-
- elementType = type_struct_container->typelem;
- if (elementType == InvalidOid)
- ereport(ERROR,
- (errcode(ERRCODE_DATATYPE_MISMATCH),
- errmsg("cannot subscript type %s because it is not an array",
- format_type_be(origContainerType))));
-
- ReleaseSysCache(type_tuple_container);
-
- return elementType;
}
/*
@@ -254,10 +226,15 @@ transformContainerType(Oid *containerType, int32 *containerTypmod)
* container. We produce an expression that represents the new container value
* with the source data inserted into the right part of the container.
*
- * For both cases, if the source container is of a domain-over-array type,
- * the result is of the base array type or its element type; essentially,
- * we must fold a domain to its base type before applying subscripting.
- * (Note that int2vector and oidvector are treated as domains here.)
+ * For both cases, this function contains only general subscripting logic while
+ * type-specific logic (e.g. type verifications and coercion) is placed in
+ * separate procedures indicated by typsubshandler. There is only one exception
+ * for now about domain-over-container: if the source container is of a
+ * domain-over-container type, the result is of the base container type or its
+ * element type; essentially, we must fold a domain to its base type before
+ * applying subscripting. (Note that int2vector and oidvector are treated as
+ * domains here.) An error will appear in the case the current container type
+ * doesn't have a subscripting procedure.
*
* pstate Parse state
* containerBase Already-transformed expression for the container as a whole
@@ -284,16 +261,12 @@ transformContainerSubscripts(ParseState *pstate,
bool isSlice = false;
List *upperIndexpr = NIL;
List *lowerIndexpr = NIL;
+ List *indexprSlice = NIL;
ListCell *idx;
SubscriptingRef *sbsref;
- /*
- * Caller may or may not have bothered to determine elementType. Note
- * that if the caller did do so, containerType/containerTypMod must be as
- * modified by transformContainerType, ie, smash domain to base type.
- */
- if (!OidIsValid(elementType))
- elementType = transformContainerType(&containerType, &containerTypMod);
+ /* Identify the actual container type and element type involved */
+ transformContainerType(&containerType, &containerTypMod);
/*
* A list containing only simple subscripts refers to a single container
@@ -327,29 +300,6 @@ transformContainerSubscripts(ParseState *pstate,
if (ai->lidx)
{
subexpr = transformExpr(pstate, ai->lidx, pstate->p_expr_kind);
- /* If it's not int4 already, try to coerce */
- subexpr = coerce_to_target_type(pstate,
- subexpr, exprType(subexpr),
- INT4OID, -1,
- COERCION_ASSIGNMENT,
- COERCE_IMPLICIT_CAST,
- -1);
- if (subexpr == NULL)
- ereport(ERROR,
- (errcode(ERRCODE_DATATYPE_MISMATCH),
- errmsg("array subscript must have type integer"),
- parser_errposition(pstate, exprLocation(ai->lidx))));
- }
- else if (!ai->is_slice)
- {
- /* Make a constant 1 */
- subexpr = (Node *) makeConst(INT4OID,
- -1,
- InvalidOid,
- sizeof(int32),
- Int32GetDatum(1),
- false,
- true); /* pass by value */
}
else
{
@@ -357,63 +307,23 @@ transformContainerSubscripts(ParseState *pstate,
subexpr = NULL;
}
lowerIndexpr = lappend(lowerIndexpr, subexpr);
+ indexprSlice = lappend(indexprSlice, ai);
}
else
Assert(ai->lidx == NULL && !ai->is_slice);
if (ai->uidx)
- {
subexpr = transformExpr(pstate, ai->uidx, pstate->p_expr_kind);
- /* If it's not int4 already, try to coerce */
- subexpr = coerce_to_target_type(pstate,
- subexpr, exprType(subexpr),
- INT4OID, -1,
- COERCION_ASSIGNMENT,
- COERCE_IMPLICIT_CAST,
- -1);
- if (subexpr == NULL)
- ereport(ERROR,
- (errcode(ERRCODE_DATATYPE_MISMATCH),
- errmsg("array subscript must have type integer"),
- parser_errposition(pstate, exprLocation(ai->uidx))));
- }
else
{
/* Slice with omitted upper bound, put NULL into the list */
Assert(isSlice && ai->is_slice);
subexpr = NULL;
}
+ subexpr = transformExpr(pstate, ai->uidx, pstate->p_expr_kind);
upperIndexpr = lappend(upperIndexpr, subexpr);
}
- /*
- * If doing an array store, coerce the source value to the right type.
- * (This should agree with the coercion done by transformAssignedExpr.)
- */
- if (assignFrom != NULL)
- {
- Oid typesource = exprType(assignFrom);
- Oid typeneeded = isSlice ? containerType : elementType;
- Node *newFrom;
-
- newFrom = coerce_to_target_type(pstate,
- assignFrom, typesource,
- typeneeded, containerTypMod,
- COERCION_ASSIGNMENT,
- COERCE_IMPLICIT_CAST,
- -1);
- if (newFrom == NULL)
- ereport(ERROR,
- (errcode(ERRCODE_DATATYPE_MISMATCH),
- errmsg("array assignment requires type %s"
- " but expression is of type %s",
- format_type_be(typeneeded),
- format_type_be(typesource)),
- errhint("You will need to rewrite or cast the expression."),
- parser_errposition(pstate, exprLocation(assignFrom))));
- assignFrom = newFrom;
- }
-
/*
* Ready to build the SubscriptingRef node.
*/
@@ -422,17 +332,30 @@ transformContainerSubscripts(ParseState *pstate,
sbsref->refassgnexpr = (Expr *) assignFrom;
sbsref->refcontainertype = containerType;
- sbsref->refelemtype = elementType;
sbsref->reftypmod = containerTypMod;
/* refcollid will be set by parse_collate.c */
sbsref->refupperindexpr = upperIndexpr;
sbsref->reflowerindexpr = lowerIndexpr;
+ sbsref->refindexprslice = indexprSlice;
sbsref->refexpr = (Expr *) containerBase;
- sbsref->refassgnexpr = (Expr *) assignFrom;
return sbsref;
}
+SubscriptRoutines*
+getSubscriptingRoutines(Oid containerType)
+{
+ RegProcedure typsubshandler = get_typsubsprocs(containerType);
+
+ if (!OidIsValid(typsubshandler))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("cannot subscript type %s because it does not support subscripting",
+ format_type_be(containerType))));
+
+ return (SubscriptRoutines *) OidFunctionCall0(typsubshandler);
+}
+
/*
* make_const
*
diff --git a/src/backend/parser/parse_target.c b/src/backend/parser/parse_target.c
index 566c517837..d7483c6538 100644
--- a/src/backend/parser/parse_target.c
+++ b/src/backend/parser/parse_target.c
@@ -685,7 +685,7 @@ transformAssignmentIndirection(ParseState *pstate,
Node *rhs,
int location)
{
- Node *result;
+ Node *result = NULL;
List *subscripts = NIL;
bool isSlice = false;
ListCell *i;
@@ -848,27 +848,21 @@ transformAssignmentIndirection(ParseState *pstate,
location);
}
- /* base case: just coerce RHS to match target type ID */
-
- result = coerce_to_target_type(pstate,
- rhs, exprType(rhs),
- targetTypeId, targetTypMod,
- COERCION_ASSIGNMENT,
- COERCE_IMPLICIT_CAST,
- -1);
- if (result == NULL)
+ /*
+ * Base case: just coerce RHS to match target type ID.
+ * It's necessary only for field selection, since for
+ * subscripting its custom code should define types.
+ */
+ if (!targetIsSubscripting)
{
- if (targetIsSubscripting)
- ereport(ERROR,
- (errcode(ERRCODE_DATATYPE_MISMATCH),
- errmsg("array assignment to \"%s\" requires type %s"
- " but expression is of type %s",
- targetName,
- format_type_be(targetTypeId),
- format_type_be(exprType(rhs))),
- errhint("You will need to rewrite or cast the expression."),
- parser_errposition(pstate, location)));
- else
+ result = coerce_to_target_type(pstate,
+ rhs, exprType(rhs),
+ targetTypeId, targetTypMod,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+
+ if (result == NULL)
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("subfield \"%s\" is of type %s"
@@ -900,29 +894,42 @@ transformAssignmentSubscripts(ParseState *pstate,
Node *rhs,
int location)
{
- Node *result;
- Oid containerType;
- int32 containerTypMod;
- Oid elementTypeId;
- Oid typeNeeded;
- Oid collationNeeded;
+ Node *result;
+ Oid containerType;
+ int32 containerTypMod;
+ Oid collationNeeded;
+ SubscriptingRef *sbsref;
+ SubscriptRoutines *sbsroutines;
Assert(subscripts != NIL);
/* Identify the actual array type and element type involved */
containerType = targetTypeId;
containerTypMod = targetTypMod;
- elementTypeId = transformContainerType(&containerType, &containerTypMod);
- /* Identify type that RHS must provide */
- typeNeeded = isSlice ? containerType : elementTypeId;
+ /* process subscripts */
+ sbsref = transformContainerSubscripts(pstate,
+ basenode,
+ containerType,
+ exprType(rhs),
+ containerTypMod,
+ subscripts,
+ rhs);
+
+ sbsroutines = getSubscriptingRoutines(sbsref->refcontainertype);
+
+ /*
+ * Let custom code provide necessary information about required types:
+ * refelemtype and refassgntype
+ */
+ sbsref = sbsroutines->prepare(rhs != NULL, sbsref);
/*
* container normally has same collation as elements, but there's an
* exception: we might be subscripting a domain over a container type. In
* that case use collation of the base type.
*/
- if (containerType == targetTypeId)
+ if (sbsref->refcontainertype == containerType)
collationNeeded = targetCollation;
else
collationNeeded = get_typcollation(containerType);
@@ -932,25 +939,22 @@ transformAssignmentSubscripts(ParseState *pstate,
NULL,
targetName,
true,
- typeNeeded,
- containerTypMod,
+ sbsref->refassgntype,
+ sbsref->reftypmod,
collationNeeded,
indirection,
next_indirection,
rhs,
location);
- /* process subscripts */
- result = (Node *) transformContainerSubscripts(pstate,
- basenode,
- containerType,
- elementTypeId,
- containerTypMod,
- subscripts,
- rhs);
+ /* Provide fully prepared subscripting information for custom validation */
+ sbsref->refassgnexpr = (Expr *) rhs;
+ sbsroutines->validate(rhs != NULL, sbsref, pstate);
+
+ result = (Node *) sbsref;
/* If target was a domain over container, need to coerce up to the domain */
- if (containerType != targetTypeId)
+ if (sbsref->refcontainertype != targetTypeId)
{
Oid resulttype = exprType(result);
diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c
index 2cbcb4b85e..fb298c11c2 100644
--- a/src/backend/utils/adt/ruleutils.c
+++ b/src/backend/utils/adt/ruleutils.c
@@ -7994,17 +7994,18 @@ get_rule_expr(Node *node, deparse_context *context,
if (need_parens)
appendStringInfoChar(buf, ')');
- /*
- * If there's a refassgnexpr, we want to print the node in the
- * format "container[subscripts] := refassgnexpr". This is
- * not legal SQL, so decompilation of INSERT or UPDATE
- * statements should always use processIndirection as part of
- * the statement-level syntax. We should only see this when
- * EXPLAIN tries to print the targetlist of a plan resulting
- * from such a statement.
- */
- if (sbsref->refassgnexpr)
+ if (IsAssignment(sbsref))
{
+ /*
+ * If there's a refassgnexpr, we want to print the node in the
+ * format "container[subscripts] := refassgnexpr". This is not
+ * legal SQL, so decompilation of INSERT or UPDATE statements
+ * should always use processIndirection as part of the
+ * statement-level syntax. We should only see this when
+ * EXPLAIN tries to print the targetlist of a plan resulting
+ * from such a statement.
+ */
+
Node *refassgnexpr;
/*
diff --git a/src/backend/utils/cache/lsyscache.c b/src/backend/utils/cache/lsyscache.c
index f3bf413829..fb264a1e57 100644
--- a/src/backend/utils/cache/lsyscache.c
+++ b/src/backend/utils/cache/lsyscache.c
@@ -3273,6 +3273,29 @@ get_range_collation(Oid rangeOid)
return InvalidOid;
}
+/*
+ * get_typsubshandler
+ *
+ * Given the type OID, return the type's subscripting procedures, if any,
+ * through pointers in arguments.
+ */
+RegProcedure
+get_typsubsprocs(Oid typid)
+{
+ HeapTuple tp;
+
+ tp = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typid));
+ if (HeapTupleIsValid(tp))
+ {
+ RegProcedure handler = ((Form_pg_type) GETSTRUCT(tp))->typsubshandler;
+ ReleaseSysCache(tp);
+
+ return handler;
+ }
+ else
+ return InvalidOid;
+}
+
/* ---------- PG_INDEX CACHE ---------- */
/*
diff --git a/src/include/c.h b/src/include/c.h
index f242e32edb..108e424df7 100644
--- a/src/include/c.h
+++ b/src/include/c.h
@@ -545,6 +545,8 @@ typedef struct
int indx[MAXDIM];
} IntArray;
+#define MAX_SUBSCRIPT_DEPTH 12
+
/* ----------------
* Variable-length datatypes all share the 'struct varlena' header.
*
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..ea237ec61d 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -221,6 +221,12 @@ CATALOG(pg_type,1247,TypeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(71,TypeRelati
*/
Oid typcollation BKI_DEFAULT(0) BKI_LOOKUP(pg_collation);
+ /*
+ * Type specific subscripting logic. If typsubshandler is NULL, it means
+ * that this type doesn't support subscripting.
+ */
+ regproc typsubshandler BKI_DEFAULT(-) BKI_LOOKUP(pg_proc);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/*
@@ -349,7 +355,8 @@ extern ObjectAddress TypeCreate(Oid newTypeOid,
int32 typeMod,
int32 typNDims,
bool typeNotNull,
- Oid typeCollation);
+ Oid typeCollation,
+ Oid subscriptingParseProcedure);
extern void GenerateTypeDependencies(HeapTuple typeTuple,
Relation typeCatalog,
diff --git a/src/include/executor/execExpr.h b/src/include/executor/execExpr.h
index dbe8649a57..52c357b2aa 100644
--- a/src/include/executor/execExpr.h
+++ b/src/include/executor/execExpr.h
@@ -19,7 +19,7 @@
/* forward references to avoid circularity */
struct ExprEvalStep;
-struct SubscriptingRefState;
+struct SubscriptRoutines;
/* Bits in ExprState->flags (see also execnodes.h for public flag bits): */
/* expression's interpreter has been initialized */
@@ -658,13 +658,13 @@ typedef struct SubscriptingRefState
/* numupper and upperprovided[] are filled at compile time */
/* at runtime, extracted subscript datums get stored in upperindex[] */
int numupper;
- bool upperprovided[MAXDIM];
- int upperindex[MAXDIM];
+ bool upperprovided[MAX_SUBSCRIPT_DEPTH];
+ Datum upperindex[MAX_SUBSCRIPT_DEPTH];
/* similarly for lower indexes, if any */
int numlower;
- bool lowerprovided[MAXDIM];
- int lowerindex[MAXDIM];
+ bool lowerprovided[MAX_SUBSCRIPT_DEPTH];
+ Datum lowerindex[MAX_SUBSCRIPT_DEPTH];
/* subscript expressions get evaluated into here */
Datum subscriptvalue;
@@ -677,6 +677,9 @@ typedef struct SubscriptingRefState
/* if we have a nested assignment, SBSREF_OLD puts old value here */
Datum prevvalue;
bool prevnull;
+
+ bool resnull;
+ struct SubscriptRoutines *sbsroutines;
} SubscriptingRefState;
diff --git a/src/include/nodes/primnodes.h b/src/include/nodes/primnodes.h
index d73be2ad46..5991f437cd 100644
--- a/src/include/nodes/primnodes.h
+++ b/src/include/nodes/primnodes.h
@@ -417,13 +417,17 @@ typedef struct SubscriptingRef
Expr xpr;
Oid refcontainertype; /* type of the container proper */
Oid refelemtype; /* type of the container elements */
+ Oid refassgntype; /* type of assignment expr that is required */
int32 reftypmod; /* typmod of the container (and elements too) */
Oid refcollid; /* OID of collation, or InvalidOid if none */
+ Oid refnestedfunc; /* OID of type-specific function to handle nested assignment */
List *refupperindexpr; /* expressions that evaluate to upper
* container indexes */
List *reflowerindexpr; /* expressions that evaluate to lower
* container indexes, or NIL for single
* container element */
+ List *refindexprslice; /* whether or not related indexpr from
+ * reflowerindexpr is a slice */
Expr *refexpr; /* the expression that evaluates to a
* container value */
@@ -431,6 +435,8 @@ typedef struct SubscriptingRef
* fetch */
} SubscriptingRef;
+#define IsAssignment(expr) ( ((SubscriptingRef*) expr)->refassgnexpr != NULL )
+
/*
* CoercionContext - distinguishes the allowed set of type casts
*
diff --git a/src/include/nodes/subscripting.h b/src/include/nodes/subscripting.h
new file mode 100644
index 0000000000..1800d5ecf5
--- /dev/null
+++ b/src/include/nodes/subscripting.h
@@ -0,0 +1,42 @@
+/*-------------------------------------------------------------------------
+ *
+ * subscripting.h
+ * API for generic type subscripting
+ *
+ * Portions Copyright (c) 1996-2018, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/nodes/subscripting.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef SUBSCRIPTING_H
+#define SUBSCRIPTING_H
+
+#include "parser/parse_node.h"
+#include "nodes/primnodes.h"
+
+struct ParseState;
+struct SubscriptingRefState;
+
+/* Callback function signatures --- see xsubscripting.sgml for more info. */
+typedef SubscriptingRef * (*SubscriptingPrepare) (bool isAssignment, SubscriptingRef *sbsef);
+
+typedef SubscriptingRef * (*SubscriptingValidate) (bool isAssignment, SubscriptingRef *sbsef,
+ struct ParseState *pstate);
+
+typedef Datum (*SubscriptingFetch) (Datum source, struct SubscriptingRefState *sbsrefstate);
+
+typedef Datum (*SubscriptingAssign) (Datum source, struct SubscriptingRefState *sbrsefstate);
+
+typedef struct SubscriptRoutines
+{
+ SubscriptingPrepare prepare;
+ SubscriptingValidate validate;
+ SubscriptingFetch fetch;
+ SubscriptingAssign assign;
+
+} SubscriptRoutines;
+
+
+#endif /* SUBSCRIPTING_H */
diff --git a/src/include/parser/parse_node.h b/src/include/parser/parse_node.h
index d25819aa28..b4736206d1 100644
--- a/src/include/parser/parse_node.h
+++ b/src/include/parser/parse_node.h
@@ -15,6 +15,7 @@
#define PARSE_NODE_H
#include "nodes/parsenodes.h"
+#include "nodes/subscripting.h"
#include "utils/queryenvironment.h"
#include "utils/relcache.h"
@@ -313,7 +314,9 @@ extern void setup_parser_errposition_callback(ParseCallbackState *pcbstate,
ParseState *pstate, int location);
extern void cancel_parser_errposition_callback(ParseCallbackState *pcbstate);
-extern Oid transformContainerType(Oid *containerType, int32 *containerTypmod);
+extern Var *make_var(ParseState *pstate, RangeTblEntry *rte, int attrno,
+ int location);
+extern void transformContainerType(Oid *containerType, int32 *containerTypmod);
extern SubscriptingRef *transformContainerSubscripts(ParseState *pstate,
Node *containerBase,
@@ -322,6 +325,7 @@ extern SubscriptingRef *transformContainerSubscripts(ParseState *pstate,
int32 containerTypMod,
List *indirection,
Node *assignFrom);
+extern SubscriptRoutines *getSubscriptingRoutines(Oid containerType);
extern Const *make_const(ParseState *pstate, Value *value, int location);
#endif /* PARSE_NODE_H */
diff --git a/src/include/utils/lsyscache.h b/src/include/utils/lsyscache.h
index fecfe1f4f6..8fc570d2e1 100644
--- a/src/include/utils/lsyscache.h
+++ b/src/include/utils/lsyscache.h
@@ -183,6 +183,7 @@ extern char *get_namespace_name(Oid nspid);
extern char *get_namespace_name_or_temp(Oid nspid);
extern Oid get_range_subtype(Oid rangeOid);
extern Oid get_range_collation(Oid rangeOid);
+extern RegProcedure get_typsubsprocs(Oid typid);
extern Oid get_index_column_opclass(Oid index_oid, int attno);
extern bool get_index_isreplident(Oid index_oid);
extern bool get_index_isvalid(Oid index_oid);
--
2.21.0
--u66dlpg6gaevsofr
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v33-0002-Subscripting-for-array.patch"
^ permalink raw reply [nested|flat] 3+ messages in thread
* [BUG]: the walsender does not update its IO statistics until it exits
@ 2025-02-25 13:42 Bertrand Drouvot <[email protected]>
0 siblings, 1 reply; 3+ messages in thread
From: Bertrand Drouvot @ 2025-02-25 13:42 UTC (permalink / raw)
To: [email protected]
Hi hackers,
while doing some tests for [1], I observed that $SUBJECT.
To observe this behavior on master:
1. create a logical replication slot
postgres=# SELECT * FROM pg_create_logical_replication_slot('logical_slot', 'test_decoding', false, true);
slot_name | lsn
--------------+------------
logical_slot | 0/40749508
(1 row)
2. create a table and add some data
postgres=# create table bdt (a int);
CREATE TABLE
postgres=# insert into bdt select a from generate_series(1,10000) a ;
INSERT 0 10000
3. starts pg_recvlogical that way
pg_recvlogical -d postgres -S logical_slot -f - --no-loop --start
4. query pg_stat_io
postgres=# select backend_type,object,context,reads,read_bytes from pg_stat_io where backend_type = 'walsender';
backend_type | object | context | reads | read_bytes
--------------+---------------+-----------+-------+------------
walsender | relation | bulkread | 0 | 0
walsender | relation | bulkwrite | 0 | 0
walsender | relation | init | 0 | 0
walsender | relation | normal | 6 | 49152
walsender | relation | vacuum | 0 | 0
walsender | temp relation | normal | 0 | 0
walsender | wal | init | |
walsender | wal | normal | 0 | 0
(8 rows)
The non zeros stats that we see here are due to the pgstat_report_stat() call in
PostgresMain() but not to the walsender decoding activity per say (proof is that
you can see that the wal object values are empty while it certainly had to read
some WAL).
5. Once ctrl-c is done for pg_recvlogical then we get:
postgres=# select backend_type,object,context,reads,read_bytes from pg_stat_io where backend_type = 'walsender';
backend_type | object | context | reads | read_bytes
--------------+---------------+-----------+-------+------------
walsender | relation | bulkread | 0 | 0
walsender | relation | bulkwrite | 0 | 0
walsender | relation | init | 0 | 0
walsender | relation | normal | 9 | 73728
walsender | relation | vacuum | 0 | 0
walsender | temp relation | normal | 0 | 0
walsender | wal | init | |
walsender | wal | normal | 98 | 793856
(8 rows)
Now we can see that the numbers increased for the relation object and that we
get non zeros numbers for the wal object too (which makes fully sense).
With the attached patch applied, we would get the same numbers already in
step 4. (means the stats are flushed without the need to wait for the walsender
to exit).
Remarks:
R1. The extra flush are done in WalSndLoop(): I believe this is the right place
for them.
R2. A test is added in 035_standby_logical_decoding.pl: while this TAP test
is already racy ([2]) that looks like a good place as we don't want pg_recvlogical
to stop/exit.
R3. The test can also be back-patched till 16_STABLE as 035_standby_logical_decoding.pl
has been introduced in 16 (and so do pg_stat_io).
R4. The test fails if the extra flushs are not applied/done, which makes fully
sense.
[1]: https://www.postgresql.org/message-id/flat/Z3zqc4o09dM/Ezyz%40ip-10-97-1-34.eu-west-3.compute.intern...
[2]: https://www.postgresql.org/message-id/Z6oRgmD8m7zBo732%40ip-10-97-1-34.eu-west-3.compute.internal
Looking forward to your feedback,
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
Attachments:
[text/x-diff] v1-0001-Flush-the-IO-statistics-of-active-walsenders.patch (2.8K, ../../[email protected]/2-v1-0001-Flush-the-IO-statistics-of-active-walsenders.patch)
download | inline diff:
From b16a0ccada3c2acc80bf733e71935677b9ab6382 Mon Sep 17 00:00:00 2001
From: Bertrand Drouvot <[email protected]>
Date: Tue, 25 Feb 2025 10:18:05 +0000
Subject: [PATCH v1] Flush the IO statistics of active walsenders
The walsender does not flush its IO statistics until it exits.
The issue is there since pg_stat_io has been introduced in a9c70b46dbe.
This commits:
1. ensures it does not wait to exit to flush its IO statistics
2. adds a test
---
src/backend/replication/walsender.c | 7 +++++++
.../recovery/t/035_standby_logical_decoding.pl | 15 +++++++++++++++
2 files changed, 22 insertions(+)
26.2% src/backend/replication/
73.7% src/test/recovery/t/
diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c
index 446d10c1a7d..3664d7d0c75 100644
--- a/src/backend/replication/walsender.c
+++ b/src/backend/replication/walsender.c
@@ -90,6 +90,7 @@
#include "utils/guc.h"
#include "utils/memutils.h"
#include "utils/pg_lsn.h"
+#include "utils/pgstat_internal.h"
#include "utils/ps_status.h"
#include "utils/timeout.h"
#include "utils/timestamp.h"
@@ -2793,6 +2794,12 @@ WalSndLoop(WalSndSendDataCallback send_data)
if (pq_flush_if_writable() != 0)
WalSndShutdown();
+ /*
+ * Report IO statistics
+ */
+ pgstat_flush_io(false);
+ (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO);
+
/* If nothing remains to be sent right now ... */
if (WalSndCaughtUp && !pq_is_send_pending())
{
diff --git a/src/test/recovery/t/035_standby_logical_decoding.pl b/src/test/recovery/t/035_standby_logical_decoding.pl
index 8903177d883..ab34d290165 100644
--- a/src/test/recovery/t/035_standby_logical_decoding.pl
+++ b/src/test/recovery/t/035_standby_logical_decoding.pl
@@ -543,6 +543,9 @@ $node_subscriber->stop;
reactive_slots_change_hfs_and_wait_for_xmins('behaves_ok_', 'vacuum_full_',
0, 1);
+# To check that an active walsender updates its IO statistics below.
+$node_standby->safe_psql('testdb', "SELECT pg_stat_reset_shared('io')");
+
# Ensure that replication slot stats are not empty before triggering the
# conflict.
$node_primary->safe_psql('testdb',
@@ -552,6 +555,18 @@ $node_standby->poll_query_until('testdb',
qq[SELECT total_txns > 0 FROM pg_stat_replication_slots WHERE slot_name = 'vacuum_full_activeslot']
) or die "replication slot stats of vacuum_full_activeslot not updated";
+# Ensure an active walsender updates its IO statistics.
+is( $node_standby->safe_psql(
+ 'testdb',
+ qq(SELECT reads > 0
+ FROM pg_catalog.pg_stat_io
+ WHERE backend_type = 'walsender'
+ AND object = 'relation'
+ AND context = 'normal')
+ ),
+ qq(t),
+ "Check that an active walsender updates its IO statistics");
+
# This should trigger the conflict
wait_until_vacuum_can_remove(
'full', 'CREATE TABLE conflict_test(x integer, y text);
--
2.34.1
^ permalink raw reply [nested|flat] 3+ messages in thread
* Re: [BUG]: the walsender does not update its IO statistics until it exits
@ 2025-02-26 06:37 Michael Paquier <[email protected]>
parent: Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 3+ messages in thread
From: Michael Paquier @ 2025-02-26 06:37 UTC (permalink / raw)
To: Bertrand Drouvot <[email protected]>; +Cc: [email protected]
On Tue, Feb 25, 2025 at 01:42:08PM +0000, Bertrand Drouvot wrote:
> Now we can see that the numbers increased for the relation object and that we
> get non zeros numbers for the wal object too (which makes fully sense).
>
> With the attached patch applied, we would get the same numbers already in
> step 4. (means the stats are flushed without the need to wait for the walsender
> to exit).
@@ -2793,6 +2794,12 @@ WalSndLoop(WalSndSendDataCallback send_data)
if (pq_flush_if_writable() != 0)
WalSndShutdown();
+ /*
+ * Report IO statistics
+ */
+ pgstat_flush_io(false);
+ (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO);
+
/* If nothing remains to be sent right now ... */
if (WalSndCaughtUp && !pq_is_send_pending())
{
That's bad, worse for a logical WAL sender, because it means that we
have no idea what kind of I/O happens in this process until it exits,
and logical WAL senders could loop forever, since v16 where we've
begun tracking I/O.
A non-forced periodic flush like you are proposing here sounds OK to
me, but the position of the flush could be positioned better in the
loop. If there is a SIGUSR2 (aka got_SIGUSR2 is true), WAL senders
would shut down, so it seems rather pointless to do a flush just
before exiting the process in WalSndDone(), no? I'd suggest to move
the flush attempt closer to where we wait for some activity, just
after WalSndKeepaliveIfNecessary().
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 3+ messages in thread
end of thread, other threads:[~2025-02-26 06:37 UTC | newest]
Thread overview: 3+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2019-01-31 21:37 [PATCH v33 1/5] Base implementation of subscripting mechanism erthalion <[email protected]>
2025-02-25 13:42 [BUG]: the walsender does not update its IO statistics until it exits Bertrand Drouvot <[email protected]>
2025-02-26 06:37 ` Re: [BUG]: the walsender does not update its IO statistics until it exits Michael Paquier <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox