public inbox for [email protected]help / color / mirror / Atom feed
[PATCH v33 1/5] Base implementation of subscripting mechanism 4+ messages / 4 participants [nested] [flat]
* [PATCH v33 1/5] Base implementation of subscripting mechanism @ 2019-01-31 21:37 erthalion <[email protected]> 0 siblings, 0 replies; 4+ messages in thread From: erthalion @ 2019-01-31 21:37 UTC (permalink / raw) Introduce all the required machinery for generalizing subscripting operation for a different data types: * subscripting handler procedure, to set up a relation between data type and corresponding subscripting logic. * subscripting routines, that help generalize a subscripting logic, since it involves few stages, namely preparation (e.g. to figure out required types), validation (to check the input and return meaningful error message), fetch (executed when we extract a value using subscripting), assign (executed when we update a data type with a new value using subscripting). Without this it would be neccessary to introduce more new fields to pg_type, which would be too invasive. All ArrayRef related logic was removed and landed as a separate subscripting implementation in the following patch from this series. The rest of the code was rearranged, to e.g. store a type of assigned value for an assign operation. Reviewed-by: Tom Lane, Arthur Zakirov --- .../pg_stat_statements/pg_stat_statements.c | 1 + src/backend/catalog/heap.c | 6 +- src/backend/catalog/pg_type.c | 15 +- src/backend/commands/typecmds.c | 77 +++++++++- src/backend/executor/execExpr.c | 25 +--- src/backend/executor/execExprInterp.c | 124 +++------------ src/backend/nodes/copyfuncs.c | 2 + src/backend/nodes/equalfuncs.c | 2 + src/backend/nodes/outfuncs.c | 2 + src/backend/nodes/readfuncs.c | 2 + src/backend/parser/parse_expr.c | 54 ++++--- src/backend/parser/parse_node.c | 141 ++++-------------- src/backend/parser/parse_target.c | 88 +++++------ src/backend/utils/adt/ruleutils.c | 21 +-- src/backend/utils/cache/lsyscache.c | 23 +++ src/include/c.h | 2 + src/include/catalog/pg_type.h | 9 +- src/include/executor/execExpr.h | 13 +- src/include/nodes/primnodes.h | 6 + src/include/nodes/subscripting.h | 42 ++++++ src/include/parser/parse_node.h | 6 +- src/include/utils/lsyscache.h | 1 + 22 files changed, 336 insertions(+), 326 deletions(-) create mode 100644 src/include/nodes/subscripting.h diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c index 14cad19afb..bf19507d32 100644 --- a/contrib/pg_stat_statements/pg_stat_statements.c +++ b/contrib/pg_stat_statements/pg_stat_statements.c @@ -2793,6 +2793,7 @@ JumbleExpr(pgssJumbleState *jstate, Node *node) JumbleExpr(jstate, (Node *) sbsref->reflowerindexpr); JumbleExpr(jstate, (Node *) sbsref->refexpr); JumbleExpr(jstate, (Node *) sbsref->refassgnexpr); + APP_JUMB(sbsref->refnestedfunc); } break; case T_FuncExpr: diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c index 3985326df6..911e2a1ffe 100644 --- a/src/backend/catalog/heap.c +++ b/src/backend/catalog/heap.c @@ -1056,7 +1056,8 @@ AddNewRelationType(const char *typeName, -1, /* typmod */ 0, /* array dimensions for typBaseType */ false, /* Type NOT NULL */ - InvalidOid); /* rowtypes never have a collation */ + InvalidOid, /* rowtypes never have a collation */ + InvalidOid); /* typsubshandler - none */ } /* -------------------------------- @@ -1335,7 +1336,8 @@ heap_create_with_catalog(const char *relname, -1, /* typmod */ 0, /* array dimensions for typBaseType */ false, /* Type NOT NULL */ - InvalidOid); /* rowtypes never have a collation */ + InvalidOid, /* rowtypes never have a collation */ + 0); /* array implementation */ pfree(relarrayname); } diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c index 79ffe317dd..bf353878f5 100644 --- a/src/backend/catalog/pg_type.c +++ b/src/backend/catalog/pg_type.c @@ -118,6 +118,7 @@ TypeShellMake(const char *typeName, Oid typeNamespace, Oid ownerId) values[Anum_pg_type_typtypmod - 1] = Int32GetDatum(-1); values[Anum_pg_type_typndims - 1] = Int32GetDatum(0); values[Anum_pg_type_typcollation - 1] = ObjectIdGetDatum(InvalidOid); + values[Anum_pg_type_typsubshandler - 1] = ObjectIdGetDatum(InvalidOid); nulls[Anum_pg_type_typdefaultbin - 1] = true; nulls[Anum_pg_type_typdefault - 1] = true; nulls[Anum_pg_type_typacl - 1] = true; @@ -158,10 +159,10 @@ TypeShellMake(const char *typeName, Oid typeNamespace, Oid ownerId) GenerateTypeDependencies(tup, pg_type_desc, NULL, - NULL, 0, false, false, + InvalidOid, false); /* Post creation hook for new shell type */ @@ -219,7 +220,8 @@ TypeCreate(Oid newTypeOid, int32 typeMod, int32 typNDims, /* Array dimensions for baseType */ bool typeNotNull, - Oid typeCollation) + Oid typeCollation, + Oid subscriptingHandlerProcedure) { Relation pg_type_desc; Oid typeObjectId; @@ -372,6 +374,7 @@ TypeCreate(Oid newTypeOid, values[Anum_pg_type_typtypmod - 1] = Int32GetDatum(typeMod); values[Anum_pg_type_typndims - 1] = Int32GetDatum(typNDims); values[Anum_pg_type_typcollation - 1] = ObjectIdGetDatum(typeCollation); + values[Anum_pg_type_typsubshandler - 1] = ObjectIdGetDatum(subscriptingHandlerProcedure); /* * initialize the default binary value for this type. Check for nulls of @@ -695,6 +698,14 @@ GenerateTypeDependencies(HeapTuple typeTuple, /* Normal dependency on the default expression. */ if (defaultExpr) recordDependencyOnExpr(&myself, defaultExpr, NIL, DEPENDENCY_NORMAL); + + if (OidIsValid(typeForm->typsubshandler)) + { + referenced.classId = ProcedureRelationId; + referenced.objectId = typeForm->typsubshandler; + referenced.objectSubId = 0; + recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL); + } } /* diff --git a/src/backend/commands/typecmds.c b/src/backend/commands/typecmds.c index 9e5938b10e..bf1e6fdc5c 100644 --- a/src/backend/commands/typecmds.c +++ b/src/backend/commands/typecmds.c @@ -115,6 +115,7 @@ static Oid findTypeSendFunction(List *procname, Oid typeOid); static Oid findTypeTypmodinFunction(List *procname); static Oid findTypeTypmodoutFunction(List *procname); static Oid findTypeAnalyzeFunction(List *procname, Oid typeOid); +static Oid findTypeSubscriptingFunction(List *procname, Oid typeOid, bool parseFunc); static Oid findRangeSubOpclass(List *opcname, Oid subtype); static Oid findRangeCanonicalFunction(List *procname, Oid typeOid); static Oid findRangeSubtypeDiffFunction(List *procname, Oid subtype); @@ -148,6 +149,7 @@ DefineType(ParseState *pstate, List *names, List *parameters) List *typmodinName = NIL; List *typmodoutName = NIL; List *analyzeName = NIL; + List *subscriptingParseName = NIL; char category = TYPCATEGORY_USER; bool preferred = false; char delimiter = DEFAULT_TYPDELIM; @@ -166,6 +168,7 @@ DefineType(ParseState *pstate, List *names, List *parameters) DefElem *typmodinNameEl = NULL; DefElem *typmodoutNameEl = NULL; DefElem *analyzeNameEl = NULL; + DefElem *subscriptingParseNameEl = NULL; DefElem *categoryEl = NULL; DefElem *preferredEl = NULL; DefElem *delimiterEl = NULL; @@ -187,6 +190,7 @@ DefineType(ParseState *pstate, List *names, List *parameters) Oid typoid; ListCell *pl; ObjectAddress address; + Oid subscriptingParseOid = InvalidOid; /* * As of Postgres 8.4, we require superuser privilege to create a base @@ -287,6 +291,8 @@ DefineType(ParseState *pstate, List *names, List *parameters) else if (strcmp(defel->defname, "analyze") == 0 || strcmp(defel->defname, "analyse") == 0) defelp = &analyzeNameEl; + else if (strcmp(defel->defname, "subscripting_handler") == 0) + defelp = &subscriptingParseNameEl; else if (strcmp(defel->defname, "category") == 0) defelp = &categoryEl; else if (strcmp(defel->defname, "preferred") == 0) @@ -357,6 +363,8 @@ DefineType(ParseState *pstate, List *names, List *parameters) typmodoutName = defGetQualifiedName(typmodoutNameEl); if (analyzeNameEl) analyzeName = defGetQualifiedName(analyzeNameEl); + if (subscriptingParseNameEl) + subscriptingParseName = defGetQualifiedName(subscriptingParseNameEl); if (categoryEl) { char *p = defGetString(categoryEl); @@ -481,6 +489,10 @@ DefineType(ParseState *pstate, List *names, List *parameters) if (analyzeName) analyzeOid = findTypeAnalyzeFunction(analyzeName, typoid); + if (subscriptingParseName) + subscriptingParseOid = findTypeSubscriptingFunction(subscriptingParseName, + typoid, true); + /* * Check permissions on functions. We choose to require the creator/owner * of a type to also own the underlying functions. Since creating a type @@ -562,7 +574,8 @@ DefineType(ParseState *pstate, List *names, List *parameters) -1, /* typMod (Domains only) */ 0, /* Array Dimensions of typbasetype */ false, /* Type NOT NULL */ - collation); /* type's collation */ + collation, /* type's collation */ + subscriptingParseOid); /* subscripting procedure */ Assert(typoid == address.objectId); /* @@ -603,7 +616,8 @@ DefineType(ParseState *pstate, List *names, List *parameters) -1, /* typMod (Domains only) */ 0, /* Array dimensions of typbasetype */ false, /* Type NOT NULL */ - collation); /* type's collation */ + collation, /* type's collation */ + 0); pfree(array_type); @@ -666,6 +680,7 @@ DefineDomain(CreateDomainStmt *stmt) Oid receiveProcedure; Oid sendProcedure; Oid analyzeProcedure; + Oid subscriptingHandlerProcedure; bool byValue; char category; char delimiter; @@ -799,6 +814,9 @@ DefineDomain(CreateDomainStmt *stmt) /* Analysis function */ analyzeProcedure = baseType->typanalyze; + /* Subscripting functions */ + subscriptingHandlerProcedure = baseType->typsubshandler; + /* Inherited default value */ datum = SysCacheGetAttr(TYPEOID, typeTup, Anum_pg_type_typdefault, &isnull); @@ -1004,7 +1022,8 @@ DefineDomain(CreateDomainStmt *stmt) basetypeMod, /* typeMod value */ typNDims, /* Array dimensions for base type */ typNotNull, /* Type NOT NULL */ - domaincoll); /* type's collation */ + domaincoll, /* type's collation */ + subscriptingHandlerProcedure); /* subscripting procedure */ /* * Create the array type that goes with it. @@ -1044,7 +1063,8 @@ DefineDomain(CreateDomainStmt *stmt) -1, /* typMod (Domains only) */ 0, /* Array dimensions of typbasetype */ false, /* Type NOT NULL */ - domaincoll); /* type's collation */ + domaincoll, /* type's collation */ + 0); /* array subscripting implementation */ pfree(domainArrayName); @@ -1159,7 +1179,8 @@ DefineEnum(CreateEnumStmt *stmt) -1, /* typMod (Domains only) */ 0, /* Array dimensions of typbasetype */ false, /* Type NOT NULL */ - InvalidOid); /* type's collation */ + InvalidOid, /* type's collation */ + InvalidOid); /* typsubshandler - none */ /* Enter the enum's values into pg_enum */ EnumValuesCreate(enumTypeAddr.objectId, stmt->vals); @@ -1199,7 +1220,8 @@ DefineEnum(CreateEnumStmt *stmt) -1, /* typMod (Domains only) */ 0, /* Array dimensions of typbasetype */ false, /* Type NOT NULL */ - InvalidOid); /* type's collation */ + InvalidOid, /* type's collation */ + 0); /* array subscripting implementation */ pfree(enumArrayName); @@ -1487,7 +1509,8 @@ DefineRange(CreateRangeStmt *stmt) -1, /* typMod (Domains only) */ 0, /* Array dimensions of typbasetype */ false, /* Type NOT NULL */ - InvalidOid); /* type's collation (ranges never have one) */ + InvalidOid, /* type's collation (ranges never have one) */ + InvalidOid); /* typsubshandler - none */ Assert(typoid == InvalidOid || typoid == address.objectId); typoid = address.objectId; @@ -1530,7 +1553,8 @@ DefineRange(CreateRangeStmt *stmt) -1, /* typMod (Domains only) */ 0, /* Array dimensions of typbasetype */ false, /* Type NOT NULL */ - InvalidOid); /* typcollation */ + InvalidOid, /* typcollation */ + 0); /* array subscripting implementation */ pfree(rangeArrayName); @@ -1881,6 +1905,43 @@ findTypeAnalyzeFunction(List *procname, Oid typeOid) return procOid; } +static Oid +findTypeSubscriptingFunction(List *procname, Oid typeOid, bool parseFunc) +{ + Oid argList[2]; + Oid procOid; + int nargs; + + if (parseFunc) + { + /* + * Subscripting parse functions always take two INTERNAL arguments and + * return INTERNAL. + */ + argList[0] = INTERNALOID; + nargs = 1; + } + else + { + /* + * Subscripting fetch/assign functions always take one typeOid + * argument, one INTERNAL argument and return typeOid. + */ + argList[0] = typeOid; + argList[1] = INTERNALOID; + nargs = 2; + } + + procOid = LookupFuncName(procname, nargs, argList, true); + if (!OidIsValid(procOid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("function %s does not exist", + func_signature_string(procname, nargs, NIL, argList)))); + + return procOid; +} + /* * Find suitable support functions and opclasses for a range type. */ diff --git a/src/backend/executor/execExpr.c b/src/backend/executor/execExpr.c index 236413f62a..ee1077ebe9 100644 --- a/src/backend/executor/execExpr.c +++ b/src/backend/executor/execExpr.c @@ -2545,18 +2545,16 @@ ExecInitSubscriptingRef(ExprEvalStep *scratch, SubscriptingRef *sbsref, { bool isAssignment = (sbsref->refassgnexpr != NULL); SubscriptingRefState *sbsrefstate = palloc0(sizeof(SubscriptingRefState)); - List *adjust_jumps = NIL; - ListCell *lc; - int i; + List *adjust_jumps = NIL; + ListCell *lc; + int i; + RegProcedure typsubshandler = get_typsubsprocs(sbsref->refcontainertype); /* Fill constant fields of SubscriptingRefState */ sbsrefstate->isassignment = isAssignment; sbsrefstate->refelemtype = sbsref->refelemtype; sbsrefstate->refattrlength = get_typlen(sbsref->refcontainertype); - get_typlenbyvalalign(sbsref->refelemtype, - &sbsrefstate->refelemlength, - &sbsrefstate->refelembyval, - &sbsrefstate->refelemalign); + sbsrefstate->sbsroutines = (SubscriptRoutines *) OidFunctionCall0(typsubshandler); /* * Evaluate array input. It's safe to do so into resv/resnull, because we @@ -2580,19 +2578,6 @@ ExecInitSubscriptingRef(ExprEvalStep *scratch, SubscriptingRef *sbsref, state->steps_len - 1); } - /* Verify subscript list lengths are within limit */ - if (list_length(sbsref->refupperindexpr) > MAXDIM) - ereport(ERROR, - (errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED), - errmsg("number of array dimensions (%d) exceeds the maximum allowed (%d)", - list_length(sbsref->refupperindexpr), MAXDIM))); - - if (list_length(sbsref->reflowerindexpr) > MAXDIM) - ereport(ERROR, - (errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED), - errmsg("number of array dimensions (%d) exceeds the maximum allowed (%d)", - list_length(sbsref->reflowerindexpr), MAXDIM))); - /* Evaluate upper subscripts */ i = 0; foreach(lc, sbsref->refupperindexpr) diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c index b812bbacee..838bb4d005 100644 --- a/src/backend/executor/execExprInterp.c +++ b/src/backend/executor/execExprInterp.c @@ -3146,8 +3146,8 @@ bool ExecEvalSubscriptingRef(ExprState *state, ExprEvalStep *op) { SubscriptingRefState *sbsrefstate = op->d.sbsref_subscript.state; - int *indexes; - int off; + Datum *indexes; + int off; /* If any index expr yields NULL, result is NULL or error */ if (sbsrefstate->subscriptnull) @@ -3167,7 +3167,7 @@ ExecEvalSubscriptingRef(ExprState *state, ExprEvalStep *op) indexes = sbsrefstate->lowerindex; off = op->d.sbsref_subscript.off; - indexes[off] = DatumGetInt32(sbsrefstate->subscriptvalue); + indexes[off] = sbsrefstate->subscriptvalue; return true; } @@ -3181,36 +3181,14 @@ void ExecEvalSubscriptingRefFetch(ExprState *state, ExprEvalStep *op) { SubscriptingRefState *sbsrefstate = op->d.sbsref.state; + SubscriptRoutines *sbsroutines = sbsrefstate->sbsroutines; /* Should not get here if source container (or any subscript) is null */ Assert(!(*op->resnull)); - if (sbsrefstate->numlower == 0) - { - /* Scalar case */ - *op->resvalue = array_get_element(*op->resvalue, - sbsrefstate->numupper, - sbsrefstate->upperindex, - sbsrefstate->refattrlength, - sbsrefstate->refelemlength, - sbsrefstate->refelembyval, - sbsrefstate->refelemalign, - op->resnull); - } - else - { - /* Slice case */ - *op->resvalue = array_get_slice(*op->resvalue, - sbsrefstate->numupper, - sbsrefstate->upperindex, - sbsrefstate->lowerindex, - sbsrefstate->upperprovided, - sbsrefstate->lowerprovided, - sbsrefstate->refattrlength, - sbsrefstate->refelemlength, - sbsrefstate->refelembyval, - sbsrefstate->refelemalign); - } + + *op->resvalue = sbsroutines->fetch(*op->resvalue, sbsrefstate); + *op->resnull = sbsrefstate->resnull; } /* @@ -3223,40 +3201,20 @@ void ExecEvalSubscriptingRefOld(ExprState *state, ExprEvalStep *op) { SubscriptingRefState *sbsrefstate = op->d.sbsref.state; + SubscriptRoutines *sbsroutines = sbsrefstate->sbsroutines; if (*op->resnull) { - /* whole array is null, so any element or slice is too */ + /* whole container is null, so any element or slice is too */ sbsrefstate->prevvalue = (Datum) 0; sbsrefstate->prevnull = true; } - else if (sbsrefstate->numlower == 0) - { - /* Scalar case */ - sbsrefstate->prevvalue = array_get_element(*op->resvalue, - sbsrefstate->numupper, - sbsrefstate->upperindex, - sbsrefstate->refattrlength, - sbsrefstate->refelemlength, - sbsrefstate->refelembyval, - sbsrefstate->refelemalign, - &sbsrefstate->prevnull); - } else { - /* Slice case */ - /* this is currently unreachable */ - sbsrefstate->prevvalue = array_get_slice(*op->resvalue, - sbsrefstate->numupper, - sbsrefstate->upperindex, - sbsrefstate->lowerindex, - sbsrefstate->upperprovided, - sbsrefstate->lowerprovided, - sbsrefstate->refattrlength, - sbsrefstate->refelemlength, - sbsrefstate->refelembyval, - sbsrefstate->refelemalign); - sbsrefstate->prevnull = false; + sbsrefstate->prevvalue = sbsroutines->fetch(*op->resvalue, sbsrefstate); + + if (sbsrefstate->numlower != 0) + sbsrefstate->prevnull = false; } } @@ -3270,59 +3228,11 @@ void ExecEvalSubscriptingRefAssign(ExprState *state, ExprEvalStep *op) { SubscriptingRefState *sbsrefstate = op->d.sbsref_subscript.state; + SubscriptRoutines *sbsroutines = sbsrefstate->sbsroutines; - /* - * For an assignment to a fixed-length container type, both the original - * container and the value to be assigned into it must be non-NULL, else - * we punt and return the original container. - */ - if (sbsrefstate->refattrlength > 0) - { - if (*op->resnull || sbsrefstate->replacenull) - return; - } - - /* - * For assignment to varlena arrays, we handle a NULL original array by - * substituting an empty (zero-dimensional) array; insertion of the new - * element will result in a singleton array value. It does not matter - * whether the new element is NULL. - */ - if (*op->resnull) - { - *op->resvalue = PointerGetDatum(construct_empty_array(sbsrefstate->refelemtype)); - *op->resnull = false; - } - - if (sbsrefstate->numlower == 0) - { - /* Scalar case */ - *op->resvalue = array_set_element(*op->resvalue, - sbsrefstate->numupper, - sbsrefstate->upperindex, - sbsrefstate->replacevalue, - sbsrefstate->replacenull, - sbsrefstate->refattrlength, - sbsrefstate->refelemlength, - sbsrefstate->refelembyval, - sbsrefstate->refelemalign); - } - else - { - /* Slice case */ - *op->resvalue = array_set_slice(*op->resvalue, - sbsrefstate->numupper, - sbsrefstate->upperindex, - sbsrefstate->lowerindex, - sbsrefstate->upperprovided, - sbsrefstate->lowerprovided, - sbsrefstate->replacevalue, - sbsrefstate->replacenull, - sbsrefstate->refattrlength, - sbsrefstate->refelemlength, - sbsrefstate->refelembyval, - sbsrefstate->refelemalign); - } + sbsrefstate->resnull = *op->resnull; + *op->resvalue = sbsroutines->assign(*op->resvalue, sbsrefstate); + *op->resnull = sbsrefstate->resnull; } /* diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c index 89c409de66..ec9b0dd97f 100644 --- a/src/backend/nodes/copyfuncs.c +++ b/src/backend/nodes/copyfuncs.c @@ -1548,8 +1548,10 @@ _copySubscriptingRef(const SubscriptingRef *from) COPY_SCALAR_FIELD(refcontainertype); COPY_SCALAR_FIELD(refelemtype); + COPY_SCALAR_FIELD(refassgntype); COPY_SCALAR_FIELD(reftypmod); COPY_SCALAR_FIELD(refcollid); + COPY_SCALAR_FIELD(refnestedfunc); COPY_NODE_FIELD(refupperindexpr); COPY_NODE_FIELD(reflowerindexpr); COPY_NODE_FIELD(refexpr); diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c index e3f33c40be..48b436f7f7 100644 --- a/src/backend/nodes/equalfuncs.c +++ b/src/backend/nodes/equalfuncs.c @@ -274,8 +274,10 @@ _equalSubscriptingRef(const SubscriptingRef *a, const SubscriptingRef *b) { COMPARE_SCALAR_FIELD(refcontainertype); COMPARE_SCALAR_FIELD(refelemtype); + COMPARE_SCALAR_FIELD(refassgntype); COMPARE_SCALAR_FIELD(reftypmod); COMPARE_SCALAR_FIELD(refcollid); + COMPARE_SCALAR_FIELD(refnestedfunc); COMPARE_NODE_FIELD(refupperindexpr); COMPARE_NODE_FIELD(reflowerindexpr); COMPARE_NODE_FIELD(refexpr); diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c index e2f177515d..70c9736c46 100644 --- a/src/backend/nodes/outfuncs.c +++ b/src/backend/nodes/outfuncs.c @@ -1194,8 +1194,10 @@ _outSubscriptingRef(StringInfo str, const SubscriptingRef *node) WRITE_OID_FIELD(refcontainertype); WRITE_OID_FIELD(refelemtype); + WRITE_OID_FIELD(refassgntype); WRITE_INT_FIELD(reftypmod); WRITE_OID_FIELD(refcollid); + WRITE_OID_FIELD(refnestedfunc); WRITE_NODE_FIELD(refupperindexpr); WRITE_NODE_FIELD(reflowerindexpr); WRITE_NODE_FIELD(refexpr); diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c index 42050ab719..1c9752c771 100644 --- a/src/backend/nodes/readfuncs.c +++ b/src/backend/nodes/readfuncs.c @@ -669,8 +669,10 @@ _readSubscriptingRef(void) READ_OID_FIELD(refcontainertype); READ_OID_FIELD(refelemtype); + READ_OID_FIELD(refassgntype); READ_INT_FIELD(reftypmod); READ_OID_FIELD(refcollid); + READ_OID_FIELD(refnestedfunc); READ_NODE_FIELD(refupperindexpr); READ_NODE_FIELD(reflowerindexpr); READ_NODE_FIELD(refexpr); diff --git a/src/backend/parser/parse_expr.c b/src/backend/parser/parse_expr.c index f69976cc8c..ea9d35429c 100644 --- a/src/backend/parser/parse_expr.c +++ b/src/backend/parser/parse_expr.c @@ -433,11 +433,13 @@ unknown_attribute(ParseState *pstate, Node *relref, const char *attname, static Node * transformIndirection(ParseState *pstate, A_Indirection *ind) { - Node *last_srf = pstate->p_last_srf; - Node *result = transformExprRecurse(pstate, ind->arg); - List *subscripts = NIL; - int location = exprLocation(result); - ListCell *i; + Node *last_srf = pstate->p_last_srf; + Node *result = transformExprRecurse(pstate, ind->arg); + SubscriptRoutines *sbsroutines; + SubscriptingRef *sbsref; + List *subscripts = NIL; + int location = exprLocation(result); + ListCell *i; /* * We have to split any field-selection operations apart from @@ -465,13 +467,20 @@ transformIndirection(ParseState *pstate, A_Indirection *ind) /* process subscripts before this field selection */ if (subscripts) - result = (Node *) transformContainerSubscripts(pstate, - result, - exprType(result), - InvalidOid, - exprTypmod(result), - subscripts, - NULL); + { + sbsref = transformContainerSubscripts(pstate, + result, + exprType(result), + InvalidOid, + exprTypmod(result), + subscripts, + NULL); + + sbsroutines = getSubscriptingRoutines(sbsref->refcontainertype); + sbsref = sbsroutines->prepare(false, sbsref); + sbsroutines->validate(false, sbsref, pstate); + result = (Node *) sbsref; + } subscripts = NIL; newresult = ParseFuncOrColumn(pstate, @@ -488,13 +497,20 @@ transformIndirection(ParseState *pstate, A_Indirection *ind) } /* process trailing subscripts, if any */ if (subscripts) - result = (Node *) transformContainerSubscripts(pstate, - result, - exprType(result), - InvalidOid, - exprTypmod(result), - subscripts, - NULL); + { + sbsref = transformContainerSubscripts(pstate, + result, + exprType(result), + InvalidOid, + exprTypmod(result), + subscripts, + NULL); + + sbsroutines = getSubscriptingRoutines(sbsref->refcontainertype); + sbsref = sbsroutines->prepare(false, sbsref); + sbsroutines->validate(false, sbsref, pstate); + result = (Node *) sbsref; + } return result; } diff --git a/src/backend/parser/parse_node.c b/src/backend/parser/parse_node.c index 6e98fe55fc..4f46d6310a 100644 --- a/src/backend/parser/parse_node.c +++ b/src/backend/parser/parse_node.c @@ -184,21 +184,12 @@ pcb_error_callback(void *arg) * transformContainerType() * Identify the types involved in a subscripting operation for container * - * - * On entry, containerType/containerTypmod identify the type of the input value - * to be subscripted (which could be a domain type). These are modified if - * necessary to identify the actual container type and typmod, and the - * container's element type is returned. An error is thrown if the input isn't - * an array type. + * On entry, containerType/containerTypmod are modified if necessary to + * identify the actual container type and typmod. */ -Oid +void transformContainerType(Oid *containerType, int32 *containerTypmod) { - Oid origContainerType = *containerType; - Oid elementType; - HeapTuple type_tuple_container; - Form_pg_type type_struct_container; - /* * If the input is a domain, smash to base type, and extract the actual * typmod to be applied to the base type. Subscripting a domain is an @@ -219,25 +210,6 @@ transformContainerType(Oid *containerType, int32 *containerTypmod) *containerType = INT2ARRAYOID; else if (*containerType == OIDVECTOROID) *containerType = OIDARRAYOID; - - /* Get the type tuple for the container */ - type_tuple_container = SearchSysCache1(TYPEOID, ObjectIdGetDatum(*containerType)); - if (!HeapTupleIsValid(type_tuple_container)) - elog(ERROR, "cache lookup failed for type %u", *containerType); - type_struct_container = (Form_pg_type) GETSTRUCT(type_tuple_container); - - /* needn't check typisdefined since this will fail anyway */ - - elementType = type_struct_container->typelem; - if (elementType == InvalidOid) - ereport(ERROR, - (errcode(ERRCODE_DATATYPE_MISMATCH), - errmsg("cannot subscript type %s because it is not an array", - format_type_be(origContainerType)))); - - ReleaseSysCache(type_tuple_container); - - return elementType; } /* @@ -254,10 +226,15 @@ transformContainerType(Oid *containerType, int32 *containerTypmod) * container. We produce an expression that represents the new container value * with the source data inserted into the right part of the container. * - * For both cases, if the source container is of a domain-over-array type, - * the result is of the base array type or its element type; essentially, - * we must fold a domain to its base type before applying subscripting. - * (Note that int2vector and oidvector are treated as domains here.) + * For both cases, this function contains only general subscripting logic while + * type-specific logic (e.g. type verifications and coercion) is placed in + * separate procedures indicated by typsubshandler. There is only one exception + * for now about domain-over-container: if the source container is of a + * domain-over-container type, the result is of the base container type or its + * element type; essentially, we must fold a domain to its base type before + * applying subscripting. (Note that int2vector and oidvector are treated as + * domains here.) An error will appear in the case the current container type + * doesn't have a subscripting procedure. * * pstate Parse state * containerBase Already-transformed expression for the container as a whole @@ -284,16 +261,12 @@ transformContainerSubscripts(ParseState *pstate, bool isSlice = false; List *upperIndexpr = NIL; List *lowerIndexpr = NIL; + List *indexprSlice = NIL; ListCell *idx; SubscriptingRef *sbsref; - /* - * Caller may or may not have bothered to determine elementType. Note - * that if the caller did do so, containerType/containerTypMod must be as - * modified by transformContainerType, ie, smash domain to base type. - */ - if (!OidIsValid(elementType)) - elementType = transformContainerType(&containerType, &containerTypMod); + /* Identify the actual container type and element type involved */ + transformContainerType(&containerType, &containerTypMod); /* * A list containing only simple subscripts refers to a single container @@ -327,29 +300,6 @@ transformContainerSubscripts(ParseState *pstate, if (ai->lidx) { subexpr = transformExpr(pstate, ai->lidx, pstate->p_expr_kind); - /* If it's not int4 already, try to coerce */ - subexpr = coerce_to_target_type(pstate, - subexpr, exprType(subexpr), - INT4OID, -1, - COERCION_ASSIGNMENT, - COERCE_IMPLICIT_CAST, - -1); - if (subexpr == NULL) - ereport(ERROR, - (errcode(ERRCODE_DATATYPE_MISMATCH), - errmsg("array subscript must have type integer"), - parser_errposition(pstate, exprLocation(ai->lidx)))); - } - else if (!ai->is_slice) - { - /* Make a constant 1 */ - subexpr = (Node *) makeConst(INT4OID, - -1, - InvalidOid, - sizeof(int32), - Int32GetDatum(1), - false, - true); /* pass by value */ } else { @@ -357,63 +307,23 @@ transformContainerSubscripts(ParseState *pstate, subexpr = NULL; } lowerIndexpr = lappend(lowerIndexpr, subexpr); + indexprSlice = lappend(indexprSlice, ai); } else Assert(ai->lidx == NULL && !ai->is_slice); if (ai->uidx) - { subexpr = transformExpr(pstate, ai->uidx, pstate->p_expr_kind); - /* If it's not int4 already, try to coerce */ - subexpr = coerce_to_target_type(pstate, - subexpr, exprType(subexpr), - INT4OID, -1, - COERCION_ASSIGNMENT, - COERCE_IMPLICIT_CAST, - -1); - if (subexpr == NULL) - ereport(ERROR, - (errcode(ERRCODE_DATATYPE_MISMATCH), - errmsg("array subscript must have type integer"), - parser_errposition(pstate, exprLocation(ai->uidx)))); - } else { /* Slice with omitted upper bound, put NULL into the list */ Assert(isSlice && ai->is_slice); subexpr = NULL; } + subexpr = transformExpr(pstate, ai->uidx, pstate->p_expr_kind); upperIndexpr = lappend(upperIndexpr, subexpr); } - /* - * If doing an array store, coerce the source value to the right type. - * (This should agree with the coercion done by transformAssignedExpr.) - */ - if (assignFrom != NULL) - { - Oid typesource = exprType(assignFrom); - Oid typeneeded = isSlice ? containerType : elementType; - Node *newFrom; - - newFrom = coerce_to_target_type(pstate, - assignFrom, typesource, - typeneeded, containerTypMod, - COERCION_ASSIGNMENT, - COERCE_IMPLICIT_CAST, - -1); - if (newFrom == NULL) - ereport(ERROR, - (errcode(ERRCODE_DATATYPE_MISMATCH), - errmsg("array assignment requires type %s" - " but expression is of type %s", - format_type_be(typeneeded), - format_type_be(typesource)), - errhint("You will need to rewrite or cast the expression."), - parser_errposition(pstate, exprLocation(assignFrom)))); - assignFrom = newFrom; - } - /* * Ready to build the SubscriptingRef node. */ @@ -422,17 +332,30 @@ transformContainerSubscripts(ParseState *pstate, sbsref->refassgnexpr = (Expr *) assignFrom; sbsref->refcontainertype = containerType; - sbsref->refelemtype = elementType; sbsref->reftypmod = containerTypMod; /* refcollid will be set by parse_collate.c */ sbsref->refupperindexpr = upperIndexpr; sbsref->reflowerindexpr = lowerIndexpr; + sbsref->refindexprslice = indexprSlice; sbsref->refexpr = (Expr *) containerBase; - sbsref->refassgnexpr = (Expr *) assignFrom; return sbsref; } +SubscriptRoutines* +getSubscriptingRoutines(Oid containerType) +{ + RegProcedure typsubshandler = get_typsubsprocs(containerType); + + if (!OidIsValid(typsubshandler)) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot subscript type %s because it does not support subscripting", + format_type_be(containerType)))); + + return (SubscriptRoutines *) OidFunctionCall0(typsubshandler); +} + /* * make_const * diff --git a/src/backend/parser/parse_target.c b/src/backend/parser/parse_target.c index 566c517837..d7483c6538 100644 --- a/src/backend/parser/parse_target.c +++ b/src/backend/parser/parse_target.c @@ -685,7 +685,7 @@ transformAssignmentIndirection(ParseState *pstate, Node *rhs, int location) { - Node *result; + Node *result = NULL; List *subscripts = NIL; bool isSlice = false; ListCell *i; @@ -848,27 +848,21 @@ transformAssignmentIndirection(ParseState *pstate, location); } - /* base case: just coerce RHS to match target type ID */ - - result = coerce_to_target_type(pstate, - rhs, exprType(rhs), - targetTypeId, targetTypMod, - COERCION_ASSIGNMENT, - COERCE_IMPLICIT_CAST, - -1); - if (result == NULL) + /* + * Base case: just coerce RHS to match target type ID. + * It's necessary only for field selection, since for + * subscripting its custom code should define types. + */ + if (!targetIsSubscripting) { - if (targetIsSubscripting) - ereport(ERROR, - (errcode(ERRCODE_DATATYPE_MISMATCH), - errmsg("array assignment to \"%s\" requires type %s" - " but expression is of type %s", - targetName, - format_type_be(targetTypeId), - format_type_be(exprType(rhs))), - errhint("You will need to rewrite or cast the expression."), - parser_errposition(pstate, location))); - else + result = coerce_to_target_type(pstate, + rhs, exprType(rhs), + targetTypeId, targetTypMod, + COERCION_ASSIGNMENT, + COERCE_IMPLICIT_CAST, + -1); + + if (result == NULL) ereport(ERROR, (errcode(ERRCODE_DATATYPE_MISMATCH), errmsg("subfield \"%s\" is of type %s" @@ -900,29 +894,42 @@ transformAssignmentSubscripts(ParseState *pstate, Node *rhs, int location) { - Node *result; - Oid containerType; - int32 containerTypMod; - Oid elementTypeId; - Oid typeNeeded; - Oid collationNeeded; + Node *result; + Oid containerType; + int32 containerTypMod; + Oid collationNeeded; + SubscriptingRef *sbsref; + SubscriptRoutines *sbsroutines; Assert(subscripts != NIL); /* Identify the actual array type and element type involved */ containerType = targetTypeId; containerTypMod = targetTypMod; - elementTypeId = transformContainerType(&containerType, &containerTypMod); - /* Identify type that RHS must provide */ - typeNeeded = isSlice ? containerType : elementTypeId; + /* process subscripts */ + sbsref = transformContainerSubscripts(pstate, + basenode, + containerType, + exprType(rhs), + containerTypMod, + subscripts, + rhs); + + sbsroutines = getSubscriptingRoutines(sbsref->refcontainertype); + + /* + * Let custom code provide necessary information about required types: + * refelemtype and refassgntype + */ + sbsref = sbsroutines->prepare(rhs != NULL, sbsref); /* * container normally has same collation as elements, but there's an * exception: we might be subscripting a domain over a container type. In * that case use collation of the base type. */ - if (containerType == targetTypeId) + if (sbsref->refcontainertype == containerType) collationNeeded = targetCollation; else collationNeeded = get_typcollation(containerType); @@ -932,25 +939,22 @@ transformAssignmentSubscripts(ParseState *pstate, NULL, targetName, true, - typeNeeded, - containerTypMod, + sbsref->refassgntype, + sbsref->reftypmod, collationNeeded, indirection, next_indirection, rhs, location); - /* process subscripts */ - result = (Node *) transformContainerSubscripts(pstate, - basenode, - containerType, - elementTypeId, - containerTypMod, - subscripts, - rhs); + /* Provide fully prepared subscripting information for custom validation */ + sbsref->refassgnexpr = (Expr *) rhs; + sbsroutines->validate(rhs != NULL, sbsref, pstate); + + result = (Node *) sbsref; /* If target was a domain over container, need to coerce up to the domain */ - if (containerType != targetTypeId) + if (sbsref->refcontainertype != targetTypeId) { Oid resulttype = exprType(result); diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c index 2cbcb4b85e..fb298c11c2 100644 --- a/src/backend/utils/adt/ruleutils.c +++ b/src/backend/utils/adt/ruleutils.c @@ -7994,17 +7994,18 @@ get_rule_expr(Node *node, deparse_context *context, if (need_parens) appendStringInfoChar(buf, ')'); - /* - * If there's a refassgnexpr, we want to print the node in the - * format "container[subscripts] := refassgnexpr". This is - * not legal SQL, so decompilation of INSERT or UPDATE - * statements should always use processIndirection as part of - * the statement-level syntax. We should only see this when - * EXPLAIN tries to print the targetlist of a plan resulting - * from such a statement. - */ - if (sbsref->refassgnexpr) + if (IsAssignment(sbsref)) { + /* + * If there's a refassgnexpr, we want to print the node in the + * format "container[subscripts] := refassgnexpr". This is not + * legal SQL, so decompilation of INSERT or UPDATE statements + * should always use processIndirection as part of the + * statement-level syntax. We should only see this when + * EXPLAIN tries to print the targetlist of a plan resulting + * from such a statement. + */ + Node *refassgnexpr; /* diff --git a/src/backend/utils/cache/lsyscache.c b/src/backend/utils/cache/lsyscache.c index f3bf413829..fb264a1e57 100644 --- a/src/backend/utils/cache/lsyscache.c +++ b/src/backend/utils/cache/lsyscache.c @@ -3273,6 +3273,29 @@ get_range_collation(Oid rangeOid) return InvalidOid; } +/* + * get_typsubshandler + * + * Given the type OID, return the type's subscripting procedures, if any, + * through pointers in arguments. + */ +RegProcedure +get_typsubsprocs(Oid typid) +{ + HeapTuple tp; + + tp = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typid)); + if (HeapTupleIsValid(tp)) + { + RegProcedure handler = ((Form_pg_type) GETSTRUCT(tp))->typsubshandler; + ReleaseSysCache(tp); + + return handler; + } + else + return InvalidOid; +} + /* ---------- PG_INDEX CACHE ---------- */ /* diff --git a/src/include/c.h b/src/include/c.h index f242e32edb..108e424df7 100644 --- a/src/include/c.h +++ b/src/include/c.h @@ -545,6 +545,8 @@ typedef struct int indx[MAXDIM]; } IntArray; +#define MAX_SUBSCRIPT_DEPTH 12 + /* ---------------- * Variable-length datatypes all share the 'struct varlena' header. * diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h index 7b37562648..ea237ec61d 100644 --- a/src/include/catalog/pg_type.h +++ b/src/include/catalog/pg_type.h @@ -221,6 +221,12 @@ CATALOG(pg_type,1247,TypeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(71,TypeRelati */ Oid typcollation BKI_DEFAULT(0) BKI_LOOKUP(pg_collation); + /* + * Type specific subscripting logic. If typsubshandler is NULL, it means + * that this type doesn't support subscripting. + */ + regproc typsubshandler BKI_DEFAULT(-) BKI_LOOKUP(pg_proc); + #ifdef CATALOG_VARLEN /* variable-length fields start here */ /* @@ -349,7 +355,8 @@ extern ObjectAddress TypeCreate(Oid newTypeOid, int32 typeMod, int32 typNDims, bool typeNotNull, - Oid typeCollation); + Oid typeCollation, + Oid subscriptingParseProcedure); extern void GenerateTypeDependencies(HeapTuple typeTuple, Relation typeCatalog, diff --git a/src/include/executor/execExpr.h b/src/include/executor/execExpr.h index dbe8649a57..52c357b2aa 100644 --- a/src/include/executor/execExpr.h +++ b/src/include/executor/execExpr.h @@ -19,7 +19,7 @@ /* forward references to avoid circularity */ struct ExprEvalStep; -struct SubscriptingRefState; +struct SubscriptRoutines; /* Bits in ExprState->flags (see also execnodes.h for public flag bits): */ /* expression's interpreter has been initialized */ @@ -658,13 +658,13 @@ typedef struct SubscriptingRefState /* numupper and upperprovided[] are filled at compile time */ /* at runtime, extracted subscript datums get stored in upperindex[] */ int numupper; - bool upperprovided[MAXDIM]; - int upperindex[MAXDIM]; + bool upperprovided[MAX_SUBSCRIPT_DEPTH]; + Datum upperindex[MAX_SUBSCRIPT_DEPTH]; /* similarly for lower indexes, if any */ int numlower; - bool lowerprovided[MAXDIM]; - int lowerindex[MAXDIM]; + bool lowerprovided[MAX_SUBSCRIPT_DEPTH]; + Datum lowerindex[MAX_SUBSCRIPT_DEPTH]; /* subscript expressions get evaluated into here */ Datum subscriptvalue; @@ -677,6 +677,9 @@ typedef struct SubscriptingRefState /* if we have a nested assignment, SBSREF_OLD puts old value here */ Datum prevvalue; bool prevnull; + + bool resnull; + struct SubscriptRoutines *sbsroutines; } SubscriptingRefState; diff --git a/src/include/nodes/primnodes.h b/src/include/nodes/primnodes.h index d73be2ad46..5991f437cd 100644 --- a/src/include/nodes/primnodes.h +++ b/src/include/nodes/primnodes.h @@ -417,13 +417,17 @@ typedef struct SubscriptingRef Expr xpr; Oid refcontainertype; /* type of the container proper */ Oid refelemtype; /* type of the container elements */ + Oid refassgntype; /* type of assignment expr that is required */ int32 reftypmod; /* typmod of the container (and elements too) */ Oid refcollid; /* OID of collation, or InvalidOid if none */ + Oid refnestedfunc; /* OID of type-specific function to handle nested assignment */ List *refupperindexpr; /* expressions that evaluate to upper * container indexes */ List *reflowerindexpr; /* expressions that evaluate to lower * container indexes, or NIL for single * container element */ + List *refindexprslice; /* whether or not related indexpr from + * reflowerindexpr is a slice */ Expr *refexpr; /* the expression that evaluates to a * container value */ @@ -431,6 +435,8 @@ typedef struct SubscriptingRef * fetch */ } SubscriptingRef; +#define IsAssignment(expr) ( ((SubscriptingRef*) expr)->refassgnexpr != NULL ) + /* * CoercionContext - distinguishes the allowed set of type casts * diff --git a/src/include/nodes/subscripting.h b/src/include/nodes/subscripting.h new file mode 100644 index 0000000000..1800d5ecf5 --- /dev/null +++ b/src/include/nodes/subscripting.h @@ -0,0 +1,42 @@ +/*------------------------------------------------------------------------- + * + * subscripting.h + * API for generic type subscripting + * + * Portions Copyright (c) 1996-2018, PostgreSQL Global Development Group + * Portions Copyright (c) 1994, Regents of the University of California + * + * src/include/nodes/subscripting.h + * + *------------------------------------------------------------------------- + */ +#ifndef SUBSCRIPTING_H +#define SUBSCRIPTING_H + +#include "parser/parse_node.h" +#include "nodes/primnodes.h" + +struct ParseState; +struct SubscriptingRefState; + +/* Callback function signatures --- see xsubscripting.sgml for more info. */ +typedef SubscriptingRef * (*SubscriptingPrepare) (bool isAssignment, SubscriptingRef *sbsef); + +typedef SubscriptingRef * (*SubscriptingValidate) (bool isAssignment, SubscriptingRef *sbsef, + struct ParseState *pstate); + +typedef Datum (*SubscriptingFetch) (Datum source, struct SubscriptingRefState *sbsrefstate); + +typedef Datum (*SubscriptingAssign) (Datum source, struct SubscriptingRefState *sbrsefstate); + +typedef struct SubscriptRoutines +{ + SubscriptingPrepare prepare; + SubscriptingValidate validate; + SubscriptingFetch fetch; + SubscriptingAssign assign; + +} SubscriptRoutines; + + +#endif /* SUBSCRIPTING_H */ diff --git a/src/include/parser/parse_node.h b/src/include/parser/parse_node.h index d25819aa28..b4736206d1 100644 --- a/src/include/parser/parse_node.h +++ b/src/include/parser/parse_node.h @@ -15,6 +15,7 @@ #define PARSE_NODE_H #include "nodes/parsenodes.h" +#include "nodes/subscripting.h" #include "utils/queryenvironment.h" #include "utils/relcache.h" @@ -313,7 +314,9 @@ extern void setup_parser_errposition_callback(ParseCallbackState *pcbstate, ParseState *pstate, int location); extern void cancel_parser_errposition_callback(ParseCallbackState *pcbstate); -extern Oid transformContainerType(Oid *containerType, int32 *containerTypmod); +extern Var *make_var(ParseState *pstate, RangeTblEntry *rte, int attrno, + int location); +extern void transformContainerType(Oid *containerType, int32 *containerTypmod); extern SubscriptingRef *transformContainerSubscripts(ParseState *pstate, Node *containerBase, @@ -322,6 +325,7 @@ extern SubscriptingRef *transformContainerSubscripts(ParseState *pstate, int32 containerTypMod, List *indirection, Node *assignFrom); +extern SubscriptRoutines *getSubscriptingRoutines(Oid containerType); extern Const *make_const(ParseState *pstate, Value *value, int location); #endif /* PARSE_NODE_H */ diff --git a/src/include/utils/lsyscache.h b/src/include/utils/lsyscache.h index fecfe1f4f6..8fc570d2e1 100644 --- a/src/include/utils/lsyscache.h +++ b/src/include/utils/lsyscache.h @@ -183,6 +183,7 @@ extern char *get_namespace_name(Oid nspid); extern char *get_namespace_name_or_temp(Oid nspid); extern Oid get_range_subtype(Oid rangeOid); extern Oid get_range_collation(Oid rangeOid); +extern RegProcedure get_typsubsprocs(Oid typid); extern Oid get_index_column_opclass(Oid index_oid, int attno); extern bool get_index_isreplident(Oid index_oid); extern bool get_index_isvalid(Oid index_oid); -- 2.21.0 --u66dlpg6gaevsofr Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v33-0002-Subscripting-for-array.patch" ^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: NUMA shared memory interleaving @ 2025-04-16 13:58 Thomas Munro <[email protected]> 0 siblings, 1 reply; 4+ messages in thread From: Thomas Munro @ 2025-04-16 13:58 UTC (permalink / raw) To: Jakub Wartak <[email protected]>; +Cc: PostgreSQL Hackers <[email protected]>; Andres Freund <[email protected]>; Tomas Vondra <[email protected]>; Bertrand Drouvot <[email protected]> On Wed, Apr 16, 2025 at 9:14 PM Jakub Wartak <[email protected]> wrote: > 2. Should we also interleave DSA/DSM for Parallel Query? (I'm not an > expert on DSA/DSM at all) I have no answers but I have speculated for years about a very specific case (without any idea where to begin due to lack of ... I guess all this sort of stuff): in ExecParallelHashJoinNewBatch(), workers split up and try to work on different batches on their own to minimise contention, and when that's not possible (more workers than batches, or finishing their existing work at different times and going to help others), they just proceed in round-robin order. A beginner thought is: if you're going to help someone working on a hash table, it would surely be best to have the CPUs and all the data on the same NUMA node. During loading, cache line ping pong would be cheaper, and during probing, it *might* be easier to tune explicit memory prefetch timing that way as it would look more like a single node system with a fixed latency, IDK (I've shared patches for prefetching before that showed pretty decent speedups, and the lack of that feature is probably a bigger problem than any of this stuff, who knows...). Another beginner thought is that the DSA allocator is a source of contention during loading: the dumbest problem is that the chunks are just too small, but it might also be interesting to look into per-node pools. Or something. IDK, just some thoughts... ^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: NUMA shared memory interleaving @ 2025-04-18 17:48 Bertrand Drouvot <[email protected]> parent: Thomas Munro <[email protected]> 0 siblings, 1 reply; 4+ messages in thread From: Bertrand Drouvot @ 2025-04-18 17:48 UTC (permalink / raw) To: Thomas Munro <[email protected]>; +Cc: Jakub Wartak <[email protected]>; PostgreSQL Hackers <[email protected]>; Andres Freund <[email protected]>; Tomas Vondra <[email protected]> Hi, On Thu, Apr 17, 2025 at 01:58:44AM +1200, Thomas Munro wrote: > On Wed, Apr 16, 2025 at 9:14 PM Jakub Wartak > <[email protected]> wrote: > > 2. Should we also interleave DSA/DSM for Parallel Query? (I'm not an > > expert on DSA/DSM at all) > > I have no answers but I have speculated for years about a very > specific case (without any idea where to begin due to lack of ... I > guess all this sort of stuff): in ExecParallelHashJoinNewBatch(), > workers split up and try to work on different batches on their own to > minimise contention, and when that's not possible (more workers than > batches, or finishing their existing work at different times and going > to help others), they just proceed in round-robin order. A beginner > thought is: if you're going to help someone working on a hash table, > it would surely be best to have the CPUs and all the data on the same > NUMA node. During loading, cache line ping pong would be cheaper, and > during probing, it *might* be easier to tune explicit memory prefetch > timing that way as it would look more like a single node system with a > fixed latency, IDK (I've shared patches for prefetching before that > showed pretty decent speedups, and the lack of that feature is > probably a bigger problem than any of this stuff, who knows...). > Another beginner thought is that the DSA allocator is a source of > contention during loading: the dumbest problem is that the chunks are > just too small, but it might also be interesting to look into per-node > pools. Or something. IDK, just some thoughts... I'm also thinking that could be beneficial for parallel workers. I think the ideal scenario would be to have the parallel workers spread across numa nodes and accessing their "local" memory first (and help with "remote" memory access if there is still more work to do "remotely"). Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com ^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: NUMA shared memory interleaving @ 2025-06-26 11:36 Jakub Wartak <[email protected]> parent: Bertrand Drouvot <[email protected]> 0 siblings, 0 replies; 4+ messages in thread From: Jakub Wartak @ 2025-06-26 11:36 UTC (permalink / raw) To: Bertrand Drouvot <[email protected]>; +Cc: Thomas Munro <[email protected]>; PostgreSQL Hackers <[email protected]>; Andres Freund <[email protected]>; Tomas Vondra <[email protected]> On Fri, Apr 18, 2025 at 7:48 PM Bertrand Drouvot <[email protected]> wrote: > > Hi, > > On Thu, Apr 17, 2025 at 01:58:44AM +1200, Thomas Munro wrote: > > On Wed, Apr 16, 2025 at 9:14 PM Jakub Wartak > > <[email protected]> wrote: > > > 2. Should we also interleave DSA/DSM for Parallel Query? (I'm not an > > > expert on DSA/DSM at all) > > > > I have no answers but I have speculated for years about a very > > specific case (without any idea where to begin due to lack of ... I > > guess all this sort of stuff): in ExecParallelHashJoinNewBatch(), > > workers split up and try to work on different batches on their own to > > minimise contention, and when that's not possible (more workers than > > batches, or finishing their existing work at different times and going > > to help others), they just proceed in round-robin order. A beginner > > thought is: if you're going to help someone working on a hash table, > > it would surely be best to have the CPUs and all the data on the same > > NUMA node. During loading, cache line ping pong would be cheaper, and > > during probing, it *might* be easier to tune explicit memory prefetch > > timing that way as it would look more like a single node system with a > > fixed latency, IDK (I've shared patches for prefetching before that > > showed pretty decent speedups, and the lack of that feature is > > probably a bigger problem than any of this stuff, who knows...). > > Another beginner thought is that the DSA allocator is a source of > > contention during loading: the dumbest problem is that the chunks are > > just too small, but it might also be interesting to look into per-node > > pools. Or something. IDK, just some thoughts... > > I'm also thinking that could be beneficial for parallel workers. I think the > ideal scenario would be to have the parallel workers spread across numa nodes and > accessing their "local" memory first (and help with "remote" memory access if > there is still more work to do "remotely"). Hi Bertrand, I've played with CPU pinning of PQ workers (via adjusting postmaster pin), but I've got quite opposite results - please see attached, especially "lat"ency against how the CPUs were assigned VS NUMA/s_b when it was not interleaved. Not that I intend to spend a lot of time researching PQ vs NUMA , but I've included interleaving of PQ shm segments too in the v4 patch in the subthread nearby. Those attached results here, were made some time ago with v1 of the patch where PQ shm segment was not interleaved. If anything it would be to hear if there are any sensible production-like scenarios/workloads when dynamic_shared_memory should be set to sysv or mmap (instead of default posix) ? Asking for Linux only, I couldn't imagine anything (?) -J. On legacy 4s/NUMA simple simplest query to `select sum(aid) from pgbench_accounts_1 (single partition, 400MB)` with pgbench -c 1 client and small s_b 4GB just taking single NUMA node, gets this: Finalize Aggregate (cost=72150.70..72150.71 rows=1 width=8) (actual time=252.817..254.409 rows=1.00 loops=1) Output: sum(aid) Buffers: shared hit=54798 -> Gather (cost=72150.49..72150.69 rows=2 width=8) (actual time=252.319..254.381 rows=3.00 loops=1) Output: (PARTIAL sum(aid)) Workers Planned: 2 Workers Launched: 2 Buffers: shared hit=54798 -> Partial Aggregate (cost=71150.49..71150.49 rows=1 width=8) (actual time=247.262..247.264 rows=1.00 loops=3) Output: PARTIAL sum(aid) Buffers: shared hit=54798 Worker 0: actual time=245.116..245.118 rows=1.00 loops=1 Buffers: shared hit=19185 Worker 1: actual time=244.797..244.800 rows=1.00 loops=1 Buffers: shared hit=19021 -> Parallel Seq Scan on public.pgbench_accounts_1 (cost=0.00..67879.99 rows=1308199 width=4) (actual time=0.011..136.895 rows=1041666.67 loop s=3) Output: aid, bid, abalance, filler Buffers: shared hit=54798 Worker 0: actual time=0.010..133.393 rows=1094053.00 loops=1 Buffers: shared hit=19185 Worker 1: actual time=0.009..134.281 rows=1084465.00 loops=1 Buffers: shared hit=19021 Observe "lat"ency metric , it primary depends on the where the process are placed: no PQ, default behavoir: progress: 57.0 s, 2.0 tps, lat 515.312 ms stddev 0.053, 0 failed progress: 58.0 s, 2.0 tps, lat 609.009 ms stddev 92.653, 0 failed progress: 59.0 s, 2.0 tps, lat 515.414 ms stddev 0.423, 0 failed progress: 60.0 s, 2.0 tps, lat 515.530 ms stddev 0.138, 0 failed progress: 61.0 s, 2.0 tps, lat 515.370 ms stddev 0.181, 0 failed progress: 62.0 s, 2.0 tps, lat 515.396 ms stddev 0.351, 0 failed progress: 63.0 s, 1.0 tps, lat 515.484 ms stddev NaN, 0 failed progress: 64.0 s, 2.0 tps, lat 515.428 ms stddev 0.035, 0 failed progress: 65.0 s, 2.0 tps, lat 515.357 ms stddev 0.089, 0 failed progress: 66.0 s, 2.0 tps, lat 608.885 ms stddev 92.633, 0 failed progress: 67.0 s, 2.0 tps, lat 515.828 ms stddev 0.501, 0 failed progress: 68.0 s, 1.0 tps, lat 603.824 ms stddev NaN, 0 failed no PQ, manual pinning (above it flunctuates a lot due to migrations): progress: 318.0 s, 3.0 tps, lat 321.714 ms stddev 0.089, 0 failed <-- bind backend to CPU on the same NUMA node, gets 1.4GB/s DRAM read on that whole socket/OS progress: 319.0 s, 3.0 tps, lat 321.630 ms stddev 0.254, 0 failed progress: 320.0 s, 3.0 tps, lat 321.655 ms stddev 0.131, 0 failed progress: 321.0 s, 3.0 tps, lat 321.555 ms stddev 0.132, 0 failed progress: 322.0 s, 3.0 tps, lat 322.027 ms stddev 0.964, 0 failed progress: 323.0 s, 2.0 tps, lat 426.245 ms stddev 49.599, 0 failed progress: 324.0 s, 2.0 tps, lat 519.873 ms stddev 0.160, 0 failed <-- bind query to CPU on different NUMA node (hits interconnect), gets only ~0.8GB/s read on that socket with s_b progress: 325.0 s, 2.0 tps, lat 519.854 ms stddev 0.488, 0 failed progress: 326.0 s, 2.0 tps, lat 519.604 ms stddev 0.285, 0 failed progress: 327.0 s, 2.0 tps, lat 519.626 ms stddev 0.190, 0 failed progress: 328.0 s, 2.0 tps, lat 520.172 ms stddev 0.064, 0 failed progress: 329.0 s, 3.0 tps, lat 409.836 ms stddev 41.466, 0 failed progress: 330.0 s, 3.0 tps, lat 325.236 ms stddev 0.248, 0 failed <-- bind backend to CPU on the same NUMA node again progress: 331.0 s, 3.0 tps, lat 325.190 ms stddev 0.261, 0 failed progress: 332.0 s, 3.0 tps, lat 325.485 ms stddev 0.751, 0 failed progress: 333.0 s, 3.0 tps, lat 325.026 ms stddev 0.065, 0 failed 520/325 ms = 1.6x; IPC jumps from 1.23 to ~2.00 just due to switch from remote to local DRAM memory access (latency) no PQ and numa=on (s_b interleaved), STABLE (low stddev), notice how optimal was ~321ms vs interleaved gets ~463ms yes it is wrose (321->463ms), but we have uniform latency, instead of high variance depending on how scheduler assigns CPU: progress: 10.0 s, 2.0 tps, lat 463.378 ms stddev 0.181, 0 failed progress: 11.0 s, 2.0 tps, lat 463.422 ms stddev 0.016, 0 failed progress: 12.0 s, 2.0 tps, lat 463.121 ms stddev 0.052, 0 failed progress: 13.0 s, 2.0 tps, lat 463.198 ms stddev 0.148, 0 failed progress: 14.0 s, 3.0 tps, lat 463.823 ms stddev 0.854, 0 failed progress: 15.0 s, 2.0 tps, lat 462.937 ms stddev 0.199, 0 failed ok let's enable PQ: 2 workers and leader participating, default: progress: 30.0 s, 7.0 tps, lat 134.692 ms stddev 4.644, 0 failed progress: 31.0 s, 8.0 tps, lat 129.489 ms stddev 1.510, 0 failed progress: 32.0 s, 7.0 tps, lat 130.506 ms stddev 3.393, 0 failed progress: 33.0 s, 8.0 tps, lat 133.785 ms stddev 1.921, 0 failed progress: 34.0 s, 7.0 tps, lat 134.561 ms stddev 6.965, 0 failed progress: 35.0 s, 8.0 tps, lat 135.438 ms stddev 5.286, 0 failed progress: 36.0 s, 7.0 tps, lat 141.760 ms stddev 15.878, 0 failed progress: 37.0 s, 7.0 tps, lat 136.749 ms stddev 5.937, 0 failed progress: 38.0 s, 7.0 tps, lat 138.258 ms stddev 8.666, 0 failed progress: 39.0 s, 7.0 tps, lat 140.263 ms stddev 8.834, 0 failed 2 workers and leader participating and postmaster CPU affinity (therefore any new pq workers) set to only CPUs from nodes having s_b + CPU-pin the leader there too: progress: 342.0 s, 9.0 tps, lat 117.306 ms stddev 0.152, 0 failed progress: 343.0 s, 8.0 tps, lat 117.265 ms stddev 0.216, 0 failed progress: 344.0 s, 9.0 tps, lat 117.216 ms stddev 0.221, 0 failed progress: 345.0 s, 9.0 tps, lat 117.233 ms stddev 0.155, 0 failed postmaster's CPU affinity (so for 2 PQ workers) being forced to remote NUMA node: progress: 346.0 s, 6.0 tps, lat 142.502 ms stddev 12.621, 0 failed progress: 347.0 s, 6.0 tps, lat 168.971 ms stddev 20.585, 0 failed progress: 348.0 s, 7.0 tps, lat 157.017 ms stddev 9.333, 0 failed progress: 349.0 s, 6.0 tps, lat 157.810 ms stddev 15.124, 0 failed progress: 350.0 s, 7.0 tps, lat 150.623 ms stddev 1.246, 0 failed progress: 351.0 s, 6.0 tps, lat 156.901 ms stddev 15.012, 0 failed 2 PQ workers on the local NUMA node to s_b + leader process forced to remote NUMA node: progress: 554.0 s, 7.0 tps, lat 130.552 ms stddev 0.256, 0 failed progress: 555.0 s, 8.0 tps, lat 130.488 ms stddev 0.144, 0 failed progress: 556.0 s, 8.0 tps, lat 130.795 ms stddev 0.171, 0 failed progress: 557.0 s, 7.0 tps, lat 130.759 ms stddev 0.250, 0 failed progress: 558.0 s, 8.0 tps, lat 130.601 ms stddev 0.217, 0 failed progress: 559.0 s, 8.0 tps, lat 130.606 ms stddev 0.151, 0 failed progress: 560.0 s, 7.0 tps, lat 130.503 ms stddev 0.217, 0 failed 2 PQ workers on remote NUMA node to s_b + leader process on local progress: 627.0 s, 6.0 tps, lat 150.555 ms stddev 0.147, 0 failed progress: 628.0 s, 7.0 tps, lat 150.675 ms stddev 0.484, 0 failed progress: 629.0 s, 7.0 tps, lat 150.617 ms stddev 0.324, 0 failed progress: 630.0 s, 6.0 tps, lat 150.410 ms stddev 0.087, 0 failed progress: 631.0 s, 7.0 tps, lat 150.493 ms stddev 0.150, 0 failed progress: 632.0 s, 7.0 tps, lat 150.566 ms stddev 0.095, 0 failed With PQ (2 workers and leader participating) with numa=on (interleaved s_b), CPU affinity default, stddev flunctuates 2-23ms progress: 39.0 s, 6.0 tps, lat 175.806 ms stddev 10.968, 0 failed progress: 40.0 s, 6.0 tps, lat 172.568 ms stddev 10.002, 0 failed progress: 41.0 s, 5.0 tps, lat 175.535 ms stddev 9.343, 0 failed progress: 42.0 s, 6.0 tps, lat 170.761 ms stddev 6.165, 0 failed progress: 43.0 s, 6.0 tps, lat 164.163 ms stddev 1.575, 0 failed progress: 44.0 s, 6.0 tps, lat 175.584 ms stddev 13.247, 0 failed progress: 45.0 s, 6.0 tps, lat 162.442 ms stddev 0.175, 0 failed but we have 4x (number of sockets) more CPU power available at the same latency/bandwidth. So if anything: - latency-wise: it would be best to place leader+all PQ workers close to s_b, provided s_b fits NUMA shared/huge page memory there and you won't need more CPU than there's on that NUMA node... (assuming e.g. hosting 4 DBs on 4-sockets each on it's own, it would be best to pin everything including shm, but PQ workers too) - capacity/TPS-wise or s_b > NUMA: just interleave to maximize bandwidth and get uniform CPU performance out of this Attachments: [text/plain] NUMA_pq_cpu_pinning_results.txt (8.2K, ../../CAKZiRmxYMPbQ4WiyJWh=Vuw_Ny+hLGH9_9FaacKRJvzZ-smm+w@mail.gmail.com/2-NUMA_pq_cpu_pinning_results.txt) download | inline: On legacy 4s/NUMA simple simplest query to `select sum(aid) from pgbench_accounts_1 (single partition, 400MB)` with pgbench -c 1 client and small s_b 4GB just taking single NUMA node, gets this: Finalize Aggregate (cost=72150.70..72150.71 rows=1 width=8) (actual time=252.817..254.409 rows=1.00 loops=1) Output: sum(aid) Buffers: shared hit=54798 -> Gather (cost=72150.49..72150.69 rows=2 width=8) (actual time=252.319..254.381 rows=3.00 loops=1) Output: (PARTIAL sum(aid)) Workers Planned: 2 Workers Launched: 2 Buffers: shared hit=54798 -> Partial Aggregate (cost=71150.49..71150.49 rows=1 width=8) (actual time=247.262..247.264 rows=1.00 loops=3) Output: PARTIAL sum(aid) Buffers: shared hit=54798 Worker 0: actual time=245.116..245.118 rows=1.00 loops=1 Buffers: shared hit=19185 Worker 1: actual time=244.797..244.800 rows=1.00 loops=1 Buffers: shared hit=19021 -> Parallel Seq Scan on public.pgbench_accounts_1 (cost=0.00..67879.99 rows=1308199 width=4) (actual time=0.011..136.895 rows=1041666.67 loop s=3) Output: aid, bid, abalance, filler Buffers: shared hit=54798 Worker 0: actual time=0.010..133.393 rows=1094053.00 loops=1 Buffers: shared hit=19185 Worker 1: actual time=0.009..134.281 rows=1084465.00 loops=1 Buffers: shared hit=19021 Observe "lat"ency metric , it primary depends on the where the process are placed: no PQ, default behavoir: progress: 57.0 s, 2.0 tps, lat 515.312 ms stddev 0.053, 0 failed progress: 58.0 s, 2.0 tps, lat 609.009 ms stddev 92.653, 0 failed progress: 59.0 s, 2.0 tps, lat 515.414 ms stddev 0.423, 0 failed progress: 60.0 s, 2.0 tps, lat 515.530 ms stddev 0.138, 0 failed progress: 61.0 s, 2.0 tps, lat 515.370 ms stddev 0.181, 0 failed progress: 62.0 s, 2.0 tps, lat 515.396 ms stddev 0.351, 0 failed progress: 63.0 s, 1.0 tps, lat 515.484 ms stddev NaN, 0 failed progress: 64.0 s, 2.0 tps, lat 515.428 ms stddev 0.035, 0 failed progress: 65.0 s, 2.0 tps, lat 515.357 ms stddev 0.089, 0 failed progress: 66.0 s, 2.0 tps, lat 608.885 ms stddev 92.633, 0 failed progress: 67.0 s, 2.0 tps, lat 515.828 ms stddev 0.501, 0 failed progress: 68.0 s, 1.0 tps, lat 603.824 ms stddev NaN, 0 failed no PQ, manual pinning (above it flunctuates a lot due to migrations): progress: 318.0 s, 3.0 tps, lat 321.714 ms stddev 0.089, 0 failed <-- bind backend to CPU on the same NUMA node, gets 1.4GB/s DRAM read on that whole socket/OS progress: 319.0 s, 3.0 tps, lat 321.630 ms stddev 0.254, 0 failed progress: 320.0 s, 3.0 tps, lat 321.655 ms stddev 0.131, 0 failed progress: 321.0 s, 3.0 tps, lat 321.555 ms stddev 0.132, 0 failed progress: 322.0 s, 3.0 tps, lat 322.027 ms stddev 0.964, 0 failed progress: 323.0 s, 2.0 tps, lat 426.245 ms stddev 49.599, 0 failed progress: 324.0 s, 2.0 tps, lat 519.873 ms stddev 0.160, 0 failed <-- bind query to CPU on different NUMA node (hits interconnect), gets only ~0.8GB/s read on that socket with s_b progress: 325.0 s, 2.0 tps, lat 519.854 ms stddev 0.488, 0 failed progress: 326.0 s, 2.0 tps, lat 519.604 ms stddev 0.285, 0 failed progress: 327.0 s, 2.0 tps, lat 519.626 ms stddev 0.190, 0 failed progress: 328.0 s, 2.0 tps, lat 520.172 ms stddev 0.064, 0 failed progress: 329.0 s, 3.0 tps, lat 409.836 ms stddev 41.466, 0 failed progress: 330.0 s, 3.0 tps, lat 325.236 ms stddev 0.248, 0 failed <-- bind backend to CPU on the same NUMA node again progress: 331.0 s, 3.0 tps, lat 325.190 ms stddev 0.261, 0 failed progress: 332.0 s, 3.0 tps, lat 325.485 ms stddev 0.751, 0 failed progress: 333.0 s, 3.0 tps, lat 325.026 ms stddev 0.065, 0 failed 520/325 ms = 1.6x; IPC jumps from 1.23 to ~2.00 just due to switch from remote to local DRAM memory access (latency) no PQ and numa=on (s_b interleaved), STABLE (low stddev), notice how optimal was ~321ms vs interleaved gets ~463ms yes it is wrose (321->463ms), but we have uniform latency, instead of high variance depending on how scheduler assigns CPU: progress: 10.0 s, 2.0 tps, lat 463.378 ms stddev 0.181, 0 failed progress: 11.0 s, 2.0 tps, lat 463.422 ms stddev 0.016, 0 failed progress: 12.0 s, 2.0 tps, lat 463.121 ms stddev 0.052, 0 failed progress: 13.0 s, 2.0 tps, lat 463.198 ms stddev 0.148, 0 failed progress: 14.0 s, 3.0 tps, lat 463.823 ms stddev 0.854, 0 failed progress: 15.0 s, 2.0 tps, lat 462.937 ms stddev 0.199, 0 failed ok let's enable PQ: 2 workers and leader participating, default: progress: 30.0 s, 7.0 tps, lat 134.692 ms stddev 4.644, 0 failed progress: 31.0 s, 8.0 tps, lat 129.489 ms stddev 1.510, 0 failed progress: 32.0 s, 7.0 tps, lat 130.506 ms stddev 3.393, 0 failed progress: 33.0 s, 8.0 tps, lat 133.785 ms stddev 1.921, 0 failed progress: 34.0 s, 7.0 tps, lat 134.561 ms stddev 6.965, 0 failed progress: 35.0 s, 8.0 tps, lat 135.438 ms stddev 5.286, 0 failed progress: 36.0 s, 7.0 tps, lat 141.760 ms stddev 15.878, 0 failed progress: 37.0 s, 7.0 tps, lat 136.749 ms stddev 5.937, 0 failed progress: 38.0 s, 7.0 tps, lat 138.258 ms stddev 8.666, 0 failed progress: 39.0 s, 7.0 tps, lat 140.263 ms stddev 8.834, 0 failed 2 workers and leader participating and postmaster CPU affinity (therefore any new pq workers) set to only CPUs from nodes having s_b + CPU-pin the leader there too: progress: 342.0 s, 9.0 tps, lat 117.306 ms stddev 0.152, 0 failed progress: 343.0 s, 8.0 tps, lat 117.265 ms stddev 0.216, 0 failed progress: 344.0 s, 9.0 tps, lat 117.216 ms stddev 0.221, 0 failed progress: 345.0 s, 9.0 tps, lat 117.233 ms stddev 0.155, 0 failed postmaster's CPU affinity (so for 2 PQ workers) being forced to remote NUMA node: progress: 346.0 s, 6.0 tps, lat 142.502 ms stddev 12.621, 0 failed progress: 347.0 s, 6.0 tps, lat 168.971 ms stddev 20.585, 0 failed progress: 348.0 s, 7.0 tps, lat 157.017 ms stddev 9.333, 0 failed progress: 349.0 s, 6.0 tps, lat 157.810 ms stddev 15.124, 0 failed progress: 350.0 s, 7.0 tps, lat 150.623 ms stddev 1.246, 0 failed progress: 351.0 s, 6.0 tps, lat 156.901 ms stddev 15.012, 0 failed 2 PQ workers on the local NUMA node to s_b + leader process forced to remote NUMA node: progress: 554.0 s, 7.0 tps, lat 130.552 ms stddev 0.256, 0 failed progress: 555.0 s, 8.0 tps, lat 130.488 ms stddev 0.144, 0 failed progress: 556.0 s, 8.0 tps, lat 130.795 ms stddev 0.171, 0 failed progress: 557.0 s, 7.0 tps, lat 130.759 ms stddev 0.250, 0 failed progress: 558.0 s, 8.0 tps, lat 130.601 ms stddev 0.217, 0 failed progress: 559.0 s, 8.0 tps, lat 130.606 ms stddev 0.151, 0 failed progress: 560.0 s, 7.0 tps, lat 130.503 ms stddev 0.217, 0 failed 2 PQ workers on remote NUMA node to s_b + leader process on local progress: 627.0 s, 6.0 tps, lat 150.555 ms stddev 0.147, 0 failed progress: 628.0 s, 7.0 tps, lat 150.675 ms stddev 0.484, 0 failed progress: 629.0 s, 7.0 tps, lat 150.617 ms stddev 0.324, 0 failed progress: 630.0 s, 6.0 tps, lat 150.410 ms stddev 0.087, 0 failed progress: 631.0 s, 7.0 tps, lat 150.493 ms stddev 0.150, 0 failed progress: 632.0 s, 7.0 tps, lat 150.566 ms stddev 0.095, 0 failed With PQ (2 workers and leader participating) with numa=on (interleaved s_b), CPU affinity default, stddev flunctuates 2-23ms progress: 39.0 s, 6.0 tps, lat 175.806 ms stddev 10.968, 0 failed progress: 40.0 s, 6.0 tps, lat 172.568 ms stddev 10.002, 0 failed progress: 41.0 s, 5.0 tps, lat 175.535 ms stddev 9.343, 0 failed progress: 42.0 s, 6.0 tps, lat 170.761 ms stddev 6.165, 0 failed progress: 43.0 s, 6.0 tps, lat 164.163 ms stddev 1.575, 0 failed progress: 44.0 s, 6.0 tps, lat 175.584 ms stddev 13.247, 0 failed progress: 45.0 s, 6.0 tps, lat 162.442 ms stddev 0.175, 0 failed but we have 4x (number of sockets) more CPU power available at the same latency/bandwidth. So if anything: - latency-wise: it would be best to place leader+all PQ workers close to s_b, provided s_b fits NUMA shared/huge page memory there and you won't need more CPU than there's on that NUMA node... (assuming e.g. hosting 4 DBs on 4-sockets each on it's own, it would be best to pin everything including shm, but PQ workers too) - capacity/TPS-wise or s_b > NUMA: just interleave to maximize bandwidth and get uniform CPU performance out of this ^ permalink raw reply [nested|flat] 4+ messages in thread
end of thread, other threads:[~2025-06-26 11:36 UTC | newest] Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2019-01-31 21:37 [PATCH v33 1/5] Base implementation of subscripting mechanism erthalion <[email protected]> 2025-04-16 13:58 Re: NUMA shared memory interleaving Thomas Munro <[email protected]> 2025-04-18 17:48 ` Re: NUMA shared memory interleaving Bertrand Drouvot <[email protected]> 2025-06-26 11:36 ` Re: NUMA shared memory interleaving Jakub Wartak <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox