Received: from malur.postgresql.org ([2a02:16a8:dc51::56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.89) (envelope-from ) id 1fSxCa-0006Fb-QX for pgsql-hackers@arkaria.postgresql.org; Wed, 13 Jun 2018 04:16:09 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1fSxCZ-0005h7-FO for pgsql-hackers@arkaria.postgresql.org; Wed, 13 Jun 2018 04:16:07 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.89) (envelope-from ) id 1fSxCZ-0005gx-7B for pgsql-hackers@lists.postgresql.org; Wed, 13 Jun 2018 04:16:07 +0000 Received: from sss.pgh.pa.us ([66.207.139.130]) by magus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.89) (envelope-from ) id 1fSxCV-0002I6-1S for pgsql-hackers@lists.postgresql.org; Wed, 13 Jun 2018 04:16:06 +0000 Received: from sss1.sss.pgh.pa.us (localhost [127.0.0.1]) by sss.pgh.pa.us (8.14.4/8.14.4) with ESMTP id w5D4Flnp000638; Wed, 13 Jun 2018 00:15:47 -0400 From: Tom Lane To: Robert Haas cc: David Rowley , Amit Langote , Dmitry Dolgov <9erthalion6@gmail.com>, Jeff Janes , Ashutosh Bapat , Pavel Stehule , PostgreSQL Hackers Subject: Re: why partition pruning doesn't work? In-reply-to: References: <7142.1527868419@sss.pgh.pa.us> <1111.1527977170@sss.pgh.pa.us> <10432.1528045900@sss.pgh.pa.us> <569483c5-a273-6537-1b5d-7a271ce439fb@lab.ntt.co.jp> <22805.1528386192@sss.pgh.pa.us> <21787.1528428171@sss.pgh.pa.us> <26279.1528670981@sss.pgh.pa.us> <17850.1528755844@sss.pgh.pa.us> <19365.1528820758@sss.pgh.pa.us> <30981.1528822481@sss.pgh.pa.us> Comments: In-reply-to Robert Haas message dated "Tue, 12 Jun 2018 16:01:39 -0400" MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----- =_aaaaaaaaaa0" Content-ID: <555.1528863299.0@sss.pgh.pa.us> Date: Wed, 13 Jun 2018 00:15:47 -0400 Message-ID: <637.1528863347@sss.pgh.pa.us> List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk ------- =_aaaaaaaaaa0 Content-Type: text/plain; charset="us-ascii" Content-ID: <555.1528863299.1@sss.pgh.pa.us> Robert Haas writes: > On Tue, Jun 12, 2018 at 12:54 PM, Tom Lane wrote: >> While I've not looked into the exact reasons for that, my first guess >> is that the partitioned table is not held open because it's not one >> of the ones to be scanned. Are you prepared to change something like >> that at this stage of the release cycle? > The partition key is immutable, so it should NOT be able to disappear > out from under us. Hm. That could be better documented. > As for whether to change it at this point in the release cycle, I > guess that, to me, depends on how invasive the fix is. It seems not to be that bad: we just need a shutdown call for the PartitionPruneState, and then we can remember the open relation there. The attached is based on David's patch from yesterday. I'm still a bit annoyed at the fmgr_info_copy calls in this. It'd be better to use the FmgrInfos in the relcache when applicable. However, mixing those with the cross-type ones would seem to require that we change the API for get_matching_hash_bounds et al from taking "FmgrInfo *" to taking "FmgrInfo **", which looks rather invasive. regards, tom lane ------- =_aaaaaaaaaa0 Content-Type: text/x-diff; name="fix_unsafe_relcache_partition_data_use.patch"; charset="us-ascii" Content-ID: <555.1528863299.2@sss.pgh.pa.us> Content-Description: fix_unsafe_relcache_partition_data_use.patch Content-Transfer-Encoding: quoted-printable diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/e= xecPartition.c index 33513ff..4eeee7c 100644 *** a/src/backend/executor/execPartition.c --- b/src/backend/executor/execPartition.c *************** adjust_partition_tlist(List *tlist, Tupl *** 1357,1367 **** * * Functions: * ! * ExecSetupPartitionPruneState: * Creates the PartitionPruneState required by each of the two pruning * functions. Details stored include how to map the partition index * returned by the partition pruning code into subplan indexes. * * ExecFindInitialMatchingSubPlans: * Returns indexes of matching subplans. Partition pruning is attempte= d * without any evaluation of expressions containing PARAM_EXEC Params. --- 1357,1370 ---- * * Functions: * ! * ExecCreatePartitionPruneState: * Creates the PartitionPruneState required by each of the two pruning * functions. Details stored include how to map the partition index * returned by the partition pruning code into subplan indexes. * + * ExecDestroyPartitionPruneState: + * Deletes a PartitionPruneState. Must be called during executor shutdo= wn. + * * ExecFindInitialMatchingSubPlans: * Returns indexes of matching subplans. Partition pruning is attempte= d * without any evaluation of expressions containing PARAM_EXEC Params. *************** adjust_partition_tlist(List *tlist, Tupl *** 1382,1389 **** */ = /* ! * ExecSetupPartitionPruneState ! * Set up the data structure required for calling * ExecFindInitialMatchingSubPlans and ExecFindMatchingSubPlans. * * 'planstate' is the parent plan node's execution state. --- 1385,1392 ---- */ = /* ! * ExecCreatePartitionPruneState ! * Build the data structure required for calling * ExecFindInitialMatchingSubPlans and ExecFindMatchingSubPlans. * * 'planstate' is the parent plan node's execution state. *************** adjust_partition_tlist(List *tlist, Tupl *** 1395,1401 **** * in each PartitionPruneInfo. */ PartitionPruneState * ! ExecSetupPartitionPruneState(PlanState *planstate, List *partitionprunei= nfo) { PartitionPruneState *prunestate; PartitionPruningData *prunedata; --- 1398,1404 ---- * in each PartitionPruneInfo. */ PartitionPruneState * ! ExecCreatePartitionPruneState(PlanState *planstate, List *partitionprune= info) { PartitionPruneState *prunestate; PartitionPruningData *prunedata; *************** ExecSetupPartitionPruneState(PlanState * *** 1435,1445 **** PartitionPruningData *pprune =3D &prunedata[i]; PartitionPruneContext *context =3D &pprune->context; PartitionDesc partdesc; - Relation rel; PartitionKey partkey; - ListCell *lc2; int partnatts; int n_steps; = /* * We must copy the subplan_map rather than pointing directly to the --- 1438,1447 ---- PartitionPruningData *pprune =3D &prunedata[i]; PartitionPruneContext *context =3D &pprune->context; PartitionDesc partdesc; PartitionKey partkey; int partnatts; int n_steps; + ListCell *lc2; = /* * We must copy the subplan_map rather than pointing directly to the *************** ExecSetupPartitionPruneState(PlanState * *** 1456,1481 **** pprune->present_parts =3D bms_copy(pinfo->present_parts); = /* ! * Grab some info from the table's relcache; lock was already obtained ! * by ExecLockNonLeafAppendTables. */ ! rel =3D relation_open(pinfo->reloid, NoLock); = ! partkey =3D RelationGetPartitionKey(rel); ! partdesc =3D RelationGetPartitionDesc(rel); = context->strategy =3D partkey->strategy; context->partnatts =3D partnatts =3D partkey->partnatts; ! context->partopfamily =3D partkey->partopfamily; ! context->partopcintype =3D partkey->partopcintype; context->partcollation =3D partkey->partcollation; context->partsupfunc =3D partkey->partsupfunc; ! context->nparts =3D pinfo->nparts; ! context->boundinfo =3D partition_bounds_copy(partdesc->boundinfo, part= key); context->planstate =3D planstate; = /* Initialize expression state for each expression we need */ - n_steps =3D list_length(pinfo->pruning_steps); context->exprstates =3D (ExprState **) palloc0(sizeof(ExprState *) * n_steps * partnatts); foreach(lc2, pinfo->pruning_steps) --- 1458,1490 ---- pprune->present_parts =3D bms_copy(pinfo->present_parts); = /* ! * We need to hold a pin on the partitioned table's relcache entry so ! * that we can rely on its copies of the table's partition key and ! * partition descriptor. We need not get a lock though; one should ! * have been acquired already by InitPlan or ! * ExecLockNonLeafAppendTables. */ ! context->partrel =3D relation_open(pinfo->reloid, NoLock); = ! partkey =3D RelationGetPartitionKey(context->partrel); ! partdesc =3D RelationGetPartitionDesc(context->partrel); ! n_steps =3D list_length(pinfo->pruning_steps); = context->strategy =3D partkey->strategy; context->partnatts =3D partnatts =3D partkey->partnatts; ! context->nparts =3D pinfo->nparts; ! context->boundinfo =3D partdesc->boundinfo; context->partcollation =3D partkey->partcollation; context->partsupfunc =3D partkey->partsupfunc; ! = ! /* We'll look up type-specific support functions as needed */ ! context->stepcmpfuncs =3D (FmgrInfo *) ! palloc0(sizeof(FmgrInfo) * n_steps * partnatts); ! = ! context->ppccontext =3D CurrentMemoryContext; context->planstate =3D planstate; = /* Initialize expression state for each expression we need */ context->exprstates =3D (ExprState **) palloc0(sizeof(ExprState *) * n_steps * partnatts); foreach(lc2, pinfo->pruning_steps) *************** ExecSetupPartitionPruneState(PlanState * *** 1527,1534 **** prunestate->execparamids =3D bms_add_members(prunestate->execparamids, pinfo->execparamids); = - relation_close(rel, NoLock); - = i++; } = --- 1536,1541 ---- *************** ExecSetupPartitionPruneState(PlanState * *** 1536,1541 **** --- 1543,1568 ---- } = /* + * ExecDestroyPartitionPruneState + * Release resources at plan shutdown. + * + * We don't bother to free any memory here, since the whole executor con= text + * will be going away shortly. We do need to release our relcache pins. + */ + void + ExecDestroyPartitionPruneState(PartitionPruneState *prunestate) + { + int i; + = + for (i =3D 0; i < prunestate->num_partprunedata; i++) + { + PartitionPruningData *pprune =3D &prunestate->partprunedata[i]; + = + relation_close(pprune->context.partrel, NoLock); + } + } + = + /* * ExecFindInitialMatchingSubPlans * Identify the set of subplans that cannot be eliminated by initial * pruning (disregarding any pruning constraints involving PARAM_EXEC diff --git a/src/backend/executor/nodeAppend.c b/src/backend/executor/node= Append.c index 6dd53e9..5ce4fb4 100644 *** a/src/backend/executor/nodeAppend.c --- b/src/backend/executor/nodeAppend.c *************** ExecInitAppend(Append *node, EState *est *** 136,143 **** /* We may need an expression context to evaluate partition exprs */ ExecAssignExprContext(estate, &appendstate->ps); = ! prunestate =3D ExecSetupPartitionPruneState(&appendstate->ps, ! node->part_prune_infos); = /* Perform an initial partition prune, if required. */ if (prunestate->do_initial_prune) --- 136,145 ---- /* We may need an expression context to evaluate partition exprs */ ExecAssignExprContext(estate, &appendstate->ps); = ! /* Create the working data structure for pruning. */ ! prunestate =3D ExecCreatePartitionPruneState(&appendstate->ps, ! node->part_prune_infos); ! appendstate->as_prune_state =3D prunestate; = /* Perform an initial partition prune, if required. */ if (prunestate->do_initial_prune) *************** ExecInitAppend(Append *node, EState *est *** 178,185 **** */ if (!prunestate->do_exec_prune) appendstate->as_valid_subplans =3D bms_add_range(NULL, 0, nplans - 1)= ; - = - appendstate->as_prune_state =3D prunestate; } else { --- 180,185 ---- *************** ExecEndAppend(AppendState *node) *** 330,335 **** --- 330,341 ---- */ for (i =3D 0; i < nplans; i++) ExecEndNode(appendplans[i]); + = + /* + * release any resources associated with run-time pruning + */ + if (node->as_prune_state) + ExecDestroyPartitionPruneState(node->as_prune_state); } = void diff --git a/src/backend/partitioning/partprune.c b/src/backend/partitioni= ng/partprune.c index 856bdd3..ee5826e 100644 *** a/src/backend/partitioning/partprune.c --- b/src/backend/partitioning/partprune.c *************** prune_append_rel_partitions(RelOptInfo * *** 436,449 **** if (contradictory) return NULL; = context.strategy =3D rel->part_scheme->strategy; context.partnatts =3D rel->part_scheme->partnatts; - context.partopfamily =3D rel->part_scheme->partopfamily; - context.partopcintype =3D rel->part_scheme->partopcintype; - context.partcollation =3D rel->part_scheme->partcollation; - context.partsupfunc =3D rel->part_scheme->partsupfunc; context.nparts =3D rel->nparts; context.boundinfo =3D rel->boundinfo; = /* These are not valid when being called from the planner */ context.planstate =3D NULL; --- 436,453 ---- if (contradictory) return NULL; = + /* Set up PartitionPruneContext */ + context.partrel =3D NULL; context.strategy =3D rel->part_scheme->strategy; context.partnatts =3D rel->part_scheme->partnatts; context.nparts =3D rel->nparts; context.boundinfo =3D rel->boundinfo; + context.partcollation =3D rel->part_scheme->partcollation; + context.partsupfunc =3D rel->part_scheme->partsupfunc; + context.stepcmpfuncs =3D (FmgrInfo *) palloc0(sizeof(FmgrInfo) * + context.partnatts * + list_length(pruning_steps)); + context.ppccontext =3D CurrentMemoryContext; = /* These are not valid when being called from the planner */ context.planstate =3D NULL; *************** perform_pruning_base_step(PartitionPrune *** 2809,2815 **** int keyno, nvalues; Datum values[PARTITION_MAX_KEYS]; ! FmgrInfo partsupfunc[PARTITION_MAX_KEYS]; = /* * There better be the same number of expressions and compare functions= . --- 2813,2820 ---- int keyno, nvalues; Datum values[PARTITION_MAX_KEYS]; ! FmgrInfo *partsupfunc; ! int stateidx; = /* * There better be the same number of expressions and compare functions= . *************** perform_pruning_base_step(PartitionPrune *** 2844,2850 **** if (lc1 !=3D NULL) { Expr *expr; - int stateidx; Datum datum; bool isnull; = --- 2849,2854 ---- *************** perform_pruning_base_step(PartitionPrune *** 2873,2891 **** return result; } = ! /* ! * If we're going to need a different comparison function than ! * the one cached in the PartitionKey, we'll need to look up ! * the FmgrInfo. ! */ cmpfn =3D lfirst_oid(lc2); Assert(OidIsValid(cmpfn)); ! if (cmpfn !=3D context->partsupfunc[keyno].fn_oid) ! fmgr_info(cmpfn, &partsupfunc[keyno]); ! else ! fmgr_info_copy(&partsupfunc[keyno], ! &context->partsupfunc[keyno], ! CurrentMemoryContext); = values[keyno] =3D datum; nvalues++; --- 2877,2901 ---- return result; } = ! /* Set up the stepcmpfuncs entry, unless we already did */ cmpfn =3D lfirst_oid(lc2); Assert(OidIsValid(cmpfn)); ! if (cmpfn !=3D context->stepcmpfuncs[stateidx].fn_oid) ! { ! /* ! * If the needed support function is the same one cached ! * in the relation's partition key, copy the cached ! * FmgrInfo. Otherwise (i.e., when we have a cross-type ! * comparison), an actual lookup is required. ! */ ! if (cmpfn =3D=3D context->partsupfunc[keyno].fn_oid) ! fmgr_info_copy(&context->stepcmpfuncs[stateidx], ! &context->partsupfunc[keyno], ! context->ppccontext); ! else ! fmgr_info_cxt(cmpfn, &context->stepcmpfuncs[stateidx], ! context->ppccontext); ! } = values[keyno] =3D datum; nvalues++; *************** perform_pruning_base_step(PartitionPrune *** 2896,2901 **** --- 2906,2918 ---- } } = + /* + * Point partsupfunc to the entry for the 0th key of this step; the + * additional support functions, if any, follow consecutively. + */ + stateidx =3D PruneCxtStateIdx(context->partnatts, opstep->step.step_id,= 0); + partsupfunc =3D &context->stepcmpfuncs[stateidx]; + = switch (context->strategy) { case PARTITION_STRATEGY_HASH: diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/= relcache.c index 3dfb1b8..dc69280 100644 *** a/src/backend/utils/cache/relcache.c --- b/src/backend/utils/cache/relcache.c *************** RelationClearRelation(Relation relation, *** 2401,2406 **** --- 2401,2407 ---- keep_tupdesc =3D equalTupleDescs(relation->rd_att, newrel->rd_att); keep_rules =3D equalRuleLocks(relation->rd_rules, newrel->rd_rules); keep_policies =3D equalRSDesc(relation->rd_rsdesc, newrel->rd_rsdesc); + /* partkey is immutable once set up, so we can always keep it */ keep_partkey =3D (relation->rd_partkey !=3D NULL); keep_partdesc =3D equalPartitionDescs(relation->rd_partkey, relation->rd_partdesc, *************** RelationClearRelation(Relation relation, *** 2445,2451 **** SWAPFIELD(Form_pg_class, rd_rel); /* ... but actually, we don't have to update newrel->rd_rel */ memcpy(relation->rd_rel, newrel->rd_rel, CLASS_TUPLE_SIZE); ! /* preserve old tupledesc and rules if no logical change */ if (keep_tupdesc) SWAPFIELD(TupleDesc, rd_att); if (keep_rules) --- 2446,2452 ---- SWAPFIELD(Form_pg_class, rd_rel); /* ... but actually, we don't have to update newrel->rd_rel */ memcpy(relation->rd_rel, newrel->rd_rel, CLASS_TUPLE_SIZE); ! /* preserve old tupledesc, rules, policies if no logical change */ if (keep_tupdesc) SWAPFIELD(TupleDesc, rd_att); if (keep_rules) *************** RelationClearRelation(Relation relation, *** 2459,2471 **** SWAPFIELD(Oid, rd_toastoid); /* pgstat_info must be preserved */ SWAPFIELD(struct PgStat_TableStatus *, pgstat_info); ! /* partition key must be preserved, if we have one */ if (keep_partkey) { SWAPFIELD(PartitionKey, rd_partkey); SWAPFIELD(MemoryContext, rd_partkeycxt); } - /* preserve old partdesc if no logical change */ if (keep_partdesc) { SWAPFIELD(PartitionDesc, rd_partdesc); --- 2460,2471 ---- SWAPFIELD(Oid, rd_toastoid); /* pgstat_info must be preserved */ SWAPFIELD(struct PgStat_TableStatus *, pgstat_info); ! /* preserve old partitioning info if no logical change */ if (keep_partkey) { SWAPFIELD(PartitionKey, rd_partkey); SWAPFIELD(MemoryContext, rd_partkeycxt); } if (keep_partdesc) { SWAPFIELD(PartitionDesc, rd_partdesc); diff --git a/src/include/executor/execPartition.h b/src/include/executor/e= xecPartition.h index 71d639f..862bf65 100644 *** a/src/include/executor/execPartition.h --- b/src/include/executor/execPartition.h *************** extern HeapTuple ConvertPartitionTupleSl *** 208,215 **** TupleTableSlot **p_my_slot); extern void ExecCleanupTupleRouting(ModifyTableState *mtstate, PartitionTupleRouting *proute); ! extern PartitionPruneState *ExecSetupPartitionPruneState(PlanState *plan= state, ! List *partitionpruneinfo); extern Bitmapset *ExecFindMatchingSubPlans(PartitionPruneState *prunesta= te); extern Bitmapset *ExecFindInitialMatchingSubPlans(PartitionPruneState *p= runestate, int nsubplans); --- 208,216 ---- TupleTableSlot **p_my_slot); extern void ExecCleanupTupleRouting(ModifyTableState *mtstate, PartitionTupleRouting *proute); ! extern PartitionPruneState *ExecCreatePartitionPruneState(PlanState *pla= nstate, ! List *partitionpruneinfo); ! extern void ExecDestroyPartitionPruneState(PartitionPruneState *prunesta= te); extern Bitmapset *ExecFindMatchingSubPlans(PartitionPruneState *prunesta= te); extern Bitmapset *ExecFindInitialMatchingSubPlans(PartitionPruneState *p= runestate, int nsubplans); diff --git a/src/include/partitioning/partprune.h b/src/include/partitioni= ng/partprune.h index e3b3bfb..09147b5 100644 *** a/src/include/partitioning/partprune.h --- b/src/include/partitioning/partprune.h *************** *** 20,68 **** = /* * PartitionPruneContext * ! * Information about a partitioned table needed to perform partition pru= ning. */ typedef struct PartitionPruneContext { ! /* Partition key information */ char strategy; int partnatts; - Oid *partopfamily; - Oid *partopcintype; - Oid *partcollation; - FmgrInfo *partsupfunc; - = - /* Number of partitions */ int nparts; - = - /* Partition boundary info */ PartitionBoundInfo boundinfo; ! = ! /* ! * This will be set when the context is used from the executor, to allo= w ! * Params to be evaluated. ! */ PlanState *planstate; - = - /* - * Array of ExprStates, indexed as per PruneCtxStateIdx; one for each - * partkey in each pruning step. Allocated if planstate is non-NULL, - * otherwise NULL. - */ ExprState **exprstates; - = - /* - * Similar array of flags, each true if corresponding 'exprstate' - * expression contains any PARAM_EXEC Params. (Can be NULL if planstat= e - * is NULL.) - */ bool *exprhasexecparam; - = - /* true if it's safe to evaluate PARAM_EXEC Params */ bool evalexecparams; } PartitionPruneContext; = #define PruneCxtStateIdx(partnatts, step_id, keyno) \ ((partnatts) * (step_id) + (keyno)) = --- 20,76 ---- = /* * PartitionPruneContext + * Stores information needed at runtime for pruning computations + * related to a single partitioned table. * ! * partrel Relcache pointer for the partitioned table, ! * if we have it open (else NULL). ! * strategy Partition strategy, e.g. LIST, RANGE, HASH. ! * partnatts Number of columns in the partition key. ! * nparts Number of partitions in this partitioned table. ! * boundinfo Partition boundary info for the partitioned table. ! * partcollation Array of partnatts elements, storing the collations of = the ! * partition key columns. ! * partsupfunc Array of FmgrInfos for the comparison or hashing functio= ns ! * associated with the partition keys (partnatts elements). ! * (This points into the partrel's partition key, typically.) ! * stepcmpfuncs Array of FmgrInfos for the comparison or hashing functi= on ! * for each pruning step and partition key. ! * ppccontext Memory context holding this PartitionPruneContext's ! * subsidiary data, such as the FmgrInfos. ! * planstate Points to the parent plan node's PlanState when called ! * during execution; NULL when called from the planner. ! * exprstates Array of ExprStates, indexed as per PruneCtxStateIdx; one ! * for each partition key in each pruning step. Allocated if ! * planstate is non-NULL, otherwise NULL. ! * exprhasexecparam Array of bools, each true if corresponding 'exprstat= e' ! * expression contains any PARAM_EXEC Params. (Can be NULL ! * if planstate is NULL.) ! * evalexecparams True if it's safe to evaluate PARAM_EXEC Params. */ typedef struct PartitionPruneContext { ! Relation partrel; char strategy; int partnatts; int nparts; PartitionBoundInfo boundinfo; ! Oid *partcollation; ! FmgrInfo *partsupfunc; ! FmgrInfo *stepcmpfuncs; ! MemoryContext ppccontext; PlanState *planstate; ExprState **exprstates; bool *exprhasexecparam; bool evalexecparams; } PartitionPruneContext; = + /* + * PruneCxtStateIdx() computes the correct index into the stepcmpfuncs[]= , + * exprstates[] and exprhasexecparam[] arrays for step step_id and + * partition key column keyno. (Note: there is code that assumes the + * entries for a given step are sequential, so this is not chosen freely= .) + */ #define PruneCxtStateIdx(partnatts, step_id, keyno) \ ((partnatts) * (step_id) + (keyno)) = ------- =_aaaaaaaaaa0--