public inbox for [email protected]help / color / mirror / Atom feed
[PATCH v1 2/2] ALTER TABLE ... DETACH CONCURRENTLY 5+ messages / 3 participants [nested] [flat]
* [PATCH v1 2/2] ALTER TABLE ... DETACH CONCURRENTLY @ 2020-08-03 23:21 Alvaro Herrera <[email protected]> 0 siblings, 0 replies; 5+ messages in thread From: Alvaro Herrera @ 2020-08-03 23:21 UTC (permalink / raw) --- doc/src/sgml/catalogs.sgml | 10 + doc/src/sgml/ref/alter_table.sgml | 25 +- src/backend/catalog/heap.c | 13 +- src/backend/catalog/index.c | 4 +- src/backend/catalog/pg_inherits.c | 51 +- src/backend/commands/indexcmds.c | 5 +- src/backend/commands/tablecmds.c | 491 ++++++++++++++---- src/backend/commands/trigger.c | 7 +- src/backend/executor/execPartition.c | 16 +- src/backend/nodes/copyfuncs.c | 1 + src/backend/nodes/equalfuncs.c | 1 + src/backend/optimizer/util/plancat.c | 2 +- src/backend/parser/gram.y | 22 +- src/backend/partitioning/partbounds.c | 6 +- src/backend/partitioning/partdesc.c | 21 +- src/backend/tcop/utility.c | 19 + src/bin/psql/describe.c | 40 +- src/include/catalog/pg_inherits.h | 7 +- src/include/nodes/parsenodes.h | 2 + src/include/parser/kwlist.h | 1 + src/include/partitioning/partdesc.h | 5 +- .../detach-partition-concurrently-1.out | 182 +++++++ .../detach-partition-concurrently-2.out | 66 +++ src/test/isolation/isolation_schedule | 2 + .../detach-partition-concurrently-1.spec | 71 +++ .../detach-partition-concurrently-2.spec | 41 ++ 26 files changed, 958 insertions(+), 153 deletions(-) create mode 100644 src/test/isolation/expected/detach-partition-concurrently-1.out create mode 100644 src/test/isolation/expected/detach-partition-concurrently-2.out create mode 100644 src/test/isolation/specs/detach-partition-concurrently-1.spec create mode 100644 src/test/isolation/specs/detach-partition-concurrently-2.spec diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml index 26fda20d19..0f8abdc2d0 100644 --- a/doc/src/sgml/catalogs.sgml +++ b/doc/src/sgml/catalogs.sgml @@ -4471,6 +4471,16 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l when using declarative partitioning. </para></entry> </row> + + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>inhdetached</structfield> <type>bool</type> + </para> + <para> + Set to true for a partition that is in the process of being detached; + false otherwise. + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index b2eb7097a9..2e67468e46 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -36,7 +36,9 @@ ALTER TABLE ALL IN TABLESPACE <replaceable class="parameter">name</replaceable> ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ATTACH PARTITION <replaceable class="parameter">partition_name</replaceable> { FOR VALUES <replaceable class="parameter">partition_bound_spec</replaceable> | DEFAULT } ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> - DETACH PARTITION <replaceable class="parameter">partition_name</replaceable> + DETACH PARTITION <replaceable class="parameter">partition_name</replaceable> [ CONCURRENTLY ] +ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> + DETACH PARTITION <replaceable class="parameter">partition_name</replaceable> FINALIZE <phrase>where <replaceable class="parameter">action</replaceable> is one of:</phrase> @@ -938,7 +940,8 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </varlistentry> <varlistentry> - <term><literal>DETACH PARTITION</literal> <replaceable class="parameter">partition_name</replaceable></term> + <term><literal>DETACH PARTITION</literal> <replaceable class="parameter">partition_name</replaceable> [ { CONCURRENTLY | FINALIZE } ]</term> + <listitem> <para> This form detaches the specified partition of the target table. The detached @@ -947,6 +950,24 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM attached to the target table's indexes are detached. Any triggers that were created as clones of those in the target table are removed. </para> + <para> + If <literal>CONCURRENTLY</literal> is specified, this process runs in two + transactions in order to avoid blocking other sessions that might be accessing + the partitioned table. During the first transaction, + <literal>SHARE UPDATE EXCLUSIVE</literal> is taken in both parent table and + partition, and the partition is marked detached; at that point, the transaction + is committed and all transactions using the partitioned table are waited for. + Once all those transactions are gone, the second stage acquires + <literal>ACCESS EXCLUSIVE</literal> on the partition, and the detach process + completes. + <literal>CONCURRENTLY</literal> is not allowed if the + partitioned table contains a default partition. + </para> + <para> + If <literal>FINALIZE</literal> is specified, complete actions of a + previous <literal>DETACH CONCURRENTLY</literal> invocation that + was cancelled or crashed. + </para> </listitem> </varlistentry> diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c index f2ca686397..7cc5cb4d51 100644 --- a/src/backend/catalog/heap.c +++ b/src/backend/catalog/heap.c @@ -2552,10 +2552,12 @@ StoreConstraints(Relation rel, List *cooked_constraints, bool is_internal) * Returns a list of CookedConstraint nodes that shows the cooked form of * the default and constraint expressions added to the relation. * - * NB: caller should have opened rel with AccessExclusiveLock, and should - * hold that lock till end of transaction. Also, we assume the caller has - * done a CommandCounterIncrement if necessary to make the relation's catalog - * tuples visible. + * NB: caller should have opened rel with some self-conflicting lock mode, + * and should hold that lock till end of transaction; for normal cases that'll + * be AccessExclusiveLock, but if caller knows that the constraint is already + * enforced by some other means, it can be ShareUpdateExclusiveLock. Also, we + * assume the caller has done a CommandCounterIncrement if necessary to make + * the relation's catalog tuples visible. */ List * AddRelationNewConstraints(Relation rel, @@ -3787,7 +3789,8 @@ StorePartitionBound(Relation rel, Relation parent, PartitionBoundSpec *bound) * relcache entry for that partition every time a partition is added or * removed. */ - defaultPartOid = get_default_oid_from_partdesc(RelationGetPartitionDesc(parent)); + defaultPartOid = + get_default_oid_from_partdesc(RelationGetPartitionDesc(parent, false)); if (OidIsValid(defaultPartOid)) CacheInvalidateRelcacheByRelid(defaultPartOid); diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c index 1be27eec52..ad545a7866 100644 --- a/src/backend/catalog/index.c +++ b/src/backend/catalog/index.c @@ -1655,7 +1655,7 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName) List *ancestors = get_partition_ancestors(oldIndexId); Oid parentIndexRelid = linitial_oid(ancestors); - DeleteInheritsTuple(oldIndexId, parentIndexRelid); + DeleteInheritsTuple(oldIndexId, parentIndexRelid, false, NULL); StoreSingleInheritance(newIndexId, parentIndexRelid, 1); list_free(ancestors); @@ -2241,7 +2241,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode) /* * fix INHERITS relation */ - DeleteInheritsTuple(indexId, InvalidOid); + DeleteInheritsTuple(indexId, InvalidOid, false, NULL); /* * We are presently too lazy to attempt to compute the new correct value diff --git a/src/backend/catalog/pg_inherits.c b/src/backend/catalog/pg_inherits.c index 17f37eb39f..f143db0ad4 100644 --- a/src/backend/catalog/pg_inherits.c +++ b/src/backend/catalog/pg_inherits.c @@ -52,7 +52,8 @@ typedef struct SeenRelsEntry * against possible DROPs of child relations. */ List * -find_inheritance_children(Oid parentrelId, LOCKMODE lockmode) +find_inheritance_children(Oid parentrelId, bool include_detached, + LOCKMODE lockmode) { List *list = NIL; Relation relation; @@ -91,7 +92,13 @@ find_inheritance_children(Oid parentrelId, LOCKMODE lockmode) while ((inheritsTuple = systable_getnext(scan)) != NULL) { + /* skip detached at caller's request */ + if (((Form_pg_inherits) GETSTRUCT(inheritsTuple))->inhdetached && + !include_detached) + continue; + inhrelid = ((Form_pg_inherits) GETSTRUCT(inheritsTuple))->inhrelid; + if (numoids >= maxoids) { maxoids *= 2; @@ -160,6 +167,9 @@ find_inheritance_children(Oid parentrelId, LOCKMODE lockmode) * given rel; caller should already have locked it). If lockmode is NoLock * then no locks are acquired, but caller must beware of race conditions * against possible DROPs of child relations. + * + * NB - No current callers of this routine are interested in children being + * concurrently detached, so there's no provision to include them. */ List * find_all_inheritors(Oid parentrelId, LOCKMODE lockmode, List **numparents) @@ -200,7 +210,8 @@ find_all_inheritors(Oid parentrelId, LOCKMODE lockmode, List **numparents) ListCell *lc; /* Get the direct children of this rel */ - currentchildren = find_inheritance_children(currentrel, lockmode); + currentchildren = find_inheritance_children(currentrel, false, + lockmode); /* * Add to the queue only those children not already seen. This avoids @@ -429,6 +440,7 @@ StoreSingleInheritance(Oid relationId, Oid parentOid, int32 seqNumber) values[Anum_pg_inherits_inhrelid - 1] = ObjectIdGetDatum(relationId); values[Anum_pg_inherits_inhparent - 1] = ObjectIdGetDatum(parentOid); values[Anum_pg_inherits_inhseqno - 1] = Int32GetDatum(seqNumber); + values[Anum_pg_inherits_inhdetached - 1] = BoolGetDatum(false); memset(nulls, 0, sizeof(nulls)); @@ -448,10 +460,21 @@ StoreSingleInheritance(Oid relationId, Oid parentOid, int32 seqNumber) * as InvalidOid, in which case all tuples matching inhrelid are deleted; * otherwise only delete tuples with the specified inhparent. * + * 'detached' is the expected state of the inhdetached flag. If the catalog + * row does not match that state, an error is raised. When used on + * partitions, the partition name must be passed, for possible error messages. + * + * If allow_detached is passed false, then an error is raised if the child is + * already marked detached. A name must be supplied in that case, for the + * error message. This should only be used with table partitions. + * XXX the name bit is pretty weird and problematic for non-partition cases; + * what if the flag check fails? + * * Returns whether at least one row was deleted. */ bool -DeleteInheritsTuple(Oid inhrelid, Oid inhparent) +DeleteInheritsTuple(Oid inhrelid, Oid inhparent, bool detached, + const char *childname) { bool found = false; Relation catalogRelation; @@ -478,6 +501,28 @@ DeleteInheritsTuple(Oid inhrelid, Oid inhparent) parent = ((Form_pg_inherits) GETSTRUCT(inheritsTuple))->inhparent; if (!OidIsValid(inhparent) || parent == inhparent) { + bool inhdetached; + + inhdetached = ((Form_pg_inherits) GETSTRUCT(inheritsTuple))->inhdetached; + + /* + * Raise error depending on state. This should only happen for + * partitions, but we have no way to cross-check. + */ + if (inhdetached && !detached) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot detach partition \"%s\"", + childname), + errdetail("The partition is being detached concurrently or has an unfinished detach."), + errhint("Use ALTER TABLE ... DETACH PARTITION ... FINALIZE to complete the pending detach operation"))); + if (!inhdetached && detached) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot complete detaching partition \"%s\"", + childname), + errdetail("There's no partial concurrent detach in progress."))); + CatalogTupleDelete(catalogRelation, &inheritsTuple->t_self); found = true; } diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c index 2baca12c5f..d032fe4af3 100644 --- a/src/backend/commands/indexcmds.c +++ b/src/backend/commands/indexcmds.c @@ -1092,7 +1092,7 @@ DefineIndex(Oid relationId, */ if (partitioned && stmt->relation && !stmt->relation->inh) { - PartitionDesc pd = RelationGetPartitionDesc(rel); + PartitionDesc pd = RelationGetPartitionDesc(rel, false); if (pd->nparts != 0) flags |= INDEX_CREATE_INVALID; @@ -1149,7 +1149,7 @@ DefineIndex(Oid relationId, */ if (!stmt->relation || stmt->relation->inh) { - PartitionDesc partdesc = RelationGetPartitionDesc(rel); + PartitionDesc partdesc = RelationGetPartitionDesc(rel, false); int nparts = partdesc->nparts; Oid *part_oids = palloc(sizeof(Oid) * nparts); bool invalidate_parent = false; @@ -3550,6 +3550,7 @@ IndexSetParentIndex(Relation partitionIdx, Oid parentOid) values[Anum_pg_inherits_inhparent - 1] = ObjectIdGetDatum(parentOid); values[Anum_pg_inherits_inhseqno - 1] = Int32GetDatum(1); + values[Anum_pg_inherits_inhdetached - 1] = BoolGetDatum(false); memset(isnull, false, sizeof(isnull)); tuple = heap_form_tuple(RelationGetDescr(pg_inherits), diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 07f4f562ee..682aa6a015 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -541,7 +541,8 @@ static PartitionSpec *transformPartitionSpec(Relation rel, PartitionSpec *partsp static void ComputePartitionAttrs(ParseState *pstate, Relation rel, List *partParams, AttrNumber *partattrs, List **partexprs, Oid *partopclass, Oid *partcollation, char strategy); static void CreateInheritance(Relation child_rel, Relation parent_rel); -static void RemoveInheritance(Relation child_rel, Relation parent_rel); +static void RemoveInheritance(Relation child_rel, Relation parent_rel, + bool allow_detached); static ObjectAddress ATExecAttachPartition(List **wqueue, Relation rel, PartitionCmd *cmd); static void AttachPartitionEnsureIndexes(Relation rel, Relation attachrel); @@ -550,7 +551,12 @@ static void QueuePartitionConstraintValidation(List **wqueue, Relation scanrel, bool validate_default); static void CloneRowTriggersToPartition(Relation parent, Relation partition); static void DropClonedTriggersFromPartition(Oid partitionId); -static ObjectAddress ATExecDetachPartition(Relation rel, RangeVar *name); +static ObjectAddress ATExecDetachPartition(List **wqueue, AlteredTableInfo *tab, + Relation rel, RangeVar *name, + bool concurrent); +static void DetachPartitionFinalize(Relation rel, Relation partRel, + bool concurrent, Oid defaultPartOid); +static ObjectAddress ATExecDetachPartitionFinalize(Relation rel, RangeVar *name); static ObjectAddress ATExecAttachPartitionIdx(List **wqueue, Relation rel, RangeVar *name); static void validatePartitionedIndex(Relation partedIdx, Relation partedTbl); @@ -984,7 +990,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId, * lock the partition so as to avoid a deadlock. */ defaultPartOid = - get_default_oid_from_partdesc(RelationGetPartitionDesc(parent)); + get_default_oid_from_partdesc(RelationGetPartitionDesc(parent, + false)); if (OidIsValid(defaultPartOid)) defaultRel = table_open(defaultPartOid, AccessExclusiveLock); @@ -3099,7 +3106,7 @@ renameatt_internal(Oid myrelid, * expected_parents will only be 0 if we are not already recursing. */ if (expected_parents == 0 && - find_inheritance_children(myrelid, NoLock) != NIL) + find_inheritance_children(myrelid, false, NoLock) != NIL) ereport(ERROR, (errcode(ERRCODE_INVALID_TABLE_DEFINITION), errmsg("inherited column \"%s\" must be renamed in child tables too", @@ -3298,7 +3305,7 @@ rename_constraint_internal(Oid myrelid, else { if (expected_parents == 0 && - find_inheritance_children(myrelid, NoLock) != NIL) + find_inheritance_children(myrelid, false, NoLock) != NIL) ereport(ERROR, (errcode(ERRCODE_INVALID_TABLE_DEFINITION), errmsg("inherited constraint \"%s\" must be renamed in child tables too", @@ -3916,7 +3923,14 @@ AlterTableGetLockLevel(List *cmds) break; case AT_DetachPartition: - cmd_lockmode = AccessExclusiveLock; + if (((PartitionCmd *) cmd->def)->concurrent) + cmd_lockmode = ShareUpdateExclusiveLock; + else + cmd_lockmode = AccessExclusiveLock; + break; + + case AT_DetachPartitionFinalize: + cmd_lockmode = ShareUpdateExclusiveLock; break; case AT_CheckNotNull: @@ -4288,6 +4302,11 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_DetachPartitionFinalize: + ATSimplePermissions(rel, ATT_TABLE); + /* No command-specific prep needed */ + pass = AT_PASS_MISC; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -4663,7 +4682,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Assert(cmd != NULL); /* ATPrepCmd ensures it must be a table */ Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE); - ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name); + ATExecDetachPartition(wqueue, tab, rel, + ((PartitionCmd *) cmd->def)->name, + ((PartitionCmd *) cmd->def)->concurrent); + break; + case AT_DetachPartitionFinalize: + ATExecDetachPartitionFinalize(rel, ((PartitionCmd *) cmd->def)->name); break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", @@ -6081,7 +6105,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel, */ if (colDef->identity && recurse && - find_inheritance_children(myrelid, NoLock) != NIL) + find_inheritance_children(myrelid, false, NoLock) != NIL) ereport(ERROR, (errcode(ERRCODE_INVALID_TABLE_DEFINITION), errmsg("cannot recursively add identity column to table that has child tables"))); @@ -6314,7 +6338,8 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel, * routines, we have to do this one level of recursion at a time; we can't * use find_all_inheritors to do it in one pass. */ - children = find_inheritance_children(RelationGetRelid(rel), lockmode); + children = + find_inheritance_children(RelationGetRelid(rel), false, lockmode); /* * If we are told not to recurse, there had better not be any child @@ -6468,7 +6493,7 @@ ATPrepDropNotNull(Relation rel, bool recurse, bool recursing) */ if (rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE) { - PartitionDesc partdesc = RelationGetPartitionDesc(rel); + PartitionDesc partdesc = RelationGetPartitionDesc(rel, false); Assert(partdesc != NULL); if (partdesc->nparts > 0 && !recurse && !recursing) @@ -7680,7 +7705,8 @@ ATExecDropColumn(List **wqueue, Relation rel, const char *colName, * routines, we have to do this one level of recursion at a time; we can't * use find_all_inheritors to do it in one pass. */ - children = find_inheritance_children(RelationGetRelid(rel), lockmode); + children = + find_inheritance_children(RelationGetRelid(rel), false, lockmode); if (children) { @@ -8144,7 +8170,8 @@ ATAddCheckConstraint(List **wqueue, AlteredTableInfo *tab, Relation rel, * routines, we have to do this one level of recursion at a time; we can't * use find_all_inheritors to do it in one pass. */ - children = find_inheritance_children(RelationGetRelid(rel), lockmode); + children = + find_inheritance_children(RelationGetRelid(rel), false, lockmode); /* * Check if ONLY was specified with ALTER TABLE. If so, allow the @@ -8759,7 +8786,7 @@ addFkRecurseReferenced(List **wqueue, Constraint *fkconstraint, Relation rel, */ if (pkrel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE) { - PartitionDesc pd = RelationGetPartitionDesc(pkrel); + PartitionDesc pd = RelationGetPartitionDesc(pkrel, false); for (int i = 0; i < pd->nparts; i++) { @@ -8893,7 +8920,7 @@ addFkRecurseReferencing(List **wqueue, Constraint *fkconstraint, Relation rel, } else if (rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE) { - PartitionDesc pd = RelationGetPartitionDesc(rel); + PartitionDesc pd = RelationGetPartitionDesc(rel, false); /* * Recurse to take appropriate action on each partition; either we @@ -10674,7 +10701,8 @@ ATExecDropConstraint(Relation rel, const char *constrName, * use find_all_inheritors to do it in one pass. */ if (!is_no_inherit_constraint) - children = find_inheritance_children(RelationGetRelid(rel), lockmode); + children = + find_inheritance_children(RelationGetRelid(rel), false, lockmode); else children = NIL; @@ -11058,7 +11086,8 @@ ATPrepAlterColumnType(List **wqueue, } } else if (!recursing && - find_inheritance_children(RelationGetRelid(rel), NoLock) != NIL) + find_inheritance_children(RelationGetRelid(rel), false, + NoLock) != NIL) ereport(ERROR, (errcode(ERRCODE_INVALID_TABLE_DEFINITION), errmsg("type of inherited column \"%s\" must be changed in child tables too", @@ -13953,7 +13982,7 @@ ATExecDropInherit(Relation rel, RangeVar *parent, LOCKMODE lockmode) */ /* Off to RemoveInheritance() where most of the work happens */ - RemoveInheritance(rel, parent_rel); + RemoveInheritance(rel, parent_rel, false); ObjectAddressSet(address, RelationRelationId, RelationGetRelid(parent_rel)); @@ -13964,12 +13993,70 @@ ATExecDropInherit(Relation rel, RangeVar *parent, LOCKMODE lockmode) return address; } +/* + * MarkInheritDetached + * + * When a partition is detached from its parent concurrently, we don't + * remove the pg_inherits row until a second transaction; as a preparatory + * step, this function marks the entry as 'detached', so that other + * transactions know to ignore the partition. + */ +static void +MarkInheritDetached(Relation child_rel, Relation parent_rel) +{ + Relation catalogRelation; + SysScanDesc scan; + ScanKeyData key; + HeapTuple inheritsTuple; + bool found = false; + + /* + * Find pg_inherits entries by inhrelid. + */ + catalogRelation = table_open(InheritsRelationId, RowExclusiveLock); + ScanKeyInit(&key, + Anum_pg_inherits_inhrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(child_rel))); + scan = systable_beginscan(catalogRelation, InheritsRelidSeqnoIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(inheritsTuple = systable_getnext(scan))) + { + HeapTuple newtup; + + if (((Form_pg_inherits) GETSTRUCT(inheritsTuple))->inhparent != + RelationGetRelid(parent_rel)) + elog(ERROR, "bad parent tuple found for partition %u", + RelationGetRelid(child_rel)); + + newtup = heap_copytuple(inheritsTuple); + ((Form_pg_inherits) GETSTRUCT(newtup))->inhdetached = true; + + CatalogTupleUpdate(catalogRelation, + &inheritsTuple->t_self, + newtup); + found = true; + } + + /* Done */ + systable_endscan(scan); + table_close(catalogRelation, RowExclusiveLock); + + if (!found) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_TABLE), + errmsg("relation \"%s\" is not a partition of relation \"%s\"", + RelationGetRelationName(child_rel), + RelationGetRelationName(parent_rel)))); +} + /* * RemoveInheritance * * Drop a parent from the child's parents. This just adjusts the attinhcount * and attislocal of the columns and removes the pg_inherit and pg_depend - * entries. + * entries. expect_detached is passed down to DeleteInheritsTuple, q.v.. * * If attinhcount goes to 0 then attislocal gets set to true. If it goes back * up attislocal stays true, which means if a child is ever removed from a @@ -13983,7 +14070,7 @@ ATExecDropInherit(Relation rel, RangeVar *parent, LOCKMODE lockmode) * Common to ATExecDropInherit() and ATExecDetachPartition(). */ static void -RemoveInheritance(Relation child_rel, Relation parent_rel) +RemoveInheritance(Relation child_rel, Relation parent_rel, bool expect_detached) { Relation catalogRelation; SysScanDesc scan; @@ -13999,7 +14086,9 @@ RemoveInheritance(Relation child_rel, Relation parent_rel) child_is_partition = true; found = DeleteInheritsTuple(RelationGetRelid(child_rel), - RelationGetRelid(parent_rel)); + RelationGetRelid(parent_rel), + expect_detached, + RelationGetRelationName(child_rel)); if (!found) { if (child_is_partition) @@ -16107,7 +16196,7 @@ QueuePartitionConstraintValidation(List **wqueue, Relation scanrel, } else if (scanrel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE) { - PartitionDesc partdesc = RelationGetPartitionDesc(scanrel); + PartitionDesc partdesc = RelationGetPartitionDesc(scanrel, false); int i; for (i = 0; i < partdesc->nparts; i++) @@ -16163,7 +16252,7 @@ ATExecAttachPartition(List **wqueue, Relation rel, PartitionCmd *cmd) * new partition will change its partition constraint. */ defaultPartOid = - get_default_oid_from_partdesc(RelationGetPartitionDesc(rel)); + get_default_oid_from_partdesc(RelationGetPartitionDesc(rel, false)); if (OidIsValid(defaultPartOid)) LockRelationOid(defaultPartOid, AccessExclusiveLock); @@ -16751,105 +16840,226 @@ CloneRowTriggersToPartition(Relation parent, Relation partition) * ALTER TABLE DETACH PARTITION * * Return the address of the relation that is no longer a partition of rel. + * + * If concurrent mode is requested, we run in two transactions. A side- + * effect is that this command cannot run in a multi-part ALTER TABLE. + * Currently, that's enforced by the grammar. + * + * The strategy for concurrency is to first modify the partition catalog + * rows to make it visible to everyone that the partition is detached, + * lock the partition against writes, and commit the transaction; anyone + * who requests the partition descriptor from that point onwards has to + * ignore such a partition. In a second transaction, we wait until all + * transactions that could have seen the partition as attached are gone, + * then we remove the rest of partition metadata (pg_inherits and + * pg_class.relpartbounds). */ static ObjectAddress -ATExecDetachPartition(Relation rel, RangeVar *name) +ATExecDetachPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, + RangeVar *name, bool concurrent) { - Relation partRel, - classRel; - HeapTuple tuple, - newtuple; - Datum new_val[Natts_pg_class]; - bool new_null[Natts_pg_class], - new_repl[Natts_pg_class]; + Relation partRel; ObjectAddress address; Oid defaultPartOid; - List *indexes; - List *fks; - ListCell *cell; /* * We must lock the default partition, because detaching this partition * will change its partition constraint. */ defaultPartOid = - get_default_oid_from_partdesc(RelationGetPartitionDesc(rel)); - if (OidIsValid(defaultPartOid)) - LockRelationOid(defaultPartOid, AccessExclusiveLock); - - partRel = table_openrv(name, ShareUpdateExclusiveLock); - - /* Ensure that foreign keys still hold after this detach */ - ATDetachCheckNoForeignKeyRefs(partRel); - - /* All inheritance related checks are performed within the function */ - RemoveInheritance(partRel, rel); - - /* Update pg_class tuple */ - classRel = table_open(RelationRelationId, RowExclusiveLock); - tuple = SearchSysCacheCopy1(RELOID, - ObjectIdGetDatum(RelationGetRelid(partRel))); - if (!HeapTupleIsValid(tuple)) - elog(ERROR, "cache lookup failed for relation %u", - RelationGetRelid(partRel)); - Assert(((Form_pg_class) GETSTRUCT(tuple))->relispartition); - - /* Clear relpartbound and reset relispartition */ - memset(new_val, 0, sizeof(new_val)); - memset(new_null, false, sizeof(new_null)); - memset(new_repl, false, sizeof(new_repl)); - new_val[Anum_pg_class_relpartbound - 1] = (Datum) 0; - new_null[Anum_pg_class_relpartbound - 1] = true; - new_repl[Anum_pg_class_relpartbound - 1] = true; - newtuple = heap_modify_tuple(tuple, RelationGetDescr(classRel), - new_val, new_null, new_repl); - - ((Form_pg_class) GETSTRUCT(newtuple))->relispartition = false; - CatalogTupleUpdate(classRel, &newtuple->t_self, newtuple); - heap_freetuple(newtuple); - + get_default_oid_from_partdesc(RelationGetPartitionDesc(rel, false)); if (OidIsValid(defaultPartOid)) { /* - * If the relation being detached is the default partition itself, - * remove it from the parent's pg_partitioned_table entry. + * Concurrent detaching when a default partition exists is not + * supported. The main problem is that the default partition + * constraint would change. And there's a definitional problem: what + * should happen to the tuples that are being inserted that belong to + * the partition being detached? Putting them on the partition being + * detached would be wrong, since they'd become "lost" after the but + * we cannot put them in the default partition either until we alter + * its partition constraint. * - * If not, we must invalidate default partition's relcache entry, as - * in StorePartitionBound: its partition constraint depends on every - * other partition's partition constraint. + * I think we could solve this problem if we effected the constraint + * change before committing the first transaction. But the lock would + * have to remain AEL and it would cause concurrent query planning to + * be blocked, so changing it that way would be even worse. */ - if (RelationGetRelid(partRel) == defaultPartOid) - update_default_partition_oid(RelationGetRelid(rel), InvalidOid); - else - CacheInvalidateRelcacheByRelid(defaultPartOid); + if (concurrent) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot detach partitions concurrently when a default partition exists"))); + LockRelationOid(defaultPartOid, AccessExclusiveLock); } - /* detach indexes too */ - indexes = RelationGetIndexList(partRel); - foreach(cell, indexes) + /* + * In concurrent mode, the partition is locked with + * ShareUpdateExclusiveLock in the first stage. + */ + partRel = table_openrv(name, concurrent ? + ShareUpdateExclusiveLock : AccessExclusiveLock); + + /* + * Check inheritance conditions and either delete the pg_inherits row + * (in non-concurrent mode) or just set the inhisdetached flag. + */ + if (!concurrent) + RemoveInheritance(partRel, rel, false); + else + MarkInheritDetached(partRel, rel); + + /* + * Ensure that foreign keys still hold after this detach. This keeps lock + * on the referencing tables, which prevent concurrent transactions from + * adding rows that we wouldn't see. For this to work in concurrent mode, + * it is critical that the partition appears as no longer attached for the + * RI queries as soon as the first transaction commits. + */ + ATDetachCheckNoForeignKeyRefs(partRel); + + /* + * Concurrent mode has to work harder; first we add a new constraint to the + * partition that matches the partition constraint. The reason for this is + * that the planner may have made optimizations that depend on the + * constraint. XXX Isn't it sufficient to invalidate the partition's + * relcache entry? + * + * Then we close our existing transaction, and in a new one wait for + * all process to catch up on the catalog updates we've done so far; at + * that point we can complete the operation. + */ + if (concurrent) { - Oid idxid = lfirst_oid(cell); - Relation idx; - Oid constrOid; + Oid partrelid, + parentrelid; + Constraint *n; + LOCKTAG tag; + char *parentrelname; + char *partrelname; - if (!has_superclass(idxid)) - continue; + /* Add constraint on partition, equivalent to the partition constraint */ + n = makeNode(Constraint); + n->contype = CONSTR_CHECK; + n->conname = NULL; + n->location = -1; + n->is_no_inherit = false; + n->raw_expr = NULL; + n->cooked_expr = nodeToString(make_ands_explicit(RelationGetPartitionQual(partRel))); + n->initially_valid = true; + n->skip_validation = true; + /* It's a re-add, since it nominally already exists */ + ATAddCheckConstraint(wqueue, tab, partRel, n, + true, false, true, ShareUpdateExclusiveLock); - Assert((IndexGetRelation(get_partition_parent(idxid), false) == - RelationGetRelid(rel))); + /* + * We're almost done now; the only traces that remain are the + * pg_inherits tuple and the partition's relpartbounds. Before we can + * remove those, we need to wait until all transactions that know that + * this is a partition are gone. + */ - idx = index_open(idxid, AccessExclusiveLock); - IndexSetParentIndex(idx, InvalidOid); + /* + * Remember relation OIDs to re-acquire them later; and relation names + * too, for error messages if something is dropped in between. + */ + partrelid = RelationGetRelid(partRel); + parentrelid = RelationGetRelid(rel); + parentrelname = MemoryContextStrdup(PortalContext, + RelationGetRelationName(rel)); + partrelname = MemoryContextStrdup(PortalContext, + RelationGetRelationName(partRel)); - /* If there's a constraint associated with the index, detach it too */ - constrOid = get_relation_idx_constraint_oid(RelationGetRelid(partRel), - idxid); - if (OidIsValid(constrOid)) - ConstraintSetParentConstraint(constrOid, InvalidOid, InvalidOid); + /* Invalidate relcache entries for the parent -- must be before close */ + CacheInvalidateRelcache(rel); + + table_close(partRel, NoLock); + table_close(rel, NoLock); + tab->rel = NULL; + + /* Make updated catalog entry visible */ + PopActiveSnapshot(); + CommitTransactionCommand(); + + StartTransactionCommand(); + + /* + * Now wait. This ensures that all queries that were planned including + * the partition are finished before we remove the rest of catalog + * entries. We don't need or indeed want to acquire this lock, though + * -- that would block later queries. + * + * We don't need to concern ourselves with waiting for a lock the + * partition itself, since we will acquire AccessExclusiveLock below. + */ + SET_LOCKTAG_RELATION(tag, MyDatabaseId, parentrelid); + WaitForLockersMultiple(list_make1(&tag), AccessExclusiveLock, false); + + /* + * Now acquire locks in both relations again. Note they may have been + * removed in the meantime, so care is required. + */ + rel = try_relation_open(parentrelid, ShareUpdateExclusiveLock); + partRel = try_relation_open(partrelid, AccessExclusiveLock); + + /* If the relations aren't there, something bad happened; bail out */ + if (rel == NULL) + { + if (partRel != NULL) /* shouldn't happen */ + elog(WARNING, "dangling partition \"%s\" remains, can't fix", + partrelname); + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("partitioned table \"%s\" was removed concurrently", + parentrelname))); + } + if (partRel == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("partition \"%s\" was removed concurrently", partrelname))); + + tab->rel = rel; - index_close(idx, NoLock); } - table_close(classRel, RowExclusiveLock); + + /* Do the final part of detaching */ + DetachPartitionFinalize(rel, partRel, concurrent, defaultPartOid); + + ObjectAddressSet(address, RelationRelationId, RelationGetRelid(partRel)); + + /* keep our lock until commit */ + table_close(partRel, NoLock); + + return address; +} + +/* + * Second part of ALTER TABLE .. DETACH. + * + * This is separate so that it can be run independently when the second + * transaction of the concurrent algorithm fails (crash or abort). + */ +static void +DetachPartitionFinalize(Relation rel, Relation partRel, bool concurrent, + Oid defaultPartOid) +{ + Relation classRel; + List *fks; + ListCell *cell; + List *indexes; + Datum new_val[Natts_pg_class]; + bool new_null[Natts_pg_class], + new_repl[Natts_pg_class]; + HeapTuple tuple, + newtuple; + + if (concurrent) + { + /* + * We can remove the pg_inherits row now. (In the non-concurrent case, + * this was already done). + */ + RemoveInheritance(partRel, rel, true); + } /* Drop any triggers that were cloned on creation/attach. */ DropClonedTriggersFromPartition(RelationGetRelid(partRel)); @@ -16922,17 +17132,98 @@ ATExecDetachPartition(Relation rel, RangeVar *name) ObjectAddressSet(constraint, ConstraintRelationId, constrOid); performDeletion(&constraint, DROP_RESTRICT, 0); } - CommandCounterIncrement(); + + /* Now we can detach indexes */ + indexes = RelationGetIndexList(partRel); + foreach(cell, indexes) + { + Oid idxid = lfirst_oid(cell); + Relation idx; + Oid constrOid; + + if (!has_superclass(idxid)) + continue; + + Assert((IndexGetRelation(get_partition_parent(idxid), false) == + RelationGetRelid(rel))); + + idx = index_open(idxid, AccessExclusiveLock); + IndexSetParentIndex(idx, InvalidOid); + + /* If there's a constraint associated with the index, detach it too */ + constrOid = get_relation_idx_constraint_oid(RelationGetRelid(partRel), + idxid); + if (OidIsValid(constrOid)) + ConstraintSetParentConstraint(constrOid, InvalidOid, InvalidOid); + + index_close(idx, NoLock); + } + + /* Update pg_class tuple */ + classRel = table_open(RelationRelationId, RowExclusiveLock); + tuple = SearchSysCacheCopy1(RELOID, + ObjectIdGetDatum(RelationGetRelid(partRel))); + if (!HeapTupleIsValid(tuple)) + elog(ERROR, "cache lookup failed for relation %u", + RelationGetRelid(partRel)); + Assert(((Form_pg_class) GETSTRUCT(tuple))->relispartition); + + /* Clear relpartbound and reset relispartition */ + memset(new_val, 0, sizeof(new_val)); + memset(new_null, false, sizeof(new_null)); + memset(new_repl, false, sizeof(new_repl)); + new_val[Anum_pg_class_relpartbound - 1] = (Datum) 0; + new_null[Anum_pg_class_relpartbound - 1] = true; + new_repl[Anum_pg_class_relpartbound - 1] = true; + newtuple = heap_modify_tuple(tuple, RelationGetDescr(classRel), + new_val, new_null, new_repl); + + ((Form_pg_class) GETSTRUCT(newtuple))->relispartition = false; + CatalogTupleUpdate(classRel, &newtuple->t_self, newtuple); + heap_freetuple(newtuple); + table_close(classRel, RowExclusiveLock); + + if (OidIsValid(defaultPartOid)) + { + /* + * If the relation being detached is the default partition itself, + * remove it from the parent's pg_partitioned_table entry. + * + * If not, we must invalidate default partition's relcache entry, as + * in StorePartitionBound: its partition constraint depends on every + * other partition's partition constraint. + */ + if (RelationGetRelid(partRel) == defaultPartOid) + update_default_partition_oid(RelationGetRelid(rel), InvalidOid); + else + CacheInvalidateRelcacheByRelid(defaultPartOid); + } /* * Invalidate the parent's relcache so that the partition is no longer * included in its partition descriptor. */ CacheInvalidateRelcache(rel); +} + +/* + * ALTER TABLE ... DETACH PARTITION ... FINALIZE + * + * To use when a DETACH PARTITION command previously did not run to + * completion; this completes the detaching process. + */ +static ObjectAddress +ATExecDetachPartitionFinalize(Relation rel, RangeVar *name) +{ + Relation partRel; + ObjectAddress address; + + partRel = table_openrv(name, AccessExclusiveLock); + + DetachPartitionFinalize(rel, partRel, true, InvalidOid); ObjectAddressSet(address, RelationRelationId, RelationGetRelid(partRel)); - /* keep our lock until commit */ table_close(partRel, NoLock); return address; @@ -17133,7 +17424,7 @@ ATExecAttachPartitionIdx(List **wqueue, Relation parentIdx, RangeVar *name) RelationGetRelationName(partIdx)))); /* Make sure it indexes a partition of the other index's table */ - partDesc = RelationGetPartitionDesc(parentTbl); + partDesc = RelationGetPartitionDesc(parentTbl, false); found = false; for (i = 0; i < partDesc->nparts; i++) { @@ -17287,7 +17578,7 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl) * If we found as many inherited indexes as the partitioned table has * partitions, we're good; update pg_index to set indisvalid. */ - if (tuples == RelationGetPartitionDesc(partedTbl)->nparts) + if (tuples == RelationGetPartitionDesc(partedTbl, false)->nparts) { Relation idxRel; HeapTuple newtup; diff --git a/src/backend/commands/trigger.c b/src/backend/commands/trigger.c index 672fccff5b..ce65abd57e 100644 --- a/src/backend/commands/trigger.c +++ b/src/backend/commands/trigger.c @@ -1054,7 +1054,7 @@ CreateTrigger(CreateTrigStmt *stmt, const char *queryString, */ if (partition_recurse) { - PartitionDesc partdesc = RelationGetPartitionDesc(rel); + PartitionDesc partdesc = RelationGetPartitionDesc(rel, false); List *idxs = NIL; List *childTbls = NIL; ListCell *l; @@ -1076,7 +1076,8 @@ CreateTrigger(CreateTrigStmt *stmt, const char *queryString, ListCell *l; List *idxs = NIL; - idxs = find_inheritance_children(indexOid, ShareRowExclusiveLock); + idxs = find_inheritance_children(indexOid, false, + ShareRowExclusiveLock); foreach(l, idxs) childTbls = lappend_oid(childTbls, IndexGetRelation(lfirst_oid(l), @@ -1537,7 +1538,7 @@ EnableDisableTrigger(Relation rel, const char *tgname, if (rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE && (TRIGGER_FOR_ROW(oldtrig->tgtype))) { - PartitionDesc partdesc = RelationGetPartitionDesc(rel); + PartitionDesc partdesc = RelationGetPartitionDesc(rel, false); int i; for (i = 0; i < partdesc->nparts; i++) diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c index fb6ce49056..ef4fd7bf90 100644 --- a/src/backend/executor/execPartition.c +++ b/src/backend/executor/execPartition.c @@ -507,6 +507,7 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate, int partidx) { ModifyTable *node = (ModifyTable *) mtstate->ps.plan; + Oid partOid = dispatch->partdesc->oids[partidx]; Relation rootrel = rootResultRelInfo->ri_RelationDesc, partrel; Relation firstResultRel = mtstate->resultRelInfo[0].ri_RelationDesc; @@ -517,7 +518,7 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate, oldcxt = MemoryContextSwitchTo(proute->memcxt); - partrel = table_open(dispatch->partdesc->oids[partidx], RowExclusiveLock); + partrel = table_open(partOid, RowExclusiveLock); leaf_part_rri = makeNode(ResultRelInfo); InitResultRelInfo(leaf_part_rri, @@ -997,7 +998,7 @@ ExecInitPartitionDispatchInfo(EState *estate, if (estate->es_partition_directory == NULL) estate->es_partition_directory = - CreatePartitionDirectory(estate->es_query_cxt); + CreatePartitionDirectory(estate->es_query_cxt, true); oldcxt = MemoryContextSwitchTo(proute->memcxt); @@ -1559,7 +1560,7 @@ ExecCreatePartitionPruneState(PlanState *planstate, if (estate->es_partition_directory == NULL) estate->es_partition_directory = - CreatePartitionDirectory(estate->es_query_cxt); + CreatePartitionDirectory(estate->es_query_cxt, true); n_part_hierarchies = list_length(partitionpruneinfo->prune_infos); Assert(n_part_hierarchies > 0); @@ -1625,9 +1626,12 @@ ExecCreatePartitionPruneState(PlanState *planstate, partrel); /* - * Initialize the subplan_map and subpart_map. Since detaching a - * partition requires AccessExclusiveLock, no partitions can have - * disappeared, nor can the bounds for any partition have changed. + * Initialize the subplan_map and subpart_map. + * + * Because we request detached partitions to be included, and + * detaching waits for old transactions, it is safe to assume that + * no partitions have disappeared since this query was planned. + * * However, new partitions may have been added. */ Assert(partdesc->nparts >= pinfo->nparts); diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c index 89c409de66..f48350bc2f 100644 --- a/src/backend/nodes/copyfuncs.c +++ b/src/backend/nodes/copyfuncs.c @@ -4670,6 +4670,7 @@ _copyPartitionCmd(const PartitionCmd *from) COPY_NODE_FIELD(name); COPY_NODE_FIELD(bound); + COPY_SCALAR_FIELD(concurrent); return newnode; } diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c index e3f33c40be..01ff684f90 100644 --- a/src/backend/nodes/equalfuncs.c +++ b/src/backend/nodes/equalfuncs.c @@ -2932,6 +2932,7 @@ _equalPartitionCmd(const PartitionCmd *a, const PartitionCmd *b) { COMPARE_NODE_FIELD(name); COMPARE_NODE_FIELD(bound); + COMPARE_SCALAR_FIELD(concurrent); return true; } diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c index 25545029d7..75fe783b41 100644 --- a/src/backend/optimizer/util/plancat.c +++ b/src/backend/optimizer/util/plancat.c @@ -2129,7 +2129,7 @@ set_relation_partition_info(PlannerInfo *root, RelOptInfo *rel, /* Create the PartitionDirectory infrastructure if we didn't already */ if (root->glob->partition_directory == NULL) root->glob->partition_directory = - CreatePartitionDirectory(CurrentMemoryContext); + CreatePartitionDirectory(CurrentMemoryContext, false); partdesc = PartitionDirectoryLookup(root->glob->partition_directory, relation); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index dbb47d4982..da0b233695 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -649,7 +649,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query); EXCLUDE EXCLUDING EXCLUSIVE EXECUTE EXISTS EXPLAIN EXPRESSION EXTENSION EXTERNAL EXTRACT - FALSE_P FAMILY FETCH FILTER FIRST_P FLOAT_P FOLLOWING FOR + FALSE_P FAMILY FETCH FILTER FINALIZE FIRST_P FLOAT_P FOLLOWING FOR FORCE FOREIGN FORWARD FREEZE FROM FULL FUNCTION FUNCTIONS GENERATED GLOBAL GRANT GRANTED GREATEST GROUP_P GROUPING GROUPS @@ -2057,12 +2057,13 @@ partition_cmd: n->subtype = AT_AttachPartition; cmd->name = $3; cmd->bound = $4; + cmd->concurrent = false; n->def = (Node *) cmd; $$ = (Node *) n; } - /* ALTER TABLE <name> DETACH PARTITION <partition_name> */ - | DETACH PARTITION qualified_name + /* ALTER TABLE <name> DETACH PARTITION <partition_name> [CONCURRENTLY] */ + | DETACH PARTITION qualified_name opt_concurrently { AlterTableCmd *n = makeNode(AlterTableCmd); PartitionCmd *cmd = makeNode(PartitionCmd); @@ -2070,8 +2071,21 @@ partition_cmd: n->subtype = AT_DetachPartition; cmd->name = $3; cmd->bound = NULL; + cmd->concurrent = $4; n->def = (Node *) cmd; + $$ = (Node *) n; + } + | DETACH PARTITION qualified_name FINALIZE + { + AlterTableCmd *n = makeNode(AlterTableCmd); + PartitionCmd *cmd = makeNode(PartitionCmd); + + n->subtype = AT_DetachPartitionFinalize; + cmd->name = $3; + cmd->bound = NULL; + cmd->concurrent = false; + n->def = (Node *) cmd; $$ = (Node *) n; } ; @@ -2086,6 +2100,7 @@ index_partition_cmd: n->subtype = AT_AttachPartition; cmd->name = $3; cmd->bound = NULL; + cmd->concurrent = false; n->def = (Node *) cmd; $$ = (Node *) n; @@ -15110,6 +15125,7 @@ unreserved_keyword: | EXTERNAL | FAMILY | FILTER + | FINALIZE | FIRST_P | FOLLOWING | FORCE diff --git a/src/backend/partitioning/partbounds.c b/src/backend/partitioning/partbounds.c index 7553d55987..966d4bbe7f 100644 --- a/src/backend/partitioning/partbounds.c +++ b/src/backend/partitioning/partbounds.c @@ -2810,7 +2810,7 @@ check_new_partition_bound(char *relname, Relation parent, PartitionBoundSpec *spec) { PartitionKey key = RelationGetPartitionKey(parent); - PartitionDesc partdesc = RelationGetPartitionDesc(parent); + PartitionDesc partdesc = RelationGetPartitionDesc(parent, true); PartitionBoundInfo boundinfo = partdesc->boundinfo; ParseState *pstate = make_parsestate(NULL); int with = -1; @@ -3984,7 +3984,7 @@ get_qual_for_list(Relation parent, PartitionBoundSpec *spec) { int i; int ndatums = 0; - PartitionDesc pdesc = RelationGetPartitionDesc(parent); + PartitionDesc pdesc = RelationGetPartitionDesc(parent, true); /* XXX correct? */ PartitionBoundInfo boundinfo = pdesc->boundinfo; if (boundinfo) @@ -4184,7 +4184,7 @@ get_qual_for_range(Relation parent, PartitionBoundSpec *spec, if (spec->is_default) { List *or_expr_args = NIL; - PartitionDesc pdesc = RelationGetPartitionDesc(parent); + PartitionDesc pdesc = RelationGetPartitionDesc(parent, true); /* XXX correct? */ Oid *inhoids = pdesc->oids; int nparts = pdesc->nparts, i; diff --git a/src/backend/partitioning/partdesc.c b/src/backend/partitioning/partdesc.c index 0f124a5254..3c21f42ea9 100644 --- a/src/backend/partitioning/partdesc.c +++ b/src/backend/partitioning/partdesc.c @@ -38,6 +38,7 @@ typedef struct PartitionDirectoryData { MemoryContext pdir_mcxt; HTAB *pdir_hash; + bool include_detached; } PartitionDirectoryData; typedef struct PartitionDirectoryEntry @@ -47,7 +48,7 @@ typedef struct PartitionDirectoryEntry PartitionDesc pd; } PartitionDirectoryEntry; -static void RelationBuildPartitionDesc(Relation rel); +static void RelationBuildPartitionDesc(Relation rel, bool include_detached); /* @@ -62,13 +63,14 @@ static void RelationBuildPartitionDesc(Relation rel); * that the data doesn't become stale. */ PartitionDesc -RelationGetPartitionDesc(Relation rel) +RelationGetPartitionDesc(Relation rel, bool include_detached) { if (rel->rd_rel->relkind != RELKIND_PARTITIONED_TABLE) return NULL; - if (unlikely(rel->rd_partdesc == NULL)) - RelationBuildPartitionDesc(rel); + if (unlikely(rel->rd_partdesc == NULL || + rel->rd_partdesc->includes_detached != include_detached)) + RelationBuildPartitionDesc(rel, include_detached); return rel->rd_partdesc; } @@ -89,7 +91,7 @@ RelationGetPartitionDesc(Relation rel) * permanently. */ static void -RelationBuildPartitionDesc(Relation rel) +RelationBuildPartitionDesc(Relation rel, bool include_detached) { PartitionDesc partdesc; PartitionBoundInfo boundinfo = NULL; @@ -111,7 +113,8 @@ RelationBuildPartitionDesc(Relation rel) * concurrently, whatever this function returns will be accurate as of * some well-defined point in time. */ - inhoids = find_inheritance_children(RelationGetRelid(rel), NoLock); + inhoids = find_inheritance_children(RelationGetRelid(rel), NoLock, + include_detached); nparts = list_length(inhoids); /* Allocate working arrays for OIDs, leaf flags, and boundspecs. */ @@ -239,6 +242,7 @@ RelationBuildPartitionDesc(Relation rel) partdesc->boundinfo = partition_bounds_copy(boundinfo, key); partdesc->oids = (Oid *) palloc(nparts * sizeof(Oid)); partdesc->is_leaf = (bool *) palloc(nparts * sizeof(bool)); + partdesc->includes_detached = include_detached; /* * Assign OIDs from the original array into mapped indexes of the @@ -281,7 +285,7 @@ RelationBuildPartitionDesc(Relation rel) * Create a new partition directory object. */ PartitionDirectory -CreatePartitionDirectory(MemoryContext mcxt) +CreatePartitionDirectory(MemoryContext mcxt, bool include_detached) { MemoryContext oldcontext = MemoryContextSwitchTo(mcxt); PartitionDirectory pdir; @@ -296,6 +300,7 @@ CreatePartitionDirectory(MemoryContext mcxt) pdir->pdir_mcxt = mcxt; pdir->pdir_hash = hash_create("partition directory", 256, &ctl, HASH_ELEM | HASH_BLOBS | HASH_CONTEXT); + pdir->include_detached = include_detached; MemoryContextSwitchTo(oldcontext); return pdir; @@ -328,7 +333,7 @@ PartitionDirectoryLookup(PartitionDirectory pdir, Relation rel) */ RelationIncrementReferenceCount(rel); pde->rel = rel; - pde->pd = RelationGetPartitionDesc(rel); + pde->pd = RelationGetPartitionDesc(rel, pdir->include_detached); Assert(pde->pd != NULL); } return pde->pd; diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c index 9b0c376c8c..57ce760edc 100644 --- a/src/backend/tcop/utility.c +++ b/src/backend/tcop/utility.c @@ -1240,6 +1240,25 @@ ProcessUtilitySlow(ParseState *pstate, AlterTableStmt *atstmt = (AlterTableStmt *) parsetree; Oid relid; LOCKMODE lockmode; + ListCell *cell; + + /* + * Disallow ALTER TABLE .. DETACH CONCURRENTLY in a + * transaction block or function. (Perhaps it could be + * allowed in a procedure, but don't hold your breath.) + */ + foreach(cell, atstmt->cmds) + { + AlterTableCmd *cmd = (AlterTableCmd *) lfirst(cell); + + /* Disallow DETACH CONCURRENTLY in a transaction block */ + if (cmd->subtype == AT_DetachPartition) + { + if (((PartitionCmd *) cmd->def)->concurrent) + PreventInTransactionBlock(isTopLevel, + "ALTER TABLE ... DETACH CONCURRENTLY"); + } + } /* * Figure out lock mode, and acquire lock. This also does diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c index 57266f4fc3..329b650e12 100644 --- a/src/bin/psql/describe.c +++ b/src/bin/psql/describe.c @@ -2109,7 +2109,12 @@ describeOneTableDetails(const char *schemaname, printfPQExpBuffer(&buf, "SELECT inhparent::pg_catalog.regclass,\n" - " pg_catalog.pg_get_expr(c.relpartbound, c.oid)"); + " pg_catalog.pg_get_expr(c.relpartbound, c.oid),\n "); + + appendPQExpBuffer(&buf, + pset.sversion >= 140000 ? "inhdetached" : + "false as inhdetached"); + /* If verbose, also request the partition constraint definition */ if (verbose) appendPQExpBufferStr(&buf, @@ -2127,17 +2132,19 @@ describeOneTableDetails(const char *schemaname, { char *parent_name = PQgetvalue(result, 0, 0); char *partdef = PQgetvalue(result, 0, 1); + char *detached = PQgetvalue(result, 0, 2); - printfPQExpBuffer(&tmpbuf, _("Partition of: %s %s"), parent_name, - partdef); + printfPQExpBuffer(&tmpbuf, _("Partition of: %s %s%s"), parent_name, + partdef, + strcmp(detached, "t") == 0 ? " DETACHED" : ""); printTableAddFooter(&cont, tmpbuf.data); if (verbose) { char *partconstraintdef = NULL; - if (!PQgetisnull(result, 0, 2)) - partconstraintdef = PQgetvalue(result, 0, 2); + if (!PQgetisnull(result, 0, 3)) + partconstraintdef = PQgetvalue(result, 0, 3); /* If there isn't any constraint, show that explicitly */ if (partconstraintdef == NULL || partconstraintdef[0] == '\0') printfPQExpBuffer(&tmpbuf, _("No partition constraint")); @@ -3179,9 +3186,18 @@ describeOneTableDetails(const char *schemaname, } /* print child tables (with additional info if partitions) */ - if (pset.sversion >= 100000) + if (pset.sversion >= 140000) printfPQExpBuffer(&buf, - "SELECT c.oid::pg_catalog.regclass, c.relkind," + "SELECT c.oid::pg_catalog.regclass, c.relkind, inhdetached," + " pg_catalog.pg_get_expr(c.relpartbound, c.oid)\n" + "FROM pg_catalog.pg_class c, pg_catalog.pg_inherits i\n" + "WHERE c.oid = i.inhrelid AND i.inhparent = '%s'\n" + "ORDER BY pg_catalog.pg_get_expr(c.relpartbound, c.oid) = 'DEFAULT'," + " c.oid::pg_catalog.regclass::pg_catalog.text;", + oid); + else if (pset.sversion >= 100000) + printfPQExpBuffer(&buf, + "SELECT c.oid::pg_catalog.regclass, c.relkind, false AS inhdetached," " pg_catalog.pg_get_expr(c.relpartbound, c.oid)\n" "FROM pg_catalog.pg_class c, pg_catalog.pg_inherits i\n" "WHERE c.oid = i.inhrelid AND i.inhparent = '%s'\n" @@ -3190,14 +3206,14 @@ describeOneTableDetails(const char *schemaname, oid); else if (pset.sversion >= 80300) printfPQExpBuffer(&buf, - "SELECT c.oid::pg_catalog.regclass, c.relkind, NULL\n" + "SELECT c.oid::pg_catalog.regclass, c.relkind, false AS inhdetached, NULL\n" "FROM pg_catalog.pg_class c, pg_catalog.pg_inherits i\n" "WHERE c.oid = i.inhrelid AND i.inhparent = '%s'\n" "ORDER BY c.oid::pg_catalog.regclass::pg_catalog.text;", oid); else printfPQExpBuffer(&buf, - "SELECT c.oid::pg_catalog.regclass, c.relkind, NULL\n" + "SELECT c.oid::pg_catalog.regclass, c.relkind, false AS inhdetached, NULL\n" "FROM pg_catalog.pg_class c, pg_catalog.pg_inherits i\n" "WHERE c.oid = i.inhrelid AND i.inhparent = '%s'\n" "ORDER BY c.relname;", @@ -3247,11 +3263,13 @@ describeOneTableDetails(const char *schemaname, else printfPQExpBuffer(&buf, "%*s %s", ctw, "", PQgetvalue(result, i, 0)); - if (!PQgetisnull(result, i, 2)) - appendPQExpBuffer(&buf, " %s", PQgetvalue(result, i, 2)); + if (!PQgetisnull(result, i, 3)) + appendPQExpBuffer(&buf, " %s", PQgetvalue(result, i, 3)); if (child_relkind == RELKIND_PARTITIONED_TABLE || child_relkind == RELKIND_PARTITIONED_INDEX) appendPQExpBufferStr(&buf, ", PARTITIONED"); + if (strcmp(PQgetvalue(result, i, 2), "t") == 0) + appendPQExpBuffer(&buf, " (DETACHED)"); if (i < tuples - 1) appendPQExpBufferChar(&buf, ','); diff --git a/src/include/catalog/pg_inherits.h b/src/include/catalog/pg_inherits.h index 0432c3ab88..fcdea9f26f 100644 --- a/src/include/catalog/pg_inherits.h +++ b/src/include/catalog/pg_inherits.h @@ -34,6 +34,7 @@ CATALOG(pg_inherits,2611,InheritsRelationId) Oid inhrelid; Oid inhparent; int32 inhseqno; + bool inhdetached; } FormData_pg_inherits; /* ---------------- @@ -44,7 +45,8 @@ CATALOG(pg_inherits,2611,InheritsRelationId) typedef FormData_pg_inherits *Form_pg_inherits; -extern List *find_inheritance_children(Oid parentrelId, LOCKMODE lockmode); +extern List *find_inheritance_children(Oid parentrelId, bool include_detached, + LOCKMODE lockmode); extern List *find_all_inheritors(Oid parentrelId, LOCKMODE lockmode, List **parents); extern bool has_subclass(Oid relationId); @@ -52,6 +54,7 @@ extern bool has_superclass(Oid relationId); extern bool typeInheritsFrom(Oid subclassTypeId, Oid superclassTypeId); extern void StoreSingleInheritance(Oid relationId, Oid parentOid, int32 seqNumber); -extern bool DeleteInheritsTuple(Oid inhrelid, Oid inhparent); +extern bool DeleteInheritsTuple(Oid inhrelid, Oid inhparent, bool allow_detached, + const char *childname); #endif /* PG_INHERITS_H */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 151bcdb7ef..d291b4d7df 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -859,6 +859,7 @@ typedef struct PartitionCmd NodeTag type; RangeVar *name; /* name of partition to attach/detach */ PartitionBoundSpec *bound; /* FOR VALUES, if attaching */ + bool concurrent; } PartitionCmd; /**************************************************************************** @@ -1845,6 +1846,7 @@ typedef enum AlterTableType AT_GenericOptions, /* OPTIONS (...) */ AT_AttachPartition, /* ATTACH PARTITION */ AT_DetachPartition, /* DETACH PARTITION */ + AT_DetachPartitionFinalize, /* DETACH PARTITION FINALIZE */ AT_AddIdentity, /* ADD IDENTITY */ AT_SetIdentity, /* SET identity column options */ AT_DropIdentity /* DROP IDENTITY */ diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h index 08f22ce211..27414b0912 100644 --- a/src/include/parser/kwlist.h +++ b/src/include/parser/kwlist.h @@ -163,6 +163,7 @@ PG_KEYWORD("false", FALSE_P, RESERVED_KEYWORD) PG_KEYWORD("family", FAMILY, UNRESERVED_KEYWORD) PG_KEYWORD("fetch", FETCH, RESERVED_KEYWORD) PG_KEYWORD("filter", FILTER, UNRESERVED_KEYWORD) +PG_KEYWORD("finalize", FINALIZE, UNRESERVED_KEYWORD) PG_KEYWORD("first", FIRST_P, UNRESERVED_KEYWORD) PG_KEYWORD("float", FLOAT_P, COL_NAME_KEYWORD) PG_KEYWORD("following", FOLLOWING, UNRESERVED_KEYWORD) diff --git a/src/include/partitioning/partdesc.h b/src/include/partitioning/partdesc.h index 70df764981..b1ee9fd5e3 100644 --- a/src/include/partitioning/partdesc.h +++ b/src/include/partitioning/partdesc.h @@ -21,6 +21,7 @@ typedef struct PartitionDescData { int nparts; /* Number of partitions */ + bool includes_detached; /* Does it include detached partitions */ Oid *oids; /* Array of 'nparts' elements containing * partition OIDs in order of the their bounds */ bool *is_leaf; /* Array of 'nparts' elements storing whether @@ -30,9 +31,9 @@ typedef struct PartitionDescData } PartitionDescData; -extern PartitionDesc RelationGetPartitionDesc(Relation rel); +extern PartitionDesc RelationGetPartitionDesc(Relation rel, bool include_detached); -extern PartitionDirectory CreatePartitionDirectory(MemoryContext mcxt); +extern PartitionDirectory CreatePartitionDirectory(MemoryContext mcxt, bool include_detached); extern PartitionDesc PartitionDirectoryLookup(PartitionDirectory, Relation); extern void DestroyPartitionDirectory(PartitionDirectory pdir); diff --git a/src/test/isolation/expected/detach-partition-concurrently-1.out b/src/test/isolation/expected/detach-partition-concurrently-1.out new file mode 100644 index 0000000000..04a0446ccb --- /dev/null +++ b/src/test/isolation/expected/detach-partition-concurrently-1.out @@ -0,0 +1,182 @@ +Parsed test spec with 3 sessions + +starting permutation: s1b s1s s2d s1s s1c s2drop s1s +step s1b: BEGIN; +step s1s: SELECT * FROM d_listp; +a + +1 +2 +step s2d: ALTER TABLE d_listp DETACH PARTITION d_listp2 CONCURRENTLY; <waiting ...> +step s1s: SELECT * FROM d_listp; +a + +1 +step s1c: COMMIT; +step s2d: <... completed> +step s2drop: DROP TABLE d_listp2; +step s1s: SELECT * FROM d_listp; +a + +1 + +starting permutation: s1b s1s s2d s1s s3s s3i s1c s3i s2drop s1s +step s1b: BEGIN; +step s1s: SELECT * FROM d_listp; +a + +1 +2 +step s2d: ALTER TABLE d_listp DETACH PARTITION d_listp2 CONCURRENTLY; <waiting ...> +step s1s: SELECT * FROM d_listp; +a + +1 +step s3s: SELECT * FROM d_listp; +a + +1 +step s3i: SELECT relpartbound IS NULL FROM pg_class where relname = 'd_listp2'; +?column? + +f +step s1c: COMMIT; +step s2d: <... completed> +step s3i: SELECT relpartbound IS NULL FROM pg_class where relname = 'd_listp2'; +?column? + +t +step s2drop: DROP TABLE d_listp2; +step s1s: SELECT * FROM d_listp; +a + +1 + +starting permutation: s1b s1s s2d s1ins s1c +step s1b: BEGIN; +step s1s: SELECT * FROM d_listp; +a + +1 +2 +step s2d: ALTER TABLE d_listp DETACH PARTITION d_listp2 CONCURRENTLY; <waiting ...> +step s1ins: INSERT INTO d_listp VALUES (1); +step s1c: COMMIT; +step s2d: <... completed> + +starting permutation: s1brr s1s s2d s1ins s1c +step s1brr: BEGIN ISOLATION LEVEL REPEATABLE READ; +step s1s: SELECT * FROM d_listp; +a + +1 +2 +step s2d: ALTER TABLE d_listp DETACH PARTITION d_listp2 CONCURRENTLY; <waiting ...> +step s1ins: INSERT INTO d_listp VALUES (1); +step s1c: COMMIT; +step s2d: <... completed> + +starting permutation: s1brr s1s s2d s1s s1c +step s1brr: BEGIN ISOLATION LEVEL REPEATABLE READ; +step s1s: SELECT * FROM d_listp; +a + +1 +2 +step s2d: ALTER TABLE d_listp DETACH PARTITION d_listp2 CONCURRENTLY; <waiting ...> +step s1s: SELECT * FROM d_listp; +a + +1 +step s1c: COMMIT; +step s2d: <... completed> + +starting permutation: s1brr s1prep s1s s2d s1s s1exec s1c +step s1brr: BEGIN ISOLATION LEVEL REPEATABLE READ; +step s1prep: PREPARE f(int) AS INSERT INTO d_listp VALUES ($1); +step s1s: SELECT * FROM d_listp; +a + +1 +2 +step s2d: ALTER TABLE d_listp DETACH PARTITION d_listp2 CONCURRENTLY; <waiting ...> +step s1s: SELECT * FROM d_listp; +a + +1 +step s1exec: EXECUTE f(1); DEALLOCATE f; +step s1c: COMMIT; +step s2d: <... completed> + +starting permutation: s1brr s1prep s1s s2d s1s s1exec2 s1c +step s1brr: BEGIN ISOLATION LEVEL REPEATABLE READ; +step s1prep: PREPARE f(int) AS INSERT INTO d_listp VALUES ($1); +step s1s: SELECT * FROM d_listp; +a + +1 +2 +step s2d: ALTER TABLE d_listp DETACH PARTITION d_listp2 CONCURRENTLY; <waiting ...> +step s1s: SELECT * FROM d_listp; +a + +1 +step s1exec2: EXECUTE f(2); DEALLOCATE f; +step s2d: <... completed> +error in steps s1exec2 s2d: ERROR: no partition of relation "d_listp" found for row +step s1c: COMMIT; + +starting permutation: s1b s1s s2d s3drop1 s1c +step s1b: BEGIN; +step s1s: SELECT * FROM d_listp; +a + +1 +2 +step s2d: ALTER TABLE d_listp DETACH PARTITION d_listp2 CONCURRENTLY; <waiting ...> +step s3drop1: DROP TABLE d_listp; <waiting ...> +step s1c: COMMIT; +step s2d: <... completed> +step s3drop1: <... completed> +error in steps s1c s2d s3drop1: ERROR: partitioned table "d_listp" was removed concurrently + +starting permutation: s1b s1s s2d s3drop2 s1c +step s1b: BEGIN; +step s1s: SELECT * FROM d_listp; +a + +1 +2 +step s2d: ALTER TABLE d_listp DETACH PARTITION d_listp2 CONCURRENTLY; <waiting ...> +step s3drop2: DROP TABLE d_listp2; <waiting ...> +step s1c: COMMIT; +step s2d: <... completed> +step s3drop2: <... completed> +error in steps s1c s2d s3drop2: ERROR: partition "d_listp2" was removed concurrently + +starting permutation: s1b s1s s2d s3detach s1c +step s1b: BEGIN; +step s1s: SELECT * FROM d_listp; +a + +1 +2 +step s2d: ALTER TABLE d_listp DETACH PARTITION d_listp2 CONCURRENTLY; <waiting ...> +step s3detach: ALTER TABLE d_listp DETACH PARTITION d_listp2; <waiting ...> +step s1c: COMMIT; +step s2d: <... completed> +step s3detach: <... completed> +error in steps s1c s2d s3detach: ERROR: cannot detach partition "d_listp2" + +starting permutation: s1b s1s s2d s3rename s1c +step s1b: BEGIN; +step s1s: SELECT * FROM d_listp; +a + +1 +2 +step s2d: ALTER TABLE d_listp DETACH PARTITION d_listp2 CONCURRENTLY; <waiting ...> +step s3rename: ALTER TABLE d_listp2 RENAME TO foobar; <waiting ...> +step s1c: COMMIT; +step s2d: <... completed> +step s3rename: <... completed> diff --git a/src/test/isolation/expected/detach-partition-concurrently-2.out b/src/test/isolation/expected/detach-partition-concurrently-2.out new file mode 100644 index 0000000000..85be707b40 --- /dev/null +++ b/src/test/isolation/expected/detach-partition-concurrently-2.out @@ -0,0 +1,66 @@ +Parsed test spec with 3 sessions + +starting permutation: s1b s1s s2d s3i1 s1c +step s1b: BEGIN; +step s1s: SELECT * FROM d_lp_fk; +a + +1 +2 +step s2d: ALTER TABLE d_lp_fk DETACH PARTITION d_lp_fk_1 CONCURRENTLY; <waiting ...> +step s3i1: INSERT INTO d_lp_fk_r VALUES (1); +ERROR: insert or update on table "d_lp_fk_r" violates foreign key constraint "d_lp_fk_r_a_fkey" +step s1c: COMMIT; +step s2d: <... completed> + +starting permutation: s1b s1s s2d s3i2 s3i2 s1c +step s1b: BEGIN; +step s1s: SELECT * FROM d_lp_fk; +a + +1 +2 +step s2d: ALTER TABLE d_lp_fk DETACH PARTITION d_lp_fk_1 CONCURRENTLY; <waiting ...> +step s3i2: INSERT INTO d_lp_fk_r VALUES (2); +step s3i2: INSERT INTO d_lp_fk_r VALUES (2); +step s1c: COMMIT; +step s2d: <... completed> + +starting permutation: s1b s1s s3i1 s2d s1c +step s1b: BEGIN; +step s1s: SELECT * FROM d_lp_fk; +a + +1 +2 +step s3i1: INSERT INTO d_lp_fk_r VALUES (1); +step s2d: ALTER TABLE d_lp_fk DETACH PARTITION d_lp_fk_1 CONCURRENTLY; +ERROR: removing partition "d_lp_fk_1" violates foreign key constraint "d_lp_fk_r_a_fkey1" +step s1c: COMMIT; + +starting permutation: s1b s1s s3i2 s2d s1c +step s1b: BEGIN; +step s1s: SELECT * FROM d_lp_fk; +a + +1 +2 +step s3i2: INSERT INTO d_lp_fk_r VALUES (2); +step s2d: ALTER TABLE d_lp_fk DETACH PARTITION d_lp_fk_1 CONCURRENTLY; <waiting ...> +step s1c: COMMIT; +step s2d: <... completed> + +starting permutation: s1b s1s s3b s2d s3i1 s1c s3c +step s1b: BEGIN; +step s1s: SELECT * FROM d_lp_fk; +a + +1 +2 +step s3b: BEGIN; +step s2d: ALTER TABLE d_lp_fk DETACH PARTITION d_lp_fk_1 CONCURRENTLY; <waiting ...> +step s3i1: INSERT INTO d_lp_fk_r VALUES (1); +ERROR: insert or update on table "d_lp_fk_r" violates foreign key constraint "d_lp_fk_r_a_fkey" +step s1c: COMMIT; +step s2d: <... completed> +step s3c: COMMIT; diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule index 218c87b24b..a9c1abc244 100644 --- a/src/test/isolation/isolation_schedule +++ b/src/test/isolation/isolation_schedule @@ -24,6 +24,8 @@ test: deadlock-hard test: deadlock-soft test: deadlock-soft-2 test: deadlock-parallel +test: detach-partition-concurrently-1 +test: detach-partition-concurrently-2 test: fk-contention test: fk-deadlock test: fk-deadlock2 diff --git a/src/test/isolation/specs/detach-partition-concurrently-1.spec b/src/test/isolation/specs/detach-partition-concurrently-1.spec new file mode 100644 index 0000000000..ca92530824 --- /dev/null +++ b/src/test/isolation/specs/detach-partition-concurrently-1.spec @@ -0,0 +1,71 @@ +# Test that detach partition concurrently makes the partition invisible at the +# correct time. + +setup +{ + DROP TABLE IF EXISTS d_listp, d_listp1, d_listp2; + CREATE TABLE d_listp (a int) PARTITION BY LIST(a); + CREATE TABLE d_listp1 PARTITION OF d_listp FOR VALUES IN (1); + CREATE TABLE d_listp2 PARTITION OF d_listp FOR VALUES IN (2); + INSERT INTO d_listp VALUES (1),(2); +} + +teardown { + DROP TABLE IF EXISTS d_listp, d_listp2; +} + +session "s1" +step "s1b" { BEGIN; } +step "s1brr" { BEGIN ISOLATION LEVEL REPEATABLE READ; } +step "s1s" { SELECT * FROM d_listp; } +step "s1ins" { INSERT INTO d_listp VALUES (1); } +step "s1prep" { PREPARE f(int) AS INSERT INTO d_listp VALUES ($1); } +step "s1exec" { EXECUTE f(1); DEALLOCATE f; } +step "s1exec2" { EXECUTE f(2); DEALLOCATE f; } +step "s1c" { COMMIT; } + +session "s2" +step "s2d" { ALTER TABLE d_listp DETACH PARTITION d_listp2 CONCURRENTLY; } +step "s2drop" { DROP TABLE d_listp2; } + +session "s3" +step "s3s" { SELECT * FROM d_listp; } +step "s3i" { SELECT relpartbound IS NULL FROM pg_class where relname = 'd_listp2'; } +step "s3drop1" { DROP TABLE d_listp; } +step "s3drop2" { DROP TABLE d_listp2; } +step "s3detach" { ALTER TABLE d_listp DETACH PARTITION d_listp2; } +step "s3rename" { ALTER TABLE d_listp2 RENAME TO foobar; } + +# The transaction that detaches hangs until it sees any older transaction +# terminate. +permutation "s1b" "s1s" "s2d" "s1s" "s1c" "s2drop" "s1s" + +# relpartbound remains set until s1 commits +# XXX this could be timing dependent :-( +permutation "s1b" "s1s" "s2d" "s1s" "s3s" "s3i" "s1c" "s3i" "s2drop" "s1s" + +# Session s1 no longer sees the partition ... +permutation "s1b" "s1s" "s2d" "s1ins" "s1c" +# ... regardless of the isolation level. +permutation "s1brr" "s1s" "s2d" "s1ins" "s1c" +# However, DETACH CONCURRENTLY is not isolation-level-correct +permutation "s1brr" "s1s" "s2d" "s1s" "s1c" + +# a prepared query is not blocked +permutation "s1brr" "s1prep" "s1s" "s2d" "s1s" "s1exec" "s1c" +permutation "s1brr" "s1prep" "s1s" "s2d" "s1s" "s1exec2" "s1c" + + +# Note: the following tests are not very interesting in isolationtester because +# the way this tool handles multiple sessions: it'll only run the drop/detach/ +# rename commands after s2 is blocked waiting. It would be more useful to +# run the DDL when s2 commits its first transaction instead. Maybe these should +# become TAP tests someday. + +# What happens if the partition or the parent table is dropped while waiting? +permutation "s1b" "s1s" "s2d" "s3drop1" "s1c" +permutation "s1b" "s1s" "s2d" "s3drop2" "s1c" +# Also try a non-concurrent detach while the other one is waiting +permutation "s1b" "s1s" "s2d" "s3detach" "s1c" +# and some other DDL +permutation "s1b" "s1s" "s2d" "s3rename" "s1c" diff --git a/src/test/isolation/specs/detach-partition-concurrently-2.spec b/src/test/isolation/specs/detach-partition-concurrently-2.spec new file mode 100644 index 0000000000..9281c80a69 --- /dev/null +++ b/src/test/isolation/specs/detach-partition-concurrently-2.spec @@ -0,0 +1,41 @@ +# Test that detach partition concurrently makes the partition safe +# for foreign keys that reference it. + +setup +{ + DROP TABLE IF EXISTS d_lp_fk, d_lp_fk_1, d_lp_fk_2, d_lp_fk_r; + + CREATE TABLE d_lp_fk (a int PRIMARY KEY) PARTITION BY LIST(a); + CREATE TABLE d_lp_fk_1 PARTITION OF d_lp_fk FOR VALUES IN (1); + CREATE TABLE d_lp_fk_2 PARTITION OF d_lp_fk FOR VALUES IN (2); + INSERT INTO d_lp_fk VALUES (1), (2); + + CREATE TABLE d_lp_fk_r (a int references d_lp_fk); +} + +teardown { DROP TABLE IF EXISTS d_lp_fk, d_lp_fk_1, d_lp_fk_2, d_lp_fk_r; } + +session "s1" +step "s1b" { BEGIN; } +step "s1s" { SELECT * FROM d_lp_fk; } +step "s1c" { COMMIT; } + +session "s2" +step "s2d" { ALTER TABLE d_lp_fk DETACH PARTITION d_lp_fk_1 CONCURRENTLY; } + +session "s3" +step "s3b" { BEGIN; } +step "s3i1" { INSERT INTO d_lp_fk_r VALUES (1); } +step "s3i2" { INSERT INTO d_lp_fk_r VALUES (2); } +step "s3c" { COMMIT; } + +# The transaction that detaches hangs until it sees any older transaction +# terminate. +permutation "s1b" "s1s" "s2d" "s3i1" "s1c" +permutation "s1b" "s1s" "s2d" "s3i2" "s3i2" "s1c" + +permutation "s1b" "s1s" "s3i1" "s2d" "s1c" +permutation "s1b" "s1s" "s3i2" "s2d" "s1c" + +# what if s3 has an uncommitted insertion? +permutation "s1b" "s1s" "s3b" "s2d" "s3i1" "s1c" "s3c" -- 2.20.1 --ikeVEW9yuYc//A+q-- ^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: Row pattern recognition @ 2026-01-13 02:41 Tatsuo Ishii <[email protected]> 0 siblings, 1 reply; 5+ messages in thread From: Tatsuo Ishii @ 2026-01-13 02:41 UTC (permalink / raw) To: [email protected]; +Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; pgsql-hackers >> What is PATH function? It's not in R010 or R020 as far as I know. >> > > A Match Path is essentially an array consisting of the CLASSIFIER() values > of all rows > within a matched sequence, arranged in chronological order. So, the PATH function is Oracle specific? Or your own proposal? >> > Longer-term, the goal is to get to MATCH_RECOGNIZE in the FROM clause for >> > full SQL standard compliance. >> >> What about MEASURES clause? Without it, many things in the standard >> cannot be implemented. >> > > I admit that my current understanding of the full SQL standard is still > evolving. However, > looking at the current codebase, I believed that the syntax I've > implemented so far > could be addressed even before mastering every detail of the standard. > That is why I have taken a 'code-first' approach for this stage. > > Of course, reaching 'FULL' compliance is undoubtedly the ultimate goal. > Given > the complexity, I believe the features should be released in several > incremental phases. I totally agree with the incremental approach. > I trust that you will help define the strategic roadmap for these releases. > As I continue to > deepen my understanding of the standard, I will do my best to contribute to > the > technical roadmap as well. Ok, here is my proposal for the strategic roadmap. This should be a starting point of the discussion. (1) The first release: this will a subset of R020 (RPR in Window clause), but will serve many of pratical uses of RPR. The following features below will not be included. - MEASURES - SUBSET - SEEK - AFTER MATCH SKIP TO - AFTER MATCH SKIP TO FIRST - AFTER MATCH SKIP TO LAST - PERMUTE Note 1: this list needs discussion and may be changed when the release is made. Note 2: Because MEASURES cannot be used in this release, all primary row patter variables in the DEFINE clause must be unqualified ones (i.e. "price", not "A.price") and they are implicitly qualified by the universal row pattern variable. In this case the universal row pattern variable is mapped to the current row, thus "price" in the DEFINE clause is evaluated in the current row. After (1) we can choose either (a) extend support for R020, i.e. add more features to the first release, especially MEASURES or (b) support a subset of R010 (MATCH_RECOGNIZE). >> > Let me know if you have any concerns or suggestions about the approach. > I will rebase my work on your latest v37 patch and move forward with that > for the next version. I’ll submit the updated patch soon. Thanks! > I completely agree that those optimizations belong in the optimizer/planner > phase. > I intentionally placed them in the parse analysis phase for now to make > debugging > easier during this initial development stage. Once the logic is stabilized, > I will move > them to the planner as you suggested to ensure proper deparsing. Understood. Looks like a good strategy. Best regards, -- Tatsuo Ishii SRA OSS K.K. English: http://www.sraoss.co.jp/index_en/ Japanese:http://www.sraoss.co.jp ^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: Row pattern recognition @ 2026-01-13 02:53 Henson Choi <[email protected]> parent: Tatsuo Ishii <[email protected]> 0 siblings, 1 reply; 5+ messages in thread From: Henson Choi @ 2026-01-13 02:53 UTC (permalink / raw) To: Tatsuo Ishii <[email protected]>; +Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; pgsql-hackers > > So, the PATH function is Oracle specific? Or your own proposal? > Match PATH is essentially the internal metadata or the trace log of which Row Pattern Variable (Classifier) was assigned to each row during the state machine's execution. Match PATH is simply a set of Classifiers stored in the internal state. > I trust that you will help define the strategic roadmap for these > releases. > > As I continue to > > deepen my understanding of the standard, I will do my best to contribute > to > > the > > technical roadmap as well. > > Ok, here is my proposal for the strategic roadmap. This should be a > starting point of the discussion. > > (1) The first release: this will a subset of R020 (RPR in Window > clause), but will serve many of pratical uses of RPR. The > following features below will not be included. > > - MEASURES > - SUBSET > - SEEK > - AFTER MATCH SKIP TO > - AFTER MATCH SKIP TO FIRST > - AFTER MATCH SKIP TO LAST > - PERMUTE > > Note 1: this list needs discussion and may be changed when the release > is made. > > Note 2: Because MEASURES cannot be used in this release, all primary > row patter variables in the DEFINE clause must be unqualified ones > (i.e. "price", not "A.price") and they are implicitly qualified by the > universal row pattern variable. In this case the universal row pattern > variable is mapped to the current row, thus "price" in the DEFINE > clause is evaluated in the current row. > > After (1) we can choose either (a) extend support for R020, i.e. add > more features to the first release, especially MEASURES or (b) support > a subset of R010 (MATCH_RECOGNIZE). > I am currently fixing the code to satisfy the existing test cases. Once the modifications are complete, I'll share the details so we can discuss the differences between the current status and the proposed roadmap. Best regards, Henson ^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: Row pattern recognition @ 2026-01-14 15:31 Henson Choi <[email protected]> parent: Henson Choi <[email protected]> 0 siblings, 1 reply; 5+ messages in thread From: Henson Choi @ 2026-01-14 15:31 UTC (permalink / raw) To: Tatsuo Ishii <[email protected]>; Jacob Champion <[email protected]>; +Cc: [email protected]; [email protected]; [email protected]; [email protected]; pgsql-hackers Hi Ishii-san, The attached patch implements streaming NFA-based pattern matching. This is based on your v37 patch and the pattern grammar work I submitted earlier. This is a testable draft for review, not production-ready yet. It allows in-depth verification of pattern optimization, pattern matching/merging, context absorption, and window behavior. Summary of changes: 1. NFA-based executor replacing regex approach - Streaming execution: processes one row at a time - Eliminates regex compilation overhead and 26-variable limit - Supports up to 252 pattern variables Note on 252 limit: Each match path stores a sequence of classifiers (one per matched row). Using 1-byte encoding keeps memory usage reasonable for long matches. Some values are reserved for pattern representation; this limit may decrease if more are needed. 200+ should be sufficient for practical use. 2. Pattern optimization moved to planner phase (as you suggested) - pg_get_viewdef() now shows original pattern - EXPLAIN shows optimized pattern - Optimizations: VAR merge (A A A -> A{3}), GROUP merge ((A B) (A B)+ -> (A B){2,}), duplicate removal, SEQ/ALT flattening, quantifier multiplication - These optimizations also enable absorption for patterns that wouldn't otherwise qualify (e.g., (A B) (A B)+ -> (A B){2,} becomes absorbable) 3. Multi-context support with context absorption - Multiple concurrent NFA contexts for overlapping matches - Absorption optimization: earlier context absorbs later ones when they reach the same endpoint Note on absorption scope: I haven't fully worked out the theoretical bounds for safe absorption yet. The current implementation covers most practically useful cases, but a complete solution would require more formal analysis. Current implementation handles cases where the behavior is clear: - Only patterns with unbounded first element (A+, A*, (A B)+, etc.) - Only when there's exactly one unbounded element at top level - Per-branch analysis for alternation patterns This may be extended as we better understand the theoretical limits. Examples: - A+ B : absorbable (A+ unbounded first, same endpoint guaranteed) - A* B : absorbable (A* unbounded first) - A B+ : NOT absorbable (unbounded not first, different start points) - A+ B+ : partial (absorbable while in A+, not after entering B+) - A? B : NOT absorbable (? is bounded: {0,1}) - A{2,} B : absorbable (unbounded quantifier at first element) - A{2,5} B : NOT absorbable (bounded quantifier) - (A B){2,}: absorbable (GROUP with unbounded quantifier at top level) - A+ | B+ : absorbable per-branch (each branch analyzed independently) - A+ | B C : partial per-branch (A+ absorbable, B C not) - (A+ B)+ : NOT YET (absorbable at first A+ entry, nested analysis not implemented) 4. Reluctant quantifiers - Will be supported in a follow-up patch - I need to study some edge cases in the spec first Also updated: parser/README, typedefs.list Testing: The rpr regression test has been expanded significantly with tests for: - Alternation and grouping patterns - Quantifiers ({n}, {n,}, {n,m}, +, *, ?) - Pattern optimization verification - Multi-context matching - Context absorption behavior - Lexical order compliance Below is a reference query designed to test NFA functionality in a clean, unambiguous way. Using explicit ARRAY flags eliminates edge cases and makes the pattern matching behavior completely predictable, allowing clear verification of multi-context execution and alternation handling. Example (multi-context overlapping match test): Uses ARRAY flags with ANY() to make each row's pattern membership explicit, making test behavior predictable and easy to verify. WITH data AS ( SELECT * FROM (VALUES (1, ARRAY['A']), (2, ARRAY['B']), (3, ARRAY['C']), (4, ARRAY['D']), (5, ARRAY['E']), (6, ARRAY['F']) ) AS t(id, flags) ) SELECT id, flags, first_value(id) OVER w AS start, last_value(id) OVER w AS end FROM data WINDOW w AS ( ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING AFTER MATCH SKIP TO NEXT ROW PATTERN (A B C D E | B C D | C D E F) DEFINE A AS 'A' = ANY(flags), B AS 'B' = ANY(flags), ... ); -- Result (SKIP TO NEXT ROW enables multi-context): id | flags | start | end ----+-------+-------+----- 1 | {A} | 1 | 5 <- A B C D E 2 | {B} | 2 | 4 <- B C D 3 | {C} | 3 | 6 <- C D E F 4 | {D} | | 5 | {E} | | 6 | {F} | | Performance: Regression test timing (rpr test): Parallel group (with other tests): Before (v37 regex): 198 ms After (NFA): 123 ms (~38% faster) After (NFA + tests): 127 ms (still ~36% faster) Single test (isolated): Before (v37 regex): 38 ms After (NFA): 26 ms (~32% faster) After (NFA + tests): 41 ms (+8%, more tests added) Note: The rpr test was moved out of the parallel group in the schedule for more accurate timing measurement. The streaming NFA approach eliminates regex compilation overhead per-row and reduces redundant DEFINE evaluations, resulting in better performance especially for complex patterns. Let me know if you have any questions or concerns. Best regards, Henson > From 07251e04688c389938dd268c28b1a00f25bec931 Mon Sep 17 00:00:00 2001 From: Henson Choi <[email protected]> Date: Wed, 14 Jan 2026 23:43:13 +0900 Subject: [PATCH] Row pattern recognition: Implement streaming NFA-based pattern matching --- src/backend/commands/explain.c | 143 ++ src/backend/executor/nodeWindowAgg.c | 2067 +++++++++--------- src/backend/nodes/copyfuncs.c | 39 + src/backend/nodes/equalfuncs.c | 33 + src/backend/nodes/outfuncs.c | 51 + src/backend/nodes/readfuncs.c | 79 + src/backend/optimizer/plan/Makefile | 1 + src/backend/optimizer/plan/createplan.c | 68 +- src/backend/optimizer/plan/meson.build | 1 + src/backend/optimizer/plan/rpr.c | 1024 +++++++++ src/backend/parser/Makefile | 1 + src/backend/parser/README | 1 + src/backend/parser/gram.y | 281 ++- src/backend/parser/meson.build | 1 + src/backend/parser/parse_clause.c | 262 +-- src/backend/parser/parse_rpr.c | 340 +++ src/backend/parser/scan.l | 4 +- src/backend/utils/adt/ruleutils.c | 125 +- src/include/nodes/execnodes.h | 63 +- src/include/nodes/parsenodes.h | 43 +- src/include/nodes/plannodes.h | 73 +- src/include/optimizer/rpr.h | 46 + src/include/parser/parse_clause.h | 3 + src/include/parser/parse_rpr.h | 22 + src/test/regress/expected/rpr.out | 2632 ++++++++++++++++++----- src/test/regress/sql/rpr.sql | 827 +++++++ src/tools/pgindent/typedefs.list | 7 + 27 files changed, 6297 insertions(+), 1940 deletions(-) create mode 100644 src/backend/optimizer/plan/rpr.c create mode 100644 src/backend/parser/parse_rpr.c create mode 100644 src/include/optimizer/rpr.h create mode 100644 src/include/parser/parse_rpr.h diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c index b7bb111688c..969c9195864 100644 --- a/src/backend/commands/explain.c +++ b/src/backend/commands/explain.c @@ -29,6 +29,7 @@ #include "nodes/extensible.h" #include "nodes/makefuncs.h" #include "nodes/nodeFuncs.h" +#include "optimizer/rpr.h" #include "parser/analyze.h" #include "parser/parsetree.h" #include "rewrite/rewriteHandler.h" @@ -117,6 +118,8 @@ static void show_window_def(WindowAggState *planstate, static void show_window_keys(StringInfo buf, PlanState *planstate, int nkeys, AttrNumber *keycols, List *ancestors, ExplainState *es); +static void append_rpr_quantifier(StringInfo buf, RPRPatternElement *elem); +static char *deparse_rpr_pattern(RPRPattern *pattern); static void show_storage_info(char *maxStorageType, int64 maxSpaceUsed, ExplainState *es); static void show_tablesample(TableSampleClause *tsc, PlanState *planstate, @@ -2889,6 +2892,134 @@ show_sortorder_options(StringInfo buf, Node *sortexpr, } } +/* + * Append quantifier suffix for a pattern element. + */ +static void +append_rpr_quantifier(StringInfo buf, RPRPatternElement *elem) +{ + if (elem->min == 1 && elem->max == 1) + return; /* no quantifier */ + else if (elem->min == 0 && elem->max == RPR_QUANTITY_INF) + appendStringInfoChar(buf, '*'); + else if (elem->min == 1 && elem->max == RPR_QUANTITY_INF) + appendStringInfoChar(buf, '+'); + else if (elem->min == 0 && elem->max == 1) + appendStringInfoChar(buf, '?'); + else if (elem->max == RPR_QUANTITY_INF) + appendStringInfo(buf, "{%d,}", elem->min); + else if (elem->min == elem->max) + appendStringInfo(buf, "{%d}", elem->min); + else + appendStringInfo(buf, "{%d,%d}", elem->min, elem->max); + + if (RPRElemIsReluctant(elem)) + appendStringInfoChar(buf, '?'); +} + +/* + * Deparse a compiled RPRPattern (bytecode) back to pattern string. + * Simple approach: output parens for each depth level as-is. + */ +static char * +deparse_rpr_pattern(RPRPattern *pattern) +{ + StringInfoData buf; + int i; + RPRDepth prevDepth = 0; + bool needSpace = false; + RPRElemIdx altSepAt = RPR_ELEMIDX_INVALID; + + if (pattern == NULL || pattern->numElements == 0) + return NULL; + + initStringInfo(&buf); + + for (i = 0; i < pattern->numElements; i++) + { + RPRPatternElement *elem = &pattern->elements[i]; + + if (RPRElemIsFin(elem)) + break; + + /* Alternation separator */ + if (altSepAt == i) + { + appendStringInfoString(&buf, " | "); + needSpace = false; + altSepAt = RPR_ELEMIDX_INVALID; + } + + if (RPRElemIsAlt(elem)) + { + /* Open parens up to ALT's content depth */ + while (prevDepth <= elem->depth) + { + if (needSpace) + appendStringInfoChar(&buf, ' '); + appendStringInfoChar(&buf, '('); + prevDepth++; + needSpace = false; + } + continue; + } + + if (RPRElemIsEnd(elem)) + { + /* Close down to END's depth, output quantifier */ + while (prevDepth > elem->depth + 1) + { + appendStringInfoChar(&buf, ')'); + prevDepth--; + } + appendStringInfoChar(&buf, ')'); + append_rpr_quantifier(&buf, elem); + prevDepth = elem->depth; + needSpace = true; + continue; + } + + if (RPRElemIsVar(elem)) + { + /* Open parens for depth increase */ + while (prevDepth < elem->depth) + { + if (needSpace) + appendStringInfoChar(&buf, ' '); + appendStringInfoChar(&buf, '('); + prevDepth++; + needSpace = false; + } + + /* Close parens for depth decrease */ + while (prevDepth > elem->depth) + { + appendStringInfoChar(&buf, ')'); + prevDepth--; + } + + if (needSpace) + appendStringInfoChar(&buf, ' '); + + appendStringInfoString(&buf, pattern->varNames[elem->varId]); + append_rpr_quantifier(&buf, elem); + needSpace = true; + + if (elem->jump != RPR_ELEMIDX_INVALID) + altSepAt = elem->jump; + } + } + + /* Close remaining */ + while (prevDepth > 0) + { + appendStringInfoChar(&buf, ')'); + prevDepth--; + } + + return buf.data; +} + /* * Show the window definition for a WindowAgg node. */ @@ -2947,6 +3078,18 @@ show_window_def(WindowAggState *planstate, List *ancestors, ExplainState *es) appendStringInfoChar(&wbuf, ')'); ExplainPropertyText("Window", wbuf.data, es); pfree(wbuf.data); + + /* Show Row Pattern Recognition pattern if present */ + if (wagg->rpPattern != NULL) + { + char *patternStr = deparse_rpr_pattern(wagg->rpPattern); + + if (patternStr != NULL) + { + ExplainPropertyText("Pattern", patternStr, es); + pfree(patternStr); + } + } } /* diff --git a/src/backend/executor/nodeWindowAgg.c b/src/backend/executor/nodeWindowAgg.c index 6a945a1a94d..cf43c1f6127 100644 --- a/src/backend/executor/nodeWindowAgg.c +++ b/src/backend/executor/nodeWindowAgg.c @@ -42,8 +42,10 @@ #include "executor/nodeWindowAgg.h" #include "miscadmin.h" #include "nodes/nodeFuncs.h" +#include "nodes/plannodes.h" #include "optimizer/clauses.h" #include "optimizer/optimizer.h" +#include "optimizer/rpr.h" #include "parser/parse_agg.h" #include "parser/parse_coerce.h" #include "regex/regex.h" @@ -173,24 +175,6 @@ typedef struct WindowStatePerAggData bool restart; /* need to restart this agg in this cycle? */ } WindowStatePerAggData; -/* - * Set of StringInfo. Used in RPR. - */ -#define STRSET_FROZEN (1 << 0) /* string is frozen */ -#define STRSET_DISCARDED (1 << 1) /* string is scheduled to be discarded */ -#define STRSET_MATCHED (1 << 2) /* string is confirmed to be matched - * with pattern */ - -typedef struct StringSet -{ - StringInfo *str_set; - Size set_size; /* current array allocation size in number of - * items */ - int set_index; /* current used size */ - int *info; /* an array of information bit per StringInfo. - * see above */ -} StringSet; - /* * Structure used by check_rpr_navigation() and rpr_navigation_walker(). */ @@ -269,44 +253,29 @@ static void register_reduced_frame_map(WindowAggState *winstate, int64 pos, static void clear_reduced_frame_map(WindowAggState *winstate); static void update_reduced_frame(WindowObject winobj, int64 pos); -static int64 evaluate_pattern(WindowObject winobj, int64 current_pos, - char *vname, StringInfo encoded_str, bool *result); - -static bool get_slots(WindowObject winobj, int64 current_pos); - -static int search_str_set(WindowAggState *winstate, - StringSet *input_str_set); -static StringSet *generate_patterns(WindowAggState *winstate, StringSet *input_str_set, - char *pattern, VariablePos *variable_pos, - char tail_pattern_initial); -static int add_pattern(WindowAggState *winstate, StringInfo old, int old_info, - StringSet *new_str_set, - char c, char *pattern, char tail_pattern_initial, - int resultlen); -static int freeze_pattern(WindowAggState *winstate, StringInfo old, int old_info, - StringSet *new_str_set, - char *pattern, int resultlen); -static char pattern_initial(WindowAggState *winstate, char *vname); -static int do_pattern_match(WindowAggState *winstate, char *pattern, char *encoded_str, int len); -static void pattern_regfree(WindowAggState *winstate); - -static StringSet *string_set_init(void); -static void string_set_add(StringSet *string_set, StringInfo str, int info); -static StringInfo string_set_get(StringSet *string_set, int index, int *flag); -static int string_set_get_size(StringSet *string_set); -static void string_set_discard(StringSet *string_set); -static VariablePos *variable_pos_init(void); -static void variable_pos_register(VariablePos *variable_pos, char initial, - int pos); -static bool variable_pos_compare(VariablePos *variable_pos, - char initial1, char initial2); -static int variable_pos_fetch(VariablePos *variable_pos, char initial, - int index); -static VariablePos *variable_pos_build(WindowAggState *winstate); - static void check_rpr_navigation(Node *node, bool is_prev); static bool rpr_navigation_walker(Node *node, void *context); +/* NFA-based pattern matching functions */ +static RPRNFAState *nfa_state_alloc(WindowAggState *winstate); +static void nfa_state_free(WindowAggState *winstate, RPRNFAState *state); +static void nfa_state_free_list(WindowAggState *winstate, RPRNFAState *list); +static RPRNFAState *nfa_state_clone(WindowAggState *winstate, int16 elemIdx, + int16 altPriority, int16 *counts, + RPRNFAState *list); +static bool nfa_evaluate_row(WindowObject winobj, int64 pos, bool *varMatched, bool *anyMatch); +static RPRNFAContext *nfa_context_alloc(WindowAggState *winstate); +static void nfa_unlink_context(WindowAggState *winstate, RPRNFAContext *ctx); +static void nfa_context_free(WindowAggState *winstate, RPRNFAContext *ctx); +static void nfa_start_context(WindowAggState *winstate, int64 startPos); +static void nfa_step_single(WindowAggState *winstate, RPRNFAContext *ctx, + RPRNFAState *state, bool *varMatched, int64 currentPos); +static void nfa_finalize_boundary(WindowAggState *winstate, RPRNFAContext *ctx, + int64 matchEndPos); +static RPRNFAContext *nfa_find_context_for_pos(WindowAggState *winstate, int64 pos); +static void nfa_remove_contexts_up_to(WindowAggState *winstate, int64 endPos); +static void nfa_absorb_contexts(WindowAggState *winstate, RPRNFAContext *excludeCtx, int64 currentPos); + /* * Not null info bit array consists of 2-bit items */ @@ -1623,6 +1592,13 @@ release_partition(WindowAggState *winstate) tuplestore_clear(winstate->buffer); winstate->partition_spooled = false; winstate->next_partition = true; + + /* Reset NFA state for new partition */ + winstate->nfaContext = NULL; + winstate->nfaContextTail = NULL; + winstate->nfaContextFree = NULL; + winstate->nfaStateFree = NULL; + winstate->nfaLastProcessedRow = -1; } /* @@ -2985,9 +2961,15 @@ ExecInitWindowAgg(WindowAgg *node, EState *estate, int eflags) /* Set up SKIP TO type */ winstate->rpSkipTo = node->rpSkipTo; - /* Set up row pattern recognition PATTERN clause */ - winstate->patternVariableList = node->patternVariable; - winstate->patternRegexpList = node->patternRegexp; + /* Set up row pattern recognition PATTERN clause (compiled NFA) */ + winstate->rpPattern = node->rpPattern; + + /* Calculate NFA state size for allocation */ + if (node->rpPattern != NULL) + { + winstate->nfaStateSize = offsetof(RPRNFAState, counts) + + sizeof(int16) * node->rpPattern->maxDepth; + } /* Set up row pattern recognition DEFINE clause */ winstate->defineInitial = node->defineInitial; @@ -3019,16 +3001,24 @@ ExecInitWindowAgg(WindowAgg *node, EState *estate, int eflags) lappend(winstate->defineClauseList, exps); } } - /* set up regular expression compiled result cache for RPR */ - winstate->patbuf = NULL; - memset(&winstate->preg, 0, sizeof(winstate->preg)); + + /* Initialize NFA free lists for row pattern matching */ + winstate->nfaContext = NULL; + winstate->nfaContextTail = NULL; + winstate->nfaContextFree = NULL; + winstate->nfaStateFree = NULL; + winstate->nfaLastProcessedRow = -1; /* - * Build variable_pos + * Allocate varMatched array for NFA evaluation. + * With the new varNames ordering (DEFINE order first), varId == defineIdx + * for all defined variables, so no mapping is needed. */ - if (winstate->defineInitial) - winstate->variable_pos = variable_pos_build(winstate); - + if (list_length(winstate->defineVariableList) > 0) + winstate->nfaVarMatched = palloc0(sizeof(bool) * + list_length(winstate->defineVariableList)); + else + winstate->nfaVarMatched = NULL; winstate->all_first = true; winstate->partition_spooled = false; winstate->more_partitions = false; @@ -3173,8 +3163,6 @@ ExecEndWindowAgg(WindowAggState *node) pfree(node->perfunc); pfree(node->peragg); - pattern_regfree(node); - outerPlan = outerPlanState(node); ExecEndNode(outerPlan); } @@ -4540,7 +4528,7 @@ WinGetFuncArgCurrent(WindowObject winobj, int argno, bool *isnull) static bool rpr_is_defined(WindowAggState *winstate) { - return winstate->patternVariableList != NIL; + return winstate->rpPattern != NULL; } /* @@ -4614,7 +4602,7 @@ row_is_in_reduced_frame(WindowObject winobj, int64 pos) break; default: - elog(ERROR, "unrecognized state: %d at: " INT64_FORMAT, + elog(ERROR, "Unrecognized state: %d at: " INT64_FORMAT, state, pos); break; } @@ -4708,1229 +4696,1196 @@ register_reduced_frame_map(WindowAggState *winstate, int64 pos, int val) /* * update_reduced_frame - * Update reduced frame info. + * Update reduced frame info using multi-context NFA pattern matching. + * + * Maintains multiple NFA contexts simultaneously, one for each potential + * match start position. This allows sharing row evaluations across contexts, + * avoiding redundant DEFINE clause evaluations when rewinding for SKIP TO + * NEXT ROW mode. + * + * Key optimizations: + * - Row evaluations (expensive DEFINE clauses) happen only once per row + * - All active contexts share the same evaluation results + * - Contexts persist across calls, enabling O(n) DEFINE evaluations */ static void update_reduced_frame(WindowObject winobj, int64 pos) { WindowAggState *winstate = winobj->winstate; - ListCell *lc1, - *lc2; - bool expression_result; - int num_matched_rows; - int64 original_pos; - bool anymatch; - StringInfo encoded_str; - StringSet *str_set; - bool greedy = false; - int64 result_pos, - i; - int init_size; + RPRNFAContext *targetCtx; + RPRNFAContext *firstCtx; + int64 matchLength = 0; + int64 currentPos; + int64 startPos; + int frameOptions = winstate->frameOptions; + bool hasLimitedFrame; + int64 frameOffset = 0; /* - * Set of pattern variables evaluated to true. Each character corresponds - * to pattern variable. Example: str_set[0] = "AB"; str_set[1] = "AC"; In - * this case at row 0 A and B are true, and A and C are true in row 1. + * Check if we have a limited frame (ROWS ... N FOLLOWING). + * Each context needs its own frame end based on matchStartRow + offset. */ - - /* initialize pattern variables set */ - str_set = string_set_init(); - - /* save original pos */ - original_pos = pos; + hasLimitedFrame = (frameOptions & FRAMEOPTION_ROWS) && + !(frameOptions & FRAMEOPTION_END_UNBOUNDED_FOLLOWING); + if (hasLimitedFrame && winstate->endOffsetValue != 0) + frameOffset = DatumGetInt64(winstate->endOffsetValue); /* - * Calculate initial memory allocation size from the number of pattern - * variables, + * Case 1: pos is before any existing context's start position. + * This means the position was already processed and determined unmatched. + * Note: contexts are added at head with increasing positions, so we need + * to find the minimum matchStartRow (the oldest context). */ - init_size = sizeof(char) * list_length(winstate->patternVariableList) + 1; + { + int64 minStartRow = INT64_MAX; + for (firstCtx = winstate->nfaContext; firstCtx != NULL; firstCtx = firstCtx->next) + { + if (firstCtx->matchStartRow < minStartRow) + minStartRow = firstCtx->matchStartRow; + } + if (minStartRow != INT64_MAX && pos < minStartRow) + { + register_reduced_frame_map(winstate, pos, RF_UNMATCHED); + return; + } + } /* - * Check if the pattern does not include any greedy quantifier. If it does - * not, we can just apply the pattern to each row. If it succeeds, we are - * done. + * Case 2: Find existing context for this pos, or create new one. */ - foreach(lc1, winstate->patternRegexpList) + targetCtx = nfa_find_context_for_pos(winstate, pos); + if (targetCtx == NULL) { - char *quantifier = strVal(lfirst(lc1)); - - if (*quantifier == '+' || *quantifier == '*' || *quantifier == '?') - { - greedy = true; - break; - } + /* No context exists - create one and start fresh */ + nfa_start_context(winstate, pos); + targetCtx = winstate->nfaContext; } /* - * Non greedy case + * Determine where to start processing. + * If we've already evaluated rows beyond pos, continue from there. + */ + startPos = Max(pos, winstate->nfaLastProcessedRow + 1); + + /* + * Process rows until target context completes or we hit boundaries. + * Each row evaluation is shared across all active contexts. */ - if (!greedy) + for (currentPos = startPos; targetCtx->states != NULL; currentPos++) { - num_matched_rows = 0; - encoded_str = makeStringInfoExt(init_size); + bool rowExists; + bool anyMatch; + RPRNFAContext *ctx; - foreach(lc1, winstate->patternVariableList) - { - char *vname = strVal(lfirst(lc1)); -#ifdef RPR_DEBUG - printf("pos:" INT64_FORMAT " pattern vname: %s\n", - pos, vname); -#endif - expression_result = false; + /* Evaluate variables for this row - done only once, shared by all contexts */ + rowExists = nfa_evaluate_row(winobj, currentPos, winstate->nfaVarMatched, &anyMatch); - /* evaluate row pattern against current row */ - result_pos = evaluate_pattern(winobj, pos, vname, - encoded_str, &expression_result); - if (!expression_result || result_pos < 0) + /* No more rows in partition? Finalize all contexts */ + if (!rowExists) + { + for (ctx = winstate->nfaContext; ctx != NULL; ctx = ctx->next) { -#ifdef RPR_DEBUG - printf("expression result is false or out of frame\n"); -#endif - register_reduced_frame_map(winstate, original_pos, - RF_UNMATCHED); - destroyStringInfo(encoded_str); - return; + if (ctx->states != NULL) + nfa_finalize_boundary(winstate, ctx, currentPos - 1); } - /* move to next row */ - pos++; - num_matched_rows++; - } - destroyStringInfo(encoded_str); -#ifdef RPR_DEBUG - printf("pattern matched\n"); -#endif - register_reduced_frame_map(winstate, original_pos, RF_FRAME_HEAD); - - for (i = original_pos + 1; i < original_pos + num_matched_rows; i++) - { - register_reduced_frame_map(winstate, i, RF_SKIPPED); + break; } - return; - } - /* - * Greedy quantifiers included. Loop over until none of pattern matches or - * encounters end of frame. - */ - for (;;) - { - result_pos = -1; + /* Update last processed row */ + winstate->nfaLastProcessedRow = currentPos; /* - * Loop over each PATTERN variable. + * Process each active context with this row's evaluation results. + * Each context has its own frame boundary based on matchStartRow. */ - anymatch = false; - - encoded_str = makeStringInfoExt(init_size); - - forboth(lc1, winstate->patternVariableList, lc2, - winstate->patternRegexpList) + for (ctx = winstate->nfaContext; ctx != NULL; ctx = ctx->next) { - char *vname = strVal(lfirst(lc1)); -#ifdef RPR_DEBUG - char *quantifier = strVal(lfirst(lc2)); - - printf("pos: " INT64_FORMAT " pattern vname: %s quantifier: %s\n", - pos, vname, quantifier); -#endif - expression_result = false; + int64 ctxFrameEnd; - /* evaluate row pattern against current row */ - result_pos = evaluate_pattern(winobj, pos, vname, - encoded_str, &expression_result); - if (expression_result) - { -#ifdef RPR_DEBUG - printf("expression result is true\n"); -#endif - anymatch = true; - } + /* Skip already-completed contexts */ + if (ctx->states == NULL) + continue; /* - * If out of frame, we are done. + * Calculate per-context frame end. + * For "ROWS BETWEEN CURRENT ROW AND N FOLLOWING", each context's + * frame end is matchStartRow + offset + 1 (exclusive). */ - if (result_pos < 0) - break; - } + if (hasLimitedFrame) + { + ctxFrameEnd = ctx->matchStartRow + frameOffset + 1; + if (currentPos >= ctxFrameEnd) + { + nfa_finalize_boundary(winstate, ctx, ctxFrameEnd - 1); + continue; + } + } - if (!anymatch) - { - /* none of patterns matched. */ - break; - } + /* First row of this context must match at least one variable */ + if (currentPos == ctx->matchStartRow && !anyMatch) + { + /* Clear states to mark as unmatched */ + nfa_state_free_list(winstate, ctx->states); + ctx->states = NULL; + continue; + } - string_set_add(str_set, encoded_str, 0); + /* Skip if this row is before context's start */ + if (currentPos < ctx->matchStartRow) + continue; -#ifdef RPR_DEBUG - printf("pos: " INT64_FORMAT " encoded_str: %s\n", - pos, encoded_str->data); -#endif + /* Process states for this context */ + { + RPRNFAState *states = ctx->states; + RPRNFAState *state; + RPRNFAState *nextState; - /* move to next row */ - pos++; + ctx->states = NULL; - if (result_pos < 0) - { - /* out of frame */ - break; + for (state = states; state != NULL; state = nextState) + { + nextState = state->next; + state->next = NULL; + nfa_step_single(winstate, ctx, state, winstate->nfaVarMatched, currentPos); + } + } } - } - if (string_set_get_size(str_set) == 0) - { - /* no match found in the first row */ - register_reduced_frame_map(winstate, original_pos, RF_UNMATCHED); - destroyStringInfo(encoded_str); - return; - } + /* + * Create a new context for the next potential start position. + * This enables overlapping match detection for SKIP TO NEXT ROW. + */ + if (anyMatch) + nfa_start_context(winstate, currentPos + 1); -#ifdef RPR_DEBUG - printf("pos: " INT64_FORMAT " encoded_str: %s\n", - pos, encoded_str->data); -#endif + /* + * Absorb redundant contexts. + * At the same elementIndex, if newer context's count <= older context's count, + * the newer context can be absorbed (for unbounded quantifiers). + * Note: Never absorb targetCtx - it's the context we're trying to complete. + */ + nfa_absorb_contexts(winstate, targetCtx, currentPos); - /* look for matching pattern variable sequence */ -#ifdef RPR_DEBUG - printf("search_str_set started\n"); -#endif - num_matched_rows = search_str_set(winstate, str_set); + /* Check if target context is now complete */ + if (targetCtx->states == NULL) + break; + } -#ifdef RPR_DEBUG - printf("search_str_set returns: %d\n", num_matched_rows); -#endif - string_set_discard(str_set); + /* + * Get match result from target context. + */ + if (targetCtx->matchEndRow >= pos) + matchLength = targetCtx->matchEndRow - pos + 1; /* - * We are at the first row in the reduced frame. Save the number of - * matched rows as the number of rows in the reduced frame. + * Register reduced frame map based on match result. */ - if (num_matched_rows <= 0) + if (matchLength <= 0) { - /* no match */ - register_reduced_frame_map(winstate, original_pos, RF_UNMATCHED); + register_reduced_frame_map(winstate, pos, RF_UNMATCHED); } else { - register_reduced_frame_map(winstate, original_pos, RF_FRAME_HEAD); - - for (i = original_pos + 1; i < original_pos + num_matched_rows; i++) + register_reduced_frame_map(winstate, pos, RF_FRAME_HEAD); + for (int64 i = pos + 1; i < pos + matchLength; i++) { register_reduced_frame_map(winstate, i, RF_SKIPPED); } } - return; -} - -/* - * search_str_set - * - * Perform pattern matching using "pattern" against input_str_set. pattern is - * a regular expression string derived from PATTERN clause. Note that the - * regular expression string is prefixed by '^' and followed by initials - * represented in a same way as str_set. str_set is a set of StringInfo. Each - * StringInfo has a string comprising initials of pattern variable strings - * being true in a row. The initials are one of [a-z], parallel to the order - * of variable names in DEFINE clause. Suppose DEFINE has variables START, UP - * and DOWN. If PATTERN has START, UP+ and DOWN, then the initials in PATTERN - * will be 'a', 'b' and 'c'. The "pattern" will be "^ab+c". - * - * variable_pos is an array representing the order of pattern variable string - * initials in PATTERN clause. For example initial 'a' potion is in - * variable_pos[0].pos[0] = 0. Note that if the pattern is "START UP DOWN UP" - * (UP appears twice), then "UP" (initial is 'b') has two position 1 and - * 3. Thus variable_pos for b is variable_pos[1].pos[0] = 1 and - * variable_pos[1].pos[1] = 3. - * - * Returns the longest number of the matching rows (greedy matching) if - * quatifier '+' or '*' is included in "pattern". - */ -static int -search_str_set(WindowAggState *winstate, StringSet *input_str_set) -{ - char *pattern; /* search regexp pattern */ - VariablePos *variable_pos; - int set_size; /* number of rows in the set */ - int resultlen; - int index; - StringSet *new_str_set; - int new_str_size; - int len; - int info; - char tail_pattern_initial; - /* - * Set last initial char to tail_pattern_initial if we can apply "tail - * pattern initial optimization". If the last regexp component in pattern - * is with '+' quatifier, set the initial to tail_pattern_initial. For - * example if pattern = "ab+", tail_pattern_initial will be 'b'. - * Otherwise, tail_pattern_initial is '\0'. + * Cleanup contexts based on SKIP mode. */ - pattern = winstate->pattern_str->data; - if (pattern[strlen(pattern) - 1] == '+') - tail_pattern_initial = pattern[strlen(pattern) - 2]; + if (matchLength > 0 && winstate->rpSkipTo == ST_PAST_LAST_ROW) + { + /* Remove all contexts with start <= matchEnd */ + nfa_remove_contexts_up_to(winstate, pos + matchLength - 1); + } else - tail_pattern_initial = '\0'; - - /* - * Generate all possible pattern variable name initials as a set of - * StringInfo named "new_str_set". For example, if we have two rows - * having "ab" (row 0) and "ac" (row 1) in the input str_set, new_str_set - * will have set of StringInfo "aa", "ac", "ba" and "bc" in the end. - */ - variable_pos = winstate->variable_pos; - new_str_set = generate_patterns(winstate, input_str_set, pattern, variable_pos, - tail_pattern_initial); - - /* - * Perform pattern matching to find out the longest match. - */ - new_str_size = string_set_get_size(new_str_set); - len = 0; - resultlen = 0; - set_size = string_set_get_size(input_str_set); - - for (index = 0; index < new_str_size; index++) { - StringInfo s; - - s = string_set_get(new_str_set, index, &info); - if (s == NULL) - continue; /* no data */ - - /* - * If the string is scheduled to be discarded, we just disregard it. - */ - if (info & STRSET_DISCARDED) - continue; + /* SKIP TO NEXT ROW or no match: just remove the target context */ + RPRNFAContext *ctx = winstate->nfaContext; - len = do_pattern_match(winstate, pattern, s->data, s->len); - if (len > resultlen) + while (ctx != NULL) { - /* remember the longest match */ - resultlen = len; - - /* - * If the size of result set is equal to the number of rows in the - * set, we are done because it's not possible that the number of - * matching rows exceeds the number of rows in the set. - */ - if (resultlen >= set_size) + if (ctx == targetCtx) + { + nfa_unlink_context(winstate, ctx); + nfa_context_free(winstate, ctx); break; + } + ctx = ctx->next; } } - - /* we no longer need new string set */ - string_set_discard(new_str_set); - - return resultlen; } /* - * generate_patterns + * NFA-based pattern matching implementation * - * Generate all possible pattern variable name initials in 'input_str_set' as - * a set of StringInfo and return it. For example, if we have two rows having - * "ab" (row 0) and "ac" (row 1) in 'input str_set', returned StringSet will - * have set of StringInfo "aa", "ac", "ba" and "bc" in the end. - * 'variable_pos' and 'tail_pattern_initial' are used for pruning - * optimization. + * These functions implement direct NFA execution using the compiled + * RPRPattern structure, avoiding regex compilation overhead. */ -static StringSet * -generate_patterns(WindowAggState *winstate, StringSet *input_str_set, - char *pattern, VariablePos *variable_pos, - char tail_pattern_initial) -{ - StringSet *old_str_set, - *new_str_set; - int index; - int set_size; - int old_set_size; - int info; - int resultlen; - StringInfo str; - int i; - char *p; - - new_str_set = string_set_init(); - set_size = string_set_get_size(input_str_set); - if (set_size == 0) /* if there's no row in input, return empty - * set */ - return new_str_set; - - resultlen = 0; - /* - * Generate initial new_string_set for input row 0. - */ - str = string_set_get(input_str_set, 0, &info); - p = str->data; +/* + * nfa_state_alloc + * + * Allocate an NFA state, reusing from freeList if available. + * freeList is stored in WindowAggState for reuse across match attempts. + * Uses flexible array member for counts[]. + */ +static RPRNFAState * +nfa_state_alloc(WindowAggState *winstate) +{ + RPRNFAState *state; + int maxDepth = winstate->rpPattern->maxDepth; - /* - * Loop over each new pattern variable char. - */ - while (*p) + /* Try to reuse from free list first */ + if (winstate->nfaStateFree != NULL) { - StringInfo new = makeStringInfo(); - - /* add pattern variable char */ - appendStringInfoChar(new, *p); - /* add new one to string set */ - string_set_add(new_str_set, new, 0); - p++; /* next pattern variable */ + state = winstate->nfaStateFree; + winstate->nfaStateFree = state->next; } - - /* - * Generate new_string_set for each input row. - */ - for (index = 1; index < set_size; index++) + else { - /* previous new str set now becomes old str set */ - old_str_set = new_str_set; - new_str_set = string_set_init(); /* create new string set */ - /* pick up input string */ - str = string_set_get(input_str_set, index, &info); - old_set_size = string_set_get_size(old_str_set); - - /* - * Loop over each row in the previous result set. - */ - for (i = 0; i < old_set_size; i++) - { - char last_old_char; - int old_str_len; - int old_info; - StringInfo old; - - old = string_set_get(old_str_set, i, &old_info); - p = old->data; - old_str_len = old->len; - if (old_str_len > 0) - last_old_char = p[old_str_len - 1]; - else - last_old_char = '\0'; - - /* Can this old set be discarded? */ - if (old_info & STRSET_DISCARDED) - continue; /* discard the old string */ + /* Allocate in partition context for proper lifetime */ + MemoryContext oldContext = MemoryContextSwitchTo(winstate->partcontext); + state = palloc(winstate->nfaStateSize); + MemoryContextSwitchTo(oldContext); + } - /* Is this old set frozen? */ - else if (old_info & STRSET_FROZEN) - { - /* if shorter match. we can discard it */ - if (old_str_len < resultlen) - continue; /* discard the shorter string */ + /* initialize state - clear all depth counts */ + state->next = NULL; + state->elemIdx = 0; + state->altPriority = 0; + /* Initialize all depth counts to 0 using memset */ + if (maxDepth > 0) + memset(state->counts, 0, sizeof(int16) * maxDepth); - /* move the old set to new_str_set */ - string_set_add(new_str_set, old, old_info); - old_str_set->str_set[i] = NULL; - continue; - } + return state; +} - /* - * loop over each pattern variable initial char in the input set. - */ - for (p = str->data; *p; p++) - { - /* - * Optimization. Check if the row's pattern variable initial - * character position is greater than or equal to the old - * set's last pattern variable initial character position. For - * example, if the old set's last pattern variable initials - * are "ab", then the new pattern variable initial can be "b" - * or "c" but can not be "a", if the initials in PATTERN is - * something like "a b c" or "a b+ c+" etc. This optimization - * is possible when we only allow "+" quantifier. - */ - if (variable_pos_compare(variable_pos, last_old_char, *p)) +/* + * nfa_state_free + * + * Return a state to the free list for later reuse. + */ +static void +nfa_state_free(WindowAggState *winstate, RPRNFAState *state) +{ + state->next = winstate->nfaStateFree; + winstate->nfaStateFree = state; +} - /* - * Satisfied the condition. Add new pattern char to - * new_str_set if it looks good. - */ - resultlen = add_pattern(winstate, old, old_info, new_str_set, *p, - pattern, tail_pattern_initial, resultlen); - else +/* + * nfa_state_free_list + * + * Free all states in a list using pfree. + */ +static void +nfa_state_free_list(WindowAggState *winstate, RPRNFAState *list) +{ + RPRNFAState *state; - /* - * The old_str did not satisfy the condition and it cannot - * be extended further. "Freeze" it. - */ - resultlen = freeze_pattern(winstate, old, old_info, - new_str_set, pattern, resultlen); - } - } - /* we no longer need old string set */ - string_set_discard(old_str_set); - } - return new_str_set; + while(list != NULL) + { + state = list; + list = list->next; + nfa_state_free(winstate, state); + } } /* - * add_pattern + * nfa_states_equal * - * Make a copy of 'old' (along with 'old_info' flag) and add new pattern char - * 'c' to it. Then add it to 'new_str_set'. 'pattern' and - * 'tail_pattern_initial' is checked to determine whether the copy is worth to - * add to new_str_set or not. The match length (possibly longer than - * 'resultlen') is returned. + * Check if two states are equivalent (same elemIdx and counts). */ -static int -add_pattern(WindowAggState *winstate, StringInfo old, int old_info, - StringSet *new_str_set, char c, char *pattern, char tail_pattern_initial, - int resultlen) +static bool +nfa_states_equal(WindowAggState *winstate, RPRNFAState *s1, RPRNFAState *s2) { - StringInfo new; - int info; - int len; + int maxDepth = winstate->rpPattern->maxDepth; - /* - * New char in the input row satisfies the condition above. - */ - new = makeStringInfoExt(old->len + 1); /* copy source string */ - appendStringInfoString(new, old->data); + if (s1->elemIdx != s2->elemIdx) + return false; - /* add pattern variable char */ - appendStringInfoChar(new, c); + if (maxDepth > 0 && + memcmp(s1->counts, s2->counts, sizeof(int16) * maxDepth) != 0) + return false; - /* - * Adhoc optimization. If the first letter in the input string is in the - * head and second position and there's no associated quatifier '+', then - * we can dicard the input because there's no chance to expand the string - * further. - * - * For example, pattern "abc" cannot match "aa". - */ - if (pattern[1] == new->data[0] && - pattern[1] == new->data[1] && - pattern[2] != '+' && - pattern[1] != pattern[2]) - { - destroyStringInfo(new); - return resultlen; - } + return true; +} - info = old_info; +/* + * nfa_add_state_unique + * + * Add a state to ctx->states at the END, only if no duplicate exists. + * Returns true if state was added, false if duplicate found (state is freed). + */ +static bool +nfa_add_state_unique(WindowAggState *winstate, RPRNFAContext *ctx, RPRNFAState *state) +{ + RPRNFAState *s; + RPRNFAState *prev = NULL; + RPRNFAState *tail = NULL; - /* - * Check if we can apply "tail pattern initial optimization". If the last - * regexp component in pattern has '+' quantifier, the component is set to - * the last pattern initial. For example if pattern is "ab+", - * tail_pattern_initial will become 'b'. Otherwise, tail_pattern_initial - * is '\0'. If the tail pattern initial optimization is possible, we do - * not need to apply regular expression match again. Suppose we have the - * previous string ended with "b" and the it was confirmed the regular - * expression match, then char 'b' can be added to the string without - * applying the regular expression match again. - */ - if (c == tail_pattern_initial) /* tail pattern initial optimization - * possible? */ + /* Check for duplicate and find tail */ + for (s = ctx->states; s != NULL; s = s->next) { - /* - * Is already confirmed to be matched with pattern? - */ - if ((info & STRSET_MATCHED) == 0) + if (nfa_states_equal(winstate, s, state)) { - /* not confirmed yet */ - len = do_pattern_match(winstate, pattern, new->data, new->len); - if (len > 0) - info = STRSET_MATCHED; /* set already confirmed flag */ - } - else - /* - * already confirmed. Use the string length as the matching length + * Duplicate found - keep lower altPriority for lexical order. + * Lower altPriority means earlier alternative in pattern. */ - len = new->len; - - /* update the longest match length if needed */ - if (len > resultlen) - resultlen = len; + if (state->altPriority < s->altPriority) + { + /* New state has better priority, replace existing */ + state->next = s->next; + if (prev == NULL) + ctx->states = state; + else + prev->next = state; + nfa_state_free(winstate, s); + return true; + } + /* Existing state has better/equal priority, discard new */ + nfa_state_free(winstate, state); + return false; + } + prev = s; + tail = s; } - /* add new StringInfo to the string set */ - string_set_add(new_str_set, new, info); + /* No duplicate, add at end */ + state->next = NULL; + if (tail == NULL) + ctx->states = state; + else + tail->next = state; - return resultlen; + return true; } /* - * freeze_pattern + * nfa_state_clone * - * "Freeze" 'old' (along with 'old_info' flag) and add it to - * 'new_str_set'. Frozen string is known to not be expanded further. The frozen - * string is check if it satisfies 'pattern'. If it does not, "discarded" - * mark is added. The discarded mark is also added if the match length is - * shorter than the current longest match length. The match length (possibly - * longer than 'resultlen') is returned. + * Clone a state with given elemIdx, altPriority and counts. + * Only copies counts up to elem->depth (not entire maxDepth). + * Prepends to the provided list and returns the new list head. */ -static int -freeze_pattern(WindowAggState *winstate, StringInfo old, int old_info, - StringSet *new_str_set, - char *pattern, int resultlen) +static RPRNFAState * +nfa_state_clone(WindowAggState *winstate, int16 elemIdx, int16 altPriority, + int16 *counts, RPRNFAState *list) { - int len; - StringInfo new; - int new_str_size; - int new_index; + RPRPattern *pattern = winstate->rpPattern; + int maxDepth = pattern->maxDepth; + RPRNFAState *state = nfa_state_alloc(winstate); + + state->elemIdx = elemIdx; + state->altPriority = altPriority; + /* nfa_state_alloc already zeroed all counts, now copy all depth levels */ + if (counts != NULL && maxDepth > 0) + memcpy(state->counts, counts, sizeof(int16) * maxDepth); + state->next = list; + + return state; +} - /* - * We are freezing this pattern string. If the pattern string length is - * shorter than the current longest string length, we don't need to keep - * it. - */ - if (old->len < resultlen) - return resultlen; +/* + * nfa_add_matched_state + * + * Record a matched state following SQL standard lexical order preference. + * Priority: lower altPriority wins (lexical order), then longer match. + */ +static void +nfa_add_matched_state(WindowAggState *winstate, RPRNFAContext *ctx, + RPRNFAState *state, int64 matchEndRow) +{ + bool shouldUpdate = false; - if (old_info & STRSET_MATCHED) - /* we don't need to apply pattern match again */ - len = old->len; - else + if (ctx->matchedState == NULL) { - /* apply pattern match */ - len = do_pattern_match(winstate, pattern, old->data, old->len); - if (len <= 0) - { - /* no match. we can discard it */ - return resultlen; - } + /* No previous match, always save */ + shouldUpdate = true; } - if (len < resultlen) + else if (state->altPriority < ctx->matchedState->altPriority) { - /* shorter match. we can discard it */ - return resultlen; + /* New state has better lexical order priority */ + shouldUpdate = true; } - - /* - * Match length is the longest so far - */ - resultlen = len; /* remember the longest match */ - - /* freeze the pattern string */ - new = makeStringInfo(); - enlargeStringInfo(new, old->len + 1); - appendStringInfoString(new, old->data); - /* set frozen mark */ - string_set_add(new_str_set, new, STRSET_FROZEN); - - /* - * Search new_str_set to find out frozen entries that have shorter match - * length. Mark them as "discard" so that they are discarded in the next - * round. - */ - new_str_size = - string_set_get_size(new_str_set) - 1; - - /* loop over new_str_set */ - for (new_index = 0; new_index < new_str_size; new_index++) + else if (state->altPriority == ctx->matchedState->altPriority && + matchEndRow > ctx->matchEndRow) { - int info; - - new = string_set_get(new_str_set, new_index, &info); + /* Same priority, but longer match */ + shouldUpdate = true; + } - /* - * If this is frozen and is not longer than the current longest match - * length, we don't need to keep this. - */ - if (info & STRSET_FROZEN && new->len < resultlen) - { - /* - * mark this set to discard in the next round - */ - info |= STRSET_DISCARDED; - new_str_set->info[new_index] = info; - } + if (shouldUpdate) + { + /* Reuse existing matchedState or allocate from free list */ + if (ctx->matchedState == NULL) + ctx->matchedState = nfa_state_alloc(winstate); + + /* Copy state data */ + memcpy(ctx->matchedState, state, winstate->nfaStateSize); + ctx->matchedState->next = NULL; + ctx->matchEndRow = matchEndRow; } - return resultlen; } /* - * do_pattern_match + * nfa_free_matched_state * - * Perform pattern match using 'pattern' against 'encoded_str' whose length is - * 'len' bytes (without null terminate). Returns matching number of rows if - * matching is succeeded. Otherwise returns 0. + * Return matchedState to free list for reuse. */ -static int -do_pattern_match(WindowAggState *winstate, char *pattern, char *encoded_str, int len) +static void +nfa_free_matched_state(WindowAggState *winstate, RPRNFAState *state) { - int plen; - int cflags = REG_EXTENDED; - size_t nmatch = 1; - int eflags = 0; - regmatch_t pmatch[1]; - int sts; - pg_wchar *data; - int data_len; - - /* - * Compile regexp if cache does not exist or existing cache is not same as - * "pattern". - */ - if (winstate->patbuf == NULL || strcmp(winstate->patbuf, pattern)) - { - MemoryContext oldContext = MemoryContextSwitchTo - (winstate->ss.ps.ps_ExprContext->ecxt_per_query_memory); - - pattern_regfree(winstate); - - /* we need to convert to char to pg_wchar */ - plen = strlen(pattern); - data = (pg_wchar *) palloc((plen + 1) * sizeof(pg_wchar)); - data_len = pg_mb2wchar_with_len(pattern, data, plen); - /* compile re */ - sts = pg_regcomp(&winstate->preg, /* compiled re */ - data, /* target pattern */ - data_len, /* length of pattern */ - cflags, /* compile option */ - C_COLLATION_OID /* collation */ - ); - pfree(data); - - if (sts != REG_OKAY) - { - /* re didn't compile (no need for pg_regfree, if so) */ - ereport(ERROR, - (errcode(ERRCODE_INVALID_REGULAR_EXPRESSION), - errmsg("invalid regular expression: %s", pattern))); - } - - /* save pattern */ - winstate->patbuf = pstrdup(pattern); - MemoryContextSwitchTo(oldContext); - } - - data = (pg_wchar *) palloc((len + 1) * sizeof(pg_wchar)); - data_len = pg_mb2wchar_with_len(encoded_str, data, len); - - /* execute the regular expression match */ - sts = pg_regexec( - &winstate->preg, /* compiled re */ - data, /* target string */ - data_len, /* length of encoded_str */ - 0, /* search start */ - NULL, /* rm details */ - nmatch, /* number of match sub re */ - pmatch, /* match result details */ - eflags); - - pfree(data); - - if (sts != REG_OKAY) - { - if (sts != REG_NOMATCH) - { - char errMsg[100]; - - pg_regerror(sts, &winstate->preg, errMsg, sizeof(errMsg)); - ereport(ERROR, - (errcode(ERRCODE_INVALID_REGULAR_EXPRESSION), - errmsg("regular expression failed: %s", errMsg))); - } - return 0; /* does not match */ - } - - len = pmatch[0].rm_eo; /* return match length */ - return len; -} + if (state != NULL) + nfa_state_free(winstate, state); +} /* - * pattern_regfree + * nfa_evaluate_row * - * Free the regcomp result for PARTTERN string. + * Evaluate all DEFINE variables for current row. + * Returns true if the row exists, false if out of partition. + * If row exists, fills varMatched array and sets *anyMatch if any variable matched. + * varMatched[i] = true if variable i matched at current row. */ -static void -pattern_regfree(WindowAggState *winstate) -{ - if (winstate->patbuf != NULL) - { - pg_regfree(&winstate->preg); /* free previous re */ - pfree(winstate->patbuf); - winstate->patbuf = NULL; - } -} - -/* - * evaluate_pattern - * - * Evaluate expression associated with PATTERN variable vname. current_pos is - * relative row position in a frame (starting from 0). If vname is evaluated - * to true, initial letters associated with vname is appended to - * encode_str. result is out paramater representing the expression evaluation - * result is true of false. - *--------- - * Return values are: - * >=0: the last match absolute row position - * otherwise out of frame. - *--------- - */ -static int64 -evaluate_pattern(WindowObject winobj, int64 current_pos, - char *vname, StringInfo encoded_str, bool *result) +static bool +nfa_evaluate_row(WindowObject winobj, int64 pos, bool *varMatched, bool *anyMatchOut) { WindowAggState *winstate = winobj->winstate; ExprContext *econtext = winstate->ss.ps.ps_ExprContext; - ListCell *lc1, - *lc2, - *lc3; - ExprState *pat; - Datum eval_result; - bool out_of_frame = false; - bool isnull; + int numDefineVars = list_length(winstate->defineVariableList); + ListCell *lc; + int varIdx = 0; + bool anyMatch = false; TupleTableSlot *slot; - forthree(lc1, winstate->defineVariableList, - lc2, winstate->defineClauseList, - lc3, winstate->defineInitial) - { - char initial; /* initial letter associated with vname */ - char *name = strVal(lfirst(lc1)); - - if (strcmp(vname, name)) - continue; - - initial = *(strVal(lfirst(lc3))); - - /* set expression to evaluate */ - pat = lfirst(lc2); - - /* get current, previous and next tuples */ - if (!get_slots(winobj, current_pos)) - { - out_of_frame = true; - } - else - { - /* evaluate the expression */ - eval_result = ExecEvalExpr(pat, econtext, &isnull); - if (isnull) - { - /* expression is NULL */ -#ifdef RPR_DEBUG - printf("expression for %s is NULL at row: " INT64_FORMAT "\n", - vname, current_pos); -#endif - *result = false; - } - else - { - if (!DatumGetBool(eval_result)) - { - /* expression is false */ -#ifdef RPR_DEBUG - printf("expression for %s is false at row: " INT64_FORMAT "\n", - vname, current_pos); -#endif - *result = false; - } - else - { - /* expression is true */ -#ifdef RPR_DEBUG - printf("expression for %s is true at row: " INT64_FORMAT "\n", - vname, current_pos); -#endif - appendStringInfoChar(encoded_str, initial); - *result = true; - } - } - - slot = winstate->temp_slot_1; - if (slot != winstate->null_slot) - ExecClearTuple(slot); - slot = winstate->prev_slot; - if (slot != winstate->null_slot) - ExecClearTuple(slot); - slot = winstate->next_slot; - if (slot != winstate->null_slot) - ExecClearTuple(slot); - - break; - } - - if (out_of_frame) - { - *result = false; - return -1; - } - } - return current_pos; -} + *anyMatchOut = false; -/* - * get_slots - * - * Get current, previous and next tuples. - * Returns false if current row is out of partition/full frame. - */ -static bool -get_slots(WindowObject winobj, int64 current_pos) -{ - WindowAggState *winstate = winobj->winstate; - TupleTableSlot *slot; - int ret; - ExprContext *econtext; - - econtext = winstate->ss.ps.ps_ExprContext; + /* + * Set up slots for current, previous, and next rows. + * We don't call get_slots() here to avoid recursion through + * row_is_in_frame -> update_reduced_frame -> nfa_match_pattern. + */ - /* set up current row tuple slot */ + /* Current row -> ecxt_outertuple */ slot = winstate->temp_slot_1; - if (!window_gettupleslot(winobj, current_pos, slot)) - { -#ifdef RPR_DEBUG - printf("current row is out of partition at:" INT64_FORMAT "\n", - current_pos); -#endif - return false; - } - ret = row_is_in_frame(winobj, current_pos, slot, false); - if (ret <= 0) - { -#ifdef RPR_DEBUG - printf("current row is out of frame at: " INT64_FORMAT "\n", - current_pos); -#endif - ExecClearTuple(slot); - return false; - } + if (!window_gettupleslot(winobj, pos, slot)) + return false; /* No row exists */ econtext->ecxt_outertuple = slot; - /* for PREV */ - if (current_pos > 0) + /* Previous row -> ecxt_scantuple (for PREV) */ + if (pos > 0) { slot = winstate->prev_slot; - if (!window_gettupleslot(winobj, current_pos - 1, slot)) - { -#ifdef RPR_DEBUG - elog(DEBUG1, "previous row is out of partition at: " INT64_FORMAT, - current_pos - 1); -#endif + if (!window_gettupleslot(winobj, pos - 1, slot)) econtext->ecxt_scantuple = winstate->null_slot; - } else - { - ret = row_is_in_frame(winobj, current_pos - 1, slot, false); - if (ret <= 0) - { -#ifdef RPR_DEBUG - elog(DEBUG1, "previous row is out of frame at: " INT64_FORMAT, - current_pos - 1); -#endif - ExecClearTuple(slot); - econtext->ecxt_scantuple = winstate->null_slot; - } - else - { - econtext->ecxt_scantuple = slot; - } - } + econtext->ecxt_scantuple = slot; } else econtext->ecxt_scantuple = winstate->null_slot; - /* for NEXT */ + /* Next row -> ecxt_innertuple (for NEXT) */ slot = winstate->next_slot; - if (!window_gettupleslot(winobj, current_pos + 1, slot)) - { -#ifdef RPR_DEBUG - printf("next row is out of partiton at: " INT64_FORMAT "\n", - current_pos + 1); -#endif + if (!window_gettupleslot(winobj, pos + 1, slot)) econtext->ecxt_innertuple = winstate->null_slot; - } else + econtext->ecxt_innertuple = slot; + + foreach(lc, winstate->defineClauseList) { - ret = row_is_in_frame(winobj, current_pos + 1, slot, false); - if (ret <= 0) + ExprState *exprState = (ExprState *) lfirst(lc); + Datum result; + bool isnull; + + /* Evaluate DEFINE expression */ + result = ExecEvalExpr(exprState, econtext, &isnull); + + if (!isnull && DatumGetBool(result)) { -#ifdef RPR_DEBUG - printf("next row is out of frame at: " INT64_FORMAT "\n", - current_pos + 1); -#endif - ExecClearTuple(slot); - econtext->ecxt_innertuple = winstate->null_slot; + varMatched[varIdx] = true; + anyMatch = true; } else - econtext->ecxt_innertuple = slot; + { + varMatched[varIdx] = false; + } + + varIdx++; + if (varIdx >= numDefineVars) + break; } - return true; + + *anyMatchOut = anyMatch; + return true; /* Row exists */ } /* - * pattern_initial + * nfa_context_alloc * - * Return pattern variable initial character - * matching with pattern variable name vname. - * If not found, return 0. + * Allocate an NFA context from free list or palloc. */ -static char -pattern_initial(WindowAggState *winstate, char *vname) +static RPRNFAContext * +nfa_context_alloc(WindowAggState *winstate) { - char initial; - char *name; - ListCell *lc1, - *lc2; + RPRNFAContext *ctx; - forboth(lc1, winstate->defineVariableList, - lc2, winstate->defineInitial) + if (winstate->nfaContextFree != NULL) { - name = strVal(lfirst(lc1)); /* DEFINE variable name */ - initial = *(strVal(lfirst(lc2))); /* DEFINE variable initial */ + ctx = winstate->nfaContextFree; + winstate->nfaContextFree = ctx->next; + } + else + { + /* Allocate in partition context for proper lifetime */ + MemoryContext oldContext = MemoryContextSwitchTo(winstate->partcontext); + ctx = palloc(sizeof(RPRNFAContext)); + MemoryContextSwitchTo(oldContext); + } + ctx->next = NULL; + ctx->prev = NULL; + ctx->states = NULL; + ctx->matchStartRow = -1; + ctx->matchEndRow = -1; + ctx->matchedState = NULL; - if (!strcmp(name, vname)) - return initial; /* found */ - } - return 0; + return ctx; } /* - * string_set_init + * nfa_unlink_context * - * Create dynamic set of StringInfo. + * Remove a context from the doubly-linked active context list. + * Updates head (nfaContext) and tail (nfaContextTail) as needed. */ -static StringSet * -string_set_init(void) +static void +nfa_unlink_context(WindowAggState *winstate, RPRNFAContext *ctx) { -/* Initial allocation size of str_set */ -#define STRING_SET_ALLOC_SIZE 1024 - - StringSet *string_set; - Size set_size; + if (ctx->prev != NULL) + ctx->prev->next = ctx->next; + else + winstate->nfaContext = ctx->next; /* was head */ - string_set = palloc0(sizeof(StringSet)); - string_set->set_index = 0; - set_size = STRING_SET_ALLOC_SIZE; - string_set->str_set = palloc(set_size * sizeof(StringInfo)); - string_set->info = palloc0(set_size * sizeof(int)); - string_set->set_size = set_size; + if (ctx->next != NULL) + ctx->next->prev = ctx->prev; + else + winstate->nfaContextTail = ctx->prev; /* was tail */ - return string_set; + ctx->next = NULL; + ctx->prev = NULL; } /* - * string_set_add + * nfa_context_free * - * Add StringInfo str to StringSet string_set. + * Return a context to free list. Also frees any states in the context. + * Note: Caller must unlink context from active list first using nfa_unlink_context. */ static void -string_set_add(StringSet *string_set, StringInfo str, int info) +nfa_context_free(WindowAggState *winstate, RPRNFAContext *ctx) { - Size set_size; - Size old_set_size; - - set_size = string_set->set_size; - if (string_set->set_index >= set_size) - { - old_set_size = set_size; - set_size *= 2; - string_set->str_set = repalloc(string_set->str_set, - set_size * sizeof(StringInfo)); - string_set->info = repalloc0(string_set->info, - old_set_size * sizeof(int), - set_size * sizeof(int)); - string_set->set_size = set_size; - } - - string_set->info[string_set->set_index] = info; - string_set->str_set[string_set->set_index++] = str; - - return; + if (ctx->states != NULL) + nfa_state_free_list(winstate, ctx->states); + if (ctx->matchedState != NULL) + nfa_free_matched_state(winstate, ctx->matchedState); + + ctx->states = NULL; + ctx->matchedState = NULL; + ctx->next = winstate->nfaContextFree; + winstate->nfaContextFree = ctx; } /* - * string_set_get + * nfa_start_context * - * Returns StringInfo specified by index. - * If there's no data yet, returns NULL. + * Start a new match context at given position. + * Adds context to winstate->nfaContext list. */ -static StringInfo -string_set_get(StringSet *string_set, int index, int *info) +static void +nfa_start_context(WindowAggState *winstate, int64 startPos) { - *info = 0; - - /* no data? */ - if (index == 0 && string_set->set_index == 0) - return NULL; - - if (index < 0 || index >= string_set->set_index) - elog(ERROR, "invalid index: %d", index); + RPRNFAContext *ctx; - *info = string_set->info[index]; + ctx = nfa_context_alloc(winstate); + ctx->matchStartRow = startPos; + ctx->states = nfa_state_alloc(winstate); /* initial state at elem 0 */ - return string_set->str_set[index]; + /* Add to head of active context list (doubly-linked) */ + ctx->next = winstate->nfaContext; + ctx->prev = NULL; + if (winstate->nfaContext != NULL) + winstate->nfaContext->prev = ctx; + else + winstate->nfaContextTail = ctx; /* first context becomes tail */ + winstate->nfaContext = ctx; } /* - * string_set_get_size + * nfa_find_context_for_pos * - * Returns the size of StringSet. - */ -static int -string_set_get_size(StringSet *string_set) -{ - return string_set->set_index; -} - -/* - * string_set_discard - * Discard StringSet. - * All memory including StringSet itself is freed. + * Find a context with the given start position. + * Returns NULL if not found. */ -static void -string_set_discard(StringSet *string_set) +static RPRNFAContext * +nfa_find_context_for_pos(WindowAggState *winstate, int64 pos) { - int i; + RPRNFAContext *ctx; - for (i = 0; i < string_set->set_index; i++) + for (ctx = winstate->nfaContext; ctx != NULL; ctx = ctx->next) { - StringInfo str = string_set->str_set[i]; - - if (str) - destroyStringInfo(str); + if (ctx->matchStartRow == pos) + return ctx; } - pfree(string_set->info); - pfree(string_set->str_set); - pfree(string_set); + return NULL; } /* - * variable_pos_init + * nfa_remove_contexts_up_to * - * Create and initialize variable postion structure + * Remove all contexts with matchStartRow <= endPos. + * Used by SKIP PAST LAST ROW to discard contexts within matched frame. */ -static VariablePos * -variable_pos_init(void) +static void +nfa_remove_contexts_up_to(WindowAggState *winstate, int64 endPos) { - VariablePos *variable_pos; + RPRNFAContext *ctx; + RPRNFAContext *next; - variable_pos = palloc(sizeof(VariablePos) * NUM_ALPHABETS); - MemSet(variable_pos, -1, sizeof(VariablePos) * NUM_ALPHABETS); - return variable_pos; + ctx = winstate->nfaContext; + while (ctx != NULL) + { + next = ctx->next; + if (ctx->matchStartRow <= endPos) + { + /* Remove this context */ + nfa_unlink_context(winstate, ctx); + nfa_context_free(winstate, ctx); + } + ctx = next; + } } /* - * variable_pos_register + * nfa_absorb_contexts + * + * Absorb newer contexts into older ones when states are fully covered. + * For pattern like A+, if older context (started earlier) has count >= newer + * context's count at the same element, the newer context produces subset matches. + * + * Absorption condition: + * - For unbounded quantifiers (max=INT32_MAX): older.counts >= newer.counts + * - For bounded quantifiers: older.counts == newer.counts * - * Register pattern variable whose initial is initial into postion index. - * pos is position of initial. - * If pos is already registered, register it at next empty slot. + * Note: List is newest-first, so we check if later nodes (older contexts) + * can absorb earlier nodes (newer contexts). */ static void -variable_pos_register(VariablePos *variable_pos, char initial, int pos) +nfa_absorb_contexts(WindowAggState *winstate, RPRNFAContext *excludeCtx, int64 currentPos) { - int index = initial - 'a'; - int slot; - int i; + RPRPattern *pattern = winstate->rpPattern; + RPRNFAContext *ctx; + int maxDepth; + + /* Need at least 2 contexts for absorption */ + if (winstate->nfaContext == NULL || winstate->nfaContext->next == NULL) + return; - if (pos < 0 || pos >= NUM_ALPHABETS) - elog(ERROR, "initial is not valid char: %c", initial); + if (pattern == NULL) + return; + + /* + * Only absorb for patterns marked as absorbable during planning. + * See computeAbsorbability() in rpr.c for criteria. + */ + if (!pattern->isAbsorbable) + return; + + maxDepth = pattern->maxDepth; + + /* + * Context absorption: A later context (started at higher row) can be + * absorbed by an earlier context if ALL states in the later context + * are "covered" by states in the earlier context. + * + * A later state is covered by an earlier state if: + * 1. They are at the same element + * 2. For unbounded elements (max == INT32_MAX): earlier.counts[d] >= later.counts[d] + * for all depths d + * 3. For bounded elements: counts must be exactly equal at all depths + * + * List is newest-first (head = highest matchStartRow). + * We iterate from head (newest) and check if older contexts can absorb it. + */ + ctx = winstate->nfaContext; - for (i = 0; i < NUM_ALPHABETS; i++) + while (ctx != NULL) { - slot = variable_pos[index].pos[i]; - if (slot < 0) + RPRNFAContext *nextCtx = ctx->next; + RPRNFAContext *older; + bool absorbed = false; + + /* Never absorb the excluded context (it's the target being completed) */ + if (ctx == excludeCtx) { - /* empty slot found */ - variable_pos[index].pos[i] = pos; - return; + ctx = nextCtx; + continue; } - } - elog(ERROR, "no empty slot for initial: %c", initial); -} -/* - * variable_pos_compare - * - * Returns true if initial1 can be followed by initial2 - */ -static bool -variable_pos_compare(VariablePos *variable_pos, char initial1, char initial2) -{ - int index1, - index2; - int pos1, - pos2; + /* Skip contexts that haven't started processing yet (just created for future row) */ + if (ctx->matchStartRow > currentPos) + { + ctx = nextCtx; + continue; + } - for (index1 = 0;; index1++) - { - pos1 = variable_pos_fetch(variable_pos, initial1, index1); - if (pos1 < 0) - break; + /* + * Handle completed contexts (states == NULL) separately. + * A completed context can be absorbed by an older completed context + * if the older one has the same or later matchEndRow. + */ + if (ctx->states == NULL) + { + /* Only completed contexts with valid matchEndRow can be absorbed */ + if (ctx->matchEndRow < 0) + { + ctx = nextCtx; + continue; + } + + /* Check if any older completed context can absorb this one */ + for (older = ctx->next; older != NULL && !absorbed; older = older->next) + { + /* Must have started earlier */ + if (older->matchStartRow >= ctx->matchStartRow) + continue; + + /* Older must also be completed with valid matchEndRow */ + if (older->states != NULL || older->matchEndRow < 0) + continue; - for (index2 = 0;; index2++) + /* + * Older context absorbs newer if it has the same or later + * matchEndRow, meaning all matches from newer are subsets. + */ + if (older->matchEndRow >= ctx->matchEndRow) + { + /* Absorb: remove ctx (newer) */ + nfa_unlink_context(winstate, ctx); + nfa_context_free(winstate, ctx); + absorbed = true; + } + } + + ctx = nextCtx; + continue; + } + + /* + * Check if all states in ctx are on absorbable branches. + * If any state is on a non-absorbable branch, skip this context. + */ { - pos2 = variable_pos_fetch(variable_pos, initial2, index2); - if (pos2 < 0) - break; - if (pos1 <= pos2) - return true; + RPRNFAState *s; + bool allAbsorbable = true; + + for (s = ctx->states; s != NULL && allAbsorbable; s = s->next) + { + RPRPatternElement *branchFirst; + + /* altPriority is the branch's first element index */ + if (s->altPriority < 0 || s->altPriority >= pattern->numElements) + continue; + + branchFirst = &pattern->elements[s->altPriority]; + if (!(branchFirst->flags & RPR_ELEM_ABSORBABLE)) + allAbsorbable = false; + } + + if (!allAbsorbable) + { + ctx = nextCtx; + continue; + } } + + /* + * Check if any older context can absorb this one. + * Older contexts have lower matchStartRow. + */ + for (older = ctx->next; older != NULL && !absorbed; older = older->next) + { + RPRNFAState *laterState; + RPRNFAState *earlierState; + bool canAbsorb; + int laterCount = 0; + int earlierCount = 0; + + /* Skip if not started earlier */ + if (older->matchStartRow >= ctx->matchStartRow) + continue; + + /* Skip contexts that haven't started processing yet */ + if (older->matchStartRow > currentPos) + continue; + + /* + * For in-progress ctx, older must also be in-progress. + * (Completed older contexts are handled above for completed ctx.) + */ + if (older->states == NULL) + continue; + + /* Count states in both contexts */ + for (laterState = ctx->states; laterState != NULL; laterState = laterState->next) + laterCount++; + for (earlierState = older->states; earlierState != NULL; earlierState = earlierState->next) + earlierCount++; + + /* Must have same number of states (same structure) */ + if (laterCount != earlierCount) + continue; + + /* + * Check if ALL states in ctx (later) are covered by states in older. + * Both must have the same set of element indices. + */ + canAbsorb = true; + for (laterState = ctx->states; laterState != NULL && canAbsorb; laterState = laterState->next) + { + bool found = false; + + for (earlierState = older->states; earlierState != NULL && !found; earlierState = earlierState->next) + { + RPRPatternElement *elem; + bool countsMatch; + int d; + + /* Must be at same element */ + if (earlierState->elemIdx != laterState->elemIdx) + continue; + + /* Must be on same branch (same altPriority) */ + if (earlierState->altPriority != laterState->altPriority) + continue; + + /* Handle invalid element index (terminal state) */ + if (laterState->elemIdx < 0 || laterState->elemIdx >= pattern->numElements) + { + found = true; + break; + } + + elem = &pattern->elements[laterState->elemIdx]; + countsMatch = true; + + if (elem->max == INT32_MAX) + { + /* Unbounded: earlier.count >= later.count at all depths */ + for (d = 0; d <= maxDepth && countsMatch; d++) + { + if (earlierState->counts[d] < laterState->counts[d]) + countsMatch = false; + } + } + else + { + /* Bounded: counts must be exactly equal at all depths */ + for (d = 0; d <= maxDepth && countsMatch; d++) + { + if (earlierState->counts[d] != laterState->counts[d]) + countsMatch = false; + } + } + + if (countsMatch) + found = true; + } + + if (!found) + canAbsorb = false; + } + + if (canAbsorb) + { + /* Absorb: remove ctx (newer) */ + nfa_unlink_context(winstate, ctx); + nfa_context_free(winstate, ctx); + absorbed = true; + } + } + + ctx = nextCtx; } - return false; } /* - * variable_pos_fetch + * nfa_finalize_boundary * - * Fetch position of pattern variable whose initial is initial, and whose index - * is index. If no postion was registered by initial, index, returns -1. + * Finalize NFA states at partition/frame boundary. + * Sets all varMatched to false and processes remaining states. */ -static int -variable_pos_fetch(VariablePos *variable_pos, char initial, int index) +static void +nfa_finalize_boundary(WindowAggState *winstate, RPRNFAContext *ctx, int64 matchEndPos) { - int pos = initial - 'a'; + RPRNFAState *states = ctx->states; + RPRNFAState *state; + RPRNFAState *nextState; + int numVars = list_length(winstate->defineVariableList); - if (pos < 0 || pos >= NUM_ALPHABETS) - elog(ERROR, "initial is not valid char: %c", initial); + ctx->states = NULL; - if (index < 0 || index >= NUM_ALPHABETS) - elog(ERROR, "index is not valid: %d", index); + for (int i = 0; i < numVars; i++) + winstate->nfaVarMatched[i] = false; - return variable_pos[pos].pos[index]; + for (state = states; state != NULL; state = nextState) + { + nextState = state->next; + state->next = NULL; + nfa_step_single(winstate, ctx, state, winstate->nfaVarMatched, matchEndPos); + } } /* - * variable_pos_build + * nfa_step_single * - * Build VariablePos structure and return it. + * Process one state through NFA for one row. + * New states are added to ctx->states. + * Match (FIN) is recorded in ctx->matchedState. + * When FIN is reached by matching (not skipping), matchEndRow is updated. */ -static VariablePos * -variable_pos_build(WindowAggState *winstate) +static void +nfa_step_single(WindowAggState *winstate, RPRNFAContext *ctx, + RPRNFAState *state, bool *varMatched, int64 currentPos) { - VariablePos *variable_pos; - StringInfo pattern_str; - int initial_index = 0; - ListCell *lc1, - *lc2; - - variable_pos = winstate->variable_pos = variable_pos_init(); - pattern_str = winstate->pattern_str = makeStringInfo(); - appendStringInfoChar(pattern_str, '^'); - - forboth(lc1, winstate->patternVariableList, - lc2, winstate->patternRegexpList) + RPRPattern *pattern = winstate->rpPattern; + RPRPatternElement *elements = pattern->elements; + RPRNFAState *pending = state; /* states to process in current row */ + + while (pending != NULL) { - char *vname = strVal(lfirst(lc1)); - char *quantifier = strVal(lfirst(lc2)); - char initial; + RPRPatternElement *elem; + bool matched; + int16 count; + int depth; - initial = pattern_initial(winstate, vname); - Assert(initial != 0); - appendStringInfoChar(pattern_str, initial); - if (quantifier[0]) - appendStringInfoChar(pattern_str, quantifier[0]); + state = pending; + pending = pending->next; + state->next = NULL; - /* - * Register the initial at initial_index. If the initial appears more - * than once, all of it's initial_index will be recorded. This could - * happen if a pattern variable appears in the PATTERN clause more - * than once like "UP DOWN UP" "UP UP UP". - */ - variable_pos_register(variable_pos, initial, initial_index); + Assert(state->elemIdx >= 0 && state->elemIdx < pattern->numElements); + elem = &elements[state->elemIdx]; + depth = elem->depth; - initial_index++; - } + if (RPRElemIsVar(elem)) + { + /* + * Variable: check if it matches current row. + * With DEFINE-first ordering, varId < numDefines has DEFINE expr, + * varId >= numDefines defaults to TRUE. + */ + int numDefines = list_length(winstate->defineVariableList); + + if (elem->varId >= numDefines) + matched = true; /* Not defined in DEFINE, defaults to TRUE */ + else + matched = varMatched[elem->varId]; + + count = state->counts[depth]; + + if (matched) + { + count++; + + if (elem->max != RPR_QUANTITY_INF && count > elem->max) + { + nfa_state_free(winstate, state); + continue; + } + + state->counts[depth] = count; + + /* Can repeat more? Clone for staying */ + if (elem->max == RPR_QUANTITY_INF || count < elem->max) + { + RPRNFAState *clone = nfa_state_clone(winstate, state->elemIdx, + state->altPriority, + state->counts, NULL); + nfa_add_state_unique(winstate, ctx, clone); + } + + /* Satisfied? Advance to next element */ + if (count >= elem->min) + { + RPRPatternElement *nextElem; + + state->counts[depth] = 0; + state->elemIdx = elem->next; + nextElem = &elements[state->elemIdx]; - return variable_pos; + if (RPRElemIsFin(nextElem)) + { + /* Match ends at current row since we matched */ + nfa_add_matched_state(winstate, ctx, state, currentPos); + nfa_state_free(winstate, state); + } + else if (RPRElemIsEnd(nextElem)) + { + /* + * END is epsilon transition - process immediately on same row. + * This ensures match end position is recorded at the row where + * the last VAR matched, not the next row. + */ + state->next = pending; + pending = state; + } + else + { + /* + * VAR, ALT - wait for next row. ALT dispatches to VARs that + * need input, so it must wait for the next row after VAR + * consumed the current row. + */ + nfa_add_state_unique(winstate, ctx, state); + } + } + else + { + nfa_state_free(winstate, state); + } + } + else + { + /* Not matched: can we skip? */ + if (count >= elem->min) + { + RPRPatternElement *nextElem; + + state->counts[depth] = 0; + state->elemIdx = elem->next; + nextElem = &elements[state->elemIdx]; + + if (RPRElemIsFin(nextElem)) + { + /* Match ends at previous row since current didn't match */ + nfa_add_matched_state(winstate, ctx, state, currentPos - 1); + nfa_state_free(winstate, state); + } + else if (RPRElemIsVar(nextElem)) + { + /* + * Current row was NOT consumed (skip case), so next VAR + * must be tried on the SAME row via pending list + */ + state->next = pending; + pending = state; + } + else + { + state->next = pending; + pending = state; + } + } + else + { + nfa_state_free(winstate, state); + } + } + } + else if (RPRElemIsFin(elem)) + { + /* Already at FIN - match ends at current row */ + nfa_add_matched_state(winstate, ctx, state, currentPos); + nfa_state_free(winstate, state); + } + else if (RPRElemIsAlt(elem)) + { + RPRElemIdx altIdx = elem->next; + bool first = true; + + /* + * ALT doesn't consume a row - it's just a dispatch point. + * All branches should be evaluated on the CURRENT row. + * Set altPriority to branch's elemIdx for lexical order tracking. + */ + while (altIdx >= 0 && altIdx < pattern->numElements) + { + RPRPatternElement *altElem = &elements[altIdx]; + + if (first) + { + state->elemIdx = altIdx; + state->altPriority = altIdx; /* lexical order */ + state->next = pending; + pending = state; + first = false; + } + else + { + pending = nfa_state_clone(winstate, altIdx, altIdx, + state->counts, pending); + } + + altIdx = altElem->jump; + } + + if (first) + nfa_state_free(winstate, state); + } + else if (RPRElemIsEnd(elem)) + { + count = state->counts[depth] + 1; + + if (count < elem->min) + { + /* + * Haven't reached minimum yet - must loop back. + * Add to ctx->states (next row) not pending (same row). + */ + state->counts[depth] = count; + for (int d = depth + 1; d < pattern->maxDepth; d++) + state->counts[d] = 0; + state->elemIdx = elem->jump; + nfa_add_state_unique(winstate, ctx, state); + } + else if (elem->max != RPR_QUANTITY_INF && count >= elem->max) + { + /* Reached maximum - must exit, continue processing */ + state->counts[depth] = 0; + state->elemIdx = elem->next; + state->next = pending; + pending = state; + } + else + { + /* + * Between min and max - can exit or continue. + * Exit state continues processing (pending). + * Loop state waits for next row (ctx->states). + */ + RPRNFAState *exitState = nfa_state_clone(winstate, elem->next, + state->altPriority, + state->counts, pending); + exitState->counts[depth] = 0; + pending = exitState; + + state->counts[depth] = count; + for (int d = depth + 1; d < pattern->maxDepth; d++) + state->counts[d] = 0; + state->elemIdx = elem->jump; + nfa_add_state_unique(winstate, ctx, state); + } + } + else + { + state->elemIdx = elem->next; + state->next = pending; + pending = state; + } + } } + diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c index ff22a04abe5..5d69805fc24 100644 --- a/src/backend/nodes/copyfuncs.c +++ b/src/backend/nodes/copyfuncs.c @@ -16,6 +16,7 @@ #include "postgres.h" #include "miscadmin.h" +#include "nodes/plannodes.h" #include "utils/datum.h" @@ -166,6 +167,44 @@ _copyBitmapset(const Bitmapset *from) return bms_copy(from); } +static RPRPattern * +_copyRPRPattern(const RPRPattern *from) +{ + RPRPattern *newnode = makeNode(RPRPattern); + + COPY_SCALAR_FIELD(numVars); + COPY_SCALAR_FIELD(maxDepth); + COPY_SCALAR_FIELD(numElements); + + /* Deep copy the varNames array */ + if (from->numVars > 0) + { + newnode->varNames = palloc0(from->numVars * sizeof(char *)); + for (int i = 0; i < from->numVars; i++) + newnode->varNames[i] = pstrdup(from->varNames[i]); + } + else + { + newnode->varNames = NULL; + } + + /* Deep copy the elements array */ + if (from->numElements > 0) + { + newnode->elements = palloc(from->numElements * sizeof(RPRPatternElement)); + memcpy(newnode->elements, from->elements, + from->numElements * sizeof(RPRPatternElement)); + } + else + { + newnode->elements = NULL; + } + + COPY_SCALAR_FIELD(isAbsorbable); + + return newnode; +} + /* * copyObjectImpl -- implementation of copyObject(); see nodes/nodes.h diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c index 3d1a1adf86e..9410790e3d3 100644 --- a/src/backend/nodes/equalfuncs.c +++ b/src/backend/nodes/equalfuncs.c @@ -20,6 +20,7 @@ #include "postgres.h" #include "miscadmin.h" +#include "nodes/plannodes.h" #include "utils/datum.h" @@ -149,6 +150,38 @@ _equalBitmapset(const Bitmapset *a, const Bitmapset *b) return bms_equal(a, b); } +static bool +_equalRPRPattern(const RPRPattern *a, const RPRPattern *b) +{ + COMPARE_SCALAR_FIELD(numVars); + COMPARE_SCALAR_FIELD(maxDepth); + COMPARE_SCALAR_FIELD(numElements); + + /* Compare varNames array */ + if (a->numVars > 0) + { + if (a->varNames == NULL || b->varNames == NULL) + return false; + for (int i = 0; i < a->numVars; i++) + { + if (strcmp(a->varNames[i], b->varNames[i]) != 0) + return false; + } + } + + /* Compare elements array */ + if (a->numElements > 0) + { + if (a->elements == NULL || b->elements == NULL) + return false; + if (memcmp(a->elements, b->elements, + a->numElements * sizeof(RPRPatternElement)) != 0) + return false; + } + + return true; +} + /* * Lists are handled specially */ diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c index c8eef2c75d2..1a70b016770 100644 --- a/src/backend/nodes/outfuncs.c +++ b/src/backend/nodes/outfuncs.c @@ -23,6 +23,7 @@ #include "nodes/bitmapset.h" #include "nodes/nodes.h" #include "nodes/pg_list.h" +#include "nodes/plannodes.h" #include "utils/datum.h" /* State flag that determines how nodeToStringInternal() should treat location fields */ @@ -718,6 +719,56 @@ _outA_Const(StringInfo str, const A_Const *node) WRITE_LOCATION_FIELD(location); } +static void +_outRPRPattern(StringInfo str, const RPRPattern *node) +{ + WRITE_NODE_TYPE("RPRPATTERN"); + + WRITE_INT_FIELD(numVars); + WRITE_INT_FIELD(maxDepth); + WRITE_INT_FIELD(numElements); + + /* Write varNames array as list of strings */ + appendStringInfoString(str, " :varNames"); + if (node->numVars > 0 && node->varNames != NULL) + { + appendStringInfoString(str, " ("); + for (int i = 0; i < node->numVars; i++) + { + if (i > 0) + appendStringInfoChar(str, ' '); + outToken(str, node->varNames[i]); + } + appendStringInfoChar(str, ')'); + } + else + appendStringInfoString(str, " <>"); + + /* Write elements array */ + appendStringInfoString(str, " :elements"); + if (node->numElements > 0 && node->elements != NULL) + { + appendStringInfoChar(str, ' '); + for (int i = 0; i < node->numElements; i++) + { + const RPRPatternElement *elem = &node->elements[i]; + + appendStringInfo(str, "(%d %u %d %d %d %d %d)", + (int) elem->varId, + (unsigned) elem->flags, + (int) elem->depth, + (int) elem->min, + (int) elem->max, + (int) elem->next, + (int) elem->jump); + } + } + else + appendStringInfoString(str, " <>"); + + WRITE_BOOL_FIELD(isAbsorbable); +} + /* * outNode - diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c index c11728c0f17..3ed31ba539d 100644 --- a/src/backend/nodes/readfuncs.c +++ b/src/backend/nodes/readfuncs.c @@ -30,6 +30,7 @@ #include "miscadmin.h" #include "nodes/bitmapset.h" +#include "nodes/plannodes.h" #include "nodes/readfuncs.h" @@ -560,6 +561,84 @@ _readExtensibleNode(void) READ_DONE(); } +static RPRPattern * +_readRPRPattern(void) +{ + READ_LOCALS(RPRPattern); + + READ_INT_FIELD(numVars); + READ_INT_FIELD(maxDepth); + READ_INT_FIELD(numElements); + + /* Read varNames array */ + token = pg_strtok(&length); /* skip :varNames */ + token = pg_strtok(&length); /* get '(' or '<>' */ + if (local_node->numVars > 0 && token[0] == '(') + { + local_node->varNames = palloc(local_node->numVars * sizeof(char *)); + for (int i = 0; i < local_node->numVars; i++) + { + token = pg_strtok(&length); + local_node->varNames[i] = debackslash(token, length); + } + token = pg_strtok(&length); /* skip ')' */ + } + else + { + local_node->varNames = NULL; + } + + /* Read elements array */ + token = pg_strtok(&length); /* skip :elements */ + token = pg_strtok(&length); /* get '(' or '<>' */ + if (local_node->numElements > 0 && token[0] == '(') + { + local_node->elements = palloc0(local_node->numElements * sizeof(RPRPatternElement)); + for (int i = 0; i < local_node->numElements; i++) + { + RPRPatternElement *elem = &local_node->elements[i]; + int varId, flags, depth, min, max, next, jump; + + /* Parse "(varId flags depth min max next jump)" */ + token = pg_strtok(&length); + varId = atoi(token); + token = pg_strtok(&length); + flags = atoi(token); + token = pg_strtok(&length); + depth = atoi(token); + token = pg_strtok(&length); + min = atoi(token); + token = pg_strtok(&length); + max = atoi(token); + token = pg_strtok(&length); + next = atoi(token); + token = pg_strtok(&length); + jump = atoi(token); + token = pg_strtok(&length); /* skip ')' */ + + elem->varId = (RPRVarId) varId; + elem->flags = (RPRElemFlags) flags; + elem->depth = (RPRDepth) depth; + elem->min = (RPRQuantity) min; + elem->max = (RPRQuantity) max; + elem->next = (RPRElemIdx) next; + elem->jump = (RPRElemIdx) jump; + + /* Read next element's '(' or end */ + if (i < local_node->numElements - 1) + token = pg_strtok(&length); /* get '(' */ + } + } + else + { + local_node->elements = NULL; + } + + READ_BOOL_FIELD(isAbsorbable); + + READ_DONE(); +} + /* * parseNodeString diff --git a/src/backend/optimizer/plan/Makefile b/src/backend/optimizer/plan/Makefile index 80ef162e484..7e0167789d8 100644 --- a/src/backend/optimizer/plan/Makefile +++ b/src/backend/optimizer/plan/Makefile @@ -19,6 +19,7 @@ OBJS = \ planagg.o \ planmain.o \ planner.o \ + rpr.o \ setrefs.o \ subselect.o diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c index c287d2d05fd..51334f80a5c 100644 --- a/src/backend/optimizer/plan/createplan.c +++ b/src/backend/optimizer/plan/createplan.c @@ -37,6 +37,7 @@ #include "optimizer/prep.h" #include "optimizer/restrictinfo.h" #include "optimizer/subselect.h" +#include "optimizer/rpr.h" #include "optimizer/tlist.h" #include "parser/parse_clause.h" #include "parser/parsetree.h" @@ -289,8 +290,9 @@ static Memoize *make_memoize(Plan *lefttree, Oid *hashoperators, static WindowAgg *make_windowagg(List *tlist, WindowClause *wc, int partNumCols, AttrNumber *partColIdx, Oid *partOperators, Oid *partCollations, int ordNumCols, AttrNumber *ordColIdx, Oid *ordOperators, Oid *ordCollations, - List *runCondition, RPSkipTo rpSkipTo, List *patternVariable, - List *patternRegexp, List *defineClause, List *defineInitial, + List *runCondition, RPSkipTo rpSkipTo, + RPRPattern *compiledPattern, + List *defineClause, List *defineInitial, List *qual, bool topWindow, Plan *lefttree); static Group *make_group(List *tlist, List *qual, int numGroupCols, @@ -2532,26 +2534,37 @@ create_windowagg_plan(PlannerInfo *root, WindowAggPath *best_path) ordNumCols++; } - /* And finally we can make the WindowAgg node */ - plan = make_windowagg(tlist, - wc, - partNumCols, - partColIdx, - partOperators, - partCollations, - ordNumCols, - ordColIdx, - ordOperators, - ordCollations, - best_path->runCondition, - wc->rpSkipTo, - wc->patternVariable, - wc->patternRegexp, - wc->defineClause, - wc->defineInitial, - best_path->qual, - best_path->topwindow, - subplan); + /* Build defineVariableList from defineClause for pattern compilation */ + { + List *defineVariableList = NIL; + + foreach(lc, wc->defineClause) + { + TargetEntry *te = (TargetEntry *) lfirst(lc); + defineVariableList = lappend(defineVariableList, + makeString(pstrdup(te->resname))); + } + + /* And finally we can make the WindowAgg node */ + plan = make_windowagg(tlist, + wc, + partNumCols, + partColIdx, + partOperators, + partCollations, + ordNumCols, + ordColIdx, + ordOperators, + ordCollations, + best_path->runCondition, + wc->rpSkipTo, + buildRPRPattern(wc->rpPattern, defineVariableList), + wc->defineClause, + wc->defineInitial, + best_path->qual, + best_path->topwindow, + subplan); + } copy_generic_path_info(&plan->plan, (Path *) best_path); @@ -6617,8 +6630,9 @@ static WindowAgg * make_windowagg(List *tlist, WindowClause *wc, int partNumCols, AttrNumber *partColIdx, Oid *partOperators, Oid *partCollations, int ordNumCols, AttrNumber *ordColIdx, Oid *ordOperators, Oid *ordCollations, - List *runCondition, RPSkipTo rpSkipTo, List *patternVariable, - List *patternRegexp, List *defineClause, List *defineInitial, + List *runCondition, RPSkipTo rpSkipTo, + RPRPattern *compiledPattern, + List *defineClause, List *defineInitial, List *qual, bool topWindow, Plan *lefttree) { WindowAgg *node = makeNode(WindowAgg); @@ -6647,8 +6661,10 @@ make_windowagg(List *tlist, WindowClause *wc, node->inRangeNullsFirst = wc->inRangeNullsFirst; node->topWindow = topWindow; node->rpSkipTo = rpSkipTo; - node->patternVariable = patternVariable; - node->patternRegexp = patternRegexp; + + /* Store compiled pattern for NFA execution */ + node->rpPattern = compiledPattern; + node->defineClause = defineClause; node->defineInitial = defineInitial; diff --git a/src/backend/optimizer/plan/meson.build b/src/backend/optimizer/plan/meson.build index c565b2adbcc..5b2381cd774 100644 --- a/src/backend/optimizer/plan/meson.build +++ b/src/backend/optimizer/plan/meson.build @@ -7,6 +7,7 @@ backend_sources += files( 'planagg.c', 'planmain.c', 'planner.c', + 'rpr.c', 'setrefs.c', 'subselect.c', ) diff --git a/src/backend/optimizer/plan/rpr.c b/src/backend/optimizer/plan/rpr.c new file mode 100644 index 00000000000..0eed6b865ae --- /dev/null +++ b/src/backend/optimizer/plan/rpr.c @@ -0,0 +1,1024 @@ +/*------------------------------------------------------------------------- + * + * rpr.c + * Row Pattern Recognition pattern compilation for planner + * + * This file contains functions for optimizing RPR pattern AST and + * compiling it to bytecode for execution by WindowAgg. + * + * Portions Copyright (c) 1996-2025, PostgreSQL Global Development Group + * Portions Copyright (c) 1994, Regents of the University of California + * + * + * IDENTIFICATION + * src/backend/optimizer/plan/rpr.c + * + *------------------------------------------------------------------------- + */ + +#include "postgres.h" + +#include "nodes/makefuncs.h" +#include "optimizer/rpr.h" + +/* Forward declarations */ +static bool rprPatternEqual(RPRPatternNode *a, RPRPatternNode *b); +static RPRPatternNode *optimizeRPRPattern(RPRPatternNode *pattern); +static void scanRPRPattern(RPRPatternNode *node, char **varNames, int *numVars, + int *numElements, RPRDepth depth, RPRDepth *maxDepth); +static void fillRPRPattern(RPRPatternNode *node, RPRPattern *pat, + int *idx, RPRDepth depth); +static RPRVarId getVarIdFromPattern(RPRPattern *pat, const char *varName); +static void computeAbsorbability(RPRPattern *pattern); + +/* + * rprPatternChildrenEqual + * Compare children of two GROUP nodes for equality. + * + * Returns true if the children lists are structurally identical. + * Used for GROUP merge optimization where we ignore outer quantifiers. + */ +static bool +rprPatternChildrenEqual(List *a, List *b) +{ + ListCell *lca, + *lcb; + + if (list_length(a) != list_length(b)) + return false; + + forboth(lca, a, lcb, b) + { + if (!rprPatternEqual((RPRPatternNode *) lfirst(lca), + (RPRPatternNode *) lfirst(lcb))) + return false; + } + + return true; +} + +/* + * rprPatternEqual + * Compare two RPRPatternNode trees for equality. + * + * Returns true if the trees are structurally identical. + */ +static bool +rprPatternEqual(RPRPatternNode *a, RPRPatternNode *b) +{ + ListCell *lca, + *lcb; + + if (a == NULL && b == NULL) + return true; + if (a == NULL || b == NULL) + return false; + + /* Must have same node type and quantifiers */ + if (a->nodeType != b->nodeType) + return false; + if (a->min != b->min || a->max != b->max) + return false; + if (a->reluctant != b->reluctant) + return false; + + switch (a->nodeType) + { + case RPR_PATTERN_VAR: + return strcmp(a->varName, b->varName) == 0; + + case RPR_PATTERN_SEQ: + case RPR_PATTERN_ALT: + case RPR_PATTERN_GROUP: + /* Must have same number of children */ + if (list_length(a->children) != list_length(b->children)) + return false; + + /* Compare each child */ + forboth(lca, a->children, lcb, b->children) + { + if (!rprPatternEqual((RPRPatternNode *) lfirst(lca), + (RPRPatternNode *) lfirst(lcb))) + return false; + } + return true; + } + + return false; /* keep compiler quiet */ +} + +/* + * optimizeRPRPattern + * Optimize RPRPatternNode tree in a single pass. + * + * Optimizations applied (bottom-up, in order per node): + * 1. Flatten nested SEQ: (A (B C)) -> (A B C) + * 2. Flatten nested ALT: (A | (B | C)) -> (A | B | C) + * 3. Unwrap GROUP{1,1}: ((A)) -> A, (A B){1,1} -> A B + * 4. Quantifier multiplication: (A{2}){3} -> A{6} + * 5. Remove duplicate alternatives: (A | B | A) -> (A | B) + * 6. Merge consecutive vars: A A A -> A{3,3} + * 7. Remove single-item SEQ/ALT wrappers + */ +static RPRPatternNode * +optimizeRPRPattern(RPRPatternNode *pattern) +{ + ListCell *lc; + List *newChildren; + + if (pattern == NULL) + return NULL; + + switch (pattern->nodeType) + { + case RPR_PATTERN_VAR: + /* Leaf node - nothing to optimize */ + return pattern; + + case RPR_PATTERN_SEQ: + { + RPRPatternNode *prev = NULL; + + /* Recursively optimize children, flatten SEQ/GROUP{1,1} */ + newChildren = NIL; + foreach(lc, pattern->children) + { + RPRPatternNode *child = (RPRPatternNode *) lfirst(lc); + RPRPatternNode *opt = optimizeRPRPattern(child); + + /* Flatten GROUP{1,1} or nested SEQ */ + if ((opt->nodeType == RPR_PATTERN_GROUP && + opt->min == 1 && opt->max == 1 && !opt->reluctant) || + opt->nodeType == RPR_PATTERN_SEQ) + { + newChildren = list_concat(newChildren, + list_copy(opt->children)); + } + else + { + newChildren = lappend(newChildren, opt); + } + } + + /* + * Merge consecutive identical VAR nodes with any quantifier. + * A{m1,M1} A{m2,M2} -> A{m1+m2, M1+M2} + * where INF + x = INF + */ + { + List *mergedChildren = NIL; + + foreach(lc, newChildren) + { + RPRPatternNode *child = (RPRPatternNode *) lfirst(lc); + + if (child->nodeType == RPR_PATTERN_VAR && !child->reluctant) + { + if (prev != NULL && + prev->nodeType == RPR_PATTERN_VAR && + strcmp(prev->varName, child->varName) == 0 && + !prev->reluctant) + { + /* + * Merge: accumulate min/max into prev. + * INF + anything = INF + */ + prev->min += child->min; + if (prev->max == RPR_QUANTITY_INF || + child->max == RPR_QUANTITY_INF) + prev->max = RPR_QUANTITY_INF; + else + prev->max += child->max; + } + else + { + /* Flush previous and start new */ + if (prev != NULL) + mergedChildren = lappend(mergedChildren, prev); + prev = child; + } + } + else + { + /* Non-mergeable - flush previous */ + if (prev != NULL) + mergedChildren = lappend(mergedChildren, prev); + mergedChildren = lappend(mergedChildren, child); + prev = NULL; + } + } + + /* Flush remaining */ + if (prev != NULL) + mergedChildren = lappend(mergedChildren, prev); + + newChildren = mergedChildren; + } + + /* + * Merge sequence prefix/suffix into GROUP with matching children. + * A B (A B)+ -> (A B){2,} + * (A B)+ A B -> (A B){2,} + * A B (A B)+ A B -> (A B){3,} + */ + { + List *groupMergedChildren = NIL; + int numChildren = list_length(newChildren); + int i; + bool merged = false; + int skipUntil = -1; /* skip suffix elements already absorbed */ + + for (i = 0; i < numChildren; i++) + { + RPRPatternNode *child = (RPRPatternNode *) list_nth(newChildren, i); + + /* Skip elements that were absorbed as suffix */ + if (i < skipUntil) + continue; + + /* + * If this is a GROUP, see if preceding/following elements + * match its children. + * GROUP's content may be wrapped in a SEQ - unwrap for comparison. + */ + if (child->nodeType == RPR_PATTERN_GROUP && !child->reluctant) + { + List *groupContent = child->children; + int groupChildCount; + int prefixLen = list_length(groupMergedChildren); + + /* + * If GROUP has single SEQ child, compare with SEQ's children. + * e.g., (A B)+ is GROUP[SEQ[A,B]], we want to compare [A,B]. + */ + if (list_length(groupContent) == 1) + { + RPRPatternNode *inner = (RPRPatternNode *) linitial(groupContent); + + if (inner->nodeType == RPR_PATTERN_SEQ) + groupContent = inner->children; + } + + groupChildCount = list_length(groupContent); + + /* + * PREFIX MERGE: Check if preceding elements match. + * Keep absorbing as long as we have matching prefixes. + */ + while (prefixLen >= groupChildCount && groupChildCount > 0) + { + List *prefixElements = NIL; + int j; + + /* Extract last groupChildCount elements from prefix */ + for (j = prefixLen - groupChildCount; j < prefixLen; j++) + { + prefixElements = lappend(prefixElements, + list_nth(groupMergedChildren, j)); + } + + /* Compare with GROUP's (possibly unwrapped) children */ + if (rprPatternChildrenEqual(prefixElements, groupContent)) + { + /* + * Match! Merge by incrementing GROUP's min. + * Remove the prefix elements from output. + */ + child->min += 1; + + /* Rebuild groupMergedChildren without matched prefix */ + { + List *trimmed = NIL; + + for (j = 0; j < prefixLen - groupChildCount; j++) + { + trimmed = lappend(trimmed, + list_nth(groupMergedChildren, j)); + } + groupMergedChildren = trimmed; + prefixLen = list_length(groupMergedChildren); + } + merged = true; + } + else + { + list_free(prefixElements); + break; + } + + list_free(prefixElements); + } + + /* + * SUFFIX MERGE: Check if following elements match. + * Keep absorbing as long as we have matching suffixes. + */ + while (i + groupChildCount < numChildren && groupChildCount > 0) + { + List *suffixElements = NIL; + int j; + int suffixStart = i + 1; + + /* Adjust for already absorbed elements */ + if (skipUntil > suffixStart) + break; + + /* Extract next groupChildCount elements as suffix */ + for (j = 0; j < groupChildCount; j++) + { + int idx = suffixStart + j; + + if (idx >= numChildren) + break; + suffixElements = lappend(suffixElements, + list_nth(newChildren, idx)); + } + + /* Compare with GROUP's children */ + if (list_length(suffixElements) == groupChildCount && + rprPatternChildrenEqual(suffixElements, groupContent)) + { + /* + * Match! Absorb suffix by incrementing min and skipping. + */ + child->min += 1; + skipUntil = suffixStart + groupChildCount; + merged = true; + /* Update i to continue suffix check after absorbed elements */ + i = skipUntil - 1; + } + else + { + list_free(suffixElements); + break; + } + + list_free(suffixElements); + } + } + + groupMergedChildren = lappend(groupMergedChildren, child); + } + + if (merged) + newChildren = groupMergedChildren; + } + + pattern->children = newChildren; + + /* Unwrap single-item SEQ */ + if (list_length(pattern->children) == 1) + return (RPRPatternNode *) linitial(pattern->children); + + return pattern; + } + + case RPR_PATTERN_ALT: + { + ListCell *lc2; + + /* Recursively optimize children, flatten nested ALT */ + newChildren = NIL; + foreach(lc, pattern->children) + { + RPRPatternNode *child = (RPRPatternNode *) lfirst(lc); + RPRPatternNode *opt = optimizeRPRPattern(child); + + if (opt->nodeType == RPR_PATTERN_ALT) + { + newChildren = list_concat(newChildren, + list_copy(opt->children)); + } + else + { + newChildren = lappend(newChildren, opt); + } + } + + /* Remove duplicate alternatives */ + { + List *uniqueChildren = NIL; + + foreach(lc, newChildren) + { + RPRPatternNode *child = (RPRPatternNode *) lfirst(lc); + bool isDuplicate = false; + + foreach(lc2, uniqueChildren) + { + if (rprPatternEqual((RPRPatternNode *) lfirst(lc2), child)) + { + isDuplicate = true; + break; + } + } + + if (!isDuplicate) + uniqueChildren = lappend(uniqueChildren, child); + } + + pattern->children = uniqueChildren; + } + + /* Unwrap single-item ALT */ + if (list_length(pattern->children) == 1) + return (RPRPatternNode *) linitial(pattern->children); + + return pattern; + } + + case RPR_PATTERN_GROUP: + /* Recursively optimize children */ + newChildren = NIL; + foreach(lc, pattern->children) + { + RPRPatternNode *child = (RPRPatternNode *) lfirst(lc); + + newChildren = lappend(newChildren, optimizeRPRPattern(child)); + } + pattern->children = newChildren; + + /* Quantifier multiplication: (A{m}){n} -> A{m*n} */ + if (list_length(pattern->children) == 1 && !pattern->reluctant) + { + RPRPatternNode *child = (RPRPatternNode *) linitial(pattern->children); + + if (child->nodeType == RPR_PATTERN_VAR && !child->reluctant) + { + if (pattern->max != RPR_QUANTITY_INF && child->max != RPR_QUANTITY_INF) + { + int64 new_min_64 = (int64) pattern->min * child->min; + int64 new_max_64 = (int64) pattern->max * child->max; + + if (new_min_64 < RPR_QUANTITY_INF && new_max_64 < RPR_QUANTITY_INF) + { + child->min = (int) new_min_64; + child->max = (int) new_max_64; + return child; + } + } + } + } + + /* Unwrap GROUP{1,1} */ + if (pattern->min == 1 && pattern->max == 1 && !pattern->reluctant) + { + if (list_length(pattern->children) == 1) + return (RPRPatternNode *) linitial(pattern->children); + + /* Multiple children: convert to SEQ */ + pattern->nodeType = RPR_PATTERN_SEQ; + } + + return pattern; + } + + return pattern; /* keep compiler quiet */ +} + +/* + * scanRPRPattern + * Single-pass scan: collect variable names and count elements. + * + * Collects unique variable names in pattern encounter order (max RPR_VARID_MAX). + * Also counts elements and tracks maximum nesting depth. + */ +static void +scanRPRPattern(RPRPatternNode *node, char **varNames, int *numVars, + int *numElements, RPRDepth depth, RPRDepth *maxDepth) +{ + ListCell *lc; + int i; + + if (node == NULL) + return; + + /* Track maximum depth */ + if (depth > *maxDepth) + *maxDepth = depth; + + switch (node->nodeType) + { + case RPR_PATTERN_VAR: + /* Count element */ + (*numElements)++; + + /* Collect variable name if not already present */ + for (i = 0; i < *numVars; i++) + { + if (strcmp(varNames[i], node->varName) == 0) + return; /* Already have this variable */ + } + + /* New variable */ + if (*numVars >= RPR_VARID_MAX) + ereport(ERROR, + (errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED), + errmsg("too many pattern variables"), + errdetail("Maximum is %d.", RPR_VARID_MAX))); + + varNames[*numVars] = node->varName; + (*numVars)++; + break; + + case RPR_PATTERN_SEQ: + /* Sequence: just recurse into children */ + foreach(lc, node->children) + { + scanRPRPattern((RPRPatternNode *) lfirst(lc), varNames, numVars, + numElements, depth, maxDepth); + } + break; + + case RPR_PATTERN_GROUP: + /* Recurse into children at increased depth */ + foreach(lc, node->children) + { + scanRPRPattern((RPRPatternNode *) lfirst(lc), varNames, numVars, + numElements, depth + 1, maxDepth); + } + + /* Add END element if group has non-trivial quantifier */ + if (node->min != 1 || node->max != 1) + (*numElements)++; + break; + + case RPR_PATTERN_ALT: + /* Count ALT start element */ + (*numElements)++; + + /* Recurse into children at increased depth */ + foreach(lc, node->children) + { + scanRPRPattern((RPRPatternNode *) lfirst(lc), varNames, numVars, + numElements, depth + 1, maxDepth); + } + break; + } +} + +/* + * getVarIdFromPattern + * Get variable ID for a variable name from RPRPattern. + * + * Returns the index of the variable in the varNames array. + */ +static RPRVarId +getVarIdFromPattern(RPRPattern *pat, const char *varName) +{ + for (int i = 0; i < pat->numVars; i++) + { + if (strcmp(pat->varNames[i], varName) == 0) + return (RPRVarId) i; + } + + /* Should not happen - variable should already be collected */ + elog(ERROR, "pattern variable \"%s\" not found", varName); + pg_unreachable(); +} + +/* + * fillRPRPattern + * Fill the pattern elements array (second pass). + * + * This traverses the AST and fills in the pre-allocated elements array. + * The idx pointer tracks the current position in the array. + */ +static void +fillRPRPattern(RPRPatternNode *node, RPRPattern *pat, int *idx, RPRDepth depth) +{ + ListCell *lc; + RPRPatternElement *elem; + int groupStartIdx; + int altStartIdx; + List *altBranchStarts; + List *altEndPositions; + + if (node == NULL) + return; + + switch (node->nodeType) + { + case RPR_PATTERN_SEQ: + /* Sequence: fill each child in order */ + foreach(lc, node->children) + { + fillRPRPattern((RPRPatternNode *) lfirst(lc), pat, idx, depth); + } + break; + + case RPR_PATTERN_VAR: + /* Variable: create element with varId */ + elem = &pat->elements[*idx]; + memset(elem, 0, sizeof(RPRPatternElement)); + elem->varId = getVarIdFromPattern(pat, node->varName); + elem->depth = depth; + elem->min = node->min; + elem->max = (node->max == INT_MAX) ? RPR_QUANTITY_INF : node->max; + elem->next = RPR_ELEMIDX_INVALID; + elem->jump = RPR_ELEMIDX_INVALID; + if (node->reluctant) + elem->flags |= RPR_ELEM_RELUCTANT; + (*idx)++; + break; + + case RPR_PATTERN_GROUP: + groupStartIdx = *idx; + + /* Fill group content at increased depth */ + foreach(lc, node->children) + { + fillRPRPattern((RPRPatternNode *) lfirst(lc), pat, idx, depth + 1); + } + + /* Add group end marker if group has non-trivial quantifier */ + if (node->min != 1 || node->max != 1) + { + elem = &pat->elements[*idx]; + memset(elem, 0, sizeof(RPRPatternElement)); + elem->varId = RPR_VARID_END; + elem->depth = depth; /* parent depth for iteration count */ + elem->min = node->min; + elem->max = (node->max == INT_MAX) ? RPR_QUANTITY_INF : node->max; + elem->next = RPR_ELEMIDX_INVALID; + elem->jump = groupStartIdx; /* jump back to group start */ + if (node->reluctant) + elem->flags |= RPR_ELEM_RELUCTANT; + (*idx)++; + } + break; + + case RPR_PATTERN_ALT: + /* Add alternation start marker */ + altStartIdx = *idx; + elem = &pat->elements[*idx]; + memset(elem, 0, sizeof(RPRPatternElement)); + elem->varId = RPR_VARID_ALT; + elem->depth = depth; + elem->min = 1; + elem->max = 1; + elem->next = RPR_ELEMIDX_INVALID; + elem->jump = RPR_ELEMIDX_INVALID; + (*idx)++; + + altBranchStarts = NIL; + altEndPositions = NIL; + + /* Fill each alternative */ + foreach(lc, node->children) + { + RPRPatternNode *alt = (RPRPatternNode *) lfirst(lc); + int branchStart = *idx; + + altBranchStarts = lappend_int(altBranchStarts, branchStart); + + fillRPRPattern(alt, pat, idx, depth + 1); + + /* Record end position if any elements were added */ + if (*idx > branchStart) + altEndPositions = lappend_int(altEndPositions, *idx - 1); + } + + /* Set ALT_START.next to first alternative */ + if (list_length(altBranchStarts) > 0) + pat->elements[altStartIdx].next = linitial_int(altBranchStarts); + + /* Set jump on first element of each alternative to next alternative */ + foreach(lc, altBranchStarts) + { + int firstElemIdx = lfirst_int(lc); + + if (lnext(altBranchStarts, lc) != NULL) + { + int nextAltStart = lfirst_int(lnext(altBranchStarts, lc)); + + pat->elements[firstElemIdx].jump = nextAltStart; + } + /* Last alternative's jump stays as -1 (default) */ + } + + /* Set next on last element of each alternative to after the alternation */ + { + int afterAltIdx = *idx; + + foreach(lc, altEndPositions) + { + int endPos = lfirst_int(lc); + + if (pat->elements[endPos].next == RPR_ELEMIDX_INVALID) + pat->elements[endPos].next = afterAltIdx; + } + } + + list_free(altBranchStarts); + list_free(altEndPositions); + break; + } +} + +/* + * buildRPRPattern + * Build flat pattern element array from AST. + * + * Optimizes the pattern tree, then uses 2-pass: count elements, allocate and fill. + * Returns NULL if pattern is NULL. + * + * This function is called from createplan.c during plan creation. + */ +RPRPattern * +buildRPRPattern(RPRPatternNode *pattern, List *defineVariableList) +{ + RPRPattern *result; + RPRPatternNode *optimized; + char *varNamesStack[RPR_VARID_MAX + 1]; /* stack array for collection */ + int numVars = 0; + int numElements = 0; + RPRDepth maxDepth = 0; + int idx; + int finIdx; + int i; + RPRPatternElement *finElem; + ListCell *lc; + + if (pattern == NULL) + return NULL; + + /* Optimize the pattern tree (planner phase) */ + optimized = optimizeRPRPattern(pattern); + + /* + * First, populate varNames from defineVariableList in order. + * This ensures varId == defineIdx for all defined variables, + * eliminating the need for varIdToDefineIdx mapping. + */ + foreach(lc, defineVariableList) + { + char *varName = strVal(lfirst(lc)); + + if (numVars >= RPR_VARID_MAX) + ereport(ERROR, + (errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED), + errmsg("too many pattern variables"), + errdetail("Maximum is %d.", RPR_VARID_MAX))); + + varNamesStack[numVars] = varName; + numVars++; + } + + /* + * Pass 1: Single scan to collect variable names and count elements. + * Variables already in varNamesStack (from DEFINE) are skipped. + * Also counts elements and tracks max depth in one traversal. + */ + scanRPRPattern(optimized, varNamesStack, &numVars, + &numElements, 0, &maxDepth); + numElements++; /* +1 for FIN marker */ + + /* Check element count limit */ + if (numElements > RPR_ELEMIDX_MAX) + ereport(ERROR, + (errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED), + errmsg("pattern too complex"), + errdetail("Pattern has %d elements, maximum is %d.", + numElements, RPR_ELEMIDX_MAX))); + + /* + * Allocate result structure with makeNode for proper NodeTag. + */ + result = makeNode(RPRPattern); + result->numVars = numVars; + result->maxDepth = maxDepth + 1; /* +1: depth is 0-based, need counts[0..maxDepth] */ + result->numElements = numElements; + + /* Build varNames as char** array */ + if (numVars > 0) + { + result->varNames = palloc(numVars * sizeof(char *)); + for (i = 0; i < numVars; i++) + result->varNames[i] = pstrdup(varNamesStack[i]); + } + else + { + result->varNames = NULL; + } + + /* Allocate elements array separately (zero-init for reserved field) */ + result->elements = palloc0(numElements * sizeof(RPRPatternElement)); + + /* + * Pass 2: Fill elements directly into pre-allocated array. + */ + idx = 0; + fillRPRPattern(optimized, result, &idx, 0); + + /* Set up next pointers for elements that don't have one yet */ + finIdx = numElements - 1; /* FIN marker is at the last position */ + for (i = 0; i < finIdx; i++) + { + if (result->elements[i].next == RPR_ELEMIDX_INVALID) + { + /* Point to next element, or to FIN if this is the last */ + result->elements[i].next = (i < finIdx - 1) ? i + 1 : finIdx; + } + } + + /* Add FIN marker at the end */ + finElem = &result->elements[finIdx]; + memset(finElem, 0, sizeof(RPRPatternElement)); + finElem->varId = RPR_VARID_FIN; + finElem->depth = 0; + finElem->min = 1; + finElem->max = 1; + finElem->next = RPR_ELEMIDX_INVALID; + finElem->jump = RPR_ELEMIDX_INVALID; + + /* Compute context absorption eligibility */ + computeAbsorbability(result); + + return result; +} + +/* + * computeAbsorbability + * Determine if pattern supports context absorption optimization. + * + * Context absorption is safe when later matches are guaranteed to be + * suffixes of earlier matches (both row positions AND content). + * + * Sets RPR_ELEM_ABSORBABLE flag on the first element of each absorbable + * branch. A branch is absorbable if: + * - It starts with an unbounded element (greedy-first) + * - It has exactly one unbounded element + * - It's not inside an unbounded GROUP (GREEDY(ALT) case) + * + * pattern->isAbsorbable is set true if ANY branch is absorbable. + */ +static void +computeAbsorbability(RPRPattern *pattern) +{ + int i; + int maxAltDepth = -1; + RPRDepth minUnboundedGroupDepth = UINT8_MAX; + bool hasTopLevelAlt = false; + + pattern->isAbsorbable = false; + + if (pattern->numElements < 2) /* At minimum: one element + FIN */ + return; + + /* + * Scan elements to find ALT structure and unbounded GROUP depth. + */ + for (i = 0; i < pattern->numElements - 1; i++) + { + RPRPatternElement *elem = &pattern->elements[i]; + + if (elem->varId == RPR_VARID_ALT) + { + if (elem->depth > maxAltDepth) + maxAltDepth = elem->depth; + if (i == 0) + hasTopLevelAlt = true; + } + else if (elem->varId == RPR_VARID_END) + { + if (elem->max == INT32_MAX && elem->depth < minUnboundedGroupDepth) + minUnboundedGroupDepth = elem->depth; + } + } + + /* + * Check for GREEDY(ALT) pattern: ALT inside unbounded GROUP. + * In this case, no branch can be absorbable. + */ + if (maxAltDepth >= 0 && maxAltDepth > minUnboundedGroupDepth) + return; + + /* + * For top-level ALT patterns, check each branch separately. + * Set ABSORBABLE flag on branches that qualify. + */ + if (hasTopLevelAlt) + { + int branchStart = 1; /* after ALT marker */ + int patternEnd = pattern->numElements - 1; + + while (branchStart < patternEnd) + { + RPRPatternElement *branchFirst = &pattern->elements[branchStart]; + int branchEnd; + int branchUnbounded = 0; + int j; + bool branchAbsorbable = true; + + /* Find branch end */ + if (branchFirst->jump != RPR_ELEMIDX_INVALID) + branchEnd = branchFirst->jump; + else + branchEnd = patternEnd; + + /* First element of branch must be unbounded */ + if (branchFirst->max != INT32_MAX) + branchAbsorbable = false; + + /* Count unbounded in this branch - must be exactly 1 */ + if (branchAbsorbable) + { + for (j = branchStart; j < branchEnd; j++) + { + RPRPatternElement *elem = &pattern->elements[j]; + + if (elem->varId == RPR_VARID_END) + { + if (elem->max == INT32_MAX) + branchUnbounded++; + } + else if (elem->varId != RPR_VARID_ALT) + { + if (elem->max == INT32_MAX) + branchUnbounded++; + } + } + + if (branchUnbounded != 1) + branchAbsorbable = false; + } + + /* Set flag on branch's first element if absorbable */ + if (branchAbsorbable) + { + branchFirst->flags |= RPR_ELEM_ABSORBABLE; + pattern->isAbsorbable = true; + } + + branchStart = branchEnd; + } + return; + } + + /* + * Non-ALT pattern: check for absorbability. + * + * Case 1: First element is unbounded (A+ B, etc.) + * Case 2: Top-level unbounded GROUP (A B){2,} - GROUP END at depth 0 + * + * In both cases, must have exactly one unbounded element/group. + */ + { + RPRPatternElement *first = &pattern->elements[0]; + int numUnbounded = 0; + bool hasTopLevelUnboundedGroup = false; + RPRElemIdx topLevelGroupEnd = RPR_ELEMIDX_INVALID; + + /* Count unbounded elements and find top-level unbounded GROUP */ + for (i = 0; i < pattern->numElements - 1; i++) + { + RPRPatternElement *elem = &pattern->elements[i]; + + if (elem->varId == RPR_VARID_END) + { + if (elem->max == INT32_MAX) + { + numUnbounded++; + /* Check if this is a top-level GROUP (depth 0) */ + if (elem->depth == 0) + { + hasTopLevelUnboundedGroup = true; + topLevelGroupEnd = i; + } + } + } + else if (elem->varId != RPR_VARID_ALT) + { + if (elem->max == INT32_MAX) + numUnbounded++; + } + } + + /* Must have exactly one unbounded element/group */ + if (numUnbounded != 1) + return; + + /* + * Case 1: First element is unbounded (e.g., A+ B) + */ + if (first->max == INT32_MAX) + { + first->flags |= RPR_ELEM_ABSORBABLE; + pattern->isAbsorbable = true; + return; + } + + /* + * Case 2: Top-level unbounded GROUP that spans entire pattern. + * e.g., (A B){2,} where GROUP END is at the last position before FIN. + * The entire pattern content is inside this unbounded group. + */ + if (hasTopLevelUnboundedGroup && + topLevelGroupEnd == pattern->numElements - 2) + { + first->flags |= RPR_ELEM_ABSORBABLE; + pattern->isAbsorbable = true; + } + } +} diff --git a/src/backend/parser/Makefile b/src/backend/parser/Makefile index 8c0fe28d63f..f4c7d605fe3 100644 --- a/src/backend/parser/Makefile +++ b/src/backend/parser/Makefile @@ -29,6 +29,7 @@ OBJS = \ parse_oper.o \ parse_param.o \ parse_relation.o \ + parse_rpr.o \ parse_target.o \ parse_type.o \ parse_utilcmd.o \ diff --git a/src/backend/parser/README b/src/backend/parser/README index e26eb437a9f..2baffa9517e 100644 --- a/src/backend/parser/README +++ b/src/backend/parser/README @@ -26,6 +26,7 @@ parse_node.c create nodes for various structures parse_oper.c handle operators in expressions parse_param.c handle Params (for the cases used in the core backend) parse_relation.c support routines for tables and column handling +parse_rpr.c handle Row Pattern Recognition parse_target.c handle the result list of the query parse_type.c support routines for data type handling parse_utilcmd.c parse analysis for utility commands (done at execution time) diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 16443e932ab..ffb950ac61d 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -209,6 +209,7 @@ static void preprocess_pub_all_objtype_list(List *all_objects_list, static void preprocess_pubobj_list(List *pubobjspec_list, core_yyscan_t yyscanner); static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query); +static RPRPatternNode *makeRPRQuantifier(int min, int max, bool reluctant); %} @@ -684,9 +685,10 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query); %type <target> row_pattern_definition %type <node> opt_row_pattern_common_syntax - row_pattern_term + row_pattern row_pattern_alt row_pattern_seq + row_pattern_term row_pattern_primary + row_pattern_quantifier_opt %type <list> row_pattern_definition_list - row_pattern %type <ival> opt_row_pattern_skip_to %type <boolean> opt_row_pattern_initial_or_seek @@ -900,7 +902,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query); %nonassoc IDENT PARTITION RANGE ROWS GROUPS PRECEDING FOLLOWING CUBE ROLLUP SET KEYS OBJECT_P SCALAR VALUE_P WITH WITHOUT PATH AFTER INITIAL SEEK PATTERN_P -%left Op OPERATOR /* multi-character ops and user-defined operators */ +%left Op OPERATOR '|' /* multi-character ops and user-defined operators */ %left '+' '-' %left '*' '/' '%' %left '^' @@ -16838,7 +16840,7 @@ opt_row_pattern_skip_to opt_row_pattern_initial_or_seek RPCommonSyntax *n = makeNode(RPCommonSyntax); n->rpSkipTo = $1; n->initial = $2; - n->rpPatterns = $5; + n->rpPattern = (RPRPatternNode *) $5; n->rpDefs = $8; $$ = (Node *) n; } @@ -16874,28 +16876,253 @@ opt_row_pattern_initial_or_seek: ; row_pattern: - row_pattern_term { $$ = list_make1($1); } - | row_pattern row_pattern_term { $$ = lappend($1, $2); } + row_pattern_alt { $$ = $1; } + ; + +row_pattern_alt: + row_pattern_seq { $$ = $1; } + | row_pattern_alt '|' row_pattern_seq + { + RPRPatternNode *n; + /* If left side is already ALT, append to it */ + if (IsA($1, RPRPatternNode) && + ((RPRPatternNode *) $1)->nodeType == RPR_PATTERN_ALT) + { + n = (RPRPatternNode *) $1; + n->children = lappend(n->children, $3); + $$ = (Node *) n; + } + else + { + n = makeNode(RPRPatternNode); + n->nodeType = RPR_PATTERN_ALT; + n->children = list_make2($1, $3); + n->min = 1; + n->max = 1; + n->location = @1; + $$ = (Node *) n; + } + } + ; + +row_pattern_seq: + row_pattern_term { $$ = $1; } + | row_pattern_seq row_pattern_term + { + RPRPatternNode *n; + /* If left side is already SEQ, append to it */ + if (IsA($1, RPRPatternNode) && + ((RPRPatternNode *) $1)->nodeType == RPR_PATTERN_SEQ) + { + n = (RPRPatternNode *) $1; + n->children = lappend(n->children, $2); + $$ = (Node *) n; + } + else + { + n = makeNode(RPRPatternNode); + n->nodeType = RPR_PATTERN_SEQ; + n->children = list_make2($1, $2); + n->min = 1; + n->max = 1; + n->location = @1; + $$ = (Node *) n; + } + } ; row_pattern_term: - ColId { $$ = (Node *) makeSimpleA_Expr(AEXPR_OP, "", (Node *)makeString($1), NULL, @1); } - | ColId '*' { $$ = (Node *) makeSimpleA_Expr(AEXPR_OP, "*", (Node *)makeString($1), NULL, @1); } - | ColId '+' { $$ = (Node *) makeSimpleA_Expr(AEXPR_OP, "+", (Node *)makeString($1), NULL, @1); } -/* - * '?' quantifier. We cannot write this directly "ColId '?'" because '?' is - * not a "self" token. So we let '?' matches Op first then check if it's '?' - * or not. - */ - | ColId Op + row_pattern_primary row_pattern_quantifier_opt + { + RPRPatternNode *n = (RPRPatternNode *) $1; + RPRPatternNode *q = (RPRPatternNode *) $2; + + n->min = q->min; + n->max = q->max; + n->reluctant = q->reluctant; + $$ = (Node *) n; + } + ; + +row_pattern_primary: + ColId { - if (strcmp("?", $2)) + RPRPatternNode *n = makeNode(RPRPatternNode); + n->nodeType = RPR_PATTERN_VAR; + n->varName = $1; + n->min = 1; + n->max = 1; + n->reluctant = false; + n->children = NIL; + n->location = @1; + $$ = (Node *) n; + } + | '(' row_pattern ')' + { + RPRPatternNode *inner = (RPRPatternNode *) $2; + RPRPatternNode *n = makeNode(RPRPatternNode); + n->nodeType = RPR_PATTERN_GROUP; + n->children = list_make1(inner); + n->min = 1; + n->max = 1; + n->reluctant = false; + n->location = @1; + $$ = (Node *) n; + } + ; + +row_pattern_quantifier_opt: + /* EMPTY */ { $$ = (Node *) makeRPRQuantifier(1, 1, false); } + | '*' { $$ = (Node *) makeRPRQuantifier(0, INT_MAX, false); } + | '+' { $$ = (Node *) makeRPRQuantifier(1, INT_MAX, false); } + | Op + { + /* Handle single Op: ? or reluctant quantifiers *?, +?, ?? */ + if (strcmp($1, "?") == 0) + $$ = (Node *) makeRPRQuantifier(0, 1, false); + else if (strcmp($1, "*?") == 0) + $$ = (Node *) makeRPRQuantifier(0, INT_MAX, true); + else if (strcmp($1, "+?") == 0) + $$ = (Node *) makeRPRQuantifier(1, INT_MAX, true); + else if (strcmp($1, "??") == 0) + $$ = (Node *) makeRPRQuantifier(0, 1, true); + else + ereport(ERROR, + errcode(ERRCODE_SYNTAX_ERROR), + errmsg("unsupported quantifier \"%s\"", $1), + parser_errposition(@1)); + } + /* RELUCTANT quantifiers (when lexer separates tokens) */ + | '*' Op + { + if (strcmp($2, "?") != 0) ereport(ERROR, errcode(ERRCODE_SYNTAX_ERROR), errmsg("unsupported quantifier"), parser_errposition(@2)); - - $$ = (Node *) makeSimpleA_Expr(AEXPR_OP, "?", (Node *)makeString($1), NULL, @1); + $$ = (Node *) makeRPRQuantifier(0, INT_MAX, true); + } + | '+' Op + { + if (strcmp($2, "?") != 0) + ereport(ERROR, + errcode(ERRCODE_SYNTAX_ERROR), + errmsg("unsupported quantifier"), + parser_errposition(@2)); + $$ = (Node *) makeRPRQuantifier(1, INT_MAX, true); + } + | Op Op + { + if (strcmp($1, "?") != 0 || strcmp($2, "?") != 0) + ereport(ERROR, + errcode(ERRCODE_SYNTAX_ERROR), + errmsg("unsupported quantifier"), + parser_errposition(@1)); + $$ = (Node *) makeRPRQuantifier(0, 1, true); + } + /* {n}, {n,}, {,m}, {n,m} quantifiers */ + | '{' Iconst '}' + { + if ($2 < 0 || $2 >= INT_MAX) + ereport(ERROR, + errcode(ERRCODE_SYNTAX_ERROR), + errmsg("quantifier bound must be between 0 and %d", INT_MAX - 1), + parser_errposition(@2)); + $$ = (Node *) makeRPRQuantifier($2, $2, false); + } + | '{' Iconst ',' '}' + { + if ($2 < 0 || $2 >= INT_MAX) + ereport(ERROR, + errcode(ERRCODE_SYNTAX_ERROR), + errmsg("quantifier bound must be between 0 and %d", INT_MAX - 1), + parser_errposition(@2)); + $$ = (Node *) makeRPRQuantifier($2, INT_MAX, false); + } + | '{' ',' Iconst '}' + { + if ($3 < 0 || $3 >= INT_MAX) + ereport(ERROR, + errcode(ERRCODE_SYNTAX_ERROR), + errmsg("quantifier bound must be between 0 and %d", INT_MAX - 1), + parser_errposition(@3)); + $$ = (Node *) makeRPRQuantifier(0, $3, false); + } + | '{' Iconst ',' Iconst '}' + { + if ($2 < 0 || $4 < 0 || $2 >= INT_MAX || $4 >= INT_MAX) + ereport(ERROR, + errcode(ERRCODE_SYNTAX_ERROR), + errmsg("quantifier bound must be between 0 and %d", INT_MAX - 1), + parser_errposition(@2)); + if ($2 > $4) + ereport(ERROR, + errcode(ERRCODE_SYNTAX_ERROR), + errmsg("quantifier minimum bound must not exceed maximum"), + parser_errposition(@2)); + $$ = (Node *) makeRPRQuantifier($2, $4, false); + } + /* Reluctant versions: {n}?, {n,}?, {,m}?, {n,m}? */ + | '{' Iconst '}' Op + { + if (strcmp($4, "?") != 0) + ereport(ERROR, + errcode(ERRCODE_SYNTAX_ERROR), + errmsg("unsupported quantifier"), + parser_errposition(@4)); + if ($2 < 0 || $2 >= INT_MAX) + ereport(ERROR, + errcode(ERRCODE_SYNTAX_ERROR), + errmsg("quantifier bound must be between 0 and %d", INT_MAX - 1), + parser_errposition(@2)); + $$ = (Node *) makeRPRQuantifier($2, $2, true); + } + | '{' Iconst ',' '}' Op + { + if (strcmp($5, "?") != 0) + ereport(ERROR, + errcode(ERRCODE_SYNTAX_ERROR), + errmsg("unsupported quantifier"), + parser_errposition(@5)); + if ($2 < 0 || $2 >= INT_MAX) + ereport(ERROR, + errcode(ERRCODE_SYNTAX_ERROR), + errmsg("quantifier bound must be between 0 and %d", INT_MAX - 1), + parser_errposition(@2)); + $$ = (Node *) makeRPRQuantifier($2, INT_MAX, true); + } + | '{' ',' Iconst '}' Op + { + if (strcmp($5, "?") != 0) + ereport(ERROR, + errcode(ERRCODE_SYNTAX_ERROR), + errmsg("unsupported quantifier"), + parser_errposition(@5)); + if ($3 < 0 || $3 >= INT_MAX) + ereport(ERROR, + errcode(ERRCODE_SYNTAX_ERROR), + errmsg("quantifier bound must be between 0 and %d", INT_MAX - 1), + parser_errposition(@3)); + $$ = (Node *) makeRPRQuantifier(0, $3, true); + } + | '{' Iconst ',' Iconst '}' Op + { + if (strcmp($6, "?") != 0) + ereport(ERROR, + errcode(ERRCODE_SYNTAX_ERROR), + errmsg("unsupported quantifier"), + parser_errposition(@6)); + if ($2 < 0 || $4 < 0 || $2 >= INT_MAX || $4 >= INT_MAX) + ereport(ERROR, + errcode(ERRCODE_SYNTAX_ERROR), + errmsg("quantifier bound must be between 0 and %d", INT_MAX - 1), + parser_errposition(@2)); + if ($2 > $4) + ereport(ERROR, + errcode(ERRCODE_SYNTAX_ERROR), + errmsg("quantifier minimum bound must not exceed maximum"), + parser_errposition(@2)); + $$ = (Node *) makeRPRQuantifier($2, $4, true); } ; @@ -16958,12 +17185,15 @@ MathOp: '+' { $$ = "+"; } | LESS_EQUALS { $$ = "<="; } | GREATER_EQUALS { $$ = ">="; } | NOT_EQUALS { $$ = "<>"; } + | '|' { $$ = "|"; } ; qual_Op: Op { $$ = list_make1(makeString($1)); } | OPERATOR '(' any_operator ')' { $$ = $3; } + | '|' + { $$ = list_make1(makeString("|")); } ; qual_all_Op: @@ -20085,6 +20315,21 @@ makeRecursiveViewSelect(char *relname, List *aliases, Node *query) return (Node *) s; } +/* + * makeRPRQuantifier + * Create an RPRPatternNode with specified quantifier bounds. + */ +static RPRPatternNode * +makeRPRQuantifier(int min, int max, bool reluctant) +{ + RPRPatternNode *n = makeNode(RPRPatternNode); + + n->min = min; + n->max = max; + n->reluctant = reluctant; + return n; +} + /* parser_init() * Initialize to parse one query string */ diff --git a/src/backend/parser/meson.build b/src/backend/parser/meson.build index 924ee87a453..a07102b37a0 100644 --- a/src/backend/parser/meson.build +++ b/src/backend/parser/meson.build @@ -16,6 +16,7 @@ backend_sources += files( 'parse_oper.c', 'parse_param.c', 'parse_relation.c', + 'parse_rpr.c', 'parse_target.c', 'parse_type.c', 'parse_utilcmd.c', diff --git a/src/backend/parser/parse_clause.c b/src/backend/parser/parse_clause.c index f450cb3310c..f98ac7cfcb5 100644 --- a/src/backend/parser/parse_clause.c +++ b/src/backend/parser/parse_clause.c @@ -37,6 +37,7 @@ #include "parser/parse_func.h" #include "parser/parse_oper.h" #include "parser/parse_relation.h" +#include "parser/parse_rpr.h" #include "parser/parse_target.h" #include "parser/parse_type.h" #include "parser/parser.h" @@ -84,8 +85,6 @@ static void checkExprIsVarFree(ParseState *pstate, Node *n, const char *constructName); static TargetEntry *findTargetlistEntrySQL92(ParseState *pstate, Node *node, List **tlist, ParseExprKind exprKind); -static TargetEntry *findTargetlistEntrySQL99(ParseState *pstate, Node *node, - List **tlist, ParseExprKind exprKind); static int get_matching_location(int sortgroupref, List *sortgrouprefs, List *exprs); static List *resolve_unique_index_expr(ParseState *pstate, InferClause *infer, @@ -96,12 +95,6 @@ static WindowClause *findWindowClause(List *wclist, const char *name); static Node *transformFrameOffset(ParseState *pstate, int frameOptions, Oid rangeopfamily, Oid rangeopcintype, Oid *inRangeFunc, Node *clause); -static void transformRPR(ParseState *pstate, WindowClause *wc, WindowDef *windef, - List **targetlist); -static List *transformDefineClause(ParseState *pstate, WindowClause *wc, WindowDef *windef, - List **targetlist); -static void transformPatternClause(ParseState *pstate, WindowClause *wc, - WindowDef *windef); /* * transformFromClause - @@ -2173,7 +2166,7 @@ findTargetlistEntrySQL92(ParseState *pstate, Node *node, List **tlist, * tlist the target list (passed by reference so we can append to it) * exprKind identifies clause type being processed */ -static TargetEntry * +TargetEntry * findTargetlistEntrySQL99(ParseState *pstate, Node *node, List **tlist, ParseExprKind exprKind) { @@ -3900,254 +3893,3 @@ transformFrameOffset(ParseState *pstate, int frameOptions, return node; } - -/* - * transformRPR - * Process Row Pattern Recognition related clauses - */ -static void -transformRPR(ParseState *pstate, WindowClause *wc, WindowDef *windef, - List **targetlist) -{ - /* - * Window definition exists? - */ - if (windef == NULL) - return; - - /* - * Row Pattern Common Syntax clause exists? - */ - if (windef->rpCommonSyntax == NULL) - return; - - /* Check Frame option. Frame must start at current row */ - if ((wc->frameOptions & FRAMEOPTION_START_CURRENT_ROW) == 0) - ereport(ERROR, - (errcode(ERRCODE_SYNTAX_ERROR), - errmsg("FRAME must start at current row when row pattern recognition is used"))); - - /* Transform AFTER MACH SKIP TO clause */ - wc->rpSkipTo = windef->rpCommonSyntax->rpSkipTo; - - /* Transform SEEK or INITIAL clause */ - wc->initial = windef->rpCommonSyntax->initial; - - /* Transform DEFINE clause into list of TargetEntry's */ - wc->defineClause = transformDefineClause(pstate, wc, windef, targetlist); - - /* Check PATTERN clause and copy to patternClause */ - transformPatternClause(pstate, wc, windef); -} - -/* - * transformDefineClause - * Process DEFINE clause and transform ResTarget into list of - * TargetEntry. - * - * XXX we only support column reference in row pattern definition search - * condition, e.g. "price". <row pattern definition variable name>.<column - * reference> is not supported, e.g. "A.price". - */ -static List * -transformDefineClause(ParseState *pstate, WindowClause *wc, WindowDef *windef, - List **targetlist) -{ - /* DEFINE variable name initials */ - static const char *defineVariableInitials = "abcdefghijklmnopqrstuvwxyz"; - - ListCell *lc, - *l; - ResTarget *restarget, - *r; - List *restargets; - List *defineClause; - char *name; - int initialLen; - int numinitials; - - /* - * If Row Definition Common Syntax exists, DEFINE clause must exist. (the - * raw parser should have already checked it.) - */ - Assert(windef->rpCommonSyntax->rpDefs != NULL); - - /* - * Check and add "A AS A IS TRUE" if pattern variable is missing in DEFINE - * per the SQL standard. - */ - restargets = NIL; - foreach(lc, windef->rpCommonSyntax->rpPatterns) - { - A_Expr *a; - bool found = false; - - if (!IsA(lfirst(lc), A_Expr)) - ereport(ERROR, - errmsg("node type is not A_Expr")); - - a = (A_Expr *) lfirst(lc); - name = strVal(a->lexpr); - - foreach(l, windef->rpCommonSyntax->rpDefs) - { - restarget = (ResTarget *) lfirst(l); - - if (!strcmp(restarget->name, name)) - { - found = true; - break; - } - } - - if (!found) - { - /* - * "name" is missing. So create "name AS name IS TRUE" ResTarget - * node and add it to the temporary list. - */ - A_Const *n; - - restarget = makeNode(ResTarget); - n = makeNode(A_Const); - n->val.boolval.type = T_Boolean; - n->val.boolval.boolval = true; - n->location = -1; - restarget->name = pstrdup(name); - restarget->indirection = NIL; - restarget->val = (Node *) n; - restarget->location = -1; - restargets = lappend((List *) restargets, restarget); - } - } - - if (list_length(restargets) >= 1) - { - /* add missing DEFINEs */ - windef->rpCommonSyntax->rpDefs = - list_concat(windef->rpCommonSyntax->rpDefs, restargets); - list_free(restargets); - } - - /* - * Check for duplicate row pattern definition variables. The standard - * requires that no two row pattern definition variable names shall be - * equivalent. - */ - restargets = NIL; - numinitials = 0; - initialLen = strlen(defineVariableInitials); - foreach(lc, windef->rpCommonSyntax->rpDefs) - { - char initial[2]; - - restarget = (ResTarget *) lfirst(lc); - name = restarget->name; - - /* - * Add DEFINE expression (Restarget->val) to the targetlist as a - * TargetEntry if it does not exist yet. Planner will add the column - * ref var node to the outer plan's target list later on. This makes - * DEFINE expression could access the outer tuple while evaluating - * PATTERN. - * - * XXX: adding whole expressions of DEFINE to the plan.targetlist is - * not so good, because it's not necessary to evalute the expression - * in the target list while running the plan. We should extract the - * var nodes only then add them to the plan.targetlist. - */ - findTargetlistEntrySQL99(pstate, (Node *) restarget->val, - targetlist, EXPR_KIND_RPR_DEFINE); - - /* - * Make sure that the row pattern definition search condition is a - * boolean expression. - */ - transformWhereClause(pstate, restarget->val, - EXPR_KIND_RPR_DEFINE, "DEFINE"); - - foreach(l, restargets) - { - char *n; - - r = (ResTarget *) lfirst(l); - n = r->name; - - if (!strcmp(n, name)) - ereport(ERROR, - (errcode(ERRCODE_SYNTAX_ERROR), - errmsg("row pattern definition variable name \"%s\" appears more than once in DEFINE clause", - name), - parser_errposition(pstate, exprLocation((Node *) r)))); - } - - /* - * Create list of row pattern DEFINE variable name's initial. We - * assign [a-z] to them (up to 26 variable names are allowed). - */ - if (numinitials >= initialLen) - { - ereport(ERROR, - (errcode(ERRCODE_SYNTAX_ERROR), - errmsg("number of row pattern definition variable names exceeds %d", - initialLen), - parser_errposition(pstate, - exprLocation((Node *) restarget)))); - } - initial[0] = defineVariableInitials[numinitials++]; - initial[1] = '\0'; - wc->defineInitial = lappend(wc->defineInitial, - makeString(pstrdup(initial))); - - restargets = lappend(restargets, restarget); - } - list_free(restargets); - - /* turns a list of ResTarget's into a list of TargetEntry's */ - defineClause = transformTargetList(pstate, windef->rpCommonSyntax->rpDefs, - EXPR_KIND_RPR_DEFINE); - - /* mark column origins */ - markTargetListOrigins(pstate, defineClause); - - /* mark all nodes in the DEFINE clause tree with collation information */ - assign_expr_collations(pstate, (Node *) defineClause); - - return defineClause; -} - -/* - * transformPatternClause - * Process PATTERN clause and return PATTERN clause in the raw parse tree - * - * windef->rpCommonSyntax must exist. - */ -static void -transformPatternClause(ParseState *pstate, WindowClause *wc, - WindowDef *windef) -{ - ListCell *lc; - - Assert(windef->rpCommonSyntax != NULL); - - wc->patternVariable = NIL; - wc->patternRegexp = NIL; - foreach(lc, windef->rpCommonSyntax->rpPatterns) - { - A_Expr *a; - char *name; - char *regexp; - - if (!IsA(lfirst(lc), A_Expr)) - ereport(ERROR, - errmsg("node type is not A_Expr")); - - a = (A_Expr *) lfirst(lc); - name = strVal(a->lexpr); - - wc->patternVariable = lappend(wc->patternVariable, makeString(pstrdup(name))); - regexp = strVal(lfirst(list_head(a->name))); - - wc->patternRegexp = lappend(wc->patternRegexp, makeString(pstrdup(regexp))); - } -} diff --git a/src/backend/parser/parse_rpr.c b/src/backend/parser/parse_rpr.c new file mode 100644 index 00000000000..de648293b27 --- /dev/null +++ b/src/backend/parser/parse_rpr.c @@ -0,0 +1,340 @@ +/*------------------------------------------------------------------------- + * + * parse_rpr.c + * handle Row Pattern Recognition in parser + * + * Portions Copyright (c) 1996-2025, PostgreSQL Global Development Group + * Portions Copyright (c) 1994, Regents of the University of California + * + * + * IDENTIFICATION + * src/backend/parser/parse_rpr.c + * + *------------------------------------------------------------------------- + */ + +#include "postgres.h" + +#include "nodes/makefuncs.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_clause.h" +#include "parser/parse_collate.h" +#include "parser/parse_expr.h" +#include "parser/parse_rpr.h" +#include "parser/parse_target.h" + +/* DEFINE variable name initials */ +static const char *defineVariableInitials = "abcdefghijklmnopqrstuvwxyz"; + +/* Forward declarations */ +static void collectRPRPatternVarNames(RPRPatternNode *node, List **varNames); +static List *transformDefineClause(ParseState *pstate, WindowClause *wc, + WindowDef *windef, List **targetlist); +static void transformPatternClause(ParseState *pstate, WindowClause *wc, + WindowDef *windef); +static RPRPatternNode *copyRPRPatternNode(RPRPatternNode *node); + +/* + * transformRPR + * Process Row Pattern Recognition related clauses + */ +void +transformRPR(ParseState *pstate, WindowClause *wc, WindowDef *windef, + List **targetlist) +{ + /* + * Window definition exists? + */ + if (windef == NULL) + return; + + /* + * Row Pattern Common Syntax clause exists? + */ + if (windef->rpCommonSyntax == NULL) + return; + + /* Check Frame option. Frame must start at current row */ + if ((wc->frameOptions & FRAMEOPTION_START_CURRENT_ROW) == 0) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("FRAME must start at current row when row pattern recognition is used"))); + + /* Transform AFTER MATCH SKIP TO clause */ + wc->rpSkipTo = windef->rpCommonSyntax->rpSkipTo; + + /* Transform SEEK or INITIAL clause */ + wc->initial = windef->rpCommonSyntax->initial; + + /* Transform DEFINE clause into list of TargetEntry's */ + wc->defineClause = transformDefineClause(pstate, wc, windef, targetlist); + + /* Check PATTERN clause and copy to patternClause */ + transformPatternClause(pstate, wc, windef); +} + +/* + * collectRPRPatternVarNames + * Collect all variable names from RPRPatternNode tree. + */ +static void +collectRPRPatternVarNames(RPRPatternNode *node, List **varNames) +{ + ListCell *lc; + + if (node == NULL) + return; + + switch (node->nodeType) + { + case RPR_PATTERN_VAR: + /* Add variable name if not already in list */ + { + bool found = false; + + foreach(lc, *varNames) + { + if (strcmp(strVal(lfirst(lc)), node->varName) == 0) + { + found = true; + break; + } + } + if (!found) + *varNames = lappend(*varNames, makeString(pstrdup(node->varName))); + } + break; + + case RPR_PATTERN_SEQ: + case RPR_PATTERN_ALT: + case RPR_PATTERN_GROUP: + /* Recurse into children */ + foreach(lc, node->children) + { + collectRPRPatternVarNames((RPRPatternNode *) lfirst(lc), varNames); + } + break; + } +} + +/* + * transformDefineClause + * Process DEFINE clause and transform ResTarget into list of + * TargetEntry. + * + * XXX we only support column reference in row pattern definition search + * condition, e.g. "price". <row pattern definition variable name>.<column + * reference> is not supported, e.g. "A.price". + */ +static List * +transformDefineClause(ParseState *pstate, WindowClause *wc, WindowDef *windef, + List **targetlist) +{ + ListCell *lc, + *l; + ResTarget *restarget, + *r; + List *restargets; + List *defineClause; + char *name; + int initialLen; + int numinitials; + List *patternVarNames = NIL; + + /* + * If Row Definition Common Syntax exists, DEFINE clause must exist. (the + * raw parser should have already checked it.) + */ + Assert(windef->rpCommonSyntax->rpDefs != NULL); + + /* + * Collect all pattern variable names from the pattern tree. + */ + collectRPRPatternVarNames(windef->rpCommonSyntax->rpPattern, &patternVarNames); + + /* + * Check and add "A AS A IS TRUE" if pattern variable is missing in DEFINE + * per the SQL standard. + */ + restargets = NIL; + foreach(lc, patternVarNames) + { + bool found = false; + + name = strVal(lfirst(lc)); + + foreach(l, windef->rpCommonSyntax->rpDefs) + { + restarget = (ResTarget *) lfirst(l); + + if (!strcmp(restarget->name, name)) + { + found = true; + break; + } + } + + if (!found) + { + /* + * "name" is missing. So create "name AS name IS TRUE" ResTarget + * node and add it to the temporary list. + */ + A_Const *n; + + restarget = makeNode(ResTarget); + n = makeNode(A_Const); + n->val.boolval.type = T_Boolean; + n->val.boolval.boolval = true; + n->location = -1; + restarget->name = pstrdup(name); + restarget->indirection = NIL; + restarget->val = (Node *) n; + restarget->location = -1; + restargets = lappend((List *) restargets, restarget); + } + } + + if (list_length(restargets) >= 1) + { + /* add missing DEFINEs */ + windef->rpCommonSyntax->rpDefs = + list_concat(windef->rpCommonSyntax->rpDefs, restargets); + list_free(restargets); + } + + /* + * Check for duplicate row pattern definition variables. The standard + * requires that no two row pattern definition variable names shall be + * equivalent. + */ + restargets = NIL; + numinitials = 0; + initialLen = strlen(defineVariableInitials); + foreach(lc, windef->rpCommonSyntax->rpDefs) + { + char initial[2]; + + restarget = (ResTarget *) lfirst(lc); + name = restarget->name; + + /* + * Add DEFINE expression (Restarget->val) to the targetlist as a + * TargetEntry if it does not exist yet. Planner will add the column + * ref var node to the outer plan's target list later on. This makes + * DEFINE expression could access the outer tuple while evaluating + * PATTERN. + * + * XXX: adding whole expressions of DEFINE to the plan.targetlist is + * not so good, because it's not necessary to evalute the expression + * in the target list while running the plan. We should extract the + * var nodes only then add them to the plan.targetlist. + */ + findTargetlistEntrySQL99(pstate, (Node *) restarget->val, + targetlist, EXPR_KIND_RPR_DEFINE); + + /* + * Make sure that the row pattern definition search condition is a + * boolean expression. + */ + transformWhereClause(pstate, restarget->val, + EXPR_KIND_RPR_DEFINE, "DEFINE"); + + foreach(l, restargets) + { + char *n; + + r = (ResTarget *) lfirst(l); + n = r->name; + + if (!strcmp(n, name)) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("row pattern definition variable name \"%s\" appears more than once in DEFINE clause", + name), + parser_errposition(pstate, exprLocation((Node *) r)))); + } + + /* + * Create list of row pattern DEFINE variable name's initial. We + * assign [a-z] to them (up to 26 variable names are allowed). + */ + if (numinitials >= initialLen) + { + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("number of row pattern definition variable names exceeds %d", + initialLen), + parser_errposition(pstate, + exprLocation((Node *) restarget)))); + } + initial[0] = defineVariableInitials[numinitials++]; + initial[1] = '\0'; + wc->defineInitial = lappend(wc->defineInitial, + makeString(pstrdup(initial))); + + restargets = lappend(restargets, restarget); + } + list_free(restargets); + + /* turns a list of ResTarget's into a list of TargetEntry's */ + defineClause = transformTargetList(pstate, windef->rpCommonSyntax->rpDefs, + EXPR_KIND_RPR_DEFINE); + + /* mark column origins */ + markTargetListOrigins(pstate, defineClause); + + /* mark all nodes in the DEFINE clause tree with collation information */ + assign_expr_collations(pstate, (Node *) defineClause); + + return defineClause; +} + +/* + * transformPatternClause + * Process PATTERN clause and return PATTERN clause in the raw parse tree + * + * windef->rpCommonSyntax must exist. + * + * The original (unoptimized) pattern is stored for deparsing (pg_get_viewdef). + * Optimization happens later in the planner phase (buildRPRPattern). + */ +static void +transformPatternClause(ParseState *pstate, WindowClause *wc, + WindowDef *windef) +{ + Assert(windef->rpCommonSyntax != NULL); + + /* Store original AST for deparsing (no optimization here) */ + wc->rpPattern = copyRPRPatternNode(windef->rpCommonSyntax->rpPattern); +} + +/* + * copyRPRPatternNode + * Deep copy an RPRPatternNode tree. + */ +static RPRPatternNode * +copyRPRPatternNode(RPRPatternNode *node) +{ + RPRPatternNode *copy; + ListCell *lc; + + if (node == NULL) + return NULL; + + copy = makeNode(RPRPatternNode); + copy->nodeType = node->nodeType; + copy->varName = node->varName ? pstrdup(node->varName) : NULL; + copy->min = node->min; + copy->max = node->max; + copy->reluctant = node->reluctant; + copy->children = NIL; + + foreach(lc, node->children) + { + copy->children = lappend(copy->children, + copyRPRPatternNode(lfirst(lc))); + } + + return copy; +} diff --git a/src/backend/parser/scan.l b/src/backend/parser/scan.l index 731d584106d..f6ca2406146 100644 --- a/src/backend/parser/scan.l +++ b/src/backend/parser/scan.l @@ -363,7 +363,7 @@ not_equals "!=" * If you change either set, adjust the character lists appearing in the * rule for "operator"! */ -self [,()\[\].;\:\+\-\*\/\%\^\<\>\=] +self [,()\[\].;\:\+\-\*\/\%\^\<\>\=\|] op_chars [\~\!\@\#\^\&\|\`\?\+\-\*\/\%\<\>\=] operator {op_chars}+ @@ -955,7 +955,7 @@ other . * that the "self" rule would have. */ if (nchars == 1 && - strchr(",()[].;:+-*/%^<>=", yytext[0])) + strchr(",()[].;:+-*/%^<>=|", yytext[0])) return yytext[0]; /* * Likewise, if what we have left is two chars, and diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c index 5ce0781edf4..b8a3ee2915b 100644 --- a/src/backend/utils/adt/ruleutils.c +++ b/src/backend/utils/adt/ruleutils.c @@ -438,10 +438,12 @@ static void get_rule_groupingset(GroupingSet *gset, List *targetlist, bool omit_parens, deparse_context *context); static void get_rule_orderby(List *orderList, List *targetList, bool force_colno, deparse_context *context); -static void get_rule_pattern(List *patternVariable, List *patternRegexp, - bool force_colno, deparse_context *context); -static void get_rule_define(List *defineClause, List *patternVariables, - bool force_colno, deparse_context *context); +static void append_pattern_quantifier(StringInfo buf, RPRPatternNode *node); +static void get_rule_pattern_node(RPRPatternNode *node, deparse_context *context); +static void get_rule_pattern(RPRPatternNode *rpPattern, bool force_colno, + deparse_context *context); +static void get_rule_define(List *defineClause, bool force_colno, + deparse_context *context); static void get_rule_windowclause(Query *query, deparse_context *context); static void get_rule_windowspec(WindowClause *wc, List *targetList, deparse_context *context); @@ -6749,37 +6751,101 @@ get_rule_orderby(List *orderList, List *targetList, } /* - * Display a PATTERN clause. + * Helper function to append quantifier string for pattern node */ static void -get_rule_pattern(List *patternVariable, List *patternRegexp, - bool force_colno, deparse_context *context) +append_pattern_quantifier(StringInfo buf, RPRPatternNode *node) +{ + bool has_quantifier = true; + + if (node->min == 1 && node->max == 1) + { + /* {1,1} = no quantifier */ + has_quantifier = false; + } + else if (node->min == 0 && node->max == INT_MAX) + appendStringInfoChar(buf, '*'); + else if (node->min == 1 && node->max == INT_MAX) + appendStringInfoChar(buf, '+'); + else if (node->min == 0 && node->max == 1) + appendStringInfoChar(buf, '?'); + else if (node->max == INT_MAX) + appendStringInfo(buf, "{%d,}", node->min); + else if (node->min == node->max) + appendStringInfo(buf, "{%d}", node->min); + else + appendStringInfo(buf, "{%d,%d}", node->min, node->max); + + if (node->reluctant && has_quantifier) + appendStringInfoChar(buf, '?'); +} + +/* + * Recursive helper to display RPRPatternNode tree + */ +static void +get_rule_pattern_node(RPRPatternNode *node, deparse_context *context) { StringInfo buf = context->buf; + ListCell *lc; const char *sep; - ListCell *lc_var, - *lc_reg = list_head(patternRegexp); - sep = ""; - appendStringInfoChar(buf, '('); - foreach(lc_var, patternVariable) - { - char *variable = strVal((String *) lfirst(lc_var)); - char *regexp = NULL; + if (node == NULL) + return; - if (lc_reg != NULL) - { - regexp = strVal((String *) lfirst(lc_reg)); + switch (node->nodeType) + { + case RPR_PATTERN_VAR: + appendStringInfoString(buf, node->varName); + append_pattern_quantifier(buf, node); + break; - lc_reg = lnext(patternRegexp, lc_reg); - } + case RPR_PATTERN_SEQ: + sep = ""; + foreach(lc, node->children) + { + appendStringInfoString(buf, sep); + get_rule_pattern_node((RPRPatternNode *) lfirst(lc), context); + sep = " "; + } + break; - appendStringInfo(buf, "%s%s", sep, variable); - if (regexp != NULL) - appendStringInfoString(buf, regexp); + case RPR_PATTERN_ALT: + sep = ""; + foreach(lc, node->children) + { + appendStringInfoString(buf, sep); + get_rule_pattern_node((RPRPatternNode *) lfirst(lc), context); + sep = " | "; + } + break; - sep = " "; + case RPR_PATTERN_GROUP: + appendStringInfoChar(buf, '('); + sep = ""; + foreach(lc, node->children) + { + appendStringInfoString(buf, sep); + get_rule_pattern_node((RPRPatternNode *) lfirst(lc), context); + sep = " "; + } + appendStringInfoChar(buf, ')'); + append_pattern_quantifier(buf, node); + break; } +} + +/* + * Display a PATTERN clause. + */ +static void +get_rule_pattern(RPRPatternNode *rpPattern, bool force_colno, + deparse_context *context) +{ + StringInfo buf = context->buf; + + appendStringInfoChar(buf, '('); + get_rule_pattern_node(rpPattern, context); appendStringInfoChar(buf, ')'); } @@ -6787,8 +6853,7 @@ get_rule_pattern(List *patternVariable, List *patternRegexp, * Display a DEFINE clause. */ static void -get_rule_define(List *defineClause, List *patternVariables, - bool force_colno, deparse_context *context) +get_rule_define(List *defineClause, bool force_colno, deparse_context *context) { StringInfo buf = context->buf; const char *sep; @@ -6922,13 +6987,12 @@ get_rule_windowspec(WindowClause *wc, List *targetList, appendStringInfoString(buf, "\n INITIAL"); needspace = true; } - if (wc->patternVariable) + if (wc->rpPattern) { if (needspace) appendStringInfoChar(buf, ' '); appendStringInfoString(buf, "\n PATTERN "); - get_rule_pattern(wc->patternVariable, wc->patternRegexp, - false, context); + get_rule_pattern(wc->rpPattern, false, context); needspace = true; } @@ -6937,8 +7001,7 @@ get_rule_windowspec(WindowClause *wc, List *targetList, if (needspace) appendStringInfoChar(buf, ' '); appendStringInfoString(buf, "\n DEFINE\n"); - get_rule_define(wc->defineClause, wc->patternVariable, - false, context); + get_rule_define(wc->defineClause, false, context); appendStringInfoChar(buf, ' '); } diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h index b87077b47cb..714c0b62700 100644 --- a/src/include/nodes/execnodes.h +++ b/src/include/nodes/execnodes.h @@ -2634,30 +2634,32 @@ typedef enum WindowAggStatus #define RF_UNMATCHED 3 /* - * Allowed PATTERN variables positions. - * Used in RPR. - * - * pos represents the pattern variable defined order in DEFINE caluase. For - * example. "DEFINE START..., UP..., DOWN ..." and "PATTERN START UP DOWN UP" - * will create: - * VariablePos[0].pos[0] = 0; START - * VariablePos[1].pos[0] = 1; UP - * VariablePos[1].pos[1] = 3; UP - * VariablePos[2].pos[0] = 2; DOWN - * - * Note that UP has two pos because UP appears in PATTERN twice. - * - * By using this strucrture, we can know which pattern variable can be followed - * by which pattern variable(s). For example, START can be followed by UP and - * DOWN since START's pos is 0, and UP's pos is 1 or 3, DOWN's pos is 2. - * DOWN can be followed by UP since UP's pos is either 1 or 3. + * RPRNFAState - single NFA state for pattern matching * + * counts[] tracks repetition counts at each nesting depth. + * altPriority tracks lexical order for alternation (lower = earlier alternative). */ -#define NUM_ALPHABETS 26 /* we allow [a-z] variable initials */ -typedef struct VariablePos +typedef struct RPRNFAState { - int pos[NUM_ALPHABETS]; /* postions in PATTERN */ -} VariablePos; + struct RPRNFAState *next; /* next state in linked list */ + int16 elemIdx; /* current pattern element index */ + int16 altPriority; /* lexical order priority (lower = preferred) */ + int16 counts[FLEXIBLE_ARRAY_MEMBER]; /* repetition counts by depth */ +} RPRNFAState; + +/* + * RPRNFAContext - context for NFA pattern matching execution + */ +typedef struct RPRNFAContext +{ + struct RPRNFAContext *next; /* next context in linked list */ + struct RPRNFAContext *prev; /* previous context (for reverse traversal) */ + RPRNFAState *states; /* active states (linked list) */ + + int64 matchStartRow; /* row where match started */ + int64 matchEndRow; /* row where match ended (-1 = no match) */ + RPRNFAState *matchedState; /* FIN state for greedy fallback (cloned) */ +} RPRNFAContext; typedef struct WindowAggState { @@ -2720,18 +2722,21 @@ typedef struct WindowAggState /* these fields are used in Row pattern recognition: */ RPSkipTo rpSkipTo; /* Row Pattern Skip To type */ - List *patternVariableList; /* list of row pattern variables names - * (list of String) */ - List *patternRegexpList; /* list of row pattern regular expressions - * ('+' or ''. list of String) */ + struct RPRPattern *rpPattern; /* compiled pattern for NFA execution */ List *defineVariableList; /* list of row pattern definition * variables (list of String) */ List *defineClauseList; /* expression for row pattern definition * search conditions ExprState list */ List *defineInitial; /* list of row pattern definition variable * initials (list of String) */ - VariablePos *variable_pos; /* list of pattern variable positions */ - StringInfo pattern_str; /* PATTERN initials */ + RPRNFAContext *nfaContext; /* active matching contexts (head) */ + RPRNFAContext *nfaContextTail; /* tail of active contexts (for reverse traversal) */ + RPRNFAContext *nfaContextPending; /* matched but awaiting earlier starts */ + RPRNFAContext *nfaContextFree; /* recycled NFA context nodes */ + RPRNFAState *nfaStateFree; /* recycled NFA state nodes */ + Size nfaStateSize; /* pre-calculated RPRNFAState size */ + bool *nfaVarMatched; /* per-row cache: varMatched[varId] for varId < numDefines */ + int64 nfaLastProcessedRow; /* last row processed by NFA (-1 = none) */ MemoryContext partcontext; /* context for partition-lifespan data */ MemoryContext aggcontext; /* shared context for aggregate working data */ @@ -2772,10 +2777,6 @@ typedef struct WindowAggState */ char *reduced_frame_map; int64 alloc_sz; /* size of the map */ - - /* regular expression compiled result cache. Used for RPR. */ - char *patbuf; /* pattern to be compiled */ - regex_t preg; /* compiled re pattern */ } WindowAggState; /* ---------------- diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index d6da01d8620..094f35248fd 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -588,9 +588,34 @@ typedef enum RPSkipTo ST_PAST_LAST_ROW, /* SKIP TO PAST LAST ROW */ } RPSkipTo; +/* + * RPRPatternNodeType - Row Pattern Recognition pattern node types + */ +typedef enum RPRPatternNodeType +{ + RPR_PATTERN_VAR, /* variable reference */ + RPR_PATTERN_SEQ, /* sequence (concatenation) */ + RPR_PATTERN_ALT, /* alternation (|) */ + RPR_PATTERN_GROUP /* group (parentheses) */ +} RPRPatternNodeType; + +/* + * RPRPatternNode - Row Pattern Recognition pattern AST node + */ +typedef struct RPRPatternNode +{ + NodeTag type; /* T_RPRPatternNode */ + RPRPatternNodeType nodeType; /* VAR, SEQ, ALT, GROUP */ + int min; /* minimum repetitions (0 for *, ?) */ + int max; /* maximum repetitions (INT_MAX for *, +) */ + bool reluctant; /* true for *?, +?, ?? */ + ParseLoc location; /* token location, or -1 */ + char *varName; /* VAR: variable name */ + List *children; /* SEQ, ALT, GROUP: child nodes */ +} RPRPatternNode; + /* * RowPatternCommonSyntax - raw representation of row pattern common syntax - * */ typedef struct RPCommonSyntax { @@ -598,7 +623,7 @@ typedef struct RPCommonSyntax RPSkipTo rpSkipTo; /* Row Pattern AFTER MATCH SKIP type */ bool initial; /* true if <row pattern initial or seek> is * initial */ - List *rpPatterns; /* PATTERN variables (list of A_Expr) */ + RPRPatternNode *rpPattern; /* PATTERN clause AST */ List *rpDefs; /* row pattern definitions clause (list of * ResTarget) */ } RPCommonSyntax; @@ -1616,8 +1641,8 @@ typedef struct GroupingSet * * "defineClause" is Row Pattern Recognition DEFINE clause (list of * TargetEntry). TargetEntry.resname represents row pattern definition - * variable name. "patternVariable" and "patternRegexp" represents PATTERN - * clause. + * variable name. "rpPattern" represents PATTERN clause as an AST tree + * (RPRPatternNode). * * The information relevant for the query jumbling is the partition clause * type and its bounds. @@ -1656,14 +1681,8 @@ typedef struct WindowClause List *defineClause; /* Row Pattern DEFINE variable initial names (list of String) */ List *defineInitial; - /* Row Pattern PATTERN variable name (list of String) */ - List *patternVariable; - - /* - * Row Pattern PATTERN regular expression quantifier ('+' or ''. list of - * String) - */ - List *patternRegexp; + /* Row Pattern PATTERN clause AST */ + RPRPatternNode *rpPattern; } WindowClause; /* diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h index 8edc7ead7e3..8d9097a8f4e 100644 --- a/src/include/nodes/plannodes.h +++ b/src/include/nodes/plannodes.h @@ -1224,6 +1224,69 @@ typedef struct Agg List *chain; } Agg; +/* ---------------- + * Row Pattern Recognition compiled pattern types + * ---------------- + */ + +/* Type definitions for RPR pattern elements */ +typedef uint8 RPRVarId; /* pattern variable ID */ +typedef uint8 RPRElemFlags; /* element flags */ +typedef uint8 RPRDepth; /* group nesting depth */ +typedef int32 RPRQuantity; /* quantifier min/max */ +typedef int16 RPRElemIdx; /* element array index */ + +/* + * RPRPatternElement - flat element for NFA pattern matching (16 bytes) + * + * Layout optimized for alignment (no padding holes): + * varId(1) + depth(1) + flags(1) + reserved(1) + min(4) + max(4) + next(2) + jump(2) + */ +typedef struct RPRPatternElement +{ + RPRVarId varId; /* variable ID, or special value for control */ + RPRDepth depth; /* group nesting depth */ + RPRElemFlags flags; /* flags (reluctant, etc.) */ + uint8 reserved; /* reserved padding byte */ + RPRQuantity min; /* quantifier minimum */ + RPRQuantity max; /* quantifier maximum */ + RPRElemIdx next; /* next element index */ + RPRElemIdx jump; /* jump target (for ALT/GROUP) */ +} RPRPatternElement; + +/* + * RPRPattern - compiled pattern for NFA execution + * + * Requires custom copy/out/read functions due to elements array. + */ +typedef struct RPRPattern +{ + pg_node_attr(custom_copy_equal, custom_read_write) + + NodeTag type; /* T_RPRPattern */ + int numVars; /* number of pattern variables */ + char **varNames; /* array of variable names (DEFINE order first) */ + RPRDepth maxDepth; /* maximum group nesting depth */ + int numElements; /* number of elements */ + RPRPatternElement *elements; /* array of pattern elements */ + + /* + * Context absorption optimization. + * + * Absorption is only safe when later matches are guaranteed to be + * suffixes of earlier matches. This requires simple pattern structure: + * + * Case 1: No ALT, single unbounded element (A+, (A B)+) + * Case 2: Top-level ALT with each branch being single unbounded (A+ | B+) + * + * Complex patterns like A B (A B)+ could theoretically be transformed to + * (A B){2,} for absorption, but this changes lexical order and is not + * implemented. Similarly, (A|B)+ cannot be absorbed because different + * start positions produce different match contents (not suffix relation). + */ + bool isAbsorbable; /* true if pattern supports context absorption */ +} RPRPattern; + /* ---------------- * window aggregate node * ---------------- @@ -1297,14 +1360,8 @@ typedef struct WindowAgg /* Row Pattern Recognition AFTER MATCH SKIP clause */ RPSkipTo rpSkipTo; /* Row Pattern Skip To type */ - /* Row Pattern PATTERN variable name (list of String) */ - List *patternVariable; - - /* - * Row Pattern RPATTERN regular expression quantifier ('+' or ''. list of - * String) - */ - List *patternRegexp; + /* Compiled Row Pattern for NFA execution */ + struct RPRPattern *rpPattern; /* Row Pattern DEFINE clause (list of TargetEntry) */ List *defineClause; diff --git a/src/include/optimizer/rpr.h b/src/include/optimizer/rpr.h new file mode 100644 index 00000000000..95324e7a343 --- /dev/null +++ b/src/include/optimizer/rpr.h @@ -0,0 +1,46 @@ +/*------------------------------------------------------------------------- + * + * rpr.h + * Row Pattern Recognition pattern compilation for planner + * + * Portions Copyright (c) 1996-2025, PostgreSQL Global Development Group + * Portions Copyright (c) 1994, Regents of the University of California + * + * src/include/optimizer/rpr.h + * + *------------------------------------------------------------------------- + */ +#ifndef OPTIMIZER_RPR_H +#define OPTIMIZER_RPR_H + +#include "nodes/parsenodes.h" +#include "nodes/plannodes.h" + +/* Limits and special values */ +#define RPR_VARID_MAX 252 /* max pattern variables: 252 */ +#define RPR_DEPTH_MAX UINT8_MAX /* max nesting depth: 255 */ +#define RPR_QUANTITY_INF INT32_MAX /* unbounded quantifier */ +#define RPR_ELEMIDX_MAX INT16_MAX /* max pattern elements */ +#define RPR_ELEMIDX_INVALID ((RPRElemIdx) -1) /* invalid index */ + +/* Special varId values for control elements (253-255) */ +#define RPR_VARID_ALT ((RPRVarId) 253) /* alternation start */ +#define RPR_VARID_END ((RPRVarId) 254) /* group end */ +#define RPR_VARID_FIN ((RPRVarId) 255) /* pattern finish */ + +/* Element flags */ +#define RPR_ELEM_RELUCTANT 0x01 /* reluctant (non-greedy) quantifier */ +#define RPR_ELEM_ABSORBABLE 0x02 /* branch supports context absorption */ + +/* Accessor macros for RPRPatternElement */ +#define RPRElemIsReluctant(e) ((e)->flags & RPR_ELEM_RELUCTANT) +#define RPRElemIsAbsorbable(e) ((e)->flags & RPR_ELEM_ABSORBABLE) +#define RPRElemIsVar(e) ((e)->varId <= RPR_VARID_MAX) +#define RPRElemIsAlt(e) ((e)->varId == RPR_VARID_ALT) +#define RPRElemIsEnd(e) ((e)->varId == RPR_VARID_END) +#define RPRElemIsFin(e) ((e)->varId == RPR_VARID_FIN) +#define RPRElemCanSkip(e) ((e)->min == 0) + +extern RPRPattern *buildRPRPattern(RPRPatternNode *pattern, List *defineVariableList); + +#endif /* OPTIMIZER_RPR_H */ diff --git a/src/include/parser/parse_clause.h b/src/include/parser/parse_clause.h index fe234611007..8aaac881f2b 100644 --- a/src/include/parser/parse_clause.h +++ b/src/include/parser/parse_clause.h @@ -52,6 +52,9 @@ extern List *addTargetToSortList(ParseState *pstate, TargetEntry *tle, extern Index assignSortGroupRef(TargetEntry *tle, List *tlist); extern bool targetIsInSortList(TargetEntry *tle, Oid sortop, List *sortList); +extern TargetEntry *findTargetlistEntrySQL99(ParseState *pstate, Node *node, + List **tlist, ParseExprKind exprKind); + /* functions in parse_jsontable.c */ extern ParseNamespaceItem *transformJsonTable(ParseState *pstate, JsonTable *jt); diff --git a/src/include/parser/parse_rpr.h b/src/include/parser/parse_rpr.h new file mode 100644 index 00000000000..40e0258d2e6 --- /dev/null +++ b/src/include/parser/parse_rpr.h @@ -0,0 +1,22 @@ +/*------------------------------------------------------------------------- + * + * parse_rpr.h + * handle Row Pattern Recognition in parser + * + * + * Portions Copyright (c) 1996-2025, PostgreSQL Global Development Group + * Portions Copyright (c) 1994, Regents of the University of California + * + * src/include/parser/parse_rpr.h + * + *------------------------------------------------------------------------- + */ +#ifndef PARSE_RPR_H +#define PARSE_RPR_H + +#include "parser/parse_node.h" + +extern void transformRPR(ParseState *pstate, WindowClause *wc, + WindowDef *windef, List **targetlist); + +#endif /* PARSE_RPR_H */ diff --git a/src/test/regress/expected/rpr.out b/src/test/regress/expected/rpr.out index cff052a66d7..cea986d6704 100644 --- a/src/test/regress/expected/rpr.out +++ b/src/test/regress/expected/rpr.out @@ -203,715 +203,1847 @@ SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER company2 | 07-10-2023 | 1300 | | | (20 rows) --- basic test with none-greedy pattern -SELECT company, tdate, price, count(*) OVER w - FROM stock - WINDOW w AS ( - PARTITION BY company - ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING - INITIAL - PATTERN (A A A) - DEFINE - A AS price >= 140 AND price <= 150 -); - company | tdate | price | count -----------+------------+-------+------- - company1 | 07-01-2023 | 100 | 0 - company1 | 07-02-2023 | 200 | 0 - company1 | 07-03-2023 | 150 | 3 - company1 | 07-04-2023 | 140 | 0 - company1 | 07-05-2023 | 150 | 0 - company1 | 07-06-2023 | 90 | 0 - company1 | 07-07-2023 | 110 | 0 - company1 | 07-08-2023 | 130 | 0 - company1 | 07-09-2023 | 120 | 0 - company1 | 07-10-2023 | 130 | 0 - company2 | 07-01-2023 | 50 | 0 - company2 | 07-02-2023 | 2000 | 0 - company2 | 07-03-2023 | 1500 | 0 - company2 | 07-04-2023 | 1400 | 0 - company2 | 07-05-2023 | 1500 | 0 - company2 | 07-06-2023 | 60 | 0 - company2 | 07-07-2023 | 1100 | 0 - company2 | 07-08-2023 | 1300 | 0 - company2 | 07-09-2023 | 1200 | 0 - company2 | 07-10-2023 | 1300 | 0 -(20 rows) - --- last_value() should remain consistent -SELECT company, tdate, price, last_value(price) OVER w +-- test using alternation (|) with sequence +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w FROM stock WINDOW w AS ( PARTITION BY company - ORDER BY tdate ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING INITIAL - PATTERN (START UP+ DOWN+) + PATTERN (START (UP | DOWN)) DEFINE START AS TRUE, UP AS price > PREV(price), DOWN AS price < PREV(price) ); - company | tdate | price | last_value -----------+------------+-------+------------ - company1 | 07-01-2023 | 100 | 140 - company1 | 07-02-2023 | 200 | - company1 | 07-03-2023 | 150 | - company1 | 07-04-2023 | 140 | - company1 | 07-05-2023 | 150 | - company1 | 07-06-2023 | 90 | 120 - company1 | 07-07-2023 | 110 | - company1 | 07-08-2023 | 130 | - company1 | 07-09-2023 | 120 | - company1 | 07-10-2023 | 130 | - company2 | 07-01-2023 | 50 | 1400 - company2 | 07-02-2023 | 2000 | - company2 | 07-03-2023 | 1500 | - company2 | 07-04-2023 | 1400 | - company2 | 07-05-2023 | 1500 | - company2 | 07-06-2023 | 60 | 1200 - company2 | 07-07-2023 | 1100 | - company2 | 07-08-2023 | 1300 | - company2 | 07-09-2023 | 1200 | - company2 | 07-10-2023 | 1300 | -(20 rows) - --- omit "START" in DEFINE but it is ok because "START AS TRUE" is --- implicitly defined. per spec. -SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w, - nth_value(tdate, 2) OVER w AS nth_second - FROM stock - WINDOW w AS ( - PARTITION BY company - ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING - INITIAL - PATTERN (START UP+ DOWN+) - DEFINE - UP AS price > PREV(price), - DOWN AS price < PREV(price) -); - company | tdate | price | first_value | last_value | nth_second -----------+------------+-------+-------------+------------+------------ - company1 | 07-01-2023 | 100 | 100 | 140 | 07-02-2023 - company1 | 07-02-2023 | 200 | | | - company1 | 07-03-2023 | 150 | | | - company1 | 07-04-2023 | 140 | | | - company1 | 07-05-2023 | 150 | | | - company1 | 07-06-2023 | 90 | 90 | 120 | 07-07-2023 - company1 | 07-07-2023 | 110 | | | - company1 | 07-08-2023 | 130 | | | - company1 | 07-09-2023 | 120 | | | - company1 | 07-10-2023 | 130 | | | - company2 | 07-01-2023 | 50 | 50 | 1400 | 07-02-2023 - company2 | 07-02-2023 | 2000 | | | - company2 | 07-03-2023 | 1500 | | | - company2 | 07-04-2023 | 1400 | | | - company2 | 07-05-2023 | 1500 | | | - company2 | 07-06-2023 | 60 | 60 | 1200 | 07-07-2023 - company2 | 07-07-2023 | 1100 | | | - company2 | 07-08-2023 | 1300 | | | - company2 | 07-09-2023 | 1200 | | | - company2 | 07-10-2023 | 1300 | | | + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | 100 | 200 + company1 | 07-02-2023 | 200 | | + company1 | 07-03-2023 | 150 | 150 | 140 + company1 | 07-04-2023 | 140 | | + company1 | 07-05-2023 | 150 | 150 | 90 + company1 | 07-06-2023 | 90 | | + company1 | 07-07-2023 | 110 | 110 | 130 + company1 | 07-08-2023 | 130 | | + company1 | 07-09-2023 | 120 | 120 | 130 + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | 50 | 2000 + company2 | 07-02-2023 | 2000 | | + company2 | 07-03-2023 | 1500 | 1500 | 1400 + company2 | 07-04-2023 | 1400 | | + company2 | 07-05-2023 | 1500 | 1500 | 60 + company2 | 07-06-2023 | 60 | | + company2 | 07-07-2023 | 1100 | 1100 | 1300 + company2 | 07-08-2023 | 1300 | | + company2 | 07-09-2023 | 1200 | 1200 | 1300 + company2 | 07-10-2023 | 1300 | | (20 rows) --- the first row start with less than or equal to 100 +-- test using alternation (|) with group quantifier SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w FROM stock WINDOW w AS ( PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING INITIAL - PATTERN (LOWPRICE UP+ DOWN+) + PATTERN (START (UP | DOWN)+) DEFINE - LOWPRICE AS price <= 100, + START AS TRUE, UP AS price > PREV(price), DOWN AS price < PREV(price) ); company | tdate | price | first_value | last_value ----------+------------+-------+-------------+------------ - company1 | 07-01-2023 | 100 | 100 | 140 + company1 | 07-01-2023 | 100 | 100 | 130 company1 | 07-02-2023 | 200 | | company1 | 07-03-2023 | 150 | | company1 | 07-04-2023 | 140 | | company1 | 07-05-2023 | 150 | | - company1 | 07-06-2023 | 90 | 90 | 120 + company1 | 07-06-2023 | 90 | | company1 | 07-07-2023 | 110 | | company1 | 07-08-2023 | 130 | | company1 | 07-09-2023 | 120 | | company1 | 07-10-2023 | 130 | | - company2 | 07-01-2023 | 50 | 50 | 1400 + company2 | 07-01-2023 | 50 | 50 | 1300 company2 | 07-02-2023 | 2000 | | company2 | 07-03-2023 | 1500 | | company2 | 07-04-2023 | 1400 | | company2 | 07-05-2023 | 1500 | | - company2 | 07-06-2023 | 60 | 60 | 1200 + company2 | 07-06-2023 | 60 | | company2 | 07-07-2023 | 1100 | | company2 | 07-08-2023 | 1300 | | company2 | 07-09-2023 | 1200 | | company2 | 07-10-2023 | 1300 | | (20 rows) --- second row raises 120% +-- test using nested alternation SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w FROM stock WINDOW w AS ( PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING INITIAL - PATTERN (LOWPRICE UP+ DOWN+) + PATTERN (START ((UP DOWN) | FLAT)+) DEFINE - LOWPRICE AS price <= 100, - UP AS price > PREV(price) * 1.2, - DOWN AS price < PREV(price) + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price), + FLAT AS price = PREV(price) ); company | tdate | price | first_value | last_value ----------+------------+-------+-------------+------------ - company1 | 07-01-2023 | 100 | 100 | 140 + company1 | 07-01-2023 | 100 | 100 | 150 company1 | 07-02-2023 | 200 | | company1 | 07-03-2023 | 150 | | - company1 | 07-04-2023 | 140 | | + company1 | 07-04-2023 | 140 | 140 | 90 company1 | 07-05-2023 | 150 | | company1 | 07-06-2023 | 90 | | - company1 | 07-07-2023 | 110 | | + company1 | 07-07-2023 | 110 | 110 | 120 company1 | 07-08-2023 | 130 | | company1 | 07-09-2023 | 120 | | company1 | 07-10-2023 | 130 | | - company2 | 07-01-2023 | 50 | 50 | 1400 + company2 | 07-01-2023 | 50 | 50 | 1500 company2 | 07-02-2023 | 2000 | | company2 | 07-03-2023 | 1500 | | - company2 | 07-04-2023 | 1400 | | + company2 | 07-04-2023 | 1400 | 1400 | 60 company2 | 07-05-2023 | 1500 | | company2 | 07-06-2023 | 60 | | - company2 | 07-07-2023 | 1100 | | + company2 | 07-07-2023 | 1100 | 1100 | 1200 company2 | 07-08-2023 | 1300 | | company2 | 07-09-2023 | 1200 | | company2 | 07-10-2023 | 1300 | | (20 rows) --- using NEXT +-- test using group with quantifier SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w FROM stock WINDOW w AS ( PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING INITIAL - PATTERN (START UPDOWN) + PATTERN ((UP DOWN)+) DEFINE - START AS TRUE, - UPDOWN AS price > PREV(price) AND price > NEXT(price) + UP AS price > PREV(price), + DOWN AS price < PREV(price) ); company | tdate | price | first_value | last_value ----------+------------+-------+-------------+------------ - company1 | 07-01-2023 | 100 | 100 | 200 - company1 | 07-02-2023 | 200 | | + company1 | 07-01-2023 | 100 | | + company1 | 07-02-2023 | 200 | 200 | 150 company1 | 07-03-2023 | 150 | | - company1 | 07-04-2023 | 140 | 140 | 150 - company1 | 07-05-2023 | 150 | | + company1 | 07-04-2023 | 140 | | + company1 | 07-05-2023 | 150 | 150 | 90 company1 | 07-06-2023 | 90 | | - company1 | 07-07-2023 | 110 | 110 | 130 - company1 | 07-08-2023 | 130 | | + company1 | 07-07-2023 | 110 | | + company1 | 07-08-2023 | 130 | 130 | 120 company1 | 07-09-2023 | 120 | | company1 | 07-10-2023 | 130 | | - company2 | 07-01-2023 | 50 | 50 | 2000 - company2 | 07-02-2023 | 2000 | | + company2 | 07-01-2023 | 50 | | + company2 | 07-02-2023 | 2000 | 2000 | 1500 company2 | 07-03-2023 | 1500 | | - company2 | 07-04-2023 | 1400 | 1400 | 1500 - company2 | 07-05-2023 | 1500 | | + company2 | 07-04-2023 | 1400 | | + company2 | 07-05-2023 | 1500 | 1500 | 60 company2 | 07-06-2023 | 60 | | - company2 | 07-07-2023 | 1100 | 1100 | 1300 - company2 | 07-08-2023 | 1300 | | + company2 | 07-07-2023 | 1100 | | + company2 | 07-08-2023 | 1300 | 1300 | 1200 company2 | 07-09-2023 | 1200 | | company2 | 07-10-2023 | 1300 | | (20 rows) --- using AFTER MATCH SKIP TO NEXT ROW +-- test using absolute threshold values (not relative PREV) +-- HIGH: price > 150, LOW: price < 100, MID: neutral range SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w FROM stock WINDOW w AS ( PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING - AFTER MATCH SKIP TO NEXT ROW INITIAL - PATTERN (START UPDOWN) + PATTERN (LOW MID* HIGH) DEFINE - START AS TRUE, - UPDOWN AS price > PREV(price) AND price > NEXT(price) + LOW AS price < 100, + MID AS price >= 100 AND price <= 150, + HIGH AS price > 150 ); company | tdate | price | first_value | last_value ----------+------------+-------+-------------+------------ - company1 | 07-01-2023 | 100 | 100 | 200 + company1 | 07-01-2023 | 100 | | company1 | 07-02-2023 | 200 | | company1 | 07-03-2023 | 150 | | - company1 | 07-04-2023 | 140 | 140 | 150 + company1 | 07-04-2023 | 140 | | company1 | 07-05-2023 | 150 | | company1 | 07-06-2023 | 90 | | - company1 | 07-07-2023 | 110 | 110 | 130 + company1 | 07-07-2023 | 110 | | company1 | 07-08-2023 | 130 | | company1 | 07-09-2023 | 120 | | company1 | 07-10-2023 | 130 | | company2 | 07-01-2023 | 50 | 50 | 2000 company2 | 07-02-2023 | 2000 | | company2 | 07-03-2023 | 1500 | | - company2 | 07-04-2023 | 1400 | 1400 | 1500 + company2 | 07-04-2023 | 1400 | | company2 | 07-05-2023 | 1500 | | - company2 | 07-06-2023 | 60 | | - company2 | 07-07-2023 | 1100 | 1100 | 1300 + company2 | 07-06-2023 | 60 | 60 | 1100 + company2 | 07-07-2023 | 1100 | | company2 | 07-08-2023 | 1300 | | company2 | 07-09-2023 | 1200 | | company2 | 07-10-2023 | 1300 | | (20 rows) --- match everything +-- test threshold-based pattern with alternation SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w FROM stock WINDOW w AS ( PARTITION BY company - ORDER BY tdate ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING - AFTER MATCH SKIP PAST LAST ROW INITIAL - PATTERN (A+) + PATTERN (LOW (MID | HIGH)+) DEFINE - A AS TRUE + LOW AS price < 100, + MID AS price >= 100 AND price <= 150, + HIGH AS price > 150 ); company | tdate | price | first_value | last_value ----------+------------+-------+-------------+------------ - company1 | 07-01-2023 | 100 | 100 | 130 + company1 | 07-01-2023 | 100 | | company1 | 07-02-2023 | 200 | | company1 | 07-03-2023 | 150 | | company1 | 07-04-2023 | 140 | | company1 | 07-05-2023 | 150 | | - company1 | 07-06-2023 | 90 | | + company1 | 07-06-2023 | 90 | 90 | 130 company1 | 07-07-2023 | 110 | | company1 | 07-08-2023 | 130 | | company1 | 07-09-2023 | 120 | | company1 | 07-10-2023 | 130 | | - company2 | 07-01-2023 | 50 | 50 | 1300 + company2 | 07-01-2023 | 50 | 50 | 1500 company2 | 07-02-2023 | 2000 | | company2 | 07-03-2023 | 1500 | | company2 | 07-04-2023 | 1400 | | company2 | 07-05-2023 | 1500 | | - company2 | 07-06-2023 | 60 | | + company2 | 07-06-2023 | 60 | 60 | 1300 company2 | 07-07-2023 | 1100 | | company2 | 07-08-2023 | 1300 | | company2 | 07-09-2023 | 1200 | | company2 | 07-10-2023 | 1300 | | (20 rows) --- backtracking with reclassification of rows --- using AFTER MATCH SKIP PAST LAST ROW -SELECT company, tdate, price, first_value(tdate) OVER w, last_value(tdate) OVER w +-- basic test with none-greedy pattern +SELECT company, tdate, price, count(*) OVER w FROM stock WINDOW w AS ( PARTITION BY company - ORDER BY tdate ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING - AFTER MATCH SKIP PAST LAST ROW INITIAL - PATTERN (A+ B+) + PATTERN (A A A) DEFINE - A AS price > 100, - B AS price > 100 + A AS price >= 140 AND price <= 150 ); - company | tdate | price | first_value | last_value -----------+------------+-------+-------------+------------ - company1 | 07-01-2023 | 100 | | - company1 | 07-02-2023 | 200 | 07-02-2023 | 07-05-2023 - company1 | 07-03-2023 | 150 | | - company1 | 07-04-2023 | 140 | | - company1 | 07-05-2023 | 150 | | - company1 | 07-06-2023 | 90 | | - company1 | 07-07-2023 | 110 | 07-07-2023 | 07-10-2023 - company1 | 07-08-2023 | 130 | | - company1 | 07-09-2023 | 120 | | - company1 | 07-10-2023 | 130 | | - company2 | 07-01-2023 | 50 | | - company2 | 07-02-2023 | 2000 | 07-02-2023 | 07-05-2023 - company2 | 07-03-2023 | 1500 | | - company2 | 07-04-2023 | 1400 | | - company2 | 07-05-2023 | 1500 | | - company2 | 07-06-2023 | 60 | | - company2 | 07-07-2023 | 1100 | 07-07-2023 | 07-10-2023 - company2 | 07-08-2023 | 1300 | | - company2 | 07-09-2023 | 1200 | | - company2 | 07-10-2023 | 1300 | | + company | tdate | price | count +----------+------------+-------+------- + company1 | 07-01-2023 | 100 | 0 + company1 | 07-02-2023 | 200 | 0 + company1 | 07-03-2023 | 150 | 3 + company1 | 07-04-2023 | 140 | 0 + company1 | 07-05-2023 | 150 | 0 + company1 | 07-06-2023 | 90 | 0 + company1 | 07-07-2023 | 110 | 0 + company1 | 07-08-2023 | 130 | 0 + company1 | 07-09-2023 | 120 | 0 + company1 | 07-10-2023 | 130 | 0 + company2 | 07-01-2023 | 50 | 0 + company2 | 07-02-2023 | 2000 | 0 + company2 | 07-03-2023 | 1500 | 0 + company2 | 07-04-2023 | 1400 | 0 + company2 | 07-05-2023 | 1500 | 0 + company2 | 07-06-2023 | 60 | 0 + company2 | 07-07-2023 | 1100 | 0 + company2 | 07-08-2023 | 1300 | 0 + company2 | 07-09-2023 | 1200 | 0 + company2 | 07-10-2023 | 1300 | 0 (20 rows) --- backtracking with reclassification of rows --- using AFTER MATCH SKIP TO NEXT ROW -SELECT company, tdate, price, first_value(tdate) OVER w, last_value(tdate) OVER w +-- test using {n} quantifier (A A A should be optimized to A{3}) +SELECT company, tdate, price, count(*) OVER w FROM stock WINDOW w AS ( PARTITION BY company - ORDER BY tdate ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING - AFTER MATCH SKIP TO NEXT ROW INITIAL - PATTERN (A+ B+) + PATTERN (A{3}) DEFINE - A AS price > 100, - B AS price > 100 + A AS price >= 140 AND price <= 150 ); - company | tdate | price | first_value | last_value -----------+------------+-------+-------------+------------ - company1 | 07-01-2023 | 100 | | - company1 | 07-02-2023 | 200 | 07-02-2023 | 07-05-2023 - company1 | 07-03-2023 | 150 | 07-03-2023 | 07-05-2023 - company1 | 07-04-2023 | 140 | 07-04-2023 | 07-05-2023 - company1 | 07-05-2023 | 150 | | - company1 | 07-06-2023 | 90 | | - company1 | 07-07-2023 | 110 | 07-07-2023 | 07-10-2023 - company1 | 07-08-2023 | 130 | 07-08-2023 | 07-10-2023 - company1 | 07-09-2023 | 120 | 07-09-2023 | 07-10-2023 - company1 | 07-10-2023 | 130 | | - company2 | 07-01-2023 | 50 | | - company2 | 07-02-2023 | 2000 | 07-02-2023 | 07-05-2023 - company2 | 07-03-2023 | 1500 | 07-03-2023 | 07-05-2023 - company2 | 07-04-2023 | 1400 | 07-04-2023 | 07-05-2023 - company2 | 07-05-2023 | 1500 | | - company2 | 07-06-2023 | 60 | | - company2 | 07-07-2023 | 1100 | 07-07-2023 | 07-10-2023 - company2 | 07-08-2023 | 1300 | 07-08-2023 | 07-10-2023 - company2 | 07-09-2023 | 1200 | 07-09-2023 | 07-10-2023 - company2 | 07-10-2023 | 1300 | | + company | tdate | price | count +----------+------------+-------+------- + company1 | 07-01-2023 | 100 | 0 + company1 | 07-02-2023 | 200 | 0 + company1 | 07-03-2023 | 150 | 3 + company1 | 07-04-2023 | 140 | 0 + company1 | 07-05-2023 | 150 | 0 + company1 | 07-06-2023 | 90 | 0 + company1 | 07-07-2023 | 110 | 0 + company1 | 07-08-2023 | 130 | 0 + company1 | 07-09-2023 | 120 | 0 + company1 | 07-10-2023 | 130 | 0 + company2 | 07-01-2023 | 50 | 0 + company2 | 07-02-2023 | 2000 | 0 + company2 | 07-03-2023 | 1500 | 0 + company2 | 07-04-2023 | 1400 | 0 + company2 | 07-05-2023 | 1500 | 0 + company2 | 07-06-2023 | 60 | 0 + company2 | 07-07-2023 | 1100 | 0 + company2 | 07-08-2023 | 1300 | 0 + company2 | 07-09-2023 | 1200 | 0 + company2 | 07-10-2023 | 1300 | 0 (20 rows) --- ROWS BETWEEN CURRENT ROW AND offset FOLLOWING -SELECT company, tdate, price, first_value(tdate) OVER w, last_value(tdate) OVER w, - count(*) OVER w +-- test using {n,} quantifier (2 or more) +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A{2,}) + DEFINE + A AS price > 100 +); + company | tdate | price | count +----------+------------+-------+------- + company1 | 07-01-2023 | 100 | 0 + company1 | 07-02-2023 | 200 | 4 + company1 | 07-03-2023 | 150 | 0 + company1 | 07-04-2023 | 140 | 0 + company1 | 07-05-2023 | 150 | 0 + company1 | 07-06-2023 | 90 | 0 + company1 | 07-07-2023 | 110 | 4 + company1 | 07-08-2023 | 130 | 0 + company1 | 07-09-2023 | 120 | 0 + company1 | 07-10-2023 | 130 | 0 + company2 | 07-01-2023 | 50 | 0 + company2 | 07-02-2023 | 2000 | 4 + company2 | 07-03-2023 | 1500 | 0 + company2 | 07-04-2023 | 1400 | 0 + company2 | 07-05-2023 | 1500 | 0 + company2 | 07-06-2023 | 60 | 0 + company2 | 07-07-2023 | 1100 | 4 + company2 | 07-08-2023 | 1300 | 0 + company2 | 07-09-2023 | 1200 | 0 + company2 | 07-10-2023 | 1300 | 0 +(20 rows) + +-- test using {n,m} quantifier (2 to 4) +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A{2,4}) + DEFINE + A AS price > 100 +); + company | tdate | price | count +----------+------------+-------+------- + company1 | 07-01-2023 | 100 | 0 + company1 | 07-02-2023 | 200 | 4 + company1 | 07-03-2023 | 150 | 0 + company1 | 07-04-2023 | 140 | 0 + company1 | 07-05-2023 | 150 | 0 + company1 | 07-06-2023 | 90 | 0 + company1 | 07-07-2023 | 110 | 4 + company1 | 07-08-2023 | 130 | 0 + company1 | 07-09-2023 | 120 | 0 + company1 | 07-10-2023 | 130 | 0 + company2 | 07-01-2023 | 50 | 0 + company2 | 07-02-2023 | 2000 | 4 + company2 | 07-03-2023 | 1500 | 0 + company2 | 07-04-2023 | 1400 | 0 + company2 | 07-05-2023 | 1500 | 0 + company2 | 07-06-2023 | 60 | 0 + company2 | 07-07-2023 | 1100 | 4 + company2 | 07-08-2023 | 1300 | 0 + company2 | 07-09-2023 | 1200 | 0 + company2 | 07-10-2023 | 1300 | 0 +(20 rows) + +-- last_value() should remain consistent +SELECT company, tdate, price, last_value(price) OVER w FROM stock WINDOW w AS ( PARTITION BY company ORDER BY tdate - ROWS BETWEEN CURRENT ROW AND 2 FOLLOWING - AFTER MATCH SKIP PAST LAST ROW + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL PATTERN (START UP+ DOWN+) DEFINE START AS TRUE, UP AS price > PREV(price), DOWN AS price < PREV(price) ); - company | tdate | price | first_value | last_value | count -----------+------------+-------+-------------+------------+------- - company1 | 07-01-2023 | 100 | 07-01-2023 | 07-03-2023 | 3 - company1 | 07-02-2023 | 200 | | | 0 - company1 | 07-03-2023 | 150 | | | 0 - company1 | 07-04-2023 | 140 | 07-04-2023 | 07-06-2023 | 3 - company1 | 07-05-2023 | 150 | | | 0 - company1 | 07-06-2023 | 90 | | | 0 - company1 | 07-07-2023 | 110 | 07-07-2023 | 07-09-2023 | 3 - company1 | 07-08-2023 | 130 | | | 0 - company1 | 07-09-2023 | 120 | | | 0 - company1 | 07-10-2023 | 130 | | | 0 - company2 | 07-01-2023 | 50 | 07-01-2023 | 07-03-2023 | 3 - company2 | 07-02-2023 | 2000 | | | 0 - company2 | 07-03-2023 | 1500 | | | 0 - company2 | 07-04-2023 | 1400 | 07-04-2023 | 07-06-2023 | 3 - company2 | 07-05-2023 | 1500 | | | 0 - company2 | 07-06-2023 | 60 | | | 0 - company2 | 07-07-2023 | 1100 | 07-07-2023 | 07-09-2023 | 3 - company2 | 07-08-2023 | 1300 | | | 0 - company2 | 07-09-2023 | 1200 | | | 0 - company2 | 07-10-2023 | 1300 | | | 0 + company | tdate | price | last_value +----------+------------+-------+------------ + company1 | 07-01-2023 | 100 | 140 + company1 | 07-02-2023 | 200 | + company1 | 07-03-2023 | 150 | + company1 | 07-04-2023 | 140 | + company1 | 07-05-2023 | 150 | + company1 | 07-06-2023 | 90 | 120 + company1 | 07-07-2023 | 110 | + company1 | 07-08-2023 | 130 | + company1 | 07-09-2023 | 120 | + company1 | 07-10-2023 | 130 | + company2 | 07-01-2023 | 50 | 1400 + company2 | 07-02-2023 | 2000 | + company2 | 07-03-2023 | 1500 | + company2 | 07-04-2023 | 1400 | + company2 | 07-05-2023 | 1500 | + company2 | 07-06-2023 | 60 | 1200 + company2 | 07-07-2023 | 1100 | + company2 | 07-08-2023 | 1300 | + company2 | 07-09-2023 | 1200 | + company2 | 07-10-2023 | 1300 | (20 rows) --- --- Aggregates --- --- using AFTER MATCH SKIP PAST LAST ROW -SELECT company, tdate, price, - first_value(price) OVER w, - last_value(price) OVER w, - max(price) OVER w, - min(price) OVER w, - sum(price) OVER w, - avg(price) OVER w, - count(price) OVER w -FROM stock -WINDOW w AS ( -PARTITION BY company -ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING -AFTER MATCH SKIP PAST LAST ROW -INITIAL -PATTERN (START UP+ DOWN+) -DEFINE -START AS TRUE, -UP AS price > PREV(price), -DOWN AS price < PREV(price) +-- omit "START" in DEFINE but it is ok because "START AS TRUE" is +-- implicitly defined. per spec. +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w, + nth_value(tdate, 2) OVER w AS nth_second + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + UP AS price > PREV(price), + DOWN AS price < PREV(price) ); - company | tdate | price | first_value | last_value | max | min | sum | avg | count -----------+------------+-------+-------------+------------+------+-----+------+-----------------------+------- - company1 | 07-01-2023 | 100 | 100 | 140 | 200 | 100 | 590 | 147.5000000000000000 | 4 - company1 | 07-02-2023 | 200 | | | | | | | 0 - company1 | 07-03-2023 | 150 | | | | | | | 0 - company1 | 07-04-2023 | 140 | | | | | | | 0 - company1 | 07-05-2023 | 150 | | | | | | | 0 - company1 | 07-06-2023 | 90 | 90 | 120 | 130 | 90 | 450 | 112.5000000000000000 | 4 - company1 | 07-07-2023 | 110 | | | | | | | 0 - company1 | 07-08-2023 | 130 | | | | | | | 0 - company1 | 07-09-2023 | 120 | | | | | | | 0 - company1 | 07-10-2023 | 130 | | | | | | | 0 - company2 | 07-01-2023 | 50 | 50 | 1400 | 2000 | 50 | 4950 | 1237.5000000000000000 | 4 - company2 | 07-02-2023 | 2000 | | | | | | | 0 - company2 | 07-03-2023 | 1500 | | | | | | | 0 - company2 | 07-04-2023 | 1400 | | | | | | | 0 - company2 | 07-05-2023 | 1500 | | | | | | | 0 - company2 | 07-06-2023 | 60 | 60 | 1200 | 1300 | 60 | 3660 | 915.0000000000000000 | 4 - company2 | 07-07-2023 | 1100 | | | | | | | 0 - company2 | 07-08-2023 | 1300 | | | | | | | 0 - company2 | 07-09-2023 | 1200 | | | | | | | 0 - company2 | 07-10-2023 | 1300 | | | | | | | 0 + company | tdate | price | first_value | last_value | nth_second +----------+------------+-------+-------------+------------+------------ + company1 | 07-01-2023 | 100 | 100 | 140 | 07-02-2023 + company1 | 07-02-2023 | 200 | | | + company1 | 07-03-2023 | 150 | | | + company1 | 07-04-2023 | 140 | | | + company1 | 07-05-2023 | 150 | | | + company1 | 07-06-2023 | 90 | 90 | 120 | 07-07-2023 + company1 | 07-07-2023 | 110 | | | + company1 | 07-08-2023 | 130 | | | + company1 | 07-09-2023 | 120 | | | + company1 | 07-10-2023 | 130 | | | + company2 | 07-01-2023 | 50 | 50 | 1400 | 07-02-2023 + company2 | 07-02-2023 | 2000 | | | + company2 | 07-03-2023 | 1500 | | | + company2 | 07-04-2023 | 1400 | | | + company2 | 07-05-2023 | 1500 | | | + company2 | 07-06-2023 | 60 | 60 | 1200 | 07-07-2023 + company2 | 07-07-2023 | 1100 | | | + company2 | 07-08-2023 | 1300 | | | + company2 | 07-09-2023 | 1200 | | | + company2 | 07-10-2023 | 1300 | | | (20 rows) --- using AFTER MATCH SKIP TO NEXT ROW -SELECT company, tdate, price, - first_value(price) OVER w, - last_value(price) OVER w, - max(price) OVER w, - min(price) OVER w, - sum(price) OVER w, - avg(price) OVER w, - count(price) OVER w -FROM stock -WINDOW w AS ( -PARTITION BY company -ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING -AFTER MATCH SKIP TO NEXT ROW -INITIAL -PATTERN (START UP+ DOWN+) -DEFINE -START AS TRUE, -UP AS price > PREV(price), -DOWN AS price < PREV(price) -); - company | tdate | price | first_value | last_value | max | min | sum | avg | count -----------+------------+-------+-------------+------------+------+------+------+-----------------------+------- - company1 | 07-01-2023 | 100 | 100 | 140 | 200 | 100 | 590 | 147.5000000000000000 | 4 - company1 | 07-02-2023 | 200 | | | | | | | 0 - company1 | 07-03-2023 | 150 | | | | | | | 0 - company1 | 07-04-2023 | 140 | 140 | 90 | 150 | 90 | 380 | 126.6666666666666667 | 3 - company1 | 07-05-2023 | 150 | | | | | | | 0 - company1 | 07-06-2023 | 90 | 90 | 120 | 130 | 90 | 450 | 112.5000000000000000 | 4 - company1 | 07-07-2023 | 110 | 110 | 120 | 130 | 110 | 360 | 120.0000000000000000 | 3 - company1 | 07-08-2023 | 130 | | | | | | | 0 - company1 | 07-09-2023 | 120 | | | | | | | 0 - company1 | 07-10-2023 | 130 | | | | | | | 0 - company2 | 07-01-2023 | 50 | 50 | 1400 | 2000 | 50 | 4950 | 1237.5000000000000000 | 4 - company2 | 07-02-2023 | 2000 | | | | | | | 0 - company2 | 07-03-2023 | 1500 | | | | | | | 0 - company2 | 07-04-2023 | 1400 | 1400 | 60 | 1500 | 60 | 2960 | 986.6666666666666667 | 3 - company2 | 07-05-2023 | 1500 | | | | | | | 0 - company2 | 07-06-2023 | 60 | 60 | 1200 | 1300 | 60 | 3660 | 915.0000000000000000 | 4 - company2 | 07-07-2023 | 1100 | 1100 | 1200 | 1300 | 1100 | 3600 | 1200.0000000000000000 | 3 - company2 | 07-08-2023 | 1300 | | | | | | | 0 - company2 | 07-09-2023 | 1200 | | | | | | | 0 - company2 | 07-10-2023 | 1300 | | | | | | | 0 +-- the first row start with less than or equal to 100 +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (LOWPRICE UP+ DOWN+) + DEFINE + LOWPRICE AS price <= 100, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | 100 | 140 + company1 | 07-02-2023 | 200 | | + company1 | 07-03-2023 | 150 | | + company1 | 07-04-2023 | 140 | | + company1 | 07-05-2023 | 150 | | + company1 | 07-06-2023 | 90 | 90 | 120 + company1 | 07-07-2023 | 110 | | + company1 | 07-08-2023 | 130 | | + company1 | 07-09-2023 | 120 | | + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | 50 | 1400 + company2 | 07-02-2023 | 2000 | | + company2 | 07-03-2023 | 1500 | | + company2 | 07-04-2023 | 1400 | | + company2 | 07-05-2023 | 1500 | | + company2 | 07-06-2023 | 60 | 60 | 1200 + company2 | 07-07-2023 | 1100 | | + company2 | 07-08-2023 | 1300 | | + company2 | 07-09-2023 | 1200 | | + company2 | 07-10-2023 | 1300 | | (20 rows) --- JOIN case -CREATE TEMP TABLE t1 (i int, v1 int); -CREATE TEMP TABLE t2 (j int, v2 int); -INSERT INTO t1 VALUES(1,10); -INSERT INTO t1 VALUES(1,11); -INSERT INTO t1 VALUES(1,12); -INSERT INTO t2 VALUES(2,10); -INSERT INTO t2 VALUES(2,11); -INSERT INTO t2 VALUES(2,12); -SELECT * FROM t1, t2 WHERE t1.v1 <= 11 AND t2.v2 <= 11; - i | v1 | j | v2 ----+----+---+---- - 1 | 10 | 2 | 10 - 1 | 10 | 2 | 11 - 1 | 11 | 2 | 10 - 1 | 11 | 2 | 11 -(4 rows) +-- second row raises 120% +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (LOWPRICE UP+ DOWN+) + DEFINE + LOWPRICE AS price <= 100, + UP AS price > PREV(price) * 1.2, + DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | 100 | 140 + company1 | 07-02-2023 | 200 | | + company1 | 07-03-2023 | 150 | | + company1 | 07-04-2023 | 140 | | + company1 | 07-05-2023 | 150 | | + company1 | 07-06-2023 | 90 | | + company1 | 07-07-2023 | 110 | | + company1 | 07-08-2023 | 130 | | + company1 | 07-09-2023 | 120 | | + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | 50 | 1400 + company2 | 07-02-2023 | 2000 | | + company2 | 07-03-2023 | 1500 | | + company2 | 07-04-2023 | 1400 | | + company2 | 07-05-2023 | 1500 | | + company2 | 07-06-2023 | 60 | | + company2 | 07-07-2023 | 1100 | | + company2 | 07-08-2023 | 1300 | | + company2 | 07-09-2023 | 1200 | | + company2 | 07-10-2023 | 1300 | | +(20 rows) + +-- using NEXT +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UPDOWN) + DEFINE + START AS TRUE, + UPDOWN AS price > PREV(price) AND price > NEXT(price) +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | 100 | 200 + company1 | 07-02-2023 | 200 | | + company1 | 07-03-2023 | 150 | | + company1 | 07-04-2023 | 140 | 140 | 150 + company1 | 07-05-2023 | 150 | | + company1 | 07-06-2023 | 90 | | + company1 | 07-07-2023 | 110 | 110 | 130 + company1 | 07-08-2023 | 130 | | + company1 | 07-09-2023 | 120 | | + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | 50 | 2000 + company2 | 07-02-2023 | 2000 | | + company2 | 07-03-2023 | 1500 | | + company2 | 07-04-2023 | 1400 | 1400 | 1500 + company2 | 07-05-2023 | 1500 | | + company2 | 07-06-2023 | 60 | | + company2 | 07-07-2023 | 1100 | 1100 | 1300 + company2 | 07-08-2023 | 1300 | | + company2 | 07-09-2023 | 1200 | | + company2 | 07-10-2023 | 1300 | | +(20 rows) + +-- using AFTER MATCH SKIP TO NEXT ROW +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + INITIAL + PATTERN (START UPDOWN) + DEFINE + START AS TRUE, + UPDOWN AS price > PREV(price) AND price > NEXT(price) +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | 100 | 200 + company1 | 07-02-2023 | 200 | | + company1 | 07-03-2023 | 150 | | + company1 | 07-04-2023 | 140 | 140 | 150 + company1 | 07-05-2023 | 150 | | + company1 | 07-06-2023 | 90 | | + company1 | 07-07-2023 | 110 | 110 | 130 + company1 | 07-08-2023 | 130 | | + company1 | 07-09-2023 | 120 | | + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | 50 | 2000 + company2 | 07-02-2023 | 2000 | | + company2 | 07-03-2023 | 1500 | | + company2 | 07-04-2023 | 1400 | 1400 | 1500 + company2 | 07-05-2023 | 1500 | | + company2 | 07-06-2023 | 60 | | + company2 | 07-07-2023 | 1100 | 1100 | 1300 + company2 | 07-08-2023 | 1300 | | + company2 | 07-09-2023 | 1200 | | + company2 | 07-10-2023 | 1300 | | +(20 rows) + +-- match everything +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + INITIAL + PATTERN (A+) + DEFINE + A AS TRUE +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | 100 | 130 + company1 | 07-02-2023 | 200 | | + company1 | 07-03-2023 | 150 | | + company1 | 07-04-2023 | 140 | | + company1 | 07-05-2023 | 150 | | + company1 | 07-06-2023 | 90 | | + company1 | 07-07-2023 | 110 | | + company1 | 07-08-2023 | 130 | | + company1 | 07-09-2023 | 120 | | + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | 50 | 1300 + company2 | 07-02-2023 | 2000 | | + company2 | 07-03-2023 | 1500 | | + company2 | 07-04-2023 | 1400 | | + company2 | 07-05-2023 | 1500 | | + company2 | 07-06-2023 | 60 | | + company2 | 07-07-2023 | 1100 | | + company2 | 07-08-2023 | 1300 | | + company2 | 07-09-2023 | 1200 | | + company2 | 07-10-2023 | 1300 | | +(20 rows) + +-- backtracking with reclassification of rows +-- using AFTER MATCH SKIP PAST LAST ROW +SELECT company, tdate, price, first_value(tdate) OVER w, last_value(tdate) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + INITIAL + PATTERN (A+ B+) + DEFINE + A AS price > 100, + B AS price > 100 +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | | + company1 | 07-02-2023 | 200 | 07-02-2023 | 07-05-2023 + company1 | 07-03-2023 | 150 | | + company1 | 07-04-2023 | 140 | | + company1 | 07-05-2023 | 150 | | + company1 | 07-06-2023 | 90 | | + company1 | 07-07-2023 | 110 | 07-07-2023 | 07-10-2023 + company1 | 07-08-2023 | 130 | | + company1 | 07-09-2023 | 120 | | + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | | + company2 | 07-02-2023 | 2000 | 07-02-2023 | 07-05-2023 + company2 | 07-03-2023 | 1500 | | + company2 | 07-04-2023 | 1400 | | + company2 | 07-05-2023 | 1500 | | + company2 | 07-06-2023 | 60 | | + company2 | 07-07-2023 | 1100 | 07-07-2023 | 07-10-2023 + company2 | 07-08-2023 | 1300 | | + company2 | 07-09-2023 | 1200 | | + company2 | 07-10-2023 | 1300 | | +(20 rows) + +-- backtracking with reclassification of rows +-- using AFTER MATCH SKIP TO NEXT ROW +SELECT company, tdate, price, first_value(tdate) OVER w, last_value(tdate) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + INITIAL + PATTERN (A+ B+) + DEFINE + A AS price > 100, + B AS price > 100 +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | | + company1 | 07-02-2023 | 200 | 07-02-2023 | 07-05-2023 + company1 | 07-03-2023 | 150 | 07-03-2023 | 07-05-2023 + company1 | 07-04-2023 | 140 | 07-04-2023 | 07-05-2023 + company1 | 07-05-2023 | 150 | | + company1 | 07-06-2023 | 90 | | + company1 | 07-07-2023 | 110 | 07-07-2023 | 07-10-2023 + company1 | 07-08-2023 | 130 | 07-08-2023 | 07-10-2023 + company1 | 07-09-2023 | 120 | 07-09-2023 | 07-10-2023 + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | | + company2 | 07-02-2023 | 2000 | 07-02-2023 | 07-05-2023 + company2 | 07-03-2023 | 1500 | 07-03-2023 | 07-05-2023 + company2 | 07-04-2023 | 1400 | 07-04-2023 | 07-05-2023 + company2 | 07-05-2023 | 1500 | | + company2 | 07-06-2023 | 60 | | + company2 | 07-07-2023 | 1100 | 07-07-2023 | 07-10-2023 + company2 | 07-08-2023 | 1300 | 07-08-2023 | 07-10-2023 + company2 | 07-09-2023 | 1200 | 07-09-2023 | 07-10-2023 + company2 | 07-10-2023 | 1300 | | +(20 rows) + +-- ROWS BETWEEN CURRENT ROW AND offset FOLLOWING +SELECT company, tdate, price, first_value(tdate) OVER w, last_value(tdate) OVER w, + count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND 2 FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value | count +----------+------------+-------+-------------+------------+------- + company1 | 07-01-2023 | 100 | 07-01-2023 | 07-03-2023 | 3 + company1 | 07-02-2023 | 200 | | | 0 + company1 | 07-03-2023 | 150 | | | 0 + company1 | 07-04-2023 | 140 | 07-04-2023 | 07-06-2023 | 3 + company1 | 07-05-2023 | 150 | | | 0 + company1 | 07-06-2023 | 90 | | | 0 + company1 | 07-07-2023 | 110 | 07-07-2023 | 07-09-2023 | 3 + company1 | 07-08-2023 | 130 | | | 0 + company1 | 07-09-2023 | 120 | | | 0 + company1 | 07-10-2023 | 130 | | | 0 + company2 | 07-01-2023 | 50 | 07-01-2023 | 07-03-2023 | 3 + company2 | 07-02-2023 | 2000 | | | 0 + company2 | 07-03-2023 | 1500 | | | 0 + company2 | 07-04-2023 | 1400 | 07-04-2023 | 07-06-2023 | 3 + company2 | 07-05-2023 | 1500 | | | 0 + company2 | 07-06-2023 | 60 | | | 0 + company2 | 07-07-2023 | 1100 | 07-07-2023 | 07-09-2023 | 3 + company2 | 07-08-2023 | 1300 | | | 0 + company2 | 07-09-2023 | 1200 | | | 0 + company2 | 07-10-2023 | 1300 | | | 0 +(20 rows) + +-- +-- Aggregates +-- +-- using AFTER MATCH SKIP PAST LAST ROW +SELECT company, tdate, price, + first_value(price) OVER w, + last_value(price) OVER w, + max(price) OVER w, + min(price) OVER w, + sum(price) OVER w, + avg(price) OVER w, + count(price) OVER w +FROM stock +WINDOW w AS ( +PARTITION BY company +ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING +AFTER MATCH SKIP PAST LAST ROW +INITIAL +PATTERN (START UP+ DOWN+) +DEFINE +START AS TRUE, +UP AS price > PREV(price), +DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value | max | min | sum | avg | count +----------+------------+-------+-------------+------------+------+-----+------+-----------------------+------- + company1 | 07-01-2023 | 100 | 100 | 140 | 200 | 100 | 590 | 147.5000000000000000 | 4 + company1 | 07-02-2023 | 200 | | | | | | | 0 + company1 | 07-03-2023 | 150 | | | | | | | 0 + company1 | 07-04-2023 | 140 | | | | | | | 0 + company1 | 07-05-2023 | 150 | | | | | | | 0 + company1 | 07-06-2023 | 90 | 90 | 120 | 130 | 90 | 450 | 112.5000000000000000 | 4 + company1 | 07-07-2023 | 110 | | | | | | | 0 + company1 | 07-08-2023 | 130 | | | | | | | 0 + company1 | 07-09-2023 | 120 | | | | | | | 0 + company1 | 07-10-2023 | 130 | | | | | | | 0 + company2 | 07-01-2023 | 50 | 50 | 1400 | 2000 | 50 | 4950 | 1237.5000000000000000 | 4 + company2 | 07-02-2023 | 2000 | | | | | | | 0 + company2 | 07-03-2023 | 1500 | | | | | | | 0 + company2 | 07-04-2023 | 1400 | | | | | | | 0 + company2 | 07-05-2023 | 1500 | | | | | | | 0 + company2 | 07-06-2023 | 60 | 60 | 1200 | 1300 | 60 | 3660 | 915.0000000000000000 | 4 + company2 | 07-07-2023 | 1100 | | | | | | | 0 + company2 | 07-08-2023 | 1300 | | | | | | | 0 + company2 | 07-09-2023 | 1200 | | | | | | | 0 + company2 | 07-10-2023 | 1300 | | | | | | | 0 +(20 rows) + +-- using AFTER MATCH SKIP TO NEXT ROW +SELECT company, tdate, price, + first_value(price) OVER w, + last_value(price) OVER w, + max(price) OVER w, + min(price) OVER w, + sum(price) OVER w, + avg(price) OVER w, + count(price) OVER w +FROM stock +WINDOW w AS ( +PARTITION BY company +ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING +AFTER MATCH SKIP TO NEXT ROW +INITIAL +PATTERN (START UP+ DOWN+) +DEFINE +START AS TRUE, +UP AS price > PREV(price), +DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value | max | min | sum | avg | count +----------+------------+-------+-------------+------------+------+------+------+-----------------------+------- + company1 | 07-01-2023 | 100 | 100 | 140 | 200 | 100 | 590 | 147.5000000000000000 | 4 + company1 | 07-02-2023 | 200 | | | | | | | 0 + company1 | 07-03-2023 | 150 | | | | | | | 0 + company1 | 07-04-2023 | 140 | 140 | 90 | 150 | 90 | 380 | 126.6666666666666667 | 3 + company1 | 07-05-2023 | 150 | | | | | | | 0 + company1 | 07-06-2023 | 90 | 90 | 120 | 130 | 90 | 450 | 112.5000000000000000 | 4 + company1 | 07-07-2023 | 110 | 110 | 120 | 130 | 110 | 360 | 120.0000000000000000 | 3 + company1 | 07-08-2023 | 130 | | | | | | | 0 + company1 | 07-09-2023 | 120 | | | | | | | 0 + company1 | 07-10-2023 | 130 | | | | | | | 0 + company2 | 07-01-2023 | 50 | 50 | 1400 | 2000 | 50 | 4950 | 1237.5000000000000000 | 4 + company2 | 07-02-2023 | 2000 | | | | | | | 0 + company2 | 07-03-2023 | 1500 | | | | | | | 0 + company2 | 07-04-2023 | 1400 | 1400 | 60 | 1500 | 60 | 2960 | 986.6666666666666667 | 3 + company2 | 07-05-2023 | 1500 | | | | | | | 0 + company2 | 07-06-2023 | 60 | 60 | 1200 | 1300 | 60 | 3660 | 915.0000000000000000 | 4 + company2 | 07-07-2023 | 1100 | 1100 | 1200 | 1300 | 1100 | 3600 | 1200.0000000000000000 | 3 + company2 | 07-08-2023 | 1300 | | | | | | | 0 + company2 | 07-09-2023 | 1200 | | | | | | | 0 + company2 | 07-10-2023 | 1300 | | | | | | | 0 +(20 rows) + +-- JOIN case +CREATE TEMP TABLE t1 (i int, v1 int); +CREATE TEMP TABLE t2 (j int, v2 int); +INSERT INTO t1 VALUES(1,10); +INSERT INTO t1 VALUES(1,11); +INSERT INTO t1 VALUES(1,12); +INSERT INTO t2 VALUES(2,10); +INSERT INTO t2 VALUES(2,11); +INSERT INTO t2 VALUES(2,12); +SELECT * FROM t1, t2 WHERE t1.v1 <= 11 AND t2.v2 <= 11; + i | v1 | j | v2 +---+----+---+---- + 1 | 10 | 2 | 10 + 1 | 10 | 2 | 11 + 1 | 11 | 2 | 10 + 1 | 11 | 2 | 11 +(4 rows) + +SELECT *, count(*) OVER w FROM t1, t2 +WINDOW w AS ( + PARTITION BY t1.i + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A) + DEFINE + A AS v1 <= 11 AND v2 <= 11 +); + i | v1 | j | v2 | count +---+----+---+----+------- + 1 | 10 | 2 | 10 | 1 + 1 | 10 | 2 | 11 | 1 + 1 | 10 | 2 | 12 | 0 + 1 | 11 | 2 | 10 | 1 + 1 | 11 | 2 | 11 | 1 + 1 | 11 | 2 | 12 | 0 + 1 | 12 | 2 | 10 | 0 + 1 | 12 | 2 | 11 | 0 + 1 | 12 | 2 | 12 | 0 +(9 rows) + +-- WITH case +WITH wstock AS ( + SELECT * FROM stock WHERE tdate < '2023-07-08' +) +SELECT tdate, price, +first_value(tdate) OVER w, +count(*) OVER w + FROM wstock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + tdate | price | first_value | count +------------+-------+-------------+------- + 07-01-2023 | 100 | 07-01-2023 | 4 + 07-02-2023 | 200 | | 0 + 07-03-2023 | 150 | | 0 + 07-04-2023 | 140 | | 0 + 07-05-2023 | 150 | | 0 + 07-06-2023 | 90 | | 0 + 07-07-2023 | 110 | | 0 + 07-01-2023 | 50 | 07-01-2023 | 4 + 07-02-2023 | 2000 | | 0 + 07-03-2023 | 1500 | | 0 + 07-04-2023 | 1400 | | 0 + 07-05-2023 | 1500 | | 0 + 07-06-2023 | 60 | | 0 + 07-07-2023 | 1100 | | 0 +(14 rows) + +-- PREV has multiple column reference +CREATE TEMP TABLE rpr1 (id INTEGER, i SERIAL, j INTEGER); +INSERT INTO rpr1(id, j) SELECT 1, g*2 FROM generate_series(1, 10) AS g; +SELECT id, i, j, count(*) OVER w + FROM rpr1 + WINDOW w AS ( + PARTITION BY id + ORDER BY i + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + INITIAL + PATTERN (START COND+) + DEFINE + START AS TRUE, + COND AS PREV(i + j + 1) < 10 +); + id | i | j | count +----+----+----+------- + 1 | 1 | 2 | 3 + 1 | 2 | 4 | 0 + 1 | 3 | 6 | 0 + 1 | 4 | 8 | 0 + 1 | 5 | 10 | 0 + 1 | 6 | 12 | 0 + 1 | 7 | 14 | 0 + 1 | 8 | 16 | 0 + 1 | 9 | 18 | 0 + 1 | 10 | 20 | 0 +(10 rows) + +-- Smoke test for larger partitions. +WITH s AS ( + SELECT v, count(*) OVER w AS c + FROM (SELECT generate_series(1, 5000) v) + WINDOW w AS ( + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + INITIAL + PATTERN ( r+ ) + DEFINE r AS TRUE + ) +) +-- Should be exactly one long match across all rows. +SELECT * FROM s WHERE c > 0; + v | c +---+------ + 1 | 5000 +(1 row) + +WITH s AS ( + SELECT v, count(*) OVER w AS c + FROM (SELECT generate_series(1, 5000) v) + WINDOW w AS ( + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + INITIAL + PATTERN ( r ) + DEFINE r AS TRUE + ) +) +-- Every row should be its own match. +SELECT count(*) FROM s WHERE c > 0; + count +------- + 5000 +(1 row) + +-- View and pg_get_viewdef tests. +CREATE TEMP VIEW v_window AS +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w, + nth_value(tdate, 2) OVER w AS nth_second + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); +SELECT * FROM v_window; + company | tdate | price | first_value | last_value | nth_second +----------+------------+-------+-------------+------------+------------ + company1 | 07-01-2023 | 100 | 100 | 140 | 07-02-2023 + company1 | 07-02-2023 | 200 | | | + company1 | 07-03-2023 | 150 | | | + company1 | 07-04-2023 | 140 | | | + company1 | 07-05-2023 | 150 | | | + company1 | 07-06-2023 | 90 | 90 | 120 | 07-07-2023 + company1 | 07-07-2023 | 110 | | | + company1 | 07-08-2023 | 130 | | | + company1 | 07-09-2023 | 120 | | | + company1 | 07-10-2023 | 130 | | | + company2 | 07-01-2023 | 50 | 50 | 1400 | 07-02-2023 + company2 | 07-02-2023 | 2000 | | | + company2 | 07-03-2023 | 1500 | | | + company2 | 07-04-2023 | 1400 | | | + company2 | 07-05-2023 | 1500 | | | + company2 | 07-06-2023 | 60 | 60 | 1200 | 07-07-2023 + company2 | 07-07-2023 | 1100 | | | + company2 | 07-08-2023 | 1300 | | | + company2 | 07-09-2023 | 1200 | | | + company2 | 07-10-2023 | 1300 | | | +(20 rows) + +SELECT pg_get_viewdef('v_window'); + pg_get_viewdef +--------------------------------------------------------------------------------------- + SELECT company, + + tdate, + + price, + + first_value(price) OVER w AS first_value, + + last_value(price) OVER w AS last_value, + + nth_value(tdate, 2) OVER w AS nth_second + + FROM stock + + WINDOW w AS (PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + + AFTER MATCH SKIP PAST LAST ROW + + INITIAL + + PATTERN (start up+ down+) + + DEFINE + + start AS true, + + up AS (price > prev(price)), + + down AS (price < prev(price)) ); +(1 row) + +-- +-- Pattern optimization tests +-- VIEW shows original pattern, EXPLAIN shows optimized pattern +-- +-- Test: duplicate alternatives removal (A | B | A)+ -> (A | B)+ +CREATE TEMP VIEW v_opt_dup AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN ((A | B | A)+) + DEFINE + A AS price > 100, + B AS price <= 100 +); +SELECT pg_get_viewdef('v_opt_dup'); -- original: ((a | b | a)+) + pg_get_viewdef +--------------------------------------------------------------------------------------- + SELECT company, + + tdate, + + price, + + count(*) OVER w AS count + + FROM stock + + WINDOW w AS (PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + + AFTER MATCH SKIP PAST LAST ROW + + INITIAL + + PATTERN ((a | b | a)+) + + DEFINE + + a AS (price > 100), + + b AS (price <= 100) ); +(1 row) + +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_dup; -- optimized: ((a | b)+) + QUERY PLAN +---------------------------------------------------------------------------------------------------- + Subquery Scan on v_opt_dup + -> WindowAgg + Window: w AS (PARTITION BY stock.company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) + Pattern: ((a | b))+ + -> Sort + Sort Key: stock.company + -> Seq Scan on stock +(7 rows) + +-- Test: duplicate group removal ((A | B)+ | (A | B)+) -> (A | B)+ +CREATE TEMP VIEW v_opt_dup_group AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN ((A | B)+ | (A | B)+) + DEFINE + A AS price > 100, + B AS price <= 100 +); +SELECT pg_get_viewdef('v_opt_dup_group'); -- original: ((a | b)+ | (a | b)+) + pg_get_viewdef +--------------------------------------------------------------------------------------- + SELECT company, + + tdate, + + price, + + count(*) OVER w AS count + + FROM stock + + WINDOW w AS (PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + + AFTER MATCH SKIP PAST LAST ROW + + INITIAL + + PATTERN ((a | b)+ | (a | b)+) + + DEFINE + + a AS (price > 100), + + b AS (price <= 100) ); +(1 row) + +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_dup_group; -- optimized: ((a | b)+) + QUERY PLAN +---------------------------------------------------------------------------------------------------- + Subquery Scan on v_opt_dup_group + -> WindowAgg + Window: w AS (PARTITION BY stock.company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) + Pattern: ((a | b))+ + -> Sort + Sort Key: stock.company + -> Seq Scan on stock +(7 rows) + +-- Test: consecutive vars merge (A A A) -> A{3} +CREATE TEMP VIEW v_opt_merge AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A A A) + DEFINE + A AS price >= 140 AND price <= 150 +); +SELECT pg_get_viewdef('v_opt_merge'); -- original: (a a a) + pg_get_viewdef +--------------------------------------------------------------------------------------- + SELECT company, + + tdate, + + price, + + count(*) OVER w AS count + + FROM stock + + WINDOW w AS (PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + + AFTER MATCH SKIP PAST LAST ROW + + INITIAL + + PATTERN (a a a) + + DEFINE + + a AS ((price >= 140) AND (price <= 150)) ); +(1 row) + +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_merge; -- optimized: a{3} + QUERY PLAN +---------------------------------------------------------------------------------------------------- + Subquery Scan on v_opt_merge + -> WindowAgg + Window: w AS (PARTITION BY stock.company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) + Pattern: a{3} + -> Sort + Sort Key: stock.company + -> Seq Scan on stock +(7 rows) + +-- Test: quantified vars merge (A A+ A) -> A{3,} +CREATE TEMP VIEW v_opt_merge_quant AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A A+ A) + DEFINE + A AS price > 100 +); +SELECT pg_get_viewdef('v_opt_merge_quant'); -- original: (a a+ a) + pg_get_viewdef +--------------------------------------------------------------------------------------- + SELECT company, + + tdate, + + price, + + count(*) OVER w AS count + + FROM stock + + WINDOW w AS (PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + + AFTER MATCH SKIP PAST LAST ROW + + INITIAL + + PATTERN (a a+ a) + + DEFINE + + a AS (price > 100) ); +(1 row) + +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_merge_quant; -- optimized: a{3,} + QUERY PLAN +---------------------------------------------------------------------------------------------------- + Subquery Scan on v_opt_merge_quant + -> WindowAgg + Window: w AS (PARTITION BY stock.company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) + Pattern: a{3,} + -> Sort + Sort Key: stock.company + -> Seq Scan on stock +(7 rows) + +-- Test: merge two unbounded (A+ A+) -> A{2,} +CREATE TEMP VIEW v_opt_merge_unbounded AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A+ A+) + DEFINE + A AS price > 100 +); +SELECT pg_get_viewdef('v_opt_merge_unbounded'); -- original: (a+ a+) + pg_get_viewdef +--------------------------------------------------------------------------------------- + SELECT company, + + tdate, + + price, + + count(*) OVER w AS count + + FROM stock + + WINDOW w AS (PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + + AFTER MATCH SKIP PAST LAST ROW + + INITIAL + + PATTERN (a+ a+) + + DEFINE + + a AS (price > 100) ); +(1 row) + +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_merge_unbounded; -- optimized: a{2,} + QUERY PLAN +---------------------------------------------------------------------------------------------------- + Subquery Scan on v_opt_merge_unbounded + -> WindowAgg + Window: w AS (PARTITION BY stock.company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) + Pattern: a{2,} + -> Sort + Sort Key: stock.company + -> Seq Scan on stock +(7 rows) + +-- Test: merge with zero-min (A* A+) -> A+ +CREATE TEMP VIEW v_opt_merge_star AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A* A+) + DEFINE + A AS price > 100 +); +SELECT pg_get_viewdef('v_opt_merge_star'); -- original: (a* a+) + pg_get_viewdef +--------------------------------------------------------------------------------------- + SELECT company, + + tdate, + + price, + + count(*) OVER w AS count + + FROM stock + + WINDOW w AS (PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + + AFTER MATCH SKIP PAST LAST ROW + + INITIAL + + PATTERN (a* a+) + + DEFINE + + a AS (price > 100) ); +(1 row) + +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_merge_star; -- optimized: a+ + QUERY PLAN +---------------------------------------------------------------------------------------------------- + Subquery Scan on v_opt_merge_star + -> WindowAgg + Window: w AS (PARTITION BY stock.company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) + Pattern: a+ + -> Sort + Sort Key: stock.company + -> Seq Scan on stock +(7 rows) + +-- Test: complex merge (A A{2} A+ A{3}) -> A{7,} +CREATE TEMP VIEW v_opt_merge_complex AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A A{2} A+ A{3}) + DEFINE + A AS price > 100 +); +SELECT pg_get_viewdef('v_opt_merge_complex'); -- original: (a a{2} a+ a{3}) + pg_get_viewdef +--------------------------------------------------------------------------------------- + SELECT company, + + tdate, + + price, + + count(*) OVER w AS count + + FROM stock + + WINDOW w AS (PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + + AFTER MATCH SKIP PAST LAST ROW + + INITIAL + + PATTERN (a a{2} a+ a{3}) + + DEFINE + + a AS (price > 100) ); +(1 row) + +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_merge_complex; -- optimized: a{7,} + QUERY PLAN +---------------------------------------------------------------------------------------------------- + Subquery Scan on v_opt_merge_complex + -> WindowAgg + Window: w AS (PARTITION BY stock.company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) + Pattern: a{7,} + -> Sort + Sort Key: stock.company + -> Seq Scan on stock +(7 rows) + +-- Test: group merge ((A B) (A B)+) -> (A B){2,} +CREATE TEMP VIEW v_opt_merge_group AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN ((A B) (A B)+) + DEFINE + A AS price > 100, + B AS price <= 100 +); +SELECT pg_get_viewdef('v_opt_merge_group'); -- original: ((a b) (a b)+) + pg_get_viewdef +--------------------------------------------------------------------------------------- + SELECT company, + + tdate, + + price, + + count(*) OVER w AS count + + FROM stock + + WINDOW w AS (PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + + AFTER MATCH SKIP PAST LAST ROW + + INITIAL + + PATTERN ((a b) (a b)+) + + DEFINE + + a AS (price > 100), + + b AS (price <= 100) ); +(1 row) + +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_merge_group; -- expected: (a b){2,} + QUERY PLAN +---------------------------------------------------------------------------------------------------- + Subquery Scan on v_opt_merge_group + -> WindowAgg + Window: w AS (PARTITION BY stock.company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) + Pattern: (a b){2,} + -> Sort + Sort Key: stock.company + -> Seq Scan on stock +(7 rows) + +-- Test: group merge A B (A B)+ -> (A B){2,} +CREATE TEMP VIEW v_opt_merge_group2 AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A B (A B)+) + DEFINE + A AS price > 100, + B AS price <= 100 +); +SELECT pg_get_viewdef('v_opt_merge_group2'); -- original: (a b (a b)+) + pg_get_viewdef +--------------------------------------------------------------------------------------- + SELECT company, + + tdate, + + price, + + count(*) OVER w AS count + + FROM stock + + WINDOW w AS (PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + + AFTER MATCH SKIP PAST LAST ROW + + INITIAL + + PATTERN (a b (a b)+) + + DEFINE + + a AS (price > 100), + + b AS (price <= 100) ); +(1 row) + +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_merge_group2; -- expected: (a b){2,} + QUERY PLAN +---------------------------------------------------------------------------------------------------- + Subquery Scan on v_opt_merge_group2 + -> WindowAgg + Window: w AS (PARTITION BY stock.company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) + Pattern: (a b){2,} + -> Sort + Sort Key: stock.company + -> Seq Scan on stock +(7 rows) + +-- Test: group merge (A B) (A B)+ (A B) -> (A B){3,} +CREATE TEMP VIEW v_opt_merge_group3 AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN ((A B) (A B)+ (A B)) + DEFINE + A AS price > 100, + B AS price <= 100 +); +SELECT pg_get_viewdef('v_opt_merge_group3'); -- original: ((a b) (a b)+ (a b)) + pg_get_viewdef +--------------------------------------------------------------------------------------- + SELECT company, + + tdate, + + price, + + count(*) OVER w AS count + + FROM stock + + WINDOW w AS (PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + + AFTER MATCH SKIP PAST LAST ROW + + INITIAL + + PATTERN ((a b) (a b)+ (a b)) + + DEFINE + + a AS (price > 100), + + b AS (price <= 100) ); +(1 row) + +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_merge_group3; -- expected: (a b){3,} + QUERY PLAN +---------------------------------------------------------------------------------------------------- + Subquery Scan on v_opt_merge_group3 + -> WindowAgg + Window: w AS (PARTITION BY stock.company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) + Pattern: (a b){3,} + -> Sort + Sort Key: stock.company + -> Seq Scan on stock +(7 rows) + +-- Test: group merge A B A B (A B)+ A B A B -> (A B){5,} +CREATE TEMP VIEW v_opt_merge_group4 AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A B A B (A B)+ A B A B) + DEFINE + A AS price > 100, + B AS price <= 100 +); +SELECT pg_get_viewdef('v_opt_merge_group4'); -- original: (a b a b (a b)+ a b a b) + pg_get_viewdef +--------------------------------------------------------------------------------------- + SELECT company, + + tdate, + + price, + + count(*) OVER w AS count + + FROM stock + + WINDOW w AS (PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + + AFTER MATCH SKIP PAST LAST ROW + + INITIAL + + PATTERN (a b a b (a b)+ a b a b) + + DEFINE + + a AS (price > 100), + + b AS (price <= 100) ); +(1 row) + +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_merge_group4; -- expected: (a b){5,} + QUERY PLAN +---------------------------------------------------------------------------------------------------- + Subquery Scan on v_opt_merge_group4 + -> WindowAgg + Window: w AS (PARTITION BY stock.company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) + Pattern: (a b){5,} + -> Sort + Sort Key: stock.company + -> Seq Scan on stock +(7 rows) + +-- Test: group merge C A B (A B)+ A B C -> C (A B){3,} C +CREATE TEMP VIEW v_opt_merge_group5 AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (C A B (A B)+ A B C) + DEFINE + A AS price > 100, + B AS price <= 100, + C AS price > 200 +); +SELECT pg_get_viewdef('v_opt_merge_group5'); -- original: (c a b (a b)+ a b c) + pg_get_viewdef +--------------------------------------------------------------------------------------- + SELECT company, + + tdate, + + price, + + count(*) OVER w AS count + + FROM stock + + WINDOW w AS (PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + + AFTER MATCH SKIP PAST LAST ROW + + INITIAL + + PATTERN (c a b (a b)+ a b c) + + DEFINE + + a AS (price > 100), + + b AS (price <= 100), + + c AS (price > 200) ); +(1 row) + +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_merge_group5; -- expected: c (a b){3,} c + QUERY PLAN +---------------------------------------------------------------------------------------------------- + Subquery Scan on v_opt_merge_group5 + -> WindowAgg + Window: w AS (PARTITION BY stock.company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) + Pattern: c (a b){3,} c + -> Sort + Sort Key: stock.company + -> Seq Scan on stock +(7 rows) + +-- Test {n} quantifier display +CREATE TEMP VIEW v_quantifier_n AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A{3}) + DEFINE + A AS price > 100 +); +SELECT pg_get_viewdef('v_quantifier_n'); + pg_get_viewdef +--------------------------------------------------------------------------------------- + SELECT company, + + tdate, + + price, + + count(*) OVER w AS count + + FROM stock + + WINDOW w AS (PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + + AFTER MATCH SKIP PAST LAST ROW + + INITIAL + + PATTERN (a{3}) + + DEFINE + + a AS (price > 100) ); +(1 row) + +-- Test {n,} quantifier display +CREATE TEMP VIEW v_quantifier_n_plus AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A{2,}) + DEFINE + A AS price > 100 +); +SELECT pg_get_viewdef('v_quantifier_n_plus'); + pg_get_viewdef +--------------------------------------------------------------------------------------- + SELECT company, + + tdate, + + price, + + count(*) OVER w AS count + + FROM stock + + WINDOW w AS (PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + + AFTER MATCH SKIP PAST LAST ROW + + INITIAL + + PATTERN (a{2,}) + + DEFINE + + a AS (price > 100) ); +(1 row) + +-- Test: flatten nested SEQ (A (B C)) -> A B C +CREATE TEMP VIEW v_opt_flatten_seq AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A (B C)) + DEFINE + A AS price > 100, + B AS price > 150, + C AS price < 150 +); +SELECT pg_get_viewdef('v_opt_flatten_seq'); -- original: (a (b c)) + pg_get_viewdef +--------------------------------------------------------------------------------------- + SELECT company, + + tdate, + + price, + + count(*) OVER w AS count + + FROM stock + + WINDOW w AS (PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + + AFTER MATCH SKIP PAST LAST ROW + + INITIAL + + PATTERN (a (b c)) + + DEFINE + + a AS (price > 100), + + b AS (price > 150), + + c AS (price < 150) ); +(1 row) + +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_flatten_seq; -- optimized: a b c + QUERY PLAN +---------------------------------------------------------------------------------------------------- + Subquery Scan on v_opt_flatten_seq + -> WindowAgg + Window: w AS (PARTITION BY stock.company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) + Pattern: a b c + -> Sort + Sort Key: stock.company + -> Seq Scan on stock +(7 rows) + +-- Test: flatten nested ALT (A | (B | C)) -> (A | B | C) +CREATE TEMP VIEW v_opt_flatten_alt AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN ((A | (B | C))+) + DEFINE + A AS price > 200, + B AS price > 100, + C AS price <= 100 +); +SELECT pg_get_viewdef('v_opt_flatten_alt'); -- original: ((a | (b | c))+) + pg_get_viewdef +--------------------------------------------------------------------------------------- + SELECT company, + + tdate, + + price, + + count(*) OVER w AS count + + FROM stock + + WINDOW w AS (PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + + AFTER MATCH SKIP PAST LAST ROW + + INITIAL + + PATTERN ((a | (b | c))+) + + DEFINE + + a AS (price > 200), + + b AS (price > 100), + + c AS (price <= 100) ); +(1 row) -SELECT *, count(*) OVER w FROM t1, t2 -WINDOW w AS ( - PARTITION BY t1.i +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_flatten_alt; -- optimized: ((a | b | c))+ + QUERY PLAN +---------------------------------------------------------------------------------------------------- + Subquery Scan on v_opt_flatten_alt + -> WindowAgg + Window: w AS (PARTITION BY stock.company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) + Pattern: ((a | b | c))+ + -> Sort + Sort Key: stock.company + -> Seq Scan on stock +(7 rows) + +-- Test: unwrap GROUP{1,1} ((A)) -> A +CREATE TEMP VIEW v_opt_unwrap_group AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING INITIAL - PATTERN (A) + PATTERN (((A))) DEFINE - A AS v1 <= 11 AND v2 <= 11 + A AS price > 100 ); - i | v1 | j | v2 | count ----+----+---+----+------- - 1 | 10 | 2 | 10 | 1 - 1 | 10 | 2 | 11 | 1 - 1 | 10 | 2 | 12 | 0 - 1 | 11 | 2 | 10 | 1 - 1 | 11 | 2 | 11 | 1 - 1 | 11 | 2 | 12 | 0 - 1 | 12 | 2 | 10 | 0 - 1 | 12 | 2 | 11 | 0 - 1 | 12 | 2 | 12 | 0 -(9 rows) +SELECT pg_get_viewdef('v_opt_unwrap_group'); -- original: (((a))) + pg_get_viewdef +--------------------------------------------------------------------------------------- + SELECT company, + + tdate, + + price, + + count(*) OVER w AS count + + FROM stock + + WINDOW w AS (PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + + AFTER MATCH SKIP PAST LAST ROW + + INITIAL + + PATTERN (((a))) + + DEFINE + + a AS (price > 100) ); +(1 row) --- WITH case -WITH wstock AS ( - SELECT * FROM stock WHERE tdate < '2023-07-08' -) -SELECT tdate, price, -first_value(tdate) OVER w, -count(*) OVER w - FROM wstock +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_unwrap_group; -- optimized: a + QUERY PLAN +---------------------------------------------------------------------------------------------------- + Subquery Scan on v_opt_unwrap_group + -> WindowAgg + Window: w AS (PARTITION BY stock.company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) + Pattern: a + -> Sort + Sort Key: stock.company + -> Seq Scan on stock +(7 rows) + +-- Test: quantifier multiplication (A{2}){3} -> A{6} +CREATE TEMP VIEW v_opt_quant_mult AS +SELECT company, tdate, price, count(*) OVER w + FROM stock WINDOW w AS ( PARTITION BY company - ORDER BY tdate ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING INITIAL - PATTERN (START UP+ DOWN+) + PATTERN ((A{2}){3}) DEFINE - START AS TRUE, - UP AS price > PREV(price), - DOWN AS price < PREV(price) + A AS price > 100 ); - tdate | price | first_value | count -------------+-------+-------------+------- - 07-01-2023 | 100 | 07-01-2023 | 4 - 07-02-2023 | 200 | | 0 - 07-03-2023 | 150 | | 0 - 07-04-2023 | 140 | | 0 - 07-05-2023 | 150 | | 0 - 07-06-2023 | 90 | | 0 - 07-07-2023 | 110 | | 0 - 07-01-2023 | 50 | 07-01-2023 | 4 - 07-02-2023 | 2000 | | 0 - 07-03-2023 | 1500 | | 0 - 07-04-2023 | 1400 | | 0 - 07-05-2023 | 1500 | | 0 - 07-06-2023 | 60 | | 0 - 07-07-2023 | 1100 | | 0 -(14 rows) +SELECT pg_get_viewdef('v_opt_quant_mult'); -- original: ((a{2}){3}) + pg_get_viewdef +--------------------------------------------------------------------------------------- + SELECT company, + + tdate, + + price, + + count(*) OVER w AS count + + FROM stock + + WINDOW w AS (PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + + AFTER MATCH SKIP PAST LAST ROW + + INITIAL + + PATTERN ((a{2}){3}) + + DEFINE + + a AS (price > 100) ); +(1 row) --- PREV has multiple column reference -CREATE TEMP TABLE rpr1 (id INTEGER, i SERIAL, j INTEGER); -INSERT INTO rpr1(id, j) SELECT 1, g*2 FROM generate_series(1, 10) AS g; -SELECT id, i, j, count(*) OVER w - FROM rpr1 +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_quant_mult; -- optimized: a{6} + QUERY PLAN +---------------------------------------------------------------------------------------------------- + Subquery Scan on v_opt_quant_mult + -> WindowAgg + Window: w AS (PARTITION BY stock.company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) + Pattern: a{6} + -> Sort + Sort Key: stock.company + -> Seq Scan on stock +(7 rows) + +-- Test: quantifier multiplication (A{2,4}){3} -> A{6,12} +CREATE TEMP VIEW v_opt_quant_mult_range AS +SELECT company, tdate, price, count(*) OVER w + FROM stock WINDOW w AS ( - PARTITION BY id - ORDER BY i + PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING - AFTER MATCH SKIP PAST LAST ROW INITIAL - PATTERN (START COND+) + PATTERN ((A{2,4}){3}) DEFINE - START AS TRUE, - COND AS PREV(i + j + 1) < 10 + A AS price > 100 ); - id | i | j | count -----+----+----+------- - 1 | 1 | 2 | 3 - 1 | 2 | 4 | 0 - 1 | 3 | 6 | 0 - 1 | 4 | 8 | 0 - 1 | 5 | 10 | 0 - 1 | 6 | 12 | 0 - 1 | 7 | 14 | 0 - 1 | 8 | 16 | 0 - 1 | 9 | 18 | 0 - 1 | 10 | 20 | 0 -(10 rows) - --- Smoke test for larger partitions. -WITH s AS ( - SELECT v, count(*) OVER w AS c - FROM (SELECT generate_series(1, 5000) v) - WINDOW w AS ( - ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING - AFTER MATCH SKIP PAST LAST ROW - INITIAL - PATTERN ( r+ ) - DEFINE r AS TRUE - ) -) --- Should be exactly one long match across all rows. -SELECT * FROM s WHERE c > 0; - v | c ----+------ - 1 | 5000 +SELECT pg_get_viewdef('v_opt_quant_mult_range'); -- original: ((a{2,4}){3}) + pg_get_viewdef +--------------------------------------------------------------------------------------- + SELECT company, + + tdate, + + price, + + count(*) OVER w AS count + + FROM stock + + WINDOW w AS (PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + + AFTER MATCH SKIP PAST LAST ROW + + INITIAL + + PATTERN ((a{2,4}){3}) + + DEFINE + + a AS (price > 100) ); (1 row) -WITH s AS ( - SELECT v, count(*) OVER w AS c - FROM (SELECT generate_series(1, 5000) v) - WINDOW w AS ( - ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING - AFTER MATCH SKIP PAST LAST ROW - INITIAL - PATTERN ( r ) - DEFINE r AS TRUE - ) -) --- Every row should be its own match. -SELECT count(*) FROM s WHERE c > 0; - count -------- - 5000 -(1 row) +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_quant_mult_range; -- optimized: a{6,12} + QUERY PLAN +---------------------------------------------------------------------------------------------------- + Subquery Scan on v_opt_quant_mult_range + -> WindowAgg + Window: w AS (PARTITION BY stock.company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) + Pattern: a{6,12} + -> Sort + Sort Key: stock.company + -> Seq Scan on stock +(7 rows) --- View and pg_get_viewdef tests. -CREATE TEMP VIEW v_window AS -SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w, - nth_value(tdate, 2) OVER w AS nth_second +-- Test: quantifier multiplication (A{2}){3,5} -> A{6,10} +CREATE TEMP VIEW v_opt_quant_mult_range2 AS +SELECT company, tdate, price, count(*) OVER w FROM stock WINDOW w AS ( PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING INITIAL - PATTERN (START UP+ DOWN+) + PATTERN ((A{2}){3,5}) DEFINE - START AS TRUE, - UP AS price > PREV(price), - DOWN AS price < PREV(price) + A AS price > 100 ); -SELECT * FROM v_window; - company | tdate | price | first_value | last_value | nth_second -----------+------------+-------+-------------+------------+------------ - company1 | 07-01-2023 | 100 | 100 | 140 | 07-02-2023 - company1 | 07-02-2023 | 200 | | | - company1 | 07-03-2023 | 150 | | | - company1 | 07-04-2023 | 140 | | | - company1 | 07-05-2023 | 150 | | | - company1 | 07-06-2023 | 90 | 90 | 120 | 07-07-2023 - company1 | 07-07-2023 | 110 | | | - company1 | 07-08-2023 | 130 | | | - company1 | 07-09-2023 | 120 | | | - company1 | 07-10-2023 | 130 | | | - company2 | 07-01-2023 | 50 | 50 | 1400 | 07-02-2023 - company2 | 07-02-2023 | 2000 | | | - company2 | 07-03-2023 | 1500 | | | - company2 | 07-04-2023 | 1400 | | | - company2 | 07-05-2023 | 1500 | | | - company2 | 07-06-2023 | 60 | 60 | 1200 | 07-07-2023 - company2 | 07-07-2023 | 1100 | | | - company2 | 07-08-2023 | 1300 | | | - company2 | 07-09-2023 | 1200 | | | - company2 | 07-10-2023 | 1300 | | | -(20 rows) - -SELECT pg_get_viewdef('v_window'); +SELECT pg_get_viewdef('v_opt_quant_mult_range2'); -- original: ((a{2}){3,5}) pg_get_viewdef --------------------------------------------------------------------------------------- SELECT company, + tdate, + price, + - first_value(price) OVER w AS first_value, + - last_value(price) OVER w AS last_value, + - nth_value(tdate, 2) OVER w AS nth_second + + count(*) OVER w AS count + FROM stock + WINDOW w AS (PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + INITIAL + - PATTERN (start up+ down+) + + PATTERN ((a{2}){3,5}) + DEFINE + - start AS true, + - up AS (price > prev(price)), + - down AS (price < prev(price)) ); + a AS (price > 100) ); (1 row) +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_quant_mult_range2; -- optimized: a{6,10} + QUERY PLAN +---------------------------------------------------------------------------------------------------- + Subquery Scan on v_opt_quant_mult_range2 + -> WindowAgg + Window: w AS (PARTITION BY stock.company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) + Pattern: a{6,10} + -> Sort + Sort Key: stock.company + -> Seq Scan on stock +(7 rows) + -- -- Error cases -- @@ -1030,7 +2162,7 @@ SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER UP AS price > PREV(1), DOWN AS price < PREV(1) ); -ERROR: unsupported quantifier +ERROR: unsupported quantifier "~" LINE 9: PATTERN (START UP~ DOWN+) ^ SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w @@ -1047,9 +2179,7 @@ SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER UP AS price > PREV(1), DOWN AS price < PREV(1) ); -ERROR: unsupported quantifier -LINE 9: PATTERN (START UP+? DOWN+) - ^ +ERROR: row pattern navigation operation's argument must include at least one column reference -- Number of row pattern definition variable names must not exceed 26 -- Ok SELECT * FROM (SELECT 1 AS x) t @@ -1094,3 +2224,513 @@ PREV(price) c1 | 07-04-2023 | 150 | 0 (4 rows) +-- Overlapping match tests (requires multi-context for correct behavior) +-- Using array flags: 'X' = ANY(flags) for multi-TRUE support +-- Test 1: A B C D E | B C D | C D E F - three overlapping patterns +-- Different end points: B C D (4), A B C D E (5), C D E F (6) +WITH test_overlap1 AS ( + SELECT * FROM (VALUES + (1, ARRAY['A']), + (2, ARRAY['B']), + (3, ARRAY['C']), + (4, ARRAY['D']), + (5, ARRAY['E']), + (6, ARRAY['F']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_overlap1 +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + PATTERN (A B C D E | B C D | C D E F) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags), + C AS 'C' = ANY(flags), + D AS 'D' = ANY(flags), + E AS 'E' = ANY(flags), + F AS 'F' = ANY(flags) +); + id | flags | match_start | match_end +----+-------+-------------+----------- + 1 | {A} | 1 | 5 + 2 | {B} | | + 3 | {C} | | + 4 | {D} | | + 5 | {E} | | + 6 | {F} | | +(6 rows) + +WITH test_overlap1 AS ( + SELECT * FROM (VALUES + (1, ARRAY['A']), + (2, ARRAY['B']), + (3, ARRAY['C']), + (4, ARRAY['D']), + (5, ARRAY['E']), + (6, ARRAY['F']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_overlap1 +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + PATTERN (A B C D E | B C D | C D E F) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags), + C AS 'C' = ANY(flags), + D AS 'D' = ANY(flags), + E AS 'E' = ANY(flags), + F AS 'F' = ANY(flags) +); + id | flags | match_start | match_end +----+-------+-------------+----------- + 1 | {A} | 1 | 5 + 2 | {B} | 2 | 4 + 3 | {C} | 3 | 6 + 4 | {D} | | + 5 | {E} | | + 6 | {F} | | +(6 rows) + +-- PAST LAST: only one match +-- TO NEXT ROW with multi-context: three matches +-- Row 1: A B C D E (1-5) +-- Row 2: B C D (2-4) <- ends first! +-- Row 3: C D E F (3-6) <- ends last! +-- Test 2: A B+ C | B+ D - long B sequence with different endings +WITH test_overlap2 AS ( + SELECT * FROM (VALUES + (1, ARRAY['A']), + (2, ARRAY['B']), + (3, ARRAY['B']), + (4, ARRAY['B']), + (5, ARRAY['B']), + (6, ARRAY['C']), + (7, ARRAY['B']), + (8, ARRAY['B']), + (9, ARRAY['B']), + (10, ARRAY['D']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_overlap2 +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + PATTERN (A B+ C | B+ D) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags), + C AS 'C' = ANY(flags), + D AS 'D' = ANY(flags) +); + id | flags | match_start | match_end +----+-------+-------------+----------- + 1 | {A} | 1 | 6 + 2 | {B} | | + 3 | {B} | | + 4 | {B} | | + 5 | {B} | | + 6 | {C} | | + 7 | {B} | 7 | 10 + 8 | {B} | | + 9 | {B} | | + 10 | {D} | | +(10 rows) + +-- Current result (correct): +-- Row 1: A B+ C (1-6) +-- Row 7-9: B+ D (7-10, 8-10, 9-10) +-- Note: Row 2-6 cannot match B+ D because Row 6 is C, not D +-- With absorption: 8-10 and 9-10 would be absorbed by 7-10 (earlier context covers later) +-- Test 3: Greedy quantifier with late failure - A B C+ D | A B +-- Pattern expects D after C+, but E comes instead ("betrayal") +WITH test_betrayal AS ( + SELECT * FROM (VALUES + (1, ARRAY['A']), + (2, ARRAY['B']), + (3, ARRAY['C']), + (4, ARRAY['C']), + (5, ARRAY['C']), + (6, ARRAY['E']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_betrayal +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + PATTERN (A B C+ D | A B) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags), + C AS 'C' = ANY(flags), + D AS 'D' = ANY(flags) +); + id | flags | match_start | match_end +----+-------+-------------+----------- + 1 | {A} | 1 | 2 + 2 | {B} | | + 3 | {C} | | + 4 | {C} | | + 5 | {C} | | + 6 | {E} | | +(6 rows) + +-- A B C+ D fails at Row 6 (E instead of D) +-- Question: Does it fallback to A B (1-2)? +-- Test 4: Lexical Order test - A B C | A B C D E +-- SQL standard: first matching alternative wins +WITH test_lexical AS ( + SELECT * FROM (VALUES + (1, ARRAY['A']), + (2, ARRAY['B']), + (3, ARRAY['C']), + (4, ARRAY['D']), + (5, ARRAY['E']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_lexical +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + PATTERN (A B C | A B C D E) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags), + C AS 'C' = ANY(flags), + D AS 'D' = ANY(flags), + E AS 'E' = ANY(flags) +); + id | flags | match_start | match_end +----+-------+-------------+----------- + 1 | {A} | 1 | 3 + 2 | {B} | | + 3 | {C} | | + 4 | {D} | | + 5 | {E} | | +(5 rows) + +-- SQL standard Lexical Order: A B C (1-3) wins (first alternative) +-- Test 4b: Reversed pattern order - A B C D E | A B C +WITH test_lexical2 AS ( + SELECT * FROM (VALUES + (1, ARRAY['A']), + (2, ARRAY['B']), + (3, ARRAY['C']), + (4, ARRAY['D']), + (5, ARRAY['E']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_lexical2 +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + PATTERN (A B C D E | A B C) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags), + C AS 'C' = ANY(flags), + D AS 'D' = ANY(flags), + E AS 'E' = ANY(flags) +); + id | flags | match_start | match_end +----+-------+-------------+----------- + 1 | {A} | 1 | 5 + 2 | {B} | | + 3 | {C} | | + 4 | {D} | | + 5 | {E} | | +(5 rows) + +-- SQL standard Lexical Order: A B C D E (1-5) wins (first alternative) +-- Test 5: Multiple TRUE in single row (overlapping pattern variables) +-- Each row matches multiple DEFINE conditions simultaneously +WITH test_multi_true AS ( + SELECT * FROM (VALUES + (1, ARRAY['A','B']), -- A and B both TRUE + (2, ARRAY['B','C']), -- B and C both TRUE + (3, ARRAY['C','D']), -- C and D both TRUE + (4, ARRAY['D','E']), -- D and E both TRUE + (5, ARRAY['E','_']) -- E only + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_multi_true +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + PATTERN (A B C D E) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags), + C AS 'C' = ANY(flags), + D AS 'D' = ANY(flags), + E AS 'E' = ANY(flags) +); + id | flags | match_start | match_end +----+-------+-------------+----------- + 1 | {A,B} | 1 | 5 + 2 | {B,C} | | + 3 | {C,D} | | + 4 | {D,E} | | + 5 | {E,_} | | +(5 rows) + +-- Row 1: A=T, B=T → matches A +-- Row 2: B=T, C=T → matches B +-- Row 3: C=T, D=T → matches C +-- Row 4: D=T, E=T → matches D +-- Row 5: E=T → matches E +-- Result: match 1-5 (A B C D E) +-- Test 6: Diagonal pattern with multi-TRUE (shifted overlap) +WITH test_diagonal AS ( + SELECT * FROM (VALUES + (1, ARRAY['A','_']), + (2, ARRAY['B','A']), + (3, ARRAY['C','B']), + (4, ARRAY['D','C']), + (5, ARRAY['_','D']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_diagonal +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + PATTERN (A B C D) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags), + C AS 'C' = ANY(flags), + D AS 'D' = ANY(flags) +); + id | flags | match_start | match_end +----+-------+-------------+----------- + 1 | {A,_} | 1 | 4 + 2 | {B,A} | 2 | 5 + 3 | {C,B} | | + 4 | {D,C} | | + 5 | {_,D} | | +(5 rows) + +-- Possible matches: +-- Start Row 1: A(1) B(2) C(3) D(4) → 1-4 +-- Start Row 2: A(2) B(3) C(4) D(5) → 2-5 (because Row 2 has A too!) +-- =================================================================== +-- Context Absorption Tests +-- =================================================================== +-- Test absorption 1: Basic A+ pattern - later contexts absorbed by earlier +WITH test_absorb_basic AS ( + SELECT * FROM (VALUES + (1, ARRAY['A']), + (2, ARRAY['A']), + (3, ARRAY['A']), + (4, ARRAY['A']), + (5, ARRAY['B']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_absorb_basic +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + PATTERN (A+) + DEFINE A AS 'A' = ANY(flags) +); + id | flags | match_start | match_end +----+-------+-------------+----------- + 1 | {A} | 1 | 4 + 2 | {A} | | + 3 | {A} | | + 4 | {A} | | + 5 | {B} | | +(5 rows) + +-- Pattern A+ is absorbable (unbounded first element, only one unbounded) +-- Without absorption: 4 matches (1-4, 2-4, 3-4, 4-4) +-- With absorption: Row 1 match (1-4), rows 2-4 absorbed +-- Test absorption 2: A+ B pattern - absorption with fixed suffix +WITH test_absorb_suffix AS ( + SELECT * FROM (VALUES + (1, ARRAY['A']), + (2, ARRAY['A']), + (3, ARRAY['A']), + (4, ARRAY['B']), + (5, ARRAY['X']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_absorb_suffix +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + PATTERN (A+ B) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags) +); + id | flags | match_start | match_end +----+-------+-------------+----------- + 1 | {A} | 1 | 4 + 2 | {A} | | + 3 | {A} | | + 4 | {B} | | + 5 | {X} | | +(5 rows) + +-- Pattern A+ B is absorbable (A+ unbounded first, B bounded suffix) +-- All potential matches end at same row (row 4 with B) +-- With absorption: Row 1 match (1-4), rows 2-3 absorbed +-- Test absorption 3: Per-branch absorption with ALT (B+ C | B+ D) +WITH test_absorb_alt AS ( + SELECT * FROM (VALUES + (1, ARRAY['B']), + (2, ARRAY['B']), + (3, ARRAY['B']), + (4, ARRAY['D']), + (5, ARRAY['X']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_absorb_alt +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + PATTERN (B+ C | B+ D) + DEFINE + B AS 'B' = ANY(flags), + C AS 'C' = ANY(flags), + D AS 'D' = ANY(flags) +); + id | flags | match_start | match_end +----+-------+-------------+----------- + 1 | {B} | 1 | 4 + 2 | {B} | | + 3 | {B} | | + 4 | {D} | | + 5 | {X} | | +(5 rows) + +-- Both branches B+ C and B+ D are absorbable (B+ unbounded first) +-- B+ D branch matches: potential 1-4, 2-4, 3-4 +-- Row 1 (1-4) absorbs Row 2 (2-4) and Row 3 (3-4) - same endpoint +-- Test absorption 4: Non-absorbable pattern (A B+ - unbounded not first) +WITH test_no_absorb AS ( + SELECT * FROM (VALUES + (1, ARRAY['A']), + (2, ARRAY['B']), + (3, ARRAY['B']), + (4, ARRAY['B']), + (5, ARRAY['X']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_no_absorb +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + PATTERN (A B+) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags) +); + id | flags | match_start | match_end +----+-------+-------------+----------- + 1 | {A} | 1 | 4 + 2 | {B} | | + 3 | {B} | | + 4 | {B} | | + 5 | {X} | | +(5 rows) + +-- Pattern A B+ is NOT absorbable (A bounded first, B+ unbounded but not first) +-- Only Row 1 can start match (only row with A), so only one match: 1-4 +-- Test absorption 5: GROUP merge enables absorption +WITH test_absorb_group AS ( + SELECT * FROM (VALUES + (1, ARRAY['A']), + (2, ARRAY['B']), + (3, ARRAY['A']), + (4, ARRAY['B']), + (5, ARRAY['A']), + (6, ARRAY['B']), + (7, ARRAY['X']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_absorb_group +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + PATTERN ((A B) (A B)+) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags) +); + id | flags | match_start | match_end +----+-------+-------------+----------- + 1 | {A} | 1 | 6 + 2 | {B} | | + 3 | {A} | | + 4 | {B} | | + 5 | {A} | | + 6 | {B} | | + 7 | {X} | | +(7 rows) + +-- Pattern optimized: (A B) (A B)+ -> (A B){2,} +-- Potential matches: 1-6 (3 reps), 3-6 (2 reps), 5-6 needs 2 reps (fail) +-- Row 1 (1-6) absorbs Row 3 (3-6) - same endpoint, top-level unbounded GROUP +-- Test absorption 6: Multiple unbounded - NOT absorbable +WITH test_multi_unbounded AS ( + SELECT * FROM (VALUES + (1, ARRAY['A']), + (2, ARRAY['A']), + (3, ARRAY['B']), + (4, ARRAY['B']), + (5, ARRAY['X']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_multi_unbounded +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + PATTERN (A+ B+) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags) +); + id | flags | match_start | match_end +----+-------+-------------+----------- + 1 | {A} | 1 | 4 + 2 | {A} | 2 | 4 + 3 | {B} | | + 4 | {B} | | + 5 | {X} | | +(5 rows) + +-- Pattern A+ B+ has TWO unbounded elements - NOT absorbable +-- Greedy A+ consumes rows 1-2, then B+ matches 3-4 +-- Row 1: 1-4, Row 2: 2-4 (different A+ counts, not absorbed) diff --git a/src/test/regress/sql/rpr.sql b/src/test/regress/sql/rpr.sql index 640e9807f59..fa8aa50fcb9 100644 --- a/src/test/regress/sql/rpr.sql +++ b/src/test/regress/sql/rpr.sql @@ -90,6 +90,91 @@ SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER DOWN AS price < PREV(price) ); +-- test using alternation (|) with sequence +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START (UP | DOWN)) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- test using alternation (|) with group quantifier +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START (UP | DOWN)+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- test using nested alternation +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START ((UP DOWN) | FLAT)+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price), + FLAT AS price = PREV(price) +); + +-- test using group with quantifier +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN ((UP DOWN)+) + DEFINE + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- test using absolute threshold values (not relative PREV) +-- HIGH: price > 150, LOW: price < 100, MID: neutral range +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (LOW MID* HIGH) + DEFINE + LOW AS price < 100, + MID AS price >= 100 AND price <= 150, + HIGH AS price > 150 +); + +-- test threshold-based pattern with alternation +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (LOW (MID | HIGH)+) + DEFINE + LOW AS price < 100, + MID AS price >= 100 AND price <= 150, + HIGH AS price > 150 +); + -- basic test with none-greedy pattern SELECT company, tdate, price, count(*) OVER w FROM stock @@ -102,6 +187,42 @@ SELECT company, tdate, price, count(*) OVER w A AS price >= 140 AND price <= 150 ); +-- test using {n} quantifier (A A A should be optimized to A{3}) +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A{3}) + DEFINE + A AS price >= 140 AND price <= 150 +); + +-- test using {n,} quantifier (2 or more) +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A{2,}) + DEFINE + A AS price > 100 +); + +-- test using {n,m} quantifier (2 to 4) +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A{2,4}) + DEFINE + A AS price > 100 +); + -- last_value() should remain consistent SELECT company, tdate, price, last_value(price) OVER w FROM stock @@ -405,6 +526,321 @@ SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER SELECT * FROM v_window; SELECT pg_get_viewdef('v_window'); +-- +-- Pattern optimization tests +-- VIEW shows original pattern, EXPLAIN shows optimized pattern +-- + +-- Test: duplicate alternatives removal (A | B | A)+ -> (A | B)+ +CREATE TEMP VIEW v_opt_dup AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN ((A | B | A)+) + DEFINE + A AS price > 100, + B AS price <= 100 +); +SELECT pg_get_viewdef('v_opt_dup'); -- original: ((a | b | a)+) +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_dup; -- optimized: ((a | b)+) + +-- Test: duplicate group removal ((A | B)+ | (A | B)+) -> (A | B)+ +CREATE TEMP VIEW v_opt_dup_group AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN ((A | B)+ | (A | B)+) + DEFINE + A AS price > 100, + B AS price <= 100 +); +SELECT pg_get_viewdef('v_opt_dup_group'); -- original: ((a | b)+ | (a | b)+) +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_dup_group; -- optimized: ((a | b)+) + +-- Test: consecutive vars merge (A A A) -> A{3} +CREATE TEMP VIEW v_opt_merge AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A A A) + DEFINE + A AS price >= 140 AND price <= 150 +); +SELECT pg_get_viewdef('v_opt_merge'); -- original: (a a a) +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_merge; -- optimized: a{3} + +-- Test: quantified vars merge (A A+ A) -> A{3,} +CREATE TEMP VIEW v_opt_merge_quant AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A A+ A) + DEFINE + A AS price > 100 +); +SELECT pg_get_viewdef('v_opt_merge_quant'); -- original: (a a+ a) +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_merge_quant; -- optimized: a{3,} + +-- Test: merge two unbounded (A+ A+) -> A{2,} +CREATE TEMP VIEW v_opt_merge_unbounded AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A+ A+) + DEFINE + A AS price > 100 +); +SELECT pg_get_viewdef('v_opt_merge_unbounded'); -- original: (a+ a+) +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_merge_unbounded; -- optimized: a{2,} + +-- Test: merge with zero-min (A* A+) -> A+ +CREATE TEMP VIEW v_opt_merge_star AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A* A+) + DEFINE + A AS price > 100 +); +SELECT pg_get_viewdef('v_opt_merge_star'); -- original: (a* a+) +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_merge_star; -- optimized: a+ + +-- Test: complex merge (A A{2} A+ A{3}) -> A{7,} +CREATE TEMP VIEW v_opt_merge_complex AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A A{2} A+ A{3}) + DEFINE + A AS price > 100 +); +SELECT pg_get_viewdef('v_opt_merge_complex'); -- original: (a a{2} a+ a{3}) +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_merge_complex; -- optimized: a{7,} + +-- Test: group merge ((A B) (A B)+) -> (A B){2,} +CREATE TEMP VIEW v_opt_merge_group AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN ((A B) (A B)+) + DEFINE + A AS price > 100, + B AS price <= 100 +); +SELECT pg_get_viewdef('v_opt_merge_group'); -- original: ((a b) (a b)+) +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_merge_group; -- expected: (a b){2,} + +-- Test: group merge A B (A B)+ -> (A B){2,} +CREATE TEMP VIEW v_opt_merge_group2 AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A B (A B)+) + DEFINE + A AS price > 100, + B AS price <= 100 +); +SELECT pg_get_viewdef('v_opt_merge_group2'); -- original: (a b (a b)+) +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_merge_group2; -- expected: (a b){2,} + +-- Test: group merge (A B) (A B)+ (A B) -> (A B){3,} +CREATE TEMP VIEW v_opt_merge_group3 AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN ((A B) (A B)+ (A B)) + DEFINE + A AS price > 100, + B AS price <= 100 +); +SELECT pg_get_viewdef('v_opt_merge_group3'); -- original: ((a b) (a b)+ (a b)) +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_merge_group3; -- expected: (a b){3,} + +-- Test: group merge A B A B (A B)+ A B A B -> (A B){5,} +CREATE TEMP VIEW v_opt_merge_group4 AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A B A B (A B)+ A B A B) + DEFINE + A AS price > 100, + B AS price <= 100 +); +SELECT pg_get_viewdef('v_opt_merge_group4'); -- original: (a b a b (a b)+ a b a b) +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_merge_group4; -- expected: (a b){5,} + +-- Test: group merge C A B (A B)+ A B C -> C (A B){3,} C +CREATE TEMP VIEW v_opt_merge_group5 AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (C A B (A B)+ A B C) + DEFINE + A AS price > 100, + B AS price <= 100, + C AS price > 200 +); +SELECT pg_get_viewdef('v_opt_merge_group5'); -- original: (c a b (a b)+ a b c) +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_merge_group5; -- expected: c (a b){3,} c + +-- Test {n} quantifier display +CREATE TEMP VIEW v_quantifier_n AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A{3}) + DEFINE + A AS price > 100 +); +SELECT pg_get_viewdef('v_quantifier_n'); + +-- Test {n,} quantifier display +CREATE TEMP VIEW v_quantifier_n_plus AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A{2,}) + DEFINE + A AS price > 100 +); +SELECT pg_get_viewdef('v_quantifier_n_plus'); + +-- Test: flatten nested SEQ (A (B C)) -> A B C +CREATE TEMP VIEW v_opt_flatten_seq AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A (B C)) + DEFINE + A AS price > 100, + B AS price > 150, + C AS price < 150 +); +SELECT pg_get_viewdef('v_opt_flatten_seq'); -- original: (a (b c)) +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_flatten_seq; -- optimized: a b c + +-- Test: flatten nested ALT (A | (B | C)) -> (A | B | C) +CREATE TEMP VIEW v_opt_flatten_alt AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN ((A | (B | C))+) + DEFINE + A AS price > 200, + B AS price > 100, + C AS price <= 100 +); +SELECT pg_get_viewdef('v_opt_flatten_alt'); -- original: ((a | (b | c))+) +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_flatten_alt; -- optimized: ((a | b | c))+ + +-- Test: unwrap GROUP{1,1} ((A)) -> A +CREATE TEMP VIEW v_opt_unwrap_group AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (((A))) + DEFINE + A AS price > 100 +); +SELECT pg_get_viewdef('v_opt_unwrap_group'); -- original: (((a))) +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_unwrap_group; -- optimized: a + +-- Test: quantifier multiplication (A{2}){3} -> A{6} +CREATE TEMP VIEW v_opt_quant_mult AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN ((A{2}){3}) + DEFINE + A AS price > 100 +); +SELECT pg_get_viewdef('v_opt_quant_mult'); -- original: ((a{2}){3}) +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_quant_mult; -- optimized: a{6} + +-- Test: quantifier multiplication (A{2,4}){3} -> A{6,12} +CREATE TEMP VIEW v_opt_quant_mult_range AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN ((A{2,4}){3}) + DEFINE + A AS price > 100 +); +SELECT pg_get_viewdef('v_opt_quant_mult_range'); -- original: ((a{2,4}){3}) +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_quant_mult_range; -- optimized: a{6,12} + +-- Test: quantifier multiplication (A{2}){3,5} -> A{6,10} +CREATE TEMP VIEW v_opt_quant_mult_range2 AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN ((A{2}){3,5}) + DEFINE + A AS price > 100 +); +SELECT pg_get_viewdef('v_opt_quant_mult_range2'); -- original: ((a{2}){3,5}) +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_quant_mult_range2; -- optimized: a{6,10} + -- -- Error cases -- @@ -565,3 +1001,394 @@ SELECT * FROM (SELECT 1 AS x) t DEFINE START AS TRUE, UP AS price > PREV(price), DOWN AS price < PREV(price) ); + + +-- Overlapping match tests (requires multi-context for correct behavior) +-- Using array flags: 'X' = ANY(flags) for multi-TRUE support + +-- Test 1: A B C D E | B C D | C D E F - three overlapping patterns +-- Different end points: B C D (4), A B C D E (5), C D E F (6) +WITH test_overlap1 AS ( + SELECT * FROM (VALUES + (1, ARRAY['A']), + (2, ARRAY['B']), + (3, ARRAY['C']), + (4, ARRAY['D']), + (5, ARRAY['E']), + (6, ARRAY['F']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_overlap1 +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + PATTERN (A B C D E | B C D | C D E F) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags), + C AS 'C' = ANY(flags), + D AS 'D' = ANY(flags), + E AS 'E' = ANY(flags), + F AS 'F' = ANY(flags) +); + +WITH test_overlap1 AS ( + SELECT * FROM (VALUES + (1, ARRAY['A']), + (2, ARRAY['B']), + (3, ARRAY['C']), + (4, ARRAY['D']), + (5, ARRAY['E']), + (6, ARRAY['F']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_overlap1 +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + PATTERN (A B C D E | B C D | C D E F) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags), + C AS 'C' = ANY(flags), + D AS 'D' = ANY(flags), + E AS 'E' = ANY(flags), + F AS 'F' = ANY(flags) +); +-- PAST LAST: only one match +-- TO NEXT ROW with multi-context: three matches +-- Row 1: A B C D E (1-5) +-- Row 2: B C D (2-4) <- ends first! +-- Row 3: C D E F (3-6) <- ends last! + +-- Test 2: A B+ C | B+ D - long B sequence with different endings +WITH test_overlap2 AS ( + SELECT * FROM (VALUES + (1, ARRAY['A']), + (2, ARRAY['B']), + (3, ARRAY['B']), + (4, ARRAY['B']), + (5, ARRAY['B']), + (6, ARRAY['C']), + (7, ARRAY['B']), + (8, ARRAY['B']), + (9, ARRAY['B']), + (10, ARRAY['D']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_overlap2 +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + PATTERN (A B+ C | B+ D) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags), + C AS 'C' = ANY(flags), + D AS 'D' = ANY(flags) +); +-- Current result (correct): +-- Row 1: A B+ C (1-6) +-- Row 7-9: B+ D (7-10, 8-10, 9-10) +-- Note: Row 2-6 cannot match B+ D because Row 6 is C, not D +-- With absorption: 8-10 and 9-10 would be absorbed by 7-10 (earlier context covers later) + +-- Test 3: Greedy quantifier with late failure - A B C+ D | A B +-- Pattern expects D after C+, but E comes instead ("betrayal") +WITH test_betrayal AS ( + SELECT * FROM (VALUES + (1, ARRAY['A']), + (2, ARRAY['B']), + (3, ARRAY['C']), + (4, ARRAY['C']), + (5, ARRAY['C']), + (6, ARRAY['E']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_betrayal +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + PATTERN (A B C+ D | A B) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags), + C AS 'C' = ANY(flags), + D AS 'D' = ANY(flags) +); +-- A B C+ D fails at Row 6 (E instead of D) +-- Question: Does it fallback to A B (1-2)? + +-- Test 4: Lexical Order test - A B C | A B C D E +-- SQL standard: first matching alternative wins +WITH test_lexical AS ( + SELECT * FROM (VALUES + (1, ARRAY['A']), + (2, ARRAY['B']), + (3, ARRAY['C']), + (4, ARRAY['D']), + (5, ARRAY['E']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_lexical +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + PATTERN (A B C | A B C D E) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags), + C AS 'C' = ANY(flags), + D AS 'D' = ANY(flags), + E AS 'E' = ANY(flags) +); +-- SQL standard Lexical Order: A B C (1-3) wins (first alternative) + +-- Test 4b: Reversed pattern order - A B C D E | A B C +WITH test_lexical2 AS ( + SELECT * FROM (VALUES + (1, ARRAY['A']), + (2, ARRAY['B']), + (3, ARRAY['C']), + (4, ARRAY['D']), + (5, ARRAY['E']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_lexical2 +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + PATTERN (A B C D E | A B C) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags), + C AS 'C' = ANY(flags), + D AS 'D' = ANY(flags), + E AS 'E' = ANY(flags) +); +-- SQL standard Lexical Order: A B C D E (1-5) wins (first alternative) + +-- Test 5: Multiple TRUE in single row (overlapping pattern variables) +-- Each row matches multiple DEFINE conditions simultaneously +WITH test_multi_true AS ( + SELECT * FROM (VALUES + (1, ARRAY['A','B']), -- A and B both TRUE + (2, ARRAY['B','C']), -- B and C both TRUE + (3, ARRAY['C','D']), -- C and D both TRUE + (4, ARRAY['D','E']), -- D and E both TRUE + (5, ARRAY['E','_']) -- E only + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_multi_true +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + PATTERN (A B C D E) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags), + C AS 'C' = ANY(flags), + D AS 'D' = ANY(flags), + E AS 'E' = ANY(flags) +); +-- Row 1: A=T, B=T → matches A +-- Row 2: B=T, C=T → matches B +-- Row 3: C=T, D=T → matches C +-- Row 4: D=T, E=T → matches D +-- Row 5: E=T → matches E +-- Result: match 1-5 (A B C D E) + +-- Test 6: Diagonal pattern with multi-TRUE (shifted overlap) +WITH test_diagonal AS ( + SELECT * FROM (VALUES + (1, ARRAY['A','_']), + (2, ARRAY['B','A']), + (3, ARRAY['C','B']), + (4, ARRAY['D','C']), + (5, ARRAY['_','D']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_diagonal +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + PATTERN (A B C D) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags), + C AS 'C' = ANY(flags), + D AS 'D' = ANY(flags) +); +-- Possible matches: +-- Start Row 1: A(1) B(2) C(3) D(4) → 1-4 +-- Start Row 2: A(2) B(3) C(4) D(5) → 2-5 (because Row 2 has A too!) + +-- =================================================================== +-- Context Absorption Tests +-- =================================================================== + +-- Test absorption 1: Basic A+ pattern - later contexts absorbed by earlier +WITH test_absorb_basic AS ( + SELECT * FROM (VALUES + (1, ARRAY['A']), + (2, ARRAY['A']), + (3, ARRAY['A']), + (4, ARRAY['A']), + (5, ARRAY['B']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_absorb_basic +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + PATTERN (A+) + DEFINE A AS 'A' = ANY(flags) +); +-- Pattern A+ is absorbable (unbounded first element, only one unbounded) +-- Without absorption: 4 matches (1-4, 2-4, 3-4, 4-4) +-- With absorption: Row 1 match (1-4), rows 2-4 absorbed + +-- Test absorption 2: A+ B pattern - absorption with fixed suffix +WITH test_absorb_suffix AS ( + SELECT * FROM (VALUES + (1, ARRAY['A']), + (2, ARRAY['A']), + (3, ARRAY['A']), + (4, ARRAY['B']), + (5, ARRAY['X']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_absorb_suffix +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + PATTERN (A+ B) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags) +); +-- Pattern A+ B is absorbable (A+ unbounded first, B bounded suffix) +-- All potential matches end at same row (row 4 with B) +-- With absorption: Row 1 match (1-4), rows 2-3 absorbed + +-- Test absorption 3: Per-branch absorption with ALT (B+ C | B+ D) +WITH test_absorb_alt AS ( + SELECT * FROM (VALUES + (1, ARRAY['B']), + (2, ARRAY['B']), + (3, ARRAY['B']), + (4, ARRAY['D']), + (5, ARRAY['X']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_absorb_alt +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + PATTERN (B+ C | B+ D) + DEFINE + B AS 'B' = ANY(flags), + C AS 'C' = ANY(flags), + D AS 'D' = ANY(flags) +); +-- Both branches B+ C and B+ D are absorbable (B+ unbounded first) +-- B+ D branch matches: potential 1-4, 2-4, 3-4 +-- Row 1 (1-4) absorbs Row 2 (2-4) and Row 3 (3-4) - same endpoint + +-- Test absorption 4: Non-absorbable pattern (A B+ - unbounded not first) +WITH test_no_absorb AS ( + SELECT * FROM (VALUES + (1, ARRAY['A']), + (2, ARRAY['B']), + (3, ARRAY['B']), + (4, ARRAY['B']), + (5, ARRAY['X']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_no_absorb +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + PATTERN (A B+) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags) +); +-- Pattern A B+ is NOT absorbable (A bounded first, B+ unbounded but not first) +-- Only Row 1 can start match (only row with A), so only one match: 1-4 + +-- Test absorption 5: GROUP merge enables absorption +WITH test_absorb_group AS ( + SELECT * FROM (VALUES + (1, ARRAY['A']), + (2, ARRAY['B']), + (3, ARRAY['A']), + (4, ARRAY['B']), + (5, ARRAY['A']), + (6, ARRAY['B']), + (7, ARRAY['X']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_absorb_group +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + PATTERN ((A B) (A B)+) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags) +); +-- Pattern optimized: (A B) (A B)+ -> (A B){2,} +-- Potential matches: 1-6 (3 reps), 3-6 (2 reps), 5-6 needs 2 reps (fail) +-- Row 1 (1-6) absorbs Row 3 (3-6) - same endpoint, top-level unbounded GROUP + +-- Test absorption 6: Multiple unbounded - NOT absorbable +WITH test_multi_unbounded AS ( + SELECT * FROM (VALUES + (1, ARRAY['A']), + (2, ARRAY['A']), + (3, ARRAY['B']), + (4, ARRAY['B']), + (5, ARRAY['X']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_multi_unbounded +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + PATTERN (A+ B+) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags) +); +-- Pattern A+ B+ has TWO unbounded elements - NOT absorbable +-- Greedy A+ consumes rows 1-2, then B+ matches 3-4 +-- Row 1: 1-4, Row 2: 2-4 (different A+ counts, not absorbed) diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index d409561f602..1d24db449ad 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2440,6 +2440,13 @@ RI_ConstraintInfo RI_QueryHashEntry RI_QueryKey RPCommonSyntax +RPRDepth +RPRElemFlags +RPRElemIdx +RPRPattern +RPRPatternElement +RPRQuantity +RPRVarId RPSkipTo RTEKind RTEPermissionInfo -- 2.50.1 (Apple Git-155) Attachments: [text/plain] 0001-NFA-based-pattern-matching.txt (312.7K, ../../CAAAe_zAgyfStcRAdm9j6QrA9p0pOk3scxpmkAkxi3goSdgxo7Q@mail.gmail.com/3-0001-NFA-based-pattern-matching.txt) download ^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: Row pattern recognition @ 2026-01-15 04:56 Tatsuo Ishii <[email protected]> parent: Henson Choi <[email protected]> 0 siblings, 0 replies; 5+ messages in thread From: Tatsuo Ishii @ 2026-01-15 04:56 UTC (permalink / raw) To: [email protected]; +Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; pgsql-hackers Hi Henson, > Hi Ishii-san, > > > The attached patch implements streaming NFA-based pattern matching. > This is based on your v37 patch and the pattern grammar work I submitted > earlier. Great! > This is a testable draft for review, not production-ready yet. It allows > in-depth verification of pattern optimization, pattern matching/merging, > context absorption, and window behavior. I squashed your patches into my v37 patches and created v38 patches. (See attached). Will it be Okay for you to work on the patches from now on? Also I have added you to the commit message as an author. > Summary of changes: Will check. Best regards, -- Tatsuo Ishii SRA OSS K.K. English: http://www.sraoss.co.jp/index_en/ Japanese:http://www.sraoss.co.jp Attachments: [application/octet-stream] v38-0001-Row-pattern-recognition-patch-for-raw-parser.patch (26.4K, ../../[email protected]/2-v38-0001-Row-pattern-recognition-patch-for-raw-parser.patch) download | inline diff: From 9a7be3281a28aefb68e03c6203608a98184eb941 Mon Sep 17 00:00:00 2001 From: Tatsuo Ishii <[email protected]> Date: Thu, 15 Jan 2026 13:26:44 +0900 Subject: [PATCH v38 1/8] Row pattern recognition patch for raw parser. The series of patches are to implement the row pattern recognition (SQL/RPR) feature. Currently the implementation is a subset of SQL/RPR (ISO/IEC 19075-2:2016). Namely, implementation of some features of R020 (WINDOW clause). R010 (MATCH_RECOGNIZE) is out of the scope of the patches. Currently following features are implemented in the patches. - PATTERN - PATTERN regular expressions (+, *, ?) - DEFINE - INITIAL - AFTER MATCH SKIP TO PAST LAST ROW - AFTER MATCH SKIP TO NEXT ROW Currently following features are not implemented in the patches. - MEASURE - SUBSET - SEEK - AFTER MATCH SKIP TO - AFTER MATCH SKIP TO FIRST - AFTER MATCH SKIP TO LAST - PATTERN regular expressions (|, () (grouping), {n}, {n,}, {n,m}, {,m}, reluctant quantifiers (*? etc.), {- and -}, ^, $, () (empty pattern) - PERMUTE - FIRST, LAST, CLASSIFIER Author: Tatsuo Ishii <[email protected]> Author: Henson Choi <[email protected]> Reviewed-by: Vik Fearing <[email protected]> Reviewed-by: Jacob Champion <[email protected]> Reviewed-by: NINGWEI CHEN <[email protected]> Reviewed-by: "David G. Johnston" <[email protected]> Reviewed-by: Chao Li <[email protected]> Reviewed-by: Henson Choi <[email protected]> Reviewed-by: Tatsuo Ishii <[email protected]> Discussion: https://postgr.es/m/20230625.210509.1276733411677577841.t-ishii%40sranhm.sra.co.jp --- src/backend/parser/gram.y | 384 ++++++++++++++++++++++++++++++-- src/backend/parser/scan.l | 4 +- src/include/nodes/parsenodes.h | 66 ++++++ src/include/parser/kwlist.h | 5 + src/include/parser/parse_node.h | 1 + 5 files changed, 441 insertions(+), 19 deletions(-) diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 713ee5c10a2..ffb950ac61d 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -209,6 +209,7 @@ static void preprocess_pub_all_objtype_list(List *all_objects_list, static void preprocess_pubobj_list(List *pubobjspec_list, core_yyscan_t yyscanner); static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query); +static RPRPatternNode *makeRPRQuantifier(int min, int max, bool reluctant); %} @@ -682,6 +683,15 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query); json_object_constructor_null_clause_opt json_array_constructor_null_clause_opt +%type <target> row_pattern_definition +%type <node> opt_row_pattern_common_syntax + row_pattern row_pattern_alt row_pattern_seq + row_pattern_term row_pattern_primary + row_pattern_quantifier_opt +%type <list> row_pattern_definition_list +%type <ival> opt_row_pattern_skip_to +%type <boolean> opt_row_pattern_initial_or_seek + /* * Non-keyword token types. These are hard-wired into the "flex" lexer. * They must be listed first so that their numeric codes do not depend on @@ -724,7 +734,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query); CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE DATA_P DATABASE DAY_P DEALLOCATE DEC DECIMAL_P DECLARE DEFAULT DEFAULTS - DEFERRABLE DEFERRED DEFINER DELETE_P DELIMITER DELIMITERS DEPENDS DEPTH DESC + DEFERRABLE DEFERRED DEFINE DEFINER DELETE_P DELIMITER DELIMITERS DEPENDS DEPTH DESC DETACH DICTIONARY DISABLE_P DISCARD DISTINCT DO DOCUMENT_P DOMAIN_P DOUBLE_P DROP @@ -740,7 +750,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query); HANDLER HAVING HEADER_P HOLD HOUR_P IDENTITY_P IF_P IGNORE_P ILIKE IMMEDIATE IMMUTABLE IMPLICIT_P IMPORT_P IN_P INCLUDE - INCLUDING INCREMENT INDENT INDEX INDEXES INHERIT INHERITS INITIALLY INLINE_P + INCLUDING INCREMENT INDENT INDEX INDEXES INHERIT INHERITS INITIAL INITIALLY INLINE_P INNER_P INOUT INPUT_P INSENSITIVE INSERT INSTEAD INT_P INTEGER INTERSECT INTERVAL INTO INVOKER IS ISNULL ISOLATION @@ -765,8 +775,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query); ORDER ORDINALITY OTHERS OUT_P OUTER_P OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER - PARALLEL PARAMETER PARSER PARTIAL PARTITION PARTITIONS PASSING PASSWORD PATH - PERIOD PLACING PLAN PLANS POLICY + PARALLEL PARAMETER PARSER PARTIAL PARTITION PARTITIONS PASSING PASSWORD PAST PATH + PATTERN_P PERIOD PLACING PLAN PLANS POLICY POSITION PRECEDING PRECISION PRESERVE PREPARE PREPARED PRIMARY PRIOR PRIVILEGES PROCEDURAL PROCEDURE PROCEDURES PROGRAM PUBLICATION @@ -777,7 +787,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query); RESET RESPECT_P RESTART RESTRICT RETURN RETURNING RETURNS REVOKE RIGHT ROLE ROLLBACK ROLLUP ROUTINE ROUTINES ROW ROWS RULE - SAVEPOINT SCALAR SCHEMA SCHEMAS SCROLL SEARCH SECOND_P SECURITY SELECT + SAVEPOINT SCALAR SCHEMA SCHEMAS SCROLL SEARCH SECOND_P SECURITY SEEK SELECT SEQUENCE SEQUENCES SERIALIZABLE SERVER SESSION SESSION_USER SET SETS SETOF SHARE SHOW SIMILAR SIMPLE SKIP SMALLINT SNAPSHOT SOME SPLIT SOURCE SQL_P STABLE STANDALONE_P @@ -860,8 +870,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query); * reference point for a precedence level that we can assign to other * keywords that lack a natural precedence level. * - * We need to do this for PARTITION, RANGE, ROWS, and GROUPS to support - * opt_existing_window_name (see comment there). + * We need to do this for PARTITION, RANGE, ROWS, GROUPS, AFTER, INITIAL, + * SEEK, PATTERN_P to support opt_existing_window_name (see comment there). * * The frame_bound productions UNBOUNDED PRECEDING and UNBOUNDED FOLLOWING * are even messier: since UNBOUNDED is an unreserved keyword (per spec!), @@ -891,7 +901,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query); %nonassoc UNBOUNDED NESTED /* ideally would have same precedence as IDENT */ %nonassoc IDENT PARTITION RANGE ROWS GROUPS PRECEDING FOLLOWING CUBE ROLLUP SET KEYS OBJECT_P SCALAR VALUE_P WITH WITHOUT PATH -%left Op OPERATOR /* multi-character ops and user-defined operators */ + AFTER INITIAL SEEK PATTERN_P +%left Op OPERATOR '|' /* multi-character ops and user-defined operators */ %left '+' '-' %left '*' '/' '%' %left '^' @@ -16625,7 +16636,8 @@ over_clause: OVER window_specification ; window_specification: '(' opt_existing_window_name opt_partition_clause - opt_sort_clause opt_frame_clause ')' + opt_sort_clause opt_frame_clause + opt_row_pattern_common_syntax ')' { WindowDef *n = makeNode(WindowDef); @@ -16637,20 +16649,21 @@ window_specification: '(' opt_existing_window_name opt_partition_clause n->frameOptions = $5->frameOptions; n->startOffset = $5->startOffset; n->endOffset = $5->endOffset; + n->rpCommonSyntax = (RPCommonSyntax *)$6; n->location = @1; $$ = n; } ; /* - * If we see PARTITION, RANGE, ROWS or GROUPS as the first token after the '(' - * of a window_specification, we want the assumption to be that there is - * no existing_window_name; but those keywords are unreserved and so could - * be ColIds. We fix this by making them have the same precedence as IDENT - * and giving the empty production here a slightly higher precedence, so - * that the shift/reduce conflict is resolved in favor of reducing the rule. - * These keywords are thus precluded from being an existing_window_name but - * are not reserved for any other purpose. + * If we see PARTITION, RANGE, ROWS, GROUPS, AFTER, INITIAL, SEEK or PATTERN_P + * as the first token after the '(' of a window_specification, we want the + * assumption to be that there is no existing_window_name; but those keywords + * are unreserved and so could be ColIds. We fix this by making them have the + * same precedence as IDENT and giving the empty production here a slightly + * higher precedence, so that the shift/reduce conflict is resolved in favor + * of reducing the rule. These keywords are thus precluded from being an + * existing_window_name but are not reserved for any other purpose. */ opt_existing_window_name: ColId { $$ = $1; } | /*EMPTY*/ %prec Op { $$ = NULL; } @@ -16819,6 +16832,315 @@ opt_window_exclusion_clause: | /*EMPTY*/ { $$ = 0; } ; +opt_row_pattern_common_syntax: +opt_row_pattern_skip_to opt_row_pattern_initial_or_seek + PATTERN_P '(' row_pattern ')' + DEFINE row_pattern_definition_list + { + RPCommonSyntax *n = makeNode(RPCommonSyntax); + n->rpSkipTo = $1; + n->initial = $2; + n->rpPattern = (RPRPatternNode *) $5; + n->rpDefs = $8; + $$ = (Node *) n; + } + | /*EMPTY*/ { $$ = NULL; } + ; + +opt_row_pattern_skip_to: + AFTER MATCH SKIP TO NEXT ROW + { + $$ = ST_NEXT_ROW; + } + | AFTER MATCH SKIP PAST LAST_P ROW + { + $$ = ST_PAST_LAST_ROW; + } + | /*EMPTY*/ + { + $$ = ST_PAST_LAST_ROW; + } + ; + +opt_row_pattern_initial_or_seek: + INITIAL { $$ = true; } + | SEEK + { + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("SEEK is not supported"), + errhint("Use INITIAL instead."), + parser_errposition(@1))); + } + | /*EMPTY*/ { $$ = true; } + ; + +row_pattern: + row_pattern_alt { $$ = $1; } + ; + +row_pattern_alt: + row_pattern_seq { $$ = $1; } + | row_pattern_alt '|' row_pattern_seq + { + RPRPatternNode *n; + /* If left side is already ALT, append to it */ + if (IsA($1, RPRPatternNode) && + ((RPRPatternNode *) $1)->nodeType == RPR_PATTERN_ALT) + { + n = (RPRPatternNode *) $1; + n->children = lappend(n->children, $3); + $$ = (Node *) n; + } + else + { + n = makeNode(RPRPatternNode); + n->nodeType = RPR_PATTERN_ALT; + n->children = list_make2($1, $3); + n->min = 1; + n->max = 1; + n->location = @1; + $$ = (Node *) n; + } + } + ; + +row_pattern_seq: + row_pattern_term { $$ = $1; } + | row_pattern_seq row_pattern_term + { + RPRPatternNode *n; + /* If left side is already SEQ, append to it */ + if (IsA($1, RPRPatternNode) && + ((RPRPatternNode *) $1)->nodeType == RPR_PATTERN_SEQ) + { + n = (RPRPatternNode *) $1; + n->children = lappend(n->children, $2); + $$ = (Node *) n; + } + else + { + n = makeNode(RPRPatternNode); + n->nodeType = RPR_PATTERN_SEQ; + n->children = list_make2($1, $2); + n->min = 1; + n->max = 1; + n->location = @1; + $$ = (Node *) n; + } + } + ; + +row_pattern_term: + row_pattern_primary row_pattern_quantifier_opt + { + RPRPatternNode *n = (RPRPatternNode *) $1; + RPRPatternNode *q = (RPRPatternNode *) $2; + + n->min = q->min; + n->max = q->max; + n->reluctant = q->reluctant; + $$ = (Node *) n; + } + ; + +row_pattern_primary: + ColId + { + RPRPatternNode *n = makeNode(RPRPatternNode); + n->nodeType = RPR_PATTERN_VAR; + n->varName = $1; + n->min = 1; + n->max = 1; + n->reluctant = false; + n->children = NIL; + n->location = @1; + $$ = (Node *) n; + } + | '(' row_pattern ')' + { + RPRPatternNode *inner = (RPRPatternNode *) $2; + RPRPatternNode *n = makeNode(RPRPatternNode); + n->nodeType = RPR_PATTERN_GROUP; + n->children = list_make1(inner); + n->min = 1; + n->max = 1; + n->reluctant = false; + n->location = @1; + $$ = (Node *) n; + } + ; + +row_pattern_quantifier_opt: + /* EMPTY */ { $$ = (Node *) makeRPRQuantifier(1, 1, false); } + | '*' { $$ = (Node *) makeRPRQuantifier(0, INT_MAX, false); } + | '+' { $$ = (Node *) makeRPRQuantifier(1, INT_MAX, false); } + | Op + { + /* Handle single Op: ? or reluctant quantifiers *?, +?, ?? */ + if (strcmp($1, "?") == 0) + $$ = (Node *) makeRPRQuantifier(0, 1, false); + else if (strcmp($1, "*?") == 0) + $$ = (Node *) makeRPRQuantifier(0, INT_MAX, true); + else if (strcmp($1, "+?") == 0) + $$ = (Node *) makeRPRQuantifier(1, INT_MAX, true); + else if (strcmp($1, "??") == 0) + $$ = (Node *) makeRPRQuantifier(0, 1, true); + else + ereport(ERROR, + errcode(ERRCODE_SYNTAX_ERROR), + errmsg("unsupported quantifier \"%s\"", $1), + parser_errposition(@1)); + } + /* RELUCTANT quantifiers (when lexer separates tokens) */ + | '*' Op + { + if (strcmp($2, "?") != 0) + ereport(ERROR, + errcode(ERRCODE_SYNTAX_ERROR), + errmsg("unsupported quantifier"), + parser_errposition(@2)); + $$ = (Node *) makeRPRQuantifier(0, INT_MAX, true); + } + | '+' Op + { + if (strcmp($2, "?") != 0) + ereport(ERROR, + errcode(ERRCODE_SYNTAX_ERROR), + errmsg("unsupported quantifier"), + parser_errposition(@2)); + $$ = (Node *) makeRPRQuantifier(1, INT_MAX, true); + } + | Op Op + { + if (strcmp($1, "?") != 0 || strcmp($2, "?") != 0) + ereport(ERROR, + errcode(ERRCODE_SYNTAX_ERROR), + errmsg("unsupported quantifier"), + parser_errposition(@1)); + $$ = (Node *) makeRPRQuantifier(0, 1, true); + } + /* {n}, {n,}, {,m}, {n,m} quantifiers */ + | '{' Iconst '}' + { + if ($2 < 0 || $2 >= INT_MAX) + ereport(ERROR, + errcode(ERRCODE_SYNTAX_ERROR), + errmsg("quantifier bound must be between 0 and %d", INT_MAX - 1), + parser_errposition(@2)); + $$ = (Node *) makeRPRQuantifier($2, $2, false); + } + | '{' Iconst ',' '}' + { + if ($2 < 0 || $2 >= INT_MAX) + ereport(ERROR, + errcode(ERRCODE_SYNTAX_ERROR), + errmsg("quantifier bound must be between 0 and %d", INT_MAX - 1), + parser_errposition(@2)); + $$ = (Node *) makeRPRQuantifier($2, INT_MAX, false); + } + | '{' ',' Iconst '}' + { + if ($3 < 0 || $3 >= INT_MAX) + ereport(ERROR, + errcode(ERRCODE_SYNTAX_ERROR), + errmsg("quantifier bound must be between 0 and %d", INT_MAX - 1), + parser_errposition(@3)); + $$ = (Node *) makeRPRQuantifier(0, $3, false); + } + | '{' Iconst ',' Iconst '}' + { + if ($2 < 0 || $4 < 0 || $2 >= INT_MAX || $4 >= INT_MAX) + ereport(ERROR, + errcode(ERRCODE_SYNTAX_ERROR), + errmsg("quantifier bound must be between 0 and %d", INT_MAX - 1), + parser_errposition(@2)); + if ($2 > $4) + ereport(ERROR, + errcode(ERRCODE_SYNTAX_ERROR), + errmsg("quantifier minimum bound must not exceed maximum"), + parser_errposition(@2)); + $$ = (Node *) makeRPRQuantifier($2, $4, false); + } + /* Reluctant versions: {n}?, {n,}?, {,m}?, {n,m}? */ + | '{' Iconst '}' Op + { + if (strcmp($4, "?") != 0) + ereport(ERROR, + errcode(ERRCODE_SYNTAX_ERROR), + errmsg("unsupported quantifier"), + parser_errposition(@4)); + if ($2 < 0 || $2 >= INT_MAX) + ereport(ERROR, + errcode(ERRCODE_SYNTAX_ERROR), + errmsg("quantifier bound must be between 0 and %d", INT_MAX - 1), + parser_errposition(@2)); + $$ = (Node *) makeRPRQuantifier($2, $2, true); + } + | '{' Iconst ',' '}' Op + { + if (strcmp($5, "?") != 0) + ereport(ERROR, + errcode(ERRCODE_SYNTAX_ERROR), + errmsg("unsupported quantifier"), + parser_errposition(@5)); + if ($2 < 0 || $2 >= INT_MAX) + ereport(ERROR, + errcode(ERRCODE_SYNTAX_ERROR), + errmsg("quantifier bound must be between 0 and %d", INT_MAX - 1), + parser_errposition(@2)); + $$ = (Node *) makeRPRQuantifier($2, INT_MAX, true); + } + | '{' ',' Iconst '}' Op + { + if (strcmp($5, "?") != 0) + ereport(ERROR, + errcode(ERRCODE_SYNTAX_ERROR), + errmsg("unsupported quantifier"), + parser_errposition(@5)); + if ($3 < 0 || $3 >= INT_MAX) + ereport(ERROR, + errcode(ERRCODE_SYNTAX_ERROR), + errmsg("quantifier bound must be between 0 and %d", INT_MAX - 1), + parser_errposition(@3)); + $$ = (Node *) makeRPRQuantifier(0, $3, true); + } + | '{' Iconst ',' Iconst '}' Op + { + if (strcmp($6, "?") != 0) + ereport(ERROR, + errcode(ERRCODE_SYNTAX_ERROR), + errmsg("unsupported quantifier"), + parser_errposition(@6)); + if ($2 < 0 || $4 < 0 || $2 >= INT_MAX || $4 >= INT_MAX) + ereport(ERROR, + errcode(ERRCODE_SYNTAX_ERROR), + errmsg("quantifier bound must be between 0 and %d", INT_MAX - 1), + parser_errposition(@2)); + if ($2 > $4) + ereport(ERROR, + errcode(ERRCODE_SYNTAX_ERROR), + errmsg("quantifier minimum bound must not exceed maximum"), + parser_errposition(@2)); + $$ = (Node *) makeRPRQuantifier($2, $4, true); + } + ; + +row_pattern_definition_list: + row_pattern_definition { $$ = list_make1($1); } + | row_pattern_definition_list ',' row_pattern_definition { $$ = lappend($1, $3); } + ; + +row_pattern_definition: + ColId AS a_expr + { + $$ = makeNode(ResTarget); + $$->name = $1; + $$->indirection = NIL; + $$->val = (Node *) $3; + $$->location = @1; + } + ; /* * Supporting nonterminals for expressions. @@ -16863,12 +17185,15 @@ MathOp: '+' { $$ = "+"; } | LESS_EQUALS { $$ = "<="; } | GREATER_EQUALS { $$ = ">="; } | NOT_EQUALS { $$ = "<>"; } + | '|' { $$ = "|"; } ; qual_Op: Op { $$ = list_make1(makeString($1)); } | OPERATOR '(' any_operator ')' { $$ = $3; } + | '|' + { $$ = list_make1(makeString("|")); } ; qual_all_Op: @@ -17958,6 +18283,7 @@ unreserved_keyword: | DECLARE | DEFAULTS | DEFERRED + | DEFINE | DEFINER | DELETE_P | DELIMITER @@ -18023,6 +18349,7 @@ unreserved_keyword: | INDEXES | INHERIT | INHERITS + | INITIAL | INLINE_P | INPUT_P | INSENSITIVE @@ -18098,7 +18425,9 @@ unreserved_keyword: | PARTITIONS | PASSING | PASSWORD + | PAST | PATH + | PATTERN_P | PERIOD | PLAN | PLANS @@ -18152,6 +18481,7 @@ unreserved_keyword: | SEARCH | SECOND_P | SECURITY + | SEEK | SEQUENCE | SEQUENCES | SERIALIZABLE @@ -18539,6 +18869,7 @@ bare_label_keyword: | DEFAULTS | DEFERRABLE | DEFERRED + | DEFINE | DEFINER | DELETE_P | DELIMITER @@ -18617,6 +18948,7 @@ bare_label_keyword: | INDEXES | INHERIT | INHERITS + | INITIAL | INITIALLY | INLINE_P | INNER_P @@ -18730,7 +19062,9 @@ bare_label_keyword: | PARTITIONS | PASSING | PASSWORD + | PAST | PATH + | PATTERN_P | PERIOD | PLACING | PLAN @@ -18789,6 +19123,7 @@ bare_label_keyword: | SCROLL | SEARCH | SECURITY + | SEEK | SELECT | SEQUENCE | SEQUENCES @@ -19980,6 +20315,21 @@ makeRecursiveViewSelect(char *relname, List *aliases, Node *query) return (Node *) s; } +/* + * makeRPRQuantifier + * Create an RPRPatternNode with specified quantifier bounds. + */ +static RPRPatternNode * +makeRPRQuantifier(int min, int max, bool reluctant) +{ + RPRPatternNode *n = makeNode(RPRPatternNode); + + n->min = min; + n->max = max; + n->reluctant = reluctant; + return n; +} + /* parser_init() * Initialize to parse one query string */ diff --git a/src/backend/parser/scan.l b/src/backend/parser/scan.l index 731d584106d..f6ca2406146 100644 --- a/src/backend/parser/scan.l +++ b/src/backend/parser/scan.l @@ -363,7 +363,7 @@ not_equals "!=" * If you change either set, adjust the character lists appearing in the * rule for "operator"! */ -self [,()\[\].;\:\+\-\*\/\%\^\<\>\=] +self [,()\[\].;\:\+\-\*\/\%\^\<\>\=\|] op_chars [\~\!\@\#\^\&\|\`\?\+\-\*\/\%\<\>\=] operator {op_chars}+ @@ -955,7 +955,7 @@ other . * that the "self" rule would have. */ if (nchars == 1 && - strchr(",()[].;:+-*/%^<>=", yytext[0])) + strchr(",()[].;:+-*/%^<>=|", yytext[0])) return yytext[0]; /* * Likewise, if what we have left is two chars, and diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 646d6ced763..2bb5c321157 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -578,6 +578,56 @@ typedef struct SortBy ParseLoc location; /* operator location, or -1 if none/unknown */ } SortBy; +/* + * AFTER MATCH row pattern skip to types in row pattern common syntax + */ +typedef enum RPSkipTo +{ + ST_NONE, /* AFTER MATCH omitted */ + ST_NEXT_ROW, /* SKIP TO NEXT ROW */ + ST_PAST_LAST_ROW, /* SKIP TO PAST LAST ROW */ +} RPSkipTo; + +/* + * RPRPatternNodeType - Row Pattern Recognition pattern node types + */ +typedef enum RPRPatternNodeType +{ + RPR_PATTERN_VAR, /* variable reference */ + RPR_PATTERN_SEQ, /* sequence (concatenation) */ + RPR_PATTERN_ALT, /* alternation (|) */ + RPR_PATTERN_GROUP /* group (parentheses) */ +} RPRPatternNodeType; + +/* + * RPRPatternNode - Row Pattern Recognition pattern AST node + */ +typedef struct RPRPatternNode +{ + NodeTag type; /* T_RPRPatternNode */ + RPRPatternNodeType nodeType; /* VAR, SEQ, ALT, GROUP */ + int min; /* minimum repetitions (0 for *, ?) */ + int max; /* maximum repetitions (INT_MAX for *, +) */ + bool reluctant; /* true for *?, +?, ?? */ + ParseLoc location; /* token location, or -1 */ + char *varName; /* VAR: variable name */ + List *children; /* SEQ, ALT, GROUP: child nodes */ +} RPRPatternNode; + +/* + * RowPatternCommonSyntax - raw representation of row pattern common syntax + */ +typedef struct RPCommonSyntax +{ + NodeTag type; + RPSkipTo rpSkipTo; /* Row Pattern AFTER MATCH SKIP type */ + bool initial; /* true if <row pattern initial or seek> is + * initial */ + RPRPatternNode *rpPattern; /* PATTERN clause AST */ + List *rpDefs; /* row pattern definitions clause (list of + * ResTarget) */ +} RPCommonSyntax; + /* * WindowDef - raw representation of WINDOW and OVER clauses * @@ -593,6 +643,7 @@ typedef struct WindowDef char *refname; /* referenced window name, if any */ List *partitionClause; /* PARTITION BY expression list */ List *orderClause; /* ORDER BY (list of SortBy) */ + RPCommonSyntax *rpCommonSyntax; /* row pattern common syntax */ int frameOptions; /* frame_clause options, see below */ Node *startOffset; /* expression for starting bound, if any */ Node *endOffset; /* expression for ending bound, if any */ @@ -1588,6 +1639,11 @@ typedef struct GroupingSet * the orderClause might or might not be copied (see copiedOrder); the framing * options are never copied, per spec. * + * "defineClause" is Row Pattern Recognition DEFINE clause (list of + * TargetEntry). TargetEntry.resname represents row pattern definition + * variable name. "rpPattern" represents PATTERN clause as an AST tree + * (RPRPatternNode). + * * The information relevant for the query jumbling is the partition clause * type and its bounds. */ @@ -1617,6 +1673,16 @@ typedef struct WindowClause Index winref; /* ID referenced by window functions */ /* did we copy orderClause from refname? */ bool copiedOrder pg_node_attr(query_jumble_ignore); + /* Row Pattern AFTER MATCH SKIP clause */ + RPSkipTo rpSkipTo; /* Row Pattern Skip To type */ + bool initial; /* true if <row pattern initial or seek> is + * initial */ + /* Row Pattern DEFINE clause (list of TargetEntry) */ + List *defineClause; + /* Row Pattern DEFINE variable initial names (list of String) */ + List *defineInitial; + /* Row Pattern PATTERN clause AST */ + RPRPatternNode *rpPattern; } WindowClause; /* diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h index f7753c5c8a8..1624f4412dc 100644 --- a/src/include/parser/kwlist.h +++ b/src/include/parser/kwlist.h @@ -129,6 +129,7 @@ PG_KEYWORD("default", DEFAULT, RESERVED_KEYWORD, BARE_LABEL) PG_KEYWORD("defaults", DEFAULTS, UNRESERVED_KEYWORD, BARE_LABEL) PG_KEYWORD("deferrable", DEFERRABLE, RESERVED_KEYWORD, BARE_LABEL) PG_KEYWORD("deferred", DEFERRED, UNRESERVED_KEYWORD, BARE_LABEL) +PG_KEYWORD("define", DEFINE, UNRESERVED_KEYWORD, BARE_LABEL) PG_KEYWORD("definer", DEFINER, UNRESERVED_KEYWORD, BARE_LABEL) PG_KEYWORD("delete", DELETE_P, UNRESERVED_KEYWORD, BARE_LABEL) PG_KEYWORD("delimiter", DELIMITER, UNRESERVED_KEYWORD, BARE_LABEL) @@ -217,6 +218,7 @@ PG_KEYWORD("index", INDEX, UNRESERVED_KEYWORD, BARE_LABEL) PG_KEYWORD("indexes", INDEXES, UNRESERVED_KEYWORD, BARE_LABEL) PG_KEYWORD("inherit", INHERIT, UNRESERVED_KEYWORD, BARE_LABEL) PG_KEYWORD("inherits", INHERITS, UNRESERVED_KEYWORD, BARE_LABEL) +PG_KEYWORD("initial", INITIAL, UNRESERVED_KEYWORD, BARE_LABEL) PG_KEYWORD("initially", INITIALLY, RESERVED_KEYWORD, BARE_LABEL) PG_KEYWORD("inline", INLINE_P, UNRESERVED_KEYWORD, BARE_LABEL) PG_KEYWORD("inner", INNER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL) @@ -342,7 +344,9 @@ PG_KEYWORD("partition", PARTITION, UNRESERVED_KEYWORD, BARE_LABEL) PG_KEYWORD("partitions", PARTITIONS, UNRESERVED_KEYWORD, BARE_LABEL) PG_KEYWORD("passing", PASSING, UNRESERVED_KEYWORD, BARE_LABEL) PG_KEYWORD("password", PASSWORD, UNRESERVED_KEYWORD, BARE_LABEL) +PG_KEYWORD("past", PAST, UNRESERVED_KEYWORD, BARE_LABEL) PG_KEYWORD("path", PATH, UNRESERVED_KEYWORD, BARE_LABEL) +PG_KEYWORD("pattern", PATTERN_P, UNRESERVED_KEYWORD, BARE_LABEL) PG_KEYWORD("period", PERIOD, UNRESERVED_KEYWORD, BARE_LABEL) PG_KEYWORD("placing", PLACING, RESERVED_KEYWORD, BARE_LABEL) PG_KEYWORD("plan", PLAN, UNRESERVED_KEYWORD, BARE_LABEL) @@ -405,6 +409,7 @@ PG_KEYWORD("scroll", SCROLL, UNRESERVED_KEYWORD, BARE_LABEL) PG_KEYWORD("search", SEARCH, UNRESERVED_KEYWORD, BARE_LABEL) PG_KEYWORD("second", SECOND_P, UNRESERVED_KEYWORD, AS_LABEL) PG_KEYWORD("security", SECURITY, UNRESERVED_KEYWORD, BARE_LABEL) +PG_KEYWORD("seek", SEEK, UNRESERVED_KEYWORD, BARE_LABEL) PG_KEYWORD("select", SELECT, RESERVED_KEYWORD, BARE_LABEL) PG_KEYWORD("sequence", SEQUENCE, UNRESERVED_KEYWORD, BARE_LABEL) PG_KEYWORD("sequences", SEQUENCES, UNRESERVED_KEYWORD, BARE_LABEL) diff --git a/src/include/parser/parse_node.h b/src/include/parser/parse_node.h index a9bffb8a78f..36fed104b32 100644 --- a/src/include/parser/parse_node.h +++ b/src/include/parser/parse_node.h @@ -51,6 +51,7 @@ typedef enum ParseExprKind EXPR_KIND_WINDOW_FRAME_RANGE, /* window frame clause with RANGE */ EXPR_KIND_WINDOW_FRAME_ROWS, /* window frame clause with ROWS */ EXPR_KIND_WINDOW_FRAME_GROUPS, /* window frame clause with GROUPS */ + EXPR_KIND_RPR_DEFINE, /* DEFINE */ EXPR_KIND_SELECT_TARGET, /* SELECT target list item */ EXPR_KIND_INSERT_TARGET, /* INSERT target list item */ EXPR_KIND_UPDATE_SOURCE, /* UPDATE assignment source item */ -- 2.43.0 [application/octet-stream] v38-0002-Row-pattern-recognition-patch-parse-analysis.patch (24.0K, ../../[email protected]/3-v38-0002-Row-pattern-recognition-patch-parse-analysis.patch) download | inline diff: From 4b7f8dfbfdefc342718c741416c3804b181aeb2b Mon Sep 17 00:00:00 2001 From: Tatsuo Ishii <[email protected]> Date: Thu, 15 Jan 2026 13:26:44 +0900 Subject: [PATCH v38 2/8] Row pattern recognition patch (parse/analysis). --- src/backend/nodes/copyfuncs.c | 39 ++++ src/backend/nodes/equalfuncs.c | 33 +++ src/backend/nodes/outfuncs.c | 51 +++++ src/backend/nodes/readfuncs.c | 79 +++++++ src/backend/parser/Makefile | 1 + src/backend/parser/README | 1 + src/backend/parser/meson.build | 1 + src/backend/parser/parse_agg.c | 7 + src/backend/parser/parse_clause.c | 10 +- src/backend/parser/parse_expr.c | 6 + src/backend/parser/parse_func.c | 3 + src/backend/parser/parse_rpr.c | 340 ++++++++++++++++++++++++++++++ src/include/parser/parse_clause.h | 3 + src/include/parser/parse_rpr.h | 22 ++ 14 files changed, 592 insertions(+), 4 deletions(-) create mode 100644 src/backend/parser/parse_rpr.c create mode 100644 src/include/parser/parse_rpr.h diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c index ff22a04abe5..5d69805fc24 100644 --- a/src/backend/nodes/copyfuncs.c +++ b/src/backend/nodes/copyfuncs.c @@ -16,6 +16,7 @@ #include "postgres.h" #include "miscadmin.h" +#include "nodes/plannodes.h" #include "utils/datum.h" @@ -166,6 +167,44 @@ _copyBitmapset(const Bitmapset *from) return bms_copy(from); } +static RPRPattern * +_copyRPRPattern(const RPRPattern *from) +{ + RPRPattern *newnode = makeNode(RPRPattern); + + COPY_SCALAR_FIELD(numVars); + COPY_SCALAR_FIELD(maxDepth); + COPY_SCALAR_FIELD(numElements); + + /* Deep copy the varNames array */ + if (from->numVars > 0) + { + newnode->varNames = palloc0(from->numVars * sizeof(char *)); + for (int i = 0; i < from->numVars; i++) + newnode->varNames[i] = pstrdup(from->varNames[i]); + } + else + { + newnode->varNames = NULL; + } + + /* Deep copy the elements array */ + if (from->numElements > 0) + { + newnode->elements = palloc(from->numElements * sizeof(RPRPatternElement)); + memcpy(newnode->elements, from->elements, + from->numElements * sizeof(RPRPatternElement)); + } + else + { + newnode->elements = NULL; + } + + COPY_SCALAR_FIELD(isAbsorbable); + + return newnode; +} + /* * copyObjectImpl -- implementation of copyObject(); see nodes/nodes.h diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c index 3d1a1adf86e..9410790e3d3 100644 --- a/src/backend/nodes/equalfuncs.c +++ b/src/backend/nodes/equalfuncs.c @@ -20,6 +20,7 @@ #include "postgres.h" #include "miscadmin.h" +#include "nodes/plannodes.h" #include "utils/datum.h" @@ -149,6 +150,38 @@ _equalBitmapset(const Bitmapset *a, const Bitmapset *b) return bms_equal(a, b); } +static bool +_equalRPRPattern(const RPRPattern *a, const RPRPattern *b) +{ + COMPARE_SCALAR_FIELD(numVars); + COMPARE_SCALAR_FIELD(maxDepth); + COMPARE_SCALAR_FIELD(numElements); + + /* Compare varNames array */ + if (a->numVars > 0) + { + if (a->varNames == NULL || b->varNames == NULL) + return false; + for (int i = 0; i < a->numVars; i++) + { + if (strcmp(a->varNames[i], b->varNames[i]) != 0) + return false; + } + } + + /* Compare elements array */ + if (a->numElements > 0) + { + if (a->elements == NULL || b->elements == NULL) + return false; + if (memcmp(a->elements, b->elements, + a->numElements * sizeof(RPRPatternElement)) != 0) + return false; + } + + return true; +} + /* * Lists are handled specially */ diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c index c8eef2c75d2..1a70b016770 100644 --- a/src/backend/nodes/outfuncs.c +++ b/src/backend/nodes/outfuncs.c @@ -23,6 +23,7 @@ #include "nodes/bitmapset.h" #include "nodes/nodes.h" #include "nodes/pg_list.h" +#include "nodes/plannodes.h" #include "utils/datum.h" /* State flag that determines how nodeToStringInternal() should treat location fields */ @@ -718,6 +719,56 @@ _outA_Const(StringInfo str, const A_Const *node) WRITE_LOCATION_FIELD(location); } +static void +_outRPRPattern(StringInfo str, const RPRPattern *node) +{ + WRITE_NODE_TYPE("RPRPATTERN"); + + WRITE_INT_FIELD(numVars); + WRITE_INT_FIELD(maxDepth); + WRITE_INT_FIELD(numElements); + + /* Write varNames array as list of strings */ + appendStringInfoString(str, " :varNames"); + if (node->numVars > 0 && node->varNames != NULL) + { + appendStringInfoString(str, " ("); + for (int i = 0; i < node->numVars; i++) + { + if (i > 0) + appendStringInfoChar(str, ' '); + outToken(str, node->varNames[i]); + } + appendStringInfoChar(str, ')'); + } + else + appendStringInfoString(str, " <>"); + + /* Write elements array */ + appendStringInfoString(str, " :elements"); + if (node->numElements > 0 && node->elements != NULL) + { + appendStringInfoChar(str, ' '); + for (int i = 0; i < node->numElements; i++) + { + const RPRPatternElement *elem = &node->elements[i]; + + appendStringInfo(str, "(%d %u %d %d %d %d %d)", + (int) elem->varId, + (unsigned) elem->flags, + (int) elem->depth, + (int) elem->min, + (int) elem->max, + (int) elem->next, + (int) elem->jump); + } + } + else + appendStringInfoString(str, " <>"); + + WRITE_BOOL_FIELD(isAbsorbable); +} + /* * outNode - diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c index c11728c0f17..3ed31ba539d 100644 --- a/src/backend/nodes/readfuncs.c +++ b/src/backend/nodes/readfuncs.c @@ -30,6 +30,7 @@ #include "miscadmin.h" #include "nodes/bitmapset.h" +#include "nodes/plannodes.h" #include "nodes/readfuncs.h" @@ -560,6 +561,84 @@ _readExtensibleNode(void) READ_DONE(); } +static RPRPattern * +_readRPRPattern(void) +{ + READ_LOCALS(RPRPattern); + + READ_INT_FIELD(numVars); + READ_INT_FIELD(maxDepth); + READ_INT_FIELD(numElements); + + /* Read varNames array */ + token = pg_strtok(&length); /* skip :varNames */ + token = pg_strtok(&length); /* get '(' or '<>' */ + if (local_node->numVars > 0 && token[0] == '(') + { + local_node->varNames = palloc(local_node->numVars * sizeof(char *)); + for (int i = 0; i < local_node->numVars; i++) + { + token = pg_strtok(&length); + local_node->varNames[i] = debackslash(token, length); + } + token = pg_strtok(&length); /* skip ')' */ + } + else + { + local_node->varNames = NULL; + } + + /* Read elements array */ + token = pg_strtok(&length); /* skip :elements */ + token = pg_strtok(&length); /* get '(' or '<>' */ + if (local_node->numElements > 0 && token[0] == '(') + { + local_node->elements = palloc0(local_node->numElements * sizeof(RPRPatternElement)); + for (int i = 0; i < local_node->numElements; i++) + { + RPRPatternElement *elem = &local_node->elements[i]; + int varId, flags, depth, min, max, next, jump; + + /* Parse "(varId flags depth min max next jump)" */ + token = pg_strtok(&length); + varId = atoi(token); + token = pg_strtok(&length); + flags = atoi(token); + token = pg_strtok(&length); + depth = atoi(token); + token = pg_strtok(&length); + min = atoi(token); + token = pg_strtok(&length); + max = atoi(token); + token = pg_strtok(&length); + next = atoi(token); + token = pg_strtok(&length); + jump = atoi(token); + token = pg_strtok(&length); /* skip ')' */ + + elem->varId = (RPRVarId) varId; + elem->flags = (RPRElemFlags) flags; + elem->depth = (RPRDepth) depth; + elem->min = (RPRQuantity) min; + elem->max = (RPRQuantity) max; + elem->next = (RPRElemIdx) next; + elem->jump = (RPRElemIdx) jump; + + /* Read next element's '(' or end */ + if (i < local_node->numElements - 1) + token = pg_strtok(&length); /* get '(' */ + } + } + else + { + local_node->elements = NULL; + } + + READ_BOOL_FIELD(isAbsorbable); + + READ_DONE(); +} + /* * parseNodeString diff --git a/src/backend/parser/Makefile b/src/backend/parser/Makefile index 8c0fe28d63f..f4c7d605fe3 100644 --- a/src/backend/parser/Makefile +++ b/src/backend/parser/Makefile @@ -29,6 +29,7 @@ OBJS = \ parse_oper.o \ parse_param.o \ parse_relation.o \ + parse_rpr.o \ parse_target.o \ parse_type.o \ parse_utilcmd.o \ diff --git a/src/backend/parser/README b/src/backend/parser/README index e26eb437a9f..2baffa9517e 100644 --- a/src/backend/parser/README +++ b/src/backend/parser/README @@ -26,6 +26,7 @@ parse_node.c create nodes for various structures parse_oper.c handle operators in expressions parse_param.c handle Params (for the cases used in the core backend) parse_relation.c support routines for tables and column handling +parse_rpr.c handle Row Pattern Recognition parse_target.c handle the result list of the query parse_type.c support routines for data type handling parse_utilcmd.c parse analysis for utility commands (done at execution time) diff --git a/src/backend/parser/meson.build b/src/backend/parser/meson.build index 924ee87a453..a07102b37a0 100644 --- a/src/backend/parser/meson.build +++ b/src/backend/parser/meson.build @@ -16,6 +16,7 @@ backend_sources += files( 'parse_oper.c', 'parse_param.c', 'parse_relation.c', + 'parse_rpr.c', 'parse_target.c', 'parse_type.c', 'parse_utilcmd.c', diff --git a/src/backend/parser/parse_agg.c b/src/backend/parser/parse_agg.c index 25ee0f87d93..5ed785ea0d5 100644 --- a/src/backend/parser/parse_agg.c +++ b/src/backend/parser/parse_agg.c @@ -584,6 +584,10 @@ check_agglevels_and_constraints(ParseState *pstate, Node *expr) errkind = true; break; + case EXPR_KIND_RPR_DEFINE: + errkind = true; + break; + /* * There is intentionally no default: case here, so that the * compiler will warn if we add a new ParseExprKind without @@ -1023,6 +1027,9 @@ transformWindowFuncCall(ParseState *pstate, WindowFunc *wfunc, case EXPR_KIND_CYCLE_MARK: errkind = true; break; + case EXPR_KIND_RPR_DEFINE: + errkind = true; + break; /* * There is intentionally no default: case here, so that the diff --git a/src/backend/parser/parse_clause.c b/src/backend/parser/parse_clause.c index e35fd25c9bb..f98ac7cfcb5 100644 --- a/src/backend/parser/parse_clause.c +++ b/src/backend/parser/parse_clause.c @@ -37,6 +37,7 @@ #include "parser/parse_func.h" #include "parser/parse_oper.h" #include "parser/parse_relation.h" +#include "parser/parse_rpr.h" #include "parser/parse_target.h" #include "parser/parse_type.h" #include "parser/parser.h" @@ -84,8 +85,6 @@ static void checkExprIsVarFree(ParseState *pstate, Node *n, const char *constructName); static TargetEntry *findTargetlistEntrySQL92(ParseState *pstate, Node *node, List **tlist, ParseExprKind exprKind); -static TargetEntry *findTargetlistEntrySQL99(ParseState *pstate, Node *node, - List **tlist, ParseExprKind exprKind); static int get_matching_location(int sortgroupref, List *sortgrouprefs, List *exprs); static List *resolve_unique_index_expr(ParseState *pstate, InferClause *infer, @@ -97,7 +96,6 @@ static Node *transformFrameOffset(ParseState *pstate, int frameOptions, Oid rangeopfamily, Oid rangeopcintype, Oid *inRangeFunc, Node *clause); - /* * transformFromClause - * Process the FROM clause and add items to the query's range table, @@ -2168,7 +2166,7 @@ findTargetlistEntrySQL92(ParseState *pstate, Node *node, List **tlist, * tlist the target list (passed by reference so we can append to it) * exprKind identifies clause type being processed */ -static TargetEntry * +TargetEntry * findTargetlistEntrySQL99(ParseState *pstate, Node *node, List **tlist, ParseExprKind exprKind) { @@ -3019,6 +3017,10 @@ transformWindowDefinitions(ParseState *pstate, rangeopfamily, rangeopcintype, &wc->endInRangeFunc, windef->endOffset); + + /* Process Row Pattern Recognition related clauses */ + transformRPR(pstate, wc, windef, targetlist); + wc->winref = winref; result = lappend(result, wc); diff --git a/src/backend/parser/parse_expr.c b/src/backend/parser/parse_expr.c index 56826db4c26..7ebcd15fa39 100644 --- a/src/backend/parser/parse_expr.c +++ b/src/backend/parser/parse_expr.c @@ -577,6 +577,7 @@ transformColumnRef(ParseState *pstate, ColumnRef *cref) case EXPR_KIND_COPY_WHERE: case EXPR_KIND_GENERATED_COLUMN: case EXPR_KIND_CYCLE_MARK: + case EXPR_KIND_RPR_DEFINE: /* okay */ break; @@ -1862,6 +1863,9 @@ transformSubLink(ParseState *pstate, SubLink *sublink) case EXPR_KIND_GENERATED_COLUMN: err = _("cannot use subquery in column generation expression"); break; + case EXPR_KIND_RPR_DEFINE: + err = _("cannot use subquery in DEFINE expression"); + break; /* * There is intentionally no default: case here, so that the @@ -3221,6 +3225,8 @@ ParseExprKindName(ParseExprKind exprKind) return "GENERATED AS"; case EXPR_KIND_CYCLE_MARK: return "CYCLE"; + case EXPR_KIND_RPR_DEFINE: + return "DEFINE"; /* * There is intentionally no default: case here, so that the diff --git a/src/backend/parser/parse_func.c b/src/backend/parser/parse_func.c index 24f6745923b..357b236a818 100644 --- a/src/backend/parser/parse_func.c +++ b/src/backend/parser/parse_func.c @@ -2783,6 +2783,9 @@ check_srf_call_placement(ParseState *pstate, Node *last_srf, int location) case EXPR_KIND_CYCLE_MARK: errkind = true; break; + case EXPR_KIND_RPR_DEFINE: + errkind = true; + break; /* * There is intentionally no default: case here, so that the diff --git a/src/backend/parser/parse_rpr.c b/src/backend/parser/parse_rpr.c new file mode 100644 index 00000000000..de648293b27 --- /dev/null +++ b/src/backend/parser/parse_rpr.c @@ -0,0 +1,340 @@ +/*------------------------------------------------------------------------- + * + * parse_rpr.c + * handle Row Pattern Recognition in parser + * + * Portions Copyright (c) 1996-2025, PostgreSQL Global Development Group + * Portions Copyright (c) 1994, Regents of the University of California + * + * + * IDENTIFICATION + * src/backend/parser/parse_rpr.c + * + *------------------------------------------------------------------------- + */ + +#include "postgres.h" + +#include "nodes/makefuncs.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_clause.h" +#include "parser/parse_collate.h" +#include "parser/parse_expr.h" +#include "parser/parse_rpr.h" +#include "parser/parse_target.h" + +/* DEFINE variable name initials */ +static const char *defineVariableInitials = "abcdefghijklmnopqrstuvwxyz"; + +/* Forward declarations */ +static void collectRPRPatternVarNames(RPRPatternNode *node, List **varNames); +static List *transformDefineClause(ParseState *pstate, WindowClause *wc, + WindowDef *windef, List **targetlist); +static void transformPatternClause(ParseState *pstate, WindowClause *wc, + WindowDef *windef); +static RPRPatternNode *copyRPRPatternNode(RPRPatternNode *node); + +/* + * transformRPR + * Process Row Pattern Recognition related clauses + */ +void +transformRPR(ParseState *pstate, WindowClause *wc, WindowDef *windef, + List **targetlist) +{ + /* + * Window definition exists? + */ + if (windef == NULL) + return; + + /* + * Row Pattern Common Syntax clause exists? + */ + if (windef->rpCommonSyntax == NULL) + return; + + /* Check Frame option. Frame must start at current row */ + if ((wc->frameOptions & FRAMEOPTION_START_CURRENT_ROW) == 0) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("FRAME must start at current row when row pattern recognition is used"))); + + /* Transform AFTER MATCH SKIP TO clause */ + wc->rpSkipTo = windef->rpCommonSyntax->rpSkipTo; + + /* Transform SEEK or INITIAL clause */ + wc->initial = windef->rpCommonSyntax->initial; + + /* Transform DEFINE clause into list of TargetEntry's */ + wc->defineClause = transformDefineClause(pstate, wc, windef, targetlist); + + /* Check PATTERN clause and copy to patternClause */ + transformPatternClause(pstate, wc, windef); +} + +/* + * collectRPRPatternVarNames + * Collect all variable names from RPRPatternNode tree. + */ +static void +collectRPRPatternVarNames(RPRPatternNode *node, List **varNames) +{ + ListCell *lc; + + if (node == NULL) + return; + + switch (node->nodeType) + { + case RPR_PATTERN_VAR: + /* Add variable name if not already in list */ + { + bool found = false; + + foreach(lc, *varNames) + { + if (strcmp(strVal(lfirst(lc)), node->varName) == 0) + { + found = true; + break; + } + } + if (!found) + *varNames = lappend(*varNames, makeString(pstrdup(node->varName))); + } + break; + + case RPR_PATTERN_SEQ: + case RPR_PATTERN_ALT: + case RPR_PATTERN_GROUP: + /* Recurse into children */ + foreach(lc, node->children) + { + collectRPRPatternVarNames((RPRPatternNode *) lfirst(lc), varNames); + } + break; + } +} + +/* + * transformDefineClause + * Process DEFINE clause and transform ResTarget into list of + * TargetEntry. + * + * XXX we only support column reference in row pattern definition search + * condition, e.g. "price". <row pattern definition variable name>.<column + * reference> is not supported, e.g. "A.price". + */ +static List * +transformDefineClause(ParseState *pstate, WindowClause *wc, WindowDef *windef, + List **targetlist) +{ + ListCell *lc, + *l; + ResTarget *restarget, + *r; + List *restargets; + List *defineClause; + char *name; + int initialLen; + int numinitials; + List *patternVarNames = NIL; + + /* + * If Row Definition Common Syntax exists, DEFINE clause must exist. (the + * raw parser should have already checked it.) + */ + Assert(windef->rpCommonSyntax->rpDefs != NULL); + + /* + * Collect all pattern variable names from the pattern tree. + */ + collectRPRPatternVarNames(windef->rpCommonSyntax->rpPattern, &patternVarNames); + + /* + * Check and add "A AS A IS TRUE" if pattern variable is missing in DEFINE + * per the SQL standard. + */ + restargets = NIL; + foreach(lc, patternVarNames) + { + bool found = false; + + name = strVal(lfirst(lc)); + + foreach(l, windef->rpCommonSyntax->rpDefs) + { + restarget = (ResTarget *) lfirst(l); + + if (!strcmp(restarget->name, name)) + { + found = true; + break; + } + } + + if (!found) + { + /* + * "name" is missing. So create "name AS name IS TRUE" ResTarget + * node and add it to the temporary list. + */ + A_Const *n; + + restarget = makeNode(ResTarget); + n = makeNode(A_Const); + n->val.boolval.type = T_Boolean; + n->val.boolval.boolval = true; + n->location = -1; + restarget->name = pstrdup(name); + restarget->indirection = NIL; + restarget->val = (Node *) n; + restarget->location = -1; + restargets = lappend((List *) restargets, restarget); + } + } + + if (list_length(restargets) >= 1) + { + /* add missing DEFINEs */ + windef->rpCommonSyntax->rpDefs = + list_concat(windef->rpCommonSyntax->rpDefs, restargets); + list_free(restargets); + } + + /* + * Check for duplicate row pattern definition variables. The standard + * requires that no two row pattern definition variable names shall be + * equivalent. + */ + restargets = NIL; + numinitials = 0; + initialLen = strlen(defineVariableInitials); + foreach(lc, windef->rpCommonSyntax->rpDefs) + { + char initial[2]; + + restarget = (ResTarget *) lfirst(lc); + name = restarget->name; + + /* + * Add DEFINE expression (Restarget->val) to the targetlist as a + * TargetEntry if it does not exist yet. Planner will add the column + * ref var node to the outer plan's target list later on. This makes + * DEFINE expression could access the outer tuple while evaluating + * PATTERN. + * + * XXX: adding whole expressions of DEFINE to the plan.targetlist is + * not so good, because it's not necessary to evalute the expression + * in the target list while running the plan. We should extract the + * var nodes only then add them to the plan.targetlist. + */ + findTargetlistEntrySQL99(pstate, (Node *) restarget->val, + targetlist, EXPR_KIND_RPR_DEFINE); + + /* + * Make sure that the row pattern definition search condition is a + * boolean expression. + */ + transformWhereClause(pstate, restarget->val, + EXPR_KIND_RPR_DEFINE, "DEFINE"); + + foreach(l, restargets) + { + char *n; + + r = (ResTarget *) lfirst(l); + n = r->name; + + if (!strcmp(n, name)) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("row pattern definition variable name \"%s\" appears more than once in DEFINE clause", + name), + parser_errposition(pstate, exprLocation((Node *) r)))); + } + + /* + * Create list of row pattern DEFINE variable name's initial. We + * assign [a-z] to them (up to 26 variable names are allowed). + */ + if (numinitials >= initialLen) + { + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("number of row pattern definition variable names exceeds %d", + initialLen), + parser_errposition(pstate, + exprLocation((Node *) restarget)))); + } + initial[0] = defineVariableInitials[numinitials++]; + initial[1] = '\0'; + wc->defineInitial = lappend(wc->defineInitial, + makeString(pstrdup(initial))); + + restargets = lappend(restargets, restarget); + } + list_free(restargets); + + /* turns a list of ResTarget's into a list of TargetEntry's */ + defineClause = transformTargetList(pstate, windef->rpCommonSyntax->rpDefs, + EXPR_KIND_RPR_DEFINE); + + /* mark column origins */ + markTargetListOrigins(pstate, defineClause); + + /* mark all nodes in the DEFINE clause tree with collation information */ + assign_expr_collations(pstate, (Node *) defineClause); + + return defineClause; +} + +/* + * transformPatternClause + * Process PATTERN clause and return PATTERN clause in the raw parse tree + * + * windef->rpCommonSyntax must exist. + * + * The original (unoptimized) pattern is stored for deparsing (pg_get_viewdef). + * Optimization happens later in the planner phase (buildRPRPattern). + */ +static void +transformPatternClause(ParseState *pstate, WindowClause *wc, + WindowDef *windef) +{ + Assert(windef->rpCommonSyntax != NULL); + + /* Store original AST for deparsing (no optimization here) */ + wc->rpPattern = copyRPRPatternNode(windef->rpCommonSyntax->rpPattern); +} + +/* + * copyRPRPatternNode + * Deep copy an RPRPatternNode tree. + */ +static RPRPatternNode * +copyRPRPatternNode(RPRPatternNode *node) +{ + RPRPatternNode *copy; + ListCell *lc; + + if (node == NULL) + return NULL; + + copy = makeNode(RPRPatternNode); + copy->nodeType = node->nodeType; + copy->varName = node->varName ? pstrdup(node->varName) : NULL; + copy->min = node->min; + copy->max = node->max; + copy->reluctant = node->reluctant; + copy->children = NIL; + + foreach(lc, node->children) + { + copy->children = lappend(copy->children, + copyRPRPatternNode(lfirst(lc))); + } + + return copy; +} diff --git a/src/include/parser/parse_clause.h b/src/include/parser/parse_clause.h index fe234611007..8aaac881f2b 100644 --- a/src/include/parser/parse_clause.h +++ b/src/include/parser/parse_clause.h @@ -52,6 +52,9 @@ extern List *addTargetToSortList(ParseState *pstate, TargetEntry *tle, extern Index assignSortGroupRef(TargetEntry *tle, List *tlist); extern bool targetIsInSortList(TargetEntry *tle, Oid sortop, List *sortList); +extern TargetEntry *findTargetlistEntrySQL99(ParseState *pstate, Node *node, + List **tlist, ParseExprKind exprKind); + /* functions in parse_jsontable.c */ extern ParseNamespaceItem *transformJsonTable(ParseState *pstate, JsonTable *jt); diff --git a/src/include/parser/parse_rpr.h b/src/include/parser/parse_rpr.h new file mode 100644 index 00000000000..40e0258d2e6 --- /dev/null +++ b/src/include/parser/parse_rpr.h @@ -0,0 +1,22 @@ +/*------------------------------------------------------------------------- + * + * parse_rpr.h + * handle Row Pattern Recognition in parser + * + * + * Portions Copyright (c) 1996-2025, PostgreSQL Global Development Group + * Portions Copyright (c) 1994, Regents of the University of California + * + * src/include/parser/parse_rpr.h + * + *------------------------------------------------------------------------- + */ +#ifndef PARSE_RPR_H +#define PARSE_RPR_H + +#include "parser/parse_node.h" + +extern void transformRPR(ParseState *pstate, WindowClause *wc, + WindowDef *windef, List **targetlist); + +#endif /* PARSE_RPR_H */ -- 2.43.0 [application/octet-stream] v38-0003-Row-pattern-recognition-patch-rewriter.patch (5.7K, ../../[email protected]/4-v38-0003-Row-pattern-recognition-patch-rewriter.patch) download | inline diff: From 26649a811ba7ad22cd7992cb46e1f23932af2bd4 Mon Sep 17 00:00:00 2001 From: Tatsuo Ishii <[email protected]> Date: Thu, 15 Jan 2026 13:26:44 +0900 Subject: [PATCH v38 3/8] Row pattern recognition patch (rewriter). --- src/backend/utils/adt/ruleutils.c | 172 ++++++++++++++++++++++++++++++ 1 file changed, 172 insertions(+) diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c index 033b625f3fc..b8a3ee2915b 100644 --- a/src/backend/utils/adt/ruleutils.c +++ b/src/backend/utils/adt/ruleutils.c @@ -438,6 +438,12 @@ static void get_rule_groupingset(GroupingSet *gset, List *targetlist, bool omit_parens, deparse_context *context); static void get_rule_orderby(List *orderList, List *targetList, bool force_colno, deparse_context *context); +static void append_pattern_quantifier(StringInfo buf, RPRPatternNode *node); +static void get_rule_pattern_node(RPRPatternNode *node, deparse_context *context); +static void get_rule_pattern(RPRPatternNode *rpPattern, bool force_colno, + deparse_context *context); +static void get_rule_define(List *defineClause, bool force_colno, + deparse_context *context); static void get_rule_windowclause(Query *query, deparse_context *context); static void get_rule_windowspec(WindowClause *wc, List *targetList, deparse_context *context); @@ -6744,6 +6750,127 @@ get_rule_orderby(List *orderList, List *targetList, } } +/* + * Helper function to append quantifier string for pattern node + */ +static void +append_pattern_quantifier(StringInfo buf, RPRPatternNode *node) +{ + bool has_quantifier = true; + + if (node->min == 1 && node->max == 1) + { + /* {1,1} = no quantifier */ + has_quantifier = false; + } + else if (node->min == 0 && node->max == INT_MAX) + appendStringInfoChar(buf, '*'); + else if (node->min == 1 && node->max == INT_MAX) + appendStringInfoChar(buf, '+'); + else if (node->min == 0 && node->max == 1) + appendStringInfoChar(buf, '?'); + else if (node->max == INT_MAX) + appendStringInfo(buf, "{%d,}", node->min); + else if (node->min == node->max) + appendStringInfo(buf, "{%d}", node->min); + else + appendStringInfo(buf, "{%d,%d}", node->min, node->max); + + if (node->reluctant && has_quantifier) + appendStringInfoChar(buf, '?'); +} + +/* + * Recursive helper to display RPRPatternNode tree + */ +static void +get_rule_pattern_node(RPRPatternNode *node, deparse_context *context) +{ + StringInfo buf = context->buf; + ListCell *lc; + const char *sep; + + if (node == NULL) + return; + + switch (node->nodeType) + { + case RPR_PATTERN_VAR: + appendStringInfoString(buf, node->varName); + append_pattern_quantifier(buf, node); + break; + + case RPR_PATTERN_SEQ: + sep = ""; + foreach(lc, node->children) + { + appendStringInfoString(buf, sep); + get_rule_pattern_node((RPRPatternNode *) lfirst(lc), context); + sep = " "; + } + break; + + case RPR_PATTERN_ALT: + sep = ""; + foreach(lc, node->children) + { + appendStringInfoString(buf, sep); + get_rule_pattern_node((RPRPatternNode *) lfirst(lc), context); + sep = " | "; + } + break; + + case RPR_PATTERN_GROUP: + appendStringInfoChar(buf, '('); + sep = ""; + foreach(lc, node->children) + { + appendStringInfoString(buf, sep); + get_rule_pattern_node((RPRPatternNode *) lfirst(lc), context); + sep = " "; + } + appendStringInfoChar(buf, ')'); + append_pattern_quantifier(buf, node); + break; + } +} + +/* + * Display a PATTERN clause. + */ +static void +get_rule_pattern(RPRPatternNode *rpPattern, bool force_colno, + deparse_context *context) +{ + StringInfo buf = context->buf; + + appendStringInfoChar(buf, '('); + get_rule_pattern_node(rpPattern, context); + appendStringInfoChar(buf, ')'); +} + +/* + * Display a DEFINE clause. + */ +static void +get_rule_define(List *defineClause, bool force_colno, deparse_context *context) +{ + StringInfo buf = context->buf; + const char *sep; + ListCell *lc_def; + + sep = " "; + + foreach(lc_def, defineClause) + { + TargetEntry *te = (TargetEntry *) lfirst(lc_def); + + appendStringInfo(buf, "%s%s AS ", sep, te->resname); + get_rule_expr((Node *) te->expr, context, false); + sep = ",\n "; + } +} + /* * Display a WINDOW clause. * @@ -6824,6 +6951,7 @@ get_rule_windowspec(WindowClause *wc, List *targetList, get_rule_orderby(wc->orderClause, targetList, false, context); needspace = true; } + /* framing clause is never inherited, so print unless it's default */ if (wc->frameOptions & FRAMEOPTION_NONDEFAULT) { @@ -6832,7 +6960,51 @@ get_rule_windowspec(WindowClause *wc, List *targetList, get_window_frame_options(wc->frameOptions, wc->startOffset, wc->endOffset, context); + needspace = true; + } + + /* RPR */ + if (wc->rpSkipTo == ST_NEXT_ROW) + { + if (needspace) + appendStringInfoChar(buf, ' '); + appendStringInfoString(buf, + "\n AFTER MATCH SKIP TO NEXT ROW "); + needspace = true; + } + else if (wc->rpSkipTo == ST_PAST_LAST_ROW) + { + if (needspace) + appendStringInfoChar(buf, ' '); + appendStringInfoString(buf, + "\n AFTER MATCH SKIP PAST LAST ROW "); + needspace = true; + } + if (wc->initial) + { + if (needspace) + appendStringInfoChar(buf, ' '); + appendStringInfoString(buf, "\n INITIAL"); + needspace = true; } + if (wc->rpPattern) + { + if (needspace) + appendStringInfoChar(buf, ' '); + appendStringInfoString(buf, "\n PATTERN "); + get_rule_pattern(wc->rpPattern, false, context); + needspace = true; + } + + if (wc->defineClause) + { + if (needspace) + appendStringInfoChar(buf, ' '); + appendStringInfoString(buf, "\n DEFINE\n"); + get_rule_define(wc->defineClause, false, context); + appendStringInfoChar(buf, ' '); + } + appendStringInfoChar(buf, ')'); } -- 2.43.0 [application/octet-stream] v38-0004-Row-pattern-recognition-patch-planner.patch (40.6K, ../../[email protected]/5-v38-0004-Row-pattern-recognition-patch-planner.patch) download | inline diff: From 259ce0040203f0e81aaa3ac625c7f9f9b4998ed8 Mon Sep 17 00:00:00 2001 From: Tatsuo Ishii <[email protected]> Date: Thu, 15 Jan 2026 13:26:44 +0900 Subject: [PATCH v38 4/8] Row pattern recognition patch (planner). --- src/backend/optimizer/plan/Makefile | 1 + src/backend/optimizer/plan/createplan.c | 64 +- src/backend/optimizer/plan/meson.build | 1 + src/backend/optimizer/plan/planner.c | 3 + src/backend/optimizer/plan/rpr.c | 1024 +++++++++++++++++++++ src/backend/optimizer/plan/setrefs.c | 27 +- src/backend/optimizer/prep/prepjointree.c | 9 + src/include/nodes/plannodes.h | 76 ++ src/include/optimizer/rpr.h | 46 + 9 files changed, 1233 insertions(+), 18 deletions(-) create mode 100644 src/backend/optimizer/plan/rpr.c create mode 100644 src/include/optimizer/rpr.h diff --git a/src/backend/optimizer/plan/Makefile b/src/backend/optimizer/plan/Makefile index 80ef162e484..7e0167789d8 100644 --- a/src/backend/optimizer/plan/Makefile +++ b/src/backend/optimizer/plan/Makefile @@ -19,6 +19,7 @@ OBJS = \ planagg.o \ planmain.o \ planner.o \ + rpr.o \ setrefs.o \ subselect.o diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c index af41ca69929..51334f80a5c 100644 --- a/src/backend/optimizer/plan/createplan.c +++ b/src/backend/optimizer/plan/createplan.c @@ -37,6 +37,7 @@ #include "optimizer/prep.h" #include "optimizer/restrictinfo.h" #include "optimizer/subselect.h" +#include "optimizer/rpr.h" #include "optimizer/tlist.h" #include "parser/parse_clause.h" #include "parser/parsetree.h" @@ -289,7 +290,10 @@ static Memoize *make_memoize(Plan *lefttree, Oid *hashoperators, static WindowAgg *make_windowagg(List *tlist, WindowClause *wc, int partNumCols, AttrNumber *partColIdx, Oid *partOperators, Oid *partCollations, int ordNumCols, AttrNumber *ordColIdx, Oid *ordOperators, Oid *ordCollations, - List *runCondition, List *qual, bool topWindow, + List *runCondition, RPSkipTo rpSkipTo, + RPRPattern *compiledPattern, + List *defineClause, List *defineInitial, + List *qual, bool topWindow, Plan *lefttree); static Group *make_group(List *tlist, List *qual, int numGroupCols, AttrNumber *grpColIdx, Oid *grpOperators, Oid *grpCollations, @@ -2530,21 +2534,37 @@ create_windowagg_plan(PlannerInfo *root, WindowAggPath *best_path) ordNumCols++; } - /* And finally we can make the WindowAgg node */ - plan = make_windowagg(tlist, - wc, - partNumCols, - partColIdx, - partOperators, - partCollations, - ordNumCols, - ordColIdx, - ordOperators, - ordCollations, - best_path->runCondition, - best_path->qual, - best_path->topwindow, - subplan); + /* Build defineVariableList from defineClause for pattern compilation */ + { + List *defineVariableList = NIL; + + foreach(lc, wc->defineClause) + { + TargetEntry *te = (TargetEntry *) lfirst(lc); + defineVariableList = lappend(defineVariableList, + makeString(pstrdup(te->resname))); + } + + /* And finally we can make the WindowAgg node */ + plan = make_windowagg(tlist, + wc, + partNumCols, + partColIdx, + partOperators, + partCollations, + ordNumCols, + ordColIdx, + ordOperators, + ordCollations, + best_path->runCondition, + wc->rpSkipTo, + buildRPRPattern(wc->rpPattern, defineVariableList), + wc->defineClause, + wc->defineInitial, + best_path->qual, + best_path->topwindow, + subplan); + } copy_generic_path_info(&plan->plan, (Path *) best_path); @@ -6610,7 +6630,10 @@ static WindowAgg * make_windowagg(List *tlist, WindowClause *wc, int partNumCols, AttrNumber *partColIdx, Oid *partOperators, Oid *partCollations, int ordNumCols, AttrNumber *ordColIdx, Oid *ordOperators, Oid *ordCollations, - List *runCondition, List *qual, bool topWindow, Plan *lefttree) + List *runCondition, RPSkipTo rpSkipTo, + RPRPattern *compiledPattern, + List *defineClause, List *defineInitial, + List *qual, bool topWindow, Plan *lefttree) { WindowAgg *node = makeNode(WindowAgg); Plan *plan = &node->plan; @@ -6637,6 +6660,13 @@ make_windowagg(List *tlist, WindowClause *wc, node->inRangeAsc = wc->inRangeAsc; node->inRangeNullsFirst = wc->inRangeNullsFirst; node->topWindow = topWindow; + node->rpSkipTo = rpSkipTo; + + /* Store compiled pattern for NFA execution */ + node->rpPattern = compiledPattern; + + node->defineClause = defineClause; + node->defineInitial = defineInitial; plan->targetlist = tlist; plan->lefttree = lefttree; diff --git a/src/backend/optimizer/plan/meson.build b/src/backend/optimizer/plan/meson.build index c565b2adbcc..5b2381cd774 100644 --- a/src/backend/optimizer/plan/meson.build +++ b/src/backend/optimizer/plan/meson.build @@ -7,6 +7,7 @@ backend_sources += files( 'planagg.c', 'planmain.c', 'planner.c', + 'rpr.c', 'setrefs.c', 'subselect.c', ) diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c index 7a6b8b749f2..2a93feee25b 100644 --- a/src/backend/optimizer/plan/planner.c +++ b/src/backend/optimizer/plan/planner.c @@ -964,6 +964,9 @@ subquery_planner(PlannerGlobal *glob, Query *parse, char *plan_name, EXPRKIND_LIMIT); wc->endOffset = preprocess_expression(root, wc->endOffset, EXPRKIND_LIMIT); + wc->defineClause = (List *) preprocess_expression(root, + (Node *) wc->defineClause, + EXPRKIND_TARGET); } parse->limitOffset = preprocess_expression(root, parse->limitOffset, diff --git a/src/backend/optimizer/plan/rpr.c b/src/backend/optimizer/plan/rpr.c new file mode 100644 index 00000000000..0eed6b865ae --- /dev/null +++ b/src/backend/optimizer/plan/rpr.c @@ -0,0 +1,1024 @@ +/*------------------------------------------------------------------------- + * + * rpr.c + * Row Pattern Recognition pattern compilation for planner + * + * This file contains functions for optimizing RPR pattern AST and + * compiling it to bytecode for execution by WindowAgg. + * + * Portions Copyright (c) 1996-2025, PostgreSQL Global Development Group + * Portions Copyright (c) 1994, Regents of the University of California + * + * + * IDENTIFICATION + * src/backend/optimizer/plan/rpr.c + * + *------------------------------------------------------------------------- + */ + +#include "postgres.h" + +#include "nodes/makefuncs.h" +#include "optimizer/rpr.h" + +/* Forward declarations */ +static bool rprPatternEqual(RPRPatternNode *a, RPRPatternNode *b); +static RPRPatternNode *optimizeRPRPattern(RPRPatternNode *pattern); +static void scanRPRPattern(RPRPatternNode *node, char **varNames, int *numVars, + int *numElements, RPRDepth depth, RPRDepth *maxDepth); +static void fillRPRPattern(RPRPatternNode *node, RPRPattern *pat, + int *idx, RPRDepth depth); +static RPRVarId getVarIdFromPattern(RPRPattern *pat, const char *varName); +static void computeAbsorbability(RPRPattern *pattern); + +/* + * rprPatternChildrenEqual + * Compare children of two GROUP nodes for equality. + * + * Returns true if the children lists are structurally identical. + * Used for GROUP merge optimization where we ignore outer quantifiers. + */ +static bool +rprPatternChildrenEqual(List *a, List *b) +{ + ListCell *lca, + *lcb; + + if (list_length(a) != list_length(b)) + return false; + + forboth(lca, a, lcb, b) + { + if (!rprPatternEqual((RPRPatternNode *) lfirst(lca), + (RPRPatternNode *) lfirst(lcb))) + return false; + } + + return true; +} + +/* + * rprPatternEqual + * Compare two RPRPatternNode trees for equality. + * + * Returns true if the trees are structurally identical. + */ +static bool +rprPatternEqual(RPRPatternNode *a, RPRPatternNode *b) +{ + ListCell *lca, + *lcb; + + if (a == NULL && b == NULL) + return true; + if (a == NULL || b == NULL) + return false; + + /* Must have same node type and quantifiers */ + if (a->nodeType != b->nodeType) + return false; + if (a->min != b->min || a->max != b->max) + return false; + if (a->reluctant != b->reluctant) + return false; + + switch (a->nodeType) + { + case RPR_PATTERN_VAR: + return strcmp(a->varName, b->varName) == 0; + + case RPR_PATTERN_SEQ: + case RPR_PATTERN_ALT: + case RPR_PATTERN_GROUP: + /* Must have same number of children */ + if (list_length(a->children) != list_length(b->children)) + return false; + + /* Compare each child */ + forboth(lca, a->children, lcb, b->children) + { + if (!rprPatternEqual((RPRPatternNode *) lfirst(lca), + (RPRPatternNode *) lfirst(lcb))) + return false; + } + return true; + } + + return false; /* keep compiler quiet */ +} + +/* + * optimizeRPRPattern + * Optimize RPRPatternNode tree in a single pass. + * + * Optimizations applied (bottom-up, in order per node): + * 1. Flatten nested SEQ: (A (B C)) -> (A B C) + * 2. Flatten nested ALT: (A | (B | C)) -> (A | B | C) + * 3. Unwrap GROUP{1,1}: ((A)) -> A, (A B){1,1} -> A B + * 4. Quantifier multiplication: (A{2}){3} -> A{6} + * 5. Remove duplicate alternatives: (A | B | A) -> (A | B) + * 6. Merge consecutive vars: A A A -> A{3,3} + * 7. Remove single-item SEQ/ALT wrappers + */ +static RPRPatternNode * +optimizeRPRPattern(RPRPatternNode *pattern) +{ + ListCell *lc; + List *newChildren; + + if (pattern == NULL) + return NULL; + + switch (pattern->nodeType) + { + case RPR_PATTERN_VAR: + /* Leaf node - nothing to optimize */ + return pattern; + + case RPR_PATTERN_SEQ: + { + RPRPatternNode *prev = NULL; + + /* Recursively optimize children, flatten SEQ/GROUP{1,1} */ + newChildren = NIL; + foreach(lc, pattern->children) + { + RPRPatternNode *child = (RPRPatternNode *) lfirst(lc); + RPRPatternNode *opt = optimizeRPRPattern(child); + + /* Flatten GROUP{1,1} or nested SEQ */ + if ((opt->nodeType == RPR_PATTERN_GROUP && + opt->min == 1 && opt->max == 1 && !opt->reluctant) || + opt->nodeType == RPR_PATTERN_SEQ) + { + newChildren = list_concat(newChildren, + list_copy(opt->children)); + } + else + { + newChildren = lappend(newChildren, opt); + } + } + + /* + * Merge consecutive identical VAR nodes with any quantifier. + * A{m1,M1} A{m2,M2} -> A{m1+m2, M1+M2} + * where INF + x = INF + */ + { + List *mergedChildren = NIL; + + foreach(lc, newChildren) + { + RPRPatternNode *child = (RPRPatternNode *) lfirst(lc); + + if (child->nodeType == RPR_PATTERN_VAR && !child->reluctant) + { + if (prev != NULL && + prev->nodeType == RPR_PATTERN_VAR && + strcmp(prev->varName, child->varName) == 0 && + !prev->reluctant) + { + /* + * Merge: accumulate min/max into prev. + * INF + anything = INF + */ + prev->min += child->min; + if (prev->max == RPR_QUANTITY_INF || + child->max == RPR_QUANTITY_INF) + prev->max = RPR_QUANTITY_INF; + else + prev->max += child->max; + } + else + { + /* Flush previous and start new */ + if (prev != NULL) + mergedChildren = lappend(mergedChildren, prev); + prev = child; + } + } + else + { + /* Non-mergeable - flush previous */ + if (prev != NULL) + mergedChildren = lappend(mergedChildren, prev); + mergedChildren = lappend(mergedChildren, child); + prev = NULL; + } + } + + /* Flush remaining */ + if (prev != NULL) + mergedChildren = lappend(mergedChildren, prev); + + newChildren = mergedChildren; + } + + /* + * Merge sequence prefix/suffix into GROUP with matching children. + * A B (A B)+ -> (A B){2,} + * (A B)+ A B -> (A B){2,} + * A B (A B)+ A B -> (A B){3,} + */ + { + List *groupMergedChildren = NIL; + int numChildren = list_length(newChildren); + int i; + bool merged = false; + int skipUntil = -1; /* skip suffix elements already absorbed */ + + for (i = 0; i < numChildren; i++) + { + RPRPatternNode *child = (RPRPatternNode *) list_nth(newChildren, i); + + /* Skip elements that were absorbed as suffix */ + if (i < skipUntil) + continue; + + /* + * If this is a GROUP, see if preceding/following elements + * match its children. + * GROUP's content may be wrapped in a SEQ - unwrap for comparison. + */ + if (child->nodeType == RPR_PATTERN_GROUP && !child->reluctant) + { + List *groupContent = child->children; + int groupChildCount; + int prefixLen = list_length(groupMergedChildren); + + /* + * If GROUP has single SEQ child, compare with SEQ's children. + * e.g., (A B)+ is GROUP[SEQ[A,B]], we want to compare [A,B]. + */ + if (list_length(groupContent) == 1) + { + RPRPatternNode *inner = (RPRPatternNode *) linitial(groupContent); + + if (inner->nodeType == RPR_PATTERN_SEQ) + groupContent = inner->children; + } + + groupChildCount = list_length(groupContent); + + /* + * PREFIX MERGE: Check if preceding elements match. + * Keep absorbing as long as we have matching prefixes. + */ + while (prefixLen >= groupChildCount && groupChildCount > 0) + { + List *prefixElements = NIL; + int j; + + /* Extract last groupChildCount elements from prefix */ + for (j = prefixLen - groupChildCount; j < prefixLen; j++) + { + prefixElements = lappend(prefixElements, + list_nth(groupMergedChildren, j)); + } + + /* Compare with GROUP's (possibly unwrapped) children */ + if (rprPatternChildrenEqual(prefixElements, groupContent)) + { + /* + * Match! Merge by incrementing GROUP's min. + * Remove the prefix elements from output. + */ + child->min += 1; + + /* Rebuild groupMergedChildren without matched prefix */ + { + List *trimmed = NIL; + + for (j = 0; j < prefixLen - groupChildCount; j++) + { + trimmed = lappend(trimmed, + list_nth(groupMergedChildren, j)); + } + groupMergedChildren = trimmed; + prefixLen = list_length(groupMergedChildren); + } + merged = true; + } + else + { + list_free(prefixElements); + break; + } + + list_free(prefixElements); + } + + /* + * SUFFIX MERGE: Check if following elements match. + * Keep absorbing as long as we have matching suffixes. + */ + while (i + groupChildCount < numChildren && groupChildCount > 0) + { + List *suffixElements = NIL; + int j; + int suffixStart = i + 1; + + /* Adjust for already absorbed elements */ + if (skipUntil > suffixStart) + break; + + /* Extract next groupChildCount elements as suffix */ + for (j = 0; j < groupChildCount; j++) + { + int idx = suffixStart + j; + + if (idx >= numChildren) + break; + suffixElements = lappend(suffixElements, + list_nth(newChildren, idx)); + } + + /* Compare with GROUP's children */ + if (list_length(suffixElements) == groupChildCount && + rprPatternChildrenEqual(suffixElements, groupContent)) + { + /* + * Match! Absorb suffix by incrementing min and skipping. + */ + child->min += 1; + skipUntil = suffixStart + groupChildCount; + merged = true; + /* Update i to continue suffix check after absorbed elements */ + i = skipUntil - 1; + } + else + { + list_free(suffixElements); + break; + } + + list_free(suffixElements); + } + } + + groupMergedChildren = lappend(groupMergedChildren, child); + } + + if (merged) + newChildren = groupMergedChildren; + } + + pattern->children = newChildren; + + /* Unwrap single-item SEQ */ + if (list_length(pattern->children) == 1) + return (RPRPatternNode *) linitial(pattern->children); + + return pattern; + } + + case RPR_PATTERN_ALT: + { + ListCell *lc2; + + /* Recursively optimize children, flatten nested ALT */ + newChildren = NIL; + foreach(lc, pattern->children) + { + RPRPatternNode *child = (RPRPatternNode *) lfirst(lc); + RPRPatternNode *opt = optimizeRPRPattern(child); + + if (opt->nodeType == RPR_PATTERN_ALT) + { + newChildren = list_concat(newChildren, + list_copy(opt->children)); + } + else + { + newChildren = lappend(newChildren, opt); + } + } + + /* Remove duplicate alternatives */ + { + List *uniqueChildren = NIL; + + foreach(lc, newChildren) + { + RPRPatternNode *child = (RPRPatternNode *) lfirst(lc); + bool isDuplicate = false; + + foreach(lc2, uniqueChildren) + { + if (rprPatternEqual((RPRPatternNode *) lfirst(lc2), child)) + { + isDuplicate = true; + break; + } + } + + if (!isDuplicate) + uniqueChildren = lappend(uniqueChildren, child); + } + + pattern->children = uniqueChildren; + } + + /* Unwrap single-item ALT */ + if (list_length(pattern->children) == 1) + return (RPRPatternNode *) linitial(pattern->children); + + return pattern; + } + + case RPR_PATTERN_GROUP: + /* Recursively optimize children */ + newChildren = NIL; + foreach(lc, pattern->children) + { + RPRPatternNode *child = (RPRPatternNode *) lfirst(lc); + + newChildren = lappend(newChildren, optimizeRPRPattern(child)); + } + pattern->children = newChildren; + + /* Quantifier multiplication: (A{m}){n} -> A{m*n} */ + if (list_length(pattern->children) == 1 && !pattern->reluctant) + { + RPRPatternNode *child = (RPRPatternNode *) linitial(pattern->children); + + if (child->nodeType == RPR_PATTERN_VAR && !child->reluctant) + { + if (pattern->max != RPR_QUANTITY_INF && child->max != RPR_QUANTITY_INF) + { + int64 new_min_64 = (int64) pattern->min * child->min; + int64 new_max_64 = (int64) pattern->max * child->max; + + if (new_min_64 < RPR_QUANTITY_INF && new_max_64 < RPR_QUANTITY_INF) + { + child->min = (int) new_min_64; + child->max = (int) new_max_64; + return child; + } + } + } + } + + /* Unwrap GROUP{1,1} */ + if (pattern->min == 1 && pattern->max == 1 && !pattern->reluctant) + { + if (list_length(pattern->children) == 1) + return (RPRPatternNode *) linitial(pattern->children); + + /* Multiple children: convert to SEQ */ + pattern->nodeType = RPR_PATTERN_SEQ; + } + + return pattern; + } + + return pattern; /* keep compiler quiet */ +} + +/* + * scanRPRPattern + * Single-pass scan: collect variable names and count elements. + * + * Collects unique variable names in pattern encounter order (max RPR_VARID_MAX). + * Also counts elements and tracks maximum nesting depth. + */ +static void +scanRPRPattern(RPRPatternNode *node, char **varNames, int *numVars, + int *numElements, RPRDepth depth, RPRDepth *maxDepth) +{ + ListCell *lc; + int i; + + if (node == NULL) + return; + + /* Track maximum depth */ + if (depth > *maxDepth) + *maxDepth = depth; + + switch (node->nodeType) + { + case RPR_PATTERN_VAR: + /* Count element */ + (*numElements)++; + + /* Collect variable name if not already present */ + for (i = 0; i < *numVars; i++) + { + if (strcmp(varNames[i], node->varName) == 0) + return; /* Already have this variable */ + } + + /* New variable */ + if (*numVars >= RPR_VARID_MAX) + ereport(ERROR, + (errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED), + errmsg("too many pattern variables"), + errdetail("Maximum is %d.", RPR_VARID_MAX))); + + varNames[*numVars] = node->varName; + (*numVars)++; + break; + + case RPR_PATTERN_SEQ: + /* Sequence: just recurse into children */ + foreach(lc, node->children) + { + scanRPRPattern((RPRPatternNode *) lfirst(lc), varNames, numVars, + numElements, depth, maxDepth); + } + break; + + case RPR_PATTERN_GROUP: + /* Recurse into children at increased depth */ + foreach(lc, node->children) + { + scanRPRPattern((RPRPatternNode *) lfirst(lc), varNames, numVars, + numElements, depth + 1, maxDepth); + } + + /* Add END element if group has non-trivial quantifier */ + if (node->min != 1 || node->max != 1) + (*numElements)++; + break; + + case RPR_PATTERN_ALT: + /* Count ALT start element */ + (*numElements)++; + + /* Recurse into children at increased depth */ + foreach(lc, node->children) + { + scanRPRPattern((RPRPatternNode *) lfirst(lc), varNames, numVars, + numElements, depth + 1, maxDepth); + } + break; + } +} + +/* + * getVarIdFromPattern + * Get variable ID for a variable name from RPRPattern. + * + * Returns the index of the variable in the varNames array. + */ +static RPRVarId +getVarIdFromPattern(RPRPattern *pat, const char *varName) +{ + for (int i = 0; i < pat->numVars; i++) + { + if (strcmp(pat->varNames[i], varName) == 0) + return (RPRVarId) i; + } + + /* Should not happen - variable should already be collected */ + elog(ERROR, "pattern variable \"%s\" not found", varName); + pg_unreachable(); +} + +/* + * fillRPRPattern + * Fill the pattern elements array (second pass). + * + * This traverses the AST and fills in the pre-allocated elements array. + * The idx pointer tracks the current position in the array. + */ +static void +fillRPRPattern(RPRPatternNode *node, RPRPattern *pat, int *idx, RPRDepth depth) +{ + ListCell *lc; + RPRPatternElement *elem; + int groupStartIdx; + int altStartIdx; + List *altBranchStarts; + List *altEndPositions; + + if (node == NULL) + return; + + switch (node->nodeType) + { + case RPR_PATTERN_SEQ: + /* Sequence: fill each child in order */ + foreach(lc, node->children) + { + fillRPRPattern((RPRPatternNode *) lfirst(lc), pat, idx, depth); + } + break; + + case RPR_PATTERN_VAR: + /* Variable: create element with varId */ + elem = &pat->elements[*idx]; + memset(elem, 0, sizeof(RPRPatternElement)); + elem->varId = getVarIdFromPattern(pat, node->varName); + elem->depth = depth; + elem->min = node->min; + elem->max = (node->max == INT_MAX) ? RPR_QUANTITY_INF : node->max; + elem->next = RPR_ELEMIDX_INVALID; + elem->jump = RPR_ELEMIDX_INVALID; + if (node->reluctant) + elem->flags |= RPR_ELEM_RELUCTANT; + (*idx)++; + break; + + case RPR_PATTERN_GROUP: + groupStartIdx = *idx; + + /* Fill group content at increased depth */ + foreach(lc, node->children) + { + fillRPRPattern((RPRPatternNode *) lfirst(lc), pat, idx, depth + 1); + } + + /* Add group end marker if group has non-trivial quantifier */ + if (node->min != 1 || node->max != 1) + { + elem = &pat->elements[*idx]; + memset(elem, 0, sizeof(RPRPatternElement)); + elem->varId = RPR_VARID_END; + elem->depth = depth; /* parent depth for iteration count */ + elem->min = node->min; + elem->max = (node->max == INT_MAX) ? RPR_QUANTITY_INF : node->max; + elem->next = RPR_ELEMIDX_INVALID; + elem->jump = groupStartIdx; /* jump back to group start */ + if (node->reluctant) + elem->flags |= RPR_ELEM_RELUCTANT; + (*idx)++; + } + break; + + case RPR_PATTERN_ALT: + /* Add alternation start marker */ + altStartIdx = *idx; + elem = &pat->elements[*idx]; + memset(elem, 0, sizeof(RPRPatternElement)); + elem->varId = RPR_VARID_ALT; + elem->depth = depth; + elem->min = 1; + elem->max = 1; + elem->next = RPR_ELEMIDX_INVALID; + elem->jump = RPR_ELEMIDX_INVALID; + (*idx)++; + + altBranchStarts = NIL; + altEndPositions = NIL; + + /* Fill each alternative */ + foreach(lc, node->children) + { + RPRPatternNode *alt = (RPRPatternNode *) lfirst(lc); + int branchStart = *idx; + + altBranchStarts = lappend_int(altBranchStarts, branchStart); + + fillRPRPattern(alt, pat, idx, depth + 1); + + /* Record end position if any elements were added */ + if (*idx > branchStart) + altEndPositions = lappend_int(altEndPositions, *idx - 1); + } + + /* Set ALT_START.next to first alternative */ + if (list_length(altBranchStarts) > 0) + pat->elements[altStartIdx].next = linitial_int(altBranchStarts); + + /* Set jump on first element of each alternative to next alternative */ + foreach(lc, altBranchStarts) + { + int firstElemIdx = lfirst_int(lc); + + if (lnext(altBranchStarts, lc) != NULL) + { + int nextAltStart = lfirst_int(lnext(altBranchStarts, lc)); + + pat->elements[firstElemIdx].jump = nextAltStart; + } + /* Last alternative's jump stays as -1 (default) */ + } + + /* Set next on last element of each alternative to after the alternation */ + { + int afterAltIdx = *idx; + + foreach(lc, altEndPositions) + { + int endPos = lfirst_int(lc); + + if (pat->elements[endPos].next == RPR_ELEMIDX_INVALID) + pat->elements[endPos].next = afterAltIdx; + } + } + + list_free(altBranchStarts); + list_free(altEndPositions); + break; + } +} + +/* + * buildRPRPattern + * Build flat pattern element array from AST. + * + * Optimizes the pattern tree, then uses 2-pass: count elements, allocate and fill. + * Returns NULL if pattern is NULL. + * + * This function is called from createplan.c during plan creation. + */ +RPRPattern * +buildRPRPattern(RPRPatternNode *pattern, List *defineVariableList) +{ + RPRPattern *result; + RPRPatternNode *optimized; + char *varNamesStack[RPR_VARID_MAX + 1]; /* stack array for collection */ + int numVars = 0; + int numElements = 0; + RPRDepth maxDepth = 0; + int idx; + int finIdx; + int i; + RPRPatternElement *finElem; + ListCell *lc; + + if (pattern == NULL) + return NULL; + + /* Optimize the pattern tree (planner phase) */ + optimized = optimizeRPRPattern(pattern); + + /* + * First, populate varNames from defineVariableList in order. + * This ensures varId == defineIdx for all defined variables, + * eliminating the need for varIdToDefineIdx mapping. + */ + foreach(lc, defineVariableList) + { + char *varName = strVal(lfirst(lc)); + + if (numVars >= RPR_VARID_MAX) + ereport(ERROR, + (errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED), + errmsg("too many pattern variables"), + errdetail("Maximum is %d.", RPR_VARID_MAX))); + + varNamesStack[numVars] = varName; + numVars++; + } + + /* + * Pass 1: Single scan to collect variable names and count elements. + * Variables already in varNamesStack (from DEFINE) are skipped. + * Also counts elements and tracks max depth in one traversal. + */ + scanRPRPattern(optimized, varNamesStack, &numVars, + &numElements, 0, &maxDepth); + numElements++; /* +1 for FIN marker */ + + /* Check element count limit */ + if (numElements > RPR_ELEMIDX_MAX) + ereport(ERROR, + (errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED), + errmsg("pattern too complex"), + errdetail("Pattern has %d elements, maximum is %d.", + numElements, RPR_ELEMIDX_MAX))); + + /* + * Allocate result structure with makeNode for proper NodeTag. + */ + result = makeNode(RPRPattern); + result->numVars = numVars; + result->maxDepth = maxDepth + 1; /* +1: depth is 0-based, need counts[0..maxDepth] */ + result->numElements = numElements; + + /* Build varNames as char** array */ + if (numVars > 0) + { + result->varNames = palloc(numVars * sizeof(char *)); + for (i = 0; i < numVars; i++) + result->varNames[i] = pstrdup(varNamesStack[i]); + } + else + { + result->varNames = NULL; + } + + /* Allocate elements array separately (zero-init for reserved field) */ + result->elements = palloc0(numElements * sizeof(RPRPatternElement)); + + /* + * Pass 2: Fill elements directly into pre-allocated array. + */ + idx = 0; + fillRPRPattern(optimized, result, &idx, 0); + + /* Set up next pointers for elements that don't have one yet */ + finIdx = numElements - 1; /* FIN marker is at the last position */ + for (i = 0; i < finIdx; i++) + { + if (result->elements[i].next == RPR_ELEMIDX_INVALID) + { + /* Point to next element, or to FIN if this is the last */ + result->elements[i].next = (i < finIdx - 1) ? i + 1 : finIdx; + } + } + + /* Add FIN marker at the end */ + finElem = &result->elements[finIdx]; + memset(finElem, 0, sizeof(RPRPatternElement)); + finElem->varId = RPR_VARID_FIN; + finElem->depth = 0; + finElem->min = 1; + finElem->max = 1; + finElem->next = RPR_ELEMIDX_INVALID; + finElem->jump = RPR_ELEMIDX_INVALID; + + /* Compute context absorption eligibility */ + computeAbsorbability(result); + + return result; +} + +/* + * computeAbsorbability + * Determine if pattern supports context absorption optimization. + * + * Context absorption is safe when later matches are guaranteed to be + * suffixes of earlier matches (both row positions AND content). + * + * Sets RPR_ELEM_ABSORBABLE flag on the first element of each absorbable + * branch. A branch is absorbable if: + * - It starts with an unbounded element (greedy-first) + * - It has exactly one unbounded element + * - It's not inside an unbounded GROUP (GREEDY(ALT) case) + * + * pattern->isAbsorbable is set true if ANY branch is absorbable. + */ +static void +computeAbsorbability(RPRPattern *pattern) +{ + int i; + int maxAltDepth = -1; + RPRDepth minUnboundedGroupDepth = UINT8_MAX; + bool hasTopLevelAlt = false; + + pattern->isAbsorbable = false; + + if (pattern->numElements < 2) /* At minimum: one element + FIN */ + return; + + /* + * Scan elements to find ALT structure and unbounded GROUP depth. + */ + for (i = 0; i < pattern->numElements - 1; i++) + { + RPRPatternElement *elem = &pattern->elements[i]; + + if (elem->varId == RPR_VARID_ALT) + { + if (elem->depth > maxAltDepth) + maxAltDepth = elem->depth; + if (i == 0) + hasTopLevelAlt = true; + } + else if (elem->varId == RPR_VARID_END) + { + if (elem->max == INT32_MAX && elem->depth < minUnboundedGroupDepth) + minUnboundedGroupDepth = elem->depth; + } + } + + /* + * Check for GREEDY(ALT) pattern: ALT inside unbounded GROUP. + * In this case, no branch can be absorbable. + */ + if (maxAltDepth >= 0 && maxAltDepth > minUnboundedGroupDepth) + return; + + /* + * For top-level ALT patterns, check each branch separately. + * Set ABSORBABLE flag on branches that qualify. + */ + if (hasTopLevelAlt) + { + int branchStart = 1; /* after ALT marker */ + int patternEnd = pattern->numElements - 1; + + while (branchStart < patternEnd) + { + RPRPatternElement *branchFirst = &pattern->elements[branchStart]; + int branchEnd; + int branchUnbounded = 0; + int j; + bool branchAbsorbable = true; + + /* Find branch end */ + if (branchFirst->jump != RPR_ELEMIDX_INVALID) + branchEnd = branchFirst->jump; + else + branchEnd = patternEnd; + + /* First element of branch must be unbounded */ + if (branchFirst->max != INT32_MAX) + branchAbsorbable = false; + + /* Count unbounded in this branch - must be exactly 1 */ + if (branchAbsorbable) + { + for (j = branchStart; j < branchEnd; j++) + { + RPRPatternElement *elem = &pattern->elements[j]; + + if (elem->varId == RPR_VARID_END) + { + if (elem->max == INT32_MAX) + branchUnbounded++; + } + else if (elem->varId != RPR_VARID_ALT) + { + if (elem->max == INT32_MAX) + branchUnbounded++; + } + } + + if (branchUnbounded != 1) + branchAbsorbable = false; + } + + /* Set flag on branch's first element if absorbable */ + if (branchAbsorbable) + { + branchFirst->flags |= RPR_ELEM_ABSORBABLE; + pattern->isAbsorbable = true; + } + + branchStart = branchEnd; + } + return; + } + + /* + * Non-ALT pattern: check for absorbability. + * + * Case 1: First element is unbounded (A+ B, etc.) + * Case 2: Top-level unbounded GROUP (A B){2,} - GROUP END at depth 0 + * + * In both cases, must have exactly one unbounded element/group. + */ + { + RPRPatternElement *first = &pattern->elements[0]; + int numUnbounded = 0; + bool hasTopLevelUnboundedGroup = false; + RPRElemIdx topLevelGroupEnd = RPR_ELEMIDX_INVALID; + + /* Count unbounded elements and find top-level unbounded GROUP */ + for (i = 0; i < pattern->numElements - 1; i++) + { + RPRPatternElement *elem = &pattern->elements[i]; + + if (elem->varId == RPR_VARID_END) + { + if (elem->max == INT32_MAX) + { + numUnbounded++; + /* Check if this is a top-level GROUP (depth 0) */ + if (elem->depth == 0) + { + hasTopLevelUnboundedGroup = true; + topLevelGroupEnd = i; + } + } + } + else if (elem->varId != RPR_VARID_ALT) + { + if (elem->max == INT32_MAX) + numUnbounded++; + } + } + + /* Must have exactly one unbounded element/group */ + if (numUnbounded != 1) + return; + + /* + * Case 1: First element is unbounded (e.g., A+ B) + */ + if (first->max == INT32_MAX) + { + first->flags |= RPR_ELEM_ABSORBABLE; + pattern->isAbsorbable = true; + return; + } + + /* + * Case 2: Top-level unbounded GROUP that spans entire pattern. + * e.g., (A B){2,} where GROUP END is at the last position before FIN. + * The entire pattern content is inside this unbounded group. + */ + if (hasTopLevelUnboundedGroup && + topLevelGroupEnd == pattern->numElements - 2) + { + first->flags |= RPR_ELEM_ABSORBABLE; + pattern->isAbsorbable = true; + } + } +} diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c index 16d200cfb46..2efeec22102 100644 --- a/src/backend/optimizer/plan/setrefs.c +++ b/src/backend/optimizer/plan/setrefs.c @@ -211,7 +211,6 @@ static List *set_windowagg_runcondition_references(PlannerInfo *root, List *runcondition, Plan *plan); - /***************************************************************************** * * SUBPLAN REFERENCES @@ -2576,6 +2575,32 @@ set_upper_references(PlannerInfo *root, Plan *plan, int rtoffset) NRM_EQUAL, NUM_EXEC_QUAL(plan)); + /* + * Modifies an expression tree in each DEFINE clause so that all Var + * nodes's varno refers to OUTER_VAR. + */ + if (IsA(plan, WindowAgg)) + { + WindowAgg *wplan = (WindowAgg *) plan; + + if (wplan->defineClause != NIL) + { + foreach(l, wplan->defineClause) + { + TargetEntry *tle = (TargetEntry *) lfirst(l); + + tle->expr = (Expr *) + fix_upper_expr(root, + (Node *) tle->expr, + subplan_itlist, + OUTER_VAR, + rtoffset, + NRM_EQUAL, + NUM_EXEC_QUAL(plan)); + } + } + } + pfree(subplan_itlist); } diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c index c80bfc88d82..63d872b029d 100644 --- a/src/backend/optimizer/prep/prepjointree.c +++ b/src/backend/optimizer/prep/prepjointree.c @@ -2510,6 +2510,15 @@ perform_pullup_replace_vars(PlannerInfo *root, parse->returningList = (List *) pullup_replace_vars((Node *) parse->returningList, rvcontext); + foreach(lc, parse->windowClause) + { + WindowClause *wc = lfirst_node(WindowClause, lc); + + if (wc->defineClause != NIL) + wc->defineClause = (List *) + pullup_replace_vars((Node *) wc->defineClause, rvcontext); + } + if (parse->onConflict) { parse->onConflict->onConflictSet = (List *) diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h index 4bc6fb5670e..8d9097a8f4e 100644 --- a/src/include/nodes/plannodes.h +++ b/src/include/nodes/plannodes.h @@ -20,6 +20,7 @@ #include "lib/stringinfo.h" #include "nodes/bitmapset.h" #include "nodes/lockoptions.h" +#include "nodes/parsenodes.h" #include "nodes/primnodes.h" @@ -1223,6 +1224,69 @@ typedef struct Agg List *chain; } Agg; +/* ---------------- + * Row Pattern Recognition compiled pattern types + * ---------------- + */ + +/* Type definitions for RPR pattern elements */ +typedef uint8 RPRVarId; /* pattern variable ID */ +typedef uint8 RPRElemFlags; /* element flags */ +typedef uint8 RPRDepth; /* group nesting depth */ +typedef int32 RPRQuantity; /* quantifier min/max */ +typedef int16 RPRElemIdx; /* element array index */ + +/* + * RPRPatternElement - flat element for NFA pattern matching (16 bytes) + * + * Layout optimized for alignment (no padding holes): + * varId(1) + depth(1) + flags(1) + reserved(1) + min(4) + max(4) + next(2) + jump(2) + */ +typedef struct RPRPatternElement +{ + RPRVarId varId; /* variable ID, or special value for control */ + RPRDepth depth; /* group nesting depth */ + RPRElemFlags flags; /* flags (reluctant, etc.) */ + uint8 reserved; /* reserved padding byte */ + RPRQuantity min; /* quantifier minimum */ + RPRQuantity max; /* quantifier maximum */ + RPRElemIdx next; /* next element index */ + RPRElemIdx jump; /* jump target (for ALT/GROUP) */ +} RPRPatternElement; + +/* + * RPRPattern - compiled pattern for NFA execution + * + * Requires custom copy/out/read functions due to elements array. + */ +typedef struct RPRPattern +{ + pg_node_attr(custom_copy_equal, custom_read_write) + + NodeTag type; /* T_RPRPattern */ + int numVars; /* number of pattern variables */ + char **varNames; /* array of variable names (DEFINE order first) */ + RPRDepth maxDepth; /* maximum group nesting depth */ + int numElements; /* number of elements */ + RPRPatternElement *elements; /* array of pattern elements */ + + /* + * Context absorption optimization. + * + * Absorption is only safe when later matches are guaranteed to be + * suffixes of earlier matches. This requires simple pattern structure: + * + * Case 1: No ALT, single unbounded element (A+, (A B)+) + * Case 2: Top-level ALT with each branch being single unbounded (A+ | B+) + * + * Complex patterns like A B (A B)+ could theoretically be transformed to + * (A B){2,} for absorption, but this changes lexical order and is not + * implemented. Similarly, (A|B)+ cannot be absorbed because different + * start positions produce different match contents (not suffix relation). + */ + bool isAbsorbable; /* true if pattern supports context absorption */ +} RPRPattern; + /* ---------------- * window aggregate node * ---------------- @@ -1293,6 +1357,18 @@ typedef struct WindowAgg /* nulls sort first for in_range tests? */ bool inRangeNullsFirst; + /* Row Pattern Recognition AFTER MATCH SKIP clause */ + RPSkipTo rpSkipTo; /* Row Pattern Skip To type */ + + /* Compiled Row Pattern for NFA execution */ + struct RPRPattern *rpPattern; + + /* Row Pattern DEFINE clause (list of TargetEntry) */ + List *defineClause; + + /* Row Pattern DEFINE variable initial names (list of String) */ + List *defineInitial; + /* * false for all apart from the WindowAgg that's closest to the root of * the plan diff --git a/src/include/optimizer/rpr.h b/src/include/optimizer/rpr.h new file mode 100644 index 00000000000..95324e7a343 --- /dev/null +++ b/src/include/optimizer/rpr.h @@ -0,0 +1,46 @@ +/*------------------------------------------------------------------------- + * + * rpr.h + * Row Pattern Recognition pattern compilation for planner + * + * Portions Copyright (c) 1996-2025, PostgreSQL Global Development Group + * Portions Copyright (c) 1994, Regents of the University of California + * + * src/include/optimizer/rpr.h + * + *------------------------------------------------------------------------- + */ +#ifndef OPTIMIZER_RPR_H +#define OPTIMIZER_RPR_H + +#include "nodes/parsenodes.h" +#include "nodes/plannodes.h" + +/* Limits and special values */ +#define RPR_VARID_MAX 252 /* max pattern variables: 252 */ +#define RPR_DEPTH_MAX UINT8_MAX /* max nesting depth: 255 */ +#define RPR_QUANTITY_INF INT32_MAX /* unbounded quantifier */ +#define RPR_ELEMIDX_MAX INT16_MAX /* max pattern elements */ +#define RPR_ELEMIDX_INVALID ((RPRElemIdx) -1) /* invalid index */ + +/* Special varId values for control elements (253-255) */ +#define RPR_VARID_ALT ((RPRVarId) 253) /* alternation start */ +#define RPR_VARID_END ((RPRVarId) 254) /* group end */ +#define RPR_VARID_FIN ((RPRVarId) 255) /* pattern finish */ + +/* Element flags */ +#define RPR_ELEM_RELUCTANT 0x01 /* reluctant (non-greedy) quantifier */ +#define RPR_ELEM_ABSORBABLE 0x02 /* branch supports context absorption */ + +/* Accessor macros for RPRPatternElement */ +#define RPRElemIsReluctant(e) ((e)->flags & RPR_ELEM_RELUCTANT) +#define RPRElemIsAbsorbable(e) ((e)->flags & RPR_ELEM_ABSORBABLE) +#define RPRElemIsVar(e) ((e)->varId <= RPR_VARID_MAX) +#define RPRElemIsAlt(e) ((e)->varId == RPR_VARID_ALT) +#define RPRElemIsEnd(e) ((e)->varId == RPR_VARID_END) +#define RPRElemIsFin(e) ((e)->varId == RPR_VARID_FIN) +#define RPRElemCanSkip(e) ((e)->min == 0) + +extern RPRPattern *buildRPRPattern(RPRPatternNode *pattern, List *defineVariableList); + +#endif /* OPTIMIZER_RPR_H */ -- 2.43.0 [application/octet-stream] v38-0005-Row-pattern-recognition-patch-executor-and-comma.patch (65.5K, ../../[email protected]/6-v38-0005-Row-pattern-recognition-patch-executor-and-comma.patch) download | inline diff: From 19dfddeb03bf6d824ecbc94c87462990581af00d Mon Sep 17 00:00:00 2001 From: Tatsuo Ishii <[email protected]> Date: Thu, 15 Jan 2026 13:26:44 +0900 Subject: [PATCH v38 5/8] Row pattern recognition patch (executor and commands). --- src/backend/commands/explain.c | 143 +++ src/backend/executor/nodeWindowAgg.c | 1779 +++++++++++++++++++++++++- src/backend/utils/adt/windowfuncs.c | 34 +- src/include/catalog/pg_proc.dat | 6 + src/include/nodes/execnodes.h | 64 + 5 files changed, 2015 insertions(+), 11 deletions(-) diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c index b7bb111688c..969c9195864 100644 --- a/src/backend/commands/explain.c +++ b/src/backend/commands/explain.c @@ -29,6 +29,7 @@ #include "nodes/extensible.h" #include "nodes/makefuncs.h" #include "nodes/nodeFuncs.h" +#include "optimizer/rpr.h" #include "parser/analyze.h" #include "parser/parsetree.h" #include "rewrite/rewriteHandler.h" @@ -117,6 +118,8 @@ static void show_window_def(WindowAggState *planstate, static void show_window_keys(StringInfo buf, PlanState *planstate, int nkeys, AttrNumber *keycols, List *ancestors, ExplainState *es); +static void append_rpr_quantifier(StringInfo buf, RPRPatternElement *elem); +static char *deparse_rpr_pattern(RPRPattern *pattern); static void show_storage_info(char *maxStorageType, int64 maxSpaceUsed, ExplainState *es); static void show_tablesample(TableSampleClause *tsc, PlanState *planstate, @@ -2889,6 +2892,134 @@ show_sortorder_options(StringInfo buf, Node *sortexpr, } } +/* + * Append quantifier suffix for a pattern element. + */ +static void +append_rpr_quantifier(StringInfo buf, RPRPatternElement *elem) +{ + if (elem->min == 1 && elem->max == 1) + return; /* no quantifier */ + else if (elem->min == 0 && elem->max == RPR_QUANTITY_INF) + appendStringInfoChar(buf, '*'); + else if (elem->min == 1 && elem->max == RPR_QUANTITY_INF) + appendStringInfoChar(buf, '+'); + else if (elem->min == 0 && elem->max == 1) + appendStringInfoChar(buf, '?'); + else if (elem->max == RPR_QUANTITY_INF) + appendStringInfo(buf, "{%d,}", elem->min); + else if (elem->min == elem->max) + appendStringInfo(buf, "{%d}", elem->min); + else + appendStringInfo(buf, "{%d,%d}", elem->min, elem->max); + + if (RPRElemIsReluctant(elem)) + appendStringInfoChar(buf, '?'); +} + +/* + * Deparse a compiled RPRPattern (bytecode) back to pattern string. + * Simple approach: output parens for each depth level as-is. + */ +static char * +deparse_rpr_pattern(RPRPattern *pattern) +{ + StringInfoData buf; + int i; + RPRDepth prevDepth = 0; + bool needSpace = false; + RPRElemIdx altSepAt = RPR_ELEMIDX_INVALID; + + if (pattern == NULL || pattern->numElements == 0) + return NULL; + + initStringInfo(&buf); + + for (i = 0; i < pattern->numElements; i++) + { + RPRPatternElement *elem = &pattern->elements[i]; + + if (RPRElemIsFin(elem)) + break; + + /* Alternation separator */ + if (altSepAt == i) + { + appendStringInfoString(&buf, " | "); + needSpace = false; + altSepAt = RPR_ELEMIDX_INVALID; + } + + if (RPRElemIsAlt(elem)) + { + /* Open parens up to ALT's content depth */ + while (prevDepth <= elem->depth) + { + if (needSpace) + appendStringInfoChar(&buf, ' '); + appendStringInfoChar(&buf, '('); + prevDepth++; + needSpace = false; + } + continue; + } + + if (RPRElemIsEnd(elem)) + { + /* Close down to END's depth, output quantifier */ + while (prevDepth > elem->depth + 1) + { + appendStringInfoChar(&buf, ')'); + prevDepth--; + } + appendStringInfoChar(&buf, ')'); + append_rpr_quantifier(&buf, elem); + prevDepth = elem->depth; + needSpace = true; + continue; + } + + if (RPRElemIsVar(elem)) + { + /* Open parens for depth increase */ + while (prevDepth < elem->depth) + { + if (needSpace) + appendStringInfoChar(&buf, ' '); + appendStringInfoChar(&buf, '('); + prevDepth++; + needSpace = false; + } + + /* Close parens for depth decrease */ + while (prevDepth > elem->depth) + { + appendStringInfoChar(&buf, ')'); + prevDepth--; + } + + if (needSpace) + appendStringInfoChar(&buf, ' '); + + appendStringInfoString(&buf, pattern->varNames[elem->varId]); + append_rpr_quantifier(&buf, elem); + needSpace = true; + + if (elem->jump != RPR_ELEMIDX_INVALID) + altSepAt = elem->jump; + } + } + + /* Close remaining */ + while (prevDepth > 0) + { + appendStringInfoChar(&buf, ')'); + prevDepth--; + } + + return buf.data; +} + /* * Show the window definition for a WindowAgg node. */ @@ -2947,6 +3078,18 @@ show_window_def(WindowAggState *planstate, List *ancestors, ExplainState *es) appendStringInfoChar(&wbuf, ')'); ExplainPropertyText("Window", wbuf.data, es); pfree(wbuf.data); + + /* Show Row Pattern Recognition pattern if present */ + if (wagg->rpPattern != NULL) + { + char *patternStr = deparse_rpr_pattern(wagg->rpPattern); + + if (patternStr != NULL) + { + ExplainPropertyText("Pattern", patternStr, es); + pfree(patternStr); + } + } } /* diff --git a/src/backend/executor/nodeWindowAgg.c b/src/backend/executor/nodeWindowAgg.c index d9b64b0f465..cf43c1f6127 100644 --- a/src/backend/executor/nodeWindowAgg.c +++ b/src/backend/executor/nodeWindowAgg.c @@ -36,18 +36,23 @@ #include "access/htup_details.h" #include "catalog/objectaccess.h" #include "catalog/pg_aggregate.h" +#include "catalog/pg_collation_d.h" #include "catalog/pg_proc.h" #include "executor/executor.h" #include "executor/nodeWindowAgg.h" #include "miscadmin.h" #include "nodes/nodeFuncs.h" +#include "nodes/plannodes.h" #include "optimizer/clauses.h" #include "optimizer/optimizer.h" +#include "optimizer/rpr.h" #include "parser/parse_agg.h" #include "parser/parse_coerce.h" +#include "regex/regex.h" #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" +#include "utils/fmgroids.h" #include "utils/expandeddatum.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -170,6 +175,15 @@ typedef struct WindowStatePerAggData bool restart; /* need to restart this agg in this cycle? */ } WindowStatePerAggData; +/* + * Structure used by check_rpr_navigation() and rpr_navigation_walker(). + */ +typedef struct NavigationInfo +{ + bool is_prev; /* true if PREV */ + int num_vars; /* number of var nodes */ +} NavigationInfo; + static void initialize_windowaggregate(WindowAggState *winstate, WindowStatePerFunc perfuncstate, WindowStatePerAgg peraggstate); @@ -206,6 +220,9 @@ static Datum GetAggInitVal(Datum textInitVal, Oid transtype); static bool are_peers(WindowAggState *winstate, TupleTableSlot *slot1, TupleTableSlot *slot2); +static int WinGetSlotInFrame(WindowObject winobj, TupleTableSlot *slot, + int relpos, int seektype, bool set_mark, + bool *isnull, bool *isout); static bool window_gettupleslot(WindowObject winobj, int64 pos, TupleTableSlot *slot); @@ -224,6 +241,40 @@ static uint8 get_notnull_info(WindowObject winobj, int64 pos, int argno); static void put_notnull_info(WindowObject winobj, int64 pos, int argno, bool isnull); +static void attno_map(Node *node); +static bool attno_map_walker(Node *node, void *context); +static int row_is_in_reduced_frame(WindowObject winobj, int64 pos); +static bool rpr_is_defined(WindowAggState *winstate); + +static void create_reduced_frame_map(WindowAggState *winstate); +static int get_reduced_frame_map(WindowAggState *winstate, int64 pos); +static void register_reduced_frame_map(WindowAggState *winstate, int64 pos, + int val); +static void clear_reduced_frame_map(WindowAggState *winstate); +static void update_reduced_frame(WindowObject winobj, int64 pos); + +static void check_rpr_navigation(Node *node, bool is_prev); +static bool rpr_navigation_walker(Node *node, void *context); + +/* NFA-based pattern matching functions */ +static RPRNFAState *nfa_state_alloc(WindowAggState *winstate); +static void nfa_state_free(WindowAggState *winstate, RPRNFAState *state); +static void nfa_state_free_list(WindowAggState *winstate, RPRNFAState *list); +static RPRNFAState *nfa_state_clone(WindowAggState *winstate, int16 elemIdx, + int16 altPriority, int16 *counts, + RPRNFAState *list); +static bool nfa_evaluate_row(WindowObject winobj, int64 pos, bool *varMatched, bool *anyMatch); +static RPRNFAContext *nfa_context_alloc(WindowAggState *winstate); +static void nfa_unlink_context(WindowAggState *winstate, RPRNFAContext *ctx); +static void nfa_context_free(WindowAggState *winstate, RPRNFAContext *ctx); +static void nfa_start_context(WindowAggState *winstate, int64 startPos); +static void nfa_step_single(WindowAggState *winstate, RPRNFAContext *ctx, + RPRNFAState *state, bool *varMatched, int64 currentPos); +static void nfa_finalize_boundary(WindowAggState *winstate, RPRNFAContext *ctx, + int64 matchEndPos); +static RPRNFAContext *nfa_find_context_for_pos(WindowAggState *winstate, int64 pos); +static void nfa_remove_contexts_up_to(WindowAggState *winstate, int64 endPos); +static void nfa_absorb_contexts(WindowAggState *winstate, RPRNFAContext *excludeCtx, int64 currentPos); /* * Not null info bit array consists of 2-bit items @@ -817,6 +868,7 @@ eval_windowaggregates(WindowAggState *winstate) * transition function, or * - we have an EXCLUSION clause, or * - if the new frame doesn't overlap the old one + * - if RPR is enabled * * Note that we don't strictly need to restart in the last case, but if * we're going to remove all rows from the aggregation anyway, a restart @@ -831,7 +883,8 @@ eval_windowaggregates(WindowAggState *winstate) (winstate->aggregatedbase != winstate->frameheadpos && !OidIsValid(peraggstate->invtransfn_oid)) || (winstate->frameOptions & FRAMEOPTION_EXCLUSION) || - winstate->aggregatedupto <= winstate->frameheadpos) + winstate->aggregatedupto <= winstate->frameheadpos || + rpr_is_defined(winstate)) { peraggstate->restart = true; numaggs_restart++; @@ -905,7 +958,22 @@ eval_windowaggregates(WindowAggState *winstate) * head, so that tuplestore can discard unnecessary rows. */ if (agg_winobj->markptr >= 0) - WinSetMarkPosition(agg_winobj, winstate->frameheadpos); + { + int64 markpos = winstate->frameheadpos; + + if (rpr_is_defined(winstate)) + { + /* + * If RPR is used, it is possible PREV wants to look at the + * previous row. So the mark pos should be frameheadpos - 1 + * unless it is below 0. + */ + markpos -= 1; + if (markpos < 0) + markpos = 0; + } + WinSetMarkPosition(agg_winobj, markpos); + } /* * Now restart the aggregates that require it. @@ -960,6 +1028,14 @@ eval_windowaggregates(WindowAggState *winstate) { winstate->aggregatedupto = winstate->frameheadpos; ExecClearTuple(agg_row_slot); + + /* + * If RPR is defined, we do not use aggregatedupto_nonrestarted. To + * avoid assertion failure below, we reset aggregatedupto_nonrestarted + * to frameheadpos. + */ + if (rpr_is_defined(winstate)) + aggregatedupto_nonrestarted = winstate->frameheadpos; } /* @@ -973,6 +1049,12 @@ eval_windowaggregates(WindowAggState *winstate) { int ret; +#ifdef RPR_DEBUG + printf("===== loop in frame starts: aggregatedupto: " INT64_FORMAT " aggregatedbase: " INT64_FORMAT "\n", + winstate->aggregatedupto, + winstate->aggregatedbase); +#endif + /* Fetch next row if we didn't already */ if (TupIsNull(agg_row_slot)) { @@ -989,9 +1071,53 @@ eval_windowaggregates(WindowAggState *winstate) agg_row_slot, false); if (ret < 0) break; + if (ret == 0) goto next_tuple; + if (rpr_is_defined(winstate)) + { +#ifdef RPR_DEBUG + printf("reduced_frame_map: %d aggregatedupto: " INT64_FORMAT " aggregatedbase: " INT64_FORMAT "\n", + get_reduced_frame_map(winstate, + winstate->aggregatedupto), + winstate->aggregatedupto, + winstate->aggregatedbase); +#endif + + /* + * If the row status at currentpos is already decided and current + * row status is not decided yet, it means we passed the last + * reduced frame. Time to break the loop. + */ + if (get_reduced_frame_map(winstate, winstate->currentpos) + != RF_NOT_DETERMINED && + get_reduced_frame_map(winstate, winstate->aggregatedupto) + == RF_NOT_DETERMINED) + break; + + /* + * Otherwise we need to calculate the reduced frame. + */ + ret = row_is_in_reduced_frame(winstate->agg_winobj, + winstate->aggregatedupto); + if (ret == -1) /* unmatched row */ + break; + + /* + * Check if current row needs to be skipped due to no match. + */ + if (get_reduced_frame_map(winstate, + winstate->aggregatedupto) == RF_SKIPPED && + winstate->aggregatedupto == winstate->aggregatedbase) + { +#ifdef RPR_DEBUG + printf("skip current row for aggregation\n"); +#endif + break; + } + } + /* Set tuple context for evaluation of aggregate arguments */ winstate->tmpcontext->ecxt_outertuple = agg_row_slot; @@ -1020,6 +1146,7 @@ next_tuple: ExecClearTuple(agg_row_slot); } + /* The frame's end is not supposed to move backwards, ever */ Assert(aggregatedupto_nonrestarted <= winstate->aggregatedupto); @@ -1243,6 +1370,7 @@ begin_partition(WindowAggState *winstate) winstate->framehead_valid = false; winstate->frametail_valid = false; winstate->grouptail_valid = false; + create_reduced_frame_map(winstate); winstate->spooled_rows = 0; winstate->currentpos = 0; winstate->frameheadpos = 0; @@ -1464,6 +1592,13 @@ release_partition(WindowAggState *winstate) tuplestore_clear(winstate->buffer); winstate->partition_spooled = false; winstate->next_partition = true; + + /* Reset NFA state for new partition */ + winstate->nfaContext = NULL; + winstate->nfaContextTail = NULL; + winstate->nfaContextFree = NULL; + winstate->nfaStateFree = NULL; + winstate->nfaLastProcessedRow = -1; } /* @@ -2237,6 +2372,11 @@ ExecWindowAgg(PlanState *pstate) CHECK_FOR_INTERRUPTS(); +#ifdef RPR_DEBUG + printf("ExecWindowAgg called. pos: " INT64_FORMAT "\n", + winstate->currentpos); +#endif + if (winstate->status == WINDOWAGG_DONE) return NULL; @@ -2345,6 +2485,17 @@ ExecWindowAgg(PlanState *pstate) /* don't evaluate the window functions when we're in pass-through mode */ if (winstate->status == WINDOWAGG_RUN) { + /* + * If RPR is defined and skip mode is next row, we need to clear + * existing reduced frame info so that we newly calculate the info + * starting from current row. + */ + if (rpr_is_defined(winstate)) + { + if (winstate->rpSkipTo == ST_NEXT_ROW) + clear_reduced_frame_map(winstate); + } + /* * Evaluate true window functions */ @@ -2511,6 +2662,9 @@ ExecInitWindowAgg(WindowAgg *node, EState *estate, int eflags) TupleDesc scanDesc; ListCell *l; + TargetEntry *te; + Expr *expr; + /* check for unsupported flags */ Assert(!(eflags & (EXEC_FLAG_BACKWARD | EXEC_FLAG_MARK))); @@ -2609,6 +2763,16 @@ ExecInitWindowAgg(WindowAgg *node, EState *estate, int eflags) winstate->temp_slot_2 = ExecInitExtraTupleSlot(estate, scanDesc, &TTSOpsMinimalTuple); + winstate->prev_slot = ExecInitExtraTupleSlot(estate, scanDesc, + &TTSOpsMinimalTuple); + + winstate->next_slot = ExecInitExtraTupleSlot(estate, scanDesc, + &TTSOpsMinimalTuple); + + winstate->null_slot = ExecInitExtraTupleSlot(estate, scanDesc, + &TTSOpsMinimalTuple); + winstate->null_slot = ExecStoreAllNullTuple(winstate->null_slot); + /* * create frame head and tail slots only if needed (must create slots in * exactly the same cases that update_frameheadpos and update_frametailpos @@ -2795,6 +2959,66 @@ ExecInitWindowAgg(WindowAgg *node, EState *estate, int eflags) winstate->inRangeAsc = node->inRangeAsc; winstate->inRangeNullsFirst = node->inRangeNullsFirst; + /* Set up SKIP TO type */ + winstate->rpSkipTo = node->rpSkipTo; + /* Set up row pattern recognition PATTERN clause (compiled NFA) */ + winstate->rpPattern = node->rpPattern; + + /* Calculate NFA state size for allocation */ + if (node->rpPattern != NULL) + { + winstate->nfaStateSize = offsetof(RPRNFAState, counts) + + sizeof(int16) * node->rpPattern->maxDepth; + } + + /* Set up row pattern recognition DEFINE clause */ + winstate->defineInitial = node->defineInitial; + winstate->defineVariableList = NIL; + winstate->defineClauseList = NIL; + if (node->defineClause != NIL) + { + /* + * Tweak arg var of PREV/NEXT so that it refers to scan/inner slot. + */ + foreach(l, node->defineClause) + { + char *name; + ExprState *exps; + + te = lfirst(l); + name = te->resname; + expr = te->expr; + +#ifdef RPR_DEBUG + printf("defineVariable name: %s\n", name); +#endif + winstate->defineVariableList = + lappend(winstate->defineVariableList, + makeString(pstrdup(name))); + attno_map((Node *) expr); + exps = ExecInitExpr(expr, (PlanState *) winstate); + winstate->defineClauseList = + lappend(winstate->defineClauseList, exps); + } + } + + /* Initialize NFA free lists for row pattern matching */ + winstate->nfaContext = NULL; + winstate->nfaContextTail = NULL; + winstate->nfaContextFree = NULL; + winstate->nfaStateFree = NULL; + winstate->nfaLastProcessedRow = -1; + + /* + * Allocate varMatched array for NFA evaluation. + * With the new varNames ordering (DEFINE order first), varId == defineIdx + * for all defined variables, so no mapping is needed. + */ + if (list_length(winstate->defineVariableList) > 0) + winstate->nfaVarMatched = palloc0(sizeof(bool) * + list_length(winstate->defineVariableList)); + else + winstate->nfaVarMatched = NULL; winstate->all_first = true; winstate->partition_spooled = false; winstate->more_partitions = false; @@ -2803,6 +3027,111 @@ ExecInitWindowAgg(WindowAgg *node, EState *estate, int eflags) return winstate; } +/* + * Rewrite varno of Var nodes that are the argument of PREV/NET so that they + * see scan tuple (PREV) or inner tuple (NEXT). Also we check the arguments + * of PREV/NEXT include at least 1 column reference. This is required by the + * SQL standard. + */ +static void +attno_map(Node *node) +{ + (void) expression_tree_walker(node, attno_map_walker, NULL); +} + +static bool +attno_map_walker(Node *node, void *context) +{ + FuncExpr *func; + int nargs; + bool is_prev; + + if (node == NULL) + return false; + + if (IsA(node, FuncExpr)) + { + func = (FuncExpr *) node; + + if (func->funcid == F_PREV || func->funcid == F_NEXT) + { + /* + * The SQL standard allows to have two more arguments form of + * PREV/NEXT. But currently we allow only 1 argument form. + */ + nargs = list_length(func->args); + if (list_length(func->args) != 1) + elog(ERROR, "PREV/NEXT must have 1 argument but function %d has %d args", + func->funcid, nargs); + + /* + * Check expr of PREV/NEXT aruguments and replace varno. + */ + is_prev = (func->funcid == F_PREV) ? true : false; + check_rpr_navigation(node, is_prev); + } + } + return expression_tree_walker(node, attno_map_walker, NULL); +} + +/* + * Rewrite varno of Var of RPR navigation operations (PREV/NEXT). + * If is_prev is true, we take care PREV, otherwise NEXT. + */ +static void +check_rpr_navigation(Node *node, bool is_prev) +{ + NavigationInfo context; + + context.is_prev = is_prev; + context.num_vars = 0; + (void) expression_tree_walker(node, rpr_navigation_walker, &context); + if (context.num_vars < 1) + ereport(ERROR, + errmsg("row pattern navigation operation's argument must include at least one column reference")); +} + +static bool +rpr_navigation_walker(Node *node, void *context) +{ + NavigationInfo *nav = (NavigationInfo *) context; + + if (node == NULL) + return false; + + switch (nodeTag(node)) + { + case T_Var: + { + Var *var = (Var *) node; + + nav->num_vars++; + + if (nav->is_prev) + { + /* + * Rewrite varno from OUTER_VAR to regular var no so that + * the var references scan tuple. + */ + var->varno = var->varnosyn; + } + else + var->varno = INNER_VAR; + } + break; + case T_Const: + case T_FuncExpr: + case T_OpExpr: + break; + + default: + ereport(ERROR, + errmsg("row pattern navigation operation's argument includes unsupported expression")); + } + return expression_tree_walker(node, rpr_navigation_walker, context); +} + + /* ----------------- * ExecEndWindowAgg * ----------------- @@ -2860,6 +3189,8 @@ ExecReScanWindowAgg(WindowAggState *node) ExecClearTuple(node->agg_row_slot); ExecClearTuple(node->temp_slot_1); ExecClearTuple(node->temp_slot_2); + ExecClearTuple(node->prev_slot); + ExecClearTuple(node->next_slot); if (node->framehead_slot) ExecClearTuple(node->framehead_slot); if (node->frametail_slot) @@ -3220,7 +3551,8 @@ window_gettupleslot(WindowObject winobj, int64 pos, TupleTableSlot *slot) return false; if (pos < winobj->markpos) - elog(ERROR, "cannot fetch row before WindowObject's mark position"); + elog(ERROR, "cannot fetch row: " INT64_FORMAT " before WindowObject's mark position: " INT64_FORMAT, + pos, winobj->markpos); oldcontext = MemoryContextSwitchTo(winstate->ss.ps.ps_ExprContext->ecxt_per_query_memory); @@ -3922,8 +4254,6 @@ WinGetFuncArgInFrame(WindowObject winobj, int argno, WindowAggState *winstate; ExprContext *econtext; TupleTableSlot *slot; - int64 abs_pos; - int64 mark_pos; Assert(WindowObjectIsValid(winobj)); winstate = winobj->winstate; @@ -3934,6 +4264,48 @@ WinGetFuncArgInFrame(WindowObject winobj, int argno, return ignorenulls_getfuncarginframe(winobj, argno, relpos, seektype, set_mark, isnull, isout); + if (WinGetSlotInFrame(winobj, slot, + relpos, seektype, set_mark, + isnull, isout) == 0) + { + econtext->ecxt_outertuple = slot; + return ExecEvalExpr((ExprState *) list_nth(winobj->argstates, argno), + econtext, isnull); + } + + if (isout) + *isout = true; + *isnull = true; + return (Datum) 0; +} + +/* + * WinGetSlotInFrame + * slot: TupleTableSlot to store the result + * relpos: signed rowcount offset from the seek position + * seektype: WINDOW_SEEK_HEAD or WINDOW_SEEK_TAIL + * set_mark: If the row is found/in frame and set_mark is true, the mark is + * moved to the row as a side-effect. + * isnull: output argument, receives isnull status of result + * isout: output argument, set to indicate whether target row position + * is out of frame (can pass NULL if caller doesn't care about this) + * + * Returns 0 if we successfullt got the slot. false if out of frame. + * (also isout is set) + */ +static int +WinGetSlotInFrame(WindowObject winobj, TupleTableSlot *slot, + int relpos, int seektype, bool set_mark, + bool *isnull, bool *isout) +{ + WindowAggState *winstate; + int64 abs_pos; + int64 mark_pos; + int num_reduced_frame; + + Assert(WindowObjectIsValid(winobj)); + winstate = winobj->winstate; + switch (seektype) { case WINDOW_SEEK_CURRENT: @@ -4000,11 +4372,25 @@ WinGetFuncArgInFrame(WindowObject winobj, int argno, winstate->frameOptions); break; } + num_reduced_frame = row_is_in_reduced_frame(winobj, + winstate->frameheadpos); + if (num_reduced_frame < 0) + goto out_of_frame; + else if (num_reduced_frame > 0) + if (relpos >= num_reduced_frame) + goto out_of_frame; break; case WINDOW_SEEK_TAIL: /* rejecting relpos > 0 is easy and simplifies code below */ if (relpos > 0) goto out_of_frame; + + /* + * RPR cares about frame head pos. Need to call + * update_frameheadpos + */ + update_frameheadpos(winstate); + update_frametailpos(winstate); abs_pos = winstate->frametailpos - 1 + relpos; @@ -4071,6 +4457,14 @@ WinGetFuncArgInFrame(WindowObject winobj, int argno, mark_pos = 0; /* keep compiler quiet */ break; } + + num_reduced_frame = row_is_in_reduced_frame(winobj, + winstate->frameheadpos + relpos); + if (num_reduced_frame < 0) + goto out_of_frame; + else if (num_reduced_frame > 0) + abs_pos = winstate->frameheadpos + relpos + + num_reduced_frame - 1; break; default: elog(ERROR, "unrecognized window seek type: %d", seektype); @@ -4089,15 +4483,13 @@ WinGetFuncArgInFrame(WindowObject winobj, int argno, *isout = false; if (set_mark) WinSetMarkPosition(winobj, mark_pos); - econtext->ecxt_outertuple = slot; - return ExecEvalExpr((ExprState *) list_nth(winobj->argstates, argno), - econtext, isnull); + return 0; out_of_frame: if (isout) *isout = true; *isnull = true; - return (Datum) 0; + return -1; } /* @@ -4128,3 +4520,1372 @@ WinGetFuncArgCurrent(WindowObject winobj, int argno, bool *isnull) return ExecEvalExpr((ExprState *) list_nth(winobj->argstates, argno), econtext, isnull); } + +/* + * rpr_is_defined + * return true if Row pattern recognition is defined. + */ +static bool +rpr_is_defined(WindowAggState *winstate) +{ + return winstate->rpPattern != NULL; +} + +/* + * ----------------- + * row_is_in_reduced_frame + * Determine whether a row is in the current row's reduced window frame + * according to row pattern matching + * + * The row must has been already determined that it is in a full window frame + * and fetched it into slot. + * + * Returns: + * = 0, RPR is not defined. + * >0, if the row is the first in the reduced frame. Return the number of rows + * in the reduced frame. + * -1, if the row is unmatched row + * -2, if the row is in the reduced frame but needed to be skipped because of + * AFTER MATCH SKIP PAST LAST ROW + * ----------------- + */ +static int +row_is_in_reduced_frame(WindowObject winobj, int64 pos) +{ + WindowAggState *winstate = winobj->winstate; + int state; + int rtn; + + if (!rpr_is_defined(winstate)) + { + /* + * RPR is not defined. Assume that we are always in the the reduced + * window frame. + */ + rtn = 0; +#ifdef RPR_DEBUG + printf("row_is_in_reduced_frame returns %d: pos: " INT64_FORMAT "\n", + rtn, pos); +#endif + return rtn; + } + + state = get_reduced_frame_map(winstate, pos); + + if (state == RF_NOT_DETERMINED) + { + update_frameheadpos(winstate); + update_reduced_frame(winobj, pos); + } + + state = get_reduced_frame_map(winstate, pos); + + switch (state) + { + int64 i; + int num_reduced_rows; + + case RF_FRAME_HEAD: + num_reduced_rows = 1; + for (i = pos + 1; + get_reduced_frame_map(winstate, i) == RF_SKIPPED; i++) + num_reduced_rows++; + rtn = num_reduced_rows; + break; + + case RF_SKIPPED: + rtn = -2; + break; + + case RF_UNMATCHED: + rtn = -1; + break; + + default: + elog(ERROR, "Unrecognized state: %d at: " INT64_FORMAT, + state, pos); + break; + } + +#ifdef RPR_DEBUG + printf("row_is_in_reduced_frame returns %d: pos: " INT64_FORMAT "\n", + rtn, pos); +#endif + return rtn; +} + +#define REDUCED_FRAME_MAP_INIT_SIZE 1024L + +/* + * create_reduced_frame_map + * Create reduced frame map + */ +static void +create_reduced_frame_map(WindowAggState *winstate) +{ + winstate->reduced_frame_map = + MemoryContextAlloc(winstate->partcontext, + REDUCED_FRAME_MAP_INIT_SIZE); + winstate->alloc_sz = REDUCED_FRAME_MAP_INIT_SIZE; + clear_reduced_frame_map(winstate); +} + +/* + * clear_reduced_frame_map + * Clear reduced frame map + */ +static void +clear_reduced_frame_map(WindowAggState *winstate) +{ + Assert(winstate->reduced_frame_map != NULL); + MemSet(winstate->reduced_frame_map, RF_NOT_DETERMINED, + winstate->alloc_sz); +} + +/* + * get_reduced_frame_map + * Get reduced frame map specified by pos + */ +static int +get_reduced_frame_map(WindowAggState *winstate, int64 pos) +{ + Assert(winstate->reduced_frame_map != NULL); + Assert(pos >= 0); + + /* + * If pos is not in the reduced frame map, it means that any info + * regarding the pos has not been registered yet. So we return + * RF_NOT_DETERMINED. + */ + if (pos >= winstate->alloc_sz) + return RF_NOT_DETERMINED; + + return winstate->reduced_frame_map[pos]; +} + +/* + * register_reduced_frame_map + * Add/replace reduced frame map member at pos. + * If there's no enough space, expand the map. + */ +static void +register_reduced_frame_map(WindowAggState *winstate, int64 pos, int val) +{ + int64 realloc_sz; + + Assert(winstate->reduced_frame_map != NULL); + + if (pos < 0) + elog(ERROR, "wrong pos: " INT64_FORMAT, pos); + + if (pos > winstate->alloc_sz - 1) + { + realloc_sz = winstate->alloc_sz * 2; + + winstate->reduced_frame_map = + repalloc(winstate->reduced_frame_map, realloc_sz); + + MemSet(winstate->reduced_frame_map + winstate->alloc_sz, + RF_NOT_DETERMINED, realloc_sz - winstate->alloc_sz); + + winstate->alloc_sz = realloc_sz; + } + + winstate->reduced_frame_map[pos] = val; +} + +/* + * update_reduced_frame + * Update reduced frame info using multi-context NFA pattern matching. + * + * Maintains multiple NFA contexts simultaneously, one for each potential + * match start position. This allows sharing row evaluations across contexts, + * avoiding redundant DEFINE clause evaluations when rewinding for SKIP TO + * NEXT ROW mode. + * + * Key optimizations: + * - Row evaluations (expensive DEFINE clauses) happen only once per row + * - All active contexts share the same evaluation results + * - Contexts persist across calls, enabling O(n) DEFINE evaluations + */ +static void +update_reduced_frame(WindowObject winobj, int64 pos) +{ + WindowAggState *winstate = winobj->winstate; + RPRNFAContext *targetCtx; + RPRNFAContext *firstCtx; + int64 matchLength = 0; + int64 currentPos; + int64 startPos; + int frameOptions = winstate->frameOptions; + bool hasLimitedFrame; + int64 frameOffset = 0; + + /* + * Check if we have a limited frame (ROWS ... N FOLLOWING). + * Each context needs its own frame end based on matchStartRow + offset. + */ + hasLimitedFrame = (frameOptions & FRAMEOPTION_ROWS) && + !(frameOptions & FRAMEOPTION_END_UNBOUNDED_FOLLOWING); + if (hasLimitedFrame && winstate->endOffsetValue != 0) + frameOffset = DatumGetInt64(winstate->endOffsetValue); + + /* + * Case 1: pos is before any existing context's start position. + * This means the position was already processed and determined unmatched. + * Note: contexts are added at head with increasing positions, so we need + * to find the minimum matchStartRow (the oldest context). + */ + { + int64 minStartRow = INT64_MAX; + for (firstCtx = winstate->nfaContext; firstCtx != NULL; firstCtx = firstCtx->next) + { + if (firstCtx->matchStartRow < minStartRow) + minStartRow = firstCtx->matchStartRow; + } + if (minStartRow != INT64_MAX && pos < minStartRow) + { + register_reduced_frame_map(winstate, pos, RF_UNMATCHED); + return; + } + } + + /* + * Case 2: Find existing context for this pos, or create new one. + */ + targetCtx = nfa_find_context_for_pos(winstate, pos); + if (targetCtx == NULL) + { + /* No context exists - create one and start fresh */ + nfa_start_context(winstate, pos); + targetCtx = winstate->nfaContext; + } + + /* + * Determine where to start processing. + * If we've already evaluated rows beyond pos, continue from there. + */ + startPos = Max(pos, winstate->nfaLastProcessedRow + 1); + + /* + * Process rows until target context completes or we hit boundaries. + * Each row evaluation is shared across all active contexts. + */ + for (currentPos = startPos; targetCtx->states != NULL; currentPos++) + { + bool rowExists; + bool anyMatch; + RPRNFAContext *ctx; + + /* Evaluate variables for this row - done only once, shared by all contexts */ + rowExists = nfa_evaluate_row(winobj, currentPos, winstate->nfaVarMatched, &anyMatch); + + /* No more rows in partition? Finalize all contexts */ + if (!rowExists) + { + for (ctx = winstate->nfaContext; ctx != NULL; ctx = ctx->next) + { + if (ctx->states != NULL) + nfa_finalize_boundary(winstate, ctx, currentPos - 1); + } + break; + } + + /* Update last processed row */ + winstate->nfaLastProcessedRow = currentPos; + + /* + * Process each active context with this row's evaluation results. + * Each context has its own frame boundary based on matchStartRow. + */ + for (ctx = winstate->nfaContext; ctx != NULL; ctx = ctx->next) + { + int64 ctxFrameEnd; + + /* Skip already-completed contexts */ + if (ctx->states == NULL) + continue; + + /* + * Calculate per-context frame end. + * For "ROWS BETWEEN CURRENT ROW AND N FOLLOWING", each context's + * frame end is matchStartRow + offset + 1 (exclusive). + */ + if (hasLimitedFrame) + { + ctxFrameEnd = ctx->matchStartRow + frameOffset + 1; + if (currentPos >= ctxFrameEnd) + { + nfa_finalize_boundary(winstate, ctx, ctxFrameEnd - 1); + continue; + } + } + + /* First row of this context must match at least one variable */ + if (currentPos == ctx->matchStartRow && !anyMatch) + { + /* Clear states to mark as unmatched */ + nfa_state_free_list(winstate, ctx->states); + ctx->states = NULL; + continue; + } + + /* Skip if this row is before context's start */ + if (currentPos < ctx->matchStartRow) + continue; + + /* Process states for this context */ + { + RPRNFAState *states = ctx->states; + RPRNFAState *state; + RPRNFAState *nextState; + + ctx->states = NULL; + + for (state = states; state != NULL; state = nextState) + { + nextState = state->next; + state->next = NULL; + nfa_step_single(winstate, ctx, state, winstate->nfaVarMatched, currentPos); + } + } + } + + /* + * Create a new context for the next potential start position. + * This enables overlapping match detection for SKIP TO NEXT ROW. + */ + if (anyMatch) + nfa_start_context(winstate, currentPos + 1); + + /* + * Absorb redundant contexts. + * At the same elementIndex, if newer context's count <= older context's count, + * the newer context can be absorbed (for unbounded quantifiers). + * Note: Never absorb targetCtx - it's the context we're trying to complete. + */ + nfa_absorb_contexts(winstate, targetCtx, currentPos); + + /* Check if target context is now complete */ + if (targetCtx->states == NULL) + break; + } + + /* + * Get match result from target context. + */ + if (targetCtx->matchEndRow >= pos) + matchLength = targetCtx->matchEndRow - pos + 1; + + /* + * Register reduced frame map based on match result. + */ + if (matchLength <= 0) + { + register_reduced_frame_map(winstate, pos, RF_UNMATCHED); + } + else + { + register_reduced_frame_map(winstate, pos, RF_FRAME_HEAD); + for (int64 i = pos + 1; i < pos + matchLength; i++) + { + register_reduced_frame_map(winstate, i, RF_SKIPPED); + } + } + + /* + * Cleanup contexts based on SKIP mode. + */ + if (matchLength > 0 && winstate->rpSkipTo == ST_PAST_LAST_ROW) + { + /* Remove all contexts with start <= matchEnd */ + nfa_remove_contexts_up_to(winstate, pos + matchLength - 1); + } + else + { + /* SKIP TO NEXT ROW or no match: just remove the target context */ + RPRNFAContext *ctx = winstate->nfaContext; + + while (ctx != NULL) + { + if (ctx == targetCtx) + { + nfa_unlink_context(winstate, ctx); + nfa_context_free(winstate, ctx); + break; + } + ctx = ctx->next; + } + } +} + +/* + * NFA-based pattern matching implementation + * + * These functions implement direct NFA execution using the compiled + * RPRPattern structure, avoiding regex compilation overhead. + */ + +/* + * nfa_state_alloc + * + * Allocate an NFA state, reusing from freeList if available. + * freeList is stored in WindowAggState for reuse across match attempts. + * Uses flexible array member for counts[]. + */ +static RPRNFAState * +nfa_state_alloc(WindowAggState *winstate) +{ + RPRNFAState *state; + int maxDepth = winstate->rpPattern->maxDepth; + + /* Try to reuse from free list first */ + if (winstate->nfaStateFree != NULL) + { + state = winstate->nfaStateFree; + winstate->nfaStateFree = state->next; + } + else + { + /* Allocate in partition context for proper lifetime */ + MemoryContext oldContext = MemoryContextSwitchTo(winstate->partcontext); + state = palloc(winstate->nfaStateSize); + MemoryContextSwitchTo(oldContext); + } + + /* initialize state - clear all depth counts */ + state->next = NULL; + state->elemIdx = 0; + state->altPriority = 0; + /* Initialize all depth counts to 0 using memset */ + if (maxDepth > 0) + memset(state->counts, 0, sizeof(int16) * maxDepth); + + return state; +} + +/* + * nfa_state_free + * + * Return a state to the free list for later reuse. + */ +static void +nfa_state_free(WindowAggState *winstate, RPRNFAState *state) +{ + state->next = winstate->nfaStateFree; + winstate->nfaStateFree = state; +} + +/* + * nfa_state_free_list + * + * Free all states in a list using pfree. + */ +static void +nfa_state_free_list(WindowAggState *winstate, RPRNFAState *list) +{ + RPRNFAState *state; + + while(list != NULL) + { + state = list; + list = list->next; + nfa_state_free(winstate, state); + } +} + +/* + * nfa_states_equal + * + * Check if two states are equivalent (same elemIdx and counts). + */ +static bool +nfa_states_equal(WindowAggState *winstate, RPRNFAState *s1, RPRNFAState *s2) +{ + int maxDepth = winstate->rpPattern->maxDepth; + + if (s1->elemIdx != s2->elemIdx) + return false; + + if (maxDepth > 0 && + memcmp(s1->counts, s2->counts, sizeof(int16) * maxDepth) != 0) + return false; + + return true; +} + +/* + * nfa_add_state_unique + * + * Add a state to ctx->states at the END, only if no duplicate exists. + * Returns true if state was added, false if duplicate found (state is freed). + */ +static bool +nfa_add_state_unique(WindowAggState *winstate, RPRNFAContext *ctx, RPRNFAState *state) +{ + RPRNFAState *s; + RPRNFAState *prev = NULL; + RPRNFAState *tail = NULL; + + /* Check for duplicate and find tail */ + for (s = ctx->states; s != NULL; s = s->next) + { + if (nfa_states_equal(winstate, s, state)) + { + /* + * Duplicate found - keep lower altPriority for lexical order. + * Lower altPriority means earlier alternative in pattern. + */ + if (state->altPriority < s->altPriority) + { + /* New state has better priority, replace existing */ + state->next = s->next; + if (prev == NULL) + ctx->states = state; + else + prev->next = state; + nfa_state_free(winstate, s); + return true; + } + /* Existing state has better/equal priority, discard new */ + nfa_state_free(winstate, state); + return false; + } + prev = s; + tail = s; + } + + /* No duplicate, add at end */ + state->next = NULL; + if (tail == NULL) + ctx->states = state; + else + tail->next = state; + + return true; +} + +/* + * nfa_state_clone + * + * Clone a state with given elemIdx, altPriority and counts. + * Only copies counts up to elem->depth (not entire maxDepth). + * Prepends to the provided list and returns the new list head. + */ +static RPRNFAState * +nfa_state_clone(WindowAggState *winstate, int16 elemIdx, int16 altPriority, + int16 *counts, RPRNFAState *list) +{ + RPRPattern *pattern = winstate->rpPattern; + int maxDepth = pattern->maxDepth; + RPRNFAState *state = nfa_state_alloc(winstate); + + state->elemIdx = elemIdx; + state->altPriority = altPriority; + /* nfa_state_alloc already zeroed all counts, now copy all depth levels */ + if (counts != NULL && maxDepth > 0) + memcpy(state->counts, counts, sizeof(int16) * maxDepth); + state->next = list; + + return state; +} + +/* + * nfa_add_matched_state + * + * Record a matched state following SQL standard lexical order preference. + * Priority: lower altPriority wins (lexical order), then longer match. + */ +static void +nfa_add_matched_state(WindowAggState *winstate, RPRNFAContext *ctx, + RPRNFAState *state, int64 matchEndRow) +{ + bool shouldUpdate = false; + + if (ctx->matchedState == NULL) + { + /* No previous match, always save */ + shouldUpdate = true; + } + else if (state->altPriority < ctx->matchedState->altPriority) + { + /* New state has better lexical order priority */ + shouldUpdate = true; + } + else if (state->altPriority == ctx->matchedState->altPriority && + matchEndRow > ctx->matchEndRow) + { + /* Same priority, but longer match */ + shouldUpdate = true; + } + + if (shouldUpdate) + { + /* Reuse existing matchedState or allocate from free list */ + if (ctx->matchedState == NULL) + ctx->matchedState = nfa_state_alloc(winstate); + + /* Copy state data */ + memcpy(ctx->matchedState, state, winstate->nfaStateSize); + ctx->matchedState->next = NULL; + ctx->matchEndRow = matchEndRow; + } +} + +/* + * nfa_free_matched_state + * + * Return matchedState to free list for reuse. + */ +static void +nfa_free_matched_state(WindowAggState *winstate, RPRNFAState *state) +{ + if (state != NULL) + nfa_state_free(winstate, state); +} + +/* + * nfa_evaluate_row + * + * Evaluate all DEFINE variables for current row. + * Returns true if the row exists, false if out of partition. + * If row exists, fills varMatched array and sets *anyMatch if any variable matched. + * varMatched[i] = true if variable i matched at current row. + */ +static bool +nfa_evaluate_row(WindowObject winobj, int64 pos, bool *varMatched, bool *anyMatchOut) +{ + WindowAggState *winstate = winobj->winstate; + ExprContext *econtext = winstate->ss.ps.ps_ExprContext; + int numDefineVars = list_length(winstate->defineVariableList); + ListCell *lc; + int varIdx = 0; + bool anyMatch = false; + TupleTableSlot *slot; + + *anyMatchOut = false; + + /* + * Set up slots for current, previous, and next rows. + * We don't call get_slots() here to avoid recursion through + * row_is_in_frame -> update_reduced_frame -> nfa_match_pattern. + */ + + /* Current row -> ecxt_outertuple */ + slot = winstate->temp_slot_1; + if (!window_gettupleslot(winobj, pos, slot)) + return false; /* No row exists */ + econtext->ecxt_outertuple = slot; + + /* Previous row -> ecxt_scantuple (for PREV) */ + if (pos > 0) + { + slot = winstate->prev_slot; + if (!window_gettupleslot(winobj, pos - 1, slot)) + econtext->ecxt_scantuple = winstate->null_slot; + else + econtext->ecxt_scantuple = slot; + } + else + econtext->ecxt_scantuple = winstate->null_slot; + + /* Next row -> ecxt_innertuple (for NEXT) */ + slot = winstate->next_slot; + if (!window_gettupleslot(winobj, pos + 1, slot)) + econtext->ecxt_innertuple = winstate->null_slot; + else + econtext->ecxt_innertuple = slot; + + foreach(lc, winstate->defineClauseList) + { + ExprState *exprState = (ExprState *) lfirst(lc); + Datum result; + bool isnull; + + /* Evaluate DEFINE expression */ + result = ExecEvalExpr(exprState, econtext, &isnull); + + if (!isnull && DatumGetBool(result)) + { + varMatched[varIdx] = true; + anyMatch = true; + } + else + { + varMatched[varIdx] = false; + } + + varIdx++; + if (varIdx >= numDefineVars) + break; + } + + *anyMatchOut = anyMatch; + return true; /* Row exists */ +} + +/* + * nfa_context_alloc + * + * Allocate an NFA context from free list or palloc. + */ +static RPRNFAContext * +nfa_context_alloc(WindowAggState *winstate) +{ + RPRNFAContext *ctx; + + if (winstate->nfaContextFree != NULL) + { + ctx = winstate->nfaContextFree; + winstate->nfaContextFree = ctx->next; + } + else + { + /* Allocate in partition context for proper lifetime */ + MemoryContext oldContext = MemoryContextSwitchTo(winstate->partcontext); + ctx = palloc(sizeof(RPRNFAContext)); + MemoryContextSwitchTo(oldContext); + } + + ctx->next = NULL; + ctx->prev = NULL; + ctx->states = NULL; + ctx->matchStartRow = -1; + ctx->matchEndRow = -1; + ctx->matchedState = NULL; + + return ctx; +} + +/* + * nfa_unlink_context + * + * Remove a context from the doubly-linked active context list. + * Updates head (nfaContext) and tail (nfaContextTail) as needed. + */ +static void +nfa_unlink_context(WindowAggState *winstate, RPRNFAContext *ctx) +{ + if (ctx->prev != NULL) + ctx->prev->next = ctx->next; + else + winstate->nfaContext = ctx->next; /* was head */ + + if (ctx->next != NULL) + ctx->next->prev = ctx->prev; + else + winstate->nfaContextTail = ctx->prev; /* was tail */ + + ctx->next = NULL; + ctx->prev = NULL; +} + +/* + * nfa_context_free + * + * Return a context to free list. Also frees any states in the context. + * Note: Caller must unlink context from active list first using nfa_unlink_context. + */ +static void +nfa_context_free(WindowAggState *winstate, RPRNFAContext *ctx) +{ + if (ctx->states != NULL) + nfa_state_free_list(winstate, ctx->states); + if (ctx->matchedState != NULL) + nfa_free_matched_state(winstate, ctx->matchedState); + + ctx->states = NULL; + ctx->matchedState = NULL; + ctx->next = winstate->nfaContextFree; + winstate->nfaContextFree = ctx; +} + +/* + * nfa_start_context + * + * Start a new match context at given position. + * Adds context to winstate->nfaContext list. + */ +static void +nfa_start_context(WindowAggState *winstate, int64 startPos) +{ + RPRNFAContext *ctx; + + ctx = nfa_context_alloc(winstate); + ctx->matchStartRow = startPos; + ctx->states = nfa_state_alloc(winstate); /* initial state at elem 0 */ + + /* Add to head of active context list (doubly-linked) */ + ctx->next = winstate->nfaContext; + ctx->prev = NULL; + if (winstate->nfaContext != NULL) + winstate->nfaContext->prev = ctx; + else + winstate->nfaContextTail = ctx; /* first context becomes tail */ + winstate->nfaContext = ctx; +} + +/* + * nfa_find_context_for_pos + * + * Find a context with the given start position. + * Returns NULL if not found. + */ +static RPRNFAContext * +nfa_find_context_for_pos(WindowAggState *winstate, int64 pos) +{ + RPRNFAContext *ctx; + + for (ctx = winstate->nfaContext; ctx != NULL; ctx = ctx->next) + { + if (ctx->matchStartRow == pos) + return ctx; + } + return NULL; +} + +/* + * nfa_remove_contexts_up_to + * + * Remove all contexts with matchStartRow <= endPos. + * Used by SKIP PAST LAST ROW to discard contexts within matched frame. + */ +static void +nfa_remove_contexts_up_to(WindowAggState *winstate, int64 endPos) +{ + RPRNFAContext *ctx; + RPRNFAContext *next; + + ctx = winstate->nfaContext; + while (ctx != NULL) + { + next = ctx->next; + if (ctx->matchStartRow <= endPos) + { + /* Remove this context */ + nfa_unlink_context(winstate, ctx); + nfa_context_free(winstate, ctx); + } + ctx = next; + } +} + +/* + * nfa_absorb_contexts + * + * Absorb newer contexts into older ones when states are fully covered. + * For pattern like A+, if older context (started earlier) has count >= newer + * context's count at the same element, the newer context produces subset matches. + * + * Absorption condition: + * - For unbounded quantifiers (max=INT32_MAX): older.counts >= newer.counts + * - For bounded quantifiers: older.counts == newer.counts + * + * Note: List is newest-first, so we check if later nodes (older contexts) + * can absorb earlier nodes (newer contexts). + */ +static void +nfa_absorb_contexts(WindowAggState *winstate, RPRNFAContext *excludeCtx, int64 currentPos) +{ + RPRPattern *pattern = winstate->rpPattern; + RPRNFAContext *ctx; + int maxDepth; + + /* Need at least 2 contexts for absorption */ + if (winstate->nfaContext == NULL || winstate->nfaContext->next == NULL) + return; + + if (pattern == NULL) + return; + + /* + * Only absorb for patterns marked as absorbable during planning. + * See computeAbsorbability() in rpr.c for criteria. + */ + if (!pattern->isAbsorbable) + return; + + maxDepth = pattern->maxDepth; + + /* + * Context absorption: A later context (started at higher row) can be + * absorbed by an earlier context if ALL states in the later context + * are "covered" by states in the earlier context. + * + * A later state is covered by an earlier state if: + * 1. They are at the same element + * 2. For unbounded elements (max == INT32_MAX): earlier.counts[d] >= later.counts[d] + * for all depths d + * 3. For bounded elements: counts must be exactly equal at all depths + * + * List is newest-first (head = highest matchStartRow). + * We iterate from head (newest) and check if older contexts can absorb it. + */ + ctx = winstate->nfaContext; + + while (ctx != NULL) + { + RPRNFAContext *nextCtx = ctx->next; + RPRNFAContext *older; + bool absorbed = false; + + /* Never absorb the excluded context (it's the target being completed) */ + if (ctx == excludeCtx) + { + ctx = nextCtx; + continue; + } + + /* Skip contexts that haven't started processing yet (just created for future row) */ + if (ctx->matchStartRow > currentPos) + { + ctx = nextCtx; + continue; + } + + /* + * Handle completed contexts (states == NULL) separately. + * A completed context can be absorbed by an older completed context + * if the older one has the same or later matchEndRow. + */ + if (ctx->states == NULL) + { + /* Only completed contexts with valid matchEndRow can be absorbed */ + if (ctx->matchEndRow < 0) + { + ctx = nextCtx; + continue; + } + + /* Check if any older completed context can absorb this one */ + for (older = ctx->next; older != NULL && !absorbed; older = older->next) + { + /* Must have started earlier */ + if (older->matchStartRow >= ctx->matchStartRow) + continue; + + /* Older must also be completed with valid matchEndRow */ + if (older->states != NULL || older->matchEndRow < 0) + continue; + + /* + * Older context absorbs newer if it has the same or later + * matchEndRow, meaning all matches from newer are subsets. + */ + if (older->matchEndRow >= ctx->matchEndRow) + { + /* Absorb: remove ctx (newer) */ + nfa_unlink_context(winstate, ctx); + nfa_context_free(winstate, ctx); + absorbed = true; + } + } + + ctx = nextCtx; + continue; + } + + /* + * Check if all states in ctx are on absorbable branches. + * If any state is on a non-absorbable branch, skip this context. + */ + { + RPRNFAState *s; + bool allAbsorbable = true; + + for (s = ctx->states; s != NULL && allAbsorbable; s = s->next) + { + RPRPatternElement *branchFirst; + + /* altPriority is the branch's first element index */ + if (s->altPriority < 0 || s->altPriority >= pattern->numElements) + continue; + + branchFirst = &pattern->elements[s->altPriority]; + if (!(branchFirst->flags & RPR_ELEM_ABSORBABLE)) + allAbsorbable = false; + } + + if (!allAbsorbable) + { + ctx = nextCtx; + continue; + } + } + + /* + * Check if any older context can absorb this one. + * Older contexts have lower matchStartRow. + */ + for (older = ctx->next; older != NULL && !absorbed; older = older->next) + { + RPRNFAState *laterState; + RPRNFAState *earlierState; + bool canAbsorb; + int laterCount = 0; + int earlierCount = 0; + + /* Skip if not started earlier */ + if (older->matchStartRow >= ctx->matchStartRow) + continue; + + /* Skip contexts that haven't started processing yet */ + if (older->matchStartRow > currentPos) + continue; + + /* + * For in-progress ctx, older must also be in-progress. + * (Completed older contexts are handled above for completed ctx.) + */ + if (older->states == NULL) + continue; + + /* Count states in both contexts */ + for (laterState = ctx->states; laterState != NULL; laterState = laterState->next) + laterCount++; + for (earlierState = older->states; earlierState != NULL; earlierState = earlierState->next) + earlierCount++; + + /* Must have same number of states (same structure) */ + if (laterCount != earlierCount) + continue; + + /* + * Check if ALL states in ctx (later) are covered by states in older. + * Both must have the same set of element indices. + */ + canAbsorb = true; + for (laterState = ctx->states; laterState != NULL && canAbsorb; laterState = laterState->next) + { + bool found = false; + + for (earlierState = older->states; earlierState != NULL && !found; earlierState = earlierState->next) + { + RPRPatternElement *elem; + bool countsMatch; + int d; + + /* Must be at same element */ + if (earlierState->elemIdx != laterState->elemIdx) + continue; + + /* Must be on same branch (same altPriority) */ + if (earlierState->altPriority != laterState->altPriority) + continue; + + /* Handle invalid element index (terminal state) */ + if (laterState->elemIdx < 0 || laterState->elemIdx >= pattern->numElements) + { + found = true; + break; + } + + elem = &pattern->elements[laterState->elemIdx]; + countsMatch = true; + + if (elem->max == INT32_MAX) + { + /* Unbounded: earlier.count >= later.count at all depths */ + for (d = 0; d <= maxDepth && countsMatch; d++) + { + if (earlierState->counts[d] < laterState->counts[d]) + countsMatch = false; + } + } + else + { + /* Bounded: counts must be exactly equal at all depths */ + for (d = 0; d <= maxDepth && countsMatch; d++) + { + if (earlierState->counts[d] != laterState->counts[d]) + countsMatch = false; + } + } + + if (countsMatch) + found = true; + } + + if (!found) + canAbsorb = false; + } + + if (canAbsorb) + { + /* Absorb: remove ctx (newer) */ + nfa_unlink_context(winstate, ctx); + nfa_context_free(winstate, ctx); + absorbed = true; + } + } + + ctx = nextCtx; + } +} + +/* + * nfa_finalize_boundary + * + * Finalize NFA states at partition/frame boundary. + * Sets all varMatched to false and processes remaining states. + */ +static void +nfa_finalize_boundary(WindowAggState *winstate, RPRNFAContext *ctx, int64 matchEndPos) +{ + RPRNFAState *states = ctx->states; + RPRNFAState *state; + RPRNFAState *nextState; + int numVars = list_length(winstate->defineVariableList); + + ctx->states = NULL; + + for (int i = 0; i < numVars; i++) + winstate->nfaVarMatched[i] = false; + + for (state = states; state != NULL; state = nextState) + { + nextState = state->next; + state->next = NULL; + nfa_step_single(winstate, ctx, state, winstate->nfaVarMatched, matchEndPos); + } +} + +/* + * nfa_step_single + * + * Process one state through NFA for one row. + * New states are added to ctx->states. + * Match (FIN) is recorded in ctx->matchedState. + * When FIN is reached by matching (not skipping), matchEndRow is updated. + */ +static void +nfa_step_single(WindowAggState *winstate, RPRNFAContext *ctx, + RPRNFAState *state, bool *varMatched, int64 currentPos) +{ + RPRPattern *pattern = winstate->rpPattern; + RPRPatternElement *elements = pattern->elements; + RPRNFAState *pending = state; /* states to process in current row */ + + while (pending != NULL) + { + RPRPatternElement *elem; + bool matched; + int16 count; + int depth; + + state = pending; + pending = pending->next; + state->next = NULL; + + Assert(state->elemIdx >= 0 && state->elemIdx < pattern->numElements); + elem = &elements[state->elemIdx]; + depth = elem->depth; + + if (RPRElemIsVar(elem)) + { + /* + * Variable: check if it matches current row. + * With DEFINE-first ordering, varId < numDefines has DEFINE expr, + * varId >= numDefines defaults to TRUE. + */ + int numDefines = list_length(winstate->defineVariableList); + + if (elem->varId >= numDefines) + matched = true; /* Not defined in DEFINE, defaults to TRUE */ + else + matched = varMatched[elem->varId]; + + count = state->counts[depth]; + + if (matched) + { + count++; + + if (elem->max != RPR_QUANTITY_INF && count > elem->max) + { + nfa_state_free(winstate, state); + continue; + } + + state->counts[depth] = count; + + /* Can repeat more? Clone for staying */ + if (elem->max == RPR_QUANTITY_INF || count < elem->max) + { + RPRNFAState *clone = nfa_state_clone(winstate, state->elemIdx, + state->altPriority, + state->counts, NULL); + nfa_add_state_unique(winstate, ctx, clone); + } + + /* Satisfied? Advance to next element */ + if (count >= elem->min) + { + RPRPatternElement *nextElem; + + state->counts[depth] = 0; + state->elemIdx = elem->next; + nextElem = &elements[state->elemIdx]; + + if (RPRElemIsFin(nextElem)) + { + /* Match ends at current row since we matched */ + nfa_add_matched_state(winstate, ctx, state, currentPos); + nfa_state_free(winstate, state); + } + else if (RPRElemIsEnd(nextElem)) + { + /* + * END is epsilon transition - process immediately on same row. + * This ensures match end position is recorded at the row where + * the last VAR matched, not the next row. + */ + state->next = pending; + pending = state; + } + else + { + /* + * VAR, ALT - wait for next row. ALT dispatches to VARs that + * need input, so it must wait for the next row after VAR + * consumed the current row. + */ + nfa_add_state_unique(winstate, ctx, state); + } + } + else + { + nfa_state_free(winstate, state); + } + } + else + { + /* Not matched: can we skip? */ + if (count >= elem->min) + { + RPRPatternElement *nextElem; + + state->counts[depth] = 0; + state->elemIdx = elem->next; + nextElem = &elements[state->elemIdx]; + + if (RPRElemIsFin(nextElem)) + { + /* Match ends at previous row since current didn't match */ + nfa_add_matched_state(winstate, ctx, state, currentPos - 1); + nfa_state_free(winstate, state); + } + else if (RPRElemIsVar(nextElem)) + { + /* + * Current row was NOT consumed (skip case), so next VAR + * must be tried on the SAME row via pending list + */ + state->next = pending; + pending = state; + } + else + { + state->next = pending; + pending = state; + } + } + else + { + nfa_state_free(winstate, state); + } + } + } + else if (RPRElemIsFin(elem)) + { + /* Already at FIN - match ends at current row */ + nfa_add_matched_state(winstate, ctx, state, currentPos); + nfa_state_free(winstate, state); + } + else if (RPRElemIsAlt(elem)) + { + RPRElemIdx altIdx = elem->next; + bool first = true; + + /* + * ALT doesn't consume a row - it's just a dispatch point. + * All branches should be evaluated on the CURRENT row. + * Set altPriority to branch's elemIdx for lexical order tracking. + */ + while (altIdx >= 0 && altIdx < pattern->numElements) + { + RPRPatternElement *altElem = &elements[altIdx]; + + if (first) + { + state->elemIdx = altIdx; + state->altPriority = altIdx; /* lexical order */ + state->next = pending; + pending = state; + first = false; + } + else + { + pending = nfa_state_clone(winstate, altIdx, altIdx, + state->counts, pending); + } + + altIdx = altElem->jump; + } + + if (first) + nfa_state_free(winstate, state); + } + else if (RPRElemIsEnd(elem)) + { + count = state->counts[depth] + 1; + + if (count < elem->min) + { + /* + * Haven't reached minimum yet - must loop back. + * Add to ctx->states (next row) not pending (same row). + */ + state->counts[depth] = count; + for (int d = depth + 1; d < pattern->maxDepth; d++) + state->counts[d] = 0; + state->elemIdx = elem->jump; + nfa_add_state_unique(winstate, ctx, state); + } + else if (elem->max != RPR_QUANTITY_INF && count >= elem->max) + { + /* Reached maximum - must exit, continue processing */ + state->counts[depth] = 0; + state->elemIdx = elem->next; + state->next = pending; + pending = state; + } + else + { + /* + * Between min and max - can exit or continue. + * Exit state continues processing (pending). + * Loop state waits for next row (ctx->states). + */ + RPRNFAState *exitState = nfa_state_clone(winstate, elem->next, + state->altPriority, + state->counts, pending); + exitState->counts[depth] = 0; + pending = exitState; + + state->counts[depth] = count; + for (int d = depth + 1; d < pattern->maxDepth; d++) + state->counts[d] = 0; + state->elemIdx = elem->jump; + nfa_add_state_unique(winstate, ctx, state); + } + } + else + { + state->elemIdx = elem->next; + state->next = pending; + pending = state; + } + } +} + diff --git a/src/backend/utils/adt/windowfuncs.c b/src/backend/utils/adt/windowfuncs.c index 78b7f05aba2..723ebc91909 100644 --- a/src/backend/utils/adt/windowfuncs.c +++ b/src/backend/utils/adt/windowfuncs.c @@ -37,11 +37,19 @@ typedef struct int64 remainder; /* (total rows) % (bucket num) */ } ntile_context; +/* + * rpr process information. + * Used for AFTER MATCH SKIP PAST LAST ROW + */ +typedef struct SkipContext +{ + int64 pos; /* last row absolute position */ +} SkipContext; + static bool rank_up(WindowObject winobj); static Datum leadlag_common(FunctionCallInfo fcinfo, bool forward, bool withoffset, bool withdefault); - /* * utility routine for *_rank functions. */ @@ -683,7 +691,7 @@ window_last_value(PG_FUNCTION_ARGS) WinCheckAndInitializeNullTreatment(winobj, true, fcinfo); result = WinGetFuncArgInFrame(winobj, 0, - 0, WINDOW_SEEK_TAIL, true, + 0, WINDOW_SEEK_TAIL, false, &isnull, NULL); if (isnull) PG_RETURN_NULL(); @@ -724,3 +732,25 @@ window_nth_value(PG_FUNCTION_ARGS) PG_RETURN_DATUM(result); } + +/* + * prev + * Dummy function to invoke RPR's navigation operator "PREV". + * This is *not* a window function. + */ +Datum +window_prev(PG_FUNCTION_ARGS) +{ + PG_RETURN_DATUM(PG_GETARG_DATUM(0)); +} + +/* + * next + * Dummy function to invoke RPR's navigation operation "NEXT". + * This is *not* a window function. + */ +Datum +window_next(PG_FUNCTION_ARGS) +{ + PG_RETURN_DATUM(PG_GETARG_DATUM(0)); +} diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 2ac69bf2df5..124884fce13 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10853,6 +10853,12 @@ { oid => '3114', descr => 'fetch the Nth row value', proname => 'nth_value', prokind => 'w', prorettype => 'anyelement', proargtypes => 'anyelement int4', prosrc => 'window_nth_value' }, +{ oid => '8126', descr => 'previous value', + proname => 'prev', provolatile => 's', prorettype => 'anyelement', + proargtypes => 'anyelement', prosrc => 'window_prev' }, +{ oid => '8127', descr => 'next value', + proname => 'next', provolatile => 's', prorettype => 'anyelement', + proargtypes => 'anyelement', prosrc => 'window_next' }, # functions for range types { oid => '3832', descr => 'I/O', diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h index f8053d9e572..89139583855 100644 --- a/src/include/nodes/execnodes.h +++ b/src/include/nodes/execnodes.h @@ -41,6 +41,7 @@ #include "nodes/plannodes.h" #include "nodes/tidbitmap.h" #include "partitioning/partdefs.h" +#include "regex/regex.h" #include "storage/condition_variable.h" #include "utils/hsearch.h" #include "utils/queryenvironment.h" @@ -2512,6 +2513,39 @@ typedef enum WindowAggStatus * tuples during spool */ } WindowAggStatus; +#define RF_NOT_DETERMINED 0 +#define RF_FRAME_HEAD 1 +#define RF_SKIPPED 2 +#define RF_UNMATCHED 3 + +/* + * RPRNFAState - single NFA state for pattern matching + * + * counts[] tracks repetition counts at each nesting depth. + * altPriority tracks lexical order for alternation (lower = earlier alternative). + */ +typedef struct RPRNFAState +{ + struct RPRNFAState *next; /* next state in linked list */ + int16 elemIdx; /* current pattern element index */ + int16 altPriority; /* lexical order priority (lower = preferred) */ + int16 counts[FLEXIBLE_ARRAY_MEMBER]; /* repetition counts by depth */ +} RPRNFAState; + +/* + * RPRNFAContext - context for NFA pattern matching execution + */ +typedef struct RPRNFAContext +{ + struct RPRNFAContext *next; /* next context in linked list */ + struct RPRNFAContext *prev; /* previous context (for reverse traversal) */ + RPRNFAState *states; /* active states (linked list) */ + + int64 matchStartRow; /* row where match started */ + int64 matchEndRow; /* row where match ended (-1 = no match) */ + RPRNFAState *matchedState; /* FIN state for greedy fallback (cloned) */ +} RPRNFAContext; + typedef struct WindowAggState { ScanState ss; /* its first field is NodeTag */ @@ -2571,6 +2605,24 @@ typedef struct WindowAggState int64 groupheadpos; /* current row's peer group head position */ int64 grouptailpos; /* " " " " tail position (group end+1) */ + /* these fields are used in Row pattern recognition: */ + RPSkipTo rpSkipTo; /* Row Pattern Skip To type */ + struct RPRPattern *rpPattern; /* compiled pattern for NFA execution */ + List *defineVariableList; /* list of row pattern definition + * variables (list of String) */ + List *defineClauseList; /* expression for row pattern definition + * search conditions ExprState list */ + List *defineInitial; /* list of row pattern definition variable + * initials (list of String) */ + RPRNFAContext *nfaContext; /* active matching contexts (head) */ + RPRNFAContext *nfaContextTail; /* tail of active contexts (for reverse traversal) */ + RPRNFAContext *nfaContextPending; /* matched but awaiting earlier starts */ + RPRNFAContext *nfaContextFree; /* recycled NFA context nodes */ + RPRNFAState *nfaStateFree; /* recycled NFA state nodes */ + Size nfaStateSize; /* pre-calculated RPRNFAState size */ + bool *nfaVarMatched; /* per-row cache: varMatched[varId] for varId < numDefines */ + int64 nfaLastProcessedRow; /* last row processed by NFA (-1 = none) */ + MemoryContext partcontext; /* context for partition-lifespan data */ MemoryContext aggcontext; /* shared context for aggregate working data */ MemoryContext curaggcontext; /* current aggregate's working data */ @@ -2598,6 +2650,18 @@ typedef struct WindowAggState TupleTableSlot *agg_row_slot; TupleTableSlot *temp_slot_1; TupleTableSlot *temp_slot_2; + + /* temporary slots for RPR */ + TupleTableSlot *prev_slot; /* PREV row navigation operator */ + TupleTableSlot *next_slot; /* NEXT row navigation operator */ + TupleTableSlot *null_slot; /* all NULL slot */ + + /* + * Each byte corresponds to a row positioned at absolute its pos in + * partition. See above definition for RF_*. Used for RPR. + */ + char *reduced_frame_map; + int64 alloc_sz; /* size of the map */ } WindowAggState; /* ---------------- -- 2.43.0 [application/octet-stream] v38-0006-Row-pattern-recognition-patch-docs.patch (11.1K, ../../[email protected]/7-v38-0006-Row-pattern-recognition-patch-docs.patch) download | inline diff: From 3167ff5ec8e2b58c67f9a9ffa349aed33b149976 Mon Sep 17 00:00:00 2001 From: Tatsuo Ishii <[email protected]> Date: Thu, 15 Jan 2026 13:26:44 +0900 Subject: [PATCH v38 6/8] Row pattern recognition patch (docs). --- doc/src/sgml/advanced.sgml | 90 ++++++++++++++++++++++++++++-- doc/src/sgml/func/func-window.sgml | 53 ++++++++++++++++++ doc/src/sgml/ref/select.sgml | 56 ++++++++++++++++++- 3 files changed, 192 insertions(+), 7 deletions(-) diff --git a/doc/src/sgml/advanced.sgml b/doc/src/sgml/advanced.sgml index 451bcb202ec..eec2a0a9346 100644 --- a/doc/src/sgml/advanced.sgml +++ b/doc/src/sgml/advanced.sgml @@ -540,13 +540,95 @@ WHERE pos < 3; two rows for each department). </para> + <para> + Row pattern common syntax can be used to perform row pattern recognition + in a query. The row pattern common syntax includes two sub + clauses: <literal>DEFINE</literal> + and <literal>PATTERN</literal>. <literal>DEFINE</literal> defines + definition variables along with an expression. The expression must be a + logical expression, which means it must + return <literal>TRUE</literal>, <literal>FALSE</literal> + or <literal>NULL</literal>. The expression may comprise column references + and functions. Window functions, aggregate functions and subqueries are + not allowed. An example of <literal>DEFINE</literal> is as follows. + +<programlisting> +DEFINE + LOWPRICE AS price <= 100, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +</programlisting> + + Note that <function>PREV</function> returns the <literal>price</literal> + column in the previous row if it's called in a context of row pattern + recognition. Thus in the second line the definition variable "UP" + is <literal>TRUE</literal> when the price column in the current row is + greater than the price column in the previous row. Likewise, "DOWN" + is <literal>TRUE</literal> when the + <literal>price</literal> column in the current row is lower than + the <literal>price</literal> column in the previous row. + </para> + <para> + Once <literal>DEFINE</literal> exists, <literal>PATTERN</literal> can be + used. <literal>PATTERN</literal> defines a sequence of rows that satisfies + conditions defined in the <literal>DEFINE</literal> clause. For example + following <literal>PATTERN</literal> defines a sequence of rows starting + with the a row satisfying "LOWPRICE", then one or more rows satisfying + "UP" and finally one or more rows satisfying "DOWN". Note that "+" means + one or more matches. Also you can use "*", which means zero or more + matches. If a sequence of rows which satisfies the PATTERN is found, in + the starting row all columns or functions are shown in the target + list. Note that aggregations only look into the matched rows, rather than + the whole frame. On the second or subsequent rows all window functions are + shown as NULL. Aggregates are NULL or 0 depending on its aggregation + definition. A count() aggregate shows 0. For rows that do not match on the + PATTERN, columns are shown AS NULL too. Example of + a <literal>SELECT</literal> using the <literal>DEFINE</literal> + and <literal>PATTERN</literal> clause is as follows. + +<programlisting> +SELECT company, tdate, price, + first_value(price) OVER w, + max(price) OVER w, + count(price) OVER w +FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + INITIAL + PATTERN (LOWPRICE UP+ DOWN+) + DEFINE + LOWPRICE AS price <= 100, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); +</programlisting> +<screen> + company | tdate | price | first_value | max | count +----------+------------+-------+-------------+-----+------- + company1 | 2023-07-01 | 100 | 100 | 200 | 4 + company1 | 2023-07-02 | 200 | | | 0 + company1 | 2023-07-03 | 150 | | | 0 + company1 | 2023-07-04 | 140 | | | 0 + company1 | 2023-07-05 | 150 | | | 0 + company1 | 2023-07-06 | 90 | 90 | 130 | 4 + company1 | 2023-07-07 | 110 | | | 0 + company1 | 2023-07-08 | 130 | | | 0 + company1 | 2023-07-09 | 120 | | | 0 + company1 | 2023-07-10 | 130 | | | 0 +(10 rows) +</screen> + </para> + <para> When a query involves multiple window functions, it is possible to write out each one with a separate <literal>OVER</literal> clause, but this is - duplicative and error-prone if the same windowing behavior is wanted - for several functions. Instead, each windowing behavior can be named - in a <literal>WINDOW</literal> clause and then referenced in <literal>OVER</literal>. - For example: + duplicative and error-prone if the same windowing behavior is wanted for + several functions. Instead, each windowing behavior can be named in + a <literal>WINDOW</literal> clause and then referenced + in <literal>OVER</literal>. For example: <programlisting> SELECT sum(salary) OVER w, avg(salary) OVER w diff --git a/doc/src/sgml/func/func-window.sgml b/doc/src/sgml/func/func-window.sgml index bcf755c9ebc..ae36e0f3135 100644 --- a/doc/src/sgml/func/func-window.sgml +++ b/doc/src/sgml/func/func-window.sgml @@ -278,6 +278,59 @@ <function>nth_value</function>. </para> + <para> + Row pattern recognition navigation functions are listed in + <xref linkend="functions-rpr-navigation-table"/>. These functions + can be used to describe DEFINE clause of Row pattern recognition. + </para> + + <table id="functions-rpr-navigation-table"> + <title>Row Pattern Navigation Functions</title> + <tgroup cols="1"> + <thead> + <row> + <entry role="func_table_entry"><para role="func_signature"> + Function + </para> + <para> + Description + </para></entry> + </row> + </thead> + + <tbody> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <indexterm> + <primary>prev</primary> + </indexterm> + <function>prev</function> ( <parameter>value</parameter> <type>anyelement</type> ) + <returnvalue>anyelement</returnvalue> + </para> + <para> + Returns the column value at the previous row; + returns NULL if there is no previous row in the window frame. + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <indexterm> + <primary>next</primary> + </indexterm> + <function>next</function> ( <parameter>value</parameter> <type>anyelement</type> ) + <returnvalue>anyelement</returnvalue> + </para> + <para> + Returns the column value at the next row; + returns NULL if there is no next row in the window frame. + </para></entry> + </row> + + </tbody> + </tgroup> + </table> + <note> <para> The SQL standard defines a <literal>FROM FIRST</literal> or <literal>FROM LAST</literal> diff --git a/doc/src/sgml/ref/select.sgml b/doc/src/sgml/ref/select.sgml index ca5dd14d627..49b3c00d9f2 100644 --- a/doc/src/sgml/ref/select.sgml +++ b/doc/src/sgml/ref/select.sgml @@ -979,8 +979,8 @@ WINDOW <replaceable class="parameter">window_name</replaceable> AS ( <replaceabl The <replaceable class="parameter">frame_clause</replaceable> can be one of <synopsis> -{ RANGE | ROWS | GROUPS } <replaceable>frame_start</replaceable> [ <replaceable>frame_exclusion</replaceable> ] -{ RANGE | ROWS | GROUPS } BETWEEN <replaceable>frame_start</replaceable> AND <replaceable>frame_end</replaceable> [ <replaceable>frame_exclusion</replaceable> ] +{ RANGE | ROWS | GROUPS } <replaceable>frame_start</replaceable> [ <replaceable>frame_exclusion</replaceable> ] [row_pattern_common_syntax] +{ RANGE | ROWS | GROUPS } BETWEEN <replaceable>frame_start</replaceable> AND <replaceable>frame_end</replaceable> [ <replaceable>frame_exclusion</replaceable> ] [row_pattern_common_syntax] </synopsis> where <replaceable>frame_start</replaceable> @@ -1087,9 +1087,59 @@ EXCLUDE NO OTHERS a given peer group will be in the frame or excluded from it. </para> + <para> + The + optional <replaceable class="parameter">row_pattern_common_syntax</replaceable> + defines the <firstterm>row pattern recognition condition</firstterm> for + this + window. <replaceable class="parameter">row_pattern_common_syntax</replaceable> + includes following subclauses. + +<synopsis> +[ { AFTER MATCH SKIP PAST LAST ROW | AFTER MATCH SKIP TO NEXT ROW } ] +[ INITIAL | SEEK ] +PATTERN ( <replaceable class="parameter">pattern_variable_name</replaceable>[*|+|?] [, ...] ) +DEFINE <replaceable class="parameter">definition_varible_name</replaceable> AS <replaceable class="parameter">expression</replaceable> [, ...] +</synopsis> + <literal>AFTER MATCH SKIP PAST LAST ROW</literal> or <literal>AFTER MATCH + SKIP TO NEXT ROW</literal> controls how to proceed to next row position + after a match found. With <literal>AFTER MATCH SKIP PAST LAST + ROW</literal> (the default) next row position is next to the last row of + previous match. On the other hand, with <literal>AFTER MATCH SKIP TO NEXT + ROW</literal> next row position is always next to the last row of previous + match. <literal>INITIAL</literal> or <literal>SEEK</literal> defines how a + successful pattern matching starts from which row in a + frame. If <literal>INITIAL</literal> is specified, the match must start + from the first row in the frame. If <literal>SEEK</literal> is specified, + the set of matching rows do not necessarily start from the first row. The + default is <literal>INITIAL</literal>. Currently + only <literal>INITIAL</literal> is supported. <literal>DEFINE</literal> + defines definition variables along with a boolean + expression. <literal>PATTERN</literal> defines a sequence of rows that + satisfies certain conditions using variables defined + in <literal>DEFINE</literal> clause. If the variable is not defined in + the <literal>DEFINE</literal> clause, it is implicitly assumed following + is defined in the <literal>DEFINE</literal> clause. + +<synopsis> +<literal>variable_name</literal> AS TRUE +</synopsis> + + Note that the maximum number of variables defined + in <literal>DEFINE</literal> clause is 26. + </para> + + <para> + The SQL standard defines more subclauses: <literal>MEASURES</literal> + and <literal>SUBSET</literal>. They are not currently supported + in <productname>PostgreSQL</productname>. Also in the standard there are + more variations in <literal>AFTER MATCH</literal> clause. + </para> + <para> The purpose of a <literal>WINDOW</literal> clause is to specify the - behavior of <firstterm>window functions</firstterm> appearing in the query's + behavior of <firstterm>window functions</firstterm> appearing in the + query's <link linkend="sql-select-list"><command>SELECT</command> list</link> or <link linkend="sql-orderby"><literal>ORDER BY</literal></link> clause. These functions -- 2.43.0 [application/octet-stream] v38-0007-Row-pattern-recognition-patch-tests.patch (155.4K, ../../[email protected]/8-v38-0007-Row-pattern-recognition-patch-tests.patch) download | inline diff: From 6a50dd3d2e29c6ff17b71ef632204cc2521e56e1 Mon Sep 17 00:00:00 2001 From: Tatsuo Ishii <[email protected]> Date: Thu, 15 Jan 2026 13:26:44 +0900 Subject: [PATCH v38 7/8] Row pattern recognition patch (tests). --- src/test/regress/expected/rpr.out | 2736 ++++++++++++++++++++++++++++ src/test/regress/parallel_schedule | 2 +- src/test/regress/sql/rpr.sql | 1394 ++++++++++++++ 3 files changed, 4131 insertions(+), 1 deletion(-) create mode 100644 src/test/regress/expected/rpr.out create mode 100644 src/test/regress/sql/rpr.sql diff --git a/src/test/regress/expected/rpr.out b/src/test/regress/expected/rpr.out new file mode 100644 index 00000000000..cea986d6704 --- /dev/null +++ b/src/test/regress/expected/rpr.out @@ -0,0 +1,2736 @@ +-- +-- Test for row pattern definition clause +-- +CREATE TEMP TABLE stock ( + company TEXT, + tdate DATE, + price INTEGER +); +INSERT INTO stock VALUES ('company1', '2023-07-01', 100); +INSERT INTO stock VALUES ('company1', '2023-07-02', 200); +INSERT INTO stock VALUES ('company1', '2023-07-03', 150); +INSERT INTO stock VALUES ('company1', '2023-07-04', 140); +INSERT INTO stock VALUES ('company1', '2023-07-05', 150); +INSERT INTO stock VALUES ('company1', '2023-07-06', 90); +INSERT INTO stock VALUES ('company1', '2023-07-07', 110); +INSERT INTO stock VALUES ('company1', '2023-07-08', 130); +INSERT INTO stock VALUES ('company1', '2023-07-09', 120); +INSERT INTO stock VALUES ('company1', '2023-07-10', 130); +INSERT INTO stock VALUES ('company2', '2023-07-01', 50); +INSERT INTO stock VALUES ('company2', '2023-07-02', 2000); +INSERT INTO stock VALUES ('company2', '2023-07-03', 1500); +INSERT INTO stock VALUES ('company2', '2023-07-04', 1400); +INSERT INTO stock VALUES ('company2', '2023-07-05', 1500); +INSERT INTO stock VALUES ('company2', '2023-07-06', 60); +INSERT INTO stock VALUES ('company2', '2023-07-07', 1100); +INSERT INTO stock VALUES ('company2', '2023-07-08', 1300); +INSERT INTO stock VALUES ('company2', '2023-07-09', 1200); +INSERT INTO stock VALUES ('company2', '2023-07-10', 1300); +SELECT * FROM stock; + company | tdate | price +----------+------------+------- + company1 | 07-01-2023 | 100 + company1 | 07-02-2023 | 200 + company1 | 07-03-2023 | 150 + company1 | 07-04-2023 | 140 + company1 | 07-05-2023 | 150 + company1 | 07-06-2023 | 90 + company1 | 07-07-2023 | 110 + company1 | 07-08-2023 | 130 + company1 | 07-09-2023 | 120 + company1 | 07-10-2023 | 130 + company2 | 07-01-2023 | 50 + company2 | 07-02-2023 | 2000 + company2 | 07-03-2023 | 1500 + company2 | 07-04-2023 | 1400 + company2 | 07-05-2023 | 1500 + company2 | 07-06-2023 | 60 + company2 | 07-07-2023 | 1100 + company2 | 07-08-2023 | 1300 + company2 | 07-09-2023 | 1200 + company2 | 07-10-2023 | 1300 +(20 rows) + +-- basic test using PREV +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w, + nth_value(tdate, 2) OVER w AS nth_second + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value | nth_second +----------+------------+-------+-------------+------------+------------ + company1 | 07-01-2023 | 100 | 100 | 140 | 07-02-2023 + company1 | 07-02-2023 | 200 | | | + company1 | 07-03-2023 | 150 | | | + company1 | 07-04-2023 | 140 | | | + company1 | 07-05-2023 | 150 | | | + company1 | 07-06-2023 | 90 | 90 | 120 | 07-07-2023 + company1 | 07-07-2023 | 110 | | | + company1 | 07-08-2023 | 130 | | | + company1 | 07-09-2023 | 120 | | | + company1 | 07-10-2023 | 130 | | | + company2 | 07-01-2023 | 50 | 50 | 1400 | 07-02-2023 + company2 | 07-02-2023 | 2000 | | | + company2 | 07-03-2023 | 1500 | | | + company2 | 07-04-2023 | 1400 | | | + company2 | 07-05-2023 | 1500 | | | + company2 | 07-06-2023 | 60 | 60 | 1200 | 07-07-2023 + company2 | 07-07-2023 | 1100 | | | + company2 | 07-08-2023 | 1300 | | | + company2 | 07-09-2023 | 1200 | | | + company2 | 07-10-2023 | 1300 | | | +(20 rows) + +-- basic test using PREV. UP appears twice +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w, + nth_value(tdate, 2) OVER w AS nth_second + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+ UP+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value | nth_second +----------+------------+-------+-------------+------------+------------ + company1 | 07-01-2023 | 100 | 100 | 150 | 07-02-2023 + company1 | 07-02-2023 | 200 | | | + company1 | 07-03-2023 | 150 | | | + company1 | 07-04-2023 | 140 | | | + company1 | 07-05-2023 | 150 | | | + company1 | 07-06-2023 | 90 | 90 | 130 | 07-07-2023 + company1 | 07-07-2023 | 110 | | | + company1 | 07-08-2023 | 130 | | | + company1 | 07-09-2023 | 120 | | | + company1 | 07-10-2023 | 130 | | | + company2 | 07-01-2023 | 50 | 50 | 1500 | 07-02-2023 + company2 | 07-02-2023 | 2000 | | | + company2 | 07-03-2023 | 1500 | | | + company2 | 07-04-2023 | 1400 | | | + company2 | 07-05-2023 | 1500 | | | + company2 | 07-06-2023 | 60 | 60 | 1300 | 07-07-2023 + company2 | 07-07-2023 | 1100 | | | + company2 | 07-08-2023 | 1300 | | | + company2 | 07-09-2023 | 1200 | | | + company2 | 07-10-2023 | 1300 | | | +(20 rows) + +-- basic test using PREV. Use '*' +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w, + nth_value(tdate, 2) OVER w AS nth_second + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP* DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value | nth_second +----------+------------+-------+-------------+------------+------------ + company1 | 07-01-2023 | 100 | 100 | 140 | 07-02-2023 + company1 | 07-02-2023 | 200 | | | + company1 | 07-03-2023 | 150 | | | + company1 | 07-04-2023 | 140 | | | + company1 | 07-05-2023 | 150 | 150 | 90 | 07-06-2023 + company1 | 07-06-2023 | 90 | | | + company1 | 07-07-2023 | 110 | 110 | 120 | 07-08-2023 + company1 | 07-08-2023 | 130 | | | + company1 | 07-09-2023 | 120 | | | + company1 | 07-10-2023 | 130 | | | + company2 | 07-01-2023 | 50 | 50 | 1400 | 07-02-2023 + company2 | 07-02-2023 | 2000 | | | + company2 | 07-03-2023 | 1500 | | | + company2 | 07-04-2023 | 1400 | | | + company2 | 07-05-2023 | 1500 | 1500 | 60 | 07-06-2023 + company2 | 07-06-2023 | 60 | | | + company2 | 07-07-2023 | 1100 | 1100 | 1200 | 07-08-2023 + company2 | 07-08-2023 | 1300 | | | + company2 | 07-09-2023 | 1200 | | | + company2 | 07-10-2023 | 1300 | | | +(20 rows) + +-- basic test using PREV. Use '?' +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w, + nth_value(tdate, 2) OVER w AS nth_second + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP? DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value | nth_second +----------+------------+-------+-------------+------------+------------ + company1 | 07-01-2023 | 100 | 100 | 140 | 07-02-2023 + company1 | 07-02-2023 | 200 | | | + company1 | 07-03-2023 | 150 | | | + company1 | 07-04-2023 | 140 | | | + company1 | 07-05-2023 | 150 | 150 | 90 | 07-06-2023 + company1 | 07-06-2023 | 90 | | | + company1 | 07-07-2023 | 110 | 110 | 120 | 07-08-2023 + company1 | 07-08-2023 | 130 | | | + company1 | 07-09-2023 | 120 | | | + company1 | 07-10-2023 | 130 | | | + company2 | 07-01-2023 | 50 | 50 | 1400 | 07-02-2023 + company2 | 07-02-2023 | 2000 | | | + company2 | 07-03-2023 | 1500 | | | + company2 | 07-04-2023 | 1400 | | | + company2 | 07-05-2023 | 1500 | 1500 | 60 | 07-06-2023 + company2 | 07-06-2023 | 60 | | | + company2 | 07-07-2023 | 1100 | 1100 | 1200 | 07-08-2023 + company2 | 07-08-2023 | 1300 | | | + company2 | 07-09-2023 | 1200 | | | + company2 | 07-10-2023 | 1300 | | | +(20 rows) + +-- test using alternation (|) with sequence +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START (UP | DOWN)) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | 100 | 200 + company1 | 07-02-2023 | 200 | | + company1 | 07-03-2023 | 150 | 150 | 140 + company1 | 07-04-2023 | 140 | | + company1 | 07-05-2023 | 150 | 150 | 90 + company1 | 07-06-2023 | 90 | | + company1 | 07-07-2023 | 110 | 110 | 130 + company1 | 07-08-2023 | 130 | | + company1 | 07-09-2023 | 120 | 120 | 130 + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | 50 | 2000 + company2 | 07-02-2023 | 2000 | | + company2 | 07-03-2023 | 1500 | 1500 | 1400 + company2 | 07-04-2023 | 1400 | | + company2 | 07-05-2023 | 1500 | 1500 | 60 + company2 | 07-06-2023 | 60 | | + company2 | 07-07-2023 | 1100 | 1100 | 1300 + company2 | 07-08-2023 | 1300 | | + company2 | 07-09-2023 | 1200 | 1200 | 1300 + company2 | 07-10-2023 | 1300 | | +(20 rows) + +-- test using alternation (|) with group quantifier +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START (UP | DOWN)+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | 100 | 130 + company1 | 07-02-2023 | 200 | | + company1 | 07-03-2023 | 150 | | + company1 | 07-04-2023 | 140 | | + company1 | 07-05-2023 | 150 | | + company1 | 07-06-2023 | 90 | | + company1 | 07-07-2023 | 110 | | + company1 | 07-08-2023 | 130 | | + company1 | 07-09-2023 | 120 | | + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | 50 | 1300 + company2 | 07-02-2023 | 2000 | | + company2 | 07-03-2023 | 1500 | | + company2 | 07-04-2023 | 1400 | | + company2 | 07-05-2023 | 1500 | | + company2 | 07-06-2023 | 60 | | + company2 | 07-07-2023 | 1100 | | + company2 | 07-08-2023 | 1300 | | + company2 | 07-09-2023 | 1200 | | + company2 | 07-10-2023 | 1300 | | +(20 rows) + +-- test using nested alternation +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START ((UP DOWN) | FLAT)+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price), + FLAT AS price = PREV(price) +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | 100 | 150 + company1 | 07-02-2023 | 200 | | + company1 | 07-03-2023 | 150 | | + company1 | 07-04-2023 | 140 | 140 | 90 + company1 | 07-05-2023 | 150 | | + company1 | 07-06-2023 | 90 | | + company1 | 07-07-2023 | 110 | 110 | 120 + company1 | 07-08-2023 | 130 | | + company1 | 07-09-2023 | 120 | | + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | 50 | 1500 + company2 | 07-02-2023 | 2000 | | + company2 | 07-03-2023 | 1500 | | + company2 | 07-04-2023 | 1400 | 1400 | 60 + company2 | 07-05-2023 | 1500 | | + company2 | 07-06-2023 | 60 | | + company2 | 07-07-2023 | 1100 | 1100 | 1200 + company2 | 07-08-2023 | 1300 | | + company2 | 07-09-2023 | 1200 | | + company2 | 07-10-2023 | 1300 | | +(20 rows) + +-- test using group with quantifier +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN ((UP DOWN)+) + DEFINE + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | | + company1 | 07-02-2023 | 200 | 200 | 150 + company1 | 07-03-2023 | 150 | | + company1 | 07-04-2023 | 140 | | + company1 | 07-05-2023 | 150 | 150 | 90 + company1 | 07-06-2023 | 90 | | + company1 | 07-07-2023 | 110 | | + company1 | 07-08-2023 | 130 | 130 | 120 + company1 | 07-09-2023 | 120 | | + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | | + company2 | 07-02-2023 | 2000 | 2000 | 1500 + company2 | 07-03-2023 | 1500 | | + company2 | 07-04-2023 | 1400 | | + company2 | 07-05-2023 | 1500 | 1500 | 60 + company2 | 07-06-2023 | 60 | | + company2 | 07-07-2023 | 1100 | | + company2 | 07-08-2023 | 1300 | 1300 | 1200 + company2 | 07-09-2023 | 1200 | | + company2 | 07-10-2023 | 1300 | | +(20 rows) + +-- test using absolute threshold values (not relative PREV) +-- HIGH: price > 150, LOW: price < 100, MID: neutral range +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (LOW MID* HIGH) + DEFINE + LOW AS price < 100, + MID AS price >= 100 AND price <= 150, + HIGH AS price > 150 +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | | + company1 | 07-02-2023 | 200 | | + company1 | 07-03-2023 | 150 | | + company1 | 07-04-2023 | 140 | | + company1 | 07-05-2023 | 150 | | + company1 | 07-06-2023 | 90 | | + company1 | 07-07-2023 | 110 | | + company1 | 07-08-2023 | 130 | | + company1 | 07-09-2023 | 120 | | + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | 50 | 2000 + company2 | 07-02-2023 | 2000 | | + company2 | 07-03-2023 | 1500 | | + company2 | 07-04-2023 | 1400 | | + company2 | 07-05-2023 | 1500 | | + company2 | 07-06-2023 | 60 | 60 | 1100 + company2 | 07-07-2023 | 1100 | | + company2 | 07-08-2023 | 1300 | | + company2 | 07-09-2023 | 1200 | | + company2 | 07-10-2023 | 1300 | | +(20 rows) + +-- test threshold-based pattern with alternation +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (LOW (MID | HIGH)+) + DEFINE + LOW AS price < 100, + MID AS price >= 100 AND price <= 150, + HIGH AS price > 150 +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | | + company1 | 07-02-2023 | 200 | | + company1 | 07-03-2023 | 150 | | + company1 | 07-04-2023 | 140 | | + company1 | 07-05-2023 | 150 | | + company1 | 07-06-2023 | 90 | 90 | 130 + company1 | 07-07-2023 | 110 | | + company1 | 07-08-2023 | 130 | | + company1 | 07-09-2023 | 120 | | + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | 50 | 1500 + company2 | 07-02-2023 | 2000 | | + company2 | 07-03-2023 | 1500 | | + company2 | 07-04-2023 | 1400 | | + company2 | 07-05-2023 | 1500 | | + company2 | 07-06-2023 | 60 | 60 | 1300 + company2 | 07-07-2023 | 1100 | | + company2 | 07-08-2023 | 1300 | | + company2 | 07-09-2023 | 1200 | | + company2 | 07-10-2023 | 1300 | | +(20 rows) + +-- basic test with none-greedy pattern +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A A A) + DEFINE + A AS price >= 140 AND price <= 150 +); + company | tdate | price | count +----------+------------+-------+------- + company1 | 07-01-2023 | 100 | 0 + company1 | 07-02-2023 | 200 | 0 + company1 | 07-03-2023 | 150 | 3 + company1 | 07-04-2023 | 140 | 0 + company1 | 07-05-2023 | 150 | 0 + company1 | 07-06-2023 | 90 | 0 + company1 | 07-07-2023 | 110 | 0 + company1 | 07-08-2023 | 130 | 0 + company1 | 07-09-2023 | 120 | 0 + company1 | 07-10-2023 | 130 | 0 + company2 | 07-01-2023 | 50 | 0 + company2 | 07-02-2023 | 2000 | 0 + company2 | 07-03-2023 | 1500 | 0 + company2 | 07-04-2023 | 1400 | 0 + company2 | 07-05-2023 | 1500 | 0 + company2 | 07-06-2023 | 60 | 0 + company2 | 07-07-2023 | 1100 | 0 + company2 | 07-08-2023 | 1300 | 0 + company2 | 07-09-2023 | 1200 | 0 + company2 | 07-10-2023 | 1300 | 0 +(20 rows) + +-- test using {n} quantifier (A A A should be optimized to A{3}) +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A{3}) + DEFINE + A AS price >= 140 AND price <= 150 +); + company | tdate | price | count +----------+------------+-------+------- + company1 | 07-01-2023 | 100 | 0 + company1 | 07-02-2023 | 200 | 0 + company1 | 07-03-2023 | 150 | 3 + company1 | 07-04-2023 | 140 | 0 + company1 | 07-05-2023 | 150 | 0 + company1 | 07-06-2023 | 90 | 0 + company1 | 07-07-2023 | 110 | 0 + company1 | 07-08-2023 | 130 | 0 + company1 | 07-09-2023 | 120 | 0 + company1 | 07-10-2023 | 130 | 0 + company2 | 07-01-2023 | 50 | 0 + company2 | 07-02-2023 | 2000 | 0 + company2 | 07-03-2023 | 1500 | 0 + company2 | 07-04-2023 | 1400 | 0 + company2 | 07-05-2023 | 1500 | 0 + company2 | 07-06-2023 | 60 | 0 + company2 | 07-07-2023 | 1100 | 0 + company2 | 07-08-2023 | 1300 | 0 + company2 | 07-09-2023 | 1200 | 0 + company2 | 07-10-2023 | 1300 | 0 +(20 rows) + +-- test using {n,} quantifier (2 or more) +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A{2,}) + DEFINE + A AS price > 100 +); + company | tdate | price | count +----------+------------+-------+------- + company1 | 07-01-2023 | 100 | 0 + company1 | 07-02-2023 | 200 | 4 + company1 | 07-03-2023 | 150 | 0 + company1 | 07-04-2023 | 140 | 0 + company1 | 07-05-2023 | 150 | 0 + company1 | 07-06-2023 | 90 | 0 + company1 | 07-07-2023 | 110 | 4 + company1 | 07-08-2023 | 130 | 0 + company1 | 07-09-2023 | 120 | 0 + company1 | 07-10-2023 | 130 | 0 + company2 | 07-01-2023 | 50 | 0 + company2 | 07-02-2023 | 2000 | 4 + company2 | 07-03-2023 | 1500 | 0 + company2 | 07-04-2023 | 1400 | 0 + company2 | 07-05-2023 | 1500 | 0 + company2 | 07-06-2023 | 60 | 0 + company2 | 07-07-2023 | 1100 | 4 + company2 | 07-08-2023 | 1300 | 0 + company2 | 07-09-2023 | 1200 | 0 + company2 | 07-10-2023 | 1300 | 0 +(20 rows) + +-- test using {n,m} quantifier (2 to 4) +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A{2,4}) + DEFINE + A AS price > 100 +); + company | tdate | price | count +----------+------------+-------+------- + company1 | 07-01-2023 | 100 | 0 + company1 | 07-02-2023 | 200 | 4 + company1 | 07-03-2023 | 150 | 0 + company1 | 07-04-2023 | 140 | 0 + company1 | 07-05-2023 | 150 | 0 + company1 | 07-06-2023 | 90 | 0 + company1 | 07-07-2023 | 110 | 4 + company1 | 07-08-2023 | 130 | 0 + company1 | 07-09-2023 | 120 | 0 + company1 | 07-10-2023 | 130 | 0 + company2 | 07-01-2023 | 50 | 0 + company2 | 07-02-2023 | 2000 | 4 + company2 | 07-03-2023 | 1500 | 0 + company2 | 07-04-2023 | 1400 | 0 + company2 | 07-05-2023 | 1500 | 0 + company2 | 07-06-2023 | 60 | 0 + company2 | 07-07-2023 | 1100 | 4 + company2 | 07-08-2023 | 1300 | 0 + company2 | 07-09-2023 | 1200 | 0 + company2 | 07-10-2023 | 1300 | 0 +(20 rows) + +-- last_value() should remain consistent +SELECT company, tdate, price, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + company | tdate | price | last_value +----------+------------+-------+------------ + company1 | 07-01-2023 | 100 | 140 + company1 | 07-02-2023 | 200 | + company1 | 07-03-2023 | 150 | + company1 | 07-04-2023 | 140 | + company1 | 07-05-2023 | 150 | + company1 | 07-06-2023 | 90 | 120 + company1 | 07-07-2023 | 110 | + company1 | 07-08-2023 | 130 | + company1 | 07-09-2023 | 120 | + company1 | 07-10-2023 | 130 | + company2 | 07-01-2023 | 50 | 1400 + company2 | 07-02-2023 | 2000 | + company2 | 07-03-2023 | 1500 | + company2 | 07-04-2023 | 1400 | + company2 | 07-05-2023 | 1500 | + company2 | 07-06-2023 | 60 | 1200 + company2 | 07-07-2023 | 1100 | + company2 | 07-08-2023 | 1300 | + company2 | 07-09-2023 | 1200 | + company2 | 07-10-2023 | 1300 | +(20 rows) + +-- omit "START" in DEFINE but it is ok because "START AS TRUE" is +-- implicitly defined. per spec. +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w, + nth_value(tdate, 2) OVER w AS nth_second + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value | nth_second +----------+------------+-------+-------------+------------+------------ + company1 | 07-01-2023 | 100 | 100 | 140 | 07-02-2023 + company1 | 07-02-2023 | 200 | | | + company1 | 07-03-2023 | 150 | | | + company1 | 07-04-2023 | 140 | | | + company1 | 07-05-2023 | 150 | | | + company1 | 07-06-2023 | 90 | 90 | 120 | 07-07-2023 + company1 | 07-07-2023 | 110 | | | + company1 | 07-08-2023 | 130 | | | + company1 | 07-09-2023 | 120 | | | + company1 | 07-10-2023 | 130 | | | + company2 | 07-01-2023 | 50 | 50 | 1400 | 07-02-2023 + company2 | 07-02-2023 | 2000 | | | + company2 | 07-03-2023 | 1500 | | | + company2 | 07-04-2023 | 1400 | | | + company2 | 07-05-2023 | 1500 | | | + company2 | 07-06-2023 | 60 | 60 | 1200 | 07-07-2023 + company2 | 07-07-2023 | 1100 | | | + company2 | 07-08-2023 | 1300 | | | + company2 | 07-09-2023 | 1200 | | | + company2 | 07-10-2023 | 1300 | | | +(20 rows) + +-- the first row start with less than or equal to 100 +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (LOWPRICE UP+ DOWN+) + DEFINE + LOWPRICE AS price <= 100, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | 100 | 140 + company1 | 07-02-2023 | 200 | | + company1 | 07-03-2023 | 150 | | + company1 | 07-04-2023 | 140 | | + company1 | 07-05-2023 | 150 | | + company1 | 07-06-2023 | 90 | 90 | 120 + company1 | 07-07-2023 | 110 | | + company1 | 07-08-2023 | 130 | | + company1 | 07-09-2023 | 120 | | + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | 50 | 1400 + company2 | 07-02-2023 | 2000 | | + company2 | 07-03-2023 | 1500 | | + company2 | 07-04-2023 | 1400 | | + company2 | 07-05-2023 | 1500 | | + company2 | 07-06-2023 | 60 | 60 | 1200 + company2 | 07-07-2023 | 1100 | | + company2 | 07-08-2023 | 1300 | | + company2 | 07-09-2023 | 1200 | | + company2 | 07-10-2023 | 1300 | | +(20 rows) + +-- second row raises 120% +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (LOWPRICE UP+ DOWN+) + DEFINE + LOWPRICE AS price <= 100, + UP AS price > PREV(price) * 1.2, + DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | 100 | 140 + company1 | 07-02-2023 | 200 | | + company1 | 07-03-2023 | 150 | | + company1 | 07-04-2023 | 140 | | + company1 | 07-05-2023 | 150 | | + company1 | 07-06-2023 | 90 | | + company1 | 07-07-2023 | 110 | | + company1 | 07-08-2023 | 130 | | + company1 | 07-09-2023 | 120 | | + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | 50 | 1400 + company2 | 07-02-2023 | 2000 | | + company2 | 07-03-2023 | 1500 | | + company2 | 07-04-2023 | 1400 | | + company2 | 07-05-2023 | 1500 | | + company2 | 07-06-2023 | 60 | | + company2 | 07-07-2023 | 1100 | | + company2 | 07-08-2023 | 1300 | | + company2 | 07-09-2023 | 1200 | | + company2 | 07-10-2023 | 1300 | | +(20 rows) + +-- using NEXT +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UPDOWN) + DEFINE + START AS TRUE, + UPDOWN AS price > PREV(price) AND price > NEXT(price) +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | 100 | 200 + company1 | 07-02-2023 | 200 | | + company1 | 07-03-2023 | 150 | | + company1 | 07-04-2023 | 140 | 140 | 150 + company1 | 07-05-2023 | 150 | | + company1 | 07-06-2023 | 90 | | + company1 | 07-07-2023 | 110 | 110 | 130 + company1 | 07-08-2023 | 130 | | + company1 | 07-09-2023 | 120 | | + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | 50 | 2000 + company2 | 07-02-2023 | 2000 | | + company2 | 07-03-2023 | 1500 | | + company2 | 07-04-2023 | 1400 | 1400 | 1500 + company2 | 07-05-2023 | 1500 | | + company2 | 07-06-2023 | 60 | | + company2 | 07-07-2023 | 1100 | 1100 | 1300 + company2 | 07-08-2023 | 1300 | | + company2 | 07-09-2023 | 1200 | | + company2 | 07-10-2023 | 1300 | | +(20 rows) + +-- using AFTER MATCH SKIP TO NEXT ROW +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + INITIAL + PATTERN (START UPDOWN) + DEFINE + START AS TRUE, + UPDOWN AS price > PREV(price) AND price > NEXT(price) +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | 100 | 200 + company1 | 07-02-2023 | 200 | | + company1 | 07-03-2023 | 150 | | + company1 | 07-04-2023 | 140 | 140 | 150 + company1 | 07-05-2023 | 150 | | + company1 | 07-06-2023 | 90 | | + company1 | 07-07-2023 | 110 | 110 | 130 + company1 | 07-08-2023 | 130 | | + company1 | 07-09-2023 | 120 | | + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | 50 | 2000 + company2 | 07-02-2023 | 2000 | | + company2 | 07-03-2023 | 1500 | | + company2 | 07-04-2023 | 1400 | 1400 | 1500 + company2 | 07-05-2023 | 1500 | | + company2 | 07-06-2023 | 60 | | + company2 | 07-07-2023 | 1100 | 1100 | 1300 + company2 | 07-08-2023 | 1300 | | + company2 | 07-09-2023 | 1200 | | + company2 | 07-10-2023 | 1300 | | +(20 rows) + +-- match everything +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + INITIAL + PATTERN (A+) + DEFINE + A AS TRUE +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | 100 | 130 + company1 | 07-02-2023 | 200 | | + company1 | 07-03-2023 | 150 | | + company1 | 07-04-2023 | 140 | | + company1 | 07-05-2023 | 150 | | + company1 | 07-06-2023 | 90 | | + company1 | 07-07-2023 | 110 | | + company1 | 07-08-2023 | 130 | | + company1 | 07-09-2023 | 120 | | + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | 50 | 1300 + company2 | 07-02-2023 | 2000 | | + company2 | 07-03-2023 | 1500 | | + company2 | 07-04-2023 | 1400 | | + company2 | 07-05-2023 | 1500 | | + company2 | 07-06-2023 | 60 | | + company2 | 07-07-2023 | 1100 | | + company2 | 07-08-2023 | 1300 | | + company2 | 07-09-2023 | 1200 | | + company2 | 07-10-2023 | 1300 | | +(20 rows) + +-- backtracking with reclassification of rows +-- using AFTER MATCH SKIP PAST LAST ROW +SELECT company, tdate, price, first_value(tdate) OVER w, last_value(tdate) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + INITIAL + PATTERN (A+ B+) + DEFINE + A AS price > 100, + B AS price > 100 +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | | + company1 | 07-02-2023 | 200 | 07-02-2023 | 07-05-2023 + company1 | 07-03-2023 | 150 | | + company1 | 07-04-2023 | 140 | | + company1 | 07-05-2023 | 150 | | + company1 | 07-06-2023 | 90 | | + company1 | 07-07-2023 | 110 | 07-07-2023 | 07-10-2023 + company1 | 07-08-2023 | 130 | | + company1 | 07-09-2023 | 120 | | + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | | + company2 | 07-02-2023 | 2000 | 07-02-2023 | 07-05-2023 + company2 | 07-03-2023 | 1500 | | + company2 | 07-04-2023 | 1400 | | + company2 | 07-05-2023 | 1500 | | + company2 | 07-06-2023 | 60 | | + company2 | 07-07-2023 | 1100 | 07-07-2023 | 07-10-2023 + company2 | 07-08-2023 | 1300 | | + company2 | 07-09-2023 | 1200 | | + company2 | 07-10-2023 | 1300 | | +(20 rows) + +-- backtracking with reclassification of rows +-- using AFTER MATCH SKIP TO NEXT ROW +SELECT company, tdate, price, first_value(tdate) OVER w, last_value(tdate) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + INITIAL + PATTERN (A+ B+) + DEFINE + A AS price > 100, + B AS price > 100 +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | | + company1 | 07-02-2023 | 200 | 07-02-2023 | 07-05-2023 + company1 | 07-03-2023 | 150 | 07-03-2023 | 07-05-2023 + company1 | 07-04-2023 | 140 | 07-04-2023 | 07-05-2023 + company1 | 07-05-2023 | 150 | | + company1 | 07-06-2023 | 90 | | + company1 | 07-07-2023 | 110 | 07-07-2023 | 07-10-2023 + company1 | 07-08-2023 | 130 | 07-08-2023 | 07-10-2023 + company1 | 07-09-2023 | 120 | 07-09-2023 | 07-10-2023 + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | | + company2 | 07-02-2023 | 2000 | 07-02-2023 | 07-05-2023 + company2 | 07-03-2023 | 1500 | 07-03-2023 | 07-05-2023 + company2 | 07-04-2023 | 1400 | 07-04-2023 | 07-05-2023 + company2 | 07-05-2023 | 1500 | | + company2 | 07-06-2023 | 60 | | + company2 | 07-07-2023 | 1100 | 07-07-2023 | 07-10-2023 + company2 | 07-08-2023 | 1300 | 07-08-2023 | 07-10-2023 + company2 | 07-09-2023 | 1200 | 07-09-2023 | 07-10-2023 + company2 | 07-10-2023 | 1300 | | +(20 rows) + +-- ROWS BETWEEN CURRENT ROW AND offset FOLLOWING +SELECT company, tdate, price, first_value(tdate) OVER w, last_value(tdate) OVER w, + count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND 2 FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value | count +----------+------------+-------+-------------+------------+------- + company1 | 07-01-2023 | 100 | 07-01-2023 | 07-03-2023 | 3 + company1 | 07-02-2023 | 200 | | | 0 + company1 | 07-03-2023 | 150 | | | 0 + company1 | 07-04-2023 | 140 | 07-04-2023 | 07-06-2023 | 3 + company1 | 07-05-2023 | 150 | | | 0 + company1 | 07-06-2023 | 90 | | | 0 + company1 | 07-07-2023 | 110 | 07-07-2023 | 07-09-2023 | 3 + company1 | 07-08-2023 | 130 | | | 0 + company1 | 07-09-2023 | 120 | | | 0 + company1 | 07-10-2023 | 130 | | | 0 + company2 | 07-01-2023 | 50 | 07-01-2023 | 07-03-2023 | 3 + company2 | 07-02-2023 | 2000 | | | 0 + company2 | 07-03-2023 | 1500 | | | 0 + company2 | 07-04-2023 | 1400 | 07-04-2023 | 07-06-2023 | 3 + company2 | 07-05-2023 | 1500 | | | 0 + company2 | 07-06-2023 | 60 | | | 0 + company2 | 07-07-2023 | 1100 | 07-07-2023 | 07-09-2023 | 3 + company2 | 07-08-2023 | 1300 | | | 0 + company2 | 07-09-2023 | 1200 | | | 0 + company2 | 07-10-2023 | 1300 | | | 0 +(20 rows) + +-- +-- Aggregates +-- +-- using AFTER MATCH SKIP PAST LAST ROW +SELECT company, tdate, price, + first_value(price) OVER w, + last_value(price) OVER w, + max(price) OVER w, + min(price) OVER w, + sum(price) OVER w, + avg(price) OVER w, + count(price) OVER w +FROM stock +WINDOW w AS ( +PARTITION BY company +ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING +AFTER MATCH SKIP PAST LAST ROW +INITIAL +PATTERN (START UP+ DOWN+) +DEFINE +START AS TRUE, +UP AS price > PREV(price), +DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value | max | min | sum | avg | count +----------+------------+-------+-------------+------------+------+-----+------+-----------------------+------- + company1 | 07-01-2023 | 100 | 100 | 140 | 200 | 100 | 590 | 147.5000000000000000 | 4 + company1 | 07-02-2023 | 200 | | | | | | | 0 + company1 | 07-03-2023 | 150 | | | | | | | 0 + company1 | 07-04-2023 | 140 | | | | | | | 0 + company1 | 07-05-2023 | 150 | | | | | | | 0 + company1 | 07-06-2023 | 90 | 90 | 120 | 130 | 90 | 450 | 112.5000000000000000 | 4 + company1 | 07-07-2023 | 110 | | | | | | | 0 + company1 | 07-08-2023 | 130 | | | | | | | 0 + company1 | 07-09-2023 | 120 | | | | | | | 0 + company1 | 07-10-2023 | 130 | | | | | | | 0 + company2 | 07-01-2023 | 50 | 50 | 1400 | 2000 | 50 | 4950 | 1237.5000000000000000 | 4 + company2 | 07-02-2023 | 2000 | | | | | | | 0 + company2 | 07-03-2023 | 1500 | | | | | | | 0 + company2 | 07-04-2023 | 1400 | | | | | | | 0 + company2 | 07-05-2023 | 1500 | | | | | | | 0 + company2 | 07-06-2023 | 60 | 60 | 1200 | 1300 | 60 | 3660 | 915.0000000000000000 | 4 + company2 | 07-07-2023 | 1100 | | | | | | | 0 + company2 | 07-08-2023 | 1300 | | | | | | | 0 + company2 | 07-09-2023 | 1200 | | | | | | | 0 + company2 | 07-10-2023 | 1300 | | | | | | | 0 +(20 rows) + +-- using AFTER MATCH SKIP TO NEXT ROW +SELECT company, tdate, price, + first_value(price) OVER w, + last_value(price) OVER w, + max(price) OVER w, + min(price) OVER w, + sum(price) OVER w, + avg(price) OVER w, + count(price) OVER w +FROM stock +WINDOW w AS ( +PARTITION BY company +ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING +AFTER MATCH SKIP TO NEXT ROW +INITIAL +PATTERN (START UP+ DOWN+) +DEFINE +START AS TRUE, +UP AS price > PREV(price), +DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value | max | min | sum | avg | count +----------+------------+-------+-------------+------------+------+------+------+-----------------------+------- + company1 | 07-01-2023 | 100 | 100 | 140 | 200 | 100 | 590 | 147.5000000000000000 | 4 + company1 | 07-02-2023 | 200 | | | | | | | 0 + company1 | 07-03-2023 | 150 | | | | | | | 0 + company1 | 07-04-2023 | 140 | 140 | 90 | 150 | 90 | 380 | 126.6666666666666667 | 3 + company1 | 07-05-2023 | 150 | | | | | | | 0 + company1 | 07-06-2023 | 90 | 90 | 120 | 130 | 90 | 450 | 112.5000000000000000 | 4 + company1 | 07-07-2023 | 110 | 110 | 120 | 130 | 110 | 360 | 120.0000000000000000 | 3 + company1 | 07-08-2023 | 130 | | | | | | | 0 + company1 | 07-09-2023 | 120 | | | | | | | 0 + company1 | 07-10-2023 | 130 | | | | | | | 0 + company2 | 07-01-2023 | 50 | 50 | 1400 | 2000 | 50 | 4950 | 1237.5000000000000000 | 4 + company2 | 07-02-2023 | 2000 | | | | | | | 0 + company2 | 07-03-2023 | 1500 | | | | | | | 0 + company2 | 07-04-2023 | 1400 | 1400 | 60 | 1500 | 60 | 2960 | 986.6666666666666667 | 3 + company2 | 07-05-2023 | 1500 | | | | | | | 0 + company2 | 07-06-2023 | 60 | 60 | 1200 | 1300 | 60 | 3660 | 915.0000000000000000 | 4 + company2 | 07-07-2023 | 1100 | 1100 | 1200 | 1300 | 1100 | 3600 | 1200.0000000000000000 | 3 + company2 | 07-08-2023 | 1300 | | | | | | | 0 + company2 | 07-09-2023 | 1200 | | | | | | | 0 + company2 | 07-10-2023 | 1300 | | | | | | | 0 +(20 rows) + +-- JOIN case +CREATE TEMP TABLE t1 (i int, v1 int); +CREATE TEMP TABLE t2 (j int, v2 int); +INSERT INTO t1 VALUES(1,10); +INSERT INTO t1 VALUES(1,11); +INSERT INTO t1 VALUES(1,12); +INSERT INTO t2 VALUES(2,10); +INSERT INTO t2 VALUES(2,11); +INSERT INTO t2 VALUES(2,12); +SELECT * FROM t1, t2 WHERE t1.v1 <= 11 AND t2.v2 <= 11; + i | v1 | j | v2 +---+----+---+---- + 1 | 10 | 2 | 10 + 1 | 10 | 2 | 11 + 1 | 11 | 2 | 10 + 1 | 11 | 2 | 11 +(4 rows) + +SELECT *, count(*) OVER w FROM t1, t2 +WINDOW w AS ( + PARTITION BY t1.i + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A) + DEFINE + A AS v1 <= 11 AND v2 <= 11 +); + i | v1 | j | v2 | count +---+----+---+----+------- + 1 | 10 | 2 | 10 | 1 + 1 | 10 | 2 | 11 | 1 + 1 | 10 | 2 | 12 | 0 + 1 | 11 | 2 | 10 | 1 + 1 | 11 | 2 | 11 | 1 + 1 | 11 | 2 | 12 | 0 + 1 | 12 | 2 | 10 | 0 + 1 | 12 | 2 | 11 | 0 + 1 | 12 | 2 | 12 | 0 +(9 rows) + +-- WITH case +WITH wstock AS ( + SELECT * FROM stock WHERE tdate < '2023-07-08' +) +SELECT tdate, price, +first_value(tdate) OVER w, +count(*) OVER w + FROM wstock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + tdate | price | first_value | count +------------+-------+-------------+------- + 07-01-2023 | 100 | 07-01-2023 | 4 + 07-02-2023 | 200 | | 0 + 07-03-2023 | 150 | | 0 + 07-04-2023 | 140 | | 0 + 07-05-2023 | 150 | | 0 + 07-06-2023 | 90 | | 0 + 07-07-2023 | 110 | | 0 + 07-01-2023 | 50 | 07-01-2023 | 4 + 07-02-2023 | 2000 | | 0 + 07-03-2023 | 1500 | | 0 + 07-04-2023 | 1400 | | 0 + 07-05-2023 | 1500 | | 0 + 07-06-2023 | 60 | | 0 + 07-07-2023 | 1100 | | 0 +(14 rows) + +-- PREV has multiple column reference +CREATE TEMP TABLE rpr1 (id INTEGER, i SERIAL, j INTEGER); +INSERT INTO rpr1(id, j) SELECT 1, g*2 FROM generate_series(1, 10) AS g; +SELECT id, i, j, count(*) OVER w + FROM rpr1 + WINDOW w AS ( + PARTITION BY id + ORDER BY i + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + INITIAL + PATTERN (START COND+) + DEFINE + START AS TRUE, + COND AS PREV(i + j + 1) < 10 +); + id | i | j | count +----+----+----+------- + 1 | 1 | 2 | 3 + 1 | 2 | 4 | 0 + 1 | 3 | 6 | 0 + 1 | 4 | 8 | 0 + 1 | 5 | 10 | 0 + 1 | 6 | 12 | 0 + 1 | 7 | 14 | 0 + 1 | 8 | 16 | 0 + 1 | 9 | 18 | 0 + 1 | 10 | 20 | 0 +(10 rows) + +-- Smoke test for larger partitions. +WITH s AS ( + SELECT v, count(*) OVER w AS c + FROM (SELECT generate_series(1, 5000) v) + WINDOW w AS ( + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + INITIAL + PATTERN ( r+ ) + DEFINE r AS TRUE + ) +) +-- Should be exactly one long match across all rows. +SELECT * FROM s WHERE c > 0; + v | c +---+------ + 1 | 5000 +(1 row) + +WITH s AS ( + SELECT v, count(*) OVER w AS c + FROM (SELECT generate_series(1, 5000) v) + WINDOW w AS ( + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + INITIAL + PATTERN ( r ) + DEFINE r AS TRUE + ) +) +-- Every row should be its own match. +SELECT count(*) FROM s WHERE c > 0; + count +------- + 5000 +(1 row) + +-- View and pg_get_viewdef tests. +CREATE TEMP VIEW v_window AS +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w, + nth_value(tdate, 2) OVER w AS nth_second + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); +SELECT * FROM v_window; + company | tdate | price | first_value | last_value | nth_second +----------+------------+-------+-------------+------------+------------ + company1 | 07-01-2023 | 100 | 100 | 140 | 07-02-2023 + company1 | 07-02-2023 | 200 | | | + company1 | 07-03-2023 | 150 | | | + company1 | 07-04-2023 | 140 | | | + company1 | 07-05-2023 | 150 | | | + company1 | 07-06-2023 | 90 | 90 | 120 | 07-07-2023 + company1 | 07-07-2023 | 110 | | | + company1 | 07-08-2023 | 130 | | | + company1 | 07-09-2023 | 120 | | | + company1 | 07-10-2023 | 130 | | | + company2 | 07-01-2023 | 50 | 50 | 1400 | 07-02-2023 + company2 | 07-02-2023 | 2000 | | | + company2 | 07-03-2023 | 1500 | | | + company2 | 07-04-2023 | 1400 | | | + company2 | 07-05-2023 | 1500 | | | + company2 | 07-06-2023 | 60 | 60 | 1200 | 07-07-2023 + company2 | 07-07-2023 | 1100 | | | + company2 | 07-08-2023 | 1300 | | | + company2 | 07-09-2023 | 1200 | | | + company2 | 07-10-2023 | 1300 | | | +(20 rows) + +SELECT pg_get_viewdef('v_window'); + pg_get_viewdef +--------------------------------------------------------------------------------------- + SELECT company, + + tdate, + + price, + + first_value(price) OVER w AS first_value, + + last_value(price) OVER w AS last_value, + + nth_value(tdate, 2) OVER w AS nth_second + + FROM stock + + WINDOW w AS (PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + + AFTER MATCH SKIP PAST LAST ROW + + INITIAL + + PATTERN (start up+ down+) + + DEFINE + + start AS true, + + up AS (price > prev(price)), + + down AS (price < prev(price)) ); +(1 row) + +-- +-- Pattern optimization tests +-- VIEW shows original pattern, EXPLAIN shows optimized pattern +-- +-- Test: duplicate alternatives removal (A | B | A)+ -> (A | B)+ +CREATE TEMP VIEW v_opt_dup AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN ((A | B | A)+) + DEFINE + A AS price > 100, + B AS price <= 100 +); +SELECT pg_get_viewdef('v_opt_dup'); -- original: ((a | b | a)+) + pg_get_viewdef +--------------------------------------------------------------------------------------- + SELECT company, + + tdate, + + price, + + count(*) OVER w AS count + + FROM stock + + WINDOW w AS (PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + + AFTER MATCH SKIP PAST LAST ROW + + INITIAL + + PATTERN ((a | b | a)+) + + DEFINE + + a AS (price > 100), + + b AS (price <= 100) ); +(1 row) + +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_dup; -- optimized: ((a | b)+) + QUERY PLAN +---------------------------------------------------------------------------------------------------- + Subquery Scan on v_opt_dup + -> WindowAgg + Window: w AS (PARTITION BY stock.company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) + Pattern: ((a | b))+ + -> Sort + Sort Key: stock.company + -> Seq Scan on stock +(7 rows) + +-- Test: duplicate group removal ((A | B)+ | (A | B)+) -> (A | B)+ +CREATE TEMP VIEW v_opt_dup_group AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN ((A | B)+ | (A | B)+) + DEFINE + A AS price > 100, + B AS price <= 100 +); +SELECT pg_get_viewdef('v_opt_dup_group'); -- original: ((a | b)+ | (a | b)+) + pg_get_viewdef +--------------------------------------------------------------------------------------- + SELECT company, + + tdate, + + price, + + count(*) OVER w AS count + + FROM stock + + WINDOW w AS (PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + + AFTER MATCH SKIP PAST LAST ROW + + INITIAL + + PATTERN ((a | b)+ | (a | b)+) + + DEFINE + + a AS (price > 100), + + b AS (price <= 100) ); +(1 row) + +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_dup_group; -- optimized: ((a | b)+) + QUERY PLAN +---------------------------------------------------------------------------------------------------- + Subquery Scan on v_opt_dup_group + -> WindowAgg + Window: w AS (PARTITION BY stock.company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) + Pattern: ((a | b))+ + -> Sort + Sort Key: stock.company + -> Seq Scan on stock +(7 rows) + +-- Test: consecutive vars merge (A A A) -> A{3} +CREATE TEMP VIEW v_opt_merge AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A A A) + DEFINE + A AS price >= 140 AND price <= 150 +); +SELECT pg_get_viewdef('v_opt_merge'); -- original: (a a a) + pg_get_viewdef +--------------------------------------------------------------------------------------- + SELECT company, + + tdate, + + price, + + count(*) OVER w AS count + + FROM stock + + WINDOW w AS (PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + + AFTER MATCH SKIP PAST LAST ROW + + INITIAL + + PATTERN (a a a) + + DEFINE + + a AS ((price >= 140) AND (price <= 150)) ); +(1 row) + +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_merge; -- optimized: a{3} + QUERY PLAN +---------------------------------------------------------------------------------------------------- + Subquery Scan on v_opt_merge + -> WindowAgg + Window: w AS (PARTITION BY stock.company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) + Pattern: a{3} + -> Sort + Sort Key: stock.company + -> Seq Scan on stock +(7 rows) + +-- Test: quantified vars merge (A A+ A) -> A{3,} +CREATE TEMP VIEW v_opt_merge_quant AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A A+ A) + DEFINE + A AS price > 100 +); +SELECT pg_get_viewdef('v_opt_merge_quant'); -- original: (a a+ a) + pg_get_viewdef +--------------------------------------------------------------------------------------- + SELECT company, + + tdate, + + price, + + count(*) OVER w AS count + + FROM stock + + WINDOW w AS (PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + + AFTER MATCH SKIP PAST LAST ROW + + INITIAL + + PATTERN (a a+ a) + + DEFINE + + a AS (price > 100) ); +(1 row) + +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_merge_quant; -- optimized: a{3,} + QUERY PLAN +---------------------------------------------------------------------------------------------------- + Subquery Scan on v_opt_merge_quant + -> WindowAgg + Window: w AS (PARTITION BY stock.company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) + Pattern: a{3,} + -> Sort + Sort Key: stock.company + -> Seq Scan on stock +(7 rows) + +-- Test: merge two unbounded (A+ A+) -> A{2,} +CREATE TEMP VIEW v_opt_merge_unbounded AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A+ A+) + DEFINE + A AS price > 100 +); +SELECT pg_get_viewdef('v_opt_merge_unbounded'); -- original: (a+ a+) + pg_get_viewdef +--------------------------------------------------------------------------------------- + SELECT company, + + tdate, + + price, + + count(*) OVER w AS count + + FROM stock + + WINDOW w AS (PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + + AFTER MATCH SKIP PAST LAST ROW + + INITIAL + + PATTERN (a+ a+) + + DEFINE + + a AS (price > 100) ); +(1 row) + +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_merge_unbounded; -- optimized: a{2,} + QUERY PLAN +---------------------------------------------------------------------------------------------------- + Subquery Scan on v_opt_merge_unbounded + -> WindowAgg + Window: w AS (PARTITION BY stock.company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) + Pattern: a{2,} + -> Sort + Sort Key: stock.company + -> Seq Scan on stock +(7 rows) + +-- Test: merge with zero-min (A* A+) -> A+ +CREATE TEMP VIEW v_opt_merge_star AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A* A+) + DEFINE + A AS price > 100 +); +SELECT pg_get_viewdef('v_opt_merge_star'); -- original: (a* a+) + pg_get_viewdef +--------------------------------------------------------------------------------------- + SELECT company, + + tdate, + + price, + + count(*) OVER w AS count + + FROM stock + + WINDOW w AS (PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + + AFTER MATCH SKIP PAST LAST ROW + + INITIAL + + PATTERN (a* a+) + + DEFINE + + a AS (price > 100) ); +(1 row) + +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_merge_star; -- optimized: a+ + QUERY PLAN +---------------------------------------------------------------------------------------------------- + Subquery Scan on v_opt_merge_star + -> WindowAgg + Window: w AS (PARTITION BY stock.company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) + Pattern: a+ + -> Sort + Sort Key: stock.company + -> Seq Scan on stock +(7 rows) + +-- Test: complex merge (A A{2} A+ A{3}) -> A{7,} +CREATE TEMP VIEW v_opt_merge_complex AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A A{2} A+ A{3}) + DEFINE + A AS price > 100 +); +SELECT pg_get_viewdef('v_opt_merge_complex'); -- original: (a a{2} a+ a{3}) + pg_get_viewdef +--------------------------------------------------------------------------------------- + SELECT company, + + tdate, + + price, + + count(*) OVER w AS count + + FROM stock + + WINDOW w AS (PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + + AFTER MATCH SKIP PAST LAST ROW + + INITIAL + + PATTERN (a a{2} a+ a{3}) + + DEFINE + + a AS (price > 100) ); +(1 row) + +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_merge_complex; -- optimized: a{7,} + QUERY PLAN +---------------------------------------------------------------------------------------------------- + Subquery Scan on v_opt_merge_complex + -> WindowAgg + Window: w AS (PARTITION BY stock.company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) + Pattern: a{7,} + -> Sort + Sort Key: stock.company + -> Seq Scan on stock +(7 rows) + +-- Test: group merge ((A B) (A B)+) -> (A B){2,} +CREATE TEMP VIEW v_opt_merge_group AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN ((A B) (A B)+) + DEFINE + A AS price > 100, + B AS price <= 100 +); +SELECT pg_get_viewdef('v_opt_merge_group'); -- original: ((a b) (a b)+) + pg_get_viewdef +--------------------------------------------------------------------------------------- + SELECT company, + + tdate, + + price, + + count(*) OVER w AS count + + FROM stock + + WINDOW w AS (PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + + AFTER MATCH SKIP PAST LAST ROW + + INITIAL + + PATTERN ((a b) (a b)+) + + DEFINE + + a AS (price > 100), + + b AS (price <= 100) ); +(1 row) + +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_merge_group; -- expected: (a b){2,} + QUERY PLAN +---------------------------------------------------------------------------------------------------- + Subquery Scan on v_opt_merge_group + -> WindowAgg + Window: w AS (PARTITION BY stock.company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) + Pattern: (a b){2,} + -> Sort + Sort Key: stock.company + -> Seq Scan on stock +(7 rows) + +-- Test: group merge A B (A B)+ -> (A B){2,} +CREATE TEMP VIEW v_opt_merge_group2 AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A B (A B)+) + DEFINE + A AS price > 100, + B AS price <= 100 +); +SELECT pg_get_viewdef('v_opt_merge_group2'); -- original: (a b (a b)+) + pg_get_viewdef +--------------------------------------------------------------------------------------- + SELECT company, + + tdate, + + price, + + count(*) OVER w AS count + + FROM stock + + WINDOW w AS (PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + + AFTER MATCH SKIP PAST LAST ROW + + INITIAL + + PATTERN (a b (a b)+) + + DEFINE + + a AS (price > 100), + + b AS (price <= 100) ); +(1 row) + +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_merge_group2; -- expected: (a b){2,} + QUERY PLAN +---------------------------------------------------------------------------------------------------- + Subquery Scan on v_opt_merge_group2 + -> WindowAgg + Window: w AS (PARTITION BY stock.company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) + Pattern: (a b){2,} + -> Sort + Sort Key: stock.company + -> Seq Scan on stock +(7 rows) + +-- Test: group merge (A B) (A B)+ (A B) -> (A B){3,} +CREATE TEMP VIEW v_opt_merge_group3 AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN ((A B) (A B)+ (A B)) + DEFINE + A AS price > 100, + B AS price <= 100 +); +SELECT pg_get_viewdef('v_opt_merge_group3'); -- original: ((a b) (a b)+ (a b)) + pg_get_viewdef +--------------------------------------------------------------------------------------- + SELECT company, + + tdate, + + price, + + count(*) OVER w AS count + + FROM stock + + WINDOW w AS (PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + + AFTER MATCH SKIP PAST LAST ROW + + INITIAL + + PATTERN ((a b) (a b)+ (a b)) + + DEFINE + + a AS (price > 100), + + b AS (price <= 100) ); +(1 row) + +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_merge_group3; -- expected: (a b){3,} + QUERY PLAN +---------------------------------------------------------------------------------------------------- + Subquery Scan on v_opt_merge_group3 + -> WindowAgg + Window: w AS (PARTITION BY stock.company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) + Pattern: (a b){3,} + -> Sort + Sort Key: stock.company + -> Seq Scan on stock +(7 rows) + +-- Test: group merge A B A B (A B)+ A B A B -> (A B){5,} +CREATE TEMP VIEW v_opt_merge_group4 AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A B A B (A B)+ A B A B) + DEFINE + A AS price > 100, + B AS price <= 100 +); +SELECT pg_get_viewdef('v_opt_merge_group4'); -- original: (a b a b (a b)+ a b a b) + pg_get_viewdef +--------------------------------------------------------------------------------------- + SELECT company, + + tdate, + + price, + + count(*) OVER w AS count + + FROM stock + + WINDOW w AS (PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + + AFTER MATCH SKIP PAST LAST ROW + + INITIAL + + PATTERN (a b a b (a b)+ a b a b) + + DEFINE + + a AS (price > 100), + + b AS (price <= 100) ); +(1 row) + +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_merge_group4; -- expected: (a b){5,} + QUERY PLAN +---------------------------------------------------------------------------------------------------- + Subquery Scan on v_opt_merge_group4 + -> WindowAgg + Window: w AS (PARTITION BY stock.company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) + Pattern: (a b){5,} + -> Sort + Sort Key: stock.company + -> Seq Scan on stock +(7 rows) + +-- Test: group merge C A B (A B)+ A B C -> C (A B){3,} C +CREATE TEMP VIEW v_opt_merge_group5 AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (C A B (A B)+ A B C) + DEFINE + A AS price > 100, + B AS price <= 100, + C AS price > 200 +); +SELECT pg_get_viewdef('v_opt_merge_group5'); -- original: (c a b (a b)+ a b c) + pg_get_viewdef +--------------------------------------------------------------------------------------- + SELECT company, + + tdate, + + price, + + count(*) OVER w AS count + + FROM stock + + WINDOW w AS (PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + + AFTER MATCH SKIP PAST LAST ROW + + INITIAL + + PATTERN (c a b (a b)+ a b c) + + DEFINE + + a AS (price > 100), + + b AS (price <= 100), + + c AS (price > 200) ); +(1 row) + +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_merge_group5; -- expected: c (a b){3,} c + QUERY PLAN +---------------------------------------------------------------------------------------------------- + Subquery Scan on v_opt_merge_group5 + -> WindowAgg + Window: w AS (PARTITION BY stock.company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) + Pattern: c (a b){3,} c + -> Sort + Sort Key: stock.company + -> Seq Scan on stock +(7 rows) + +-- Test {n} quantifier display +CREATE TEMP VIEW v_quantifier_n AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A{3}) + DEFINE + A AS price > 100 +); +SELECT pg_get_viewdef('v_quantifier_n'); + pg_get_viewdef +--------------------------------------------------------------------------------------- + SELECT company, + + tdate, + + price, + + count(*) OVER w AS count + + FROM stock + + WINDOW w AS (PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + + AFTER MATCH SKIP PAST LAST ROW + + INITIAL + + PATTERN (a{3}) + + DEFINE + + a AS (price > 100) ); +(1 row) + +-- Test {n,} quantifier display +CREATE TEMP VIEW v_quantifier_n_plus AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A{2,}) + DEFINE + A AS price > 100 +); +SELECT pg_get_viewdef('v_quantifier_n_plus'); + pg_get_viewdef +--------------------------------------------------------------------------------------- + SELECT company, + + tdate, + + price, + + count(*) OVER w AS count + + FROM stock + + WINDOW w AS (PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + + AFTER MATCH SKIP PAST LAST ROW + + INITIAL + + PATTERN (a{2,}) + + DEFINE + + a AS (price > 100) ); +(1 row) + +-- Test: flatten nested SEQ (A (B C)) -> A B C +CREATE TEMP VIEW v_opt_flatten_seq AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A (B C)) + DEFINE + A AS price > 100, + B AS price > 150, + C AS price < 150 +); +SELECT pg_get_viewdef('v_opt_flatten_seq'); -- original: (a (b c)) + pg_get_viewdef +--------------------------------------------------------------------------------------- + SELECT company, + + tdate, + + price, + + count(*) OVER w AS count + + FROM stock + + WINDOW w AS (PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + + AFTER MATCH SKIP PAST LAST ROW + + INITIAL + + PATTERN (a (b c)) + + DEFINE + + a AS (price > 100), + + b AS (price > 150), + + c AS (price < 150) ); +(1 row) + +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_flatten_seq; -- optimized: a b c + QUERY PLAN +---------------------------------------------------------------------------------------------------- + Subquery Scan on v_opt_flatten_seq + -> WindowAgg + Window: w AS (PARTITION BY stock.company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) + Pattern: a b c + -> Sort + Sort Key: stock.company + -> Seq Scan on stock +(7 rows) + +-- Test: flatten nested ALT (A | (B | C)) -> (A | B | C) +CREATE TEMP VIEW v_opt_flatten_alt AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN ((A | (B | C))+) + DEFINE + A AS price > 200, + B AS price > 100, + C AS price <= 100 +); +SELECT pg_get_viewdef('v_opt_flatten_alt'); -- original: ((a | (b | c))+) + pg_get_viewdef +--------------------------------------------------------------------------------------- + SELECT company, + + tdate, + + price, + + count(*) OVER w AS count + + FROM stock + + WINDOW w AS (PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + + AFTER MATCH SKIP PAST LAST ROW + + INITIAL + + PATTERN ((a | (b | c))+) + + DEFINE + + a AS (price > 200), + + b AS (price > 100), + + c AS (price <= 100) ); +(1 row) + +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_flatten_alt; -- optimized: ((a | b | c))+ + QUERY PLAN +---------------------------------------------------------------------------------------------------- + Subquery Scan on v_opt_flatten_alt + -> WindowAgg + Window: w AS (PARTITION BY stock.company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) + Pattern: ((a | b | c))+ + -> Sort + Sort Key: stock.company + -> Seq Scan on stock +(7 rows) + +-- Test: unwrap GROUP{1,1} ((A)) -> A +CREATE TEMP VIEW v_opt_unwrap_group AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (((A))) + DEFINE + A AS price > 100 +); +SELECT pg_get_viewdef('v_opt_unwrap_group'); -- original: (((a))) + pg_get_viewdef +--------------------------------------------------------------------------------------- + SELECT company, + + tdate, + + price, + + count(*) OVER w AS count + + FROM stock + + WINDOW w AS (PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + + AFTER MATCH SKIP PAST LAST ROW + + INITIAL + + PATTERN (((a))) + + DEFINE + + a AS (price > 100) ); +(1 row) + +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_unwrap_group; -- optimized: a + QUERY PLAN +---------------------------------------------------------------------------------------------------- + Subquery Scan on v_opt_unwrap_group + -> WindowAgg + Window: w AS (PARTITION BY stock.company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) + Pattern: a + -> Sort + Sort Key: stock.company + -> Seq Scan on stock +(7 rows) + +-- Test: quantifier multiplication (A{2}){3} -> A{6} +CREATE TEMP VIEW v_opt_quant_mult AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN ((A{2}){3}) + DEFINE + A AS price > 100 +); +SELECT pg_get_viewdef('v_opt_quant_mult'); -- original: ((a{2}){3}) + pg_get_viewdef +--------------------------------------------------------------------------------------- + SELECT company, + + tdate, + + price, + + count(*) OVER w AS count + + FROM stock + + WINDOW w AS (PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + + AFTER MATCH SKIP PAST LAST ROW + + INITIAL + + PATTERN ((a{2}){3}) + + DEFINE + + a AS (price > 100) ); +(1 row) + +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_quant_mult; -- optimized: a{6} + QUERY PLAN +---------------------------------------------------------------------------------------------------- + Subquery Scan on v_opt_quant_mult + -> WindowAgg + Window: w AS (PARTITION BY stock.company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) + Pattern: a{6} + -> Sort + Sort Key: stock.company + -> Seq Scan on stock +(7 rows) + +-- Test: quantifier multiplication (A{2,4}){3} -> A{6,12} +CREATE TEMP VIEW v_opt_quant_mult_range AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN ((A{2,4}){3}) + DEFINE + A AS price > 100 +); +SELECT pg_get_viewdef('v_opt_quant_mult_range'); -- original: ((a{2,4}){3}) + pg_get_viewdef +--------------------------------------------------------------------------------------- + SELECT company, + + tdate, + + price, + + count(*) OVER w AS count + + FROM stock + + WINDOW w AS (PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + + AFTER MATCH SKIP PAST LAST ROW + + INITIAL + + PATTERN ((a{2,4}){3}) + + DEFINE + + a AS (price > 100) ); +(1 row) + +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_quant_mult_range; -- optimized: a{6,12} + QUERY PLAN +---------------------------------------------------------------------------------------------------- + Subquery Scan on v_opt_quant_mult_range + -> WindowAgg + Window: w AS (PARTITION BY stock.company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) + Pattern: a{6,12} + -> Sort + Sort Key: stock.company + -> Seq Scan on stock +(7 rows) + +-- Test: quantifier multiplication (A{2}){3,5} -> A{6,10} +CREATE TEMP VIEW v_opt_quant_mult_range2 AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN ((A{2}){3,5}) + DEFINE + A AS price > 100 +); +SELECT pg_get_viewdef('v_opt_quant_mult_range2'); -- original: ((a{2}){3,5}) + pg_get_viewdef +--------------------------------------------------------------------------------------- + SELECT company, + + tdate, + + price, + + count(*) OVER w AS count + + FROM stock + + WINDOW w AS (PARTITION BY company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + + AFTER MATCH SKIP PAST LAST ROW + + INITIAL + + PATTERN ((a{2}){3,5}) + + DEFINE + + a AS (price > 100) ); +(1 row) + +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_quant_mult_range2; -- optimized: a{6,10} + QUERY PLAN +---------------------------------------------------------------------------------------------------- + Subquery Scan on v_opt_quant_mult_range2 + -> WindowAgg + Window: w AS (PARTITION BY stock.company ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) + Pattern: a{6,10} + -> Sort + Sort Key: stock.company + -> Seq Scan on stock +(7 rows) + +-- +-- Error cases +-- +-- row pattern definition variable name must not appear more than once +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price), + UP AS price > PREV(price) +); +ERROR: row pattern definition variable name "up" appears more than once in DEFINE clause +LINE 11: UP AS price > PREV(price), + ^ +-- subqueries in DEFINE clause are not supported +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START LOWPRICE) + DEFINE + START AS TRUE, + LOWPRICE AS price < (SELECT 100) +); +ERROR: cannot use subquery in DEFINE expression +LINE 11: LOWPRICE AS price < (SELECT 100) + ^ +-- aggregates in DEFINE clause are not supported +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START LOWPRICE) + DEFINE + START AS TRUE, + LOWPRICE AS price < count(*) +); +ERROR: aggregate functions are not allowed in DEFINE +LINE 11: LOWPRICE AS price < count(*) + ^ +-- FRAME must start at current row when row pattern recognition is used +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); +ERROR: FRAME must start at current row when row pattern recognition is used +-- SEEK is not supported +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + SEEK + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); +ERROR: SEEK is not supported +LINE 8: SEEK + ^ +HINT: Use INITIAL instead. +-- PREV's argument must have at least 1 column reference +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(1), + DOWN AS price < PREV(1) +); +ERROR: row pattern navigation operation's argument must include at least one column reference +-- Unsupported quantifier +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + INITIAL + PATTERN (START UP~ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(1), + DOWN AS price < PREV(1) +); +ERROR: unsupported quantifier "~" +LINE 9: PATTERN (START UP~ DOWN+) + ^ +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + INITIAL + PATTERN (START UP+? DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(1), + DOWN AS price < PREV(1) +); +ERROR: row pattern navigation operation's argument must include at least one column reference +-- Number of row pattern definition variable names must not exceed 26 +-- Ok +SELECT * FROM (SELECT 1 AS x) t + WINDOW w AS ( + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + PATTERN (a b c d e f g h i j k l m n o p q r s t u v w x y z) + DEFINE a AS TRUE +); + x +--- + 1 +(1 row) + +-- Error +SELECT * FROM (SELECT 1 AS x) t + WINDOW w AS ( + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + PATTERN (a b c d e f g h i j k l m n o p q r s t u v w x y z aa) + DEFINE a AS TRUE +); +ERROR: number of row pattern definition variable names exceeds 26 + CREATE TEMP TABLE stock_null (company TEXT, tdate DATE, price INTEGER); + INSERT INTO stock_null VALUES ('c1', '2023-07-01', 100); + INSERT INTO stock_null VALUES ('c1', '2023-07-02', NULL); -- NULL in middle + INSERT INTO stock_null VALUES ('c1', '2023-07-03', 200); + INSERT INTO stock_null VALUES ('c1', '2023-07-04', 150); + SELECT company, tdate, price, count(*) OVER w AS match_count + FROM stock_null + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + PATTERN (START UP DOWN) + DEFINE START AS TRUE, UP AS price > PREV(price), DOWN AS price < +PREV(price) + ); + company | tdate | price | match_count +---------+------------+-------+------------- + c1 | 07-01-2023 | 100 | 0 + c1 | 07-02-2023 | | 0 + c1 | 07-03-2023 | 200 | 0 + c1 | 07-04-2023 | 150 | 0 +(4 rows) + +-- Overlapping match tests (requires multi-context for correct behavior) +-- Using array flags: 'X' = ANY(flags) for multi-TRUE support +-- Test 1: A B C D E | B C D | C D E F - three overlapping patterns +-- Different end points: B C D (4), A B C D E (5), C D E F (6) +WITH test_overlap1 AS ( + SELECT * FROM (VALUES + (1, ARRAY['A']), + (2, ARRAY['B']), + (3, ARRAY['C']), + (4, ARRAY['D']), + (5, ARRAY['E']), + (6, ARRAY['F']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_overlap1 +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + PATTERN (A B C D E | B C D | C D E F) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags), + C AS 'C' = ANY(flags), + D AS 'D' = ANY(flags), + E AS 'E' = ANY(flags), + F AS 'F' = ANY(flags) +); + id | flags | match_start | match_end +----+-------+-------------+----------- + 1 | {A} | 1 | 5 + 2 | {B} | | + 3 | {C} | | + 4 | {D} | | + 5 | {E} | | + 6 | {F} | | +(6 rows) + +WITH test_overlap1 AS ( + SELECT * FROM (VALUES + (1, ARRAY['A']), + (2, ARRAY['B']), + (3, ARRAY['C']), + (4, ARRAY['D']), + (5, ARRAY['E']), + (6, ARRAY['F']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_overlap1 +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + PATTERN (A B C D E | B C D | C D E F) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags), + C AS 'C' = ANY(flags), + D AS 'D' = ANY(flags), + E AS 'E' = ANY(flags), + F AS 'F' = ANY(flags) +); + id | flags | match_start | match_end +----+-------+-------------+----------- + 1 | {A} | 1 | 5 + 2 | {B} | 2 | 4 + 3 | {C} | 3 | 6 + 4 | {D} | | + 5 | {E} | | + 6 | {F} | | +(6 rows) + +-- PAST LAST: only one match +-- TO NEXT ROW with multi-context: three matches +-- Row 1: A B C D E (1-5) +-- Row 2: B C D (2-4) <- ends first! +-- Row 3: C D E F (3-6) <- ends last! +-- Test 2: A B+ C | B+ D - long B sequence with different endings +WITH test_overlap2 AS ( + SELECT * FROM (VALUES + (1, ARRAY['A']), + (2, ARRAY['B']), + (3, ARRAY['B']), + (4, ARRAY['B']), + (5, ARRAY['B']), + (6, ARRAY['C']), + (7, ARRAY['B']), + (8, ARRAY['B']), + (9, ARRAY['B']), + (10, ARRAY['D']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_overlap2 +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + PATTERN (A B+ C | B+ D) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags), + C AS 'C' = ANY(flags), + D AS 'D' = ANY(flags) +); + id | flags | match_start | match_end +----+-------+-------------+----------- + 1 | {A} | 1 | 6 + 2 | {B} | | + 3 | {B} | | + 4 | {B} | | + 5 | {B} | | + 6 | {C} | | + 7 | {B} | 7 | 10 + 8 | {B} | | + 9 | {B} | | + 10 | {D} | | +(10 rows) + +-- Current result (correct): +-- Row 1: A B+ C (1-6) +-- Row 7-9: B+ D (7-10, 8-10, 9-10) +-- Note: Row 2-6 cannot match B+ D because Row 6 is C, not D +-- With absorption: 8-10 and 9-10 would be absorbed by 7-10 (earlier context covers later) +-- Test 3: Greedy quantifier with late failure - A B C+ D | A B +-- Pattern expects D after C+, but E comes instead ("betrayal") +WITH test_betrayal AS ( + SELECT * FROM (VALUES + (1, ARRAY['A']), + (2, ARRAY['B']), + (3, ARRAY['C']), + (4, ARRAY['C']), + (5, ARRAY['C']), + (6, ARRAY['E']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_betrayal +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + PATTERN (A B C+ D | A B) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags), + C AS 'C' = ANY(flags), + D AS 'D' = ANY(flags) +); + id | flags | match_start | match_end +----+-------+-------------+----------- + 1 | {A} | 1 | 2 + 2 | {B} | | + 3 | {C} | | + 4 | {C} | | + 5 | {C} | | + 6 | {E} | | +(6 rows) + +-- A B C+ D fails at Row 6 (E instead of D) +-- Question: Does it fallback to A B (1-2)? +-- Test 4: Lexical Order test - A B C | A B C D E +-- SQL standard: first matching alternative wins +WITH test_lexical AS ( + SELECT * FROM (VALUES + (1, ARRAY['A']), + (2, ARRAY['B']), + (3, ARRAY['C']), + (4, ARRAY['D']), + (5, ARRAY['E']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_lexical +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + PATTERN (A B C | A B C D E) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags), + C AS 'C' = ANY(flags), + D AS 'D' = ANY(flags), + E AS 'E' = ANY(flags) +); + id | flags | match_start | match_end +----+-------+-------------+----------- + 1 | {A} | 1 | 3 + 2 | {B} | | + 3 | {C} | | + 4 | {D} | | + 5 | {E} | | +(5 rows) + +-- SQL standard Lexical Order: A B C (1-3) wins (first alternative) +-- Test 4b: Reversed pattern order - A B C D E | A B C +WITH test_lexical2 AS ( + SELECT * FROM (VALUES + (1, ARRAY['A']), + (2, ARRAY['B']), + (3, ARRAY['C']), + (4, ARRAY['D']), + (5, ARRAY['E']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_lexical2 +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + PATTERN (A B C D E | A B C) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags), + C AS 'C' = ANY(flags), + D AS 'D' = ANY(flags), + E AS 'E' = ANY(flags) +); + id | flags | match_start | match_end +----+-------+-------------+----------- + 1 | {A} | 1 | 5 + 2 | {B} | | + 3 | {C} | | + 4 | {D} | | + 5 | {E} | | +(5 rows) + +-- SQL standard Lexical Order: A B C D E (1-5) wins (first alternative) +-- Test 5: Multiple TRUE in single row (overlapping pattern variables) +-- Each row matches multiple DEFINE conditions simultaneously +WITH test_multi_true AS ( + SELECT * FROM (VALUES + (1, ARRAY['A','B']), -- A and B both TRUE + (2, ARRAY['B','C']), -- B and C both TRUE + (3, ARRAY['C','D']), -- C and D both TRUE + (4, ARRAY['D','E']), -- D and E both TRUE + (5, ARRAY['E','_']) -- E only + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_multi_true +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + PATTERN (A B C D E) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags), + C AS 'C' = ANY(flags), + D AS 'D' = ANY(flags), + E AS 'E' = ANY(flags) +); + id | flags | match_start | match_end +----+-------+-------------+----------- + 1 | {A,B} | 1 | 5 + 2 | {B,C} | | + 3 | {C,D} | | + 4 | {D,E} | | + 5 | {E,_} | | +(5 rows) + +-- Row 1: A=T, B=T → matches A +-- Row 2: B=T, C=T → matches B +-- Row 3: C=T, D=T → matches C +-- Row 4: D=T, E=T → matches D +-- Row 5: E=T → matches E +-- Result: match 1-5 (A B C D E) +-- Test 6: Diagonal pattern with multi-TRUE (shifted overlap) +WITH test_diagonal AS ( + SELECT * FROM (VALUES + (1, ARRAY['A','_']), + (2, ARRAY['B','A']), + (3, ARRAY['C','B']), + (4, ARRAY['D','C']), + (5, ARRAY['_','D']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_diagonal +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + PATTERN (A B C D) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags), + C AS 'C' = ANY(flags), + D AS 'D' = ANY(flags) +); + id | flags | match_start | match_end +----+-------+-------------+----------- + 1 | {A,_} | 1 | 4 + 2 | {B,A} | 2 | 5 + 3 | {C,B} | | + 4 | {D,C} | | + 5 | {_,D} | | +(5 rows) + +-- Possible matches: +-- Start Row 1: A(1) B(2) C(3) D(4) → 1-4 +-- Start Row 2: A(2) B(3) C(4) D(5) → 2-5 (because Row 2 has A too!) +-- =================================================================== +-- Context Absorption Tests +-- =================================================================== +-- Test absorption 1: Basic A+ pattern - later contexts absorbed by earlier +WITH test_absorb_basic AS ( + SELECT * FROM (VALUES + (1, ARRAY['A']), + (2, ARRAY['A']), + (3, ARRAY['A']), + (4, ARRAY['A']), + (5, ARRAY['B']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_absorb_basic +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + PATTERN (A+) + DEFINE A AS 'A' = ANY(flags) +); + id | flags | match_start | match_end +----+-------+-------------+----------- + 1 | {A} | 1 | 4 + 2 | {A} | | + 3 | {A} | | + 4 | {A} | | + 5 | {B} | | +(5 rows) + +-- Pattern A+ is absorbable (unbounded first element, only one unbounded) +-- Without absorption: 4 matches (1-4, 2-4, 3-4, 4-4) +-- With absorption: Row 1 match (1-4), rows 2-4 absorbed +-- Test absorption 2: A+ B pattern - absorption with fixed suffix +WITH test_absorb_suffix AS ( + SELECT * FROM (VALUES + (1, ARRAY['A']), + (2, ARRAY['A']), + (3, ARRAY['A']), + (4, ARRAY['B']), + (5, ARRAY['X']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_absorb_suffix +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + PATTERN (A+ B) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags) +); + id | flags | match_start | match_end +----+-------+-------------+----------- + 1 | {A} | 1 | 4 + 2 | {A} | | + 3 | {A} | | + 4 | {B} | | + 5 | {X} | | +(5 rows) + +-- Pattern A+ B is absorbable (A+ unbounded first, B bounded suffix) +-- All potential matches end at same row (row 4 with B) +-- With absorption: Row 1 match (1-4), rows 2-3 absorbed +-- Test absorption 3: Per-branch absorption with ALT (B+ C | B+ D) +WITH test_absorb_alt AS ( + SELECT * FROM (VALUES + (1, ARRAY['B']), + (2, ARRAY['B']), + (3, ARRAY['B']), + (4, ARRAY['D']), + (5, ARRAY['X']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_absorb_alt +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + PATTERN (B+ C | B+ D) + DEFINE + B AS 'B' = ANY(flags), + C AS 'C' = ANY(flags), + D AS 'D' = ANY(flags) +); + id | flags | match_start | match_end +----+-------+-------------+----------- + 1 | {B} | 1 | 4 + 2 | {B} | | + 3 | {B} | | + 4 | {D} | | + 5 | {X} | | +(5 rows) + +-- Both branches B+ C and B+ D are absorbable (B+ unbounded first) +-- B+ D branch matches: potential 1-4, 2-4, 3-4 +-- Row 1 (1-4) absorbs Row 2 (2-4) and Row 3 (3-4) - same endpoint +-- Test absorption 4: Non-absorbable pattern (A B+ - unbounded not first) +WITH test_no_absorb AS ( + SELECT * FROM (VALUES + (1, ARRAY['A']), + (2, ARRAY['B']), + (3, ARRAY['B']), + (4, ARRAY['B']), + (5, ARRAY['X']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_no_absorb +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + PATTERN (A B+) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags) +); + id | flags | match_start | match_end +----+-------+-------------+----------- + 1 | {A} | 1 | 4 + 2 | {B} | | + 3 | {B} | | + 4 | {B} | | + 5 | {X} | | +(5 rows) + +-- Pattern A B+ is NOT absorbable (A bounded first, B+ unbounded but not first) +-- Only Row 1 can start match (only row with A), so only one match: 1-4 +-- Test absorption 5: GROUP merge enables absorption +WITH test_absorb_group AS ( + SELECT * FROM (VALUES + (1, ARRAY['A']), + (2, ARRAY['B']), + (3, ARRAY['A']), + (4, ARRAY['B']), + (5, ARRAY['A']), + (6, ARRAY['B']), + (7, ARRAY['X']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_absorb_group +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + PATTERN ((A B) (A B)+) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags) +); + id | flags | match_start | match_end +----+-------+-------------+----------- + 1 | {A} | 1 | 6 + 2 | {B} | | + 3 | {A} | | + 4 | {B} | | + 5 | {A} | | + 6 | {B} | | + 7 | {X} | | +(7 rows) + +-- Pattern optimized: (A B) (A B)+ -> (A B){2,} +-- Potential matches: 1-6 (3 reps), 3-6 (2 reps), 5-6 needs 2 reps (fail) +-- Row 1 (1-6) absorbs Row 3 (3-6) - same endpoint, top-level unbounded GROUP +-- Test absorption 6: Multiple unbounded - NOT absorbable +WITH test_multi_unbounded AS ( + SELECT * FROM (VALUES + (1, ARRAY['A']), + (2, ARRAY['A']), + (3, ARRAY['B']), + (4, ARRAY['B']), + (5, ARRAY['X']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_multi_unbounded +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + PATTERN (A+ B+) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags) +); + id | flags | match_start | match_end +----+-------+-------------+----------- + 1 | {A} | 1 | 4 + 2 | {A} | 2 | 4 + 3 | {B} | | + 4 | {B} | | + 5 | {X} | | +(5 rows) + +-- Pattern A+ B+ has TWO unbounded elements - NOT absorbable +-- Greedy A+ consumes rows 1-2, then B+ matches 3-4 +-- Row 1: 1-4, Row 2: 2-4 (different A+ counts, not absorbed) diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule index 021d57f66bb..f3f4ca7dfbe 100644 --- a/src/test/regress/parallel_schedule +++ b/src/test/regress/parallel_schedule @@ -102,7 +102,7 @@ test: publication subscription # Another group of parallel tests # select_views depends on create_view # ---------- -test: select_views portals_p2 foreign_key cluster dependency guc bitmapops combocid tsearch tsdicts foreign_data window xmlmap functional_deps advisory_lock indirect_toast equivclass stats_rewrite +test: select_views portals_p2 foreign_key cluster dependency guc bitmapops combocid tsearch tsdicts foreign_data window xmlmap functional_deps advisory_lock indirect_toast equivclass stats_rewrite rpr # ---------- # Another group of parallel tests (JSON related) diff --git a/src/test/regress/sql/rpr.sql b/src/test/regress/sql/rpr.sql new file mode 100644 index 00000000000..fa8aa50fcb9 --- /dev/null +++ b/src/test/regress/sql/rpr.sql @@ -0,0 +1,1394 @@ +-- +-- Test for row pattern definition clause +-- + +CREATE TEMP TABLE stock ( + company TEXT, + tdate DATE, + price INTEGER +); +INSERT INTO stock VALUES ('company1', '2023-07-01', 100); +INSERT INTO stock VALUES ('company1', '2023-07-02', 200); +INSERT INTO stock VALUES ('company1', '2023-07-03', 150); +INSERT INTO stock VALUES ('company1', '2023-07-04', 140); +INSERT INTO stock VALUES ('company1', '2023-07-05', 150); +INSERT INTO stock VALUES ('company1', '2023-07-06', 90); +INSERT INTO stock VALUES ('company1', '2023-07-07', 110); +INSERT INTO stock VALUES ('company1', '2023-07-08', 130); +INSERT INTO stock VALUES ('company1', '2023-07-09', 120); +INSERT INTO stock VALUES ('company1', '2023-07-10', 130); +INSERT INTO stock VALUES ('company2', '2023-07-01', 50); +INSERT INTO stock VALUES ('company2', '2023-07-02', 2000); +INSERT INTO stock VALUES ('company2', '2023-07-03', 1500); +INSERT INTO stock VALUES ('company2', '2023-07-04', 1400); +INSERT INTO stock VALUES ('company2', '2023-07-05', 1500); +INSERT INTO stock VALUES ('company2', '2023-07-06', 60); +INSERT INTO stock VALUES ('company2', '2023-07-07', 1100); +INSERT INTO stock VALUES ('company2', '2023-07-08', 1300); +INSERT INTO stock VALUES ('company2', '2023-07-09', 1200); +INSERT INTO stock VALUES ('company2', '2023-07-10', 1300); + +SELECT * FROM stock; + +-- basic test using PREV +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w, + nth_value(tdate, 2) OVER w AS nth_second + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- basic test using PREV. UP appears twice +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w, + nth_value(tdate, 2) OVER w AS nth_second + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+ UP+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- basic test using PREV. Use '*' +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w, + nth_value(tdate, 2) OVER w AS nth_second + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP* DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- basic test using PREV. Use '?' +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w, + nth_value(tdate, 2) OVER w AS nth_second + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP? DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- test using alternation (|) with sequence +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START (UP | DOWN)) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- test using alternation (|) with group quantifier +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START (UP | DOWN)+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- test using nested alternation +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START ((UP DOWN) | FLAT)+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price), + FLAT AS price = PREV(price) +); + +-- test using group with quantifier +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN ((UP DOWN)+) + DEFINE + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- test using absolute threshold values (not relative PREV) +-- HIGH: price > 150, LOW: price < 100, MID: neutral range +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (LOW MID* HIGH) + DEFINE + LOW AS price < 100, + MID AS price >= 100 AND price <= 150, + HIGH AS price > 150 +); + +-- test threshold-based pattern with alternation +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (LOW (MID | HIGH)+) + DEFINE + LOW AS price < 100, + MID AS price >= 100 AND price <= 150, + HIGH AS price > 150 +); + +-- basic test with none-greedy pattern +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A A A) + DEFINE + A AS price >= 140 AND price <= 150 +); + +-- test using {n} quantifier (A A A should be optimized to A{3}) +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A{3}) + DEFINE + A AS price >= 140 AND price <= 150 +); + +-- test using {n,} quantifier (2 or more) +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A{2,}) + DEFINE + A AS price > 100 +); + +-- test using {n,m} quantifier (2 to 4) +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A{2,4}) + DEFINE + A AS price > 100 +); + +-- last_value() should remain consistent +SELECT company, tdate, price, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- omit "START" in DEFINE but it is ok because "START AS TRUE" is +-- implicitly defined. per spec. +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w, + nth_value(tdate, 2) OVER w AS nth_second + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- the first row start with less than or equal to 100 +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (LOWPRICE UP+ DOWN+) + DEFINE + LOWPRICE AS price <= 100, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- second row raises 120% +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (LOWPRICE UP+ DOWN+) + DEFINE + LOWPRICE AS price <= 100, + UP AS price > PREV(price) * 1.2, + DOWN AS price < PREV(price) +); + +-- using NEXT +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UPDOWN) + DEFINE + START AS TRUE, + UPDOWN AS price > PREV(price) AND price > NEXT(price) +); + +-- using AFTER MATCH SKIP TO NEXT ROW +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + INITIAL + PATTERN (START UPDOWN) + DEFINE + START AS TRUE, + UPDOWN AS price > PREV(price) AND price > NEXT(price) +); + +-- match everything + +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + INITIAL + PATTERN (A+) + DEFINE + A AS TRUE +); + +-- backtracking with reclassification of rows +-- using AFTER MATCH SKIP PAST LAST ROW +SELECT company, tdate, price, first_value(tdate) OVER w, last_value(tdate) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + INITIAL + PATTERN (A+ B+) + DEFINE + A AS price > 100, + B AS price > 100 +); + +-- backtracking with reclassification of rows +-- using AFTER MATCH SKIP TO NEXT ROW +SELECT company, tdate, price, first_value(tdate) OVER w, last_value(tdate) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + INITIAL + PATTERN (A+ B+) + DEFINE + A AS price > 100, + B AS price > 100 +); + +-- ROWS BETWEEN CURRENT ROW AND offset FOLLOWING +SELECT company, tdate, price, first_value(tdate) OVER w, last_value(tdate) OVER w, + count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND 2 FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- +-- Aggregates +-- + +-- using AFTER MATCH SKIP PAST LAST ROW +SELECT company, tdate, price, + first_value(price) OVER w, + last_value(price) OVER w, + max(price) OVER w, + min(price) OVER w, + sum(price) OVER w, + avg(price) OVER w, + count(price) OVER w +FROM stock +WINDOW w AS ( +PARTITION BY company +ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING +AFTER MATCH SKIP PAST LAST ROW +INITIAL +PATTERN (START UP+ DOWN+) +DEFINE +START AS TRUE, +UP AS price > PREV(price), +DOWN AS price < PREV(price) +); + +-- using AFTER MATCH SKIP TO NEXT ROW +SELECT company, tdate, price, + first_value(price) OVER w, + last_value(price) OVER w, + max(price) OVER w, + min(price) OVER w, + sum(price) OVER w, + avg(price) OVER w, + count(price) OVER w +FROM stock +WINDOW w AS ( +PARTITION BY company +ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING +AFTER MATCH SKIP TO NEXT ROW +INITIAL +PATTERN (START UP+ DOWN+) +DEFINE +START AS TRUE, +UP AS price > PREV(price), +DOWN AS price < PREV(price) +); + +-- JOIN case +CREATE TEMP TABLE t1 (i int, v1 int); +CREATE TEMP TABLE t2 (j int, v2 int); +INSERT INTO t1 VALUES(1,10); +INSERT INTO t1 VALUES(1,11); +INSERT INTO t1 VALUES(1,12); +INSERT INTO t2 VALUES(2,10); +INSERT INTO t2 VALUES(2,11); +INSERT INTO t2 VALUES(2,12); + +SELECT * FROM t1, t2 WHERE t1.v1 <= 11 AND t2.v2 <= 11; + +SELECT *, count(*) OVER w FROM t1, t2 +WINDOW w AS ( + PARTITION BY t1.i + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A) + DEFINE + A AS v1 <= 11 AND v2 <= 11 +); + +-- WITH case +WITH wstock AS ( + SELECT * FROM stock WHERE tdate < '2023-07-08' +) +SELECT tdate, price, +first_value(tdate) OVER w, +count(*) OVER w + FROM wstock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- PREV has multiple column reference +CREATE TEMP TABLE rpr1 (id INTEGER, i SERIAL, j INTEGER); +INSERT INTO rpr1(id, j) SELECT 1, g*2 FROM generate_series(1, 10) AS g; +SELECT id, i, j, count(*) OVER w + FROM rpr1 + WINDOW w AS ( + PARTITION BY id + ORDER BY i + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + INITIAL + PATTERN (START COND+) + DEFINE + START AS TRUE, + COND AS PREV(i + j + 1) < 10 +); + +-- Smoke test for larger partitions. +WITH s AS ( + SELECT v, count(*) OVER w AS c + FROM (SELECT generate_series(1, 5000) v) + WINDOW w AS ( + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + INITIAL + PATTERN ( r+ ) + DEFINE r AS TRUE + ) +) +-- Should be exactly one long match across all rows. +SELECT * FROM s WHERE c > 0; + +WITH s AS ( + SELECT v, count(*) OVER w AS c + FROM (SELECT generate_series(1, 5000) v) + WINDOW w AS ( + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + INITIAL + PATTERN ( r ) + DEFINE r AS TRUE + ) +) +-- Every row should be its own match. +SELECT count(*) FROM s WHERE c > 0; + +-- View and pg_get_viewdef tests. +CREATE TEMP VIEW v_window AS +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w, + nth_value(tdate, 2) OVER w AS nth_second + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +SELECT * FROM v_window; +SELECT pg_get_viewdef('v_window'); + +-- +-- Pattern optimization tests +-- VIEW shows original pattern, EXPLAIN shows optimized pattern +-- + +-- Test: duplicate alternatives removal (A | B | A)+ -> (A | B)+ +CREATE TEMP VIEW v_opt_dup AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN ((A | B | A)+) + DEFINE + A AS price > 100, + B AS price <= 100 +); +SELECT pg_get_viewdef('v_opt_dup'); -- original: ((a | b | a)+) +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_dup; -- optimized: ((a | b)+) + +-- Test: duplicate group removal ((A | B)+ | (A | B)+) -> (A | B)+ +CREATE TEMP VIEW v_opt_dup_group AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN ((A | B)+ | (A | B)+) + DEFINE + A AS price > 100, + B AS price <= 100 +); +SELECT pg_get_viewdef('v_opt_dup_group'); -- original: ((a | b)+ | (a | b)+) +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_dup_group; -- optimized: ((a | b)+) + +-- Test: consecutive vars merge (A A A) -> A{3} +CREATE TEMP VIEW v_opt_merge AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A A A) + DEFINE + A AS price >= 140 AND price <= 150 +); +SELECT pg_get_viewdef('v_opt_merge'); -- original: (a a a) +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_merge; -- optimized: a{3} + +-- Test: quantified vars merge (A A+ A) -> A{3,} +CREATE TEMP VIEW v_opt_merge_quant AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A A+ A) + DEFINE + A AS price > 100 +); +SELECT pg_get_viewdef('v_opt_merge_quant'); -- original: (a a+ a) +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_merge_quant; -- optimized: a{3,} + +-- Test: merge two unbounded (A+ A+) -> A{2,} +CREATE TEMP VIEW v_opt_merge_unbounded AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A+ A+) + DEFINE + A AS price > 100 +); +SELECT pg_get_viewdef('v_opt_merge_unbounded'); -- original: (a+ a+) +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_merge_unbounded; -- optimized: a{2,} + +-- Test: merge with zero-min (A* A+) -> A+ +CREATE TEMP VIEW v_opt_merge_star AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A* A+) + DEFINE + A AS price > 100 +); +SELECT pg_get_viewdef('v_opt_merge_star'); -- original: (a* a+) +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_merge_star; -- optimized: a+ + +-- Test: complex merge (A A{2} A+ A{3}) -> A{7,} +CREATE TEMP VIEW v_opt_merge_complex AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A A{2} A+ A{3}) + DEFINE + A AS price > 100 +); +SELECT pg_get_viewdef('v_opt_merge_complex'); -- original: (a a{2} a+ a{3}) +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_merge_complex; -- optimized: a{7,} + +-- Test: group merge ((A B) (A B)+) -> (A B){2,} +CREATE TEMP VIEW v_opt_merge_group AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN ((A B) (A B)+) + DEFINE + A AS price > 100, + B AS price <= 100 +); +SELECT pg_get_viewdef('v_opt_merge_group'); -- original: ((a b) (a b)+) +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_merge_group; -- expected: (a b){2,} + +-- Test: group merge A B (A B)+ -> (A B){2,} +CREATE TEMP VIEW v_opt_merge_group2 AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A B (A B)+) + DEFINE + A AS price > 100, + B AS price <= 100 +); +SELECT pg_get_viewdef('v_opt_merge_group2'); -- original: (a b (a b)+) +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_merge_group2; -- expected: (a b){2,} + +-- Test: group merge (A B) (A B)+ (A B) -> (A B){3,} +CREATE TEMP VIEW v_opt_merge_group3 AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN ((A B) (A B)+ (A B)) + DEFINE + A AS price > 100, + B AS price <= 100 +); +SELECT pg_get_viewdef('v_opt_merge_group3'); -- original: ((a b) (a b)+ (a b)) +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_merge_group3; -- expected: (a b){3,} + +-- Test: group merge A B A B (A B)+ A B A B -> (A B){5,} +CREATE TEMP VIEW v_opt_merge_group4 AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A B A B (A B)+ A B A B) + DEFINE + A AS price > 100, + B AS price <= 100 +); +SELECT pg_get_viewdef('v_opt_merge_group4'); -- original: (a b a b (a b)+ a b a b) +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_merge_group4; -- expected: (a b){5,} + +-- Test: group merge C A B (A B)+ A B C -> C (A B){3,} C +CREATE TEMP VIEW v_opt_merge_group5 AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (C A B (A B)+ A B C) + DEFINE + A AS price > 100, + B AS price <= 100, + C AS price > 200 +); +SELECT pg_get_viewdef('v_opt_merge_group5'); -- original: (c a b (a b)+ a b c) +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_merge_group5; -- expected: c (a b){3,} c + +-- Test {n} quantifier display +CREATE TEMP VIEW v_quantifier_n AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A{3}) + DEFINE + A AS price > 100 +); +SELECT pg_get_viewdef('v_quantifier_n'); + +-- Test {n,} quantifier display +CREATE TEMP VIEW v_quantifier_n_plus AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A{2,}) + DEFINE + A AS price > 100 +); +SELECT pg_get_viewdef('v_quantifier_n_plus'); + +-- Test: flatten nested SEQ (A (B C)) -> A B C +CREATE TEMP VIEW v_opt_flatten_seq AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A (B C)) + DEFINE + A AS price > 100, + B AS price > 150, + C AS price < 150 +); +SELECT pg_get_viewdef('v_opt_flatten_seq'); -- original: (a (b c)) +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_flatten_seq; -- optimized: a b c + +-- Test: flatten nested ALT (A | (B | C)) -> (A | B | C) +CREATE TEMP VIEW v_opt_flatten_alt AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN ((A | (B | C))+) + DEFINE + A AS price > 200, + B AS price > 100, + C AS price <= 100 +); +SELECT pg_get_viewdef('v_opt_flatten_alt'); -- original: ((a | (b | c))+) +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_flatten_alt; -- optimized: ((a | b | c))+ + +-- Test: unwrap GROUP{1,1} ((A)) -> A +CREATE TEMP VIEW v_opt_unwrap_group AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (((A))) + DEFINE + A AS price > 100 +); +SELECT pg_get_viewdef('v_opt_unwrap_group'); -- original: (((a))) +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_unwrap_group; -- optimized: a + +-- Test: quantifier multiplication (A{2}){3} -> A{6} +CREATE TEMP VIEW v_opt_quant_mult AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN ((A{2}){3}) + DEFINE + A AS price > 100 +); +SELECT pg_get_viewdef('v_opt_quant_mult'); -- original: ((a{2}){3}) +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_quant_mult; -- optimized: a{6} + +-- Test: quantifier multiplication (A{2,4}){3} -> A{6,12} +CREATE TEMP VIEW v_opt_quant_mult_range AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN ((A{2,4}){3}) + DEFINE + A AS price > 100 +); +SELECT pg_get_viewdef('v_opt_quant_mult_range'); -- original: ((a{2,4}){3}) +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_quant_mult_range; -- optimized: a{6,12} + +-- Test: quantifier multiplication (A{2}){3,5} -> A{6,10} +CREATE TEMP VIEW v_opt_quant_mult_range2 AS +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN ((A{2}){3,5}) + DEFINE + A AS price > 100 +); +SELECT pg_get_viewdef('v_opt_quant_mult_range2'); -- original: ((a{2}){3,5}) +EXPLAIN (COSTS OFF) SELECT * FROM v_opt_quant_mult_range2; -- optimized: a{6,10} + +-- +-- Error cases +-- + +-- row pattern definition variable name must not appear more than once +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price), + UP AS price > PREV(price) +); + +-- subqueries in DEFINE clause are not supported +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START LOWPRICE) + DEFINE + START AS TRUE, + LOWPRICE AS price < (SELECT 100) +); + +-- aggregates in DEFINE clause are not supported +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START LOWPRICE) + DEFINE + START AS TRUE, + LOWPRICE AS price < count(*) +); + +-- FRAME must start at current row when row pattern recognition is used +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- SEEK is not supported +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + SEEK + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- PREV's argument must have at least 1 column reference +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(1), + DOWN AS price < PREV(1) +); + +-- Unsupported quantifier +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + INITIAL + PATTERN (START UP~ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(1), + DOWN AS price < PREV(1) +); + +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + INITIAL + PATTERN (START UP+? DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(1), + DOWN AS price < PREV(1) +); + +-- Number of row pattern definition variable names must not exceed 26 + +-- Ok +SELECT * FROM (SELECT 1 AS x) t + WINDOW w AS ( + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + PATTERN (a b c d e f g h i j k l m n o p q r s t u v w x y z) + DEFINE a AS TRUE +); + +-- Error +SELECT * FROM (SELECT 1 AS x) t + WINDOW w AS ( + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + PATTERN (a b c d e f g h i j k l m n o p q r s t u v w x y z aa) + DEFINE a AS TRUE +); + + CREATE TEMP TABLE stock_null (company TEXT, tdate DATE, price INTEGER); + INSERT INTO stock_null VALUES ('c1', '2023-07-01', 100); + INSERT INTO stock_null VALUES ('c1', '2023-07-02', NULL); -- NULL in middle + INSERT INTO stock_null VALUES ('c1', '2023-07-03', 200); + INSERT INTO stock_null VALUES ('c1', '2023-07-04', 150); + + SELECT company, tdate, price, count(*) OVER w AS match_count + FROM stock_null + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + PATTERN (START UP DOWN) + DEFINE START AS TRUE, UP AS price > PREV(price), DOWN AS price < +PREV(price) + ); + + +-- Overlapping match tests (requires multi-context for correct behavior) +-- Using array flags: 'X' = ANY(flags) for multi-TRUE support + +-- Test 1: A B C D E | B C D | C D E F - three overlapping patterns +-- Different end points: B C D (4), A B C D E (5), C D E F (6) +WITH test_overlap1 AS ( + SELECT * FROM (VALUES + (1, ARRAY['A']), + (2, ARRAY['B']), + (3, ARRAY['C']), + (4, ARRAY['D']), + (5, ARRAY['E']), + (6, ARRAY['F']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_overlap1 +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + PATTERN (A B C D E | B C D | C D E F) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags), + C AS 'C' = ANY(flags), + D AS 'D' = ANY(flags), + E AS 'E' = ANY(flags), + F AS 'F' = ANY(flags) +); + +WITH test_overlap1 AS ( + SELECT * FROM (VALUES + (1, ARRAY['A']), + (2, ARRAY['B']), + (3, ARRAY['C']), + (4, ARRAY['D']), + (5, ARRAY['E']), + (6, ARRAY['F']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_overlap1 +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + PATTERN (A B C D E | B C D | C D E F) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags), + C AS 'C' = ANY(flags), + D AS 'D' = ANY(flags), + E AS 'E' = ANY(flags), + F AS 'F' = ANY(flags) +); +-- PAST LAST: only one match +-- TO NEXT ROW with multi-context: three matches +-- Row 1: A B C D E (1-5) +-- Row 2: B C D (2-4) <- ends first! +-- Row 3: C D E F (3-6) <- ends last! + +-- Test 2: A B+ C | B+ D - long B sequence with different endings +WITH test_overlap2 AS ( + SELECT * FROM (VALUES + (1, ARRAY['A']), + (2, ARRAY['B']), + (3, ARRAY['B']), + (4, ARRAY['B']), + (5, ARRAY['B']), + (6, ARRAY['C']), + (7, ARRAY['B']), + (8, ARRAY['B']), + (9, ARRAY['B']), + (10, ARRAY['D']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_overlap2 +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + PATTERN (A B+ C | B+ D) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags), + C AS 'C' = ANY(flags), + D AS 'D' = ANY(flags) +); +-- Current result (correct): +-- Row 1: A B+ C (1-6) +-- Row 7-9: B+ D (7-10, 8-10, 9-10) +-- Note: Row 2-6 cannot match B+ D because Row 6 is C, not D +-- With absorption: 8-10 and 9-10 would be absorbed by 7-10 (earlier context covers later) + +-- Test 3: Greedy quantifier with late failure - A B C+ D | A B +-- Pattern expects D after C+, but E comes instead ("betrayal") +WITH test_betrayal AS ( + SELECT * FROM (VALUES + (1, ARRAY['A']), + (2, ARRAY['B']), + (3, ARRAY['C']), + (4, ARRAY['C']), + (5, ARRAY['C']), + (6, ARRAY['E']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_betrayal +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + PATTERN (A B C+ D | A B) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags), + C AS 'C' = ANY(flags), + D AS 'D' = ANY(flags) +); +-- A B C+ D fails at Row 6 (E instead of D) +-- Question: Does it fallback to A B (1-2)? + +-- Test 4: Lexical Order test - A B C | A B C D E +-- SQL standard: first matching alternative wins +WITH test_lexical AS ( + SELECT * FROM (VALUES + (1, ARRAY['A']), + (2, ARRAY['B']), + (3, ARRAY['C']), + (4, ARRAY['D']), + (5, ARRAY['E']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_lexical +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + PATTERN (A B C | A B C D E) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags), + C AS 'C' = ANY(flags), + D AS 'D' = ANY(flags), + E AS 'E' = ANY(flags) +); +-- SQL standard Lexical Order: A B C (1-3) wins (first alternative) + +-- Test 4b: Reversed pattern order - A B C D E | A B C +WITH test_lexical2 AS ( + SELECT * FROM (VALUES + (1, ARRAY['A']), + (2, ARRAY['B']), + (3, ARRAY['C']), + (4, ARRAY['D']), + (5, ARRAY['E']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_lexical2 +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + PATTERN (A B C D E | A B C) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags), + C AS 'C' = ANY(flags), + D AS 'D' = ANY(flags), + E AS 'E' = ANY(flags) +); +-- SQL standard Lexical Order: A B C D E (1-5) wins (first alternative) + +-- Test 5: Multiple TRUE in single row (overlapping pattern variables) +-- Each row matches multiple DEFINE conditions simultaneously +WITH test_multi_true AS ( + SELECT * FROM (VALUES + (1, ARRAY['A','B']), -- A and B both TRUE + (2, ARRAY['B','C']), -- B and C both TRUE + (3, ARRAY['C','D']), -- C and D both TRUE + (4, ARRAY['D','E']), -- D and E both TRUE + (5, ARRAY['E','_']) -- E only + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_multi_true +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + PATTERN (A B C D E) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags), + C AS 'C' = ANY(flags), + D AS 'D' = ANY(flags), + E AS 'E' = ANY(flags) +); +-- Row 1: A=T, B=T → matches A +-- Row 2: B=T, C=T → matches B +-- Row 3: C=T, D=T → matches C +-- Row 4: D=T, E=T → matches D +-- Row 5: E=T → matches E +-- Result: match 1-5 (A B C D E) + +-- Test 6: Diagonal pattern with multi-TRUE (shifted overlap) +WITH test_diagonal AS ( + SELECT * FROM (VALUES + (1, ARRAY['A','_']), + (2, ARRAY['B','A']), + (3, ARRAY['C','B']), + (4, ARRAY['D','C']), + (5, ARRAY['_','D']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_diagonal +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + PATTERN (A B C D) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags), + C AS 'C' = ANY(flags), + D AS 'D' = ANY(flags) +); +-- Possible matches: +-- Start Row 1: A(1) B(2) C(3) D(4) → 1-4 +-- Start Row 2: A(2) B(3) C(4) D(5) → 2-5 (because Row 2 has A too!) + +-- =================================================================== +-- Context Absorption Tests +-- =================================================================== + +-- Test absorption 1: Basic A+ pattern - later contexts absorbed by earlier +WITH test_absorb_basic AS ( + SELECT * FROM (VALUES + (1, ARRAY['A']), + (2, ARRAY['A']), + (3, ARRAY['A']), + (4, ARRAY['A']), + (5, ARRAY['B']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_absorb_basic +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + PATTERN (A+) + DEFINE A AS 'A' = ANY(flags) +); +-- Pattern A+ is absorbable (unbounded first element, only one unbounded) +-- Without absorption: 4 matches (1-4, 2-4, 3-4, 4-4) +-- With absorption: Row 1 match (1-4), rows 2-4 absorbed + +-- Test absorption 2: A+ B pattern - absorption with fixed suffix +WITH test_absorb_suffix AS ( + SELECT * FROM (VALUES + (1, ARRAY['A']), + (2, ARRAY['A']), + (3, ARRAY['A']), + (4, ARRAY['B']), + (5, ARRAY['X']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_absorb_suffix +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + PATTERN (A+ B) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags) +); +-- Pattern A+ B is absorbable (A+ unbounded first, B bounded suffix) +-- All potential matches end at same row (row 4 with B) +-- With absorption: Row 1 match (1-4), rows 2-3 absorbed + +-- Test absorption 3: Per-branch absorption with ALT (B+ C | B+ D) +WITH test_absorb_alt AS ( + SELECT * FROM (VALUES + (1, ARRAY['B']), + (2, ARRAY['B']), + (3, ARRAY['B']), + (4, ARRAY['D']), + (5, ARRAY['X']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_absorb_alt +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + PATTERN (B+ C | B+ D) + DEFINE + B AS 'B' = ANY(flags), + C AS 'C' = ANY(flags), + D AS 'D' = ANY(flags) +); +-- Both branches B+ C and B+ D are absorbable (B+ unbounded first) +-- B+ D branch matches: potential 1-4, 2-4, 3-4 +-- Row 1 (1-4) absorbs Row 2 (2-4) and Row 3 (3-4) - same endpoint + +-- Test absorption 4: Non-absorbable pattern (A B+ - unbounded not first) +WITH test_no_absorb AS ( + SELECT * FROM (VALUES + (1, ARRAY['A']), + (2, ARRAY['B']), + (3, ARRAY['B']), + (4, ARRAY['B']), + (5, ARRAY['X']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_no_absorb +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + PATTERN (A B+) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags) +); +-- Pattern A B+ is NOT absorbable (A bounded first, B+ unbounded but not first) +-- Only Row 1 can start match (only row with A), so only one match: 1-4 + +-- Test absorption 5: GROUP merge enables absorption +WITH test_absorb_group AS ( + SELECT * FROM (VALUES + (1, ARRAY['A']), + (2, ARRAY['B']), + (3, ARRAY['A']), + (4, ARRAY['B']), + (5, ARRAY['A']), + (6, ARRAY['B']), + (7, ARRAY['X']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_absorb_group +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + PATTERN ((A B) (A B)+) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags) +); +-- Pattern optimized: (A B) (A B)+ -> (A B){2,} +-- Potential matches: 1-6 (3 reps), 3-6 (2 reps), 5-6 needs 2 reps (fail) +-- Row 1 (1-6) absorbs Row 3 (3-6) - same endpoint, top-level unbounded GROUP + +-- Test absorption 6: Multiple unbounded - NOT absorbable +WITH test_multi_unbounded AS ( + SELECT * FROM (VALUES + (1, ARRAY['A']), + (2, ARRAY['A']), + (3, ARRAY['B']), + (4, ARRAY['B']), + (5, ARRAY['X']) + ) AS t(id, flags) +) +SELECT id, flags, first_value(id) OVER w AS match_start, last_value(id) OVER w AS match_end +FROM test_multi_unbounded +WINDOW w AS ( + ORDER BY id + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + PATTERN (A+ B+) + DEFINE + A AS 'A' = ANY(flags), + B AS 'B' = ANY(flags) +); +-- Pattern A+ B+ has TWO unbounded elements - NOT absorbable +-- Greedy A+ consumes rows 1-2, then B+ matches 3-4 +-- Row 1: 1-4, Row 2: 2-4 (different A+ counts, not absorbed) -- 2.43.0 [application/octet-stream] v38-0008-Row-pattern-recognition-patch-typedefs.list.patch (1.3K, ../../[email protected]/9-v38-0008-Row-pattern-recognition-patch-typedefs.list.patch) download | inline diff: From 725031df342b30ead381d38bba1a2dbfd927efd9 Mon Sep 17 00:00:00 2001 From: Tatsuo Ishii <[email protected]> Date: Thu, 15 Jan 2026 13:26:44 +0900 Subject: [PATCH v38 8/8] Row pattern recognition patch (typedefs.list). --- src/tools/pgindent/typedefs.list | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 14dec2d49c1..031701ef479 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -1767,6 +1767,7 @@ NamedLWLockTrancheRequest NamedTuplestoreScan NamedTuplestoreScanState NamespaceInfo +NavigationInfo NestLoop NestLoopParam NestLoopState @@ -2439,6 +2440,15 @@ RI_CompareKey RI_ConstraintInfo RI_QueryHashEntry RI_QueryKey +RPCommonSyntax +RPRDepth +RPRElemFlags +RPRElemIdx +RPRPattern +RPRPatternElement +RPRQuantity +RPRVarId +RPSkipTo RTEKind RTEPermissionInfo RWConflict @@ -2816,6 +2826,7 @@ SimpleStringListCell SingleBoundSortItem SinglePartitionSpec Size +SkipContext SkipPages SkipSupport SkipSupportData @@ -2915,6 +2926,7 @@ StreamStopReason String StringInfo StringInfoData +StringSet StripnullState SubLink SubLinkType @@ -3253,6 +3265,7 @@ VarString VarStringSortSupport Variable VariableAssignHook +VariablePos VariableSetKind VariableSetStmt VariableShowStmt -- 2.43.0 ^ permalink raw reply [nested|flat] 5+ messages in thread
end of thread, other threads:[~2026-01-15 04:56 UTC | newest] Thread overview: 5+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2020-08-03 23:21 [PATCH v1 2/2] ALTER TABLE ... DETACH CONCURRENTLY Alvaro Herrera <[email protected]> 2026-01-13 02:41 Re: Row pattern recognition Tatsuo Ishii <[email protected]> 2026-01-13 02:53 ` Re: Row pattern recognition Henson Choi <[email protected]> 2026-01-14 15:31 ` Re: Row pattern recognition Henson Choi <[email protected]> 2026-01-15 04:56 ` Re: Row pattern recognition Tatsuo Ishii <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox