public inbox for [email protected]
help / color / mirror / Atom feedRe: SQL:2011 application time
6+ messages / 5 participants
[nested] [flat]
* Re: SQL:2011 application time
@ 2022-01-05 16:07 Peter Eisentraut <[email protected]>
0 siblings, 1 reply; 6+ messages in thread
From: Peter Eisentraut @ 2022-01-05 16:07 UTC (permalink / raw)
To: Paul A Jungwirth <[email protected]>; Corey Huinker <[email protected]>; +Cc: Jaime Casanova <[email protected]>; PostgreSQL Hackers <[email protected]>
On 21.11.21 02:51, Paul A Jungwirth wrote:
> Here are updated patches. They are rebased and clean up some of my
> TODOs.
This patch set looks very interesting. It's also very big, so it's
difficult to see how to get a handle on it. I did a pass through it
to see if there were any obvious architectural or coding style
problems. I also looked at some of your TODO comments to see if I had
something to contribute there.
I'm confused about how to query tables based on application time
periods. Online, I see examples using AS OF, but in the SQL standard
I only see this used for system time, which we are not doing here.
What is your understanding of that?
v10-0001-Add-PERIODs.patch
src/backend/commands/tablecmds.c
Might be worth explaining somewhere why AT_PASS_ADD_PERIOD needs to be
its own pass. -- Ah, this is explained in ATPrepCmd(). Maybe that is
okay, but I would tend to prefer a comprehensive explanation here
rather than sprinkled around.
make_period_not_backward(): Hardcoding the name of the operator as "<"
is not good. You should perhaps lookup the less-than operator in the
type cache. Look around for TYPECACHE_LT_OPR for how this is usually done.
validate_period(): Could use an explanatory comment. There are a
bunch of output arguments, and it's not clear what all of this is
supposed to do, and what "validating" is in this context.
MergeAttributes(): I would perhaps initially just prohibit inheritance
situations that involve periods on either side. (It should work for
partitioning, IMO, but that should be easier to arrange.)
AlterTableGetLockLevel(): The choice of AccessExclusiveLock looks
correct. I think the whole thing can also be grouped with some of the
other "affects concurrent SELECTs" cases?
Maybe the node type Period could have a slightly more specific name,
perhaps PeriodDef, analogous to ColumnDef?
I didn't follow why indexes would have periods, for example, the new
period field in IndexStmt. Is that explained anywhere?
While reading this patch I kept wondering whether it would be possible
to fold periods into pg_attribute, perhaps with negative attribute
numbers. Have you looked into something like that? No doubt it's
also complicated, but it might simplify some things, like the name
conflict checking.
v10-0002-Add-temporal-PRIMARY-KEY-and-UNIQUE-constraints.patch
src/backend/catalog/Catalog.pm: I see you use this change in the
subsequent patches, but I would recommend skipping all this. The
comments added are kind of redundant with the descr fields anyway.
transformIndexConstraint(): As above, we can't look up the && operator
by name. In this case, I suppose we should look it up through the
index AM support operators.
Further, the additions to this function are very complicated and not
fully explained. I'm suspicious about things like
findNewOrOldColumn() -- generally we should look up columns by number
not name. Perhaps you can add a header comment or split out the code
further into smaller functions.
pg_dump.c getIndexes() has been refactored since to make
version-specific additions easier. But your patch is now failing to
apply because of this.
Of course, the main problem in this patch is that for most uses it
requires btree_gist. I think we should consider moving that into
core, or at least the support for types that are most relevant to this
functionality, specifically the date/time types. Aside from user
convenience, this would also allow writing more realistic test cases.
v10-0003-Add-UPDATE-DELETE-FOR-PORTION-OF.patch
Use of MINVALUE and MAXVALUE for unbounded seems problematic to me.
(If it is some value, it is not really larger than any value.) We
have the keyword UNBOUNDED, which seems better suited.
src/backend/access/brin/brin_minmax_multi.c
These renaming changes seem unrelated (but still seem like a good
idea). Should they be progressed separately?
Again, some hardcoded operator name lookup in this patch.
I don't understand why a temporal primary key is required for doing
UPDATE FOR PORTION OF. I don't see this in the standard.
v10-0004-Add-temporal-FOREIGN-KEYs.patch
Do we really need different trigger names depending on whether the
foreign key is temporal?
range_as_string() doesn't appear to be used anywhere.
I ran out of steam on this patch, it's very big. But it seems sound
in general.
How to proceed. I suppose we could focus on committing 0001 and 0002
first. That would be a sensible feature set even if the remaining
patches did not make a release. I do feel we need to get btree_gist
into core. That might be a big job by itself. I'm also bemused why
btree_gist is so bloated compared to btree_gin. btree_gin uses macros
to eliminate duplicate code where btree_gist is full of
copy-and-paste. So there are some opportunities there to make things
more compact. Is there anything else you think we can do as
preparatory work to make the main patches more manageable?
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: SQL:2011 application time
@ 2022-01-05 22:03 Corey Huinker <[email protected]>
parent: Peter Eisentraut <[email protected]>
0 siblings, 1 reply; 6+ messages in thread
From: Corey Huinker @ 2022-01-05 22:03 UTC (permalink / raw)
To: Peter Eisentraut <[email protected]>; +Cc: Paul A Jungwirth <[email protected]>; Jaime Casanova <[email protected]>; PostgreSQL Hackers <[email protected]>
On Wed, Jan 5, 2022 at 11:07 AM Peter Eisentraut <
[email protected]> wrote:
> On 21.11.21 02:51, Paul A Jungwirth wrote:
> > Here are updated patches. They are rebased and clean up some of my
> > TODOs.
>
> This patch set looks very interesting. It's also very big, so it's
> difficult to see how to get a handle on it. I did a pass through it
> to see if there were any obvious architectural or coding style
> problems. I also looked at some of your TODO comments to see if I had
> something to contribute there.
>
> I'm confused about how to query tables based on application time
> periods. Online, I see examples using AS OF, but in the SQL standard
> I only see this used for system time, which we are not doing here.
> What is your understanding of that?
>
Paul has previously supplied me with this document
https://cs.ulb.ac.be/public/_media/teaching/infoh415/tempfeaturessql2011.pdf
and that formed the basis of a lot of my questions a few months earlier.
There was similar work being done for system periods, which are a bit
simpler but require a side (history) table to be created. I was picking
people's brains about some aspects of system versioning to see if I could
help bringing that into this already very large patchset, but haven't yet
felt like I had done enough research to post it.
It is my hope that we can at least get the syntax for both application and
system versioning committed, even if it's just stubbed in with
not-yet-supported errors.
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: SQL:2011 application time
@ 2022-01-06 14:44 Vik Fearing <[email protected]>
parent: Corey Huinker <[email protected]>
0 siblings, 1 reply; 6+ messages in thread
From: Vik Fearing @ 2022-01-06 14:44 UTC (permalink / raw)
To: Corey Huinker <[email protected]>; Peter Eisentraut <[email protected]>; +Cc: Paul A Jungwirth <[email protected]>; Jaime Casanova <[email protected]>; PostgreSQL Hackers <[email protected]>
On 1/5/22 11:03 PM, Corey Huinker wrote:
>
> There was similar work being done for system periods, which are a bit
> simpler but require a side (history) table to be created.
This is false. SYSTEM_TIME periods do not need any kind of history.
This was one of the problems I had with Surafel's attempt because it was
confusing the period with SYSTEM VERSIONING. Versioning needs the
period but the inverse is not true.
--
Vik Fearing
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: SQL:2011 application time
@ 2022-01-06 18:08 Paul A Jungwirth <[email protected]>
parent: Vik Fearing <[email protected]>
0 siblings, 1 reply; 6+ messages in thread
From: Paul A Jungwirth @ 2022-01-06 18:08 UTC (permalink / raw)
To: Vik Fearing <[email protected]>; +Cc: Corey Huinker <[email protected]>; Peter Eisentraut <[email protected]>; Jaime Casanova <[email protected]>; PostgreSQL Hackers <[email protected]>
On Thu, Jan 6, 2022 at 6:45 AM Vik Fearing <[email protected]> wrote:
>
> On 1/5/22 11:03 PM, Corey Huinker wrote:
> >
> > There was similar work being done for system periods, which are a bit
> > simpler but require a side (history) table to be created.
>
> This is false. SYSTEM_TIME periods do not need any kind of history.
> This was one of the problems I had with Surafel's attempt because it was
> confusing the period with SYSTEM VERSIONING. Versioning needs the
> period but the inverse is not true.
This is an interesting point. Syntactically, there are three different
things: the generated started/end columns, the period declaration, and
the WITH SYSTEM VERSIONING modifier to the table. You could declare a
system period without making the table versioned. Practically speaking
I don't know why you'd ever create a system period without a versioned
table (do you know of any uses Vik?), but perhaps we can exploit the
separation to add system periods in the same patch that adds
application periods.
The first two bits of syntax *are* tied together: you need columns
with GENERATED ALWAYS AS ROW START/END to declare the system period,
and less intuitively the standard says you can't use AS ROW START/END
unless those columns appear in a system period (2.e.v.2 under Part 2:
Foundation, 11.3 <table definition>). Personally I'd be willing to
ignore that latter requirement. For one thing, what does Postgres do
with the columns if you drop the period? Dropping the columns
altogether seems very harsh, so I guess you'd just remove the
GENERATED clause.
Another weird thing is that you don't (can't) say STORED for those
columns. But they are certainly stored somewhere. I would store the
values just like any other column (even if non-current rows get moved
to a separate table). Also then you don't have to do anything extra
when the GENERATED clause is dropped.
If we wanted to support system-time periods without building all of
system versioning, what would that look like? At first I thought it
would be a trivial addition to part-1 of the patch here, but the more
I think about it the more it seems to deserve its own patch.
One rule I think we should follow is that using a non-system-versioned
table (with a system period) should get you to the same place as using
a system-versioned table and then removing the system versioning. But
the standard says that dropping system versioning should automatically
drop all historical records (2 under Part 2: Foundation, 11.30 <drop
system versioning clause>). That actually makes sense though: when you
do DML we automatically update the start/end columns, but we don't
save copies of the previous data (and incidentally the end column will
always be the max value.) So there is a use case, albeit a thin one:
you get a Rails-like updated_at column that is maintained
automatically by your RDBMS. That is pretty easy, but I think I'd
still break it out into a separate patch. I'm happy to work on that as
something that builds on top of my part-1 patch here.
Yours,
Paul
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: SQL:2011 application time
@ 2022-01-06 18:52 Corey Huinker <[email protected]>
parent: Paul A Jungwirth <[email protected]>
0 siblings, 0 replies; 6+ messages in thread
From: Corey Huinker @ 2022-01-06 18:52 UTC (permalink / raw)
To: Paul A Jungwirth <[email protected]>; +Cc: Vik Fearing <[email protected]>; Peter Eisentraut <[email protected]>; Jaime Casanova <[email protected]>; PostgreSQL Hackers <[email protected]>
>
>
> But
> the standard says that dropping system versioning should automatically
> drop all historical records (2 under Part 2: Foundation, 11.30 <drop
> system versioning clause>). That actually makes sense though: when you
> do DML we automatically update the start/end columns, but we don't
> save copies of the previous data (and incidentally the end column will
> always be the max value.)
This is what I was referring to when I mentioned a side-table.
deleting history would be an O(1) operation. Any other
misunderstandings are all mine.
^ permalink raw reply [nested|flat] 6+ messages in thread
* [PATCH v9 5/7] Row pattern recognition patch (docs).
@ 2023-10-04 05:51 Tatsuo Ishii <[email protected]>
0 siblings, 0 replies; 6+ messages in thread
From: Tatsuo Ishii @ 2023-10-04 05:51 UTC (permalink / raw)
---
doc/src/sgml/advanced.sgml | 52 ++++++++++++++++++++++++++++++++++
doc/src/sgml/func.sgml | 54 ++++++++++++++++++++++++++++++++++++
doc/src/sgml/ref/select.sgml | 38 +++++++++++++++++++++++--
3 files changed, 142 insertions(+), 2 deletions(-)
diff --git a/doc/src/sgml/advanced.sgml b/doc/src/sgml/advanced.sgml
index 755c9f1485..f39ec8f2d5 100644
--- a/doc/src/sgml/advanced.sgml
+++ b/doc/src/sgml/advanced.sgml
@@ -537,6 +537,58 @@ WHERE pos < 3;
<literal>rank</literal> less than 3.
</para>
+ <para>
+ Row pattern common syntax can be used with row pattern common syntax to
+ perform row pattern recognition in a query. Row pattern common syntax
+ includes two sub clauses. <literal>DEFINE</literal> defines definition
+ variables along with an expression. The expression must be a logical
+ expression, which means it must
+ return <literal>TRUE</literal>, <literal>FALSE</literal>
+ or <literal>NULL</literal>. Moreover if the expression comprises a column
+ reference, it must be the argument of <function>rpr</function>. An example
+ of <literal>DEFINE</literal> is as follows.
+
+<programlisting>
+DEFINE
+ LOWPRICE AS price <= 100,
+ UP AS price > PREV(price),
+ DOWN AS price < PREV(price)
+</programlisting>
+
+ Note that <function>PREV</function> returns price column in the previous
+ row if it's called in a context of row pattern recognition. So in the
+ second line means the definition variable "UP" is <literal>TRUE</literal>
+ when price column in the current row is greater than the price column in
+ the previous row. Likewise, "DOWN" is <literal>TRUE</literal> when when
+ price column in the current row is lower than the price column in the
+ previous row.
+ </para>
+ <para>
+ Once <literal>DEFINE</literal> exists, <literal>PATTERN</literal> can be
+ used. <literal>PATTERN</literal> defines a sequence of rows that satisfies
+ certain conditions. For example following <literal>PATTERN</literal>
+ defines that a row starts with the condition "LOWPRICE", then one or more
+ rows satisfy "UP" and finally one or more rows satisfy "DOWN". If a
+ sequence of rows found, rpr returns the column at the starting row.
+ Example of a <literal>SELECT</literal> using the <literal>DEFINE</literal>
+ and <literal>PATTERN</literal> clause is as follows.
+
+<programlisting>
+SELECT company, tdate, price, max(price) OVER w FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ AFTER MATCH SKIP PAST LAST ROW
+ INITIAL
+ PATTERN (LOWPRICE UP+ DOWN+)
+ DEFINE
+ LOWPRICE AS price <= 100,
+ UP AS price > PREV(price),
+ DOWN AS price < PREV(price)
+);
+</programlisting>
+ </para>
+
<para>
When a query involves multiple window functions, it is possible to write
out each one with a separate <literal>OVER</literal> clause, but this is
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index f1ad64c3d6..6c46ea4355 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21780,6 +21780,7 @@ SELECT count(*) FROM sometable;
returns <literal>NULL</literal> if there is no such row.
</para></entry>
</row>
+
</tbody>
</tgroup>
</table>
@@ -21819,6 +21820,59 @@ SELECT count(*) FROM sometable;
Other frame specifications can be used to obtain other effects.
</para>
+ <para>
+ Row pattern recognition navigation functions are listed in
+ <xref linkend="functions-rpr-navigation-table"/>. These functions
+ can be used to describe DEFINE clause of Row pattern recognition.
+ </para>
+
+ <table id="functions-rpr-navigation-table">
+ <title>Row Pattern Navigation Functions</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ Function
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>prev</primary>
+ </indexterm>
+ <function>prev</function> ( <parameter>value</parameter> <type>anyelement</type> )
+ <returnvalue>anyelement</returnvalue>
+ </para>
+ <para>
+ Returns the column value at the previous row;
+ returns NULL if there is no previous row in the window frame.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>next</primary>
+ </indexterm>
+ <function>next</function> ( <parameter>value</parameter> <type>anyelement</type> )
+ <returnvalue>anyelement</returnvalue>
+ </para>
+ <para>
+ Returns the column value at the next row;
+ returns NULL if there is no next row in the window frame.
+ </para></entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+
<note>
<para>
The SQL standard defines a <literal>RESPECT NULLS</literal> or
diff --git a/doc/src/sgml/ref/select.sgml b/doc/src/sgml/ref/select.sgml
index 42d78913cf..522ad9dd70 100644
--- a/doc/src/sgml/ref/select.sgml
+++ b/doc/src/sgml/ref/select.sgml
@@ -969,8 +969,8 @@ WINDOW <replaceable class="parameter">window_name</replaceable> AS ( <replaceabl
The <replaceable class="parameter">frame_clause</replaceable> can be one of
<synopsis>
-{ RANGE | ROWS | GROUPS } <replaceable>frame_start</replaceable> [ <replaceable>frame_exclusion</replaceable> ]
-{ RANGE | ROWS | GROUPS } BETWEEN <replaceable>frame_start</replaceable> AND <replaceable>frame_end</replaceable> [ <replaceable>frame_exclusion</replaceable> ]
+{ RANGE | ROWS | GROUPS } <replaceable>frame_start</replaceable> [ <replaceable>frame_exclusion</replaceable> ] [row_pattern_common_syntax]
+{ RANGE | ROWS | GROUPS } BETWEEN <replaceable>frame_start</replaceable> AND <replaceable>frame_end</replaceable> [ <replaceable>frame_exclusion</replaceable> ] [row_pattern_common_syntax]
</synopsis>
where <replaceable>frame_start</replaceable>
@@ -1077,6 +1077,40 @@ EXCLUDE NO OTHERS
a given peer group will be in the frame or excluded from it.
</para>
+ <para>
+ The
+ optional <replaceable class="parameter">row_pattern_common_syntax</replaceable>
+ defines the <firstterm>row pattern recognition condition</firstterm> for
+ this
+ window. <replaceable class="parameter">row_pattern_common_syntax</replaceable>
+ includes following subclauses. <literal>AFTER MATCH SKIP PAST LAST
+ ROW</literal> or <literal>AFTER MATCH SKIP TO NEXT ROW</literal> controls
+ how to proceed to next row position after a match
+ found. With <literal>AFTER MATCH SKIP PAST LAST ROW</literal> (the
+ default) next row position is next to the last row of previous match. On
+ the other hand, with <literal>AFTER MATCH SKIP TO NEXT ROW</literal> next
+ row position is always next to the last row of previous
+ match. <literal>DEFINE</literal> defines definition variables along with a
+ boolean expression. <literal>PATTERN</literal> defines a sequence of rows
+ that satisfies certain conditions using variables defined
+ in <literal>DEFINE</literal> clause. If the variable is not defined in
+ the <literal>DEFINE</literal> clause, it is implicitly assumed
+ following is defined in the <literal>DEFINE</literal> clause.
+
+<synopsis>
+<literal>variable_name</literal> AS TRUE
+</synopsis>
+
+ Note that the maximu number of variables defined
+ in <literal>DEFINE</literal> clause is 26.
+
+<synopsis>
+[ AFTER MATCH SKIP PAST LAST ROW | AFTER MATCH SKIP TO NEXT ROW ]
+PATTERN <replaceable class="parameter">pattern_variable_name</replaceable>[+] [, ...]
+DEFINE <replaceable class="parameter">definition_varible_name</replaceable> AS <replaceable class="parameter">expression</replaceable> [, ...]
+</synopsis>
+ </para>
+
<para>
The purpose of a <literal>WINDOW</literal> clause is to specify the
behavior of <firstterm>window functions</firstterm> appearing in the query's
--
2.25.1
----Next_Part(Wed_Oct__4_15_03_28_2023_821)--
Content-Type: Text/X-Patch; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
filename="v9-0006-Row-pattern-recognition-patch-tests.patch"
^ permalink raw reply [nested|flat] 6+ messages in thread
end of thread, other threads:[~2023-10-04 05:51 UTC | newest]
Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2022-01-05 16:07 Re: SQL:2011 application time Peter Eisentraut <[email protected]>
2022-01-05 22:03 ` Corey Huinker <[email protected]>
2022-01-06 14:44 ` Vik Fearing <[email protected]>
2022-01-06 18:08 ` Paul A Jungwirth <[email protected]>
2022-01-06 18:52 ` Corey Huinker <[email protected]>
2023-10-04 05:51 [PATCH v9 5/7] Row pattern recognition patch (docs). Tatsuo Ishii <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox