public inbox for [email protected]help / color / mirror / Atom feed
[PATCH 7/8] Change ALTER TABLESPACE to use the pending-sync infrastructure 6+ messages / 4 participants [nested] [flat]
* [PATCH 7/8] Change ALTER TABLESPACE to use the pending-sync infrastructure @ 2019-03-25 11:39 Kyotaro Horiguchi <[email protected]> 0 siblings, 0 replies; 6+ messages in thread From: Kyotaro Horiguchi @ 2019-03-25 11:39 UTC (permalink / raw) Apply heap_register_sync() to ATLER TABLESPACE stuff. --- src/backend/commands/tablecmds.c | 54 +++++++++++++++++++++------------------- 1 file changed, 28 insertions(+), 26 deletions(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 54ce52eaae..aabb3806f6 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -84,7 +84,6 @@ #include "storage/lmgr.h" #include "storage/lock.h" #include "storage/predicate.h" -#include "storage/smgr.h" #include "utils/acl.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -11891,7 +11890,7 @@ copy_relation_data(SMgrRelation src, SMgrRelation dst, { PGAlignedBlock buf; Page page; - bool use_wal; + bool use_wal = false; bool copying_initfork; BlockNumber nblocks; BlockNumber blkno; @@ -11906,12 +11905,33 @@ copy_relation_data(SMgrRelation src, SMgrRelation dst, copying_initfork = relpersistence == RELPERSISTENCE_UNLOGGED && forkNum == INIT_FORKNUM; - /* - * We need to log the copied data in WAL iff WAL archiving/streaming is - * enabled AND it's a permanent relation. - */ - use_wal = XLogIsNeeded() && - (relpersistence == RELPERSISTENCE_PERMANENT || copying_initfork); + if (relpersistence == RELPERSISTENCE_PERMANENT || copying_initfork) + { + /* + * We need to log the copied data in WAL iff WAL archiving/streaming + * is enabled AND it's a permanent relation. + */ + if (XLogIsNeeded()) + use_wal = true; + + /* + * If the rel is WAL-logged, must fsync at commit. We do the same to + * ensure that the toast table gets fsync'd too. (For a temp or + * unlogged rel we don't care since the data will be gone after a + * crash anyway.) + * + * It's obvious that we must do this when not WAL-logging the + * copy. It's less obvious that we have to do it even if we did + * WAL-log the copied pages. The reason is that since we're copying + * outside shared buffers, a CHECKPOINT occurring during the copy has + * no way to flush the previously written data to disk (indeed it + * won't know the new rel even exists). A crash later on would replay + * WAL from the checkpoint, therefore it wouldn't replay our earlier + * WAL entries. If we do not fsync those pages here, they might still + * not be on disk when the crash occurs. + */ + RecordPendingSync(dst, forkNum); + } nblocks = smgrnblocks(src, forkNum); @@ -11948,24 +11968,6 @@ copy_relation_data(SMgrRelation src, SMgrRelation dst, */ smgrextend(dst, forkNum, blkno, buf.data, true); } - - /* - * If the rel is WAL-logged, must fsync before commit. We use heap_sync - * to ensure that the toast table gets fsync'd too. (For a temp or - * unlogged rel we don't care since the data will be gone after a crash - * anyway.) - * - * It's obvious that we must do this when not WAL-logging the copy. It's - * less obvious that we have to do it even if we did WAL-log the copied - * pages. The reason is that since we're copying outside shared buffers, a - * CHECKPOINT occurring during the copy has no way to flush the previously - * written data to disk (indeed it won't know the new rel even exists). A - * crash later on would replay WAL from the checkpoint, therefore it - * wouldn't replay our earlier WAL entries. If we do not fsync those pages - * here, they might still not be on disk when the crash occurs. - */ - if (relpersistence == RELPERSISTENCE_PERMANENT || copying_initfork) - smgrimmedsync(dst, forkNum); } /* -- 2.16.3 ----Next_Part(Tue_Mar_26_16_35_07_2019_128)-- Content-Type: Text/X-Patch; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="v9-0008-Optimize-WAL-logging-on-btree-bulk-insertion.patch" ^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Question about non-blocking mode in libpq @ 2023-11-01 02:16 Tom Lane <[email protected]> 2023-11-01 12:47 ` Re: Question about non-blocking mode in libpq Bruce Momjian <[email protected]> 0 siblings, 1 reply; 6+ messages in thread From: Tom Lane @ 2023-11-01 02:16 UTC (permalink / raw) To: Bruce Momjian <[email protected]>; +Cc: Yugo NAGATA <[email protected]>; Alvaro Herrera <[email protected]>; pgsql-hackers Bruce Momjian <[email protected]> writes: > On Tue, Oct 31, 2023 at 09:11:06PM -0400, Tom Lane wrote: >> What I'm objecting to is removal of the bit about "if they need to be >> called again". That provides a hint that retry is the appropriate >> response to a failure. Admittedly, it's not 100% clear, but your >> version makes it 0% clear. > I thought the original docs said you had to re-call on failure (it would > not block but it would fail if it could not be sent), while we are now > saying that it will be queued in the input buffer. For these functions in nonblock mode, failure means "we didn't queue it". > Is retry really something we need to mention now? If out of memory is > our only failure case now ("unable to enlarge the buffer because OOM"), > is retry really a realistic option? Well, ideally the application would do something to alleviate the OOM problem before retrying. I don't know if we want to go so far as to discuss that. I do object to giving the impression that failure is impossible, which I think your proposed wording does. An orthogonal issue with your latest wording is that it's unclear whether *unsuccessful* calls to these functions will block. regards, tom lane ^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Question about non-blocking mode in libpq 2023-11-01 02:16 Re: Question about non-blocking mode in libpq Tom Lane <[email protected]> @ 2023-11-01 12:47 ` Bruce Momjian <[email protected]> 2023-11-13 18:01 ` Re: Question about non-blocking mode in libpq Bruce Momjian <[email protected]> 0 siblings, 1 reply; 6+ messages in thread From: Bruce Momjian @ 2023-11-01 12:47 UTC (permalink / raw) To: Tom Lane <[email protected]>; +Cc: Yugo NAGATA <[email protected]>; Alvaro Herrera <[email protected]>; pgsql-hackers On Tue, Oct 31, 2023 at 10:16:07PM -0400, Tom Lane wrote: > Bruce Momjian <[email protected]> writes: > > On Tue, Oct 31, 2023 at 09:11:06PM -0400, Tom Lane wrote: > >> What I'm objecting to is removal of the bit about "if they need to be > >> called again". That provides a hint that retry is the appropriate > >> response to a failure. Admittedly, it's not 100% clear, but your > >> version makes it 0% clear. > > > I thought the original docs said you had to re-call on failure (it would > > not block but it would fail if it could not be sent), while we are now > > saying that it will be queued in the input buffer. > > For these functions in nonblock mode, failure means "we didn't queue it". > > > Is retry really something we need to mention now? If out of memory is > > our only failure case now ("unable to enlarge the buffer because OOM"), > > is retry really a realistic option? > > Well, ideally the application would do something to alleviate the > OOM problem before retrying. I don't know if we want to go so far > as to discuss that. I do object to giving the impression that > failure is impossible, which I think your proposed wording does. > > An orthogonal issue with your latest wording is that it's unclear > whether *unsuccessful* calls to these functions will block. Okay, I see your point now. Here is an updated patch that addresses both issues. -- Bruce Momjian <[email protected]> https://momjian.us EDB https://enterprisedb.com Only you can decide what is important to you. Attachments: [text/x-diff] block.diff (898B, ../../[email protected]/2-block.diff) download | inline diff: diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml index 64b2910fee..ffff0e528d 100644 --- a/doc/src/sgml/libpq.sgml +++ b/doc/src/sgml/libpq.sgml @@ -5316,11 +5316,12 @@ int PQsetnonblocking(PGconn *conn, int arg); </para> <para> - In the nonblocking state, calls to + In the nonblocking state, successful calls to <xref linkend="libpq-PQsendQuery"/>, <xref linkend="libpq-PQputline"/>, <xref linkend="libpq-PQputnbytes"/>, <xref linkend="libpq-PQputCopyData"/>, - and <xref linkend="libpq-PQendcopy"/> will not block but instead return - an error if they need to be called again. + and <xref linkend="libpq-PQendcopy"/> will not block; their changes + are stored in the local output buffer until they are flushed. + Unsuccessful calls will return an error and must be retried. </para> <para> ^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Question about non-blocking mode in libpq 2023-11-01 02:16 Re: Question about non-blocking mode in libpq Tom Lane <[email protected]> 2023-11-01 12:47 ` Re: Question about non-blocking mode in libpq Bruce Momjian <[email protected]> @ 2023-11-13 18:01 ` Bruce Momjian <[email protected]> 2023-11-13 19:05 ` Re: Question about non-blocking mode in libpq Bruce Momjian <[email protected]> 0 siblings, 1 reply; 6+ messages in thread From: Bruce Momjian @ 2023-11-13 18:01 UTC (permalink / raw) To: Tom Lane <[email protected]>; +Cc: Yugo NAGATA <[email protected]>; Alvaro Herrera <[email protected]>; pgsql-hackers On Wed, Nov 1, 2023 at 08:47:33AM -0400, Bruce Momjian wrote: > On Tue, Oct 31, 2023 at 10:16:07PM -0400, Tom Lane wrote: > > Bruce Momjian <[email protected]> writes: > > > On Tue, Oct 31, 2023 at 09:11:06PM -0400, Tom Lane wrote: > > >> What I'm objecting to is removal of the bit about "if they need to be > > >> called again". That provides a hint that retry is the appropriate > > >> response to a failure. Admittedly, it's not 100% clear, but your > > >> version makes it 0% clear. > > > > > I thought the original docs said you had to re-call on failure (it would > > > not block but it would fail if it could not be sent), while we are now > > > saying that it will be queued in the input buffer. > > > > For these functions in nonblock mode, failure means "we didn't queue it". > > > > > Is retry really something we need to mention now? If out of memory is > > > our only failure case now ("unable to enlarge the buffer because OOM"), > > > is retry really a realistic option? > > > > Well, ideally the application would do something to alleviate the > > OOM problem before retrying. I don't know if we want to go so far > > as to discuss that. I do object to giving the impression that > > failure is impossible, which I think your proposed wording does. > > > > An orthogonal issue with your latest wording is that it's unclear > > whether *unsuccessful* calls to these functions will block. > > Okay, I see your point now. Here is an updated patch that addresses > both issues. Patch applied to master. -- Bruce Momjian <[email protected]> https://momjian.us EDB https://enterprisedb.com Only you can decide what is important to you. ^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Question about non-blocking mode in libpq 2023-11-01 02:16 Re: Question about non-blocking mode in libpq Tom Lane <[email protected]> 2023-11-01 12:47 ` Re: Question about non-blocking mode in libpq Bruce Momjian <[email protected]> 2023-11-13 18:01 ` Re: Question about non-blocking mode in libpq Bruce Momjian <[email protected]> @ 2023-11-13 19:05 ` Bruce Momjian <[email protected]> 0 siblings, 0 replies; 6+ messages in thread From: Bruce Momjian @ 2023-11-13 19:05 UTC (permalink / raw) To: Tom Lane <[email protected]>; +Cc: Yugo NAGATA <[email protected]>; Alvaro Herrera <[email protected]>; pgsql-hackers On Mon, Nov 13, 2023 at 01:01:32PM -0500, Bruce Momjian wrote: > On Wed, Nov 1, 2023 at 08:47:33AM -0400, Bruce Momjian wrote: > > On Tue, Oct 31, 2023 at 10:16:07PM -0400, Tom Lane wrote: > > > Bruce Momjian <[email protected]> writes: > > > > On Tue, Oct 31, 2023 at 09:11:06PM -0400, Tom Lane wrote: > > > >> What I'm objecting to is removal of the bit about "if they need to be > > > >> called again". That provides a hint that retry is the appropriate > > > >> response to a failure. Admittedly, it's not 100% clear, but your > > > >> version makes it 0% clear. > > > > > > > I thought the original docs said you had to re-call on failure (it would > > > > not block but it would fail if it could not be sent), while we are now > > > > saying that it will be queued in the input buffer. > > > > > > For these functions in nonblock mode, failure means "we didn't queue it". > > > > > > > Is retry really something we need to mention now? If out of memory is > > > > our only failure case now ("unable to enlarge the buffer because OOM"), > > > > is retry really a realistic option? > > > > > > Well, ideally the application would do something to alleviate the > > > OOM problem before retrying. I don't know if we want to go so far > > > as to discuss that. I do object to giving the impression that > > > failure is impossible, which I think your proposed wording does. > > > > > > An orthogonal issue with your latest wording is that it's unclear > > > whether *unsuccessful* calls to these functions will block. > > > > Okay, I see your point now. Here is an updated patch that addresses > > both issues. > > Patch applied to master. My apologies, I forgot this needed to backpatched, so done now. -- Bruce Momjian <[email protected]> https://momjian.us EDB https://enterprisedb.com Only you can decide what is important to you. ^ permalink raw reply [nested|flat] 6+ messages in thread
* [PATCH v26 6/9] Row pattern recognition patch (docs). @ 2024-12-30 12:44 Tatsuo Ishii <[email protected]> 0 siblings, 0 replies; 6+ messages in thread From: Tatsuo Ishii @ 2024-12-30 12:44 UTC (permalink / raw) --- doc/src/sgml/advanced.sgml | 82 ++++++++++++++++++++++++++++++++++++ doc/src/sgml/func.sgml | 54 ++++++++++++++++++++++++ doc/src/sgml/ref/select.sgml | 38 ++++++++++++++++- 3 files changed, 172 insertions(+), 2 deletions(-) diff --git a/doc/src/sgml/advanced.sgml b/doc/src/sgml/advanced.sgml index 755c9f1485..b0b1d1c51e 100644 --- a/doc/src/sgml/advanced.sgml +++ b/doc/src/sgml/advanced.sgml @@ -537,6 +537,88 @@ WHERE pos < 3; <literal>rank</literal> less than 3. </para> + <para> + Row pattern common syntax can be used to perform row pattern recognition + in a query. The row pattern common syntax includes two sub + clauses: <literal>DEFINE</literal> + and <literal>PATTERN</literal>. <literal>DEFINE</literal> defines + definition variables along with an expression. The expression must be a + logical expression, which means it must + return <literal>TRUE</literal>, <literal>FALSE</literal> + or <literal>NULL</literal>. The expression may comprise column references + and functions. Window functions, aggregate functions and subqueries are + not allowed. An example of <literal>DEFINE</literal> is as follows. + +<programlisting> +DEFINE + LOWPRICE AS price <= 100, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +</programlisting> + + Note that <function>PREV</function> returns the price column in the + previous row if it's called in a context of row pattern recognition. Thus in + the second line the definition variable "UP" is <literal>TRUE</literal> + when the price column in the current row is greater than the price column + in the previous row. Likewise, "DOWN" is <literal>TRUE</literal> when when + the price column in the current row is lower than the price column in the + previous row. + </para> + <para> + Once <literal>DEFINE</literal> exists, <literal>PATTERN</literal> can be + used. <literal>PATTERN</literal> defines a sequence of rows that satisfies + certain conditions. For example following <literal>PATTERN</literal> + defines that a row starts with the condition "LOWPRICE", then one or more + rows satisfy "UP" and finally one or more rows satisfy "DOWN". Note that + "+" means one or more matches. Also you can use "*", which means zero or + more matches. If a sequence of rows which satisfies the PATTERN is found, + in the starting row of the sequence of rows all window functions and + aggregates are shown in the target list. Note that aggregations only look + into the matched rows, rather than whole frame. On the second or + subsequent rows all window functions are NULL. Aggregates are NULL or 0 + (count case) depending on its aggregation definition. For rows that do not + match on the PATTERN, all window functions and aggregates are shown AS + NULL too, except count showing 0. This is because the rows do not match, + thus they are in an empty frame. Example of a <literal>SELECT</literal> + using the <literal>DEFINE</literal> and <literal>PATTERN</literal> clause + is as follows. + +<programlisting> +SELECT company, tdate, price, + first_value(price) OVER w, + max(price) OVER w, + count(price) OVER w +FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + INITIAL + PATTERN (LOWPRICE UP+ DOWN+) + DEFINE + LOWPRICE AS price <= 100, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); +</programlisting> +<screen> + company | tdate | price | first_value | max | count +----------+------------+-------+-------------+-----+------- + company1 | 2023-07-01 | 100 | 100 | 200 | 4 + company1 | 2023-07-02 | 200 | | | 0 + company1 | 2023-07-03 | 150 | | | 0 + company1 | 2023-07-04 | 140 | | | 0 + company1 | 2023-07-05 | 150 | | | 0 + company1 | 2023-07-06 | 90 | 90 | 130 | 4 + company1 | 2023-07-07 | 110 | | | 0 + company1 | 2023-07-08 | 130 | | | 0 + company1 | 2023-07-09 | 120 | | | 0 + company1 | 2023-07-10 | 130 | | | 0 +(10 rows) +</screen> + </para> + <para> When a query involves multiple window functions, it is possible to write out each one with a separate <literal>OVER</literal> clause, but this is diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 47370e581a..17a38c4046 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -23338,6 +23338,7 @@ SELECT count(*) FROM sometable; returns <literal>NULL</literal> if there is no such row. </para></entry> </row> + </tbody> </tgroup> </table> @@ -23377,6 +23378,59 @@ SELECT count(*) FROM sometable; Other frame specifications can be used to obtain other effects. </para> + <para> + Row pattern recognition navigation functions are listed in + <xref linkend="functions-rpr-navigation-table"/>. These functions + can be used to describe DEFINE clause of Row pattern recognition. + </para> + + <table id="functions-rpr-navigation-table"> + <title>Row Pattern Navigation Functions</title> + <tgroup cols="1"> + <thead> + <row> + <entry role="func_table_entry"><para role="func_signature"> + Function + </para> + <para> + Description + </para></entry> + </row> + </thead> + + <tbody> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <indexterm> + <primary>prev</primary> + </indexterm> + <function>prev</function> ( <parameter>value</parameter> <type>anyelement</type> ) + <returnvalue>anyelement</returnvalue> + </para> + <para> + Returns the column value at the previous row; + returns NULL if there is no previous row in the window frame. + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <indexterm> + <primary>next</primary> + </indexterm> + <function>next</function> ( <parameter>value</parameter> <type>anyelement</type> ) + <returnvalue>anyelement</returnvalue> + </para> + <para> + Returns the column value at the next row; + returns NULL if there is no next row in the window frame. + </para></entry> + </row> + + </tbody> + </tgroup> + </table> + <note> <para> The SQL standard defines a <literal>RESPECT NULLS</literal> or diff --git a/doc/src/sgml/ref/select.sgml b/doc/src/sgml/ref/select.sgml index d7089eac0b..7e1c9989ba 100644 --- a/doc/src/sgml/ref/select.sgml +++ b/doc/src/sgml/ref/select.sgml @@ -969,8 +969,8 @@ WINDOW <replaceable class="parameter">window_name</replaceable> AS ( <replaceabl The <replaceable class="parameter">frame_clause</replaceable> can be one of <synopsis> -{ RANGE | ROWS | GROUPS } <replaceable>frame_start</replaceable> [ <replaceable>frame_exclusion</replaceable> ] -{ RANGE | ROWS | GROUPS } BETWEEN <replaceable>frame_start</replaceable> AND <replaceable>frame_end</replaceable> [ <replaceable>frame_exclusion</replaceable> ] +{ RANGE | ROWS | GROUPS } <replaceable>frame_start</replaceable> [ <replaceable>frame_exclusion</replaceable> ] [row_pattern_common_syntax] +{ RANGE | ROWS | GROUPS } BETWEEN <replaceable>frame_start</replaceable> AND <replaceable>frame_end</replaceable> [ <replaceable>frame_exclusion</replaceable> ] [row_pattern_common_syntax] </synopsis> where <replaceable>frame_start</replaceable> @@ -1077,6 +1077,40 @@ EXCLUDE NO OTHERS a given peer group will be in the frame or excluded from it. </para> + <para> + The + optional <replaceable class="parameter">row_pattern_common_syntax</replaceable> + defines the <firstterm>row pattern recognition condition</firstterm> for + this + window. <replaceable class="parameter">row_pattern_common_syntax</replaceable> + includes following subclauses. <literal>AFTER MATCH SKIP PAST LAST + ROW</literal> or <literal>AFTER MATCH SKIP TO NEXT ROW</literal> controls + how to proceed to next row position after a match + found. With <literal>AFTER MATCH SKIP PAST LAST ROW</literal> (the + default) next row position is next to the last row of previous match. On + the other hand, with <literal>AFTER MATCH SKIP TO NEXT ROW</literal> next + row position is always next to the last row of previous + match. <literal>DEFINE</literal> defines definition variables along with a + boolean expression. <literal>PATTERN</literal> defines a sequence of rows + that satisfies certain conditions using variables defined + in <literal>DEFINE</literal> clause. If the variable is not defined in + the <literal>DEFINE</literal> clause, it is implicitly assumed + following is defined in the <literal>DEFINE</literal> clause. + +<synopsis> +<literal>variable_name</literal> AS TRUE +</synopsis> + + Note that the maximu number of variables defined + in <literal>DEFINE</literal> clause is 26. + +<synopsis> +[ AFTER MATCH SKIP PAST LAST ROW | AFTER MATCH SKIP TO NEXT ROW ] +PATTERN <replaceable class="parameter">pattern_variable_name</replaceable>[+] [, ...] +DEFINE <replaceable class="parameter">definition_varible_name</replaceable> AS <replaceable class="parameter">expression</replaceable> [, ...] +</synopsis> + </para> + <para> The purpose of a <literal>WINDOW</literal> clause is to specify the behavior of <firstterm>window functions</firstterm> appearing in the query's -- 2.25.1 ----Next_Part(Mon_Dec_30_22_37_18_2024_171)-- Content-Type: Text/X-Patch; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="v26-0007-Row-pattern-recognition-patch-tests.patch" ^ permalink raw reply [nested|flat] 6+ messages in thread
end of thread, other threads:[~2024-12-30 12:44 UTC | newest] Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2019-03-25 11:39 [PATCH 7/8] Change ALTER TABLESPACE to use the pending-sync infrastructure Kyotaro Horiguchi <[email protected]> 2023-11-01 02:16 Re: Question about non-blocking mode in libpq Tom Lane <[email protected]> 2023-11-01 12:47 ` Re: Question about non-blocking mode in libpq Bruce Momjian <[email protected]> 2023-11-13 18:01 ` Re: Question about non-blocking mode in libpq Bruce Momjian <[email protected]> 2023-11-13 19:05 ` Re: Question about non-blocking mode in libpq Bruce Momjian <[email protected]> 2024-12-30 12:44 [PATCH v26 6/9] Row pattern recognition patch (docs). Tatsuo Ishii <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox