pg.ddx.io pgsql-committers@postgresql.org mailing list archive
help / color / mirror / Atom feedpgsql: Allow using the updated tuple while moving it to a different par
5+ messages / 4 participants
[nested] [flat]
* pgsql: Allow using the updated tuple while moving it to a different par
@ 2018-07-12 07:33 Amit Kapila <akapila@postgresql.org>
0 siblings, 1 reply; 5+ messages in thread
From: Amit Kapila @ 2018-07-12 07:33 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Allow using the updated tuple while moving it to a different partition.
An update that causes the tuple to be moved to a different partition was
missing out on re-constructing the to-be-updated tuple, based on the latest
tuple in the update chain. Instead, it's simply deleting the latest tuple
and inserting a new tuple in the new partition based on the old tuple.
Commit 2f17844104 didn't consider this case, so some of the updates were
getting lost.
In passing, change the argument order for output parameter in ExecDelete
and add some commentary about it.
Reported-by: Pavan Deolasee
Author: Amit Khandekar, with minor changes by me
Reviewed-by: Dilip Kumar, Amit Kapila and Alvaro Herrera
Backpatch-through: 11
Discussion: https://postgr.es/m/CAJ3gD9fRbEzDqdeDq1jxqZUb47kJn+tQ7=Bcgjc8quqKsDViKQ@mail.gmail.com
Branch
------
master
Details
-------
https://git.postgresql.org/pg/commitdiff/40ca70ebcc9d0bc1c02937b27c44b2766617e790
Modified Files
--------------
src/backend/commands/trigger.c | 22 ++++++-
src/backend/executor/execReplication.c | 2 +-
src/backend/executor/nodeModifyTable.c | 73 +++++++++++++++------
src/include/commands/trigger.h | 3 +-
.../isolation/expected/partition-key-update-4.out | 60 +++++++++++++++++
src/test/isolation/isolation_schedule | 1 +
.../isolation/specs/partition-key-update-4.spec | 76 ++++++++++++++++++++++
7 files changed, 214 insertions(+), 23 deletions(-)
^ permalink raw reply [nested|flat] 5+ messages in thread
* pgsql: Allow using the updated tuple while moving it to a different par
@ 2018-07-12 07:33 Amit Kapila <akapila@postgresql.org>
0 siblings, 0 replies; 5+ messages in thread
From: Amit Kapila @ 2018-07-12 07:33 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Allow using the updated tuple while moving it to a different partition.
An update that causes the tuple to be moved to a different partition was
missing out on re-constructing the to-be-updated tuple, based on the latest
tuple in the update chain. Instead, it's simply deleting the latest tuple
and inserting a new tuple in the new partition based on the old tuple.
Commit 2f17844104 didn't consider this case, so some of the updates were
getting lost.
In passing, change the argument order for output parameter in ExecDelete
and add some commentary about it.
Reported-by: Pavan Deolasee
Author: Amit Khandekar, with minor changes by me
Reviewed-by: Dilip Kumar, Amit Kapila and Alvaro Herrera
Backpatch-through: 11
Discussion: https://postgr.es/m/CAJ3gD9fRbEzDqdeDq1jxqZUb47kJn+tQ7=Bcgjc8quqKsDViKQ@mail.gmail.com
Branch
------
REL_11_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/0527df732beb8c2d584c7e4ea5aeb05cf92e14a2
Modified Files
--------------
src/backend/commands/trigger.c | 22 ++++++-
src/backend/executor/execReplication.c | 2 +-
src/backend/executor/nodeModifyTable.c | 73 +++++++++++++++------
src/include/commands/trigger.h | 3 +-
.../isolation/expected/partition-key-update-4.out | 60 +++++++++++++++++
src/test/isolation/isolation_schedule | 1 +
.../isolation/specs/partition-key-update-4.spec | 76 ++++++++++++++++++++++
7 files changed, 214 insertions(+), 23 deletions(-)
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: pgsql: Allow using the updated tuple while moving it to a different par
@ 2018-07-12 07:59 Michael Paquier <michael@paquier.xyz>
parent: Amit Kapila <akapila@postgresql.org>
0 siblings, 1 reply; 5+ messages in thread
From: Michael Paquier @ 2018-07-12 07:59 UTC (permalink / raw)
To: Amit Kapila <akapila@postgresql.org>; +Cc: pgsql-committers@lists.postgresql.org
On Thu, Jul 12, 2018 at 07:33:54AM +0000, Amit Kapila wrote:
> Allow using the updated tuple while moving it to a different partition.
>
> An update that causes the tuple to be moved to a different partition was
> missing out on re-constructing the to-be-updated tuple, based on the latest
> tuple in the update chain. Instead, it's simply deleting the latest tuple
> and inserting a new tuple in the new partition based on the old tuple.
> Commit 2f17844104 didn't consider this case, so some of the updates were
> getting lost.
>
> In passing, change the argument order for output parameter in ExecDelete
> and add some commentary about it.
Looks unrelated, but flaviventris (a snake?) has just complained:
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=flaviventris&dt=2018-07-12%2007%3A36%3A0...
--
Michael
Attachments:
signature.asc (application/pgp-signature, 832B) download
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: pgsql: Allow using the updated tuple while moving it to a different par
@ 2018-07-12 11:14 Amit Kapila <amit.kapila16@gmail.com>
parent: Michael Paquier <michael@paquier.xyz>
0 siblings, 1 reply; 5+ messages in thread
From: Amit Kapila @ 2018-07-12 11:14 UTC (permalink / raw)
To: Michael Paquier <michael@paquier.xyz>; +Cc: Amit Kapila <akapila@postgresql.org>; pgsql-committers@lists.postgresql.org
On Thu, Jul 12, 2018 at 1:29 PM, Michael Paquier <michael@paquier.xyz> wrote:
> On Thu, Jul 12, 2018 at 07:33:54AM +0000, Amit Kapila wrote:
>> Allow using the updated tuple while moving it to a different partition.
>>
>> An update that causes the tuple to be moved to a different partition was
>> missing out on re-constructing the to-be-updated tuple, based on the latest
>> tuple in the update chain. Instead, it's simply deleting the latest tuple
>> and inserting a new tuple in the new partition based on the old tuple.
>> Commit 2f17844104 didn't consider this case, so some of the updates were
>> getting lost.
>>
>> In passing, change the argument order for output parameter in ExecDelete
>> and add some commentary about it.
>
> Looks unrelated, but flaviventris (a snake?) has just complained:
> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=flaviventris&dt=2018-07-12%2007%3A36%3A0...
>
Yeah, I am confused as to how this commit can lead to that failure.
Anyway, just to clarify my understanding, it seems that master node
has executed an Insert statement (INSERT INTO replayed(val) SELECT
coalesce(max(val),0) + 1 AS newval FROM replayed RETURNING val) and
waiting for standby to sync up. In the meantime, some background
process (probably walsender ?) crashed. Are you seeing anything
different? Any other clue?
By the way, I could see the same (appears to be same) failure in other
buildfarm (skink, serinus, etc.) logs in some previous runs:
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=skink&dt=2017-11-30%2008%3A47%3A34
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=serinus&dt=2018-05-18%2020%3A14%3A01
--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: pgsql: Allow using the updated tuple while moving it to a different par
@ 2018-07-12 21:57 Tom Lane <tgl@sss.pgh.pa.us>
parent: Amit Kapila <amit.kapila16@gmail.com>
0 siblings, 0 replies; 5+ messages in thread
From: Tom Lane @ 2018-07-12 21:57 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: Michael Paquier <michael@paquier.xyz>; Amit Kapila <akapila@postgresql.org>; pgsql-committers@lists.postgresql.org
Amit Kapila <amit.kapila16@gmail.com> writes:
> On Thu, Jul 12, 2018 at 1:29 PM, Michael Paquier <michael@paquier.xyz> wrote:
>> Looks unrelated, but flaviventris (a snake?) has just complained:
>> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=flaviventris&dt=2018-07-12%2007%3A36%3A0...
> Yeah, I am confused as to how this commit can lead to that failure.
It didn't; looks like just a random failure to me. The test is timing
out:
ok 26 - catalog xmin of cascaded slot still null with hs_feedback reset
# re-enabling hot_standby_feedback and disabling while stopped
### Reloading node "standby_2"
# Running: pg_ctl -D /home/bf/build/buildfarm-flaviventris/HEAD/pgsql.build/src/test/recovery/tmp_check/t_001_stream_rep_standby_2_data/pgdata reload
server signaled
Waiting for replication conn standby_1's replay_lsn to pass '0/30A1720' on master
done
Waiting for replication conn standby_2's replay_lsn to pass '0/30A1720' on standby_1
done
### Stopping node "standby_2" using mode fast
# Running: pg_ctl -D /home/bf/build/buildfarm-flaviventris/HEAD/pgsql.build/src/test/recovery/tmp_check/t_001_stream_rep_standby_2_data/pgdata -m fast stop
waiting for server to shut down.... done
server stopped
# No postmaster PID for node "standby_2"
#
Timed out waiting for slot xmins to advance at t/001_stream_rep.pl line 157.
This looks to me like it's probably a low-probability timing problem in
the test script itself. The relevant part of the script is
$node_master->safe_psql('postgres', qq[INSERT INTO tab_int VALUES (11000);]);
replay_check();
$node_standby_2->safe_psql('postgres',
'ALTER SYSTEM SET hot_standby_feedback = off;');
$node_standby_2->stop;
($xmin, $catalog_xmin) =
get_slot_xmins($node_standby_1, $slotname_2, "xmin IS NOT NULL");
isnt($xmin, '', 'xmin of cascaded slot non-null with postgres shut down');
where the failure is occurring because get_slot_xmins() never sees the
expected state. I'm suspicious that this is happening because the script
is making no attempt to ensure the relative timing of events on the three
servers --- notably, issuing a "reload" does nothing to guarantee that
the server has actually seen and responded to the parameter change.
As an extreme case, I can reproduce the failure exactly with this:
inline patch 1: src/test/recovery/t/001_stream_rep.pl (+1/-1 lines) download inline-1.patch
diff --git a/src/test/recovery/t/001_stream_rep.pl b/src/test/recovery/t/001_stream_rep.pl
index a0d3e8f..0824184 100644
--- a/src/test/recovery/t/001_stream_rep.pl
+++ b/src/test/recovery/t/001_stream_rep.pl
@@ -286,7 +286,7 @@ is($catalog_xmin, '',
note "re-enabling hot_standby_feedback and disabling while stopped";
$node_standby_2->safe_psql('postgres',
'ALTER SYSTEM SET hot_standby_feedback = on;');
-$node_standby_2->reload;
+# $node_standby_2->reload;
$node_master->safe_psql('postgres', qq[INSERT INTO tab_int VALUES (11000);]);
replay_check();
end of thread, other threads:[~2018-07-12 21:57 UTC | newest]
Thread overview: 5+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2018-07-12 07:33 pgsql: Allow using the updated tuple while moving it to a different par Amit Kapila <akapila@postgresql.org>
2018-07-12 07:59 ` Michael Paquier <michael@paquier.xyz>
2018-07-12 11:14 ` Amit Kapila <amit.kapila16@gmail.com>
2018-07-12 21:57 ` Tom Lane <tgl@sss.pgh.pa.us>
2018-07-12 07:33 pgsql: Allow using the updated tuple while moving it to a different par Amit Kapila <akapila@postgresql.org>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox