public inbox for [email protected]
help / color / mirror / Atom feed[PATCH v16 03/10] Add tests on pg_ls_dir before changing it
5+ messages / 3 participants
[nested] [flat]
* [PATCH v16 03/10] Add tests on pg_ls_dir before changing it
@ 2020-03-17 18:16 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 5+ messages in thread
From: Justin Pryzby @ 2020-03-17 18:16 UTC (permalink / raw)
---
src/test/regress/expected/misc_functions.out | 18 ++++++++++++++++++
src/test/regress/sql/misc_functions.sql | 5 +++++
2 files changed, 23 insertions(+)
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index d3acb98d04..2e87c548eb 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -201,6 +201,24 @@ select count(*) > 0 from
t
(1 row)
+select * from (select pg_ls_dir('.', false, true) as name) as ls where ls.name='.'; -- include_dot_dirs=true
+ name
+------
+ .
+(1 row)
+
+select * from (select pg_ls_dir('.', false, false) as name) as ls where ls.name='.'; -- include_dot_dirs=false
+ name
+------
+(0 rows)
+
+select pg_ls_dir('does not exist', true, false); -- ok with missingok=true
+ pg_ls_dir
+-----------
+(0 rows)
+
+select pg_ls_dir('does not exist'); -- fails with missingok=false
+ERROR: could not open directory "does not exist": No such file or directory
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 094e8f8296..f6857ad177 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -60,6 +60,11 @@ select count(*) > 0 from
where spcname = 'pg_default') pts
join pg_database db on pts.pts = db.oid;
+select * from (select pg_ls_dir('.', false, true) as name) as ls where ls.name='.'; -- include_dot_dirs=true
+select * from (select pg_ls_dir('.', false, false) as name) as ls where ls.name='.'; -- include_dot_dirs=false
+select pg_ls_dir('does not exist', true, false); -- ok with missingok=true
+select pg_ls_dir('does not exist'); -- fails with missingok=false
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0004-Add-pg_ls_dir_metadata-to-list-a-dir-with-file-m.patch"
^ permalink raw reply [nested|flat] 5+ messages in thread
* Assertion being hit during WAL replay
@ 2023-04-11 18:48 Tom Lane <[email protected]>
0 siblings, 1 reply; 5+ messages in thread
From: Tom Lane @ 2023-04-11 18:48 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: [email protected]
I have seen this failure a couple of times recently while
testing code that caused crashes and restarts:
#2 0x00000000009987e3 in ExceptionalCondition (
conditionName=conditionName@entry=0xb31bc8 "mode == RBM_NORMAL || mode == RBM_ZERO_ON_ERROR || mode == RBM_ZERO_AND_LOCK",
fileName=fileName@entry=0xb31c15 "bufmgr.c",
lineNumber=lineNumber@entry=892) at assert.c:66
#3 0x0000000000842d73 in ExtendBufferedRelTo (eb=...,
fork=fork@entry=MAIN_FORKNUM, strategy=strategy@entry=0x0,
flags=flags@entry=3, extend_to=extend_to@entry=1,
mode=mode@entry=RBM_ZERO_AND_CLEANUP_LOCK) at bufmgr.c:891
#4 0x00000000005cc398 in XLogReadBufferExtended (rlocator=...,
forknum=MAIN_FORKNUM, blkno=0, mode=mode@entry=RBM_ZERO_AND_CLEANUP_LOCK,
recent_buffer=<optimized out>) at xlogutils.c:527
#5 0x00000000005cc697 in XLogReadBufferForRedoExtended (
record=record@entry=0x1183b98, block_id=block_id@entry=0 '\000',
mode=mode@entry=RBM_NORMAL, get_cleanup_lock=get_cleanup_lock@entry=true,
buf=buf@entry=0x7ffd98e3ea94) at xlogutils.c:391
#6 0x000000000055df59 in heap_xlog_prune (record=0x1183b98) at heapam.c:8779
#7 heap2_redo (record=0x1183b98) at heapam.c:10015
#8 0x00000000005ca430 in ApplyWalRecord (replayTLI=<synthetic pointer>,
record=0x7f8f7afbcb60, xlogreader=<optimized out>)
at ../../../../src/include/access/xlog_internal.h:379
It's not clear to me whether this Assert is wrong, or
XLogReadBufferForRedoExtended shouldn't be using
RBM_ZERO_AND_CLEANUP_LOCK, or the Assert is correctly protecting an
unimplemented case in ExtendBufferedRelTo that we now need to implement.
In any case, I'm pretty sure Andres broke it in 26158b852, because
I hadn't seen it before this weekend.
regards, tom lane
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: Assertion being hit during WAL replay
@ 2023-04-11 19:56 Andres Freund <[email protected]>
parent: Tom Lane <[email protected]>
0 siblings, 1 reply; 5+ messages in thread
From: Andres Freund @ 2023-04-11 19:56 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: [email protected]
Hi,
On 2023-04-11 14:48:44 -0400, Tom Lane wrote:
> I have seen this failure a couple of times recently while
> testing code that caused crashes and restarts:
Do you have a quick repro recipe?
> #2 0x00000000009987e3 in ExceptionalCondition (
> conditionName=conditionName@entry=0xb31bc8 "mode == RBM_NORMAL || mode == RBM_ZERO_ON_ERROR || mode == RBM_ZERO_AND_LOCK",
> fileName=fileName@entry=0xb31c15 "bufmgr.c",
> lineNumber=lineNumber@entry=892) at assert.c:66
> #3 0x0000000000842d73 in ExtendBufferedRelTo (eb=...,
> fork=fork@entry=MAIN_FORKNUM, strategy=strategy@entry=0x0,
> flags=flags@entry=3, extend_to=extend_to@entry=1,
> mode=mode@entry=RBM_ZERO_AND_CLEANUP_LOCK) at bufmgr.c:891
> #4 0x00000000005cc398 in XLogReadBufferExtended (rlocator=...,
> forknum=MAIN_FORKNUM, blkno=0, mode=mode@entry=RBM_ZERO_AND_CLEANUP_LOCK,
> recent_buffer=<optimized out>) at xlogutils.c:527
> #5 0x00000000005cc697 in XLogReadBufferForRedoExtended (
> record=record@entry=0x1183b98, block_id=block_id@entry=0 '\000',
> mode=mode@entry=RBM_NORMAL, get_cleanup_lock=get_cleanup_lock@entry=true,
> buf=buf@entry=0x7ffd98e3ea94) at xlogutils.c:391
> #6 0x000000000055df59 in heap_xlog_prune (record=0x1183b98) at heapam.c:8779
> #7 heap2_redo (record=0x1183b98) at heapam.c:10015
> #8 0x00000000005ca430 in ApplyWalRecord (replayTLI=<synthetic pointer>,
> record=0x7f8f7afbcb60, xlogreader=<optimized out>)
> at ../../../../src/include/access/xlog_internal.h:379
>
> It's not clear to me whether this Assert is wrong, or
> XLogReadBufferForRedoExtended shouldn't be using
> RBM_ZERO_AND_CLEANUP_LOCK, or the Assert is correctly protecting an
> unimplemented case in ExtendBufferedRelTo that we now need to implement.
Hm. It's not implemented because I didn't quite see how it'd make sense to
pass RBM_ZERO_AND_CLEANUP_LOCK when extending the relation, but given how
relation extension is done "implicitly" during recovery, that's too narrow a
view. It's trivial to add.
I wonder if we should eventually redefine the RBM* things into a bitmask.
> In any case, I'm pretty sure Andres broke it in 26158b852, because
> I hadn't seen it before this weekend.
Yea, that's clearly the fault of 26158b852.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: Assertion being hit during WAL replay
@ 2023-04-11 20:54 Tom Lane <[email protected]>
parent: Andres Freund <[email protected]>
0 siblings, 1 reply; 5+ messages in thread
From: Tom Lane @ 2023-04-11 20:54 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: [email protected]
Andres Freund <[email protected]> writes:
> On 2023-04-11 14:48:44 -0400, Tom Lane wrote:
>> I have seen this failure a couple of times recently while
>> testing code that caused crashes and restarts:
> Do you have a quick repro recipe?
Here's something related to what I hit that time:
diff --git a/src/backend/optimizer/plan/subselect.c b/src/backend/optimizer/plan/subselect.c
index 052263aea6..d43a7c7bcb 100644
--- a/src/backend/optimizer/plan/subselect.c
+++ b/src/backend/optimizer/plan/subselect.c
@@ -2188,6 +2188,7 @@ SS_charge_for_initplans(PlannerInfo *root, RelOptInfo *final_rel)
void
SS_attach_initplans(PlannerInfo *root, Plan *plan)
{
+ Assert(root->init_plans == NIL);
plan->initPlan = root->init_plans;
}
You won't get through initdb with this, but if you install this change
into a successfully init'd database and then "make installcheck-parallel",
it will crash and then fail to recover, at least a lot of the time.
regards, tom lane
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: Assertion being hit during WAL replay
@ 2023-04-11 22:03 Andres Freund <[email protected]>
parent: Tom Lane <[email protected]>
0 siblings, 0 replies; 5+ messages in thread
From: Andres Freund @ 2023-04-11 22:03 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: [email protected]
Hi,
On 2023-04-11 16:54:53 -0400, Tom Lane wrote:
> Andres Freund <[email protected]> writes:
> > On 2023-04-11 14:48:44 -0400, Tom Lane wrote:
> >> I have seen this failure a couple of times recently while
> >> testing code that caused crashes and restarts:
>
> > Do you have a quick repro recipe?
>
> Here's something related to what I hit that time:
>
> diff --git a/src/backend/optimizer/plan/subselect.c b/src/backend/optimizer/plan/subselect.c
> index 052263aea6..d43a7c7bcb 100644
> --- a/src/backend/optimizer/plan/subselect.c
> +++ b/src/backend/optimizer/plan/subselect.c
> @@ -2188,6 +2188,7 @@ SS_charge_for_initplans(PlannerInfo *root, RelOptInfo *final_rel)
> void
> SS_attach_initplans(PlannerInfo *root, Plan *plan)
> {
> + Assert(root->init_plans == NIL);
> plan->initPlan = root->init_plans;
> }
>
> You won't get through initdb with this, but if you install this change
> into a successfully init'd database and then "make installcheck-parallel",
> it will crash and then fail to recover, at least a lot of the time.
Ah, that allowed me to reproduce. Thanks.
Took me a bit to understand how we actually get into this situation. A PRUNE
record for relation+block that doesn't exist during recovery. That doesn't
commonly happen outside of PITR or such, because we obviously need a block
with content to generate the PRUNE. The way it does happen here, is that the
relation is vacuumed and then truncated. Then we crash. Thus we end up with a
PRUNE record for a block that doesn't exist on disk.
Which is also why the test is quite timing sensitive.
Seems like it'd be good to have a test that covers this scenario. There's
plenty code around it that doesn't currently get exercised.
None of the existing tests seem like a great fit. I guess it could be added to
013_crash_restart, but that really focuses on something else.
So I guess I'll write a 036_notsureyet.pl...
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 5+ messages in thread
end of thread, other threads:[~2023-04-11 22:03 UTC | newest]
Thread overview: 5+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2020-03-17 18:16 [PATCH v16 03/10] Add tests on pg_ls_dir before changing it Justin Pryzby <[email protected]>
2023-04-11 18:48 Assertion being hit during WAL replay Tom Lane <[email protected]>
2023-04-11 19:56 ` Re: Assertion being hit during WAL replay Andres Freund <[email protected]>
2023-04-11 20:54 ` Re: Assertion being hit during WAL replay Tom Lane <[email protected]>
2023-04-11 22:03 ` Re: Assertion being hit during WAL replay Andres Freund <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox