public inbox for [email protected]
help / color / mirror / Atom feedRe: pg_basebackup check vs Windows file path limits
4+ messages / 3 participants
[nested] [flat]
* Re: pg_basebackup check vs Windows file path limits
@ 2023-11-11 15:18 Andrew Dunstan <[email protected]>
0 siblings, 1 reply; 4+ messages in thread
From: Andrew Dunstan @ 2023-11-11 15:18 UTC (permalink / raw)
To: Alexander Lakhin <[email protected]>; Daniel Gustafsson <[email protected]>; +Cc: pgsql-hackers
Hi, Alexander
On 2023-11-11 Sa 08:00, Alexander Lakhin wrote:
> Hello Andrew,
>
> 08.07.2023 18:52, Andrew Dunstan wrote:
>>> Since this test is passing on HEAD which has slightly shorter paths, I'm wondering if we should change this:
>>>
>>> rename("$pgdata/pg_replslot", "$tempdir/pg_replslot")
>>> or BAIL_OUT "could not move $pgdata/pg_replslot";
>>> dir_symlink("$tempdir/pg_replslot", "$pgdata/pg_replslot")
>>> or BAIL_OUT "could not symlink to $pgdata/pg_replslot";
>>>
>>> to use the much shorter $sys_tempdir created a few lines below.
>>>
>> Pushed a tested fix along those lines.
>>
>
> Today I've started up my Windows VM to run some tests and discovered a
> test
> failure caused by that fix (e213de8e7):
> >meson test
> Ok: 246
> Expected Fail: 0
> Fail: 1
> Unexpected Pass: 0
> Skipped: 14
> Timeout: 0
>
> ...\010_pg_basebackup\log\regress_log_010_pg_basebackup.txt contains:
> [04:42:45.321](0.291s) Bail out! could not move
> T:\postgresql\build/testrun/pg_basebackup/010_pg_basebackup\data/t_010_pg_basebackup_main_data/pgdata/pg_replslot
>
> With a diagnostic print added before rename() in 010_pg_basebackup.pl,
> I see:
> rename("T:\postgresql\build/testrun/pg_basebackup/010_pg_basebackup\data/t_010_pg_basebackup_main_data/pgdata/pg_replslot",
> "C:\Users\User\AppData\Local\Temp\fGT76tZUWr/pg_replslot")
> That is, I have the postgres source tree and the user tempdir placed on
> different disks.
>
> perldoc on rename() says that it usually doesn't work across filesystem
> boundaries, so I think it's not a Windows-specific issue.
>
>
Hmm, maybe we should be using File::Copy::move() instead of rename().
The docco for that says:
If possible, move() will simply rename the file. Otherwise, it
copies the file to the new location and deletes the original. If an
error occurs during this copy-and-delete process, you may be left
with a (possibly partial) copy of the file under the destination
name.
Can you try it out?
cheers
andrew
--
Andrew Dunstan
EDB:https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: pg_basebackup check vs Windows file path limits
@ 2023-11-11 17:00 Alexander Lakhin <[email protected]>
parent: Andrew Dunstan <[email protected]>
0 siblings, 1 reply; 4+ messages in thread
From: Alexander Lakhin @ 2023-11-11 17:00 UTC (permalink / raw)
To: Andrew Dunstan <[email protected]>; Daniel Gustafsson <[email protected]>; +Cc: pgsql-hackers
11.11.2023 18:18, Andrew Dunstan wrote:
>
> Hmm, maybe we should be using File::Copy::move() instead of rename(). The docco for that says:
>
> If possible, move() will simply rename the file. Otherwise, it
> copies the file to the new location and deletes the original. If an
> error occurs during this copy-and-delete process, you may be left
> with a (possibly partial) copy of the file under the destination
> name.
Unfortunately, I've stumbled upon inability of File::Copy::move()
to move directories across filesystems, exactly as described here:
https://stackoverflow.com/questions/17628039/filecopy-move-directories-accross-drives-in-windows-not...
(I'm sorry for not looking above rename() where this stated explicitly:
# On Windows use the short location to avoid path length issues.
# Elsewhere use $tempdir to avoid file system boundary issues with moving.
So this issue affects Windows only.)
Best regards,
Alexander
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: pg_basebackup check vs Windows file path limits
@ 2023-11-12 14:09 Andrew Dunstan <[email protected]>
parent: Alexander Lakhin <[email protected]>
0 siblings, 0 replies; 4+ messages in thread
From: Andrew Dunstan @ 2023-11-12 14:09 UTC (permalink / raw)
To: Alexander Lakhin <[email protected]>; Daniel Gustafsson <[email protected]>; +Cc: pgsql-hackers
On 2023-11-11 Sa 12:00, Alexander Lakhin wrote:
> 11.11.2023 18:18, Andrew Dunstan wrote:
>>
>> Hmm, maybe we should be using File::Copy::move() instead of rename().
>> The docco for that says:
>>
>> If possible, move() will simply rename the file. Otherwise, it
>> copies the file to the new location and deletes the original. If an
>> error occurs during this copy-and-delete process, you may be left
>> with a (possibly partial) copy of the file under the destination
>> name.
>
> Unfortunately, I've stumbled upon inability of File::Copy::move()
> to move directories across filesystems, exactly as described here:
> https://stackoverflow.com/questions/17628039/filecopy-move-directories-accross-drives-in-windows-not...
>
> (I'm sorry for not looking above rename() where this stated explicitly:
> # On Windows use the short location to avoid path length issues.
> # Elsewhere use $tempdir to avoid file system boundary issues with moving.
> So this issue affects Windows only.)
>
>
*sigh*
A probable workaround is to use a temp directory on the same device the
test is building on. Just set it up and set your environment TEMPDIR to
point to it, and I think it will be OK (i.e. I havent tested it).
But that doesn't mean I'm not searching for a better solution. Maybe
Alvaro's suggestion nearby will help.
cheers
andrew
--
Andrew Dunstan
EDB:https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 4+ messages in thread
* [PATCH v51 07/10] Check for transaction block early in ExecRepack
@ 2026-04-03 15:23 Srinath Reddy Sadipiralla <[email protected]>
0 siblings, 0 replies; 4+ messages in thread
From: Srinath Reddy Sadipiralla @ 2026-04-03 15:23 UTC (permalink / raw)
Currently, executing REPACK (CONCURRENTLY) without a table name inside a
transaction block throws the error "REPACK CONCURRENTLY requires explicit
table name" instead of the expected transaction block error. This occurs
because ExecRepack() validates the parsed options and missing relation
before verifying the transaction state.
This behavior is inconsistent with other utility commands like VACUUM
,REINDEX, etc; which invoke PreventInTransactionBlock() at the very start
of their execution to properly reject execution inside user transactions
before validating targets.
Add PreventInTransactionBlock to the top of ExecRepack() to enforce the
transaction block restriction early. This prevents the user from fixing
a missing table error only to immediately hit a transaction block error,
and also ensures consistency with rest of the commands.
Author: Srinath Reddy Sadipiralla <[email protected]>
---
src/backend/commands/repack.c | 26 ++++++++++++--------------
1 file changed, 12 insertions(+), 14 deletions(-)
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index d6e446d582d..d4c1f0e7652 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -292,6 +292,18 @@ ExecRepack(ParseState *pstate, RepackStmt *stmt, bool isTopLevel)
MyProc->statusFlags |= PROC_IN_CONCURRENT_REPACK;
ProcGlobal->statusFlags[MyProc->pgxactoff] = MyProc->statusFlags;
LWLockRelease(ProcArrayLock);
+
+ /*
+ * Make sure we're not in a transaction block.
+ *
+ * The reason is that repack_setup_logical_decoding() could wait
+ * indefinitely for our XID to complete. (The deadlock detector would
+ * not recognize it because we'd be waiting for ourselves, i.e. no
+ * real lock conflict.) It would be possible to run in a transaction
+ * block if we had no XID, but this restriction is simpler for users
+ * to understand and we don't lose anything.
+ */
+ PreventInTransactionBlock(isTopLevel, "REPACK (CONCURRENTLY)");
}
/*
@@ -523,21 +535,7 @@ cluster_rel(RepackCommand cmd, Relation OldHeap, Oid indexOid,
* replica index OID.
*/
if (concurrent)
- {
- /*
- * Make sure we're not in a transaction block.
- *
- * The reason is that repack_setup_logical_decoding() could wait
- * indefinitely for our XID to complete. (The deadlock detector would
- * not recognize it because we'd be waiting for ourselves, i.e. no
- * real lock conflict.) It would be possible to run in a transaction
- * block if we had no XID, but this restriction is simpler for users
- * to understand and we don't lose anything.
- */
- PreventInTransactionBlock(isTopLevel, "REPACK (CONCURRENTLY)");
-
check_repack_concurrently_requirements(OldHeap, &ident_idx);
- }
/* Check for user-requested abort. */
CHECK_FOR_INTERRUPTS();
--
2.47.3
--r2slln3zpmilwu22
Content-Type: text/x-diff; charset=utf-8
Content-Disposition: attachment;
filename="v51-0008-Introduce-an-option-to-make-logical-replication-.patch"
^ permalink raw reply [nested|flat] 4+ messages in thread
end of thread, other threads:[~2026-04-03 15:23 UTC | newest]
Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2023-11-11 15:18 Re: pg_basebackup check vs Windows file path limits Andrew Dunstan <[email protected]>
2023-11-11 17:00 ` Alexander Lakhin <[email protected]>
2023-11-12 14:09 ` Andrew Dunstan <[email protected]>
2026-04-03 15:23 [PATCH v51 07/10] Check for transaction block early in ExecRepack Srinath Reddy Sadipiralla <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox