pg.ddx.io pgsql-bugs@postgresql.org mailing list archive
help / color / mirror / Atom feedBUG #16484: pg_regress fails with --outputdir parameter
8+ messages / 4 participants
[nested] [flat]
Commits from this thread:
2020-06-13 e78900afd2 Create by default sql/ and expected/ for output directory in pg_regress
* BUG #16484: pg_regress fails with --outputdir parameter
@ 2020-06-08 05:45 PG Bug reporting form <noreply@postgresql.org>
0 siblings, 1 reply; 8+ messages in thread
From: PG Bug reporting form @ 2020-06-08 05:45 UTC (permalink / raw)
To: pgsql-bugs@lists.postgresql.org; +Cc: r.zharkov@postgrespro.ru
The following bug has been logged on the website:
Bug reference: 16484
Logged by: Roman Zharkov
Email address: r.zharkov@postgrespro.ru
PostgreSQL version: 13beta1
Operating system: linux, windows
Description:
Hello,
pg_regress fails when i try to change the output directory with
"--outputdir" parameter.
This happens bacause pg_regress creates the output directory, but doesn't
create sql and expected subdirectories.
Here is an example:
$ ./pg_regress int8 --dlpath=.
(using postmaster on Unix socket, default port)
============== dropping database "regression" ==============
DROP DATABASE
============== creating database "regression" ==============
CREATE DATABASE
ALTER DATABASE
============== running regression test queries ==============
test int8 ... ok 122 ms
=====================
All 1 tests passed.
=====================
$ ./pg_regress int8 --dlpath=. --outputdir=~regress_output
(using postmaster on Unix socket, default port)
pg_regress: could not open file "~regress_output/sql/largeobject.sql" for
writing: Нет такого файла или каталога
$ mkdir ~regress_output/sql
$ ./pg_regress int8 --dlpath=. --outputdir=~regress_output
(using postmaster on Unix socket, default port)
pg_regress: could not open file
"~regress_output/expected/create_function_2.out" for writing: Нет такого
файла или каталога
$ mkdir ~regress_output/expected
$ ./pg_regress int8 --dlpath=. --outputdir=~regress_output
(using postmaster on Unix socket, default port)
============== dropping database "regression" ==============
DROP DATABASE
============== creating database "regression" ==============
CREATE DATABASE
ALTER DATABASE
============== running regression test queries ==============
test int8 ... ok 174 ms
=====================
All 1 tests passed.
=====================
This patch can fix the issue:
inline patch 1: src/test/regress/pg_regress.c (+0/-0 lines) download inline-1.patch
diff --git a/src/test/regress/pg_regress.c b/src/test/regress/pg_regress.c
index 38b2b1e8e1b..1eb1122d237 100644
--- a/src/test/regress/pg_regress.c
+++ b/src/test/regress/pg_regress.c
@@ -465,6 +465,7 @@ convert_sourcefiles_in(const char *source_subdir, const
char *dest_dir, const ch
{
char testtablespace[MAXPGPATH];
char indir[MAXPGPATH];
+ char output_subdir[MAXPGPATH];
struct stat st;
int ret;
char **name;
@@ -473,6 +474,11 @@ convert_sourcefiles_in(const char *source_subdir, const
char *dest_dir, const ch
snprintf(indir, MAXPGPATH, "%s/%s", inputdir, source_subdir);
+ /* Create dest_subdir directory if it not exists */
+ snprintf(output_subdir, MAXPGPATH, "%s/%s", dest_dir, dest_subdir);
+ if (!directory_exists(output_subdir))
+ make_directory(output_subdir);
+
/* Check that indir actually exists and is a directory */
ret = stat(indir, &st);
if (ret != 0 || !S_ISDIR(st.st_mode))
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: BUG #16484: pg_regress fails with --outputdir parameter
@ 2020-06-08 09:09 Daniel Gustafsson <daniel@yesql.se>
parent: PG Bug reporting form <noreply@postgresql.org>
0 siblings, 1 reply; 8+ messages in thread
From: Daniel Gustafsson @ 2020-06-08 09:09 UTC (permalink / raw)
To: r.zharkov@postgrespro.ru; pgsql-bugs@lists.postgresql.org
> On 8 Jun 2020, at 07:45, PG Bug reporting form <noreply@postgresql.org> wrote:
> Hello,
> pg_regress fails when i try to change the output directory with
> "--outputdir" parameter.
> This happens bacause pg_regress creates the output directory, but doesn't
> create sql and expected subdirectories.
Nice catch, testing the attached patch does indeed make pg_regress run without
errors if outputdir doesn't exist.
While looking at it though, I noticed that we nearby use stat() in a single
place while the rest use check_directory(). Is there a reason to not use
check_directory consistently as per the attached diff?
cheers ./daniel
Attachments:
pg_regress_stat.diff (application/octet-stream, 832B) download
diff --git a/src/test/regress/pg_regress.c b/src/test/regress/pg_regress.c
index 38b2b1e8e1..ed0e3c780d 100644
--- a/src/test/regress/pg_regress.c
+++ b/src/test/regress/pg_regress.c
@@ -465,8 +465,6 @@ convert_sourcefiles_in(const char *source_subdir, const char *dest_dir, const ch
{
char testtablespace[MAXPGPATH];
char indir[MAXPGPATH];
- struct stat st;
- int ret;
char **name;
char **names;
int count = 0;
@@ -474,8 +472,7 @@ convert_sourcefiles_in(const char *source_subdir, const char *dest_dir, const ch
snprintf(indir, MAXPGPATH, "%s/%s", inputdir, source_subdir);
/* Check that indir actually exists and is a directory */
- ret = stat(indir, &st);
- if (ret != 0 || !S_ISDIR(st.st_mode))
+ if (!directory_exists(indir))
{
/*
* No warning, to avoid noise in tests that do not have these
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: BUG #16484: pg_regress fails with --outputdir parameter
@ 2020-06-09 05:33 Michael Paquier <michael@paquier.xyz>
parent: Daniel Gustafsson <daniel@yesql.se>
0 siblings, 1 reply; 8+ messages in thread
From: Michael Paquier @ 2020-06-09 05:33 UTC (permalink / raw)
To: Daniel Gustafsson <daniel@yesql.se>; +Cc: r.zharkov@postgrespro.ru; pgsql-bugs@lists.postgresql.org
On Mon, Jun 08, 2020 at 11:09:42AM +0200, Daniel Gustafsson wrote:
> Nice catch, testing the attached patch does indeed make pg_regress run without
> errors if outputdir doesn't exist.
No objections to that. That seems more useful than having to recreate
them before running any custom script specifying --outputdir.
> While looking at it though, I noticed that we nearby use stat() in a single
> place while the rest use check_directory(). Is there a reason to not use
> check_directory consistently as per the attached diff?
Indeed. And two extra things are that you can just simplify
src/bin/pg_upgrade/test.sh and src/tools/msvc/vcregress.pl so as they
don't create manually those repositories. This report is not really
annoying for the average user in my opinion, so there may be little
point in backpatching this stuff.
And this leads me to the attached.
--
Michael
Attachments:
pg_regress_stat_v3.diff (text/x-diff, 1.9K) download
diff --git a/src/bin/pg_upgrade/test.sh b/src/bin/pg_upgrade/test.sh
index 10a28d8133..7ff06de6d1 100644
--- a/src/bin/pg_upgrade/test.sh
+++ b/src/bin/pg_upgrade/test.sh
@@ -106,8 +106,6 @@ outputdir="$temp_root/regress"
EXTRA_REGRESS_OPTS="$EXTRA_REGRESS_OPTS --outputdir=$outputdir"
export EXTRA_REGRESS_OPTS
mkdir "$outputdir"
-mkdir "$outputdir"/sql
-mkdir "$outputdir"/expected
mkdir "$outputdir"/testtablespace
logdir=`pwd`/log
diff --git a/src/test/regress/pg_regress.c b/src/test/regress/pg_regress.c
index 38b2b1e8e1..b29de5edd3 100644
--- a/src/test/regress/pg_regress.c
+++ b/src/test/regress/pg_regress.c
@@ -465,17 +465,20 @@ convert_sourcefiles_in(const char *source_subdir, const char *dest_dir, const ch
{
char testtablespace[MAXPGPATH];
char indir[MAXPGPATH];
- struct stat st;
- int ret;
+ char test_subdir[MAXPGPATH];
char **name;
char **names;
int count = 0;
snprintf(indir, MAXPGPATH, "%s/%s", inputdir, source_subdir);
+ /* Create dest_subdir directory if it not exists */
+ snprintf(test_subdir, MAXPGPATH, "%s/%s", dest_dir, dest_subdir);
+ if (!directory_exists(test_subdir))
+ make_directory(test_subdir);
+
/* Check that indir actually exists and is a directory */
- ret = stat(indir, &st);
- if (ret != 0 || !S_ISDIR(st.st_mode))
+ if (!directory_exists(indir))
{
/*
* No warning, to avoid noise in tests that do not have these
diff --git a/src/tools/msvc/vcregress.pl b/src/tools/msvc/vcregress.pl
index 4a53a004b7..3365ee578c 100644
--- a/src/tools/msvc/vcregress.pl
+++ b/src/tools/msvc/vcregress.pl
@@ -571,8 +571,6 @@ sub upgradecheck
my $outputdir = "$tmp_root/regress";
my @EXTRA_REGRESS_OPTS = ("--outputdir=$outputdir");
mkdir "$outputdir" || die $!;
- mkdir "$outputdir/sql" || die $!;
- mkdir "$outputdir/expected" || die $!;
mkdir "$outputdir/testtablespace" || die $!;
my $logdir = "$topdir/src/bin/pg_upgrade/log";
signature.asc (application/pgp-signature, 832B) download
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: BUG #16484: pg_regress fails with --outputdir parameter
@ 2020-06-09 05:55 Tom Lane <tgl@sss.pgh.pa.us>
parent: Michael Paquier <michael@paquier.xyz>
0 siblings, 1 reply; 8+ messages in thread
From: Tom Lane @ 2020-06-09 05:55 UTC (permalink / raw)
To: Michael Paquier <michael@paquier.xyz>; +Cc: Daniel Gustafsson <daniel@yesql.se>; r.zharkov@postgrespro.ru; pgsql-bugs@lists.postgresql.org
Michael Paquier <michael@paquier.xyz> writes:
> And this leads me to the attached.
That will create the output directory even if indir doesn't exist, which
seems likely to lead to bogus directories hanging around in places where
we don't need them --- plus it seems rather ugly to drop this into the
middle of the sequence concerned with indir. So shouldn't the new code
be after the exit for indir not existing?
regards, tom lane
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: BUG #16484: pg_regress fails with --outputdir parameter
@ 2020-06-09 08:28 Daniel Gustafsson <daniel@yesql.se>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 8+ messages in thread
From: Daniel Gustafsson @ 2020-06-09 08:28 UTC (permalink / raw)
To: Michael Paquier <michael@paquier.xyz>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; r.zharkov@postgrespro.ru, PostgreSQL mailing lists <pgsql-bugs@lists.postgresql.org>
> On 9 Jun 2020, at 07:55, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
> Michael Paquier <michael@paquier.xyz> writes:
>> And this leads me to the attached.
FWIW I agree with the extended cleanup you proposed but I also agree that it
doesn't seem worth to backpatch.
> That will create the output directory even if indir doesn't exist, which
> seems likely to lead to bogus directories hanging around in places where
> we don't need them --- plus it seems rather ugly to drop this into the
> middle of the sequence concerned with indir. So shouldn't the new code
> be after the exit for indir not existing?
Makes sense, +1 for moving it after indir processing.
Looking at the diff, I noticed this hunk:
inline patch 1: src/tools/msvc/vcregress.pl (+0/-2 lines) download inline-1.patch
--- a/src/tools/msvc/vcregress.pl
+++ b/src/tools/msvc/vcregress.pl
@@ -571,8 +571,6 @@ sub upgradecheck
my $outputdir = "$tmp_root/regress";
my @EXTRA_REGRESS_OPTS = ("--outputdir=$outputdir");
mkdir "$outputdir" || die $!;
- mkdir "$outputdir/sql" || die $!;
- mkdir "$outputdir/expected" || die $!;
mkdir "$outputdir/testtablespace" || die $!;
On Windows we don't want testtablespace to exist whilst on non-Windows we do.
I wonder if this is a logical copy-paste from test.sh which could be removed as
well?
cheers ./daniel
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: BUG #16484: pg_regress fails with --outputdir parameter
@ 2020-06-11 08:27 Michael Paquier <michael@paquier.xyz>
parent: Daniel Gustafsson <daniel@yesql.se>
0 siblings, 1 reply; 8+ messages in thread
From: Michael Paquier @ 2020-06-11 08:27 UTC (permalink / raw)
To: Daniel Gustafsson <daniel@yesql.se>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; r.zharkov@postgrespro.ru, PostgreSQL mailing lists <pgsql-bugs@lists.postgresql.org>
On Tue, Jun 09, 2020 at 10:28:32AM +0200, Daniel Gustafsson wrote:
>> On 9 Jun 2020, at 07:55, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> FWIW I agree with the extended cleanup you proposed but I also agree that it
> doesn't seem worth to backpatch.
Thanks.
>> That will create the output directory even if indir doesn't exist, which
>> seems likely to lead to bogus directories hanging around in places where
>> we don't need them --- plus it seems rather ugly to drop this into the
>> middle of the sequence concerned with indir. So shouldn't the new code
>> be after the exit for indir not existing?
>
> Makes sense, +1 for moving it after indir processing.
Makes sense. Done this way in the updated version attached.
> Looking at the diff, I noticed this hunk:
>
> --- a/src/tools/msvc/vcregress.pl
> +++ b/src/tools/msvc/vcregress.pl
> @@ -571,8 +571,6 @@ sub upgradecheck
> my $outputdir = "$tmp_root/regress";
> my @EXTRA_REGRESS_OPTS = ("--outputdir=$outputdir");
> mkdir "$outputdir" || die $!;
> - mkdir "$outputdir/sql" || die $!;
> - mkdir "$outputdir/expected" || die $!;
> mkdir "$outputdir/testtablespace" || die $!;
>
> On Windows we don't want testtablespace to exist whilst on non-Windows we do.
> I wonder if this is a logical copy-paste from test.sh which could be removed as
> well?
Yes, we could remove this part in vcregress.pl. And as you visibly
noted the cleanup of testtablespace is taken care of in
src/test/regress/GNUmakefile for the non-Windows case, while for MSVC
we do this work in pg_regress.c. I have left this path creation
intentionally actually because there is a patch to improve this area
in the works and we have to check this area of the code so leaving it
behind makes grepping for testtablespace harder to forget (did not get
back to it yet, but I am planning to do so):
https://www.postgresql.org/message-id/20200219.142519.437573253063431435.horikyota.ntt@gmail.com
--
Michael
Attachments:
pg_regress_stat_v4.patch (text/x-diff, 2.2K) download
diff --git a/src/bin/pg_upgrade/test.sh b/src/bin/pg_upgrade/test.sh
index 10a28d8133..7ff06de6d1 100644
--- a/src/bin/pg_upgrade/test.sh
+++ b/src/bin/pg_upgrade/test.sh
@@ -106,8 +106,6 @@ outputdir="$temp_root/regress"
EXTRA_REGRESS_OPTS="$EXTRA_REGRESS_OPTS --outputdir=$outputdir"
export EXTRA_REGRESS_OPTS
mkdir "$outputdir"
-mkdir "$outputdir"/sql
-mkdir "$outputdir"/expected
mkdir "$outputdir"/testtablespace
logdir=`pwd`/log
diff --git a/src/test/regress/pg_regress.c b/src/test/regress/pg_regress.c
index 38b2b1e8e1..3013a493c2 100644
--- a/src/test/regress/pg_regress.c
+++ b/src/test/regress/pg_regress.c
@@ -465,8 +465,7 @@ convert_sourcefiles_in(const char *source_subdir, const char *dest_dir, const ch
{
char testtablespace[MAXPGPATH];
char indir[MAXPGPATH];
- struct stat st;
- int ret;
+ char outdir_sub[MAXPGPATH];
char **name;
char **names;
int count = 0;
@@ -474,8 +473,7 @@ convert_sourcefiles_in(const char *source_subdir, const char *dest_dir, const ch
snprintf(indir, MAXPGPATH, "%s/%s", inputdir, source_subdir);
/* Check that indir actually exists and is a directory */
- ret = stat(indir, &st);
- if (ret != 0 || !S_ISDIR(st.st_mode))
+ if (!directory_exists(indir))
{
/*
* No warning, to avoid noise in tests that do not have these
@@ -491,6 +489,11 @@ convert_sourcefiles_in(const char *source_subdir, const char *dest_dir, const ch
snprintf(testtablespace, MAXPGPATH, "%s/testtablespace", outputdir);
+ /* Create the "dest" subdirectory if not present */
+ snprintf(outdir_sub, MAXPGPATH, "%s/%s", dest_dir, dest_subdir);
+ if (!directory_exists(outdir_sub))
+ make_directory(outdir_sub);
+
#ifdef WIN32
/*
diff --git a/src/tools/msvc/vcregress.pl b/src/tools/msvc/vcregress.pl
index 4a53a004b7..3365ee578c 100644
--- a/src/tools/msvc/vcregress.pl
+++ b/src/tools/msvc/vcregress.pl
@@ -571,8 +571,6 @@ sub upgradecheck
my $outputdir = "$tmp_root/regress";
my @EXTRA_REGRESS_OPTS = ("--outputdir=$outputdir");
mkdir "$outputdir" || die $!;
- mkdir "$outputdir/sql" || die $!;
- mkdir "$outputdir/expected" || die $!;
mkdir "$outputdir/testtablespace" || die $!;
my $logdir = "$topdir/src/bin/pg_upgrade/log";
signature.asc (application/pgp-signature, 832B) download
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: BUG #16484: pg_regress fails with --outputdir parameter
@ 2020-06-11 09:59 Daniel Gustafsson <daniel@yesql.se>
parent: Michael Paquier <michael@paquier.xyz>
0 siblings, 1 reply; 8+ messages in thread
From: Daniel Gustafsson @ 2020-06-11 09:59 UTC (permalink / raw)
To: Michael Paquier <michael@paquier.xyz>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; r.zharkov@postgrespro.ru, PostgreSQL mailing lists <pgsql-bugs@lists.postgresql.org>
> On 11 Jun 2020, at 10:27, Michael Paquier <michael@paquier.xyz> wrote:
> I have left this path creation intentionally actually because there is a patch
> to improve this area in the works and we have to check this area of the code so
> leaving it behind makes grepping for testtablespace harder to forget
Aha, I hadn't seen that patch, but I agree it makes sense to leave it for now
then. +1 on the attached.
cheers ./daniel
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: BUG #16484: pg_regress fails with --outputdir parameter
@ 2020-06-13 05:12 Michael Paquier <michael@paquier.xyz>
parent: Daniel Gustafsson <daniel@yesql.se>
0 siblings, 0 replies; 8+ messages in thread
From: Michael Paquier @ 2020-06-13 05:12 UTC (permalink / raw)
To: Daniel Gustafsson <daniel@yesql.se>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; r.zharkov@postgrespro.ru, PostgreSQL mailing lists <pgsql-bugs@lists.postgresql.org>
On Thu, Jun 11, 2020 at 11:59:20AM +0200, Daniel Gustafsson wrote:
> Aha, I hadn't seen that patch, but I agree it makes sense to leave it for now
> then. +1 on the attached.
Thanks. I have been through the patch once again, and applied it to
HEAD. Worth noting that we have the same number of sql/ and expected/
repositories in the tree with check-world before and after this patch
(1 extra sql/ path is generated, 4 extra ones for expected/).
--
Michael
Attachments:
signature.asc (application/pgp-signature, 832B) download
^ permalink raw reply [nested|flat] 8+ messages in thread
end of thread, other threads:[~2020-06-13 05:12 UTC | newest]
Thread overview: 8+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2020-06-08 05:45 BUG #16484: pg_regress fails with --outputdir parameter PG Bug reporting form <noreply@postgresql.org>
2020-06-08 09:09 ` Daniel Gustafsson <daniel@yesql.se>
2020-06-09 05:33 ` Michael Paquier <michael@paquier.xyz>
2020-06-09 05:55 ` Tom Lane <tgl@sss.pgh.pa.us>
2020-06-09 08:28 ` Daniel Gustafsson <daniel@yesql.se>
2020-06-11 08:27 ` Michael Paquier <michael@paquier.xyz>
2020-06-11 09:59 ` Daniel Gustafsson <daniel@yesql.se>
2020-06-13 05:12 ` Michael Paquier <michael@paquier.xyz>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox