pg.ddx.io pgsql-bugs@postgresql.org mailing list archive
help / color / mirror / Atom feedMisleading/inaccurate error message from pg_basebackup
8+ messages / 5 participants
[nested] [flat]
* Misleading/inaccurate error message from pg_basebackup
@ 2024-01-22 16:47 c@osss.net
2024-01-23 10:27 ` Re: Misleading/inaccurate error message from pg_basebackup Daniel Gustafsson <daniel@yesql.se>
2024-01-24 09:20 ` Re: Misleading/inaccurate error message from pg_basebackup Christoph Berg <myon@debian.org>
0 siblings, 2 replies; 8+ messages in thread
From: c@osss.net @ 2024-01-22 16:47 UTC (permalink / raw)
To: pgsql-bugs@lists.postgresql.org
pg_basebackup can throw an error which is inaccurate and misleading:
$ mkdir /var/lib/postgresql/14
$ ls -ld /var/lib/postgresql/14
drwxr-x--- 2 postgres postgres 6 Jan 22 16:08 /var/lib/postgresql/14
$ pg_basebackup --version
Error: /var/lib/postgresql/14/main is not accessible; please fix the directory permissions (/var/lib/postgresql/14/ should be world readable)
Reading that error suggests that I need to make the directory /var/lib/postgresql/14 world-readable, but doing so does not fix the problem, as the actual problem is that the main subdirectory does not exist:
$ chmod a+rx /var/lib/postgresql/14
$ ls -ld /var/lib/postgresql/14
drwxr-xr-x 2 postgres postgres 6 Jan 22 16:08 /var/lib/postgresql/14
$ pg_basebackup --version
Error: /var/lib/postgresql/14/main is not accessible; please fix the directory permissions (/var/lib/postgresql/14/ should be world readable)
In fact it does not need to be world readable at all - the subdirectory just needs to be created:
$ chmod o-rx /var/lib/postgresql/14
$ mkdir /var/lib/postgresql/14/main
$ ls -ld /var/lib/postgresql/14{,/main}
drwxr-x--- 3 postgres postgres 19 Jan 22 16:14 /var/lib/postgresql/14
drwxr-x--- 2 postgres postgres 6 Jan 22 16:14 /var/lib/postgresql/14/main
$ pg_basebackup --version
pg_basebackup (PostgreSQL) 14.10 (Ubuntu 14.10-1.pgdg20.04+1)
This check is being done in cases where it's unnecessary, as it shouldn't matter at all when running a simple --version or --help, anyways.
--
Regards,
Casey
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Misleading/inaccurate error message from pg_basebackup
2024-01-22 16:47 Misleading/inaccurate error message from pg_basebackup c@osss.net
@ 2024-01-23 10:27 ` Daniel Gustafsson <daniel@yesql.se>
2024-01-23 17:37 ` Re: Misleading/inaccurate error message from pg_basebackup Casey <c@osss.net>
1 sibling, 1 reply; 8+ messages in thread
From: Daniel Gustafsson @ 2024-01-23 10:27 UTC (permalink / raw)
To: c@osss.net; +Cc: pgsql-bugs@lists.postgresql.org
> On 22 Jan 2024, at 17:47, c@osss.net wrote:
> pg_basebackup can throw an error which is inaccurate and misleading:
> $ pg_basebackup --version
> Error: /var/lib/postgresql/14/main is not accessible; please fix the directory permissions (/var/lib/postgresql/14/ should be world readable)
>
> In fact it does not need to be world readable at all - the subdirectory just needs to be created:
> This check is being done in cases where it's unnecessary, as it shouldn't matter at all when running a simple --version or --help, anyways.
There is no such check before --version or --help, and no such check at all in
pg_basebackup. Whatever is raising that error probably isn't postgres or
pg_basebackup. When pg_basebackup logs an error it looks like this:
pg_basebackup: error: /var/lib/postgresql/14/main is not accessible
Something else is doing this on your system.
--
Daniel Gustafsson
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Misleading/inaccurate error message from pg_basebackup
2024-01-22 16:47 Misleading/inaccurate error message from pg_basebackup c@osss.net
2024-01-23 10:27 ` Re: Misleading/inaccurate error message from pg_basebackup Daniel Gustafsson <daniel@yesql.se>
@ 2024-01-23 17:37 ` Casey <c@osss.net>
2024-01-23 23:35 ` Re: Misleading/inaccurate error message from pg_basebackup Michael Paquier <michael@paquier.xyz>
0 siblings, 1 reply; 8+ messages in thread
From: Casey @ 2024-01-23 17:37 UTC (permalink / raw)
To: Daniel Gustafsson <daniel@yesql.se>; +Cc: pgsql-bugs@lists.postgresql.org
Hmm, this must be a problem with the Debian packaging then.
root@prime-or1-pg-truth-2:~# file /usr/bin/pg_basebackup
/usr/bin/pg_basebackup: symbolic link to ../share/postgresql-common/pg_wrapper
Do you know the correct place to file an appropriate report for the Debian package maintainers?
Thanks,
--
Casey
> On Jan 23, 2024, at 5:27 AM, Daniel Gustafsson <daniel@yesql.se> wrote:
>
>> On 22 Jan 2024, at 17:47, c@osss.net wrote:
>
>> pg_basebackup can throw an error which is inaccurate and misleading:
>
>> $ pg_basebackup --version
>> Error: /var/lib/postgresql/14/main is not accessible; please fix the directory permissions (/var/lib/postgresql/14/ should be world readable)
>>
>> In fact it does not need to be world readable at all - the subdirectory just needs to be created:
>
>> This check is being done in cases where it's unnecessary, as it shouldn't matter at all when running a simple --version or --help, anyways.
>
> There is no such check before --version or --help, and no such check at all in
> pg_basebackup. Whatever is raising that error probably isn't postgres or
> pg_basebackup. When pg_basebackup logs an error it looks like this:
>
> pg_basebackup: error: /var/lib/postgresql/14/main is not accessible
>
> Something else is doing this on your system.
>
> --
> Daniel Gustafsson
>
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Misleading/inaccurate error message from pg_basebackup
2024-01-22 16:47 Misleading/inaccurate error message from pg_basebackup c@osss.net
2024-01-23 10:27 ` Re: Misleading/inaccurate error message from pg_basebackup Daniel Gustafsson <daniel@yesql.se>
2024-01-23 17:37 ` Re: Misleading/inaccurate error message from pg_basebackup Casey <c@osss.net>
@ 2024-01-23 23:35 ` Michael Paquier <michael@paquier.xyz>
0 siblings, 0 replies; 8+ messages in thread
From: Michael Paquier @ 2024-01-23 23:35 UTC (permalink / raw)
To: Casey <c@osss.net>; +Cc: Daniel Gustafsson <daniel@yesql.se>; pgsql-bugs@lists.postgresql.org, Christoph Berg <myon@debian.org>
On Tue, Jan 23, 2024 at 12:37:57PM -0500, Casey wrote:
> Hmm, this must be a problem with the Debian packaging then.
>
> root@prime-or1-pg-truth-2:~# file /usr/bin/pg_basebackup
> /usr/bin/pg_basebackup: symbolic link to ../share/postgresql-common/pg_wrapper
>
> Do you know the correct place to file an appropriate report for the Debian package maintainers?
Christoph Berg would be the correct person. I am adding him in CC for
comments.
--
Michael
Attachments:
signature.asc (application/pgp-signature, 832B) download
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Misleading/inaccurate error message from pg_basebackup
2024-01-22 16:47 Misleading/inaccurate error message from pg_basebackup c@osss.net
@ 2024-01-24 09:20 ` Christoph Berg <myon@debian.org>
2024-01-24 12:48 ` Re: Misleading/inaccurate error message from pg_basebackup Christoph Berg <myon@debian.org>
1 sibling, 1 reply; 8+ messages in thread
From: Christoph Berg @ 2024-01-24 09:20 UTC (permalink / raw)
To: c@osss.net; +Cc: pgsql-bugs@lists.postgresql.org
Re: c@osss.net
> pg_basebackup can throw an error which is inaccurate and misleading:
>
> $ mkdir /var/lib/postgresql/14
Hi Casey,
what previous error/message lead you to create that directory? The
data directory should be existing before.
What does `pg_lsclusters` report?
Where is the actual data directory of that cluster?
Christoph
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Misleading/inaccurate error message from pg_basebackup
2024-01-22 16:47 Misleading/inaccurate error message from pg_basebackup c@osss.net
2024-01-24 09:20 ` Re: Misleading/inaccurate error message from pg_basebackup Christoph Berg <myon@debian.org>
@ 2024-01-24 12:48 ` Christoph Berg <myon@debian.org>
2024-01-29 17:40 ` Re: Misleading/inaccurate error message from pg_basebackup Christoph Berg <myon@debian.org>
0 siblings, 1 reply; 8+ messages in thread
From: Christoph Berg @ 2024-01-24 12:48 UTC (permalink / raw)
To: Casey Shobe <c@osss.net>; +Cc: PostgreSQL Bugs <pgsql-bugs@lists.postgresql.org>
Re: Casey Shobe
> Below is pasted my initial message, which gives more context and detail. Let me know if anything is still inclear after this. The context is that I use Patroni to run a multi-node cluster, and WAL-G creates a hidden directory within the wal directory which I did not initially notice when I otherwise emptied it before reinitializing a node after replacing disk for the data volume. This led to a fair bit of time wasted looking for the wrong problem:
I did reply to your initially message and all the questions are still
open.
Christoph
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Misleading/inaccurate error message from pg_basebackup
2024-01-22 16:47 Misleading/inaccurate error message from pg_basebackup c@osss.net
2024-01-24 09:20 ` Re: Misleading/inaccurate error message from pg_basebackup Christoph Berg <myon@debian.org>
2024-01-24 12:48 ` Re: Misleading/inaccurate error message from pg_basebackup Christoph Berg <myon@debian.org>
@ 2024-01-29 17:40 ` Christoph Berg <myon@debian.org>
2024-01-29 21:14 ` Re: Misleading/inaccurate error message from pg_basebackup Casey <c@osss.net>
0 siblings, 1 reply; 8+ messages in thread
From: Christoph Berg @ 2024-01-29 17:40 UTC (permalink / raw)
To: Casey <c@osss.net>; +Cc: PostgreSQL Bugs <pgsql-bugs@lists.postgresql.org>
Re: Casey
> I thought that I addressed your inquiries as best as I was able. Can you please clarify any remaining questions?
What did you do to make you believe that you had to "mkdir" in the
first place?
Also, please keep it on the list.
> > On Jan 24, 2024, at 6:48 AM, Christoph Berg <myon@debian.org> wrote:
> >
> > Re: Casey Shobe
> >> Below is pasted my initial message, which gives more context and detail. Let me know if anything is still inclear after this. The context is that I use Patroni to run a multi-node cluster, and WAL-G creates a hidden directory within the wal directory which I did not initially notice when I otherwise emptied it before reinitializing a node after replacing disk for the data volume. This led to a fair bit of time wasted looking for the wrong problem:
> >
> > I did reply to your initially message and all the questions are still
> > open.
> >
> > Christoph
Christoph
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Misleading/inaccurate error message from pg_basebackup
2024-01-22 16:47 Misleading/inaccurate error message from pg_basebackup c@osss.net
2024-01-24 09:20 ` Re: Misleading/inaccurate error message from pg_basebackup Christoph Berg <myon@debian.org>
2024-01-24 12:48 ` Re: Misleading/inaccurate error message from pg_basebackup Christoph Berg <myon@debian.org>
2024-01-29 17:40 ` Re: Misleading/inaccurate error message from pg_basebackup Christoph Berg <myon@debian.org>
@ 2024-01-29 21:14 ` Casey <c@osss.net>
0 siblings, 0 replies; 8+ messages in thread
From: Casey @ 2024-01-29 21:14 UTC (permalink / raw)
To: Christoph Berg <myon@debian.org>; +Cc: pgsql-bugs@lists.postgresql.org
I didn't believe I had to mkdir, those were just test cases to illustrate the problem in isolation. I had been trying to reinitialize a node after replacing disks used for the data volume, using Patroni. When that failed due to a pg_basebackup error, it removed the data directory.
To be clear, I'm mounting separate volumes at:
/var/lib/postgresql
/var/lib/postgresql/wal
The data and wal directories are a couple levels under those:
/var/lib/postgresql/14/main
/var/lib/postgresql/wal/14/main
So when Patroni ran into a pg_basebackup error, it removed /var/lib/postgresq/14/main and /var/lib/postgresql/14. It also does not log the specific generated pg_basebackup command. As I couldn't tell why it was erroring, I tried to recreate that command myself based on the configuration and defaults. When I did that, I didn't think about specifying the path for it as /usr/lib/postgresql/14/bin as I ought to have, but just relied on what was in my path, which turned out to be the wrapper script.
The actual problem turned out to be that I thought that I'd cleared out all the contents of the wal directory, but I'd inadvertently left a hidden file sitting in there. Anyways during the process of debugging this, I didn't have the database running, and didn't have the data directory existing. I wanted to look at pg_basebackup --help, and that would not work, throwing the error about the data directory not existing. I should have focused on the first part of the error message, that /var/lib/postgresq/14/main was not accessible, but instead I got distracted by the second part, telling me to fix the directory permissions on /var/lib/postgresql/14 making it world-readable. Well it didn't actually need to be world-readable, and we don't want it to be world-readable. Regardless, I tried making it world-readable, and was confused as to why pg_basebackup threw the same error message. Once I created the /main subdirectory, ignoring the complaint about world-readability, I was able to get a different error that pointed me to the actual problem:
pg_basebackup: error: directory "/var/lib/postgresql/wal/14/main" exists but is not empty
The point is that the error I ran into when the data directory (/main) did not exist under /var/lib/postgresq//14, is incorrect, and led me to being confused and wasting some time wondering what was wrong rather than getting to the actual problem. "please fix the directory permissions (/var/lib/postgresql/14/ should be world readable)" is misleading as there was no need to follow that instruction and it distracted from the more relevant and correct message printed just before it ("/var/lib/postgresql/14/main is not accessible"). Furthermore, pg_basebackup --help should ideally work regardless of that, as does the upstream binary.
Hope this helps,
--
Casey
> On Jan 29, 2024, at 11:40 AM, Christoph Berg <myon@debian.org> wrote:
>
> Re: Casey
>> I thought that I addressed your inquiries as best as I was able. Can you please clarify any remaining questions?
>
> What did you do to make you believe that you had to "mkdir" in the
> first place?
>
> Also, please keep it on the list.
>
>>> On Jan 24, 2024, at 6:48 AM, Christoph Berg <myon@debian.org> wrote:
>>>
>>> Re: Casey Shobe
>>>> Below is pasted my initial message, which gives more context and detail. Let me know if anything is still inclear after this. The context is that I use Patroni to run a multi-node cluster, and WAL-G creates a hidden directory within the wal directory which I did not initially notice when I otherwise emptied it before reinitializing a node after replacing disk for the data volume. This led to a fair bit of time wasted looking for the wrong problem:
>>>
>>> I did reply to your initially message and all the questions are still
>>> open.
>>>
>>> Christoph
>
> Christoph
^ permalink raw reply [nested|flat] 8+ messages in thread
end of thread, other threads:[~2024-01-29 21:14 UTC | newest]
Thread overview: 8+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2024-01-22 16:47 Misleading/inaccurate error message from pg_basebackup c@osss.net
2024-01-23 10:27 ` Daniel Gustafsson <daniel@yesql.se>
2024-01-23 17:37 ` Casey <c@osss.net>
2024-01-23 23:35 ` Michael Paquier <michael@paquier.xyz>
2024-01-24 09:20 ` Christoph Berg <myon@debian.org>
2024-01-24 12:48 ` Christoph Berg <myon@debian.org>
2024-01-29 17:40 ` Christoph Berg <myon@debian.org>
2024-01-29 21:14 ` Casey <c@osss.net>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox