pg.ddx.io pgsql-admin@postgresql.org mailing list archive
help / color / mirror / Atom feedFwd: failed to setup barman backup when Posgres is running in Podman Container
4+ messages / 4 participants
[nested] [flat]
* Fwd: failed to setup barman backup when Posgres is running in Podman Container
@ 2024-01-10 16:14 duc hiep ha <haduchiep@googlemail.com>
2024-01-15 07:57 ` Re: failed to setup barman backup when Posgres is running in Podman Container =?Windows-1252?Q?Gabriel_Guillem_Barcel=F3_Soteras?= <gbarcelo@parlamentib.es>
2024-01-16 08:54 ` Re: failed to setup barman backup when Posgres is running in Podman Container Romas Kontrimas <r.kontrimas@gmail.com>
0 siblings, 2 replies; 4+ messages in thread
From: duc hiep ha @ 2024-01-10 16:14 UTC (permalink / raw)
To: pgsql-admin@lists.postgresql.org
Dear All,
I am trying to use Barman to back up PostgreSQL, which is running in a
Podman container. However, I encounter the error "WAL archive: FAILED," as
shown below:
#barman check testdb
Server testdb:
WAL archive: FAILED (please make sure WAL shipping is set up)
PostgreSQL: OK
superuser: OK
PostgreSQL streaming: OK
wal_level: OK
replication slot: OK
directories: OK
retention policy settings: OK
backup maximum age: OK (no last_backup_maximum_age provided)
compression settings: OK
failed backups: OK (there are 0 failed backups)
minimum redundancy requirements: OK (have 0 backups, expected at least 0)
pg_basebackup: OK
pg_basebackup compatible: OK
pg_basebackup supports tablespaces mapping: OK
pg_receivexlog: OK
pg_receivexlog compatible: OK
receive-wal running: OK
archiver errors: OK
I have identified that normally, when the PostgreSQL database is installed
on a regular VM (not in a container), the archive_command parameter under
postgresql.conf looks like this:
archive_command = 'rsync -a %p barman@barman-backup-server-ip
:/data/barman/main-db-server/incoming/%f'
However, when PostgreSQL is installed on a Podman container, I cannot use
the above archive_command because within the PostgreSQL container, it
doesn't understand rsync, ssh, and the barman user. That's why I have to
use a normal copy command like:
archive_command = 'cp -i %p /srv/archive/%f'
I then mounted this wal-files folder on
'/data/barman/main-db-server/incoming/%f'. As a result, all these wal_files
have the PostgreSQL ownership and not the barman ownership. The barman user
has no privileges to change these wal-files, which is why the WAL archive
is marked as FAILED.
Do you know how to solve this issue? do I have to install Barman and
PostgreSQL in the same Pod container in order to backup PostgreSQL
container? Or is there another solution to this problem?
Thank you for your support in advance.
Best,
Ha
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: failed to setup barman backup when Posgres is running in Podman Container
2024-01-10 16:14 Fwd: failed to setup barman backup when Posgres is running in Podman Container duc hiep ha <haduchiep@googlemail.com>
@ 2024-01-15 07:57 ` =?Windows-1252?Q?Gabriel_Guillem_Barcel=F3_Soteras?= <gbarcelo@parlamentib.es>
1 sibling, 0 replies; 4+ messages in thread
From: Gabriel Guillem Barceló Soteras @ 2024-01-15 07:57 UTC (permalink / raw)
To: duc hiep ha <haduchiep@googlemail.com>; pgsql-admin@lists.postgresql.org <pgsql-admin@lists.postgresql.org>
I don’t have experience with Barman but if in-container tooling is limited you may try the following:
* Map postgre’s uid:gid with same numbers as barman’s when mounting volumes. Check –volume idmap option https://docs.podman.io/en/latest/markdown/podman-run.1.html#volume-v-source-volume-host-dir-containe...
--uidmap and --gidmap may be useful too
* Run a Temporal or side container (root) that copies it for you.
* Run a postgresql server container with ssh and rsync packages, still postgres will have to know about ‘external’ barman user
From: duc hiep ha <haduchiep@googlemail.com>
Date: Thursday, 11 January 2024 at 10:13
To: pgsql-admin@lists.postgresql.org <pgsql-admin@lists.postgresql.org>
Subject: Fwd: failed to setup barman backup when Posgres is running in Podman Container
Dear All,
I am trying to use Barman to back up PostgreSQL, which is running in a Podman container. However, I encounter the error "WAL archive: FAILED," as shown below:
#barman check testdb
Server testdb:
WAL archive: FAILED (please make sure WAL shipping is set up)
PostgreSQL: OK
superuser: OK
PostgreSQL streaming: OK
wal_level: OK
replication slot: OK
directories: OK
retention policy settings: OK
backup maximum age: OK (no last_backup_maximum_age provided)
compression settings: OK
failed backups: OK (there are 0 failed backups)
minimum redundancy requirements: OK (have 0 backups, expected at least 0)
pg_basebackup: OK
pg_basebackup compatible: OK
pg_basebackup supports tablespaces mapping: OK
pg_receivexlog: OK
pg_receivexlog compatible: OK
receive-wal running: OK
archiver errors: OK
I have identified that normally, when the PostgreSQL database is installed on a regular VM (not in a container), the archive_command parameter under postgresql.conf looks like this:
archive_command = 'rsync -a %p barman@barman-backup-server-ip:/data/barman/main-db-server/incoming/%f'
However, when PostgreSQL is installed on a Podman container, I cannot use the above archive_command because within the PostgreSQL container, it doesn't understand rsync, ssh, and the barman user. That's why I have to use a normal copy command like:
archive_command = 'cp -i %p /srv/archive/%f'
I then mounted this wal-files folder on '/data/barman/main-db-server/incoming/%f'. As a result, all these wal_files have the PostgreSQL ownership and not the barman ownership. The barman user has no privileges to change these wal-files, which is why the WAL archive is marked as FAILED.
Do you know how to solve this issue? do I have to install Barman and PostgreSQL in the same Pod container in order to backup PostgreSQL container? Or is there another solution to this problem?
Thank you for your support in advance.
Best,
Ha
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: failed to setup barman backup when Posgres is running in Podman Container
2024-01-10 16:14 Fwd: failed to setup barman backup when Posgres is running in Podman Container duc hiep ha <haduchiep@googlemail.com>
@ 2024-01-16 08:54 ` Romas Kontrimas <r.kontrimas@gmail.com>
2024-01-16 11:43 ` Re: failed to setup barman backup when Posgres is running in Podman Container Holger Jakobs <holger@jakobs.com>
1 sibling, 1 reply; 4+ messages in thread
From: Romas Kontrimas @ 2024-01-16 08:54 UTC (permalink / raw)
To: duc hiep ha <haduchiep@googlemail.com>; +Cc: pgsql-admin@lists.postgresql.org
Hi,
If I understand correctly, you already have WAL streaming setup and it's in
OK state (setting in in conf streaming_archiver = on). Barman will
continuously receive and archive transaction logs. Why would you still need
standard archive_command?
https://docs.pgbarman.org/release/3.9.0/#wal-streaming
On Thu, 11 Jan 2024 at 11:13, duc hiep ha <haduchiep@googlemail.com> wrote:
>
> Dear All,
>
>
> I am trying to use Barman to back up PostgreSQL, which is running in a
> Podman container. However, I encounter the error "WAL archive: FAILED," as
> shown below:
>
> #barman check testdb
> Server testdb:
> WAL archive: FAILED (please make sure WAL shipping is set up)
> PostgreSQL: OK
> superuser: OK
> PostgreSQL streaming: OK
> wal_level: OK
> replication slot: OK
> directories: OK
> retention policy settings: OK
> backup maximum age: OK (no last_backup_maximum_age provided)
> compression settings: OK
> failed backups: OK (there are 0 failed backups)
> minimum redundancy requirements: OK (have 0 backups, expected at least 0)
> pg_basebackup: OK
> pg_basebackup compatible: OK
> pg_basebackup supports tablespaces mapping: OK
> pg_receivexlog: OK
> pg_receivexlog compatible: OK
> receive-wal running: OK
> archiver errors: OK
>
> I have identified that normally, when the PostgreSQL database is installed
> on a regular VM (not in a container), the archive_command parameter under
> postgresql.conf looks like this:
>
> archive_command = 'rsync -a %p barman@barman-backup-server-ip
> :/data/barman/main-db-server/incoming/%f'
>
> However, when PostgreSQL is installed on a Podman container, I cannot use
> the above archive_command because within the PostgreSQL container, it
> doesn't understand rsync, ssh, and the barman user. That's why I have to
> use a normal copy command like:
>
> archive_command = 'cp -i %p /srv/archive/%f'
>
> I then mounted this wal-files folder on
> '/data/barman/main-db-server/incoming/%f'. As a result, all these wal_files
> have the PostgreSQL ownership and not the barman ownership. The barman user
> has no privileges to change these wal-files, which is why the WAL archive
> is marked as FAILED.
>
> Do you know how to solve this issue? do I have to install Barman and
> PostgreSQL in the same Pod container in order to backup PostgreSQL
> container? Or is there another solution to this problem?
>
> Thank you for your support in advance.
>
> Best,
> Ha
>
>
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: failed to setup barman backup when Posgres is running in Podman Container
2024-01-10 16:14 Fwd: failed to setup barman backup when Posgres is running in Podman Container duc hiep ha <haduchiep@googlemail.com>
2024-01-16 08:54 ` Re: failed to setup barman backup when Posgres is running in Podman Container Romas Kontrimas <r.kontrimas@gmail.com>
@ 2024-01-16 11:43 ` Holger Jakobs <holger@jakobs.com>
0 siblings, 0 replies; 4+ messages in thread
From: Holger Jakobs @ 2024-01-16 11:43 UTC (permalink / raw)
To: pgsql-admin@lists.postgresql.org
Am 16.01.24 um 09:54 schrieb Romas Kontrimas:
> Hi,
>
> If I understand correctly, you already have WAL streaming setup and
> it's in OK state (setting in in conf streaming_archiver = on). Barman
> will continuously receive and archive transaction logs. Why would you
> still need standard archive_command?
> https://docs.pgbarman.org/release/3.9.0/#wal-streaming
>
>
>
> On Thu, 11 Jan 2024 at 11:13, duc hiep ha <haduchiep@googlemail.com>
> wrote:
>
>
> Dear All,
>
>
> I am trying to use Barman to back up PostgreSQL, which is running
> in a Podman container. However, I encounter the error "WAL
> archive: FAILED," as shown below:
>
> #barman check testdb
> Server testdb:
> WAL archive: FAILED (please make sure WAL shipping is set up)
> PostgreSQL: OK
> superuser: OK
> PostgreSQL streaming: OK
> wal_level: OK
> replication slot: OK
> directories: OK
> retention policy settings: OK
> backup maximum age: OK (no last_backup_maximum_age provided)
> compression settings: OK
> failed backups: OK (there are 0 failed backups)
> minimum redundancy requirements: OK (have 0 backups, expected
> at least 0)
> pg_basebackup: OK
> pg_basebackup compatible: OK
> pg_basebackup supports tablespaces mapping: OK
> pg_receivexlog: OK
> pg_receivexlog compatible: OK
> receive-wal running: OK
> archiver errors: OK
>
> I have identified that normally, when the PostgreSQL database is
> installed on a regular VM (not in a container), the
> archive_command parameter under postgresql.conf looks like this:
>
> archive_command = 'rsync -a %p
> barman@barman-backup-server-ip:/data/barman/main-db-server/incoming/%f'
>
> However, when PostgreSQL is installed on a Podman container, I
> cannot use the above archive_command because within the PostgreSQL
> container, it doesn't understand rsync, ssh, and the barman user.
> That's why I have to use a normal copy command like:
>
> archive_command = 'cp -i %p /srv/archive/%f'
>
> I then mounted this wal-files folder on
> '/data/barman/main-db-server/incoming/%f'. As a result, all these
> wal_files have the PostgreSQL ownership and not the barman
> ownership. The barman user has no privileges to change these
> wal-files, which is why the WAL archive is marked as FAILED.
>
> Do you know how to solve this issue? do I have to install Barman
> and PostgreSQL in the same Pod container in order to backup
> PostgreSQL container? Or is there another solution to this problem?
>
> Thank you for your support in advance.
>
> Best,
> Ha
>
> Why would you still need standard archive_command?
You need it only if the streaming secondary servers are offline for a
while and cannot stream old WAL files from the primary because the have
been deleted there.
Archive shipping can then come to the rescue, so that the standby server
can get the missing WAL file from the archive. Subsequently, it will
continue to stream directly from the primary.
Regards,
Holger
--
Holger Jakobs, Bergisch Gladbach, Tel. +49-178-9759012
Attachments:
[application/pgp-signature] OpenPGP_signature (202B, ../../1973cf63-3c87-c48e-f91e-89f39c455b7e@jakobs.com/3-OpenPGP_signature)
download
^ permalink raw reply [nested|flat] 4+ messages in thread
end of thread, other threads:[~2024-01-16 11:43 UTC | newest]
Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2024-01-10 16:14 Fwd: failed to setup barman backup when Posgres is running in Podman Container duc hiep ha <haduchiep@googlemail.com>
2024-01-15 07:57 ` =?Windows-1252?Q?Gabriel_Guillem_Barcel=F3_Soteras?= <gbarcelo@parlamentib.es>
2024-01-16 08:54 ` Romas Kontrimas <r.kontrimas@gmail.com>
2024-01-16 11:43 ` Holger Jakobs <holger@jakobs.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox