public inbox for [email protected]
help / color / mirror / Atom feedVerify or dump pgdata files
5+ messages / 4 participants
[nested] [flat]
* Verify or dump pgdata files
@ 2024-10-19 23:45 Murthy Nunna <[email protected]>
2024-10-21 03:33 ` Re: Verify or dump pgdata files Muhammad Ikram <[email protected]>
2024-10-21 18:31 ` Re: Verify or dump pgdata files Alvaro Herrera <[email protected]>
0 siblings, 2 replies; 5+ messages in thread
From: Murthy Nunna @ 2024-10-19 23:45 UTC (permalink / raw)
To: pgsql-admin
Hi,
We are currently using pg_waldump to verify integrity of WAL files after archiving them in external storage. If the WAL is incomplete pg_waldump returns error, which is great.
Similarly, I am wondering if there is any utility that can be used against data files in /pgdata. Just to verify after "rsync" of /pgdata.
Thanks in advance for your response.
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: Verify or dump pgdata files
2024-10-19 23:45 Verify or dump pgdata files Murthy Nunna <[email protected]>
@ 2024-10-21 03:33 ` Muhammad Ikram <[email protected]>
2024-10-21 15:51 ` RE: Verify or dump pgdata files Murthy Nunna <[email protected]>
1 sibling, 1 reply; 5+ messages in thread
From: Muhammad Ikram @ 2024-10-21 03:33 UTC (permalink / raw)
To: Murthy Nunna <[email protected]>; +Cc: pgsql-admin
Hi,
Could you use diff ? e.g.
diff -qr /var/lib/pgsql/15/data /tmp/data
Or
pg_checksums --verify --data-dir=/pgdata (for checksum enabled databases)
or do checksum on source and target and take diff between two checksum
generated files e.g.
cd /pgdata
find . -type f -exec sha256sum {} \; > /tmp/source_checksums.txt
Regards,
Ikram
On Sun, Oct 20, 2024 at 4:45 AM Murthy Nunna <[email protected]> wrote:
> Hi,
>
>
>
> We are currently using pg_waldump to verify integrity of WAL files after
> archiving them in external storage. If the WAL is incomplete pg_waldump
> returns error, which is great.
>
>
>
> Similarly, I am wondering if there is any utility that can be used against
> data files in /pgdata. Just to verify after “rsync” of /pgdata.
>
>
>
> Thanks in advance for your response.
>
>
>
>
>
>
>
>
>
^ permalink raw reply [nested|flat] 5+ messages in thread
* RE: Verify or dump pgdata files
2024-10-19 23:45 Verify or dump pgdata files Murthy Nunna <[email protected]>
2024-10-21 03:33 ` Re: Verify or dump pgdata files Muhammad Ikram <[email protected]>
@ 2024-10-21 15:51 ` Murthy Nunna <[email protected]>
2024-10-21 16:21 ` Re: Verify or dump pgdata files Ron Johnson <[email protected]>
0 siblings, 1 reply; 5+ messages in thread
From: Murthy Nunna @ 2024-10-21 15:51 UTC (permalink / raw)
To: Muhammad Ikram <[email protected]>; +Cc: pgsql-admin
Hi Muhammad,
Thank you for responding to my query.
I don’t think checksum method will work in my situation because the source file would have changed by the time I run checksum. So, the checksum (or diff) will be different.
I am doing rsync of active cluster.
What I am looking for is physical (structural) verification of target file such as beginning block and end block, beginning of file and end of file etc. All I want to know is “completeness” and “readability” of the file by postgres.
Thanks.
From: Muhammad Ikram <[email protected]>
Sent: Sunday, October 20, 2024 10:34 PM
To: Murthy Nunna <[email protected]>
Cc: [email protected]
Subject: Re: Verify or dump pgdata files
[EXTERNAL] – This message is from an external sender
Hi,
Could you use diff ? e.g.
diff -qr /var/lib/pgsql/15/data /tmp/data
Or
pg_checksums --verify --data-dir=/pgdata (for checksum enabled databases)
or do checksum on source and target and take diff between two checksum generated files e.g.
cd /pgdata
find . -type f -exec sha256sum {} \; > /tmp/source_checksums.txt
Regards,
Ikram
On Sun, Oct 20, 2024 at 4:45 AM Murthy Nunna <[email protected]<mailto:[email protected]>> wrote:
Hi,
We are currently using pg_waldump to verify integrity of WAL files after archiving them in external storage. If the WAL is incomplete pg_waldump returns error, which is great.
Similarly, I am wondering if there is any utility that can be used against data files in /pgdata. Just to verify after “rsync” of /pgdata.
Thanks in advance for your response.
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: Verify or dump pgdata files
2024-10-19 23:45 Verify or dump pgdata files Murthy Nunna <[email protected]>
2024-10-21 03:33 ` Re: Verify or dump pgdata files Muhammad Ikram <[email protected]>
2024-10-21 15:51 ` RE: Verify or dump pgdata files Murthy Nunna <[email protected]>
@ 2024-10-21 16:21 ` Ron Johnson <[email protected]>
0 siblings, 0 replies; 5+ messages in thread
From: Ron Johnson @ 2024-10-21 16:21 UTC (permalink / raw)
To: pgsql-admin
Use PgBackRest or pg_basebackup. Not only do they correctly handle wal
files, but do checksums.
On Mon, Oct 21, 2024 at 11:51 AM Murthy Nunna <[email protected]> wrote:
> Hi Muhammad,
>
>
>
> Thank you for responding to my query.
>
>
>
> I don’t think checksum method will work in my situation because the source
> file would have changed by the time I run checksum. So, the checksum (or
> diff) will be different.
>
> I am doing rsync of active cluster.
>
>
>
> What I am looking for is physical (structural) verification of target file
> such as beginning block and end block, beginning of file and end of file
> etc. All I want to know is “completeness” and “readability” of the file by
> postgres.
>
>
>
> Thanks.
>
>
>
>
>
> *From:* Muhammad Ikram <[email protected]>
> *Sent:* Sunday, October 20, 2024 10:34 PM
> *To:* Murthy Nunna <[email protected]>
> *Cc:* [email protected]
> *Subject:* Re: Verify or dump pgdata files
>
>
>
> [EXTERNAL] – This message is from an external sender
>
> Hi,
>
>
>
> Could you use diff ? e.g.
>
>
>
> diff -qr /var/lib/pgsql/15/data /tmp/data
>
>
>
> Or
>
>
>
> pg_checksums --verify --data-dir=/pgdata (for checksum enabled databases)
>
>
>
> or do checksum on source and target and take diff between two checksum
> generated files e.g.
>
>
>
> cd /pgdata
> find . -type f -exec sha256sum {} \; > /tmp/source_checksums.txt
>
>
>
>
>
> Regards,
>
> Ikram
>
>
>
>
>
>
>
> On Sun, Oct 20, 2024 at 4:45 AM Murthy Nunna <[email protected]> wrote:
>
> Hi,
>
>
>
> We are currently using pg_waldump to verify integrity of WAL files after
> archiving them in external storage. If the WAL is incomplete pg_waldump
> returns error, which is great.
>
>
>
> Similarly, I am wondering if there is any utility that can be used against
> data files in /pgdata. Just to verify after “rsync” of /pgdata.
>
>
>
> Thanks in advance for your response.
>
>
>
>
>
>
>
>
>
>
--
Death to <Redacted>, and butter sauce.
Don't boil me, I'm still alive.
<Redacted> crustacean!
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: Verify or dump pgdata files
2024-10-19 23:45 Verify or dump pgdata files Murthy Nunna <[email protected]>
@ 2024-10-21 18:31 ` Alvaro Herrera <[email protected]>
1 sibling, 0 replies; 5+ messages in thread
From: Alvaro Herrera @ 2024-10-21 18:31 UTC (permalink / raw)
To: Murthy Nunna <[email protected]>; +Cc: pgsql-admin
On 2024-Oct-19, Murthy Nunna wrote:
> Similarly, I am wondering if there is any utility that can be used
> against data files in /pgdata. Just to verify after "rsync" of
> /pgdata.
Maybe you could use pg_verifybackup for this, assuming you do your
"rsync" with pg_basebackup.
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 5+ messages in thread
end of thread, other threads:[~2024-10-21 18:31 UTC | newest]
Thread overview: 5+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2024-10-19 23:45 Verify or dump pgdata files Murthy Nunna <[email protected]>
2024-10-21 03:33 ` Muhammad Ikram <[email protected]>
2024-10-21 15:51 ` Murthy Nunna <[email protected]>
2024-10-21 16:21 ` Ron Johnson <[email protected]>
2024-10-21 18:31 ` Alvaro Herrera <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox