public inbox for [email protected]help / color / mirror / Atom feed
Verify or dump pgdata files 5+ messages / 4 participants [nested] [flat]
* Verify or dump pgdata files @ 2024-10-19 23:45 Murthy Nunna <[email protected]> 0 siblings, 2 replies; 5+ messages in thread From: Murthy Nunna @ 2024-10-19 23:45 UTC (permalink / raw) To: pgsql-admin Hi, We are currently using pg_waldump to verify integrity of WAL files after archiving them in external storage. If the WAL is incomplete pg_waldump returns error, which is great. Similarly, I am wondering if there is any utility that can be used against data files in /pgdata. Just to verify after "rsync" of /pgdata. Thanks in advance for your response. ^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: Verify or dump pgdata files @ 2024-10-21 03:33 Muhammad Ikram <[email protected]> parent: Murthy Nunna <[email protected]> 1 sibling, 1 reply; 5+ messages in thread From: Muhammad Ikram @ 2024-10-21 03:33 UTC (permalink / raw) To: Murthy Nunna <[email protected]>; +Cc: pgsql-admin Hi, Could you use diff ? e.g. diff -qr /var/lib/pgsql/15/data /tmp/data Or pg_checksums --verify --data-dir=/pgdata (for checksum enabled databases) or do checksum on source and target and take diff between two checksum generated files e.g. cd /pgdata find . -type f -exec sha256sum {} \; > /tmp/source_checksums.txt Regards, Ikram On Sun, Oct 20, 2024 at 4:45 AM Murthy Nunna <[email protected]> wrote: > Hi, > > > > We are currently using pg_waldump to verify integrity of WAL files after > archiving them in external storage. If the WAL is incomplete pg_waldump > returns error, which is great. > > > > Similarly, I am wondering if there is any utility that can be used against > data files in /pgdata. Just to verify after “rsync” of /pgdata. > > > > Thanks in advance for your response. > > > > > > > > > ^ permalink raw reply [nested|flat] 5+ messages in thread
* RE: Verify or dump pgdata files @ 2024-10-21 15:51 Murthy Nunna <[email protected]> parent: Muhammad Ikram <[email protected]> 0 siblings, 1 reply; 5+ messages in thread From: Murthy Nunna @ 2024-10-21 15:51 UTC (permalink / raw) To: Muhammad Ikram <[email protected]>; +Cc: pgsql-admin Hi Muhammad, Thank you for responding to my query. I don’t think checksum method will work in my situation because the source file would have changed by the time I run checksum. So, the checksum (or diff) will be different. I am doing rsync of active cluster. What I am looking for is physical (structural) verification of target file such as beginning block and end block, beginning of file and end of file etc. All I want to know is “completeness” and “readability” of the file by postgres. Thanks. From: Muhammad Ikram <[email protected]> Sent: Sunday, October 20, 2024 10:34 PM To: Murthy Nunna <[email protected]> Cc: [email protected] Subject: Re: Verify or dump pgdata files [EXTERNAL] – This message is from an external sender Hi, Could you use diff ? e.g. diff -qr /var/lib/pgsql/15/data /tmp/data Or pg_checksums --verify --data-dir=/pgdata (for checksum enabled databases) or do checksum on source and target and take diff between two checksum generated files e.g. cd /pgdata find . -type f -exec sha256sum {} \; > /tmp/source_checksums.txt Regards, Ikram On Sun, Oct 20, 2024 at 4:45 AM Murthy Nunna <[email protected]<mailto:[email protected]>> wrote: Hi, We are currently using pg_waldump to verify integrity of WAL files after archiving them in external storage. If the WAL is incomplete pg_waldump returns error, which is great. Similarly, I am wondering if there is any utility that can be used against data files in /pgdata. Just to verify after “rsync” of /pgdata. Thanks in advance for your response. ^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: Verify or dump pgdata files @ 2024-10-21 16:21 Ron Johnson <[email protected]> parent: Murthy Nunna <[email protected]> 0 siblings, 0 replies; 5+ messages in thread From: Ron Johnson @ 2024-10-21 16:21 UTC (permalink / raw) To: pgsql-admin Use PgBackRest or pg_basebackup. Not only do they correctly handle wal files, but do checksums. On Mon, Oct 21, 2024 at 11:51 AM Murthy Nunna <[email protected]> wrote: > Hi Muhammad, > > > > Thank you for responding to my query. > > > > I don’t think checksum method will work in my situation because the source > file would have changed by the time I run checksum. So, the checksum (or > diff) will be different. > > I am doing rsync of active cluster. > > > > What I am looking for is physical (structural) verification of target file > such as beginning block and end block, beginning of file and end of file > etc. All I want to know is “completeness” and “readability” of the file by > postgres. > > > > Thanks. > > > > > > *From:* Muhammad Ikram <[email protected]> > *Sent:* Sunday, October 20, 2024 10:34 PM > *To:* Murthy Nunna <[email protected]> > *Cc:* [email protected] > *Subject:* Re: Verify or dump pgdata files > > > > [EXTERNAL] – This message is from an external sender > > Hi, > > > > Could you use diff ? e.g. > > > > diff -qr /var/lib/pgsql/15/data /tmp/data > > > > Or > > > > pg_checksums --verify --data-dir=/pgdata (for checksum enabled databases) > > > > or do checksum on source and target and take diff between two checksum > generated files e.g. > > > > cd /pgdata > find . -type f -exec sha256sum {} \; > /tmp/source_checksums.txt > > > > > > Regards, > > Ikram > > > > > > > > On Sun, Oct 20, 2024 at 4:45 AM Murthy Nunna <[email protected]> wrote: > > Hi, > > > > We are currently using pg_waldump to verify integrity of WAL files after > archiving them in external storage. If the WAL is incomplete pg_waldump > returns error, which is great. > > > > Similarly, I am wondering if there is any utility that can be used against > data files in /pgdata. Just to verify after “rsync” of /pgdata. > > > > Thanks in advance for your response. > > > > > > > > > > -- Death to <Redacted>, and butter sauce. Don't boil me, I'm still alive. <Redacted> crustacean! ^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: Verify or dump pgdata files @ 2024-10-21 18:31 Alvaro Herrera <[email protected]> parent: Murthy Nunna <[email protected]> 1 sibling, 0 replies; 5+ messages in thread From: Alvaro Herrera @ 2024-10-21 18:31 UTC (permalink / raw) To: Murthy Nunna <[email protected]>; +Cc: pgsql-admin On 2024-Oct-19, Murthy Nunna wrote: > Similarly, I am wondering if there is any utility that can be used > against data files in /pgdata. Just to verify after "rsync" of > /pgdata. Maybe you could use pg_verifybackup for this, assuming you do your "rsync" with pg_basebackup. -- Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/ ^ permalink raw reply [nested|flat] 5+ messages in thread
end of thread, other threads:[~2024-10-21 18:31 UTC | newest] Thread overview: 5+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2024-10-19 23:45 Verify or dump pgdata files Murthy Nunna <[email protected]> 2024-10-21 03:33 ` Muhammad Ikram <[email protected]> 2024-10-21 15:51 ` Murthy Nunna <[email protected]> 2024-10-21 16:21 ` Ron Johnson <[email protected]> 2024-10-21 18:31 ` Alvaro Herrera <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox