Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1t2jAy-006cOW-75 for pgsql-admin@arkaria.postgresql.org; Mon, 21 Oct 2024 03:33:48 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1t2jAw-003v6w-HJ for pgsql-admin@arkaria.postgresql.org; Mon, 21 Oct 2024 03:33:46 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1t2jAw-003v6n-5g for pgsql-admin@lists.postgresql.org; Mon, 21 Oct 2024 03:33:46 +0000 Received: from mail-pf1-x436.google.com ([2607:f8b0:4864:20::436]) by magus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94.2) (envelope-from ) id 1t2jAu-0029cd-DQ for pgsql-admin@postgresql.org; Mon, 21 Oct 2024 03:33:45 +0000 Received: by mail-pf1-x436.google.com with SMTP id d2e1a72fcca58-71e681bc315so2485325b3a.0 for ; Sun, 20 Oct 2024 20:33:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1729481622; x=1730086422; darn=postgresql.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=XGvb4G446r3M3qagJsOMVXnFTLmRGo6hsjpcY1loLFQ=; b=jqEEO9TnIb9B4hlGBH7VJNhgYdMTWHs+KnuNB0c/hZWSq9sUiB20kxWw+Adn3UaQnN bqFv9US7rO3OOgjrJs5vZebWnBd50SJTD6EwvR3Qnp2E7rl5waaiKPVBMXcSFiqQj+k6 mRlMNkO8Tuy+yEyK/EEABmauJ4MOUXXRX/IgFhp9+ZpawwetQ8dE03D2wlom/Mf0cvPT ZCQTKX6M/qE/ehcjsHQrH67Pj4G6l7l9nB9NoT70hkA/46ZUTnE0XxvGfNbP+O+B8VVk jZM+RCz6PFTuwaiZIISq69tYkrHgs0Rn266ZFrpvBMeQ+sAfYuKpzH4FWBL584dSE4BC vDsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729481622; x=1730086422; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=XGvb4G446r3M3qagJsOMVXnFTLmRGo6hsjpcY1loLFQ=; b=HFo665SbrASe5JIxTk7xF8mYXcbIKvV2fxq8uWpk1nuWeXf48Q/oTUYQt74fO+2vF6 6scPhpwiZXUM+j7Y0hDpdLKxl0p/ipPlPXp3g+BxGtG4t+WPuIOJJ7Z9TwIZm2K7QTQ7 eLYXXWPMPZCfZWSgIP9pC1ZGkih6FaftaxSL3nGXzri6z1k7t6qx0hETtXUG9hPwOZ9t AXEl7KOi3rLLPn967wwUoUAlun4mnMlG29G7N6KhgBgXh2C6Fr1UISLtIncNqiTDBJ80 4tSu+K46/GiZDr0BlNVq2jeTQ6mtHPL/TeVnZm4g5eHcHxlY7Gdz4iHkRwtFZLAHEaYD HuVQ== X-Gm-Message-State: AOJu0YyAIX+/lfdrVfdoOIpHVpkjWL6Zd8qM1btTazLzSJMPn4m7v6ys AMRyjQ0v0AeF9zGnKLVnk3TJwRagiGXWi2TLfwVlrKa2pQtYW0mfOviLYwcm3FaPIhIVJULtaMo 8Xo3to5oO+Zgq1HLpDQNNqe8FgiI= X-Google-Smtp-Source: AGHT+IHYUNzycFzdAXRqBlgqPe1A3q4mF0z/e1Ke4MgKkkljfwM3+hNiw0yUlCrap5gbXB7wXEY2Pn1TaE9aOzsz9Ho= X-Received: by 2002:a05:6a00:1493:b0:71e:667c:8384 with SMTP id d2e1a72fcca58-71ea4259bcemr15453008b3a.9.1729481622400; Sun, 20 Oct 2024 20:33:42 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Muhammad Ikram Date: Mon, 21 Oct 2024 08:33:31 +0500 Message-ID: Subject: Re: Verify or dump pgdata files To: Murthy Nunna Cc: "pgsql-admin@postgresql.org" Content-Type: multipart/alternative; boundary="000000000000bfba2f0624f451f8" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --000000000000bfba2f0624f451f8 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi, Could you use diff ? e.g. diff -qr /var/lib/pgsql/15/data /tmp/data Or pg_checksums --verify --data-dir=3D/pgdata (for checksum enabled databases= ) or do checksum on source and target and take diff between two checksum generated files e.g. cd /pgdata find . -type f -exec sha256sum {} \; > /tmp/source_checksums.txt Regards, Ikram On Sun, Oct 20, 2024 at 4:45=E2=80=AFAM Murthy Nunna wrot= e: > Hi, > > > > We are currently using pg_waldump to verify integrity of WAL files after > archiving them in external storage. If the WAL is incomplete pg_waldump > returns error, which is great. > > > > Similarly, I am wondering if there is any utility that can be used agains= t > data files in /pgdata. Just to verify after =E2=80=9Crsync=E2=80=9D of /p= gdata. > > > > Thanks in advance for your response. > > > > > > > > > --000000000000bfba2f0624f451f8 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi,

Could you use diff ? e.g.

diff -qr /var/lib/pgsql/15/data /tmp/data
Or

pg_checksums --verify --data-dir=3D= /pgdata=C2=A0 (for checksum enabled databases)

or = do checksum on source and target and take diff between two checksum generat= ed files e.g.

cd /pgdata
find . -type f -exec s= ha256sum {} \; > /tmp/source_checksums.txt

=
Regards,
Ikram


=
= On Sun, Oct 20, 2024 at 4:45=E2=80=AFAM Murthy Nunna <mnunna@fnal.gov> wrote:

Hi,

=C2=A0<= /span>

We are currently usin= g pg_waldump to verify integrity of WAL files after archiving them in exter= nal storage. If the WAL is incomplete pg_waldump returns error, which is gr= eat.

=C2=A0<= /span>

Similarly, I am wonde= ring if there is any utility that can be used against data files in /pgdata= . Just to verify after =E2=80=9Crsync=E2=80=9D of /pgdata.

=C2=A0<= /span>

Thanks in advance for= your response.

=C2=A0<= /span>

=C2=A0<= /span>

=C2=A0<= /span>

=C2=A0<= /span>

--000000000000bfba2f0624f451f8--