Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.89) (envelope-from ) id 1f3PYi-0001xM-5N for pgsql-committers@arkaria.postgresql.org; Tue, 03 Apr 2018 17:17:24 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1f3PYh-0002M3-2E for pgsql-committers@arkaria.postgresql.org; Tue, 03 Apr 2018 17:17:23 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.89) (envelope-from ) id 1f3PYg-0002Lu-Qj for pgsql-committers@lists.postgresql.org; Tue, 03 Apr 2018 17:17:22 +0000 Received: from mail-lf0-x235.google.com ([2a00:1450:4010:c07::235]) by magus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1f3PYd-0004RN-11 for pgsql-committers@lists.postgresql.org; Tue, 03 Apr 2018 17:17:22 +0000 Received: by mail-lf0-x235.google.com with SMTP id u3-v6so21285242lff.5 for ; Tue, 03 Apr 2018 10:17:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hagander-net.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=ql4sgXpLYwcQkgIFz4FIMtcEoYDIaVeYkHhOhDAP0A4=; b=p00d3E7ZZRfEOxdifgdnaQWFKDvj1ShC4TV6oj3LKqNnc94ZYAnw6Aaud0gpYTaZcz dZIkzBU2ZbSKyMrFK0v/WLptl1dbytK5bavzQFollmPMYkbgAEZiTYt6FIjEFCtakeco /k14CzANYhM+sHnbNjWpUCFpML95DZUAnoe5UAcVzNRaXYCvYDQLcXhJehYu0vgg2mec hBZ/JVCsMaEfhcSrSXG5lFk8Y9AxjLMAIQ+0HlMNIR4Pon4daexdXcucgekUl307rrE3 TNg0WEHTI+9n5A5Ba4vYUTCpYBkiKEWYVgiXy2YbmUcOl8hJKtyFfFEtVZpqpJF0DjTw IsTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=ql4sgXpLYwcQkgIFz4FIMtcEoYDIaVeYkHhOhDAP0A4=; b=EgGmYQTtcBdDq2zFl2/i/Pq5lFDc/CJbYbOuNYOBNY/vpgioQxHjlEt5Yb+1KUsX3B WuA3tyAdJKeqXjm0tNmKVjwuWb+1DQyupTn0W7Ze/IEFLXIsuRCNXEnqBTsI4fFOE5iu Pl8SDiCNtqdD5Scic7rxj+gcd54BQFoNXC9b/h1GOhu3X6rwBm3shQEmuPV0tooO8CO5 Jgh3oJWWqUqnvQGF4RRU/KxvjCvp0duHMjcO3givvi6rtSiUyCI4uUlzT6x22NIIyR/x G35gs2v2kkC9vJCGa3JG5o6azfGDRsJhGSUcA7/Orix7fbLEql7LCeUxyxVUOmUdDC07 1z1Q== X-Gm-Message-State: ALQs6tAEISllmejae7r404nOZtTu1NRtttDwOOuR7vBNw8BoKFOmRIcp sBnjlnljFwEvIXkYvrdvUi7SJlQDtOgdv9VSkkhdp+EF X-Google-Smtp-Source: AIpwx49BPWWg9hvXgYKvSAHpUPkmSDvdVpXIjK8PgE7MjiYl0UJ6eEi43+tUGDjDnVXxs6yW4pcYSimqq2vQdjP7BFk= X-Received: by 10.46.134.89 with SMTP id i25mr6159763ljj.121.1522775837824; Tue, 03 Apr 2018 10:17:17 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a19:9748:0:0:0:0:0 with HTTP; Tue, 3 Apr 2018 10:17:16 -0700 (PDT) In-Reply-To: <20180403171356.v4ji5sfhbc4sggij@alap3.anarazel.de> References: <20180403171356.v4ji5sfhbc4sggij@alap3.anarazel.de> From: Magnus Hagander Date: Tue, 3 Apr 2018 19:17:16 +0200 Message-ID: Subject: Re: pgsql: Validate page level checksums in base backups To: Andres Freund , Michael Banck , David Steele Cc: PostgreSQL-development , pgsql-committers@lists.postgresql.org Content-Type: multipart/alternative; boundary="f4f5e80eea8ce306a80568f4e063" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk --f4f5e80eea8ce306a80568f4e063 Content-Type: text/plain; charset="UTF-8" On Tue, Apr 3, 2018 at 7:13 PM, Andres Freund wrote: > On 2018-04-03 11:52:26 +0000, Magnus Hagander wrote: > > Validate page level checksums in base backups > > > > When base backups are run over the replication protocol (for example > > using pg_basebackup), verify the checksums of all data blocks if > > checksums are enabled. If checksum failures are encountered, log them > > as warnings but don't abort the backup. > > > > This becomes the default behaviour in pg_basebackup (provided checksums > > are enabled on the server), so add a switch (-k) to disable the checks > > if necessary. > > > > Author: Michael Banck > > Reviewed-By: Magnus Hagander, David Steele > > Discussion: https://postgr.es/m/20180228180856.GE13784@ > nighthawk.caipicrew.dd-dns.de > > Hm, something isn't quite right with the test: > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm= > pogona&dt=2018-04-03%2016%3A10%3A01 > > # WARNING: 6 total checksum verification failures > # pg_basebackup: checksum error occured > # ' > # doesn't match '(?^s:^WARNING.*7 total checksum verification > failures)' > Yeah, there's clearly a second problem here. I'm stuck in meetings most of tonight, so I might not get to it right away. If someonen else figures it out meanwhile please do, otherwise I'll get to it as soon as I can. -- Magnus Hagander Me: https://www.hagander.net/ Work: https://www.redpill-linpro.com/ --f4f5e80eea8ce306a80568f4e063 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Tue, Apr 3, 2018 at 7:13 PM, Andres Freund <andres@anarazel.de= > wrote:
On 20= 18-04-03 11:52:26 +0000, Magnus Hagander wrote:
> Validate page level checksums in base backups
>
> When base backups are run over the replication protocol (for example > using pg_basebackup), verify the checksums of all data blocks if
> checksums are enabled. If checksum failures are encountered, log them<= br> > as warnings but don't abort the backup.
>
> This becomes the default behaviour in pg_basebackup (provided checksum= s
> are enabled on the server), so add a switch (-k) to disable the checks=
> if necessary.
>
> Author: Michael Banck
> Reviewed-By: Magnus Hagander, David Steele
> Discussion: https://pos= tgr.es/m/20180228180856.GE13784@nighthawk.caipicrew.dd-dns.de=

Hm, something isn't quite right with the test:
h= ttps://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=3Dpogona&a= mp;dt=3D2018-04-03%2016%3A10%3A01

# WARNING:=C2=A0 6 total checksum verification failures
# pg_basebackup: checksum error occured
# '
#=C2=A0 =C2=A0 =C2=A0doesn't match '(?^s:^WARNING.*7 total checksum= verification failures)'

Yeah, ther= e's clearly a second problem here.

I'm stu= ck in meetings most of tonight, so I might not get to it right away. If som= eonen else figures it out meanwhile please do, otherwise I'll get to it= as soon as I can.=C2=A0


--
=C2=A0Magnus Hagander
=C2=A0Me: https://www.hagander.net/
=C2=A0Work: <= a href=3D"http://www.redpill-linpro.com/" target=3D"_blank">https://www.red= pill-linpro.com/
--f4f5e80eea8ce306a80568f4e063--