Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1guHhH-000807-6J for pgsql-hackers@arkaria.postgresql.org; Thu, 14 Feb 2019 14:09:03 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1guHgH-0005ko-Vv for pgsql-hackers@arkaria.postgresql.org; Thu, 14 Feb 2019 14:08:01 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1guHgH-0005kh-OT for pgsql-hackers@lists.postgresql.org; Thu, 14 Feb 2019 14:08:01 +0000 Received: from gauss.credativ.com ([93.94.130.89]) by magus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1guHgE-0004K1-65 for pgsql-hackers@lists.postgresql.org; Thu, 14 Feb 2019 14:08:01 +0000 Received: from gauss.credativ.com (localhost [127.0.0.1]) by gauss.credativ.com (Postfix) with ESMTP id 18C3E1E2890; Thu, 14 Feb 2019 15:07:57 +0100 (CET) Received: from kohn.fritz.box (ppp-88-217-107-116.dynamic.mnet-online.de [88.217.107.116]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: mba@gauss.credativ.com) by gauss.credativ.com (Postfix) with ESMTPSA id C14451E1AEB; Thu, 14 Feb 2019 15:07:56 +0100 (CET) Message-ID: <1550153276.796.35.camel@credativ.de> Subject: [Patch] checksumming-related buglets in pg_verify_checksums/pg_basebackup TAP tests From: Michael Banck To: PostgreSQL Developers Cc: Michael Paquier , Magnus Hagander Date: Thu, 14 Feb 2019 15:07:56 +0100 Organization: credativ GmbH Content-Type: multipart/mixed; boundary="=-VZU7HG8iQysFFWKHW4c8" X-Mailer: Evolution 3.22.6-1+deb9u1 Mime-Version: 1.0 List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk --=-VZU7HG8iQysFFWKHW4c8 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 8bit Hi, while hacking on pg_verify_checksums and looking at hexdumps, I noticed that the TAP tests of pg_verify_checksums (and pg_basebackup from which it was copy-pasted) actually write "305c305c[...]" (i.e. literal backslashes and number 0s) instead of "000[...]" into the to-be- corrupted relfilenodes due to wrong quoting. This also revealed a second bug in the pg_basebackup test suite where the offset for the corruption in the second file was wrong, so it actually never got corrupted, and the tests only passed due to the above twice than expected number of written bytes. The write() probably overflowed into an adjacent block so that the total number of corrupted blocks was as expected by addident. Oops, my bad, patch attached. Michael -- Michael Banck Projektleiter / Senior Berater Tel.: +49 2166 9901-171 Fax: +49 2166 9901-100 Email: michael.banck@credativ.de credativ GmbH, HRB Mönchengladbach 12080 USt-ID-Nummer: DE204566209 Trompeterallee 108, 41189 Mönchengladbach Geschäftsführung: Dr. Michael Meskes, Jörg Folz, Sascha Heuer Unser Umgang mit personenbezogenen Daten unterliegt folgenden Bestimmungen: https://www.credativ.de/datenschutz --=-VZU7HG8iQysFFWKHW4c8 Content-Disposition: attachment; filename="taptest_checksum_fixes.patch" Content-Type: text/x-patch; name="taptest_checksum_fixes.patch"; charset="UTF-8" Content-Transfer-Encoding: base64 ZGlmZiAtLWdpdCBhL3NyYy9iaW4vcGdfYmFzZWJhY2t1cC90LzAxMF9wZ19iYXNlYmFja3VwLnBs IGIvc3JjL2Jpbi9wZ19iYXNlYmFja3VwL3QvMDEwX3BnX2Jhc2ViYWNrdXAucGwKaW5kZXggM2Ux YzM4NjNjNC4uMzM4NjlmZWNjOSAxMDA2NDQKLS0tIGEvc3JjL2Jpbi9wZ19iYXNlYmFja3VwL3Qv MDEwX3BnX2Jhc2ViYWNrdXAucGwKKysrIGIvc3JjL2Jpbi9wZ19iYXNlYmFja3VwL3QvMDEwX3Bn X2Jhc2ViYWNrdXAucGwKQEAgLTQ5OSw3ICs0OTksNyBAQCBteSAkYmxvY2tfc2l6ZSA9ICRub2Rl LT5zYWZlX3BzcWwoJ3Bvc3RncmVzJywgJ1NIT1cgYmxvY2tfc2l6ZTsnKTsKIHN5c3RlbV9vcl9i YWlsICdwZ19jdGwnLCAnLUQnLCAkcGdkYXRhLCAnc3RvcCc7CiBvcGVuICRmaWxlLCAnKzwnLCAi JHBnZGF0YS8kZmlsZV9jb3JydXB0MSI7CiBzZWVrKCRmaWxlLCAkcGFnZWhlYWRlcl9zaXplLCAw KTsKLXN5c3dyaXRlKCRmaWxlLCAnXDBcMFwwXDBcMFwwXDBcMFwwJyk7CitzeXN3cml0ZSgkZmls ZSwgIlwwXDBcMFwwXDBcMFwwXDBcMCIpOwogY2xvc2UgJGZpbGU7CiBzeXN0ZW1fb3JfYmFpbCAn cGdfY3RsJywgJy1EJywgJHBnZGF0YSwgJ3N0YXJ0JzsKIApAQCAtNTE4LDcgKzUxOCw3IEBAIGZv ciBteSAkaSAoMSAuLiA1KQogewogCW15ICRvZmZzZXQgPSAkcGFnZWhlYWRlcl9zaXplICsgJGkg KiAkYmxvY2tfc2l6ZTsKIAlzZWVrKCRmaWxlLCAkb2Zmc2V0LCAwKTsKLQlzeXN3cml0ZSgkZmls ZSwgJ1wwXDBcMFwwXDBcMFwwXDBcMCcpOworCXN5c3dyaXRlKCRmaWxlLCAiXDBcMFwwXDBcMFww XDBcMFwwIik7CiB9CiBjbG9zZSAkZmlsZTsKIHN5c3RlbV9vcl9iYWlsICdwZ19jdGwnLCAnLUQn LCAkcGdkYXRhLCAnc3RhcnQnOwpAQCAtNTM0LDggKzUzNCw4IEBAIHJtdHJlZSgiJHRlbXBkaXIv YmFja3VwX2NvcnJ1cHQyIik7CiAjIGluZHVjZSBjb3JydXB0aW9uIGluIGEgc2Vjb25kIGZpbGUK IHN5c3RlbV9vcl9iYWlsICdwZ19jdGwnLCAnLUQnLCAkcGdkYXRhLCAnc3RvcCc7CiBvcGVuICRm aWxlLCAnKzwnLCAiJHBnZGF0YS8kZmlsZV9jb3JydXB0MiI7Ci1zZWVrKCRmaWxlLCA0MDAwLCAw KTsKLXN5c3dyaXRlKCRmaWxlLCAnXDBcMFwwXDBcMFwwXDBcMFwwJyk7CitzZWVrKCRmaWxlLCAk cGFnZWhlYWRlcl9zaXplLCAwKTsKK3N5c3dyaXRlKCRmaWxlLCAiXDBcMFwwXDBcMFwwXDBcMFww Iik7CiBjbG9zZSAkZmlsZTsKIHN5c3RlbV9vcl9iYWlsICdwZ19jdGwnLCAnLUQnLCAkcGdkYXRh LCAnc3RhcnQnOwogCmRpZmYgLS1naXQgYS9zcmMvYmluL3BnX3ZlcmlmeV9jaGVja3N1bXMvdC8w MDJfYWN0aW9ucy5wbCBiL3NyYy9iaW4vcGdfdmVyaWZ5X2NoZWNrc3Vtcy90LzAwMl9hY3Rpb25z LnBsCmluZGV4IDUyNTBiNWE3MjguLjc0YWQ1YWQ3MjMgMTAwNjQ0Ci0tLSBhL3NyYy9iaW4vcGdf dmVyaWZ5X2NoZWNrc3Vtcy90LzAwMl9hY3Rpb25zLnBsCisrKyBiL3NyYy9iaW4vcGdfdmVyaWZ5 X2NoZWNrc3Vtcy90LzAwMl9hY3Rpb25zLnBsCkBAIC00NSw3ICs0NSw3IEBAIHN1YiBjaGVja19y ZWxhdGlvbl9jb3JydXB0aW9uCiAJIyBUaW1lIHRvIGNyZWF0ZSBzb21lIGNvcnJ1cHRpb24KIAlv cGVuIG15ICRmaWxlLCAnKzwnLCAiJHBnZGF0YS8kZmlsZV9jb3JydXB0ZWQiOwogCXNlZWsoJGZp bGUsICRwYWdlaGVhZGVyX3NpemUsIDApOwotCXN5c3dyaXRlKCRmaWxlLCAnXDBcMFwwXDBcMFww XDBcMFwwJyk7CisJc3lzd3JpdGUoJGZpbGUsICJcMFwwXDBcMFwwXDBcMFwwXDAiKTsKIAljbG9z ZSAkZmlsZTsKIAogCSMgQ2hlY2tzdW0gY2hlY2tzIG9uIHNpbmdsZSByZWxmaWxlbm9kZSBmYWls Cg== --=-VZU7HG8iQysFFWKHW4c8--