From: Stas Kelvich <s.kelvich@postgrespro.ru>
To: PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Subject: unused_oids script is broken with bsd sed
Date: Wed, 25 Apr 2018 13:22:41 +0300
Message-ID: <37D774E4-FE1F-437E-B3D2-593F314B7505@postgrespro.ru> (raw)
Hi.
BSD sed in macOS doesn't understand word boundary operator "\b". So after
372728b0d49 unused_oids doesn’t match all oids in new *.dat files marking
all of them as unused.
It is possible to write more portable regexps, e.g. change "\b" to
something like "^.*{*.*", but it seems easier for feature use to just
rewrite unused_oids in perl to match duplicate_oids. Also add in-place
complain about duplicates instead of running uniq through oids array.
--
Stas Kelvich
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
Attachments:0001-Rewrite-unused_oids-in-perl.patch (application/octet-stream, 3.1K) download
From 756aeca9f6bf48c795cb84de7a131e5db8a4b33e Mon Sep 17 00:00:00 2001
From: Stas Kelvich <stanconn@gmail.com>
Date: Wed, 25 Apr 2018 13:00:28 +0300
Subject: [PATCH] Rewrite unused_oids in perl
unused_oids script was broken with bsd sed after 372728b0d49. It was
possible to write more portable regexps but it seems easier to just rewrite
unused_oids in perl to match duplicate_oids. Also add in-place complain
about duplicates instead of running uniq through oids array.
---
src/include/catalog/unused_oids | 75 ++++++++++++++++++++++++-----------------
1 file changed, 45 insertions(+), 30 deletions(-)
diff --git a/src/include/catalog/unused_oids b/src/include/catalog/unused_oidsindex f71222d50d..c225cbaa70 100755--- a/src/include/catalog/unused_oids+++ b/src/include/catalog/unused_oids@@ -1,4 +1,4 @@-#!/bin/sh+#!/usr/bin/perl
#
# unused_oids
#
@@ -16,42 +16,57 @@
# run this script in src/include/catalog.
#
+use strict;+use warnings;-AWK="awk"+BEGIN+{+ @ARGV = (glob("pg_*.h"), glob("pg_*.dat"), qw(indexing.h toasting.h));+}-# Get FirstBootstrapObjectId from access/transam.h-FIRSTOBJECTID=`grep '#define[ ]*FirstBootstrapObjectId' ../access/transam.h | $AWK '{ print $3 }'`-export FIRSTOBJECTID+my $AWK = 'awk';+my $FirstObjectId =`grep '#define[ ]*FirstBootstrapObjectId' ../access/transam.h | $AWK '{ print \$3 }'`;+if ($FirstObjectId !~ /^\d+$/)+{+ die "Counld not find FirstBootstrapObjectId in ../access/transam.h";+}+my @oids = ($FirstObjectId);
# this part (down to the uniq step) should match the duplicate_oids script
# note: we exclude BKI_BOOTSTRAP relations since they are expected to have
# matching data entries in pg_class.dat and pg_type.dat
+while (<>)+{+ next if /^CATALOG\(.*BKI_BOOTSTRAP/;+ next+ unless /\boid *=> *'(\d+)'/+ || /^CATALOG\([^,]*, *(\d+).*BKI_ROWTYPE_OID\((\d+),/+ || /^CATALOG\([^,]*, *(\d+)/+ || /^DECLARE_INDEX\([^,]*, *(\d+)/+ || /^DECLARE_UNIQUE_INDEX\([^,]*, *(\d+)/+ || /^DECLARE_TOAST\([^,]*, *(\d+), *(\d+)/;+ push @oids, $1;+ push @oids, $2 if $2;+}-cat pg_*.h pg_*.dat toasting.h indexing.h |-egrep -v -e '^CATALOG\(.*BKI_BOOTSTRAP' | \-sed -n -e 's/.*\boid *=> *'\''\([0-9][0-9]*\)'\''.*$/\1/p' \- -e 's/^CATALOG([^,]*, *\([0-9][0-9]*\).*BKI_ROWTYPE_OID(\([0-9][0-9]*\),.*$/\1,\2/p' \- -e 's/^CATALOG([^,]*, *\([0-9][0-9]*\).*$/\1/p' \- -e 's/^DECLARE_INDEX([^,]*, *\([0-9][0-9]*\).*$/\1/p' \- -e 's/^DECLARE_UNIQUE_INDEX([^,]*, *\([0-9][0-9]*\).*$/\1/p' \- -e 's/^DECLARE_TOAST([^,]*, *\([0-9][0-9]*\), *\([0-9][0-9]*\).*$/\1,\2/p' | \-tr ',' '\n' | \-sort -n | \-uniq | \-$AWK '-BEGIN {- last = 0;-}-/^[0-9]/ {- if ($1 > last + 1) {- if ($1 > last + 2) {- print last + 1, "-", $1 - 1;- } else {- print last + 1;+my $prev_oid = 0;+foreach my $oid (sort { $a <=> $b } @oids)+{+ if ($oid > $prev_oid + 1)+ {+ if ($oid > $prev_oid + 2)+ {+ print "@{[ $prev_oid + 1 ]} - @{[ $oid - 1 ]}\n";+ }+ else+ {+ print "@{[ $prev_oid + 1 ]}\n";
}
}
- last = $1;+ elsif ($oid == $prev_oid)+ {+ print "Duplicate oid detected: $oid\n";+ exit 1;+ }+ $prev_oid = $oid;
}
-END {- print last + 1, "-", ENVIRON["FIRSTOBJECTID"]-1;-}'--
2.16.2
=
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: s.kelvich@postgrespro.ru
Subject: Re: unused_oids script is broken with bsd sed
In-Reply-To: <37D774E4-FE1F-437E-B3D2-593F314B7505@postgrespro.ru>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox