From: Andrew Dunstan <andrew.dunstan@2ndquadrant.com>
Subject: Re: unused_oids script is broken with bsd sed
To: John Naylor <jcnaylor@gmail.com>, Stas Kelvich <s.kelvich@postgrespro.ru>
Cc: Tom Lane <tgl@sss.pgh.pa.us>,
 PostgreSQL Hackers <pgsql-hackers@postgresql.org>
References: <37D774E4-FE1F-437E-B3D2-593F314B7505@postgrespro.ru>
 <34a54c54-9132-1254-2bc3-d3489fb79728@2ndQuadrant.com>
 <4473.1524665917@sss.pgh.pa.us>
 <4DE444D1-DDE9-460F-AC2F-EB1B73D94A2D@postgrespro.ru>
 <CAJVSVGU2w6rvTXghDe7-Y3x9Ca0833VmnC3HpaYKGLw4xznpJA@mail.gmail.com>
Message-ID: <25f3ae08-5347-bd63-7866-05c135fdcfa3@2ndQuadrant.com>
Date: Wed, 25 Apr 2018 11:48:13 -0400
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
 Thunderbird/52.4.0
MIME-Version: 1.0
In-Reply-To: 
 <CAJVSVGU2w6rvTXghDe7-Y3x9Ca0833VmnC3HpaYKGLw4xznpJA@mail.gmail.com>
Content-Type: multipart/mixed;
 boundary="------------639912E45C740DCCD9F2CFF4"
Content-Language: en-MW
Precedence: bulk

This is a multi-part message in MIME format.
--------------639912E45C740DCCD9F2CFF4
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


On 04/25/2018 10:55 AM, John Naylor wrote:
> On 4/25/18, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:
>>> On 25 Apr 2018, at 17:18, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>>>  I think we should rewrite
>>> both of them to use the Catalog.pm infrastructure.
>> Okay, seems reasonable. I'll put shared code in Catalog.pm and
>> update patch.
> I don't think you need any new code in Catalog.pm, I believe the
> suggestion was just to use that module as a stable interface to the
> data. Looking at your patch, I'll mention that we have an idiom for
> extracting #define'd OID symbols, e.g.:
>
> my $FirstBootstrapObjectId = Catalog::FindDefinedSymbol(
> 	'access/transam.h', \@include_path, 'FirstBootstrapObjectId');
>
> This is preferred over using awk, which would have its own portability
> issues (Windows for starters).
>
> While I'm thinking out loud, it might be worthwhile to patch genbki.pl
> for the duplicate test, since they're run at the same time anyway (see
> catalog/Makefile), and we've already read all the data.
>


The logic for getting the set of oids should be centralized, if not in
Catalog.pm then in a script which serves both for dup0licate_oids and
unused_oids.

Here is something I cobbled together for the latter approach. It could
probably improve by using Catalog::FindDefinedSymbol()

cheers

andrew

-- 
Andrew Dunstan                https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


--------------639912E45C740DCCD9F2CFF4
Content-Type: application/x-perl;
 name="unused_oids.pl"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename="unused_oids.pl"

#!/usr/bin/perl

use strict;
use warnings;

my %oidcounts;

BEGIN
{
	@ARGV = (glob("pg_*.h"), glob("pg_*.dat"), qw(indexing.h toasting.h));
}

while (<>)
{
	next if /^CATALOG\(.*BKI_BOOTSTRAP/;
	next
	  unless /\boid *=> *'(\d+)'/
		  || /^CATALOG\([^,]*, *(\d+).*BKI_ROWTYPE_OID\((\d+),/
		  || /^CATALOG\([^,]*, *(\d+)/
		  || /^DECLARE_INDEX\([^,]*, *(\d+)/
		  || /^DECLARE_UNIQUE_INDEX\([^,]*, *(\d+)/
		  || /^DECLARE_TOAST\([^,]*, *(\d+), *(\d+)/;
	$oidcounts{$1}++;
	$oidcounts{$2}++ if $2;
}

if ($0 eq 'duplicate_oids')
{
	my $found = 0;

	foreach my $oid (sort { $a <=> $b } keys %oidcounts)
	{
		next unless $oidcounts{$oid} > 1;
		$found = 1;
		print "$oid\n";
	}

	exit $found;
}
elsif ($0 eq 'unused_oids')
{
	my $firstobj;
	open (my $handle,"../access/transam.h") || die "opening transam.h";
	while (<$handle>)
	{
		next unless /^#define\s+FirstBootstrapObjectId\s+(\d+)/;
		$firstobj = $1;
		last;
	}
	close($handle);
	my @sortedoids = sort { $a <=> $b } keys %oidcounts;
	my $last = 0;
	foreach my $oid (@sortedoids )
	{
		if ($oid > $last + 1)
		{
			if ($oid > $last + 2)
			{
				print $last + 1, " - ", $oid - 1, "\n";
			}
			else
			{
				print $last + 1,"\n";
			}
		}
		$last = $oid;
	}
	print $last + 1," - ", $firstobj - 1,"\n";
	exit 0;
}

--------------639912E45C740DCCD9F2CFF4--