pg.ddx.io pgsql-hackers@postgresql.org mailing list archive
help / color / mirror / Atom feedReproducible builds: genbki.pl vs schemapg.h
9+ messages / 5 participants
[nested] [flat]
Commits from this thread:
2017-12-21 c98c35cd08 Avoid putting build-location-dependent strings into generated files.
* Reproducible builds: genbki.pl vs schemapg.h
@ 2017-12-15 10:22 Christoph Berg <christoph.berg@credativ.de>
0 siblings, 1 reply; 9+ messages in thread
From: Christoph Berg @ 2017-12-15 10:22 UTC (permalink / raw)
To: pgsql-hackers
Hi,
Debian's reproducible builds project has revealed that the full build
path gets embedded into server/catalog/schemapg.h:
/*-------------------------------------------------------------------------
*
* schemapg.h
* Schema_pg_xxx macros for use by relcache.c
*
* Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
* NOTES
* ******************************
* *** DO NOT EDIT THIS FILE! ***
* ******************************
*
* It has been GENERATED by /build/postgresql-11-05gRdu/build/../src/backend/catalog/genbki.pl
*
*-------------------------------------------------------------------------
*/
This information will vary on rebuilding the binaries, needlessly
causing different checksums.
I'm proposing to strip that down to "It has been GENERATED genbki.pl".
Patch attached.
Christoph
Attachments:
reproducible-genbki.patch (text/x-diff, 674B) download
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
new file mode 100644
index 256a9c9..30153e6
*** a/src/backend/catalog/genbki.pl
--- b/src/backend/catalog/genbki.pl
*************** print $schemapg <<EOM;
*** 340,346 ****
* *** DO NOT EDIT THIS FILE! ***
* ******************************
*
! * It has been GENERATED by $0
*
*-------------------------------------------------------------------------
*/
--- 340,346 ----
* *** DO NOT EDIT THIS FILE! ***
* ******************************
*
! * It has been GENERATED by genbki.pl
*
*-------------------------------------------------------------------------
*/
* Re: Reproducible builds: genbki.pl vs schemapg.h
@ 2017-12-15 15:29 Tom Lane <tgl@sss.pgh.pa.us>
parent: Christoph Berg <christoph.berg@credativ.de>
0 siblings, 1 reply; 9+ messages in thread
From: Tom Lane @ 2017-12-15 15:29 UTC (permalink / raw)
To: Christoph Berg <christoph.berg@credativ.de>; +Cc: pgsql-hackers
Christoph Berg <christoph.berg@credativ.de> writes:
> Debian's reproducible builds project has revealed that the full build
> path gets embedded into server/catalog/schemapg.h:
genbki.pl is hardly our only script that prints its $0 ...
regards, tom lane
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Reproducible builds: genbki.pl vs schemapg.h
@ 2017-12-15 18:13 Christoph Berg <christoph.berg@credativ.de>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 9+ messages in thread
From: Christoph Berg @ 2017-12-15 18:13 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-hackers
Re: Tom Lane 2017-12-15 <9616.1513351766@sss.pgh.pa.us>
> Christoph Berg <christoph.berg@credativ.de> writes:
> > Debian's reproducible builds project has revealed that the full build
> > path gets embedded into server/catalog/schemapg.h:
>
> genbki.pl is hardly our only script that prints its $0 ...
As per
https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/postgresql-10.html,
that's the only place that makes it into the resulting binary.
I wouldn't be sending a patch if it didn't fix the issue.
Christoph
--
Senior Berater, Tel.: +49 2166 9901 187
credativ GmbH, HRB Mönchengladbach 12080, USt-ID-Nummer: DE204566209
Trompeterallee 108, 41189 Mönchengladbach
Geschäftsführung: Dr. Michael Meskes, Jörg Folz, Sascha Heuer
pgp fingerprint: 5C48 FE61 57F4 9179 5970 87C6 4C5A 6BAB 12D2 A7AE
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Reproducible builds: genbki.pl vs schemapg.h
@ 2017-12-15 22:52 Michael Paquier <michael.paquier@gmail.com>
parent: Christoph Berg <christoph.berg@credativ.de>
0 siblings, 1 reply; 9+ messages in thread
From: Michael Paquier @ 2017-12-15 22:52 UTC (permalink / raw)
To: Christoph Berg <christoph.berg@credativ.de>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On Sat, Dec 16, 2017 at 3:13 AM, Christoph Berg
<christoph.berg@credativ.de> wrote:
> Re: Tom Lane 2017-12-15 <9616.1513351766@sss.pgh.pa.us>
>> Christoph Berg <christoph.berg@credativ.de> writes:
>> > Debian's reproducible builds project has revealed that the full build
>> > path gets embedded into server/catalog/schemapg.h:
>>
>> genbki.pl is hardly our only script that prints its $0 ...
>
> As per
> https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/postgresql-10.html,
> that's the only place that makes it into the resulting binary.
> I wouldn't be sending a patch if it didn't fix the issue.
Why not fixing that? Reproducible builds are a trend of these days,
and what's proposed here is really simple to make PG more compliant
with this purpose in mind.
--
Michael
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Reproducible builds: genbki.pl vs schemapg.h
@ 2017-12-15 23:21 Andres Freund <andres@anarazel.de>
parent: Michael Paquier <michael.paquier@gmail.com>
0 siblings, 2 replies; 9+ messages in thread
From: Andres Freund @ 2017-12-15 23:21 UTC (permalink / raw)
To: Michael Paquier <michael.paquier@gmail.com>; +Cc: Christoph Berg <christoph.berg@credativ.de>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On 2017-12-16 07:52:41 +0900, Michael Paquier wrote:
> On Sat, Dec 16, 2017 at 3:13 AM, Christoph Berg
> <christoph.berg@credativ.de> wrote:
> > Re: Tom Lane 2017-12-15 <9616.1513351766@sss.pgh.pa.us>
> >> Christoph Berg <christoph.berg@credativ.de> writes:
> >> > Debian's reproducible builds project has revealed that the full build
> >> > path gets embedded into server/catalog/schemapg.h:
> >>
> >> genbki.pl is hardly our only script that prints its $0 ...
> >
> > As per
> > https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/postgresql-10.html,
> > that's the only place that makes it into the resulting binary.
> > I wouldn't be sending a patch if it didn't fix the issue.
>
> Why not fixing that? Reproducible builds are a trend of these days,
> and what's proposed here is really simple to make PG more compliant
> with this purpose in mind.
It's not like $0 instead of a hardcoded name in the header actually buys
us anything afaict.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Reproducible builds: genbki.pl vs schemapg.h
@ 2017-12-15 23:36 Peter Geoghegan <pg@bowt.ie>
parent: Andres Freund <andres@anarazel.de>
1 sibling, 0 replies; 9+ messages in thread
From: Peter Geoghegan @ 2017-12-15 23:36 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Michael Paquier <michael.paquier@gmail.com>; Christoph Berg <christoph.berg@credativ.de>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On Fri, Dec 15, 2017 at 3:21 PM, Andres Freund <andres@anarazel.de> wrote:
>> Why not fixing that? Reproducible builds are a trend of these days,
>> and what's proposed here is really simple to make PG more compliant
>> with this purpose in mind.
>
> It's not like $0 instead of a hardcoded name in the header actually buys
> us anything afaict.
+1.
I think that reproducible builds are a worthwhile goal, and I welcome
Christoph's continued work on them.
--
Peter Geoghegan
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Reproducible builds: genbki.pl vs schemapg.h
@ 2017-12-15 23:39 Tom Lane <tgl@sss.pgh.pa.us>
parent: Andres Freund <andres@anarazel.de>
1 sibling, 1 reply; 9+ messages in thread
From: Tom Lane @ 2017-12-15 23:39 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Michael Paquier <michael.paquier@gmail.com>; Christoph Berg <christoph.berg@credativ.de>; pgsql-hackers
Andres Freund <andres@anarazel.de> writes:
> On 2017-12-16 07:52:41 +0900, Michael Paquier wrote:
>> On Sat, Dec 16, 2017 at 3:13 AM, Christoph Berg
>> <christoph.berg@credativ.de> wrote:
>>> Re: Tom Lane 2017-12-15 <9616.1513351766@sss.pgh.pa.us>
>>>> genbki.pl is hardly our only script that prints its $0 ...
>>> As per
>>> https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/postgresql-10.html,
>>> that's the only place that makes it into the resulting binary.
I'm fairly confused by this claim. Since the string in question is in a
comment, it really shouldn't affect built binaries at all. I can believe
that it would affect the non-binary contents of the finished package,
because we include schemapg.h as one of the installed headers. However,
we also include fmgroids.h, which also has an expanded version of $0
in it. So if schemapg.h is affected by build path, why isn't fmgroids.h?
In my build, neither one of these files contains any path information;
I speculate that you need to use a VPATH build to have an issue, or
maybe Debian's build environment does something even weirder. But
I feel confident in saying that if indeed fmgroids.h is invariant for
you today, that's a phase-of-the-moon behavior that will break someday
if we don't make a similar change in Gen_fmgrtab.pl. Or else we should
propagate what's preventing it back to genbki.pl.
plperl's text2macro.pl is also emitting $0, and there may be other places
I missed (I grepped for 'DO NOT EDIT', not for $0 per se). We seem not
to install the output file of that one, perlchunks.h, but that's a
decision that somebody might change too.
> It's not like $0 instead of a hardcoded name in the header actually buys
> us anything afaict.
Agreed so far as the script name goes. However, two out of three of these
scripts also print their input file names, and I'm suspicious that that
output is also gonna change in a VPATH build. I'm a little less inclined
to buy the claim that we're not losing anything if we suppress that :-(
regards, tom lane
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Reproducible builds: genbki.pl vs schemapg.h
@ 2017-12-16 15:16 Christoph Berg <christoph.berg@credativ.de>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 9+ messages in thread
From: Christoph Berg @ 2017-12-16 15:16 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Andres Freund <andres@anarazel.de>; Michael Paquier <michael.paquier@gmail.com>; pgsql-hackers
Re: Tom Lane 2017-12-16 <5525.1513381145@sss.pgh.pa.us>
> >>> As per
> >>> https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/postgresql-10.html,
> >>> that's the only place that makes it into the resulting binary.
>
> I'm fairly confused by this claim. Since the string in question is in a
> comment, it really shouldn't affect built binaries at all. I can believe
> that it would affect the non-binary contents of the finished package,
"Binary" in that context was the .deb package file (in contrast to the
.dsc source package).
> In my build, neither one of these files contains any path information;
> I speculate that you need to use a VPATH build to have an issue, or
> maybe Debian's build environment does something even weirder.
This is a VPATH build, yes.
> > It's not like $0 instead of a hardcoded name in the header actually buys
> > us anything afaict.
>
> Agreed so far as the script name goes. However, two out of three of these
> scripts also print their input file names, and I'm suspicious that that
> output is also gonna change in a VPATH build. I'm a little less inclined
> to buy the claim that we're not losing anything if we suppress that :-(
Well, patching this instance of $0 would fix a binary-package
variation in practise. Of course there might be more issues waiting to
come into effect, but I don't see why that would be an argument
against fixing the current issue.
Christoph
--
Senior Berater, Tel.: +49 2166 9901 187
credativ GmbH, HRB Mönchengladbach 12080, USt-ID-Nummer: DE204566209
Trompeterallee 108, 41189 Mönchengladbach
Geschäftsführung: Dr. Michael Meskes, Jörg Folz, Sascha Heuer
pgp fingerprint: 5C48 FE61 57F4 9179 5970 87C6 4C5A 6BAB 12D2 A7AE
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Reproducible builds: genbki.pl vs schemapg.h
@ 2017-12-16 15:27 Tom Lane <tgl@sss.pgh.pa.us>
parent: Christoph Berg <christoph.berg@credativ.de>
0 siblings, 0 replies; 9+ messages in thread
From: Tom Lane @ 2017-12-16 15:27 UTC (permalink / raw)
To: Christoph Berg <christoph.berg@credativ.de>; +Cc: Andres Freund <andres@anarazel.de>; Michael Paquier <michael.paquier@gmail.com>; pgsql-hackers
Christoph Berg <christoph.berg@credativ.de> writes:
>> Agreed so far as the script name goes. However, two out of three of these
>> scripts also print their input file names, and I'm suspicious that that
>> output is also gonna change in a VPATH build. I'm a little less inclined
>> to buy the claim that we're not losing anything if we suppress that :-(
> Well, patching this instance of $0 would fix a binary-package
> variation in practise. Of course there might be more issues waiting to
> come into effect, but I don't see why that would be an argument
> against fixing the current issue.
I think we're talking at cross-purposes. I'm not saying we should not fix
this problem. I'm saying that the proposed fix appears incomplete, which
means that (a) even if it solves your problem, it probably does not solve
related problems for other people; (b) since it's not clear why this
patch is apparently sufficient for you, I'd like to understand that in
some detail before deeming the problem solved; and (c) leaving instances
of the problematic code in our tree is just about guaranteed to mean
you'll have the same problem in future, when somebody either copies that
coding pattern into some new script or tweaks the way those existing
scripts are being used.
regards, tom lane
^ permalink raw reply [nested|flat] 9+ messages in thread
end of thread, other threads:[~2017-12-16 15:27 UTC | newest]
Thread overview: 9+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2017-12-15 10:22 Reproducible builds: genbki.pl vs schemapg.h Christoph Berg <christoph.berg@credativ.de>
2017-12-15 15:29 ` Tom Lane <tgl@sss.pgh.pa.us>
2017-12-15 18:13 ` Christoph Berg <christoph.berg@credativ.de>
2017-12-15 22:52 ` Michael Paquier <michael.paquier@gmail.com>
2017-12-15 23:21 ` Andres Freund <andres@anarazel.de>
2017-12-15 23:36 ` Peter Geoghegan <pg@bowt.ie>
2017-12-15 23:39 ` Tom Lane <tgl@sss.pgh.pa.us>
2017-12-16 15:16 ` Christoph Berg <christoph.berg@credativ.de>
2017-12-16 15:27 ` Tom Lane <tgl@sss.pgh.pa.us>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox