Subject: Re: pgsql: Move snowball_create.sql creation into perl file

From: Tom Lane <[email-masked]>
Date: Tue, 23 May 2023 14:51:03 -0400

[ dropping -packagers ]

Andres Freund <[email-masked]> writes:
> I guess I need to go and check how long the "release" tarball generation
> takes...

It's quick except for the documentation-generating steps.  Maybe
we could test that part only once?

			regards, tom lane

---

From: Andres Freund <[email-masked]>
Date: Wed, 24 May 2023 14:24:37 -0700

Hi,

> we could test that part only once?

First thing I noticed that 'make dist' doesn't work in a vpath, failing in a
somewhat obscure way (likely because in a vpath build the the copy from the
source dir doesn't include GNUMakefile). Do we expect it to work?

Besides docs, the slowest part appears to be gzip --best and then bzip2, as
those runs serially and takes 11 and 13 seconds respectively here...

The first thing I tried was:
  make -j8 dist GZIP=pigz BZIP2=pbzip2

unfortunately that results in

pigz: abort: cannot provide files in GZIP environment variable

echo GZIP=pigz >> src/Makefile.custom
echo BZIP2=pbzip2 >> src/Makefile.custom

reduces that to

real	1m6.472s
user	1m28.316s
sys	0m5.340s

real	0m54.811s
user	1m42.078s
sys	0m6.183s

still not great...

OTOH, we currently already build the docs as part of the CompilerWarnings
test. I don't think there's a reason to test that twice?

For me make distcheck currently fails:

In file included from ../../src/include/postgres.h:46,
                 from hashfn.c:24:
../../src/include/utils/elog.h:79:10: fatal error: utils/errcodes.h: No such file or directory
   79 | #include "utils/errcodes.h"
      |          ^~~~~~~~~~~~~~~~~~
compilation terminated.
make[3]: *** [<builtin>: hashfn.o] Error 1

at first I thought it was due to my use of -j8 - but it doesn't even work
without that.

That's due to MAKELEVEL:

submake-generated-headers:
ifndef NO_GENERATED_HEADERS
ifeq ($(MAKELEVEL),0)
	$(MAKE) -C $(top_builddir)/src/backend generated-headers
endif
endif

So the distcheck target needs to reset MAKELEVEL=0 - unless somebody has a
better idea?

Separately, it's somewhat confusing that we include errcodes.h etc in
src/backend/utils, rather than its final location, in src/include/utils. It
works, even without perl, because copying the file doesn't require perl, it's
just generating it...

Greetings,

Andres Freund

---

From: Tom Lane <[email-masked]>
Date: Wed, 24 May 2023 17:35:52 -0400

> source dir doesn't include GNUMakefile). Do we expect it to work?

Don't see how it could possibly be useful in a vpath, because you'd have
the real source files and the generated files in different trees.

			regards, tom lane

---

From: Peter Eisentraut <[email-masked]>
Date: Fri, 26 May 2023 09:02:33 +0200

> source dir doesn't include GNUMakefile). Do we expect it to work?

I don't think so.

> works, even without perl, because copying the file doesn't require perl, it's

The "copying" is actually a symlink, right?  I don't think we want to 
ship symlinks in the tarball?

---

From: Andres Freund <[email-masked]>
Date: Sat, 27 May 2023 11:47:13 -0700

Hi,

> > source dir doesn't include GNUMakefile). Do we expect it to work?

Maybe we should just error out in that case, instead of failing in an obscure
way down the line?

> The "copying" is actually a symlink, right?  I don't think we want to ship
> symlinks in the tarball?

Fair point - still seems we should just create the files in the right
directory instead of doing it in the wrong place and then creating symlinks to
make them accessible...

Greetings,

Andres Freund

---

From: Peter Eisentraut <[email-masked]>
Date: Wed, 31 May 2023 06:52:03 -0400

> directory instead of doing it in the wrong place and then creating symlinks to

Right.  I think the reason this was set up this way is that with make it 
is generally dubious to create target files outside of the current 
directory.