Subject: Re: make installcheck-world in a clean environment
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>,
 Robert Haas <robertmhaas@gmail.com>,
 "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
References: <bfee872b-ab16-ec7c-8f8a-b3949add1c1b@gmail.com>
 <CA+TgmoafXiYMj6ROd09PxLFQ_dpurP1FJOL4ZwDonCkPeSubWg@mail.gmail.com>
 <aff02bd6-20c9-0b6e-4e3c-a3428ce6568b@gmail.com>
 <CA+TgmoZV08_6SXmag0hFkOurgWPjks6Jwyh+DeFXsCxLWLVjfw@mail.gmail.com>
 <24834.1525712835@sss.pgh.pa.us>
 <ec02a440-0346-4bdc-5118-0f7970d33c16@gmail.com>
 <b36c96e5-05cf-aa0f-90a5-2276887170ce@2ndquadrant.com>
 <beb142a3-aaf6-f6d7-2aae-69f7d65fb3d4@gmail.com>
 <d50a2101-c8cc-1456-fd4c-c12edfa2549b@2ndquadrant.com>
 <0401efec-68f1-679d-3ea3-21d4e8dd11af@gmail.com>
 <15981.1532988967@sss.pgh.pa.us> <26960.1532994120@sss.pgh.pa.us>
From: Alexander Lakhin <exclusion@gmail.com>
Message-ID: <8d6aebaa-d119-b702-f8a3-e0261e4cb586@gmail.com>
Date: Tue, 31 Jul 2018 14:40:05 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
 Thunderbird/52.9.1
MIME-Version: 1.0
In-Reply-To: <26960.1532994120@sss.pgh.pa.us>
Content-Type: multipart/alternative;
 boundary="------------02BBDD6D2F12D8805CEB1633"
Content-Language: en-US
Precedence: bulk

This is a multi-part message in MIME format.
--------------02BBDD6D2F12D8805CEB1633
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

31.07.2018 02:42, Tom Lane wrote:
> I wrote:
>> The original complaint about ecpg remains; I'm not terribly excited
>> about messing with that.
> Well, I got curious as to why we were seeing such a weird error,
> and eventually traced it to this stuff in ecpg/test/Makefile.regress:
>
> # Standard way to invoke the ecpg preprocessor
> ECPG = ../../preproc/ecpg --regression -I$(srcdir)/../../include -I$(srcdir)
>
> # Files that most or all ecpg preprocessor test outputs depend on
> ECPG_TEST_DEPENDENCIES = ../../preproc/ecpg$(X) \
> 	$(srcdir)/../regression.h \
> 	$(srcdir)/../../include/sqlca.h \
> 	$(srcdir)/../../include/sqlda.h \
> 	$(srcdir)/../../include/sqltypes.h \
> 	$(srcdir)/../../include/sql3types.h
>
> %.c: %.pgc $(ECPG_TEST_DEPENDENCIES)
> 	$(ECPG) -o $@ $<
>
> Now, the fine gmake manual quoth:
>
>     `%' in a prerequisite of a pattern rule stands for the same stem
>     that was matched by the `%' in the target.  In order for the pattern
>     rule to apply, its target pattern must match the file name under
>     consideration and all of its prerequisites (after pattern substitution)
>     must name files that exist or can be made.
>
> So the problem is that (after make clean) ../../preproc/ecpg doesn't
> exist, and the Makefiles under test/ have no idea how to build it,
> and thus this pattern rule is inapplicable.  Thus you end up getting
> "No rule to make target" errors.
Yes, it's exactly the problem I was trying to fix.
> While it seems straightforward to fix this for "make check" scenarios ---
> just go make ecpg before trying to make the tests --- I think these rules
> are very surprising for "make installcheck" cases.  You'd expect "make
> installcheck" to test the installed ecpg, as indeed Alexander pointed out
> at the start of the thread.  But it's not doing that.
>
> Alexander's USE_INSTALLED_ASSETS patch attempted to fix that, and I now
> see the point of it, but it still seems pretty hacky and invasive (why
> should an ecpg-only problem be touching stuff outside ecpg)?  Also,
> unless I'm still missing something, it doesn't fix the problem with
> "make clean check": ecpg won't have been built before we try to build the
> test programs.
In fact, after fixing ECPG usage with installcheck, I found that "make
installcheck" then rebuilds libpostgres, libpgport and libpq (for
installcheck-world). (I mentioned it in this thread before.) Later I
proposed a more comprehensive patch that allows us to make installcheck
cleanly (without building something).
So the problem is not ecpg-only, the ecpg's failure to build is
prominent part of it, but there are another assets getting built under
the hood.
> I'd suggest trying to simplify the USE_INSTALLED_ASSETS patch so it
> doesn't muck with the rules for building pg_regress.  I don't find
> that to be very relevant to the problem.
I can simplify the patch to fix only the ECPG failure, and then prepare
a distinct patch for libs, if it makes sense.


Best regards,
------

Alexander Lakhin
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company


--------------02BBDD6D2F12D8805CEB1633
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 7bit

<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    <div class="moz-cite-prefix">31.07.2018 02:42, Tom Lane wrote:<br>
    </div>
    <blockquote type="cite" cite="mid:26960.1532994120@sss.pgh.pa.us">
      <pre wrap="">I wrote:
</pre>
      <blockquote type="cite">
        <pre wrap="">The original complaint about ecpg remains; I'm not terribly excited
about messing with that.
</pre>
      </blockquote>
      <pre wrap="">
Well, I got curious as to why we were seeing such a weird error,
and eventually traced it to this stuff in ecpg/test/Makefile.regress:

# Standard way to invoke the ecpg preprocessor
ECPG = ../../preproc/ecpg --regression -I$(srcdir)/../../include -I$(srcdir)

# Files that most or all ecpg preprocessor test outputs depend on
ECPG_TEST_DEPENDENCIES = ../../preproc/ecpg$(X) \
	$(srcdir)/../regression.h \
	$(srcdir)/../../include/sqlca.h \
	$(srcdir)/../../include/sqlda.h \
	$(srcdir)/../../include/sqltypes.h \
	$(srcdir)/../../include/sql3types.h

%.c: %.pgc $(ECPG_TEST_DEPENDENCIES)
	$(ECPG) -o $@ $&lt;

Now, the fine gmake manual quoth:

    `%' in a prerequisite of a pattern rule stands for the same stem
    that was matched by the `%' in the target.  In order for the pattern
    rule to apply, its target pattern must match the file name under
    consideration and all of its prerequisites (after pattern substitution)
    must name files that exist or can be made.

So the problem is that (after make clean) ../../preproc/ecpg doesn't
exist, and the Makefiles under test/ have no idea how to build it,
and thus this pattern rule is inapplicable.  Thus you end up getting
"No rule to make target" errors.</pre>
    </blockquote>
    Yes, it's exactly the problem I was trying to fix.<br>
    <blockquote type="cite" cite="mid:26960.1532994120@sss.pgh.pa.us">
      <pre wrap="">While it seems straightforward to fix this for "make check" scenarios ---
just go make ecpg before trying to make the tests --- I think these rules
are very surprising for "make installcheck" cases.  You'd expect "make
installcheck" to test the installed ecpg, as indeed Alexander pointed out
at the start of the thread.  But it's not doing that.

Alexander's USE_INSTALLED_ASSETS patch attempted to fix that, and I now
see the point of it, but it still seems pretty hacky and invasive (why
should an ecpg-only problem be touching stuff outside ecpg)?  Also,
unless I'm still missing something, it doesn't fix the problem with
"make clean check": ecpg won't have been built before we try to build the
test programs.</pre>
    </blockquote>
    In fact, after fixing ECPG usage with installcheck, I found that
    "make installcheck" then rebuilds libpostgres, libpgport and libpq
    (for installcheck-world). (I mentioned it in this thread before.)
    Later I proposed a more comprehensive patch that allows us to make
    installcheck cleanly (without building something).<br>
    So the problem is not ecpg-only, the ecpg's failure to build is
    prominent part of it, but there are another assets getting built
    under the hood.<br>
    <blockquote type="cite" cite="mid:26960.1532994120@sss.pgh.pa.us">
      <pre wrap="">I'd suggest trying to simplify the USE_INSTALLED_ASSETS patch so it
doesn't muck with the rules for building pg_regress.  I don't find
that to be very relevant to the problem.</pre>
    </blockquote>
    I can simplify the patch to fix only the ECPG failure, and then
    prepare a distinct patch for libs, if it makes sense. <br>
    <br>
    <br>
    <font color="#000000"><span style="background: #ffffff">Best
        regards,</span></font><font color="#000000"><span
        style="background: #ffffff"><br>
        ------ </span></font><br>
    <font color="#000000"><span style="background: #ffffff"></span></font>
    <p><font color="#000000"><span style="background: #ffffff">
          Alexander Lakhin <br>
          Postgres Professional: <a class="moz-txt-link-freetext"
            href="http://www.postgrespro.com">http://www.postgrespro.com</a>
          <br>
          The Russian Postgres Company</span></font></p>
  </body>
</html>

--------------02BBDD6D2F12D8805CEB1633--