public inbox for [email protected]
help / color / mirror / Atom feedNew and unified 9.5 spec file is in git
11+ messages / 3 participants
[nested] [flat]
* New and unified 9.5 spec file is in git
@ 2016-01-19 21:44 Devrim GÜNDÜZ <[email protected]>
0 siblings, 1 reply; 11+ messages in thread
From: Devrim GÜNDÜZ @ 2016-01-19 21:44 UTC (permalink / raw)
To: pgsql-pkg-yum
Hi,
After various attempts to make RHEL 5 happy, I pushed unified
PostgreSQL 9.5 spec file to git, which is supposed to work for all
distros.
The spec file also adds plpython3 subpackage (on supported distros),
including patches from Fedora for Python 3.5 tests (on supported
distros), adds dtrace support (on supported distros), and some cosmetic
cleanup.
I'm about to push packages to repo. I tested them, hopefully I did not
miss anything :-)
If anyone could take a look at it, it would be awesome.
Regards,
--
Devrim GÜNDÜZ
Principal Systems Engineer @ EnterpriseDB: http://www.enterprisedb.com
PostgreSQL Danışmanı/Consultant, Red Hat Certified Engineer
Twitter: @DevrimGunduz , @DevrimGunduzTR
Attachments:
[application/pgp-signature] signature.asc (819B, 2-signature.asc)
download
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: New and unified 9.5 spec file is in git
@ 2016-01-20 02:15 Craig Ringer <[email protected]>
parent: Devrim GÜNDÜZ <[email protected]>
0 siblings, 2 replies; 11+ messages in thread
From: Craig Ringer @ 2016-01-20 02:15 UTC (permalink / raw)
To: Devrim GÜNDÜZ <[email protected]>; +Cc: pgsql-pkg-yum
On 20 January 2016 at 05:44, Devrim GÜNDÜZ <[email protected]> wrote:
>
> Hi,
>
> After various attempts to make RHEL 5 happy, I pushed unified
> PostgreSQL 9.5 spec file to git, which is supposed to work for all
> distros.
>
Great! That's wonderful news.
I hope you got some use out of the work I sent you a while ago on that.
Apologies that I didn't find the time to follow up on it.
> If anyone could take a look at it, it would be awesome.
>
I'll look right now.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: New and unified 9.5 spec file is in git
@ 2016-01-20 03:52 Craig Ringer <[email protected]>
parent: Craig Ringer <[email protected]>
1 sibling, 3 replies; 11+ messages in thread
From: Craig Ringer @ 2016-01-20 03:52 UTC (permalink / raw)
To: Devrim GÜNDÜZ <[email protected]>; +Cc: pgsql-pkg-yum
On 20 January 2016 at 10:15, Craig Ringer <[email protected]> wrote:
> If anyone could take a look at it, it would be awesome.
>>
>
> I'll look right now.
>
It built perfectly with mock on Fedora 21 for a F23 target. Brilliant!
To make it clear that they're the same is it worth having a common/
directory at the same level as EL-5, F-23, etc and symlinking the specfile
to the various subdirs from there, along with any identical patches etc?
Here's the full build process I used:
sudo yum -y install rpmdevtools mock
git clone git://git.postgresql.org/git/pgrpms.git
cd pgrpms/rpm/redhat/9.5/postgresql/F-23
spectool --get-files postgresql-9.5.spec
mock -r fedora-23-x86_64 --buildsrpm --sources . --spec postgresql-9.5.spec
cp
/var/lib/mock/fedora-23-x86_64/result/postgresql95-9.5.0-2PGDG.fc23.src.rpm
.
mock -r fedora-23-x86_64 --rebuild postgresql95-9.5.0-2PGDG.fc23.src.rpm
I'm really happy about this, since it means the build depends are now
correct and it can be built without needing a dedicated build machine using
nothing but mock. No need to manually set up dependencies anymore, set up a
build box, etc. No need to juggle x86_64 and i386 either, you can run
builds for *everything* from a single F23 box or VM or docker or whatever.
If the same process is followed for other packages then it'll become
possible to build the whole PGDG system with mockchain with no custom build
machines, building anywhere that mock is available. No accidental
undeclared build or runtime dependencies. Since you seem to be using
Docker to prep build envs now it's less of an issue than when the build
boxes are hand-maintained, but the great thing about mock is that it gets
all the build depends info straight from the spec file.
I also rebuilt the F23 package for F21 just by changing the -r argument to
mock. Same for building it for i386 on an x86_64 host.
The EL-5 package built happily under mock for -r epel-5-i386 too. I didn't
test install it.
Thanks so much. This is great and I really hope you got some use out of my
prior work on it.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: New and unified 9.5 spec file is in git
@ 2016-01-20 04:22 Craig Ringer <[email protected]>
parent: Craig Ringer <[email protected]>
2 siblings, 1 reply; 11+ messages in thread
From: Craig Ringer @ 2016-01-20 04:22 UTC (permalink / raw)
To: Devrim GÜNDÜZ <[email protected]>; +Cc: pgsql-pkg-yum
I've uploaded a branch with a couple of small improvements. See
https://github.com/2ndQuadrant/pgdg-pgrpms .
git remote add 2ndq https://github.com/2ndQuadrant/pgdg-pgrpms.git
git fetch 2ndq
# view it
git log -p remotes/2ndq/add-builddepends
# merge it
git merge remotes/2ndq/add-builddepends
This is the main patch of interest:
https://github.com/2ndQuadrant/pgdg-pgrpms/commit/60524156129b27b9cf3836621a0d05afb658d60e
A further suggested improvement, which I haven't put in that tree but
wanted to flag as an idea - detect the target based on the directory name
so there's no need to have a different Makefile for each target dir and
they can all just be symlinks.
For each dist's Makefile:
-DIST=.rhel5
SPECFILE="postgresql-9.5.spec"
+# This gibberish gets the immediate parent directory's name, prepends
RPMDIST_
+# and looks it up as a Make variable lazily on first evaluation. That way
we
+# don't require a new Makefile for each build directory.
+#
+# The RPMDIST_ vars are defined in Makefile.global
+#
+DIST=$(RPMDIST_$(shell basename $(dir $(abspath $(lastword
$(MAKEFILE_LIST))))))
then in Makefile.global:
+# These mappings could be automated with $(foreach ...) and $(eval ...)
+# but there hardly seems any point.
+RPMDIST_EL-5=.rhel5
+RPMDIST_EL-6=.rhel6
+RPMDIST_EL-7=.rhel7
+RPMDIST_F-21=.fc21
+RPMDIST_F-22=.fc22
+RPMDIST_F-23=.fc23
Then EL-5, EL-6, EL-7, F-22 and F-23 can be deleted and replaced with
common/ which is symlinks to each of them. That shouldn't upset the build
system; otherwise I'd suggest switching moving all files up a level,
removing the dist subtrees, and using "make DIST=EL-5" or whatever to
control the target.
Less duplication in the source tree, no need to remember to copy files
around, easier to track history.
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: New and unified 9.5 spec file is in git
@ 2016-01-20 04:33 Jeff Frost <[email protected]>
parent: Craig Ringer <[email protected]>
2 siblings, 1 reply; 11+ messages in thread
From: Jeff Frost @ 2016-01-20 04:33 UTC (permalink / raw)
To: Craig Ringer <[email protected]>; +Cc: Devrim GÜNDÜZ <[email protected]>; pgsql-pkg-yum
> On Jan 19, 2016, at 19:52, Craig Ringer <[email protected]> wrote:
>
> On 20 January 2016 at 10:15, Craig Ringer <[email protected]> wrote:
>
>>> If anyone could take a look at it, it would be awesome.
>>
>> I'll look right now.
>
>
> It built perfectly with mock on Fedora 21 for a F23 target. Brilliant!
>
> To make it clear that they're the same is it worth having a common/ directory at the same level as EL-5, F-23, etc and symlinking the specfile to the various subdirs from there, along with any identical patches etc?
>
> Here's the full build process I used:
>
> sudo yum -y install rpmdevtools mock
> git clone git://git.postgresql.org/git/pgrpms.git
> cd pgrpms/rpm/redhat/9.5/postgresql/F-23
> spectool --get-files postgresql-9.5.spec
> mock -r fedora-23-x86_64 --buildsrpm --sources . --spec postgresql-9.5.spec
> cp /var/lib/mock/fedora-23-x86_64/result/postgresql95-9.5.0-2PGDG.fc23.src.rpm .
> mock -r fedora-23-x86_64 --rebuild postgresql95-9.5.0-2PGDG.fc23.src.rpm
>
> I'm really happy about this, since it means the build depends are now correct and it can be built without needing a dedicated build machine using nothing but mock. No need to manually set up dependencies anymore, set up a build box, etc. No need to juggle x86_64 and i386 either, you can run builds for *everything* from a single F23 box or VM or docker or whatever.
>
> If the same process is followed for other packages then it'll become possible to build the whole PGDG system with mockchain with no custom build machines, building anywhere that mock is available. No accidental undeclared build or runtime dependencies. Since you seem to be using Docker to prep build envs now it's less of an issue than when the build boxes are hand-maintained, but the great thing about mock is that it gets all the build depends info straight from the spec file.
>
> I also rebuilt the F23 package for F21 just by changing the -r argument to mock. Same for building it for i386 on an x86_64 host.
>
> The EL-5 package built happily under mock for -r epel-5-i386 too. I didn't test install it.
>
> Thanks so much. This is great and I really hope you got some use out of my prior work on it.
>
This makes me so excited! I can't wait to try it and work on getting the other packages set up this way as well!
Great job you guys!
> --
> Craig Ringer http://www.2ndQuadrant.com/
> PostgreSQL Development, 24x7 Support, Training & Services
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: New and unified 9.5 spec file is in git
@ 2016-01-20 20:11 Devrim GÜNDÜZ <[email protected]>
parent: Craig Ringer <[email protected]>
1 sibling, 0 replies; 11+ messages in thread
From: Devrim GÜNDÜZ @ 2016-01-20 20:11 UTC (permalink / raw)
To: Craig Ringer <[email protected]>; +Cc: pgsql-pkg-yum
Hi Craig,
On Wed, 2016-01-20 at 10:15 +0800, Craig Ringer wrote:
> I hope you got some use out of the work I sent you a while ago on
> that. Apologies that I didn't find the time to follow up on it.
Even though I did not use your actual work, I must admit that I learned
*a lot* from your spec. Apologies for not crediting you over there.
Thank you very much!
Regards,
--
Devrim GÜNDÜZ
Principal Systems Engineer @ EnterpriseDB: http://www.enterprisedb.com
PostgreSQL Danışmanı/Consultant, Red Hat Certified Engineer
Twitter: @DevrimGunduz , @DevrimGunduzTR
Attachments:
[application/pgp-signature] signature.asc (819B, 2-signature.asc)
download
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: New and unified 9.5 spec file is in git
@ 2016-01-20 20:17 Devrim GÜNDÜZ <[email protected]>
parent: Craig Ringer <[email protected]>
2 siblings, 1 reply; 11+ messages in thread
From: Devrim GÜNDÜZ @ 2016-01-20 20:17 UTC (permalink / raw)
To: Craig Ringer <[email protected]>; +Cc: pgsql-pkg-yum
Hi,
On Wed, 2016-01-20 at 11:52 +0800, Craig Ringer wrote:
> It built perfectly with mock on Fedora 21 for a F23 target.
> Brilliant!
Thanks for checking!
> To make it clear that they're the same is it worth having a common/
> directory at the same level as EL-5, F-23, etc and symlinking the
> specfile to the various subdirs from there, along with any identical
> patches etc?
That would be the next step. Let's finish the actual work first.
> I'm really happy about this, since it means the build depends are now
> correct and it can be built without needing a dedicated build machine
> using nothing but mock. No need to manually set up dependencies
> anymore, set up a build box, etc. No need to juggle x86_64 and i386
> either, you can run builds for *everything* from a single F23 box or
> VM or docker or whatever.
Well, we are building 100+ packages in 9.5 set. Setting up a new mock
environment everytime would slow us down, IMHO. What do you think?
> If the same process is followed for other packages
*All* of the *recently added* spec files are unified nowadays :) This
also applies to some specs that I found time to rework on (like
postgis, repmgr, etc.). The repo is not fully unified yet, but we are
on it.
> Since you seem to be using Docker to prep build envs now
Err, no. We are using actual VMs (powered by libvirtd+qemu-kvm)
> it's less of an issue than when the build boxes are hand-maintained,
> but the great thing about mock is that it gets all the build depends
> info straight from the spec file.
All deps are already installed on our build servers. If not, we install
them before building a new package.
> The EL-5 package built happily under mock for -r epel-5-i386 too. I
> didn't
> test install it.
This is great testing. Appreciated, thanks!
> Thanks so much. This is great and I really hope you got some use out
> of my prior work on it.
You are welcome. As I wrote a few mins ago, I really was inspired from
your work.
Regards,
--
Devrim GÜNDÜZ
Principal Systems Engineer @ EnterpriseDB: http://www.enterprisedb.com
PostgreSQL Danışmanı/Consultant, Red Hat Certified Engineer
Twitter: @DevrimGunduz , @DevrimGunduzTR
Attachments:
[application/pgp-signature] signature.asc (819B, 2-signature.asc)
download
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: New and unified 9.5 spec file is in git
@ 2016-01-20 20:18 Devrim GÜNDÜZ <[email protected]>
parent: Craig Ringer <[email protected]>
0 siblings, 1 reply; 11+ messages in thread
From: Devrim GÜNDÜZ @ 2016-01-20 20:18 UTC (permalink / raw)
To: Craig Ringer <[email protected]>; +Cc: pgsql-pkg-yum
Hi,
On Wed, 2016-01-20 at 12:22 +0800, Craig Ringer wrote:
> This is the main patch of interest:
>
> https://github.com/2ndQuadrant/pgdg-pgrpms/commit/60524156129b27b9cf3
> 836621a0d05afb658d60e
Oh, why do we need -devel packages for postgresql95-devel package?
Regards,
--
Devrim GÜNDÜZ
Principal Systems Engineer @ EnterpriseDB: http://www.enterprisedb.com
PostgreSQL Danışmanı/Consultant, Red Hat Certified Engineer
Twitter: @DevrimGunduz , @DevrimGunduzTR
Attachments:
[application/pgp-signature] signature.asc (819B, 2-signature.asc)
download
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: New and unified 9.5 spec file is in git
@ 2016-01-20 20:18 Devrim GÜNDÜZ <[email protected]>
parent: Jeff Frost <[email protected]>
0 siblings, 0 replies; 11+ messages in thread
From: Devrim GÜNDÜZ @ 2016-01-20 20:18 UTC (permalink / raw)
To: Jeff Frost <[email protected]>; Craig Ringer <[email protected]>; +Cc: pgsql-pkg-yum
Hi Jeff,
On Tue, 2016-01-19 at 20:33 -0800, Jeff Frost wrote:
> This makes me so excited! I can't wait to try it and work on getting
> the other packages set up this way as well!
>
> Great job you guys!
Thanks!
Regards,
--
Devrim GÜNDÜZ
Principal Systems Engineer @ EnterpriseDB: http://www.enterprisedb.com
PostgreSQL Danışmanı/Consultant, Red Hat Certified Engineer
Twitter: @DevrimGunduz , @DevrimGunduzTR
Attachments:
[application/pgp-signature] signature.asc (819B, 2-signature.asc)
download
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: New and unified 9.5 spec file is in git
@ 2016-01-21 01:15 Craig Ringer <[email protected]>
parent: Devrim GÜNDÜZ <[email protected]>
0 siblings, 0 replies; 11+ messages in thread
From: Craig Ringer @ 2016-01-21 01:15 UTC (permalink / raw)
To: Devrim GÜNDÜZ <[email protected]>; +Cc: pgsql-pkg-yum
On 21 January 2016 at 04:18, Devrim GÜNDÜZ <[email protected]> wrote:
> > This is the main patch of interest:
> >
> > https://github.com/2ndQuadrant/pgdg-pgrpms/commit/60524156129b27b9cf3
> > 836621a0d05afb658d60e
>
> Oh, why do we need -devel packages for postgresql95-devel package?
>
>
So that when you install the postgresql95-devel package it's reliably
usable.
If we don't depend on gettext-devel etc, then where PostgreSQL uses those
headers in its own headers you'll get errors at compile time unless they
happen to be installed because of the requirements of other unrelated
packages.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: New and unified 9.5 spec file is in git
@ 2016-01-21 01:21 Craig Ringer <[email protected]>
parent: Devrim GÜNDÜZ <[email protected]>
0 siblings, 0 replies; 11+ messages in thread
From: Craig Ringer @ 2016-01-21 01:21 UTC (permalink / raw)
To: Devrim GÜNDÜZ <[email protected]>; +Cc: pgsql-pkg-yum
On 21 January 2016 at 04:17, Devrim GÜNDÜZ <[email protected]> wrote:
> > I'm really happy about this, since it means the build depends are now
> > correct and it can be built without needing a dedicated build machine
> > using nothing but mock. No need to manually set up dependencies
> > anymore, set up a build box, etc. No need to juggle x86_64 and i386
> > either, you can run builds for *everything* from a single F23 box or
> > VM or docker or whatever.
>
> Well, we are building 100+ packages in 9.5 set. Setting up a new mock
> environment everytime would slow us down, IMHO. What do you think?
>
mock caches the build env so in practice it's very fast. mockchain helps
even more.
mock is what Koji uses and what's used to build all of Fedora and EPEL, so
it gets a fair bit of performance attention. There _is_ a performance cost
vs building bare on a preconfigured build box but it's (IMO) greatly offset
by the increased reliability and consistency of package builds. You don't
risk having undeclared dependencies, building against a version other than
the one declared in the spec file, etc.
> > If the same process is followed for other packages
>
> *All* of the *recently added* spec files are unified nowadays :)
Great!
> > it's less of an issue than when the build boxes are hand-maintained,
> > but the great thing about mock is that it gets all the build depends
> > info straight from the spec file.
>
> All deps are already installed on our build servers. If not, we install
> them before building a new package.
>
That can cause issues though. For example I think that's how the Java 8
problems a while ago happened. The spec file declared a dependency on
openjdk 7 but the rpms were built against what was locally installed, which
is openjdk 8.
The nice thing with mock is that the buildroot can contain only exactly the
required packages for a build. No extras. It provides a lot more confidence
that the spec matches the actual build and that anyone can reproduce the
package build consistently without a special hand-created build env.
I've been using it for the BDR and pglogical builds and it does a pretty
good job. It's helped me find a number of bugs and oversights too.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
^ permalink raw reply [nested|flat] 11+ messages in thread
end of thread, other threads:[~2016-01-21 01:21 UTC | newest]
Thread overview: 11+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2016-01-19 21:44 New and unified 9.5 spec file is in git Devrim GÜNDÜZ <[email protected]>
2016-01-20 02:15 ` Craig Ringer <[email protected]>
2016-01-20 03:52 ` Craig Ringer <[email protected]>
2016-01-20 04:22 ` Craig Ringer <[email protected]>
2016-01-20 20:18 ` Devrim GÜNDÜZ <[email protected]>
2016-01-21 01:15 ` Craig Ringer <[email protected]>
2016-01-20 04:33 ` Jeff Frost <[email protected]>
2016-01-20 20:18 ` Devrim GÜNDÜZ <[email protected]>
2016-01-20 20:17 ` Devrim GÜNDÜZ <[email protected]>
2016-01-21 01:21 ` Craig Ringer <[email protected]>
2016-01-20 20:11 ` Devrim GÜNDÜZ <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox