public inbox for [email protected]
help / color / mirror / Atom feedRPM Morgue
8+ messages / 4 participants
[nested] [flat]
* RPM Morgue
@ 2018-06-25 01:42 Craig Ringer <[email protected]>
2018-06-25 07:27 ` Re: RPM Morgue Christoph Berg <[email protected]>
2018-06-26 23:40 ` Re: RPM Morgue Devrim Gündüz <[email protected]>
0 siblings, 2 replies; 8+ messages in thread
From: Craig Ringer @ 2018-06-25 01:42 UTC (permalink / raw)
To: Devrim Gündüz <[email protected]>; pgsql-pkg-yum
Devrim, team:
The apt.postgresql.org crew have a package morgue for old versions at
atalia.postgresql.org/morgue/ . It's not a full repo, but you can fish out
needed packages manually and install them. This is a *lifesaver* when
trying to examine a core file a customer system where debuginfo wasn't
installed, or trying to reproduce a subtle version-specific issue.
Do you have anything like that? If not, do you have any interest in it? I'd
really like to get something like it going, and rather than creating one
in-house at 2ndQ where nobody else lands up benefiting, it might make sense
to help out with one for the community.
What I'm thinking of is a selective rsync to an archive repo, where
packages get copied but never get deleted, then createrepo_c runs in
incremental mode (--update --retain-old-md-by-age=5d) to index them.
So it doesn't upset https://yum.postgresql.org/, but
https://yum-archive.postgresql.org/ or whatever can keep a deep history of
packages.
An alternative is to ditch the repo indexing and use an AWS S3 bucket to
host an unindexed package slush pile, like the apt morgue. createrepo can't
run sensibly on s3-hosted files. But s3 is cheap - 2c/gb/month, or less if
you go for infrequent access mode.
Thoughts?
I can always set up my own mirror inhouse for 2ndQ, but I'd rather work
with you.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: RPM Morgue
2018-06-25 01:42 RPM Morgue Craig Ringer <[email protected]>
@ 2018-06-25 07:27 ` Christoph Berg <[email protected]>
1 sibling, 0 replies; 8+ messages in thread
From: Christoph Berg @ 2018-06-25 07:27 UTC (permalink / raw)
To: pgsql-pkg-yum
Re: Craig Ringer 2018-06-25 <CAMsr+YHGvn-7nyFUncdVHR_Q5i2Kc19jy_RAvmt-g9O5iMKUqg@mail.gmail.com>
> The apt.postgresql.org crew have a package morgue for old versions at
> atalia.postgresql.org/morgue/ . It's not a full repo, but you can fish out
> needed packages manually and install them. This is a *lifesaver* when
> trying to examine a core file a customer system where debuginfo wasn't
> installed, or trying to reproduce a subtle version-specific issue.
Fwiw, the reason I set it up is because our repo software (reprepro)
doesn't support keeping the past N versions of every package around.
The yum repo keeps the last 3 versions in place, I believe. But of
course that doesn't help if $customer didn't upgrade for $long.
Christoph
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: RPM Morgue
2018-06-25 01:42 RPM Morgue Craig Ringer <[email protected]>
@ 2018-06-26 23:40 ` Devrim Gündüz <[email protected]>
2018-06-27 02:49 ` Re: RPM Morgue Craig Ringer <[email protected]>
2018-06-27 20:35 ` Re: RPM Morgue Christoph Berg <[email protected]>
2018-06-28 11:14 ` Re: RPM Morgue GOLLET Nicolas <[email protected]>
1 sibling, 3 replies; 8+ messages in thread
From: Devrim Gündüz @ 2018-06-26 23:40 UTC (permalink / raw)
To: Craig Ringer <[email protected]>; pgsql-pkg-yum
Hi Craig,
On Mon, 2018-06-25 at 09:42 +0800, Craig Ringer wrote:
> The apt.postgresql.org crew have a package morgue for old versions at
> atalia.postgresql.org/morgue/ . It's not a full repo, but you can fish out
> needed packages manually and install them. This is a *lifesaver* when
> trying to examine a core file a customer system where debuginfo wasn't
> installed, or trying to reproduce a subtle version-specific issue.
>
> Do you have anything like that? If not, do you have any interest in it? I'd
> really like to get something like it going, and rather than creating one
> in-house at 2ndQ where nobody else lands up benefiting, it might make sense
> to help out with one for the community.
>
> What I'm thinking of is a selective rsync to an archive repo, where
> packages get copied but never get deleted, then createrepo_c runs in
> incremental mode (--update --retain-old-md-by-age=5d) to index them.
>
> So it doesn't upset https://yum.postgresql.org/, but
> https://yum-archive.postgresql.org/ or whatever can keep a deep history of
> packages.
>
> An alternative is to ditch the repo indexing and use an AWS S3 bucket to
> host an unindexed package slush pile, like the apt morgue. createrepo can't
> run sensibly on s3-hosted files. But s3 is cheap - 2c/gb/month, or less if
> you go for infrequent access mode.
>
> Thoughts?
I'm not against keeping the old package, but there are two things:
* We need to ask sysadmins team, because this means a lot of extra disk space.
* Building older packages is not that hard with the RPMs as you know -- just
change the version number, and run make rpm11 (or whatever). I'm not sure that
keeping the old packages are worth the hassle.
Again, I'm not against the idea as long as you can convince sysadmin team, and
also write scripts to pull the packages :)
Cheers,
--
Devrim Gündüz
EnterpriseDB: https://www.enterprisedb.com
PostgreSQL Consultant, Red Hat Certified Engineer
Twitter: @DevrimGunduz , @DevrimGunduzTR
Attachments:
[application/pgp-signature] signature.asc (833B, 2-signature.asc)
download
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: RPM Morgue
2018-06-25 01:42 RPM Morgue Craig Ringer <[email protected]>
2018-06-26 23:40 ` Re: RPM Morgue Devrim Gündüz <[email protected]>
@ 2018-06-27 02:49 ` Craig Ringer <[email protected]>
2 siblings, 0 replies; 8+ messages in thread
From: Craig Ringer @ 2018-06-27 02:49 UTC (permalink / raw)
To: Devrim Gündüz <[email protected]>; +Cc: pgsql-pkg-yum
On 27 June 2018 at 07:40, Devrim Gündüz <[email protected]> wrote:
>
> I'm not against keeping the old package, but there are two things:
>
> * We need to ask sysadmins team, because this means a lot of extra disk
> space.
>
I'm not averse to hosting elsewhere, I just know Pg infra likes to keep
control where they can.
* Building older packages is not that hard with the RPMs as you know -- just
> change the version number, and run make rpm11 (or whatever). I'm not sure
> that
> keeping the old packages are worth the hassle.
>
Unfortunately that's not useful. You need *exactly matching* debuginfo
packages for the same build-IDs. You can't just rebuild the suite and
extract the debuginfo packages to install on another host.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: RPM Morgue
2018-06-25 01:42 RPM Morgue Craig Ringer <[email protected]>
2018-06-26 23:40 ` Re: RPM Morgue Devrim Gündüz <[email protected]>
@ 2018-06-27 20:35 ` Christoph Berg <[email protected]>
2 siblings, 0 replies; 8+ messages in thread
From: Christoph Berg @ 2018-06-27 20:35 UTC (permalink / raw)
To: Devrim Gündüz <[email protected]>; +Cc: Craig Ringer <[email protected]>; pgsql-pkg-yum
Re: Devrim Gündüz 2018-06-27 <[email protected]>
> I'm not against keeping the old package, but there are two things:
>
> * We need to ask sysadmins team, because this means a lot of extra disk space.
The apt morgue is 74 GB atm. It's not mirrored anywhere - which isn't
ideal, but the at means we only need to keep an eye on the local disk
space, and not on N mirror hosts which all might have slightly
different limits.
The initial discussion with the sysadmins was that everyone kind of
agreed that the service is useful, but we couldn't settle for any
definition of what resources should be allocated for the next N years.
So in the end I just put it somewhere, and that's where it's living
now. (I don't blame them for that, it's difficult to work with
statements like "this could be any size, because we don't know how
many packages there will be in 3 years".)
Christoph
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: RPM Morgue
2018-06-25 01:42 RPM Morgue Craig Ringer <[email protected]>
2018-06-26 23:40 ` Re: RPM Morgue Devrim Gündüz <[email protected]>
@ 2018-06-28 11:14 ` GOLLET Nicolas <[email protected]>
2018-06-28 11:31 ` Re: RPM Morgue Christoph Berg <[email protected]>
2 siblings, 1 reply; 8+ messages in thread
From: GOLLET Nicolas @ 2018-06-28 11:14 UTC (permalink / raw)
To: "Devrim G?nd?z" <[email protected]>; +Cc: "Craig Ringer" <[email protected]>; pgsql-pkg-yum
Hi Craig, I had also thought that setting up a public morgue would be a good thing, I recently set up an unofficial public morgue (WIP): http://pgyum-morgue.ng.pe/ (WIP) I'm missing some RPM, if you want us to work together on setting it up I'm available;) I had thought not to put all the packages in the morgue but the most important ones: - postgresql (RPMs but also an archive containing all RPMs (libs, contrib, debuginfo, ...) by version (WIP). - slony - postgis ... ? Classified by architecture and RHLE version... See you soon! ++ Nicolas ---- On mer., 27 juin 2018 01:40:54 +0200 Devrim Gündüz <[email protected]> wrote ---- Hi Craig, On Mon, 2018-06-25 at 09:42 +0800, Craig Ringer wrote: > The apt.postgresql.org crew have a package morgue for old versions at > atalia.postgresql.org/morgue/ . It's not a full repo, but you can fish out > needed packages manually and install them. This is a *lifesaver* when > trying to examine a core file a customer system where debuginfo wasn't > installed, or trying to reproduce a subtle version-specific issue. > > Do you have anything like that? If not, do you have any interest in it? I'd > really like to get something like it going, and rather than creating one > in-house at 2ndQ where nobody else lands up benefiting, it might make sense > to help out with one for the community. > > What I'm thinking of is a selective rsync to an archive repo, where > packages get copied but never get deleted, then createrepo_c runs in > incremental mode (--update --retain-old-md-by-age=5d) to index them. > > So it doesn't upset https://yum.postgresql.org/, but > https://yum-archive.postgresql.org/ or whatever can keep a deep history of > packages. > > An alternative is to ditch the repo indexing and use an AWS S3 bucket to > host an unindexed package slush pile, like the apt morgue. createrepo can't > run sensibly on s3-hosted files. But s3 is cheap - 2c/gb/month, or less if > you go for infrequent access mode. > > Thoughts? I'm not against keeping the old package, but there are two things: * We need to ask sysadmins team, because this means a lot of extra disk space. * Building older packages is not that hard with the RPMs as you know -- just change the version number, and run make rpm11 (or whatever). I'm not sure that keeping the old packages are worth the hassle. Again, I'm not against the idea as long as you can convince sysadmin team, and also write scripts to pull the packages :) Cheers, -- Devrim Gündüz EnterpriseDB: https://www.enterprisedb.com PostgreSQL Consultant, Red Hat Certified Engineer Twitter: @DevrimGunduz , @DevrimGunduzTR
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: RPM Morgue
2018-06-25 01:42 RPM Morgue Craig Ringer <[email protected]>
2018-06-26 23:40 ` Re: RPM Morgue Devrim Gündüz <[email protected]>
2018-06-28 11:14 ` Re: RPM Morgue GOLLET Nicolas <[email protected]>
@ 2018-06-28 11:31 ` Christoph Berg <[email protected]>
2018-06-29 01:38 ` Re: RPM Morgue Craig Ringer <[email protected]>
0 siblings, 1 reply; 8+ messages in thread
From: Christoph Berg @ 2018-06-28 11:31 UTC (permalink / raw)
To: GOLLET Nicolas <[email protected]>; +Cc: Devrim Gündüz <[email protected]>; Craig Ringer <[email protected]>; pgsql-pkg-yum
Re: GOLLET Nicolas 2018-06-28 <[email protected]>
> I had also thought that setting up a public morgue would be a good thing, I
> recently set up an unofficial public morgue (WIP): http://pgyum-morgue.ng.pe/
> (WIP)
>
> I'm missing some RPM, if you want us to work together on setting it up I'm
> available;)
> I had thought not to put all the packages in the morgue but the most important
> ones:
Why not include everything? Filtering is extra effort, and even if you
get the filters right, they will soon become outdated.
Christoph
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: RPM Morgue
2018-06-25 01:42 RPM Morgue Craig Ringer <[email protected]>
2018-06-26 23:40 ` Re: RPM Morgue Devrim Gündüz <[email protected]>
2018-06-28 11:14 ` Re: RPM Morgue GOLLET Nicolas <[email protected]>
2018-06-28 11:31 ` Re: RPM Morgue Christoph Berg <[email protected]>
@ 2018-06-29 01:38 ` Craig Ringer <[email protected]>
0 siblings, 0 replies; 8+ messages in thread
From: Craig Ringer @ 2018-06-29 01:38 UTC (permalink / raw)
To: Christoph Berg <[email protected]>; GOLLET Nicolas <[email protected]>; Devrim Gündüz <[email protected]>; Craig Ringer <[email protected]>; pgsql-pkg-yum
On 28 June 2018 at 19:31, Christoph Berg <[email protected]> wrote:
> Re: GOLLET Nicolas 2018-06-28 <164461a65cc.e862b2a0141287.
> [email protected]>
> > I had also thought that setting up a public morgue would be a good
> thing, I
> > recently set up an unofficial public morgue (WIP):
> http://pgyum-morgue.ng.pe/
> > (WIP)
> >
> > I'm missing some RPM, if you want us to work together on setting it up
> I'm
> > available;)
> > I had thought not to put all the packages in the morgue but the most
> important
> > ones:
>
> Why not include everything? Filtering is extra effort, and even if you
> get the filters right, they will soon become outdated.
>
>
Right. Also, you're most likely to need postgres debuginfo, which is also
the biggest single package.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
^ permalink raw reply [nested|flat] 8+ messages in thread
end of thread, other threads:[~2018-06-29 01:38 UTC | newest]
Thread overview: 8+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2018-06-25 01:42 RPM Morgue Craig Ringer <[email protected]>
2018-06-25 07:27 ` Christoph Berg <[email protected]>
2018-06-26 23:40 ` Devrim Gündüz <[email protected]>
2018-06-27 02:49 ` Craig Ringer <[email protected]>
2018-06-27 20:35 ` Christoph Berg <[email protected]>
2018-06-28 11:14 ` GOLLET Nicolas <[email protected]>
2018-06-28 11:31 ` Christoph Berg <[email protected]>
2018-06-29 01:38 ` Craig Ringer <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox