pg.ddx.io pgsql-hackers@postgresql.org mailing list archive
help / color / mirror / Atom feedpostpone next week's release
78+ messages / 17 participants
[nested] [flat]
* postpone next week's release
@ 2015-05-29 18:02 Robert Haas <robertmhaas@gmail.com>
0 siblings, 3 replies; 78+ messages in thread
From: Robert Haas @ 2015-05-29 18:02 UTC (permalink / raw)
To: pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
Hi,
I think we should postpone next week's release. I have been hard at
work on the multixact-related bugs that were reported in 9.4.2 and
9.3.7, and the subsequent bugs found by code-reading, but getting them
all fixed by Monday doesn't seem realistic. Such fixes should have
careful review, and not be dashed into the tree under time pressure.
We could do the release anyway to relieve the pain caused by the
fsync-pgdata hard-failure problem, but it seems to me that if we do
that, we're just going to end up having to do yet another release
almost right away. I think it would be better to wait and do one
release that fixes both sets of issues.
Thoughts?
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: postpone next week's release
@ 2015-05-29 18:53 Bruce Momjian <bruce@momjian.us>
parent: Robert Haas <robertmhaas@gmail.com>
2 siblings, 0 replies; 78+ messages in thread
From: Bruce Momjian @ 2015-05-29 18:53 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On Fri, May 29, 2015 at 02:02:43PM -0400, Robert Haas wrote:
> Hi,
>
> I think we should postpone next week's release. I have been hard at
> work on the multixact-related bugs that were reported in 9.4.2 and
> 9.3.7, and the subsequent bugs found by code-reading, but getting them
> all fixed by Monday doesn't seem realistic. Such fixes should have
> careful review, and not be dashed into the tree under time pressure.
>
> We could do the release anyway to relieve the pain caused by the
> fsync-pgdata hard-failure problem, but it seems to me that if we do
> that, we're just going to end up having to do yet another release
> almost right away. I think it would be better to wait and do one
> release that fixes both sets of issues.
It does seem wise to make sure we have all these items fixed. We have
PR'ed the recovery failure issue so I think we are good at this point.
I see having to put out another multi-xact-only fix release the week
after as being a bigger negative.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ Everyone has their own god. +
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: postpone next week's release
@ 2015-05-29 18:54 Stephen Frost <sfrost@snowman.net>
parent: Robert Haas <robertmhaas@gmail.com>
2 siblings, 2 replies; 78+ messages in thread
From: Stephen Frost @ 2015-05-29 18:54 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
* Robert Haas (robertmhaas@gmail.com) wrote:
> I think we should postpone next week's release. I have been hard at
> work on the multixact-related bugs that were reported in 9.4.2 and
> 9.3.7, and the subsequent bugs found by code-reading, but getting them
> all fixed by Monday doesn't seem realistic. Such fixes should have
> careful review, and not be dashed into the tree under time pressure.
>
> We could do the release anyway to relieve the pain caused by the
> fsync-pgdata hard-failure problem, but it seems to me that if we do
> that, we're just going to end up having to do yet another release
> almost right away. I think it would be better to wait and do one
> release that fixes both sets of issues.
Agreed.
I just caution that we appreciate PGCon coming up and that we do our
best to avoid running into a case where we have to push it further due
to everyone being at the conference.
Thanks!
Stephen
Attachments:
signature.asc (application/pgp-signature, 818B) download
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-29 18:57 Bruce Momjian <bruce@momjian.us>
parent: Stephen Frost <sfrost@snowman.net>
1 sibling, 0 replies; 78+ messages in thread
From: Bruce Momjian @ 2015-05-29 18:57 UTC (permalink / raw)
To: Stephen Frost <sfrost@snowman.net>; +Cc: Robert Haas <robertmhaas@gmail.com>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On Fri, May 29, 2015 at 02:54:31PM -0400, Stephen Frost wrote:
> * Robert Haas (robertmhaas@gmail.com) wrote:
> > I think we should postpone next week's release. I have been hard at
> > work on the multixact-related bugs that were reported in 9.4.2 and
> > 9.3.7, and the subsequent bugs found by code-reading, but getting them
> > all fixed by Monday doesn't seem realistic. Such fixes should have
> > careful review, and not be dashed into the tree under time pressure.
> >
> > We could do the release anyway to relieve the pain caused by the
> > fsync-pgdata hard-failure problem, but it seems to me that if we do
> > that, we're just going to end up having to do yet another release
> > almost right away. I think it would be better to wait and do one
> > release that fixes both sets of issues.
>
> Agreed.
>
> I just caution that we appreciate PGCon coming up and that we do our
> best to avoid running into a case where we have to push it further due
> to everyone being at the conference.
This brings up the issue of when we want to do 9.5 beta. Ideas?
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ Everyone has their own god. +
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-29 19:09 Magnus Hagander <magnus@hagander.net>
parent: Robert Haas <robertmhaas@gmail.com>
2 siblings, 2 replies; 78+ messages in thread
From: Magnus Hagander @ 2015-05-29 19:09 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On Fri, May 29, 2015 at 8:02 PM, Robert Haas <robertmhaas@gmail.com> wrote:
> Hi,
>
> I think we should postpone next week's release. I have been hard at
> work on the multixact-related bugs that were reported in 9.4.2 and
> 9.3.7, and the subsequent bugs found by code-reading, but getting them
> all fixed by Monday doesn't seem realistic. Such fixes should have
> careful review, and not be dashed into the tree under time pressure.
>
> We could do the release anyway to relieve the pain caused by the
> fsync-pgdata hard-failure problem, but it seems to me that if we do
> that, we're just going to end up having to do yet another release
> almost right away. I think it would be better to wait and do one
> release that fixes both sets of issues.
>
> Thoughts?
>
I'm a bit split on this.
We *definitely* don't want to release the multixact fix without it being
carefully reviewed, that's the part I'm not split about :) And I fully
appreciate we can't have that done by monday.
However, the file-permission thing seems to hit quite a few people (have we
ever had this many bug reports after a minor release), which means wed
really want to get that out quickly.
Do you have any feeling of how likely people are to actually hit the
multixact one? I've followed some of that impressive debugging you guys
did, and I know it's a pretty critical bug if you hit it, but how
wide-spread will it be?
I guess one option we could do is encourage packagers to push updated
packages (-2 versions) basically. But if we do that, perhaps we might as
well release anyway?
AIUI, the permission thing won't actually be very likely to affect Windows
users. And Windows packages are the ones that take by far the most work to
make. Perhaps we should consider skipping making packages of that version
on Windows, and then plan to push yet another minor one or two weeks later,
that goes out on all platforms?
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-29 19:11 Magnus Hagander <magnus@hagander.net>
parent: Stephen Frost <sfrost@snowman.net>
1 sibling, 2 replies; 78+ messages in thread
From: Magnus Hagander @ 2015-05-29 19:11 UTC (permalink / raw)
To: Stephen Frost <sfrost@snowman.net>; +Cc: Robert Haas <robertmhaas@gmail.com>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On Fri, May 29, 2015 at 8:54 PM, Stephen Frost <sfrost@snowman.net> wrote:
> * Robert Haas (robertmhaas@gmail.com) wrote:
> > I think we should postpone next week's release. I have been hard at
> > work on the multixact-related bugs that were reported in 9.4.2 and
> > 9.3.7, and the subsequent bugs found by code-reading, but getting them
> > all fixed by Monday doesn't seem realistic. Such fixes should have
> > careful review, and not be dashed into the tree under time pressure.
> >
> > We could do the release anyway to relieve the pain caused by the
> > fsync-pgdata hard-failure problem, but it seems to me that if we do
> > that, we're just going to end up having to do yet another release
> > almost right away. I think it would be better to wait and do one
> > release that fixes both sets of issues.
>
> Agreed.
>
> I just caution that we appreciate PGCon coming up and that we do our
> best to avoid running into a case where we have to push it further due
> to everyone being at the conference.
>
If we plan it, we certainly *can* make a release during pgcon. If that's
what the reasonable timing comes down to, I think getting these fixes out
definitely has to be considered more important than the conference, so a
few of us will just have to take a break...
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-29 19:18 Robert Haas <robertmhaas@gmail.com>
parent: Magnus Hagander <magnus@hagander.net>
1 sibling, 1 reply; 78+ messages in thread
From: Robert Haas @ 2015-05-29 19:18 UTC (permalink / raw)
To: Magnus Hagander <magnus@hagander.net>; +Cc: pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On Fri, May 29, 2015 at 3:09 PM, Magnus Hagander <magnus@hagander.net> wrote:
> Do you have any feeling of how likely people are to actually hit the
> multixact one? I've followed some of that impressive debugging you guys did,
> and I know it's a pretty critical bug if you hit it, but how wide-spread
> will it be?
That precise problem has been reported a few times, but it may not be
widespread. I don't know. My bigger concern is that, at present,
taking a base backup is broken. I haven't figured out the exact
reproduction scenario, but I think it's something like this:
- begin base backup
- checkpoint happens, truncating pg_multixact
- at this point pg_multixact gets copied
- end base backup
I think what will happen on replay is that replaying the checkpoint,
it will try to reference pg_multixact files that don't exist any more
and die with a fatal error.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-29 19:31 Stephen Frost <sfrost@snowman.net>
parent: Magnus Hagander <magnus@hagander.net>
1 sibling, 0 replies; 78+ messages in thread
From: Stephen Frost @ 2015-05-29 19:31 UTC (permalink / raw)
To: Magnus Hagander <magnus@hagander.net>; +Cc: Robert Haas <robertmhaas@gmail.com>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
* Magnus Hagander (magnus@hagander.net) wrote:
> On Fri, May 29, 2015 at 8:54 PM, Stephen Frost <sfrost@snowman.net> wrote:
>
> > * Robert Haas (robertmhaas@gmail.com) wrote:
> > > I think we should postpone next week's release. I have been hard at
> > > work on the multixact-related bugs that were reported in 9.4.2 and
> > > 9.3.7, and the subsequent bugs found by code-reading, but getting them
> > > all fixed by Monday doesn't seem realistic. Such fixes should have
> > > careful review, and not be dashed into the tree under time pressure.
> > >
> > > We could do the release anyway to relieve the pain caused by the
> > > fsync-pgdata hard-failure problem, but it seems to me that if we do
> > > that, we're just going to end up having to do yet another release
> > > almost right away. I think it would be better to wait and do one
> > > release that fixes both sets of issues.
> >
> > Agreed.
> >
> > I just caution that we appreciate PGCon coming up and that we do our
> > best to avoid running into a case where we have to push it further due
> > to everyone being at the conference.
>
> If we plan it, we certainly *can* make a release during pgcon. If that's
> what the reasonable timing comes down to, I think getting these fixes out
> definitely has to be considered more important than the conference, so a
> few of us will just have to take a break...
I don't disagree with you about any of that, just wanted to make mention
of the timing.
Thanks!
Stephen
Attachments:
signature.asc (application/pgp-signature, 818B) download
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-29 19:32 Joshua D. Drake <jd@commandprompt.com>
parent: Robert Haas <robertmhaas@gmail.com>
0 siblings, 0 replies; 78+ messages in thread
From: Joshua D. Drake @ 2015-05-29 19:32 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; Magnus Hagander <magnus@hagander.net>; +Cc: pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On 05/29/2015 12:18 PM, Robert Haas wrote:
>
> On Fri, May 29, 2015 at 3:09 PM, Magnus Hagander <magnus@hagander.net> wrote:
>> Do you have any feeling of how likely people are to actually hit the
>> multixact one? I've followed some of that impressive debugging you guys did,
>> and I know it's a pretty critical bug if you hit it, but how wide-spread
>> will it be?
>
> That precise problem has been reported a few times, but it may not be
> widespread. I don't know. My bigger concern is that, at present,
> taking a base backup is broken.
This I think is the bigger issue. They both are horrible but basebackup
being broken is rather... egregious.
JD
--
Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-29 19:32 Tom Lane <tgl@sss.pgh.pa.us>
parent: Magnus Hagander <magnus@hagander.net>
1 sibling, 3 replies; 78+ messages in thread
From: Tom Lane @ 2015-05-29 19:32 UTC (permalink / raw)
To: Magnus Hagander <magnus@hagander.net>; +Cc: Stephen Frost <sfrost@snowman.net>; Robert Haas <robertmhaas@gmail.com>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
Magnus Hagander <magnus@hagander.net> writes:
> On Fri, May 29, 2015 at 8:54 PM, Stephen Frost <sfrost@snowman.net> wrote:
>> I just caution that we appreciate PGCon coming up and that we do our
>> best to avoid running into a case where we have to push it further due
>> to everyone being at the conference.
> If we plan it, we certainly *can* make a release during pgcon. If that's
> what the reasonable timing comes down to, I think getting these fixes out
> definitely has to be considered more important than the conference, so a
> few of us will just have to take a break...
I think there's no way that we wait more than one additional week to push
the fsync fix. So the problem is not with scheduling the update releases,
it's with whether we can also fit in a 9.5 beta release before PGCon.
(I can't see doing a beta *during* PGCon week. I for one am going to be
on an airplane at the time I'd normally have to be Doing Release Stuff.)
I know Josh doesn't like to do beta1 releases concurrently with back
branches because it confuses the PR messaging. But we could make an
exception perhaps; or do all those releases the same week but announce
the beta the day after the bugfix releases.
Or we just let the beta slide till after PGCon, but then I think we're
missing some excitement factor.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-29 19:35 Magnus Hagander <magnus@hagander.net>
parent: Tom Lane <tgl@sss.pgh.pa.us>
2 siblings, 1 reply; 78+ messages in thread
From: Magnus Hagander @ 2015-05-29 19:35 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Stephen Frost <sfrost@snowman.net>; Robert Haas <robertmhaas@gmail.com>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On Fri, May 29, 2015 at 9:32 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Magnus Hagander <magnus@hagander.net> writes:
> > On Fri, May 29, 2015 at 8:54 PM, Stephen Frost <sfrost@snowman.net>
> wrote:
> >> I just caution that we appreciate PGCon coming up and that we do our
> >> best to avoid running into a case where we have to push it further due
> >> to everyone being at the conference.
>
> > If we plan it, we certainly *can* make a release during pgcon. If that's
> > what the reasonable timing comes down to, I think getting these fixes out
> > definitely has to be considered more important than the conference, so a
> > few of us will just have to take a break...
>
> I think there's no way that we wait more than one additional week to push
> the fsync fix. So the problem is not with scheduling the update releases,
> it's with whether we can also fit in a 9.5 beta release before PGCon.
>
I think 9.5 beta has to stand back. The question is what we do with the
potentially two minor releases. Then we can slot in the beta whenever.
If we do the minor as currently planned, can we do another one the week
after to deal with the multixact issues? (scheduling wise we're going to
have to do one the week after *regardless*, the question is if we can make
two different ones, or if we need to fold them into one)
(I can't see doing a beta *during* PGCon week. I for one am going to be
> on an airplane at the time I'd normally have to be Doing Release Stuff.)
>
Agreed. We can push a *minor* during pgcon, but not beta.
I know Josh doesn't like to do beta1 releases concurrently with back
> branches because it confuses the PR messaging. But we could make an
> exception perhaps; or do all those releases the same week but announce
> the beta the day after the bugfix releases.
>
I can't comment on the PR parts, I'll leave that to Josh.
>
> Or we just let the beta slide till after PGCon, but then I think we're
> missing some excitement factor.
>
Well, most of the people going to pgcon know it already. And most of the
excitement affects people who are not at pgcon (simply based on that most
of our users are not at pgcon). If doing it the week after pgcon is what
ends up making sense once weve figured out what to do with the minors, then
so be it, IMNSHO.
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-29 19:37 Stephen Frost <sfrost@snowman.net>
parent: Tom Lane <tgl@sss.pgh.pa.us>
2 siblings, 0 replies; 78+ messages in thread
From: Stephen Frost @ 2015-05-29 19:37 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Magnus Hagander <magnus@hagander.net>; Robert Haas <robertmhaas@gmail.com>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
* Tom Lane (tgl@sss.pgh.pa.us) wrote:
> (I can't see doing a beta *during* PGCon week. I for one am going to be
> on an airplane at the time I'd normally have to be Doing Release Stuff.)
[...]
> Or we just let the beta slide till after PGCon, but then I think we're
> missing some excitement factor.
Personally, I'd be all for a "watch Tom do the 9.5 beta release!"
Unconference slot...
:)
(mostly kidding, but I'm 100% sure it'd draw a huge crowd..)
Thanks!
Stephen
Attachments:
signature.asc (application/pgp-signature, 818B) download
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-29 19:43 Tom Lane <tgl@sss.pgh.pa.us>
parent: Magnus Hagander <magnus@hagander.net>
0 siblings, 1 reply; 78+ messages in thread
From: Tom Lane @ 2015-05-29 19:43 UTC (permalink / raw)
To: Magnus Hagander <magnus@hagander.net>; +Cc: Stephen Frost <sfrost@snowman.net>; Robert Haas <robertmhaas@gmail.com>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
Magnus Hagander <magnus@hagander.net> writes:
> On Fri, May 29, 2015 at 9:32 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> I think there's no way that we wait more than one additional week to push
>> the fsync fix. So the problem is not with scheduling the update releases,
>> it's with whether we can also fit in a 9.5 beta release before PGCon.
> I think 9.5 beta has to stand back. The question is what we do with the
> potentially two minor releases. Then we can slot in the beta whenever.
> If we do the minor as currently planned, can we do another one the week
> after to deal with the multixact issues? (scheduling wise we're going to
> have to do one the week after *regardless*, the question is if we can make
> two different ones, or if we need to fold them into one)
I suppose we could, but it doubles the amount of release gruntwork
involved, and it doesn't exactly make us look good to our users either.
I believe Christoph indicated that he was going to cherry-pick the fsync
patch and push out an intermediate Debian package with that fix, so at
least for that community there is not an urgent reason to get out a set
of releases with only the fsync fixes and not the multixact fixes. I'm
not clear though on how many of the other reports we heard came from
Debian users. (Some of them did, but maybe not all.)
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-29 19:46 Stephen Frost <sfrost@snowman.net>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 78+ messages in thread
From: Stephen Frost @ 2015-05-29 19:46 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Magnus Hagander <magnus@hagander.net>; Robert Haas <robertmhaas@gmail.com>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
* Tom Lane (tgl@sss.pgh.pa.us) wrote:
> Magnus Hagander <magnus@hagander.net> writes:
> > On Fri, May 29, 2015 at 9:32 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> >> I think there's no way that we wait more than one additional week to push
> >> the fsync fix. So the problem is not with scheduling the update releases,
> >> it's with whether we can also fit in a 9.5 beta release before PGCon.
>
> > I think 9.5 beta has to stand back. The question is what we do with the
> > potentially two minor releases. Then we can slot in the beta whenever.
>
> > If we do the minor as currently planned, can we do another one the week
> > after to deal with the multixact issues? (scheduling wise we're going to
> > have to do one the week after *regardless*, the question is if we can make
> > two different ones, or if we need to fold them into one)
>
> I suppose we could, but it doubles the amount of release gruntwork
> involved, and it doesn't exactly make us look good to our users either.
Agreed. Makes it look like we can't manage to figure out our bugs and
put fixes for them together in sensible releases..
Thanks!
Stephen
Attachments:
signature.asc (application/pgp-signature, 818B) download
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-29 19:47 Magnus Hagander <magnus@hagander.net>
parent: Stephen Frost <sfrost@snowman.net>
0 siblings, 1 reply; 78+ messages in thread
From: Magnus Hagander @ 2015-05-29 19:47 UTC (permalink / raw)
To: Stephen Frost <sfrost@snowman.net>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Robert Haas <robertmhaas@gmail.com>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On Fri, May 29, 2015 at 9:46 PM, Stephen Frost <sfrost@snowman.net> wrote:
> * Tom Lane (tgl@sss.pgh.pa.us) wrote:
> > Magnus Hagander <magnus@hagander.net> writes:
> > > On Fri, May 29, 2015 at 9:32 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > >> I think there's no way that we wait more than one additional week to
> push
> > >> the fsync fix. So the problem is not with scheduling the update
> releases,
> > >> it's with whether we can also fit in a 9.5 beta release before PGCon.
> >
> > > I think 9.5 beta has to stand back. The question is what we do with the
> > > potentially two minor releases. Then we can slot in the beta whenever.
> >
> > > If we do the minor as currently planned, can we do another one the week
> > > after to deal with the multixact issues? (scheduling wise we're going
> to
> > > have to do one the week after *regardless*, the question is if we can
> make
> > > two different ones, or if we need to fold them into one)
> >
> > I suppose we could, but it doubles the amount of release gruntwork
> > involved, and it doesn't exactly make us look good to our users either.
>
> Agreed. Makes it look like we can't manage to figure out our bugs and
> put fixes for them together in sensible releases..
>
The flipside of that is that we have a bug fix that's preventing peoples
databases from starting, and we're the intentionally delaying the shipment
of it. Though i guess a mitigating fact there is that it is very easy to
manually recover from that. But it's painful if your db server restarts
awhen you're not around...
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-29 19:53 Stephen Frost <sfrost@snowman.net>
parent: Magnus Hagander <magnus@hagander.net>
0 siblings, 0 replies; 78+ messages in thread
From: Stephen Frost @ 2015-05-29 19:53 UTC (permalink / raw)
To: Magnus Hagander <magnus@hagander.net>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Robert Haas <robertmhaas@gmail.com>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
* Magnus Hagander (magnus@hagander.net) wrote:
> On Fri, May 29, 2015 at 9:46 PM, Stephen Frost <sfrost@snowman.net> wrote:
>
> > * Tom Lane (tgl@sss.pgh.pa.us) wrote:
> > > Magnus Hagander <magnus@hagander.net> writes:
> > > > On Fri, May 29, 2015 at 9:32 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > > >> I think there's no way that we wait more than one additional week to
> > push
> > > >> the fsync fix. So the problem is not with scheduling the update
> > releases,
> > > >> it's with whether we can also fit in a 9.5 beta release before PGCon.
> > >
> > > > I think 9.5 beta has to stand back. The question is what we do with the
> > > > potentially two minor releases. Then we can slot in the beta whenever.
> > >
> > > > If we do the minor as currently planned, can we do another one the week
> > > > after to deal with the multixact issues? (scheduling wise we're going
> > to
> > > > have to do one the week after *regardless*, the question is if we can
> > make
> > > > two different ones, or if we need to fold them into one)
> > >
> > > I suppose we could, but it doubles the amount of release gruntwork
> > > involved, and it doesn't exactly make us look good to our users either.
> >
> > Agreed. Makes it look like we can't manage to figure out our bugs and
> > put fixes for them together in sensible releases..
> >
>
> The flipside of that is that we have a bug fix that's preventing peoples
> databases from starting, and we're the intentionally delaying the shipment
> of it. Though i guess a mitigating fact there is that it is very easy to
> manually recover from that. But it's painful if your db server restarts
> awhen you're not around...
And we have *another* fix for a *data corruption* bug which is coming in
the following *week*.
Yes, I think delaying a week to get both in is better than putting out a
fix for one bug when we *know* there's a data corruption bug sitting in
that code, and we're putting out a fix for it the following week.
If we were talking about a month-long delay, that'd be one thing, but
that isn't the impression I've got about what we're talking about.
Thanks!
Stephen
Attachments:
signature.asc (application/pgp-signature, 818B) download
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-29 19:56 Bruce Momjian <bruce@momjian.us>
parent: Tom Lane <tgl@sss.pgh.pa.us>
2 siblings, 1 reply; 78+ messages in thread
From: Bruce Momjian @ 2015-05-29 19:56 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Magnus Hagander <magnus@hagander.net>; Stephen Frost <sfrost@snowman.net>; Robert Haas <robertmhaas@gmail.com>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On Fri, May 29, 2015 at 03:32:57PM -0400, Tom Lane wrote:
> I know Josh doesn't like to do beta1 releases concurrently with back
> branches because it confuses the PR messaging. But we could make an
> exception perhaps; or do all those releases the same week but announce
> the beta the day after the bugfix releases.
>
> Or we just let the beta slide till after PGCon, but then I think we're
> missing some excitement factor.
I am unclear if we are anywhere near ready for beta1 even in June. Are
we?
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ Everyone has their own god. +
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-29 19:57 Stephen Frost <sfrost@snowman.net>
parent: Bruce Momjian <bruce@momjian.us>
0 siblings, 1 reply; 78+ messages in thread
From: Stephen Frost @ 2015-05-29 19:57 UTC (permalink / raw)
To: Bruce Momjian <bruce@momjian.us>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Magnus Hagander <magnus@hagander.net>; Robert Haas <robertmhaas@gmail.com>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
* Bruce Momjian (bruce@momjian.us) wrote:
> On Fri, May 29, 2015 at 03:32:57PM -0400, Tom Lane wrote:
> > I know Josh doesn't like to do beta1 releases concurrently with back
> > branches because it confuses the PR messaging. But we could make an
> > exception perhaps; or do all those releases the same week but announce
> > the beta the day after the bugfix releases.
> >
> > Or we just let the beta slide till after PGCon, but then I think we're
> > missing some excitement factor.
>
> I am unclear if we are anywhere near ready for beta1 even in June. Are
> we?
I'm all about having that discussion... but can we do it on another
thread or at least wait til we've decided about the back-branch
releases? They are clearly the more important issue to consider.
Thanks!
Stephen
Attachments:
signature.asc (application/pgp-signature, 818B) download
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-29 20:01 Tom Lane <tgl@sss.pgh.pa.us>
parent: Stephen Frost <sfrost@snowman.net>
0 siblings, 4 replies; 78+ messages in thread
From: Tom Lane @ 2015-05-29 20:01 UTC (permalink / raw)
To: Stephen Frost <sfrost@snowman.net>; +Cc: Bruce Momjian <bruce@momjian.us>; Magnus Hagander <magnus@hagander.net>; Robert Haas <robertmhaas@gmail.com>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
Stephen Frost <sfrost@snowman.net> writes:
> * Bruce Momjian (bruce@momjian.us) wrote:
>> I am unclear if we are anywhere near ready for beta1 even in June. Are
>> we?
> I'm all about having that discussion... but can we do it on another
> thread or at least wait til we've decided about the back-branch
> releases? They are clearly the more important issue to consider.
It's the same discussion though, ie what releases are we expecting to
get out in the next couple of weeks.
It's possible that we ought to give up on a pre-conference beta.
Certainly a whole lot of time that I'd hoped would go into reviewing
9.5 feature commits has instead gone into back-branch bug chasing this
week.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-29 20:03 Stephen Frost <sfrost@snowman.net>
parent: Tom Lane <tgl@sss.pgh.pa.us>
3 siblings, 1 reply; 78+ messages in thread
From: Stephen Frost @ 2015-05-29 20:03 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Bruce Momjian <bruce@momjian.us>; Magnus Hagander <magnus@hagander.net>; Robert Haas <robertmhaas@gmail.com>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
* Tom Lane (tgl@sss.pgh.pa.us) wrote:
> It's possible that we ought to give up on a pre-conference beta.
> Certainly a whole lot of time that I'd hoped would go into reviewing
> 9.5 feature commits has instead gone into back-branch bug chasing this
> week.
I guess that's what I'm getting at. We need to take care of the
back-branches and that means pushing beta back. I fully expect a good
discussion on when to release beta when we get closer on that, but we're
not going to be close while we have outstanding big back-branch bugs.
Thanks!
Stephen
Attachments:
signature.asc (application/pgp-signature, 818B) download
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-29 20:07 Bruce Momjian <bruce@momjian.us>
parent: Tom Lane <tgl@sss.pgh.pa.us>
3 siblings, 0 replies; 78+ messages in thread
From: Bruce Momjian @ 2015-05-29 20:07 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Stephen Frost <sfrost@snowman.net>; Magnus Hagander <magnus@hagander.net>; Robert Haas <robertmhaas@gmail.com>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On Fri, May 29, 2015 at 04:01:00PM -0400, Tom Lane wrote:
> Stephen Frost <sfrost@snowman.net> writes:
> > * Bruce Momjian (bruce@momjian.us) wrote:
> >> I am unclear if we are anywhere near ready for beta1 even in June. Are
> >> we?
>
> > I'm all about having that discussion... but can we do it on another
> > thread or at least wait til we've decided about the back-branch
> > releases? They are clearly the more important issue to consider.
>
> It's the same discussion though, ie what releases are we expecting to
> get out in the next couple of weeks.
Agreed. If we want to put out beta1 before PGCon, I need to start on
the release notes on Monday.
> It's possible that we ought to give up on a pre-conference beta.
> Certainly a whole lot of time that I'd hoped would go into reviewing
> 9.5 feature commits has instead gone into back-branch bug chasing this
> week.
Based on what has transpired in the past two weeks, I am thinking we
need to move _slower_, not faster. I am concerned we have focused so
much on new features that we have taken our eye off of reliability.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ Everyone has their own god. +
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-29 20:08 Joshua D. Drake <jd@commandprompt.com>
parent: Stephen Frost <sfrost@snowman.net>
0 siblings, 0 replies; 78+ messages in thread
From: Joshua D. Drake @ 2015-05-29 20:08 UTC (permalink / raw)
To: Stephen Frost <sfrost@snowman.net>; Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Bruce Momjian <bruce@momjian.us>; Magnus Hagander <magnus@hagander.net>; Robert Haas <robertmhaas@gmail.com>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On 05/29/2015 01:03 PM, Stephen Frost wrote:
> * Tom Lane (tgl@sss.pgh.pa.us) wrote:
>> It's possible that we ought to give up on a pre-conference beta.
>> Certainly a whole lot of time that I'd hoped would go into reviewing
>> 9.5 feature commits has instead gone into back-branch bug chasing this
>> week.
>
> I guess that's what I'm getting at. We need to take care of the
> back-branches and that means pushing beta back.
+1
JD
--
The most kicking donkey PostgreSQL Infrastructure company in existence.
The oldest, the most experienced, the consulting company to the stars.
Command Prompt, Inc. http://www.commandprompt.com/ +1 -503-667-4564 -
24x7 - 365 - Proactive and Managed Professional Services!
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-29 20:29 Robert Haas <robertmhaas@gmail.com>
parent: Tom Lane <tgl@sss.pgh.pa.us>
3 siblings, 1 reply; 78+ messages in thread
From: Robert Haas @ 2015-05-29 20:29 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Stephen Frost <sfrost@snowman.net>; Bruce Momjian <bruce@momjian.us>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On Fri, May 29, 2015 at 4:01 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> It's possible that we ought to give up on a pre-conference beta.
> Certainly a whole lot of time that I'd hoped would go into reviewing
> 9.5 feature commits has instead gone into back-branch bug chasing this
> week.
I'm personally kind of astonished that we're even thinking about beta
so soon. I mean, we at least need to go through the stuff listed
here, I think:
https://wiki.postgresql.org/wiki/PostgreSQL_9.5_Open_Items
The bigger issue is: what's NOT on that list that should be? I think
we need to devote some cycles to figuring that out, and I sure haven't
had any this week.
In any case, I think the negative PR that we're going to get from not
getting this multixact stuff taken care of is going to far outweigh
any positive PR from getting 9.5beta1 out a little sooner, especially
if 9.5beta1 is bug-ridden because we gave it no time to settle.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-29 20:37 Tom Lane <tgl@sss.pgh.pa.us>
parent: Robert Haas <robertmhaas@gmail.com>
0 siblings, 2 replies; 78+ messages in thread
From: Tom Lane @ 2015-05-29 20:37 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: Stephen Frost <sfrost@snowman.net>; Bruce Momjian <bruce@momjian.us>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
Robert Haas <robertmhaas@gmail.com> writes:
> I'm personally kind of astonished that we're even thinking about beta
> so soon. I mean, we at least need to go through the stuff listed
> here, I think:
> https://wiki.postgresql.org/wiki/PostgreSQL_9.5_Open_Items
Well, maybe we ought to call it an alpha not a beta, but I think we ought
to put out some kind of release that we can encourage people to test.
What you are suggesting is that we serialize resolution of the known
issues with discovery of new issues, and that's not an efficient use of
time. Especially seeing that we're approaching the summer season where
we won't get much input at all.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-29 21:04 Andres Freund <andres@anarazel.de>
parent: Tom Lane <tgl@sss.pgh.pa.us>
1 sibling, 1 reply; 78+ messages in thread
From: Andres Freund @ 2015-05-29 21:04 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Robert Haas <robertmhaas@gmail.com>; Stephen Frost <sfrost@snowman.net>; Bruce Momjian <bruce@momjian.us>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On 2015-05-29 16:37:00 -0400, Tom Lane wrote:
> Well, maybe we ought to call it an alpha not a beta, but I think we ought
> to put out some kind of release that we can encourage people to test.
I also do think it's important that we put out a beta (or alpha)
relatively soon. Both because we actually need input to find out what
works and what doesn't and also because it pushes us to tie up loose
ends.
A beta with open items isn't that bad a thing? There's many bigger
projects doing 4-8 betas releases before a major one; and most of them
have open items at the indvidual beta's release times.
I think we should define/document it so that there's no hard goal of
being compatible for beta releases and that the compatibility goal
starts with the first release candidate, and not the betas.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-29 21:12 Bruce Momjian <bruce@momjian.us>
parent: Andres Freund <andres@anarazel.de>
0 siblings, 2 replies; 78+ messages in thread
From: Bruce Momjian @ 2015-05-29 21:12 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Robert Haas <robertmhaas@gmail.com>; Stephen Frost <sfrost@snowman.net>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On Fri, May 29, 2015 at 11:04:59PM +0200, Andres Freund wrote:
> On 2015-05-29 16:37:00 -0400, Tom Lane wrote:
> > Well, maybe we ought to call it an alpha not a beta, but I think we ought
> > to put out some kind of release that we can encourage people to test.
>
> I also do think it's important that we put out a beta (or alpha)
> relatively soon. Both because we actually need input to find out what
> works and what doesn't and also because it pushes us to tie up loose
> ends.
>
> A beta with open items isn't that bad a thing? There's many bigger
> projects doing 4-8 betas releases before a major one; and most of them
> have open items at the indvidual beta's release times.
>
> I think we should define/document it so that there's no hard goal of
> being compatible for beta releases and that the compatibility goal
> starts with the first release candidate, and not the betas.
Do we need release notes for an alpha? Once I do the release notes, it
is possible to miss subtle changes in the code that aren't mentioned in
commit messages.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ Everyone has their own god. +
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-29 21:14 Andres Freund <andres@anarazel.de>
parent: Bruce Momjian <bruce@momjian.us>
1 sibling, 0 replies; 78+ messages in thread
From: Andres Freund @ 2015-05-29 21:14 UTC (permalink / raw)
To: Bruce Momjian <bruce@momjian.us>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Robert Haas <robertmhaas@gmail.com>; Stephen Frost <sfrost@snowman.net>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On May 29, 2015 2:12:24 PM PDT, Bruce Momjian <bruce@momjian.us> wrote:
>On Fri, May 29, 2015 at 11:04:59PM +0200, Andres Freund wrote:
>> On 2015-05-29 16:37:00 -0400, Tom Lane wrote:
>> > Well, maybe we ought to call it an alpha not a beta, but I think we
>ought
>> > to put out some kind of release that we can encourage people to
>test.
>>
>> I also do think it's important that we put out a beta (or alpha)
>> relatively soon. Both because we actually need input to find out what
>> works and what doesn't and also because it pushes us to tie up loose
>> ends.
>>
>> A beta with open items isn't that bad a thing? There's many bigger
>> projects doing 4-8 betas releases before a major one; and most of
>them
>> have open items at the indvidual beta's release times.
>>
>> I think we should define/document it so that there's no hard goal of
>> being compatible for beta releases and that the compatibility goal
>> starts with the first release candidate, and not the betas.
>
>Do we need release notes for an alpha? Once I do the release notes, it
>is possible to miss subtle changes in the code that aren't mentioned in
>commit messages.
Yes I think so. Otherwise it's pretty useless for people not following closely. I see little point in explicitly delaying release note work any further.
Andres
---
Please excuse brevity and formatting - I am writing this on my mobile phone.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-29 21:37 Tom Lane <tgl@sss.pgh.pa.us>
parent: Bruce Momjian <bruce@momjian.us>
1 sibling, 1 reply; 78+ messages in thread
From: Tom Lane @ 2015-05-29 21:37 UTC (permalink / raw)
To: Bruce Momjian <bruce@momjian.us>; +Cc: Andres Freund <andres@anarazel.de>; Robert Haas <robertmhaas@gmail.com>; Stephen Frost <sfrost@snowman.net>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
Bruce Momjian <bruce@momjian.us> writes:
> Do we need release notes for an alpha? Once I do the release notes, it
> is possible to miss subtle changes in the code that aren't mentioned in
> commit messages.
If the commit message isn't clear about something, you'd likely miss the
issue anyway, no? Anyway, once the release notes are in the tree, we
could expect that anyone committing a user-visible semantics change should
update the release notes themselves.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-29 22:02 Robert Haas <robertmhaas@gmail.com>
parent: Tom Lane <tgl@sss.pgh.pa.us>
1 sibling, 1 reply; 78+ messages in thread
From: Robert Haas @ 2015-05-29 22:02 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Stephen Frost <sfrost@snowman.net>; Bruce Momjian <bruce@momjian.us>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On Fri, May 29, 2015 at 4:37 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Robert Haas <robertmhaas@gmail.com> writes:
>> I'm personally kind of astonished that we're even thinking about beta
>> so soon. I mean, we at least need to go through the stuff listed
>> here, I think:
>> https://wiki.postgresql.org/wiki/PostgreSQL_9.5_Open_Items
>
> Well, maybe we ought to call it an alpha not a beta, but I think we ought
> to put out some kind of release that we can encourage people to test.
> What you are suggesting is that we serialize resolution of the known
> issues with discovery of new issues, and that's not an efficient use of
> time. Especially seeing that we're approaching the summer season where
> we won't get much input at all.
Well, I think we ought to take at least a few weeks to try to do a bit
of code review and clean up what we can from the open items list.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-29 22:28 Bruce Momjian <bruce@momjian.us>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 78+ messages in thread
From: Bruce Momjian @ 2015-05-29 22:28 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Andres Freund <andres@anarazel.de>; Robert Haas <robertmhaas@gmail.com>; Stephen Frost <sfrost@snowman.net>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On Fri, May 29, 2015 at 05:37:13PM -0400, Tom Lane wrote:
> Bruce Momjian <bruce@momjian.us> writes:
> > Do we need release notes for an alpha? Once I do the release notes, it
> > is possible to miss subtle changes in the code that aren't mentioned in
> > commit messages.
>
> If the commit message isn't clear about something, you'd likely miss the
> issue anyway, no? Anyway, once the release notes are in the tree, we
I often do research in the git tree to get details on the feature beyond
just looking at the commit or the patch.
> could expect that anyone committing a user-visible semantics change should
> update the release notes themselves.
Yes, that would be nice.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ Everyone has their own god. +
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-29 22:33 Andres Freund <andres@anarazel.de>
parent: Robert Haas <robertmhaas@gmail.com>
0 siblings, 2 replies; 78+ messages in thread
From: Andres Freund @ 2015-05-29 22:33 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Stephen Frost <sfrost@snowman.net>; Bruce Momjian <bruce@momjian.us>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On 2015-05-29 18:02:36 -0400, Robert Haas wrote:
> Well, I think we ought to take at least a few weeks to try to do a bit
> of code review and clean up what we can from the open items list.
Why? A large portion of the input required to go from beta towards a
release is from actual users. To see when things break, what confuses
them and such.
I don't see why that requires that there are no minor entries in the
open items list - and that's what currently is on it. Neither does it
seem to be a problem to do code review concurrently to user beta
testing. We obviously can't start a beta if things crash left and
right, but I don't think that's the situation right now?
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-30 00:07 Stephen Frost <sfrost@snowman.net>
parent: Andres Freund <andres@anarazel.de>
1 sibling, 0 replies; 78+ messages in thread
From: Stephen Frost @ 2015-05-30 00:07 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Robert Haas <robertmhaas@gmail.com>; Tom Lane <tgl@sss.pgh.pa.us>; Bruce Momjian <bruce@momjian.us>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
* Andres Freund (andres@anarazel.de) wrote:
> On 2015-05-29 18:02:36 -0400, Robert Haas wrote:
> > Well, I think we ought to take at least a few weeks to try to do a bit
> > of code review and clean up what we can from the open items list.
>
> Why? A large portion of the input required to go from beta towards a
> release is from actual users. To see when things break, what confuses
> them and such.
>
> I don't see why that requires that there are no minor entries in the
> open items list - and that's what currently is on it. Neither does it
> seem to be a problem to do code review concurrently to user beta
> testing. We obviously can't start a beta if things crash left and
> right, but I don't think that's the situation right now?
Agreed.
Thanks!
Stephen
Attachments:
signature.asc (application/pgp-signature, 818B) download
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-30 03:56 Robert Haas <robertmhaas@gmail.com>
parent: Andres Freund <andres@anarazel.de>
1 sibling, 2 replies; 78+ messages in thread
From: Robert Haas @ 2015-05-30 03:56 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Stephen Frost <sfrost@snowman.net>; Bruce Momjian <bruce@momjian.us>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On Fri, May 29, 2015 at 6:33 PM, Andres Freund <andres@anarazel.de> wrote:
> On 2015-05-29 18:02:36 -0400, Robert Haas wrote:
>> Well, I think we ought to take at least a few weeks to try to do a bit
>> of code review and clean up what we can from the open items list.
>
> Why? A large portion of the input required to go from beta towards a
> release is from actual users. To see when things break, what confuses
> them and such.
I have two concerns:
1. I'm concerned that once we release beta, any idea about reverting a
feature or fixing something that is broken will get harder, because
people will say "well, we can't do that after we've released a beta".
I confess to particularly wanting a solution to the item listed as
"custom-join has no way to construct Plan nodes of child Path nodes",
the history of which I'll avoid recapitulating until I'm sure I can do
it while maintaining my blood pressure at safe levels.
2. Also, if we're going to make significant multixact-related changes
to 9.5 to try to improve reliability, as you proposed on the other
thread, then it would be nice to do that before beta, so that it gets
tested. Of course, someone is bound to point out that we could make
those changes in time for beta2, and people could test that. But in
practice I think that'll just mean that stuff is only out there for
let's say 2 months before we put it in a major release, which ain't
much.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-30 04:08 Tom Lane <tgl@sss.pgh.pa.us>
parent: Robert Haas <robertmhaas@gmail.com>
1 sibling, 4 replies; 78+ messages in thread
From: Tom Lane @ 2015-05-30 04:08 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: Andres Freund <andres@anarazel.de>; Stephen Frost <sfrost@snowman.net>; Bruce Momjian <bruce@momjian.us>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
Robert Haas <robertmhaas@gmail.com> writes:
> On Fri, May 29, 2015 at 6:33 PM, Andres Freund <andres@anarazel.de> wrote:
>> Why? A large portion of the input required to go from beta towards a
>> release is from actual users. To see when things break, what confuses
>> them and such.
> I have two concerns:
> 1. I'm concerned that once we release beta, any idea about reverting a
> feature or fixing something that is broken will get harder, because
> people will say "well, we can't do that after we've released a beta".
> I confess to particularly wanting a solution to the item listed as
> "custom-join has no way to construct Plan nodes of child Path nodes",
> the history of which I'll avoid recapitulating until I'm sure I can do
> it while maintaining my blood pressure at safe levels.
> 2. Also, if we're going to make significant multixact-related changes
> to 9.5 to try to improve reliability, as you proposed on the other
> thread, then it would be nice to do that before beta, so that it gets
> tested. Of course, someone is bound to point out that we could make
> those changes in time for beta2, and people could test that. But in
> practice I think that'll just mean that stuff is only out there for
> let's say 2 months before we put it in a major release, which ain't
> much.
I think your position is completely nuts. The GROUPING SETS code is
desperately in need of testing. The custom-plan code is desperately
in need of fixing and testing. The multixact code is desperately
in need of testing. The open-items list has several other problems
besides those. All of those problems are independent. If we insist
on tackling them serially rather than in parallel, 9.5 might not come
out till 2017.
I agree that we are not in a position to promise features won't change.
So let's call it an alpha not a beta --- but for heaven's sake let's
try to move forward on all these issues, not just some of them.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-30 04:17 Andres Freund <andres@anarazel.de>
parent: Robert Haas <robertmhaas@gmail.com>
1 sibling, 0 replies; 78+ messages in thread
From: Andres Freund @ 2015-05-30 04:17 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Stephen Frost <sfrost@snowman.net>; Bruce Momjian <bruce@momjian.us>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On May 29, 2015 8:56:40 PM PDT, Robert Haas <robertmhaas@gmail.com> wrote:
>On Fri, May 29, 2015 at 6:33 PM, Andres Freund <andres@anarazel.de>
>wrote:
>> On 2015-05-29 18:02:36 -0400, Robert Haas wrote:
>>> Well, I think we ought to take at least a few weeks to try to do a
>bit
>>> of code review and clean up what we can from the open items list.
>>
>> Why? A large portion of the input required to go from beta towards a
>> release is from actual users. To see when things break, what confuses
>> them and such.
>
>I have two concerns:
>
>1. I'm concerned that once we release beta, any idea about reverting a
>feature or fixing something that is broken will get harder, because
>people will say "well, we can't do that after we've released a beta".
>I confess to particularly wanting a solution to the item listed as
>"custom-join has no way to construct Plan nodes of child Path nodes",
>the history of which I'll avoid recapitulating until I'm sure I can do
>it while maintaining my blood pressure at safe levels.
I think we should just document that this a beta and that changes are to be expected. And have a release candidate once that's not the case.
I agree that it'd be very good of the custom join issue gets fixed. But I don't see a beta prohibiting it. Independently from that in going to ask a Citus colleague to make sure that pg-shard can use this.
>2. Also, if we're going to make significant multixact-related changes
>to 9.5 to try to improve reliability, as you proposed on the other
>thread, then it would be nice to do that before beta, so that it gets
>tested. Of course, someone is bound to point out that we could make
>those changes in time for beta2, and people could test that. But in
>practice I think that'll just mean that stuff is only out there for
>let's say 2 months before we put it in a major release, which ain't
>much.
There seems to be enough other stuff in die need of testing that I don't think that's sufficient cause, even though I understand the sentiment.
Andres
---
Please excuse brevity and formatting - I am writing this on my mobile phone.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-30 04:25 Andres Freund <andres@anarazel.de>
parent: Tom Lane <tgl@sss.pgh.pa.us>
3 siblings, 0 replies; 78+ messages in thread
From: Andres Freund @ 2015-05-30 04:25 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; Robert Haas <robertmhaas@gmail.com>; +Cc: Stephen Frost <sfrost@snowman.net>; Bruce Momjian <bruce@momjian.us>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On May 29, 2015 9:08:07 PM PDT, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>I think your position is completely nuts.
Yeehaa.
> The GROUPING SETS code is
>desperately in need of testing. The custom-plan code is desperately
>in need of fixing and testing. The multixact code is desperately
>in need of testing.
And the array/plpgsql changes and upsert, and...
Andres
---
Please excuse brevity and formatting - I am writing this on my mobile phone.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-30 06:38 Noah Misch <noah@leadboat.com>
parent: Tom Lane <tgl@sss.pgh.pa.us>
3 siblings, 0 replies; 78+ messages in thread
From: Noah Misch @ 2015-05-30 06:38 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Stephen Frost <sfrost@snowman.net>; Bruce Momjian <bruce@momjian.us>; Magnus Hagander <magnus@hagander.net>; Robert Haas <robertmhaas@gmail.com>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On Fri, May 29, 2015 at 04:01:00PM -0400, Tom Lane wrote:
> Stephen Frost <sfrost@snowman.net> writes:
> > * Bruce Momjian (bruce@momjian.us) wrote:
> >> I am unclear if we are anywhere near ready for beta1 even in June. Are
> >> we?
>
> > I'm all about having that discussion... but can we do it on another
> > thread or at least wait til we've decided about the back-branch
> > releases? They are clearly the more important issue to consider.
>
> It's the same discussion though, ie what releases are we expecting to
> get out in the next couple of weeks.
+1 for Stephen's thought to decide about back-branch releases first and to
Magnus's sentiment upthread that beta has to stand back while we schedule
them. In other words, the feedback between these two scheduling decisions
ought to be one-way: bringing today's supported branches to a state we can be
content about deserves first pick from the calendar.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-30 12:56 Bruce Momjian <bruce@momjian.us>
parent: Tom Lane <tgl@sss.pgh.pa.us>
3 siblings, 2 replies; 78+ messages in thread
From: Bruce Momjian @ 2015-05-30 12:56 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Robert Haas <robertmhaas@gmail.com>; Andres Freund <andres@anarazel.de>; Stephen Frost <sfrost@snowman.net>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On Sat, May 30, 2015 at 12:08:07AM -0400, Tom Lane wrote:
> desperately in need of testing. The custom-plan code is desperately
> in need of fixing and testing. The multixact code is desperately
> in need of testing. The open-items list has several other problems
> besides those. All of those problems are independent. If we insist
> on tackling them serially rather than in parallel, 9.5 might not come
> out till 2017.
2017? Really? Is there any need for that hyperbole?
Frankly, based on how I feel now, I would have no problem doing 9.5 in
2016 and saying we have a lot of retooling to do. We could say we have
gotten too far out ahead of ourselves and we need to regroup and
restructure the code.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ Everyone has their own god. +
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-30 13:11 Bruce Momjian <bruce@momjian.us>
parent: Bruce Momjian <bruce@momjian.us>
1 sibling, 1 reply; 78+ messages in thread
From: Bruce Momjian @ 2015-05-30 13:11 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Robert Haas <robertmhaas@gmail.com>; Andres Freund <andres@anarazel.de>; Stephen Frost <sfrost@snowman.net>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On Sat, May 30, 2015 at 08:56:53AM -0400, Bruce Momjian wrote:
> On Sat, May 30, 2015 at 12:08:07AM -0400, Tom Lane wrote:
> > desperately in need of testing. The custom-plan code is desperately
> > in need of fixing and testing. The multixact code is desperately
> > in need of testing. The open-items list has several other problems
> > besides those. All of those problems are independent. If we insist
> > on tackling them serially rather than in parallel, 9.5 might not come
> > out till 2017.
>
> 2017? Really? Is there any need for that hyperbole?
>
> Frankly, based on how I feel now, I would have no problem doing 9.5 in
> 2016 and saying we have a lot of retooling to do. We could say we have
> gotten too far out ahead of ourselves and we need to regroup and
> restructure the code.
Actually, barrelling ahead to get releases out is how we got into this
mess in the first place. I would vote we put the 9.5 release on hold
while we do an honest assessment of where we are. In hindsight, we
should have known to do this even before 9.4 was released.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ Everyone has their own god. +
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-30 14:06 Robert Haas <robertmhaas@gmail.com>
parent: Tom Lane <tgl@sss.pgh.pa.us>
3 siblings, 1 reply; 78+ messages in thread
From: Robert Haas @ 2015-05-30 14:06 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Andres Freund <andres@anarazel.de>; Stephen Frost <sfrost@snowman.net>; Bruce Momjian <bruce@momjian.us>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On Sat, May 30, 2015 at 12:08 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> I think your position is completely nuts. The GROUPING SETS code is
> desperately in need of testing. The custom-plan code is desperately
> in need of fixing and testing. The multixact code is desperately
> in need of testing. The open-items list has several other problems
> besides those. All of those problems are independent. If we insist
> on tackling them serially rather than in parallel, 9.5 might not come
> out till 2017.
If that means it's stable, +1 from me.
I dispute, on every level, the notion that not releasing a beta means
that we can't work on things in parallel. We can work on all of the
things on the open items list in parallel right now. We can also
test. And in fact, we should test. It's entirely appropriate to test
our own stuff before we ask other people to test it. It's also
appropriate to fix the things that we already know are broken before
we ask other people to test it.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-30 14:11 Joshua D. Drake <jd@commandprompt.com>
parent: Bruce Momjian <bruce@momjian.us>
0 siblings, 0 replies; 78+ messages in thread
From: Joshua D. Drake @ 2015-05-30 14:11 UTC (permalink / raw)
To: Bruce Momjian <bruce@momjian.us>; Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Robert Haas <robertmhaas@gmail.com>; Andres Freund <andres@anarazel.de>; Stephen Frost <sfrost@snowman.net>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On 05/30/2015 06:11 AM, Bruce Momjian wrote:
>> 2017? Really? Is there any need for that hyperbole?
>>
>> Frankly, based on how I feel now, I would have no problem doing 9.5 in
>> 2016 and saying we have a lot of retooling to do. We could say we have
>> gotten too far out ahead of ourselves and we need to regroup and
>> restructure the code.
>
> Actually, barrelling ahead to get releases out is how we got into this
> mess in the first place. I would vote we put the 9.5 release on hold
> while we do an honest assessment of where we are. In hindsight, we
> should have known to do this even before 9.4 was released.
>
It seems that we all are forgetting one of the fundamental concepts of
open source development:
Q. When will release X be?
A. When it is done.
A delay because of quality concerns shows the integrity of the project.
Sincerely,
JD
--
Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-30 15:45 Bruce Momjian <bruce@momjian.us>
parent: Robert Haas <robertmhaas@gmail.com>
0 siblings, 2 replies; 78+ messages in thread
From: Bruce Momjian @ 2015-05-30 15:45 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Andres Freund <andres@anarazel.de>; Stephen Frost <sfrost@snowman.net>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On Sat, May 30, 2015 at 10:06:52AM -0400, Robert Haas wrote:
> If that means it's stable, +1 from me.
>
> I dispute, on every level, the notion that not releasing a beta means
> that we can't work on things in parallel. We can work on all of the
> things on the open items list in parallel right now. We can also
> test. And in fact, we should test. It's entirely appropriate to test
> our own stuff before we ask other people to test it. It's also
> appropriate to fix the things that we already know are broken before
> we ask other people to test it.
Let me share something that people have told me privately but don't want
to state publicly (at least with attribution), and that is that we have
seen great increases in feature development (often funded), without a
corresponding increase development efforts focused on stability. The
fact Alvaro has had to almost single-handedly fix multi-xact bug until
very recently is testament to that.
The bottom line is that we just can't keep going on like this. The fact
we put out a release two weeks ago, then need to put out a fix release
for that, but we have more multi-xact bugs to fix and can't decide if we
should do one or two minor releases, and are pushing out an alpha of 9.5
because we know we aren't ready for a beta, just confirms my analysis.
I hate to be the bearer of bad news, but I think bad news is what we
must face.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ Everyone has their own god. +
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-30 18:10 Robert Haas <robertmhaas@gmail.com>
parent: Bruce Momjian <bruce@momjian.us>
1 sibling, 3 replies; 78+ messages in thread
From: Robert Haas @ 2015-05-30 18:10 UTC (permalink / raw)
To: Bruce Momjian <bruce@momjian.us>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Andres Freund <andres@anarazel.de>; Stephen Frost <sfrost@snowman.net>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On Sat, May 30, 2015 at 11:45 AM, Bruce Momjian <bruce@momjian.us> wrote:
> On Sat, May 30, 2015 at 10:06:52AM -0400, Robert Haas wrote:
>> If that means it's stable, +1 from me.
>>
>> I dispute, on every level, the notion that not releasing a beta means
>> that we can't work on things in parallel. We can work on all of the
>> things on the open items list in parallel right now. We can also
>> test. And in fact, we should test. It's entirely appropriate to test
>> our own stuff before we ask other people to test it. It's also
>> appropriate to fix the things that we already know are broken before
>> we ask other people to test it.
>
> Let me share something that people have told me privately but don't want
> to state publicly (at least with attribution), and that is that we have
> seen great increases in feature development (often funded), without a
> corresponding increase development efforts focused on stability. The
> fact Alvaro has had to almost single-handedly fix multi-xact bug until
> very recently is testament to that.
It's clear - at least to me - that we need to put more resources into
stabilizing the new multixact system. This is killing us. If we can't
stabilize this, people will go use some other database.
Equally importantly, we need to make sure that we never release
something comparably broken ever again. And that's why I'm not
sanguine about shipping what we've got without adequate reflection.
What, in this release, could break things badly? RLS? Grouping sets?
Heikki's WAL format changes? That last one sounds really scary to me;
it's painful if not impossible to fix the WAL format in a minor
release.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-30 19:26 Peter Geoghegan <pg@heroku.com>
parent: Bruce Momjian <bruce@momjian.us>
1 sibling, 1 reply; 78+ messages in thread
From: Peter Geoghegan @ 2015-05-30 19:26 UTC (permalink / raw)
To: Bruce Momjian <bruce@momjian.us>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Robert Haas <robertmhaas@gmail.com>; Andres Freund <andres@anarazel.de>; Stephen Frost <sfrost@snowman.net>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On Sat, May 30, 2015 at 5:56 AM, Bruce Momjian <bruce@momjian.us> wrote:
> Frankly, based on how I feel now, I would have no problem doing 9.5 in
> 2016 and saying we have a lot of retooling to do. We could say we have
> gotten too far out ahead of ourselves and we need to regroup and
> restructure the code.
I wouldn't mind doing that, but I think it's premature to conclude
that it's necessary to wait quite that long to release.
--
Peter Geoghegan
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-30 19:46 Peter Geoghegan <pg@heroku.com>
parent: Robert Haas <robertmhaas@gmail.com>
2 siblings, 1 reply; 78+ messages in thread
From: Peter Geoghegan @ 2015-05-30 19:46 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: Bruce Momjian <bruce@momjian.us>; Tom Lane <tgl@sss.pgh.pa.us>; Andres Freund <andres@anarazel.de>; Stephen Frost <sfrost@snowman.net>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On Sat, May 30, 2015 at 11:10 AM, Robert Haas <robertmhaas@gmail.com> wrote:
>> Let me share something that people have told me privately but don't want
>> to state publicly (at least with attribution), and that is that we have
>> seen great increases in feature development (often funded), without a
>> corresponding increase development efforts focused on stability. The
>> fact Alvaro has had to almost single-handedly fix multi-xact bug until
>> very recently is testament to that.
>
> It's clear - at least to me - that we need to put more resources into
> stabilizing the new multixact system. This is killing us. If we can't
> stabilize this, people will go use some other database.
+1. I don't grok the MultiXact code as some people do, but even still,
I think problems have been ongoing for so long now that we must change
course. FWIW, my perception from afar is that the problems haven't
really tapered off, and we'd be better off taking a fresh approach.
> Equally importantly, we need to make sure that we never release
> something comparably broken ever again. And that's why I'm not
> sanguine about shipping what we've got without adequate reflection.
As you said, there was a failure to appreciate the interactions with
VACUUM. That should have made us more introspective about what we
didn't know and couldn't know during during 9.3 development, but it
didn't.
> What, in this release, could break things badly? RLS? Grouping sets?
> Heikki's WAL format changes? That last one sounds really scary to me;
> it's painful if not impossible to fix the WAL format in a minor
> release.
I think we actually have learned some lessons here. MultiXacts were a
somewhat unusual case for a couple of reasons that I need not rehash.
In contrast, Heikki's WAL format changes (just for example) are
fundamentally just a restructuring to the existing format. Sure, there
could be bugs, but I think that it's fundamentally different to the
9.3 MultiXact stuff, in that the MultiXact stuff appears to be
stubbornly difficult to stabilize over months and years. That feels
like something that is unlikely to be true for anything that made it
into 9.5.
--
Peter Geoghegan
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-30 20:47 Andres Freund <andres@anarazel.de>
parent: Bruce Momjian <bruce@momjian.us>
1 sibling, 2 replies; 78+ messages in thread
From: Andres Freund @ 2015-05-30 20:47 UTC (permalink / raw)
To: Bruce Momjian <bruce@momjian.us>; +Cc: Robert Haas <robertmhaas@gmail.com>; Tom Lane <tgl@sss.pgh.pa.us>; Stephen Frost <sfrost@snowman.net>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
Hi Bruce, Everyone,
On 2015-05-30 11:45:59 -0400, Bruce Momjian wrote:
> Let me share something that people have told me privately but don't want
> to state publicly (at least with attribution), and that is that we have
> seen great increases in feature development (often funded), without a
> corresponding increase development efforts focused on stability.
Yes, I have seen and heard that too. What I think is also important that
in turn our adoption has outpaced feature development (and thus
transitively stability work).
> The bottom line is that we just can't keep going on like this. The fact
> we put out a release two weeks ago, then need to put out a fix release
> for that, but we have more multi-xact bugs to fix and can't decide if we
> should do one or two minor releases, and are pushing out an alpha of 9.5
> because we know we aren't ready for a beta, just confirms my analysis.
I don't think that alone confirms very much.
> I hate to be the bearer of bad news, but I think bad news is what we
> must face.
Well, the question is what we do with that observation. Personally I
think it's not a new one. This point has been made repeatedly, including
at most if not all developer meetings I attended. I definitely had
conversations around it both in person, on IM and on list.
I don't think it's primarily a problem of lack of review; although that
is a large problem. I think the biggest systematic problem is that the
compound complexity of postgres has increased dramatically over the
years. Features have added complexity little by little, each not
incrementally not looking that bad. But very little has been done to
manage complexity. Since 8.0 the codesize has roughly doubled, but
little has been done to manage the increased complexity. Few new
abstractions have been introduced and the structure of the code is
largely the same.
As a somewhat extreme example, let's look at StartupXLOG(). In 8.0 it
was ~500 LOC, in master it's ~1500. The interactions in 8.0 were
complex, they have gotten much more complex since. It fullfills lots of
different roles, all in one function:
(roughly in the order things happen, but simplified)
* Read the control file/determine whether we crashed
* recovery.conf handling
* backup label handling
* tablespace map handling (huh, I missed that this was added directly to
StartupXLOG. What a bad idea)
* Determine whether we're doing archive recovery, read the relevant
checkpoint if so
* relcache init file removal
* timeline switch handling
* Loading the checkpoint we're starting from
* Initialization of a lot of subsystems
* crash recovery/replay
* Including pgstat, unlogged table, exported snapshot handling
* iff hot standby, some more subsystems are initialized here
* hot standby state handling
* replay process intialization
* crash replay itself, including
* progress tracking
* recovery pause handling
* nextxid tracking
* timeline increase handling
* hot standby state handling
* unlogged relations handling
* archive recovery handling
* creation/initialization of the end of recovery checkpoint
* timeline increment if failover
* subsystem initialization iff !hot_standby
* end of recovery actions
Yes. that's one routine. And, to make things even funnier, half of that
routine isn't exercised by our tests.
You can argue that this is an outlier, but I don't think so. Heapam, the
planner, etc. have similar cases.
And I think this, to some degree, explains a lot of the multixact
problems. While there were a few "simple bugs", most of them were
interactions between the various subsystems that are rather intricate.
So, I think we have built up a lot of technical debt. And very little
effort has been made to fix that; and in the cases where people have the
reception has often been cool, because refactoring things obviously will
destabilize in the short term, even if it fixes problems in the long
term. I don't think that's sustainable.
We can't improve the situation by just delaying the 9.5 release or
something like that. We need to actively work on making the codebase
easier to understand and better tested. But that is actual development
work, and shouldn't happen at the tail end of a release.
Regards,
Andres
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-30 21:02 Andres Freund <andres@anarazel.de>
parent: Robert Haas <robertmhaas@gmail.com>
2 siblings, 1 reply; 78+ messages in thread
From: Andres Freund @ 2015-05-30 21:02 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: Bruce Momjian <bruce@momjian.us>; Tom Lane <tgl@sss.pgh.pa.us>; Stephen Frost <sfrost@snowman.net>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On 2015-05-30 14:10:36 -0400, Robert Haas wrote:
> It's clear - at least to me - that we need to put more resources into
> stabilizing the new multixact system. This is killing us. If we can't
> stabilize this, people will go use some other database.
I agree. Perhaps I don't see things quite as direly, but then I didn't
just spend weeks on the issue. I remember that I was incredibly
frustrated around 9.3.2 because I'd spent weeks on fixing issued around
this and it just never seemed to stop.
> Equally importantly, we need to make sure that we never release
> something comparably broken ever again. And that's why I'm not
> sanguine about shipping what we've got without adequate reflection.
I think you're inferring something wrong here. A beta/alpha *is* getting
feedback on how good/bad things are. It's just one source of such
information, but we don't have that many others.
As explained in the email I sent before this, I think a lot of the
problems come from too complex code (with barely any testing). But we're
not going to be able to clean this up in 9.5. This will be a longer term
effort.
If we, without further changes, decide to let the release slip to, say,
Q1 2016, the only thing that'll happen is to happen that 9.6 will have
larger, more complex features. With barely any additional review and
testing done. There was very little, if any, additional testing/review
outside jsonb due to the 9.4 slippage.
I don't think the problems have much to do with the release schedule.
> What, in this release, could break things badly?
> RLS?
Mostly localized to users of the feature. Niche use case.
> Grouping sets?
Few changes to code unless grouping sets are used.
> Heikki's WAL format changes?
Yes, that's quite invasive. On the other hand, I can't think of another
feature that had as much invested in tooling to detect problem.
What's more:
* Upsert - it's probably the most complex feature in 9.5. It's quite
localized though.
* The locking changes, a good amount of potential for subtle problems
* The signal handling, sinval, client communication changes. Little to
none problems so far, but it's complex stuff. These changes are an
example of potential for problems due to changes to reduce
complexity...
Greetings,
Andres Freund
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-30 21:19 Tom Lane <tgl@sss.pgh.pa.us>
parent: Andres Freund <andres@anarazel.de>
0 siblings, 1 reply; 78+ messages in thread
From: Tom Lane @ 2015-05-30 21:19 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Robert Haas <robertmhaas@gmail.com>; Bruce Momjian <bruce@momjian.us>; Stephen Frost <sfrost@snowman.net>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
Andres Freund <andres@anarazel.de> writes:
> * The signal handling, sinval, client communication changes. Little to
> none problems so far, but it's complex stuff. These changes are an
> example of potential for problems due to changes to reduce
> complexity...
As far as that goes, it's quite clear from the buildfarm that the
atomics stuff is not very stable on non-mainstream architectures.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-30 21:23 Andres Freund <andres@anarazel.de>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 0 replies; 78+ messages in thread
From: Andres Freund @ 2015-05-30 21:23 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Robert Haas <robertmhaas@gmail.com>; Bruce Momjian <bruce@momjian.us>; Stephen Frost <sfrost@snowman.net>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On May 30, 2015 2:19:00 PM PDT, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>Andres Freund <andres@anarazel.de> writes:
>> * The signal handling, sinval, client communication changes. Little
>to
>> none problems so far, but it's complex stuff. These changes are an
>> example of potential for problems due to changes to reduce
>> complexity...
>
>As far as that goes, it's quite clear from the buildfarm that the
>atomics stuff is not very stable on non-mainstream architectures.
Is that the case? So far it seems to primarily be a problem of the, old, barrier emulation being buggy (non reentrant). And that being visible due to the new barrier in the latch code.
If not be surprised if there were more bugs, don't get me wrong, this is highly platform dependant stuff.
---
Please excuse brevity and formatting - I am writing this on my mobile phone.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-30 22:48 David Steele <david@pgmasters.net>
parent: Robert Haas <robertmhaas@gmail.com>
2 siblings, 1 reply; 78+ messages in thread
From: David Steele @ 2015-05-30 22:48 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; Bruce Momjian <bruce@momjian.us>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Andres Freund <andres@anarazel.de>; Stephen Frost <sfrost@snowman.net>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On 5/30/15 2:10 PM, Robert Haas wrote:
> What, in this release, could break things badly? RLS? Grouping sets?
> Heikki's WAL format changes? That last one sounds really scary to me;
> it's painful if not impossible to fix the WAL format in a minor
> release.
I would argue Heikki's WAL stuff is a perfect case for releasing a
public alpha/beta soon. I'd love to test PgBackRest with an "official"
9.5dev build. The PgBackRest test suite has lots of tests that run on
versions 8.3+ and might well shake out any bugs that are lying around.
In fact, I've added a new feature based on monitoring the thread and I'm
interested to see how that pans out.
--
- David Steele
david@pgmasters.net
Attachments:
signature.asc (application/pgp-signature, 818B) download
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-31 00:38 Joshua D. Drake <jd@commandprompt.com>
parent: David Steele <david@pgmasters.net>
0 siblings, 1 reply; 78+ messages in thread
From: Joshua D. Drake @ 2015-05-31 00:38 UTC (permalink / raw)
To: David Steele <david@pgmasters.net>; Robert Haas <robertmhaas@gmail.com>; Bruce Momjian <bruce@momjian.us>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Andres Freund <andres@anarazel.de>; Stephen Frost <sfrost@snowman.net>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On 05/30/2015 03:48 PM, David Steele wrote:
> On 5/30/15 2:10 PM, Robert Haas wrote:
>> What, in this release, could break things badly? RLS? Grouping sets?
>> Heikki's WAL format changes? That last one sounds really scary to me;
>> it's painful if not impossible to fix the WAL format in a minor
>> release.
>
> I would argue Heikki's WAL stuff is a perfect case for releasing a
> public alpha/beta soon. I'd love to test PgBackRest with an "official"
> 9.5dev build. The PgBackRest test suite has lots of tests that run on
> versions 8.3+ and might well shake out any bugs that are lying around.
You are right. Clone git, run it nightly automated and please, please
report anything you find. There is no reason for a tagged release for
that. Consider it a custom, purpose built, build-test farm.
Sincerely,
JD
--
Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-31 01:51 David Steele <david@pgmasters.net>
parent: Joshua D. Drake <jd@commandprompt.com>
0 siblings, 3 replies; 78+ messages in thread
From: David Steele @ 2015-05-31 01:51 UTC (permalink / raw)
To: Joshua D. Drake <jd@commandprompt.com>; Robert Haas <robertmhaas@gmail.com>; Bruce Momjian <bruce@momjian.us>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Andres Freund <andres@anarazel.de>; Stephen Frost <sfrost@snowman.net>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On 5/30/15 8:38 PM, Joshua D. Drake wrote:
>
> On 05/30/2015 03:48 PM, David Steele wrote:
>> On 5/30/15 2:10 PM, Robert Haas wrote:
>>> What, in this release, could break things badly? RLS? Grouping sets?
>>> Heikki's WAL format changes? That last one sounds really scary to me;
>>> it's painful if not impossible to fix the WAL format in a minor
>>> release.
>>
>> I would argue Heikki's WAL stuff is a perfect case for releasing a
>> public alpha/beta soon. I'd love to test PgBackRest with an "official"
>> 9.5dev build. The PgBackRest test suite has lots of tests that run on
>> versions 8.3+ and might well shake out any bugs that are lying around.
>
> You are right. Clone git, run it nightly automated and please, please
> report anything you find. There is no reason for a tagged release for
> that. Consider it a custom, purpose built, build-test farm.
Sure - I can write code to do that. But then why release a beta at all?
--
- David Steele
david@pgmasters.net
Attachments:
signature.asc (application/pgp-signature, 818B) download
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-31 02:24 Bruce Momjian <bruce@momjian.us>
parent: Peter Geoghegan <pg@heroku.com>
0 siblings, 1 reply; 78+ messages in thread
From: Bruce Momjian @ 2015-05-31 02:24 UTC (permalink / raw)
To: Peter Geoghegan <pg@heroku.com>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Robert Haas <robertmhaas@gmail.com>; Andres Freund <andres@anarazel.de>; Stephen Frost <sfrost@snowman.net>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On Sat, May 30, 2015 at 12:26:11PM -0700, Peter Geoghegan wrote:
> On Sat, May 30, 2015 at 5:56 AM, Bruce Momjian <bruce@momjian.us> wrote:
> > Frankly, based on how I feel now, I would have no problem doing 9.5 in
> > 2016 and saying we have a lot of retooling to do. We could say we have
> > gotten too far out ahead of ourselves and we need to regroup and
> > restructure the code.
>
> I wouldn't mind doing that, but I think it's premature to conclude
> that it's necessary to wait quite that long to release.
I agree it probably wouldn't take until 2016, but if does take until
2016, we have to be fine with that. What I am saying is we can't just
continue to focus on hitting target dates and assume everything will be
fine, because it isn't.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ Everyone has their own god. +
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-31 02:38 Joshua D. Drake <jd@commandprompt.com>
parent: David Steele <david@pgmasters.net>
2 siblings, 0 replies; 78+ messages in thread
From: Joshua D. Drake @ 2015-05-31 02:38 UTC (permalink / raw)
To: David Steele <david@pgmasters.net>; Robert Haas <robertmhaas@gmail.com>; Bruce Momjian <bruce@momjian.us>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Andres Freund <andres@anarazel.de>; Stephen Frost <sfrost@snowman.net>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On 05/30/2015 06:51 PM, David Steele wrote:
> On 5/30/15 8:38 PM, Joshua D. Drake wrote:
>>
>> On 05/30/2015 03:48 PM, David Steele wrote:
>>> On 5/30/15 2:10 PM, Robert Haas wrote:
>>>> What, in this release, could break things badly? RLS? Grouping sets?
>>>> Heikki's WAL format changes? That last one sounds really scary to me;
>>>> it's painful if not impossible to fix the WAL format in a minor
>>>> release.
>>>
>>> I would argue Heikki's WAL stuff is a perfect case for releasing a
>>> public alpha/beta soon. I'd love to test PgBackRest with an "official"
>>> 9.5dev build. The PgBackRest test suite has lots of tests that run on
>>> versions 8.3+ and might well shake out any bugs that are lying around.
>>
>> You are right. Clone git, run it nightly automated and please, please
>> report anything you find. There is no reason for a tagged release for
>> that. Consider it a custom, purpose built, build-test farm.
>
> Sure - I can write code to do that. But then why release a beta at all?
1. Continuous testing (especially automated) is a great thing (see
Buildfarm)
2. The rules for patches change a bit when we move to Beta
3. We may be able to fix a problem now (or soon) that you might catch
before Beta.
Sincerely,
J
--
Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: postpone next week's release
@ 2015-05-31 02:40 David G. Johnston <david.g.johnston@gmail.com>
parent: Bruce Momjian <bruce@momjian.us>
0 siblings, 0 replies; 78+ messages in thread
From: David G. Johnston @ 2015-05-31 02:40 UTC (permalink / raw)
To: Bruce Momjian <bruce@momjian.us>; +Cc: Peter Geoghegan <pg@heroku.com>; Tom Lane <tgl@sss.pgh.pa.us>; Robert Haas <robertmhaas@gmail.com>; Andres Freund <andres@anarazel.de>; Stephen Frost <sfrost@snowman.net>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On Saturday, May 30, 2015, Bruce Momjian <bruce@momjian.us> wrote:
> On Sat, May 30, 2015 at 12:26:11PM -0700, Peter Geoghegan wrote:
> > On Sat, May 30, 2015 at 5:56 AM, Bruce Momjian <bruce@momjian.us
> <javascript:;>> wrote:
> > > Frankly, based on how I feel now, I would have no problem doing 9.5 in
> > > 2016 and saying we have a lot of retooling to do. We could say we have
> > > gotten too far out ahead of ourselves and we need to regroup and
> > > restructure the code.
> >
> > I wouldn't mind doing that, but I think it's premature to conclude
> > that it's necessary to wait quite that long to release.
>
> I agree it probably wouldn't take until 2016, but if does take until
> 2016, we have to be fine with that. What I am saying is we can't just
> continue to focus on hitting target dates and assume everything will be
> fine, because it isn't.
>
>
On a slightly tangential note: I'm not prepared to defend doing so but it
seems worth at least considering whether we should continue supporting 9.0
beyond this October.
I don't think it should be be de-supported until at least a couple of 9.5
point releases have been found to be stable.
David J.
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-31 02:48 Bruce Momjian <bruce@momjian.us>
parent: Andres Freund <andres@anarazel.de>
1 sibling, 1 reply; 78+ messages in thread
From: Bruce Momjian @ 2015-05-31 02:48 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Robert Haas <robertmhaas@gmail.com>; Tom Lane <tgl@sss.pgh.pa.us>; Stephen Frost <sfrost@snowman.net>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On Sat, May 30, 2015 at 10:47:27PM +0200, Andres Freund wrote:
> > The bottom line is that we just can't keep going on like this. The fact
> > we put out a release two weeks ago, then need to put out a fix release
> > for that, but we have more multi-xact bugs to fix and can't decide if we
> > should do one or two minor releases, and are pushing out an alpha of 9.5
> > because we know we aren't ready for a beta, just confirms my analysis.
>
> I don't think that alone confirms very much.
Huh? In what world is that release timeline ever reasonable? It points
to a serious problem.
> > I hate to be the bearer of bad news, but I think bad news is what we
> > must face.
>
> Well, the question is what we do with that observation. Personally I
> think it's not a new one. This point has been made repeatedly, including
> at most if not all developer meetings I attended. I definitely had
> conversations around it both in person, on IM and on list.
Well, I think we stop what we are doing, focus on restructuring,
testing, and reviewing areas that historically have had problems, and
when we are done, we can look to go to 9.5 beta. What we don't want to
do is to push out more code and get back into a
wack-a-bug-as-they-are-found mode, which obviously did not serve us well
for multi-xact, and which is what releasing a beta will do, and of
course, more commit-fests, and more features.
If we have to totally stop feature development until we are all happy
with the code we have, so be it. If people feel they have to get into
cleanup mode or they will never get to add a feature to Postgres again,
so be it. If people say, heh, I am not going to do anything and just
come back when cleanup is done (by someone else), then we will end up
with a smaller but more dedicated development team, and I am fine with
that too. I am suggesting that until everyone is happy with the code we
have, we should not move forward. Forget 9.5 feature testing --- we
don't even have 9.3 and 9.4 working to my satisfaction yet, and I bet
others share my opinion. We do not want to look back on this period and
say _this_ is when Postgres lost its reputation for reliability, and
when other databases took that reputation from us.
> I don't think it's primarily a problem of lack of review; although that
> is a large problem. I think the biggest systematic problem is that the
> compound complexity of postgres has increased dramatically over the
> years. Features have added complexity little by little, each not
> incrementally not looking that bad. But very little has been done to
> manage complexity. Since 8.0 the codesize has roughly doubled, but
> little has been done to manage the increased complexity. Few new
> abstractions have been introduced and the structure of the code is
> largely the same.
>
> As a somewhat extreme example, let's look at StartupXLOG(). In 8.0 it
> was ~500 LOC, in master it's ~1500. The interactions in 8.0 were
> complex, they have gotten much more complex since. It fullfills lots of
> different roles, all in one function:
Yep, great please to start our work.
> So, I think we have built up a lot of technical debt. And very little
> effort has been made to fix that; and in the cases where people have the
> reception has often been cool, because refactoring things obviously will
> destabilize in the short term, even if it fixes problems in the long
> term. I don't think that's sustainable.
Agreed.
> We can't improve the situation by just delaying the 9.5 release or
> something like that. We need to actively work on making the codebase
> easier to understand and better tested. But that is actual development
> work, and shouldn't happen at the tail end of a release.
It should start right now, and then, once we are happy with our code, we
can take periodic breaks to revisit the exact issues you describe. What
I am saying is that we shouldn't wait until after 9.5 beta or after 9.5
final, or after the next commitfest or whatever. We have already waited
too long to do this.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ Everyone has their own god. +
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-31 11:15 Michael Paquier <michael.paquier@gmail.com>
parent: Bruce Momjian <bruce@momjian.us>
0 siblings, 1 reply; 78+ messages in thread
From: Michael Paquier @ 2015-05-31 11:15 UTC (permalink / raw)
To: Bruce Momjian <bruce@momjian.us>; +Cc: Andres Freund <andres@anarazel.de>; Robert Haas <robertmhaas@gmail.com>; Tom Lane <tgl@sss.pgh.pa.us>; Stephen Frost <sfrost@snowman.net>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On Sun, May 31, 2015 at 11:48 AM, Bruce Momjian wrote:
> On Sat, May 30, 2015 at 10:47:27PM +0200, Andres Freund wrote:
>> So, I think we have built up a lot of technical debt. And very little
>> effort has been made to fix that; and in the cases where people have the
>> reception has often been cool, because refactoring things obviously will
>> destabilize in the short term, even if it fixes problems in the long
>> term. I don't think that's sustainable.
>
> Agreed.
+1. Complexity has increased, and we are actually never at 100% sure
that a given bug fix does not have side effects on other things, hence
I think that a portion of this technical debt is the lack of
regression test coverage, for both existing features and platforms
(like Windows). The thing is that complexity has increased, but for
example for many features we lack test coverage, thinking mainly
replication-related stuff here. Of course we will never get to a level
of 100% of confidence with just the test coverage and the buildfarm,
but we should at least try to get closer to such a goal.
Those are things I am really willing to work on in the very short term
for what it's worth (of course not only that as
reviewing/refactoring/testing existing things is as well damn
important). Now improving the test coverage requires new
infrastructure, so those are new features, and that's perhaps not
dedicated to 9.5, except if we consider that this is part of this
technical debt accumulated among the years. Honestly I think it is.
>> We can't improve the situation by just delaying the 9.5 release or
>> something like that. We need to actively work on making the codebase
>> easier to understand and better tested. But that is actual development
>> work, and shouldn't happen at the tail end of a release.
>
> It should start right now, and then, once we are happy with our code, we
> can take periodic breaks to revisit the exact issues you describe. What
> I am saying is that we shouldn't wait until after 9.5 beta or after 9.5
> final, or after the next commitfest or whatever. We have already waited
> too long to do this.
Definitely.
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-31 11:24 Robert Haas <robertmhaas@gmail.com>
parent: Peter Geoghegan <pg@heroku.com>
0 siblings, 0 replies; 78+ messages in thread
From: Robert Haas @ 2015-05-31 11:24 UTC (permalink / raw)
To: Peter Geoghegan <pg@heroku.com>; +Cc: Bruce Momjian <bruce@momjian.us>; Tom Lane <tgl@sss.pgh.pa.us>; Andres Freund <andres@anarazel.de>; Stephen Frost <sfrost@snowman.net>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On Sat, May 30, 2015 at 3:46 PM, Peter Geoghegan <pg@heroku.com> wrote:
>> What, in this release, could break things badly? RLS? Grouping sets?
>> Heikki's WAL format changes? That last one sounds really scary to me;
>> it's painful if not impossible to fix the WAL format in a minor
>> release.
>
> I think we actually have learned some lessons here. MultiXacts were a
> somewhat unusual case for a couple of reasons that I need not rehash.
>
> In contrast, Heikki's WAL format changes (just for example) are
> fundamentally just a restructuring to the existing format. Sure, there
> could be bugs, but I think that it's fundamentally different to the
> 9.3 MultiXact stuff, in that the MultiXact stuff appears to be
> stubbornly difficult to stabilize over months and years. That feels
> like something that is unlikely to be true for anything that made it
> into 9.5.
I hope you're right. But I don't think any of us foresaw just how bad
the MultiXact thing was likely to be either.
In fact, I think to some extent we may STILL be in denial about how bad it is.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-31 13:50 Bruce Momjian <bruce@momjian.us>
parent: Michael Paquier <michael.paquier@gmail.com>
0 siblings, 1 reply; 78+ messages in thread
From: Bruce Momjian @ 2015-05-31 13:50 UTC (permalink / raw)
To: Michael Paquier <michael.paquier@gmail.com>; +Cc: Andres Freund <andres@anarazel.de>; Robert Haas <robertmhaas@gmail.com>; Tom Lane <tgl@sss.pgh.pa.us>; Stephen Frost <sfrost@snowman.net>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On Sun, May 31, 2015 at 08:15:38PM +0900, Michael Paquier wrote:
> On Sun, May 31, 2015 at 11:48 AM, Bruce Momjian wrote:
> > On Sat, May 30, 2015 at 10:47:27PM +0200, Andres Freund wrote:
> >> So, I think we have built up a lot of technical debt. And very little
> >> effort has been made to fix that; and in the cases where people have the
> >> reception has often been cool, because refactoring things obviously will
> >> destabilize in the short term, even if it fixes problems in the long
> >> term. I don't think that's sustainable.
> >
> > Agreed.
>
> +1. Complexity has increased, and we are actually never at 100% sure
> that a given bug fix does not have side effects on other things, hence
> I think that a portion of this technical debt is the lack of
> regression test coverage, for both existing features and platforms
> (like Windows). The thing is that complexity has increased, but for
> example for many features we lack test coverage, thinking mainly
> replication-related stuff here. Of course we will never get to a level
> of 100% of confidence with just the test coverage and the buildfarm,
> but we should at least try to get closer to such a goal.
FYI, I realize that one additional thing that has discouraged code
reorganization is the additional backpatch overhead. I think we now
need to accept that our reorganization-adverse approach might have cost
us some reliability, and that reorganization is going to add work to
backpatching.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ Everyone has their own god. +
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-31 14:03 Bruce Momjian <bruce@momjian.us>
parent: Bruce Momjian <bruce@momjian.us>
0 siblings, 2 replies; 78+ messages in thread
From: Bruce Momjian @ 2015-05-31 14:03 UTC (permalink / raw)
To: Michael Paquier <michael.paquier@gmail.com>; +Cc: Andres Freund <andres@anarazel.de>; Robert Haas <robertmhaas@gmail.com>; Tom Lane <tgl@sss.pgh.pa.us>; Stephen Frost <sfrost@snowman.net>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On Sun, May 31, 2015 at 09:50:25AM -0400, Bruce Momjian wrote:
> > +1. Complexity has increased, and we are actually never at 100% sure
> > that a given bug fix does not have side effects on other things, hence
> > I think that a portion of this technical debt is the lack of
> > regression test coverage, for both existing features and platforms
> > (like Windows). The thing is that complexity has increased, but for
> > example for many features we lack test coverage, thinking mainly
> > replication-related stuff here. Of course we will never get to a level
> > of 100% of confidence with just the test coverage and the buildfarm,
> > but we should at least try to get closer to such a goal.
>
> FYI, I realize that one additional thing that has discouraged code
> reorganization is the additional backpatch overhead. I think we now
> need to accept that our reorganization-adverse approach might have cost
> us some reliability, and that reorganization is going to add work to
> backpatching.
Actually, code reorganization in HEAD might cause backpatching to be
more buggy, reducing reliability --- obviously we need to have a
discussion about that.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ Everyone has their own god. +
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-31 15:49 Noah Misch <noah@leadboat.com>
parent: David Steele <david@pgmasters.net>
2 siblings, 1 reply; 78+ messages in thread
From: Noah Misch @ 2015-05-31 15:49 UTC (permalink / raw)
To: David Steele <david@pgmasters.net>; +Cc: Joshua D. Drake <jd@commandprompt.com>; Robert Haas <robertmhaas@gmail.com>; Bruce Momjian <bruce@momjian.us>; Tom Lane <tgl@sss.pgh.pa.us>; Andres Freund <andres@anarazel.de>; Stephen Frost <sfrost@snowman.net>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On Sat, May 30, 2015 at 09:51:04PM -0400, David Steele wrote:
> On 5/30/15 8:38 PM, Joshua D. Drake wrote:
> > On 05/30/2015 03:48 PM, David Steele wrote:
> >> I would argue Heikki's WAL stuff is a perfect case for releasing a
> >> public alpha/beta soon. I'd love to test PgBackRest with an "official"
> >> 9.5dev build. The PgBackRest test suite has lots of tests that run on
> >> versions 8.3+ and might well shake out any bugs that are lying around.
> >
> > You are right. Clone git, run it nightly automated and please, please
> > report anything you find. There is no reason for a tagged release for
> > that. Consider it a custom, purpose built, build-test farm.
>
> Sure - I can write code to do that. But then why release a beta at all?
It's largely for the benefit of folks planning manual, or otherwise high-cost,
testing. If you budget for just one big test per year, make it a test of
beta1. For inexpensive testing, you may as well ignore beta and test git
master daily or weekly.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-31 15:55 Tom Lane <tgl@sss.pgh.pa.us>
parent: Bruce Momjian <bruce@momjian.us>
1 sibling, 2 replies; 78+ messages in thread
From: Tom Lane @ 2015-05-31 15:55 UTC (permalink / raw)
To: Bruce Momjian <bruce@momjian.us>; +Cc: Michael Paquier <michael.paquier@gmail.com>; Andres Freund <andres@anarazel.de>; Robert Haas <robertmhaas@gmail.com>; Stephen Frost <sfrost@snowman.net>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
Bruce Momjian <bruce@momjian.us> writes:
>> FYI, I realize that one additional thing that has discouraged code
>> reorganization is the additional backpatch overhead. I think we now
>> need to accept that our reorganization-adverse approach might have cost
>> us some reliability, and that reorganization is going to add work to
>> backpatching.
> Actually, code reorganization in HEAD might cause backpatching to be
> more buggy, reducing reliability --- obviously we need to have a
> discussion about that.
Commit 6b700301c36e380eb4972ab72c0e914cae60f9fd is a recent real example.
Not that that should dissuade us from ever doing any reorganizations,
but it's foolish to discount back-patching costs.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-31 16:09 David Steele <david@pgmasters.net>
parent: Noah Misch <noah@leadboat.com>
0 siblings, 1 reply; 78+ messages in thread
From: David Steele @ 2015-05-31 16:09 UTC (permalink / raw)
To: Noah Misch <noah@leadboat.com>; +Cc: Joshua D. Drake <jd@commandprompt.com>; Robert Haas <robertmhaas@gmail.com>; Bruce Momjian <bruce@momjian.us>; Tom Lane <tgl@sss.pgh.pa.us>; Andres Freund <andres@anarazel.de>; Stephen Frost <sfrost@snowman.net>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On 5/31/15 11:49 AM, Noah Misch wrote:
> On Sat, May 30, 2015 at 09:51:04PM -0400, David Steele wrote:
>> On 5/30/15 8:38 PM, Joshua D. Drake wrote:
>>> On 05/30/2015 03:48 PM, David Steele wrote:
>>>> I would argue Heikki's WAL stuff is a perfect case for releasing a
>>>> public alpha/beta soon. I'd love to test PgBackRest with an "official"
>>>> 9.5dev build. The PgBackRest test suite has lots of tests that run on
>>>> versions 8.3+ and might well shake out any bugs that are lying around.
>>>
>>> You are right. Clone git, run it nightly automated and please, please
>>> report anything you find. There is no reason for a tagged release for
>>> that. Consider it a custom, purpose built, build-test farm.
>>
>> Sure - I can write code to do that. But then why release a beta at all?
>
> It's largely for the benefit of folks planning manual, or otherwise high-cost,
> testing. If you budget for just one big test per year, make it a test of
> beta1. For inexpensive testing, you may as well ignore beta and test git
> master daily or weekly.
I've gotten to the point of (relatively) high-cost coding/testing. The
removal of checkpoint_segments and pause_on_recovery are leading to
refactoring of not only the regressions tests but the actual backup
code. 9.5 and 8.3 are the only versions that require exceptions in the
code base.
I've already done basic testing against 9.5 by disabling certain tests.
Now I'm at the point where I need to start modifying code to take new
9.5 features/changes into account and make sure the regression tests
work for 8.3-9.5 with the fewest number of exceptions possible.
From the perspective of backup/restore testing, 9.5 has the most changes
since 9.0. I'd like to know that the API at least is stable before
investing the time in new development.
Perhaps I'm just misunderstanding the nature of the discussion.
--
- David Steele
david@pgmasters.net
Attachments:
signature.asc (application/pgp-signature, 818B) download
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-31 16:24 Bruce Momjian <bruce@momjian.us>
parent: Tom Lane <tgl@sss.pgh.pa.us>
1 sibling, 0 replies; 78+ messages in thread
From: Bruce Momjian @ 2015-05-31 16:24 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Michael Paquier <michael.paquier@gmail.com>; Andres Freund <andres@anarazel.de>; Robert Haas <robertmhaas@gmail.com>; Stephen Frost <sfrost@snowman.net>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On Sun, May 31, 2015 at 11:55:44AM -0400, Tom Lane wrote:
> Bruce Momjian <bruce@momjian.us> writes:
> >> FYI, I realize that one additional thing that has discouraged code
> >> reorganization is the additional backpatch overhead. I think we now
> >> need to accept that our reorganization-adverse approach might have cost
> >> us some reliability, and that reorganization is going to add work to
> >> backpatching.
>
> > Actually, code reorganization in HEAD might cause backpatching to be
> > more buggy, reducing reliability --- obviously we need to have a
> > discussion about that.
>
> Commit 6b700301c36e380eb4972ab72c0e914cae60f9fd is a recent real example.
> Not that that should dissuade us from ever doing any reorganizations,
> but it's foolish to discount back-patching costs.
Yep.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ Everyone has their own god. +
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-05-31 20:16 Andres Freund <andres@anarazel.de>
parent: Tom Lane <tgl@sss.pgh.pa.us>
1 sibling, 0 replies; 78+ messages in thread
From: Andres Freund @ 2015-05-31 20:16 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Bruce Momjian <bruce@momjian.us>; Michael Paquier <michael.paquier@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Stephen Frost <sfrost@snowman.net>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On 2015-05-31 11:55:44 -0400, Tom Lane wrote:
> Bruce Momjian <bruce@momjian.us> writes:
> >> FYI, I realize that one additional thing that has discouraged code
> >> reorganization is the additional backpatch overhead. I think we now
> >> need to accept that our reorganization-adverse approach might have cost
> >> us some reliability, and that reorganization is going to add work to
> >> backpatching.
>
> > Actually, code reorganization in HEAD might cause backpatching to be
> > more buggy, reducing reliability --- obviously we need to have a
> > discussion about that.
>
> Commit 6b700301c36e380eb4972ab72c0e914cae60f9fd is a recent real example.
> Not that that should dissuade us from ever doing any reorganizations,
> but it's foolish to discount back-patching costs.
On the other hand, that code is a complete maintenance nightmare. If
there weren't literally dozens of places that needed to be touched to
add a single parameter, it'd be far less likely for such a mistake to be
made. Right now significant portions of the file differ between the
branches, despite primarily minor feature additions...
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-06-01 01:17 Michael Paquier <michael.paquier@gmail.com>
parent: Bruce Momjian <bruce@momjian.us>
1 sibling, 0 replies; 78+ messages in thread
From: Michael Paquier @ 2015-06-01 01:17 UTC (permalink / raw)
To: Bruce Momjian <bruce@momjian.us>; +Cc: Andres Freund <andres@anarazel.de>; Robert Haas <robertmhaas@gmail.com>; Tom Lane <tgl@sss.pgh.pa.us>; Stephen Frost <sfrost@snowman.net>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On Sun, May 31, 2015 at 11:03 PM, Bruce Momjian <bruce@momjian.us> wrote:
> On Sun, May 31, 2015 at 09:50:25AM -0400, Bruce Momjian wrote:
>> > +1. Complexity has increased, and we are actually never at 100% sure
>> > that a given bug fix does not have side effects on other things, hence
>> > I think that a portion of this technical debt is the lack of
>> > regression test coverage, for both existing features and platforms
>> > (like Windows). The thing is that complexity has increased, but for
>> > example for many features we lack test coverage, thinking mainly
>> > replication-related stuff here. Of course we will never get to a level
>> > of 100% of confidence with just the test coverage and the buildfarm,
>> > but we should at least try to get closer to such a goal.
>>
>> FYI, I realize that one additional thing that has discouraged code
>> reorganization is the additional backpatch overhead. I think we now
>> need to accept that our reorganization-adverse approach might have cost
>> us some reliability, and that reorganization is going to add work to
>> backpatching.
>
> Actually, code reorganization in HEAD might cause backpatching to be
> more buggy, reducing reliability --- obviously we need to have a
> discussion about that.
As a result, IMO all the folks gathering to PGCon (won't be there
sorry, but I read the MLs) should have a talk about that and define a
clear list of items to tackle in terms of reorganization for 9.5, and
then update this page:
https://wiki.postgresql.org/wiki/PostgreSQL_9.5_Open_Items
This does not prevent to move on with all the current items and
continue reviewing existing features that have been pushed of course.
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-06-01 13:38 Tom Lane <tgl@sss.pgh.pa.us>
parent: Magnus Hagander <magnus@hagander.net>
1 sibling, 0 replies; 78+ messages in thread
From: Tom Lane @ 2015-06-01 13:38 UTC (permalink / raw)
To: Magnus Hagander <magnus@hagander.net>; +Cc: Robert Haas <robertmhaas@gmail.com>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
Magnus Hagander <magnus@hagander.net> writes:
> On Fri, May 29, 2015 at 8:02 PM, Robert Haas <robertmhaas@gmail.com> wrote:
>> I think we should postpone next week's release.
> I'm a bit split on this.
> We *definitely* don't want to release the multixact fix without it being
> carefully reviewed, that's the part I'm not split about :) And I fully
> appreciate we can't have that done by monday.
> However, the file-permission thing seems to hit quite a few people (have we
> ever had this many bug reports after a minor release), which means wed
> really want to get that out quickly.
After dithering over the weekend, the majority view on -core seems to be
that we should go ahead with making a release today for the fsync issue.
We'll plan another release next week, or whenever the dust seems to have
settled on the multixact issue(s).
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-06-01 16:12 Jim Nasby <Jim.Nasby@BlueTreble.com>
parent: Bruce Momjian <bruce@momjian.us>
0 siblings, 1 reply; 78+ messages in thread
From: Jim Nasby @ 2015-06-01 16:12 UTC (permalink / raw)
To: Bruce Momjian <bruce@momjian.us>; Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Andres Freund <andres@anarazel.de>; Robert Haas <robertmhaas@gmail.com>; Stephen Frost <sfrost@snowman.net>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On 5/29/15 5:28 PM, Bruce Momjian wrote:
>> could expect that anyone committing a user-visible semantics change should
>> >update the release notes themselves.
> Yes, that would be nice.
FWIW, I've always wondered why we don't create an empty next-version
release notes as part of stamping a major release and expect patch
authors to add to it. I realize that likely creates merge conflicts, but
that seems less work than doing it all at the end. (Or maybe each patch
just creates a file and the final process is pulling all the files
together.)
--
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Data in Trouble? Get it in Treble! http://BlueTreble.com
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-06-01 16:32 Tom Lane <tgl@sss.pgh.pa.us>
parent: Jim Nasby <Jim.Nasby@BlueTreble.com>
0 siblings, 1 reply; 78+ messages in thread
From: Tom Lane @ 2015-06-01 16:32 UTC (permalink / raw)
To: Jim Nasby <Jim.Nasby@bluetreble.com>; +Cc: Bruce Momjian <bruce@momjian.us>; Andres Freund <andres@anarazel.de>; Robert Haas <robertmhaas@gmail.com>; Stephen Frost <sfrost@snowman.net>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
Jim Nasby <Jim.Nasby@bluetreble.com> writes:
> FWIW, I've always wondered why we don't create an empty next-version
> release notes as part of stamping a major release and expect patch
> authors to add to it. I realize that likely creates merge conflicts, but
> that seems less work than doing it all at the end. (Or maybe each patch
> just creates a file and the final process is pulling all the files
> together.)
There are good reasons to write the release notes all in one batch:
otherwise you don't get any uniformity of editorial style.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-06-01 16:34 Andres Freund <andres@anarazel.de>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 78+ messages in thread
From: Andres Freund @ 2015-06-01 16:34 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Jim Nasby <Jim.Nasby@bluetreble.com>; Bruce Momjian <bruce@momjian.us>; Robert Haas <robertmhaas@gmail.com>; Stephen Frost <sfrost@snowman.net>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On 2015-06-01 12:32:21 -0400, Tom Lane wrote:
> There are good reasons to write the release notes all in one batch:
> otherwise you don't get any uniformity of editorial style.
I agree that that's a good reason for major releases, I do however
wonder if it'd not be a good idea to do differently for backpatched
bugfixes. It's imo a good thing to force committers to write a release
notice at the same time they're backpatching. The memory is fresh, and
the commit message is more likely to contain pertinent details.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-06-01 16:56 Tom Lane <tgl@sss.pgh.pa.us>
parent: Andres Freund <andres@anarazel.de>
0 siblings, 1 reply; 78+ messages in thread
From: Tom Lane @ 2015-06-01 16:56 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Jim Nasby <Jim.Nasby@bluetreble.com>; Bruce Momjian <bruce@momjian.us>; Robert Haas <robertmhaas@gmail.com>; Stephen Frost <sfrost@snowman.net>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
Andres Freund <andres@anarazel.de> writes:
> On 2015-06-01 12:32:21 -0400, Tom Lane wrote:
>> There are good reasons to write the release notes all in one batch:
>> otherwise you don't get any uniformity of editorial style.
> I agree that that's a good reason for major releases, I do however
> wonder if it'd not be a good idea to do differently for backpatched
> bugfixes. It's imo a good thing to force committers to write a release
> notice at the same time they're backpatching. The memory is fresh, and
> the commit message is more likely to contain pertinent details.
We do expect committers to write commit log messages that contain
appropriate raw material for the release notes. That's not the same
as expecting them to prepare an actual, sgml-marked-up, release note
entry that's in good English and occupies a reasonable amount of space
relative to other items.
Jim's point about merge problems is very pertinent as well. In the
first place, if we had running release notes like that, they'd often
differ from one branch to the next, making back-patching rather annoying.
In the second place, SGML is so bulky that the patch context you'd be
working with would frequently look like not much more than
</para>
</listitem>
<listitem>
<para>
making it very easy for the hunks to be misapplied.
Lastly, we have recently adopted a practice of labeling release note
entries with the associated commit hashes. I dunno how much value that
really has, but it would be entirely impossible to write such labels
in advance of pushing the fixes.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-06-01 18:09 Josh Berkus <josh@agliodbs.com>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 0 replies; 78+ messages in thread
From: Josh Berkus @ 2015-06-01 18:09 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; Andres Freund <andres@anarazel.de>; +Cc: Jim Nasby <Jim.Nasby@bluetreble.com>; Bruce Momjian <bruce@momjian.us>; Robert Haas <robertmhaas@gmail.com>; Stephen Frost <sfrost@snowman.net>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
All,
Just my $0.02 on PR: it has never been a PR problem to do multiple
update releases, as long as we could provide a good reason for doing so
(like: fix A is available now and we didn't want to hold it back waiting
for fix B).
It's always a practical question of (a) packaging and (b) deployment.
That is, we can get packager fatigue where some updates don't get
packaged, and we can get user fatigue where they start ignoring updates.
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-06-02 05:44 Noah Misch <noah@leadboat.com>
parent: David Steele <david@pgmasters.net>
0 siblings, 0 replies; 78+ messages in thread
From: Noah Misch @ 2015-06-02 05:44 UTC (permalink / raw)
To: David Steele <david@pgmasters.net>; +Cc: Joshua D. Drake <jd@commandprompt.com>; Robert Haas <robertmhaas@gmail.com>; Bruce Momjian <bruce@momjian.us>; Tom Lane <tgl@sss.pgh.pa.us>; Andres Freund <andres@anarazel.de>; Stephen Frost <sfrost@snowman.net>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On Sun, May 31, 2015 at 12:09:16PM -0400, David Steele wrote:
> On 5/31/15 11:49 AM, Noah Misch wrote:
> > On Sat, May 30, 2015 at 09:51:04PM -0400, David Steele wrote:
> >> Sure - I can write code to do that. But then why release a beta at all?
> >
> > It's largely for the benefit of folks planning manual, or otherwise high-cost,
> > testing. If you budget for just one big test per year, make it a test of
> > beta1. For inexpensive testing, you may as well ignore beta and test git
> > master daily or weekly.
>
> I've gotten to the point of (relatively) high-cost coding/testing. The
> removal of checkpoint_segments and pause_on_recovery are leading to
> refactoring of not only the regressions tests but the actual backup
> code. 9.5 and 8.3 are the only versions that require exceptions in the
> code base.
>
> I've already done basic testing against 9.5 by disabling certain tests.
> Now I'm at the point where I need to start modifying code to take new
> 9.5 features/changes into account and make sure the regression tests
> work for 8.3-9.5 with the fewest number of exceptions possible.
Release of beta1 is the cue for that sort of work.
> From the perspective of backup/restore testing, 9.5 has the most changes
> since 9.0. I'd like to know that the API at least is stable before
> investing the time in new development.
Its API will be as good as pgsql-hackers could make it; beta1 is also a call
for help discovering API problems we overlooked. Subsequent API changes are
usually reactions to beta test reports.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-06-03 17:56 Stefan Kaltenbrunner <stefan@kaltenbrunner.cc>
parent: David Steele <david@pgmasters.net>
2 siblings, 0 replies; 78+ messages in thread
From: Stefan Kaltenbrunner @ 2015-06-03 17:56 UTC (permalink / raw)
To: David Steele <david@pgmasters.net>; Bruce Momjian <bruce@momjian.us>; +Cc: Joshua D. Drake <jd@commandprompt.com>; Robert Haas <robertmhaas@gmail.com>; Tom Lane <tgl@sss.pgh.pa.us>; Andres Freund <andres@anarazel.de>; Stephen Frost <sfrost@snowman.net>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On 05/31/2015 03:51 AM, David Steele wrote:
> On 5/30/15 8:38 PM, Joshua D. Drake wrote:
>>
>> On 05/30/2015 03:48 PM, David Steele wrote:
>>> On 5/30/15 2:10 PM, Robert Haas wrote:
>>>> What, in this release, could break things badly? RLS? Grouping sets?
>>>> Heikki's WAL format changes? That last one sounds really scary to me;
>>>> it's painful if not impossible to fix the WAL format in a minor
>>>> release.
>>>
>>> I would argue Heikki's WAL stuff is a perfect case for releasing a
>>> public alpha/beta soon. I'd love to test PgBackRest with an "official"
>>> 9.5dev build. The PgBackRest test suite has lots of tests that run on
>>> versions 8.3+ and might well shake out any bugs that are lying around.
>>
>> You are right. Clone git, run it nightly automated and please, please
>> report anything you find. There is no reason for a tagged release for
>> that. Consider it a custom, purpose built, build-test farm.
>
> Sure - I can write code to do that. But then why release a beta at all?
FWIW: we also carry "official" snapshots on the download site (
https://ftp.postgresql.org/pub/snapshot/dev/) that you could use if you
dont want git directly - those even receive some form of QA (for a
snapshot to be posted it is required to pass a full buildfarm run on the
buildbox).
Stefan
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-06-04 08:51 Heikki Linnakangas <hlinnaka@iki.fi>
parent: Andres Freund <andres@anarazel.de>
1 sibling, 2 replies; 78+ messages in thread
From: Heikki Linnakangas @ 2015-06-04 08:51 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; Bruce Momjian <bruce@momjian.us>; +Cc: Robert Haas <robertmhaas@gmail.com>; Tom Lane <tgl@sss.pgh.pa.us>; Stephen Frost <sfrost@snowman.net>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On 05/30/2015 11:47 PM, Andres Freund wrote:
> I don't think it's primarily a problem of lack of review; although that
> is a large problem. I think the biggest systematic problem is that the
> compound complexity of postgres has increased dramatically over the
> years. Features have added complexity little by little, each not
> incrementally not looking that bad. But very little has been done to
> manage complexity. Since 8.0 the codesize has roughly doubled, but
> little has been done to manage the increased complexity. Few new
> abstractions have been introduced and the structure of the code is
> largely the same.
>
> As a somewhat extreme example, let's look at StartupXLOG(). In 8.0 it
> was ~500 LOC, in master it's ~1500. The interactions in 8.0 were
> complex, they have gotten much more complex since. It fullfills lots of
> different roles, all in one function:
>
> (roughly in the order things happen, but simplified)
> * Read the control file/determine whether we crashed
> * recovery.conf handling
> * backup label handling
> * tablespace map handling (huh, I missed that this was added directly to
> StartupXLOG. What a bad idea)
> * Determine whether we're doing archive recovery, read the relevant
> checkpoint if so
> * relcache init file removal
> * timeline switch handling
> * Loading the checkpoint we're starting from
> * Initialization of a lot of subsystems
> * crash recovery/replay
> * Including pgstat, unlogged table, exported snapshot handling
> * iff hot standby, some more subsystems are initialized here
> * hot standby state handling
> * replay process intialization
> * crash replay itself, including
> * progress tracking
> * recovery pause handling
> * nextxid tracking
> * timeline increase handling
> * hot standby state handling
> * unlogged relations handling
> * archive recovery handling
> * creation/initialization of the end of recovery checkpoint
> * timeline increment if failover
> * subsystem initialization iff !hot_standby
> * end of recovery actions
>
> Yes. that's one routine. And, to make things even funnier, half of that
> routine isn't exercised by our tests.
>
> You can argue that this is an outlier, but I don't think so. Heapam, the
> planner, etc. have similar cases.
>
> And I think this, to some degree, explains a lot of the multixact
> problems. While there were a few "simple bugs", most of them were
> interactions between the various subsystems that are rather intricate.
I think this explanation is wrong. I agree that there are many places
that would be good to refactor - like StartupXLOG() - but the multixact
code was not too bad in that regard. IIRC the patch included some
refactoring, it added some new helper functions in heapam.c, for
example. You can argue that it didn't do enough of it, but that was not
the big issue.
The big issue was at the architecture level. Basically, we liked
vacuuming of XIDs and clog so much that we decided that it'd be nice if
you had to vacuum multixids too, in order to not lose data. Many of the
bugs and issues were not new - we had multixids before - but we upped
the ante and turned minor locking bugs into data loss. And that had
nothing to do with the code structure - we'd have similar issues if we
had rewritten everything java, with the same design.
So, I'm all for refactoring and adding abstractions where it makes
sense, but it's not going to solve design problems.
- Heikki
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-06-04 09:17 Andres Freund <andres@anarazel.de>
parent: Heikki Linnakangas <hlinnaka@iki.fi>
1 sibling, 0 replies; 78+ messages in thread
From: Andres Freund @ 2015-06-04 09:17 UTC (permalink / raw)
To: Heikki Linnakangas <hlinnaka@iki.fi>; +Cc: Bruce Momjian <bruce@momjian.us>; Robert Haas <robertmhaas@gmail.com>; Tom Lane <tgl@sss.pgh.pa.us>; Stephen Frost <sfrost@snowman.net>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On 2015-06-04 11:51:44 +0300, Heikki Linnakangas wrote:
> I think this explanation is wrong. I agree that there are many places that
> would be good to refactor - like StartupXLOG() - but the multixact code was
> not too bad in that regard. IIRC the patch included some refactoring, it
> added some new helper functions in heapam.c, for example. You can argue that
> it didn't do enough of it, but that was not the big issue.
Yea, but the bugs were more around the interactions to other parts of
the system. Like e.g. crash recovery, which now is about bug 7 or
so. And those are the ones that are hard to understand.
> The big issue was at the architecture level. Basically, we liked vacuuming
> of XIDs and clog so much that we decided that it'd be nice if you had to
> vacuum multixids too, in order to not lose data. Many of the bugs and issues
> were not new - we had multixids before - but we upped the ante and turned
> minor locking bugs into data loss. And that had nothing to do with the code
> structure - we'd have similar issues if we had rewritten everything java,
> with the same design.
I think we're probably just using slightly different terms here - for me
one very good way of fixing some structurally bad things *is* improving
the design.
If you look at the bugs around multixacts: The first few were around
ctid-chaining, hard to find and fix because there's about 8-10 places
implementing it with slight differences. The next bunch were around
vacuuming, some of them oversights, a good bunch of them more
fundamental. Crash recovery wasn't thought about (lack of
testing/review), and more generally the new code tripped over bad old
decisions (hey, wraparound is ok!). Then there were a bunch of stupid
bugs in crash-recovery (testing mainly), and larger scale bugs (hey, let's
access stuff during recovery). Then there's the whole row level locking
code - which is by now among the hardest to understand code in
postgres - and voila it contained a bunch of oversights that were hard
to spot.
So yes, I think nicer code to work with would have prevented us from
making a significant portion of these. It might have also made us
realize earlier how significant the increase in complexity was.
> So, I'm all for refactoring and adding abstractions where it makes sense,
> but it's not going to solve design problems.
I personally don't really see the multixact changes being that bad on
the overall design. It pretty much just extended an earlier design. Now
that wasn't great, but I don't think too many people had realized that
at that point. The biggest problem was underestimating the complexity.
Greetings,
Andres Freund
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-06-04 09:27 Heikki Linnakangas <hlinnaka@iki.fi>
parent: Heikki Linnakangas <hlinnaka@iki.fi>
1 sibling, 0 replies; 78+ messages in thread
From: Heikki Linnakangas @ 2015-06-04 09:27 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Bruce Momjian <bruce@momjian.us>; Robert Haas <robertmhaas@gmail.com>; Tom Lane <tgl@sss.pgh.pa.us>; Stephen Frost <sfrost@snowman.net>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On 06/04/2015 12:17 PM, Andres Freund wrote:
> On 2015-06-04 11:51:44 +0300, Heikki Linnakangas wrote:
>> So, I'm all for refactoring and adding abstractions where it makes sense,
>> but it's not going to solve design problems.
>
> I personally don't really see the multixact changes being that bad on
> the overall design. It pretty much just extended an earlier design. Now
> that wasn't great, but I don't think too many people had realized that
> at that point. The biggest problem was underestimating the complexity.
Yeah, many of the issues were pre-existing, and would've been good to
fix anyway.
The multixact issues remind me of the another similar thing we did: the
visibility map. It too was non-critical when it was first introduced,
but later we started using it for index-only-scans, and it suddenly
became important that it's up-to-date and crash-safe. We did uncover
some bugs in that area when index-only-scans were introduced, similar to
the multixact bugs, only not as bad because it didn't lead to data loss.
I don't have any point to make with that comparison, but it was similar
in many ways.
- Heikki
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 78+ messages in thread
* Re: [CORE] postpone next week's release
@ 2015-06-04 09:28 Simon Riggs <simon@2ndQuadrant.com>
parent: Tom Lane <tgl@sss.pgh.pa.us>
3 siblings, 0 replies; 78+ messages in thread
From: Simon Riggs @ 2015-06-04 09:28 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Robert Haas <robertmhaas@gmail.com>; Andres Freund <andres@anarazel.de>; Stephen Frost <sfrost@snowman.net>; Bruce Momjian <bruce@momjian.us>; Magnus Hagander <magnus@hagander.net>; pgsql-hackers; pgsql-core <pgsql-core@postgresql.org>
On 30 May 2015 at 05:08, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Robert Haas <robertmhaas@gmail.com> writes:
> > On Fri, May 29, 2015 at 6:33 PM, Andres Freund <andres@anarazel.de>
> wrote:
> >> Why? A large portion of the input required to go from beta towards a
> >> release is from actual users. To see when things break, what confuses
> >> them and such.
>
> > I have two concerns:
>
> > 1. I'm concerned that once we release beta, any idea about reverting a
> > feature or fixing something that is broken will get harder, because
> > people will say "well, we can't do that after we've released a beta".
> > I confess to particularly wanting a solution to the item listed as
> > "custom-join has no way to construct Plan nodes of child Path nodes",
> > the history of which I'll avoid recapitulating until I'm sure I can do
> > it while maintaining my blood pressure at safe levels.
>
> > 2. Also, if we're going to make significant multixact-related changes
> > to 9.5 to try to improve reliability, as you proposed on the other
> > thread, then it would be nice to do that before beta, so that it gets
> > tested. Of course, someone is bound to point out that we could make
> > those changes in time for beta2, and people could test that. But in
> > practice I think that'll just mean that stuff is only out there for
> > let's say 2 months before we put it in a major release, which ain't
> > much.
>
> I think your position is completely nuts. The GROUPING SETS code is
> desperately in need of testing. The custom-plan code is desperately
> in need of fixing and testing. The multixact code is desperately
> in need of testing. The open-items list has several other problems
> besides those. All of those problems are independent. If we insist
> on tackling them serially rather than in parallel, 9.5 might not come
> out till 2017.
>
> I agree that we are not in a position to promise features won't change.
> So let's call it an alpha not a beta --- but for heaven's sake let's
> try to move forward on all these issues, not just some of them.
>
I think releasing 9.5 in some form NOW will aid its software quality.
We've never linked Beta release date to final release date, so if the
quality proves to be as poor as some people think then the list of bugs
will show that and we release later.
AFAIK beta period is exactly the time when we are allowed to pull features
from the release. I welcome the idea that we test it, if its stable and it
works we release it. If doesn't, we pull it.
Not releasing our software yet making a list of our fears doesn't work
towards a solution. Our fears will make us shout at each other too, so I
for one would rather skip that part and do some practical actions.
--
Simon Riggs http://www.2ndQuadrant.com/
<http://www.2ndquadrant.com/;
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
^ permalink raw reply [nested|flat] 78+ messages in thread
end of thread, other threads:[~2015-06-04 09:28 UTC | newest]
Thread overview: 78+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2015-05-29 18:02 postpone next week's release Robert Haas <robertmhaas@gmail.com>
2015-05-29 18:53 ` Bruce Momjian <bruce@momjian.us>
2015-05-29 18:54 ` Stephen Frost <sfrost@snowman.net>
2015-05-29 18:57 ` Bruce Momjian <bruce@momjian.us>
2015-05-29 19:11 ` Magnus Hagander <magnus@hagander.net>
2015-05-29 19:31 ` Stephen Frost <sfrost@snowman.net>
2015-05-29 19:32 ` Tom Lane <tgl@sss.pgh.pa.us>
2015-05-29 19:35 ` Magnus Hagander <magnus@hagander.net>
2015-05-29 19:43 ` Tom Lane <tgl@sss.pgh.pa.us>
2015-05-29 19:46 ` Stephen Frost <sfrost@snowman.net>
2015-05-29 19:47 ` Magnus Hagander <magnus@hagander.net>
2015-05-29 19:53 ` Stephen Frost <sfrost@snowman.net>
2015-05-29 19:37 ` Stephen Frost <sfrost@snowman.net>
2015-05-29 19:56 ` Bruce Momjian <bruce@momjian.us>
2015-05-29 19:57 ` Stephen Frost <sfrost@snowman.net>
2015-05-29 20:01 ` Tom Lane <tgl@sss.pgh.pa.us>
2015-05-29 20:03 ` Stephen Frost <sfrost@snowman.net>
2015-05-29 20:08 ` Joshua D. Drake <jd@commandprompt.com>
2015-05-29 20:07 ` Bruce Momjian <bruce@momjian.us>
2015-05-29 20:29 ` Robert Haas <robertmhaas@gmail.com>
2015-05-29 20:37 ` Tom Lane <tgl@sss.pgh.pa.us>
2015-05-29 21:04 ` Andres Freund <andres@anarazel.de>
2015-05-29 21:12 ` Bruce Momjian <bruce@momjian.us>
2015-05-29 21:14 ` Andres Freund <andres@anarazel.de>
2015-05-29 21:37 ` Tom Lane <tgl@sss.pgh.pa.us>
2015-05-29 22:28 ` Bruce Momjian <bruce@momjian.us>
2015-06-01 16:12 ` Jim Nasby <Jim.Nasby@BlueTreble.com>
2015-06-01 16:32 ` Tom Lane <tgl@sss.pgh.pa.us>
2015-06-01 16:34 ` Andres Freund <andres@anarazel.de>
2015-06-01 16:56 ` Tom Lane <tgl@sss.pgh.pa.us>
2015-06-01 18:09 ` Josh Berkus <josh@agliodbs.com>
2015-05-29 22:02 ` Robert Haas <robertmhaas@gmail.com>
2015-05-29 22:33 ` Andres Freund <andres@anarazel.de>
2015-05-30 00:07 ` Stephen Frost <sfrost@snowman.net>
2015-05-30 03:56 ` Robert Haas <robertmhaas@gmail.com>
2015-05-30 04:08 ` Tom Lane <tgl@sss.pgh.pa.us>
2015-05-30 04:25 ` Andres Freund <andres@anarazel.de>
2015-05-30 12:56 ` Bruce Momjian <bruce@momjian.us>
2015-05-30 13:11 ` Bruce Momjian <bruce@momjian.us>
2015-05-30 14:11 ` Joshua D. Drake <jd@commandprompt.com>
2015-05-30 19:26 ` Peter Geoghegan <pg@heroku.com>
2015-05-31 02:24 ` Bruce Momjian <bruce@momjian.us>
2015-05-31 02:40 ` David G. Johnston <david.g.johnston@gmail.com>
2015-05-30 14:06 ` Robert Haas <robertmhaas@gmail.com>
2015-05-30 15:45 ` Bruce Momjian <bruce@momjian.us>
2015-05-30 18:10 ` Robert Haas <robertmhaas@gmail.com>
2015-05-30 19:46 ` Peter Geoghegan <pg@heroku.com>
2015-05-31 11:24 ` Robert Haas <robertmhaas@gmail.com>
2015-05-30 21:02 ` Andres Freund <andres@anarazel.de>
2015-05-30 21:19 ` Tom Lane <tgl@sss.pgh.pa.us>
2015-05-30 21:23 ` Andres Freund <andres@anarazel.de>
2015-05-30 22:48 ` David Steele <david@pgmasters.net>
2015-05-31 00:38 ` Joshua D. Drake <jd@commandprompt.com>
2015-05-31 01:51 ` David Steele <david@pgmasters.net>
2015-05-31 02:38 ` Joshua D. Drake <jd@commandprompt.com>
2015-05-31 15:49 ` Noah Misch <noah@leadboat.com>
2015-05-31 16:09 ` David Steele <david@pgmasters.net>
2015-06-02 05:44 ` Noah Misch <noah@leadboat.com>
2015-06-03 17:56 ` Stefan Kaltenbrunner <stefan@kaltenbrunner.cc>
2015-05-30 20:47 ` Andres Freund <andres@anarazel.de>
2015-05-31 02:48 ` Bruce Momjian <bruce@momjian.us>
2015-05-31 11:15 ` Michael Paquier <michael.paquier@gmail.com>
2015-05-31 13:50 ` Bruce Momjian <bruce@momjian.us>
2015-05-31 14:03 ` Bruce Momjian <bruce@momjian.us>
2015-05-31 15:55 ` Tom Lane <tgl@sss.pgh.pa.us>
2015-05-31 16:24 ` Bruce Momjian <bruce@momjian.us>
2015-05-31 20:16 ` Andres Freund <andres@anarazel.de>
2015-06-01 01:17 ` Michael Paquier <michael.paquier@gmail.com>
2015-06-04 08:51 ` Heikki Linnakangas <hlinnaka@iki.fi>
2015-06-04 09:17 ` Andres Freund <andres@anarazel.de>
2015-06-04 09:27 ` Heikki Linnakangas <hlinnaka@iki.fi>
2015-06-04 09:28 ` Simon Riggs <simon@2ndQuadrant.com>
2015-05-30 04:17 ` Andres Freund <andres@anarazel.de>
2015-05-30 06:38 ` Noah Misch <noah@leadboat.com>
2015-05-29 19:09 ` Magnus Hagander <magnus@hagander.net>
2015-05-29 19:18 ` Robert Haas <robertmhaas@gmail.com>
2015-05-29 19:32 ` Joshua D. Drake <jd@commandprompt.com>
2015-06-01 13:38 ` Tom Lane <tgl@sss.pgh.pa.us>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox