Subject: Feature freeze progress report

From: Bruce Momjian <[email-masked]>
Date: Thu, 26 Apr 2007 23:13:58 -0400 (EDT)

Now that we are half-way though the scheduled feature freeze, I wanted
to share my thoughts about this period.

Having just pushed all open items into the patches queue or 8.4 hold
queue, I am seeing that we have many more in-process patches than we
normally do at this stage of the process.  I think there are three
reasons for this:

1)  The patches are not necessarily larger, but are more complex because
much most of the easy TODO items have already been written for previous
PostgreSQL releases.

2)  We have a number of new developers who took on some of these complex
TODO items, and some of the TODO items were significantly beyond the
developer capabilities at the start of the process.

3)  Many of the complex patches are hard to review because they deal
with very complex areas of the code, like reliability or transaction
semantics.

Our community could probably handle a few of these complex patches, but
the volume for this release is significantly higher than previous
releases.  The community is doing a good job of giving patch writers
feedback and new patch versions are getting generated.  However, this
amount of patch churn is not normal.

There are a few possible results that might come out of this:

1)  Feature freeze will be much longer.
2)  More patches will be postponed for later releases than usual.
3)  Most patches will be included but beta will be longer because
    of bug fixing.
4)  Most patches will be included but beta will not be any longer.

I think we all hope for #4, but right now, I don't know the probability
of that.  We are going to have to think creatively in the coming weeks
to increase the likelihood of a #4 result.  However, right now, I can't
think of what we can do to improve the odds.  I think the community has
to come up with ideas on how to accomplish this.

[ FYI, I leave on a 2-week trip tomorrow/Friday.]

-- 
  Bruce Momjian  <[email-masked]>          http://momjian.us
  EnterpriseDB                               http://www.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

---

From: "Simon Riggs" <[email-masked]>
Date: Sat, 28 Apr 2007 21:40:35 +0100

> amount of patch churn is not normal.

I think it is probably going to be normal from now on. We now a
significant number of reasonably prolific developers.

> I think the community has to come up with ideas on how to accomplish this.

My thinking is to move to a two stage release process: Do one
"production" release annually, and one "dev" release at the 6 month
mid-point. That way each new release contains a manageable number of new
features and we have a realistic chance of integrating them
successfully. Support companies would then have the option to support
both releases, or just the main production release. Leading edge users,
of which we have many, would then benefit from more frequent additional
features.

I would also suggest that 8.3 be labelled a dev release. We have a
reasonable number of fairly invasive patches, so we need a mechanism to
integrate them with reduced risk.

With those two suggestions, the prod release would freeze on Sep 30 and
the dev release on Mar 31. This would then put us into the same
situation as Linux, where odd-numbered releases are dev and
even-numbered are main releases. Everyone would understand our decision
to take this action, as well as immediately understanding how this
process will work in the future.

By agreeing this now, we can then punt a reasonable number of patches
back to the main release, later this year. The de-selected patches still
have a second chance of making it into a release available in 2007 and
this will diffuse the various tensions and difficulties we now have.

Without this, we face a long wait. 8.2 took 4 months to go through beta
and release, so 8.3 could well take 6 months. If we allow 8.3 to be a
mega-release then it might take much longer than that: increases in
complexity have a non-linear effect on software quality/robustness.
Adding reviewers or committers isn't going to dramatically change that.
IMHO the only sensible thing to do is to make releases more frequent so
that each one is still a manageable size. 

The alternative is to somehow select patches to wait until the next
release, a full year away. That is unlikely to be an easy process and
nobody really wants to volunteer their own or others' patches.

Realistically, people won't speed up the frequency they upgrade their
software and we certainly don't want to increase the number of
production releases in circulation that we must support. This set of
proposals is a realistic way forward from where we are and will be
easily explained to people only briefly in touch with our project.

Whether or not this is accepted, I'm happy to offer assistance wherever
the core team directs to improve the current situation.

-- 
  Simon Riggs

---

From: Stefan Kaltenbrunner <[email-masked]>
Date: Sun, 29 Apr 2007 13:36:00 +0200

> of which we have many, would then benefit from more frequent additional

I'm not really convinced that this is good idea at all - it would lead
to further fragmentation of developer resources (likely more versions to
support and more frequent releases which to put quite a load on
developers by itself). And a lot of issues only get found in the field
and I'm unsure if a releases labled "dev" might get the critical mass in
terms of testing (if that would be true we would find most of the bugs
during beta imho).
99% of the people only use what is declared "stable" and already has a
number of minor releases - and the other 1% is already reading -hackers ...

> integrate them with reduced risk.

I would rather like to see patches we don't are confident enough in to
be dropped from 8.3 and moved to 8.4 - the goal should not be jamming as
much patches into a single release s we can (because they are proposed)
but rather putting those in that meet the quality bar and we trust in.

> process will work in the future.

I don't want to see the current linux model - which is basically that
2.6 is a continous development trunk with snapshots (ie "releases") done
every few months that only get supported until the next release or so.
Note that all the distributions spend considerable amount of time
stabilizing those kernels and have to put additional resources into
maintaining those over the years because upstream is not doing that any
more.
Look into the recent discussion about releasing 2.6.21 with a number of
KNOWN regressions for example.

> that each one is still a manageable size. 

again - the bottleneck right now seems to be reviewer capacity coupled
with a large number of intrusive patches. So the most logical answer is
to drop those patches we are not fully confident in and try to get them
in during 8.4.

> nobody really wants to volunteer their own or others' patches.

see above

Stefan

---

From: "Dawid Kuroczko" <[email-masked]>
Date: Sun, 29 Apr 2007 13:43:58 +0200

> of which we have many, would then benefit from more frequent additional

This would mean we would have to have a very well tested upgrade path
for odd releases (8.2 -> 8.4).

Also it probably would mean that analytical functions or recursive queries
should be postponed until 8.5 (as they didn't end up inside 8.3, and 8.4
would be "stable" release).

I think that with introducing stable/devel version we are risking that devel
versions will be less used in production environments (meaning less testing)
and as a result they can lengthen the development cycle.  Currently every
release is stable, therefore we don't accept "experimental patches" unless
they are really good idea.  Then there is beta sequence, and then a stable
release.  With introducing dev release, we give green light to more
"experimental"
patches, and then devote dev release as a ripening period for them (equivalent
of current pre-releases, I imagine).  And then we release stable relese (without
"experimental" patches; experimental patches are postponed until devel release,
and devel release twice the number of experimental patches).

I think we should not go into stable/devel release cycle without carefully
thinking if it will serve us good.  I am afraid this will make many people
stick with stable releases and will make upgrades harder (do you remember
how hard it was to go from Linux 2.2 to 2.4, and from 2.4 to 2.6?).

  Regards,
      Dawid

---

From: Heikki Linnakangas <[email-masked]>
Date: Sun, 29 Apr 2007 14:34:44 +0100

> of which we have many, would then benefit from more frequent additional

I like the idea of draining the patch queue mid-way through the release 
cycle. That'll hopefully encourage people to submit patches earlier in 
the release cycle, knowing they will be reviewed. It'll also give people 
working on external projects, drivers and tools, a checkpoint to sync with.

But I don't like the idea of making a release out of it. Who would use 
such a release? No one in production. Making a release comes with a 
cost, even if it's just a dev release.

One could also argue that we don't need the mid-cycle checkpoint, if we 
just keep the patch queue empty all the time. In the end, it comes down 
to how many people we have actively reviewing patches and giving 
feedback (I agree that it's not a linear relationship as you pointed out 
later in your mail, though). I believe a mid-cycle checkpoint would help 
by directing efforts to review, just like the pre-release feature freeze 
does.

> integrate them with reduced risk.

I have no reason to believe that the next release will have less patches 
in it, so if we went down that path we could never release a stable 
release. If we have reasonable doubts about the stability of a patch, it 
should not be included. That said, all patches come with a risk.

> process will work in the future.

We're having a short 8.3 cycle because we wanted to shift our release 
schedule from Autumn to Spring. That would get us back to releasing in 
Autumn.

-- 
   Heikki Linnakangas

---

From: Alvaro Herrera <[email-masked]>
Date: Sun, 29 Apr 2007 11:40:06 -0400

> but rather putting those in that meet the quality bar and we trust in.

Yeah; the agreement we had was that 8.3 would be a short release.  So if
we're going to take too long to review and apply the outstanding patches
we have, we should rather push them to 8.4, get 8.3 released quickly and
then go on with the regular annual release.  The postponed patches can
be reviewed and committed early in 8.4, instead of at the last minute in
8.3.  Sounds like a smarter, safer move.

(The only complication would be the pgindent changes which could cause
merge problems for some patches.  It would be good to have a mechanism
to "update" a patch over pgindent easily.)

-- 
Alvaro Herrera                                http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

---

From: Gregory Stark <[email-masked]>
Date: Sun, 29 Apr 2007 16:47:58 +0100

"Heikki Linnakangas" <[email-masked]> writes:

> I like the idea of draining the patch queue mid-way through the release cycle.
> That'll hopefully encourage people to submit patches earlier in the release
> cycle, knowing they will be reviewed. It'll also give people working on
> external projects, drivers and tools, a checkpoint to sync with.
>
> But I don't like the idea of making a release out of it. Who would use such a
> release? No one in production. Making a release comes with a cost, even if it's
> just a dev release.

On other projects people use these snapshots or dev releases because checking
stuff out from CVS is likely to get you a source tree that won't even build
let alone run cleanly. It's also nice to have everyone using the same
checkouts when report bugs or submitting patches.

But it's not because we're afraid some user will run a CVS checkout that
Postgres CVS is kept clean. Postgres CVS is kept clean because that's just the
way the Postgres developers think it should work. Doing regular snapshot
releases isn't going to cause that to get worse.

> One could also argue that we don't need the mid-cycle checkpoint, if we just
> keep the patch queue empty all the time. In the end, it comes down to how many
> people we have actively reviewing patches and giving feedback

I would argue that. In fact I would argue it would be *easier* to keep the
patch queue empty all the time than to spend months reviewing and merging
six-month-old patches once a year. But I still have hope this is a problem
that will fix itself naturally with time.

-- 
  Gregory Stark

---

From: Gregory Stark <[email-masked]>
Date: Sun, 29 Apr 2007 16:50:24 +0100

"Alvaro Herrera" <[email-masked]> writes:

> The postponed patches can be reviewed and committed early in 8.4, instead of
> at the last minute in 8.3.

Given that some of those patches have been in the queue since last September
is there any reason to think they'll get reviewed early in this cycle if they
weren't last cycle?

---

From: Lukas Kahwe Smith <[email-masked]>
Date: Sun, 29 Apr 2007 09:30:22 -0700

> 8.3.  Sounds like a smarter, safer move.

Hmm, I do not have an overview on this, but like Alvaro mentions, the 
shorter release cycles for 8.3 was done because we felt that a number of 
patches that were originally slated for 8.2 were almost but not quite 
ready for 8.2. So are all of those patches from back then ready to go 
into 8.3? If not then it would indicate that fast tracking a release 
cycle for patches there are not quite there yet is not paying off?

Otherwise, if all/most of the patches originally planned for 8.2 have 
made it into 8.3, everything is good. If new additions are not yet ready 
then they will just get bumped to 8.4, just like the changes that got 
bumped to 8.3.

regards,
Lukas

---

From: Tom Lane <[email-masked]>
Date: Sun, 29 Apr 2007 12:53:30 -0400

Stefan Kaltenbrunner <[email-masked]> writes:
>> mid-point.

> developers by itself).

That's my reaction too.  The overhead of a "dev" release would be just
as high as a full release, and we don't really have enough manpower
to do two releases a year.  We *definitely* haven't got enough manpower
to double the number of back branches we are trying to keep patched.
So this could only work if dev releases are abandoned from a support
perspective when the next full release comes out, and that will entirely
guarantee that no DBA will use one in production.

			regards, tom lane

---

From: Tom Lane <[email-masked]>
Date: Sun, 29 Apr 2007 13:07:46 -0400

> cycle for patches there are not quite there yet is not paying off?

In fact several of the major ones are still not ready, so I think that
experience is evidence that we couldn't handle a six-month release cycle
anyway.  We'll still stick to the announced 8.3 schedule though, since
part of the reason for it was to rotate around to a spring release time
instead of a fall release time, on the thought that that might work more
conveniently for many people.

			regards, tom lane

---

From: Dave Page <[email-masked]>
Date: Sun, 29 Apr 2007 20:04:05 +0100

> working on external projects, drivers and tools, a checkpoint to sync with.

This is important for me - the pgAdmin development cycle follows that of 
PostgreSQL's for very obvious reasons, however *after* we enter 'feature 
freeze' we can still end up adding significant amounts of new code. Why? 
Becvause during PostgreSQL's feature freeze, patches are applied which 
add new functionality we need to support. We can't code for the new 
features when patches are submitted because we don't know if they'll go 
in, or how much they'll change when they do.

This means that there is a huge rush of new code in pgAdmin's 
development cycle, right at the time when we should be testing - making 
the release process more and more rushed as each release of PostgreSQL 
gets more efficient and adds more and more new features.

Sooner or later with things working the way they are now, we *will* 
reach a breaking point at which pgAdmin simply won't be ready when 
PostgreSQL is released.

> cost, even if it's just a dev release.

Agreed. That would have the opposite effect of what should happen.

I like the idea of having a sync point mid cycle, however, what I'd like 
to see even more is an improved system in which we put less pressure on 
the few committers we have, and give them more freedom to commit patches 
they may not understand fully themselves by having an improved community 
review and feedback process to give the patches the approval they need. 
Doing so might allow us to keep the queue of a more or less fixed, short 
length throughout the cycle. There would be a few advantages to this:

- The problem described above practically vanishes.
- Developers see their work accepted more quickly, giving them the 
confidence to produce more.
- Developers are able to build on their earlier work, knowing that it's 
believed to be reasonably sound, unlike now when they may not know for 
months.

I believe we can achieve this with a couple of relatively minor changes:

1) *Slowly* introduce more committers. The are developers gaining 
experience all the time, and as they become trusted by the existing 
committers/core they can be 'promoted' to alleviate some of the pressure 
on the existing guys.

2) Introduce a new patch management system. I suggest a web interface 
through which patches be submitted. This would assign them an ID number, 
and forward them to the patches list. The system would track any 
responses to the initial email, logging the thread automatically and 
making it available through the web interface. Posts from 
trusted/experienced developers might be highlighted so that committers 
can see at a glance if any of the more experienced guys have commented 
on the patch yet. A status flag could easily include a status flag to 
mark them as "won't accept", "committed", "under review" or "under 
revision". If left at "under review" for too long, the patch might be 
highlighted, and if at "under revision" for too long, the patch author 
might be automatically requested to send a status report.

There are potentially a number of benefits to such a system:

- No patches will get lost
- Bruce's time is feed up from the mundane work of tracking patches, to 
allow him to spend time developing, reviewing/committing and all the 
other great jobs he does for the community.
- Status of patches can be seen at a glance.
- *All* discussion of a patch can be logged in one place, allowing the 
committer to put more reliance on the knowledge and experience of his 
peers, rather than being expected to fully understand the minutae of 
every patch - thus allowing him to commit more.

Well, I'll stop there as this is getting long winded - I'm sure others 
will have their own ideas about how we can improve our processes for 
future releases; one thing I'm certain of though, is that we absolutely 
must strive to improve them somehow as whilst they has served us well in 
the past, the current process is starting to show that it just won't 
scale as the project grows.

Regards, Dave

---

From: Stefan Kaltenbrunner <[email-masked]>
Date: Sun, 29 Apr 2007 21:30:36 +0200

Dave Page wrote:
>> I like the idea of draining the patch queue mid-way through the
>> release cycle. That'll hopefully encourage people to submit patches
>> earlier in the release cycle, knowing they will be reviewed. It'll
>> also give people working on external projects, drivers and tools, a
>> checkpoint to sync with.
> 

this is indeed an issue - but there is nothing that says that pgadminIII
has to support all the new features of a backend the they it get
released. pgadmin is decoupled from the min development cycle anyway so
adding support for a new features a few months later sounds not too much
an issue.

> reach a breaking point at which pgAdmin simply won't be ready when

well if it still works but does not yet support $wizzbangnewfeature that
sounds ok too me ?

[...]

> might be automatically requested to send a status report.

this sounds like trying to reinvent a real bugtracking system with an
email interface ...

[...]

> scale as the project grows.

not sure I fully agree here - I think we could do a bit better on the
"bug tracking front" but it is a bit unclear if we cn honestly sy that
"the current process" does not work or is not going to scale in the
future. Complex patches need time - sometimes much more time than a
release or even two release cycles - it's unclear if trying to get those
in more agressively (by having more commiters or applying them earlier)
might not actually cause more harm due to -HEAD getting less stable and
causing developers to spend time hunting regressions.

Stefan

---

From: Dave Page <[email-masked]>
Date: Sun, 29 Apr 2007 21:15:59 +0100

> adding support for a new features a few months later sounds not too much

No it's not decoupled; it runs almost exactly in sync. Our most popular
platform ships with pgAdmin as standard because thats what the users of
that platform expect. Not shipping with pgAdmin (or a functionally
complete one) would be like shipping SQL Server without Enterprise Manager.

> well if it still works but does not yet support $wizzbangnewfeature that

Who's to say it will? Changes to pg_database have required a new release
in the past.

> this sounds like trying to reinvent a real bugtracking system with an

More or less - but one that's simple by design, and specifically for
tracking what happens with our patches with the absolute minimum amount
of change required to our existing communication methods.

> release or even two release cycles - 

I'm not specifically talking about complex patches (nor am I talking at
all about bug tracking) - there are a variety of patches in the queue,
of varying complexity. Some have been there for months, and worse, some
of them recieved little or no feedback when submitted leaving the
authors completely in the dark about whether their work will be
included, whether further changes are required, or whether they should
continue with additional enhancements.

it's unclear if trying to get those
> causing developers to spend time hunting regressions.

I'm not advocating committing patches that might destabilize the code,
I'm suggesting making it easier for individual committers to make use of
the knowledge and experience of everyone else in the community, whilst
at the same time reducing the reliance on their own experience. Even now
we occasionally see patches getting committed that (for example) Tom has
rejected months earlier. At the very least a tracker should help prevent
that happening, at best it will help committers work faster and more
effectively because they have all the relevant discussion in front of them.

Regards, Dave.

---

From: "Marc G. Fournier" <[email-masked]>
Date: Sun, 29 Apr 2007 18:15:11 -0300

> causing developers to spend time hunting regressions.

re: -HEAD getting less stable ... we wouldn't be adding just anyone as a 
committer, only those that are trusted to know what they are doing ... even 
'slapping in a patch', I would expect it to have some sort of preliminary 
review by *someone* with a bit of knowledge in that aspect of the code ...

- ----
Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email . [email-masked]                              MSN . [email-masked]
Yahoo . yscrappy               Skype: hub.org        ICQ . 7615664

---

From: Lukas Kahwe Smith <[email-masked]>
Date: Sun, 29 Apr 2007 14:52:03 -0700

> this sounds like trying to reinvent a real bugtracking system with an

I think the angle of this suggestion is to approach things from the 
perspective of trying to automate more according to how people are 
currently working instead of shoehorning an existing solution. It seems 
that most folks on -hackers prefer email based systems. The proposals 
sounds like it would not change anything much for people who choose to 
ignore the web interface and as such it has some appeal to it.

regards,
Lukas

---

From: Andrew Dunstan <[email-masked]>
Date: Sun, 29 Apr 2007 19:38:21 -0400

Dave Page wrote:
>
>> But I don't like the idea of making a release out of it. Who would 
>> use such a release? No one in production. Making a release comes with 
>> a cost, even if it's just a dev release.
>
> I like the idea of having a sync point mid cycle, however, what I'd 
> like to see even more is an improved system in which we put less 
> pressure on the few committers we have, and give them more freedom to 
> commit patches they may not understand fully themselves by having an 
> improved community review and feedback process to give the patches the 
> approval they need. Doing so might allow us to keep the queue of a 
> more or less fixed, short length throughout the cycle. There would be 
> a few advantages to this:
>
>

I don't think we need a sync point. I think we need to get better at 
setting expectations and at managing the patch queue so that it gets 
drained better all the time. Nothing can be more frustrating for patch 
authors than to have patches in the queue for a very long time. They 
bitrot, and we sometime end up throwing curly questions at authors long 
after the issues are hot in their minds. I'd like to see us set 
ourselves some targets for handling patches. Something like: patches 
held over from feature freeze from the previous release will be reviewed 
within two months of the tree re-opening, and all other patches will be 
reviewed within one month of being submitted. That implies that one 
month after feature freeze the tree will only be open for bug fixes. Any 
patches unapplied at that time would be held over. Maybe that would give 
pgAdmin and friends enough head room to catch up.

cheers

andrew

---

From: "Marc G. Fournier" <[email-masked]>
Date: Sun, 29 Apr 2007 23:27:12 -0300

- --On Sunday, April 29, 2007 19:38:21 -0400 Andrew Dunstan
<[email-masked]> 
wrote:

> patches held over from feature freeze from the previous
> release will be reviewed within two months of the tree re-opening

a. why 2 months?  shouldn't they be given priority, period?
b. what happens after 2 months?  we delete them?

- ----

---

From: Tom Lane <[email-masked]>
Date: Mon, 30 Apr 2007 00:31:06 -0400

Dave Page <[email-masked]> writes:
> they may not understand fully themselves

That is a recipe for disaster :-(.  The real problem I see with the big
patches that are currently in the queue is that I'm not sure even the
authors understand the patches (or more accurately, all their potential
consequences) completely.  Telling committers they should apply such
patches without having understood them either is just going to lead to
an unfixably broken system.

[ thinks for a bit... ]  What we need to expand is not so much the pool
of committers as the pool of reviewers.  If a patch had been signed off
on by X number of reasonably-qualified people then it'd be fair to
consider that it could be committed.  The tracking system you suggest
could make that sort of approach manageable.

			regards, tom lane

---

From: Stefan Kaltenbrunner <[email-masked]>
Date: Mon, 30 Apr 2007 08:01:04 +0200

> complete one) would be like shipping SQL Server without Enterprise Manager.

Interesting ... so if you have a new feature (or a number of them) -
that is not directly depending on some sort of new backend feature - in
pgadmin you "delay" it until the next postgresql mjor release ?
To be honest I personally have not used pgadmin in years and I have no
idea what SQL-Server would be with or without Enterprise Manager so I
actually don't really know enough to further speculate on this ...

> Who's to say it will? Changes to pg_database have required a new release

hmm true - but I imagine that a change to a catalog like pg_database is
not the kind of feature you need to rush lot's of code in (again
speculating here) ?

> of change required to our existing communication methods.

ah - ok

> continue with additional enhancements.

that one I agree with - heck even people very close to the project are
sometimes unclear about the status of this patch or that patch.
Part of that could probably be solved by the simple tracker you are
proposing - another way might be to promote more usage of the developer
wiki.

Stefan

---

From: Dave Page <[email-masked]>
Date: Mon, 30 Apr 2007 08:32:48 +0100

> could make that sort of approach manageable.

Err, I thought that was precisely what I was suggesting in my second
point :-). To reiterate:

- Expand the pool of committers (slowly, and as appropriate - not for
the sake of it). There inevitably is and will continue to be more work
for experienced committers. We should consider 'promoting' those
developers that become experienced and trusted.

- Use a tracking system to enable the committers to rely more on the
experience of the users. Ideas we have discussed here in the office
~(which I didn't mention earlier) included a scoring system, where
trusted developers (who aren't necessarily committers) can score patches
or veto them if there are real problems spotted and to highlight the
comments from those experienced people to make it easy to spot what they
say.

Regards, Dave.

---

From: Dave Page <[email-masked]>
Date: Mon, 30 Apr 2007 08:44:01 +0100

> pgadmin you "delay" it until the next postgresql mjor release ?

It's not so much that we delay the new features, it's just that we
follow the same development schedule, just with a shorter beta/feature
freeze period. We try to release just before PostgreSQL to avoid our
respective advocacy efforts trampling each other - but that's usually a
bit of a guessing game in itself!

> actually don't really know enough to further speculate on this ...

I imagine the %age of SQL Server users that *don't* use Enterprise
Manager is close to zero. It's a platform on which everything is
expected to be a GUI.

> not the kind of feature you need to rush lot's of code in (again

No, but it's exactly the reason why we release with/just before
PostgreSQL. If we were offset by six months, we might find ourselves
having to do compatibility releases mid-cycle for the latest PostgreSQL
release. A change in pg_database such as we had previously wouldn't be
an issue for the stable branch, but the changes to op classes etc. in
8.3 certainly would be of great concern.

> proposing - another way might be to promote more usage of the developer

Yep, true - though the reason I promote the use of the tracker is that
it could be implemented with minimal invasiveness into the existing
process, such that it automatically captures all discussion on the
topic, whereas I imagine some might object to repeatedly
visting/re-reading/editting a wiki to discuss a patch.

Regards, Dave

---

From: Magnus Hagander <[email-masked]>
Date: Mon, 30 Apr 2007 09:46:21 +0200

> pgadmin you "delay" it until the next postgresql mjor release ?

Yes.

<snip>

> proposing - another way might be to promote more usage of the developer

The advantage of a "proper system" would be that it'd be consistent. Using
the wiki could get very unstructured (unstructured being a *feature* of a
wiki). A specialised system will be able to enforce how things look and are
worked with, and makes the *use* of the system easier. Using the wiki makes
installing it easier, since it's already there, at the cost of usage.

//Magnus

---

From: Stefan Kaltenbrunner <[email-masked]>
Date: Mon, 30 Apr 2007 10:07:52 +0200

> bit of a guessing game in itself!

ah ok - makes sense

> Manager is close to zero. It's a platform on which everything is

I will take your word on that - Iäm neither a Windows nor a MSSQL Server 
user :-)

> 8.3 certainly would be of great concern.

hmm - understood. I guess I simply speculated that doing a pgadmin 
release might be much more lightweight than doing a PostgreSQL release 
(how many "backbranches" is pgadmin supporting?". I think I now 
understand why you are doing it that way though ...

> visting/re-reading/editting a wiki to discuss a patch.

you are probably right here too - though I can see some value in a wiki 
too. Some things that come to mind are subproject specific todo 
lists(like the XMLTodo) or the Wishlist (which is a rather abstracted 
thing to point people to that ask "what can we expect in $release if we 
are really really lucky" and don't want to parse the pgpatches list 
bruce keeps)

Stefan

---

From: Dave Page <[email-masked]>
Date: Mon, 30 Apr 2007 09:15:42 +0100

> release might be much more lightweight than doing a PostgreSQL release

Oh, it is - but then the work is done mainly by me, with Guillaume
handling the translations and other people producing some of the binary
builds. We don't have anything like to pool of developers that
PostgreSQL does.

> understand why you are doing it that way though ...

We don't support any back branches because each version of pgAdmin III
supports PostgreSQL >= 7.3. When a new major release comes out, we drop
support for the previous. We do have the advantage of not having to
worry about on-disk upgrading data formats :-)

Regards,Dave

---

From: Heikki Linnakangas <[email-masked]>
Date: Mon, 30 Apr 2007 09:18:36 +0100

> could make that sort of approach manageable.

Agreed, committing a patch is not much work. If all the patches in the 
queue were perfect and just waiting for committing, one person could 
just commit them all in a day.

What takes time is reviewing. We have people capable of reviewing 
patches, we should encourage them to do that (that includes myself). 
Maybe we should have a more official protocol of "signing-off" patches. 
And part of the review is refactoring and commenting and fixing tiny 
bugs that the original author missed. I've refrained from doing that 
myself because I'm afraid I'd step on the authors toes or joggle the 
elbow of a committer.

One problem with the current patch queue is that it's really hard to get 
a picture of where each patch stands. There's many different reasons why 
a patch can get stalled in the patch queue. Looking at the patches there 
now:

Design issues have been raised:
- index advisor (messy API)
- full page writes improvement, code update (increases WAL size)
- dead space map (uses a fixed size shared memory block)

Dependency on other patches:
- scan-resistant buffer cache
- group commit
- error correction for n_dead_tuples

Do we want this or not? -category
- POSIX shared mem support

Unfinished:
- bitmap indexes

Author is working on patch, based on comments
- heap page diagnostic functions
- load distributed checkpoint

Author is waiting for feedback on how to proceed:
- clustered indexes / grouped index tuples
- concurrent psql patch

(not exhaustive list, just examples)

The above list reflects my view of the status of the patches. Others 
might not agree, and that's a problem in itself: it's not clear even to 
a patch author why a patch isn't moving forward. Author might think a 
patch is just about to be committed, while others are waiting for the 
author to fix an issue raised earlier. Or an author might think there's 
a problem with his patch, while it just hasn't been committed because of 
a dependency to another patch.

If we had a 1-2 lines status blurp attached to each patch in the queue, 
like "waiting for review", "author is fixing issue XX", etc., that might 
help. Bruce would need to do that if we keep the current patch queue 
system unmodified otherwise, or we'd need to switch to something else.

---

From: Bruce Momjian <[email-masked]>
Date: Mon, 30 Apr 2007 07:06:55 -0400 (EDT)

> I'm not really convinced that this is good idea at all - it would lead

Agreed, a bad idea.

> but rather putting those in that meet the quality bar and we trust in.

I don't want us postponing patches for a later release if the patch will
be no easier to apply later than it is right now, i.e. if it is "buckle
down" now or "buckle down" later, we might as well do it now, hence my
desire to move things along now rather than when our backs are against
the wall to get a release out.

Of course, patches were will learn more by waiting for 8.4 should
probably be kept for that release.

> to drop those patches we are not fully confident in and try to get them

We are not going to magically have more time or be more confident for
8.4.  If a patch needs more research or time, we should hold it, but not
having time to review it isn't a reason to hold it -- it just
bottlenecks the next release.

---

From: Bruce Momjian <[email-masked]>
Date: Mon, 30 Apr 2007 07:09:38 -0400 (EDT)

> working on external projects, drivers and tools, a checkpoint to sync with.

Aside from a few complex patches all the patches in the queue are from
work completed just before feature freeze --- we have been draining it
during the entire release.  I think for the few complex patches that
have been in there for a while, the problem is that reviewing them is
going to be so hard, no one has done it.  Now that we are in feature
freeze, we have to do it --- but the idea that we have somehow just been
holding patches during the whole release isn't true.

---

From: Bruce Momjian <[email-masked]>
Date: Mon, 30 Apr 2007 07:15:22 -0400 (EDT)

> 8.3.  Sounds like a smarter, safer move.

Because we are dealing with this now, and not later, we have time to
give all patches the appropriate review time --- we don't need to panic
yet and start throwing patches to 8.4, especially since we might have
even more patches for 8.4.

---

From: Bruce Momjian <[email-masked]>
Date: Mon, 30 Apr 2007 07:16:46 -0400 (EDT)

> is there any reason to think they'll get reviewed early in this cycle if they

Agreed, though most patches are from late March, just before feature
freeze.  I don't see any really old stuff except the index advisor,
which isn't ready for application because the user and backend API need
work.

---

From: Bruce Momjian <[email-masked]>
Date: Mon, 30 Apr 2007 07:19:20 -0400 (EDT)

> then they will just get bumped to 8.4, just like the changes that got 

The patches _might_ be ready.  Please re-read my earlier posting that
started this thread -- the major problem is new developers adding
complex features, and the difficulty of reviewing all of that.

Fortunately I have gotten approval from EnterpriseDB for Heikki to spend
full-time helping with the 8.3 patch queue, and he, Tom and I have
already been over many of the items via private email and will be moving
forward.

---

From: Bruce Momjian <[email-masked]>
Date: Mon, 30 Apr 2007 07:23:09 -0400 (EDT)

> might be automatically requested to send a status report.

It would be interesting if such a system could be made to work simply,
but I am afraid that isn't possible.  What happens now is that as people
post email comments about the patches, I add the emails to the patches
queue.  It would be interesting to put comments on the patches
themselves, but in many cases the opinions about patches are too candid
to put in public so committers email each other to give status reports.

---

From: Bruce Momjian <[email-masked]>
Date: Mon, 30 Apr 2007 07:27:10 -0400 (EDT)

> continue with additional enhancements.

Agreed.  Remember that patches queue is just patches that no one has
dealt with.  It was never designed to be a community thing, but Tom and
others do pull from it as necessary.  If the community dealt with all
patches, I wouldn't have to add anything to the queue.

> effectively because they have all the relevant discussion in front of them.

This gets back to the same issue as a bug trackers --- the information
has to be managed or it just becomes a dumping ground, and who is going
to do that if the community can't even comment on some patches.

---

From: Bruce Momjian <[email-masked]>
Date: Mon, 30 Apr 2007 07:28:51 -0400 (EDT)

> pgAdmin and friends enough head room to catch up.

This is what already happens, and if it doesn't happen sometimes, it is
just because people are too busy.  Making arbitrary deadlines isn't
going to help, partly because we have little control over our community.

---

From: Bruce Momjian <[email-masked]>
Date: Mon, 30 Apr 2007 07:29:54 -0400 (EDT)

> could make that sort of approach manageable.

I am still unclear how the patch would get into such a system, and how
we would add comments, apply, and later remove it, without causing us
even more work.

---

From: Bruce Momjian <[email-masked]>
Date: Mon, 30 Apr 2007 07:31:45 -0400 (EDT)

Every patch in the queue is ready for review.  If we have bounced
something back for the author to fix, it gets removed from the queue. 

As far as adding comments, again, this wasn't meant to be a community
resource, just my tracking system, and I have given feedback on the
status to any committers who wants to know about a patch.

---------------------------------------------------------------------------

---

From: Andrew Dunstan <[email-masked]>
Date: Mon, 30 Apr 2007 07:54:40 -0400

Marc G. Fournier wrote
> a. why 2 months?  shouldn't they be given priority, period?

Yes, but they don't always seem to get it.

> b. what happens after 2 months?  we delete them?

Of course not. What I am suggesting is that we set ourselves review 
targets. The if we start to get in danger of missing them the authors 
and/or anyone else can legitimately start ringing alarm bells.

cheers

andrew

---

From: Dave Page <[email-masked]>
Date: Mon, 30 Apr 2007 12:56:35 +0100

> we would add comments, apply, and later remove it, without causing us

In my original message I described my thinking:

- Developer submits patch, with additional info through a web interface.

- The web interface formats an email containing the patch description,
patch and any other info entered, assigns it a patch number, and
forwards it to the patches list.

- Discussion ensues on list as per normal. The tracking system monitors
the list and automatically attaches any posts to the patch that have the
appropriate reference in the subject line.

- Community members and committers can review the entire discussion
through the systems web interface. Updated patches could be submitted
online.

- Committers log into the system when necessary to alter the status
(committed, rejected, awaiting revision, under review etc), or the queue
name (8.3, 8.4 etc). This could also be done automagically via email
keywords from specific addresses.

You would no longer need to manually manage the queue, and the
committers would simply need to tweak the status flag as required.

Regards, Dave

---

From: Dave Page <[email-masked]>
Date: Mon, 30 Apr 2007 13:00:31 +0100

Bruce Momjian wrote:
> queue.  

This what I propose to automate.

> It would be interesting to put comments on the patches

Any out of band discussion (between you and Tom for example) would
presumably be summarised on list anyway by one or other of you. There
would be no reason I could see to need to be any more candid than to
reject a patch outright, giving a list of reasons why - all of which can
and should be public. If you feel a need to vent about the author for
any other reason, well, thats why we have pubs in the UK :-).

Regards, Dave

---

From: Zdenek Kotala <[email-masked]>
Date: Mon, 30 Apr 2007 14:59:51 +0200

> committers would simply need to tweak the status flag as required.

You can look on Apache Derby project. I think everything what you 
mentioned there Derby project is using.

See http://db.apache.org/derby/

		Zdenek

---

From: Zdenek Kotala <[email-masked]>
Date: Mon, 30 Apr 2007 15:10:31 +0200

> complete one) would be like shipping SQL Server without Enterprise Manager.

And also from another point of view Postgres and related version of 
PgAdmin must fit in same release cycle windows of OS distributions. When 
OS release is out new feature is not usually accepted and OS should be 
shipped with old version pgAdmin.

And bigger problem then new feature in pgAdmin is 
pg_upgrade/pg_migrator. Upgrade procedure must be finished at same time 
as new release, but some upgrade functions are possible coding only 
after feature freeze or when all affected patches are committed.

If we want to have this feature (upgrade) in postgres we would adjust 
release cycle anyway.

		Zdenek

---

From: Magnus Hagander <[email-masked]>
Date: Mon, 30 Apr 2007 15:17:36 +0200

> patches, I wouldn't have to add anything to the queue.

I think that summarises the problem with it fairly well. It was never
designde to be a community thing. But we need a community thing now (we
didn't at the time, probably). Oh, and you are of course no less community
than anybody else ;-)

//Magnus

---

From: Josh Berkus <[email-masked]>
Date: Mon, 30 Apr 2007 10:04:25 -0700

Heikki,

> schedule from Autumn to Spring. That would get us back to releasing in

Er, no.  We wanted to change the cycle to avoid having Feature Freeze occur at 
midsummer (N. hemisphere) when many of our committers are unavailable due to 
conferences or vacation.  

-----------

Question #559: Would changing version control systems help us on this at all?  
I'm specifically thinking of preventing bitrot; would using a decentralized 
VCS allow patch authors to easily prevent bitrot on their own?

-----------

I do like the idea of a web management interface for patches.  It has a number 
of additional advantages:

-- Advocacy volunteers would know what's under development and thus what they 
can talk about at user groups

-- Advanced users who are interested in a specific patch could download that 
patch early, test it for their own applications, and supply feedback to the 
community even before feature freeze.

-- A more organized queue would make backporting by the backports project 
easier.

-- We could save the patches by applied date and index them, and then have a 
place to point users when they ask: "When was X fixed?  Do I *have* to 
upgrade to 8.1 or just 8.0?"

-- It would make it easier to manage Google Summer of Code students and their 
work.

-- The status of a partially complete patch abandoned by its author would be 
much clearer and thus more likely to get picked up by someone else.

-- The patch manager could eventually be integrated with the Buildfarm to do 
automated patch testing.

Overall, I think this would be an excellent direction to move for 8.4.  As web 
applications go, it doesn't even sound hard; I think I could write it if I 
weren't on airplanes all the time.

-- 
Josh Berkus
PostgreSQL @ Sun
San Francisco

---

From: Bruce Momjian <[email-masked]>
Date: Mon, 30 Apr 2007 18:25:04 -0400 (EDT)

> committers would simply need to tweak the status flag as required.

Sounds interesting, but I am not sure how that is going to track
multiple versions of the patch, or changes in the email subject.

The bottom line is that there is a lot of thinking that the patch queue
is so large because no one knows what to do.  "Oh, if we were better
communicators, more would be done".  The patch queue is large because we
have lots of March 31 patches, and because we don't have enough people
to review them quickly.

The people who have expressed interest in reviewing patches already know
where we stand on the patches, and a status email of where we are each
patch will be posted shortly.  I just can't do it this time because I am
traveling.

If you want to try a tracking system, go ahead, just pull from the
patches email list, and somehow try to grab discussion from
hackers/patches on this patches, and give a way to manually update the
patch status via a web site.  If your system works, I will not need to
maintain a separate patches queue, but I will keep doing it until we
know the web site idea will work, just in case.

---

From: Dave Page <[email-masked]>
Date: Tue, 01 May 2007 08:56:15 +0100

Bruce Momjian wrote:
> multiple versions of the patch, 

They could easily be submitted through the web interface as revisions to
the original version.

> or changes in the email subject.

We'd need to keep the reference number in there, but other than that it
could change. We could also examine the in-reply-to header of every
email and attach based on that, but I'm not sure that's necessary.

> have lots of March 31 patches, and because we don't have enough people

Thats is true, but there are also patches in there that have been there
for quite some time adn have had little or no feedback. Consider
Heikki's grouped index items patch
(http://momjian.us/mhonarc/patches/msg00032.html). That thread starts on
7th March when he posted an update because the old one had bitrotted.
There are six messages in the thread on the patch queue, four of which
say "message no available", and the last means little without the
preceeding messages.

When a committer comes to look at this, if he's not sure of something
he's going to be wasting time searching the archives to find all the
different thread fragments that make up the various discussions on the
topic, and those related to each individual revision of the patch. That
has got to be part of what makes reviewing patches both hard, and
uninteresting work. If we can make it easier and quicker, even with just
the current committers we might have found that one of you were able
(and keen) to review much more quickly.

> patch will be posted shortly.  I just can't do it this time because I am

This is precisely the sort of trivial task that I believe we can make
much easier for you, if not eliminate altogether.

> patch status via a web site.  

OK, I will give it some thought. Obviously I'm not going to do much more
before release though.

> If your system works, I will not need to

I would expect nothing less :-)

Regards, Dave.

---

From: Jim Nasby <[email-masked]>
Date: Tue, 1 May 2007 13:27:40 +0100

Two more ideas for the manager, now that we seem to have consensus to  
build one.

> upgrade to 8.1 or just 8.0?"

This should also make doing the release notes substantially easier,  
though since there's probably some stuff that wouldn't go through the  
patch queue we'd want commits from the patch queue to be marked  
differently.

That brings up another idea for the patch management webapp...  
presumably it could handle most/all of the process of actually  
committing a patch. Granted, that's not a huge amount of work, but  
it's silly to have committers do by hand that hich could be automated.
--
Jim Nasby                                            [email-masked]
EnterpriseDB      http://enterprisedb.com      512.569.9461 (cell)

---

From: Bruce Momjian <[email-masked]>
Date: Tue, 1 May 2007 10:52:25 -0400 (EDT)

> (and keen) to review much more quickly.

The bottom line is if you had a system that was 100% perfect in
capturing all information about a patch, it only helps us 2% toward
reviewing the patch, and what is the cost of keeping 100% information?

---

From: Dave Page <[email-masked]>
Date: Tue, 01 May 2007 16:26:03 +0100

> reviewing the patch, and what is the cost of keeping 100% information?

2% for you or Tom reviewing a recently discussed, run-of-the mill patch. 
I suspect that %age will rise as the patch complexity increases and the 
reviewers experience decreases - which is exactly the situation that it 
would help to improve.

Also note that I'm not saying I can produce a system that's 100% correct 
- just one that will capture the posts that keep the patch ID in their 
subject line *automatically* - meaning you don't have to worry about 
keeping threads for the existing queue or tracking the patch status.

Regards Dave

---

From: Josh Berkus <[email-masked]>
Date: Tue, 1 May 2007 09:43:19 -0700

Bruce, 

> reviewers experience decreases - which is exactly the situation that it

Moreover, what I'm looking for is tools which will:
1) allow existing reviewers to make better use of the time that they have, and
2) encourage/assist new reviewers in helping out, and
3) not bottleneck on the availability of a single project member

The current patch-queue process is failing to scale with the project: every 
release it gets to be more work for you & Tom to integrate the patches.  We 
need to think of new approaches to make the review process scale.  As a 
pointed example, you're about to go on tour for 2 weeks and patch review will 
stall while you're gone.  That's not sustainable.

If you don't think that a web tool will help, then what *do* you think will 
help?  Just "soldiering on" isn't really an answer, and I notice that you're 
very quick to come up with reasons why anything we might try will fail, but 
extremely reluctant to make suggestions for improvement.

==============

Dave,

> keeping threads for the existing queue or tracking the patch status.

Is there a reason why the system needs to be primarily based on e-mail?  I was 
thinking that the patch manager would be entirely a web tool, with people 
submitting and modifying a patch directly through a web interface.  This 
would be lots easier to build than an e-mail based system, and also far more 
useful from a monitoring standpoint.  I've worked with e-mail based systems 
like RT and OTRS, and frankly they're extremely high-maintenance and suffer a 
large amount of "lost" information.

We could also build a number of other things into the web tool, like a "You 
are submitting this patch under BSD" disclaimer and pointers to the Developer 
FAQ and other relevant documents.

-- 
Josh Berkus
PostgreSQL @ Sun
San Francisco

---

From: Dave Page <[email-masked]>
Date: Tue, 01 May 2007 18:21:55 +0100

Josh,

> large amount of "lost" information.

The reason for basing the system on email is simply that it minimises 
the changes required in the community process. If it were entirely web 
based, we'd have to change the way we all work to discuss patches in a 
forum style, rather than a list style. I have a sneaking suspicion that 
at least one of our most valued contributors might object to that.

As long as the patch were initially submitted through the web interface 
so that it got assigned an ID, we could automatically track the initial, 
and followup threads on any of the lists as long as the ID is retained 
in the subject line.

> FAQ and other relevant documents.

Oh for sure. We could even do silly stuff like try to automatically 
determine if the patch is in diff -c format ;-)

Regards, Dave

---

From: Josh Berkus <[email-masked]>
Date: Tue, 1 May 2007 10:55:43 -0700

Dave,

> at least one of our most valued contributors might object to that.

Hmmm, yeah, I can see.  However, I think it's possible to separate the patch 
discussion from patch submission; that is, to submit/edit/update patch code 
through the interface, but to do the discussion either through e-mail or the 
interface.  

> and followup threads on any of the lists as long as the ID is retained

Yes, and any changes to the patch code itself, as well.  My concern with 
making the tool e-mail centric is that, based on e-mail based tools I've 
worked with, I'm afraid that the tool will be clunky/buggy enough not to be 
an improvement over the current process -- too much of e-mail format is 
optional varies by MUA.  If e-mail is limited to commentary, though, I don't 
see that as a problem.  And, of course, it would be easy for the tool to send 
e-mail to pgsql-patches.

If many people are going to block on using a web tool for submitting new 
versions of a patch, claiming responsibility for review, etc., though, then 
we should probably abandon this discussion right here. No new tool is going 
to work if we have people who won't make any changes at all in their work 
habits.

> determine if the patch is in diff -c format ;-)

No!  Dave, you're a real radical sometimes.  

-- 
Josh Berkus
PostgreSQL @ Sun
San Francisco

---

From: "Simon Riggs" <[email-masked]>
Date: Tue, 01 May 2007 19:09:50 +0100

> stall while you're gone.  That's not sustainable.

This seems a reasonable observation on events.

I'll come up with ideas to help, if asked, but I'd like to follow a
proposal from Core on this issue.

---

From: Andrew Dunstan <[email-masked]>
Date: Tue, 01 May 2007 16:10:07 -0400

> to work if we have people who won't make any changes at all in their work 

We could do most of what people want using a tracker, but that 
discussion always seems to end up nowhere. And, as I have noted before, 
use of a tracker will become a total mess unless there are adequate 
resources to keep it clean and up to date (e.g. to put links in items to 
mailing list discussions, update state etc.) So if the commercial 
backers of PostgreSQL want better management of the project, maybe they 
need to find some resources to help out.

cheers

andrew

---

From: Magnus Hagander <[email-masked]>
Date: Tue, 01 May 2007 22:23:07 +0200

> through the interface, but to do the discussion either through e-mail or the 

I'd just keep all discussion in email, no need to do that off the web.
As long as you can *read* all the references on the web.

> see that as a problem.  And, of course, it would be easy for the tool to send 

I believe that's what Dave is proposing. If we want to implement
something like "reviewer signoff" (which we'd at least eventually want
to do, IMHO), doing that through the web interface primarily (for a nice
"tally" table) is probably a good start - it's a lot easier than parsing
emails.

//Magnus

---

From: Josh Berkus <[email-masked]>
Date: Tue, 1 May 2007 13:31:38 -0700

Andrew,

> So if the commercial

I don't think they really care, or we'd have heard something by now.  I 
think this is up to us PG developers.

-- 
--Josh

Josh Berkus
PostgreSQL @ Sun
San Francisco

---

From: Andrew Dunstan <[email-masked]>
Date: Tue, 01 May 2007 16:42:30 -0400

> think this is up to us PG developers.

Well, I have no confidence that any formal system will succeed without 
someone trusted by core and committers stepping up to the plate to do 
the required ongoing legwork.

As for voting on patches, that seems a most un-postgres-like way of 
doing things. What is more, it assumes that multiple people will be 
reviewing patches. Our trouble right now is finding even one qualified 
reviewer with enough time for some patches.

cheers

andrew

---

From: Bruce Momjian <[email-masked]>
Date: Tue, 1 May 2007 19:09:24 -0400 (EDT)

> stall while you're gone.  That's not sustainable.

I am traveling --- I left on Friday.  I am in Sydney now.

As far as scaling, patch information isn't our problem right now.  If
someone wants to help we can give them up-to-date information on exactly
how to deal with the patch.  It is people doing the work that is the
bottleneck.

> extremely reluctant to make suggestions for improvement.

Well, if I had something better, I would be doing it already.

There isn't an improved solution to every problem.

We have this discussion regularly --- things aren't going well, so there
must be a better way.  I know things aren't going well because I created
this thread, but I don't see collecting patch information as solving
this issue.  What we actually need are more people doing things,
"soldiering on".

As an example, how is patch information going to help us review HOT or
group-item-index?  There is frankly more information about these in the
archives than someone could reasonable read.  What someone needs is a
summary of where we are now on the patches, and lots of time.

FYI, Tom, Heikki, I need one of you to post the list of patches and
where we think we are on each one, even if the list is imperfect.

---

From: Bruce Momjian <[email-masked]>
Date: Tue, 1 May 2007 19:11:09 -0400 (EDT)

> reviewer with enough time for some patches.

The typical use-case is that someone is going to like the patch, but
what X changed in it, so a simple vote isn't going to work, and neither
is automatic patch application.  Rarely is a patch applied unmodified by
the applier.

---

From: Arturo Perez <[email-masked]>
Date: Tue, 01 May 2007 19:46:26 -0400

> Two more ideas for the manager, now that we seem to have consensus to  

One other thing a webapp would allow that would help grow the community.  
If the patches are all in a public place then reviewer wannabees can get 
their feet wet relatively easily.  

Some may argue this is already possible but I, personally, don't even 
know where to look for patches.

-arturo

---

From: Josh Berkus <[email-masked]>
Date: Tue, 1 May 2007 21:51:51 -0700

Bruce,

> summary of where we are now on the patches, and lots of time.

The idea is to provide ways for other people to help where they can and to 
provide better feedback to patch submitters so that they fix their own issues 
faster.  Also, lesser PostgreSQL hackers than you could take on reviewing the 
"small" patches, leaving you to devote all of your attention to the "big" 
patches.

Actually, that can happen with the current system. The real blocker there is 
that some people, particularly Tom, work so fast that there's no chance for a 
new reviewer to tackle the easy stuff.  Maybe the real solution is to 
encourage some of our other contributors to get their feet wet with easy 
patches so that they can help with the big ones later on?

That is, if the problem is people and not tools, then what are we doing to 
train up the people we need?

-- 
Josh Berkus
PostgreSQL @ Sun
San Francisco

---

From: Tom Lane <[email-masked]>
Date: Wed, 02 May 2007 01:19:45 -0400

> patches so that they can help with the big ones later on?

Yeah, I hear what you say.  This is particularly a problem for small bug
fixes: I tend to zing small bugs quickly, first because I enjoy finding/
fixing them and second because I worry that they'll fall off the radar
screen if not fixed.  But I am well aware that fixing those sorts of
issues is a great way to learn your way around the code (I think that's
largely how I learned whatever I know about Postgres).  I'd be more
willing to stand aside and let someone else do it if I had confidence
that issues wouldn't get forgotten.  So in a roundabout way we come back
to the idea that we need a bug tracker (NOT a patch tracker), plus
people putting in the effort to make sure it stays a valid source
of up-to-date info.  Without the latter it won't really be useful.

			regards, tom lane

---

From: Magnus Hagander <[email-masked]>
Date: Wed, 2 May 2007 09:34:38 +0200

<snip>

> where we think we are on each one, even if the list is imperfect.

I think you just contradicted yourself. Information isn ot the problem, but
you need more information...

I think this is a fairly clear indication that we do need a better way to
track this information.

//Magnus

---

From: Heikki Linnakangas <[email-masked]>
Date: Wed, 02 May 2007 09:23:49 +0100

> of up-to-date info.  Without the latter it won't really be useful.

A great way to learn would be to look at the patches in the queue, and 
find bugs in them. There's a lot more bugs to be found in submitted 
patches than in PostgreSQL itself. A patch tracker would help with that.

I'm in favor of some kind of a patch tracker. It doesn't need to be too 
fancy, but if for each patch we had:

Patch name:	Kitchen sink addition to planner
Latest patch:	kitchen-sink-v123.patch, click to download
Summary:	Adds a kitchen-sink node type to the planner to enable liquid 
queries.
Status:		Will be rejected unless race conditions are fixed. Needs 
performance testing.
Discussions:	<links to mail threads, like in the current patch queue>

That wouldn't necessarily help committers directly, but it'd give more 
visibility to the patches. That would encourage more people to review 
and test patches. And it'd make it clear what the status of all the 
patches are to anyone who's interested.

---

From: Bruce Momjian <[email-masked]>
Date: Wed, 2 May 2007 06:44:12 -0400 (EDT)

> I think this is a fairly clear indication that we do need a better way to

No, my point is that 100% information is already available by looking at
email archives.  What we need is a short description of where we are on
each patch --- that is a manual process, not something that can be
automated.

Tom has posted it --- tell me how we will get such a list in an
automated manner.

---

From: Bruce Momjian <[email-masked]>
Date: Wed, 2 May 2007 06:46:45 -0400 (EDT)

> train up the people we need?

We seem to handle trivial patches just fine.  The current problem is
that the remaining patches require domain or subsystem-specific
knowledge to apply, e.g. XML or WAL, and those skills are available in a
limited number of people.  If I had the expertise in those areas, I
would have applied the patches already.

---

From: Gregory Stark <[email-masked]>
Date: Wed, 02 May 2007 12:59:27 +0100

"Bruce Momjian" <[email-masked]> writes:

> We seem to handle trivial patches just fine.  

You keep saying that but I think it's wrong. There are trivial patches that
were submitted last year that are still sitting in the queue.

In fact I claim we handle complex patches better than trivial ones. HOT, LDC,
DSM etc receive tons of feedback and acquire a momentum of their own.
Admittedly GII is a counter-example though.

On the other hand trivial patches tend to interest relatively few people and
have little urgency.

> The current problem is that the remaining patches require domain or
> subsystem-specific knowledge to apply, e.g. XML or WAL, and those skills are
> available in a limited number of people. If I had the expertise in those
> areas, I would have applied the patches already.

Well, I claim it's often the trivial patches that require the domain-specific
knowledge you describe. If they were major patches they would touch more parts
of the system. But that means they should be easy to commit if you could just
fill in the missing knowledge.

Could you pick a non-committer with the domain-specific knowledge you think a
patch needs and ask for their analysis of the patch then commit it yourself?
You can still review it for general code quality and trust the non-committer's
review of whether the domain-specific change is correct.

---

From: Bruce Momjian <[email-masked]>
Date: Wed, 2 May 2007 08:11:12 -0400 (EDT)

> were submitted last year that are still sitting in the queue.

You seem to be looking at something different than me.  Which patches?

> review of whether the domain-specific change is correct.

We are already pushing out patches to people with domain-specific
knowledge.  Tom posted that summary today.

---

From: Andrew Dunstan <[email-masked]>
Date: Wed, 02 May 2007 08:33:32 -0400

Tom Lane wrote:
> So in a roundabout way we come back

Hallelujah Brother!

BTW, a bug tracker can be used as a patch tracker, although the reverse 
isn't true. For example, the BZ people use BZ that way, in fact - most 
patches arrive as attachments to bugs. And trackers can be used just as 
well for tracking features as well as bugs.

cheers

andrew

---

From: Chris Browne <[email-masked]>
Date: Wed, 02 May 2007 13:27:00 +0000

> Hallelujah Brother!
>
> BTW, a bug tracker can be used as a patch tracker, although the
> reverse isn't true. For example, the BZ people use BZ that way, in
> fact - most patches arrive as attachments to bugs. And trackers can be
> used just as well for tracking features as well as bugs.

Well, Command Prompt set up a Bugzilla instance specifically so people
could track PG bugs.  If only someone took interest and started using
it...
-- 
"cbbrowne","@","linuxfinances.info"
http://cbbrowne.com/info/lisp.html
Do Roman paramedics refer to IV's as "4's"?

---

From: Alvaro Herrera <[email-masked]>
Date: Wed, 2 May 2007 09:28:29 -0400

> >of up-to-date info.  Without the latter it won't really be useful.

Amen

> well for tracking features as well as bugs.

The pidgin (previously known as Gaim) guys also use it that way.  They
add a bug for each thing they want to change, even new features, and
track the patches in there.  Then they have a list of issues that should
be solved for each release, so it's easy to see which ones are still
missing using their Trac interface.

http://developer.pidgin.im/roadmap

So the status email that Tom sent yesterday would be a very simple thing
to generate, just looking at the "bugs to fix" page.

I'm not saying we should use Trac, mainly because I hate how it
(doesn't) interact with email.  But it does say that a bug tracker can
be useful to us.

-- 
Alvaro Herrera                                http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

---

From: Magnus Hagander <[email-masked]>
Date: Wed, 2 May 2007 19:37:14 +0200

On Wed, May 02, 2007 at 06:44:12AM -0400, Bruce Momjian wrote:
> email archives.

Yes, and hard to find.

>  What we need is a short description of where we are on

Right. But it can be presented in a central way and incrementally updated.

> Tom has posted it --- tell me how we will get such a list in an

There's no way to get it automated, but you can get it incrementally
updated.

//Magnus

---

From: Josh Berkus <[email-masked]>
Date: Wed, 2 May 2007 11:09:01 -0700

Bruce, all,

> Tom has posted it --- tell me how we will get such a list in an

Several of us have already suggested a method.  If we want the information to 
be up-to-date, then the patch manager, or bug tracker, needs to be a required 
part of the approval & application process, NOT an optional accessory.  That 
is, if patches & bug fixes can come in, get modified, get approved & applied 
entirely on pgsql-patches or pgsql-bugs without ever touching the tracker 
tool, then the tracker tool will be permanently out of date and useless.

It's going to require the people who are doing the majority of the bug hunting 
& patch review to change the way they work, with the idea that any extra time 
associated with the new tool will be offset by being able to spread the work 
more and having information easy to find later, for you as well as others.  
Tom seems to be willing; are you?

====================

> Discussions:    <links to mail threads, like in the current patch queue>

... this brings up another reason we could use a tracker.  I now have access 
to a performance testing lab and staff.  However, these people are NOT going 
to follow 3 different high-traffic mailing lists in order to keep up with 
which patches to test.  As a result, they haven't done much testing of 8.3 
patches; they're depenant on me to keep them updated on new patch versions 
and known issues and I'm on the road a lot.

If I had a web tool I could point them to where they could simply download the 
current version of the patch, test, and comment a report, we'd get a LOT more 
useful performance feedback from Sun.  I suspect the same is true of Unisys.

-- 
Josh Berkus
PostgreSQL @ Sun
San Francisco

---

From: "Joshua D. Drake" <[email-masked]>
Date: Wed, 02 May 2007 11:27:24 -0700

> Tom seems to be willing; are you?

Hello,

Well according to himself the last time this came up:

http://archives.postgresql.org/pgsql-hackers/2006-08/msg01253.php

No, he isn't.

Sincerely,

Joshua D. Drake

-- 

       === The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive  PostgreSQL solutions since 1997
              http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL Replication: http://www.commandprompt.com/products/

---

From: "Andrew Dunstan" <[email-masked]>
Date: Wed, 2 May 2007 15:59:35 -0400 (EDT)

> could track PG bugs.  If only someone took interest and started using

I lost interest last time around when it seemed clear to me that there was
not enough traction.

Maybe there is this time around.

The effort Tom mentions above is crucial to success.

cheers

andrew

---

From: Dave Page <[email-masked]>
Date: Wed, 02 May 2007 21:32:19 +0100

> http://archives.postgresql.org/pgsql-hackers/2006-08/msg01253.php

The last paragraph of 
http://archives.postgresql.org/pgsql-hackers/2007-04/msg01258.php is 
somewhat more positive regarding a patch tracker.

Regards, Dave

---

From: Tom Lane <[email-masked]>
Date: Thu, 03 May 2007 01:30:39 -0400

> were submitted last year that are still sitting in the queue.

Um ... which ones exactly?  I don't see *anything* in the current queue
that is utterly without issues, other than Heikki's ReadOrZeroBuffer
patch which certainly doesn't date from last year (and besides, has
now been applied).

I'm a bit more worried about stuff that may have slid through the
cracks, like the Darwin SysV-vs-Posix-semaphores patch that I complained
of in my triage listing.  But the stuff that is listed on Bruce's patch
queue page is not "trivial".  It's either large or has got problems.

			regards, tom lane

---

From: Gregory Stark <[email-masked]>
Date: Thu, 03 May 2007 12:31:26 +0100

"Tom Lane" <[email-masked]> writes:

> that is utterly without issues

I didn't mean they were without issues. Only that they weren't complex patches
requiring broad knowledge of many parts of the system.

Anyways, I think we're well on our way to improving the situation so I don't
want to drag out my negativity any longer.

---

From: Bruce Momjian <[email-masked]>
Date: Thu, 3 May 2007 07:32:12 -0400 (EDT)

> Tom seems to be willing; are you?

No, I think it will be more work than benefit, and Tom and I will still
be doing the bulk of it, so we will have something pretty but more work
for people who are already too busy.

This is so funny, I don't know how to respond, only to ask if Dtrace  
will help us here.  ;-)  I admit having such a system would improve the
chances of commercial help by a miniscule percentage, but the cost to
patch submitters/managers is so high in proportion to be humorous.

We have _ample_ evidence that the problem is lack of people able to
review patches, and yet there is this discussion to track patches
better.  It reminds me of someone who has lost their keys in an alley,
but is looking for them in the street because the light is better there.

To be more concrete, during normal development, we seem to have no
problem keeping track of patches, so it is only during feature freeze
that it is a problem.  Now, everyone admits that having patches tracked
on a web interface is going to be more work for patch managers and
importantly also patch submitters.  So do we do the web work just so we
can have a more orderly feature freeze, or do we just accept that Tom is
going to have to post a summary of where we are on patches periodically
during feature freeze and not do the web work for every patch?

I have added a link on the patches queue so you can download the emails
in mbox format.  If someone wants to take that and make a nice web site
out of it and keep that up-to-date during our feature freeze period,
that would probably help. (I can even point the patch queue to a new
URL.)  If no one is willing to do it, I question how much help we would
get with a web patch system.

Heck, if I was around, I would throw up a txt file on the web (using
txt2html) with the status of each patch and keep it current.  I think I
have done that for some past releases.  Why doesn't someone do that and
see how it goes?  Thanks to Tom, you know have a current status of each
patch and who is supposed to work on it.

I am not anti-big-solution, but I don't want a big solution if it isn't
going to help.  If no one is doing even a trivial solution, I don't want
to try a big one.

Get rid of gborg and let's talk.

Why am I having to spend hours in Syndey saying the same thing?  Why
don't you guys go ahead and change things, and when they fail, I will
still be around.

---

From: Csaba Nagy <[email-masked]>
Date: Thu, 03 May 2007 13:47:06 +0200

> but is looking for them in the street because the light is better there.

Bruce, I guess the analogy fails on the fact that you're not looking for
a key, but for people, and I thought "better light" will attract people
to find you instead of you to need to search...

I'm an outsider regarding postgres development, so my opinion does not
count, but the gut feeling is that more light attracts better people.
Not to mention it can help people get better...

Cheers,
Csaba.

---

From: Bruce Momjian <[email-masked]>
Date: Thu, 3 May 2007 07:51:18 -0400 (EDT)

> to find you instead of you to need to search...

I believe the problem is not that there isn't enough information, but
not enough people able to do the work.  Seeking solutions in areas that
aren't helping was the illustration.

---

From: Csaba Nagy <[email-masked]>
Date: Thu, 03 May 2007 13:59:32 +0200

> aren't helping was the illustration.

Yes Bruce, but you're failing to see that a more structured information
infrastructure will attract more people to do the work which could
eventually solve the problem you're having in the first place
(contradicting your argument that it won't help), at the cost of some
possible additional work (which I actually think you're overestimating
the amount of - it's probably more like a learning curve than actual
additional work).

The fact that there is enough information is not relevant, it must be in
the right place too - too much information or hard to find information
is sometimes worst than no information.

Cheers,
Csaba.

---

From: Dave Page <[email-masked]>
Date: Thu, 03 May 2007 13:01:00 +0100

> aren't helping was the illustration.

OK, so how do we attract more people if not by making the job easier for 
them? It's not like there aren't plenty of very clever people here - we 
just need to encourage them to chip in - and making the task less 
daunting by making *all* the information they need readily available 
seems a good start too me.

And heck, we're database people - storing and retrieving the data to do 
a job is exactly what we do!

Regards, Dave.

---

From: Andrew Dunstan <[email-masked]>
Date: Thu, 03 May 2007 10:28:47 -0400

> aren't helping was the illustration.

There are multiple problems.

Of course no amount of technology will provide us with a bigger pool of 
qualified reviewers. That doesn't mean our present management methods 
are good - you seem to be just about the only person left who thinks 
they are.  Moving from using a quill to using a fountain pen or even a 
ballpoint doesn't mean we have more writers or better writers. But it 
does make writing easier and is therefore a good thing to do.

cheers

andrew

---

From: "Joshua D. Drake" <[email-masked]>
Date: Thu, 03 May 2007 09:43:29 -0700

> aren't helping was the illustration.

*sigh* Bruce, why is this so hard with you? Let's try this a different way.

In the beginning there was Archie. Archie was good but only people that 
were intimately familiar with its use got much benefit.

Then there was Gopher. He was cool, got Bill Murray to do really bad 
things to a golf course but he provided more structured information if 
in a strictly hierarchical fashion. Again it was used, but only by those 
in the know.

The came the World Wide Web. A fascinated World Wide Portal, with no 
map... It brought, Yahoo!, Alta Vista and Lycos.

All of a sudden, anyone could find what they were looking for and in the 
process more people joined the community.

The rest is history (and I think my timeline between archie and gopher 
might be off) but the point is, everything grows up and reach a point 
where they need a more structured, easy to find, easy to collaborate 
method of processing data. Any data, not just patches or bugs.

You are stuck in Gopher land, I don't know why but it is at is is. 
Perhaps it is type to at least jump to using Yahoo!?

Sincerely,

Joshua D. Drake

--

---

From: Josh Berkus <[email-masked]>
Date: Thu, 3 May 2007 10:26:43 -0700

Bruce,

> Get rid of gborg and let's talk.

Touche'.

Actually, AFAICT, the only active thing left on GBorg is WWW.  If we move 
that, we can shut it down.  Any objections?

> don't you guys go ahead and change things, and when they fail, I will

You're acting as a majority of one, here, Bruce.  The reason why any solution 
anyone else tries *will* fail is because you will refuse to participate in 
it.  That's why people are trying to persuade you instead of just going ahead 
and doing it; you have the power to effectively sandbag anything anyone else 
does, so that's why nobody wants to put effort into it if you're opposed.

-- 
Josh Berkus
PostgreSQL @ Sun
San Francisco

---

From: "Joshua D. Drake" <[email-masked]>
Date: Thu, 03 May 2007 10:47:48 -0700

> that, we can shut it down.  Any objections?

This should be a different thread *but* to my knowledge there is more 
than WWW active on Gborg. Or at least there is some data that people 
still wished moved.

Gborg needs to be shutdown in a scheduled manner. I proposed a schedule 
last year but it was largely overlooked:

http://archives.postgresql.org/pgsql-general/2006-08/msg01167.php

My rather bold attempt to reach all people associated with Gborg was not 
met with positive responses either (granted due to a mistake on my end).

http://people.planetpostgresql.org/joshua/index.php?/archives/15-Blacklisting-PostgreSQL.Org,-Spam-and-hordes-of-geeks-Oh-My!.html

> does, so that's why nobody wants to put effort into it if you're opposed.

That pretty much sums it up. Bruce you can either come to the party with 
a smile or a grin. It is your party. If you come with a smile everyone 
will be a lot happier.

Sincerely,

Joshua D. Drake

--

---

From: "Marc G. Fournier" <[email-masked]>
Date: Thu, 03 May 2007 15:20:16 -0300

Why not just send a notice out stated that Gborg will be shutdown as of June 
1st ... give a finite deadline to move things over to pgfoundry ... just 
because we 'shut down' the site on June 1st, it doesn't mean we are going to 
wipe it all out, we can just put a Redirect on the web server on gborg over to 
pgfoundry so that ppl can't go *to* gborg's web site ... we can also make the 
CVS 'read-only', so that developers can't update the CVS there, but ppl can 
still download the code ...

- --On Thursday, May 03, 2007 10:47:48 -0700 "Joshua D. Drake" 
<[email-masked]> wrote:

> This should be a different thread *but* to my knowledge there is more than
> WWW active on Gborg. Or at least there is some data that people still wished
> moved.
>
> Gborg needs to be shutdown in a scheduled manner. I proposed a schedule last
> year but it was largely overlooked:
>
> My rather bold attempt to reach all people associated with Gborg was not met
> with positive responses either (granted due to a mistake on my end).
>
> http://people.planetpostgresql.org/joshua/index.php?/archives/15-Blacklisting
> -PostgreSQL.Org,-Spam-and-hordes-of-geeks-Oh-My!.html
>
>>
>> You're acting as a majority of one, here, Bruce.  The reason why any
>> solution  anyone else tries *will* fail is because you will refuse to
>> participate in  it.  That's why people are trying to persuade you instead of
>> just going ahead  and doing it; you have the power to effectively sandbag
>> anything anyone else  does, so that's why nobody wants to put effort into it
>> if you're opposed.
>
>
> That pretty much sums it up. Bruce you can either come to the party with a
> smile or a grin. It is your party. If you come with a smile everyone will be
> a lot happier.
>
>
> Sincerely,
>
> Joshua D. Drake
>
>
>
> --
>
> ---------------------------(end of broadcast)---------------------------
> TIP 9: In versions below 8.0, the planner will ignore your desire to
>        choose an index scan if your joining column's datatypes do not
>        match

- ----

---

From: Chris Ryan <[email-masked]>
Date: Thu, 3 May 2007 12:12:20 -0700 (PDT)

I was just getting ready to suggest such an approach. We could
email all the project admins for the reamaining projects with the
dead-line. Backup the information and tell people who to contact in
order to claim whatever information they want. Once the dead-line is
past you can simply shutdown all the gborg services. Those who don't
claim their information either have already moved someplace else or
don't care about what is on gborg anymore.

   Additionally if it were desired we could place tarballs of the
cvsrepositories and whatever download files where uploaded for each
project on some ftp server if anyone was interested in preserving that
ifnormation for posterity.

   It would not be difficult for me to get a list of email addresses
and project names of those admins on gborg.

Chris Ryan

--- "Marc G. Fournier" <[email-masked]> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> 
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.5 (FreeBSD)
> 
> iD8DBQFGOifh4QvfyHIvDvMRAvKSAJ9l87k9s8wp4nnw5W4xyCQJIgwNOQCeMLXt
> bOqtGrCOZ8WnbrhAdYKOGKY=
> =vJmp
> -----END PGP SIGNATURE-----
> 
> 

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com

---

From: Robert Treat <[email-masked]>
Date: Fri, 04 May 2007 14:24:21 -0400

On Wednesday 02 May 2007 01:19, Tom Lane wrote:
> > Actually, that can happen with the current system. The real blocker there
> > is that some people, particularly Tom, work so fast that there's no
> > chance for a new reviewer to tackle the easy stuff.  Maybe the real
> > solution is to encourage some of our other contributors to get their feet
> > wet with easy patches so that they can help with the big ones later on?
>

Maybe you just need to have a 1 week clock skew when reading pgsql-bugs? 

-- 
Robert Treat
Build A Brighter LAMP :: Linux Apache {middleware} PostgreSQL

---

From: Robert Treat <[email-masked]>
Date: Fri, 04 May 2007 14:46:41 -0400

I think this is the apprach joshua tried the first time and it backfired... I 
think we need a more personal approach.  I'm willing to put time into this if 
people want a new point man (I don't think Joshua will mind, lmk if you do) 
but it will have to wait untill after pgcon.

---

From: Bruce Momjian <[email-masked]>
Date: Fri, 4 May 2007 21:39:16 -0400 (EDT)

> is sometimes worst than no information.

If I can't get help on a simple request of maintaining an 8.3 status web
page, I think more help is unlikely and I am not interested in doing
more work to find out.

---

From: Bruce Momjian <[email-masked]>
Date: Fri, 4 May 2007 21:41:17 -0400 (EDT)

Josh Berkus wrote:
> > Why am I having to spend hours in Syndey saying the same thing? ?Why

People aren't willing to hel pme in even a simple task of maintaining an
8.3 patches status page, so why would they want to help with something
larger.  I am not going to make my job harder only to find out no one
wants to help.

---

From: Bruce Momjian <[email-masked]>
Date: Fri, 4 May 2007 22:00:25 -0400 (EDT)

I have already responded to all the email comments.  Here is my idea of
moving forward.  There are basically three interrelated issues:

1)  bug tracking
2)  getting more people to review complex patches 
3)  patch tracking

I am not going to go into #1, except to say that the problem has always
been the effort needed to track the bugs vs. the value.

I am not going to go into #2, except to say that this is going to require
a personal approach to assisting developers.  One thing I am going to do
is to add an item to the developer's FAQ outlining how patches are analyzed
for application, which should help both patch submitters and committers
(sample attached).

As for #3, again, I don't want us to take on a burdensome patch tracking
process that is more effort than it is worth, and the lack of people
jumping to even manage a simple web page for current 8.3 patches has me
questioning what kind of support a burdensome tracking system would
receive.

What I think we can do simply is to have our email software automatically
number emails submitted to the patches list that already don't have a
number.  This way, all followups, even if moved to the hackers list, would
maintain that patch number, and if an updated version is posted, the user
would keep the same number in the email subject.

Once that is done, it should be trivial to write a web application that
will track the patches based on the subject line, something like
"PATCH#555".  Committers can include that patch string in the commit
message, so committed patches can be automatically removed from the open
patch queue.  The only tricky part is to handle patches that are rejected
or kept for a later release.  That would probably require web access to
adjust.

One thing that has always bothered me about tracking patches is that when
an email to the patches list is bounced for discussion to the hackers
list, there isn't any good way for outside people to track the full patch
discussion because we don't track threads that move to different mailing
lists.  The special patch number would make such tracking easier.

The good thing about such a simple system is that it has a very light
burden on patch submitters and committers.  The major work is done by the
web application, but that can all be programmed.

  + If your life is a hard drive, Christ can be your backup. +
Patch committers check several things before applying a patch:

1)  Follows the SQL standard or community agreed-upon behavior
2)  Style merges seamlessly into the surrounding code
3)  Written as simply and efficiently as possible
4)  Uses the available PostgreSQL subsystems properly
5)  Contains sufficient comments
6)  Contains code that works on all supported operating systems
7)  Has proper documentation
8)  Passes all regression tests
9)  Behaves as expected, even under unusual cirumstances
10)  Contains no reliability risks
11)  Does not overly complicate the source code
12)  If performance-related, it should have a measureable performance benefit
13)  Is of sufficient usefulness to the average PostgreSQL user
14)  Follows existing PostgreSQL coding standards

---

From: Andrew Dunstan <[email-masked]>
Date: Fri, 04 May 2007 22:28:52 -0400

Bruce,

I will say publicly what I have said to others privately. Forgive me if 
I'm a bit blunter than usual. I do not see any value in this at all. 
What we need to track are problems to be solved, be they bugs or 
features, not particular patches. Tracking patches simply comes too late 
in the process.

I think that your attitude to the use of bug/feature trackers is quite 
unreasonable, and certainly in opposition to what seems to be the 
majority opinion among developers. It's a great pity that you are so 
utterly resistant to use of tracking software. The only reason that this 
system, at best a half measure in almost everyone's eyes, is being 
proposed, as far as I can see, is that you will not agree to use 
anything else.

So if this goes ahead and proves to be of little value, I hope that you 
will relent and agree the use of proper tracking software like almost 
every other open source project uses. It really is time that PostgreSQL 
managed to advance beyond thinking that email lists are the greatest 
management tool since sliced bread. It's just indefensible in 2007.

cheers

a disappointed andrew

---

From: Bruce Momjian <[email-masked]>
Date: Fri, 4 May 2007 23:38:23 -0400 (EDT)

> management tool since sliced bread. It's just indefensible in 2007.

As I said before, I am involved in patches only when a patch isn't
addressed.  If a new system works, I will have nothing to do, which is
good.

If you want me to believe it will work better than what we do now, I
can't.  Prove me wrong.  Forget about what I think.  Do something and
stop talking about it.

What I am not going to do is to do 2x more work and get 2% more help,
which is what I fear.

---

From: "Robert Haas" <[email-masked]>
Date: Sat, 5 May 2007 07:57:36 -0400

> larger.  I am not going to make my job harder only to find out no one

I thought about volunteering to do this, but:

1. I am a little warry of inserting myself (as an outsider) into a major
controversy as my first contribution to the project.

2. It seems like it would be difficult or impossible for an outsider to
do this well.  Essentially, I'd have to read every message on -hackers,
-patches, and -committers, and try to figure out which of those messages
amounted to a change in status for which patches, and then update the
status of the patches.

Example: Tom says "what about XYZ?  ISTM this will have to wait for
8.4".  The person who wrote the patch replies with "I think XYZ is not
an issue because of ABC."  It's not clear (at least to me) whether the
patch is now in play for 8.3 again or whether it's still on hold.

In addition, if some discussion is happening via private email (which it
sounds like it is), then this wouldn't be complete even if it were done
perfectly.

I write web-based workflow applications for a living, so in theory I'm
more amenable to the idea of helping out in that way.  But it seems to
me that right now there's no consensus on whether we need this at all,
and if so what it should do.

I don't really want to get involved in the central argument about what
the "right" way of doing this is, but I think Bruce's proposal to put a
patch number in every email that hasn't got one can't possibly be any
worse than what we're doing now, and it might be better, so why not?
I'm even willing help with this if there is consensus on it.

Thanks,

...Robert

---

From: Zdenek Kotala <[email-masked]>
Date: Sat, 05 May 2007 16:24:38 +0200

I would like to add one point:

> 8)  Passes all regression tests

   8.5) Contains regression test(s) which covered performed changes

	Zdenek

---

From: Bruce Momjian <[email-masked]>
Date: Sat, 5 May 2007 10:33:23 -0400 (EDT)

OK, item modified to:

	<li>Passes all regression tests, and if needed, adds new
	ones</li>