Subject: Getting a move on for 8.2 beta

From: Tom Lane <[email-masked]>
Date: Fri, 01 Sep 2006 11:50:54 -0400

(I've already bounced this off the core committee, but it's time for
wider discussion.)

September is upon us and it doesn't seem like we are ready to ship
a beta.  I think it's time to start making some hard choices.

In the main code I see these outstanding features/patches:

* bitmap indexes
* updatable views
* GUC variable reload + refactoring (previously applied and reverted)
* plpython improvements (needs review by someone who knows plpython)
* plpgsql improvements for returning record types
* patch to build on VC
* make libpq default client_encoding setting from locale?

In contrib we've got:

* new ISBN/etc module
* hstore (finally proposed for inclusion)
* new sslinfo module
* pgstattuple changes
* removing the deadwood

These are not all the open issues, for sure, but these are what I think
have to be resolved before we can go beta.  Everything else is in the
category of "bug fixes", and none of it seems like a showstopper (in
fact a lot of the small stuff on my todo list is bugs reported against
8.1, so those certainly are not beta-stoppers).

My feeling is that we ought to bounce bitmap indexes and updatable views
as not being ready, accept all the contrib stuff, and try to get the
other items done in time for a beta at, say, the end of next week.

I'm not thrilled about postponing those two large items till 8.3, but
we are a month past feature freeze and we do not have credible patches
in hand for either.  The alternative is to be willing to slip the
schedule until whenever they are ready, and we have learned the hard way
that that's not good project management.

Comments?

			regards, tom lane

---

From: "Joshua D. Drake" <[email-masked]>
Date: Fri, 01 Sep 2006 11:42:10 -0700

> in hand for either.

I know there is a lot of "backend" work that has been done for 8.2 but 
the two features you are suggesting we bounce, are probably the most 
visible of marketable features we will have for this release. Especially 
updateable views.

> The alternative is to be willing to slip the

Well I don't like the idea either, but if it isn't done, it isn't done. 
How often do we hear the complaint about software that is shipped before 
it is done?

Outside of feature freeze, we really haven't had AFAICS a "solid" 
release schedule. Is it going to be push Beta for a month? Does that 
push us till the end of the year?

I don't know which is the better solution, but you asked for comments :)

Sincerely,

Joshua D. Drake

> 
> Comments?
> 
> 			regards, tom lane
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 3: Have you checked our extensive FAQ?
> 
>                http://www.postgresql.org/docs/faq
> 

-- 

    === The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
    Providing the most comprehensive  PostgreSQL solutions since 1997
              http://www.commandprompt.com/

---

From: Alvaro Herrera <[email-masked]>
Date: Fri, 1 Sep 2006 14:52:05 -0400

> In the main code I see these outstanding features/patches:

IMHO these two should be bounced.  They are big features that still seem
to need a lot of work and will probably take long before they are ready
for inclusion.  Interested parties should continue working on them with
an eye to be included *at the start* of the 8.3 development cycle.

> * hstore (finally proposed for inclusion)

These all seem to be manageable within reasonable time.

I can help with the contrib stuff.

> other items done in time for a beta at, say, the end of next week.

+1.

-- 
Alvaro Herrera                                http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

---

From: Josh Berkus <[email-masked]>
Date: Fri, 1 Sep 2006 12:07:38 -0700

Tom,

> * plpgsql improvements for returning record types

What's VC?

> * hstore (finally proposed for inclusion)

This is just a delete.  I've fixed the pgFoundry permissions issues and 
will be loading the last CVS snapshot today.  At that point, Bruce can 
delete stuff.

Can you be a little clearer on which things are non-working patches, and 
which things are pending due to lack of review?   I'm all for bouncing 
stuff that's not ready to go, but bouncing stuff because we haven't 
recruited enough code reviewers is just bad practice.

-- 
--Josh

Josh Berkus
PostgreSQL @ Sun
San Francisco

---

From: Tom Lane <[email-masked]>
Date: Fri, 01 Sep 2006 15:20:24 -0400

Josh Berkus <[email-masked]> writes:
> which things are pending due to lack of review?

The only things I currently want to reject as not having provided a
timely patch are bitmap indexes and updatable views.  The other stuff
I mentioned is in need of review, but I don't currently have a reason
to think it can't get in.

> I'm all for bouncing 

Well, it's taken us the full month to get this far through the queue, so
I'd sure like to have more people on board reviewing next time.  Alvaro
helped but I miss Neil Conway, and some other people have helped in the
past.  However --- the present situation has nothing to do with lack of
reviewers, it's lack of time to finish the patches.

			regards, tom lane

---

From: Stephen Frost <[email-masked]>
Date: Fri, 1 Sep 2006 15:28:36 -0400

> other items done in time for a beta at, say, the end of next week.

I agree with bouncing these to get the beta out, though I hate having
to wait another year for them.  If things change in some way and we have
the opportunity to fit one in this time, I'd rather see updatable views
than bitmap indexes, personally.

Overall, I really think 8.2 is going to be an excellent release.  I wish
autovacuum could have been enabled by default and I'd just like to ask,
now and I'll try to remember again once 8.2 is out, please let's turn it
on by default for 8.3 (and early on so we get some good testing of it).

	Thanks,

		Stephen

---

From: Tom Lane <[email-masked]>
Date: Fri, 01 Sep 2006 16:10:01 -0400

> visible of marketable features we will have for this release. Especially 

Josh said nearly the same in core's discussion, but come now.  The
updateable-view patch (in its current form) does not offer *one single
thing* you can't do today, indeed for many PG releases past; it just
saves writing out some tedious rules.  If that's the most marketable
feature in 8.2 we're already in trouble.  I don't think it is anyway.
Looking at the CVS logs I see

	INSERT/UPDATE/DELETE RETURNING
	VALUES lists
	CREATE INDEX CONCURRENTLY
	pl/pgsql debugger (OK, not in core, but available)
	restartable WAL recovery
	constraint exclusion works on UPDATE/DELETE queries
	multiple-argument aggregate functions
	SQL2003-standard statistical aggregates
	COPY (SELECT ...) TO ...
	pg_dump selectivity options
	customizable timezone names
	GIN indexes
	FILLFACTOR for indexes and tables

and that's just back to the beginning of July, so it probably represents
about a quarter of the work that's gone into 8.2, but I got tired of
reading the logs at that point.

We have this discussion every single release cycle: "hey, if we wait X
amount more time then we can have cool features Y and Z".  But there's
always another X, Y, and Z.  More importantly, this view ignores the
very real disadvantages of not getting already-finished cool features
A through W out to our users sooner.

			regards, tom lane

---

From: "Joshua D. Drake" <[email-masked]>
Date: Fri, 01 Sep 2006 13:33:46 -0700

> saves writing out some tedious rules.

Sure, I can write the rules, you can write the rules but we can do 
rollup queries using unions too, but we still don't have rollup :)

Perception is everything and we don't have Updateable views until we 
don't have to write those rules (yes, its stupid)...

>  If that's the most marketable

Let's take multiple-argument aggregate functions for example... The 
marketable portion of that is limited. 85% of the people we are 
marketing too don't even know what that means. Heck, I haven't even run 
into the pervious limitation :)

Versus SQL2003-standard statistical aggregates. That wins hands down 
just because of the SQL2003-standard part of the string :)

COPY (SELECT...) TO, is only marketable to current PostgreSQL users. 
Those who are not users don't know the limitation and it is likely not 
something they would look for when evaluating PostgreSQL.

For my current customers the things that are the most marketable (note 
current customers) is probably:

  * CREATE INDEX CONCURRENTLY (this is a huge in production boon)

  * GIN indexes (after explanation)

  * constraint exclusion works on UPDATE/DELETE queries (We are using 
alot of tp now, so this is great)

  * Bitmap Indexes

It does not mean all those features are useful, they definitely are. I 
am just trying to look at it from at:

WHIZ* BANG* POW* perspective.

What makes the analyst (and I know this isn't the communities problem) 
say, "Holy crap batman, we need to be talking to more people about this 
database". I think those would be Updateable views, Bitmap Indexes and 
the SQL2003-standard statistical aggregates.

Sincerely,

Joshua D. Drake

--

---

From: Bruce Momjian <[email-masked]>
Date: Fri, 1 Sep 2006 17:20:39 -0400 (EDT)

> > * plpgsql improvements for returning record types

MS Visual C++

-- 
  Bruce Momjian   [email-masked]
  EnterpriseDB    http://www.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

---

From: Alvaro Herrera <[email-masked]>
Date: Fri, 1 Sep 2006 17:50:56 -0400

> reviewers, it's lack of time to finish the patches.

Also, for next time it would be good to have the reviewers got through
the patches *before* feature freeze, so that developers get early
feedback.  (Which in turn means more time to finish the patches).

-- 
Alvaro Herrera                                http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

---

From: Josh Berkus <[email-masked]>
Date: Fri, 1 Sep 2006 15:09:21 -0700

Bruce,

> MS Visual C++

Given that this could lead to us recruiting more developers out of our 
Windows user base, I'd prioritize it.

Overall, I think this whole process is pointing up that there's a problem 
with our historic Feature Freeze+1 month|beta|RCs cycle.  Given the number 
of developers and companies involved and the increasing size of the code, 
we need to develop a new schedule that allows us to get draft patches (or 
at least full specifications) eariler than the month before feature 
freeze.  In short,the issue is that Feature Freeze isn't just Feature 
Freeze, it's "final patch submission".

If you look at the two "incomplete" patches, and the "misfired" one 
(Bitmaps, Updatable Views, and WITH RECURSIVE) all of them are patches 
where the submitter had been working on them months ago, and might have 
made the release (or let us know they weren't on schedule) if we'd held 
them to an earlier deadline. As I recall, it was the same situation last 
year: getting large patches dropped on us out of the blue the week before 
feature freeze, which then didnt' make it in.

Therefore I propose that we adopt the following schedule for 8.3, assuming 
an September 1 Beta date:

June 1: Specification Freeze: specifications for all new features due
July 1: Feature Freeze: Draft patches and any minor tweaks due
August 1: Final (completed, mostly debugged) patches due
September 1: First Beta

-- 
--Josh

Josh Berkus
PostgreSQL @ Sun
San Francisco

---

From: Alvaro Herrera <[email-masked]>
Date: Fri, 1 Sep 2006 18:22:26 -0400

> WHIZ* BANG* POW* perspective.

Holy crap, Batman!  This database can do

INSERT INTO foo VALUES (1,1, 'so long'), (42, 2, 'and thanks'),
(142857, 3, 'for all the fish')

now!  We should be talking to more people about that!

That will make some MySQL users happy at least a bit :-)

---

From: Bruce Momjian <[email-masked]>
Date: Fri, 1 Sep 2006 18:23:30 -0400 (EDT)

> reviewers, it's lack of time to finish the patches.

I did try to get us additional help in reviewing.  Neil was unavailable,
and Alvaro could only give part of his time.

---

From: Peter Eisentraut <[email-masked]>
Date: Sat, 2 Sep 2006 00:25:54 +0200

Joshua D. Drake wrote:
> marketing too don't even know what that means. Heck, I haven't even
> run into the pervious limitation :)

As far as market analysis goes, I'd guess that 100% of the people who 
already use PostgreSQL by definition don't need any new features and 
would rather see incremental improvements across the board, which is 
what this or any release would give them.

-- 
Peter Eisentraut
http://developer.postgresql.org/~petere/

---

From: Bruce Momjian <[email-masked]>
Date: Fri, 1 Sep 2006 18:25:58 -0400 (EDT)

> August 1: Final (completed, mostly debugged) patches due

No rejiggering is going to get people to complete things they didn't
complete under the old system.  The plan you list above is what we did
for this release.

---

From: Peter Eisentraut <[email-masked]>
Date: Sat, 2 Sep 2006 00:28:56 +0200

Tom Lane wrote:
> Well, it's taken us the full month to get this far through the queue,
> so I'd sure like to have more people on board reviewing next time. 
> Alvaro helped but I miss Neil Conway, and some other people have
> helped in the past.  However --- the present situation has nothing to
> do with lack of reviewers, it's lack of time to finish the patches.

Not that I've had particularly plenty of time to give, but what's 
concerning me is that while we're moving toward an increasingly rigid 
process, we don't really have the process support to go with it.  How 
would anyone even know what patches are pending review?  Something to 
think about ...

---

From: Tom Lane <[email-masked]>
Date: Fri, 01 Sep 2006 18:55:27 -0400

> and Alvaro could only give part of his time.

It strikes me that setting feature freeze in midsummer might not be the
best strategy for having manpower available to review --- people tend to
be on vacation in August.  Maybe the answer is just to move the dates a
bit one way or the other.

			regards, tom lane

---

From: "Joshua D. Drake" <[email-masked]>
Date: Fri, 01 Sep 2006 16:01:31 -0700

> June 1: Specification Freeze: specifications for all new features due

I would suggest the above be:

June 1: Specification Freeze: specifications for all new features due & 
approved.

The last thing we want it to spend all of june arguing about how 
something should be implemented and thus no real coding gets done.

> August 1: Final (completed, mostly debugged) patches due

other then that w00t!

Joshua D. Drake

--

---

From: Tom Lane <[email-masked]>
Date: Fri, 01 Sep 2006 19:01:50 -0400

> Windows user base, I'd prioritize it.

Yeah, that's why I listed it as a major item.  I haven't had a chance to
look at the patch, but if it's not too ugly I would like to get it in
this time rather than next --- that could lead directly to having more
people available next time.

> them to an earlier deadline.

Perhaps, but I'm not sure what we can or should do about it.  Moving
deadlines up will either create a dead zone where we all sit around
twiddling our thumbs, or people will keep on coding till the last minute
anyway.

I think having a few patches that don't make the deadline isn't a bad
thing: it means we didn't have people sitting idle.  It's not like the
work will go to waste --- those things can still get in in the next
devel cycle.

			regards, tom lane

---

From: "Joshua D. Drake" <[email-masked]>
Date: Fri, 01 Sep 2006 16:17:22 -0700

> That will make some MySQL users happy at least a bit :-)

To be fair, it will make some of our customers happy to.. they can now 
get rid of the weird union hack they had to do instead :)

Joshua D. Drake

--

---

From: Gavin Sherry <[email-masked]>
Date: Sat, 2 Sep 2006 09:32:14 +1000 (EST)

> other items done in time for a beta at, say, the end of next week.

For what it's worth, Jie and I hope to have finished the bitmap streaming
this weekend which is the only outstanding issue with the code as it
currently stands. I now realise (nothing like hindsight) that we should
have posted a patch before we looked at streaming bitmaps because they've
proved more fiddly to implement than I thought :-(. At that point, we'd
more or less addressed the issues I'd raised when I posted at feature
freeze time.

Thanks,

Gavin

---

From: Josh Berkus <[email-masked]>
Date: Fri, 1 Sep 2006 16:39:41 -0700

Bruce, Tom, all:

> complete under the old system.  

It'll help the new people.  A lot of people -- if not most -- submitting 
their first major patch to PostgreSQL dramatically underestimate the 
amount of fix-up that's going to be required, and assume that there won't 
be a spec discussion, which there often is.  By getting them to submit a 
little at a time, *earlier*, we can avoid doing those things at the last 
minute.

Alternately, we can just make sure that first-time patchers have mentors 
who check progress well before feature freeze.

> The plan you list above is what we did 
> for this release.

No, it's not.   There's a bunch of patches which we had nothing on -- not 
spec, not design draft, not anything -- until we got them on July 20th.  
Our current system is to have only one deadline, at which point you're 
expected to have 85% of the patch done and up to PostgreSQL standards.  
That's quite a bit of "jumping in with both feet" for a newbie.

> and Alvaro could only give part of his time

Asking two people is not exactly an all-out effort to get reviewers.  

> bit one way or the other.

We've discussed that issue before, yes.  Since we're proposing a new 
roadmap process for 8.3, and will likely be dealing with a lot of major 
patches, maybe that's the release to delay?

However, as PR maven I do want to point out that doing the final release in 
December would be a bad idea.  Hard to get news coverage.  Also, we'd have 
the same issue with people being gone.

-- 
--Josh

Josh Berkus
PostgreSQL @ Sun
San Francisco

---

From: Tom Lane <[email-masked]>
Date: Fri, 01 Sep 2006 19:42:08 -0400

> currently stands.

AFAIR, you told me "it'll be done this weekend" last week.  And the week
before.

In any case, it's going to be a large complicated patch, and if I spend
all next week reviewing it then I won't make any progress on the other
items on my to-do list.  At some point we have to say "this has slipped
too far, we have to hold it over for 8.3".

			regards, tom lane

---

From: Gregory Stark <[email-masked]>
Date: Sat, 02 Sep 2006 00:42:54 +0100

> twiddling our thumbs, or people will keep on coding till the last minute

I don't think that's what would happen at all. In fact what I think would
happen is that it would free up people like you to work on stuff you're
interested in instead of reviewing patches.

As an extreme example consider the new Linux release cycle. They have a
non-freeze period of a couple weeks, followed by months of frozen time. Users
who want to try out new features on different hardware can do so and often
turn up problems developers reviewing the patches missed. Developers spend the
months of frozen time working on new patches which, if they're ready on time,
all go into the queue in the first few weeks of a new release. If they miss
the window they'll make the next one.

As a result they don't get things like what we saw with 8.0 where major new
features like PITR, nested transactions and so on all hit the tree in the
final days before the freeze. Instead they go in early and if they turn out to
be immature they get pulled long before the actual release.

I would love to see the bitmap indexes and updatable views get merged into the
tree just weeks after the release comes out rather than disappear again until
just before the next release. If I were a user depending on these new features
it would give me a hell of a lot more confidence if people could say they've
been in the CVS version for months and nobody's had problems with them than to
be told they were reviewed by one person three weeks ago and nobody else has
had much of a chance to use it.

-- 
  Gregory Stark

---

From: Tom Lane <[email-masked]>
Date: Fri, 01 Sep 2006 19:50:56 -0400

Josh Berkus <[email-masked]> writes:
> Our current system is to have only one deadline,

Well, no, it's not.  We have told people till we're blue in the face
"post early, post often".  Now I will plead guilty to not always having
spent as much time giving feedback on draft patches as I should've, but
the process is pretty clear.  As I see it the main problem is people
undertaking patches off in corners somewhere rather than discussing
their work on the mailing lists while they do it.

If you want more process in this we could institute rules like "feature
patches will be rejected out of hand if there wasn't previously a spec
proposal posted to pghackers".  I don't think this is really a good
idea, but I'm not sure what else to do.

			regards, tom lane

---

From: Peter Eisentraut <[email-masked]>
Date: Sat, 2 Sep 2006 02:09:35 +0200

Tom Lane wrote:
> "post early, post often".  Now I will plead guilty to not always
> having spent as much time giving feedback on draft patches as I
> should've, but the process is pretty clear.  As I see it the main
> problem is people undertaking patches off in corners somewhere rather
> than discussing their work on the mailing lists while they do it.

Again, process support.  If all we can offer people is to post 
multi-megabyte patches to the mailing list every month, that totally 
doesn't help.  We'd need ways to track the progress on these things: 
what was the specification for that patch, where was the discussion on 
it, what has changed in the patch since the last time, since the time 
before last time, what is left to be done, who has worked on it, etc.  
Figuring out the answer to those questions from a mailing list archive 
is tedious to the point that no one wants to do it.

---

From: Bruce Momjian <[email-masked]>
Date: Fri, 1 Sep 2006 20:29:42 -0400 (EDT)

> would anyone even know what patches are pending review?  Something to 

They are in the patches queue?

	http://momjian.postgresql.org/cgi-bin/pgpatches

---

From: Bruce Momjian <[email-masked]>
Date: Fri, 1 Sep 2006 20:32:47 -0400 (EDT)

> little at a time, *earlier*, we can avoid doing those things at the last 

That is in the developer's FAQ.

> That's quite a bit of "jumping in with both feet" for a newbie.

Right.  The developer's FAQ says they should follow a process.  Making
another process doesn't mean they will follow that either.

> Asking two people is not exactly an all-out effort to get reviewers.  

Well, not sure what else I can do.  Those are the people who used to
help out a lot.

> patches, maybe that's the release to delay?

Moving it away from summer might help, yea.

---

From: Bruce Momjian <[email-masked]>
Date: Fri, 1 Sep 2006 20:34:05 -0400 (EDT)

> idea, but I'm not sure what else to do.

Well stated.

---

From: Bruce Momjian <[email-masked]>
Date: Fri, 1 Sep 2006 20:36:32 -0400 (EDT)

> is tedious to the point that no one wants to do it.

Uh, Tom has been tracking Gavin on the bitmap patch every week for
weeks, and I pummelled EnterpriseDB/Jonah over the recursive query
patch.  Neither effort was very fruitful, but tracking wasn't what made
them fail.  I am not saying tracking is wrong, but rather tracking would
not have helped make these things happen faster.

---

From: "Joshua D. Drake" <[email-masked]>
Date: Fri, 01 Sep 2006 17:37:46 -0700

> 	http://momjian.postgresql.org/cgi-bin/pgpatches

Side note::

Perhaps this would be better served from the main website under the 
developer and community sections.

Joshua D. Drake

--

---

From: Bruce Momjian <[email-masked]>
Date: Fri, 1 Sep 2006 20:38:57 -0400 (EDT)

> developer and community sections.

It pulls my a mailbox file I use, and it does instant updates as soon as
I change it.  It is a URL.  Why do people care where it is?

---

From: "Joshua D. Drake" <[email-masked]>
Date: Fri, 01 Sep 2006 18:01:54 -0700

> I change it.  It is a URL.  Why do people care where it is?

I would think that it would be easier to find :)

Joshua D. Drake

--

---

From: Andrew Dunstan <[email-masked]>
Date: Fri, 01 Sep 2006 21:24:57 -0400

> not have helped make these things happen faster.

It would make the process more transparent, which is something several 
people have expressed a desire for.

cheers

andrew

---

From: Tom Lane <[email-masked]>
Date: Fri, 01 Sep 2006 21:37:58 -0400

> the window they'll make the next one.

That doesn't seem particularly appealing for our purposes.  What it
sounds like is that all the development happens somewhere outside the
main tree, and every so often everybody tries to land a bunch of large
patches at once.  That's going to be a mess if the patches interact at
all, and it certainly isn't going to address my primary concern of
encouraging a publicly-visible development process instead of having
people hiding in a corner until they can drop a large patch on us.

> I would love to see the bitmap indexes and updatable views get merged
> into the tree just weeks after the release comes out rather than
> disappear again until just before the next release.

Here I agree - partly.  With some continuing effort these patches could
be in fine shape to apply by the time we branch CVS for 8.3 development.
However, our traditional approach to beta period is that it's supposed
to be a "quiet time" where people focus on testing, debugging, and
documentation.  Shepherding people who are doing major development
during that period is likely to be a serious distraction from making the
release ready and high-quality.

			regards, tom lane

---

From: Peter Eisentraut <[email-masked]>
Date: Sat, 2 Sep 2006 03:41:36 +0200

Bruce Momjian wrote:
> It pulls my a mailbox file I use, and it does instant updates as soon
> as I change it.  It is a URL.  Why do people care where it is?

The complaint has been that not enough people help with reviewing 
patches.  That would indicate that the problem is not location but 
scalability.  If everything has to go through your private mailbox, 
then it's not a very obvious process to outsiders.

---

From: Bruce Momjian <[email-masked]>
Date: Fri, 1 Sep 2006 21:43:25 -0400 (EDT)

> I would think that it would be easier to find :)

We have a link from the developer's page:

	http://www.postgresql.org/developer/roadmap

---

From: Bruce Momjian <[email-masked]>
Date: Fri, 1 Sep 2006 21:46:02 -0400 (EDT)

> then it's not a very obvious process to outsiders.

Well, you can grab items from there and apply them.  I will remove them
from the mailbox when I see the commit.   You have another system that
will not be significantly more work?  Any you can apply them right from
the email lists so they never get to the queue.

---

From: Alvaro Herrera <[email-masked]>
Date: Fri, 1 Sep 2006 21:52:09 -0400

> the email lists so they never get to the queue.

Another novel idea: have a "bugtracker" of sorts where selected people
can add "feature requests" and append patches to them.  There, people
could add "fake" feature requests describing the stuff they are working
on, and publish the patches they have for them.

---

From: Peter Eisentraut <[email-masked]>
Date: Sat, 2 Sep 2006 03:58:04 +0200

Bruce Momjian wrote:
> patch.

Great, but where is this documented, so others know about this?

> Neither effort was very fruitful, but tracking wasn't what 
> made them fail.  I am not saying tracking is wrong, but rather
> tracking would not have helped make these things happen faster.

The fallacy here is assuming that all these things should be 
single-person tasks.  As long as we only have one coder and 
one "manager", we don't need much process support, but then we're 
pretty nearly at the point we're now, where two or three people review 
patches while the rest just sits around and wonders what this feature 
freeze thing is supposed to be about.

I can tell you plenty of stories about the updatable views patch.  One 
month after feature freeze, we notice that we didn't even have an 
accepted design specification.  I'm sure it was posted sometime, but 
how do we find it now?  People complain unjustly that the patch was 
posted at the last minute, but in fact updated patches and information 
have been posted regularly for more than one year.  But it's impossible 
to tie these things together unless you are mailing list crawling 
software with artificial intelligence capabilities.  And during the 
last two weeks, no make that six months, Bernd has spent half his time 
analyzing and reverting breakage that well-meaning reviewers had 
injected into his patch, with the other half possibly spent keeping the 
patch up to date with the moving development tree.

There is, of course, no silver bullet.  But more successful involvement 
of people who are not in the inner circle needs more support in many 
ways.

---

From: Peter Eisentraut <[email-masked]>
Date: Sat, 2 Sep 2006 04:07:15 +0200

Bruce Momjian wrote:
> Well, you can grab items from there and apply them.  I will remove
> them from the mailbox when I see the commit.   You have another
> system that will not be significantly more work?  Any you can apply
> them right from the email lists so they never get to the queue.

What the current system lacks is documentation, which makes the process 
behind those lists known to others, and the ability to interact with 
the system.  If I can't see any feedback in the system for the actions 
I do, then the system is not very useful to me.

---

From: Peter Eisentraut <[email-masked]>
Date: Sat, 2 Sep 2006 04:08:18 +0200

Alvaro Herrera wrote:
> I've been bitten by having stuff rejected because there was no
> immediate use for them; so I had to maintain them in a private tree,
> where they were subsequently broken by someone else's parallel
> development.

A better version control system would surely seem to help here and in 
other instances.

---

From: Bruce Momjian <[email-masked]>
Date: Fri, 1 Sep 2006 22:09:50 -0400 (EDT)

> Great, but where is this documented, so others know about this?

I see no value in documenting it.

> of people who are not in the inner circle needs more support in many 

I do things only if others do not.  If committers applied patches as
they came in, the patch queue would be empty, and if others tracked open
issues, I wouldn't have to.

---

From: Peter Eisentraut <[email-masked]>
Date: Sat, 2 Sep 2006 04:10:40 +0200

Alvaro Herrera wrote:
> Another novel idea: have a "bugtracker" of sorts where selected
> people can add "feature requests" and append patches to them.  There,
> people could add "fake" feature requests describing the stuff they
> are working on, and publish the patches they have for them.

Certainly, much of this discussion is trying to yell "tracking 
software".  Track patches, track comments, track open items, and so on.  
This doesn't have to be the same as a bug tracker, though.

---

From: Bruce Momjian <[email-masked]>
Date: Fri, 1 Sep 2006 22:11:16 -0400 (EDT)

> I do, then the system is not very useful to me.

What do you want to happen?  Just apply the patch?  You want to log into
a system to say you applied it?  If people are not doing applications
when they have no overhead, I doubt they will do it when there are
additional steps.

---

From: Bruce Momjian <[email-masked]>
Date: Fri, 1 Sep 2006 22:12:20 -0400 (EDT)

> This doesn't have to be the same as a bug tracker, though.

OK, why do we use email then?  Move it all into a tracker?  If not, who
is going to move info from email to the tracking system?

---

From: "Jonah H. Harris" <[email-masked]>
Date: Fri, 1 Sep 2006 22:13:07 -0400

On 9/1/06, Bruce Momjian <[email-masked]> wrote:
> I pummelled Jonah over the recursive query patch.

He did.  Trust me on this... think I still have some bruises too :)

> Neither effort was very fruitful, but tracking wasn't what made

This is correct.  I just had too much stuff to do and there wasn't
enough time to get the hierarchical query patch worthy of someone
spending their time on a review.

IMHO, tracking occasionally is alright.  However, as a developer,
being "pummelled" does sometimes get irritating; even to the point
that I will sometimes discontinue working on something because it's
too much of a hassle for me to spend my free-time on.  Though, this
was not the case for hierarchical queries at all; that really was due
to a lack of time.

There's got to be a "happy medium" in which we can keep our status
updated without it becoming an irritation.  Has anyone looked at
something like dotProject?  It may be something we could use for
development.  Of course, there's lots of other tools... but it would
be nice if we had a central location for each task's status so that we
don't have to resort to searching email and/or archives.

-- 
Jonah H. Harris, Software Architect | phone: 732.331.1300
EnterpriseDB Corporation            | fax: 732.331.1301
33 Wood Ave S, 2nd Floor            | [email-masked]
Iselin, New Jersey 08830            | http://www.enterprisedb.com/

---

From: Peter Eisentraut <[email-masked]>
Date: Sat, 2 Sep 2006 04:14:23 +0200

> I see no value in documenting it.

The value in documenting it is that the general awareness about the 
progress of development is raised, so more people know where and how to 
get involved, as opposed to only three people knowing and the rest 
finding out one month after feature freeze that the patch isn't ready.

> they came in, the patch queue would be empty, and if others tracked
> open issues, I wouldn't have to.

Fair enough.  Perhaps we should find others with ideas about better 
tracking and the willingness to implement them.

---

From: Gregory Stark <[email-masked]>
Date: 02 Sep 2006 00:03:17 -0400

Peter Eisentraut <[email-masked]> writes:

> doesn't help.  

You're right, it's a pain for developers to have to keep shipping around
patches over and over. It means anyone else who wants to look at your code is
probably looking at an old version, there's no convenient way to convert from
incremental patches to cumulative patches, etc.

There are tools to solve this but they're not "process support", they're
revision control systems. If CVS didn't suck so hard I would suggest that the
natural thing to do would be to grant CVS access far and wide and have people
create branches for their proposed changes. The existing set of committers
would become the set of people allowed to commit to HEAD. Sadly CVS makes this
about as much work as the current system and the risks are much higher.

> We'd need ways to track the progress on these things: 

Frankly these things all sound pretty trivial. 

-- 
greg

---

From: Gregory Stark <[email-masked]>
Date: 02 Sep 2006 00:20:26 -0400

> people hiding in a corner until they can drop a large patch on us.

Well that's basically what's been happening except they drop it on us a day
before feature freeze giving you a frantic month of trying to make it work.

The reason people go off on their own and develop without help are manifold
but I'm not sure the release process is one of them. They're not going to be
able to commit their incomplete code to the main CVS regardless of whether
there's a freeze or not anyways. 

They can work openly on new features during the feature freeze and beta
period, they just won't have the expectation that they'll squeeze it into this
release. Instead they'll know that if it's ready for the patch landing period
it'll be in the next release.

> release ready and high-quality.

Well on the flip side you have months to do that work instead of having
pressure to rework whole patches in short order. Perhaps it helps that at
least in the case of Linux it's mostly different people handling the releases
and the raw development. I'm not sure we have enough manpower for that.

-- 
greg

---

From: Lukas Kahwe Smith <[email-masked]>
Date: Sat, 02 Sep 2006 08:12:47 +0200

> people have expressed a desire for.

Yes, the processes seems to work by having two of the most important 
people waste time on getting information anyone else could collect, or 
that the developer himself could quickly provide and keep up to date.

Furthermore the process expects these two people to know who to ask, 
what to look for etc. Wouldn't it be great if someone could just decide: 
"hey I know postgres, I have some unexpected spare time, I want to do 
some code reviewing on patch x, y and z"

If this all works through some issue tracker, a wiki or a combination of 
both, the end result is transparency, the opportunity to take load off 
from people that have important other things to do and a chance at 
getting unexpected help.

For example I have no expertise in coding on Postgres, but I think I 
would be able to collect information from this mailinglist (like specs, 
url's etc.) and put them in some issue tracker or wiki. I have done 
exactly the same for PHP [1] (though there are rarely specs thrown 
around in PHP, so my PHP todo list is not much more than a simple bullet 
list of todo's with a name and occasional URL's to additional information).

regards,
Lukas

[1] http://oss.backendmedia.com/PHPTODO/

---

From: Lukas Kahwe Smith <[email-masked]>
Date: Sat, 02 Sep 2006 09:00:35 +0200

> list of todo's with a name and occasional URL's to additional information).

Actually I should add that I went ahead and created the PHP todo list on 
my own, without any official blessing and one by one internals developer 
have joined. Now its actively used in the entire release process.

This is probably the best approach to go about doing this for PostgreSQL 
as well. That way there is no additional work, and instead it can show 
if it actually helps people. And if it does more people will start to 
contribute to it. If not then someone who did not contribute to the code 
anyways wasted a bit of time to get to know the PostgreSQL todo items 
really well. ;)

regards,
Lukas

---

From: David Fetter <[email-masked]>
Date: Sat, 2 Sep 2006 03:55:28 -0700

> He did.  Trust me on this... think I still have some bruises too :)

That wasn't productive.  Getting it out in public that your available
time didn't match the size of the project in, say, February or even
April, would have been.

> >them fail.  I am not saying tracking is wrong, but rather tracking
> >would not have helped make these things happen faster.
> 

I know it's hard to judge from the inside, which is why this process
needs more public visibility, early and often, to the rest of the
community.  It does nobody any good to have Bruce bawl you out
repeatedly in private, and then spring it on the rest of us in June
that you weren't even close.

> too much of a hassle for me to spend my free-time on.

That's an excellent outcome for the project, by the way.  When
somebody is feeling "pummeled," it's frequently a signal that they
really don't have the time they thought they did, and the sooner and
more widely such a situation is known, the better.

> Though, this was not the case for hierarchical queries at all; that
> really was due to a lack of time.
> 
> be nice if we had a central location for each task's status so that
> we don't have to resort to searching email and/or archives.

...and we're back to the infamous tarbaby of bug/issue trackers.  It's
time we took that head-on instead of dancing around it.

As expressed many times earlier, such a system must be accessible,
both for read and write, by email.  What other things must it do?
Should it do?  Must it avoid?

Cheers,
D
-- 
David Fetter <[email-masked]> http://fetter.org/
phone: +1 415 235 3778        AIM: dfetter666
                              Skype: davidfetter

Remember to vote!

---

From: David Fetter <[email-masked]>
Date: Sat, 2 Sep 2006 04:14:03 -0700

On Fri, Sep 01, 2006 at 09:46:02PM -0400, Bruce Momjian wrote:
> Peter Eisentraut wrote:
> > Bruce Momjian wrote:
> > > It pulls my a mailbox file I use, and it does instant updates as
> > > soon as I change it.  It is a URL.  Why do people care where it
> > > is?
> > 
> > scalability.  If everything has to go through your private
> > mailbox, then it's not a very obvious process to outsiders.
> 
> them from the mailbox when I see the commit.

This, in essence, is the problem.  You're using the first person
singular here, and it's become obvious that one person can't keep up
with this any more.  The project, thanks in part to your hard work,
has grown past that stage.

> You have another system that will not be significantly more work?

If it's significantly more work spread among many people, that is a
perfectly viable solution, just as long as it doesn't cause any one
person to be overloaded.

We need to get this project more bus-proof, whether it's the patch
queue or a server people depend on that's silently down for a week
with no ETAs in sight for getting any piece of it back on line.

Bruce, are you ready to help create a more bus-proof situation?  If
so, what specific steps would you like to take in that direction?

Cheers,
D

Remember to vote!

---

From: Martijn van Oosterhout <[email-masked]>
Date: Sat, 2 Sep 2006 14:48:34 +0200

> have joined. Now its actively used in the entire release process.

That is the approach we should be taking. I see too many people wait
for an "official" solution when in fact the first person to come up
with a todo list that people like better than the current one will win.

I remember something about setting up a wiki for a todo list and pie in
the sky list. I thought it had promise, but until the wiki is there we
won't know...

Have a nice day,
-- 
Martijn van Oosterhout   <[email-masked]>   http://svana.org/kleptog/
> From each according to his ability. To each according to his ability to litigate.

---

From: Peter Eisentraut <[email-masked]>
Date: Sat, 2 Sep 2006 15:49:16 +0200

Martijn van Oosterhout wrote:
> I remember something about setting up a wiki for a todo list and pie
> in the sky list. I thought it had promise, but until the wiki is
> there we won't know...

I think the wiki is the prerequisite for many ideas about alternative 
tracking and documentation mechanisms.  I just wonder what the hold up 
is on it.

---

From: Robert Treat <[email-masked]>
Date: Sat, 2 Sep 2006 10:07:57 -0400

> has grown past that stage.

AFAICT Bruce is not the bottleneck and our problem is not that multiple people 
are reviewing the same item and duplicating effort, so I think your making 
conclusions based not on the evidence but instead on a desired outcome.  

No offense, a whole lot of this thread seems to be positioned that way, but 
the real problem seems to be we do not have enough patch reviewers.  ISTM the 
questions we should be asking are who can actually help out with patch review 
and then ask those people why they haven't done it.   If folks like Peter, 
Andrew, Magnus, Simon, Joe, and Niel all say that they are not reviewing 
patches because they can't find the patches that need review, they can't 
figure out who is reviewing what, or they don't think anyone is paying 
attention when they do review something, then I think we have a serious 
problem and we certainly need to change processes.  What I think you'll find 
is that they are all just busy working on other things, which in that case I 
think we need to figure out how to motivate them to focus on the patch queue 
rather than other items.    IMHO

-- 
Robert Treat
Build A Brighter LAMP :: Linux Apache {middleware} PostgreSQL

---

From: Tom Lane <[email-masked]>
Date: Sat, 02 Sep 2006 10:15:01 -0400

> as well.

I agree.  Look at the most successful recent process change around here:
the buildfarm.  Andrew Dunstan took it upon himself to make that happen.
He built it, and they came.

No bug/issue tracker, or anything else, is going to be successful unless
somebody commits enough time to make it so.  I've noted a whole lot of
enthusiasm for having a tracker in these recent discussions, but a
remarkable shortage of individuals stepping up to do the work.

			regards, tom lane

---

From: "Greg Sabino Mullane" <[email-masked]>
Date: Sat, 2 Sep 2006 15:20:46 -0000

> remarkable shortage of individuals stepping up to do the work.

There are a number of reasons for this, not least of which is the
enormous and ever-changing requirements such a system would have to
have. My company has even offered resources to get something going,
and I think I convinced Bruce at one point of the usefulness of a
PG bug/feature tracking system. However, the problem is that existing
software is never going to be good enough.

The buildfarm is an excellent example of this. If Andrew had tried
to create it with something "off-the-shelf", it probably would have
taken longer to get started, been more complicated than it needed
to be, and would not work as well as it now does. However, it's been
a smashing success, so much so that MySQL is now envious[1] and is
trying to make their own version of it[2].

[1] http://www.jpipes.com/index.php?/archives/93-MySQL-Build-Farm-Part-1.html
[2] http://forge.mysql.com/wiki/MySQL_Build_Farm_Initiative

I've been thinking about this a lot since before the Summit, and the
only solution I see is to design something specifically for us.
Rather than get bogged down in details about how it will work and
what technologies it will be using, I'd like to share my ideas on
how it will work from an end-user perspective. There's a simple web
interface. You can use it to get the answer to questions like this:

What are all the open bugs for the current release?
Who is working on the updateable views patch?
Which bugs were fixed in 8.1.3?
What's the feature history of psql since 2003?
What are some easy TODO items that need getting done?
Just when does Tom Lane sleep?
What features were added in 8.1?
What are the current unapplied patches?
What bugs were fixed in tsearch2 from version 7.4.5 to 7.4.12?

Yes, maintaining it will be a royal pain in the butt. But my theory has
been "if you build it, they will come". It will require a lot of human
interaction, as automation only takes you so far, especially when trying
to parse mailing list messages. But if we eventually get a decent-size
group of people who each put in a little work on it, it should be
quite possible to keep it up to date. (Also, this would be a great way
for people to help the project who don't have the time and/or skills
to do coding).

The idea is to allow everything to happen as it already does, e.g. no
extra burdens for Tom, no extra processes anywhere. However, all the
existing information is gathered into one centralized and searchable place.
The primary sources will include the code, the docs, and the mailing lists.
Especially the mailing lists.

So, rather than go off in a corner and start coding, how about I start a
pgfoundry project for this and document my existing ideas? Any objections
to giving this a try?

- --
Greg Sabino Mullane [email-masked]  [email-masked]
End Point Corporation 610-983-9073
PGP Key: 0x14964AC8 200609021107
http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8

---

From: Tom Lane <[email-masked]>
Date: Sat, 02 Sep 2006 11:28:58 -0400

Alvaro Herrera <[email-masked]> writes:
> Here's a completely novel idea: accept incremental patches.

I don't think it's as novel as all that --- personally I've always
preferred to tackle large projects incrementally.

> I've been bitten by having stuff rejected because there was no immediate
> use for them;

Maybe you failed to make the larger picture clear.  If you'd gotten the
pghackers list to agree to some plan like "I want to accomplish X, and
to get there I want to do A then B then C", I doubt we'd have rejected a
subsequent patch to do A (assuming it doesn't break anything else, of
course).

			regards, tom lane

---

From: Theo Schlossnagle <[email-masked]>
Date: Sat, 2 Sep 2006 11:41:48 -0400

> preferred to tackle large projects incrementally.

I think that accepting incremental patches to a mainline is _bad_ and  
that accepting them in general is good (as you have to go through  
that process outside of version control sometimes anyway).  Sticking  
them in CVS however can get a bit messy.  This is where other version  
control systems that have a bit better branching and merging support  
has an advantage as people can work in the repository on their  
project in separate branches and pulling them all back together again  
(once an overall satisfaction metric has been reached) is not  
excruciatingly painful.  Where am I going with this?  From my  
experience accepting incremental patches is a _bad_ idea unless you  
have a VCS that has really makes it _easy_ to manage them.  Sounds  
like more work than worth on the postgres project as it is now.

Additionally, what problem is accepting incremental patches supposed  
to solve?

// Theo Schlossnagle
// CTO -- http://www.omniti.com/~jesus/
// OmniTI Computer Consulting, Inc. -- http://www.omniti.com/

---

From: Tom Lane <[email-masked]>
Date: Sat, 02 Sep 2006 11:42:25 -0400

> only solution I see is to design something specifically for us.

Well, nobody's going to accuse you of thinking too small ;-).  Sounds
great to me, though, if you think you can pull it off.  Have at it!

> [etc]

That sounds fine for the "output" part of things, but where does the
"input" come from?  ISTM that it's been the input side that's been the
real bone of contention in all the discussions of specific tracking
software.

BTW, another "output" thing you might consider is "having draft release
notes ready-to-go on demand".  Currently, Bruce prepares the release
notes on the basis of a very tedious scan of the CVS commit logs.
If this sort of stuff were being dropped into a tracker as it went
into the CVS tree, at least the research part of making the notes would
be zero-effort (or perhaps better to say that the work would be spread
out instead of concentrated).

			regards, tom lane

---

From: Robert Treat <[email-masked]>
Date: Sat, 2 Sep 2006 12:00:59 -0400

> out instead of concentrated).

FWIW I have never understood why we don't require patch submitters/committers 
to update the release notes when they do the patch.  I know Bruce says that 
it's not a problem for him to do it, but it sure would help out some of the 
other folks.

---

From: Bruce Momjian <[email-masked]>
Date: Sat, 2 Sep 2006 12:02:52 -0400 (EDT)

> pgfoundry project for this and document my existing ideas? Any objections

Good approach.  Let me tell you what doesn't work well currently:

	What is fixed in current CVS that was not on the TODO list
		already, and marked with a dash?
	What fixes/bugs are being worked on but not yet in the patch queue?
	What are all the features/fixes in each release?

Right now, the release notes show a list of all the significant items in
each release, but it isn't available until the release, and it isn't
complete (because it would be unreadable by ordinary users).  And there
is no tracking of individual items in progress except by individual
developers.  Magnus, for example, tracks Win32 items, and Tom tracks
backend stuff.  I track whatever no one else tracks.

As far as tracking, right now, +50% of items come in and are discussed
via email, not a bug form.  How does that information get into a
tracking system without huge extra work?  We can do it, and it would
give people nice feedback, but is the effort worth the benefit?

---

From: Tom Lane <[email-masked]>
Date: Sat, 02 Sep 2006 12:03:10 -0400

> Additionally, what problem is accepting incremental patches supposed  

Keeping the individual patches reviewable is one useful goal.

We may be talking at cross-purposes here.  The sort of thing I think
Alvaro is imagining is something like what I did a year or two back when
I wanted to make the executor treat plan trees as read-only --- if
memory serves, I did that in three or four commits spread over a week or
two.  I could have done it in one massive patch but it would have been
unreadable to me or anyone else.  And for that matter, the reason for
doing it at all was as part of a much larger concept that's still
unfinished (better caching and invalidation of plans).  When you're
talking about tasks of that magnitude, stepwise improvement is the only
way you are going to get there at all.

I really don't have any faith in the concept of doing very large tasks
on a development branch and expecting to land them in one huge merge.
For a nearby counterexample look at Postgres-R, which never has and
probably never will manage to produce a patch that could apply to the
core project's CVS head.  At least not without thinking of some
incremental way to do it, because by the time they bring all their
changes up to HEAD, the tree has always drifted under them.  There is
no magic version control system that will fix that.

			regards, tom lane

---

From: Bruce Momjian <[email-masked]>
Date: Sat, 2 Sep 2006 12:05:32 -0400 (EDT)

> notes on the basis of a very tedious scan of the CVS commit logs.

Having an intermediate release note list, beyond the dashed TODO items,
might be helpful.  So the idea is to track work in progress, and
completed items.  The question is always whether the effort is worth the
benefit.  We can try and see how it works.

> out instead of concentrated).

Well, multiple commits often end up as a single entry (and many commits
don't need to show up in the release notes), so the tracker would just
give me a backup when creating the release notes.

---

From: "Greg Sabino Mullane" <[email-masked]>
Date: Sat, 2 Sep 2006 16:05:34 -0000

> That will make some MySQL users happy at least a bit :-)

It will certainly make porting applications from MySQL to
Postgres a lot easier, and such porting is where a lot more
effort needs to be put, IMO. (I mean more people working
on porting applications, not adding mysqlisms to the code :)

- --
Greg Sabino Mullane [email-masked]
End Point Corporation
PGP Key: 0x14964AC8 200609021200
http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8

---

From: Bruce Momjian <[email-masked]>
Date: Sat, 2 Sep 2006 12:06:15 -0400 (EDT)

> it's not a problem for him to do it, but it sure would help out some of the 

I can't even get documentation for many patches.  I am hesitant to add
even more burden.  I would prefer they concentrate on documentation.

---

From: Robert Treat <[email-masked]>
Date: Sat, 2 Sep 2006 12:07:16 -0400

> too far, we have to hold it over for 8.3".

Perhaps if some other committers would take ownership of other items on 
your "to be done list", that would spread the workload enough to keep 
everything going?  (Yes, this does require some of the other folks to step 
up... responses to your original email with "I will commit to finishing this 
item this week" would probably be enough)

---

From: "Joshua D. Drake" <[email-masked]>
Date: Sat, 02 Sep 2006 09:11:06 -0700

Tom Lane wrote:
> two. 

To second this, Alvaro is constantly beating our (cmd) other developers 
to do this, so I would guess that you are correct :).

I find also that this method allows someone like me, who can read C and 
understand good parts of it to get the gist of what is going on without 
trying to grok the whole thing. Large patches make it very difficult.

Sincerely,

Joshua D. Drake

--

---

From: "Joshua D. Drake" <[email-masked]>
Date: Sat, 02 Sep 2006 09:15:01 -0700

Greg Sabino Mullane wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> 
> Tom Lane sagely noted:

Although I applaud Greg's efforts, I think it makes more sense to take 
something and make it our own instead of trying to build something from 
scratch.

I think that Bugzilla will probably serve our purpose for most things.

Greg, how about you and I (I am talking with Dunstan on this as well) 
set up Bugzilla and just doing an small group audit of how it could (or 
couldn't) work and report back?

Sincerely,

Joshua D. Drake

--

---

From: Lukas Kahwe Smith <[email-masked]>
Date: Sat, 02 Sep 2006 18:16:17 +0200

> rather than other items.    IMHO

I think that if all the patches are listed with all relevant context 
information on a webpage, then people can more easily jump in when they 
unexpectedly have time (or prefer to procrastinate some other real world 
thing they should rather work on). Right now if you have a few hours to 
spare you do not have all the information readily available. Even worse 
by inquiring for the information you might feel you are commiting more 
than you really wanted to. This kind of information needs to be right 
there without any person having to actively provide it upon inquiry.

regards,
Lukas

---

From: Bruce Momjian <[email-masked]>
Date: Sat, 2 Sep 2006 12:17:19 -0400 (EDT)

> questions we should be asking are who can actually help out with patch review 

Let me explain what I do.  I subscribe to most PostgreSQL email lists. 
I basically try to make sure every bug, patch, or idea is processed.  If
it is a bug via user error, did they get correct information.  If it is
valid PostgreSQL bug, was it fixed or added to the TODO list.  If it was
a patch, was it applied or rejected.  If it is a feature request, was it
added to the TODO list or rejected.

That's about it, folks.  Not a lot of fancy magic.  As other people take
up these items, my workload will be reduced.  I do not take ownership of
any of these things.  I merely jump on them when they are _not_ dealt
with by others.

What I am unclear about is how all of this traffic is going to be pushed
into a system that cleanly categorizes it so others can understand it.
It can be done, but what is the cost/benefit to it?

Perhaps if email could be pushed into the system, and somehow
categorized as bug, patch, or feature request, and somehow removed only
when it is dealt with.  That is what I do now via my mailbox.

---

From: Lukas Kahwe Smith <[email-masked]>
Date: Sat, 02 Sep 2006 18:18:13 +0200

> tracking and documentation mechanisms.  I just wonder what the hold up 

We have a wiki already:
http://wiki.postgresql.org/

It could be a bit faster, but its there.

I even already started on a little documentation effort:
http://wiki.postgresql.org/index.php/pgwiki:Variations

regards,
Lukas

---

From: Lukas Kahwe Smith <[email-masked]>
Date: Sat, 02 Sep 2006 18:20:57 +0200

> for people to help the project who don't have the time and/or skills

I think we should start with the wiki and then add in whatever we notice 
can be automated. This way we may not initially be able to cover all the 
bases, but it will mean that whatever we do automate actually matches 
real world requirements.

> Especially the mailing lists.

Yeah, let the ones work on this that do not do any of the pgsql coding.

regards,
Lukas

---

From: Peter Eisentraut <[email-masked]>
Date: Sat, 2 Sep 2006 18:21:29 +0200

Greg Sabino Mullane wrote:
> have.

> The buildfarm is an excellent example of this.

The build farm is not an example of this.  There isn't any build-farm 
software out there.  There is, however, plenty of issue tracking 
software out there.

Just pointing that out.  Have at it.  I for one would rather set up 
existing issue trackers and work with them.  If I could just find 
hardware to host it.  Not that I've looked, though.

---

From: Bruce Momjian <[email-masked]>
Date: Sat, 2 Sep 2006 12:22:48 -0400 (EDT)

> there without any person having to actively provide it upon inquiry.

OK, how does that happen without a lot of work, or moving all discussion
on to a web-based system?

---

From: "Joshua D. Drake" <[email-masked]>
Date: Sat, 02 Sep 2006 09:25:41 -0700

> It could be a bit faster, but its there.

That wiki is wrong. :) It was set up wrong and configured wrong. It was 
supposed to be for developers only.

There is also another wiki that is a trac based that was set up at dave 
pages request (for slaves_to_www).

> http://wiki.postgresql.org/index.php/pgwiki:Variations

And note it has already been mentioned that, that wiki was the wrong 
place to put it. Not that I am slamming your efforts, I think it was 
really good that you did the docs :). They are just in the wrong place.

Sincerely,

Joshua D. Drake

> TIP 9: In versions below 8.0, the planner will ignore your desire to
>       choose an index scan if your joining column's datatypes do not
>       match
> 

--

---

From: Lukas Kahwe Smith <[email-masked]>
Date: Sat, 02 Sep 2006 18:27:16 +0200

> on to a web-based system?

No discussion takes place on the lists and IRC and busy bees make sure 
that the progress/decisions etc get updated on the static wiki. Wiki's 
suck at discussions, but they are great to store the decisions made so 
that anyone can get himself upto speed and things do not get lost over time.

It seems that you have been the only busy bee so far, and we definitely 
need more for this to work.

regards,
Lukas

---

From: Lukas Kahwe Smith <[email-masked]>
Date: Sat, 02 Sep 2006 18:28:55 +0200

> pages request (for slaves_to_www).

Setup something better, until then we can start using it to generate 
content.

> really good that you did the docs :). They are just in the wrong place.

No problem. Its easy to move content, its much harder to generate it.

regards,
Lukas

---

From: Bruce Momjian <[email-masked]>
Date: Sat, 2 Sep 2006 12:40:34 -0400 (EDT)

> need more for this to work.

Yea, I was afraid that was the answer.  :-(

---

From: Martijn van Oosterhout <[email-masked]>
Date: Sat, 2 Sep 2006 18:55:39 +0200

> http://wiki.postgresql.org/

I must have missed the annoucement, oh well...

Now I'm only familiar with twiki so maybe this sounds silly but:

Does it support sections? Like can you have a developer section, a
patches section, a TODO section, etc. And then have differing
permissions per section?

So I was thinking a TODO section with the list as front page and each
tem having a page within that section. Create a parallel page
Wannabe-todo-tiems that can be edited by anyone and lock the main page
to just a few. Then all some developer has to do is scan the wannabe
list and copy the links to the main list.

Is this kind of setup ossible?

Have a nice day,

---

From: Lukas Kahwe Smith <[email-masked]>
Date: Sat, 02 Sep 2006 19:09:42 +0200

> Yea, I was afraid that was the answer.  :-(

But we have a few volunteers, like me for example.
Now don't say "I was afraid that was the answer" again or I might feel 
insulted :)

regards,
Lukas

---

From: Dave Page <[email-masked]>
Date: Sat, 02 Sep 2006 18:31:51 +0100

> out instead of concentrated).

We have the developers update the CHANGELOG file with each non-trivial
change in pgAdmin. It works very well for us and producing release
announcements etc. is a trivial task. The overhead on each developer is
virtually zero as well as they have to compose some text for the commit
message anyway.

http://www.pgadmin.org/development/changelog.php

Regards, Dave.

---

From: Dave Page <[email-masked]>
Date: Sat, 02 Sep 2006 18:38:30 +0100

> http://wiki.postgresql.org/index.php/pgwiki:Variations

The wiki will be going shortly as it's not been setup in the way that was
agreed, nor is it being used for it's intended purpose. I'll put it right
when I get time from dealing with releases of everything I seem to be
involved in.

Regards, Dave.

---

From: "Joshua D. Drake" <[email-masked]>
Date: Sat, 02 Sep 2006 12:01:28 -0700

> Now I'm only familiar with twiki so maybe this sounds silly but:

wiki.postgresql.org is dead. :). It wasn't supposed to be an open wiki 
like that.

Sincerely,

Joshua D. Drake

--

---

From: Martijn van Oosterhout <[email-masked]>
Date: Sat, 2 Sep 2006 21:16:14 +0200

> when I get time from dealing with releases of everything I seem to be

I beleive the idea was to not make the site editable by the public?
Does't that kind of defeat the purpose of getting more people to
takeover work?

Have a nice day,

---

From: Dave Page <[email-masked]>
Date: Sat, 02 Sep 2006 20:28:27 +0100

> Does't that kind of defeat the purpose of getting more people to

That was one of the more flexible points as far as I'm concerned. What was
more important was to make it clear it was a *developer* resource and not an
alternative techdocs site or in fact a site intended in any way for end
users.

See my next email for more...

Regards, Dave.

---

From: Andrew Dunstan <[email-masked]>
Date: Sat, 02 Sep 2006 21:11:51 -0400

> hardware to host it.  Not that I've looked, though.

What is more, a good bug tracking system is orders of magnitude more 
complex that the buildfarm.

If we want something sooner rather than later, let's use something off 
the shelf and tailor it.

cheers

andrew

---

From: "Joshua D. Drake" <[email-masked]>
Date: Sat, 02 Sep 2006 18:54:02 -0700

> If we want something sooner rather than later, let's use something off 

http://pgbugs.commandprompt.com (still need to configure email).

Joshua D. Drake

> TIP 5: don't forget to increase your free space map settings
> 

--

---

From: Peter Eisentraut <[email-masked]>
Date: Sun, 3 Sep 2006 09:18:00 +0200

> http://pgbugs.commandprompt.com (still need to configure email).

Thank you for that.

I think an issue tracking system for patches and such may need to be 
distinct from a bug-tracking system such as bugzilla, but let's get one 
thing after another up.

---

From: "Joshua D. Drake" <[email-masked]>
Date: Sun, 03 Sep 2006 08:01:37 -0700

> distinct from a bug-tracking system such as bugzilla, but let's get one 

I made a mistake on this install and installed the dev version. I will 
fix this weekend.

I was also thinking the domain: tracker.postgresql.org?

Sincerely,

Joshua D. Drake

--

---

From: Peter Eisentraut <[email-masked]>
Date: Sun, 3 Sep 2006 18:27:48 +0200

> I was also thinking the domain: tracker.postgresql.org?

Certainly not for bugzilla.

---

From: Andrew Dunstan <[email-masked]>
Date: Sun, 03 Sep 2006 12:41:23 -0400

> He built it, and they came.

:-)

The difference is that the buildfarm could get going with effort only 
from me and a handful of early adopters, while a tracker probably needs 
higher level of initial community buyin. Nevertheless, I take your point.

> remarkable shortage of individuals stepping up to do the work.

You are right that it will need ongoing effort.

There are discussions happening about resources. Stay tuned.

cheers

andrew

---

From: Carlo Florendo <[email-masked]>
Date: Mon, 04 Sep 2006 16:24:52 +0800

> (142857, 3, 'for all the fish')

I just lurk here at pgsql-hackers.

That function would be very cool.

Thanks!

Best Regards,

Carlo Florendo
Astra Philippines Inc.
www.astra.ph

---

From: "Magnus Hagander" <[email-masked]>
Date: Mon, 4 Sep 2006 13:35:42 +0200

> Right now, the release notes show a list of all the significant
> items in each release, but it isn't available until the release,
> and it isn't complete (because it would be unreadable by ordinary
> users).  And there is no tracking of individual items in progress
> except by individual developers.  Magnus, for example, tracks Win32
> items, and Tom tracks backend stuff.  I track whatever no one else
> tracks.

Well, I *try*. I find myself not managing quite as well as I'd like ;-)
So I for one would definitely appreciate a tool that would help with
that - provided that the tool fits the development model, not the other
way around. 

(most other things seems to have been said already this time around, so
I won't repeat arguments others have made)

//Magnus

---

From: Neil Conway <[email-masked]>
Date: Wed, 06 Sep 2006 20:15:59 -0400

Bruce Momjian wrote:
>> to update the release notes when they do the patch.

I've suggested this more than once in the past -- I think it would be a 
clear improvement over the status quo. Updating the release notes 
incrementally would lead to more accurate and complete release notes: 
more accurate because the description for a feature would be written at 
the same time as the feature itself, and more complete because it would 
be harder to unintentionally omit discussion of a new feature. It would 
also help communicate to users what features will be in the next release 
of Postgres, which is certainly good from a PR point of view (a certain 
Swedish software company is very fond of talking about the features it 
will be adding in future releases, for example...) Finally, it would 
remove the need for a sequential scan of the CVS history, which I'm sure 
is pretty time-consuming, and delays the beta process.

> even more burden.  I would prefer they concentrate on documentation.

The first revision of a patch often doesn't include documentation 
updates, but in that case the submitter should be promptly told what 
they need to fix; I think the same would apply here. In practice, if 
you're committing a patch, you *should* understand it well enough to 
write a release note entry for it, so the burden might end up falling on 
committers, anyway.

-Neil

---

From: Kevin Brown <[email-masked]>
Date: Thu, 7 Sep 2006 12:05:15 -0700

> bit one way or the other.

Hmm...but if you're going to do that, why not do that now: push the
beta date back by, say, a month (or however long you had in mind) for
this cycle.  That way, the two major patches that are likely to be
dropped for this cycle stand a chance to make it into this release,
and you accomplish your goal of moving the dates a bit all at the same
time.

-- 
Kevin Brown					      [email-masked]

---

From: "Jim C. Nasby" <[email-masked]>
Date: Thu, 7 Sep 2006 14:33:05 -0500

> virtually zero as well as they have to compose some text for the commit

Another possibility would be to annotate commit messages that should get
added to the release notes and have something automagically cull those
out of CVS.
-- 
Jim C. Nasby, Database Architect                   [email-masked]
512.569.9461 (cell)                         http://jim.nasby.net

---

From: "Jim C. Nasby" <[email-masked]>
Date: Thu, 7 Sep 2006 14:36:31 -0500

> distinct from a bug-tracking system such as bugzilla, but let's get one 

Actually, I've generally found bugzilla to be a decent tool for
general-purpose tracking. Feature requests and what-not need a lot of
the same capabilities of a bug tracker, and having everything in one
tool is often very handy.
-- 
Jim C. Nasby, Database Architect                   [email-masked]
512.569.9461 (cell)                         http://jim.nasby.net

---

From: "Jim C. Nasby" <[email-masked]>
Date: Wed, 13 Sep 2006 00:40:29 -0500

> on by default for 8.3 (and early on so we get some good testing of it).

Can someone put this on the TODO, just so we (hopefully) don't forget
about it?
-- 
Jim C. Nasby, Database Architect                   [email-masked]
512.569.9461 (cell)                         http://jim.nasby.net