Subject: source documentation tool doxygen

From: Joachim Wieland <[email-masked]>
Date: Mon, 16 Jan 2006 09:49:36 +0100

I've created a browsable source tree "documentation", it's done with the
doxygen tool.

http://www.mcknight.de/pgsql-doxygen/cvshead/html/

There was a discussion about this some time ago, Jonathan Gardner proposed
it here:

http://archives.postgresql.org/pgsql-hackers/2004-03/msg00748.php

quite a few people found it useful but it somehow got stuck. The reason was
apparently that you'd have to tweak your comments in order to profit from
it as much as possible.

However I still think it's a nice-to-have among the online documentation
and it gives people quite new to the code the possibility to easily check
what is where defined and how... What do you think?

doxygen can also produce a pdf but I haven't succeeded in doing that so far,
pdflatex keeps bailing out. Has anybody else succeeded building this yet?

Joachim

-- 
Joachim Wieland                                              [email-masked]
C/ Usandizaga 12 1°B                                           ICQ: 37225940
20002 Donostia / San Sebastian (Spain)                     GPG key available

---

From: Thomas Hallgren <[email-masked]>
Date: Mon, 16 Jan 2006 12:38:16 +0100

I wish I've had this when I started working with PostgreSQL. This looks really good. Very 
useful indeed, even without the comments. What kind of changes are needed in order to get 
the comments in?

Regards,
Thomas Hallgren

---

From: Kim Bisgaard <[email-masked]>
Date: Mon, 16 Jan 2006 13:44:50 +0100

Try following the link (the Doxygen icon) - it has both a tutorial and
extensive doc.

Regards,
Kim Bisgaard

Thomas Hallgren wrote:

> I wish I've had this when I started working with PostgreSQL. This 
> looks really good. Very useful indeed, even without the comments. What 
> kind of changes are needed in order to get the comments in?
>
> Regards,
> Thomas Hallgren
>
> Joachim Wieland wrote:
>
> ---------------------------(end of broadcast)---------------------------
> TIP 2: Don't 'kill -9' the postmaster
>

---

From: "Andrew Dunstan" <[email-masked]>
Date: Mon, 16 Jan 2006 07:57:53 -0600 (CST)

Thomas Hallgren said:
> I wish I've had this when I started working with PostgreSQL. This looks
> really good. Very  useful indeed, even without the comments. What kind
> of changes are needed in order to get  the comments in?
>

I too have done this. But retrofitting Doxygen style comments to the
PostgreSQL source code would be a big undertaking. Maintaining it, which
would be another task for reviewers/committers, would also be a pain unless
there were some automated checking tool.

cheers

andrew

---

From: Thomas Hallgren <[email-masked]>
Date: Mon, 16 Jan 2006 19:21:52 +0100

Andrew,
Why not publish it as it stands today? Changing comments can be done in due time, no need to 
rush it. Or are the comments in some special format today that is used by some other tool?

What I'm trying to say is that for people like me, this would be very useful. Just clicking 
on a structure and see what file it's defined in is great. I don't care much if the comments 
are not perfect from day one (or ever for that matter). I can click on the file and read the 
comments right there if I want to.

If the comments have no special format today, well then perhaps the Doxygen format could be 
a good format to gradually adopt? No need to add checking etc. at this point. Perhaps later 
on (years from now) when most of the comments use that format.

Regards,
Thomas Hallgren

Andrew Dunstan wrote:
> TIP 3: Have you checked our extensive FAQ?
> 
>                http://www.postgresql.org/docs/faq
>

---

From: Neil Conway <[email-masked]>
Date: Mon, 16 Jan 2006 13:31:40 -0500

> there were some automated checking tool.

I don't think it would be all that painful. There would be no need to
convert the entire source tree to use proper Doxygen-style comments in
one fell swoop: individual files and modules can be converted whenever
anyone gets the inclination to do so. I don't think the maintenance
burden would be very substantial, either.

-Neil

---

From: Tom Lane <[email-masked]>
Date: Mon, 16 Jan 2006 13:51:38 -0500

> burden would be very substantial, either.

In the previous go-round on this topic, I seem to recall some concern
about side-effects, to wit reducing the readability of the comments for
ordinary non-doxygen code browsing.  I'd be quite against taking any
noticeable hit in that direction.

A quick look through the doxygen manual doesn't make it sound too
invasive, but I am worried about how well it will coexist with pgindent.
It seems both tools think they can dictate the meaning of the characters
immediately after /* of a comment block.

			regards, tom lane

---

From: Jonathan Gardner <[email-masked]>
Date: Mon, 16 Jan 2006 12:45:37 -0800

> immediately after /* of a comment block.

In my experimentation, I found that there are really three paths for using 
Doxygen on code.

(1) Do nothing to the code. Doxygen in this state works pretty well. You can 
browse the code and you can find references to things. Some of the comments 
are picked up correctly as-is. It does a really good job of formatting the 
code for HTML.

(2) Do minimally invasive changes to make Doxygen recognize comments better. 
Example: Move comments from behind to in-front, or vice-versa. Put an extra 
'*' after '/*' to denote special documentation. This makes Doxygen especially 
useful, because most things already have proper documentation and bringing 
that out to the person who browses Doxygen's output is very good. All of a 
sudden, we have a manual that reads like a book describing how PostgreSQL is 
put together PLUS we still have the marked-up code.

(3) Do maximally invasive changes to the comments. This includes things like 
using Doxygen-specific markup, putting in LaTeX for formulas, and putting in 
cross-references. This is a huge amount of work (initial + maintenance) and 
doesn't add much benefit beyond what (2) has. It makes the book read more 
like a book, but it was already pretty readable.

It's not a question of WHETHER to use Doxygen. You can use it whether or not 
anyone else wants to. It's really a question of how far do we bend backwards 
to accomodate Doxygen. I'd say we just stick to moving comments around and 
putting in the extra '*'s. Let's not encourage people to put in Doxygen 
markup that isn't obvious. If someone contributes code with Doxygen mark-up, 
we won't turn them down but we won't promise we'll keep their markup as 
pretty as it was when they submitted it.

-- 
Jonathan Gardner
[email-masked]

---

From: Michael Glaesemann <[email-masked]>
Date: Tue, 17 Jan 2006 06:51:15 +0900

> immediately after /* of a comment block.

I haven't looked at it yet, but might there not be a way to have a  
preprocessing step where the current comment format is converted to  
something doxygen-friendly? pg_indent2doxygen or something? Then the  
current comment style could be maintained and doxygen could get what  
it likes.

Michael Glaesemann
grzm myrealbox com

---

From: Joachim Wieland <[email-masked]>
Date: Tue, 17 Jan 2006 00:04:19 +0100

> current comment style could be maintained and doxygen could get what

It's not about getting in all the comments, it's mostly about comments
describing functions / makros / typedefs / structures / global variables
and the like.

Comments in function bodies will just stay I guess.

However we could try to find some de facto rules about comments to get most
of the matches: For example a comment starting on column 0 of the line is
probably one you're looking for vs. one that has some whitespace at the
beginning is not.

Of course this would still need manual review for every file.

Joachim

---

From: Bruce Momjian <[email-masked]>
Date: Mon, 16 Jan 2006 18:32:07 -0500 (EST)

Wow, looks great.  Is that URL stable?  Can we link to it from the
PostgreSQL developers page?

	http://www.postgresql.org/developer/coding

---------------------------------------------------------------------------

Joachim Wieland wrote:
> C/ Usandizaga 12 1?B                                           ICQ: 37225940

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  [email-masked]               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

---

From: "Marc G. Fournier" <[email-masked]>
Date: Mon, 16 Jan 2006 19:40:20 -0400 (AST)

the only question I have ... is there any way of improving that right 
index?  Love the 'detail pages', but somehow making the right index more 
'tree like' might make navigation a bit easier ...

On Mon, 16 Jan 2006, Bruce Momjian wrote:

> TIP 5: don't forget to increase your free space map settings
>

----
Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email: [email-masked]           Yahoo!: yscrappy              ICQ: 7615664

---

From: Michael Glaesemann <[email-masked]>
Date: Tue, 17 Jan 2006 09:34:06 +0900

On Jan 17, 2006, at 8:40 , Marc G. Fournier wrote:

>
> the only question I have ... is there any way of improving that  
> right index?  Love the 'detail pages', but somehow making the right  
> index more 'tree like' might make navigation a bit easier ...

Along those lines, I wonder if a CSS couldn't be worked up to  
integrate the look with the rest of the site.

Michael Glaesemann
grzm myrealbox com

---

From: Robert Treat <[email-masked]>
Date: Mon, 16 Jan 2006 19:42:35 -0500

This was my plan all along, just been waiting for someone to make it work with 
the postgresql code and then send instructions to the postgresql web team on 
how to set it up. 

Robert Treat

On Monday 16 January 2006 18:32, Bruce Momjian wrote:
> > There was a discussion about this some time ago, Jonathan Gardner
> > proposed it here:
> >
> > quite a few people found it useful but it somehow got stuck. The reason
> > was apparently that you'd have to tweak your comments in order to profit
> > from it as much as possible.
> >
> > doxygen can also produce a pdf but I haven't succeeded in doing that so
> > far, pdflatex keeps bailing out. Has anybody else succeeded building this
> > yet?
> >
> >
> >
> > Joachim
> >
> > --
> > Joachim Wieland                                             
> > [email-masked] C/ Usandizaga 12 1?B                                     
> >      ICQ: 37225940 20002 Donostia / San Sebastian (Spain)                
> >     GPG key available
> >

-- 
Robert Treat
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL

---

From: Tom Lane <[email-masked]>
Date: Tue, 17 Jan 2006 00:15:02 -0500

> PostgreSQL developers page?

The thing seems to have only the very vaguest grasp on whether it is
parsing C or C++ ... or should I say that it is convinced it is parsing
C++ despite all evidence to the contrary?  I'd be happier with the
pretty pictures if they had some connection to reality.

			regards, tom lane

---

From: Joachim Wieland <[email-masked]>
Date: Tue, 17 Jan 2006 11:58:45 +0100

> pretty pictures if they had some connection to reality.

True. Doxygen is developped for documenting C++ source code. There is
however an option OPTIMIZE_OUTPUT_FOR_C which is set. There are a few more
options for class and collaboration graphs as well as a switch to create a
graphical class hierarchy, all of them are turned on at the moment. So it
might get more concise but we'll have to play around with it to see what is
useful and what isn't.

Joachim

---

From: Joachim Wieland <[email-masked]>
Date: Tue, 17 Jan 2006 12:21:37 +0100

On Mon, Jan 16, 2006 at 07:42:35PM -0500, Robert Treat wrote:
> This was my plan all along, just been waiting for someone to make it work
> with the postgresql code and then send instructions to the postgresql web
> team on how to set it up. 

I volunteer to tell you after I've found out for myself ;-)

Some details: What I have put online on my website occupies about 240 MBs of
disk space and gets built in 1.5h on my PIII 800 Laptop. Removing useless
graphs this can be reduced to less than 200 MBs I imagine.

Do you want to put it on the postgresql.org site nevertheless? Is it too big
to be mirrored and should be recreated on every webserver? We might need
one copy for the last version of every major release as well as one for
cvs. The latter should get updated regularly of course but I figure it would
be sufficient to do that once a week...

Joachim

---

From: Joachim Wieland <[email-masked]>
Date: Tue, 17 Jan 2006 12:24:46 +0100

> integrate the look with the rest of the site.

Yes, it's stylesheet based. However I don't know yet to what extend you can
change the look. It allows for a custom header and footer as well. The
postgresql logo on top would be nice but the navigation menu on the left has
to be sacrificed for more space.

Joachim

---

From: "Andrew Dunstan" <[email-masked]>
Date: Tue, 17 Jan 2006 06:42:39 -0600 (CST)

Joachim Wieland said:
> Do you want to put it on the postgresql.org site nevertheless? Is it
> too big to be mirrored and should be recreated on every webserver? We
> might need one copy for the last version of every major release as well
> as one for cvs. The latter should get updated regularly of course but I
> figure it would be sufficient to do that once a week...
>

The overwhelming amount of development work gets done against HEAD. I would
start with a once a day run against HEAD, and possibly one against the
latest stable branch (currently REL8_1_STABLE in cvs). That would get you
99% of the possible benefit, I think. I don't see any virtue in doing it
against a release as opposed to a stable branch - this is to help
development efforts so it should be against what people should be basing
their development efforts on.

cheers

andrew

---

From: Tom Lane <[email-masked]>
Date: Tue, 17 Jan 2006 10:18:23 -0500

"Andrew Dunstan" <[email-masked]> writes:
> 99% of the possible benefit, I think.

I agree --- I see no reason for us to maintain such documentation for
anything except HEAD.  If somebody really wants documentation for a
stable branch, they can build it themselves.

			regards, tom lane