Subject: Switching to XML

From: David Blewett <[email-masked]>
Date: Fri, 08 Dec 2006 09:09:18 -0500

Hi All:

I read a thread from July explaining the current status of moving to XML
[1].

Peter, you mention that if someone has a clear need you would be open to
switching. Let me explain the problems I've encountered with SGML.

A few months ago, I split up the 8.1 pdf into 2 volumes so that I could
get a hardbound copy from Lulu. You can get those here [2-3] (I'm not
making any money, it's the pure cost of production). For 8.2, I wanted
to go deeper and be able to distribute the final version to bookstores.
In order to do that, each volume needs to be under 700 pages.

I started work on modifying the SGML to use a set of 3 volumes, split at
roughly 500 page intervals. I wanted to generate individual ToC's and
indexes for each volume. I started to modify the SGML to include a
"role" attribute for each indexterm, to tell what volume it was part of.
This was done with a simple sed script. When I went to generate the
actual indexes, I hit a brick wall. Apparently, the SGML toolchain
cannot handle typed indexes as described here [4]. Only the XML
toolchain currently handles them.

I am working with Joshua Drake on creating a Lulu account for the
fundraising group. We would then move the volumes I did for 8.1 to their
account, and raise the price so that any profit went to them. The
ability to order the 8.2 manual from bookstores could increase the
visibility of the project as a whole.

David Blewett

1.
http://groups.google.com/group/pgsql.docs/browse_thread/thread/37ff3a011bb705d7
2. http://www.lulu.com/content/455020
3. http://www.lulu.com/content/464949
4. http://www.xml.com/pub/a/2004/07/14/dbndx.html?page=3

---

From: "Joshua D. Drake" <[email-masked]>
Date: Fri, 08 Dec 2006 07:39:54 -0800

On Fri, 2006-12-08 at 09:09 -0500, David Blewett wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Hi All:
> 

To add to this, it would take minutes to transform XML versus the days
it takes the SGML.

Joshua D. Drake

> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.3 (MingW32)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
> 
> iD8DBQFFeXIOZmlc6wNjtLYRAo2yAKCZT1NbsklCd8djADdv48MuLELG7wCfYYG3
> ypUw8LU3g++GaY6Dz2vUi+I=
> =lC+c
> -----END PGP SIGNATURE-----
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Have you searched our list archives?
> 
>                http://archives.postgresql.org
> 
-- 

      === The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive  PostgreSQL solutions since 1997
             http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate

---

From: Peter Eisentraut <[email-masked]>
Date: Fri, 8 Dec 2006 19:02:11 +0100

David Blewett wrote:
> I started work on modifying the SGML to use a set of 3 volumes, split
> at roughly 500 page intervals. I wanted to generate individual ToC's
> and indexes for each volume. I started to modify the SGML to include
> a "role" attribute for each indexterm, to tell what volume it was
> part of. This was done with a simple sed script. When I went to
> generate the actual indexes, I hit a brick wall. Apparently, the SGML
> toolchain cannot handle typed indexes as described here [4]. Only the
> XML toolchain currently handles them.

But no one is forcing you to use "the SGML toolchain".  I requote the 
message you cited:

 First of all, moving to DocBook XML will not do anything in the way of 
 improving our output processing abilities.  Any tool that you can use 
 on DocBook SGML can also be used on DocBook XML and vice versa.

-- 
Peter Eisentraut
http://developer.postgresql.org/~petere/

---

From: Josh Berkus <[email-masked]>
Date: Fri, 08 Dec 2006 13:36:35 -0500

Peter,

>  on DocBook SGML can also be used on DocBook XML and vice versa.

As I said then, this is absolutely untrue.  OpenOffice.org, for example, 
works with DocBook XML but not SGML.  There are also a plethora of XML 
editing and publishing tools which can been used for Docbook XML which 
are not available for SGML.  A simple look at this page: 
http://wiki.docbook.org/topic/DocBookAuthoringTools
.... shows that there are more than twice as many authoring tools which 
support only XML as support SGML -- and that most of the tools which 
support SGML are out-of-maintenance.

--Josh Berkus

---

From: "Joshua D. Drake" <[email-masked]>
Date: Fri, 08 Dec 2006 11:07:51 -0800

On Fri, 2006-12-08 at 13:36 -0500, Josh Berkus wrote:
> As I said then, this is absolutely untrue. 

You are correct Josh. I am not sure why Peter ignores this simple fact.

>  OpenOffice.org, for example, 

Yep.

Sincerely,

Joshua D. Drake

> TIP 2: Don't 'kill -9' the postmaster
> 
--

---

From: Peter Eisentraut <[email-masked]>
Date: Fri, 8 Dec 2006 21:15:52 +0100

Josh Berkus wrote:
> As I said then, this is absolutely untrue.  OpenOffice.org, for
> example, works with DocBook XML but not SGML.

What does "works" mean?

> There are also a 
> plethora of XML editing and publishing tools which can been used for
> Docbook XML which are not available for SGML.

That has nothing to do with processing.

---

From: "Joshua D. Drake" <[email-masked]>
Date: Fri, 08 Dec 2006 12:21:34 -0800

> > example, works with DocBook XML but not SGML.

You can create, edit, convert, save, and open docbook xml in
OpenOfice.org.

> That has nothing to do with processing.

You are correct but XML does give us more modern and efficient toolsets
for processing. XSLT, FOP + Xerces for example.

Sincerely,

Joshua D. Drake

--

---

From: Peter Eisentraut <[email-masked]>
Date: Fri, 8 Dec 2006 21:58:05 +0100

> You can create, edit, convert, save, and open docbook xml in

Sure, there are more editing options with DocBook XML.  No one disputes 
that.  But the question at hand was about processing the DocBook.

> You are correct but XML does give us more modern and efficient
> toolsets for processing. XSLT, FOP + Xerces for example.

You can do that right now.

---

From: "Joshua D. Drake" <[email-masked]>
Date: Fri, 08 Dec 2006 13:26:00 -0800

> that.  But the question at hand was about processing the DocBook.

Yes which is generated from our use of SGML which is the core of this
problem and the core of the question as a whole.

SGML is making working with the documentation *harder*.

We have people that *DO NOT* contribute because of this SGML
requirement. They have what I consider extremely valid reasons, namely
it is dumb to require a writer to use emacs or write tags explictly.

Hell, the only reason I have even bothered to contribute what little I
have to the docs is because I wrote a book in SGML, thus it is a no
brainer to me. Others aren't so tortured as to have done the same.

There is a long standing support within the community to move to XML
including:

Josh Berkus
Josh Drake
Robert Treat
Andrew Dunslane
David Blewett
David Fetter
Devrim Gunduz
Darcy Buskermolen

And that is just from #postgresql

The french team also uses Docbook XML and they can generate a PDF in 30
minutes... it takes us DAYS because of the SGML.

How about we stopping using assembling and move to C shall we?

Sincerely,

Joshua D. Drake

--

---

From: Scott Marlowe <[email-masked]>
Date: Fri, 08 Dec 2006 15:45:57 -0600

> it is dumb to require a writer to use emacs or write tags explictly.

I'm one.  I had the vacuuming stuff about written, but couldn't figure
out the tool set this summer, and didn't have the time.  Does OO make it
pretty much painless to work with?  I couldn't find any tools that I
felt comfortable using for SGML / Docbooks.

---

From: "Joshua D. Drake" <[email-masked]>
Date: Fri, 08 Dec 2006 13:51:56 -0800

> felt comfortable using for SGML / Docbooks.

OO treats Docbook like a normal document. You will however loose styles
(like bold, italic). It does support tables, it understands
transformation from things like sect1 (from OO heading1) etc...

The style loss is to be expected because Docbook doesn't contain
representation data. That belongs to a style sheet.

Sincerely,

Joshua D. Drake

--

---

From: Scott Marlowe <[email-masked]>
Date: Fri, 08 Dec 2006 15:54:00 -0600

> representation data. That belongs to a style sheet.

and let me add that I'm not really anti-sgml docbook, I just couldn't
find a "starter set" for editing the stuff.  It seemed like everything I
found on docbook xml was written for people who already use docbook xml.

---

From: "Joshua D. Drake" <[email-masked]>
Date: Fri, 08 Dec 2006 14:03:03 -0800

> found on docbook xml was written for people who already use docbook xml.

Nor am I anti-sgml. I am however anti-noncontribution, if people are not
contributing because of our sgml but would if it is xml, that is a no
brainer.

Further, here is a real world problem that our toolset creates...

I take 5 minutes, change the stylesheet for SGML. I want to see what my
changes will look like... 3 days later, I will know.

That is stupid. If it was XML, it would be 30 minutes. That is a
workable timeframe.

Sincerely,

Joshua D. Drake

> TIP 7: You can help support the PostgreSQL project by donating at
> 
>                 http://www.postgresql.org/about/donate
> 
--

---

From: Oisin Glynn <[email-masked]>
Date: Fri, 08 Dec 2006 17:10:37 -0500

>>                 http://www.postgresql.org/about/donate
I am not an expert in any of this but would say the one time I had 
something I would have added to a doc (windows Vs *nix differences in 
copy command) I was in knots with the whole SGML thing and figured it 
would be quicker to just post an answer on the mailing list so it would 
be immortalized, it should not be so hard to contribute. I feel there 
are people who maybe cannot code or add in that way  but could add to 
docs if it was easier. I had an easier time setting up a pgAdmin build 
than figuring out the SGML stuff.

Oisin

---

From: Mario <[email-masked]>
Date: Fri, 8 Dec 2006 22:49:51 -0300

> And that is just from #postgresql

  And now in Chile, I wrote an app to translate postgres documentation
(l10n.postgresql.cl). I took SGML sources, transform to XML and to POT
files

  The results are good! but is it not the best. To move from SGML to
XML is a hard work.

-- 
http://www.advogato.org/person/mgonzalez/

---

From: Guillaume Lelarge <[email-masked]>
Date: Sat, 09 Dec 2006 09:21:12 +0100

> SGML is making working with the documentation *harder*.

+1

> brainer to me. Others aren't so tortured as to have done the same.

I'm not so sure it will help you find more contributors. I'm part of a
project which aims to translate HOWTO from TLDP. They don't find
contributors and we too have really hard times to find contributors
despite the fact we try to only use DocBook XML (TLDP use DocBook SGML,
DocBook XML and LinuxDoc formats).

Did you try to use OpenOffice.org with DocBook ? I tried once and it was
a complete disaster. But it was a long time ago. I will try again this
week-end.

> minutes... it takes us DAYS because of the SGML.

In fact, we need 15 minutes to build HTML files and 10 minutes to build
PDF file. To be completely honest, I don't seem to be able to build PDF
file for 8.2.0 release. I must have made a mistake (or perhaps a lot of
:) ).

Regards.

-- 
Guillaume.
<!-- http://abs.traduc.org/
     http://lfs.traduc.org/
     http://docs.postgresqlfr.org/ -->

---

From: Theo Kramer <[email-masked]>
Date: Sat, 09 Dec 2006 11:07:30 +0200

On Fri, 2006-12-08 at 13:26 -0800, Joshua D. Drake wrote:
<snip>

>From a total outsider's point of view I have to disagree. It took me a
couple of minutes to figure out how to make the tiny change I did the
other day by looking at the rest of the sgml (managed to get the diff
the wrong way around but not the sgml :/).

> it is dumb to require a writer to use emacs or write tags explictly.

Again would have to disagree - surely if someone really wants to
contribute they could provide their input in plain text, and someone on
the list could then integrate those contribs.

> brainer to me. Others aren't so tortured as to have done the same.

I would hate to hand edit the stuff generated by something like
OpenOffice.org.

> minutes... it takes us DAYS because of the SGML.

Here I agree - 30 minutes vs days is a good reason - as long as editing
with an ascii editor is not taken away.

-- 
Regards
Theo

---

From: Peter Eisentraut <[email-masked]>
Date: Sat, 9 Dec 2006 11:59:19 +0100

Joshua D. Drake wrote:
> I take 5 minutes, change the stylesheet for SGML. I want to see what
> my changes will look like... 3 days later, I will know.
>

I feel like I'm talkling to a wall here, but I'm going to say this one 
last time:

Any processing toolchain that you can use with DocBook XML you can use 
with DocBook SGML and vice versa.

If you know of a way to create a PDF off DocBook XML in 30 minutes, 
please tell us.

---

From: Tom Lane <[email-masked]>
Date: Sat, 09 Dec 2006 13:18:46 -0500

> support SGML are out-of-maintenance.

This is confusing authoring tools (ie, stuff for more or less WYSIWYG
editing of the document source) with output generation tools.

As for authoring tools, show me one that produces SGML or XML that's
reasonably readable, and I might worry about allowing people to use it.
Most of the ones I've seen would render the doc sources unreadable for
anyone not using an authoring tool (possibly even the very same
authoring tool).  We are not going to move in that direction
because it would piss off the people who do the bulk of the work now.

			regards, tom lane

---

From: Tom Lane <[email-masked]>
Date: Sat, 09 Dec 2006 13:32:41 -0500

>> minutes... it takes us DAYS because of the SGML.

Has anyone looked into actually fixing the performance problem?

oprofile results for jade trying to produce tex output from our docs are
suggestive of a localized performance issue:

samples  %        symbol name
2082917  98.5829  OpenJade_DSSSL::PairNodeListObj::nodeListFirst(OpenJade_DSSSL::EvalContext&, OpenJade_DSSSL::Interpreter&)
9713      0.4597  OpenJade_DSSSL::PairNodeListObj::nodeListRest(OpenJade_DSSSL::EvalContext&, OpenJade_DSSSL::Interpreter&)
5019      0.2375  OpenJade_DSSSL::AppendSosofoObj::traceSubObjects(Collector&) const
3571      0.1690  Collector::collect()
1938      0.0917  OpenJade_DSSSL::FlowObj::traceSubObjects(Collector&) const

I attached to the process with gdb and found it nested four thousand (!)
call levels deep in OpenJade_DSSSL::PairNodeListObj::nodeListFirst and
OpenJade_DSSSL::PairNodeListObj::nodeListRest calls.  Meanwhile, looking
at the output-so-far-emitted makes me think it was working on a fairly
large <programlisting> example.  The last little bit is:

{asis}\def\InputWhitespaceTreatment%
{preserve}}\Seq%
{}\Seq%
{}~~~~\endSeq{}/*
\Seq%
{}~~~~\endSeq{}~*~testlibpq2.c
\Seq%
{}~~~~\endSeq{}~*~~~~~~Test~of~the~asynchronous~notification~interface
\Seq%
{}~~~~\endSeq{}~*
\Seq%
{}~~~~\endSeq{}~*~Start~this~program,~then~from~psql~in~another~window~do
\Seq%
{}~~~~\endSeq{}~*~~~NOTIFY~TBL2;
\Seq%
{}~~~~\endSeq{}~*~Repeat~four~times~to~get~this~program~to~exit.
\Seq%
{}~~~~\endSeq{}~*
\Seq%
{}~~~~\endSeq{}~*~Or,~if~you~want~to~get~fancy,~try~this:
\Seq%
{}~~~~\endSeq{}~*~populate~a~database~with~th

What it looks like to me is that there is some bit of stupidity that is
producing a deeply nested list representation of a <programlisting>
section, probably one list level per character in the text, making the
runtime O(N^2) or worse in the length of the <programlisting>.  (The
particular example it's stuck on here is about 10K characters.)

Since jade does not go into this kind of spiral when producing html
output from the same sources, I suggest that it's not jade's fault,
but rather crummy coding in the sgml-to-tex conversion scripts it's
using.  I don't know enough about those to know where to look, but maybe
someone here does?

			regards, tom lane

---

From: "Joshua D. Drake" <[email-masked]>
Date: Sat, 09 Dec 2006 12:01:39 -0800

> I would hate to hand edit the stuff generated by something like

As would I.

> with an ascii editor is not taken away.

I would never expect to take editing away.. I personally would likely
just use bluefish, not Openoffice.

Sincerely,

Joshua D. Drake

--

---

From: "Joshua D. Drake" <[email-masked]>
Date: Sat, 09 Dec 2006 13:05:01 -0800

> If you know of a way to create a PDF off DocBook XML in 30 minutes, 

See the post on this thread from :

Guillaume Lelarge

10 minutes to generate a pdf.

Sincerely,

Joshua D. Drake

--

---

From: Peter Eisentraut <[email-masked]>
Date: Sun, 10 Dec 2006 00:32:54 +0100

> 10 minutes to generate a pdf.

Right.  So why don't we just use that?

---

From: "Joshua D. Drake" <[email-masked]>
Date: Sat, 09 Dec 2006 17:51:38 -0800

> Right.  So why don't we just use that?

How about the fact that we are putting undue burden on our fellow
regional projects?

How about it isn't in english?

Joshua D. Drake

--

---

From: Peter Eisentraut <[email-masked]>
Date: Sun, 10 Dec 2006 03:13:49 +0100

Joshua D. Drake wrote:
> > > > If you know of a way to create a PDF off DocBook XML in 30
> > > > minutes, please tell us.
> > >

Well, I didn't actually find a post titled "10 minutes to generate a 
pdf." anywhere, so I assumed you referred to another one of his posts 
which described how they used FOP to build PDF documentation.  That 
method does not burden anyone unduly and it is in English.  But maybe 
we're not talking about the same thing.

---

From: Peter Eisentraut <[email-masked]>
Date: Sun, 10 Dec 2006 03:15:46 +0100

Tom Lane wrote:
> using.

Right.  I fixed that, so now it takes about 15 minutes to build the 
whole thing.

---

From: Mark Kirkwood <[email-masked]>
Date: Sun, 10 Dec 2006 15:54:35 +1300

+1

With regards to the current setup discouraging contribution - I've done 
one (or maybe two) doc submissions and ISTR that it was not very 
difficult - even as a complete SGML newbie - to get up and running, make 
my changes and build the (HTML) docs.

Cheers

Mark

---

From: Tom Lane <[email-masked]>
Date: Sat, 09 Dec 2006 22:19:05 -0500

> Right.  I fixed that, so now it takes about 15 minutes to build the 

Actually, I just finished fixing what seems to be the underlying problem
in jade: it's got a spectacularly bad implementation of linked lists.
In PG-code terms, if the list is built by successive lappends(), then
both lfirst() and lnext() take O(N) time for an N-element list, thus
iterating through the list in the usual way is O(N^2) ... and not even
with a reasonably small multiplier, because it stresses the hell out of
memory allocation and garbage collection while it's at it.  (Which may
well mean that it's really more like O(N^3), since the garbage collector
will iterate over every allocated object every so often.)

With the patch it takes me about 5 minutes to do the jade step of the
PDF build, using this morning's SGML sources.  (I don't know how to set
the TeX configuration to get the pdfjadetex steps to go through, so I
dunno about total time.)

However, I have no idea what it'll take to get this patch propagated
into the copies people actually use, so your fix sounds good for the
short term.

			regards, tom lane

diff -cr openjade-1.3.2.orig/style/ELObj.cxx openjade-1.3.2/style/ELObj.cxx
*** openjade-1.3.2.orig/style/ELObj.cxx	Fri Jan 11 10:48:38 2002
--- openjade-1.3.2/style/ELObj.cxx	Sat Dec  9 21:58:36 2006
***************
*** 1044,1049 ****
--- 1044,1054 ----
    return new (interp) ReverseNodeListObj(this);
  }
  
+ NodeListObj *NodeListObj::nodeListAppend(NodeListObj *newTail, Interpreter &interp)
+ {
+   return new (interp) HeadNodeListObj(this, newTail, interp);
+ }
+ 
  long NodeListObj::nodeListLength(EvalContext &context, Interpreter &interp)
  {
    NodeListObj *nl = this;
***************
*** 1217,1236 ****
  
  NodeListObj *PairNodeListObj::nodeListRest(EvalContext &context, Interpreter &interp)
  {
!   if (!head_ || !head_->nodeListFirst(context, interp))
      return tail_->nodeListRest(context, interp);
    NodeListObj *tem = head_->nodeListRest(context, interp);
    ELObjDynamicRoot protect(interp, tem);
!   return new (interp) PairNodeListObj(tem, tail_);
  }
  
  NodeListObj *PairNodeListObj::nodeListChunkRest(EvalContext &context, Interpreter &interp, bool &chunk)
  {
!   if (!head_ || !head_->nodeListFirst(context, interp))
      return tail_->nodeListChunkRest(context, interp, chunk);
    NodeListObj *tem = head_->nodeListChunkRest(context, interp, chunk);
    ELObjDynamicRoot protect(interp, tem);
!   return new (interp) PairNodeListObj(tem, tail_);
  }
  
  void PairNodeListObj::traceSubObjects(Collector &c) const
--- 1222,1260 ----
  
  NodeListObj *PairNodeListObj::nodeListRest(EvalContext &context, Interpreter &interp)
  {
!   if (!head_)
!     return tail_->nodeListRest(context, interp);
!   if (!head_->nodeListFirst(context, interp)) {
!     head_ = 0;
      return tail_->nodeListRest(context, interp);
+   }
    NodeListObj *tem = head_->nodeListRest(context, interp);
    ELObjDynamicRoot protect(interp, tem);
!   if (!tem || !tem->nodeListFirst(context, interp))
!     return tail_;
!   return tem->nodeListAppend(tail_, interp);
  }
  
  NodeListObj *PairNodeListObj::nodeListChunkRest(EvalContext &context, Interpreter &interp, bool &chunk)
  {
!   if (!head_)
!     return tail_->nodeListChunkRest(context, interp, chunk);
!   if (!head_->nodeListFirst(context, interp)) {
!     head_ = 0;
      return tail_->nodeListChunkRest(context, interp, chunk);
+   }
    NodeListObj *tem = head_->nodeListChunkRest(context, interp, chunk);
    ELObjDynamicRoot protect(interp, tem);
!   if (!tem || !tem->nodeListFirst(context, interp))
!     return tail_;
!   return tem->nodeListAppend(tail_, interp);
! }
! 
! NodeListObj *PairNodeListObj::nodeListAppend(NodeListObj *newTail, Interpreter &interp)
! {
!   NodeListObj *tem = new (interp) PairNodeListObj(tail_, newTail);
!   ELObjDynamicRoot protect(interp, tem);
!   return new (interp) PairNodeListObj(head_, tem);
  }
  
  void PairNodeListObj::traceSubObjects(Collector &c) const
***************
*** 1239,1244 ****
--- 1263,1384 ----
    c.trace(tail_);
  }
  
+ CellNodeListObj::CellNodeListObj(NodeListObj *head)
+ : head_(head), next_(0)
+ {
+   hasSubObjects_ = 1;
+ }
+ 
+ NodePtr CellNodeListObj::nodeListFirst(EvalContext &context, Interpreter &interp)
+ {
+   if (head_) {
+     NodePtr nd(head_->nodeListFirst(context, interp));
+     if (nd)
+       return nd;
+     head_ = 0;
+   }
+   if (next_)
+     return next_->nodeListFirst(context, interp);
+   return NodePtr();
+ }
+ 
+ NodeListObj *CellNodeListObj::nodeListRest(EvalContext &context, Interpreter &interp)
+ {
+   if (!head_) {
+     if (next_)
+       return next_->nodeListRest(context, interp);
+     return interp.makeEmptyNodeList();
+   }
+   if (!head_->nodeListFirst(context, interp)) {
+     head_ = 0;
+     if (next_)
+       return next_->nodeListRest(context, interp);
+     return interp.makeEmptyNodeList();
+   }
+   NodeListObj *tem = head_->nodeListRest(context, interp);
+   ELObjDynamicRoot protect(interp, tem);
+   if (!tem || !tem->nodeListFirst(context, interp)) {
+     if (next_)
+       return next_;
+     return interp.makeEmptyNodeList();
+   }
+   if (next_)
+     return tem->nodeListAppend(next_, interp);
+   return tem;
+ }
+ 
+ NodeListObj *CellNodeListObj::nodeListChunkRest(EvalContext &context, Interpreter &interp, bool &chunk)
+ {
+   if (!head_) {
+     if (next_)
+       return next_->nodeListChunkRest(context, interp, chunk);
+     chunk = 0;
+     return interp.makeEmptyNodeList();
+   }
+   if (!head_->nodeListFirst(context, interp)) {
+     head_ = 0;
+     if (next_)
+       return next_->nodeListChunkRest(context, interp, chunk);
+     chunk = 0;
+     return interp.makeEmptyNodeList();
+   }
+   NodeListObj *tem = head_->nodeListChunkRest(context, interp, chunk);
+   ELObjDynamicRoot protect(interp, tem);
+   if (!tem || !tem->nodeListFirst(context, interp)) {
+     if (next_)
+       return next_;
+     return interp.makeEmptyNodeList();
+   }
+   if (next_)
+     return tem->nodeListAppend(next_, interp);
+   return tem;
+ }
+ 
+ void CellNodeListObj::traceSubObjects(Collector &c) const
+ {
+   c.trace(head_);
+   c.trace(next_);
+ }
+ 
+ HeadNodeListObj::HeadNodeListObj(NodeListObj *first, NodeListObj *second, Interpreter &interp)
+ : first_(0), last_(0)
+ {
+   hasSubObjects_ = 1;
+   ELObjDynamicRoot protect(interp, this);
+   first_ = new (interp) CellNodeListObj(first);
+   last_ = new (interp) CellNodeListObj(second);
+   first_->next_ = last_;
+ }
+ 
+ NodePtr HeadNodeListObj::nodeListFirst(EvalContext &context, Interpreter &interp)
+ {
+   return first_->nodeListFirst(context, interp);
+ }
+ 
+ NodeListObj *HeadNodeListObj::nodeListRest(EvalContext &context, Interpreter &interp)
+ {
+   return first_->nodeListRest(context, interp);
+ }
+ 
+ NodeListObj *HeadNodeListObj::nodeListChunkRest(EvalContext &context, Interpreter &interp, bool &chunk)
+ {
+   return first_->nodeListChunkRest(context, interp, chunk);
+ }
+ 
+ NodeListObj *HeadNodeListObj::nodeListAppend(NodeListObj *newTail, Interpreter &interp)
+ {
+   CellNodeListObj *tem = new (interp) CellNodeListObj(newTail);
+   last_->next_ = tem;
+   last_ = tem;
+   return this;
+ }
+ 
+ void HeadNodeListObj::traceSubObjects(Collector &c) const
+ {
+   c.trace(first_);
+   // first_ will recurse to the rest
+ }
+ 
  ReverseNodeListObj::ReverseNodeListObj(NodeListObj *nl)
  : nl_(nl), reversed_(0)
  {
diff -cr openjade-1.3.2.orig/style/ELObj.h openjade-1.3.2/style/ELObj.h
*** openjade-1.3.2.orig/style/ELObj.h	Fri Jan 11 10:48:38 2002
--- openjade-1.3.2/style/ELObj.h	Sat Dec  9 20:14:23 2006
***************
*** 426,431 ****
--- 426,432 ----
    virtual NodeListObj *nodeListChunkRest(EvalContext &, Interpreter &, bool &chunk);
    virtual NodePtr nodeListRef(long, EvalContext &, Interpreter &);
    virtual NodeListObj *nodeListReverse(EvalContext &, Interpreter &);
+   virtual NodeListObj *nodeListAppend(NodeListObj *, Interpreter &);
    virtual long nodeListLength(EvalContext &, Interpreter &);
    virtual bool suppressError();
  };
***************
*** 493,504 ****
--- 494,533 ----
    NodePtr nodeListFirst(EvalContext &, Interpreter &);
    NodeListObj *nodeListRest(EvalContext &, Interpreter &);
    NodeListObj *nodeListChunkRest(EvalContext &, Interpreter &, bool &);
+   NodeListObj *nodeListAppend(NodeListObj *, Interpreter &);
    void traceSubObjects(Collector &) const;
  private:
    NodeListObj *head_; // may be null
    NodeListObj *tail_;
  };
  
+ class CellNodeListObj : public NodeListObj {
+ public:
+   CellNodeListObj(NodeListObj *);
+   NodePtr nodeListFirst(EvalContext &, Interpreter &);
+   NodeListObj *nodeListRest(EvalContext &, Interpreter &);
+   NodeListObj *nodeListChunkRest(EvalContext &, Interpreter &, bool &);
+   void traceSubObjects(Collector &) const;
+ private:
+   NodeListObj *head_; // may be null
+   CellNodeListObj *next_; // may be null
+   friend class HeadNodeListObj; // needs access to next_ link
+ };
+ 
+ class HeadNodeListObj : public NodeListObj {
+ public:
+   HeadNodeListObj(NodeListObj *, NodeListObj *, Interpreter &);
+   NodePtr nodeListFirst(EvalContext &, Interpreter &);
+   NodeListObj *nodeListRest(EvalContext &, Interpreter &);
+   NodeListObj *nodeListChunkRest(EvalContext &, Interpreter &, bool &);
+   NodeListObj *nodeListAppend(NodeListObj *, Interpreter &);
+   void traceSubObjects(Collector &) const;
+ private:
+   // there are always at least two cells in the list, hence first_ != last_
+   CellNodeListObj *first_;
+   CellNodeListObj *last_;
+ };
+ 
  inline
  bool ELObj::equal(ELObj &obj1, ELObj &obj2)
  {
diff -cr openjade-1.3.2.orig/style/primitive.cxx openjade-1.3.2/style/primitive.cxx
*** openjade-1.3.2.orig/style/primitive.cxx	Wed Mar 13 07:21:22 2002
--- openjade-1.3.2/style/primitive.cxx	Sat Dec  9 21:48:01 2006
***************
*** 3846,3870 ****
  
  DEFPRIMITIVE(NodeList, argc, argv, context, interp, loc)
  {
!   if (argc == 0)
      return interp.makeEmptyNodeList();
!   int i = argc - 1;
!   NodeListObj *nl = argv[i]->asNodeList();
    if (!nl)
      return argError(interp, loc,
! 		    InterpreterMessages::notANodeList, i, argv[i]);
!   if (i > 0) {
!     ELObjDynamicRoot protect(interp, nl);
!     for (;;) {
!       i--;
        NodeListObj *tem = argv[i]->asNodeList();
        if (!tem)
!         return argError(interp, loc,
! 	                InterpreterMessages::notANodeList, i, argv[i]);
!       nl = new (interp) PairNodeListObj(tem, nl);
!       if (i == 0)
! 	break;
!       protect = nl;
      }
    }
    return nl;
--- 3846,3868 ----
  
  DEFPRIMITIVE(NodeList, argc, argv, context, interp, loc)
  {
!   if (argc <= 0)
      return interp.makeEmptyNodeList();
!   NodeListObj *nl = argv[0]->asNodeList();
    if (!nl)
      return argError(interp, loc,
! 		    InterpreterMessages::notANodeList, 0, argv[0]);
!   if (argc > 1) {
!     ELObjDynamicRoot protect(interp);
!     ELObjDynamicRoot protect2(interp);
!     for (int i = 1; i < argc; i++) {
!       protect = nl;
        NodeListObj *tem = argv[i]->asNodeList();
        if (!tem)
! 	return argError(interp, loc,
! 			InterpreterMessages::notANodeList, i, argv[i]);
!       protect2 = tem;
!       nl = nl->nodeListAppend(tem, interp);
      }
    }
    return nl;

---

From: Tom Lane <[email-masked]>
Date: Sat, 09 Dec 2006 23:53:09 -0500

I wrote:
> PDF build, using this morning's SGML sources.

Ooops, that was with an -O0 build for debugging.  After rebuilding with
the normal -O2 optimization, it takes about 70 seconds, just about
exactly the same as for the HTML-output case.

			regards, tom lane

---

From: "Joshua D. Drake" <[email-masked]>
Date: Sat, 09 Dec 2006 21:13:27 -0800

> we're not talking about the same thing.

I meant, *their* xml file is not english. So it doesn't do any good.
Remember this whole current thread started because we are trying to
create print ready documentation, which we currently do not have.

Sincerely,

Joshua D. Drake

--

---

From: Tom Lane <[email-masked]>
Date: Sun, 10 Dec 2006 00:24:57 -0500

> the TeX configuration to get the pdfjadetex steps to go through, so I

OK, I read Peter's notes about suitable TeX configuration, and with
that plus suitably patched openjade I get this from a standing start:

$ time make postgres.pdf
... much output ...
Output written on postgres.pdf (1670 pages, 13561269 bytes).
Transcript written on postgres.log.

real    8m23.949s
user    8m14.263s
sys     0m9.241s
$

But my goodness it produces a lot of TeX warnings :-(.  I can understand
that no one was gonna look into those with a three-day turnaround time,
but maybe now we can think about cleaning them up?  Personally I
haven't used TeX/LaTeX in anger in about ten years, but most of what
scrolled past looked easily fixable, and at least some of it would
translate to visible ugliness in the output.

			regards, tom lane

---

From: "Joshua D. Drake" <[email-masked]>
Date: Sat, 09 Dec 2006 21:27:25 -0800

> > SGML is making working with the documentation *harder*.

I believe this point is being overlooked is people's willingness to code
a solution ;).

Guillaume could you please tell us *why* you ported the SGML to XML
before doing your translation and processing? Perhaps that will carry
more weight to this discussion then my hand waving.

Sincerely,

Joshua D. Drake

--

---

From: "Joshua D. Drake" <[email-masked]>
Date: Sat, 09 Dec 2006 21:28:07 -0800

Mario could you please tell us *why* you ported the SGML to XML
before doing your translation and processing? Perhaps that will carry
more weight to this discussion then my hand waving.

Sincerely,

Joshua D. Drake

--

---

From: David Fetter <[email-masked]>
Date: Sat, 9 Dec 2006 21:39:26 -0800

> > SGML is making working with the documentation *harder*.

Thanks to Peter and Tom for making the PDF build faster, but the more
general problem, which is that the SGML does not actually do the same
things that XML does, no matter how many times Peter so asserts,
remains.

In addition to the long-standing problem that there is no way to edit
the SGML docs with any known GUI tool, we have a particular use case,
namely producing a multi-volume set suitable for printing as books.

> > requirement.  They have what I consider extremely valid reasons,
> > namely it is dumb to require a writer to use emacs or write tags
> > explictly.

Peter, if you have a working example of a GUI tool that can be used
with the SGML source in its current form, the burden of proof is on
you to demonstrate it.  Another flat assertion from you of some kind
of mathematical equivalence between SGML and XML will *not* do the
trick.

> > Hell, the only reason I have even bothered to contribute what
> > little I have to the docs is because I wrote a book in SGML, thus
> > it is a no brainer to me. Others aren't so tortured as to have
> > done the same.
> 
> I'm not so sure it will help you find more contributors. I'm part of
> a project which aims to translate HOWTO from TLDP. They don't find
> despite the fact we try to only use DocBook XML (TLDP use DocBook
> SGML, DocBook XML and LinuxDoc formats).
> 
> Did you try to use OpenOffice.org with DocBook ? I tried once and it
> was a complete disaster. But it was a long time ago. I will try
> again this week-end.

I gave it a try post-patches, and it's still a disaster 3284 pages of
un-rendered XML.

> > The french team also uses Docbook XML and they can generate a PDF
> > in 30 minutes... it takes us DAYS because of the SGML.
> 
> In fact, we need 15 minutes to build HTML files and 10 minutes to
> build PDF file. To be completely honest, I don't seem to be able to
> build PDF file for 8.2.0 release. I must have made a mistake (or
> perhaps a lot of :) ).

It'll be nice to have the document building cycle shorter, but the
point here is that we need to enter the 21st century.  That Tom found
a need to fork a document tool, i.e. take ownership of a whole large
piece of software, that being what forking means, is a neon sign that
means, "we're stuck with broken tools."

Cheers,
D
-- 
David Fetter <[email-masked]> http://fetter.org/
phone: +1 415 235 3778        AIM: dfetter666
                              Skype: davidfetter

Remember to vote!

---

From: "Joshua D. Drake" <[email-masked]>
Date: Sat, 09 Dec 2006 21:56:35 -0800

> namely producing a multi-volume set suitable for printing as books.

Well yes and no. There isn't really any good WYSIWYG tools, but there
are plenty of graphical validating editors for Docbook XML. I actually
don't agree with JoshBs argument on this one as most major FOSS (Gnome,
KDE, Linux) have moved to Docbook XML without the need of graphical
editors.

I do think the tools in general can be a little daunting and perhaps a
pgfoundry project that had a perl script that would download and
configure a self contained jail of tools for working with the PostgreSQL
docs would be good.

> of mathematical equivalence between SGML and XML will *not* do the

I have to agree here, the reality is that the tools that are designed to
work with XML work with XML. Yes you *can* in theory make them work with
SGML, and some may work directly out of the box (Emacs would for editing
for example), not all of them would.

> I gave it a try post-patches, and it's still a disaster 3284 pages of

Well keep in mind that all XML is unrendered :). OpenOffice will do
things like this:

<emphasis>Hi</emphasis>
<para></para>
<para></para>
Bye

Which equates to:

Hi

Bye

We don't do that in the PostgreSQL sgml (thank goodness) so it comes out
really narly :). However it wouldn't take much to clean up small
sections for people.

> means, "we're stuck with broken tools."

That is a big concern for me as well. I don't think Tom expects us to
maintain the code, and we can certainly try to get the patch submitted
to the open jade project which would definitely help. However, the
movement of the open jade projects appears basically dead.

Moving to XML based tools does give us a more modern and developed
toolset.

Sincerely,

Joshua D. Drake

--

---

From: Tom Lane <[email-masked]>
Date: Sun, 10 Dec 2006 01:02:46 -0500

David Fetter <[email-masked]> writes:
> the SGML docs with any known GUI tool,

The point not in evidence is that there exists a GUI tool we'd accept
for editing XML-format docs.  Can you point to some that don't mess up
XML source code to the point of being unreadable?

> ... That Tom found

Um, I spend all day every day on making sometimes-comparable
improvements in Postgres.  Will you abandon Postgres as soon as
someone points out a(nother) serious performance bug in it?

As for "fork", I have no intention of forking anything --- that patch
is already submitted upstream.

			regards, tom lane

---

From: Peter Eisentraut <[email-masked]>
Date: Sun, 10 Dec 2006 09:43:38 +0100

Joshua D. Drake wrote:
> I do think the tools in general can be a little daunting and perhaps
> a pgfoundry project that had a perl script that would download and
> configure a self contained jail of tools for working with the
> PostgreSQL docs would be good.

Do you find any deficiencies in the setup instructions we have in the 
documentation?

---

From: Peter Eisentraut <[email-masked]>
Date: Sun, 10 Dec 2006 10:08:09 +0100

> of mathematical equivalence between SGML and XML will *not* do the

Well, most of you are obviously led into twisting every single thread on 
this mailing list related to some sort of tool trouble into a thread 
that SGML is completely unusable for unrelated reasons such as that 
there are no editing tools or that it takes too long to build.  I have 
no interest in these sorts of uninformed discussions.

The question here was by David Blewett about creating multiple indexes.  
Has anyone bothered to look into that?  My experimentation, by the way, 
shows that it works exactly as described in the cited document.

> means, "we're stuck with broken tools."

And which nonbroken tools for creating print output do you have up your 
sleeve in the new XML world?  And what is so wonderous about these 
tools that we can't use them today?

---

From: Guillaume Lelarge <[email-masked]>
Date: Sun, 10 Dec 2006 11:38:30 +0100

> more weight to this discussion then my hand waving.

I use DocBook XML a lot more than SGML on some translation's projects.

I switch to DocBook XML because I find it easier to customize
stylesheets. Take a look at DSSSL and take a look at XSLT : DSSL just
hurts my eyes, I found XSLT more simple to understand *and* customize.
There's been a lot of posts on this subject on some of the Linux From
Scratch project. And they've done a pretty good job with HTML and PDF
builts. So I took their stylesheets, Makefiles. I tried to understand
what they do, customize a little more to get what the french team have now.

So first argument, the one that makes me switch, is ease of
customization. Second one is also really important to me. XSLT
stylesheets and xsltproc are under active development. You can take a
look here :
  http://sourceforge.net/project/showfiles.php?group_id=21935#files
DSSSL 1.79 has been packaged one 2004-11-03, XSL 1.71.1 has been
released on 2006-10-19. And don't get me started on tools. I've seen
recent releases of xsltproc, none of jade, openjade or any of those tools.

Last point : just look at other free projects. Take a look at Gnome and
KDE for example. They all use DocBook XML with customized stylesheets. I
don't mean that because lots of projects use DocBook XML, they all are
right and us wrong. I don't even mean that we should switch to follow
new buzz. I just mean people use it, publish information on what they do
and how they do it. Can someone find more informations on how to build
documentation with SGML than he can with XML ?

I don't want to say we can't do all these with DocBook SGML. Tom fixed
some stuff on openjade to make it process quickier. Great. We can also
customize DSSSL stylesheets. We can, we can... yes we can do a lot of
things with SGML and DSSSL. When I say "We", I should say "You" because
I don't want to get this burden. I mean, fixing a tool is great... but I
really prefer to see Tom working on PostgreSQL new astounding features
than on an old unmaintained and unused tool. I can't code as Tom. But I
can help on documentation with tools, but tools that are maintained. You
want to put more documentation on the manual but don't want to write it
with DocBook XML ? I'm OK with it. Just give me a plain ASCII file or an
openoffice file, I will handle the tagging, check correctness of the XML
file, build the documentation.

I say "I" but perhaps "we" would be more appropriate.

Perhaps it is time to create a real documentation team. More people will
be able to get involved with this great project.

---

From: Alvaro Herrera <[email-masked]>
Date: Sun, 10 Dec 2006 09:20:34 -0300

> more weight to this discussion then my hand waving.

The reason is that he uses xml2pot, which AFAIK only handles XML and not
SGML.

-- 
Alvaro Herrera                                http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

---

From: David Blewett <[email-masked]>
Date: Sun, 10 Dec 2006 09:40:54 -0500

Guillaume Lelarge wrote:
<snip>

> Second one is also really important to me. XSLT

<snip>

> than on an old unmaintained and unused tool. 

I think this is a good point. I've heard a saying in #postgresql on
IRC: "First rate database, second rate tools..." I think that is a
shame.

David

---

From: David Blewett <[email-masked]>
Date: Sun, 10 Dec 2006 09:44:05 -0500

> > shows that it works exactly as described in the cited document.

Peter:

Can you explain the process to me then? I couldn't figure out how to
 generate different HTML.index files, or how to split the one
generated into different volumeindex.sgml files. Can you enlighten me?

David

PS:
Sorry if this gets posted twice, I'm having issues this morning...

---

From: "Joshua D. Drake" <[email-masked]>
Date: Sun, 10 Dec 2006 08:03:54 -0800

> Do you find any deficiencies in the setup instructions we have in the 

Where are they in the documentation?

Sincerely,

Joshua D. Drake

--

---

From: "Joshua D. Drake" <[email-masked]>
Date: Sun, 10 Dec 2006 08:10:23 -0800

> no interest in these sorts of uninformed discussions.

Bah! That is certainly not what is happening here. No one has said that
they are completely unusable. Even I have said they are usable but that
they can be daunting.

The point of this discussion comes down to some very simple things.

1. Tools. SGML tools are not as actively developed as the XML ones. No
you can not use XML tools with SGML as effectively.

If I am wrong on #1 Peter, fine. Prove it.  You keep telling us we are
wrong but are unwilling to share where or how. 

2. We have two regional projects that take a lot of hard earned time to
work around the lackluster state of our source documentation, namely
SGML. This hard earned time would better be spent on other things and
could be if you would stop stonewalling and let us move to XML.

3. You are the only one that is aruging *for* staying SGML. 

> shows that it works exactly as described in the cited document.

Which cited document? However this still doesn't solve #2

Lastly, even if I am completely cranked on #1 and #3, #2 is reason
enough to move to XML regardless of your opinion.

We need to move to XML for the regional projects.

Sincerely,

Joshua D. Drake

--

---

From: Peter Eisentraut <[email-masked]>
Date: Sun, 10 Dec 2006 17:37:09 +0100

Joshua D. Drake wrote:
> 1. Tools. SGML tools are not as actively developed as the XML ones.
> No you can not use XML tools with SGML as effectively.
>
> If I am wrong on #1 Peter, fine. Prove it.  You keep telling us we
> are wrong but are unwilling to share where or how.

Well, such statements are quite impossible to reason about because 
a "tool" can be just about anything.  There is no doubt that SGML tools 
are not as actively developed as the XML ones.  That could, for 
example, be because SGML is much older and stabilized and the tools 
have matured.

But none of this interests me unless someone can come up with a specific 
completion of the following sentence: "We need to do $ACTION because 
that allows us to use tool $TOOL, which is more efficient at doing 
$TASK than the currently used tool."

The only completion I have heard about so far is ACTION='switch source 
to XML', TOOL='OpenOffice', and TASK='editing'.  Others have pointed 
out that while OpenOffice can indeed edit DocBook XML, it wouldn't be 
very useful for that purpose in our case.  But I'm interested in 
hearing more about that.

> 2. We have two regional projects that take a lot of hard earned time
> to work around the lackluster state of our source documentation,
> namely SGML.

I know of one regional project that has done documentation work, the 
French one.  (I'm sorry that I missed the other one.)  Their reason for 
moving to XML was (a) to use FOP for (b) producing print output more 
efficiently.  But you can use FOP right now, so (a) is not a good 
reason, and we have fixed the print output generation now, so (b) isn't 
a reason either.

> This hard earned time would better be spent on other 
> things and could be if you would stop stonewalling and let us move to
> XML.

I'm not stonewalling anything.  If someone wants to prepare a case for 
moving to XML, be my guest.  I'm just here to debunk the reasons for 
moving to XML which are wrong.

---

From: Peter Eisentraut <[email-masked]>
Date: Sun, 10 Dec 2006 17:38:31 +0100

Joshua D. Drake wrote:
> > Do you find any deficiencies in the setup instructions we have in
> > the documentation?
>

Appendix G

---

From: "Joshua D. Drake" <[email-masked]>
Date: Sun, 10 Dec 2006 08:49:41 -0800

> example, be because SGML is much older and stabilized and the tools 

O.k. again... what tools? Either provide some specific solutions or back
out of the conversation.

The standard tools used in the FOSS arena (specifically linux) for
Docbook are OpenJade and DSSSL. OpenJade sees very little development. 

What tools do you use? Are they the same listed in Appendix G (I am
assuming so)

> $TASK than the currently used tool."

We need to do migrate our docs to XML because that allows us to use
xml-po, which is more efficient at helping translation than the
currently used process.

Close enough?

> reason, and we have fixed the print output generation now, so (b) isn't 

No, they also wanted easier stylesheet maintenance. 

Sincerely,

Joshua D. Drake

--

---

From: "Joshua D. Drake" <[email-masked]>
Date: Sun, 10 Dec 2006 08:51:24 -0800

> moving to XML which are wrong.

O.k. how about this:

We want to move to XML. We have community members that want to move to
XML. We have regional projects that want to move to XML. 

You can not debunk any of those reasons.

The fact is, they want to.

Until you can come up with a reason why we *shouldn't* move to XML. I
believe this issue is settled.

Sincerely,

Joshua D. Drake

--

---

From: "Joshua D. Drake" <[email-masked]>
Date: Sun, 10 Dec 2006 08:52:23 -0800

> > Where are they in the documentation?

These are good instruction, but not anywhere as simple as:

Download this script.
Run this script.
type make, make pdf, make rtf etc...

Sincerely,

Joshua D. Drake

--

---

From: Guillaume Lelarge <[email-masked]>
Date: Sun, 10 Dec 2006 18:01:36 +0100

> reason, and we have fixed the print output generation now, so (b) isn't 

We didn't switch to use FOP. We switched to build better PDF files
(better --> prettier). Perhaps we could have done the same with SGML but
I had a great way to do it with XML and XSLT (and FOP), a way I can
understand, customize, enhance, etc.

The last PDF available on the website has some bad issues. Just take a
look at table 8-13 (page 805), table 8-18 (page 113). One more time, I
don't say you can't fix this in SGML (and I think we already talk of
this some time ago). But *I* won't do it. And I think some people are
willing to work on this but they also want to have XML documentation.

Perhaps we can switch to XML just because people who want to get
involved with PostgreSQL documentation want to use XML ?

---

From: Tom Lane <[email-masked]>
Date: Sun, 10 Dec 2006 12:11:29 -0500

> believe this issue is settled.

How about "the people who are actually doing the work today don't
want to change to XML"?

I'm not very excited by hopes that all sorts of documentation manpower
will suddenly materialize if only the docs were in XML not SGML.  *That*
is the part of this argument for which no evidence has been produced.
Where are the people who know the code well enough to document it
but refuse to work on the docs because they're not XML?

(And no, Peter is not the only one objecting.)

			regards, tom lane

---

From: Peter Eisentraut <[email-masked]>
Date: Sun, 10 Dec 2006 18:23:14 +0100

Joshua D. Drake wrote:
> O.k. again... what tools? Either provide some specific solutions or
> back out of the conversation.

No, that is up to you, because you started handwaving about "tools".  I 
know my tools, but then I'm not the one complaining that we don't have 
enough or the right ones or whatever.

> xml-po, which is more efficient at helping translation than the

Well, that seems reasonable, but I looked around for something 
called "xmlpo" or "xml-po", but I could find neither a binary nor a 
package by that name on my system.  Googling suggests that it is a 
KDE-specific tool.  Can someone clarify?

> No, they also wanted easier stylesheet maintenance.

$EDITOR stylesheet.xsl

---

From: "Joshua D. Drake" <[email-masked]>
Date: Sun, 10 Dec 2006 09:23:16 -0800

> How about "the people who are actually doing the work today don't

O.k. that's obviously reasonable enough but nobody has stepped up and
said. "Hey, I am doing the work here, I don't want to use XML."

I can easily provide the counter argument that says, "Why don't the
users of SGML want to migrate to XML?" Not one person has stepped up and
said it will cause me undue amounts of grief, pain, or even... I just
hate XML.

> will suddenly materialize if only the docs were in XML not SGML.

I am not a believe of suddenly materializing manpower either ;). However
I do know that the regional projects would be greatly appreciative. 

>   *That*

Where are the people that know the code well enough to document it but
refuse to work on the docs because they are XML?

I am not trying to be sarcastic here, honestly but not even in this
email have you said: "I don't like XML, I don't want to move to it
because..."

> (And no, Peter is not the only one objecting.)

O.k. why then? Why are you objecting? Peter has yet to provide a reason
either?

What we have here is a dozen or so people that want to move to XML and
have stated publicly that they want to do so, as well as the reasons
why.

We also have two people who have said, that they want to stay with SGML
but have provided zero reason why.

Sincerely,

Joshua D. Drake

--

---

From: Peter Eisentraut <[email-masked]>
Date: Sun, 10 Dec 2006 18:25:15 +0100

Joshua D. Drake wrote:

> We want to move to XML. We have community members that want to move
> to XML. We have regional projects that want to move to XML.

"Because I want to" is not a reason in my mind.

> believe this issue is settled.

I have previously listed the issues in favor and against.  Check the 
start of this thread.

---

From: Peter Eisentraut <[email-masked]>
Date: Sun, 10 Dec 2006 18:26:36 +0100

> type make, make pdf, make rtf etc...

Then start writing that script.

---

From: Peter Eisentraut <[email-masked]>
Date: Sun, 10 Dec 2006 18:28:51 +0100

Guillaume Lelarge wrote:
> (better --> prettier). Perhaps we could have done the same with SGML
> but I had a great way to do it with XML and XSLT (and FOP), a way I
> can understand, customize, enhance, etc.

Well, we've got XSLT stylesheets, so use them.

---

From: "Joshua D. Drake" <[email-masked]>
Date: Sun, 10 Dec 2006 09:40:03 -0800

> Then start writing that script.

I plan on it. That is why I brought it up.

Joshua D. Drake

--

---

From: "Joshua D. Drake" <[email-masked]>
Date: Sun, 10 Dec 2006 09:42:00 -0800

> KDE-specific tool.  Can someone clarify?

Sorry, xml2pot

Sincerely,

Joshua D. Drake

--

---

From: "Joshua D. Drake" <[email-masked]>
Date: Sun, 10 Dec 2006 09:51:28 -0800

> I have previously listed the issues in favor and against.  Check the 

The only thing I see is that you don't want to move because all the
tools that are available for docbook sgml are also available for docbook
xml.

Am I missing anything?

Joshua D. Drake

--

---

From: Peter Eisentraut <[email-masked]>
Date: Sun, 10 Dec 2006 19:07:10 +0100

Joshua D. Drake wrote:
> tools that are available for docbook sgml are also available for
> docbook xml.

Link posted by David Blewett: 
http://groups.google.com/group/pgsql.docs/browse_thread/thread/37ff3a011bb705d7
-- 
Peter Eisentraut
http://developer.postgresql.org/~petere/

---

From: "Joshua D. Drake" <[email-masked]>
Date: Sun, 10 Dec 2006 10:20:06 -0800

> http://groups.google.com/group/pgsql.docs/browse_thread/thread/37ff3a011bb705d7

Ahh that would explain why I couldn't find it. :). Thanks.

Sincerely,

Joshua D. Drake

--

---

From: Peter Eisentraut <[email-masked]>
Date: Sun, 10 Dec 2006 21:40:21 +0100

> Sorry, xml2pot

OK, found it.  Works fine.

---

From: Mario <[email-masked]>
Date: Sun, 10 Dec 2006 18:29:58 -0300

> > KDE-specific tool.  Can someone clarify?

  no, is xml2po  and you can find in debian in a package called gnome-doc-utils

---

From: Mario <[email-masked]>
Date: Sun, 10 Dec 2006 19:30:37 -0300

> The reason is that he uses xml2pot, which AFAIK only handles XML and not

  Yes, that's right, I used and I'm using xml2po. Cause, we wanted to
create an easy way to translate manuals and the only tool I know for
that is xml2po. That tool AFAIK works with XML only, that's way I did
that port and also we wanted to translate online, so the current
configuration is

  SGML -> XML -> POT -> WEBAPP -> PO_TRANSLATED -> XML_TRANSLATED

  AND from the XML_TRANSLATED we are able to (thanks to openjade)
generate the HTML files in +10 minutes

---

From: David Blewett <[email-masked]>
Date: Sun, 10 Dec 2006 20:18:07 -0500

> $TASK than the currently used tool."

Sorry to butt in, but I'm still not sure how to do multiple indexes
in one volume using the DSSSL stylesheets. The docs I pointed to
were for the XSL stylesheets. After some discussion in #docbook, it
was said that it was a recent addition to those stylesheets.

David

---

From: Peter Eisentraut <[email-masked]>
Date: Mon, 11 Dec 2006 10:13:40 +0100

Am Montag, 11. Dezember 2006 02:18 schrieb David Blewett:
> in one volume using the DSSSL stylesheets.

I'm not sure either.

> The docs I pointed to were for the XSL stylesheets.

Right.  So using the XSL(T) stylesheet instead of the DSSSL stylesheet would 
be a step in the right direction.  Actually the 8.2 branch currently only has 
an XSLT stylesheet for HTML output.  In 8.3devel I've added one for XSL-FO 
output, which you can invoke by "make postgres-{A4|US}.fo".  (You will 
probably want yet a different paper format for your book.)

---

From: Josh Berkus <[email-masked]>
Date: Mon, 11 Dec 2006 09:54:52 -0800

Folks,

Tom and Peter correctly point out that discussion of production tools and 
authoring tools are separate, and only come together if there are tools for 
XML which solve both issues, an assertion which is not yet proven.

I am one of the champions of XML simply because I know for a fact that there 
are no real authoring tools* for SGML.  However, I do *not* know that the 
tools for XML produce acceptable output.

So, questions to answer:

1) Are there enhanced tools for Docbook XML, WYSWYG or otherwise, which make 
doc authoring easier and produce correct output for PostgreSQL Docs?
1) a) is there some way we can try various tools and check output?

2) If (1), would switching to XML make doc writing with Emacs/VIM harder?

3) If (1), will having authoring tools available increase the number of doc 
contributors?

Regarding (3), I believe that having authoring tools would increase our pool 
based on two things:
a) Anectodal evidence, including several people (like me) who were prevented 
from doing significant doc clean-up by the amount of labor required to 
restructure large SGML documents.
b) That the vast majority of the OSS world is no longer using SGML, meaning 
that even for hand-editing people who work on other projects will be more 
familiar with XML and thus better able to edit it.

While these two may not be enough to answer "should we migrate", they are 
enough to make it worth trying to find the answer to (1), since if all of the 
Docbook XML authoring tools are crappy the whole discussion is moot.

So, can someone help me come up with a way to validate a Docbook XML version 
of the PostgreSQL docs so that I (and others, hopefully) can start testing 
tools?

* = an authoring tool is one which makes generation of the document 
easier/faster than hand-editing text files.  No such tool exists for SGML -- 
even the Emacs toolkit merely does validation.

-- 
Josh Berkus
PostgreSQL @ Sun
San Francisco

---

From: David Blewett <[email-masked]>
Date: Mon, 11 Dec 2006 13:10:18 -0500

> probably want yet a different paper format for your book.)

Excuse my ignorance, but I thought XSLT stylesheets can only be used
on XML files?

David Blewett

---

From: "Joshua D. Drake" <[email-masked]>
Date: Mon, 11 Dec 2006 10:12:56 -0800

> doc authoring easier and produce correct output for PostgreSQL Docs?

Anything that you produce from a WYSWYG editor is going to have to be
massaged to work with PostgreSQL.Org docs.

> 1) a) is there some way we can try various tools and check output?

OpenOffice.Org, WordPerfect, Kword, Abiword. The first two (last I
checked) were the only mature software supporting Docbook output.

> 2) If (1), would switching to XML make doc writing with Emacs/VIM harder?

Absolutely not. 

> 3) If (1), will having authoring tools available increase the number of doc 

It might. However, getting authors will always be difficult. Having
clear direction from the website on how to contribute documentation.
This should include how to contribute if you don't know Docbook. Right
now, you have to dig all the way to Appendix G (who reads apepedixes?). 

Something like this would also be helpful:

http://www.tldp.org/LDP/LDP-Author-Guide/html/index.html

> restructure large SGML documents.

Well to be honest, there isn't really anyway around that that I know of.
OpenOffice.Org will allow you to do what you want, but someone is going
to have to clean up after you.

Now, that is not neccessarily a road block. It is certainly plausible
that someone could write a section (although I think plain text would be
better here) or rewrite a section and submit it for factoring. Factoring
would involve any Docbook that needed to be done.

Usually the Docbook that is required is trivial.

> familiar with XML and thus better able to edit it.

Well that is certainly true.

>From the source tree:

make postgres.xml

Then validate as you see fit. Just FYI, it does validate.

> even the Emacs toolkit merely does validation.

What is it specifically you are looking for then? Because that is what
Docbook is, XML (or sgml) the only thing you need to do is validate. 

http://www.commandprompt.com/community/emacs_sgml/

If you use OpenOffice you will have to set up a style to correct mirror
the PostgreSQL Docbook usage.

Is that perhaps useful? An OpenOffice style that people could download
that is documented and can be used for creating reasonably useful
Docbook for the community?

Sincerely,

Joshua D. Drake

--

---

From: "Joshua D. Drake" <[email-masked]>
Date: Mon, 11 Dec 2006 10:21:21 -0800

On Mon, 2006-12-11 at 13:10 -0500, David Blewett wrote:
> Hash: SHA256

AFAIK they can only be used on XML files, but you can get XML from the
postgresql sgml sources by using make postgres.xml .

Of course your changes that you make from release to release for volumes
won't be permanent and we will continue to waste time because people
have an odd affection for SGML like it was some kind of pet dog.

Joshua D. Drake

> Version: GnuPG v1.4.5 (MingW32)
> iD8DBQFFfZ8KZmlc6wNjtLYRCEsTAJ0YF/5/enV2w7ibvjSYENTdKjZs3QCgophl
> tZVTZsrFceHD3f+KKn6Drqo=
> =zhwv
--

---

From: Tom Lane <[email-masked]>
Date: Mon, 11 Dec 2006 13:34:01 -0500

> checked) were the only mature software supporting Docbook output.

I tried opening postgres.xml in OpenOffice 2.0, and it just showed me
the raw text and markup --- no indication that it understood xml at all.
Is there some special incantation needed?

			regards, tom lane

---

From: "Joshua D. Drake" <[email-masked]>
Date: Mon, 11 Dec 2006 10:40:34 -0800

> Is there some special incantation needed?

I found that I had to remove the <? xml declartion and only have the
docbook dtd declartion.

Warning, it is ugly :) but it does indeed work. 

Sincerely,

Joshua D. Drake

--

---

From: "Joshua D. Drake" <[email-masked]>
Date: Mon, 11 Dec 2006 10:48:11 -0800

> http://groups.google.com/group/pgsql.docs/browse_thread/thread/37ff3a011bb705d7

Per this link Peter you cite:
"""
- Marked sections don't work anymore; would need to use DocBook-specific
profiling mechanism, which isn't all that elegant. 

- More typing: Things like <abc>foo</> and other abbreviations won't 
work anymore; all attributes would need to be quoted, etc. 

- doc/src/sgml/*.sgml will look silly for filenames. 

- Someone would need to do the conversion.  I understand that the
French 
translation team might have patches available. 

So that is it.  In my mind, there is no clear winner, but if someone has
a concrete need for XML, I don't see a problem with it. 
"""

Your first disadvantage I can not comment on.

Your second disadvantage is fixed with any reasonable editor including
emacs.

Your third comment is a filename change.

Your fourth is resolved, we have the conversion per make postgres.xml.
We could even reasonably break it all out again.

I would add to advantages that:

1. We wouldn't have to have this discussion every six months :)

2. It will make life easier for regional projects as they are already
going through the pain of moving to XML for each release.

3. It would help some sub projects including the printed documentation
project that Blewitt and I are working on because we could submit our
patches as XML directly back to the community.

Lastly: There are several who have mentioned a concrete need. Is the
need solved with the existing tools? Yes, but not without extra steps.

So I would ask that you either give us further reason why you don't want
to move to XML or honor what you say above, in that you don't have a
problem with it.

Sincerely,

Joshua D. Drake

--

---

From: Tom Lane <[email-masked]>
Date: Mon, 11 Dec 2006 14:08:03 -0500

> Warning, it is ugly :) but it does indeed work. 

Only for exceedingly small values of "work".  I tried that, and got
a display showing all the text run together in a single paragraph.
I changed one word and re-saved, and got a file that had basically
nothing to do with the original --- it's been arbitrarily reformatted,
and most of the original markup is gone.

Perhaps later versions are better, but the OO version currently available
in Fedora 5 appears utterly useless for our purposes.

Can you show me an authoring tool that does *not* think it's OK to
mangle the low-level text in "semantically irrelevant" ways?  We might
as well forget about the concept of a doc patch if people start using
tools like this --- if a one-word change results in diffs across the
whole file, the tool is not usable.

			regards, tom lane

---

From: "Joshua D. Drake" <[email-masked]>
Date: Mon, 11 Dec 2006 11:23:28 -0800

> mangle the low-level text in "semantically irrelevant" ways?

No. :) I mentioned this previously. Any word processor is going to blow
stuff away in an ugly way. The closest we could get is:

Create a custom style in OpenOffice that works in a similar fashion to
PostgreSQL.

Use Tidy and or a custom script to clean after the fact.

IMHO the only tools for Docbook that are usable are editors, preferrably
validating editors such as Emacs, Bluefish (actually Word Perfect does
this too). We are not going to get what we want out of WYSIWYG.

>   We might

You are correct, we would have to run some kind of clean script and then
diff between the two versions.

Sincerely,

Joshua D. Drake

--

---

From: Josh Berkus <[email-masked]>
Date: Mon, 11 Dec 2006 11:27:37 -0800

Josh,

> massaged to work with PostgreSQL.Org docs.

*sigh* too bad Lyx only writes DocBook and doesn't read it.  

> > easier/faster than hand-editing text files.  No such tool exists for
> > SGML -- even the Emacs toolkit merely does validation.
>

"Authoring Tool" means "not always hand-editing tags".  Right now, I can't 
do anything with Emacs SGML that I couldn't do with Wordpad or Pico, 
except validate.

Let me give you an example of a doc change which needing to hand-edit SGML 
prevented me from making:  I wanted to clean up runtime.sgml by adding an 
alpha index, and clearly breaking out data like defaults, set time, etc.   
This meant changing 70% of the document, including some significant 
re-arranging.

After 5 hours of editing, I discovered that I was only 1/3 done ... and 
that there had been incompatible changes in CVS in the meantime.  I gave 
up.

If the docs were WYSWYG-editable or if we had any authoring tool which 
would automatically fix tags, I would have been able to make the changes 
and submit a patch in < 4 hours, not 15 hours+ it would have taken to 
complete the project.   

-- 
--Josh

Josh Berkus
PostgreSQL @ Sun
San Francisco

---

From: "Joshua D. Drake" <[email-masked]>
Date: Mon, 11 Dec 2006 11:36:02 -0800

> do anything with Emacs SGML that I couldn't do with Wordpad or Pico, 

That is certainly not true. Emacs will correctly associate your tags
without having to write those tags, if you use Emacs correctly. As will
Bluefish.

http://www.commandprompt.com/community/emacs_sgml/x108

Bluefish is a little easier with this IMHO but it gives the same
capabilities. 

> that there had been incompatible changes in CVS in the meantime.  I gave 

Well the CVS issue will be one that always exists.

> would automatically fix tags,

O.k. well you will you lost my support here. I guarantee you I would not
submit patches if I had to work with problems associated with doing
things in OpenOffice.Org format. I would not have a problem helping get
a clean up script together that would work with Docbook output from
OpenOffice.org though.

Docbook is specifically designed for technical documentation. 

>  I would have been able to make the changes 

Wait... You want a tool to automatically fix tags? You do need to know
Docbook to write the docs JoshB. That means you need to know which tags
are relevant to what. Something like emacs will make this easier because
it will tell you what tags are valid for each section of the document
you are in. 

Sincerely,

Joshua D. Drake

--

---

From: Tom Lane <[email-masked]>
Date: Mon, 11 Dec 2006 14:53:53 -0500

> Your fourth is resolved, we have the conversion per make postgres.xml.

We do not have that --- have you looked at the output?  It's nigh
unreadable.  Same problem as with the authoring tools: that software
doesn't think that formatting of the XML source text is important.

			regards, tom lane

---

From: "Joshua D. Drake" <[email-masked]>
Date: Mon, 11 Dec 2006 11:59:12 -0800

On Mon, 2006-12-11 at 14:53 -0500, Tom Lane wrote:
> unreadable.

You are correct which is actually the argument that Gleu made to me
about moving to XML. I did find however that if you ran postgres.xml
through tidy it cleaned things up nicely.

Secondly the make postgres.xml would be a one time thing. I would
happilly take the time to get postgres.xml into a format that was
reasonable and similar to how we currently do the postgres.sgml docs.

>   Same problem as with the authoring tools: that software

Well not authoring editors though. Emacs or Bluefish won't blow that
stuff away (just realized Bluefish doesn't validate :( ).

--

---

From: Josh Berkus <[email-masked]>
Date: Mon, 11 Dec 2006 12:07:23 -0800

Josh,

> it will tell you what tags are valid for each section of the document

Sure, but that doesn't do any good if you want to cut-and-paste a section 
and fix the tags.

Anyway, from the sound of it whether or not there are better authoring 
tools for XML than SGML is "undetermined pending further investigation."

-- 
--Josh

Josh Berkus
PostgreSQL @ Sun
San Francisco

---

From: Tom Lane <[email-masked]>
Date: Mon, 11 Dec 2006 15:07:37 -0500

"Joshua D. Drake" <[email-masked]> writes:
> Secondly the make postgres.xml would be a one time thing.

Agreed, the cost of conversion is one-time ... but it's not small.
Aside from getting the files themselves converted, there's the effort
for people to find, install, and learn suitable tools, not to mention
come up to speed on the differences between xml and sgml.  (I assume
there are some significant ones, else why are we having this discussion?)

The real problem here is that you've still failed to establish any
sizable benefit from converting.  As best I can tell at the moment,
the acceptable options for editing XML will be about the same as they
are for SGML: emacs, and not a lot else.  I don't really see why I
should have to start spelling out every closing tag for a no-op like
that.

			regards, tom lane

---

From: "Joshua D. Drake" <[email-masked]>
Date: Mon, 11 Dec 2006 12:43:27 -0800

> there are some significant ones, else why are we having this discussion?)

Well this actually comes back to one of Peter's points :). You shouldn't
need to update your tools. If you installed Docbook tools, it is going
to come with DSSSL and XSL. OpenJade will process the XML as will any of
the current db2, docbook2 etc.. scripts.

As far as learning the differences between xml and sgml, they are very
subtle because it is still Docbook.

http://www.oreilly.com/catalog/docbook/chapter/book/appb.html
http://www.docbook.org/tdg/en/html/ch02.html

For most things that anyone would be doing, they will see extremely
little difference. I didn't even notice the difference because I never
used short tags.

> are for SGML: emacs, and not a lot else.

Well basically, editing :) any editor although some make it easier as I
mentioned Emacs, BlueFish, Kate etc...

The benefits are (in my mind):

1. At least some of our regionals are already using XML. Let's make it a
little easier for them to do so.

2. It would make the new printed documentation project easier for David
Blewitt (the guy that is doing it)

3. DocBook XML is where FOSS documentation is moving or has moved
depending on the project. Should any project want to integrate any part
of our docs to their docs or vice versa, it will be easier for them if
we follow the similar standard of using Docbook XML. 

4. Their is active and progressive development with XSL and Docbook.
There is not (that I can see) the same level with DSSSL and Docbbok.

5. The format objects development (FOP) seems to be better maintained
then the dblatex/db2latex/pdflatex stuff. FOP does not appear to work
with SGML.

6. Nobody has really come up with a reason *not* to move to XML that
can't easily be addressed.

7. And this may be ethereal but I do believe that if we were following
in the footsteps of other FOSS projects we may be able to get writers
from those projects. Even only 1 of them.

>   I don't really see why I

Fair enough but if you are using a decent editor, it will take care of
that for you.

To be honest. I do not care. I have zero preference to XML or SGML
because I know them both. I do however care, when I have regionals and
contributors trying to give to the community that are having difficulty
doing so *because* we use SGML versus XML.

It is enough for me, that I have PostgreSQLFR, Chile and the printed
book project all clammoring for XML support to give more weight to their
arguments. Each of the three have presented their specific problems with
the SGML. Not to mention people such as Andrew Dunstan and the others. 

Maybe the people demand isn't the concern. Okay, but switching to XML is
not hurting anyone. It's not like we are trying to change languages...
only the DTD :)

Sincerely,

Joshua D. Drake

--

---

From: "Magnus Hagander" <[email-masked]>
Date: Mon, 11 Dec 2006 21:56:28 +0100

> The real problem here is that you've still failed to 
> establish any sizable benefit from converting.  As best I can 
> tell at the moment, the acceptable options for editing XML 
> will be about the same as they are for SGML: emacs, and not a 
> lot else.  I don't really see why I should have to start 
> spelling out every closing tag for a no-op like that.

Visual Studio has a *very* nice XML editor these days, and they don't do
SGML...

(ok, ok, that's a joke. It's true, sure, but it's not very relevant...)

//Magnus

---

From: "Joshua D. Drake" <[email-masked]>
Date: Mon, 11 Dec 2006 13:07:03 -0800

> (ok, ok, that's a joke. It's true, sure, but it's not very relevant...)

Actually it might be for Windows developers wishing to develop Windows
specific sections. How does it work with Docbook?

Sincerely,

Joshua D. Drake

> TIP 6: explain analyze is your friend
> 
--

---

From: David Blewett <[email-masked]>
Date: Mon, 11 Dec 2006 16:08:09 -0500

> should have to start spelling out every closing tag for a no-op like

In regards to cost of conversion: The osx tool that the makefile
uses to generate postgres.xml has more options available to tweak
the output (not resolve external files into the main file, no
newlines in tags, etc.) After adding a few of those options +
htmltidy, the result is pretty close to the original SGML, barring a
bare newline between <para>'s for example.

The benefit I'm talking about is the ability to do multiple indexes
in one document, which is not possible with DSSSL. In order to do
this, I would have to run the conversion *every time* I wanted to
work on a new release of the manual to bookstores.

David Blewett

---

From: Scott Marlowe <[email-masked]>
Date: Mon, 11 Dec 2006 15:20:57 -0600

> doesn't think that formatting of the XML source text is important.

Would this help:

http://xmlindent.sourceforge.net/

I use it all the time at work, where we have LOTS of tools that make big
one line xml files for us.

---

From: Tom Lane <[email-masked]>
Date: Mon, 11 Dec 2006 16:23:15 -0500

> work on a new release of the manual to bookstores.

Yeah, but for purely downstream work like that, what's the objection to
just running the osx conversion?  It seems to take just a few seconds.

I'm still not seeing where we get return on our investment for
converting the master source files from one format to the other.

			regards, tom lane

---

From: "Magnus Hagander" <[email-masked]>
Date: Mon, 11 Dec 2006 23:00:35 +0100

> establish any 
> > > the acceptable options for editing XML will be about the same as 
> > > they are for SGML: emacs, and not a lot else.  I don't really see 
> > > why I should have to start spelling out every closing tag for a 
> > > no-op like that.
> > 
> > Visual Studio has a *very* nice XML editor these days, and 
> they don't 
> > do SGML...
> > 
> Actually it might be for Windows developers wishing to 
> develop Windows specific sections. How does it work with Docbook?

It works just fine with any XML inlcluding docbook. It does your basic
syntax highlighting, autoindenting and things like that. You can teach
it about things like docbook to get autocomplete on tags and attributes,
but I don't know if anybody has done that (I certainly haven't).

//Magnus

---

From: Tom Lane <[email-masked]>
Date: Mon, 11 Dec 2006 17:02:12 -0500

"Magnus Hagander" <[email-masked]> writes:
>>> they don't do SGML...

> but I don't know if anybody has done that (I certainly haven't).

How is it on the point of not introducing random reformatting into the
XML source text?

			regards, tom lane

---

From: "Magnus Hagander" <[email-masked]>
Date: Mon, 11 Dec 2006 23:04:57 +0100

> >>> Visual Studio has a *very* nice XML editor these days, and they 
> >>> don't do SGML...
> >>> 
> >> Actually it might be for Windows developers wishing to develop 
> >> Windows specific sections. How does it work with Docbook?
> 
> > it about things like docbook to get autocomplete on tags and 
> > attributes, but I don't know if anybody has done that (I 
> certainly haven't).
> 
> How is it on the point of not introducing random reformatting 
> into the XML source text?

it doesn't do it, period. Just to be clear, it's a XML *source* editor.
Just like emacs, excpet prettier ;-)

(and just for the record, these days even the GUI editors in Visual
Studio generally do a very good job of not breaking your manually tuned
markup, but there is no GUI editor for docbook)

//Magnus

---

From: Guillaume Lelarge <[email-masked]>
Date: Tue, 12 Dec 2006 01:31:02 +0100

> Excuse my ignorance, but I thought XSLT stylesheets can only be used

That's right. To use Peter's work, you need to checkout CVS HEAD, cd
into doc/src/sgml, "make postgres.xml" which build a valid XML file.

Next step depends on the output format you want to get. To get HTML
output from XSLT file, execute "make xslthtml". This target use
stylesheet.xsl which customizes a bit the XSLT stylesheets for XHTML
available from docbook.sourceforge.net. To get .fo output (required to
build a PDF file), "make postgres-A4.fo". This command will use the
stylesheet-fo.xsl file. This file is just a pointer to XSLT stylesheets
for PDF available from docbook.sourceforge.net. Simple but effective.

A summary :
 * to build HTML manual with XSLT :
   make postgres.xml
   make xslthtml
 * to build PDF manual with XSLT :
   make postgres.xml
   make postgres-A4.fo
   fop.sh postgres-A4.fo postgres-A4.pdf

I think my fop install is broken because I'm not able to build the PDF
file...

Peter, if I said something stupid, please, tell it. This is just what I
understood from your mail and the Makefiles/xslt files you provide.

Regards.

---

From: David Blewett <[email-masked]>
Date: Mon, 11 Dec 2006 20:39:11 -0500

> converting the master source files from one format to the other.

Does the designation of "downstream work" apply here? I was
intending to contribute back any changes required, to enable anyone
else to build the multiple volume version for printing. I wanted it
to become an easily maintainable way for the project to have an
up-to-date version of our manual available to bookstores, or
directly available through lulu.com.

I also would like to reiterate the fact that the localization
efforts would appreciate not having to re-do their work for each
release. Do we really want to continue considering this addition to
the project as well as their work to be "second-class citizens" wrt
the rest of the distribution?

David Blewett

---

From: Tom Lane <[email-masked]>
Date: Mon, 11 Dec 2006 20:47:14 -0500

David Blewett <[email-masked]> writes:
> else to build the multiple volume version for printing.

Changes required where?  AFAICS the useful content of your work would
probably be in the stylesheets, which there is already an XML version
of.

> release.

How exactly does working from an osx translation rather than hand-edited
sources handicap them?  Or even more to the point, which part of their
work don't they have to redo anyway?  We certainly have never hesitated
to whack the documentation around in small ways and large...

			regards, tom lane

---

From: Guillaume Lelarge <[email-masked]>
Date: Tue, 12 Dec 2006 08:58:07 +0100

Tom Lane a ecrit le 12/12/2006 02:47:
> [...]

I don't know for others but I think I can speak for the french team :)

I also don't quite understand the whole point on the translation. I use 
Kompare to... hmmm... compare one release to another. I merge *manually* 
the differences, and then I translate them. I also need to transform 
SGML tags to XML tags (</> stuff and no ending tags). Actually, the 
merge/translate step is much more difficult.

If some projects prefer to use po (with xml2pot or po4all), "make 
postgres.xml" is the required step. The XML file built this way seems 
valid to me.

Regards.

-- 
Guillaume.

---

From: Peter Eisentraut <[email-masked]>
Date: Tue, 12 Dec 2006 08:59:44 +0100

Joshua D. Drake wrote:
> then the dblatex/db2latex/pdflatex stuff.

Check the facts.  The latest stable FOP release was more than three 
years ago.  The latest stable dblatex release was less than a month 
ago.

---

From: Peter Eisentraut <[email-masked]>
Date: Tue, 12 Dec 2006 09:07:14 +0100

Joshua D. Drake wrote:
> Of course your changes that you make from release to release for
> volumes won't be permanent

Well, of course he'd make his changes in the sources, not in the 
intermediate XML file.  Or he is talking about a kind of changes which 
I cannot fathom at the moment.

---

From: Peter Eisentraut <[email-masked]>
Date: Tue, 12 Dec 2006 09:08:07 +0100

> Excuse my ignorance, but I thought XSLT stylesheets can only be used

FWIW, you can use XSLT stylesheets on SGML as well, but we don't do 
that.

---

From: Peter Eisentraut <[email-masked]>
Date: Tue, 12 Dec 2006 09:10:18 +0100

>  * to build HTML manual with XSLT :

Just the last command should be enough.

>  * to build PDF manual with XSLT :

Same here.

> I think my fop install is broken because I'm not able to build the
> PDF file...

I have been using FOP on and off for many years and I've never seen it 
produce any real document longer than two pages, so I'm not surprised.

---

From: Guillaume Lelarge <[email-masked]>
Date: Tue, 12 Dec 2006 09:16:35 +0100

> produce any real document longer than two pages, so I'm not surprised.

It produces PDF file for the PostgreSQL french manual. I really have no 
problem to build the 1098 pages PDF file for 8.1.5 release.

-- 
Guillaume.

---

From: Peter Eisentraut <[email-masked]>
Date: Tue, 12 Dec 2006 16:26:41 +0100

Josh Berkus wrote:
> Anyway, from the sound of it whether or not there are better
> authoring tools for XML than SGML is "undetermined pending further
> investigation."

I'll be the first to welcome better editing options, but so far I have 
only heard them being theorized about.

---

From: "Joshua D. Drake" <[email-masked]>
Date: Tue, 12 Dec 2006 09:42:30 -0800

On Tue, 2006-12-12 at 08:59 +0100, Peter Eisentraut wrote:
> years ago. 

Actually it was about 8 months ago. 0.92 Beta. 

>  The latest stable dblatex release was less than a month 
> ago.

I have found dblatex to not work very well (as in at all) but that could
very well be my installation.

Joshua D. Drake

--

---

From: "Joshua D. Drake" <[email-masked]>
Date: Tue, 12 Dec 2006 09:43:51 -0800

> produce any real document longer than two pages, so I'm not surprised.

I was able to easily (once I got the environment sorted out) generate
the entire french documentation with FOP.

Joshua D. Drake

--

---

From: Peter Eisentraut <[email-masked]>
Date: Tue, 12 Dec 2006 20:27:58 +0100

> Actually it was about 8 months ago. 0.92 Beta.

That is not a stable release.  And the fact that the latest beta is 8 
months old doesn't impress me either.
-- 
Peter Eisentraut
http://developer.postgresql.org/~petere/

---

From: "Joshua D. Drake" <[email-masked]>
Date: Tue, 12 Dec 2006 11:49:08 -0800

> months old doesn't impress me either.

I am not going to argue with you further. I have better things to do.

Sincerely,

Joshua D. Drake

--

---

From: Bruce Momjian <[email-masked]>
Date: Tue, 12 Dec 2006 17:44:48 -0500 (EST)

> because it would piss off the people who do the bulk of the work now.

And diffs would be either very large or useless.

-- 
  Bruce Momjian   [email-masked]
  EnterpriseDB    http://www.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

---

From: Bruce Momjian <[email-masked]>
Date: Wed, 13 Dec 2006 12:14:18 -0500 (EST)

I have read the two long XML threads, and see these issues for
converting to XML:

o authoring, in two parts:

	o WYSIWYG - this seems unattainable because such editors are
	  going to modify the surrounding XML, which will affect
	  hand-edited changes

	o tag assistance - this seems possible, e.g. tag completion,
	  drop down list of tags available for this section

o output generation - the XML toolchain is more widely used, but Tom has
  fixed the performance issue we have had for years.  The fact that the
  tool maintainers were unable to fix it themselves reflects badly on
  their maintenance of this toolchain.

o translation - does XML have better tools to assist in translating our
  manuals, and is this better than outputting XML via our makefile?

It would be nice to use htmltidy to clean up the xml before each
release, but I wonder whether it understand the tags well enough to
provide better formatting than we do now manually.

Tom, I think we need that SGML patch placed on our ftp site and a
reference to it made in our documentation.  I can do it of you want.

---

From: Josh Berkus <[email-masked]>
Date: Thu, 14 Dec 2006 12:58:43 -0800

Bruce,

> 	  going to modify the surrounding XML, which will affect

It would be more accurate to say that we have not identified a WYSWYG tool 
which does not mess up the source.  There may be one, it would just take a 
fair amount of testing to find it.

> 	  drop down list of tags available for this section

Yes, and graphically-enhanced tag support using XML IDEs.  Command-line tag 
completion is available via Emacs-SGML, but there are a lot more tools 
which do it for XML (including Emacs).

-- 
--Josh

Josh Berkus
PostgreSQL @ Sun
San Francisco

---

From: Scott Marlowe <[email-masked]>
Date: Thu, 14 Dec 2006 15:23:17 -0600

> fair amount of testing to find it.

Is this strictly a question of indentation, or one of actually mangling
tags and such?

---

From: Tom Lane <[email-masked]>
Date: Thu, 14 Dec 2006 16:34:06 -0500

> Is this strictly a question of indentation, or one of actually mangling

What we need is something that does not change regions of the file that
the user did not intend to modify.  I think horizonal-white-space-only
changes could be worked around if the user is careful to use diff
--ignore-space-change when submitting the patch, but I suspect that an
editor that thinks it can mangle whitespace will also figure that it can
change line boundaries, and then diff will never be able to extract any
signal from that noise.

			regards, tom lane

---

From: Scott Marlowe <[email-masked]>
Date: Thu, 14 Dec 2006 15:37:01 -0600

> change line boundaries, and then diff will never be able to extract any

Ahhhh.  I see.

Well, I don't mind teaching myself some docbook, but the first time I
looked at adding to the docs I was overwhelmed with what I didn't know,
and didn't even see a starting point.  I'll look in the appendices and
see what I can see.

---

From: Bruce Momjian <[email-masked]>
Date: Thu, 14 Dec 2006 21:45:52 -0500 (EST)

I have applied the attached documentation patch to HEAD and 8.2.X to
document the use of Tom's patch to openjade to get ouput in a reasonable
amount of time.

---------------------------------------------------------------------------

  + If your life is a hard drive, Christ can be your backup. +
Index: doc/src/sgml/docguide.sgml
===================================================================
RCS file: /cvsroot/pgsql/doc/src/sgml/docguide.sgml,v
retrieving revision 1.61
diff -c -c -r1.61 docguide.sgml
*** doc/src/sgml/docguide.sgml	10 Dec 2006 20:46:03 -0000	1.61
--- doc/src/sgml/docguide.sgml	15 Dec 2006 02:39:37 -0000
***************
*** 115,120 ****
--- 115,123 ----
         <acronym>DSSSL</acronym> stylesheets), as well as a number of
         related tools.  <productname>Jade</productname> is now being
         maintained by the OpenJade group, no longer by James Clark.
+        (If generating Postscript or PDF output, you will need to
+        compile from source and use a special patch to get output
+        in a reasonable amount of time.)
        </para>
       </listitem>
      </varlistentry>
***************
*** 295,300 ****
--- 298,311 ----
          installed and you want to install the rest of the tool chain
          locally.)
         </para>
+ 
+        <para>
+         OpenJade release 1.3.2 and perhaps earlier and later releases
+         have a known bug that causes Postscript and PDF output
+         generation to take days.  This <ulink
+         url="http://archives.postgresql.org/pgsql-docs/2006-12/msg00064.php">patch</ulink>
+         fixes the problem and generates output in a few minutes.
+        </para>
        </step>
  
        <step id="doc-openjade-install">

---

From: David Fetter <[email-masked]>
Date: Thu, 14 Dec 2006 22:15:16 -0800

On Thu, Dec 14, 2006 at 04:34:06PM -0500, Tom Lane wrote:
> >> It would be more accurate to say that we have not identified a
> >> WYSWYG tool which does not mess up the source.  There may be one,
> >> it would just take a fair amount of testing to find it.
> 
> > Is this strictly a question of indentation, or one of actually
> > mangling tags and such?
> 
> What we need is something that does not change regions of the file
> that the user did not intend to modify.  I think
> horizonal-white-space-only changes could be worked around if the
> user is careful to use diff --ignore-space-change when submitting
> the patch, but I suspect that an editor that thinks it can mangle
> whitespace will also figure that it can change line boundaries, and
> then diff will never be able to extract any signal from that noise.

Could post-processing with tidy clean this up?  I'm pretty sure that
tidy's output is deterministic, at least if the editor hasn't mangled
tags...

Cheers,
D

Remember to vote!

---

From: "Joshua D. Drake" <[email-masked]>
Date: Thu, 14 Dec 2006 22:39:24 -0800

> tidy's output is deterministic, at least if the editor hasn't mangled

Yes tidy makes things pretty darn clean. However, there is one thing
that it won't do and that is remove extra tags. Thus something like
OpenOffice is still a no-op.

Joshua D. Drake

--

---

From: Peter Eisentraut <[email-masked]>
Date: Fri, 15 Dec 2006 10:07:52 +0100

> document the use of Tom's patch to openjade to get ouput in a reasonable

This is no longer accurate in 8.3.  A workaround has been installed in the 
stylesheets to arrive at the same result.

---

From: Bruce Momjian <[email-masked]>
Date: Fri, 15 Dec 2006 11:50:09 -0500 (EST)

> stylesheets to arrive at the same result.

OK, I have talked to Peter via IM and now understand that his change to
stylesheet.dsl avoids the bug Tom found and fixed, so instead of
mentioning the patch in the SGML docs, I added a comment in the
stylesheet.dsl file and pointed to the URL.

Also, the double-run of the PDF does not fix the index.  Only a proper
bookindex.sgml does, that, so I removed the double runs and either Peter
or I will work on issuing a warning when the bookindex.sgml file isn't
proper, so people know to run it again.

Revert patch attached and applied to HEAD and 8.2.X.

  + If your life is a hard drive, Christ can be your backup. +
Index: doc/src/sgml/Makefile
===================================================================
RCS file: /cvsroot/pgsql/doc/src/sgml/Makefile,v
retrieving revision 1.85
diff -c -c -r1.85 Makefile
*** doc/src/sgml/Makefile	15 Dec 2006 01:20:59 -0000	1.85
--- doc/src/sgml/Makefile	15 Dec 2006 16:29:03 -0000
***************
*** 99,104 ****
--- 99,107 ----
  
  COLLATEINDEX := LC_ALL=C $(PERL) $(COLLATEINDEX) -f -g
  
+ # If HTML.index does not exist, create a dummy bookindex.sgml.  During the
+ # next build, create bookindex.sgml with the proper index contents.  A proper
+ # bookindex.sgml is required to have an index in the output.
  ifeq (,$(wildcard HTML.index))
  bookindex.sgml:
  	$(COLLATEINDEX) -o $@ -N
***************
*** 136,164 ****
  
  JADE.tex.call = $(JADE) $(JADEFLAGS) $(SGMLINCLUDE) $(CATALOG) -d $(srcdir)/stylesheet.dsl -t tex -V tex-backend -i output-print
  
! %-A4.tex-ps: %.sgml $(ALLSGML) stylesheet.dsl
! 	# multiple runs are necessary to create proper index entries
! 	$(JADE.tex.call) -V texdvi-output -V '%paper-type%'=A4 -o $@ $<
  	$(JADE.tex.call) -V texdvi-output -V '%paper-type%'=A4 -o $@ $<
  
! %-US.tex-ps: %.sgml $(ALLSGML) stylesheet.dsl
! 	# multiple runs are necessary to create proper index entries
! 	$(JADE.tex.call) -V texdvi-output -V '%paper-type%'=USletter -o $@ $<
  	$(JADE.tex.call) -V texdvi-output -V '%paper-type%'=USletter -o $@ $<
  
! %-A4.tex-pdf: %.sgml $(ALLSGML) stylesheet.dsl
! 	# multiple runs are necessary to create proper index entries
! 	$(JADE.tex.call) -V texpdf-output -V '%paper-type%'=A4 -o $@ $<
  	$(JADE.tex.call) -V texpdf-output -V '%paper-type%'=A4 -o $@ $<
  
! %-US.tex-pdf: %.sgml $(ALLSGML) stylesheet.dsl
! 	# multiple runs are necessary to create proper index entries
! 	$(JADE.tex.call) -V texpdf-output -V '%paper-type%'=USletter -o $@ $<
  	$(JADE.tex.call) -V texpdf-output -V '%paper-type%'=USletter -o $@ $<
  
  %.dvi: %.tex-ps
  	@rm -f $*.aux $*.log
! 	# multiple runs are necessary to create proper index entries
  	jadetex $<
  	jadetex $<
  	jadetex $<
--- 139,159 ----
  
  JADE.tex.call = $(JADE) $(JADEFLAGS) $(SGMLINCLUDE) $(CATALOG) -d $(srcdir)/stylesheet.dsl -t tex -V tex-backend -i output-print
  
! %-A4.tex-ps: %.sgml $(ALLSGML) stylesheet.dsl bookindex.sgml
  	$(JADE.tex.call) -V texdvi-output -V '%paper-type%'=A4 -o $@ $<
  
! %-US.tex-ps: %.sgml $(ALLSGML) stylesheet.dsl bookindex.sgml
  	$(JADE.tex.call) -V texdvi-output -V '%paper-type%'=USletter -o $@ $<
  
! %-A4.tex-pdf: %.sgml $(ALLSGML) stylesheet.dsl bookindex.sgml
  	$(JADE.tex.call) -V texpdf-output -V '%paper-type%'=A4 -o $@ $<
  
! %-US.tex-pdf: %.sgml $(ALLSGML) stylesheet.dsl bookindex.sgml
  	$(JADE.tex.call) -V texpdf-output -V '%paper-type%'=USletter -o $@ $<
  
  %.dvi: %.tex-ps
  	@rm -f $*.aux $*.log
! 	# multiple runs are necessary to create proper intra-document links
  	jadetex $<
  	jadetex $<
  	jadetex $<
***************
*** 169,175 ****
  
  %.pdf: %.tex-pdf
  	@rm -f $*.aux $*.log $*.out
! 	# multiple runs are necessary to create proper index entries
  	pdfjadetex $<
  	pdfjadetex $<
  	pdfjadetex $<
--- 164,170 ----
  
  %.pdf: %.tex-pdf
  	@rm -f $*.aux $*.log $*.out
! 	# multiple runs are necessary to create proper intra-document links
  	pdfjadetex $<
  	pdfjadetex $<
  	pdfjadetex $<
Index: doc/src/sgml/docguide.sgml
===================================================================
RCS file: /cvsroot/pgsql/doc/src/sgml/docguide.sgml,v
retrieving revision 1.62
diff -c -c -r1.62 docguide.sgml
*** doc/src/sgml/docguide.sgml	15 Dec 2006 02:44:28 -0000	1.62
--- doc/src/sgml/docguide.sgml	15 Dec 2006 16:29:03 -0000
***************
*** 115,123 ****
         <acronym>DSSSL</acronym> stylesheets), as well as a number of
         related tools.  <productname>Jade</productname> is now being
         maintained by the OpenJade group, no longer by James Clark.
-        (If generating Postscript or PDF output, you will need to
-        compile from source and use a special patch to get output
-        in a reasonable amount of time.)
        </para>
       </listitem>
      </varlistentry>
--- 115,120 ----
***************
*** 298,311 ****
          installed and you want to install the rest of the tool chain
          locally.)
         </para>
- 
-        <para>
-         OpenJade release 1.3.2 and perhaps earlier and later releases
-         have a known bug that causes Postscript and PDF output
-         generation to take days.  This <ulink
-         url="http://archives.postgresql.org/pgsql-docs/2006-12/msg00064.php">patch</ulink>
-         fixes the problem and generates output in a few minutes.
-        </para>
        </step>
  
        <step id="doc-openjade-install">
--- 295,300 ----
Index: doc/src/sgml/stylesheet.dsl
===================================================================
RCS file: /cvsroot/pgsql/doc/src/sgml/stylesheet.dsl,v
retrieving revision 1.31
diff -c -c -r1.31 stylesheet.dsl
*** doc/src/sgml/stylesheet.dsl	10 Dec 2006 01:55:29 -0000	1.31
--- doc/src/sgml/stylesheet.dsl	15 Dec 2006 16:29:03 -0000
***************
*** 280,285 ****
--- 280,293 ----
  (define %refentry-new-page%     #t)
  (define %refentry-keep%         #f)
  
+ ;; Indentation of verbatim environments.  (This should really be done
+ ;; with start-indent in DSSSL.)
+ ;; Use of indentation in this area exposes a bug in openjade,
+ ;; http://archives.postgresql.org/pgsql-docs/2006-12/msg00064.php
+ ;; (define %indent-programlisting-lines% "    ")
+ ;; (define %indent-screen-lines% "    ")
+ ;; (define %indent-synopsis-lines% "    ")
+ 
  
  ;; Default graphic format: Jadetex wants eps, pdfjadetex wants pdf.
  ;; (Note that pdfjadetex will not accept eps, that's why we need to

---

From: Peter Eisentraut <[email-masked]>
Date: Fri, 15 Dec 2006 22:39:36 +0100

Bruce Momjian wrote:
> Also, the double-run of the PDF does not fix the index.  Only a
> proper bookindex.sgml does, that, so I removed the double runs and
> either Peter or I will work on issuing a warning when the
> bookindex.sgml file isn't proper, so people know to run it again.

Producing a warning isn't hard, but the question is where to put it so 
it doesn't get buried in all the other output.

---

From: Bruce Momjian <[email-masked]>
Date: Sat, 16 Dec 2006 22:17:48 -0500 (EST)

> it doesn't get buried in all the other output.

I think the Makefile rule just emits it at the end of its processing.

Also, for testing, I was thinking you could use 'wc -l bookindex.sgml',
and if the file length is less thatn 25 lines, it is a fake file.

---

From: David Blewett <[email-masked]>
Date: Wed, 20 Dec 2006 20:33:24 -0500

> into the copies people actually use, so your fix sounds good for the

I submitted the patch to gentoo's bugzilla:
http://bugs.gentoo.org/show_bug.cgi?id=158605

David Blewett

---

From: David Blewett <[email-masked]>
Date: Fri, 22 Dec 2006 05:38:44 -0500

> http://bugs.gentoo.org/show_bug.cgi?id=158605

Hi Tom:

There's been a question posted to that bug. Would you mind replying
to it? You don't have to create an account there. I can post your reply.

David

Quoted,  [email-masked]:
"Very interesting. I'd of course love to add a patch like this,
however, it seems not so easy to follow. Not that the patch itself
is overly complicated, but openjade sources are, let's say, less
than ideal.

If you or the original author of the patch could comment the
changes, specially those in style/primitive.cxx regarding to the
protect and protect2 objects, it would help a lot.

Also, for future reference, please create your patches as unified
diffs when submitting them to resources such as this bugzilla (many
other projects encourage this as well).

Thanks."

---

From: Tom Lane <[email-masked]>
Date: Fri, 22 Dec 2006 10:18:02 -0500

> to it? You don't have to create an account there. I can post your reply.

I haven't gotten any reply from my posting to the upstream mailing list
at sourceforge :-( so it's hard to tell if anything will happen there.
I don't really approve of fixing this on a distro-by-distro basis, but
maybe we have no other choice.

Anyway, what I said to them was

------- Forwarded Message

Date:    Sat, 09 Dec 2006 23:35:54 -0500
From:    Tom Lane <[email-masked]>
To:      [email-masked]
Subject: Performance problems with NodeListObj lists

It's been folklore for some time at the PostgreSQL project
(www.postgresql.org) that running our documentation through jade to
produce TeX output requires about three days on current hardware :-(
Today I got motivated to look into why, and what I find is a pretty
localized problem, as illustrated by this oprofile trace:

samples  %        symbol name

While I've not traced through the stylesheets to determine exactly what
causes this, the behavior seen at the C++ level is that long NodeListObj
lists (on the order of 10K elements) are built by successive "append"
operations using the NodeList primitive.  This produces a
PairNodeListObj tree that's nested in the wrong direction; if you'll
pardon some crude ASCII art:

                PairNodeListObj
                  /           \
            PairNodeListObj    e
              /           \
        PairNodeListObj    d
          /           \
    PairNodeListObj    c
       /      \
      a        b

This means that for an N-element list, nodeListFirst requires O(N) time
to recurse down to the first element, and so does nodeListRest.  So
iterating through the list in the usual way requires O(N^2) time.
To add to the problem, nodeListRest generates another PairNodeListObj
on each call, imposing a ton of mostly-useless load on the memory
allocation/garbage collection machinery.

Attached is a first-cut attempt at fixing this.  With this patch,
jade gets through the Postgres doc sources in 5 minutes rather than
the reported three days.  (I've never actually built the sources on
the machine I'm testing on, so these numbers are not strictly
comparable.)

As patched, PairNodeListObj is used only in ReverseNodeListObj;
it's tempting to get rid of it entirely by merging it somehow
with the new CellNodeListObj class.  But since I've never looked
at the jade sources before today, I figured I'd better send it
in without any more ado, so someone who knows better can tell me
why this is not gonna work ...

			regards, tom lane

[ plus patch, stripped ]

------- End of Forwarded Message

The point of the patch is to introduce a version of NodeListObj that
tracks both the head and the tail of the list it represents, so that
successive appends can produce a tree that's right-deep instead of
left-deep.  The individual list cells (CellNodeListObj) just have
Lisp-style car and cdr pointers, while HeadNodeListObj is the top of
the list and remembers the current first and last cells.

As for the changes in primitive.cxx, assuming that I have wrapped my
head around the usage of ELObjDynamicRoot correctly, the original
coding is actually wrong because it invokes PairNodeListObj() without
protecting the "tem" pointer.  If that were to point to a
freshly-created, not-otherwise-referenced object, then if any garbage
collection occurred during PairNodeListObj()'s memory allocation request
you'd have a problem.  I suppose this bug has not been seen because
no version of asNodeList() that is actually used here returns a freshly
created object, but it sure seems like an accident waiting to happen.
So I modified the code to be sure that both nl and tem are
ELObjDynamicRoot-protected during the nodeListAppend call.

			regards, tom lane

---

From: Alvaro Herrera <[email-masked]>
Date: Fri, 22 Dec 2006 13:54:58 -0300

> willing to work on this but they also want to have XML documentation.

I checked the PDF available on
http://www.postgresql.org/files/documentation/pdf/8.2/postgresql-8.2-A4.pdf
and it does have the issues Guillaume mentions: table 8-13 (on page 105)
is certainly mangled (columns too wide), and table 8-18 is mangled too
(a row's content is too wide for the columns, so the text on one column
overlaps the text on the next).  This problem can also be seen on other
tables, for example 9-5 and 9-6.

I printed one page and it looks just as silly, so it's not just a
display problem.

Also, the US version is lacking the index -- I suppose it was built
without the complete index.sgml file.

---

From: Guillaume Lelarge <[email-masked]>
Date: Fri, 22 Dec 2006 18:09:20 +0100

> tables, for example 9-5 and 9-6.

There's a way to fix this : adding colspec's tags for each column which 
need a specific col width.

For example :
   <colspec colnum="1" colwidth="1.3*"/>

Regards.

-- 
Guillaume.