Subject: Proposal: syntax highlight in html manual

From: Daniele Varrazzo <[email-masked]>
Date: Wed, 13 Apr 2011 11:31:45 +0100

Hello,

when I wrote the docs for the GMP extension
(http://pgmp.projects.postgresql.org/) I started improving the syntax
highlight produced by pygments
(https://bitbucket.org/dvarrazzo/pygments-postgres) for the PostgreSQL
SQL dialect. I've also added specific lexers for PL/pgSQL and
interactive psql sessions. The lexer handles all the PG constructs,
has a list of keywords and datatypes parsed from the docs, and can
also dispatch the content of a $$ string to a different lexer based on
a related LANGUAGE keyword nearby (e.g. highlighting a PL/Python
function using the Python lexer). Here you can see the result on the
"regression document" I am using to develop the lexer:

- http://pgmp.projects.postgresql.org/highlight/psql.html
- http://pgmp.projects.postgresql.org/highlight/postgres.html

Is there any interest in applying highlighted syntax to the html
rendering of the manual?

If there is, I think the rendering should be performed as a
post-processing step on the html output and should be a totally
optional phase: we may do it for the website but have the docs
generation not to fail if the tools (python, pygments) are missing.
There would be the need to tag every snippet in the docs with the
correct language: I think the correct way is to use the "role"
attribute in the docbook tags generating the snippets (screen,
programlisting, synopsis...): its value can be propagated to the html
(e.g. as a css class) using a suitable docbook configuration (see
<http://www.sagehill.net/docbookxsl/HtmlCustomEx.html#CustomClassValues>,
albeit a test I've done in that direction failed - but I'm completely
clueless about debugging the docbooc tool chain).

I've scraped all the docs snippets (about 3k) into a database and
written an interactive tool to tag them with a language: I'm using the
tool to find snippets to test the lexer with and immediately check the
result. Examples here (these are static pages, not the live tool):

- http://pgmp.projects.postgresql.org/highlight/snippets/plpgsql-control-structures.html
- http://pgmp.projects.postgresql.org/highlight/snippets/plpython-data.html

The result of the tagging may be used to patch the docs, injecting the
role in the sgml source.

If you like the idea I can work at the missing parts, e.g. fixing the
ambiguities in the examples (missing psql prompts etc), tag all the
snippets, write the script to postprocess the html (which should also
put back the result of the docbook semantic tagging to have e.g.
"replaceable" rendered in italic), maybe have a specific lexer for the
synopsis etc. Otherwise I'll just hack on the lexer until I'm happy
and contribute it back to the pygments project. On your side I just
expect to have the role propagation fixed in the xslt and of course to
accept the doc patches produced.

Let me know if you like the idea.

Best regards,

-- Daniele

---

From: Tom Lane <[email-masked]>
Date: Wed, 13 Apr 2011 11:02:02 -0400

> Is there any interest in applying highlighted syntax to the html

FWIW, there are very few things in the world that I hate more than
randomly multicolored text.  Almost always, some of it is unreadable
in a particular viewing context because of poor contrast.  And as
often as not, the stuff that does stand out isn't the most important
parts of the text.

So no, this isn't a direction I want to see the Postgres documentation
take.

			regards, tom lane

---

From: Susanne Ebrecht <[email-masked]>
Date: Wed, 13 Apr 2011 17:48:52 +0200

> So no, this isn't a direction I want to see the Postgres documentation
Tom,

as color blind (blue blind (tritanope)) person I just can say thanks for 
that
statement.

Colored text / background is always horrible for color blinds.
The best always is black on white or vise versa.

Susanne
P.S.: btw. my google chrome color filter usually makes the postgresql 
elephant
either green or pink

-- 
Susanne Ebrecht - 2ndQuadrant
PostgreSQL Development, 24x7 Support, Training and Services
www.2ndQuadrant.com

---

From: "Kevin Grittner" <[email-masked]>
Date: Wed, 13 Apr 2011 11:14:49 -0500

> Is there any interest in applying highlighted syntax to the html
 
When using an editor I like some color highlighting, as long as I
can control the colors.  Documentation is very different, especially
since the user probably wouldn't have any control over the colors. 
The examples in the links above are very hard for me to read, and
definitely *not* something I would want in the documentation.
 
There might be room to do something limited and subtle, like maybe
making comments a dark gray while other text is black or possibly
making computer-generated output very dark blue versus black for
user input.  It should be subtle enough not to draw attention to it,
but rather to give subtle cues.  If the display screams "HEY! LOOK
HERE!" about anything, it's just wrong.
 
Overall, I'm dubious about whether it would be worth the effort,
even if you can come up with a scheme which people like.
 
-Kevin

---

From: Thom Brown <[email-masked]>
Date: Wed, 13 Apr 2011 17:21:49 +0100

> even if you can come up with a scheme which people like.

If we had any sort of syntax highlighting in the docs, I'd only expect
keywords in bold, and everything else regular.  No colour changes.
Saying that, it would then interfere with examples that have
emphasised text, such as here:
http://www.postgresql.org/docs/9.0/static/ddl-constraints.html#DDL-CONSTRAINTS-FK

And even if it didn't, I don't think it brings enough benefit to
warrant implementation.

-- 
Thom Brown
Twitter: @darkixion
IRC (freenode): dark_ixion
Registered Linux user: #516935

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

---

From: Peter Eisentraut <[email-masked]>
Date: Wed, 13 Apr 2011 21:34:48 +0300

> clueless about debugging the docbooc tool chain).

I guess the verdict is that we shouldn't do it by default, but what we
could do anyway is language tag the code as you describe.  Then users
could either hook in client-side Javascript to do the highlighting or
whoever wants to can produce an alternative colorful pygments-based
version.

---

From: Daniele Varrazzo <[email-masked]>
Date: Wed, 13 Apr 2011 20:20:23 +0100

> whoever wants to can produce an alternative colorful pygments-based

Fine for me, the motivation for not wanting highlight in the official
docs are pretty solid. If you fancy implementing the role to class
propagation I may complete the tagging and provide patches for the
sgml, but I expect it to be pretty much everything.

Regards,

-- Daniele