Subject: Documentation and explanatory diagrams

From: Rafael Martinez <[email-masked]>
Date: Mon, 05 Apr 2010 15:33:19 +0200

Hello

I am wondering if we have ever evaluated the use of explanatory diagrams
in the PostgreSQL documentation.

I have always thought that a good diagram is worth a thousand words. I
think the PostgreSQL manual would improve its quality if we get some
good diagrams on the right places.

They will help novice and advanced users who are not involved in the
development of PostgreSQL, to get a quick overview  of some central
components in a PostgreSQL system when they read the documentation.

Some of the possible diagrams that come to my mind in none particular
order are:

 - PITR
 - Streaming Replication
 - Hot Standby
 - Write-Ahead Log
 - System Catalogs
 - Architectural Fundamentals overview
 - PostgreSQL server overview
 - Disk layout
 - Database Roles and Privileges
 - Database Physical Storage
 - Security and authentication

What do you think about the idea? I could start working on an initial
version of some of these diagrams if you are interested in the idea.

regards,
- --
 Rafael Martinez, <[email-masked]>
 Center for Information Technology Services
 University of Oslo, Norway

 PGP Public Key: http://folk.uio.no/rafael/

---

From: Oleg Bartunov <[email-masked]>
Date: Mon, 5 Apr 2010 17:53:55 +0400 (MSD)

That would be very nice to have !

Oleg
On Mon, 5 Apr 2010, Rafael Martinez wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hello
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
>
> iEYEARECAAYFAku55ocACgkQBhuKQurGihRbJQCfVA9SYVWkyfudFg7XPDlLXYeV
> tKAAmgMjg1tfU63sf0RTMRoFTtfsn/Pj
> =2+9o
> -----END PGP SIGNATURE-----
>
>

 	Regards,
 		Oleg
_____________________________________________________________
Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru),
Sternberg Astronomical Institute, Moscow University, Russia
Internet: [email-masked], http://www.sai.msu.su/~megera/
phone: +007(495)939-16-83, +007(495)939-23-83

---

From: Tom Lane <[email-masked]>
Date: Mon, 05 Apr 2010 10:02:12 -0400

> in the PostgreSQL documentation.

There actually used to be some.  We've moved away from them, I believe,
because of the difficulty of maintenance; which stems mainly from the
lack of any universally-available tools.  Unless you can tell me what
you're going to draw the diagram with and how people on Unix, Windows,
or Mac could modify it later, I'm going to not be very interested ...

			regards, tom lane

---

From: Rafael Martinez <[email-masked]>
Date: Mon, 05 Apr 2010 16:52:09 +0200

> or Mac could modify it later, I'm going to not be very interested ...

Well, I was thinking about DIA [1]. It runs on Unix, Windows and Mac.
It loads and saves diagrams to a custom XML format and it can export
diagrams to a number of formats, including EPS, SVG, XFIG, WMF and PNG.

I have to admit that I have only use this program on Linux.

[1] http://live.gnome.org/Dia/

regards
- --

---

From: Peter Eisentraut <[email-masked]>
Date: Mon, 05 Apr 2010 20:27:38 +0300

> diagrams to a number of formats, including EPS, SVG, XFIG, WMF and PNG.

Preferably, any tool that we would use would save a reasonably plain
text source file that we could check into VCS and would provide a tool
for automatically converting to a variety of target formats.  For
example, graphviz could work well.  (Not saying that graphviz is the
right tool for producing the kinds of diagrams that you want, but it
provides the right toolchain interfaces.)  Dia might, but it would be
interesting to see how human-readable that XML format really is.  (This
is necessary for change tracking.  I would like to use a diff tool to
see what happened to a diagram over various revisions.  If opening the
file in the editing tool, changing one bit, and saving it produces a
completely different machine-readable-only XML mush, then it's no good.)

---

From: Greg Smith <[email-masked]>
Date: Mon, 05 Apr 2010 15:30:46 -0400

> diagrams to a number of formats, including EPS, SVG, XFIG, WMF and PNG.

I keep a copy of Windows around to run exactly four programs:  Word, 
Excel, Quicken, and Visio, the program Dia was partially inspired by.  
Dia has such a terrible UI that I will boot an entire VM just to use 
Visio instead.

In any case, the sort of diagrams you're asking about are created 
regularly, they just show up in presentations created for conferences 
and the like rather than appearing in the official documentation.  
http://wiki.postgresql.org/wiki/PostgreSQL_Related_Slides_and_Presentations 
is a good starting point.  The short list at 
http://www.postgresql.org/developer/coding has two of the better ones 
around, Tom's "A Tour of PostgreSQL Internals" and Bruce's "Database 
Internals Presentation", which has now expanded to be "PostgreSQL 
Internals Through Pictures" at 
http://momjian.us/main/writings/pgsql/internalpics.pdf

Given that's how the work is being done by community members right now, 
I think you'd make better progress asking how to index the material in 
those slides better and provide a hotlist of good slides on the Wiki.  I 
consider fighting the battle to get the diagrams directly into the docs 
the way it would need to be done to be both impractical and a waste of 
resources, given many of these diagrams do already exist (and new ones 
are constantly generated) if you follow the presentation scene.

-- 
Greg Smith  2ndQuadrant US  Baltimore, MD
PostgreSQL Training, Services and Support
[email-masked]   www.2ndQuadrant.us

---

From: Alvaro Herrera <[email-masked]>
Date: Mon, 5 Apr 2010 16:45:42 -0400

> completely different machine-readable-only XML mush, then it's no good.)

It's ugly, but it seems to work reasonably.  You have to explicitely
tell it not to compress the .dia files.

-- 
Alvaro Herrera                                http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

---

From: Rafael Martinez <[email-masked]>
Date: Tue, 06 Apr 2010 00:58:27 +0200

Hello

I mentioned Dia because is the one I have experience with and I know it
works with different operative systems. It is not perfect but in my
opinion it is good enough and it does a decent job.

With Dia:

* We have a program that works on Unix, Windows and Mac.
* We can create/update a diagram in an easy way without having to learn
  a new complex/powerful system.
* We can save the diagrams in a plain text format that can be check into
  CVS.
* We can convert to a variety of target formats with commandline tools.
  I have had good experience converting to png and eps.

I have made a quick check of some of your concerns:

* Changing one bit a diagram and saving it, does not produce a
  completely different machine-readable-only XML mush. It changes only
  the portion of the code affected by your changes. A diff will show
  only the lines affected by your changes.

* It looks like the objects in a diagram are saved in the same order
  they were created.

* Modified objects keep their position in the saved file.

* The XML format used is human-readable and not so difficult to
  understand but it has so much information that it would not be a good
  idea to edit it manually. Straightforward changes will not be
  difficult to do manually, complex changes have to be done via the
  program.

regards,
- --

---

From: Rafael Martinez <[email-masked]>
Date: Tue, 06 Apr 2010 00:59:51 +0200

Greg Smith wrote:
[.......]
> are constantly generated) if you follow the presentation scene.

I still think that the right place to have explanatory diagrams is in
the official documentation and not a wiki or external document with
pointers to other documents/presentations.

They would make some important parts of the documentation much more
easier to understand. This is especially important for new users of
PostgreSQL and users who are trying to get an overview of PostgreSQL in
a evaluation process.

I know that most contributors have more than enough to do with the new
release, and I don't want to create more work at this time. So maybe it
will be better to fight this battle after 9.0 is released. :-)

regards
- --

---

From: Bruce Momjian <[email-masked]>
Date: Tue, 13 Apr 2010 21:02:09 -0400 (EDT)

> will be better to fight this battle after 9.0 is released. :-)

OK, DIA sounds like it meets the requirements, and it would be good to
get some images into our documentation.  I wonder if it could import
some of my xfig diagrams;  if so, I could probably keep the images you
took from me updated in CVS.

-- 
  Bruce Momjian  <[email-masked]>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com

---

From: Rafael Martinez <[email-masked]>
Date: Fri, 23 Apr 2010 12:00:46 +0200

Bruce Momjian wrote:
[...]
> took from me updated in CVS.

Where can I get these xfig diagrams? I can play with them and see how it
works to import/convert them.

- --

---

From: Bruce Momjian <[email-masked]>
Date: Sat, 24 Apr 2010 21:44:24 -0400 (EDT)

> works to import/convert them.

I would have to email them to you;  here is one for testing:

	http://momjian.us/expire/mvcc.fig

---

From: Rafael Martinez <[email-masked]>
Date: Mon, 26 Apr 2010 09:53:13 +0200

> 	http://momjian.us/expire/mvcc.fig

I have been playing with this file and DIA-0.96.1 on Linux and this is
the result:

* Importing/converting from xfig to dia format worked with almost no
problems.

* DIA complained that it could not find 'Latex font 16' and that it
would use 'sans' instead. An explanation about this could be what they
explain in this url: http://projects.gnome.org/dia/fonts.html

* The diagram was imported automatically with the right layout but
without the red color in the boxes. This was fixed without problems
manually.

After converting this file to DIA I have generated several other formats
from it. Here are they: http://folk.uio.no/rafael/pgsql_diagrams/

* png, jpeg and eps are generated with the right layout, color and font
size.

* xfig is generated with the right layout and color but the font size is
too big

* svg has big problems with the font size under google chrome and is
'almost' right under firefox.

- --

---

From: Bruce Momjian <[email-masked]>
Date: Mon, 26 Apr 2010 10:07:30 -0400 (EDT)

> 'almost' right under firefox.

Looks good.  I have created a tarball of all my xfig figures:

	http://momjian.us/expire/xfig.tgz

While they are licensed as Creative Commons Attribution, you can
consider them BSD-licensed for inclusion into our Postgres docs.

Let me know what else I can do to help.