Subject: PGDATA confusion

From: Thom Brown <[email-masked]>
Date: Sat, 15 Oct 2011 02:24:46 +0100

Hi,

I notice that in the man page and the page for pg_ctl in the
documentation (http://www.postgresql.org/docs/current/static/app-pg-ctl.html)
it states that the -D parameter should point to the directory which
contains the database files:

"Specifies the file system location of the database files. If this is
omitted, the environment variable PGDATA is used."

This isn't necessarily true.  It will only need to know where the
postgresql.conf file is.  In Debian and Ubuntu, for example, the
"database files" reside in /var/lib/postgresql/x.x/main/, and the
configuration files in /etc/postgresql/x.x/main/, so a user might
assume they need to point it to the former as that's where the actual
database itself can be found, when in fact it needs to be the latter.

This same inaccuracy affects the "postgres" command
(http://www.postgresql.org/docs/current/static/app-postgres.html) to a
lesser extent:

"Specifies the file system location of the data directory or
configuration file(s)."

Which is it?  This suggests it can be either, when only the latter
matters.  If it also happens to be the data directory, that's
unimportant.

To add further confusion, the page describing PGDATA
(http://www.postgresql.org/docs/current/static/storage-file-layout.html)
refers to it like so:

"All the data needed for a database cluster is stored within the
cluster's data directory, commonly referred to as PGDATA...

*snip*

The PGDATA directory contains several subdirectories and control
files... ...In addition to these required items, the cluster
configuration files postgresql.conf, pg_hba.conf, and pg_ident.conf
are traditionally stored in PGDATA (although in PostgreSQL 8.0 and
later, it is possible to keep them elsewhere)."

Traditionally but not necessarily, and not by default in Debian and
Ubuntu.  In fact Gentoo (and therefore probably Sabayon too) has also
elected for this separation of data files and config files as of 9.0.

So if one set PGDATA to somewhere which had no database files at all,
but just postgresql.conf, it could still work (assuming it, in turn,
set data_directory correctly), but not vice versa.  It would make more
sense to call it PGCONFIG, although I'm not proposing that, especially
since PGDATA makes sense when it comes to initdb.

There are probably plenty of other places in the docs which also don't
adequately describe PGDATA or -D.

Any disagreements?  If not, should I write a patch (since someone will
probably accuse me of volunteering anyway) or would someone like to
commit some adjustments?

-- 
Thom Brown
Twitter: @darkixion
IRC (freenode): dark_ixion
Registered Linux user: #516935

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

---

From: Thom Brown <[email-masked]>
Date: Mon, 31 Oct 2011 17:46:55 +0200

> probably accuse me of volunteering anyway) or would someone like to

No opinions on this?

---

From: Bruce Momjian <[email-masked]>
Date: Fri, 4 Nov 2011 12:32:13 -0400 (EDT)

Thom Brown wrote:
> > set data_directory correctly), but not vice versa. ?It would make more
> > Any disagreements? ?If not, should I write a patch (since someone will

Yes.  I had kept it to deal with later.  Please work on a doc patch to
try to clean this up.  pg_upgrade just went through this confusion and I
also was unhappy at how vague things are in this area.

Things got very confusing with pg_upgrade when PGDATA pointed to the
configuration directory and the data_directory GUC pointed to the data
directory.

-- 
  Bruce Momjian  <[email-masked]>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com

  + It's impossible for everything to be true. +

---

From: Bruce Momjian <[email-masked]>
Date: Wed, 15 Aug 2012 23:00:21 -0400

> configuration directory and the data_directory GUC pointed to the data

I have applied the attached doc patch for PG 9.3 to clarify PGDATA.

  + It's impossible for everything to be true. +
diff --git a/doc/src/sgml/ref/pg_ctl-ref.sgml b/doc/src/sgml/ref/pg_ctl-ref.sgml
new file mode 100644
index 90725d9..7a4c2be
*** a/doc/src/sgml/ref/pg_ctl-ref.sgml
--- b/doc/src/sgml/ref/pg_ctl-ref.sgml
*************** PostgreSQL documentation
*** 261,267 ****
        <term><option>--pgdata <replaceable class="parameter">datadir</replaceable></option></term>
        <listitem>
         <para>
!         Specifies the file system location of the database files.  If
          this is omitted, the environment variable
          <envar>PGDATA</envar> is used.
         </para>
--- 261,267 ----
        <term><option>--pgdata <replaceable class="parameter">datadir</replaceable></option></term>
        <listitem>
         <para>
!         Specifies the file system location of the database configuration files.  If
          this is omitted, the environment variable
          <envar>PGDATA</envar> is used.
         </para>
diff --git a/doc/src/sgml/ref/postgres-ref.sgml b/doc/src/sgml/ref/postgres-ref.sgml
new file mode 100644
index a1f36e1..41745fb
*** a/doc/src/sgml/ref/postgres-ref.sgml
--- b/doc/src/sgml/ref/postgres-ref.sgml
*************** PostgreSQL documentation
*** 179,186 ****
        <term><option>-D <replaceable class="parameter">datadir</replaceable></option></term>
        <listitem>
         <para>
!         Specifies the file system location of the data directory or
!         configuration file(s).  See
          <xref linkend="runtime-config-file-locations"> for details.
         </para>
        </listitem>
--- 179,186 ----
        <term><option>-D <replaceable class="parameter">datadir</replaceable></option></term>
        <listitem>
         <para>
!         Specifies the file system location of the database
!         configuration files.  See
          <xref linkend="runtime-config-file-locations"> for details.
         </para>
        </listitem>
diff --git a/doc/src/sgml/storage.sgml b/doc/src/sgml/storage.sgml
new file mode 100644
index bd2dca3..e0a93c1
*** a/doc/src/sgml/storage.sgml
--- b/doc/src/sgml/storage.sgml
*************** directories.
*** 19,25 ****
  </para>
  
  <para>
! All the data needed for a database cluster is stored within the cluster's data
  directory, commonly referred to as <varname>PGDATA</> (after the name of the
  environment variable that can be used to define it).  A common location for
  <varname>PGDATA</> is <filename>/var/lib/pgsql/data</>.  Multiple clusters,
--- 19,26 ----
  </para>
  
  <para>
! Traditionally, the configuration and data files used by a database
! cluster are stored together within the cluster's data
  directory, commonly referred to as <varname>PGDATA</> (after the name of the
  environment variable that can be used to define it).  A common location for
  <varname>PGDATA</> is <filename>/var/lib/pgsql/data</>.  Multiple clusters,
*************** these required items, the cluster config
*** 33,39 ****
  <filename>postgresql.conf</filename>, <filename>pg_hba.conf</filename>, and
  <filename>pg_ident.conf</filename> are traditionally stored in
  <varname>PGDATA</> (although in <productname>PostgreSQL</productname> 8.0 and
! later, it is possible to keep them elsewhere).
  </para>
  
  <table tocentry="1" id="pgdata-contents-table">
--- 34,40 ----
  <filename>postgresql.conf</filename>, <filename>pg_hba.conf</filename>, and
  <filename>pg_ident.conf</filename> are traditionally stored in
  <varname>PGDATA</> (although in <productname>PostgreSQL</productname> 8.0 and
! later, it is possible to place them elsewhere).
  </para>
  
  <table tocentry="1" id="pgdata-contents-table">

---

From: Thom Brown <[email-masked]>
Date: Thu, 16 Aug 2012 08:30:48 +0100

> I have applied the attached doc patch for PG 9.3 to clarify PGDATA.

Thanks Bruce.

-- 
Thom

---

From: Thom Brown <[email-masked]>
Date: Sun, 14 Apr 2013 08:56:12 +0100

>> I have applied the attached doc patch for PG 9.3 to clarify PGDATA.

I've found another unfortunate inconsistency.

PGDATA is not necessarily the same location in these 2 commands:

pg_ctl start -D DATADIR
pg_ctl stop -D DATADIR

The first one requires that the postgresql.conf file be located in the
specified directory.  The second one needs to find the pid file.  On
Debian/Ubuntu/Linux Mint/Gentoo (and probably most other Linux
distros), it would mean 2 different locations for each:

pg_ctl start -D /etc/postgresql/9.2/main/
pg_ctl stop -D /var/lib/postgresql/9.2/main/

pg_ctl --help confusingly tells us that DATADIR is the "location of
the database storage area".  But this clearly isn't true when starting
a cluster.

--
Thom

-- 
Sent via pgsql-docs mailing list ([email-masked])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-docs

---

From: Tom Lane <[email-masked]>
Date: Sun, 14 Apr 2013 10:50:46 -0400

> pg_ctl stop -D /var/lib/postgresql/9.2/main/

This is one of the reasons why an external config file isn't as great
an idea as some people think.

I wonder whether we shouldn't simply remove the ability for
postgresql.conf to exist outside the data directory (which would be
mechanized by removing the ability to set data_directory to something
other than the place where the config file is found).  People who prefer
to keep their config somewhere else can reduce the in-the-directory file
to just "include /some/other/file".  But otherwise, this would get rid
of a confusing and completely unnecessary inconsistency between
different installations.

			regards, tom lane