Subject: libpq options

From: [email-masked]
Date: Wed, 22 Nov 2017 13:27:14 +0000

The following documentation comment has been logged on the website:

Page: https://www.postgresql.org/docs/9.6/static/libpq-connect.html
Description:

Hi everyone, 

The documentation refers the libpq connection option &quot;replication&quot; here in
&quot;Streaming Replication Protocol&quot; page;

https://www.postgresql.org/docs/current/static/protocol-replication.html

However the option &quot;replication&quot; is not clarified (or even listed) in libpq
options page.

https://www.postgresql.org/docs/current/static/libpq-connect.html#LIBPQ-PARAMKEYWORDS

I searched the source and find that pg_conn struct has this 

struct pg_conn
{
...
  char *replication; /* connect as the replication standby? */
...
}

Adding something simple like above and referring back to one of replication
pages would be good I think. 

thanks

---

From: Michael Paquier <[email-masked]>
Date: Wed, 22 Nov 2017 23:36:34 +0900

> pages would be good I think.

Yeah, it is mainly a developer option which is why I guess it is not
documented. Like you, I think it should be added as part of the
connection parameter, and mentioned it a couple of days back:
https://www.postgresql.org/message-id/CAB7nPqQAtKfG3H%2BuK11JNivtJtZYE9yVCrPuejRMjp8tUDe0nQ%40mail.gmail.com
-- 
Michael

---

From: Michael Paquier <[email-masked]>
Date: Sat, 25 Nov 2017 19:05:54 +0900

> https://www.postgresql.org/message-id/CAB7nPqQAtKfG3H%2BuK11JNivtJtZYE9yVCrPuejRMjp8tUDe0nQ%40mail.gmail.com

Attached is a patch as an attempt to bring together the best of both
worlds. The idea is to move the description of how the connection
parameter replication works from the replication protocol page into
the section of libpq dedicated to connection parameters, and add links
between both sections.

Thoughts?
-- 
Michael
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index 4703309254..09bfcbbec3 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -1222,6 +1222,24 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname
       </listitem>
      </varlistentry>
 
+     <varlistentry id="libpq-connect-replication" xreflabel="replication">
+      <term><literal>replication</literal></term>
+      <listitem>
+      <para>
+       This option determines if a backend should use the replication
+       protocol. A Boolean value of <literal>true</literal> tells the backend
+       to go into walsender mode, wherein a small set of replication commands
+       can be issued instead of SQL statements. Only the simple query protocol
+       can be used in walsender mode. Passing <literal>database</literal>
+       as the value instructs walsender to connect to the database specified
+       in the <literal>dbname</literal> parameter, which will allow the
+       connection to be used for logical replication from that database.
+       For a detailed description about the replication protocol, consult
+       <xref linkend="protocol-replication"/>.
+      </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry id="libpq-connect-sslmode" xreflabel="sslmode">
       <term><literal>sslmode</literal></term>
       <listitem>
diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index 8174e3defa..b85a3edda9 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -1630,15 +1630,9 @@ supported at the moment is <literal>tls-unique</literal>, defined in RFC 5929.
 
 <para>
 To initiate streaming replication, the frontend sends the
-<literal>replication</literal> parameter in the startup message. A Boolean value
-of <literal>true</literal> tells the backend to go into walsender mode, wherein a
-small set of replication commands can be issued instead of SQL statements. Only
-the simple query protocol can be used in walsender mode.
-Replication commands are logged in the server log when
+<xref linkend="libpq-connect-replication"/> connection parameter in the
+startup message. Replication commands are logged in the server log when
 <xref linkend="guc-log-replication-commands"/> is enabled.
-Passing <literal>database</literal> as the value instructs walsender to connect to
-the database specified in the <literal>dbname</literal> parameter, which will allow
-the connection to be used for logical replication from that database.
 </para>
 <para>
  For the purpose of testing replication commands, you can make a replication

---

From: Şahap Aşçı <[email-masked]>
Date: Tue, 28 Nov 2017 11:00:14 +0300

I think this solves the consistency issue that i am talking about. Well, i
am just looking from documentation user point of view.

If it's developer only option and shouldn't be documented then maybe adding
a 'IDENTIFY_SYSTEM' parameter to pg_receivexlog is more appropriate.
like this pg_receivexlog ...  --identify-system

On Sat, Nov 25, 2017 at 1:05 PM, Michael Paquier <[email-masked]>
wrote:

> > https://www.postgresql.org/message-id/CAB7nPqQAtKfG3H%
> 2BuK11JNivtJtZYE9yVCrPuejRMjp8tUDe0nQ%40mail.gmail.com
>

-- 
Şahap Aşçı
<https://tr.linkedin.com/in/sahapasci>

---

From: Michael Paquier <[email-masked]>
Date: Tue, 28 Nov 2017 17:21:37 +0900

On Tue, Nov 28, 2017 at 5:00 PM, Şahap Aşçı <[email-masked]> wrote:
> I think this solves the consistency issue that i am talking about. Well, i am just looking from documentation user point of view.

Thanks. Input is always welcome. I have added an entry in the commit
fest app so as this is not lost:
https://commitfest.postgresql.org/16/1387/
-- 
Michael

---

From: Vik Fearing <[email-masked]>
Date: Tue, 9 Jan 2018 17:53:57 +0100

> https://commitfest.postgresql.org/16/1387/

I approve of this change.

Ready for Committer.
-- 
Vik Fearing                                          +33 6 46 75 15 36
http://2ndQuadrant.fr     PostgreSQL : Expertise, Formation et Support

---

From: Andres Freund <[email-masked]>
Date: Thu, 1 Mar 2018 01:35:54 -0800

On 2017-11-25 19:05:54 +0900, Michael Paquier wrote:
> +       protocol.

"should"? The backend will, and the client will have to do so as well.

> A Boolean value of <literal>true</literal> tells the backend
> +       can be issued instead of SQL statements.

This actually is wrong now I think. Petr?

Greetings,

Andres Freund

---

From: Michael Paquier <[email-masked]>
Date: Fri, 2 Mar 2018 10:50:38 +0900

> This actually is wrong now I think. Petr?

On more or less HEAD:
$ psql -X -d "replication=1 dbname=postgres"
postgres=# create table aa (a int);
ERROR:  cannot execute SQL commands in WAL sender for physical replication
$ psql -X -d "replication=database dbname=postgres"
postgres=# create table aa (a int);
CREATE TABLE

So one needs to use replication=database in order to be able to issue
normal SQL statements, while replication=true enforces physical
replication where this cannot happen (no connection to a specified
database).
--
Michael

---

From: Petr Jelinek <[email-masked]>
Date: Fri, 2 Mar 2018 10:44:13 +0100

> "should"? The backend will, and the client will have to do so as well.

I think so as well.

> This actually is wrong now I think. Petr?

If the replication is true, we don't have database connection so only
replication commands can be executed.

SQL commands work when replication is set to database.

-- 
  Petr Jelinek                  http://www.2ndQuadrant.com/
  PostgreSQL Development, 24x7 Support, Training & Services

---

From: Magnus Hagander <[email-masked]>
Date: Fri, 2 Mar 2018 12:58:50 +0100

> replication where this cannot happen (no connection to a specified

To nitpick:

+       protocol. A Boolean value of <literal>true</literal> tells the
backend

We don't really have boolean values here, do we? It's just the string true
that's treated as a boolean by the backend. It just sounds really weird to
me when written that way. Particularly since the next sentence talks about
passing "database" as the same thing.

I know that's pretty much nitpicking, but I want to make sure I haven't
actually missed/forgotten how some part of that one is handled..

It also talks separately about "going into walsender mode" (=physical
replication) and "instructs the walsender to connect to the database". I
think that's a bit confusing. I suggest just calling it "physical
replication mode" and "logical replication mode", and not bother mentioning
walsender since that's quite internal.

-- 
 Magnus Hagander
 Me: https://www.hagander.net/ <http://www.hagander.net/>
 Work: https://www.redpill-linpro.com/ <http://www.redpill-linpro.com/>

---

From: Michael Paquier <[email-masked]>
Date: Mon, 5 Mar 2018 13:44:39 +0900

> actually missed/forgotten how some part of that one is handled..

Yes, that's indeed a bit inconsistent with the other parameters able to
use boolean-like parameters, like sslcompression or sslmode (deprecated
in favor requiressl), still those two ones are frontend-only
parameters, so they are just able to use "1" or "0".  Replication is
part of the startup packet which uses parse_bool() so valid values can
be true, false, yes, no, on, off, 1 or 0.  And it is even possible to
use upper-case characters.  So why not documenting all that precisely
then?

> walsender since that's quite internal.

Indeed, this term is pretty spread across the documentation as well.

I have taken into account this feedback, and created the new version
attached, for a v2.

Thoughts?
--
Michael
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index 2a8e1f2e07..8934559239 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -1262,6 +1262,57 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname
       </listitem>
      </varlistentry>
 
+     <varlistentry id="libpq-connect-replication" xreflabel="replication">
+      <term><literal>replication</literal></term>
+      <listitem>
+      <para>
+       This option determines if a backend should use the replication
+       protocol.  For a detailed description about the replication protocol,
+       consult <xref linkend="protocol-replication"/>. The following values,
+       which are case-insensitive, are supported:
+       <variablelist>
+        <varlistentry>
+         <term>
+          <literal>true</literal>, <literal>on</literal>,
+           <literal>yes</literal> or <literal>1</literal>
+         </term>
+         <listitem>
+          <para>
+           The backend goes into physical replication mode, wherein a small
+           set of replication commands can be issued instead of SQL statements.
+           Only the simple query protocol can be used in this case.
+          </para>
+         </listitem>
+        </varlistentry>
+
+        <varlistentry>
+         <term><literal>database</literal></term>
+         <listitem>
+          <para>
+           The backend goes into logical replication mode, as the value
+           instructs the backend to connect to the database specified in
+           the <literal>dbname</literal> parameter.  Only the simple query
+           protocol can be used in this case.
+          </para>
+         </listitem>
+        </varlistentry>
+
+        <varlistentry>
+         <term>
+          <literal>false</literal>, <literal>off</literal>,
+           <literal>no</literal> or <literal>0</literal>
+         </term>
+         <listitem>
+          <para>
+           The backend is a regular one, which is the default behavior.
+          </para>
+         </listitem>
+        </varlistentry>
+       </variablelist>
+      </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry id="libpq-connect-sslmode" xreflabel="sslmode">
       <term><literal>sslmode</literal></term>
       <listitem>
diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index 3cec9e0b0c..2e231efac0 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -1635,15 +1635,9 @@ that cannot support <literal>tls-unique</literal> for some reason.
 
 <para>

---

From: Peter Eisentraut <[email-masked]>
Date: Tue, 6 Mar 2018 21:05:38 -0500

> I have taken into account this feedback, and created the new version

I didn't like so much shrinking down the protocol section.  I have often
wanted *more* detail there, not less.  So I combined your patch for the
libpq chapter and added a bit more in the protocol chapter as well.

Committed.

-- 
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

---

From: Michael Paquier <[email-masked]>
Date: Wed, 7 Mar 2018 11:20:49 +0900

> libpq chapter and added a bit more in the protocol chapter as well.

Thanks for the commit, my idea was to limit duplication between both
sections.  If things get changed at some point, this avoids parts of the
documentation to rot.

 <para>
-The commands accepted in walsender mode are:
+Replication commands are logged in the server log when
+<xref linkend="guc-log-replication-commands"/> is enabled.
+</para>
+
+<para>
+The commands accepted in replication mode are:
Good addition here.
--
Michael