pg.ddx.io pgsql-docs@postgresql.org mailing list archive
help / color / mirror / Atom feedreplication wordsmithing / clarifications
11+ messages / 3 participants
[nested] [flat]
Commits from this thread:
2020-10-02 8550cbd0ba doc: Improve some documentation about HA and replication
* replication wordsmithing / clarifications
@ 2020-09-11 23:16 Robert Treat <rob@xzilla.net>
2020-09-12 01:45 ` Re: replication wordsmithing / clarifications Michael Paquier <michael@paquier.xyz>
0 siblings, 1 reply; 11+ messages in thread
From: Robert Treat @ 2020-09-11 23:16 UTC (permalink / raw)
To: pgsql-docs
Howdy folks,
A recent discussion on slack prompted me to read through the
high-availability section of the docs, turning in to some suggested
changes that hopefully clarify how the various replication options
work (specifically around logical and trigger-based solutions). A
proposed patch is attached, lmk if you have questions.
Robert Treat
https://xzilla.net
Attachments:
replication-wordsmithing.patch (application/octet-stream, 4.3K) download
diff --git a/doc/src/sgml/high-availability.sgml b/doc/src/sgml/high-availability.sgml
index beb309e668..77cf69252a 100644
--- a/doc/src/sgml/high-availability.sgml
+++ b/doc/src/sgml/high-availability.sgml
@@ -165,10 +165,10 @@ protocol to make nodes agree on a serializable transactional order.
Logical replication allows a database server to send a stream of data
modifications to another server. <productname>PostgreSQL</productname>
logical replication constructs a stream of logical data modifications
- from the WAL. Logical replication allows the data changes from
- individual tables to be replicated. Logical replication doesn't require
- a particular server to be designated as a primary or a replica but allows
- data to flow in multiple directions. For more information on logical
+ from the WAL. Logical replication allows replication of data changes on
+ a per-table basis. In addition, a server that is publishing its own
+ changes can also subscribe to changes from another server, allowing data
+ to flow in multiple directions. For more information on logical
replication, see <xref linkend="logical-replication"/>. Through the
logical decoding interface (<xref linkend="logicaldecoding"/>),
third-party extensions can also provide similar functionality.
@@ -177,22 +177,24 @@ protocol to make nodes agree on a serializable transactional order.
</varlistentry>
<varlistentry>
- <term>Trigger-Based Master-Standby Replication</term>
+ <term>Trigger-Based Primary-Replica Replication</term>
<listitem>
<para>
- A master-standby replication setup sends all data modification
- queries to the master server. The master server asynchronously
- sends data changes to the standby server. The standby can answer
- read-only queries while the master server is running. The
- standby server is ideal for data warehouse queries.
+ A trigger-based replication setup typically funnels data modification
+ queries to a designated primary server. Operating on a per-table basis,
+ the primary server sends data changes (typically) asynchronously to the
+ replica node(s). Replica nodes can answer queries while the primary is
+ running, and may allow some local data changes or write activity. This
+ form of replication is often used for offloading large analytical or data
+ warehouse queries.
</para>
<para>
- <productname>Slony-I</productname> is an example of this type of replication, with per-table
- granularity, and support for multiple standby servers. Because it
- updates the standby server asynchronously (in batches), there is
- possible data loss during fail over.
+ <productname>Slony-I</productname> is an example of this type of
+ replication, with per-table granularity, and support for multiple standby
+ servers. Because it updates the standby server asynchronously (in
+ batches), there is possible data loss during fail over.
</para>
</listitem>
</varlistentry>
@@ -215,14 +217,10 @@ protocol to make nodes agree on a serializable transactional order.
<function>random()</function>, <function>CURRENT_TIMESTAMP</function>, and
sequences can have different values on different servers.
This is because each server operates independently, and because
- SQL queries are broadcast (and not actual modified rows). If
+ SQL queries are broadcast rather than actual data changes. If
this is unacceptable, either the middleware or the application
- must query such values from a single server and then use those
- values in write queries. Another option is to use this replication
- option with a traditional primary-standby setup, i.e., data modification
- queries are sent only to the primary and are propagated to the
- standby servers via primary-standby replication, not by the replication
- middleware. Care must also be taken that all
+ must determine such values from a single source and then use those
+ values in write queries. Care must also be taken that all
transactions either commit or abort on all servers, perhaps
using two-phase commit (<xref linkend="sql-prepare-transaction"/>
and <xref linkend="sql-commit-prepared"/>).
* Re: replication wordsmithing / clarifications
2020-09-11 23:16 replication wordsmithing / clarifications Robert Treat <rob@xzilla.net>
@ 2020-09-12 01:45 ` Michael Paquier <michael@paquier.xyz>
2020-09-12 09:52 ` Re: replication wordsmithing / clarifications Magnus Hagander <magnus@hagander.net>
2020-09-12 16:40 ` Re: replication wordsmithing / clarifications Robert Treat <rob@xzilla.net>
0 siblings, 2 replies; 11+ messages in thread
From: Michael Paquier @ 2020-09-12 01:45 UTC (permalink / raw)
To: Robert Treat <rob@xzilla.net>; +Cc: pgsql-docs
On Fri, Sep 11, 2020 at 07:16:43PM -0400, Robert Treat wrote:
> A recent discussion on slack prompted me to read through the
> high-availability section of the docs, turning in to some suggested
> changes that hopefully clarify how the various replication options
> work (specifically around logical and trigger-based solutions). A
> proposed patch is attached, lmk if you have questions.
A portion of what you are suggesting here is in line with 9e101cf,
which looks like a good thing. Most of the changes look like
improvements to me, and here are some comments.
+ queries to a designated primary server. Operating on a per-tablebasis,
+ the primary server sends data changes (typically) asynchronously to the
+ replica node(s). Replica nodes can answer queries while the primary is
Could it be better to use "one or more replica clusters" here?
This is because each server operates independently, and because
- SQL queries are broadcast (and not actual modified rows). If
+ SQL queries are broadcast rather than actual data changes. If
this is unacceptable, either the middleware or the application
- must query such values from a single server and then use those
- values in write queries. Another option is to use this replication
- option with a traditional primary-standby setup, i.e., data modification
- queries are sent only to the primary and are propagated to the
- standby servers via primary-standby replication, not by the replication
- middleware. Care must also be taken that all
+ must determine such values from a single source and then use those
+ values in write queries. Care must also be taken that all
Why do you remove the sentence about the extra option available to the
user here, about the traditional primary-standby setup?
--
Michael
Attachments:
signature.asc (application/pgp-signature, 832B) download
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: replication wordsmithing / clarifications
2020-09-11 23:16 replication wordsmithing / clarifications Robert Treat <rob@xzilla.net>
2020-09-12 01:45 ` Re: replication wordsmithing / clarifications Michael Paquier <michael@paquier.xyz>
@ 2020-09-12 09:52 ` Magnus Hagander <magnus@hagander.net>
2020-09-14 08:09 ` Re: replication wordsmithing / clarifications Michael Paquier <michael@paquier.xyz>
1 sibling, 1 reply; 11+ messages in thread
From: Magnus Hagander @ 2020-09-12 09:52 UTC (permalink / raw)
To: Michael Paquier <michael@paquier.xyz>; +Cc: Robert Treat <rob@xzilla.net>; pgsql-docs
On Sat, Sep 12, 2020 at 3:46 AM Michael Paquier <michael@paquier.xyz> wrote:
> On Fri, Sep 11, 2020 at 07:16:43PM -0400, Robert Treat wrote:
> > A recent discussion on slack prompted me to read through the
> > high-availability section of the docs, turning in to some suggested
> > changes that hopefully clarify how the various replication options
> > work (specifically around logical and trigger-based solutions). A
> > proposed patch is attached, lmk if you have questions.
>
> A portion of what you are suggesting here is in line with 9e101cf,
> which looks like a good thing. Most of the changes look like
> improvements to me, and here are some comments.
>
+1.
>
> + queries to a designated primary server. Operating on a
> per-tablebasis,
> + the primary server sends data changes (typically) asynchronously to
> the
> + replica node(s). Replica nodes can answer queries while the primary
> is
> Could it be better to use "one or more replica clusters" here?
>
In particular, why is the primary a "server" and the replica a "node"? That
caught my eye for inconsistency -- but changing node to cluster will be
equally inconsistent, just in a different way. Why not just call them both
servers?
--
Magnus Hagander
Me: https://www.hagander.net/ <http://www.hagander.net/;
Work: https://www.redpill-linpro.com/ <http://www.redpill-linpro.com/;
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: replication wordsmithing / clarifications
2020-09-11 23:16 replication wordsmithing / clarifications Robert Treat <rob@xzilla.net>
2020-09-12 01:45 ` Re: replication wordsmithing / clarifications Michael Paquier <michael@paquier.xyz>
2020-09-12 09:52 ` Re: replication wordsmithing / clarifications Magnus Hagander <magnus@hagander.net>
@ 2020-09-14 08:09 ` Michael Paquier <michael@paquier.xyz>
0 siblings, 0 replies; 11+ messages in thread
From: Michael Paquier @ 2020-09-14 08:09 UTC (permalink / raw)
To: Magnus Hagander <magnus@hagander.net>; +Cc: Robert Treat <rob@xzilla.net>; pgsql-docs
On Sat, Sep 12, 2020 at 11:52:11AM +0200, Magnus Hagander wrote:
> In particular, why is the primary a "server" and the replica a "node"? That
> caught my eye for inconsistency -- but changing node to cluster will be
> equally inconsistent, just in a different way. Why not just call them both
> servers?
Makes sense.
--
Michael
Attachments:
signature.asc (application/pgp-signature, 832B) download
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: replication wordsmithing / clarifications
2020-09-11 23:16 replication wordsmithing / clarifications Robert Treat <rob@xzilla.net>
2020-09-12 01:45 ` Re: replication wordsmithing / clarifications Michael Paquier <michael@paquier.xyz>
@ 2020-09-12 16:40 ` Robert Treat <rob@xzilla.net>
2020-09-14 08:11 ` Re: replication wordsmithing / clarifications Michael Paquier <michael@paquier.xyz>
1 sibling, 1 reply; 11+ messages in thread
From: Robert Treat @ 2020-09-12 16:40 UTC (permalink / raw)
To: Michael Paquier <michael@paquier.xyz>; +Cc: pgsql-docs
On Fri, Sep 11, 2020 at 9:46 PM Michael Paquier <michael@paquier.xyz> wrote:
>
> On Fri, Sep 11, 2020 at 07:16:43PM -0400, Robert Treat wrote:
> > A recent discussion on slack prompted me to read through the
> > high-availability section of the docs, turning in to some suggested
> > changes that hopefully clarify how the various replication options
> > work (specifically around logical and trigger-based solutions). A
> > proposed patch is attached, lmk if you have questions.
>
> A portion of what you are suggesting here is in line with 9e101cf,
> which looks like a good thing. Most of the changes look like
> improvements to me, and here are some comments.
>
> + queries to a designated primary server. Operating on a per-tablebasis,
> + the primary server sends data changes (typically) asynchronously to the
> + replica node(s). Replica nodes can answer queries while the primary is
> Could it be better to use "one or more replica clusters" here?
>
maybe "to one or more replicas." or maybe "to one or more replica
targets." I'd avoid the word cluster because you could set it up to
send to multiple databases in the same postgres cluster.
> This is because each server operates independently, and because
> - SQL queries are broadcast (and not actual modified rows). If
> + SQL queries are broadcast rather than actual data changes. If
> this is unacceptable, either the middleware or the application
> - must query such values from a single server and then use those
> - values in write queries. Another option is to use this replication
> - option with a traditional primary-standby setup, i.e., data modification
> - queries are sent only to the primary and are propagated to the
> - standby servers via primary-standby replication, not by the replication
> - middleware. Care must also be taken that all
> + must determine such values from a single source and then use those
> + values in write queries. Care must also be taken that all
> Why do you remove the sentence about the extra option available to the
> user here, about the traditional primary-standby setup?
If you think about it, this sentence is essentially saying "If you
don't like the downsides of this replication method, you can use an
entirely different replication method." Well sure, you could, but once
you are using a different method, you are no longer using the method
we are trying to describe. In addition, there are several other
systems you could use beyond the suggested one, so suggestion this
specific one felt mis-leading or out of place. And it is not something
we talk about in any of the other sections (after all, every
replication system has trade-offs that might not be acceptable, that's
why we have so many of them).
Robert Treat
https://xzilla.net
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: replication wordsmithing / clarifications
2020-09-11 23:16 replication wordsmithing / clarifications Robert Treat <rob@xzilla.net>
2020-09-12 01:45 ` Re: replication wordsmithing / clarifications Michael Paquier <michael@paquier.xyz>
2020-09-12 16:40 ` Re: replication wordsmithing / clarifications Robert Treat <rob@xzilla.net>
@ 2020-09-14 08:11 ` Michael Paquier <michael@paquier.xyz>
2020-09-15 17:47 ` Re: replication wordsmithing / clarifications Robert Treat <rob@xzilla.net>
0 siblings, 1 reply; 11+ messages in thread
From: Michael Paquier @ 2020-09-14 08:11 UTC (permalink / raw)
To: Robert Treat <rob@xzilla.net>; +Cc: pgsql-docs
On Sat, Sep 12, 2020 at 12:40:45PM -0400, Robert Treat wrote:
> maybe "to one or more replicas." or maybe "to one or more replica
> targets." I'd avoid the word cluster because you could set it up to
> send to multiple databases in the same postgres cluster.
Magnus has given a better suggestion: to just use "servers".
> If you think about it, this sentence is essentially saying "If you
> don't like the downsides of this replication method, you can use an
> entirely different replication method." Well sure, you could, but once
> you are using a different method, you are no longer using the method
> we are trying to describe. In addition, there are several other
> systems you could use beyond the suggested one, so suggestion this
> specific one felt mis-leading or out of place. And it is not something
> we talk about in any of the other sections (after all, every
> replication system has trade-offs that might not be acceptable, that's
> why we have so many of them).
OK, I see your point. Seeing like that, it makes sense.
--
Michael
Attachments:
signature.asc (application/pgp-signature, 832B) download
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: replication wordsmithing / clarifications
2020-09-11 23:16 replication wordsmithing / clarifications Robert Treat <rob@xzilla.net>
2020-09-12 01:45 ` Re: replication wordsmithing / clarifications Michael Paquier <michael@paquier.xyz>
2020-09-12 16:40 ` Re: replication wordsmithing / clarifications Robert Treat <rob@xzilla.net>
2020-09-14 08:11 ` Re: replication wordsmithing / clarifications Michael Paquier <michael@paquier.xyz>
@ 2020-09-15 17:47 ` Robert Treat <rob@xzilla.net>
2020-09-16 06:45 ` Re: replication wordsmithing / clarifications Michael Paquier <michael@paquier.xyz>
0 siblings, 1 reply; 11+ messages in thread
From: Robert Treat @ 2020-09-15 17:47 UTC (permalink / raw)
To: Michael Paquier <michael@paquier.xyz>; +Cc: pgsql-docs
On Mon, Sep 14, 2020 at 4:11 AM Michael Paquier <michael@paquier.xyz> wrote:
>
> On Sat, Sep 12, 2020 at 12:40:45PM -0400, Robert Treat wrote:
> > maybe "to one or more replicas." or maybe "to one or more replica
> > targets." I'd avoid the word cluster because you could set it up to
> > send to multiple databases in the same postgres cluster.
>
> Magnus has given a better suggestion: to just use "servers".
>
I think I like targets over servers, since it could be a different
server or the same server or even the same database, but happy either
way.
> > If you think about it, this sentence is essentially saying "If you
> > don't like the downsides of this replication method, you can use an
> > entirely different replication method." Well sure, you could, but once
> > you are using a different method, you are no longer using the method
> > we are trying to describe. In addition, there are several other
> > systems you could use beyond the suggested one, so suggestion this
> > specific one felt mis-leading or out of place. And it is not something
> > we talk about in any of the other sections (after all, every
> > replication system has trade-offs that might not be acceptable, that's
> > why we have so many of them).
>
> OK, I see your point. Seeing like that, it makes sense.
LMK if you'd like an updated patch. Also I wrote this against master,
but could verify it against 13 if folks wanted to backpatch (it might
apply cleanly though)
Robert Treat
https://xzilla.net
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: replication wordsmithing / clarifications
2020-09-11 23:16 replication wordsmithing / clarifications Robert Treat <rob@xzilla.net>
2020-09-12 01:45 ` Re: replication wordsmithing / clarifications Michael Paquier <michael@paquier.xyz>
2020-09-12 16:40 ` Re: replication wordsmithing / clarifications Robert Treat <rob@xzilla.net>
2020-09-14 08:11 ` Re: replication wordsmithing / clarifications Michael Paquier <michael@paquier.xyz>
2020-09-15 17:47 ` Re: replication wordsmithing / clarifications Robert Treat <rob@xzilla.net>
@ 2020-09-16 06:45 ` Michael Paquier <michael@paquier.xyz>
2020-10-02 01:39 ` Re: replication wordsmithing / clarifications Michael Paquier <michael@paquier.xyz>
0 siblings, 1 reply; 11+ messages in thread
From: Michael Paquier @ 2020-09-16 06:45 UTC (permalink / raw)
To: Robert Treat <rob@xzilla.net>; +Cc: pgsql-docs
On Tue, Sep 15, 2020 at 01:47:07PM -0400, Robert Treat wrote:
> I think I like targets over servers, since it could be a different
> server or the same server or even the same database, but happy either
> way.
I'd say the opposite here: servers over targets. But that's just one
opinion.
> LMK if you'd like an updated patch. Also I wrote this against master,
> but could verify it against 13 if folks wanted to backpatch (it might
> apply cleanly though)
If you can send a patch, that would be great. As that's an
improvement, I would go for HEAD only.
--
Michael
Attachments:
signature.asc (application/pgp-signature, 832B) download
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: replication wordsmithing / clarifications
2020-09-11 23:16 replication wordsmithing / clarifications Robert Treat <rob@xzilla.net>
2020-09-12 01:45 ` Re: replication wordsmithing / clarifications Michael Paquier <michael@paquier.xyz>
2020-09-12 16:40 ` Re: replication wordsmithing / clarifications Robert Treat <rob@xzilla.net>
2020-09-14 08:11 ` Re: replication wordsmithing / clarifications Michael Paquier <michael@paquier.xyz>
2020-09-15 17:47 ` Re: replication wordsmithing / clarifications Robert Treat <rob@xzilla.net>
2020-09-16 06:45 ` Re: replication wordsmithing / clarifications Michael Paquier <michael@paquier.xyz>
@ 2020-10-02 01:39 ` Michael Paquier <michael@paquier.xyz>
2020-10-04 17:08 ` Re: replication wordsmithing / clarifications Robert Treat <rob@xzilla.net>
0 siblings, 1 reply; 11+ messages in thread
From: Michael Paquier @ 2020-10-02 01:39 UTC (permalink / raw)
To: Robert Treat <rob@xzilla.net>; +Cc: pgsql-docs
On Wed, Sep 16, 2020 at 03:45:32PM +0900, Michael Paquier wrote:
> If you can send a patch, that would be great. As that's an
> improvement, I would go for HEAD only.
I have gone through this patch set again, and applied most of the
suggested changes on HEAD as of 8550cbd, updating while on it some
extra instances of "master" to "primary" to be more consistent with
the other changes of this patch.
--
Michael
Attachments:
signature.asc (application/pgp-signature, 832B) download
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: replication wordsmithing / clarifications
2020-09-11 23:16 replication wordsmithing / clarifications Robert Treat <rob@xzilla.net>
2020-09-12 01:45 ` Re: replication wordsmithing / clarifications Michael Paquier <michael@paquier.xyz>
2020-09-12 16:40 ` Re: replication wordsmithing / clarifications Robert Treat <rob@xzilla.net>
2020-09-14 08:11 ` Re: replication wordsmithing / clarifications Michael Paquier <michael@paquier.xyz>
2020-09-15 17:47 ` Re: replication wordsmithing / clarifications Robert Treat <rob@xzilla.net>
2020-09-16 06:45 ` Re: replication wordsmithing / clarifications Michael Paquier <michael@paquier.xyz>
2020-10-02 01:39 ` Re: replication wordsmithing / clarifications Michael Paquier <michael@paquier.xyz>
@ 2020-10-04 17:08 ` Robert Treat <rob@xzilla.net>
2020-10-05 01:25 ` Re: replication wordsmithing / clarifications Michael Paquier <michael@paquier.xyz>
0 siblings, 1 reply; 11+ messages in thread
From: Robert Treat @ 2020-10-04 17:08 UTC (permalink / raw)
To: Michael Paquier <michael@paquier.xyz>; +Cc: pgsql-docs
On Thu, Oct 1, 2020 at 9:39 PM Michael Paquier <michael@paquier.xyz> wrote:
>
> On Wed, Sep 16, 2020 at 03:45:32PM +0900, Michael Paquier wrote:
> > If you can send a patch, that would be great. As that's an
> > improvement, I would go for HEAD only.
>
> I have gone through this patch set again, and applied most of the
> suggested changes on HEAD as of 8550cbd, updating while on it some
> extra instances of "master" to "primary" to be more consistent with
> the other changes of this patch.
Merci monsieur!
Robert Treat
https://xzilla.net
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: replication wordsmithing / clarifications
2020-09-11 23:16 replication wordsmithing / clarifications Robert Treat <rob@xzilla.net>
2020-09-12 01:45 ` Re: replication wordsmithing / clarifications Michael Paquier <michael@paquier.xyz>
2020-09-12 16:40 ` Re: replication wordsmithing / clarifications Robert Treat <rob@xzilla.net>
2020-09-14 08:11 ` Re: replication wordsmithing / clarifications Michael Paquier <michael@paquier.xyz>
2020-09-15 17:47 ` Re: replication wordsmithing / clarifications Robert Treat <rob@xzilla.net>
2020-09-16 06:45 ` Re: replication wordsmithing / clarifications Michael Paquier <michael@paquier.xyz>
2020-10-02 01:39 ` Re: replication wordsmithing / clarifications Michael Paquier <michael@paquier.xyz>
2020-10-04 17:08 ` Re: replication wordsmithing / clarifications Robert Treat <rob@xzilla.net>
@ 2020-10-05 01:25 ` Michael Paquier <michael@paquier.xyz>
0 siblings, 0 replies; 11+ messages in thread
From: Michael Paquier @ 2020-10-05 01:25 UTC (permalink / raw)
To: Robert Treat <rob@xzilla.net>; +Cc: pgsql-docs
On Sun, Oct 04, 2020 at 01:08:56PM -0400, Robert Treat wrote:
> Merci monsieur!
De rien, monsieur.
--
Michael
Attachments:
signature.asc (application/pgp-signature, 832B) download
^ permalink raw reply [nested|flat] 11+ messages in thread
end of thread, other threads:[~2020-10-05 01:25 UTC | newest]
Thread overview: 11+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2020-09-11 23:16 replication wordsmithing / clarifications Robert Treat <rob@xzilla.net>
2020-09-12 01:45 ` Michael Paquier <michael@paquier.xyz>
2020-09-12 09:52 ` Magnus Hagander <magnus@hagander.net>
2020-09-14 08:09 ` Michael Paquier <michael@paquier.xyz>
2020-09-12 16:40 ` Robert Treat <rob@xzilla.net>
2020-09-14 08:11 ` Michael Paquier <michael@paquier.xyz>
2020-09-15 17:47 ` Robert Treat <rob@xzilla.net>
2020-09-16 06:45 ` Michael Paquier <michael@paquier.xyz>
2020-10-02 01:39 ` Michael Paquier <michael@paquier.xyz>
2020-10-04 17:08 ` Robert Treat <rob@xzilla.net>
2020-10-05 01:25 ` Michael Paquier <michael@paquier.xyz>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox