From: Achilleas Mantzios <achill@matrix.gatewaynet.com>
To: pgsql-admin@lists.postgresql.org
Subject: Re: Logical replication and wal segment retention
Date: Wed, 27 Feb 2019 14:57:46 +0200
Message-ID: <59567e0a-78e4-ed1c-f5b1-033aafcfb68a@matrix.gatewaynet.com> (raw)
In-Reply-To: <20190227125242.6ixy4jkzcuj55cdl@vault.lan>
References: <695C083D-AFB5-431B-B022-ED442CE94794@gmail.com>
<20190227125242.6ixy4jkzcuj55cdl@vault.lan>
On 27/2/19 2:52 μ.μ., Johannes Truschnigg wrote:
> Hi Jay,
>
> On Wed, Feb 27, 2019 at 07:40:26AM -0500, John Scalia wrote:
>> Hello, folks,
>>
>> Yesterday, I had a small file system fill up, due to some logical
>> replication testing we had been performing. We had been testing IBM’s IIDR
>> system and apparently it had built a logical replication slot on my server.
>> When the test was completed, nobody removed the slot, so WAL segments
>> stopped being dropped. Now I can understand the difficulty separating what
>> physical versus logical replication needs from the WAL segments, but as
>> logical replication is database specific not cluster wide, this behavior was
>> a little unexpected, since the WAL segments are cluster wide. Are WAL
>> segments going to pile up whenever something drops a logical replication
>> connection? I’ve seen it, but it seems like this could be a bad thing.
> Since Logical Replication is piggybacked on Physical Replication, you cannot
> use the first without having the latter. And yes, what you experienced is one
> of the dangers of using replication slots when having a busy database (i.e.
> producing lots of WAL) and a filesystem with little excess space. Under these
> circumstances, it is imperative to monitor for (and alert on) anything going
> awry with your replication slot consumers, and/or the size of your wal/xlog
> directory. It's a feature of replication slots to work that way - but one that
> may end up biting you.
A logical approach for replication slots would be to accept a parameter regarding max WAL files to retain, after which newer WALs will be removed and the primary server saved. Pretty much like :
--archive-push-queue-max argument of pgbackrest .
>
--
Achilleas Mantzios
IT DEV Lead
IT DEPT
Dynacom Tankers Mgmt
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-admin@postgresql.org
Cc: achill@matrix.gatewaynet.com, pgsql-admin@lists.postgresql.org
Subject: Re: Logical replication and wal segment retention
In-Reply-To: <59567e0a-78e4-ed1c-f5b1-033aafcfb68a@matrix.gatewaynet.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox