MIME-Version: 1.0
In-Reply-To: <4E0CABC3.7000101@agliodbs.com>
References: <BANLkTi==TTzHDqWzwJDjmOf__8YuA7L1jw@mail.gmail.com>
	<BANLkTi=QScpDTkUw3iw6JaO15uUvx_P4qg@mail.gmail.com>
	<BANLkTimarB9eLhHpW73ja2oGvCaWNQuD6g@mail.gmail.com>
	<BANLkTi=hR3vGUDX-SQf=nFcL9ycmrQ6Fcg@mail.gmail.com>
	<BANLkTikPdjn98yjSR-VYywyjMe64wdrdDQ@mail.gmail.com>
	<26007.1303309175@sss.pgh.pa.us>
	<BANLkTikqGottQvyBzrXsARkaWMB2OEMGEA@mail.gmail.com>
	<27149.1303312506@sss.pgh.pa.us>
	<BANLkTin=22tyhK5zaNm7Nt-Q+MHcsNHdLg@mail.gmail.com>
	<BANLkTikFhVOhmf_XKERe=MkU3Ht_K7posQ@mail.gmail.com>
	<BANLkTi=COgk+8cieWxhM7YN9guYXOrDb_g@mail.gmail.com>
	<BANLkTingeD+XrxN-gDv0pzUMHrPcHWSF+A@mail.gmail.com>
	<BANLkTim8rKuUbDP=TxWmtfvZk8C1vmgmsw@mail.gmail.com>
	<BANLkTi=ufQk9i2Ria1U-c0VDUMUNxKjPgg@mail.gmail.com>
	<BANLkTi=33oiYV36cM_nOW7_Pi3t4E=P32g@mail.gmail.com>
	<4E0BD74B.7020105@agliodbs.com>
	<BANLkTimntX0B4ao_YKZemo7VtF8C3vy3-g@mail.gmail.com>
	<BANLkTi=nvWefCymUz8_mT_UK16trw-si6Q@mail.gmail.com>
	<4E0CABC3.7000101@agliodbs.com>
Date: Thu, 30 Jun 2011 13:05:48 -0400
Message-ID: <BANLkTin4OhdSywpOP2n2gt+vt+fTBYBKHA@mail.gmail.com>
Subject: Re: time-delayed standbys
From: Robert Haas <robertmhaas@gmail.com>
To: Josh Berkus <josh@agliodbs.com>
Cc: pgsql-hackers@postgresql.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On Thu, Jun 30, 2011 at 1:00 PM, Josh Berkus <josh@agliodbs.com> wrote:
> On 6/30/11 2:00 AM, Simon Riggs wrote:
>>>> Manual (or scripted) intervention is always necessary if you reach dis=
k
>>>> >> 100% full.
>>> >
>>> > Wow, that's a pretty crappy failure mode... but I don't think we need
>>> > to fix it just on account of this patch. =A0It would be nice to fix, =
of
>>> > course.
>> How is that different to running out of space in the main database?
>>
>> If I try to pour a pint of milk into a small cup, I don't blame the cup.
>
> I have to agree with Simon here. =A0;-)
>
> We can do some things to make this easier for administrators, but
> there's no way to "solve" the problem. =A0And the things we could do woul=
d
> have to be advanced optional modes which aren't on by default, so they
> wouldn't really help the DBA with poor planning skills. =A0Here's my
> suggestions:
>
> 1) Have a utility (pg_archivecleanup?) which checks if we have more than
> a specific settings's worth of archive_logs, and breaks replication and
> deletes the archive logs if we hit that number. =A0This would also requir=
e
> some way for the standby to stop replicating *without* becoming a
> standalone server, which I don't think we currently have.
>
> 2) Have a setting where, regardless of standby_delay settings, the
> standby will interrupt any running queries and start applying logs as
> fast as possible if it hits a certain number of unapplied archive logs.
> =A0Of course, given the issues we had with standby_delay, I'm not sure I
> want to complicate it further.
>
> I think we've already fixed the biggest issue in 9.1, since we now have
> a limit on the number of WALs the master will keep if archiving is
> failing ... yes? =A0That's the only big *avoidable* failure mode we have,
> where a failing standby effectively shuts down the master.

I'm not sure we changed anything in this area for 9.1.  Am I wrong?
wal_keep_segments was present in 9.0.  Using that instead of archiving
is a reasonable way to bound the amount of disk space that can get
used, at the cost of possibly needing to rebuild the standby if things
get too far behind.  Of course, in any version, you could also use an
archive_command that will remove old files to make space if the disk
is full, with the same downside: if the standby isn't done with those
files, you're now in for a rebuild.

--=20
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company