From: Tom Lane <tgl@sss.pgh.pa.us>
To: Noah Misch <noah@leadboat.com>
cc: Tomas Vondra <tomas.vondra@2ndquadrant.com>, pgsql-hackers@postgresql.org
Subject: Re: Spurious "apparent wraparound" via SimpleLruTruncate() rounding
In-reply-to: <20200330052809.GB2324620@rfd.leadboat.com>
References: <20190202083822.GC32531@gust.leadboat.com>
 <20190214072623.GA1139206@rfd.leadboat.com>
 <20200105003355.n2atv7thx7dqiqbt@development>
 <20200105011916.GC2270238@rfd.leadboat.com> <2271.1584655492@sss.pgh.pa.us>
 <20200322083452.GB1918808@rfd.leadboat.com> <13116.1585168951@sss.pgh.pa.us>
 <20200330052809.GB2324620@rfd.leadboat.com>
Comments: In-reply-to Noah Misch <noah@leadboat.com>
	message dated "Sun, 29 Mar 2020 22:28:09 -0700"
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-ID: <1958.1586198769.1@sss.pgh.pa.us>
Content-Transfer-Encoding: quoted-printable
Date: Mon, 06 Apr 2020 14:46:09 -0400
Message-ID: <1959.1586198769@sss.pgh.pa.us>
Precedence: bulk

Noah Misch <noah@leadboat.com> writes:
> On Wed, Mar 25, 2020 at 04:42:31PM -0400, Tom Lane wrote:
>> So I think what we're actually trying to accomplish here is to
>> ensure that instead of deleting up to half of the SLRU space
>> before the cutoff, we delete up to half-less-one-segment.
>> Maybe it should be half-less-two-segments, just to provide some
>> cushion against edge cases.  Reading the first comment in
>> SetTransactionIdLimit makes one not want to trust too much in
>> arguments based on the exact value of xidWrapLimit, while for
>> the other SLRUs it was already unclear whether the edge cases
>> were exactly right.

> That could be interesting insurance.  While it would be sad for us to mi=
ss an
> edge case and print "must be vacuumed within 2 transactions" when wrap h=
as
> already happened, reaching that message implies the DBA burned ~1M XIDs,=
 all
> in single-user mode.  More plausible is FreezeMultiXactId() overrunning =
the
> limit by tens of segments.  Hence, if we do buy this insurance, let's sk=
ip far
> more segments.  For example, instead of unlinking segments representing =
up to
> 2^31 past XIDs, we could divide that into an upper half that we unlink a=
nd a
> lower half.  The lower half will stay in place; eventually, XID consumpt=
ion
> will overwrite it.  Truncation behavior won't change until the region of=
 CLOG
> for pre-oldestXact XIDs exceeds 256 MiB.  Beyond that threshold,
> vac_truncate_clog() will unlink the upper 256 MiB and leave the rest.  C=
LOG
> maximum would rise from 512 MiB to 768 MiB.  Would that be worthwhile?

Hmm.  I'm not particularly concerned about the disk-space-consumption
angle, but I do wonder about whether we'd be sacrificing the ability to
recover cleanly from a situation that the code does let you get into.
However, as long as we're sure that the system will ultimately reuse/
recycle a not-deleted old segment file without complaint, it's hard to
find much fault with your proposal.  Temporarily wasting some disk
space is a lot more palatable than corrupting data, and these code
paths are necessarily not terribly well tested.  So +1 for more
insurance.

			regards, tom lane