From: "Wong, Yi Wen" <yiwong@amazon.com>
To: Alvaro Herrera <alvherre@alvh.no-ip.org>,
        Michael Paquier
	<michael.paquier@gmail.com>
CC: Peter Geoghegan <pg@bowt.ie>, "Wood, Dan" <hexpert@amazon.com>,
        "pgsql-bugs@postgresql.org" <pgsql-bugs@postgresql.org>
Subject: Re: Old row version in hot chain become visible after a freeze
Thread-Topic: [BUGS] Old row version in hot chain become visible after a
 freeze
Thread-Index: AQHTJ10zEH9T5bMCDUufad1KxDA7S6Kp0agK
Date: Thu, 7 Sep 2017 20:02:34 +0000
Message-ID: <1504814554687.86636@amazon.com>
References: 
 <CAB7nPqTf70_QF7JnJZ=trZLamrYrm3soHbMUFLFQduDo39VBZQ@mail.gmail.com>,<20170906221217.fnmoipi5zi5w3yrl@alvherre.pgsql>
In-Reply-To: <20170906221217.fnmoipi5zi5w3yrl@alvherre.pgsql>
Accept-Language: en-US
Content-Language: en-US
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Precedence: Bulk
Precedence: bulk
Sender: pgsql-bugs-owner@postgresql.org


________________________________________
From: pgsql-bugs-owner@postgresql.org <pgsql-bugs-owner@postgresql.org> on =
behalf of Alvaro Herrera <alvherre@alvh.no-ip.org>
Sent: Wednesday, September 6, 2017 3:12 PM
To: Michael Paquier
Cc: Peter Geoghegan; Wood, Dan; pgsql-bugs@postgresql.org
Subject: Re: [BUGS] Old row version in hot chain become visible after a fre=
eze

>Michael Paquier wrote:

>> frame #4: 0x00000001098fba6b postgres`FreezeMultiXactId(multi=3D34,
>> t_infomask=3D4930, cutoff_xid=3D897, cutoff_multi=3D30,
>> flags=3D0x00007fff56372fae) + 1179 at heapam.c:6532
>>    6529                 * Since the tuple wasn't marked HEAPTUPLE_DEAD b=
y vacuum, the
>>    6530                 * update Xid cannot possibly be older than the x=
id cutoff.
>>    6531                 */
>> -> 6532                Assert(!TransactionIdIsValid(update_xid) ||
>>    6533                       !TransactionIdPrecedes(update_xid, cutoff_=
xid));
>>    6534
>>    6535                /*
>> (lldb) p update_xid
>> (TransactionId) $0 =3D 896
>> (lldb) p cutoff_xid
>> (TransactionId) $1 =3D 897

>So, looking at this closely, I think there is a bigger problem here: if
>we use any of the proposed patches or approaches, we risk leaving an old
>Xid in a tuple (because of skipping the heap_tuple_prepare_freeze on a
>tuple which remains in the heap with live Xids), followed by later
>truncating pg_multixact / pg_clog removing a segment that might be
>critical to resolving this tuple status later on.

>I think doing the tuple freezing dance for any tuple we don't remove
>from the heap is critical, not optional.  Maybe a later HOT pruning
>would save you from actual disaster, but I think it'd be a bad idea to
>rely on that.

There is actually another case where HEAPTUPLE_DEAD tuples may be kept and =
have
prepare_freeze skipped on them entirely.

lazy_record_dead_tuple may fail to record the heap for later pruning
for lazy_vacuum_heap if there is already a sufficiently large number of dea=
d tuples
in the array:

/*
 * The array shouldn't overflow under normal behavior, but perhaps it
 * could if we are given a really small maintenance_work_mem. In that
 * case, just forget the last few tuples (we'll get 'em next time).
 */
if (vacrelstats->num_dead_tuples < vacrelstats->max_dead_tuples)
...

It looks like we don't even check if the lazy_record_dead_tuple operation a=
ctually did any
actual recording in the tupgone case...

 if (tupgone)
 {
    lazy_record_dead_tuple(vacrelstats, &(tuple.t_self));
    HeapTupleHeaderAdvanceLatestRemovedXid(tuple.t_data,
                                                                           =
              &vacrelstats->latestRemovedXid);
    tups_vacuumed +=3D 1;
    has_dead_tuples =3D true;
}

It's probably unsafe for any operations calling TruncateXXX code to assume =
that old Xids don't stick around
after a vacuum in itself.

Thanks,
Yi Wen

--=20
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs