Subject: Re: prion failed with ERROR: missing chunk number 0 for toast value
 14334 in
 pg_toast_2619
To: Andres Freund <andres@anarazel.de>, Tom Lane <tgl@sss.pgh.pa.us>
CC: Peter Geoghegan <pg@bowt.ie>, Heikki Linnakangas <hlinnaka@iki.fi>,
	pgsql-hackers <pgsql-hackers@postgresql.org>, Justin Pryzby
	<pryzby@telsasoft.com>
References: <f03ea04a-9b77-e371-9ab9-182cb35db1f9@iki.fi>
 <2472976.1621117273@sss.pgh.pa.us> <2591376.1621196582@sss.pgh.pa.us>
 <CAH2-Wz=Pdfhd0Pc3g6odxkgBP9P6nsp1uaiR0JYRac9fYyYGrQ@mail.gmail.com>
 <2595981.1621203681@sss.pgh.pa.us>
 <20210516223513.outnhdfibafroqc4@alap3.anarazel.de>
 <2601445.1621204973@sss.pgh.pa.us>
 <20210516232833.qfrjcb7jgbzzpo5h@alap3.anarazel.de>
From: "Drouvot, Bertrand" <bdrouvot@amazon.com>
Message-ID: <7d8045e5-c23f-4090-3553-6092804441e3@amazon.com>
Date: Mon, 17 May 2021 20:14:40 +0200
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.10.1
MIME-Version: 1.0
In-Reply-To: <20210516232833.qfrjcb7jgbzzpo5h@alap3.anarazel.de>
Content-Type: multipart/alternative;
	boundary="------------88C5175CBBAF6094E6D70238"
Content-Language: en-US
Archived-At: 
 <https://www.postgresql.org/message-id/7d8045e5-c23f-4090-3553-6092804441e3%40amazon.com>
Precedence: bulk

--------------88C5175CBBAF6094E6D70238
Content-Type: text/plain; charset="utf-8"; format=flowed
Content-Transfer-Encoding: 8bit

Hi,

On 5/17/21 1:28 AM, Andres Freund wrote:
> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
>
>
>
> Hi,
>
> On 2021-05-16 18:42:53 -0400, Tom Lane wrote:
>> Andres Freund <andres@anarazel.de> writes:
>>> Why would it not be safe?
>> I'm just wondering about the catalog tuples set up by pg_upgrade
>> itself.  If they're all frozen then they probably don't matter to
>> this, but it might take some thought.
> There shouldn't be any catalog objects (vs tuples) set up by pg_upgrade
> at the time of the resetwal, as far as I can see. copy_xact_xlog_xid(),
> which includes the resetwal calls, is done before any new objects are
> created/restored.
>
> The only thing that happens before copy_xact_xlog_xid() is
> prepare_new_cluster(), which analyzes/freezes the catalog of the new
> cluster. Of course that does create new stats tuples for catalog tables,
> but if the freezing of those doesn't work, we'd be in deep trouble
> regardless of which concrete oldestXid value we choose - that happens
> with xids as they are in a freshly initdb' cluster, which might be in
> the future in the old cluster, or might have aborted. Their pg_xact will
> be overwritten in copy_xact_xlog_xid().


FWIW a patch proposal to copy the oldest unfrozen XID during pg_upgrade 
(it adds a new (- u) parameter to pg_resetwal) has been submitted a 
couple of weeks ago, see: https://commitfest.postgresql.org/33/3105/

I was also wondering if:

  * We should keep the old behavior in case pg_resetwal -x is being used
    without -u?  (The proposed patch does not set an arbitrary oldestXID
    anymore in  case -x is used)
  * We should ensure that the xid provided with -x or -u is
     >= FirstNormalTransactionId (Currently the only check is that it is
    # 0)?

Thanks

Bertrand


--------------88C5175CBBAF6094E6D70238
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: 8bit

<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  </head>
  <body>
    <p>Hi,<br>
    </p>
    <div class="moz-cite-prefix">On 5/17/21 1:28 AM, Andres Freund
      wrote:<br>
    </div>
    <blockquote type="cite"
      cite="mid:20210516232833.qfrjcb7jgbzzpo5h@alap3.anarazel.de">
      <pre class="moz-quote-pre" wrap="">CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.


Hi,

On 2021-05-16 18:42:53 -0400, Tom Lane wrote:
</pre>
      <blockquote type="cite">
        <pre class="moz-quote-pre" wrap="">Andres Freund <a class="moz-txt-link-rfc2396E" href="mailto:andres@anarazel.de">&lt;andres@anarazel.de&gt;</a> writes:
</pre>
        <blockquote type="cite">
          <pre class="moz-quote-pre" wrap="">Why would it not be safe?
</pre>
        </blockquote>
        <pre class="moz-quote-pre" wrap="">
I'm just wondering about the catalog tuples set up by pg_upgrade
itself.  If they're all frozen then they probably don't matter to
this, but it might take some thought.
</pre>
      </blockquote>
      <pre class="moz-quote-pre" wrap="">
There shouldn't be any catalog objects (vs tuples) set up by pg_upgrade
at the time of the resetwal, as far as I can see. copy_xact_xlog_xid(),
which includes the resetwal calls, is done before any new objects are
created/restored.

The only thing that happens before copy_xact_xlog_xid() is
prepare_new_cluster(), which analyzes/freezes the catalog of the new
cluster. Of course that does create new stats tuples for catalog tables,
but if the freezing of those doesn't work, we'd be in deep trouble
regardless of which concrete oldestXid value we choose - that happens
with xids as they are in a freshly initdb' cluster, which might be in
the future in the old cluster, or might have aborted. Their pg_xact will
be overwritten in copy_xact_xlog_xid().
</pre>
    </blockquote>
    <p><br>
      FWIW a patch proposal to copy the oldest unfrozen XID during
      pg_upgrade (it adds a new (- u) parameter to pg_resetwal) has been
      submitted a couple of weeks ago, see:
      <a class="moz-txt-link-freetext" href="https://commitfest.postgresql.org/33/3105/">https://commitfest.postgresql.org/33/3105/</a></p>
    <p>I was also wondering if: <br>
    </p>
    <ul>
      <li>We should keep the old behavior in case pg_resetwal -x is
        being used without -u?  (The proposed patch does not set an
        arbitrary oldestXID anymore in  case -x is used)</li>
      <li>We should ensure that the xid provided with -x or -u is
        &gt;= FirstNormalTransactionId (Currently the only check is that
        it is # 0)?</li>
    </ul>
    <p>Thanks</p>
    <p>Bertrand</p>
  </body>
</html>

--------------88C5175CBBAF6094E6D70238--