Date: Fri, 6 Jun 2014 12:47:27 -0400
From: Alvaro Herrera <alvherre@2ndquadrant.com>
To: Serge Negodyuck <petr@petrovich.kiev.ua>
Cc: pgsql-bugs@postgresql.org
Subject: Re: BUG #8673: Could not open file
 "pg_multixact/members/xxxx" on slave during hot_standby
Message-ID: <20140606164727.GU5146@eldon.alvh.no-ip.org>
References: <E1Vq1BO-00064N-Qn@wrigleys.postgresql.org>
 <20131209142525.GA1123@awork2.anarazel.de>
 <CABKyZDFWn+ty_m88sq2j5UThowb2Qwd0125BNx_19va2Fn57dQ@mail.gmail.com>
 <20131209150345.GB1123@awork2.anarazel.de>
 <CABKyZDFRtDzcY1=sEznC2ivmcM5p1yPQvC8qLL8XLviNF2kp7A@mail.gmail.com>
 <20131209182701.GD9519@awork2.anarazel.de>
 <20140103014616.GB7035@eldon.alvh.no-ip.org>
 <CABKyZDEV49nUYJej9Hv-hPUS=VUmMt1LmVV=YW=fV5X8nKtdtw@mail.gmail.com>
 <20140606013943.GT5146@eldon.alvh.no-ip.org>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="9Ek0hoCL9XbhcSqy"
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20140606013943.GT5146@eldon.alvh.no-ip.org>
User-Agent: Mutt/1.5.21 (2010-09-15)
Precedence: bulk
Sender: pgsql-bugs-owner@postgresql.org


--9Ek0hoCL9XbhcSqy
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit

Alvaro Herrera wrote:
> Serge Negodyuck wrote:
> 
> > 2014-06-02 08:20:55 EEST 172.18.10.4 db PANIC: could not access status of
> > transaction 2080547
> > 2014-06-02 08:20:55 EEST 172.18.10.4 db DETAIL: Could not open file
> > "pg_multixact/members/14078": No such file or directory.
> > 2014-06-02 08:20:55 EEST 172.18.10.4 db CONTEXT: SQL statement "  UPDATE
> > ....."
> 
> So as it turns out, this was caused because the arithmetic to handle the
> wraparound case neglected to handle multixacts with more members than
> the number that fit in the last page(s) of the last segment, leading to
> a number of pages in the 14078 segment (or whatever the last segment is
> for a given BLCKSZ) to fail to be initialized.  This patch is a rework
> of that arithmetic, although it seems little bit too obscure.

After some simplification I think it should be clearer.  Thanks Andres
for commenting offlist.

There is a different way to compute the "difference" proposed by Andres,
without using if/else; the idea is to cast the values to int64 and then
clamp.  It would be something like

	uint64	diff64;

	diff64 = Min((uint64) offset + nmembers,
			(uint64) offset + MULTIXACT_MEMBERS_PER_PAGE);
	difference = (uint32) Min(diff64, MaxMultiXactOffset);

(There are other ways to formulate this, of course, but this seems to me
to be the most readable one).  I am undecided between this and the one I
propose in the patch, so I've stuck with the patch.

-- 
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--9Ek0hoCL9XbhcSqy
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
 filename="multixact-offset-wraparound-2.patch"

*** a/src/backend/access/transam/multixact.c
--- b/src/backend/access/transam/multixact.c
***************
*** 133,138 ****
--- 133,151 ----
  #define MULTIXACT_MEMBERS_PER_PAGE	\
  	(MULTIXACT_MEMBERGROUPS_PER_PAGE * MULTIXACT_MEMBERS_PER_MEMBERGROUP)
  
+ /*
+  * Because the number of items per page is not a divisor of the last item
+  * number (member 0xFFFFFFFF), the last segment does not use the maximum number
+  * of pages, and moreover the last used page therein does not use the same
+  * number of items as previous pages.  (Another way to say it is that the
+  * 0xFFFFFFFF member is somewhere in the middle of the last page, so the page
+  * has some empty space after that item.)
+  *
+  * This constant is the number of members in the last page of the last segment.
+  */
+ #define MAX_MEMBERS_IN_LAST_MEMBERS_PAGE \
+ 		((uint32) ((0xFFFFFFFF % MULTIXACT_MEMBERS_PER_PAGE) + 1))
+ 
  /* page in which a member is to be found */
  #define MXOffsetToMemberPage(xid) ((xid) / (TransactionId) MULTIXACT_MEMBERS_PER_PAGE)
  
***************
*** 2278,2283 **** ExtendMultiXactMember(MultiXactOffset offset, int nmembers)
--- 2291,2297 ----
  	{
  		int			flagsoff;
  		int			flagsbit;
+ 		uint32		difference;
  
  		/*
  		 * Only zero when at first entry of a page.
***************
*** 2302,2322 **** ExtendMultiXactMember(MultiXactOffset offset, int nmembers)
  		 * Advance to next page, taking care to properly handle the wraparound
  		 * case.  OK if nmembers goes negative.
  		 */
- 		if ((unsigned int) (offset + nmembers) < offset)
- 		{
- 			uint32		difference = offset + MULTIXACT_MEMBERS_PER_PAGE;
  
! 			nmembers -= (unsigned int) (MULTIXACT_MEMBERS_PER_PAGE - difference);
! 			offset = 0;
  		}
  		else
- 		{
- 			int			difference;
- 
  			difference = MULTIXACT_MEMBERS_PER_PAGE - offset % MULTIXACT_MEMBERS_PER_PAGE;
! 			nmembers -= difference;
! 			offset += difference;
! 		}
  	}
  }
  
--- 2316,2335 ----
  		 * Advance to next page, taking care to properly handle the wraparound
  		 * case.  OK if nmembers goes negative.
  		 */
  
! 		if (offset + MAX_MEMBERS_IN_LAST_MEMBERS_PAGE < offset)
! 		{
! 			/*
! 			 * This is the last page of the last segment; we can compute the
! 			 * number of items in it without modulo arithmetic.
! 			 */
! 			difference = MaxMultiXactOffset - offset + 1;
  		}
  		else
  			difference = MULTIXACT_MEMBERS_PER_PAGE - offset % MULTIXACT_MEMBERS_PER_PAGE;
! 
! 		nmembers -= difference;
! 		offset += difference;
  	}
  }
  
*** a/src/include/access/multixact.h
--- b/src/include/access/multixact.h
***************
*** 25,30 ****
--- 25,32 ----
  
  #define MultiXactIdIsValid(multi) ((multi) != InvalidMultiXactId)
  
+ #define MaxMultiXactOffset	((MultiXactOffset) 0xFFFFFFFF)
+ 
  /* Number of SLRU buffers to use for multixact */
  #define NUM_MXACTOFFSET_BUFFERS		8
  #define NUM_MXACTMEMBER_BUFFERS		16

--9Ek0hoCL9XbhcSqy
Content-Type: text/plain
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0


-- 
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

--9Ek0hoCL9XbhcSqy--