From: Alvaro Herrera <alvherre@2ndquadrant.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Serge Negodyuck <petr@petrovich.kiev.ua>
Cc: pgsql-bugs@postgresql.org
Subject: Re: BUG #8673: Could not open file "pg_multixact/members/xxxx" on slave during hot_standby
Date: Fri, 6 Jun 2014 14:08:26 -0400
Message-ID: <20140606180826.GW5146@eldon.alvh.no-ip.org> (raw)
In-Reply-To: <6687.1402074269@sss.pgh.pa.us>
References: <20131209142525.GA1123@awork2.anarazel.de>
<CABKyZDFWn+ty_m88sq2j5UThowb2Qwd0125BNx_19va2Fn57dQ@mail.gmail.com>
<20131209150345.GB1123@awork2.anarazel.de>
<CABKyZDFRtDzcY1=sEznC2ivmcM5p1yPQvC8qLL8XLviNF2kp7A@mail.gmail.com>
<20131209182701.GD9519@awork2.anarazel.de>
<20140103014616.GB7035@eldon.alvh.no-ip.org>
<CABKyZDEV49nUYJej9Hv-hPUS=VUmMt1LmVV=YW=fV5X8nKtdtw@mail.gmail.com>
<20140606013943.GT5146@eldon.alvh.no-ip.org>
<20140606164727.GU5146@eldon.alvh.no-ip.org>
<6687.1402074269@sss.pgh.pa.us>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-bugs>
Tom Lane wrote:
> Alvaro Herrera <alvherre@2ndquadrant.com> writes:
> > After some simplification I think it should be clearer. Thanks Andres
> > for commenting offlist.
>
> I find this not only unreadable but a bit scary:
>
> ! if (offset + MAX_MEMBERS_IN_LAST_MEMBERS_PAGE < offset)
>
> I'm worried whether some compilers might not decide that that's a
> can't-happen situation, and optimize the test away entirely. I think
> Andres' proposal is probably safer (though it needs comments).
Well, somebody would have to work it out fully and test it -- I can't
spend more time on this issue. If somebody is interested in doing that,
I used this little program and make sure that it "zeroes" all pages, for
various block sizes, initial offsets and "step sizes" (#members in each
multi) -- I was able to reproduce Serge's issues with the original
coding, and then watch issues fly by as I tried several approaches until
I settled on the patch as submitted.
--
Álvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugsAttachments:multixact.c (text/x-csrc, 3.4K) download
#include <stdio.h>
#include <stdlib.h>
#define BLCKSZ (8192 * 1)
typedef char bool;
typedef unsigned int uint32;
typedef uint32 TransactionId;
typedef uint32 MultiXactOffset;
typedef unsigned long int uint64;
#define MaxMultiXactOffset ((MultiXactOffset) 0xFFFFFFFF)
#define Min(x, y) ((x) < (y) ? (x) : (y))
#define true ((bool) 1)
#define false ((bool) 0)
#define SLRU_PAGES_PER_SEGMENT 32
#define MXACT_MEMBER_BITS_PER_XACT 8
#define MXACT_MEMBER_FLAGS_PER_BYTE 1
#define MXACT_MEMBER_XACT_BITMASK ((1 << MXACT_MEMBER_BITS_PER_XACT) - 1)
/* how many full bytes of flags are there in a group? */
#define MULTIXACT_FLAGBYTES_PER_GROUP 4
#define MULTIXACT_MEMBERS_PER_MEMBERGROUP \
(MULTIXACT_FLAGBYTES_PER_GROUP * MXACT_MEMBER_FLAGS_PER_BYTE)
/* size in bytes of a complete group */
#define MULTIXACT_MEMBERGROUP_SIZE \
(sizeof(TransactionId) * MULTIXACT_MEMBERS_PER_MEMBERGROUP + MULTIXACT_FLAGBYTES_PER_GROUP)
#define MULTIXACT_MEMBERGROUPS_PER_PAGE (BLCKSZ / MULTIXACT_MEMBERGROUP_SIZE)
#define MULTIXACT_MEMBERS_PER_PAGE \
(MULTIXACT_MEMBERGROUPS_PER_PAGE * MULTIXACT_MEMBERS_PER_MEMBERGROUP)
/*
* Because the number of items per page is not a divisor of the last item
* number (member 0xFFFFFFFF), the last segment does not use the maximum number
* of pages, and moreover the last used page therein does not use the same
* number of items as previous pages. (Another way to say it is that the
* 0xFFFFFFFF member is somewhere in the middle of the last page, so the page
* has some empty space after that item.)
*
* This constant is the number of members in the last page of the last segment.
*/
#define MAX_MEMBERS_IN_LAST_MEMBERS_PAGE \
((uint32) ((0xFFFFFFFF % MULTIXACT_MEMBERS_PER_PAGE) + 1))
/* page in which a member is to be found */
#define MXOffsetToMemberPage(xid) ((xid) / (TransactionId) MULTIXACT_MEMBERS_PER_PAGE)
#define MXOffsetToFlagsOffset(xid) \
((((xid) / (TransactionId) MULTIXACT_MEMBERS_PER_MEMBERGROUP) % \
(TransactionId) MULTIXACT_MEMBERGROUPS_PER_PAGE) * \
(TransactionId) MULTIXACT_MEMBERGROUP_SIZE)
#define MXOffsetToFlagsBitShift(xid) \
(((xid) % (TransactionId) MULTIXACT_MEMBERS_PER_MEMBERGROUP) * \
MXACT_MEMBER_BITS_PER_XACT)
static void
ZeroMultiXactMemberPage(int pageno, bool writeXlog)
{
printf("zeroing page %u: page %u in segment %05X\n", pageno,
pageno % SLRU_PAGES_PER_SEGMENT,
pageno / SLRU_PAGES_PER_SEGMENT);
}
static void
ExtendMultiXactMember(MultiXactOffset offset, int nmembers)
{
while (nmembers > 0)
{
int flagsoff;
int flagsbit;
uint64 diff64;
uint32 difference;
flagsoff = MXOffsetToFlagsOffset(offset);
flagsbit = MXOffsetToFlagsBitShift(offset);
if (flagsoff == 0 && flagsbit == 0)
{
int pageno;
pageno = MXOffsetToMemberPage(offset);
ZeroMultiXactMemberPage(pageno, true);
}
if (offset + MAX_MEMBERS_IN_LAST_MEMBERS_PAGE < offset)
{
difference = MaxMultiXactOffset - offset + 1;
}
else
difference = MULTIXACT_MEMBERS_PER_PAGE - offset % MULTIXACT_MEMBERS_PER_PAGE;
nmembers -= difference;
offset += difference;
}
}
int
main(int argc, char **argv)
{
MultiXactOffset offset;
int stepsize;
if (argc != 3)
{
fprintf(stderr, "arguments: initial_offset step_size\n");
exit(0);
}
offset = strtoul(argv[1], NULL, 10);
printf("initial offset: %u\n", offset);
stepsize = strtoul(argv[2], NULL, 10);
printf("step size: %d\n", stepsize);
for (; ; offset += stepsize)
ExtendMultiXactMember(offset, stepsize);
return 0;
}
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: alvherre@2ndquadrant.com, tgl@sss.pgh.pa.us, petr@petrovich.kiev.ua, pgsql-bugs@postgresql.org
Subject: Re: BUG #8673: Could not open file "pg_multixact/members/xxxx" on slave during hot_standby
In-Reply-To: <20140606180826.GW5146@eldon.alvh.no-ip.org>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox