Message-ID: <4FE2279C.2070506@enterprisedb.com>
Date: Wed, 20 Jun 2012 22:42:20 +0300
From: Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:10.0.4) Gecko/20120510 Icedove/10.0.4
MIME-Version: 1.0
To: Fujii Masao <masao.fujii@gmail.com>
CC: Magnus Hagander <magnus@hagander.net>,
	Robert Haas <robertmhaas@gmail.com>,
	Andres Freund <andres@2ndquadrant.com>, pgsql-hackers@postgresql.org
Subject: Re: WAL format changes
References: <4FDA5136.6080206@enterprisedb.com>
	<201206142358.12431.andres@2ndquadrant.com>
	<CA+TgmobG9VWWvLXeK0XOxoWXtza8JTFgSu2eoAtVFucxH6gLBw@mail.gmail.com>
	<4FDF6E8E.4080900@enterprisedb.com>
	<4FE034D0.5070900@enterprisedb.com>
	<CA+TgmoZu=UMKLJDQ5FNRqPnomiGOSdEvzaj_Gg9CJzcPnvuxgQ@mail.gmail.com>
	<CABUevExOgqc+XqsOQs_B-TT8w4iC1iPGofz2EPejUKkGJpPF4w@mail.gmail.com>
	<CAHGQGwFa0mDNrVqwdq1ObVtb+aA6Tq=0m6QTynr6qtO2GSDtSQ@mail.gmail.com>
In-Reply-To: 
 <CAHGQGwFa0mDNrVqwdq1ObVtb+aA6Tq=0m6QTynr6qtO2GSDtSQ@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

On 20.06.2012 20:43, Fujii Masao wrote:
> On Wed, Jun 20, 2012 at 8:19 PM, Magnus Hagander<magnus@hagander.net>  wrote:
>> On Tue, Jun 19, 2012 at 5:57 PM, Robert Haas<robertmhaas@gmail.com>  wrote:
>>> On Tue, Jun 19, 2012 at 4:14 AM, Heikki Linnakangas
>>> <heikki.linnakangas@enterprisedb.com>  wrote:
>>>> Well, that was easier than I thought. Attached is a patch to make XLogRecPtr
>>>> a uint64, on top of my other WAL format patches. I think we should go ahead
>>>> with this.
>>>
>>> +1.
>>>
>>>> The LSNs on pages are still stored in the old format, to avoid changing the
>>>> on-disk format and breaking pg_upgrade. The XLogRecPtrs stored the control
>>>> file and WAL are changed, however, so an initdb (or at least pg_resetxlog)
>>>> is required.
>>>
>>> Seems fine.
>>>
>>>> Should we keep the old representation in the replication protocol messages?
>>>> That would make it simpler to write a client that works with different
>>>> server versions (like pg_receivexlog). Or, while we're at it, perhaps we
>>>> should mandate network-byte order for all the integer and XLogRecPtr fields
>>>> in the replication protocol. That would make it easier to write a client
>>>> that works across different architectures, in>= 9.3. The contents of the
>>>> WAL would of course be architecture-dependent, but it would be nice if
>>>> pg_receivexlog and similar tools could nevertheless be
>>>> architecture-independent.
>>>
>>> I share Andres' question about how we're doing this already.  I think
>>> if we're going to break this, I'd rather do it in 9.3 than 5 years
>>> from now.  At this point it's just a minor annoyance, but it'll
>>> probably get worse as people write more tools that understand WAL.
>>
>> If we are looking at breaking it, and we are especially concerned
>> about something like pg_receivexlog... Is it something we could/should
>> change in the protocl *now* for 9.2, to make it non-broken in any
>> released version? As in, can we extract just the protocol change and
>> backpatch that to 9.2beta?
>
> pg_receivexlog in 9.2 cannot handle correctly the WAL location "FF"
> (which was skipped in 9.2 or before). For example, pg_receivexlog calls
> XLByteAdvance() which always skips "FF". So even if we change the protocol,
> ISTM pg_receivexlog in 9.2 cannot work well with the server in 9.3 which
> might send "FF". No?

Yeah, you can't use pg_receivexlog from 9.2 against a 9.3 server. We 
can't really promise compatibility when using an older client against a 
newer server, but we can try to be backwards-compatible in the other 
direction. I'm thinking of using a 9.3 pg_receivexlog against a 9.2 server.

But I guess Robert is right and we shouldn't worry about 
backwards-compatibility at this point. Instead, let's try to get the 
protocol right, so that we can more easily provide 
backwards-compatibility in the future. Like, using a 9.4 pg_receivexlog 
against a 9.3 server.

-- 
   Heikki Linnakangas
   EnterpriseDB   http://www.enterprisedb.com