From: Andres Freund <andres@2ndquadrant.com>
Organization: 2ndquadrant.com
To: Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>
Subject: Re: WAL format changes
Date: Wed, 20 Jun 2012 00:30:38 +0200
User-Agent: KMail/1.13.7 (Linux/3.4.0-andres; KDE/4.8.3; x86_64; ; )
Cc: pgsql-hackers@postgresql.org,
 Robert Haas <robertmhaas@gmail.com>
References: <4FDA5136.6080206@enterprisedb.com>
	<201206191746.41027.andres@2ndquadrant.com>
	<4FE0FC36.9080409@enterprisedb.com>
In-Reply-To: <4FE0FC36.9080409@enterprisedb.com>
MIME-Version: 1.0
Content-Type: Text/Plain;
  charset="iso-8859-15"
Content-Transfer-Encoding: 7bit
Message-Id: <201206200030.38683.andres@2ndquadrant.com>

Hi,

On Wednesday, June 20, 2012 12:24:54 AM Heikki Linnakangas wrote:
> On 19.06.2012 18:46, Andres Freund wrote:
> > On Tuesday, June 19, 2012 10:14:08 AM Heikki Linnakangas wrote:
> >> Well, that was easier than I thought. Attached is a patch to make
> >> XLogRecPtr a uint64, on top of my other WAL format patches. I think we
> >> should go ahead with this.
> > 
> > Cool. You plan to merge XLogSegNo with XLogRecPtr in that case? I am not
> > sure if having two representations which just have a constant factor
> > inbetween really makes sense.

> I wasn't planning to, it didn't even occur to me that we might be able
> to get rid of XLogSegNo to be honest. There's places that deal whole
> segments, rather than with specific byte positions in the WAL, so I
> think XLogSegNo makes more sense in that context. Take
> XLogArchiveNotifySeg(), for example. It notifies the archiver that a
> given segment is ready for archiving, so we pass an XLogSegNo to
> identify that segment as an argument. I suppose we could pass an
> XLogRecPtr that points to the beginning of the segment instead, but it
> doesn't really feel like an improvement to me.
I am not sure its a win either, was just wondering because they now are that 
similar.

> >> Should we keep the old representation in the replication protocol
> >> messages? That would make it simpler to write a client that works with
> >> different server versions (like pg_receivexlog). Or, while we're at it,
> >> perhaps we should mandate network-byte order for all the integer and
> >> XLogRecPtr fields in the replication protocol.
> > 
> > The replication protocol uses pq_sendint for integers which should take
> > care of converting to big endian already. I don't think anything but the
> > wal itself is otherwise transported in a binary fashion? So I don't
> > think there is any such architecture dependency in the protocol
> > currently?
> We only use pg_sendint() for the few values exchanged in the handshake
> before we start replicating, but once we begin, we just send structs
> 
> around. For example, in ProcessStandbyReplyMessage():
> > static void
> > ProcessStandbyReplyMessage(void)
> > {
> > 
> > 	StandbyReplyMessage reply;
> > 	
> > 	pq_copymsgbytes(&reply_message, (char *) &reply,
> > 	sizeof(StandbyReplyMessage));
> > 	
>  > ...
> 
> After that, we just the fields in the reply struct like in any other
> struct.
Yes, forgot that, true. I guess the best fix would be to actually send normal 
messages instead of CopyData ones? Much more to type though...

Andres
-- 
 Andres Freund	                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services