Date: Thu, 15 May 2014 22:16:51 +0200
From: Andres Freund <andres@2ndquadrant.com>
To: Bruce Momjian <bruce@momjian.us>
Cc: Robert Haas <robertmhaas@gmail.com>,
 Heikki Linnakangas <hlinnakangas@vmware.com>,
 Rajeev rastogi <rajeev.rastogi@huawei.com>, Ants Aasma <ants@cybertec.at>,
 PostgreSQL-development <pgsql-hackers@postgresql.org>,
 Markus Wanner <markus@bluegap.ch>
Subject: Re: Proposal for CSN based snapshots
Message-ID: <20140515201651.GD16014@awork2.anarazel.de>
References: 
 <CA+CSw_uSzXidemci-nU5fJY-qXD0DE0ZWz-j1ASTT_ffBjXo9Q@mail.gmail.com>
 <51B6C97B.1070000@bluegap.ch>
 <BF2827DCCE55594C8D7A8F7FFD3AB7713DDBC758@SZXEML508-MBX.china.huawei.com>
 <5370D323.1070606@vmware.com>
 <20140512144112.GB9619@awork2.anarazel.de>
 <5370E267.2020904@vmware.com> <20140515183436.GD25053@momjian.us>
 <CA+TgmoYiqga4KWz_1=O8D6ivmTAJfY=-fxO6Nk+D-o3M0M60=Q@mail.gmail.com>
 <20140515200632.GB16014@awork2.anarazel.de>
 <20140515201349.GK25053@momjian.us>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20140515201349.GK25053@momjian.us>
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org

On 2014-05-15 16:13:49 -0400, Bruce Momjian wrote:
> On Thu, May 15, 2014 at 10:06:32PM +0200, Andres Freund wrote:
> > > If the larger clog size is a show-stopper (and I'm not sure I have an
> > > intelligent opinion on that just yet), one way to get around the
> > > problem would be to summarize CLOG entries after-the-fact.  Once an
> > > XID precedes the xmin of every snapshot, we don't need to know the
> > > commit LSN any more.  So we could read the old pg_clog files and write
> > > new summary files.  Since we don't need to care about subcommitted
> > > transactions either, we could get by with just 1 bit per transaction,
> > > 1 = committed, 0 = aborted.  Once we've written and fsync'd the
> > > summary files, we could throw away the original files.  That might
> > > leave us with a smaller pg_clog than what we have today.
> > 
> > I think the easiest way for now would be to have pg_clog with the same
> > format as today and a rangewise much smaller pg_csn storing the lsns
> > that are needed. That'll leave us with pg_upgrade'ability without
> > needing to rewrite pg_clog during the upgrade.
> 
> Yes, I like the idea of storing the CSN separately.  One reason the
> 2-bit clog is so good is that we know we have atomic 1-byte writes on
> all platforms.

I don't think we rely on that anywhere. And in fact we don't have the
ability to do so for arbitrary bytes - lots of platforms can do that
only on specifically aligned bytes.

We rely on being able to atomically (as in either before/after no torn
value) write/read TransactionIds, but that's it I think?

>  Can we assume atomic 64-bit writes?

Not on 32bit platforms.

Greetings,

Andres Freund

-- 
 Andres Freund	                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers