pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  

help / color / mirror / Atom feed
From: Heikki Linnakangas <hlinnakangas@vmware.com>
To: Greg Stark <stark@mit.edu>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Andres Freund <andres@2ndquadrant.com>
Cc: Alvaro Herrera <alvherre@2ndquadrant.com>
Cc: Bruce Momjian <bruce@momjian.us>
Cc: Rajeev rastogi <rajeev.rastogi@huawei.com>
Cc: Ants Aasma <ants@cybertec.at>
Cc: PostgreSQL-development <pgsql-hackers@postgresql.org>
Cc: Markus Wanner <markus@bluegap.ch>
Subject: Re: Proposal for CSN based snapshots
Date: Fri, 13 Jun 2014 13:24:19 +0300
Message-ID: <539AD153.9000004@vmware.com> (raw)
In-Reply-To: <CAM-w4HMG-YeTfWc3r_excej3Kd04q4zDDXR0hxShVbij211ABA@mail.gmail.com>
References: <51B6C97B.1070000@bluegap.ch>
	<BF2827DCCE55594C8D7A8F7FFD3AB7713DDBC758@SZXEML508-MBX.china.huawei.com>
	<5370D323.1070606@vmware.com>
	<20140512144112.GB9619@awork2.anarazel.de>
	<5370E267.2020904@vmware.com>
	<20140515183436.GD25053@momjian.us>
	<CA+TgmoYiqga4KWz_1=O8D6ivmTAJfY=-fxO6Nk+D-o3M0M60=Q@mail.gmail.com>
	<20140515200632.GB16014@awork2.anarazel.de>
	<20140515213714.GB7857@eldon.alvh.no-ip.org>
	<53889CCB.7030005@vmware.com>
	<20140530152721.GC30516@awork2.anarazel.de>
	<5388D02D.6040305@vmware.com>
	<CA+TgmoZ4cP_1Pavwn+VX74_ksDhHf-whO5FFR1j+NdZqONu8MA@mail.gmail.com>
	<CAM-w4HMG-YeTfWc3r_excej3Kd04q4zDDXR0hxShVbij211ABA@mail.gmail.com>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-hackers>

On 06/03/2014 05:53 PM, Greg Stark wrote:
> On Tue, Jun 3, 2014 at 2:55 PM, Robert Haas <robertmhaas@gmail.com> wrote:
>> That doesn't address Bruce's concern about CLOG disk consumption.
>
> Well we would only need the xid->lsn mapping for transactions since
> globalxmin. Anything older we would just need the committed bit. So we
> could maintain two structures, one like our current clog going back
> until the freeze_max_age and one with 32-bits (or 64 bits?) per xid
> but only going back as far as the globalxmin. There are a myriad of
> compression techniques we could use on a sequence of mostly similar
> mostly increasing numbers in a small range too. But I suspect they
> wouldn't really be necessary.

Yeah, that seems like a better design, after all.

Attached is a new patch. It now keeps the current pg_clog unchanged, but 
adds a new pg_csnlog besides it. pg_csnlog is more similar to 
pg_subtrans than pg_clog: it's not WAL-logged, is reset at startup, and 
segments older than GlobalXmin can be truncated.

This addresses the disk space consumption, and simplifies pg_upgrade.

There are no other significant changes in this new version, so it's 
still very much WIP. But please take a look!

- Heikki

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Attachments:

  csn-2.patch.gz (application/gzip, 107.3K) download

view thread (113+ messages)  latest in thread

Message-ID: <539AD153.9000004@vmware.com>
Permalink:  ../539AD153.9000004@vmware.com/
Also on:    postgresql.org/message-id/539AD153.9000004@vmware.com

 · 

reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-hackers@postgresql.org
  Cc: hlinnakangas@vmware.com, stark@mit.edu, robertmhaas@gmail.com, andres@2ndquadrant.com, alvherre@2ndquadrant.com, bruce@momjian.us, rajeev.rastogi@huawei.com, ants@cybertec.at, markus@bluegap.ch
  Subject: Re: Proposal for CSN based snapshots
  In-Reply-To: <539AD153.9000004@vmware.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox