Date: Mon, 1 May 2017 01:03:48 -0700
From: Andres Freund <andres@anarazel.de>
To: Petr Jelinek <petr.jelinek@2ndquadrant.com>
Cc: PostgreSQL-development <pgsql-hackers@postgresql.org>,
	Erik Rijkers <er@xs4all.nl>, Robert Haas <robertmhaas@gmail.com>,
	Craig Ringer <craig@2ndquadrant.com>
Subject: Re: snapbuild woes
Message-ID: <20170501080348.vb6mj5whdex3f7l7@alap3.anarazel.de>
References: <7286ebe9-cb1b-cf1a-f85b-fbe3402ea0bf@2ndquadrant.com>
 <9a897b86-46e1-9915-ee4c-da02e4ff6a95@2ndquadrant.com>
 <51f65289-54f8-2256-d107-937d662d69f1@2ndquadrant.com>
 <f41d64c7-1ae2-fac7-bf7f-f24dfe2d33bd@2ndquadrant.com>
 <20170413002959.apytyi5lkknauauj@alap3.anarazel.de>
 <0d06dfc1-0bc5-c3ab-3555-24e2f275d7fa@2ndquadrant.com>
 <30242bc6-eca4-b7bb-670e-8d0458753a8c@2ndquadrant.com>
 <20170501005921.qcqd7w2njpv3av6s@alap3.anarazel.de>
 <20170501013529.sejtgo2acxbvq26q@alap3.anarazel.de>
 <209861fc-718f-0a7b-3a1d-36512865c209@2ndquadrant.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <209861fc-718f-0a7b-3a1d-36512865c209@2ndquadrant.com>
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org

On 2017-05-01 03:54:49 +0200, Petr Jelinek wrote:
> I agree with adding running, I think that's good thing even for the per
> transaction tracking and snapshot exports - we could use the newly added
> field to get rid of the issue we have with 'snapshot too large' when
> there were many aborted transactions while we waited for running ones to
> finish.

I'm not sure of that - what I was proposing would only track this for
the ->running substructure.  How'd that help?


> But, I still think we need to restart the tracking after new
> xl_running_xacts. Reason for that is afaics any of the catalog snapshots
> that we assigned to transactions at the end of SnapBuildCommitTxn might
> be corrupted otherwise as they were built before we knew one of the
> supposedly running txes was actually already committed and that
> transaction might have done catalog changes.

I'm afraid you're right.  But I think this is even more complicated: The
argument in your version that this can only happen once, seems to also
be holey: Just imagine a pg_usleep(3000 * 1000000) right before
ProcArrayEndTransaction() and enjoy the picture.

Wonder if we should just (re-)add a stage between SNAPBUILD_START and
SNAPBUILD_FULL_SNAPSHOT.  Enter SNAPBUILD_BUILD_INITIAL_SNAPSHOT at the
first xl_running_xacts, wait for all transactions to end with my
approach, while populating SnapBuild->committed, only then start
collecting changes for transactions (i.e. return true from
SnapBuildProcessChange()), return true once all xacts have finished
again.  That'd presumably be a bit easier to understand, more robust -
and slower.

- Andres


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers