Subject: Re: snapbuild woes
To: Erik Rijkers <er@xs4all.nl>
References: <f37e975c-908f-858e-707f-058d3b1eb214@2ndquadrant.com>
 <20161212214227.qxspxdqqrprfvnz3@alap3.anarazel.de>
 <194608bf-d403-b566-3578-4439e70a8119@2ndquadrant.com>
 <20161212223338.bmhswrdaipttvcve@alap3.anarazel.de>
 <7023017e-8b08-eaba-396c-80baf0a793c0@2ndquadrant.com>
 <7286ebe9-cb1b-cf1a-f85b-fbe3402ea0bf@2ndquadrant.com>
 <5094adef5a2c1cc3f56c18a0252176dd@xs4all.nl>
Cc: Andres Freund <andres@anarazel.de>,
 PostgreSQL-development <pgsql-hackers@postgresql.org>,
 Robert Haas <robertmhaas@gmail.com>, Craig Ringer <craig@2ndquadrant.com>
From: Petr Jelinek <petr.jelinek@2ndquadrant.com>
Message-ID: <e85172af-724c-3550-ca9c-ce155db8bccc@2ndquadrant.com>
Date: Wed, 22 Feb 2017 13:07:46 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101
 Thunderbird/45.7.0
MIME-Version: 1.0
In-Reply-To: <5094adef5a2c1cc3f56c18a0252176dd@xs4all.nl>
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: 7bit
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org


On 22/02/17 11:29, Erik Rijkers wrote:
> On 2017-02-22 03:05, Petr Jelinek wrote:
>>
>> So to summarize attached patches:
>> 0001 - Fixes performance issue where we build tons of snapshots that we
>> don't need which kills CPU.
>>
>> 0002 - Disables the use of ondisk historical snapshots for initial
>> consistent snapshot export as it may result in corrupt data. This
>> definitely needs backport.
>>
>> 0003 - Fixes bug where we might never reach snapshot on busy server due
>> to race condition in xl_running_xacts logging. The original use of extra
>> locking does not seem to be enough in practice. Once we have agreed fix
>> for this it's probably worth backpatching. There are still some comments
>> that need updating, this is more of a PoC.
>>
> 
> I am not not entirely sure what to expect.  Should a server with these 3
> patches do initial data copy or not?  The sgml seems to imply there is
> not inital data copy.  But my test does copy something.
> 

Not by itself (without the copy patch), those fixes are for snapshots.

> 
> With
>> 0001-Skip-unnecessary-snapshot-builds.patch
>> 0002-Don-t-use-on-disk-snapshots-for-snapshot-export-in-l.patch
>> 0003-Fix-xl_running_xacts-usage-in-snapshot-builder.patch
> 
> the consistent (but wrong) end state is always that only one of the four
> pgbench tables, pgbench_history, is replicated (always correctly).
> 
> Below is the output from the test (I've edited the lines for email)
> (below, a,b,t,h stand for: pgbench_accounts, pgbench_branches,
> pgbench_tellers, pgbench_history)
> (master on port 6972, replica on port 6973.)
> 
> port
> 6972 a,b,t,h: 100000      1     10    347
> 6973 a,b,t,h:      0      0      0    347
> 
> a,b,t,h: a68efc81a  2c27f7ba5  128590a57  1e4070879   master
> a,b,t,h: d41d8cd98  d41d8cd98  d41d8cd98  1e4070879   replica NOK
> 
> The md5-initstrings are from a md5 of the whole content of each table
> (an ordered select *)
> 
> I repeated this a few times: of course, the number of rows in
> pgbench_history varies a bit but otherwise it is always the same: 3
> empty replica tables, pgbench_history replicated correctly.
> 

That's actually correct behaviour without the initial copy patch, it
replicates changes, but since the 3 tables only get updates there is
nothing to replicate as there is no data downstream. However inserts
will of course work fine even without data downstream.

-- 
  Petr Jelinek                  http://www.2ndQuadrant.com/
  PostgreSQL Development, 24x7 Support, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers