From: Petr Jelinek <petr.jelinek@2ndquadrant.com>
To: Andres Freund <andres@anarazel.de>
To: Noah Misch <noah@leadboat.com>
Cc: pgsql-hackers@postgresql.org, Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Cc: Erik Rijkers <er@xs4all.nl>
Cc: David Steele <david@pgmasters.net>
Cc: Robert Haas <robertmhaas@gmail.com>
Cc: Craig Ringer <craig@2ndquadrant.com>
Cc: pgsql-hackers-owner@postgresql.org
Subject: Re: snapbuild woes
Date: Fri, 12 May 2017 18:40:33 +0200
Message-ID: <b39c9f0d-5fc7-bfd7-7495-74cc4f450785@2ndquadrant.com> (raw)
In-Reply-To: <c8d39fad-d163-7a6d-512c-03c4999a8e67@2ndquadrant.com>
References: <20170413045812.GA2917375@tornado.leadboat.com>
<20170417020404.GB2988768@tornado.leadboat.com>
<20170418041657.hvwss6hz73ghozp5@alap3.anarazel.de>
<20170422053621.2qtvurecfatet2ei@alap3.anarazel.de>
<20170428043444.GD787347@rfd.leadboat.com>
<D8279AAF-3532-4F66-9D31-1BEEDBAC0819@anarazel.de>
<20170504054516.GA850485@rfd.leadboat.com>
<F8C3AE44-4507-4C26-9B50-112A929D276D@anarazel.de>
<20170511215155.bnigoevqoe4hejmg@alap3.anarazel.de>
<20170511215426.5u2itcy5mruo65on@alap3.anarazel.de>
<20170512013127.ytncnqvlluheandz@alap3.anarazel.de>
<c8d39fad-d163-7a6d-512c-03c4999a8e67@2ndquadrant.com>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-hackers>
On 12/05/17 10:57, Petr Jelinek wrote:
> On 12/05/17 03:31, Andres Freund wrote:
>> On 2017-05-11 14:54:26 -0700, Andres Freund wrote:
>>> On 2017-05-11 14:51:55 -0700, wrote:
>>>> On 2017-05-08 00:10:12 -0700, Andres Freund wrote:
>>>>> I plan to commit the next pending patch after the back branch releases
>>>>> are cut - it's an invasive fix and the issue doesn't cause corruption
>>>>> "just" slow slot creation. So it seems better to wait for a few days,
>>>>> rather than hurry it into the release.
>>>>
>>>> Now that that's done, here's an updated version of that patch. Note the
>>>> new logic to trigger xl_running_xact's to be logged at the right spot.
>>>> Works well in my testing.
>>>>
>>>> I plan to commit this fairly soon, unless somebody wants a bit more time
>>>> to look into it.
>>>>
>>>> Unless somebody protests, I'd like to slightly revise how the on-disk
>>>> snapshots are stored on master - given the issues this bug/commit showed
>>>> with the current method - but I can see one could argue that that should
>>>> rather be done next release.
>>>
>>> As usual I forgot my attachement.
>>
>> This patch also seems to offer a way to do your 0005 in, possibly, more
>> efficient manner. We don't ever need to assume a transaction is
>> timetravelling anymore. Could you check whether the attached, hasty,
>> patch resolves the performance problems you measured? Also, do you have
>> a "reference" workload for that?
>>
>
> Hmm, well it helps but actually now that we don't track individual
> running transactions anymore it got much less effective (my version of
> 0005 does as well).
>
> The example workload I test with is:
> session 1: open transaction, do a write, keep it open
> session 2: pgbench -M simple -N -c 10 -P 1 -T 5
> session 3: run CREATE_REPLICATION_SLOT LOGICAL in walsender
> session 2: pgbench -M simple -N -c 10 -P 1 -T 20
> session 1: commit
>
> And wait for session 3 to finish slot creation, takes about 20 mins on
> my laptop without patches, minute and half with your patches for 0004
> and 0005 (or with your 0004 and my 0005) and about 2s with my original
> 0004 and 0005.
>
> What makes it slow is the constant qsorting IIRC.
>
Btw I still think that we should pursue the approach you proposed. I
think we can deal with the qsorting in some other ways (ordered insert
perhaps?) later. What you propose fixes the correctness, makes
performance less awful in the above workload and also makes the
snapbuild bit easier to read.
--
Petr Jelinek http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: petr.jelinek@2ndquadrant.com, andres@anarazel.de, noah@leadboat.com, peter.eisentraut@2ndquadrant.com, er@xs4all.nl, david@pgmasters.net, robertmhaas@gmail.com, craig@2ndquadrant.com, pgsql-hackers-owner@postgresql.org
Subject: Re: snapbuild woes
In-Reply-To: <b39c9f0d-5fc7-bfd7-7495-74cc4f450785@2ndquadrant.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox