From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: Konstantin Knizhnik <k.knizhnik@postgrespro.ru>
Cc: Alexey Kondratov <a.kondratov@postgrespro.ru>
Cc: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Cc: Erik Rijkers <er@xs4all.nl>
Cc: Masahiko Sawada <sawada.mshk@gmail.com>
Cc: PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Subject: Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions
Date: Mon, 16 Sep 2019 22:57:19 +0200
Message-ID: <20190916205719.mnyytrsredcuf2or@development> (raw)
In-Reply-To: <50a87b09-a5cd-bb13-1d10-41d4147a8a95@postgrespro.ru>
References: <126e3a30-a7f5-ede4-2bd0-b217749c8b33@postgrespro.ru>
<c511125d-33d7-5209-98ce-5c9977f3e437@2ndquadrant.com>
<f4cf3883-5ce2-6829-bb42-3038bfc7b4d9@postgrespro.ru>
<8eda5118-2dd0-79a1-4fe9-eec7e334de17@postgrespro.ru>
<20190828190646.s3bjs22urff32k56@development>
<bb84dcf8-0a7d-5c8b-9cea-7186dc3acc3c@postgrespro.ru>
<20190829184824.kmrbchrk2ged6vjw@development>
<322e40c4-8ca7-6c34-2544-28a6d95989c2@postgrespro.ru>
<b25ce80e-f536-78c8-d5c8-a5df3e230785@postgrespro.ru>
<50a87b09-a5cd-bb13-1d10-41d4147a8a95@postgrespro.ru>
On Mon, Sep 16, 2019 at 10:29:18PM +0300, Konstantin Knizhnik wrote:
>
>
>On 16.09.2019 19:54, Alexey Kondratov wrote:
>>On 30.08.2019 18:59, Konstantin Knizhnik wrote:
>>>
>>>I think that instead of defining savepoints it is simpler and more
>>>efficient to use
>>>
>>>BeginInternalSubTransaction +
>>>ReleaseCurrentSubTransaction/RollbackAndReleaseCurrentSubTransaction
>>>
>>>as it is done in PL/pgSQL (pl_exec.c).
>>>Not sure if it can pr
>>>
>>
>>Both BeginInternalSubTransaction and DefineSavepoint use
>>PushTransaction() internally for a normal subtransaction start. So
>>they seems to be identical from the performance perspective, which
>>is also stated in the comment section:
>
>Yes, definitely them are using the same mechanism and most likely
>provides similar performance.
>But BeginInternalSubTransaction does not require to generate some
>savepoint name which seems to be redundant in this case.
>
>
>>
>>Anyway, I've performed a profiling of my apply worker (flamegraph is
>>attached) and it spends the vast amount of time (>90%) applying
>>changes. So the problem is not in the savepoints their-self, but in
>>the fact that we first apply all changes and then abort all the
>>work. Not sure, that it is possible to do something in this case.
>>
>
>Looks like the only way to increase apply speed is to do it in
>parallel: make it possible to concurrently execute non-conflicting
>transactions.
>
True, although it seems like a massive can of worms to me. I'm not aware
a way to identify non-conflicting transactions in advance, so it would
have to be implemented as optimistic apply, with a detection and
recovery from conflicts.
I'm not against doing that, and I'm willing to spend some time on revies
etc. but it seems like a completely separate effort.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: tomas.vondra@2ndquadrant.com, k.knizhnik@postgrespro.ru, a.kondratov@postgrespro.ru, peter.eisentraut@2ndquadrant.com, er@xs4all.nl, sawada.mshk@gmail.com
Subject: Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions
In-Reply-To: <20190916205719.mnyytrsredcuf2or@development>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox