Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1i9wgq-0004j7-1t for pgsql-hackers@arkaria.postgresql.org; Mon, 16 Sep 2019 19:29:36 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1i9wgo-0000Ha-7t for pgsql-hackers@arkaria.postgresql.org; Mon, 16 Sep 2019 19:29:34 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1i9wgn-0000H5-OX for pgsql-hackers@lists.postgresql.org; Mon, 16 Sep 2019 19:29:33 +0000 Received: from cyclops.postgrespro.ru ([93.174.131.138] helo=mail.postgrespro.ru) by makus.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1i9wgf-0005Sz-IO for pgsql-hackers@postgresql.org; Mon, 16 Sep 2019 19:29:32 +0000 Received: from localhost (localhost [127.0.0.1]) by mail.postgrespro.ru (Postfix) with ESMTP id CCB6C21C22FC; Mon, 16 Sep 2019 22:29:20 +0300 (MSK) X-Virus-Scanned: Debian amavisd-new at postgrespro.ru X-Spam-Flag: NO X-Spam-Score: 0 X-Spam-Level: X-Spam-Status: No, score=x tagged_above=-99 required=4 WHITELISTED tests=[] autolearn=unavailable Received: from [192.168.28.74] (unknown [192.168.28.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mail.postgrespro.ru (Postfix) with ESMTPSA id A7AAB21C1F6B; Mon, 16 Sep 2019 22:29:19 +0300 (MSK) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=postgrespro.ru; s=mail; t=1568662160; bh=YiiLmnpSN3/GfILzacpcdiKShgupFqdH/bDDyitfs+k=; h=Subject:To:Cc:References:From:Date:In-Reply-To; b=mqhtpc/dITIMm7cldvjBfFBP4XCDVbCA7iJK9h3rU5e34wiEuij88cbG2mYdwGs3/ m3OgYZHQWej1Ub2w2/imvDcTsTDQ0jwb+VmCXDvIhvf1eiDkMGvo5sKej+c2ArU9pb D5BZ1Bb5eYNbcuVMIpbrUMyee2TO1CFhYcIimqqc= Subject: Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions To: Alexey Kondratov , Tomas Vondra Cc: Peter Eisentraut , Erik Rijkers , Masahiko Sawada , PostgreSQL Hackers References: <8940f374-6651-26d0-cefe-6c719454b6fe@2ndquadrant.com> <40c38758-04b5-74f4-c963-cf300f9e5dff@postgrespro.ru> <126e3a30-a7f5-ede4-2bd0-b217749c8b33@postgrespro.ru> <8eda5118-2dd0-79a1-4fe9-eec7e334de17@postgrespro.ru> <20190828190646.s3bjs22urff32k56@development> <20190829184824.kmrbchrk2ged6vjw@development> <322e40c4-8ca7-6c34-2544-28a6d95989c2@postgrespro.ru> From: Konstantin Knizhnik Message-ID: <50a87b09-a5cd-bb13-1d10-41d4147a8a95@postgrespro.ru> Date: Mon, 16 Sep 2019 22:29:18 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk On 16.09.2019 19:54, Alexey Kondratov wrote: > On 30.08.2019 18:59, Konstantin Knizhnik wrote: >> >> I think that instead of defining savepoints it is simpler and more >> efficient to use >> >> BeginInternalSubTransaction + >> ReleaseCurrentSubTransaction/RollbackAndReleaseCurrentSubTransaction >> >> as it is done in PL/pgSQL (pl_exec.c). >> Not sure if it can pr >> > > Both BeginInternalSubTransaction and DefineSavepoint use > PushTransaction() internally for a normal subtransaction start. So > they seems to be identical from the performance perspective, which is > also stated in the comment section: Yes, definitely them are using the same mechanism and most likely provides similar performance. But BeginInternalSubTransaction does not require to generate some savepoint name which seems to be redundant in this case. > > Anyway, I've performed a profiling of my apply worker (flamegraph is > attached) and it spends the vast amount of time (>90%) applying > changes. So the problem is not in the savepoints their-self, but in > the fact that we first apply all changes and then abort all the work. > Not sure, that it is possible to do something in this case. > Looks like the only way to increase apply speed is to do it in parallel: make it possible to concurrently execute non-conflicting transactions.