Date: Sun, 29 Mar 2020 17:31:05 +0200
From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: Dilip Kumar <dilipbalaut@gmail.com>,
	Michael Paquier <michael@paquier.xyz>,
	Peter Eisentraut <peter.eisentraut@2ndquadrant.com>,
	PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Subject: Re: PATCH: logical_work_mem and logical streaming of large
 in-progress transactions
Message-ID: <20200329153105.mdryxzz562tg65pk@development>
References: 
 <CAFiTN-t1YNmoBf7k1kUrUue4q1Tf3GXjGwZTseNLivKmr9sz1Q@mail.gmail.com>
 <CAFiTN-vR7jQBP75FpvE=-4Ca-3jqsJA55ZDct0B8UtE0x13gCQ@mail.gmail.com>
 <CAFiTN-uNzEyY3YL1EZabWcG=Bah1nO8hno+-UaAkCDcfCGYC0g@mail.gmail.com>
 <20200303214646.6adway2z5wnapjem@development>
 <CAFiTN-sn0Bd4Q72Cn9t8-mZp1sxWRv2SmUoeR_7J05SqNTR--g@mail.gmail.com>
 <CAA4eK1KyrP6m53FQV5v0gysEnaTHPdc2xYT0KeNN+fhOcedRwA@mail.gmail.com>
 <CAFiTN-uBd3p50s=kfEiSbg93_Fnf_VU+haS4FqBZAm7PmbX69g@mail.gmail.com>
 <CAA4eK1LkLU5frY4kZGO38d8m6R504H+GbfkwsQLAp7sQ-4oqbw@mail.gmail.com>
 <20200329005905.ooxq25qyjwn2mvxz@development>
 <CAA4eK1LvUDNaCJUZEh5BhV31U6OnNWP=OCUmyZM2k4JGx5CXvQ@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Disposition: inline
In-Reply-To: 
 <CAA4eK1LvUDNaCJUZEh5BhV31U6OnNWP=OCUmyZM2k4JGx5CXvQ@mail.gmail.com>
Precedence: bulk

On Sun, Mar 29, 2020 at 11:19:21AM +0530, Amit Kapila wrote:
>On Sun, Mar 29, 2020 at 6:29 AM Tomas Vondra
><tomas.vondra@2ndquadrant.com> wrote:
>>
>> On Sat, Mar 28, 2020 at 03:29:34PM +0530, Amit Kapila wrote:
>> >On Sat, Mar 28, 2020 at 2:19 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
>> >
>> >How about if instead of writing an XLOG_XACT_ASSIGNMENT WAL, we set a
>> >flag in TransactionStateData and then log that as special information
>> >whenever we write next WAL record for a new subtransaction?  Then
>> >during recovery, we can only call ProcArrayApplyXidAssignment when we
>> >find that special flag is set in a WAL record.  One idea could be to
>> >use a flag bit in XLogRecord.xl_info.  If that is feasible then the
>> >solution can work as it is now, without any overhead or change in the
>> >way we maintain KnownAssignedXids.
>> >
>>
>> Ummm, how is that different from what the patch is doing now? I mean, we
>> only write the top-level XID for the first WAL record in each subxact,
>> right? Or what would be the difference with your approach?
>>
>
>We have to do what the patch is currently doing and additionally, we
>will set this flag after PGPROC_MAX_CACHED_SUBXIDS which would allow
>us to call ProcArrayApplyXidAssignment during WAL replay only after
>PGPROC_MAX_CACHED_SUBXIDS number of subxacts.  It will help us in
>clearing the KnownAssignedXids at the same time as we do now, so no
>additional performance overhead.
>

Hmmm. So we'd still log assignment twice? Or would we keep just the
immediate assignments (embedded into xlog records), and cache the
subxids on the replica somehow?


regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services