pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  

help / color / mirror / Atom feed
From: Tomas Vondra <tomas.vondra@enterprisedb.com>
To: Anastasia Lubennikova <a.lubennikova@postgrespro.ru>
To: Tomas Vondra <tomas.vondra@2ndquadrant.com>
Cc: Tatsuo Ishii <ishii@sraoss.co.jp>
Cc: lubennikovaav@gmail.com
Cc: pgsql-hackers@lists.postgresql.org
Cc: pavan.deolasee@gmail.com
Cc: ibrar.ahmad@gmail.com
Subject: Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits
Date: Sat, 16 Jan 2021 23:18:19 +0100
Message-ID: <13a5b2dc-4fe9-d4a6-01df-d2c7bba8fd2c@enterprisedb.com> (raw)
In-Reply-To: <58359211-c505-1ee6-c5fd-78db84afb46f@postgrespro.ru>
References: <CABOikdN-ptGv0mZntrK2Q8OtfUuAjqaYMGmkdU1dCKFtUxVLrg@mail.gmail.com>
	<CALtqXTfpEOiyJchYQoUKw9pvEPhVgnkxL=JvKNj57BTxva8bTA@mail.gmail.com>
	<160382617166.1204.475752573208579406.pgcf@coridan.postgresql.org>
	<20201028.144653.966121790856478893.t-ishii@sraoss.co.jp>
	<20201030004252.d63rwclys4bniluu@development>
	<1d2e0f4a-6ab8-10f1-c177-af97475df001@postgrespro.ru>
	<20201102144453.tumlk7qhw3gzn2dk@development>
	<7e6a006f-c59a-93fc-3ddf-f9c661319bc8@2ndquadrant.com>
	<75f3760f-1077-07c1-aea4-975d5c127d6d@postgrespro.ru>
	<f6e0af44-b147-d27f-cc21-2404b4f11f0e@enterprisedb.com>
	<93b767d6-4cde-7326-8de4-d71d81817c16@postgrespro.ru>
	<8669d452-d078-bdf9-5a62-6f1acd55f3c6@enterprisedb.com>
	<58359211-c505-1ee6-c5fd-78db84afb46f@postgrespro.ru>

On 1/16/21 4:11 PM, Anastasia Lubennikova wrote:
>
 > ...
> 
> As Pavan correctly figured it out before the problem is that 
> RelationGetBufferForTuple() moves to the next page, losing free space in 
> the block:
> 
>  > ... I see that a relcache invalidation arrives
>  > after 1st and then after every 32672th block is filled. That clears the
>  > rel->rd_smgr field and we lose the information about the saved target
>  > block. The code then moves to extend the relation again and thus 
> skips the
>  > previously less-than-half filled block, losing the free space in that 
> block.
> 
> The reason of this cache invalidation is vm_extend() call, which happens 
> every 32762 blocks.
> 
> RelationGetBufferForTuple() tries to use the last page, but for some 
> reason this code is under 'use_fsm' check. And COPY FROM doesn't use fsm 
> (see TABLE_INSERT_SKIP_FSM).
> 
> 
>          /*
>           * If the FSM knows nothing of the rel, try the last page 
> before we
>           * give up and extend.  This avoids one-tuple-per-page syndrome 
> during
>           * bootstrapping or in a recently-started system.
>           */
>          if (targetBlock == InvalidBlockNumber)
>          {
>              BlockNumber nblocks = RelationGetNumberOfBlocks(relation);
>              if (nblocks > 0)
>                  targetBlock = nblocks - 1;
>          }
> 
> 
> I think we can use this code without regard to 'use_fsm'. With this 
> change, the number of toast rel pages is correct. The patch is attached.
> 

Thanks for the updated patch, this version looks OK to me - I've marked 
it as RFC. I'll do a bit more testing, review, and then I'll get it 
committed.

regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company





view thread (71+ messages)  latest in thread

Message-ID: <13a5b2dc-4fe9-d4a6-01df-d2c7bba8fd2c@enterprisedb.com>
Permalink:  ../13a5b2dc-4fe9-d4a6-01df-d2c7bba8fd2c@enterprisedb.com/
Also on:    postgresql.org/message-id/13a5b2dc-4fe9-d4a6-01df-d2c7bba8fd2c@enterprisedb.com

 · 

reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-hackers@postgresql.org
  Cc: tomas.vondra@enterprisedb.com, a.lubennikova@postgrespro.ru, tomas.vondra@2ndquadrant.com, ishii@sraoss.co.jp, lubennikovaav@gmail.com, pgsql-hackers@lists.postgresql.org, pavan.deolasee@gmail.com, ibrar.ahmad@gmail.com
  Subject: Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits
  In-Reply-To: <13a5b2dc-4fe9-d4a6-01df-d2c7bba8fd2c@enterprisedb.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox