pg.ddx.io  pgsql-committers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Amit Kapila <akapila@postgresql.org>
To: pgsql-committers@lists.postgresql.org
Subject: pgsql: Optimize fileset usage in apply worker.
Date: Thu, 02 Sep 2021 02:55:11 +0000
Message-ID: <E1mLcsh-0000Ud-QO@gemulon.postgresql.org> (raw)

Optimize fileset usage in apply worker.

Use one fileset for the entire worker lifetime instead of using
separate filesets for each streaming transaction. Now, the
changes/subxacts files for every streaming transaction will be
created under the same fileset and the files will be deleted
after the transaction is completed.

This patch extends the BufFileOpenFileSet and BufFileDeleteFileSet
APIs to allow users to specify whether to give an error on missing
files.

Author: Dilip Kumar, based on suggestion by Thomas Munro
Reviewed-by: Hou Zhijie, Masahiko Sawada, Amit Kapila
Discussion: https://postgr.es/m/E1mCC6U-0004Ik-Fs@gemulon.postgresql.org

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/31c389d8de915b705ee06c7a6e9246e20f36b9dc

Modified Files
--------------
src/backend/replication/logical/launcher.c |   6 +-
src/backend/replication/logical/worker.c   | 257 ++++++-----------------------
src/backend/storage/file/buffile.c         |  22 ++-
src/backend/utils/sort/logtape.c           |   2 +-
src/backend/utils/sort/sharedtuplestore.c  |   3 +-
src/include/replication/worker_internal.h  |  10 +-
src/include/storage/buffile.h              |   5 +-
7 files changed, 86 insertions(+), 219 deletions(-)



Message-ID: <E1mLcsh-0000Ud-QO@gemulon.postgresql.org>
Permalink:  ../E1mLcsh-0000Ud-QO@gemulon.postgresql.org/
Also on:    postgresql.org/message-id/E1mLcsh-0000Ud-QO@gemulon.postgresql.org

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-committers@postgresql.org
  Cc: akapila@postgresql.org
  Subject: Re: pgsql: Optimize fileset usage in apply worker.
  In-Reply-To: <E1mLcsh-0000Ud-QO@gemulon.postgresql.org>
  References: <E1mLcsh-0000Ud-QO@gemulon.postgresql.org>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox