Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pI7aI-0006J1-Jt for pgsql-hackers@arkaria.postgresql.org; Wed, 18 Jan 2023 12:30:30 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1pI7aF-0004IR-TU for pgsql-hackers@arkaria.postgresql.org; Wed, 18 Jan 2023 12:30:27 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pI7aF-0004IH-IV for pgsql-hackers@lists.postgresql.org; Wed, 18 Jan 2023 12:30:27 +0000 Received: from mail-oi1-x232.google.com ([2607:f8b0:4864:20::232]) by makus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1pI7aD-0006vl-7V for pgsql-hackers@lists.postgresql.org; Wed, 18 Jan 2023 12:30:26 +0000 Received: by mail-oi1-x232.google.com with SMTP id o66so28443711oia.6 for ; Wed, 18 Jan 2023 04:30:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=Lzfjbm1/vhufylNeU7HYPI9QU0kF1POPDedvPGtSg2U=; b=ZKJ/SRwRyT56oTI36ylVUtcBROJGRy8hJjuC51f+QEgpXXGOT4e7F9KuoNW7CcW6N6 unAt6mV+u/0za1edNI67BsgrpSoLransJV83HLSg06KozYlGb3neDgi7xA/26GX3pZrc 6hGwn/63gfuKB1Kp7GkBbvqW+MOimMh6XY4ia3uQ8VtQaq69PhZsOLGHR+dVIuTESNtr BMcnkGrWX0YNK8F0NiVT6hsKvqdVIsZedSk1DyIGr+/B02Ksr4UpWeH7lLhv6Y1kZecO MeM5UiJXCS3+fUylXt9j4NOr/mXd0aUcKLRJM3kpK4ZQ+Ocaj4EeltWNKldZxoAOw9E1 OgSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Lzfjbm1/vhufylNeU7HYPI9QU0kF1POPDedvPGtSg2U=; b=Gnl9d1cDuiURWk9RMDapphhqrKRa979FPITsI8prxggLlHAeUYYfCvqqNBh34oMb7H HzEtgPx4EWsAyp1W9uHYCh4QiKwx4HkqVdyN8ljpsn60ZLuvImRL/LU9ieXhMR17Q67a zL1RttH5gPpEOjim07F+xGW4gAczSH/OyBMbVJvkjgEXaQTXgUOQqxLHrIfaJRAT4WFf a+HbX83VB+bdvL/8HrbDvAjnRqr9yYcb9fCxujf88XgKxhNNhNqrLBwy2udYqZxGFv9Q rj3xRmuFHt8+co1yZ1YayOfuB3Aa9nXhgFBnuir/6UI/JW/pSxhVpfVTG7Dh9aep/97O mngA== X-Gm-Message-State: AFqh2krXuFoNt+qARLByvI0roam4WKS7zJq2S+WaMBIqx4WOywP9aLrI fJWqGaSTRhpWHHG10M4Yc+Sfqgd+LQtePtIp0Kw= X-Google-Smtp-Source: AMrXdXvwhYErsY06dB9hSX7g75FAKTmlNvz/GnBmzmqMzFdFCR5FH3g56QgMv4J26j/Kaa1KOhG25sug9FGscJ/jv84= X-Received: by 2002:a05:6808:15a:b0:35a:8ab0:70f0 with SMTP id h26-20020a056808015a00b0035a8ab070f0mr414030oie.272.1674045024348; Wed, 18 Jan 2023 04:30:24 -0800 (PST) MIME-Version: 1.0 References: <0271e197-2cfe-4627-9c11-47535cc7fdf3@www.fastmail.com> <8cc4f31f-240a-4590-94a8-a50bdf4f4d1c@www.fastmail.com> In-Reply-To: From: Amit Kapila Date: Wed, 18 Jan 2023 18:00:12 +0530 Message-ID: Subject: Re: Logical replication timeout problem To: Ashutosh Bapat Cc: "wangw.fnst@fujitsu.com" , Fabrice Chapuis , Euler Taveira , Masahiko Sawada , "Hayato Kuroda (Fujitsu)" , Peter Smith , Simon Riggs , Petr Jelinek , "tanghy.fnst@fujitsu.com" , PostgreSQL Hackers , Ajin Cherian Content-Type: text/plain; charset="UTF-8" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk On Wed, Jan 18, 2023 at 5:37 PM Ashutosh Bapat wrote: > > On Wed, Jan 18, 2023 at 1:49 PM wangw.fnst@fujitsu.com > wrote: > > > > On Wed, Jan 18, 2023 at 13:29 PM Amit Kapila wrote: > > > On Tue, Jan 17, 2023 at 6:41 PM Ashutosh Bapat > > > wrote: > > > > > > > > On Tue, Jan 17, 2023 at 3:34 PM Amit Kapila wrote: > > > > > > > > > > > > > > > > I am a bit worried about the indirections that the wrappers and hooks > > > > > > create. Output plugins call OutputPluginUpdateProgress() in callbacks > > > > > > but I don't see why ReorderBufferProcessTXN() needs a callback to > > > > > > call OutputPluginUpdateProgress. > > > > > > > > > > > > > > > > Yeah, I think we can do it as we are doing the previous approach but > > > > > we need an additional wrapper (update_progress_cb_wrapper()) as the > > > > > current patch has so that we can add error context information. This > > > > > is similar to why we have a wrapper for all other callbacks like > > > > > change_cb_wrapper. > > > > > > > > > > > > > Ultimately OutputPluginUpdateProgress() will be called - which in turn > > > > will call ctx->update_progress. > > > > > > > > > > No, update_progress_cb_wrapper() should directly call > > > ctx->update_progress(). The key reason to have a > > > update_progress_cb_wrapper() is that it allows us to add error context > > > information (see the usage of output_plugin_error_callback). > > > > I think it makes sense. This also avoids the need for every output plugin to > > implement the callback. So I tried to improve the patch based on this approach. > > > > And I tried to add some comments for this new callback to distinguish it from > > ctx->update_progress. > > Comments don't help when using cscope or some such code browsing tool. > Better to use a different variable name. > + /* + * Callback to be called when updating progress during sending data of a + * transaction (and its subtransactions) to the output plugin. + */ + ReorderBufferUpdateProgressCB update_progress; Are you suggesting changing the name of the above variable? If so, how about apply_progress, progress, or updateprogress? If you don't like any of these then feel free to suggest something else. If we change the variable name then accordingly, we need to update ReorderBufferUpdateProgressCB as well. -- With Regards, Amit Kapila.