Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1oYEZW-0007gs-Hi for pgsql-hackers@arkaria.postgresql.org; Tue, 13 Sep 2022 22:40:02 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1oYEZV-0008Gu-F7 for pgsql-hackers@arkaria.postgresql.org; Tue, 13 Sep 2022 22:40:01 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1oYEZU-0008Ga-W2 for pgsql-hackers@lists.postgresql.org; Tue, 13 Sep 2022 22:40:01 +0000 Received: from mail-io1-xd34.google.com ([2607:f8b0:4864:20::d34]) by makus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1oYEZR-0008N2-Lv for pgsql-hackers@lists.postgresql.org; Tue, 13 Sep 2022 22:39:59 +0000 Received: by mail-io1-xd34.google.com with SMTP id d8so7809210iof.11 for ; Tue, 13 Sep 2022 15:39:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=telsasoft-com.20210112.gappssmtp.com; s=20210112; h=user-agent:in-reply-to:content-disposition:mime-version:references :message-id:subject:cc:to:from:date:from:to:cc:subject:date; bh=ZooydJen6Mc7FoNoEHuGS3hkHJoGV8DUfQH0Ze3Y2/Q=; b=48XbPNPpC1RbNGQ3fv6kO3ITmXvIIJ0VcKc6HOCE6d1tSs0MUMosktdVwkCRrKbvEm +WI5CCg0P/uuGcldyrl0ovLz27ptRvyOtTryTCAdmbA2UTcKvJv8yhiaSup6AJa/l/mO QYYzSyYveVkykPTYRNlmqFPSR6KbP5OONdvHuwI9+UTJVnP1mmwZmdQQkSjIaxB4NaPB mmncInV55LJYnuby3aOiP0B0D9GfbI+VAym9x0paEktV/BnrcdwwZw8aeYsqfj9aQpdm Rmd+TjwDGMhmiruO8XfH4JDR1AoLgZMR9RuTq7bUIG05bRSHQUg8Ul54Z+Z3yXwRDcYi GOtA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=user-agent:in-reply-to:content-disposition:mime-version:references :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date; bh=ZooydJen6Mc7FoNoEHuGS3hkHJoGV8DUfQH0Ze3Y2/Q=; b=Xox1CzwP/aAVq2aItuxY1QorOhLmOx7UZoKTiDxT9x8uh/R+vC6X6zHgd+fEXtNmud /mCZjlHY2I/0Av2cIWq52OZCdCVdAtBOJ959Ga9VfH+GLIAjDnxRsDe6PVZoogLO9Syx diGV2n8CReWO0LoBsSSuIDB60cqBSjBNSMDMFK4dceCcay+L+oPEm21KODR41dgPSg5E /YHGlEvzsNV+BgNKlgeFy5S+y67qLykD0/lLWVtjI31KPeWZRin5UsV6eivRtubcoU6r ob4jUx4sk/8gaPYQbvHFyx7bl5M3o0SM5MN3OVqkHvLKocFSmSHZepsOLct4ZPMyO7Ut QVxQ== X-Gm-Message-State: ACgBeo0RANV7TZOXENPw/fq5UA/wCnj8c0Mdx/P6zQjIRT/GwS+Sq4Vr ctfI1ALALaVYeO2+5U7mRLF6rw== X-Google-Smtp-Source: AA6agR5xSOChrhRxClxNDKqamHUxkrO9XK0C9QldB3V/FvZkknStHXyJU8TsYOdkD2UsRVCjOigT4g== X-Received: by 2002:a05:6602:2b14:b0:67b:8976:2945 with SMTP id p20-20020a0566022b1400b0067b89762945mr15337376iov.82.1663108796742; Tue, 13 Sep 2022 15:39:56 -0700 (PDT) Received: from pryzbyj.telsasoft (charmander.telsasoft.com. [50.244.222.1]) by smtp.gmail.com with ESMTPSA id y21-20020a056602165500b0068a82e9b780sm1958335iow.28.2022.09.13.15.39.55 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 13 Sep 2022 15:39:56 -0700 (PDT) Received: by pryzbyj.telsasoft (Postfix, from userid 1000) id 27466800EB6; Tue, 13 Sep 2022 17:39:55 -0500 (CDT) Date: Tue, 13 Sep 2022 17:39:55 -0500 From: Justin Pryzby To: Michael Paquier Cc: Thomas Munro , Robert Haas , Aleksander Alekseev , pgsql-hackers@lists.postgresql.org Subject: Re: Background writer and checkpointer in crash recovery Message-ID: <20220913223954.GU31833@telsasoft.com> References: <162763450191.31964.15384962987091520173.pgcf@coridan.postgresql.org> <20220912005443.GB31833@telsasoft.com> <20220913021311.GN31833@telsasoft.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.4 (2018-02-28) List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk On Tue, Sep 13, 2022 at 01:32:11PM +0900, Michael Paquier wrote: > On Mon, Sep 12, 2022 at 09:13:11PM -0500, Justin Pryzby wrote: > > Like this, maybe. > > > > It's similar to what I suggested to consider backpatching here: > > https://www.postgresql.org/message-id/20201214032224.GA30237%40telsasoft.com > > At the same time, df9274adf has been done because the end-of-recovery > checkpoint done potentially by the startup process if the checkpointer > was not up at the end of recovery would take long. Any of the actions > taken in the startup process when finishing recovery are not that > costly, on the contrary, as far as I know. Your interpretation may be a bit different from mine. The central problem for me was that the startup process said "recovering NNN" after it was done recovering NNN. To fix that, df9274adf took the approach of overwriting the existing PS with "end-of-recovery checkpoint", which also adds some extra value... > I am not opposed to clearing the ps display when we are at this state > of the game for the startup process, but rather than clearing it we > could switch provide something more useful that shows what's > happening. Say a simple "performing end-of-recovery actions", for > example.. ...but the minimal solution (which 2 years ago I suggested could've been backpatched) is to *clear* the PS (in master branch at the time, that would've been before also changing it to say stuff about the checkpoint). If we'd done that, I probably wouldn't be griping about it now, so my preference is to *clear* the display as soon as the WAL processing is done; further overwriting it with additional stuff can be left as a future enhancement (like "syncing FS" that I brought up last year, and maybe everything else that calls ereport_startup_progress()).