Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1m8Fzd-0002gW-0m for pgsql-hackers@arkaria.postgresql.org; Tue, 27 Jul 2021 05:51:05 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1m8Fzb-0000YP-Qz for pgsql-hackers@arkaria.postgresql.org; Tue, 27 Jul 2021 05:51:03 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1m8Fzb-0000V6-Hv for pgsql-hackers@lists.postgresql.org; Tue, 27 Jul 2021 05:51:03 +0000 Received: from mail-wr1-x42d.google.com ([2a00:1450:4864:20::42d]) by makus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1m8FzV-00008T-3u for pgsql-hackers@lists.postgresql.org; Tue, 27 Jul 2021 05:51:02 +0000 Received: by mail-wr1-x42d.google.com with SMTP id j2so13658378wrx.9 for ; Mon, 26 Jul 2021 22:50:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=aiven.io; s=google; h=from:to:subject:date:message-id:organization:mime-version :content-transfer-encoding; bh=e/wmufN1/DeXk9qP2cEpxSd5h4njzZM2Ei8BUYqo3os=; b=XUY4GQtET0T3zHRNjVikJZq0/dYeSVX4FUlZz4A1bArlnVB6XAmGGM6+NroMPT4Co9 Y0nIzyIz52ERdmIUTFXobp2+yrlbkcd4v88LRHATHGOn51YZWT42WX0FWnxdNicK3BZk RwPa3sMX/fLNsoiCiOBfgVF7rucllrK9y0YCI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:organization :mime-version:content-transfer-encoding; bh=e/wmufN1/DeXk9qP2cEpxSd5h4njzZM2Ei8BUYqo3os=; b=JwKSrnY5vwiinsCI9pVsWK6oqEz46v2gEm5Y0nkyopChN7xuHH+IHf4X2cR9Sx1+sU dvVGxLqdII9Vf3x/DnBmQzkHsDSXwCC0tlAPxHHfrvNIX5xpb5EtgFUYdTxSyYeReka3 b1aOSDNwXRqhCDltlQju0WFv+SkOxU7kQ/2DlAmiOOCUe2UMEZfRLpBO87E2d94ukWFY ZLx3c5HVLa744EK2LI3atMHCVCbZ1uQvbnZEOjKj8fMUo5OozkcDQ2h2mXOC1HmBaNBS Fk86b9S6v9s+hG6yZNjMOiXEj6cL/gXfRQFCew0+G4hDmPee99GgfzDl/L+W2Wc5ohi3 ELfw== X-Gm-Message-State: AOAM532BObD7rH/BA7hS5BvDVBmw3xdnIllrYI/0uyZzbiVKFtFBohkz au1DrwvC/Bop9H0JN52E6nk3dMPHkbQJbPv9 X-Google-Smtp-Source: ABdhPJxmt7qSQn6QqCn2f509R8G0w42kYJlI0uiD9K/jIEhLx/jLshAO0q5w8uXLVWkp/cdI/LBsNw== X-Received: by 2002:adf:f1cd:: with SMTP id z13mr4243130wro.210.1627365054214; Mon, 26 Jul 2021 22:50:54 -0700 (PDT) Received: from aivenronan.localnet (static-176-158-121-96.ftth.abo.bbox.fr. [176.158.121.96]) by smtp.gmail.com with ESMTPSA id n5sm1862200wri.31.2021.07.26.22.50.53 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 26 Jul 2021 22:50:53 -0700 (PDT) From: Ronan Dunklau To: pgsql-hackers@lists.postgresql.org Subject: pg_receivewal starting position Date: Tue, 27 Jul 2021 07:50:39 +0200 Message-ID: <18708360.4lzOvYHigE@aivenronan> Organization: aiven MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk Hello, I've notived that pg_receivewal logic for deciding which LSN to start streaming at consists of: - looking up the latest WAL file in our destination folder, and resume from here - if there isn't, use the current flush location instead. This behaviour surprised me when using it with a replication slot: I was expecting it to start streaming at the last flushed location from the replication slot instead. If you consider a backup tool which will take pg_receivewal's output and transfer it somewhere else, using the replication slot position would be the easiest way to ensure we don't miss WAL files. Does that make sense ? I don't know if it should be the default, toggled by a command line flag, or if we even should let the user provide a LSN. I'd be happy to implement any of that if we agree. -- Ronan Dunklau