Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1qSZ0I-006LpF-Os for pgsql-admin@arkaria.postgresql.org; Sun, 06 Aug 2023 08:20:46 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1qSZ0G-002wJG-QD for pgsql-admin@arkaria.postgresql.org; Sun, 06 Aug 2023 08:20:45 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1qSZ0G-002wJ7-FN for pgsql-admin@lists.postgresql.org; Sun, 06 Aug 2023 08:20:44 +0000 Received: from mail-pl1-x631.google.com ([2607:f8b0:4864:20::631]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94.2) (envelope-from ) id 1qSZ0A-000oY5-Cx for pgsql-admin@lists.postgresql.org; Sun, 06 Aug 2023 08:20:43 +0000 Received: by mail-pl1-x631.google.com with SMTP id d9443c01a7336-1bc3d94d40fso31136825ad.3 for ; Sun, 06 Aug 2023 01:20:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1691310037; x=1691914837; h=in-reply-to:from:references:to:content-language:subject:user-agent :mime-version:date:message-id:from:to:cc:subject:date:message-id :reply-to; bh=wq41DiSsyLFvMMNORl3RfqgJBViE7Aal6Gjmln9tNnU=; b=NPUsYiMeoCXKVZDlHzrGE96dyzlxPG9ehq816toIblkcZN/Zzm3h32SSP+c2YhxvPj 1cqim9SwIAo3Dua3q/hb78o7P1NtxgKVG3CT57xkKbf3/Abl+TGZBpoHqjh2qnWp9NOl OTnJbtUECrzekmBHJKhpCIrn4CZuLRrF2l2mKQF2RSieCJVlzrsuXUmG/z6BIBdTrW06 EVvdEvZ8101D3mSamVQby08xrpEFc6607fwGbDsS6vMlZXlvI5VWUTZkJF9ZyrTnewhB FbYMdlxi7bU2E09bfBpujI6Rg7zbXq2FrsQmM0IZG7Cs9S5ZJArg8bzax6GEJLeiXuXW 58nQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691310037; x=1691914837; h=in-reply-to:from:references:to:content-language:subject:user-agent :mime-version:date:message-id:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=wq41DiSsyLFvMMNORl3RfqgJBViE7Aal6Gjmln9tNnU=; b=UoW2ue7Wdn0IGaFVSE87mGqPmtFCjuC4HW7+/1wRL2owvI+0ghtg5T3qjBpK912mnd l4iI2nhXTCWkTKXsjdTfwzp6AFzJ92F1WkkkgskvQV8HvsG+ntQqUpeLL/B3rtLVS5oc 9+p5s7tUlhv7GltsXi4lC68R8FM7ye/StO1W2mMdpPNHBckQdm75bMPvZapon9S4AyYa p+jAXfREWO+6qjrz2jLHbUVBQTPHxUq4OkSnzr9p5X8buW8aiNbho9zndCqMaTlJcYjK GxK4KZEyUVXOc7u8RVjRN1d+s6/zB7+S7kPesHljzg3ISbI/At6UuStObzvzUloI2Zf1 4o6Q== X-Gm-Message-State: AOJu0YxLeSgeZWmw+1HDA1TnQvy2cRzWkKzO4CyG2P9DwFpKoR7k1uPg LVidAOKRgJQkLFcOEFPDgzyfegd1yLM= X-Google-Smtp-Source: AGHT+IFRcxmLZYU1vhZkQUFySG5kjSpMvyw3QkBU1YSMdbYOVYk0JkJ8Ha9/w+bXFEvpYPpb/Ti9cw== X-Received: by 2002:a17:902:b193:b0:1b8:66f6:87a3 with SMTP id s19-20020a170902b19300b001b866f687a3mr6018528plr.52.1691310037081; Sun, 06 Aug 2023 01:20:37 -0700 (PDT) Received: from ?IPV6:2600:1700:bd2c:b810:9444:c670:2c09:c643? ([2600:1700:bd2c:b810:9444:c670:2c09:c643]) by smtp.googlemail.com with ESMTPSA id t5-20020a25c305000000b00c5ec980da48sm1763112ybf.9.2023.08.06.01.20.36 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 06 Aug 2023 01:20:36 -0700 (PDT) Content-Type: multipart/alternative; boundary="------------ad0K8Md8tlXgmLPmK0nmsPj3" Message-ID: Date: Sun, 6 Aug 2023 03:20:36 -0500 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.11.0 Subject: Re: Parallel WAL Archival Options Content-Language: en-US To: pgsql-admin@lists.postgresql.org References: From: Ron In-Reply-To: List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk This is a multi-part message in MIME format. --------------ad0K8Md8tlXgmLPmK0nmsPj3 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit On 8/6/23 02:43, Nikhil Shetty wrote: > Hi Team, > > I would like to know which backup&restore tools will be better for > scenarios where the database is generating around 400 WALs per minute. If my math is correct, 400x 16MB WAL files per minute is 400*(16*2^20)/60*8 / 10^6 = 895 MBits *per second*.  Plus overhead. That's about *1Gbit/second*.  Definitely nothing to sneeze at. > We are using wal-g but it is not able to keep pace with the wal > generation. We increased the upload streams to 256 but no luck Uploads to a *remote* server? Does wal-g compress files before sending them across the wire?  By how much?  Are you CPU or IO bound by having to compress that much data? -- Born in Arizona, moved to Babylonia. --------------ad0K8Md8tlXgmLPmK0nmsPj3 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit On 8/6/23 02:43, Nikhil Shetty wrote:
Hi Team,

I would like to know which backup&restore tools will be better for scenarios where the database is generating around 400 WALs per minute.

If my math is correct, 400x 16MB WAL files per minute is 400*(16*2^20)/60*8 / 10^6 = 895 MBits per second.  Plus overhead.

That's about 1Gbit/second.  Definitely nothing to sneeze at.

We are using wal-g but it is not able to keep pace with the wal generation. We increased the upload streams to 256 but no luck

Uploads to a remote server? 

Does wal-g compress files before sending them across the wire?  By how much?  Are you CPU or IO bound by having to compress that much data?

--
Born in Arizona, moved to Babylonia.
--------------ad0K8Md8tlXgmLPmK0nmsPj3--