Subject: Re: refactoring basebackup.c
To: Jeevan Ladhe <jeevanladhe.os@gmail.com>,
 Robert Haas <robertmhaas@gmail.com>
Cc: "Shinoda, Noriyoshi (PN Japan FSIP)" <noriyoshi.shinoda@hpe.com>,
 Justin Pryzby <pryzby@telsasoft.com>, Dipesh Pandit
 <dipesh.pandit@gmail.com>, Abhijit Menon-Sen <ams@toroid.org>,
 Dmitry Dolgov <9erthalion6@gmail.com>,
 Jeevan Ladhe <jeevan.ladhe@enterprisedb.com>,
 Mark Dilger <mark.dilger@enterprisedb.com>,
 "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
References: 
 <CA+TgmoboJJE6M+u4f2E9vKaKGVq6jKync81Bi1jFW8QChscwOg@mail.gmail.com>
 <CA+TgmoYzYVwkDpwHru5+Lf5f0u4cTN6FeBLsJeHxXTBS7v=11A@mail.gmail.com>
 <YgPEhynPPLUFf7x3@toroid.org>
 <CAN1g5_FEDTQXk8h_bUQcc0ULKco=Tv=_QGuiwJ73Qay=T2monQ@mail.gmail.com>
 <CANm22Cj4qakrKy5Evkt29tnW9WqqgfB1u3O_btwYGhPp_M18pw@mail.gmail.com>
 <CAN1g5_FeDmiA9D8wdG2W6Lkq5CpubxOAqTmd2et9hsinTJtsMQ@mail.gmail.com>
 <CANm22Cg9cArXEaYgHVZhCnzPLfqXCZLAzjwTq7Fc0quXRPfbxA@mail.gmail.com>
 <CAN1g5_FrApD0OFbWMF8PRF9W+EAPEV6wS7cCJp-h0qMe7cg8aA@mail.gmail.com>
 <CA+TgmoYUDEJga2qV_XbAZ=pGEBaOsgFmzZ6Ac4_sRwOm_+UeHA@mail.gmail.com>
 <CANm22CgPb1qZRHxqr6jzP2Xt_c3=6TiCBv_AB5G9eqBP90XDgw@mail.gmail.com>
 <20220211152944.GD31460@telsasoft.com>
 <CA+Tgmobo_ZQpRw3rDKM0H6iOBEv4ccA=-7bGo4OSXeHGiWQm4g@mail.gmail.com>
 <PH7PR84MB188512B88392DD495D8F1E3EEE319@PH7PR84MB1885.NAMPRD84.PROD.OUTLOOK.COM>
 <CA+TgmoZeUPgY3TJZMuiFQyFfzpZEtQhjTU2UaKirD_E4Gk_2sQ@mail.gmail.com>
 <CANm22CjESZPRiGB5Cs3EJQhsKs8R5dbfYt8Z=ni4=TLMbcW9iw@mail.gmail.com>
From: tushar <tushar.ahuja@enterprisedb.com>
Message-ID: <6a2e3bf5-100c-8730-c49b-29932eef7d0e@enterprisedb.com>
Date: Tue, 15 Feb 2022 22:24:48 +0530
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: 
 <CANm22CjESZPRiGB5Cs3EJQhsKs8R5dbfYt8Z=ni4=TLMbcW9iw@mail.gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US
Archived-At: 
 <https://www.postgresql.org/message-id/6a2e3bf5-100c-8730-c49b-29932eef7d0e%40enterprisedb.com>
Precedence: bulk

On 2/15/22 6:48 PM, Jeevan Ladhe wrote:
> Please find the attached updated version of patch for ZSTD server side
Thanks, Jeevan, I again tested with the attached patch, and as mentioned 
the crash is fixed now.

also, I tested with different labels with gzip V/s zstd against data 
directory size which is 29GB and found these results

====
./pg_basebackup  -t server:/tmp/<directory> 
--compress=server-zstd:<label>  -Xnone -n -N --no-estimate-size -v

--compress=server-zstd:1 =  compress directory size is  1.3GB
--compress=server-zstd:4 = compress  directory size is  1.3GB
--compress=server-zstd:7 = compress  directory size is  1.2GB
--compress=server-zstd:12 = compress directory size is 1.2GB
====

===
./pg_basebackup  -t server:/tmp/<directooy> 
--compress=server-gzip:<label>  -Xnone -n -N --no-estimate-size -v

--compress=server-gzip:1 =  compress directory size is  1.8GB
--compress=server-gzip:4 = compress  directory size is  1.6GB
--compress=server-gzip:9 = compress  directory size is  1.6GB
===

-- 
regards,tushar
EnterpriseDB  https://www.enterprisedb.com/
The Enterprise PostgreSQL Company