Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pbrP0-0004Ax-EX for pgsql-hackers@arkaria.postgresql.org; Mon, 13 Mar 2023 23:16:26 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1pbrOy-0004HT-No for pgsql-hackers@arkaria.postgresql.org; Mon, 13 Mar 2023 23:16:24 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pbrOy-0004HJ-A2 for pgsql-hackers@lists.postgresql.org; Mon, 13 Mar 2023 23:16:24 +0000 Received: from mail-ed1-x535.google.com ([2a00:1450:4864:20::535]) by makus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1pbrOu-0001mN-Cu for pgsql-hackers@lists.postgresql.org; Mon, 13 Mar 2023 23:16:22 +0000 Received: by mail-ed1-x535.google.com with SMTP id o12so55140882edb.9 for ; Mon, 13 Mar 2023 16:16:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=enterprisedb.com; s=google; t=1678749379; h=in-reply-to:references:cc:to:from:content-language:subject :user-agent:mime-version:date:message-id:from:to:cc:subject:date :message-id:reply-to; bh=XQnFVLLZQoUFQrQfwPIzs3tOCD6OX33efVxscJ6JLRw=; b=eX0/yxmCv3cYsz10WZs/MVdiU/XLMRdckM/fGauXZcyitJfeuxXPpV3Nx0hWAk4xsQ 9aXUsoc+lFrPtRzbeYpL2Or1k88zY36juhVSChKcr+Efp1qxaBSNGC/LXYUxJyNt3TXc gLUo7a6LRIOnonm19MSS+34yGPQ87D0fI3My68o6b5zz8Qe19ZMkZftUwYkTTQzwEzkA wFAYkKO+KYjTHBEmF6ZD3w/s6SUuCiil8N0IVj14guqAOpEludH8PHWJJosko2SS/7+5 iaTMX3pscmvnjAshbNyEQfCDq8sc2GC0Ve5s0kfTnyyCoHIwLkp96CeLGiaQw9Co3aJF nGrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678749379; h=in-reply-to:references:cc:to:from:content-language:subject :user-agent:mime-version:date:message-id:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=XQnFVLLZQoUFQrQfwPIzs3tOCD6OX33efVxscJ6JLRw=; b=lcbW0z1Irw7QAWhSJ8ttnHZVITBRhGXwpC+vTwjER7tjLOa+zyE4bm+KdX5hqXApmS rzPNfSH3yIhzxrarWwp85/Qtsuearf0lVg2Wp3jMH2dqtibYiYOGZcnHuy2H9g6ODSN9 4XYafCVp/rJEwdEwJQfazj9HM2YLvGeZs0c/AbQyYyOHhHC9C3YQR5b0Bi4LwcaP7VZJ R2Mfkd6yRxKBe4NGSAJ/Mj/hywVXkrse74/vlxldQeuiZqh8a1uZ0EzCSuOhxvBQpCT1 7AtV6K8WohuHHd2qKz2tJjpqzFERtLRs5piEHgZUzXNSgURJq0CwAqSPUkZsy+R6tFr5 IoMA== X-Gm-Message-State: AO0yUKVzZsRtVOEWbofwI4FJnH2stQ2787fLN7UjGeB2AzYc0d43v5SA vB8OFr+4NMVS4/T1nk57Cp9shSITp22xHdYi6hte7WhefmVasRnYGQ16hEyZTwe5JtkScSj+Q5W 4vf1UxH85u4Gqj3gzATEzEFidMk5D5vJb+zGIS4HT0Limg8lLWhWoQwDz+dOCUj+isB3hsB30aH 50zaI4XSEt4KIV+rcFkzxj8xmM9smPpnNpfUHgu+2o7LmbEIL4+LZx X-Google-Smtp-Source: AK7set+ITM3QR3sFAEYhJCu3V+UmnKEFz7M+ULw4UEylus/jcIncwqENama6JFuREB9Ql+KzYNV7iQ== X-Received: by 2002:a17:906:2995:b0:92b:e64a:f6 with SMTP id x21-20020a170906299500b0092be64a00f6mr302024eje.6.1678749378862; Mon, 13 Mar 2023 16:16:18 -0700 (PDT) Received: from [10.137.0.17] (ip-86-49-228-162.bb.vodafone.cz. [86.49.228.162]) by smtp.gmail.com with ESMTPSA id q20-20020a50c354000000b004c09f0ba24dsm296447edb.48.2023.03.13.16.16.17 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 13 Mar 2023 16:16:18 -0700 (PDT) Content-Type: multipart/mixed; boundary="------------ZtIPqGsoXUOho8roRIWETKDN" Message-ID: <74f02417-d18a-71dd-a406-8af41cbeb6c4@enterprisedb.com> Date: Tue, 14 Mar 2023 00:16:16 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.8.0 Subject: Re: lz4 --rm on Ubuntu 18.04 (Add LZ4 compression to pg_dump) Content-Language: en-US From: Tomas Vondra To: Michael Paquier Cc: Daniel Gustafsson , Christoph Berg , Tomas Vondra , PostgreSQL Hackers References: <819f6883-f7cf-74fd-e918-4c06a9d76d0b@enterprisedb.com> <0e8e53fa-5087-2151-dcd6-a1d3ad3ed493@enterprisedb.com> In-Reply-To: <0e8e53fa-5087-2151-dcd6-a1d3ad3ed493@enterprisedb.com> X-CLOUD-SEC-AV-Info: enterprisedb,google_mail,monitor X-CLOUD-SEC-AV-Sent: true X-Gm-Spam: 0 X-Gm-Phishy: 0 List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk This is a multi-part message in MIME format. --------------ZtIPqGsoXUOho8roRIWETKDN Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit On 3/9/23 19:00, Tomas Vondra wrote: > > > On 3/9/23 01:30, Michael Paquier wrote: >> On Thu, Mar 09, 2023 at 12:39:08AM +0100, Tomas Vondra wrote: >>> IMO we should fix that. We have a bunch of buildfarm members running on >>> Ubuntu 18.04 (or older) - it's true none of them seems to be running TAP >>> tests. But considering how trivial the fix is ... >>> >>> Barring objections, I'll push a fix early next week. >> >> +1, better to change that, thanks. Actually, would --rm be OK even on >> Windows? As far as I can see, the CI detects a LZ4 command for the >> VS2019 environment but not the liblz4 libraries that would be needed >> to trigger the set of tests. > > Thanks for noticing that. I'll investigate next week. > So, here's a fix that should (I think) replace the 'lz4 --rm' with a simple 'rm'. I have two doubts about this, though: 1) I haven't found a simple way to inject additional command into the test. The pg_dump runs have a predefined list of "steps" to run: -- compress_cmd -- glob_patterns -- command_like -- restore_cmd and I don't think there's a good place to inject the 'rm' so I ended up adding a 'cleanup_cmd' right after 'compress_cmd'. But it seems a bit strange / hacky. Maybe there's a better way? 2) I wonder if Windows will know what 'rm' means. I haven't found any TAP test doing 'rm' and don't see 'rm' in any $ENV either. That being said, I have no idea how to make this work on our Windows CI. As mentioned, the environment is missing the lz4 library - there's a setup_additional_packages_script: | REM choco install -y --no-progress ... in the .yml file, but AFAICS the chocolatey does not have lz4 :-/ regards -- Tomas Vondra EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company --------------ZtIPqGsoXUOho8roRIWETKDN Content-Type: text/x-patch; charset=UTF-8; name="lz4-rm-tap-test.patch" Content-Disposition: attachment; filename="lz4-rm-tap-test.patch" Content-Transfer-Encoding: base64 ZGlmZiAtLWdpdCBhL3NyYy9iaW4vcGdfZHVtcC90LzAwMl9wZ19kdW1wLnBsIGIvc3JjL2Jp bi9wZ19kdW1wL3QvMDAyX3BnX2R1bXAucGwKaW5kZXggOWMzNTQyMTNjZS4uYjNkY2Y2ZmY2 ZCAxMDA2NDQKLS0tIGEvc3JjL2Jpbi9wZ19kdW1wL3QvMDAyX3BnX2R1bXAucGwKKysrIGIv c3JjL2Jpbi9wZ19kdW1wL3QvMDAyX3BnX2R1bXAucGwKQEAgLTE3OCwxMSArMTc4LDE4IEBA IG15ICVwZ2R1bXBfcnVucyA9ICgKIAkJY29tcHJlc3NfY21kID0+IHsKIAkJCXByb2dyYW0g PT4gJEVOVnsnTFo0J30sCiAJCQlhcmdzICAgID0+IFsKLQkJCQknLXonLCAnLWYnLCAnLS1y bScsCisJCQkJJy16JywgJy1mJywKIAkJCQkiJHRlbXBkaXIvY29tcHJlc3Npb25fbHo0X2Rp ci9ibG9icy50b2MiLAogCQkJCSIkdGVtcGRpci9jb21wcmVzc2lvbl9sejRfZGlyL2Jsb2Jz LnRvYy5sejQiLAogCQkJXSwKIAkJfSwKKwkJIyByZW1vdmUgdGhlIHNvdXJjZSAodW5jb21w cmVzc2VkKSAudG9jIGZpbGUKKwkJY2xlYW51cF9jbWQgPT4geworCQkJcHJvZ3JhbSA9PiAn cm0nLAorCQkJYXJncyAgICA9PiBbCisJCQkJIiR0ZW1wZGlyL2NvbXByZXNzaW9uX2x6NF9k aXIvYmxvYnMudG9jIiwKKwkJCV0sCisJCX0sCiAJCSMgVmVyaWZ5IHRoYXQgZGF0YSBmaWxl cyB3ZXJlIGNvbXByZXNzZWQKIAkJZ2xvYl9wYXR0ZXJucyA9PiBbCiAJCQkiJHRlbXBkaXIv Y29tcHJlc3Npb25fbHo0X2Rpci90b2MuZGF0IiwKQEAgLTQyNzQsNiArNDI4MSwyMCBAQCBm b3JlYWNoIG15ICRydW4gKHNvcnQga2V5cyAlcGdkdW1wX3J1bnMpCiAJCWNvbW1hbmRfb2so XEBmdWxsX2NvbXByZXNzX2NtZCwgIiRydW46IGNvbXByZXNzaW9uIGNvbW1hbmRzIik7CiAJ fQogCisJaWYgKCRwZ2R1bXBfcnVuc3skcnVufS0+e2NsZWFudXBfY21kfSkKKwl7CisJCW15 ICgkY2xlYW51cF9jbWQpID0gJHBnZHVtcF9ydW5zeyRydW59LT57Y2xlYW51cF9jbWR9Owor CQlteSAkY2xlYW51cF9wcm9ncmFtID0gJGNsZWFudXBfY21kLT57cHJvZ3JhbX07CisKKwkJ IyBTa2lwIHRoZSByZXN0IG9mIHRoZSB0ZXN0IGlmIHRoZSBjb21wcmVzc2lvbiBwcm9ncmFt IGlzCisJCSMgbm90IGRlZmluZWQuCisJCW5leHQgaWYgKCFkZWZpbmVkKCRjbGVhbnVwX2Nt ZCkgfHwgJGNsZWFudXBfY21kIGVxICcnKTsKKworCQlteSBAZnVsbF9jbGVhbnVwX2NtZCA9 CisJCSAgKCRjbGVhbnVwX2NtZC0+e3Byb2dyYW19LCBAeyAkY2xlYW51cF9jbWQtPnthcmdz fSB9KTsKKwkJY29tbWFuZF9vayhcQGZ1bGxfY2xlYW51cF9jbWQsICIkcnVuOiBjbGVhbnVw IGNvbW1hbmRzIik7CisJfQorCiAJaWYgKCRwZ2R1bXBfcnVuc3skcnVufS0+e2dsb2JfcGF0 dGVybnN9KQogCXsKIAkJbXkgJGdsb2JfcGF0dGVybnMgPSAkcGdkdW1wX3J1bnN7JHJ1bn0t PntnbG9iX3BhdHRlcm5zfTsK --------------ZtIPqGsoXUOho8roRIWETKDN--