Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lLSDx-0005LK-Gp for pgsql-hackers@arkaria.postgresql.org; Sun, 14 Mar 2021 15:00:09 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1lLSDw-0005X3-BI for pgsql-hackers@arkaria.postgresql.org; Sun, 14 Mar 2021 15:00:08 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lLSDw-0005Ww-31 for pgsql-hackers@lists.postgresql.org; Sun, 14 Mar 2021 15:00:08 +0000 Received: from mail-lf1-x12a.google.com ([2a00:1450:4864:20::12a]) by magus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1lLSDt-0004DC-01 for pgsql-hackers@postgresql.org; Sun, 14 Mar 2021 15:00:07 +0000 Received: by mail-lf1-x12a.google.com with SMTP id p21so52877689lfu.11 for ; Sun, 14 Mar 2021 08:00:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:from:to:cc:references:message-id:date:user-agent :mime-version:in-reply-to:content-language; bh=57KMqPZdPICdI4lpSXOsVmB/Lj6XHVtgz0pq2Hs/0AA=; b=kYWgLqSeR81MuFwi4jx1jqoh3Tj3cx9FIwLB7PVJg4p2bEU3GmxG5d/7nZusxiL43v BMAAMsjCfL403yqiGkNykvzttwS6NLFa7o6o1iwmdOlAtGihoutoDTaph1OL1ABv6Svk Tu+4fcU45RxC9jR2Ik/Ra04Sy8bAQNMZ9ut3EmSEAoMxfFCG4PqqJaWH9Kr/eVCxqYZd SqOfZvvfewuY1fqo+hNFf19sDJ9MQY+jdU0xpqw8t0t6LHE4zMOi6n/Jz2vhEneG0QDP YAmaUU45AOHarS04oi4JdLhyJWmrfy1WaS3Z1AAFPBH4Ep1SgChdukxUVin6/0jVNDMs ndiQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:from:to:cc:references:message-id:date :user-agent:mime-version:in-reply-to:content-language; bh=57KMqPZdPICdI4lpSXOsVmB/Lj6XHVtgz0pq2Hs/0AA=; b=I9jkVchg2/pQlap9thWlvoIvyhatMWqQN0tDeEi9RzMKyGd62yMcRZsOgC86j8H6WM JksJAfdxqPKaCEa0z1OJRMht2GmGXeBq+u/7XZRSCFid6nphYQ9Y9hCftWJz8bBZtxLr Ge4AwiVihOnz3/vWRQjzntsw1tQ97qKTDCLjdL+wygDMs2Hv0y/CAjS0BO3AZ3BCs4bx oFj7/Qqmoi8oJ3KLKZpCpUr3WcOb8dTvDeW1MOOJLBbTcJwlqLzzuXAajW1LnWgDCYqE 6Rh6Kj2FqDFBPeZ18HrbWBVqnFGhMf2+v7FhjRiKksSF6o2cBLsk8+LuZJJGqBaKIwwh f0iQ== X-Gm-Message-State: AOAM533YWb51weqybpAToBuUUOGS3cZkikE3oySBjOIFVNNQX1r0Fhq6 DbeLOnTuQ7iyhQLU5HTUcXTs5PrX2m0= X-Google-Smtp-Source: ABdhPJy6VmiX/RHtBIj+q4zyg1o/ODXn/kGJhSCwvxi0iQIjN4TZ630hjT7n2KpOSuWmEkVfHKCAFA== X-Received: by 2002:a19:791:: with SMTP id 139mr5421598lfh.436.1615734003391; Sun, 14 Mar 2021 08:00:03 -0700 (PDT) Received: from [1.0.0.7] ([178.155.6.222]) by smtp.gmail.com with ESMTPSA id w24sm2654443ljh.19.2021.03.14.08.00.02 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 14 Mar 2021 08:00:02 -0700 (PDT) Subject: Re: More time spending with "delete pending" From: Alexander Lakhin To: pgsql-hackers Cc: Tom Lane , Justin Pryzby References: <20201115011112.GW30691@telsasoft.com> <979f4ee6-9960-e37f-f3ee-f458e87777b0@gmail.com> <995713.1605725597@sss.pgh.pa.us> <1001775.1605731995@sss.pgh.pa.us> <3366bfde-183f-60be-62ab-8ae81b946bac@gmail.com> <1006364.1605738508@sss.pgh.pa.us> <20729794-e2bf-bd7a-f01b-d132f89fd0d7@gmail.com> Message-ID: Date: Sun, 14 Mar 2021 18:00:00 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: <20729794-e2bf-bd7a-f01b-d132f89fd0d7@gmail.com> Content-Type: multipart/mixed; boundary="------------13D8790EF69A818C69C27BC9" Content-Language: en-US List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk This is a multi-part message in MIME format. --------------13D8790EF69A818C69C27BC9 Content-Type: multipart/alternative; boundary="------------38CD6CFA3F1BE71B6DDD7B51" --------------38CD6CFA3F1BE71B6DDD7B51 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Hello hackers, 22.11.2020 18:59, Alexander Lakhin wrote: > 19.11.2020 01:28, Tom Lane wrote: > >> Hmm, that is an interesting question isn't it. Here's a search going >> back a full year. There are a few in v12 --- interestingly, all on >> the statistics file, none from pg_ctl --- and none in v13. Of course, >> v13 has only existed as a separate branch since 2020-06-07. >> >> There's also a buildfarm test-coverage artifact involved. The bulk >> of the HEAD reports are from dory and walleye, neither of which are >> building v13 as yet, because of unclear problems [1]. I think those >> two animals build much more frequently than our other Windows animals, >> too, so the fact that they have more may be just because of that and >> not because they're somehow more susceptible. Because of that, I'm not >> sure that we have enough evidence to say that v13 is better than HEAD. >> If there is some new bug, it's post-v12, but maybe not post-v13. But >> v12 is evidently not perfect either. > The failures on HEAD (on dory and walleye) need another investigation > (maybe they are caused by modified stat()...). To revive this topic and make sure that the issue still persists, I've searched through the buildfarm logs (for HEAD and REL_13_STABLE) and collected the following failures of the misc_functions test with the "Permission denied" error: bowerbird HEAD (13 runs last week, 500 total, 18 failed) - REL_13_STABLE (4 runs last week, 237 total, 8 failed) - drongo HEAD (25 runs last week, 500 total, 24 failed) https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=drongo&dt=2020-12-27%2019%3A01%3A50 https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=drongo&dt=2021-01-18%2001%3A01%3A55 REL_13_STABLE (5 runs last week, 255 total, 2 failed) - hamerkop HEAD - (8 runs last week, 501 total, 20 failed) https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=hamerkop&dt=2021-02-22%2010%3A20%3A00 fairywren HEAD - (27 runs last week, 500 total, 28 failed) https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=fairywren&dt=2021-02-22%2012%3A03%3A35 https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=fairywren&dt=2021-01-28%2000%3A04%3A50 https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=fairywren&dt=2021-01-19%2023%3A29%3A38 https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=fairywren&dt=2021-01-17%2018%3A08%3A11 https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=fairywren&dt=2020-11-30%2009%3A57%3A15 https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=fairywren&dt=2020-11-19%2006%3A02%3A22 https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=fairywren&dt=2020-11-18%2018%3A02%3A04 https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=fairywren&dt=2020-09-16%2019%3A48%3A21 REL_13_STABLE (5 runs last week, 302 total, 5 failed) https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=fairywren&dt=2021-02-26%2003%3A03%3A42 https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=fairywren&dt=2020-10-20%2007%3A50%3A44 https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=fairywren&dt=2020-09-10%2006%3A02%3A12 https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=fairywren&dt=2020-09-04%2019%3A58%3A39 woodlouse HEAD (33 runs last week, 500 total, 5 failed) - REL_13_STABLE (4 runs last week, 325 total, 1 failed) https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=woodlouse&dt=2021-01-20%2011%3A42%3A30 dory HEAD (52 runs last week, 500 total, 25 failed) - whelk HEAD (31 runs last week, 500 total, 11 failed) - REL_13_STABLE (7 runs last week, 224 total, 1 failed) - (I've not considered lorikeet, jacana, and walleye as they are using alternative compilers.) So on some machines there wasn't a single failure (probably it depends on hardware), but where it fails, such failures account for a significant share of all failures. I believe that the patch attached to [1] should fix this issue. The patch still applies to master and makes the demotest (attached to [2]) pass. Also I've prepared a trivial patch that makes pgwin32_open() use the original stat() function (as in the proposed change for _pgstat64()). https://www.postgresql.org/message-id/979f4ee6-9960-e37f-f3ee-f458e87777b0%40gmail.com https://www.postgresql.org/message-id/c3427edf-d7c0-ff57-90f6-b5de3bb62709%40gmail.com Best regards, Alexander --------------38CD6CFA3F1BE71B6DDD7B51 Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: 7bit
Hello hackers,

22.11.2020 18:59, Alexander Lakhin wrote:
19.11.2020 01:28, Tom Lane wrote:


Hmm, that is an interesting question isn't it.  Here's a search going
back a full year.  There are a few in v12 --- interestingly, all on
the statistics file, none from pg_ctl --- and none in v13.  Of course,
v13 has only existed as a separate branch since 2020-06-07.

There's also a buildfarm test-coverage artifact involved.  The bulk
of the HEAD reports are from dory and walleye, neither of which are
building v13 as yet, because of unclear problems [1].  I think those
two animals build much more frequently than our other Windows animals,
too, so the fact that they have more may be just because of that and
not because they're somehow more susceptible.  Because of that, I'm not
sure that we have enough evidence to say that v13 is better than HEAD.
If there is some new bug, it's post-v12, but maybe not post-v13.  But
v12 is evidently not perfect either.

The failures on HEAD (on dory and walleye) need another investigation
(maybe they are caused by modified stat()...).
To revive this topic and make sure that the issue still persists, I've searched through the buildfarm logs (for HEAD and REL_13_STABLE) and collected the following failures of the misc_functions test with the "Permission denied" error:

bowerbird
HEAD (13 runs last week, 500 total, 18 failed)
-

REL_13_STABLE (4 runs last week, 237 total, 8 failed)
-

drongo
HEAD (25 runs last week, 500 total, 24 failed)
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=drongo&dt=2020-12-27%2019%3A01%3A50
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=drongo&dt=2021-01-18%2001%3A01%3A55

REL_13_STABLE (5 runs last week, 255 total, 2 failed)
-

hamerkop
HEAD - (8 runs last week, 501 total, 20 failed)
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=hamerkop&dt=2021-02-22%2010%3A20%3A00

fairywren
HEAD - (27 runs last week, 500 total, 28 failed)
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=fairywren&dt=2021-02-22%2012%3A03%3A35
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=fairywren&dt=2021-01-28%2000%3A04%3A50
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=fairywren&dt=2021-01-19%2023%3A29%3A38
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=fairywren&dt=2021-01-17%2018%3A08%3A11
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=fairywren&dt=2020-11-30%2009%3A57%3A15
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=fairywren&dt=2020-11-19%2006%3A02%3A22
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=fairywren&dt=2020-11-18%2018%3A02%3A04
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=fairywren&dt=2020-09-16%2019%3A48%3A21

REL_13_STABLE (5 runs last week, 302 total, 5 failed)
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=fairywren&dt=2021-02-26%2003%3A03%3A42
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=fairywren&dt=2020-10-20%2007%3A50%3A44
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=fairywren&dt=2020-09-10%2006%3A02%3A12
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=fairywren&dt=2020-09-04%2019%3A58%3A39

woodlouse
HEAD (33 runs last week, 500 total, 5 failed)
-

REL_13_STABLE (4 runs last week, 325 total, 1 failed)
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=woodlouse&dt=2021-01-20%2011%3A42%3A30

dory
HEAD (52 runs last week, 500 total, 25 failed)
-

whelk
HEAD (31 runs last week, 500 total, 11 failed)
-

REL_13_STABLE (7 runs last week, 224 total, 1 failed)
-
(I've not considered lorikeet, jacana, and walleye as they are using alternative compilers.)
So on some machines there wasn't a single failure (probably it depends on hardware), but where it fails, such failures account for a significant share of all failures.
I believe that the patch attached to [1] should fix this issue. The patch still applies to master and makes the demotest (attached to [2]) pass. Also I've prepared a trivial patch that makes pgwin32_open() use the original stat() function (as in the proposed change for _pgstat64()).

https://www.postgresql.org/message-id/979f4ee6-9960-e37f-f3ee-f458e87777b0%40gmail.com
https://www.postgresql.org/message-id/c3427edf-d7c0-ff57-90f6-b5de3bb62709%40gmail.com

Best regards,
Alexander
--------------38CD6CFA3F1BE71B6DDD7B51-- --------------13D8790EF69A818C69C27BC9 Content-Type: text/x-patch; charset=UTF-8; name="use-microsoft_native_stat-in-pgwin32_open.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="use-microsoft_native_stat-in-pgwin32_open.patch" diff --git a/src/port/open.c b/src/port/open.c index 14c6debba9..4bd11ca9d9 100644 --- a/src/port/open.c +++ b/src/port/open.c @@ -157,9 +157,9 @@ pgwin32_open(const char *fileName, int fileFlags,...) { if (loops < 10) { - struct stat st; + struct microsoft_native_stat st; - if (stat(fileName, &st) != 0) + if (microsoft_native_stat(fileName, &st) != 0) { pg_usleep(100000); loops++; --------------13D8790EF69A818C69C27BC9--