Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1ofBkI-000625-Ok for pgsql-hackers@arkaria.postgresql.org; Mon, 03 Oct 2022 03:03:54 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1ofBkH-0006tU-HZ for pgsql-hackers@arkaria.postgresql.org; Mon, 03 Oct 2022 03:03:53 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1ofBkH-0006tL-7o for pgsql-hackers@lists.postgresql.org; Mon, 03 Oct 2022 03:03:53 +0000 Received: from mail-ej1-x62e.google.com ([2a00:1450:4864:20::62e]) by makus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1ofBkE-0007oY-Uf for pgsql-hackers@lists.postgresql.org; Mon, 03 Oct 2022 03:03:52 +0000 Received: by mail-ej1-x62e.google.com with SMTP id rk17so19452841ejb.1 for ; Sun, 02 Oct 2022 20:03:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date; bh=XcJ2h3r+/t7n/9SLKyqQb7FNTcUJIksAM9DZW/BGIDs=; b=mOcgnmHN0AlF+IUBjN1nH2Sln7N54RtmCEmml0JOrMfyhch2Sf/qwQVQpt8FEBhyAN s9LjvrGkRkiEwxYanfIvnplFNqG8ioSFSW6jYmMWLx5CDHaLlAeS7QS4pkEht5HO16I8 sVPXOebl8LZ7x0EisHUPa2zDOvRoODE8ViIYDnkYgKADpRKwZaRj/Gb5mplxGxidOtVz 5FsXcLAa2AhfujBOe1mn2y0pPINKS6sf5/pOY/X0PBJashRGSf/vxHaPvYwjLIfXE0BF 1C5I2w6ADSeNF4ChOMKtZkc2Qe3YhdbzZazU3a9/vK1FfC2fVea1grcS/5QPeLDqy7WT oqAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date; bh=XcJ2h3r+/t7n/9SLKyqQb7FNTcUJIksAM9DZW/BGIDs=; b=U/5zdE2CA5pXAQUvK/CG8V3IgA7rQxGSiEBdmeA7Xw1mNVPl4IDIFCLnX8jQRb7h3o cBDBJzn95M+x3eDbD2DE6EY8S10x3FgsREZftKknhjnjpwuZolD1I8iXJ2EJmCi7wjpn SVA6u1iP3DNllnAcTa3b8joFmIZoJ5zQ0hnLVbvSgej0twnIrzudV5JCw33LL0vH5N4d 7GYEEtBDzSa9Wd+/mPh3D29Y5SMY01vAmZI3V9pnr3T/FRZk2X7pkmeT90xGDqqgQSDG LZ1hcunSDam6jK2gDDHSjGb06puIsgH2g4AuR7+riwilQ2y6zHRyfk4i6rnnl3ZhP/Qo nsPg== X-Gm-Message-State: ACrzQf3bVP+pjrGUU1icdpVRK3PNb1VYrq0aqyh/o8kPS9lVsf4dLv1Y cQv1RNe3Gfz/MHYLFsHJPmOTMizUOMfsvJNX87E= X-Google-Smtp-Source: AMsMyM5z2UCruH4nKjYliKTFExma2s70uy2sTh/x+wUmIoqb+EZvUG9hlxN70Iar4b4C2fztvKxoqRfRbr/Fsfg0eFo= X-Received: by 2002:a17:906:4ac1:b0:780:3448:ff06 with SMTP id u1-20020a1709064ac100b007803448ff06mr13752528ejt.403.1664766229345; Sun, 02 Oct 2022 20:03:49 -0700 (PDT) MIME-Version: 1.0 References: <20220919213217.ptqfdlcc5idk5xup@awork3.anarazel.de> <20220920013122.GA31833@telsasoft.com> In-Reply-To: From: Thomas Munro Date: Mon, 3 Oct 2022 16:03:12 +1300 Message-ID: Subject: Re: pg_upgrade test failure To: Michael Paquier Cc: Justin Pryzby , Andres Freund , pgsql-hackers@lists.postgresql.org Content-Type: text/plain; charset="UTF-8" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk On Mon, Oct 3, 2022 at 1:40 PM Michael Paquier wrote: > On Mon, Oct 03, 2022 at 12:10:06PM +1300, Thomas Munro wrote: > > I think something like the attached should do the right thing for > > STATUS_DELETE_PENDING (sort of "ENOENT-in-progress"). unlink() goes > > back to being blocking (sleep+retry until eventually we reach ENOENT > > or we time out and give up with EACCES), but we still distinguish it > > from true ENOENT so we have a fast exit in that case. This is passing > > CI, but not tested yet. > > if (lstat(path, &st) < 0) > - return -1; > + { > + if (lstat_error_was_status_delete_pending()) > + is_lnk = false; > + else > + return -1; > + } > + else > + is_lnk = S_ISLNK(st.st_mode); > Sorry, I don't remember all the details in this area, but a directory > can never be marked as STATUS_DELETE_PENDING with some of its contents > still inside, right? That's my understanding, yes: just like Unix, you can't remove a directory with something in it. Unlike Unix, that includes files that have been unlinked but are still open somewhere. (Note that in this case it's not exactly a real directory, it's a junction point, which is a directory but it doesn't have contents, it is a reparse point pointing somewhere else, so I suspect that it can't really suffer from ENOTEMPTY, but it probably can suffer from 'someone has it open for a short time because they are concurrently stat-ing it'.) > If it has some contents, forcing unlink() all > the time would be fine? Here's why I think it's probably OK to use unlink() unconditionally after detecting STATUS_DELETE_PENDING. AFAICT there is no way to even find out if it's a file or a junction in this state, but it doesn't matter: we are not waiting for rmdir() or unlink() to succeed, we are waiting for it to fail with an error other than EACCES, most likely ENOENT (or to time out, perhaps because someone held the file open for 11 seconds, or because EACCES was actually telling us about a permissions problem). EACCES is the errno for many things including STATUS_DELETE_PENDING and also "you called unlink() but it's a directory" (should be EPERM according to POSIX, or EISDIR according to Linux). Both of those reasons imply that the zombie directory entry still exists, and we don't care which of those reasons triggered it. So I think that setting is_lnk = false is good enough here. Do you see a hole in it?