Subject: Re: add non-option reordering to in-tree getopt_long

From: Kyotaro Horiguchi <[email-masked]>
Date: Tue, 13 Jun 2023 12:00:01 +0900 (JST)

At Fri, 9 Jun 2023 16:22:57 -0700, Nathan Bossart <[email-masked]> wrote in 
> While working on 2dcd157, I noticed cfbot failures for Windows due to tests
> with commands that had non-options specified before options.  For example,
> "createuser myrole -a myadmin" specified a non-option (myrole) before the
> option/argument pair (-a admin).  To get the tests passing for Windows,
> non-options must be placed at the end (e.g., "createuser -a myadmin
> myrole").  This same problem was encountered while working on 08951a7 [0],
> but it was again fortunately caught with cfbot.  Others have not been so
> lucky [1] [2] [3].

While I don't see it as reason to change the behavior, I do believe
the change could be beneficial from a user's perspective.

> The reason for this discrepancy is because Windows uses the in-tree
> implementation of getopt_long(), which differs from the other
> implementations I've found in that it doesn't reorder non-options to the
> end of argv by default.  Instead, it returns -1 as soon as the first
> non-option is found, even if there are other options listed afterwards.  By
> moving non-options to the end of argv, getopt_long() can parse all
> specified options and return -1 when only non-options remain.  The
> implementations I reviewed all reorder argv unless the POSIXLY_CORRECT
> environment variable is set or the "optstring" argument begins with '+'.
>
> The best reasons I can think of to keep the current behavior are 1)
> reordering involves writing to the original argv array, which could be
> risky (as noted by Tom [4]) and 2) any systems with a getopt_long()
> implementation that doesn't reorder non-options could begin failing tests
> that take advantage of this behavior.  However, this quirk in the in-tree
> getopt_long() is periodically missed by hackers, the only systems I'm aware
> of that use it are Windows and AIX, and every other implementation of
> getopt_long() I saw reorders non-options by default.  Furthermore, C99
> omits const decorations in main()'s signature, so modifying argv might not
> be too scary.

POSIXLY_CORRECT appears to be intended for debugging or feature
validation. If we know we can always rearrange argv on those
platforms, we don't need it.  I would suggest that we turn on the new
feature at the compile time on those platforms where we know this
rearrangement works, instead of switching at runtime.

> Thus, I propose introducing this non-option reordering behavior but
> allowing it to be disabled via the POSIXLY_CORRECT environment variable.
> The attached patch is my first attempt at implementing this proposal.  I
> don't think we need to bother with handling '+' at the beginning of
> "optstring" since it seems unlikely to be used in PostgreSQL, but it would
> probably be easy enough to add if folks want it.
> 
> I briefly looked at getopt() and concluded that we should probably retain
> its POSIX-compliant behavior for non-options, as reordering support seems
> much less universal than with getopt_long().  AFAICT all client utilities
> use getopt_long(), anyway.
> 
> Thoughts?

As far as I can see, getopt_long on Rocky9 does *not* rearrange argv
until it reaches the end of the array. But it won't matter much.

regards.

-- 
Kyotaro Horiguchi
NTT Open Source Software Center

---

From: Nathan Bossart <[email-masked]>
Date: Mon, 12 Jun 2023 22:13:43 -0700

> rearrangement works, instead of switching at runtime.

I'd be okay with leaving it out wherever possible.  I'm curious whether any
supported systems do not allow this.

> until it reaches the end of the array. But it won't matter much.

Do you mean that it rearranges argv once all the options have been
returned, or that it doesn't rearrange argv at all?

-- 
Nathan Bossart
Amazon Web Services: https://aws.amazon.com

---

From: Kyotaro Horiguchi <[email-masked]>
Date: Tue, 13 Jun 2023 16:02:01 +0900 (JST)

At Mon, 12 Jun 2023 22:13:43 -0700, Nathan Bossart <[email-masked]> wrote in 
> supported systems do not allow this.

Hmm. from the initial mail, I got the impression that AIX and Windows
allow this, so I thought that we can do that for them.  While there
could be other platforms that allow it, perhaps we don't need to go as
far as extending this until we come across another platform that does.

> returned, or that it doesn't rearrange argv at all?

I meant the former. argv remains unaltered until getopt_long returns
-1. Once it does, non-optional arguments are being collected at the
end of argv.  (But in my opinion, that behavior isn't very significant
in this context..)

regards.

---

From: Nathan Bossart <[email-masked]>
Date: Tue, 13 Jun 2023 15:36:57 -0700

> far as extending this until we come across another platform that does.

Windows seems to allow rearranging argv, based upon cfbot's results.  I do
not know about AIX.  In any case, C99 explicitly mentions that argv should
be modifiable.

> end of argv.  (But in my opinion, that behavior isn't very significant

Got it.