Unix-like platforms, Windows with the experimental PSNativeWindowsTildeExpansion feature: Native tilde expansion unexpectedly expands tilde in variable references #21323

Activity

1. Michael Klement (mklement0)

   added

   Needs-TriageThe issue is new and needs to be triaged by a work group.The issue is new and needs to be triaged by a work group.

   on Mar 9, 2024
2. MatejKafka commented on Mar 10, 2024

   

   Matěj Kafka (MatejKafka)

   on Mar 10, 2024

   Contributor

   More actions

   Problem is that when you don't expand the ~ when calling native command with variable arguments, ~ stops working for PowerShell functions which internally call native commands, which might be surprising. As much as I despise the behavior, I see the rationale for the current design.
3. mklement0 commented on Mar 10, 2024

   

   Michael Klement (mklement0)

   on Mar 10, 2024

   Last edited by mklement0

   ContributorAuthor

   More actions

   Matěj Kafka (@MatejKafka), there's an unresolvable tension when different worlds collide, and I understand that there's no ideal solution here:

   Introducing POSIX-like shell expansions invariably introduces a concept that is foreign to PowerShell: that whether characters such as ~ and * are individually escaped / inside quoted strings or not matters.

   To PowerShell itself, these distinctions do not matter, given that ~ and * are not metacharacters at the shell level - they are passed as-is to the target PowerShell command, which may or may not choose to assign special meaning to them - such as an initial ~ referring to the underlying provider's home location in provider cmdlets, and * representing a wildcard metacharacter in arguments passed to parameters that happen to support wildcard expressions.

   If PowerShell were to strictly adhere to this model, neither ls ~ nor ls *.txt would work on Unix-like platforms, given that the /bin/ls utility neither knows what ~ refers to nor can locate a file-system item literally named *.txt.

   For a more seamless integration of external utilities (aka native programs) on Unix, PowerShell (Core) has from the beginning decided to emulate these specific shell expansion (tilde expansion and pathname expansion (globbing)), which invariably entailed introducing the aforementioned foreign concept into PowerShell; without it, you couldn't distinguish between - say - using *.txt to mean "all files in the current directory with extension .txt" and verbatim *.txt

   These concepts are more foreign to Windows users:

   • With respect to pathname expansion (globbing), the issue doesn't arise: this emulation can never be introduced on Windows, as it would break backward compatibility: external utilities on Windows that rely on globbing must perform it themselves, and PowerShell suddenly doing it for them and passing multiple arguments that result from globbing, say, *.txt could break such utilities.

   • However, tilde expansion is less foreign: ~ does have established semantics in PowerShell (but not Windows in general, and not in cmd.exe): it refers to a provider's home location, which notably doesn't necessarily mean the file-system provider's home location, but that's a separate issue: in the context of external utilities, it only ever makes sense to interpret ~ as the file-system's home location, i.e. as $HOME.

     • As such, extending these semantics to external utilities is a more natural extension - even though the escaping/quoting part isn't.
     • The awkwardness comes from needing to understand the latter part, which includes knowing that unquoted ~ therefore cannot be used in variable values, because PowerShell inherently doesn't allow defining string variables without quoting.
       • By contrast, in POSIX-like shells, you can use unquoted values in variable assignments, and therefore take advantage of the usual shell expansions; e.g., p=~ is equivalent to p="$HOME"
       • You cannot do the same in PowerShell, and therefore need to know that ~ in argument mode, when calling external utilities, is the same as $HOME, so in variable assignments $p = $HOME is required.
4. Steve Lee (SteveL-MSFT)

   added

   WG-NeedsReviewNeeds a review by the labeled Working GroupNeeds a review by the labeled Working Group

   WG-Languageparser, language semanticsparser, language semantics

   on Mar 11, 2024