Repository navigation
Native globbing doesn't respect bypassing via the escape character (backtick) in unquoted arguments #15959
Description
Prerequisites
- Write a descriptive title.
- Make sure you are able to repro it on the latest released version
- Search the existing issues.
- Refer to the FAQ.
- Refer to Differences between Windows PowerShell 5.1 and PowerShell.
Steps to reproduce
When calling external programs on Unix-like platforms, native globbing can be bypassed on demand with single- or double-quoting an argument; e.g., /bin/echo '*'
The same should work with selectively escaping globbing metacharacters with ` in unquoted arguments, but currently doesn't.
# On Unix
/bin/echo `*Expected behavior
*Actual behavior
The names of all files and directories are output, because globbing was unexpectedly still applied.Error details
No response
Environment data
PowerShell Core 7.2.0-preview.8 on Unix-like platforms.Visuals
No response
Activity
237dmitry commentedon Aug 20, 2021
Why expected behavior is
*output? In bashecho *outputs string with all files and directories in current location (space separated). In pwsh/bin/echo `*outputs string of unhidden files and directoriesmklement0 commentedon Aug 20, 2021
Because
`*is an escaped*character, to be used verbatim - just like'*', which does bypass native globbing.237dmitry commentedon Aug 20, 2021
Escaping is not equivalent quoting. You pass the escaped character to receive it as is. So that it is not lost or it is not interpreted in some other way. So
/bin/echogets everything as it is. Maybe I'm wrong, but that's how I understand this.mklement0 commentedon Aug 20, 2021
Yes, escaping is equivalent to quoting - just two different ways of achieving the same effect; compare
Write-Output `|toWrite-Output '|'That's precisely the point, and passing
*as-is means just that: the external executable/bin/echoshould receive*verbatim and therefore also print it verbatim.Note that no (native) shell is involved in such a call.
It is PowerShell that emulates a feature of POSIX-compatible shells - hence the name native globbing - and the external utility being invoked sees the results of that. And if I'm telling PowerShell to treat a character verbatim, it should therefore not apply this emulation - irrespective of the syntax form used.
237dmitry commentedon Aug 20, 2021
Perhaps the
*character is a special character for/bin/echo, which should еxplicitly be in quotation marks.mklement0 commentedon Aug 20, 2021
No,
*has no special meaning to/bin/echo.This is purely about PowerShell respecting the explicitly signaled intent to treat a character verbatim, which if unquoted or unescaped would have special meaning (a situational metacharacter that only acts a such when (a) calling external programs and (b) on Unix-like platforms only).
(The same applies to
?and[.)mklement0 commentedon Aug 20, 2021
And, to be clear, programs on Unix receive arguments as an array of verbatim values - they have no information about what particular shell with what particular quoting/escaping syntax was used to translate a command line into that array of verbatim values. For instance,
/bin/echo `|,/bin/echo '|'and/bin/echo "|"all result in the same invocation of/bin/echo, namely with an argument array whose 2nd element contains verbatim|(the 1st element is the executable name or path itself).vexx32 commentedon Sep 3, 2021
WG-Engine discussed this issue in today's meeting.
Our opinion is that this issue with wildcards is not exclusive to the native command globbing behaviour — although, yes, the ability to use quotes to quietly bypass the globbing behaviour is specific to this experimental feature.
Given that there is currently a workaround that will likely suffice for most use cases, which Michael Klement (@mklement0) outlines in the initial post, we would prefer effort be spent investigating more complete fixes for the issues with wildcard escaping that currently exist, rather than trying to design exceptions to the wildcard handling specifically for this feature. If a robust solution can be found to that issue (see #7999; there is also some relevant discussion in #7399), it would become trivial to incorporate the solution into the behaviour of this feature as well.
Ideally the functionality of this feature should match the behaviour of wildcard globbing as implemented in provider cmdlets.
Given that, we don't think this specific issue makes sense to fix in isolation, and it would be likely to result in additional confusion when such a workaround would subsequently fail to work with wildcards in other areas of pwsh.
vexx32 commentedon Sep 5, 2021
Resolving this one as a duplicate of #7999 as the root cause is ultimately the wildcard handling.
mklement0 commentedon Sep 6, 2021
Just to note an important distinction, Rain Sallow (/u/ta11ow) (@vexx32):
Backtick escaping inconsistent #7999 is about how cmdlets / providers treat wildcard expressions that they see after the parameter binder has bound arguments. That is,
Get-Item *,Get-Item `*(the bareword parsing eats the`),Get-Item '*', andGet-Item "*"are - and should be - equivalent, becauseGet-Itemsees verbatim*in all cases.By contrast, we're talking about an intentional anomaly when calling external programs on Unix, in service of emulating a different shell's behavior: as in POSIX-compatible shells, what matters is whether the argument is quoted / has individual metacharacters escaped (e.g., from Bash, compare
echo *withecho \*/echo '*'/echo "*"), in which case no globbing must be applied.vexx32 commentedon Sep 6, 2021
Yeah, I'd agree that the code path here is somewhat separate to the one originally mentioned in #7999. However, the problem is likely caused by similar handling of input -- both use WildcardPattern -- and I think we were all more or less in agreement that it would counterintuitive to implement a partial fix that only addresses this issue instead of generalising the fix to also resolve the original issue for #7999.
I think it's definitely worth mentioning that in #7999 and we can incorporate this facet into the original issue as well. 🙂