WildcardPattern.Escape() does not escape lone backticks correctly

Prerequisites

Write a descriptive title.
Make sure you are able to repro it on the latest released version
Search the existing issues.
Refer to the FAQ.
Refer to Differences between Windows PowerShell 5.1 and PowerShell.

Steps to reproduce

WildcardPattern.Escape() does not escape `, and as a result it's unreliable for escaping input for use in wildcard comparisons:

Define a string literal containing a single backtick character:

$string = 'abc`def'

Create an escaped copy of the string value:

$escaped = [WildcardPattern]::Escape($string)

Perform wildcard comparison and observe that it does not yield $true as expected:

$string -like $escaped

(That is: [WildcardPattern]::Escape($string) should produce a string value equivalent to 'abc``def', not return it as-is)

Expected behavior

PS> $string -like $escaped
True

Actual behavior

PS> $string -like $escaped
False

Error details

No response

Environment data

Name                           Value
----                           -----
PSVersion                      7.1.4
PSEdition                      Core
GitCommitId                    7.1.4
OS                             Microsoft Windows 10.0.19042
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

Visuals

No response

Wildcards in LIKE meant the DOS wildcard chars. Things like ?,*,[] right? I doubt back-tick is considered a wildcard needing escaping.

I'm aware that ` isn't a wildcard char in DOS, but PowerShell isn't DOS.

If it doesn't need escaping, then you'd expect 'a`b' -like 'a`b' to work, no?

Yeah it means "treat the next character as a literal" regardless of whether the next character would otherwise be treated as a literal or a wildcard.

right. but my feeling is that it's not the [WildcardPattern] method at fault, but the -LIKE operation. It is broken.

I mean... both WildcardPattern.Escape() and -like are wildcard methods (-like maps to WildcardPattern.IsMatch() from memory). So while one is clearly broken, it's still WildcardPattern that's broken, either way.

And you can't really change the wildcard behavior without it being a rather large breaking change. Fixing the escape method should be bucket 3 I think.

Bruce Payette (@bpayette)

/cc Bruce Payette (@bpayette)

Engine WG discussed this today, we spent a little time digging into the code to try to figure out what's going on. We think the crux of the issue is likely to be that we're doing essentially the same check two slightly different ways, depending on which API you're calling for WildcardPattern.

The first is here (note that backtick is included here):

PowerShell/src/System.Management.Automation/engine/regex.cs

Line 64 in 4ae539f

    
           private static readonly char[] s_specialChars = new[] { '*', '?', '[', ']', '`' };

And the second is here:

PowerShell/src/System.Management.Automation/engine/regex.cs

Lines 401 to 404 in 4ae539f

    
           private static bool IsWildcardChar(char ch) 
        
           { 
        
               return (ch == '*') || (ch == '?') || (ch == '[') || (ch == ']'); 
        
           }

The latter is used by Escape(), along with a few other places, while the former is used when constructing the pattern object. There needs to be more investigation and testing, but it looks like the issue here may be caused by that discrepancy.

Ilya (@iSazonov)

Ilya (@iSazonov) and myself have looked into it a bit more and it should be as simple as remove backtick from SpecialChars. With recent PR change it seems to be working.

Mathias R. Jessen (@IISResetMe) When you get time would appreciate a review, this one was a funny one 😄.

I've realised this as well seems to break for Argument completers which rely on WildcardPattern.IsMatch in C#. Specifically for completers which want to complete CRLF.

XUnit Repro:

[Theory]
[InlineData("`r*", "`r`n", true)]
public void TestIsMatch_String(string pattern, string input, bool result)
{
    Assert.Equal(result, WildcardPattern.Get(pattern, WildcardOptions.IgnoreCase).IsMatch(input));
}

Result:

[xUnit.net 00:00:00.70]     PSTests.Parallel.WildcardPatternTests.TestIsMatch_String(pattern: "`r*", input: "`r`n", result: True) [FAIL]
[xUnit.net 00:00:00.70]       Assert.Equal() Failure: Values differ
[xUnit.net 00:00:00.70]       Expected: True
[xUnit.net 00:00:00.70]       Actual:   False

If you remove backtick from SpecialChars this works as expected.

PowerShell on the other hand does not seem to be broken:

> [WildcardPattern]::Get("`r*", [System.Management.Automation.WildcardOptions]::IgnoreCase).IsMatch("`r`n")
True

So I am confused 😄.

So I am confused 😄.

Single quoted vs double quoted?

Mathias R. Jessen (@IISResetMe)

Mathias R. Jessen (@IISResetMe) Do you want to pull PR with fix?

Mathias R. Jessen (@IISResetMe)

Coming back to this one, I believe just escaping directly in the WildardPattern.Escape() method was the right solution: ArmaanMcleod@8b98336

With some Pester tests:

It "Should be <result> for backtick comparison <lhs> <operator> <rhs>" -TestCases @(
    @{ lhs = 'abc`def'; operator = '-like'; rhs = 'abc`def'; result = $false }
    @{ lhs = 'abc`def'; operator = '-like'; rhs = 'abc``def'; result = $true }
    @{ lhs = 'abc`def'; operator = '-like'; rhs = 'abc````def'; result = $false }
    @{ lhs = 'abc``def'; operator = '-like'; rhs = 'abc````def'; result = $true }
    @{ lhs = 'abc`def'; operator = '-like'; rhs = [WildcardPattern]::Escape('abc`def'); result = $true }
    @{ lhs = [WildcardPattern]::Escape('abc`def'); operator = '-eq'; rhs = 'abc``def'; result = $true }
    @{ lhs = [WildcardPattern]::Escape('abc`def'); operator = '-eq'; rhs = 'abc`def'; result = $false }
    @{ lhs = [WildcardPattern]::Escape('abc``def'); operator = '-eq'; rhs = 'abc````def'; result = $true }
 ) {
    param($lhs, $operator, $rhs, $result)
    $expression = "'$lhs' $operator '$rhs'"
    Invoke-Expression $expression | Should -Be $result
}

And results:

    [+] Should be $false for backtick comparison abc`def -like abc`def 3ms
    [+] Should be $true for backtick comparison abc`def -like abc``def 1ms
    [+] Should be $false for backtick comparison abc`def -like abc````def 1ms
    [+] Should be $true for backtick comparison abc``def -like abc````def 2ms
    [+] Should be $true for backtick comparison abc`def -like abc``def 1ms
    [+] Should be $true for backtick comparison abc``def -eq abc``def 13ms
    [+] Should be $false for backtick comparison abc``def -eq abc`def 14ms
    [+] Should be $true for backtick comparison abc````def -eq abc````def 11ms

Mathias R. Jessen (@IISResetMe) Let me know if you see any problems with above 🙂.

@{ lhs = [WildcardPattern]::Escape('abc`def'); operator = '-eq'; rhs = 'abc``def'; result = $true }

I wonder how it is true.

Ilya (@iSazonov)

Ilya (@iSazonov) [WildcardPattern]::Escape('abc`def') would just turn into abc``def after escaping, and then it's just a direct -eq comparison which would return true.

Currently this doesn't work since WildcardPattern.Escape is not escaping backticks so it returns false.

But right side is WildcardPattern where backtick is special char and effective right string will be 'abc`def' vs effective left string 'abc``def'.

Armaan Mcleod (@ArmaanMcleod)

But right side is WildcardPattern

No, -eq will treat it as a normal string literal with backticks in it

Armaan Mcleod (@ArmaanMcleod) lgtm, although I'd double up on test cases where the result of Escape is passed as RHS (since that's specifically what we're aiming to correct here)

But right side is WildcardPattern

No, -eq will treat it as a normal string literal with backticks in it

So such tests useless in this context. They could be separate, although I don't see much point in that.

We need tests specifically for [WildcardPattern]::Escape() and [WildcardPattern]::UnEscape()

So such tests useless in this context.

Fair point, agreed

In that case it might be better just keeping the -like tests in Pester and the Escape/UnEscape tests in XUnit which test the conversion.

Ill create a new PR anyways and we can discuss there.

Yes, it males sense to add 1-2 tests for -like in Pester and full set of xUnit tests for Escape and UnEscape methods.

Mathias R. Jessen (@IISResetMe)

📣 Hey Mathias R. Jessen (@IISResetMe), how did we do? We would love to hear your feedback with the link below! 🗣️

🔗 https://aka.ms/PSRepoFeedback

Microsoft Forms

Navigation Menu

Repository navigation

WildcardPattern.Escape() does not escape lone backticks correctly #16306

Prerequisites

Steps to reproduce

Expected behavior

Actual behavior

Error details

Environment data

Visuals

14 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	private static bool IsWildcardChar(char ch)
	{
	return (ch == '*') \|\| (ch == '?') \|\| (ch == '[') \|\| (ch == ']');
	}

WildcardPattern.Escape() does not escape lone backticks correctly #16306

Description

Prerequisites

Steps to reproduce

Expected behavior

Actual behavior

Error details

Environment data

Visuals

Activity

lukeb1961 commented on Oct 27, 2021

IISResetMe commented on Oct 27, 2021

SeeminglyScience commented on Oct 27, 2021

lukeb1961 commented on Oct 27, 2021

vexx32 commented on Oct 28, 2021

SeeminglyScience commented on Oct 28, 2021

vexx32 commented on Oct 28, 2021

vexx32 commented on Oct 28, 2021

14 remaining items

ArmaanMcleod commented on Jan 30, 2025

ArmaanMcleod commented on Feb 10, 2025

iSazonov commented on Feb 10, 2025

iSazonov commented on Mar 18, 2025

ArmaanMcleod commented on Mar 19, 2025

iSazonov commented on Mar 19, 2025

ArmaanMcleod commented on Mar 19, 2025

iSazonov commented on Mar 19, 2025

IISResetMe commented on Mar 19, 2025

iSazonov commented on Mar 19, 2025

IISResetMe commented on Mar 19, 2025

ArmaanMcleod commented on Mar 19, 2025

iSazonov commented on Mar 20, 2025

microsoft-github-policy-service commented on Mar 25, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions