Passing an escaped wildcard expression to -Path in file-creating cmdlets causes the escaped representation to be used as the literal filename

Note: This seems to affect all file-creating cmdlets - including > and >> - with the laudable exception of Set-Content and Add-Content.

Conceptually related: #7999

Say you want to create a file literally named file[1].txt.

In order to successfully pass this name to a -Path parameter (or -FilePath for -OutFile), it must be escaped as file`[1`].txt, because -Path parameters interpret their arguments as wildcard expressions.

While such an escaped path is accepted, the problem is that it isn't retranslated into its literal equivalent on file creation, resulting in the escaped representation being used as the filename; in the example above, you end up with a file literally named file`[1`].txt instead of the intended file[1].txt

This problem can generally be avoided by passing the filename to the -LiteralPath parameter instead, but that is not yet always an option - see #3174 - and, of course, the bug is worth fixing either way.

Steps to reproduce

Run the following Pester tests:

Describe '-Path file-creation test' {
  BeforeAll { Push-Location TestDrive:\ }
  AfterAll { Pop-Location }
  It "Escaped wildcard paths are created as their literal equivalents" {
   'hi' | Out-File -FilePath 'file`[1`].txt'
    Test-Path -LiteralPath 'file[1].txt' | Should -Be $true
    Test-Path -LiteralPath 'file`[1`].txt' | Should -Be $false
 }
}

Expected behavior

The tests should pass.

Actual behavior

The tests fail, because the actual name of the file created is literally file`[1`].txt, i.e., the escaped name.

Environment data

PowerShell Core 6.2.0 / Windows PowerShell v5.1

#9258 is related to this but I am not sure if it covers all situations.

It feels to me the right behavior for Out-File -FilePath may be:

when searching for an existing file, wildcard and escaped wildcard characters are respected, meaning Out-File -FilePath [ab].txt should be able to target the existing file a.txt, and Out-File -FilePath '[out].txt' should be able to target the existing file [out].txt.
when no existing file is found, the provided path to -FilePath should be used as literal path to create the new file, just like what New-Item does, meaning Out-File -FilePath [hello].txt should create '[hello].txt' when no existing file found that matches the wildcard pattern.
redirection operator > and >> may be better always use -LiteralPath parameter when invoking Out-File. But this will be a breaking change.

The part I don't like about this proposal is that Out-File -FilePath would target different files depending on if an existing file that matches the wildcard pattern exists or not. For example, Out-File -FilePath [ab].txt would target a.txt if that file already exists, and [ab].txt if no matching file found.

This is a general issue that exists for all cmdlets that do operations that search existing files before create new files.

#9258 is an attempt to address the issue reported here for Out-File -FilePath. But I think this is a general issue that should be fixed in a wider scope.

I have always been surprised that the behavior of the path (literalpath) parameters varies in different cmdlets. This causes an endless stream of fixes.
It would be great if the paths (literalpaths) always behaved in the same way, and any deviations were regulated by separate parameters.

It feels to me the right behavior for Out-File -FilePath may be:

when searching for an existing file, wildcard and escaped wildcard characters are respected, meaning Out-File -FilePath [ab].txt should be able to target the existing file a.txt, and Out-File -FilePath '[out].txt' should be able to target the existing file [out].txt.

when no existing file is found, the provided path to -FilePath should be used as literal path to create the new file, just like what New-Item does, meaning Out-File -FilePath [hello].txt should create '[hello].txt' when no existing file found that matches the wildcard pattern.

redirection operator > and >> may be better always use -LiteralPath parameter when invoking Out-File. But this will be a breaking change.

The part I don't like about this proposal is that Out-File -FilePath would target different files depending on if an existing file that matches the wildcard pattern exists or not. For example, Out-File -FilePath [ab].txt would target a.txt if that file already exists, and [ab].txt if no matching file found.

I believe it is not a good idea to have a single parameter handling two different usages that might conflict with each other. For example, how do we tell if a user typed an incorrect wildcard accidentally, or a literal filename intentionally? To keep the things simple, I would prefer to use the existing Path/LiteralPath concept:

Make -FilePath an alias of -Path
Add -LiteralPath to the cmdlet

As for > and >>, would it be possible to add configurable settings (e.g. environment variable) to define whether it (or even the engine itself) accepts -Path or -LiteralPath by default when it is called from console/script-file, to give users more control?

kwkam

Agreed re consistency, kwkam.

The larger issue, however, is that output file paths should never have been treated as wildcards to begin with, which takes us back to #4726, #6729, and also #9225

P.S.: The -FilePath parameters already are aliased to -Path:

Command           Name                   Aliases
-------           ----                   -------
Export-FormatData Path                   {FilePath}
Invoke-Command    FilePath               {PSPath}
Out-File          FilePath               {Path}
Set-TraceSource   FilePath               {PSPath, Path}
Start-Process     FilePath               {PSPath, Path}
Tee-Object        FilePath               {Path}
Trace-Command     FilePath               {PSPath, Path}

Note that PSPath is yet another alias for some of them, which is also problematic, because PSPath should be the alias of -LiteralPath, to ensure treatment of pipeline input from, e.g., Get-ChildItem as the literal paths that they are.

Michael Klement (@mklement0)

As Michael Klement (@mklement0) pointed in #6729 writing using wildcards to multiple destinations is dangerous.
While changing this behavior in Set-Content/Add-Content would be huge breaking change and I guess it is not for Out-File (>,>>), it is seem reasonable to switch Out-File (>,>>) to LiteralPath by default, specially it is not obvious for user that >,>> are mapped to Out-File, I guess nobody use the complex behavior in the redirection.

Also PSPath alias already points to LiteralPath in Out-File. (So if path comes from pipeline by name it binds to LiteralPath but I guess it is not mainstream scenario).

@PowerShell/powershell-committee reviewed this. The case we believe is inconsistent is out-file file``[1``].txt where you would properly expect this to be treated as literal as you are escaping the escape character. Looking at the traces, the parser unescapes it once and the parameter binder is sending file`[1`].txt to the cmdlet. The cmdlet calls into the FileSystemProvider which receives this but as it's processing the path, during normalization it appears it gets unescaped to just file[1].txt and later the code sees this as a valid wildcard pattern and does globbing.

Given that the current FileSystemProvider is sensitive to changes, we recommend that this scenario be addressed in FileSystemProvider v2 rather than trying to fix this in v1 and risk regression.

This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you.

This issue has been marked as "No Activity" as there has been no activity for 6 months. It has been closed for housekeeping purposes.

Navigation Menu

Repository navigation

Passing an escaped wildcard expression to -Path in file-creating cmdlets causes the escaped representation to be used as the literal filename #9475

Steps to reproduce

Expected behavior

Actual behavior

Environment data

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Passing an escaped wildcard expression to -Path in file-creating cmdlets causes the escaped representation to be used as the literal filename #9475

Description

Steps to reproduce

Expected behavior

Actual behavior

Environment data

Activity

kwkam commented on May 7, 2019

daxian-dbw commented on May 13, 2019

daxian-dbw commented on May 13, 2019

iSazonov commented on May 13, 2019

kwkam commented on May 14, 2019

mklement0 commented on May 22, 2019

iSazonov commented on Dec 1, 2021

SteveL-MSFT commented on Dec 1, 2021

microsoft-github-policy-service commented on Nov 16, 2023

microsoft-github-policy-service commented on Nov 16, 2023

microsoft-github-policy-service commented on Nov 16, 2023

microsoft-github-policy-service commented on Nov 23, 2023

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions