Get-ChildItem -Recurse treats literals passed to -Path like wildcard expressions

The 1st positional argument passed to Get-ChildItem binds to -Path, meaning that wildcard expressions are supported.

However, values that are literal paths - values that do not contain wildcard metacharacters such as * - should be treated as such.

Without -Recurse, that is already the case: A nonexistent literal path triggers an error.
With -Recurse:
- The nonexistent path (assuming that it is a mere filename or its parent path exists) is treated like a wildcard expression that happens to match nothing.
- Additionally, this matching is apparently performed in all the directories in the subtree.

Workaround (and generally the better choice if you know a path to be a literal one):
Get-ChildItem -Recurse -LiteralPath <path>

Update: Get-ChildItem -Recurse <name> may also yield false positives, by matching <name> anywhere in the subtree rather than just in the current location - see below.

Steps to reproduce

Get-ChildItem NoSuch
Get-ChildItem NoSuch -Recurse

Expected behavior

Both commands should report an error re nonexistent item NoSuch.

Actual behavior

Get-ChildItem NoSuch -Recurse produces no output at all (neither success nor error), and $? is set to $True.

Environment data

PowerShell Core v6.0.0-rc.2 (v6.0.0-rc.2) on Microsoft Windows 7 Enterprise  (64-bit; v6.1.7601)
Windows PowerShell v5.1.14409.1012 on Microsoft Windows 7 Enterprise  (64-bit; v6.1.7601)

The inconsistent behavior is wrong, but with the -literalpath workaround and checking the returned count, I'm not sure if we would change this. It seems to me that if this was v1, I would have -Path and -LiteralPath be the root of the search and require -Filter for filtering rather than mixing it with the path.

I understand that the behavior is useful if the only / last path component is an actual wildcard pattern (e.g., Get-ChildItem *.txt -Recurse - though it gets confusing with Get-ChildItem /tmp/*.txt -Recurse), but with literal components you not only may get quiet no-ops when there should be an error, you may also get false positives, which is more problematic:

> mkdir -p /tmp/sub/tmp2; sl /tmp; Get-ChildItem -Recurse tmp2  | % fullname
/tmp/sub/tmp2

Note how tmp2 is unexpectedly matched anywhere in the subtree of /tmp instead of complaining that /tmp has no child item named tmp2.

Needless to say, this can have grave consequences.

It's fine to keep the existing behavior with actual wildcard expressions in the last path component, but my vote is to fix the behavior with literals.

@PowerShell/powershell-committee discussed this and is concerned about the compatibility risk with existing scripts. We acknowledge that the behavior is surprising and would defer to potentially writing a new FileSystemProvider in the future to resolve these types of issues. Feel free to mention me on other ones that should be addressed in FileSystemProvider v2.

To the developer for this issue,
I noticed this issue is relevant to this PR: #5896

Work on a new FileSystemProvider should probably wait until all the changes to System.IO in .NET core is shipped. They are doing lots on high perf/low allocation wrt file system. Can barely wait! :)

See for example dotnet/designs#24

It works as expected in v2. As far as I remember, per the PowerShell Team explanation, this was a design decision in order to match CMD dir behaviour and improve the user experience. There was an issue Microsoft Connect 766100 but half of Connect issues were removed, including this one.

Roman Kuzmin 1967-2026 (@nightroman)

@PowerShell/powershell-committee - Roman Kuzmin 1967-2026 (@nightroman) 100% is correct. This is a deliberate feature implemented to make it easier (and more friendly to cmd users) to recursively search for a file name. It is extremely useful, likely broadly used and consequently extremely likely to break both scripts and the user experience if it is changed.

If the consensus is that the risk of breaking existing scripts is too high, the best we can do is to document the pitfalls - and the pit can be deep, as my example with the false positives shows.

But let's be clear that while the ability to match on all levels of the subtree is indeed very useful, it is regrettable that such a treacherous implementation was chosen (even if as a nod to cmd users) - all the more so, given that the correct implementation of the feature is also available (-Filter, -Include).

This is a extremely dangerous when used as such:

Get-ChildItem -Path "C:\DoesNotExist" -Recurse | ForEach-Object { Remove-Item -Path $_.FullName -Recurse }

This will enumerate all files on the C drive and start deleting them, where my, what I think is reasonable, expectation, is that it would try to enumerate the files in the specified path and if none exist then quit immediately not begin trawling through every file.

It seems that if we know the path is literal (c:\foo or even .\foo) we shouldn't treat that as a search and that might be acceptable from a compatibility point of view

The code which implement the behavior is here

Comments in the code says the current behavior is explicit design:
"If we're recursing, do some path fixups to match user expectations:"

There are two base scenario:

Get-ChildItem -Recurse searchname (searchname is not an absolute or relative path)
Get-ChildItem -Recurse searchpath (searchpath is \searchname)

Let's see what is real user expectations for these scenarios.

Using simple search name with Recurse parameter says that we want to find the name in all directory tree. The expectation looks very native.
If I use a path I expect it works as path
Before the issue I didn't even know dir -Recurse c:\name is transforming into dir -Recurse c:\ -Include "name". The behavior is unexpected for me. If I came across this, I would consider it a bug and use a final slash to fix the situation dir -Recurse c:\name\.

It is clear why this design arose - this is a generalization of the first scenario - if an user is on C:\ and want to search on D:\ the user uses dir -Recurse c:\name instead of dir -Recurse name.
The key here is that the user must explicitly know how it works. I could speculate that no one knows about this since it is not documented and you can only find out about it from the comments in the code. This is an incredible situation.

Also as said above #5699 (comment) the design (for second scenario) has a dangerous side effect because the behavior differ if is existence container or no. Thus, if an user create a script with confidence that the path exists the script works well until somebody removes the directory, after that, the script will unexpectedly enumerate whole directory tree which can lead to disastrous consequences.

That we see for second scenario:

unexpected
dangerous
users don't know the design and therefore cannot rely on it

Thus, we can conclude that:

this should be fixed
this is not a breaking change
if you still believe it is a breaking change it is a breaking change under Bracket 3.

Ilya (@iSazonov)

Hear, hear, Ilya (@iSazonov).

I think that even letting Get-ChildItem name -recurse - i.e. searching by literal item name (or relative path) only (bound to -Path by default) - in the entire subtree is ultimately too treacherous, which is why I originally suggested that only wildcard patterns (e.g., name*.txt) trigger this at-every-level-of-the-subtree matching.

Ultimately, however, I think that even wildcard patterns shouldn't trigger this behavior, to avoid everlasting confusion and prevent unexpected, potentially destructive results.

Instead, I think the fix should be:

Only ever treat -Path arguments as recursion starting points (the way -LiteralPath arguments are already treated).
- That is, an argument - if it contains wildcards - should be globbed as-is (which in the name-only and relative-path cases means relative to the current directory only), and recursion should then only be performed on the resulting, literal path(s).
For at-every-level-of-the-subtree matching, -Filter or -Include must be used.
- Given that -Filter is the second positional argument, that is not much of a hardship: gci . name*.txt -r

Yes, that is more of a breaking change than the previously proposed fixes, but to me is justified by:

then having consistent behavior that is easily conceptualized and documented.
avoiding unexpected, potentially destructive behavior.

Also, Steve Lee (@SteveL-MSFT), quoting from the committee decision above:

would defer to potentially writing a new FileSystemProvider in the future to resolve these types of issues.

Can you please clarify what that means in practical terms? Different cmdlet names? ...?

would defer to potentially writing a new FileSystemProvider in the future to resolve these types of issues.

Can you please clarify what that means in practical terms? Different cmdlet names? ...?

In threory, we should convert FileSystemProvider in an loadable PowerShell subsystem. But I think it is impossible since most of issues is not in FileSystem class only but in Globber, cmdlets and other parts - we would have to rewrite whole PowerShell :-)

Yes, that is more of a breaking change than the previously proposed fixes, but to me is justified by:

It is clear from the previous discussion that it is unbelievable it will even be approved, not just implemented.

I'd suggest to open new issue for second scenario from my previous post. I suppose the removal of this unpleasant side effect will be approved quickly.

we would have to rewrite whole PowerShell :-)

Great idea! I have a few issues lined up in #6745 that we could then address too. (Just kidding.)
That is unfortunate.

Personally, I'm only interested in the fundamental fix proposed above.

So I'll leave it to you or someone else to write up a new issue for the partial fix you propose.

If you won't implement a breaking change (just think, if you'd bitten the bullet when this first issue was raised...waiting is only compounding the issue further), please do something to prevent users from accidentally deleting all their data. I nearly just destroyed our entire fleet. This is madness.

(PS 7.1.3)

FWIW, the behavior of using recursion with a literal path (no wildcards) is not without precedent. You get the same behavior in MSDOS or Windows CMD.

Here is the CMD example:

D:\temp>tree test /F
Folder PATH listing for volume DATA
Volume serial number is 6466-2168
D:\TEMP\TEST
│   fizzbuzz.txt
│
├───buzz
│       buzz.txt
│       fizzbuzz.txt
│
└───fizz
        buzz.txt

D:\temp>dir /s test\fizzbuzz.txt
 Volume in drive D is DATA
 Volume Serial Number is 6466-2168

 Directory of D:\temp\test

11/15/2023  02:13 PM                10 fizzbuzz.txt
               1 File(s)             10 bytes

 Directory of D:\temp\test\buzz

11/15/2023  02:11 PM                10 fizzbuzz.txt
               1 File(s)             10 bytes

     Total Files Listed:
               2 File(s)             20 bytes
               0 Dir(s)  1,915,708,940,288 bytes free

Here is PowerShell (7.4):

PS> Get-ChildItem -Path .\test\fizzbuzz.txt -Recurse


    Directory: D:\temp\test\buzz


Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
-a----        11/15/2023   2:11 PM             10 fizzbuzz.txt


    Directory: D:\temp\test


Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
-a----        11/15/2023   2:13 PM             10 fizzbuzz.txt

This issue has not had any activity in 6 months, if there is no further activity in 7 days, the issue will be closed automatically.

Activity in this case refers only to comments on the issue. If the issue is closed and you are the author, you can re-open the issue using the button below. Please add more information to be considered during retriage. If you are not the author but the issue is impacting you after it has been closed, please submit a new issue with updated details and a link to this issue and the original.

Navigation Menu

Repository navigation

Get-ChildItem -Recurse treats literals passed to -Path like wildcard expressions #5699

Steps to reproduce

Expected behavior

Actual behavior

Environment data

14 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Get-ChildItem -Recurse treats literals passed to -Path like wildcard expressions #5699

Description

Steps to reproduce

Expected behavior

Actual behavior

Environment data

Activity

SteveL-MSFT commented on Dec 16, 2017

mklement0 commented on Dec 17, 2017

SteveL-MSFT commented on Jan 3, 2018

marckassay commented on Jan 29, 2018

powercode commented on Jan 30, 2018

nightroman commented on Jan 30, 2018

BrucePay commented on Mar 16, 2018

mklement0 commented on Mar 17, 2018

Atheuz commented on Apr 12, 2018

SteveL-MSFT commented on Apr 12, 2018

14 remaining items

iSazonov commented on Apr 21, 2021

mklement0 commented on Apr 21, 2021

iSazonov commented on Apr 22, 2021

iSazonov commented on Apr 22, 2021

mklement0 commented on Apr 22, 2021

whitTech commented on Aug 11, 2021

sdwheeler commented on Nov 15, 2023

microsoft-github-policy-service commented on May 14, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions