Translating PSDrive-based path arguments for external (native) executables is overzealous (PSNativePSPathResolution)

Dustin L. Howett (@DHowett)

PR #12386 introduced experimental feature PSNativePSPathResolution, which as of (at least) PowerShell Core 7.1.0-preview.7 has become a default feature.

For problems with this feature in general, see Bypassing translation of PSDrive-based path arguments for external (native) executables is obscure and inadequate (PSNativePSPathResolution) #13644

Its purpose is to translate PowerShell-drive-based paths to native filesystem paths, because external executables only understand the latter.

This translation is currently overzealous in that it nonsensically translates an argument that starts with : (which clearly isn't a drive-based path).

Dustin L. Howett (@DHowett) has pinpointed the problematic behavior to:

PowerShell/src/System.Management.Automation/engine/NativeCommandParameterBinder.cs

Lines 388 to 406 in 3effa20

    
           if (path.Contains(':')) 
        
           { 
        
               LocationGlobber globber = new LocationGlobber(context.SessionState); 
        
               try 
        
               { 
        
                   ProviderInfo providerInfo; 
        
                   // replace the argument with resolved path if it's a filesystem path 
        
                   string pspath = globber.GetProviderPath(path, out providerInfo); 
        
                   if (string.Equals(providerInfo.Name, FileSystemProvider.ProviderName, StringComparison.OrdinalIgnoreCase)) 
        
                   { 
        
                       path = pspath; 
        
                   } 
        
               } 
        
               catch 
        
               { 
        
                   // if it's not a provider path, do nothing 
        
               } 
        
           }

Steps to reproduce

On Unix:

/bin/echo :foo | Should -Be ':foo'

On Windows:

cmd /c echo :foo | Should -Be ':foo'

Expected behavior

The test should succeed.

Actual behavior

The test fails, because the the current location's path is unexpectedly prepended to :

Expected strings to be the same, but they were different. Expected length: 4 Actual length:   44 Strings differ at index 0.
Expected: ':foo' But was:  '/Users/jdoe...'

Environment data

PowerShell Core 7.1.0-preview.7

The bug is not in PSNativePSPathResolution. It is in globber.GetProviderPath(). Try dir : for demo.

You're right: ~~the bug also affects~~ The behavior also applies to arguments to cmdlets that expect item paths, such as Get-ChildItem (dir).

And it also surfaces if there's no globbing involved whatsoever: Get-ChildItem -LiteralPath :.

As such, it goes back to Windows PowerShell.

[Update: : is simply interpreted as the start of a relative path.]

However, with respect to external executables it only surfaces since PSNativePSPathResolution was made a regular feature.

It is not clear how should we react on ':name' or 'name:name' in both cases.

In IsAbsolutePath() I see the comment:

                // If the : is the first character in the path then we
                // must assume that it is part of the path, and not
                // delimiting the drive name.

So for dir : scenario it is by design and for first scenario we could do the same if first char is ':' but it it is still not clear what we should do for name:name in both scenarios.

Current behavior for path like name1:name2 - throw if drive name1 is not found. Should we change this and consider name1:name2 as file/directory name (if drive name1 is not found) because we allow :name?

Dustin L. Howett (@DHowett)

I think we're missing the point here quite a bit tbh.

The problem that was originally raised on #13638 is that because we're looking for paths on all input parameters, we're breaking some use cases where an argument string is literally just thing:thing or even just :thing because PS is treating that as a relative path and it's being resolved to a full path.

Dustin L. Howett (@DHowett)'s original example was:

& git log ":(top,exclude).git/" ":(top,exclude)**/*.rec"

That's not meant to be a path in the same way that PowerShell tends to think of paths. Some of these need to passed along without being resolved at all, not as a file name/directory name or anything. With PSNativePSPathResolution a standard feature, if PS thinks it's a relative path it's resolving it to what it thinks the full path "should" be. Which means an argument like this will never be passed to the executable verbatim. Which is going to be endlessly problematic.

If we get name1:name2 and name1 isn't a recognised physical drive or PSDrive, IMO it should be passed along verbatim.

Which is going to be endlessly problematic.

It seems only reliable way to implement PSNativePSPathResolution it is to know that native application expects a path.

Rain Sallow (/u/ta11ow) (@vexx32)

If we get name1:name2 and name1 isn't a recognised physical drive or PSDrive, IMO it should be passed along verbatim.

Indeed, Rain Sallow (/u/ta11ow) (@vexx32), and that is actually how it mostly already works (except that action is only required for PSDrives, not native filesystem drives), with an argument-initial : constituting the only exception that amounts to a bug, as described in the OP.

So the bottom line is: An argument that starts with : is by definition not a path based on a PS drive and therefore no attempt should be made to translate it to a native path.

Note, however, that that doesn't necessarily mean verbatim, because on Unix globbing may still have to be performed on an (unquoted) :-prefixed argument, given that there you may have, say, a subdir named :foo containing .txt files, which should be glob-expanded by something like /bin/echo :foo/*.txt (to pass individual arguments :foo/bar.txt, ...)

I got confused by the behavior of Get-ChildItem : (and other provider cmdlets), but that is really a separate issue:

A filesystem-path-argument-initial : is interpreted as a relative path, but on Windows (unlike on Unix), file/directory names mustn't contain :, so there is no point in trying to interpret as such.

Doing so is probably not a big deal in practice, (cmd.exe similarly yields a File Not Found), but the current error message, which claims nonexistence (Cannot find path '...' because it does not exist) with the full path based on the current location is a bit confusing.

Oh, : is a valid item name on Unix? Didn't know that. I guess I assumed it wasn't considered valid because that's what the path separator usually is on Unix systems. 🤔

Ilya (@iSazonov)

It seems only reliable way to implement PSNativePSPathResolution it is to know that native application expect a path.

Ilya (@iSazonov), unfortunately, that is impossible, and the current feature is guesswork, for sure, but I suspect that it will work seamlessly in the vast majority of cases.

That said, it's important:

To document the magic in detail.
To provide a way to bypass the magic on demand.
- Currently, using single-quoting ('...') rather than double-quoting ("...") provides this bypass, but I think that is conceptually problematic and obscure and only works with literal arguments to boot.

Oh, : is a valid item name on Unix? Didn't know that. I guess I assumed it wasn't considered valid because that's what the path separator usually is on Unix systems

Rain Sallow (/u/ta11ow) (@vexx32), yes, that's surprising, but note that, similarly, ; is a valid filename character on Windows.

Incidentally, single quoting won't work for the use case that led me to discover the issue.

My original code was closer to...

& git log (Get-ExcludedLogPatterns)

...which I do not believe I can single-quote (or use --% on!)

unfortunately, that is impossible, and the current feature is guesswork, for sure,

I will reference this from #13428 native cmdlet class :-)

unfortunately, that is impossible, and the current feature is guesswork, for sure, but I suspect that it will work seamlessly in the vast majority of cases.

In the case we need an way to temporary disable the feature. It can fall us in bad history like --% and quotes.

Dustin L. Howett (@DHowett)

Dustin L. Howett (@DHowett), with additional work - which invariably involves creating an auxiliary environment variable that pollutes the child process' environment - you can use --% to work around the problem, but it is obviously cumbersome :

PS> $env:_aux_arg = ':foo'; cmd --% /c echo %_aux_arg%
:foo

Ilya (@iSazonov), implementing wrappers around all external CLIs that accept file-system paths is obviously impossible.

In the case we need a way to temporary disable the feature. It can fall us in bad history like --% and quotes.

I agree, and, as stated, the current mechanism - single-quoting - is both obscure and inadequate, due to only working with literals.

I'm not sure there's a good in-call solution, and that makes me wonder whether this feature - as helpful as it may be in many cases - is worth implementing at all.

Travis Plunk (@TravisEz13)

Travis Plunk (@TravisEz13) You say about 7.1 label here but a label in the PR is 7.0 - typo?

Looking the code I found that expansion of ~ (Home) is weird - it is on Windows only.

This should really have the option of being controlled via a variable like PSNativeCommandArgumentPassing does, so it can be toggled off temporarily without restarting the terminal.

Navigation Menu

Repository navigation

Translating PSDrive-based path arguments for external (native) executables is overzealous (PSNativePSPathResolution) #13640

Steps to reproduce

Expected behavior

Actual behavior

Environment data

13 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	if (path.Contains(':'))
	{
	LocationGlobber globber = new LocationGlobber(context.SessionState);
	try
	{
	ProviderInfo providerInfo;

	// replace the argument with resolved path if it's a filesystem path
	string pspath = globber.GetProviderPath(path, out providerInfo);
	if (string.Equals(providerInfo.Name, FileSystemProvider.ProviderName, StringComparison.OrdinalIgnoreCase))
	{
	path = pspath;
	}
	}
	catch
	{
	// if it's not a provider path, do nothing
	}
	}

Translating PSDrive-based path arguments for external (native) executables is overzealous (PSNativePSPathResolution) #13640

Description

Steps to reproduce

Expected behavior

Actual behavior

Environment data

Activity

iSazonov commented on Sep 16, 2020

mklement0 commented on Sep 16, 2020

iSazonov commented on Sep 16, 2020

iSazonov commented on Sep 16, 2020

iSazonov commented on Sep 16, 2020

vexx32 commented on Sep 16, 2020

iSazonov commented on Sep 16, 2020

mklement0 commented on Sep 16, 2020

vexx32 commented on Sep 16, 2020

mklement0 commented on Sep 16, 2020

DHowett commented on Sep 16, 2020

iSazonov commented on Sep 16, 2020

mklement0 commented on Sep 16, 2020

13 remaining items

iSazonov commented on Oct 14, 2020

iSazonov commented on Dec 15, 2021

God-damnit-all commented on Jun 19, 2022

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions