Tokenizer treats . as a separator when parsing a parameter token, which causes problem to argument parsing for native commands

I use powershell to run build commands like
mvn --% install -Dmaven.test.skip -Dmaven.repo.local=.m2/repository -Dbuild.deployAtEnd=true -Dgib.disable

I NEVER expect the dot to act like a property selection operator. I don't want to have to quote every time I use a dot, and I'd rather not have to specify --% for every command.

Is it reasonable to expect a setting in pwsh.runtimeconfig.json to disable parsing of command arguments? Call it dumb mode or whatever.

I see other people are also doing strange things to work around this https://stackoverflow.com/questions/18923315/using-in-powershell

Unless I'm missing something, you shouldn't need to use --% most of the time. That command you have there will work as-is without it, I'd think.

You're not the first person to suggest that, but the fact is I've never seen it work without --%, on many machines running powershell 5 or 7. There's even a difference in the syntax highlighting

What makes you think I don't need the --% ?

Yeah, there is a difference without --%:

Using --% in PS:

BTW there is some work going on WRT argument parsing in 7.2 - see PR #15408. However whether the mode is set to Standard or Legacy, PS still parses that mvn command line differently than cmd.exe does.

That feels like a bug, tbh. I don't see any good reason for the . characters to have special meaning. There aren't any variables or other special tokens involved here.

It looks . is used as a separator when tokenizing text like a parameter (e.g. -xxx)

PS:24> function bar { param($param) $param }
PS:25> bar -param.abcdef
.abcdef

Another native command argument parsing issue related to tokenizer: #15279

This seems the same as the older #6291. I fail to understand if this a confirmed bug and if this will be solved in a future version of powershell.

Francesco Pretto (@ceztko)

Good point, Francesco Pretto (@ceztko). In fact,

Parameter parsing/passing: an unquoted argument that starts with a "-" (hyphen) is broken in two at the first "." (period) #6291

is just one of several related bugs, which the following search surfaces ("Parameter parsing.passing" in the title, 8 issues as of this writing):

https://github.com/PowerShell/PowerShell/issues?q=is%3Aissue+is%3Aopen+parameter+parsing%2Fpassing+in%3Atitle

GitHub
Issues · PowerShell/PowerShell
PowerShell for every system! Contribute to PowerShell/PowerShell development by creating an account on GitHub.

I ran into this again on macOS with the updated native command passing:

g++ ./hello.cpp -target arm64-apple-macos11 -mmacosx-version-min=11.0 -o hello_arm  
clang: error: no such file or directory: '.0'

You can quote the 11.0 to get around this or use --%. The issue seems to be that -mmacosx-version-min=11 is considered a parameter name by PowerShell, followed by a .0 as a ConstantExpressionAst. Possible fix is to recognize this situation in the NativeCommandProcessor and fix up the resulting parameter passed to the native app.

Keith Hill (@rkeithhill)

Keith Hill (@rkeithhill), it's the same old bug, #6291, and it is both unrelated to and long predates $PSNativeCommandArgumentPassing, as are the other linked bugs.

The simplest workaround is to quote the whole argument: '-mmacosx-version-min=11.0'
If the token contains no spaces or other PowerShell metacharacters, `-escaping the initial - is enough:
`-mmacosx-version-min=11.0

Another one of those, "why? why" issues with PowerShell.

It's great that PowerShell has extra features and some simplifications. But why should it complicate standard commands? Like why should we change the way we invoke Java since 1997 just because we start using PowerShell? (I can't guarantee that Java has supported -D since 1997, but it's been around quite a while.)

Also, I can't use --% in commands which invoke a script. For example with this script "fast.ps1"

./mvnw clean install $args -D"surefire.skip"

And this command fast --% something

The token "--%" gets passed to Maven. So I'm forced to quote.

Delany (@delanym)

Delany (@delanym) What's your $PSNativeCommandArgumentPassing value?

Will 保哥 (@doggy8088)

Will 保哥 (@doggy8088) i just installed Powershell 7.4.1 and the value is "Windows"

https://learn.microsoft.com/en-us/powershell/scripting/learn/experimental-features?view=powershell-7.4#psnativecommandargumentpassing

It makes no difference whether this property is set to Windows, Standard or Legacy.

Using Experimental Features in PowerShell - PowerShell
Lists the currently available experimental features and how to use them.

Maybe this issue should be kept open/assigned a milestone?

Navigation Menu

Repository navigation

Tokenizer treats `.` as a separator when parsing a parameter token, which causes problem to argument parsing for native commands #15541

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Tokenizer treats . as a separator when parsing a parameter token, which causes problem to argument parsing for native commands #15541

Description

Activity

vexx32 commented on Jun 8, 2021

delanym commented on Jun 8, 2021

rkeithhill commented on Jun 8, 2021

vexx32 commented on Jun 8, 2021

daxian-dbw commented on Jun 9, 2021

ceztko commented on May 6, 2022

mklement0 commented on Sep 8, 2022

rkeithhill commented on Nov 28, 2022

mklement0 commented on Nov 28, 2022

garretwilson commented on Mar 16, 2023

delanym commented on Sep 7, 2023

doggy8088 commented on Mar 6, 2024

delanym commented on Mar 6, 2024

delanym commented on Mar 6, 2024

ceztko commented on Sep 10, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Tokenizer treats `.` as a separator when parsing a parameter token, which causes problem to argument parsing for native commands #15541