Repository navigation
PSParser Tokenize with Switch Default fails #13462
Description
Trouble when trying to Tokenize a script containing a Switch-statement with a Default case in Powershell 7.1.0-preview.5. This works as expected in Powershell 7.0.3 (stable). If one does comment-out the default case, it works as expected. This has been reproduced on more than one computer.
Steps to reproduce
Create a file containing a simple function that has a switch-statement with a default case (see below). Save that file. Then try to tokenize that file (also see below) from a console running Powershell 7.1.0-preview.5.
# Save this as a file called 'TEST.ps1'
Function Test {
Param (
$String
)
Switch ($String) {
"test" {
"sfdsf"
}
"test2" {
"sfdsf"
}
default {
"sdfsdf"
}
}
}# From a Powershell 7.1.0-preview.5 prompt execute the following:
$scripts = Get-ChildItem -Path "C:\__TEMP\TEST.ps1"
Foreach ($file in $scripts) {
$contents = Get-Content -Path $file.fullname -ErrorAction Stop
$errors = $null
$null = [System.Management.Automation.PSParser]::Tokenize($contents, [ref]$errors)
$errors.Count -eq 0
}Expected behavior
True
Actual behavior
MethodInvocationException:
Line |
4 | $null = [System.Management.Automation.PSParser]::Tokenize($conten …
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| Exception calling "Tokenize" with "2" argument(s): "Index was outside the bounds of the array."
TrueEnvironment data
Name Value
---- -----
PSVersion 7.1.0-preview.5
PSEdition Core
GitCommitId 7.1.0-preview.5
OS Microsoft Windows 10.0.19041
Platform Win32NT
PSCompatibleVersions {1.0, 2.0, 3.0, 4.0}
PSRemotingProtocolVersion 2.3
SerializationVersion 1.1.0.1
WSManStackVersion 3.0
Activity
vexx32 commentedon Aug 17, 2020
PSParser is legacy and I'm not sure it's actually supported in a meaningful way.
Instead, you can use:
My guess is the PSParser code path was probably broken with #10487 cc Steve Lee (@SteveL-MSFT) Rob Holt (@rjmholt) not sure PSParser is really supported / what would need to be done to "fix" it following that change, or whether it's worth looking into as there are fully functional and supported alternatives available.
LAltinell commentedon Aug 17, 2020
Thank you for your prompt reply!
Perfect, that works for me. Hopefully this will help someone else as well.
rjmholt commentedon Aug 17, 2020
This sounds like something we should still fix though -- the
PSParserAPI needs to be maintained for broad module compatibilityrjmholt commentedon Aug 18, 2020
Full stack trace:
rjmholt commentedon Aug 18, 2020
The problem is that this mapping hasn't been maintained:
PowerShell/src/System.Management.Automation/engine/lang/interface/PSToken.cs
Lines 110 to 307 in 4b9b078
That's ostensibly easy to fix, but:
basedoesn't seem to be in thereclasskeyword?rjmholt commentedon Aug 18, 2020
/cc Dongbo Wang (@daxian-dbw)
daxian-dbw commentedon Aug 18, 2020
Rob Holt (@rjmholt) Is this specific regression caused by #10487?
rjmholt commentedon Aug 18, 2020
I believe so
daxian-dbw commentedon Aug 18, 2020
baseseems to be working fine:[Update] I took a look at the code, and the keyword
baseworks just by accident.rjmholt commentedon Aug 18, 2020
That looks like this:
PowerShell/src/System.Management.Automation/engine/lang/interface/PSToken.cs
Line 306 in 4b9b078
daxian-dbw commentedon Aug 18, 2020
Yeah. Let's first fix this regression by simply updating
s_tokenKindMapping. And also open a new issue to discuss how to deal withPSParserin the long term to keep it up-to-date.