Repository navigation
invoke-webrequest and invoke-restmethod -headers parameter is more restrictive on PowerShell Core than on PowerShell Native #2895
Description
Im working with an API that seems to not comply with the accepted standard for if-match header (value string must be enclosed in quotes), although this is apparently quite common and is accommodated in native PowerShell iwr/irm. PowerShell core behaves differently and rejects the header when building the request. Modifying the API (commercial product) to comply is not an option unfortunately.
Steps to reproduce
$headers = @{}
$headers.Add("if-match","12345")
Invoke-WebRequest -Uri "http://httpbin.org/headers" -Headers $headersExpected behavior
Works on native PowerShell
Invoke-WebRequest -Uri "http://httpbin.org/headers" -Headers $headers
StatusCode : 200
StatusDescription : OK
Content : {
"headers": {
"Host": "httpbin.org",
"If-Match": "12345",
"User-Agent": "Mozilla/5.0 (Windows NT; Windows NT 10.0; en-AU) WindowsPowerShell/5.1.14393.206"
}
}
RawContent : HTTP/1.1 200 OK
Connection: keep-alive
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true
Content-Length: 182
Content-Type: application/json
Date: Fri, 16 Dec 2016 05:06:38 GM...
Forms : {}
Headers : {[Connection, keep-alive], [Access-Control-Allow-Origin, *], [Access-Control-Allow-Credentials, true], [Content-Length, 182]...}
Images : {}
InputFields : {}
Links : {}
ParsedHtml : mshtml.HTMLDocumentClass
RawContentLength : 182 Actual behavior
Fails on linux
Invoke-WebRequest -Uri "http://httpbin.org/headers" -Headers $headers
Invoke-WebRequest : The format of value '12345' is invalid.
At line:1 char:1
+ Invoke-WebRequest -Uri "http://httpbin.org/headers" -Headers $headers
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [Invoke-WebRequest], FormatException
+ FullyQualifiedErrorId : System.FormatException,Microsoft.PowerShell.Commands.InvokeWebRequestCommandNote: enclosing the value in quotes allows the header to be accepted by iwr/irm, but the API I'm attempting to consume from PowerShell Core does not accept the value when enclosed in quotes.
Environment data
Name Value
---- -----
PSVersion 6.0.0-alpha
PSEdition Core
PSCompatibleVersions {1.0, 2.0, 3.0, 4.0...}
BuildVersion 3.0.0.0
GitCommitId v6.0.0-alpha.14
CLRVersion
WSManStackVersion 3.0
PSRemotingProtocolVersion 2.3
SerializationVersion 1.1.0.1
Activity
7 remaining items
SteveL-MSFT commentedon Jun 13, 2017
Dan Travison (@dantraMSFT) can you open an issue in corefx and link it to this?
dantraMSFT commentedon Jun 13, 2017
Will do, Steve.
Nick Bradford (@nmbradford) Would you mind providing a couple of example header values and the details of what you are doing to work around it. I'd like to be specific when I create the issue.
Thanks.
nmbradford commentedon Jun 13, 2017
no prob. My initial post has steps to repro for the failure scenario. For the header to be accepted on Core, try this (with quoted string):
In order to work around it, I'm not using IWR at all any more, instead constructing the httpclient request object directly and using the DefaultRequestHeaders.TryAddWithoutValidation() method rather than the DefaultRequestHeaders.add() method that IWR is using.
dantraMSFT commentedon Jun 13, 2017
I've created an issue in the CoreFX repo: https://github.com/dotnet/corefx/issues/20994
SteveL-MSFT commentedon Jun 13, 2017
Perhaps another way to solve this would be to expose a
-SkipHeaderValidationswitch? cc Joey Aiello (@joeyaiello)dantraMSFT commentedon Jun 21, 2017
Joey Aiello (@joeyaiello) are you fine with Steve's suggestion or do you have other ideas?
joeyaiello commentedon Jun 21, 2017
I would go with
-SkipHeaderValidation, yeah. ( Steve Lee (@SteveL-MSFT) changed it from-NoHeaderValidation😉 )nmbradford commentedon Jun 22, 2017
Agree - this would be an easier workaround, but then iwr and irm would behave differently on Windows/non windows platforms (meaning scripts would not be portable without platform detection)?
SteveL-MSFT commentedon Jun 22, 2017
Nick Bradford (@nmbradford) it would behave the same on Windows and Linux as long as you're using PSCore6. We have already added new capabilities to the webcmdlets that won't be in Windows PowerShell 5.1
jimwAtMs commentedon Jan 24, 2018
I assume running 6.0 release has this fix, but passing in empty header gets a format exception, but on PS 5 it does not.
This works on 5
Invoke-WebRequest -Uri $uri -Method Post -Headers @{Authorization=""} params
On 6 it gets this error. Removing the Authorization header gets around it.
Invoke-WebRequest : The format of value '' is invalid.
markekraus commentedon Jan 24, 2018
JimW (@jimwAtMs) PowerShell Core now has strict header parsing on by default. Any headers passed to
-Headersthat are standards defined (such asAuthorization) will result in a format error if the value is not standards compliant. You can bypass header validation with-SkipHeaderValidationhttps://github.com/PowerShell/PowerShell/blob/master/docs/BREAKINGCHANGES.md#changes-to-web-cmdlets
More info: https://get-powershellblog.blogspot.com/2017/11/powershell-core-web-cmdlets-in-depth.html#L08
jimwAtMs commentedon Jan 24, 2018
Thanks Mark Kraus (@markekraus)