Add User Supplied ServerCertificateCustomValidationCallback Support to Web Cmdlets

Problem

Currently users only have 2 options for dealing with Server SSL/TLS Certificates with Invoke-WebRequest and Invoke-RestMethod: the default validation and to skip validation. Some scenarios warrant tighter security on web requests where a certificate is not fully trusted by the host environment but is known to be trusted by the user. This could include internal web APIs that use a self signed certificate with a specific thumbprint or from a known CA that is not trusted by the host. Or if a user wishes to ensure a certain CA/Thumbprint/Subject is blocked (a known bad actor).

Also [System.Net.ServicePointManager]::ServerCertificateValidationCallback has no effect in Core and HttpClient only uses the settings provided by HttpClientHandler.

Proposal

Add a parameter of type Func<HttpRequestMessage,X509Certificate2,X509Chain,SslPolicyErrors,Boolean> that accepts a ScriptBlock to both Web Cmdlets. This is to be set on HttpClientHandler.ServerCertificateCustomValidationCallback. The -SkipCertificateCheck would have priority, meaning if both were supplied either a parameter exception is thrown or -SkipCertificateCheck would be applied and the callback ignored.

The Parameter will be named CertificateValidationScript

For consideration

What to name this parameter?
Throw or no throw on clash with -SkipCertificateCheck?

For name of the parameter we could start discussion with ServerCertificateCustomValidationCallback.

As for ScriptBlock we need remember about binary cmdlets too.

I don't think Callback fits well into the PowerShell user's domain. Also, Custom is a bit redundant IMO.

How about ServerCertificateValidationAction? Or may since the other parameter is just -SkipCertificateCheck (not SkipServerCertificateCheck), maybe the parameter should be just CertificateValidationAction?

Ilya (@iSazonov)

Ilya (@iSazonov) I believe there is a way to accept both binary and shell friendly callbacks. I need to find an example to borrow from, but I'm sure I have seen that done somewhere before. I just can't remember where. But before I go down that rabit hole, is this something that really needs considering? The Web Cmdlets inherit from PSCmdLet, not Cmdlet there is a bit of an understanding there that it will be running inside a PowerShell runspace, a ScriptBlock in that scenario is probably no more or less cumbersome than Func<HttpRequestMessage,X509Certificate2,X509Chain,SslPolicyErrors,Boolean>

Keith Hill (@rkeithhill) Action hmm. I don't think that is a PowerShell friendly name either. Script maybe. CertificateValidationScript, CertificateValidationOverride hmm.

Action is used in a number of commands including Set-PSBreakpoint and Register-ObjectEvent. Fortunately, this sort of thing is easy to analyze in PowerShell e.g.:

15:10ms> (gcm -CommandType Cmdlet).ParameterSets.Parameters.Where({$_.ParameterType -match 'ScriptBlock'}) | Group Name
| Sort Count -Descending

Count Name                      Group
----- ----                      -----
   15 ScriptBlock               {System.Management.Automation.CommandParameterInfo, System.Management.Automation.Com...
    7 Action                    {System.Management.Automation.CommandParameterInfo, System.Management.Automation.Com...
    3 InitializationScript      {System.Management.Automation.CommandParameterInfo, System.Management.Automation.Com...
    3 Expression                {System.Management.Automation.CommandParameterInfo, System.Management.Automation.Com...
    1 TransactedScript          {System.Management.Automation.CommandParameterInfo}
    1 FilterScript              {System.Management.Automation.CommandParameterInfo}
    1 End                       {System.Management.Automation.CommandParameterInfo}
    1 Process                   {System.Management.Automation.CommandParameterInfo}
    1 Begin                     {System.Management.Automation.CommandParameterInfo}
    1 RemainingScripts          {System.Management.Automation.CommandParameterInfo}

However, both Action and ScriptBlock are used as the entire parameter name. OTOH Script is used as a common suffix. So your suggestion of CertificateValidationScript is probably the best parameter name.

As for ScriptBlock we need remember about binary cmdlets too.

You can always expose a public property that is not marked with [Parameter] that is a delegate type.

Since I don't deal with binaries that talk directly to PowerShell, what exactly is the problem?

From what I can tell, if I type the parameter as a Func<HttpRequestMessage,X509Certificate2,X509Chain,SslPolicyErrors,Boolean> which is the same type as HttpClientHandler.ServerCertificateCustomValidationCallback, PowerShell script users will be able to pass a ScriptBlock to it.

Will this some how cause problems for Binary users?

When you are writing a binary cmdlet you can invoke other binary cmdlets. While you can create ScriptBlocks in C#, I guess it might be a little clunky to use that instead of a delegate. OTOH I'm not sure how common this scenario is. I've done this before but pretty rarely and never with something that took a delegate/ScriptBlock. OTOH if your parameter type is already a delegate then problem solved. :-)

All good then, that was the plan all along. I just wasn't clear in my proposal. 😄

Side note: The Web Cmdlets derive from PSCmdlet and not Cmdlet and cannot be invoked that way. They can only be invoked using PowerShell.Create(), AddCommand(), and AddParameter(). I guess a ScriptBlock would still be problematic that way.

OK, I have a working example here.

1 Problem: since the callback is run async the runspace is not available in that thread. To work around this, I am wrapping the provided delegate with code that creates the runspace, calls the delegate, then cleans up the runspace. This works fine. However, this means that there is no access to the current session state (variables, modules, functions, etc).

I'm not sure it needs to. If this is properly documented that the code in the script runs in its own context, it should be fine. I don't want to make an overcomplicated feature for something that has a limited target audience.

Thoughts and Feedback welcome.

# emulate -SkipCertificateCheck
$script = { return $True }
Invoke-RestMethod https://expired.badssl.com/ -CertificateValidationScript $Script

# Block all Certs
$script = { return $False }
Invoke-RestMethod https://google.com/ -CertificateValidationScript $Script

# -SkipCertificateCheck overrides -CertificateValidationScript
$script = { return $False }
Invoke-RestMethod https://expired.badssl.com/ -CertificateValidationScript $Script -SkipCertificateCheck

# Accept a .NET Delegate:
Invoke-RestMethod https://expired.badssl.com/ -CertificateValidationScript ([System.Net.Http.HttpClientHandler]::DangerousAcceptAnyServerCertificateValidator)

Jason Shirk (@lzybkr)

Jason Shirk (@lzybkr) Could you please comment - what is right way to implement the callback parameter?

I'm not sure how important it is to support delegate callbacks - I'd imagine these cmdlets are mostly called from PowerShell scripts, so I would just take a scriptblock.

This mostly solves the problem of invoking the scriptblock without a runspace because PowerShell marshals the scriptblock.invoke() call back to the runspace creating the scriptblock - though this can cause a hang if that runspace is blocked in a .Net method.

Jason Shirk (@lzybkr)

Jason Shirk (@lzybkr) so if I set handler.ServerCertificateCustomValidationCallback with the supplied script block, and that callback is called async, it will work with the current scope and without needing to initialize a runspace?

It depends on what is happening on the cmdlet's thread.

First, you will need to convert the scriptblock to the appropriate delegate type with LanguagePrimitives.ConvertTo<DelegateType>(scriptblock).

But as I mentioned - if the cmdlet's thread is stuck in code outside of PowerShell, there is no way for that thread to invoke the scriptblock - it will instead hang waiting until PowerShell gets control of the thread again.

In an ideal case, you have:

Pipeline thread (cmdlet's thread):
Starts some async code.
Receive and write some output.
Background thread
scriptblock.Invoke() - sends event to Runspace thread, blocking until event is processed

The important thing is to reach this code on the pipeline thread. This happens naturally if you execute some PowerShell script or write to the pipeline.

If neither of those things are happening, you need to do something like this assuming you can regain control after calling into some non-PowerShell code. Async methods normally give you control, so hopefully you can make this work.

alx9r

alx9r Thanks! I will have to look at what you provided this weekend. Have you tested any of this with the HttpClientHandler.ServerCertificateCustomValidationCallback? It appears to be a slightly different beast in CoreFX than what was on the ServicePointManager in full CLR.

Mark Kraus (@markekraus)

Mark Kraus (@markekraus) I did do a bit of testing of HttpClientHandler.ServerCertificateCustomValidationCallback using one of the v6.0.0-beta.8. I didn't notice any obvious differences in behavior.

It appears to be a slightly different beast in CoreFX than what was on the ServicePointManager in full CLR.

I haven't worked with ServicePointManager so I can't really speak to that. The work I talked about in my last post is all with full 4.7 WebRequestHandler.ServerCertificateValidationCallback (see here) which looks rather similar to core 2.0 HttpClientHandler.ServerCertificateCustomValidationCallback. I would have used HttpClientHandler instead of WebRequestHandler, but it didn't expose ServerCertificateCustomValidationCallback until 4.7.1 which was only released a couple of days ago.

Just to let you know, the callback currently (as of .Net Core 2.0) does not work on all platforms (e.g. OSX). The two modes that you are referring to (default validation and skip validation) are the only consistent modes currently supported by .Net Core. There is hope for this to get fixed with the next version of .Net Core.
I am trying to get this documented: https://github.com/dotnet/corefx/issues/24774
It might be ok to handle the PlatformNotSupported exception and inform the user that the parameter(s) are not supported on your platform.

Atanas Atanasov (@atanasa)

Atanas Atanasov (@atanasa) Yup, I'm aware. Though in some instances it seems to work and in others it doesn't. It was passing our macOS tests for awhile and then stopped. I finally got a loaner mac to troubleshoot, but I just haven't had time (real job getting in the way ☹️ ). We have a similar issues with -Certificate and certificate based authentication with the web cmdlets.

I have a PowerShell 5.1 Module that implements a custom class from ICertificatePolicy to be used with [System.Net.ServicePointManager]::CertificatePolicy to allow for normal Certificate Validation with support for overriding the Common Name validation by providing a custom whitelist of acceptable Common Names which is useful when connecting to HTTPS via IP address rather than FQDN but you still wish to validate the server certificate.

It would be useful to have this capability again across the Web Cmdlets without having to make my own replica Cmdlets, based around HttpClient, just to support this, or, having to use SkipCertificateCheck and lose all protection.

This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you.

This issue has been marked as "No Activity" as there has been no activity for 6 months. It has been closed for housekeeping purposes.

Navigation Menu

Repository navigation

Add User Supplied ServerCertificateCustomValidationCallback Support to Web Cmdlets #4899

Problem

Proposal

For consideration

26 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Add User Supplied ServerCertificateCustomValidationCallback Support to Web Cmdlets #4899

Description

Problem

Proposal

For consideration

Activity

iSazonov commented on Sep 23, 2017

rkeithhill commented on Sep 23, 2017

markekraus commented on Sep 23, 2017

rkeithhill commented on Sep 23, 2017

rkeithhill commented on Sep 23, 2017

markekraus commented on Sep 24, 2017

rkeithhill commented on Sep 24, 2017

markekraus commented on Sep 24, 2017

markekraus commented on Sep 24, 2017

iSazonov commented on Sep 25, 2017

lzybkr commented on Sep 25, 2017

markekraus commented on Sep 25, 2017

lzybkr commented on Sep 25, 2017

26 remaining items

markekraus commented on Oct 19, 2017

alx9r commented on Oct 20, 2017

atanasa commented on Oct 24, 2017

markekraus commented on Oct 24, 2017

Viajaz commented on Apr 6, 2022

microsoft-github-policy-service commented on Nov 16, 2023

microsoft-github-policy-service commented on Nov 16, 2023

microsoft-github-policy-service commented on Nov 16, 2023

microsoft-github-policy-service commented on Nov 23, 2023

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions