PS 6/7 are 4-5x slower than PS 5 due to not using web connection pool

I'm calling an API repeatedly (500,000 times) and I noticed it runs 4-5x faster on PowerShell 5. The code below is not the actual API, but a minimal example that demos the issue.

Steps to reproduce

I'm running this on 2 different "machines":

PowerShell 5 and 6 on Windows 10 (physical)
PowerShell 5 and 7 on Windows Server (VM)

The results are pretty consistent. I noticed PowerShell 6 or 7 for Mac was really slow -- that's why I started benchmarking.

You can adjust $max on your machine to get a more reasonable run duration for comparison.

PowerShell Version	Duration (sec.)
5	1
6	4
7	5

<#
Ver.   Sec
----   ---
   5     1
   6     4
   7     5
#>

$max = 10
$start = Get-Date -Format "HH:mm:ss"
foreach ($i in 1..$max) {
    Invoke-RestMethod "https://postman-echo.com/get?i=$i"
}
$end = Get-Date -Format "HH:mm:ss"
write-host "started at" $start
write-host "  ended at" $end

Unless somebody is aware of why this might be, I'm willing to take a look.

PowerShell 6/7 seems to use the HttpClient class which takes about 5-7 seconds of my system under PowerShell 7.0.1 or Windows PowerShell 5:

Measure-Command -Expression `
{
   Add-Type -Assembly System.Net.Http
   $Max = 100
   $Size = 0
   ForEach ($I in 1..$Max) 
   {
      $Url = "http://postman-echo.com/get?i=$i"
      $Client = [System.Net.Http.Httpclient]::new()
      $Request = [System.Net.Http.HttpRequestMessage]::new('Get',$Url)
      $CancelToken = [System.Threading.CancellationToken]::new($false)
      $Option = [System.Net.Http.HttpCompletionOption]::ResponseContentRead
      $Response = $Client.SendAsync($Request, $Option, $CancelToken).GetAwaiter().GetResult();
      $Size += $Response.Content.ReadAsByteArrayAsync().Result.Length
   }
   Write-Host 'Total Size' $Size
}

Windows PowerShell uses System.Net.WebRequest; this request completes in about 2-3 seconds on my system under Windows PowerShell:

Measure-Command -Expression `
{
    $Max = 100
    $Size = 0
    ForEach ($I in 1..$Max) {
        $Url = "http://postman-echo.com/get?i=$i"
        $Request = [System.Net.WebRequest]::Create($Url)
        $Response = $request.BeginGetResponse($null, $null)
        $Response.AsyncWaitHandle.WaitOne()
        $Result= $Request.EndGetResponse($Response)
        $Reader = [System.IO.BinaryReader]::new($Result.GetResponseStream());
        $Size += $Reader.ReadBytes($Result.ContentLength).Length
    }
   Write-Host 'Total Size' $Size
}

Interestingly, this latter code also runs much 2x-3x slower under PowerShell 6/7 compared to Windows PowerShell. This kind of surprised me.

So the question may actually be three-fold: 1) Why the switch to HttpClient? This is not an obvious problem at first glance... just a curiosity. 2) Why is performance so much different between HttpClient and WebRequest under Windows PowerShell, and 3) Why the slowdown in WebRequest from Windows PowerShell to PowerShell Core (may be an underlying DotNet Core question).

WebClient was simply nonexistent in .NET Core during pretty much all of the development of 6.x, and we needed web cmdlets to work with. Even now, it's only really been brought back in for compatibility and isn't recommended for new code as I understand it.

Nor sure about much beyond that, though. 🙂

PowerShell 5 is using a single persistent TCP connection, PowerShell 7 is opening a new connection for each HTTP request.

If the -DisableKeepAlive parameter is used with Invoke-RestMethod there is no difference in performance.

Bryan Berns (@NoMoreFood)

Bryan Berns (@NoMoreFood) If you set $Request.KeepAlive = $false in your example the performance is also the same.

PowerShell 7 actually sends the Connection: Keep-Alive header, but than opens a new connection for every request.

That's really good info! Thank you.

I did a little more digging and here's what I found. Node.js (using node-fetch) and Python (2/3, using requests) are also very slow, but JS in Chrome is as fast as PowerShell 5. I also used Wireshark, and noticed that the slow ones seem to negotiate HTTPS each time, but the fast ones do it only once. If you try it using HTTP instead of HTTPS, it's much faster. Unfortunately, the code I have to run requires HTTPS. hth.

// JavaScript

// To run this in a browser, first go to https://postman-echo.com/get to avoid CORS issues.

const fetch = require("node-fetch"); // Only for node.js. Remove for Chrome.

(async function () {
    const MAX = 10;
    const start = new Date();
    for (var i = 0; i < MAX; i++) {
        const r = await fetch(`https://postman-echo.com/get?i=${i}`);
        const d = await r.json();
    }
    const end = new Date();
    console.log(start);
    console.log(end);
    console.log(end - start);
})();

Python without connection pooling -- see below for an example using connection pooling

# Python 3 (without connection pooling)

import requests
from datetime import datetime

MAX = 10
start = datetime.now()
for i in range(MAX):
    requests.get(f'https://postman-echo.com/get?i={i}').json()
end = datetime.now()
print("started at", start)
print("  ended at", end)
print(end - start)

Cory Nelson (@scalablecory)

It appears that ~~TCP KeepAlive~~HTTP persistent connections are supported in certain circumstances. If a singular HttpClient is declared and reused for each request, TCP connection reuse occurs and performance is much better, especially when using HTTPS:

$Client = [System.Net.Http.HttpClient]::new()
$Client.BaseAddress = "https://postman-echo.com"
(Measure-Command {
    (1..100).ForEach{
        $Request = [System.Net.Http.HttpRequestMessage]::new('Get', "/get?i=$_")
        $Response = $Client.SendAsync($Request).GetAwaiter().GetResult();
        Write-Output $Response.Content.ReadAsStringAsync().Result
    }
}).TotalSeconds

Edited to correct conflation between TCP KeepAlive and HTTP KeepAlive (persistent connections). (Thanks Cory Nelson (@scalablecory))

The problem here is that the HttpClient needs to be reused between calls of Invoke-RestMethod. This enables connection pooling.

Note that this is not TCP Keepalive (a technology to detect broken idle connections), and the two issues linked above will not help to resolve this.

So the the difference in perfomance is due the fact Windows PowerShell 5 pools connections between seperate instances of [System.Net.WebRequest], and PowerShell 7 does not.

As far as I can see, the only way to ensure use of a persistent connection (in PowerShell 7) is to use a raw HttpClient instead of the cmdlets Invoke-RestMethod/Invoke-WebRequest. Perhaps we could open a new issue to discuss whether to implement this functionality.

I guess I was only partially correct above. Python requests has a Session() object which allows for connection pooling. This only takes about 1 second as opposed to about 4 seconds for the example in my earlier comment (which doesn't use connection pooling).

# Python 3 (with connection pooling)

import requests
from datetime import datetime

session = requests.Session()

MAX = 10
start = datetime.now()
for i in range(MAX):
    session.get(f'https://postman-echo.com/get?i={i}').json()
end = datetime.now()
print('started at', start)
print('  ended at', end)
print(end - start)

HttpClient creates SocketsHttpHandler where connection pool is implemented.

If we will share the HttpClient or SocketsHttpHandler per process how do we resolve config conflicts between runspaces?
It seems we can not utilize a global pool.

~~Perhaps we could add new parameter in web cmdlets to get shared SocketsHttpHandler.~~
We could enhance WebRequestSession with SocketsHttpHandler.

The only disadvantage is the complexity it adds to an already quite complex cmdlet.

Perhaps there are unexpected obstacles there, but I don't see them now. Most likely the changes will be minimal.

The only one I see is that users could still change items on the web session through direct assignment in their scripts. However this would be a fairly strange thing to do I would hope. The implementation might need to be pushed into the web session class to detect when this happens.

I have made changes to detect when the WebSession has changed between invocations on my branch here. Please have a look and let me know if this approach is likely to work.

master...stevenebutler:PowerShell:12764-support-persistent-http-connections-in-cmdlets

CarloToso

Looking at this code I think it is too confusing. We need to do a refactor first. My main idea is based on the fact that both handler and HttpClient can't be changed after the start. That means there is no point in calling the GetHttpClient() method on the fly. In other words, all initializations should be in BeginProcessing() and the cmdlets could be IDisposable. I hope it possible.
Perhaps CarloToso knows the code better and could refactor it.

You can still call multiple invocations of iwr in sequence using the web session parameter to tie them together and most of the time I would expect gethttpclient to be called only once per invocation, so I’m not sure moving the call to beginprocessing will make much difference for most use cases. A refactoring is definitely not something I’d be willing to take on so I guess I’ll leave this issue the way it is for now.

stevenebutler

stevenebutler I tested your code and the speedup was impressive, please keep working on it and open a PR.
It also passes all the WebCmdlets tests.
Test:

Measure-Command {
    Write-Host (Invoke-WebRequest https://postman-echo.com/get?i=1 -SessionVariable Session).StatusCode
    1..10 | ForEach-Object {
        Invoke-WebRequest https://postman-echo.com/get?i=$_ -WebSession $Session
    }
}

Ps 7.3.2 --> 5538 milliseconds
your code --> 468 milliseconds

You can still call multiple invocations of iwr in sequence using the web session parameter to tie them together and most of the time I would expect gethttpclient to be called only once per invocation, so I’m not sure moving the call to beginprocessing will make much difference for most use cases. A refactoring is definitely not something I’d be willing to take on so I guess I’ll leave this issue the way it is for now.

Don't give up! That's just one person's opinion. The speed improvement is seriously impressive, I would really like to see a PR of this too.

I tested your code and the speedup was impressive, please keep working on it and open a PR.

To keep working on it I'd need to have some idea of what needs to be done to make it acceptable to merge.

I can start that process through a PR if that is the recommended course of action or if there's some improvements that should be made first, then I can try that first. I would assume at minimum it would need to merge with changes in #19109

I tested your code and the speedup was impressive, please keep working on it and open a PR.

To keep working on it I'd need to have some idea of what needs to be done to make it acceptable to merge.

I can start that process through a PR if that is the recommended course of action or if there's some improvements that should be made first, then I can try that first. I would assume at minimum it would need to merge with changes in #19109

Best way to find out is to submit what you have as-is and see what they say about it.

stevenebutler

stevenebutler You should merge with changes in #19109 and open a [W.I.P] PR, then continue working on it there.

@ImportTaste responding to your query on my repo that got lost when I force pushed some changes...

I wouldn't be able to make this an exteranl cmdlet without copying most of the existing code base into the third party module and I'm not really willing to do that.

The reason you'd have to do that is it depends on a lot of internal interfaces that an external module can't access. It also adds internal properties to the WebRequestSession you'd have to do externally as well.

If you're desperate for an interim solution you can write your own simplified PowerShell script that uses httpclienthandler/httpclient directly that supports only what you need. That is what I have been doing but it was relatively simple since I don't need 95% of the features on IWR.

Navigation Menu

Repository navigation

PS 6/7 are 4-5x slower than PS 5 due to not using web connection pool #12764

Steps to reproduce

43 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

PS 6/7 are 4-5x slower than PS 5 due to not using web connection pool #12764

Description

Steps to reproduce

Activity

NoMoreFood commented on May 29, 2020

NoMoreFood commented on May 30, 2020

vexx32 commented on May 30, 2020

xtqqczze commented on May 31, 2020

xtqqczze commented on May 31, 2020

xtqqczze commented on May 31, 2020

xtqqczze commented on May 31, 2020

gabrielsroka commented on May 31, 2020

xtqqczze commented on Jun 1, 2020

scalablecory commented on Jun 4, 2020

xtqqczze commented on Jun 4, 2020

gabrielsroka commented on Jul 5, 2020

iSazonov commented on Jul 6, 2020

43 remaining items

iSazonov commented on Feb 5, 2023

stevenebutler commented on Feb 5, 2023

stevenebutler commented on Feb 5, 2023

iSazonov commented on Feb 6, 2023

stevenebutler commented on Feb 6, 2023

CarloToso commented on Feb 12, 2023

God-damnit-all commented on Feb 13, 2023

stevenebutler commented on Feb 14, 2023

God-damnit-all commented on Feb 14, 2023

CarloToso commented on Feb 14, 2023

stevenebutler commented on Feb 19, 2023

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions