Repository navigation
Get-ChildItem 3x: Replace NoteProperties on FileSystem- and RegistryProvider by class backed CodeProperties. #7501
Description
When using Get-ChildItem, the ProviderBase adds these note properties:
PSChildName
PSDrive
PSIsContainer
PSParentPath
PSPath
PSProvider
This is quite expensive, both in terms of memory and execution time.
By replacing the noteproperties with a classes that lazily can compute most of the values, we can save both storage and time.
Timing:
| Command | 6.1-preview | After change | Speedup |
|---|---|---|---|
| GCI -rec -for d:\source\powershell | 0.446 | 0.144 | 3.1 x |
| GCI -rec -for c:\windows -ea:0 | 12.24 | 6.05 | 2 x |
| GCI -rec hkcu:\ -ea:0 | 7.86 | 2.99 | 2.6 x |
Cred to Jason Shirk (@lzybkr) - this was his idea, not mine.
Reacted by Patrick Meinecke, Joel Bennett and Michael Klement
Activity
powercode commentedon Aug 11, 2018
Memory:
The yellow is after the change.
This was measured before and after running
A quite substantial memory reduction.
powercode commentedon Aug 11, 2018
One thing to consider is if it is OK to add CodeProperties to
System.IO.FileInfoandSystem.IO.DirectoryInfo.Jaykul commentedon Aug 12, 2018
The classes in question (e.g. system.io.fileinfo) are sealed, meaning you can't just inherit from them... so what did you test? (Nevermind, I looked at the PR).
So you can definitely add properties in the TypeData as CodeProperty (resulting in a lazy load, but still C# evaluation) instead of adding them as pre-calculated NoteProperty ... There is no question of whether that's "ok" (To implement the property as a static on a helper class). I'd say, if TypeData properties work for output from provider cmdlets, then it makes sense to me to do it that way!
I think to get the full picture of perf. you'd need to test it piping through other commands, since so many of them use the PSPath property, for instance
get-childitem | get-item... or something.powercode commentedon Aug 13, 2018
Joel Bennett (@Jaykul) I was more thinking about the use case when you instantiate the FileInfo by newing it up. Then it will not have the backing property object, and these CodeProperties will still be there, but empty.
That is a change from before, when these properties was not there at all, since they are added per instance by the provider.
iSazonov commentedon Aug 13, 2018
Should we ask PowerShell Committee? /cc Steve Lee (@SteveL-MSFT) Dongbo Wang (@daxian-dbw)
[-]Replace NoteProperties on FileSystem- and RegistryProvider by class backed CodeProperties.[/-][+]Get-ChildItem 3x: Replace NoteProperties on FileSystem- and RegistryProvider by class backed CodeProperties.[/+]Jaykul commentedon Aug 14, 2018
Staffan Gustafsson (@powercode) I would still say that I can't imagine how adding those properties through ETS could cause a problem. 😁
SteveL-MSFT commentedon Sep 19, 2018
@PowerShell/powershell-committee reviewed this. Would suggest putting this feature behind an experimental flag to determine if this breaks any customers.
powercode commentedon Sep 20, 2018
I completely agree. This is kind of an experiment right now - I'm also looking at derived PSObject types and changing the binders. That seems to have more potential performance wise. Both should probably be behind an experimental flag.
microsoft-github-policy-service commentedon Nov 16, 2023
This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you.
microsoft-github-policy-service commentedon Nov 16, 2023
This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you.
microsoft-github-policy-service commentedon Nov 16, 2023
This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you.
microsoft-github-policy-service commentedon Nov 23, 2023
This issue has been marked as "No Activity" as there has been no activity for 6 months. It has been closed for housekeeping purposes.