Directly constructed .NET type instances lack properties that ones output by cmdlets have

Update: See also this later proposal to provide the provider properties via class-backed CodeProperty members rather than per-instance NoteProperty members.

Get-ChildItem 3x: Replace NoteProperties on FileSystem- and RegistryProvider by class backed CodeProperties. #7501

From what I can tell, this discrepancy has always existed, so there may be a good reason for it, but it deserves an explanation / documentation.

Specifically, where do the extra properties come from? They appear not to be part of the ETS type data as returned by Get-TypeData System.IO.DirectoryInfo (and there's also none for the parent type, Get-TypeData System.IO.FileSystemInfo).

The absence of the .PSPath property from the directly constructed [System.IO.DirectoryInfo] and [System.IO.FileInfo] instances is especially problematic, because pipeline-binding to -LiteralPath parameters is based on it.

The types used are just an example; the same discrepancy exists for other types, such as [datetime]::now vs. Get-Date.

Steps to reproduce

# Construct  seemingly identical [System.IO.DirectoryInfo] instances
([System.IO.DirectoryInfo] '/').psextended | Format-List
'---'
(Get-Item '/').psextended | Format-List

Expected behavior

PSPath        : Microsoft.PowerShell.Core\FileSystem::/
PSParentPath  : 
PSChildName   : 
PSDrive       : /
PSProvider    : Microsoft.PowerShell.Core\FileSystem
PSIsContainer : True
Mode          : d-r---
BaseName      : /
Target        : 
LinkType      : 

---

PSPath        : Microsoft.PowerShell.Core\FileSystem::/
PSParentPath  : 
PSChildName   : 
PSDrive       : /
PSProvider    : Microsoft.PowerShell.Core\FileSystem
PSIsContainer : True
Mode          : d-r---
BaseName      : /
Target        : 
LinkType      :

Actual behavior

Mode     : d-r---
BaseName : /
Target   : 
LinkType : 

---

PSPath        : Microsoft.PowerShell.Core\FileSystem::/
PSParentPath  : 
PSChildName   : 
PSDrive       : /
PSProvider    : Microsoft.PowerShell.Core\FileSystem
PSIsContainer : True
Mode          : d-r---
BaseName      : /
Target        : 
LinkType      :

As you can see, the PS-prefixed properties only exist on the instance output by Get-Item.

Environment data

PowerShell Core v6.0.0-beta.4 on macOS 10.12.5
PowerShell Core v6.0.0-beta.4 on Ubuntu 16.04.2 LTS
PowerShell Core v6.0.0-beta.4 on Microsoft Windows 10 Pro (64-bit; v10.0.15063)
Windows PowerShell v5.1.15063.413 on Microsoft Windows 10 Pro (64-bit; v10.0.15063)

All provider cmdlets add these properties when writing objects to the pipeline, you can see the code here.

It would be better for performance if we used ETS, but there are likely cases where we would need to add these properties to override the ETS value, e.g. if a PSDrive was used in the path.

Jason Shirk (@lzybkr)

Thanks, Jason Shirk (@lzybkr), good to know.

Hypothetically, is it technically feasible to combine the two approaches? (ETS properties that are selectively overridden by the provider cmdlets, if needed?)

Also, Get-Date is not a provider cmdlet.
Does that mean that individual cmdlets also add additional properties (such as .DisplayHint in the case of Get-Date)?
Is that common, and is there an easy way to discover it across all cmdlets?

We add DisplayHint here (In Set-Date too.)
Although output type is defined as Datetime we modify it.

Ilya (@iSazonov)

Thanks, Ilya (@iSazonov) - good to know.

My overall concern is that bypassing the ETS can lead to both subtle and confusing bugs.

It is counter-intuitive for two [System.DateTime] instances to behave differently, solely based on how they were obtained.

If bypassing cannot be avoided for technical reasons (as it sounds is the case with provider cmdlets), the additional properties should at least be documented.

The trick is that it's not C# type - it is really a PSObject with same name.

Ilya (@iSazonov)

Ilya (@iSazonov):

Understood, but that fact is normally hidden from users.

Even the ETS properties are a departure from the underlying raw .NET type, but the point is that it can be confusing that something that presents as a given type in PowerShell can have different properties in different contexts.

Michael Klement (@mklement0)

Michael Klement (@mklement0) - I'm sure this is documented somewhere, but there are 3 kinds of members - instance (added via an api or Add-Member), type (added via types.ps1xml or Update-TypeData) - and native - .Net or AD or CIM or whatever the underlying object provides.

Instance members hide type members, and type members hide native members, so combining is definitely doable.

There isn't a super easy way to find all the places where members are added, but searching for new NoteProperty will find many of the locations - there are other types of members you can add of course, so you'd want to search for those too, but NoteProperty is the most common.

Jason Shirk (@lzybkr)

Jason Shirk (@lzybkr) we are planning to review about_types soon, if this fits in there would you mind giving us some additional details during review?
thanks!

Zachary Alexander (@zjalexander)

Zachary Alexander (@zjalexander) - sure, ask away when you have specific questions or something for me to review.

From what I can tell, this discrepancy has always existed, so there may be a good reason for it, but it deserves an explanation / documentation.

Instance members are added imperatively by cmdlets, etc, including directly with Add-Member or through the API:

PS[1] (103) >  $x = 2
PS[1] (104) >  $x.PSObject.Members.Add([System.Management.Automation.PSNoteProperty]::new("hi", "there"))
PS[1] (105) >  $x.Hi
there

Basically there is know way of knowing what properties are being added where so there's no way that

[io.file]::new("c:\temp\mypath.txt")

can add the properties that the provider would have added. Conversely, [io.file]::new() knows nothing about the current provider drive or path so you can't pass relative paths to these APIs.

More generally, we explicitly decided not to try wrapping .NET APIs because it was too fragile. There are just to many APIs to wrap. The result would be patchy and inconsistent. Currently it is at least self-consistent.

do folks feel like this has been adequately answered at this point

Speaking as a [folks] instance: personally, yes.

if so, do folks have an idea of where it should be doc'd?

about_Types.ps1xml has been mentioned, but the problem discussed here is not ETS-related.
I therefore suggest about_Object_Creation.

Incidentally, the latter lacks documentation of the PSv5 static ::new() method for calling constructors - see MicrosoftDocs/PowerShell-Docs#3209 - and deserves a bit of an overhaul in general - see MicrosoftDocs/PowerShell-Docs#3210

On second thought:

I think a note should be added to about_Types.ps1xm as well, based on @lzybkr's comment above.

The relationship between ETS and instance properties added via Add-Member is already mentioned there, but there's nothing about instances properties being automatically added by providers or (other) cmdlets.

A note in about_Object_Creation should then say that directly constructed instances (via ::new() / New-Object) may lack properties that instances of the same type obtained via cmdlets may have.

It strikes me as somewhat odd that out of all the cmdlets, New-Object doesn't add the additional properties. 😕

Rain Sallow (/u/ta11ow) (@vexx32)

Rain Sallow (/u/ta11ow) (@vexx32) think of it this way. You could have two cmdlets CmdletA and CmdletB that extend [System.Diagnostics.Process] in different ways. One adds a PropA and the other adds PropB (I would also expect both cmdlets to add to .pstypenames so one might be System.Diagnostics.Process#PropA and the other System.Diagnostics.Process#PropB (useful for formatting, for example). New-Object doesn't know anything about CmdletA nor CmdletB and thus doesn't know about the additional properties. Same reason ::new() wouldn't either.

I guess I'm just not really... clear... on why some objects are done this way, whereas others simply have new types implemented that inherit from the original. 😄

Rain Sallow (/u/ta11ow) (@vexx32)

Rain Sallow (/u/ta11ow) (@vexx32) I agree from a user perspective, it can be confusing why a [Process] is not the same as another [Process]. But I think the pros having this support in PowerShell outweighs the cons.

Michael Klement (@mklement0)

I don't think there's anything inherently wrong with the approach... but I do think Michael Klement (@mklement0) has a point in asking whether we ought to roll them into some kind of more standard member -- perhaps we should have a different binding point for these properties for consistency... Though I'm sure that's no small ask. 😄

Steve Lee (@SteveL-MSFT)

Steve Lee (@SteveL-MSFT) describes a current pattern, but it doesn't make it a good pattern.

To use the Get-Process example, I think Get-Process -IncludeUserName is a bit gross. UserName seems important enough that it should always be available on a Process object, and ETS was designed for just that scenario.

If I want the user name in the output, then I'd rather see Get-Process | Format-Table -View ProcessWithUserName or something like that.

To expand a bit on how we abuse ETS unnecessarily in the name of formatting, I'll point out an idea first introduced by Kirk Munro (@KirkMunro) in FormatPx.

The basic idea is that formatting directives are attached to an object without relying on the typename. This has the benefit of binding your formats in a script, e.g.:

function Get-Process
{
    # Use a very basic default format
    [System.Diagnostics.Process]::GetProcesses() | Format-Table Id,ProcessName
}

And use the function like this:

PS> $x = Get-Process
PS> $x[0].ProcessName
cmd
PS> $x

Id  ProcessName
==  ===========
11  cmd
22  powershell

PS> $x | Format-Table Id,SI,ProcessName

Id  SI  ProcessName
==  ==  ===========
11   1  cmd
22   2  powershell

So in this example, my function Get-Process fully specifies the default formatting while writing instances of System.Diagnostics.Process that can be used normally and later have a different format applied.

FormatPx works like this using the existing format system, and I've implemented this idea in PSMore which is just barely functional as a prototype.

In summary, I'd like to see PowerShell move away from adding members to each instance and use type members as much as possible. I think this would be a win from a performance and usability point of view.

I would love to see the Format cmdlets work like that -- still return usable objects, just with the formatting directives altered -- wow!

That would save a significant amount of confusion for newbies, give us a way to simply and easily specify default formats programatically, the whole deal. Love it!

#7857 for tracking PSMore.

Michael Klement (@mklement0)

Michael Klement (@mklement0) I want close this issue. If you see something should be documented please open issue in Docs repo.

Related to the PSMore/FormatPx discussion: #10463.

Navigation Menu

Repository navigation

Directly constructed .NET type instances lack properties that ones output by cmdlets have #4347

Steps to reproduce

Expected behavior

Actual behavior

Environment data

1 remaining item

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Directly constructed .NET type instances lack properties that ones output by cmdlets have #4347

Description

Steps to reproduce

Expected behavior

Actual behavior

Environment data

Activity

lzybkr commented on Jul 26, 2017

mklement0 commented on Jul 26, 2017

iSazonov commented on Jul 26, 2017

mklement0 commented on Jul 26, 2017

iSazonov commented on Jul 26, 2017

mklement0 commented on Jul 26, 2017

lzybkr commented on Jul 26, 2017

zjalexander commented on Oct 11, 2017

lzybkr commented on Oct 11, 2017

BrucePay commented on May 3, 2018

1 remaining item

mklement0 commented on Oct 30, 2018

mklement0 commented on Oct 30, 2018

vexx32 commented on Oct 30, 2018

SteveL-MSFT commented on Oct 31, 2018

vexx32 commented on Oct 31, 2018

SteveL-MSFT commented on Oct 31, 2018

vexx32 commented on Oct 31, 2018

lzybkr commented on Oct 31, 2018

vexx32 commented on Oct 31, 2018

iSazonov commented on Oct 31, 2018

iSazonov commented on Sep 28, 2019

KirkMunro commented on Sep 30, 2019

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions