Improving the discoverability of PowerShell variables

Continue #4216 to discuss improving the discoverability of PowerShell variables.

Problem: It is hard for users to discover PowerShell automatic (system) variables and get help (their description).

Idias:

Enhance cmdlets to provide better discoverability, tab-completion, etc.
1.1. Get-Variable -AutomaticVariable/-PowerShell/-System/-Special / -PreferenceVariable
1.2 Get-Help -AutomaticVariable/-SpecialVariable / -PreferenceVariable
~~Add new namespace $pspref or $psvar~~
Enhance the variable: provider with additional properties IsAutomatic and IsPreference.

Also enhance the variable: provider with additional property IsAutomatic

Steve Lee (@SteveL-MSFT)

Steve Lee (@SteveL-MSFT):

Good idea; while we're at it: IsPreference would be useful too.

Programmatic discovery of preference variables has been suggested before, on uservoice.com.

I still think the introduction of namespaces is worth considering, given that, with the current approach of sticking all automatic / preference variables in the same namespace as user-defined variables, any introduction of new automatic / preference variables comes with the risk of breaking existing scripts that happen to use variables of the same name.

Technically, a single namespace such as $ps: would suffice, containing the union of preference variables and automatic variables, including the proposed platform-abstracted "environment" variables (PS-internal variables that provide cross-platform information about the environment, such as the location of the directory for temporary files).

Given how namespace notation currently works, an underlying ps: drive would be required, though a conceivable departure could be to instead use Get-Variable with switches such as the new ones proposed in the initial post for filtered discovery of all variables, including those in the $ps: namespace.

Internally we use term AutomaticVariables in different way then public.

Should we use SpecialVariables that is used internally in public too?

We have 7 well-known preference variable - I think there's no point in highlighting them.

Re automatic variables: While this discrepancy is unfortunate, automatic variable is a well-established and documented user-facing term.

It sounds to me that the way to resolve this is to make a note in the source code or change the variable name(s), such as to AutomaticEngineVariable.

Re preference variables:

Similarly, the about_preference_variables help topic lists many more preference variables than the 7 named as such in the source code:

> (Get-Help about_preference_variables) -split '\r?\n' | Select-String '^\s+(\$\w+)\s{2,}' | % { (-split $_)[0] }

$ConfirmPreference
$DebugPreference
$ErrorActionPreference
$ErrorView
$FormatEnumerationLimit
$InformationPreference
$LogCommandHealthEvent
$LogCommandLifecycleEvent
$LogEngineHealthEvent
$LogEngineLifecycleEvent
$LogProviderLifecycleEvent
$LogProviderHealthEvent
$MaximumAliasCount
$MaximumDriveCount
$MaximumErrorCount
$MaximumFunctionCount
$MaximumHistoryCount
$MaximumVariableCount
$OFS
$OutputEncoding
$ProgressPreference
$PSDefaultParameterValues
$PSEmailServer
$PSModuleAutoLoadingPreference
$PSSessionApplicationName
$PSSessionConfigurationName
$PSSessionOption
$VerbosePreference
$WarningPreference
$WhatIfPreference

On a side note: $OFS is currently documented as both an automatic and a preference variable, whereas I think it should only be the latter - see MicrosoftDocs/PowerShell-Docs#1566

I'll address the need for discoverability of preference variables separately.

Similarly, the about_preference_variables help topic lists many more preference variables than the 7 named as such in the source code

It's a misleading internal/public terminology again.

As for the need for programmatic discoverability of automatic / preference variables:

PowerShell (as well as .NET) has a strong tradition of "self-awareness" through reflection.

Generally, you'd want to know about special attributes / behavior of variables: whether a variable is defined by PowerShell and whether a variable modifies PowerShell's behavior is certainly worth knowing about.

This is especially true currently, given that these variables live in the same namespace as user variables, but the argument applies in general.
Again, my vote is to move these variables into a separate namespace, while grandfathering in the current variables for backward compatibility.

In particular, with respect to preference variables, #4568 shows why a programmatic way to discover all preference variable is important (though I wish this particular need would go away via a built-in way to honor all caller preference-variable values in the context of functions inside script modules).

Michael Klement (@mklement0)

Michael Klement (@mklement0) My 2 cents, if it helps: many years ago I created a PowerShell module framework for a client and ran into this issue. The environment was very heterogeneous; OK, not cross-OS like Core but every flavor of Windows that supported PS v1 and above. Using a separate namespace - an in-memory-only $psf: drive - solved the problem perfectly:

clean abstraction separate from all other types of PS variables, 3rd party modules, user-specific settings, etc.
good discoverability: I created a function to walk the $psf: tree and output all variables and values; you could also manually walk the tree if you wanted using tab completion.
easy to override per machine if necessary: user could run $psf:Folders.Temp = <whatever> in the shell to update for just that session or update their $profile to permanently override the value set when bootstrapping.
easy to customize and maintain. The tree was basically a structure of hash tables so it worked well in PowerShell, including tab completion. The tree could be as deep and / or wide as needed.

All that said: this solution did (and still does) run on 100+ machines, not the millions you have to consider. Plus there is backward compatibility to think about. But if you are interested in making a fresh start with Core, a separate namespace / settings drive would be nice. You could also add some structure, improving organization and discoverability, i.e. $psf:Folders.Home , $psf:Folders.Temp , $psf:Preferences.Verbose , etc.

Dan Ward (@DTW-DanWard)

Thanks, Dan Ward (@DTW-DanWard) - good stuff. So you created an in-memory $psf: drive by implementing your own provider?

I agree that a separate namespace makes sense - and PowerShell has the sophistication in terms of features to pull it off (unlike other shells).
The backward-compatibility issue could be solved by continuing to surface the defined-so-far PS-controlled variables in the user-variable namespace while also exposing them via the new namespace.

Michael Klement (@mklement0)

Yep Michael Klement (@mklement0) a custom provider. It was a great way to provide a default location for settings that could be quite different based on the machine: dev machines vs. servers, web servers vs. databases vs. , dev environment vs. UAT vs prod, etc. And even though I typically only used it with machine-level settings (not app-level), it grew quickly enough that I still had to organize the contents into 'folders': $psf:Folders, $psf:Applications, $psf:Applications.Database, $psf:Applications.Database.SqlCmd = <path>, etc.

Michael Klement (@mklement0)

Michael Klement (@mklement0) RE: $ProgressPreference -- it has been my experience that setting $ProgressPreference is only reliable in the current scope. I'm not sure if other cmdlets simply routinely modify it themselves for some reason, or if there are issues in the underlying code that handles the variable.

This, combined with the very... clunky... nature of Write-Progress for many purposes which we would typically want to use it for, results in me simply... not... using it, at all. Write-Progress is great when it works, but by and large it is much more difficult to use than one would expect, and the effort one needs to input to get it working satisfactorily is typically far more than the task is worth.

Rain Sallow (/u/ta11ow) (@vexx32)

Rain Sallow (/u/ta11ow) (@vexx32) seems like you should open a separate issue on Write-Progress and suggestions on how to make it more usable

@mkelement0 you keep doubling down and making it even more complicated...

I think your proposed benefits are just an illusion. The name collisions and inconsistently applied prefixes are all still there, because as everyone keeps saying: you can't get rid of the current variables without breaking the world. People have already been told There's not better discoverability, you've actually buried the things in a folder ... people would need to read a document to even learn about that and this document would have to explain the old values as well as the new provider, because most people need to write code that works in older versions of PowerShell ... like 6.1

To me, it seems obvious that a simple property (or additional value in the options enum) is a much simpler option that keeps the actual code backwards-compatible, while still enhancing visibility (especially paired with a change to the view).

P.S. using : as a separator for the variable namespace would be really confusing, because it's already the separator for scopes. Don't you write $global:ProgressPreference? I never see code using the variable cmdlets just to access global or script (or local) scope -- it's usually only used when you need to set options, or access a scope by index.

P.P.S. I'm strongly opposed to adding hierarchy to a provider that's already scoped - even a new provider would be better.

Joel Bennett (@Jaykul)

Joel Bennett (@Jaykul)

The name collisions and inconsistently applied prefixes are all still there, because as everyone keeps saying: you can't get rid of the current variables without breaking the world.

They will continue to be there for backward compatibility.
They will be documented as deprecated (without fear of removal), and users will be encouraged to use the new namespace(s) going forward.
PSSA will provide guidance, as suggested.

There's not better discoverability, you've actually buried the things in a folder

Something like Get-Variable ps:* or Get-Variable -Namespace ps seems pretty discoverable to me.

To me, it seems obvious that a simple property (or additional value in the options enum) is a much simpler option

Tagging variables is fine, but it does not address the primary problem with how preference / automatic variables are currently maintained: every time a new one needs to be introduced, existing scripts may break.

people would need to read a document to even learn about that

Yes, when a new feature is introduced, people need to learn about it, if they want to avail themselves of it (they don't have to, thanks to backward compatibility, though they'll hopefully see the benefits of the new behavior).

most people need to write code that works in older versions of PowerShell .

By that reasoning, all innovation in PowerShell should be put on hold.

That improvements may take a while before they are usable in production is no reason not to introduce them.

using : as a separator for the variable namespace would be really confusing

I think it would be just fine in practice, given that : already does double duty as the scope-modifier separator ($global:LASTEXITCODE) and as the drive-name separator with namespace notation ($env:HOME).
~~The only thing new would be combining the two, but all you'll need to remember is that the scope modifier goes first~~ As is already the case^[1], the namespace (drive name) goes first (and creating drives / sub-namespaces named global, script or local would have to be prevented).

I'm strongly opposed to adding hierarchy to a provider that's already scoped - even a new provider would be better.

You needn't think of it as a hierarchy; think of it as partitioning into sub-namespaces.
That is, you'd access variables in the default namespace as before ($foo) or with a namespace qualifier in a given namespace ($ps:OutputEncoding).

Re new provider: obviously, most of the logic is already in the Variable provider and we're talking about simply partitioning the set of variables.

^{[1] Even though it's not supported consistently, the current namespace notation requires that the scope modifier follow the drive name; e.g., $function:global:help returns the definition of the help function in the global scope.}

Correction re sequence of namespace (drive name) and scope modifiers: as is already supported (albeit inconsistently), the namespace goes first (e.g., $function:global:help) - I've updated my previous comment accordingly.

This is a pretty rough read honestly. Given that there is this big conversation regarding creating variable pathing. My question is how does the proposed arguments add to and not obfuscate the base idea that $scope:variable exists.. as well as not muddy the waters diverging too far away from how variables work within the confines of C# and other scripting languages.

I believe adding a path would detract so much from the language that it would confuse a lot of people about how scoping works. Especially if path'ing would exist in the global space. Making Garbage Collection a little bit more murky.

Instead I recommend adding an Attribute to the Variable. and the ability to look up the variable by AttributeType. [PSAutomaticVariableAttribute].

So example code would be
Get-Variable * -AttributeType [PSAutomaticVariableAttribute]
Which would return $_, $PSItem, $Error.

I believe this would also avoid an entire rewrite of PSVariable module and help keep the language similar to other languages (C#/Python/Ruby/etc) and help implement a Pretty Awesome Feature.

romero126

I agree. Having variable paths is a nice idea in concept, but in practice it just means burying them further down out of the immediately accessible values.

Maintaining the PS prefix for any automatic variables like this would seem to be the best option, supplemented with the ability to specifically search for them within the variable provider by adding these designation(s) as a property or attribute of the variables themselves, per romero126's suggestions.

Rain Sallow (/u/ta11ow) (@vexx32)

Rain Sallow (/u/ta11ow) (@vexx32): As someone once said:

I think that while by and large PowerShell tends to backwards compatibility there are cases where it makes sense to break a few eggs to make a better omelet. 😄

And here's the good news: No eggs need harming in the implementation of my proposal:
Grandfathering in the existing variables means old code won't break.

Maintaining the PS prefix for any automatic variables like this would seem to be the best option

Yes, going forward you can commit to PS-prefixing future automatic variables, but that can still only be a convention, if you want to preserve backward compatibility.

Yes, implementing sub-namespaces would be a new concept that users will have to learn, but I think it's worth it in the long run.
It also fits in well with namespace variable notation (e.g., $env:HOME), which isn't currently well-known as a general concept, but deserves to be.

How was this issue resolved? Is there now any way to detect that a variable is a preference variable?

Ilya (@iSazonov)

How was this issue resolved? Is there now any way to detect that a variable is a preference variable?

Ilya (@iSazonov) Sorry to bug you but could you confirm how this was completed or is this no longer being pursued?

RiverHeart

RiverHeart This looks like nobody ended up taking on the mantle to work on this effort. It has been over 5+ years since this was looked at, and no development cycles have been given towards this ask. As nice as it would be to have this feature without somebody contributing to this open source project we probably won't see this feature being prioritized.

But that is just my 2c.

How was this issue resolved? Is there now any way to detect that a variable is a preference variable?

@iSazonov Sorry to bug you but could you confirm how this was completed or is this no longer being pursued?

Navigation Menu

Repository navigation

Improving the discoverability of PowerShell variables #4394

7 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Improving the discoverability of PowerShell variables #4394

Description

Activity

SteveL-MSFT commented on Aug 1, 2017

mklement0 commented on Aug 12, 2017

mklement0 commented on Aug 12, 2017

iSazonov commented on Aug 14, 2017

iSazonov commented on Aug 14, 2017

mklement0 commented on Aug 14, 2017

iSazonov commented on Aug 14, 2017

mklement0 commented on Aug 14, 2017

DTW-DanWard commented on Sep 3, 2017

mklement0 commented on Sep 4, 2017

DTW-DanWard commented on Sep 4, 2017

7 remaining items

vexx32 commented on Oct 9, 2018

SteveL-MSFT commented on Oct 9, 2018

Jaykul commented on Oct 10, 2018

mklement0 commented on Oct 10, 2018

mklement0 commented on Oct 17, 2018

romero126 commented on Oct 23, 2018

vexx32 commented on Oct 23, 2018

mklement0 commented on Dec 13, 2018

MatejKafka commented on Dec 18, 2023

RiverHeart commented on Jul 8, 2025

romero126 commented on Jul 8, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions