If a block inside a function contains return, all output is written to the pipeline, even if assigned to a variable

Prerequisites

Write a descriptive title.
Make sure you are able to repro it on the latest released version
Search the existing issues.
Refer to the FAQ.
Refer to Differences between Windows PowerShell 5.1 and PowerShell.

Steps to reproduce

The issue manifests whenever a function contains a block (if, foreach, try,...), which writes some output and then calls return. All output from the block is written into the output pipeline, even if the output of the block should be redirected into a variable, or a pipeline inside the function:

function x([switch]$p) {
    $null = foreach ($a in @(1)) {
        echo "this should not be visible"
        if ($p) {return}
    }
}

function y([switch]$p) {
    $null = try {echo "this should not be visible"; if ($p) {return}} catch {}
}

Expected behavior

PS> x
PS> y
PS> x -p
PS> y -p

Actual behavior

PS> x
PS> y
PS> x -p
this should not be visible
PS> y -p
this should not be visible

Error details

No response

Environment data

Name                           Value
----                           -----
PSVersion                      7.3.0-preview.6
PSEdition                      Core
GitCommitId                    7.3.0-preview.6
OS                             Microsoft Windows 10.0.19044
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

Visuals

No response

Matěj Kafka (@MatejKafka)

Matěj Kafka (@MatejKafka) it's a duplicate of #3669

Matěj Kafka (@MatejKafka)

Matěj Kafka (@MatejKafka) it's a duplicate of #3669

No, I don't believe it is. The referenced issue discusses the case where output is NOT assigned to $null (or ignored in some other way). This issue describes what I believe to be a bug in how the output is directed, even when assigned to $null.

Matěj Kafka (@MatejKafka)

Matěj Kafka (@MatejKafka) it's a duplicate of #3669

No, I don't believe it is. The referenced issue discusses the case where output is NOT assigned to $null (or ignored in some other way). This issue describes what I believe to be a bug in how the output is directed, even when assigned to $null.

You're right, I read too quickly, sorry 😅

Matěj Kafka (@MatejKafka)

Matěj Kafka (@MatejKafka) This is the minimal reproduction without any function

$a = if ($true) {
    Write-Output "this should not be visible"
    return "this should not be visible too"
}

$null -eq $a # true

$b = if ($true) {
    Write-Output "this should not be visible"
}
$null -eq $b # false


$c = if ($true) {
    return "this should not be visible too"
}
$null -eq $c # true

I'd be OK with the last case being left as is (so that "return X" always returns X from the function).

It's the first case that seems wrong to me, as there's an "action at a distance", with an unrelated return statement causing previous output to be treated as function output.

https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_return?view=powershell-7.2

I understand that return is an accumulator, so the behavior is compliant, but the documentation should be more explicit and detailed.

If anyone else can confirm, we might ask for a documentation upgrade.

about Return - PowerShell
Exits the current scope, which can be a function, script, or script block.

This is fairly weird behaviour, it seems like the return in the if statement is bypassing the $null = accumulator as I would not expect this to output anything

function test {
    $null = if ($true) {
        "out"
        return
    }
}

I can't say I've seen this documented and considering if you omit the return the if captures the output as expected.

Matěj Kafka (@MatejKafka)

If the PowerShell engine indeed acts the way as expected by Matěj Kafka (@MatejKafka) (except that I don't think it should accumulate anything else than the succes stream), it might be a way out for the leaking pipeline challenge (see also: #15781) where everything in the (success stream) pipeline is then consumed by the assignment except for what is explicitly returned by Return <object> or Write-Output [-InputObject] <PSObject[]>...

This is fairly weird behaviour, it seems like the return in the if statement is bypassing the $null = accumulator as I would not expect this to output anything
function test {
    $null = if ($true) {
        "out"
        return
    }
}

It is not intuitive that's for sure. And I don't think it is as consistent as it might be. But "accumulator" piqued my curiosity. Here are multiple evolutions of the same bit of code

function test {
   while ($null -eq $x) {
         $global:foo = foreach ($x in 1..5) {
            if ($x -eq 5) { return } 
            else {$x} 
        }
        "after the loop, x =$x "
    }
    "After the while"
}

This does what the OP saw. It looks like it should put the values into foo but it doesn't and they become "output"

#531 PS5 ~\Documents\windowsPowershell> $foo = $null 

#532 PS5 ~\Documents\windowsPowershell> $foo 

#533 PS5 ~\Documents\windowsPowershell> test
1
2
3
4

#534 PS5 ~\Documents\windowsPowershell> $foo

We [should] know that (without the return) the code would NOT put 1 into foo, then make foo an array and add 2 as member , then add 3 etc. - most of have us have learnt at a loop with $results = $results + $LatestItem gets slow for many items and $result = <loop> is to be preferred because PowerShell accumulates the results for us and assigns them in a single operation after the for loop has run.

We do not really think about the "Call something, and when execution returns the output you want will be a standard buffer" which underpins all of this, any more than we think about

  $val = foreach ($x in 1..5) {
            if ($x -eq 5) { throw } 
            else {$x} 
        }

putting 1..4 in the output buffer and then having throw remove them (they must have gone somewhere but they don't reach $val).
In this case return has inserted a GOTO between putting the results into the buffer and reading them back; Return, Break, Continue are all forms of GOTO by other means (see below before arguing! :-) ) - return means execution goes to whatever called this function instead of the next step of this function..

We can see the same thing with break, where (again) the behaviour is not intuitive. Someone who has learnt what return does in this case, but not why, might expect this to do the same

function test {
    while ($null -eq $x) {
         $global:foo = foreach ($x in 1..5) {
            if ($x -eq 5) { break  } 
            else {$x} 
        }
        "after the loop x =$x "
    }

"After the while"
}

All that has changed is the return has been replaced with a break - which exits the for but not the while so execution continues where it would have done without break

#540 PS5 ~\Documents\windowsPowershell> $foo = $null 

#541 PS5 ~\Documents\windowsPowershell> $foo

#542 PS5 ~\Documents\windowsPowershell> test
after the loop x =5 
After the while

#543 PS5 ~\Documents\windowsPowershell> $foo 
1
2
3
4

I said "wait before arguing about goto" - in this case break is just making the "if_not_condition run_the_remainder_of_the_code" construction less unwieldy - the point it is going to isn't arbitrary it is what would be the next step anyway. But it can go to other places using labels.

function test {
 :myLabel    while ($null -eq $x) {
         $global:foo = foreach ($x in 1..5) {
            if ($x -eq 5) { break myLabel } 
            else {$x} 
        }
        "after the loop x =$x "
    }

"After the while"
}

This is just the last version with a label but now the output looks like the first version.

#545 PS5 ~\Documents\windowsPowershell> $foo = $null 

#546 PS5 ~\Documents\windowsPowershell> $foo

#547 PS5 ~\Documents\windowsPowershell> test
1
2
3
4
After the while

#548 PS5 ~\Documents\windowsPowershell> $foo

I guess the TL;DR version for the above is "Don't put a GOTO in the right hand side of an assignment"

But just when think we know how these things behave What about

function test2 {

try     {"Hello" ; return}
catch   {"Never runs"}
finally {"Good bye"} 
}

If return is immediate successful exit (like throw is an immediate failure) as we have seen finally would never run but this code produces the following output

#532 PS5 ~\Documents\windowsPowershell> test2
Hello
Good bye

Nothing the runs after after the try/catch/finally but the goto effect isn't immediate

return causes the expression to end before no assignment is made.
return can be seen as goto "}" of function
I think it's by design,
$var = Return, Break, Continue ------ This is a bad way of writing code, it is recommended to prohibit

James O'Neill (@jhoneill)

James O'Neill (@jhoneill) That's a very insightful explanation, thanks. :)

I'd consider it more intuitive for the following code:

$Result = if ($true) {
  echo "value"
  return
}

to be interpreted as

try {
  if ($true) {
    accumulate "value"
    return # and flush accumulator
  }
} finally {
  $Result = accumulator
}

rather than as

if ($true) {
  accumulate "value"
  return # and flush accumulator
}
$Result = accumulator

, but I think that we are in agreement here about what's the intuitive behavior.

kasini3000 This pattern is actually kinda useful:

function TryToProcessStuff($Stuff) {
  $PreprocessedStuff = foreach ($s in $Stuff) {
    # if preprocessing fails, return without doing anything more
    try {Preprocess $s} catch {return}
  }
  # do more processing
}

Usually, you would just let the exception bubble up, but in some cases, you want to silence it and treat failure by just not returning any results.

Definitely a bug. Here's a gist with the disassembly.

When an assignment like this happens, the output pipe is temporarily changed to a new one that just writes to a list. This swap happens in a try block, and in the finally it resets back to the old pipe and sends anything still remaining in the temp list to the old pipe.

Line 85 is where the list is supposed to be cleared. PipelineOps.PipelineResult unwraps the list into a scalar or array and clears the list. But that's not in a finally, so when the return on line 60 happens, the clear never happens but the pipe is still flushed.

Dongbo Wang (@daxian-dbw)

Tbh I'm not 100% sure why it flushes there at all, it seems like a logic flaw. I can't quickly think of a case where we make a temporary output pipe and then want to flush it back to the original pipe afterwards. I'm inclined to say every time we generate this pattern, we're expecting the list to be empty from PipelineResult by the time that finally hits.

As a side note, I wonder if there's a good reason that we're building a list period when $null = is used. Seems like potentially a significant chunk of overhead when we could just use a null pipe instead.

/cc Dongbo Wang (@daxian-dbw) James Truher (@JamesWTruher)

Reopen this issue because the fix (#17856) causes a regression reported in #20739, and thus it will be reverted by #21368.
Will let the engine WG review this issue and decide on whether this should be fixed and how to fix it if so.

The Engine WG reviewed this and we decided to mark this as wontfix. This behavior has existed since version 2 and unfortunately we've found that without overhauling that area of the compiler - changing it leads to different breaking changes (see the comment above this one for the attempt).

This issue has been marked as won't fix and has not had any activity for 1 day. It has been closed for housekeeping purposes.

Matěj Kafka (@MatejKafka)

📣 Hey Matěj Kafka (@MatejKafka), how did we do? We would love to hear your feedback with the link below! 🗣️

🔗 https://aka.ms/PSRepoFeedback

Microsoft Forms

Navigation Menu

Repository navigation

If a block inside a function contains `return`, all output is written to the pipeline, even if assigned to a variable #17836

Prerequisites

Steps to reproduce

Expected behavior

Actual behavior

Error details

Environment data

Visuals

4 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

If a block inside a function contains return, all output is written to the pipeline, even if assigned to a variable #17836

Description

Prerequisites

Steps to reproduce

Expected behavior

Actual behavior

Error details

Environment data

Visuals

Activity

fMichaleczek commented on Aug 3, 2022

MatejKafka commented on Aug 3, 2022

fMichaleczek commented on Aug 3, 2022

fMichaleczek commented on Aug 3, 2022

MatejKafka commented on Aug 3, 2022

fMichaleczek commented on Aug 4, 2022

jborean93 commented on Aug 4, 2022

iRon7 commented on Aug 4, 2022

jhoneill commented on Aug 4, 2022

kasini3000 commented on Aug 4, 2022

MatejKafka commented on Aug 4, 2022

SeeminglyScience commented on Aug 5, 2022

SeeminglyScience commented on Aug 5, 2022

4 remaining items

daxian-dbw commented on Mar 24, 2024

SeeminglyScience commented on Apr 1, 2024

microsoft-github-policy-service commented on Apr 3, 2024

microsoft-github-policy-service commented on Apr 3, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

If a block inside a function contains `return`, all output is written to the pipeline, even if assigned to a variable #17836