Repository navigation
Need a full set of encoding tests #3488
Description
Now we don't have a full set of encoding tests (only for redirections). We need to create them during future Encoding RFC implementation (?).
In #3467 (discussion Issue #3248) we fix Default/OEM encoding behavior PowerShell Core on Windows (as in Windows PowerShell) but don't add tests (waiting the RFC). The simplest test (from Michael Klement (@mklement0)) is:
# Setup:
# Create a no-BOM UTF-8 file with the following literal content:
# 'ö'
# which, when executed as a PS script, should echo 'ö' back, IF the script
# was correctly decoded from UTF-8 by PS.
# UTF-8 bytes: 0x27 (single quote), 0xc3 0xb6 (encoding of 'ö', U+00F6), 0x27 (single quote)
[byte[]] (0x27, 0xc3, 0xb6, 0x27) | Set-Content -Encoding Byte /tmp/$PID.ps1
# Test: See if the 'ö' is echoed back correctly.
# Should return $True
'ö' -eq (& /tmp/$PID.ps1)We need test for:
- File provider cmdlets
- Utility (and other) cmdlets
- Redirections
- Console Host
- Web cmdlets (Discussion Issue Webcmdlets should parse the <html><head><meta charset="foo"> attribute for the correct encoding if not in http header #3267)
- PowerShell engine source file reading
Activity
mklement0 commentedon Apr 5, 2017
Ilya (@iSazonov):
The test snippet tests how PowerShell itself reads source code that has no BOM, so can you please add that as a separate action item (check box) to the list of tests?
(Perhaps that behavior is implied by the other tests, but I think it should still be a separate one.)
(And, on a more quibbly note: Can you add the word "character" before "encoding" to the title, so that it's clear what kind of encoding is being referred to?)
And here are the
Set-Content/Get-Contenttest snippets with BOM-less UTF-8 encoding in effect:iSazonov commentedon Apr 5, 2017
We have never used "character" before so it can confuse us even more. (?)
mklement0 commentedon Apr 5, 2017
Ilya (@iSazonov):
Not that it will matter much in this instance, but just for the record and for future instances where the distinction may matter:
We are discussing character encodings here.
Encoding is a far more generic term, of which character encoding is just one instance.
Given that the issues in this repo span all sorts of topics, giving sufficient context is preferable.
P.S.: In light of the above, I would have preferred the title "Default Character Encoding" for the RFC.
iSazonov commentedon Apr 5, 2017
Michael Klement (@mklement0) In the code we use "encoding". So let us defer the question to the discretion of the mantainers.
mklement0 commentedon Apr 5, 2017
Ilya (@iSazonov):
Sounds good.
It's perfectly understandable to shorten "character encoding" to "encoding" in a given, narrow context, where there's no risk of ambiguity.
My only point is that in a larger context the added specificity can be helpful - both for searching topics by keywords and for quick comprehension.