Repository navigation
ConvertFrom-Json silently throws away duplicated keys with identical casing #5199
Description
If the JSON contains duplicate keys with the same casing, then only the last key is used and the others are silently thrown away. The code in JsonObject.cs here actually caters for this case and would throw an error but it never gets to that stage because JsonConvert.DeserializeObject here does not throw when there are duplicate keys and simply decides to take the last one.
Steps to reproduce
'{ "key":"value1", "key":"value2" }' | ConvertFrom-JsonExpected behavior
Either a warning that duplicated keys were disregarded or an error stating that duplicate keys are not allowed unless e.g something like. a -Force switch is used. Given that this would be a breaking change, it will probably have to be a warning.
Actual behavior
Only the last key is used when there are duplicate keys
key
---
value2Environment data
> $PSVersionTable
Name Value
---- -----
PSVersion 6.0.0-beta.8
PSEdition Core
GitCommitId v6.0.0-beta.8-20-g0d5a6fbd71c31043d0eda82f7b0e2d4fb0431322
OS Microsoft Windows 10.0.15063
Platform Win32NT
PSCompatibleVersions {1.0, 2.0, 3.0, 4.0...}
PSRemotingProtocolVersion 2.3
SerializationVersion 1.1.0.1
WSManStackVersion 3.0
Activity
[-]ConvertFrom-Json silently throws away duplicated keys[/-][+]ConvertFrom-Json silently throws away duplicated keys with identical casing[/+]iSazonov commentedon Oct 23, 2017
It is really bug - Json don't allow duplicate keys with the same case.
SteveL-MSFT commentedon Oct 25, 2017
As best as I can tell JSON doesn't require member names to be unique, only recommended.
iSazonov commentedon Oct 25, 2017
:-) I believe implementations "MUST" follow "SHOULD recommendations".
In any case I think
JsonConvert.DeserializeObjectworks incorrectly.bergmeister commentedon Oct 25, 2017
I found the first 2 answers here interesting and their statement is basically that although it is valid JSON from a puristic point of view, JSON parsing libraries have and can choose different behaviours without conflicting with the standard. Normally one should trust the foundation, which is
Newtonsoft.Json, but I am with Ilya (@iSazonov) that it's behaviour is sub-optimal. If there is a way of at least detecting the duplicates then at least a warning would be nice. A detection should be possible using a custom converter type when callingJsonConvert.DeserializeObjectbut it might result in a higher computational cost.iSazonov commentedon Oct 25, 2017
If the standard say "SHOULD" it allows for a reasonable deviation from the standard. This means that the API must have the option to turn on or off duplicate keys.
bergmeister commentedon Oct 25, 2017
According to the author of
Newtonsoft.Jsonin his comment here it does not seem to be possible, therefore a different library might need to be used unfortunately.SteveL-MSFT commentedon Oct 26, 2017
Is there a known use case that depends on duplicate keys? Otherwise, based on the standard's recommendation, I don't think we must support it and warning the user is sufficient.
iSazonov commentedon Oct 26, 2017
Neton.Json is very popular library - if they don't fix (won't fix) we can ignore this in code and only add notes in docs.
microsoft-github-policy-service commentedon Nov 16, 2023
This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you.
microsoft-github-policy-service commentedon Nov 16, 2023
This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you.
microsoft-github-policy-service commentedon Nov 16, 2023
This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you.
microsoft-github-policy-service commentedon Nov 23, 2023
This issue has been marked as "No Activity" as there has been no activity for 6 months. It has been closed for housekeeping purposes.