You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
convertto-json bigint is not properly serialized #9207
JavaScript implementations cannot read large integers >53 bytes precision, without loss of precision, so it looks like there's no clearly correct answer here, and the main choices are:
Pick what another system does and do that; Newtonsoft Json.Net encodes larger integers than JavaScript can read safely and risks compatibility problems, Twitter encodes large numbers as strings and cautions developers to expect that, Google V8 refuses to serialize BigInts saying it isn't supported.
Encode small values, but error for BigIntegers representing larger values. (Might be unpredictable)
Do what it does now, with that object encoding, which isn't useful, but is consistent - doesn't promise anything which it can't deliver.
Defer until such time as ECMAScript / Google have agreed ways of serialising BigInts to JSON, and do what they do.
Background links:
ECMA-404 JSON standard doesn't put a limit on how large a value can be serialized as an integer in JSON, but JavaScript is limited to reading 53 bytes precision so it can't read integers larger than that, and returns truncated values. Twitter works around this by using strings to encode its 64-bit numeric identifiers and cautions developers to expect that.
I think JavaScript is one of thousands of use-cases for JSON and probably a much small minority among those employed in PowerShell. The primary use cases for JSON in PowerShell are serializing and desalinizing data for interaction with REST APIs or for consumption or storage as settings. I see no reason to deny serializing BigInt. There are possibly REST APIs that would depend on such things.
But, I'd be more interested in what the .NET Core 3.0 JSON implementation does with them. If we still plan to move from NewtonSoft to the .NET Core native implementation, it probably behooves us to do as they do.
Note that when such software is used, numbers that are integers and
are in the range [-(2^53)+1, (2^53)-1] are interoperable in the
sense that implementations will agree exactly on their numeric
values.
We already support Int64 conversion which is out of range for general interoperability. Int64.MaxValue 9223372036854775807 is greater than Number.MAX_SAFE_INTEGER 9007199254740991. There is no strict limit on this per the RFC, just a warning with regards to interoperability.
If PowerShell was written in Java, then interoperability with jdk12 would be critical. However, we are based on .NET Core so we are bound by their support and limitations. Thus, it is far more important for us to consider .NET Core 3+'s implementation than any other language for any area not explicitly restricted by RFC-8259.
I would like to note that if we do end up converting BigInt to a JSON Number by, it will be a breaking change. It could be implemented in a non-breaking manner as a configurable option. However, I suspect what most users would consider the default behavior would be to convert BigInt to JSON Number. Whether we wish to not make it option or what is the default behavior needs to be considered carefully.
This issue creates inconsistent behavior with desktop PowerShell 5 though. Since ConvertFrom-Json prioritizes BigInteger over Decimal in PSCore, you can't go from JSON back to JSON without breaking the result so long as BigInteger isn't serialized correctly. This doesn't repro in Windows PowerShell.
This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you.
microsoft-github-policy-service commented on Nov 16, 2023
This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you.
microsoft-github-policy-service commented on Nov 16, 2023
This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you.
Activity
HumanEquivalentUnit commentedon Mar 22, 2019
JavaScript implementations cannot read large integers >53 bytes precision, without loss of precision, so it looks like there's no clearly correct answer here, and the main choices are:
Background links:
ECMA-404 JSON standard doesn't put a limit on how large a value can be serialized as an integer in JSON, but JavaScript is limited to reading 53 bytes precision so it can't read integers larger than that, and returns truncated values. Twitter works around this by using strings to encode its 64-bit numeric identifiers and cautions developers to expect that.
Chrome's JavaScript engine V8 gained support for BigInt as a data type in 2018 but cannot serialize them to JSON, with
VM134:1 Uncaught TypeError: Do not know how to serialize a BigInt.ECMAScript has an open issue for what to do about JSON serialization of BigInts, appears inconclusive, with lots of discussion about compatibility.
iSazonov commentedon Mar 23, 2019
/cc Mark Kraus (@markekraus) Have you thoughts?
markekraus commentedon Mar 23, 2019
I think JavaScript is one of thousands of use-cases for JSON and probably a much small minority among those employed in PowerShell. The primary use cases for JSON in PowerShell are serializing and desalinizing data for interaction with REST APIs or for consumption or storage as settings. I see no reason to deny serializing BigInt. There are possibly REST APIs that would depend on such things.
But, I'd be more interested in what the .NET Core 3.0 JSON implementation does with them. If we still plan to move from NewtonSoft to the .NET Core native implementation, it probably behooves us to do as they do.
iSazonov commentedon Mar 23, 2019
Mark Kraus (@markekraus) If we don't still have an issue tracking this please open new issue. Thanks!
markekraus commentedon Mar 24, 2019
I disagree. RFC-8259 is the the standard. How any language implements JSON should be based on that standard.
The relevant part of the RFC is here https://tools.ietf.org/html/rfc8259#section-6
We already support Int64 conversion which is out of range for general interoperability. Int64.MaxValue
9223372036854775807is greater than Number.MAX_SAFE_INTEGER9007199254740991. There is no strict limit on this per the RFC, just a warning with regards to interoperability.If PowerShell was written in Java, then interoperability with jdk12 would be critical. However, we are based on .NET Core so we are bound by their support and limitations. Thus, it is far more important for us to consider .NET Core 3+'s implementation than any other language for any area not explicitly restricted by RFC-8259.
markekraus commentedon Mar 24, 2019
I would like to note that if we do end up converting BigInt to a JSON Number by, it will be a breaking change. It could be implemented in a non-breaking manner as a configurable option. However, I suspect what most users would consider the default behavior would be to convert BigInt to JSON Number. Whether we wish to not make it option or what is the default behavior needs to be considered carefully.
zackste commentedon Jan 14, 2020
This issue creates inconsistent behavior with desktop PowerShell 5 though. Since ConvertFrom-Json prioritizes BigInteger over Decimal in PSCore, you can't go from JSON back to JSON without breaking the result so long as BigInteger isn't serialized correctly. This doesn't repro in Windows PowerShell.
Windows PowerShell 5.1.18362.145:
The intermediate type is interpreted as Decimal.
PSCore 7.0.0-rc.1
The intermediate type is interpreted as BigInteger.
microsoft-github-policy-service commentedon Nov 16, 2023
This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you.
microsoft-github-policy-service commentedon Nov 16, 2023
This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you.
microsoft-github-policy-service commentedon Nov 16, 2023
This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you.
JimbobSquelchy commentedon Nov 16, 2023
I'm running latest 7.3.9 version of PowerShell. Running the command provided by Zack (@zackste) still produces the incorrect output:
microsoft-github-policy-service commentedon Nov 23, 2023
This issue has been marked as "No Activity" as there has been no activity for 6 months. It has been closed for housekeeping purposes.