Serializing bigint value to JSON emits incorrect JSON

Prerequisites

Write a descriptive title.
Make sure you are able to repro it on the latest released version
Search the existing issues.
Refer to the FAQ.
Refer to Differences between Windows PowerShell 5.1 and PowerShell.

Steps to reproduce

PS> $i = [bigint]"93839802329326"
PS> $obj = @{"val" = $i}
PS> ConvertTo-Json $obj 

{
    "val":  {
                "IsPowerOfTwo":  false,
                "IsZero":  false,      
                "IsOne":  false,       
                "IsEven":  true,
                "Sign":  1
            }
}

Note that casting the variable inline as [int64] does work correctly:

PS> $i = [bigint]"93839802329326"
PS> $obj = @{"val" = [int64]$i}
PS> ConvertTo-Json $obj 
{
    "val":  93839802329326
}

and casting the variable inline as [bigint] does not:

PS> $i = [bigint]"93839802329326"
PS> $obj = @{"val" = [bigint]$i}
PS> ConvertTo-Json $obj 
{
  "val": {
    "IsPowerOfTwo": false,
    "IsZero": false,
    "IsOne": false,
    "IsEven": true,
    "Sign": 1
  }
}

Expected behavior

{
    "val":  93839802329326
}

Actual behavior

{
    "val":  {
                "IsPowerOfTwo":  false,
                "IsZero":  false,      
                "IsOne":  false,       
                "IsEven":  true,
                "Sign":  1
            }
}

Error details

No response

Environment data

Name                           Value
----                           -----
PSVersion                      7.4.0
PSEdition                      Core
GitCommitId                    7.4.0
OS                             Microsoft Windows 10.0.19045
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

Visuals

No response

Good point, but your issue is a duplicate of the following one (which fell victim to the dreaded "Resolution-No Activity" bot-assigned label and - often inappropriate - associated automatic closure):

convertto-json bigint is not properly serialized #9207

I guess this needs some authoritive answer as to what should happen here. I see two valid options to go with:

1. Serialize to the raw number (my vote)

[System.Numerics.BigInteger]::new("9223372036854775808") | ConvertTo-Json
# 9223372036854775808

This would technically be a breaking change as now the properties are no longer serialized so the output is different but it does mirror how such a value is deserialized making it roundtrippable. While a breaking change I feel like the chances of people relying on such a format is minimal as it's mostly useless today and even if they do really want it they can create such an object manually before serialization.

# BigInteger
('9223372036854775808' | ConvertFrom-Json).GetType()

2. Add a `Value` key to the output

[System.Numerics.BigInteger]::new("9223372036854775808") | ConvertTo-Json
# {
#   "IsPowerOfTwo": true,
#   "IsZero": false,
#   "IsOne": false,
#   "IsEven": true,
#   "Sign": 1,
#   "Value": 9223372036854775808
# }

This is backwards compatible as we only have an extra property. We do run the risk if BigInteger ever adds a Value property itself or if someone has added this as an ETS property.

What I hate to see is a new -Switch parameter to decide this, I don't think it warrants the extra complexity it adds to the UX to be able to have the old and "new" behaviour for a BigInteger. I think pwsh should just do the right thing out of the box.

I'm happy to implement the fix but we would need an answer from the working group as to what route to go with before I do the work.

I think serialising to the value itself (option 1) makes the most sense because it's consistent with behaviour in other number scenarios. I was quite confused with the output and was fortunate to find some clues here.

While technically a breaking change, given that the value itself is missing I would guess that there aren't many hard dependencies on those properties today. Consequently, I think the most likely scenario is deserialising stored JSON that has inadvertently included the current structure.

As it turns out, BigInteger cannot be round tripped at all so I think that this a .NET runtime bug.

using System.Numerics;
using System.Text.Json;

var val = new Val
{
    BigInt = new BigInteger(179032540293482304),
    Long = 41555532453456346
};

var json = JsonSerializer.Serialize(val);
Console.WriteLine(json);

var obj = JsonSerializer.Deserialize<Val>(json);
Console.WriteLine($"{obj.BigInt} {obj.Long}");

public class Val
{
    public BigInteger BigInt { get; set; }
    public long Long { get; set; }
}

Output:

{"BigInt":{"IsPowerOfTwo":false,"IsZero":false,"IsOne":false,"IsEven":true,"Sign":1},"Long":41555532453456346}
0 41555532453456346

When deserialising to a strong type, backwards compatibility could maintain the existing behaviour and default the value to zero. In fuzzy type scenarios like powershell, I agree with you that it needs discussion amongst those who understand the internals.

rifuller

rifuller, note that it isn't System.Text.Json that underlies the PowerShell cmdlets, but third-party library Json.NET.

The PowerShell cmdlets already support deserializing arbitrarily large integer values (see below), so it makes sense to also serialize them as such.

# Existing support for *deserializing* large integer values:

# Too big to be [Int64] -> 
# [bigint] in PS Core, [decimal] in WinPS
('10000000000000000000' | ConvertFrom-Json).GetType().Name

# Too big to be [decimal] -> 
# [bigint] in PS Core, [double] in WinPS
('100000000000000000000000000000' | ConvertFrom-Json).GetType().Name

As an aside: note the difference in types chosen between Windows PowerShell and PowerShell Core.

While there can generally be no expectation of round-tripping - given that the specific original number type isn't recorded on to-JSON conversion - on deserialization it would arguably have made more sense to go from Int64 -> Decimal -> BigInteger, instead of going straight from Int64 to BigInteger.

As an aside, re a potential .NET bug: System.Text.Json by design doesn't support [bigint] (the largest supported type capable of storing integers accurately is [decimal]), but allows implementing custom converters to support arbitrary types.

I wanted to see if it was actually possible in the first place to do option 1 and opened #21000. It is possible so just waiting to see what the WG says about this option.

I wanted to see if it was actually possible in the first place to do option 1 and opened #21000. It is possible so just waiting to see what the WG says about this option.

Oh, thanks for putting up a PR. I'm working around my issue by hard casting it so I'll just keep an eye on my Github emails for any updates.

rifuller

📣 Hey rifuller, how did we do? We would love to hear your feedback with the link below! 🗣️

🔗 https://aka.ms/PSRepoFeedback

Microsoft Forms

Navigation Menu

Repository navigation

Serializing bigint value to JSON emits incorrect JSON #20989

Prerequisites

Steps to reproduce

Expected behavior

Actual behavior

Error details

Environment data

Visuals

1. Serialize to the raw number (my vote)

2. Add a `Value` key to the output

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Serializing bigint value to JSON emits incorrect JSON #20989

Description

Prerequisites

Steps to reproduce

Expected behavior

Actual behavior

Error details

Environment data

Visuals

Activity

mklement0 commented on Jan 3, 2024

jborean93 commented on Jan 4, 2024

1. Serialize to the raw number (my vote)

2. Add a Value key to the output

argium commented on Jan 4, 2024

mklement0 commented on Jan 4, 2024

jborean93 commented on Jan 5, 2024

argium commented on Jan 9, 2024

microsoft-github-policy-service commented on Feb 27, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

2. Add a `Value` key to the output