ConvertFrom-Json silently throws away duplicated keys with identical casing

If the JSON contains duplicate keys with the same casing, then only the last key is used and the others are silently thrown away. The code in JsonObject.cs here actually caters for this case and would throw an error but it never gets to that stage because JsonConvert.DeserializeObject here does not throw when there are duplicate keys and simply decides to take the last one.

Steps to reproduce

'{ "key":"value1", "key":"value2" }' | ConvertFrom-Json

Expected behavior

Either a warning that duplicated keys were disregarded or an error stating that duplicate keys are not allowed unless e.g something like. a -Force switch is used. Given that this would be a breaking change, it will probably have to be a warning.

Actual behavior

Only the last key is used when there are duplicate keys

key
---
value2

Environment data

> $PSVersionTable
Name                           Value
----                           -----
PSVersion                      6.0.0-beta.8
PSEdition                      Core
GitCommitId                    v6.0.0-beta.8-20-g0d5a6fbd71c31043d0eda82f7b0e2d4fb0431322
OS                             Microsoft Windows 10.0.15063
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

It is really bug - Json don't allow duplicate keys with the same case.

As best as I can tell JSON doesn't require member names to be unique, only recommended.

:-) I believe implementations "MUST" follow "SHOULD recommendations".

In any case I think JsonConvert.DeserializeObject works incorrectly.

Ilya (@iSazonov)

I found the first 2 answers here interesting and their statement is basically that although it is valid JSON from a puristic point of view, JSON parsing libraries have and can choose different behaviours without conflicting with the standard. Normally one should trust the foundation, which is Newtonsoft.Json, but I am with Ilya (@iSazonov) that it's behaviour is sub-optimal. If there is a way of at least detecting the duplicates then at least a warning would be nice. A detection should be possible using a custom converter type when calling JsonConvert.DeserializeObject but it might result in a higher computational cost.

If the standard say "SHOULD" it allows for a reasonable deviation from the standard. This means that the API must have the option to turn on or off duplicate keys.

According to the author of Newtonsoft.Json in his comment here it does not seem to be possible, therefore a different library might need to be used unfortunately.

Is there a known use case that depends on duplicate keys? Otherwise, based on the standard's recommendation, I don't think we must support it and warning the user is sufficient.

Neton.Json is very popular library - if they don't fix (won't fix) we can ignore this in code and only add notes in docs.

This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you.

This issue has been marked as "No Activity" as there has been no activity for 6 months. It has been closed for housekeeping purposes.

Navigation Menu

Repository navigation

ConvertFrom-Json silently throws away duplicated keys with identical casing #5199

Steps to reproduce

Expected behavior

Actual behavior

Environment data

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

ConvertFrom-Json silently throws away duplicated keys with identical casing #5199

Description

Steps to reproduce

Expected behavior

Actual behavior

Environment data

Activity

iSazonov commented on Oct 23, 2017

SteveL-MSFT commented on Oct 25, 2017

iSazonov commented on Oct 25, 2017

bergmeister commented on Oct 25, 2017

iSazonov commented on Oct 25, 2017

bergmeister commented on Oct 25, 2017

SteveL-MSFT commented on Oct 26, 2017

iSazonov commented on Oct 26, 2017

microsoft-github-policy-service commented on Nov 16, 2023

microsoft-github-policy-service commented on Nov 16, 2023

microsoft-github-policy-service commented on Nov 16, 2023

microsoft-github-policy-service commented on Nov 23, 2023

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions