DateTimeOffset does not parse a valid ISO-8601 datetime string

Description

DateTimeOffset does not correctly parse a valid ISO-8601 datetime string. Wikipedia claims the current ISO-8601 definition prefers a comma as the decimal mark for the lowest time order present:

However, DateTimeOffset is not capable of parsing a datetime string with a comma decimal mark for the lowest time order when that time order is seconds:

        static void Main(string[] args)
        {
            static void Test()
            {
                var dateString = "2021-04-23T13:04:17,307642270+02:00";

                DateTimeOffset date = DateTimeOffset.MinValue;

                try
                {
                    date = DateTimeOffset.Parse(dateString);
                }
                catch (Exception e)
                {
                    Console.WriteLine($"date: {e.Message}");
                }
                finally
                {
                    Console.WriteLine($"date: {date}");
                }
            }

            Test();
        }

Configuration

Code above running on .NET 5.0.202 on Windows 20H2 (OS Build 19042.928).

@maryamariyan

/cc globalization crew (@maryamariyan @michaelgsharp @safern @tarekgh) in case you all have further thoughts

@chris-steema

@chris-steema You are right that the comma can replace the period in the ISO-8601 but currently we don't support that. We only support the format we use when formatting the DateTimeOffset which always uses the period. You may workaround it by calling ParseExact and passing the format yyyy'-'MM'-'dd'T'HH':'mm':'ss','fffffffzzz.

XML Schema Part 2: Datatypes Second Edition seems to specify only . for fractional seconds. Thus, even if DateTimeOffset.Parse is changed to support comma as well, I think XmlConvert should be left as is.

@tarekgh

Thank you @tarekgh, you're right, please let me explain why this is a concern for us. Our .NET app runs on Linux, and collates information from both Linux and Windows systems. In short, the issue is this:

tofol@neptune:~$ date --iso-8601=ns
2021-04-23T22:07:49,016860795+02:00

Linux appears to understand the decimal mark of ISO-8601 as a comma. What this means on our system, when receiving hundreds of inputs a second from both Windows and Linux, is a degradation of performance as we have to call TryParse to determine which interpretation of ISO-8601 is arriving to our system.

@chris-steema

@chris-steema I am not objecting we try to fix that in the .NET. I was just pointing at some temporary workaround.

Also, are all dates you are receiving in ISO format? I am asking because if this is the case, you can check

if (dateString.Length >= 20 && dateString[19] == ',')
{
    DateTimeOffset dto = DateTimeOffset.ParseExact(dateString, "yyyy'-'MM'-'dd'T'HH':'mm':'ss','fffffffzzz", null);
}

This should mitigate the performance concern.

Although DateTimeOffset.Parse (as opposed to ParseExact) allows excess digits of fractional seconds, it looks like you cannot change the separator character there by cloning CultureInfo.InvariantCulture and then setting its NumberFormat.NumberDecimalSeparator or anything else, because . is hardcoded near the ParseFraction calls:

runtime/src/libraries/System.Private.CoreLib/src/System/Globalization/DateTimeParse.cs

Lines 647 to 654 in 2f740ad

    
           char nextCh = str.Value[str.Index]; 
        
           if (nextCh == '.') 
        
           { 
        
               // While ParseFraction can fail, it just means that there were no digits after 
        
               // the dot. In this case ParseFraction just removes the dot. This is actually 
        
               // valid for cultures like Albanian, that join the time marker to the time with 
        
               // with a dot: e.g. "9:03.MD" 
        
               ParseFraction(ref str, out raw.fraction);

runtime/src/libraries/System.Private.CoreLib/src/System/Globalization/DateTimeParse.cs

Lines 2980 to 2982 in 2f740ad

    
           if (str.Match('.')) 
        
           { 
        
               if (!ParseFraction(ref str, out partSecond))

If .NET removed the following length check, then I think you could use "FFFFFFFFF" (upper case so that it also accepts fewer than nine digits):

runtime/src/libraries/System.Private.CoreLib/src/System/Globalization/DateTimeParse.cs

Line 4052 in 2f740ad

if (tokenLen <= DateTimeFormat.MaxSecondsFractionDigits)

Besides, it is a bit misleading that failing the length check causes result.SetBadDateTimeFailure() to be called even though the problem is in the format string (can never match any input) and not in the input string.

allows excess digits of fractional seconds

.... note that, last time I checked, parsing code handled excess digits in one of two ways (depending on the precise path)

Truncating the excess digits (personally, this is what I prefer)
Rounding UP to the next tick (and may have been done with double...? Been a while since I looked).

@chris-steema

@chris-steema please try the following code:

        public static DateTimeOffset ParseDateTimeOffset(string dateTimeOffsetString)
        {
            DateTimeOffset dateTimeOffset;

            if (dateTimeOffsetString.Length >= 20 && dateTimeOffsetString.Length < 100 && dateTimeOffsetString[19] == ',')
            {
                Span<char> newDateTime = stackalloc char[dateTimeOffsetString.Length];
                dateTimeOffsetString.AsSpan().CopyTo(newDateTime);
                newDateTime[19] = '.';
                dateTimeOffset = DateTimeOffset.Parse(newDateTime);
            }
            else
            {
                dateTimeOffset = DateTimeOffset.Parse(dateTimeOffsetString);
            }

            return dateTimeOffset;
        }

@tarekgh

Thanks again @tarekgh. Maybe just as easy to do:

                try
                {
                    if (dateString.Length >= 20 && dateString[19] == ',')
                    {
                        dateString = string.Concat(dateString.Select((x, i) => 
                        {
                            if (i == 19) return '.';
                            else return x;
                        }));
                    }

                    date = DateTimeOffset.Parse(dateString);
                }

Until the feature request has been implemented.

@chris-steema

@chris-steema my proposal was trying to avoid allocating any new string. but if this is not issue to you, then that is ok to do what you have proposed.

refactor(dotnet#51740): Extracting some constants.

fix(dotnet#51740): Adding comma as valid time fraction delimiter.

Navigation Menu

Repository navigation

DateTimeOffset does not parse a valid ISO-8601 datetime string #51740

Description

Configuration

2 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	char nextCh = str.Value[str.Index];
	if (nextCh == '.')
	{
	// While ParseFraction can fail, it just means that there were no digits after
	// the dot. In this case ParseFraction just removes the dot. This is actually
	// valid for cultures like Albanian, that join the time marker to the time with
	// with a dot: e.g. "9:03.MD"
	ParseFraction(ref str, out raw.fraction);

	if (str.Match('.'))
	{
	if (!ParseFraction(ref str, out partSecond))

DateTimeOffset does not parse a valid ISO-8601 datetime string #51740

Description

Description

Configuration

Activity

GrabYourPitchforks commented on Apr 23, 2021

tarekgh commented on Apr 23, 2021

KalleOlaviNiemitalo commented on Apr 23, 2021

chris-steema commented on Apr 23, 2021

tarekgh commented on Apr 23, 2021

tarekgh commented on Apr 23, 2021

2 remaining items

KalleOlaviNiemitalo commented on Apr 26, 2021

Clockwork-Muse commented on Apr 26, 2021

tarekgh commented on Apr 26, 2021

chris-steema commented on Apr 26, 2021

tarekgh commented on Apr 26, 2021

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions