Michael Klement (@mklement0) can you look at the test failures?

Steve Lee (@SteveL-MSFT), I'll focus on fixing the fundamental problem first. A preliminary investigation showed that the problem was just one of duplicates in the output, strangely; applying Select-Object -Unique before counting / comparing made the problem go away.

Steve Lee (@SteveL-MSFT), looking into this further my conclusion is that this PR has simply unmasked an unrelated problem that should be investigated separately.

For now, I've restored the semantics of the original test by passing -PowerShellVersion 5.0 to the Get-FormatData commands in the original tests, which makes them succeed.

As for what may be going on:

The current format-data definitions contain duplicates:

Some types are covered both by a dedicated definition (where .TypeNames references a single type) and by a multi-type definition (where .TypeNames contains multiple type names).

$dataWithMultipleTypes = Get-FormatData -PowerShellVersion 5.1 | Where { $_.TypeNames.Count -gt 1 }
$dataWithSingleType = Get-FormatData -PowerShellVersion 5.1 | Where { $_.TypeNames.Count -eq 1 }

# Find all single-type definitions also covered by multiple-type definitions.
$dataWithSingleType | ? { foreach ($fd in $dataWithMultipleTypes) { if ($fd.TypeNames -contains $_) { return $true } } }

This currently yields:

TypeNames                                                        FormatViewDefinition
---------                                                        --------------------
{System.Security.Cryptography.X509Certificates.X509Certificate2} {ThumbprintTable}
{System.Management.Automation.ApplicationInfo}                   {ApplicationInfo, System.Management.Automation.ApplicationInfo}
{System.Management.Automation.ScriptInfo}                        {ScriptInfo, System.Management.Automation.ScriptInfo}
{System.Management.Automation.ExternalScriptInfo}                {ExternalScriptInfo}
{System.Management.Automation.FunctionInfo}                      {FunctionInfo}
{System.Management.Automation.FilterInfo}                        {FilterInfo}
{System.Management.Automation.AliasInfo}                         {AliasInfo, System.Management.Automation.AliasInfo}
{System.Management.Automation.CmdletInfo}                        {System.Management.Automation.CmdletInfo}
{System.Management.Automation.PSDriveInfo}                       {System.Management.Automation.PSDriveInfo}

• Is there a good reason for this duplication?
• If so, what are the precedence rules, and is Update-FormatData expected to load the duplicates too?

Michael Klement (@mklement0) adding | % { Get-FormatData -PowerShellVersion 5.1 -TypeName $_.TypeNames } to your example, it seems that the duplication is for a base type and derived types. So that seems to be fine. As for ordering, with the exception of Out-of-Band types (like errors), it should go in order of the psobject.typenames list.

Steve Lee (@SteveL-MSFT)

There is genuine duplication, with dormant duplicates among the built-in formats - see #11307

I suspect that this duplication ultimately causes the round-trip Get-FormatData test to fail with -PowerShellVersion 5.1 (or this fix in place), but that's a separate issue - see #11308

Steve Lee (@SteveL-MSFT), are there concerns around the public method added to InternalTestHooks? Apart from that, I think this is ready to be merged.

Michael Klement (@mklement0) Please have a look at the merge conflicts

Aditya Patwardhan (@adityapatwardhan), I've resolved the merge conflict.

The PowerShell-CI-static-analysis failure seems like a transient one ("SocketException: Network is unreachable") - not sure how to restart the test.

PoshChan-Bot (@PoshChan) please retry static

Steve Lee (@SteveL-MSFT), successfully started retry of PowerShell-CI-static-analysis

Michael Klement (@mklement0) There seems to be another merge conflict. Please have a look.

Aditya Patwardhan (@adityapatwardhan) , I've resolved the (trivial) conflict in favor of preferring a whitespace after the unary form of the , operator (, <operand> rather than ,<operand>), which looks cleaner to me (it's also what auto-formatting in VSCode does).

Michael Klement (@mklement0) Thanks for your contribution!

🎉v7.1.0-preview.2 has been released which incorporates this pull request.:tada:

Handy links:

• Release Notes

Steve Lee (@SteveL-MSFT)

Please see the update I posted yesterday that still exists in PS 7.2.0 to an issue I previously reported about issues with Get-FormatData / Export-FormatData / Update-FormatData.

Here's the link with a detailed example that is 100% reproducible with PS 7.2.0

#9300

A quick summary would be that:

A roundtrip of format data using Get-FormatData / Export-FormatData / Update-FormatData results in broken output.

The 2 main issues are:

When format data has a custom control it is not returned by Get-FormatData and therefore is lost on the roundtrip.

When format data contains more than 1 TypeName, Get-FormatData seems to get both but ExportFormat data only exports the first 1 so additional types are lost in the roundtrip.

This can be seen in the PS source code

PowerShell-7.2.0/src/System.Management.Automation/FormatAndOutput/DefaultFormatters/FileSystem_format_ps1xml.cs

where the custom control is created and shared between views and where a view has more than 1 TypeName.

It is frustrating to see these problems still exist in the latest PS when they have existed and reported so long ago.

Get-FormatData / Export-FormatData / Update-FormatData are GREAT feaures but the *.format.ps1xml files should never have been removed until the replace actually was fully functional.

Fixing Export-FormatData to include all TypeNames seems like it should be a fairly easy fix since Get-FormatData appears to have them.

Fixing the CustomControl situation seems more complicated since it is shared.

Here's a thought:

DirectoryInfo / FileInfo objects have the PSParentPath note property which looks like this:

string PSParentPath=Microsoft.PowerShell.Core\FileSystem::/home/joe

Add a new note property ParentPath

string ParentPath=/home/joe

Then all of the places that are using the shared custom control for their GroupBy could just be changed to reference ParentPath and there would no longer be a need for the custom control!

        <GroupBy>
            <PropertyName>ParentPath</PropertyName>
        </GroupBy>

This seems like a fairly easy change and by eliminating the shared custom control you remove the more difficult situation to fix.

What do you think?

Joe