Upload packages iOS native test xcresult bundle on test failure

On native iOS Xcode build test failures upload the xcresult bundle to the logs location so we can open it in Xcode and see the screenshots, etc. This is possible as of #121535.

This is where it would be updated:
https://github.com/flutter/packages/blob/903f03add169c39d3f19bc70980af3f0862869a9/script/tool/lib/src/common/xcode.dart#L34-L56

Here is how it's done in the framework repo:

flutter/dev/devicelab/lib/framework/ios.dart

Lines 194 to 219 in 874804e

    
           final String resultBundleTemp = Directory.systemTemp.createTempSync('flutter_xcresult.').path; 
        
           final String resultBundlePath = path.join(resultBundleTemp, 'result'); 
        
           final int testResultExit = await exec( 
        
             'xcodebuild', 
        
             <String>[ 
        
               '-workspace', 
        
               'Runner.xcworkspace', 
        
               '-scheme', 
        
               'Runner', 
        
               '-configuration', 
        
               configuration, 
        
               '-destination', 
        
               destination, 
        
               '-resultBundlePath', 
        
               resultBundlePath, 
        
               'test', 
        
               'COMPILER_INDEX_STORE_ENABLE=NO', 
        
               if (developmentTeam != null) 
        
                 'DEVELOPMENT_TEAM=$developmentTeam', 
        
               if (codeSignStyle != null) 
        
                 'CODE_SIGN_STYLE=$codeSignStyle', 
        
               if (provisioningProfile != null) 
        
                 'PROVISIONING_PROFILE_SPECIFIER=$provisioningProfile', 
        
             ], 
        
             workingDirectory: platformDirectory, 
        
             canFail: true,

flutter/dev/devicelab/lib/framework/ios.dart

Lines 226 to 241 in 874804e

    
           if (xcresultBundle.existsSync()) { 
        
             // Zip the test results to the artifacts directory for upload. 
        
             final String zipPath = path.join(dumpDirectory.path, 
        
                 '$testName-${DateTime.now().toLocal().toIso8601String()}.zip'); 
        
             await exec( 
        
               'zip', 
        
               <String>[ 
        
                 '-r', 
        
                 '-9', 
        
                 '-q', 
        
                 zipPath, 
        
                 path.basename(xcresultBundle.path), 
        
               ], 
        
               workingDirectory: resultBundleTemp, 
        
               canFail: true, // Best effort to get the logs. 
        
             );

where dumpDirectory is platform.environment['FLUTTER_LOGS_DIR']

flutter/dev/devicelab/lib/framework/host_agent.dart

Lines 26 to 28 in 36e09ff

    
           final String? directoryPath = _platform.environment['FLUTTER_LOGS_DIR']; 
        
           if (directoryPath != null) { 
        
             _dumpDirectory = _fileSystem.directory(directoryPath)..createSync(recursive: true);

This would particularly be helpful when tests are run in parallel because evidently it doesn't show the failures in stdout:
https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket/8754096679487031185/+/u/Run_package_tests/native_test/stdout

@stuartmorgan

@stuartmorgan I thought we had an issue for this but I can't find it.

random thought - maybe we can share some common infra for these 2 repos?

[ci] Upload screenshots, logs, and Xcode test results for drive and i…

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.

[ci] Upload screenshots, logs, and Xcode test results for drive and i…

Navigation Menu

Repository navigation

Upload packages iOS native test xcresult bundle on test failure #144795

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	final String resultBundleTemp = Directory.systemTemp.createTempSync('flutter_xcresult.').path;
	final String resultBundlePath = path.join(resultBundleTemp, 'result');
	final int testResultExit = await exec(
	'xcodebuild',
	<String>[
	'-workspace',
	'Runner.xcworkspace',
	'-scheme',
	'Runner',
	'-configuration',
	configuration,
	'-destination',
	destination,
	'-resultBundlePath',
	resultBundlePath,
	'test',
	'COMPILER_INDEX_STORE_ENABLE=NO',
	if (developmentTeam != null)
	'DEVELOPMENT_TEAM=$developmentTeam',
	if (codeSignStyle != null)
	'CODE_SIGN_STYLE=$codeSignStyle',
	if (provisioningProfile != null)
	'PROVISIONING_PROFILE_SPECIFIER=$provisioningProfile',
	],
	workingDirectory: platformDirectory,
	canFail: true,

	if (xcresultBundle.existsSync()) {
	// Zip the test results to the artifacts directory for upload.
	final String zipPath = path.join(dumpDirectory.path,
	'$testName-${DateTime.now().toLocal().toIso8601String()}.zip');
	await exec(
	'zip',
	<String>[
	'-r',
	'-9',
	'-q',
	zipPath,
	path.basename(xcresultBundle.path),
	],
	workingDirectory: resultBundleTemp,
	canFail: true, // Best effort to get the logs.
	);

	final String? directoryPath = _platform.environment['FLUTTER_LOGS_DIR'];
	if (directoryPath != null) {
	_dumpDirectory = _fileSystem.directory(directoryPath)..createSync(recursive: true);

Upload packages iOS native test xcresult bundle on test failure #144795

Description

Activity

jmagman commented on Mar 7, 2024

hellohuanlin commented on Mar 8, 2024

github-actions commented on Nov 20, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions