Use app deltas for faster and smaller incremental iOS app installs

@anirudhb

Figure out how to get app deltas working in Xcode/ios-deploy to avoid recopying the same files across installs.

@anirudhb started this investigation #60408 (comment):

fwiw I did try the new ios-deploy with app deltas, but it seems that Flutter builds the app in such a way that app deltas copies the entire app again (including Flutter.framework, which shouldn't've even changed.)

The app install time is extremely noticeable over wireless, whereas over USB it's not as significant.

and #60408 (comment):

I did some investigation on this, and the gist of it is that Xcode only uses modification time to determine whether to copy a file or not. Some things in the build that change modification time include code signing and framework thinning. Cocoapods additionally does this as well (this could be an upstream issue). After modifying the scripts to preserve modification time across codesign/thin, ~~Xcode was successfully able to use app deltas!~~ edit: Crazy run errors including a white screen on launch.

A proper implementation of this would probably involve hash comparisons, since Xcode doesn't do this for some reason when computing app deltas.

Also ios-control/ios-deploy#475 (comment)

This feature will bring second and subsequent install time down from minutes to seconds. The incremental app installation with app deltas only works if the files in the app bundle haven't been modified (by time stamp) since the last installation.

The dart compilation is already skipped on second builds, so the modifications that would need to be skippable are:

Thinning the App framework Stop trying to thin iOS App.framework #76834
Thinning the Flutter framework Move iOS Flutter.framework thinning into copy assemble build target #77007
Bitcode stripping the Flutter framework Bitcode strip Flutter.framework in assemble build target #77329
Codesigning Move iOS codesigning into assemble build target #77664
Embedding the App and Flutter frameworks
Adding NSBonjourServices to the Info.plist

Then:

Implement app deltas (this issue)
Add benchmarks for a second-build to make sure build time doesn't regress Add debug_second_compile_millis benchmark metric #77420

This would be a superb fix!

The Thin Binary build script inputs would be (with USE_RECURSIVE_SCRIPT_INPUTS_IN_SCRIPT_PHASES=YES):

${BUILT_PRODUCTS_DIR}/App.framework
${BUILT_PRODUCTS_DIR}/Flutter.framework
${TARGET_BUILD_DIR}/${INFOPLIST_PATH}

and outputs:

${TARGET_BUILD_DIR}/${FRAMEWORKS_FOLDER_PATH}/App.framework
${TARGET_BUILD_DIR}/${FRAMEWORKS_FOLDER_PATH}/Flutter.framework

Adding NSBonjourServices to the Info.plist

However the Info.plist change is the thing that's tripping the app deltas up. On an incremental install, Xcode will regenerate the Info.plist if it's been changed. When it changes, flutter needs to re-add the NSBonjourServices key. And they thrash so the app always has a timestamp change and the entire app is re-installed every time.

That Info.plist change is only necessary when --publish-port is true (which is by default false on flutter drive), maybe there's a way to pass that info along so the Info.plist can be changed only when needed...

PR for non-prebuilt binaries (the normal flutter run flow without --use-application-binary) at #77756.

I ran some numbers (on my machine) building and installing the new non-trivial gallery app after a flutter clean
The first number is the amount of time flutter run takes where I force an exit as soon as the app is launched. The second number is the same, run a second time with no clean.

	stable	master	this PR
1st	1:15.08	1:14.95	1:17.53
2nd	48.379	45.479	31.096

So the improvements with thinning (#76834 and #77007), bitcode stripping (#77329), and codesigning (#77664) improved matters by ~3 seconds. App deltas gave us another ~14 seconds.

#67580 (comment) sounds like this is a recent-ish regression right? Would #77420 have caught this if something similar happened again?

However the Info.plist change is the thing that's tripping the app deltas up. On an incremental install, Xcode will regenerate the Info.plist if it's been changed. When it changes, flutter needs to re-add the NSBonjourServices key. And they thrash so the app always has a timestamp change and the entire app is re-installed every time.

That wasn't true at all, I thought the entire app is re-installed every time because the app was being re-signed, caused by the Info.plist change. The problem was actually because flutter was copying around the app bundle to a new directory every time it built, which was updating all the timestamps and defeating the incremental installer.

flutter/packages/flutter_tools/lib/src/ios/mac.dart

Lines 433 to 441 in 047dcc7

    
           if (globals.fs.isDirectorySync(outputDir)) { 
        
             // Previous output directory might have incompatible artifacts 
        
             // (for example, kernel binary files produced from previous run). 
        
             globals.fs.directory(outputDir).deleteSync(recursive: true); 
        
           } 
        
           copyDirectory( 
        
             globals.fs.directory(expectedOutputDirectory), 
        
             globals.fs.directory(outputDir), 
        
           );

I fixed that by making it an rsync instead #77756.

flutter/packages/flutter_tools/lib/src/ios/mac.dart

Lines 435 to 440 in 08b225e

    
           // rsync instead of copy to maintain timestamps to support incremental 
        
           // app install deltas. Use --delete to remove incompatible artifacts 
        
           // (for example, kernel binary files produced from previous run). 
        
           await globals.processUtils.run( 
        
             <String>[ 
        
               'rsync',

#67580 (comment) sounds like this is a recent-ish regression right? Would #77420 have caught this if something similar happened again?

I don't think it's a regression, it's just preventing Thin Binary from being skippable with input and output files from #67580 (comment). Which isn't a big deal now that app deltas are working, that script phase only takes a few milliseconds to run a second time, and while it does make the app binary resign and therefore reupload to the device, it's a really small file. The small amount of time is probably not worth regressing the local network permission behavior. The main chunk of remaining time is coming from elsewhere.

The remaining work being tracking is to implement app deltas with flutter run --use-application-binary.

The flutter run case was done in #77756 and is in 2.2.0-10.0.pre. #90842 is tracking the less-common --use-application-binary case.
Closing in favor of that more focused issue.

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.

Navigation Menu

Repository navigation

Use app deltas for faster and smaller incremental iOS app installs #67580

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	if (globals.fs.isDirectorySync(outputDir)) {
	// Previous output directory might have incompatible artifacts
	// (for example, kernel binary files produced from previous run).
	globals.fs.directory(outputDir).deleteSync(recursive: true);
	}
	copyDirectory(
	globals.fs.directory(expectedOutputDirectory),
	globals.fs.directory(outputDir),
	);

	// rsync instead of copy to maintain timestamps to support incremental
	// app install deltas. Use --delete to remove incompatible artifacts
	// (for example, kernel binary files produced from previous run).
	await globals.processUtils.run(
	<String>[
	'rsync',

Use app deltas for faster and smaller incremental iOS app installs #67580

Description

Activity

xster commented on Oct 13, 2020

jmagman commented on Mar 5, 2021

xster commented on Mar 5, 2021

jmagman commented on Mar 9, 2021

jmagman commented on Mar 10, 2021

xster commented on Mar 10, 2021

jmagman commented on Mar 11, 2021

jmagman commented on Mar 11, 2021

jmagman commented on Sep 28, 2021

github-actions commented on Oct 12, 2021

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions