Mac LUCI builders' Xcode should include iOS 14 SDK

For this PR to build, it needs to have at least iOS SDK version 13.4, and it's on 13.0 right now.

An example of a build failing because of this is this iOS Engine build

@godofredoc

Any ETA on this? It's blocking some feature work.

Does this need be updated across engine, sdk and devicelab?

We can move the xcode version on the sdk and engine with a small amount of effort but for devicelab that can take up to mid q4.

Hmm. I think that if the engine build is updated, then the engine artifacts should work on the devicelab.

@zanderso

\cc @zanderso @jonahwilliams @jmagman any objections with updating the engine ios version to 13.4?

Do you mean add the iOS 13.4 SDK to the current version of Xcode? We can do that with some symlinking with Xcode, I think.

We can't upgrade the engine builder clang version (Xcode version) beyond the minimum Xcode version enforced in the tool, or we will break bitcode again. Current minimum is Xcode 11.0.0, which shipped the iOS 13.0 SDK.
See internal documentation and internal post-mortem.

@nturgut

\cc @nturgut who I think has added newer SDKs to older versions of Xcode in LUCI.
\cc @dnfield to bring down the hammer on us all if we break bitcode again.

The builders automatically select the mac sdk correlated to the requested ios version. We only provide xcode version.

We can update both sdk and engine xcode version at the same time but that will most likely break the devicelab.

Yes, I think just adding the SDK would work.

I have Xcode 12 installed (no symlink SDK tricks) and when I run it, it works. The difference I see is that the Xcode 12 toolchain swapped out -ios_simulator_version_min 8.0.0 for -platform_version ios-simulator 8.0.0 14.0. Not sure if that's the problem, though.

  "<redacted>Xcode-12.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld" -demangle -lto_library <redacted>Xcode-12.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/libLTO.dylib 
  -no_deduplicate 
  -dynamic 
  -dylib 
  -arch x86_64 
  -dylib_install_name @rpath/App.framework/App 
  -platform_version ios-simulator 8.0.0 14.0 
  -bitcode_bundle 
  -bitcode_process_mode marker 
  -syslibroot <redacted>Xcode-12.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator14.0.sdk 
  -o /<redacted>/engine/src/flutter/testing/scenario_app/./build/ios/App.framework/App 
  -rpath @executable_path/Frameworks -rpath @loader_path/Frameworks 
  /var/folders/bx/pyplw7v92lj58gm3_lztz5sc00mfq2/T/moo-789812.o -lSystem <redacted>Xcode-12.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/12.0.0/lib/darwin/libclang_rt.iossim.a

I have a bad feeling TAPI had to get a lot more sophisticated with archs/targets with the macOS ARM changes, and the Xcode 11 toolchain won't be able to handle it. Xcode 11 SDK is using tapi-tbd-v3 and looks like:

--- !tapi-tbd-v3
archs:           [ i386, x86_64 ]
...
platform:        ios
...
  - archs:           [ i386, x86_64 ]

and Xcode 12 SDK is on version 4:

tbd-version:     4
targets:         [ i386-macos, x86_64-macos, x86_64-maccatalyst, arm64-macos, arm64-maccatalyst ]
...
parent-umbrella:
  - targets:         [ i386-macos, i386-maccatalyst, x86_64-macos, x86_64-maccatalyst, 
                       arm64-macos, arm64-maccatalyst, arm64e-macos, arm64e-maccatalyst ]
...

@LongCatIsLooong

@LongCatIsLooong Do you know if the engine frameworks build with iOS 14 SDK against the Xcode 11 toolchain? I'm not sure how to override the buildtools clang paths.

I feel like it would have the same problem, but if it does happen to work, can we use the Xcode 11 hybrid package on the engine framework builders only, as opposed to any of the other builders running tests, etc?

they do (at least some of them).
I'll open a test pr and point SYSROOT to the ios 13 sdk, and see if there's anything else needs fixing.

Update: that was it. Everything passed on the mac ios engine try builder.

But if it's pointing on the iOS 13 SYSROOT then it's no better than we have now, right?

I was able to get it working by symlinking the Xcode 12 linker into the Xcode 11 toolchain.

$ ln -s /path/to/Xcode-12.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld /path/to/Xcode-11.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld

The only place I had to use the iOS 13 sdk is the scenario app: https://github.com/flutter/engine/pull/21614/files, the ios frameworks are generated using the iOS 14 sdk. Does the scenario app have to use the iOS 14 sdk?

It looks like it's happening building the framework, too, though not sure why it's a warning and not an error.
https://logs.chromium.org/logs/flutter/led/weiyuhuang_google.com/13a6e979e3da140e383a604d97ef5dc322152c5ca429a9d4404599256d244540/+/steps/build_ios_debug_sim/0/stdout

[3791/3800] SOLINK libFlutter.dylib libFlutter.dylib.TOC
ld: warning: building for iOS Simulator, but linking in .tbd file (/opt/s/w/ir/cache/osx_sdk/XCode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator14.0.sdk/usr/lib/system/libsystem_kernel.tbd) built for macOS
ld: warning: building for iOS Simulator, but linking in .tbd file (/opt/s/w/ir/cache/osx_sdk/XCode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator14.0.sdk/usr/lib/system/libsystem_platform.tbd) built for macOS
ld: warning: building for iOS Simulator, but linking in .tbd file (/opt/s/w/ir/cache/osx_sdk/XCode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator14.0.sdk/usr/lib/system/libsystem_pthread.tbd) built for macOS

Removing the bitcode marker flag used to compile the scenario app turns that into a warning. I think it probably hit this line:
https://github.com/apple-opensource/ld64/blob/fd3feabb0a1eb18ab5d7910f3c3a5eed99cef6ab/src/ld/PlatformSupport.cpp#L156-L158

The warning only shows up when building simulator targets it seems. Bitcode must be disabled when building with --simulator right?
Still, not sure if these warnings are a sign of a real problem or not. Should we bundle ld with the ios 14 sdk, or use the stock sdk for building the scenario app?

@LongCatIsLooong

@LongCatIsLooong We support bitcode for add-to-app, which can be run in the simulator (with the bitcode marker). If all you changed was the SDK and some stuff for the scenario app, it's probably fine if the framework compiles. If you also turned off bitcode during framework builds, that's a problem.

@godofredoc

Looks like flutter-team-archive/engine#20972 was able to build on CI. Closing.

Thanks for the help @godofredoc and @jmagman!

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.

Navigation Menu

Repository navigation

Mac LUCI builders' Xcode should include iOS 14 SDK #65901

22 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Mac LUCI builders' Xcode should include iOS 14 SDK #65901

Description

Activity

gspencergoog commented on Sep 23, 2020

godofredoc commented on Sep 23, 2020

godofredoc commented on Sep 24, 2020

gspencergoog commented on Sep 24, 2020

godofredoc commented on Sep 24, 2020

jmagman commented on Sep 24, 2020

jmagman commented on Sep 24, 2020

godofredoc commented on Sep 24, 2020

godofredoc commented on Sep 24, 2020

gspencergoog commented on Sep 24, 2020

22 remaining items

jmagman commented on Oct 5, 2020

jmagman commented on Oct 6, 2020

jmagman commented on Oct 6, 2020

LongCatIsLooong commented on Oct 6, 2020

jmagman commented on Oct 6, 2020

LongCatIsLooong commented on Oct 6, 2020

jmagman commented on Oct 6, 2020

LongCatIsLooong commented on Oct 6, 2020

jmagman commented on Oct 6, 2020

LongCatIsLooong commented on Oct 13, 2020

github-actions commented on Aug 11, 2021

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions