[webview] Integration tests hang locally on macOS

Currently none of us are able to successfully run webview_flutter_wkwebview integration tests locally with macOS as the target; test that try to load a URL and then wait for it to load are hanging indefinitely. What's odd is that the tests still run successfully on CI (we've verified that they are running and passing, not just silently not running). I'm investigating, but filing this to track the investigation (and hopefully provide context for an eventual fix).

So far I've determined:

The local server started in the integration test is not receiving the requests at all (determined by logging requests to a file after turning off sandboxing).
The tests pass if I use real websites instead of the local server (but we don't want to do that; using real websites was a source of OOB failures and flake).
If I log the server address at the start of the run, then load one of its test URLs from a browser while the integration test is hanging waiting for the load that never happens, I do get a response, and I see the request logged.

Something to do with MacOS network permissions and/or dialog boxes?

My current theory is tightening of NSAppTransportSecurity defaults; it sounds like macOS 14 changed the default behaviors for local connections. I'll dig into that more tomorrow.

From https://developer.apple.com/documentation/macos-release-notes/macos-14-release-notes:

Fixed: For apps linked on or after iOS 17 and aligned OS versions, App Transport Security now requires secure connections to external IP addresses by default. For more information on these requirements, see Preventing Insecure Network Connections. Exceptions can be made using NSExceptionDomains, which now supports exceptions for individual IP addresses and ranges specified in classless inter-domain routing (CIDR) notation. (101967030)

I haven't been able to get any exceptions to work yet though, and nscurl --ats-diagnostics shows a fail for every configuration it tries.

I'm seeing flaky hangs in CI in what looks like the same tests, and spot checking is that they are hanging on macOS 14 machines, but passing on macOS 13. Which would line up with the ATS changes, although that could be a coincidence.

Strange, just changing the in-process test server to bind to the loopback address seems to be enough to resolve this locally (on macOS 15). I'm not clear on why we don't need any ATS plist entries, but I'll take it.

[webview_flutter] Fix macOS 14+ test hangs (flutter#7953)

Thanks Stuart for investigating and fixing this, much appreciated! 💙

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.

[webview_flutter] Fix macOS 14+ test hangs (#7953)

[webview_flutter] Fix macOS 14+ test hangs (flutter#7953)

Navigation Menu

Repository navigation

[webview] Integration tests hang locally on macOS #157449

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[webview] Integration tests hang locally on macOS #157449

Description

Activity

stuartmorgan-g commented on Oct 23, 2024

matanlurey commented on Oct 23, 2024

stuartmorgan-g commented on Oct 23, 2024

stuartmorgan-g commented on Oct 24, 2024

stuartmorgan-g commented on Oct 29, 2024

stuartmorgan-g commented on Oct 29, 2024

loic-sharma commented on Nov 4, 2024

github-actions commented on Nov 18, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions