Repository navigation
[tool] GA4 Analytics events are sent when running devicelab tests locally #151560
Description
Discovered while working on #151518 (details).
Repro
- Enable analytics/telemetry with
flutter --enable-analytics - Clear/delete your local analytics log file, this is usually stored in
<home directory>/.dart-tool/dart-flutter-telemetry.log. We'll use this file later to see what analytics events were sent. - Run a devicelab test locally. See dev/devicelab/README.md for full instructions, though I will provide steps here
- In a terminal navigate to dev/devicelab.
- Run a devicelab test that invokes
flutter(probably all of them). On my macOS machine that's iOS-development ready, I ran../../bin/cache/dart-sdk/bin/dart bin/test_runner.dart test -t native_assets_ios_simulator.
Expected behavior
Since this is a devicelab test, we should not send any analytics events regardless of whether analytics is enabled on the host machine.
Observed behavior
Open your dart-flutter-telemetry.log and search for "tool":{"value":"flutter-tool"}. This will be contained in flutter tool analytics events that will be sent off. Notice that there will be several events logged.
Activity
andrewkolos commentedon Feb 5, 2025
I've reprod this on current
master.The tool code responsible for determining whether or not we are running on a bot can be found here:
flutter/packages/flutter_tools/lib/src/base/bot_detector.dart
Line 27 in 5491c8c
@christopherfujino are you aware of any environment variables that are good candidates for the tool to inspect on current CI? Looking at a random postsubmit test OSinfo output, I see
USERis set tochrome-bot. Perhaps we could use that. Alternatively, we could configure infra to always haveBOTset totrue.andrewkolos commentedon Feb 5, 2025
Regardless of how we patch this issue, perhaps we should implement some sort of integration test on CI that will signal to us that bot detection doesn't work. This issue is probably resulting in a lot of noise to CLI usage analytics data.
andrewkolos commentedon Feb 5, 2025
Oops, I was conflating this issue (which is about running tests locally) with the potential issue of sending analytics events on CI. Regardless, I think this question still stands.