Skip to content

fix(flutter_tools): check runtime env vars before cached bot state - #190978

Merged
kevmoo merged 3 commits into
flutter:masterfrom
kevmoo:bot-detector-env-override
Aug 13, 2026
Merged

kevmoo merged 3 commits into
flutter:masterfrom
kevmoo:bot-detector-env-override

Conversation

@kevmoo

@kevmoo kevmoo commented Aug 12, 2026 •

Copy link
Copy Markdown
Contributor

Description

BotDetector currently latches detection results to persistent disk state (tool_state), causing two issues:

  • Pre-baked CI Images: When Docker/VM images run flutter doctor during build, BotDetector writes {"is-bot": false} to disk. When executed in CI (GITHUB_ACTIONS=true / CI=true), the cached false short-circuits evaluation, bypassing runtime CI environment variables.
  • Ephemeral Env Var Latching: In-memory env vars are process-level state (<1µs lookup). Persisting them to disk latches transient subshell/script context into tool_state for future interactive runs.

Changes

  • Fast in-memory checks first: Reorder BotDetector.isRunningOnBot to evaluate runtime env vars (BOT=false, CI, GITHUB_ACTIONS, TRAVIS, BORG_ALLOC_DIR, etc.) before reading persistent cache.
  • Zero disk writes on env checks: Remove _persistentToolState.setIsRunningOnBot from in-memory checks.
  • Restrict cache to hardware/IMDS probes: _persistentToolState.isRunningOnBot now only caches _azureDetector.isRunningOnAzure to avoid 250ms–1s timeout stalls on non-Azure machines.
  • Tests: Add unit tests in bot_detector_test.dart verifying runtime env overrides and cache isolation across runs.

@flutter-dashboard flutter-dashboard Bot added the CICD Run CI/CD label Aug 12, 2026
@github-actions github-actions Bot added the tool Affects the "flutter" command-line tool. See also t: labels. label Aug 12, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request modifies BotDetector to check environment variables before returning a cached isRunningOnBot value, allowing runtime environment variables to override a cached false state, and adds corresponding tests. Feedback points out that caching isRunningOnBot as false will bypass the Azure detector (which relies on an HTTP request rather than environment variables) on subsequent runs, preventing it from detecting an Azure environment.

Comment thread packages/flutter_tools/lib/src/base/bot_detector.dart
@kevmoo
kevmoo requested a review from bkonyi August 12, 2026 05:48

@zanderso zanderso left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A bunch of existing test expectations are now checking that expect(persistentToolState.isRunningOnBot, isNull). The new test expectations for persistentToolState.isRunningOnBot all check for isFalse. Are there any remaining tests that still check that persistentToolState.isRunningOnBot is correctly toggled to true when it should be?

Comment thread packages/flutter_tools/lib/src/base/bot_detector.dart
@kevmoo

kevmoo commented Aug 12, 2026

Copy link
Copy Markdown
Contributor Author

A bunch of existing test expectations are now checking that expect(persistentToolState.isRunningOnBot, isNull). The new test expectations for persistentToolState.isRunningOnBot all check for isFalse. Are there any remaining tests that still check that persistentToolState.isRunningOnBot is correctly toggled to true when it should be?

Yes! Lines 113–140 in bot_detector_test.dart still verify persistentToolState.isRunningOnBot toggles to true when Azure IMDS is reachable (returns true when azure metadata is reachable and caches azure bot detection results across instances).

The env var tests now check for isNull (or unchanged isFalse) because we intentionally removed disk writes from in-memory environment checks to prevent ephemeral script/subshell env vars from poisoning tool_state.

zanderso
zanderso previously approved these changes Aug 13, 2026
bkonyi
bkonyi previously approved these changes Aug 13, 2026
@kevmoo kevmoo added the autosubmit Merge PR when tree becomes green via auto submit App label Aug 13, 2026
@auto-submit auto-submit Bot removed the autosubmit Merge PR when tree becomes green via auto submit App label Aug 13, 2026
@auto-submit

auto-submit Bot commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

autosubmit label was removed for flutter/flutter/190978, because - The status or check suite Dashboard Checks has failed. Please fix the issues identified (or deflake) before re-applying this label.

@kevmoo
kevmoo dismissed stale reviews from bkonyi and zanderso via 30bf7c0 August 13, 2026 16:30
@kevmoo
kevmoo requested a review from a team as a code owner August 13, 2026 16:30
@github-actions github-actions Bot added platform-ios iOS applications specifically team-ios Owned by iOS platform team team-web Owned by Web platform team labels Aug 13, 2026
@kevmoo

kevmoo commented Aug 13, 2026

Copy link
Copy Markdown
Contributor Author

@bkonyi @zanderso - could I get another nod, please...

@zanderso

Copy link
Copy Markdown
Member

It looks like you merged some unrelated changes into this PR.

kevmoo added 3 commits August 13, 2026 16:52
When container images (e.g. docker build) or CI machine templates run
flutter doctor during image construction, BotDetector evaluates
isRunningOnBot = false (as no CI environment variables are present)
and writes {"is-bot": false} to tool_state.

When that container subsequently runs inside a CI pipeline (e.g.
GitHub Actions with GITHUB_ACTIONS=true or CI=true), the cached
false was returned immediately, completely bypassing runtime
environment variable checks.

* Reorder BotDetector.isRunningOnBot to evaluate in-memory environment
  variables before reading cached persistent state.
* Add unit tests verifying environment variable overrides take precedence
  over cached persistent state.
…tent tool state

When container images (e.g. docker build) or CI machine templates run
flutter doctor during image construction, BotDetector evaluated
isRunningOnBot = false (as no CI environment variables were present)
and wrote {"is-bot": false} to tool_state.

When that container subsequently executed inside a CI pipeline (e.g.
GitHub Actions with GITHUB_ACTIONS=true or CI=true), the cached
false was returned immediately from disk, completely bypassing
runtime environment variable checks.

Furthermore, in-memory environment variables (CI, GITHUB_ACTIONS,
TRAVIS, BORG_ALLOC_DIR, etc.) are fast lookups that reflect transient
process/subshell context and must not be persisted to disk, as writing
is-bot: true on encountering CI=true would permanently poison a
developer's workstation cache for subsequent interactive commands.

* Reorder BotDetector.isRunningOnBot to evaluate in-memory environment
  variables (BOT=false, CI, GITHUB_ACTIONS, TRAVIS, CIRRUS_CI,
  etc.) before reading cached persistent state.
* Remove persistent state mutations from in-memory environment checks,
  restricting PersistentToolState.isRunningOnBot caching strictly to
  AzureDetector (to avoid repeated network timeout loops on non-Azure
  machines).
* Add unit tests verifying environment variable overrides take precedence
  at runtime without mutating or poisoning persistent disk state.
… DevOps

* Add TF_BUILD check to BotDetector.isRunningOnBot to explicitly
  detect Azure Pipelines / DevOps CI runners in memory without probing IMDS.
* Add unit test in bot_detector_test.dart.
@kevmoo
kevmoo force-pushed the bot-detector-env-override branch from 30bf7c0 to 73a9f4d Compare August 13, 2026 16:53
@kevmoo

kevmoo commented Aug 13, 2026

Copy link
Copy Markdown
Contributor Author

@zanderso sorry. fixed

@github-actions github-actions Bot removed platform-ios iOS applications specifically team-ios Owned by iOS platform team team-web Owned by Web platform team labels Aug 13, 2026

@zanderso zanderso left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please monitor telemetry from master channel for anomalies after landing.

@kevmoo kevmoo added the autosubmit Merge PR when tree becomes green via auto submit App label Aug 13, 2026
@auto-submit auto-submit Bot removed the autosubmit Merge PR when tree becomes green via auto submit App label Aug 13, 2026
@auto-submit

auto-submit Bot commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

autosubmit label was removed for flutter/flutter/190978, because - The status or check suite Dashboard Checks has failed. Please fix the issues identified (or deflake) before re-applying this label.

@kevmoo
kevmoo added this pull request to the merge queue Aug 13, 2026
Merged via the queue into flutter:master with commit 0e46b7d Aug 13, 2026
23 checks passed
@kevmoo
kevmoo deleted the bot-detector-env-override branch August 13, 2026 19:31
@kevmoo

kevmoo commented Aug 13, 2026

Copy link
Copy Markdown
Contributor Author

Please monitor telemetry from master channel for anomalies after landing.

http://cl/964235879

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CICD Run CI/CD tool Affects the "flutter" command-line tool. See also t: labels.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants