Skip to content

Revert: Allow resetting test invariants in addTearDown - #193383

Merged
dkwingsmt merged 1 commit into
flutter:masterfrom
flutteractionsbot:revert-192082-1790380495
Sep 26, 2026
Merged

dkwingsmt merged 1 commit into
flutter:masterfrom
flutteractionsbot:revert-192082-1790380495

Conversation

@flutteractionsbot

Copy link
Copy Markdown
Contributor

Reverts: Allow resetting test invariants in addTearDown

Initiated by: @b-luk

Reason for reverting: Breaks tree with tool_integration_tests failures. Example: https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20tool_integration_tests_2/1329/overview

Original PR Author: @dkwingsmt

Reviewed By: @justinmc

The original PR description is provided below:

Fixes #110488.

Background: The testWidgets Lifecycle

Understanding this change requires distinguishing between two layers of Futures and two distinct verification steps in testWidgets:

1. Two Layers of Futures

  • testBody() Future: The closure containing the user's test logic.
  • runTest() Future: The overall async harness returned by binding.runTest(). It manages the test zone, unmounts the widget tree, and resolves only after testBody() and test-specific verifications finish.

2. Two Distinct Verification Phases

  • Test-specific verifications (invariantTester): Passed as a callback to runTest and executed synchronously inside _runTestBody immediately after the testBody() Future completes and the widget tree is unmounted. In testWidgets, this runs tester._endOfTestVerifications to verify that tickers and semantics handles were disposed.
  • Framework-level invariant checks: Verifies global framework state, including ensuring foundation/rendering/widgets debug flags were reset and no non-periodic timers or animations remain.

3. The Teardown Lifecycle & Root Cause of #110488

When testWidgets runs:

  1. testWidgets registers test_package.addTearDown(binding.postTest) with package:test.
  2. testWidgets calls and returns binding.runTest(...).
  3. Inside testBody(), users may register cleanups using addTearDown (for example, addTearDown(() => debugDefaultTargetPlatformOverride = null)).
  4. package:test executes teardowns in LIFO (reverse registration) order only after the Future returned by runTest completes.

Previously, framework-level invariant checks ran at the end of _runTestBody before the runTest Future completed. As a result, they ran before package:test executed any user-registered addTearDown callbacks, causing tests that clean up debug flags via addTearDown to fail invariant verification (#110488).

The following diagram shows the code flow after this PR:

testWidgets(description, (tester) async { ... })
 │
 ├── 1. Registration Phase
 │    └── test_package.addTearDown(binding.postTest)     [Teardown Stack: #1]
 │
 ├── 2. Test Execution Phase: binding.runTest(...)
 │    │
 │    │   [ Future 1: testBody() ]
 │    ├──► Run testBody(tester)
 │    │     └── User calls addTearDown(...)              [Teardown Stack: #2]
 │    │
 │    │   [ Post-testBody cleanup (inside runTest) ]
 │    ├──► Unmount widget tree (runApp)
 │    ├──► Verification 1: invariantTester()             (tickers, semantics)
 │    └──► Set _shouldVerifyInvariants = true
 │
 │    [ Future 2: runTest() completes ]
 │
 └── 3. Teardown Phase: package:test executes in LIFO order
      │
      ├──► [Step 1] User Teardowns (Stack #2)
      │     └── e.g., debugDefaultTargetPlatformOverride = null
      │
      └──► [Step 2] binding.postTest() (Stack #1)
            ├── Verification 2: Framework Invariants     (debug vars, timers)
            ├── Reset Binding State                      (_currentFakeAsync, etc.)
            └── reportTestException(...)                 (if Verification 2 failed)

Changes & Design Rationale

1. Defer Framework Invariant Checks to postTest

Framework invariant checks (_verifyInvariants(), _verifyAutoUpdateGoldensUnset, etc.) are moved from _runTestBody into TestWidgetsFlutterBinding.postTest.
Because testWidgets registers binding.postTest before running the test body, package:test's LIFO teardown ordering guarantees that all user-registered addTearDown callbacks execute before binding.postTest.

2. Clarifying the postTest Error Contract

Previously, postTest only performed state cleanup and did not check invariants. Moving invariant checks into postTest introduced the question of how invariant failures should be propagated: allowing postTest to throw, or absorbing errors and reporting them via reportTestException.

We explicitly define the contract of postTest to not throw exceptions, reporting any invariant failures via reportTestException:

  • Rationale & Trade-offs:
    • Alternative considered (throwing): If super.postTest() threw on an invariant failure, package:test would still capture the failure. However, subclasses (AutomatedTestWidgetsFlutterBinding, LiveTestWidgetsFlutterBinding, or custom bindings) override postTest to reset their own state (e.g. nulling _currentFakeAsync / _clock, or setting _inTest = false). Allowing super.postTest() to throw would require every subclass to defensively wrap super.postTest() in try ... finally. If any subclass omitted this, an invariant failure would leave the binding in a dirty state and trigger cascading !inTest failures in subsequent tests.
    • Chosen approach (non-throwing): Absorbing errors in postTest and forwarding them to reportTestException guarantees that all cleanup in base and subclass implementations executes sequentially without defensive try ... finally boilerplate, while still cleanly failing the test.

Why this does not break existing behavior:
In the previous implementation, when invariant assertions failed inside _runTestBody, they were caught by _testZone.handleUncaughtError and forwarded to reportTestException (which delegates to package:test.registerException). They never escaped runTest as unhandled thrown exceptions. Catching invariant failures in postTest and routing them to reportTestException preserves this exact error reporting pipeline, diagnostic output, and failure semantics without disrupting teardown execution.

3. Preserving Existing Behavior on Failed Tests (_shouldVerifyInvariants)

Historically, _runTestBody only verified invariants if the test body did not fail:

if (_pendingExceptionDetails == null) {
  // We only try to clean up and verify invariants if we didn't already
  // fail. If we got an exception already, then we instead leave everything
  // alone so that we don't cause more spurious errors.
  ...
  _verifyInvariants();
}

When invariant checks are moved to postTest, _pendingExceptionDetails is no longer available because it has already been reported and cleared by testCompletionHandler when runTest completed.

To preserve this exact existing behavior across the lifecycle boundary, _shouldVerifyInvariants is introduced. It is set to true at the end of _runTestBody only when _pendingExceptionDetails == null. If a test fails in its body, _shouldVerifyInvariants remains false, and postTest skips invariant checks—ensuring no behavior change and preventing spurious errors (e.g. active animations on an unmounted tree) from masking the real failure.

4. Documentation Updates

Updated the doc comments on:

  • TestWidgetsFlutterBinding.postTest: Documents the non-throwing contract, the @mustCallSuper requirement, and error reporting via reportTestException.
  • TestWidgetsFlutterBinding.runTest: Clarifies the distinction between invariantTester (which runs before runTest completes) and framework invariant checks (which run in postTest after runTest and user teardowns complete).

Tests

  • Added unit tests in packages/flutter_test/test/bindings_invariants_test.dart verifying that foundation variables (debugDefaultTargetPlatformOverride, debugDoublePrecision, debugBrightnessOverride), autoUpdateGoldenFiles, ErrorWidget.builder, and shouldPropagateDevicePointerEvents can all be reset using addTearDown.
  • Added test cases verifying invariant failures in postTest and ensuring invariant checks are skipped when the test body throws.

Pre-launch Checklist

If you need help, consider asking for advice on the #hackers-new channel on Discord.

If this change needs to override an active code freeze, provide a comment explaining why. The code freeze workflow can be overridden by code reviewers. See pinned issues for any active code freezes with guidance.

Note: The Flutter team is currently trialing the use of Gemini Code Assist for GitHub. Comments from the gemini-code-assist bot should not be taken as authoritative feedback from the Flutter team. If you find its comments useful you can update your code accordingly, but if you are unsure or disagree with the feedback, please feel free to wait for a Flutter team member's review for guidance on which automated comments should be addressed.

@flutter-dashboard flutter-dashboard Bot added the CICD Run CI/CD label Sep 25, 2026
@github-actions github-actions Bot added a: tests "flutter test", flutter_test, or one of our tests framework flutter/packages/flutter repository. See also f: labels. labels Sep 25, 2026
@b-luk
b-luk requested a review from dkwingsmt September 25, 2026 23:55
@b-luk b-luk added the emergency Jump the queue; land PR in front of all others; only use for emergencies label Sep 25, 2026
@flutter-dashboard

Copy link
Copy Markdown

Detected the emergency label.

If you add the autosubmit label, the bot will wait until all presubmits pass but ignore the tree status, allowing fixes for tree breakages while still validating that they don't break any existing presubmits.

The "Merge" button is also unlocked. To bypass presubmits as well as the tree status, press the GitHub "Add to Merge Queue".

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the invariant verification process in TestWidgetsFlutterBinding by executing the checks directly within _runTestBody instead of delaying them to postTest. It also updates the verification methods to report errors using FlutterError.reportError rather than throwing them directly, and cleans up associated state variables and unused methods. Additionally, feedback was provided to simplify the call to _verifyAutoUpdateGoldensUnset by removing a redundant isBrowser check.

invariantTester();
_verifyPostPumpInvariants();
_shouldVerifyInvariants = true;
_verifyAutoUpdateGoldensUnset(autoUpdateGoldensBeforeTest && !isBrowser);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The && !isBrowser check is redundant here because autoUpdateGoldensBeforeTest is already defined as autoUpdateGoldenFiles && !isBrowser on line 1954. Simplifying this call improves readability.

      _verifyAutoUpdateGoldensUnset(autoUpdateGoldensBeforeTest);
References
  1. Suggest simplification and refactoring: Assess whether the code can be made simpler or refactored to enhance readability and maintainability. (link)

@dkwingsmt dkwingsmt left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thank you filing the revert promptly.

@dkwingsmt dkwingsmt added the autosubmit Merge PR when tree becomes green via auto submit App label Sep 26, 2026
@dkwingsmt
dkwingsmt added this pull request to the merge queue Sep 26, 2026
Merged via the queue into flutter:master with commit 5f7822f Sep 26, 2026
32 of 33 checks passed
@flutter-dashboard flutter-dashboard Bot removed the autosubmit Merge PR when tree becomes green via auto submit App label Sep 26, 2026
@flutteractionsbot
flutteractionsbot deleted the revert-192082-1790380495 branch September 26, 2026 01:23
DanTup pushed a commit to DanTup/flutter that referenced this pull request Sep 28, 2026
)

Reverts: [Allow resetting test invariants in
`addTearDown`](flutter#192082)

Initiated by: @b-luk

Reason for reverting: Breaks tree with tool_integration_tests failures.
Example:
https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20tool_integration_tests_2/1329/overview

Original PR Author: @dkwingsmt

Reviewed By: @justinmc

The original PR description is provided below:

Fixes flutter#110488.

### Background: The `testWidgets` Lifecycle
Understanding this change requires distinguishing between two layers of
`Future`s and two distinct verification steps in `testWidgets`:

#### 1. Two Layers of `Future`s
* **`testBody()` Future**: The closure containing the user's test logic.
* **`runTest()` Future**: The overall async harness returned by
`binding.runTest()`. It manages the test zone, unmounts the widget tree,
and resolves only after `testBody()` and test-specific verifications
finish.

#### 2. Two Distinct Verification Phases
* **Test-specific verifications (`invariantTester`)**: Passed as a
callback to `runTest` and executed synchronously inside `_runTestBody`
immediately after the `testBody()` Future completes and the widget tree
is unmounted. In `testWidgets`, this runs
`tester._endOfTestVerifications` to verify that tickers and semantics
handles were disposed.
* **Framework-level invariant checks**: Verifies global framework state,
including ensuring foundation/rendering/widgets debug flags were reset
and no non-periodic timers or animations remain.

#### 3. The Teardown Lifecycle & Root Cause of flutter#110488
When `testWidgets` runs:
1. `testWidgets` registers `test_package.addTearDown(binding.postTest)`
with `package:test`.
2. `testWidgets` calls and returns `binding.runTest(...)`.
3. Inside `testBody()`, users may register cleanups using `addTearDown`
(for example, `addTearDown(() => debugDefaultTargetPlatformOverride =
null)`).
4. `package:test` executes teardowns in **LIFO (reverse registration)
order** only *after* the Future returned by `runTest` completes.

Previously, framework-level invariant checks ran at the end of
`_runTestBody` *before* the `runTest` Future completed. As a result,
they ran before `package:test` executed any user-registered
`addTearDown` callbacks, causing tests that clean up debug flags via
`addTearDown` to fail invariant verification (flutter#110488).

The following diagram shows the code flow after this PR:
```
testWidgets(description, (tester) async { ... })
 │
 ├── 1. Registration Phase
 │    └── test_package.addTearDown(binding.postTest)     [Teardown Stack: #1]
 │
 ├── 2. Test Execution Phase: binding.runTest(...)
 │    │
 │    │   [ Future 1: testBody() ]
 │    ├──► Run testBody(tester)
 │    │     └── User calls addTearDown(...)              [Teardown Stack: #2]
 │    │
 │    │   [ Post-testBody cleanup (inside runTest) ]
 │    ├──► Unmount widget tree (runApp)
 │    ├──► Verification 1: invariantTester()             (tickers, semantics)
 │    └──► Set _shouldVerifyInvariants = true
 │
 │    [ Future 2: runTest() completes ]
 │
 └── 3. Teardown Phase: package:test executes in LIFO order
      │
      ├──► [Step 1] User Teardowns (Stack #2)
      │     └── e.g., debugDefaultTargetPlatformOverride = null
      │
      └──► [Step 2] binding.postTest() (Stack #1)
            ├── Verification 2: Framework Invariants     (debug vars, timers)
            ├── Reset Binding State                      (_currentFakeAsync, etc.)
            └── reportTestException(...)                 (if Verification 2 failed)
```
### Changes & Design Rationale
#### 1. Defer Framework Invariant Checks to `postTest`

Framework invariant checks (`_verifyInvariants()`,
`_verifyAutoUpdateGoldensUnset`, etc.) are moved from `_runTestBody`
into `TestWidgetsFlutterBinding.postTest`.
Because `testWidgets` registers `binding.postTest` before running the
test body, `package:test`'s LIFO teardown ordering guarantees that all
user-registered `addTearDown` callbacks execute before
`binding.postTest`.

#### 2. Clarifying the `postTest` Error Contract

Previously, `postTest` only performed state cleanup and did not check
invariants. Moving invariant checks into `postTest` introduced the
question of how invariant failures should be propagated: allowing
`postTest` to throw, or absorbing errors and reporting them via
`reportTestException`.

We explicitly define the contract of `postTest` to **not throw
exceptions**, reporting any invariant failures via
`reportTestException`:

* **Rationale & Trade-offs**:
* *Alternative considered (throwing)*: If `super.postTest()` threw on an
invariant failure, `package:test` would still capture the failure.
However, subclasses (`AutomatedTestWidgetsFlutterBinding`,
`LiveTestWidgetsFlutterBinding`, or custom bindings) override `postTest`
to reset their own state (e.g. nulling `_currentFakeAsync` / `_clock`,
or setting `_inTest = false`). Allowing `super.postTest()` to throw
would require every subclass to defensively wrap `super.postTest()` in
`try ... finally`. If any subclass omitted this, an invariant failure
would leave the binding in a dirty state and trigger cascading `!inTest`
failures in subsequent tests.
* *Chosen approach (non-throwing)*: Absorbing errors in `postTest` and
forwarding them to `reportTestException` guarantees that all cleanup in
base and subclass implementations executes sequentially without
defensive `try ... finally` boilerplate, while still cleanly failing the
test.

**Why this does not break existing behavior:**
In the previous implementation, when invariant assertions failed inside
`_runTestBody`, they were caught by `_testZone.handleUncaughtError` and
forwarded to `reportTestException` (which delegates to
`package:test.registerException`). They never escaped `runTest` as
unhandled thrown exceptions. Catching invariant failures in `postTest`
and routing them to `reportTestException` preserves this exact error
reporting pipeline, diagnostic output, and failure semantics without
disrupting teardown execution.

#### 3. Preserving Existing Behavior on Failed Tests
(`_shouldVerifyInvariants`)

Historically, `_runTestBody` only verified invariants if the test body
did not fail:

```dart
if (_pendingExceptionDetails == null) {
  // We only try to clean up and verify invariants if we didn't already
  // fail. If we got an exception already, then we instead leave everything
  // alone so that we don't cause more spurious errors.
  ...
  _verifyInvariants();
}
```

When invariant checks are moved to `postTest`,
`_pendingExceptionDetails` is no longer available because it has already
been reported and cleared by `testCompletionHandler` when `runTest`
completed.

To preserve this exact existing behavior across the lifecycle boundary,
`_shouldVerifyInvariants` is introduced. It is set to `true` at the end
of `_runTestBody` only when `_pendingExceptionDetails == null`. If a
test fails in its body, `_shouldVerifyInvariants` remains `false`, and
`postTest` skips invariant checks—ensuring no behavior change and
preventing spurious errors (e.g. active animations on an unmounted tree)
from masking the real failure.

#### 4. Documentation Updates
Updated the doc comments on:
* `TestWidgetsFlutterBinding.postTest`: Documents the non-throwing
contract, the `@mustCallSuper` requirement, and error reporting via
`reportTestException`.
* `TestWidgetsFlutterBinding.runTest`: Clarifies the distinction between
`invariantTester` (which runs before `runTest` completes) and framework
invariant checks (which run in `postTest` after `runTest` and user
teardowns complete).

### Tests
- Added unit tests in
`packages/flutter_test/test/bindings_invariants_test.dart` verifying
that foundation variables (`debugDefaultTargetPlatformOverride`,
`debugDoublePrecision`, `debugBrightnessOverride`),
`autoUpdateGoldenFiles`, `ErrorWidget.builder`, and
`shouldPropagateDevicePointerEvents` can all be reset using
`addTearDown`.
- Added test cases verifying invariant failures in `postTest` and
ensuring invariant checks are skipped when the test body throws.

## Pre-launch Checklist

- [ ] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [ ] I read the [AI contribution guidelines] and understand my
responsibilities, or I am not using AI tools.
- [ ] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [ ] I read and followed the [Flutter Style Guide], including [Features
we expect every widget to implement].
- [ ] I signed the [CLA].
- [ ] I listed at least one issue that this PR fixes in the description
above.
- [ ] I updated/added relevant documentation (doc comments with `///`).
- [ ] I added new tests to check the change I am making, or this PR is
[test-exempt].
- [ ] I followed the [breaking change policy] and added [Data Driven
Fixes] where supported.
- [ ] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel
on [Discord].

If this change needs to override an active code freeze, provide a
comment explaining why. The code freeze workflow can be overridden by
code reviewers. See pinned issues for any active code freezes with
guidance.

**Note**: The Flutter team is currently trialing the use of [Gemini Code
Assist for
GitHub](https://developers.google.com/gemini-code-assist/docs/review-github-code).
Comments from the `gemini-code-assist` bot should not be taken as
authoritative feedback from the Flutter team. If you find its comments
useful you can update your code accordingly, but if you are unsure or
disagree with the feedback, please feel free to wait for a Flutter team
member's review for guidance on which automated comments should be
addressed.

<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#overview
[AI contribution guidelines]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#ai-contribution-guidelines
[Tree Hygiene]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md
[test-exempt]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md
[Features we expect every widget to implement]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md#features-we-expect-every-widget-to-implement
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#handling-breaking-changes
[Discord]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Chat.md
[Data Driven Fixes]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Data-driven-Fixes.md
DanTup pushed a commit to DanTup/flutter that referenced this pull request Sep 28, 2026
)

Reverts: [Allow resetting test invariants in
`addTearDown`](flutter#192082)

Initiated by: @b-luk

Reason for reverting: Breaks tree with tool_integration_tests failures.
Example:
https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20tool_integration_tests_2/1329/overview

Original PR Author: @dkwingsmt

Reviewed By: @justinmc

The original PR description is provided below:

Fixes flutter#110488.

### Background: The `testWidgets` Lifecycle
Understanding this change requires distinguishing between two layers of
`Future`s and two distinct verification steps in `testWidgets`:

#### 1. Two Layers of `Future`s
* **`testBody()` Future**: The closure containing the user's test logic.
* **`runTest()` Future**: The overall async harness returned by
`binding.runTest()`. It manages the test zone, unmounts the widget tree,
and resolves only after `testBody()` and test-specific verifications
finish.

#### 2. Two Distinct Verification Phases
* **Test-specific verifications (`invariantTester`)**: Passed as a
callback to `runTest` and executed synchronously inside `_runTestBody`
immediately after the `testBody()` Future completes and the widget tree
is unmounted. In `testWidgets`, this runs
`tester._endOfTestVerifications` to verify that tickers and semantics
handles were disposed.
* **Framework-level invariant checks**: Verifies global framework state,
including ensuring foundation/rendering/widgets debug flags were reset
and no non-periodic timers or animations remain.

#### 3. The Teardown Lifecycle & Root Cause of flutter#110488
When `testWidgets` runs:
1. `testWidgets` registers `test_package.addTearDown(binding.postTest)`
with `package:test`.
2. `testWidgets` calls and returns `binding.runTest(...)`.
3. Inside `testBody()`, users may register cleanups using `addTearDown`
(for example, `addTearDown(() => debugDefaultTargetPlatformOverride =
null)`).
4. `package:test` executes teardowns in **LIFO (reverse registration)
order** only *after* the Future returned by `runTest` completes.

Previously, framework-level invariant checks ran at the end of
`_runTestBody` *before* the `runTest` Future completed. As a result,
they ran before `package:test` executed any user-registered
`addTearDown` callbacks, causing tests that clean up debug flags via
`addTearDown` to fail invariant verification (flutter#110488).

The following diagram shows the code flow after this PR:
```
testWidgets(description, (tester) async { ... })
 │
 ├── 1. Registration Phase
 │    └── test_package.addTearDown(binding.postTest)     [Teardown Stack: #1]
 │
 ├── 2. Test Execution Phase: binding.runTest(...)
 │    │
 │    │   [ Future 1: testBody() ]
 │    ├──► Run testBody(tester)
 │    │     └── User calls addTearDown(...)              [Teardown Stack: #2]
 │    │
 │    │   [ Post-testBody cleanup (inside runTest) ]
 │    ├──► Unmount widget tree (runApp)
 │    ├──► Verification 1: invariantTester()             (tickers, semantics)
 │    └──► Set _shouldVerifyInvariants = true
 │
 │    [ Future 2: runTest() completes ]
 │
 └── 3. Teardown Phase: package:test executes in LIFO order
      │
      ├──► [Step 1] User Teardowns (Stack #2)
      │     └── e.g., debugDefaultTargetPlatformOverride = null
      │
      └──► [Step 2] binding.postTest() (Stack #1)
            ├── Verification 2: Framework Invariants     (debug vars, timers)
            ├── Reset Binding State                      (_currentFakeAsync, etc.)
            └── reportTestException(...)                 (if Verification 2 failed)
```
### Changes & Design Rationale
#### 1. Defer Framework Invariant Checks to `postTest`

Framework invariant checks (`_verifyInvariants()`,
`_verifyAutoUpdateGoldensUnset`, etc.) are moved from `_runTestBody`
into `TestWidgetsFlutterBinding.postTest`.
Because `testWidgets` registers `binding.postTest` before running the
test body, `package:test`'s LIFO teardown ordering guarantees that all
user-registered `addTearDown` callbacks execute before
`binding.postTest`.

#### 2. Clarifying the `postTest` Error Contract

Previously, `postTest` only performed state cleanup and did not check
invariants. Moving invariant checks into `postTest` introduced the
question of how invariant failures should be propagated: allowing
`postTest` to throw, or absorbing errors and reporting them via
`reportTestException`.

We explicitly define the contract of `postTest` to **not throw
exceptions**, reporting any invariant failures via
`reportTestException`:

* **Rationale & Trade-offs**:
* *Alternative considered (throwing)*: If `super.postTest()` threw on an
invariant failure, `package:test` would still capture the failure.
However, subclasses (`AutomatedTestWidgetsFlutterBinding`,
`LiveTestWidgetsFlutterBinding`, or custom bindings) override `postTest`
to reset their own state (e.g. nulling `_currentFakeAsync` / `_clock`,
or setting `_inTest = false`). Allowing `super.postTest()` to throw
would require every subclass to defensively wrap `super.postTest()` in
`try ... finally`. If any subclass omitted this, an invariant failure
would leave the binding in a dirty state and trigger cascading `!inTest`
failures in subsequent tests.
* *Chosen approach (non-throwing)*: Absorbing errors in `postTest` and
forwarding them to `reportTestException` guarantees that all cleanup in
base and subclass implementations executes sequentially without
defensive `try ... finally` boilerplate, while still cleanly failing the
test.

**Why this does not break existing behavior:**
In the previous implementation, when invariant assertions failed inside
`_runTestBody`, they were caught by `_testZone.handleUncaughtError` and
forwarded to `reportTestException` (which delegates to
`package:test.registerException`). They never escaped `runTest` as
unhandled thrown exceptions. Catching invariant failures in `postTest`
and routing them to `reportTestException` preserves this exact error
reporting pipeline, diagnostic output, and failure semantics without
disrupting teardown execution.

#### 3. Preserving Existing Behavior on Failed Tests
(`_shouldVerifyInvariants`)

Historically, `_runTestBody` only verified invariants if the test body
did not fail:

```dart
if (_pendingExceptionDetails == null) {
  // We only try to clean up and verify invariants if we didn't already
  // fail. If we got an exception already, then we instead leave everything
  // alone so that we don't cause more spurious errors.
  ...
  _verifyInvariants();
}
```

When invariant checks are moved to `postTest`,
`_pendingExceptionDetails` is no longer available because it has already
been reported and cleared by `testCompletionHandler` when `runTest`
completed.

To preserve this exact existing behavior across the lifecycle boundary,
`_shouldVerifyInvariants` is introduced. It is set to `true` at the end
of `_runTestBody` only when `_pendingExceptionDetails == null`. If a
test fails in its body, `_shouldVerifyInvariants` remains `false`, and
`postTest` skips invariant checks—ensuring no behavior change and
preventing spurious errors (e.g. active animations on an unmounted tree)
from masking the real failure.

#### 4. Documentation Updates
Updated the doc comments on:
* `TestWidgetsFlutterBinding.postTest`: Documents the non-throwing
contract, the `@mustCallSuper` requirement, and error reporting via
`reportTestException`.
* `TestWidgetsFlutterBinding.runTest`: Clarifies the distinction between
`invariantTester` (which runs before `runTest` completes) and framework
invariant checks (which run in `postTest` after `runTest` and user
teardowns complete).

### Tests
- Added unit tests in
`packages/flutter_test/test/bindings_invariants_test.dart` verifying
that foundation variables (`debugDefaultTargetPlatformOverride`,
`debugDoublePrecision`, `debugBrightnessOverride`),
`autoUpdateGoldenFiles`, `ErrorWidget.builder`, and
`shouldPropagateDevicePointerEvents` can all be reset using
`addTearDown`.
- Added test cases verifying invariant failures in `postTest` and
ensuring invariant checks are skipped when the test body throws.

## Pre-launch Checklist

- [ ] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [ ] I read the [AI contribution guidelines] and understand my
responsibilities, or I am not using AI tools.
- [ ] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [ ] I read and followed the [Flutter Style Guide], including [Features
we expect every widget to implement].
- [ ] I signed the [CLA].
- [ ] I listed at least one issue that this PR fixes in the description
above.
- [ ] I updated/added relevant documentation (doc comments with `///`).
- [ ] I added new tests to check the change I am making, or this PR is
[test-exempt].
- [ ] I followed the [breaking change policy] and added [Data Driven
Fixes] where supported.
- [ ] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel
on [Discord].

If this change needs to override an active code freeze, provide a
comment explaining why. The code freeze workflow can be overridden by
code reviewers. See pinned issues for any active code freezes with
guidance.

**Note**: The Flutter team is currently trialing the use of [Gemini Code
Assist for
GitHub](https://developers.google.com/gemini-code-assist/docs/review-github-code).
Comments from the `gemini-code-assist` bot should not be taken as
authoritative feedback from the Flutter team. If you find its comments
useful you can update your code accordingly, but if you are unsure or
disagree with the feedback, please feel free to wait for a Flutter team
member's review for guidance on which automated comments should be
addressed.

<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#overview
[AI contribution guidelines]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#ai-contribution-guidelines
[Tree Hygiene]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md
[test-exempt]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md
[Features we expect every widget to implement]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md#features-we-expect-every-widget-to-implement
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#handling-breaking-changes
[Discord]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Chat.md
[Data Driven Fixes]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Data-driven-Fixes.md
auto-submit Bot pushed a commit to flutter/packages that referenced this pull request Sep 29, 2026
…#13060)

Manual roll Flutter from 4fcd90be0045 to c5a061b18fa2 (222 revisions)

Manual roll requested by [email protected]

flutter/flutter@4fcd90b...c5a061b

2026-09-28 [email protected] [cross imports] navigator_test.dart (flutter/flutter#190954)
2026-09-28 [email protected] Revert: Reland: Only render views that need to be rendered  (flutter/flutter#193360)
2026-09-28 [email protected] ci(bringup): android_intent_security_test is green (flutter/flutter#193469)
2026-09-28 [email protected] Sync CHANGELOG.md from stable (flutter/flutter#193020)
2026-09-28 [email protected] Roll Fuchsia Linux SDK from ukukV5lEkKabtOATk... to QdgqP02_cYpRQQQNN... (flutter/flutter#193454)
2026-09-28 [email protected] Roll pub packages (flutter/flutter#193460)
2026-09-28 [email protected] Un-nest sceneBuildDuration and windowRenderDuration in web SceneBuilderRecorder (flutter/flutter#193260)
2026-09-28 [email protected] Fix unresolved doc comment references in dev/integration_tests and dev/benchmarks (flutter/flutter#193433)
2026-09-28 [email protected] Roll Packages from e55e7ac to ba0364a (9 revisions) (flutter/flutter#193450)
2026-09-28 [email protected] [tool] Require explicit dependency injection for FlutterDevice and FlutterDevice.create (flutter/flutter#192830)
2026-09-28 [email protected] Roll pub packages (flutter/flutter#193442)
2026-09-28 [email protected] [flutter_tools] Include data assets from hooks when pubspec.yaml is empty (flutter/flutter#193434)
2026-09-28 [email protected] Roll pub packages (flutter/flutter#193438)
2026-09-28 [email protected] Fix unresolved doc comment references in Material and Cupertino (flutter/flutter#193283)
2026-09-28 [email protected] Fix doc references to Material and Cupertino in the widgets library and its tests (flutter/flutter#193334)
2026-09-26 [email protected] wasm: enforce WasmGC opt-in capability checks and Firefox < 147 guard (flutter/flutter#193180)
2026-09-26 [email protected] [Android] Refuse external setters of engine entrypoint and cached engine arguments via `Intent`s (flutter/flutter#190249)
2026-09-26 [email protected] [web] Unskip TextPainter.getWordBoundary test (flutter/flutter#193378)
2026-09-26 [email protected] Specify non-obvious types in pattern variable declarations (flutter/flutter#192632)
2026-09-26 [email protected] Removes feedback loop from advanced filters without offscreen msaa and framebufferfetch (flutter/flutter#193306)
2026-09-26 [email protected] [web] Unskip 8 passing image tests in painting and rendering (flutter/flutter#193377)
2026-09-26 [email protected] Revert: Allow resetting test invariants in `addTearDown` (flutter/flutter#193383)
2026-09-25 [email protected] [tool] Migrate CoverageCollector to modular dependency injection (flutter/flutter#192924)
2026-09-25 [email protected] Fix more unresolved doc comment references (flutter/flutter#193336)
2026-09-25 [email protected] Allow resetting test invariants in `addTearDown` (flutter/flutter#192082)
2026-09-25 [email protected] [a11y] Add a semantics role for slider (flutter/flutter#193324)
2026-09-25 [email protected] Fix plugin tests after Pigeon plugin template changes (flutter/flutter#193302)
2026-09-25 [email protected] [Android 17] Bump Standard Test Apps in flutter/flutter to AGP 9.3.1 (flutter/flutter#193263)
2026-09-25 [email protected] SSBO-based gradients in UberSDF (flutter/flutter#192962)
2026-09-25 [email protected] [Linux] Handle FlView being destroyed before rendering is complete. (flutter/flutter#193268)
2026-09-25 [email protected] [flutter_tools] Handle Windows reserved characters in test target path (flutter/flutter#191900)
2026-09-25 [email protected] docs(tools): nit: say hook/build.dart in CMake native assets comment (flutter/flutter#192205)
2026-09-25 [email protected] [web] Enable Firefox Skwasm UI CI suites, configure COI configs, and fail fast on loader rejections (flutter/flutter#193187)
2026-09-25 [email protected] [Android 17] Bumped Engine Dependencies to 9.3.1 (flutter/flutter#193265)
2026-09-25 [email protected] [web] Omit group role on menu scrollables and assign region role to named routes (flutter/flutter#192965)
2026-09-25 [email protected] Roll Skia from 8eedeed98e79 to f441ca223b2b (5 revisions) (flutter/flutter#193353)
2026-09-25 [email protected] Roll Fuchsia Linux SDK from EbPpoJW-Lnsu-8dyZ... to ukukV5lEkKabtOATk... (flutter/flutter#193326)
2026-09-25 [email protected] Roll Packages from 431ea69 to e55e7ac (6 revisions) (flutter/flutter#193351)
2026-09-25 [email protected] Support DynamicLibrary.codeAsset in Flutter (flutter/flutter#188947)
2026-09-25 [email protected] Roll Dart SDK from 75d9e87d4e3d to 0e7642b85457 (1 revision) (flutter/flutter#193346)
2026-09-25 [email protected] Exclude iOS PRs from macOS link (flutter/flutter#193315)
2026-09-25 [email protected] Roll Skia from 6357608543ec to 8eedeed98e79 (4 revisions) (flutter/flutter#193341)
2026-09-25 [email protected] Roll Dart SDK from d3aedb186aab to 75d9e87d4e3d (3 revisions) (flutter/flutter#193335)
2026-09-25 [email protected] Cache the paint offset adjusted line metrics in TextPainter (flutter/flutter#191223)
...
biggs0125 pushed a commit to biggs0125/flutter that referenced this pull request Oct 2, 2026
)

This PR relands flutter#192082 after fixing the error in integration tests as
described in flutter#193383.

I've manually verified that the integration test fails on flutter#192082 and
passes now.

### Why it failed

This integration test runs
`dev/automated_tests/flutter_test/ticker_test.dart`, which leaves a
`Ticker` running. It expects that test to fail with an "An animation is
still running even after the widget tree was disposed." error. After
flutter#192082, `ticker_test.dart` passed instead.

The leaked ticker is caught by
`SchedulerBinding.debugAssertNoTransientCallbacks`. Unlike most
invariant checks, it doesn't throw; it reports the failure through
`FlutterError.reportError`. During a test, `FlutterError.onError` stores
reported errors in `_pendingExceptionDetails`, and the test completion
handler reports them when `runTest` finishes. Changing the behavior of
this method would be a breaking change to the public API in the
framework, so instead the test framework should be changed to pick up
this error.

Before flutter#192082, the check ran inside `_runTestBody`, so the completion
handler picked up the error. flutter#192082 moved the check to `postTest`,
which runs after the completion handler. The `try/catch` in `postTest`
only catches thrown errors, and `postTest` then clears
`_pendingExceptionDetails`, so the error was silently dropped and the
test passed.

#### What this reland changes

See the [2nd
commit](flutter@c617498)
for the list of changes.

- After running the invariant checks, `postTest` also picks up
`_pendingExceptionDetails` and reports it. If a check also threw, the
thrown error is reported and the other one is printed to the console so
it isn't lost.
- `postTest` now passes the test description to `reportTestException`
instead of `''`, so the output includes `The test description was: ...`
again. The integration test's expected output relies on that line.
- The four `_verify*Unset` helpers go back to
`FlutterError.reportError`, which is what they used before flutter#192082. Now
that `postTest` handles reported errors, they don't need to throw, and
they behave the same as before. flutter#192082 made this change with the hope
to simplify and unify the code flow, but now that we have to pick up
`_pendingExceptionDetails` anyway, there's no reason to make this
change.

#### Tests

- Added a test to `bindings_invariants_test.dart`. It leaks a `Ticker`
and checks that `postTest` reports the error along with the test
description.
- `test_test.dart: flutter test should report a nice error when a Ticker
is left running` passes again.

### What the 3rd commit changes

The [3rd
commit](flutter@6866134)
refactors the `bindings_invariants_test.dart` to ensure that the
`postTest` is run (and invariants are reset) even when errors occur
during the test body.

## Pre-launch Checklist

- [ ] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [ ] I read the [AI contribution guidelines] and understand my
responsibilities, or I am not using AI tools.
- [ ] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [ ] I read and followed the [Flutter Style Guide], including [Features
we expect every widget to implement].
- [ ] I signed the [CLA].
- [ ] I listed at least one issue that this PR fixes in the description
above.
- [ ] I updated/added relevant in-code documentation (doc comments with
`///`).
- [ ] If this PR introduces a new feature or capability, I created and
linked a website documentation issue or PR in [flutter/website] (or
verified none is needed).
- [ ] I added new tests to check the change I am making, or this PR is
[test-exempt].
- [ ] I followed the [breaking change policy] and added [Data Driven
Fixes] where supported.
- [ ] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel
on [Discord].

If this change needs to override an active code freeze, provide a
comment explaining why. The code freeze workflow can be overridden by
code reviewers. See pinned issues for any active code freezes with
guidance.

**Note**: The Flutter team is currently trialing the use of [Gemini Code
Assist for
GitHub](https://developers.google.com/gemini-code-assist/docs/review-github-code).
Comments from the `gemini-code-assist` bot should not be taken as
authoritative feedback from the Flutter team. If you find its comments
useful you can update your code accordingly, but if you are unsure or
disagree with the feedback, please feel free to wait for a Flutter team
member's review for guidance on which automated comments should be
addressed.

<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#overview
[AI contribution guidelines]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#ai-contribution-guidelines
[Tree Hygiene]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md
[test-exempt]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md
[Features we expect every widget to implement]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md#features-we-expect-every-widget-to-implement
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[flutter/website]: https://github.com/flutter/website
[breaking change policy]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#handling-breaking-changes
[Discord]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Chat.md
[Data Driven Fixes]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Data-driven-Fixes.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

a: tests "flutter test", flutter_test, or one of our tests CICD Run CI/CD emergency Jump the queue; land PR in front of all others; only use for emergencies framework flutter/packages/flutter repository. See also f: labels.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow debugDefaultTargetPlatformOverride to be resetted with addTearDown

3 participants