Skip to content

Fix some latent test bugs (state leaked between tests) - #2457

Merged
chrisbobbe merged 2 commits into
zulip:mainfrom
chrisbobbe:test-order-fixes
Sep 21, 2026
Merged

chrisbobbe merged 2 commits into
zulip:mainfrom
chrisbobbe:test-order-fixes

Conversation

@chrisbobbe

Copy link
Copy Markdown
Member

Discovered in the Claude-powered audit that led to #2454 and #2455.

(These aren't active flakes, but we still shouldn't leak state between tests like this, such that the suite's results depend on what order the tests are run in.)

@chrisbobbe chrisbobbe added the maintainer review PR ready for review by Zulip maintainers label Sep 17, 2026

@rajveermalviya rajveermalviya left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @chrisbobbe! LGTM, please go ahead and merge.

chrisbobbe and others added 2 commits September 21, 2026 16:15
…eWidget

These two pump a widget tree containing a GlobalStoreWidget without
`addTearDown(testBinding.reset)`. Each is currently the last test in
its file, so nothing fails. But if someone adds a test after one of
them and it mounts a GlobalStoreWidget, the store from these would
still be loaded, and that test would trip
TestZulipBinding.getGlobalStoreUniquely -- the assertion firing in the
new test rather than in these two, making it confusing to track down.

So add `addTearDown(testBinding.reset)` to both.

Co-Authored-By: Claude Opus 5 <[email protected]>
The file-scope `testNavObserver` writes into `late` locals that only
`prepare` initializes, so a test pumping the tree via `prepareBanner`
would hit an uninitialized `pushedRoutes` on the first navigation.
Nothing fails in the default test order, where `prepare` always runs
first.

Move the reset into a `setUp`, so the state is ready for every test
no matter what order the tests are run in, and no matter which
helper pumps the tree.

Co-Authored-By: Claude Opus 5 <[email protected]>
@chrisbobbe
chrisbobbe merged commit bbec50b into zulip:main Sep 21, 2026
4 checks passed
@chrisbobbe

Copy link
Copy Markdown
Member Author

Thanks for the review! Done.

@chrisbobbe
chrisbobbe deleted the test-order-fixes branch September 21, 2026 23:26
chrisbobbe added a commit to chrisbobbe/tests that referenced this pull request Oct 8, 2026
Our tests were disabled in flutter#478, quite understandably: they were
flaking (flutter/flutter#178364, flutter/flutter#179122), and we
went two weeks without responding, where registry/template.test
asks contacts to respond within about 72 hours.

We've since diagnosed and fixed that flake (a test was implicitly
depending on the date/time on a real clock):
  zulip/zulip-flutter#2454

We also fixed another, rarer flake of the same kind, which showed up
in a Claude-assisted audit:
  zulip/zulip-flutter#2455

(The audit also uncovered some cases where the behavior of a file's
tests would depend on the order in which the tests ran, and we fixed
those too: zulip/zulip-flutter#2457.)

So we're proposing re-enablement. The new pin is
zulip/zulip-flutter's `main` on 2026-10-08, which includes all
three fixes.

Greg has left full-time work on Zulip:
  https://blog.zulip.com/2026/05/15/announcing-zulip-foundation/
I (Chris) am now the mobile-app lead. This commit removes Greg's
contact and adds my coworker Rajesh. On GitHub, we're @chrisbobbe
and @rajveermalviya.
@chrisbobbe chrisbobbe mentioned this pull request Oct 8, 2026
4 of 8 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

maintainer review PR ready for review by Zulip maintainers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants