Skip to content

tools/run_tests.sh: fix --select-class scope leak; make leak-sweep opt-in only - #44

Merged
Thrameos merged 1 commit into
masterfrom
backport/run-tests-select-and-leaksweep-safety
Sep 6, 2026
Merged

Thrameos merged 1 commit into
masterfrom
backport/run-tests-select-and-leaksweep-safety

Conversation

@Thrameos

@Thrameos Thrameos commented Sep 5, 2026

Copy link
Copy Markdown
Owner

Summary

  • --select-class was silently additive to the script's hardcoded --select-package tests.junittests (JUnit ORs selectors rather than intersecting them), so a "run just this one class" invocation was actually running the whole 123-test suite -- discovered live while trying to reproduce the CefDownloadItemTest CI flake with a narrow, low-risk local repro. Backports the fix already made for this on coverage/phase1-value-objects-phase2-handlers (commit 7678597), unmodified.
  • Compounding that bug: the suite's leak-sweep/process-isolated tests (@Tag("leak-sweep")/@Tag("process-isolated")) spawn a pool of up to 16 concurrent isolated CEF subprocesses, each its own browser+renderer+GPU+zygote process tree. A "run just this one class" invocation that silently widened back out to the whole suite therefore also silently ran the full leak-sweep pass at the end -- fanning out to dozens of real windows/processes with no indication that was about to happen, reliably exhausting a dev machine's memory (8G+swap) to the point of a full VM hang/crash, twice in one session.
  • leak-sweep/process-isolated must only ever be triggered deliberately via tools/run_leak_sweep_isolated.sh; run_tests.sh now excludes both tags unconditionally, with no flag on this script able to re-enable them.

Test plan

  • bash -n tools/run_tests.sh (syntax check)
  • Verified --select-class tests.junittests.CefDownloadItemTest now runs only that class's 2 tests (previously ran all 123 + leak-sweep)
  • CI

🤖 Generated with Claude Code

https://claude.ai/code/session_01HBFmQs6JDypYP5qdC99yDq

@codecov

codecov Bot commented Sep 5, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 38.01%. Comparing base (be68fe1) to head (8ce45b9).

Additional details and impacted files
@@            Coverage Diff            @@
##             master      #44   +/-   ##
=========================================
  Coverage     38.01%   38.01%           
  Complexity      735      735           
=========================================
  Files           243      243           
  Lines         14863    14863           
  Branches       2449     2449           
=========================================
  Hits           5650     5650           
+ Misses         8011     8008    -3     
- Partials       1202     1205    +3     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Thrameos added a commit that referenced this pull request Sep 5, 2026
preliminary update racing ahead of onBeforeDownload

CefDownloadItemTest.cancelDuringDownloadInvokesNCancel() flaked
intermittently in CI across multiple unrelated backport PRs (#38, #39, #41,
#42), always failing the same assertion: "onBeforeDownload was never
invoked". Root-caused by reproducing it locally (100% deterministic once a
low-risk, narrowly-scoped repro was possible -- see PR #44's run_tests.sh
fix, needed first to run just this one test without also running the whole
suite + leak-sweep) and adding temporary native fprintf instrumentation
(not included in this commit) at DownloadHandler::OnBeforeDownload/
OnDownloadUpdated and ClientHandler::GetDownloadHandler.

That instrumentation showed CEF can deliver a preliminary
OnDownloadUpdated() notification -- for a valid, real CefDownloadItem --
*before* OnBeforeDownload() has been dispatched to the handler at all. This
test's onDownloadUpdated() used to treat "not complete" on the very first
update it saw as "must actively cancel it", immediately calling
callback.cancel() and terminateTest() -- racing ahead of onBeforeDownload
and ending the test before it ever got a chance to run, which is exactly
what "onBeforeDownload was never invoked" captures.

Fix: ignore any onDownloadUpdated() notification that arrives before this
handler's own onBeforeDownload() has recorded its decision
(gotBeforeDownload[0]). Verified via direct reproduction: failed 3/3 in
isolation before the fix (including against a heavily-used local CEF
profile where it failed 100% of the time), passed cleanly every time after
(5+ consecutive solo runs, plus the full class together) with no change to
downloadCompletesAndWritesRealFile's behavior.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01HBFmQs6JDypYP5qdC99yDq
…t-in only

--select-class was silently additive to the script's own hardcoded
--select-package tests.junittests (JUnit ORs selectors rather than
intersecting them), so a "run just this one class" invocation actually ran
the whole 123-test suite -- discovered live while trying to reproduce the
CefDownloadItemTest flake with a narrow, low-risk repro. Backports the fix
already made for this on coverage/phase1-value-objects-phase2-handlers
(commit 7678597), unmodified.

Compounding that bug: the suite's leak-sweep/process-isolated tests (tagged
"leak-sweep"/"process-isolated") spawn a pool of up to 16 concurrent
isolated CEF subprocesses, each its own browser+renderer+GPU+zygote process
tree. A "run just this one class" invocation that silently widened back out
to the whole suite therefore also silently ran the full leak-sweep pass at
the end -- fanning out to dozens of real windows/processes with no
indication that was about to happen, and reliably exhausting this
particular dev machine's memory (8G+swap) to the point of a full VM
hang/crash, twice in one session. leak-sweep must only ever be triggered
deliberately via tools/run_leak_sweep_isolated.sh; it's now excluded here
unconditionally, with no flag on this script able to re-enable it.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01HBFmQs6JDypYP5qdC99yDq
@Thrameos
Thrameos force-pushed the backport/run-tests-select-and-leaksweep-safety branch from 1084086 to 8ce45b9 Compare September 5, 2026 21:23
@Thrameos Thrameos mentioned this pull request Sep 6, 2026
2 of 3 tasks
@Thrameos Thrameos added the bug Something isn't working label Sep 6, 2026
@Thrameos
Thrameos merged commit 2c9e9b8 into master Sep 6, 2026
9 checks passed
@Thrameos
Thrameos deleted the backport/run-tests-select-and-leaksweep-safety branch September 6, 2026 02:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant