Skip to content

Add recurring shared release qualification - #175

Merged
Shengyu Fu (shengyfu) merged 4 commits into
mainfrom
shengyfu-shared-release-qualification
Oct 6, 2026
Merged

Shengyu Fu (shengyfu) merged 4 commits into
mainfrom
shengyfu-shared-release-qualification

Conversation

@shengyfu

@shengyfu Shengyu Fu (shengyfu) commented Oct 6, 2026 •

Copy link
Copy Markdown
Member

Summary

  • Extend the existing native Linux/macOS/Windows CI matrix with bounded jobs, weekly/manual qualification, locked builds/tests, and independent vendor/ignore unit/integration tests including doctests. Existing Rust, Unix installer, and Python agent integration coverage stays intact.
  • On Linux, validate both complete locked Cargo dependency graphs: exactly one ignore 0.4.25, directly resolved from tgrep-core to vendor/ignore; check all four fuzz targets with stable Cargo and existing libfuzzer tooling. Guard all three lockfiles against changes.
  • Perform the documented cargo install --path tgrep-cli --locked --root <private-root> and exercise that exact release executable: ordinary on-disk index and server content/files (default and hidden), shared attach/readiness/refresh, scan parity, stale-daemon fallback over a stale ordinary index, restart, detach, and worktree deletion while the daemon remains alive. Both indexed and filesystem-scan controls require backend diagnostics.
  • Eighteen focused helper tests cover graph/backend/parity rejection, lockfile mutation, deadlines, private installation, process-tree cleanup, partial-marker retries, and failure-log ordering. Windows children launch suspended, join an owned kill-on-close Job Object, then resume through documented thread APIs; cleanup waits for zero active descendants, even after direct-parent exit. Assignment/resume failures are fatal and reaped. Brief executable-image cleanup denials are logged/retried for at most five seconds; persistent failures remain fatal.
  • Generic root-level scripts/test_*.py discovery runs on every OS when files exist, alongside explicit nested suites. Standalone e9 has no root script tests, so that step skips rather than accepting Python 3.14's empty-inventory exit 5. Later benchmark tests are picked up without naming unmerged files or swallowing failures; the coordinator independently verified 14 benchmark tests execute in the combined snapshot.
  • Document scope, cost and reproduction in CONTRIBUTING.md. No runtime integration, dependency upgrades, vendored source edits, benchmark implementation dependencies, or release/signing changes.

Coverage and cost

  • PRs/main pushes retain the existing three-OS build/test matrix and add one Linux release install. Full installed-release coverage runs on all three platforms weekly (Monday 08:00 UTC) and via workflow_dispatch.
  • Test jobs are fail-fast disabled and capped at 40 minutes. Helper commands/readiness and cleanup are bounded. Private fixtures/installations are removed after owned processes stop; Cargo's normal target/download reuse is preserved.
  • Standalone main-based PR starting at e9d55db. Normal workspace tests automatically pick up later shared parity/lifecycle additions.

Final validation

Final head: e7b6250.

  • Automatic PR CI passed on Windows, macOS and native Linux: https://github.com/microsoft/tgrep/actions/runs/37423760430
  • Exact-head manual qualification passed on all three OSes, including each privately installed executable and unchanged lockfiles: https://github.com/microsoft/tgrep/actions/runs/37423775317
  • Latest completed Copilot review recommends approval with no findings. Initial scan-control, marker/log race, Windows orphan-tree, and local-content coverage findings were addressed with regressions; all inline threads resolved.
  • Locally: Windows full locked workspace/bench builds, Rust suite, independent vendor tests/doctests, final 18 helper tests, and final installed lifecycle. Native ext4 Linux full locked workspace/bench builds/tests, vendor tests/doctests, both dependency graphs/four fuzz targets, existing Python integration/installer checks, fmt/clippy, and installed qualification; final helper suite also independently passed in the coordinator's native Linux combined snapshot.
  • Workflow YAML/matrix/triggers/pinned actions verified. All three lockfiles unchanged. Native logs retained; temporary fixtures/snapshots cleaned.

An initial superseded macOS run hit the unchanged live-RSS exact-equality test; subsequent exact-head checks are green without weakening/skipping that test or making unrelated runtime changes. The helper intentionally uses stable text/file parity; exhaustive JSON parity and performance measurements remain in their independent workstreams.

Check locked vendored ignore and fuzz graphs in CI; qualify private installed release binaries on Linux PRs and all native platforms weekly or manually.

Co-authored-by: Copilot App <[email protected]>
Copilot AI balanced review requested due to automatic review settings October 6, 2026 05:54

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Scan-backend validation can falsely pass, and marker/log races can make qualification unreliable.

Review effort: Balanced
Findings: 1 High severity · 2 Medium severity

Open (3)
What changed in this PR

Adds recurring cross-platform release qualification for locked dependencies, vendored tests, and installed shared-daemon workflows.

Changes:

  • Extends CI with scheduled/manual qualification and bounded jobs.
  • Adds dependency and installed-binary qualification helpers with tests.
  • Documents qualification scope and reproduction steps.
File Description
.github/​workflows/​ci.yml Adds locked, cross-platform qualification jobs.
scripts/​qualification/​qualify.py Implements dependency and installed-release checks.
scripts/​qualification/​test_qualify.py Tests qualification failure and cleanup paths.
CONTRIBUTING.md Documents recurring qualification procedures.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread scripts/qualification/qualify.py
Comment thread scripts/qualification/qualify.py Outdated
Comment thread scripts/qualification/qualify.py Outdated
Require filesystem evidence for the scan control, retry partial registration markers, and read failure logs after reaping the owned server. Cover these review findings with focused regressions and discover future root-level script tests without accepting test failures.

Co-authored-by: Copilot App <[email protected]>
Copilot AI balanced review requested due to automatic review settings October 6, 2026 06:05

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Windows process-tree cleanup can leave descendants running when the direct child exits first.

Review effort: Balanced
Findings: None

Resolved since last review (3)
Previously missed (1)

In code that hasn't changed since last review

Medium severity Windows descendants survive after direct child exits

scripts/​qualification/​qualify.py:64

On Windows, tree cleanup is skipped as soon as the direct child has exited. A command can spawn a descendant that closes/redirects its inherited pipes and then exit; poll() is non-None, so taskkill /T is never called and the descendant can survive qualification or hold the temporary tree open. There is also a race where the child exits between this check and taskkill, whose failure is accepted because poll() then succeeds. Track these processes in a Windows Job Object (or another tree-lifetime mechanism independent of the parent remaining alive), and cover the exited-parent/live-descendant case.

Launch suspended, assign an owned kill-on-close Job Object, then resume through documented thread APIs. Wait for an empty job and release owned handles; test redirected descendants, assignment/resume failures, and bounded explicit Windows cleanup retries.

Co-authored-by: Copilot App <[email protected]>
Copilot AI balanced review requested due to automatic review settings October 6, 2026 06:20

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The advertised local-index content-search parity is not exercised.

Review effort: Balanced
Findings: None

Previously missed (1)

In code that hasn't changed since last review

Medium severity Local-index parity test omits content search

scripts/​qualification/​qualify.py:262

The local-index qualification only exercises --files; no content search runs before the server starts. A release with broken local indexed matching could therefore pass despite the PR and CONTRIBUTING.md claiming ordinary local-index search parity. Reuse parity here so both file listing and content search are checked against --no-index.

Check both default and hidden local content/file results against proven filesystem scans and require local-only content diagnostics.

Co-authored-by: Copilot App <[email protected]>
Copilot AI balanced review requested due to automatic review settings October 6, 2026 06:26

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The implementation is bounded, comprehensively tested, and the current all-platform qualification run succeeded.

Review effort: Balanced
Findings: None

@shengyfu
Shengyu Fu (shengyfu) merged commit a2c42a5 into main Oct 6, 2026
17 checks passed
@shengyfu
Shengyu Fu (shengyfu) deleted the shengyfu-shared-release-qualification branch October 6, 2026 15:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants