Repository navigation
Add cmb2-playwright-qa agent skill and Playwright QA environment config - #1564
Merged
Merged
Conversation
…ugin Co-Authored-By: Claude Fable 5 <[email protected]> Claude-Session: https://claude.ai/code/session_014MBFkWgYTTtvSKHXJcQYea
Three gaps a run surfaces only after it is already wasted: A skill that describes how to run browser QA but never when not to gets used for changes with no rendered output at all. Route sanitization, escaping and REST permission work to PHPUnit, and a single "does it render?" look to claude-in-chrome, before a Playwright run is stood up. Ordinal artifact names (claim-01-before.png) push the cost onto the reviewer, who must replay the run to learn what the screenshot was meant to prove. Name the file after the claim instead; the bundled exploratory script now models it. pass/fail left no honest home for a claim that could not be run, so an infrastructure dependency or a deliberately preserved site state had to be logged as a failure or dropped. Skipped is now a first-class result and always carries its reason. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01RX5t3Pbg1wSBjz5FKqzNE3
2 tasks done
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## develop #1564 +/- ##
==========================================
Coverage 68.41% 68.41%
Complexity 1763 1763
==========================================
Files 52 52
Lines 4774 4774
==========================================
Hits 3266 3266
Misses 1508 1508
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds the
cmb2-playwright-qaagent skill: a documented, repeatable way to turn a PR's testing procedure, a review discussion, or an issue repro into a temporary evidence-focused Playwright run against the isolated wp-env suite — claim-level screenshots, a filled report template, and criteria for when ad-hoc coverage should graduate intotests/playwright/.Alongside the skill (in
.agents/skills/cmb2-playwright-qa/, symlinked from.claude/skills/), this carries the Playwright environment config its instructions reference:.env.local.example,SKIP_WP_SERVER/SKIP_WP_CHECKsupport viadotenv, atestIgnorefor exploratory scripts, and a gitignoredtests/playwright/.qa/artifacts directory.Motivation and Context
Manual browser QA of CMB2 changes kept being reinvented per PR — each run rediscovering the same environment traps (auth-state refresh, the tests env deactivating the plugin on re-provision). The skill captures the workflow and the traps once, so any agent or contributor can produce reviewable, claim-by-claim evidence instead of "I clicked around and it looked fine."
No linked issue.
Risk Level
Minimal: developer tooling only. No runtime plugin code is touched — the diff is agent-skill docs, Playwright config, and an exploratory-script template.
Testing procedure
The skill has been exercised end-to-end by two full QA runs against the isolated wp-env tests site (
.wp-env-tests.json, port 2623), most recently a 7-claim admin-notice matrix with all claims passing and every report link verified to resolve.npm run test:e2e(the durable suite) is unaffected: the newtestIgnorekeeps exploratory.qa/scripts out of it.Types of changes
Checklist:
Screenshots
N/A (developer tooling; no UI changes).
🤖 Generated with Claude Code
https://claude.ai/code/session_01CWsz3J4MfzkanHyPrf3DU8