Repository navigation
Allow the 60s aggregate workflow budget to be raised (env var or CLI flag) #460
Description
Version: 2.11.0
MAX_WORKFLOW_SECONDS = 60.0 in skillspector/state.py is the aggregate deadline for a whole scan
workflow, consumed by WorkflowResourceBudget.remaining_seconds(). On a large skill the budget is
exhausted part-way through, and the scan returns a partial result with truncation_reasons
populated and coverage_percent below 100.
There is currently no way to raise it: no CLI flag sets a budget or timeout, and the only
environment override anywhere in the package is SKILLSPECTOR_OSV_TIMEOUT in
nodes/analyzers/osv_client.py.
Why this is a problem rather than a preference. We use a complete scan as an intake gate on
third-party skills — a skill is not accepted until it has a scan with full coverage. A skill large
enough to reliably exceed 60 seconds therefore cannot pass the gate at all, because it cannot
produce the artifact the gate requires. Observed on a 147-file skill: five of six whole-skill
attempts truncated, with coverage varying between 0% and 100% run to run purely on timing.
A truncated result is not simply a subset of a complete one, which is what makes the partial
output hard to work with. On the same skill, the truncated whole-skill run reported 69 findings at
63.3% coverage, while a complete run over disjoint decomposed units reported 29 findings at
100%. Units scanned in isolation lose reference-graph relationships that cross unit boundaries, so
more coverage legitimately produced fewer findings. A consumer cannot treat a truncated run as "the
complete run minus what it missed".
Request. Make the aggregate budget configurable, keeping 60s as the default.
WorkflowResourceBudget.max_seconds is already a defaulted dataclass field, so either would work:
- a
SKILLSPECTOR_MAX_WORKFLOW_SECONDSenvironment variable, mirroring the existing
SKILLSPECTOR_OSV_TIMEOUThandling exactly; or - a
--max-secondsCLI flag threaded into the budget at construction.
An env var alone would be enough for our case and matches precedent already in the codebase.
Current workaround, and why it is not sufficient. We decompose a large skill into disjoint
subdirectory units, scan each within the 60s budget, recurse where a unit still truncates, and
aggregate (sum severities, max the score, file-count-weighted mean coverage). It does produce a
complete scan, but it costs the cross-unit context described above and takes roughly 20 minutes of
wall clock for one skill, so it cannot run interactively.
Activity
rioyu123 commentedon Sep 1, 2026
Would you like a single SKILLSPECTOR_MAX_WORKFLOW_SECONDS setting to apply to both direct graph scans and the aggregate --transitive traversal, while keeping 60 seconds as the default for both?
There are currently two independent 60-second defaults (WorkflowResourceBudget.max_seconds and _TransitiveBudget.max_seconds). I can prepare a focused implementation that resolves the environment value at budget construction time, accepts only finite positive values, and preserves explicit constructor overrides, but I want to confirm the shared direct/transitive semantics before changing the public contract.
Saif-03 commentedon Sep 1, 2026
@rioyu123
Yes, one SKILLSPECTOR_MAX_WORKFLOW_SECONDS covering both the direct scan and the --transitive aggregate is exactly what I need, no reason to keep them separate. Default of 60s for both is fine. Heads up, PR #468 looks like it's already implementing this if you want to coordinate rather than duplicate. Thanks
rioyu123 commentedon Sep 1, 2026
Thanks for confirming. PR #468 covers the shared setting and the 60-second default for both direct and transitive scans, and its CI is green. I'll avoid duplicating the work and defer to that implementation.
Patrick-Erichsen commentedon Sep 4, 2026
Adding a concrete public ClawHub example for this request and #468:
@byungkyu/api-gateway, version 1.2.0.CRITICAL / DO_NOT_INSTALL, with 98AE1"analysis-evasion" findings for reference files that are present in the package openclaw/clawhub#3603SKILL.mdand 167 Markdown reference documents.The author reports reproducing the failure directly, without ClawHub or an LLM, on the downloaded skill tree:
Their report records failed execution, 0% completeness, and all 235 files partially inspected. It includes 287
runtime_limitledger exceptions and 186 unresolved-reference exceptions, while the risk assessment is score 100 / CRITICAL / DO_NOT_INSTALL with 695 findings. Of those, 98 are AE1 findings referring to files the author says are present locally. The original issue contains the report excerpts and file-presence checks.Verification boundary: I independently checked the exact-version public ClawHub API on September 4, 2026: the SkillSpector summary still shows score 100, CRITICAL, DO_NOT_INSTALL, and 695 findings, while the separate ClawScan judge returns benign. I have not independently rerun the author's raw SkillSpector timeout reproduction or established that increasing the deadline alone resolves it.
This would be a useful whole-skill regression case for the proposed
SKILLSPECTOR_MAX_WORKFLOW_SECONDSsetting. Keeping the existing default and allowing an operator to choose a higher finite budget would avoid having to split the skill and lose cross-file reference context. Please also clarify how the override interacts with the independent discovery/context and other stage-level limits, so a higher aggregate budget does not leave an unexpected shorter ceiling in place.ClawHub separately drops execution-completeness metadata from its stored summary; that is our reporting issue to address, not a request for SkillSpector to treat an incomplete scan as clean.
chrisknvidia commentedon Sep 4, 2026
Adding another reproducible case: identical skill content passes with SkillSpector 2.9.6 and consistently returns incomplete execution with 2.11.0 because
meta_analyzerreaches the shared workflow runtime limit. No skill behavior change was involved. This provides another concrete reproducer for #460; we can retest it against #468.