Repository navigation
Conversation
The SC8 shipped-bytecode walk stopped after 5 s of wall-clock time. Analyzers run concurrently in one process, so the walk spent most of that time waiting for the GIL behind CPU-bound analyzers. Skills that ship no bytecode got a static runtime_limit ledger row at a different folder on every run, which SkillEvaluator reports as a Critical "skillspector scan did not execute reliably" (SKILLEVAL-17). Measure the 5 s processing budget with time.thread_time(), as static_yara already does for rule loading. Wall-clock time stays bounded by the remaining workflow deadline, or by MAX_WORKFLOW_SECONDS for callers outside a workflow. Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
chrisknvidia
left a comment
There was a problem hiding this comment.
Reviewed commit 922317e167a47d2ac3819c28edb481519f90d2e2. Looks good to me; no blocking findings.
I checked CPU-time accounting in _scan_shipped_bytecode, propagation of the remaining workflow wall-clock budget, limitation reporting, and the existing traversal/count/finding bounds. All 36 focused SC8 and budget-configuration tests passed.
I also exercised the actual filesystem walk under concurrent CPU-thread load: the baseline hit its wall-clock limit, while this version completed without a limitation. Positive controls still reported shipped bytecode, and an explicit wall-clock allowance still stopped the walk with runtime_limit.
Validation was local on macOS. No live LLM-provider or production validation was performed.
Problem
The SC8 shipped-bytecode walk (
_scan_shipped_bytecode) stops afterMAX_SC8_ANALYSIS_SECONDS = 5.0of wall-clock time. All analyzers run in parallel threads in one process. Eachos.scandirstep releases the GIL, and the walk then waits behind CPU-bound analyzers to get it back. A walk that needs a few milliseconds of work can therefore run out of time.In a customer report (SKILLEVAL-17), a skill with no
__pycache__,.pycor.pyofiles got a staticruntime_limitledger row. The folder in it changed on every run, because it was wherever the walk had reached when time ran out. The SC8 walk started about 190 s into the scan. SkillEvaluator reports any staticruntime_limitas a Critical "skillspector scan did not execute reliably", so the row blocked submission.Reproduction: the real v2.12.0
_scan_shipped_bytecodeon a tree with the customer's layout (148 files, 50 folders, no bytecode):runtime_limitatfeatures/safetyruntime_limitFix
time.thread_time(), so only the walk thread's own CPU time counts.static_yaraalready does this for rule loading.timeout_seconds), whichresolve_inputalways sets for graph scans. Callers outside a workflow fall back toMAX_WORKFLOW_SECONDS.Tests
limit_seconds == MAX_SC8_ANALYSIS_SECONDS.MAX_WORKFLOW_SECONDSwhen there is none.tests/nodes/analyzers/test_sc8_shipped_bytecode.py: 19 passed. Supply-chain, static-pattern, workflow and transitive selection oftests/nodes/analyzersandtests/unit: 1018 passed.🤖 Generated with Claude Code