We’ve integrated Tencent’s AI-Infra-Guard (AIG) into ClawScan, the open-source command-line tool that powers security review on ClawHub. Every skill and plugin uploaded to ClawHub now runs through AIG as part of its security review.
\nOur ClawHub Security Signals paper showed how differently scanners can read the same skill. Each brings its own view of risk, and preserving those differences gives us more evidence to work with.
\nClawScan runs AIG and NVIDIA’s SkillSpector security scanner independently on each skill or plugin. An AI judge then reviews both scanners’ findings alongside the uploaded files and makes a final security assessment.
\nContributors can test different scanners, AI models, and review instructions against the same benchmark to measure whether a change improves the results.
\nTencent describes AIG’s skill scanner as “an LLM-driven multi-stage code audit and vulnerability review pipeline.” It can follow the relationship between a skill’s instructions and the scripts, dependencies, and data flows behind them.
\nAIG reviews nine categories of risk, from instruction hijacking and memory poisoning to remote payload execution, unauthorized access, persistence, and insecure dependencies.
\nWe worked with Tencent to evaluate ClawScan on a fixed 556-case subset of SkillTrustBench, the public benchmark developed by Tencent and the Chinese University of Hong Kong, Shenzhen.
\nSkillTrustBench includes benign, suspicious, and malicious skills across nine risk categories. It tests whether scanners catch risky behavior, avoid flagging legitimate skills, and distinguish security flaws from malicious intent.
\nThe evaluation helped us validate the scanner settings and the combined review process. Tencent found that AIG and SkillSpector surfaced different risks even when using the same AI model.
\nAcross those 556 cases, ClawScan matched 86.9% of the benchmark’s labels and correctly classified 98.6% of malicious cases.
\nWe now share anonymized cases where the scanners disagree, along with confirmed false positives, with Tencent. These examples feed into regression tests and improvements to AIG’s detection rules and review process. We evaluate those changes through ClawScan, creating a feedback loop that improves both projects.
\nKeeping this work open lets contributors build on the integration and bring their own evidence to the next round of improvements.
\nWe’re grateful to the Tencent team for contributing their scanner, benchmark, and time to this work, from reviewing individual results to resolving production issues.
","headings":[{"level":2,"text":"AIG is now part of ClawHub review","id":"aig-is-now-part-of-clawhub-review"},{"level":2,"text":"How ClawScan works","id":"how-clawscan-works"},{"level":2,"text":"What Tencent AIG adds","id":"what-tencent-aig-adds"},{"level":2,"text":"Benchmarking the combined system","id":"benchmarking-the-combined-system"},{"level":2,"text":"Improving both projects together","id":"improving-both-projects-together"},{"level":2,"text":"Acknowledgments","id":"acknowledgments"}]}}