We’ve integrated Tencent’s AI-Infra-Guard (AIG) into ClawScan, the open-source command-line tool that powers security review on ClawHub. Every skill and plugin uploaded to ClawHub now runs through AIG as part of its security review.
Our ClawHub Security Signals paper showed how differently scanners can read the same skill. Each brings its own view of risk, and preserving those differences gives us more evidence to work with.
ClawScan runs AIG and NVIDIA’s SkillSpector security scanner independently on each skill or plugin. An AI judge then reviews both scanners’ findings alongside the uploaded files and makes a final security assessment.
Contributors can test different scanners, AI models, and review instructions against the same benchmark to measure whether a change improves the results.
What Tencent AIG adds
Tencent describes AIG’s skill scanner as “an LLM-driven multi-stage code audit and vulnerability review pipeline.” It can follow the relationship between a skill’s instructions and the scripts, dependencies, and data flows behind them.
AIG reviews nine categories of risk, from instruction hijacking and memory poisoning to remote payload execution, unauthorized access, persistence, and insecure dependencies.
Benchmarking the combined system
We worked with Tencent to evaluate ClawScan on a fixed 556-case subset of SkillTrustBench, the public benchmark developed by Tencent and the Chinese University of Hong Kong, Shenzhen.
SkillTrustBench includes benign, suspicious, and malicious skills across nine risk categories. It tests whether scanners catch risky behavior, avoid flagging legitimate skills, and distinguish security flaws from malicious intent.
The evaluation helped us validate the scanner settings and the combined review process. Tencent found that AIG and SkillSpector surfaced different risks even when using the same AI model.
Across those 556 cases, ClawScan matched 86.9% of the benchmark’s labels and correctly classified 98.6% of malicious cases.
Improving both projects together
We now share anonymized cases where the scanners disagree, along with confirmed false positives, with Tencent. These examples feed into regression tests and improvements to AIG’s detection rules and review process. We evaluate those changes through ClawScan, creating a feedback loop that improves both projects.
Keeping this work open lets contributors build on the integration and bring their own evidence to the next round of improvements.
Acknowledgments
We’re grateful to the Tencent team for contributing their scanner, benchmark, and time to this work, from reviewing individual results to resolving production issues.