Open-source release · 227 audited skills

Audit agent skills before they touch your system.

Should you install this skill? See the safety and utility evidence for every skill before you trust it.

Try

skills audited end to end

flagged unsafe to install

exploits confirmed in sandbox

independent judge items

See it in action

See SkillAudit in action.

A 17-second walkthrough, from picking a skill to reading its safety and utility verdict.

01Method · four axes

One execution pass. Four independent verdicts.

The same paired run produces utility, efficiency, cost, and safety. We never blend them into a single number, because a skill that helps can still be the one that leaks your keys.

Axis · Safety

Does it stay in its sandbox?

A static scan plus a live containerized run record every file write and outbound packet. Each finding is weighted by existence and exploitability confidence.

41of 227 audited skills triggered a confirmed exploit in our sandbox. Nearly one in five.

93 exploits confirmed · 241 findings flagged

Axis · Utility

Does it help?

Mean pass-rate gain over matched with / without pairs.

+1.0lifts the pass rate by a full 100 points

+0.5lifts the pass rate by 50 points

0no measurable change from the skill

range [0, 1]

Axis · Efficiency

Is it faster?

Relative wall-clock time saved versus the no-skill baseline.

1.0runs in near-zero time next to the baseline

0same wall-clock as the no-skill run

< 0slower than running without it

range (-∞, 1]

Axis · Cost

Is it cheaper?

Relative effective input tokens saved on the same runs.

0same input tokens as the no-skill run

< 0uses more tokens than the baseline

range (-∞, 1]

Axis · Utility distribution

Most skills help a little. A few help a lot.

Pass-rate gain across 226 skills with a valid paired run. The long tail on the right is where the genuinely capability-adding skills live.

85of 226 skills gain under 0.1 pass rate

23clear a 0.5 gain, the genuine capability adds

buckets of 0.1 from 0 to 1.0 pass-rate gain

02Evidence

Every skill, plotted by what matters.

One dot per audited skill: utility gain against safety score, sized by findings. Many look helpful, but 41 triggered a confirmed exploit. The most dangerous sit in the lower right: they help the agent and are not safe to run.

Safe (80 to 100) Caution (60 to 80) Unsafe (below 60) dot size = findings

03Capability by domain

What skills add, domain by domain.

Resolution rate across nine professional-domain groups (O*NET occupation taxonomy), per model and harness. The pale fill is the lift skills add over the no-skill baseline. That lift is the whole point: skills make agents materially more capable, which is exactly why an unvetted one is worth worrying about. Hover an axis to inspect a domain; click any agent to compare, up to four at a time.

rings at 20 to 100%

dashed line = without skills · pale fill = with skills · the gap between them is the skill lift

04Leaderboard

Leaderboard.

Sorted by utility by default. High utility does not mean safe to install. Click a column to re-sort.

Skill ▲▼	Domain ▲▼	Utility ▲▼	Safety ▲▼	Findings ▲▼	Verdict ▲▼

View all 227 audits

05Risky skills

Riskiest skills you should know about.

93 confirmed exploits across 41 skills

06Pipeline

How a skill gets scored.

Every skill runs the same five-step pipeline. One execution pass, four axes, full evidence trail.

Profile

Static scan of SKILL.md, scripts, and dependencies. Each finding gets an existence confidence.

static_scan.json

Generate

Capability-targeted scenarios with binary judge items, calibrated to baseline difficulty.

scenarios/U*.yaml

Execute

Containerized paired runs, with and without the skill. The sandbox records file diff and network in real time.

fs.diff · net.log

Judge

An LLM judge scores each item. The security judge composes existence by exploitability against the trace.

judges/*.json

Report

One report, four scores, every judge item and finding rationale kept for reproduction.

skill_report.json

07Reproduce

Cite and reproduce.

BibTeX

@misc{skillaudit2026,
  title  = {SkillAudit: From Fixed-Suite Benchmarking to
            Skill-Centered Assessment},
  author = {SkillAudit Contributors},
  year   = {2026},
  eprint = {2606.22613},
  archivePrefix = {arXiv},
  primaryClass  = {cs.AI},
  note   = {https://skillaudit.github.io/}
}

Read on arXiv ↗

Reproduce

Every skill_report.json with judge items, finding rationale, paired numbers and severity weighting, bundled for all 227 evaluated skills.

Download benchmark.json

~4.5 MB · schema skillaudit_benchmark_v1 · frozen 2026-05-04

A Chromium extension renders this verdict directly on GitHub and the four skill marketplaces, at the moment of install.