Claude Code, Skill
Hermes Jailbench
Jailbreak regression benchmark for LLM endpoints with repeatable known-pattern attacks and deterministic scoring
Compatibility: Not stated.
Grade C
One Medium finding. Read the findings before installing.
Not a guarantee. A hygiene check of the source at the commit shown, not a security audit.
Rescanned at commit 4af699364b22.
PERMISSION_SHELL_UNDECLARED
Why this grade
- PERMISSION_SHELL_UNDECLARED Medium
Runs shell commands that the manifest or description does not mention.
hermes_jailbench/evidence.py, line 106
return subprocess.run(
Any High, or two or more Medium findings, give D. One Medium gives C. Low findings or a security Note give B. Each code names a finding behind the letter.
Grades are a hygiene check of the source at the commit shown, on the date checked, not a security audit or a guarantee. The free directory grades are automated. Before release, a PlugVet engineer reviews every paid report and any grade we hold back, including all F grades, against the source code and the evidence we quote.
What was checked
Plain notes from the hygiene checks.
Permissions
Concern
requests shell access. No manifest field names shell or command execution.
PERMISSION_SHELL_UNDECLARED
hermes_jailbench/evidence.py, line 106
return subprocess.run(
Install and update
Pass
No install-time script found, and dependencies are pinned or absent.
Network
Concern
can reach a host that is not written as a fixed address.
NETWORK_NO_FIXED_HOST
hermes_jailbench/providers.py, line 266
with urllib.request.urlopen(request, timeout=self.timeout) as response:
Secrets
Concern
reads a credential or environment file.
SECRETS_ENV_READ SECRETS_PATH_DOTENV
hermes_jailbench/cli.py, line 460
api_key = args.api_key or os.environ.get("OPENAI_API_KEY")
hermes_jailbench/cli.py, line 462
api_key = args.api_key or os.environ.get("ANTHROPIC_API_KEY")
.gitignore, line 58
.env
Show more
.gitignore, line 59
.env.local
Agent instructions
Pass
No instruction to ignore earlier instructions or to hide a step.
Obfuscation
Pass
No encoded or minified payload found.
Licence
Pass
A licence file is present (LICENSE).
Maintainer activity
Pass
The latest commit is within the last 365 days.
Note
Hygiene check of commit 4af699364b22. requests shell access. No manifest field names shell or command execution. can reach a host that is not written as a fixed address. reads a credential or environment file.
Date and source
Date checked:
github.com
Commit graded: 4af699364b222010b6814ebbe07cdb3c64c6adbe
This link goes to the original host. PlugVet does not host or check what you download from there, and installing it is your own decision and risk.