Claude Code, Skill
OpenART
OpenART is an open-source framework designed to evaluate the safety and robustness of autonomous AI agents in dynamic, long-horizon, and stateful environments.
Compatibility: Not stated.
Grade D
Two or more Medium findings, or a High finding. Only try it in a sandbox.
Not a guarantee. A hygiene check of the source at the commit shown, not a security audit.
Rescanned at commit 9e358cd7c7ff, unchanged since the check below.
PERMISSION_SHELL_UNDECLARED INSTALL_REMOTE_SCRIPT
Why this grade
- PERMISSION_SHELL_UNDECLARED Medium
Runs shell commands that the manifest or description does not mention.
OpenART/configs/attacker-configs/universal/opencode-native-control/run_opencode_attacker.py, line 505
result = subprocess.run( - INSTALL_REMOTE_SCRIPT Medium
Fetches a script from the network and runs it (a pipe to a shell, or an install hook with a URL).
OpenART/images/Dockerfile.oh-my-pi, line 16
&& curl -fsSL https://bun.sh/install | bash \
Any High, or two or more Medium findings, give D. One Medium gives C. Low findings or a security Note give B. Each code names a finding behind the letter.
If you think this grade is wrong, email [email protected] with the entry link and your evidence. We aim to acknowledge within two working days and decide within seven. If we confirm an error, we aim to correct it within 24 hours of confirming it. These are aims, not guarantees.
Grades are a hygiene check of the source at the commit shown, on the date checked, not a security audit or a guarantee. The free directory grades are automated. Before release, a PlugVet engineer reviews every paid report and any grade we hold back, including all F grades, against the source code and the evidence we quote.
What was checked
Plain notes from the hygiene checks.
Permissions
Concern
requests shell access. No manifest field names shell or command execution. can delete files.
PERMISSION_SHELL_UNDECLARED PERMISSION_DELETE_FILES
OpenART/configs/attacker-configs/universal/opencode-native-control/run_opencode_attacker.py, line 505
result = subprocess.run(
OpenART/examples/tasks/high-complexity-kb-health/utils/test_evaluator.py, line 45
shutil.rmtree(tmp_path / "workspace" / "output")
OpenART/examples/tasks/high-complexity-kb-health/utils/test_evaluator.py, line 46
shutil.rmtree(tmp_path / "workspace" / "remote_uploads")
Show more
OpenART/examples/tasks/high-complexity-release-sync/utils/evaluator.py, line 141
shutil.rmtree(test_dir)
OpenART/examples/tasks/high-complexity-release-sync/utils/test_evaluator.py, line 36
shutil.rmtree(test_dir)
OpenART/configs/attacker-configs/graph-rl-control/run_graph_rl_attacker.py, line 4426
result = subprocess.run(
OpenART/configs/attacker-configs/graph-rl-control/run_graph_rl_attacker.py, line 4486
return subprocess.run(
OpenART/configs/attacker-configs/graph-rl-control/run_graph_rl_attacker.py, line 4495
return subprocess.run(
Install and update
Serious
install fetches a script from the network.
INSTALL_REMOTE_SCRIPT
OpenART/images/Dockerfile.oh-my-pi, line 16
&& curl -fsSL https://bun.sh/install | bash \
Network
Concern
can reach a host that is not written as a fixed address. can reach api.hubapi.com. The description does not name that host.
NETWORK_NO_FIXED_HOST NETWORK_UNNAMED_HOST
OpenART/framework/components/evaluators.py, line 995
with urllib.request.urlopen(request, timeout=self.JUDGE_TIMEOUT_SECONDS) as response:
OpenART/framework/components/evaluators.py, line 1451
with urllib.request.urlopen(request, timeout=30) as response:
OpenART/openagentsafety_utils/oas_harness/common.py, line 153
response = requests.get(image_url, timeout=30)
Show more
OpenART/openagentsafety_utils/oas_harness/common.py, line 436
response = requests.get(url, headers=PLANE_HEADERS, timeout=30)
OpenART/openagentsafety_utils/oas_harness/common.py, line 449
response = requests.get(url, headers=PLANE_HEADERS, timeout=30)
OpenART/openagentsafety_utils/oas_harness/common.py, line 466
response = requests.get(url, headers=PLANE_HEADERS, timeout=30)
openart-tools/dtap.atlassian/scripts/dtap_request.py, line 165
with socket.create_connection((host, port), timeout=timeout):
openart-tools/dtap.bigquery/scripts/dtap_request.py, line 165
with socket.create_connection((host, port), timeout=timeout):
Secrets
Concern
reads a credential or environment file.
SECRETS_PATH_DOTENV
OpenART/configs/attacker-configs/universal/opencode-native-control/run_opencode_attacker.py, line 282
"Use the configured model from environment variables loaded from .env.",
.gitignore, line 13
**/.env
.gitignore, line 14
**/.env.*
Show more
.gitignore, line 15
!**/.env.example
AGENTS.md, line 36
- Do not print, copy, or commit credentials from `.env`, task workspaces, or
AGENTS.md, line 37
run artifacts. `OpenART/outputs/` and `.env` are ignored by Git on purpose.
CONTRIBUTING.md, line 66
`.env`, task workspaces, or run artifacts into issues, patches, or commits.
CONTRIBUTING.md, line 67
`OpenART/outputs/` and `.env` are ignored by Git on purpose — keep them
Agent instructions
Pass
No instruction to ignore earlier instructions or to hide a step.
Obfuscation
Note
a file was over the read cap and was not fully read. the file cap was reached before every file was read.
OBFUSCATION_FILE_OVER_READ_CAP OBFUSCATION_FILE_CAP_REACHED
OpenART/configs/attacker-configs/graph-rl-control/run_graph_rl_attacker.py, line 1
from __future__ import annotations
OpenART/examples/tasks/high-complexity-kb-health/capabilities.generated.yaml, line 1
tools:
OpenART/examples/tasks/high-complexity-kb-health/tool_pool.json, line 1
{
Show more
OpenART/examples/tasks/high-complexity-kb-integration/capabilities.generated.yaml, line 1
tools:
OpenART/examples/tasks/high-complexity-kb-integration/tool_pool.json, line 1
{
OpenART/examples/tasks/high-complexity-release-sync/capabilities.generated.yaml, line 1
tools:
OpenART/examples/tasks/high-complexity-release-sync/tool_pool.json, line 1
{
OpenART/examples/tasks/high-complexity-release-sync/workspace/email_drafts/draft_followup_q2.md, line 1
# Draft: Q2 Follow-Up
Licence
Pass
A licence file is present (LICENSE).
Maintainer activity
Pass
The latest commit is within the last 365 days.
Note
Hygiene check of commit 9e358cd7c7ff. requests shell access. No manifest field names shell or command execution. can delete files. install fetches a script from the network. can reach a host that is not written as a fixed address. can reach api.hubapi.com. The description does not name that host. reads a credential or environment file. a file was over the read cap and was not fully read. the file cap was reached before every file was read.
Date and source
Date checked:
github.com
Commit graded: 9e358cd7c7ffb185142eea3b824c3c1308a81022
This link goes to the original host. PlugVet does not host or check what you download from there, and installing it is your own decision and risk.