Claude Code, Skill
cultivar
A CLI tool to help you write tests for skills, test them across agents, and iterate until they work, from the Pinecone DevRel team.
Compatibility: Not stated.
Grade C
One Medium finding. Read the findings before installing.
Not a guarantee. A hygiene check of the source at the commit shown, not a security audit.
PERMISSION_SHELL_UNDECLARED
Why this grade
- PERMISSION_SHELL_UNDECLARED Medium
Runs shell commands that the manifest or description does not mention.
evals/hello.py, line 225
proc = subprocess.run(
Any High, or two or more Medium findings, give D. One Medium gives C. Low findings or a security Note give B. Each code names a finding behind the letter.
Grades are a hygiene check of the source at the commit shown, on the date checked, not a security audit or a guarantee. The free directory grades are automated. Before release, a PlugVet engineer reviews every paid report and any grade we hold back, including all F grades, against the source code and the evidence we quote.
What was checked
Plain notes from the hygiene checks.
Permissions
Concern
requests shell access. No manifest field names shell or command execution. can delete files.
PERMISSION_SHELL_UNDECLARED PERMISSION_DELETE_FILES
evals/hello.py, line 225
proc = subprocess.run(
evals/hello.py, line 272
shutil.rmtree(smoke_root, ignore_errors=True)
evals/run.py, line 167
setup_result = subprocess.run(t["setup"], shell=True, capture_output=True, text=True)
Show more
evals/run.py, line 214
verify_result = subprocess.run(
evals/run.py, line 246
td_result = subprocess.run(t["teardown"], shell=True, capture_output=True, text=True)
evals/run.py, line 659
result = subprocess.run(cmd)
evals/runners/base.py, line 67
result = subprocess.run(cmd, capture_output=True, text=True, timeout=timeout, **kwargs)
evals/runners/gemini.py, line 98
subprocess.run(
Install and update
Note
The docs document an installer that pipes a download into a shell.
INSTALL_DOCUMENTED_INSTALLER
evals/remote/modal_runner.py, line 29
"curl -fsSL https://deb.nodesource.com/setup_22.x | bash -",
Network
Concern
can reach a host that is not written as a fixed address.
NETWORK_NO_FIXED_HOST
evals/framework/grader.py, line 320
with urllib.request.urlopen(req, timeout=20) as r:
Secrets
Concern
reads a credential or environment file.
SECRETS_ENV_READ SECRETS_PATH_DOTENV
evals/cli.py, line 12
load_dotenv(dotenv_path=Path.cwd() / ".env", override=False)
evals/hello.py, line 71
has_key = bool(os.environ.get("ANTHROPIC_API_KEY"))
evals/hello.py, line 77
"hint": "" if has_key else "Add to .env in cwd or export in shell - or pass --no-grade.",
Show more
evals/hello.py, line 80
has_ts_key = bool(os.environ.get("TYPESAFE_API_KEY"))
evals/hello.py, line 106
modal_auth = (Path.home() / ".modal.toml").exists() or bool(os.environ.get("MODAL_TOKEN_ID"))
evals/run.py, line 614
f"Add {key_name}={example} to a .env in this directory (auto-loaded),\n"
evals/framework/grader.py, line 31
if not os.environ.get("ANTHROPIC_API_KEY"):
evals/framework/grader.py, line 36
" - Add [bold]ANTHROPIC_API_KEY=sk-ant-...[/bold] to a .env file "
evals/framework/grader.py, line 213
WORKDIR_INCLUDE_NAMES = {"Dockerfile", "Makefile", "requirements.txt", "pyproject.toml", "package.json", ".env.example"}
evals/framework/grader.py, line 1011
Keys are auto-loaded from .env in cwd, and only the backends actually used
evals/framework/typesafe_grader.py, line 127
if not os.environ.get("TYPESAFE_API_KEY"):
evals/framework/typesafe_grader.py, line 132
" - Add [bold]TYPESAFE_API_KEY=...[/bold] to a .env file "
evals/remote/modal_runner.py, line 17
SECRET_NAME = os.environ.get("CULTIVAR_MODAL_SECRET", "eval-sandbox-secrets")
.env.example, line 1
# Copy this file to .env and fill in the values you need. Auto-loaded by
Agent instructions
Pass
No instruction to ignore earlier instructions or to hide a step.
Obfuscation
Note
a file was over the read cap and was not fully read.
OBFUSCATION_FILE_OVER_READ_CAP
uv.lock, line 1
version = 1
Licence
Pass
A licence file is present (LICENSE).
Maintainer activity
Pass
The latest commit is within the last 365 days.
Note
Hygiene check of commit 7a4b035e04eb. requests shell access. No manifest field names shell or command execution. can delete files. The docs document an installer that pipes a download into a shell. can reach a host that is not written as a fixed address. reads a credential or environment file. a file was over the read cap and was not fully read.
Date and source
Date checked:
github.com
Commit graded: 7a4b035e04ebe9e85869f27866306c2ab76dbe31
This link goes to the original host. PlugVet does not host or check what you download from there, and installing it is your own decision and risk.