Claude Code, Skill
proofrag
Evaluate a RAG/LLM app: generate a golden set from your docs, run LLM-as-judge + retrieval metrics, and produce a shareable HTML scorecard with a CI gate.
Compatibility: Not stated.
Grade C
One Medium finding. Read the findings before installing.
Not a guarantee. A hygiene check of the source at the commit shown, not a security audit.
Rescanned at commit 7c944e1528b4, unchanged since the check below.
NETWORK_HOST_FROM_INPUT
Why this grade
- NETWORK_HOST_FROM_INPUT Medium
A file takes a URL or endpoint as input and has no fixed host list.
case_studies/http_rfc_metadata/download_corpus.py, line 70
with urllib.request.urlopen(request, timeout=30) as response:
Any High, or two or more Medium findings, give D. One Medium gives C. Low findings or a security Note give B. Each code names a finding behind the letter.
If you think this grade is wrong, email [email protected] with the entry link and your evidence. We aim to acknowledge within two working days and decide within seven. If we confirm an error, we aim to correct it within 24 hours of confirming it. These are aims, not guarantees.
Grades are a hygiene check of the source at the commit shown, on the date checked, not a security audit or a guarantee. The free directory grades are automated. Before release, a PlugVet engineer reviews every paid report and any grade we hold back, including all F grades, against the source code and the evidence we quote.
What was checked
Plain notes from the hygiene checks.
Permissions
Note
requests shell access. The manifest names this permission.
PERMISSION_SHELL_DECLARED
devtools/lint.py, line 31
return subprocess.call(cmd)
Install and update
Pass
No install-time script found, and dependencies are pinned or absent.
Network
Serious
accepts a host from input, with no fixed list in this file.
NETWORK_HOST_FROM_INPUT
case_studies/http_rfc_metadata/download_corpus.py, line 70
with urllib.request.urlopen(request, timeout=30) as response:
case_studies/python_concurrency/download_corpus.py, line 60
with urllib.request.urlopen(request, timeout=30) as response:
Secrets
Concern
reads a credential or environment file.
SECRETS_ENV_READ SECRETS_PATH_DOTENV
devtools/lint.py, line 22
".env.example",
src/proofrag/llm.py, line 35
if os.environ.get("ANTHROPIC_API_KEY"):
src/proofrag/llm.py, line 38
if os.environ.get("OPENAI_API_KEY") or os.environ.get("OPENAI_BASE_URL"):
Show more
src/proofrag/llm.py, line 141
key = os.environ.get("OPENAI_API_KEY") or ("not-needed" if base else None)
src/proofrag/backends/deepeval_backend.py, line 43
key = os.environ.get("OPENAI_API_KEY") or ("not-needed" if base else None)
src/proofrag/backends/jev_backend.py, line 167
key = os.environ.get("TYPESAFE_API_KEY", "").strip()
.env.example, line 1
# Copy this file to `.env` and fill in your key: cp .env.example .env
.env.example, line 2
# `.env` is gitignored - never commit real keys.
.env.example, line 4
# proofrag does not auto-load .env. Load it before running, e.g.:
.env.example, line 5
# set -a && source .env && set +a
.gitignore, line 16
# secrets - never commit
.gitignore, line 17
.env
.gitignore, line 19
!.env.example
Agent instructions
Pass
No instruction to ignore earlier instructions or to hide a step.
Obfuscation
Note
a file was over the read cap and was not fully read.
OBFUSCATION_FILE_OVER_READ_CAP
uv.lock, line 1
version = 1
Licence
Pass
A licence file is present (LICENSE).
Maintainer activity
Pass
The latest commit is within the last 365 days.
Note
Hygiene check of commit 7c944e1528b4. requests shell access. The manifest names this permission. accepts a host from input, with no fixed list in this file. reads a credential or environment file. a file was over the read cap and was not fully read.
Date and source
Date checked:
github.com
Commit graded: 7c944e1528b4c24660686028496ce06dcc17c70f
This link goes to the original host. PlugVet does not host or check what you download from there, and installing it is your own decision and risk.