Grades are a hygiene check, not a security guarantee.

MCP, MCP server

RAGScore

RAGScore, an MCP server listed in the PlugVet directory.

Compatibility: Not stated.

Grade D

Two or more Medium findings, or a High finding. Only try it in a sandbox.

Graded by PlugVetRubric v2

Not a guarantee. A hygiene check of the source at the commit shown, not a security audit.

Rescanned at commit 3fbc6e8bb21c, unchanged since the check below.

PERMISSION_SHELL_UNDECLARED INSTALL_REMOTE_SCRIPT

Why this grade

  • PERMISSION_SHELL_UNDECLARED Medium

    Runs shell commands that the manifest or description does not mention.

    examples/complete_demo.ipynb, line 253 "# subprocess.Popen(\"ollama serve\", shell=True, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)\n",

  • INSTALL_REMOTE_SCRIPT Medium

    Fetches a script from the network and runs it (a pipe to a shell, or an install hook with a URL).

    examples/complete_demo.ipynb, line 251 "# !curl -fsSL https://ollama.com/install.sh | sh\n",

Any High, or two or more Medium findings, give D. One Medium gives C. Low findings or a security Note give B. Each code names a finding behind the letter.

If you think this grade is wrong, email [email protected] with the entry link and your evidence. We aim to acknowledge within two working days and decide within seven. If we confirm an error, we aim to correct it within 24 hours of confirming it. These are aims, not guarantees.

Grades are a hygiene check of the source at the commit shown, on the date checked, not a security audit or a guarantee. The free directory grades are automated. Before release, a PlugVet engineer reviews every paid report and any grade we hold back, including all F grades, against the source code and the evidence we quote.

What was checked

Plain notes from the hygiene checks.

  • Permissions

    Concern

    requests shell access. No manifest field names shell or command execution.

    PERMISSION_SHELL_UNDECLARED

    examples/complete_demo.ipynb, line 253

    "# subprocess.Popen(\"ollama serve\", shell=True, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)\n",

    examples/demo.ipynb, line 81

    "subprocess.Popen(\"ollama serve\", shell=True, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)\n",

    examples/ollama_local_demo.ipynb, line 52

    "subprocess.run([\"apt-get\", \"install\", \"-y\", \"-q\", \"zstd\"], check=True)\n",

    Show more

    examples/ollama_local_demo.ipynb, line 53

    "subprocess.run([\"pip\", \"install\", \"-q\", \"ragscore[notebook]>=0.7.7\", \"numpy\"], check=True)\n",

    examples/ollama_local_demo.ipynb, line 57

    "result = subprocess.run(\n",

    examples/ollama_local_demo.ipynb, line 68

    "subprocess.Popen([ollama_bin, \"serve\"], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)\n",

    examples/ollama_local_demo.ipynb, line 73

    "subprocess.run([ollama_bin, \"pull\", \"qwen2.5:1.5b\"], check=True)\n",

  • Install and update

    Serious

    install fetches a script from the network.

    INSTALL_REMOTE_SCRIPT

    requirements.txt, line 6

    pypdf>=4.0.0

    examples/complete_demo.ipynb, line 251

    "# !curl -fsSL https://ollama.com/install.sh | sh\n",

    examples/demo.ipynb, line 76

    "!curl -fsSL https://ollama.com/install.sh | sh\n",

    Show more

    examples/ollama_local_demo.ipynb, line 58

    " \"curl -fsSL https://ollama.ai/install.sh | sh\",\n",

  • Network

    Concern

    can reach a host that is not written as a fixed address.

    NETWORK_NO_FIXED_HOST

    src/ragscore/cli.py, line 107

    with urllib.request.urlopen(req, timeout=2) as resp:

    src/ragscore/providers/factory.py, line 207

    response = requests.get("http://localhost:11434/api/tags", timeout=2)

    src/ragscore/providers/ollama_provider.py, line 95

    response = requests.get(f"{self.base_url}/api/tags", timeout=5)

    Show more

    src/ragscore/providers/ollama_provider.py, line 162

    response = requests.post(

    src/ragscore/providers/ollama_provider.py, line 220

    response = requests.post(

    src/ragscore/providers/ollama_provider.py, line 238

    response = requests.get(f"{self.base_url}/api/tags", timeout=10)

    examples/ollama_local_demo.ipynb, line 137

    " return json.loads(urllib.request.urlopen(req).read())[\"embeddings\"][0]\n",

    examples/ollama_local_demo.ipynb, line 148

    " return json.loads(urllib.request.urlopen(req).read())[\"message\"][\"content\"]\n",

  • Secrets

    Concern

    reads a credential or environment file.

    SECRETS_ENV_READ SECRETS_PATH_DOTENV

    src/ragscore/config.py, line 6

    # Load environment variables from .env file

    src/ragscore/config.py, line 48

    raise ValueError(f"{env_var} not found. Please set it in your environment or .env file.")

    src/ragscore/config.py, line 53

    DASHSCOPE_API_KEY = os.getenv("DASHSCOPE_API_KEY")

    Show more

    src/ragscore/config.py, line 58

    POSTHOG_API_KEY = os.getenv("POSTHOG_API_KEY", "phc_19wLYHD0eHcMQel7WUBz2cXUHu22extqJXyqrXX9GuW")

    src/ragscore/exceptions.py, line 30

    f"Or in your .env file: {env_var}=your-key"

    src/ragscore/mcp_server.py, line 46

    if os.getenv("ANTHROPIC_API_KEY"):

    src/ragscore/providers/anthropic_provider.py, line 43

    self.api_key = api_key or os.getenv("ANTHROPIC_API_KEY")

    src/ragscore/providers/dashscope_provider.py, line 49

    api_key = api_key or os.getenv("DASHSCOPE_API_KEY")

    src/ragscore/providers/generic_provider.py, line 122

    self.api_key = api_key or os.getenv("LLM_API_KEY", "")

    src/ragscore/providers/openai_provider.py, line 55

    api_key = api_key or os.getenv("OPENAI_API_KEY")

    src/ragscore/providers/openai_provider.py, line 184

    api_key = api_key or os.getenv("AZURE_OPENAI_API_KEY")

    .gitignore, line 5

    # Environment variables (NEVER commit API keys!)

    .gitignore, line 6

    .env

    .gitignore, line 7

    .env.local

    CHANGELOG.md, line 252

    - Removed RAGScore Pro marketing from .env.example

    CHANGELOG.md, line 367

    - Ensure `.env` file has `DASHSCOPE_API_KEY`

  • Agent instructions

    Pass

    No instruction to ignore earlier instructions or to hide a step.

  • Obfuscation

    Pass

    No encoded or minified payload found.

  • Licence

    Pass

    A licence file is present (LICENSE).

  • Maintainer activity

    Pass

    The latest commit is within the last 365 days.

Get the full report on this tool, £49

Note

Hygiene check of commit 3fbc6e8bb21c. requests shell access. No manifest field names shell or command execution. install fetches a script from the network. can reach a host that is not written as a fixed address. reads a credential or environment file.

Date and source

Date checked:

View source

github.com

Commit graded: 3fbc6e8bb21c99d50a8b6d7a6d57bf7a2100d21c

This link goes to the original host. PlugVet does not host or check what you download from there, and installing it is your own decision and risk.

Read how this letter is chosen