MCP, MCP server
PyreCrawl
One command gives any AI agent the whole web. Scrape, extract, crawl, map, and search — self-hosted, no API keys, no rate limits, no subscription.
Compatibility: Not stated.
Grade B
Minor issues only, no Medium or High. Worth a quick look at the findings below before installing.
Not a guarantee. A hygiene check of the source at the commit shown, not a security audit.
Rescanned at commit 657ad6576613, unchanged since the check below.
NETWORK_NO_FIXED_HOST PERMISSION_SHELL_DECLARED INSTALL_DOCUMENTED_INSTALLER OBFUSCATION_FILE_OVER_READ_CAP
Why this grade
- NETWORK_NO_FIXED_HOST Low
Makes a network call whose host is not written as a fixed address.
scripts/stats.py, line 27
with urllib.request.urlopen(req, timeout=15) as resp: - PERMISSION_SHELL_DECLARED Note
Runs shell commands, and the manifest or description says so.
scripts/probe_stdio.py, line 12
proc = subprocess.Popen( - INSTALL_DOCUMENTED_INSTALLER Note
Documentation shows an installer that pipes a download into a shell.
README.md, line 76
curl -LsSf https://astral.sh/uv/install.sh | sh - OBFUSCATION_FILE_OVER_READ_CAP Note
A file was over the read cap and was not fully read.
uv.lock, line 1
version = 1
Any High, or two or more Medium findings, give D. One Medium gives C. Low findings or a security Note give B. Each code names a finding behind the letter.
Grades are a hygiene check of the source at the commit shown, on the date checked, not a security audit or a guarantee. The free directory grades are automated. Before release, a PlugVet engineer reviews every paid report and any grade we hold back, including all F grades, against the source code and the evidence we quote.
What was checked
Plain notes from the hygiene checks.
Permissions
Note
requests shell access. The manifest names this permission.
PERMISSION_SHELL_DECLARED
scripts/probe_stdio.py, line 12
proc = subprocess.Popen(
scripts/probe_v2.py, line 19
proc = subprocess.Popen(
scripts/stats.py, line 56
raw = subprocess.run(
Show more
scripts/test_tempmail_mcp.py, line 16
proc = subprocess.Popen(
src/pyrecrawl/cli.py, line 280
rc = subprocess.call(s)
src/pyrecrawl/updater.py, line 110
rc = subprocess.call([uv, "tool", "upgrade", "pyrecrawl"])
Install and update
Note
README documents an installer that pipes a download into a shell.
INSTALL_DOCUMENTED_INSTALLER
README.md, line 76
curl -LsSf https://astral.sh/uv/install.sh | sh
Network
Concern
can reach a host that is not written as a fixed address.
NETWORK_NO_FIXED_HOST
scripts/stats.py, line 27
with urllib.request.urlopen(req, timeout=15) as resp:
src/pyrecrawl/engines.py, line 642
return StealthyFetcher.fetch(url, **kwargs)
src/pyrecrawl/engines.py, line 917
return _SF2.fetch(url, headless=True, network_idle=True, timeout=60000)
Show more
src/pyrecrawl/engines.py, line 1498
limit: how many search results to fetch (DDG free tier works fine
src/pyrecrawl/engines.py, line 1776
with urllib.request.urlopen(req, timeout=timeout) as resp:
src/pyrecrawl/telemetry.py, line 96
with urllib.request.urlopen(req, timeout=3) as resp:
src/pyrecrawl/updater.py, line 37
with urllib.request.urlopen(req, timeout=timeout) as resp:
workers/telemetry/src/worker.js, line 13
async fetch(request, env) {
workers/telemetry/src/worker.js, line 78
const r = await fetch("https://pypistats.org/api/packages/pyrecrawl/overall", {
Secrets
Pass
No read of a credential file or secret environment variable found.
Agent instructions
Pass
No instruction to ignore earlier instructions or to hide a step.
Obfuscation
Note
a file was over the read cap and was not fully read.
OBFUSCATION_FILE_OVER_READ_CAP
uv.lock, line 1
version = 1
Licence
Pass
A licence file is present (LICENSE).
Maintainer activity
Pass
The latest commit is within the last 365 days.
Note
Hygiene check of commit 657ad6576613. requests shell access. The manifest names this permission. README documents an installer that pipes a download into a shell. can reach a host that is not written as a fixed address. a file was over the read cap and was not fully read.
Date and source
Date checked:
github.com
Commit graded: 657ad65766130bb7958e9bc5344fc5f0696fabe5
This link goes to the original host. PlugVet does not host or check what you download from there, and installing it is your own decision and risk.