Guide · Rubric and roundups
Best MCP server for scraping: the rubric, not a ranking
How we compare scraping MCP servers: a ten-row rubric, every table sourced and dated, and seven servers in category tables. No scores, no rankings.
Guide · Updated
This article was drafted with AI assistance from the sources listed below and edited and published by Teneo Labs. Disclosure: this site is run by Teneo Labs, the company behind Live Web Data API, which this article mentions.
- Transport and auth: stdio, Streamable HTTP, SSE; API key, bearer token or OAuth
- Tool surface: how many tools the server registers, and whether you can filter them
- Output: formats, main-content extraction and a per-call size cap
- Failure behavior: error text the model can act on, and the cost of a failed call
- Freshness: live fetch, cache with a max age, or no control
- Scope: JavaScript pages, interaction, crawling, search, structured extraction
- Hosting and license: hosted, self-hosted or both, and under which license
- Pricing shape, compliance posture and maintenance (last release date)
Most pages that answer "best MCP server for scraping" are rankings. This one is a rubric. A ranking tells you who won; a rubric tells you why a server fits one job and not another, and lets you check every claim yourself. The tables below cover seven servers in four categories. Each value comes from the project's own documentation and was checked on 2026-10-27.
How we check
We read each project's README, docs and recent release notes. A value goes into the table only if a primary source states it. Where the docs are silent, the cell says so instead of guessing. We do not run speed or accuracy benchmarks, so there are none here, and we do not copy numbers from other people's benchmarks into the tables.
Every table has a check date. MCP servers change fast: Firecrawl's MCP server, for example, now lists a full profile of more than 25 tools, many more than its first releases. A row older than 90 days, or older than a release that touches it, gets re-checked.
Rubric v1 explained
The rubric has ten rows. They are the questions a team actually argues about before standardizing on a server.
- Transport and auth. Local stdio, a remote Streamable HTTP URL, or both. Remote servers need a key, a bearer token or OAuth. This decides whether a server works in your client and whether it can run without anything installed.
- Tool surface. A client that loads tools upfront describes every registered tool to the model in each session; Claude Code defers them by default through tool search. Fewer tools means less context spent before the first page is read. Some servers let you include or exclude tools in the URL.
- Output. Markdown, text, HTML or JSON; main content only or the whole page; and a per-call cap on characters or tokens.
- Failure behavior. Does the error tell the model what to do next? Does a failed call cost anything?
- Freshness. Live fetch every time, a cache with a maximum age, or no control at all.
- Scope. JavaScript rendering, interaction, crawling, search and structured extraction. More scope is not better if your job does not need it.
- Hosting and license. Hosted only, self-hosted only, or both, and under which license.
- Pricing shape. Credits per page, pay per event, browser minutes, or your own machines. See the pricing models guide.
- Compliance posture. An acceptable-use policy, a stated position on robots.txt, and what is logged.
- Maintenance. The date of the last release and how active the repository is.
Category tables (checked 2026-10-27)
Every table has all ten rubric rows plus a source row. "Not documented" means the source linked in that column does not state the value; it is not a judgment about the project.
Browser-control servers
| Row | Playwright MCP | Scrapling MCP |
|---|---|---|
| Transport and auth | stdio by default; HTTP with --port | stdio by default; Streamable HTTP with --http and an auth token |
| Tool surface | About 25 core tools (browser_navigate, browser_snapshot, browser_click …); optional groups via --caps | 16 tools, from make_request to session and browser tools |
| Output | Accessibility snapshots; screenshots as a separate tool | Markdown, HTML, text or an AI snapshot; main content only by default |
| Failure behavior | Not documented | Not documented |
| Freshness | Live page in a browser; no cache control documented | Live request; no cache control documented |
| Scope | Interaction and JavaScript pages; no crawl tool | Requests, sessions and browser fetches; Markdown extraction |
| Hosting and license | Your machine or Docker; Apache-2.0 | Your machine; BSD-3-Clause |
| Pricing shape | Free to run; you pay compute | Free to run; you pay compute |
| Compliance | No robots.txt stance documented | HTTP mode requires a token by default; no robots.txt stance documented |
| Maintenance | Latest release 2026-09-28 | Latest release 2026-10-17 |
| Source | README, releases | MCP docs, releases |
Crawl platforms
| Row | Firecrawl MCP | Apify MCP |
|---|---|---|
| Transport and auth | stdio, local Streamable HTTP, hosted URL; API key, OAuth endpoint for hosted use | Hosted Streamable HTTP at mcp.apify.com, or stdio via npx; OAuth or bearer token |
| Tool surface | Full profile lists 25+ tools: scrape, map, crawl, search, interact, agent, monitors | Default set plus any Actor you add; the tools parameter narrows it |
| Output | formats option, onlyMainContent | Whatever each Actor returns |
| Failure behavior | Rate-limit errors surfaced to the client; automatic retries | Depends on the Actor |
| Freshness | Not documented in the README | Not documented |
| Scope | Scrape, map, crawl, search, browser interaction | Whatever the added Actors do |
| Hosting and license | Hosted, or self-hosted via FIRECRAWL_API_URL; MCP server MIT, core AGPL-3.0 | Hosted or local stdio; some Actors only via the hosted server; MIT |
| Pricing shape | Credits per request; error-status pages cost 1 credit | Per Actor: pay per event or pay per usage |
| Compliance | No robots.txt stance in the README | Not documented in the README |
| Maintenance | Latest MCP server release 2026-08-26 | Not documented |
| Source | README, pricing, releases | README, pricing models |
Self-hosted crawler
| Row | Crawl4AI (Docker server) |
|---|---|
| Transport and auth | SSE at /mcp/sse and WebSocket at /mcp/ws |
| Tool surface | 7 tools: md, html, screenshot, pdf, execute_js, crawl, ask |
| Output | Markdown or HTML per page, plus screenshots and PDFs |
| Failure behavior | Not documented on the self-hosting page |
| Freshness | Not documented on the self-hosting page |
| Scope | Crawling, JavaScript execution, screenshots |
| Hosting and license | Your infrastructure; Apache-2.0, with an attribution clause in the README |
| Pricing shape | Free to run; you pay compute |
| Compliance | Not documented on the self-hosting page |
| Maintenance | v0.9.5 released 2026-10-14 |
| Source | Self-hosting guide, releases |
Single-page content APIs
| Row | Jina MCP | Live Web Data API |
|---|---|---|
| Transport and auth | Remote Streamable HTTP; bearer key optional for some tools | Remote Streamable HTTP, bearer key |
| Tool surface | 12 tools; filter with include_tools / exclude_tools | 2 tools: get_page_contents, get_account_status |
| Output | read_url returns Markdown; max_tokens caps it | Markdown, text or HTML; max_characters per call |
| Failure behavior | Timeouts of 30 or 60 seconds; the README notes blocked pages can come back as ordinary content | One actionable error sentence; failed calls not charged |
| Freshness | Not documented in the README | Always live; no cache |
| Scope | Read, search (key needed), screenshots, reranking | One public page per call; the MCP tool has no JavaScript rendering, crawl, search or extraction |
| Hosting and license | Hosted; repository Apache-2.0 | Hosted only |
| Pricing shape | Billed in tokens; keyless use at lower rate limits | Credits per successful page |
| Compliance | Not documented in the README | Acceptable-use policy; honoring robots.txt is not promised (Terms, Section 8.6) |
| Maintenance | Not documented | Not documented (hosted service, no public release log) |
| Source | README | MCP docs |
What this means for your server choice
The tables do not crown a winner, because the categories answer different jobs. If your agent clicks and types, the browser rows matter. If it maps a site, look at the crawl platforms, and if you are leaving one, the Firecrawl MCP alternatives guide sorts the options by reason. If data must stay on your machines, read the hosted vs self-hosted guide. If it reads known public pages, the content APIs keep the context smallest; the output and token cost guide shows why. For the compliance row, the responsible scraping guide turns it into a review checklist. The fastest way to a shortlist is the pick-by-job guide.
Changelog
- Rubric v1, 2026-10-27. First published version: ten rows, seven servers, four categories.
- 2026-10-28. Tool-surface row description: tool definitions reach the model in every session only in clients that load them upfront; Claude Code defers them by default through tool search. Table values unchanged.
Where Live Web Data API sits in these tables
Live Web Data API is checked on the same ten rows as the other servers, including the rows where it falls short: no JavaScript rendering in the MCP tool, no crawl and no search. If reading known public pages is your job, read the MCP docs. Sign-up is for businesses only. Only Business, Enterprise and Custom sign-ups are reviewed before checkout, usually within 1 business day.
Frequently asked questions
Why don't you give each server a score out of 10?
Because nobody can reproduce it. A score mixes ten different questions into one number and hides the trade-off you care about. We publish the values and the source link for each row, so you can weight them for your own job.
How often are the rubric rows re-checked?
When a covered project ships a release that touches a row, and at least every 90 days. Each table carries its check date, and changes are logged in the changelog at the end of this guide.
Can I suggest a server or report a wrong row?
Yes, through the contact page. We re-check the row against the project's docs or release notes and log the change with the new date.
Sources
- firecrawl-mcp-server README (GitHub) (Documentation)
- Playwright MCP README (GitHub) (Documentation)
- Crawl4AI self-hosting guide and MCP endpoints (Documentation)
- Apify MCP server README (GitHub) (Documentation)
- Jina AI remote MCP server README (GitHub) (Documentation)
- Scrapling MCP server docs (Documentation)
- Crawl4AI releases (GitHub) (Release notes)
- Playwright MCP releases (GitHub) (Release notes)
- Scrapling releases (GitHub) (Release notes)
- firecrawl-mcp-server releases (GitHub) (Release notes)
- Firecrawl pricing (Documentation)
- Apify docs: Actors in Store and pricing models (Documentation)
- Claude Code docs: MCP, including tool search (Documentation)