13 tools · $0.005/op · Open source

Web intelligence APIs
for AI agents

Scrape, validate, detect, audit, and monitor — 13 developer tools at $0.005 per operation. Built for automation pipelines, MCP servers, and AI agent workflows.

Try TheCrawler Freenpm install thecrawler
5 free per run·No credit card·AGPL-3.0

Flagship Tools

Six production-grade APIs. Each solves a specific web intelligence problem.

TheCrawler

$0.005

Web scraper with LLM-ready markdown

Cheerio + Playwright (adaptive)Turndown markdown with GFMHeading-aware RAG chunkingPDF & DOCX extraction+6 more

Tech Stack Detector

$0.02

Detect 5,772 technologies on any website

5,772 technology signatures105 categoriesVersion detectionConfidence scoring+3 more

Email Validator Pro

$0.005

SMTP verification with deliverability scoring

SMTP socket verificationMX record lookup with cacheCatch-all server detection5,365 disposable domain list+3 more

Broken Link Checker

$0.005

Find every dead link on your site

Recursive site crawlExternal link verificationSeverity classificationrobots.txt parsing+3 more

Lighthouse Auditor

$0.005

Batch performance audits at scale

PageSpeed Insights APICore Web Vitals (LCP, FID, CLS)Performance/SEO/Accessibility scoresCompetitor comparison mode+2 more

Website Change Monitor

$0.005

Know when any page changes

Text, hash, or selector modesVisual diff outputWebhook notificationsChange history tracking+1 more

Utility Tools

Seven additional tools for DNS, SSL, security, documents, and more.

DNS/WHOIS Suite

$0.004

Full DNS records + WHOIS in one call

Domain Age Checker

$0.004

RDAP-powered domain age lookup

SSL Monitor

$0.004

Certificate chain inspection + expiry alerts

Security Headers

$0.004

OWASP header audit with A+ to F grading

PDF Toolkit

$0.004

Extract text + metadata from any PDF

QR Code Toolkit

$0.004

Generate + decode QR codes

Google Sheets R/W

$0.004

Read, write, and transform Sheets programmatically

Fair pricing

Pay per operation. No subscriptions required. Free tier on every tool.

ToolOur PriceAlternativeTheir Price
TheCrawler$0.005/pageFirecrawl$0.005–0.03/page
Tech Stack Detector$0.02/URLWappalyzer$0.05–0.09/lookup
Email Validator Pro$0.005/emailZeroBounce$0.008–0.01/email
Broken Link Checker$0.005/linkScreaming Frog$259/year
Lighthouse Auditor$0.005/auditGTmetrix$12–99/month
Website Change Monitor$0.005/checkVisualping$50–200/month
Free
5 operations per run
No credit card required
$0.005
Per operation (flagships)
$0.004 for utility tools
{ }
JSON output
Structured data, ready for your pipeline

Four ways to access

Cloud, local, AI agent, or self-hosted. Your choice.

Apify Store

Run any tool directly on Apify. Free tier included.

apify.com/accurate_pouch

npm / CLI

Install TheCrawler locally. Zero cloud dependency.

npm install thecrawler

MCP Server

Add to Claude, Cursor, or any MCP-compatible AI.

npx thecrawler-mcp

REST API

Self-host the API server on your infrastructure.

npx thecrawler-api