Crawl Readiness

Check what 50+ AI crawlers can read on a website and whether a page permits an AI use

Official docs outline

Extra Credit builds an outline of Crawl Readiness's docs the first time your work uses it. Until then, start at the official documentation.

Cloudflare Trace Get IP Address, Timestamp, User Agent, Country Code, IATA, HTTP Version, TLS/SSL Version & More
Codeberg Git hosting, repos and issues via Gitea API
Codex Online Compiler for Various Languages
Contentful Images Used to retrieve and apply transformations to images
CORS Proxy Get around the dreaded CORS error by using this proxy as a middle man
CountAPI Free and simple counting service. You can use it to track page hits and specific events
crawlbrulee EU-native web scraping for AI agents & developers: turn any page into markdown, screenshots and more
CurlShot Screenshots, PDFs and scrolling videos of any URL, HTML or Markdown; free tier
Diagrams.so Generate editable draw.io architecture diagrams from text or infrastructure code
DigitalOcean Status Status of all DigitalOcean services
DigMyName Domain availability and registrar pricing across 52 TLDs
Domain Info by LifeStep DNS records, SPF/DMARC detection and TLS certificate expiry