All projects matching “unblock”

Unblocker for AI agents: send {url} and get the page back as clean Markdown, with title, description, language, final URL and HTTP status. Gets through Cloudflare and other common bot protection and can render JavaScript-heavy pages. Set format to html or text for those instead. A page that can't be fetched (blocked, error status or empty) is not charged. No API key, proxy or account: pay per call

Retrieve a public HTTPS URL from the US, Germany, Japan, Brazil, or India. Use HTTP mode for lightweight retrieval, or browser mode to render JavaScript pages. For pages with bot protection, set max_tier to L2 for managed unblocking at $0.05 USDC; L0/L1 costs $0.02 USDC. Returns the fetched body and an Ed25519-signed receipt for provenance.

Cheap robots.txt preflight check before you call the Minifetch /extract and /run/seo-page-audit endpoints: returns whether Minifetch may fetch a target URL (allow/deny) plus its crawl-delay. If blocked and the site is not yours try our external proxy routes. If blocked and the site is yours or to set a custom crawl-delay read https://minifetch.com/tutorials/unblock-minifetch Checks the URL as give