All projects matching “fetch url”

Fetch any public URL and return its content as base64 with the content type and a ready-to-use data URI. Embed images, PDFs and files directly in LLM and API calls without downloading them yourself. Clean JSON.

Convert HTML to tidy markdown ($0.005 USDC via x402). Pass html (string) and/or url (public https to fetch). At least one required; if both, prefers html and warns. URL fetches use the same allowlist as WELL-002 (https only — http/file/data/localhost/private IPs rejected; 1 MB body, ~10s timeout, max 5 redirects). Returns JSON { source, url?, final_url?, content_type?, markdown, warnings }.

Webpage text extraction: extract clean text from any webpage. Extract the clean readable text from a public webpage with title, headings, links and SHA-256; add format=markdown for Markdown or render=true to render JavaScript pages. Web read / fetch URL / scrape URL to markdown — public webpage to clean text or Markdown with headings, links and SHA-256, for RAG and research. Batch up to 10 URLs; o

Check a URL before fetching it: is it alive, final URL after redirects, HTTP status, content type, HTML title, and whether robots.txt allows your user-agent. Link checker and fetch preflight for agents.

Retrieve a public HTTPS URL from the US, Germany, Japan, Brazil, or India. Use HTTP mode for lightweight retrieval, or browser mode to render JavaScript pages. For pages with bot protection, set max_tier to L2 for managed unblocking at $0.05 USDC; L0/L1 costs $0.02 USDC. Returns the fetched body and an Ed25519-signed receipt for provenance.

Fetch a URL and extract structured metadata from its HTML: title, description, canonical URL, language, favicon URL, RSS/Atom feed links, Open Graph tags (og:title, og:image, og:type, og:description, og:url), Twitter Card tags, and other meta tags (author, keywords, robots). Use this to get Open Graph data, preview cards, or site information without parsing HTML yourself. Send {"url": "https://...

Public sources for agent research: one You.com search plus fetch of up to three public HTTPS pages. Returns structured JSON with page text, metadata, source URLs, fetch timestamps and content hashes for source checking and company research. 0.01 USDC on Base per call via x402 v2; no signup. No browser rendering or synthesized answer. Generate a private request_id locally and reuse it on pending re

Fetch a live https URL and score whether the deploy is actually there. POST {"url":"https://your.site","min_score":80} ($0.02). Default x402-seller profile: HTTPS, HSTS, llms.txt, catalog, OpenAPI, MCP, health. Optional claims[] or profile:"web". Returns 0–100, evidence, do_this_first, ci_failed. Humans: GET /v1/tools/claim-check/report?host=your.site. Badge goes stale after 7 days.

Fetch one URL from a residential IP and get the raw response as JSON: status, final URL, headers and body (text or base64), up to 10 MB. GET, HEAD or POST, your own headers, an optional country. No JavaScript rendering. You pay for the megabytes delivered; a fetch that fails is not charged. Pay per call in stablecoins.

Web page scrape: one URL fetched and rendered by Firecrawl (headless browser), returned as markdown, html, rawHtml and/or links. Not charged when the page cannot be fetched.

Notarize any public web page or API response: fetches the URL server-side, archives the exact bytes, and returns a tamper-evident ed25519 attestation recording the final URL, UTC timestamp, HTTP status, content type, byte length and sha256 hash. Establishes provenance and a chain of custody for what a source said at a point in time, so the signed record can serve as the evidence layer in a later d

Viewport or full page, optional element selector, dark mode and cookie banner removal. Returns base64 image plus a short lived fetch URL, byte size, load time and the final HTTP status of the page.

Answer a focused question from 1-3 caller-supplied text sources, with exact excerpts, source hashes, offsets, model-assessed conflicts and evidence gaps. No URL fetching or source authentication. Quote presence is checked; semantic truth/support is not independently proved. 0.01 USDC per completed brief, including insufficient/conflicting evidence; malformed output and execution failures are not s

Extract text from a digital PDF into page-numbered JSON with SHA-256 hashes. No account or API key. 0.005 USDC on Base per successful document via x402. Maximum 2 MiB and 25 pages. No OCR, summaries, URL fetching, or verification of document claims.

Fetch a URL and extract clean main-content text with title, description, word count, and char count; boilerplate (nav/footer/sidebar/ads) is stripped.

Fetch a URL and return clean, token-efficient markdown

Fetch a URL and return its text content

Fetch any URL and, without paying, decode its x402 402 Payment Required challenge: price (atomic and formatted for known stablecoins), asset, network, payTo, resource, description, timeout and any input schema. Flags MPP and Bazaar discovery extensions. Returns free:true when no payment is needed. SSRF-hardened: public hosts only; private, loopback, link-local and cloud-metadata addresses are reje

How many <audio> elements does the page have? Fetches the URL and counts audio tags (plus, where relevant, how many have alt/src/href/type attributes), and the page's total element count for scale. Structural audits and template comparisons in one number. $0.01 per page.

Fetches a URL and returns clean article text/markdown plus metadata. Deterministic, static HTML only, no JS rendering.

Fetch any public URL and return a clean, readable markdown or JSON extract for AI agents. Pass the target as ?url=<encoded>. Ideal when you need to read webpages, docs, or articles without a browser. Blocks script/style tags, normalizes links and images, strips HTML.

Returns the page's readable text plus an Ed25519-signed record (final URL after redirects, fetch time, status, sha256 of the body and of the text) that verifies offline. Charged only for 2xx text pages; refused URLs cost nothing.

Cheap robots.txt preflight check before you call the Minifetch /extract and /run/seo-page-audit endpoints: returns whether Minifetch may fetch a target URL (allow/deny) plus its crawl-delay. If blocked and the site is not yours try our external proxy routes. If blocked and the site is yours or to set a custom crawl-delay read https://minifetch.com/tutorials/unblock-minifetch Checks the URL as give

Fetch a public URL with SSRF protection, clean extraction, and semantic prompt-injection analysis.

Fetch a public URL and return clean LLM-ready markdown from static HTML.

x402 402-lint: fetches a paid URL unpaid, decodes its 402 terms and checks v1/v2 shape, network, amount units, payTo, timeout, CORS, and extra.name/version against the token's on-chain EIP-712 domain.

Fetch a public URL as markdown from the public HTML (Readability)

Fetch any public URL and report whether its HTTP 402 challenge is a schema-valid x402 v2 PaymentRequired, payable in Base USDC, and catalog-ready (bazaar discovery declaration + facilitator support).

Fetch a public URL and return its main content as markdown.

Web content extraction: fetch any public URL and get clean readable text, title, meta description, OpenGraph data, headings, links and word count as JSON. Built for AI agents that need web page content without HTML noise.