All projects matching “fetch url”

Web content extraction: fetch any public URL and get clean readable text, title, meta description, OpenGraph data, headings, links and word count as JSON. Built for AI agents that need web page content without HTML noise.

Fetch one public HTTP(S) URL for research, extraction, or agent workflows. Returns the final URL, upstream status, content type, and decoded content. Private, local, metadata, and reserved destinations are blocked; redirects are revalidated and response size and duration are bounded.

Fetch a public https URL and extract HTML head metadata ($0.005 USDC via x402). https only — http, file, data, localhost, and private IPs are rejected. Caps: 1 MB body, ~10s timeout, max 5 redirects. Returns JSON { url, final_url, content_type, title?, description?, canonical?, lang?, open_graph, twitter, warnings }.

Fetch a public https URL that returns text and compute byte/char/line/word counts plus SHA-256 ($0.005 USDC via x402). https only — http, file, data, localhost, and private IPs are rejected. Caps: 1 MB body, ~10s timeout, max 5 redirects. Rejects image/audio/video/octet-stream. Returns JSON { url, final_url, content_type, byte_length, char_count, line_count, word_count, sha256, warnings }.

Headless scraper proxy for AI agents at $0.01 per accepted scrape. Fetch a public HTTPS URL and return clean LLM-ready Markdown. Agent-safety guarantee: output is validated as clean text before serving; binary or garbled content is rejected with no charge.

Headless scraper proxy for AI agents at $0.01 per accepted scrape. Fetch a public HTTPS URL and return clean LLM-ready Markdown. Agent-safety guarantee: output is validated as clean text before serving; binary or garbled content is rejected with no charge.

Extract a readable article from supplied HTML, including title, author, language, metadata, clean Markdown, links and tables. Uses Mozilla Readability with a fallback for short pages. No remote fetches or script execution; URLs resolve relative links.

Fetch a public https URL, compute SHA-256 of the response body, and report whether it changed vs a prior sha256/etag ($0.01 USDC via x402). Sticky: store sha256 and call again later. Optional previous_sha256, previous_etag, include_body_if_changed. https only — http, file, data, localhost, and private IPs rejected. Caps: 1 MB body, ~10s timeout, max 5 redirects. Returns JSON { url, final_url, cont

Ysaere Agent Page Extract — $0.02 USDC, no account needed. Input: {"url":"https://ysaere.com","max_chars"?:12000}. Fetches that public https page and returns clean readable text, title, meta description, headings, word_count, and a v2:sha512: content_hash in this response. Failed fetches refused before payment. Wallet only. No account price.

Give this API a link to any website's sitemap.xml file and it will fetch and parse it, returning every URL found inside. If the sitemap links to other sitemaps (called sitemap indexes), it will follow and extract those too. Each URL is returned with any extra information available, such as when the page was last updated and how often it changes. Useful for SEO audits, content inventories, broken l

Fetch a public https CSV URL and parse it into headers + rows ($0.005 USDC via x402). https only — http, file, data, localhost, and private IPs are rejected. Caps: 1 MB body, ~10s timeout, max 5 redirects, max 5000 data rows (truncated with warning). Optional has_header (default true). Returns JSON { url, final_url, content_type, headers, rows, row_count, warnings }.

Headless browser rendering for JavaScript-heavy pages and SPAs: full stealth Chromium render, then clean markdown plus title, meta description, and canonical URL. Use when a plain fetch returns an empty shell or a bot wall.

Scrape website page to Markdown. Fetch any public web page URL and get clean, LLM-ready text or Markdown: main content, code blocks, tables, links and headings, for RAG and agent context. Static HTML, Markdown or text; optional section ID. Default 20,000 output characters, max 100,000; input 2 MiB. No JavaScript rendering (see /extract/rendered), logins or paywalls. Empty pages, bad sections, brow

Pay $0.01 USDC to make an outbound HTTP request through the proxy. Supports all standard HTTP methods, custom headers (including auth), and JSON or raw bodies. Responses are returned as text or base64-encoded binary. Max response size 10 MB, timeout 30s. Each request performs one upstream fetch and returns the final URL plus response metadata.

Fetch a public http(s) URL and return Markdown. Pay-per-call via x402 USDC.

Fetches a public web page URL and returns its main article content as clean Markdown with title, byline and excerpt; strips navigation, ads and scripts. Useful for RAG ingestion and LLM context.

Fetches a public http(s) URL (GET, up to 5 redirects, 10s timeout, body never read) and grades its security headers A+ to F. Returns score, present and missing headers, per-header findings with copy-paste fix values, and cookie flag issues.

Independently fetch, hash, timestamp and archive a URL or API response; returns a signed certificate. Merkle-anchored on Base hourly. Neutral third-party evidence for agent commerce disputes.

Read any public web page as clean Markdown: main-content extraction (navigation, footers, scripts and hidden elements removed), headings, lists, tables, code blocks, links (absolute, deduplicated), images, plus title, description, canonical URL, language, author and publish date, and a content hash + fetch time as evidence. JSON and plain-text URLs are returned as-is. No JavaScript rendering. $0.0

Checks whether a URL/domain is safe to fetch, cite, or follow: flags phantom-squatted or typosquatted brand domains, unregistered or brand-new domains, and known malicious infrastructure (URLhaus). Returns an ALLOW/WARN/BLOCK verdict before your agent trusts the link.

Fetch a YouTube transcript from a video URL or 11-char id (video_id, also accepted as ref or url); returns a presigned download URL to the transcript

Verify a claim against 2-4 source URLs: cited passages, cross-source agreement, verdict: consistent | conflicting | insufficient_evidence. Evidence, not truth. Use when your agent needs to verify a claim against independent sources before acting on it. Give a claim plus 2-4 source URLs; each source is extracted (three-fetch cross-check, confidence-scored), the checkable figures are compared across

Submit one verifiable fact on the machine-payments beat for editorial review. Body: {"contributor_wallet": "0x... (Base, receives the bounty)", "category": "machine_payments_metrics|x402_endpoint_news|protocol_governance|pricing_change|other", "fact_statement": "one specific checkable claim, 20-600 chars", "source_url": "https://..."}. Agent-run review within ~48h fetches your source and checks it

Inspect citation overlap before counting corroboration. Supply 2-10 URLs, optional article HTML and citation chains. Returns duplicate URLs, shared links and supplied-source groups with locations. No fetching, truth or independence verdict. See /openapi.json for schemas and examples.

URL-to-Markdown web extraction and context compression for AI agents. Fetch a public web page and return clean, task-relevant, citation-verifiable LLM context with exact excerpts, locators, hashes, and estimated tokens saved. Use before an agent summarizes, answers questions from, or cites a long or noisy URL. Deterministic and model-free. Public HTML only; no PDFs, logins, or JavaScript-only page

Fetch a site's robots.txt and evaluate crawl permissions: given a URL (plus optional extra paths) and a user-agent, return whether each path is allowed or disallowed, the rule that matched, the user-agent group, the crawl-delay and any declared sitemaps. Implements the Robots Exclusion Protocol (RFC 9309) with longest-match-wins, Allow-over-Disallow tie-breaking and * / $ wildcards.

Get structured metadata for any public web page in one call: title, meta description, canonical URL, Open Graph and Twitter card tags, favicons, JSON-LD schema types, language and the heading outline. Use it to build link previews and social cards, classify or dedupe URLs, and run SEO and AI-readability audits - far cheaper than fetching and parsing the whole page.

Get structured metadata for any public web page in one call: title, meta description, canonical URL, Open Graph and Twitter card tags, favicons, JSON-LD schema types, language and the heading outline. Use it to build link previews and social cards, classify or dedupe URLs, and run SEO and AI-readability audits - far cheaper than fetching and parsing the whole page.

Search the web and read the top pages in ONE call: 10 results (title, url, snippet, date) plus the clean text of the best pages, with fetch time and sha256. French and Canada by default when the query is French; official-sources mode (governments, registries, courts, laws); freshness filter; site: restriction. No results = HTTP 404 = you are not charged. Input: query, optional read (0-5), official

Japan Stablecoin Payment Monitor: weekly change feed for stablecoin payment services in Japan — new launches, pilots, partnerships, fee changes, supported assets and chains, and closures. Each event is dated, categorized and tied to an official source URL. Pass changedSince=YYYY-MM-DD (your last run date) to fetch only new events; an empty changes list explicitly means no change. Without changedSi