All projects matching “scrape”

Domain intel bundle: DNS records (A/AAAA/MX/NS/TXT/CAA/CNAME), RDAP registration (registrar, created/expires, status, nameservers) and TLS certificate (issuer, days left) in one call.

Use when you need to discover arXiv e-prints by keyword, title, author or field query: pass any arXiv search_query string (e.g. "transformer attention", "au:vaswani", "ti:\"attention is all you need\"", "cat:cs.LG AND all:diffusion"). Returns up to 25 papers with id, title, summary, authors (names + affiliation without emails), primary category, categories, published/updated, comment, journal ref,

Use when you have a likely PyPI project name (or a few tokens) and need to resolve it to live projects: tries the query as-is, PEP 503 normalization (lowercase, [-_.] → -), and hyphen/underscore/dot variants, then returns up to "limit" matching projects with summary, version, requires_python and license. This is name resolution, not full-text discovery — prefer exact package names when known. Auth

Use when you need structured, machine-readable facts rather than prose: give a Wikidata ID (Q95) or a name (resolved to the best-matching item) and get its label, description, aliases, Wikipedia link and up to ~50 curated properties with human-readable labels: instance of, country, location, coordinates, inception/dissolution, official website, headquarters, industry, employees, revenue, stock exc

Use when you need the official facts about a US-listed or SEC-registered company: resolves a ticker or CIK and returns the EDGAR profile (legal name, all tickers and exchanges, SIC code and industry, entity type, filer category such as Large Accelerated Filer, state/country of incorporation, fiscal year end, business and mailing address, company phone, website if filed, former names) plus its 10 m

Use when you need to know whether a website or mail server has a valid HTTPS certificate or when it expires: opens a live TLS connection (SNI) to the host and returns whether the chain is publicly trusted (with the exact error, e.g. CERT_HAS_EXPIRED or self-signed), whether the certificate matches the hostname, validity dates and days remaining, subject, issuer, SAN list, key type, TLS protocol, c

Use when you need the list of pages a site publishes (to crawl it, find recent posts, or audit SEO): give a site URL (the sitemap is discovered from robots.txt Sitemap: lines, then /sitemap.xml, /sitemap_index.xml, /wp-sitemap.xml) or a sitemap URL directly. Returns the page URLs with lastmod, changefreq and priority; for a sitemap index, returns its child sitemaps and (expand=true, default) the U

Fetch a public web page and return clean LLM-ready Markdown (Readability main-content extraction, nav/scripts stripped) plus title, byline, excerpt and OpenGraph metadata. {fullPage:true} keeps the whole body; {maxChars} caps output (default 100k). No JS rendering.

MicroScrape: structured field extraction via caller-supplied CSS selectors (via MCP)

Use when you need what viewers say about a YouTube video, e.g. sentiment, feedback mining, FAQ or product research: up to 100 top-level comments with text, like count, reply count, relative publish time and pinned / by-creator flags, plus the total comment count. Commenter names, handles and channel IDs are intentionally NOT returned. Videos with comments off and errors are not charged.

Fetch a public web page and return its title, description, clean Markdown (scripts, styles, nav chrome removed) and the list of links. No model call, deterministic.