The scalable web scraping and crawling library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.

Pack an entire repository into a single, AI-friendly file. Perfect for when you need to feed your codebase to Large Language Models (LLMs) or other AI tools like Claude, ChatGPT, DeepSeek, Perplexity, Gemini, Gemma, Llama, Grok, and more.

Get the main content of any page as Markdown.

Extract the Readable Content from an HTML Document

Scrape metadata from any URL using Open Graph, JSON-LD, HTML meta tags, and smart fallbacks.

AI-powered query language for web scraping and automation. It uses natural language selectors to find data on any page, including authenticated content. AgentQL queries are self-healing as UI changes and work across similar sites. Users can define structured data output, making AgentQL versatile for developers and data scientists.

A super simple site crawler and broken link checker

Download website to local directory (including all css, images, js, etc.)

Extract main article, main image and meta data from URL

The fast, flexible, and elegant library for parsing and manipulating HTML and XML.