scraping

🔥 The Web Data API for AI - Power AI agents with clean web data

🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!

Scrapy, a fast high-level web crawling & scraping framework for Python.

🕵️‍♂️ Collect a dossier on a person by username from 3000+ sites

Python scraper based on AI

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Custom Selenium Chromedriver | Zero-Config | Passes ALL bot mitigation systems (like Distil / Imperva/ Datadadome / CloudFlare IUAM)

Stealth headless browser for AI agents — bypass Cloudflare, bot detection, and anti-scraping. Drop-in Puppeteer/Playwright replacement.

A browser testing and web crawling library for PHP and Symfony

Web crawler and scraper for Rust

Linkedin Automation Tool: Describe your product. Define your target market. The AI finds the leads for you.

List of anti-detect and humanizing tools and browsers, including captcha solvers and sms-activation.

AgentQL is a suite of tools for connecting your AI to the web. Featuring a query language and Playwright integrations for interacting with elements and extracting data quickly, precisely, and at scale. Includes REST API, Python and JavaScript SDKs, browser debugger.

The agent that turns websites into APIs!

Jekyll-based static site for The Programming Historian

Python quick start guides to get the most out of Oxylabs' Web Scraper API free trial.

A framework for creating semi-automatic web content extractors

Library for Rapid (Web) Crawler and Scraper Development

SEO python scraper to extract data from major searchengine result pages. Extract data like url, title, snippet, richsnippet and the type from searchresults for given keywords. Detect Ads or make automated screenshots. You can also fetch text content of urls provided in searchresults or by your own. It's usefull for SEO and business related research tasks.