Crawlee is a TypeScript library for Node.js that simplifies web scraping and browser automation. It helps you extract various file types like HTML, PDF, and images, making it ideal for feeding data to AI models. You can use it with Puppeteer, Playwright, Cheerio, and more, supporting both headful and headless browser modes with built-in proxy rotation.
What problem does it solve?
Build reliable web crawlers for data extraction and automation.
Who should use it?
Developers needing to build and manage complex web scraping tasks for data analysis or AI model training.
Crawlee covers your crawling and scraping end-to-end and helps you build reliable scrapers. Fast.
Your crawlers will appear human-like and fly under the radar of modern bot protections even with the default configuration. Crawlee gives you the tools to crawl the web for links, scrape data, and store it to disk or cloud while staying configurable to suit your project's needs.
