LibreCrawl
A web-based multi-tenant crawler for SEO analysis and website auditing.
🌐 Website: librecrawl.com
Demo no longer available cause people thought it was a prod environ, it isnt, it was a demo to get a taste before installing
API Documentation: https://librecrawl.com/api/docs/
LibreCrawl will always be free and open source. If it's replacing your $259/year Screaming Frog license, deepcrawl license or sitebulb license, buy me a coffee.
What it does
LibreCrawl crawls websites and gives you detailed information about pages, links, SEO elements, and performance. It's built as a web application using Python Flask with a modern web interface supporting multiple concurrent users.
Features
- 🚀 Multi-tenancy - Multiple users can crawl simultaneously with isolated sessions
- 🎨 Custom CSS styling - Personalize the UI with your own CSS themes
- 💾 Browser localStorage persistence - Settings saved per browser
- 🔄 JavaScript rendering for dynamic content (React, Vue, Angular, etc.)
- 📊 SEO analysis - Extract titles, meta descriptions, headings, etc.
- 🔗 Link analysis - Track internal and external links with detailed relationship mapping
- 📈 PageSpeed Insights integration - Analyze Core Web Vitals
- 💾 Multiple export formats - CSV, JSON, or XML
- 🔍 Issue detection - Automated SEO issue identification
- ⚡ Real-time crawling progress with live statistics
Getting started