🔥🕷️ Crawl4AI: Open-source LLM Friendly Web Scrapper

Try It Now

🤔 Why building this?

In recent times, we've seen numerous startups emerging, riding the AI hype wave and charging for services that should rightfully be accessible to everyone. 🌍💸 One for example is to scrap and crawl a web page, and transform it o a form suitable for LLM. We don't think one should build a business out of this, but definilty should be opened source. So if you possess the skills to build such things and you have such philosphy you should join our "Robinhood" band and help set these products free. 🆓🤝

⚙️ Installation

To install and run Crawl4AI locally or on your own service, the best way is to use Docker. 🐳 Follow these steps:

  1. Clone the GitHub repository: 📥 git clone https://github.com/unclecode/crawl4ai.git
  2. Navigate to the project directory: 📂 cd crawl4ai
  3. Build the Docker image: 🛠️ docker build -t crawl4ai . On Mac, follow: 🍎 docker build --platform linux/amd64 -t crawl4ai .
  4. Run the Docker container: ▶️ docker run -p 8000:80 crawl4ai

For more detailed instructions and advanced configuration options, please refer to the 📚 GitHub repository.