ayrisdev/crawl4ai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

chore: Update installation instructions with support for different modes

Browse Source
This commit is contained in:
unclecode
2024-06-30 00:22:17 +08:00
parent 940df4631f
commit b58af3349c
1 changed files with 38 additions and 38 deletions
Download Patch File Download Diff File Expand all files Collapse all files

76
docs/md/installation.md
View File

@@ -9,49 +9,49 @@ There are three ways to use Crawl4AI:
Crawl4AI offers flexible installation options to suit various use cases. Choose the option that best fits your needs: Crawl4AI offers flexible installation options to suit various use cases. Choose the option that best fits your needs:
1. **Default Installation** (Basic functionality): - **Default Installation** (Basic functionality):
```bash ```bash
virtualenv venv virtualenv venv
source venv/bin/activate source venv/bin/activate
pip install "crawl4ai @ git+https://github.com/unclecode/crawl4ai.git" pip install "crawl4ai @ git+https://github.com/unclecode/crawl4ai.git"
``` ```
Use this for basic web crawling and scraping tasks. Use this for basic web crawling and scraping tasks.
2. **Installation with PyTorch** (For advanced text clustering): - **Installation with PyTorch** (For advanced text clustering):
```bash ```bash
virtualenv venv virtualenv venv
source venv/bin/activate source venv/bin/activate
pip install "crawl4ai[torch] @ git+https://github.com/unclecode/crawl4ai.git" pip install "crawl4ai[torch] @ git+https://github.com/unclecode/crawl4ai.git"
``` ```
Choose this if you need the CosineSimilarity cluster strategy. Choose this if you need the CosineSimilarity cluster strategy.
3. **Installation with Transformers** (For summarization and Hugging Face models): - **Installation with Transformers** (For summarization and Hugging Face models):
```bash ```bash
virtualenv venv virtualenv venv
source venv/bin/activate source venv/bin/activate
pip install "crawl4ai[transformer] @ git+https://github.com/unclecode/crawl4ai.git" pip install "crawl4ai[transformer] @ git+https://github.com/unclecode/crawl4ai.git"
``` ```
Opt for this if you require text summarization or plan to use Hugging Face models. Opt for this if you require text summarization or plan to use Hugging Face models.
4. **Full Installation** (All features): - **Full Installation** (All features):
```bash ```bash
virtualenv venv virtualenv venv
source venv/bin/activate source venv/bin/activate
pip install "crawl4ai[all] @ git+https://github.com/unclecode/crawl4ai.git" pip install "crawl4ai[all] @ git+https://github.com/unclecode/crawl4ai.git"
``` ```
This installs all dependencies for full functionality. This installs all dependencies for full functionality.
5. **Development Installation** (For contributors): - **Development Installation** (For contributors):
```bash ```bash
virtualenv venv virtualenv venv
source venv/bin/activate source venv/bin/activate
git clone https://github.com/unclecode/crawl4ai.git git clone https://github.com/unclecode/crawl4ai.git
cd crawl4ai cd crawl4ai
pip install -e ".[all]" pip install -e ".[all]"
``` ```
Use this if you plan to modify the source code. Use this if you plan to modify the source code.
💡 After installation, it's recommended to run the following CLI command to load the required models. This is optional but will boost the performance and speed of the crawler. You need to do this only once: 💡 After installation, if you have used "torch", "transformer" or "all", it's recommended to run the following CLI command to load the required models. This is optional but will boost the performance and speed of the crawler. You need to do this only once, this is only for when you install using []
```bash ```bash
crawl4ai-download-models crawl4ai-download-models
``` ```