Compare commits
2 Commits
main-v0.2.
...
v0.2.72
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
b58af3349c | ||
|
|
940df4631f |
11
CHANGELOG.md
11
CHANGELOG.md
@@ -1,5 +1,16 @@
|
||||
# Changelog
|
||||
|
||||
## [v0.2.72] - 2024-06-30
|
||||
|
||||
This release brings exciting updates and improvements to our project! 🎉
|
||||
|
||||
* 📚 **Documentation Updates**: Our documentation has been revamped to reflect the latest changes and additions.
|
||||
* 🚀 **New Modes in setup.py**: We've added support for three new modes in setup.py: default, torch, and transformers. This enhances the project's flexibility and usability.
|
||||
* 🐳 **Docker File Updates**: The Docker file has been updated to ensure seamless compatibility with the new modes and improvements.
|
||||
* 🕷️ **Temporary Solution for Headless Crawling**: We've implemented a temporary solution to overcome issues with crawling websites in headless mode.
|
||||
|
||||
These changes aim to improve the overall user experience, provide more flexibility, and enhance the project's performance. We're thrilled to share these updates with you and look forward to continuing to evolve and improve our project!
|
||||
|
||||
## [0.2.71] - 2024-06-26
|
||||
|
||||
**Improved Error Handling and Performance** 🚧
|
||||
|
||||
@@ -9,49 +9,49 @@ There are three ways to use Crawl4AI:
|
||||
|
||||
Crawl4AI offers flexible installation options to suit various use cases. Choose the option that best fits your needs:
|
||||
|
||||
1. **Default Installation** (Basic functionality):
|
||||
```bash
|
||||
virtualenv venv
|
||||
source venv/bin/activate
|
||||
pip install "crawl4ai @ git+https://github.com/unclecode/crawl4ai.git"
|
||||
```
|
||||
Use this for basic web crawling and scraping tasks.
|
||||
- **Default Installation** (Basic functionality):
|
||||
```bash
|
||||
virtualenv venv
|
||||
source venv/bin/activate
|
||||
pip install "crawl4ai @ git+https://github.com/unclecode/crawl4ai.git"
|
||||
```
|
||||
Use this for basic web crawling and scraping tasks.
|
||||
|
||||
2. **Installation with PyTorch** (For advanced text clustering):
|
||||
```bash
|
||||
virtualenv venv
|
||||
source venv/bin/activate
|
||||
pip install "crawl4ai[torch] @ git+https://github.com/unclecode/crawl4ai.git"
|
||||
```
|
||||
Choose this if you need the CosineSimilarity cluster strategy.
|
||||
- **Installation with PyTorch** (For advanced text clustering):
|
||||
```bash
|
||||
virtualenv venv
|
||||
source venv/bin/activate
|
||||
pip install "crawl4ai[torch] @ git+https://github.com/unclecode/crawl4ai.git"
|
||||
```
|
||||
Choose this if you need the CosineSimilarity cluster strategy.
|
||||
|
||||
3. **Installation with Transformers** (For summarization and Hugging Face models):
|
||||
```bash
|
||||
virtualenv venv
|
||||
source venv/bin/activate
|
||||
pip install "crawl4ai[transformer] @ git+https://github.com/unclecode/crawl4ai.git"
|
||||
```
|
||||
Opt for this if you require text summarization or plan to use Hugging Face models.
|
||||
- **Installation with Transformers** (For summarization and Hugging Face models):
|
||||
```bash
|
||||
virtualenv venv
|
||||
source venv/bin/activate
|
||||
pip install "crawl4ai[transformer] @ git+https://github.com/unclecode/crawl4ai.git"
|
||||
```
|
||||
Opt for this if you require text summarization or plan to use Hugging Face models.
|
||||
|
||||
4. **Full Installation** (All features):
|
||||
```bash
|
||||
virtualenv venv
|
||||
source venv/bin/activate
|
||||
pip install "crawl4ai[all] @ git+https://github.com/unclecode/crawl4ai.git"
|
||||
```
|
||||
This installs all dependencies for full functionality.
|
||||
- **Full Installation** (All features):
|
||||
```bash
|
||||
virtualenv venv
|
||||
source venv/bin/activate
|
||||
pip install "crawl4ai[all] @ git+https://github.com/unclecode/crawl4ai.git"
|
||||
```
|
||||
This installs all dependencies for full functionality.
|
||||
|
||||
5. **Development Installation** (For contributors):
|
||||
```bash
|
||||
virtualenv venv
|
||||
source venv/bin/activate
|
||||
git clone https://github.com/unclecode/crawl4ai.git
|
||||
cd crawl4ai
|
||||
pip install -e ".[all]"
|
||||
```
|
||||
Use this if you plan to modify the source code.
|
||||
- **Development Installation** (For contributors):
|
||||
```bash
|
||||
virtualenv venv
|
||||
source venv/bin/activate
|
||||
git clone https://github.com/unclecode/crawl4ai.git
|
||||
cd crawl4ai
|
||||
pip install -e ".[all]"
|
||||
```
|
||||
Use this if you plan to modify the source code.
|
||||
|
||||
💡 After installation, it's recommended to run the following CLI command to load the required models. This is optional but will boost the performance and speed of the crawler. You need to do this only once:
|
||||
💡 After installation, if you have used "torch", "transformer" or "all", it's recommended to run the following CLI command to load the required models. This is optional but will boost the performance and speed of the crawler. You need to do this only once, this is only for when you install using []
|
||||
```bash
|
||||
crawl4ai-download-models
|
||||
```
|
||||
|
||||
Reference in New Issue
Block a user