Compare commits
4 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2c2362b4d3 | ||
|
|
612ed3fef2 | ||
|
|
fb2a6d0d04 | ||
|
|
19d3d39115 |
2
.gitignore
vendored
2
.gitignore
vendored
@@ -182,3 +182,5 @@ docs/examples/.chainlit/*
|
|||||||
|
|
||||||
local/
|
local/
|
||||||
.files/
|
.files/
|
||||||
|
|
||||||
|
a.txt
|
||||||
@@ -1,5 +1,9 @@
|
|||||||
# Changelog
|
# Changelog
|
||||||
|
|
||||||
|
## [0.2.6] - 2024-06-22
|
||||||
|
### Fixed
|
||||||
|
- Fix issue #19: Update Dockerfile to ensure compatibility across multiple platforms.
|
||||||
|
|
||||||
## [0.2.5] - 2024-06-18
|
## [0.2.5] - 2024-06-18
|
||||||
### Added
|
### Added
|
||||||
- Added five important hooks to the crawler:
|
- Added five important hooks to the crawler:
|
||||||
|
|||||||
@@ -54,7 +54,12 @@ EXPOSE 80
|
|||||||
|
|
||||||
# Download models call cli "crawl4ai-download-models"
|
# Download models call cli "crawl4ai-download-models"
|
||||||
RUN crawl4ai-download-models
|
RUN crawl4ai-download-models
|
||||||
# RUN python crawl4ai/model_loader.py
|
|
||||||
|
# Instakk mkdocs
|
||||||
|
RUN pip install mkdocs mkdocs-terminal
|
||||||
|
|
||||||
|
# Call mkdocs to build the documentation
|
||||||
|
RUN mkdocs build
|
||||||
|
|
||||||
# Run uvicorn
|
# Run uvicorn
|
||||||
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "80", "--workers", "4"]
|
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "80", "--workers", "4"]
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
# Crawl4AI v0.2.5 🕷️🤖
|
# Crawl4AI v0.2.6 🕷️🤖
|
||||||
|
|
||||||
[](https://github.com/unclecode/crawl4ai/stargazers)
|
[](https://github.com/unclecode/crawl4ai/stargazers)
|
||||||
[](https://github.com/unclecode/crawl4ai/network/members)
|
[](https://github.com/unclecode/crawl4ai/network/members)
|
||||||
@@ -13,6 +13,8 @@ Crawl4AI simplifies web crawling and data extraction, making it accessible for l
|
|||||||
- Use as REST API: [](https://colab.research.google.com/drive/1zODYjhemJ5bUmYceWpVoBMVpd0ofzNBZ?usp=sharing)
|
- Use as REST API: [](https://colab.research.google.com/drive/1zODYjhemJ5bUmYceWpVoBMVpd0ofzNBZ?usp=sharing)
|
||||||
- Use as Python library: [](https://colab.research.google.com/drive/1wz8u30rvbq6Scodye9AGCw8Qg_Z8QGsk)
|
- Use as Python library: [](https://colab.research.google.com/drive/1wz8u30rvbq6Scodye9AGCw8Qg_Z8QGsk)
|
||||||
|
|
||||||
|
✨ visit our [Documentation Website](https://crawl4ai.com/mkdocs/)
|
||||||
|
|
||||||
## Features ✨
|
## Features ✨
|
||||||
|
|
||||||
- 🆓 Completely free and open-source
|
- 🆓 Completely free and open-source
|
||||||
@@ -47,7 +49,7 @@ crawler.warmup()
|
|||||||
result = crawler.run(url="https://www.nbcnews.com/business")
|
result = crawler.run(url="https://www.nbcnews.com/business")
|
||||||
|
|
||||||
# Print the extracted content
|
# Print the extracted content
|
||||||
print(result.extracted_content)
|
print(result.markdown)
|
||||||
```
|
```
|
||||||
|
|
||||||
### Extract Structured Data from Web Pages 📊
|
### Extract Structured Data from Web Pages 📊
|
||||||
@@ -98,7 +100,7 @@ print(result.extracted_content)
|
|||||||
|
|
||||||
## Documentation 📚
|
## Documentation 📚
|
||||||
|
|
||||||
For detailed documentation, including installation instructions, advanced features, and API reference, visit our [Documentation Website](https://craw4ai.com/mkdocs/).
|
For detailed documentation, including installation instructions, advanced features, and API reference, visit our [Documentation Website](https://crawl4ai.com/mkdocs/).
|
||||||
|
|
||||||
## Contributing 🤝
|
## Contributing 🤝
|
||||||
|
|
||||||
|
|||||||
@@ -1,5 +1,9 @@
|
|||||||
# Changelog
|
# Changelog
|
||||||
|
|
||||||
|
## [0.2.6] - 2024-06-22
|
||||||
|
### Fixed
|
||||||
|
- Fix issue #19: Update Dockerfile to ensure compatibility across multiple platforms.
|
||||||
|
|
||||||
## [0.2.5] - 2024-06-18
|
## [0.2.5] - 2024-06-18
|
||||||
### Added
|
### Added
|
||||||
- Added five important hooks to the crawler:
|
- Added five important hooks to the crawler:
|
||||||
|
|||||||
@@ -25,7 +25,7 @@
|
|||||||
<header class="bg-zinc-950 text-lime-500 py-4 flex">
|
<header class="bg-zinc-950 text-lime-500 py-4 flex">
|
||||||
|
|
||||||
<div class="mx-auto px-4">
|
<div class="mx-auto px-4">
|
||||||
<h1 class="text-2xl font-bold">🔥🕷️ Crawl4AI: Web Data for your Thoughts v0.2.5</h1>
|
<h1 class="text-2xl font-bold">🔥🕷️ Crawl4AI: Web Data for your Thoughts</h1>
|
||||||
</div>
|
</div>
|
||||||
<div class="mx-auto px-4 flex font-bold text-xl gap-2">
|
<div class="mx-auto px-4 flex font-bold text-xl gap-2">
|
||||||
<span>📊 Total Website Processed</span>
|
<span>📊 Total Website Processed</span>
|
||||||
|
|||||||
2
setup.py
2
setup.py
@@ -33,7 +33,7 @@ class CustomInstallCommand(install):
|
|||||||
|
|
||||||
setup(
|
setup(
|
||||||
name="Crawl4AI",
|
name="Crawl4AI",
|
||||||
version="0.2.5",
|
version="0.2.6",
|
||||||
description="🔥🕷️ Crawl4AI: Open-source LLM Friendly Web Crawler & Scrapper",
|
description="🔥🕷️ Crawl4AI: Open-source LLM Friendly Web Crawler & Scrapper",
|
||||||
long_description=open("README.md").read(),
|
long_description=open("README.md").read(),
|
||||||
long_description_content_type="text/markdown",
|
long_description_content_type="text/markdown",
|
||||||
|
|||||||
Reference in New Issue
Block a user