ayrisdev/crawl4ai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
203 Commits 150 Branches 42 Tags
886622cb1effe57f20de9bace68831ac1290e059
Commit Graph

203 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Unclecode
886622cb1e Merge branch 'main' of https://github.com/unclecode/crawl4ai 2024-06-29 16:23:44 +00:00
unclecode
b58af3349c chore: Update installation instructions with support for different modes v0.2.72 2024-06-30 00:22:17 +08:00
unclecode
940df4631f Update ChangeLog 2024-06-30 00:18:40 +08:00
unclecode
685706e0aa Update version, and change log 2024-06-30 00:17:43 +08:00
unclecode
7b0979e134 Update Redme and Docker file 2024-06-30 00:15:43 +08:00
unclecode
61ae2de841 1/Update setup.py to support following modes: 
- default (most frequent mode)
- torch
- transformers
- all
2/ Update Docker file
3/ Update documentation as well.
 2024-06-30 00:15:29 +08:00
unclecode
5b28eed2c0 Add a temporary solution for when we can't crawl websites in headless mode. 2024-06-29 23:25:50 +08:00
unclecode
f8a11779fe Update change log 2024-06-26 16:48:36 +08:00
Unclecode
13dc254438 Merge branch 'main' of https://github.com/unclecode/crawl4ai 2024-06-26 07:35:06 +00:00
unclecode
d11a83c232 ## [0.2.71] 2024-06-26 
• Refactored `crawler_strategy.py` to handle exceptions and improve error messages
• Improved `get_content_of_website_optimized` function in `utils.py` for better performance
• Updated `utils.py` with latest changes
• Migrated to `ChromeDriverManager` for resolving Chrome driver download issues
v0.2.71 		2024-06-26 15:34:15 +08:00
unclecode
3255c7a3fa Update CHANGELOG.md with recent commits 2024-06-26 15:20:34 +08:00
unclecode
4756d0a532 Refactor crawler_strategy.py to handle exceptions and improve error messages 2024-06-26 15:04:33 +08:00
unclecode
7ba2142363 chore: Refactor get_content_of_website_optimized function in utils.py 2024-06-26 14:43:09 +08:00
Unclecode
096929153f Merge branch 'main' of https://github.com/unclecode/crawl4ai 2024-06-26 05:45:25 +00:00
unclecode
96d1eb0d0d Some updated ins utils.py 2024-06-26 13:03:03 +08:00
unclecode
144cfa0eda Switch to ChromeDriverManager due some issues with download the chrome driver 2024-06-26 13:00:17 +08:00
unclecode
a0dff192ae Update README for speed example 2024-06-24 23:06:12 +08:00
unclecode
1fffeeedd2 Update Readme: Showcase the speed 2024-06-24 23:02:08 +08:00
unclecode
f51b078042 Update reame example. 2024-06-24 22:54:29 +08:00
unclecode
b6023a51fb Add star chart 2024-06-24 22:47:46 +08:00
Unclecode
7e95c38acb Merge branch 'main' of https://github.com/unclecode/crawl4ai 2024-06-24 14:40:48 +00:00
unclecode
78cfad8b2f chore: Update version to 0.2.7 and improve extraction function speed v0.2.7 2024-06-24 22:39:56 +08:00
Unclecode
c697bf23e4 Merge branch 'main' of https://github.com/unclecode/crawl4ai 2024-06-22 16:37:27 +00:00
Unclecode
b951d34ed0 chore: Update fetch URL to use HTTPS 2024-06-22 16:37:21 +00:00
unclecode
68b3dff74a Update CSS 2024-06-23 00:36:03 +08:00
unclecode
bfc4abd6e8 Update documents 2024-06-22 20:57:03 +08:00
Unclecode
c8a10dc455 Merge branch 'main' of https://github.com/unclecode/crawl4ai 2024-06-22 12:54:41 +00:00
unclecode
8c77a760fc Fixed: 
- Redirect "/" to mkdocs
 2024-06-22 20:54:32 +08:00
Unclecode
9e0ded8da0 Merge branch 'main' of https://github.com/unclecode/crawl4ai 2024-06-22 12:41:52 +00:00
unclecode
b9bf8ac9d7 Fix mounting the "/" to mkdocs site folder 2024-06-22 20:41:39 +08:00
Unclecode
48c27899b7 Merge branch 'main' of https://github.com/unclecode/crawl4ai 2024-06-22 12:38:14 +00:00
unclecode
d6182bedd7 chore: 
- Add demo page to the new mkdocs
- Set website home page to mkdocs
 2024-06-22 20:36:01 +08:00
unclecode
2217904876 Update .gitignore 2024-06-22 18:12:12 +08:00
unclecode
2c2362b4d3 issue 19 is resolved 
- Update Dockerfile to install mkdocs and build documentation
v0.2.6 		2024-06-22 17:18:00 +08:00
unclecode
612ed3fef2 chore: Update print statement to use markdown format 2024-06-21 19:10:13 +08:00
unclecode
fb2a6d0d04 chore: Update documentation link in README.md 2024-06-21 18:05:18 +08:00
unclecode
19d3d39115 Update Marge the DOCS branch 2024-06-21 18:04:13 +08:00
Unclecode
3c32b0abed Merge branch 'main' of https://github.com/unclecode/crawl4ai 2024-06-21 09:58:17 +00:00
unclecode
c1413e6916 chore: Update documentation link in README.md 2024-06-21 17:57:47 +08:00
unclecode
e7705e661a ADD MKDocs 2024-06-21 17:56:54 +08:00
unclecode
21b110bfd7 Update LLMExtractionStrategy to disable chunking if specified, Add example of summarization for a web page. 2024-06-19 19:03:35 +08:00
unclecode
1fcb573909 chore: Update table of contents in README.md 2024-06-19 18:53:22 +08:00
Unclecode
a215ec08d6 Merge branch 'main' of https://github.com/unclecode/crawl4ai 2024-06-19 10:51:31 +00:00
unclecode
0f6c5f5453 chore: Update configuration values, create new example, and update Dockerfile and README 2024-06-19 18:50:58 +08:00
unclecode
350ca1511b chore: Update configuration values, create new example, and update Dockerfile and README 2024-06-19 18:48:20 +08:00
unclecode
539263a8ba chore: Update configuration values for chunk token threshold, overlap rate, and minimum word threshold. Create a new example for LLMExtraction Strategy, update Dockerfile, and README 2024-06-19 18:32:20 +08:00
unclecode
3f0e265baf Merge branch 'format-inline-tags' 2024-06-19 00:48:38 +08:00
unclecode
21e2538e57 Update quickstart.py 2024-06-19 00:37:53 +08:00
Unclecode
5d3fef45f7 Merge branch 'main' of https://github.com/unclecode/crawl4ai 2024-06-18 12:02:29 +00:00
unclecode
480902bd66 Update README 2024-06-18 20:02:21 +08:00