ayrisdev/crawl4ai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
497 Commits 150 Branches 42 Tags
7c0fa269a67e383dd7aa85bbb79547fd94b4ca47
Commit Graph

2 Commits

Author SHA1 Message Date
UncleCode
e130fd8db9 Implement new async crawler features and stability updates 
- Introduced new async crawl strategy with session management.
  - Added BrowserManager for improved browser management.
  - Enhanced documentation, focusing on storage state and usage examples.
  - Improved error handling and logging for sessions.
  - Added JavaScript snippets for customizing navigator properties.
 2024-12-10 17:55:29 +08:00
UncleCode
6ec4cb33ca Enhance Markdown generation and external content control 
- Integrate customized html2text library for flexible Markdown output
- Add options to exclude external links and images
- Improve content scraping efficiency and error handling
- Update AsyncPlaywrightCrawlerStrategy for faster closing
- Enhance CosineStrategy with generic embedding model loading
 2024-10-20 18:56:58 +08:00