UncleCode 768aa06ceb feat(crawler): Enhance stealth and flexibility, improve error handling ...

- Implement playwright_stealth for better bot detection avoidance
- Add user simulation and navigator override options
- Improve iframe processing and browser selection
- Enhance error reporting and debugging capabilities
- Optimize image processing and parallel crawling
- Add new example for user simulation feature
- Added support for including links in Markdown content, by definin g a new flag `include_links_on_markdown` in `crawl` method.

2024-10-17 21:37:48 +08:00

..

examples

feat(crawler): Enhance stealth and flexibility, improve error handling

2024-10-17 21:37:48 +08:00

md

Update documents and README

2024-09-25 16:52:11 +08:00

md _sync

Push async version last changes for merge to main branch

2024-09-24 20:52:08 +08:00

.DS_Store

Push async version last changes for merge to main branch

2024-09-24 20:52:08 +08:00

chunking_strategies.json

chore: Update NlpSentenceChunking constructor parameters to None

2024-05-17 17:00:43 +08:00

extraction_strategies.json

Update:

2024-05-16 17:31:44 +08:00