UncleCode 825c78a048 refactor(dispatcher): migrate to modular dispatcher system with enhanced monitoring ...

Reorganize dispatcher functionality into separate components:
- Create dedicated dispatcher classes (MemoryAdaptive, Semaphore)
- Add RateLimiter for smart request throttling
- Implement CrawlerMonitor for real-time progress tracking
- Move dispatcher config from CrawlerRunConfig to separate classes

BREAKING CHANGE: Dispatcher configuration moved from CrawlerRunConfig to dedicated dispatcher classes. Users need to update their configuration approach for multi-URL crawling.

2025-01-11 21:10:27 +08:00

..

assets

- User agent

2024-06-08 17:59:42 +08:00

amazon_product_extraction_direct_url.py

Commit Message:

2024-12-29 20:05:18 +08:00

amazon_product_extraction_using_hooks.py

Commit Message:

2024-12-29 20:05:18 +08:00

amazon_product_extraction_using_use_javascript.py

Commit Message:

2024-12-29 20:05:18 +08:00

async_webcrawler_multiple_urls_example.py

Enhance AsyncWebCrawler with smart waiting and screenshot capabilities

2024-10-02 17:34:56 +08:00

browser_optimization_example.py

Enhance crawler capabilities and documentation

2024-12-25 21:34:31 +08:00

chainlit.md

Add research assistant example using Chainlit

2024-06-04 22:43:09 +08:00

crawlai_vs_firecrawl.py

chore: Update crawl4ai package with AsyncWebCrawler and JsonCssExtractionStrategy

2024-09-03 23:32:27 +08:00

dispatcher_example.py

refactor(dispatcher): migrate to modular dispatcher system with enhanced monitoring

2025-01-11 21:10:27 +08:00

docker_example.py

Enhance Docker support and improve installation process

2024-11-29 20:52:51 +08:00

extraction_strategies_example.py

Enhance crawler capabilities and documentation

2024-12-25 21:34:31 +08:00

full_page_screenshot_and_pdf_export.md

Enhance crawler capabilities and documentation

2024-12-25 21:34:31 +08:00

hello_world.py

refactor():

2025-01-02 17:53:30 +08:00

hooks_example.py

refactor(docs): reorganize documentation structure and update styles

2025-01-07 20:49:50 +08:00

language_support_example.py

Enhance AsyncWebCrawler with smart waiting and screenshot capabilities

2024-10-02 17:34:56 +08:00

llm_extraction_openai_pricing.py

Fix #340 example llm_extraction (#358)

2024-12-24 19:56:07 +08:00

quickstart_async.config.py

Commit Message:

2024-12-29 20:05:18 +08:00

quickstart_async.py

Commit Message:

2024-12-29 20:05:18 +08:00

quickstart_sync.py

Create series of quickstart files.

2024-09-04 15:33:24 +08:00

quickstart_v0.ipynb

docs(urls): update documentation URLs to new domain

2025-01-09 16:22:41 +08:00

quickstart.ipynb

Update documents, upload new version of quickstart.

2024-10-30 20:39:35 +08:00

research_assistant.py

Push async version last changes for merge to main branch

2024-09-24 20:52:08 +08:00

rest_call.py

- User agent

2024-06-08 17:59:42 +08:00

sample_ecommerce.html

Push async version last changes for merge to main branch

2024-09-24 20:52:08 +08:00

ssl_example.py

Commit Message:

2024-12-26 15:17:07 +08:00

storage_state_tutorial.md

Implement new async crawler features and stability updates

2024-12-10 17:55:29 +08:00

summarize_page.py

## [v0.2.74] - 2024-07-08

2024-07-08 16:33:25 +08:00

tutorial_dynamic_clicks.md

Add PDF & screenshot functionality, new tutorial

2024-12-10 20:10:39 +08:00

v0_4_24_walkthrough.py

refactor(build): simplify setup.py configuration

2025-01-01 15:52:01 +08:00

v0.3.74.overview.py

feat: add enhanced markdown generation example with citations and file output

2024-11-22 20:14:58 +08:00