unclecode 68e9144ce3 feat: Enhance crawling control and LLM extraction flexibility ...

- Add before_retrieve_html hook and delay_before_return_html option
- Implement flexible page_timeout for smart_wait function
- Support extra_args and custom headers in LLM extraction
- Allow arbitrary kwargs in AsyncWebCrawler initialization
- Improve perform_completion_with_backoff for custom API calls
- Update examples with new features and diverse LLM providers

2024-10-12 14:48:22 +08:00

17 KiB

Raw Blame History

View Raw