ayrisdev/crawl4ai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Refactor AsyncCrawlerStrategy to return AsyncCrawlResponse

Browse Source 
This commit refactors the AsyncCrawlerStrategy class in the async_crawler_strategy.py file to modify the return types of the crawl and crawl_many methods. Instead of returning strings, these methods now return instances of the AsyncCrawlResponse class from the pydantic module. The AsyncCrawlResponse class contains the crawled HTML, response headers, and status code. This change improves the clarity and consistency of the code.
This commit is contained in:
unclecode
2024-09-12 15:49:49 +08:00
parent eb131bebdf
commit 396f430022
3 changed files with 44 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
crawl4ai/models.py
View File

@@ -17,4 +17,6 @@ class CrawlResult(BaseModel):
extracted_content: Optional[str] = None
metadata: Optional[dict] = None
error_message: Optional[str] = None
session_id: Optional[str] = None
session_id: Optional[str] = None
responser_headers: Optional[dict] = None
status_code: Optional[int] = None