refactor(deep-crawling): reorganize deep crawling strategies and add new implementations
Split deep crawling code into separate strategy files for better organization and maintainability. Added new BFF (Best First) and DFS crawling strategies. Introduced base strategy class and common types. BREAKING CHANGE: Deep crawling implementation has been split into multiple files. Import paths for deep crawling strategies have changed.
This commit is contained in:
14
crawl4ai/types.py
Normal file
14
crawl4ai/types.py
Normal file
@@ -0,0 +1,14 @@
|
||||
from typing import TYPE_CHECKING, Union
|
||||
|
||||
AsyncWebCrawler = Union['AsyncWebCrawlerType'] # Note the string literal
|
||||
CrawlerRunConfig = Union['CrawlerRunConfigType']
|
||||
CrawlResult = Union['CrawlResultType']
|
||||
RunManyReturn = Union['RunManyReturnType']
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from . import (
|
||||
AsyncWebCrawler as AsyncWebCrawlerType,
|
||||
CrawlerRunConfig as CrawlerRunConfigType,
|
||||
CrawlResult as CrawlResultType,
|
||||
RunManyReturn as RunManyReturnType,
|
||||
)
|
||||
Reference in New Issue
Block a user