refactor(pdf): improve PDF processor dependency handling

Make PyPDF2 an optional dependency and improve import handling in PDF processor.
Move imports inside methods to allow for lazy loading and better error handling.
Add new 'pdf' optional dependency group in pyproject.toml.
Clean up unused imports and remove deprecated files.

BREAKING CHANGE: PyPDF2 is now an optional dependency. Users need to install with 'pip install crawl4ai[pdf]' to use PDF processing features.
This commit is contained in:
UncleCode
2025-02-25 22:27:55 +08:00
parent 71ce01c9e1
commit 4bcd4cbda1
8 changed files with 67 additions and 388 deletions

View File

@@ -6,6 +6,7 @@ from crawl4ai import (
CacheMode,
DefaultMarkdownGenerator,
PruningContentFilter,
CrawlResult
)
@@ -20,10 +21,12 @@ async def main():
)
),
)
result = await crawler.arun(
url="https://www.helloworld.org", config=crawler_config
result : CrawlResult = await crawler.arun(
# url="https://www.helloworld.org", config=crawler_config
url="https://www.kidocode.com", config=crawler_config
)
print(result.markdown_v2.raw_markdown[:500])
# print(result.model_dump())
if __name__ == "__main__":