fix(models): make model fields optional with default values

Make fields in MediaItem and Link models optional with default values to prevent validation errors when data is incomplete. Also expose BaseDispatcher in __init__ and fix markdown field handling in database manager. BREAKING CHANGE: MediaItem and Link model fields are now optional with default values which may affect existing code expecting required fields.
2025-01-15 22:58:14 +08:00
parent 20c027b79c
commit 9d694da939
6 changed files with 681 additions and 12 deletions
--- a/crawl4ai/init.py
+++ b/crawl4ai/init.py
@@ -23,6 +23,7 @@ from .async_dispatcher import (
    RateLimiter,
    CrawlerMonitor,
    DisplayMode,
+    BaseDispatcher
 )

 __all__ = [
@@ -43,6 +44,7 @@ __all__ = [
    "DefaultMarkdownGenerator",
    "PruningContentFilter",
    "BM25ContentFilter",
+    "BaseDispatcher",
    "MemoryAdaptiveDispatcher",
    "SemaphoreDispatcher",
    "RateLimiter",