docs: update README for version 0.3.743 with new features, enhancements, and contributor acknowledgments

README.md

@@ -11,20 +11,15 @@
 
 Crawl4AI simplifies asynchronous web crawling and data extraction, making it accessible for large language models (LLMs) and AI applications. 🆓🌐
 
-## New in 0.3.74 ✨
+## New in 0.3.743 ✨  
 
-- 🚀 **Blazing Fast Scraping**: Significantly improved scraping speed.  
+🚀 **Improved ManagedBrowser Configuration**: Dynamic host and port support for more flexible browser management.  
-- 📥 **Download Manager**: Integrated file crawling, downloading, and tracking within `CrawlResult`.  
+📝 **Enhanced Markdown Generation**: New generator class for better formatting and customization.  
-- 📝 **Markdown Strategy**: Flexible system for custom markdown generation and formats.  
+⚡ **Fast HTML Formatting**: Significantly optimized HTML formatting in the web crawler.  
-- 🔗 **LLM-Friendly Citations**: Auto-converts links to numbered citations with reference lists.  
+🛠️ **Utility & Sanitization Upgrades**: Improved sanitization and expanded utility functions for streamlined workflows.  
-- 🔎 **Markdown Filter**: BM25-based content extraction for cleaner, relevant markdown.  
+👥 **Acknowledgments**: Added contributor details and pull request acknowledgments for better transparency.  
-- 🖼️ **Image Extraction**: Supports `srcset`, `picture`, and responsive image formats.  
+📖 **Documentation Updates**: Clearer usage scenarios and updated guidance for better user onboarding.  
-- 🗂️ **Local/Raw HTML**: Crawl `file://` paths and raw HTML (`raw:`) directly.  
+🧪 **Test Adjustments**: Refined tests to align with recent class name changes.  
-- 🤖 **Browser Control**: Custom browser setups with stealth integration to bypass bots.  
-- ☁️ **API & Cache Boost**: CORS, static serving, and enhanced filesystem-based caching.  
-- 🐳 **API Gateway**: Run as an API service with secure token authentication.  
-- 🛠️ **Database Upgrades**: Optimized for larger content sets with faster caching.  
-- 🐛 **Bug Fixes**: Resolved browser context issues, memory leaks, and improved error handling.
 
 
 ## Try it Now!
@@ -35,31 +30,85 @@ Crawl4AI simplifies asynchronous web crawling and data extraction, making it acc
 
 ## Features ✨
 
-- 🆓 Completely free and open-source
+<details open>
-- 🚀 Blazing fast performance, outperforming many paid services
+<summary>🚀 <strong>Performance & Scalability</strong></summary>
-- 🤖 LLM-friendly output formats (JSON, cleaned HTML, markdown)
+
-- 🌐 Multi-browser support (Chromium, Firefox, WebKit)
+- ⚡ **Blazing Fast Scraping**: Outperforms many paid services with cutting-edge optimization.
-- 🌍 Supports crawling multiple URLs simultaneously
+- 🔄 **Asynchronous Architecture**: Enhanced performance for complex multi-page crawling.
-- 🎨 Extracts and returns all media tags (Images, Audio, and Video)
+- ⚡ **Dynamic HTML Formatting**: New, fast HTML formatting for streamlined workflows.
-- 🔗 Extracts all external and internal links
+- 🗂️ **Large Dataset Optimization**: Improved caching for handling massive content sets.
-- 📚 Extracts metadata from the page
+
-- 🔄 Custom hooks for authentication, headers, and page modifications
+</details>
-- 🕵️ User-agent customization
+
-- 🖼️ Takes screenshots of pages with enhanced error handling
+<details>
-- 📜 Executes multiple custom JavaScripts before crawling
+<summary>🔎 <strong>Extraction Capabilities</strong></summary>
-- 📊 Generates structured output without LLM using JsonCssExtractionStrategy
+
-- 📚 Various chunking strategies: topic-based, regex, sentence, and more
+- 🖼️ **Comprehensive Media Support**: Extracts images, audio, video, and responsive image formats like `srcset` and `picture`.
-- 🧠 Advanced extraction strategies: cosine clustering, LLM, and more
+- 📚 **Advanced Content Chunking**: Topic-based, regex, sentence-level, and cosine clustering strategies.
-- 🎯 CSS selector support for precise data extraction
+- 🎯 **Precise Data Extraction**: Supports CSS selectors and keyword-based refinements.
-- 📝 Passes instructions/keywords to refine extraction
+- 🔗 **All-Inclusive Link Crawling**: Extracts internal and external links.
-- 🔒 Proxy support with authentication for enhanced access
+- 📝 **Markdown Generation**: Enhanced markdown generator class for custom, clean, LLM-friendly outputs.
-- 🔄 Session management for complex multi-page crawling
+- 🏷️ **Metadata Extraction**: Fetches metadata directly from pages.
-- 🌐 Asynchronous architecture for improved performance
+
-- 🖼️ Improved image processing with lazy-loading detection
+</details>
-- 🕰️ Enhanced handling of delayed content loading
+
-- 🔑 Custom headers support for LLM interactions
+<details>
-- 🖼️ iframe content extraction for comprehensive analysis
+<summary>🌐 <strong>Browser Integration</strong></summary>
-- ⏱️ Flexible timeout and delayed content retrieval options
+
+- 🌍 **Multi-Browser Support**: Works with Chromium, Firefox, and WebKit.
+- 🖥️ **ManagedBrowser with Dynamic Config**: Flexible host/port control for tailored setups.
+- ⚙️ **Custom Browser Hooks**: Authentication, headers, and page modifications.
+- 🕶️ **Stealth Mode**: Bypasses bot detection with advanced techniques.
+- 📸 **Screenshots & JavaScript Execution**: Takes screenshots and executes custom JavaScript before crawling.
+
+</details>
+
+<details>
+<summary>📁 <strong>Input/Output Flexibility</strong></summary>
+
+- 📂 **Local & Raw HTML Crawling**: Directly processes `file://` paths and raw HTML.
+- 🌐 **Custom Headers for LLM**: Tailored headers for enhanced AI interactions.
+- 🛠️ **Structured Output Options**: Supports JSON, cleaned HTML, and markdown outputs.
+
+</details>
+
+<details>
+<summary>🔧 <strong>Utility & Debugging</strong></summary>
+
+- 🛡️ **Error Handling**: Robust error management for seamless execution.
+- 🔐 **Session Management**: Handles complex, multi-page interactions.
+- 🧹 **Utility Functions**: Enhanced sanitization and flexible extraction helpers.
+- 🕰️ **Delayed Content Loading**: Improved handling of lazy-loading and dynamic content.
+
+</details>
+
+<details>
+<summary>🔐 <strong>Security & Accessibility</strong></summary>
+
+- 🕵️ **Proxy Support**: Enables authenticated access for restricted pages.
+- 🚪 **API Gateway**: Deploy as an API service with secure token authentication.
+- 🌐 **CORS & Static Serving**: Enhanced support for filesystem-based caching and cross-origin requests.
+
+</details>
+
+<details>
+<summary>🌟 <strong>Community & Documentation</strong></summary>
+
+- 🙌 **Contributor Acknowledgments**: Recognition for pull requests and contributions.
+- 📖 **Clear Documentation**: Simplified and updated for better onboarding and usage.
+
+</details>
+
+<details>
+<summary>🎯 <strong>Cutting-Edge Features</strong></summary>
+
+- 🛠️ **BM25-Based Markdown Filtering**: Extracts cleaner, context-relevant markdown.
+- 📚 **LLM-Friendly Citations**: Auto-converts links to numbered citations with reference lists.
+- 📡 **IFrame Content Extraction**: Comprehensive analysis for embedded content.
+- 🕰️ **Flexible Content Retrieval**: Combines timing-based strategies for reliable extractions.
+
+</details>
+
 
 ## Installation 🛠️