ayrisdev/crawl4ai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
db6ad7a79d9c5b5b5fdb2bfa23937ae7f0c50f10
crawl4ai/crawl4ai
History
Nasrin 004d514f33 Merge pull request #1265 from unclecode/feature/nasrin-cli-deep-crawl 
Feature/CLI - deep-crawl: Add --deep-crawl CLI option with BFS/DFS/Best-First strategies and fix serialization error. ref #874
2025-07-23 09:40:33 +02:00
..
components
feat(monitor): add real-time crawler monitoring system with memory management
2025-03-12 19:05:24 +08:00
crawlers
Update all documentation to import extraction strategies directly from crawl4ai.
2025-06-10 18:08:27 +08:00
deep_crawling
Merge branch '2025-JUN-1' into next-MAY
2025-07-09 09:41:03 +02:00
html2text
2025 feb alpha 1 (#685)
2025-02-19 14:13:17 +08:00
js_snippet
fix: Add a nominal wait time for remove overlay elements since it's already controllable through delay_before_return_html
2025-04-14 12:39:05 +05:30
legacy
refactor(core): reorganize project structure and remove legacy code
2025-01-30 19:35:06 +08:00
processors/pdf
fix(pdf): add timeout to PDF downloads to prevent hanging (#1141)
2025-05-23 16:05:44 +08:00
script
Add C4A-Script support and documentation
2025-06-07 23:07:19 +08:00
__init__.py
feat: cleanup unused code and enhance documentation for v0.7.1
2025-07-17 11:35:16 +02:00
__version__.py
chore: update version to 0.7.1
2025-07-17 11:37:41 +02:00
adaptive_crawler copy.py
feat(crawl4ai): Implement adaptive crawling feature
2025-07-04 15:16:53 +08:00
adaptive_crawler.py
feat(crawl4ai): Implement adaptive crawling feature
2025-07-04 15:16:53 +08:00
async_configs.py
Release v0.7.0-r1: The Adaptive Intelligence Update
2025-07-12 18:51:13 +08:00
async_crawler_strategy.py
Merge branch '2025-JUN-1' into next-MAY
2025-07-09 09:41:03 +02:00
async_database.py
WIP: logger migriate to rich
2025-04-11 00:44:43 +08:00
async_dispatcher.py
Set memory_wait_timeout default to 10 minutes (#1193)
2025-06-10 15:47:03 +08:00
async_logger.py
Merge branch '2025-MAY-2' into next-MAY
2025-07-08 11:46:13 +02:00
async_url_seeder.py
Release v0.7.0-r1: The Adaptive Intelligence Update
2025-07-12 18:51:13 +08:00
async_webcrawler.py
Merge pull request #1265 from unclecode/feature/nasrin-cli-deep-crawl
2025-07-23 09:40:33 +02:00
browser_manager.py
feat: cleanup unused code and enhance documentation for v0.7.1
2025-07-17 11:35:16 +02:00
browser_profiler.py
Merge branch 'next' into 2025-MAY-2
2025-06-02 20:26:40 +02:00
cache_context.py
Apply Ruff Corrections
2025-01-13 19:19:58 +08:00
chunking_strategy.py
refactor(core): reorganize project structure and remove legacy code
2025-01-30 19:35:06 +08:00
cli.py
Merge branch '2025-MAY-2' into next-MAY
2025-07-08 11:46:13 +02:00
config.py
refactor(proxy): move ProxyConfig to async_configs and improve LLM token handling
2025-04-15 22:27:18 +08:00
content_filter_strategy.py
Add use_stemming option to BM25ContentFilter (#1192)
2025-06-10 15:44:45 +08:00
content_scraping_strategy.py
feat: cleanup unused code and enhance documentation for v0.7.1
2025-07-17 11:35:16 +02:00
docker_client.py
fix(auth): add token authorization header in request preparation to ensure authenticated requests are made
2025-05-21 13:27:17 +08:00
extraction_strategy.py
Merge branch '2025-MAY-2' of https://github.com/unclecode/crawl4ai into 2025-MAY-2
2025-06-13 11:26:17 +02:00
hub.py
feat(logger): add abstract logger base class and file logger implementation
2025-02-23 21:23:41 +08:00
install.py
feat(docs): enhance documentation UI with ToC and GitHub stats
2025-04-14 20:46:32 +08:00
link_preview.py
refactor(link_extractor): remove link_extractor and rename to link_preview
2025-06-27 21:54:22 +08:00
markdown_generation_strategy.py
feat(markdown): add content source selection for markdown generation
2025-04-17 20:13:53 +08:00
migrations.py
Apply Ruff Corrections
2025-01-13 19:19:58 +08:00
model_loader.py
Apply Ruff Corrections
2025-01-13 19:19:58 +08:00
models.py
feat: Add advanced link head extraction with three-layer scoring system (#1)
2025-06-27 20:06:04 +08:00
prompts.py
Add C4A-Script support and documentation
2025-06-07 23:07:19 +08:00
proxy_strategy.py
refactor(proxy): move ProxyConfig to async_configs and improve LLM token handling
2025-04-15 22:27:18 +08:00
ssl_certificate.py
refactor(proxy): move ProxyConfig to async_configs and improve LLM token handling
2025-04-15 22:27:18 +08:00
types.py
feat: Add AsyncUrlSeeder for intelligent URL discovery and filtering
2025-06-03 23:27:12 +08:00
user_agent_generator.py
feat(browser): add BrowserProfiler class for identity-based browsing
2025-03-02 20:32:29 +08:00
utils.py
fix: Improve comments for article metadata extraction in extract_metadata functions. ref #1105
2025-07-08 12:54:33 +02:00