ayrisdev/crawl4ai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
fde4e9f0c64aa2870da447cce00a9b3f7fcf1322
crawl4ai/tests
History
unclecode fde4e9f0c6 Add prefetch mode for two-phase deep crawling 
- Add `prefetch` parameter to CrawlerRunConfig
- Add `quick_extract_links()` function for fast link extraction
- Add short-circuit in aprocess_html() for prefetch mode
- Add 42 tests (unit, integration, regression)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-25 01:55:08 +00:00
..
adaptive
Release/v0.7.6 (#1556)
2025-10-22 20:41:06 +08:00
async
Fix: capture current page URL to reflect JavaScript navigation and add test for delayed redirects. ref #1268
2025-12-02 13:00:54 +01:00
async_assistant
test(async_assistant): add new tests for extract pipeline
2025-06-23 10:44:27 +08:00
browser
Fix CDP connection handling: support WS URLs and proper cleanup
2025-12-18 22:04:52 +08:00
cache_validation
Some debugging for caching
2025-12-21 04:48:03 +00:00
cli
feat(cli): add command line interface with comprehensive features
2025-02-10 16:58:52 +08:00
deep_crawling
Add crash recovery for deep crawl strategies
2025-12-22 14:51:10 +00:00
deep_crwaling
#1167 Add PHP MIME types to ContentTypeFilter for better file handling
2025-06-09 11:49:33 +08:00
docker
parameterized tests
2025-12-01 16:19:19 -05:00
general
Merge branch 'develop' into fix/wrong_url_raw
2025-11-24 13:54:07 +02:00
hub
refactor(crawler): improve HTML handling and cleanup codebase
2025-02-07 21:56:27 +08:00
loggers
Release prep (#749)
2025-02-28 19:53:35 +08:00
mcp
feat(docker): update Docker deployment for v0.6.0
2025-04-22 22:35:25 +08:00
memory
Release/v0.7.6 (#1556)
2025-10-22 20:41:06 +08:00
profiler
chore(profile-test): fix filename typo ( test_crteate_profile.py → test_create_profile.py )
2025-06-12 14:38:32 +03:00
proxy
Release/v0.7.6 (#1556)
2025-10-22 20:41:06 +08:00
releases
test(releases): Add test cases for release 0.7.0
2025-07-11 22:27:18 +08:00
unit
#1559 :Add tests for sitemap parsing and URL normalization in AsyncUrlSeeder
2025-11-10 14:15:54 +08:00
__init__.py
- Test all methods
2024-05-14 21:27:41 +08:00
check_dependencies.py
refactor: replace PyPDF2 with pypdf across the codebase. ref #1412
2025-12-03 10:59:18 +01:00
docker_example.py
docs: remove CRAWL4AI_API_TOKEN references and use correct endpoints in Docker example scripts (#1015)
2025-08-09 19:37:22 +05:30
test_arun_many.py
feat: Add URL-specific crawler configurations for multi-URL crawling
2025-08-02 19:10:36 +08:00
test_cli_docs.py
Apply Ruff Corrections
2025-01-13 19:19:58 +08:00
test_config_matching_only.py
feat: Add URL-specific crawler configurations for multi-URL crawling
2025-08-02 19:10:36 +08:00
test_config_selection.py
feat: Add URL-specific crawler configurations for multi-URL crawling
2025-08-02 19:10:36 +08:00
test_docker_api_with_llm_provider.py
feat(docker): add flexible LLM provider configuration
2025-08-05 14:09:54 +08:00
test_docker.py
Fix: README.md urls list
2025-04-29 16:26:35 +02:00
test_link_extractor.py
feat: cleanup unused code and enhance documentation for v0.7.1
2025-07-17 11:35:16 +02:00
test_llm_extraction_parallel_issue_1055.py
This commit resolves issue #1055 where LLM extraction was blocking async
2025-11-06 11:22:45 +01:00
test_llm_simple_url.py
refactor: Update LLMTableExtraction examples and tests
2025-08-15 18:47:31 +08:00
test_llmtxt.py
Apply Ruff Corrections
2025-01-13 19:19:58 +08:00
test_main.py
Fix: README.md urls list
2025-04-29 16:26:35 +02:00
test_memory_macos.py
refactor(utils): move memory utilities to utils and update imports
2025-08-17 19:14:55 +08:00
test_multi_config.py
fix: Correct URL matcher fallback behavior and improve memory monitoring
2025-08-03 16:50:54 +08:00
test_normalize_url.py
#1103 fix(url): enhance URL normalization to handle invalid schemes and trailing slashes
2025-05-19 13:51:16 +08:00
test_prefetch_integration.py
Add prefetch mode for two-phase deep crawling
2025-12-25 01:55:08 +00:00
test_prefetch_mode.py
Add prefetch mode for two-phase deep crawling
2025-12-25 01:55:08 +00:00
test_prefetch_regression.py
Add prefetch mode for two-phase deep crawling
2025-12-25 01:55:08 +00:00
test_preserve_https_for_internal_links.py
feat: add preserve_https_for_internal_links flag to maintain HTTPS during crawling. Ref #1410
2025-08-28 17:38:40 +08:00
test_pyopenssl_security_fix.py
test: add verification tests for pyOpenSSL security update
2025-10-23 06:57:25 +00:00
test_pyopenssl_update.py
test: add verification tests for pyOpenSSL security update
2025-10-23 06:57:25 +00:00
test_scraping_strategy.py
Release prep (#749)
2025-02-28 19:53:35 +08:00
test_virtual_scroll.py
feat: Add virtual scroll support for modern web scraping
2025-06-29 20:41:37 +08:00
test_web_crawler.py
Update all documentation to import extraction strategies directly from crawl4ai.
2025-06-10 18:08:27 +08:00
test_webhook_feature.sh
test: add comprehensive webhook feature test script
2025-10-22 00:35:07 +00:00
WEBHOOK_TEST_README.md
test: add comprehensive webhook feature test script
2025-10-22 00:35:07 +00:00