ayrisdev/crawl4ai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
18ad3ef1599a4c7565692a8368b16fb2789de4a6
crawl4ai/tests
History
Soham Kukreti 18ad3ef159 fix: Implement base tag support in link extraction (#1147) 
- Extract base href from <head><base> tag using XPath in _process_element method
- Use base URL as the primary URL for link normalization when present
- Add error handling with logging for malformed or problematic base tags
- Maintain backward compatibility when no base tag is present
- Add test to verify the functionality of the base tag extraction.
2025-08-08 20:11:57 +05:30
..
adaptive
feat(crawl4ai): Implement adaptive crawling feature
2025-07-04 15:16:53 +08:00
async
fix: Implement base tag support in link extraction (#1147)
2025-08-08 20:11:57 +05:30
async_assistant
test(async_assistant): add new tests for extract pipeline
2025-06-23 10:44:27 +08:00
browser
feat(browser): implement browser pooling and page pre-warming
2025-03-31 21:55:07 +08:00
cli
feat(cli): add command line interface with comprehensive features
2025-02-10 16:58:52 +08:00
deep_crwaling
#1167 Add PHP MIME types to ContentTypeFilter for better file handling
2025-06-09 11:49:33 +08:00
docker
fix: handle infinity values in JSON serialization for API responses
2025-07-15 13:49:07 +02:00
general
Merge branch '2025-JUN-1' into next-MAY
2025-07-09 09:41:03 +02:00
hub
refactor(crawler): improve HTML handling and cleanup codebase
2025-02-07 21:56:27 +08:00
loggers
Release prep (#749)
2025-02-28 19:53:35 +08:00
mcp
feat(docker): update Docker deployment for v0.6.0
2025-04-22 22:35:25 +08:00
memory
feat(mcp): Implement MCP protocol and enhance server capabilities
2025-04-21 22:22:02 +08:00
profiler
feat(browser): improve browser profile management and cleanup
2025-04-29 23:04:32 +08:00
releases
test(releases): Add test cases for release 0.7.0
2025-07-11 22:27:18 +08:00
__init__.py
- Test all methods
2024-05-14 21:27:41 +08:00
check_dependencies.py
feat: add stealth mode and enhance undetected browser support
2025-07-17 16:59:10 +08:00
docker_example.py
Fix: README.md urls list
2025-04-29 16:26:35 +02:00
test_arun_many.py
feat: Add URL-specific crawler configurations for multi-URL crawling
2025-08-02 19:10:36 +08:00
test_cli_docs.py
Apply Ruff Corrections
2025-01-13 19:19:58 +08:00
test_config_matching_only.py
feat: Add URL-specific crawler configurations for multi-URL crawling
2025-08-02 19:10:36 +08:00
test_config_selection.py
feat: Add URL-specific crawler configurations for multi-URL crawling
2025-08-02 19:10:36 +08:00
test_docker_api_with_llm_provider.py
feat(docker): add flexible LLM provider configuration
2025-08-05 14:09:54 +08:00
test_docker.py
Fix: README.md urls list
2025-04-29 16:26:35 +02:00
test_link_extractor.py
feat: cleanup unused code and enhance documentation for v0.7.1
2025-07-17 11:35:16 +02:00
test_llmtxt.py
Apply Ruff Corrections
2025-01-13 19:19:58 +08:00
test_main.py
Fix: README.md urls list
2025-04-29 16:26:35 +02:00
test_memory_macos.py
fix: Correct URL matcher fallback behavior and improve memory monitoring
2025-08-03 16:50:54 +08:00
test_multi_config.py
fix: Correct URL matcher fallback behavior and improve memory monitoring
2025-08-03 16:50:54 +08:00
test_normalize_url.py
#1103 fix(url): enhance URL normalization to handle invalid schemes and trailing slashes
2025-05-19 13:51:16 +08:00
test_scraping_strategy.py
Release prep (#749)
2025-02-28 19:53:35 +08:00
test_virtual_scroll.py
feat: Add virtual scroll support for modern web scraping
2025-06-29 20:41:37 +08:00
test_web_crawler.py
Update all documentation to import extraction strategies directly from crawl4ai.
2025-06-10 18:08:27 +08:00