crawl4ai

adaptive_crawling

fix: allow custom LLM providers for adaptive crawler embedding config. ref: #1291

2025-09-09 12:49:55 +08:00

assets

feat: Add virtual scroll support for modern web scraping

2025-06-29 20:41:37 +08:00

c4a_script

fix(docs): standardize C4A-Script tutorial, add CLI identity-based crawling, and add sponsorship CTA

2025-10-03 22:00:46 +05:30

cli

feat(cli): add command line interface with comprehensive features

2025-02-10 16:58:52 +08:00

docker

feat(docker-api): add job-based polling endpoints for crawl and LLM tasks

2025-05-01 21:24:52 +08:00

markdown

feat(markdown): add content source selection for markdown generation

2025-04-17 20:13:53 +08:00

undetectability

feat: add stealth mode and enhance undetected browser support

2025-07-17 16:59:10 +08:00

url_seeder

docs(tutorial_url_seeder): refine summary and next steps, enhance agentic design patterns section

2025-06-05 16:20:58 +08:00

website-to-api

feat: Add comprehensive website to API example with frontend

2025-08-24 18:52:37 +05:30

amazon_product_extraction_direct_url.py

Update all documentation to import extraction strategies directly from crawl4ai.

2025-06-10 18:08:27 +08:00

amazon_product_extraction_using_hooks.py

Update all documentation to import extraction strategies directly from crawl4ai.

2025-06-10 18:08:27 +08:00

amazon_product_extraction_using_use_javascript.py

Update all documentation to import extraction strategies directly from crawl4ai.

2025-06-10 18:08:27 +08:00

arun_vs_arun_many.py

feat(crawler): optimize single URL handling and add performance comparison

2025-03-13 22:15:15 +08:00

async_webcrawler_multiple_urls_example.py

Apply Ruff Corrections

2025-01-13 19:19:58 +08:00

browser_optimization_example.py

Release prep (#749 )

2025-02-28 19:53:35 +08:00

builtin_browser_example.py

feat(browser): add builtin browser management system

2025-03-20 12:13:59 +08:00

chainlit.md

Add research assistant example using Chainlit

2024-06-04 22:43:09 +08:00

crawlai_vs_firecrawl.py

Apply Ruff Corrections

2025-01-13 19:19:58 +08:00

crawler_monitor_example.py

feat(monitor): add real-time crawler monitoring system with memory management

2025-03-12 19:05:24 +08:00

crypto_analysis_example.py

feat(models): add dedicated tables field to CrawlResult

2025-04-24 18:36:25 +08:00

deepcrawl_example.py

fix(docs): correct section numbering in deepcrawl_example.py tutorial

2025-03-04 20:57:33 +08:00

demo_multi_config_clean.py

fix: Correct URL matcher fallback behavior and improve memory monitoring

2025-08-03 16:50:54 +08:00

dispatcher_example.py

feat(content): add target_elements parameter for selective content extraction

2025-03-10 18:54:51 +08:00

docker_client_hooks_example.py

feat: Add hooks utility for function-based hooks with Docker client integration. ref #1377

2025-10-13 12:34:08 +08:00

docker_config_obj.py

refactor(config): enhance serialization and config handling

2025-02-19 17:23:25 +08:00

docker_example.py

docs: remove CRAWL4AI_API_TOKEN references and use correct endpoints in Docker example scripts (#1015 )

2025-08-09 19:37:22 +05:30

docker_hooks_examples.py

docs: update docker_hooks_examples.py with comprehensive examples and improved structure

2025-10-22 16:29:19 +02:00

docker_python_rest_api.py

feat(crawler): optimize single URL handling and add performance comparison

2025-03-13 22:15:15 +08:00

docker_python_sdk.py

refactor(config): enhance serialization and config handling

2025-02-19 17:23:25 +08:00

docker_webhook_example.py

fix: update Crawl4AI Docker container port from 11234 to 11235

2025-10-22 13:14:11 +02:00

extraction_strategies_examples.py

Update all documentation to import extraction strategies directly from crawl4ai.

2025-06-10 18:08:27 +08:00

full_page_screenshot_and_pdf_export.md

docs + fix: Update example for full page screenshot & PDF export. Fix the bug Error: crawl4ai.async_webcrawler.AsyncWebCrawler.aprocess_html() got multiple values for keyword argument - for screenshot param. https://github.com/unclecode/crawl4ai/issues/822#issuecomment-2732602118

2025-03-18 17:20:24 +05:30

hello_world_undetected.py

feat: add stealth mode and enhance undetected browser support

2025-07-17 16:59:10 +08:00

hello_world.py

feat(crawler): add session management and view-source support

2025-05-08 17:13:35 +08:00

hooks_example.py

Apply Ruff Corrections

2025-01-13 19:19:58 +08:00

identity_based_browsing.py

feat(profiles): add CLI command for crawling with browser profiles

2025-03-02 21:33:33 +08:00

language_support_example.py

Apply Ruff Corrections

2025-01-13 19:19:58 +08:00

link_head_extraction_example.py

feat: cleanup unused code and enhance documentation for v0.7.1

2025-07-17 11:35:16 +02:00

llm_extraction_openai_pricing.py

refactor: Update LLM extraction example with the updated structure

2025-06-12 12:23:03 +02:00

llm_markdown_generator.py

feat(browser): add standalone CDP browser launch and lxml extraction strategy

2025-03-07 20:55:56 +08:00

llm_table_extraction_example.py

refactor: Update LLMTableExtraction examples and tests

2025-08-15 18:47:31 +08:00

network_console_capture_example.py

Fixed capturing console messages for case the url is the local file. Update docker configuration (work in progress)

2025-04-10 23:22:38 +08:00

proxy_rotation_demo.py

feat(proxy): add proxy rotation strategy

2025-02-09 18:49:10 +08:00

quickstart_examples_set_1.py

refactor(proxy): move ProxyConfig to async_configs and improve LLM token handling

2025-04-15 22:27:18 +08:00

quickstart_examples_set_2.py

Update all documentation to import extraction strategies directly from crawl4ai.

2025-06-10 18:08:27 +08:00

quickstart.ipynb

Update all documentation to import extraction strategies directly from crawl4ai.

2025-06-10 18:08:27 +08:00

quickstart.py

Update all documentation to import extraction strategies directly from crawl4ai.

2025-06-10 18:08:27 +08:00

README_BUILTIN_BROWSER.md

feat(browser): add builtin browser management system

2025-03-20 12:13:59 +08:00

regex_extraction_quickstart.py

feat(extraction): add RegexExtractionStrategy for pattern-based extraction

2025-05-02 21:15:24 +08:00

research_assistant.py

Apply Ruff Corrections

2025-01-13 19:19:58 +08:00

rest_call.py

Apply Ruff Corrections

2025-01-13 19:19:58 +08:00

sample_ecommerce.html

Push async version last changes for merge to main branch

2024-09-24 20:52:08 +08:00

scraping_strategies_performance.py

Squashed commit of the following:

2025-08-04 19:02:01 +08:00

serp_api_project_11_feb.py

Release prep (#749 )

2025-02-28 19:53:35 +08:00

session_id_example.py

feat(crawler): add session management and view-source support

2025-05-08 17:13:35 +08:00

simple_anti_bot_examples.py

docs: add simple anti-bot examples

2025-07-17 17:05:35 +08:00

ssl_example.py

Apply Ruff Corrections

2025-01-13 19:19:58 +08:00

stealth_mode_example.py

feat: add stealth mode and enhance undetected browser support

2025-07-17 16:59:10 +08:00

stealth_mode_quick_start.py

feat: add stealth mode and enhance undetected browser support

2025-07-17 16:59:10 +08:00

stealth_test_simple.py

feat: add stealth mode and enhance undetected browser support

2025-07-17 16:59:10 +08:00

storage_state_tutorial.md

Implement new async crawler features and stability updates

2024-12-10 17:55:29 +08:00

summarize_page.py

Update all documentation to import extraction strategies directly from crawl4ai.

2025-06-10 18:08:27 +08:00

table_extraction_example.py

feat: 🚀 Introduce revolutionary LLMTableExtraction with intelligent chunking for massive tables

2025-08-14 18:21:24 +08:00

tutorial_dynamic_clicks.md

Add PDF & screenshot functionality, new tutorial

2024-12-10 20:10:39 +08:00

tutorial_v0.5.py

Update all documentation to import extraction strategies directly from crawl4ai.

2025-06-10 18:08:27 +08:00

undetected_simple_demo.py

feat: add stealth mode and enhance undetected browser support

2025-07-17 16:59:10 +08:00

use_geo_location.py

feat(browser): add geolocation, locale and timezone support

2025-04-21 23:20:59 +08:00

virtual_scroll_example.py

feat: Add virtual scroll support for modern web scraping

2025-06-29 20:41:37 +08:00