UncleCode c612f9a852 feat(profiles): add CLI command for crawling with browser profiles ...

Adds new functionality to crawl websites using saved browser profiles directly from the CLI.
This includes:
- New CLI option to use profiles for crawling
- Helper functions for profile-based crawling
- Fixed type hints for config parameters
- Updated example to show browser window by default

This makes it easier for users to leverage saved browser profiles for crawling without writing code.

2025-03-02 21:33:33 +08:00

..

assets

- User agent

2024-06-08 17:59:42 +08:00

cli

feat(cli): add command line interface with comprehensive features

2025-02-10 16:58:52 +08:00

amazon_product_extraction_direct_url.py

Apply Ruff Corrections

2025-01-13 19:19:58 +08:00

amazon_product_extraction_using_hooks.py

Apply Ruff Corrections

2025-01-13 19:19:58 +08:00

amazon_product_extraction_using_use_javascript.py

Apply Ruff Corrections

2025-01-13 19:19:58 +08:00

async_webcrawler_multiple_urls_example.py

Apply Ruff Corrections

2025-01-13 19:19:58 +08:00

browser_optimization_example.py

Release prep (#749)

2025-02-28 19:53:35 +08:00

chainlit.md

Add research assistant example using Chainlit

2024-06-04 22:43:09 +08:00

crawlai_vs_firecrawl.py

Apply Ruff Corrections

2025-01-13 19:19:58 +08:00

deepcrawl.py

Release prep (#749)

2025-02-28 19:53:35 +08:00

dispatcher_example.py

docs(examples): update demo scripts and fix output formats

2025-01-22 20:40:03 +08:00

docker_config_obj.py

refactor(config): enhance serialization and config handling

2025-02-19 17:23:25 +08:00

docker_example.py

Apply Ruff Corrections

2025-01-13 19:19:58 +08:00

docker_python_rest_api.py

refactor(config): enhance serialization and config handling

2025-02-19 17:23:25 +08:00

docker_python_sdk.py

refactor(config): enhance serialization and config handling

2025-02-19 17:23:25 +08:00

extraction_strategies_examples.py

Release prep (#749)

2025-02-28 19:53:35 +08:00

full_page_screenshot_and_pdf_export.md

Enhance crawler capabilities and documentation

2024-12-25 21:34:31 +08:00

hello_world.py

feat(browser): add BrowserProfiler class for identity-based browsing

2025-03-02 20:32:29 +08:00

hooks_example.py

Apply Ruff Corrections

2025-01-13 19:19:58 +08:00

identity_based_browsing.py

feat(profiles): add CLI command for crawling with browser profiles

2025-03-02 21:33:33 +08:00

language_support_example.py

Apply Ruff Corrections

2025-01-13 19:19:58 +08:00

llm_extraction_openai_pricing.py

Feat/llm config (#724)

2025-02-21 15:41:37 +08:00

llm_markdown_generator.py

Feat/llm config (#724)

2025-02-21 15:41:37 +08:00

proxy_rotation_demo.py

feat(proxy): add proxy rotation strategy

2025-02-09 18:49:10 +08:00

quickstart_async.config.py

Release prep (#749)

2025-02-28 19:53:35 +08:00

quickstart_async.py

Release prep (#749)

2025-02-28 19:53:35 +08:00

quickstart_sync.py

Feat/llm config (#724)

2025-02-21 15:41:37 +08:00

quickstart_v0.ipynb

docs(urls): update documentation URLs to new domain

2025-01-09 16:22:41 +08:00

quickstart.ipynb

Release prep (#749)

2025-02-28 19:53:35 +08:00

research_assistant.py

Apply Ruff Corrections

2025-01-13 19:19:58 +08:00

rest_call.py

Apply Ruff Corrections

2025-01-13 19:19:58 +08:00

sample_ecommerce.html

Push async version last changes for merge to main branch

2024-09-24 20:52:08 +08:00

scraping_strategies_performance.py

docs(examples): update demo scripts and fix output formats

2025-01-22 20:40:03 +08:00

serp_api_project_11_feb.py

Release prep (#749)

2025-02-28 19:53:35 +08:00

ssl_example.py

Apply Ruff Corrections

2025-01-13 19:19:58 +08:00

storage_state_tutorial.md

Implement new async crawler features and stability updates

2024-12-10 17:55:29 +08:00

summarize_page.py

Apply Ruff Corrections

2025-01-13 19:19:58 +08:00

tutorial_dynamic_clicks.md

Add PDF & screenshot functionality, new tutorial

2024-12-10 20:10:39 +08:00

tutorial_v0.5.py

Release prep (#749)

2025-02-28 19:53:35 +08:00