UncleCode 16f918621f Merge branch 'main' of https://github.com/unclecode/crawl4ai

2024-11-07 19:30:22 +08:00

..

html2text

Enhance Markdown generation and external content control

2024-10-20 18:56:58 +08:00

__init__.py

Refactor version management and remove deprecated gitignore.dev file

2024-11-04 16:51:59 +08:00

_version.py

chore(version): bump version to 0.3.73

2024-11-05 20:05:58 +08:00

async_crawler_strategy.py

Improved database management and error handling, updated README instructions, refined .gitignore, enhanced async web crawling capabilities, and updated dependencies.

2024-11-04 13:22:13 +08:00

async_database.py

Improved database management and error handling, updated README instructions, refined .gitignore, enhanced async web crawling capabilities, and updated dependencies.

2024-11-04 13:22:13 +08:00

async_webcrawler.py

feat(core): Release v0.3.73 with Browser Takeover and Docker Support

2024-11-05 20:04:18 +08:00

chunking_strategy.py

[v0.3.71] Enhance chunking strategies and improve overall performance

2024-10-19 18:36:59 +08:00

config.py

feat: enhance crawler with overlay removal and improved screenshot capabilities

2024-10-24 20:22:47 +08:00

content_cleaning_strategy.py

[v0.3.72] Enhance content extraction and proxy support

2024-10-22 20:19:22 +08:00

content_scrapping_strategy.py

Refactored the CustomHTML2Text class in content_scrapping_strategy.py to remove the handling logic for header tags (h1-h6), which are now commented out. This cleanup improves code readability and reduces maintenance overhead.

2024-11-06 21:50:09 +08:00

crawler_strategy.py

Push async version last changes for merge to main branch

2024-09-24 20:52:08 +08:00

database.py

## [v0.2.74] - 2024-07-08

2024-07-08 16:33:25 +08:00

extraction_strategy.py

feat: enhance crawler with overlay removal and improved screenshot capabilities

2024-10-24 20:22:47 +08:00

model_loader.py

[v0.3.71] Enhance chunking strategies and improve overall performance

2024-10-19 18:36:59 +08:00

models.py

[v0.3.72] Enhance content extraction and proxy support

2024-10-22 20:19:22 +08:00

prompts.py

feat: Enhance crawler flexibility and LLM extraction capabilities

2024-10-14 21:03:28 +08:00

train.py

Remove dependency on Spacy model.

2024-05-17 15:08:03 +08:00

utils.py

Merge pull request #229 from bizrockman/main

2024-11-06 07:31:07 +01:00

web_crawler.back.py

vital: Right now, only raw html is retrived from datbase, therefore, css selector and other filter will be executed every time.

2024-06-08 18:37:40 +08:00

web_crawler.py

feat: Enhance crawler flexibility and LLM extraction capabilities

2024-10-14 21:03:28 +08:00