UncleCode c6a605ccce feat(filters): add reverse option to URLPatternFilter ...

Adds a new 'reverse' parameter to URLPatternFilter that allows inverting the filter's logic. When reverse=True, URLs that would normally match are rejected and vice versa.

Also removes unused 'scraped_html' from WebScrapingStrategy output to reduce memory usage.

BREAKING CHANGE: WebScrapingStrategy no longer returns 'scraped_html' in its output dictionary

2025-03-08 18:54:41 +08:00

..

__init__.py

Release prep (#749)

2025-02-28 19:53:35 +08:00

base_strategy.py

feat(profiles): add CLI command for crawling with browser profiles

2025-03-02 21:33:33 +08:00

bff_strategy.py

refactor(core): replace float('inf') with math.inf

2025-03-04 18:23:55 +08:00

bfs_strategy.py

feat(deep-crawling): improve URL normalization and domain filtering

2025-03-06 22:45:57 +08:00

crazy.py

refactor(config): enhance serialization and config handling

2025-02-19 17:23:25 +08:00

dfs_strategy.py

refactor(deep-crawl): add max_pages limit and improve crawl control

2025-03-03 21:51:11 +08:00

filters.py

feat(filters): add reverse option to URLPatternFilter

2025-03-08 18:54:41 +08:00

scorers.py

2025 feb alpha 1 (#685)

2025-02-19 14:13:17 +08:00