Aravind Karnam 6e78c56dda Refactor: Removed all scheduling logic from scraper. From now scraper expects arun_many to handle all scheduling. Scraper will only do traversal, validations, compliance checks, URL filtering and scoring etc. Reformatted some of the scraper files with Black code formatter

2025-01-21 18:44:43 +05:30

..

assets

Refactor mission section in README and add mission diagram

2024-10-31 16:38:56 +08:00

deprecated

docs: update README badges and Docker section, reorganize documentation structure

2024-12-31 19:45:02 +08:00

examples

feat(browser): improve browser context management and add shared data support

2025-01-19 17:12:03 +08:00

md_v2

Merge branch 'next' into feature/scraper

2025-01-21 12:35:45 +05:30

notebooks

Update documents, upload new version of quickstart.

2024-10-30 20:39:35 +08:00

scraper

Refactor: Removed all scheduling logic from scraper. From now scraper expects arun_many to handle all scheduling. Scraper will only do traversal, validations, compliance checks, URL filtering and scoring etc. Reformatted some of the scraper files with Black code formatter

2025-01-21 18:44:43 +05:30