AHMET YILMAZ c003cb6e4f fix #1563 (cdp): resolve page leaks and race conditions in concurrent crawling ...

Fix memory leaks and race conditions when using arun_many() with managed CDP browsers. Each crawl now gets proper page isolation with automatic cleanup while maintaining shared browser context.

Key fixes:
- Close non-session pages after crawling to prevent tab accumulation
- Add thread-safe page creation with locks to avoid concurrent access
- Improve page lifecycle management for managed vs non-managed browsers
- Keep session pages alive for authentication persistence
- Prevent TOCTOU (time-of-check-time-of-use) race conditions

This ensures stable parallel crawling without memory growth or browser instability.

2025-11-07 15:42:37 +08:00

2.0 KiB

Raw Permalink Blame History

View Raw