feat(crawler): add network request and console message capturing
Implement comprehensive network request and console message capturing functionality: - Add capture_network_requests and capture_console_messages config parameters - Add network_requests and console_messages fields to models - Implement Playwright event listeners to capture requests, responses, and console output - Create detailed documentation and examples - Add comprehensive tests This feature enables deep visibility into web page activity for debugging, security analysis, performance profiling, and API discovery in web applications.
This commit is contained in:
20
parameter_updates.txt
Normal file
20
parameter_updates.txt
Normal file
@@ -0,0 +1,20 @@
|
||||
The file /docs/md_v2/api/parameters.md should be updated to include the new network and console capturing parameters.
|
||||
|
||||
Here's what needs to be updated:
|
||||
|
||||
1. Change section title from:
|
||||
```
|
||||
### G) **Debug & Logging**
|
||||
```
|
||||
to:
|
||||
```
|
||||
### G) **Debug, Logging & Capturing**
|
||||
```
|
||||
|
||||
2. Add new parameters to the table:
|
||||
```
|
||||
| **`capture_network_requests`** | `bool` (False) | Captures all network requests, responses, and failures during the crawl. Available in `result.network_requests`. |
|
||||
| **`capture_console_messages`** | `bool` (False) | Captures all browser console messages (logs, warnings, errors) during the crawl. Available in `result.console_messages`. |
|
||||
```
|
||||
|
||||
These changes demonstrate how to use the new network and console capturing features in the CrawlerRunConfig.
|
||||
Reference in New Issue
Block a user