Add Crawl4AI Assistant Chrome Extension
- Created manifest.json for the Crawl4AI Assistant extension. - Added popup HTML, CSS, and JS files for the extension interface. - Included icons and favicon for the extension. - Implemented functionality for schema capture and code generation. - Updated index.md to reflect the availability of the new extension. - Enhanced LLM Context Builder layout and styles for consistency. - Adjusted global styles for better branding and responsiveness.
@@ -4,7 +4,11 @@
|
||||
"Bash(cd:*)",
|
||||
"Bash(python3:*)",
|
||||
"Bash(python:*)",
|
||||
"Bash(grep:*)"
|
||||
"Bash(grep:*)",
|
||||
"Bash(mkdir:*)",
|
||||
"Bash(cp:*)",
|
||||
"Bash(rm:*)",
|
||||
"Bash(true)"
|
||||
]
|
||||
},
|
||||
"enableAllProjectMcpServers": false
|
||||
|
||||
BIN
docs/md_v2/apps/assets/DankMono-Bold.woff2
Normal file
BIN
docs/md_v2/apps/assets/DankMono-Italic.woff2
Normal file
BIN
docs/md_v2/apps/assets/DankMono-Regular.woff2
Normal file
123
docs/md_v2/apps/crawl4ai-assistant/README.md
Normal file
@@ -0,0 +1,123 @@
|
||||
# Crawl4AI Chrome Extension
|
||||
|
||||
Visual schema and script builder for Crawl4AI - Build extraction schemas by clicking on webpage elements!
|
||||
|
||||
## 🚀 Features
|
||||
|
||||
- **Visual Schema Builder**: Click on elements to build extraction schemas
|
||||
- **Smart Element Selection**: Container and field selection with visual feedback
|
||||
- **Code Generation**: Generates complete Python code with LLM integration
|
||||
- **Beautiful Dark UI**: Consistent with Crawl4AI's design language
|
||||
- **One-Click Download**: Get your generated code instantly
|
||||
|
||||
## 📦 Installation
|
||||
|
||||
### Method 1: Load Unpacked Extension (Recommended for Development)
|
||||
|
||||
1. Open Chrome and navigate to `chrome://extensions/`
|
||||
2. Enable "Developer mode" in the top right corner
|
||||
3. Click "Load unpacked"
|
||||
4. Select the `crawl4ai-assistant` folder
|
||||
5. The extension icon (🚀🤖) will appear in your toolbar
|
||||
|
||||
### Method 2: Generate Icons First
|
||||
|
||||
If you want proper icons:
|
||||
|
||||
1. Open `icons/generate_icons.html` in your browser
|
||||
2. Right-click each canvas and save as:
|
||||
- `icon-16.png`
|
||||
- `icon-48.png`
|
||||
- `icon-128.png`
|
||||
3. Then follow Method 1 above
|
||||
|
||||
## 🎯 How to Use
|
||||
|
||||
### Building a Schema
|
||||
|
||||
1. **Navigate to any website** you want to extract data from
|
||||
2. **Click the Crawl4AI extension icon** in your toolbar
|
||||
3. **Click "Schema Builder"** to start the capture mode
|
||||
4. **Select a container element**:
|
||||
- Hover over elements (they'll highlight in blue)
|
||||
- Click on a repeating container (e.g., product card, article block)
|
||||
5. **Select fields within the container**:
|
||||
- Elements will now highlight in green
|
||||
- Click on each piece of data you want to extract
|
||||
- Name each field (e.g., "title", "price", "description")
|
||||
6. **Generate the code**:
|
||||
- Click "Generate Code" in the extension popup
|
||||
- A Python file will automatically download
|
||||
|
||||
### Running the Generated Code
|
||||
|
||||
The downloaded Python file contains:
|
||||
|
||||
```python
|
||||
# 1. The HTML snippet of your selected container
|
||||
HTML_SNIPPET = """..."""
|
||||
|
||||
# 2. The extraction query based on your selections
|
||||
EXTRACTION_QUERY = """..."""
|
||||
|
||||
# 3. Functions to generate and test the schema
|
||||
async def generate_schema():
|
||||
# Generates the extraction schema using LLM
|
||||
|
||||
async def test_extraction():
|
||||
# Tests the schema on the actual website
|
||||
```
|
||||
|
||||
To use it:
|
||||
|
||||
1. Install Crawl4AI: `pip install crawl4ai`
|
||||
2. Run the script: `python crawl4ai_schema_*.py`
|
||||
3. The script will generate a `generated_schema.json` file
|
||||
4. Use this schema in your Crawl4AI projects!
|
||||
|
||||
## 🎨 Visual Feedback
|
||||
|
||||
- **Blue dashed outline**: Container selection mode
|
||||
- **Green dashed outline**: Field selection mode
|
||||
- **Solid blue outline**: Selected container
|
||||
- **Solid green outline**: Selected fields
|
||||
- **Floating toolbar**: Shows current mode and selection status
|
||||
|
||||
## ⌨️ Keyboard Shortcuts
|
||||
|
||||
- **ESC**: Cancel current capture session
|
||||
|
||||
## 🔧 Technical Details
|
||||
|
||||
- Built with Manifest V3 for security and performance
|
||||
- Pure client-side - no data sent to external servers
|
||||
- Generates code that uses Crawl4AI's LLM integration
|
||||
- Smart selector generation prioritizes stable attributes
|
||||
|
||||
## 🐛 Troubleshooting
|
||||
|
||||
### Extension doesn't load
|
||||
- Make sure you're in Developer Mode
|
||||
- Check the console for any errors
|
||||
- Ensure all files are in the correct directories
|
||||
|
||||
### Can't select elements
|
||||
- Some websites may block extensions
|
||||
- Try refreshing the page
|
||||
- Make sure you clicked "Schema Builder" first
|
||||
|
||||
### Generated code doesn't work
|
||||
- Ensure you have Crawl4AI installed
|
||||
- Check that you have an LLM API key configured
|
||||
- Make sure the website structure hasn't changed
|
||||
|
||||
## 🤝 Contributing
|
||||
|
||||
This extension is part of the Crawl4AI project. Contributions are welcome!
|
||||
|
||||
- Report issues: [GitHub Issues](https://github.com/unclecode/crawl4ai/issues)
|
||||
- Join discussion: [Discord](https://discord.gg/crawl4ai)
|
||||
|
||||
## 📄 License
|
||||
|
||||
Same as Crawl4AI - see main project for details.
|
||||
BIN
docs/md_v2/apps/crawl4ai-assistant/assets/DankMono-Bold.woff2
Normal file
BIN
docs/md_v2/apps/crawl4ai-assistant/assets/DankMono-Italic.woff2
Normal file
BIN
docs/md_v2/apps/crawl4ai-assistant/assets/DankMono-Regular.woff2
Normal file
539
docs/md_v2/apps/crawl4ai-assistant/assistant.css
Normal file
@@ -0,0 +1,539 @@
|
||||
/* Crawl4AI Assistant Landing Page Styles */
|
||||
|
||||
/* Font Face Definitions */
|
||||
@font-face {
|
||||
font-family: 'Dank Mono';
|
||||
src: url('../assets/DankMono-Regular.woff2') format('woff2');
|
||||
font-weight: 400;
|
||||
font-style: normal;
|
||||
font-display: swap;
|
||||
}
|
||||
|
||||
@font-face {
|
||||
font-family: 'Dank Mono';
|
||||
src: url('../assets/DankMono-Bold.woff2') format('woff2');
|
||||
font-weight: 700;
|
||||
font-style: normal;
|
||||
font-display: swap;
|
||||
}
|
||||
|
||||
@font-face {
|
||||
font-family: 'Dank Mono';
|
||||
src: url('../assets/DankMono-Italic.woff2') format('woff2');
|
||||
font-weight: 400;
|
||||
font-style: italic;
|
||||
font-display: swap;
|
||||
}
|
||||
|
||||
:root {
|
||||
--primary-green: #0fbbaa;
|
||||
--primary-pink: #f380f5;
|
||||
--bg-dark: #070708;
|
||||
--bg-secondary: #1a1a1a;
|
||||
--bg-tertiary: #3f3f44;
|
||||
--text-primary: #e8e9ed;
|
||||
--text-secondary: #a3abba;
|
||||
--text-accent: #d5cec0;
|
||||
--border-color: #3f3f44;
|
||||
--code-bg: #070708;
|
||||
--font-primary: 'Dank Mono', -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, monospace;
|
||||
--font-code: 'Dank Mono', 'Monaco', 'Menlo', 'Consolas', monospace;
|
||||
}
|
||||
|
||||
* {
|
||||
box-sizing: border-box;
|
||||
margin: 0;
|
||||
padding: 0;
|
||||
}
|
||||
|
||||
body {
|
||||
background-color: var(--bg-dark);
|
||||
color: var(--text-primary);
|
||||
font-family: var(--font-primary);
|
||||
line-height: 1.6;
|
||||
overflow-x: hidden;
|
||||
}
|
||||
|
||||
/* Terminal Container */
|
||||
.terminal-container {
|
||||
min-height: 100vh;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
}
|
||||
|
||||
/* Header */
|
||||
.header {
|
||||
background: var(--bg-secondary);
|
||||
border-bottom: 1px solid var(--border-color);
|
||||
padding: 1.5rem 0;
|
||||
position: sticky;
|
||||
top: 0;
|
||||
z-index: 100;
|
||||
backdrop-filter: blur(10px);
|
||||
background: rgba(26, 26, 26, 0.95);
|
||||
}
|
||||
|
||||
.header-content {
|
||||
max-width: 1200px;
|
||||
margin: 0 auto;
|
||||
padding: 0 2rem;
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
.logo-section {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 1rem;
|
||||
}
|
||||
|
||||
.logo {
|
||||
width: 48px;
|
||||
height: 48px;
|
||||
}
|
||||
|
||||
.logo-section h1 {
|
||||
font-size: 1.75rem;
|
||||
font-weight: 700;
|
||||
color: var(--text-primary);
|
||||
}
|
||||
|
||||
.tagline {
|
||||
font-size: 0.875rem;
|
||||
color: var(--text-secondary);
|
||||
margin-top: 0.25rem;
|
||||
}
|
||||
|
||||
.nav-links {
|
||||
display: flex;
|
||||
gap: 2rem;
|
||||
}
|
||||
|
||||
.nav-link {
|
||||
color: var(--text-secondary);
|
||||
text-decoration: none;
|
||||
font-size: 0.875rem;
|
||||
transition: color 0.2s ease;
|
||||
}
|
||||
|
||||
.nav-link:hover {
|
||||
color: var(--primary-green);
|
||||
}
|
||||
|
||||
/* Content */
|
||||
.content {
|
||||
flex: 1;
|
||||
max-width: 1200px;
|
||||
margin: 0 auto;
|
||||
padding: 0 2rem;
|
||||
width: 100%;
|
||||
}
|
||||
|
||||
/* Video Section */
|
||||
.video-section {
|
||||
margin: 3rem 0;
|
||||
}
|
||||
|
||||
.video-wrapper {
|
||||
position: relative;
|
||||
border-radius: 12px;
|
||||
overflow: hidden;
|
||||
box-shadow: 0 8px 32px rgba(0, 0, 0, 0.6);
|
||||
border: 1px solid var(--border-color);
|
||||
}
|
||||
|
||||
.demo-video {
|
||||
width: 100%;
|
||||
height: auto;
|
||||
display: block;
|
||||
}
|
||||
|
||||
/* Terminal Windows */
|
||||
.terminal-window {
|
||||
background: var(--bg-secondary);
|
||||
border: 1px solid var(--border-color);
|
||||
border-radius: 8px;
|
||||
overflow: hidden;
|
||||
margin-bottom: 2rem;
|
||||
box-shadow: 0 4px 16px rgba(0, 0, 0, 0.4);
|
||||
}
|
||||
|
||||
.terminal-header {
|
||||
background: #2a2a2a;
|
||||
padding: 0.75rem 1rem;
|
||||
border-bottom: 1px solid var(--border-color);
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 0.5rem;
|
||||
}
|
||||
|
||||
.terminal-header::before {
|
||||
content: '';
|
||||
display: flex;
|
||||
gap: 8px;
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
.terminal-header::before {
|
||||
content: '● ● ●';
|
||||
color: #ff5f57;
|
||||
font-size: 12px;
|
||||
letter-spacing: 8px;
|
||||
}
|
||||
|
||||
.terminal-title {
|
||||
font-size: 0.875rem;
|
||||
color: var(--text-secondary);
|
||||
margin-left: 1rem;
|
||||
}
|
||||
|
||||
.terminal-content {
|
||||
padding: 2rem;
|
||||
}
|
||||
|
||||
/* Features Grid */
|
||||
.features-grid {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(auto-fit, minmax(250px, 1fr));
|
||||
gap: 1.5rem;
|
||||
margin-top: 2rem;
|
||||
}
|
||||
|
||||
.feature-card {
|
||||
background: var(--bg-tertiary);
|
||||
padding: 1.5rem;
|
||||
border-radius: 8px;
|
||||
border: 1px solid var(--border-color);
|
||||
transition: all 0.2s ease;
|
||||
}
|
||||
|
||||
.feature-card:hover {
|
||||
border-color: var(--primary-green);
|
||||
transform: translateY(-2px);
|
||||
box-shadow: 0 4px 16px rgba(15, 187, 170, 0.2);
|
||||
}
|
||||
|
||||
.feature-icon {
|
||||
font-size: 2rem;
|
||||
margin-bottom: 1rem;
|
||||
display: block;
|
||||
}
|
||||
|
||||
.feature-card h3 {
|
||||
font-size: 1.125rem;
|
||||
margin-bottom: 0.5rem;
|
||||
color: var(--text-primary);
|
||||
}
|
||||
|
||||
.feature-card p {
|
||||
font-size: 0.875rem;
|
||||
color: var(--text-secondary);
|
||||
}
|
||||
|
||||
/* Installation Steps */
|
||||
.installation-steps {
|
||||
margin-top: 1.5rem;
|
||||
}
|
||||
|
||||
.step {
|
||||
display: flex;
|
||||
gap: 1.5rem;
|
||||
margin-bottom: 2rem;
|
||||
align-items: flex-start;
|
||||
}
|
||||
|
||||
.step-number {
|
||||
background: var(--primary-green);
|
||||
color: var(--bg-dark);
|
||||
width: 40px;
|
||||
height: 40px;
|
||||
border-radius: 50%;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
font-weight: bold;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
|
||||
.step-content h4 {
|
||||
margin-bottom: 0.5rem;
|
||||
color: var(--text-primary);
|
||||
}
|
||||
|
||||
.step-content p {
|
||||
color: var(--text-secondary);
|
||||
margin-bottom: 1rem;
|
||||
}
|
||||
|
||||
.download-button {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
gap: 0.75rem;
|
||||
background: var(--primary-green);
|
||||
color: var(--bg-dark);
|
||||
padding: 0.75rem 1.5rem;
|
||||
border-radius: 6px;
|
||||
text-decoration: none;
|
||||
font-weight: 600;
|
||||
transition: all 0.2s ease;
|
||||
margin-top: 1rem;
|
||||
}
|
||||
|
||||
.download-button:hover {
|
||||
background: #1fcbba;
|
||||
transform: translateY(-2px);
|
||||
box-shadow: 0 4px 16px rgba(15, 187, 170, 0.3);
|
||||
}
|
||||
|
||||
.button-icon {
|
||||
font-size: 1.25rem;
|
||||
}
|
||||
|
||||
/* Usage Flow */
|
||||
.usage-flow {
|
||||
margin-top: 1.5rem;
|
||||
}
|
||||
|
||||
.usage-step {
|
||||
background: var(--bg-tertiary);
|
||||
padding: 1.5rem;
|
||||
border-radius: 8px;
|
||||
margin-bottom: 1rem;
|
||||
border: 1px solid var(--border-color);
|
||||
}
|
||||
|
||||
.usage-header {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 1rem;
|
||||
margin-bottom: 0.75rem;
|
||||
}
|
||||
|
||||
.usage-icon {
|
||||
font-size: 1.5rem;
|
||||
}
|
||||
|
||||
.usage-step h4 {
|
||||
margin: 0;
|
||||
color: var(--text-primary);
|
||||
}
|
||||
|
||||
.usage-step p {
|
||||
color: var(--text-secondary);
|
||||
margin-bottom: 0.5rem;
|
||||
}
|
||||
|
||||
/* Code Snippets */
|
||||
.code-snippet {
|
||||
background: var(--code-bg);
|
||||
padding: 0.75rem;
|
||||
border-radius: 4px;
|
||||
margin-top: 0.5rem;
|
||||
font-family: var(--font-code);
|
||||
font-size: 0.875rem;
|
||||
border: 1px solid var(--border-color);
|
||||
}
|
||||
|
||||
.comment {
|
||||
color: var(--text-secondary);
|
||||
font-style: italic;
|
||||
}
|
||||
|
||||
/* Code Section */
|
||||
pre {
|
||||
margin: 0;
|
||||
overflow-x: auto;
|
||||
}
|
||||
|
||||
code {
|
||||
font-family: var(--font-code);
|
||||
font-size: 0.875rem;
|
||||
line-height: 1.6;
|
||||
}
|
||||
|
||||
.keyword {
|
||||
color: var(--primary-green);
|
||||
font-weight: bold;
|
||||
}
|
||||
|
||||
.string {
|
||||
color: var(--primary-pink);
|
||||
}
|
||||
|
||||
.function {
|
||||
color: #ff3c74;
|
||||
}
|
||||
|
||||
/* Coming Soon Section */
|
||||
.coming-soon-section {
|
||||
margin: 4rem 0;
|
||||
}
|
||||
|
||||
.coming-soon-section h2 {
|
||||
font-size: 2rem;
|
||||
margin-bottom: 2rem;
|
||||
color: var(--text-primary);
|
||||
}
|
||||
|
||||
.intro-text {
|
||||
font-size: 1.125rem;
|
||||
color: var(--text-secondary);
|
||||
margin-bottom: 2rem;
|
||||
}
|
||||
|
||||
.coming-features {
|
||||
display: grid;
|
||||
gap: 1.5rem;
|
||||
margin-top: 2rem;
|
||||
}
|
||||
|
||||
.coming-feature {
|
||||
background: var(--bg-tertiary);
|
||||
padding: 2rem;
|
||||
border-radius: 12px;
|
||||
border: 1px solid var(--border-color);
|
||||
transition: all 0.3s ease;
|
||||
}
|
||||
|
||||
.coming-feature:hover {
|
||||
border-color: var(--primary-green);
|
||||
transform: translateY(-2px);
|
||||
box-shadow: 0 8px 24px rgba(15, 187, 170, 0.2);
|
||||
}
|
||||
|
||||
.feature-header {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 1rem;
|
||||
margin-bottom: 1rem;
|
||||
}
|
||||
|
||||
.feature-badge {
|
||||
background: var(--primary-green);
|
||||
color: var(--bg-dark);
|
||||
padding: 0.25rem 0.75rem;
|
||||
border-radius: 20px;
|
||||
font-size: 0.75rem;
|
||||
font-weight: 600;
|
||||
text-transform: uppercase;
|
||||
}
|
||||
|
||||
.coming-feature h3 {
|
||||
font-size: 1.25rem;
|
||||
color: var(--text-primary);
|
||||
margin: 0;
|
||||
}
|
||||
|
||||
.coming-feature p {
|
||||
color: var(--text-secondary);
|
||||
margin-bottom: 1rem;
|
||||
line-height: 1.6;
|
||||
}
|
||||
|
||||
.feature-preview {
|
||||
background: var(--bg-secondary);
|
||||
padding: 1rem;
|
||||
border-radius: 6px;
|
||||
font-family: var(--font-code);
|
||||
font-size: 0.875rem;
|
||||
color: var(--text-accent);
|
||||
border: 1px solid var(--border-color);
|
||||
}
|
||||
|
||||
.stay-tuned {
|
||||
text-align: center;
|
||||
margin-top: 3rem;
|
||||
padding: 2rem;
|
||||
background: var(--bg-tertiary);
|
||||
border-radius: 12px;
|
||||
border: 1px solid var(--border-color);
|
||||
}
|
||||
|
||||
.stay-tuned p {
|
||||
font-size: 1.125rem;
|
||||
color: var(--text-primary);
|
||||
}
|
||||
|
||||
.stay-tuned a {
|
||||
color: var(--primary-green);
|
||||
text-decoration: none;
|
||||
font-weight: 600;
|
||||
}
|
||||
|
||||
.stay-tuned a:hover {
|
||||
text-decoration: underline;
|
||||
}
|
||||
|
||||
/* Footer */
|
||||
.footer {
|
||||
background: var(--bg-secondary);
|
||||
border-top: 1px solid var(--border-color);
|
||||
margin-top: 4rem;
|
||||
padding: 3rem 0 2rem;
|
||||
}
|
||||
|
||||
.footer-content {
|
||||
max-width: 1200px;
|
||||
margin: 0 auto;
|
||||
padding: 0 2rem;
|
||||
display: grid;
|
||||
grid-template-columns: repeat(auto-fit, minmax(250px, 1fr));
|
||||
gap: 3rem;
|
||||
}
|
||||
|
||||
.footer-section h4 {
|
||||
margin-bottom: 1rem;
|
||||
color: var(--text-primary);
|
||||
}
|
||||
|
||||
.footer-section ul {
|
||||
list-style: none;
|
||||
}
|
||||
|
||||
.footer-section li {
|
||||
margin-bottom: 0.5rem;
|
||||
}
|
||||
|
||||
.footer-section a {
|
||||
color: var(--text-secondary);
|
||||
text-decoration: none;
|
||||
transition: color 0.2s ease;
|
||||
}
|
||||
|
||||
.footer-section a:hover {
|
||||
color: var(--primary-green);
|
||||
}
|
||||
|
||||
.footer-bottom {
|
||||
text-align: center;
|
||||
margin-top: 2rem;
|
||||
padding-top: 2rem;
|
||||
border-top: 1px solid var(--border-color);
|
||||
color: var(--text-secondary);
|
||||
}
|
||||
|
||||
/* Responsive */
|
||||
@media (max-width: 768px) {
|
||||
.header-content {
|
||||
flex-direction: column;
|
||||
gap: 1.5rem;
|
||||
}
|
||||
|
||||
.nav-links {
|
||||
gap: 1rem;
|
||||
}
|
||||
|
||||
.features-grid {
|
||||
grid-template-columns: 1fr;
|
||||
}
|
||||
|
||||
.step {
|
||||
flex-direction: column;
|
||||
gap: 1rem;
|
||||
}
|
||||
|
||||
.coming-soon-section h2 {
|
||||
font-size: 1.5rem;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,39 @@
|
||||
// Service worker for Crawl4AI Assistant
|
||||
|
||||
// Handle messages from content script
|
||||
chrome.runtime.onMessage.addListener((message, sender, sendResponse) => {
|
||||
if (message.action === 'downloadCode') {
|
||||
try {
|
||||
// Create a data URL for the Python code
|
||||
const dataUrl = 'data:text/plain;charset=utf-8,' + encodeURIComponent(message.code);
|
||||
|
||||
// Download the file
|
||||
chrome.downloads.download({
|
||||
url: dataUrl,
|
||||
filename: message.filename || 'crawl4ai_schema.py',
|
||||
saveAs: true
|
||||
}, (downloadId) => {
|
||||
if (chrome.runtime.lastError) {
|
||||
console.error('Download failed:', chrome.runtime.lastError);
|
||||
sendResponse({ success: false, error: chrome.runtime.lastError.message });
|
||||
} else {
|
||||
console.log('Download started with ID:', downloadId);
|
||||
sendResponse({ success: true, downloadId: downloadId });
|
||||
}
|
||||
});
|
||||
} catch (error) {
|
||||
console.error('Error creating download:', error);
|
||||
sendResponse({ success: false, error: error.message });
|
||||
}
|
||||
|
||||
return true; // Keep the message channel open for async response
|
||||
}
|
||||
|
||||
return false;
|
||||
});
|
||||
|
||||
// Clean up on extension install/update
|
||||
chrome.runtime.onInstalled.addListener(() => {
|
||||
// Clear any stored state
|
||||
chrome.storage.local.clear();
|
||||
});
|
||||
738
docs/md_v2/apps/crawl4ai-assistant/content/content.js
Normal file
@@ -0,0 +1,738 @@
|
||||
// Content script for Crawl4AI Assistant
|
||||
class SchemaBuilder {
|
||||
constructor() {
|
||||
this.mode = null;
|
||||
this.container = null;
|
||||
this.fields = [];
|
||||
this.overlay = null;
|
||||
this.toolbar = null;
|
||||
this.highlightBox = null;
|
||||
this.selectedElements = new Set();
|
||||
this.isPaused = false;
|
||||
this.codeModal = null;
|
||||
|
||||
this.handleMouseMove = this.handleMouseMove.bind(this);
|
||||
this.handleClick = this.handleClick.bind(this);
|
||||
this.handleKeyPress = this.handleKeyPress.bind(this);
|
||||
}
|
||||
|
||||
start() {
|
||||
this.mode = 'container';
|
||||
this.createOverlay();
|
||||
this.createToolbar();
|
||||
this.attachEventListeners();
|
||||
this.updateToolbar();
|
||||
}
|
||||
|
||||
stop() {
|
||||
this.detachEventListeners();
|
||||
this.overlay?.remove();
|
||||
this.toolbar?.remove();
|
||||
this.highlightBox?.remove();
|
||||
this.removeAllHighlights();
|
||||
this.mode = null;
|
||||
this.container = null;
|
||||
this.fields = [];
|
||||
this.selectedElements.clear();
|
||||
}
|
||||
|
||||
createOverlay() {
|
||||
// Create highlight box
|
||||
this.highlightBox = document.createElement('div');
|
||||
this.highlightBox.className = 'c4ai-highlight-box';
|
||||
document.body.appendChild(this.highlightBox);
|
||||
}
|
||||
|
||||
createToolbar() {
|
||||
this.toolbar = document.createElement('div');
|
||||
this.toolbar.className = 'c4ai-toolbar';
|
||||
this.toolbar.innerHTML = `
|
||||
<div class="c4ai-toolbar-titlebar">
|
||||
<div class="c4ai-titlebar-dots">
|
||||
<button class="c4ai-dot c4ai-dot-close" id="c4ai-close"></button>
|
||||
<button class="c4ai-dot c4ai-dot-minimize"></button>
|
||||
<button class="c4ai-dot c4ai-dot-maximize"></button>
|
||||
</div>
|
||||
<img src="${chrome.runtime.getURL('icons/icon-16.png')}" class="c4ai-titlebar-icon" alt="Crawl4AI">
|
||||
<div class="c4ai-titlebar-title">Crawl4AI Schema Builder</div>
|
||||
</div>
|
||||
<div class="c4ai-toolbar-content">
|
||||
<div class="c4ai-toolbar-status">
|
||||
<div class="c4ai-status-item">
|
||||
<span class="c4ai-status-label">Mode:</span>
|
||||
<span class="c4ai-status-value" id="c4ai-mode">Select Container</span>
|
||||
</div>
|
||||
<div class="c4ai-status-item">
|
||||
<span class="c4ai-status-label">Container:</span>
|
||||
<span class="c4ai-status-value" id="c4ai-container">Not selected</span>
|
||||
</div>
|
||||
</div>
|
||||
<div class="c4ai-fields-list" id="c4ai-fields-list" style="display: none;">
|
||||
<div class="c4ai-fields-header">Selected Fields:</div>
|
||||
<ul class="c4ai-fields-items" id="c4ai-fields-items"></ul>
|
||||
</div>
|
||||
<div class="c4ai-toolbar-hint" id="c4ai-hint">
|
||||
Click on a container element (e.g., product card, article, etc.)
|
||||
</div>
|
||||
<div class="c4ai-toolbar-actions">
|
||||
<button id="c4ai-pause" class="c4ai-action-btn c4ai-pause-btn">
|
||||
<span class="c4ai-pause-icon">⏸</span> Pause
|
||||
</button>
|
||||
<button id="c4ai-generate" class="c4ai-action-btn c4ai-generate-btn">
|
||||
<span class="c4ai-generate-icon">⚡</span> Generate Code
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
`;
|
||||
document.body.appendChild(this.toolbar);
|
||||
|
||||
// Add event listeners for toolbar buttons
|
||||
document.getElementById('c4ai-pause').addEventListener('click', () => this.togglePause());
|
||||
document.getElementById('c4ai-generate').addEventListener('click', () => this.stopAndGenerate());
|
||||
document.getElementById('c4ai-close').addEventListener('click', () => this.stop());
|
||||
|
||||
// Make toolbar draggable
|
||||
this.makeDraggable(this.toolbar);
|
||||
}
|
||||
|
||||
attachEventListeners() {
|
||||
document.addEventListener('mousemove', this.handleMouseMove, true);
|
||||
document.addEventListener('click', this.handleClick, true);
|
||||
document.addEventListener('keydown', this.handleKeyPress, true);
|
||||
}
|
||||
|
||||
detachEventListeners() {
|
||||
document.removeEventListener('mousemove', this.handleMouseMove, true);
|
||||
document.removeEventListener('click', this.handleClick, true);
|
||||
document.removeEventListener('keydown', this.handleKeyPress, true);
|
||||
}
|
||||
|
||||
handleMouseMove(e) {
|
||||
if (this.isPaused) return;
|
||||
|
||||
const element = document.elementFromPoint(e.clientX, e.clientY);
|
||||
if (element && !this.isOurElement(element)) {
|
||||
this.highlightElement(element);
|
||||
}
|
||||
}
|
||||
|
||||
handleClick(e) {
|
||||
if (this.isPaused) return;
|
||||
|
||||
const element = e.target;
|
||||
|
||||
if (this.isOurElement(element)) {
|
||||
return;
|
||||
}
|
||||
|
||||
e.preventDefault();
|
||||
e.stopPropagation();
|
||||
|
||||
if (this.mode === 'container') {
|
||||
this.selectContainer(element);
|
||||
} else if (this.mode === 'field') {
|
||||
this.selectField(element);
|
||||
}
|
||||
}
|
||||
|
||||
handleKeyPress(e) {
|
||||
if (e.key === 'Escape') {
|
||||
this.stop();
|
||||
}
|
||||
}
|
||||
|
||||
isOurElement(element) {
|
||||
return element.classList.contains('c4ai-highlight-box') ||
|
||||
element.classList.contains('c4ai-toolbar') ||
|
||||
element.closest('.c4ai-toolbar') ||
|
||||
element.closest('.c4ai-field-dialog') ||
|
||||
element.closest('.c4ai-code-modal');
|
||||
}
|
||||
|
||||
makeDraggable(element) {
|
||||
let isDragging = false;
|
||||
let startX, startY, initialX, initialY;
|
||||
|
||||
const titlebar = element.querySelector('.c4ai-toolbar-titlebar');
|
||||
|
||||
titlebar.addEventListener('mousedown', (e) => {
|
||||
// Don't drag if clicking on buttons
|
||||
if (e.target.classList.contains('c4ai-dot')) return;
|
||||
|
||||
isDragging = true;
|
||||
startX = e.clientX;
|
||||
startY = e.clientY;
|
||||
|
||||
const rect = element.getBoundingClientRect();
|
||||
initialX = rect.left;
|
||||
initialY = rect.top;
|
||||
|
||||
element.style.transition = 'none';
|
||||
titlebar.style.cursor = 'grabbing';
|
||||
});
|
||||
|
||||
document.addEventListener('mousemove', (e) => {
|
||||
if (!isDragging) return;
|
||||
|
||||
const deltaX = e.clientX - startX;
|
||||
const deltaY = e.clientY - startY;
|
||||
|
||||
element.style.left = `${initialX + deltaX}px`;
|
||||
element.style.top = `${initialY + deltaY}px`;
|
||||
element.style.right = 'auto';
|
||||
});
|
||||
|
||||
document.addEventListener('mouseup', () => {
|
||||
if (isDragging) {
|
||||
isDragging = false;
|
||||
element.style.transition = '';
|
||||
titlebar.style.cursor = 'grab';
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
togglePause() {
|
||||
this.isPaused = !this.isPaused;
|
||||
const pauseBtn = document.getElementById('c4ai-pause');
|
||||
if (this.isPaused) {
|
||||
pauseBtn.innerHTML = '<span class="c4ai-play-icon">▶</span> Resume';
|
||||
pauseBtn.classList.add('c4ai-paused');
|
||||
this.highlightBox.style.display = 'none';
|
||||
} else {
|
||||
pauseBtn.innerHTML = '<span class="c4ai-pause-icon">⏸</span> Pause';
|
||||
pauseBtn.classList.remove('c4ai-paused');
|
||||
}
|
||||
}
|
||||
|
||||
stopAndGenerate() {
|
||||
if (!this.container || this.fields.length === 0) {
|
||||
alert('Please select a container and at least one field before generating code.');
|
||||
return;
|
||||
}
|
||||
|
||||
const code = this.generateCode();
|
||||
this.showCodeModal(code);
|
||||
}
|
||||
|
||||
highlightElement(element) {
|
||||
const rect = element.getBoundingClientRect();
|
||||
this.highlightBox.style.cssText = `
|
||||
left: ${rect.left + window.scrollX}px;
|
||||
top: ${rect.top + window.scrollY}px;
|
||||
width: ${rect.width}px;
|
||||
height: ${rect.height}px;
|
||||
display: block;
|
||||
`;
|
||||
|
||||
if (this.mode === 'container') {
|
||||
this.highlightBox.className = 'c4ai-highlight-box c4ai-container-mode';
|
||||
} else {
|
||||
this.highlightBox.className = 'c4ai-highlight-box c4ai-field-mode';
|
||||
}
|
||||
}
|
||||
|
||||
selectContainer(element) {
|
||||
// Remove previous container highlight
|
||||
if (this.container) {
|
||||
this.container.element.classList.remove('c4ai-selected-container');
|
||||
}
|
||||
|
||||
this.container = {
|
||||
element: element,
|
||||
html: element.outerHTML,
|
||||
selector: this.generateSelector(element),
|
||||
tagName: element.tagName.toLowerCase()
|
||||
};
|
||||
|
||||
element.classList.add('c4ai-selected-container');
|
||||
this.mode = 'field';
|
||||
this.updateToolbar();
|
||||
this.updateStats();
|
||||
}
|
||||
|
||||
selectField(element) {
|
||||
// Don't select the container itself
|
||||
if (element === this.container.element) {
|
||||
return;
|
||||
}
|
||||
|
||||
// Check if already selected - if so, deselect it
|
||||
if (this.selectedElements.has(element)) {
|
||||
this.deselectField(element);
|
||||
return;
|
||||
}
|
||||
|
||||
// Must be inside the container
|
||||
if (!this.container.element.contains(element)) {
|
||||
return;
|
||||
}
|
||||
|
||||
this.showFieldDialog(element);
|
||||
}
|
||||
|
||||
deselectField(element) {
|
||||
// Remove from fields array
|
||||
this.fields = this.fields.filter(f => f.element !== element);
|
||||
|
||||
// Remove from selected elements set
|
||||
this.selectedElements.delete(element);
|
||||
|
||||
// Remove visual selection
|
||||
element.classList.remove('c4ai-selected-field');
|
||||
|
||||
// Update UI
|
||||
this.updateToolbar();
|
||||
this.updateStats();
|
||||
}
|
||||
|
||||
showFieldDialog(element) {
|
||||
const dialog = document.createElement('div');
|
||||
dialog.className = 'c4ai-field-dialog';
|
||||
|
||||
const rect = element.getBoundingClientRect();
|
||||
dialog.style.cssText = `
|
||||
left: ${rect.left + window.scrollX}px;
|
||||
top: ${rect.bottom + window.scrollY + 10}px;
|
||||
`;
|
||||
|
||||
dialog.innerHTML = `
|
||||
<div class="c4ai-field-dialog-content">
|
||||
<h4>Name this field:</h4>
|
||||
<input type="text" id="c4ai-field-name" placeholder="e.g., title, price, description" autofocus>
|
||||
<div class="c4ai-field-preview">
|
||||
<strong>Content:</strong> ${element.textContent.trim().substring(0, 50)}...
|
||||
</div>
|
||||
<div class="c4ai-field-actions">
|
||||
<button id="c4ai-field-save">Save</button>
|
||||
<button id="c4ai-field-cancel">Cancel</button>
|
||||
</div>
|
||||
</div>
|
||||
`;
|
||||
|
||||
document.body.appendChild(dialog);
|
||||
|
||||
const input = dialog.querySelector('#c4ai-field-name');
|
||||
const saveBtn = dialog.querySelector('#c4ai-field-save');
|
||||
const cancelBtn = dialog.querySelector('#c4ai-field-cancel');
|
||||
|
||||
const save = () => {
|
||||
const fieldName = input.value.trim();
|
||||
if (fieldName) {
|
||||
this.fields.push({
|
||||
name: fieldName,
|
||||
value: element.textContent.trim(),
|
||||
element: element,
|
||||
selector: this.generateSelector(element, this.container.element)
|
||||
});
|
||||
|
||||
element.classList.add('c4ai-selected-field');
|
||||
this.selectedElements.add(element);
|
||||
this.updateToolbar();
|
||||
this.updateStats();
|
||||
}
|
||||
dialog.remove();
|
||||
};
|
||||
|
||||
const cancel = () => {
|
||||
dialog.remove();
|
||||
};
|
||||
|
||||
saveBtn.addEventListener('click', save);
|
||||
cancelBtn.addEventListener('click', cancel);
|
||||
input.addEventListener('keypress', (e) => {
|
||||
if (e.key === 'Enter') save();
|
||||
if (e.key === 'Escape') cancel();
|
||||
});
|
||||
|
||||
input.focus();
|
||||
}
|
||||
|
||||
generateSelector(element, context = document) {
|
||||
// Try to generate a robust selector
|
||||
if (element.id) {
|
||||
return `#${CSS.escape(element.id)}`;
|
||||
}
|
||||
|
||||
// Check for data attributes (most stable)
|
||||
const dataAttrs = ['data-testid', 'data-id', 'data-test', 'data-cy'];
|
||||
for (const attr of dataAttrs) {
|
||||
const value = element.getAttribute(attr);
|
||||
if (value) {
|
||||
return `[${attr}="${value}"]`;
|
||||
}
|
||||
}
|
||||
|
||||
// Check for aria-label
|
||||
if (element.getAttribute('aria-label')) {
|
||||
return `[aria-label="${element.getAttribute('aria-label')}"]`;
|
||||
}
|
||||
|
||||
// Try semantic HTML elements with text
|
||||
const tagName = element.tagName.toLowerCase();
|
||||
if (['button', 'a', 'h1', 'h2', 'h3', 'h4', 'h5', 'h6'].includes(tagName)) {
|
||||
const text = element.textContent.trim();
|
||||
if (text && text.length < 50) {
|
||||
// Use tag name with partial text match
|
||||
return `${tagName}`;
|
||||
}
|
||||
}
|
||||
|
||||
// Check for simple, non-utility classes
|
||||
const classes = Array.from(element.classList)
|
||||
.filter(c => !c.startsWith('c4ai-')) // Exclude our classes
|
||||
.filter(c => !c.includes('[') && !c.includes('(') && !c.includes(':')) // Exclude utility classes
|
||||
.filter(c => c.length < 30); // Exclude very long classes
|
||||
|
||||
if (classes.length > 0 && classes.length <= 3) {
|
||||
const selector = classes.map(c => `.${CSS.escape(c)}`).join('');
|
||||
try {
|
||||
if (context.querySelectorAll(selector).length === 1) {
|
||||
return selector;
|
||||
}
|
||||
} catch (e) {
|
||||
// Invalid selector, continue
|
||||
}
|
||||
}
|
||||
|
||||
// Use nth-child with simple parent tag
|
||||
const parent = element.parentElement;
|
||||
if (parent && parent !== context) {
|
||||
const siblings = Array.from(parent.children);
|
||||
const index = siblings.indexOf(element) + 1;
|
||||
// Just use parent tag name to avoid recursion
|
||||
const parentTag = parent.tagName.toLowerCase();
|
||||
return `${parentTag} > ${tagName}:nth-child(${index})`;
|
||||
}
|
||||
|
||||
// Final fallback
|
||||
return tagName;
|
||||
}
|
||||
|
||||
updateToolbar() {
|
||||
document.getElementById('c4ai-mode').textContent =
|
||||
this.mode === 'container' ? 'Select Container' : 'Select Fields';
|
||||
|
||||
document.getElementById('c4ai-container').textContent =
|
||||
this.container ? `${this.container.tagName} ✓` : 'Not selected';
|
||||
|
||||
// Update fields list
|
||||
const fieldsList = document.getElementById('c4ai-fields-list');
|
||||
const fieldsItems = document.getElementById('c4ai-fields-items');
|
||||
|
||||
if (this.fields.length > 0) {
|
||||
fieldsList.style.display = 'block';
|
||||
fieldsItems.innerHTML = this.fields.map(field => `
|
||||
<li class="c4ai-field-item">
|
||||
<span class="c4ai-field-name">${field.name}</span>
|
||||
<span class="c4ai-field-value">${field.value.substring(0, 30)}${field.value.length > 30 ? '...' : ''}</span>
|
||||
</li>
|
||||
`).join('');
|
||||
} else {
|
||||
fieldsList.style.display = 'none';
|
||||
}
|
||||
|
||||
const hint = document.getElementById('c4ai-hint');
|
||||
if (this.mode === 'container') {
|
||||
hint.textContent = 'Click on a container element (e.g., product card, article, etc.)';
|
||||
} else if (this.fields.length === 0) {
|
||||
hint.textContent = 'Click on fields inside the container to extract (title, price, etc.)';
|
||||
} else {
|
||||
hint.innerHTML = `Continue selecting fields or click <strong>Stop & Generate</strong> to finish.`;
|
||||
}
|
||||
}
|
||||
|
||||
updateStats() {
|
||||
chrome.runtime.sendMessage({
|
||||
action: 'updateStats',
|
||||
stats: {
|
||||
container: !!this.container,
|
||||
fields: this.fields.length
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
removeAllHighlights() {
|
||||
document.querySelectorAll('.c4ai-selected-container').forEach(el => {
|
||||
el.classList.remove('c4ai-selected-container');
|
||||
});
|
||||
document.querySelectorAll('.c4ai-selected-field').forEach(el => {
|
||||
el.classList.remove('c4ai-selected-field');
|
||||
});
|
||||
}
|
||||
|
||||
generateCode() {
|
||||
const fieldDescriptions = this.fields.map(f =>
|
||||
`- ${f.name} (example: "${f.value.substring(0, 50)}...")`
|
||||
).join('\n');
|
||||
|
||||
return `#!/usr/bin/env python3
|
||||
"""
|
||||
Generated by Crawl4AI Chrome Extension
|
||||
URL: ${window.location.href}
|
||||
Generated: ${new Date().toISOString()}
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
from pathlib import Path
|
||||
from crawl4ai import AsyncWebCrawler, BrowserConfig, CrawlerRunConfig
|
||||
from crawl4ai.extraction_strategy import JsonCssExtractionStrategy
|
||||
|
||||
# HTML snippet of the selected container element
|
||||
HTML_SNIPPET = """
|
||||
${this.container.html}
|
||||
"""
|
||||
|
||||
# Extraction query based on your field selections
|
||||
EXTRACTION_QUERY = """
|
||||
Create a JSON CSS extraction schema to extract the following fields:
|
||||
${fieldDescriptions}
|
||||
|
||||
The schema should handle multiple ${this.container.tagName} elements on the page.
|
||||
Each item should be extracted as a separate object in the results array.
|
||||
"""
|
||||
|
||||
async def generate_schema():
|
||||
"""Generate extraction schema using LLM"""
|
||||
print("🔧 Generating extraction schema...")
|
||||
|
||||
try:
|
||||
# Generate the schema using Crawl4AI's built-in LLM integration
|
||||
schema = JsonCssExtractionStrategy.generate_schema(
|
||||
html=HTML_SNIPPET,
|
||||
query=EXTRACTION_QUERY,
|
||||
)
|
||||
|
||||
# Save the schema for reuse
|
||||
schema_path = Path('generated_schema.json')
|
||||
with open(schema_path, 'w') as f:
|
||||
json.dump(schema, f, indent=2)
|
||||
|
||||
print("✅ Schema generated successfully!")
|
||||
print(f"📄 Schema saved to: {schema_path}")
|
||||
print("\\nGenerated schema:")
|
||||
print(json.dumps(schema, indent=2))
|
||||
|
||||
return schema
|
||||
|
||||
except Exception as e:
|
||||
print(f"❌ Error generating schema: {e}")
|
||||
return None
|
||||
|
||||
async def test_extraction(url: str = "${window.location.href}"):
|
||||
"""Test the generated schema on the actual webpage"""
|
||||
print("\\n🧪 Testing extraction on live webpage...")
|
||||
|
||||
# Load the generated schema
|
||||
try:
|
||||
with open('generated_schema.json', 'r') as f:
|
||||
schema = json.load(f)
|
||||
except FileNotFoundError:
|
||||
print("❌ Schema file not found. Run generate_schema() first.")
|
||||
return
|
||||
|
||||
# Configure browser
|
||||
browser_config = BrowserConfig(
|
||||
headless=True,
|
||||
verbose=False
|
||||
)
|
||||
|
||||
# Configure extraction
|
||||
crawler_config = CrawlerRunConfig(
|
||||
extraction_strategy=JsonCssExtractionStrategy(schema=schema)
|
||||
)
|
||||
|
||||
async with AsyncWebCrawler(config=browser_config) as crawler:
|
||||
result = await crawler.arun(
|
||||
url=url,
|
||||
config=crawler_config
|
||||
)
|
||||
|
||||
if result.success and result.extracted_content:
|
||||
data = json.loads(result.extracted_content)
|
||||
print(f"\\n✅ Successfully extracted {len(data)} items!")
|
||||
|
||||
# Save results
|
||||
with open('extracted_data.json', 'w') as f:
|
||||
json.dump(data, f, indent=2)
|
||||
|
||||
# Show sample results
|
||||
print("\\n📊 Sample results (first 2 items):")
|
||||
for i, item in enumerate(data[:2], 1):
|
||||
print(f"\\nItem {i}:")
|
||||
for key, value in item.items():
|
||||
print(f" {key}: {value}")
|
||||
else:
|
||||
print("❌ Extraction failed:", result.error_message)
|
||||
|
||||
if __name__ == "__main__":
|
||||
# Step 1: Generate the schema from HTML snippet
|
||||
asyncio.run(generate_schema())
|
||||
|
||||
# Step 2: Test extraction on the live webpage
|
||||
# Uncomment the line below to test extraction:
|
||||
# asyncio.run(test_extraction())
|
||||
|
||||
print("\\n🎯 Next steps:")
|
||||
print("1. Review the generated schema in 'generated_schema.json'")
|
||||
print("2. Uncomment the test_extraction() line to test on the live site")
|
||||
print("3. Use the schema in your Crawl4AI projects!")
|
||||
`;
|
||||
|
||||
return code;
|
||||
}
|
||||
|
||||
showCodeModal(code) {
|
||||
// Create modal
|
||||
this.codeModal = document.createElement('div');
|
||||
this.codeModal.className = 'c4ai-code-modal';
|
||||
this.codeModal.innerHTML = `
|
||||
<div class="c4ai-code-modal-content">
|
||||
<div class="c4ai-code-modal-header">
|
||||
<h2>Generated Python Code</h2>
|
||||
<button class="c4ai-close-modal" id="c4ai-close-modal">✕</button>
|
||||
</div>
|
||||
<div class="c4ai-code-modal-body">
|
||||
<pre class="c4ai-code-block"><code class="language-python">${this.escapeHtml(code)}</code></pre>
|
||||
</div>
|
||||
<div class="c4ai-code-modal-footer">
|
||||
<button class="c4ai-action-btn c4ai-cloud-btn" id="c4ai-run-cloud" disabled>
|
||||
<span>☁️</span> Run on C4AI Cloud (Coming Soon)
|
||||
</button>
|
||||
<button class="c4ai-action-btn c4ai-download-btn" id="c4ai-download-code">
|
||||
<span>⬇</span> Download Code
|
||||
</button>
|
||||
<button class="c4ai-action-btn c4ai-copy-btn" id="c4ai-copy-code">
|
||||
<span>📋</span> Copy to Clipboard
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
`;
|
||||
|
||||
document.body.appendChild(this.codeModal);
|
||||
|
||||
// Add event listeners
|
||||
document.getElementById('c4ai-close-modal').addEventListener('click', () => {
|
||||
this.codeModal.remove();
|
||||
this.codeModal = null;
|
||||
// Don't stop the capture session
|
||||
});
|
||||
|
||||
document.getElementById('c4ai-download-code').addEventListener('click', () => {
|
||||
chrome.runtime.sendMessage({
|
||||
action: 'downloadCode',
|
||||
code: code,
|
||||
filename: `crawl4ai_schema_${Date.now()}.py`
|
||||
}, (response) => {
|
||||
if (response && response.success) {
|
||||
const btn = document.getElementById('c4ai-download-code');
|
||||
const originalHTML = btn.innerHTML;
|
||||
btn.innerHTML = '<span>✓</span> Downloaded!';
|
||||
setTimeout(() => {
|
||||
btn.innerHTML = originalHTML;
|
||||
}, 2000);
|
||||
} else {
|
||||
console.error('Download failed:', response?.error);
|
||||
alert('Download failed. Please check your browser settings.');
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
document.getElementById('c4ai-copy-code').addEventListener('click', () => {
|
||||
navigator.clipboard.writeText(code).then(() => {
|
||||
const btn = document.getElementById('c4ai-copy-code');
|
||||
btn.innerHTML = '<span>✓</span> Copied!';
|
||||
setTimeout(() => {
|
||||
btn.innerHTML = '<span>📋</span> Copy to Clipboard';
|
||||
}, 2000);
|
||||
});
|
||||
});
|
||||
|
||||
// Apply syntax highlighting if possible
|
||||
this.applySyntaxHighlighting();
|
||||
}
|
||||
|
||||
escapeHtml(text) {
|
||||
const div = document.createElement('div');
|
||||
div.textContent = text;
|
||||
return div.innerHTML;
|
||||
}
|
||||
|
||||
applySyntaxHighlighting() {
|
||||
// Simple Python syntax highlighting - using a different approach
|
||||
const codeElement = this.codeModal.querySelector('.language-python');
|
||||
const code = codeElement.textContent;
|
||||
|
||||
// Split by lines to handle line-by-line
|
||||
const lines = code.split('\n');
|
||||
const highlightedLines = lines.map(line => {
|
||||
let highlightedLine = this.escapeHtml(line);
|
||||
|
||||
// Skip if line is empty
|
||||
if (!highlightedLine.trim()) return highlightedLine;
|
||||
|
||||
// Comments (lines starting with #)
|
||||
if (highlightedLine.trim().startsWith('#')) {
|
||||
return `<span class="c4ai-comment">${highlightedLine}</span>`;
|
||||
}
|
||||
|
||||
// Triple quoted strings
|
||||
if (highlightedLine.includes('"""')) {
|
||||
highlightedLine = highlightedLine.replace(/(""".*?""")/g, '<span class="c4ai-string">$1</span>');
|
||||
}
|
||||
|
||||
// Regular strings - single and double quotes
|
||||
highlightedLine = highlightedLine.replace(/(["'])([^"']*)\1/g, '<span class="c4ai-string">$1$2$1</span>');
|
||||
|
||||
// Keywords - only highlight if not inside a string
|
||||
const keywords = ['import', 'from', 'async', 'def', 'await', 'try', 'except', 'with', 'as', 'for', 'if', 'else', 'elif', 'return', 'print', 'open', 'and', 'or', 'not', 'in', 'is', 'class', 'self', 'None', 'True', 'False', '__name__', '__main__'];
|
||||
|
||||
keywords.forEach(keyword => {
|
||||
// Use word boundaries and lookahead/lookbehind to ensure we're not in a string
|
||||
const regex = new RegExp(`\\b(${keyword})\\b(?![^<]*</span>)`, 'g');
|
||||
highlightedLine = highlightedLine.replace(regex, '<span class="c4ai-keyword">$1</span>');
|
||||
});
|
||||
|
||||
// Functions (word followed by parenthesis)
|
||||
highlightedLine = highlightedLine.replace(/\b([a-zA-Z_]\w*)\s*\(/g, '<span class="c4ai-function">$1</span>(');
|
||||
|
||||
return highlightedLine;
|
||||
});
|
||||
|
||||
codeElement.innerHTML = highlightedLines.join('\n');
|
||||
}
|
||||
}
|
||||
|
||||
// Initialize
|
||||
let schemaBuilder = null;
|
||||
|
||||
// Listen for messages from popup
|
||||
chrome.runtime.onMessage.addListener((message, sender, sendResponse) => {
|
||||
switch (message.action) {
|
||||
case 'startSchemaCapture':
|
||||
if (!schemaBuilder) {
|
||||
schemaBuilder = new SchemaBuilder();
|
||||
}
|
||||
schemaBuilder.start();
|
||||
sendResponse({ success: true });
|
||||
break;
|
||||
|
||||
case 'stopCapture':
|
||||
if (schemaBuilder) {
|
||||
schemaBuilder.stop();
|
||||
schemaBuilder = null;
|
||||
}
|
||||
sendResponse({ success: true });
|
||||
break;
|
||||
|
||||
case 'generateCode':
|
||||
if (schemaBuilder) {
|
||||
const code = schemaBuilder.generateCode();
|
||||
schemaBuilder.showCodeModal(code);
|
||||
}
|
||||
sendResponse({ success: true });
|
||||
break;
|
||||
}
|
||||
|
||||
return true;
|
||||
});
|
||||
561
docs/md_v2/apps/crawl4ai-assistant/content/overlay.css
Normal file
@@ -0,0 +1,561 @@
|
||||
/* Crawl4AI Assistant Overlay Styles */
|
||||
|
||||
/* Font Face Definitions */
|
||||
@font-face {
|
||||
font-family: 'Dank Mono';
|
||||
src: url('../assets/DankMono-Regular.woff2') format('woff2');
|
||||
font-weight: 400;
|
||||
font-style: normal;
|
||||
font-display: swap;
|
||||
}
|
||||
|
||||
@font-face {
|
||||
font-family: 'Dank Mono';
|
||||
src: url('../assets/DankMono-Bold.woff2') format('woff2');
|
||||
font-weight: 700;
|
||||
font-style: normal;
|
||||
font-display: swap;
|
||||
}
|
||||
|
||||
@font-face {
|
||||
font-family: 'Dank Mono';
|
||||
src: url('../assets/DankMono-Italic.woff2') format('woff2');
|
||||
font-weight: 400;
|
||||
font-style: italic;
|
||||
font-display: swap;
|
||||
}
|
||||
|
||||
:root {
|
||||
--font-primary: 'Dank Mono', -apple-system, BlinkMacSystemFont, sans-serif;
|
||||
--font-code: 'Dank Mono', 'Monaco', 'Menlo', 'Consolas', monospace;
|
||||
}
|
||||
|
||||
/* Highlight box for hovering */
|
||||
.c4ai-highlight-box {
|
||||
position: absolute;
|
||||
pointer-events: none;
|
||||
z-index: 999999;
|
||||
transition: all 0.1s ease;
|
||||
display: none;
|
||||
}
|
||||
|
||||
.c4ai-highlight-box.c4ai-container-mode {
|
||||
border: 3px dashed #0fbbaa;
|
||||
background: rgba(15, 187, 170, 0.1);
|
||||
box-shadow: 0 0 0 2px rgba(15, 187, 170, 0.3);
|
||||
}
|
||||
|
||||
.c4ai-highlight-box.c4ai-field-mode {
|
||||
border: 2px dashed #f380f5;
|
||||
background: rgba(243, 128, 245, 0.1);
|
||||
box-shadow: 0 0 0 2px rgba(243, 128, 245, 0.3);
|
||||
}
|
||||
|
||||
/* Selected elements */
|
||||
.c4ai-selected-container {
|
||||
outline: 3px solid #0fbbaa !important;
|
||||
outline-offset: 2px;
|
||||
background: rgba(15, 187, 170, 0.05) !important;
|
||||
}
|
||||
|
||||
.c4ai-selected-field {
|
||||
outline: 2px solid #f380f5 !important;
|
||||
outline-offset: 1px;
|
||||
background: rgba(243, 128, 245, 0.1) !important;
|
||||
position: relative;
|
||||
}
|
||||
|
||||
.c4ai-selected-field::after {
|
||||
content: attr(data-c4ai-field);
|
||||
position: absolute;
|
||||
top: -24px;
|
||||
left: 0;
|
||||
background: #f380f5;
|
||||
color: #000;
|
||||
padding: 2px 8px;
|
||||
border-radius: 4px;
|
||||
font-size: 12px;
|
||||
font-weight: bold;
|
||||
font-family: var(--font-primary);
|
||||
z-index: 999999;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
/* Toolbar */
|
||||
.c4ai-toolbar {
|
||||
position: fixed;
|
||||
top: 20px;
|
||||
right: 20px;
|
||||
background: #070708;
|
||||
border: 1px solid #3f3f44;
|
||||
border-radius: 8px;
|
||||
z-index: 999999;
|
||||
font-family: var(--font-primary);
|
||||
box-shadow: 0 8px 32px rgba(0, 0, 0, 0.8);
|
||||
width: 320px;
|
||||
color: #e8e9ed;
|
||||
overflow: hidden;
|
||||
}
|
||||
|
||||
.c4ai-toolbar * {
|
||||
font-family: inherit;
|
||||
box-sizing: border-box;
|
||||
}
|
||||
|
||||
/* macOS-style titlebar */
|
||||
.c4ai-toolbar-titlebar {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
padding: 12px;
|
||||
background: #1a1a1a;
|
||||
border-bottom: 1px solid #3f3f44;
|
||||
cursor: grab;
|
||||
user-select: none;
|
||||
}
|
||||
|
||||
.c4ai-titlebar-dots {
|
||||
display: flex;
|
||||
gap: 8px;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
|
||||
.c4ai-dot {
|
||||
width: 12px;
|
||||
height: 12px;
|
||||
border-radius: 50%;
|
||||
border: none;
|
||||
cursor: pointer;
|
||||
transition: opacity 0.2s ease;
|
||||
padding: 0;
|
||||
}
|
||||
|
||||
.c4ai-dot-close {
|
||||
background: #ff5f57;
|
||||
}
|
||||
|
||||
.c4ai-dot-close:hover {
|
||||
background: #ff3030;
|
||||
}
|
||||
|
||||
.c4ai-dot-minimize {
|
||||
background: #ffbd2e;
|
||||
}
|
||||
|
||||
.c4ai-dot-minimize:hover {
|
||||
background: #ffaa00;
|
||||
}
|
||||
|
||||
.c4ai-dot-maximize {
|
||||
background: #28ca42;
|
||||
}
|
||||
|
||||
.c4ai-dot-maximize:hover {
|
||||
background: #1eb533;
|
||||
}
|
||||
|
||||
.c4ai-titlebar-icon {
|
||||
width: 16px;
|
||||
height: 16px;
|
||||
margin-left: 8px;
|
||||
margin-right: 8px;
|
||||
}
|
||||
|
||||
.c4ai-titlebar-title {
|
||||
flex: 1;
|
||||
text-align: left;
|
||||
font-size: 13px;
|
||||
font-weight: 600;
|
||||
color: #e8e9ed;
|
||||
}
|
||||
|
||||
.c4ai-toolbar-content {
|
||||
padding: 16px;
|
||||
}
|
||||
|
||||
.c4ai-toolbar-status {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 8px;
|
||||
margin-bottom: 16px;
|
||||
padding: 12px;
|
||||
background: #3f3f44;
|
||||
border-radius: 8px;
|
||||
}
|
||||
|
||||
.c4ai-status-item {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
.c4ai-status-label {
|
||||
font-size: 12px;
|
||||
color: #a3abba;
|
||||
text-transform: uppercase;
|
||||
}
|
||||
|
||||
.c4ai-status-value {
|
||||
font-size: 14px;
|
||||
font-weight: 600;
|
||||
color: #e8e9ed;
|
||||
}
|
||||
|
||||
.c4ai-toolbar-hint {
|
||||
margin-top: 16px;
|
||||
font-size: 13px;
|
||||
line-height: 1.4;
|
||||
color: #d5cec0;
|
||||
padding: 12px;
|
||||
background: #3f3f44;
|
||||
border-radius: 8px;
|
||||
border-left: 3px solid #0fbbaa;
|
||||
}
|
||||
|
||||
.c4ai-toolbar-hint strong {
|
||||
color: #0fbbaa;
|
||||
font-weight: 600;
|
||||
}
|
||||
|
||||
/* Fields list */
|
||||
.c4ai-fields-list {
|
||||
margin-top: 16px;
|
||||
padding: 12px;
|
||||
background: #3f3f44;
|
||||
border-radius: 8px;
|
||||
border: 1px solid #3f3f44;
|
||||
}
|
||||
|
||||
.c4ai-fields-header {
|
||||
font-size: 12px;
|
||||
text-transform: uppercase;
|
||||
color: #a3abba;
|
||||
margin-bottom: 8px;
|
||||
font-weight: 600;
|
||||
}
|
||||
|
||||
.c4ai-fields-items {
|
||||
list-style: none;
|
||||
margin: 0;
|
||||
padding: 0;
|
||||
}
|
||||
|
||||
.c4ai-field-item {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
padding: 6px 8px;
|
||||
margin-bottom: 4px;
|
||||
background: #070708;
|
||||
border-radius: 4px;
|
||||
font-size: 12px;
|
||||
}
|
||||
|
||||
.c4ai-field-name {
|
||||
font-weight: 600;
|
||||
color: #f380f5;
|
||||
margin-right: 8px;
|
||||
}
|
||||
|
||||
.c4ai-field-value {
|
||||
color: #a3abba;
|
||||
font-size: 11px;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
white-space: nowrap;
|
||||
flex: 1;
|
||||
text-align: right;
|
||||
}
|
||||
|
||||
/* Field naming dialog */
|
||||
.c4ai-field-dialog {
|
||||
position: absolute;
|
||||
background: #070708;
|
||||
border: 2px solid #f380f5;
|
||||
border-radius: 12px;
|
||||
padding: 16px;
|
||||
z-index: 999999;
|
||||
box-shadow: 0 8px 32px rgba(0, 0, 0, 0.8);
|
||||
font-family: -apple-system, BlinkMacSystemFont, sans-serif;
|
||||
color: #e8e9ed;
|
||||
min-width: 300px;
|
||||
}
|
||||
|
||||
.c4ai-field-dialog h4 {
|
||||
margin: 0 0 12px 0;
|
||||
font-size: 14px;
|
||||
color: #e8e9ed;
|
||||
}
|
||||
|
||||
.c4ai-field-dialog input {
|
||||
width: 100%;
|
||||
padding: 8px 12px;
|
||||
background: #3f3f44;
|
||||
border: 1px solid #3f3f44;
|
||||
border-radius: 6px;
|
||||
color: #e8e9ed;
|
||||
font-size: 14px;
|
||||
margin-bottom: 12px;
|
||||
outline: none;
|
||||
}
|
||||
|
||||
.c4ai-field-dialog input:focus {
|
||||
border-color: #f380f5;
|
||||
box-shadow: 0 0 0 2px rgba(243, 128, 245, 0.2);
|
||||
}
|
||||
|
||||
.c4ai-field-preview {
|
||||
font-size: 12px;
|
||||
color: #999;
|
||||
margin-bottom: 12px;
|
||||
padding: 8px;
|
||||
background: #1a1a1a;
|
||||
border-radius: 6px;
|
||||
word-break: break-word;
|
||||
}
|
||||
|
||||
.c4ai-field-preview strong {
|
||||
color: #ccc;
|
||||
}
|
||||
|
||||
.c4ai-field-actions {
|
||||
display: flex;
|
||||
gap: 8px;
|
||||
justify-content: flex-end;
|
||||
}
|
||||
|
||||
.c4ai-field-actions button {
|
||||
padding: 6px 16px;
|
||||
border: none;
|
||||
border-radius: 6px;
|
||||
font-size: 13px;
|
||||
font-weight: 600;
|
||||
cursor: pointer;
|
||||
transition: all 0.2s ease;
|
||||
}
|
||||
|
||||
#c4ai-field-save {
|
||||
background: #f380f5;
|
||||
color: #070708;
|
||||
}
|
||||
|
||||
#c4ai-field-save:hover {
|
||||
background: #e370e5;
|
||||
transform: translateY(-1px);
|
||||
}
|
||||
|
||||
#c4ai-field-cancel {
|
||||
background: #3f3f44;
|
||||
color: #e8e9ed;
|
||||
}
|
||||
|
||||
#c4ai-field-cancel:hover {
|
||||
background: #4f4f54;
|
||||
}
|
||||
|
||||
/* Toolbar action buttons */
|
||||
.c4ai-toolbar-actions {
|
||||
display: flex;
|
||||
gap: 8px;
|
||||
margin-top: 16px;
|
||||
padding-top: 16px;
|
||||
border-top: 1px solid #2a2a2a;
|
||||
}
|
||||
|
||||
.c4ai-action-btn {
|
||||
flex: 1;
|
||||
padding: 8px 16px;
|
||||
background: #3f3f44;
|
||||
border: 1px solid #3f3f44;
|
||||
border-radius: 6px;
|
||||
color: #e8e9ed;
|
||||
font-size: 13px;
|
||||
font-weight: 600;
|
||||
cursor: pointer;
|
||||
transition: all 0.2s ease;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
gap: 6px;
|
||||
}
|
||||
|
||||
.c4ai-action-btn:hover {
|
||||
background: #4f4f54;
|
||||
border-color: #5f5f64;
|
||||
transform: translateY(-1px);
|
||||
}
|
||||
|
||||
.c4ai-pause-btn {
|
||||
background: #09b5a5;
|
||||
border-color: #09b5a5;
|
||||
color: #070708;
|
||||
}
|
||||
|
||||
.c4ai-pause-btn:hover {
|
||||
background: #0ac5b5;
|
||||
}
|
||||
|
||||
.c4ai-pause-btn.c4ai-paused {
|
||||
background: #ff3c74;
|
||||
border-color: #ff3c74;
|
||||
}
|
||||
|
||||
.c4ai-generate-btn {
|
||||
background: #0fbbaa;
|
||||
border-color: #0fbbaa;
|
||||
color: #070708;
|
||||
}
|
||||
|
||||
.c4ai-generate-btn:hover {
|
||||
background: #1fcbba;
|
||||
}
|
||||
|
||||
/* Code modal */
|
||||
.c4ai-code-modal {
|
||||
position: fixed;
|
||||
top: 0;
|
||||
left: 0;
|
||||
right: 0;
|
||||
bottom: 0;
|
||||
background: rgba(0, 0, 0, 0.8);
|
||||
z-index: 999999;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
padding: 20px;
|
||||
}
|
||||
|
||||
.c4ai-code-modal-content {
|
||||
background: #070708;
|
||||
border: 2px solid #3f3f44;
|
||||
border-radius: 16px;
|
||||
width: 90%;
|
||||
max-width: 900px;
|
||||
max-height: 90vh;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
box-shadow: 0 16px 64px rgba(0, 0, 0, 0.9);
|
||||
}
|
||||
|
||||
.c4ai-code-modal-header {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
padding: 20px 24px;
|
||||
border-bottom: 1px solid #2a2a2a;
|
||||
}
|
||||
|
||||
.c4ai-code-modal-header h2 {
|
||||
margin: 0;
|
||||
font-size: 20px;
|
||||
color: #fff;
|
||||
font-family: -apple-system, BlinkMacSystemFont, sans-serif;
|
||||
}
|
||||
|
||||
.c4ai-close-modal {
|
||||
background: none;
|
||||
border: none;
|
||||
color: #999;
|
||||
font-size: 24px;
|
||||
cursor: pointer;
|
||||
padding: 0;
|
||||
width: 32px;
|
||||
height: 32px;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
border-radius: 4px;
|
||||
transition: all 0.2s ease;
|
||||
}
|
||||
|
||||
.c4ai-close-modal:hover {
|
||||
background: #2a2a2a;
|
||||
color: #fff;
|
||||
}
|
||||
|
||||
.c4ai-code-modal-body {
|
||||
flex: 1;
|
||||
overflow: auto;
|
||||
padding: 24px;
|
||||
}
|
||||
|
||||
.c4ai-code-block {
|
||||
background: #3f3f44;
|
||||
border: 1px solid #3f3f44;
|
||||
border-radius: 8px;
|
||||
padding: 20px;
|
||||
overflow-x: auto;
|
||||
margin: 0;
|
||||
}
|
||||
|
||||
.c4ai-code-block code {
|
||||
font-family: var(--font-code);
|
||||
font-size: 13px;
|
||||
line-height: 1.6;
|
||||
color: #e8e9ed;
|
||||
white-space: pre;
|
||||
display: block;
|
||||
}
|
||||
|
||||
/* Syntax highlighting */
|
||||
.c4ai-keyword {
|
||||
color: #0fbbaa;
|
||||
font-weight: bold;
|
||||
}
|
||||
|
||||
.c4ai-string {
|
||||
color: #f380f5;
|
||||
}
|
||||
|
||||
.c4ai-comment {
|
||||
color: #a3abba;
|
||||
font-style: italic;
|
||||
}
|
||||
|
||||
.c4ai-function {
|
||||
color: #ff3c74;
|
||||
}
|
||||
|
||||
.c4ai-code-modal-footer {
|
||||
display: flex;
|
||||
gap: 12px;
|
||||
padding: 20px 24px;
|
||||
border-top: 1px solid #2a2a2a;
|
||||
justify-content: flex-end;
|
||||
}
|
||||
|
||||
.c4ai-download-btn {
|
||||
background: #0fbbaa;
|
||||
color: #070708;
|
||||
border: none;
|
||||
}
|
||||
|
||||
.c4ai-download-btn:hover {
|
||||
background: #1fcbba;
|
||||
}
|
||||
|
||||
.c4ai-copy-btn {
|
||||
background: #3f3f44;
|
||||
border-color: #3f3f44;
|
||||
}
|
||||
|
||||
.c4ai-copy-btn:hover {
|
||||
background: #4f4f54;
|
||||
border-color: #5f5f64;
|
||||
}
|
||||
|
||||
.c4ai-cloud-btn {
|
||||
background: #3f3f44;
|
||||
border-color: #3f3f44;
|
||||
opacity: 0.6;
|
||||
cursor: not-allowed;
|
||||
}
|
||||
|
||||
.c4ai-cloud-btn:disabled {
|
||||
opacity: 0.6;
|
||||
cursor: not-allowed;
|
||||
}
|
||||
|
||||
.c4ai-cloud-btn:hover:disabled {
|
||||
background: #3f3f44;
|
||||
transform: none;
|
||||
}
|
||||
BIN
docs/md_v2/apps/crawl4ai-assistant/crawl4ai-assistant-v1.0.0.zip
Normal file
BIN
docs/md_v2/apps/crawl4ai-assistant/crawl4ai-assistant-v1.0.1.zip
Normal file
BIN
docs/md_v2/apps/crawl4ai-assistant/icons/favicon.ico
Normal file
|
After Width: | Height: | Size: 3.4 KiB |
BIN
docs/md_v2/apps/crawl4ai-assistant/icons/icon-128.png
Normal file
|
After Width: | Height: | Size: 1.6 KiB |
BIN
docs/md_v2/apps/crawl4ai-assistant/icons/icon-16.png
Normal file
|
After Width: | Height: | Size: 1.6 KiB |
BIN
docs/md_v2/apps/crawl4ai-assistant/icons/icon-48.png
Normal file
|
After Width: | Height: | Size: 1.6 KiB |
328
docs/md_v2/apps/crawl4ai-assistant/index.html
Normal file
@@ -0,0 +1,328 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>Crawl4AI Assistant - Chrome Extension for Visual Web Scraping</title>
|
||||
<link rel="stylesheet" href="assistant.css">
|
||||
</head>
|
||||
<body>
|
||||
<div class="terminal-container">
|
||||
<div class="header">
|
||||
<div class="header-content">
|
||||
<div class="logo-section">
|
||||
<img src="../../img/favicon-32x32.png" alt="Crawl4AI Logo" class="logo">
|
||||
<div>
|
||||
<h1>Crawl4AI Assistant</h1>
|
||||
<p class="tagline">Chrome Extension for Visual Web Scraping</p>
|
||||
</div>
|
||||
</div>
|
||||
<nav class="nav-links">
|
||||
<a href="../../" class="nav-link">← Back to Docs</a>
|
||||
<a href="../" class="nav-link">All Apps</a>
|
||||
<a href="https://github.com/unclecode/crawl4ai" class="nav-link" target="_blank">GitHub</a>
|
||||
</nav>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="content">
|
||||
<!-- Video Section -->
|
||||
<section class="video-section">
|
||||
<div class="video-wrapper">
|
||||
<video autoplay loop muted playsinline class="demo-video">
|
||||
<source src="demo.mp4" type="video/mp4">
|
||||
Your browser does not support the video tag.
|
||||
</video>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<!-- Introduction -->
|
||||
<section class="intro-section">
|
||||
<div class="terminal-window">
|
||||
<div class="terminal-header">
|
||||
<span class="terminal-title">About Crawl4AI Assistant</span>
|
||||
</div>
|
||||
<div class="terminal-content">
|
||||
<p>Transform any website into structured data with just a few clicks! The Crawl4AI Assistant Chrome Extension lets you visually select elements on any webpage and automatically generates Python code for web scraping.</p>
|
||||
|
||||
<div class="features-grid">
|
||||
<div class="feature-card">
|
||||
<span class="feature-icon">🎯</span>
|
||||
<h3>Visual Selection</h3>
|
||||
<p>Click on any element to select it - no CSS selectors needed</p>
|
||||
</div>
|
||||
<div class="feature-card">
|
||||
<span class="feature-icon">📊</span>
|
||||
<h3>Schema Builder</h3>
|
||||
<p>Build extraction schemas by clicking on container and field elements</p>
|
||||
</div>
|
||||
<div class="feature-card">
|
||||
<span class="feature-icon">🐍</span>
|
||||
<h3>Python Code</h3>
|
||||
<p>Get production-ready Crawl4AI code with LLM extraction</p>
|
||||
</div>
|
||||
<div class="feature-card">
|
||||
<span class="feature-icon">🎨</span>
|
||||
<h3>Beautiful UI</h3>
|
||||
<p>Draggable toolbar with macOS-style interface</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<!-- Quick Start -->
|
||||
<section class="quickstart-section">
|
||||
<h2>Quick Start</h2>
|
||||
<div class="terminal-window">
|
||||
<div class="terminal-header">
|
||||
<span class="terminal-title">Installation</span>
|
||||
</div>
|
||||
<div class="terminal-content">
|
||||
<div class="installation-steps">
|
||||
<div class="step">
|
||||
<span class="step-number">1</span>
|
||||
<div class="step-content">
|
||||
<h4>Download the Extension</h4>
|
||||
<p>Get the latest release from GitHub or use the button below</p>
|
||||
<a href="crawl4ai-assistant-v1.0.1.zip" class="download-button" download>
|
||||
<span class="button-icon">↓</span>
|
||||
Download Extension (v1.0.1)
|
||||
</a>
|
||||
</div>
|
||||
</div>
|
||||
<div class="step">
|
||||
<span class="step-number">2</span>
|
||||
<div class="step-content">
|
||||
<h4>Load in Chrome</h4>
|
||||
<p>Navigate to <code>chrome://extensions/</code> and enable Developer Mode</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="step">
|
||||
<span class="step-number">3</span>
|
||||
<div class="step-content">
|
||||
<h4>Load Unpacked</h4>
|
||||
<p>Click "Load unpacked" and select the extracted extension folder</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<!-- Usage Guide -->
|
||||
<section class="usage-section">
|
||||
<h2>How to Use</h2>
|
||||
<div class="terminal-window">
|
||||
<div class="terminal-header">
|
||||
<span class="terminal-title">Step-by-Step Guide</span>
|
||||
</div>
|
||||
<div class="terminal-content">
|
||||
<div class="usage-flow">
|
||||
<div class="usage-step">
|
||||
<div class="usage-header">
|
||||
<span class="usage-icon">1️⃣</span>
|
||||
<h4>Start Schema Builder</h4>
|
||||
</div>
|
||||
<p>Click the extension icon and select "Schema Builder" to begin</p>
|
||||
</div>
|
||||
|
||||
<div class="usage-step">
|
||||
<div class="usage-header">
|
||||
<span class="usage-icon">2️⃣</span>
|
||||
<h4>Select Container</h4>
|
||||
</div>
|
||||
<p>Click on a container element (e.g., product card, article, listing)</p>
|
||||
<div class="code-snippet">
|
||||
<span class="comment"># Container will be highlighted in green</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="usage-step">
|
||||
<div class="usage-header">
|
||||
<span class="usage-icon">3️⃣</span>
|
||||
<h4>Select Fields</h4>
|
||||
</div>
|
||||
<p>Click on individual fields inside the container and name them</p>
|
||||
<div class="code-snippet">
|
||||
<span class="comment"># Fields will be highlighted in pink</span>
|
||||
<span class="comment"># Examples: title, price, description, image</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="usage-step">
|
||||
<div class="usage-header">
|
||||
<span class="usage-icon">4️⃣</span>
|
||||
<h4>Generate Code</h4>
|
||||
</div>
|
||||
<p>Click "Stop & Generate" to create your Python extraction code</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<!-- Generated Code Example -->
|
||||
<section class="code-section">
|
||||
<h2>Generated Code Example</h2>
|
||||
<div class="terminal-window">
|
||||
<div class="terminal-header">
|
||||
<span class="terminal-title">example_extraction.py</span>
|
||||
</div>
|
||||
<div class="terminal-content">
|
||||
<pre><code><span class="keyword">import</span> asyncio
|
||||
<span class="keyword">import</span> json
|
||||
<span class="keyword">from</span> crawl4ai <span class="keyword">import</span> AsyncWebCrawler, CrawlerRunConfig
|
||||
<span class="keyword">from</span> crawl4ai.extraction_strategy <span class="keyword">import</span> JsonCssExtractionStrategy
|
||||
|
||||
<span class="keyword">async</span> <span class="keyword">def</span> <span class="function">extract_products</span>():
|
||||
<span class="comment"># Schema generated from your visual selection</span>
|
||||
schema = {
|
||||
<span class="string">"name"</span>: <span class="string">"Product Catalog"</span>,
|
||||
<span class="string">"baseSelector"</span>: <span class="string">"div.product-card"</span>, <span class="comment"># Container you clicked</span>
|
||||
<span class="string">"fields"</span>: [
|
||||
{
|
||||
<span class="string">"name"</span>: <span class="string">"title"</span>,
|
||||
<span class="string">"selector"</span>: <span class="string">"h3.product-title"</span>,
|
||||
<span class="string">"type"</span>: <span class="string">"text"</span>
|
||||
},
|
||||
{
|
||||
<span class="string">"name"</span>: <span class="string">"price"</span>,
|
||||
<span class="string">"selector"</span>: <span class="string">"span.price"</span>,
|
||||
<span class="string">"type"</span>: <span class="string">"text"</span>
|
||||
},
|
||||
{
|
||||
<span class="string">"name"</span>: <span class="string">"description"</span>,
|
||||
<span class="string">"selector"</span>: <span class="string">"p.description"</span>,
|
||||
<span class="string">"type"</span>: <span class="string">"text"</span>
|
||||
},
|
||||
{
|
||||
<span class="string">"name"</span>: <span class="string">"image"</span>,
|
||||
<span class="string">"selector"</span>: <span class="string">"img.product-image"</span>,
|
||||
<span class="string">"type"</span>: <span class="string">"attribute"</span>,
|
||||
<span class="string">"attribute"</span>: <span class="string">"src"</span>
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
<span class="comment"># Create extraction strategy</span>
|
||||
extraction_strategy = JsonCssExtractionStrategy(schema, verbose=<span class="keyword">True</span>)
|
||||
|
||||
<span class="comment"># Configure the crawler</span>
|
||||
config = CrawlerRunConfig(
|
||||
extraction_strategy=extraction_strategy
|
||||
)
|
||||
|
||||
<span class="keyword">async</span> <span class="keyword">with</span> AsyncWebCrawler() <span class="keyword">as</span> crawler:
|
||||
result = <span class="keyword">await</span> crawler.arun(
|
||||
url=<span class="string">"https://example.com/products"</span>,
|
||||
config=config
|
||||
)
|
||||
|
||||
<span class="comment"># Parse the extracted data</span>
|
||||
products = json.loads(result.extracted_content)
|
||||
<span class="keyword">print</span>(<span class="string">f"Extracted {len(products)} products"</span>)
|
||||
|
||||
<span class="comment"># Display first product</span>
|
||||
<span class="keyword">if</span> products:
|
||||
<span class="keyword">print</span>(json.dumps(products[0], indent=2))
|
||||
|
||||
<span class="keyword">return</span> products
|
||||
|
||||
<span class="comment"># Run the extraction</span>
|
||||
<span class="keyword">if</span> __name__ == <span class="string">"__main__"</span>:
|
||||
asyncio.run(extract_products())</code></pre>
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<!-- Coming Soon Section -->
|
||||
<section class="coming-soon-section">
|
||||
<h2>Coming Soon: Even More Power</h2>
|
||||
<div class="terminal-window">
|
||||
<div class="terminal-header">
|
||||
<span class="terminal-title">Future Features</span>
|
||||
</div>
|
||||
<div class="terminal-content">
|
||||
<p class="intro-text">We're continuously expanding C4AI Assistant with powerful new features to make web scraping even easier:</p>
|
||||
|
||||
<div class="coming-features">
|
||||
<div class="coming-feature">
|
||||
<div class="feature-header">
|
||||
<span class="feature-badge">Cloud</span>
|
||||
<h3>Run on C4AI Cloud</h3>
|
||||
</div>
|
||||
<p>Execute your extraction directly in the cloud without setting up any local environment. Just click "Run on Cloud" and get your data instantly.</p>
|
||||
<div class="feature-preview">
|
||||
<code>☁️ Instant results • Auto-scaling</code>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="coming-feature">
|
||||
<div class="feature-header">
|
||||
<span class="feature-badge">Direct</span>
|
||||
<h3>Get CrawlResult Without Code</h3>
|
||||
</div>
|
||||
<p>Skip the code generation entirely! Get extracted data directly in the extension as a CrawlResult object, ready to download as JSON.</p>
|
||||
<div class="feature-preview">
|
||||
<code>📊 One-click extraction • No Python needed • Export to JSON/CSV</code>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="coming-feature">
|
||||
<div class="feature-header">
|
||||
<span class="feature-badge">AI</span>
|
||||
<h3>Smart Schema Suggestions</h3>
|
||||
</div>
|
||||
<p>AI-powered field detection that automatically suggests the most likely data fields on any page, making schema building even faster.</p>
|
||||
<div class="feature-preview">
|
||||
<code>🤖 Auto-detect fields • Smart naming • Pattern recognition</code>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="coming-feature">
|
||||
<div class="feature-header">
|
||||
<span class="feature-badge">Script</span>
|
||||
<h3>C4A Script Builder</h3>
|
||||
</div>
|
||||
<p>Visual automation script builder for complex interactions - fill forms, click buttons, handle pagination, all without writing code.</p>
|
||||
<div class="feature-preview">
|
||||
<code>🎯 Visual automation • Record & replay • Export as C4A script</code>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="stay-tuned">
|
||||
<p>🚀 Stay tuned for updates! Follow our <a href="https://github.com/unclecode/crawl4ai" target="_blank">GitHub</a> for the latest releases.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<!-- Footer -->
|
||||
<footer class="footer">
|
||||
<div class="footer-content">
|
||||
<div class="footer-section">
|
||||
<h4>Resources</h4>
|
||||
<ul>
|
||||
<li><a href="https://github.com/unclecode/crawl4ai">GitHub Repository</a></li>
|
||||
<li><a href="../../">Documentation</a></li>
|
||||
<li><a href="https://discord.gg/jP8KfhDhyN">Discord Community</a></li>
|
||||
</ul>
|
||||
</div>
|
||||
<div class="footer-section">
|
||||
<h4>Connect</h4>
|
||||
<ul>
|
||||
<li><a href="https://twitter.com/unclecode">Twitter @unclecode</a></li>
|
||||
<li><a href="https://github.com/unclecode">GitHub @unclecode</a></li>
|
||||
</ul>
|
||||
</div>
|
||||
</div>
|
||||
<div class="footer-bottom">
|
||||
<p>Made with 🚀 by the Crawl4AI team</p>
|
||||
</div>
|
||||
</footer>
|
||||
</div>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
44
docs/md_v2/apps/crawl4ai-assistant/manifest.json
Normal file
@@ -0,0 +1,44 @@
|
||||
{
|
||||
"manifest_version": 3,
|
||||
"name": "Crawl4AI Assistant",
|
||||
"version": "1.0.1",
|
||||
"description": "Visual schema and script builder for Crawl4AI - Build extraction schemas by clicking on elements",
|
||||
"permissions": [
|
||||
"activeTab",
|
||||
"storage",
|
||||
"downloads"
|
||||
],
|
||||
"host_permissions": [
|
||||
"<all_urls>"
|
||||
],
|
||||
"action": {
|
||||
"default_popup": "popup/popup.html",
|
||||
"default_icon": {
|
||||
"16": "icons/icon-16.png",
|
||||
"48": "icons/icon-48.png",
|
||||
"128": "icons/icon-128.png"
|
||||
}
|
||||
},
|
||||
"content_scripts": [
|
||||
{
|
||||
"matches": ["<all_urls>"],
|
||||
"js": ["content/content.js"],
|
||||
"css": ["content/overlay.css"],
|
||||
"run_at": "document_idle"
|
||||
}
|
||||
],
|
||||
"background": {
|
||||
"service_worker": "background/service-worker.js"
|
||||
},
|
||||
"icons": {
|
||||
"16": "icons/icon-16.png",
|
||||
"48": "icons/icon-48.png",
|
||||
"128": "icons/icon-128.png"
|
||||
},
|
||||
"web_accessible_resources": [
|
||||
{
|
||||
"resources": ["icons/*", "assets/*"],
|
||||
"matches": ["<all_urls>"]
|
||||
}
|
||||
]
|
||||
}
|
||||
BIN
docs/md_v2/apps/crawl4ai-assistant/popup/icons/favicon.ico
Normal file
|
After Width: | Height: | Size: 3.4 KiB |
BIN
docs/md_v2/apps/crawl4ai-assistant/popup/icons/icon-128.png
Normal file
|
After Width: | Height: | Size: 1.6 KiB |
BIN
docs/md_v2/apps/crawl4ai-assistant/popup/icons/icon-16.png
Normal file
|
After Width: | Height: | Size: 1.6 KiB |
BIN
docs/md_v2/apps/crawl4ai-assistant/popup/icons/icon-48.png
Normal file
|
After Width: | Height: | Size: 1.6 KiB |
324
docs/md_v2/apps/crawl4ai-assistant/popup/popup.css
Normal file
@@ -0,0 +1,324 @@
|
||||
/* Font Face Definitions */
|
||||
@font-face {
|
||||
font-family: 'Dank Mono';
|
||||
src: url('../assets/DankMono-Regular.woff2') format('woff2');
|
||||
font-weight: 400;
|
||||
font-style: normal;
|
||||
font-display: swap;
|
||||
}
|
||||
|
||||
@font-face {
|
||||
font-family: 'Dank Mono';
|
||||
src: url('../assets/DankMono-Bold.woff2') format('woff2');
|
||||
font-weight: 700;
|
||||
font-style: normal;
|
||||
font-display: swap;
|
||||
}
|
||||
|
||||
@font-face {
|
||||
font-family: 'Dank Mono';
|
||||
src: url('../assets/DankMono-Italic.woff2') format('woff2');
|
||||
font-weight: 400;
|
||||
font-style: italic;
|
||||
font-display: swap;
|
||||
}
|
||||
|
||||
:root {
|
||||
--font-primary: 'Dank Mono', -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, monospace;
|
||||
}
|
||||
|
||||
* {
|
||||
margin: 0;
|
||||
padding: 0;
|
||||
box-sizing: border-box;
|
||||
}
|
||||
|
||||
body {
|
||||
width: 380px;
|
||||
font-family: var(--font-primary);
|
||||
background: #0a0a0a;
|
||||
color: #e0e0e0;
|
||||
}
|
||||
|
||||
.popup-container {
|
||||
padding: 20px;
|
||||
}
|
||||
|
||||
header {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 16px;
|
||||
margin-bottom: 20px;
|
||||
padding-bottom: 16px;
|
||||
border-bottom: 1px solid #2a2a2a;
|
||||
}
|
||||
|
||||
.logo {
|
||||
width: 48px;
|
||||
height: 48px;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
|
||||
.header-content {
|
||||
flex: 1;
|
||||
}
|
||||
|
||||
header h1 {
|
||||
font-size: 20px;
|
||||
font-weight: 600;
|
||||
color: #fff;
|
||||
margin: 0 0 4px 0;
|
||||
}
|
||||
|
||||
.header-stats {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 12px;
|
||||
}
|
||||
|
||||
.github-stars {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 6px;
|
||||
color: #999;
|
||||
text-decoration: none;
|
||||
font-size: 13px;
|
||||
transition: color 0.2s ease;
|
||||
}
|
||||
|
||||
.github-stars:hover {
|
||||
color: #4a9eff;
|
||||
}
|
||||
|
||||
.github-icon {
|
||||
flex-shrink: 0;
|
||||
}
|
||||
|
||||
.mode-selector {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 12px;
|
||||
margin-bottom: 20px;
|
||||
}
|
||||
|
||||
.mode-button {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 16px;
|
||||
padding: 16px;
|
||||
background: #1a1a1a;
|
||||
border: 2px solid #2a2a2a;
|
||||
border-radius: 12px;
|
||||
cursor: pointer;
|
||||
transition: all 0.2s ease;
|
||||
width: 100%;
|
||||
text-align: left;
|
||||
}
|
||||
|
||||
.mode-button:hover:not(:disabled) {
|
||||
background: #252525;
|
||||
border-color: #4a9eff;
|
||||
transform: translateY(-2px);
|
||||
}
|
||||
|
||||
.mode-button:disabled {
|
||||
opacity: 0.5;
|
||||
cursor: not-allowed;
|
||||
}
|
||||
|
||||
.mode-button .icon {
|
||||
font-size: 32px;
|
||||
width: 48px;
|
||||
height: 48px;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
background: #252525;
|
||||
border-radius: 8px;
|
||||
}
|
||||
|
||||
.mode-button.schema .icon {
|
||||
background: #1e3a5f;
|
||||
}
|
||||
|
||||
.mode-button.script .icon {
|
||||
background: #3a1e5f;
|
||||
}
|
||||
|
||||
.mode-info h3 {
|
||||
font-size: 16px;
|
||||
color: #fff;
|
||||
margin-bottom: 4px;
|
||||
}
|
||||
|
||||
.mode-info p {
|
||||
font-size: 13px;
|
||||
color: #999;
|
||||
line-height: 1.4;
|
||||
}
|
||||
|
||||
.active-session {
|
||||
background: #1a1a1a;
|
||||
border: 2px solid #4a9eff;
|
||||
border-radius: 12px;
|
||||
padding: 16px;
|
||||
margin-bottom: 20px;
|
||||
}
|
||||
|
||||
.active-session.hidden {
|
||||
display: none;
|
||||
}
|
||||
|
||||
.session-header {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 8px;
|
||||
margin-bottom: 12px;
|
||||
}
|
||||
|
||||
.status-dot {
|
||||
width: 8px;
|
||||
height: 8px;
|
||||
background: #4a9eff;
|
||||
border-radius: 50%;
|
||||
animation: pulse 2s infinite;
|
||||
}
|
||||
|
||||
@keyframes pulse {
|
||||
0%, 100% { opacity: 1; }
|
||||
50% { opacity: 0.5; }
|
||||
}
|
||||
|
||||
.session-title {
|
||||
font-size: 14px;
|
||||
font-weight: 600;
|
||||
color: #4a9eff;
|
||||
}
|
||||
|
||||
.session-stats {
|
||||
display: flex;
|
||||
gap: 20px;
|
||||
margin-bottom: 16px;
|
||||
padding: 12px;
|
||||
background: #0a0a0a;
|
||||
border-radius: 8px;
|
||||
}
|
||||
|
||||
.stat {
|
||||
flex: 1;
|
||||
}
|
||||
|
||||
.stat-label {
|
||||
display: block;
|
||||
font-size: 11px;
|
||||
color: #666;
|
||||
text-transform: uppercase;
|
||||
margin-bottom: 4px;
|
||||
}
|
||||
|
||||
.stat-value {
|
||||
font-size: 14px;
|
||||
color: #fff;
|
||||
font-weight: 600;
|
||||
}
|
||||
|
||||
.session-actions {
|
||||
display: flex;
|
||||
gap: 8px;
|
||||
}
|
||||
|
||||
.action-button {
|
||||
flex: 1;
|
||||
padding: 10px 16px;
|
||||
border: none;
|
||||
border-radius: 8px;
|
||||
font-size: 14px;
|
||||
font-weight: 600;
|
||||
cursor: pointer;
|
||||
transition: all 0.2s ease;
|
||||
}
|
||||
|
||||
.action-button.primary {
|
||||
background: #4a9eff;
|
||||
color: #000;
|
||||
}
|
||||
|
||||
.action-button.primary:hover:not(:disabled) {
|
||||
background: #3a8eef;
|
||||
transform: translateY(-1px);
|
||||
}
|
||||
|
||||
.action-button.primary:disabled {
|
||||
background: #2a4a7f;
|
||||
color: #666;
|
||||
cursor: not-allowed;
|
||||
}
|
||||
|
||||
.action-button.secondary {
|
||||
background: #2a2a2a;
|
||||
color: #fff;
|
||||
}
|
||||
|
||||
.action-button.secondary:hover {
|
||||
background: #3a3a3a;
|
||||
}
|
||||
|
||||
.instructions {
|
||||
background: #1a1a1a;
|
||||
border-radius: 12px;
|
||||
padding: 16px;
|
||||
margin-bottom: 16px;
|
||||
}
|
||||
|
||||
.instructions h4 {
|
||||
font-size: 14px;
|
||||
margin-bottom: 12px;
|
||||
color: #fff;
|
||||
}
|
||||
|
||||
.instructions ol {
|
||||
padding-left: 20px;
|
||||
}
|
||||
|
||||
.instructions li {
|
||||
font-size: 13px;
|
||||
line-height: 1.6;
|
||||
color: #ccc;
|
||||
margin-bottom: 6px;
|
||||
}
|
||||
|
||||
footer {
|
||||
padding-top: 16px;
|
||||
border-top: 1px solid #2a2a2a;
|
||||
}
|
||||
|
||||
.social-links {
|
||||
display: flex;
|
||||
justify-content: center;
|
||||
gap: 16px;
|
||||
}
|
||||
|
||||
.social-link {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 6px;
|
||||
color: #999;
|
||||
text-decoration: none;
|
||||
font-size: 12px;
|
||||
transition: all 0.2s ease;
|
||||
padding: 6px 12px;
|
||||
border-radius: 6px;
|
||||
background: #1a1a1a;
|
||||
}
|
||||
|
||||
.social-link:hover {
|
||||
color: #0fbbaa;
|
||||
background: #2a2a2a;
|
||||
transform: translateY(-1px);
|
||||
}
|
||||
|
||||
.social-link svg {
|
||||
width: 16px;
|
||||
height: 16px;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
103
docs/md_v2/apps/crawl4ai-assistant/popup/popup.html
Normal file
@@ -0,0 +1,103 @@
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<link rel="stylesheet" href="popup.css">
|
||||
</head>
|
||||
<body>
|
||||
<div class="popup-container">
|
||||
<header>
|
||||
<img src="icons/icon-48.png" class="logo" alt="Crawl4AI">
|
||||
<div class="header-content">
|
||||
<h1>Crawl4AI Assistant</h1>
|
||||
<div class="header-stats">
|
||||
<a href="https://github.com/unclecode/crawl4ai" target="_blank" class="github-stars">
|
||||
<svg class="github-icon" viewBox="0 0 16 16" width="16" height="16">
|
||||
<path fill="currentColor" d="M8 0C3.58 0 0 3.58 0 8c0 3.54 2.29 6.53 5.47 7.59.4.07.55-.17.55-.38 0-.19-.01-.82-.01-1.49-2.01.37-2.53-.49-2.69-.94-.09-.23-.48-.94-.82-1.13-.28-.15-.68-.52-.01-.53.63-.01 1.08.58 1.23.82.72 1.21 1.87.87 2.33.66.07-.52.28-.87.51-1.07-1.78-.2-3.64-.89-3.64-3.95 0-.87.31-1.59.82-2.15-.08-.2-.36-1.02.08-2.12 0 0 .67-.21 2.2.82.64-.18 1.32-.27 2-.27.68 0 1.36.09 2 .27 1.53-1.04 2.2-.82 2.2-.82.44 1.1.16 1.92.08 2.12.51.56.82 1.27.82 2.15 0 3.07-1.87 3.75-3.65 3.95.29.25.54.73.54 1.48 0 1.07-.01 1.93-.01 2.2 0 .21.15.46.55.38A8.013 8.013 0 0016 8c0-4.42-3.58-8-8-8z"></path>
|
||||
</svg>
|
||||
<span id="stars-count">Loading...</span>
|
||||
</a>
|
||||
</div>
|
||||
</div>
|
||||
</header>
|
||||
|
||||
<div class="mode-selector">
|
||||
<button id="schema-mode" class="mode-button schema">
|
||||
<div class="icon">📊</div>
|
||||
<div class="mode-info">
|
||||
<h3>Schema Builder</h3>
|
||||
<p>Click elements to build extraction schemas</p>
|
||||
</div>
|
||||
</button>
|
||||
|
||||
<button id="script-mode" class="mode-button script" disabled>
|
||||
<div class="icon">🎯</div>
|
||||
<div class="mode-info">
|
||||
<h3>Script Builder</h3>
|
||||
<p>Coming soon - Build automation scripts</p>
|
||||
</div>
|
||||
</button>
|
||||
</div>
|
||||
|
||||
<div id="active-session" class="active-session hidden">
|
||||
<div class="session-header">
|
||||
<span class="status-dot"></span>
|
||||
<span class="session-title">Schema Capture Active</span>
|
||||
</div>
|
||||
<div class="session-stats">
|
||||
<div class="stat">
|
||||
<span class="stat-label">Container:</span>
|
||||
<span id="container-status" class="stat-value">Not selected</span>
|
||||
</div>
|
||||
<div class="stat">
|
||||
<span class="stat-label">Fields:</span>
|
||||
<span id="fields-count" class="stat-value">0</span>
|
||||
</div>
|
||||
</div>
|
||||
<div class="session-actions">
|
||||
<button id="generate-code" class="action-button primary" disabled>
|
||||
<span>Generate Code</span>
|
||||
</button>
|
||||
<button id="stop-capture" class="action-button secondary">
|
||||
<span>Stop Capture</span>
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="instructions">
|
||||
<h4>How to use:</h4>
|
||||
<ol>
|
||||
<li>Click "Schema Builder" to start</li>
|
||||
<li>Click on a container element (e.g., product card)</li>
|
||||
<li>Click individual fields inside and name them</li>
|
||||
<li>Generate Python code when done</li>
|
||||
</ol>
|
||||
</div>
|
||||
|
||||
<footer>
|
||||
<div class="social-links">
|
||||
<a href="https://docs.crawl4ai.com" target="_blank" class="social-link">
|
||||
<svg viewBox="0 0 24 24" width="16" height="16">
|
||||
<path fill="currentColor" d="M12 2C6.48 2 2 6.48 2 12s4.48 10 10 10 10-4.48 10-10S17.52 2 12 2zm-2 15l-5-5 1.41-1.41L10 14.17l7.59-7.59L19 8l-9 9z"/>
|
||||
</svg>
|
||||
<span>Docs</span>
|
||||
</a>
|
||||
<a href="https://twitter.com/unclecode" target="_blank" class="social-link">
|
||||
<svg viewBox="0 0 24 24" width="16" height="16">
|
||||
<path fill="currentColor" d="M18.244 2.25h3.308l-7.227 8.26 8.502 11.24H16.17l-5.214-6.817L4.99 21.75H1.68l7.73-8.835L1.254 2.25H8.08l4.713 6.231zm-1.161 17.52h1.833L7.084 4.126H5.117z"/>
|
||||
</svg>
|
||||
<span>@unclecode</span>
|
||||
</a>
|
||||
<a href="https://discord.gg/jP8KfhDhyN" target="_blank" class="social-link">
|
||||
<svg viewBox="0 0 24 24" width="16" height="16">
|
||||
<path fill="currentColor" d="M19.27 5.33C17.94 4.71 16.5 4.26 15 4a.09.09 0 00-.07.03c-.18.33-.39.76-.53 1.09a16.09 16.09 0 00-4.8 0c-.14-.34-.35-.76-.54-1.09-.01-.02-.04-.03-.07-.03-1.5.26-2.93.71-4.27 1.33-.01 0-.02.01-.03.02-2.72 4.07-3.47 8.03-3.1 11.95 0 .02.01.04.03.05 1.8 1.32 3.53 2.12 5.24 2.65.03.01.06 0 .07-.02.4-.55.76-1.13 1.07-1.74.02-.04 0-.08-.04-.09-.57-.22-1.11-.48-1.64-.78-.04-.02-.04-.08-.01-.11.11-.08.22-.17.33-.25.02-.02.05-.02.07-.01 3.44 1.57 7.15 1.57 10.55 0 .02-.01.05-.01.07.01.11.09.22.17.33.26.04.03.04.09-.01.11-.52.31-1.07.56-1.64.78-.04.01-.05.06-.04.09.32.61.68 1.19 1.07 1.74.03.01.06.02.09.01 1.72-.53 3.45-1.33 5.25-2.65.02-.01.03-.03.03-.05.44-4.53-.73-8.46-3.1-11.95-.01-.01-.02-.02-.04-.02zM8.52 14.91c-1.03 0-1.89-.95-1.89-2.12s.84-2.12 1.89-2.12c1.06 0 1.9.96 1.89 2.12 0 1.17-.84 2.12-1.89 2.12zm6.97 0c-1.03 0-1.89-.95-1.89-2.12s.84-2.12 1.89-2.12c1.06 0 1.9.96 1.89 2.12 0 1.17-.83 2.12-1.89 2.12z"/>
|
||||
</svg>
|
||||
<span>Discord</span>
|
||||
</a>
|
||||
</div>
|
||||
</footer>
|
||||
</div>
|
||||
|
||||
<script src="popup.js"></script>
|
||||
</body>
|
||||
</html>
|
||||
112
docs/md_v2/apps/crawl4ai-assistant/popup/popup.js
Normal file
@@ -0,0 +1,112 @@
|
||||
// Popup script for Crawl4AI Assistant
|
||||
let activeMode = null;
|
||||
|
||||
document.addEventListener('DOMContentLoaded', () => {
|
||||
// Fetch GitHub stars
|
||||
fetchGitHubStars();
|
||||
|
||||
// Check current state
|
||||
chrome.storage.local.get(['captureMode', 'captureStats'], (data) => {
|
||||
if (data.captureMode) {
|
||||
activeMode = data.captureMode;
|
||||
showActiveSession(data.captureStats || {});
|
||||
}
|
||||
});
|
||||
|
||||
// Mode buttons
|
||||
document.getElementById('schema-mode').addEventListener('click', () => {
|
||||
startSchemaCapture();
|
||||
});
|
||||
|
||||
// Session actions
|
||||
document.getElementById('generate-code').addEventListener('click', () => {
|
||||
generateCode();
|
||||
});
|
||||
|
||||
document.getElementById('stop-capture').addEventListener('click', () => {
|
||||
stopCapture();
|
||||
});
|
||||
});
|
||||
|
||||
async function fetchGitHubStars() {
|
||||
try {
|
||||
const response = await fetch('https://api.github.com/repos/unclecode/crawl4ai');
|
||||
const data = await response.json();
|
||||
const stars = data.stargazers_count;
|
||||
|
||||
// Format the number (e.g., 1.2k)
|
||||
let formattedStars;
|
||||
if (stars >= 1000) {
|
||||
formattedStars = (stars / 1000).toFixed(1) + 'k';
|
||||
} else {
|
||||
formattedStars = stars.toString();
|
||||
}
|
||||
|
||||
document.getElementById('stars-count').textContent = `⭐ ${formattedStars}`;
|
||||
} catch (error) {
|
||||
console.error('Failed to fetch GitHub stars:', error);
|
||||
document.getElementById('stars-count').textContent = '⭐ 2k+';
|
||||
}
|
||||
}
|
||||
|
||||
function startSchemaCapture() {
|
||||
chrome.tabs.query({ active: true, currentWindow: true }, (tabs) => {
|
||||
chrome.tabs.sendMessage(tabs[0].id, {
|
||||
action: 'startSchemaCapture'
|
||||
}, (response) => {
|
||||
if (response && response.success) {
|
||||
// Close the popup to let user interact with the page
|
||||
window.close();
|
||||
}
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
function showActiveSession(stats) {
|
||||
document.querySelector('.mode-selector').style.display = 'none';
|
||||
document.getElementById('active-session').classList.remove('hidden');
|
||||
|
||||
updateSessionStats(stats);
|
||||
}
|
||||
|
||||
function updateSessionStats(stats) {
|
||||
document.getElementById('container-status').textContent =
|
||||
stats.container ? 'Selected ✓' : 'Not selected';
|
||||
document.getElementById('fields-count').textContent = stats.fields || 0;
|
||||
|
||||
// Enable generate button if we have container and fields
|
||||
document.getElementById('generate-code').disabled =
|
||||
!stats.container || stats.fields === 0;
|
||||
}
|
||||
|
||||
function generateCode() {
|
||||
chrome.tabs.query({ active: true, currentWindow: true }, (tabs) => {
|
||||
chrome.tabs.sendMessage(tabs[0].id, {
|
||||
action: 'generateCode'
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
function stopCapture() {
|
||||
chrome.tabs.query({ active: true, currentWindow: true }, (tabs) => {
|
||||
chrome.tabs.sendMessage(tabs[0].id, {
|
||||
action: 'stopCapture'
|
||||
}, () => {
|
||||
// Reset UI
|
||||
document.querySelector('.mode-selector').style.display = 'flex';
|
||||
document.getElementById('active-session').classList.add('hidden');
|
||||
activeMode = null;
|
||||
|
||||
// Clear storage
|
||||
chrome.storage.local.remove(['captureMode', 'captureStats']);
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
// Listen for stats updates from content script
|
||||
chrome.runtime.onMessage.addListener((message, sender, sendResponse) => {
|
||||
if (message.action === 'updateStats') {
|
||||
updateSessionStats(message.stats);
|
||||
chrome.storage.local.set({ captureStats: message.stats });
|
||||
}
|
||||
});
|
||||
@@ -186,19 +186,20 @@ Our apps are designed to make Crawl4AI more accessible and powerful. Whether you
|
||||
</div>
|
||||
|
||||
<div class="app-card">
|
||||
<span class="app-status status-coming-soon">Coming Soon</span>
|
||||
<h3>🏗️ Schema Builder</h3>
|
||||
<span class="app-status status-available">Available</span>
|
||||
<h3>🔍 Crawl4AI Assistant (Chrome Extension)</h3>
|
||||
<p class="app-description">
|
||||
Visually design JSON extraction schemas by pointing and clicking on webpage elements. No more guessing CSS selectors!
|
||||
Visual schema builder Chrome extension - click on webpage elements to generate extraction schemas and Python code!
|
||||
</p>
|
||||
<ul class="app-features">
|
||||
<li>Point-and-click selector generation</li>
|
||||
<li>Automatic schema inference</li>
|
||||
<li>Live extraction preview</li>
|
||||
<li>Export to multiple formats</li>
|
||||
<li>Visual element selection</li>
|
||||
<li>Container & field selection modes</li>
|
||||
<li>Smart selector generation</li>
|
||||
<li>Complete Python code generation</li>
|
||||
<li>One-click installation</li>
|
||||
</ul>
|
||||
<div class="app-action">
|
||||
<a href="#" class="app-btn disabled">Coming Soon</a>
|
||||
<a href="crawl4ai-assistant/" class="app-btn">Install Extension →</a>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
@@ -7,10 +7,25 @@
|
||||
<link rel="stylesheet" href="llmtxt.css">
|
||||
</head>
|
||||
<body>
|
||||
<div class="container">
|
||||
<header class="header">
|
||||
<h1><span class="logo">🚀🤖</span> Crawl4AI LLM Context Builder</h1>
|
||||
</header>
|
||||
<div class="terminal-container">
|
||||
<div class="header">
|
||||
<div class="header-content">
|
||||
<div class="logo-section">
|
||||
<img src="../../img/favicon-32x32.png" alt="Crawl4AI Logo" class="logo">
|
||||
<div>
|
||||
<h1>Crawl4AI LLM Context Builder</h1>
|
||||
<p class="tagline">Multi-Dimensional Context for AI Assistants</p>
|
||||
</div>
|
||||
</div>
|
||||
<nav class="nav-links">
|
||||
<a href="../../" class="nav-link">← Back to Docs</a>
|
||||
<a href="../" class="nav-link">All Apps</a>
|
||||
<a href="https://github.com/unclecode/crawl4ai" class="nav-link" target="_blank">GitHub</a>
|
||||
</nav>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="content">
|
||||
|
||||
<section class="intro">
|
||||
<div class="intro-header">
|
||||
@@ -135,6 +150,7 @@
|
||||
</table>
|
||||
</div>
|
||||
</section>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script src="llmtxt.js"></script>
|
||||
|
||||
@@ -1,5 +1,30 @@
|
||||
/* Terminal Theme CSS for LLM Context Builder */
|
||||
|
||||
/* Font Face Definitions */
|
||||
@font-face {
|
||||
font-family: 'Dank Mono';
|
||||
src: url('../assets/DankMono-Regular.woff2') format('woff2');
|
||||
font-weight: 400;
|
||||
font-style: normal;
|
||||
font-display: swap;
|
||||
}
|
||||
|
||||
@font-face {
|
||||
font-family: 'Dank Mono';
|
||||
src: url('../assets/DankMono-Bold.woff2') format('woff2');
|
||||
font-weight: 700;
|
||||
font-style: normal;
|
||||
font-display: swap;
|
||||
}
|
||||
|
||||
@font-face {
|
||||
font-family: 'Dank Mono';
|
||||
src: url('../assets/DankMono-Italic.woff2') format('woff2');
|
||||
font-weight: 400;
|
||||
font-style: italic;
|
||||
font-display: swap;
|
||||
}
|
||||
|
||||
:root {
|
||||
--background-color: #070708;
|
||||
--font-color: #e8e9ed;
|
||||
@@ -13,6 +38,9 @@
|
||||
--border-color: #3f3f44;
|
||||
--hover-bg: #1a1a1c;
|
||||
--success-color: #50ff50;
|
||||
--bg-secondary: #1a1a1a;
|
||||
--primary-green: #0fbbaa;
|
||||
--font-primary: 'Dank Mono', dm, Monaco, Courier New, monospace;
|
||||
}
|
||||
|
||||
* {
|
||||
@@ -22,38 +50,88 @@
|
||||
body {
|
||||
margin: 0;
|
||||
padding: 0;
|
||||
font-family: dm, Monaco, Courier New, monospace;
|
||||
font-family: var(--font-primary);
|
||||
font-size: 14px;
|
||||
line-height: 1.5;
|
||||
background-color: var(--background-color);
|
||||
color: var(--font-color);
|
||||
}
|
||||
|
||||
.container {
|
||||
/* Terminal Container */
|
||||
.terminal-container {
|
||||
min-height: 100vh;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
}
|
||||
|
||||
/* Header - matching assistant layout */
|
||||
.header {
|
||||
background: var(--bg-secondary);
|
||||
border-bottom: 1px solid var(--border-color);
|
||||
padding: 1.5rem 0;
|
||||
position: sticky;
|
||||
top: 0;
|
||||
z-index: 100;
|
||||
backdrop-filter: blur(10px);
|
||||
background: rgba(26, 26, 26, 0.95);
|
||||
}
|
||||
|
||||
.header-content {
|
||||
max-width: 1200px;
|
||||
margin: 0 auto;
|
||||
padding: 20px;
|
||||
padding: 0 2rem;
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
/* Header */
|
||||
.header {
|
||||
text-align: center;
|
||||
margin-bottom: 40px;
|
||||
border-bottom: 1px dashed var(--tertiary-color);
|
||||
padding-bottom: 20px;
|
||||
}
|
||||
|
||||
.header h1 {
|
||||
font-size: 24px;
|
||||
color: var(--primary-color);
|
||||
margin: 0;
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 2px;
|
||||
.logo-section {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 1rem;
|
||||
}
|
||||
|
||||
.logo {
|
||||
font-size: 28px;
|
||||
vertical-align: middle;
|
||||
width: 48px;
|
||||
height: 48px;
|
||||
}
|
||||
|
||||
.logo-section h1 {
|
||||
font-size: 1.75rem;
|
||||
font-weight: 700;
|
||||
color: var(--font-color);
|
||||
margin: 0;
|
||||
}
|
||||
|
||||
.tagline {
|
||||
font-size: 0.875rem;
|
||||
color: var(--tertiary-color);
|
||||
margin-top: 0.25rem;
|
||||
}
|
||||
|
||||
.nav-links {
|
||||
display: flex;
|
||||
gap: 2rem;
|
||||
}
|
||||
|
||||
.nav-link {
|
||||
color: var(--tertiary-color);
|
||||
text-decoration: none;
|
||||
font-size: 0.875rem;
|
||||
transition: color 0.2s ease;
|
||||
}
|
||||
|
||||
.nav-link:hover {
|
||||
color: var(--primary-green);
|
||||
}
|
||||
|
||||
/* Content */
|
||||
.content {
|
||||
flex: 1;
|
||||
max-width: 1200px;
|
||||
margin: 0 auto;
|
||||
padding: 2rem;
|
||||
width: 100%;
|
||||
}
|
||||
|
||||
/* Intro Section */
|
||||
@@ -446,6 +524,15 @@ body {
|
||||
|
||||
/* Responsive Design */
|
||||
@media (max-width: 768px) {
|
||||
.header-content {
|
||||
flex-direction: column;
|
||||
gap: 1.5rem;
|
||||
}
|
||||
|
||||
.nav-links {
|
||||
gap: 1rem;
|
||||
}
|
||||
|
||||
.preset-options {
|
||||
flex-direction: column;
|
||||
}
|
||||
@@ -454,7 +541,7 @@ body {
|
||||
grid-template-columns: 1fr;
|
||||
}
|
||||
|
||||
.container {
|
||||
padding: 10px;
|
||||
.content {
|
||||
padding: 1rem;
|
||||
}
|
||||
}
|
||||
@@ -28,7 +28,8 @@
|
||||
--secondary-color: #d5cec0;
|
||||
--tertiary-color: #a3abba;
|
||||
--primary-dimmed-color: #09b5a5; /* Updated to the brand color */
|
||||
--primary-color: #0fbbaa; /* Updated to the brand color */
|
||||
/* --primary-color: #0fbbaa; */
|
||||
--primary-color: #50ffff; /* Updated to the brand color */
|
||||
--accent-color: rgb(243, 128, 245);
|
||||
--error-color: #ff3c74;
|
||||
--progress-bar-background: #3f3f44;
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
<div class="llmtxt-container">
|
||||
I<div class="llmtxt-container">
|
||||
<iframe id="llmtxt-frame" src="../../llmtxt/index.html" width="100%" style="border:none; display: block;" title="Crawl4AI LLM Context Builder"></iframe>
|
||||
</div>
|
||||
|
||||
|
||||