@intent-systems/nexus 2026.1.5-3

package/skills/onboarding/docs/skill-deep-dives/brave-search.md

@@ -0,0 +1,601 @@
+# Skill Deep-Dive: brave-search
+
+**Skill**: `brave-search`
+**Category**: Free (Web Search & Content Extraction)
+**Setup Difficulty**: 🟢 Easy
+**Time Estimate**: ~5 minutes
+**Platform**: Cross-platform (macOS, Windows, Linux)
+
+---
+
+## What It Enables
+
+| Capability | Description | Example |
+|------------|-------------|---------|
+| Web search | Search the web programmatically | `node search.mjs "Anthropic Claude"` |
+| Content extraction | Extract article text from URLs | `node content.mjs "https://example.com/article"` |
+| Headless operation | No browser required | Pure API calls |
+| Result customization | Control number of results | `node search.mjs "query" -n 10` |
+| Combined search+content | Search and extract in one step | `node search.mjs "query" --content` |
+
+---
+
+## Dependencies
+
+### Hard Dependencies (Required)
+- [ ] `node` + `npm` — For running JavaScript scripts
+- [ ] `BRAVE_API_KEY` — Free API key from Brave Search
+- [ ] Internet connection — For API calls
+
+### Soft Dependencies (Recommended)
+- [ ] `1password` skill — For storing API key securely
+- [ ] `jq` — For parsing JSON output (optional)
+
+### This Skill Enables
+- [ ] Agent web access — Agent can search and read web content
+- [ ] Research automation — Gather information from multiple sources
+- [ ] Content summarization — Extract and analyze articles
+- [ ] Fact checking — Verify information from web sources
+
+---
+
+## Installation
+
+### Prerequisites Check
+
+```bash
+# Check for Node.js
+which node && node --version || echo "❌ Need Node.js"
+
+# Check for npm
+which npm && npm --version || echo "❌ Need npm"
+
+# Check for API key
+echo $BRAVE_API_KEY | grep -q . && echo "✅ API key set" || echo "⚠️  Need BRAVE_API_KEY"
+```
+
+### Installation Steps
+
+#### Step 1: Get Brave Search API Key
+
+```bash
+# 1. Visit: https://brave.com/search/api/
+# 2. Sign up for free account (2,000 queries/month free)
+# 3. Create API key
+
+# Store in 1Password
+op item create \
+  --category "API Credential" \
+  --title "Brave Search API Key" \
+  --vault Personal \
+  "credential[password]=YOUR_API_KEY_HERE"
+
+# Add to shell profile
+echo 'export BRAVE_API_KEY="op://Personal/Brave Search API Key/credential"' >> ~/.zshrc
+echo 'eval "$(op plugin run -- zsh)"' >> ~/.zshrc
+source ~/.zshrc
+
+# Or set directly (not recommended)
+export BRAVE_API_KEY="your-key-here"
+```
+
+#### Step 2: Verify Scripts are Available
+
+```bash
+# Navigate to brave-search skill directory
+cd ~/nexus/skills/brave-search
+
+# List scripts
+ls scripts/
+# Should show: search.mjs, content.mjs
+
+# Make sure they're executable
+chmod +x scripts/*.mjs
+```
+
+### Verification
+
+```bash
+# Check API key is set
+echo $BRAVE_API_KEY | grep -q . && echo "✅ API key set"
+
+# Test search
+cd ~/nexus/skills/brave-search
+node scripts/search.mjs "test query" -n 3
+
+# Test content extraction
+node scripts/content.mjs "https://example.com"
+```
+
+---
+
+## Configuration
+
+### Required: Set API Key
+
+```bash
+# Option 1: Via 1Password (recommended)
+export BRAVE_API_KEY="op://Personal/Brave Search API Key/credential"
+eval "$(op plugin run -- zsh)"
+
+# Option 2: Direct export (less secure)
+export BRAVE_API_KEY="your-key-here"
+
+# Option 3: Add to .env file (if skill supports it)
+echo "BRAVE_API_KEY=your-key-here" >> ~/nexus/skills/brave-search/.env
+
+# Verify
+echo $BRAVE_API_KEY
+```
+
+### Optional: Create Aliases
+
+```bash
+# Add to ~/.zshrc or ~/.bashrc
+alias brave-search='node ~/nexus/skills/brave-search/scripts/search.mjs'
+alias brave-content='node ~/nexus/skills/brave-search/scripts/content.mjs'
+
+# Usage:
+brave-search "Anthropic Claude"
+brave-content "https://example.com/article"
+```
+
+---
+
+## User Actions Required
+
+1. **Create Brave Search account**: Visit brave.com/search/api and sign up (free)
+2. **Generate API key**: Create API key in Brave Search dashboard
+3. **Store API key**: Save in 1Password or environment variable
+
+---
+
+## Common Failure Points
+
+### Failure: "BRAVE_API_KEY not set"
+
+**Symptom**: Scripts fail with "API key required" error
+
+**Cause**: Environment variable not set
+
+**Recovery**:
+```bash
+# Check if key is set
+echo $BRAVE_API_KEY
+
+# If empty, set it
+export BRAVE_API_KEY="your-key-here"
+
+# Or load from 1Password
+export BRAVE_API_KEY=$(op read "op://Personal/Brave Search API Key/credential")
+
+# Verify
+echo $BRAVE_API_KEY
+
+# Add to shell profile for persistence
+echo 'export BRAVE_API_KEY=$(op read "op://Personal/Brave Search API Key/credential")' >> ~/.zshrc
+source ~/.zshrc
+```
+
+**Prevention**: Set API key before using skill
+
+---
+
+### Failure: "API rate limit exceeded"
+
+**Symptom**: API returns 429 Too Many Requests error
+
+**Cause**: Exceeded free tier limit (2,000 queries/month)
+
+**Recovery**:
+```bash
+# Check your usage at: https://brave.com/search/api/dashboard
+
+# Options:
+# 1. Wait for monthly reset
+# 2. Upgrade to paid plan
+# 3. Use caching to reduce API calls
+
+# Implement simple cache
+CACHE_FILE="/tmp/brave-search-$(echo "$QUERY" | md5).json"
+if [ -f "$CACHE_FILE" ] && [ $(find "$CACHE_FILE" -mmin -60) ]; then
+    cat "$CACHE_FILE"
+else
+    node search.mjs "$QUERY" | tee "$CACHE_FILE"
+fi
+```
+
+**Prevention**: Monitor usage, implement caching, avoid redundant queries
+
+---
+
+### Failure: Content extraction returns empty/incomplete
+
+**Symptom**: `content.mjs` returns little or no text from URL
+
+**Cause**: Website is heavily JavaScript-based or blocks extraction
+
+**Recovery**:
+```bash
+# Brave Search content extraction is best-effort
+# For JavaScript-heavy sites, use alternative methods:
+
+# Option 1: Use summarize skill with Firecrawl fallback
+# nexus run summarize "https://example.com"
+
+# Option 2: Use computer-use skill with browser automation
+# interpreter "Navigate to URL and extract content"
+
+# Option 3: Try different extraction tool (jina, trafilatura, etc.)
+```
+
+**Prevention**: Test extraction on target sites before relying on it
+
+---
+
+### Failure: "Cannot find module" error
+
+**Symptom**: Node.js can't find script or dependencies
+
+**Cause**: Wrong working directory or missing dependencies
+
+**Recovery**:
+```bash
+# Navigate to skill directory
+cd ~/nexus/skills/brave-search
+
+# Check if scripts exist
+ls scripts/search.mjs scripts/content.mjs
+
+# Install dependencies if package.json exists
+if [ -f package.json ]; then
+    npm install
+fi
+
+# Try again
+node scripts/search.mjs "test query"
+```
+
+**Prevention**: Always run scripts from skill directory or use absolute paths
+
+---
+
+## Automation Opportunities
+
+| Step | Automatable? | Tool | Notes |
+|------|--------------|------|-------|
+| Get API key | ❌ No | - | Requires account creation |
+| Store API key | ✅ Yes | bash | 1Password integration |
+| Web search | ✅ Yes | bash | `node search.mjs "query"` |
+| Content extraction | ✅ Yes | bash | `node content.mjs "url"` |
+| Parse results | ✅ Yes | bash | Use `jq` for JSON parsing |
+| Cache results | ✅ Yes | bash | Store in /tmp with timestamp |
+
+### Automation Script: Research Assistant
+
+```bash
+#!/bin/bash
+# Research a topic: search + extract top results
+
+QUERY="$1"
+NUM_RESULTS="${2:-5}"
+
+if [ -z "$QUERY" ]; then
+    echo "Usage: $0 'search query' [num_results]"
+    exit 1
+fi
+
+echo "🔍 Searching for: $QUERY"
+echo ""
+
+cd ~/nexus/skills/brave-search
+
+# Search and get URLs
+RESULTS=$(node scripts/search.mjs "$QUERY" -n "$NUM_RESULTS" 2>/dev/null)
+
+# Extract and display URLs
+echo "$RESULTS" | jq -r '.web.results[].url' | while read -r URL; do
+    echo "📄 Extracting: $URL"
+    node scripts/content.mjs "$URL" 2>/dev/null | head -20
+    echo ""
+    echo "---"
+    echo ""
+done
+
+echo "✅ Research complete!"
+```
+
+### Caching Wrapper
+
+```bash
+#!/bin/bash
+# Cache Brave Search results to avoid rate limits
+
+CACHE_DIR="/tmp/brave-search-cache"
+mkdir -p "$CACHE_DIR"
+
+QUERY="$1"
+CACHE_KEY=$(echo "$QUERY" | md5)
+CACHE_FILE="$CACHE_DIR/$CACHE_KEY.json"
+CACHE_AGE=3600  # 1 hour
+
+if [ -f "$CACHE_FILE" ] && [ $(( $(date +%s) - $(stat -f %m "$CACHE_FILE") )) -lt $CACHE_AGE ]; then
+    echo "🗂️  Using cached results"
+    cat "$CACHE_FILE"
+else
+    echo "🔍 Fetching fresh results"
+    cd ~/nexus/skills/brave-search
+    node scripts/search.mjs "$QUERY" | tee "$CACHE_FILE"
+fi
+```
+
+---
+
+## Wow Moments
+
+| Moment | What Happens | When to Demo |
+|--------|--------------|--------------|
+| Instant web access | Agent searches web without browser | When user asks about current events |
+| Content extraction | Pure text from articles, no ads/clutter | When demonstrating research capabilities |
+| Structured results | JSON output for programmatic use | When showing automation potential |
+| No browser overhead | Fast results, no browser startup time | Comparing to manual browsing |
+| API-based research | Gather info from 10+ sources in seconds | Research automation use case |
+
+### Demo Script
+
+```
+Agent: "Let me show you how I can search the web and extract content. I'll research a topic for you:"
+
+cd ~/nexus/skills/brave-search
+node scripts/search.mjs "Anthropic Claude API" -n 5
+
+Agent: "I found several relevant results. Now let me extract the content from the top article:"
+
+# Get first URL from results
+URL=$(node scripts/search.mjs "Anthropic Claude API" -n 1 | jq -r '.web.results[0].url')
+node scripts/content.mjs "$URL"
+
+Agent: "Notice how I extracted the pure article text without ads or navigation? This lets me read and analyze content programmatically."
+
+Agent: "And the best part - all of this happens headlessly, no browser required. I can research multiple topics simultaneously."
+```
+
+---
+
+## Usage Examples
+
+### Basic Search
+
+```bash
+# Simple search
+cd ~/nexus/skills/brave-search
+node scripts/search.mjs "Anthropic Claude"
+
+# Limit results
+node scripts/search.mjs "machine learning" -n 10
+
+# Search with content extraction
+node scripts/search.mjs "climate change" --content
+
+# Combine: search 3 results and extract content
+node scripts/search.mjs "quantum computing" -n 3 --content
+```
+
+### Content Extraction
+
+```bash
+# Extract article content
+node scripts/content.mjs "https://example.com/article"
+
+# Save to file
+node scripts/content.mjs "https://example.com/article" > article.txt
+
+# Extract multiple URLs
+for url in "url1" "url2" "url3"; do
+    node scripts/content.mjs "$url" > "$(echo $url | md5).txt"
+done
+```
+
+### Parsing Results with jq
+
+```bash
+# Search and extract just URLs
+node scripts/search.mjs "topic" | jq -r '.web.results[].url'
+
+# Get titles and URLs
+node scripts/search.mjs "topic" | jq -r '.web.results[] | "\(.title): \(.url)"'
+
+# Get just descriptions
+node scripts/search.mjs "topic" | jq -r '.web.results[].description'
+
+# Count results
+node scripts/search.mjs "topic" | jq '.web.results | length'
+```
+
+---
+
+## Integration with Other Skills
+
+| Skill | Integration | Notes |
+|-------|-------------|-------|
+| **1password** | Store API key | Secure credential management |
+| **qmd** | Index extracted content | Search web results semantically |
+| **summarize** | Summarize extracted content | Fallback: Use Firecrawl for JS sites |
+| **aix** | Provide web context to AI | AI can research before answering |
+
+---
+
+## Testing Checklist
+
+- [ ] API key obtained from Brave Search
+- [ ] API key stored in 1Password
+- [ ] Environment variable set: `echo $BRAVE_API_KEY`
+- [ ] Scripts exist: `ls ~/nexus/skills/brave-search/scripts/`
+- [ ] Basic search works: `node search.mjs "test" -n 3`
+- [ ] Content extraction works: `node content.mjs "https://example.com"`
+- [ ] Search with content: `node search.mjs "test" --content`
+- [ ] JSON parsing: `node search.mjs "test" | jq`
+- [ ] Rate limit check: Verify usage on dashboard
+- [ ] Caching: Test cache wrapper script
+
+---
+
+## Notes for Onboarding Agent
+
+- **API key required**: Always check if user has Brave Search API key
+- **Free tier**: Emphasize 2,000 queries/month free (generous for most users)
+- **When to recommend**:
+  - User wants agent to access current web information
+  - User needs content extraction for research
+  - User wants headless web search (no browser)
+  - User does automated research workflows
+- **When NOT to recommend**:
+  - User doesn't need web search capabilities
+  - User prefers WebSearch built-in tool (if available)
+  - User uncomfortable with third-party API
+- **Value proposition**: "Give your agent web access without opening a browser"
+- **Time estimate**: 5 minutes (including API key setup)
+- **Content extraction limitations**: Best for article sites; struggles with JS-heavy apps
+- **Alternative tools**:
+  - WebSearch: Built-in tool (if available)
+  - computer-use: For JS-heavy sites
+  - summarize skill: Has Firecrawl fallback
+- **Cost awareness**: Free tier sufficient for most users; monitor usage
+
+---
+
+## API Response Structure
+
+### Search Response
+
+```json
+{
+  "query": {
+    "original": "Anthropic Claude",
+    "show_strict_warning": false
+  },
+  "web": {
+    "results": [
+      {
+        "title": "Claude - Anthropic",
+        "url": "https://www.anthropic.com/claude",
+        "description": "Claude is a family of AI assistants...",
+        "age": "2024-01-15T00:00:00Z",
+        "language": "en"
+      }
+    ]
+  }
+}
+```
+
+### Content Extraction Response
+
+```
+Title: Article Title
+URL: https://example.com/article
+
+Article content text here...
+Multiple paragraphs...
+```
+
+---
+
+## Performance Considerations
+
+### Rate Limits
+
+- **Free tier**: 2,000 queries/month
+- **Paid tiers**: 5,000 to 50,000+ queries/month
+- **No per-second limits**: Can query rapidly within monthly quota
+
+### Caching Strategies
+
+```bash
+# Cache by query hash
+QUERY_HASH=$(echo "$QUERY" | md5)
+CACHE_FILE="/tmp/brave-cache-$QUERY_HASH.json"
+
+# Cache for 1 hour
+if [ -f "$CACHE_FILE" ] && [ $(find "$CACHE_FILE" -mmin -60) ]; then
+    cat "$CACHE_FILE"
+else
+    node search.mjs "$QUERY" | tee "$CACHE_FILE"
+fi
+```
+
+### Content Extraction Limitations
+
+- **Best for**: News articles, blogs, documentation
+- **Struggles with**: SPAs, JavaScript-heavy sites, streaming content
+- **Fallbacks**: Use computer-use or summarize skill for complex sites
+
+---
+
+## Advanced: Batch Research
+
+```bash
+#!/bin/bash
+# Research multiple topics in batch
+
+TOPICS=(
+    "Anthropic Claude API"
+    "LangChain documentation"
+    "vector databases comparison"
+)
+
+for TOPIC in "${TOPICS[@]}"; do
+    echo "🔍 Researching: $TOPIC"
+    node scripts/search.mjs "$TOPIC" -n 3 --content > "research-$(echo $TOPIC | tr ' ' '_').txt"
+    sleep 1  # Be nice to API
+done
+
+echo "✅ Batch research complete!"
+ls -lh research-*.txt
+```
+
+---
+
+## Comparison: brave-search vs WebSearch Tool
+
+| Feature | brave-search | WebSearch Tool |
+|---------|-------------|----------------|
+| API key required | ✅ Yes | ❌ No |
+| Setup time | 5 minutes | 0 minutes |
+| Content extraction | ✅ Yes | ❌ No |
+| Rate limits | 2,000/month free | Unknown |
+| Programmatic use | ✅ Yes | 🟡 Limited |
+| Offline caching | ✅ Yes | ❌ No |
+| JSON output | ✅ Yes | ❌ No |
+
+**When to use brave-search**:
+- Need content extraction
+- Want to cache results
+- Building automation
+- Need structured JSON output
+
+**When to use WebSearch tool**:
+- Quick one-off searches
+- No API key setup
+- Built-in to Claude
+
+---
+
+## Security Considerations
+
+- **API key**: Treat as sensitive credential (store in 1Password)
+- **Search queries**: Sent to Brave (review privacy policy)
+- **Content extraction**: Brave fetches and parses URLs (not you directly)
+- **No browser fingerprinting**: Pure API, no cookies/tracking
+- **HTTPS**: All API calls encrypted in transit
+
+---
+
+## Changelog
+
+| Date | Change |
+|------|--------|
+| 2026-01-12 | Initial deep-dive created |
+
+---
+
+*Template version: 1.0*