@miphamai/cli 0.24.4 → 0.24.6

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@miphamai/cli",
-  "version": "0.24.4",
+  "version": "0.24.6",
   "description": "Mipham Code — Multi-model open-core intelligent coding terminal by MiphamAI",
   "keywords": [
     "ai",

package/skills/standard/mipham-code-setup.SKILL.md

@@ -286,9 +286,9 @@ providers:
 
 ### 5.1 — Built-in Skills
 
-Mipham Code ships with 15 built-in skills loaded automatically:
+Mipham Code ships with 17 built-in skills loaded automatically:
 
-- **Standard (12)**: code-review, compassionate-communication, doc-generator, github-ops, memory, mipham-code-setup, security-review, self-review, superpower, tdd, web-access, web-search
+- **Standard (14)**: code-review, compassionate-communication, doc-generator, github-ops, memory, mipham-code-setup, security-review, self-review, superpower, systematic-debugging, tdd, test-driven-development, web-access, web-search
 - **Mipham (3)**: om-artifact, om-model-optimize, om-security
 
 ### 5.2 — Community Skills

package/skills/standard/web-access.SKILL.md

@@ -1,54 +1,213 @@
 ---
 name: web-access
-description: Web access, scraping, browser automation, and online content retrieval — search engines, page fetching, login-required sites, social media
-version: 1.0.0
+description: All network operations — web search, page fetching, authenticated browsing, social media scraping, dynamic page rendering. Routes to the correct tool (WebSearch/WebFetch/ComputerUse browser) based on the task.
+version: 2.0.0
+user-invocable: true
+allowed-tools:
+  - Bash
+  - WebFetch
+  - WebSearch
+  - ComputerUse
+  - Read
 ---
 
-# Web Access Skill
+# Web Access — Executable Workflow
 
-Handle all network operations: search, web scraping, authenticated browsing, and dynamic page rendering.
+**Type**: Flexible — use the decision tree to route to the right tool, then adapt to the specific site.
 
-## When to Use
+**Purpose**: All network-bound operations go through this skill. It routes the request to the correct underlying tool and handles authentication, rendering, and extraction strategy.
 
-- **Web search**: Finding current information, documentation, news
-- **Page scraping**: Extracting content from web pages
-- **Authenticated access**: Sites requiring login (via browser automation)
-- **Social media**: Content from Xiaohongshu, Weibo, Twitter, etc.
-- **Dynamic content**: JavaScript-rendered pages requiring a real browser
-- **API interaction**: REST/GraphQL endpoints with proper auth
+**Triggers**: "search for", "look up", "find information about", "fetch this URL", "scrape", "browser", "login to", "check this website", "web", "online"
 
-## Tools
+---
+
+## Phase 0: Route to the Right Tool (ALWAYS RUN FIRST)
+
+```
+User request involves network?
+├── Search engine query (find/discover/look up)?
+│   └── → WebSearch tool
+│       Query best practices: specific + versioned + technical terms
+│
+├── Read a known URL (docs/article/API)?
+│   └── → WebFetch tool
+│       HTTP auto-upgrades to HTTPS, HTML converts to markdown
+│       Cached for 15 minutes — re-fetch only if stale
+│
+├── Login-required site? JavaScript SPA? Form submission?
+│   └── → ComputerUse browser automation
+│       browser_navigate → browser_snapshot → browser_click
+│
+├── Social media (Xiaohongshu, Weibo, Twitter, etc.)?
+│   └── → ComputerUse browser (render JS, handle auth)
+│       OR → WebFetch if public page
+│
+└── API endpoint (REST/GraphQL)?
+    └── → WebFetch with prompt for structured extraction
+```
+
+---
+
+## Phase 1: Web Search
+
+Use `WebSearch` for discovery queries — finding documentation, news, troubleshooting, comparisons.
+
+### Query Construction
+
+```
+❌ "React"                           → too broad
+❌ "React problems"                  → ambiguous
+✅ "React 19 useEffect double mount fix 2026"  → specific + versioned
+✅ "Next.js 14 App Router caching behavior"     → targeted
+```
+
+### Domain Filtering
+
+Use `allowed_domains` for authoritative sources:
+
+- `docs.github.com` — GitHub docs
+- `nextjs.org` — Next.js official
+- `developer.mozilla.org` — MDN
+- `nodejs.org` — Node.js official
+
+Use `blocked_domains` to exclude noise (e.g., exclude `w3schools.com` when looking for MDN).
+
+### Verification
+
+- Cross-reference claims across 2+ independent sources
+- Prefer results from current year
+- Authority: official docs > well-known blogs > Stack Overflow > random forums
 
-### WebSearch
+### Source Attribution
 
-General web search for current information. Use for:
+Always end responses with:
 
-- Documentation lookups, API references
-- News and current events
-- Technical troubleshooting
-- Technology comparisons
+```markdown
+Sources:
 
-### WebFetch
+- [Title](URL) — brief note
+```
+
+---
+
+## Phase 2: Web Fetch
+
+Use `WebFetch` for reading a specific URL.
+
+### What it does
+
+- Auto-upgrades HTTP → HTTPS
+- Converts HTML to Markdown (headings, links, images, lists, code blocks)
+- Strips scripts, styles, nav, header, footer before conversion
+- Caches results for 15 minutes per URL
+- Detects cross-host redirects and reports them
+- Truncates content at 100K characters
+
+### Prompt Parameter
+
+Use `prompt` to guide extraction focus:
+
+```
+WebFetch: url="https://docs.example.com", prompt="find the authentication API section"
+```
+
+### When NOT to use WebFetch
+
+- Search queries → use WebSearch
+- Login-required pages → use ComputerUse browser
+- Large file downloads → use Bash + curl/wget
+- API endpoints returning JSON → WebFetch works, returns raw JSON
+
+---
 
-Fetch and parse a single URL. Use for:
+## Phase 3: Browser Automation (ComputerUse)
 
-- Reading documentation pages
-- Checking API responses
-- Extracting article content
-- Verifying links
+Use `ComputerUse` for interactive browsing — login, form submission, JavaScript rendering.
 
-### Browser Automation
+### Available Actions
 
-For sites requiring login or JavaScript rendering:
+| Action             | Purpose                                     |
+| ------------------ | ------------------------------------------- |
+| `browser_navigate` | Go to a URL                                 |
+| `browser_snapshot` | Capture accessibility tree (page structure) |
+| `browser_click`    | Click an element by UID                     |
+| `screenshot`       | Capture visible viewport                    |
+| `launch`           | Open a desktop application                  |
 
-- Authenticated sessions
-- Single-page applications (SPAs)
-- Form submissions and multi-step workflows
+### Workflow for Authenticated Sites
+
+```
+1. browser_navigate → login page
+2. browser_snapshot → find form fields (UIDs)
+3. Ask user for credentials (NEVER auto-fill)
+4. browser_click → submit
+5. browser_navigate → target page
+6. browser_snapshot → extract content
+```
+
+### Workflow for SPAs (React/Vue/Angular)
+
+```
+1. browser_navigate → SPA URL
+2. Wait 2-3 seconds (JavaScript render)
+3. browser_snapshot → extract rendered content
+```
+
+### Prerequisites
+
+- Playwright must be installed: `npm install playwright`
+- First launch opens a visible browser window (headless: false)
+
+---
+
+## Phase 4: Extraction & Synthesis
+
+After fetching content (via any method):
+
+### Content Extraction
+
+1. Identify relevant sections using the prompt/h3 headings
+2. Extract key facts, code examples, API signatures
+3. Note the source URL for attribution
+
+### Cross-Referencing
+
+1. Verify technical claims across 2+ sources
+2. Flag contradictions between sources
+3. Note version/deprecation warnings
+
+### Output Format
+
+```markdown
+## [Topic]
+
+[Key finding with source attribution]
+
+### Details
+
+[Structured content from page]
+
+Sources:
+
+- [Title](URL)
+```
+
+---
 
 ## Security Rules
 
-- Never submit credentials without explicit user approval
-- Respect robots.txt and rate limiting
+- **Never submit credentials** without explicit user approval
+- Respect `robots.txt` and rate limiting
 - Do not scrape PII or sensitive data
-- Validate all URLs against SSRF before fetching
-- Only HTTPS for remote requests
+- All URLs validated against SSRF before fetching
+- Only HTTPS for remote requests (HTTP auto-upgraded)
+- Cross-host redirects reported to caller (not silently followed)
+
+---
+
+## When NOT to Use This Skill
+
+- Pure logic / algorithmic questions (reasoning, not research)
+- Questions answerable from code already in context
+- Opinions / subjective recommendations (search for data, not consensus)
+- Downloading large binaries → use Bash + curl

package/skills/standard/web-search.SKILL.md

@@ -1,75 +1,176 @@
 ---
 name: web-search
-description: Search the web for current information — documentation, news, technical references, and troubleshooting
-version: 2.0.0
+description: Search the web for current information — documentation, news, technical references, troubleshooting, and research. Routes queries through Brave Search API with domain filtering and source verification.
+version: 3.0.0
+user-invocable: true
+allowed-tools:
+  - WebSearch
+  - WebFetch
 ---
 
-# Web Search
+# Web Search — Executable Workflow
 
-Use `WebSearch` tool to find current, accurate information from the web.
+**Type**: Flexible — follow the query construction rules strictly, then adapt verification depth to the task.
 
-## When to Use
+**Purpose**: Find accurate, current information from the web. This skill covers query formulation, domain filtering, result verification, and when to follow up with WebFetch for deep reading.
 
-- **Library/framework docs**: API references, migration guides, version-specific features
-- **Current events**: News, releases, incidents (anything after training cutoff)
-- **Troubleshooting**: Error messages, stack traces, known issues
-- **Comparisons**: Technology trade-offs, benchmark data
-- **Code examples**: Real-world usage patterns, configuration snippets
+**Triggers**: "search for", "look up", "find", "what is", "how to", "latest", "current", "news about", "documentation for", "research"
 
-## Query Best Practices
+---
 
-### Be Specific
+## Phase 0: Decide Whether to Search (ALWAYS RUN FIRST)
 
 ```
-❌ "React"                              → too broad
-❌ "React problems"                     → ambiguous
-✅ "React 19 useEffect double mount fix" → specific, versioned
-✅ "Next.js 14 App Router caching behavior 2026" → targeted
+Question involves...
+├── Current events, news, recent releases?
+│   └── YES → Search (model training cutoff limitation)
+│
+├── Library/framework documentation?
+│   └── YES → Search (version-specific, up-to-date)
+│
+├── Error messages, stack traces?
+│   └── YES → Search (known issues, fixes)
+│
+├── Technology comparisons, benchmarks?
+│   └── YES → Search (current data)
+│
+├── Pure logic, algorithms, math?
+│   └── NO → Reason directly (no external data needed)
+│
+├── Question answerable from code in context?
+│   └── NO → Use existing context (faster, no network)
+│
+└── Opinion / subjective?
+    └── MAYBE → Search for data points, not consensus
 ```
 
-### Use Technical Terms
+---
+
+## Phase 1: Construct the Query
+
+### Rules (apply in order)
+
+1. **Be specific**: include version numbers, dates, proper nouns
+2. **Use technical terms**: framework/language jargon over natural language
+3. **Include context**: OS, environment, constraints if relevant
+4. **English preferred**: technical content is richer in English
+
+### Examples
 
 ```
-❌ "how to make website fast"
+❌ "React"                                → too broad
+❌ "React problems"                       → ambiguous
+❌ "how to make website fast"             → natural language
+✅ "React 19 useEffect double mount fix"  → specific + versioned
 ✅ "Core Web Vitals LCP optimization Next.js 14"
+✅ "Prisma 5 findMany nested include filter TypeScript"
+✅ "playwright click button not working 2026"
+```
+
+### For Chinese-Language Queries
+
+Chinese queries work but yield fewer technical results:
+
+```
+✅ "React 19 useEffect 执行两次 修复"     → mixed language for best results
+✅ "Vue 3 Composition API 最佳实践 2026"
 ```
 
-### Include Version Numbers
+---
+
+## Phase 2: Filter & Verify Results
+
+### Domain Authority Tiers
 
+| Tier              | Domains                                                               | Weight  |
+| ----------------- | --------------------------------------------------------------------- | ------- |
+| **Official**      | docs.github.com, nextjs.org, nodejs.org, python.org, rust-lang.org    | Highest |
+| **Authoritative** | developer.mozilla.org, web.dev, kubernetes.io                         | High    |
+| **Trusted**       | stackoverflow.com (high-score), dev.to, medium.com (verified authors) | Medium  |
+| **Low**           | personal blogs, random forums, w3schools                              | Low     |
+
+### Use allowed_domains for targeted searches
+
+```json
+{ "query": "Next.js caching", "allowed_domains": ["nextjs.org", "github.com"] }
 ```
-❌ "prisma query"
-✅ "Prisma 5.0 findMany with nested include filter"
+
+### Use blocked_domains to exclude noise
+
+```json
+{ "query": "JavaScript array methods", "blocked_domains": ["w3schools.com"] }
 ```
 
-### Domain Filtering
+### Cross-Reference Rule
 
-Use `allowed_domains` for authoritative sources:
+- **Critical claims** (API behavior, security): 2+ independent sources
+- **Code examples**: test before recommending
+- **Version info**: check publish date (prefer current year)
 
-- `docs.github.com` — GitHub documentation
-- `nextjs.org` — Next.js official docs
-- `developer.mozilla.org` — MDN Web Docs
-- `nodejs.org` — Node.js official
+---
 
-## Verification
+## Phase 3: Deep Read (When Needed)
 
-- **Cross-reference**: Verify claims across 2+ independent sources
-- **Recency**: Prefer results from the current year; note article dates
-- **Authority**: Official docs > well-known blogs > Stack Overflow > random forums
-- **Cite sources**: Always include source URLs in responses
+After search returns results, decide whether to deep-read:
 
-## Source Attribution
+```
+Search result looks promising?
+├── Snippet answers the question fully?
+│   └── → Use snippet + cite source (done)
+│
+├── Need code examples / detailed API docs?
+│   └── → WebFetch the page URL
+│       Use prompt to focus extraction
+│
+├── Multiple sources needed for verification?
+│   └── → WebFetch top 2-3 results
+│       Cross-reference and flag contradictions
+│
+└── Page is JavaScript SPA / login-walled?
+    └── → Delegate to web-access skill (ComputerUse browser)
+```
+
+---
 
-After answering from search results, end with:
+## Phase 4: Report Results
+
+### Format
 
 ```markdown
+## [Topic]
+
+[Answer with inline citations]
+
+### Details (if deep-read was done)
+
+[Structured content from fetched pages]
+
 Sources:
 
-- [Title](URL) — brief note
-- [Title](URL) — brief note
+- [Title](URL) — [1-sentence note on what was found there]
+- [Title](URL) — [1-sentence note]
 ```
 
-## When NOT to Search
+### Attribution Rules
+
+- Always include source URLs
+- Note if a source is official docs vs community
+- Flag outdated content (e.g., "article from 2024, may be stale")
+- Distinguish between facts (need citation) and reasoning (your own)
+
+---
+
+## Search API Configuration
+
+Web search uses **Brave Search API** (free tier: 2,000 queries/month).
+
+If search returns "not configured":
+
+1. Get a free API key at https://brave.com/search/api/
+2. Set: `export BRAVE_API_KEY="BSA..."`
+3. Restart Mipham Code
+
+Alternatives (additional API keys supported):
 
-- Pure logic or algorithmic questions (reasoning, not research)
-- Questions answerable from code already in context
-- Opinions and subjective recommendations (search for data, not consensus)
+- `TAVILY_API_KEY` — https://tavily.com
+- `SERPAPI_API_KEY` — https://serpapi.com

package/src/tools/network/web-fetch.ts

@@ -1,21 +1,180 @@
 import type { ToolDefinition } from '../../shared/index.ts'
 import { validateUrl } from '../../security/url'
 
+// ── In-memory cache (15-min TTL per URL) ──
+
+interface CacheEntry {
+  content: string
+  timestamp: number
+}
+
+const CACHE_TTL = 15 * 60 * 1000 // 15 minutes
+const cache = new Map<string, CacheEntry>()
+
+function getCached(url: string): string | undefined {
+  const entry = cache.get(url)
+  if (!entry) return undefined
+  if (Date.now() - entry.timestamp > CACHE_TTL) {
+    cache.delete(url)
+    return undefined
+  }
+  return entry.content
+}
+
+function setCache(url: string, content: string): void {
+  // Evict oldest entries if cache grows too large (max 200 URLs)
+  if (cache.size >= 200) {
+    const oldest = [...cache.entries()].sort((a, b) => a[1].timestamp - b[1].timestamp)
+    for (let i = 0; i < 20 && oldest[i]; i++) {
+      cache.delete(oldest[i]![0])
+    }
+  }
+  cache.set(url, { content, timestamp: Date.now() })
+}
+
+// ── HTTP→HTTPS upgrade ──
+
+function upgradeToHttps(url: string): string {
+  if (url.startsWith('http://')) {
+    return url.replace('http://', 'https://')
+  }
+  return url
+}
+
+// ── HTML → Markdown conversion ──
+
+function htmlToMarkdown(html: string, baseUrl: string): string {
+  let text = html
+
+  // Remove scripts, styles, nav, header, footer
+  text = text.replace(/<script[^>]*>[\s\S]*?<\/script>/gi, '')
+  text = text.replace(/<style[^>]*>[\s\S]*?<\/style>/gi, '')
+  text = text.replace(/<nav[^>]*>[\s\S]*?<\/nav>/gi, '')
+  text = text.replace(/<header[^>]*>[\s\S]*?<\/header>/gi, '')
+  text = text.replace(/<footer[^>]*>[\s\S]*?<\/footer>/gi, '')
+
+  // Convert headings
+  text = text.replace(/<h1[^>]*>([\s\S]*?)<\/h1>/gi, (_, c) => `\n# ${stripTags(c).trim()}\n`)
+  text = text.replace(/<h2[^>]*>([\s\S]*?)<\/h2>/gi, (_, c) => `\n## ${stripTags(c).trim()}\n`)
+  text = text.replace(/<h3[^>]*>([\s\S]*?)<\/h3>/gi, (_, c) => `\n### ${stripTags(c).trim()}\n`)
+  text = text.replace(/<h4[^>]*>([\s\S]*?)<\/h4>/gi, (_, c) => `\n#### ${stripTags(c).trim()}\n`)
+  text = text.replace(/<h5[^>]*>([\s\S]*?)<\/h5>/gi, (_, c) => `\n##### ${stripTags(c).trim()}\n`)
+  text = text.replace(/<h6[^>]*>([\s\S]*?)<\/h6>/gi, (_, c) => `\n###### ${stripTags(c).trim()}\n`)
+
+  // Convert links: <a href="...">text</a> → [text](url)
+  text = text.replace(/<a[^>]*href=["']([^"']*)["'][^>]*>([\s\S]*?)<\/a>/gi, (_, href, content) => {
+    const resolved = resolveUrl(href, baseUrl)
+    return `[${stripTags(content).trim()}](${resolved})`
+  })
+
+  // Convert images: <img ... src="..." ...> → ![alt](url)
+  text = text.replace(
+    /<img[^>]*src=["']([^"']*)["'][^>]*alt=["']([^"']*)["'][^>]*\/?>/gi,
+    (_, src, alt) => {
+      const resolved = resolveUrl(src, baseUrl)
+      return `![${alt || ''}](${resolved})`
+    },
+  )
+  text = text.replace(/<img[^>]*src=["']([^"']*)["'][^>]*\/?>/gi, (_, src) => {
+    const resolved = resolveUrl(src, baseUrl)
+    return `![](${resolved})`
+  })
+
+  // Convert lists
+  text = text.replace(/<li[^>]*>([\s\S]*?)<\/li>/gi, (_, c) => `- ${stripTags(c).trim()}\n`)
+  text = text.replace(/<\/ul>/gi, '\n')
+  text = text.replace(/<\/ol>/gi, '\n')
+
+  // Convert code blocks
+  text = text.replace(/<pre[^>]*><code[^>]*>([\s\S]*?)<\/code><\/pre>/gi, (_, c) => {
+    const decoded = c
+      .replace(/&lt;/g, '<')
+      .replace(/&gt;/g, '>')
+      .replace(/&amp;/g, '&')
+      .replace(/&quot;/g, '"')
+    return `\n\`\`\`\n${decoded.trim()}\n\`\`\`\n`
+  })
+  text = text.replace(/<code[^>]*>([\s\S]*?)<\/code>/gi, (_, c) => `\`${c.trim()}\``)
+
+  // Convert inline formatting
+  text = text.replace(/<strong[^>]*>([\s\S]*?)<\/strong>/gi, '**$1**')
+  text = text.replace(/<b[^>]*>([\s\S]*?)<\/b>/gi, '**$1**')
+  text = text.replace(/<em[^>]*>([\s\S]*?)<\/em>/gi, '*$1*')
+  text = text.replace(/<i[^>]*>([\s\S]*?)<\/i>/gi, '*$1*')
+
+  // Convert paragraph and line break tags
+  text = text.replace(/<br\s*\/?>/gi, '\n')
+  text = text.replace(/<\/p>/gi, '\n\n')
+  text = text.replace(/<p[^>]*>/gi, '')
+
+  // Strip remaining HTML tags
+  text = text.replace(/<[^>]*>/g, '')
+
+  // Decode HTML entities
+  text = text.replace(/&lt;/g, '<')
+  text = text.replace(/&gt;/g, '>')
+  text = text.replace(/&amp;/g, '&')
+  text = text.replace(/&quot;/g, '"')
+  text = text.replace(/&#x27;/g, "'")
+  text = text.replace(/&#39;/g, "'")
+  text = text.replace(/&nbsp;/g, ' ')
+
+  // Collapse whitespace (preserve intentional line breaks)
+  text = text
+    .split('\n')
+    .map((l) => l.replace(/\s+/g, ' ').trim())
+    .join('\n')
+  text = text.replace(/\n{3,}/g, '\n\n')
+
+  return text.trim()
+}
+
+function stripTags(html: string): string {
+  return html.replace(/<[^>]*>/g, '')
+}
+
+function resolveUrl(href: string, baseUrl: string): string {
+  try {
+    return new URL(href, baseUrl).toString()
+  } catch {
+    return href
+  }
+}
+
+// ── Tool Definition ──
+
 export const webFetchTool: ToolDefinition = {
   name: 'WebFetch',
-  description: 'Fetch content from a URL and process into markdown.',
+  description:
+    'Fetches a URL, converts the page to markdown. HTTP is upgraded to HTTPS. Cross-host redirects are returned to the caller. Responses are cached for 15 minutes per URL.',
   category: 'network',
   permission: 'auto',
   parameters: {
     type: 'object',
     properties: {
       url: { type: 'string', format: 'uri', description: 'URL to fetch' },
-      prompt: { type: 'string', description: 'Prompt to run on fetched content' },
+      prompt: {
+        type: 'string',
+        description:
+          'What to extract from the page (e.g., "find the API docs for authentication"). The tool returns the full page; the prompt helps focus extraction.',
+      },
     },
     required: ['url'],
   },
   async execute(params, _ctx) {
-    const url = params.url as string
+    const rawUrl = params.url as string
+    const prompt = (params.prompt as string) || ''
+    const url = upgradeToHttps(rawUrl)
+
+    // Check cache
+    const cached = getCached(url)
+    if (cached) {
+      return {
+        success: true,
+        content: cached,
+        metadata: { cached: true, url },
+      }
+    }
 
     // SSRF protection: validate URL before fetching
     const validationError = validateUrl(url)
@@ -28,25 +187,29 @@ export const webFetchTool: ToolDefinition = {
       const timer = setTimeout(() => controller.abort(), 30_000) // 30s timeout
 
       const response = await fetch(url, {
-        headers: { 'User-Agent': 'Mipham-Code/0.5.0' },
+        headers: {
+          'User-Agent': 'Mipham-Code/0.24.0',
+          Accept: 'text/html,application/xhtml+xml,*/*',
+        },
         redirect: 'follow',
         signal: controller.signal,
       })
 
       clearTimeout(timer)
 
-      if (!response.ok) {
-        return {
-          success: false,
-          content: '',
-          error: `HTTP ${response.status}: ${response.statusText}`,
+      // Cross-host redirect reporting
+      if (response.url) {
+        const finalHost = new URL(response.url).hostname
+        const originalHost = new URL(url).hostname
+        if (finalHost !== originalHost) {
+          return {
+            success: true,
+            content: `Redirected to: ${response.url}\n\nFetch from this URL directly to retrieve content.`,
+            metadata: { redirected: true, originalUrl: url, finalUrl: response.url },
+          }
         }
-      }
 
-      // SSRF defense: re-validate the resolved URL after redirects.
-      // `redirect: 'follow'` may have landed on an internal IP — check again.
-      // Guard: response.url may not exist in some environments (e.g. mocked fetch).
-      if (response.url) {
+        // SSRF defense: re-validate the resolved URL after redirects
         const redirectError = validateUrl(response.url)
         if (redirectError) {
           return {
@@ -57,14 +220,43 @@ export const webFetchTool: ToolDefinition = {
         }
       }
 
-      const html = await response.text()
-      // Simple HTML-to-text extraction
-      const text = html
-        .replace(/<[^>]*>/g, ' ')
-        .replace(/\s+/g, ' ')
-        .trim()
-        .slice(0, 50_000)
-      return { success: true, content: text }
+      // Determine content type; only convert HTML to markdown
+      const contentType = response.headers.get('content-type') || ''
+      const isHtml = contentType.includes('text/html') || contentType.includes('application/xhtml')
+
+      if (!response.ok) {
+        return {
+          success: false,
+          content: '',
+          error: `HTTP ${response.status}: ${response.statusText}`,
+        }
+      }
+
+      let content: string
+
+      if (isHtml) {
+        const html = await response.text()
+        const baseUrl = response.url || url
+        content = htmlToMarkdown(html, baseUrl)
+      } else {
+        // Plain text / JSON / etc. — return as-is
+        content = await response.text()
+      }
+
+      // Truncate to 100K characters
+      if (content.length > 100_000) {
+        content = content.slice(0, 100_000) + '\n\n... (truncated)'
+      }
+
+      // Cache the result
+      setCache(url, content)
+
+      // Include prompt context if provided
+      const header = prompt
+        ? `── WebFetch: ${url} ──\nPrompt: ${prompt}\n\n`
+        : `── WebFetch: ${url} ──\n\n`
+
+      return { success: true, content: header + content, metadata: { url, size: content.length } }
     } catch (err) {
       const message =
         err instanceof Error && err.name === 'AbortError'

package/src/tools/network/web-search.ts

@@ -1,8 +1,85 @@
 import type { ToolDefinition } from '../../shared/index.ts'
 
+/**
+ * Brave Search API integration.
+ *
+ * API: https://api.search.brave.com/res/v1/web/search
+ * Free tier: 2000 queries/month (no credit card required)
+ * Sign up: https://brave.com/search/api/
+ *
+ * Set BRAVE_API_KEY in your environment to enable web search.
+ * Falls back to a helpful message when the key is not configured.
+ */
+
+const BRAVE_API = 'https://api.search.brave.com/res/v1/web/search'
+const MAX_RESULTS = 10
+
+interface BraveWebResult {
+  title: string
+  url: string
+  description: string
+}
+
+interface BraveAPIResponse {
+  web?: {
+    results?: BraveWebResult[]
+  }
+}
+
+function getApiKey(): string | undefined {
+  return process.env.BRAVE_API_KEY || undefined
+}
+
+function filterByDomains(
+  results: BraveWebResult[],
+  allowed?: string[],
+  blocked?: string[],
+): BraveWebResult[] {
+  let filtered = results
+
+  if (allowed && allowed.length > 0) {
+    filtered = filtered.filter((r) => {
+      try {
+        const host = new URL(r.url).hostname
+        return allowed.some((d) => host === d || host.endsWith('.' + d))
+      } catch {
+        return false
+      }
+    })
+  }
+
+  if (blocked && blocked.length > 0) {
+    filtered = filtered.filter((r) => {
+      try {
+        const host = new URL(r.url).hostname
+        return !blocked.some((d) => host === d || host.endsWith('.' + d))
+      } catch {
+        return true
+      }
+    })
+  }
+
+  return filtered
+}
+
+function formatResults(results: BraveWebResult[]): string {
+  if (results.length === 0) {
+    return 'No results found.'
+  }
+
+  return results
+    .map((r, i) => {
+      const title = r.title.replace(/\n/g, ' ').trim()
+      const desc = (r.description || '').replace(/\n/g, ' ').trim()
+      return `${i + 1}. **${title}**\n   ${r.url}\n   ${desc}`
+    })
+    .join('\n\n')
+}
+
 export const webSearchTool: ToolDefinition = {
   name: 'WebSearch',
-  description: 'Search the web. Returns result titles and URLs.',
+  description:
+    'Search the web via Brave Search API. Returns result blocks with titles, URLs, and descriptions. Set BRAVE_API_KEY to enable.',
   category: 'network',
   permission: 'auto',
   parameters: {
@@ -24,9 +101,106 @@ export const webSearchTool: ToolDefinition = {
   },
   async execute(params, _ctx) {
     const query = params.query as string
-    return {
-      success: true,
-      content: `WebSearch query: "${query}". Results would appear here when search API is configured.`,
+    const allowedDomains = params.allowed_domains as string[] | undefined
+    const blockedDomains = params.blocked_domains as string[] | undefined
+
+    const apiKey = getApiKey()
+    if (!apiKey) {
+      return {
+        success: true,
+        content: [
+          '── Web Search (not configured) ──',
+          '',
+          `Query: "${query}"`,
+          '',
+          'Web search is not yet configured. To enable it:',
+          '',
+          '1. Get a free API key at https://brave.com/search/api/',
+          '   (2,000 queries/month, no credit card required)',
+          '',
+          '2. Set the key in your environment:',
+          '   export BRAVE_API_KEY="BSA..."',
+          '',
+          '3. Restart Mipham Code and try again.',
+          '',
+          'Alternatives (add API key for any):',
+          '   TAVILY_API_KEY   — https://tavily.com',
+          '   SERPAPI_API_KEY  — https://serpapi.com',
+        ].join('\n'),
+      }
+    }
+
+    try {
+      const url = new URL(BRAVE_API)
+      url.searchParams.set('q', query)
+      url.searchParams.set('count', String(MAX_RESULTS))
+
+      const controller = new AbortController()
+      const timer = setTimeout(() => controller.abort(), 15_000) // 15s timeout
+
+      const response = await fetch(url.toString(), {
+        headers: {
+          Accept: 'application/json',
+          'Accept-Encoding': 'gzip',
+          'X-Subscription-Token': apiKey,
+        },
+        signal: controller.signal,
+      })
+
+      clearTimeout(timer)
+
+      if (!response.ok) {
+        const body = await response.text().catch(() => '')
+        // 429 = rate limit, 401 = bad key
+        if (response.status === 429) {
+          return {
+            success: false,
+            content: '',
+            error:
+              'Brave Search API rate limit reached (2,000/month on free tier). Try again later.',
+          }
+        }
+        if (response.status === 401) {
+          return {
+            success: false,
+            content: '',
+            error: 'Invalid BRAVE_API_KEY. Check your key at https://brave.com/search/api/.',
+          }
+        }
+        return {
+          success: false,
+          content: '',
+          error: `Brave Search API error (${response.status}): ${body.slice(0, 200)}`,
+        }
+      }
+
+      const data = (await response.json()) as BraveAPIResponse
+      const rawResults = data.web?.results || []
+
+      const filtered = filterByDomains(rawResults, allowedDomains, blockedDomains)
+      const formatted = formatResults(filtered)
+
+      return {
+        success: true,
+        content: [
+          `── Web Search: "${query}" ──`,
+          `${filtered.length} result(s)`,
+          '',
+          formatted,
+          '',
+          rawResults.length > filtered.length
+            ? `(${rawResults.length - filtered.length} result(s) filtered by domain rules)`
+            : '',
+        ]
+          .filter(Boolean)
+          .join('\n'),
+      }
+    } catch (err) {
+      const message =
+        err instanceof Error && err.name === 'AbortError'
+          ? 'Search timed out (15s). Try a more specific query.'
+          : `Search failed: ${String(err)}`
+      return { success: false, content: '', error: message }
     }
   },
 }