sfcc-dev-mcp 1.0.15 → 1.0.19

package/.github/pull_request_template.md

@@ -50,7 +50,7 @@
 - [ ] README.md updated (if user-facing changes)
 - [ ] copilot-instructions.md updated (if architectural changes)
 - [ ] API documentation updated
-- [ ] Best practices guides updated
+- [ ] Agent skills updated (ai-instructions/skills)
 - [ ] Code comments added/updated
 - [ ] No documentation changes needed
 

package/.github/skills/agent-authoring/SKILL.md

@@ -0,0 +1,66 @@
+---
+name: agent-authoring
+description: How to design, structure, and validate custom Copilot agents for this workspace.
+---
+
+# Agent Authoring Skill
+Guidelines for creating and maintaining custom GitHub Copilot agents (.agent.md) tailored to this repository.
+
+## When to Use
+- You need a new Copilot agent focused on a workflow (e.g., Storybook tests, log triage).
+- You want to adjust tools/persona for existing agents.
+- You are adding workspace-level agents under .github/agents or profile-level agents for reuse.
+
+## Quick Start
+1) Create `.github/agents/<agent-name>.agent.md` (VS Code auto-detects this folder/extension).
+2) Add YAML frontmatter with only supported fields: `name`, `description`, `tools`, `target`, `infer`, `mcp-servers`, `handoffs`.
+3) In the body, write concise instructions: task focus, guardrails, run/validate steps, and handoffs.
+4) Keep it under ~200-300 lines; link out to docs instead of pasting everything.
+
+## Frontmatter Cheatsheet
+- `name`: Display name; defaults to file name if omitted.
+- `description`: Placeholder text in chat input; keep short.
+- `tools`: List allowed tools. For MCP servers, use `<server>/*` to include all tools or enumerate specific ones.
+- `target`: `github-copilot` for Copilot agents (default), `vscode` for VS Code target.
+- `infer`: Allow subagent use (default true). Set false if you want to restrict subagents.
+- `mcp-servers`: For Copilot agents, point to MCP server configs (paths to server JSON). Use only if you need to pin servers.
+- `handoffs`: Optional array of next-step buttons `{ label, agent, prompt?, send? }`.
+
+## Body Structure (recommended)
+- **Purpose and scope**: What this agent does and what it must not do.
+- **Operating defaults**: Where to run commands, what to inspect first, preferred scripts.
+- **Playbooks**: Ordered steps for common flows (run, debug, fix, validate).
+- **Guardrails**: Safe-edit rules (scope, no unrelated changes, ask before destructive ops).
+- **Validation**: How to verify work (tests/linters), how to summarize outputs.
+- **Links**: Point to repo docs or skills for deeper detail.
+
+## Tool Selection Patterns
+- Editing agent: include `workspace`, `terminal`, `search`; add `fetch` for docs; add MCP toolsets only if required.
+- Read-only agent: omit editing tools; keep `fetch`/`search`.
+- Narrow the tool list to prevent unsafe actions (e.g., no `terminal` for planning-only agents).
+
+## Handoffs
+Use handoffs to guide sequential flows (e.g., Plan -> Implement -> Review). Keep labels short and make prompts actionable. Use `send: false` unless you want auto-submit.
+
+## Quality Bar
+- Minimal, task-focused instructions; avoid boilerplate.
+- No unsupported frontmatter fields (reject `argument-hint`, etc.).
+- Prefer scoped commands over global installs; mention working directory if non-root.
+- Add guardrails for destructive commands (never propose `git reset --hard`).
+- Keep instructions ASCII and concise; use fenced code blocks for commands.
+
+## Validation Checklist
+- [ ] Supported frontmatter only; fields spelled correctly.
+- [ ] Tools list matches the agent’s intent; no over-broad capabilities.
+- [ ] Clear run/validate steps with default commands.
+- [ ] Guardrails called out.
+- [ ] Links resolve (relative paths within repo or official docs).
+
+## Examples
+- Location: `.github/agents/storybook-interaction-tests.agent.md` (interaction test fixer playbook).
+- Planning agent: frontmatter with `tools: ['fetch', 'search']`, body instructs to gather context and produce a plan, handoff to implementation agent.
+
+## Maintenance Tips
+- Keep agents in sync with repo workflows (scripts, test commands). Update after script changes.
+- Remove stale handoffs when target agents are renamed or removed.
+- Prefer additive changes; avoid reformatting existing agents without need.

package/.github/skills/mcp-log-debugging/SKILL.md

@@ -0,0 +1,69 @@
+````skill
+---
+name: mcp-log-debugging
+description: How to capture and read sfcc-dev-mcp server logs for debugging tool responses, preflight notices, and OCAPI/WebDAV calls. Use this when investigating missing notices or unexpected tool output.
+---
+
+# MCP Log Debugging Skill
+
+Use this skill to collect and analyze the MCP server logs that are written outside the workspace (in the OS temp directory). Logs are always file-based and debug logging is opt-in.
+
+## Enable Debug Logs
+
+Start the server with debug enabled to get `[DEBUG]` entries:
+
+```bash
+node dist/main.js --debug true
+```
+
+If you run from source with ts-node or a runner wrapper, pass `--debug true` after your command. Debug mode is sticky per process; restart the server to toggle it.
+
+## Locate the Log Directory
+
+Logs live under the OS temp dir in `sfcc-mcp-logs/`. Print the absolute path:
+
+```bash
+node -e "const os=require('os');const path=require('path');console.log(path.join(os.tmpdir(),'sfcc-mcp-logs'));"
+```
+
+Key files:
+- `sfcc-mcp-debug.log` – debug/info
+- `sfcc-mcp-error.log` – errors
+
+## Capture Logs into Workspace (safe for MCP tools)
+
+Because the log directory is outside the workspace, copy a tail into the repo for inspection:
+
+```bash
+mkdir -p logs && \
+  tail -n 200 "$(node -e "const os=require('os');const path=require('path');console.log(path.join(os.tmpdir(),'sfcc-mcp-logs','sfcc-mcp-debug.log'));")" \
+  > logs/sfcc-mcp-debug-tail.txt
+```
+
+Then open `logs/sfcc-mcp-debug-tail.txt` inside the workspace. Adjust `-n` as needed.
+
+## What to Look For
+
+- `[InstructionAdvisor] Status` entries: shows workspaceRoot, hasAgents, hasSkills, detectedSkillsDir, missingSkills
+- "Issuing" messages: confirms preflight notice generation path
+- `Full response for <tool>`: verifies whether the notice was merged into the `content`
+- OCAPI/WebDAV client lines: token reuse, request URLs, timings
+- Handler timings: `Performance: <tool> took Nms`
+
+## Common Debug Flows
+
+1) **Missing preflight notice**
+   - Ensure debug is on, capture tail, search for `InstructionAdvisor`.
+   - If `hasAgents=true && hasSkills=true`, notice is suppressed (expected).
+   - If `workspaceRoot` is undefined, use `sync_agent_instructions` with `destinationType:"temp"` or set `preferredRoot`.
+
+2) **Tool response shape surprises**
+   - Check `Full response for <tool>` preview to confirm the server returned the expected content before client UI formatting.
+
+3) **OCAPI authentication issues**
+   - Look for `Attempting to get access token` / `Using existing valid token` pairs and request URLs.
+
+## Safety
+
+Logger masks common secrets (passwords, client secrets, tokens). Still avoid sharing entire logs publicly; redact hostnames and business data when necessary.
+````

package/.github/skills/mcp-programmatic-testing/SKILL.md

@@ -0,0 +1,142 @@
+---
+name: mcp-programmatic-testing
+description: Guide for writing JavaScript programmatic MCP server tests using Node.js test runner. Use this when asked to create complex multi-step MCP tests or debug programmatic test files (*.programmatic.test.js).
+---
+
+# MCP Programmatic Testing Skill
+
+Use this skill when writing or debugging `*.programmatic.test.js` files for Model Context Protocol servers.
+
+## When to Use Programmatic Tests
+
+- Complex business logic requiring code execution
+- Multi-step workflows with state management  
+- Dynamic test case generation
+- Integration with existing JavaScript test suites
+
+For basic functional testing, prefer YAML tests (see `mcp-yaml-testing` skill).
+
+## Basic Test Structure
+
+```javascript
+import { test, describe, before, after, beforeEach } from 'node:test';
+import { strict as assert } from 'node:assert';
+import { connect } from 'mcp-aegis';
+
+describe('[SERVER_NAME] Programmatic Tests', () => {
+  let client;
+
+  before(async () => {
+    client = await connect('./aegis.config.docs-only.json');
+  });
+
+  after(async () => {
+    if (client?.connected) {
+      await client.disconnect();
+    }
+  });
+
+  beforeEach(() => {
+    // CRITICAL: Clear buffers to prevent test interference
+    client.clearAllBuffers();
+  });
+
+  test('should list available tools', async () => {
+    const tools = await client.listTools();
+    assert.ok(Array.isArray(tools), 'Tools should be array');
+    assert.ok(tools.length > 0, 'Should have at least one tool');
+  });
+
+  test('should execute tool successfully', async () => {
+    const result = await client.callTool('[TOOL_NAME]', { param: 'value' });
+    assert.ok(result.content, 'Should return content');
+    assert.equal(result.isError, false, 'Should not be error');
+  });
+});
+```
+
+## Critical: Buffer Management
+
+**Always include `beforeEach` with buffer clearing** to prevent test flakiness:
+
+```javascript
+beforeEach(() => {
+  client.clearAllBuffers();  // Recommended - comprehensive
+  // OR: client.clearStderr();  // Minimum - stderr only
+});
+```
+
+Without this, stderr from one test can leak into subsequent tests, causing random failures.
+
+## Critical: No Concurrent Requests
+
+**Never use `Promise.all()` with MCP requests**. MCP uses single stdio with shared buffers:
+
+```javascript
+// ❌ NEVER DO THIS
+const results = await Promise.all(tools.map(t => client.callTool(t.name, {})));
+
+// ✅ ALWAYS DO THIS
+const results = [];
+for (const tool of tools) {
+  const result = await client.callTool(tool.name, {});
+  results.push(result);
+}
+```
+
+## Test Execution Commands
+
+```bash
+# Run individual MCP programmatic test
+node --test tests/mcp/node/specific-test.programmatic.test.js
+
+# Run all MCP programmatic tests
+npm run test:mcp:node
+
+# Watch mode for development
+node --test --watch tests/mcp/node/*.programmatic.test.js
+
+# ❌ WRONG: Don't use npm test with individual files
+# npm test -- tests/mcp/node/specific-test.programmatic.test.js  # Runs Jest!
+```
+
+## API Reference
+
+```javascript
+// Connection
+const client = await connect('./aegis.config.json');
+await client.disconnect();
+
+// Tool operations
+const tools = await client.listTools();
+const result = await client.callTool(name, arguments);
+
+// Debugging
+const stderr = client.getStderr();
+client.clearStderr();
+client.clearAllBuffers();
+```
+
+## Performance Testing Guidelines
+
+**Performance testing is discouraged in programmatic tests** due to CI variability.
+
+If timing validation is required, use lenient thresholds:
+
+```javascript
+// ❌ Too strict for CI
+assert.ok(duration < 50, 'Should be under 50ms');
+
+// ✅ CI-friendly
+assert.ok(duration < 500, 'Should be under 500ms');
+```
+
+## Discovery Before Testing
+
+Use aegis query to discover response formats before writing tests:
+
+```bash
+npx aegis query search_sfcc_classes 'query:catalog' --config ./aegis.config.docs-only.json
+```
+
+Then use the discovered format to write assertions.

package/.github/skills/mcp-yaml-testing/SKILL.md

@@ -0,0 +1,136 @@
+---
+name: mcp-yaml-testing
+description: Guide for writing and debugging YAML-based declarative MCP server tests using Aegis. Use this when asked to create, fix, or debug MCP YAML tests (*.test.mcp.yml files).
+---
+
+# MCP YAML Testing Skill
+
+Use this skill when writing or debugging `*.test.mcp.yml` files for Model Context Protocol servers.
+
+## Golden Rule: Discovery First
+
+**CRITICAL**: Before writing ANY YAML test, discover actual response formats:
+
+```bash
+# Test successful execution
+npx aegis query [tool_name] '[valid_params]' --config "./aegis.config.docs-only.json"
+
+# Test failure scenarios  
+npx aegis query [tool_name] '[invalid_params]' --config "./aegis.config.docs-only.json"
+npx aegis query [tool_name] '' --config "./aegis.config.docs-only.json"  # Empty params
+```
+
+## Parameter Formats
+
+### Pipe Format (Recommended)
+```bash
+npx aegis query read_file 'path:test.txt' --config "./aegis.config.docs-only.json"
+npx aegis query calculator 'operation:add|a:5|b:3' --config "./aegis.config.docs-only.json"
+
+# Nested via dot notation
+npx aegis query api_client 'config.host:localhost|config.port:8080' --config "./aegis.config.docs-only.json"
+```
+
+### JSON Format (Complex Structures)
+```bash
+npx aegis query complex_tool '{"config": {"host": "localhost"}, "data": [1,2,3]}' --config "./aegis.config.docs-only.json"
+```
+
+## Basic Test Structure
+
+```yaml
+description: "Test suite description"
+tests:
+  - it: "Test description"
+    request:
+      jsonrpc: "2.0"
+      id: "unique-id"
+      method: "tools/list"  # or "tools/call"
+      params: {}
+    expect:
+      response:
+        jsonrpc: "2.0"
+        id: "unique-id"
+        result: {}
+      stderr: "toBeEmpty"
+```
+
+## Common Pattern Reference
+
+```yaml
+# TYPE VALIDATION
+result:
+  tools: "match:type:array"
+  count: "match:type:number"
+
+# STRING PATTERNS
+result:
+  text: "match:contains:substring"
+  name: "match:startsWith:prefix"
+  pattern: "match:regex:\\d{4}-\\d{2}-\\d{2}"  # Double-escape backslashes!
+
+# ARRAY PATTERNS
+result:
+  tools: "match:arrayLength:3"
+  tools:
+    match:arrayElements:
+      match:partial:  # Flexible - only validate specified fields
+        name: "match:regex:^[a-z_]+$"
+        description: "match:contains:tool"
+
+# NEGATION
+result:
+  tools: "match:not:arrayLength:0"
+  text: "match:not:contains:error"
+```
+
+## Common Mistakes to Avoid
+
+```yaml
+# ❌ WRONG - Duplicate YAML keys
+result:
+  tools: "match:arrayLength:1"
+  tools: ["read_file"]  # OVERWRITES above!
+
+# ❌ WRONG - Missing double backslash
+result:
+  text: "match:regex:\d+"  # Wrong
+  text: "match:regex:\\d+"  # Correct
+
+# ❌ WRONG - Assuming structure without discovery
+result:
+  tools: "match:arrayLength:3"  # Did you verify this exists?
+```
+
+## Test Execution Commands
+
+```bash
+# Run YAML tests (docs-only mode)
+npm run test:mcp:yaml
+
+# Run YAML tests (full mode with credentials)
+npm run test:mcp:yaml:full
+
+# Run specific test file
+npx aegis "tests/mcp/yaml/specific.test.mcp.yml" --config "./aegis.config.docs-only.json"
+
+# Debug mode
+npx aegis "tests/**/*.test.mcp.yml" --config "./aegis.config.docs-only.json" --verbose --debug
+```
+
+## Discovery Examples for This Project
+
+```bash
+# List available tools
+npx aegis query --config ./aegis.config.docs-only.json
+
+# Documentation tools
+npx aegis query search_sfcc_classes 'query:catalog' --config ./aegis.config.docs-only.json
+npx aegis query get_sfcc_class_info 'className:dw.catalog.Product' --config ./aegis.config.docs-only.json
+
+# Agent instructions / skills bootstrap
+npx aegis query sync_agent_instructions 'destinationType:temp|dryRun:true' --config ./aegis.config.docs-only.json
+
+# Full mode tools (requires credentials)
+npx aegis query get_system_object_definitions --config ./aegis.config.with-dw.json
+```

package/.github/skills/salesforce-developer-site-scraper/SKILL.md

@@ -0,0 +1,102 @@
+---
+name: salesforce-developer-site-scraper
+description: 'Scrape Salesforce Developer documentation into clean Markdown using headless Chromium, Readability, and the docs content API fallback. Use when content is async or blocked by OneTrust cookie banners.'
+license: Forward Proprietary
+compatibility: VS Code 1.x+, Node.js 18+
+---
+
+# Salesforce Developer Site Scraper
+
+Use this skill to capture Salesforce Developer documentation pages as clean Markdown even when content loads asynchronously or is blocked by OneTrust cookie banners.
+
+## When to Use This Skill
+
+- A Salesforce Developer doc page renders key content after async requests.
+- A OneTrust cookie banner hides content until consent is accepted.
+- You need a readable Markdown snapshot for Apex, LWC, or platform docs.
+- NOT for: high-volume crawling or scraping behind access restrictions.
+
+## Prerequisites
+
+- Node.js 18+
+- npm dependencies for the script (see below)
+
+## How to Use
+
+### 1) Install script dependencies (if not done already)
+
+```bash
+npm install playwright @mozilla/readability jsdom turndown
+```
+
+### 2) Run the script
+
+```bash
+node skills/salesforce-developer-site-scraper/scripts/scrape-to-markdown.js \
+  --url "https://developer.salesforce.com/docs/atlas.en-us.apexcode.meta/apexcode/apex_intro.htm" \
+  --out "./artifacts/online-research/apex_intro.md" \
+  --consent-selector "#onetrust-accept-btn-handler" \
+  --wait 2000
+```
+
+## Script Options
+
+| Option | Required | Description |
+| --- | --- | --- |
+| `--url` | Yes | Target URL to fetch and extract. |
+| `--out` | Yes | Output Markdown file path. |
+| `--consent-selector` | No | CSS or text selector for a cookie banner accept button. |
+| `--wait` | No | Milliseconds to wait after navigation or consent click. |
+| `--content-selector` | No | Extract only this element instead of Readability parsing. |
+| `--remove-selectors` | No | Comma-separated selectors to remove before extraction. |
+| `--cookie` | No | Consent/session cookie, e.g. `name=value;domain=example.com;path=/`. |
+| `--storage-state` | No | Playwright storage state JSON file to reuse consent/session. |
+| `--timeout` | No | Navigation timeout in ms (default 45000). |
+| `--no-default-removals` | No | Disable default cookie/consent element removals. |
+
+## Compliance Notes
+
+- Respect robots.txt and site terms before scraping.
+- Use consent cookies or storage state only when you have permission.
+- Avoid collecting personal data unless you have a legal basis.
+
+## Examples
+
+### Example: Reuse a consent state
+
+```bash
+node skills/salesforce-developer-site-scraper/scripts/scrape-to-markdown.js \
+  --url "https://developer.salesforce.com/docs/atlas.en-us.apexcode.meta/apexcode/apex_intro.htm" \
+  --out "./artifacts/online-research/apex_intro.md" \
+  --storage-state "./artifacts/online-research/consent-state.json"
+```
+
+### Example: Extract a specific content container
+
+```bash
+node skills/salesforce-developer-site-scraper/scripts/scrape-to-markdown.js \
+  --url "https://developer.salesforce.com/docs/atlas.en-us.apexcode.meta/apexcode/apex_intro.htm" \
+  --out "./artifacts/online-research/apex_intro.md" \
+  --content-selector "main article"
+```
+
+## Troubleshooting
+
+### Issue: Output is empty or too short
+
+**Solution**: Add `--wait` or provide a `--content-selector` for the primary content node.
+
+### Issue: Cookie banner blocks content
+
+**Solution**: Provide `--consent-selector` (OneTrust) or reuse a `--storage-state` with consent already saved.
+
+## References
+
+- Playwright Docs: https://playwright.dev/docs/intro
+- Playwright Cookies: https://playwright.dev/docs/api/class-browsercontext#browser-context-add-cookies
+- Playwright Storage State: https://playwright.dev/docs/auth#reuse-authentication-state
+- Readability (Mozilla): https://github.com/mozilla/readability
+- DOMParser (MDN): https://developer.mozilla.org/en-US/docs/Web/API/DOMParser
+- Robots Exclusion Protocol (RFC 9309): https://www.rfc-editor.org/rfc/rfc9309
+- GDPR: https://eur-lex.europa.eu/eli/reg/2016/679/oj
+- ePrivacy Directive: https://eur-lex.europa.eu/eli/dir/2002/58/oj