sunpeak 0.18.13 → 0.19.1

package/README.md

@@ -16,7 +16,7 @@
 [![TypeScript](https://img.shields.io/badge/TypeScript-5.9-blue?style=flat&logo=typescript&label=ts&color=FFB800&logoColor=white&labelColor=000035)](https://www.typescriptlang.org/)
 [![React](https://img.shields.io/badge/React-19-blue?style=flat&logo=react&label=react&color=FFB800&logoColor=white&labelColor=000035)](https://reactjs.org/)
 
-Inspector, testing framework, and app framework for MCP Apps.
+Inspector, testing framework, and runtime framework for MCP servers and MCP Apps.
 
 [Demo (Hosted)](https://sunpeak.ai/inspector) ~
 [Demo (Video)](https://cdn.sunpeak.ai/sunpeak-demo-prod.mp4) ~
@@ -28,7 +28,7 @@ Inspector, testing framework, and app framework for MCP Apps.
 
 ### 1. Inspector
 
-Test any MCP server in replicated ChatGPT and Claude runtimes — no sunpeak project required.
+Manually test any MCP server in replicated ChatGPT and Claude runtimes.
 
 ```bash
 sunpeak inspect --server http://localhost:8000/mcp
@@ -45,15 +45,25 @@ sunpeak inspect --server http://localhost:8000/mcp
 - Multi-host inspector replicating ChatGPT and Claude runtimes
 - Toggle themes, display modes, device types from the sidebar or URL params
 - Call real tool handlers or use simulation fixtures for mock data
-- Built into `sunpeak dev` for framework users
 
 ### 2. Testing Framework
 
-E2E tests against simulated hosts and live tests against real production hosts.
+Automatically test any MCP server against replicated ChatGPT and Claude runtimes.
 
-- **Simulations**: JSON fixtures defining reproducible tool states ([example below](#simulation))
-- **E2E tests**: Playwright + `createInspectorUrl` against the inspector ([example below](#inspector))
-- **Live tests**: Automated browser tests against real ChatGPT via `sunpeak/test`
+```ts
+import { test, expect } from 'sunpeak/test';
+
+test('review tool renders title', async ({ mcp }) => {
+  const result = await mcp.callTool('review-diff');
+  const app = result.app();
+  await expect(app.locator('h1:has-text("Refactor")')).toBeVisible();
+});
+```
+
+- **Works for any MCP server**: `sunpeak test init` scaffolds tests for Python, Go, TS, or any language
+- **MCP-native assertions**: `toBeError()`, `toHaveTextContent()`, `toHaveStructuredContent()`
+- **Multi-host**: Tests run against ChatGPT and Claude hosts automatically
+- **Live tests**: Automated browser tests against real ChatGPT via `sunpeak/test/live`
 
 ### 3. App Framework
 
@@ -92,135 +102,25 @@ sunpeak new
 
 ## CLI
 
+**Testing** (works with any MCP server):
+
 | Command                               | Description                                 |
 | ------------------------------------- | ------------------------------------------- |
-| `sunpeak new [name] [resources]`      | Create a new project                        |
-| `sunpeak dev`                         | Start dev server + inspector + MCP endpoint |
-| `sunpeak inspect --server <url\|cmd>` | Inspect any MCP server (standalone)         |
-| `sunpeak build`                       | Build resources + tools for production      |
-| `sunpeak start`                       | Start production MCP server                 |
-| `sunpeak upgrade`                     | Upgrade sunpeak to latest version           |
-
-## Example App
-
-Example `Resource`, `Simulation`, and testing file (using the `Inspector`) for an [MCP resource](https://sunpeak.ai/docs/mcp-apps/mcp/resources) called "Review".
-
-### `Resource` Component
-
-Each resource `.tsx` file exports both the React component and the [MCP resource](https://sunpeak.ai/docs/mcp-apps/mcp/resources) metadata:
-
-```tsx
-// src/resources/review/review.tsx
-
-import { useToolData } from 'sunpeak';
-import type { ResourceConfig } from 'sunpeak';
-
-export const resource: ResourceConfig = {
-  description: 'Visualize and review a code change',
-  _meta: { ui: { csp: { resourceDomains: ['https://cdn.example.com'] } } },
-};
-
-export function ReviewResource() {
-  const { output: data } = useToolData<unknown, { title: string }>();
-
-  return <h1>Review: {data?.title}</h1>;
-}
-```
-
-### Tool File
-
-Each tool `.ts` file exports metadata (with an optional resource link for UI tools), a Zod schema, and a handler:
-
-```ts
-// src/tools/review-diff.ts
-
-import { z } from 'zod';
-import type { AppToolConfig, ToolHandlerExtra } from 'sunpeak/mcp';
-
-export const tool: AppToolConfig = {
-  resource: 'review',
-  title: 'Diff Review',
-  description: 'Show a review dialog for a proposed code diff',
-  annotations: { readOnlyHint: false },
-  _meta: { ui: { visibility: ['model', 'app'] } },
-};
-
-export const schema = {
-  changesetId: z.string().describe('Unique identifier for the changeset'),
-  title: z.string().describe('Title describing the changes'),
-};
-
-type Args = z.infer<z.ZodObject<typeof schema>>;
-
-export default async function (args: Args, extra: ToolHandlerExtra) {
-  return { structuredContent: { title: args.title, sections: [] } };
-}
-```
-
-### `Simulation`
-
-Simulation files provide fixture data for testing UIs. Each references a tool by filename and contains the mock input/output:
-
-```jsonc
-// tests/simulations/review-diff.json
-
-{
-  "tool": "review-diff",                      // References src/tools/review-diff.ts
-  "userMessage": "Refactor the auth module to use JWT tokens.",
-  "toolInput": {
-    "changesetId": "cs_789",
-    "title": "Refactor Authentication Module"
-  },
-  "toolResult": {
-    "structuredContent": {
-      "title": "Refactor Authentication Module",
-      "sections": [...]
-    }
-  }
-}
-```
-
-### `Inspector`
-
-```bash
-├── tests/e2e/
-│   └── review.spec.ts # This! (not pictured above for simplicity)
-└── package.json
-```
-
-The `Inspector` allows you to set **host state** (like host platform, light/dark mode) via URL params, which can be rendered alongside your `Simulation`s and tested via pre-configured Playwright end-to-end tests (`.spec.ts`).
-
-Using the `Inspector` and `Simulation`s, you can test all possible App states locally and automatically across hosts (ChatGPT, Claude)!
-
-```ts
-// tests/e2e/review.spec.ts
-
-import { test, expect } from '@playwright/test';
-import { createInspectorUrl } from 'sunpeak/inspector';
-
-const hosts = ['chatgpt', 'claude'] as const;
-
-for (const host of hosts) {
-  test.describe(`Review Resource [${host}]`, () => {
-    test.describe('Light Mode', () => {
-      test('should render review title with correct styles', async ({ page }) => {
-        const params = { simulation: 'review-diff', theme: 'light', host }; // Set sim & host state.
-        await page.goto(createInspectorUrl(params));
-
-        // Resource content renders inside an iframe
-        const iframe = page.frameLocator('iframe');
-        const title = iframe.locator('h1:has-text("Refactor Authentication Module")');
-        await expect(title).toBeVisible();
-
-        const color = await title.evaluate((el) => window.getComputedStyle(el).color);
-
-        // Light mode should render dark text.
-        expect(color).toBe('rgb(13, 13, 13)');
-      });
-    });
-  });
-}
-```
+| `sunpeak inspect --server <url\|cmd>` | Inspect any MCP server in the inspector     |
+| `sunpeak test`                        | Run e2e tests against the inspector         |
+| `sunpeak test init`                   | Scaffold test infrastructure into a project |
+| `sunpeak test --unit`                 | Run unit tests (Vitest)                     |
+| `sunpeak test --live`                 | Run live tests against real hosts           |
+
+**App framework** (for sunpeak projects):
+
+| Command                          | Description                                 |
+| -------------------------------- | ------------------------------------------- |
+| `sunpeak new [name] [resources]` | Create a new project                        |
+| `sunpeak dev`                    | Start dev server + inspector + MCP endpoint |
+| `sunpeak build`                  | Build resources + tools for production      |
+| `sunpeak start`                  | Start production MCP server                 |
+| `sunpeak upgrade`                | Upgrade sunpeak to latest version           |
 
 ## Coding Agent Skill
 

package/bin/commands/test-init.mjs

@@ -0,0 +1,305 @@
+import { existsSync, readFileSync, writeFileSync, mkdirSync } from 'fs';
+import { join } from 'path';
+import * as p from '@clack/prompts';
+
+/**
+ * sunpeak test init — Scaffold test infrastructure for MCP servers.
+ *
+ * Detects project type and scaffolds accordingly:
+ * - Non-JS projects: self-contained tests/sunpeak/ directory
+ * - JS/TS projects: root-level config + test files
+ * - sunpeak projects: migrate to defineConfig()
+ */
+export async function testInit(args = []) {
+  p.intro('Setting up sunpeak tests');
+
+  // Parse --server flag from CLI args
+  const serverIdx = args.indexOf('--server');
+  const cliServer =
+    serverIdx !== -1 && args[serverIdx + 1]
+      ? args[serverIdx + 1]
+      : undefined;
+
+  const projectType = detectProjectType();
+
+  if (projectType === 'sunpeak') {
+    await initSunpeakProject();
+  } else if (projectType === 'js') {
+    await initJsProject(cliServer);
+  } else {
+    await initExternalProject(cliServer);
+  }
+
+  p.outro('Done!');
+}
+
+function detectProjectType() {
+  const cwd = process.cwd();
+  const pkgPath = join(cwd, 'package.json');
+
+  if (existsSync(pkgPath)) {
+    try {
+      const pkg = JSON.parse(readFileSync(pkgPath, 'utf-8'));
+      const deps = { ...pkg.dependencies, ...pkg.devDependencies };
+      if ('sunpeak' in deps) return 'sunpeak';
+      return 'js';
+    } catch {
+      return 'js';
+    }
+  }
+
+  // Non-JS project (Python, Go, Rust, etc.)
+  return 'external';
+}
+
+async function getServerConfig(cliServer) {
+  // If provided via --server flag, detect type automatically
+  if (cliServer) {
+    if (cliServer.startsWith('http://') || cliServer.startsWith('https://')) {
+      return { type: 'url', value: cliServer };
+    }
+    return { type: 'command', value: cliServer };
+  }
+
+  const serverType = await p.select({
+    message: 'How does your MCP server start?',
+    options: [
+      { value: 'command', label: 'Command (e.g., python server.py)' },
+      { value: 'url', label: 'HTTP URL (e.g., http://localhost:8000/mcp)' },
+      { value: 'later', label: 'Configure later' },
+    ],
+  });
+
+  if (p.isCancel(serverType)) process.exit(0);
+
+  if (serverType === 'command') {
+    const command = await p.text({
+      message: 'Server start command:',
+      placeholder: 'python src/server.py',
+    });
+    if (p.isCancel(command)) process.exit(0);
+    return { type: 'command', value: command };
+  }
+
+  if (serverType === 'url') {
+    const url = await p.text({
+      message: 'Server URL:',
+      placeholder: 'http://localhost:8000/mcp',
+    });
+    if (p.isCancel(url)) process.exit(0);
+    return { type: 'url', value: url };
+  }
+
+  return { type: 'later' };
+}
+
+function generateServerConfigBlock(server, relativeTo = '.') {
+  if (server.type === 'later') {
+    return `  // TODO: Configure your MCP server connection
+  // server: {
+  //   command: 'python',
+  //   args: ['server.py'],
+  // },`;
+  }
+  if (server.type === 'url') {
+    return `  server: {
+    url: '${server.value}',
+  },`;
+  }
+  // Parse command into command + args
+  const parts = server.value.split(/\s+/);
+  const cmd = parts[0];
+  const args = parts.slice(1);
+  // Make paths relative from test directory
+  const relativeArgs = args.map((a) =>
+    a.startsWith('/') || a.startsWith('./') || a.startsWith('../')
+      ? `'${relativeTo}/${a}'`
+      : `'${a}'`
+  );
+  return `  server: {
+    command: '${cmd}',
+    args: [${relativeArgs.join(', ')}],
+  },`;
+}
+
+async function initExternalProject(cliServer) {
+  p.log.info('Detected non-JS project. Creating self-contained test directory.');
+
+  const server = await getServerConfig(cliServer);
+  const testDir = join(process.cwd(), 'tests', 'sunpeak');
+
+  if (existsSync(testDir)) {
+    p.log.warn('tests/sunpeak/ already exists. Skipping scaffold.');
+    return;
+  }
+
+  mkdirSync(testDir, { recursive: true });
+
+  // package.json
+  writeFileSync(
+    join(testDir, 'package.json'),
+    JSON.stringify(
+      {
+        private: true,
+        type: 'module',
+        devDependencies: {
+          sunpeak: 'latest',
+          '@playwright/test': 'latest',
+        },
+        scripts: {
+          test: 'sunpeak test',
+        },
+      },
+      null,
+      2
+    ) + '\n'
+  );
+
+  // sunpeak.config.ts (used as playwright config)
+  const serverBlock = generateServerConfigBlock(server, '../..');
+  writeFileSync(
+    join(testDir, 'playwright.config.ts'),
+    `import { defineConfig } from 'sunpeak/test/config';
+
+export default defineConfig({
+${serverBlock}
+});
+`
+  );
+
+  // tsconfig.json
+  writeFileSync(
+    join(testDir, 'tsconfig.json'),
+    JSON.stringify(
+      {
+        compilerOptions: {
+          target: 'ES2022',
+          module: 'ESNext',
+          moduleResolution: 'bundler',
+          strict: true,
+          esModuleInterop: true,
+        },
+      },
+      null,
+      2
+    ) + '\n'
+  );
+
+  // smoke test — runnable out of the box, verifies the server is reachable
+  writeFileSync(
+    join(testDir, 'smoke.test.ts'),
+    `import { test, expect } from 'sunpeak/test';
+
+test('server is reachable and inspector loads', async ({ mcp }) => {
+  // Verify the inspector page loads successfully
+  await expect(mcp.page.locator('#root')).not.toBeEmpty();
+});
+
+// Uncomment and customize for your tools:
+// test('my tool renders correctly', async ({ mcp }) => {
+//   const result = await mcp.callTool('your-tool', { key: 'value' });
+//   expect(result).not.toBeError();
+//
+//   // If your tool has a UI:
+//   // const app = result.app();
+//   // await expect(app.getByText('Hello')).toBeVisible();
+// });
+`
+  );
+
+  p.log.success('Created tests/sunpeak/ with config and starter test.');
+  p.log.step('Next steps:');
+  p.log.message('  cd tests/sunpeak');
+  p.log.message('  npm install');
+  p.log.message('  npx playwright install chromium');
+  p.log.message('  npx sunpeak test');
+}
+
+async function initJsProject(cliServer) {
+  p.log.info('Detected JS/TS project. Adding test config at project root.');
+
+  const server = await getServerConfig(cliServer);
+  const cwd = process.cwd();
+
+  // Create playwright.config.ts
+  const configPath = join(cwd, 'playwright.config.ts');
+  if (existsSync(configPath)) {
+    p.log.warn('playwright.config.ts already exists. Skipping config creation.');
+  } else {
+    const serverBlock = generateServerConfigBlock(server);
+    writeFileSync(
+      configPath,
+      `import { defineConfig } from 'sunpeak/test/config';
+
+export default defineConfig({
+${serverBlock}
+});
+`
+    );
+    p.log.success('Created playwright.config.ts');
+  }
+
+  // Create test directory and smoke test
+  const testDir = join(cwd, 'tests', 'e2e');
+  mkdirSync(testDir, { recursive: true });
+
+  const testPath = join(testDir, 'smoke.test.ts');
+  if (!existsSync(testPath)) {
+    writeFileSync(
+      testPath,
+      `import { test, expect } from 'sunpeak/test';
+
+test('server is reachable and inspector loads', async ({ mcp }) => {
+  await expect(mcp.page.locator('#root')).not.toBeEmpty();
+});
+
+// Uncomment and customize for your tools:
+// test('my tool renders correctly', async ({ mcp }) => {
+//   const result = await mcp.callTool('your-tool', { key: 'value' });
+//   expect(result).not.toBeError();
+//
+//   // If your tool has a UI:
+//   // const app = result.app();
+//   // await expect(app.getByText('Hello')).toBeVisible();
+// });
+`
+    );
+    p.log.success('Created tests/e2e/smoke.test.ts');
+  }
+
+  p.log.step('Next steps:');
+  p.log.message('  npm install -D sunpeak @playwright/test');
+  p.log.message('  npx playwright install chromium');
+  p.log.message('  npx sunpeak test');
+}
+
+async function initSunpeakProject() {
+  p.log.info('Detected sunpeak project. Updating config to use defineConfig().');
+
+  const cwd = process.cwd();
+  const configPath = join(cwd, 'playwright.config.ts');
+
+  if (existsSync(configPath)) {
+    const content = readFileSync(configPath, 'utf-8');
+    if (content.includes('sunpeak/test/config')) {
+      p.log.info('Config already uses sunpeak/test/config. Nothing to do.');
+      return;
+    }
+  }
+
+  writeFileSync(
+    configPath,
+    `import { defineConfig } from 'sunpeak/test/config';
+
+export default defineConfig();
+`
+  );
+
+  p.log.success('Updated playwright.config.ts to use defineConfig()');
+  p.log.step('Migrate test files:');
+  p.log.message('  Replace: import { test, expect } from "@playwright/test"');
+  p.log.message('  With:    import { test, expect } from "sunpeak/test"');
+  p.log.message('');
+  p.log.message('  Use the `mcp` fixture instead of raw page navigation.');
+  p.log.message('  See sunpeak docs for migration examples.');
+}

package/bin/commands/test.mjs

@@ -0,0 +1,83 @@
+import { spawn } from 'child_process';
+import { existsSync } from 'fs';
+import { join } from 'path';
+
+/**
+ * sunpeak test — Run MCP server tests.
+ *
+ * sunpeak test              Run e2e tests (Playwright)
+ * sunpeak test init         Scaffold test infrastructure
+ * sunpeak test --unit       Run unit tests (vitest)
+ * sunpeak test --live       Run live tests against real hosts
+ * sunpeak test [pattern]    Pass through to Playwright
+ */
+export async function runTest(args) {
+  // Handle `sunpeak test init` subcommand
+  if (args[0] === 'init') {
+    const { testInit } = await import('./test-init.mjs');
+    await testInit(args.slice(1));
+    return;
+  }
+
+  const isUnit = args.includes('--unit');
+  const isLive = args.includes('--live');
+  const filteredArgs = args.filter((a) => a !== '--unit' && a !== '--live');
+
+  if (isUnit) {
+    // Run vitest
+    const child = spawn('pnpm', ['exec', 'vitest', 'run', ...filteredArgs], {
+      stdio: 'inherit',
+      env: { ...process.env },
+    });
+    child.on('exit', (code) => process.exit(code ?? 1));
+    return;
+  }
+
+  // Find the appropriate Playwright config
+  let configArgs = [];
+  if (isLive) {
+    const liveConfig = findConfig([
+      'tests/live/playwright.config.ts',
+      'tests/live/playwright.config.js',
+    ]);
+    if (liveConfig) {
+      configArgs = ['--config', liveConfig];
+    } else {
+      console.error('No live test config found at tests/live/playwright.config.ts');
+      process.exit(1);
+    }
+  } else {
+    // Default: e2e tests — look for config in standard locations
+    const e2eConfig = findConfig([
+      'playwright.config.ts',
+      'playwright.config.js',
+      'sunpeak.config.ts',
+      'sunpeak.config.js',
+    ]);
+    if (e2eConfig) {
+      configArgs = ['--config', e2eConfig];
+    }
+    // If no config found, let Playwright use its defaults
+  }
+
+  const child = spawn(
+    'pnpm',
+    ['exec', 'playwright', 'test', ...configArgs, ...filteredArgs],
+    {
+      stdio: 'inherit',
+      env: {
+        ...process.env,
+        SUNPEAK_DEV_OVERLAY: process.env.SUNPEAK_DEV_OVERLAY ?? 'false',
+      },
+    }
+  );
+  child.on('exit', (code) => process.exit(code ?? 1));
+}
+
+function findConfig(candidates) {
+  for (const candidate of candidates) {
+    const full = join(process.cwd(), candidate);
+    if (existsSync(full)) return candidate;
+  }
+  return null;
+}

package/bin/lib/inspect/inspect-config.mjs

@@ -2,16 +2,19 @@
  * Playwright config factory for inspect mode (BYOS — Bring Your Own Server).
  *
  * Generates a complete Playwright config that starts `sunpeak inspect` as the
- * webServer and runs e2e tests against the inspector. Follows the same pattern
- * as `defineLiveConfig` for live tests.
+ * webServer and runs e2e tests against the inspector.
  *
  * Usage in playwright.config.ts:
  *   import { defineInspectConfig } from 'sunpeak/test/inspect/config';
  *   export default defineInspectConfig({
  *     server: 'http://localhost:8000/mcp',
  *   });
+ *
+ * Note: For new projects, prefer `defineConfig` from 'sunpeak/test/config'
+ * which auto-detects the project type and handles both sunpeak projects
+ * and external servers.
  */
-import { getPortSync } from '../get-port.mjs';
+import { createBaseConfig, resolvePorts } from '../test/base-config.mjs';
 
 /**
  * Create a complete Playwright config for testing an external MCP server.
@@ -19,7 +22,7 @@ import { getPortSync } from '../get-port.mjs';
  * @param {Object} options
  * @param {string} options.server - MCP server URL or stdio command (required)
  * @param {string} [options.testDir='tests/e2e'] - Test directory
- * @param {string} [options.simulationsDir='tests/simulations'] - Simulation JSON directory
+ * @param {string} [options.simulationsDir] - Simulation JSON directory
  * @param {string[]} [options.hosts=['chatgpt', 'claude']] - Host shells to test
  * @param {string} [options.name] - App name in inspector chrome
  * @param {Object} [options.use] - Additional Playwright `use` options
@@ -39,12 +42,12 @@ export function defineInspectConfig(options) {
     throw new Error('defineInspectConfig: `server` option is required');
   }
 
-  const port = Number(process.env.SUNPEAK_TEST_PORT) || getPortSync(6776);
-  const sandboxPort = Number(process.env.SUNPEAK_SANDBOX_PORT) || getPortSync(24680);
+  const { port, sandboxPort } = resolvePorts();
 
   // Build the sunpeak inspect command
   const serverArg = server.includes(' ') ? `"${server}"` : server;
   const command = [
+    `SUNPEAK_SANDBOX_PORT=${sandboxPort}`,
     'npx sunpeak inspect',
     `--server ${serverArg}`,
     ...(simulationsDir ? [`--simulations ${simulationsDir}`] : []),
@@ -52,25 +55,14 @@ export function defineInspectConfig(options) {
     ...(name ? [`--name "${name}"`] : []),
   ].join(' ');
 
-  return {
+  return createBaseConfig({
+    hosts,
     testDir,
-    fullyParallel: true,
-    forbidOnly: !!process.env.CI,
-    retries: process.env.CI ? 2 : 1,
-    // Limit workers to avoid overwhelming the double-iframe sandbox proxy.
-    workers: process.env.CI ? 1 : 2,
-    reporter: 'list',
-    use: {
-      baseURL: `http://localhost:${port}`,
-      trace: 'on-first-retry',
-      ...userUse,
-    },
-    projects: hosts.map((host) => ({ name: host })),
+    port,
+    use: userUse,
     webServer: {
-      command: `SUNPEAK_SANDBOX_PORT=${sandboxPort} ${command}`,
-      url: `http://localhost:${port}/health`,
-      reuseExistingServer: !process.env.CI,
-      timeout: 60_000,
+      command,
+      healthUrl: `http://localhost:${port}/health`,
     },
-  };
+  });
 }