premanmcp 0.4.0 → 0.7.0

package/README.md

@@ -4,26 +4,33 @@ Turn your APIs into MCP tools that coding agents can discover, call, test, and a
 
 PreMan is agent-first API infrastructure. It lets backend teams expose endpoints to AI coding agents through MCP, add an auth layer around those tools, and see exactly which agent called what.
 
-## Install
+## Connect
 
 ```bash
-npm exec -y premanmcp@latest -- install
+npm exec -y premanmcp@latest -- connect
 ```
 
+Pick your coding agent from the list — Cursor, Claude Code, or Codex — and PreMan
+writes that agent's MCP config for you. No hand-edited config anywhere.
+
 Local development form:
 
 ```bash
-node bin/cli.js install
+node bin/cli.js connect
 ```
 
-First-time users are prompted for email, OTP, and password directly in the terminal. PreMan creates or connects the account, generates an API key, saves it to `~/.preman/credentials.json`, then writes a `preman` MCP server into `~/.cursor/mcp.json`:
+First-time users are prompted for email, OTP, and password directly in the terminal.
+PreMan creates or connects the account, generates an API key, saves it to
+`~/.preman/credentials.json`, then writes a `preman` MCP server into the config your
+agent actually reads (`~/.cursor/mcp.json`, Claude Code's MCP config, or
+`~/.codex/config.toml`):
 
 ```json
 {
   "mcpServers": {
     "preman": {
       "command": "npm",
-      "args": ["exec", "-y", "--package", "premanmcp@latest", "--", "premanmcp"],
+      "args": ["exec", "-y", "premanmcp@latest", "--"],
       "env": {
         "PREMAN_BACKEND": "https://api.preman.live",
         "PREMAN_FRONTEND": "https://app.preman.live"
@@ -33,9 +40,34 @@ First-time users are prompted for email, OTP, and password directly in the termi
 }
 ```
 
-Restart Cursor or toggle the MCP server off/on after install. The MCP server loads the saved key automatically.
+Restart your agent afterwards, then ask it to `run preman_status` to finish linking.
+
+`connect` reads the config back after writing it, and prints a copy-paste snippet if it
+cannot confirm the entry landed. It then waits for your agent to check in — that
+check-in is the only real proof the agent loaded the server.
+
+Once the agent has checked in, `connect` walks you into your first test: it runs one
+against an endpoint you already have, or prints the discovery brief to hand your agent,
+followed by `preman endpoints setup` and `preman test`. Pass `--no-guide` to skip it.
+
+Useful flags: `--agent cursor|claude-code|codex` skips the picker, `--project` writes
+project-local config, `--print` shows the config without writing it, `--no-guide` skips
+the guided first run.
+
+In CI or any non-interactive shell, run `connect --agent <name> --api-key pm_live_…`.
+Without `--agent` there is nothing to prompt on, so `connect` prints ready-to-paste
+setup blocks for all three agents and exits 2.
+
+`preman install` still exists and does the Cursor-only half of this.
 
-You can also create or connect your account before install:
+### Cloud dispatch (optional)
+
+`connect` offers to store a coding-agent credential — a Cursor API key, or a Claude
+Code routine token and id. With one saved, PreMan can start an agent run for you when
+it finds a failing endpoint instead of handing back a prompt to paste. Press Enter to
+skip; everything else still works.
+
+You can also create or connect your account first:
 
 ```bash
 npm exec -y premanmcp@latest -- login
@@ -44,13 +76,13 @@ npm exec -y premanmcp@latest -- login
 You can also pass the key directly:
 
 ```bash
-npm exec -y premanmcp@latest -- install --api-key pm_live_xxx
+npm exec -y premanmcp@latest -- connect --api-key pm_live_xxx
 ```
 
-For project-local installation:
+For project-local config:
 
 ```bash
-npm exec -y premanmcp@latest -- install --project
+npm exec -y premanmcp@latest -- connect --project
 ```
 
 ## What It Does
@@ -63,6 +95,7 @@ npm exec -y premanmcp@latest -- install --project
 - Supports hosted MCPs with consumer tokens for customer-facing agent access.
 - Records per-call observability so teams can audit which agent did what.
 - Hands failing-endpoint alerts to your agent as fix tasks (`preman_get_fix_task` → repro curl → `preman_complete_fix_task`).
+- Connects your production logs from the terminal (`connect_logs`): your agent asks where the logs live, then deploys a read-only CloudFormation role for AWS, or wires a shipper to the ingest endpoint for everything else.
 
 ## Common Agent Commands
 
@@ -92,22 +125,24 @@ Show me the audit log for this hosted MCP.
 Pull my pending PreMan fix tasks and fix the failing endpoint.
 ```
 
-## Cursor
+```text
+Connect my production logs to PreMan.
+```
 
-The installer targets Cursor by default:
+## Cursor
 
 ```bash
-npm exec -y premanmcp@latest -- install
+npm exec -y premanmcp@latest -- connect --agent cursor
 ```
 
-Manual Cursor config:
+Manual Cursor config, if you would rather write it yourself:
 
 ```json
 {
   "mcpServers": {
     "preman": {
       "command": "npm",
-      "args": ["exec", "-y", "--package", "premanmcp@latest", "--", "premanmcp"]
+      "args": ["exec", "-y", "premanmcp@latest", "--"]
     }
   }
 }
@@ -116,18 +151,33 @@ Manual Cursor config:
 ## Claude Code
 
 ```bash
-claude mcp add preman -- npm exec -y --package premanmcp@latest -- premanmcp
+npm exec -y premanmcp@latest -- connect --agent claude-code
+```
+
+Equivalent manual command:
+
+```bash
+claude mcp add preman -- npm exec -y premanmcp@latest --
 ```
 
+## Codex
+
+```bash
+npm exec -y premanmcp@latest -- connect --agent codex
+```
+
+Writes an `[mcp_servers.preman]` block into `~/.codex/config.toml`.
+
 ## CLI
 
 ```bash
-npm exec -y premanmcp@latest --       # Start the MCP server
-npm exec -y premanmcp@latest -- login # Create/login and generate a PreMan API key
-npm exec -y premanmcp@latest -- install          # Install Cursor MCP config
-npm exec -y premanmcp@latest -- install --project # Write .cursor/mcp.json in current repo
-npm exec -y premanmcp@latest -- install --print   # Print config without writing
-npm exec -y premanmcp@latest -- install --api-key KEY # Install with your PreMan API key
+npm exec -y premanmcp@latest -- connect            # Pick an agent and connect it
+npm exec -y premanmcp@latest -- connect --agent codex   # Skip the picker
+npm exec -y premanmcp@latest -- connect --project  # Write project-local config
+npm exec -y premanmcp@latest -- connect --print    # Print config without writing
+npm exec -y premanmcp@latest --         # Start the MCP server
+npm exec -y premanmcp@latest -- login   # Create/login and generate a PreMan API key
+npm exec -y premanmcp@latest -- install # Cursor-only installer (legacy)
 ```
 
 Options:

package/bin/api_tools.js

@@ -0,0 +1,215 @@
+/**
+ * `preman endpoints` and `preman test` — the API-testing surface of the CLI.
+ *
+ * Both are thin proxies over `POST /mcp/call-tool`, exactly like the MCP
+ * server's tools, so the CLI and an agent calling the same tool see identical
+ * behaviour. Requires a pm_live_ key (flag, env, or stored credentials).
+ */
+
+import { readFileSync } from "node:fs";
+
+import { callBackendJson, makeArgs, resolveApiKey } from "./shared.js";
+
+export const ENDPOINTS_HELP = `
+Endpoints:
+  preman endpoints list [--status s] [--method m] [--workbench] [--json-out]
+      List endpoints saved in PreMan (add --workbench for runnable requests).
+  preman endpoints discover [--framework hint]
+      Print the codebase-discovery brief for your coding agent to execute.
+  preman endpoints setup (--file endpoints.json | --ids id1,id2) [--base-url url] [--no-workbench]
+      Register discovered endpoints and set them up as runnable requests.
+`;
+
+export const TEST_HELP = `
+Testing:
+  preman test <request-or-endpoint-id> [options]
+      Generate scenario tests for one endpoint and run them.
+      --scenario "..."       Add a described scenario (repeatable)
+      --no-run               Generate only, don't execute
+      --allow-writes         Permit mutating cases / write endpoints
+      --code                 Emit unit-test files (prints filename + content)
+      --framework pytest|jest  Framework for --code (default pytest)
+      --stress               Run a bounded load test instead
+      --duration n --rps n --concurrency n   Stress parameters
+      --json-out             Print the raw JSON result
+`;
+
+class CliError extends Error {
+  constructor(message, exitCode = 1) {
+    super(message);
+    this.exitCode = exitCode;
+  }
+}
+
+export async function callTool(args, tool, toolArguments) {
+  const token = resolveApiKey(args);
+  if (!token) {
+    throw new CliError(
+      "No PreMan API key. Run `preman login` or pass --api-key pm_live_...",
+      2,
+    );
+  }
+  const result = await callBackendJson(args, "POST", "/mcp/call-tool", {
+    json: { tool, arguments: toolArguments },
+    token,
+  });
+  if (!result.ok) {
+    const detail = result.detail || result.message || result.raw || "backend error";
+    throw new CliError(`${tool} failed: ${result.status_code} ${detail}`);
+  }
+  return result;
+}
+
+function printJson(value) {
+  process.stdout.write(`${JSON.stringify(value, null, 2)}\n`);
+}
+
+function collectRepeated(commandArgs, flag) {
+  const out = [];
+  for (let i = 0; i < commandArgs.length; i += 1) {
+    if (commandArgs[i] === flag && commandArgs[i + 1]) out.push(commandArgs[i + 1]);
+  }
+  return out;
+}
+
+export async function endpointsCommand(commandArgs) {
+  const sub = commandArgs[0] && !commandArgs[0].startsWith("-") ? commandArgs[0] : "list";
+  const rest = sub === commandArgs[0] ? commandArgs.slice(1) : commandArgs;
+  const args = makeArgs(rest);
+
+  if (sub === "list") {
+    const toolArguments = {
+      status: args.value("--status", "") || undefined,
+      method: args.value("--method", "") || undefined,
+      include_workbench: args.has("--workbench"),
+      limit: Number(args.value("--limit", "50")),
+    };
+    const result = await callTool(args, "get_endpoints", toolArguments);
+    if (args.has("--json-out")) return printJson(result);
+    const rows = result.endpoints || [];
+    process.stdout.write(`${rows.length} registry endpoint(s)\n`);
+    for (const ep of rows) {
+      const status = ep.status ? ` [${ep.status}]` : "";
+      process.stdout.write(`  ${ep.method || "GET"} ${ep.path_template || ep.url || ""}${status} ${ep.id || ""}\n`);
+    }
+    const workbench = result.workbench_requests || [];
+    if (workbench.length) {
+      process.stdout.write(`${workbench.length} runnable workbench request(s)\n`);
+      for (const r of workbench) {
+        process.stdout.write(`  ${r.method} ${r.url} ${r.id}\n`);
+      }
+    }
+    return undefined;
+  }
+
+  if (sub === "discover") {
+    const result = await callTool(args, "discover_endpoints_from_codebase", {
+      base_path: args.value("--path", "."),
+      framework_hint: args.value("--framework", "") || undefined,
+    });
+    if (args.has("--json-out")) return printJson(result);
+    for (const line of result.instructions || []) process.stdout.write(`${line}\n`);
+    process.stdout.write(
+      "\nHand this brief to your coding agent, then run `preman endpoints setup --file endpoints.json`.\n",
+    );
+    return undefined;
+  }
+
+  if (sub === "setup") {
+    const file = args.value("--file", "");
+    const ids = args.value("--ids", "");
+    const toolArguments = {
+      base_url: args.value("--base-url", "") || undefined,
+      setup_workbench: !args.has("--no-workbench"),
+    };
+    if (file) {
+      let parsed;
+      try {
+        parsed = JSON.parse(readFileSync(file, "utf8"));
+      } catch (error) {
+        throw new CliError(`could not read ${file}: ${error.message}`, 2);
+      }
+      toolArguments.endpoints = Array.isArray(parsed) ? parsed : parsed.endpoints;
+    } else if (ids) {
+      toolArguments.endpoint_ids = ids.split(",").map((s) => s.trim()).filter(Boolean);
+    } else {
+      throw new CliError("pass --file endpoints.json or --ids id1,id2", 2);
+    }
+    const result = await callTool(args, "register_discovered_endpoints", toolArguments);
+    if (args.has("--json-out")) return printJson(result);
+    process.stdout.write(
+      `registered ${result.registered || 0} endpoint(s), skipped ${result.skipped || 0}\n`,
+    );
+    const requests = result.workbench_request_ids || [];
+    if (requests.length) {
+      process.stdout.write(`runnable as: ${requests.join(", ")}\n`);
+    }
+    return undefined;
+  }
+
+  throw new CliError(`unknown endpoints subcommand: ${sub}${ENDPOINTS_HELP}`, 2);
+}
+
+/** Print the scenario list and run outcome from generate_endpoint_tests. */
+export function printTestSummary(result) {
+  process.stdout.write(`${result.count || 0} scenario(s) for ${result.request_id}\n`);
+  for (const c of result.cases || []) {
+    process.stdout.write(`  - ${c.name} (${c.kind})\n`);
+  }
+  const run = result.run;
+  if (!run) return;
+  process.stdout.write(
+    `run ${run.status}: ${run.passed}/${run.total} passed, ${run.skipped} skipped` +
+    `${run.aborted_reason ? `, aborted: ${run.aborted_reason}` : ""}\n`,
+  );
+  for (const o of run.outcomes || []) {
+    if (o.status !== "passed") {
+      process.stdout.write(`  FAIL ${o.case_name}: ${o.response_status ?? o.error ?? ""}\n`);
+    }
+  }
+}
+
+export async function testCommand(commandArgs) {
+  const target = commandArgs[0] && !commandArgs[0].startsWith("-") ? commandArgs[0] : "";
+  const rest = target ? commandArgs.slice(1) : commandArgs;
+  const args = makeArgs(rest);
+  if (!target) {
+    throw new CliError(`pass a request or endpoint id${TEST_HELP}`, 2);
+  }
+
+  if (args.has("--stress")) {
+    const result = await callTool(args, "run_stress_test", {
+      target,
+      duration_seconds: Number(args.value("--duration", "15")),
+      rps: Number(args.value("--rps", "5")),
+      concurrency: Number(args.value("--concurrency", "5")),
+      allow_writes: args.has("--allow-writes"),
+    });
+    if (args.has("--json-out")) return printJson(result);
+    process.stdout.write(
+      `stress ${result.status}: ${result.total_requests} requests, ` +
+      `${Math.round((result.error_rate || 0) * 1000) / 10}% errors, ` +
+      `p50 ${result.p50_ms ?? "n/a"}ms p95 ${result.p95_ms ?? "n/a"}ms p99 ${result.p99_ms ?? "n/a"}ms, ` +
+      `${Math.round((result.throughput_rps || 0) * 10) / 10} rps\n`,
+    );
+    if (result.aborted_reason) process.stdout.write(`aborted: ${result.aborted_reason}\n`);
+    return undefined;
+  }
+
+  const result = await callTool(args, "generate_endpoint_tests", {
+    target,
+    scenarios: collectRepeated(rest, "--scenario"),
+    run: !args.has("--no-run"),
+    allow_writes: args.has("--allow-writes"),
+    max_cases: Number(args.value("--max-cases", "10")),
+    include_code: args.has("--code"),
+    test_framework: args.value("--framework", "pytest"),
+  });
+  if (args.has("--json-out")) return printJson(result);
+
+  printTestSummary(result);
+  for (const artifact of result.code_artifacts || []) {
+    process.stdout.write(`\n===== ${artifact.filename} =====\n${artifact.content}\n`);
+  }
+  return undefined;
+}