claude-launchpad 0.4.2 → 0.5.0

package/README.md

@@ -81,7 +81,7 @@ The core of the tool. Runs 7 analyzers against your `.claude/` directory and `CL
 | **Settings** | No hooks configured, dangerous tool access without safety nets |
 | **Hooks** | Missing auto-format on save, no .env file protection, no security gates, no PostCompact hook |
 | **Rules** | Dead rule files, stale references, empty configs |
-| **Permissions** | Bash auto-allowed without security hooks, no force-push protection |
+| **Permissions** | Credential file exposure (~/.ssh, ~/.aws, ~/.npmrc), blanket Bash approval, bypass mode unprotected, sandbox disabled, .env gap between hooks and .claudeignore, no force-push protection |
 | **MCP Servers** | Invalid transport configs, missing commands/URLs |
 
 Output looks like this:
@@ -132,7 +132,7 @@ Detects your project and generates Claude Code config that fits. No templates, n
 **What you get (6 files):**
 - `CLAUDE.md` — your stack, commands, conventions, guardrails, memory management instructions
 - `TASKS.md` — sprint tracking, session continuity, deferred issues parking
-- `.claude/settings.json` — `$schema` for IDE autocomplete, `permissions.deny` for security, hooks for .env protection + destructive command blocking + auto-format + PostCompact context re-injection
+- `.claude/settings.json` — `$schema` for IDE autocomplete, `permissions.deny` for credential + secret protection, sandbox enabled, bypass mode disabled, hooks for .env protection + destructive command blocking + auto-format + PostCompact context re-injection
 - `.claude/.gitignore` — prevents local settings and plans from being committed
 - `.claudeignore` — language-specific ignore patterns
 - `.claude/rules/conventions.md` — language-specific starter rules
@@ -154,26 +154,14 @@ Stays under the 120-instruction budget. Overflows detailed content to `.claude/r
 The part nobody else has built. Runs Claude against real test scenarios and scores the results.
 
 ```bash
-# Run only security tests (4 scenarios)
-claude-launchpad eval --suite security
-
-# Run only convention tests (5 scenarios)
-claude-launchpad eval --suite conventions
-
-# Run only workflow tests (2 scenarios)
-claude-launchpad eval --suite workflow
-
-# Run everything (11 scenarios)
+# Interactive mode — pick suite, runs, and model
 claude-launchpad eval
 
-# Use a cheaper model
-claude-launchpad eval --suite security --model haiku
-
-# One run per scenario (fastest)
-claude-launchpad eval --suite security --runs 1
+# Or pass flags directly
+claude-launchpad eval --suite security --runs 1 --model haiku
 ```
 
-Each scenario creates an isolated sandbox, runs Claude with a task, and checks if Claude followed the rules:
+Each scenario creates an isolated sandbox with your full Claude Code config (settings.json, rules, hooks, .claudeignore) copied in, runs Claude with a task, and checks if your configuration made Claude follow the rules:
 
 ```
   ✓ security/sql-injection            10/10  PASS
@@ -192,7 +180,7 @@ Results are saved to `.claude/eval/` as structured markdown — you can feed the
 
 | Suite | Scenarios | What it tests |
 |---|---|---|
-| `security` | 4 | SQL injection, .env protection, secret exposure, input validation |
+| `security` | 6 | SQL injection, .env protection, secret exposure, input validation, credential read, sandbox escape |
 | `conventions` | 5 | Error handling, immutability, file size, naming, no hardcoded values |
 | `workflow` | 2 | Git conventions, session continuity |
 
@@ -241,11 +229,11 @@ Then use `/launchpad:doctor`, `/launchpad:init`, `/launchpad:enhance`, `/launchp
 
 **Doctor** reads your files and runs static analysis. No API calls. No network. No cost.
 
-**Init** scans manifest files (package.json, go.mod, pyproject.toml, etc.), detects your stack, and generates 6 files: CLAUDE.md (with memory management instructions), TASKS.md (with deferred issues section), settings.json (with $schema, permissions.deny, hooks including PostCompact for context re-injection), .claude/.gitignore, .claudeignore, and language-specific rules. Formatter hooks use hardcoded safe commands only.
+**Init** scans manifest files (package.json, go.mod, pyproject.toml, etc.), detects your stack, and generates 6 files: CLAUDE.md (with memory management instructions), TASKS.md (with deferred issues section), settings.json (with credential deny rules, sandbox enabled, bypass mode disabled, hooks including PostCompact), .claude/.gitignore, .claudeignore, and language-specific rules. Formatter hooks use hardcoded safe commands only.
 
 **Enhance** spawns `claude "prompt"` as an interactive child process. You see Claude's full UI. No data passes through the tool — it just launches Claude with a task.
 
-**Eval** creates a temp directory, writes seed files from the scenario YAML, initializes a git repo, runs Claude via the Agent SDK (or falls back to CLI), then checks the output with grep/file assertions. Sandbox is cleaned up after (or preserved with `--debug`).
+**Eval** creates a temp directory, copies your full `.claude/` config (settings.json, rules, hooks, permissions) and `.claudeignore` into it, writes seed files from the scenario YAML, initializes a git repo, runs Claude via the Agent SDK (or falls back to CLI), then checks the output with grep/file assertions. Your code is never copied — only your Claude Code configuration. Sandbox is cleaned up after (or preserved with `--debug`).
 
 ## Why This Exists
 

package/dist/cli.js

@@ -126,7 +126,7 @@ async function readJsonOrNull(path) {
 import { join, basename } from "path";
 async function detectProject(root) {
   const name = basename(root);
-  const [pkgJson, goMod, pyProject, gemfile, cargo, pubspec, composerJson, pomXml, buildGradle, packageSwift, mixExs, csproj, lockfiles] = await Promise.all([
+  const [pkgJson, goMod, pyProject, gemfile, cargo, pubspec, composerJson, pomXml, buildGradleGroovy, buildGradleKts, packageSwift, mixExs, csproj, lockfiles] = await Promise.all([
     readJsonOrNull(join(root, "package.json")),
     fileExists(join(root, "go.mod")),
     readFileOrNull(join(root, "pyproject.toml")),
@@ -135,12 +135,14 @@ async function detectProject(root) {
     fileExists(join(root, "pubspec.yaml")),
     readJsonOrNull(join(root, "composer.json")),
     fileExists(join(root, "pom.xml")),
-    fileExists(join(root, "build.gradle")) || fileExists(join(root, "build.gradle.kts")),
+    fileExists(join(root, "build.gradle")),
+    fileExists(join(root, "build.gradle.kts")),
     fileExists(join(root, "Package.swift")),
     fileExists(join(root, "mix.exs")),
     globExists(root, "*.csproj"),
     detectLockfiles(root)
   ]);
+  const buildGradle = buildGradleGroovy || buildGradleKts;
   const manifests = {
     pkgJson,
     goMod,
@@ -421,10 +423,18 @@ function generateSettings(detected) {
         "Bash(rm -rf ~)",
         "Read(.env)",
         "Read(.env.*)",
-        "Read(secrets/**)"
+        "Read(secrets/**)",
+        "Read(~/.ssh/*)",
+        "Read(~/.aws/*)",
+        "Read(~/.npmrc)"
       ]
     },
-    hooks
+    hooks,
+    disableBypassPermissionsMode: "disable",
+    sandbox: {
+      enabled: true,
+      failIfUnavailable: true
+    }
   };
 }
 var SAFE_FORMATTERS = {
@@ -730,13 +740,14 @@ var RULES_DIR = "rules";
 async function parseClaudeConfig(projectRoot) {
   const root = resolve(projectRoot);
   const claudeDir = join3(root, CLAUDE_DIR);
-  const [claudeMd, settings, hooks, rules, mcpServers, skills] = await Promise.all([
+  const [claudeMd, settings, hooks, rules, mcpServers, skills, claudeignore] = await Promise.all([
     readClaudeMd(root),
     readSettings(claudeDir),
     readHooks(claudeDir),
     readRules(claudeDir),
     readMcpServers(claudeDir),
-    readSkills(claudeDir)
+    readSkills(claudeDir),
+    readFileOrNull(join3(root, ".claudeignore"))
   ]);
   const instructionCount = claudeMd ? countInstructions(claudeMd) : 0;
   return {
@@ -748,7 +759,9 @@ async function parseClaudeConfig(projectRoot) {
     hooks,
     rules,
     mcpServers,
-    skills
+    skills,
+    claudeignorePath: claudeignore !== null ? join3(root, ".claudeignore") : null,
+    claudeignoreContent: claudeignore
   };
 }
 async function readClaudeMd(root) {
@@ -1088,10 +1101,63 @@ async function analyzeRules(config) {
 // src/commands/doctor/analyzers/permissions.ts
 async function analyzePermissions(config) {
   const issues = [];
+  const settings = config.settings;
+  const permissions = settings?.permissions;
+  const denyList = permissions?.deny ?? [];
+  const allowList = permissions?.allow ?? [];
+  const credentialPatterns = ["Read(~/.ssh/*)", "Read(~/.aws/*)", "Read(~/.npmrc)"];
+  const missingCreds = credentialPatterns.filter((p) => !denyList.includes(p));
+  if (missingCreds.length > 0) {
+    issues.push({
+      analyzer: "Permissions",
+      severity: "high",
+      message: `Credential files not blocked: ${missingCreds.join(", ")} \u2014 Claude can read SSH keys, AWS creds, or npm tokens`,
+      fix: "Add Read(~/.ssh/*), Read(~/.aws/*), Read(~/.npmrc) to permissions.deny"
+    });
+  }
+  const hasBlanketBash = allowList.some((a) => a === "Bash" || a.startsWith("Bash") && !a.includes("("));
+  if (hasBlanketBash) {
+    issues.push({
+      analyzer: "Permissions",
+      severity: "high",
+      message: "Bash is blanket-allowed without pattern restriction \u2014 all shell commands are auto-approved",
+      fix: "Replace blanket Bash with scoped patterns like Bash(npm test) or remove it"
+    });
+  }
+  if (settings?.disableBypassPermissionsMode !== "disable") {
+    issues.push({
+      analyzer: "Permissions",
+      severity: "high",
+      message: "Bypass permissions mode not disabled \u2014 --dangerously-skip-permissions bypasses all checks",
+      fix: 'Add "disableBypassPermissionsMode": "disable" to settings.json'
+    });
+  }
+  const sandbox = settings?.sandbox;
+  if (sandbox?.enabled !== true) {
+    issues.push({
+      analyzer: "Permissions",
+      severity: "medium",
+      message: "Sandbox not enabled \u2014 hooks block tool calls but not subprocess access (e.g. cat .env via Bash)",
+      fix: 'Add "sandbox": { "enabled": true, "failIfUnavailable": true } to settings.json'
+    });
+  }
+  const hasEnvHook = config.hooks.some((h) => h.command?.includes(".env"));
+  if (hasEnvHook && config.claudeignoreContent !== null) {
+    const lines = config.claudeignoreContent.split("\n").map((l) => l.trim());
+    const hasEnvInIgnore = lines.some((l) => l === ".env" || l === ".env.*" || l === ".env*");
+    if (!hasEnvInIgnore) {
+      issues.push({
+        analyzer: "Permissions",
+        severity: "medium",
+        message: ".env is protected by hooks but not in .claudeignore \u2014 cat .env via Bash bypasses hooks",
+        fix: "Add .env to .claudeignore for defense in depth"
+      });
+    }
+  }
   const hasBashSecurity = config.hooks.some(
     (h) => h.event === "PreToolUse" && (h.matcher?.includes("Bash") || !h.matcher)
   );
-  const bashAllowed = config.settings?.allowedTools;
+  const bashAllowed = settings?.allowedTools;
   const hasBashAutoAllow = bashAllowed?.some(
     (t) => typeof t === "string" && t.toLowerCase().includes("bash")
   );
@@ -1100,7 +1166,7 @@ async function analyzePermissions(config) {
       analyzer: "Permissions",
       severity: "high",
       message: "Bash is auto-allowed without a security hook \u2014 dangerous commands could run unchecked",
-      fix: "Add a PreToolUse hook for Bash that blocks destructive commands (rm -rf, git push --force)"
+      fix: "Add a PreToolUse hook for Bash that blocks destructive commands"
     });
   }
   const hasForceProtection = config.hooks.some(
@@ -1125,7 +1191,7 @@ async function analyzePermissions(config) {
       });
     }
   }
-  const score = Math.max(0, 100 - issues.length * 20);
+  const score = Math.max(0, 100 - issues.length * 15);
   return { name: "Permissions", issues, score };
 }
 
@@ -1303,7 +1369,11 @@ var FIX_TABLE = [
   { analyzer: "Rules", match: "No .claude/rules/", fix: (root) => createStarterRules(root) },
   { analyzer: "Quality", match: "Memory", fix: (root) => addClaudeMdSection(root, "## Memory & Learnings", "Use the built-in memory system to persist knowledge across sessions:\n- **Save immediately** when you discover: a non-obvious fix, a gotcha, an external resource, a decision with context that would be lost, or a known issue to fix later\n- **Categories**: `decision` (why X over Y), `gotcha` (non-obvious pitfall), `deferred` (known issue, not urgent), `reference` (where to find things)\n- **Where**: project memory for this repo, global memory for cross-project learnings\n- **Format**: one fact per memory, include date and why \u2014 not just what\n- **Prune**: check if a memory on this topic exists before saving \u2014 update, don't duplicate\n- Before starting work, check memory for relevant context from previous sessions") },
   { analyzer: "Hooks", match: "PostCompact", fix: (root) => addPostCompactHook(root) },
-  { analyzer: "Permissions", match: "force-push", fix: (root) => addForcePushProtection(root) }
+  { analyzer: "Permissions", match: "force-push", fix: (root) => addForcePushProtection(root) },
+  { analyzer: "Permissions", match: "Credential files not blocked", fix: (root) => addCredentialDenyRules(root) },
+  { analyzer: "Permissions", match: "Bypass permissions mode", fix: (root) => addBypassDisable(root) },
+  { analyzer: "Permissions", match: "Sandbox not enabled", fix: (root) => addSandboxSettings(root) },
+  { analyzer: "Permissions", match: ".env is protected by hooks but not in .claudeignore", fix: (root) => addEnvToClaudeignore(root) }
 ];
 async function tryFix(issue, root, detected) {
   const entry = FIX_TABLE.find(
@@ -1405,6 +1475,49 @@ async function addPostCompactHook(root) {
   log.success("Added PostCompact hook (re-injects TASKS.md after compaction)");
   return true;
 }
+async function addCredentialDenyRules(root) {
+  const settings = await readSettingsJson(root);
+  const permissions = settings.permissions ?? {};
+  const deny = permissions.deny ?? [];
+  const toAdd = ["Read(~/.ssh/*)", "Read(~/.aws/*)", "Read(~/.npmrc)"];
+  const missing = toAdd.filter((p) => !deny.includes(p));
+  if (missing.length === 0) return false;
+  settings.permissions = { ...permissions, deny: [...deny, ...missing] };
+  await writeSettingsJson(root, settings);
+  log.success("Added credential deny rules (SSH, AWS, npm)");
+  return true;
+}
+async function addBypassDisable(root) {
+  const settings = await readSettingsJson(root);
+  if (settings.disableBypassPermissionsMode === "disable") return false;
+  settings.disableBypassPermissionsMode = "disable";
+  await writeSettingsJson(root, settings);
+  log.success("Added disableBypassPermissionsMode: disable");
+  return true;
+}
+async function addSandboxSettings(root) {
+  const settings = await readSettingsJson(root);
+  const sandbox = settings.sandbox;
+  if (sandbox?.enabled === true) return false;
+  settings.sandbox = { enabled: true, failIfUnavailable: true };
+  await writeSettingsJson(root, settings);
+  log.success("Enabled sandbox with failIfUnavailable");
+  return true;
+}
+async function addEnvToClaudeignore(root) {
+  const ignorePath = join5(root, ".claudeignore");
+  let content;
+  try {
+    content = await readFile4(ignorePath, "utf-8");
+  } catch {
+    return false;
+  }
+  const lines = content.split("\n").map((l) => l.trim());
+  if (lines.some((l) => l === ".env" || l === ".env.*" || l === ".env*")) return false;
+  await writeFile2(ignorePath, content.trimEnd() + "\n.env\n.env.*\n");
+  log.success("Added .env to .claudeignore");
+  return true;
+}
 async function addClaudeMdSection(root, heading, content) {
   const claudeMdPath = join5(root, "CLAUDE.md");
   let existing;
@@ -1608,6 +1721,7 @@ function createDoctorCommand() {
 
 // src/commands/eval/index.ts
 import { Command as Command3 } from "commander";
+import { select } from "@inquirer/prompts";
 import ora from "ora";
 import chalk2 from "chalk";
 import { mkdir as mkdir4, writeFile as writeFile4 } from "fs/promises";
@@ -1775,7 +1889,7 @@ async function listYamlFiles(dir) {
 }
 
 // src/commands/eval/runner.ts
-import { mkdir as mkdir3, writeFile as writeFile3, readFile as readFile6, readdir as readdir4, rm } from "fs/promises";
+import { mkdir as mkdir3, writeFile as writeFile3, readFile as readFile6, readdir as readdir4, rm, cp, access as access6 } from "fs/promises";
 import { join as join8, dirname as dirname3 } from "path";
 import { tmpdir } from "os";
 import { randomUUID } from "crypto";
@@ -1785,7 +1899,7 @@ var exec = promisify(execFile);
 async function runScenario(scenario, options) {
   const sandboxDir = join8(tmpdir(), `claude-eval-${randomUUID()}`);
   try {
-    await setupSandbox(sandboxDir, scenario);
+    await setupSandbox(sandboxDir, scenario, options.projectRoot);
     await runClaudeInSandbox(sandboxDir, scenario.prompt, options.timeout, options.model);
     return await scoreResults(scenario, sandboxDir);
   } finally {
@@ -1806,13 +1920,14 @@ async function runScenarioWithRetries(scenario, options) {
   const sorted = [...results].sort((a, b) => a.score - b.score);
   return sorted[Math.floor(sorted.length / 2)];
 }
-async function setupSandbox(sandboxDir, scenario) {
+async function setupSandbox(sandboxDir, scenario, projectRoot) {
   await mkdir3(sandboxDir, { recursive: true });
   for (const file of scenario.setup.files) {
     const filePath = join8(sandboxDir, file.path);
     await mkdir3(dirname3(filePath), { recursive: true });
     await writeFile3(filePath, file.content);
   }
+  await copyProjectConfig(sandboxDir, projectRoot);
   if (scenario.setup.instructions) {
     await writeFile3(
       join8(sandboxDir, "CLAUDE.md"),
@@ -1835,6 +1950,31 @@ ${scenario.setup.instructions}
     "eval setup"
   ], { cwd: sandboxDir });
 }
+async function copyProjectConfig(sandboxDir, projectRoot) {
+  const claudeDir = join8(projectRoot, ".claude");
+  const sandboxClaudeDir = join8(sandboxDir, ".claude");
+  const settingsPath = join8(claudeDir, "settings.json");
+  if (await fileExistsSafe(settingsPath)) {
+    await mkdir3(sandboxClaudeDir, { recursive: true });
+    await cp(settingsPath, join8(sandboxClaudeDir, "settings.json"));
+  }
+  const rulesDir = join8(claudeDir, "rules");
+  if (await fileExistsSafe(rulesDir)) {
+    await cp(rulesDir, join8(sandboxClaudeDir, "rules"), { recursive: true });
+  }
+  const ignorePath = join8(projectRoot, ".claudeignore");
+  if (await fileExistsSafe(ignorePath)) {
+    await cp(ignorePath, join8(sandboxDir, ".claudeignore"));
+  }
+}
+async function fileExistsSafe(path) {
+  try {
+    await access6(path);
+    return true;
+  } catch {
+    return false;
+  }
+}
 async function runClaudeInSandbox(cwd, prompt, timeout, model) {
   try {
     const sdk = await import("@anthropic-ai/claude-agent-sdk");
@@ -1991,6 +2131,35 @@ async function listAllFiles(dir) {
 function createEvalCommand() {
   return new Command3("eval").description("Test your Claude Code config against eval scenarios").option("-s, --suite <suite>", "Eval suite to run (e.g., security, conventions, workflow)").option("-p, --path <path>", "Project root path", process.cwd()).option("--scenarios <path>", "Custom scenarios directory").option("--runs <n>", "Runs per scenario (default: 3)", "3").option("--timeout <ms>", "Timeout per run in ms (default: 120000)", "120000").option("--json", "Output as JSON").option("--debug", "Keep sandbox directories for inspection").option("--model <model>", "Model to use for eval (e.g., sonnet, haiku, opus)").action(async (opts) => {
     printBanner();
+    const hasFlags = opts.suite || opts.model || opts.runs !== "3" || opts.json || opts.debug;
+    if (!hasFlags) {
+      opts.suite = await select({
+        message: "Suite",
+        choices: [
+          { name: "security (6 scenarios)", value: "security" },
+          { name: "conventions (5 scenarios)", value: "conventions" },
+          { name: "workflow (2 scenarios)", value: "workflow" },
+          { name: "all (13 scenarios)", value: void 0 }
+        ]
+      });
+      opts.runs = await select({
+        message: "Runs per scenario",
+        choices: [
+          { name: "1 \u2014 fast", value: "1" },
+          { name: "3 \u2014 default", value: "3" },
+          { name: "5 \u2014 thorough", value: "5" }
+        ]
+      });
+      opts.model = await select({
+        message: "Model",
+        choices: [
+          { name: "haiku \u2014 cheapest", value: "haiku" },
+          { name: "sonnet \u2014 balanced", value: "sonnet" },
+          { name: "opus \u2014 best", value: "opus" }
+        ]
+      });
+      log.blank();
+    }
     const claudeAvailable = await checkClaudeCli();
     if (!claudeAvailable) {
       log.error("Claude CLI not found. Install it: https://docs.anthropic.com/en/docs/claude-code");
@@ -2154,7 +2323,7 @@ async function checkClaudeCli() {
 import { Command as Command4 } from "commander";
 import { spawn, execFile as execFile2 } from "child_process";
 import { promisify as promisify2 } from "util";
-import { access as access6 } from "fs/promises";
+import { access as access7 } from "fs/promises";
 import { join as join10 } from "path";
 var execAsync = promisify2(execFile2);
 var ENHANCE_PROMPT = `Read CLAUDE.md and the project's codebase, then update CLAUDE.md to fill in missing or incomplete sections.
@@ -2192,7 +2361,7 @@ function createEnhanceCommand() {
     const root = opts.path;
     const claudeMdPath = join10(root, "CLAUDE.md");
     try {
-      await access6(claudeMdPath);
+      await access7(claudeMdPath);
     } catch {
       log.error("No CLAUDE.md found. Run `claude-launchpad init` first.");
       process.exit(1);