ace-interview-prep 0.1.3 → 0.1.4

package/README.md

@@ -1,10 +1,9 @@
 # ace
 
 [![npm version](https://img.shields.io/npm/v/ace-interview-prep.svg)](https://www.npmjs.com/package/ace-interview-prep)
-[![CI](https://github.com/neel/ace-interview-prep/actions/workflows/ci.yml/badge.svg)](https://github.com/neel/ace-interview-prep/actions/workflows/ci.yml)
 [![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
 
-A CLI tool for staff-engineer-level frontend interview preparation. Scaffolds questions with test cases, tracks progress with scorecards, and provides LLM-powered feedback.
+A CLI tool for interview prep focusing on frontend. Scaffolds questions with test cases, tracks progress with scorecards, and provides LLM-powered feedback.
 
 ## Install
 
@@ -12,12 +11,6 @@ A CLI tool for staff-engineer-level frontend interview preparation. Scaffolds qu
 npm install -g ace-interview-prep
 ```
 
-Or run directly with npx:
-
-```bash
-npx ace-interview-prep help
-```
-
 ## Quick Start
 
 ### 1. Configure API Keys
@@ -27,24 +20,41 @@ ace setup
 ```
 
 Stores your OpenAI / Anthropic API keys in `~/.ace/config.json` (one-time, works across all workspaces).
+If both keys are valid, `ace setup` prompts you to choose a default provider (`openai` or `anthropic`).
 
 ```bash
 # Non-interactive
 ace setup --openai-key sk-... --anthropic-key sk-ant-...
+
+# Set default provider explicitly when both keys are configured
+ace setup --openai-key sk-... --anthropic-key sk-ant-... --default-provider anthropic
 ```
 
 ### 2. Initialize a Workspace
 
-Navigate to any folder where you want to practice:
+Navigate to any folder where you want to practice, then run:
 
 ```bash
 ace init
 ```
 
-Creates a `questions/` directory and vitest config files. Then install the test dependencies:
+`ace init` bootstraps the workspace and installs dependencies for you. It:
 
 ```bash
-npm install vitest happy-dom @testing-library/jest-dom
+# Creates/updates workspace files
+# - questions/
+# - package.json (adds test scripts + devDependencies if missing)
+# - tsconfig.json
+# - vitest.config.ts
+# - vitest.setup.ts
+#
+# Installs dependencies automatically via npm install
+```
+
+If you need to regenerate workspace files:
+
+```bash
+ace init --force
 ```
 
 ### 3. Practice
@@ -69,6 +79,7 @@ ace list
 ### 4. Test, Review, Track
 
 All commands below work in three modes:
+
 - **Interactive** — run with no arguments to pick from a selectable list
 - **Direct** — pass a slug to target a specific question
 - **All** — pass `--all` to operate on every question
@@ -96,6 +107,23 @@ ace reset debounce    # specific question
 ace reset --all       # reset everything (with confirmation)
 ```
 
+### 5. Dispute Potentially Incorrect Tests
+
+Use this when your implementation appears correct but a generated test assertion might be wrong.
+
+```bash
+# Dispute interactively (pick a question)
+ace dispute
+
+# Dispute a specific question
+ace dispute debounce
+
+# Optional: force a provider for dispute analysis
+ace dispute debounce --provider anthropic
+```
+
+If the verdict says the test is incorrect (or ambiguous), ace can apply a corrected test file and re-run tests.
+
 ## Question Categories
 
 | Category | Slug | Type | Focus |
@@ -126,11 +154,13 @@ ace reset --all       # reset everything (with confirmation)
 - `~/.ace/.env` — fallback (dotenv format)
 - Environment variables — final fallback
 
-**Workspace** — each workspace gets its own `questions/` directory and test config.
+Typical `~/.ace/config.json` keys:
 
-## Seed Questions
+- `OPENAI_API_KEY`
+- `ANTHROPIC_API_KEY`
+- `default_provider` (set automatically or via `ace setup --default-provider ...`)
 
-Ships with 8 starter questions (one per category) so you can begin practicing immediately after install.
+**Workspace** — each workspace gets its own `questions/` directory and test config.
 
 ## Development
 

package/dist/commands/dispute.js

@@ -0,0 +1,226 @@
+import fs from "node:fs";
+import path from "node:path";
+import { spawnSync } from "node:child_process";
+import prompts from "prompts";
+import chalk from "chalk";
+import { findQuestion, readScorecard, writeScorecard, promptForSlug } from "../lib/scorecard.js";
+import { CATEGORIES, isDesignCategory } from "../lib/categories.js";
+import { chat, requireProvider } from "../lib/llm.js";
+import { resolveWorkspaceRoot, isWorkspaceInitialized } from "../lib/paths.js";
+const PROMPTS_DIR = path.resolve(import.meta.dirname, "../prompts");
+function loadPrompt(name) {
+  return fs.readFileSync(path.join(PROMPTS_DIR, name), "utf-8");
+}
+function extractJSON(text) {
+  const match = text.match(/```json\s*([\s\S]*?)```/);
+  if (match) return match[1].trim();
+  const jsonMatch = text.match(/\{[\s\S]*\}/);
+  if (jsonMatch) return jsonMatch[0];
+  return text;
+}
+function parseArgs(args) {
+  let slug;
+  let provider;
+  for (let i = 0; i < args.length; i++) {
+    const arg = args[i];
+    if (arg === "--provider" && args[i + 1]) {
+      provider = args[++i];
+    } else if (!arg.startsWith("--")) {
+      slug = arg;
+    }
+  }
+  return { slug, provider };
+}
+function runTestsAndCapture(projectRoot, questionDir) {
+  const result = spawnSync("npx", ["vitest", "run", questionDir], {
+    cwd: projectRoot,
+    encoding: "utf-8"
+  });
+  const stdout = result.stdout || "";
+  const stderr = result.stderr || "";
+  const output = [stdout, stderr].filter(Boolean).join("\n");
+  const exitCode = typeof result.status === "number" ? result.status : 1;
+  return { output, exitCode };
+}
+async function run(args) {
+  const projectRoot = resolveWorkspaceRoot();
+  if (!isWorkspaceInitialized(projectRoot)) {
+    console.error(chalk.red("\nError: Workspace not initialized."));
+    console.error(chalk.dim("Run `ace init` in this folder first.\n"));
+    process.exit(1);
+  }
+  const parsed = parseArgs(args);
+  let selectedSlug = parsed.slug;
+  if (!selectedSlug) {
+    selectedSlug = await promptForSlug();
+    if (!selectedSlug) return;
+  }
+  const question = findQuestion(selectedSlug);
+  if (!question) {
+    console.error(chalk.red(`Question not found: ${selectedSlug}`));
+    return;
+  }
+  if (isDesignCategory(question.category)) {
+    console.log(chalk.yellow(`"${selectedSlug}" is a system design question \u2014 no tests to dispute.`));
+    return;
+  }
+  const config = CATEGORIES[question.category];
+  const readmePath = path.join(question.dir, "README.md");
+  const readme = fs.existsSync(readmePath) ? fs.readFileSync(readmePath, "utf-8") : "";
+  if (!readme.trim()) {
+    console.error(chalk.red("No README.md found for this question."));
+    return;
+  }
+  let solutionContent = "";
+  for (const f of config.solutionFiles) {
+    const fp = path.join(question.dir, f);
+    if (fs.existsSync(fp)) {
+      const content = fs.readFileSync(fp, "utf-8");
+      solutionContent += `
+--- ${f} ---
+${content}
+`;
+    }
+  }
+  if (!solutionContent.trim() || solutionContent.includes("// TODO: implement")) {
+    console.error(chalk.yellow("Solution appears to be the default stub. Write your solution first!"));
+    return;
+  }
+  let testContent = "";
+  let testFilePath = "";
+  for (const f of config.testFiles) {
+    const fp = path.join(question.dir, f);
+    if (fs.existsSync(fp)) {
+      testContent += fs.readFileSync(fp, "utf-8");
+      testFilePath = fp;
+    }
+  }
+  if (!testContent.trim()) {
+    console.error(chalk.red("No test file found for this question."));
+    return;
+  }
+  console.log(chalk.cyan(`
+Running tests for "${selectedSlug}" to capture failures...
+`));
+  const initialRun = runTestsAndCapture(projectRoot, question.dir);
+  const testOutput = initialRun.output;
+  if (initialRun.exitCode === 0) {
+    console.log(chalk.green("All tests are passing \u2014 nothing to dispute."));
+    return;
+  }
+  const provider = requireProvider(parsed.provider);
+  console.log(chalk.yellow("\nFailing tests detected. Sending to LLM for analysis...\n"));
+  const systemPrompt = loadPrompt("test-dispute.md");
+  const userContent = `## Problem Statement
+${readme}
+
+## Solution Code
+${solutionContent}
+
+## Test File
+${testContent}
+
+## Test Failure Output
+\`\`\`
+${testOutput}
+\`\`\``;
+  const messages = [
+    { role: "system", content: systemPrompt },
+    { role: "user", content: userContent }
+  ];
+  const spinner = ["|", "/", "-", "\\"];
+  let spinIdx = 0;
+  const interval = setInterval(() => {
+    process.stdout.write(`\r${chalk.cyan(spinner[spinIdx++ % spinner.length])} Analyzing...`);
+  }, 120);
+  let response;
+  try {
+    response = await chat(provider, messages, true);
+  } finally {
+    clearInterval(interval);
+    process.stdout.write("\r" + " ".repeat(30) + "\r");
+  }
+  let result;
+  try {
+    result = JSON.parse(extractJSON(response));
+  } catch {
+    console.error(chalk.red("Failed to parse LLM response."));
+    console.log(chalk.dim(response));
+    return;
+  }
+  const verdictColors = {
+    test_incorrect: chalk.green,
+    solution_incorrect: chalk.red,
+    ambiguous: chalk.yellow
+  };
+  const verdictLabels = {
+    test_incorrect: "Test is incorrect",
+    solution_incorrect: "Solution has a bug",
+    ambiguous: "Ambiguous specification"
+  };
+  const color = verdictColors[result.verdict] || chalk.white;
+  const label = verdictLabels[result.verdict] || result.verdict;
+  console.log(chalk.bold(`
+Verdict: ${color(label)}
+`));
+  console.log(result.summary);
+  console.log(chalk.dim("\n--- Details ---\n"));
+  console.log(result.details);
+  if (result.failingTests?.length) {
+    console.log(chalk.dim("\n--- Per-Test Breakdown ---\n"));
+    for (const t of result.failingTests) {
+      const tColor = verdictColors[t.verdict] || chalk.white;
+      console.log(`  ${tColor("\u25CF")} ${chalk.bold(t.testName)}: ${tColor(verdictLabels[t.verdict] || t.verdict)}`);
+      console.log(`    ${t.explanation}`);
+      if (t.fixedAssertion) {
+        console.log(chalk.dim(`    Fix: ${t.fixedAssertion}`));
+      }
+    }
+  }
+  if ((result.verdict === "test_incorrect" || result.verdict === "ambiguous") && result.fixedTestCode) {
+    console.log("");
+    const { confirm } = await prompts({
+      type: "confirm",
+      name: "confirm",
+      message: "Apply the corrected test file?",
+      initial: true
+    });
+    if (confirm) {
+      fs.writeFileSync(testFilePath, result.fixedTestCode, "utf-8");
+      console.log(chalk.green(`
+Test file updated: ${path.relative(projectRoot, testFilePath)}`));
+      console.log(chalk.cyan("\nRe-running tests to verify...\n"));
+      const verifyRun = runTestsAndCapture(projectRoot, question.dir);
+      const verifyOutput = verifyRun.output;
+      console.log(verifyOutput);
+      const scorecard = readScorecard(question.category, selectedSlug);
+      if (scorecard) {
+        const passMatch = verifyOutput.match(/(\d+)\s+passed/);
+        const failMatch = verifyOutput.match(/(\d+)\s+failed/);
+        const passed = passMatch ? parseInt(passMatch[1], 10) : 0;
+        const failed = failMatch ? parseInt(failMatch[1], 10) : 0;
+        const total = passed + failed;
+        if (total > 0) {
+          if (scorecard.attempts.length === 0) {
+            scorecard.attempts.push({ attempt: 1, testsTotal: 0, testsPassed: 0, llmScore: null });
+          }
+          const current = scorecard.attempts[scorecard.attempts.length - 1];
+          current.testsTotal = total;
+          current.testsPassed = passed;
+          scorecard.status = passed === total ? "solved" : "attempted";
+          writeScorecard(question.category, selectedSlug, scorecard);
+          const resultColor = passed === total ? chalk.green : chalk.yellow;
+          console.log(resultColor(`Scorecard updated: ${passed}/${total} tests passed`));
+        }
+      }
+    } else {
+      console.log(chalk.dim("No changes made."));
+    }
+  } else if (result.verdict === "solution_incorrect" && result.hint) {
+    console.log(chalk.dim("\n--- Hint ---\n"));
+    console.log(chalk.yellow(result.hint));
+  }
+}
+export {
+  run
+};

package/dist/commands/setup.js

@@ -93,17 +93,42 @@ async function run(args) {
     anthropicValid = result.valid;
     anthropicError = result.error;
   }
+  if (openaiValid && anthropicValid) {
+    let defaultProvider = parsed["default-provider"];
+    if (!defaultProvider) {
+      const currentDefault = final.default_provider || "openai";
+      const { selected } = await prompts({
+        type: "select",
+        name: "selected",
+        message: "Default LLM provider:",
+        choices: [
+          { title: "OpenAI", value: "openai" },
+          { title: "Anthropic", value: "anthropic" }
+        ],
+        initial: currentDefault === "anthropic" ? 1 : 0
+      });
+      defaultProvider = selected;
+    }
+    if (defaultProvider === "openai" || defaultProvider === "anthropic") {
+      saveGlobalAceConfig({ default_provider: defaultProvider });
+    }
+  } else if (openaiValid) {
+    saveGlobalAceConfig({ default_provider: "openai" });
+  } else if (anthropicValid) {
+    saveGlobalAceConfig({ default_provider: "anthropic" });
+  }
+  const updated = loadAceConfig();
   console.log(chalk.cyan("\n\u256D\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u256E"));
   console.log(chalk.cyan("\u2502") + chalk.bold("  ace status") + "                            " + chalk.cyan("\u2502"));
   console.log(chalk.cyan("\u251C\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524"));
-  if (final.OPENAI_API_KEY) {
-    const detail = openaiValid ? maskApiKey(final.OPENAI_API_KEY) : openaiError || "validation failed";
+  if (updated.OPENAI_API_KEY) {
+    const detail = openaiValid ? maskApiKey(updated.OPENAI_API_KEY) : openaiError || "validation failed";
     printStatusLine("OpenAI key", openaiValid, detail);
   } else {
     printStatusLine("OpenAI key", null, "not configured");
   }
-  if (final.ANTHROPIC_API_KEY) {
-    const detail = anthropicValid ? maskApiKey(final.ANTHROPIC_API_KEY) : anthropicError || "validation failed";
+  if (updated.ANTHROPIC_API_KEY) {
+    const detail = anthropicValid ? maskApiKey(updated.ANTHROPIC_API_KEY) : anthropicError || "validation failed";
     printStatusLine("Anthropic key", anthropicValid, detail);
   } else {
     printStatusLine("Anthropic key", null, "not configured");
@@ -111,10 +136,17 @@ async function run(args) {
   console.log(chalk.cyan("\u251C\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524"));
   const ready = openaiValid === true || anthropicValid === true;
   printStatusLine("Ready", ready, ready ? "at least one provider configured" : "no valid API keys");
+  if (updated.default_provider) {
+    const providerName = updated.default_provider === "openai" ? "OpenAI" : "Anthropic";
+    printStatusLine("Default provider", true, providerName.toLowerCase());
+  }
   console.log(chalk.cyan("\u2570\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u256F\n"));
   if (!ready && (openaiValid === false || anthropicValid === false)) {
-    console.log(chalk.yellow("Warning: No valid API keys configured."));
-    console.log(chalk.dim("Run `ace setup` again with valid keys to use LLM features.\n"));
+    console.log(chalk.yellow("No valid API keys configured. You need at least one to use LLM features.\n"));
+    console.log(chalk.dim("Generate an API key from either provider:\n"));
+    console.log(chalk.cyan("  OpenAI    ") + chalk.dim(" https://platform.openai.com/api-keys"));
+    console.log(chalk.cyan("  Anthropic ") + chalk.dim(" https://console.anthropic.com/settings/keys"));
+    console.log(chalk.dim("\nThen run ") + chalk.cyan("ace setup") + chalk.dim(" again to configure your key.\n"));
   }
 }
 export {

package/dist/index.js

@@ -10,7 +10,8 @@ const COMMANDS = {
   test: () => import("./commands/test.js"),
   feedback: () => import("./commands/feedback.js"),
   reset: () => import("./commands/reset.js"),
-  score: () => import("./commands/score.js")
+  score: () => import("./commands/score.js"),
+  dispute: () => import("./commands/dispute.js")
 };
 function printHelp() {
   console.log(`
@@ -30,6 +31,7 @@ ${chalk.bold("Question Commands:")}
   ${chalk.green("feedback")}   Get LLM code review or design review
   ${chalk.green("reset")}      Reset a question to unanswered state
   ${chalk.green("score")}      View scorecard for a question
+  ${chalk.green("dispute")}    Challenge a potentially incorrect test case
 
 ${chalk.bold("Examples:")}
 
@@ -68,6 +70,10 @@ ${chalk.bold("Examples:")}
   ace reset
   ace reset debounce
   ace reset --all
+  
+  ${chalk.dim("# Dispute a test you think is wrong")}
+  ace dispute
+  ace dispute debounce
 `);
 }
 async function main() {

package/dist/lib/llm.js

@@ -17,9 +17,14 @@ function getAvailableProviders() {
   return providers;
 }
 function getDefaultProvider() {
-  const providers = getAvailableProviders();
-  if (providers.length === 0) return null;
-  return providers[0];
+  const config = getConfig();
+  const available = getAvailableProviders();
+  if (available.length === 0) return null;
+  if (config.default_provider && available.includes(config.default_provider)) {
+    return config.default_provider;
+  }
+  if (available.includes("openai")) return "openai";
+  return available[0];
 }
 function requireProvider(preferred) {
   const config = getConfig();
@@ -44,7 +49,7 @@ async function callOpenAI(messages, jsonMode = false) {
   const config = getConfig();
   const client = new OpenAI({ apiKey: config.OPENAI_API_KEY });
   const response = await client.chat.completions.create({
-    model: "gpt-4o",
+    model: "gpt-5.2",
     messages: messages.map((m) => ({ role: m.role, content: m.content })),
     temperature: 0.7,
     max_tokens: 4096,
@@ -58,7 +63,7 @@ async function callAnthropic(messages, _jsonMode = false) {
   const systemMsg = messages.find((m) => m.role === "system");
   const nonSystemMessages = messages.filter((m) => m.role !== "system");
   const response = await client.messages.create({
-    model: "claude-sonnet-4-20250514",
+    model: "claude-sonnet-4-5-20250929",
     max_tokens: 4096,
     ...systemMsg ? { system: systemMsg.content } : {},
     messages: nonSystemMessages.map((m) => ({
@@ -80,7 +85,7 @@ async function streamOpenAI(messages) {
   const config = getConfig();
   const client = new OpenAI({ apiKey: config.OPENAI_API_KEY });
   const stream = await client.chat.completions.create({
-    model: "gpt-4o",
+    model: "gpt-5.2",
     messages: messages.map((m) => ({ role: m.role, content: m.content })),
     temperature: 0.7,
     max_tokens: 4096,
@@ -101,7 +106,7 @@ async function streamAnthropic(messages) {
   const systemMsg = messages.find((m) => m.role === "system");
   const nonSystemMessages = messages.filter((m) => m.role !== "system");
   const stream = client.messages.stream({
-    model: "claude-sonnet-4-20250514",
+    model: "claude-sonnet-4-5-20250929",
     max_tokens: 4096,
     ...systemMsg ? { system: systemMsg.content } : {},
     messages: nonSystemMessages.map((m) => ({
@@ -140,7 +145,7 @@ async function validateAnthropicKey(apiKey) {
   try {
     const client = new Anthropic({ apiKey });
     await client.messages.create({
-      model: "claude-sonnet-4-20250514",
+      model: "claude-sonnet-4-5-20250929",
       max_tokens: 1,
       messages: [{ role: "user", content: "hi" }]
     });

package/dist/prompts/question-generate.md

@@ -66,6 +66,24 @@ Return a JSON object with:
 - No `signature`, `testCode`, or `solutionCode`
 - The description must include a clear **Requirements** section that candidates can use to structure their design
 
+## Common Test Mistakes to Avoid
+
+Before finalizing `testCode`, mentally execute each test case against a correct implementation step-by-step. Verify that every expected value is accurate.
+
+- **Wrong expected values**: Double-check computed outputs (sorted arrays, mathematical results, string transformations) by tracing through the algorithm by hand
+- **Off-by-one errors**: Verify boundary indices, slice ranges, and loop counts in expected outputs
+- **Incorrect sort order**: Ensure expected output matches the exact sort direction (ascending vs descending) and sort key specified in the problem
+- **Async timing**: For debounce/throttle/timer tests, ensure timing assertions match the described delay behavior — account for whether the function fires on the leading edge, trailing edge, or both
+- **Floating point**: Use `toBeCloseTo` instead of `toBe` for floating point comparisons
+- **Reference vs value equality**: Use `toEqual` for deep object/array comparisons, not `toBe`
+- **Hardcoded magic values**: Never guess an expected value — derive it from the problem constraints and input
+
+If a test involves a computed result, add a brief inline comment explaining how the expected value was derived, e.g.:
+```
+// [1,2,3] -> sum = 6, mean = 6/3 = 2
+expect(mean([1, 2, 3])).toBe(2);
+```
+
 ## Quality Guidelines
 
 - Questions should be achievable within the suggested time for the category and difficulty

package/dist/prompts/test-dispute.md

@@ -0,0 +1,54 @@
+# Test Case Dispute Analyst
+
+You are an expert code reviewer specializing in test correctness. A candidate believes their solution is correct but one or more test cases are failing. Your job is to determine whether the **test** or the **solution** is at fault.
+
+## Input
+
+You will receive:
+- **Problem statement**: The question README describing expected behavior
+- **Solution code**: The candidate's implementation
+- **Test file**: The full test file
+- **Test failure output**: The actual vitest output showing which tests failed and why
+
+## Analysis Process
+
+1. Read the problem statement carefully to understand the **specified behavior**
+2. Trace the candidate's solution logic step-by-step
+3. For each failing test:
+   - Determine the **expected output** according to the problem statement (not the test)
+   - Trace the solution's actual output for that input
+   - Compare both to the test's expected value
+4. Classify the root cause as one of:
+   - `test_incorrect` — the test's expected value or assertion is wrong
+   - `solution_incorrect` — the solution has a bug
+   - `ambiguous` — the problem statement is unclear and both interpretations are valid
+
+## Output Format
+
+**IMPORTANT**: Your response MUST be valid JSON wrapped in ```json code fences. No other text before or after.
+
+```json
+{
+  "verdict": "test_incorrect | solution_incorrect | ambiguous",
+  "summary": "One-sentence summary of the finding",
+  "details": "Detailed explanation with step-by-step trace showing why the test or solution is wrong",
+  "failingTests": [
+    {
+      "testName": "name of the failing test",
+      "verdict": "test_incorrect | solution_incorrect | ambiguous",
+      "explanation": "Why this specific test is wrong or why the solution fails it",
+      "fixedAssertion": "The corrected expect(...) line, if test_incorrect. Omit if solution_incorrect."
+    }
+  ],
+  "fixedTestCode": "The complete corrected test file content (only if verdict is test_incorrect or ambiguous). Omit entirely if verdict is solution_incorrect.",
+  "hint": "A nudge toward the bug in the solution (only if verdict is solution_incorrect, without revealing the answer). Omit if test_incorrect."
+}
+```
+
+## Rules
+
+- Be precise: trace actual values, not hand-wavy reasoning
+- When the problem statement is the source of truth, favor it over both the test and the solution
+- If the test is wrong, provide the complete corrected test file — do not leave placeholders
+- If the solution is wrong, give a helpful hint without giving away the full fix
+- If ambiguous, explain both valid interpretations and provide a corrected test file that matches the more common/standard interpretation

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ace-interview-prep",
-  "version": "0.1.3",
+  "version": "0.1.4",
   "description": "CLI tool for frontend interview preparation — scaffolds questions with tests, tracks progress, and provides LLM-powered feedback",
   "type": "module",
   "bin": {

package/questions/design-be/url-shortener/README.md

@@ -1,23 +0,0 @@
-# Design a URL Shortener
-
-**Category:** Backend Design  
-**Difficulty:** Medium  
-**Suggested Time:** ~40 minutes
-
----
-
-## Problem
-
-Design a URL shortening service (like bit.ly) from a **backend perspective**.
-
-Consider the following aspects:
-
-- **Encoding scheme** — How to generate short codes (base62, UUID, etc.)
-- **Database design** — Schema for URLs, mappings, metadata
-- **Read/write ratio** — Optimize for heavy read traffic
-- **Caching** — Cache hot short URLs for fast redirects
-- **Analytics** — Click tracking, geographic data
-- **Expiration** — Optional TTL for short links
-- **Horizontal scaling** — Sharding, load balancing
-
-Walk through your design, data flow, and trade-offs. Use the `notes.md` file to capture your solution.

package/questions/design-be/url-shortener/notes.md

@@ -1,27 +0,0 @@
-# Design a URL Shortener — Design Notes
-
-## Requirements
-
-### Functional Requirements
-<!-- List the core features and user-facing requirements -->
-
-### Non-Functional Requirements
-<!-- Performance, scalability, availability, security, etc. -->
-
-## High-Level Architecture
-<!-- Describe the overall system architecture -->
-
-## Data Model
-<!-- Define key data structures, schemas, etc. -->
-
-## API Design
-<!-- Endpoint design, data contracts, etc. -->
-
-## Deep Dive
-<!-- Pick 1-2 areas to go deep on -->
-
-## Scaling Considerations
-<!-- How does this scale? -->
-
-## Trade-offs
-<!-- What trade-offs did you make? -->

package/questions/design-be/url-shortener/scorecard.json

@@ -1 +0,0 @@
-{"title":"Design a URL Shortener","category":"design-be","difficulty":"medium","suggestedTime":40,"status":"untouched","attempts":[],"llmFeedback":null}

package/questions/design-fe/news-feed/README.md

@@ -1,22 +0,0 @@
-# Design a News Feed
-
-**Category:** Frontend Design  
-**Difficulty:** Hard  
-**Suggested Time:** ~55 minutes
-
----
-
-## Problem
-
-Design a social media news feed (like Facebook or Twitter feed) from a **frontend perspective**.
-
-Consider the following aspects:
-
-- **Infinite scroll** — Loading more content as the user scrolls
-- **Real-time updates** — New posts appearing without a full refresh
-- **Optimistic UI** — Immediate feedback for likes, comments, shares
-- **Caching** — Stale-while-revalidate, cache invalidation strategies
-- **Virtualization** — Rendering only visible items for performance
-- **Accessibility** — Screen readers, keyboard navigation, focus management
-
-Walk through your design, data flow, and trade-offs. Use the `notes.md` file to capture your solution.