@skyramp/mcp 0.0.64 → 0.1.0-rc.1

package/build/playwright/traceRecordingPrompt.js

@@ -14,44 +14,38 @@ export function registerTraceRecordingPrompt(server) {
                 role: "user",
                 content: {
                     type: "text",
-                    text: `## Skyramp Trace Recording & UI Test Generation
-
-You have access to Playwright browser tools that let you interact with web applications.
-Use these tools to record a trace of browser interactions, then generate a Skyramp UI test from that trace.
-
-### Flow
-
-1. **Navigate**: ALWAYS call \`browser_navigate\` with the target URL as the very first step, even if the browser seems to already be on that page. This ensures a clean state.
-2. **Understand the page**: Call \`browser_snapshot\` to see the current page state (ARIA tree).
-3. **Interact**: Use \`browser_click\`, \`browser_type\`, \`browser_select_option\`, etc. to perform the user interactions described in the prompt.
-4. **Repeat steps 2-3** until all interactions are complete. Assertions are automatically added at strategic points during export.
-5. **Export the trace**: Call \`skyramp_export_zip\` with an output path (e.g. \`skyramp_export.zip\`). This produces a zip containing the JSONL trace and HAR network recording. Assertions are auto-injected based on API calls detected in the HAR.
-6. **Generate the test**: Call \`skyramp_ui_test_generation\` with \`playwrightInput\` set to the absolute path of the zip file from step 5.
-
-### Tips
-- **To type into a field**: Just use \`browser_type\` — it automatically clears the field and types the new value. Do NOT press Ctrl+A or any keyboard shortcuts before typing.
-- If a \`browser_click\` or \`browser_type\` fails because the element reference is stale (page updated), call \`browser_snapshot\` to refresh the page state and retry.
-- Use \`browser_snapshot\` liberally — it helps you understand what elements are available.
-- The trace automatically deduplicates retries: if you navigate back to the start URL and redo steps, only the last complete attempt is exported.
-- After generating the test, the tool will suggest running \`skyramp_modularization\` for code quality.
-- **Dropdown/Select components**: For custom dropdowns (Radix, MUI, etc.) that show as \`combobox\` in the snapshot, do NOT use \`browser_select_option\` — it only works on native \`<select>\` elements. Instead: (1) click the combobox to open the dropdown, (2) call \`browser_snapshot\` to see the options in a \`listbox\`, (3) click the desired \`option\`. This three-step pattern is required for all custom dropdown components.
-- **Always take a snapshot after each interaction** that changes the page (click, form submit, navigation) to see the updated state before proceeding.
-
-### Critical rules for clicking
-- **NEVER click container/wrapper divs** (e.g. elements with "container" in their test-id). Always click the actual interactive element inside: a \`button\`, \`link\`, or \`input\`.
-- When the snapshot shows a container with a button inside, click the **button**, not the container. For example, if you see \`div "add-order-products-container" > button "Add"\`, click the button "Add", not the container.
-- To submit forms, click the submit \`button\` (e.g. "Add Order", "Submit"), never the form container.
-- After selecting a product from a dropdown, click the "Add" button to confirm, not the surrounding container.
+                    text: `## Skyramp UI Test Recording
+
+You are a Skyramp Integration Architect. Your role is to record browser interactions with zero hallucination: every action must be grounded in what \`browser_snapshot\` returns. If an element is not visible in the snapshot, do not interact with it.
+
+### Required workflow
+
+Before starting, output a \`<thinking>\` block that maps each step of the user's intent to the specific browser interactions required. Do not call any tool until this mapping is complete.
+
+Then execute in strict order:
+
+1. **Navigate**: Call \`browser_navigate\` with the target URL. Always do this first, even if the browser appears to be on the correct page.
+2. **Snapshot**: Call \`browser_snapshot\` to get the current ARIA tree and element refs.
+3. **Interact**: Call the appropriate tool (\`browser_click\`, \`browser_type\`, \`browser_hover\`, etc.) using refs from the snapshot.
+4. **Repeat steps 2–3** for each user action until all steps are complete.
+5. **Export**: Call \`skyramp_export_zip\` with \`outputPath\` set to the absolute zip path (same directory and base name as the test file, replacing \`.spec.ts\` with \`.zip\`). Do NOT ask the user first — call it automatically.
+6. **Generate**: Call \`skyramp_ui_test_generation\` with \`playwrightInput\` set to the absolute zip path from step 5.
+
+### Cross-tool rules
+
+- **After every action that changes the page**, call \`browser_snapshot\` before the next interaction — refs become stale after navigation, clicks that trigger page updates, and form submissions.
+- **Iframe content** appears inline in the snapshot — interact with those elements using their refs normally.
+- **Trace deduplication**: if you retry from the start URL, only the last complete attempt is exported.
+- **After generating the test**, run \`skyramp_modularization\` for code quality.
 
 ### Assertions
-If the user requests assertions, you MUST call \`browser_assert\` at the appropriate points. Always provide the \`expected\` value.
-- \`type: "text"\` — verify element contains expected text (e.g., product name appears after creation)
-- \`type: "value"\` — verify input field has expected value (e.g., price field shows "29.99")
-
-### Important
-- Do NOT ask the user before calling \`skyramp_export_zip\` — call it automatically as the final step.
-- Do NOT write JSONL or HAR files manually — the export tool handles everything.
-- Do NOT reuse existing zip files from previous sessions — always record fresh.
+Call \`browser_assert\` when the user requests verification. Always provide the \`expected\` value.
+- \`type: "text"\` — verify an element contains expected text
+- \`type: "value"\` — verify an input field has an expected value
+
+### Constraints
+- Do NOT write JSONL or HAR files manually — \`skyramp_export_zip\` handles everything.
+- Do NOT reuse zip files from previous sessions — always record fresh.
 `,
                 },
             },

package/build/prompts/architectPersona.js

@@ -0,0 +1,19 @@
+/**
+ * Skyramp Integration Architect persona injected into generation tool descriptions.
+ *
+ * In TestBot environments (ENABLE_SKYRAMP_TESTBOT=true), the persona is injected
+ * once as a system prompt via `claude --system-prompt` rather than repeating it in
+ * every tool description. In that case this string is omitted from the tool description
+ * to avoid wasting context tokens.
+ *
+ * In IDE/MCP-direct environments, it is included in each tool description so the
+ * model has the role context available without a separate system prompt.
+ */
+export const SKYRAMP_ARCHITECT_PERSONA = `You are acting as a Skyramp Integration Architect. Your responsibility is to map the user's test intent to the Skyramp generation spec with precision. No guessing — derive all parameters from the codebase, workspace config, and provided context only.`;
+/**
+ * Returns the persona prefix for use in tool descriptions.
+ * Returns an empty string when running inside TestBot (persona is injected via system prompt instead).
+ */
+export function getPersonaPrefix() {
+    return process.env.ENABLE_SKYRAMP_TESTBOT ? '' : `${SKYRAMP_ARCHITECT_PERSONA}\n\n`;
+}

package/build/prompts/test-maintenance/drift-analysis-prompt.js

@@ -30,7 +30,16 @@ No existing Skyramp tests found in repository.
 `;
     const scannedSection = scannedEndpoints.length > 0
         ? `## Scanned Endpoints (${scannedEndpoints.length})
-${scannedEndpoints.map((ep) => `- ${Array.isArray(ep.methods) ? ep.methods.join("|") : ep.method} ${ep.path}`).join("\n")}
+${scannedEndpoints.map((ep) => {
+            let methods;
+            if (Array.isArray(ep.methods)) {
+                methods = ep.methods.map((m) => (typeof m === "string" ? m : m.method)).join("|");
+            }
+            else {
+                methods = ep.method;
+            }
+            return `- ${methods} ${ep.path}`;
+        }).join("\n")}
 `
         : "";
     // In inline mode (testbot), skip the context header — existing tests and diff
@@ -70,9 +79,5 @@ ${buildUpdateExecutionRules()}
 
 ${buildAddRecommendationGuidelines()}
 
-${buildDriftOutputChecklist(existingTests.length, newEndpointCount, inlineMode)}
-
-After completing the assessment above, call \`skyramp_actions\` with \`stateFile: "${stateFile}"\`
-
-**CRITICAL**: Do NOT create any .json or .md files. Only call skyramp_actions when done.`;
+${buildDriftOutputChecklist(existingTests.length, newEndpointCount, inlineMode, stateFile)}`;
 }

package/build/prompts/test-maintenance/drift-analysis-prompt.test.js

@@ -0,0 +1,49 @@
+import { buildDriftAnalysisPrompt } from "./drift-analysis-prompt.js";
+describe("buildDriftAnalysisPrompt - scanned endpoints rendering", () => {
+    // Reproduces the [object Object] bug: skeletonEndpoints from analyzeChangesTool
+    // stores methods as objects { method: string, ... }, not plain strings.
+    const skeletonMethodObjects = [
+        {
+            path: "/api/v1/",
+            methods: [{ method: "GET", description: "", queryParams: [], authRequired: true, sourceFile: "main.py", interactions: [] }],
+            resourceGroup: "v1",
+            pathParams: [],
+        },
+        {
+            path: "/api/v1/orders",
+            methods: [
+                { method: "GET", description: "", queryParams: [], authRequired: true, sourceFile: "orders.py", interactions: [] },
+                { method: "POST", description: "", queryParams: [], authRequired: true, sourceFile: "orders.py", interactions: [] },
+            ],
+            resourceGroup: "orders",
+            pathParams: [],
+        },
+    ];
+    it("renders HTTP methods as strings, not [object Object]", () => {
+        const prompt = buildDriftAnalysisPrompt({
+            existingTests: [],
+            scannedEndpoints: skeletonMethodObjects,
+            repositoryPath: "/repo",
+            stateFile: "/tmp/state.json",
+        });
+        expect(prompt).not.toContain("[object Object]");
+        expect(prompt).toContain("GET /api/v1/");
+        expect(prompt).toContain("GET|POST /api/v1/orders");
+        // CTA should appear exactly once (not duplicated)
+        const ctaCount = (prompt.match(/call `skyramp_actions`/g) || []).length;
+        expect(ctaCount).toBe(1);
+    });
+    it("also works with plain string methods (ScannedEndpoint format)", () => {
+        const stringMethods = [
+            { path: "/api/v1/products", methods: ["GET", "POST"], sourceFile: "products.py" },
+        ];
+        const prompt = buildDriftAnalysisPrompt({
+            existingTests: [],
+            scannedEndpoints: stringMethods,
+            repositoryPath: "/repo",
+            stateFile: "/tmp/state.json",
+        });
+        expect(prompt).not.toContain("[object Object]");
+        expect(prompt).toContain("GET|POST /api/v1/products");
+    });
+});

package/build/prompts/test-maintenance/driftAnalysisSections.js

@@ -163,12 +163,14 @@ Apply to **new test functions you are adding** and **existing functions that cov
 
 ${ENHANCE_ASSERTIONS_FOR_INTEGRATION_AND_CONTRACTPROVIDER}`;
 }
-export function buildDriftOutputChecklist(existingTestCount, newEndpointCount, inlineMode = false) {
+export function buildDriftOutputChecklist(existingTestCount, newEndpointCount, inlineMode = false, stateFile) {
     const finalStep = inlineMode
         ? `### Final step
 Apply all maintenance actions (UPDATE / REGENERATE / DELETE) directly by editing the test files. New test generation (ADD) is handled separately in the next step.`
         : `### Final step
-After completing all assessments above, call \`skyramp_actions\` with the stateFile to execute the recommended changes.`;
+After completing all assessments above, call \`skyramp_actions\` with \`stateFile: "${stateFile}"\` to execute the recommended changes.
+
+**CRITICAL**: Do NOT create any .json or .md files. Only call skyramp_actions when done.`;
     // In inline mode, existing test counts are unknown at prompt-build time —
     // they come from skyramp_analyze_changes at runtime. Skip the count headers.
     const existingTestSection = inlineMode

package/build/prompts/test-recommendation/test-recommendation-prompt.js

@@ -1,6 +1,7 @@
 import * as crypto from "crypto";
 import { WorkspaceAuthType } from "../../utils/workspaceAuth.js";
 import { buildToolWorkflows, buildTestPatternGuidelines, buildTestQualityCriteria, buildTestExamples, buildGenerationRules, getAuthSnippets, MAX_TESTS_TO_GENERATE, MAX_RECOMMENDATIONS, MAX_CRITICAL_TESTS, } from "./recommendationSections.js";
+import { CATEGORY_PRIORITY, TEST_CATEGORIES } from "../../types/TestRecommendation.js";
 function formatTestLocations(locs) {
     const entries = Object.entries(locs || {});
     if (entries.length === 0)
@@ -9,18 +10,11 @@ function formatTestLocations(locs) {
         "  If a GENERATE item's resource path matches a path listed here, UPDATE that file instead of creating a new one.\n" +
         entries.map(([type, files]) => "  - [" + type + "] " + files).join("\n");
 }
-const CATEGORY_PRIORITY = {
-    new_endpoint: "CRITICAL", // diff-direct scenarios always fill GENERATE slots first
-    security_boundary: "HIGH",
-    business_rule: "HIGH",
-    data_integrity: "HIGH",
-    breaking_change: "HIGH",
-    auth: "HIGH",
-    workflow: "MEDIUM",
-    "error-handling": "MEDIUM",
-    "data-validation": "MEDIUM",
-    crud: "LOW",
-};
+// ── Priority-tier ordering (replaces numeric CATEGORY_WEIGHTS) ──
+// Categories map to HIGH / MEDIUM / LOW tiers.
+// Within a tier, novelty (new > modified > existing) breaks ties,
+// then cross-resource, step count, and finally the deterministic SHA-256 seed.
+// CATEGORY_PRIORITY and PriorityTier imported from ../../types/TestRecommendation.js
 const PRIORITY_ORDER = { CRITICAL: 4, HIGH: 3, MEDIUM: 2, LOW: 1 };
 const NOVELTY_ORDER = { new: 3, modified: 2, existing: 1 };
 function classifyNovelty(scenario, diffContext) {
@@ -201,11 +195,32 @@ Seed: ${seed} | Endpoints: ${endpointCount} | Budget: ${generateItems.length + (
 
 **Step 0 — Existing-test cross-check (MANDATORY before executing anything)**
 For every GENERATE item below, check its endpoint path and test type against the Existing Tests list (further down in the prompt).
-- **Contract tests**: If an existing contract test already covers that resource path → UPDATE the existing file instead of creating a new one. This does NOT count toward \`newTestsCreated\` — backfill from ADDITIONAL candidates to fill the open ADD slot.
+- **Contract tests**: If an existing contract test already covers that resource path → UPDATE the existing file instead of creating a new one. This does NOT count toward \`newTestsCreated\` — backfill from ADDITIONAL candidates to fill the open ADD slot using this priority order:
+  1. **PR-endpoint edge cases first**: Look for integration test candidates covering error paths, boundary values, or alternative scenarios for the SAME endpoints changed in the PR diff. If no suitable candidate exists in ADDITIONAL, derive one from your source-code enrichment findings. These are always the highest-value backfill.
+  2. **Same-resource other scenarios**: Other HTTP methods or flows on the same resource group touched by the PR.
+  3. **Cross-resource workflows involving the PR endpoint**: Integration scenarios that include the PR's changed endpoint as one of the steps.
+  4. **Unrelated endpoint coverage (last resort)**: Tests for endpoints with no connection to the PR diff, only when ALL options above have been exhausted or would only produce UPDATEs (not new files).
+  **NEVER backfill with a test for a completely unrelated resource (e.g. \`POST /reviews\` when the PR only changes \`/orders\`) if any PR-endpoint edge-case integration test is feasible.**
 - **Integration/scenario tests**: Always generate as a new file via the scenario pipeline, even if an existing integration test covers the same resource. A new multi-step scenario is a distinct test. Count it toward \`newTestsCreated\`.
 - **UI tests**: Always generate as a new file. Count toward \`newTestsCreated\`.
 
-**Step 1 — Source-Code Enrichment (MANDATORY before executing anything)**
+**Step 1 — Diversity check (MANDATORY before executing anything)**
+Review the GENERATE list and verify that each item exercises a **distinct code path** — not just different input values on the same path.
+
+**What NOT to do (these are all violations — if you catch yourself doing any of these, STOP and replace one item):**
+- Do NOT generate two integration tests that both send a successful PUT/PATCH to the same endpoint and only differ in the request body values (e.g. 10% discount vs 5% discount vs 100% discount — these are the SAME test with different numbers)
+- Do NOT generate two tests with the same step sequence (e.g. both are POST→PUT→GET or both are POST→PUT) where the only variation is the payload
+- Do NOT count a "boundary value" as a separate test if the code path is identical to the happy path (e.g. discount=100% still returns 200 just like discount=10% — that is the same code path)
+- Do NOT use different scenario names to disguise duplicate tests (e.g. "orders-put-add-items-recalculate" and "orders-put-new-endpoint-happy-path" are duplicates if both POST an order then PUT with items and expect 200)
+
+**What TO do — each GENERATE item must exercise a different code path. Good diversity means a mix of:**
+- One **happy-path** integration test (the richest scenario: create prerequisites → call the new endpoint → verify computed fields and child collections)
+- One **error-path** test (trigger a distinct HTTP error status: 404 for non-existent resource, 422 for invalid input, 400 for malformed request — pick whichever the source code actually handles)
+- One **state-variation** test (different operation on the same endpoint that hits different logic: empty items array, removing items instead of adding, updating quantity without changing products)
+
+For each duplicate pair found, keep the richer item and replace the other with a test from a different category above. The replacement still targets the same PR endpoint and counts as a GENERATE item. Move the displaced item to ADDITIONAL.
+
+**Step 2 — Source-Code Enrichment (MANDATORY before executing anything)**
 Read the source code for ALL changed files. Look for:
 - **Auth middleware** (passport, jwt.verify, authMiddleware, @requires_auth, Depends(get_current_user), @UseGuards, EnsureSessionDep, session middleware) — if found, override \`authHeader\` and \`authScheme\` in scenario and contract tool calls even if workspace.yml says authType: none. Exception: for \`skyramp_integration_test_generation\` with \`scenarioFile\`, omit auth params entirely if workspace has \`api.authType\` set (workspace handles it); if workspace has no \`authType\`, pass \`authHeader\` only.
 - Business rules and formulas (e.g. total_cost = compute * rate + memory * rate)
@@ -227,7 +242,7 @@ Quality gate — ask all three questions:
   2. "Does this test exercise a real workflow or catch a real bug?" → YES = at least MEDIUM
   3. "Does this test cover a mutation that modifies child items and triggers total/amount recalculation?" → YES = HIGH priority, and prefer it for GENERATE over simple single-field update tests for the same endpoint
 
-Assign category: security_boundary | business_rule | data_integrity | breaking_change | auth | workflow | error-handling | data-validation | crud
+Assign category: ${TEST_CATEGORIES.join(" | ")}
 
 ${buildTestPatternGuidelines()}
 
@@ -247,7 +262,7 @@ When a qualifying candidate is inserted: place it HIGH before MEDIUM before LOW;
 
 **Unique constraints:** Unique-constraint scenarios (duplicate POST → expect 409) are pre-drafted for all resources. Before keeping them, check whether the storage backend actually enforces uniqueness — look for SQL \`UNIQUE\` indexes, Mongoose \`unique: true\`, Prisma \`@unique\`, or explicit duplicate-check logic in the source. If the backend is Redis, an in-memory store, or a schema-less DB with no explicit unique constraint in the changed files, move the unique-constraint scenario to ADDITIONAL with a note that enforcement is unconfirmed — do NOT generate it as a GENERATE item.
 
-**Step 2 — Execute merged plan in rank order**
+**Step 3 — Execute merged plan in rank order**
 Replace any scenario that pairs unrelated resources with one reflecting actual FK relationships in the codebase.
 Use realistic request bodies from source code schemas; verify response data (not just status codes).
 
@@ -266,7 +281,7 @@ ${buildGenerationRules(isUIOnlyPR)}
 
 **Critical-category minimum:** At least ${Math.min(MAX_CRITICAL_TESTS, maxGen)} of the ${maxGen} GENERATE items MUST be from HIGH-priority categories (security_boundary, business_rule, data_integrity, breaking_change). The pre-ranked plan below already prioritises this — only override if source-code enrichment reveals a higher-value candidate.
 
-### GENERATE (process these EXACTLY as listed, in order — do NOT reorder or replace any item with a different scenario; if Step 0 converts an item to UPDATE, backfill the ADD slot from ADDITIONAL)
+### GENERATE (process these EXACTLY as listed, in order — after completing Steps 0–2 above; if Step 0 converts an item to UPDATE, backfill the ADD slot from ADDITIONAL following the priority order in Step 0)
 
 ${generateBlocks || "  (no pre-ranked generate items — draft your own based on endpoint analysis)"}${reserveUIGenSlot ? `