@skyramp/mcp 0.1.4 → 0.1.5

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -143,8 +143,8 @@ When a diff adds a new HTTP method to a resource, UPDATE covers **all** existing
143
143
 
144
144
  ### PATCH/PUT with child collections (MANDATORY)
145
145
  When updating a contract or integration test for a PATCH or PUT endpoint whose request/response includes a child collection array (e.g. \`items\`, \`products\`, \`line_items\`):
146
- 1. The request body MUST include the child array with at least one item containing the Foreign Key field (e.g. \`product_id\`) and a \`quantity\` field.
147
- 2. Assert each item's Foreign Key field and \`quantity\` match the sent values.
146
+ 1. The request body MUST include the child array with at least one item containing the FK field (e.g. \`product_id\`) and a \`quantity\` field.
147
+ 2. Assert each item's FK field and \`quantity\` match the sent values.
148
148
  3. Assert the top-level computed total (e.g. \`total_amount\`) equals the expected math from the items.
149
149
  A test that only sends/asserts metadata (discount, status, notes) without asserting the items array is INCOMPLETE and will produce false passes even when the items/total logic is broken.
150
150
 
@@ -52,10 +52,10 @@ The ranked test recommendation catalog is pre-built and shown below (after the s
52
52
  **Your only job is to present it.**
53
53
 
54
54
  1. Fill in every \`<…from source>\` placeholder using the field names, computed formulas, and auth details you found in Steps 1–2.
55
- 2. Output the completed catalog **exactly as formatted**, preserving whatever test-type section headings are already present in the catalog. Do NOT restructure, reorder, rename sections, invent missing sections, or generate a new format.
55
+ 2. Output the completed catalog **exactly as formatted grouped by test type (### E2E / ### UI / ### Integration / ### Contract)**. Do NOT restructure, reorder, rename sections, or generate a new format.
56
56
  3. Do NOT call any Skyramp generation tools. The catalog shows ready-to-use tool calls that can be executed on demand.
57
57
 
58
- **If** Steps 1–2 revealed additional scenarios the catalog does not cover (e.g. a computed formula or Foreign Key relationship that was missed), you may optionally call \`skyramp_recommend_tests\` with \`stateFile: "${p.stateFile ?? p.sessionId}"\` and \`enrichedScenarios\` to regenerate a more complete catalog — but only after presenting the current one.`;
58
+ **If** Steps 1–2 revealed additional scenarios the catalog does not cover (e.g. a computed formula or FK relationship that was missed), you may optionally call \`skyramp_recommend_tests\` with \`stateFile: "${p.stateFile ?? p.sessionId}"\` and \`enrichedScenarios\` to regenerate a more complete catalog — but only after presenting the current one.`;
59
59
  const hasJavaFiles = p.candidateRouteFiles?.some(f => /\.(java|kt)$/.test(f)) ?? false;
60
60
  const routeFilesSection = p.candidateRouteFiles && p.candidateRouteFiles.length > 0
61
61
  ? `\nRoute/controller files found by static scan (read these to discover endpoints — the regex-based catalog below may be incomplete for your framework):\n${p.candidateRouteFiles.map(f => `- ${f}`).join("\n")}\n`
@@ -1,4 +1,4 @@
1
- import { isContractConsumerModeEnabled, resolveServiceDetailsRef } from "../../utils/featureFlags.js";
1
+ import { isContractConsumerModeEnabled } from "../../utils/featureFlags.js";
2
2
  import { WorkspaceAuthType, getAuthScheme, isAuthorizationHeaderName, AUTH_MIDDLEWARE_PATTERNS_STR } from "../../utils/workspaceAuth.js";
3
3
  // Cached at module-load — the flag is process-wide and cannot change per call.
4
4
  const CONSUMER_MODE_ENABLED = isContractConsumerModeEnabled();
@@ -42,45 +42,13 @@ Before calling any tool, replace every \`<from source>\` placeholder in the tool
42
42
  }
43
43
  export function buildReasoningProtocol() {
44
44
  return `<reasoning_protocol>
45
- ## Coverage Reasoning Block (MANDATORY — complete BEFORE your Budget Plan)
46
-
47
- Before committing to a Budget Plan and test list, produce a <thinking> block that enumerates ALL testable surfaces introduced or affected by this PR. This prevents narrow focus on a single endpoint/method.
48
-
49
- **For backend-only PRs**, your thinking MUST cover:
50
- 1. **All HTTP methods affected** — if a new validation/service method is added, trace ALL callers (not just createOne — also updateOne, updateMany, deleteOne). List every HTTP method × endpoint pair.
51
- 2. **Error paths per method** — for each endpoint-method, what error codes does the source code return? (400, 401, 403, 404, 409, 422). Each distinct error path is a potential test.
52
- 3. **Cross-service impact** — does the change affect other services that import the modified module? Those endpoints need coverage too.
53
- 4. **Data migrations** — if a migration exists, can its effect be verified via an API call? (e.g. backfill → GET should return the backfilled value)
54
-
55
- **For frontend-only PRs**, your thinking MUST cover:
56
- 1. **Component integration** — which routes render the changed component? Each route is a test target.
57
- 2. **User interactions** — what actions can a user perform on the changed component? (click, type, select, drag). Each distinct action flow is a test.
58
- 3. **State variations** — what different states does the component render? (empty, loading, error, populated, edge values)
59
-
60
- **For mixed (frontend + backend) PRs**, your thinking MUST cover:
61
- 1. All backend surfaces (methods 1–4 above)
62
- 2. All frontend surfaces (methods 1–3 above)
63
- 3. **E2E bridges** — which frontend components call the changed backend endpoints? Those are E2E test candidates that cover both layers in one test.
64
-
65
- **Output format in your thinking block:**
66
- \`\`\`
67
- Testable surfaces:
68
- - POST /permissions → happy path (201), invalid fields (422), missing collection (400)
69
- - PATCH /permissions/:id → update with valid fields (200), update with invalid fields (422)
70
- - GET /items/:collection?aggregate → with allowed fields (200), with forbidden fields (403)
71
- - UI: permissions field selector → add field, remove field, wildcard toggle
72
- Total distinct surfaces: N
73
- \`\`\`
74
-
75
- Your Budget Plan total MUST be ≥ the number of GENERATE slots and reflect the breadth of surfaces found. If you found 8 distinct surfaces but only budget 3 tests, you are under-covering the PR.
76
-
77
45
  ## Parameter Grounding Rule
78
46
  Before each GENERATE tool call, confirm WHERE each key value comes from:
79
47
 
80
48
  - **requestBody / responseBody fields** → source code schema (Zod, Pydantic, DTO), enriched scenario, or OpenAPI spec. **The generation tool rejects empty \`{}\` request bodies for POST/PUT/PATCH** — read the source schema first if the fields are unknown.
81
49
  - **endpointURL** → workspace \`baseUrl\` + endpoint path (both required — never path alone)
82
50
  - **authHeader / authScheme** → workspace config or OpenAPI \`securitySchemes\`
83
- - **Foreign Key path params** → chained from a prior step's response (check the actual field name — it may be \`id\`, \`uuid\`, \`_id\`, or a resource-specific \`*_id\` field). The chaining source can be a response body (POST or GET), a response header (e.g. \`Location\`), or a cookie — not hardcoded
51
+ - **FK path params** → chained from a prior step's response (check the actual field name — it may be \`id\`, \`uuid\`, \`_id\`, or a resource-specific \`*_id\` field). The chaining source can be a response body (POST or GET), a response header (e.g. \`Location\`), or a cookie — not hardcoded
84
52
  - **Names / string values** → realistic; append timestamp suffix to avoid re-run conflicts
85
53
 
86
54
  ## Ranking Rule
@@ -142,11 +110,11 @@ export function buildTestPatternGuidelines() {
142
110
  - **Middleware chains**: If auth/rate-limit/logging middleware exists, test the chain (e.g., rate limit hit → auth still checked → correct error returned)
143
111
  - **N+1 query risk**: If list endpoints join related data (e.g., orders with products), test with large datasets
144
112
  - **State machines**: If resources have status transitions (draft→published→archived), test invalid transitions (e.g., archived→draft should fail)
145
- - **Cascade deletes**: Only recommend after reading source code to confirm which resource holds the Foreign Key. The resource with the Foreign Key is the child; the one it points to is the parent. Example: if orders.product_id references products, then products is the parent — deleting a product tests whether orders are protected or cascade-deleted. Getting this backwards (treating the child as the parent) produces a nonsensical test.
113
+ - **Cascade deletes**: Only recommend after reading source code to confirm which resource holds the FK. The resource with the FK is the child; the one it points to is the parent. Example: if orders.product_id references products, then products is the parent — deleting a product tests whether orders are protected or cascade-deleted. Getting this backwards (treating the child as the parent) produces a nonsensical test.
146
114
  - **Race conditions**: If concurrent writes are possible (inventory deduction, counter increment), test concurrent requests
147
115
  - **Computed fields**: If response contains derived values (total, average, count), verify computation with known inputs (e.g., total_cost = compute_seconds * rate + memory_mb * rate + external_cost)
148
116
  - **Mutation with collection modification**: If PUT/PATCH endpoints accept arrays of child items (e.g., order line items, cart products, invoice entries), test adding/removing items and verify that derived totals (e.g., total_amount, subtotal, item_count) are recalculated correctly. This is the most common source of user-reported bugs — always prioritize it for GENERATE over simple field-update tests.
149
- The PATCH/PUT request body should include the child collection array field(s) defined for that endpoint (e.g., "items" with Foreign Key references like "product_id" and a quantity field) chained from prior POST responses. A PATCH that only sends metadata fields (e.g., discount_type, status, notes) without modifying the child collection is NOT a valid mutation-recalc test — it will pass even when the item/total logic is broken. Before writing assertions, inspect the source code or OpenAPI spec to identify (1) the actual child collection field name and its Foreign Key/quantity/price sub-fields, and (2) how derived totals are calculated (including any discounts, taxes, or fees). Then assert: the child Foreign Key fields match chained IDs, quantities match sent values, and totals match the computation from the source code
117
+ The PATCH/PUT request body should include the child collection array field(s) defined for that endpoint (e.g., "items" with FK references like "product_id" and a quantity field) chained from prior POST responses. A PATCH that only sends metadata fields (e.g., discount_type, status, notes) without modifying the child collection is NOT a valid mutation-recalc test — it will pass even when the item/total logic is broken. Before writing assertions, inspect the source code or OpenAPI spec to identify (1) the actual child collection field name and its FK/quantity/price sub-fields, and (2) how derived totals are calculated (including any discounts, taxes, or fees). Then assert: the child FK fields match chained IDs, quantities match sent values, and totals match the computation from the source code
150
118
  - **Webhook/event side effects**: If endpoints trigger async operations, test that side effects occur (e.g., POST /orders triggers notification)
151
119
  - **Cross-user isolation**: If resources are owned by users, test that user B cannot access/modify user A's resources (GET /users/{other_id}/data → 403 Forbidden)
152
120
  - **Range/boundary invariants**: If business rules cap values (max retries, min balance, discount ≤ subtotal), test the boundary (e.g., set retries to max+1 → expect rejection)
@@ -160,7 +128,7 @@ that step B depends on (e.g., create product → create order referencing that p
160
128
  verify order contains correct product). Single-resource CRUD alone is not an integration test.
161
129
  Use actual field names and values from the source code schema or OpenAPI schema (not \`{}\` or invented field names); verify response data, not just status codes.
162
130
  When a PUT/PATCH updates a resource with child collections (e.g., order items), the request body
163
- MUST include the child array with Foreign Key references chained from prior steps — and assertions MUST
131
+ MUST include the child array with FK references chained from prior steps — and assertions MUST
164
132
  verify the actual child items in the response (product_id, quantity, unit_price), not just
165
133
  top-level metadata like discount or status.
166
134
 
@@ -214,7 +182,7 @@ Before finalizing your output, verify:
214
182
  6. **Real request shapes**: requestBody for POST/PUT/PATCH uses actual field names from source (not \`{}\`). GET search/filter uses \`queryParams\`, not \`requestBody\`.
215
183
  7. **scenarioFile**: \`skyramp_integration_test_generation\` uses the exact \`filePath\` returned by \`skyramp_batch_scenario_test_generation\` — not a guessed or hardcoded filename.
216
184
  8. **bugCatchingTarget**: Every GENERATE integration test that targets a business rule, formula, or constraint has a non-empty \`bugCatchingTarget\`.
217
- 9. **Foreign Key chaining**: In multi-step integration tests, path params sourced from a prior step's response (e.g. \`order_id\` from step 1) use \`chainsFrom\` — not hardcoded IDs.
185
+ 9. **FK chaining**: In multi-step integration tests, path params sourced from a prior step's response (e.g. \`order_id\` from step 1) use \`chainsFrom\` — not hardcoded IDs.
218
186
  10. **Concrete scenario names**: No GENERATE item uses a placeholder name ending in a numeric suffix (e.g. \`ui-test-for-changed-component-1\`, \`ui-test-from-trace-2\`). Derive the name from the actual changed component or flow: if the diff touches \`LinkCard.tsx\`, the scenario name should be \`link-card-pin-toggle\` or \`link-card-edit-description\`, not \`ui-test-for-changed-component-1\`. The changed file list is available above — use it.
219
187
  </verification>`;
220
188
  }
@@ -225,7 +193,7 @@ export function buildFewShotExamples() {
225
193
  **Parameter grounding**:
226
194
  - baseURL: "http://localhost:8000" (workspace api.baseUrl)
227
195
  - steps[0].requestBody fields "name", "price": ProductCreate schema fields (src/models/product.py)
228
- - steps[1].requestBody "product_id": Foreign Key to products — chained from step 0 response id
196
+ - steps[1].requestBody "product_id": FK to products — chained from step 0 response id
229
197
  - steps[1].requestBody "quantity": OrderCreate schema field (src/models/order.py)
230
198
  - responseBody "total_amount": 89.97 = 29.99 × 3 — from order total formula (src/services/order_service.py: total = sum(item.price * item.quantity))
231
199
  - authHeader/authScheme: workspace config (Authorization / Bearer)
@@ -343,7 +311,7 @@ ${authGuidance}
343
311
  **For multi-endpoint workflows (integration tests) — Batch Scenario → Integration pipeline:**
344
312
  1. Call \`skyramp_batch_scenario_test_generation\` with ALL steps in a single call: \`scenarioName\`, \`destination\`,
345
313
  \`baseURL\`, \`${authCallParams}\`, and a \`steps\` array where each element has \`method\`, \`path\`, \`requestBody\` OR \`queryParams\`, \`responseBody\`, \`statusCode\`.
346
- \`statusCode\` is requireddetermine the expected status code from the source code for each step.
314
+ \`statusCode\` is optionaldefaults: POST→201, DELETE→204, GET/PUT/PATCH→200. Only override for non-standard codes.
347
315
  **OpenAPI spec is NOT required.** \`apiSchema\` is OPTIONAL — omit it if no spec exists.
348
316
  **CRITICAL — Query params vs request body:**
349
317
  - For **POST/PUT/PATCH**: use \`requestBody\` with realistic field values from source code schemas.
@@ -383,12 +351,12 @@ ${CONSUMER_MODE_ENABLED ? `**Contract test mode selection — set based on this
383
351
  Only provider-side contract tests are supported. Pass \`providerMode: true\` for new or modified endpoints this codebase owns.`}
384
352
 
385
353
  **For UI tests:**
386
- 1. \`browser_navigate\` to the target URL (from ${resolveServiceDetailsRef().baseUrlRef})
354
+ 1. \`browser_navigate\` to the target URL (from workspace \`api.baseUrl\`)
387
355
  2. \`browser_snapshot\` to see the page (ARIA tree)
388
356
  3. Interact using \`browser_click\`, \`browser_type\`, \`browser_fill_form\`, etc.
389
357
  4. \`browser_snapshot\` after each interaction that changes the page
390
358
  5. \`skyramp_export_zip\` with an **absolute** output path: \`<repositoryPath>/.skyramp/<test_name>_trace.zip\`
391
- 6. \`skyramp_ui_test_generation\` with \`playwrightInput\` = the **absolute** path of the exported zip, and \`outputDir\` = ${resolveServiceDetailsRef().frontendTestDirRef} (e.g. \`frontend/tests\`). Do NOT use the backend service's testDirectory — UI tests must go in the frontend service's test directory.
359
+ 6. \`skyramp_ui_test_generation\` with \`playwrightInput\` = the **absolute** path of the exported zip, and \`outputDir\` = the **frontend** service's \`testDirectory\` from workspace.yml (e.g. \`frontend/tests\`). Do NOT use the backend service's testDirectory — UI tests must go in the frontend service's test directory.
392
360
 
393
361
  Tips: For custom dropdowns (Radix, MUI): click combobox → snapshot → click option (NOT \`browser_select_option\`).
394
362
 
@@ -4,7 +4,6 @@ import { logger } from "../../utils/logger.js";
4
4
  import { buildRecommendationPrompt } from "./test-recommendation-prompt.js";
5
5
  import { ScenarioSource, AnalysisScope } from "../../types/RepositoryAnalysis.js";
6
6
  import { SCENARIO_CATEGORIES } from "../../types/TestRecommendation.js";
7
- import { inferExpectedStatus } from "../../utils/httpDefaults.js";
8
7
  export function mergeEnrichedScenarios(serverScenarios, raw) {
9
8
  const rejectionNotes = [];
10
9
  let parsed;
@@ -55,7 +54,11 @@ export function mergeEnrichedScenarios(serverScenarios, raw) {
55
54
  requestBody: st.requestBody,
56
55
  queryParams: st.queryParams,
57
56
  responseBody: st.responseBody,
58
- expectedStatusCode: st.expectedStatusCode ?? inferExpectedStatus(String(st.method ?? "GET")),
57
+ // Default status code by method if omitted to avoid `statusCode: undefined` in tool calls
58
+ expectedStatusCode: st.expectedStatusCode ??
59
+ (String(st.method ?? "").toUpperCase() === "POST" ? 201
60
+ : String(st.method ?? "").toUpperCase() === "DELETE" ? 204
61
+ : 200),
59
62
  expectedResponseFields: st.expectedResponseFields,
60
63
  bodyMustInclude: st.bodyMustInclude,
61
64
  chainsFrom: st.chainsFrom,
@@ -4,9 +4,8 @@ import { WorkspaceAuthType, getDefaultAuthHeader, AUTH_MIDDLEWARE_PATTERNS_STR }
4
4
  import { logger } from "../../utils/logger.js";
5
5
  import { extractResourceFromPath } from "../../utils/routeParsers.js";
6
6
  import { buildArchitectPreamble, buildContextFetchingGuidance, buildReasoningProtocol, buildToolWorkflows, buildTestPatternGuidelines, buildTestQualityCriteria, buildFewShotExamples, buildVerificationChecklist, buildGenerationRules, getAuthSnippets, MAX_TESTS_TO_GENERATE, MAX_RECOMMENDATIONS, MAX_CRITICAL_TESTS, } from "./recommendationSections.js";
7
- import { CATEGORY_PRIORITY, PRIORITY_TIER_ORDER, TEST_CATEGORIES } from "../../types/TestRecommendation.js";
7
+ import { CATEGORY_PRIORITY, TEST_CATEGORIES } from "../../types/TestRecommendation.js";
8
8
  import { buildScopeAssessmentSection, isFrontendFile } from "./scopeAssessment.js";
9
- import { resolveServiceDetailsRef } from "../../utils/featureFlags.js";
10
9
  function formatTestLocations(locs) {
11
10
  const entries = Object.entries(locs || {});
12
11
  if (entries.length === 0)
@@ -27,8 +26,8 @@ function formatTestLocations(locs) {
27
26
  // Categories map to HIGH / MEDIUM / LOW tiers.
28
27
  // Within a tier, novelty (new > modified > existing) breaks ties,
29
28
  // then cross-resource, step count, and finally the deterministic SHA-256 seed.
30
- // Single source of truth for priority ordering — imported from types.
31
- const PRIORITY_ORDER = PRIORITY_TIER_ORDER;
29
+ // CATEGORY_PRIORITY and PriorityTier imported from ../../types/TestRecommendation.js
30
+ const PRIORITY_ORDER = { CRITICAL: 4, HIGH: 3, MEDIUM: 2, LOW: 1 };
32
31
  const NOVELTY_ORDER = { new: 3, modified: 2, existing: 1 };
33
32
  function classifyNovelty(scenario, diffContext) {
34
33
  if (!diffContext)
@@ -134,6 +133,12 @@ function buildExternalCoverageSet(testLocations) {
134
133
  }
135
134
  // ── Execution Plan (replaces pre-ranked + scenarios + heuristic sections) ──
136
135
  function buildFullRepoRecommendations(scored, topN, baseUrl, authHeaderValue, authSchemeSnippet, authTypeValue, isFrontendProject = false, isFrontendOnlyProject = false, externalCoverage = new Set()) {
136
+ // Full-repo mode only — percentage-based UI/E2E slot targets (15% each, floor 1).
137
+ const rawE2E = isFrontendProject ? Math.max(1, Math.round(topN * 0.15)) : 0;
138
+ const rawUI = isFrontendProject ? Math.max(1, Math.round(topN * 0.15)) : 0;
139
+ const slotsFloor = Math.floor(topN / 2);
140
+ const minE2ESlots = Math.min(rawE2E, slotsFloor);
141
+ const minUISlots = Math.min(rawUI, Math.max(0, topN - minE2ESlots));
137
142
  const authRef = authHeaderValue
138
143
  ? `, authHeader: "${authHeaderValue}"${authSchemeSnippet}`
139
144
  : `, authHeader: <check OpenAPI securitySchemes or auth middleware; "" if confirmed unauthenticated>`;
@@ -162,9 +167,11 @@ function buildFullRepoRecommendations(scored, topN, baseUrl, authHeaderValue, au
162
167
  return true;
163
168
  })
164
169
  : scored;
165
- // All backend slots UI/E2E split is determined by the LLM's Budget Plan
166
- // (via buildScopeAssessmentSection), not by hardcoded percentage allocation.
167
- const allItems = scoredFiltered.slice(0, topN);
170
+ // For full-stack repos, carve out E2E and UI slots before filling with backend tests.
171
+ const backendSlotCount = isFrontendProject
172
+ ? Math.max(0, topN - minE2ESlots - minUISlots)
173
+ : topN;
174
+ const allItems = scoredFiltered.slice(0, backendSlotCount);
168
175
  const byType = new Map();
169
176
  for (const t of TYPE_ORDER)
170
177
  byType.set(t, []);
@@ -188,7 +195,7 @@ function buildFullRepoRecommendations(scored, topN, baseUrl, authHeaderValue, au
188
195
  return [
189
196
  `**${rank}. ${title}**`,
190
197
  ` ${s.description}`,
191
- ` ${step.method} ${step.path}${step.expectedStatusCode ? ` \u2192 ${step.expectedStatusCode}` : ""}`,
198
+ ` ${step.method} ${step.path} \u2192 ${step.expectedStatusCode}`,
192
199
  ` Tool: \`skyramp_contract_test_generation({ endpointURL: "${endpointURL}", method: "${step.method}"${authRef}${dataParam} })\``,
193
200
  ` From source: fill in requestData field names and the specific production boundary this validates`,
194
201
  ].join("\n");
@@ -197,7 +204,7 @@ function buildFullRepoRecommendations(scored, topN, baseUrl, authHeaderValue, au
197
204
  const stepLines = s.steps.map(st => {
198
205
  const isBody = ["POST", "PUT", "PATCH"].includes(st.method);
199
206
  const bodyHint = isBody ? ` \u2014 body: <${st.method} ${st.path} required fields from source>` : "";
200
- return ` ${st.order}. ${st.method} ${st.path}${st.expectedStatusCode ? ` \u2192 ${st.expectedStatusCode}` : ""}: ${st.description}${bodyHint}`;
207
+ return ` ${st.order}. ${st.method} ${st.path} \u2192 ${st.expectedStatusCode}: ${st.description}${bodyHint}`;
201
208
  }).join("\n");
202
209
  const isTraceBased = testType === "e2e" || testType === "ui";
203
210
  let toolCallsBlock;
@@ -243,7 +250,7 @@ function buildFullRepoRecommendations(scored, topN, baseUrl, authHeaderValue, au
243
250
  dataParam = `, requestBody: <${st.method} ${st.path} required fields from source code>`;
244
251
  }
245
252
  }
246
- return ` { method: "${st.method}", path: "${st.path}"${st.expectedStatusCode ? `, statusCode: ${st.expectedStatusCode}` : ""}${dataParam} }`;
253
+ return ` { method: "${st.method}", path: "${st.path}", statusCode: ${st.expectedStatusCode}${dataParam} }`;
247
254
  }).join(",\n");
248
255
  toolCallsBlock = [
249
256
  ` skyramp_batch_scenario_test_generation({ scenarioName: "${s.scenarioName}", destination: "${destinationHost}", baseURL: "${baseUrl}"${scenarioAuthRef}, steps: [\n${batchSteps}\n ] })`,
@@ -275,11 +282,55 @@ function buildFullRepoRecommendations(scored, topN, baseUrl, authHeaderValue, au
275
282
  const entries = items.map((item, i) => renderItem(item, globalRank + i + 1)).join("\n\n");
276
283
  return `### ${label} (${items.length})\n\n${entries}`;
277
284
  });
278
- const sections = backendSections.join("\n\n");
285
+ // Pre-allocate E2E and UI placeholder sections for full-stack repos.
286
+ const e2eSectionParts = [];
287
+ const uiSectionParts = [];
288
+ if (isFrontendProject) {
289
+ for (let i = 0; i < minE2ESlots; i++) {
290
+ const rank = i + 1;
291
+ e2eSectionParts.push(`**${rank}. E2E User Journey ${i + 1}**\n` +
292
+ ` End-to-end test covering a complete user journey through the frontend and backend.\n` +
293
+ ` To generate: record a browser trace, then call the generation tool.\n` +
294
+ ` browser_navigate({ url: "${baseUrl}" }) \u2192 exercise key user flow \u2192 skyramp_export_zip({ outputPath: "<repo>/.skyramp/e2e_journey_${i + 1}.zip" })\n` +
295
+ ` Tool: \`skyramp_e2e_test_generation({ playwrightInput: "<repo>/.skyramp/e2e_journey_${i + 1}.zip"${authHeaderOnlyRef} })\`\n` +
296
+ ` From source: read frontend components and their API calls to identify the highest-value user journey`);
297
+ }
298
+ for (let i = 0; i < minUISlots; i++) {
299
+ const rank = minE2ESlots + i + 1;
300
+ uiSectionParts.push(`**${rank}. UI Component Test ${i + 1}**\n` +
301
+ ` Test key UI component interactions and state changes.\n` +
302
+ ` To generate: record a browser trace, then call the generation tool.\n` +
303
+ ` browser_navigate({ url: "${baseUrl}" }) \u2192 interact with UI components \u2192 skyramp_export_zip({ outputPath: "<repo>/.skyramp/ui_component_${i + 1}.zip" })\n` +
304
+ ` Tool: \`skyramp_ui_test_generation({ playwrightInput: "<repo>/.skyramp/ui_component_${i + 1}.zip"${authHeaderOnlyRef} })\`\n` +
305
+ ` From source: read frontend component files to identify interactions, form submissions, and state transitions`);
306
+ }
307
+ // Offset backend section ranks by the number of E2E + UI placeholders
308
+ const offset = minE2ESlots + minUISlots;
309
+ backendSections.forEach((_, idx) => {
310
+ const t = TYPE_ORDER.filter(t => (byType.get(t) ?? []).length > 0)[idx];
311
+ if (!t)
312
+ return;
313
+ const items = byType.get(t);
314
+ const label = TYPE_LABEL[t];
315
+ let globalRank = offset;
316
+ for (const prev of TYPE_ORDER) {
317
+ if (prev === t)
318
+ break;
319
+ globalRank += (byType.get(prev) ?? []).length;
320
+ }
321
+ backendSections[idx] = `### ${label} (${items.length})\n\n${items.map((item, i) => renderItem(item, globalRank + i + 1)).join("\n\n")}`;
322
+ });
323
+ }
324
+ const allSections = [
325
+ ...(e2eSectionParts.length > 0 ? [`### E2E (${e2eSectionParts.length})\n\n${e2eSectionParts.join("\n\n")}`] : []),
326
+ ...(uiSectionParts.length > 0 ? [`### UI (${uiSectionParts.length})\n\n${uiSectionParts.join("\n\n")}`] : []),
327
+ ...backendSections,
328
+ ];
329
+ const sections = allSections.join("\n\n");
279
330
  const frontendTierNote = isFrontendOnlyProject
280
- ? `\n\n**Frontend repo:** add E2E and UI tests only no integration or contract tests. The number of each is determined by the UI/E2E percentage you committed to in your Budget Plan above.`
331
+ ? `\n\n**Frontend repo:** supplement MUST include at least ${minE2ESlots} E2E test${minE2ESlots > 1 ? "s" : ""} (\`skyramp_e2e_test_generation\`) and at least ${minUISlots} UI test${minUISlots > 1 ? "s" : ""} (\`skyramp_ui_test_generation\`). Do NOT add integration or contract tests.`
281
332
  : isFrontendProject
282
- ? `\n\n**Full-stack repo:** add E2E and UI tests alongside backend tests. Fill your Budget Plan's UI/E2E percentage first, then use remaining slots for backend tests (Tiers 1-4 above).`
333
+ ? `\n\n**Full-stack repo:** supplement MUST include at least ${minE2ESlots} E2E test${minE2ESlots > 1 ? "s" : ""} (\`skyramp_e2e_test_generation\`) and at least ${minUISlots} UI test${minUISlots > 1 ? "s" : ""} (\`skyramp_ui_test_generation\`). Add these before exhausting backend tiers.`
283
334
  : "";
284
335
  const repoSupplementNote = supplementCount > 0
285
336
  ? `
@@ -306,9 +357,9 @@ function buildFullRepoRecommendations(scored, topN, baseUrl, authHeaderValue, au
306
357
  </supplement_guidance>`
307
358
  : "";
308
359
  const typeMixText = isFrontendOnlyProject
309
- ? `This is a frontend repo. Focus on E2E and UI tests only. Do NOT add integration or contract tests. Split between E2E and UI based on the percentage in your Budget Plan above.`
360
+ ? `This is a frontend repo. Focus on E2E and UI tests only. Include at least ${minE2ESlots} E2E test${minE2ESlots > 1 ? "s" : ""} (\`skyramp_e2e_test_generation\`) and at least ${minUISlots} UI test${minUISlots > 1 ? "s" : ""} (\`skyramp_ui_test_generation\`). Do NOT add integration or contract tests.`
310
361
  : isFrontendProject
311
- ? `This is a full-stack repo. Coverage ranking: E2E > UI > Integration > Contract. Use \`skyramp_e2e_test_generation\` for E2E and \`skyramp_ui_test_generation\` for UI tests. Split between frontend and backend tests based on the percentage in your Budget Plan above.`
362
+ ? `This is a full-stack repo. Coverage ranking: E2E > UI > Integration > Contract. Include at least ${minE2ESlots} E2E test${minE2ESlots > 1 ? "s" : ""} (\`skyramp_e2e_test_generation\`) and at least ${minUISlots} UI test${minUISlots > 1 ? "s" : ""} (\`skyramp_ui_test_generation\`), in addition to backend integration and contract tests.`
312
363
  : `Focus on integration and contract tests for all API endpoints.`;
313
364
  return `## Test Recommendations — ${topN} total (grouped by test type)
314
365
 
@@ -337,7 +388,7 @@ Before filling in tool call parameters for each item, use the analysis data alre
337
388
  - Computed/derived response fields and their formulas — assert exact values; read source for formula details not captured in the analysis
338
389
  - Auth middleware — set authHeader/authScheme from the repository context above; FastAPI HTTPBearer → 403 not 401
339
390
  - Storage backend — if Redis or schema-less, discard unique-constraint and cascade-delete scenarios
340
- - Delete behavior — read the route handler to determine actual response code (hard-delete may use 204, soft-delete/cancel may use 200)
391
+ - Delete behavior — hard-delete 204; soft-delete/cancel 200
341
392
 
342
393
  ${buildTestQualityCriteria()}
343
394
 
@@ -351,6 +402,16 @@ ${buildTestQualityCriteria()}
351
402
  </enrichment_notes>`;
352
403
  }
353
404
  function buildExecutionPlan(scored, maxGen, topN, baseUrl, authHeaderValue, authSchemeSnippet, authTypeValue, seed, endpointCount, isUIOnlyPR, hasFrontendChanges = false, hasTraces = false, externalCoverage = new Set(), relevantExternalTestPaths = []) {
405
+ const frontendUrl = "<frontend_url>";
406
+ // Slot allocation:
407
+ // - UI-only PR: all GENERATE slots are UI placeholders (no pre-ranked backend scenarios)
408
+ // - Mixed PR: last GENERATE slot is a UI placeholder; remaining slots are backend
409
+ // - Backend-only PR: all GENERATE slots are backend scenarios
410
+ const backendGenerateCount = isUIOnlyPR
411
+ ? 0
412
+ : hasFrontendChanges
413
+ ? Math.max(0, maxGen - 1)
414
+ : maxGen;
354
415
  // Filter out scenarios whose primary method + resource + test type is already covered by external tests.
355
416
  // Method-aware: an external test covering GET /orders won't block PUT /orders scenarios.
356
417
  // This is the programmatic complement to the prompt-level Step 0 dedup instructions.
@@ -364,10 +425,8 @@ function buildExecutionPlan(scored, maxGen, topN, baseUrl, authHeaderValue, auth
364
425
  return true;
365
426
  })
366
427
  : scored;
367
- // All pre-ranked backend scenarios go into GENERATE slots (up to maxGen).
368
- // UI/E2E split is determined by the LLM's Budget Plan — not hardcoded here.
369
- const generateItems = scoredAfterExternalDedup.slice(0, Math.min(maxGen, scoredAfterExternalDedup.length));
370
- const rawAdditionalItems = scoredAfterExternalDedup.slice(maxGen, topN);
428
+ const generateItems = scoredAfterExternalDedup.slice(0, Math.min(backendGenerateCount, scoredAfterExternalDedup.length));
429
+ const rawAdditionalItems = scoredAfterExternalDedup.slice(backendGenerateCount, topN);
371
430
  // Filter additional items whose primary resource + test type already appear in GENERATE
372
431
  const generatedCoverage = new Set(generateItems.map(item => scenarioCoverageKey(item.scenario)));
373
432
  const additionalItems = rawAdditionalItems.filter(item => !generatedCoverage.has(scenarioCoverageKey(item.scenario)));
@@ -380,10 +439,47 @@ function buildExecutionPlan(scored, maxGen, topN, baseUrl, authHeaderValue, auth
380
439
  : authHeaderValue
381
440
  ? `, authHeader: "${authHeaderValue}"`
382
441
  : `, authHeader: <check OpenAPI securitySchemes or auth middleware; "" if confirmed unauthenticated>`;
383
- // UI-only PR: provide guidance template for the LLM to derive UI tests from changed files.
384
- // The LLM's Budget Plan (100% UI for UI-only PRs) determines how many to generate.
385
- const uiOnlyGenerateGuidance = isUIOnlyPR
386
- ? `**UI-only PR derive ${maxGen} UI tests from changed frontend files.**\nEach test must target a distinct changed component or user flow. Use \`skyramp_ui_test_generation\` to generate each test.`
442
+ // UI-only: all GENERATE slots are UI test placeholders (one per changed component/flow)
443
+ const uiGenerateBlocks = isUIOnlyPR
444
+ ? Array.from({ length: maxGen }, (_, i) => {
445
+ const rank = i + 1;
446
+ const zipPath = `<repositoryPath>/.skyramp/ui_test_${rank}_trace.zip`;
447
+ return hasTraces
448
+ ? (`**#${rank} — GENERATE** | ui | workflow | new\n` +
449
+ `Scenario: ui-test-from-trace-${rank} (rename from the actual changed component/flow)\n` +
450
+ `Validates: UI interactions for a changed frontend component or flow.\n\n` +
451
+ `**Tool**: \`skyramp_ui_test_generation({ playwrightInput: "<discovered_trace_file_path>", outputDir: "<frontend service testDirectory from workspace.yml e.g. frontend/tests>" })\``)
452
+ : (`**#${rank} — GENERATE** | ui | workflow | new\n` +
453
+ `Scenario: ui-test-for-changed-component-${rank} (rename from the actual changed component/flow)\n` +
454
+ `Validates: UI interactions for changed frontend component/flow ${rank}.\n\n` +
455
+ `**Tool workflow:**\n` +
456
+ ` 1. \`browser_navigate({ url: "${frontendUrl}" })\`\n` +
457
+ ` 2. Interact with the changed component (read the diff to identify which component changed and what interactions it supports)\n` +
458
+ ` 3. \`browser_snapshot()\` after each key interaction\n` +
459
+ ` 4. \`skyramp_export_zip({ outputPath: "${zipPath}" })\` — absolute path\n` +
460
+ ` 5. \`skyramp_ui_test_generation({ playwrightInput: "${zipPath}", outputDir: "<frontend service testDirectory from workspace.yml e.g. frontend/tests>" })\`\n\n` +
461
+ `Each item must target a distinct changed component or user flow.`);
462
+ }).join("\n\n")
463
+ : "";
464
+ // Mixed PR: reserve the last GENERATE slot for a UI test for the changed frontend components.
465
+ // Guard: skip when maxGen=0 (caller explicitly requested no generation)
466
+ const uiRank = generateItems.length + 1;
467
+ const uiPlaceholderBlock = (hasFrontendChanges && !isUIOnlyPR && maxGen > 0)
468
+ ? hasTraces
469
+ ? (`**#${uiRank} — GENERATE** | ui | workflow | new\n` +
470
+ `Scenario: ui-test-for-changed-components (rename from the actual changed component/flow)\n` +
471
+ `Validates: UI interactions for the changed frontend components in this PR.\n\n` +
472
+ `**Tool**: \`skyramp_ui_test_generation({ playwrightInput: "<discovered_trace_file_path>", outputDir: "<frontend service testDirectory from workspace.yml e.g. frontend/tests>" })\``)
473
+ : (`**#${uiRank} — GENERATE** | ui | workflow | new\n` +
474
+ `Scenario: ui-test-for-changed-components (rename from the actual changed component/flow)\n` +
475
+ `Validates: UI interactions for the changed frontend components in this PR.\n\n` +
476
+ `**Tool workflow:**\n` +
477
+ ` 1. \`browser_navigate({ url: "${frontendUrl}" })\`\n` +
478
+ ` 2. Interact with the changed component (read the diff to identify which component changed and what interactions it supports)\n` +
479
+ ` 3. \`browser_snapshot()\` after each key interaction\n` +
480
+ ` 4. \`skyramp_export_zip({ outputPath: "<repositoryPath>/.skyramp/ui_mixed_pr_trace.zip" })\` — absolute path\n` +
481
+ ` 5. \`skyramp_ui_test_generation({ playwrightInput: "<repositoryPath>/.skyramp/ui_mixed_pr_trace.zip", outputDir: "<frontend service testDirectory from workspace.yml e.g. frontend/tests>" })\`\n\n` +
482
+ `Derive scenario name and steps from the actual changed frontend files.`)
387
483
  : "";
388
484
  const generateBlocks = generateItems.map((item, i) => {
389
485
  const rank = i + 1;
@@ -400,7 +496,7 @@ function buildExecutionPlan(scored, maxGen, topN, baseUrl, authHeaderValue, auth
400
496
  ? `\n authHeader: "${authHeaderValue}"${authSchemeSnippet}`
401
497
  : `\n authHeader: <resolve from workspace or OpenAPI securitySchemes>; authScheme: <if Authorization>`;
402
498
  return (`**#${rank} — GENERATE** | ${testType} | ${s.category} | ${item.novelty}\n` +
403
- `${step.method} ${step.path}${step.expectedStatusCode ? ` → ${step.expectedStatusCode}` : ""}\n` +
499
+ `${step.method} ${step.path} → ${step.expectedStatusCode}\n` +
404
500
  `Validates: ${s.description}\n\n` +
405
501
  `**Context for generation**:\n` +
406
502
  ` Endpoint URL: ${endpointURL}${requestBodyData}${authContext}\n\n` +
@@ -423,7 +519,7 @@ function buildExecutionPlan(scored, maxGen, topN, baseUrl, authHeaderValue, auth
423
519
  const bodyData = st.requestBody && Object.keys(st.requestBody).length > 0
424
520
  ? ` [use requestBody: ${JSON.stringify(st.requestBody)} — pass as JSON string in tool call]`
425
521
  : "";
426
- return ` ${st.order}. ${st.method} ${st.path}${st.expectedStatusCode ? ` → ${st.expectedStatusCode}` : ""}: ${st.description}${chains}${bodyHint}${bodyData}${responseHint}`;
522
+ return ` ${st.order}. ${st.method} ${st.path} → ${st.expectedStatusCode}: ${st.description}${chains}${bodyHint}${bodyData}${responseHint}`;
427
523
  }).join("\n");
428
524
  let destinationHost = "localhost";
429
525
  try {
@@ -434,7 +530,9 @@ function buildExecutionPlan(scored, maxGen, topN, baseUrl, authHeaderValue, auth
434
530
  const authContext = authHeaderValue
435
531
  ? `authHeader: "${authHeaderValue}"${authSchemeSnippet}`
436
532
  : "authHeader: <resolve from workspace or OpenAPI securitySchemes>; authScheme: <if Authorization>";
437
- const prereqNote = `\n**Prerequisite discovery**: Check for Foreign Key fields (product_id, user_id, order_id) in the endpoint's request body. If found, prepend a step to create that prerequisite resource first, then chain its primary key field into the dependent step using template variable syntax. Check the actual field name from the response body (\`id\`, \`uuid\`, \`_id\`, etc.), response header (\`Location\`), or cookie — do not assume \`id\`.`;
533
+ const prereqNote = s.category === "new_endpoint"
534
+ ? `\n**Prerequisite discovery**: Check for FK fields (product_id, user_id, order_id) in the endpoint's request body. If found, prepend a step to create that prerequisite resource first, then chain its primary key field into the dependent step using template variable syntax. Check the actual field name from the response body (\`id\`, \`uuid\`, \`_id\`, etc.), response header (\`Location\`), or cookie — do not assume \`id\`.`
535
+ : "";
438
536
  const bugLine = s.bugCatchingTarget
439
537
  ? `**Bug to catch**: ${s.bugCatchingTarget}\n`
440
538
  : "";
@@ -463,16 +561,17 @@ function buildExecutionPlan(scored, maxGen, topN, baseUrl, authHeaderValue, auth
463
561
  const s = item.scenario;
464
562
  const testType = s.testType ?? (s.steps.length === 1 ? "contract" : "integration");
465
563
  const target = s.steps.length === 1
466
- ? `${s.steps[0].method} ${s.steps[0].path}${s.steps[0].expectedStatusCode ? ` → ${s.steps[0].expectedStatusCode}` : ""}`
564
+ ? `${s.steps[0].method} ${s.steps[0].path} → ${s.steps[0].expectedStatusCode}`
467
565
  : `Scenario: ${s.scenarioName} (${s.steps.map(st => `${st.method} ${st.path}`).join(" → ")})`;
468
566
  return `#${rank} [ADDITIONAL] | ${testType} | ${s.category} | ${item.novelty}\n ${target}\n Validates: ${s.description}`;
469
567
  }).join("\n\n");
470
- // UI/E2E guidance — the LLM adds UI/E2E items as its Budget Plan dictates.
471
- // Only rendered for non-UI-only PRs (UI-only PRs have dedicated guidance above).
472
- const uiGuidance = (!isUIOnlyPR && hasFrontendChanges) ? `
473
- **UI/E2E tests (add per your Budget Plan):** If your Budget Plan allocates UI/E2E slots, add them here. Tool workflow for each new item:
568
+ // UI/E2E guidance — the LLM adds as many as its Budget Plan calls for.
569
+ // Note: if a UI test already occupies a GENERATE slot (uiPlaceholderBlock), that slot
570
+ // satisfies the UI generate count do not add it again in ADDITIONAL.
571
+ const uiGuidance = !isUIOnlyPR ? `
572
+ **UI/E2E tests (add per your Budget Plan):** If your Budget Plan requires UI/E2E items beyond what is already in your GENERATE list, append an [ADDITIONAL] entry for each. If a UI test already occupies a GENERATE slot above, that slot satisfies your UI/E2E generate count — do NOT add it again to ADDITIONAL. Tool workflow for each new item:
474
573
  - **E2E**: ${hasTraces ? "Use discovered trace/recording files with `skyramp_e2e_test_generation`." : "Add to additionalRecommendations with a note that both a backend API trace (`skyramp_start_trace_collection` / `skyramp_stop_trace_collection`) and a browser Playwright recording must be collected in a live environment first. Do NOT attempt `skyramp_e2e_test_generation` without both traces present."}
475
- - **UI**: ${hasTraces ? "Use an existing Playwright `.zip` trace with `skyramp_ui_test_generation`." : `Record a trace using \`browser_navigate\` + \`browser_snapshot\` + \`skyramp_export_zip\`, then call \`skyramp_ui_test_generation({ playwrightInput: "<zip_path>", outputDir: "<frontend_test_directory>" })\`. Resolve \`<frontend_test_directory>\` from ${resolveServiceDetailsRef().frontendTestDirRef}.`}
574
+ - **UI**: ${hasTraces ? "Use an existing Playwright `.zip` trace with `skyramp_ui_test_generation`." : "Record a trace using `browser_navigate` + `browser_snapshot` + `skyramp_export_zip`, then call `skyramp_ui_test_generation({ playwrightInput: \"<zip_path>\", outputDir: \"<frontend testDirectory from workspace.yml>\" })`."}
476
575
  Derive scenario names and steps from the actual changed frontend files. If your Budget Plan calls for 0% UI/E2E, omit this entirely.` : "";
477
576
  const supplementNote = `\n**If your Budget Plan total exceeds the pre-ranked items listed above:** draft additional tests from source-code enrichment (Step 1). For each new or changed endpoint, identify boundary or variation scenarios — formula parameters, search/filter constraints, required field validation. Only after exhausting PR-specific scenarios, add generic patterns (auth boundary → 401, non-existent ID → 404). Do NOT supplement with tests whose endpoint + test type match a GENERATE item.`;
478
577
  // ── PR / branch-diff mode: execution plan ────────────────────────────────
@@ -499,7 +598,7 @@ ${externalTestFilesList}For every GENERATE item below, check its endpoint path a
499
598
 
500
599
  **Step 1 — Source-Code Enrichment (before executing anything)**
501
600
  Read the source code for ALL changed files. Before generating each recommendation, quote the relevant source code in a <source_evidence> block — include the route handler signature, request body schema fields, response shape, and any computed field formulas. Use these quotes to derive tool call parameters. Look for:
502
- - **Auth middleware** — check for known signals (${AUTH_MIDDLEWARE_PATTERNS_STR}). If any match, override \`authHeader\` and \`authScheme\` even if ${resolveServiceDetailsRef().authSourceRef} says authType: none. **If no known signal matches but the diff shows security-adjacent code** (decorators like \`@requiresRole\`/\`@Protected\`, function names like \`validateToken\`/\`checkPermission\`/\`verifyHMAC\`, or imports from auth/security packages), read the relevant source file to determine the actual auth scheme before proceeding. Auth handling for \`skyramp_integration_test_generation\` with \`scenarioFile\` is covered in the Tool Workflows section below.
601
+ - **Auth middleware** — check for known signals (${AUTH_MIDDLEWARE_PATTERNS_STR}). If any match, override \`authHeader\` and \`authScheme\` even if workspace.yml says authType: none. **If no known signal matches but the diff shows security-adjacent code** (decorators like \`@requiresRole\`/\`@Protected\`, function names like \`validateToken\`/\`checkPermission\`/\`verifyHMAC\`, or imports from auth/security packages), read the relevant source file to determine the actual auth scheme before proceeding. Auth handling for \`skyramp_integration_test_generation\` with \`scenarioFile\` is covered in the Tool Workflows section below.
503
602
  - Business rules and formulas (e.g. total_cost = compute * rate + memory * rate)
504
603
  - State transitions and domain constraints (e.g. budget cannot drop below current spend)
505
604
  - Validation logic (field constraints, cross-field dependencies)
@@ -534,7 +633,7 @@ If these conditions are not met, add it to ADDITIONAL only — do NOT displace a
534
633
  When a qualifying candidate is inserted: place it HIGH before MEDIUM before LOW; within the same priority, source-code-derived candidates go BEFORE structural ones. Re-number ranks after insertion. The top ${maxGen} ranked items become GENERATE candidates.
535
634
 
536
635
  **Source-code validation gates (apply during Step 1):**
537
- - **Cascade vs referential integrity**: If both a cascade-delete and a delete-blocked scenario appear for the same resource pair, keep only the one matching the source Foreign Key delete policy (ON DELETE CASCADE / cascade=True / onDelete: 'CASCADE' → keep cascade-delete; RESTRICT/PROTECT/no annotation → keep delete-blocked). Remove the inapplicable variant.
636
+ - **Cascade vs referential integrity**: If both a cascade-delete and a delete-blocked scenario appear for the same resource pair, keep only the one matching the source FK delete policy (ON DELETE CASCADE / cascade=True / onDelete: 'CASCADE' → keep cascade-delete; RESTRICT/PROTECT/no annotation → keep delete-blocked). Remove the inapplicable variant.
538
637
  - **Unique constraints**: Unique-constraint scenarios (duplicate POST → 409) are pre-drafted for all resources. Confirm enforcement before keeping: SQL UNIQUE index, Mongoose unique: true, Prisma @unique, or explicit duplicate-check code. If the backend is Redis, schema-less, or has no explicit constraint in the changed files, move to ADDITIONAL with a note — do NOT generate.
539
638
 
540
639
  **Step 2 — Diversity check (using enriched knowledge from Step 1)**
@@ -550,7 +649,7 @@ For each pair of GENERATE items, ask: same HTTP method + path + step sequence +
550
649
  Same step sequence with only payload differences (e.g. 10% vs 5% discount both returning 200) = same code path = duplicate. Different scenario names do not make duplicate tests distinct.
551
650
 
552
651
  **Step 3 — Execute merged plan in rank order**
553
- Replace any scenario that pairs unrelated resources with one reflecting actual Foreign Key relationships in the codebase.
652
+ Replace any scenario that pairs unrelated resources with one reflecting actual FK relationships in the codebase.
554
653
  Use the field names and values from the \`<source_evidence>\` blocks you quoted in Step 1 to fill all tool call parameters. Prefer reusing Step 1 evidence when it already resolves a placeholder, but if a placeholder cannot be replaced with concrete values from files already read, you may read the specific schema, model, or handler file needed to resolve it. Assert response field values, not just status codes.
555
654
 
556
655
  ${buildTestQualityCriteria()}
@@ -566,10 +665,10 @@ ${buildGenerationRules(isUIOnlyPR)}
566
665
  ### GENERATE (process these EXACTLY as listed, in order — after completing Steps 0–2 above; if Step 0 converts an item to UPDATE, backfill the ADD slot from ADDITIONAL following the priority order in Step 0)
567
666
 
568
667
  ${isUIOnlyPR
569
- ? (uiOnlyGenerateGuidance || " (no UI generate items — derive scenarios from changed frontend files)")
570
- : (generateBlocks || " (no pre-ranked generate items — draft your own based on endpoint analysis)")}
668
+ ? (uiGenerateBlocks || " (no UI generate items — derive scenarios from changed frontend files)")
669
+ : ([generateBlocks, uiPlaceholderBlock].filter(Boolean).join("\n\n") || " (no pre-ranked generate items — draft your own based on endpoint analysis)")}
571
670
 
572
- **VERIFICATION CHECK**: Before proceeding, verify your generate list covers the same endpoints and test types as the items above. Add genuinely new scenarios to ADDITIONAL instead. One retry on failure then skip to next item.
671
+ **COMPLIANCE CHECK**: Before proceeding, verify your generate list matches the items above. If you plan to generate a scenario with a different name than what is listed (e.g. you want to generate "order-update-discount-calculation" but the plan says "orders-patch-add-items-recalculate"), STOP — use the plan's scenario name and steps. Add your alternative to ADDITIONAL instead. One retry on failure then skip to next item.
573
672
 
574
673
  ### ADDITIONAL (list in additionalRecommendations in this order after Step 1 insertion)
575
674
 
@@ -604,17 +703,10 @@ export function buildRecommendationPrompt(analysis, analysisScope = AnalysisScop
604
703
  const hasFrontendChanges = isDiffScope && diffContext
605
704
  ? filteredChangedFiles.some(f => isFrontendFile(f))
606
705
  : false;
607
- // Backend changes detected if:
608
- // 1. Endpoints directly matched from changed files (new/modified/removed), OR
609
- // 2. Changed files are in backend service/model/middleware directories (affectedServices non-empty)
610
- // but couldn't be mapped to specific endpoints (service-layer changes like services/items.ts)
611
706
  const hasApiChanges = isDiffScope && diffContext
612
707
  ? (diffContext.newEndpoints.length > 0 || diffContext.modifiedEndpoints.length > 0 || (diffContext.removedEndpoints?.length ?? 0) > 0)
613
708
  : false;
614
- const hasBackendServiceChanges = isDiffScope && diffContext
615
- ? (diffContext.affectedServices.length > 0 && filteredChangedFiles.some(f => !isFrontendFile(f) && /\.(ts|js|py|java|go|rb|rs|cs)$/.test(f)))
616
- : false;
617
- const isUIOnlyPR = hasFrontendChanges && !hasApiChanges && !hasBackendServiceChanges;
709
+ const isUIOnlyPR = hasFrontendChanges && !hasApiChanges;
618
710
  const hasTraces = (analysis.artifacts?.traceFiles?.length ?? 0) > 0 ||
619
711
  (analysis.artifacts?.playwrightRecordings?.length ?? 0) > 0;
620
712
  // ── Mode preamble ──
@@ -814,11 +906,17 @@ ${detailBlocks}
814
906
  const na = NOVELTY_ORDER[a.novelty], nb = NOVELTY_ORDER[b.novelty];
815
907
  if (nb !== na)
816
908
  return nb - na;
817
- // Deterministic tiebreaker: when priority and novelty are equal, sort by
818
- // scenario name then seeded hash. This ensures identical inputs always produce
819
- // the same ordering regardless of runtime locale or JS sort stability — important
820
- // because the LLM receives a ranked list and would otherwise produce inconsistent
821
- // recommendations across runs for equally-ranked scenarios.
909
+ const crossA = a.scenario.steps.length > 2 ? 1 : 0;
910
+ const crossB = b.scenario.steps.length > 2 ? 1 : 0;
911
+ if (crossB !== crossA)
912
+ return crossB - crossA;
913
+ if (b.scenario.steps.length !== a.scenario.steps.length)
914
+ return b.scenario.steps.length - a.scenario.steps.length;
915
+ const errorA = a.scenario.steps.some(s => s.interactionType === "error" || s.interactionType === "edge-case") ? 1 : 0;
916
+ const errorB = b.scenario.steps.some(s => s.interactionType === "error" || s.interactionType === "edge-case") ? 1 : 0;
917
+ if (errorB !== errorA)
918
+ return errorB - errorA;
919
+ // Use locale-independent comparison to avoid runtime-locale non-determinism
822
920
  const nameA = a.scenario.scenarioName;
823
921
  const nameB = b.scenario.scenarioName;
824
922
  if (nameA < nameB)
@@ -853,23 +951,12 @@ ${detailBlocks}
853
951
  mainSection = buildExecutionPlan(scored, maxGen, topN, analysis.apiEndpoints.baseUrl, authHeaderValue, authSchemeSnippet, authTypeValue, seed, endpointCount, isUIOnlyPR, hasFrontendChanges, hasTraces, externalCoverage, analysis.existingTests.relevantExternalTestPaths ?? []);
854
952
  }
855
953
  else {
856
- // Endpoint discovery hint: when backend service files changed but no endpoints were
857
- // directly matched, guide the LLM to trace from service files → controllers → routes.
858
- const endpointDiscoveryHint = hasBackendServiceChanges && diffContext
859
- ? `
860
- **Endpoint Discovery Required** — the diff modifies backend service files (affected services: ${diffContext.affectedServices.join(", ")}) that don't directly define routes. You MUST:
861
- 1. Read the Routing entry-point files listed above
862
- 2. Trace which controllers/routers import the affected services
863
- 3. Identify the specific HTTP endpoints those controllers register
864
- 4. Use discovered endpoints as your GENERATE targets (contract + integration tests)
865
- Do NOT default to UI-only tests — this PR has backend logic changes that require API-level testing.`
866
- : "";
867
954
  mainSection = `
868
955
  ## Draft Your Execution Plan
869
956
 
870
- No pre-drafted scenarios available.${endpointDiscoveryHint}
957
+ No pre-drafted scenarios available.
871
958
 
872
- ${buildScopeAssessmentSection(topN, maxGen, isUIOnlyPR)}
959
+ ${buildScopeAssessmentSection(topN, maxGen)}
873
960
 
874
961
  Draft tests from the endpoint interactions and source code above, following the same tool pipeline described in Tool Workflows below. Prioritize critical categories: security_boundary > data_integrity > business_rule > workflow > crud.
875
962
 
@@ -877,8 +964,6 @@ For each test: pick the highest-impact endpoint(s), draft a realistic scenario w
877
964
 
878
965
  **Honor your Budget Plan: produce exactly the total you committed to (GENERATE + ADDITIONAL). No fewer, no padding with low-value tests.**
879
966
 
880
- **Coverage breadth enforcement:** Your GENERATE items must span DIFFERENT HTTP methods or endpoints from your Coverage Reasoning surfaces. If you identified 5+ testable surfaces but all GENERATE items target the same method + path (e.g. all POST /permissions), you are violating diversity. Spread GENERATE slots across distinct surfaces; put remaining surfaces in ADDITIONAL recommendations.
881
-
882
967
  ## Recommendation Stability
883
968
  - **Carry forward** previous additionalRecommendations that still apply — match by scenarioName (multi-step) or endpoint (single-endpoint). Re-derive category and priority from test content.
884
969
  - **Only drop** a previous recommendation if its target endpoint was removed, its business logic changed, or it is now covered by a generated test.