@qulib/mcp 0.8.2 → 0.10.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +41 -14
- package/dist/index.js +176 -24
- package/package.json +13 -7
package/README.md
CHANGED
|
@@ -43,22 +43,51 @@ For verbose server-side stderr logs while troubleshooting host wiring, add:
|
|
|
43
43
|
}
|
|
44
44
|
```
|
|
45
45
|
|
|
46
|
-
##
|
|
46
|
+
## MCP tools
|
|
47
|
+
|
|
48
|
+
| Tool | Purpose |
|
|
49
|
+
|---|---|
|
|
50
|
+
| **`qulib_score_confidence`** | **Flagship.** Fuses evidence from `qulib_analyze_app`, `qulib_score_automation`, and `qulib_score_api` into one verdict: **ship / caution / hold / block** with a 0–100 confidence score, L1–L5 level, per-source contributions, honesty notes, and recommended next checks. Pass `url` and/or `repoPath`. |
|
|
51
|
+
| `qulib_analyze_app` | Live-app quality scan: release confidence (0–100), axe-core a11y, broken links, console errors, prioritized gaps. Default payload is summary-first; pass `includeFullReport: true` for all scenarios. Optional form-login / storage-state auth. *(Canonical form; legacy alias `analyze_app` kept for backwards compatibility.)* |
|
|
52
|
+
| `qulib_score_automation` | Score a local repo's test-automation maturity across six dimensions (test coverage breadth, framework adoption, test-id hygiene, CI integration, auth test coverage, component test ratio) — plus a conditional 7th dimension (API coverage) when API endpoints are detected. Returns overall 0–100, level (L1–L5), and top recommendations. Each dimension carries `applicability`; score normalizes over applicable dimensions only. |
|
|
53
|
+
| `qulib_score_api` | Discover API endpoints in a repo and score their test coverage. Tier1=OpenAPI specs, Tier2=framework routes (Next.js, Express, Fastify, NestJS), Tier3=heuristic opt-in (tRPC). Returns an api-test-coverage dimension score with per-endpoint evidence. |
|
|
54
|
+
| `qulib_scaffold_tests` | Generate a ready-to-run test scaffold (Cypress config + spec files) by crawling a deployed URL. Returns `generatedTests` and `projectConfig` so an agent can write files directly. Pass `recipes` (e.g. `["auth","a11y"]`) to append proven test patterns. Supported framework: `cypress-e2e` (default); `playwright` is not yet implemented. |
|
|
55
|
+
| `qulib_explore_auth` | List all sign-in paths (OAuth, SSO, forms, magic link) and what the agent must collect before `qulib_analyze_app`. Prefer on unfamiliar apps. *(Canonical form; legacy alias `explore_auth` kept for backwards compatibility.)* |
|
|
56
|
+
| `qulib_detect_auth` | Single-pass auth pattern guess with a recommendation. Lighter than `qulib_explore_auth`. *(Canonical form; legacy alias `detect_auth` kept for backwards compatibility.)* |
|
|
57
|
+
| `analyze_app` | Legacy alias for `qulib_analyze_app`. Identical behavior; kept for backwards compatibility through v1.0. |
|
|
58
|
+
| `explore_auth` | Legacy alias for `qulib_explore_auth`. Identical behavior; kept for backwards compatibility through v1.0. |
|
|
59
|
+
| `detect_auth` | Legacy alias for `qulib_detect_auth`. Identical behavior; kept for backwards compatibility through v1.0. |
|
|
60
|
+
|
|
61
|
+
**Example — flagship confidence call:**
|
|
47
62
|
|
|
48
|
-
|
|
63
|
+
```
|
|
64
|
+
qulib_score_confidence({ url: "https://example.com", repoPath: "/path/to/repo" })
|
|
65
|
+
```
|
|
66
|
+
|
|
67
|
+
Returns a verdict like:
|
|
68
|
+
|
|
69
|
+
```json
|
|
70
|
+
{
|
|
71
|
+
"releaseConfidence": {
|
|
72
|
+
"verdict": "caution",
|
|
73
|
+
"confidenceScore": 54,
|
|
74
|
+
"level": 3,
|
|
75
|
+
"label": "Moderate confidence — proceed with known risks",
|
|
76
|
+
"topRisks": ["Low crawl coverage (2 pages)", "No CI integration detected"],
|
|
77
|
+
"recommendedNextChecks": ["Add CI pipeline", "Increase crawl depth"],
|
|
78
|
+
"honestyNotes": ["API coverage: not_applicable (no API endpoints found — excluded from score)"]
|
|
79
|
+
}
|
|
80
|
+
}
|
|
81
|
+
```
|
|
49
82
|
|
|
50
|
-
|
|
51
|
-
- **`analyze_app`** — quality scan (optional form-login or storage-state auth). **Default payload is summary-first:** `summary`, `topGaps`, `costIntelligenceSummary`, `nextDeterministicChecks`, small previews. Set **`includeFullReport: true`** for the full `analyzeApp` result (all scenarios). Optional harness overrides: **`llmMaxOutputTokensPerCall`**, **`llmTokenBudget`** (legacy), **`testGenerationLimit`**, **`enableLlmScenarios`** (default true when omitted).
|
|
52
|
-
- **`detect_auth(url, timeoutMs?)`** — single-pattern auth guess with a short recommendation (lighter than `explore_auth`).
|
|
53
|
-
- **`qulib_score_automation(repoPath, includeFullDimensions?)`** — score a local automation repo across six dimensions (test coverage breadth, framework adoption, test-id hygiene, CI integration, auth test coverage, component test ratio). Returns an overall 0–100 score, maturity level (L1–L5), and top recommendations. Each dimension carries an **`applicability`** field (`applicable` / `not_applicable` / `unknown`); the overall score normalizes across applicable dimensions only so absent capabilities never get silent partial credit. **`repoPath`** must be an absolute path on the MCP host. Pass **`includeFullDimensions: true`** for per-dimension evidence and reasons.
|
|
83
|
+
### `analyze_app` detail
|
|
54
84
|
|
|
55
|
-
|
|
85
|
+
- **Default payload:** `summary`, `topGaps`, `costIntelligenceSummary`, `nextDeterministicChecks`, small previews.
|
|
86
|
+
- **`includeFullReport: true`** — full `gapAnalysis` (all scenarios) and full `repoInventory`.
|
|
87
|
+
- **`agentSummary: true`** — compact gate-decision payload (`pass`/`warn`/`fail`) for CI orchestrators.
|
|
88
|
+
- Optional harness overrides: **`llmMaxOutputTokensPerCall`**, **`llmTokenBudget`** (legacy), **`testGenerationLimit`**, **`enableLlmScenarios`**.
|
|
56
89
|
|
|
57
|
-
|
|
58
|
-
- Accessibility violations (axe-core, WCAG 2 A/AA)
|
|
59
|
-
- Broken links
|
|
60
|
-
- Console errors and coverage warnings
|
|
61
|
-
- Prioritized gaps with severity
|
|
90
|
+
Returns: release confidence score (0–100), accessibility violations (axe-core, WCAG 2 A/AA), broken links, console errors and coverage warnings, prioritized gaps with severity.
|
|
62
91
|
|
|
63
92
|
Supports optional form-login auth for scanning authenticated pages. If auth is required but not configured, the scan can stop early with `mode: auth-required` and guidance in `detectedAuth` / the decision log.
|
|
64
93
|
|
|
@@ -110,8 +139,6 @@ When the model sees **`unrecognizedButtons`**, it can ask the user to register a
|
|
|
110
139
|
| Size | Small: top gaps, cost summary, next checks, `repoInventorySummary` (counts only) | Full `gapAnalysis` (all scenarios) and full `repoInventory` (test files, missing test IDs) |
|
|
111
140
|
| When to use | Routine agent turns, chat context limits | Deep dives, exporting full scenario JSON |
|
|
112
141
|
|
|
113
|
-
> **0.4.2 note:** The compact response now ships `repoInventorySummary` (route/test/missing-id counts plus framework verdict) instead of the full `repoInventory`. Agents that need the raw `testFiles` or `missingTestIds` arrays should pass `includeFullReport: true`.
|
|
114
|
-
|
|
115
142
|
Example (full):
|
|
116
143
|
|
|
117
144
|
```json
|
package/dist/index.js
CHANGED
|
@@ -14,7 +14,8 @@ import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js';
|
|
|
14
14
|
import { StdioServerTransport } from '@modelcontextprotocol/sdk/server/stdio.js';
|
|
15
15
|
const requirePkg = createRequire(import.meta.url);
|
|
16
16
|
const pkg = requirePkg('../package.json');
|
|
17
|
-
import { analyzeApp, detectAuth, exploreAuth, scanRepo, computeAutomationMaturity, scaffoldTests, discoverApiSurfaceWithRepo, computeApiCoverage, } from '@qulib/core';
|
|
17
|
+
import { analyzeApp, detectAuth, exploreAuth, scanRepo, computeAutomationMaturity, scaffoldTests, discoverApiSurfaceWithRepo, computeApiCoverage, computeReleaseConfidence, buildConfidenceInputFromQulib, } from '@qulib/core';
|
|
18
|
+
import { RecipeIdSchema } from '@qulib/core';
|
|
18
19
|
import { z } from 'zod';
|
|
19
20
|
import { buildAnalyzeAppMcpPayload } from './analyze-app-mcp-payload.js';
|
|
20
21
|
import { log } from './logger.js';
|
|
@@ -42,6 +43,18 @@ const mcpProgressLog = {
|
|
|
42
43
|
error: (message) => log.error(message),
|
|
43
44
|
debug: (message) => log.debug(message),
|
|
44
45
|
};
|
|
46
|
+
// ---------------------------------------------------------------------------
|
|
47
|
+
// Naming convergence — 0.10 (non-breaking aliases)
|
|
48
|
+
//
|
|
49
|
+
// Three legacy tools predate the qulib_ prefix convention:
|
|
50
|
+
// explore_auth → qulib_explore_auth (alias)
|
|
51
|
+
// detect_auth → qulib_detect_auth (alias)
|
|
52
|
+
// analyze_app → qulib_analyze_app (alias)
|
|
53
|
+
//
|
|
54
|
+
// The legacy names keep working unchanged. New integrations should prefer the
|
|
55
|
+
// qulib_ forms. Both will coexist through 1.0; the legacy names are marked
|
|
56
|
+
// "alias" in their descriptions. Removal is planned for 1.0.
|
|
57
|
+
// ---------------------------------------------------------------------------
|
|
45
58
|
// NOTE: MCP `auth` shape intentionally flattens the core `AuthConfigSchema` so an LLM
|
|
46
59
|
// can populate it without nested objects. We translate it back into core's nested
|
|
47
60
|
// `AuthConfig` (with `credentials: { username, password }` and `selectors: { ... }`)
|
|
@@ -117,14 +130,11 @@ const DetectAuthToolInputSchema = z.object({
|
|
|
117
130
|
url: z.string().url().describe('Full URL of the deployed app or login page'),
|
|
118
131
|
timeoutMs: z.number().int().positive().optional().describe('Page load timeout in milliseconds (default 15000)'),
|
|
119
132
|
});
|
|
120
|
-
|
|
121
|
-
description: 'Use this BEFORE analyze_app when scanning unfamiliar apps. Returns all detected sign-in paths with per-path requirements describing what credentials or actions the agent must collect from the user before calling analyze_app. Combines built-in OAuth/SSO labels, user-local patterns from ~/.qulib/providers.json, and heuristic unknown buttons.',
|
|
122
|
-
inputSchema: ExploreAuthToolInputSchema,
|
|
123
|
-
}, async ({ url, timeoutMs }) => {
|
|
133
|
+
async function handleExploreAuth({ url, timeoutMs, }) {
|
|
124
134
|
try {
|
|
125
|
-
log.info(`explore_auth
|
|
135
|
+
log.info(`explore_auth url=${url} timeoutMs=${timeoutMs ?? 20000}`);
|
|
126
136
|
const result = await exploreAuth(url, timeoutMs, mcpProgressLog);
|
|
127
|
-
log.info(`explore_auth
|
|
137
|
+
log.info(`explore_auth done authRequired=${result.authRequired} paths=${result.authPaths.length}`);
|
|
128
138
|
return {
|
|
129
139
|
content: [{ type: 'text', text: JSON.stringify(result, null, 2) }],
|
|
130
140
|
};
|
|
@@ -134,18 +144,24 @@ mcpServer.registerTool('explore_auth', {
|
|
|
134
144
|
log.error(`explore_auth failed: ${msg}`);
|
|
135
145
|
return toolError('QULIB_AUTH_EXPLORE_FAILED', msg, err instanceof Error ? err.stack : undefined);
|
|
136
146
|
}
|
|
137
|
-
}
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
147
|
+
}
|
|
148
|
+
const EXPLORE_AUTH_DESCRIPTION = 'Use this BEFORE analyze_app when scanning unfamiliar apps. Returns all detected sign-in paths with per-path requirements describing what credentials or actions the agent must collect from the user before calling analyze_app. Combines built-in OAuth/SSO labels, user-local patterns from ~/.qulib/providers.json, and heuristic unknown buttons.';
|
|
149
|
+
mcpServer.registerTool('explore_auth', {
|
|
150
|
+
description: `${EXPLORE_AUTH_DESCRIPTION} (Alias for new integrations: qulib_explore_auth)`,
|
|
151
|
+
inputSchema: ExploreAuthToolInputSchema,
|
|
152
|
+
}, handleExploreAuth);
|
|
153
|
+
mcpServer.registerTool('qulib_explore_auth', {
|
|
154
|
+
description: `${EXPLORE_AUTH_DESCRIPTION} (Canonical qulib_ form; explore_auth is the legacy alias kept for backwards compatibility.)`,
|
|
155
|
+
inputSchema: ExploreAuthToolInputSchema,
|
|
156
|
+
}, handleExploreAuth);
|
|
157
|
+
async function handleDetectAuth({ url, timeoutMs, }) {
|
|
142
158
|
try {
|
|
143
|
-
log.info(`detect_auth
|
|
159
|
+
log.info(`detect_auth url=${url} timeoutMs=${timeoutMs ?? 15000}`);
|
|
144
160
|
const result = await detectAuth(url, timeoutMs, mcpProgressLog);
|
|
145
161
|
const providerSummary = result.oauthButtons.length > 0
|
|
146
162
|
? result.oauthButtons.map((b) => b.provider).join(', ')
|
|
147
163
|
: result.provider ?? 'none';
|
|
148
|
-
log.info(`detect_auth
|
|
164
|
+
log.info(`detect_auth done type=${result.type} providers=${providerSummary} automatable=${result.type === 'form-login'}`);
|
|
149
165
|
return {
|
|
150
166
|
content: [{ type: 'text', text: JSON.stringify(result, null, 2) }],
|
|
151
167
|
};
|
|
@@ -155,11 +171,17 @@ mcpServer.registerTool('detect_auth', {
|
|
|
155
171
|
log.error(`detect_auth failed: ${msg}`);
|
|
156
172
|
return toolError('QULIB_AUTH_DETECT_FAILED', msg, err instanceof Error ? err.stack : undefined);
|
|
157
173
|
}
|
|
158
|
-
}
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
|
|
162
|
-
|
|
174
|
+
}
|
|
175
|
+
const DETECT_AUTH_DESCRIPTION = 'Detect the authentication pattern used by a deployed web app. Returns the auth type (form-login, oauth, magic-link, none, or unknown) and a recommendation for how to configure qulib to scan past it.';
|
|
176
|
+
mcpServer.registerTool('detect_auth', {
|
|
177
|
+
description: `${DETECT_AUTH_DESCRIPTION} (Alias for new integrations: qulib_detect_auth)`,
|
|
178
|
+
inputSchema: DetectAuthToolInputSchema,
|
|
179
|
+
}, handleDetectAuth);
|
|
180
|
+
mcpServer.registerTool('qulib_detect_auth', {
|
|
181
|
+
description: `${DETECT_AUTH_DESCRIPTION} (Canonical qulib_ form; detect_auth is the legacy alias kept for backwards compatibility.)`,
|
|
182
|
+
inputSchema: DetectAuthToolInputSchema,
|
|
183
|
+
}, handleDetectAuth);
|
|
184
|
+
async function handleAnalyzeApp(input) {
|
|
163
185
|
try {
|
|
164
186
|
const successIndicator = input.auth?.type === 'form-login' &&
|
|
165
187
|
input.auth.successUrlContains !== undefined &&
|
|
@@ -224,7 +246,16 @@ mcpServer.registerTool('analyze_app', {
|
|
|
224
246
|
log.error(`analyze_app failed: ${msg}`);
|
|
225
247
|
return toolError('QULIB_SCAN_FAILED', msg, err instanceof Error ? err.stack : undefined);
|
|
226
248
|
}
|
|
227
|
-
}
|
|
249
|
+
}
|
|
250
|
+
const ANALYZE_APP_DESCRIPTION = 'Analyze a deployed web app for quality gaps. Default response is summary-first (top gaps, cost summary, next checks). Set includeFullReport for the full gapAnalysis. Set agentSummary for the compact gate-decision payload (pass/warn/fail with honesty notes) — use this when calling from a CI gate or orchestrator. Optional llmMaxOutputTokensPerCall / llmTokenBudget (legacy), testGenerationLimit, enableLlmScenarios align with @qulib/core HarnessConfig.';
|
|
251
|
+
mcpServer.registerTool('analyze_app', {
|
|
252
|
+
description: `${ANALYZE_APP_DESCRIPTION} (Alias for new integrations: qulib_analyze_app)`,
|
|
253
|
+
inputSchema: AnalyzeInputSchema,
|
|
254
|
+
}, handleAnalyzeApp);
|
|
255
|
+
mcpServer.registerTool('qulib_analyze_app', {
|
|
256
|
+
description: `${ANALYZE_APP_DESCRIPTION} (Canonical qulib_ form; analyze_app is the legacy alias kept for backwards compatibility.)`,
|
|
257
|
+
inputSchema: AnalyzeInputSchema,
|
|
258
|
+
}, handleAnalyzeApp);
|
|
228
259
|
mcpServer.registerTool('qulib_score_automation', {
|
|
229
260
|
description: 'Score an automation repository for QA maturity across six dimensions: test coverage breadth, framework adoption, test-id hygiene, CI integration, auth test coverage, and component test ratio. Returns an overall score (0–100), maturity level (L1–L5), and prioritized recommendations.',
|
|
230
261
|
inputSchema: ScoreAutomationInputSchema,
|
|
@@ -266,7 +297,7 @@ const ScaffoldTestsInputSchema = z.object({
|
|
|
266
297
|
framework: z
|
|
267
298
|
.enum(['cypress-e2e', 'playwright'])
|
|
268
299
|
.optional()
|
|
269
|
-
.describe('Test framework to generate. Default: cypress-e2e'),
|
|
300
|
+
.describe('Test framework to generate. Default and recommended: cypress-e2e. playwright is accepted but not yet implemented (returns an error).'),
|
|
270
301
|
maxPagesToScan: z
|
|
271
302
|
.number()
|
|
272
303
|
.int()
|
|
@@ -274,18 +305,31 @@ const ScaffoldTestsInputSchema = z.object({
|
|
|
274
305
|
.max(20)
|
|
275
306
|
.optional()
|
|
276
307
|
.describe('Max pages to crawl when running analyze_app internally. Default: 10'),
|
|
308
|
+
recipes: z
|
|
309
|
+
.array(RecipeIdSchema)
|
|
310
|
+
.optional()
|
|
311
|
+
.describe('Optional list of reusable test-pattern recipes to append to the scaffold. ' +
|
|
312
|
+
'Each recipe adds proven NQ-2/CaseLoom-derived scenarios: ' +
|
|
313
|
+
'"auth" = login/logout/protected-route flows; ' +
|
|
314
|
+
'"a11y" = heading/landmark/title accessibility checks; ' +
|
|
315
|
+
'"nav" = deep-link/browser-back/404 handling; ' +
|
|
316
|
+
'"seed" = data-seeding/state-reset helpers. ' +
|
|
317
|
+
'Recipe scenarios are APPENDED to crawl-derived scenarios — they never replace them. ' +
|
|
318
|
+
'Example: ["auth", "a11y"] adds 6 ready-to-run test scenarios.'),
|
|
277
319
|
});
|
|
278
320
|
mcpServer.registerTool('qulib_scaffold_tests', {
|
|
279
|
-
description: 'Generate a ready-to-run test scaffold for a deployed web app. Crawls the URL, identifies quality gaps and user flows, then produces framework-specific test files
|
|
321
|
+
description: 'Generate a ready-to-run test scaffold for a deployed web app. Crawls the URL, identifies quality gaps and user flows, then produces framework-specific test files plus the project config and package.json deps. Returns generatedTests (array of {filename, code, outputPath}) and projectConfig so an agent can write the files directly to a repo without any manual test-writing. Supported framework: cypress-e2e (default). playwright scaffold is experimental and not yet implemented. Optionally pass recipes (e.g. ["auth","a11y"]) to append proven NQ-2/CaseLoom-derived test patterns for common flows — auth adds login/logout/protected-route tests, a11y adds heading/landmark/title checks, nav adds deep-link/404 tests, seed adds state-reset helpers.',
|
|
280
322
|
inputSchema: ScaffoldTestsInputSchema,
|
|
281
|
-
}, async ({ url, framework, maxPagesToScan }) => {
|
|
323
|
+
}, async ({ url, framework, maxPagesToScan, recipes }) => {
|
|
282
324
|
try {
|
|
283
|
-
|
|
325
|
+
const recipesLog = recipes && recipes.length > 0 ? ` recipes=[${recipes.join(',')}]` : '';
|
|
326
|
+
log.info(`qulib_scaffold_tests url=${url} framework=${framework ?? 'cypress-e2e'} maxPagesToScan=${maxPagesToScan ?? 10}${recipesLog}`);
|
|
284
327
|
const result = await scaffoldTests(url, {
|
|
285
328
|
framework: framework ?? 'cypress-e2e',
|
|
286
329
|
maxPagesToScan: maxPagesToScan ?? 10,
|
|
287
330
|
progressLog: mcpProgressLog,
|
|
288
331
|
telemetry: telemetrySink,
|
|
332
|
+
...(recipes && recipes.length > 0 && { recipes }),
|
|
289
333
|
});
|
|
290
334
|
return {
|
|
291
335
|
content: [
|
|
@@ -359,5 +403,113 @@ mcpServer.registerTool('qulib_score_api', {
|
|
|
359
403
|
return toolError('QULIB_API_SCORE_FAILED', msg, err instanceof Error ? err.stack : undefined);
|
|
360
404
|
}
|
|
361
405
|
});
|
|
406
|
+
// ---------------------------------------------------------------------------
|
|
407
|
+
// qulib_score_confidence — P3 Release Confidence Aggregator
|
|
408
|
+
// Composes existing collectors (analyze_app, qulib_score_automation,
|
|
409
|
+
// qulib_score_api) into one fused Release Confidence verdict. Honors the
|
|
410
|
+
// tool-explosion guardrail by composing, not fanning out (index.ts lines 4–10).
|
|
411
|
+
// ---------------------------------------------------------------------------
|
|
412
|
+
const ScoreConfidenceInputSchema = z.object({
|
|
413
|
+
url: z.string().url().optional().describe('URL of the deployed app to analyze (runs analyze_app if provided)'),
|
|
414
|
+
repoPath: z
|
|
415
|
+
.string()
|
|
416
|
+
.optional()
|
|
417
|
+
.describe('Absolute path to the repository (runs qulib_score_automation + qulib_score_api if provided)'),
|
|
418
|
+
includeViews: z
|
|
419
|
+
.object({
|
|
420
|
+
replay: z.boolean().optional().describe('Include the Replay provenance trace in the response'),
|
|
421
|
+
})
|
|
422
|
+
.optional()
|
|
423
|
+
.describe('Optional projection flags — which views to include beyond the Release Confidence view'),
|
|
424
|
+
subject: z
|
|
425
|
+
.object({
|
|
426
|
+
kind: z.enum(['release', 'pr', 'deploy', 'app', 'repo']).optional(),
|
|
427
|
+
ref: z.string().optional(),
|
|
428
|
+
tenantId: z.string().optional(),
|
|
429
|
+
})
|
|
430
|
+
.optional()
|
|
431
|
+
.describe('Subject metadata for the confidence verdict; defaults are inferred from url/repoPath'),
|
|
432
|
+
});
|
|
433
|
+
mcpServer.registerTool('qulib_score_confidence', {
|
|
434
|
+
description: 'Compute a fused Release Confidence verdict by composing qulib evidence collectors. ' +
|
|
435
|
+
'Given a URL and/or repo path, runs analyze_app / qulib_score_automation / qulib_score_api as applicable, ' +
|
|
436
|
+
'then fuses the signals into one verdict (ship | caution | hold | block) with a 0–100 confidence score, ' +
|
|
437
|
+
'L1–L5 level, per-source contributions, honesty notes for any excluded/unknown source, and recommended next checks. ' +
|
|
438
|
+
'Returns the Release Confidence view. Pass includeViews.replay for the full provenance trace.',
|
|
439
|
+
inputSchema: ScoreConfidenceInputSchema,
|
|
440
|
+
}, async ({ url, repoPath, includeViews, subject }) => {
|
|
441
|
+
try {
|
|
442
|
+
const subjectRef = subject?.ref ?? url ?? repoPath ?? 'unknown';
|
|
443
|
+
const subjectKind = subject?.kind ?? (url && repoPath ? 'release' : url ? 'app' : 'repo');
|
|
444
|
+
const tenantId = subject?.tenantId ?? 'default';
|
|
445
|
+
const confidenceSubject = { kind: subjectKind, ref: subjectRef, tenantId };
|
|
446
|
+
// Collect evidence from whichever collectors apply.
|
|
447
|
+
let analyzeResult;
|
|
448
|
+
let maturityResult;
|
|
449
|
+
let apiCoverageResult;
|
|
450
|
+
if (url) {
|
|
451
|
+
log.info(`qulib_score_confidence: running analyze_app url=${url}`);
|
|
452
|
+
const harnessConfig = {
|
|
453
|
+
maxPagesToScan: 10,
|
|
454
|
+
maxDepth: 3,
|
|
455
|
+
minPagesForConfidence: 3,
|
|
456
|
+
timeoutMs: 30000,
|
|
457
|
+
retryCount: 0,
|
|
458
|
+
llmTokenBudget: 4096,
|
|
459
|
+
testGenerationLimit: 5,
|
|
460
|
+
enableLlmScenarios: false,
|
|
461
|
+
readOnlyMode: true,
|
|
462
|
+
requireHumanReview: false,
|
|
463
|
+
failOnConsoleError: false,
|
|
464
|
+
explorer: 'playwright',
|
|
465
|
+
defaultAdapter: 'playwright',
|
|
466
|
+
adapters: ['playwright'],
|
|
467
|
+
};
|
|
468
|
+
analyzeResult = await analyzeApp({
|
|
469
|
+
url,
|
|
470
|
+
writeArtifacts: false,
|
|
471
|
+
config: harnessConfig,
|
|
472
|
+
progressLog: mcpProgressLog,
|
|
473
|
+
telemetry: telemetrySink,
|
|
474
|
+
});
|
|
475
|
+
}
|
|
476
|
+
if (repoPath) {
|
|
477
|
+
const abs = validateAbsoluteRepoPath(repoPath);
|
|
478
|
+
log.info(`qulib_score_confidence: running qulib_score_automation + qulib_score_api repoPath=${abs}`);
|
|
479
|
+
const repo = await scanRepo(abs);
|
|
480
|
+
maturityResult = computeAutomationMaturity(repo);
|
|
481
|
+
const apiSurface = await discoverApiSurfaceWithRepo(abs, repo, { enableTier3: false });
|
|
482
|
+
apiCoverageResult = computeApiCoverage(repo, apiSurface);
|
|
483
|
+
}
|
|
484
|
+
// Build the evidence bundle from qulib's own collectors.
|
|
485
|
+
const confidenceInput = buildConfidenceInputFromQulib({
|
|
486
|
+
analyze: analyzeResult,
|
|
487
|
+
maturity: maturityResult,
|
|
488
|
+
apiCoverage: apiCoverageResult,
|
|
489
|
+
subject: confidenceSubject,
|
|
490
|
+
});
|
|
491
|
+
// Run the pure scorer.
|
|
492
|
+
const rc = computeReleaseConfidence(confidenceInput);
|
|
493
|
+
// Build the response payload (Release Confidence view is always included).
|
|
494
|
+
const payload = { releaseConfidence: rc };
|
|
495
|
+
if (includeViews?.replay) {
|
|
496
|
+
const { buildReplay } = await import('@qulib/core');
|
|
497
|
+
payload['replay'] = buildReplay(confidenceInput, rc);
|
|
498
|
+
}
|
|
499
|
+
log.info(`qulib_score_confidence done verdict=${rc.verdict} confidenceScore=${rc.confidenceScore ?? 'null'} ` +
|
|
500
|
+
`level=${rc.level} evidenceSources=${confidenceInput.evidence.length}`);
|
|
501
|
+
return {
|
|
502
|
+
content: [{ type: 'text', text: JSON.stringify(payload, null, 2) }],
|
|
503
|
+
};
|
|
504
|
+
}
|
|
505
|
+
catch (err) {
|
|
506
|
+
const msg = err instanceof Error ? err.message : String(err);
|
|
507
|
+
if (msg.includes('repoPath must')) {
|
|
508
|
+
return toolError('QULIB_INPUT_INVALID', msg, undefined);
|
|
509
|
+
}
|
|
510
|
+
log.error(`qulib_score_confidence failed: ${msg}`);
|
|
511
|
+
return toolError('QULIB_CONFIDENCE_FAILED', msg, err instanceof Error ? err.stack : undefined);
|
|
512
|
+
}
|
|
513
|
+
});
|
|
362
514
|
const transport = new StdioServerTransport();
|
|
363
515
|
await mcpServer.connect(transport);
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@qulib/mcp",
|
|
3
|
-
"version": "0.
|
|
4
|
-
"description": "MCP server for Qulib — AI-callable
|
|
3
|
+
"version": "0.10.0",
|
|
4
|
+
"description": "MCP server for Qulib — AI-callable release confidence. Seven tools: fused verdict, live-app scan, automation maturity, API coverage, test scaffold, and auth tools.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"author": "Tapesh Nagarwal",
|
|
7
7
|
"homepage": "https://github.com/TapeshN/qulib#readme",
|
|
@@ -28,25 +28,31 @@
|
|
|
28
28
|
],
|
|
29
29
|
"scripts": {
|
|
30
30
|
"build": "npm --prefix ../.. run build -w @qulib/core && tsc && chmod +x dist/index.js",
|
|
31
|
+
"prepublishOnly": "npm run build",
|
|
31
32
|
"dev": "tsx src/index.ts",
|
|
32
|
-
"test": "node --import tsx/esm --test src/__tests__/summarize-analyze-result.test.ts src/__tests__/analyze-app-mcp-payload.test.ts"
|
|
33
|
+
"test": "node --import tsx/esm --test src/__tests__/summarize-analyze-result.test.ts src/__tests__/analyze-app-mcp-payload.test.ts src/__tests__/score-confidence-mcp.test.ts src/__tests__/naming-aliases.test.ts"
|
|
33
34
|
},
|
|
34
35
|
"dependencies": {
|
|
35
36
|
"@modelcontextprotocol/sdk": "^1.0.0",
|
|
36
|
-
"@qulib/core": "0.
|
|
37
|
+
"@qulib/core": "0.10.0",
|
|
37
38
|
"zod": "^3.23.0"
|
|
38
39
|
},
|
|
39
40
|
"devDependencies": {
|
|
40
41
|
"@types/node": "^20.0.0",
|
|
41
|
-
"tsx": "^4.
|
|
42
|
+
"tsx": "^4.22.4",
|
|
42
43
|
"typescript": "^5.4.0"
|
|
43
44
|
},
|
|
44
45
|
"keywords": [
|
|
45
46
|
"mcp",
|
|
47
|
+
"release-confidence",
|
|
46
48
|
"qa",
|
|
47
49
|
"quality",
|
|
50
|
+
"ship-verdict",
|
|
51
|
+
"automation-maturity",
|
|
48
52
|
"accessibility",
|
|
49
|
-
"
|
|
50
|
-
"
|
|
53
|
+
"ai",
|
|
54
|
+
"ci-gate",
|
|
55
|
+
"playwright",
|
|
56
|
+
"web-quality"
|
|
51
57
|
]
|
|
52
58
|
}
|