agentic-qe 3.6.9 → 3.6.11

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (117) hide show
  1. package/.claude/skills/.validation/schemas/skill-eval.schema.json +11 -1
  2. package/.claude/skills/pr-review/SKILL.md +2 -2
  3. package/.claude/skills/qcsd-production-swarm/SKILL.md +2781 -0
  4. package/.claude/skills/qcsd-production-swarm/evals/qcsd-production-swarm.yaml +246 -0
  5. package/.claude/skills/qcsd-production-swarm/schemas/output.json +505 -0
  6. package/.claude/skills/qcsd-production-swarm/scripts/validate-config.json +25 -0
  7. package/.claude/skills/skills-manifest.json +5 -5
  8. package/package.json +1 -1
  9. package/scripts/benchmark-hnsw-loading.ts +480 -0
  10. package/scripts/benchmark-kg-assisted.ts +725 -0
  11. package/scripts/collect-production-telemetry.sh +291 -0
  12. package/scripts/detect-skill-conflicts.ts +347 -0
  13. package/scripts/eval-driven-workflow.ts +704 -0
  14. package/scripts/run-skill-eval.ts +210 -10
  15. package/scripts/score-skill-quality.ts +511 -0
  16. package/v3/CHANGELOG.md +44 -0
  17. package/v3/assets/skills/pr-review/SKILL.md +2 -2
  18. package/v3/dist/cli/bundle.js +1526 -700
  19. package/v3/dist/cli/commands/code.d.ts.map +1 -1
  20. package/v3/dist/cli/commands/code.js +9 -85
  21. package/v3/dist/cli/commands/code.js.map +1 -1
  22. package/v3/dist/cli/commands/coverage.d.ts.map +1 -1
  23. package/v3/dist/cli/commands/coverage.js +3 -28
  24. package/v3/dist/cli/commands/coverage.js.map +1 -1
  25. package/v3/dist/cli/commands/hooks.d.ts.map +1 -1
  26. package/v3/dist/cli/commands/hooks.js +143 -2
  27. package/v3/dist/cli/commands/hooks.js.map +1 -1
  28. package/v3/dist/cli/commands/security.d.ts.map +1 -1
  29. package/v3/dist/cli/commands/security.js +3 -29
  30. package/v3/dist/cli/commands/security.js.map +1 -1
  31. package/v3/dist/cli/commands/test.d.ts.map +1 -1
  32. package/v3/dist/cli/commands/test.js +11 -58
  33. package/v3/dist/cli/commands/test.js.map +1 -1
  34. package/v3/dist/cli/utils/file-discovery.d.ts +27 -0
  35. package/v3/dist/cli/utils/file-discovery.d.ts.map +1 -0
  36. package/v3/dist/cli/utils/file-discovery.js +105 -0
  37. package/v3/dist/cli/utils/file-discovery.js.map +1 -0
  38. package/v3/dist/coordination/task-executor.d.ts.map +1 -1
  39. package/v3/dist/coordination/task-executor.js +304 -44
  40. package/v3/dist/coordination/task-executor.js.map +1 -1
  41. package/v3/dist/domains/code-intelligence/coordinator.d.ts.map +1 -1
  42. package/v3/dist/domains/code-intelligence/coordinator.js +8 -1
  43. package/v3/dist/domains/code-intelligence/coordinator.js.map +1 -1
  44. package/v3/dist/domains/code-intelligence/services/metric-collector/index.d.ts.map +1 -1
  45. package/v3/dist/domains/code-intelligence/services/metric-collector/index.js +10 -0
  46. package/v3/dist/domains/code-intelligence/services/metric-collector/index.js.map +1 -1
  47. package/v3/dist/domains/code-intelligence/services/metric-collector/interfaces.d.ts +7 -1
  48. package/v3/dist/domains/code-intelligence/services/metric-collector/interfaces.d.ts.map +1 -1
  49. package/v3/dist/domains/code-intelligence/services/metric-collector/interfaces.js +10 -1
  50. package/v3/dist/domains/code-intelligence/services/metric-collector/interfaces.js.map +1 -1
  51. package/v3/dist/domains/code-intelligence/services/metric-collector/loc-counter.js +34 -10
  52. package/v3/dist/domains/code-intelligence/services/metric-collector/loc-counter.js.map +1 -1
  53. package/v3/dist/domains/coverage-analysis/services/hnsw-index.d.ts +9 -0
  54. package/v3/dist/domains/coverage-analysis/services/hnsw-index.d.ts.map +1 -1
  55. package/v3/dist/domains/coverage-analysis/services/hnsw-index.js +38 -3
  56. package/v3/dist/domains/coverage-analysis/services/hnsw-index.js.map +1 -1
  57. package/v3/dist/domains/test-generation/generators/jest-vitest-generator.d.ts.map +1 -1
  58. package/v3/dist/domains/test-generation/generators/jest-vitest-generator.js +58 -6
  59. package/v3/dist/domains/test-generation/generators/jest-vitest-generator.js.map +1 -1
  60. package/v3/dist/domains/test-generation/generators/mocha-generator.d.ts.map +1 -1
  61. package/v3/dist/domains/test-generation/generators/mocha-generator.js +79 -7
  62. package/v3/dist/domains/test-generation/generators/mocha-generator.js.map +1 -1
  63. package/v3/dist/domains/test-generation/generators/pytest-generator.d.ts +4 -0
  64. package/v3/dist/domains/test-generation/generators/pytest-generator.d.ts.map +1 -1
  65. package/v3/dist/domains/test-generation/generators/pytest-generator.js +77 -10
  66. package/v3/dist/domains/test-generation/generators/pytest-generator.js.map +1 -1
  67. package/v3/dist/domains/test-generation/interfaces/test-generator.interface.d.ts +21 -0
  68. package/v3/dist/domains/test-generation/interfaces/test-generator.interface.d.ts.map +1 -1
  69. package/v3/dist/domains/test-generation/interfaces.d.ts +21 -0
  70. package/v3/dist/domains/test-generation/interfaces.d.ts.map +1 -1
  71. package/v3/dist/domains/test-generation/services/test-generator.d.ts +22 -0
  72. package/v3/dist/domains/test-generation/services/test-generator.d.ts.map +1 -1
  73. package/v3/dist/domains/test-generation/services/test-generator.js +163 -3
  74. package/v3/dist/domains/test-generation/services/test-generator.js.map +1 -1
  75. package/v3/dist/init/init-wizard-hooks.d.ts +8 -1
  76. package/v3/dist/init/init-wizard-hooks.d.ts.map +1 -1
  77. package/v3/dist/init/init-wizard-hooks.js +47 -39
  78. package/v3/dist/init/init-wizard-hooks.js.map +1 -1
  79. package/v3/dist/init/phases/07-hooks.d.ts +11 -1
  80. package/v3/dist/init/phases/07-hooks.d.ts.map +1 -1
  81. package/v3/dist/init/phases/07-hooks.js +46 -50
  82. package/v3/dist/init/phases/07-hooks.js.map +1 -1
  83. package/v3/dist/init/settings-merge.d.ts +35 -0
  84. package/v3/dist/init/settings-merge.d.ts.map +1 -0
  85. package/v3/dist/init/settings-merge.js +140 -0
  86. package/v3/dist/init/settings-merge.js.map +1 -0
  87. package/v3/dist/integrations/agentic-flow/model-router/router.js +1 -1
  88. package/v3/dist/integrations/agentic-flow/model-router/router.js.map +1 -1
  89. package/v3/dist/integrations/agentic-flow/model-router/score-calculator.d.ts.map +1 -1
  90. package/v3/dist/integrations/agentic-flow/model-router/score-calculator.js +18 -3
  91. package/v3/dist/integrations/agentic-flow/model-router/score-calculator.js.map +1 -1
  92. package/v3/dist/integrations/agentic-flow/model-router/signal-collector.d.ts +3 -3
  93. package/v3/dist/integrations/agentic-flow/model-router/signal-collector.d.ts.map +1 -1
  94. package/v3/dist/integrations/agentic-flow/model-router/signal-collector.js +18 -0
  95. package/v3/dist/integrations/agentic-flow/model-router/signal-collector.js.map +1 -1
  96. package/v3/dist/kernel/unified-memory-hnsw.d.ts +29 -0
  97. package/v3/dist/kernel/unified-memory-hnsw.d.ts.map +1 -1
  98. package/v3/dist/kernel/unified-memory-hnsw.js +136 -0
  99. package/v3/dist/kernel/unified-memory-hnsw.js.map +1 -1
  100. package/v3/dist/kernel/unified-memory.d.ts +2 -2
  101. package/v3/dist/kernel/unified-memory.d.ts.map +1 -1
  102. package/v3/dist/kernel/unified-memory.js +7 -9
  103. package/v3/dist/kernel/unified-memory.js.map +1 -1
  104. package/v3/dist/learning/qe-hooks.d.ts.map +1 -1
  105. package/v3/dist/learning/qe-hooks.js +34 -3
  106. package/v3/dist/learning/qe-hooks.js.map +1 -1
  107. package/v3/dist/mcp/bundle.js +1403 -425
  108. package/v3/dist/mcp/handlers/domain-handler-configs.d.ts.map +1 -1
  109. package/v3/dist/mcp/handlers/domain-handler-configs.js +40 -31
  110. package/v3/dist/mcp/handlers/domain-handler-configs.js.map +1 -1
  111. package/v3/dist/mcp/handlers/task-handlers.d.ts.map +1 -1
  112. package/v3/dist/mcp/handlers/task-handlers.js +68 -5
  113. package/v3/dist/mcp/handlers/task-handlers.js.map +1 -1
  114. package/v3/dist/mcp/protocol-server.d.ts.map +1 -1
  115. package/v3/dist/mcp/protocol-server.js +16 -2
  116. package/v3/dist/mcp/protocol-server.js.map +1 -1
  117. package/v3/package.json +1 -1
@@ -0,0 +1,511 @@
1
+ #!/usr/bin/env npx tsx
2
+ /**
3
+ * Skill Quality Scorer
4
+ * Deterministic quality scoring for SKILL.md files across 8 dimensions.
5
+ *
6
+ * Usage:
7
+ * npx tsx scripts/score-skill-quality.ts
8
+ * npx tsx scripts/score-skill-quality.ts --skill api-testing-patterns
9
+ * npx tsx scripts/score-skill-quality.ts --suggestions
10
+ * npx tsx scripts/score-skill-quality.ts --json
11
+ * npx tsx scripts/score-skill-quality.ts --min-score 70
12
+ */
13
+
14
+ import { readFileSync, writeFileSync, existsSync, readdirSync, statSync } from 'fs';
15
+ import { join, dirname } from 'path';
16
+
17
+ // ============================================================================
18
+ // TYPES
19
+ // ============================================================================
20
+
21
+ interface DimensionScores {
22
+ description: number; // 0-10
23
+ frontmatter: number; // 0-10
24
+ conciseness: number; // 0-10
25
+ actionability: number; // 0-10
26
+ structure: number; // 0-10
27
+ triggers: number; // 0-10
28
+ codeExamples: number; // 0-10
29
+ metadata: number; // 0-10
30
+ }
31
+
32
+ interface SkillScore {
33
+ name: string;
34
+ score: number;
35
+ grade: string;
36
+ dimensions: DimensionScores;
37
+ suggestions: string[];
38
+ }
39
+
40
+ // ============================================================================
41
+ // CONSTANTS
42
+ // ============================================================================
43
+
44
+ const SKILLS_DIR = '.claude/skills';
45
+ const PLATFORM_PREFIXES = ['v3-', 'flow-nexus-', 'agentdb-', 'reasoningbank-', 'swarm-'];
46
+
47
+ const EXPECTED_FRONTMATTER_FIELDS = [
48
+ 'name', 'description', 'category', 'priority', 'tokenEstimate',
49
+ 'agents', 'tags', 'trust_tier', 'validation', 'dependencies',
50
+ 'implementation_status', 'optimization_version', 'last_optimized',
51
+ 'quick_reference_card',
52
+ ];
53
+
54
+ const TRIGGER_TERMS = [
55
+ 'owasp', 'pact', 'k6', 'artillery', 'jmeter', 'wcag', 'gdpr', 'hipaa',
56
+ 'soc2', 'pci', 'jest', 'vitest', 'playwright', 'cypress', 'selenium',
57
+ 'supertest', 'graphql', 'rest', 'grpc', 'openapi', 'swagger', 'postman',
58
+ 'cucumber', 'gherkin', 'bdd', 'tdd', 'mutation', 'stryker', 'pitest',
59
+ 'sonarqube', 'eslint', 'prettier', 'docker', 'kubernetes', 'terraform',
60
+ 'kafka', 'rabbitmq', 'redis', 'postgresql', 'mongodb', 'dynamodb',
61
+ 'oauth', 'jwt', 'saml', 'oidc', 'xss', 'sqli', 'csrf', 'ssrf',
62
+ 'sast', 'dast', 'iast', 'sca', 'sbom', 'cve',
63
+ ];
64
+
65
+ const WEIGHTS: Record<keyof DimensionScores, number> = {
66
+ description: 20,
67
+ frontmatter: 15,
68
+ conciseness: 15,
69
+ actionability: 15,
70
+ structure: 15,
71
+ triggers: 10,
72
+ codeExamples: 5,
73
+ metadata: 5,
74
+ };
75
+
76
+ // ============================================================================
77
+ // FRONTMATTER PARSER (from detect-skill-conflicts.ts)
78
+ // ============================================================================
79
+
80
+ function parseYamlFrontmatter(content: string): Record<string, unknown> {
81
+ const match = content.match(/^---\n([\s\S]*?)\n---/);
82
+ if (!match) return {};
83
+
84
+ const result: Record<string, unknown> = {};
85
+ const lines = match[1].split('\n');
86
+ let inNested = false;
87
+ let nestedKey = '';
88
+ const nestedObj: Record<string, unknown> = {};
89
+
90
+ for (const line of lines) {
91
+ if (!line.trim() || line.trim().startsWith('#')) continue;
92
+ const indent = line.search(/\S/);
93
+
94
+ if (indent > 0 && inNested) {
95
+ const kv = line.trim().match(/^([\w_]+):\s*(.+)$/);
96
+ if (kv) nestedObj[kv[1]] = parseYamlValue(kv[2]);
97
+ continue;
98
+ }
99
+ if (indent === 0 && inNested) {
100
+ result[nestedKey] = { ...nestedObj };
101
+ inNested = false;
102
+ }
103
+
104
+ const kv = line.trim().match(/^([\w_]+):\s*(.*)$/);
105
+ if (kv) {
106
+ const [, key, value] = kv;
107
+ if (!value || value.trim() === '') {
108
+ inNested = true;
109
+ nestedKey = key;
110
+ Object.keys(nestedObj).forEach(k => delete nestedObj[k]);
111
+ } else {
112
+ result[key] = parseYamlValue(value.trim());
113
+ }
114
+ }
115
+ }
116
+ if (inNested) result[nestedKey] = { ...nestedObj };
117
+ return result;
118
+ }
119
+
120
+ function parseYamlValue(value: string): unknown {
121
+ const t = value.trim();
122
+ if ((t.startsWith('"') && t.endsWith('"')) || (t.startsWith("'") && t.endsWith("'")))
123
+ return t.slice(1, -1);
124
+ if (t.startsWith('[') && t.endsWith(']'))
125
+ return t.slice(1, -1).split(',').map(i => {
126
+ const s = i.trim();
127
+ return (s.startsWith('"') || s.startsWith("'")) ? s.slice(1, -1) : s;
128
+ });
129
+ if (t === 'true') return true;
130
+ if (t === 'false') return false;
131
+ const n = Number(t);
132
+ if (!isNaN(n) && t !== '') return n;
133
+ return t;
134
+ }
135
+
136
+ // ============================================================================
137
+ // SKILL DISCOVERY (from detect-skill-conflicts.ts)
138
+ // ============================================================================
139
+
140
+ function getProjectRoot(): string {
141
+ let dir = process.cwd();
142
+ while (dir !== '/') {
143
+ if (existsSync(join(dir, 'package.json'))) return dir;
144
+ dir = dirname(dir);
145
+ }
146
+ return process.cwd();
147
+ }
148
+
149
+ interface SkillFile {
150
+ name: string;
151
+ content: string;
152
+ frontmatter: Record<string, unknown>;
153
+ body: string;
154
+ }
155
+
156
+ function discoverSkills(projectRoot: string, filterSkill?: string): SkillFile[] {
157
+ const skillsPath = join(projectRoot, SKILLS_DIR);
158
+ if (!existsSync(skillsPath)) {
159
+ console.error(`Skills directory not found: ${skillsPath}`);
160
+ process.exit(1);
161
+ }
162
+
163
+ const skills: SkillFile[] = [];
164
+ for (const entry of readdirSync(skillsPath)) {
165
+ if (entry.startsWith('.') || !statSync(join(skillsPath, entry)).isDirectory()) continue;
166
+ if (PLATFORM_PREFIXES.some(p => entry.startsWith(p))) continue;
167
+ if (filterSkill && entry !== filterSkill) continue;
168
+
169
+ const mdPath = existsSync(join(skillsPath, entry, 'SKILL.md'))
170
+ ? join(skillsPath, entry, 'SKILL.md')
171
+ : existsSync(join(skillsPath, entry, 'skill.md'))
172
+ ? join(skillsPath, entry, 'skill.md')
173
+ : null;
174
+ if (!mdPath) continue;
175
+
176
+ const content = readFileSync(mdPath, 'utf-8');
177
+ const frontmatter = parseYamlFrontmatter(content);
178
+ const bodyMatch = content.match(/^---[\s\S]*?---\s*\n([\s\S]*)$/);
179
+ const body = bodyMatch ? bodyMatch[1] : content;
180
+
181
+ skills.push({ name: entry, content, frontmatter, body });
182
+ }
183
+
184
+ return skills.sort((a, b) => a.name.localeCompare(b.name));
185
+ }
186
+
187
+ // ============================================================================
188
+ // SCORING DIMENSIONS
189
+ // ============================================================================
190
+
191
+ function scoreDescription(fm: Record<string, unknown>, body: string): { score: number; suggestions: string[] } {
192
+ let pts = 0;
193
+ const suggestions: string[] = [];
194
+ const desc = String(fm.description || '');
195
+
196
+ if (!desc) {
197
+ suggestions.push('Add a "description" field to frontmatter');
198
+ return { score: 0, suggestions };
199
+ }
200
+
201
+ // Length sweet spot: 50-200 chars
202
+ if (desc.length >= 50 && desc.length <= 200) pts += 4;
203
+ else if (desc.length >= 30 && desc.length <= 250) pts += 2;
204
+ else suggestions.push(`Description length ${desc.length} chars — aim for 50-200`);
205
+
206
+ // "Use when" trigger phrase
207
+ if (/use when/i.test(desc)) pts += 3;
208
+ else suggestions.push('Add "Use when..." trigger phrase to description');
209
+
210
+ // Named tools/standards in description
211
+ const descLower = desc.toLowerCase();
212
+ const mentionedTools = TRIGGER_TERMS.filter(t => descLower.includes(t));
213
+ if (mentionedTools.length >= 2) pts += 3;
214
+ else if (mentionedTools.length === 1) pts += 2;
215
+ else suggestions.push('Mention specific tools/standards in description (e.g., Pact, k6, OWASP)');
216
+
217
+ return { score: Math.min(10, pts), suggestions };
218
+ }
219
+
220
+ function scoreFrontmatter(fm: Record<string, unknown>): { score: number; suggestions: string[] } {
221
+ const suggestions: string[] = [];
222
+ const present = EXPECTED_FRONTMATTER_FIELDS.filter(f => fm[f] !== undefined);
223
+ const ratio = present.length / EXPECTED_FRONTMATTER_FIELDS.length;
224
+ const score = Math.round(ratio * 10);
225
+
226
+ const missing = EXPECTED_FRONTMATTER_FIELDS.filter(f => fm[f] === undefined);
227
+ if (missing.length > 0 && missing.length <= 5) {
228
+ suggestions.push(`Add missing frontmatter: ${missing.join(', ')}`);
229
+ } else if (missing.length > 5) {
230
+ suggestions.push(`Add ${missing.length} missing frontmatter fields (e.g., ${missing.slice(0, 3).join(', ')})`);
231
+ }
232
+
233
+ return { score, suggestions };
234
+ }
235
+
236
+ function scoreConciseness(body: string): { score: number; suggestions: string[] } {
237
+ const suggestions: string[] = [];
238
+ const lines = body.split('\n').length;
239
+
240
+ let score: number;
241
+ if (lines >= 200 && lines <= 400) score = 10;
242
+ else if (lines >= 150 && lines <= 500) score = 8;
243
+ else if (lines >= 100 && lines <= 600) score = 6;
244
+ else if (lines >= 50 && lines <= 800) score = 4;
245
+ else if (lines > 1200) { score = 1; suggestions.push(`Body is ${lines} lines — aim for 200-400 for optimal readability`); }
246
+ else if (lines > 800) { score = 3; suggestions.push(`Body is ${lines} lines — consider trimming to 200-400`); }
247
+ else if (lines < 50) { score = 3; suggestions.push(`Body is only ${lines} lines — expand with tables, examples, decision matrices`); }
248
+ else score = 4;
249
+
250
+ return { score, suggestions };
251
+ }
252
+
253
+ function scoreActionability(body: string): { score: number; suggestions: string[] } {
254
+ let pts = 0;
255
+ const suggestions: string[] = [];
256
+
257
+ // <default_to_action> block
258
+ if (body.includes('<default_to_action>')) pts += 5;
259
+ else suggestions.push('Add a <default_to_action> block with immediate action steps');
260
+
261
+ // Verb-led numbered steps (e.g., "1. IDENTIFY", "2. TEST")
262
+ const numberedSteps = body.match(/^\d+\.\s+[A-Z]{2,}/gm);
263
+ if (numberedSteps && numberedSteps.length >= 3) pts += 3;
264
+ else if (numberedSteps && numberedSteps.length >= 1) pts += 1;
265
+ else suggestions.push('Use verb-led numbered steps (e.g., "1. IDENTIFY...", "2. TEST...")');
266
+
267
+ // Decision matrix (pattern selection, when-to-use table)
268
+ if (/quick pattern selection|when to use|decision matrix|pattern selection/i.test(body)) pts += 2;
269
+ else if (/\|.*\|.*\|/m.test(body)) pts += 1;
270
+
271
+ return { score: Math.min(10, pts), suggestions };
272
+ }
273
+
274
+ function scoreStructure(body: string): { score: number; suggestions: string[] } {
275
+ let pts = 0;
276
+ const suggestions: string[] = [];
277
+
278
+ // Quick Reference Card
279
+ if (/quick reference card/i.test(body)) pts += 3;
280
+ else suggestions.push('Add a "Quick Reference Card" section');
281
+
282
+ // Related Skills / Integration section
283
+ if (/related skills|related commands|integration with/i.test(body)) pts += 2;
284
+ else suggestions.push('Add a "Related Skills" section');
285
+
286
+ // Remember / Rules / Best Practices
287
+ if (/^##?\s*(remember|rules|best practices|critical)/im.test(body)) pts += 2;
288
+
289
+ // 3+ markdown tables
290
+ const tables = body.match(/\|.*\|.*\|/g);
291
+ if (tables && tables.length >= 6) pts += 2; // header+row = 2 lines per table, so 6 = 3+ tables
292
+ else if (tables && tables.length >= 4) pts += 1;
293
+ else suggestions.push('Add more markdown tables for quick-scan reference');
294
+
295
+ // Agent Coordination section
296
+ if (/agent coordination/i.test(body)) pts += 1;
297
+
298
+ return { score: Math.min(10, pts), suggestions };
299
+ }
300
+
301
+ function scoreTriggers(fm: Record<string, unknown>, body: string): { score: number; suggestions: string[] } {
302
+ const suggestions: string[] = [];
303
+ const desc = String(fm.description || '').toLowerCase();
304
+ const first60 = body.split('\n').slice(0, 60).join('\n').toLowerCase();
305
+ const combined = desc + ' ' + first60;
306
+
307
+ const found = TRIGGER_TERMS.filter(t => combined.includes(t));
308
+ let score: number;
309
+ if (found.length >= 5) score = 10;
310
+ else if (found.length >= 3) score = 7;
311
+ else if (found.length >= 1) score = 4;
312
+ else { score = 1; suggestions.push('Add specific tool/standard names (OWASP, Pact, k6, WCAG, etc.) in description and intro'); }
313
+
314
+ return { score, suggestions };
315
+ }
316
+
317
+ function scoreCodeExamples(body: string): { score: number; suggestions: string[] } {
318
+ const suggestions: string[] = [];
319
+ const blocks = body.match(/```[\s\S]*?```/g);
320
+ const count = blocks ? blocks.length : 0;
321
+
322
+ let score: number;
323
+ if (count >= 4) score = 10;
324
+ else if (count >= 2) score = 7;
325
+ else if (count === 1) score = 4;
326
+ else { score = 0; suggestions.push('Add fenced code examples to illustrate usage'); }
327
+
328
+ return { score, suggestions };
329
+ }
330
+
331
+ function scoreMetadata(fm: Record<string, unknown>): { score: number; suggestions: string[] } {
332
+ let pts = 0;
333
+ const suggestions: string[] = [];
334
+
335
+ // Tags (3+)
336
+ const tags = Array.isArray(fm.tags) ? fm.tags : [];
337
+ if (tags.length >= 3) pts += 3;
338
+ else if (tags.length >= 1) pts += 1;
339
+ else suggestions.push('Add 3+ tags to frontmatter');
340
+
341
+ // Valid category
342
+ if (fm.category) pts += 3;
343
+ else suggestions.push('Add a "category" field to frontmatter');
344
+
345
+ // Priority set
346
+ if (fm.priority) pts += 2;
347
+ else suggestions.push('Add a "priority" field to frontmatter');
348
+
349
+ // tokenEstimate present
350
+ if (fm.tokenEstimate !== undefined) pts += 2;
351
+ else suggestions.push('Add a "tokenEstimate" field to frontmatter');
352
+
353
+ return { score: Math.min(10, pts), suggestions };
354
+ }
355
+
356
+ // ============================================================================
357
+ // SCORING ORCHESTRATION
358
+ // ============================================================================
359
+
360
+ function scoreSkill(skill: SkillFile): SkillScore {
361
+ const allSuggestions: string[] = [];
362
+
363
+ const desc = scoreDescription(skill.frontmatter, skill.body);
364
+ const fm = scoreFrontmatter(skill.frontmatter);
365
+ const conc = scoreConciseness(skill.body);
366
+ const act = scoreActionability(skill.body);
367
+ const str = scoreStructure(skill.body);
368
+ const trg = scoreTriggers(skill.frontmatter, skill.body);
369
+ const code = scoreCodeExamples(skill.body);
370
+ const meta = scoreMetadata(skill.frontmatter);
371
+
372
+ const dimensions: DimensionScores = {
373
+ description: desc.score,
374
+ frontmatter: fm.score,
375
+ conciseness: conc.score,
376
+ actionability: act.score,
377
+ structure: str.score,
378
+ triggers: trg.score,
379
+ codeExamples: code.score,
380
+ metadata: meta.score,
381
+ };
382
+
383
+ // Weighted score out of 100
384
+ let weighted = 0;
385
+ for (const [key, weight] of Object.entries(WEIGHTS)) {
386
+ weighted += (dimensions[key as keyof DimensionScores] / 10) * weight;
387
+ }
388
+ const score = Math.round(weighted);
389
+
390
+ allSuggestions.push(...desc.suggestions, ...fm.suggestions, ...conc.suggestions,
391
+ ...act.suggestions, ...str.suggestions, ...trg.suggestions, ...code.suggestions, ...meta.suggestions);
392
+
393
+ const grade = score >= 85 ? 'A' : score >= 70 ? 'B' : score >= 55 ? 'C' : score >= 40 ? 'D' : 'F';
394
+
395
+ return { name: skill.name, score, grade, dimensions, suggestions: allSuggestions };
396
+ }
397
+
398
+ // ============================================================================
399
+ // OUTPUT
400
+ // ============================================================================
401
+
402
+ function formatResults(scores: SkillScore[], showSuggestions: boolean): string {
403
+ const sep = '='.repeat(120);
404
+ const dimHeaders = 'Desc FM Conc Act Str Trg Code Meta';
405
+ const header = `Skill${' '.repeat(39)}Score Grade ${dimHeaders}`;
406
+ const divider = '-'.repeat(120);
407
+
408
+ let out = `\n${sep}\nSKILL QUALITY SCORER\n${scores.length} AQE skills analyzed\n${sep}\n${header}\n${divider}\n`;
409
+
410
+ for (const s of scores) {
411
+ const d = s.dimensions;
412
+ const name = s.name.length > 42 ? s.name.slice(0, 39) + '...' : s.name.padEnd(42);
413
+ out += `${name} ${String(s.score).padStart(3)} ${s.grade} ${String(d.description).padStart(2)} ${String(d.frontmatter).padStart(2)} ${String(d.conciseness).padStart(2)} ${String(d.actionability).padStart(2)} ${String(d.structure).padStart(2)} ${String(d.triggers).padStart(2)} ${String(d.codeExamples).padStart(2)} ${String(d.metadata).padStart(2)}\n`;
414
+ }
415
+
416
+ // Summary
417
+ const grades = { A: 0, B: 0, C: 0, D: 0, F: 0 };
418
+ for (const s of scores) grades[s.grade as keyof typeof grades]++;
419
+ const avg = Math.round(scores.reduce((sum, s) => sum + s.score, 0) / scores.length);
420
+ const sorted = [...scores].sort((a, b) => a.score - b.score);
421
+ const median = sorted.length % 2 === 0
422
+ ? Math.round((sorted[sorted.length / 2 - 1].score + sorted[sorted.length / 2].score) / 2)
423
+ : sorted[Math.floor(sorted.length / 2)].score;
424
+
425
+ out += `${sep}\nSUMMARY: A=${grades.A} B=${grades.B} C=${grades.C} D=${grades.D} F=${grades.F} | avg=${avg} median=${median}\n${sep}\n`;
426
+
427
+ if (showSuggestions) {
428
+ out += '\nIMPROVEMENT SUGGESTIONS:\n';
429
+ for (const s of scores) {
430
+ if (s.suggestions.length === 0) continue;
431
+ out += `\n ${s.name} (${s.grade}, ${s.score}):\n`;
432
+ for (const sug of s.suggestions) {
433
+ out += ` - ${sug}\n`;
434
+ }
435
+ }
436
+ }
437
+
438
+ return out;
439
+ }
440
+
441
+ // ============================================================================
442
+ // MAIN
443
+ // ============================================================================
444
+
445
+ function main() {
446
+ const args = process.argv.slice(2);
447
+ let filterSkill: string | undefined;
448
+ let showSuggestions = false;
449
+ let jsonOutput = false;
450
+ let minScore: number | undefined;
451
+
452
+ for (let i = 0; i < args.length; i++) {
453
+ switch (args[i]) {
454
+ case '--skill': filterSkill = args[++i]; break;
455
+ case '--suggestions': showSuggestions = true; break;
456
+ case '--json': jsonOutput = true; break;
457
+ case '--min-score': minScore = parseInt(args[++i], 10); break;
458
+ case '--help': case '-h':
459
+ console.log(`
460
+ Skill Quality Scorer
461
+
462
+ Usage:
463
+ npx tsx scripts/score-skill-quality.ts [options]
464
+
465
+ Options:
466
+ --skill <name> Score a single skill
467
+ --suggestions Show improvement suggestions
468
+ --json Write results to scripts/skill-quality-scores.json
469
+ --min-score <n> Only show skills scoring >= n
470
+ --help Show this help
471
+ `);
472
+ process.exit(0);
473
+ }
474
+ }
475
+
476
+ const projectRoot = getProjectRoot();
477
+ const skills = discoverSkills(projectRoot, filterSkill);
478
+
479
+ if (skills.length === 0) {
480
+ console.error(filterSkill ? `Skill not found: ${filterSkill}` : 'No skills found');
481
+ process.exit(1);
482
+ }
483
+
484
+ let scores = skills.map(scoreSkill);
485
+ scores.sort((a, b) => b.score - a.score);
486
+
487
+ if (minScore !== undefined) {
488
+ scores = scores.filter(s => s.score >= minScore);
489
+ }
490
+
491
+ console.log(formatResults(scores, showSuggestions));
492
+
493
+ if (jsonOutput) {
494
+ const jsonPath = join(projectRoot, 'scripts', 'skill-quality-scores.json');
495
+ const data = {
496
+ generatedAt: new Date().toISOString(),
497
+ skillCount: scores.length,
498
+ scores: scores.map(s => ({
499
+ name: s.name,
500
+ score: s.score,
501
+ grade: s.grade,
502
+ dimensions: s.dimensions,
503
+ suggestions: s.suggestions,
504
+ })),
505
+ };
506
+ writeFileSync(jsonPath, JSON.stringify(data, null, 2));
507
+ console.log(`JSON results written to ${jsonPath}`);
508
+ }
509
+ }
510
+
511
+ main();
package/v3/CHANGELOG.md CHANGED
@@ -5,6 +5,50 @@ All notable changes to Agentic QE will be documented in this file.
5
5
  The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
6
6
  and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7
7
 
8
+ ## [3.6.11] - 2026-02-18
9
+
10
+ ### Added
11
+
12
+ - **Smart settings merge for `aqe init --auto`** — Init now detects existing AQE hooks and updates them in-place instead of duplicating. Non-AQE user hooks are preserved. Both modular and wizard init paths share the same merge logic.
13
+ - **Multi-language file discovery** — New shared `file-discovery.ts` utility supports 12+ languages (TypeScript, Python, Go, Rust, Java, Kotlin, Ruby, C#, PHP, Swift, C/C++, Scala). Used by all CLI commands.
14
+ - **Accuracy indicator on metrics** (#281) — `ProjectMetrics` now includes an `accuracy` field (`accurate` | `approximate`) so consumers know whether LOC/test counts come from real file analysis or estimation.
15
+
16
+ ### Fixed
17
+
18
+ - **MCP server crash on malformed requests** (#274) — Added try/catch safety wrapper on `onRequest` handler. Malformed tool calls now return structured errors instead of killing the MCP connection.
19
+ - **`code_index` hardcoded to JS/TS only** (#275) — Replaced inline TypeScript-only file walkers in `code`, `coverage`, `security`, and `test` CLI commands with the shared multi-language discovery utility.
20
+ - **`test_generate_enhanced` temp file extension** (#274) — Temp file now uses the correct extension based on the `language` parameter instead of always `.ts`.
21
+ - **Coverage handler returned synthetic file paths** (#276) — Removed fabricated `src/module0.ts` placeholders from coverage results. Only real file paths are returned.
22
+ - **Model router fallback score went negative** (#278) — Fixed `fallbackToLLM: -1` with `Math.max(0, ...)`. Improved complexity scoring with proper tier thresholds and domain-specific floors.
23
+ - **HNSW index panic on dimension mismatch** (#277) — `validateVector()` now resizes vectors (averaging to shrink, zero-padding to grow) instead of throwing on 768→128 mismatch.
24
+ - **LOC counter accuracy for node-native fallback** (#281) — Expanded `excludeDirs` with Python, Java, and framework directories. Added max file size guard (2MB), depth limit, and dot-directory exclusion. Renamed source from `'fallback'` to `'node-native'` since it reads real files.
25
+ - **Init wizard missing `--json` flags on hooks** — Legacy wizard path hooks now include `--json` flags matching the working configuration.
26
+ - **MCP server name mismatch** — Changed from `'aqe'` to `'agentic-qe'` with migration for old entries.
27
+
28
+ ### Changed
29
+
30
+ - **Cross-language security scanning** — `task-executor.ts` now applies regex-based security patterns (hardcoded secrets, SQL injection, CORS misconfig) to non-JS/TS files during `test_generate_enhanced`.
31
+ - **Model routing enrichment** — `task_orchestrate` and `model_route` MCP handlers now infer domain and criticality from task descriptions for better routing decisions.
32
+
33
+ ## [3.6.10] - 2026-02-18
34
+
35
+ ### Added
36
+
37
+ - **QCSD Production Telemetry Swarm** (#271) — 5th and final QCSD lifecycle phase. Collects DORA metrics, deployment frequency, change failure rate, and MTTR via GitHub API. Includes auto-trigger workflow after npm publish and cross-phase memory hooks (Loop 5: CI/CD → Production).
38
+ - **Eval-driven development workflow** — New `scripts/eval-driven-workflow.ts` for creating, running, and iterating on skill evaluations as the primary development loop.
39
+ - **Deterministic skill quality scorer** — `scripts/score-skill-quality.ts` computes reproducible quality scores for skills based on structure, coverage, and eval results.
40
+ - **Skill conflict detector** — `scripts/detect-skill-conflicts.ts` identifies overlapping skill scopes and trigger conflicts across the fleet.
41
+ - **Eval runner P1 grading improvements** — Negative control test cases (inverted pass logic for "skill should decline" tests), finding count enforcement, schema validation, weighted grading rubric (completeness/accuracy/actionability), and adaptive rubric with dynamic keyword extraction.
42
+ - **KG-assisted test generation** — Knowledge Graph integration into test generators with 276x faster HNSW vector loading. All generators (Jest/Vitest, Mocha, Pytest) now emit mock declarations from KG dependency analysis.
43
+
44
+ ### Fixed
45
+
46
+ - **Self-learning hooks not recording from CLI** — Hook interactions from CLI commands were silently dropped. Now all CLI hook interactions record learning data.
47
+
48
+ ### Security
49
+
50
+ - **Dependency updates** (#272) — Bumped `tar` 7.5.7→7.5.9 (symlink path traversal fix) and `ajv` 8.17.1→8.18.0 (CVE-2025-69873 ReDoS mitigation).
51
+
8
52
  ## [3.6.9] - 2026-02-17
9
53
 
10
54
  ### Fixed
@@ -24,8 +24,8 @@ Read the complete diff and PR description. Do not skim — read every changed fi
24
24
 
25
25
  ### 2. Scope Check
26
26
  - Only analyze AQE/QE skills (NOT Claude Flow platform skills)
27
- - Platform skills to EXCLUDE: v3-*, flow-nexus-*, agentdb-*, reasoningbank-*, swarm-advanced, swarm-orchestration
28
- - If the PR touches skills, verify the count/scope matches expectations (~63 AQE skills)
27
+ - Platform skills to EXCLUDE: v3-*, flow-nexus-*, agentdb-*, reasoningbank-*, swarm-*, github-*, hive-mind-advanced, hooks-automation, iterative-loop, stream-chain, skill-builder, sparc-methodology, pair-programming, release, debug-loop, aqe-v2-v3-migration
28
+ - If the PR touches skills, verify the count/scope matches expectations (~75 AQE skills)
29
29
  - Flag any platform skill changes that may have leaked into an AQE-focused PR
30
30
 
31
31
  ### 3. Summarize Changes