agentic-qe 3.6.6 → 3.6.8
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/skills/skills-manifest.json +1 -1
- package/package.json +3 -1
- package/scripts/benchmark-aqe-baseline.ts +614 -0
- package/v3/CHANGELOG.md +41 -0
- package/v3/dist/cli/bundle.js +970 -350
- package/v3/dist/cli/commands/hooks.d.ts.map +1 -1
- package/v3/dist/cli/commands/hooks.js +172 -31
- package/v3/dist/cli/commands/hooks.js.map +1 -1
- package/v3/dist/coordination/dynamic-scaling/dynamic-scaler.d.ts +21 -0
- package/v3/dist/coordination/dynamic-scaling/dynamic-scaler.d.ts.map +1 -1
- package/v3/dist/coordination/dynamic-scaling/dynamic-scaler.js +76 -0
- package/v3/dist/coordination/dynamic-scaling/dynamic-scaler.js.map +1 -1
- package/v3/dist/coordination/mincut/mincut-persistence.d.ts +4 -0
- package/v3/dist/coordination/mincut/mincut-persistence.d.ts.map +1 -1
- package/v3/dist/coordination/mincut/mincut-persistence.js +30 -0
- package/v3/dist/coordination/mincut/mincut-persistence.js.map +1 -1
- package/v3/dist/coordination/mincut/queen-integration.d.ts.map +1 -1
- package/v3/dist/coordination/mincut/queen-integration.js +7 -2
- package/v3/dist/coordination/mincut/queen-integration.js.map +1 -1
- package/v3/dist/coordination/mincut/time-crystal.d.ts +22 -0
- package/v3/dist/coordination/mincut/time-crystal.d.ts.map +1 -1
- package/v3/dist/coordination/mincut/time-crystal.js +95 -0
- package/v3/dist/coordination/mincut/time-crystal.js.map +1 -1
- package/v3/dist/domains/code-intelligence/coordinator.d.ts.map +1 -1
- package/v3/dist/domains/code-intelligence/coordinator.js +6 -3
- package/v3/dist/domains/code-intelligence/coordinator.js.map +1 -1
- package/v3/dist/domains/code-intelligence/services/knowledge-graph.d.ts.map +1 -1
- package/v3/dist/domains/code-intelligence/services/knowledge-graph.js +19 -34
- package/v3/dist/domains/code-intelligence/services/knowledge-graph.js.map +1 -1
- package/v3/dist/domains/coverage-analysis/services/hnsw-index.d.ts.map +1 -1
- package/v3/dist/domains/coverage-analysis/services/hnsw-index.js +5 -3
- package/v3/dist/domains/coverage-analysis/services/hnsw-index.js.map +1 -1
- package/v3/dist/domains/security-compliance/services/scanners/sast-scanner.d.ts.map +1 -1
- package/v3/dist/domains/security-compliance/services/scanners/sast-scanner.js +7 -3
- package/v3/dist/domains/security-compliance/services/scanners/sast-scanner.js.map +1 -1
- package/v3/dist/early-exit/early-exit-controller.d.ts +20 -0
- package/v3/dist/early-exit/early-exit-controller.d.ts.map +1 -1
- package/v3/dist/early-exit/early-exit-controller.js +72 -0
- package/v3/dist/early-exit/early-exit-controller.js.map +1 -1
- package/v3/dist/feedback/coverage-learner.d.ts +19 -0
- package/v3/dist/feedback/coverage-learner.d.ts.map +1 -1
- package/v3/dist/feedback/coverage-learner.js +134 -0
- package/v3/dist/feedback/coverage-learner.js.map +1 -1
- package/v3/dist/feedback/feedback-loop.d.ts +10 -1
- package/v3/dist/feedback/feedback-loop.d.ts.map +1 -1
- package/v3/dist/feedback/feedback-loop.js +20 -1
- package/v3/dist/feedback/feedback-loop.js.map +1 -1
- package/v3/dist/feedback/index.d.ts +1 -1
- package/v3/dist/feedback/index.d.ts.map +1 -1
- package/v3/dist/feedback/index.js +1 -1
- package/v3/dist/feedback/index.js.map +1 -1
- package/v3/dist/feedback/test-outcome-tracker.d.ts +19 -0
- package/v3/dist/feedback/test-outcome-tracker.d.ts.map +1 -1
- package/v3/dist/feedback/test-outcome-tracker.js +114 -0
- package/v3/dist/feedback/test-outcome-tracker.js.map +1 -1
- package/v3/dist/governance/compliance-reporter.d.ts +13 -0
- package/v3/dist/governance/compliance-reporter.d.ts.map +1 -1
- package/v3/dist/governance/compliance-reporter.js +63 -0
- package/v3/dist/governance/compliance-reporter.js.map +1 -1
- package/v3/dist/governance/continue-gate-integration.d.ts +8 -0
- package/v3/dist/governance/continue-gate-integration.d.ts.map +1 -1
- package/v3/dist/governance/continue-gate-integration.js +50 -2
- package/v3/dist/governance/continue-gate-integration.js.map +1 -1
- package/v3/dist/governance/evolution-pipeline-integration.d.ts +13 -0
- package/v3/dist/governance/evolution-pipeline-integration.d.ts.map +1 -1
- package/v3/dist/governance/evolution-pipeline-integration.js +53 -0
- package/v3/dist/governance/evolution-pipeline-integration.js.map +1 -1
- package/v3/dist/index.d.ts +1 -1
- package/v3/dist/index.d.ts.map +1 -1
- package/v3/dist/index.js +1 -1
- package/v3/dist/index.js.map +1 -1
- package/v3/dist/init/migration/data-migrator.d.ts.map +1 -1
- package/v3/dist/init/migration/data-migrator.js +5 -6
- package/v3/dist/init/migration/data-migrator.js.map +1 -1
- package/v3/dist/init/phases/07-hooks.d.ts.map +1 -1
- package/v3/dist/init/phases/07-hooks.js +26 -5
- package/v3/dist/init/phases/07-hooks.js.map +1 -1
- package/v3/dist/integrations/agentic-flow/reasoning-bank/experience-replay.d.ts +4 -2
- package/v3/dist/integrations/agentic-flow/reasoning-bank/experience-replay.d.ts.map +1 -1
- package/v3/dist/integrations/agentic-flow/reasoning-bank/experience-replay.js +68 -42
- package/v3/dist/integrations/agentic-flow/reasoning-bank/experience-replay.js.map +1 -1
- package/v3/dist/integrations/ruvector/ast-complexity.d.ts +8 -0
- package/v3/dist/integrations/ruvector/ast-complexity.d.ts.map +1 -1
- package/v3/dist/integrations/ruvector/ast-complexity.js +45 -0
- package/v3/dist/integrations/ruvector/ast-complexity.js.map +1 -1
- package/v3/dist/integrations/ruvector/attention-wrapper.d.ts +18 -1
- package/v3/dist/integrations/ruvector/attention-wrapper.d.ts.map +1 -1
- package/v3/dist/integrations/ruvector/attention-wrapper.js +60 -2
- package/v3/dist/integrations/ruvector/attention-wrapper.js.map +1 -1
- package/v3/dist/integrations/ruvector/coverage-router.d.ts +8 -0
- package/v3/dist/integrations/ruvector/coverage-router.d.ts.map +1 -1
- package/v3/dist/integrations/ruvector/coverage-router.js +45 -0
- package/v3/dist/integrations/ruvector/coverage-router.js.map +1 -1
- package/v3/dist/integrations/ruvector/diff-risk-classifier.d.ts +8 -0
- package/v3/dist/integrations/ruvector/diff-risk-classifier.d.ts.map +1 -1
- package/v3/dist/integrations/ruvector/diff-risk-classifier.js +45 -0
- package/v3/dist/integrations/ruvector/diff-risk-classifier.js.map +1 -1
- package/v3/dist/integrations/ruvector/graph-boundaries.d.ts +8 -0
- package/v3/dist/integrations/ruvector/graph-boundaries.d.ts.map +1 -1
- package/v3/dist/integrations/ruvector/graph-boundaries.js +45 -0
- package/v3/dist/integrations/ruvector/graph-boundaries.js.map +1 -1
- package/v3/dist/integrations/ruvector/index.d.ts.map +1 -1
- package/v3/dist/integrations/ruvector/index.js +5 -20
- package/v3/dist/integrations/ruvector/index.js.map +1 -1
- package/v3/dist/integrations/ruvector/persistent-q-router.d.ts +7 -3
- package/v3/dist/integrations/ruvector/persistent-q-router.d.ts.map +1 -1
- package/v3/dist/integrations/ruvector/persistent-q-router.js +7 -3
- package/v3/dist/integrations/ruvector/persistent-q-router.js.map +1 -1
- package/v3/dist/integrations/ruvector/q-learning-router.d.ts +13 -0
- package/v3/dist/integrations/ruvector/q-learning-router.d.ts.map +1 -1
- package/v3/dist/integrations/ruvector/q-learning-router.js +67 -0
- package/v3/dist/integrations/ruvector/q-learning-router.js.map +1 -1
- package/v3/dist/kernel/hybrid-backend.d.ts +1 -0
- package/v3/dist/kernel/hybrid-backend.d.ts.map +1 -1
- package/v3/dist/kernel/hybrid-backend.js +39 -0
- package/v3/dist/kernel/hybrid-backend.js.map +1 -1
- package/v3/dist/kernel/kernel.d.ts.map +1 -1
- package/v3/dist/kernel/kernel.js +8 -0
- package/v3/dist/kernel/kernel.js.map +1 -1
- package/v3/dist/kernel/unified-memory.d.ts +1 -0
- package/v3/dist/kernel/unified-memory.d.ts.map +1 -1
- package/v3/dist/kernel/unified-memory.js +123 -4
- package/v3/dist/kernel/unified-memory.js.map +1 -1
- package/v3/dist/learning/experience-capture-middleware.d.ts.map +1 -1
- package/v3/dist/learning/experience-capture-middleware.js +45 -18
- package/v3/dist/learning/experience-capture-middleware.js.map +1 -1
- package/v3/dist/learning/pattern-store.d.ts.map +1 -1
- package/v3/dist/learning/pattern-store.js +12 -61
- package/v3/dist/learning/pattern-store.js.map +1 -1
- package/v3/dist/learning/qe-reasoning-bank.d.ts.map +1 -1
- package/v3/dist/learning/qe-reasoning-bank.js +15 -2
- package/v3/dist/learning/qe-reasoning-bank.js.map +1 -1
- package/v3/dist/learning/token-tracker.d.ts +23 -0
- package/v3/dist/learning/token-tracker.d.ts.map +1 -1
- package/v3/dist/learning/token-tracker.js +91 -0
- package/v3/dist/learning/token-tracker.js.map +1 -1
- package/v3/dist/mcp/bundle.js +945 -375
- package/v3/dist/mcp/entry.js +21 -1
- package/v3/dist/mcp/entry.js.map +1 -1
- package/v3/dist/routing/routing-feedback.d.ts +21 -0
- package/v3/dist/routing/routing-feedback.d.ts.map +1 -1
- package/v3/dist/routing/routing-feedback.js +95 -0
- package/v3/dist/routing/routing-feedback.js.map +1 -1
- package/v3/dist/shared/sql-safety.d.ts.map +1 -1
- package/v3/dist/shared/sql-safety.js +2 -0
- package/v3/dist/shared/sql-safety.js.map +1 -1
- package/v3/package.json +1 -1
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "agentic-qe",
|
|
3
|
-
"version": "3.6.
|
|
3
|
+
"version": "3.6.8",
|
|
4
4
|
"description": "Agentic Quality Engineering V3 - Domain-Driven Design Architecture with 13 Bounded Contexts, O(log n) coverage analysis, ReasoningBank learning, 59 specialized QE agents, mathematical Coherence verification, deep Claude Flow integration",
|
|
5
5
|
"main": "./v3/dist/index.js",
|
|
6
6
|
"types": "./v3/dist/index.d.ts",
|
|
@@ -86,6 +86,8 @@
|
|
|
86
86
|
"cli-progress": "^3.12.0",
|
|
87
87
|
"commander": "^14.0.1",
|
|
88
88
|
"fast-glob": "^3.3.3",
|
|
89
|
+
"fast-json-patch": "^3.1.1",
|
|
90
|
+
"jose": "^6.1.3",
|
|
89
91
|
"hnswlib-node": "^3.0.0",
|
|
90
92
|
"lodash": "^4.17.23",
|
|
91
93
|
"ora": "^5.4.1",
|
|
@@ -0,0 +1,614 @@
|
|
|
1
|
+
#!/usr/bin/env tsx
|
|
2
|
+
/**
|
|
3
|
+
* AQE Baseline Performance Benchmark
|
|
4
|
+
*
|
|
5
|
+
* Measures current AQE performance characteristics for comparison
|
|
6
|
+
* with an RVF-based implementation. Covers:
|
|
7
|
+
*
|
|
8
|
+
* 1. Cold start (DB open + HNSW init)
|
|
9
|
+
* 2. Pattern insert throughput
|
|
10
|
+
* 3. Vector search latency + recall
|
|
11
|
+
* 4. KV read/write throughput
|
|
12
|
+
* 5. Memory footprint
|
|
13
|
+
* 6. File size
|
|
14
|
+
*
|
|
15
|
+
* Run: npx tsx scripts/benchmark-aqe-baseline.ts
|
|
16
|
+
*/
|
|
17
|
+
|
|
18
|
+
import Database from 'better-sqlite3';
|
|
19
|
+
import * as fs from 'fs';
|
|
20
|
+
import * as path from 'path';
|
|
21
|
+
import * as crypto from 'crypto';
|
|
22
|
+
|
|
23
|
+
// ── Config ──────────────────────────────────────────────────────────
|
|
24
|
+
|
|
25
|
+
const BENCH_DIR = path.join('/tmp', 'aqe-benchmark-' + Date.now());
|
|
26
|
+
const DB_PATH = path.join(BENCH_DIR, 'bench.db');
|
|
27
|
+
const DIMENSIONS = 384; // AQE default
|
|
28
|
+
const VECTOR_COUNTS = [100, 500, 1000, 5000];
|
|
29
|
+
const QUERY_COUNT = 100;
|
|
30
|
+
const KV_COUNT = 1000;
|
|
31
|
+
const SEARCH_K = 10;
|
|
32
|
+
|
|
33
|
+
// ── Helpers ─────────────────────────────────────────────────────────
|
|
34
|
+
|
|
35
|
+
function randomVector(dim: number): Float32Array {
|
|
36
|
+
const v = new Float32Array(dim);
|
|
37
|
+
for (let i = 0; i < dim; i++) v[i] = Math.random() * 2 - 1;
|
|
38
|
+
return v;
|
|
39
|
+
}
|
|
40
|
+
|
|
41
|
+
function normalizeVector(v: Float32Array): Float32Array {
|
|
42
|
+
let norm = 0;
|
|
43
|
+
for (let i = 0; i < v.length; i++) norm += v[i] * v[i];
|
|
44
|
+
norm = Math.sqrt(norm);
|
|
45
|
+
if (norm > 0) for (let i = 0; i < v.length; i++) v[i] /= norm;
|
|
46
|
+
return v;
|
|
47
|
+
}
|
|
48
|
+
|
|
49
|
+
function cosineSimilarity(a: Float32Array, b: Float32Array): number {
|
|
50
|
+
let dot = 0, normA = 0, normB = 0;
|
|
51
|
+
for (let i = 0; i < a.length; i++) {
|
|
52
|
+
dot += a[i] * b[i];
|
|
53
|
+
normA += a[i] * a[i];
|
|
54
|
+
normB += b[i] * b[i];
|
|
55
|
+
}
|
|
56
|
+
return dot / (Math.sqrt(normA) * Math.sqrt(normB));
|
|
57
|
+
}
|
|
58
|
+
|
|
59
|
+
function bruteForceKNN(
|
|
60
|
+
query: Float32Array,
|
|
61
|
+
vectors: Float32Array[],
|
|
62
|
+
k: number
|
|
63
|
+
): { id: number; similarity: number }[] {
|
|
64
|
+
const scored = vectors.map((v, i) => ({ id: i, similarity: cosineSimilarity(query, v) }));
|
|
65
|
+
scored.sort((a, b) => b.similarity - a.similarity);
|
|
66
|
+
return scored.slice(0, k);
|
|
67
|
+
}
|
|
68
|
+
|
|
69
|
+
function percentile(arr: number[], p: number): number {
|
|
70
|
+
const sorted = [...arr].sort((a, b) => a - b);
|
|
71
|
+
const idx = Math.ceil((p / 100) * sorted.length) - 1;
|
|
72
|
+
return sorted[Math.max(0, idx)];
|
|
73
|
+
}
|
|
74
|
+
|
|
75
|
+
function formatMs(ms: number): string {
|
|
76
|
+
if (ms < 0.01) return `${(ms * 1000).toFixed(1)}µs`;
|
|
77
|
+
if (ms < 1) return `${ms.toFixed(2)}ms`;
|
|
78
|
+
if (ms < 1000) return `${ms.toFixed(1)}ms`;
|
|
79
|
+
return `${(ms / 1000).toFixed(2)}s`;
|
|
80
|
+
}
|
|
81
|
+
|
|
82
|
+
function formatBytes(bytes: number): string {
|
|
83
|
+
if (bytes < 1024) return `${bytes}B`;
|
|
84
|
+
if (bytes < 1024 * 1024) return `${(bytes / 1024).toFixed(1)}KB`;
|
|
85
|
+
return `${(bytes / (1024 * 1024)).toFixed(1)}MB`;
|
|
86
|
+
}
|
|
87
|
+
|
|
88
|
+
// ── SQLite Schema (mirrors AQE unified-memory) ─────────────────────
|
|
89
|
+
|
|
90
|
+
function createSchema(db: ReturnType<typeof Database>) {
|
|
91
|
+
db.pragma('journal_mode = WAL');
|
|
92
|
+
db.pragma('mmap_size = 268435456');
|
|
93
|
+
db.pragma('cache_size = -64000');
|
|
94
|
+
db.pragma('busy_timeout = 5000');
|
|
95
|
+
|
|
96
|
+
db.exec(`
|
|
97
|
+
CREATE TABLE IF NOT EXISTS kv_store (
|
|
98
|
+
key TEXT PRIMARY KEY,
|
|
99
|
+
value TEXT NOT NULL,
|
|
100
|
+
namespace TEXT DEFAULT 'default',
|
|
101
|
+
created_at INTEGER DEFAULT (strftime('%s','now')),
|
|
102
|
+
updated_at INTEGER DEFAULT (strftime('%s','now')),
|
|
103
|
+
expires_at INTEGER,
|
|
104
|
+
tags TEXT DEFAULT '[]'
|
|
105
|
+
);
|
|
106
|
+
CREATE INDEX IF NOT EXISTS idx_kv_namespace ON kv_store(namespace);
|
|
107
|
+
|
|
108
|
+
CREATE TABLE IF NOT EXISTS qe_patterns (
|
|
109
|
+
id TEXT PRIMARY KEY,
|
|
110
|
+
domain TEXT NOT NULL,
|
|
111
|
+
type TEXT NOT NULL,
|
|
112
|
+
name TEXT NOT NULL,
|
|
113
|
+
description TEXT,
|
|
114
|
+
embedding BLOB,
|
|
115
|
+
confidence REAL DEFAULT 0.5,
|
|
116
|
+
access_count INTEGER DEFAULT 0,
|
|
117
|
+
tier TEXT DEFAULT 'short-term',
|
|
118
|
+
metadata TEXT DEFAULT '{}',
|
|
119
|
+
created_at INTEGER DEFAULT (strftime('%s','now')),
|
|
120
|
+
updated_at INTEGER DEFAULT (strftime('%s','now'))
|
|
121
|
+
);
|
|
122
|
+
CREATE INDEX IF NOT EXISTS idx_patterns_domain ON qe_patterns(domain);
|
|
123
|
+
CREATE INDEX IF NOT EXISTS idx_patterns_tier ON qe_patterns(tier);
|
|
124
|
+
|
|
125
|
+
CREATE TABLE IF NOT EXISTS rl_q_values (
|
|
126
|
+
state TEXT NOT NULL,
|
|
127
|
+
action TEXT NOT NULL,
|
|
128
|
+
q_value REAL NOT NULL DEFAULT 0.0,
|
|
129
|
+
visit_count INTEGER DEFAULT 0,
|
|
130
|
+
last_reward REAL DEFAULT 0.0,
|
|
131
|
+
updated_at INTEGER DEFAULT (strftime('%s','now')),
|
|
132
|
+
PRIMARY KEY (state, action)
|
|
133
|
+
);
|
|
134
|
+
`);
|
|
135
|
+
}
|
|
136
|
+
|
|
137
|
+
// ── Benchmarks ──────────────────────────────────────────────────────
|
|
138
|
+
|
|
139
|
+
interface BenchmarkResult {
|
|
140
|
+
name: string;
|
|
141
|
+
value: number;
|
|
142
|
+
unit: string;
|
|
143
|
+
formatted: string;
|
|
144
|
+
details?: string;
|
|
145
|
+
}
|
|
146
|
+
|
|
147
|
+
const results: BenchmarkResult[] = [];
|
|
148
|
+
|
|
149
|
+
function record(name: string, value: number, unit: string, details?: string) {
|
|
150
|
+
const formatted = unit === 'ms' ? formatMs(value) :
|
|
151
|
+
unit === 'bytes' ? formatBytes(value) :
|
|
152
|
+
unit === 'MB' ? `${value.toFixed(1)}MB` :
|
|
153
|
+
unit === 'ops/s' ? `${Math.round(value).toLocaleString()} ops/s` :
|
|
154
|
+
unit === '%' ? `${value.toFixed(1)}%` :
|
|
155
|
+
`${value}`;
|
|
156
|
+
results.push({ name, value, unit, formatted, details });
|
|
157
|
+
}
|
|
158
|
+
|
|
159
|
+
async function benchmarkColdStart() {
|
|
160
|
+
console.log('\n── 1. Cold Start ──────────────────────────────────');
|
|
161
|
+
|
|
162
|
+
// Measure DB open + schema creation
|
|
163
|
+
const t0 = performance.now();
|
|
164
|
+
const db = new Database(DB_PATH);
|
|
165
|
+
createSchema(db);
|
|
166
|
+
const t1 = performance.now();
|
|
167
|
+
record('cold_start_db_open', t1 - t0, 'ms', 'DB open + schema + WAL + pragma');
|
|
168
|
+
console.log(` DB open + schema: ${formatMs(t1 - t0)}`);
|
|
169
|
+
|
|
170
|
+
// Measure HNSW init via @ruvector/gnn
|
|
171
|
+
let hnswInitMs = 0;
|
|
172
|
+
try {
|
|
173
|
+
const { default: gnn } = await import('@ruvector/gnn');
|
|
174
|
+
const { HNSWIndex: GNNIndex } = gnn as any;
|
|
175
|
+
|
|
176
|
+
const t2 = performance.now();
|
|
177
|
+
const idx = new GNNIndex({ dimension: DIMENSIONS, metric: 'cosine', M: 16 });
|
|
178
|
+
await idx.initialize?.();
|
|
179
|
+
const t3 = performance.now();
|
|
180
|
+
hnswInitMs = t3 - t2;
|
|
181
|
+
record('cold_start_hnsw_init', hnswInitMs, 'ms', '@ruvector/gnn HNSW init (empty)');
|
|
182
|
+
console.log(` HNSW init (empty): ${formatMs(hnswInitMs)}`);
|
|
183
|
+
} catch {
|
|
184
|
+
// Fallback: hnswlib-node
|
|
185
|
+
try {
|
|
186
|
+
const hnswlib = await import('hnswlib-node');
|
|
187
|
+
const { HierarchicalNSW } = hnswlib.default || hnswlib;
|
|
188
|
+
|
|
189
|
+
const t2 = performance.now();
|
|
190
|
+
const idx = new HierarchicalNSW('cosine', DIMENSIONS);
|
|
191
|
+
idx.initIndex({ maxElements: 10000, m: 16, efConstruction: 200 });
|
|
192
|
+
const t3 = performance.now();
|
|
193
|
+
hnswInitMs = t3 - t2;
|
|
194
|
+
record('cold_start_hnsw_init', hnswInitMs, 'ms', 'hnswlib-node HNSW init (empty)');
|
|
195
|
+
console.log(` HNSW init (empty): ${formatMs(hnswInitMs)}`);
|
|
196
|
+
} catch {
|
|
197
|
+
console.log(' HNSW init: SKIPPED (no HNSW library available)');
|
|
198
|
+
}
|
|
199
|
+
}
|
|
200
|
+
|
|
201
|
+
record('cold_start_total', (t1 - t0) + hnswInitMs, 'ms', 'Total cold start');
|
|
202
|
+
console.log(` Total cold start: ${formatMs((t1 - t0) + hnswInitMs)}`);
|
|
203
|
+
|
|
204
|
+
db.close();
|
|
205
|
+
}
|
|
206
|
+
|
|
207
|
+
async function benchmarkPatternInsert() {
|
|
208
|
+
console.log('\n── 2. Pattern Insert Throughput ────────────────────');
|
|
209
|
+
|
|
210
|
+
for (const count of VECTOR_COUNTS) {
|
|
211
|
+
const db = new Database(DB_PATH);
|
|
212
|
+
createSchema(db);
|
|
213
|
+
|
|
214
|
+
const insertStmt = db.prepare(`
|
|
215
|
+
INSERT OR REPLACE INTO qe_patterns (id, domain, type, name, description, embedding, confidence, tier)
|
|
216
|
+
VALUES (?, ?, ?, ?, ?, ?, ?, ?)
|
|
217
|
+
`);
|
|
218
|
+
|
|
219
|
+
const vectors: Float32Array[] = [];
|
|
220
|
+
const t0 = performance.now();
|
|
221
|
+
|
|
222
|
+
const insertAll = db.transaction(() => {
|
|
223
|
+
for (let i = 0; i < count; i++) {
|
|
224
|
+
const vec = normalizeVector(randomVector(DIMENSIONS));
|
|
225
|
+
vectors.push(vec);
|
|
226
|
+
const embedding = Buffer.from(vec.buffer);
|
|
227
|
+
insertStmt.run(
|
|
228
|
+
`pattern-${i}`,
|
|
229
|
+
'test-generation',
|
|
230
|
+
'test-pattern',
|
|
231
|
+
`Pattern ${i}`,
|
|
232
|
+
`Test pattern ${i} for benchmarking`,
|
|
233
|
+
embedding,
|
|
234
|
+
Math.random(),
|
|
235
|
+
i % 3 === 0 ? 'long-term' : 'short-term'
|
|
236
|
+
);
|
|
237
|
+
}
|
|
238
|
+
});
|
|
239
|
+
insertAll();
|
|
240
|
+
|
|
241
|
+
const t1 = performance.now();
|
|
242
|
+
const totalMs = t1 - t0;
|
|
243
|
+
const opsPerSec = (count / totalMs) * 1000;
|
|
244
|
+
|
|
245
|
+
record(`insert_${count}_total`, totalMs, 'ms');
|
|
246
|
+
record(`insert_${count}_ops`, opsPerSec, 'ops/s');
|
|
247
|
+
console.log(` ${count} patterns: ${formatMs(totalMs)} (${Math.round(opsPerSec).toLocaleString()} ops/s)`);
|
|
248
|
+
|
|
249
|
+
// Measure file size
|
|
250
|
+
const fileSize = fs.statSync(DB_PATH).size;
|
|
251
|
+
record(`filesize_${count}_patterns`, fileSize, 'bytes');
|
|
252
|
+
console.log(` File size: ${formatBytes(fileSize)}`);
|
|
253
|
+
|
|
254
|
+
db.close();
|
|
255
|
+
fs.unlinkSync(DB_PATH);
|
|
256
|
+
// Remove WAL/SHM files too
|
|
257
|
+
try { fs.unlinkSync(DB_PATH + '-wal'); } catch {}
|
|
258
|
+
try { fs.unlinkSync(DB_PATH + '-shm'); } catch {}
|
|
259
|
+
}
|
|
260
|
+
}
|
|
261
|
+
|
|
262
|
+
async function benchmarkVectorSearch() {
|
|
263
|
+
console.log('\n── 3. Vector Search (Brute Force - SQLite BLOBs) ──');
|
|
264
|
+
|
|
265
|
+
for (const count of VECTOR_COUNTS) {
|
|
266
|
+
const db = new Database(DB_PATH);
|
|
267
|
+
createSchema(db);
|
|
268
|
+
|
|
269
|
+
// Insert vectors
|
|
270
|
+
const vectors: Float32Array[] = [];
|
|
271
|
+
const insertStmt = db.prepare(`
|
|
272
|
+
INSERT OR REPLACE INTO qe_patterns (id, domain, type, name, embedding, confidence)
|
|
273
|
+
VALUES (?, ?, ?, ?, ?, ?)
|
|
274
|
+
`);
|
|
275
|
+
|
|
276
|
+
const insertAll = db.transaction(() => {
|
|
277
|
+
for (let i = 0; i < count; i++) {
|
|
278
|
+
const vec = normalizeVector(randomVector(DIMENSIONS));
|
|
279
|
+
vectors.push(vec);
|
|
280
|
+
insertStmt.run(`p-${i}`, 'test-gen', 'pattern', `P${i}`, Buffer.from(vec.buffer), Math.random());
|
|
281
|
+
}
|
|
282
|
+
});
|
|
283
|
+
insertAll();
|
|
284
|
+
|
|
285
|
+
// Benchmark: load all vectors from DB and brute-force search
|
|
286
|
+
const queryVec = normalizeVector(randomVector(DIMENSIONS));
|
|
287
|
+
const latencies: number[] = [];
|
|
288
|
+
|
|
289
|
+
for (let q = 0; q < QUERY_COUNT; q++) {
|
|
290
|
+
const qv = q === 0 ? queryVec : normalizeVector(randomVector(DIMENSIONS));
|
|
291
|
+
|
|
292
|
+
const t0 = performance.now();
|
|
293
|
+
|
|
294
|
+
// Load from DB (simulates AQE's in-memory vector scan)
|
|
295
|
+
const rows = db.prepare('SELECT id, embedding FROM qe_patterns WHERE embedding IS NOT NULL').all() as any[];
|
|
296
|
+
const scored: { id: string; sim: number }[] = [];
|
|
297
|
+
for (const row of rows) {
|
|
298
|
+
const stored = new Float32Array(new Uint8Array(row.embedding).buffer);
|
|
299
|
+
scored.push({ id: row.id, sim: cosineSimilarity(qv, stored) });
|
|
300
|
+
}
|
|
301
|
+
scored.sort((a, b) => b.sim - a.sim);
|
|
302
|
+
const _topK = scored.slice(0, SEARCH_K);
|
|
303
|
+
|
|
304
|
+
const t1 = performance.now();
|
|
305
|
+
latencies.push(t1 - t0);
|
|
306
|
+
}
|
|
307
|
+
|
|
308
|
+
const p50 = percentile(latencies, 50);
|
|
309
|
+
const p95 = percentile(latencies, 95);
|
|
310
|
+
const p99 = percentile(latencies, 99);
|
|
311
|
+
|
|
312
|
+
record(`search_${count}_p50`, p50, 'ms', 'Brute force from SQLite BLOBs');
|
|
313
|
+
record(`search_${count}_p95`, p95, 'ms');
|
|
314
|
+
record(`search_${count}_p99`, p99, 'ms');
|
|
315
|
+
console.log(` ${count} vectors: p50=${formatMs(p50)} p95=${formatMs(p95)} p99=${formatMs(p99)}`);
|
|
316
|
+
|
|
317
|
+
db.close();
|
|
318
|
+
fs.unlinkSync(DB_PATH);
|
|
319
|
+
try { fs.unlinkSync(DB_PATH + '-wal'); } catch {}
|
|
320
|
+
try { fs.unlinkSync(DB_PATH + '-shm'); } catch {}
|
|
321
|
+
}
|
|
322
|
+
}
|
|
323
|
+
|
|
324
|
+
async function benchmarkHNSWSearch() {
|
|
325
|
+
console.log('\n── 4. Vector Search (HNSW - In-Memory) ────────────');
|
|
326
|
+
|
|
327
|
+
let HNSWConstructor: any = null;
|
|
328
|
+
|
|
329
|
+
// Try @ruvector/gnn first
|
|
330
|
+
try {
|
|
331
|
+
const gnn = await import('@ruvector/gnn');
|
|
332
|
+
const mod = (gnn as any).default || gnn;
|
|
333
|
+
if (mod.HNSWIndex) {
|
|
334
|
+
HNSWConstructor = { type: 'gnn', ctor: mod.HNSWIndex };
|
|
335
|
+
}
|
|
336
|
+
} catch {}
|
|
337
|
+
|
|
338
|
+
// Fallback to hnswlib-node
|
|
339
|
+
if (!HNSWConstructor) {
|
|
340
|
+
try {
|
|
341
|
+
const hnswlib = await import('hnswlib-node');
|
|
342
|
+
const mod = (hnswlib as any).default || hnswlib;
|
|
343
|
+
HNSWConstructor = { type: 'hnswlib', ctor: mod.HierarchicalNSW };
|
|
344
|
+
} catch {}
|
|
345
|
+
}
|
|
346
|
+
|
|
347
|
+
if (!HNSWConstructor) {
|
|
348
|
+
console.log(' SKIPPED: No HNSW library available');
|
|
349
|
+
return;
|
|
350
|
+
}
|
|
351
|
+
|
|
352
|
+
console.log(` Using: ${HNSWConstructor.type}`);
|
|
353
|
+
|
|
354
|
+
for (const count of VECTOR_COUNTS) {
|
|
355
|
+
const vectors: Float32Array[] = [];
|
|
356
|
+
for (let i = 0; i < count; i++) {
|
|
357
|
+
vectors.push(normalizeVector(randomVector(DIMENSIONS)));
|
|
358
|
+
}
|
|
359
|
+
|
|
360
|
+
// Build HNSW index
|
|
361
|
+
const t0 = performance.now();
|
|
362
|
+
let index: any;
|
|
363
|
+
|
|
364
|
+
if (HNSWConstructor.type === 'gnn') {
|
|
365
|
+
index = new HNSWConstructor.ctor({ dimension: DIMENSIONS, metric: 'cosine', M: 16 });
|
|
366
|
+
for (let i = 0; i < count; i++) {
|
|
367
|
+
index.addPoint(Array.from(vectors[i]), i);
|
|
368
|
+
}
|
|
369
|
+
} else {
|
|
370
|
+
index = new HNSWConstructor.ctor('cosine', DIMENSIONS);
|
|
371
|
+
index.initIndex({ maxElements: count + 1000, m: 16, efConstruction: 200 });
|
|
372
|
+
for (let i = 0; i < count; i++) {
|
|
373
|
+
index.addPoint(Array.from(vectors[i]), i);
|
|
374
|
+
}
|
|
375
|
+
}
|
|
376
|
+
|
|
377
|
+
const buildMs = performance.now() - t0;
|
|
378
|
+
record(`hnsw_build_${count}`, buildMs, 'ms', `${HNSWConstructor.type} build time`);
|
|
379
|
+
|
|
380
|
+
// Search
|
|
381
|
+
const latencies: number[] = [];
|
|
382
|
+
let totalRecall = 0;
|
|
383
|
+
|
|
384
|
+
if (HNSWConstructor.type === 'hnswlib') {
|
|
385
|
+
index.setEfSearch(100);
|
|
386
|
+
}
|
|
387
|
+
|
|
388
|
+
for (let q = 0; q < Math.min(QUERY_COUNT, count); q++) {
|
|
389
|
+
const queryVec = normalizeVector(randomVector(DIMENSIONS));
|
|
390
|
+
const queryArr = Array.from(queryVec);
|
|
391
|
+
|
|
392
|
+
const t1 = performance.now();
|
|
393
|
+
let hnswResults: number[];
|
|
394
|
+
|
|
395
|
+
if (HNSWConstructor.type === 'gnn') {
|
|
396
|
+
const res = index.searchKNN(queryArr, SEARCH_K);
|
|
397
|
+
hnswResults = res.neighbors || res.ids || [];
|
|
398
|
+
} else {
|
|
399
|
+
const res = index.searchKnn(queryArr, SEARCH_K);
|
|
400
|
+
hnswResults = Array.from(res.neighbors);
|
|
401
|
+
}
|
|
402
|
+
const t2 = performance.now();
|
|
403
|
+
latencies.push(t2 - t1);
|
|
404
|
+
|
|
405
|
+
// Compute recall vs brute force
|
|
406
|
+
const bruteForce = bruteForceKNN(queryVec, vectors, SEARCH_K);
|
|
407
|
+
const bfIds = new Set(bruteForce.map(r => r.id));
|
|
408
|
+
const hits = hnswResults.filter(id => bfIds.has(id)).length;
|
|
409
|
+
totalRecall += hits / SEARCH_K;
|
|
410
|
+
}
|
|
411
|
+
|
|
412
|
+
const searchCount = Math.min(QUERY_COUNT, count);
|
|
413
|
+
const avgRecall = (totalRecall / searchCount) * 100;
|
|
414
|
+
const p50 = percentile(latencies, 50);
|
|
415
|
+
const p95 = percentile(latencies, 95);
|
|
416
|
+
|
|
417
|
+
record(`hnsw_search_${count}_p50`, p50, 'ms');
|
|
418
|
+
record(`hnsw_search_${count}_p95`, p95, 'ms');
|
|
419
|
+
record(`hnsw_search_${count}_recall`, avgRecall, '%');
|
|
420
|
+
|
|
421
|
+
console.log(` ${count} vectors: build=${formatMs(buildMs)} search p50=${formatMs(p50)} p95=${formatMs(p95)} recall@${SEARCH_K}=${avgRecall.toFixed(1)}%`);
|
|
422
|
+
}
|
|
423
|
+
}
|
|
424
|
+
|
|
425
|
+
async function benchmarkKVOps() {
|
|
426
|
+
console.log('\n── 5. KV Store Throughput ──────────────────────────');
|
|
427
|
+
|
|
428
|
+
const db = new Database(DB_PATH);
|
|
429
|
+
createSchema(db);
|
|
430
|
+
|
|
431
|
+
const insertStmt = db.prepare(`
|
|
432
|
+
INSERT OR REPLACE INTO kv_store (key, value, namespace) VALUES (?, ?, ?)
|
|
433
|
+
`);
|
|
434
|
+
const getStmt = db.prepare('SELECT value FROM kv_store WHERE key = ? AND namespace = ?');
|
|
435
|
+
|
|
436
|
+
// Write throughput
|
|
437
|
+
const t0 = performance.now();
|
|
438
|
+
const writeAll = db.transaction(() => {
|
|
439
|
+
for (let i = 0; i < KV_COUNT; i++) {
|
|
440
|
+
insertStmt.run(`key-${i}`, JSON.stringify({ data: `value-${i}`, ts: Date.now() }), 'bench');
|
|
441
|
+
}
|
|
442
|
+
});
|
|
443
|
+
writeAll();
|
|
444
|
+
const writeMs = performance.now() - t0;
|
|
445
|
+
const writeOps = (KV_COUNT / writeMs) * 1000;
|
|
446
|
+
|
|
447
|
+
record('kv_write_total', writeMs, 'ms', `${KV_COUNT} writes`);
|
|
448
|
+
record('kv_write_ops', writeOps, 'ops/s');
|
|
449
|
+
console.log(` Write ${KV_COUNT}: ${formatMs(writeMs)} (${Math.round(writeOps).toLocaleString()} ops/s)`);
|
|
450
|
+
|
|
451
|
+
// Read throughput
|
|
452
|
+
const t1 = performance.now();
|
|
453
|
+
for (let i = 0; i < KV_COUNT; i++) {
|
|
454
|
+
getStmt.get(`key-${i}`, 'bench');
|
|
455
|
+
}
|
|
456
|
+
const readMs = performance.now() - t1;
|
|
457
|
+
const readOps = (KV_COUNT / readMs) * 1000;
|
|
458
|
+
|
|
459
|
+
record('kv_read_total', readMs, 'ms', `${KV_COUNT} reads`);
|
|
460
|
+
record('kv_read_ops', readOps, 'ops/s');
|
|
461
|
+
console.log(` Read ${KV_COUNT}: ${formatMs(readMs)} (${Math.round(readOps).toLocaleString()} ops/s)`);
|
|
462
|
+
|
|
463
|
+
// Random read
|
|
464
|
+
const t2 = performance.now();
|
|
465
|
+
for (let i = 0; i < KV_COUNT; i++) {
|
|
466
|
+
const key = `key-${Math.floor(Math.random() * KV_COUNT)}`;
|
|
467
|
+
getStmt.get(key, 'bench');
|
|
468
|
+
}
|
|
469
|
+
const randReadMs = performance.now() - t2;
|
|
470
|
+
const randReadOps = (KV_COUNT / randReadMs) * 1000;
|
|
471
|
+
|
|
472
|
+
record('kv_random_read_total', randReadMs, 'ms', `${KV_COUNT} random reads`);
|
|
473
|
+
record('kv_random_read_ops', randReadOps, 'ops/s');
|
|
474
|
+
console.log(` Random read ${KV_COUNT}: ${formatMs(randReadMs)} (${Math.round(randReadOps).toLocaleString()} ops/s)`);
|
|
475
|
+
|
|
476
|
+
db.close();
|
|
477
|
+
fs.unlinkSync(DB_PATH);
|
|
478
|
+
try { fs.unlinkSync(DB_PATH + '-wal'); } catch {}
|
|
479
|
+
try { fs.unlinkSync(DB_PATH + '-shm'); } catch {}
|
|
480
|
+
}
|
|
481
|
+
|
|
482
|
+
async function benchmarkMemoryFootprint() {
|
|
483
|
+
console.log('\n── 6. Memory Footprint ────────────────────────────');
|
|
484
|
+
|
|
485
|
+
const baseline = process.memoryUsage();
|
|
486
|
+
|
|
487
|
+
// Load vectors into memory (simulating HNSW index rebuild)
|
|
488
|
+
const vectors: Float32Array[] = [];
|
|
489
|
+
for (let i = 0; i < 5000; i++) {
|
|
490
|
+
vectors.push(normalizeVector(randomVector(DIMENSIONS)));
|
|
491
|
+
}
|
|
492
|
+
|
|
493
|
+
const afterVectors = process.memoryUsage();
|
|
494
|
+
const vectorMB = (afterVectors.heapUsed - baseline.heapUsed) / (1024 * 1024);
|
|
495
|
+
record('memory_5000_vectors', vectorMB, 'MB', `5000 × ${DIMENSIONS}-dim Float32Array in memory`);
|
|
496
|
+
console.log(` 5000 vectors: ${vectorMB.toFixed(1)}MB heap`);
|
|
497
|
+
console.log(` Per vector: ${((vectorMB * 1024 * 1024) / 5000).toFixed(0)} bytes`);
|
|
498
|
+
console.log(` Theoretical: ${(5000 * DIMENSIONS * 4 / (1024 * 1024)).toFixed(1)}MB raw`);
|
|
499
|
+
}
|
|
500
|
+
|
|
501
|
+
async function benchmarkQValueOps() {
|
|
502
|
+
console.log('\n── 7. Q-Value Operations ──────────────────────────');
|
|
503
|
+
|
|
504
|
+
const db = new Database(DB_PATH);
|
|
505
|
+
createSchema(db);
|
|
506
|
+
|
|
507
|
+
const upsertStmt = db.prepare(`
|
|
508
|
+
INSERT INTO rl_q_values (state, action, q_value, visit_count, last_reward)
|
|
509
|
+
VALUES (?, ?, ?, ?, ?)
|
|
510
|
+
ON CONFLICT(state, action) DO UPDATE SET
|
|
511
|
+
q_value = excluded.q_value,
|
|
512
|
+
visit_count = visit_count + 1,
|
|
513
|
+
last_reward = excluded.last_reward,
|
|
514
|
+
updated_at = strftime('%s','now')
|
|
515
|
+
`);
|
|
516
|
+
|
|
517
|
+
const lookupStmt = db.prepare(`
|
|
518
|
+
SELECT action, q_value FROM rl_q_values
|
|
519
|
+
WHERE state = ?
|
|
520
|
+
ORDER BY q_value DESC
|
|
521
|
+
LIMIT 5
|
|
522
|
+
`);
|
|
523
|
+
|
|
524
|
+
// Write Q-values
|
|
525
|
+
const states = 100;
|
|
526
|
+
const actionsPerState = 10;
|
|
527
|
+
const t0 = performance.now();
|
|
528
|
+
const writeAll = db.transaction(() => {
|
|
529
|
+
for (let s = 0; s < states; s++) {
|
|
530
|
+
for (let a = 0; a < actionsPerState; a++) {
|
|
531
|
+
upsertStmt.run(`state-${s}`, `action-${a}`, Math.random(), 1, Math.random());
|
|
532
|
+
}
|
|
533
|
+
}
|
|
534
|
+
});
|
|
535
|
+
writeAll();
|
|
536
|
+
const writeMs = performance.now() - t0;
|
|
537
|
+
console.log(` Write ${states * actionsPerState} Q-values: ${formatMs(writeMs)}`);
|
|
538
|
+
record('qvalue_write_1000', writeMs, 'ms');
|
|
539
|
+
|
|
540
|
+
// Lookup best actions
|
|
541
|
+
const lookupLatencies: number[] = [];
|
|
542
|
+
for (let q = 0; q < 200; q++) {
|
|
543
|
+
const state = `state-${Math.floor(Math.random() * states)}`;
|
|
544
|
+
const t1 = performance.now();
|
|
545
|
+
lookupStmt.all(state);
|
|
546
|
+
lookupLatencies.push(performance.now() - t1);
|
|
547
|
+
}
|
|
548
|
+
|
|
549
|
+
const p50 = percentile(lookupLatencies, 50);
|
|
550
|
+
const p95 = percentile(lookupLatencies, 95);
|
|
551
|
+
record('qvalue_lookup_p50', p50, 'ms');
|
|
552
|
+
record('qvalue_lookup_p95', p95, 'ms');
|
|
553
|
+
console.log(` Lookup best action: p50=${formatMs(p50)} p95=${formatMs(p95)}`);
|
|
554
|
+
|
|
555
|
+
db.close();
|
|
556
|
+
fs.unlinkSync(DB_PATH);
|
|
557
|
+
try { fs.unlinkSync(DB_PATH + '-wal'); } catch {}
|
|
558
|
+
try { fs.unlinkSync(DB_PATH + '-shm'); } catch {}
|
|
559
|
+
}
|
|
560
|
+
|
|
561
|
+
// ── Main ────────────────────────────────────────────────────────────
|
|
562
|
+
|
|
563
|
+
async function main() {
|
|
564
|
+
console.log('╔═══════════════════════════════════════════════════════════╗');
|
|
565
|
+
console.log('║ AQE Baseline Performance Benchmark ║');
|
|
566
|
+
console.log('║ Dimensions: 384 | Metric: cosine ║');
|
|
567
|
+
console.log('╚═══════════════════════════════════════════════════════════╝');
|
|
568
|
+
|
|
569
|
+
console.log(`\n Platform: ${process.platform} ${process.arch}`);
|
|
570
|
+
console.log(` Node: ${process.version}`);
|
|
571
|
+
console.log(` Bench dir: ${BENCH_DIR}`);
|
|
572
|
+
console.log(` Timestamp: ${new Date().toISOString()}`);
|
|
573
|
+
|
|
574
|
+
fs.mkdirSync(BENCH_DIR, { recursive: true });
|
|
575
|
+
|
|
576
|
+
await benchmarkColdStart();
|
|
577
|
+
await benchmarkPatternInsert();
|
|
578
|
+
await benchmarkVectorSearch();
|
|
579
|
+
await benchmarkHNSWSearch();
|
|
580
|
+
await benchmarkKVOps();
|
|
581
|
+
await benchmarkMemoryFootprint();
|
|
582
|
+
await benchmarkQValueOps();
|
|
583
|
+
|
|
584
|
+
// ── Summary table ──
|
|
585
|
+
console.log('\n╔═══════════════════════════════════════════════════════════╗');
|
|
586
|
+
console.log('║ SUMMARY TABLE ║');
|
|
587
|
+
console.log('╠═══════════════════════════════════════════════════════════╣');
|
|
588
|
+
console.log('║ Metric │ Value ║');
|
|
589
|
+
console.log('╟──────────────────────────────────┼───────────────────────╢');
|
|
590
|
+
for (const r of results) {
|
|
591
|
+
const name = r.name.padEnd(32).slice(0, 32);
|
|
592
|
+
const val = r.formatted.padStart(21).slice(-21);
|
|
593
|
+
console.log(`║ ${name} │ ${val} ║`);
|
|
594
|
+
}
|
|
595
|
+
console.log('╚══════════════════════════════════╧═══════════════════════╝');
|
|
596
|
+
|
|
597
|
+
// Write JSON results
|
|
598
|
+
const jsonPath = path.join(BENCH_DIR, 'results.json');
|
|
599
|
+
fs.writeFileSync(jsonPath, JSON.stringify({
|
|
600
|
+
timestamp: new Date().toISOString(),
|
|
601
|
+
platform: `${process.platform} ${process.arch}`,
|
|
602
|
+
node: process.version,
|
|
603
|
+
dimensions: DIMENSIONS,
|
|
604
|
+
results,
|
|
605
|
+
}, null, 2));
|
|
606
|
+
console.log(`\n Results saved to: ${jsonPath}`);
|
|
607
|
+
|
|
608
|
+
// Cleanup
|
|
609
|
+
try {
|
|
610
|
+
fs.rmSync(BENCH_DIR, { recursive: true });
|
|
611
|
+
} catch {}
|
|
612
|
+
}
|
|
613
|
+
|
|
614
|
+
main().catch(console.error);
|
package/v3/CHANGELOG.md
CHANGED
|
@@ -5,6 +5,47 @@ All notable changes to Agentic QE will be documented in this file.
|
|
|
5
5
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
|
6
6
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
7
|
|
|
8
|
+
## [3.6.8] - 2026-02-15
|
|
9
|
+
|
|
10
|
+
### Fixed
|
|
11
|
+
|
|
12
|
+
- **MCP tools failing with "Fleet not initialized"** — Every MCP tool call required a prior `fleet_init` call, which Claude Code doesn't do automatically. Fleet now auto-initializes on MCP server startup with default configuration (hierarchical topology, hybrid memory).
|
|
13
|
+
- **Experience persistence gap** — Learning system (`ExperienceReplay`) read from a separate `experiences` table while the capture middleware wrote to `captured_experiences`, so new experiences were never visible to the learning system. Unified to use `captured_experiences` as the single source of truth.
|
|
14
|
+
- **Split-brain database from relative dbPath** — `UnifiedMemoryManager` accepted relative paths (e.g. `.agentic-qe/memory.db`) which could create duplicate databases when CWD differed from project root. Now resolves relative paths through `findProjectRoot()`.
|
|
15
|
+
- **Missing runtime dependency `fast-json-patch`** (#262) — Package was marked as external in the MCP esbuild bundle but not listed in root `package.json`, causing `ERR_MODULE_NOT_FOUND` on fresh installs. Added `fast-json-patch` and `jose` to root dependencies.
|
|
16
|
+
|
|
17
|
+
### Changed
|
|
18
|
+
|
|
19
|
+
- **Data migrator** — v2-to-v3 migration now writes to `captured_experiences` table instead of the removed `experiences` table.
|
|
20
|
+
|
|
21
|
+
## [3.6.7] - 2026-02-14
|
|
22
|
+
|
|
23
|
+
### Fixed
|
|
24
|
+
|
|
25
|
+
- **Database corruption from concurrent VACUUM** — Automated VACUUM in HybridBackend ran every 10 minutes while 3+ hook processes held concurrent SQLite connections, causing intermittent "database disk image is malformed" errors. Removed automated VACUUM entirely; SQLite WAL mode handles space reclamation safely without it.
|
|
26
|
+
- **Dual-database split-brain** — `findProjectRoot()` found the first `.agentic-qe` directory walking up, which resolved to `v3/.agentic-qe` instead of root. Changed to find the topmost `.agentic-qe`, preventing shadow databases from forming.
|
|
27
|
+
- **Vector table bloat (124MB)** — Every HNSW index insert wrote vectors to SQLite's `vectors` table via `storeVector()`, accumulating 30K+ rows (124MB). Removed persistence since the in-memory HNSW index is the source of truth for search.
|
|
28
|
+
- **Cross-domain transfer re-running every session** — The seed flag (`reasoning-bank:cross-domain-seeded`) was set after `seedCrossDomainPatterns()`, but the 10-second hooks init timeout could fire mid-transfer, preventing the flag from ever being written. Moved the flag write before the transfer so it persists even on timeout.
|
|
29
|
+
- **Stop hook JSON validation error** — Stop hook output included `hookSpecificOutput` with `hookEventName: "Stop"`, but Claude Code's hook schema only accepts this field for PreToolUse, UserPromptSubmit, and PostToolUse events. Simplified to plain JSON output.
|
|
30
|
+
- **Hypergraph database path** — Code-intelligence coordinator used a relative path `.agentic-qe/hypergraph.db`, which created a shadow directory when run from `v3/`. Fixed to use `findProjectRoot()`.
|
|
31
|
+
- **Seed flag namespace collision** — Graph-boundaries kv_store writes used wrong namespace, causing cross-domain seed flag to not persist correctly.
|
|
32
|
+
|
|
33
|
+
### Added
|
|
34
|
+
|
|
35
|
+
- **File guardian hook** — Pre-edit hook that protects critical files (database files, WAL files, config) from accidental modification.
|
|
36
|
+
- **Command bouncer hook** — Pre-command hook that blocks dangerous shell commands (rm -rf on databases, VACUUM on live DB) with configurable rules.
|
|
37
|
+
- **Context injection hook** — Injects relevant context (recent patterns, domain knowledge) into prompts via UserPromptSubmit hooks.
|
|
38
|
+
- **Tier 2/3 kv_store persistence** — Learning experiences and routing feedback now persist to kv_store for cross-session retention.
|
|
39
|
+
- **Coverage learner feedback loop** — Tracks test outcomes and coverage trends to improve future test generation.
|
|
40
|
+
- **Token usage tracker** — Records token consumption per operation for cost analysis and optimization.
|
|
41
|
+
- **Q-learning router** — Reinforcement learning-based routing that improves agent selection over time.
|
|
42
|
+
- **RuVector WASM integrations** — AST complexity analyzer, coverage router, diff-risk classifier, and graph boundaries modules for fast local inference.
|
|
43
|
+
|
|
44
|
+
### Changed
|
|
45
|
+
|
|
46
|
+
- **Knowledge graph cache-only mode** — Code-intelligence knowledge graph no longer persists to PostgreSQL by default, preventing errors when PG is unavailable. Uses in-memory graph with SQLite-backed concept nodes/edges.
|
|
47
|
+
- **Pattern store simplified** — Removed unused code paths and reduced complexity in pattern persistence layer.
|
|
48
|
+
|
|
8
49
|
## [3.6.6] - 2026-02-13
|
|
9
50
|
|
|
10
51
|
### Fixed
|