capman 0.5.4 → 0.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +46 -0
- package/CODEBASE.md +111 -66
- package/README.md +45 -4
- package/bin/lib/cmd-generate.js +200 -40
- package/bin/lib/cmd-help.js +3 -0
- package/dist/cjs/cache.d.ts.map +1 -1
- package/dist/cjs/cache.js +22 -5
- package/dist/cjs/cache.js.map +1 -1
- package/dist/cjs/engine.d.ts +53 -1
- package/dist/cjs/engine.d.ts.map +1 -1
- package/dist/cjs/engine.js +252 -17
- package/dist/cjs/engine.js.map +1 -1
- package/dist/cjs/generator.d.ts.map +1 -1
- package/dist/cjs/generator.js +7 -1
- package/dist/cjs/generator.js.map +1 -1
- package/dist/cjs/index.d.ts +1 -0
- package/dist/cjs/index.d.ts.map +1 -1
- package/dist/cjs/index.js +3 -1
- package/dist/cjs/index.js.map +1 -1
- package/dist/cjs/learning.d.ts.map +1 -1
- package/dist/cjs/learning.js +51 -30
- package/dist/cjs/learning.js.map +1 -1
- package/dist/cjs/matcher.d.ts +69 -9
- package/dist/cjs/matcher.d.ts.map +1 -1
- package/dist/cjs/matcher.js +328 -43
- package/dist/cjs/matcher.js.map +1 -1
- package/dist/cjs/parser.d.ts.map +1 -1
- package/dist/cjs/parser.js +15 -8
- package/dist/cjs/parser.js.map +1 -1
- package/dist/cjs/resolver.d.ts +1 -0
- package/dist/cjs/resolver.d.ts.map +1 -1
- package/dist/cjs/resolver.js +16 -5
- package/dist/cjs/resolver.js.map +1 -1
- package/dist/cjs/schema.d.ts +64 -46
- package/dist/cjs/schema.d.ts.map +1 -1
- package/dist/cjs/schema.js +2 -1
- package/dist/cjs/schema.js.map +1 -1
- package/dist/cjs/types.d.ts +8 -2
- package/dist/cjs/types.d.ts.map +1 -1
- package/dist/cjs/version.d.ts +1 -1
- package/dist/cjs/version.js +1 -1
- package/dist/esm/cache.js +22 -5
- package/dist/esm/engine.d.ts +53 -1
- package/dist/esm/engine.js +255 -20
- package/dist/esm/generator.js +7 -1
- package/dist/esm/index.d.ts +1 -0
- package/dist/esm/index.js +1 -0
- package/dist/esm/learning.js +52 -31
- package/dist/esm/matcher.d.ts +69 -9
- package/dist/esm/matcher.js +321 -42
- package/dist/esm/parser.js +15 -8
- package/dist/esm/resolver.d.ts +1 -0
- package/dist/esm/resolver.js +16 -6
- package/dist/esm/schema.d.ts +64 -46
- package/dist/esm/schema.js +2 -1
- package/dist/esm/types.d.ts +8 -2
- package/dist/esm/version.d.ts +1 -1
- package/dist/esm/version.js +1 -1
- package/package.json +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,52 @@ All notable changes to capman are documented here.
|
|
|
4
4
|
|
|
5
5
|
---
|
|
6
6
|
|
|
7
|
+
## [0.5.5] — 2026-05-03
|
|
8
|
+
### Fixed
|
|
9
|
+
|
|
10
|
+
**Critical:**
|
|
11
|
+
- LLM rate-limit slot now refunded on failure — `recordLLMFailure()` decrements `llmCallsThisMinute`. Previously burned slot was never returned, causing premature rate exhaustion under sustained errors and silent degradation to keyword-only matching
|
|
12
|
+
- `maxLLMCallsPerMinute: 0` now returns a clear `'LLM disabled'` message instead of arithmetic confusion
|
|
13
|
+
- Anthropic/OpenAI/OpenRouter error responses now read via `res.text()` before `res.ok` check — `res.json()` on a non-200 with malformed body was masking the real API error with a parse exception
|
|
14
|
+
- Empty string API keys (`""`) now correctly rejected — `??` operator passed empty strings through as truthy; replaced with `.trim() ||` across all three provider env vars
|
|
15
|
+
|
|
16
|
+
**High:**
|
|
17
|
+
- Privacy trace step now correctly shows `'fail'` when auth would block — previously always pushed `status: 'pass'` regardless of auth state, making the trace misleading for unauthenticated requests
|
|
18
|
+
- `preBoostMatchResult` is now an explicit shallow copy (`{ ...matchResult, candidates: matchResult.candidates.slice() }`) instead of a reference alias — prevents accidental mutation corrupting the pre-boost snapshot
|
|
19
|
+
- `'manage'` keyword no longer causes false admin classification in OpenAPI parser — replaced substring check with word-boundary regex `\b(admin|administrator|backoffice|back-office|internal|superuser)\b`. Operations like `manageWishlist`, `fileManager` are no longer misclassified as admin
|
|
20
|
+
- `'context'` and `'static'` param sources removed from schema and types — were schema-valid but silently dropped at runtime, injecting `null` into URLs with no error. Schema now only accepts `'user_query'` and `'session'`. Parser updated to map Swagger 2.x body/formData params to `'user_query'`
|
|
21
|
+
- Swagger 2.0 `scheme` no longer hardcoded to `https` — respects declared `schemes` array, prefers `https` over `http` when both present
|
|
22
|
+
- `process.exit()` removed from library signal handlers — `FileLearningStore` SIGTERM/SIGINT handlers were calling `process.exit(0)`, hijacking application shutdown sequence. Handlers now flush only. Handler references stored as module-level variables and cleaned up via `unregisterExitHandlers()` when `activeStores` is empty after `destroy()`
|
|
23
|
+
- LLM `extracted_params` validated against declared capability params — previously cast directly to `Record<string, string | null>`. Nested objects now produce `null` instead of `"[object Object]"` in URLs. Unknown keys dropped. Numbers and booleans coerced to string
|
|
24
|
+
|
|
25
|
+
**Medium:**
|
|
26
|
+
- `ENOENT` vs corruption now distinguished in `FileCache` and `FileLearningStore` load — bare `catch {}` replaced with code check; only non-ENOENT errors emit a warning
|
|
27
|
+
- `loadPromise` now resets on rejection — previously a failed load cached the rejected promise permanently, causing all subsequent calls to fail forever
|
|
28
|
+
- `validateNavParam` allowlist aligned with `validateApiPathParam` — dots, colons, `@` now permitted. Allows deep links (`myapp://path`), domain-qualified values (`auth.tokens`), versioned routes (`v1:resource`)
|
|
29
|
+
- `manifest.app` sanitized before LLM prompt injection — strips newlines, tabs, and control characters that could break prompt structure
|
|
30
|
+
- `buildCacheKey` now used on cache write — engine stores results under both `normalizeQuery` key (exact phrasing) and `buildCacheKey` (capability + params semantic key). Differently-phrased queries resolving to the same capability share cache entries
|
|
31
|
+
- Description scoring normalized against `Math.min(descWords.length, 10)` — previously `overlap / totalWords` penalized rich documentation. Long descriptions no longer score lower than short ones for the same keyword overlap
|
|
32
|
+
- Silent `baseUrl` placeholder in parser now emits a `logger.warn` — generated configs with no server URL were silently broken at runtime
|
|
33
|
+
- `writeManifest` now validates output path stays within working directory — public API had no guard, only the CLI wrapper did
|
|
34
|
+
- `scoreCapability` converted from O(n²) to O(n) — `qWords.includes(w)` inside loops replaced with a `Set` built once per `match()` call. At 500 capabilities × 30 query words: ~300,000 ops → ~15,000 ops
|
|
35
|
+
|
|
36
|
+
**Low / Cosmetic:**
|
|
37
|
+
- `export class CapmanEngine` indentation fixed — extra leading 2-space indent removed
|
|
38
|
+
- Trailing whitespace artifact removed from `matcher.ts` line 89
|
|
39
|
+
- Rate-limiter call sites in `_runMatch()` now have comment noting shared quota between `ask()` and `explain()`
|
|
40
|
+
- Learning boost skips high-confidence LLM matches — `applyBoostToMatchResult()` now accepts `resolvedVia` and returns early when `resolvedVia === 'llm' && confidence > 80`. Avoids learning signal incorrectly overriding strong LLM decisions
|
|
41
|
+
- `matchWithLLM()` sanitizes capability `description` and `examples` fields via `sanitizeForPrompt()` before LLM prompt injection — strips newlines, delimiters, leading braces. Defence-in-depth on top of caller-side sanitization
|
|
42
|
+
- `resolveApi` JSDoc updated — documents that partial failure scenario does not surface which endpoints succeeded, and notes planned `partialSuccess` field in future version
|
|
43
|
+
- `extractParams` JSDoc updated — documents fallback word extraction limitation and future `pattern` field plan
|
|
44
|
+
- `matchWithLLM` security JSDoc updated — notes current single-string prompt limitation for system/user message separation
|
|
45
|
+
|
|
46
|
+
### Tests
|
|
47
|
+
- 99 tests passing (up from 97)
|
|
48
|
+
- Added: `'manage'` false admin classification test
|
|
49
|
+
- Added: nav params with dots/colons allowed test
|
|
50
|
+
|
|
51
|
+
---
|
|
52
|
+
|
|
7
53
|
## [0.5.4] — 2026-04-29
|
|
8
54
|
### Added
|
|
9
55
|
- `engine.loadManifest(manifest)` — hot-reloads the manifest without creating a new engine instance. Preserves cache, learning history, and rate limiter state. Clears cache automatically since cached results from the old manifest are no longer valid
|
package/CODEBASE.md
CHANGED
|
@@ -28,6 +28,7 @@ All TypeScript types and interfaces. No logic — pure declarations.
|
|
|
28
28
|
|
|
29
29
|
Key exports:
|
|
30
30
|
- `Capability`, `CapabilityParam`, `Manifest`, `CapmanConfig` — core data shapes
|
|
31
|
+
- `CapabilityParam.source` — `'user_query' | 'session'` only. `context` and `static` removed in v0.5.5 — were schema-valid but silently dropped at runtime
|
|
31
32
|
- `MatchResult` — what `match()` returns, including `candidates: MatchCandidate[]`
|
|
32
33
|
- `MatchCandidate` — `{ capabilityId, score, matched }` — all scored candidates
|
|
33
34
|
- `ResolveResult`, `ApiCallResult` — what `resolve()` returns, with `status` and `data`
|
|
@@ -48,8 +49,12 @@ Key exports:
|
|
|
48
49
|
|
|
49
50
|
Notable rules:
|
|
50
51
|
- `id` must match `/^[a-z0-9_]+$/` — snake_case only
|
|
51
|
-
- `description` minimum 10 characters
|
|
52
|
+
- `description` minimum 10 characters, maximum 500 characters
|
|
53
|
+
- `examples` each entry maximum 200 characters
|
|
54
|
+
- `source` only accepts `'user_query'` or `'session'`
|
|
52
55
|
- Capability IDs must be unique within a manifest
|
|
56
|
+
- `baseUrl` required when any capability uses `api` or `hybrid` resolver
|
|
57
|
+
- `CapmanConfig` refined — `baseUrl` must be present when any API/hybrid capability exists
|
|
53
58
|
|
|
54
59
|
---
|
|
55
60
|
|
|
@@ -59,13 +64,13 @@ Manifest lifecycle — create, read, write, validate.
|
|
|
59
64
|
Key exports:
|
|
60
65
|
- `generate(config)` → `Manifest` — converts config to manifest, deep-copies capabilities
|
|
61
66
|
- `loadConfig(path?)` → `CapmanConfig` — loads `capman.config.js` via `require()`
|
|
62
|
-
- `writeManifest(manifest, path?)` — writes `manifest.json
|
|
67
|
+
- `writeManifest(manifest, path?)` — writes `manifest.json`. Output path validated against working directory — throws on path traversal
|
|
63
68
|
- `readManifest(path?)` → `Manifest` — reads and Zod-validates `manifest.json`
|
|
64
69
|
- `validate(manifest)` → `ValidationResult`
|
|
65
70
|
- `generateStarterConfig()` → `string`
|
|
66
71
|
- `VERSION` — current version string, auto-generated by `scripts/version.js`
|
|
67
72
|
|
|
68
|
-
Note: `loadConfig()` uses `require()` internally — CJS config files only. ESM config files (`.mjs` or `"type": "module"`)
|
|
73
|
+
Note: `loadConfig()` uses `require()` internally — CJS config files only. ESM config files (`.mjs` or `"type": "module"`) produce a clear error with migration instructions. Full ESM config support is planned for v0.6.
|
|
69
74
|
|
|
70
75
|
---
|
|
71
76
|
|
|
@@ -73,28 +78,38 @@ Note: `loadConfig()` uses `require()` internally — CJS config files only. ESM
|
|
|
73
78
|
Intent matching — keyword scoring and LLM-based matching.
|
|
74
79
|
|
|
75
80
|
Key exports:
|
|
76
|
-
- `match(query, manifest)` → `MatchResult` — scores all capabilities, returns winner + all candidates
|
|
81
|
+
- `match(query, manifest, options?)` → `MatchResult` — scores all capabilities, returns winner + all candidates
|
|
82
|
+
- `options.fuzzyMatch` — enable Fuse.js fuzzy matching (default: false)
|
|
83
|
+
- `options.fuzzyThreshold` — Fuse.js threshold 0.0–1.0 (default: 0.4)
|
|
77
84
|
- `matchWithLLM(query, manifest, { llm })` → `MatchResult` — LLM-based matching
|
|
78
85
|
- Query passed as `JSON.stringify({ user_query })` with system instructions before user data
|
|
79
86
|
- `USER_QUERY_START/END` delimiters separate instructions from user-controlled content
|
|
80
|
-
-
|
|
87
|
+
- Capability `description` and `examples` sanitized via `sanitizeForPrompt()` before injection
|
|
88
|
+
- `manifest.app` sanitized — strips newlines/control chars, capped at 100 chars
|
|
89
|
+
- LLM `extracted_params` validated against declared capability params — nested objects → null, numbers/booleans → string, unknown keys dropped
|
|
90
|
+
- Parse failures throw `LLMParseError` — not counted toward circuit breaker
|
|
81
91
|
- Errors propagate to caller — no internal try/catch
|
|
82
92
|
- `extractParams(query, capability)` → `Record<string, string | null>` — direct param extraction
|
|
83
93
|
- `resolverToIntent(capability)` → intent string — converts resolver type to intent
|
|
84
94
|
- `STOPWORDS` — set of words filtered from scoring and learning index
|
|
95
|
+
- `LLMParseError` — typed error class for LLM parse failures
|
|
96
|
+
- `MatchOptions` — `{ fuzzyMatch?, fuzzyThreshold? }`
|
|
85
97
|
|
|
86
|
-
Scoring algorithm
|
|
87
|
-
- Examples: `Math.max` across all examples
|
|
88
|
-
- Description
|
|
89
|
-
- Name
|
|
90
|
-
-
|
|
98
|
+
Scoring algorithm:
|
|
99
|
+
- Examples: `Math.max` across all examples — best single match up to 60 points. Quality beats quantity — a capability with 10 weak examples no longer beats one with a single precise example
|
|
100
|
+
- Description: `Math.min(overlap / Math.min(descWords.length, 10), 1) * 30` — normalized against a cap of 10 words to avoid penalizing rich documentation
|
|
101
|
+
- Name: up to 10 points
|
|
102
|
+
- Fuzzy (optional): Fuse.js flat corpus — one entry per example/description/name, one index per `match()` call, results grouped by capability, best hit merged via `Math.max` with keyword score
|
|
103
|
+
|
|
104
|
+
Performance: `qWordSet` built once as `Set<string>` per `match()` call — O(1) `.has()` lookups replace O(n) `Array.includes` in all scoring loops.
|
|
91
105
|
|
|
92
106
|
Param extraction:
|
|
93
107
|
- `isIdParam` — single token (e.g. `order_id=1234`)
|
|
94
108
|
- `isNavParam` — single token after nav keywords (`to`, `open`, `show`)
|
|
95
109
|
- Multi-word — joined with `-` (e.g. `product=blue-jacket`)
|
|
96
|
-
- Required param fallback — only accepts identifier-shaped last word (rejects generic nouns)
|
|
110
|
+
- Required param fallback — only accepts identifier-shaped last word (rejects generic nouns and category words)
|
|
97
111
|
- Optional params stay `null` if no keyword match found
|
|
112
|
+
- Session params return `null` — injected by resolver from auth context, not extracted from query
|
|
98
113
|
|
|
99
114
|
---
|
|
100
115
|
|
|
@@ -104,16 +119,19 @@ Capability execution — API calls, navigation, hybrid.
|
|
|
104
119
|
Key exports:
|
|
105
120
|
- `resolve(matchResult, params, options)` → `ResolveResult`
|
|
106
121
|
- Enforces privacy before executing
|
|
107
|
-
- Injects `auth.userId` into session params
|
|
122
|
+
- Injects `auth.userId` into session params per-endpoint — only where `{param}` placeholder exists in that specific endpoint's path
|
|
108
123
|
- Supports `dryRun: true` — returns call plan without executing
|
|
109
|
-
- Retries with `AbortController` timeout on failure
|
|
124
|
+
- Retries with `AbortController` timeout on failure — safe methods (GET/HEAD/OPTIONS) only by default
|
|
125
|
+
- `retryAllMethods: true` opt-in for retrying write operations
|
|
110
126
|
- Returns `status` and parsed `data` from API response
|
|
111
127
|
- `null` and `undefined` params never written into URLs
|
|
112
|
-
-
|
|
113
|
-
-
|
|
128
|
+
- Both API and nav param values URL-encoded via `encodeURIComponent`
|
|
129
|
+
- API path params validated against `[a-zA-Z0-9_\-.:@]+` allowlist — prevents path traversal
|
|
130
|
+
- Nav params validated against same allowlist — rejects `/` and shell metacharacters
|
|
131
|
+
- `checkPrivacy(capability, auth)` → `string | null` — exported. Used by engine to populate privacy trace step accurately
|
|
114
132
|
|
|
115
133
|
`ResolveOptions`:
|
|
116
|
-
- `baseUrl`, `auth`, `dryRun`, `retries`, `timeoutMs`, `headers`, `fetch`
|
|
134
|
+
- `baseUrl`, `auth`, `dryRun`, `retries`, `timeoutMs`, `retryAllMethods`, `headers`, `fetch`
|
|
117
135
|
|
|
118
136
|
Privacy enforcement:
|
|
119
137
|
- `public` — always allowed
|
|
@@ -122,51 +140,61 @@ Privacy enforcement:
|
|
|
122
140
|
|
|
123
141
|
Debug logging: param values and `auth.userId` are redacted as `[REDACTED]` — never logged in plaintext.
|
|
124
142
|
|
|
143
|
+
⚠️ Parallel execution: multi-endpoint capabilities fire all endpoints simultaneously via `Promise.all()`. If one fails, side effects from successful endpoints cannot be rolled back. Use single-endpoint capabilities for operations requiring ordering or rollback.
|
|
144
|
+
|
|
125
145
|
---
|
|
126
146
|
|
|
127
147
|
### `src/cache.ts`
|
|
128
148
|
Pluggable cache backends.
|
|
129
149
|
|
|
130
150
|
Key exports:
|
|
131
|
-
- `CacheStore` interface — `get(key)`, `set(key, result)`, `clear()`, `size()`
|
|
132
|
-
- `MemoryCache` — in-memory Map, 512-entry cap
|
|
133
|
-
- `FileCache` — async `fs.promises` read/write, 2048-entry cap
|
|
151
|
+
- `CacheStore` interface — `get(key, ttlMs?)`, `set(key, result)`, `clear()`, `size()`
|
|
152
|
+
- `MemoryCache` — in-memory Map, 512-entry LRU cap
|
|
153
|
+
- `FileCache` — async `fs.promises` read/write, 2048-entry LRU cap. Atomic writes via `.tmp` + rename. Concurrent load serialized via `loadPromise`
|
|
134
154
|
- `ComboCache` — memory-first with file fallback
|
|
135
|
-
- `normalizeQuery(query)` — lowercase + trim + collapse whitespace → cache key
|
|
136
|
-
- `buildCacheKey(query, capabilityId, params)` —
|
|
155
|
+
- `normalizeQuery(query)` — lowercase + trim + strip punctuation + collapse whitespace → cache key. Punctuation stripped so `"show orders!"` and `"show orders"` share the same key
|
|
156
|
+
- `buildCacheKey(query, capabilityId, params)` — semantic key using capability + params. Engine writes under both `normalizeQuery` and `buildCacheKey` — differently-phrased queries resolving to the same capability share cache entries
|
|
137
157
|
|
|
138
|
-
Security: Only `public` capabilities are cached. Non-public (`user_owned`, `admin`) are never cached — prevents auth bypass where one user's cached match is served to another.
|
|
158
|
+
Security: Only `public` capabilities are cached. Non-public (`user_owned`, `admin`) are never cached — prevents auth bypass where one user's cached match is served to another. Cache written only after successful resolution — failed resolutions do not poison the cache.
|
|
139
159
|
|
|
140
160
|
Notes:
|
|
141
161
|
- `FileCache` and `ComboCache` are single-instance only — concurrent writers will corrupt
|
|
162
|
+
- `loadPromise` resets on rejection — failed loads allow retry rather than caching the rejection permanently
|
|
142
163
|
- For multi-instance deployments, use a Redis adapter (planned v0.6)
|
|
143
164
|
|
|
144
165
|
---
|
|
145
166
|
|
|
146
167
|
### `src/learning.ts`
|
|
147
|
-
Usage analytics and keyword index —
|
|
168
|
+
Usage analytics and keyword index — incremental, PII-safe.
|
|
148
169
|
|
|
149
170
|
Key exports:
|
|
150
|
-
- `LearningStore` interface — `record(entry)`, `getStats()`, `getTopCapabilities(limit)`, `getIndex()`
|
|
151
|
-
- `FileLearningStore` — persists to `.capman/learning.json`, caps at 10,000 entries. Saves
|
|
171
|
+
- `LearningStore` interface — `record(entry)`, `getStats()`, `getTopCapabilities(limit)`, `getIndex()`, `destroy()`
|
|
172
|
+
- `FileLearningStore` — persists to `.capman/learning.json`, caps at 10,000 entries. Saves debounced (5s) with synchronous flush on process exit via `flushSync()`
|
|
152
173
|
- `MemoryLearningStore` — in-memory only, used in tests
|
|
153
174
|
- `LearningIndex` — internal class shared by both stores. Maintains keyword index and stats counters incrementally. Eliminates ~80 lines of duplication
|
|
154
175
|
|
|
155
176
|
`LearningEntry`:
|
|
156
|
-
- `query
|
|
177
|
+
- `query` — stored as tokenized keywords only, never raw text. PII (emails, names, IDs) stripped before persistence
|
|
178
|
+
- `capabilityId`, `confidence`, `intent`, `extractedParams`
|
|
157
179
|
- `resolvedVia: 'keyword' | 'llm' | 'cache'`
|
|
158
180
|
- `timestamp`
|
|
159
181
|
|
|
160
182
|
`KeywordStats` (from `getStats()`):
|
|
161
|
-
- `index` — `{ word → { capabilityId → hitCount } }` — used by engine for learning boost
|
|
183
|
+
- `index` — `{ word → { capabilityId → hitCount } }` — used by engine for learning boost. Returns `structuredClone` — callers cannot corrupt internal state
|
|
162
184
|
- `totalQueries`, `llmQueries`, `cacheHits`, `outOfScope`
|
|
163
185
|
|
|
164
186
|
Performance:
|
|
165
|
-
- Index
|
|
187
|
+
- Index maintained incrementally in `record()` — O(w) per entry
|
|
166
188
|
- `getStats()` returns cached counters — O(1), no rebuild
|
|
167
|
-
- `
|
|
168
|
-
|
|
169
|
-
|
|
189
|
+
- Prune uses `subtractFromIndex()` — O(pruned × w), not full rebuild
|
|
190
|
+
|
|
191
|
+
Exit handling:
|
|
192
|
+
- Process exit handlers registered once via module-level `registerExitHandlers()`
|
|
193
|
+
- Handler references stored as `exitHandler`, `sigTermHandler`, `sigIntHandler`
|
|
194
|
+
- Removed via `unregisterExitHandlers()` when `activeStores` is empty after `destroy()`
|
|
195
|
+
- Does NOT call `process.exit()` — library must not hijack application shutdown
|
|
196
|
+
|
|
197
|
+
`destroy()` — async, awaits final flush before removing from registry. On the `LearningStore` interface — callable through the interface type.
|
|
170
198
|
|
|
171
199
|
---
|
|
172
200
|
|
|
@@ -176,36 +204,40 @@ The recommended API — orchestrates matching, caching, learning, and tracing.
|
|
|
176
204
|
Key exports:
|
|
177
205
|
- `CapmanEngine` class
|
|
178
206
|
- `EngineOptions` — all constructor options
|
|
179
|
-
- `fuzzyMatch` — enable Fuse.js fuzzy matching (default: false)
|
|
180
|
-
- `fuzzyThreshold` — Fuse.js threshold 0.0–1.0 (default: 0.4)
|
|
181
207
|
- `EngineResult` — `{ match, resolution, resolvedVia, durationMs, trace }`
|
|
182
208
|
|
|
183
209
|
⚠️ **Concurrency:** `CapmanEngine` is not safe for sharing across concurrent async request handlers. The LLM rate limiter, circuit breaker, and learning index cache are instance-level mutable state. Create one engine per request in server deployments, or use `cheap` mode for shared instances.
|
|
184
210
|
|
|
185
211
|
`CapmanEngine` methods:
|
|
186
212
|
- `ask(query, overrides?)` → `EngineResult` — full pipeline: cache → match → boost → resolve → learn
|
|
187
|
-
- `explain(query)` → `ExplainResult` — match + boost only, no execution, no cache/learning write
|
|
213
|
+
- `explain(query)` → `ExplainResult` — match + boost only, no execution, no cache/learning write. Shares LLM quota with `ask()` — explain() counts against rate limit
|
|
214
|
+
- `loadManifest(manifest)` — hot-reloads manifest without losing learning history, rate limiter state, or cache. Cache is cleared automatically. Rate limiter state intentionally preserved — LLM provider unchanged
|
|
188
215
|
- `getStats()` → `KeywordStats | null`
|
|
189
216
|
- `getTopCapabilities(limit?)` → `Array<{ id, hits }>`
|
|
190
217
|
- `clearCache()`
|
|
191
|
-
|
|
218
|
+
|
|
219
|
+
`EngineOptions` highlights:
|
|
220
|
+
- `fuzzyMatch` — enable Fuse.js fuzzy matching (default: false)
|
|
221
|
+
- `fuzzyThreshold` — Fuse.js threshold 0.0–1.0 (default: 0.4)
|
|
222
|
+
- `cacheTtlMs` — optional TTL for cache entries in ms (default: no expiry)
|
|
223
|
+
- `maxLLMCallsPerMinute` — rate limit (default: 60). Set to 0 to disable LLM entirely
|
|
224
|
+
- `llmCooldownMs`, `llmCircuitBreakerThreshold`, `llmCircuitBreakerResetMs`
|
|
192
225
|
|
|
193
226
|
Matching pipeline in `ask()`:
|
|
194
|
-
1. Cache check — return immediately on hit (public capabilities only)
|
|
195
|
-
2. Match — `cheap` / `balanced` / `accurate` mode
|
|
196
|
-
3. Privacy check —
|
|
197
|
-
4. Learning boost — up to +15 points
|
|
198
|
-
5. Cache set —
|
|
227
|
+
1. Cache check — return immediately on hit (public capabilities only). Re-extracts params fresh from current query
|
|
228
|
+
2. Match — `cheap` / `balanced` / `accurate` mode dispatch via `_runMatch()`
|
|
229
|
+
3. Privacy check — uses `checkPrivacy()` from `resolver.ts`. Correctly shows `'fail'` for blocked requests
|
|
230
|
+
4. Learning boost — up to +15 points. Skipped in `cheap` mode, skipped if all candidates score 0 (no keyword signal), skipped if `resolvedVia === 'llm'` and `confidence > 80`
|
|
231
|
+
5. Cache set — writes under both `normalizeQuery` and `buildCacheKey` (public only, after successful resolution only)
|
|
199
232
|
6. Resolve — actual API call or nav
|
|
200
233
|
7. Reasoning build — human-readable array
|
|
201
234
|
8. Learning record — pre-boost result recorded to prevent feedback loop
|
|
202
235
|
|
|
203
|
-
LLM rate limiting
|
|
204
|
-
- `maxLLMCallsPerMinute` —
|
|
205
|
-
- `
|
|
206
|
-
- `
|
|
207
|
-
-
|
|
208
|
-
- Parse failures (`LLM_PARSE_ERROR`) do NOT count toward circuit breaker — only network failures do
|
|
236
|
+
LLM rate limiting:
|
|
237
|
+
- `maxLLMCallsPerMinute` — slot reserved in `checkLLMAllowed()`, refunded in `recordLLMFailure()`
|
|
238
|
+
- `maxLLMCallsPerMinute: 0` returns `'LLM disabled'` message immediately
|
|
239
|
+
- Parse failures (`LLMParseError`) do NOT count toward circuit breaker — only network failures do
|
|
240
|
+
- Rate-limiter state shared between `ask()` and `explain()`
|
|
209
241
|
|
|
210
242
|
---
|
|
211
243
|
|
|
@@ -214,13 +246,18 @@ OpenAPI/Swagger → capman config converter.
|
|
|
214
246
|
|
|
215
247
|
Key exports:
|
|
216
248
|
- `parseOpenAPI(specPathOrUrl)` → `ParseResult`
|
|
217
|
-
- Accepts local file path or HTTP URL
|
|
218
|
-
- Parses JSON natively; YAML requires `js-yaml`
|
|
249
|
+
- Accepts local file path or HTTP URL (10s timeout via `AbortController`)
|
|
250
|
+
- Parses JSON natively; YAML requires `js-yaml` (distinguished via `err.code === 'MODULE_NOT_FOUND'`)
|
|
219
251
|
- Converts every path+method into a `Capability`
|
|
220
252
|
- Infers privacy from security schemes and tags
|
|
221
|
-
- Extracts path/query/body params
|
|
253
|
+
- Extracts path/query/body params — all mapped to `'user_query'` source
|
|
222
254
|
- Generates examples from operation summaries
|
|
223
255
|
|
|
256
|
+
Privacy inference:
|
|
257
|
+
- Admin classification uses word-boundary regex `\b(admin|administrator|backoffice|back-office|internal|superuser)\b` — `'manage'` alone no longer triggers admin. Operations like `manageWishlist`, `fileManager` correctly classified as non-admin
|
|
258
|
+
- Swagger 2.x `schemes` array respected — prefers `https` over `http` when both declared
|
|
259
|
+
- Missing server URL emits `logger.warn` instead of silently using placeholder
|
|
260
|
+
|
|
224
261
|
`ParseResult`: `{ config, stats: { total, skipped, warnings } }`
|
|
225
262
|
|
|
226
263
|
Supported: OpenAPI 3.x, Swagger 2.x (JSON or YAML)
|
|
@@ -234,7 +271,7 @@ Key exports:
|
|
|
234
271
|
- `logger` — singleton with `debug()`, `info()`, `warn()`, `error()`
|
|
235
272
|
- `setLogLevel(level)` — `'silent' | 'error' | 'warn' | 'info' | 'debug'`
|
|
236
273
|
|
|
237
|
-
Note: The manifest version compatibility warning uses `console.warn` directly (not `logger.warn`) so it is always visible regardless of log level.
|
|
274
|
+
Note: The manifest version compatibility warning uses `console.warn` directly (not `logger.warn`) so it is always visible regardless of log level. Raw query text never appears at info level — only query length logged at info, full query at debug.
|
|
238
275
|
|
|
239
276
|
---
|
|
240
277
|
|
|
@@ -245,7 +282,7 @@ Notable:
|
|
|
245
282
|
- `ask(query, manifest, options?)` — convenience function, delegates to `CapmanEngine`
|
|
246
283
|
- Marked `@deprecated` — use `CapmanEngine` directly for full features
|
|
247
284
|
- `MatchMode` — `'cheap' | 'balanced' | 'accurate'`
|
|
248
|
-
- `extractParams`, `resolverToIntent`, `STOPWORDS` — exported for advanced use cases
|
|
285
|
+
- `extractParams`, `resolverToIntent`, `STOPWORDS`, `LLMParseError` — exported for advanced use cases
|
|
249
286
|
|
|
250
287
|
---
|
|
251
288
|
|
|
@@ -255,22 +292,24 @@ Notable:
|
|
|
255
292
|
Entry point only (~20 lines). Routes `command` to the correct module.
|
|
256
293
|
|
|
257
294
|
### `bin/lib/shared.js`
|
|
258
|
-
Exports: `args`, `command`, `flags`, `getFlag`, `c`, `log`, `header`, `
|
|
295
|
+
Exports: `args`, `command`, `flags`, `posArgs`, `getFlag`, `c`, `log`, `header`, `requireSrc`
|
|
259
296
|
|
|
260
|
-
`getFlag(name)` — exits with error if flag is present but has no value
|
|
261
|
-
`posArgs` — positional arguments after POSIX `--` sentinel. Allows queries starting with `--` to be passed without flag interpretation
|
|
297
|
+
- `getFlag(name)` — exits with error if flag is present but has no value
|
|
298
|
+
- `posArgs` — positional arguments after POSIX `--` sentinel. Allows queries starting with `--` to be passed without flag interpretation (e.g. `capman run -- "--help me find orders"`)
|
|
262
299
|
|
|
263
300
|
### `bin/lib/cmd-generate.js`
|
|
264
301
|
Three generation paths: `--from` (OpenAPI), `--ai` (LLM-assisted), manual.
|
|
265
302
|
Output paths validated via `safeOutputPath()` — rejects traversal outside working directory.
|
|
266
303
|
Contains `buildAIPrompt()` and `callLLM()`.
|
|
304
|
+
- All three providers (Anthropic, OpenAI, OpenRouter) read response via `res.text()` before `res.ok` check — prevents JSON parse errors masking real API errors
|
|
305
|
+
- Empty string API keys rejected — `.trim() ||` used instead of `??`
|
|
267
306
|
|
|
268
307
|
### `bin/lib/cmd-init.js` — creates `capman.config.js`
|
|
269
308
|
### `bin/lib/cmd-validate.js` — validates `manifest.json`
|
|
270
309
|
### `bin/lib/cmd-inspect.js` — prints all capabilities
|
|
271
310
|
### `bin/lib/cmd-demo.js` — live demo with hardcoded e-commerce manifest
|
|
272
|
-
### `bin/lib/cmd-run.js` — runs a query, `--debug` shows all candidates
|
|
273
|
-
### `bin/lib/cmd-explain.js` — runs `engine.explain()` and prints full breakdown
|
|
311
|
+
### `bin/lib/cmd-run.js` — runs a query, `--debug` shows all candidates. Supports `--` sentinel via `posArgs`
|
|
312
|
+
### `bin/lib/cmd-explain.js` — runs `engine.explain()` and prints full breakdown. Supports `--` sentinel via `posArgs`
|
|
274
313
|
### `bin/lib/cmd-help.js` — usage and command list
|
|
275
314
|
|
|
276
315
|
---
|
|
@@ -278,16 +317,16 @@ Contains `buildAIPrompt()` and `callLLM()`.
|
|
|
278
317
|
## tests/
|
|
279
318
|
|
|
280
319
|
### `tests/matcher.test.ts` — 17 tests
|
|
281
|
-
Keyword scoring, OOS detection, param extraction, LLM edge cases (hallucinated ID, undefined reasoning), example
|
|
320
|
+
Keyword scoring, OOS detection, param extraction, LLM edge cases (hallucinated ID, undefined reasoning), example quality-over-quantity (Math.max)
|
|
282
321
|
|
|
283
322
|
### `tests/resolver.test.ts` — 27 tests
|
|
284
|
-
API/nav/hybrid resolvers, privacy enforcement, session injection, null params, nav open redirect, API path
|
|
323
|
+
API/nav/hybrid resolvers, privacy enforcement, session injection, null params, nav open redirect, API path traversal rejection, multi-endpoint session param isolation, LRU eviction, nav params with dots/colons allowed
|
|
285
324
|
|
|
286
325
|
### `tests/engine.test.ts` — 44 tests
|
|
287
326
|
`ask()`, `explain()`, caching, learning, matching modes, trace, rate limiting, manifest version check, learning boost, query validation (TypeError/RangeError guards), LRU eviction, fuzzy matching (typos, cheap mode bypass, default disabled, strict threshold), `loadManifest()` hot-reload (cache cleared, learning preserved)
|
|
288
327
|
|
|
289
|
-
### `tests/parser.test.ts` —
|
|
290
|
-
OpenAPI capability extraction, privacy inference, param extraction, base URL, error handling
|
|
328
|
+
### `tests/parser.test.ts` — 11 tests
|
|
329
|
+
OpenAPI capability extraction, privacy inference, param extraction, base URL, error handling, manage/admin false-positive regression
|
|
291
330
|
|
|
292
331
|
---
|
|
293
332
|
|
|
@@ -306,9 +345,9 @@ Prebuild script. Reads `version` from `package.json` and writes `src/version.ts`
|
|
|
306
345
|
| `tsconfig.esm.json` | ESM build → `dist/esm/` with `.d.ts` |
|
|
307
346
|
| `package.json` | Version, exports map, scripts, dependencies |
|
|
308
347
|
| `.github/workflows/ci.yml` | Build + test + verify both dist outputs on every push |
|
|
348
|
+
| `.gitignore` | Includes `.capman/` — cache and learning files must never be committed |
|
|
309
349
|
| `CHANGELOG.md` | All notable changes per version |
|
|
310
350
|
| `CODEBASE.md` | This file |
|
|
311
|
-
| `ROADMAP_v0.5.0.md` | Prioritized fix and feature roadmap |
|
|
312
351
|
|
|
313
352
|
---
|
|
314
353
|
|
|
@@ -319,19 +358,25 @@ Developer writes capman.config.js
|
|
|
319
358
|
↓
|
|
320
359
|
capman generate
|
|
321
360
|
↓
|
|
322
|
-
generator.ts → manifest.json
|
|
361
|
+
generator.ts → manifest.json (path-guarded write)
|
|
323
362
|
↓
|
|
324
363
|
CapmanEngine.ask("user query")
|
|
325
364
|
↓
|
|
326
|
-
cache.ts → cache hit?
|
|
365
|
+
cache.ts → cache hit? re-extract params fresh → return
|
|
366
|
+
↓
|
|
367
|
+
matcher.ts → score all capabilities (Set-based O(n))
|
|
368
|
+
→ optional Fuse.js fuzzy pass (single index)
|
|
369
|
+
→ pick winner
|
|
370
|
+
↓
|
|
371
|
+
checkPrivacy() → privacy trace step (pass or fail)
|
|
327
372
|
↓
|
|
328
|
-
|
|
373
|
+
learning.ts → apply boost (+0 to +15) — skipped for high-confidence LLM
|
|
329
374
|
↓
|
|
330
|
-
|
|
375
|
+
cache.ts → write under normalizeQuery + buildCacheKey (public, on success only)
|
|
331
376
|
↓
|
|
332
377
|
resolver.ts → enforce privacy → call API or navigate
|
|
333
378
|
↓
|
|
334
|
-
learning.ts → record pre-boost match result
|
|
379
|
+
learning.ts → record pre-boost match result (tokenized, PII-stripped)
|
|
335
380
|
↓
|
|
336
381
|
EngineResult → { match, resolution, trace, resolvedVia }
|
|
337
382
|
```
|
package/README.md
CHANGED
|
@@ -227,6 +227,25 @@ const engine = new CapmanEngine({
|
|
|
227
227
|
})
|
|
228
228
|
```
|
|
229
229
|
|
|
230
|
+
### Fuzzy Matching
|
|
231
|
+
|
|
232
|
+
Enable opt-in fuzzy matching to catch typos and slight paraphrases:
|
|
233
|
+
|
|
234
|
+
```typescript
|
|
235
|
+
const engine = new CapmanEngine({
|
|
236
|
+
manifest,
|
|
237
|
+
mode: 'balanced',
|
|
238
|
+
fuzzyMatch: true, // enable Fuse.js fuzzy matching
|
|
239
|
+
fuzzyThreshold: 0.4, // 0.0 = exact only, 1.0 = match anything (default: 0.4)
|
|
240
|
+
})
|
|
241
|
+
|
|
242
|
+
// Now catches typos: "Shwo me artciles" → matches "Show me articles"
|
|
243
|
+
// Also catches near-matches Fuse.js considers similar
|
|
244
|
+
```
|
|
245
|
+
|
|
246
|
+
Fuzzy matching never runs in `cheap` mode. It is additive — fuzzy can only
|
|
247
|
+
help a capability reach the confidence threshold, never hurt it.
|
|
248
|
+
|
|
230
249
|
---
|
|
231
250
|
|
|
232
251
|
## Caching + Learning
|
|
@@ -249,6 +268,28 @@ const top = await engine.getTopCapabilities(3)
|
|
|
249
268
|
|
|
250
269
|
---
|
|
251
270
|
|
|
271
|
+
### Hot-reloading manifests
|
|
272
|
+
|
|
273
|
+
Swap the manifest without creating a new engine instance — preserves cache,
|
|
274
|
+
learning history, and rate limiter state:
|
|
275
|
+
|
|
276
|
+
```typescript
|
|
277
|
+
const newManifest = generate(updatedConfig)
|
|
278
|
+
await engine.loadManifest(newManifest)
|
|
279
|
+
// Cache is cleared automatically — stale entries from old manifest are gone
|
|
280
|
+
```
|
|
281
|
+
|
|
282
|
+
### Cleaning up
|
|
283
|
+
|
|
284
|
+
Call `destroy()` on file-backed stores when done to flush pending data and
|
|
285
|
+
deregister process exit handlers:
|
|
286
|
+
|
|
287
|
+
```typescript
|
|
288
|
+
await engine.learning?.destroy()
|
|
289
|
+
```
|
|
290
|
+
|
|
291
|
+
---
|
|
292
|
+
|
|
252
293
|
## Privacy + Auth
|
|
253
294
|
|
|
254
295
|
Privacy scope is enforced **per capability**, before resolution happens:
|
|
@@ -320,8 +361,6 @@ const result = await engine.ask('show my orders', {
|
|
|
320
361
|
|---|---|
|
|
321
362
|
| `user_query` | Extracted from the user's query |
|
|
322
363
|
| `session` | Injected from `auth.userId` automatically |
|
|
323
|
-
| `context` | Provided by the caller |
|
|
324
|
-
| `static` | Fixed value, never changes |
|
|
325
364
|
|
|
326
365
|
---
|
|
327
366
|
|
|
@@ -337,8 +376,10 @@ const result = await engine.ask('show my orders', {
|
|
|
337
376
|
**Current limits:**
|
|
338
377
|
- Real-time infra status (is the server down?)
|
|
339
378
|
- UI-only state with no API backing
|
|
340
|
-
- Very ambiguous queries — use `mode: 'accurate'` with an LLM
|
|
341
|
-
- Multi-instance deployments
|
|
379
|
+
- Very ambiguous queries with no keyword signal — use `mode: 'accurate'` with an LLM, or enable `fuzzyMatch: true`
|
|
380
|
+
- Multi-instance deployments: `FileCache` and `FileLearningStore` are single-instance only — concurrent writers will corrupt the file. Use separate instances per process or a shared Redis adapter
|
|
381
|
+
- `FileLearningStore` saves are debounced — up to 5s of learning data could be lost on `SIGKILL` (not SIGTERM/SIGINT which are handled)
|
|
382
|
+
- Parallel multi-endpoint capabilities: if one endpoint fails, side effects from successful endpoints cannot be rolled back. Use single-endpoint capabilities for operations requiring ordering or rollback
|
|
342
383
|
|
|
343
384
|
---
|
|
344
385
|
|