@devilsdev/rag-pipeline-utils 2.4.5 → 2.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (258) hide show
  1. package/.ragrc.schema.json +83 -192
  2. package/CHANGELOG.md +431 -0
  3. package/README.md +418 -117
  4. package/SECURITY.md +5 -5
  5. package/dist/experimental.cjs +2 -0
  6. package/dist/experimental.d.ts +1 -0
  7. package/dist/experimental.mjs +5 -0
  8. package/dist/index.cjs +6442 -1355
  9. package/dist/index.d.ts +129 -2
  10. package/dist/index.mjs +1 -1
  11. package/dist/testing.cjs +2 -0
  12. package/dist/testing.d.ts +1 -0
  13. package/dist/testing.mjs +5 -0
  14. package/dist/types/adapters/ai-sdk.d.ts +53 -0
  15. package/dist/types/agentic/agentic-pipeline.d.ts +103 -0
  16. package/dist/types/agentic/iterative-retriever.d.ts +85 -0
  17. package/dist/types/agentic/query-planner.d.ts +145 -0
  18. package/dist/types/agentic/self-critique.d.ts +45 -0
  19. package/dist/types/ai/adaptive-retrieval.d.ts +44 -0
  20. package/dist/types/ai/federation/federated-learning-coordinator.d.ts +62 -0
  21. package/dist/types/ai/multimodal-processing.d.ts +32 -0
  22. package/dist/types/ai/multimodal.d.ts +39 -0
  23. package/dist/types/ai/retrieval-engine.d.ts +37 -0
  24. package/dist/types/ai/training/model-training-orchestrator.d.ts +31 -0
  25. package/dist/types/cache/component-keys.d.ts +7 -0
  26. package/dist/types/cache/memory-cache.d.ts +54 -0
  27. package/dist/types/cache/pipeline-cache.d.ts +66 -0
  28. package/dist/types/cache/redis-cache.d.ts +46 -0
  29. package/dist/types/chunking/chunking-engine.d.ts +70 -0
  30. package/dist/types/chunking/strategies/fixed-size.d.ts +17 -0
  31. package/dist/types/chunking/strategies/recursive.d.ts +20 -0
  32. package/dist/types/chunking/strategies/semantic.d.ts +19 -0
  33. package/dist/types/chunking/strategies/sentence.d.ts +13 -0
  34. package/dist/types/chunking/strategies/structure-aware.d.ts +19 -0
  35. package/dist/types/citation/citation-tracker.d.ts +64 -0
  36. package/dist/types/citation/hallucination-detector.d.ts +40 -0
  37. package/dist/types/citation/source-mapper.d.ts +37 -0
  38. package/dist/types/connectors/anthropic-connector.d.ts +97 -0
  39. package/dist/types/connectors/base-connector.d.ts +87 -0
  40. package/dist/types/connectors/cohere-connector.d.ts +94 -0
  41. package/dist/types/connectors/connector-utils.d.ts +110 -0
  42. package/dist/types/connectors/local-embedder.d.ts +95 -0
  43. package/dist/types/connectors/memory-retriever.d.ts +99 -0
  44. package/dist/types/connectors/ollama-connector.d.ts +115 -0
  45. package/dist/types/connectors/openai-connector.d.ts +129 -0
  46. package/dist/types/connectors/rag-prompt.d.ts +40 -0
  47. package/dist/types/core/create-pipeline.d.ts +8 -0
  48. package/dist/types/core/plugin-registry.d.ts +136 -0
  49. package/dist/types/core/stage-contracts.d.ts +36 -0
  50. package/dist/types/core/validate-plugin.d.ts +45 -0
  51. package/dist/types/cost/cost-calculator.d.ts +152 -0
  52. package/dist/types/cost/cost-tracker.d.ts +126 -0
  53. package/dist/types/cost/token-budget.d.ts +110 -0
  54. package/dist/types/dag/core/error-context.d.ts +239 -0
  55. package/dist/types/dag/core/execution-scheduler.d.ts +191 -0
  56. package/dist/types/dag/core/topology-validation.d.ts +226 -0
  57. package/dist/types/dag/dag-engine.d.ts +156 -0
  58. package/dist/types/dag/dag-node.d.ts +37 -0
  59. package/dist/types/dashboard/dashboard-generator.d.ts +127 -0
  60. package/dist/types/dashboard/metrics-aggregator.d.ts +218 -0
  61. package/dist/types/debug/execution-tracer.d.ts +57 -0
  62. package/dist/types/debug/trace-inspector.d.ts +31 -0
  63. package/dist/types/ecosystem/plugin-certification.d.ts +169 -0
  64. package/dist/types/enterprise/data-governance.d.ts +203 -0
  65. package/dist/types/enterprise/multi-tenancy.d.ts +137 -0
  66. package/dist/types/enterprise/sso-integration.d.ts +420 -0
  67. package/dist/types/evaluate/context-metrics.d.ts +55 -0
  68. package/dist/types/evaluate/faithfulness.d.ts +41 -0
  69. package/dist/types/evaluate/groundedness.d.ts +60 -0
  70. package/dist/types/evaluate/pipeline-evaluator.d.ts +88 -0
  71. package/dist/types/evaluate/relevance.d.ts +21 -0
  72. package/dist/types/evaluate/scoring.d.ts +123 -0
  73. package/dist/types/evaluate/semantic-similarity.d.ts +14 -0
  74. package/dist/types/experimental.d.ts +15 -0
  75. package/dist/types/graph/entity-extractor.d.ts +84 -0
  76. package/dist/types/graph/graph-index.d.ts +49 -0
  77. package/dist/types/graph/graph-retriever.d.ts +81 -0
  78. package/dist/types/graph/knowledge-graph.d.ts +129 -0
  79. package/dist/types/grounding/check-answer.d.ts +101 -0
  80. package/dist/types/guardrails/guardrails-pipeline.d.ts +63 -0
  81. package/dist/types/guardrails/injection-detector.d.ts +47 -0
  82. package/dist/types/guardrails/post-generation-guard.d.ts +61 -0
  83. package/dist/types/guardrails/pre-retrieval-guard.d.ts +64 -0
  84. package/dist/types/guardrails/retrieval-guard.d.ts +57 -0
  85. package/dist/types/mcp/mcp-server.d.ts +85 -0
  86. package/dist/types/mcp/mcp-tool-builder.d.ts +55 -0
  87. package/dist/types/reranker/cascade-reranker.d.ts +40 -0
  88. package/dist/types/reranker/embedding-reranker.d.ts +77 -0
  89. package/dist/types/reranker/scoring-reranker.d.ts +41 -0
  90. package/dist/types/retrieval/bm25-search.d.ts +43 -0
  91. package/dist/types/retrieval/file-vector-store.d.ts +99 -0
  92. package/dist/types/retrieval/hybrid-retriever.d.ts +35 -0
  93. package/dist/types/retrieval/rank-fusion.d.ts +10 -0
  94. package/dist/types/security/jwt-config.d.ts +80 -0
  95. package/dist/types/security/jwt-validator.d.ts +195 -0
  96. package/dist/types/security/plugin-sandbox.d.ts +158 -0
  97. package/dist/types/security/plugin-signature-verifier.d.ts +139 -0
  98. package/dist/types/security/validator.d.ts +5 -0
  99. package/dist/types/streaming/embedding-stream.d.ts +65 -0
  100. package/dist/types/streaming/sse-adapter.d.ts +59 -0
  101. package/dist/types/streaming/stream-router.d.ts +88 -0
  102. package/dist/types/streaming/streaming-embedder.d.ts +99 -0
  103. package/dist/types/streaming/websocket-adapter.d.ts +71 -0
  104. package/dist/types/templates/scaffold.d.ts +55 -0
  105. package/dist/types/templates/template-registry.d.ts +63 -0
  106. package/dist/types/testing.d.ts +168 -0
  107. package/dist/types/utils/embed-query.d.ts +20 -0
  108. package/dist/types/utils/logger.d.ts +50 -0
  109. package/dist/types/utils/rate-limiter.d.ts +66 -0
  110. package/dist/types/utils/retry.d.ts +27 -0
  111. package/dist/types/utils/run-abort.d.ts +34 -0
  112. package/dist/types/utils/secure-logging.d.ts +51 -0
  113. package/dist/types/utils/structured-logger.d.ts +126 -0
  114. package/package.json +72 -33
  115. package/src/adapters/ai-sdk.js +215 -0
  116. package/src/agentic/agentic-pipeline.js +75 -7
  117. package/src/agentic/iterative-retriever.js +47 -10
  118. package/src/agentic/query-planner.js +329 -78
  119. package/src/cache/component-keys.js +38 -0
  120. package/src/cache/memory-cache.js +87 -0
  121. package/src/cache/pipeline-cache.js +260 -0
  122. package/src/cache/redis-cache.js +83 -0
  123. package/src/citation/citation-tracker.js +6 -4
  124. package/src/citation/hallucination-detector.js +2 -2
  125. package/src/citation/source-mapper.js +15 -28
  126. package/src/cli/commands/ai.js +106 -0
  127. package/src/cli/commands/plugin-hub/event-handlers.js +6 -0
  128. package/src/cli/commands/plugin-hub/hub-commands.js +0 -74
  129. package/src/cli/commands/plugin-hub/local-commands.js +143 -0
  130. package/src/cli/commands/plugin-hub.js +114 -49
  131. package/src/cli/diagnostics/config-checker.js +76 -63
  132. package/src/cli/diagnostics/environment-checker.js +62 -31
  133. package/src/cli/diagnostics/plugin-checker.js +36 -89
  134. package/src/cli/doctor-command.js +39 -5
  135. package/src/cli/enhanced-cli-commands.js +104 -233
  136. package/src/cli/handlers/ai-handler.js +364 -0
  137. package/src/cli/handlers/diagnostics-handler.js +78 -31
  138. package/src/cli/handlers/ingest-handler.js +28 -22
  139. package/src/cli/handlers/init-handler.js +148 -25
  140. package/src/cli/handlers/query-handler.js +203 -25
  141. package/src/cli/handlers/run-handler.js +16 -167
  142. package/src/cli/handlers/serve-handler.js +60 -0
  143. package/src/cli/services/bench-stats.js +54 -0
  144. package/src/cli/services/benchmark.js +161 -0
  145. package/src/cli/services/feedback.js +155 -0
  146. package/src/cli/services/instrument.js +83 -0
  147. package/src/cli/services/rag-service.js +361 -0
  148. package/src/cli/shared/cli-utils.js +24 -7
  149. package/src/cli/shared/config-context.js +21 -0
  150. package/src/cli/wizard/config-builder.js +81 -0
  151. package/src/cli/wizard/init-wizard.js +602 -0
  152. package/src/config/component-factory.js +125 -0
  153. package/src/config/enhanced-ragrc-schema.js +0 -2
  154. package/src/config/load-config.js +29 -10
  155. package/src/config/load-plugin-config.js +46 -2
  156. package/src/config/validate-schema.js +4 -1
  157. package/src/connectors/anthropic-connector.js +115 -57
  158. package/src/connectors/base-connector.js +74 -19
  159. package/src/connectors/cohere-connector.js +108 -34
  160. package/src/connectors/connector-utils.js +302 -0
  161. package/src/connectors/local-embedder.js +63 -5
  162. package/src/connectors/memory-retriever.js +125 -14
  163. package/src/connectors/ollama-connector.js +112 -51
  164. package/src/connectors/openai-connector.js +153 -44
  165. package/src/connectors/rag-prompt.js +88 -0
  166. package/src/core/create-pipeline.js +307 -61
  167. package/src/core/performance/parallel-processor.js +62 -1
  168. package/src/core/plugin-registry.js +125 -26
  169. package/src/core/stage-contracts.js +36 -0
  170. package/src/core/validate-plugin.js +85 -0
  171. package/src/cost/cost-calculator.js +205 -32
  172. package/src/debug/execution-tracer.js +14 -4
  173. package/src/dev/dev-server.js +173 -11
  174. package/src/dev/hot-reload.js +44 -4
  175. package/src/ecosystem/plugin-checks.js +181 -0
  176. package/src/ecosystem/plugin-hub.js +50 -125
  177. package/src/ecosystem/plugin-installer.js +298 -0
  178. package/src/ecosystem/plugin-probe.js +127 -0
  179. package/src/ecosystem/plugin-scanner.js +185 -0
  180. package/src/ecosystem/tar-extract.js +114 -0
  181. package/src/enterprise/audit-logging.js +158 -29
  182. package/src/enterprise/sso-integration.js +716 -228
  183. package/src/evaluate/context-metrics.js +28 -16
  184. package/src/evaluate/evaluator.js +3 -3
  185. package/src/evaluate/faithfulness.js +18 -14
  186. package/src/evaluate/groundedness.js +43 -12
  187. package/src/evaluate/pipeline-evaluator.js +35 -8
  188. package/src/evaluate/scoring.js +251 -3
  189. package/src/evaluate/semantic-similarity.js +79 -0
  190. package/src/experimental.js +71 -0
  191. package/src/grounding/check-answer.js +173 -0
  192. package/src/guardrails/guardrails-pipeline.js +69 -7
  193. package/src/guardrails/injection-detector.js +283 -0
  194. package/src/guardrails/post-generation-guard.js +19 -0
  195. package/src/guardrails/pre-retrieval-guard.js +36 -36
  196. package/src/guardrails/retrieval-guard.js +58 -8
  197. package/src/index.js +58 -1
  198. package/src/loader/csv-loader.js +3 -2
  199. package/src/loader/document-reader.js +91 -0
  200. package/src/loader/html-loader.js +3 -1
  201. package/src/loader/markdown-loader.js +3 -2
  202. package/src/reranker/cascade-reranker.js +3 -0
  203. package/src/reranker/embedding-reranker.js +136 -21
  204. package/src/reranker/llm-reranker.js +79 -12
  205. package/src/reranker/scoring-reranker.js +3 -0
  206. package/src/retrieval/bm25-search.js +12 -6
  207. package/src/retrieval/file-vector-store.js +241 -0
  208. package/src/retrieval/hybrid-retriever.js +22 -19
  209. package/src/security/jwt-config.js +13 -2
  210. package/src/security/jwt-validator.js +139 -34
  211. package/src/security/plugin-sandbox.js +18 -1
  212. package/src/security/plugin-signature-verifier.js +245 -70
  213. package/src/testing.js +254 -0
  214. package/src/utils/deprecate.js +69 -0
  215. package/src/utils/embed-query.js +33 -0
  216. package/src/utils/input-sanitizer.js +19 -4
  217. package/src/utils/logger.js +34 -4
  218. package/src/utils/run-abort.js +102 -0
  219. package/src/ai/federated-learning.js +0 -211
  220. package/src/ai/federation.js +0 -71
  221. package/src/ai/index.js +0 -22
  222. package/src/ai/model-trainer.js +0 -217
  223. package/src/ai/model-training.js +0 -202
  224. package/src/ai/multimodal/multi-modal-processor.js +0 -345
  225. package/src/ai/orchestrator.js +0 -65
  226. package/src/ai/retrieval/adaptive-retrieval-engine.js +0 -398
  227. package/src/cli/commands/ai-ml/adaptive-commands.js +0 -178
  228. package/src/cli/commands/ai-ml/federated-commands.js +0 -207
  229. package/src/cli/commands/ai-ml/index.js +0 -17
  230. package/src/cli/commands/ai-ml/multimodal-commands.js +0 -187
  231. package/src/cli/commands/ai-ml/shared.js +0 -51
  232. package/src/cli/commands/ai-ml/training-commands.js +0 -228
  233. package/src/cli/commands/ai-ml.js +0 -386
  234. package/src/cli/commands/docs.js +0 -228
  235. package/src/cli/commands/dx.js +0 -538
  236. package/src/cli/commands/wizard.js +0 -627
  237. package/src/cli/input-sanitization-middleware.js +0 -356
  238. package/src/cli/interactive-wizard.js +0 -373
  239. package/src/cli/templates/development.template.json +0 -64
  240. package/src/cli/templates/minimal.template.json +0 -33
  241. package/src/cli/templates/production.template.json +0 -104
  242. package/src/cli/templates/testing.template.json +0 -45
  243. package/src/cli/wizard/configuration.js +0 -148
  244. package/src/cli/wizard/observability.js +0 -79
  245. package/src/cli/wizard/performance.js +0 -76
  246. package/src/cli/wizard/plugin-selection.js +0 -297
  247. package/src/cli/wizard/preview-save.js +0 -172
  248. package/src/cli/wizard/project-setup.js +0 -66
  249. package/src/core/pipeline-factory.js +0 -59
  250. package/src/dag/enhanced-dag-engine.js +0 -894
  251. package/src/dx/index.js +0 -40
  252. package/src/dx/integration-templates.js +0 -662
  253. package/src/dx/performance-profiler.js +0 -968
  254. package/src/dx/realtime-debugger.js +0 -755
  255. package/src/dx/visual-pipeline-builder.js +0 -578
  256. package/src/ingest.js +0 -59
  257. package/src/pipeline/pipeline.js +0 -18
  258. package/src/query.js +0 -117
package/CHANGELOG.md CHANGED
@@ -9,6 +9,434 @@ Older history (pre-2.4.0) is preserved in the
9
9
  [GitHub releases](https://github.com/DevilsDev/rag-pipeline-utils/releases)
10
10
  page.
11
11
 
12
+ ## [Unreleased]
13
+
14
+ ## [2.6.0] - 2026-10-29
15
+
16
+ ### Added
17
+
18
+ - **`OpenAIConnector` `baseURL`** for OpenAI-compatible APIs (Groq,
19
+ Together, the Hugging Face router, vLLM, LM Studio, Ollama's `/v1`
20
+ endpoint). With a custom `baseURL` the key is never read from
21
+ `OPENAI_API_KEY`, so an OpenAI key is not sent to another provider; pass
22
+ `apiKey`, or omit it for local servers that need none.
23
+ - **`validatePlugin(stage, plugin, { registry })`** returns
24
+ `{ valid, errors, warnings }` for a plugin without throwing, for use in
25
+ plugin tests. `registry: true` also applies the rules
26
+ `pluginRegistry.register()` enforces.
27
+ - **`ParallelEmbedder` `mode`** (`'auto'`, `'batch'`, `'single'`).
28
+ - **`checkAnswer({ question, answer, sources })`** checks an answer from any
29
+ framework (LangChain, LlamaIndex, Mastra, the AI SDK, your own code)
30
+ against its sources: `grounded`, `groundedness`, the `unsupported`
31
+ sentences, per-sentence `citations` and `evaluation` scores. Sources may be
32
+ strings, `{ id, content }`, LangChain Documents or LlamaIndex nodes;
33
+ `normalizeSources` is exported for the conversion alone.
34
+ - **Caching**: `createRagPipeline({ cache })` caches query embeddings,
35
+ retrieval results and answers (`true`, a store, or per layer with time to
36
+ live). `MemoryCache` (LRU and TTL) and `RedisCache` (node-redis or
37
+ ioredis) are included; any `get`/`set` store works. A layer caches only
38
+ when its component identifies itself (`getCacheKey(role)` or `cacheKey`;
39
+ all built-ins do), retrieval is keyed by the store's state and the query
40
+ vector, answers by the exact documents and generation options, and
41
+ answers are not cached when the temperature is above 0. Retrieval is
42
+ cached before `filterResults` and the reranker, so guardrail ACLs run on
43
+ every request. Nothing from a failed, aborted or timed-out run, or from an
44
+ unfinished stream, is stored. Results carry `cache` per layer and the
45
+ pipeline has `cacheStats()`.
46
+ - **`@devilsdev/rag-pipeline-utils/testing`**: deterministic, offline test
47
+ helpers. `HashEmbedder`, `EchoLLM` (answers with the top document or a set
48
+ answer, records every call, streams), `fixtures.documents` and
49
+ `createTestPipeline()`. Shares classes with the main entry and is not
50
+ loaded by it.
51
+ - **`computeSemanticSimilarity(answer, reference, { embedder })`**: cosine
52
+ similarity of the two embeddings, 0 to 1. `checkAnswer` adds it as
53
+ `evaluation.scores.semanticSimilarity` when given a `referenceAnswer` and
54
+ an `embedder`.
55
+ - **`rag-pipeline query`** options: `--trace` (time per stage), `--explain`
56
+ (passage scores, and the passages supporting each answer sentence, with
57
+ unsupported sentences flagged), `--stats` (latency, estimated tokens and
58
+ cost), `--streaming` and `--timeout <ms>`; global `--quiet`. `--json`
59
+ includes trace, stats and explanation when asked.
60
+ - **`createAiSdkAdapter({ ai, embeddingModel, languageModel })`** wraps
61
+ Vercel AI SDK models as an `embedder` and an `llm` for
62
+ `createRagPipeline()`, with streaming, `embedMany` batching and the run's
63
+ `AbortSignal`. You pass the AI SDK functions, so it is not a dependency and
64
+ works with AI SDK 5, 6 and 7 from ESM or CommonJS. The RAG system prompt
65
+ is sent as both `instructions` (AI SDK 7) and `system` (5 and 6). Tested
66
+ against AI SDK 6 and 7 with their mock models.
67
+
68
+ ### Changed
69
+
70
+ - **Faster CLI.** Commands load only their own dependencies: `--help`
71
+ takes about 0.5 s and `query` about 0.5 s instead of about 3.6 s. Loading
72
+ the CLI drops from 1281 modules and 93 packages to 55 and 13. The HTML,
73
+ Markdown and CSV loaders load `jsdom`, `marked` and `csv-parse` when they
74
+ read a file.
75
+ - **Faster import.** `require('@devilsdev/rag-pipeline-utils')` takes about
76
+ 125 ms instead of about 515 ms and loads 31 modules from 9 packages
77
+ instead of 245 from 54 (published bundle, Node 22). `axios`,
78
+ `jsonwebtoken`, `ajv` and `chokidar` now load when the feature that needs
79
+ them is first used (the Anthropic, Cohere and Ollama connectors,
80
+ `JWTValidator`, plugin registration and config validation, hot reload).
81
+ The shared `pluginRegistry` loads its contracts on first use; registries
82
+ created with `new PluginRegistry()` still load them in the constructor.
83
+
84
+ ### Fixed
85
+
86
+ - **`ExecutionTracer.exportTraces('json')`** threw "circular structure"
87
+ with the default `captureOutputs: true`: traces stored the live result,
88
+ which carries the trace. Traces now store JSON snapshots, which also keeps
89
+ later changes to a result out of its recorded trace.
90
+ - **`ParallelEmbedder`** failed with the built-in connectors, which embed
91
+ one string at a time, because it always called `embed(batch)` with an
92
+ array. In the default `'auto'` mode it now embeds texts one by one for
93
+ those connectors and uses `embedBatch(texts)` when an embedder has it;
94
+ other embedders are still called with the array.
95
+
96
+ ## [2.5.0] - 2026-10-28
97
+
98
+ ### Added
99
+
100
+ - **Cancellation**: `createRagPipeline` creates an AbortController per run
101
+ and passes its signal to every component (`retriever.retrieve({ signal })`,
102
+ `embedder.embedQuery/embed(text, { signal })`, `reranker.rerank(q, docs,
103
+ { signal })`, `llm.generate(q, docs, { signal })`); a new `options.signal`
104
+ lets callers cancel. All built-in connectors and rerankers honour it.
105
+ `AgenticPipeline` and `IterativeRetriever` propagate it too.
106
+ - `QueryPlanner.plan(query, { llm, signal })`: LLM-proposed sub-queries with
107
+ strict JSON validation and heuristic fallback (returns a Promise);
108
+ `AgenticPipeline` option `enableLlmPlanning` (default `false`).
109
+ - `checkNumericConsistency`, `extractNumbers`, `detectInjection` and
110
+ `PRICING_METADATA` exports.
111
+ - Opt-in `detectInjection` on `RetrievalGuard` to scan retrieved documents for
112
+ indirect prompt injection.
113
+ - `CostCalculator`: pricing provenance (`source`, `asOf`, `verified`), a
114
+ `pricing` override, one-time unknown-model warnings
115
+ (`RAG_UNKNOWN_MODEL_PRICING`), longest-match model resolution.
116
+ - `EmbeddingReranker` `cacheSize` (LRU of document embeddings) and
117
+ `concurrency` options; `MemoryRetriever` `eviction: 'fifo'` option.
118
+ - Plugin signatures: `computeContentDigest()`, a signed `contentDigest` that
119
+ binds plugin code, opt-in `requireContentDigest` and `requireSignedPlugins`,
120
+ and `register()`'s optional `{ content }` argument.
121
+ - `JWTValidator` options `singleUseTokens` (default `true`) and
122
+ `maxTrackedJtis` (default 100000).
123
+ - `AuditLogger` reads its key from `encryptionKey` or
124
+ `RAG_AUDIT_ENCRYPTION_KEY`; `requireEncryption: true` makes a missing key an
125
+ error.
126
+ - `.env.example` for `docker compose`.
127
+ - `npm run test:serial` and `npm run format:check`.
128
+ - **Working CLI workflow**: `rag-pipeline init`, `ingest <path>`, `query
129
+ <question>` and `run` now do real work against a persistent local index.
130
+ `init` writes a pipeline of built-in components (`--template local |
131
+ openai | ollama | anthropic`, or interactive prompts in a terminal).
132
+ - **`FileVectorStore`** (exported): JSON-file vector index with atomic
133
+ writes, embedder/dimension tracking and per-source re-ingestion.
134
+ - **`.ragrc.json` pipeline resolution**: entries `{ stage, name, config }`
135
+ resolve to built-in components, custom plugin modules under `plugins`, or
136
+ plugins registered on `pluginRegistry`.
137
+ - **`rag-pipeline serve`** and a working **`DevServer`**: local HTTP API
138
+ (`GET /health`, `POST /query`) bound to 127.0.0.1, JSON only, 64 KB body
139
+ limit, no CORS.
140
+ - `LocalEmbedder.getState()` / `setState()`.
141
+ - **`rag-pipeline hub install <source>`** installs plugins from a local
142
+ directory, `.tgz`, single `.js` file, `npm:<package>` or a registry
143
+ (sha256-verified). It contains extraction to `.rag/plugins`, runs a static
144
+ security scan (high-risk findings block unless `--allow-risky`), checks the
145
+ stage contract in a separate process, and registers the plugin in
146
+ `.ragrc.json`. `PluginHub.installPlugin()` uses the same installer.
147
+ - **`rag-pipeline hub certify submit|check <path>`** scores a local plugin
148
+ from its files (contract, security scan, tests, documentation) against the
149
+ BASIC/VERIFIED/ENTERPRISE automated thresholds.
150
+ - **`rag-pipeline hub list`** lists plugins registered in the config.
151
+ - **`rag-pipeline init` setup wizard**: chooses the embedder, LLM, retriever and an optional reranker, using only built-ins or custom plugin modules. Connector options are pre-filled with the connector defaults. Each custom plugin is loaded and its stage method checked before it is accepted. The wizard reports which API key environment variables are missing, can run a connection test on request, and previews the JSON before writing.
152
+ - **`rag-pipeline ai`**, built on the configured pipeline and index:
153
+ `ai train embedding [path]` refits the `local` TF-IDF embedder and
154
+ re-embeds the index; `ai benchmark` reports measured p50/p95/mean latency
155
+ per stage; `ai feedback <question> <docId> --rating up|down` stores votes in
156
+ `.rag/feedback.json` and `ai search` applies them as a capped,
157
+ explained score adjustment; `ai multimodal process <path>` reports type,
158
+ characters and chunks for text files (images exit 1: no built-in connector
159
+ accepts image input).
160
+
161
+ ### Deprecated
162
+
163
+ - **Experimental modules moved to `@devilsdev/rag-pipeline-utils/experimental`**:
164
+ `ModelTrainingOrchestrator`, `FederatedLearningCoordinator`,
165
+ `MultiModalProcessor`, `AdaptiveRetrievalEngine`, `SSOManager`,
166
+ `SAMLProvider`, `OAuth2Provider`, `ActiveDirectoryProvider`, `OIDCProvider`,
167
+ `TenantManager`, `ResourceMonitor`, `DataGovernanceManager` /
168
+ `DataGovernance`, `PluginSandbox` and `PluginCertification`. They are
169
+ simulated or incomplete (random metrics, SSO that accepts any code outside
170
+ production and does not verify SAML or OIDC tokens, random quotas, erasure
171
+ that deletes nothing, a non-functional sandbox, fixed certification
172
+ scores); see the Experimental docs page. Their main-entry exports still
173
+ work but emit a one-time `DeprecationWarning` (`RAG_DEP_*` codes) when
174
+ used, are marked `@deprecated` in the types, and will be removed from the
175
+ main entry in the next major release. The same classes are shared by both
176
+ entries, so `instanceof` checks keep working.
177
+
178
+ ### Removed
179
+
180
+ - Unused CLI config templates under `src/cli/templates/` (old format,
181
+ referenced a schema path that did not exist).
182
+ - Ten unreachable duplicate modules under `src/ai/` and the internal
183
+ `src/dag/enhanced-dag-engine.js` (never reachable from the package entries;
184
+ the exported classes are unchanged).
185
+ - semantic-release configuration and dependencies (releases use
186
+ `npm publish --provenance` from the release workflow), duplicate
187
+ `.eslintrc.json` and `.babelrc`, about 50 one-off fix scripts and their
188
+ `ci:*` npm scripts, the tracked `junit.xml`, and 14 test suites that never
189
+ exercised `src/` (fake pipelines, mocks, empty timing loops).
190
+ - CLI placeholders that printed success without doing anything (`ingest`,
191
+ `query`, `doctor`, `validate`, `plugin search|install|publish`) and
192
+ options that had no effect (`--trace`, `--stats`, `--parallel`,
193
+ `--streaming`, `--max-memory`, `--preview`, `--validate`, `--explain`,
194
+ `--abort-timeout`, `--quiet`, `config show --format`).
195
+ - The interactive wizard built on hard-coded mock registry data.
196
+ - The unregistered `src/cli/commands/ai-ml*` command modules: federated
197
+ learning, LLM fine-tuning, `ai train status|deploy`, `ai dashboard`,
198
+ adaptive profiles and multimodal `search`/`describe`. They called methods
199
+ missing from the `src/ai` engines, exited 0 on failure and printed
200
+ hard-coded or simulated numbers. The `src/ai` library modules are unchanged.
201
+ - Unreachable modules that were never exported or registered: `src/dx/`
202
+ (including a realtime debugger exposing an unauthenticated WebSocket),
203
+ `src/cli/commands/{dx,docs}.js`, `src/cli/input-sanitization-middleware.js`,
204
+ `src/cli/plugin-marketplace-commands.js`, `src/query.js`, `src/ingest.js`,
205
+ `src/pipeline/pipeline.js`, `src/core/pipeline-factory.js`. The package
206
+ `exports` map only exposes `.` and `./cli`, so these were never public.
207
+
208
+ ### Fixed
209
+
210
+ - **`evaluateRagDataset`** scores the generated answer; it scored the
211
+ retrieved documents, so every case failed with zero scores.
212
+ - **`RetrievalGuard`** with `enforceACL` also reads a top-level `acl` /
213
+ `requiredRole` on results that have `metadata` (it read only `metadata`
214
+ then).
215
+ - **`createRagPipeline`**: timeouts now cancel work and no stage starts after
216
+ an abort (a timeout still resolves `{ success: false, error: 'Execution
217
+ timeout' }`; a caller abort resolves `'Execution aborted'`). Without an llm
218
+ it runs retrieval only and returns `answer: null` instead of `'ok'`. Stream
219
+ mode with `citations` or `evaluate` ends with one `{ token: '', done: true,
220
+ citations, evaluation }` chunk. The artificial 5 ms delay is removed.
221
+ - **Connectors** retry only transient failures (network, 408/409/425/429/5xx)
222
+ with exponential backoff, jitter and `Retry-After`; SDK-internal retries are
223
+ disabled. API keys are read at construction, and calls without a key fail
224
+ fast with `MISSING_API_KEY`.
225
+ - **`MemoryRetriever`** enforces `maxDocuments` and rejects mismatched vector
226
+ dimensions instead of truncating them.
227
+ - **`AuditLogger`** no longer encrypts with a random per-process key that made
228
+ logs unreadable after a restart; legacy `createCipher` records decrypt on
229
+ Node 22+.
230
+ - **`JWTValidator` / `jwt-config`** honour an explicit `clockTolerance` /
231
+ `notBeforeTolerance` of 0 (it became 30); the expired-token test is enabled.
232
+ - **`loadPluginsFromJson`** awaits registration and verifies signed manifests
233
+ and plugin files before importing them.
234
+ - **`rag-pipeline init`**: the connection test aborts its calls after the 15 s
235
+ timeout and the CLI exits afterwards.
236
+ - **Benchmarks** accept `--iterations=N`, no longer abort on the retry
237
+ benchmarks' deliberate warm-up failures, and exit when done.
238
+ - **Docs generation** no longer hangs: the JSDoc regex backtracked
239
+ exponentially on an unterminated `/**`.
240
+ - **Tests** run in parallel again (a suite destroyed the Jest worker's own
241
+ stdio streams), no longer rewrite tracked fixtures, and coverage works on
242
+ Windows (the test setup no longer patches `path.resolve`).
243
+ - **Package import**: `require()` / `import` of 2.4.5 failed in every clean
244
+ install with `Cannot find module 'isolated-vm'`. `isolated-vm` is now
245
+ loaded only when `PluginSandbox` runs and is an optional peer dependency;
246
+ `pino-pretty` is an optional peer as well. Added the missing `ajv-formats`
247
+ and `marked` dependencies, pinned `chalk` to 4.x (5.x is ESM-only), removed
248
+ unused `dotenv`. Importing the bundle no longer prints "Missing Contract"
249
+ warnings. A tarball smoke test (`npm run smoke:pack`) now runs in CI and
250
+ before publish.
251
+ - **`createRagPipeline`**: the `embedder` result was computed and discarded,
252
+ so vector retrievers never received a `queryVector`. Plugins registered on
253
+ the exported `pluginRegistry` are now resolvable by name, and the timeout
254
+ timer is cleared when a run finishes.
255
+ - **`MemoryRetriever`** throws a clear error when no `queryVector` is given
256
+ instead of crashing inside cosine similarity.
257
+ - **README / docs quick start** now passes an `embedder`; as previously
258
+ written it could not retrieve anything from `MemoryRetriever`.
259
+ - **Citations** with zero retrieved documents no longer fail the run; every
260
+ sentence is reported as unsupported.
261
+ - **`HybridRetriever`** passes the full request (including `queryVector`) to
262
+ `retrieve()`, so vector results are no longer silently dropped, and throws
263
+ when every retriever fails.
264
+ - **Guardrails**: the retrieval guard (ACL, relevance, freshness) now runs
265
+ before generation via the new `options.filterResults` hook; removed
266
+ documents are reported by id only; strict-mode blocks no longer return the
267
+ raw answer; PII is redacted in citations and evaluation details too.
268
+ - **`AuditLogger`**: tenant/user/action filters never matched, so queries and
269
+ compliance reports returned every tenant's records. Encryption now uses
270
+ `createCipheriv` with a random IV per field (it threw on Node 22 and reused
271
+ the nonce on Node 18/20). The integrity hash now covers action and tenant.
272
+ - **`InputSanitizer`**: a regex state bug let command-injection payloads
273
+ through after a previously blocked input.
274
+ - **`LLMReranker`** sends an actual ranking prompt instead of the raw
275
+ question, keeps documents the LLM omits, and honours `topK`.
276
+ - **Agentic RAG**: with an `embedder`, each sub-query and refined query is
277
+ embedded on its own instead of reusing the original query vector.
278
+ - **TypeScript**: 52 runtime exports (citations, guardrails, hybrid
279
+ retrieval, agentic RAG, connectors, rerankers, GraphRAG, ...) had no
280
+ declarations. They are now generated from JSDoc at build time into
281
+ `dist/types/` and re-exported from `dist/index.d.ts`; the build fails if
282
+ any export is undeclared, and the smoke test compiles a TypeScript consumer.
283
+ - **`doctor` / `validate`** report real findings (unknown components,
284
+ missing stages, missing API keys, broken custom plugins) and exit 1 on
285
+ errors. `doctor --auto-fix` no longer lets callers re-enable a shell, no
286
+ longer strips Windows path separators, and runs npm on Windows.
287
+ - **`hub`** (alias `plugin`) commands are registered. There is no default
288
+ registry (the old default host does not exist): pass `--registry <url>` or
289
+ set `RAG_PLUGIN_REGISTRY` for registry operations.
290
+ - **Hot reload** ignore patterns work again (chokidar 4 dropped glob support)
291
+ and only JavaScript changes trigger a reload.
292
+ - `loadRagConfig()` accepts a config file path as well as a directory and no
293
+ longer logs to stdout.
294
+ - `LocalEmbedder` produced all-zero vectors for corpora of two or fewer
295
+ documents (IDF is now smoothed).
296
+ - **Logger**: console-style calls such as `logger.error("Failed:", msg)` no
297
+ longer log the message as one key per character.
298
+
299
+ ### Changed
300
+
301
+ - **Documentation**: the README is rewritten with use cases whose code runs
302
+ in CI (`__tests__/docs/readme-examples.test.js` executes every README
303
+ example), and `.ragrc.json`'s JSON schema (`.ragrc.schema.json`, shipped in
304
+ the package) now describes the `pipeline` + `plugins` format the CLI and
305
+ `loadRagConfig()` actually use.
306
+ - **Metric changes**:
307
+ - Context recall now measures how much of a reference (ground-truth) answer
308
+ the retrieved documents support. Pass it via
309
+ `PipelineEvaluator.evaluate(result, { referenceAnswer })` or
310
+ `run({ options: { evaluate: true, referenceAnswer } })`. Without one,
311
+ `scores.contextRecall` is omitted, `details.contextRecall` is
312
+ `{ score: null, skipped: true, reason }`, and `metadata.metricsSkipped`
313
+ lists it.
314
+ - `computeGroundedness` is a weighted mean over the available components;
315
+ without a reference it is `(0.4 * faithfulness + 0.3 * contextPrecision)
316
+ / 0.7`.
317
+ - Claim support checks numbers: a sentence whose numbers (digits, decimals,
318
+ percentages, number words) are not in the source scores below 0.4 and is
319
+ not grounded. This affects citations, faithfulness, recall,
320
+ `PostGenerationGuard` and `SelfCritiqueChecker`. "20" equals "twenty".
321
+ - **`createRagPipeline`** results always include `answer` when an llm is
322
+ configured.
323
+ - **`QueryPlanner`** splits only at clause or question boundaries, keeps
324
+ "salt and pepper"-style phrases intact, and expands comparisons into one
325
+ sub-query per entity plus the comparison; results carry `source`.
326
+ - **Prompts**: built-in connectors put retrieved context in the user message
327
+ as delimited, untrusted reference data, with a system prompt telling the
328
+ model to treat it as data. Answers may differ.
329
+ - **`PreRetrievalGuard`** returns the query unchanged (no partial deletion) and
330
+ reports detections in `injection`; strict mode still blocks. Detection
331
+ normalises input first. To cut false positives, bare "act as", "jailbreak",
332
+ uppercase "DAN", mid-line "system:" and "you are now ready" are no longer
333
+ flagged; chat role markers, prompt-exfiltration phrasing and normalisation
334
+ bypasses now are.
335
+ - **`CostCalculator`** prices `gpt-4o-mini-*` and `claude-3-5-sonnet-*`
336
+ correctly (previously gpt-4 pricing and $0); `customPricing` entries without
337
+ `unit` default to per-1000 tokens. Built-in prices are marked unverified:
338
+ their `asOf` is when they were added, not when they were checked.
339
+ - **`EmbeddingReranker`** limits concurrency to 4 by default; call
340
+ `clearCache()` after retraining an embedder.
341
+ - **`AuditLogger`** with encryption enabled and no key now disables
342
+ encryption with a warning instead of using a throwaway key.
343
+ - **Plugin trust store** is empty by default (the placeholder key is gone).
344
+ - **`docker compose`** requires `POSTGRES_PASSWORD`, `REDIS_PASSWORD` and
345
+ `GRAFANA_ADMIN_PASSWORD`, publishes ports on 127.0.0.1 only, and drops the
346
+ debugger port and the Prometheus lifecycle API; the image build fails on
347
+ critical runtime vulnerabilities.
348
+ - **CI**: one `ci.yml` workflow (required checks `Test & Lint (18|20|22)`)
349
+ that also enforces `format:check` and the tarball smoke test, and runs for
350
+ pull requests into any branch. `npm test` runs Jest in parallel.
351
+ - **`DevServer`** now really listens: `start()` binds port 3000 by default
352
+ (pass `port: 0` for a free port), so existing callers may see
353
+ `EADDRINUSE`. Default host is `127.0.0.1` (was `localhost`), default
354
+ `watchPaths` is `[]` (was `./src/plugins`, `./src/mocks`), and requests
355
+ must use a loopback `Host` header while bound to loopback.
356
+ - **`loadRagConfig()`** error messages now include details
357
+ (`Config validation failed: ...`, `Invalid JSON in config file <path>: ...`)
358
+ and it no longer logs.
359
+ - **`rag-pipeline plugin` / `hub`** registry operations require
360
+ `--registry <url>` or `RAG_PLUGIN_REGISTRY`. `hub certify submit` takes a
361
+ local plugin path and reports computed scores; it previously issued
362
+ certificates from fixed placeholder scores. `PluginHub.installPlugin()` now
363
+ installs into `<project>/.rag/plugins` and registers the plugin in the
364
+ config (it previously only wrote `metadata.json` to `localCacheDir`), and
365
+ `getInstalledPlugins()` reads the config.
366
+ - **`FileVectorStore`** index files store mostly-zero vectors sparsely and
367
+ round values to 7 significant digits (about 10x smaller for the `local`
368
+ embedder).
369
+
370
+ - **Grounding scores** (citations, faithfulness, context precision/recall,
371
+ groundedness, `PostGenerationGuard`, `SelfCritiqueChecker`) now measure the
372
+ share of a sentence's content words found in a document, optionally
373
+ IDF-weighted. They previously used Jaccard similarity against the whole
374
+ document, which shrinks with document length: an answer copied verbatim
375
+ from a ~70-word chunk scored 0. Scores will be noticeably higher for
376
+ grounded answers.
377
+ - Default per-sentence similarity threshold raised from 0.3 to 0.5
378
+ (`CitationTracker.similarityThreshold`, `PipelineEvaluator.threshold`,
379
+ faithfulness/recall/hallucination-detector `threshold`). Aggregate
380
+ thresholds (`groundednessThreshold`, `approvalThreshold`) are unchanged.
381
+ - The tokenizer keeps non-ASCII letters (`Política` no longer becomes
382
+ `poltica`) and splits on newlines and hyphens instead of gluing words
383
+ together. This also affects BM25, BLEU/ROUGE and the scoring reranker.
384
+ - `RetrievalGuard` with `enforceACL` now fails closed when
385
+ `context.userPermissions` is missing.
386
+ - `CitationTracker.track()` non-array error message is now
387
+ "retrievedDocs must be an array".
388
+
389
+ ### Security
390
+
391
+ - Production dependencies: `npm audit --omit=dev` reports no high or critical
392
+ findings (axios ^1.21.0; overrides for fast-uri, follow-redirects,
393
+ form-data, ws and zod).
394
+ - Experimental SSO: the development bypass that accepted any authorization
395
+ code requires `devBypass: true` and a non-production `NODE_ENV` and is
396
+ refused in production; OIDC id_tokens are validated (JWKS signature, iss,
397
+ aud, exp, nonce); OAuth2/OIDC use PKCE (S256); `redirect_uri` comes only
398
+ from configuration; callbacks must match the stored login request's
399
+ provider.
400
+ - Plugin signatures: Ed25519 verification works (`crypto.verify` with
401
+ KeyObjects); a signed `contentDigest` binds plugin code and is always
402
+ enforced when present.
403
+ - `AuditLogger` compares HMAC signatures in constant time; `JWTValidator`'s
404
+ replay store is bounded and expires entries.
405
+ - Retrieved documents are no longer placed in the LLM system role; prompt
406
+ injection detection normalises homoglyphs, leetspeak and zero-width
407
+ characters.
408
+ - `.secrets` is no longer tracked in git (rotate anything it held) and is
409
+ excluded from Docker builds.
410
+
411
+ ### Known limitations
412
+
413
+ - Grounding is lexical plus a numeric-consistency check. Not checked:
414
+ negation ("not twenty days"), units ("20 hours" vs "20 days"), ordinals, a
415
+ lone "one", differently split decimals, and claims that combine facts from
416
+ several documents. Document-side matching is lenient ("twenty-five" in a
417
+ source also supports "twenty" or "five").
418
+ - The `openai` SDK stays on 4.x: newer majors need Node 20+ and will come with
419
+ 3.0. `csv-parse` has one moderate advisory whose fix is a major release.
420
+ - Experimental SSO cannot refresh OIDC sessions, and `SAMLProvider` refuses
421
+ to authenticate.
422
+ - `AuditLogger`'s HMAC signing key is random per process unless `signingKey`
423
+ is passed, so `verifyIntegrity` flags records written before a restart.
424
+
425
+ ## [2.4.5] - 2026-04-19
426
+
427
+ ### Changed
428
+
429
+ - **Supply chain**: first release published via GitHub Actions OIDC trusted
430
+ publishing with `--provenance` attached. Tarball now ships with a SLSA
431
+ build provenance attestation signed via Sigstore; consumers can verify
432
+ the git-commit → tarball-bytes chain with `npm audit signatures`. No
433
+ long-lived `NPM_TOKEN` involved.
434
+ - Publish workflow (`.github/workflows/release.yml`): switched from
435
+ `NODE_AUTH_TOKEN` to OIDC, upgraded npm CLI to 11.5.1+ at job start,
436
+ added `--provenance` and `--ignore-scripts` to the publish step.
437
+
438
+ No API, behavior, or documented-surface changes from 2.4.4.
439
+
12
440
  ## [2.4.4] - 2026-04-16
13
441
 
14
442
  ### Added
@@ -124,6 +552,9 @@ First public release on the 2.4 line, introducing the v2.4 feature set.
124
552
  For release notes prior to v2.4.0, see
125
553
  [github.com/DevilsDev/rag-pipeline-utils/releases](https://github.com/DevilsDev/rag-pipeline-utils/releases).
126
554
 
555
+ [2.6.0]: https://github.com/DevilsDev/rag-pipeline-utils/releases/tag/v2.6.0
556
+ [2.5.0]: https://github.com/DevilsDev/rag-pipeline-utils/releases/tag/v2.5.0
557
+ [2.4.5]: https://github.com/DevilsDev/rag-pipeline-utils/releases/tag/v2.4.5
127
558
  [2.4.4]: https://github.com/DevilsDev/rag-pipeline-utils/releases/tag/v2.4.4
128
559
  [2.4.3]: https://github.com/DevilsDev/rag-pipeline-utils/releases/tag/v2.4.3
129
560
  [2.4.2]: https://github.com/DevilsDev/rag-pipeline-utils/releases/tag/v2.4.2