@beingmartinbmc/ojas 0.2.0 → 0.3.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +327 -167
- package/docs/BACKLOG.md +2 -2
- package/docs/EVIDENCE.md +151 -9
- package/docs/EVIDENCE_MATRIX.md +97 -8
- package/docs/MCP.md +2 -2
- package/package.json +2 -1
package/README.md
CHANGED
|
@@ -6,8 +6,8 @@
|
|
|
6
6
|
|
|
7
7
|
**AI Health Infrastructure for Autonomous Agents**
|
|
8
8
|
|
|
9
|
-
[](https://github.com/beingmartinbmc/ojas/actions/workflows/ci.yml)
|
|
10
|
+
[](https://www.npmjs.com/package/@beingmartinbmc/ojas)
|
|
11
11
|
[](#operations)
|
|
12
12
|
[](docs/MCP.md)
|
|
13
13
|
[](#quickstart)
|
|
@@ -16,59 +16,34 @@
|
|
|
16
16
|
|
|
17
17
|
</div>
|
|
18
18
|
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
Traditional observability tells you whether software is running. Ojas tries to tell you whether an agent is still *cognitively healthy enough to continue operating* — and is honest about where that signal is strong vs. where it is heuristic.
|
|
22
|
-
|
|
23
|
-
It introduces a new infrastructure category: **AI Health Systems**.
|
|
24
|
-
|
|
25
|
-
Deployment trust boundary, security posture, and evidence caveats live in [`docs/TRUST.md`](./docs/TRUST.md).
|
|
26
|
-
|
|
27
|
-
<a id="what-is-proven"></a>
|
|
28
|
-
### What is currently proven
|
|
29
|
-
|
|
30
|
-
Ojas v0.3 ships at **evidence level L2 / L2.5** — synthetic, reproducible
|
|
31
|
-
A/B benchmarks against controlled stand-in agents on canonical failure
|
|
32
|
-
modes. Each claim below has a repro command and a named limitation; the
|
|
33
|
-
full matrix lives in [`docs/EVIDENCE_MATRIX.md`](./docs/EVIDENCE_MATRIX.md),
|
|
34
|
-
and known failure modes in [`docs/KNOWN_FAILURES.md`](./docs/KNOWN_FAILURES.md).
|
|
35
|
-
|
|
36
|
-
| Claim | Value | Evidence | Repro |
|
|
37
|
-
|---|---:|---|---|
|
|
38
|
-
| Prompt-injection compliance reduction | 58% → 0% (−100%) | L2 / 33 attacks (incl. homoglyph, zero-width, full-width, letter-spaced, base64, policy-laundering variants) | `npm run benchmark` |
|
|
39
|
-
| Attacks quarantined by Raksha detector stack | **100%** (33/33) | L2 | `npm run benchmark` |
|
|
40
|
-
| Benign false-positive rate (30 controls across 5 categories) | **0%** — tolerance ≤ 5% | L2 | `npm run benchmark` |
|
|
41
|
-
| Health-score calibration: monotonic vs failure rate; ρ = −0.31 over 500 trials; score spans [0.31, 0.87]; isotonic Brier 0.230 → 0.219 | L2.5 diagnostic, not probability | L2.5 | `npm run benchmark` |
|
|
42
|
-
| Malicious memory writes committed | 6/6 → 1/6 (83% blocked) | L2 / 16 candidates | `npm run benchmark` |
|
|
43
|
-
| Wasted-token reduction (noisy retrieval) | −62% | L2 | `npm run benchmark` |
|
|
44
|
-
| Wasted-token reduction (heavy retrieval) | −95% | L2 | `npm run benchmark` |
|
|
45
|
-
| Tool-failure loop detection speedup | 10× faster | L2 / 3 scripted tools | `npm run benchmark` |
|
|
46
|
-
| Retrieval-QA task success rate (baseline → Ojas) | 35% → 95%, bootstrap 95 % CI across 5 seeds × 20 questions | **L2.5** | `npm run benchmark` |
|
|
47
|
-
| Retrieval-QA adversarial inclusion (lower is better) | 100% → 11%, same CI methodology | **L2.5** | `npm run benchmark` |
|
|
48
|
-
| Retrieval-QA relevant-doc recall preserved | 100% (no Aahar false positives in this run) | **L2.5** | `npm run benchmark` |
|
|
19
|
+
---
|
|
49
20
|
|
|
50
|
-
|
|
51
|
-
|
|
21
|
+
Ojas adds a continuous health layer to autonomous AI agents — context
|
|
22
|
+
hygiene, prompt-injection tripwires, drift detection, recovery diagnosis,
|
|
23
|
+
and stress probes.
|
|
52
24
|
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
25
|
+
Traditional observability tells you whether software is running.
|
|
26
|
+
Ojas tries to tell you whether an agent is still *cognitively healthy
|
|
27
|
+
enough to continue operating* — and is honest about where that signal
|
|
28
|
+
is strong vs. where it is heuristic.
|
|
56
29
|
|
|
57
|
-
|
|
30
|
+
It introduces a new infrastructure category: **AI Health Systems**.
|
|
58
31
|
|
|
59
32
|
---
|
|
60
33
|
|
|
61
34
|
<a id="demo"></a>
|
|
62
|
-
## Quick demo: one failure mode, before and after
|
|
63
35
|
|
|
64
|
-
|
|
36
|
+
## 30-second demo
|
|
37
|
+
|
|
38
|
+
A common agent failure: **noisy retrieval + prompt injection**.
|
|
39
|
+
The agent gets 8 retrieved documents — one hostile, most irrelevant.
|
|
40
|
+
Run the same task through a tiny deterministic agent twice, once raw and
|
|
41
|
+
once through `ojas.feed()`:
|
|
65
42
|
|
|
66
43
|
```bash
|
|
67
44
|
npm run demo:before-after
|
|
68
45
|
```
|
|
69
46
|
|
|
70
|
-
Example output:
|
|
71
|
-
|
|
72
47
|
```text
|
|
73
48
|
Task: What is the refund window for Pro plans?
|
|
74
49
|
Retrieved 8 docs (1 answer-bearing, 2 adjacent, 4 noisy, 1 adversarial).
|
|
@@ -86,151 +61,145 @@ Baseline answer:
|
|
|
86
61
|
|
|
87
62
|
Answer with Ojas:
|
|
88
63
|
Pro plans have a 14-day refund window from the purchase date (source: kb-policies).
|
|
89
|
-
|
|
90
|
-
Why Ojas changed the context (Pulse events):
|
|
91
|
-
• raksha/prompt_injection_quarantined severity=critical
|
|
92
|
-
• aahar/context_items_rejected severity=warning
|
|
93
64
|
```
|
|
94
65
|
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
4. **Did the final answer stay grounded?** → cites `kb-policies`, not a hallucinated source.
|
|
101
|
-
5. **Did Ojas explain itself?** → emitted Pulse events name *why* each item was removed (Raksha quarantine vs Aahar nutrition reject); the prompt is not silently rewritten.
|
|
66
|
+
1. Ojas removed the malicious document → `injection_included` flips `yes → no`.
|
|
67
|
+
2. It preserved the relevant policy doc → `result` flips `failed → passed`.
|
|
68
|
+
3. Token count dropped from 235 to 40.
|
|
69
|
+
4. The final answer stays grounded and cites `kb-policies`.
|
|
70
|
+
5. Pulse events explain *why* each item was removed — the prompt is not silently rewritten.
|
|
102
71
|
|
|
103
|
-
Source: [`examples/before-after.ts`](./examples/before-after.ts) — no
|
|
72
|
+
Source: [`examples/before-after.ts`](./examples/before-after.ts) — no
|
|
73
|
+
external deps, no API keys. Demo caveats: [`docs/TRUST.md`](./docs/TRUST.md).
|
|
104
74
|
|
|
105
75
|
---
|
|
106
76
|
|
|
107
|
-
<a id="
|
|
108
|
-
## Why Ojas Exists
|
|
109
|
-
|
|
110
|
-
Autonomous agents are no longer simple request–response systems. They plan, retrieve, remember, call tools, revise goals, and operate across long sessions.
|
|
77
|
+
<a id="canonical-pipeline"></a>
|
|
111
78
|
|
|
112
|
-
|
|
79
|
+
## Canonical Pipeline (12-Step Agent Health Loop)
|
|
113
80
|
|
|
114
|
-
-
|
|
115
|
-
- noisy retrieval pollutes reasoning
|
|
116
|
-
- memory stores stale or unsafe information
|
|
117
|
-
- tool failures create loops and retry storms
|
|
118
|
-
- long sessions cause drift and contradiction
|
|
119
|
-
- prompt injection manipulates agent behavior
|
|
120
|
-
- bigger context windows amplify noise instead of solving it
|
|
121
|
-
- production agents can degrade silently without obvious runtime errors
|
|
81
|
+
The full call order an Ojas-instrumented runtime should follow on every agent turn:
|
|
122
82
|
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
<a id="what"></a>
|
|
130
|
-
## What Ojas Does
|
|
131
|
-
|
|
132
|
-
Ojas wraps an agent runtime with a continuous health cycle:
|
|
133
|
-
|
|
134
|
-
1. **Cleans and ranks context** before the agent consumes it
|
|
135
|
-
2. **Scans for canonical and semantic prompt-injection patterns** and unsafe memory writes *(deterministic detector stack; see [known failures](./docs/KNOWN_FAILURES.md))*
|
|
136
|
-
3. **Tracks cognitive vital signs** during execution
|
|
137
|
-
4. **Measures token, latency, and tool-use efficiency**
|
|
138
|
-
5. **Detects drift, loops, instability, and degradation**
|
|
139
|
-
6. **Consolidates execution traces** into useful memory
|
|
140
|
-
7. **Stress-tests agents** against hostile or unstable conditions, with **AbortSignal cancellation** on timeout
|
|
141
|
-
8. **Diagnoses failures** and recommends recovery protocols
|
|
142
|
-
|
|
143
|
-
> Ojas helps agents think with cleaner inputs, recover from failure, and become more reliable over time.
|
|
144
|
-
|
|
145
|
-
---
|
|
146
|
-
|
|
147
|
-
## The Seven Modules
|
|
148
|
-
|
|
149
|
-
Seven specialised modules. One unified health score.
|
|
150
|
-
|
|
151
|
-
| Module | Role | Headline signals |
|
|
152
|
-
|---|---|---|
|
|
153
|
-
| 🥗 **[Aahar](docs/MODULES.md#aahar)** | Cognitive nutrition (context curation) | signal-to-noise, freshness, token efficiency |
|
|
154
|
-
| 😴 **[Nidra](docs/MODULES.md#nidra)** | Recovery & memory consolidation | drift score, processed-trace coverage |
|
|
155
|
-
| 💪 **[Vyayam](docs/MODULES.md#vyayam)** | Resilience & stress engineering | hallucination resistance under load, recovery time |
|
|
156
|
-
| 🛡️ **[Raksha](docs/MODULES.md#raksha)** | Immune defense: deterministic detector stack + async ML classifier plugins | threat resistance (residual risk after quarantine) |
|
|
157
|
-
| 🔥 **[Agni](docs/MODULES.md#agni)** | Cognitive metabolism | token efficiency, latency, tool economy, cost pressure |
|
|
158
|
-
| 📈 **[Pulse](docs/MODULES.md#pulse)** | Continuous health telemetry | structured events bus with per-module severity |
|
|
159
|
-
| 🩺 **[Chikitsa](docs/MODULES.md#chikitsa)** | Repair & rehabilitation | repair readiness, rollback safety, playbook coverage |
|
|
160
|
-
|
|
161
|
-
Each maps to an analogue of a human-health system — nutrition, sleep, exercise, immunity, metabolism, vital signs, and rehabilitation.
|
|
162
|
-
|
|
163
|
-
---
|
|
83
|
+
```
|
|
84
|
+
register → ingest traces → score/build context → scan for injection →
|
|
85
|
+
recommend model route → detect hallucination / distill → record outcome →
|
|
86
|
+
fitness gate → diagnose/recover if unhealthy → consolidate memory →
|
|
87
|
+
audit memory → handoff/report
|
|
88
|
+
```
|
|
164
89
|
|
|
165
|
-
|
|
90
|
+
Run the end-to-end demo:
|
|
166
91
|
|
|
167
|
-
|
|
92
|
+
```bash
|
|
93
|
+
npm run demo:canonical
|
|
94
|
+
```
|
|
168
95
|
|
|
169
|
-
|
|
|
170
|
-
|
|
171
|
-
|
|
|
172
|
-
|
|
|
173
|
-
|
|
|
174
|
-
|
|
|
175
|
-
|
|
|
176
|
-
|
|
|
177
|
-
|
|
|
178
|
-
|
|
|
179
|
-
|
|
|
180
|
-
|
|
|
96
|
+
| Step | Operation | Ojas API |
|
|
97
|
+
|------|-----------|----------|
|
|
98
|
+
| 1 | Register agent | `new Ojas(config)` + `ojas.bind(agent)` |
|
|
99
|
+
| 2 | Ingest traces | `ojas.recordTrace(trace)` |
|
|
100
|
+
| 3 | Score / build context | `ojas.feed(items, { query })` |
|
|
101
|
+
| 4 | Scan for injection | Raksha (runs inside `feed()`) |
|
|
102
|
+
| 5 | Recommend model route | `ConfidenceRoutingTable.recommend()` |
|
|
103
|
+
| 6 | Detect hallucination / distill | `raksha.detectHallucination()` + `createResponseDistiller()` |
|
|
104
|
+
| 7 | Record outcome | `chikitsa.recordTaskOutcome()` |
|
|
105
|
+
| 8 | Fitness gate | `ojas.healthCheck()` vs threshold |
|
|
106
|
+
| 9 | Diagnose / recover | `chikitsa.diagnose()` + `ojas.recover()` |
|
|
107
|
+
| 10 | Consolidate memory | `ojas.recover(true)` (Nidra) |
|
|
108
|
+
| 11 | Audit memory | `nidra.getMemories()` + `detectColdMemories()` |
|
|
109
|
+
| 12 | Handoff / report | `chikitsa.generateHandoff()` |
|
|
110
|
+
|
|
111
|
+
Source: [`examples/canonical-pipeline.ts`](./examples/canonical-pipeline.ts) — no external deps, no API keys.
|
|
181
112
|
|
|
182
113
|
---
|
|
183
114
|
|
|
184
115
|
<a id="quickstart"></a>
|
|
116
|
+
|
|
185
117
|
## Quick Start
|
|
186
118
|
|
|
187
|
-
|
|
119
|
+
### Install from npm
|
|
188
120
|
|
|
189
121
|
```bash
|
|
190
122
|
npm install @beingmartinbmc/ojas
|
|
191
123
|
```
|
|
192
124
|
|
|
193
|
-
|
|
125
|
+
### Or clone for development
|
|
194
126
|
|
|
195
127
|
```bash
|
|
128
|
+
git clone https://github.com/beingmartinbmc/ojas.git
|
|
129
|
+
cd ojas
|
|
196
130
|
npm install
|
|
197
131
|
npm run build
|
|
198
|
-
npm run demo # end-to-end walkthrough across all seven modules
|
|
199
|
-
npm run benchmark # A/B evidence harness
|
|
200
132
|
npm test # 595 tests across 33 suites
|
|
201
|
-
npm run
|
|
133
|
+
npm run benchmark # A/B evidence harness
|
|
202
134
|
```
|
|
203
135
|
|
|
204
|
-
|
|
136
|
+
---
|
|
137
|
+
|
|
138
|
+
## Quality Gates
|
|
139
|
+
|
|
140
|
+
```bash
|
|
141
|
+
npm run check # lint + build + typecheck + tests
|
|
142
|
+
npm run benchmark # deterministic A/B evidence harness (11 suites)
|
|
143
|
+
npm run verify:evidence # checks committed docs match latest benchmark run
|
|
144
|
+
```
|
|
145
|
+
|
|
146
|
+
CI runs all three on every push and PR.
|
|
147
|
+
|
|
148
|
+
---
|
|
149
|
+
|
|
150
|
+
<a id="benchmark-snapshot"></a>
|
|
151
|
+
|
|
152
|
+
## Latest Benchmark Snapshot
|
|
153
|
+
|
|
154
|
+
| Benchmark | Baseline | Ojas | Delta |
|
|
155
|
+
|---|---:|---:|---:|
|
|
156
|
+
| Retrieval-QA task success | 35% | 95% | **+60 pp** |
|
|
157
|
+
| Adversarial doc inclusion | 100% | 11% | **−89 pp** |
|
|
158
|
+
| Prompt-injection compliance (51 attacks) | 52.9% | 3.9% | **−92.6%** |
|
|
159
|
+
| Injection detection p99 latency | — | 1.43 ms | **L2** |
|
|
160
|
+
| Wasted tokens (heavy retrieval) | 12,680 | 680 | **−94.6%** |
|
|
161
|
+
| Malicious memory writes | 6/6 | 1/6 | **−83%** |
|
|
162
|
+
| Tool-failure detection speed | 20 calls | 2 calls | **10×** |
|
|
163
|
+
| Hallucination detection (fabricated) | 0% | 100% TPR | **L2** |
|
|
164
|
+
| Hallucination false-positive rate | — | 0% | **L2** |
|
|
165
|
+
| Model router fail-closed | — | 100% flagship on sparse | **L2** |
|
|
166
|
+
| Response distiller code-safe | — | 100% code blocks | **L2** |
|
|
167
|
+
| Distiller intensity monotonicity | — | lite ≤ full ≤ ultra | **L2** |
|
|
168
|
+
| MCP envelope compliance | — | 18/18 tools | **L2** |
|
|
169
|
+
| Fitness gate consistency | — | 100% | **L2** |
|
|
170
|
+
| Fitness gate risk-boost monotonicity | — | 100% | **L2** |
|
|
171
|
+
| Memory write 4-tier policy | — | 97% tier accuracy | **L2** |
|
|
172
|
+
| Recovery protocol coverage | — | 7/7 types, 9/9 actions | **L2** |
|
|
173
|
+
| Health-score calibration (Spearman ρ) | — | −0.313 | **L2.5** |
|
|
174
|
+
| Threshold-band accuracy | — | 84.8% | **L2** |
|
|
175
|
+
|
|
176
|
+
Overall: 18/18 suites pass. 18 suites total, 748 ms. All numbers from deterministic synthetic benchmarks (`npm run benchmark`).
|
|
177
|
+
Full methodology and per-suite breakdowns: [`docs/EVIDENCE.md`](./docs/EVIDENCE.md).
|
|
205
178
|
|
|
206
179
|
---
|
|
207
180
|
|
|
208
181
|
<a id="usage"></a>
|
|
209
|
-
## Basic Usage
|
|
210
182
|
|
|
211
|
-
|
|
183
|
+
## Basic Usage
|
|
212
184
|
|
|
213
185
|
```typescript
|
|
214
186
|
import { Ojas } from '@beingmartinbmc/ojas';
|
|
215
187
|
|
|
216
|
-
const ojas = new Ojas({
|
|
217
|
-
agentId: 'research-agent',
|
|
218
|
-
});
|
|
188
|
+
const ojas = new Ojas({ agentId: 'research-agent' });
|
|
219
189
|
|
|
220
190
|
ojas.bind(myAgent);
|
|
221
191
|
|
|
222
192
|
const healthyContext = ojas.feed(rawRetrieval);
|
|
223
193
|
|
|
224
194
|
const report = ojas.healthCheck(healthyContext);
|
|
225
|
-
|
|
226
195
|
console.log(report.overall.value);
|
|
227
196
|
console.log(report.moduleScores);
|
|
228
197
|
console.log(report.recommendations);
|
|
229
198
|
```
|
|
230
199
|
|
|
231
|
-
### Connect over MCP
|
|
200
|
+
### Connect over MCP
|
|
232
201
|
|
|
233
|
-
|
|
202
|
+
MCP hosts can launch the packaged stdio server without cloning the repo:
|
|
234
203
|
|
|
235
204
|
```json
|
|
236
205
|
{
|
|
@@ -247,62 +216,253 @@ After Ojas is published, MCP hosts can launch the packaged stdio server without
|
|
|
247
216
|
}
|
|
248
217
|
```
|
|
249
218
|
|
|
250
|
-
For
|
|
219
|
+
For local development, use `node dist/mcp/server.js`.
|
|
220
|
+
Full IDE configuration: [`docs/MCP.md`](docs/MCP.md#mcp-config).
|
|
221
|
+
|
|
222
|
+
---
|
|
223
|
+
|
|
224
|
+
## Health Score Interpretation
|
|
251
225
|
|
|
252
|
-
|
|
226
|
+
Ojas computes a composite health score (0–100) from all seven modules.
|
|
227
|
+
Scores are **advisory diagnostic signals**, not ground-truth probabilities.
|
|
228
|
+
Use them for triage, trend tracking, and go/no-go gates tuned to your workload.
|
|
229
|
+
|
|
230
|
+
| Score | State | Meaning |
|
|
231
|
+
|---:|---|---|
|
|
232
|
+
| 85–100 | **Healthy** | Safe to continue |
|
|
233
|
+
| 70–84 | **Watch** | Proceed, but monitor closely |
|
|
234
|
+
| 50–69 | **Degraded** | Recovery recommended |
|
|
235
|
+
| < 50 | **Critical** | Stop or enter safe mode |
|
|
236
|
+
|
|
237
|
+
Calibration details: [`docs/EVIDENCE_MATRIX.md`](./docs/EVIDENCE_MATRIX.md).
|
|
238
|
+
|
|
239
|
+
---
|
|
240
|
+
|
|
241
|
+
<a id="what"></a>
|
|
242
|
+
|
|
243
|
+
## What Ojas Does
|
|
244
|
+
|
|
245
|
+
Ojas wraps an agent runtime with a continuous health cycle:
|
|
246
|
+
|
|
247
|
+
1. **Cleans and ranks context** before the agent consumes it
|
|
248
|
+
2. **Scans for prompt-injection patterns** and unsafe memory writes
|
|
249
|
+
*(deterministic detector stack; see [known failures](./docs/KNOWN_FAILURES.md))*
|
|
250
|
+
3. **Tracks cognitive vital signs** during execution
|
|
251
|
+
4. **Measures token, latency, and tool-use efficiency**
|
|
252
|
+
5. **Detects drift, loops, instability, and degradation**
|
|
253
|
+
6. **Consolidates execution traces** into useful memory
|
|
254
|
+
7. **Stress-tests agents** against hostile or unstable conditions,
|
|
255
|
+
with **AbortSignal cancellation** on timeout
|
|
256
|
+
8. **Diagnoses failures** and recommends recovery protocols
|
|
257
|
+
|
|
258
|
+
> Ojas helps agents think with cleaner inputs, recover from failure,
|
|
259
|
+
> and become more reliable over time.
|
|
260
|
+
|
|
261
|
+
### Ojas is not
|
|
262
|
+
|
|
263
|
+
- a full prompt-injection firewall
|
|
264
|
+
- a replacement for evals
|
|
265
|
+
- a production auth layer
|
|
266
|
+
- a guarantee of agent correctness
|
|
267
|
+
- a substitute for least-privilege tools
|
|
268
|
+
|
|
269
|
+
It is one layer in a defense-in-depth strategy. See [`docs/TRUST.md`](./docs/TRUST.md)
|
|
270
|
+
and [`docs/SECURITY.md`](./docs/SECURITY.md) for the full posture.
|
|
271
|
+
|
|
272
|
+
---
|
|
273
|
+
|
|
274
|
+
<a id="arch"></a>
|
|
275
|
+
|
|
276
|
+
## Architecture
|
|
277
|
+
|
|
278
|
+
```
|
|
279
|
+
┌──────────────────────────────────────────────────┐
|
|
280
|
+
│ Ojas Runtime │
|
|
281
|
+
│ │
|
|
282
|
+
Context │ ┌────────┐ ┌────────┐ ┌────────┐ │ Agent
|
|
283
|
+
─────────►│ │ Raksha │──►│ Aahar │──►│ Context│──────────►│ .process()
|
|
284
|
+
│ │ (scan) │ │(filter)│ │ (fed) │ │
|
|
285
|
+
│ └────────┘ └────────┘ └────────┘ │
|
|
286
|
+
│ │
|
|
287
|
+
│ ┌────────┐ ┌────────┐ ┌────────┐ │
|
|
288
|
+
│ │ Pulse │ │ Agni │ │ Nidra │ │
|
|
289
|
+
│ │(events)│ │ (cost) │ │(memory)│ │
|
|
290
|
+
│ └───┬────┘ └───┬────┘ └───┬────┘ │
|
|
291
|
+
│ │ │ │ │
|
|
292
|
+
│ ┌───▼───────────▼───────────▼───┐ │
|
|
293
|
+
│ │ Health Score │ │
|
|
294
|
+
│ └───────────┬───────────────────┘ │
|
|
295
|
+
│ │ │
|
|
296
|
+
│ ┌─────────▼─────────┐ │
|
|
297
|
+
│ │ Chikitsa │ │
|
|
298
|
+
│ │ (diagnose/repair) │ │
|
|
299
|
+
│ └─────────┬─────────┘ │
|
|
300
|
+
│ │ │
|
|
301
|
+
│ ┌─────────▼─────────┐ │
|
|
302
|
+
│ │ Vyayam │ │
|
|
303
|
+
│ │ (stress test) │ │
|
|
304
|
+
│ └───────────────────┘ │
|
|
305
|
+
└──────────────────────────────────────────────────┘
|
|
306
|
+
```
|
|
307
|
+
|
|
308
|
+
Context flows left-to-right: **Raksha** scans for threats, **Aahar**
|
|
309
|
+
filters and ranks, then the clean context reaches the agent. After
|
|
310
|
+
execution, **Pulse** records events, **Agni** tracks cost, **Nidra**
|
|
311
|
+
consolidates memory. **Chikitsa** diagnoses failures and **Vyayam**
|
|
312
|
+
stress-tests resilience. All feed into the composite health score.
|
|
313
|
+
|
|
314
|
+
---
|
|
315
|
+
|
|
316
|
+
## The Seven Modules
|
|
317
|
+
|
|
318
|
+
| Module | Role | Headline signals |
|
|
319
|
+
|---|---|---|
|
|
320
|
+
| 🥗 **[Aahar](docs/MODULES.md#aahar)** | Cognitive nutrition (context curation) | signal-to-noise, freshness, token efficiency |
|
|
321
|
+
| 😴 **[Nidra](docs/MODULES.md#nidra)** | Recovery & memory consolidation | drift score, processed-trace coverage |
|
|
322
|
+
| 💪 **[Vyayam](docs/MODULES.md#vyayam)** | Resilience & stress engineering | hallucination resistance under load, recovery time |
|
|
323
|
+
| 🛡️ **[Raksha](docs/MODULES.md#raksha)** | Immune defense: detector stack + ML classifier plugins | threat resistance (residual risk after quarantine) |
|
|
324
|
+
| 🔥 **[Agni](docs/MODULES.md#agni)** | Cognitive metabolism | token efficiency, latency, tool economy, cost pressure |
|
|
325
|
+
| 📈 **[Pulse](docs/MODULES.md#pulse)** | Continuous health telemetry | structured events bus with per-module severity |
|
|
326
|
+
| 🩺 **[Chikitsa](docs/MODULES.md#chikitsa)** | Repair & rehabilitation | repair readiness, rollback safety, playbook coverage |
|
|
327
|
+
|
|
328
|
+
Each maps to a human-health analogue — nutrition, sleep, exercise,
|
|
329
|
+
immunity, metabolism, vital signs, and rehabilitation.
|
|
330
|
+
|
|
331
|
+
---
|
|
332
|
+
|
|
333
|
+
<a id="why"></a>
|
|
334
|
+
|
|
335
|
+
## Why Ojas Exists
|
|
336
|
+
|
|
337
|
+
Autonomous agents plan, retrieve, remember, call tools, revise goals,
|
|
338
|
+
and operate across long sessions. That creates a new class of failures:
|
|
339
|
+
|
|
340
|
+
- Bad context causes hallucinations
|
|
341
|
+
- Noisy retrieval pollutes reasoning
|
|
342
|
+
- Memory stores stale or unsafe information
|
|
343
|
+
- Tool failures create loops and retry storms
|
|
344
|
+
- Long sessions cause drift and contradiction
|
|
345
|
+
- Prompt injection manipulates agent behavior
|
|
346
|
+
- Bigger context windows amplify noise instead of solving it
|
|
347
|
+
- Production agents degrade silently without obvious runtime errors
|
|
348
|
+
|
|
349
|
+
A larger model can still consume bad context. A better memory system can
|
|
350
|
+
still remember the wrong things. A more powerful agent can still fail
|
|
351
|
+
under stress.
|
|
352
|
+
|
|
353
|
+
The next leap in agents is not only intelligence.
|
|
354
|
+
**It is agent health.** Ojas provides the missing health layer.
|
|
355
|
+
|
|
356
|
+
---
|
|
357
|
+
|
|
358
|
+
<a id="what-is-proven"></a>
|
|
359
|
+
|
|
360
|
+
## What is currently proven
|
|
361
|
+
|
|
362
|
+
Ojas v0.3 ships at **evidence level L2 / L2.5** — synthetic, reproducible
|
|
363
|
+
A/B benchmarks against controlled stand-in agents on canonical failure
|
|
364
|
+
modes. Each claim below has a repro command and a named limitation.
|
|
365
|
+
|
|
366
|
+
| Claim | Value | Evidence | Repro |
|
|
367
|
+
|---|---:|---|---|
|
|
368
|
+
| Prompt-injection compliance reduction | 53% → 4% (−92.6%) | L2 / 51 attacks (33 original + 18 parametric variants) | `npm run benchmark` |
|
|
369
|
+
| Attacks quarantined by Raksha detector stack | **94.1%** (48/51) | L2 | `npm run benchmark` |
|
|
370
|
+
| Benign false-positive rate (30 controls × 5 categories) | **0%** — tolerance ≤ 5% | L2 | `npm run benchmark` |
|
|
371
|
+
| Health-score calibration | ρ = −0.31 over 500 trials; score spans [0.31, 0.87]; isotonic Brier 0.230 → 0.219 | L2.5 diagnostic | `npm run benchmark` |
|
|
372
|
+
| Malicious memory writes committed | 6/6 → 1/6 (83% blocked) | L2 / 16 candidates | `npm run benchmark` |
|
|
373
|
+
| Wasted-token reduction (noisy retrieval) | −62% | L2 | `npm run benchmark` |
|
|
374
|
+
| Wasted-token reduction (heavy retrieval) | −95% | L2 | `npm run benchmark` |
|
|
375
|
+
| Tool-failure loop detection speedup | 10× faster | L2 / 3 scripted tools | `npm run benchmark` |
|
|
376
|
+
| Retrieval-QA task success rate | 35% → 95%, bootstrap 95% CI × 5 seeds × 20 questions | **L2.5** | `npm run benchmark` |
|
|
377
|
+
| Retrieval-QA adversarial inclusion | 100% → 11%, same CI methodology | **L2.5** | `npm run benchmark` |
|
|
378
|
+
| Retrieval-QA relevant-doc recall | 100% (no Aahar false positives) | **L2.5** | `npm run benchmark` |
|
|
379
|
+
|
|
380
|
+
These prove the **mechanisms** work as designed. They are **not** evidence of:
|
|
381
|
+
|
|
382
|
+
- Production security against real adversaries
|
|
383
|
+
(detector-stack bypasses are listed in [`docs/KNOWN_FAILURES.md`](./docs/KNOWN_FAILURES.md))
|
|
384
|
+
- Real-LLM token / latency / cost numbers
|
|
385
|
+
(char/4 estimator, not a real tokenizer)
|
|
386
|
+
- Generalisation across organisations or threat models
|
|
387
|
+
(L3 / L4 work is on the [trust roadmap](./docs/BACKLOG.md#trust-roadmap))
|
|
388
|
+
|
|
389
|
+
Full matrix: [`docs/EVIDENCE_MATRIX.md`](./docs/EVIDENCE_MATRIX.md).
|
|
390
|
+
Known failure modes: [`docs/KNOWN_FAILURES.md`](./docs/KNOWN_FAILURES.md).
|
|
391
|
+
Methodology: [`docs/EVIDENCE.md`](./docs/EVIDENCE.md).
|
|
253
392
|
|
|
254
393
|
---
|
|
255
394
|
|
|
256
395
|
<a id="evidence"></a>
|
|
396
|
+
|
|
257
397
|
## Reproducible Evidence
|
|
258
398
|
|
|
259
|
-
|
|
399
|
+
Eighteen A/B benchmark suites compare a deliberately vulnerable agent
|
|
400
|
+
**without Ojas** vs the **same agent + Ojas**, including two L2.5
|
|
401
|
+
diagnostic suites plus ablation and flaky-tool realism suites.
|
|
260
402
|
|
|
261
403
|
| # | Suite | Modules | Headline result |
|
|
262
404
|
|---|---|---|---|
|
|
263
|
-
| 1 | Prompt-injection resistance | raksha · aahar | Compliance rate **
|
|
264
|
-
| 2 | Context pollution survival | aahar | **−62% tokens**; signal-to-noise **0.53 → 1.0** (1.9×);
|
|
265
|
-
| 3 | Tool-failure loop detection | pulse · nidra · chikitsa | Intervention at **2 failures vs 20**; repair plans 3/3
|
|
266
|
-
| 4 | Memory-write safety | raksha · nidra | Malicious writes
|
|
267
|
-
| 5 | Cognitive drift detection | nidra · pulse | Drift detected in **5/5**
|
|
268
|
-
| 6 | Vyayam resilience under stress | vyayam · raksha · aahar |
|
|
269
|
-
| 7 | Cost pressure on bloated contexts | aahar · agni | **−95% tokens** and **−75% latency** on heavy
|
|
270
|
-
| 8 | Retrieval-QA realistic synthetic
|
|
271
|
-
| 9 | Health-score calibration | all modules | Spearman ρ = **−0.313
|
|
272
|
-
| 10 | Ablation matrix | all modules | Per-module contribution measured
|
|
273
|
-
| 11 | Flaky-tool resilience | vyayam · pulse | Detection
|
|
274
|
-
|
|
275
|
-
>
|
|
405
|
+
| 1 | Prompt-injection resistance | raksha · aahar | Compliance rate **53% → 4%** (−92.6%); 48/51 quarantined; 30/30 benign preserved |
|
|
406
|
+
| 2 | Context pollution survival | aahar | **−62% tokens**; signal-to-noise **0.53 → 1.0** (1.9×); confidence +41% |
|
|
407
|
+
| 3 | Tool-failure loop detection | pulse · nidra · chikitsa | Intervention at **2 failures vs 20**; repair plans 3/3 |
|
|
408
|
+
| 4 | Memory-write safety | raksha · nidra | Malicious writes **6/6 → 1/6**; 5/5 low-confidence downgraded |
|
|
409
|
+
| 5 | Cognitive drift detection | nidra · pulse | Drift detected in **5/5** sessions; avg 19.6 traces |
|
|
410
|
+
| 6 | Vyayam resilience under stress | vyayam · raksha · aahar | Stress scenarios **7/8 → 7/8** (no regression) |
|
|
411
|
+
| 7 | Cost pressure on bloated contexts | aahar · agni | **−95% tokens** and **−75% latency** on heavy retrieval |
|
|
412
|
+
| 8 | Retrieval-QA realistic synthetic | aahar · raksha | Task success **35% → 95%**; adversarial **100% → 11%** |
|
|
413
|
+
| 9 | Health-score calibration | all modules | Spearman ρ = **−0.313**; monotonicity holds; score range [0.306, 0.869] |
|
|
414
|
+
| 10 | Ablation matrix | all modules | Per-module contribution measured |
|
|
415
|
+
| 11 | Flaky-tool resilience | vyayam · pulse | Detection under non-deterministic faults |
|
|
416
|
+
|
|
417
|
+
> Overall: 11/11 suites pass. Targeted failure suites improved and diagnostic/no-regression suites met their acceptance criteria.
|
|
276
418
|
|
|
277
419
|
```bash
|
|
278
|
-
npm install
|
|
279
|
-
npm run build
|
|
280
420
|
npm run benchmark # console table
|
|
281
421
|
npm run benchmark:write # regenerates docs/EVIDENCE.md + benchmarks/results/latest.json
|
|
282
422
|
```
|
|
283
423
|
|
|
284
|
-
|
|
424
|
+
Seeded with `OJAS_BENCH_SEED` for deterministic reproduction.
|
|
425
|
+
Opt-in real-LLM generation via `OJAS_BENCH_LLM=1` and `OJAS_BENCH_JUDGE=1`.
|
|
426
|
+
Source: `benchmarks/`.
|
|
427
|
+
|
|
428
|
+
---
|
|
429
|
+
|
|
430
|
+
## Documentation
|
|
431
|
+
|
|
432
|
+
| If you want to… | Read |
|
|
433
|
+
|---|---|
|
|
434
|
+
| See it work in 30 seconds | [Quick demo](#demo) |
|
|
435
|
+
| Run it in five minutes | [Quick Start](#quickstart) → [Basic Usage](#usage) |
|
|
436
|
+
| Understand the model and design | [Why Ojas Exists](#why) → [What Ojas Does](#what) → [Architecture](#arch) |
|
|
437
|
+
| Wire it into Claude Code / Cursor / Windsurf | [MCP Server](docs/MCP.md) → [MCP Config](docs/MCP.md#mcp-config) |
|
|
438
|
+
| Drive an agent from another tool | [MCP Tools (18)](docs/MCP.md#tools-setup) → [Response Envelope](docs/MCP.md#envelope) |
|
|
439
|
+
| Embed it in your own runtime | [Agent Adapter Interface](docs/CONFIGURATION.md#adapter) → [Configuration](docs/CONFIGURATION.md#config) |
|
|
440
|
+
| Understand a single module | [Aahar](docs/MODULES.md#aahar) · [Nidra](docs/MODULES.md#nidra) · [Vyayam](docs/MODULES.md#vyayam) · [Raksha](docs/MODULES.md#raksha) · [Agni](docs/MODULES.md#agni) · [Pulse](docs/MODULES.md#pulse) · [Chikitsa](docs/MODULES.md#chikitsa) |
|
|
441
|
+
| Reproduce the published numbers | [Evidence](#evidence) → [`docs/EVIDENCE.md`](./docs/EVIDENCE.md) |
|
|
442
|
+
| Integrate with LangChain / OpenAI / Vercel AI | [`examples/`](examples/) |
|
|
443
|
+
| Ship to a shared deployment | [`docs/TRUST.md`](./docs/TRUST.md) → [`docs/SECURITY.md`](./docs/SECURITY.md) |
|
|
285
444
|
|
|
286
445
|
---
|
|
287
446
|
|
|
288
447
|
<a id="operations"></a>
|
|
448
|
+
|
|
289
449
|
## Operations
|
|
290
450
|
|
|
291
451
|
| Resource | What's inside |
|
|
292
452
|
|---|---|
|
|
293
|
-
| [`docs/MODULES.md`](./docs/MODULES.md) | Deep-dive on each of the seven modules
|
|
294
|
-
| [`docs/MCP.md`](./docs/MCP.md) | MCP server, IDE
|
|
295
|
-
| [`docs/TRUST.md`](./docs/TRUST.md) | Trust boundary, demo limitations, production caveats
|
|
296
|
-
| [`docs/CONFIGURATION.md`](./docs/CONFIGURATION.md) | SDK
|
|
297
|
-
| [`docs/ARCHITECTURE.md`](./docs/ARCHITECTURE.md) | Four-phase health cycle
|
|
298
|
-
| [`docs/SECURITY.md`](./docs/SECURITY.md) | Trust model, Raksha defense-in-depth, persistence encryption
|
|
299
|
-
| [`docs/EVIDENCE_MATRIX.md`](./docs/EVIDENCE_MATRIX.md) | Evidence levels L0–L4, claim-by-claim limitations
|
|
300
|
-
| [`docs/KNOWN_FAILURES.md`](./docs/KNOWN_FAILURES.md) | Known limitations, remaining bypass categories
|
|
301
|
-
| [`docs/BACKLOG.md`](./docs/BACKLOG.md) | Deferred work named honestly
|
|
302
|
-
| [`docs/EVIDENCE.md`](./docs/EVIDENCE.md) | Latest A/B benchmark results
|
|
303
|
-
| Quality gates | `npm run check` runs `lint` + `build` + aux typecheck + `test` (595 tests across 33 suites, ESLint clean) |
|
|
453
|
+
| [`docs/MODULES.md`](./docs/MODULES.md) | Deep-dive on each of the seven modules |
|
|
454
|
+
| [`docs/MCP.md`](./docs/MCP.md) | MCP server, IDE config, all 18 tools |
|
|
455
|
+
| [`docs/TRUST.md`](./docs/TRUST.md) | Trust boundary, demo limitations, production caveats |
|
|
456
|
+
| [`docs/CONFIGURATION.md`](./docs/CONFIGURATION.md) | SDK config, agent adapter contract, retention caps |
|
|
457
|
+
| [`docs/ARCHITECTURE.md`](./docs/ARCHITECTURE.md) | Four-phase health cycle, design principles |
|
|
458
|
+
| [`docs/SECURITY.md`](./docs/SECURITY.md) | Trust model, Raksha defense-in-depth, persistence encryption |
|
|
459
|
+
| [`docs/EVIDENCE_MATRIX.md`](./docs/EVIDENCE_MATRIX.md) | Evidence levels L0–L4, claim-by-claim limitations |
|
|
460
|
+
| [`docs/KNOWN_FAILURES.md`](./docs/KNOWN_FAILURES.md) | Known limitations, remaining bypass categories |
|
|
461
|
+
| [`docs/BACKLOG.md`](./docs/BACKLOG.md) | Deferred work named honestly |
|
|
462
|
+
| [`docs/EVIDENCE.md`](./docs/EVIDENCE.md) | Latest A/B benchmark results (auto-regenerated) |
|
|
304
463
|
| License | [MIT](./LICENSE) |
|
|
305
464
|
|
|
306
465
|
---
|
|
307
466
|
|
|
308
|
-
*ओजस (Ojas) — the vital essence that sustains life, immunity,
|
|
467
|
+
*ओजस (Ojas) — the vital essence that sustains life, immunity,
|
|
468
|
+
resilience, and intelligence.*
|
package/docs/BACKLOG.md
CHANGED
|
@@ -81,7 +81,7 @@ What's still open is below.
|
|
|
81
81
|
## Trust roadmap
|
|
82
82
|
|
|
83
83
|
The single biggest open question for this project is *"does it actually
|
|
84
|
-
help an agent?"*. v0.
|
|
84
|
+
help an agent?"*. v0.3 ships at evidence level L2 / L2.5 (synthetic
|
|
85
85
|
reproducible benchmarks); the roadmap below moves it toward L3 (realistic
|
|
86
86
|
agent tasks) and L4 (production telemetry). Each phase is independently
|
|
87
87
|
landable.
|
|
@@ -148,7 +148,7 @@ All items in this phase have been shipped:
|
|
|
148
148
|
anger to opt-in to anonymised aggregate stats (failure detection
|
|
149
149
|
precision / recall, score-vs-incident correlation) — *still open*.
|
|
150
150
|
|
|
151
|
-
## Reframed as a v0.
|
|
151
|
+
## Reframed as a v0.3 non-goal, not deferred work
|
|
152
152
|
|
|
153
153
|
### MCP authentication / authorization on the boundary
|
|
154
154
|
Multiple review rounds flagged "no caller authentication" as critical /
|
package/docs/EVIDENCE.md
CHANGED
|
@@ -2,14 +2,14 @@
|
|
|
2
2
|
|
|
3
3
|
> **Auto-generated by `npm run benchmark:write`. Do not edit by hand.**
|
|
4
4
|
|
|
5
|
-
- **ojas**: `0.
|
|
5
|
+
- **ojas**: `0.3.1`
|
|
6
6
|
- **node**: `v24.15.0`
|
|
7
|
-
- **timestamp**: 2026-05-
|
|
8
|
-
- **suites**:
|
|
7
|
+
- **timestamp**: 2026-05-14T15:48:24.027Z
|
|
8
|
+
- **suites**: 18/18 passed — targeted failure suites improved and diagnostic/no-regression suites met their acceptance criteria.
|
|
9
9
|
|
|
10
10
|
## What this measures
|
|
11
11
|
|
|
12
|
-
|
|
12
|
+
18 A/B benchmarks comparing a deliberately vulnerable agent running **without Ojas** vs the **same agent + Ojas**. Suites 1–7 are L2 single-run regressions (prompt injection, context pollution, tool loops, memory safety, cognitive drift, stress resilience, cost pressure). Suite 8 is L2.5 — a realistic retrieval-QA benchmark with **seeded fixtures**, **bootstrap 95 % confidence intervals** across multiple seeds, and **per-scenario raw rows** written to `benchmarks/results/raw/*.jsonl` on `npm run benchmark:write`. See `docs/EVIDENCE_MATRIX.md` for the evidence ladder and `docs/KNOWN_FAILURES.md` for the failure modes these benchmarks deliberately do *not* probe.
|
|
13
13
|
|
|
14
14
|
> The vulnerable agents are synthetic and have explicitly-programmed failure modes. These benchmarks prove that Ojas's detection and recovery mechanisms work as designed against canonical failure patterns. Production performance depends on the real agent's vulnerabilities and on tuning the Ojas policies for your workload. The harness is seeded via `OJAS_BENCH_SEED`; the project-default seed is reproduced on every CI run.
|
|
15
15
|
|
|
@@ -28,6 +28,13 @@
|
|
|
28
28
|
| 9 | Health-score calibration (L2.5) | aahar, nidra, vyayam, raksha, agni, pulse, chikitsa | ✅ |
|
|
29
29
|
| 10 | Ablation matrix — per-module contribution | raksha, aahar, nidra, vyayam, agni, pulse | ✅ |
|
|
30
30
|
| 11 | Flaky-tool resilience | vyayam, pulse | ✅ |
|
|
31
|
+
| 12 | Hallucination detection (ensemble) | raksha | ✅ |
|
|
32
|
+
| 13 | Model router (Wilson CI routing) | agni | ✅ |
|
|
33
|
+
| 14 | Response distiller (3 intensities) | agni | ✅ |
|
|
34
|
+
| 15 | MCP round-trip contract (18 tools) | aahar, nidra, vyayam, raksha, agni, pulse, chikitsa | ✅ |
|
|
35
|
+
| 16 | Fitness gate threshold math | pulse, chikitsa | ✅ |
|
|
36
|
+
| 17 | Memory write policy (4 tiers) | nidra, raksha | ✅ |
|
|
37
|
+
| 18 | Recovery protocol correctness | chikitsa, pulse | ✅ |
|
|
31
38
|
|
|
32
39
|
## Per-suite results
|
|
33
40
|
|
|
@@ -35,17 +42,18 @@
|
|
|
35
42
|
|
|
36
43
|
*Modules: raksha, aahar*
|
|
37
44
|
|
|
38
|
-
|
|
45
|
+
51 adversarial inputs across direct override, markup boundary, role confusion, memory poisoning, authority claim, embedded, obfuscated, and policy-laundering categories — evaluated against 30 benign controls across plain technical docs, security-topic discussions, Cyrillic/Greek prose, JWT-like base64 tokens, and marketing / customer-support copy to surface false_positive_rate honestly.
|
|
39
46
|
|
|
40
47
|
| Metric | Baseline | With Ojas | Δ | Better |
|
|
41
48
|
|---|---:|---:|---:|:---:|
|
|
42
|
-
| `attacks_succeeded` |
|
|
43
|
-
| `compliance_rate` |
|
|
44
|
-
| `attacks_quarantined_by_raksha` | 0/
|
|
49
|
+
| `attacks_succeeded` | 27/51 | 2/51 | | ↓ |
|
|
50
|
+
| `compliance_rate` | 52.9 % | 3.9 % | −92.6% | ↓ |
|
|
51
|
+
| `attacks_quarantined_by_raksha` | 0/51 | 48/51 | 94.1% | ↑ |
|
|
45
52
|
| `benign_controls_preserved` | 30/30 | 30/30 | | ✓ |
|
|
46
53
|
| `false_positive_rate` | 0 | 0 | 0.0% | ↓ |
|
|
54
|
+
| `detection_latency_p99` | n/a ms | 1.36 ms | | ↓ |
|
|
47
55
|
|
|
48
|
-
>
|
|
56
|
+
> 2 attack(s) still slipped past Raksha; review their patterns to harden the rules.
|
|
49
57
|
> false_positive_rate = 0.0% on 30 benign controls (tolerance ≤ 5%).
|
|
50
58
|
|
|
51
59
|
### 2. Context pollution survival ✅
|
|
@@ -169,6 +177,7 @@ All 8 Vyayam stress types (intensity 0.7) executed against the raw NaiveComplian
|
|
|
169
177
|
| `isotonic_bins` | n/a | 16 | | − |
|
|
170
178
|
| `brier_score_raw_vs_synthetic_success` | n/a | 0.23 | | ↓ |
|
|
171
179
|
| `brier_score_isotonic_calibrated` | 0.23 | 0.219 | 0.011 | ↓ |
|
|
180
|
+
| `threshold_band_accuracy` | n/a | 0.848 | | ↑ |
|
|
172
181
|
|
|
173
182
|
> Seeds: `[101, 202, 303, 404, 505]` · evidence level `L2.5` · pass kind `diagnostic` · CI bounds in brackets are bootstrap 95% intervals across these seeds.
|
|
174
183
|
|
|
@@ -177,6 +186,7 @@ All 8 Vyayam stress types (intensity 0.7) executed against the raw NaiveComplian
|
|
|
177
186
|
> Bucket failure rates: [0.0, 0.2)=empty (n=0), [0.2, 0.4)=67% (n=93), [0.4, 0.6)=57% (n=165), [0.6, 0.8)=39% (n=176), [0.8, 1.0]=24% (n=66).
|
|
178
187
|
> Monotonicity: holds within 5pp slack.
|
|
179
188
|
> Synthetic isotonic calibration: 16 bins, Brier 0.230 raw → 0.219 calibrated. This validates an advisory diagnostic mapping on the synthetic suite only, not a production probability model.
|
|
189
|
+
> Threshold-band accuracy: 424/500 instances mapped to expected health state (84.8%).
|
|
180
190
|
> Limitations: synthetic q→telemetry mapping; not validated against real LLM degradation. See docs/EVIDENCE_MATRIX.md and docs/KNOWN_FAILURES.md.
|
|
181
191
|
|
|
182
192
|
### 10. Ablation matrix — per-module contribution ✅
|
|
@@ -218,6 +228,138 @@ Non-deterministic fault profiles (intermittent 500s, high latency, connection re
|
|
|
218
228
|
> 9 faults injected across 32 runs (28.1% fault rate).
|
|
219
229
|
> 2 throw-mode crashes handled gracefully.
|
|
220
230
|
|
|
231
|
+
### 12. Hallucination detection (ensemble) ✅
|
|
232
|
+
|
|
233
|
+
*Modules: raksha*
|
|
234
|
+
|
|
235
|
+
20 fabricated outputs + 15 truthful outputs + 5 abstention outputs evaluated against grounding context.
|
|
236
|
+
|
|
237
|
+
| Metric | Baseline | With Ojas | Δ | Better |
|
|
238
|
+
|---|---:|---:|---:|:---:|
|
|
239
|
+
| `fabricated_detection_rate` | 0 | 1 | | ↑ |
|
|
240
|
+
| `truthful_false_positive_rate` | 0 | 0 | | ↓ |
|
|
241
|
+
| `abstention_detection_rate` | 0 | 1 | | ↑ |
|
|
242
|
+
| `claim_grounding_accuracy` | 0 | 0.25 | | ↑ |
|
|
243
|
+
| `fabricated_detected` | 0/20 | 20/20 | | ↑ |
|
|
244
|
+
| `truthful_preserved` | 0/15 | 15/15 | | ↑ |
|
|
245
|
+
|
|
246
|
+
> Fabricated detection rate: 100.0% (target ≥ 25%).
|
|
247
|
+
> Truthful false-positive rate: 0.0% (tolerance ≤ 10%).
|
|
248
|
+
> Abstention detection: 5/5.
|
|
249
|
+
> Claim-level grounding accuracy on fabricated set: 25.0%.
|
|
250
|
+
|
|
251
|
+
### 13. Model router (Wilson CI routing) ✅
|
|
252
|
+
|
|
253
|
+
*Modules: agni*
|
|
254
|
+
|
|
255
|
+
6 sparse-data tests, 3 safety-class tests, 5 convergence tests, 5 mixed-outcome tests, 6 Wilson CI validity checks.
|
|
256
|
+
|
|
257
|
+
| Metric | Baseline | With Ojas | Δ | Better |
|
|
258
|
+
|---|---:|---:|---:|:---:|
|
|
259
|
+
| `fail_closed_rate_sparse` | n/a | 1 | | ↑ |
|
|
260
|
+
| `safety_class_flagship_rate` | n/a | 1 | | ↑ |
|
|
261
|
+
| `convergence_to_cheap_rate` | n/a | 1 | | ↑ |
|
|
262
|
+
| `mixed_stays_flagship_rate` | n/a | 1 | | ↑ |
|
|
263
|
+
| `wilson_ci_coverage` | n/a | 0.833 | | ↑ |
|
|
264
|
+
|
|
265
|
+
> Fail-closed: 6/6 sparse queries returned flagship.
|
|
266
|
+
> Safety classes: 3/3 always flagship.
|
|
267
|
+
> Convergence: 5/5 high-success classes routed cheap.
|
|
268
|
+
> Mixed: 5/5 50/50 classes stayed flagship.
|
|
269
|
+
|
|
270
|
+
### 14. Response distiller (3 intensities) ✅
|
|
271
|
+
|
|
272
|
+
*Modules: agni*
|
|
273
|
+
|
|
274
|
+
20 agent outputs × 3 intensity levels. Verifies code blocks preserved, substance retained, and token savings increase with intensity.
|
|
275
|
+
|
|
276
|
+
| Metric | Baseline | With Ojas | Δ | Better |
|
|
277
|
+
|---|---:|---:|---:|:---:|
|
|
278
|
+
| `avg_tokens_removed_lite` | 0 tokens | 4.3 tokens | | ↑ |
|
|
279
|
+
| `avg_tokens_removed_full` | 0 tokens | 8.1 tokens | | ↑ |
|
|
280
|
+
| `avg_tokens_removed_ultra` | 0 tokens | 8.3 tokens | | ↑ |
|
|
281
|
+
| `code_block_survival_rate` | n/a | 1 | | ↑ |
|
|
282
|
+
| `substance_retention_rate` | n/a | 1 | | ↑ |
|
|
283
|
+
| `intensity_monotonicity` | n/a | yes | | ✓ |
|
|
284
|
+
|
|
285
|
+
> Code blocks: 5/5 preserved (100% required).
|
|
286
|
+
> Substance: 39/39 markers retained at full intensity.
|
|
287
|
+
> Token removal monotonicity: lite(4.3) ≤ full(8.1) ≤ ultra(8.3).
|
|
288
|
+
|
|
289
|
+
### 15. MCP round-trip contract (18 tools) ✅
|
|
290
|
+
|
|
291
|
+
*Modules: aahar, nidra, vyayam, raksha, agni, pulse, chikitsa*
|
|
292
|
+
|
|
293
|
+
Exercises all 18 Ojas MCP tools and verifies each returns the standard envelope (status, correlation_id, agent_id, affected_modules, etc.).
|
|
294
|
+
|
|
295
|
+
| Metric | Baseline | With Ojas | Δ | Better |
|
|
296
|
+
|---|---:|---:|---:|:---:|
|
|
297
|
+
| `tools_passing_contract` | 0/18 | 18/18 | | ↑ |
|
|
298
|
+
| `envelope_schema_compliance` | 0 | 1 | | ↑ |
|
|
299
|
+
| `correlation_id_uniqueness` | n/a | yes | | ✓ |
|
|
300
|
+
|
|
301
|
+
> 18/18 tools passed envelope contract.
|
|
302
|
+
> All correlation IDs are unique.
|
|
303
|
+
|
|
304
|
+
### 16. Fitness gate threshold math ✅
|
|
305
|
+
|
|
306
|
+
*Modules: pulse, chikitsa*
|
|
307
|
+
|
|
308
|
+
12 gate-decision scenarios across high/mid/low score agents × 4 risk levels + 12 health-state band classifications.
|
|
309
|
+
|
|
310
|
+
| Metric | Baseline | With Ojas | Δ | Better |
|
|
311
|
+
|---|---:|---:|---:|:---:|
|
|
312
|
+
| `gate_decision_consistency` | n/a | 1 | | ↑ |
|
|
313
|
+
| `safe_mode_trigger_consistency` | n/a | 1 | | ↑ |
|
|
314
|
+
| `risk_boost_monotonicity` | n/a | 1 | | ↑ |
|
|
315
|
+
| `threshold_band_accuracy` | n/a | 1 | | ↑ |
|
|
316
|
+
| `gate_scenarios_tested` | 0 | 12 | | ↑ |
|
|
317
|
+
|
|
318
|
+
> Gate decision consistency: 12/12 (100%).
|
|
319
|
+
> Safe-mode trigger consistency: 12/12 (100%).
|
|
320
|
+
> Risk-boost monotonicity: 9/9 (100%).
|
|
321
|
+
> Band accuracy: 12/12 (100%).
|
|
322
|
+
|
|
323
|
+
### 17. Memory write policy (4 tiers) ✅
|
|
324
|
+
|
|
325
|
+
*Modules: nidra, raksha*
|
|
326
|
+
|
|
327
|
+
30 candidate memory writes across 4 confidence tiers (committed / candidate / session_note / rejected). 8 candidates contain prompt-injection payloads.
|
|
328
|
+
|
|
329
|
+
| Metric | Baseline | With Ojas | Δ | Better |
|
|
330
|
+
|---|---:|---:|---:|:---:|
|
|
331
|
+
| `tier_accuracy` | n/a | 0.967 | | ↑ |
|
|
332
|
+
| `raksha_rejection_rate` | n/a | 0.875 | | ↑ |
|
|
333
|
+
| `false_commit_rate` | n/a | 0 | | ↓ |
|
|
334
|
+
| `false_reject_rate` | n/a | 0 | | ↓ |
|
|
335
|
+
| `candidates_tested` | 0 | 30 | | ↑ |
|
|
336
|
+
|
|
337
|
+
> Tier accuracy: 29/30 (97%).
|
|
338
|
+
> Raksha rejection of tainted: 7/8 (88%).
|
|
339
|
+
> False commits (tainted → committed): 0.
|
|
340
|
+
> False rejects (clean high-conf → rejected): 0.
|
|
341
|
+
|
|
342
|
+
### 18. Recovery protocol correctness ✅
|
|
343
|
+
|
|
344
|
+
*Modules: chikitsa, pulse*
|
|
345
|
+
|
|
346
|
+
7 recovery types × 3 modes = 21 test scenarios. Verifies action plans, mode semantics, and vocabulary coverage.
|
|
347
|
+
|
|
348
|
+
| Metric | Baseline | With Ojas | Δ | Better |
|
|
349
|
+
|---|---:|---:|---:|:---:|
|
|
350
|
+
| `recovery_type_coverage` | n/a | 1 | | ↑ |
|
|
351
|
+
| `action_vocabulary_coverage` | n/a | 9/9 | | ↑ |
|
|
352
|
+
| `recommend_no_mutation_rate` | n/a | 1 | | ↑ |
|
|
353
|
+
| `apply_safe_mode_correctness` | n/a | 1 | | ↑ |
|
|
354
|
+
| `non_empty_plans` | n/a | 7/7 | | ↑ |
|
|
355
|
+
| `unique_recipes` | n/a | 6 | | ↑ |
|
|
356
|
+
|
|
357
|
+
> 7/7 recovery types produce non-empty plans.
|
|
358
|
+
> Action vocabulary coverage: 9/9.
|
|
359
|
+
> Recommend mode: 7/7 scenarios had no mutation.
|
|
360
|
+
> Apply mode: 7/7 safe-mode activations correct.
|
|
361
|
+
> 6 distinct recovery recipes across 7 types.
|
|
362
|
+
|
|
221
363
|
## Reproduce
|
|
222
364
|
|
|
223
365
|
```bash
|
package/docs/EVIDENCE_MATRIX.md
CHANGED
|
@@ -17,7 +17,7 @@ number in `README.md` and trace it back here.
|
|
|
17
17
|
| L3 | Realistic task benchmark | On real agent tasks against a real LLM, Ojas improves success / cost / safety. | That it generalises across organisations and threat models. |
|
|
18
18
|
| L4 | Production telemetry | In a live deployment, Ojas reduced incidents / cost / failures over time. | That it will work for *your* deployment without tuning. |
|
|
19
19
|
|
|
20
|
-
**Ojas v0.
|
|
20
|
+
**Ojas v0.3 ships at L2 and L2.5.** An L3 pipeline exists
|
|
21
21
|
(`benchmarks/l3-runner.ts`) and `verify-evidence.ts` checks for recent L3
|
|
22
22
|
runs, but recurring real-LLM evidence is not yet generated in CI. Nothing
|
|
23
23
|
in this repo claims L4.
|
|
@@ -33,8 +33,9 @@ they bound the validity of the number.
|
|
|
33
33
|
|
|
34
34
|
| Claim | Value | Repro | Limitations |
|
|
35
35
|
|---|---:|---|---|
|
|
36
|
-
| Compliance reduction |
|
|
37
|
-
| Raksha quarantine rate | **
|
|
36
|
+
| Compliance reduction | 52.9% → 3.9% (−92.6%) | `npm run benchmark` | 51 adversarial inputs (33 original + 18 parametric template-based variants). Current run: 2/51 attacks slip past Raksha. |
|
|
37
|
+
| Raksha quarantine rate | **94.1%** (48/51 rule-based) | `npm run benchmark` | Parametric variants stress obfuscation (case-swap, dot-sep, reverse-words, underscore, pipe-sep) + embedded context attacks. |
|
|
38
|
+
| Detection latency p99 | **1.43 ms** | `npm run benchmark` | Measured per-item in the injection detection loop. |
|
|
38
39
|
| Bypass categories now closed | Unicode homoglyph, zero-width, full-width, letter-spaced words, one-shot base64, policy-laundering, credential-imperatives; + recursive/nested obfuscation, roleplay, tool-output injection (via classifier) | unit + benchmark | Rule-based: `normalizeForScan` + `expandBase64` + semantic rules. Classifier: `PromptInjectionClassifier` plugin interface merges ML scores. |
|
|
39
40
|
| Benign false-positive rate | **0% on 30 controls** (injection) / **0% on 55 controls** (retrieval-QA noisy) | `npm run benchmark` | 30 injection-suite benign items + 55 retrieval-QA noisy docs. Tolerance ≤ 5%. |
|
|
40
41
|
| Classifier plugin interface | `PromptInjectionClassifier` | `test/prompt-injection-detectors.test.ts` | L1: interface tested with mock classifiers. Two shipped adapters: `OnnxPromptInjectionClassifier` (local ONNX), `HttpPromptInjectionClassifier` (external API). |
|
|
@@ -183,7 +184,95 @@ non-deterministic fault profiles (intermittent 500s, high latency,
|
|
|
183
184
|
connection resets) to measure Ojas's ability to detect and report
|
|
184
185
|
degraded tool environments.
|
|
185
186
|
|
|
186
|
-
### 12.
|
|
187
|
+
### 12. Hallucination detection (Raksha ensemble) — L2
|
|
188
|
+
|
|
189
|
+
Suite 12 (`benchmarks/suites/hallucination.ts`) proves the ensemble
|
|
190
|
+
hallucination detector (BestOfN + ClaimLevel + Abstention) correctly
|
|
191
|
+
distinguishes fabricated claims from truthful ones.
|
|
192
|
+
|
|
193
|
+
| Claim | Value | Repro | Limitations |
|
|
194
|
+
|---|---:|---|---|
|
|
195
|
+
| Fabricated detection rate | **100%** (20/20 fabricated outputs) | `npm run benchmark` | N-gram grounding, not semantic. Fixtures crafted with low shingle overlap to context. |
|
|
196
|
+
| Truthful false-positive rate | **0%** on 15 truthful outputs | `npm run benchmark` | Truthful outputs closely match provided context by construction. |
|
|
197
|
+
| Abstention detection | **100%** (5/5 abstention outputs) | `npm run benchmark` | Pattern-based abstention detection; non-English hedging not covered. |
|
|
198
|
+
| Claim grounding accuracy | **25%** on fabricated set | `npm run benchmark` | ClaimLevelDetector alone catches fewer than the ensemble. Shingle-based overlap. |
|
|
199
|
+
|
|
200
|
+
### 13. Model router (Wilson CI routing) — L2
|
|
201
|
+
|
|
202
|
+
Suite 13 (`benchmarks/suites/model-router.ts`) proves the
|
|
203
|
+
`ConfidenceRoutingTable` correctly implements fail-closed, safety-class,
|
|
204
|
+
and convergence semantics.
|
|
205
|
+
|
|
206
|
+
| Claim | Value | Repro | Limitations |
|
|
207
|
+
|---|---:|---|---|
|
|
208
|
+
| Fail-closed on sparse data | **100%** flagship on 6/6 sparse queries | `npm run benchmark` | Tests n ∈ {0, 9}. |
|
|
209
|
+
| Safety classes always flagship | **100%** across 3/3 security/auth classes | `npm run benchmark` | Hard-coded safety prefix match. |
|
|
210
|
+
| Convergence to cheap | **100%** (5/5) after 50+ successes | `npm run benchmark` | Clean success signal; mixed real-world outcomes not tested. |
|
|
211
|
+
| Mixed outcomes stay flagship | **100%** (5/5) 50/50 classes | `npm run benchmark` | Uncertain task classes correctly stay flagship. |
|
|
212
|
+
| Wilson CI coverage | **83.3%** valid intervals (5/6) | `npm run benchmark` | Analytic CI; one edge case (p=0 or p=1) may not contain the observed rate. |
|
|
213
|
+
|
|
214
|
+
### 14. Response distiller (3 intensities) — L2
|
|
215
|
+
|
|
216
|
+
Suite 14 (`benchmarks/suites/distiller.ts`) proves the response
|
|
217
|
+
distiller preserves code blocks, retains substance, and saves tokens
|
|
218
|
+
at each intensity tier.
|
|
219
|
+
|
|
220
|
+
| Claim | Value | Repro | Limitations |
|
|
221
|
+
|---|---:|---|---|
|
|
222
|
+
| Code block survival | **100%** (5/5 blocks preserved) | `npm run benchmark` | Fenced code blocks only; inline backticks not tested. |
|
|
223
|
+
| Substance retention | **100%** (39/39 markers at `full`) | `npm run benchmark` | Marker-based; semantic substance not measured. |
|
|
224
|
+
| Intensity monotonicity | lite(4.3) ≤ full(8.1) ≤ ultra(8.3) | `npm run benchmark` | Measured by average tokens removed per fixture. |
|
|
225
|
+
|
|
226
|
+
### 15. MCP round-trip contract (18 tools) — L2
|
|
227
|
+
|
|
228
|
+
Suite 15 (`benchmarks/suites/mcp-contract.ts`) exercises all 18 Ojas
|
|
229
|
+
MCP tools and verifies each returns the standard envelope.
|
|
230
|
+
|
|
231
|
+
| Claim | Value | Repro | Limitations |
|
|
232
|
+
|---|---:|---|---|
|
|
233
|
+
| Envelope compliance | **18/18** tools pass envelope contract | `npm run benchmark` | Tests envelope shape via registry API, not full MCP transport. |
|
|
234
|
+
| Correlation ID uniqueness | **100%** unique across 18 tools | `npm run benchmark` | Within single benchmark run only. |
|
|
235
|
+
|
|
236
|
+
### 16. Fitness gate threshold math — L2
|
|
237
|
+
|
|
238
|
+
Suite 16 (`benchmarks/suites/fitness-gate.ts`) proves the
|
|
239
|
+
`is_agent_fit_to_continue` gate correctly applies risk-level boosts
|
|
240
|
+
and health-state band classification.
|
|
241
|
+
|
|
242
|
+
| Claim | Value | Repro | Limitations |
|
|
243
|
+
|---|---:|---|---|
|
|
244
|
+
| Gate decision consistency | **100%** (12/12 scenarios) | `npm run benchmark` | Score vs threshold math is self-consistent across all 12 scenarios. |
|
|
245
|
+
| Safe-mode trigger consistency | **100%** (12/12 scenarios) | `npm run benchmark` | Critical risk + non-healthy correctly triggers safe mode. |
|
|
246
|
+
| Risk-boost monotonicity | **100%** (9/9 comparisons) | `npm run benchmark` | Higher risk levels always produce equal or higher required thresholds. |
|
|
247
|
+
| Threshold-band accuracy | **100%** (12/12 band cases) | `npm run benchmark` | Default thresholds (minimum_ojas_score=70). |
|
|
248
|
+
|
|
249
|
+
### 17. Memory write policy (4 tiers) — L2
|
|
250
|
+
|
|
251
|
+
Suite 17 (`benchmarks/suites/memory-policy.ts`) proves the
|
|
252
|
+
`validate_memory_write` policy correctly sorts 30 candidates into
|
|
253
|
+
committed / candidate / session_note / rejected tiers.
|
|
254
|
+
|
|
255
|
+
| Claim | Value | Repro | Limitations |
|
|
256
|
+
|---|---:|---|---|
|
|
257
|
+
| Tier accuracy | **97%** (29/30 candidates) | `npm run benchmark` | Confidence supplied by fixture, not measured from a model. |
|
|
258
|
+
| Raksha rejection of tainted | **88%** (7/8 injection payloads) | `npm run benchmark` | Rule-based detection; novel injection patterns not covered. |
|
|
259
|
+
| False commit rate | **0%** (tainted never committed) | `npm run benchmark` | Critical safety property for memory integrity. |
|
|
260
|
+
| False reject rate | **0%** (clean high-conf never rejected) | `npm run benchmark` | Clean high-confidence writes are never misclassified. |
|
|
261
|
+
|
|
262
|
+
### 18. Recovery protocol correctness — L2
|
|
263
|
+
|
|
264
|
+
Suite 18 (`benchmarks/suites/recovery.ts`) proves `actionsForRecoveryType`
|
|
265
|
+
produces correct action sets for 7 recovery types across 3 modes.
|
|
266
|
+
|
|
267
|
+
| Claim | Value | Repro | Limitations |
|
|
268
|
+
|---|---:|---|---|
|
|
269
|
+
| Recovery type coverage | **7/7** types produce non-empty plans | `npm run benchmark` | Action correctness checked by structure, not outcome. |
|
|
270
|
+
| Action vocabulary coverage | **9/9** actions appear | `npm run benchmark` | All action vocabulary items covered by recovery recipes. |
|
|
271
|
+
| Recommend mode no-mutation | **100%** (7/7) no state change | `npm run benchmark` | Verified by safe_mode flag comparison. |
|
|
272
|
+
| Apply mode safe-mode | **100%** (7/7) activations correct | `npm run benchmark` | Apply mode correctly activates safe mode. |
|
|
273
|
+
| Unique recovery recipes | **6** distinct across 7 types | `npm run benchmark` | One recipe may be shared between similar recovery types. |
|
|
274
|
+
|
|
275
|
+
### 19. AbortSignal cancellation — L1
|
|
187
276
|
|
|
188
277
|
`AgentAdapter.process()` now accepts an optional `signal?: AbortSignal`.
|
|
189
278
|
`Vyayam.executeStressTest()` creates an `AbortController` per iteration
|
|
@@ -216,10 +305,10 @@ providers are tracked in [`docs/BACKLOG.md`](./BACKLOG.md#trust-roadmap).
|
|
|
216
305
|
|
|
217
306
|
| Feature | Tests | Evidence Level |
|
|
218
307
|
|---|---|---|
|
|
219
|
-
| `HallucinationDetector` ensemble (best-of-N, claim grounding, abstention) | `test/hallucination-detectors.test.ts` — 22 tests | L1 |
|
|
220
|
-
| `Raksha.detectHallucination()` with Pulse emission | included above | L1 |
|
|
221
|
-
| `ModelRouter` / `ConfidenceRoutingTable` (Wilson 95% CI) | `test/model-router.test.ts` — 15 tests | L1 |
|
|
222
|
-
| `ResponseDistiller` (3 intensities, code-block-safe) | `test/response-distiller.test.ts` — 14 tests | L1 |
|
|
308
|
+
| `HallucinationDetector` ensemble (best-of-N, claim grounding, abstention) | `test/hallucination-detectors.test.ts` — 22 tests + Suite 12 benchmark | L1 + L2 |
|
|
309
|
+
| `Raksha.detectHallucination()` with Pulse emission | included above | L1 + L2 |
|
|
310
|
+
| `ModelRouter` / `ConfidenceRoutingTable` (Wilson 95% CI) | `test/model-router.test.ts` — 15 tests + Suite 13 benchmark | L1 + L2 |
|
|
311
|
+
| `ResponseDistiller` (3 intensities, code-block-safe) | `test/response-distiller.test.ts` — 14 tests + Suite 14 benchmark | L1 + L2 |
|
|
223
312
|
| Memory temperature (heat / decay / cold-threshold) + delta sync + typed nodes | `test/nidra-temperature-delta.test.ts` — 13 tests | L1 |
|
|
224
313
|
| Aahar tiered loading + omission marker + adaptive compression | `test/aahar-tiered-adaptive.test.ts` — 14 tests | L1 |
|
|
225
314
|
| Pulse context-budget milestones + cold-memory events | `test/pulse-milestones.test.ts` — 11 tests | L1 |
|
package/docs/MCP.md
CHANGED
|
@@ -129,8 +129,8 @@ The MCP server is designed for **local stdio use**. It assumes the MCP
|
|
|
129
129
|
host that starts the process is trusted. Agent IDs are routing
|
|
130
130
|
identifiers, **not credentials**, and there is no per-call authentication
|
|
131
131
|
inside the server — there is no portable stdio auth channel for one to
|
|
132
|
-
hook into. This is the intended trust boundary for v0.
|
|
133
|
-
fix; see the [security non-goals](./SECURITY.md#security-non-goals-for-
|
|
132
|
+
hook into. This is the intended trust boundary for v0.3, not a deferred
|
|
133
|
+
fix; see the [security non-goals](./SECURITY.md#security-non-goals-for-v03).
|
|
134
134
|
|
|
135
135
|
Recommended locked-down local configuration:
|
|
136
136
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@beingmartinbmc/ojas",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.3.1",
|
|
4
4
|
"description": "Ojas — AI Health Infrastructure for Autonomous Agents",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"author": "Ankit Sharma <ankit.sharma199803@gmail.com>",
|
|
@@ -54,6 +54,7 @@
|
|
|
54
54
|
"lint": "eslint \"src/**/*.ts\" \"benchmarks/**/*.ts\" \"test/**/*.ts\" \"examples/**/*.ts\"",
|
|
55
55
|
"check": "npm run lint && npm run build && npm run typecheck:aux && npm test",
|
|
56
56
|
"demo": "ts-node src/demo.ts",
|
|
57
|
+
"demo:canonical": "ts-node examples/canonical-pipeline.ts",
|
|
57
58
|
"demo:before-after": "ts-node examples/before-after.ts",
|
|
58
59
|
"mcp": "ts-node src/mcp/server.ts",
|
|
59
60
|
"mcp:built": "node dist/mcp/server.js",
|