@visiq/harness 0.2.20 → 0.2.21
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +114 -0
- package/dist/index.d.ts +158 -0
- package/dist/index.js +4 -4
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -62,6 +62,120 @@ variables (option takes precedence, then the env var):
|
|
|
62
62
|
| `endpoint` | `VISIQ_ENDPOINT` | Backend endpoint URL. Optional — defaults to the managed SaaS host `https://api.visiqlabs.com`; set it explicitly only for onprem / self-hosted deployments. |
|
|
63
63
|
| `agentId` | `VISIQ_AGENT_ID` | Stable agent identity. If unset it is derived (package name → hostname) and auto-provisioned in monitor mode. |
|
|
64
64
|
| `hitlTimeoutMs` | `VISIQ_HITL_TIMEOUT_MS` | Max wait (ms) for a human to resolve an `approval_required` decision before failing closed. Default `120000`. |
|
|
65
|
+
| `timeoutMs` | `VISIQ_TIMEOUT_MS` | Per-evaluate network timeout before the action pre-gate fails **closed** on a stuck backend. Default 5s. Raise it for high-latency paths. |
|
|
66
|
+
| `onInstrumentFailure` | `VISIQ_ON_INSTRUMENT_FAILURE` | `'warn'` \| `'throw'` — what to do when a framework is detected but ZERO tools are instrumented. Unset is context-sensitive: an agent that ADVERTISES tools but instruments none fails **closed**; a genuinely tool-less agent only warns. |
|
|
67
|
+
| `failMode` | `VISIQ_FAIL_MODE` | `'open'` (default) \| `'closed'` — posture for **VisIQ's own** failures. Policy outcomes are unaffected. |
|
|
68
|
+
| `sessionId` | *(none — deliberately)* | Explicit session key for sequence-aware rules (`input.session.*`). See [Sessions](#sessions-and-sequence-aware-rules). |
|
|
69
|
+
| `identity` | *(none — deliberately)* | Per-call identity attestation for identity-binding rules. See [Identity](#identity-attestation). |
|
|
70
|
+
|
|
71
|
+
Two options have **no environment variable, on purpose**. `sessionId` and
|
|
72
|
+
`identity` are per-conversation and per-call facts; a process-wide env var would
|
|
73
|
+
apply one value to every request an agent-per-request server handles — fusing
|
|
74
|
+
unrelated users into one session, or attesting one user's principal for
|
|
75
|
+
everybody. Both are strictly worse than leaving them unset.
|
|
76
|
+
|
|
77
|
+
## Sessions and sequence-aware rules
|
|
78
|
+
|
|
79
|
+
A rule can condition on what the session has already done
|
|
80
|
+
(`input.session.event_count`, `input.session.action_counts.*`,
|
|
81
|
+
`input.session.retrieved_data_categories`, …). That needs a **session key**, and
|
|
82
|
+
the key comes from your application.
|
|
83
|
+
|
|
84
|
+
Resolution order, highest first:
|
|
85
|
+
|
|
86
|
+
1. `sessionId` passed to `visiq(target, { sessionId })`
|
|
87
|
+
2. LangChain/LangGraph only: `configurable.thread_id` on the call config
|
|
88
|
+
3. LangChain/LangGraph only: `configurable.session_id`
|
|
89
|
+
4. a fresh id per top-level run (so a caller who threads nothing is unaffected)
|
|
90
|
+
|
|
91
|
+
```ts
|
|
92
|
+
// LangGraph / LangChain — nothing to configure: thread your conversation id and
|
|
93
|
+
// the two turns share a session.
|
|
94
|
+
await graph.invoke(input, { configurable: { thread_id: conversationId } });
|
|
95
|
+
|
|
96
|
+
// Every other framework — name the conversation on the wrap.
|
|
97
|
+
const agent = visiq(new Agent({ model, tools }), { sessionId: conversationId });
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
### Reach, per framework
|
|
101
|
+
|
|
102
|
+
<!-- BEGIN GENERATED session-reach-table (src/session-reach.ts) -->
|
|
103
|
+
| framework | native conversation key | `sessionId` option | spans runs | concurrent runs isolated |
|
|
104
|
+
|---|---|---|---|---|
|
|
105
|
+
| `langchain` | yes | yes | yes | yes |
|
|
106
|
+
| `vercel_ai` | no | yes | yes | yes |
|
|
107
|
+
| `mastra` | no | yes | yes | yes |
|
|
108
|
+
| `voltagent` | no | yes | yes | yes |
|
|
109
|
+
| `llamaindex` | no | yes | yes | yes |
|
|
110
|
+
| `openai_agents` | no | yes | yes | no |
|
|
111
|
+
| `semantic_kernel` | no | yes | yes | no |
|
|
112
|
+
| `bare_tool` | no | yes | yes | no |
|
|
113
|
+
<!-- END GENERATED session-reach-table -->
|
|
114
|
+
|
|
115
|
+
- **native conversation key** — the harness reads a durable id from the
|
|
116
|
+
framework's own config, so you configure nothing.
|
|
117
|
+
- **concurrent runs isolated** — two simultaneous runs of ONE wrapped object in
|
|
118
|
+
one process do not share a key. Where this is `no`, the framework exposes no
|
|
119
|
+
per-run context the harness can hang a scope on. For `semantic_kernel` and
|
|
120
|
+
`bare_tool`, a host key you supply is by definition shared by everything that
|
|
121
|
+
uses it — pass a *per-conversation* value, not a process-wide one.
|
|
122
|
+
|
|
123
|
+
### ⚠️ Sessions accumulate IN-PROCESS
|
|
124
|
+
|
|
125
|
+
The trajectory is folded into an in-memory map on the harness's own runtime:
|
|
126
|
+
**512 live sessions (LRU), 30-minute idle expiry**, per process. Two turns that
|
|
127
|
+
land on two replicas therefore see two **fresh, empty** sessions, and a
|
|
128
|
+
sequence-aware rule can be correct, deployed and silently inert. If you run more
|
|
129
|
+
than one replica, pin conversation affinity at your load balancer, or accept that
|
|
130
|
+
`input.session.*` conditions only see the turns one replica handled.
|
|
131
|
+
|
|
132
|
+
## Identity attestation
|
|
133
|
+
|
|
134
|
+
Some rules bind a privileged action to a principal the session has **proven**:
|
|
135
|
+
|
|
136
|
+
```rego
|
|
137
|
+
deny if not input.normalized.write.subject_id
|
|
138
|
+
in input.session.identity.attested_subjects
|
|
139
|
+
```
|
|
140
|
+
|
|
141
|
+
Supply the principal per call:
|
|
142
|
+
|
|
143
|
+
```ts
|
|
144
|
+
const agent = visiq(new Agent({ model, tools }), {
|
|
145
|
+
sessionId: conversationId,
|
|
146
|
+
identity: () => {
|
|
147
|
+
const req = requestContext.getStore(); // YOUR request-scoped context
|
|
148
|
+
return req && { attested: [req.userId], subjectId: req.targetUserId };
|
|
149
|
+
},
|
|
150
|
+
});
|
|
151
|
+
```
|
|
152
|
+
|
|
153
|
+
| field | meaning |
|
|
154
|
+
|---|---|
|
|
155
|
+
| `attested` | principals your application can PROVE for this call — an IdP/session assertion, your request context, a verification tool's result. The only field that writes `attested_subjects`. |
|
|
156
|
+
| `claimed` | principals the **conversation** merely asserted. Grants nothing; exists so a rule can see the divergence. |
|
|
157
|
+
| `subjectId` | the human this action acts UPON. Becomes `normalized.write.subject_id`. |
|
|
158
|
+
|
|
159
|
+
**⚠️ VisIQ cannot verify this.** The harness runs inside your process, so
|
|
160
|
+
`attested` is an assertion by your application — recorded, folded and
|
|
161
|
+
rule-matchable, never independently checked. Source it from a channel the
|
|
162
|
+
untrusted party cannot author. **A principal the model read out of the
|
|
163
|
+
conversation belongs in `claimed`; if you cannot tell the two apart, pass
|
|
164
|
+
neither.**
|
|
165
|
+
|
|
166
|
+
Two behaviours worth knowing:
|
|
167
|
+
|
|
168
|
+
- The fold runs **after** the decision, so a principal attested on turn *N* is
|
|
169
|
+
visible from turn *N+1*. That is the intended pattern: a verification step
|
|
170
|
+
registers the principal, then the privileged action binds against it.
|
|
171
|
+
- With **no** `identity` supplied, a rule of the shape above does not fire at
|
|
172
|
+
all — its left operand is absent, so the condition is undefined and a
|
|
173
|
+
lower-priority rule decides. It is not a safe default; it is a no-op. Supply
|
|
174
|
+
`subjectId` to arm the rule.
|
|
175
|
+
|
|
176
|
+
Values are bounded to the control plane's own limits (≤32 principals of ≤255
|
|
177
|
+
characters). Oversized input is truncated rather than rejected — a rejected
|
|
178
|
+
event would fail the whole evaluation, which by default proceeds *ungoverned*.
|
|
65
179
|
|
|
66
180
|
## Multi-agent delegation
|
|
67
181
|
|
package/dist/index.d.ts
CHANGED
|
@@ -14,6 +14,118 @@ export declare const DEFAULT_FAIL_MODE: FailMode;
|
|
|
14
14
|
* must degrade to the DEFAULT, never invent strictness.
|
|
15
15
|
*/
|
|
16
16
|
export declare function normalizeFailMode(raw: unknown): FailMode | undefined;
|
|
17
|
+
/**
|
|
18
|
+
* IDENTITY ATTESTATION — the first-party seam an application uses to tell VisIQ
|
|
19
|
+
* *who* a governed call is being made for.
|
|
20
|
+
*
|
|
21
|
+
* ── WHY THIS FILE EXISTS ────────────────────────────────────────────────────
|
|
22
|
+
* The control plane teaches rule authors an identity-binding rule of the shape
|
|
23
|
+
*
|
|
24
|
+
* deny if not input.normalized.write.subject_id
|
|
25
|
+
* in input.session.identity.attested_subjects
|
|
26
|
+
*
|
|
27
|
+
* and `@visiq/runtime` has accepted `event.identity` since the fold shipped
|
|
28
|
+
* (`unified-runtime.ts` — `attested_subjects` / `claimed_subjects` /
|
|
29
|
+
* `acted_subjects`). But until this module landed, **this SDK never produced
|
|
30
|
+
* `event.identity` on any path**: there was not one `identity:` producer site in
|
|
31
|
+
* `packages/visiq-sdk-ts/src/`. This is the producer.
|
|
32
|
+
*
|
|
33
|
+
* ⚠️ AND THE DIRECTION OF THAT GAP IS THE OPPOSITE OF THE OBVIOUS READING.
|
|
34
|
+
* "`attested_subjects` is empty, so the rule denies everything" is wrong.
|
|
35
|
+
* MEASURED against the committed wasm (`identity-attestation.cell.test.ts`):
|
|
36
|
+
*
|
|
37
|
+
* what the harness sent taught binding rule
|
|
38
|
+
* ──────────────────────────────────────────── ───────────────────
|
|
39
|
+
* nothing (every @visiq/harness deployment) PERMIT ← silent no-op
|
|
40
|
+
* `normalized.write.subject_id`, attested [] DENY ← fail-closed
|
|
41
|
+
* `identity.attested` but NO `subjectId` PERMIT ← silent no-op
|
|
42
|
+
*
|
|
43
|
+
* The rule's LEFT operand is `input.normalized.write.subject_id`. Absent, the
|
|
44
|
+
* `in` relation is undefined, the rule body fails, the deny arm never fires, and
|
|
45
|
+
* a lower-priority catch-all decides. So the pre-seam behaviour was a SILENT
|
|
46
|
+
* FAIL-OPEN on the identity-binding rule class, not a safe deny — and it is the
|
|
47
|
+
* SUBJECT, not the attested set, that arms the rule. That is why
|
|
48
|
+
* {@link identityNormalized} materializes `normalized.write.subject_id` from
|
|
49
|
+
* `subjectId` rather than only forwarding `identity`.
|
|
50
|
+
*
|
|
51
|
+
* ── THE TRUST BOUNDARY, AND WHAT THE PLATFORM CAN AND CANNOT DO ─────────────
|
|
52
|
+
* **VisIQ CANNOT VERIFY WHAT YOU PASS HERE.** The harness runs inside your
|
|
53
|
+
* process; there is no channel by which the control plane could re-derive an
|
|
54
|
+
* IdP assertion from an in-process call. `attested` is therefore an ASSERTION BY
|
|
55
|
+
* YOUR APPLICATION, recorded and folded as such. What the platform guarantees is
|
|
56
|
+
* narrow and worth stating exactly:
|
|
57
|
+
*
|
|
58
|
+
* • it is the ONLY thing that writes `input.session.identity.attested_subjects`;
|
|
59
|
+
* • it is a grow-only set, so the fold stays commutative;
|
|
60
|
+
* • a principal that reaches `claimed` NEVER reaches `attested`, so a rule can
|
|
61
|
+
* SEE the divergence between "who the model says this is" and "who the
|
|
62
|
+
* application proved".
|
|
63
|
+
*
|
|
64
|
+
* So the security property is YOURS to uphold, and it is a single rule:
|
|
65
|
+
*
|
|
66
|
+
* **`attested` must come from a channel the untrusted party cannot author** —
|
|
67
|
+
* an IdP/session assertion, your own request context, or the RESULT of a
|
|
68
|
+
* verification tool. **NEVER a principal the model merely read out of the
|
|
69
|
+
* conversation.** Anything model-derived belongs in `claimed`, which grants
|
|
70
|
+
* nothing. A caller that cannot tell the two apart must pass NEITHER; passing
|
|
71
|
+
* a conversational claim as `attested` silently reintroduces exactly the
|
|
72
|
+
* forgery the binding exists to stop.
|
|
73
|
+
*
|
|
74
|
+
* ── BOUNDS (G003), AND WHY THEY ARE HERE AND NOT ONLY AT THE GATE ───────────
|
|
75
|
+
* `UnifiedRuntime` Zod-validates identity to `≤32` entries of `1..255` chars
|
|
76
|
+
* (`unified-runtime.ts` `evaluateEventSchema`). A violation THROWS out of
|
|
77
|
+
* `evaluate()`, which the harness treats as a HARNESS failure routed by
|
|
78
|
+
* `failMode` — i.e. fail-OPEN by default. So an application that handed us a
|
|
79
|
+
* 300-character subject id would make EVERY governed call in that process run
|
|
80
|
+
* UNGOVERNED, with only a warning. Normalizing here converts that into a
|
|
81
|
+
* silently-bounded value, exactly as `resolveLangchainSessionId` bounds the
|
|
82
|
+
* session key for the same reason.
|
|
83
|
+
*
|
|
84
|
+
* Truncation direction is deliberate: dropping entries from `attested` can only
|
|
85
|
+
* SHRINK the proven set, so a binding rule denies MORE, never less.
|
|
86
|
+
*/
|
|
87
|
+
/**
|
|
88
|
+
* An identity attestation for one governed call.
|
|
89
|
+
*
|
|
90
|
+
* Every field is optional; supply only what your application can actually
|
|
91
|
+
* establish for THIS call.
|
|
92
|
+
*/
|
|
93
|
+
export interface VisiqIdentity {
|
|
94
|
+
/**
|
|
95
|
+
* Principals this call is authorised to act for, established through a channel
|
|
96
|
+
* the untrusted party cannot author (an IdP/session assertion, your request
|
|
97
|
+
* context, a verification tool's RESULT).
|
|
98
|
+
*
|
|
99
|
+
* This is the ONLY field that writes `input.session.identity.attested_subjects`
|
|
100
|
+
* — the set a binding rule GRANTS on. Never put a model-derived principal here.
|
|
101
|
+
*/
|
|
102
|
+
attested?: string[];
|
|
103
|
+
/**
|
|
104
|
+
* Principals the CONVERSATION merely asserted (the model read a name out of
|
|
105
|
+
* user text). Grants nothing; it exists so a rule can see the divergence from
|
|
106
|
+
* `attested` rather than be blind to it.
|
|
107
|
+
*/
|
|
108
|
+
claimed?: string[];
|
|
109
|
+
/**
|
|
110
|
+
* The human this action ACTS UPON (`write/subject_id`) — the subject a binding
|
|
111
|
+
* rule checks against the attested set. Falls back to
|
|
112
|
+
* `normalized.write.subject_id` when omitted.
|
|
113
|
+
*/
|
|
114
|
+
subjectId?: string;
|
|
115
|
+
}
|
|
116
|
+
/**
|
|
117
|
+
* A per-call resolver. Called ONCE PER GOVERNED EVALUATION, so an
|
|
118
|
+
* agent-per-request server can read its own request-scoped context (its
|
|
119
|
+
* AsyncLocalStorage, its framework's request object) inside the callback and
|
|
120
|
+
* return a different principal per call — the shape a static object cannot
|
|
121
|
+
* express.
|
|
122
|
+
*
|
|
123
|
+
* It must not throw; if it does, the throw is reported and the event carries NO
|
|
124
|
+
* identity (see {@link makeIdentityResolver}).
|
|
125
|
+
*/
|
|
126
|
+
export type VisiqIdentityResolver = () => VisiqIdentity | undefined | null;
|
|
127
|
+
/** What `VisiqOptions.identity` accepts: a fixed attestation, or a per-call resolver. */
|
|
128
|
+
export type VisiqIdentityOption = VisiqIdentity | VisiqIdentityResolver;
|
|
17
129
|
export interface VisiqOptions {
|
|
18
130
|
/**
|
|
19
131
|
* Agent identity. Optional — resolution order is this option →
|
|
@@ -23,6 +135,52 @@ export interface VisiqOptions {
|
|
|
23
135
|
* for a stable, rule-friendly name.
|
|
24
136
|
*/
|
|
25
137
|
agentId?: string;
|
|
138
|
+
/**
|
|
139
|
+
* Explicit SESSION key for sequence-aware rules (`input.session.*`), overriding
|
|
140
|
+
* the conversation id this wrapper otherwise reads from LangChain's own config
|
|
141
|
+
* (`configurable.thread_id`, then `configurable.session_id`).
|
|
142
|
+
*
|
|
143
|
+
* Set it when the host knows the conversation boundary and LangChain does not.
|
|
144
|
+
* LEAVE IT UNSET for an agent-per-request server: a fixed value there would
|
|
145
|
+
* fuse every request in the process into one ever-growing session, which is the
|
|
146
|
+
* opposite failure to the one this resolution fixes — accumulating evidence
|
|
147
|
+
* across unrelated users rather than losing it between turns of one.
|
|
148
|
+
*/
|
|
149
|
+
sessionId?: string;
|
|
150
|
+
/**
|
|
151
|
+
* IDENTITY ATTESTATION for every governed call this wrap makes — the seam that
|
|
152
|
+
* populates `input.session.identity.attested_subjects`, the set an
|
|
153
|
+
* identity-binding rule GRANTS on.
|
|
154
|
+
*
|
|
155
|
+
* Accepts either a fixed attestation or, preferably, a RESOLVER called once
|
|
156
|
+
* per governed evaluation so an agent-per-request server can read its own
|
|
157
|
+
* request context per call:
|
|
158
|
+
*
|
|
159
|
+
* ```ts
|
|
160
|
+
* visiq(agent, {
|
|
161
|
+
* identity: () => {
|
|
162
|
+
* const req = requestContext.getStore(); // YOUR request ALS
|
|
163
|
+
* return req ? { attested: [req.userId], subjectId: req.targetUserId } : undefined;
|
|
164
|
+
* },
|
|
165
|
+
* });
|
|
166
|
+
* ```
|
|
167
|
+
*
|
|
168
|
+
* ⚠️ **VisIQ CANNOT VERIFY THIS.** The harness runs inside your process, so
|
|
169
|
+
* `attested` is an assertion BY YOUR APPLICATION — recorded, folded and
|
|
170
|
+
* rule-matchable, but never independently checked by the control plane. It is
|
|
171
|
+
* therefore yours to source from a channel the untrusted party cannot author
|
|
172
|
+
* (an IdP/session assertion, your request context, a verification tool's
|
|
173
|
+
* RESULT). A principal the MODEL read out of the conversation belongs in
|
|
174
|
+
* `claimed`, which grants nothing and exists so a rule can see the divergence.
|
|
175
|
+
* A caller that cannot tell the two apart must pass NEITHER.
|
|
176
|
+
*
|
|
177
|
+
* Leaving it unset is safe and is the default: `attested_subjects` stays empty
|
|
178
|
+
* and any identity-binding rule denies — the fail-closed direction. Values are
|
|
179
|
+
* bounded to the runtime's own limits (≤32 entries of ≤255 chars) so an
|
|
180
|
+
* oversized principal cannot throw out of `evaluate()` and fail the whole
|
|
181
|
+
* process OPEN.
|
|
182
|
+
*/
|
|
183
|
+
identity?: VisiqIdentityOption;
|
|
26
184
|
/** Backend API key. Falls back to VISIQ_API_KEY env var. */
|
|
27
185
|
apiKey?: string;
|
|
28
186
|
/** Backend endpoint URL. Falls back to VISIQ_ENDPOINT env var. */
|