@warlock.js/ai-panoptic 5.24.0 → 5.25.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +6 -0
- package/llms-full.txt +869 -869
- package/package.json +4 -4
- package/skills/evaluate-system-prompt/SKILL.md +86 -86
- package/skills/export-traces/SKILL.md +184 -184
- package/skills/observe-with-panoptic/SKILL.md +175 -175
- package/skills/query-traces/SKILL.md +203 -203
- package/skills/use-local-dashboard/SKILL.md +221 -221
|
@@ -1,175 +1,175 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: observe-with-panoptic
|
|
3
|
-
description: 'Wire @warlock.js/ai-panoptic into an agent/workflow/supervisor/orchestrator — declaratively via `ai.config({ panoptic })` (registers panoptic on core''s Observer seam) or with the one-call `panoptic({ exporters })` subscriber. Triggers: `panoptic`, `Panoptic`, `PanopticOptions`, `PanopticConfig`, `ai.config({ panoptic })`, `observeAll`, per-flow `observe`, `FlowObserveOption`, `registerObserver`, `Observer`, `.attach`, `.middleware`, `.collect`, `.toTrace`, `observe.attach(agent)`, `panoptic().middleware()`, `completedEvents`, `middlewareName`; ''observe a warlock agent'', ''trace an agent run'', ''attach observability to a workflow/supervisor'', ''observe every flow by default'', ''add a tracing middleware'', ''collect an orchestrator turn report'', ''wire OTel/Langfuse into my agent''; typical import `import { ai } from "@warlock.js/ai"` + `import "@warlock.js/ai-panoptic"` (or `import { panoptic } from "@warlock.js/ai-panoptic"`). Skip: the zero-setup local dashboard — `@warlock.js/ai-panoptic/use-local-dashboard/SKILL.md`; writing/choosing an exporter (the sink end) — `@warlock.js/ai-panoptic/export-traces/SKILL.md`; the core report shape / running the agent itself — `@warlock.js/ai/run-ai-agent/SKILL.md`.'
|
|
4
|
-
---
|
|
5
|
-
|
|
6
|
-
# Observe a run with Panoptic — the one-call subscriber
|
|
7
|
-
|
|
8
|
-
There are two levels. **Declarative** (`ai.config({ panoptic })`) is the high-level path: panoptic registers itself on core's generic `Observer` seam and every observed flow routes its report in automatically — no per-target wiring. **Imperative** (`panoptic(...)`) is the underlying subscriber, with three explicit feed paths. The declarative path is sugar over the imperative one; both reuse the same collection pipeline.
|
|
9
|
-
|
|
10
|
-
## Declarative — `ai.config({ panoptic })` + the Observer seam
|
|
11
|
-
|
|
12
|
-
The simplest wiring. Importing `@warlock.js/ai-panoptic` (even bare, for side effects) subscribes panoptic to core's config seam; a later `ai.config({ panoptic })` then builds the collector and **registers it once on core's `Observer` registry** via `registerObserver`. Core itself never imports panoptic — it depends only on the structural `Observer` interface (`{ collect(report) }`), the dependency inversion that keeps the two sides decoupled.
|
|
13
|
-
|
|
14
|
-
```ts
|
|
15
|
-
import { ai } from "@warlock.js/ai";
|
|
16
|
-
import "@warlock.js/ai-panoptic"; // side-effect wiring — do this once at boot
|
|
17
|
-
|
|
18
|
-
ai.config({
|
|
19
|
-
panoptic: {
|
|
20
|
-
exporters: [consoleExporter()], // optional; a store-shaped exporter also feeds the dashboard
|
|
21
|
-
observeAll: true, // observe every flow by default
|
|
22
|
-
dashboard: true, // optional local viewer — see use-local-dashboard
|
|
23
|
-
},
|
|
24
|
-
});
|
|
25
|
-
```
|
|
26
|
-
|
|
27
|
-
`PanopticConfig`: `exporters?` (the collector's sinks), `observeAll?` (default `false`), `dashboard?` (`boolean | DashboardOptions`). Applying it is **idempotent** — the observer registers once, the dashboard starts at most once; repeat `ai.config(...)` merges only refresh `observeAll`.
|
|
28
|
-
|
|
29
|
-
### `observeAll` + per-flow `observe`
|
|
30
|
-
|
|
31
|
-
Once a collector is registered, which flows actually route to it is controlled by core's `observe` seam:
|
|
32
|
-
|
|
33
|
-
| Setting | Effect |
|
|
34
|
-
|---|---|
|
|
35
|
-
| `observeAll: true` (config) | Every `ai.agent` / `ai.workflow` / `ai.supervisor` / `ai.team` that left `observe` unset routes its completed report to the registered observers. |
|
|
36
|
-
| `observe: true` (per-flow) | Force-observe **this** flow even when `observeAll` is off → the globally registered observers. |
|
|
37
|
-
| `observe: false` (per-flow) | Opt **this** flow out entirely, even when `observeAll` is on. |
|
|
38
|
-
| `observe: <Observer>` (per-flow) | A **flow-local** collector — only this flow's report, routed only to it (the global observers are skipped). A panoptic flow-local collector implements `Observer`, so pass it directly. |
|
|
39
|
-
| (unset, `observeAll: false`) | Not observed — opt-in default, identical to legacy behavior. |
|
|
40
|
-
|
|
41
|
-
```ts
|
|
42
|
-
ai.agent({ model, observe: true }); // always traced
|
|
43
|
-
ai.workflow({ name: "billing", steps, observe: true }); // always traced
|
|
44
|
-
ai.supervisor({ name: "triage", ..., observe: false }); // never traced, even under observeAll
|
|
45
|
-
|
|
46
|
-
const flowLocal = panoptic({ exporters: [fileExporter({ path: "billing.jsonl" })] });
|
|
47
|
-
ai.team({ name: "research", ..., observe: flowLocal }); // only this flow, only this collector
|
|
48
|
-
```
|
|
49
|
-
|
|
50
|
-
The flow awaits each observer's `collect`, and a throwing observer is **swallowed** — observability never breaks the run (mirrors `onUsage` / `onComplete`). `observe` lives on **all four** flow configs — `ai.agent`, `ai.workflow`, `ai.supervisor`, and `ai.team` (the agent additionally supports the imperative `attach()` / `middleware()` paths below).
|
|
51
|
-
|
|
52
|
-
## Imperative — `panoptic(...)`, the one-call subscriber
|
|
53
|
-
|
|
54
|
-
`panoptic(...)` builds a collector, registers your exporters on it, and hands back a subscriber with **three ways to feed traces** in. All three converge on the same collector, so each run reaches every exporter exactly once. Use it when you want explicit control, a flow-local collector, or to wire a target the declarative seam doesn't cover (the orchestrator).
|
|
55
|
-
|
|
56
|
-
```ts
|
|
57
|
-
import { panoptic } from "@warlock.js/ai-panoptic";
|
|
58
|
-
import { consoleExporter, otelExporter } from "@warlock.js/ai-panoptic";
|
|
59
|
-
|
|
60
|
-
const observe = panoptic({
|
|
61
|
-
exporters: [consoleExporter(), otelExporter({ tracerName: "my-app" })],
|
|
62
|
-
});
|
|
63
|
-
```
|
|
64
|
-
|
|
65
|
-
## Three feed paths
|
|
66
|
-
|
|
67
|
-
| Path | Use when | API |
|
|
68
|
-
|---|---|---|
|
|
69
|
-
| **Events** | You want every run captured automatically and the primitive emits a `*.completed` event (agent / workflow / supervisor). | `observe.attach(target)` |
|
|
70
|
-
| **Middleware** | The app already composes cross-cutting concerns through the agent middleware pipeline. | `panoptic().middleware()` |
|
|
71
|
-
| **Direct** | The primitive carries no result-bearing event (the orchestrator), or you captured a report out of band. | `observe.collect(report)` |
|
|
72
|
-
|
|
73
|
-
### 1. Attach to the event stream
|
|
74
|
-
|
|
75
|
-
`attach` subscribes to the terminal `*.completed` events and returns a **detach** function:
|
|
76
|
-
|
|
77
|
-
```ts
|
|
78
|
-
const agent = ai.agent({ model });
|
|
79
|
-
const detach = observe.attach(agent); // agent | workflow | supervisor
|
|
80
|
-
|
|
81
|
-
await agent.execute("Summarize this");
|
|
82
|
-
|
|
83
|
-
// ...later, on teardown:
|
|
84
|
-
detach();
|
|
85
|
-
await observe.shutdown();
|
|
86
|
-
```
|
|
87
|
-
|
|
88
|
-
By default Panoptic subscribes to `agent.completed`, `workflow.completed`, and `supervisor.completed` — the terminal events that carry the finalized `result.report`. These fire **once per run regardless of outcome** (the matching `*.error` fires first on failure, then `*.completed` still fires), so attaching captures completed, failed, and cancelled runs alike. Panoptic silently ignores the names a given target doesn't emit, so the same subscriber attaches to any of the three primitives.
|
|
89
|
-
|
|
90
|
-
Narrow or widen the surface with `completedEvents`:
|
|
91
|
-
|
|
92
|
-
```ts
|
|
93
|
-
panoptic({ exporters, completedEvents: ["agent.completed"] }); // agents only
|
|
94
|
-
```
|
|
95
|
-
|
|
96
|
-
### 2. Wire it as agent middleware
|
|
97
|
-
|
|
98
|
-
```ts
|
|
99
|
-
const observe = panoptic({ exporters: [langfuseExporter({ /* ... */ })] });
|
|
100
|
-
|
|
101
|
-
const agent = ai.agent({
|
|
102
|
-
model,
|
|
103
|
-
middleware: [observe.middleware()],
|
|
104
|
-
});
|
|
105
|
-
```
|
|
106
|
-
|
|
107
|
-
The middleware feeds the collector from the `execute`-level `after` (success) and `onError` (failure) hooks. It never returns a value, so it never mutates the agent's result. Register more than one Panoptic middleware on the same agent by passing a distinct `middlewareName`:
|
|
108
|
-
|
|
109
|
-
```ts
|
|
110
|
-
panoptic({ middlewareName: "panoptic-secondary" }).middleware();
|
|
111
|
-
```
|
|
112
|
-
|
|
113
|
-
### 3. Collect a report directly
|
|
114
|
-
|
|
115
|
-
The **orchestrator** emits `orchestrator.turn.*` events that carry only session identity — no result. Feed its report straight in:
|
|
116
|
-
|
|
117
|
-
```ts
|
|
118
|
-
const result = await orchestrator.execute(input, { sessionId });
|
|
119
|
-
await observe.collect(result.report);
|
|
120
|
-
```
|
|
121
|
-
|
|
122
|
-
`collect(report)` also covers any report captured out of band (e.g. replayed from a store).
|
|
123
|
-
|
|
124
|
-
## Failure isolation
|
|
125
|
-
|
|
126
|
-
An observability fault never escapes onto the AI run's hot path:
|
|
127
|
-
|
|
128
|
-
- The collector wraps every exporter invocation in `Promise.allSettled`, so a throwing exporter can't crash the run or block its siblings.
|
|
129
|
-
- The event handler and the middleware additionally swallow the `collect` rejection, so even a collector-level fault stays contained.
|
|
130
|
-
|
|
131
|
-
A broken backend degrades to *missing telemetry*, never a failed agent run.
|
|
132
|
-
|
|
133
|
-
## Lifecycle
|
|
134
|
-
|
|
135
|
-
```ts
|
|
136
|
-
await observe.flush(); // drain buffered exporters (between batches / checkpoints)
|
|
137
|
-
await observe.shutdown(); // flush + release every exporter (process teardown)
|
|
138
|
-
```
|
|
139
|
-
|
|
140
|
-
Call `shutdown()` on teardown so buffered exporters (`fileExporter` batches, the Langfuse client) send their last records before exit.
|
|
141
|
-
|
|
142
|
-
## Bring your own collector or store
|
|
143
|
-
|
|
144
|
-
The in-memory trace store doubles as an `ExporterContract`, so the simplest way to retain runs for later querying is to pass it as an exporter — it then receives every trace alongside your other sinks:
|
|
145
|
-
|
|
146
|
-
```ts
|
|
147
|
-
import { panoptic, createInMemoryTraceStore } from "@warlock.js/ai-panoptic";
|
|
148
|
-
|
|
149
|
-
const store = createInMemoryTraceStore(); // an ExporterContract + queryable store
|
|
150
|
-
const observe = panoptic({ exporters: [store, consoleExporter()] });
|
|
151
|
-
|
|
152
|
-
// store now receives every trace — query/aggregate it later:
|
|
153
|
-
store.query({ status: "failed" });
|
|
154
|
-
store.aggregate({ sessionId });
|
|
155
|
-
```
|
|
156
|
-
|
|
157
|
-
Pass a pre-built `collector` instead when you want to share one collector across several subscribers (each subscriber's `attach` / `middleware` feeds the same exporters):
|
|
158
|
-
|
|
159
|
-
```ts
|
|
160
|
-
import { panoptic, createCollector } from "@warlock.js/ai-panoptic";
|
|
161
|
-
|
|
162
|
-
const collector = createCollector().use(store);
|
|
163
|
-
const observe = panoptic({ collector, exporters: [consoleExporter()] });
|
|
164
|
-
```
|
|
165
|
-
|
|
166
|
-
Add exporters after construction with `observe.use(exporter)` (chainable, deduped by `ExporterContract.name`). `observe.toTrace(report)` projects a report into a `Trace` **without** dispatching — handy for inspection or tests.
|
|
167
|
-
|
|
168
|
-
> The store's full query / aggregate / capacity surface is its own skill — see [`query-traces/SKILL.md`](@warlock.js/ai-panoptic/query-traces/SKILL.md).
|
|
169
|
-
|
|
170
|
-
## See also
|
|
171
|
-
|
|
172
|
-
- [`@warlock.js/ai-panoptic/use-local-dashboard/SKILL.md`](@warlock.js/ai-panoptic/use-local-dashboard/SKILL.md) — the zero-setup local viewer the `ai.config({ panoptic: { dashboard } })` switch starts over the collector's store.
|
|
173
|
-
- [`@warlock.js/ai-panoptic/export-traces/SKILL.md`](@warlock.js/ai-panoptic/export-traces/SKILL.md) — the sink end: choosing/writing an exporter, the `gen_ai.*` / `warlock.*` attribute mapping.
|
|
174
|
-
- [`@warlock.js/ai-panoptic/query-traces/SKILL.md`](@warlock.js/ai-panoptic/query-traces/SKILL.md) — retaining traces in the queryable in-memory store and slicing them (`query` / `aggregate` / `capacity`).
|
|
175
|
-
- [`@warlock.js/ai/run-ai-agent/SKILL.md`](@warlock.js/ai/run-ai-agent/SKILL.md) — running the agent and the `BaseReport` shape Panoptic consumes.
|
|
1
|
+
---
|
|
2
|
+
name: observe-with-panoptic
|
|
3
|
+
description: 'Wire @warlock.js/ai-panoptic into an agent/workflow/supervisor/orchestrator — declaratively via `ai.config({ panoptic })` (registers panoptic on core''s Observer seam) or with the one-call `panoptic({ exporters })` subscriber. Triggers: `panoptic`, `Panoptic`, `PanopticOptions`, `PanopticConfig`, `ai.config({ panoptic })`, `observeAll`, per-flow `observe`, `FlowObserveOption`, `registerObserver`, `Observer`, `.attach`, `.middleware`, `.collect`, `.toTrace`, `observe.attach(agent)`, `panoptic().middleware()`, `completedEvents`, `middlewareName`; ''observe a warlock agent'', ''trace an agent run'', ''attach observability to a workflow/supervisor'', ''observe every flow by default'', ''add a tracing middleware'', ''collect an orchestrator turn report'', ''wire OTel/Langfuse into my agent''; typical import `import { ai } from "@warlock.js/ai"` + `import "@warlock.js/ai-panoptic"` (or `import { panoptic } from "@warlock.js/ai-panoptic"`). Skip: the zero-setup local dashboard — `@warlock.js/ai-panoptic/use-local-dashboard/SKILL.md`; writing/choosing an exporter (the sink end) — `@warlock.js/ai-panoptic/export-traces/SKILL.md`; the core report shape / running the agent itself — `@warlock.js/ai/run-ai-agent/SKILL.md`.'
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Observe a run with Panoptic — the one-call subscriber
|
|
7
|
+
|
|
8
|
+
There are two levels. **Declarative** (`ai.config({ panoptic })`) is the high-level path: panoptic registers itself on core's generic `Observer` seam and every observed flow routes its report in automatically — no per-target wiring. **Imperative** (`panoptic(...)`) is the underlying subscriber, with three explicit feed paths. The declarative path is sugar over the imperative one; both reuse the same collection pipeline.
|
|
9
|
+
|
|
10
|
+
## Declarative — `ai.config({ panoptic })` + the Observer seam
|
|
11
|
+
|
|
12
|
+
The simplest wiring. Importing `@warlock.js/ai-panoptic` (even bare, for side effects) subscribes panoptic to core's config seam; a later `ai.config({ panoptic })` then builds the collector and **registers it once on core's `Observer` registry** via `registerObserver`. Core itself never imports panoptic — it depends only on the structural `Observer` interface (`{ collect(report) }`), the dependency inversion that keeps the two sides decoupled.
|
|
13
|
+
|
|
14
|
+
```ts
|
|
15
|
+
import { ai } from "@warlock.js/ai";
|
|
16
|
+
import "@warlock.js/ai-panoptic"; // side-effect wiring — do this once at boot
|
|
17
|
+
|
|
18
|
+
ai.config({
|
|
19
|
+
panoptic: {
|
|
20
|
+
exporters: [consoleExporter()], // optional; a store-shaped exporter also feeds the dashboard
|
|
21
|
+
observeAll: true, // observe every flow by default
|
|
22
|
+
dashboard: true, // optional local viewer — see use-local-dashboard
|
|
23
|
+
},
|
|
24
|
+
});
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
`PanopticConfig`: `exporters?` (the collector's sinks), `observeAll?` (default `false`), `dashboard?` (`boolean | DashboardOptions`). Applying it is **idempotent** — the observer registers once, the dashboard starts at most once; repeat `ai.config(...)` merges only refresh `observeAll`.
|
|
28
|
+
|
|
29
|
+
### `observeAll` + per-flow `observe`
|
|
30
|
+
|
|
31
|
+
Once a collector is registered, which flows actually route to it is controlled by core's `observe` seam:
|
|
32
|
+
|
|
33
|
+
| Setting | Effect |
|
|
34
|
+
|---|---|
|
|
35
|
+
| `observeAll: true` (config) | Every `ai.agent` / `ai.workflow` / `ai.supervisor` / `ai.team` that left `observe` unset routes its completed report to the registered observers. |
|
|
36
|
+
| `observe: true` (per-flow) | Force-observe **this** flow even when `observeAll` is off → the globally registered observers. |
|
|
37
|
+
| `observe: false` (per-flow) | Opt **this** flow out entirely, even when `observeAll` is on. |
|
|
38
|
+
| `observe: <Observer>` (per-flow) | A **flow-local** collector — only this flow's report, routed only to it (the global observers are skipped). A panoptic flow-local collector implements `Observer`, so pass it directly. |
|
|
39
|
+
| (unset, `observeAll: false`) | Not observed — opt-in default, identical to legacy behavior. |
|
|
40
|
+
|
|
41
|
+
```ts
|
|
42
|
+
ai.agent({ model, observe: true }); // always traced
|
|
43
|
+
ai.workflow({ name: "billing", steps, observe: true }); // always traced
|
|
44
|
+
ai.supervisor({ name: "triage", ..., observe: false }); // never traced, even under observeAll
|
|
45
|
+
|
|
46
|
+
const flowLocal = panoptic({ exporters: [fileExporter({ path: "billing.jsonl" })] });
|
|
47
|
+
ai.team({ name: "research", ..., observe: flowLocal }); // only this flow, only this collector
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
The flow awaits each observer's `collect`, and a throwing observer is **swallowed** — observability never breaks the run (mirrors `onUsage` / `onComplete`). `observe` lives on **all four** flow configs — `ai.agent`, `ai.workflow`, `ai.supervisor`, and `ai.team` (the agent additionally supports the imperative `attach()` / `middleware()` paths below).
|
|
51
|
+
|
|
52
|
+
## Imperative — `panoptic(...)`, the one-call subscriber
|
|
53
|
+
|
|
54
|
+
`panoptic(...)` builds a collector, registers your exporters on it, and hands back a subscriber with **three ways to feed traces** in. All three converge on the same collector, so each run reaches every exporter exactly once. Use it when you want explicit control, a flow-local collector, or to wire a target the declarative seam doesn't cover (the orchestrator).
|
|
55
|
+
|
|
56
|
+
```ts
|
|
57
|
+
import { panoptic } from "@warlock.js/ai-panoptic";
|
|
58
|
+
import { consoleExporter, otelExporter } from "@warlock.js/ai-panoptic";
|
|
59
|
+
|
|
60
|
+
const observe = panoptic({
|
|
61
|
+
exporters: [consoleExporter(), otelExporter({ tracerName: "my-app" })],
|
|
62
|
+
});
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
## Three feed paths
|
|
66
|
+
|
|
67
|
+
| Path | Use when | API |
|
|
68
|
+
|---|---|---|
|
|
69
|
+
| **Events** | You want every run captured automatically and the primitive emits a `*.completed` event (agent / workflow / supervisor). | `observe.attach(target)` |
|
|
70
|
+
| **Middleware** | The app already composes cross-cutting concerns through the agent middleware pipeline. | `panoptic().middleware()` |
|
|
71
|
+
| **Direct** | The primitive carries no result-bearing event (the orchestrator), or you captured a report out of band. | `observe.collect(report)` |
|
|
72
|
+
|
|
73
|
+
### 1. Attach to the event stream
|
|
74
|
+
|
|
75
|
+
`attach` subscribes to the terminal `*.completed` events and returns a **detach** function:
|
|
76
|
+
|
|
77
|
+
```ts
|
|
78
|
+
const agent = ai.agent({ model });
|
|
79
|
+
const detach = observe.attach(agent); // agent | workflow | supervisor
|
|
80
|
+
|
|
81
|
+
await agent.execute("Summarize this");
|
|
82
|
+
|
|
83
|
+
// ...later, on teardown:
|
|
84
|
+
detach();
|
|
85
|
+
await observe.shutdown();
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
By default Panoptic subscribes to `agent.completed`, `workflow.completed`, and `supervisor.completed` — the terminal events that carry the finalized `result.report`. These fire **once per run regardless of outcome** (the matching `*.error` fires first on failure, then `*.completed` still fires), so attaching captures completed, failed, and cancelled runs alike. Panoptic silently ignores the names a given target doesn't emit, so the same subscriber attaches to any of the three primitives.
|
|
89
|
+
|
|
90
|
+
Narrow or widen the surface with `completedEvents`:
|
|
91
|
+
|
|
92
|
+
```ts
|
|
93
|
+
panoptic({ exporters, completedEvents: ["agent.completed"] }); // agents only
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
### 2. Wire it as agent middleware
|
|
97
|
+
|
|
98
|
+
```ts
|
|
99
|
+
const observe = panoptic({ exporters: [langfuseExporter({ /* ... */ })] });
|
|
100
|
+
|
|
101
|
+
const agent = ai.agent({
|
|
102
|
+
model,
|
|
103
|
+
middleware: [observe.middleware()],
|
|
104
|
+
});
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
The middleware feeds the collector from the `execute`-level `after` (success) and `onError` (failure) hooks. It never returns a value, so it never mutates the agent's result. Register more than one Panoptic middleware on the same agent by passing a distinct `middlewareName`:
|
|
108
|
+
|
|
109
|
+
```ts
|
|
110
|
+
panoptic({ middlewareName: "panoptic-secondary" }).middleware();
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
### 3. Collect a report directly
|
|
114
|
+
|
|
115
|
+
The **orchestrator** emits `orchestrator.turn.*` events that carry only session identity — no result. Feed its report straight in:
|
|
116
|
+
|
|
117
|
+
```ts
|
|
118
|
+
const result = await orchestrator.execute(input, { sessionId });
|
|
119
|
+
await observe.collect(result.report);
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
`collect(report)` also covers any report captured out of band (e.g. replayed from a store).
|
|
123
|
+
|
|
124
|
+
## Failure isolation
|
|
125
|
+
|
|
126
|
+
An observability fault never escapes onto the AI run's hot path:
|
|
127
|
+
|
|
128
|
+
- The collector wraps every exporter invocation in `Promise.allSettled`, so a throwing exporter can't crash the run or block its siblings.
|
|
129
|
+
- The event handler and the middleware additionally swallow the `collect` rejection, so even a collector-level fault stays contained.
|
|
130
|
+
|
|
131
|
+
A broken backend degrades to *missing telemetry*, never a failed agent run.
|
|
132
|
+
|
|
133
|
+
## Lifecycle
|
|
134
|
+
|
|
135
|
+
```ts
|
|
136
|
+
await observe.flush(); // drain buffered exporters (between batches / checkpoints)
|
|
137
|
+
await observe.shutdown(); // flush + release every exporter (process teardown)
|
|
138
|
+
```
|
|
139
|
+
|
|
140
|
+
Call `shutdown()` on teardown so buffered exporters (`fileExporter` batches, the Langfuse client) send their last records before exit.
|
|
141
|
+
|
|
142
|
+
## Bring your own collector or store
|
|
143
|
+
|
|
144
|
+
The in-memory trace store doubles as an `ExporterContract`, so the simplest way to retain runs for later querying is to pass it as an exporter — it then receives every trace alongside your other sinks:
|
|
145
|
+
|
|
146
|
+
```ts
|
|
147
|
+
import { panoptic, createInMemoryTraceStore } from "@warlock.js/ai-panoptic";
|
|
148
|
+
|
|
149
|
+
const store = createInMemoryTraceStore(); // an ExporterContract + queryable store
|
|
150
|
+
const observe = panoptic({ exporters: [store, consoleExporter()] });
|
|
151
|
+
|
|
152
|
+
// store now receives every trace — query/aggregate it later:
|
|
153
|
+
store.query({ status: "failed" });
|
|
154
|
+
store.aggregate({ sessionId });
|
|
155
|
+
```
|
|
156
|
+
|
|
157
|
+
Pass a pre-built `collector` instead when you want to share one collector across several subscribers (each subscriber's `attach` / `middleware` feeds the same exporters):
|
|
158
|
+
|
|
159
|
+
```ts
|
|
160
|
+
import { panoptic, createCollector } from "@warlock.js/ai-panoptic";
|
|
161
|
+
|
|
162
|
+
const collector = createCollector().use(store);
|
|
163
|
+
const observe = panoptic({ collector, exporters: [consoleExporter()] });
|
|
164
|
+
```
|
|
165
|
+
|
|
166
|
+
Add exporters after construction with `observe.use(exporter)` (chainable, deduped by `ExporterContract.name`). `observe.toTrace(report)` projects a report into a `Trace` **without** dispatching — handy for inspection or tests.
|
|
167
|
+
|
|
168
|
+
> The store's full query / aggregate / capacity surface is its own skill — see [`query-traces/SKILL.md`](@warlock.js/ai-panoptic/query-traces/SKILL.md).
|
|
169
|
+
|
|
170
|
+
## See also
|
|
171
|
+
|
|
172
|
+
- [`@warlock.js/ai-panoptic/use-local-dashboard/SKILL.md`](@warlock.js/ai-panoptic/use-local-dashboard/SKILL.md) — the zero-setup local viewer the `ai.config({ panoptic: { dashboard } })` switch starts over the collector's store.
|
|
173
|
+
- [`@warlock.js/ai-panoptic/export-traces/SKILL.md`](@warlock.js/ai-panoptic/export-traces/SKILL.md) — the sink end: choosing/writing an exporter, the `gen_ai.*` / `warlock.*` attribute mapping.
|
|
174
|
+
- [`@warlock.js/ai-panoptic/query-traces/SKILL.md`](@warlock.js/ai-panoptic/query-traces/SKILL.md) — retaining traces in the queryable in-memory store and slicing them (`query` / `aggregate` / `capacity`).
|
|
175
|
+
- [`@warlock.js/ai/run-ai-agent/SKILL.md`](@warlock.js/ai/run-ai-agent/SKILL.md) — running the agent and the `BaseReport` shape Panoptic consumes.
|