@warlock.js/ai-panoptic 5.27.0 → 5.29.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -4,6 +4,18 @@ All notable changes to `@warlock.js/ai-panoptic` are documented in this file.
4
4
 
5
5
  The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). `@warlock.js/*` packages are released in lockstep — every package shares the same version number, so a version below may list only the changes that affected this package.
6
6
 
7
+ ## 5.29.0 - 2026-10-01
8
+
9
+ ### Changed
10
+
11
+ - Lockstep release; package APIs are unchanged. Skills refer to other topics by name instead of file paths, so cross-references keep working after `agent-kit` copies them into an app.
12
+
13
+ ## 5.28.0 - 2026-10-01
14
+
15
+ ### Changed
16
+
17
+ - Lockstep release; package APIs are unchanged. `llms.txt` now lists every skill topic with its description (frontmatter in CRLF files was skipped).
18
+
7
19
  ## 5.27.0 - 2026-10-01
8
20
 
9
21
  ### Changed
package/llms-full.txt CHANGED
@@ -8,12 +8,12 @@
8
8
 
9
9
  ---
10
10
  name: evaluate-system-prompt
11
- description: 'Grade a single trace''s LAST captured system prompt from the dashboard drawer — the dashboard''s only write-capable feature, config-gated via `DashboardOptions.evaluate` (`ai.config({ panoptic: { dashboard: { evaluate: { model, instructions } } } })`). Triggers: `evaluate`, `EvaluateConfig`, `EvaluateVerdict`, `evaluateSystemPrompt`, `extractLastSystemPrompt`, `findSpanById`, "Evaluate system prompt" button, `POST .../spans/:spanId/evaluate`, grading a prompt from a real trace, judging a captured system prompt, per-run instructions override, judge model factory; ''evaluate this trace''s prompt'', ''grade the system prompt from the dashboard'', ''judge a captured run'', ''score a live system prompt''. Skip: grading raw prompt TEXT you already have in code (no trace involved) — `@warlock.js/ai`''s `ai.prompts().validate({ criteria })`, the same judge machinery this wraps; a DATASET of test cases run through an agent — `@warlock.js/ai`''s `ai.dataset` + `agent.eval`; viewing/filtering/grouping traces without grading them — `@warlock.js/ai-panoptic/use-local-dashboard/SKILL.md`; capturing the content this feature reads — `@warlock.js/ai-panoptic/export-traces/SKILL.md` (`captureContent`).'
11
+ description: 'Grade a single trace''s LAST captured system prompt from the dashboard drawer — the dashboard''s only write-capable feature, config-gated via `DashboardOptions.evaluate` (`ai.config({ panoptic: { dashboard: { evaluate: { model, instructions } } } })`). Triggers: `evaluate`, `EvaluateConfig`, `EvaluateVerdict`, `evaluateSystemPrompt`, `extractLastSystemPrompt`, `findSpanById`, "Evaluate system prompt" button, `POST .../spans/:spanId/evaluate`, grading a prompt from a real trace, judging a captured system prompt, per-run instructions override, judge model factory; ''evaluate this trace''s prompt'', ''grade the system prompt from the dashboard'', ''judge a captured run'', ''score a live system prompt''. Skip: grading raw prompt TEXT you already have in code (no trace involved) — `@warlock.js/ai`''s `ai.prompts().validate({ criteria })`, the same judge machinery this wraps; a DATASET of test cases run through an agent — `@warlock.js/ai`''s `ai.dataset` + `agent.eval`; viewing/filtering/grouping traces without grading them — the `use-local-dashboard` topic; capturing the content this feature reads — the `export-traces` topic (`captureContent`).'
12
12
  ---
13
13
 
14
14
  # Evaluate a trace's system prompt from the dashboard
15
15
 
16
- A drawer button that grades the LAST `{role: "system"}` message captured on the selected span — not the whole conversation, not the tool code, just the system prompt that was actually in effect for that run. It's the dashboard's only write-capable feature: every other route is `GET`-only and read-only by design (see `use-local-dashboard/SKILL.md`'s security note); this one POSTs, and it's off unless you explicitly opt in.
16
+ A drawer button that grades the LAST `{role: "system"}` message captured on the selected span — not the whole conversation, not the tool code, just the system prompt that was actually in effect for that run. It's the dashboard's only write-capable feature: every other route is `GET`-only and read-only by design (see the `use-local-dashboard` topic's security note); this one POSTs, and it's off unless you explicitly opt in.
17
17
 
18
18
  ## Enable it
19
19
 
@@ -89,8 +89,8 @@ curl -X POST 'http://127.0.0.1:4319/api/traces/tr_1/spans/sp_1/evaluate' \
89
89
 
90
90
  ## See also
91
91
 
92
- - [`@warlock.js/ai-panoptic/use-local-dashboard/SKILL.md`](@warlock.js/ai-panoptic/use-local-dashboard/SKILL.md) — the dashboard this feature is a drawer action of; its security note now says "every route is GET-only and read-only EXCEPT the one opt-in write route `evaluate` enables."
93
- - [`@warlock.js/ai-panoptic/export-traces/SKILL.md`](@warlock.js/ai-panoptic/export-traces/SKILL.md) — `captureContent` (and `fullHistory`), the prerequisite for `span.input` to carry anything gradable.
92
+ - The `use-local-dashboard` topic — the dashboard this feature is a drawer action of; its security note now says "every route is GET-only and read-only EXCEPT the one opt-in write route `evaluate` enables."
93
+ - The `export-traces` topic — `captureContent` (and `fullHistory`), the prerequisite for `span.input` to carry anything gradable.
94
94
  - `ai.prompts().validate({ criteria })` in `@warlock.js/ai` — the sibling capability this reuses: grading raw prompt TEXT you already have in code, no trace or dashboard involved.
95
95
 
96
96
 
@@ -98,7 +98,7 @@ curl -X POST 'http://127.0.0.1:4319/api/traces/tr_1/spans/sp_1/evaluate' \
98
98
 
99
99
  ---
100
100
  name: export-traces
101
- description: 'Send @warlock.js/ai-panoptic traces to an observability backend via pluggable exporters, including capturing prompt/response content onto spans. Triggers: `consoleExporter`, `fileExporter`, `otelExporter`, `langfuseExporter`, `ExporterContract`, `ContentCaptureOptions`, `captureContent`, `redactContent`, `fullHistory`, `ContentRedactor`, `toGenAiAttributes`, `walkSpans`, `totalCostUsd`, `GEN_AI_ATTRIBUTES`, `WARLOCK_ATTRIBUTES`, `reportToTrace`, `reportToSpan`, `extractSpanAttributes`, `normalizeError`, `formatSpanLine`, `createPanopticMiddleware`; ''export AI traces to OpenTelemetry'', ''send traces to Langfuse'', ''log traces to console / a file'', ''capture the full prompt/conversation onto a span'', ''gen_ai semantic conventions'', ''write a custom exporter''; typical import `import { otelExporter, langfuseExporter, consoleExporter, fileExporter } from "@warlock.js/ai-panoptic"`. Skip: wiring the subscriber/collector into a run — `@warlock.js/ai-panoptic/observe-with-panoptic/SKILL.md`; the core report shape — `@warlock.js/ai/run-ai-agent/SKILL.md`.'
101
+ description: 'Send @warlock.js/ai-panoptic traces to an observability backend via pluggable exporters, including capturing prompt/response content onto spans. Triggers: `consoleExporter`, `fileExporter`, `otelExporter`, `langfuseExporter`, `ExporterContract`, `ContentCaptureOptions`, `captureContent`, `redactContent`, `fullHistory`, `ContentRedactor`, `toGenAiAttributes`, `walkSpans`, `totalCostUsd`, `GEN_AI_ATTRIBUTES`, `WARLOCK_ATTRIBUTES`, `reportToTrace`, `reportToSpan`, `extractSpanAttributes`, `normalizeError`, `formatSpanLine`, `createPanopticMiddleware`; ''export AI traces to OpenTelemetry'', ''send traces to Langfuse'', ''log traces to console / a file'', ''capture the full prompt/conversation onto a span'', ''gen_ai semantic conventions'', ''write a custom exporter''; typical import `import { otelExporter, langfuseExporter, consoleExporter, fileExporter } from "@warlock.js/ai-panoptic"`. Skip: wiring the subscriber/collector into a run — the `observe-with-panoptic` topic; the core report shape — the `run-ai-agent` topic of the `warlock-js-ai` skill.'
102
102
  ---
103
103
 
104
104
  # Export traces — the sink end of Panoptic
@@ -277,16 +277,16 @@ Reach for `reportToTrace` when you already hold a finished `result.report` and w
277
277
 
278
278
  ## See also
279
279
 
280
- - [`@warlock.js/ai-panoptic/observe-with-panoptic/SKILL.md`](@warlock.js/ai-panoptic/observe-with-panoptic/SKILL.md) — the source end: wiring the `panoptic(...)` subscriber into a run (`attach` / `middleware` / `collect`). The `collector` these exporters register on comes from there.
281
- - [`@warlock.js/ai-panoptic/query-traces/SKILL.md`](@warlock.js/ai-panoptic/query-traces/SKILL.md) — the in-memory trace store: a sibling sink that *retains* traces (instead of shipping them out) so you can `query` / `aggregate` them after the fact.
282
- - [`@warlock.js/ai/run-ai-agent/SKILL.md`](@warlock.js/ai/run-ai-agent/SKILL.md) — running the agent and the `BaseReport` shape Panoptic projects into a `Trace`.
280
+ - The `observe-with-panoptic` topic — the source end: wiring the `panoptic(...)` subscriber into a run (`attach` / `middleware` / `collect`). The `collector` these exporters register on comes from there.
281
+ - The `query-traces` topic — the in-memory trace store: a sibling sink that *retains* traces (instead of shipping them out) so you can `query` / `aggregate` them after the fact.
282
+ - The `run-ai-agent` topic of the `warlock-js-ai` skill — running the agent and the `BaseReport` shape Panoptic projects into a `Trace`.
283
283
 
284
284
 
285
285
  ## observe-with-panoptic `@warlock.js/ai-panoptic/observe-with-panoptic/SKILL.md`
286
286
 
287
287
  ---
288
288
  name: observe-with-panoptic
289
- description: 'Wire @warlock.js/ai-panoptic into an agent/workflow/supervisor/orchestrator — declaratively via `ai.config({ panoptic })` (registers panoptic on core''s Observer seam) or with the one-call `panoptic({ exporters })` subscriber. Triggers: `panoptic`, `Panoptic`, `PanopticOptions`, `PanopticConfig`, `ai.config({ panoptic })`, `observeAll`, per-flow `observe`, `FlowObserveOption`, `registerObserver`, `Observer`, `.attach`, `.middleware`, `.collect`, `.toTrace`, `observe.attach(agent)`, `panoptic().middleware()`, `completedEvents`, `middlewareName`; ''observe a warlock agent'', ''trace an agent run'', ''attach observability to a workflow/supervisor'', ''observe every flow by default'', ''add a tracing middleware'', ''collect an orchestrator turn report'', ''wire OTel/Langfuse into my agent''; typical import `import { ai } from "@warlock.js/ai"` + `import "@warlock.js/ai-panoptic"` (or `import { panoptic } from "@warlock.js/ai-panoptic"`). Skip: the zero-setup local dashboard — `@warlock.js/ai-panoptic/use-local-dashboard/SKILL.md`; writing/choosing an exporter (the sink end) — `@warlock.js/ai-panoptic/export-traces/SKILL.md`; the core report shape / running the agent itself — `@warlock.js/ai/run-ai-agent/SKILL.md`.'
289
+ description: 'Wire @warlock.js/ai-panoptic into an agent/workflow/supervisor/orchestrator — declaratively via `ai.config({ panoptic })` (registers panoptic on core''s Observer seam) or with the one-call `panoptic({ exporters })` subscriber. Triggers: `panoptic`, `Panoptic`, `PanopticOptions`, `PanopticConfig`, `ai.config({ panoptic })`, `observeAll`, per-flow `observe`, `FlowObserveOption`, `registerObserver`, `Observer`, `.attach`, `.middleware`, `.collect`, `.toTrace`, `observe.attach(agent)`, `panoptic().middleware()`, `completedEvents`, `middlewareName`; ''observe a warlock agent'', ''trace an agent run'', ''attach observability to a workflow/supervisor'', ''observe every flow by default'', ''add a tracing middleware'', ''collect an orchestrator turn report'', ''wire OTel/Langfuse into my agent''; typical import `import { ai } from "@warlock.js/ai"` + `import "@warlock.js/ai-panoptic"` (or `import { panoptic } from "@warlock.js/ai-panoptic"`). Skip: the zero-setup local dashboard — the `use-local-dashboard` topic; writing/choosing an exporter (the sink end) — the `export-traces` topic; the core report shape / running the agent itself — the `run-ai-agent` topic of the `warlock-js-ai` skill.'
290
290
  ---
291
291
 
292
292
  # Observe a run with Panoptic — the one-call subscriber
@@ -451,21 +451,21 @@ const observe = panoptic({ collector, exporters: [consoleExporter()] });
451
451
 
452
452
  Add exporters after construction with `observe.use(exporter)` (chainable, deduped by `ExporterContract.name`). `observe.toTrace(report)` projects a report into a `Trace` **without** dispatching — handy for inspection or tests.
453
453
 
454
- > The store's full query / aggregate / capacity surface is its own skill — see [`query-traces/SKILL.md`](@warlock.js/ai-panoptic/query-traces/SKILL.md).
454
+ > The store's full query / aggregate / capacity surface is its own skill — see the `query-traces` topic.
455
455
 
456
456
  ## See also
457
457
 
458
- - [`@warlock.js/ai-panoptic/use-local-dashboard/SKILL.md`](@warlock.js/ai-panoptic/use-local-dashboard/SKILL.md) — the zero-setup local viewer the `ai.config({ panoptic: { dashboard } })` switch starts over the collector's store.
459
- - [`@warlock.js/ai-panoptic/export-traces/SKILL.md`](@warlock.js/ai-panoptic/export-traces/SKILL.md) — the sink end: choosing/writing an exporter, the `gen_ai.*` / `warlock.*` attribute mapping.
460
- - [`@warlock.js/ai-panoptic/query-traces/SKILL.md`](@warlock.js/ai-panoptic/query-traces/SKILL.md) — retaining traces in the queryable in-memory store and slicing them (`query` / `aggregate` / `capacity`).
461
- - [`@warlock.js/ai/run-ai-agent/SKILL.md`](@warlock.js/ai/run-ai-agent/SKILL.md) — running the agent and the `BaseReport` shape Panoptic consumes.
458
+ - The `use-local-dashboard` topic — the zero-setup local viewer the `ai.config({ panoptic: { dashboard } })` switch starts over the collector's store.
459
+ - The `export-traces` topic — the sink end: choosing/writing an exporter, the `gen_ai.*` / `warlock.*` attribute mapping.
460
+ - The `query-traces` topic — retaining traces in the queryable in-memory store and slicing them (`query` / `aggregate` / `capacity`).
461
+ - The `run-ai-agent` topic of the `warlock-js-ai` skill — running the agent and the `BaseReport` shape Panoptic consumes.
462
462
 
463
463
 
464
464
  ## query-traces `@warlock.js/ai-panoptic/query-traces/SKILL.md`
465
465
 
466
466
  ---
467
467
  name: query-traces
468
- description: 'Retain @warlock.js/ai-panoptic traces in a queryable store — in-memory, or cache-backed so they survive a process restart — and slice them after the fact: get one run by id, list a session, filter failed runs in a window, roll up usage + cost. Triggers: `createInMemoryTraceStore`, `createCacheTraceStore`, `CacheTraceStoreHandle`, `CacheTraceStoreOptions`, `CacheDriverInput`, `store.ready`, `TraceStoreContract`, `TraceQuery`, `TraceAggregate`, `InMemoryTraceStoreOptions`, `store.query`, `store.aggregate`, `store.get`, `store.add`, `store.clear`, `store.size`, `capacity`, `prefix`, `onError`, `sumUsage`, `emptyUsage`, `matchTrace`; ''query collected traces'', ''aggregate AI cost per session'', ''how much did this session spend'', ''list failed runs'', ''retain traces in memory'', ''persist traces across a restart'', ''cache-backed trace store'', ''Redis trace store'', ''roll up token usage''; typical import `import { createInMemoryTraceStore, createCacheTraceStore } from "@warlock.js/ai-panoptic"`. Skip: sending traces to an external backend (OTel/Langfuse/console/file) — `@warlock.js/ai-panoptic/export-traces/SKILL.md`; wiring the subscriber into a run — `@warlock.js/ai-panoptic/observe-with-panoptic/SKILL.md`.'
468
+ description: 'Retain @warlock.js/ai-panoptic traces in a queryable store — in-memory, or cache-backed so they survive a process restart — and slice them after the fact: get one run by id, list a session, filter failed runs in a window, roll up usage + cost. Triggers: `createInMemoryTraceStore`, `createCacheTraceStore`, `CacheTraceStoreHandle`, `CacheTraceStoreOptions`, `CacheDriverInput`, `store.ready`, `TraceStoreContract`, `TraceQuery`, `TraceAggregate`, `InMemoryTraceStoreOptions`, `store.query`, `store.aggregate`, `store.get`, `store.add`, `store.clear`, `store.size`, `capacity`, `prefix`, `onError`, `sumUsage`, `emptyUsage`, `matchTrace`; ''query collected traces'', ''aggregate AI cost per session'', ''how much did this session spend'', ''list failed runs'', ''retain traces in memory'', ''persist traces across a restart'', ''cache-backed trace store'', ''Redis trace store'', ''roll up token usage''; typical import `import { createInMemoryTraceStore, createCacheTraceStore } from "@warlock.js/ai-panoptic"`. Skip: sending traces to an external backend (OTel/Langfuse/console/file) — the `export-traces` topic; wiring the subscriber into a run — the `observe-with-panoptic` topic.'
469
469
  ---
470
470
 
471
471
  # Query traces — the in-memory queryable store
@@ -492,7 +492,7 @@ const failed = store.query({ status: "failed" });
492
492
  const spend = store.aggregate({ sessionId: "session-42" });
493
493
  ```
494
494
 
495
- > The store is also a valid `collector.use(...)` target directly — `panoptic({ collector: createCollector().use(store) })` — see [`observe-with-panoptic/SKILL.md`](@warlock.js/ai-panoptic/observe-with-panoptic/SKILL.md) for bring-your-own-collector wiring.
495
+ > The store is also a valid `collector.use(...)` target directly — `panoptic({ collector: createCollector().use(store) })` — see the `observe-with-panoptic` topic for bring-your-own-collector wiring.
496
496
 
497
497
  ## The store contract
498
498
 
@@ -593,7 +593,7 @@ const store = createInMemoryTraceStore({ capacity: 1000 });
593
593
 
594
594
  Overwriting an existing `traceId` refreshes its insertion position, so a re-collected run is treated as newest for eviction — eviction stays honest under overwrite.
595
595
 
596
- > The in-memory store has no secondary indexes: `get` / `add` are O(1), `query` / `aggregate` are O(n) scans. That is the right trade-off for dev/test and modest-volume runtime use. For high-volume or durable retention, write a custom `TraceStoreContract` (or export to a backend with [`export-traces`](@warlock.js/ai-panoptic/export-traces/SKILL.md)) backed by a real datastore.
596
+ > The in-memory store has no secondary indexes: `get` / `add` are O(1), `query` / `aggregate` are O(n) scans. That is the right trade-off for dev/test and modest-volume runtime use. For high-volume or durable retention, write a custom `TraceStoreContract` (or export to a backend with the `export-traces` topic) backed by a real datastore.
597
597
 
598
598
  ## Persisting across a restart — `createCacheTraceStore`
599
599
 
@@ -644,7 +644,7 @@ type CacheTraceStoreHandle = TraceStoreContract & ExporterContract & {
644
644
 
645
645
  Ordering uses an **internal monotonic counter**, not `Date.now()`, so the persisted index stays a stable total order even when many traces land in the same millisecond — FIFO eviction stays honest across a restart.
646
646
 
647
- > The usual path is declarative: `ai.config({ panoptic: { cache, dashboard: true } })` constructs this store, awaits `ready()`, and wires it under the dashboard for you — see [`use-local-dashboard/SKILL.md`](@warlock.js/ai-panoptic/use-local-dashboard/SKILL.md). Reach for `createCacheTraceStore` directly only when you manage the store / collector yourself.
647
+ > The usual path is declarative: `ai.config({ panoptic: { cache, dashboard: true } })` constructs this store, awaits `ready()`, and wires it under the dashboard for you — see the `use-local-dashboard` topic. Reach for `createCacheTraceStore` directly only when you manage the store / collector yourself.
648
648
 
649
649
  ## Summing usage yourself
650
650
 
@@ -663,16 +663,16 @@ for (const trace of store.query({ sessionId })) {
663
663
 
664
664
  ## See also
665
665
 
666
- - [`@warlock.js/ai-panoptic/observe-with-panoptic/SKILL.md`](@warlock.js/ai-panoptic/observe-with-panoptic/SKILL.md) — wiring the `panoptic(...)` subscriber and bring-your-own-collector so the store fills as runs complete.
667
- - [`@warlock.js/ai-panoptic/export-traces/SKILL.md`](@warlock.js/ai-panoptic/export-traces/SKILL.md) — sending traces to an external backend instead of (or alongside) retaining them in memory; writing a custom sink.
668
- - [`@warlock.js/ai/run-ai-agent/SKILL.md`](@warlock.js/ai/run-ai-agent/SKILL.md) — the `BaseReport` the collector projects into the `Trace` this store retains.
666
+ - The `observe-with-panoptic` topic — wiring the `panoptic(...)` subscriber and bring-your-own-collector so the store fills as runs complete.
667
+ - The `export-traces` topic — sending traces to an external backend instead of (or alongside) retaining them in memory; writing a custom sink.
668
+ - The `run-ai-agent` topic of the `warlock-js-ai` skill — the `BaseReport` the collector projects into the `Trace` this store retains.
669
669
 
670
670
 
671
671
  ## use-local-dashboard `@warlock.js/ai-panoptic/use-local-dashboard/SKILL.md`
672
672
 
673
673
  ---
674
674
  name: use-local-dashboard
675
- description: 'Run the zero-setup local Panoptic dashboard — a loopback `node:http` server over a trace store served from `ai.config({ panoptic: { dashboard } })`, with a two-pane call-tree + metadata UI (light/dark/system theme, search/filter, session + prompt-version grouping, cost heatmap, Gantt timeline, errors filter, hash deep-links) and an optional cache-backed persistent store. Triggers: `dashboard`, `DashboardOptions`, `DashboardHandle`, `ai.config({ panoptic })`, `panoptic.dashboard`, `panoptic.cache`, `port: 4319`, `basePath`, `open`, `authToken`, `allowedHosts`, `/api/traces`, `/api/aggregate`, `/api/traces/:id`, `createRequestHandler`, `dashboardHtml`, `sanitizeHref`, `ServeConfig`, `TraceStoreContract`, `createCacheTraceStore`, theme, dark mode, call tree, metadata panel, cost heatmap, timeline / Gantt, errors only, deep-link, group by session, prompt version, group by type, type stats panel, started / ended timestamps, team type; ''open a local AI trace dashboard'', ''view agent traces in the browser'', ''self-hosted panoptic UI'', ''inspect traces without Langfuse'', ''spin up a dev observability server'', ''dark mode dashboard'', ''persist traces across restart'', ''group traces by session / prompt version / type'', ''cost heatmap'', ''timeline / waterfall view'', ''filter failed runs in the dashboard'', ''per-type stats panel'', ''is the dashboard bearer token safe'', ''markdown links XSS in trace content''; typical import `import { ai } from "@warlock.js/ai"` + `import "@warlock.js/ai-panoptic"`. Skip: shipping traces to a backend (OTel / Langfuse / file) — `@warlock.js/ai-panoptic/export-traces/SKILL.md`; the queryable store API the dashboard reads — `@warlock.js/ai-panoptic/query-traces/SKILL.md`; wiring the subscriber by hand — `@warlock.js/ai-panoptic/observe-with-panoptic/SKILL.md`.'
675
+ description: 'Run the zero-setup local Panoptic dashboard — a loopback `node:http` server over a trace store served from `ai.config({ panoptic: { dashboard } })`, with a two-pane call-tree + metadata UI (light/dark/system theme, search/filter, session + prompt-version grouping, cost heatmap, Gantt timeline, errors filter, hash deep-links) and an optional cache-backed persistent store. Triggers: `dashboard`, `DashboardOptions`, `DashboardHandle`, `ai.config({ panoptic })`, `panoptic.dashboard`, `panoptic.cache`, `port: 4319`, `basePath`, `open`, `authToken`, `allowedHosts`, `/api/traces`, `/api/aggregate`, `/api/traces/:id`, `createRequestHandler`, `dashboardHtml`, `sanitizeHref`, `ServeConfig`, `TraceStoreContract`, `createCacheTraceStore`, theme, dark mode, call tree, metadata panel, cost heatmap, timeline / Gantt, errors only, deep-link, group by session, prompt version, group by type, type stats panel, started / ended timestamps, team type; ''open a local AI trace dashboard'', ''view agent traces in the browser'', ''self-hosted panoptic UI'', ''inspect traces without Langfuse'', ''spin up a dev observability server'', ''dark mode dashboard'', ''persist traces across restart'', ''group traces by session / prompt version / type'', ''cost heatmap'', ''timeline / waterfall view'', ''filter failed runs in the dashboard'', ''per-type stats panel'', ''is the dashboard bearer token safe'', ''markdown links XSS in trace content''; typical import `import { ai } from "@warlock.js/ai"` + `import "@warlock.js/ai-panoptic"`. Skip: shipping traces to a backend (OTel / Langfuse / file) — the `export-traces` topic; the queryable store API the dashboard reads — the `query-traces` topic; wiring the subscriber by hand — the `observe-with-panoptic` topic.'
676
676
  ---
677
677
 
678
678
  # The local Panoptic dashboard — zero-setup trace viewing
@@ -791,7 +791,7 @@ ai.config({
791
791
  });
792
792
  ```
793
793
 
794
- Panoptic awaits the store's `ready()` hydration **before** the server starts serving, so a restart surfaces previously-persisted traces immediately. Reads stay synchronous (served from an in-memory mirror); writes go through to the cache fire-and-forget, so a flaky cache never throws into the collector's hot path. A store-shaped exporter you pass via `exporters` still wins over `cache`. The store building block (`createCacheTraceStore`) and its durability contract live in [`query-traces/SKILL.md`](@warlock.js/ai-panoptic/query-traces/SKILL.md).
794
+ Panoptic awaits the store's `ready()` hydration **before** the server starts serving, so a restart surfaces previously-persisted traces immediately. Reads stay synchronous (served from an in-memory mirror); writes go through to the cache fire-and-forget, so a flaky cache never throws into the collector's hot path. A store-shaped exporter you pass via `exporters` still wins over `cache`. The store building block (`createCacheTraceStore`) and its durability contract live in the `query-traces` topic.
795
795
 
796
796
  ## The low-level building block — `dashboard(store, options)`
797
797
 
@@ -838,7 +838,7 @@ type DashboardHandle = {
838
838
 
839
839
  ## The JSON API
840
840
 
841
- Every route is **`GET`-only and read-only** (non-`GET` → `405`), mounted under `basePath` — EXCEPT the one opt-in `POST .../spans/:spanId/evaluate` route `DashboardOptions.evaluate` enables (see [`evaluate-system-prompt/SKILL.md`](@warlock.js/ai-panoptic/evaluate-system-prompt/SKILL.md)); absent that config, it 405s like everything else. The page polls the `GET` routes; you can curl them too. Store shapes are already JSON-safe, so responses are a plain `JSON.stringify`.
841
+ Every route is **`GET`-only and read-only** (non-`GET` → `405`), mounted under `basePath` — EXCEPT the one opt-in `POST .../spans/:spanId/evaluate` route `DashboardOptions.evaluate` enables (see the `evaluate-system-prompt` topic); absent that config, it 405s like everything else. The page polls the `GET` routes; you can curl them too. Store shapes are already JSON-safe, so responses are a plain `JSON.stringify`.
842
842
 
843
843
  | Route | Returns |
844
844
  |---|---|
@@ -855,7 +855,7 @@ curl 'http://127.0.0.1:4319/api/traces?status=failed'
855
855
  curl 'http://127.0.0.1:4319/api/aggregate?sessionId=sess_42'
856
856
  ```
857
857
 
858
- (The exact filter keys and the aggregate shape live in [`query-traces/SKILL.md`](@warlock.js/ai-panoptic/query-traces/SKILL.md).)
858
+ (The exact filter keys and the aggregate shape live in the `query-traces` topic.)
859
859
 
860
860
  ## Advanced — build the handler yourself
861
861
 
@@ -887,9 +887,9 @@ These three findings (link sanitization + the two token-handling items above) ca
887
887
 
888
888
  ## See also
889
889
 
890
- - [`@warlock.js/ai-panoptic/observe-with-panoptic/SKILL.md`](@warlock.js/ai-panoptic/observe-with-panoptic/SKILL.md) — the `ai.config({ panoptic })` wiring and the underlying `panoptic(...)` subscriber that fills the store.
891
- - [`@warlock.js/ai-panoptic/query-traces/SKILL.md`](@warlock.js/ai-panoptic/query-traces/SKILL.md) — the `TraceStoreContract` (`query` / `get` / `aggregate`) the dashboard reads, the `TraceQuery` filter, and capacity eviction.
892
- - [`@warlock.js/ai-panoptic/export-traces/SKILL.md`](@warlock.js/ai-panoptic/export-traces/SKILL.md) — shipping traces to OTel / Langfuse / a file instead of (or alongside) the local dashboard.
893
- - [`@warlock.js/ai-panoptic/evaluate-system-prompt/SKILL.md`](@warlock.js/ai-panoptic/evaluate-system-prompt/SKILL.md) — the drawer's "Evaluate system prompt" action, the dashboard's one write route.
890
+ - The `observe-with-panoptic` topic — the `ai.config({ panoptic })` wiring and the underlying `panoptic(...)` subscriber that fills the store.
891
+ - The `query-traces` topic — the `TraceStoreContract` (`query` / `get` / `aggregate`) the dashboard reads, the `TraceQuery` filter, and capacity eviction.
892
+ - The `export-traces` topic — shipping traces to OTel / Langfuse / a file instead of (or alongside) the local dashboard.
893
+ - The `evaluate-system-prompt` topic — the drawer's "Evaluate system prompt" action, the dashboard's one write route.
894
894
 
895
895
 
package/llms.txt CHANGED
@@ -6,8 +6,8 @@
6
6
 
7
7
  ## Skills
8
8
 
9
- - [evaluate-system-prompt](@warlock.js/ai-panoptic/evaluate-system-prompt/SKILL.md): (no description)
10
- - [export-traces](@warlock.js/ai-panoptic/export-traces/SKILL.md): (no description)
11
- - [observe-with-panoptic](@warlock.js/ai-panoptic/observe-with-panoptic/SKILL.md): (no description)
12
- - [query-traces](@warlock.js/ai-panoptic/query-traces/SKILL.md): (no description)
13
- - [use-local-dashboard](@warlock.js/ai-panoptic/use-local-dashboard/SKILL.md): (no description)
9
+ - [evaluate-system-prompt](@warlock.js/ai-panoptic/evaluate-system-prompt/SKILL.md): Grade a single trace's LAST captured system prompt from the dashboard drawer — the dashboard's only write-capable feature, config-gated via `DashboardOptions.evaluate` (`ai.config({ panoptic: { dashboard: { evaluate: { model, instructions } } } })`). Triggers: `evaluate`, `EvaluateConfig`, `EvaluateVerdict`, `evaluateSystemPrompt`, `extractLastSystemPrompt`, `findSpanById`, "Evaluate system prompt" button, `POST .../spans/:spanId/evaluate`, grading a prompt from a real trace, judging a captured system prompt, per-run instructions override, judge model factory; 'evaluate this trace's prompt', 'grade the system prompt from the dashboard', 'judge a captured run', 'score a live system prompt'. Skip: grading raw prompt TEXT you already have in code (no trace involved) — `@warlock.js/ai`'s `ai.prompts().validate({ criteria })`, the same judge machinery this wraps; a DATASET of test cases run through an agent — `@warlock.js/ai`'s `ai.dataset` + `agent.eval`; viewing/filtering/grouping traces without grading them — the `use-local-dashboard` topic; capturing the content this feature reads — the `export-traces` topic (`captureContent`).
10
+ - [export-traces](@warlock.js/ai-panoptic/export-traces/SKILL.md): Send @warlock.js/ai-panoptic traces to an observability backend via pluggable exporters, including capturing prompt/response content onto spans. Triggers: `consoleExporter`, `fileExporter`, `otelExporter`, `langfuseExporter`, `ExporterContract`, `ContentCaptureOptions`, `captureContent`, `redactContent`, `fullHistory`, `ContentRedactor`, `toGenAiAttributes`, `walkSpans`, `totalCostUsd`, `GEN_AI_ATTRIBUTES`, `WARLOCK_ATTRIBUTES`, `reportToTrace`, `reportToSpan`, `extractSpanAttributes`, `normalizeError`, `formatSpanLine`, `createPanopticMiddleware`; 'export AI traces to OpenTelemetry', 'send traces to Langfuse', 'log traces to console / a file', 'capture the full prompt/conversation onto a span', 'gen_ai semantic conventions', 'write a custom exporter'; typical import `import { otelExporter, langfuseExporter, consoleExporter, fileExporter } from "@warlock.js/ai-panoptic"`. Skip: wiring the subscriber/collector into a run — the `observe-with-panoptic` topic; the core report shape — the `run-ai-agent` topic of the `warlock-js-ai` skill.
11
+ - [observe-with-panoptic](@warlock.js/ai-panoptic/observe-with-panoptic/SKILL.md): Wire @warlock.js/ai-panoptic into an agent/workflow/supervisor/orchestrator — declaratively via `ai.config({ panoptic })` (registers panoptic on core's Observer seam) or with the one-call `panoptic({ exporters })` subscriber. Triggers: `panoptic`, `Panoptic`, `PanopticOptions`, `PanopticConfig`, `ai.config({ panoptic })`, `observeAll`, per-flow `observe`, `FlowObserveOption`, `registerObserver`, `Observer`, `.attach`, `.middleware`, `.collect`, `.toTrace`, `observe.attach(agent)`, `panoptic().middleware()`, `completedEvents`, `middlewareName`; 'observe a warlock agent', 'trace an agent run', 'attach observability to a workflow/supervisor', 'observe every flow by default', 'add a tracing middleware', 'collect an orchestrator turn report', 'wire OTel/Langfuse into my agent'; typical import `import { ai } from "@warlock.js/ai"` + `import "@warlock.js/ai-panoptic"` (or `import { panoptic } from "@warlock.js/ai-panoptic"`). Skip: the zero-setup local dashboard — the `use-local-dashboard` topic; writing/choosing an exporter (the sink end) — the `export-traces` topic; the core report shape / running the agent itself — the `run-ai-agent` topic of the `warlock-js-ai` skill.
12
+ - [query-traces](@warlock.js/ai-panoptic/query-traces/SKILL.md): Retain @warlock.js/ai-panoptic traces in a queryable store — in-memory, or cache-backed so they survive a process restart — and slice them after the fact: get one run by id, list a session, filter failed runs in a window, roll up usage + cost. Triggers: `createInMemoryTraceStore`, `createCacheTraceStore`, `CacheTraceStoreHandle`, `CacheTraceStoreOptions`, `CacheDriverInput`, `store.ready`, `TraceStoreContract`, `TraceQuery`, `TraceAggregate`, `InMemoryTraceStoreOptions`, `store.query`, `store.aggregate`, `store.get`, `store.add`, `store.clear`, `store.size`, `capacity`, `prefix`, `onError`, `sumUsage`, `emptyUsage`, `matchTrace`; 'query collected traces', 'aggregate AI cost per session', 'how much did this session spend', 'list failed runs', 'retain traces in memory', 'persist traces across a restart', 'cache-backed trace store', 'Redis trace store', 'roll up token usage'; typical import `import { createInMemoryTraceStore, createCacheTraceStore } from "@warlock.js/ai-panoptic"`. Skip: sending traces to an external backend (OTel/Langfuse/console/file) — the `export-traces` topic; wiring the subscriber into a run — the `observe-with-panoptic` topic.
13
+ - [use-local-dashboard](@warlock.js/ai-panoptic/use-local-dashboard/SKILL.md): Run the zero-setup local Panoptic dashboard — a loopback `node:http` server over a trace store served from `ai.config({ panoptic: { dashboard } })`, with a two-pane call-tree + metadata UI (light/dark/system theme, search/filter, session + prompt-version grouping, cost heatmap, Gantt timeline, errors filter, hash deep-links) and an optional cache-backed persistent store. Triggers: `dashboard`, `DashboardOptions`, `DashboardHandle`, `ai.config({ panoptic })`, `panoptic.dashboard`, `panoptic.cache`, `port: 4319`, `basePath`, `open`, `authToken`, `allowedHosts`, `/api/traces`, `/api/aggregate`, `/api/traces/:id`, `createRequestHandler`, `dashboardHtml`, `sanitizeHref`, `ServeConfig`, `TraceStoreContract`, `createCacheTraceStore`, theme, dark mode, call tree, metadata panel, cost heatmap, timeline / Gantt, errors only, deep-link, group by session, prompt version, group by type, type stats panel, started / ended timestamps, team type; 'open a local AI trace dashboard', 'view agent traces in the browser', 'self-hosted panoptic UI', 'inspect traces without Langfuse', 'spin up a dev observability server', 'dark mode dashboard', 'persist traces across restart', 'group traces by session / prompt version / type', 'cost heatmap', 'timeline / waterfall view', 'filter failed runs in the dashboard', 'per-type stats panel', 'is the dashboard bearer token safe', 'markdown links XSS in trace content'; typical import `import { ai } from "@warlock.js/ai"` + `import "@warlock.js/ai-panoptic"`. Skip: shipping traces to a backend (OTel / Langfuse / file) — the `export-traces` topic; the queryable store API the dashboard reads — the `query-traces` topic; wiring the subscriber by hand — the `observe-with-panoptic` topic.
package/package.json CHANGED
@@ -18,9 +18,9 @@
18
18
  "peerDependencies": {
19
19
  "@opentelemetry/api": "*",
20
20
  "@opentelemetry/sdk-trace-base": "*",
21
- "@warlock.js/ai": "5.27.0",
22
- "@warlock.js/cache": "5.27.0",
23
- "@warlock.js/logger": "5.27.0",
21
+ "@warlock.js/ai": "5.29.0",
22
+ "@warlock.js/cache": "5.29.0",
23
+ "@warlock.js/logger": "5.29.0",
24
24
  "langfuse": "*"
25
25
  },
26
26
  "peerDependenciesMeta": {
@@ -40,7 +40,7 @@
40
40
  "dependencies": {
41
41
  "langfuse": "^3.38.20"
42
42
  },
43
- "version": "5.27.0",
43
+ "version": "5.29.0",
44
44
  "main": "./cjs/index.cjs",
45
45
  "module": "./esm/index.mjs",
46
46
  "types": "./esm/index.d.mts",
package/skills/README.md CHANGED
@@ -1,25 +1,25 @@
1
1
  # `@warlock.js/ai-panoptic` — skills index
2
2
 
3
- Per-task skills. All cross-references use the form `@warlock.js/<pkg>/<skill>/SKILL.md`.
3
+ Per-task skills. Cross-references name the topic ("the `<topic>` topic"), and for another package add its skill ("the `<topic>` topic of the `warlock-js-<pkg>` skill").
4
4
 
5
5
  ## Skills
6
6
 
7
- ### [`observe-with-panoptic/`](./observe-with-panoptic/SKILL.md)
7
+ ### `observe-with-panoptic`
8
8
 
9
9
  Wire Panoptic into a run — declaratively via `ai.config({ panoptic })` (registers panoptic on core's `Observer` seam; `observeAll` + per-flow `observe` decide which workflow/supervisor/team runs route in), or imperatively with the one-call `panoptic({ exporters })` subscriber (`attach()` to an event stream, `middleware()` for the agent pipeline, or `collect()` a report directly — the orchestrator turn). Covers the side-effect import, the three imperative feed paths, the default `*.completed` event surface and `completedEvents` override, failure isolation, `flush()` / `shutdown()`, and bringing your own collector/store. Load when attaching observability to a primitive, observing every flow by default, adding a tracing middleware, or collecting an orchestrator turn report.
10
10
 
11
- ### [`export-traces/`](./export-traces/SKILL.md)
11
+ ### `export-traces`
12
12
 
13
13
  Send Panoptic traces to an observability backend via pluggable exporters — `consoleExporter` / `fileExporter` (zero-dep) and `otelExporter` (OpenTelemetry GenAI semantic conventions) / `langfuseExporter` (both lazily import their optional-peer SDK). Covers picking an exporter, the `ExporterContract`, the `gen_ai.*` / `warlock.*` attribute mapping, opt-in content capture (`captureContent` / `redactContent` / `fullHistory` to emit the whole `CapturedMessage[]`), the shared `walkSpans` / `toGenAiAttributes` / `totalCostUsd` utilities, and authoring a custom exporter. Load when wiring `collector.use(...)` with an exporter, exporting to OTel / Langfuse / console / a file, capturing prompt/response content onto spans, or writing a custom sink.
14
14
 
15
- ### [`query-traces/`](./query-traces/SKILL.md)
15
+ ### `query-traces`
16
16
 
17
17
  Retain traces in a queryable store — in-memory (`createInMemoryTraceStore`) or **cache-backed and restart-durable** (`createCacheTraceStore` over any `@warlock.js/cache` `CacheDriver` / async factory, with a synchronous in-memory read mirror, write-through persistence, `ready()` hydration, `prefix` / `capacity` / `onError`) — and slice them after the fact: `get` / `query` / `aggregate` / `clear` / `size`, the `TraceQuery` filter (id / session / status / time window), the `TraceAggregate` usage + cost + status rollup, `capacity` FIFO eviction, and the pure `sumUsage` / `emptyUsage` / `matchTrace` helpers. Load when querying collected traces, aggregating AI cost/usage per session, listing failed runs, retaining traces in memory, or persisting them across a restart.
18
18
 
19
- ### [`use-local-dashboard/`](./use-local-dashboard/SKILL.md)
19
+ ### `use-local-dashboard`
20
20
 
21
21
  Run the zero-setup local Panoptic dashboard — a loopback `node:http` server over a trace store served from `ai.config({ panoptic: { dashboard } })`. Covers the redesigned UI (light/dark/system theme, two-pane nested call tree + metadata panel, colour-coded Title-case type labels including the first-class `Team` type, started/ended wall-clock timestamps, arrow-coded `↓in · ↑out · total` tokens, per-node rollup cost, hide-zero rows, Warlock logo), the new dashboard features (client-side search/filter, group-by-session + group-by-prompt + group-by-type, per-type stats panel, cost heatmap, Gantt timeline, errors-only toggle, hash deep-links) and their pure `trace-filter.ts` helpers (`filterTraces` / `groupBySession` / `groupByPrompt` / `rollupCost` / `maxNodeCost` / `heatIntensity` / `tracePromptKey`), the persistent `cache` store wiring, the declarative path (which store it reads, idempotent wiring), the low-level `dashboard(store, options)` building block, `DashboardOptions` (port `4319`, loopback host, `basePath`, `open`, `title`), the `DashboardHandle`, the read-only `/api/traces` / `/api/aggregate` / `/api/traces/:id` JSON API, `createRequestHandler` / `dashboardHtml` for embedding, prompt-version linkage, and the loopback-only security posture. Load when opening a local AI trace viewer in the browser, theming/filtering it, inspecting traces without a backend, persisting them across a restart, or spinning up a dev observability server.
22
22
 
23
- ### [`evaluate-system-prompt/`](./evaluate-system-prompt/SKILL.md)
23
+ ### `evaluate-system-prompt`
24
24
 
25
25
  Grade a trace's LAST captured system prompt from the drawer — the dashboard's only write-capable feature, config-gated via `DashboardOptions.evaluate` (`{ model, instructions }`). Covers `evaluateSystemPrompt` / `extractLastSystemPrompt` / `findSpanById`, the "Evaluate system prompt" button + editable-instructions panel, the `POST .../spans/:spanId/evaluate` route and its status codes, and how it reuses `@warlock.js/ai`'s `judgePromptBody` (the same machinery `ai.prompts().validate()` runs) rather than a second judging implementation. Load when grading a live/captured system prompt from the dashboard, wiring `evaluate` into `ai.config`, or embedding the evaluate building blocks elsewhere.
@@ -1,11 +1,11 @@
1
1
  ---
2
2
  name: evaluate-system-prompt
3
- description: 'Grade a single trace''s LAST captured system prompt from the dashboard drawer — the dashboard''s only write-capable feature, config-gated via `DashboardOptions.evaluate` (`ai.config({ panoptic: { dashboard: { evaluate: { model, instructions } } } })`). Triggers: `evaluate`, `EvaluateConfig`, `EvaluateVerdict`, `evaluateSystemPrompt`, `extractLastSystemPrompt`, `findSpanById`, "Evaluate system prompt" button, `POST .../spans/:spanId/evaluate`, grading a prompt from a real trace, judging a captured system prompt, per-run instructions override, judge model factory; ''evaluate this trace''s prompt'', ''grade the system prompt from the dashboard'', ''judge a captured run'', ''score a live system prompt''. Skip: grading raw prompt TEXT you already have in code (no trace involved) — `@warlock.js/ai`''s `ai.prompts().validate({ criteria })`, the same judge machinery this wraps; a DATASET of test cases run through an agent — `@warlock.js/ai`''s `ai.dataset` + `agent.eval`; viewing/filtering/grouping traces without grading them — `@warlock.js/ai-panoptic/use-local-dashboard/SKILL.md`; capturing the content this feature reads — `@warlock.js/ai-panoptic/export-traces/SKILL.md` (`captureContent`).'
3
+ description: 'Grade a single trace''s LAST captured system prompt from the dashboard drawer — the dashboard''s only write-capable feature, config-gated via `DashboardOptions.evaluate` (`ai.config({ panoptic: { dashboard: { evaluate: { model, instructions } } } })`). Triggers: `evaluate`, `EvaluateConfig`, `EvaluateVerdict`, `evaluateSystemPrompt`, `extractLastSystemPrompt`, `findSpanById`, "Evaluate system prompt" button, `POST .../spans/:spanId/evaluate`, grading a prompt from a real trace, judging a captured system prompt, per-run instructions override, judge model factory; ''evaluate this trace''s prompt'', ''grade the system prompt from the dashboard'', ''judge a captured run'', ''score a live system prompt''. Skip: grading raw prompt TEXT you already have in code (no trace involved) — `@warlock.js/ai`''s `ai.prompts().validate({ criteria })`, the same judge machinery this wraps; a DATASET of test cases run through an agent — `@warlock.js/ai`''s `ai.dataset` + `agent.eval`; viewing/filtering/grouping traces without grading them — the `use-local-dashboard` topic; capturing the content this feature reads — the `export-traces` topic (`captureContent`).'
4
4
  ---
5
5
 
6
6
  # Evaluate a trace's system prompt from the dashboard
7
7
 
8
- A drawer button that grades the LAST `{role: "system"}` message captured on the selected span — not the whole conversation, not the tool code, just the system prompt that was actually in effect for that run. It's the dashboard's only write-capable feature: every other route is `GET`-only and read-only by design (see `use-local-dashboard/SKILL.md`'s security note); this one POSTs, and it's off unless you explicitly opt in.
8
+ A drawer button that grades the LAST `{role: "system"}` message captured on the selected span — not the whole conversation, not the tool code, just the system prompt that was actually in effect for that run. It's the dashboard's only write-capable feature: every other route is `GET`-only and read-only by design (see the `use-local-dashboard` topic's security note); this one POSTs, and it's off unless you explicitly opt in.
9
9
 
10
10
  ## Enable it
11
11
 
@@ -81,6 +81,6 @@ curl -X POST 'http://127.0.0.1:4319/api/traces/tr_1/spans/sp_1/evaluate' \
81
81
 
82
82
  ## See also
83
83
 
84
- - [`@warlock.js/ai-panoptic/use-local-dashboard/SKILL.md`](@warlock.js/ai-panoptic/use-local-dashboard/SKILL.md) — the dashboard this feature is a drawer action of; its security note now says "every route is GET-only and read-only EXCEPT the one opt-in write route `evaluate` enables."
85
- - [`@warlock.js/ai-panoptic/export-traces/SKILL.md`](@warlock.js/ai-panoptic/export-traces/SKILL.md) — `captureContent` (and `fullHistory`), the prerequisite for `span.input` to carry anything gradable.
84
+ - The `use-local-dashboard` topic — the dashboard this feature is a drawer action of; its security note now says "every route is GET-only and read-only EXCEPT the one opt-in write route `evaluate` enables."
85
+ - The `export-traces` topic — `captureContent` (and `fullHistory`), the prerequisite for `span.input` to carry anything gradable.
86
86
  - `ai.prompts().validate({ criteria })` in `@warlock.js/ai` — the sibling capability this reuses: grading raw prompt TEXT you already have in code, no trace or dashboard involved.
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  name: export-traces
3
- description: 'Send @warlock.js/ai-panoptic traces to an observability backend via pluggable exporters, including capturing prompt/response content onto spans. Triggers: `consoleExporter`, `fileExporter`, `otelExporter`, `langfuseExporter`, `ExporterContract`, `ContentCaptureOptions`, `captureContent`, `redactContent`, `fullHistory`, `ContentRedactor`, `toGenAiAttributes`, `walkSpans`, `totalCostUsd`, `GEN_AI_ATTRIBUTES`, `WARLOCK_ATTRIBUTES`, `reportToTrace`, `reportToSpan`, `extractSpanAttributes`, `normalizeError`, `formatSpanLine`, `createPanopticMiddleware`; ''export AI traces to OpenTelemetry'', ''send traces to Langfuse'', ''log traces to console / a file'', ''capture the full prompt/conversation onto a span'', ''gen_ai semantic conventions'', ''write a custom exporter''; typical import `import { otelExporter, langfuseExporter, consoleExporter, fileExporter } from "@warlock.js/ai-panoptic"`. Skip: wiring the subscriber/collector into a run — `@warlock.js/ai-panoptic/observe-with-panoptic/SKILL.md`; the core report shape — `@warlock.js/ai/run-ai-agent/SKILL.md`.'
3
+ description: 'Send @warlock.js/ai-panoptic traces to an observability backend via pluggable exporters, including capturing prompt/response content onto spans. Triggers: `consoleExporter`, `fileExporter`, `otelExporter`, `langfuseExporter`, `ExporterContract`, `ContentCaptureOptions`, `captureContent`, `redactContent`, `fullHistory`, `ContentRedactor`, `toGenAiAttributes`, `walkSpans`, `totalCostUsd`, `GEN_AI_ATTRIBUTES`, `WARLOCK_ATTRIBUTES`, `reportToTrace`, `reportToSpan`, `extractSpanAttributes`, `normalizeError`, `formatSpanLine`, `createPanopticMiddleware`; ''export AI traces to OpenTelemetry'', ''send traces to Langfuse'', ''log traces to console / a file'', ''capture the full prompt/conversation onto a span'', ''gen_ai semantic conventions'', ''write a custom exporter''; typical import `import { otelExporter, langfuseExporter, consoleExporter, fileExporter } from "@warlock.js/ai-panoptic"`. Skip: wiring the subscriber/collector into a run — the `observe-with-panoptic` topic; the core report shape — the `run-ai-agent` topic of the `warlock-js-ai` skill.'
4
4
  ---
5
5
 
6
6
  # Export traces — the sink end of Panoptic
@@ -179,6 +179,6 @@ Reach for `reportToTrace` when you already hold a finished `result.report` and w
179
179
 
180
180
  ## See also
181
181
 
182
- - [`@warlock.js/ai-panoptic/observe-with-panoptic/SKILL.md`](@warlock.js/ai-panoptic/observe-with-panoptic/SKILL.md) — the source end: wiring the `panoptic(...)` subscriber into a run (`attach` / `middleware` / `collect`). The `collector` these exporters register on comes from there.
183
- - [`@warlock.js/ai-panoptic/query-traces/SKILL.md`](@warlock.js/ai-panoptic/query-traces/SKILL.md) — the in-memory trace store: a sibling sink that *retains* traces (instead of shipping them out) so you can `query` / `aggregate` them after the fact.
184
- - [`@warlock.js/ai/run-ai-agent/SKILL.md`](@warlock.js/ai/run-ai-agent/SKILL.md) — running the agent and the `BaseReport` shape Panoptic projects into a `Trace`.
182
+ - The `observe-with-panoptic` topic — the source end: wiring the `panoptic(...)` subscriber into a run (`attach` / `middleware` / `collect`). The `collector` these exporters register on comes from there.
183
+ - The `query-traces` topic — the in-memory trace store: a sibling sink that *retains* traces (instead of shipping them out) so you can `query` / `aggregate` them after the fact.
184
+ - The `run-ai-agent` topic of the `warlock-js-ai` skill — running the agent and the `BaseReport` shape Panoptic projects into a `Trace`.
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  name: observe-with-panoptic
3
- description: 'Wire @warlock.js/ai-panoptic into an agent/workflow/supervisor/orchestrator — declaratively via `ai.config({ panoptic })` (registers panoptic on core''s Observer seam) or with the one-call `panoptic({ exporters })` subscriber. Triggers: `panoptic`, `Panoptic`, `PanopticOptions`, `PanopticConfig`, `ai.config({ panoptic })`, `observeAll`, per-flow `observe`, `FlowObserveOption`, `registerObserver`, `Observer`, `.attach`, `.middleware`, `.collect`, `.toTrace`, `observe.attach(agent)`, `panoptic().middleware()`, `completedEvents`, `middlewareName`; ''observe a warlock agent'', ''trace an agent run'', ''attach observability to a workflow/supervisor'', ''observe every flow by default'', ''add a tracing middleware'', ''collect an orchestrator turn report'', ''wire OTel/Langfuse into my agent''; typical import `import { ai } from "@warlock.js/ai"` + `import "@warlock.js/ai-panoptic"` (or `import { panoptic } from "@warlock.js/ai-panoptic"`). Skip: the zero-setup local dashboard — `@warlock.js/ai-panoptic/use-local-dashboard/SKILL.md`; writing/choosing an exporter (the sink end) — `@warlock.js/ai-panoptic/export-traces/SKILL.md`; the core report shape / running the agent itself — `@warlock.js/ai/run-ai-agent/SKILL.md`.'
3
+ description: 'Wire @warlock.js/ai-panoptic into an agent/workflow/supervisor/orchestrator — declaratively via `ai.config({ panoptic })` (registers panoptic on core''s Observer seam) or with the one-call `panoptic({ exporters })` subscriber. Triggers: `panoptic`, `Panoptic`, `PanopticOptions`, `PanopticConfig`, `ai.config({ panoptic })`, `observeAll`, per-flow `observe`, `FlowObserveOption`, `registerObserver`, `Observer`, `.attach`, `.middleware`, `.collect`, `.toTrace`, `observe.attach(agent)`, `panoptic().middleware()`, `completedEvents`, `middlewareName`; ''observe a warlock agent'', ''trace an agent run'', ''attach observability to a workflow/supervisor'', ''observe every flow by default'', ''add a tracing middleware'', ''collect an orchestrator turn report'', ''wire OTel/Langfuse into my agent''; typical import `import { ai } from "@warlock.js/ai"` + `import "@warlock.js/ai-panoptic"` (or `import { panoptic } from "@warlock.js/ai-panoptic"`). Skip: the zero-setup local dashboard — the `use-local-dashboard` topic; writing/choosing an exporter (the sink end) — the `export-traces` topic; the core report shape / running the agent itself — the `run-ai-agent` topic of the `warlock-js-ai` skill.'
4
4
  ---
5
5
 
6
6
  # Observe a run with Panoptic — the one-call subscriber
@@ -165,11 +165,11 @@ const observe = panoptic({ collector, exporters: [consoleExporter()] });
165
165
 
166
166
  Add exporters after construction with `observe.use(exporter)` (chainable, deduped by `ExporterContract.name`). `observe.toTrace(report)` projects a report into a `Trace` **without** dispatching — handy for inspection or tests.
167
167
 
168
- > The store's full query / aggregate / capacity surface is its own skill — see [`query-traces/SKILL.md`](@warlock.js/ai-panoptic/query-traces/SKILL.md).
168
+ > The store's full query / aggregate / capacity surface is its own skill — see the `query-traces` topic.
169
169
 
170
170
  ## See also
171
171
 
172
- - [`@warlock.js/ai-panoptic/use-local-dashboard/SKILL.md`](@warlock.js/ai-panoptic/use-local-dashboard/SKILL.md) — the zero-setup local viewer the `ai.config({ panoptic: { dashboard } })` switch starts over the collector's store.
173
- - [`@warlock.js/ai-panoptic/export-traces/SKILL.md`](@warlock.js/ai-panoptic/export-traces/SKILL.md) — the sink end: choosing/writing an exporter, the `gen_ai.*` / `warlock.*` attribute mapping.
174
- - [`@warlock.js/ai-panoptic/query-traces/SKILL.md`](@warlock.js/ai-panoptic/query-traces/SKILL.md) — retaining traces in the queryable in-memory store and slicing them (`query` / `aggregate` / `capacity`).
175
- - [`@warlock.js/ai/run-ai-agent/SKILL.md`](@warlock.js/ai/run-ai-agent/SKILL.md) — running the agent and the `BaseReport` shape Panoptic consumes.
172
+ - The `use-local-dashboard` topic — the zero-setup local viewer the `ai.config({ panoptic: { dashboard } })` switch starts over the collector's store.
173
+ - The `export-traces` topic — the sink end: choosing/writing an exporter, the `gen_ai.*` / `warlock.*` attribute mapping.
174
+ - The `query-traces` topic — retaining traces in the queryable in-memory store and slicing them (`query` / `aggregate` / `capacity`).
175
+ - The `run-ai-agent` topic of the `warlock-js-ai` skill — running the agent and the `BaseReport` shape Panoptic consumes.
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  name: query-traces
3
- description: 'Retain @warlock.js/ai-panoptic traces in a queryable store — in-memory, or cache-backed so they survive a process restart — and slice them after the fact: get one run by id, list a session, filter failed runs in a window, roll up usage + cost. Triggers: `createInMemoryTraceStore`, `createCacheTraceStore`, `CacheTraceStoreHandle`, `CacheTraceStoreOptions`, `CacheDriverInput`, `store.ready`, `TraceStoreContract`, `TraceQuery`, `TraceAggregate`, `InMemoryTraceStoreOptions`, `store.query`, `store.aggregate`, `store.get`, `store.add`, `store.clear`, `store.size`, `capacity`, `prefix`, `onError`, `sumUsage`, `emptyUsage`, `matchTrace`; ''query collected traces'', ''aggregate AI cost per session'', ''how much did this session spend'', ''list failed runs'', ''retain traces in memory'', ''persist traces across a restart'', ''cache-backed trace store'', ''Redis trace store'', ''roll up token usage''; typical import `import { createInMemoryTraceStore, createCacheTraceStore } from "@warlock.js/ai-panoptic"`. Skip: sending traces to an external backend (OTel/Langfuse/console/file) — `@warlock.js/ai-panoptic/export-traces/SKILL.md`; wiring the subscriber into a run — `@warlock.js/ai-panoptic/observe-with-panoptic/SKILL.md`.'
3
+ description: 'Retain @warlock.js/ai-panoptic traces in a queryable store — in-memory, or cache-backed so they survive a process restart — and slice them after the fact: get one run by id, list a session, filter failed runs in a window, roll up usage + cost. Triggers: `createInMemoryTraceStore`, `createCacheTraceStore`, `CacheTraceStoreHandle`, `CacheTraceStoreOptions`, `CacheDriverInput`, `store.ready`, `TraceStoreContract`, `TraceQuery`, `TraceAggregate`, `InMemoryTraceStoreOptions`, `store.query`, `store.aggregate`, `store.get`, `store.add`, `store.clear`, `store.size`, `capacity`, `prefix`, `onError`, `sumUsage`, `emptyUsage`, `matchTrace`; ''query collected traces'', ''aggregate AI cost per session'', ''how much did this session spend'', ''list failed runs'', ''retain traces in memory'', ''persist traces across a restart'', ''cache-backed trace store'', ''Redis trace store'', ''roll up token usage''; typical import `import { createInMemoryTraceStore, createCacheTraceStore } from "@warlock.js/ai-panoptic"`. Skip: sending traces to an external backend (OTel/Langfuse/console/file) — the `export-traces` topic; wiring the subscriber into a run — the `observe-with-panoptic` topic.'
4
4
  ---
5
5
 
6
6
  # Query traces — the in-memory queryable store
@@ -27,7 +27,7 @@ const failed = store.query({ status: "failed" });
27
27
  const spend = store.aggregate({ sessionId: "session-42" });
28
28
  ```
29
29
 
30
- > The store is also a valid `collector.use(...)` target directly — `panoptic({ collector: createCollector().use(store) })` — see [`observe-with-panoptic/SKILL.md`](@warlock.js/ai-panoptic/observe-with-panoptic/SKILL.md) for bring-your-own-collector wiring.
30
+ > The store is also a valid `collector.use(...)` target directly — `panoptic({ collector: createCollector().use(store) })` — see the `observe-with-panoptic` topic for bring-your-own-collector wiring.
31
31
 
32
32
  ## The store contract
33
33
 
@@ -128,7 +128,7 @@ const store = createInMemoryTraceStore({ capacity: 1000 });
128
128
 
129
129
  Overwriting an existing `traceId` refreshes its insertion position, so a re-collected run is treated as newest for eviction — eviction stays honest under overwrite.
130
130
 
131
- > The in-memory store has no secondary indexes: `get` / `add` are O(1), `query` / `aggregate` are O(n) scans. That is the right trade-off for dev/test and modest-volume runtime use. For high-volume or durable retention, write a custom `TraceStoreContract` (or export to a backend with [`export-traces`](@warlock.js/ai-panoptic/export-traces/SKILL.md)) backed by a real datastore.
131
+ > The in-memory store has no secondary indexes: `get` / `add` are O(1), `query` / `aggregate` are O(n) scans. That is the right trade-off for dev/test and modest-volume runtime use. For high-volume or durable retention, write a custom `TraceStoreContract` (or export to a backend with the `export-traces` topic) backed by a real datastore.
132
132
 
133
133
  ## Persisting across a restart — `createCacheTraceStore`
134
134
 
@@ -179,7 +179,7 @@ type CacheTraceStoreHandle = TraceStoreContract & ExporterContract & {
179
179
 
180
180
  Ordering uses an **internal monotonic counter**, not `Date.now()`, so the persisted index stays a stable total order even when many traces land in the same millisecond — FIFO eviction stays honest across a restart.
181
181
 
182
- > The usual path is declarative: `ai.config({ panoptic: { cache, dashboard: true } })` constructs this store, awaits `ready()`, and wires it under the dashboard for you — see [`use-local-dashboard/SKILL.md`](@warlock.js/ai-panoptic/use-local-dashboard/SKILL.md). Reach for `createCacheTraceStore` directly only when you manage the store / collector yourself.
182
+ > The usual path is declarative: `ai.config({ panoptic: { cache, dashboard: true } })` constructs this store, awaits `ready()`, and wires it under the dashboard for you — see the `use-local-dashboard` topic. Reach for `createCacheTraceStore` directly only when you manage the store / collector yourself.
183
183
 
184
184
  ## Summing usage yourself
185
185
 
@@ -198,6 +198,6 @@ for (const trace of store.query({ sessionId })) {
198
198
 
199
199
  ## See also
200
200
 
201
- - [`@warlock.js/ai-panoptic/observe-with-panoptic/SKILL.md`](@warlock.js/ai-panoptic/observe-with-panoptic/SKILL.md) — wiring the `panoptic(...)` subscriber and bring-your-own-collector so the store fills as runs complete.
202
- - [`@warlock.js/ai-panoptic/export-traces/SKILL.md`](@warlock.js/ai-panoptic/export-traces/SKILL.md) — sending traces to an external backend instead of (or alongside) retaining them in memory; writing a custom sink.
203
- - [`@warlock.js/ai/run-ai-agent/SKILL.md`](@warlock.js/ai/run-ai-agent/SKILL.md) — the `BaseReport` the collector projects into the `Trace` this store retains.
201
+ - The `observe-with-panoptic` topic — wiring the `panoptic(...)` subscriber and bring-your-own-collector so the store fills as runs complete.
202
+ - The `export-traces` topic — sending traces to an external backend instead of (or alongside) retaining them in memory; writing a custom sink.
203
+ - The `run-ai-agent` topic of the `warlock-js-ai` skill — the `BaseReport` the collector projects into the `Trace` this store retains.
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  name: use-local-dashboard
3
- description: 'Run the zero-setup local Panoptic dashboard — a loopback `node:http` server over a trace store served from `ai.config({ panoptic: { dashboard } })`, with a two-pane call-tree + metadata UI (light/dark/system theme, search/filter, session + prompt-version grouping, cost heatmap, Gantt timeline, errors filter, hash deep-links) and an optional cache-backed persistent store. Triggers: `dashboard`, `DashboardOptions`, `DashboardHandle`, `ai.config({ panoptic })`, `panoptic.dashboard`, `panoptic.cache`, `port: 4319`, `basePath`, `open`, `authToken`, `allowedHosts`, `/api/traces`, `/api/aggregate`, `/api/traces/:id`, `createRequestHandler`, `dashboardHtml`, `sanitizeHref`, `ServeConfig`, `TraceStoreContract`, `createCacheTraceStore`, theme, dark mode, call tree, metadata panel, cost heatmap, timeline / Gantt, errors only, deep-link, group by session, prompt version, group by type, type stats panel, started / ended timestamps, team type; ''open a local AI trace dashboard'', ''view agent traces in the browser'', ''self-hosted panoptic UI'', ''inspect traces without Langfuse'', ''spin up a dev observability server'', ''dark mode dashboard'', ''persist traces across restart'', ''group traces by session / prompt version / type'', ''cost heatmap'', ''timeline / waterfall view'', ''filter failed runs in the dashboard'', ''per-type stats panel'', ''is the dashboard bearer token safe'', ''markdown links XSS in trace content''; typical import `import { ai } from "@warlock.js/ai"` + `import "@warlock.js/ai-panoptic"`. Skip: shipping traces to a backend (OTel / Langfuse / file) — `@warlock.js/ai-panoptic/export-traces/SKILL.md`; the queryable store API the dashboard reads — `@warlock.js/ai-panoptic/query-traces/SKILL.md`; wiring the subscriber by hand — `@warlock.js/ai-panoptic/observe-with-panoptic/SKILL.md`.'
3
+ description: 'Run the zero-setup local Panoptic dashboard — a loopback `node:http` server over a trace store served from `ai.config({ panoptic: { dashboard } })`, with a two-pane call-tree + metadata UI (light/dark/system theme, search/filter, session + prompt-version grouping, cost heatmap, Gantt timeline, errors filter, hash deep-links) and an optional cache-backed persistent store. Triggers: `dashboard`, `DashboardOptions`, `DashboardHandle`, `ai.config({ panoptic })`, `panoptic.dashboard`, `panoptic.cache`, `port: 4319`, `basePath`, `open`, `authToken`, `allowedHosts`, `/api/traces`, `/api/aggregate`, `/api/traces/:id`, `createRequestHandler`, `dashboardHtml`, `sanitizeHref`, `ServeConfig`, `TraceStoreContract`, `createCacheTraceStore`, theme, dark mode, call tree, metadata panel, cost heatmap, timeline / Gantt, errors only, deep-link, group by session, prompt version, group by type, type stats panel, started / ended timestamps, team type; ''open a local AI trace dashboard'', ''view agent traces in the browser'', ''self-hosted panoptic UI'', ''inspect traces without Langfuse'', ''spin up a dev observability server'', ''dark mode dashboard'', ''persist traces across restart'', ''group traces by session / prompt version / type'', ''cost heatmap'', ''timeline / waterfall view'', ''filter failed runs in the dashboard'', ''per-type stats panel'', ''is the dashboard bearer token safe'', ''markdown links XSS in trace content''; typical import `import { ai } from "@warlock.js/ai"` + `import "@warlock.js/ai-panoptic"`. Skip: shipping traces to a backend (OTel / Langfuse / file) — the `export-traces` topic; the queryable store API the dashboard reads — the `query-traces` topic; wiring the subscriber by hand — the `observe-with-panoptic` topic.'
4
4
  ---
5
5
 
6
6
  # The local Panoptic dashboard — zero-setup trace viewing
@@ -119,7 +119,7 @@ ai.config({
119
119
  });
120
120
  ```
121
121
 
122
- Panoptic awaits the store's `ready()` hydration **before** the server starts serving, so a restart surfaces previously-persisted traces immediately. Reads stay synchronous (served from an in-memory mirror); writes go through to the cache fire-and-forget, so a flaky cache never throws into the collector's hot path. A store-shaped exporter you pass via `exporters` still wins over `cache`. The store building block (`createCacheTraceStore`) and its durability contract live in [`query-traces/SKILL.md`](@warlock.js/ai-panoptic/query-traces/SKILL.md).
122
+ Panoptic awaits the store's `ready()` hydration **before** the server starts serving, so a restart surfaces previously-persisted traces immediately. Reads stay synchronous (served from an in-memory mirror); writes go through to the cache fire-and-forget, so a flaky cache never throws into the collector's hot path. A store-shaped exporter you pass via `exporters` still wins over `cache`. The store building block (`createCacheTraceStore`) and its durability contract live in the `query-traces` topic.
123
123
 
124
124
  ## The low-level building block — `dashboard(store, options)`
125
125
 
@@ -166,7 +166,7 @@ type DashboardHandle = {
166
166
 
167
167
  ## The JSON API
168
168
 
169
- Every route is **`GET`-only and read-only** (non-`GET` → `405`), mounted under `basePath` — EXCEPT the one opt-in `POST .../spans/:spanId/evaluate` route `DashboardOptions.evaluate` enables (see [`evaluate-system-prompt/SKILL.md`](@warlock.js/ai-panoptic/evaluate-system-prompt/SKILL.md)); absent that config, it 405s like everything else. The page polls the `GET` routes; you can curl them too. Store shapes are already JSON-safe, so responses are a plain `JSON.stringify`.
169
+ Every route is **`GET`-only and read-only** (non-`GET` → `405`), mounted under `basePath` — EXCEPT the one opt-in `POST .../spans/:spanId/evaluate` route `DashboardOptions.evaluate` enables (see the `evaluate-system-prompt` topic); absent that config, it 405s like everything else. The page polls the `GET` routes; you can curl them too. Store shapes are already JSON-safe, so responses are a plain `JSON.stringify`.
170
170
 
171
171
  | Route | Returns |
172
172
  |---|---|
@@ -183,7 +183,7 @@ curl 'http://127.0.0.1:4319/api/traces?status=failed'
183
183
  curl 'http://127.0.0.1:4319/api/aggregate?sessionId=sess_42'
184
184
  ```
185
185
 
186
- (The exact filter keys and the aggregate shape live in [`query-traces/SKILL.md`](@warlock.js/ai-panoptic/query-traces/SKILL.md).)
186
+ (The exact filter keys and the aggregate shape live in the `query-traces` topic.)
187
187
 
188
188
  ## Advanced — build the handler yourself
189
189
 
@@ -215,7 +215,7 @@ These three findings (link sanitization + the two token-handling items above) ca
215
215
 
216
216
  ## See also
217
217
 
218
- - [`@warlock.js/ai-panoptic/observe-with-panoptic/SKILL.md`](@warlock.js/ai-panoptic/observe-with-panoptic/SKILL.md) — the `ai.config({ panoptic })` wiring and the underlying `panoptic(...)` subscriber that fills the store.
219
- - [`@warlock.js/ai-panoptic/query-traces/SKILL.md`](@warlock.js/ai-panoptic/query-traces/SKILL.md) — the `TraceStoreContract` (`query` / `get` / `aggregate`) the dashboard reads, the `TraceQuery` filter, and capacity eviction.
220
- - [`@warlock.js/ai-panoptic/export-traces/SKILL.md`](@warlock.js/ai-panoptic/export-traces/SKILL.md) — shipping traces to OTel / Langfuse / a file instead of (or alongside) the local dashboard.
221
- - [`@warlock.js/ai-panoptic/evaluate-system-prompt/SKILL.md`](@warlock.js/ai-panoptic/evaluate-system-prompt/SKILL.md) — the drawer's "Evaluate system prompt" action, the dashboard's one write route.
218
+ - The `observe-with-panoptic` topic — the `ai.config({ panoptic })` wiring and the underlying `panoptic(...)` subscriber that fills the store.
219
+ - The `query-traces` topic — the `TraceStoreContract` (`query` / `get` / `aggregate`) the dashboard reads, the `TraceQuery` filter, and capacity eviction.
220
+ - The `export-traces` topic — shipping traces to OTel / Langfuse / a file instead of (or alongside) the local dashboard.
221
+ - The `evaluate-system-prompt` topic — the drawer's "Evaluate system prompt" action, the dashboard's one write route.