@handsforbots/semantic-event-observability 0.0.0-stage → 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (55) hide show
  1. package/LICENSE.md +9 -0
  2. package/README.md +141 -2
  3. package/adapters/genericEventBus.js +20 -0
  4. package/adapters/instrumentChannel.js +22 -0
  5. package/core/CorrelationContext.js +49 -0
  6. package/core/EventBuffer.js +37 -0
  7. package/core/MetricsRegistry.js +200 -0
  8. package/core/PhaseTracker.js +143 -0
  9. package/core/Policy.js +87 -0
  10. package/core/SessionTracker.js +23 -0
  11. package/core/TraceContextBridge.js +74 -0
  12. package/core/TraceMapper.js +167 -0
  13. package/core/TurnMetricsCollector.js +19 -0
  14. package/core/TurnModel.js +87 -0
  15. package/core/baggage.d.ts +12 -0
  16. package/core/baggage.js +68 -0
  17. package/core/bucketEventName.js +11 -0
  18. package/core/createObservability.js +611 -0
  19. package/core/definePhaseModel.js +14 -0
  20. package/core/eventInstrumentation.js +21 -0
  21. package/core/isErrorEvent.js +9 -0
  22. package/core/telemetryAttributes.js +44 -0
  23. package/core/turnSpanNaming.js +73 -0
  24. package/docs/architecture.md +106 -0
  25. package/docs/custom-exporter.md +135 -0
  26. package/docs/exporters.md +180 -0
  27. package/docs/getting-started.md +192 -0
  28. package/docs/grafana-validation.md +71 -0
  29. package/docs/handsforbots-adapter.md +64 -0
  30. package/docs/handsforbots-roadmap.md +193 -0
  31. package/docs/metrics-roadmap.md +145 -0
  32. package/docs/publishing.md +55 -0
  33. package/docs/renaming.md +65 -0
  34. package/docs/roadmap.md +467 -0
  35. package/docs/tail-sampling.md +99 -0
  36. package/exporters/console.js +29 -0
  37. package/exporters/devPanel.js +204 -0
  38. package/exporters/faro.js +71 -0
  39. package/exporters/index.js +58 -0
  40. package/exporters/langfuse.js +161 -0
  41. package/exporters/langsmith.js +168 -0
  42. package/exporters/llmExporterState.js +84 -0
  43. package/exporters/memory.js +18 -0
  44. package/exporters/otel.js +236 -0
  45. package/exporters/otelTraceBackend.js +51 -0
  46. package/exporters/webVitals.js +73 -0
  47. package/grafana/semantic-event-observability.json +250 -0
  48. package/grafana/semantic-event-observability.lgtm.json +399 -0
  49. package/index.d.ts +312 -0
  50. package/index.js +31 -0
  51. package/package.json +80 -3
  52. package/packageIdentity.js +41 -0
  53. package/utils/id.js +10 -0
  54. package/utils/probeOptional.js +42 -0
  55. package/utils/safeJson.js +74 -0
@@ -0,0 +1,71 @@
1
+ # Grafana dashboard validation
2
+
3
+ Checklist and automated smoke test for bundled dashboards targeting **`sevo_*`** metrics.
4
+
5
+ ## Dashboard files
6
+
7
+ | File | Use |
8
+ |------|-----|
9
+ | `grafana/semantic-event-observability.json` | Import template (select Loki/Tempo/Prometheus) |
10
+ | `grafana/semantic-event-observability.lgtm.json` | Pre-provisioned in `examples/` LGTM stack |
11
+
12
+ ## Automated smoke test
13
+
14
+ ```bash
15
+ cd packages/semantic-event-observability
16
+ npm test -- test/grafana-dashboard.test.js
17
+ ```
18
+
19
+ The test asserts each dashboard JSON includes PromQL for core P0/P1 metrics:
20
+
21
+ - `sevo_turn_duration_ms`
22
+ - `sevo_phase_duration_ms`
23
+ - `sevo_phase_wait_ms`
24
+ - `sevo_events_dropped_total`
25
+ - `sevo_state_gauge`
26
+ - `sevo_turns_total`
27
+ - `sevo_bus_events_total`
28
+
29
+ ## Manual LGTM validation
30
+
31
+ 1. Start the example stack with observability enabled (`examples/OBSERVABILITY.md`).
32
+ 2. Open Grafana → **Semantic Event Observability** dashboard.
33
+ 3. Send a few chat turns in the Vite example.
34
+ 4. Confirm within ~30s (metric export interval):
35
+
36
+ | Panel | Expected |
37
+ |-------|----------|
38
+ | Turn duration p95 | Non-empty after turns complete |
39
+ | Phase duration p95 | `backend` (or configured phase) after backend round-trip |
40
+ | Phase wait p95 | Non-zero after turn → backend delay |
41
+ | Turn outcomes rate | `completed` series |
42
+ | Backend queue depth | Reflects orchestrator queue |
43
+ | Bus event rate | `core.input`, `core.output_ready`, etc. |
44
+ | Dropped telemetry | Zero or low with full sample rate |
45
+ | Tempo / Recent turns | Turn traces with phase children |
46
+
47
+ 5. Drilldown → Metrics: search `sevo_` — all P0 instruments should appear.
48
+
49
+ ## Tempo trace checks
50
+
51
+ - Service: `semantic-event-observability`
52
+ - Root span: `turn:core.input` (or `invoke_agent` when `turnRootSpan` enabled)
53
+ - Child spans: `phase:backend`, `event:*`
54
+
55
+ ## When panels stay empty
56
+
57
+ | Symptom | Likely cause |
58
+ |---------|----------------|
59
+ | All Prometheus panels empty | OTel metrics export not wired; check `getMeter` in exporter config |
60
+ | Traces only, no metrics | MeterProvider not registered globally |
61
+ | Histogram panels empty but `sevo_*_count` exists in Explore | PromQL metric name mismatch — OTel must not set `unit: 'ms'` on instruments already named `*_ms` (otherwise Prometheus gets `*_milliseconds_bucket` while dashboards query `*_bucket`) |
62
+ | Phase / status / queue panels empty | Label mismatch — metric labels are `phase`, `status`, `key` (not `sevo_phase`, etc.) |
63
+ | Bus events empty | Bus not instrumented or no traffic |
64
+ | Phase wait empty | No `phases` configured or phases start immediately on turn start |
65
+ | p95 shows NaN briefly | Need ongoing chat traffic in the selected time range; histogram `rate()` needs counter movement across scrapes |
66
+
67
+ ## Related
68
+
69
+ - [metrics-roadmap.md](./metrics-roadmap.md)
70
+ - [tail-sampling.md](./tail-sampling.md)
71
+ - [exporters.md](./exporters.md)
@@ -0,0 +1,64 @@
1
+ # Hands for Bots adapter
2
+
3
+ Hands for Bots v2 integrates through the plugin [`@handsforbots/observability`](../../observability/src/index.ts). The v1 adapter (`adapters/handsforbots.js`, which instrumented the v1 `eventEmitter`) was removed together with v1.
4
+
5
+ ## Scope
6
+
7
+ The plugin instruments the **semantic flow** of the v2 kernel. It does **not** monitor whether the application or its backends are up.
8
+
9
+ | Covered | Not covered (use another stack) |
10
+ |---------|--------------------------------|
11
+ | Turn latency (`turn.started` → `turn.done` / `turn.error` / `turn.aborted`) | Uptime / synthetic HTTP probes |
12
+ | Phase per route: `transport`, `direct` (menu), `capture`, `agent` (WebMCP), `push` | Backend `/health` endpoints |
13
+ | Actions by origin (`action.invoked` / `action.failed`, origin `user` / `assistant` / `agent`) | Kubernetes liveness/readiness |
14
+ | Signals by modality (`signal.text`, `signal.transcript`, `signal.image`…), without content by default | "Site down" alerting |
15
+ | Stimuli (`stimulus.ui.effect`, `stimulus.action.call`, `stimulus.error`…), token deltas excluded | |
16
+ | Perceived latency: `h4b_first_response_ms` (input → first visible response) and `h4b_turn_duration_ms`, labelled by route | |
17
+ | Telemetry export health (`sevo_exporter_errors_total`) | |
18
+
19
+ ## Usage
20
+
21
+ ```ts
22
+ import { createH4B } from '@handsforbots/core'
23
+ import { observability } from '@handsforbots/observability'
24
+
25
+ const h4b = createH4B({
26
+ plugins: [
27
+ observability({
28
+ environment: 'production',
29
+ sampleRate: 0.2,
30
+ maxEventsPerMinute: 120,
31
+ exporters: ['memory', 'faro', 'otel', 'langfuse', 'langsmith'],
32
+ exporterConfig: {
33
+ langfuse: { project: 'handsforbots' },
34
+ langsmith: { projectName: 'handsforbots' },
35
+ },
36
+ // includeContent: true, // only if your privacy policy allows logging message text
37
+ }),
38
+ ],
39
+ })
40
+ ```
41
+
42
+ All `createObservability` options are accepted. The instance is available as a service: `h4b.get('observability')`.
43
+
44
+ ### Trace propagation to the backend
45
+
46
+ Pass trace headers to HTTP-based transports so a turn is one trace from the browser to the LLM:
47
+
48
+ ```ts
49
+ agui({ url: '/api/agent', headers: () => h4b.get('observability')?.getTraceHeaders() ?? {} })
50
+ ```
51
+
52
+ ### Experiments
53
+
54
+ Conversation attributes (`h4b.setConversationAttributes`, ADR 0017) go on every span and, through `getTraceHeaders()`, to the backend as W3C `baggage`. Exposures (`h4b.expose`) are recorded as `experiment.exposed`, also the ones from before the plugin mounted (`late: true`). Two counters, never sampled, are recorded: `h4b_experiment_exposed_total` and `h4b_conversation_engaged_total`, the conversation's first visitor turn. On the server, `readConversationAttributes` from `@handsforbots/observability/server` reads the header. See [Experiments](../../../docs/en-us/experiments.md).
55
+
56
+ ## Turn and phase model
57
+
58
+ | Kernel event | Semantic event |
59
+ |--------------|----------------|
60
+ | first `turn.status` of a turn | `turn.started` |
61
+ | `turn.status` `acting` with route R | `route.R.start` |
62
+ | `turn.status` `done` / `error` / `aborted` | `route.R.end`, then `turn.done` / `turn.error` / `turn.aborted` |
63
+
64
+ Exports: `H4B_TURN_START_EVENTS`, `H4B_TURN_END_EVENTS`, `H4B_PHASES` from `@handsforbots/observability`.
@@ -0,0 +1,193 @@
1
+ # Hands for Bots roadmap (adapter consumer)
2
+
3
+ > **Note (v2):** this roadmap was written for the v1 integration (`core.input`, `core.calling_backend`…). In v2 the integration is the [`@handsforbots/observability`](../../observability/src/index.ts) plugin; see [handsforbots-adapter.md](./handsforbots-adapter.md) for the current event model. Items that still apply (traceparent, gen_ai spans, feedback per turn, Langfuse) carry over with the v2 names.
4
+
5
+ Metrics, phases, and integration work specific to **Hands for Bots** as a consumer of [Semantic Event Observability](./roadmap.md).
6
+
7
+ > **Library metrics (`sevo_*`):** [metrics-roadmap.md](./metrics-roadmap.md)
8
+ > **Integration guide:** [handsforbots-adapter.md](./handsforbots-adapter.md)
9
+
10
+ ---
11
+
12
+ ## Relationship to the library
13
+
14
+ Hands for Bots does **not** extend the lib core. It provides:
15
+
16
+ 1. **Adapter preset** — `adapters/handsforbots.js`
17
+ 2. **Observability output plugin** — `Plugins/Output/Observability/`
18
+ 3. **Domain metrics** — `hfb_*` via `recordMetric()` and future plugin instrumentation
19
+ 4. **Phase preset** — `HFB_PHASE_MODEL` in adapter (passed to `createObservability`)
20
+ 5. **Backend correlation** — trace propagation from orchestrator / backend cores
21
+
22
+ The adapter and Observability plugin do **not** provide uptime monitoring, health endpoints, or platform probes. Those stay in deploy/infra and backend stacks — see [library scope](./architecture.md#scope).
23
+
24
+ ```mermaid
25
+ flowchart LR
26
+ HFB["Hands for Bots"]
27
+ Adapter["handsforbots.js adapter"]
28
+ SEO["Semantic Event Observability"]
29
+ Plugin["Observability plugin"]
30
+
31
+ HFB --> Plugin --> Adapter --> SEO
32
+ HFB -->|"hfb_* metrics"| SEO
33
+ HFB -->|"gen_ai.* traces"| Tempo["Tempo / backend"]
34
+ SEO -->|"sevo_* + traces"| LGTM["Grafana LGTM"]
35
+ ```
36
+
37
+ ---
38
+
39
+ ## HfB turn & phase preset
40
+
41
+ Configured in `adapters/handsforbots.js` and passed to `createObservability`:
42
+
43
+ ```javascript
44
+ export const HFB_TURN_START_EVENTS = ['core.input']
45
+ export const HFB_TURN_END_EVENTS = ['core.output_ready']
46
+
47
+ export const HFB_PHASE_MODEL = definePhaseModel([
48
+ {
49
+ id: 'backend',
50
+ startEvent: 'core.calling_backend',
51
+ endEvent: 'core.backend_responded',
52
+ },
53
+ // render: wall-clock after backend_responded until turn.end (lib computes render_ms)
54
+ ])
55
+
56
+ export const HFB_SEMANTIC_EVENTS = [ /* allowlist for high-signal events */ ]
57
+ ```
58
+
59
+ **State provider (HfB-specific):**
60
+
61
+ ```javascript
62
+ {
63
+ queueDepth: bot.orchestrator.queue.length,
64
+ callingBackend: bot.orchestrator.calling_backend,
65
+ redirectInput: bot.redirectInput,
66
+ }
67
+ ```
68
+
69
+ ---
70
+
71
+ ## Metric naming (`hfb_*`)
72
+
73
+ | Prefix | Owner |
74
+ |---------|-------|
75
+ | `hfb_` | Hands for Bots cores, plugins, adapter |
76
+ | `sevo_` | Library only — never defined in HfB code |
77
+
78
+ **Recommended labels:** `plugin`, `backend`, `modality` (`text` \| `voice`), `environment`.
79
+
80
+ ---
81
+
82
+ ## Development checklist
83
+
84
+ Status: `[ ]` not started · `[~]` partial · `[x]` done
85
+
86
+ ### P0 — Adapter alignment with lib abstractions
87
+
88
+ | # | Item | Status |
89
+ |---|------|--------|
90
+ | H0.1 | Move phase preset from otel exporter to adapter `HFB_PHASE_MODEL` | [x] |
91
+ | H0.2 | Wire adapter to `PhaseModel` when lib Phase B ships | [x] |
92
+ | H0.3 | Enable Observability plugin in example stack with `sevo_*` validation | [~] |
93
+ | H0.4 | Document adapter preset in [handsforbots-adapter.md](./handsforbots-adapter.md) | [~] |
94
+
95
+ ---
96
+
97
+ ### P1 — Orchestration & modality metrics
98
+
99
+ | # | Metric | Source event | Status |
100
+ |---|--------|--------------|--------|
101
+ | H1.1 | `hfb_input_total{plugin}` | `core.input` | [ ] |
102
+ | H1.2 | `hfb_output_ready_total` | `core.output_ready` | [ ] |
103
+ | H1.3 | `hfb_backend_requests_total{backend,status}` | `core.calling_backend` + errors | [ ] |
104
+ | H1.4 | Turn labels `input_plugin` on `sevo_*` via adapter metadata | `core.input` payload | [x] |
105
+ | H1.5 | `sevo_queue_wait_ms` data — emit via phase or custom span | input → calling_backend | [ ] |
106
+ | H1.6 | Backend error → `error.type` on phase span | orchestrator / backend | [ ] |
107
+
108
+ ---
109
+
110
+ ### P2 — UI lifecycle & session
111
+
112
+ | # | Metric | Source event | Status |
113
+ |---|--------|--------------|--------|
114
+ | H2.1 | `hfb_bot_load_duration_ms` | `core.loaded` | [ ] |
115
+ | H2.2 | `hfb_ui_load_duration_ms{plugin}` | `core.ui_loaded` | [ ] |
116
+ | H2.3 | `hfb_all_ui_loaded_total` | `core.all_ui_loaded` | [ ] |
117
+ | H2.4 | `hfb_session_renewals_total` | `core.renew_session` | [ ] |
118
+ | H2.5 | `hfb_history_cleared_total` | `core.history_cleared` | [ ] |
119
+ | H2.6 | `hfb_bc_messages_total` | BroadcastChannel (adapter) | [ ] |
120
+ | H2.7 | `hfb_redirect_input_total` | `redirectInput` state | [ ] |
121
+ | H2.8 | `hfb_action_success_total{plugin}` | `core.action_success` | [ ] |
122
+ | H2.9 | `hfb_mcp_tool_feedback_total` | `mcp.tool_feedback_received` | [ ] |
123
+ | H2.10 | Plugin metrics cookbook in docs | `recordMetric()` examples | [ ] |
124
+
125
+ ---
126
+
127
+ ### P2 — Voice (HfB Voice core)
128
+
129
+ | # | Metric | Source | Status |
130
+ |---|--------|--------|--------|
131
+ | H2.V1 | `hfb_voice_session_duration_s` | Voice input/output | [ ] |
132
+ | H2.V2 | `hfb_voice_stt_duration_ms` | SpeechRecognition / Vosk | [ ] |
133
+ | H2.V3 | `hfb_voice_endpoint_delay_ms` | silence → send | [ ] |
134
+ | H2.V4 | `hfb_voice_tts_ttfb_ms` | Voice output | [ ] |
135
+ | H2.V5 | `hfb_voice_interruptions_total` | barge-in during TTS | [ ] |
136
+
137
+ Industry targets (voice): mouth-to-ear P95 < 1.1–1.4 s; STT < 500 ms; TTS TTFB < 250 ms ([Twilio guide](https://www.twilio.com/en-us/blog/developers/best-practices/guide-core-latency-ai-voice-agents)).
138
+
139
+ ---
140
+
141
+ ### P3 — Backend integration (HfB cores)
142
+
143
+ Backend work stays in **BotOrchestrator** and backend cores. The lib provides trace propagation only.
144
+
145
+ | # | Item | Owner | Status |
146
+ |---|------|-------|--------|
147
+ | H3.1 | Inject `traceparent` on `sendToBackend` fetch | BotOrchestrator | [ ] |
148
+ | H3.2 | `gen_ai.client.operation.duration` | OpenAI / UniversalLLM core | [ ] |
149
+ | H3.3 | `gen_ai.client.token.usage` | OpenAI / UniversalLLM core | [ ] |
150
+ | H3.4 | MCP / tool execution child spans | MCPHelper | [ ] |
151
+ | H3.5 | Rasa backend span or `backend=rasa` label | Rasa core | [ ] |
152
+ | H3.6 | End-to-end trace demo in `examples/OBSERVABILITY.md` | Examples | [ ] |
153
+ | H3.7 | Backend observability guide | `docs/en-us/` or `docs-dev/` | [ ] |
154
+
155
+ | Question | `sevo_*` (lib) | `hfb_*` (HfB) | `gen_ai.*` (backend) |
156
+ |----------|---------------|---------------|----------------------|
157
+ | User wait time | `sevo_turn_duration_ms` | — | — |
158
+ | Backend wall time | `sevo_phase_duration_ms{phase="backend"}` | — | `gen_ai.client.operation.duration` |
159
+ | Render / speak time | `sevo_phase_duration_ms` or render attr | `hfb_voice_tts_ttfb_ms` | — |
160
+ | Input modality | label on seo | `hfb_input_total` | — |
161
+ | Token cost | — | — | `gen_ai.client.token.usage` |
162
+
163
+ ---
164
+
165
+ ### P4 — Quality & product (HfB + Langfuse)
166
+
167
+ | # | Item | Status |
168
+ |---|------|--------|
169
+ | H4.1 | User feedback plugin linked to `turnId` | [ ] |
170
+ | H4.2 | Langfuse project config in Observability plugin | [ ] |
171
+ | H4.3 | Online eval sampling on production turns | [ ] |
172
+ | H4.4 | Unify or deprecate Analytics plugin vs observability pipeline | [ ] |
173
+
174
+ ---
175
+
176
+ ## Implementation pattern for HfB metrics
177
+
178
+ Prefer thin instrumentation in cores/plugins; emit via the observability instance:
179
+
180
+ ```javascript
181
+ // In a plugin
182
+ this.bot.observability?.recordMetric('hfb_gallery_open_total', 1, { plugin: 'ImageGallery' })
183
+ ```
184
+
185
+ For automatic bus-driven metrics, a future **HfB metrics helper** (in the adapter, not lib core) may subscribe to `HFB_SEMANTIC_EVENTS` and increment counters — keeps plugins declarative.
186
+
187
+ ---
188
+
189
+ ## Changelog
190
+
191
+ | Date | Change |
192
+ |------|--------|
193
+ | 2026-06-15 | H0.1–H0.2 done; `HFB_PHASE_MODEL` in adapter |
@@ -0,0 +1,145 @@
1
+ # Metrics roadmap (library)
2
+
3
+ Development checklist for **`sevo_*` metrics only** — emitted by Semantic Event Observability.
4
+
5
+ > **Not in this document:** Hands for Bots product metrics (`hfb_*`) → [handsforbots-roadmap.md](./handsforbots-roadmap.md).
6
+ > **Abstractions plan:** [roadmap.md](./roadmap.md).
7
+
8
+ ---
9
+
10
+ ## Out of scope
11
+
12
+ These concerns are intentionally **not** `sevo_*` metrics:
13
+
14
+ | Concern | Owner | Examples |
15
+ |---------|-------|----------|
16
+ | Service uptime | Infra / synthetics | HTTP probes, Grafana Cloud checks, “site down” alerts |
17
+ | Backend health | Backend deploy | Rasa `/api/health`, LLM server health |
18
+ | Platform probes | Kubernetes / Docker | `livenessProbe`, `readinessProbe` on app containers |
19
+
20
+ `sevo_exporter_errors_total` measures **telemetry export failures**, not application availability. See [architecture.md](./architecture.md#scope).
21
+
22
+ ---
23
+
24
+ ## Naming rules
25
+
26
+ | Rule | Example |
27
+ |------|---------|
28
+ | Prefix | `sevo_` |
29
+ | Unit suffix | `_ms`, `_total`, `_bytes` |
30
+ | Histograms | Turn duration, phase duration |
31
+ | Counters | Turns, drops, events emitted |
32
+ | Gauges | State keys from `stateProvider` |
33
+
34
+ **Standard labels (low cardinality):**
35
+
36
+ - `environment`
37
+ - `phase` — phase id from PhaseModel
38
+ - `status` — `completed` \| `abandoned` \| `error`
39
+ - `reason` — policy drop reason
40
+ - `event_type` — `bus.trigger` \| `turn.end` \| …
41
+
42
+ Never label with `sessionId`, `turnId`, or raw payload text.
43
+
44
+ ---
45
+
46
+ ## Industry mapping (front-end)
47
+
48
+ | Industry metric | Library metric | Notes |
49
+ |-----------------|----------------|-------|
50
+ | End-to-end turn latency | `sevo_turn_duration_ms` | Client-perceived wait |
51
+ | Per-stage latency | `sevo_phase_duration_ms{phase}` | PhaseModel-driven |
52
+ | Turn success rate | `sevo_turns_total{status}` | completed vs abandoned |
53
+ | Telemetry loss | `sevo_events_dropped_total{reason}` | Policy transparency |
54
+ | Queue / backlog pressure | `sevo_state_gauge{key="queueDepth"}` | From stateProvider |
55
+ | Throughput | `rate(sevo_turns_total{status="completed"})` | Derived in Grafana |
56
+
57
+ Backend `gen_ai.*` metrics are **not** emitted by this lib. Hosts join traces via [TraceContextBridge](./roadmap.md#6-tracecontextbridge-new--backend-correlation).
58
+
59
+ ---
60
+
61
+ ## Checklist
62
+
63
+ Status: `[ ]` not started · `[~]` partial · `[x]` done
64
+
65
+ ### P0 — SLO readiness
66
+
67
+ | # | Metric | Type | Trigger | Status |
68
+ |---|--------|------|---------|--------|
69
+ | M0.1 | `sevo_turn_duration_ms` | histogram | `turn.end` → `durationMs` | [x] |
70
+ | M0.2 | `sevo_phase_duration_ms` | histogram | PhaseTracker phase end | [x] |
71
+ | M0.3 | `sevo_turns_total` | counter | `status=completed` on turn end | [x] |
72
+ | M0.4 | `sevo_turns_total` | counter | `status=abandoned` on new turn without end | [x] |
73
+ | M0.5 | `sevo_events_dropped_total` | counter | `Policy.recordDrop(reason)` | [x] |
74
+ | M0.6 | `sevo_state_gauge` | gauge | `stateProvider()` keys | [x] |
75
+ | M0.7 | OTel Metrics API wiring | — | `MetricsRegistry` → exporter | [x] |
76
+ | M0.8 | Grafana dashboard smoke test | — | `grafana/semantic-event-observability.json` | [x] |
77
+
78
+ **Implementation:** single `MetricsRegistry` module; OTel exporter reads from registry (see [roadmap.md](./roadmap.md#4-metricsregistry-new--single-source-of-truth)).
79
+
80
+ **SLO examples:**
81
+
82
+ ```promql
83
+ histogram_quantile(0.95, sum(rate(sevo_turn_duration_ms_bucket[5m])) by (le))
84
+ sum(rate(sevo_turns_total{status="abandoned"}[5m])) / sum(rate(sevo_turns_total[5m]))
85
+ sum(increase(sevo_events_dropped_total[5m])) by (reason)
86
+ sum(increase(sevo_session_turns_total[5m])) by (status, reason)
87
+ ```
88
+
89
+ ---
90
+
91
+ ### P1 — Reliability & visibility
92
+
93
+ | # | Metric | Type | Trigger | Status |
94
+ |---|--------|------|---------|--------|
95
+ | M1.1 | `sevo_phase_wait_ms` | histogram | turn start → phase start | [x] |
96
+ | M1.2 | `sevo_turns_total` | counter | `status=error` when error heuristic fires | [x] |
97
+ | M1.3 | `sevo_events_emitted_total` | counter | `type` + bucketed `name` | [x] |
98
+ | M1.4 | `sevo_exporter_errors_total` | counter | exporter catch blocks | [x] |
99
+ | M1.5 | `sevo_active_turns` | gauge | open turns (diagnostic) | [x] |
100
+ | M1.6 | Span `error.type` attribute | — | pluggable `isError(event)` | [x] |
101
+
102
+ ---
103
+
104
+ ### P2 — Session & host-agnostic depth
105
+
106
+ | # | Metric | Type | Trigger | Status |
107
+ |---|--------|------|---------|--------|
108
+ | M2.1 | `sevo_session_turns_total` | counter | session boundary (configurable) | [x] |
109
+ | M2.2 | `sevo_custom_metrics_total` | counter | `recordMetric()` with name allowlist | [x] |
110
+ | M2.3 | `sevo_bus_events_total` | counter | `bus.trigger` by bucketed name | [x] |
111
+ | M2.4 | `sevo_listener_duration_ms` | histogram | `wrapListeners: true` | [x] |
112
+ | M2.5 | `sevo_web_vital` | histogram | `webVitals` exporter | [x] |
113
+
114
+ ---
115
+
116
+ ## `recordMetric()` convention
117
+
118
+ Host apps may call `observability.recordMetric(name, value, labels)` for domain metrics.
119
+
120
+ | Rule | Detail |
121
+ |------|--------|
122
+ | Lib does not rename | Host owns prefix (`hfb_`, `myapp_`) |
123
+ | Policy | Subject to `metric.gauge` policy gate |
124
+ | Export | All exporters receive `onMetric` |
125
+ | Recommendation | Document host prefix in adapter; avoid `sevo_` in `recordMetric` |
126
+ | Allowlist counter | Set `customMetricAllowlist: ['hfb_*']` to increment `sevo_custom_metrics_total` |
127
+
128
+ ---
129
+
130
+ ## Grafana
131
+
132
+ Dashboard templates: `grafana/semantic-event-observability.json`, `grafana/semantic-event-observability.lgtm.json`.
133
+
134
+ Panels expect P0 metrics. Host-specific panels belong in host repos, not lib dashboards.
135
+
136
+ Validation: [grafana-validation.md](./grafana-validation.md) · `npm test -- test/grafana-dashboard.test.js`
137
+
138
+ ---
139
+
140
+ ## Changelog
141
+
142
+ | Date | Change |
143
+ |------|--------|
144
+ | 2026-06-15 | Initial combined doc (lib + HfB) |
145
+ | 2026-06-15 | Phase A metrics implemented (`MetricsRegistry`, `TurnMetricsCollector`, OTel Metrics API) |
@@ -0,0 +1,55 @@
1
+ # Publishing to npm
2
+
3
+ The package lives at `packages/semantic-event-observability/` and is named **`@handsforbots/semantic-event-observability`**.
4
+
5
+ ## Single source of truth
6
+
7
+ | Field | Source |
8
+ |-------|--------|
9
+ | Package name / scope | `packageIdentity.js` → `PACKAGE_NAME` |
10
+ | Version | `packageIdentity.js` → `PACKAGE_VERSION` **and** `package.json` → `version` |
11
+ | Global debug key | Derived from `PACKAGE_SLUG` via `getPackageIdentity()` |
12
+ | OTel service name | `identity.otelServiceName` (= slug) |
13
+
14
+ **Before every publish**, keep `package.json` version in sync with `PACKAGE_VERSION` in `packageIdentity.js`.
15
+
16
+ ## Pre-publish checklist
17
+
18
+ ```bash
19
+ cd packages/semantic-event-observability
20
+ npm test
21
+ npm pack --dry-run # verify files list
22
+ ```
23
+
24
+ Confirm `files` in `package.json` includes everything consumers need (`index.js`, `index.d.ts`, `core/`, `adapters/`, `exporters/`, `utils/`).
25
+
26
+ ## Publish (maintainers)
27
+
28
+ ```bash
29
+ npm login
30
+ npm publish --access public
31
+ ```
32
+
33
+ Scoped package `@handsforbots/*` requires `--access public` on first publish.
34
+
35
+ ## Consumer install
36
+
37
+ ```bash
38
+ npm i @handsforbots/semantic-event-observability
39
+ ```
40
+
41
+ Optional peers (install only what you use):
42
+
43
+ ```bash
44
+ npm i @opentelemetry/api @grafana/faro-web-sdk
45
+ ```
46
+
47
+ TypeScript consumers get types from `index.d.ts` automatically (`"types"` field in `package.json`).
48
+
49
+ ## Monorepo / local development
50
+
51
+ Hands for Bots imports the lib from the repo path. No publish required for local work. The Vite example uses the same source tree.
52
+
53
+ ## Renaming the package
54
+
55
+ If the public npm name changes, update **`PACKAGE_SLUG`** in `packageIdentity.js` only. Storage keys, globals, and dashboard UIDs derive from it. See [renaming.md](./renaming.md).
@@ -0,0 +1,65 @@
1
+ # Renaming the package
2
+
3
+ The library is published under a **working title**. To rename without breaking integrations:
4
+
5
+ ## 1. Edit `packageIdentity.js`
6
+
7
+ ```javascript
8
+ export const PACKAGE_SLUG = 'your-new-name'
9
+ export const PACKAGE_DISPLAY_NAME = 'Your New Name'
10
+ export const PACKAGE_SCOPE = '@your-org' // optional
11
+ ```
12
+
13
+ ## 2. Update `package.json`
14
+
15
+ ```json
16
+ {
17
+ "name": "@your-org/your-new-name"
18
+ }
19
+ ```
20
+
21
+ ## 3. Derived identifiers (automatic)
22
+
23
+ `getPackageIdentity()` builds these from `PACKAGE_SLUG`:
24
+
25
+ | Derived | Example |
26
+ |---------|---------|
27
+ | npm name | `@your-org/your-new-name` |
28
+ | Global debug API | `__YOUR_NEW_NAME__` (slug uppercased, `-` → `_`) |
29
+ | localStorage debug key | `your-new-name:debug` |
30
+ | localStorage panel key | `your-new-name:panel` |
31
+ | OTel `service.name` | `your-new-name` |
32
+ | Grafana dashboard UID | `your-new-name` |
33
+
34
+ **Do not hardcode these strings elsewhere.** Import from `packageIdentity.js` or `getPackageIdentity()`.
35
+
36
+ ## 4. After rename checklist
37
+
38
+ - [ ] Update README and docs references (human-readable name only)
39
+ - [ ] Re-import Grafana dashboard (UID changes with slug)
40
+ - [ ] Update collector rules / Loki labels if they filter on `service_name`
41
+ - [ ] Notify consumers to update localStorage debug key
42
+ - [ ] Publish new npm package (old name can remain as deprecated alias if needed)
43
+
44
+ ## 5. Hands for Bots import path
45
+
46
+ Hands for Bots consumes the library through its plugin `@handsforbots/observability`, which imports `createObservability` from the package name. After a rename, only that import changes:
47
+
48
+ ```javascript
49
+ import { createObservability } from '@your-org/your-new-name'
50
+ ```
51
+
52
+ ## 6. Alias strategy for npm
53
+
54
+ When renaming publicly, you can publish both packages for one major cycle:
55
+
56
+ ```json
57
+ // old package.json
58
+ { "name": "@handsforbots/semantic-event-observability", "deprecated": "Use @handsforbots/new-name" }
59
+ ```
60
+
61
+ Re-export from the old entry point:
62
+
63
+ ```javascript
64
+ export * from '@handsforbots/new-name'
65
+ ```