@latitude-data/openclaw-telemetry 0.0.4 → 0.0.5

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (7) hide show
  1. package/README.md +40 -14
  2. package/dist/plugin.d.ts +55 -65
  3. package/dist/plugin.d.ts.map +1 -1
  4. package/dist/plugin.js +728 -460
  5. package/dist/plugin.js.map +1 -1
  6. package/openclaw.plugin.json +1 -1
  7. package/package.json +1 -1
package/README.md CHANGED
@@ -46,29 +46,55 @@ Shows a plan, asks for confirmation, then runs `openclaw plugins uninstall @lati
46
46
 
47
47
  ## What gets sent
48
48
 
49
- For each agent run, the plugin emits one trace with three span kinds:
49
+ For each agent run, the plugin emits one trace shaped like the actual run:
50
50
 
51
- - **`interaction`** — the agent run. Carries `openclaw.session.key`, `openclaw.agent.id`, `openclaw.agent.name`, aggregated token usage across all LLM calls, run duration, success/error status, and the first user prompt.
52
- - **`llm_request`** — one per LLM call. Carries provider, request/response model, `gen_ai.system_instructions`, `gen_ai.input.messages` (full history + current prompt), `gen_ai.output.messages` (assistant text + tool_call parts), and full token usage (input/output/cache_read/cache_creation/total) — under both canonical `gen_ai.*` keys and legacy aliases.
53
- - **`tool_execution`** — one per tool call. Canonical `gen_ai.tool.*` attributes: `name`, `call.id`, `call.arguments`, `call.result`. Failures set `error.type`, `error.message`, and OTel status code 2.
51
+ ```
52
+ agent (root, traceId = hash(runId))
53
+ ├─ compaction (0..1, rare; budget-triggered)
54
+ ├─ model_call (1..N, one per provider API call)
55
+ ├─ tool_call: foo (between model_calls; sibling of agent)
56
+ ├─ model_call
57
+ ├─ tool_call: bar
58
+ ├─ subagent (0..N — the child's full agent tree nests under here)
59
+ │ └─ agent
60
+ │ ├─ model_call
61
+ │ └─ tool_call: ...
62
+ └─ model_call (final)
63
+ ```
64
+
65
+ Five span kinds:
66
+
67
+ - **`agent`** — root of the run. Carries `openclaw.session.key`, `openclaw.agent.id`, `openclaw.agent.name`, aggregated token usage across all generations, run duration, success/error status, the first user prompt, and the full final message list. This is where attempt-aggregate `gen_ai.*` lands.
68
+ - **`model_call`** — one per actual provider API call inside the run. Carries provider, request/response model, `openclaw.api`, `openclaw.transport`, per-call duration, outcome, error category, time-to-first-byte, request payload bytes, response stream bytes, upstream request id hash, and `gen_ai.input.messages` snapshotted at the moment that generation started. Per-call output messages and per-call token usage aren't surfaced by OpenClaw today (attempt-aggregate only); those stay on `agent`.
69
+ - **`tool_call:<name>`** — one per tool invocation. Canonical `gen_ai.tool.*` attributes: `name`, `call.id`, `call.arguments`, `call.result`. Sibling of `agent`, NOT child of `model_call` — tools run between generations, not during them.
70
+ - **`compaction`** — rare; fires when OpenClaw hits the message budget mid-run. Records before/after message counts and the compacted-out count.
71
+ - **`subagent`** — one per child run spawned by this agent. The child's entire `agent` subtree (its own `model_call`s, `tool_call`s, even further-nested `subagent`s) parents itself underneath via cross-runId trace propagation, so a spawn tree is one waterfall in one trace.
54
72
 
55
- Every span carries `openclaw.agent.id` and `openclaw.agent.name`. Multi-agent OpenClaw setups (sub-agents) naturally produce spans tagged with the invoking agent's id, letting you filter and group by agent in the Latitude UI.
73
+ Every span carries `openclaw.agent.id` and `openclaw.agent.name`. Multi-agent setups produce spans tagged with the invoking agent's id, letting you filter and group by agent in the Latitude UI.
56
74
 
57
- All spans share the run id and trace id so they group together.
75
+ All spans share the same `traceId` so they group as one trace per agent run (and one trace per spawn tree, by virtue of the subagent linkage).
76
+
77
+ ### Backend caveat: Codex / Claude-Code-style providers
78
+
79
+ OpenClaw's `model_call_started` / `model_call_ended` hooks fire from its `selection` layer, which wraps the agent's `streamFn` invocation. For "agentic" backends (Codex, Claude Code) the inner generations happen inside the backend's own loop and don't surface as separate `model_call` events. Result: a Codex-backed run shows ONE `model_call` per attempt instead of N. Anthropic and OpenAI direct don't have this issue. The fix is upstream in OpenClaw — out of scope for this plugin.
58
80
 
59
81
  ## How it works
60
82
 
61
- OpenClaw ships typed plugin hooks that fire per-LLM-call with the complete payload (`src/plugins/hook-types.ts` in the OpenClaw source). We subscribe to:
83
+ We subscribe to OpenClaw's typed plugin hooks (`src/plugins/hook-types.ts` upstream). The model is "one span per paired before/after (or start/end) event":
84
+
85
+ | Span | Start hook | End hook |
86
+ | --- | --- | --- |
87
+ | `agent` | `before_agent_start` | `agent_end` |
88
+ | `model_call` | `model_call_started` | `model_call_ended` |
89
+ | `tool_call` | `before_tool_call` | `after_tool_call` |
90
+ | `compaction` | `before_compaction` | `after_compaction` |
91
+ | `subagent` | `subagent_spawned` | `subagent_ended` |
62
92
 
63
- - `llm_input` — full system prompt, prompt text, history messages, provider, model.
64
- - `llm_output` — assistant text, last assistant message, full token usage (input/output/cacheRead/cacheWrite/total), resolved provider/model ref.
65
- - `before_tool_call` / `after_tool_call` — tool name, arguments, result, error, duration.
66
- - `agent_end` — run completion signal. This is when we build the OTLP trace and POST it.
67
- - `session_start` — currently a no-op; reserved for future session-level metadata.
93
+ Two more hooks (`llm_input`, `llm_output`) are subscribed to for **content only** — they don't open or close spans, they just enrich the `agent` span with attempt-aggregate data and seed the rolling history snapshot used by per-call `model_call.gen_ai.input.messages`.
68
94
 
69
- OpenClaw runs LLM hooks **fire-and-forget** (see [`src/plugins/hooks.ts`](https://github.com/openclaw/openclaw/blob/main/src/plugins/hooks.ts) — `runLlmInput`/`runLlmOutput` are documented as parallel, and the call site in [`src/agents/pi-embedded-runner/run/attempt.ts`](https://github.com/openclaw/openclaw/blob/main/src/agents/pi-embedded-runner/run/attempt.ts) wraps them with `void hookRunner.run*(...).catch(...)`). Our handlers can never slow down the agent loop.
95
+ The hook system runs handlers fire-and-forget (see [`src/plugins/hooks.ts`](https://github.com/openclaw/openclaw/blob/main/src/plugins/hooks.ts) upstream), so nothing we do here can slow the agent loop. The one exception is `before_tool_call`, which is a `runModifyingHook` — our handler returns `undefined` so OpenClaw dispatches the tool normally. Returning anything else (e.g. `{block: true}`) would block every tool call.
70
96
 
71
- **No runtime wrapping.** Unlike existing third-party OpenClaw observability plugins that try to monkey-patch `@mariozechner/pi-ai` (and run into jiti's CJS/ESM module isolation), we stay inside the supported plugin API. The hooks give us everything, at lower risk of breaking on OpenClaw updates.
97
+ **No runtime wrapping.** Unlike third-party OpenClaw observability plugins that try to monkey-patch `@mariozechner/pi-ai` (and run into jiti's CJS/ESM module isolation), we stay inside the supported plugin API. The hooks give us everything, at lower risk of breaking on OpenClaw updates.
72
98
 
73
99
  ## Configuration reference
74
100
 
package/dist/plugin.d.ts CHANGED
@@ -20,68 +20,56 @@ interface Logger {
20
20
  warn: (msg: string) => void;
21
21
  }
22
22
  //#endregion
23
- //#region src/types.d.ts
24
- interface OpenClawLlmUsage {
25
- input?: number;
26
- output?: number;
27
- cacheRead?: number;
28
- cacheWrite?: number;
29
- total?: number;
30
- }
31
- interface LlmCallRecord {
32
- runId: string;
33
- sessionId: string;
34
- sessionKey: string | undefined;
35
- agentId: string | undefined;
36
- provider: string;
37
- requestModel: string;
38
- responseModel: string | undefined;
39
- resolvedRef: string | undefined;
40
- systemPrompt: string | undefined;
41
- prompt: string;
42
- historyMessages: unknown[];
43
- imagesCount: number;
44
- assistantTexts: string[];
45
- lastAssistant: unknown;
46
- usage: OpenClawLlmUsage | undefined;
47
- startMs: number;
48
- endMs: number | undefined;
49
- error: string | undefined;
50
- toolCalls: ToolCallRecord[];
51
- }
52
- interface ToolCallRecord {
53
- toolCallId: string;
54
- toolName: string;
55
- params: Record<string, unknown>;
56
- result: unknown;
57
- error: string | undefined;
23
+ //#region src/span-builder.d.ts
24
+ /**
25
+ * Builds the per-trace span tree for an OpenClaw agent run from the granular
26
+ * paired hooks. Replaces the older `turn-builder.ts` model that collapsed the
27
+ * whole attempt into a single `llm_request` span — that shape was wrong on
28
+ * two counts: `llm_input` / `llm_output` fire ONCE per attempt (not per
29
+ * generation), and an attempt is a sequence of generations interleaved with
30
+ * tool executions.
31
+ *
32
+ * Span set this builder produces:
33
+ *
34
+ * agent (root)
35
+ * ├─ compaction (0..1, rare)
36
+ * ├─ model_call (1..N, one per provider API call)
37
+ * ├─ tool_call: ... (interleaved between model_calls; siblings of agent)
38
+ * ├─ subagent (0..N; child agent runs nest INSIDE these via
39
+ * │ └─ agent ... cross-runId trace propagation)
40
+ * └─ model_call (final)
41
+ *
42
+ * Tool spans are siblings of `agent`, not children of `model_call`, because
43
+ * tools run BETWEEN generations — not during them. Nesting under model_call
44
+ * would falsely imply concurrency.
45
+ *
46
+ * `llm_input` / `llm_output` are NOT span boundaries here. They're data-only
47
+ * feeds that enrich the parent `agent` span (full message history, output
48
+ * messages, aggregate token usage).
49
+ */
50
+ interface SpanRecord {
51
+ /** Stable id for the span (16 hex chars). */
52
+ spanId: string;
53
+ /** Span tree id (32 hex chars). */
54
+ traceId: string;
55
+ /** Empty string for root agent spans, parent's spanId otherwise. */
56
+ parentSpanId: string;
57
+ /** OpenClaw event noun (`agent` / `model_call` / `tool_call` / `compaction` / `subagent`). */
58
+ name: string;
58
59
  startMs: number;
59
60
  endMs: number | undefined;
60
- durationMs: number | undefined;
61
- agentId: string | undefined;
61
+ /** Free-form attribute bag — flattened to OTLP key/value at emit time. */
62
+ attrs: Record<string, AttrValue>;
63
+ /** Status — set at close from the event payload's outcome/error. */
64
+ outcome?: "ok" | "error";
65
+ errorMessage?: string | undefined;
62
66
  }
63
- interface RunRecord {
67
+ type AttrValue = string | number | boolean | unknown[] | Record<string, unknown> | undefined;
68
+ interface BuildResult {
69
+ /** Run id this batch belongs to. */
64
70
  runId: string;
65
- sessionId: string | undefined;
66
- sessionKey: string | undefined;
67
- agentId: string | undefined;
68
- workspaceDir: string | undefined;
69
- messageProvider: string | undefined;
70
- trigger: string | undefined;
71
- channelId: string | undefined;
72
- modelProviderId: string | undefined;
73
- modelId: string | undefined;
74
- startMs: number;
75
- endMs: number | undefined;
76
- success: boolean | undefined;
77
- error: string | undefined;
78
- llmCalls: LlmCallRecord[];
79
- /**
80
- * Tool calls queued by `before_tool_call` but not yet matched to an `llm_call`
81
- * because the most recent `llm_output` has already closed the call for this
82
- * runId. Retained so `after_tool_call` can still complete the record.
83
- */
84
- orphanTools: ToolCallRecord[];
71
+ /** All spans ready to be exported (agent + everything beneath it). */
72
+ spans: SpanRecord[];
85
73
  }
86
74
  //#endregion
87
75
  //#region src/plugin.d.ts
@@ -112,17 +100,19 @@ interface RegisterOptions {
112
100
  * Hook to observe the emitted run right before it's posted. Used by tests;
113
101
  * not a stable public API.
114
102
  */
115
- onEmit?: (run: RunRecord) => void;
103
+ onEmit?: (result: BuildResult) => void;
116
104
  }
117
105
  /**
118
106
  * Register the Latitude plugin against an OpenClaw plugin API. OpenClaw calls
119
- * this once at plugin activation; we wire up `llm_input`, `llm_output`, tool
120
- * and lifecycle hooks to stream traces to Latitude.
107
+ * this once at plugin activation; we wire up the granular paired hooks
108
+ * (model_call_started/_ended, before_/after_tool_call, before_/after_compaction,
109
+ * subagent_spawned/_ended, before_agent_start/agent_end) plus the
110
+ * data-only feeds (llm_input/llm_output) that enrich the agent span.
121
111
  *
122
- * Every handler is fire-and-forget on OpenClaw's side (see
123
- * `src/plugins/hooks.ts` — runLlmInput/runLlmOutput are documented as
124
- * parallel and wrapped with `.catch()` at the call site in attempt.ts), so
125
- * nothing we do here can slow the agent loop.
112
+ * Every typed hook on OpenClaw's side fires fire-and-forget for non-modifying
113
+ * hooks; before_tool_call is a `runModifyingHook` where returning anything
114
+ * other than undefined blocks the tool call. Our handler returns nothing —
115
+ * keep it that way.
126
116
  */
127
117
  declare function registerLatitudePlugin(api: OpenClawPluginApiLike, opts?: RegisterOptions): void;
128
118
  //#endregion
package/dist/plugin.d.ts.map CHANGED
@@ -1 +1 @@
1
- {"version":3,"file":"plugin.d.ts","names":[],"sources":["../src/config.ts","../src/logger.ts","../src/types.ts","../src/plugin.ts"],"mappings":";UAAiB,MAAA;EACf,MAAA;EACA,OAAA;EACA,OAAA;EACA,OAAA;EACA,KAAA;EAHA;;;;;;EAUA,uBAAA;AAAA;;;UCVe,MAAA;EACf,KAAA,GAAQ,GAAA;EACR,IAAA,GAAO,GAAA;AAAA;;;UCwDQ,gBAAA;EACf,KAAA;EACA,MAAA;EACA,SAAA;EACA,UAAA;EACA,KAAA;AAAA;AAAA,UA4De,aAAA;EACf,KAAA;EACA,SAAA;EACA,UAAA;EACA,OAAA;EACA,QAAA;EACA,YAAA;EACA,aAAA;EACA,WAAA;EACA,YAAA;EACA,MAAA;EACA,eAAA;EACA,WAAA;EACA,cAAA;EACA,aAAA;EACA,KAAA,EAAO,gBAAA;EACP,OAAA;EACA,KAAA;EACA,KAAA;EACA,SAAA,EAAW,cAAA;AAAA;AAAA,UAGI,cAAA;EACf,UAAA;EACA,QAAA;EACA,MAAA,EAAQ,MAAA;EACR,MAAA;EACA,KAAA;EACA,OAAA;EACA,KAAA;EACA,UAAA;EACA,OAAA;AAAA;AAAA,UAGe,SAAA;EACf,KAAA;EACA,SAAA;EACA,UAAA;EACA,OAAA;EACA,YAAA;EACA,eAAA;EACA,OAAA;EACA,SAAA;EACA,eAAA;EACA,OAAA;EACA,OAAA;EACA,KAAA;EACA,OAAA;EACA,KAAA;EACA,QAAA,EAAU,aAAA;ECnHmC;;;;;EDyH7C,WAAA,EAAa,cAAA;AAAA;;;;;;;;;;;;;;UCzJE,qBAAA;EACf,MAAA,GAAS,MAAA;EACT,YAAA,GAAe,MAAA;EACf,EAAA,qBACE,QAAA,EAAU,CAAA,EACV,OAAA,GAAU,KAAA,WAAgB,GAAA,uBAC1B,IAAA;IAAS,QAAA;EAAA;AAAA;AAAA,UAII,eAAA;EFjCf;EEmCA,MAAA,GAAS,MAAA;EFnCS;EEqClB,MAAA,GAAS,MAAA;;;;ADmBX;ECdE,MAAA,IAAU,GAAA,EAAK,SAAA;AAAA;;;;;;;;;AD+EjB;;iBClEwB,sBAAA,CAAuB,GAAA,EAAK,qBAAA,EAAuB,IAAA,GAAM,eAAA"}
1
+ {"version":3,"file":"plugin.d.ts","names":[],"sources":["../src/config.ts","../src/logger.ts","../src/span-builder.ts","../src/plugin.ts"],"mappings":";UAAiB,MAAA;EACf,MAAA;EACA,OAAA;EACA,OAAA;EACA,OAAA;EACA,KAAA;EAHA;;;;;;EAUA,uBAAA;AAAA;;;UCVe,MAAA;EACf,KAAA,GAAQ,GAAA;EACR,IAAA,GAAO,GAAA;AAAA;;;ADJT;;;;;;;;;;;;;;;ACEA;;;;;;;;;;;ADFA,UEsDiB,UAAA;;EAEf,MAAA;EAFyB;EAIzB,OAAA;EAQa;EANb,YAAA;EAFA;EAIA,IAAA;EACA,OAAA;EACA,KAAA;EAAA;EAEA,KAAA,EAAO,MAAA,SAAe,SAAA;EAAf;EAEP,OAAA;EACA,YAAA;AAAA;AAAA,KAGU,SAAA,2CAAoD,MAAA;AAAA,UAoD/C,WAAA;EApDL;EAsDV,KAAA;;EAEA,KAAA,EAAO,UAAA;AAAA;;;;;;;;;;;;;;UChGQ,qBAAA;EACf,MAAA,GAAS,MAAA;EACT,YAAA,GAAe,MAAA;EACf,EAAA,qBACE,QAAA,EAAU,CAAA,EACV,OAAA,GAAU,KAAA,WAAgB,GAAA,uBAC1B,IAAA;IAAS,QAAA;EAAA;AAAA;AAAA,UAII,eAAA;EFtCf;EEwCA,MAAA,GAAS,MAAA;EFxCS;EE0ClB,MAAA,GAAS,MAAA;;;;ADQX;ECHE,MAAA,IAAU,MAAA,EAAQ,WAAA;AAAA;;;;;;;;;;;;;iBAeI,sBAAA,CAAuB,GAAA,EAAK,qBAAA,EAAuB,IAAA,GAAM,eAAA"}