@senad-d/observme 0.1.0

package/.env.example

@@ -0,0 +1,57 @@
+# ObservMe extension environment example.
+#
+# Option A: export these variables in your shell before starting Pi.
+# Option B: copy this file to .env in the project root and fill values there.
+# System environment variables override values from .env. Never commit .env.
+
+# Core extension settings
+OBSERVME_ENABLED=true
+OBSERVME_ENVIRONMENT=development
+OBSERVME_TENANT=local-dev
+
+# Local OpenTelemetry Collector endpoint used by the bundled observability-stack.
+OBSERVME_OTLP_ENDPOINT=http://localhost:4318
+OBSERVME_OTLP_PROTOCOL=http/protobuf
+
+# Optional OTLP bearer token for remote/secured collectors. Leave blank for local dev.
+OBSERVME_OTLP_TOKEN=
+
+# Grafana query settings for /obs health, /obs cost, /obs tools, /obs logs, /obs agents, and /obs trace.
+OBSERVME_GRAFANA_URL=https://localhost:3000
+
+# Choose one Grafana auth mode.
+# Preferred: create a Grafana service-account token with Viewer access and set it here.
+# OBSERVME_GRAFANA_TOKEN=glsa_your_service_account_token
+
+# Local fallback: use Basic auth. For the bundled stack, copy the password from
+# observability-stack/secrets/grafana_admin_password.
+OBSERVME_GRAFANA_USERNAME=admin
+OBSERVME_GRAFANA_PASSWORD=
+
+# Bundled stack datasource UIDs. Change only if your Grafana datasource UIDs differ.
+OBSERVME_GRAFANA_TEMPO_DATASOURCE_UID=tempo
+OBSERVME_GRAFANA_LOKI_DATASOURCE_UID=loki
+OBSERVME_GRAFANA_PROMETHEUS_DATASOURCE_UID=prometheus
+
+# Required for the bundled local HTTPS endpoint with a self-signed certificate.
+# Keep false for production CAs.
+OBSERVME_GRAFANA_TLS_INSECURE_SKIP_VERIFY=true
+
+# Avoid observability.local resolving to an unreachable IPv6 loopback first.
+OBSERVME_GRAFANA_PREFER_IPV4=true
+
+# Optional capture/debug toggles. Keep disabled unless you intentionally need local debugging.
+OBSERVME_CAPTURE_PROMPTS=false
+OBSERVME_CAPTURE_RESPONSES=false
+OBSERVME_CAPTURE_THINKING=false
+OBSERVME_CAPTURE_TOOL_ARGUMENTS=false
+OBSERVME_CAPTURE_TOOL_RESULTS=false
+OBSERVME_CAPTURE_BASH_COMMANDS=false
+OBSERVME_CAPTURE_BASH_OUTPUT=false
+OBSERVME_CAPTURE_FILE_PATHS=false
+OBSERVME_REDACTION_ENABLED=true
+# Required when redaction is enabled and any capture flag above is true.
+# Generate a project-local value with: openssl rand -hex 32
+OBSERVME_HASH_SALT=
+OBSERVME_ALLOW_UNSAFE_CAPTURE=false
+OBSERVME_ALLOW_INSECURE_TRANSPORT=true

package/CHANGELOG.md

@@ -0,0 +1,52 @@
+# Changelog
+
+## Unreleased
+
+### Added
+
+- Expanded the Grafana suite with SLO Health, Trace Journey, Agent Node Graphs, and LLM Conversations dashboards, plus richer multi-agent, cost, latency, tool, model, log, and export-health views.
+- Added correlated, content-safe telemetry for tool results, agent runs, workflows, subagent lifecycle events, interactive Bash executions, and configuration failures.
+- Added W3C trace continuation for launcher-propagated parent contexts, with sanitized lineage validation and fail-safe span-link/log fallbacks.
+- Added deterministic Pi runtime, Collector, Grafana-stack, dashboard, lifecycle, packaging, and configuration validation coverage.
+- Added SonarQube-compatible LCOV output and focused tests for partial Pi runtimes, lifecycle races, fallback paths, privacy controls, and telemetry contracts.
+
+### Changed
+
+- Split Pi event handling into focused lifecycle, agent/turn, LLM, tool/Bash, and session modules while preserving the `registerHandlers()` facade.
+- Centralized telemetry conventions, content-capture policy, sensitive-value rejection, diagnostic sanitization, Grafana transport, `/obs` command plumbing, and trusted-project configuration bootstrap.
+- Improved all dashboards with emitted-label-safe queries, selected-range calculations, zero/no-data states, bounded tables, canonical SLO formulas, and time-preserving Loki/Tempo drill-downs.
+- Updated the Cost dashboard to aggregate spend across providers and models, preserve token totals from short-lived sessions, and display sub-cent values accurately.
+- Updated README, dashboard, integration, and operator documentation to match the current command surface, package contents, privacy defaults, and validation workflows.
+- Made strict TypeScript checks, offline coverage, deterministic fixtures, and Docker-backed integration validation more reliable in CI.
+
+### Fixed
+
+- Fixed session, workflow, agent, turn, LLM, tool, Bash, histogram, active-span, failure/recovery, and session-count telemetry accuracy.
+- Fixed lifecycle serialization, duplicate session replacement, bounded shutdown/flush behavior, backfill cancellation, parallel tool correlation, and bounded agent-tree state cleanup.
+- Hardened configuration validation, project-root path confinement, endpoint security, custom redaction patterns, tenant-salted hashing, and environment propagation.
+- Restored trusted-project redacted LLM capture and OpenAI Responses-style prompt capture while keeping capture disabled and redaction enabled by default.
+- Remediated SonarQube maintainability and security findings across parsing, regexes, fallback handling, diagnostics, and default assignment.
+- Fixed npm/CI dependency installation, package contents, README command drift, Grafana/Loki schema handling, and dashboard provisioning/query validation.
+
+## 0.1.0 - 2026-07-07
+
+### Added
+
+- Bootstrapped the `@senad-d/observme` Pi extension with layered configuration, semantic conventions, session-scoped OpenTelemetry exporters, and bounded lifecycle management.
+- Instrumented sessions, workflows, agents, turns, LLM requests, tools, interactive Bash, subagents, model/thinking changes, compaction, and branches with traces, metrics, and structured logs.
+- Added privacy-first redaction, salted hashing, truncation, path scrubbing, secret detection, opt-in content capture, and trusted-project `.env`/`.pi/observme.yaml` support.
+- Added `/obs` commands for status, health, session details, cost, traces, links, tools, errors, logs, agents, and bounded current-session backfill.
+- Added Grafana clients for Prometheus, Loki, and Tempo with authenticated, timeout-bounded, secret-safe transport and query validation.
+- Added dashboards for overview, cost, latency, tools, models, errors, branches/compactions, agents/subagents, export health, logs/LLM I/O, conversations, trace journeys, and agent node graphs.
+- Added Prometheus alerts, SLO definitions, Collector/Grafana-stack examples, compatibility documentation, and production/operator guidance.
+- Added unit, contract, cardinality, privacy, exporter-failure, Pi RPC, Collector, Grafana-stack, chaos, lifecycle, packaging, and synthetic performance tests.
+
+### Changed
+
+- Reworked the original project specifications into session-sized tasks and reconciled production documentation with telemetry, privacy, lineage, validation, and packaging contracts.
+- Replaced template scaffolding and metadata with the ObservMe extension factory, package identity, documentation, and shipped companion assets.
+
+### Fixed
+
+- Hardened duplicate session startup, exporter timeout behavior, command parsing, query readiness, dashboard datasource UIDs, Loki labels, active-session trace guidance, and integration-test isolation.
+- Corrected latency/size histogram emission, short-window PromQL, content-capture visibility, configuration diagnostics, and local Grafana authentication/TLS handling.

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Senad Dizdarević
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/ObservMe-Production-Docs/00-README.md

@@ -0,0 +1,79 @@
+# ObservMe Production Documentation
+
+**ObservMe** is a production-grade Pi extension for observability. It captures Pi session, turn, tool, LLM, branch, compaction, and error events, translates them into OpenTelemetry telemetry, and exports them to a durable external observability stack such as OpenTelemetry Collector, Grafana Tempo, Grafana Loki, Prometheus, Mimir, or Grafana Cloud.
+
+ObservMe is intentionally **not** a local analytics database. Pi agents are ephemeral. Observability data must leave the Pi agent process quickly, safely, and reliably. When multiple Pi agents or subagents are running, ObservMe must also preserve workflow and agent-tree lineage so operators can see which parent agent spawned each child agent and how the full orchestrated workload behaved.
+
+## Core Design
+
+```text
+Pi Agent
+  └── ObservMe Pi Extension
+        ├── OTEL Traces  ──► OTEL Collector ──► Tempo
+        ├── OTEL Metrics ──► OTEL Collector ──► Prometheus / Mimir
+        └── OTEL Logs    ──► OTEL Collector ──► Loki
+
+Grafana
+  ├── Tempo datasource
+  ├── Loki datasource
+  └── Prometheus/Mimir datasource
+```
+
+## Document Set
+
+| File | Purpose |
+|---|---|
+| `01-requirements-and-scope.md` | Product goals, non-goals, personas, requirements, rollout phases |
+| `02-reference-architecture.md` | Full system architecture and component responsibilities |
+| `03-pi-event-and-session-model.md` | Pi session/event sources and how ObservMe interprets them |
+| `04-telemetry-semantic-conventions.md` | ObservMe attribute, metric, log, and span naming specification |
+| `05-otel-pipeline-and-collector.md` | OTLP exporter strategy and production Collector configurations |
+| `06-security-privacy-redaction.md` | Prompt safety, secret redaction, PII handling, tenant isolation |
+| `07-extension-implementation-blueprint.md` | TypeScript implementation architecture, modules, lifecycle, code patterns |
+| `08-query-grafana-integration.md` | `/obs` commands, Grafana/Tempo/Loki/Prometheus query integration |
+| `09-dashboards-alerts-slos.md` | Dashboard pack, PromQL, LogQL, TraceQL, alerts, SLOs |
+| `10-testing-release-operations.md` | Test strategy, chaos testing, performance validation, release process |
+| `11-deployment-runbooks.md` | Local, Docker Compose, Kubernetes, and production runbooks |
+| `12-configuration-reference.md` | Complete ObservMe configuration schema and examples |
+| `13-source-notes.md` | Official documentation cross-check notes and external assumptions |
+| `pi-session-format.md` | Pi session JSONL reference copied and adapted from Pi's official session-format documentation |
+
+## Production Principles
+
+1. **OTLP-first.** ObservMe emits OpenTelemetry traces, metrics, and logs.
+2. **No durable local telemetry storage.** Only bounded in-memory queues are allowed.
+3. **Fail open.** If observability fails, Pi must continue operating.
+4. **Privacy by default.** Prompt and response content capture is disabled by default.
+5. **Pi-native and OTEL-compatible semantics.** ObservMe uses `observme.*` and `pi.*` for Pi-specific concepts and official OpenTelemetry `gen_ai.*` attributes where they fit; it does not depend on OpenInference.
+6. **Agent lineage by design.** Agent and subagent telemetry carries `pi.agent.*` lineage attributes and, where possible, W3C trace context so parent/child relationships survive process boundaries.
+7. **Workflow/tree observability.** Orchestrator workloads are modeled as agent trees with `pi.workflow.*` correlation, depth, fan-out, wait/join, orphan, and critical-path signals so operators can understand both single-agent and multi-agent execution.
+8. **Collector recommended.** Direct-to-backend export is allowed only for development or constrained environments.
+9. **Backend-neutral.** Tempo/Loki/Prometheus are reference backends, not hard dependencies.
+
+## Minimum Viable Production Release
+
+A production-ready `1.0.0` release must include:
+
+- Extension load and health checks
+- Session, workflow, agent-run, turn, LLM, tool, subagent-spawn, agent wait/join, compaction, branch, model-change telemetry
+- Multi-agent tree metrics for active agents, depth, fan-out, orphan agents, trace-context propagation failures, child failures, and workflow duration
+- OTLP trace exporter
+- OTLP metric exporter
+- OTLP log exporter
+- Configurable redaction and capture controls
+- Bounded queues and exporter timeouts
+- Grafana dashboard JSON
+- Collector reference configs
+- Tests for event mapping, redaction, exporter failure, and cardinality
+- `/obs status`, `/obs health`, `/obs session`, `/obs cost`, `/obs trace`, `/obs agents` commands
+
+## Source References
+
+This document set assumes:
+
+- Pi extensions provide documented lifecycle hooks, commands, tools, custom entries, session events, and queryable session state.
+- Pi can run multiple extension instances or spawn additional Pi processes through tools/subagent patterns; cross-process workflow and parent/child lineage requires explicit ObservMe correlation and trace-context propagation.
+- Pi sessions are JSONL files with typed entries such as `message`, `compaction`, `branch_summary`, `model_change`, `thinking_level_change`, `custom`, `custom_message`, `label`, and `session_info`.
+- OpenTelemetry Collector receives, processes, and exports telemetry via pipelines.
+- OpenTelemetry GenAI semantic conventions are the official OTEL namespace for GenAI client/agent spans, attributes, and metrics, but many GenAI fields are still experimental/evolving in SDK packages; ObservMe extends them for Pi-specific workflow concepts.
+- Grafana Tempo, Loki, and Prometheus/Mimir are reference storage systems for traces, logs, and metrics.

package/ObservMe-Production-Docs/01-requirements-and-scope.md

@@ -0,0 +1,207 @@
+# ObservMe Requirements and Scope
+
+## 1. Product Definition
+
+ObservMe is a Pi extension that instruments Pi agents and sends observability data to an external telemetry platform using OpenTelemetry.
+
+ObservMe must answer questions such as:
+
+- Which Pi sessions are running now?
+- Which sessions are expensive?
+- Which models are used most often?
+- Which tools fail or run slowly?
+- Which prompts or tool calls caused high latency?
+- How often do agents branch or compact context?
+- Which projects, users, or environments generate the most cost?
+- What happened in an ephemeral agent after it disappeared?
+- Which parent agent spawned a subagent, and what did the child agent do?
+- How deep and wide did an orchestrated multi-agent workflow become?
+- Which child or grandchild agent was on the critical path?
+- Did any child agent become orphaned, fail to report lineage, or continue after the parent finished?
+- Did one orchestrator prompt amplify cost, latency, or tool usage through excessive fan-out?
+
+## 2. Personas
+
+### Platform Engineering
+
+Needs centralized observability, retention, SLOs, alerts, and tenant isolation.
+
+### AI Platform Owner
+
+Needs cost, model usage, prompt efficiency, failure analysis, and quality signals.
+
+### Developer Using Pi
+
+Needs quick session diagnostics from inside Pi via `/obs` commands.
+
+### Security / Compliance
+
+Needs prompt capture off by default, redaction controls, audit trails, and safe handling of code and secrets.
+
+## 3. Hard Requirements
+
+### 3.1 External Durability
+
+ObservMe must not depend on local Pi storage for telemetry durability. Pi sessions may be ephemeral or deleted. Telemetry must be exported to a central durable backend as close to real time as possible.
+
+### 3.2 No Durable Local Telemetry Database
+
+ObservMe must not create SQLite, local JSONL telemetry archives, parquet files, or long-lived caches. Allowed local state:
+
+- In-memory span registry
+- In-memory metric instruments
+- Bounded retry queue inside OTEL SDK/exporter
+- Optional one-line session/agent correlation custom entry if explicitly enabled
+
+### 3.3 OTEL Native
+
+ObservMe must use OpenTelemetry APIs and OTLP exporters for traces, metrics, and logs. Where OpenTelemetry GenAI semantic conventions apply, ObservMe should emit the official `gen_ai.*` attributes in addition to Pi-specific `pi.*` attributes.
+
+### 3.4 Collector First
+
+The default topology is:
+
+```text
+ObservMe -> OTEL Collector -> Backends
+```
+
+Direct export to Tempo, Loki, Prometheus, or Grafana Cloud is allowed but not the default production recommendation.
+
+### 3.5 Fail Open
+
+ObservMe must never block Pi agent execution because the collector or backend is down.
+
+Required behavior:
+
+- Export failures are logged locally only as minimal warnings through Pi UI when debug is enabled.
+- Telemetry is dropped when queues are full.
+- Tool calls, model calls, and sessions must continue.
+
+### 3.6 Privacy by Default
+
+Default settings:
+
+```yaml
+capture:
+  prompts: false
+  responses: false
+  thinking: false
+  toolArguments: false
+  toolResults: false
+  bashCommands: false
+  bashOutput: false
+  filePaths: false
+```
+
+Metadata such as token counts, duration, status, model, provider, tool name, agent role, subagent depth, and error class is captured by default. High-cardinality identifiers such as `pi.workflow.id`, `pi.workflow.root_agent_id`, `pi.agent.id`, `pi.agent.parent_id`, `pi.agent.run.id`, `pi.session.id`, trace IDs, and tool-call IDs are allowed on spans/logs but must not be promoted to metric labels by default.
+
+### 3.7 Multi-Agent Workflow and Subagent Lineage
+
+Adding a single `agent_id` is necessary but not sufficient for multi-agent observability. ObservMe must model the workflow and lineage explicitly:
+
+```text
+pi.workflow.id          # generated workflow/root execution id; high cardinality, traces/logs only
+pi.workflow.root_agent_id
+pi.agent.id             # current logical Pi agent runtime
+pi.agent.parent_id      # parent agent when this is a subagent
+pi.agent.root_id        # root agent in the tree
+pi.agent.depth          # 0 for root, 1+ for subagents
+pi.agent.spawn.id       # individual spawn operation id when known
+pi.agent.spawn.tool_call_id
+```
+
+Preferred behavior:
+
+1. Generate `pi.agent.id` at session runtime startup unless supplied by trusted parent context.
+2. When a tool/extension spawns a subagent, pass W3C `traceparent`/`tracestate` plus ObservMe lineage environment variables to the child process.
+3. If trace context is propagated, make the subagent session span a descendant of the parent spawn/tool span.
+4. If trace context cannot be propagated, emit the child as a separate trace with span links and the `pi.agent.parent_id`/`pi.agent.root_id` attributes.
+5. Track agent-tree depth, fan-out, active children, orphaned children, trace-context propagation failures, child failures, and parent wait/join latency.
+6. Keep workflow IDs and agent IDs out of metric labels; use traces and logs for per-workflow/per-agent drill-down, and low-cardinality labels such as `agent_role`, `agent_capability`, `subagent_depth`, `spawn_type`, or `spawn_reason` for aggregate metrics.
+
+## 4. Non-Goals
+
+ObservMe is not:
+
+- A local trace database
+- A replacement for Grafana
+- A replacement for OTEL Collector
+- An OpenInference implementation
+- A replacement for official OpenTelemetry GenAI semantic conventions
+- A vendor-specific AI observability SDK
+- A prompt evaluation framework
+- A policy enforcement extension
+- A session replay system by default
+
+Session replay can be enabled only with explicit content capture settings.
+
+## 5. Production Capabilities
+
+### Traces
+
+- Session root span
+- Agent-run spans
+- Turn spans
+- LLM request spans
+- Tool call spans
+- Subagent-spawn spans
+- Agent wait/join spans for multi-agent critical-path analysis
+- Bash execution spans
+- Compaction spans
+- Branch spans
+- Extension error spans
+
+### Metrics
+
+- Token counters
+- Cost counters
+- Turn counters
+- Tool call counters
+- Error counters
+- Latency histograms
+- Queue/drop counters
+- Export health metrics
+- Agent-run and subagent-spawn counters
+- Workflow counters and duration histograms
+- Active-agent gauges
+- Agent-tree depth, width, and fan-out histograms
+- Orphan-agent and trace-context propagation failure counters
+- Parent wait/join duration histograms
+- Child-agent failure and recovery counters
+
+### Logs
+
+- Structured session events
+- Structured tool events
+- Structured model events
+- Structured workflow, agent-run, subagent-spawn, wait/join, orphan-agent, and trace-context propagation events
+- Redacted error logs
+- Optional prompt and response logs
+
+### Query Commands
+
+- `/obs status`
+- `/obs health`
+- `/obs session`
+- `/obs cost`
+- `/obs trace`
+- `/obs tools`
+- `/obs errors`
+- `/obs logs`
+- `/obs link`
+- `/obs agents`
+
+## 6. Success Criteria
+
+ObservMe is production ready when:
+
+1. Telemetry from an ephemeral Pi agent survives after the agent is deleted.
+2. Grafana shows session traces in Tempo.
+3. Loki contains structured event logs for sessions and tools.
+4. Prometheus/Mimir contains token, cost, latency, and failure metrics.
+5. The extension survives backend outages without breaking Pi.
+6. Redaction tests prevent secrets from leaving the agent.
+7. Cardinality limits prevent observability stack overload.
+8. Users can open a Grafana trace link from Pi.
+9. Parent agents and spawned subagents can be correlated without adding high-cardinality workflow IDs or agent IDs to metrics.
+10. Orchestrator workflows expose depth, fan-out, active-agent, orphan, cost-amplification, and critical-path signals without adding workflow IDs or agent IDs to metric labels.