@dv.nghiem/flowdeck 0.4.11 → 0.5.0

package/src/skills/telemetry-steward/SKILL.md

@@ -0,0 +1,191 @@
+---
+name: telemetry-steward
+description: Lightweight append-only telemetry layer for tracking session health, agent performance, and decision quality across FlowDeck operations.
+origin: FlowDeck
+---
+
+# Telemetry Steward
+
+FlowDeck generates a steady stream of operational signals — context compactions, agent handoffs, recorded decisions, readiness scores. Telemetry Steward captures these as structured, append-only events so operators can observe patterns, diagnose drift, and validate improvement over time.
+
+## Purpose
+
+Telemetry answers three questions:
+
+1. **Context health** — Are we pruning, compacting, or checkpointing effectively? Are token savings trending up or down?
+2. **Agent performance** — Which agents are fastest? Which fail most often? Where does routing latency spike?
+3. **Decision quality** — Are decisions backed by evidence? Is confidence calibrated to actual risk?
+
+It is not a control plane. It does not trigger actions. It exists purely for observability, retrospection, and baseline-setting.
+
+## Storage
+
+All events are written to `.codebase/TELEMETRY.jsonl`.
+
+- One JSON object per line. No outer array.
+- Append only. Never rewrite the file in place.
+- Each line is self-contained and independently parseable.
+- Invalid lines are skipped during read, not repaired.
+
+## Event Schema
+
+Every event has a top-level `type` field. All other fields are type-specific.
+
+### `context_action`
+
+Recorded when the context steward prunes, compacts, or checkpoints.
+
+```json
+{
+  "type": "context_action",
+  "action": "prune",
+  "tokens_before": 12400,
+  "tokens_after": 8200,
+  "timestamp": "2026-06-11T09:23:17.000Z"
+}
+```
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `action` | string | `"prune"`, `"compact"`, or `"checkpoint"` |
+| `tokens_before` | integer | Context size in tokens before the action |
+| `tokens_after` | integer | Context size in tokens after the action |
+| `timestamp` | ISO 8601 | When the action occurred |
+
+### `agent_routing`
+
+Recorded when the orchestrator dispatches work to an agent.
+
+```json
+{
+  "type": "agent_routing",
+  "agent": "backend-coder",
+  "category": "implementation",
+  "task_type": "bugfix",
+  "duration_ms": 14520,
+  "success": true,
+  "timestamp": "2026-06-11T09:45:02.000Z"
+}
+```
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `agent` | string | Agent that executed the task |
+| `category` | string | High-level bucket: `"implementation"`, `"research"`, `"review"`, `"debug"`, `"docs"` |
+| `task_type` | string | Specific task class: `"feature"`, `"bugfix"`, `"refactor"`, `"plan"`, `"audit"` |
+| `duration_ms` | integer | Wall-clock time from dispatch to completion |
+| `success` | boolean | Did the agent report success? |
+| `timestamp` | ISO 8601 | When routing completed |
+
+### `decision_recorded`
+
+Recorded when a decision is persisted via `decision-trace`.
+
+```json
+{
+  "type": "decision_recorded",
+  "decision_id": "auth-refactor-2026-06-11",
+  "risk_level": "medium",
+  "confidence": 0.85,
+  "has_evidence": true,
+  "timestamp": "2026-06-11T10:12:44.000Z"
+}
+```
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `decision_id` | string | Identifier from `decision-trace` |
+| `risk_level` | string | `"low"`, `"medium"`, or `"high"` |
+| `confidence` | number | 0.0 to 1.0, if available |
+| `has_evidence` | boolean | Were evidence entries provided? |
+| `timestamp` | ISO 8601 | When the decision was recorded |
+
+### `readiness_check`
+
+Recorded after a deploy-check or pre-flight verification run.
+
+```json
+{
+  "type": "readiness_check",
+  "status": "pass",
+  "score": 0.94,
+  "failing_checks": ["dependency-audit"],
+  "timestamp": "2026-06-11T11:00:00.000Z"
+}
+```
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `status` | string | `"pass"`, `"warn"`, or `"fail"` |
+| `score` | number | 0.0 to 1.0 aggregate readiness score |
+| `failing_checks` | string[] | List of check names that did not pass |
+| `timestamp` | ISO 8601 | When the check completed |
+
+## Retention Policy
+
+- **Active window**: 90 days of events remain in `.codebase/TELEMETRY.jsonl`.
+- **Monthly rotation**: At the start of each month, rename the current file to `.codebase/TELEMETRY-YYYY-MM.jsonl` and start a fresh `TELEMETRY.jsonl`.
+- **No automatic deletion**: Archived monthly files are never deleted without explicit operator action. If disk space is a concern, the operator moves old archives to cold storage.
+- **Rationale**: Aggressive rotation destroys the long-term pattern signal. Three months of continuous data is the minimum for detecting trends like "agent routing latency creeps up on Fridays" or "context savings degrade after 20+ message sessions."
+
+## Consumption Patterns
+
+### Dashboard
+
+A dashboard agent or external tool reads `.codebase/TELEMETRY.jsonl` directly and renders:
+
+- Daily context savings (tokens_before - tokens_after)
+- Agent success rate by category
+- Decision confidence distribution
+- Readiness score trend over the last 30 days
+
+### Failure Replay Engine
+
+Before recording a new failure, the failure-replay engine queries telemetry for correlated signals:
+
+- Did `agent_routing` success rate drop before this failure?
+- Were there recent `decision_recorded` entries with low confidence and no evidence?
+- Did `readiness_check` scores degrade in the days leading up to the incident?
+
+### Agent Performance Baselines
+
+Agents query their own historical telemetry to set expectations:
+
+- "My median `backend-coder` bugfix duration is 8 minutes. This task is taking 25 minutes — something is wrong."
+- "Context actions in this repo typically save 35% of tokens. Today's savings are 12% — the compaction strategy may need tuning."
+
+## Aggregation Recipe
+
+**Weekly context savings**
+
+To compute tokens reclaimed by pruning in the last 7 days:
+
+1. Filter lines where `type == "context_action"` and `action == "prune"`.
+2. For each matching line, compute `tokens_before - tokens_after`.
+3. Sum the differences.
+
+```bash
+jq -c 'select(.type == "context_action" and .action == "prune") | (.tokens_before - .tokens_after)' .codebase/TELEMETRY.jsonl | awk '{s+=$1} END {print s}'
+```
+
+## Anti-Patterns
+
+- **Do not store secrets or PII in telemetry.** Event payloads must never contain API keys, tokens, user emails, or conversation content. If you need to correlate with a sensitive ID, hash it first.
+- **Do not use telemetry for real-time control.** Telemetry is observability, not a trigger. Do not write events and then read them in the same session to decide what to do next. Use state files or direct signals for control logic.
+- **Do not rotate too aggressively.** Rotating or truncating `TELEMETRY.jsonl` more often than monthly makes it impossible to detect multi-week patterns like gradual agent slowdown or declining decision quality.
+
+## Cross-References
+
+| Component | Relationship |
+|-----------|-------------|
+| `context-steward` | Emits `context_action` events during prune/compact/checkpoint cycles |
+| `decision-trace` | Emits `decision_recorded` events for every recorded decision |
+| `failure-replay-engine` | Queries telemetry for pre-failure signal correlation |
+| dashboard | Reads `TELEMETRY.jsonl` to render operational charts |
+
+## Guidance
+
+- Write events synchronously at the point of action. Do not buffer or batch — a single line append is cheap and eliminates flush complexity.
+- If `TELEMETRY.jsonl` is missing on first write, create it. Do not fail the operation because telemetry is unavailable.
+- Validate event structure before appending. A malformed line corrupts nothing (readers skip it), but it wastes space and loses signal.
+- Prefer explicit `null` over omitting a field. A missing `confidence` is ambiguous (not recorded vs. not applicable); `confidence: null` is clear.

package/dist/services/rtk-manager.d.ts

@@ -1,80 +0,0 @@
-/**
- * rtk-manager — runtime integration for https://github.com/rtk-ai/rtk
- *
- * rtk is a CLI proxy that compresses noisy terminal output (git, npm, test
- * runners, linters, docker, etc.) by 60-90% before it reaches the model
- * context. It works by prefixing supported commands: `rtk git status`.
- *
- * This manager provides:
- * - Detection of an installed rtk binary
- * - Optional agent-triggered initialization (rtk init -g)
- * - Status reporting for diagnostics
- * - wrapCommandArgs() for explicit wrapping in FlowDeck's own spawnSync calls
- *
- * DESIGN NOTES:
- * - No auto-install: downloading + executing a remote shell script is a
- *   supply-chain risk. Users install rtk manually; this plugin detects it.
- * - No startup mutation: init is agent-triggered via rtk-setup tool only.
- * - Live detection: no state cache that goes stale across machines/reinstalls.
- * - Bash hook caveat: `rtk init -g` writes to Claude Code / Copilot global
- *   config. Whether that hook fires in OpenCode's non-interactive bash sessions
- *   depends on the runtime. Explicit wrapping via wrapCommandArgs() is the
- *   reliable alternative.
- */
-export interface RtkDetection {
-    installed: boolean;
-    binPath?: string;
-    version?: string;
-    error?: string;
-}
-export interface RtkInitResult {
-    success: boolean;
-    log: string;
-    telemetryDisabled: boolean;
-    error?: string;
-}
-export interface RtkStatus {
-    installed: boolean;
-    binPath?: string;
-    version?: string;
-    initAttempted: boolean;
-    initSuccess: boolean;
-    telemetryDisabled: boolean;
-    installInstructions?: string;
-}
-/**
- * Locate and verify the rtk binary. Checks PATH first, then known install
- * locations. Returns the first working binary found.
- */
-export declare function detectRtk(): RtkDetection;
-/**
- * Run `rtk init -g` to install the bash hook for Claude Code / Copilot,
- * then immediately run `rtk telemetry disable` to explicitly opt out.
- *
- * Telemetry is disabled by default per rtk docs, but we make it explicit
- * and persistent so consent is never accidentally given in future versions.
- * The `RTK_TELEMETRY_DISABLED=1` env var injected by shell-env-hook provides
- * an additional belt-and-suspenders block at the session level.
- *
- * Note on bash hook: `rtk init -g` writes to Claude Code / Copilot global
- * config. Whether that hook fires in OpenCode's non-interactive bash sessions
- * depends on the runtime configuration. Use `$RTK_BIN <cmd>` explicitly as
- * a reliable alternative when RTK_INSTALLED=true in the environment.
- */
-export declare function initRtk(binPath: string): RtkInitResult;
-/**
- * Return current rtk status. Always performs a live detection check.
- */
-export declare function getRtkStatus(opts?: {
-    runInit?: boolean;
-}): RtkStatus;
-/**
- * Wrap a command with rtk if the binary is available and policy allows it.
- * Returns `[cmd, ...args]` unchanged when rtk is unavailable or policy says no.
- *
- * Usage:
- *   const [c, ...a] = wrapCommandArgs("git", ["status"], "/home/user/.local/bin/rtk")
- *   spawnSync(c, a, { ... })
- */
-export declare function wrapCommandArgs(cmd: string, args: string[], binPath: string | undefined): [string, ...string[]];
-//# sourceMappingURL=rtk-manager.d.ts.map

package/dist/services/rtk-manager.d.ts.map

@@ -1 +0,0 @@
-{"version":3,"file":"rtk-manager.d.ts","sourceRoot":"","sources":["../../src/services/rtk-manager.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;;;GAsBG;AAQH,MAAM,WAAW,YAAY;IAC3B,SAAS,EAAE,OAAO,CAAA;IAClB,OAAO,CAAC,EAAE,MAAM,CAAA;IAChB,OAAO,CAAC,EAAE,MAAM,CAAA;IAChB,KAAK,CAAC,EAAE,MAAM,CAAA;CACf;AAED,MAAM,WAAW,aAAa;IAC5B,OAAO,EAAE,OAAO,CAAA;IAChB,GAAG,EAAE,MAAM,CAAA;IACX,iBAAiB,EAAE,OAAO,CAAA;IAC1B,KAAK,CAAC,EAAE,MAAM,CAAA;CACf;AAED,MAAM,WAAW,SAAS;IACxB,SAAS,EAAE,OAAO,CAAA;IAClB,OAAO,CAAC,EAAE,MAAM,CAAA;IAChB,OAAO,CAAC,EAAE,MAAM,CAAA;IAChB,aAAa,EAAE,OAAO,CAAA;IACtB,WAAW,EAAE,OAAO,CAAA;IACpB,iBAAiB,EAAE,OAAO,CAAA;IAC1B,mBAAmB,CAAC,EAAE,MAAM,CAAA;CAC7B;AAYD;;;GAGG;AACH,wBAAgB,SAAS,IAAI,YAAY,CAsBxC;AAED;;;;;;;;;;;;;GAaG;AACH,wBAAgB,OAAO,CAAC,OAAO,EAAE,MAAM,GAAG,aAAa,CAsCtD;AAED;;GAEG;AACH,wBAAgB,YAAY,CAAC,IAAI,CAAC,EAAE;IAAE,OAAO,CAAC,EAAE,OAAO,CAAA;CAAE,GAAG,SAAS,CA+BpE;AAED;;;;;;;GAOG;AACH,wBAAgB,eAAe,CAAC,GAAG,EAAE,MAAM,EAAE,IAAI,EAAE,MAAM,EAAE,EAAE,OAAO,EAAE,MAAM,GAAG,SAAS,GAAG,CAAC,MAAM,EAAE,GAAG,MAAM,EAAE,CAAC,CAI/G"}

package/dist/services/rtk-policy.d.ts

@@ -1,26 +0,0 @@
-/**
- * rtk-policy — command wrapping policy for rtk integration.
- *
- * Determines which commands benefit from rtk output compression and which
- * should be passed through unchanged. The policy is intentionally conservative:
- * when in doubt, don't wrap.
- *
- * Wrapping benefits:
- * - Commands with large, repetitive, or progress-heavy output
- * - Commands where compressed output preserves the signal
- *
- * Do NOT wrap:
- * - Commands where raw output is required for correctness
- * - Commands with already-compact output (git rev-parse, git diff --name-only)
- * - Commands used for installing rtk itself (curl, sh)
- * - Structured/programmatic output (codegraph, jq, etc.)
- * - OS notification tools (notify-send, osascript, powershell)
- */
-/**
- * Determine whether a command should be wrapped with rtk.
- * Returns true only when compression is expected to improve signal quality.
- */
-export declare function shouldWrapWithRtk(cmd: string, args: string[]): boolean;
-/** Returns the full list of supported commands for diagnostics/documentation. */
-export declare function getSupportedCommands(): string[];
-//# sourceMappingURL=rtk-policy.d.ts.map

package/dist/services/rtk-policy.d.ts.map

@@ -1 +0,0 @@
-{"version":3,"file":"rtk-policy.d.ts","sourceRoot":"","sources":["../../src/services/rtk-policy.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;GAiBG;AA4DH;;;GAGG;AACH,wBAAgB,iBAAiB,CAAC,GAAG,EAAE,MAAM,EAAE,IAAI,EAAE,MAAM,EAAE,GAAG,OAAO,CAatE;AAED,iFAAiF;AACjF,wBAAgB,oBAAoB,IAAI,MAAM,EAAE,CAE/C"}

package/dist/tools/rtk-setup.d.ts

@@ -1,22 +0,0 @@
-import { type ToolDefinition } from "@opencode-ai/plugin";
-/**
- * rtk-setup tool — agent-callable tool for rtk lifecycle management.
- *
- * Provides:
- * - Detection: is rtk installed and where?
- * - Optional init: run `rtk init -g` to install the bash hook
- * - Status report: for diagnostics in the current workflow
- * - Supported commands list: which commands benefit from rtk wrapping
- *
- * Agents should call this tool when:
- * - They want to verify rtk is available before running commands
- * - The workflow involves heavy CLI usage (git, tests, linting, docker)
- * - After a user has installed rtk and wants to activate it
- *
- * Note on bash hook: `rtk init -g` writes to Claude Code / Copilot global
- * config. In OpenCode's non-interactive bash sessions the hook may not fire
- * automatically. Use `$RTK_BIN <cmd>` explicitly as a reliable alternative
- * when RTK_INSTALLED=true in the environment.
- */
-export declare const rtkSetupTool: ToolDefinition;
-//# sourceMappingURL=rtk-setup.d.ts.map

package/dist/tools/rtk-setup.d.ts.map

@@ -1 +0,0 @@
-{"version":3,"file":"rtk-setup.d.ts","sourceRoot":"","sources":["../../src/tools/rtk-setup.ts"],"names":[],"mappings":"AAAA,OAAO,EAAQ,KAAK,cAAc,EAAE,MAAM,qBAAqB,CAAA;AAI/D;;;;;;;;;;;;;;;;;;GAkBG;AACH,eAAO,MAAM,YAAY,EAAE,cA0EzB,CAAA"}

package/docs/reference/rtk.md

@@ -1,162 +0,0 @@
-# rtk Integration
-
-FlowDeck integrates [rtk](https://github.com/rtk-ai/rtk) — a Rust CLI proxy that compresses noisy terminal output (git, npm, test runners, linters, Docker, and more) by 60–90% before it reaches the model context.
-
----
-
-## What rtk does
-
-rtk acts as a transparent proxy in front of supported CLI commands:
-
-```bash
-rtk git status     # same as git status, but output compressed 60-90%
-rtk npm test       # same as npm test, but noise filtered out
-rtk tsc --noEmit   # TypeScript compiler errors, signal-only
-```
-
-This reduces the number of tokens consumed by verbose CLI output — lowering cost and improving signal quality for agents that read shell output.
-
----
-
-## Detection
-
-FlowDeck detects rtk automatically at session startup. No configuration is required.
-
-Detection checks in order:
-1. `rtk --version` via `PATH`
-2. `~/.local/bin/rtk` (default install location on Linux/macOS)
-3. `/usr/local/bin/rtk`
-
-Detection is performed once per session and cached (zero overhead per bash call).
-
----
-
-## Environment Variables Injected
-
-When rtk is detected, FlowDeck injects the following into **every bash tool execution** via the `shell.env` hook:
-
-| Variable | Value | Description |
-|----------|-------|-------------|
-| `RTK_INSTALLED` | `"true"` / `"false"` | Whether rtk was found at session start |
-| `RTK_BIN` | e.g. `/home/user/.local/bin/rtk` | Full path to the rtk binary (only when installed) |
-| `RTK_TELEMETRY_DISABLED` | `"1"` | Always set when rtk is installed — blocks telemetry |
-
-Agents can use these vars directly in bash commands:
-
-```bash
-if [ "$RTK_INSTALLED" = "true" ]; then
-  $RTK_BIN git log --oneline -20
-else
-  git log --oneline -20
-fi
-```
-
----
-
-## Telemetry
-
-FlowDeck **always disables rtk telemetry**. Two layers of protection:
-
-1. **`rtk telemetry disable`** — run automatically after every `rtk-setup init`. Stores an explicit opt-out in rtk's local config (`~/.local/share/rtk/`).
-2. **`RTK_TELEMETRY_DISABLED=1`** — injected into every bash session by FlowDeck's `shell.env` hook. Blocks telemetry at the env-var level regardless of stored consent state.
-
-Both mechanisms are active independently. The env var alone is sufficient to suppress all telemetry pings even if the config opt-out is somehow lost.
-
-See [rtk TELEMETRY.md](https://github.com/rtk-ai/rtk/blob/develop/docs/TELEMETRY.md) for what rtk would collect if telemetry were enabled.
-
----
-
-## Supported Commands
-
-The following commands benefit from rtk compression. FlowDeck's wrapping policy (`rtk-policy.ts`) uses this list:
-
-| Command | What gets compressed |
-|---------|----------------------|
-| `git status` | Staged / unstaged file listing |
-| `git log` | Commit history |
-| `git diff` | Full diff output |
-| `git show` | Commit show output |
-| `npm test` / `bun test` | Test runner output and summaries |
-| `tsc` | TypeScript compiler diagnostics |
-| `eslint` / `biome` / `oxlint` | Lint output |
-| `jest` / `vitest` / `pytest` | Test output |
-| `cargo` | Rust build / test output |
-| `docker` | Container and image listings |
-| `kubectl` | Kubernetes resource listings |
-| `gh` | GitHub CLI output |
-| `pnpm` / `yarn` / `npx` | Package manager output |
-
-**Commands that are never wrapped** (raw output required or already compact):
-
-| Command | Reason |
-|---------|--------|
-| `git rev-parse` | Returns a single hash — already minimal |
-| `git diff --name-only` / `--name-status` / `--stat` | Already compact listing |
-| `git ls-files`, `git config`, `git symbolic-ref` | Compact structured output |
-| `codegraph` | Programmatic structured output — must not be modified |
-| `curl` | Used for downloads — raw output required |
-| `sh`, `bash`, `node`, `python` | Shell interpreters — must not be intercepted |
-
----
-
-## Setup Tool
-
-Agents can check rtk status or trigger initialization via the `rtk-setup` tool:
-
-### Check status
-
-```
-rtk-setup (action: "status")
-```
-
-Returns current detection result, binary path, version, and instructions if rtk is not installed.
-
-### Initialize bash hook
-
-```
-rtk-setup (action: "init")
-```
-
-Runs `rtk init -g` to install the bash rewriting hook, then immediately runs `rtk telemetry disable`. Reports both outcomes.
-
-**Bash hook caveat:** `rtk init -g` writes to Claude Code / Copilot global config. Whether the hook fires automatically in OpenCode's non-interactive bash sessions depends on the runtime. Using `$RTK_BIN <cmd>` explicitly is always reliable.
-
----
-
-## Installing rtk
-
-FlowDeck does **not** auto-install rtk. Auto-executing a remote shell script is a supply-chain risk. Install manually:
-
-```bash
-# Linux / macOS
-curl -fsSL https://raw.githubusercontent.com/rtk-ai/rtk/refs/heads/master/install.sh | sh
-```
-
-After installation, add `~/.local/bin` to your PATH if not already present, then verify:
-
-```bash
-rtk --version
-```
-
-FlowDeck will detect the binary automatically on the next session start.
-
----
-
-## No rtk? No problem.
-
-rtk is entirely optional. If rtk is not installed:
-- `RTK_INSTALLED=false` is injected (no `RTK_BIN` or `RTK_TELEMETRY_DISABLED`)
-- All commands run as normal — no change to behavior
-- The `rtk-setup` tool returns install instructions instead of status
-- All FlowDeck workflows remain fully functional
-
----
-
-## Files
-
-| File | Purpose |
-|------|---------|
-| `src/services/rtk-manager.ts` | Detection (`detectRtk`), init (`initRtk`), status (`getRtkStatus`), wrapping (`wrapCommandArgs`) |
-| `src/services/rtk-policy.ts` | Command wrapping policy — supported list, compact-git exclusions, `shouldWrapWithRtk()` |
-| `src/tools/rtk-setup.ts` | Agent-callable `rtk-setup` tool |
-| `src/hooks/shell-env-hook.ts` | Injects `RTK_INSTALLED`, `RTK_BIN`, `RTK_TELEMETRY_DISABLED` into bash sessions |