better-dsh 0.2.4 → 0.2.5

package/docs/80_upstream-docs-and-skills/hermes-dev-skill/src/docs/developer-guide/relay-shared-metrics.md

@@ -0,0 +1,487 @@
+---
+title: "Relay Shared Metrics"
+description: "NeMo Relay shared metrics: what is exported, consent and retention, staging validation"
+---
+
+# NeMo Relay Shared Metrics
+
+Hermes includes NeMo Relay as a normal runtime dependency on platforms for
+which Relay publishes a native wheel. The shared-metrics integration is built
+into Hermes and does not require a Hermes observability plugin. Hermes remains
+importable without Relay on other native targets. Those targets use an
+explicit reduced-capability no-op host:
+Hermes execution remains available, while Relay scopes, middleware, plugins,
+and subscribers are unavailable. The `hermes-agent[nemo-relay]` extra remains
+as a no-op compatibility alias for existing installation commands.
+
+> [!WARNING]
+> This removes the Hermes `observability/nemo_relay` plugin. Existing users
+> must remove `observability/nemo_relay` (or its legacy `nemo_relay` alias)
+> from `plugins.enabled` and move exporter configuration into a Relay
+> `plugins.toml` selected with `HERMES_NEMO_RELAY_PLUGINS_TOML`. The legacy
+> `HERMES_NEMO_RELAY_ATOF_*` and `HERMES_NEMO_RELAY_ATIF_*` variables no
+> longer activate exporters. Without the new variable, Hermes does not run
+> Relay plugin discovery, configuration layering, middleware, or exporters.
+
+Hermes requires NeMo Relay 0.8.3 or later within the 0.8 release line. That
+line provides the provider-codec and canonical tool-result contracts Hermes
+uses for managed provider and tool calls.
+
+## Runtime Dependency and Data Boundary
+
+Hermes installs the platform-specific `nemo-relay` native wheel from the
+bounded `>=0.8.3,<0.9` dependency range. The published package is built from
+the [NVIDIA NeMo Relay repository](https://github.com/NVIDIA/NeMo-Relay).
+Unsupported platforms use the explicit no-op runtime described above rather
+than downloading a different implementation.
+
+Operator-supplied typed native plugins must be rebuilt for Relay 0.8. `grpc-v1`
+workers must be regenerated and rebuilt when they use tool callbacks, tool
+execution intercepts, or manual tool-end APIs.
+
+When Relay managed execution is active, the provider request and response pass
+through that native module in the Hermes process so configured interceptors can
+operate on the real call. This is separate from the shared-metrics data
+contract. Shared-metrics mode installs no rich-observability network exporter,
+and its subscriber
+accepts only the versioned, allowlisted projection described below. The
+opt-in package sender described in Appendix A is the only outbound path, it
+transmits nothing unless the user enables both `enabled` and `send`, and it
+sends whole packages rather than live spans. Enabling a
+separately configured rich-observability or dynamic plugin can create a
+different data path and requires its own policy review.
+
+Collection remains off unless Hermes policy enables it:
+
+```yaml
+telemetry:
+  shared_metrics:
+    enabled: true
+```
+
+This choice is read from the profile's own `config.yaml`. A machine-managed
+configuration overlay cannot enable or disable shared metrics on the profile's
+behalf.
+
+Relay plugin activation is owned by the native runtime and remains explicitly
+opt-in. Set `HERMES_NEMO_RELAY_PLUGINS_TOML` to a selected `plugins.toml` to
+activate configured middleware, exporters, or dynamic plugins. When the
+variable is unset, Hermes does not invoke Relay's plugin initializer, so Relay
+does not perform plugin configuration discovery or layering. When it is set
+and the selected file loads successfully, Relay discovers supported user and
+system `plugins.toml` files and layers the selected static configuration over
+them. Repository-local `.nemo-relay/plugins.toml` files are ignored. Dynamic
+`[[plugins.dynamic]]` records are loaded from the selected file only. If the
+selected file cannot be loaded, Hermes reports the error and does not invoke
+Relay initialization or fall back to ambient discovery.
+
+## Session-Span Segmentation for Continuous Sessions
+
+Relay exports a span when its scope closes. A continuous gateway session can
+remain open for days, so its session span remains open even though each turn
+span is exported normally. Optional segmentation rotates only the session
+scope at a turn boundary:
+
+```yaml
+gateway:
+  telemetry:
+    session_segments:
+      on_compaction: false  # rotate after context compaction
+      max_turns: 0          # 0 = unlimited; N = turns per segment
+```
+
+| Key | Default | Behavior |
+|---|---:|---|
+| `on_compaction` | `false` | Rotate after compaction completes, at the next turn boundary. |
+| `max_turns` | `0` | Rotate after every N completed turns; `0` disables the cap. |
+
+Both defaults preserve one session scope for the full session. Rotated spans
+retain the same `session_id` and add `hermes.session.segment` plus
+`hermes.session.segment_reason` (`compaction` or `max_turns`).
+
+## Process-Wide Plugin Policy and Profile Isolation
+
+Relay plugin configuration is a process-level deployment choice, not a Hermes
+profile setting. The first hosted profile triggers lazy initialization, and
+every additional profile hosted by that Hermes process shares the resulting
+static middleware, dynamic plugins, subscribers, exporters, and guardrail
+policy. After initialization succeeds, Hermes logs:
+
+```text
+Relay plugins are active process-wide and apply to all profiles hosted by this Hermes process.
+```
+
+Profile scopes still preserve causal isolation inside that shared policy.
+ATIF groups events by their top-level Agent scope, so simultaneous profile
+sessions produce separate trajectories rather than one mixed trajectory.
+ATOF and other global subscribers observe events from every hosted profile.
+Static and dynamic middleware likewise runs for managed calls from every
+profile.
+
+A worker plugin running in a separate worker process does not create a
+per-profile security boundary. One process-wide activation dispatches calls
+from all hosted profiles to that worker while preserving the invoking
+profile's Relay scope stack. Native dynamic plugins are loaded into the Hermes
+process and share the same policy boundary.
+
+Run profiles in separate Hermes processes when they require different trust
+levels, plugin credentials, exporter destinations, or guardrail policies.
+This process-wide plugin contract does not change each profile's independent
+shared-metrics consent, local SQLite state, or ATIF trajectory grouping.
+
+Hermes core owns one Relay host and one isolated Relay session scope per Hermes
+session. Core lifecycle producers use
+`agent.relay_runtime` to obtain the shared session handle or
+run Relay scope, LLM, tool, and mark APIs in that session context. New product
+marks do not require Hermes plugin registration. Shared-metrics marks must
+still contain only fields approved by the versioned allowlist; the hard
+dependency does not change the collection or privacy policy.
+
+## Current Slices
+
+The current vertical slices record pseudonymous profile activity, logical
+model calls, top-level task runs, tool and approval outcomes, and skill
+lifecycle and reuse:
+
+```text
+Hermes turn, API, tool, and approval hooks
+  -> Relay session, task, LLM, tool, and mark lifecycle
+  -> Hermes shared-metrics subscriber
+  -> SQLite counters
+  -> immutable JSON delta package
+```
+
+Hermes sends an empty `LLMRequest` into the metrics-owned lifecycle. This does
+not describe the separate managed-execution call through the native runtime
+documented above. The terminal metrics event contains the model identifier and
+provider route that Hermes used for the logical call, such as
+`nvidia/nemotron-3-ultra` through `openrouter`. These identifiers are
+lowercased and structurally bounded, but they are not normalized through a
+checked-in model catalog. Pricing and model-family classification belong to
+the metrics backend. Prompts, responses, endpoints, errors, session IDs, task
+IDs, and request IDs are not included in the metrics event or package.
+New calls use `hermes.model_route.count`. The previous
+`hermes.model_call.count` contract remains readable only so pending local
+counters created by older builds can be exported without losing data.
+
+The first consented session start emits an empty `hermes.client.active` Relay
+mark. The profile-scoped subscriber creates a random UUID install identity and
+uses a transactional compare-and-set to record at most one client-active
+counter in any rolling 24-hour window. The metric has no dimensions; Hermes
+version, OS family, architecture, and install method remain bounded package
+resources. Concurrent Hermes processes share the SQLite latch, so simultaneous
+starts cannot double-count one install. A later session or task can attempt the
+mark again, but the subscriber suppresses it until the rolling window expires.
+
+Each task run is a Relay `Function` scope named `hermes.task_run`, parented to
+the owning Hermes session. The start counter contains only bounded execution
+surface and entrypoint values. The terminal counter contains bounded outcome,
+end reason, termination status, duration, logical model-call count, terminal
+tool-call count, and provider-retry count buckets. Retries are additional
+provider attempts for the same Hermes API request ID; they do not inflate the
+logical model-call count. Tool calls are deduplicated by their Hermes tool-call
+ID after a terminal tool result is observed. The outer `AIAgent` execution
+boundary closes the task for normal returns, early returns, exceptions, and
+cancellations. Active task ownership follows the task ID if Hermes rotates its
+conversation session during context compression.
+
+Each tool invocation is represented by a Relay tool lifecycle named
+`hermes.tool_call`. The terminal counter contains only bounded tool category,
+outcome, approval outcome, latency, and explicit retry-count buckets. Hermes
+derives the category from the toolset already declared in its runtime registry;
+custom and unrecognized toolsets collapse to `other` rather than exporting
+tool or plugin names. Hermes does not infer retries from repeated tool names or
+adjacent calls; when the
+hook does not provide an explicit retry relationship, the retry bucket is
+`unknown`. Approval decisions are emitted as `hermes.tool_approval` marks and
+recorded as attributed to a tool call or explicitly `unattributed`. Tool names,
+call IDs, arguments, results, commands, descriptions, and error text are not
+included in shared-metrics events or packages. A started tool that is still
+open when its task terminates is closed as failed, timed out, or cancelled and
+remains in the task's tool-count bucket.
+
+Successful skill mutations emit `hermes.skill.lifecycle` marks with only a
+bounded action and provenance. Successful loads emit `hermes.skill.load`
+marks with bounded provenance, first-use or reuse state, reuse-after-patch
+state, and a use-count bucket. Hermes derives reuse and patch-generation
+continuity transactionally in its existing `skills/.usage.json` state; skill
+names and exact counts or generations never enter Relay metrics events,
+SQLite dimensions, or packages. A use after a new patch is counted once as
+`reused_after_patch`; later uses remain ordinary reuse until another patch.
+Task-outcome attribution after a patch remains deferred until its window and
+multi-skill semantics are defined.
+
+Local state is written under:
+
+```text
+$HERMES_HOME/telemetry/shared_metrics/metrics.sqlite3
+$HERMES_HOME/telemetry/shared_metrics/outbox/*.json
+```
+
+The database keeps transactional aggregate and package-outbox state. Package
+files are immutable delta documents that conform to a closed JSON schema and
+are written with atomic replacement. Each package records the Hermes version,
+OS family, architecture, and install method as bounded client resources.
+Unrecognized platform or installation values are exported as `unknown`; raw
+platform strings, hostnames, and paths are never included. Fully packaged
+aggregate rows and successfully exported package rows and files are retained
+locally for 30 days. Pending package rows and counters with unexported deltas
+are never pruned.
+Package schema v1 remains unchanged for existing outbox files. New packages
+use v2, which accepts both the retired model-call contract and the current
+model-route contract so upgrades can drain pending counters safely.
+
+Each package contains an `install_id` generated as a random UUID. Despite the
+schema field name, its current scope is one `HERMES_HOME`, so it is more
+precisely a persistent pseudonymous profile identifier. It is not derived from
+hardware, account, host, path, or credential data. It remains stable across
+packages from that profile and can therefore link those local packages.
+Deleting `$HERMES_HOME/telemetry/shared_metrics` resets the identifier together
+with all aggregates and package files.
+
+Remote delivery is opt-in and off by default. Reusing the persistent local
+identifier remotely required a separate product and privacy decision covering
+consent, identity scope, reset behavior, retention, and deletion — that
+decision has been made.
+
+> Those decisions are recorded in
+> [Appendix A](#appendix-a-remote-exporter-decisions-phase-2), and the exporter
+> implementing them has shipped. Collection alone still transmits nothing: the
+> sender runs only when `telemetry.shared_metrics.send` is also true. Each
+> transmitted package carries the stable `install_id` as-is (product decision,
+> 2026-08-27 — see A.2 for the record, including the superseded
+> HMAC-pseudonym design).
+
+The install identity is scoped to one `HERMES_HOME`. To reset it, stop Hermes
+processes and remove `$HERMES_HOME/telemetry/shared_metrics`. This deliberately
+removes the old identity, aggregate database, and queued local packages
+together; the next consented session creates a new identity. Disabling shared
+metrics stops new collection but does not silently delete previously collected
+local state.
+
+## Smoke Test
+
+Run a real Hermes CLI turn against the deterministic local model server:
+
+```bash
+./.venv/bin/python scripts/smoke_nemo_relay_shared_metrics.py
+```
+
+The script uses the installed `nemo-relay` dependency by default. Pass
+`--relay-python ../nemo-relay/python` only when testing a locally built Relay
+binding.
+
+The smoke has the local model request a real `read_file` tool call before its
+final response, then drives create, load, reuse, patch, edit, stale, archive,
+restore, and install skill transitions through the installed Relay binding. It
+verifies model, provider, task, tool, and skill counters in SQLite, validates
+all exported delta packages against the closed schema, verifies the
+pseudonymous client-active counter, and checks that prompt, response, tool-call
+ID, tool-result, and skill-name canaries are absent from the packages.
+
+## Appendix A: Remote Exporter Decisions (Phase 2)
+
+Status: **implemented.** This appendix answers the product and
+privacy questions that "Current Slices" defers to a future remote exporter. It
+records what was decided and why, so the reasoning survives the implementation.
+
+Sending is off by default and requires both `telemetry.shared_metrics.enabled`
+and `telemetry.shared_metrics.send`.
+
+The exporter sends the package files already written under
+`$HERMES_HOME/telemetry/shared_metrics/outbox/` to the Hermes telemetry ingest
+service. That service validates only the envelope (`schema_version` plus a UUID
+`package_id`) and stores the body verbatim in S3.
+
+### A.1 Consent
+
+Transmission is a **separate opt-in** from collection, under a new config key:
+
+```yaml
+telemetry:
+  shared_metrics:
+    enabled: false   # collect locally
+    send: false      # NEW: transmit to the Nous telemetry service
+```
+
+- `send` defaults to **false**. Collection alone never transmits.
+- `send` requires `enabled`. It does **not** imply it: a transmission flag must
+  not silently switch on collection. `send: true` with `enabled: false` warns
+  and does nothing.
+- Like `enabled`, `send` is profile-owned and is not overridden by
+  managed-scope configuration.
+
+**A package is only sent when its whole period falls inside a recorded
+consent window.** Consent is stored as explicit intervals in the shared-
+metrics SQLite store (`send_consent_windows`): a window opens when `send:
+true` is first observed, is confirmed forward by every later observation,
+and closes — at the last *confirmed* moment, never at the wall clock — when
+`send: false` is observed. A single reconciler derives this table from the
+config on every process start, so wizard changes, hand-edits to
+`config.yaml`, and mid-pass revocations all take the same path, and no
+transition can be missed by any of them.
+
+Any package whose period predates the first window, falls between windows,
+or runs past the newest confirmed moment is excluded — the gate fails
+closed. A fresh package therefore waits at most one process start after its
+period completes before becoming eligible.
+
+The gate is on the **period**, not on the package's creation time. One period
+is split across several packages created on different days: a day's first
+package is written that day, and a tail package for the same period typically
+follows the next day. Gating on creation time would send a period's tail while
+dropping its head, reporting a **silently undercounted** day. Gating on the
+period keeps consent forward-only and every transmitted period complete.
+
+Local history can be up to 30 days old, and that data was collected under a
+promise that nothing is uploaded. Honouring consent forward-only costs at most
+30 days of backlog we never had permission to send.
+
+### A.2 Identity scope — the stable install_id is transmitted as-is
+
+**Decision record.** The original design of this exporter (and revisions 1–8
+of this appendix) transmitted a keyed pseudonym instead of the identifier:
+`HMAC-SHA256(key = locally-held rotating salt, message = install_id)`, with
+the salt rotating every 30 days. On **2026-08-27**, before the feature
+shipped (zero consented users, zero production transmissions), the product
+owner decided the analytical need is a **stable cross-window identity** —
+retention curves, longitudinal install behaviour — which rotation by design
+destroys. The pseudonymization layer was removed in full rather than
+weakened in place.
+
+What is transmitted now:
+
+- Each package carries `install_id` verbatim: the persistent, profile-scoped
+  random UUID described above.
+- It is generated locally (`uuid4`), contains no hardware, account, user, or
+  machine-derived information, and identifies a *profile*, not a person.
+- It is stable until the user deletes the shared-metrics directory, which
+  regenerates it (see A.4).
+
+Consequences stated plainly rather than papered over:
+
+- Packages from one profile correlate **indefinitely**, not per-window.
+  Long-term linkability of one install's daily envelope sequence is now the
+  designed behaviour, not a residue.
+- The A.3 residue analysis of the old design (stable `resource` tuple +
+  contiguous periods bridging rotation windows) is moot — there is no window
+  boundary left to bridge.
+- The setup wizard's consent language states this identity model explicitly;
+  it was updated in the same change that removed the derivation, so no
+  consent was ever collected under the old wording in any shipped build.
+
+**Byte-identical resends still hold.** The transmitted id is recorded on the
+row (`sent_install_id`) when the package is first prepared, and the wire body
+is always rebuilt from that recorded value, so a retry rebuilds identical
+bytes. The contract requires this: resending a `package_id` with different
+content is undefined behaviour. (With a stable id the recorded copy is no
+longer load-bearing against rotation — it remains as the audit column and as
+cheap insurance against any future change to identity semantics.)
+
+### A.3 Rotation — removed (decision record)
+
+Salt rotation was deleted together with the derivation (product decision,
+2026-08-27). This section is retained as a record of what the earlier design
+did and why the removal was accepted:
+
+- Rotation existed to bound long-term linkability: one identity per 30-day
+  window, unrelated identities across windows.
+- The documented residue (see git history for the full analysis): the
+  envelope's stable, low-entropy `resource` tuple plus contiguous daily
+  periods could plausibly bridge windows for rare configurations anyway, so
+  the boundary was a cost-raiser, not a wall.
+- The product need that killed it: cross-window continuity is precisely what
+  retention analysis requires. A boundary that mostly inconveniences honest
+  analysis while only raising costs for a determined correlator was judged
+  the wrong trade once stable identity became a requirement.
+
+There is no salt in the store, no rotation schedule, and no derived
+identifier anywhere in the pipeline.
+
+### A.4 Reset behavior
+
+Removing `$HERMES_HOME/telemetry/shared_metrics` still resets local identity,
+aggregates, and package files, exactly as documented above. Two honest
+qualifications now apply:
+
+- Reset regenerates `install_id`, so subsequent packages transmit a **new**
+  identity. Local reset does give a new remote identity.
+- Reset **cannot unsend**. Packages already transmitted remain in the ingest
+  service's storage under the identifier they were sent with. There is no
+  read-back or delete API in the v1 contract.
+
+Setting `send: false` stops transmission immediately: consent is re-read
+before every package, so a pass already in flight stops after the package it
+is currently sending rather than draining its whole batch. It does not delete
+previously transmitted packages, and it does not stop local collection.
+
+Turning sending off also **closes the consent window** — at the last moment
+consent was actually observed, not at the wall clock. Packages whose periods
+fall between one window and the next are never transmitted, even if sending
+is later re-enabled, and this holds for any number of on/off cycles, across
+hand-edits with no process running, and under a clock that jumps in either
+direction (window opens are clamped above every timestamp already in the
+store; observation marks advance by a bounded step per call, so one glitched
+forward sample cannot drag the confirmation horizon years ahead; a close
+never lands after the closing observation's own clock).
+Unlike the earlier single moving opt-in date, closing and reopening does NOT
+discard the still-undelivered backlog from a previous consented window —
+those packages stay inside their own interval and remain eligible.
+
+One deliberate upgrade-path consequence: packages exported under the
+pre-interval consent model (before `send_consent_windows` existed) predate
+the first recorded window and are therefore never transmitted after an
+upgrade. This is the fail-closed direction — re-importing the old moving
+day-stamp to release them would re-import the semantics five review rounds
+showed to be unsound — and it costs at most the undelivered backlog, never
+collected data.
+
+### A.5 Retention
+
+- **Local:** unchanged — 30 days for successfully exported history, and pending
+  deltas are kept until exported. Send state does **not** extend local
+  retention: a package that could never be sent is still pruned at 30 days.
+  Unbounded local growth against a permanently unreachable endpoint is a worse
+  failure than losing metrics from an install that has been broken for a month.
+- **Remote:** raw packages are retained in S3 without expiry in production and
+  for 30 days in staging.
+
+### A.6 Deletion
+
+There is no remote deletion path in the v1 contract, and this appendix does not
+invent one. What a user can do:
+
+| Action | Effect |
+|---|---|
+| `send: false` | No further packages leave the machine |
+| `enabled: false` | Collection stops; existing local state remains |
+| Remove `.../shared_metrics` | Local identity, aggregates, and files reset; future sends use a new install_id |
+| Delete already-sent data | Not self-service — requires an operator acting on the S3 bucket |
+
+If a deletion-on-request obligation is ever taken on, the lookup path is now
+direct: the user's `install_id` (readable from their local store) is the key
+their data is stored under. Building the service-side delete API remains a
+new product decision, not an implementation detail.
+
+### A.7 What the outbox directory is
+
+Recorded because it was misread once during Phase 2 planning, in a way that
+would have deleted user data.
+
+The directory is **local history, not a send-queue**. `package_outbox` is the
+SQLite table; its `exported_at` column means "written to disk", not "sent".
+Files are immutable and pruned **by age alone**.
+
+The ingest contract says senders should delete a package from their outbox on
+`202`. **The exporter does not do this.** Deleting on acknowledgement would
+repurpose the user's 30-day local history as a transmission queue and destroy
+state they were promised. Send state lives in new columns on the
+`package_outbox` table instead; the files are untouched by transmission.
+
+### A.8 Scope note
+
+The `install_id` field inside the package body is transmitted as the
+generator wrote it (rewritten from the row's frozen `sent_install_id`, which
+records the same value). No other payload field changes, nothing is added,
+and the service treats the whole body as opaque. Payload schema evolution
+therefore stays a sender-side concern, as before.

package/docs/80_upstream-docs-and-skills/hermes-dev-skill/src/docs/developer-guide/secret-source-plugin.md

@@ -0,0 +1,192 @@
+---
+sidebar_position: 9
+title: "Secret Source Plugins"
+description: "How to build a secret-manager backend plugin for Hermes Agent"
+---
+
+# Building a Secret Source Plugin
+
+Secret sources resolve provider credentials from an external secret manager (a vault, a password manager, an OS keystore, a custom script) into environment variables at process startup — after `~/.hermes/.env` loads, before Hermes reads credentials. Bitwarden, 1Password, and a generic command-helper source ship in-tree; **every other backend is a plugin**. This guide covers building one.
+
+:::tip
+The bundled set is deliberately closed, same policy as [memory providers](/developer-guide/memory-provider-plugin): PRs adding new vault backends under `agent/secret_sources/` are closed with a pointer to this guide. Publish your backend as a standalone plugin repo and share it in the Nous Research Discord (`#plugins-skills-and-skins`).
+:::
+
+## First-process bootstrap timing
+
+`load_hermes_dotenv()` often runs at import time **before** plugins register.
+Hermes then re-pulls secrets after plugin discovery when any **enabled**
+plugin secret source is configured. Enablement uses the source's
+`is_enabled(cfg)` contract; the standard form is
+`secrets.<name>.enabled: true`, while custom activation remains supported.
+That closes the "replace Bitwarden with my vault" first-process gap (#64177).
+
+- Re-pull is idempotent and fail-open (never blocks startup).
+- `hermes update` never resolves external sources — not in the updater process
+  and not in the import-health probes it spawns. Nothing in the update path
+  needs credentials, and a slow helper would otherwise be misreported as an
+  import failure (#110823).
+- Sources only supply env vars through the orchestrator; there is **no**
+  plugin API to dump other plugins' or the user's entire secret store beyond
+  what your source's own config allows.
+- Reading `os.environ` after load is possible for any in-process code — the
+  trust boundary remains "enabled plugins run with agent privilege".
+
+## What the framework owns vs. what you own
+
+The orchestrator (`agent.secret_sources.registry.apply_all`) owns everything security- and precedence-sensitive, so a backend cannot get it wrong:
+
+| Framework owns | You own |
+|---|---|
+| Source ordering, mapped-vs-bulk precedence | Fetching values from your backend |
+| First-claim-wins conflict handling + warnings | Validating your reference format |
+| `override_existing` semantics (never crosses sources) | Talking to your CLI/SDK/API |
+| Protected bootstrap tokens | Declaring which env var IS your bootstrap token |
+| Per-source wall-clock timeout | Keeping `fetch()` reasonably fast |
+| Per-var provenance + `(from X)` labels | A human-readable `label` |
+| `os.environ` writes | Nothing — you never touch the environment |
+
+## Directory structure
+
+```
+~/.hermes/plugins/my-vault/
+├── plugin.yaml      # name, description
+└── __init__.py      # SecretSource subclass + register(ctx)
+```
+
+## The SecretSource ABC
+
+Implement `agent.secret_sources.base.SecretSource`. One method is required:
+
+```python
+from pathlib import Path
+
+from agent.secret_sources.base import (
+    ErrorKind,
+    FetchResult,
+    SecretSource,
+    run_secret_cli,
+)
+
+
+class MyVaultSource(SecretSource):
+    name = "myvault"          # config section key: secrets.myvault
+    label = "My Vault"        # used in startup lines + provenance labels
+    shape = "mapped"          # "mapped" (explicit VAR→ref map) or "bulk" (project dump)
+    scheme = "mv"             # optional: unique URI scheme you own (mv://...)
+
+    def fetch(self, cfg: dict, home_path: Path) -> FetchResult:
+        """Resolve secrets. MUST NOT raise. MUST NOT prompt."""
+        result = FetchResult()
+        token = os.environ.get("MYVAULT_TOKEN", "").strip()
+        if not token:
+            result.error = "secrets.myvault.enabled is true but MYVAULT_TOKEN is not set."
+            result.error_kind = ErrorKind.NOT_CONFIGURED
+            return result
+
+        try:
+            proc = run_secret_cli(
+                ["myvault-cli", "export", "--json"],
+                allow_env=["MYVAULT_TOKEN"],   # ONLY your auth vars — never full os.environ
+                timeout=30,
+            )
+        except RuntimeError as exc:           # spawn failure / timeout
+            result.error = str(exc)
+            result.error_kind = ErrorKind.BINARY_MISSING
+            return result
+
+        if proc.returncode != 0:
+            result.error = f"myvault-cli exited {proc.returncode}: {proc.stderr[:200]}"
+            result.error_kind = ErrorKind.AUTH_FAILED
+            return result
+
+        result.secrets = parse_your_output(proc.stdout)  # {ENV_VAR: value}
+        return result
+
+    def protected_env_vars(self, cfg: dict):
+        # Your bootstrap token — no source (including yours) may ever overwrite it.
+        return frozenset({"MYVAULT_TOKEN"})
+```
+
+### Contract rules (enforced, not suggestions)
+
+- **`fetch()` never raises.** Errors go in `result.error` + `result.error_kind`. A raising fetch is contained by the orchestrator and reported as `INTERNAL` — a contract violation, not a feature.
+- **`fetch()` never prompts.** Startup runs in non-TTY contexts (gateway, cron, Docker). `run_secret_cli()` closes stdin so a prompting helper fails fast. Interactive auth belongs in your CLI setup flow, never on the startup path.
+- **Sync, within budget.** The orchestrator enforces a wall-clock timeout (default 120s, user-tunable via `secrets.<name>.timeout_seconds`). Exceeding it reports `TIMEOUT` and your result is discarded.
+- **You fetch; the orchestrator applies.** Return the mapping you *would* contribute. Never write `os.environ` yourself — you'd bypass precedence, conflict detection, and provenance.
+- **API versioning.** `SecretSource.api_version` defaults to the current `SECRET_SOURCE_API_VERSION`. The registry skips (with a warning) sources built against a different version instead of crashing startup.
+
+### Choosing your `shape`
+
+- `mapped` — the user explicitly binds env-var names to references in config (like 1Password's `env:` map). Strongest intent: mapped claims beat bulk claims on contested vars.
+- `bulk` — you inject a whole project/folder of secrets implicitly (like Bitwarden BSM). Yields to mapped sources.
+
+### Optional hooks
+
+| Method | Default | Override when |
+|---|---|---|
+| `is_enabled(cfg)` | `cfg.get("enabled")` | Custom activation logic |
+| `override_existing(cfg)` | `cfg.get("override_existing", False)` | You want a different default (both bundled sources default `True` for rotation) |
+| `protected_env_vars(cfg)` | empty | You have a bootstrap token (you almost certainly do) |
+| `fetch_timeout_seconds(cfg)` | 120s | Your backend needs a different budget |
+| `config_schema()` | `{}` | Declare config keys for setup surfaces |
+| `remediation(kind, cfg)` | generic per-`ErrorKind` hints | You want failure warnings to point at your own fix-it command (e.g. the bundled sources return `Run hermes secrets <name> token…` for `AUTH_FAILED`). Must be a pure kind→string mapping: no I/O, never raises. Return `""` to suppress the hint. |
+
+## Subprocess safety: use `run_secret_cli()`
+
+If your backend shells out to a CLI, use the shared helper instead of `subprocess.run` directly. It gives you the audited posture for free: argv-only (no `shell=True`), a **minimal allowlisted child environment** (by the time sources run, `os.environ` holds every credential Hermes knows — never hand that to a child process), `NO_COLOR` + ANSI-scrubbed stderr, stdin closed, timeout → clean `RuntimeError`. Pass user-supplied reference strings after a `--` terminator in your argv so they can never parse as flags.
+
+## Registering
+
+```python
+# __init__.py
+def register(ctx):
+    ctx.register_secret_source(MyVaultSource())
+```
+
+Registration is rejected (with a log warning, never a crash) for: non-`SecretSource` instances, invalid/duplicate names, a `scheme` another source owns, wrong `api_version`, or a `shape` outside `mapped`/`bulk`.
+
+:::note Timing
+Plugin discovery runs later in startup than the first `load_hermes_dotenv()` call. Immediately after discovery, Hermes re-pulls enabled plugin secret sources (`reset_secret_source_cache()` + `load_hermes_dotenv()`), so the discovering process *does* pick them up — see [First-process bootstrap timing](#first-process-bootstrap-timing) above (#64177). The re-pull is fail-open and skipped when no plugin source is enabled. Any code that reads `os.environ` during the plugin module's import or `register(ctx)` still runs before the re-pull and cannot depend on credentials supplied by that same source; keep credentialed work inside `fetch()`. Gateway, cron, and subagent processes perform the same discovery/re-pull sequence. The re-pull (and the per-fire cron re-pull) resets only the resolving home's cache, so under a multiplex gateway sibling profiles keep their hydrated snapshots; and a re-pull whose keys already sit in the process environment (`skipped_existing`, e.g. the previous apply's own write-back) still records the home's effective values, so `override_existing` is never required just to survive a re-pull.
+:::
+
+## Users configure it like any other source
+
+```yaml
+secrets:
+  sources: [myvault, bitwarden]   # optional ordering
+  myvault:
+    enabled: true
+    # ... your config_schema keys
+```
+
+Multi-source precedence, conflict warnings, and `(from My Vault)` provenance labels all work automatically — see the [user-facing secrets docs](/user-guide/secrets/) for the precedence ladder.
+
+## Validate with the conformance kit
+
+Subclass the kit from the Hermes repo (`tests/secret_sources/conformance.py`) in your plugin's tests:
+
+```python
+import pytest
+from tests.secret_sources.conformance import SecretSourceConformance
+
+class TestMyVaultConformance(SecretSourceConformance):
+    @pytest.fixture
+    def source(self):
+        return MyVaultSource()
+```
+
+It checks the rules that break other people when violated: never-raises on malformed config, machine-readable error kinds, disabled-by-default, positive timeouts, valid protected-var names, and a full `apply_all()` round trip. Green conformance is the review bar for calling a backend contract-compliant.
+
+## ErrorKind reference
+
+| Kind | Meaning |
+|---|---|
+| `NOT_CONFIGURED` | Enabled but missing token / project / map |
+| `BINARY_MISSING` | Helper CLI not found or not executable |
+| `AUTH_FAILED` / `AUTH_EXPIRED` | Bad / expired credentials |
+| `REF_INVALID` | A secret reference failed validation |
+| `NETWORK` | Transport-level failure |
+| `EMPTY_VALUE` | Backend returned nothing for a ref — never apply `""` over a good credential |
+| `TIMEOUT` | Fetch exceeded its budget |
+| `INTERNAL` | Anything else (bug, unexpected shape) |