stable-harness 0.0.7 → 0.0.9

package/docs/compatibility-matrix.md

@@ -0,0 +1,139 @@
+# Stable Harness Compatibility Matrix
+
+This document tracks compatibility work against `agent-harness` while preserving the clean runtime boundary.
+
+`stable-harness` should not replicate `agent-harness` as a project. It should only keep the compatibility surface needed for downstream workspaces while rebuilding runtime-owned capabilities as small, typed, pluggable modules.
+
+## Classification
+
+Every compatibility item must use one of these decisions:
+
+- `passthrough`: the upstream backend already owns the capability
+- `runtime wrapper`: `stable-harness` adds lifecycle, events, governance, persistence, or protocol access around upstream execution
+- `plugin capability`: runtime-owned but optional and replaceable
+- `downstream workspace`: application-specific behavior owned by EasyNet, Flev, or another workspace
+- `do not build`: duplicates upstream execution semantics or hardcodes downstream behavior
+
+## Current EasyNet Gate
+
+EasyNet currently uses this local compatibility path:
+
+```text
+stable-harness -> file:../stable-harness
+```
+
+The passing gate proves that `stable-harness` can run the current EasyNet E2E suite. It does not prove full `agent-harness` parity.
+
+The current migration implementation lives under `runtime/compat`. New runtime capabilities should not extend that folder by default; they should move into typed packages or pluggable runtime modules.
+
+The root package now exposes only native stable runtime entrypoints. The `agent-harness` API remains available only from `./compat/agent-harness.js` for explicit compatibility imports.
+
+Verified gates:
+
+- `npm test` in EasyNet
+- `npm run test:botbotgo:full` in EasyNet
+- `npm run test:stable-native-cli` in EasyNet
+- stable-harness package tests for explicit compat import support through `./compat/agent-harness.js`
+- CLI compatibility through EasyNet's package-local `node_modules/.bin/botbotgo`
+- native typed tool execution through EasyNet's package-local `node_modules/.bin/botbotgo --tool`
+- native CLI coverage through EasyNet's package-local `node_modules/.bin/stable-harness`
+
+## Compatibility Rows
+
+| Capability | Current owner | Decision | Current status | Next action |
+| --- | --- | --- | --- | --- |
+| Public compat API: `createAgentHarness`, `request`, `stop` | `compat/agent-harness` | runtime wrapper | Minimal EasyNet-compatible facade exists only on the explicit compat subpath | Keep as a thin migration facade; move implementation behind core runtime contracts |
+| `botbotgo` CLI entrypoint | stable CLI facade | runtime wrapper | Natural-language requests stay on the compatibility runner; typed `--tool` requests now use native stable runtime/tool gateway | Continue moving typed surfaces first; switch natural-language requests only after upstream DeepAgents delegation/tool-calling satisfies the real matrix |
+| Workspace YAML loading | stable runtime | plugin capability | Loads EasyNet agent/model/tool/skill/memory config; EasyNet config now uses `apiVersion: stable-harness.dev/v1` | Move package-local loader into `@stable-harness/workspace-yaml` as the single implementation |
+| DeepAgents execution semantics | DeepAgents | passthrough | Native adapter can call upstream `createDeepAgent`; EasyNet direct tests use native runtime/tool calls, while natural-language `botbotgo` compatibility still uses the compatibility runner | Move downstream natural-language CLI flows from compat runner to native adapter when upstream tool calling is stable |
+| Subagent delegation semantics | DeepAgents | passthrough | Current runner asks model for a routed agent list and emits stable events; direct user-text route shortcuts were removed from compat | Keep event projection; move actual delegation to upstream DeepAgents task/subagent primitives |
+| Delegation plan events | stable runtime | runtime wrapper | Compat emits `delegation.plan` and `delegation.start`; native trace now projects observed upstream DeepAgents `task` tool calls as `delegation.start` and `delegation.completed` entries without selecting agents locally | Keep as normalized observability only; do not synthesize task calls or route from text |
+| Explicit tool execution | stable runtime tool gateway | runtime wrapper | Native `RuntimeRequest.toolCall` invokes only the selected agent's declared tools through `toolGateway`; EasyNet tool matrix, contract tests, and native CLI E2E no longer use compat `toolName`/`args`; the compat direct-invoke request fields were removed | Keep direct tool execution native-only and continue migrating remaining natural-language compat paths separately |
+| Upstream tool calling | upstream backend | passthrough | DeepAgents receives tool gateway tools, but the current remote Ollama path is not yet stable enough to replace compat tool selection | Keep as upstream passthrough; do not add prompt or keyword repair in the adapter |
+| Tool start/result events | stable runtime | runtime wrapper | Native runtime emits `runtime.tool.direct.started` and `runtime.tool.direct.completed`; DeepAgents adapter emits upstream tool events under `runtime.adapter.event`; `projectRuntimeTrace` exposes a normalized trace view consumed by native CLI `--trace` and `--trace-json`; native CLI now streams `--trace` rows as events happen, so long/timeout runs expose progress before finalization; EasyNet asserts native `subscribe` tool events without compat `dataListener` | Continue moving downstream CLI trace checks from compat dataListener to native trace output |
+| Tool argument generation | upstream backend | passthrough | Compatibility runner uses constrained model prompts; user-input JSON tool-call pre-execution was removed from compat | Remove from default DeepAgents adapter; keep only optional tool gateway repair/validation if configured |
+| Required evidence tools | stable runtime policy | plugin capability | Native runtime validates `executionContract.requiredEvidenceTools` from emitted tool events and fails incomplete runs; compat repair for missing required evidence was removed | Keep as validation policy only; do not synthesize or repair tool calls from this contract |
+| Required planning checkpoint | stable runtime policy | plugin capability | Native runtime validates `executionContract.requiresPlan` by observing `write_todos` tool events and fails incomplete runs | Keep as validation policy only; TODO execution remains upstream/back-end owned |
+| TODO trace projection | stable runtime event/view | runtime wrapper | Native `projectRuntimeTrace` projects observed upstream `write_todos` tool completions as `plan.updated`; `readPlanTodos` normalizes structured TODO payloads for presentation; stable no longer provides local `write_todos` or `read_todos` tools | Keep as presentation/event projection only; do not parse TODO text to create tool calls |
+| Built-in `write_todos` and `read_todos` | DeepAgents | passthrough/runtime wrapper | DeepAgents owns its built-in `write_todos`; stable observes upstream `write_todos`/`task` tool calls through middleware without injecting same-name tools; the stable tool gateway planning-tool implementation was removed | Keep observing upstream built-ins; do not synthesize planning or delegation calls |
+| Model provider path | upstream backend adapter | passthrough | Minimal Ollama prompted JSON path supports EasyNet | Move provider handling into backend adapter config; avoid stable-owned model execution semantics |
+| Result compression/final response | runtime optional layer | plugin capability | Not formalized; current runner returns model/fallback text | Design as optional output policy with typed config |
+| Approvals/HITL | stable governance | plugin capability | Not implemented in compatibility runner | Build as independent governance capability |
+| Sandbox policy | stable governance | plugin capability | Not implemented in compatibility runner | Build as independent sandbox policy capability |
+| Run persistence/recovery | stable runtime | plugin capability | Not implemented in compatibility runner | Build as independent run store and recovery capability |
+| Memory lifecycle | stable runtime | plugin capability | Not implemented beyond config load | Build memory lifecycle outside backend execution semantics |
+| Replay/artifacts/traces | stable runtime | plugin capability | Runtime records events per run and exposes `projectRuntimeTrace`; native CLI can print stable trace rows or JSON trace; HTTP exposes `/runs/{requestId}/trace` | Add protocol streaming/export over the same event model |
+| EasyNet specialist prompts and tool ownership | EasyNet workspace | downstream workspace | Owned by EasyNet YAML and resources | Keep out of generic runtime |
+| Finance, Kubernetes, QA, release, secretary rules | EasyNet workspace | downstream workspace | Encoded in EasyNet agents, tools, and skills | Do not move into `stable-harness` runtime |
+| Natural-language keyword routing | none | do not build | Forbidden | Use typed config, metadata, runtime state, or upstream primitives only |
+| TODO text to tool-call inference | none | do not build | Forbidden | Use upstream tool calls or typed runtime policy only |
+
+## Current Technical Debt
+
+The EasyNet compatibility runner is intentionally temporary. It contains behavior that must be split before `stable-harness` becomes a clean production runtime:
+
+- `runtime/compat` should remain a migration-only layer and must not become the default stable runtime core
+- routing prompt and parser should become either upstream DeepAgents delegation passthrough or an optional typed router capability
+- natural-language compat tool selection should be replaced by upstream DeepAgents tool calling; direct `toolName`/`args` invocation moved to native `RuntimeRequest.toolCall`, and execution contracts moved to native validation
+- local TODO built-ins were removed; native planning evidence must come from observed upstream DeepAgents `write_todos`
+- prompted JSON tool choice should not be the stable default when upstream tool calling exists
+- compatibility CLI output should be replaced by native trace projection where the behavior is present in runtime events
+
+Current migration evidence:
+
+- `stable-harness` native CLI can run an EasyNet specialist through the real remote Ollama model for non-tool synthesis.
+- `stable-harness` native runtime can load EasyNet workspace tools through `createModuleToolGateway`.
+- `stable-harness` native runtime and native CLI can execute an explicit EasyNet `git_command` request through `RuntimeRequest.toolCall` and `createModuleToolGateway`, without using `runtime/compat`.
+- EasyNet's package-local `botbotgo --tool` now executes typed tool calls through native stable runtime/tool gateway and emits native trace rows; natural-language `botbotgo` requests remain on compat.
+- EasyNet's model/context matrix can run in native diagnostic mode with `EASYNET_E2E_NATIVE=1`; this drives `botbotgo --native --trace` while preserving the default compat matrix.
+- EasyNet now has `npm run test:stable-native-diagnostics`, a non-compat-readiness gate that requires native natural-language runs to emit stable trace before they are allowed to fail on known upstream orchestration gaps.
+- `stable-harness` native CLI `--trace` and `--trace-json` project `runtime.tool.direct.started` and `runtime.tool.direct.completed` from runtime events; EasyNet native CLI E2E asserts this trace without compat.
+- `stable-harness` native trace projects observed `write_todos` completions as `plan.updated`; `readPlanTodos` reads structured TODO payloads for presentation, and does not synthesize TODO calls or parse TODO text.
+- DeepAgents adapter now installs a `StableHarnessObserver` middleware that observes upstream built-in `write_todos` and `task` tool calls and emits stable `runtime.adapter.event` records without registering or replacing those tools.
+- The same observer now records upstream DeepAgents filesystem built-ins (`ls`, `read_file`, `write_file`, `edit_file`, `glob`, `grep`) so native traces expose policy-denied filesystem attempts without replaying or replacing upstream tools.
+- Native `projectRuntimeTrace` now projects observed upstream DeepAgents `task` calls as stable delegation trace entries, preserving the upstream `subagent_type` as detail when present.
+- Native DeepAgents adapter treats `config.builtinTools.modelExposed` as an explicit governance switch: `false` hides upstream `task`, `["task"]` restricts task targets to workspace-declared subagents, and omitted config preserves upstream DeepAgents task behavior including `general-purpose`.
+- EasyNet native contract tests now assert that invalid upstream task targets such as `research-analyst` are blocked by the stable adapter policy and are not locally remapped to an EasyNet specialist.
+- EasyNet native contract tests also assert that specialists with `modelExposed: false` block upstream `task` completely.
+- Native DeepAgents adapter maps stable `config.builtinTools.filesystem: false` to upstream DeepAgents filesystem permissions that deny `read` and `write` on `/**`, unless the workspace explicitly provides `config.deepagents.permissions`.
+- Native CLI `--trace` streams runtime trace rows before completion, making native adapter hangs/timeouts observable.
+- Compatibility CLI output formatting is centralized in `formatCompatDelta`, which consumes the shared structured TODO reader instead of parsing TODO payloads in the CLI entrypoint.
+- EasyNet native contract tests assert `runtime.tool.direct.started` and `runtime.tool.direct.completed` through stable runtime `subscribe`, without compat `dataListener`.
+- Native workspace loader now discovers `resources/skills/*/SKILL.md` as stable skill inventory and reads each agent's `skills` and `memory` refs without executing them.
+- Native workspace loader reads agent metadata descriptions, and the DeepAgents adapter passes those descriptions through for upstream subagent definitions.
+- EasyNet skill metadata contract now consumes native `workspace.skills` inventory instead of the legacy `runtime/skills/skill-metadata.js` helper.
+- Native workspace loader records agent source paths and discovers module tool ids from `export const <toolId> = tool(...)`, so tool inventory no longer depends on filename-only compatibility behavior.
+- EasyNet contract and real integration tests no longer import the legacy `workspace/compile.js`; they use native `loadWorkspaceFromYaml` for model, agent, tool, skill, memory, source, and response-format inventory.
+- EasyNet specialist response formats are explicit in workspace YAML instead of relying on the legacy compiler's default response-format completion.
+- EasyNet tests and scripts no longer import `compat/agent-harness.js`; remaining EasyNet compatibility coverage goes through the package-local `botbotgo` CLI while native coverage uses `stable-harness` runtime/CLI entrypoints.
+- `stable-harness` native runtime validates EasyNet release `requiredEvidenceTools` and reports missing evidence without compat repair prompts.
+- `stable-harness` native runtime validates EasyNet specialist `requiresPlan` and reports missing `write_todos` checkpoints without synthesizing TODO calls.
+- EasyNet's real specialist tool matrix now executes through native stable `toolCall`; compat request `toolName`/`args` no longer exists as a migration path.
+- EasyNet real and full matrix scripts now resolve `botbotgo` from the workspace package bin instead of a checkout-internal `stable-harness/dist/cli.js` path.
+- EasyNet workspace YAML now declares `apiVersion: stable-harness.dev/v1`; the old `agent-harness/v1alpha1` name is no longer the application config identity.
+- Compatibility runner no longer exposes local `write_todos` or `read_todos`; visible planning evidence in native paths must come from observed upstream DeepAgents built-ins, not stable-owned duplicate tools.
+- DeepAgents 1.9.1 rejects custom tools named `write_todos` because planning is a built-in upstream tool; the native adapter intentionally does not inject stable planning tools over upstream built-ins.
+- Native EasyNet research still is not a full replacement for compat: real runs can fail before synthesis when the upstream model selects the built-in `task` tool with an invalid agent type such as `research-analyst`, and repeated checks also hit remote model `502 Bad Gateway` or 120s timeouts; this must be solved by upstream delegation/tool behavior, endpoint stability, or typed workspace configuration, not by runtime keyword routing.
+- A native diagnostic matrix sample against EasyNet `research-stock` now observes stable trace rows and classifies upstream filesystem access as `native_filesystem_denied` when EasyNet disables filesystem built-ins; the current DeepAgents API still exposes filesystem middleware by default, so this remains an upstream/workspace configuration blocker rather than a stable runtime routing feature.
+- The native diagnostic gate now separates stable readiness from compat-style business assertions: `passed` means native trace/readiness when `EASYNET_E2E_REQUIRE_NATIVE_TRACE=1`, while `businessPassed` keeps the old full-matrix success signal for natural-language migration work. Filesystem-denied native blockers must include a concrete upstream builtin trace such as `agent.tool.start:grep`.
+- Compatibility runner no longer pre-executes JSON tool calls parsed directly from user input; explicit execution belongs to native typed `RuntimeRequest.toolCall`.
+- Compatibility runner no longer routes directly from user text patterns such as explicit `delegate/route/委托 <agentId>`; routing now goes through the compat model-routing prompt until upstream DeepAgents delegation replaces it.
+- Native upstream DeepAgents tool-calling with the current remote Ollama path is not yet a replacement for the compatibility runner; a direct `git_command` request hung until terminated. Keep compat tool-selection behavior until upstream tool-calling is stable or an explicit typed tool-execution policy replaces it.
+
+## Acceptance Rule
+
+A compatibility feature is acceptable only when all of these are true:
+
+- it does not duplicate a current DeepAgents execution primitive in the default path
+- it can be disabled, replaced, or moved behind typed config
+- it has focused tests that assert runtime contracts rather than downstream business cases
+- it keeps downstream product logic in the workspace
+- it improves migration without expanding the public runtime API unnecessarily
+
+## Next Cleanup Sequence
+
+1. Replace the compatibility runner's default DeepAgents path with upstream `createDeepAgent`.
+2. Replace natural-language compat tool selection with upstream DeepAgents tool calling once the remote Ollama tool-call path is stable.
+3. Move remaining CLI trace checks from compat `dataListener` to native `projectRuntimeTrace` where native runtime events exist.
+4. Keep only the explicit `./compat/agent-harness.js` facade until downstream workspaces can use the native stable runtime API directly.
+5. Add tests for each runtime capability boundary before broadening beyond EasyNet.

package/docs/engineering-rules.md

@@ -0,0 +1,111 @@
+# Stable Harness Engineering Rules
+
+These rules are mandatory for all implementation work in this repository.
+
+## Size Limits
+
+Hard limits:
+
+- Source code file: 500 lines maximum.
+- Function: 100 lines maximum.
+- Direct files per folder: 10 files maximum.
+
+The limits apply to source code and project-authored configuration. Generated output under `dist`, `coverage`, or package manager folders is ignored.
+
+## How to Stay Under the Limits
+
+Split by product boundary:
+
+- runtime contract
+- workspace parsing
+- adapter integration
+- protocol adapter
+- policy evaluation
+- memory store
+- CLI command
+- test fixture
+
+Do not split by vague utility names when a product boundary is available.
+
+## Layering Rules
+
+`core` must stay backend-neutral.
+
+`workspace-yaml` may compile backend-specific config, but it must not execute backend behavior.
+
+Adapters may import upstream frameworks. Core packages must not import adapters.
+
+Protocol packages may import core runtime contracts. They must not import backend SDKs directly.
+
+Governance packages may inspect structured policy, metadata, tool IDs, approval state, and runtime records. They must not inspect user prose to infer intent.
+
+## Generic Runtime Rules
+
+Native runtime code must stay generic. Do not add behavior that only makes sense for one downstream workspace, business domain, benchmark case, prompt wording, model family, or backend quirk.
+
+New concepts are allowed only when `stable-harness` clearly owns the runtime/control-plane capability. If an upstream backend already has a concept for the behavior, expose or configure that concept through adapter passthrough instead of inventing a stable-owned replacement.
+
+Every runtime capability must be modular: it needs a narrow interface, explicit config, focused tests, and a replacement point. Capabilities such as approvals, events, memory, tool gateway, replay, artifacts, sandbox policy, protocol serving, and adapter execution must not be bundled into one non-replaceable subsystem.
+
+Workflow graphs and workflow routing are runtime inventory, not generic intent inference. Route selection must come from typed config, explicit request metadata, operator selection, or a future pluggable router capability; it must not be hardcoded from natural-language prompts.
+
+Recovery logic may react to structured adapter events only through typed
+runtime config. Core recovery code must not know backend tool names, downstream
+error strings, or application-specific retry instructions.
+
+Workspace YAML must not default an agent to a specific backend. `Agent.spec.backend`
+is required so scaffolded examples can choose DeepAgents first without making
+the generic schema DeepAgents-shaped.
+
+Native workspace loading must preserve adapter and backend names as configured.
+Compatibility aliases belong in explicit compat modules or adapter-owned config,
+not in the generic YAML schema.
+
+## Feature Preflight Rules
+
+Before adding any runtime, adapter, tool, or compatibility feature:
+
+- Inspect the current upstream framework behavior first.
+- Prove that `stable-harness` actually owns the product capability.
+- Prefer upstream passthrough when DeepAgents, LangChain, or another backend already provides the primitive.
+- Avoid adding a stable-owned product concept when typed upstream config or adapter passthrough is enough.
+- Do not add local versions of upstream built-in tools such as DeepAgents planning, delegation, filesystem, or memory primitives.
+- If the capability is only needed to reproduce old `agent-harness` output, keep it out of native runtime and require a documented migration reason.
+- If there is no current product need, do not implement the feature.
+
+## Testing Rules
+
+Every new runtime behavior needs at least one focused test.
+
+Tests should verify boundaries:
+
+- YAML compiles into typed runtime state
+- adapters receive explicit config
+- runtime emits stable events
+- governance decisions use structured policy
+- forbidden keyword heuristics are absent from generic runtime packages
+
+## Documentation Rules
+
+Every product-visible behavior should be documented before or with implementation.
+
+Documentation should explain:
+
+- which layer owns the behavior
+- what is public product surface
+- what remains backend-specific
+- what is intentionally not supported
+
+## Review Checklist
+
+Before merging a change, verify:
+
+- no source file exceeds 500 lines
+- no function exceeds 100 lines
+- no folder has more than 10 direct files
+- no generic runtime code contains downstream domain heuristics
+- no adapter reimplements upstream semantics unnecessarily
+- every new feature has an upstream capability check and product need
+- every new native concept has a generic runtime owner and is not an upstream duplicate
+- every new capability can be enabled, disabled, replaced, and tested independently
+- no public API grows without a product-boundary reason