coding-agent-harness 1.0.4 → 1.0.6

package/docs-release/architecture/system-explainer/en-US/01-system-overview.md

@@ -0,0 +1,227 @@
+# 01 — System Overview
+
+## What this is and what problem it solves
+
+Before AI coding tools (Codex, Claude Code, Gemini CLI) became widespread,
+"task management" for developers meant Jira tickets or GitHub issues —
+tools designed for humans that Agents can't read and can't derive verifiable state from.
+
+**Coding Agent Harness** solves a core problem:
+
+> When an Agent is working in your repository, how do you ensure its work is traceable,
+> gated, and reviewable?
+
+It's not an Agent itself, and it's not a task management tool for humans.
+It's a **repository-native operating layer** — it gives Agents executable structured context
+so they can resume execution from files without relying on previous chat memory.
+
+The core design philosophy in one sentence:
+
+> **Store important state in Markdown files that Agents can read, then use the CLI to derive
+> status, checks, migration plans, and Dashboard views from those files.**
+
+---
+
+## Why it's called a "Harness"
+
+"Harness" in an engineering context means a constraining device — something used to constrain,
+guide, and measure a system's behavior without replacing it. Just like a "test harness" in
+testing isn't the tests themselves, but the infrastructure that lets tests be organized,
+executed, and verified.
+
+Coding Agent Harness isn't a replacement for Agents — it's a harness for them:
+- Tasks have a lifecycle (create → execute → review → closeout)
+- Reviews have gates (Agents can't approve their own work)
+- State is recorded (every change is written to Markdown, git-blammable)
+- Humans have a review entry point (local Dashboard + Workbench review confirmation)
+
+---
+
+## The essential difference from Jira / Linear / GitHub Issues
+
+| | Jira / Linear / GitHub Issues | Coding Agent Harness |
+| --- | --- | --- |
+| Designed for | Human collaboration | Agent execution + human review |
+| State lives in | External SaaS database | Markdown files in the repository |
+| Can Agents read it? | Requires API integration | Read files directly |
+| Can you git diff it? | No | Yes |
+| Works offline? | No | Yes |
+| Can resume execution from files? | No | Yes |
+
+---
+
+## Level 0 — Four main blocks
+
+Start at the highest level. The whole system is made up of four blocks:
+
+```mermaid
+flowchart LR
+  A["📦 Package\nPublished npm package\n(CLI + templates + standards + Presets)"]
+  B["📁 Target Repo\nUser's project repository\n(doc tree + state files)"]
+  C["⚙️ Runtime\nRuntime engine\n(scan + check + generate)"]
+  D["👤 Human\nHuman review entry point\n(Dashboard + review confirmation)"]
+
+  A -->|"scaffold + validate"| B
+  B -->|"read"| C
+  C -->|"generate"| D
+  D -->|"review-confirm"| B
+```
+
+- **Package**: What you `npm install` — contains the CLI, templates, standards docs, and Preset packages
+- **Target Repo**: Your project, where harness creates a `coding-agent-harness/` tree to record task state
+- **Runtime**: The CLI runtime that scans the doc tree, validates compliance, and generates the Dashboard
+- **Human**: Views the Dashboard in a browser, performs review confirmation in the Workbench
+
+Note the direction of this loop: **Package writes to Target Repo, Runtime reads from Target Repo,
+Human writes back to Target Repo via review-confirm**.
+The whole system is a read-write loop centered on Markdown files, with no hidden state.
+
+---
+
+## Level 1 — What's inside each block
+
+### What's in the Package
+
+```mermaid
+flowchart TD
+  PKG["📦 Package\ncoding-agent-harness@npm"]
+
+  PKG --> CLI["harness CLI\ndist/harness.mjs\nSingle command entry point"]
+  PKG --> Lib["Core library\nscripts/lib/\n~30 modules, 6 functional layers"]
+  PKG --> Templates["Task templates\ntemplates/\nTask scaffold files (task_plan / visual_map etc.)"]
+  PKG --> References["Operating standards\nreferences/\nSpec docs that can be copied to target repos"]
+  PKG --> Presets["Preset packages\npresets/\nReusable task methods (legacy-migration / module etc.)"]
+```
+
+The Package is **read-only** — it provides tools and templates but stores no state.
+All state lives in the Target Repo.
+
+### What's in the Target Repo
+
+```mermaid
+flowchart TD
+  REPO["📁 Target Repo\nYour project repository"]
+
+  REPO --> Entry["AGENTS.md\nAgent entry point and routing\n(tells Agents where to find context)"]
+  REPO --> Manifest["coding-agent-harness/harness.yaml\nEnabled capabilities and structure"]
+  REPO --> Harness["coding-agent-harness/\nHarness workspace"]
+  REPO --> Docs["docs/\nReference docs and project standards"]
+
+  Harness --> Planning["planning/\nTask directory + module directory"]
+  Harness --> Ledger["governance/generated/Harness-Ledger.md\nGlobal ledger (all tasks summarized)"]
+  Harness --> Walkthrough["task-local walkthrough.md\nCloseout evidence"]
+  Docs --> Reference["coding-agent-harness/governance/standards/\nLocal operating standards (copied from Package)"]
+  Docs --> Governance["01-GOVERNANCE/\nLesson library"]
+```
+
+Each task corresponds to a directory under `coding-agent-harness/planning/tasks/<task-id>/`,
+containing files like `task_plan.md`, `progress.md`, `visual_map.md`, `review.md`, etc.
+
+### What the Runtime does
+
+```mermaid
+flowchart TD
+  RT["⚙️ Runtime\nharness CLI runtime"]
+
+  RT --> Scan["Scan\nRead all task files\nParse state / phases / review / Lessons"]
+  RT --> Check["Check\n9 validators verify compliance\nOutput failures / warnings"]
+  RT --> Generate["Generate\nOutput Dashboard HTML + JSON index\nOutput governance index tables"]
+  RT --> Migrate["Migrate\nAnalyze legacy project gaps\nGenerate migration action queue"]
+  RT --> Lifecycle["Lifecycle\nExecute state transition commands\nWrite Governance Sync"]
+```
+
+The Runtime is **stateless** — every run re-reads from Markdown files from scratch,
+caching no intermediate state (except file watching in `harness dev`).
+
+---
+
+## Level 2 — Core concept glossary
+
+| Concept | One-line explanation | Where |
+| --- | --- | --- |
+| **Task** | A unit of work with a lifecycle | `coding-agent-harness/planning/tasks/<id>/` |
+| **Budget** | Task complexity: `simple` / `standard` / `complex`, determines gate strictness | `task_plan.md` |
+| **Phase** | An execution phase in the Visual Map, with state and completion | `visual_map.md` |
+| **Capability** | Optional feature module, e.g. `dashboard`, `module-parallel` | `coding-agent-harness/harness.yaml` |
+| **Review Gate** | A review gate that blocks task completion, requires human confirmation to pass | `review.md` |
+| **Governance Sync** | Atomic operation that auto-updates the global ledger on task state changes | `coding-agent-harness/governance/generated/Harness-Ledger.md` |
+| **Preset** | A reusable task method package, e.g. `legacy-migration`, `module` | `presets/<id>/` |
+| **Lesson** | Reusable knowledge distilled from a task | `coding-agent-harness/governance/lessons/` |
+| **Tombstone** | Marker for soft-deleted / merged / superseded tasks | Special block in `task_plan.md` |
+| **lifecycleState** | Queue classification derived from task state + review state combined | Derived at runtime, not stored in files |
+
+---
+
+## Level 2 — Design decisions
+
+### Why Markdown instead of a database
+
+This is the most frequently asked question.
+
+**Reasons for choosing Markdown**:
+
+1. **Agent-readable**: All major AI coding tools can read and write Markdown without special APIs
+2. **Git-native**: State changes can be diffed, blamed, and rolled back — audit trail is built in
+3. **Human-readable**: State can be viewed directly without tools, reducing tool dependency
+4. **Works offline**: No external service dependency, works without network
+5. **Portable**: Switching Agent tools doesn't require data migration
+6. **Single source of truth**: Avoids drift between Markdown, JSON, and SQLite as three separate facts
+
+**Trade-offs**:
+
+- Parsing Markdown is slower than querying a database (but not a bottleneck at task management scale)
+- Format constraints must be maintained by validators rather than enforced by database schema
+- Concurrent writes need file locking (`governance-sync` lock mechanism)
+
+**Alternatives considered and rejected**:
+- **SQLite**: Not git-diff-friendly, introduces binary files, and current scale (hundreds of tasks) doesn't need it
+- **JSON**: Good for machine parsing but not for Agents understanding narrative context
+- **YAML/TOML**: Not suited for carrying long-form content like briefs and execution strategies
+
+### Why an npm package instead of SaaS
+
+Agents need to read and write state on the local filesystem. SaaS would introduce network
+dependency, authentication, and latency, breaking Agents' autonomous execution capability.
+An npm package lets any environment that can run Node.js use it directly, with no account
+or network required. `package.json` `dependencies` is empty — zero runtime dependencies.
+
+### Why review-confirm must be a manual operation
+
+`review-confirm` is the **only operation in the entire system that cannot be automatically
+executed by an Agent**.
+
+The reason:
+
+> Agents cannot approve their own work.
+
+This boundary wasn't there from the start. Initially the Dashboard workbench review action
+had no Agent/Human distinction. Later, through competitive analysis (Taskr competitive intake),
+"Agent auto-confirming review" was identified as a P0 risk, which led to introducing the
+Git commit gate: `review-confirm` writes human confirmation audit fields with Git
+`user.name` / `user.email` to the task `INDEX.md`, and makes two atomic Git commits:
+the first commits the confirmation fields, and the second commits the final audit
+record containing the first commit's SHA.
+This Git commit is an **auditable human signature** proving a real human reviewed the task.
+
+### Why derived state isn't stored in files
+
+`lifecycleState`, `taskQueues`, and `reviewQueueState` are derived state that gets
+recalculated on every run and never written back to Markdown files. Three reasons:
+
+1. **Avoid fact drift**: If derived state were also written to files, there would be two sources
+   of truth, and either one going stale would cause false reports
+2. **Prevent gate bypass**: If Agents could directly modify derived fields, they could bypass
+   the review-confirm gate
+3. **Governance rules as code**: The scanner's derivation rules are themselves a machine-readable
+   expression of governance rules — recalculating on every run is equivalent to re-executing
+   the governance check every time
+
+---
+
+## Next steps
+
+- Want to understand how the code is organized → [02-module-dependency.md](02-module-dependency.md)
+- Want to understand how a task flows from start to finish → [03-task-lifecycle.md](03-task-lifecycle.md)
+- Want to understand what the validators check → [04-check-and-governance.md](04-check-and-governance.md)
+- Want to understand where Dashboard data comes from → [05-data-flow.md](05-data-flow.md)
+- Want to understand how Presets and migration work → [06-preset-and-migration.md](06-preset-and-migration.md)

package/docs-release/architecture/system-explainer/en-US/02-module-dependency.md

@@ -0,0 +1,263 @@
+# 02 — Code Module Dependencies
+
+## Level 0 — Where's the entry point
+
+All commands come through a single file:
+
+```mermaid
+flowchart LR
+  User["User / Agent\n$ harness <command> [target]"] -->|"parse args + dispatch"| Entry["dist/harness.mjs\nSingle CLI entry point"]
+```
+
+`harness.mjs` does two things: parses command-line arguments, then dispatches to the
+corresponding command module or calls the core library directly.
+It contains no business logic itself.
+
+---
+
+## Level 1 — How commands are dispatched
+
+```mermaid
+flowchart TD
+  Entry["dist/harness.mjs"]
+
+  Entry -->|"dashboard\ndev"| DashCmd["dist/commands/dashboard-command.mjs\nDashboard generation + dynamic serving"]
+  Entry -->|"migrate-plan\nmigrate-run\nmigrate-verify"| MigCmd["dist/commands/migration-command.mjs\nMigration three-phase commands"]
+  Entry -->|"new-task / task-start\ntask-phase / task-review\ntask-complete / review-confirm\ntask-tombstone"| TaskCmd["dist/commands/task-command.mjs\nTask lifecycle commands"]
+  Entry -->|"preset catalog\npreset install\npreset uninstall"| PresetCmd["dist/commands/preset-command.mjs\nPreset management commands"]
+  Entry -->|"check / status / init\ngovernance / lesson-promote\n..."| Core["dist/lib/harness-core.mjs\n(called directly)"]
+```
+
+Four command modules each own one domain; other commands call `harness-core.mjs` directly.
+
+**Why this split**: Command modules handle commands with complex interaction logic
+(multi-step, reading/writing multiple files, user prompts), while simple query commands
+(`check`, `status`) are cleaner calling the core library directly.
+
+---
+
+## Level 2 — What is harness-core.mjs
+
+`harness-core.mjs` is a **facade** — it contains no business logic itself,
+it just re-exports everything from all modules under `lib/`.
+
+The benefit of this design: external code only needs to
+`import from "./lib/harness-core.mjs"` to get all functionality,
+without knowing which sub-module something lives in.
+
+```mermaid
+flowchart TD
+  Core["harness-core.mjs\n(pure re-export facade)"]
+
+  Core --> G1["① Core utilities layer\ncore-shared + markdown-utils"]
+  Core --> G2["② Task scanning layer\ntask-scanner + review-model + lesson-candidates"]
+  Core --> G3["③ Check and governance layer\ncheck-profiles + governance-sync + governance-index"]
+  Core --> G4["④ Dashboard layer\ndashboard-data + dashboard-writer + workbench"]
+  Core --> G5["⑤ Task lifecycle layer\ntask-lifecycle + review-gates + review-confirm"]
+  Core --> G6["⑥ Migration and Preset layer\nmigration-planner + preset-registry + tombstone"]
+```
+
+Let's expand each layer.
+
+---
+
+## Level 3 — Six functional layers in detail
+
+### ① Core utilities layer
+
+These two modules are the foundation for all other modules — almost every module imports them:
+
+```mermaid
+flowchart LR
+  CoreShared["core-shared.mjs\nPath resolution / constant enums\nFile read/write / locale handling\nTemplate rendering"]
+  MarkdownUtils["markdown-utils.mjs\nMarkdown table extraction\nRow updates / column lookup\nDependency list splitting"]
+```
+
+`core-shared` defines all allowed enum values — it's the "type system" for the whole system:
+
+| Enum | Allowed values |
+| --- | --- |
+| `allowedTaskStates` | `not_started / planned / in_progress / review / blocked / done` |
+| `allowedTaskBudgets` | `simple / standard / complex` |
+| `allowedPhaseStates` | `planned / in_progress / review / blocked / done / skipped` |
+| `allowedCapabilities` | `core / module-parallel / subagent-worker / adversarial-review / ...` |
+
+`markdown-utils` provides structured operations on Markdown tables — this is the technical
+foundation that lets the whole system derive state from Markdown files.
+
+---
+
+### ② Task scanning layer
+
+Responsible for reading all files under `coding-agent-harness/planning/tasks/` and parsing them into
+structured data:
+
+```mermaid
+flowchart TD
+  TaskScanner["task-scanner.mjs\nScans all task directories\nParses state / budget / phases / metadata"]
+
+  TaskScanner --> ReviewModel["task-review-model.mjs\nReview confirmation parsing\nLifecycle queue derivation\nTombstone parsing"]
+  TaskScanner --> LessonCandidates["task-lesson-candidates.mjs\nLesson candidate status parsing\nDecision completion determination"]
+
+  ReviewModel --> CoreShared
+  ReviewModel --> MarkdownUtils
+  TaskScanner --> CoreShared
+  TaskScanner --> MarkdownUtils
+```
+
+`task-review-model` contains several key **derivation functions** — they don't read files,
+they compute new state from already-parsed data:
+
+| Function | Input | Output |
+| --- | --- | --- |
+| `deriveLifecycleState()` | taskState + reviewStatus + tombstone | `lifecycleState` (queue classification) |
+| `deriveTaskQueues()` | lifecycleState + materials + lessons | `taskQueues[]` (which queues it belongs to) |
+| `deriveReviewQueueState()` | findings + confirmation | `reviewQueueState` |
+| `parseTaskTombstone()` | task_plan.md content | soft-delete / merge / superseded state |
+
+These derivation functions are **pure functions** — same input always produces same output,
+making them easy to test and debug.
+
+---
+
+### ③ Check and governance layer
+
+Responsible for validating compliance and maintaining atomic writes to global indexes:
+
+```mermaid
+flowchart TD
+  CheckProfiles["check-profiles.mjs\nbuildStatus() orchestrates 9 validators\nReturns failures + warnings + tasks"]
+
+  CheckProfiles --> V1["validateCapabilities\nCapability registry consistency"]
+  CheckProfiles --> V2["validateReviewSchema\nreview.md structure"]
+  CheckProfiles --> V3["validateVisualMaps\nvisual_map compliance"]
+  CheckProfiles --> V4["validatePlanContracts\nTask contract markers"]
+  CheckProfiles --> V5["validateTaskPresetContracts\nPreset contracts"]
+  CheckProfiles --> V6["validateContextDocs\nContext doc completeness"]
+  CheckProfiles --> V7["validateGovernanceTableBoundaries\nTable boundaries"]
+  CheckProfiles --> V8["validateSubagentAuthorization\nSubagent authorization"]
+  CheckProfiles --> V9["validateTaskCompletionConsistency\nCompletion consistency"]
+
+  CheckProfiles --> GitSummary["git-status-summary.mjs\nGit status summary (dirty files etc.)"]
+
+  GovSync["governance-sync.mjs\nAtomic lock + row-level update + Git commit\n(auto-called on task state changes)"]
+  GovIndex["governance-index-generator.mjs\nRebuilds global index tables\n(manually triggered)"]
+  GovIndex --> GovSync
+```
+
+**Important distinction**: `governance-sync` and `check-profiles` have no dependency on each other.
+- `check-profiles`: read-only, validates state, writes no files
+- `governance-sync`: write-only, updates the ledger, does no validation
+
+---
+
+### ④ Dashboard layer
+
+Responsible for converting scan results into an HTML Dashboard:
+
+```mermaid
+flowchart TD
+  DashData["dashboard-data.mjs\nbuildDashboardBundle()\nCollects status + documents + tables + graph + adoption"]
+
+  DashData --> CheckProfiles["check-profiles.mjs\n(calls buildStatus)"]
+  DashData --> DashWriter["dashboard-writer.mjs\nWrites HTML + JSON files\n(static snapshot mode)"]
+  DashData --> StatusRenderer["status-dashboard-renderer.mjs\nRenders status summary text"]
+
+  DashWorkbench["dashboard-workbench.mjs\nDev dynamic serving\nHTTP server + file watching + auto-refresh\n(harness dev command)"]
+```
+
+`DashWorkbench` and `DashData` / `DashWriter` are **independent**:
+- `DashData` + `DashWriter`: generates static snapshots (read-only)
+- `DashWorkbench`: starts a local HTTP server, supports Workbench write operations
+
+---
+
+### ⑤ Task lifecycle layer
+
+Responsible for executing all task state transition commands:
+
+```mermaid
+flowchart TD
+  TaskLifecycle["task-lifecycle.mjs\nLifecycle command implementations\nnew-task / task-start / task-phase\ntask-review / task-complete"]
+
+  TaskLifecycle --> ReviewGates["task-lifecycle/review-gates.mjs\nGate validation logic\n(checks before entering review)"]
+  TaskLifecycle --> ReviewConfirm["task-lifecycle/review-confirm.mjs\nHuman confirmation execution\n(review-confirm command)"]
+  TaskLifecycle --> TextUtils["task-lifecycle/text-utils.mjs\nText append utilities\n(appending content to Markdown files)"]
+  TaskLifecycle --> GovSync["governance-sync.mjs\nSync ledger on state changes"]
+  TaskLifecycle --> MigPreset["task-migration-preset.mjs\nMigration Preset context injection"]
+
+  ReviewConfirm --> GitGate["review-confirm-git-gate.mjs\nGit atomic commit gate\n(writes human confirmation block + commit)"]
+```
+
+`review-confirm` is the most special command in the entire lifecycle layer — it's the only
+operation that requires a Git atomic commit, and the only one that cannot be automatically
+executed by an Agent (see design decisions in [01-system-overview.md](01-system-overview.md)).
+
+---
+
+### ⑥ Migration and Preset layer
+
+```mermaid
+flowchart TD
+  PresetReg["preset-registry.mjs\nReads presets/ YAML\nValidates package completeness\nLayered discovery (project / user / bundled)"]
+  PresetEngine["preset-engine.mjs\nExecutes Preset entrypoints\n(template / script / check types)"]
+  PresetAudit["preset-audit-contracts.mjs\nValidates Preset contract completeness"]
+  PresetResource["preset-resource-contracts.mjs\nValidates Preset resource declarations"]
+
+  MigPlanner["migration-planner.mjs\nAnalyzes target repo gaps\nGenerates migration action queue"]
+  MigSupport["migration-support.mjs\nSession management / locale detection\nGit status check / full-cutover verification"]
+  Tombstone["task-tombstone-commands.mjs\nSoft-delete / merge / reopen commands"]
+
+  LessonSed["task-lesson-sedimentation.mjs\nLesson sedimentation task creation"]
+  LessonMaint["lesson-maintenance.mjs\nLesson library maintenance"]
+  TaskIndex["task-index.mjs\nTask index generation"]
+
+  MigPlanner --> MigSupport
+  PresetEngine --> PresetReg
+```
+
+---
+
+## Complete dependency map (reference)
+
+If you've understood the layering above, this diagram serves as a lookup index:
+
+```mermaid
+flowchart TD
+  Entry["harness.mjs"] --> DashCmd & MigCmd & TaskCmd & PresetCmd & Core["harness-core.mjs"]
+
+  Core --> CoreShared & MarkdownUtils
+  Core --> TaskScanner --> ReviewModel & LessonCandidates
+  Core --> CheckProfiles --> GitSummary
+  Core --> GovSync
+  Core --> GovIndex --> GovSync
+  Core --> DashData --> DashWriter & StatusRenderer
+  Core --> DashWorkbench
+  Core --> TaskLifecycle --> ReviewGates & ReviewConfirm & TextUtils & GovSync & MigPreset
+  ReviewConfirm --> GitGate
+  Core --> PresetReg
+  Core --> PresetEngine --> PresetReg
+  Core --> MigPlanner --> MigSupport
+  Core --> Tombstone
+  Core --> LessonSed
+  Core --> LessonMaint
+  Core --> TaskIndex
+```
+
+---
+
+## Level 2 — Module naming patterns
+
+Understanding naming patterns helps you locate code quickly:
+
+| Prefix / suffix | Meaning | Examples |
+| --- | --- | --- |
+| `task-` | Task-related | `task-scanner`, `task-lifecycle`, `task-review-model` |
+| `dashboard-` | Dashboard-related | `dashboard-data`, `dashboard-writer`, `dashboard-workbench` |
+| `governance-` | Governance / ledger-related | `governance-sync`, `governance-index-generator` |
+| `migration-` | Migration-related | `migration-planner`, `migration-support` |
+| `preset-` | Preset-related | `preset-registry`, `preset-engine`, `preset-audit-contracts` |
+| `check-` | Validators | `check-profiles`, `check-module-parallel` |
+| `-command.mjs` | CLI command modules | `task-command`, `dashboard-command` |
+| `-utils.mjs` | Utility functions | `markdown-utils`, `text-utils` |
+| `-gates.mjs` | Gate logic | `review-gates`, `review-confirm-git-gate` |