@harness-engineering/cli 1.25.7 → 1.26.1

package/dist/agents/skills/codex/harness-knowledge-pipeline/SKILL.md

@@ -0,0 +1,277 @@
+# Harness Knowledge Pipeline
+
+> 4-phase knowledge extraction, reconciliation, drift detection, and remediation with convergence loop. Keeps the business knowledge graph current and identifies coverage gaps.
+
+## When to Use
+
+- When you want a full knowledge health check: extract signals, detect drift, find gaps
+- After adding new code, connectors, or diagram files that contain business knowledge
+- As a periodic hygiene check to ensure the knowledge graph reflects the codebase
+- When `--drift-check` is needed in CI to gate on knowledge freshness
+- NOT for manually authoring knowledge files (write to `docs/knowledge/` directly)
+- NOT for ingesting a single source (use `ingest_source` MCP tool directly)
+- NOT for graph queries or exploration (use `ask_graph` or `query_graph`)
+
+## Process
+
+1. **EXTRACT:** Run code signal extractors, diagram parsers, BusinessKnowledgeIngestor, and KnowledgeLinker against the project
+2. **RECONCILE:** Build pre-extraction snapshot (current graph state) and post-extraction snapshot, then run StructuralDriftDetector
+3. **DETECT:** Partition findings by classification (new/stale/drifted/contradicting), generate gap report
+4. **REMEDIATE (only with `--fix`):** Apply safe remediations, convergence loop up to 5 iterations
+
+## Iron Law
+
+**Auto-remediate stale findings only. Prompt for drifted and contradicting.** Never auto-resolve a `contradicting` finding — the human must decide which source is authoritative.
+
+## Flags
+
+| Flag              | Effect                                                           |
+| ----------------- | ---------------------------------------------------------------- |
+| `--fix`           | Enable convergence-based auto-remediation (default: detect-only) |
+| `--ci`            | Non-interactive: apply safe fixes only, report everything else   |
+| `--domain <name>` | Limit pipeline to specific knowledge domain                      |
+
+## Shared Context Object
+
+All phases read from and write to a shared `KnowledgePipelineContext`:
+
+```typescript
+interface KnowledgePipelineContext {
+  extractionSnapshot: KnowledgeSnapshot;
+  currentSnapshot: KnowledgeSnapshot;
+  driftResult: DriftResult;
+  gapReport: GapReport;
+  remediationsApplied: string[];
+  iteration: number;
+  verdict: 'pass' | 'warn' | 'fail';
+}
+```
+
+---
+
+## Phase 1: EXTRACT
+
+Extract business knowledge from all available sources into a fresh snapshot.
+
+1. **Code Signal Extractors:** Run `ExtractionRunner` against the project directory.
+   - Test description extractor
+   - Enum/constant extractor
+   - Validation rule extractor
+   - API path extractor
+   - Output goes to `.harness/knowledge/extracted/`
+
+2. **Diagram Parsers:** Run `DiagramParser.ingest()` for diagram-as-code files.
+   - Scans for `.mmd`, `.mermaid`, `.d2`, `.puml`, `.plantuml` files
+   - Extracts entities and relationships into `business_concept` nodes
+
+3. **Connector Sync** (if configured): Trigger connector sync for Jira, Slack, Confluence.
+   - Only runs if connectors are configured in `harness.config.json`
+   - Skip silently if no connectors
+
+4. **Build Fresh Snapshot:** Collect all extraction results into a `KnowledgeSnapshot`.
+   - Each entry has: `id`, `type`, `contentHash`, `source`, `name`
+
+**Output:** `context.extractionSnapshot` populated.
+
+---
+
+## Phase 2: RECONCILE
+
+Compare the fresh extraction against the current graph state.
+
+1. **Build Current Snapshot:** Read all `business_*` nodes from GraphStore.
+   - Map each node to a `KnowledgeSnapshotEntry`
+   - Use node ID as entry ID, compute contentHash from node content
+
+2. **Run Drift Detection:** Call `StructuralDriftDetector.detect(current, fresh)`.
+   - Produces `DriftResult` with findings classified as `new`, `drifted`, `stale`, `contradicting`
+   - Produces `driftScore` (0.0 - 1.0)
+
+**Output:** `context.currentSnapshot` and `context.driftResult` populated.
+
+---
+
+## Phase 3: DETECT
+
+Partition drift findings and classify safety.
+
+1. **Partition by classification:**
+
+   | Classification  | Severity | Safety        | Action                    |
+   | --------------- | -------- | ------------- | ------------------------- |
+   | `new`           | low      | safe          | Stage automatically       |
+   | `stale`         | high     | probably-safe | Auto-remove (source gone) |
+   | `drifted`       | medium   | probably-safe | Prompt for confirmation   |
+   | `contradicting` | critical | unsafe        | Never auto-resolve        |
+
+2. **Generate Drift Report:** Present findings grouped by classification:
+
+   ```
+   KNOWLEDGE PIPELINE -- Drift Detection
+
+   Drift Score: 0.23 (12 findings / 52 entries)
+
+   NEW (4 findings, safe):
+     - business_rule:payments:refund-window [extractor]
+     - business_concept:auth:session-timeout [diagram]
+     ...
+
+   STALE (3 findings, probably-safe):
+     - business_fact:billing:old-rate-limit [extractor] — source file deleted
+     ...
+
+   DRIFTED (4 findings, requires confirmation):
+     - business_rule:checkout:tax-calculation [extractor] — content changed
+     ...
+
+   CONTRADICTING (1 finding, requires human decision):
+     - business_term:Settlement SLA — extractor says 24h, linker says 48h
+   ```
+
+3. **Generate Gap Report:** Run `KnowledgeStagingAggregator.generateGapReport()`.
+   - Lists domains with entry counts
+   - Write to `.harness/knowledge/gaps.md`
+
+**Output:** Drift report and gap report written. `context.gapReport` populated.
+
+---
+
+## Phase 4: REMEDIATE (only with `--fix`)
+
+Apply safe fixes and converge.
+
+1. **Stage `new` findings:** Write to `.harness/knowledge/staged/pipeline-staged.jsonl` via `KnowledgeStagingAggregator`.
+
+2. **Auto-remove `stale` findings:** Remove nodes from graph where source file no longer exists.
+   - Log each removal
+
+3. **Present `drifted` findings:** Ask the human to confirm each update.
+   - In `--ci` mode: skip, report as unresolved
+   - If confirmed: update the graph node with fresh content
+
+4. **Surface `contradicting` findings:** Never auto-resolve.
+   - Present both sources and ask the human which is authoritative
+   - In `--ci` mode: report as unresolved
+
+5. **Convergence Check:** After remediation:
+   - Re-run EXTRACT + RECONCILE + DETECT
+   - Count remaining findings
+   - If `newCount >= previousCount`: STOP (converged or no progress)
+   - If `newCount < previousCount`: continue (max 5 iterations)
+
+### Convergence Loop
+
+```
+maxIterations = 5
+previousCount = findings.length
+
+while iteration < maxIterations:
+  1. Apply safe remediations (new → stage, stale → remove)
+  2. Present probably-safe remediations (skip in --ci)
+  3. Log unsafe findings (never auto-apply)
+  4. Re-run EXTRACT + RECONCILE + DETECT
+  5. newCount = remaining findings
+  6. if newCount >= previousCount: STOP (converged)
+  7. previousCount = newCount; iteration++
+```
+
+---
+
+## Verdict Rules
+
+After all phases complete, compute verdict:
+
+| Verdict | Condition                                         |
+| ------- | ------------------------------------------------- |
+| `pass`  | Zero unresolved findings after pipeline           |
+| `warn`  | Only `new` findings remain (low severity)         |
+| `fail`  | Any `contradicting`, `stale`, or `drifted` remain |
+
+Present verdict summary:
+
+```
+KNOWLEDGE PIPELINE -- Verdict: WARN
+
+Findings: 2 new (staged), 0 stale, 0 drifted, 0 contradicting
+Gap Report: 3 domains, 12 total entries
+Drift Score: 0.04
+```
+
+## Harness Integration
+
+- **`ingest_source`** -- MCP tool for ingesting diagram sources (`source: 'diagrams'`)
+- **`ExtractionRunner`** -- Code signal extraction from `@harness-engineering/graph`
+- **`DiagramParser`** -- Diagram-as-code parsing from `@harness-engineering/graph`
+- **`StructuralDriftDetector`** -- Drift detection from `@harness-engineering/graph`
+- **`KnowledgeStagingAggregator`** -- Staging and gap reporting from `@harness-engineering/graph`
+- **`harness validate`** -- Run after remediation to verify project health
+
+## Success Criteria
+
+1. All 4 phases execute in sequence (EXTRACT, RECONCILE, DETECT, REMEDIATE)
+2. Drift detection correctly classifies findings into 4 categories
+3. Auto-remediation only applies to safe/probably-safe findings
+4. Contradicting findings are never auto-resolved
+5. Convergence loop terminates when finding count stops decreasing (max 5 iterations)
+6. Gap report lists domains with entry counts
+7. Verdict accurately reflects remaining finding state
+8. `--ci` mode is fully non-interactive
+
+## Examples
+
+### Example: Running detect-only pipeline
+
+```
+$ harness knowledge-pipeline
+
+KNOWLEDGE PIPELINE -- Verdict: WARN
+
+  Drift Score: 0.12
+  Findings: 3 new, 0 stale, 0 drifted, 0 contradicting
+  Extraction: 12 code signals, 5 diagrams, 2 linker facts, 4 business knowledge
+  Gaps: 2 domains, 16 total entries
+```
+
+### Example: Running with fix mode
+
+```
+$ harness knowledge-pipeline --fix
+
+KNOWLEDGE PIPELINE -- Verdict: PASS
+
+  Drift Score: 0.00
+  Findings: 0 new, 0 stale, 0 drifted, 0 contradicting
+  Extraction: 12 code signals, 5 diagrams, 2 linker facts, 4 business knowledge
+  Gaps: 2 domains, 19 total entries
+  Convergence: 2 iterations
+  Remediations: 3 applied
+```
+
+### Example: CI gate mode
+
+```
+$ harness knowledge-pipeline --ci --drift-check
+# Exits 1 if unresolved drift exists
+```
+
+## Rationalizations to Reject
+
+| Rationalization                                                                      | Reality                                                                                                                                             |
+| ------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------- |
+| "The contradicting findings have an obvious resolution, so I can auto-fix them"      | The Iron Law: contradicting findings require human decision. Different sources may have different authority levels that only the human understands. |
+| "Drift score is low (0.05), so I can skip the DETECT phase and go straight to PASS"  | Every finding must be classified regardless of drift score. A low score with one critical contradiction is still a FAIL.                            |
+| "The stale entries might come back if the source is restored, so I should keep them" | Stale means the source is gone now. If it comes back, the next extraction will find it as NEW. Keeping stale entries pollutes the graph.            |
+| "I can run the convergence loop more than 5 times to get to zero findings"           | The 5-iteration cap is a safety bound. If findings aren't decreasing, more iterations won't help — the remaining issues need human intervention.    |
+
+## Gates
+
+- **No auto-resolution of contradicting findings.** EVER. Present both sources, ask the human.
+- **No skipping DETECT.** Even in `--fix` mode, DETECT must run before REMEDIATE.
+- **No convergence beyond 5 iterations.** If findings aren't decreasing, stop and report.
+- **No REMEDIATE without `--fix`.** Default mode is detect-and-report only.
+
+## Escalation
+
+- **When no graph is available:** Run extractors and diagram parsers only. Skip reconciliation (no current snapshot). Report extraction results as gap analysis.
+- **When connectors fail:** Log the error, continue with available sources. Note which connectors were skipped in the report.
+- **When convergence stalls at iteration 1:** All remaining findings require human intervention. Present them and stop.

package/dist/agents/skills/codex/harness-knowledge-pipeline/skill.yaml

@@ -0,0 +1,60 @@
+name: harness-knowledge-pipeline
+version: "1.0.0"
+description: 4-phase knowledge extraction, reconciliation, drift detection, and remediation with convergence loop
+stability: static
+cognitive_mode: constructive-architect
+triggers:
+  - manual
+platforms:
+  - claude-code
+  - gemini-cli
+  - cursor
+  - codex
+tools:
+  - Bash
+  - Read
+  - Write
+  - Edit
+  - Glob
+  - Grep
+cli:
+  command: harness skill run harness-knowledge-pipeline
+  args:
+    - name: path
+      description: Project root path
+      required: false
+    - name: fix
+      description: Enable auto-remediation of safe findings (default detect-only)
+      required: false
+    - name: ci
+      description: Non-interactive mode, skip probably-safe fixes
+      required: false
+    - name: domain
+      description: Limit pipeline to specific knowledge domain
+      required: false
+mcp:
+  tool: run_skill
+  input:
+    skill: harness-knowledge-pipeline
+    path: string
+type: rigid
+tier: 2
+phases:
+  - name: extract
+    description: Run code signal extractors, diagram parsers, and connector sync
+    required: true
+  - name: reconcile
+    description: Build snapshots and run structural drift detection
+    required: true
+  - name: detect
+    description: Partition findings by severity and safety classification
+    required: true
+  - name: remediate
+    description: Apply safe fixes, prompt for unsafe, convergence loop
+    required: false
+state:
+  persistent: false
+  files:
+    - .harness/knowledge/staged/pipeline-staged.jsonl
+    - .harness/knowledge/gaps.md
+depends_on: []

package/dist/agents/skills/codex/harness-planning/SKILL.md

@@ -54,6 +54,21 @@ When no arguments are provided (standalone invocation), discover spec from conte
 Work backward from the goal. Start with "what must be true when we are done?"
 
 1. **State the goal.** One sentence. What does the system do when this plan is complete?
+
+1b. **Load skill recommendations.** After loading the spec, check for skill recommendations:
+
+- If `docs/changes/<feature>/SKILLS.md` exists alongside the spec: parse the Apply and Reference tiers. These inform task annotation in Phase 2.
+- If SKILLS.md is missing but a spec exists: run the advisor inline using `advise_skills` MCP tool to generate SKILLS.md.
+- If neither SKILLS.md nor a spec exists: emit a one-line note:
+
+  ```
+  Note: No skill recommendations found. Run the advisor to discover
+  relevant design, framework, and knowledge skills:
+    harness advise-skills --spec-path <path>
+  ```
+
+Store the parsed skill list for use in Phase 2 task annotation.
+
 2. **Derive observable truths.** What can be observed (running a command, opening a browser, reading a file) that proves the goal is met? Be specific:
    - BAD: "The API handles errors"
    - GOOD: "GET /api/users/nonexistent returns 404 with `{ error: 'User not found' }` body"
@@ -190,7 +205,16 @@ Report progress: `**[Phase 2/4]** DECOMPOSE — mapping file structure and creat
    - **Exact commit message**
    - **`harness validate`** as the final step
 
-5. **Include checkpoints.** Mark tasks requiring human input:
+5. **Skill annotations.** If skill recommendations were loaded in Phase 1, annotate each task with relevant skills from the Apply and Reference tiers:
+
+   ```
+   ### Task 3: Implement dark mode toggle
+   **Skills:** `design-dark-mode` (apply), `a11y-color-contrast` (reference)
+   ```
+
+   Match skills to tasks based on keyword and domain overlap between the task description and the skill's purpose/keywords. Only annotate when the match is relevant to the specific task.
+
+6. **Include checkpoints.** Mark tasks requiring human input:
    - `[checkpoint:human-verify]` — Pause, show result, wait for confirmation
    - `[checkpoint:decision]` — Pause, present options, wait for choice
    - `[checkpoint:human-action]` — Pause, instruct human on required action

package/dist/agents/skills/codex/initialize-harness-project/SKILL.md

@@ -29,6 +29,17 @@
 
 4. **Recommend the target adoption level.** For new projects, start with basic unless the team has harness experience. For existing projects, recommend one level up from current. Present the recommendation and wait for confirmation.
 
+5. **Classify the project shape — product/service or test-suite?** Check these signals; if any match, the project is a test suite and the rest of this skill's flow changes:
+   - Repo or package name matches `*test*`, `*-e2e*`, `*-qa*`, `*-automation*`
+   - `package.json` has `@playwright/test`, `cypress`, `webdriverio`, `mocha`, or `testcafe` as a direct dep
+   - Top-level `tests/`, `e2e/`, `specs/`, or `playwright/` directories are the primary source tree
+   - Config files like `playwright.config.*`, `cypress.config.*`, `wdio.conf.*`
+   - No production runtime — build output consumed only by other test repos (shared library)
+
+   **If a test suite:** complete Phase 2 (scaffolding) here, then dispatch to `initialize-test-suite-project` for Phase 3 configuration and Phase 4 verification, then return here for Phase 4 step 4+ (knowledge graph, roadmap nudge, final commit). The test-suite skill owns archetype selection, layer variants, tags, reporters, and the custom report.
+
+   **If a product/service:** continue with the rest of this skill as written.
+
 ### Phase 2: SCAFFOLD — Generate Project Structure
 
 1. **Run `harness init` with the appropriate flags:**
@@ -72,6 +83,8 @@
    - **No:** Set `i18n.enabled: false` in `harness.config.json`. The `harness-i18n-process` skill will still fire gentle prompts for unconfigured projects when features touch user-facing strings.
    - **Not sure yet:** Skip i18n configuration entirely. Do not set `i18n.enabled`. The project can enable i18n later by running `harness-i18n-workflow` directly.
 
+6. **Test-suite projects only — dispatch to `initialize-test-suite-project`.** If Phase 1 step 5 classified this as a test suite, invoke `initialize-test-suite-project` now and let it own archetype selection, shared-library decision, layer variants (A self-contained vs B consumer), ESLint flat-config fix, tag taxonomy, reporter stack, custom report, and the "prove the guards fire" verification. Return here for Phase 4 step 4+ (knowledge graph, roadmap, commit). Product and service projects skip this step entirely.
+
 ### Phase 4: VALIDATE — Confirm Everything Works
 
 1. **Run `harness validate`** to verify the full configuration. This checks:
@@ -107,6 +120,7 @@ This creates the `.harness/graph/` directory and populates it with the project's
 - **`harness validate`** — Verify the full project configuration is valid and complete.
 - **`harness check-deps`** — Verify dependency constraints match the actual codebase (intermediate and above).
 - **`harness-i18n-workflow configure` + `harness-i18n-workflow scaffold`** — Invoked during Phase 3 if the project will support multiple languages. Sets up i18n configuration and translation file structure.
+- **`initialize-test-suite-project`** — Sub-skill. Invoked during Phase 3 step 6 when Phase 1 step 5 classified the project as a test suite. Owns archetype selection, shared-library vs in-repo decision, layer variants, tag taxonomy, reporter stack, custom report, and "prove the guards fire" verification.
 - **Roadmap nudge** — After successful initialization, inform the user about `/harness:roadmap --create` for setting up project-level feature tracking. Informational only; does not create the roadmap.
 
 ## Success Criteria
@@ -120,15 +134,17 @@ This creates the `.harness/graph/` directory and populates it with the project's
 - The adoption level matches what was agreed upon with the human
 - All generated files are committed in a single atomic commit
 - i18n configuration is set if the human chose to enable it during init
+- For test suites: `initialize-test-suite-project` ran to completion and its Success Criteria are also met
 
 ## Rationalizations to Reject
 
-| Rationalization                                                          | Why It Is Wrong                                                                                                                        |
-| ------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------- |
-| "The generated AGENTS.md template looks fine -- no need to customize it" | Phase 3 says do not blindly accept generated content. Without project-specific descriptions, agents receive generic instructions.      |
-| "We should start at the advanced level since we want full coverage"      | The skill recommends basic for new projects. Each level builds on the previous. Jumping to advanced creates misconfigured rules.       |
-| "I will skip the i18n question to keep setup fast"                       | Phase 3 requires asking about i18n and recording the decision. Skipping creates ambiguity about whether the omission was intentional.  |
-| "Validation passed, so the project is ready"                             | Phase 4 includes harness check-deps for intermediate+ projects and knowledge graph initialization. Validation alone is not sufficient. |
+| Rationalization                                                          | Why It Is Wrong                                                                                                                                                                                                           |
+| ------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| "The generated AGENTS.md template looks fine -- no need to customize it" | Phase 3 says do not blindly accept generated content. Without project-specific descriptions, agents receive generic instructions.                                                                                         |
+| "We should start at the advanced level since we want full coverage"      | The skill recommends basic for new projects. Each level builds on the previous. Jumping to advanced creates misconfigured rules.                                                                                          |
+| "I will skip the i18n question to keep setup fast"                       | Phase 3 requires asking about i18n and recording the decision. Skipping creates ambiguity about whether the omission was intentional.                                                                                     |
+| "Validation passed, so the project is ready"                             | Phase 4 includes harness check-deps for intermediate+ projects and knowledge graph initialization. Validation alone is not sufficient.                                                                                    |
+| "This is a test suite, we'll configure layers in this skill"             | Phase 3 step 6 dispatches to `initialize-test-suite-project` for archetype selection, layer variants, and the rest. Do not inline test-suite-specific configuration here — the sub-skill owns it and carries the gotchas. |
 
 ## Examples
 

package/dist/agents/skills/codex/initialize-harness-project/skill.yaml

@@ -32,4 +32,5 @@ tier: 1
 state:
   persistent: false
   files: []
-depends_on: []
+depends_on:
+  - initialize-test-suite-project