@hallucination-studio/harness-engine 1.0.0 → 1.0.1

package/skills/harness-engine/references/workflow.md

@@ -0,0 +1,62 @@
+# Workflow
+
+Use this skill in two passes.
+
+## Pass 1: Analyze and Confirm
+
+Run `analyze` before editing repository docs.
+
+Ask the human only about facts that cannot be derived safely from the repo, especially:
+
+- product domain and top-level outcomes
+- intended users or operators
+- production reliability expectations
+- security or compliance constraints
+- frontend experience bar
+- canonical external references worth pinning inside `docs/references/`
+
+Do not ask for facts that can be inferred from source layout, dependency manifests, or existing docs.
+
+Also inspect the analysis for:
+
+- missing durable knowledge that should be written during the task
+- missing execution-plan state
+- which SOPs should be referenced in the generated router docs
+
+## Pass 2: Init
+
+Run `sample-answers`, fill the answers, then run `init`.
+
+Use `init` for both first-time adoption and managed-harness reconciliation. It creates a new harness when none exists, and refreshes managed harness files plus backfills newly introduced managed files when an existing managed harness is detected. Unmanaged user files are preserved unless `--force` is explicitly used.
+
+After the script runs, read the generated docs once and tighten weak generic phrases before handing off.
+
+## Ongoing Use
+
+After the scaffold exists:
+
+- treat harness commands as Codex's execution interface, not as steps the user must manually run
+- translate user intent like "complete this", "continue this later", "pause until X", "stop this", or "defer this" into the matching continuation decision yourself
+- read `docs/exec-plans/workstreams.md` before resuming interrupted or long-running work
+- create or reuse an execution plan before any repository-mutating work, including code, docs, configuration, tests, dependencies, build/release scripts, generated templates, runtime behavior, migrations, cleanup, or review fixes
+- use `plan-start` instead of creating plan files manually when possible
+- use `acceptance-set` before implementation so the active plan has a ready, task-specific Acceptance Contract
+- log durable facts during execution instead of waiting until the end
+- follow the matching SOP for architecture, UI, observability, or knowledge capture work
+- route repository-mutating requests through Harness Task Intake in `AGENTS.md`; route product, frontend, backend, architecture, data/state, security, performance, and reliability issues through the Issue Workflows branch when the request is a bug or regression
+- encode durable knowledge back into the repository before closing the task
+- mark logged knowledge items as written after updating the permanent docs; the `knowledge-mark-written` evidence must be exact text already present in the destination doc, not a paraphrase
+- log every defect found by tests, evals, browser validation, or code review with `defect-log`
+- resolve logged defects only after fixing the implementation and citing passing validation with `defect-resolve`
+- run `quality-score` after implementation and validation, with evidence notes for every dimension tied to the ready Acceptance Contract
+- if `quality-score` fails, implement the `## Rework Required` items and score again
+- use `continuation-set` before every `plan-close`; `continue` and `pause` update the workstream ledger automatically after required fields validate
+- do not ask the user to invoke `continuation-set`, `plan-close`, or `check`; run them and summarize blocked reasons or successful state changes
+- use `clean` when local skill installs or generated evidence need cleanup or were already committed; review dry-run output first, then apply, commit, and push the staged removals
+- use `plan-close` to verify no durable knowledge is left stranded in the active plan
+- before `plan-close`, replace generic plan placeholders with task-specific scope, constraints, steps, validation, and completion notes; delete unused ad hoc durable-knowledge TODOs
+- run the installed manager `check` command before handoff; active plans require ready Acceptance Contracts, while completed plans require passing Quality Results
+- preview stale generated evidence with `evidence-prune` when `docs/generated/` contains old screenshots, DOM dumps, layout summaries, or smoke outputs; review the dry-run output before using `--apply`
+- do not add CI to the target repository unless the human explicitly asks for it
+
+No-plan exceptions are limited to pure question answering, read-only investigation, showing command output, or status reporting with no file changes. If files will change, enter the plan lifecycle first.

package/skills/harness-engine/scripts/harness_engine/__init__.py

@@ -0,0 +1 @@
+"""Harness Engine Python runtime package."""

package/skills/harness-engine/scripts/harness_engine/analysis.py

@@ -0,0 +1,240 @@
+from .common import *
+from .templates import is_harness_owned_text
+
+def detect_languages(files):
+    ext_map = {}
+    for file_path in files:
+        suffix = Path(file_path).suffix.lower()
+        if suffix:
+            ext_map[suffix] = ext_map.get(suffix, 0) + 1
+    mapping = {
+        ".js": "JavaScript",
+        ".jsx": "JavaScript",
+        ".ts": "TypeScript",
+        ".tsx": "TypeScript",
+        ".sh": "Shell",
+        ".bash": "Shell",
+        ".zsh": "Shell",
+        ".py": "Python",
+        ".rb": "Ruby",
+        ".go": "Go",
+        ".rs": "Rust",
+        ".java": "Java",
+        ".kt": "Kotlin",
+        ".swift": "Swift",
+    }
+    languages = []
+    for ext, language in mapping.items():
+        if ext in ext_map and language not in languages:
+            languages.append(language)
+    return languages
+
+
+def read_json_if_exists(path):
+    if not path.exists():
+        return None
+    try:
+        return json.loads(path.read_text())
+    except json.JSONDecodeError:
+        return None
+
+
+def detect_frameworks(repo):
+    frameworks = []
+    package_json = read_json_if_exists(repo / "package.json")
+    if package_json:
+        deps = {}
+        deps.update(package_json.get("dependencies", {}))
+        deps.update(package_json.get("devDependencies", {}))
+        dep_names = set(deps.keys())
+        known = {
+            "next": "Next.js",
+            "react": "React",
+            "vue": "Vue",
+            "svelte": "Svelte",
+            "vite": "Vite",
+            "express": "Express",
+            "nestjs": "NestJS",
+        }
+        for key, label in known.items():
+            if key in dep_names and label not in frameworks:
+                frameworks.append(label)
+    if (repo / "pyproject.toml").exists():
+        text = (repo / "pyproject.toml").read_text()
+        if "fastapi" in text.lower():
+            frameworks.append("FastAPI")
+        if "django" in text.lower():
+            frameworks.append("Django")
+        if "flask" in text.lower():
+            frameworks.append("Flask")
+    return frameworks
+
+
+def detect_package_managers(repo):
+    package_managers = []
+    markers = {
+        "package-lock.json": "npm",
+        "pnpm-lock.yaml": "pnpm",
+        "yarn.lock": "yarn",
+        "bun.lockb": "bun",
+        "pyproject.toml": "uv/pip",
+        "requirements.txt": "pip",
+        "go.mod": "go",
+        "Cargo.toml": "cargo",
+    }
+    for marker, label in markers.items():
+        if (repo / marker).exists():
+            package_managers.append(label)
+    return package_managers
+
+
+def detect_frontend_style_files(files):
+    style_files = []
+    style_markers = (
+        ".css",
+        ".scss",
+        ".sass",
+        ".less",
+        "tailwind.config.js",
+        "tailwind.config.ts",
+        "postcss.config.js",
+        "postcss.config.ts",
+        "theme.js",
+        "theme.ts",
+        "tokens.js",
+        "tokens.ts",
+        "tokens.json",
+    )
+    path_keywords = (
+        "/styles/",
+        "/style/",
+        "/theme/",
+        "/themes/",
+        "/tokens/",
+        "/components/",
+        "/ui/",
+        "/app/",
+        "/src/",
+    )
+    for file_path in files:
+        lower = file_path.lower()
+        if lower.endswith(style_markers) or any(keyword in f"/{lower}" for keyword in path_keywords):
+            if lower.endswith((".js", ".jsx", ".ts", ".tsx", ".css", ".scss", ".sass", ".less", ".json")):
+                style_files.append(file_path)
+    return sorted(dict.fromkeys(style_files))[:20]
+
+
+def list_repo_files(repo):
+    ignored = {".git", ".codex", "node_modules", ".next", "dist", "build", "__pycache__"}
+    results = []
+    for root, dirs, files in os.walk(repo):
+        dirs[:] = [d for d in dirs if d not in ignored]
+        for filename in files:
+            path = Path(root, filename)
+            results.append(str(path.relative_to(repo)))
+    return sorted(results)
+
+
+def detect_existing_managed_files(repo):
+    managed = []
+    for relative_path in list(ROOT_FILES.keys()) + list(DOC_FILES.keys()) + list(FRONTEND_DOC_FILES.keys()):
+        path = repo / relative_path
+        if path.exists():
+            try:
+                if is_harness_owned_text(path.read_text()):
+                    managed.append(relative_path)
+            except UnicodeDecodeError:
+                continue
+    return managed
+
+
+def analyze_repo(repo):
+    files = list_repo_files(repo)
+    languages = detect_languages(files)
+    frameworks = detect_frameworks(repo)
+    package_managers = detect_package_managers(repo)
+    has_frontend = any(name in frameworks for name in ["Next.js", "React", "Vue", "Svelte", "Vite"]) or any(
+        file.endswith((".tsx", ".jsx", ".css", ".scss")) for file in files
+    )
+    frontend_style_files = detect_frontend_style_files(files) if has_frontend else []
+    existing_managed = detect_existing_managed_files(repo)
+    existing_harness = [
+        file for file in ["AGENTS.md", "ARCHITECTURE.md", "docs/PLANS.md", "docs/SECURITY.md"] if (repo / file).exists()
+    ]
+    missing_exec_plan_state = [
+        path
+        for path in [
+            "docs/exec-plans/active/README.md",
+            "docs/exec-plans/active/_template.md",
+            "docs/exec-plans/completed/README.md",
+        ]
+        if not (repo / path).exists()
+    ]
+    missing_sops = [
+        path
+        for path in [
+            "docs/sops/layered-domain-architecture-setup.md",
+            "docs/sops/encode-unseen-knowledge.md",
+            "docs/sops/local-observability-feedback-loop.md",
+            "docs/sops/chrome-devtools-ui-validation-loop.md",
+            "docs/sops/evidence-first-eval-loop.md",
+        ]
+        if not (repo / path).exists()
+    ]
+    durable_knowledge_targets = [
+        "ARCHITECTURE.md",
+        "docs/product-specs/",
+        "docs/RELIABILITY.md",
+        "docs/SECURITY.md",
+        "docs/references/",
+    ]
+    if has_frontend:
+        durable_knowledge_targets.insert(2, "docs/design-docs/")
+
+    inferred_answers = {
+        "project_name": repo.name,
+        "languages": languages,
+        "frameworks": frameworks,
+        "package_managers": package_managers,
+        "frontend_scope": (
+            "A frontend surface likely exists."
+            if has_frontend
+            else "No obvious frontend surface detected from the repository."
+        ),
+        "frontend_style_files": frontend_style_files,
+    }
+
+    human_confirmations = []
+    for question in QUESTION_CATALOG:
+        if question["id"] in {"frontend_stack_notes", "design_style_direction"} and not has_frontend:
+            continue
+        human_confirmations.append(question)
+
+    analysis = {
+        "project_name": repo.name,
+        "repo_path": str(repo.resolve()),
+        "languages": languages,
+        "frameworks": frameworks,
+        "package_managers": package_managers,
+        "has_frontend": has_frontend,
+        "frontend_style_files": frontend_style_files,
+        "inferred_answers": inferred_answers,
+        "existing_harness_files": existing_harness,
+        "existing_managed_files": existing_managed,
+        "missing_exec_plan_state": missing_exec_plan_state,
+        "missing_sops": missing_sops,
+        "durable_knowledge_targets": durable_knowledge_targets,
+        "human_confirmations": human_confirmations,
+        "harness_state": "existing" if existing_harness or existing_managed else "new",
+        "recommended_action": "init",
+        "notes": [
+            "Ask the human only the confirmations that the repository cannot answer safely.",
+            "If unmanaged harness files already exist, preserve them unless the human explicitly requests replacement.",
+            "Create execution-plan state before expecting agents to keep repository-mutating work synchronized.",
+            "Use SOPs to turn recurring architecture, UI, observability, and knowledge-capture work into mechanical loops.",
+            "Write durable facts into permanent docs instead of leaving them trapped inside plans or chat history.",
+        ],
+    }
+    return analysis
+
+

package/skills/harness-engine/scripts/harness_engine/checks.py

@@ -0,0 +1,287 @@
+from .common import *
+from .templates import DEFAULT_KNOWLEDGE_PLACEHOLDER, is_managed_text
+from .plans import active_plan_dir, completed_plan_dir, load_plan_state, sync_state_from_markdown, specific_acceptance_issues, criteria_fingerprint, find_section
+from .knowledge import extract_knowledge_items, parse_knowledge_item, destination_contains_fact
+from .plans import open_defects_for_plan
+from .continuation import phase_continuity_issues, continuation_decision_issues, workstreams_path
+
+def check_harness(repo):
+    required_files = [
+        "AGENTS.md",
+        "ARCHITECTURE.md",
+        "docs/PLANS.md",
+        "docs/QUALITY_SCORE.md",
+        "docs/RELIABILITY.md",
+        "docs/SECURITY.md",
+        "docs/exec-plans/workstreams.md",
+        "docs/exec-plans/active/README.md",
+        "docs/exec-plans/active/_template.md",
+        "docs/exec-plans/completed/README.md",
+        "docs/sops/encode-unseen-knowledge.md",
+    ]
+    issues = []
+    for relative_path in required_files:
+        if not (repo / relative_path).exists():
+            issues.append(
+                {
+                    "severity": "error",
+                    "code": "missing-required-file",
+                    "path": relative_path,
+                    "message": f"Required harness file is missing: {relative_path}",
+                }
+            )
+
+    active_dir = active_plan_dir(repo)
+    if active_dir.exists():
+        for plan_path in sorted(active_dir.glob("*.md")):
+            if plan_path.name in {"README.md", "_template.md"}:
+                continue
+            relative_plan = str(plan_path.relative_to(repo))
+            plan_text = plan_path.read_text()
+            state = load_plan_state(plan_path)
+            if state is None:
+                issues.append(
+                    {
+                        "severity": "error",
+                        "code": "missing-plan-sidecar",
+                        "path": relative_plan,
+                        "message": "Active plan is missing structured JSON sidecar metadata. Run migration or recreate it with plan-start.",
+                    }
+                )
+            else:
+                state = sync_state_from_markdown(plan_path, state)
+                contract = state.get("acceptance_contract", {})
+                criteria = contract.get("criteria") or {}
+                acceptance_issues = specific_acceptance_issues(criteria)
+                current_fingerprint = criteria_fingerprint(criteria)
+                if contract.get("status") != "ready" or acceptance_issues:
+                    issues.append(
+                        {
+                            "severity": "error",
+                            "code": "acceptance-contract-not-ready",
+                            "path": relative_plan,
+                            "message": "Active plan must have a ready, task-specific Acceptance Contract before implementation.",
+                            "acceptance_issues": acceptance_issues,
+                        }
+                    )
+                elif contract.get("fingerprint") != current_fingerprint:
+                    issues.append(
+                        {
+                            "severity": "error",
+                            "code": "acceptance-fingerprint-stale",
+                            "path": relative_plan,
+                            "message": "Active plan Acceptance Contract fingerprint does not match current criteria.",
+                        }
+                    )
+            if find_section(plan_text.splitlines(), "## Acceptance Contract") is None:
+                issues.append(
+                    {
+                        "severity": "error",
+                        "code": "missing-acceptance-contract",
+                        "path": relative_plan,
+                        "message": "Active plan is missing an Acceptance Contract section.",
+                    }
+                )
+            if find_section(plan_text.splitlines(), "## Quality Result") is None:
+                issues.append(
+                    {
+                        "severity": "error",
+                        "code": "missing-quality-result",
+                        "path": relative_plan,
+                        "message": "Active plan is missing a Quality Result section.",
+                    }
+                )
+            for defect in open_defects_for_plan(plan_text):
+                issues.append(
+                    {
+                        "severity": "error",
+                        "code": "open-defect",
+                        "path": relative_plan,
+                        "id": defect["id"],
+                        "defect_severity": defect["severity"],
+                        "message": f"Active plan has an unresolved defect: {defect['summary']}",
+                    }
+                )
+            issues.extend(phase_continuity_issues(repo, plan_path, plan_text))
+            for item in extract_knowledge_items(plan_text):
+                if item == DEFAULT_KNOWLEDGE_PLACEHOLDER:
+                    continue
+                parsed = parse_knowledge_item(item)
+                if not parsed:
+                    issues.append(
+                        {
+                            "severity": "error",
+                            "code": "unparseable-knowledge-item",
+                            "path": relative_plan,
+                            "message": f"Knowledge item is not parseable: {item}",
+                        }
+                    )
+                    continue
+                if parsed["status"] == "open":
+                    issues.append(
+                        {
+                            "severity": "error",
+                            "code": "open-durable-knowledge",
+                            "path": relative_plan,
+                            "destination": parsed["destination"],
+                            "message": f"Durable knowledge is still open: {parsed['fact']}",
+                        }
+                    )
+                else:
+                    verification_text = parsed["evidence"] or parsed["fact"]
+                    if destination_contains_fact(repo, parsed["destination"], verification_text):
+                        continue
+                    issues.append(
+                        {
+                            "severity": "error",
+                            "code": "missing-written-knowledge",
+                            "path": relative_plan,
+                            "destination": parsed["destination"],
+                            "message": f"Marked knowledge evidence is missing from destination: {verification_text}",
+                        }
+                    )
+
+    completed_dir = completed_plan_dir(repo)
+    if completed_dir.exists():
+        for plan_path in sorted(completed_dir.glob("*.md")):
+            if plan_path.name == "README.md":
+                continue
+            relative_plan = str(plan_path.relative_to(repo))
+            state = load_plan_state(plan_path)
+            if state is None:
+                issues.append(
+                    {
+                        "severity": "error",
+                        "code": "missing-plan-sidecar",
+                        "path": relative_plan,
+                        "message": "Completed plan is missing structured JSON sidecar metadata. Run migration or recreate structured plan history.",
+                    }
+                )
+                continue
+            state = sync_state_from_markdown(plan_path, state)
+            contract = state.get("acceptance_contract", {})
+            criteria = contract.get("criteria") or {}
+            current_fingerprint = criteria_fingerprint(criteria)
+            quality = state.get("quality_result", {})
+            if contract.get("status") != "ready":
+                issues.append(
+                    {
+                        "severity": "error",
+                        "code": "completed-acceptance-contract-not-ready",
+                        "path": relative_plan,
+                        "message": "Completed plan must have a ready Acceptance Contract.",
+                    }
+                )
+            if quality.get("status") != "pass":
+                issues.append(
+                    {
+                        "severity": "error",
+                        "code": "completed-quality-result-not-passing",
+                        "path": relative_plan,
+                        "message": "Completed plan must have a passing Quality Result.",
+                    }
+                )
+            if quality.get("criteria_fingerprint") != current_fingerprint:
+                issues.append(
+                    {
+                        "severity": "error",
+                        "code": "completed-quality-fingerprint-stale",
+                        "path": relative_plan,
+                        "message": "Completed plan Quality Result was not scored against the current Acceptance Contract.",
+                    }
+                )
+            issues.extend(continuation_decision_issues(repo, plan_path, plan_path.read_text()))
+
+    ledger = workstreams_path(repo)
+    if ledger.exists():
+        for index, line in enumerate(ledger.read_text().splitlines(), start=1):
+            stripped = line.strip()
+            if not stripped.startswith("|") or stripped.startswith("| ---") or stripped.startswith("| ID |"):
+                continue
+            cells = [cell.strip() for cell in stripped.strip("|").split("|")]
+            if len(cells) != 6:
+                continue
+            workstream_id, _, current_plan, last_completed_plan, _, _ = cells
+            for label, plan_value in [
+                ("current plan", current_plan),
+                ("last completed plan", last_completed_plan),
+            ]:
+                if plan_value in {"", "none", "n/a", "-"}:
+                    continue
+                if not (repo / plan_value).exists():
+                    issues.append(
+                        {
+                            "severity": "error",
+                            "code": "missing-workstream-plan-reference",
+                            "path": str(ledger.relative_to(repo)),
+                            "line": index,
+                            "workstream": workstream_id,
+                            "message": f"Workstream {workstream_id} references missing {label}: {plan_value}",
+                        }
+                    )
+
+    return {
+        "repo": str(repo),
+        "status": "pass" if not issues else "fail",
+        "issue_count": len(issues),
+        "issues": issues,
+    }
+
+
+def docs_text_for_reference_scan(repo):
+    docs_root = repo / "docs"
+    chunks = []
+    roots = [repo / "AGENTS.md", repo / "ARCHITECTURE.md"]
+    if docs_root.exists():
+        roots.extend(path for path in docs_root.rglob("*") if path.is_file())
+    for path in roots:
+        if not path.exists() or not path.is_file():
+            continue
+        try:
+            chunks.append(path.read_text())
+        except UnicodeDecodeError:
+            continue
+    return "\n".join(chunks)
+
+
+def evidence_prune_candidates(repo, root="docs/generated", older_than_days=14):
+    evidence_root = (repo / root).resolve()
+    if not evidence_root.exists():
+        return []
+    try:
+        evidence_root.relative_to(repo.resolve())
+    except ValueError as error:
+        raise ValueError(f"Evidence root must be inside repo: {root}") from error
+
+    now = time.time()
+    max_age_seconds = older_than_days * 24 * 60 * 60
+    docs_text = docs_text_for_reference_scan(repo)
+    candidates = []
+    for path in sorted(evidence_root.rglob("*")):
+        if not path.is_file():
+            continue
+        relative_path = str(path.relative_to(repo))
+        try:
+            content = path.read_text()
+        except UnicodeDecodeError:
+            content = ""
+        if is_managed_text(content):
+            continue
+        age_seconds = now - path.stat().st_mtime
+        if age_seconds < max_age_seconds:
+            continue
+        if relative_path in docs_text or path.name in docs_text:
+            continue
+        candidates.append(
+            {
+                "path": relative_path,
+                "age_days": round(age_seconds / (24 * 60 * 60), 1),
+                "reason": (
+                    f"unreferenced file under {root} older than {older_than_days} days "
+                    "and not a managed starter"
+                ),
+            }
+        )
+    return candidates
+
+