@ahmed-g-gad/apothem 0.1.1

package/src/apothem/conformity/README.md

@@ -0,0 +1,173 @@
+<!-- SPDX-License-Identifier: MIT -->
+
+# conformity
+
+> **Role.** Pre-emission conformity validators. `gate.py` orchestrates a fleet of mechanical `*_grep` matchers (plus the link checker) against a Write/Edit input, returning a pass/fail verdict before the artifact is emitted.
+
+## Orchestrator
+
+| File | Purpose |
+|------|---------|
+| `gate.py` | The conformity-gate orchestrator — dispatches every matcher against a Write/Edit input and aggregates the verdicts. |
+| `__init__.py` | Package marker. |
+
+## Matcher families
+
+### Authorship & structure
+
+| File | Checks |
+|------|--------|
+| `file_header_grep.py` | Canonical authorship-header presence and form. |
+| `frontmatter_grep.py` | Required frontmatter fields on artifact classes that carry them. |
+| `frontmatter_value_grep.py` | Frontmatter `enum`/`pattern` values against the per-class JSON Schema (agents, commands, skills) — the value-level counterpart to `frontmatter_grep`'s key check. |
+| `naming_grep.py` | Naming-convention conformance. |
+| `binding_reciprocity_grep.py` | Reciprocal five-direction bindings — half-edge detection. |
+| `always_on_budget_grep.py` | Always-on rule body token budget (the 500-token ceiling). |
+| `agent_capability_grep.py` | Every harness declares its agentic-capability matrix. |
+| `agents_md_coverage_grep.py` | Stale agent-companion files under the root-only AGENTS.md convention. |
+| `registry_capability_consistency_grep.py` | Registry capability cells not backed by install / materializer / projection evidence. |
+| `recommend_next_step_grep.py` | `## Recommended Next Step` block presence on command / skill terminal surfaces. |
+| `determinism_grep.py` | Deterministic output shape across command and skill surfaces. |
+| `no_toplevel_docs_grep.py` | A top-level `docs/` directory at the repository root. |
+
+### Prose & narrative discipline
+
+| File | Checks |
+|------|--------|
+| `hedging_grep.py` | Hedging vocabulary in prescriptive prose. |
+| `option_annotation_grep.py` | Per-option `(Recommended)` label-to-body bind (H6): canonical capital postfix iff body `recommendation: recommended`, no spurious or lowercase postfix, at most one recommended option in a single-select set. |
+| `completion_claim_grep.py` | Unsupported absolute-guarantee completion claims in cohort prose (spec §4.2 honesty) — e.g., "guaranteed to pass", "always works", "100% complete", "cannot fail". |
+| `diagram_staleness_grep.py` | Diagram verification dates against the structure they abstract. |
+| `multi_surface_coherence_grep.py` | Coherence across paired/multi-surface artifacts. |
+| `plain_language_grep.py` | Mechanistic / harness-internal vocabulary in user-facing prose. |
+| `token_efficiency_grep.py` | Filler phrases, throat-clearing openers, content-free qualifiers. |
+| `redundancy_grep.py` | Substantively-duplicated paragraphs across the governed corpus. |
+| `agnosticism_grep.py` | Harness bias and re-introduced enforcement presets in shipped surfaces. |
+
+### Plans discipline
+
+| File | Checks |
+|------|--------|
+| `no_global_plans_grep.py` | Writes to a global plans directory. |
+| `plans_discipline_language_grep.py` | Plans-discipline language violations. |
+| `orphan_output_grep.py` | Orphan outputs — no consumer / no index entry / no producer attribution. |
+| `user_confirm_grep.py` | Unfilled `<USER-CONFIRM:…>` placeholders. |
+| `plan_suite_structure_grep.py` | Plan-suite structure — suite-locality, closed vocabulary, numeric-prefix discipline. |
+| `plan_next_step_consistency_grep.py` | Plan-suite infra files' Recommended-Next-Step footer consistency (advisory). |
+
+### Code craft
+
+| File | Checks |
+|------|--------|
+| `bare_except_grep.py` | Bare / overly-broad `except` clauses. |
+| `commented_out_code_grep.py` | Commented-out code blocks. |
+| `magic_number_grep.py` | Magic numbers in logic without named constants. |
+
+### Supply chain & security
+
+| File | Checks |
+|------|--------|
+| `secret_leak_grep.py` | Secret literals committed to source — vendor-prefixed credential patterns plus a high-entropy heuristic, with a canonical-banner allow-list. |
+| `unpinned_action_grep.py` | Unpinned GitHub Actions `uses:` references. |
+| `permissions_minimum_scope_grep.py` | Every workflow declares a minimum-scope `permissions:` block. |
+| `harden_runner_grep.py` | Every workflow job opens with a conformant harden-runner step. |
+| `workflow_concurrency_grep.py` | Concurrency group + per-job timeout discipline on workflows. |
+| `oidc_trusted_publishing_grep.py` | Release workflows use OIDC trusted publishing. |
+| `cross_platform_matrix_grep.py` | CI declares a cross-platform OS × Python matrix. |
+| `editorconfig_presence_grep.py` | Canonical `.editorconfig` present at the project root. |
+| `gitattributes_presence_grep.py` | Repo-root `.gitattributes` carries the canonical contract. |
+
+### Release & branding
+
+| File | Checks |
+|------|--------|
+| `production_ready_pr_grep.py` | Same-change-set production-readiness discipline. |
+| `license_author_consistency_grep.py` | Verifies the root LICENSE carries an author line. |
+| `brand_mark_grep.py` | Brand-mark usage. |
+| `smoke_install_grep.py` | Install-script smoke-check markers. |
+| `copilot_instructions_presence_grep.py` | `copilot-instructions.md` presence. |
+| `dynamism_grep.py` | Static-string version embeds in dynamism-required surfaces. |
+| `static_version_grep.py` | Version-bearing sites resolve dynamically, not as static literals. |
+| `semver_stability_grep.py` | Semver stability (the change-set-scoped M15 mechanical fraction). |
+| `conventional_commit_grep.py` | HEAD commit messages against the Conventional-Commits grammar. |
+| `freshness_token_grep.py` | Freshness-narrative phrases on shipped public surfaces. |
+
+### Link integrity
+
+| File | Checks |
+|------|--------|
+| `link_check.py` | Validate Markdown internal links against the on-disk reference graph. |
+
+## Operating contract
+
+`gate.py` is the single dispatch surface. It owns two matcher registries —
+`GREP_MODULES` (per-Write matchers with a `check(content, path)` signature, run
+on the Write/Edit body) and `STANDALONE_MODULES` (corpus walkers that take a
+root directory and walk the tree, git index, or a fixed surface set) — plus the
+CLI surface (`--all`, `--check <name>`, `--list`, `--hook`, `--strict`) and the
+scope/short-circuit logic. The invariants a matcher must hold to:
+
+- **Advisory by default.** The gate reports findings and exits zero so the write
+  proceeds; strict blocking is opt-in via `--strict` or a truthy
+  `APOTHEM_CONFORMITY_STRICT`. A matcher's standalone `to_json()` carries
+  `"advisory": true`, and its CLI `_main` may exit non-zero on findings (the
+  orchestrator treats the standalone exit code as the strict signal). Never
+  hard-block a write from inside a matcher.
+- **Fail-open isolation.** The orchestrator wraps every matcher load and
+  `check()` call; a raised exception is recorded and surfaced, never swallowed,
+  and never fail-closes the write. Do not catch-and-suppress inside a matcher;
+  let the orchestrator's boundary handle errors.
+- **Dataclass result shape.** Matchers return a frozen `GrepResult` carrying
+  `passed: bool` and `findings: list[Finding]`; `Finding` carries the surface,
+  kind/issue, and a `detail` string. The orchestrator duck-types
+  `getattr(result, "passed", ...)` and `result.findings`, so the field names are
+  the contract.
+- **Registry-name forms.** `STANDALONE_MODULES` entries are stored hyphenated
+  (`reference-token-grep`) for CLI ergonomics; the on-disk filename is
+  underscored (`reference_token_grep.py`). `GREP_MODULES` entries are
+  underscored. `_resolve_validator` normalizes both forms.
+- **Plan/scope short-circuits are gate-owned.** `.plans/` paths, out-of-scope
+  writes, and harness runtime-state subtrees (`projects/`, `memory/`)
+  short-circuit to a silent pass in the orchestrator. A per-Write matcher MUST
+  NOT itself special-case `.plans/`; rely on the gate's `_is_plan_suite_path`
+  short-circuit.
+- Denylist/schema data a matcher consumes lives in [`../schemas/`](../schemas/),
+  never inlined in the module.
+
+## Adding or modifying a matcher
+
+To **add a matcher**:
+
+1. Author `<name>_grep.py` here. Define frozen `Finding` and `GrepResult`
+   dataclasses; `GrepResult.to_json()` emits `"advisory": true`. Expose the
+   entry callable — **per-Write**:
+   `check(content: str, path: Path | None = None) -> GrepResult`; **corpus
+   walker**: `check(root: Path) -> GrepResult` plus a `_main(argv)` that prints
+   JSON and returns `EXIT_PASS` (0) / `EXIT_FAIL` (2).
+2. **Register** in `gate.py`: add the underscored name to `GREP_MODULES` for
+   per-Write, OR the hyphenated name to `STANDALONE_MODULES` for a corpus walker.
+3. Add **fixtures and tests** under `tests/conformity/<name>/` (pass/fail cases)
+   and a test module under `tests/conformity/`. For standalone walkers, also
+   register the matcher in `tests/conformity/test_standalone_greps_inprocess.py`
+   so it runs in-process in the suite.
+4. Keep within the **per-grep wall-clock budget** (`PER_GREP_BUDGET_SECONDS` in
+   `gate.py`); the hook has a 10s `PreToolUse` ceiling and the orchestrator flags
+   any matcher approaching the per-grep budget as a watch item. Cheap structural
+   scans run before expensive regex sweeps.
+
+To **modify or remove a matcher**: keep the registry entry, fixtures, and the
+in-process test list in sync in the same change-set; an orphaned entry or a
+registered name with no script surfaces as a load error.
+
+Validate every change here:
+
+```bash
+python -m apothem.conformity.gate --all .
+python -m pytest tests/conformity
+python -m ruff check .
+python -m mypy src/apothem/cli/ src/apothem/harnesses/
+```
+
+## Related
+
+- [`hooks/`](../hooks/) — the dispatcher that fires the gate on PreToolUse Write/Edit events.

package/src/apothem/conformity/__init__.py

@@ -0,0 +1 @@
+# SPDX-License-Identifier: MIT

package/src/apothem/conformity/_grep_base.py

@@ -0,0 +1,93 @@
+# SPDX-License-Identifier: MIT
+
+"""Shared scaffolding for path-based conformity matchers.
+
+Why this module exists. The path-based conformity matchers
+(``check(content, path) -> GrepResult``) each defined a byte-identical
+``GrepResult`` dataclass, an identical ``to_json`` serialiser, and a
+near-identical ``_read_input`` / ``_main`` / ``__main__`` skeleton. Hoisting
+that boilerplate here removes the duplication while preserving every matcher's
+observable behaviour: the JSON report shape, the exit codes, and the
+stdin-or-path input contract are unchanged.
+
+Scope. This base serves matchers whose canonical report payload is exactly
+``{grep, path, passed, findings}`` — optionally carrying a ``note`` emitted
+only when present. Matchers with ``root``-based payloads, extra report keys,
+or always-emitted ``advisory`` flags keep their own ``GrepResult`` because
+their serialised shape diverges from this one.
+"""
+
+from __future__ import annotations
+
+import json
+import sys
+from collections.abc import Callable
+from dataclasses import asdict, dataclass, field
+from pathlib import Path
+from typing import Any, Final, Protocol
+
+EXIT_PASS: Final[int] = 0
+EXIT_FAIL: Final[int] = 2
+STDIN_FLAG: Final[str] = "--stdin"
+
+
+class _Finding(Protocol):
+    """Structural type for a per-matcher finding dataclass."""
+
+
+@dataclass(frozen=True)
+class GrepResult:
+    """Canonical path-based matcher report.
+
+    Pre-conditions: ``findings`` holds the matcher-specific finding dataclasses
+    (each a frozen ``@dataclass``). Post-conditions: ``to_json`` emits the
+    canonical ``{grep, path, passed, findings}`` payload, including ``note``
+    only when it is not ``None``.
+    """
+
+    grep: str
+    path: str | None
+    passed: bool
+    findings: list[Any] = field(default_factory=list)
+    note: str | None = None
+
+    def to_json(self) -> str:
+        payload: dict[str, object] = {
+            "grep": self.grep,
+            "path": self.path,
+            "passed": self.passed,
+            "findings": [asdict(f) for f in self.findings],
+        }
+        if self.note is not None:
+            payload["note"] = self.note
+        return json.dumps(payload, indent=2)
+
+
+def read_input(argv: list[str]) -> tuple[str, Path | None]:
+    """Return ``(content, path)`` from a path argument or stdin.
+
+    Pre-conditions: ``argv`` is ``sys.argv`` (``argv[0]`` is the script name).
+    Post-conditions: when ``argv[1]`` is a path (not ``--stdin``), the file is
+    read and returned with its ``Path``; otherwise stdin is read with a
+    ``None`` path.
+    """
+    if len(argv) >= 2 and argv[1] != STDIN_FLAG:
+        path = Path(argv[1])
+        return path.read_text(encoding="utf-8"), path
+    return sys.stdin.read(), None
+
+
+def run_grep(
+    check: Callable[[str, Path | None], GrepResult],
+    argv: list[str],
+) -> int:
+    """Run ``check`` over the resolved input, print the report, return the exit.
+
+    Pre-conditions: ``check`` returns a :class:`GrepResult`. Post-conditions:
+    the JSON report is printed to stdout; the return is ``EXIT_PASS`` when the
+    result passed, ``EXIT_FAIL`` otherwise.
+    """
+    content, path = read_input(argv)
+    result = check(content, path)
+    print(result.to_json())
+    return EXIT_PASS if result.passed else EXIT_FAIL

package/src/apothem/conformity/agent_capability_grep.py

@@ -0,0 +1,306 @@
+# SPDX-License-Identifier: MIT
+
+"""Verify every harness declares its agentic-capability matrix.
+
+Why this validator exists. The agent-capability discipline (per
+``rules/agent-capability-discipline.md``) requires every harness in
+the 17-harness cohort to declare a structured capability matrix in its
+template tree so adapter authors, operators, and downstream tooling can
+reason about the harness's agentic surface without inspecting source.
+The installed machine-readable projection names thirteen fields:
+``mcp_servers`` (list of MCP servers the harness can host),
+``sub_agent_dispatch`` (bool — whether the harness spawns sub-agents),
+``custom_command_support`` (yes/no/discovery-pending),
+``recommended_postfix_rendering`` (how recommended option labels render),
+``long_context_compaction`` (native or profile-managed long-session
+continuity), ``context_ignore_surface`` (native ignore surface or declared
+absence), ``standard_convention_pin`` (the sibling pin file),
+``tool_surface_restrictions`` (list of tool-surface constraints),
+``system_prompt_template_path`` (str pointer to the harness's system-prompt
+template), ``agent_memory_surface`` (str naming the harness's memory
+surface), ``layered_context_surface`` (native hierarchy or profile-managed
+projection), ``lsp_symbol_navigation`` (native/plugin-backed symbol lookup
+or tracked gap), and ``hook_learning_capture`` (where recurring hook lessons
+are persisted without pass-class chatter).
+
+Detection. For each harness directory at
+``src/apothem/harnesses/<harness>/`` that exists, look for one of three
+declaration files: ``capabilities.yml`` / ``capabilities.yaml`` /
+``agent_capabilities.md``. The YAML form is parsed; the Markdown form is
+inspected for the thirteen field markers as ``- field_name:`` lines. A
+harness whose directory is present but whose declaration is absent OR
+whose declaration is missing any required projection field produces a
+finding.
+
+Pre-materialization tolerance. Some harness directories may not yet exist
+on disk in early repository states. The validator is absence-tolerant: when zero harness
+directories are present, it exits 0 with an informational
+``harnesses_not_yet_materialized`` message so the gate does not block
+the early absence-tolerant state. Once at least one harness directory exists, the
+validator enforces the discipline on every present harness.
+
+Exit semantics. Exits 0 when every present harness declares a complete
+capability matrix (or when zero harness directories exist). Exits 2 on
+any present harness lacking the declaration or missing any required
+field. The exit-2 convention matches the conformity-gate orchestrator's
+``EXIT_FAIL`` constant.
+"""
+
+from __future__ import annotations
+
+import json
+import sys
+from dataclasses import asdict, dataclass, field
+from pathlib import Path
+from typing import Final
+
+import yaml
+
+GREP_NAME: Final[str] = "agent-capability-grep"
+RULE_ANCHOR: Final[str] = "agent-capability-discipline (per-harness capability matrix)"
+
+EXIT_PASS: Final[int] = 0
+EXIT_FAIL: Final[int] = 2
+
+# The 17-harness cohort. Each entry is the directory name (package key)
+# under src/apothem/harnesses/. The cohort is derived from the registry's
+# SUPPORTED_PACKAGE_KEYS so a newly-registered adapter (e.g. codebuddy,
+# kiro, trae, zed) is validated without editing this tuple. When the
+# registry cannot be imported (an unusual mount layout the standalone
+# subprocess cannot reach), the validator falls back to the static cohort
+# so it never silently shrinks its sweep below the known full set.
+_FALLBACK_HARNESS_COHORT: Final[tuple[str, ...]] = (
+    "antigravity",
+    "claude_code",
+    "codex",
+    "cursor",
+    "gemini_cli",
+    "github_copilot",
+    "hermes",
+    "kimi_code",
+    "open_claw",
+    "opencode",
+    "qwen_code",
+    "windsurf",
+    "codebuddy",
+    "kiro",
+    "trae",
+    "zed",
+    "glm",
+)
+
+
+def _resolve_cohort() -> tuple[str, ...]:
+    """Return the harness package-key cohort, derived from the registry.
+
+    Imports ``SUPPORTED_PACKAGE_KEYS`` from the harness registry so the
+    cohort tracks the registered adapter set automatically. Falls back to
+    the static full cohort if the import fails.
+    """
+    try:
+        from apothem.lib.harness_registry import SUPPORTED_PACKAGE_KEYS
+    except Exception:  # import boundary: fall back to the static cohort
+        return _FALLBACK_HARNESS_COHORT
+    return tuple(SUPPORTED_PACKAGE_KEYS)
+
+
+HARNESS_COHORT: Final[tuple[str, ...]] = _resolve_cohort()
+
+# The required capability-matrix fields for the installed projection.
+REQUIRED_FIELDS: Final[tuple[str, ...]] = (
+    "mcp_servers",
+    "sub_agent_dispatch",
+    "custom_command_support",
+    "recommended_postfix_rendering",
+    "long_context_compaction",
+    "context_ignore_surface",
+    "layered_context_surface",
+    "lsp_symbol_navigation",
+    "hook_learning_capture",
+    "standard_convention_pin",
+    "tool_surface_restrictions",
+    "system_prompt_template_path",
+    "agent_memory_surface",
+)
+
+# Declaration filenames the validator probes per harness, in priority order.
+DECLARATION_FILENAMES: Final[tuple[str, ...]] = (
+    "capabilities.yml",
+    "capabilities.yaml",
+    "agent_capabilities.md",
+)
+
+
+@dataclass(frozen=True)
+class Finding:
+    """One harness lacking a complete capability matrix."""
+
+    harness: str
+    detail: str
+    missing_fields: list[str] = field(default_factory=list)
+    rule: str = RULE_ANCHOR
+
+
+@dataclass(frozen=True)
+class GrepResult:
+    """Aggregated walk result for a single cohort sweep."""
+
+    grep: str
+    root: str
+    harnesses_present: int
+    harnesses_total: int
+    passed: bool
+    informational: str | None = None
+    findings: list[Finding] = field(default_factory=list)
+
+    def to_json(self) -> str:
+        payload = {
+            "grep": self.grep,
+            "root": self.root,
+            "harnesses_present": self.harnesses_present,
+            "harnesses_total": self.harnesses_total,
+            "passed": self.passed,
+            "informational": self.informational,
+            "findings": [asdict(f) for f in self.findings],
+        }
+        return json.dumps(payload, indent=2)
+
+
+def _harnesses_dir(root: Path) -> Path:
+    return root / "src" / "apothem" / "harnesses"
+
+
+def _find_declaration(harness_dir: Path) -> Path | None:
+    """Probe the harness directory tree for a declaration file.
+
+    Looks at the harness root and at ``templates/`` inside it (the
+    rule allows either location). Returns the first match in
+    ``DECLARATION_FILENAMES`` priority order, or None when absent.
+    """
+    candidate_dirs = (harness_dir, harness_dir / "templates")
+    for directory in candidate_dirs:
+        for filename in DECLARATION_FILENAMES:
+            candidate = directory / filename
+            if candidate.is_file():
+                return candidate
+    return None
+
+
+def _parse_yaml_declaration(path: Path) -> dict[str, object]:
+    """Load a YAML declaration into a dict. Empty or non-mapping → {}."""
+    try:
+        raw = path.read_text(encoding="utf-8")
+    except OSError:
+        return {}
+    try:
+        loaded = yaml.safe_load(raw)
+    except yaml.YAMLError:
+        return {}
+    if isinstance(loaded, dict):
+        return loaded
+    return {}
+
+
+def _parse_markdown_declaration(path: Path) -> dict[str, object]:
+    """Scan a Markdown declaration for ``- field_name:`` field markers.
+
+    The Markdown form admits any prose around the field markers; this
+    parser is presence-only — it records each required field's
+    presence (mapping to a placeholder True value) without inspecting
+    the field's substantive value. The YAML form is the rigorous
+    surface; the Markdown form is the lightweight alternative.
+    """
+    try:
+        text = path.read_text(encoding="utf-8")
+    except OSError:
+        return {}
+    found: dict[str, object] = {}
+    for line in text.splitlines():
+        stripped = line.strip()
+        for field_name in REQUIRED_FIELDS:
+            # Accept `- field_name:` or `field_name:` at line start.
+            if stripped.startswith(f"- {field_name}:") or stripped.startswith(
+                f"{field_name}:"
+            ):
+                found[field_name] = True
+    return found
+
+
+def _missing_fields(declaration: dict[str, object]) -> list[str]:
+    return [f for f in REQUIRED_FIELDS if f not in declaration]
+
+
+def check(root: Path) -> GrepResult:
+    """Walk the harness cohort under root; flag incomplete declarations."""
+    harnesses_dir = _harnesses_dir(root)
+    findings: list[Finding] = []
+    present_count = 0
+    for harness in HARNESS_COHORT:
+        harness_dir = harnesses_dir / harness
+        if not harness_dir.is_dir():
+            continue
+        present_count += 1
+        declaration_path = _find_declaration(harness_dir)
+        if declaration_path is None:
+            findings.append(
+                Finding(
+                    harness=harness,
+                    detail=(
+                        "no capability declaration found; expected one of "
+                        f"{list(DECLARATION_FILENAMES)} under {harness_dir} "
+                        "or its templates/ subdirectory"
+                    ),
+                    missing_fields=list(REQUIRED_FIELDS),
+                )
+            )
+            continue
+        if declaration_path.suffix in {".yml", ".yaml"}:
+            declaration = _parse_yaml_declaration(declaration_path)
+        else:
+            declaration = _parse_markdown_declaration(declaration_path)
+        missing = _missing_fields(declaration)
+        if missing:
+            findings.append(
+                Finding(
+                    harness=harness,
+                    detail=(
+                        f"declaration at {declaration_path} is missing "
+                        f"{len(missing)} of {len(REQUIRED_FIELDS)} required "
+                        "capability-matrix fields"
+                    ),
+                    missing_fields=missing,
+                )
+            )
+    informational: str | None = None
+    if present_count == 0:
+        informational = (
+            "harnesses_not_yet_materialized: zero of "
+            f"{len(HARNESS_COHORT)} harness directories present under "
+            f"{harnesses_dir}; validator is in an absence-tolerant pass"
+        )
+    return GrepResult(
+        grep=GREP_NAME,
+        root=str(root),
+        harnesses_present=present_count,
+        harnesses_total=len(HARNESS_COHORT),
+        passed=not findings,
+        informational=informational,
+        findings=findings,
+    )
+
+
+def _read_input(argv: list[str]) -> Path:
+    if len(argv) >= 2:
+        return Path(argv[1])
+    return Path.cwd()
+
+
+def _main(argv: list[str]) -> int:
+    root = _read_input(argv)
+    result = check(root)
+    print(result.to_json())
+    return EXIT_PASS if result.passed else EXIT_FAIL
+
+
+if __name__ == "__main__":
+    sys.exit(_main(sys.argv))