@event4u/agent-config 2.19.0 → 2.20.0

package/docs/profiles.md

@@ -0,0 +1,91 @@
+# Six entry paths — by `profile.id`
+
+Each block below is the first-screen for one shipped profile. The
+`profile.id` written by `/onboard` selects the anchor; the block names
+the audience, the first three things the agent does for that role,
+and the exact commands and skills wired into the profile YAML at
+[`.agent-src.uncompressed/profiles/<id>.yml`](../.agent-src.uncompressed/profiles/).
+
+The summary table at the top of [`README.md`](../README.md) is the
+one-page index; the prose below is the deep version.
+
+<a id="profile-developer"></a>
+## 👩‍💻 `developer` — IC engineer
+
+Implement a ticket end-to-end, fix CI red, run a self-review before
+the PR. `/implement-ticket` refines the ticket, plans, edits, tests,
+and verifies; `/work` is the free-form sibling; `/review-changes`
+dispatches five judges (bug, security, tests, quality, architecture)
+on the local diff. Stack-aware skills cover Laravel · Symfony ·
+Next.js · React · Node. **Preset default: `balanced`.**
+[Profile YAML](../.agent-src.uncompressed/profiles/developer.yml) ·
+[Role guide](getting-started-by-role.md#developer-the-original-audience).
+
+<a id="profile-content_creator"></a>
+## ✍️ `content_creator` — writers, ghostwriters, marketers
+
+Draft in someone else's voice, plan a quarter of content, ship a
+launch announcement. `/ghostwriter` fetches and writes against a
+public-figure voice profile; `/post-as` is the same primitive for
+your own voice (`.agent-user.md`); `voice-and-tone-design` and
+`messaging-architecture` lock the brand frame before any copy ships.
+**Preset default: `balanced`.**
+[Profile YAML](../.agent-src.uncompressed/profiles/content_creator.yml) ·
+[Role guide](getting-started-by-role.md#creator-writer-marketer-indie-content-shop).
+
+<a id="profile-founder"></a>
+## 🚀 `founder` — solo / early-stage founder
+
+Sharpen a fuzzy idea, rank what to build, write the why-now slide.
+`/challenge-me` runs a grill-style interview that turns a vague plan
+into a copyable pitch; `/council` polls external AIs for a neutral
+second opinion; `rice-prioritization` ranks the backlog;
+`vision-articulation` and `fundraising-narrative` shape the
+internal-vs-external story. **Preset default: `fast`.**
+[Profile YAML](../.agent-src.uncompressed/profiles/founder.yml) ·
+[Role guide](getting-started-by-role.md#founder-early-stage-operator-wearing-every-hat).
+
+<a id="profile-agency"></a>
+## 🏛 `agency` — multi-client delivery shop
+
+Refine a fuzzy client ask into an estimated, AC-tight ticket; turn a
+phase into a roadmap; ship per client without losing decision
+provenance. `/refine-ticket` rewrites the ticket and surfaces top-5
+risks; `estimate-ticket` sizes and splits; `decision-record` anchors
+the trade-off in an ADR before code starts. **Preset default:
+`strict`.**
+[Profile YAML](../.agent-src.uncompressed/profiles/agency.yml) ·
+[Role guide](getting-started-by-role.md#consultant-advisory-freelance-fractional).
+
+<a id="profile-finance"></a>
+## 💼 `finance` — CFO / fractional finance / FP&A
+
+Build a DCF, stress-test the plan, frame the runway call. `dcf-modeling`
+walks the WACC / terminal-value / 5-year-hold reasoning; `forecasting`
+reconciles top-down vs bottom-up; `scenario-modeling` produces the
+base / upside / downside cuts; `runway-cognition` frames the
+fundraise-vs-cut-vs-grow decision. **Preset default: `strict`.**
+[Profile YAML](../.agent-src.uncompressed/profiles/finance.yml) ·
+[Role guide](getting-started-by-role.md#finance--ops-cfo-controller-ops-lead-founder-finance).
+
+<a id="profile-ops"></a>
+## 🛡 `ops` — RevOps, support, SRE-adjacent
+
+Threat-model a change before it ships, command the incident when it
+breaks, build the dashboard that catches it next time.
+`/threat-model` enumerates abuse cases and trust boundaries before
+the first line of code; `incident-commander` frames severity and
+post-mortem; `dashboard-design` chooses the right RED / USE / Golden
+Signal panel. **Preset default: `strict`.**
+[Profile YAML](../.agent-src.uncompressed/profiles/ops.yml) ·
+[Role guide](getting-started-by-role.md#finance--ops-cfo-controller-ops-lead-founder-finance).
+
+> **Universal AI Agent OS, not "for developers only".** The same
+> orchestration core also drives non-software trades. Worked-example
+> user types ship for [galabau](../.agent-src.uncompressed/user-types/galabau-field-crew.md),
+> [metalworking](../.agent-src.uncompressed/user-types/metalworking-shop.md),
+> [truck driving](../.agent-src.uncompressed/user-types/truck-driver.md) —
+> with a [scaffold](../.agent-src.uncompressed/user-types/_template/) for
+> contributing your own.
+
+→ [Public catalog](catalog.md) (all rules, skills, commands, guidelines) · [Skills only](skills-catalog.md) · [llms.txt](../llms.txt)

package/package.json

@@ -1,6 +1,6 @@
 {
     "name": "@event4u/agent-config",
-    "version": "2.19.0",
+    "version": "2.20.0",
     "description": "Shared agent configuration \u2014 skills, rules, commands, guidelines, and templates for AI coding tools",
     "license": "MIT",
     "private": false,

package/scripts/_cli/cmd_explain.py

@@ -0,0 +1,250 @@
+"""``agent-config explain`` — print the decision chain behind an outcome.
+
+Step-15 Phase 1 item 3. Answers the silent "why did the agent do that?"
+question by showing which inputs the loader / router consulted, in what
+order, and which one won. Read-only; never edits state, never dispatches
+network calls. Three subjects in the v1 surface:
+
+* ``config``           — full resolution chain for the active profile +
+                         preset. Uses :mod:`scripts.config.profiles` and
+                         :mod:`scripts.config.presets`; surfaces source
+                         (pack / profile / user / env / runtime / default)
+                         and per-knob overrides.
+* ``rule <name>``      — kernel vs tier-1 vs tier-2 placement plus the
+                         declared trigger list from ``router.json``.
+* ``route <text>``     — given prompt text, returns every tier-1 rule
+                         whose trigger list matches plus kernel rules
+                         (always active).
+
+Exit codes: ``0`` clean, ``1`` not found / no match, ``2`` invocation
+error (bad project root, malformed ``router.json``).
+"""
+from __future__ import annotations
+
+import argparse
+import json
+import os
+import sys
+from pathlib import Path
+from typing import Any
+
+from scripts._lib.agent_settings import (
+    DEFAULT_PROJECT_FILE,
+    ProjectRootError,
+    load_agent_settings,
+    resolve_project_root,
+)
+from scripts.config import presets, profiles
+
+ROUTER_FILENAME = "router.json"
+
+
+def _resolve_root(arg: str | None) -> tuple[Path, str]:
+    try:
+        return resolve_project_root(arg, cwd=Path.cwd())
+    except ProjectRootError as exc:
+        print(f"❌  explain: {exc}", file=sys.stderr)
+        raise SystemExit(2) from exc
+
+
+def _load_user_settings(project_root: Path) -> dict[str, Any]:
+    path = project_root / DEFAULT_PROJECT_FILE
+    if not path.exists():
+        return {}
+    return load_agent_settings(project_path=path) or {}
+
+
+def _load_router(project_root: Path) -> dict[str, Any]:
+    path = project_root / ROUTER_FILENAME
+    if not path.exists():
+        return {}
+    try:
+        return json.loads(path.read_text(encoding="utf-8"))
+    except (OSError, json.JSONDecodeError) as exc:
+        print(f"❌  explain: cannot read {path}: {exc}", file=sys.stderr)
+        raise SystemExit(2) from exc
+
+
+def _explain_config(project_root: Path, *, as_json: bool) -> int:
+    settings = _load_user_settings(project_root)
+    resolved_profile = profiles.resolve_profile(
+        project_root=project_root,
+        user_settings=settings,
+    )
+    resolved_preset = presets.resolve_preset(
+        project_root=project_root,
+        user_settings=settings,
+        profile_preset_id=resolved_profile.preset_id,
+    )
+    payload = {
+        "project_root": str(project_root),
+        "profile": {
+            "id": resolved_profile.id,
+            "source": resolved_profile.source,
+            "preset_id": resolved_profile.preset_id,
+            "warning": resolved_profile.warning,
+        },
+        "preset": {
+            "id": resolved_preset.id,
+            "source": resolved_preset.source,
+            "overrides": list(resolved_preset.overrides),
+            "knobs": resolved_preset.knobs,
+        },
+        "env": {
+            profiles.PROFILE_ID_ENV: os.environ.get(profiles.PROFILE_ID_ENV),
+            presets.PRESET_ID_ENV: os.environ.get(presets.PRESET_ID_ENV),
+        },
+    }
+    if as_json:
+        json.dump(payload, sys.stdout, indent=2, sort_keys=True)
+        sys.stdout.write("\n")
+        return 0
+    print(f"  📍  project_root: {project_root}")
+    print()
+    print(f"  profile.id:   {resolved_profile.id}  (source: {resolved_profile.source})")
+    if resolved_profile.warning:
+        print(f"  ⚠️   {resolved_profile.warning}")
+    print(f"  preset.id:    {resolved_preset.id}  (source: {resolved_preset.source})")
+    if resolved_preset.overrides:
+        print(f"  overrides:    {', '.join(resolved_preset.overrides)}")
+    cost = resolved_preset.knobs.get("cost", {})
+    if cost:
+        print(
+            f"  cost caps:    daily ${cost.get('daily_max_usd')} · "
+            f"weekly ${cost.get('weekly_max_usd')} · "
+            f"monthly ${cost.get('monthly_max_usd')}",
+        )
+    autonomy = resolved_preset.knobs.get("autonomy", {})
+    if autonomy:
+        print(f"  autonomy:     default={autonomy.get('default')}")
+    return 0
+
+
+def _find_rule(router: dict[str, Any], name: str) -> tuple[str, dict[str, Any]] | None:
+    if name in router.get("kernel", []):
+        return "kernel", {"id": name, "triggers": [{"always": True}]}
+    for tier in ("tier_1", "tier_2"):
+        for entry in router.get(tier, []):
+            if entry.get("id") == name:
+                return tier, entry
+    return None
+
+
+def _explain_rule(project_root: Path, name: str, *, as_json: bool) -> int:
+    router = _load_router(project_root)
+    found = _find_rule(router, name)
+    if found is None:
+        print(f"❌  explain: rule {name!r} not found in router", file=sys.stderr)
+        return 1
+    tier, entry = found
+    payload = {"rule": name, "tier": tier, "entry": entry}
+    if as_json:
+        json.dump(payload, sys.stdout, indent=2, sort_keys=True)
+        sys.stdout.write("\n")
+        return 0
+    print(f"  rule:    {name}")
+    print(f"  tier:    {tier}")
+    triggers = entry.get("triggers") or []
+    print(f"  triggers ({len(triggers)}):")
+    for trig in triggers:
+        print(f"    · {trig}")
+    routes = entry.get("routes_to") or []
+    if routes:
+        print(f"  routes_to: {', '.join(routes)}")
+    return 0
+
+
+def _matches_trigger(trigger: dict[str, Any], text: str, lowered: str) -> str | None:
+    """Return a human-readable match reason, or ``None`` for no match."""
+    if "keyword" in trigger:
+        kw = str(trigger["keyword"]).lower()
+        if kw and kw in lowered:
+            return f"keyword: {kw}"
+    if "phrase" in trigger:
+        ph = str(trigger["phrase"]).lower()
+        if ph and ph in lowered:
+            return f"phrase: {ph}"
+    if "path_prefix" in trigger:
+        prefix = str(trigger["path_prefix"])
+        if prefix and prefix in text:
+            return f"path_prefix: {prefix}"
+    return None
+
+
+def _explain_route(project_root: Path, text: str, *, as_json: bool) -> int:
+    router = _load_router(project_root)
+    lowered = text.lower()
+    matches: list[dict[str, Any]] = []
+    for entry in router.get("tier_1", []):
+        for trig in entry.get("triggers", []) or []:
+            reason = _matches_trigger(trig, text, lowered)
+            if reason is not None:
+                matches.append({
+                    "id": entry["id"], "tier": "tier_1", "reason": reason,
+                })
+                break
+    payload = {
+        "input": text,
+        "kernel_always": list(router.get("kernel", [])),
+        "tier_1_matches": matches,
+    }
+    if as_json:
+        json.dump(payload, sys.stdout, indent=2, sort_keys=True)
+        sys.stdout.write("\n")
+        return 0
+    print(f"  input: {text!r}")
+    print()
+    print(f"  kernel (always active, {len(payload['kernel_always'])}):")
+    for kid in payload["kernel_always"]:
+        print(f"    · {kid}")
+    print()
+    print(f"  tier-1 matches ({len(matches)}):")
+    if not matches:
+        print("    · (no trigger matched — only kernel rules active)")
+        return 1
+    for match in matches:
+        print(f"    · {match['id']}  ({match['reason']})")
+    return 0
+
+
+def main(argv: list[str] | None = None) -> int:
+    parser = argparse.ArgumentParser(
+        prog="agent-config explain",
+        description=(
+            "Print the decision chain behind a configuration or routing "
+            "outcome. Read-only; no network calls."
+        ),
+    )
+    parser.add_argument(
+        "subject", choices=("config", "rule", "route"),
+        help="what to explain",
+    )
+    parser.add_argument(
+        "target", nargs="?", default=None,
+        help="rule name (for 'rule') or prompt text (for 'route')",
+    )
+    parser.add_argument(
+        "--project", default=None,
+        help="project root (defaults to anchor walk from cwd)",
+    )
+    parser.add_argument(
+        "--json", action="store_true", dest="as_json",
+        help="emit JSON instead of human-readable text",
+    )
+    opts = parser.parse_args(argv)
+    project_root, _origin = _resolve_root(opts.project)
+    if opts.subject == "config":
+        return _explain_config(project_root, as_json=opts.as_json)
+    if opts.target is None:
+        print(
+            f"❌  explain: '{opts.subject}' requires a target argument",
+            file=sys.stderr,
+        )
+        return 2
+    if opts.subject == "rule":
+        return _explain_rule(project_root, opts.target, as_json=opts.as_json)
+    return _explain_route(project_root, opts.target, as_json=opts.as_json)
+
+
+if __name__ == "__main__":  # pragma: no cover
+    raise SystemExit(main())

package/scripts/_lib/bench_cost.py

@@ -0,0 +1,138 @@
+# Cost capture for `scripts/bench_run.py` — step-4 Phase 2 Step 2.
+#
+# Reads Claude Code session jsonl summaries (one summary line per session)
+# from agents/cost-tracking/sessions.jsonl — produced by scripts/cost/track.mjs
+# — and aggregates totals using model rates from bench/pricing.yaml.
+#
+# Returns the dict shape declared in docs/contracts/benchmark-report-schema.md
+# § JSON schema (v1) `cost`. When the source jsonl is missing, returns the
+# `unavailable` sentinel block (NEVER silently drops, per schema invariant).
+"""Cost capture helper for the bench runner."""
+from __future__ import annotations
+
+import json
+from pathlib import Path
+from typing import Any
+
+try:
+    import yaml
+except ImportError:  # pragma: no cover — bench_run handles the same import
+    yaml = None  # type: ignore[assignment]
+
+UNKNOWN_TIER = "unknown"
+TIER_KEYS = ("haiku", "sonnet", "opus", UNKNOWN_TIER)
+
+
+def load_pricing(pricing_path: Path) -> tuple[dict[str, dict[str, float]], str | None]:
+    """Return ({tier: rates}, oldest_sourced_on) from bench/pricing.yaml."""
+    if yaml is None or not pricing_path.is_file():
+        return {}, None
+    data = yaml.safe_load(pricing_path.read_text(encoding="utf-8")) or {}
+    rates: dict[str, dict[str, float]] = {}
+    oldest: str | None = None
+    for row in data.get("models", []):
+        tier = row.get("tier")
+        if not tier:
+            continue
+        rates[tier] = {
+            "input": float(row.get("input", 0.0)),
+            "output": float(row.get("output", 0.0)),
+            "cache_write": float(row.get("cache_write", 0.0)),
+            "cache_read": float(row.get("cache_read", 0.0)),
+        }
+        sourced = row.get("sourced_on")
+        # YAML 1.1 parses ISO dates to datetime.date; coerce to ISO string.
+        if sourced is not None and not isinstance(sourced, str):
+            sourced = sourced.isoformat() if hasattr(sourced, "isoformat") else str(sourced)
+        if isinstance(sourced, str) and (oldest is None or sourced < oldest):
+            oldest = sourced
+    return rates, oldest
+
+
+def _empty_totals() -> dict[str, int | float]:
+    return {
+        "input_tokens": 0,
+        "output_tokens": 0,
+        "cache_read_input_tokens": 0,
+        "cache_creation_input_tokens": 0,
+        "total_cost_usd": 0.0,
+    }
+
+
+def _empty_per_tier() -> dict[str, dict[str, int | float]]:
+    return {t: {"messages": 0, "cost_usd": 0.0} for t in TIER_KEYS}
+
+
+def unavailable_block(reason: str, source: str, pricing_sourced_on: str | None) -> dict[str, Any]:
+    """Schema-compliant `cost` block when no session jsonl is readable."""
+    return {
+        "source": "unavailable",
+        "reason": reason,
+        "scanned_path": source,
+        "sessions_scanned": 0,
+        "totals": _empty_totals(),
+        "per_tier": _empty_per_tier(),
+        "pricing_sourced_on": pricing_sourced_on,
+    }
+
+
+def aggregate_sessions(
+    sessions_jsonl: Path,
+    pricing_path: Path,
+) -> dict[str, Any]:
+    """Read agents/cost-tracking/sessions.jsonl and aggregate per-tier totals."""
+    rates, pricing_sourced_on = load_pricing(pricing_path)
+    if not sessions_jsonl.is_file():
+        return unavailable_block(
+            reason="sessions_jsonl_missing",
+            source=str(sessions_jsonl),
+            pricing_sourced_on=pricing_sourced_on,
+        )
+
+    totals = _empty_totals()
+    per_tier = _empty_per_tier()
+    sessions_scanned = 0
+
+    for line in sessions_jsonl.read_text(encoding="utf-8").splitlines():
+        if not line.strip():
+            continue
+        try:
+            summary = json.loads(line)
+        except json.JSONDecodeError:
+            continue
+        sessions_scanned += 1
+        for _model, slot in (summary.get("byModel") or {}).items():
+            tier = slot.get("tier", UNKNOWN_TIER)
+            if tier not in per_tier:
+                tier = UNKNOWN_TIER
+            totals["input_tokens"] += int(slot.get("input_tokens", 0))
+            totals["output_tokens"] += int(slot.get("output_tokens", 0))
+            totals["cache_read_input_tokens"] += int(slot.get("cache_read_input_tokens", 0))
+            totals["cache_creation_input_tokens"] += int(slot.get("cache_creation_input_tokens", 0))
+            cost = float(slot.get("cost_usd", 0.0))
+            # Recompute from rates if upstream cost is zero AND we have rates;
+            # otherwise trust the upstream attribution (it priced at capture time).
+            if cost == 0.0 and tier in rates:
+                r = rates[tier]
+                cost = (
+                    int(slot.get("input_tokens", 0)) / 1e6 * r["input"]
+                    + int(slot.get("output_tokens", 0)) / 1e6 * r["output"]
+                    + int(slot.get("cache_creation_input_tokens", 0)) / 1e6 * r["cache_write"]
+                    + int(slot.get("cache_read_input_tokens", 0)) / 1e6 * r["cache_read"]
+                )
+            per_tier[tier]["messages"] += int(slot.get("messages", 0))
+            per_tier[tier]["cost_usd"] += cost
+            totals["total_cost_usd"] += cost
+
+    # Round currency to 6 decimals for stable diffs.
+    totals["total_cost_usd"] = round(float(totals["total_cost_usd"]), 6)
+    for t in per_tier.values():
+        t["cost_usd"] = round(float(t["cost_usd"]), 6)
+
+    return {
+        "source": str(sessions_jsonl),
+        "sessions_scanned": sessions_scanned,
+        "totals": totals,
+        "per_tier": per_tier,
+        "pricing_sourced_on": pricing_sourced_on,
+    }

package/scripts/_lib/bench_quality.py

@@ -0,0 +1,118 @@
+# Quality probe for `scripts/bench_run.py` — step-4 Phase 2 Step 3.
+#
+# Each prompt declares `rubric.must_include` / `must_not_include` or a
+# `quality_assertion` regex (per docs/contracts/benchmark-corpus-spec.md).
+# When an agent-output file is passed via --agent-output, we score the
+# assertions against actual output. Without it, we emit `not_collected`
+# per docs/contracts/benchmark-report-schema.md § quality invariants.
+"""Quality probe helper for the bench runner."""
+from __future__ import annotations
+
+import json
+import re
+from pathlib import Path
+from typing import Any
+
+
+def _eval_rubric(rubric: dict[str, Any], output: str) -> tuple[bool, str]:
+    """Apply rubric.must_include / must_not_include / length_words to output."""
+    for phrase in rubric.get("must_include") or []:
+        if phrase not in output:
+            return False, f"missing: {phrase!r}"
+    for phrase in rubric.get("must_not_include") or []:
+        if phrase in output:
+            return False, f"forbidden: {phrase!r}"
+    bounds = rubric.get("length_words") or {}
+    if bounds:
+        words = len(output.split())
+        lo, hi = bounds.get("min", 0), bounds.get("max", 0)
+        if lo and words < lo:
+            return False, f"length<{lo}: {words}"
+        if hi and words > hi:
+            return False, f"length>{hi}: {words}"
+    return True, "ok"
+
+
+def _eval_regex(pattern: str, output: str) -> tuple[bool, str]:
+    try:
+        compiled = re.compile(pattern, re.MULTILINE)
+    except re.error as exc:
+        return False, f"bad_regex: {exc}"
+    return (bool(compiled.search(output)), "ok" if compiled.search(output) else "no_match")
+
+
+def _format_rubric(rubric: dict[str, Any]) -> str:
+    parts = []
+    if rubric.get("must_include"):
+        parts.append(f"must_include={rubric['must_include']}")
+    if rubric.get("must_not_include"):
+        parts.append(f"must_not_include={rubric['must_not_include']}")
+    if rubric.get("length_words"):
+        parts.append(f"length_words={rubric['length_words']}")
+    return " ".join(parts) or "<empty>"
+
+
+def score_corpus(
+    prompts: list[dict[str, Any]],
+    agent_output_path: Path | None,
+) -> dict[str, Any]:
+    """Return the `quality` block per benchmark-report-schema § quality."""
+    declared = [
+        p for p in prompts
+        if (p.get("rubric") or {}).get("must_include")
+        or (p.get("rubric") or {}).get("must_not_include")
+        or (p.get("rubric") or {}).get("length_words")
+        or p.get("quality_assertion")
+    ]
+    total_declared = len(declared)
+
+    if agent_output_path is None or not agent_output_path.is_file():
+        return {
+            "source": "not_collected",
+            "prompts_with_assertion": total_declared,
+            "prompts_passing": 0,
+            "quality_score": 0.0,
+            "per_prompt": [
+                {
+                    "id": p["id"],
+                    "assertion": p.get("quality_assertion") or _format_rubric(p.get("rubric") or {}),
+                    "assertion_kind": "quality_assertion" if p.get("quality_assertion") else "rubric",
+                    "passed": "not_collected",
+                }
+                for p in declared
+            ],
+        }
+
+    outputs = json.loads(agent_output_path.read_text(encoding="utf-8"))
+    per_prompt: list[dict[str, Any]] = []
+    passing = 0
+    for p in declared:
+        pid = p["id"]
+        output_text = str(outputs.get(pid, ""))
+        rubric = p.get("rubric") or {}
+        regex = p.get("quality_assertion")
+        if regex:
+            ok, _why = _eval_regex(regex, output_text)
+            kind = "quality_assertion"
+            assertion = regex
+        else:
+            ok, _why = _eval_rubric(rubric, output_text)
+            kind = "rubric"
+            assertion = _format_rubric(rubric)
+        per_prompt.append({
+            "id": pid,
+            "assertion": assertion,
+            "assertion_kind": kind,
+            "passed": ok,
+        })
+        if ok:
+            passing += 1
+
+    score = round(passing / total_declared, 4) if total_declared else 0.0
+    return {
+        "source": str(agent_output_path),
+        "prompts_with_assertion": total_declared,
+        "prompts_passing": passing,
+        "quality_score": score,
+        "per_prompt": per_prompt,
+    }