@intentsolutions/audit-harness 1.1.8 → 1.2.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -4,6 +4,56 @@ All notable changes are recorded here. Format follows [Keep a Changelog](https:/
4
4
 
5
5
  ## [Unreleased]
6
6
 
7
+ _Nothing yet._
8
+
9
+ ### Riding a future v2.1 routine release (descoped from 1.2.0)
10
+
11
+ - **OTel event-name polish (iah-E07b/c).** The `agent.rollout.gate.evaluated` and `gate.decision.emitted` event names are already locked + tested on main (PRs #78, #81 per NORMATIVE `intent-eval-lab/000-docs/067-AT-SPEC`). Any further attribute-schema polish on those events is deferred to a routine v2.1 release rather than headlined here — it is additive telemetry refinement, not a 1.2.0 capability boundary.
12
+
13
+ ## [1.2.1] - 2026-06-16
14
+
15
+ A patch release: release-pipeline supply-chain hardening (polyglot signing) plus
16
+ dev-dependency bumps. No CLI surface, runtime behavior, or API boundary changes —
17
+ the published artifacts are byte-identical in behavior to 1.2.0; only the release
18
+ machinery and dev tooling moved.
19
+
20
+ ### Changed — polyglot release signing wired into the publish pipeline (#90)
21
+
22
+ - **crates.io build-provenance attestation.** The `publish-crates` leg now emits a
23
+ GitHub build-provenance attestation for the published crate artifact, extending the
24
+ signed-supply-chain guarantee to the Rust distribution.
25
+ - **sigstore-python wheel + sdist signing.** The `publish-pypi` leg now signs the built
26
+ wheel and sdist with `sigstore-python` (keyless Fulcio OIDC + Rekor), so the PyPI
27
+ distribution carries verifiable provenance alongside the existing npm sigstore path.
28
+ - **crates.io publish is now active.** With `CARGO_REGISTRY_TOKEN` provisioned as a
29
+ repository secret, the `publish-crates` leg goes live on this tag — closing the
30
+ polyglot publish loop (npm + PyPI + crates.io all publish + sign from one tag).
31
+
32
+ ### Changed — dev-dependency bumps
33
+
34
+ - Bump `eslint` from 9.39.4 to 10.5.0 (#71).
35
+ - Bump `jeremylongshore/intent-rollout-gate` GitHub Action pin (#86).
36
+ - Bump `crate-ci/typos` from 1.29.4 to 1.47.2 (#87).
37
+
38
+ ## [1.2.0] - 2026-06-15
39
+
40
+ A minor release: the read-only "comprehensive audit, on any repo" brain (`classify` → `conform` → `audit` → `scan` → `currency`), the kernel-emitting evidence path (`emit-evidence` Evidence Bundle, E04), the provider credential gate (`cred-gate`, E08), shared vendorable lint configs (#85), and a golden-master fitness function — all additive, with the zero-runtime-dependency guarantee preserved.
41
+
42
+ ### Release narrative (what shipped since 1.1.8)
43
+
44
+ - **`emit-evidence` Evidence Bundle emitter (E04).** The CI-only signed-evidence path emits the harness's own deterministic self-gate as a kernel `gate-result/v1` row inside an `EvidenceBundle`, cosign-signs the canonical bytes (Fulcio OIDC + Rekor), and publishes a `report-manifest.json` the dashboard re-verifies at ingest. Detail under "CI-only signed evidence emit" below.
45
+ - **Provider credential gate (`cred-gate`, E08).** A new gate that asserts provider credentials PASS/FAIL with full redaction + spillover coverage (`scripts/cred-gate.sh`, fixtures via PR #80).
46
+ - **Shared, vendorable lint configs (#85).** `.audit-harness-configs/` (markdownlint / yamllint / ruff / shellcheck) is the canonical config set the IEP repos vendor + extend; `install.sh` now vendors both `scripts/` and `configs/`.
47
+ - **Dogfood AAR (iah-E10d).** First-downstream-adopter run captured at `000-docs/013-AA-AACR-rollout-gate-dogfood-iah-E10-2026-06-15.md`.
48
+
49
+ ### Apache-2.0 §4(d) NOTICE obligation — satisfied
50
+
51
+ `NOTICE` is present at the repo root, listed in `package.json#files` (ships in the npm tarball), included in the Python sdist + Rust crate distributions, AND vendored into `.audit-harness/` by `install.sh` (see "`install.sh` vendors NOTICE" below). The §4(d) attribution-travels-with-distribution obligation holds across npm, PyPI, crates.io, and the vendored-install path.
52
+
53
+ ### Why minor, not patch
54
+
55
+ Multiple new CLI verbs (`classify`, `conform`, `audit`, `scan`, `currency`, `cred-gate`) and new authored feature surfaces (shared lint configs, golden-master suite, the CI-only evidence emit). Per SemVer this is a minor bump. No CLI command was renamed or removed; the change is purely additive and the published tarball stays zero-runtime-dependency.
56
+
7
57
  ### Added — golden-master suite for gherkin-lint + crap-score stdout shapes (iah-golden-master)
8
58
 
9
59
  A fitness function that pins the raw stdout of the two scorers whose output is a downstream contract.
@@ -17,6 +17,7 @@ const COMMANDS = {
17
17
  'init': { script: 'harness-hash.sh', args: ['--init'] },
18
18
  'list': { script: 'harness-hash.sh', args: ['--list'] },
19
19
  'escape-scan': { script: 'escape-scan.sh', args: [] },
20
+ 'cred-gate': { script: 'cred-gate.sh', args: [] },
20
21
  'arch': { script: 'arch-check.sh', args: [] },
21
22
  'bias': { script: 'bias-count.sh', args: [] },
22
23
  'gherkin-lint': { script: 'gherkin-lint.sh', args: [] },
@@ -35,7 +36,7 @@ const COMMANDS = {
35
36
  // classify is intentionally NOT here: it emits a meaningful kill-switched profile
36
37
  // itself (every gate enforcement=disabled). verify/init/list always run.
37
38
  const KILLABLE_GATES = new Set([
38
- 'escape-scan', 'arch', 'bias', 'gherkin-lint', 'crap', 'emit-evidence',
39
+ 'escape-scan', 'cred-gate', 'arch', 'bias', 'gherkin-lint', 'crap', 'emit-evidence',
39
40
  ]);
40
41
 
41
42
  function usage() {
@@ -50,6 +51,15 @@ Commands:
50
51
  list List currently pinned files
51
52
  escape-scan <source> Scan a diff for escape attempts
52
53
  source: --staged | --range A..B | - (stdin) | path.patch
54
+ cred-gate [args...] Provider credential PASS/FAIL gate (iah-E08, CISO
55
+ binding DR-010 S1Q5). Reads a candidate artifact (the
56
+ JSON about to be signed/emitted) on stdin or --input and
57
+ FAILs (exit 1) if a declared secret value leaks verbatim,
58
+ a known provider-key shape is embedded, or the artifact
59
+ serializes the process environment (env-var spillover).
60
+ Offline + read-only. --secret-env NAME (repeatable)
61
+ declares a secret by env-var name; --json emits a
62
+ gate-result/v1 envelope. See docs/cred-gate.md.
53
63
  arch Run architecture-rule checks (Wall 7)
54
64
  bias Count test-bias patterns (tautology, smoke-only, etc.)
55
65
  gherkin-lint Advisory Gherkin quality check
@@ -0,0 +1,131 @@
1
+ # `cred-gate` — provider credential PASS/FAIL gate (iah-E08)
2
+
3
+ CISO non-negotiable per DR-010 S1Q5. Before any provider abstraction is allowed
4
+ to flow data into an Evidence Bundle / OTel signal / gate-result envelope, the
5
+ `cred-gate` gate proves — deterministically and offline — that:
6
+
7
+ 1. **Credential redaction** — no provider secret VALUE appears verbatim in the
8
+ candidate artifact (the JSON the runner is about to sign, the OTel line it is
9
+ about to emit, any log it captures). A leaked API key in a signed,
10
+ Rekor-anchored in-toto Statement is irreversible.
11
+ 2. **No env-var spillover** — the candidate artifact does not blindly serialize
12
+ the process environment. A provider key need not be named to leak: a wholesale
13
+ `env` dump spills every secret at once.
14
+
15
+ ## Usage
16
+
17
+ ```bash
18
+ # Candidate on stdin (the artifact about to be emitted/signed):
19
+ producer | audit-harness cred-gate
20
+
21
+ # Candidate from a file:
22
+ audit-harness cred-gate --input candidate.json
23
+
24
+ # Declare secrets by env-var NAME (the VALUE is read from the environment and
25
+ # never appears on the command line / in `ps`):
26
+ audit-harness cred-gate --secret-env ANTHROPIC_API_KEY --secret-env OPENAI_API_KEY < cand.json
27
+
28
+ # Emit a gate-result/v1 envelope, pipe-ready for emit-evidence:
29
+ audit-harness cred-gate --json < candidate.json | audit-harness emit-evidence
30
+ ```
31
+
32
+ ## Exit codes
33
+
34
+ | Code | Meaning |
35
+ | ---- | ------- |
36
+ | `0` | **PASS** — no secret value present, no provider-key shape, no env-var spillover |
37
+ | `1` | **FAIL** — a secret value leaked OR a provider-key shape matched OR env-var spillover detected |
38
+ | `2` | usage / input error (no candidate, unreadable `--input`) |
39
+
40
+ ## What it detects
41
+
42
+ ### Detected provider-key shapes (value-agnostic catalog)
43
+
44
+ These match the on-the-wire SHAPE of a known provider key, so a raw key is caught
45
+ even when it was not declared via `--secret-env`. Patterns are intentionally
46
+ specific to keep the false-positive rate low.
47
+
48
+ | Name | Shape (regex fragment) |
49
+ | ---- | ---------------------- |
50
+ | `anthropic-key` | `sk-ant-…` |
51
+ | `openai-key` | `sk-…` / `sk-proj-…` (excludes `sk-ant-`) |
52
+ | `groq-key` | `gsk_…` |
53
+ | `nvidia-key` | `nvapi-…` |
54
+ | `aws-access-key-id` | `AKIA…` |
55
+ | `google-api-key` | `AIza…` |
56
+ | `github-token` | `ghp_` / `gho_` / `ghs_` / `ghr_` / `ghu_…` |
57
+ | `slack-token` | `xoxb-` / `xoxa-` / `xoxp-` / `xoxr-` / `xoxs-…` |
58
+ | `private-key-block` | `-----BEGIN … PRIVATE KEY-----` |
59
+
60
+ ### Env-var spillover heuristics
61
+
62
+ | Name | What it catches |
63
+ | ---- | --------------- |
64
+ | `process-env-spread` | `...process.env` (JS object spread of the whole environment) |
65
+ | `os-environ-dump` | `dict(os.environ)` / a bare `os.environ` serialized into JSON |
66
+ | `env-block-key` | an `"env"` / `"environ"` / `"environment"` object key whose value is a `{…}` block |
67
+ | `printenv-capture` | a `printenv` / `/usr/bin/env` invocation captured into the artifact |
68
+
69
+ A spillover match is a hard **FAIL**: an environment dump inside a to-be-signed
70
+ artifact is exactly the irreversible leak this gate exists to stop.
71
+
72
+ ## False-positive posture
73
+
74
+ - **Declared secrets shorter than 8 chars are ignored** — a 1-char "secret"
75
+ would false-positive on virtually any artifact and is not a real credential.
76
+ - **The word "environment" in prose is NOT a spillover** — only the structural
77
+ `"env"/"environment": { … }` block shape, the `...process.env` spread, the
78
+ `os.environ` dump, or a `printenv` capture flag. (See the `tests/cred-gate`
79
+ FP-guard assertion.)
80
+ - The shape catalog is conservative by design; promotion from advisory to
81
+ blocking elsewhere in the harness follows `docs/gate-promotion.md`.
82
+
83
+ ## No re-leak guarantee
84
+
85
+ When a declared secret leaks, the FAIL finding **never echoes the secret value
86
+ back**. It reports only the value's length and a non-reversible SHA-256
87
+ fingerprint prefix, so the finding is actionable without re-leaking. The
88
+ `tests/cred-gate` suite asserts this explicitly.
89
+
90
+ ## Remediation when the gate FAILs
91
+
92
+ | Finding kind | Fix |
93
+ | ------------ | --- |
94
+ | `secret-value-leak` | Remove the literal secret from the artifact. Pass an opaque reference (key NAME, a hash, or a vault path) instead of the value. |
95
+ | `secret-shape-match` | A raw provider key is embedded. Strip it; if it is a real credential, treat it as compromised and rotate. |
96
+ | `env-spillover` | Stop serializing the whole environment. Allowlist the specific non-secret fields you actually need (`os.getenv("X")` per key), never `dict(os.environ)` / `{...process.env}`. |
97
+
98
+ ## Safety + scope
99
+
100
+ - **Offline + read-only**: never contacts a provider, never reads a real key
101
+ from disk, never writes.
102
+ - **Secret values via env-var NAME only**: `--secret-env NAME` reads `$NAME`
103
+ through indirect expansion; the value never appears on `argv` (so it is not
104
+ visible to `ps`), and the candidate + secret blob are passed to the python
105
+ analyzer through the environment, not the command line.
106
+ - **Kill-switch aware**: `cred-gate` is in `KILLABLE_GATES`, so
107
+ `AUDIT_HARNESS_DISABLE=1` no-ops it (exit 0, banner) like the other gates.
108
+ - **Timeout aware**: `AUDIT_HARNESS_TIMEOUT=N` supervises it like every gate.
109
+
110
+ ## CI (iah-E08c)
111
+
112
+ The `cred-gate` CI lane in `.github/workflows/ci.yml` runs
113
+ `tests/cred-gate/run-cred-gate-tests.sh`, which proves the credential-redaction
114
+ fixtures (E08a), the env-var spillover fixtures (E08b), the `--json` envelope
115
+ round-trip, and — because the same suite also exercises `emit-evidence.sh` — the
116
+ `gate.decision.emitted` OTel event (iah-E07b), which fires per the NORMATIVE
117
+ runtime event taxonomy (intent-eval-lab `067-AT-SPEC` § 2.2) with the
118
+ `gate.decision` enum `{pass, fail, advisory, error}` and the kernel-pinned
119
+ attribute spelling. Both the redaction group AND the spillover group must pass
120
+ for the lane to be green (iah-E08c "both must pass").
121
+
122
+ The fixture suite covers the **full catalog**: every provider-key shape in
123
+ `SHAPE_PATTERNS` (anthropic, openai, groq, nvidia, AWS, Google, GitHub, Slack,
124
+ private-key block) has a FAILing fixture built from a **synthetic, non-real**
125
+ value, and every `SPILLOVER_PATTERNS` heuristic (`process-env-spread`,
126
+ `os-environ-dump`, `env-block-key`, `printenv-capture`) has its own fixture — so
127
+ a regression in any single regex cannot ship silently green. Two PASS guards
128
+ (a non-matching value, and benign "environment" prose) pin the false-positive
129
+ posture. All fixtures are inline in the runner: synthetic secret values are
130
+ injected into the local environment for the duration of one assertion and never
131
+ touch `argv` (passed by env-var NAME via `--secret-env`).
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@intentsolutions/audit-harness",
3
- "version": "1.1.8",
3
+ "version": "1.2.1",
4
4
  "description": "Deterministic test-enforcement harness — escape-scan, hash-pinning, CRAP, architecture checks, bias detection, Gherkin lint. Companion to the audit-tests and implement-tests Claude Code skills.",
5
5
  "license": "Apache-2.0",
6
6
  "author": "Jeremy Longshore <jeremy@intentsolutions.io>",
@@ -46,7 +46,7 @@
46
46
  },
47
47
  "devDependencies": {
48
48
  "@eslint/js": "^9.39.4",
49
- "eslint": "^9.39.4",
49
+ "eslint": "^10.5.0",
50
50
  "lefthook": "^1.13.6"
51
51
  },
52
52
  "publishConfig": {
@@ -0,0 +1,120 @@
1
+ #!/usr/bin/env bash
2
+ # check-wrapper-sync.sh — assert the bundled wrapper-script mirrors are byte-identical
3
+ # to their canonical source under scripts/.
4
+ #
5
+ # WHY THIS EXISTS
6
+ # ---------------
7
+ # The Node package (bin/audit-harness.js) dispatches to the CANONICAL scripts under
8
+ # scripts/. The Python wrapper (intent-audit-harness on PyPI) and the Rust wrapper
9
+ # (intent-audit-harness on crates.io) cannot reach those canonical files at install
10
+ # time, so each BUNDLES a copy:
11
+ #
12
+ # * python/src/intent_audit_harness/scripts/<name> (packaged into the wheel)
13
+ # * rust/scripts/<name> (include_bytes!'d into the binary)
14
+ #
15
+ # Those copies are hand-maintained. On 2026-05-24 they were found ~1 month stale:
16
+ # the bundled crap-score.py was missing v1.1.1's --json evidence envelope, the
17
+ # `which_or_none("go")` PATH guard (silent crash on Go-less hosts), and the
18
+ # rglob->os.walk directory pruning. A user running
19
+ # `pip install intent-audit-harness && audit-harness crap` got the OLD gate.
20
+ # (Tracking bead: iah-python-wrapper-scripts-sync / bd_000-projects-65k4.)
21
+ #
22
+ # This gate makes that class of drift IMPOSSIBLE to merge silently: every bundled
23
+ # mirror MUST be a byte-for-byte copy of its canonical source. There is no
24
+ # wrapper-only delta — both wrappers invoke the script verbatim via bash/python3.
25
+ #
26
+ # RESYNC (when this gate REDs)
27
+ # ----------------------------
28
+ # bash scripts/check-wrapper-sync.sh --fix # copy canonical -> both mirrors
29
+ # then review + commit the result.
30
+ #
31
+ # Exit codes:
32
+ # 0 all mirrors in sync (or --fix completed)
33
+ # 1 drift detected (and not in --fix mode)
34
+ set -euo pipefail
35
+
36
+ # Resolve repo root from this script's own location so the gate works regardless
37
+ # of the caller's CWD (CI runs it from the repo root; a dev may run it elsewhere).
38
+ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
39
+ REPO_ROOT="$(cd "${SCRIPT_DIR}/.." && pwd)"
40
+ CANONICAL_DIR="${REPO_ROOT}/scripts"
41
+
42
+ # The set of scripts the Python + Rust wrappers DISPATCH. Keep this in lock-step
43
+ # with:
44
+ # * python/src/intent_audit_harness/cli.py (COMMANDS dict)
45
+ # * rust/src/main.rs (SCRIPTS array)
46
+ # If a wrapper starts dispatching a new canonical script, add it here AND to both
47
+ # wrapper sources, and copy it into both mirror dirs.
48
+ MIRRORED_SCRIPTS=(
49
+ "harness-hash.sh"
50
+ "escape-scan.sh"
51
+ "arch-check.sh"
52
+ "bias-count.sh"
53
+ "gherkin-lint.sh"
54
+ "crap-score.py"
55
+ )
56
+
57
+ # Each mirror directory that bundles a copy of the canonical scripts.
58
+ MIRROR_DIRS=(
59
+ "python/src/intent_audit_harness/scripts"
60
+ "rust/scripts"
61
+ )
62
+
63
+ FIX=0
64
+ if [[ "${1:-}" == "--fix" ]]; then
65
+ FIX=1
66
+ fi
67
+
68
+ drift_found=0
69
+ missing_canonical=0
70
+
71
+ for name in "${MIRRORED_SCRIPTS[@]}"; do
72
+ canonical="${CANONICAL_DIR}/${name}"
73
+ if [[ ! -f "${canonical}" ]]; then
74
+ echo "ERROR: canonical source missing: scripts/${name}" >&2
75
+ missing_canonical=1
76
+ continue
77
+ fi
78
+ for mdir in "${MIRROR_DIRS[@]}"; do
79
+ mirror="${REPO_ROOT}/${mdir}/${name}"
80
+ if [[ ! -f "${mirror}" ]]; then
81
+ echo "DRIFT: missing mirror ${mdir}/${name} (expected a copy of scripts/${name})" >&2
82
+ drift_found=1
83
+ if [[ "${FIX}" -eq 1 ]]; then
84
+ cp -f "${canonical}" "${mirror}"
85
+ echo " fixed: created ${mdir}/${name}"
86
+ fi
87
+ continue
88
+ fi
89
+ if ! diff -q "${canonical}" "${mirror}" >/dev/null 2>&1; then
90
+ echo "DRIFT: ${mdir}/${name} differs from canonical scripts/${name}" >&2
91
+ drift_found=1
92
+ if [[ "${FIX}" -eq 1 ]]; then
93
+ cp -f "${canonical}" "${mirror}"
94
+ echo " fixed: resynced ${mdir}/${name}"
95
+ fi
96
+ fi
97
+ done
98
+ done
99
+
100
+ if [[ "${missing_canonical}" -eq 1 ]]; then
101
+ echo "FAIL: one or more canonical scripts are missing — cannot verify mirror sync." >&2
102
+ exit 1
103
+ fi
104
+
105
+ if [[ "${FIX}" -eq 1 ]]; then
106
+ echo "check-wrapper-sync: --fix complete. Review + commit the resynced mirrors."
107
+ exit 0
108
+ fi
109
+
110
+ if [[ "${drift_found}" -eq 1 ]]; then
111
+ echo "" >&2
112
+ echo "FAIL: bundled wrapper mirrors are out of sync with canonical scripts/." >&2
113
+ echo " The Python (PyPI) and Rust (crates.io) packages would ship STALE gates." >&2
114
+ echo " Resync with: bash scripts/check-wrapper-sync.sh --fix" >&2
115
+ echo " then review + commit the result." >&2
116
+ exit 1
117
+ fi
118
+
119
+ echo "check-wrapper-sync: OK — all ${#MIRRORED_SCRIPTS[@]} bundled mirrors match canonical in ${#MIRROR_DIRS[@]} wrapper dirs."
120
+ exit 0
@@ -0,0 +1,238 @@
1
+ #!/usr/bin/env bash
2
+ # cred-gate.sh — Provider credential PASS/FAIL gate (iah-E08).
3
+ #
4
+ # CISO non-negotiable per DR-010 S1Q5: before any provider abstraction is allowed
5
+ # to flow data into an Evidence Bundle / OTel signal / gate-result envelope, two
6
+ # things MUST hold and are gated here, deterministically and offline:
7
+ #
8
+ # 1. CREDENTIAL REDACTION — no provider secret VALUE appears verbatim in the
9
+ # candidate artifact (the JSON the runner is about to sign, the OTel line it
10
+ # is about to emit, any log it captures). A leaked API key in a signed,
11
+ # Rekor-anchored Statement is irreversible.
12
+ #
13
+ # 2. ENV-VAR SPILLOVER — the candidate artifact does not blindly serialize the
14
+ # process environment (e.g. an `env` dump, a `process.env` spread, or a
15
+ # "context": {<all env>} block). A provider key need not be named to leak:
16
+ # a wholesale env dump spills every secret at once.
17
+ #
18
+ # This gate is READ-ONLY and OFFLINE. It never contacts a provider, never reads
19
+ # a real key from disk, and never writes. It inspects the candidate artifact you
20
+ # hand it (stdin or --input) against the secret values present in the environment
21
+ # (referenced by NAME via --secret-env, so the values never appear on the command
22
+ # line) plus a built-in catalog of provider-key SHAPES.
23
+ #
24
+ # It emits a gate-result/v1 envelope on stdout (--json) suitable for piping to
25
+ # emit-evidence, OR a human-readable PASS/FAIL summary (default).
26
+ #
27
+ # Usage:
28
+ # bash cred-gate.sh --input candidate.json
29
+ # <producer> | bash cred-gate.sh # candidate on stdin
30
+ # bash cred-gate.sh --secret-env ANTHROPIC_API_KEY --secret-env OPENAI_API_KEY < cand.json
31
+ # bash cred-gate.sh --json < candidate.json | bash emit-evidence.sh
32
+ #
33
+ # Flags:
34
+ # --input PATH Read the candidate artifact from PATH instead of stdin.
35
+ # --secret-env NAME Treat $NAME's VALUE as a secret that must NOT appear in the
36
+ # candidate. Repeatable. The value is read from the
37
+ # environment by name — it is never passed on argv.
38
+ # --json Emit a gate-result/v1 envelope (JSON) instead of text.
39
+ # --gate-id ID Override the gate_id in the envelope (default: provider-cred-gate).
40
+ # --help, -h Print help.
41
+ #
42
+ # Exit codes:
43
+ # 0 — PASS (no secret value present; no env-var spillover detected)
44
+ # 1 — FAIL (a secret value leaked OR an env-var spillover pattern matched)
45
+ # 2 — usage / input error (no candidate, unreadable --input)
46
+ #
47
+ # Failure-mode docs (iah-E08d): see docs/cred-gate.md for the catalog of detected
48
+ # shapes, the spillover heuristics, the false-positive posture, and remediation.
49
+
50
+ set -euo pipefail
51
+
52
+ # Bash version floor: align with the rest of the harness (jcgw).
53
+ [ "${BASH_VERSINFO:-0}" -ge 4 ] || { echo 'audit-harness requires bash >= 4' >&2; exit 2; }
54
+
55
+ INPUT="-"
56
+ EMIT_JSON=0
57
+ GATE_ID="provider-cred-gate"
58
+ SECRET_ENVS=()
59
+
60
+ while [[ $# -gt 0 ]]; do
61
+ case "$1" in
62
+ --input) INPUT="$2"; shift 2 ;;
63
+ --secret-env) SECRET_ENVS+=("$2"); shift 2 ;;
64
+ --json) EMIT_JSON=1; shift ;;
65
+ --gate-id) GATE_ID="$2"; shift 2 ;;
66
+ --help|-h) sed -n '2,46p' "$0"; exit 0 ;;
67
+ *) echo "cred-gate: unknown flag $1" >&2; exit 2 ;;
68
+ esac
69
+ done
70
+
71
+ # --- Read the candidate artifact ---
72
+ if [[ "$INPUT" == "-" ]]; then
73
+ CANDIDATE=$(cat)
74
+ else
75
+ if [[ ! -r "$INPUT" ]]; then
76
+ echo "cred-gate: cannot read $INPUT" >&2
77
+ exit 2
78
+ fi
79
+ CANDIDATE=$(cat "$INPUT")
80
+ fi
81
+
82
+ if [[ -z "$CANDIDATE" ]]; then
83
+ echo "cred-gate: empty candidate artifact" >&2
84
+ exit 2
85
+ fi
86
+
87
+ # Resolve the gate input hash (sha256 of the candidate bytes) so the emitted
88
+ # envelope's input_hash is coherent with what was actually inspected.
89
+ INPUT_HASH="sha256:$(printf '%s' "$CANDIDATE" | sha256sum | cut -d' ' -f1)"
90
+ # The policy is this script's own bytes — a content address of the gate logic.
91
+ POLICY_HASH="sha256:$(sha256sum "$0" | cut -d' ' -f1)"
92
+
93
+ # --- Collect the secret VALUES to redaction-check (by env-var name) ---
94
+ # Built as a NUL-delimited blob so values with newlines/spaces stay intact and
95
+ # never touch argv.
96
+ SECRET_VALUES_BLOB=""
97
+ for name in "${SECRET_ENVS[@]:-}"; do
98
+ [[ -z "$name" ]] && continue
99
+ # Indirect expansion: read $name's value without it ever appearing on argv.
100
+ val="${!name:-}"
101
+ # Skip empty / trivially short values: a 1-char "secret" would false-positive
102
+ # on virtually any artifact and is not a real credential.
103
+ [[ ${#val} -lt 8 ]] && continue
104
+ SECRET_VALUES_BLOB+="$val"$'\0'
105
+ done
106
+
107
+ # --- Deterministic analysis in python (offline; values via env, not argv) ---
108
+ # We pass the candidate + the secret blob + the catalog knobs through the
109
+ # environment so no secret value is ever visible in `ps`.
110
+ RESULT=$(
111
+ CANDIDATE="$CANDIDATE" \
112
+ SECRET_VALUES_BLOB="$SECRET_VALUES_BLOB" \
113
+ GATE_ID="$GATE_ID" \
114
+ python3 - <<'PY'
115
+ import json
116
+ import os
117
+ import re
118
+ import sys
119
+
120
+ candidate = os.environ["CANDIDATE"]
121
+
122
+ findings = [] # list of {"kind": ..., "detail": ...}
123
+
124
+ # --- 1. Credential redaction: explicit secret VALUES must not appear verbatim ---
125
+ blob = os.environ.get("SECRET_VALUES_BLOB", "")
126
+ secret_values = [v for v in blob.split("\0") if v]
127
+ for val in secret_values:
128
+ if val in candidate:
129
+ # NEVER echo the secret. Report only its length + a non-reversible
130
+ # fingerprint so the finding is actionable without re-leaking.
131
+ import hashlib
132
+
133
+ fp = hashlib.sha256(val.encode("utf-8")).hexdigest()[:12]
134
+ findings.append(
135
+ {
136
+ "kind": "secret-value-leak",
137
+ "detail": (
138
+ "a declared secret value (len=%d, sha256:%s...) appears "
139
+ "verbatim in the candidate artifact" % (len(val), fp)
140
+ ),
141
+ }
142
+ )
143
+
144
+ # --- 2. Credential redaction: provider-key SHAPES (value-agnostic catalog) ---
145
+ # Each pattern matches the literal on-the-wire shape of a known provider key.
146
+ # A match means a raw key is embedded even if it was not declared via
147
+ # --secret-env. Patterns are intentionally specific to keep the FP rate low.
148
+ SHAPE_PATTERNS = [
149
+ ("anthropic-key", r"sk-ant-[A-Za-z0-9_-]{20,}"),
150
+ # OpenAI keys start sk- but NOT sk-ant- (that's anthropic, matched above).
151
+ # The negative lookahead keeps the two findings disjoint.
152
+ ("openai-key", r"sk-(?!ant-)(?:proj-)?[A-Za-z0-9_-]{20,}"),
153
+ ("groq-key", r"gsk_[A-Za-z0-9]{20,}"),
154
+ ("nvidia-key", r"nvapi-[A-Za-z0-9_-]{20,}"),
155
+ ("aws-access-key-id", r"AKIA[0-9A-Z]{16}"),
156
+ ("google-api-key", r"AIza[0-9A-Za-z_-]{35}"),
157
+ ("github-token", r"gh[posru]_[A-Za-z0-9]{36,}"),
158
+ ("slack-token", r"xox[baprs]-[A-Za-z0-9-]{10,}"),
159
+ ("private-key-block", r"-----BEGIN (?:RSA |EC |OPENSSH )?PRIVATE KEY-----"),
160
+ ]
161
+ for name, pattern in SHAPE_PATTERNS:
162
+ if re.search(pattern, candidate):
163
+ findings.append(
164
+ {
165
+ "kind": "secret-shape-match",
166
+ "detail": "candidate contains a value matching the %s key shape"
167
+ % name,
168
+ }
169
+ )
170
+
171
+ # --- 3. Env-var spillover: wholesale environment serialization ---
172
+ # A provider key need not be NAMED to leak — a blanket env dump spills every
173
+ # secret at once. We flag the structural patterns that serialize the whole
174
+ # environment into the artifact.
175
+ SPILLOVER_PATTERNS = [
176
+ ("process-env-spread", r"\.\.\.\s*process\.env\b"),
177
+ ("os-environ-dump", r"\bdict\(\s*os\.environ\s*\)|\bos\.environ\b\s*[,}\]]"),
178
+ ("env-block-key", r'"(?:env|environ|environment)"\s*:\s*\{'),
179
+ ("printenv-capture", r"\b(?:printenv|/usr/bin/env)\b"),
180
+ ]
181
+ # These are heuristics: matching one is an ADVISORY-grade structural smell, but
182
+ # combined with an actual secret leak it is a hard FAIL. We treat any spillover
183
+ # match as a finding so the gate FAILs — an env dump in a to-be-signed artifact
184
+ # is exactly the irreversible leak this gate exists to stop.
185
+ for name, pattern in SPILLOVER_PATTERNS:
186
+ if re.search(pattern, candidate):
187
+ findings.append(
188
+ {
189
+ "kind": "env-spillover",
190
+ "detail": "candidate serializes the process environment via "
191
+ "the %s pattern" % name,
192
+ }
193
+ )
194
+
195
+ result = "FAIL" if findings else "PASS"
196
+ print(json.dumps({"result": result, "findings": findings}))
197
+ PY
198
+ )
199
+
200
+ # --- Parse the python result ---
201
+ GATE_RESULT=$(printf '%s' "$RESULT" | python3 -c "import json,sys; print(json.load(sys.stdin)['result'])")
202
+ FINDINGS_JSON=$(printf '%s' "$RESULT" | python3 -c "import json,sys; print(json.dumps(json.load(sys.stdin)['findings']))")
203
+ FINDING_COUNT=$(printf '%s' "$RESULT" | python3 -c "import json,sys; print(len(json.load(sys.stdin)['findings']))")
204
+
205
+ # --- Emit ---
206
+ if [[ "$EMIT_JSON" -eq 1 ]]; then
207
+ GATE_ID="$GATE_ID" GATE_RESULT="$GATE_RESULT" INPUT_HASH="$INPUT_HASH" \
208
+ POLICY_HASH="$POLICY_HASH" FINDINGS_JSON="$FINDINGS_JSON" \
209
+ python3 - <<'PY'
210
+ import json
211
+ import os
212
+
213
+ env = {
214
+ "gate_id": os.environ["GATE_ID"],
215
+ "result": os.environ["GATE_RESULT"],
216
+ "input_hash": os.environ["INPUT_HASH"],
217
+ "policy_hash": os.environ["POLICY_HASH"],
218
+ "metadata": {"findings": json.loads(os.environ["FINDINGS_JSON"])},
219
+ }
220
+ if env["result"] == "FAIL":
221
+ env["failure_mode"] = "provider_credential_leak"
222
+ print(json.dumps(env, separators=(",", ":")))
223
+ PY
224
+ else
225
+ if [[ "$GATE_RESULT" == "PASS" ]]; then
226
+ echo "cred-gate: PASS — no provider secret value present, no env-var spillover detected"
227
+ else
228
+ echo "cred-gate: FAIL — $FINDING_COUNT credential finding(s):" >&2
229
+ printf '%s' "$FINDINGS_JSON" | python3 -c "
230
+ import json, sys
231
+ for f in json.load(sys.stdin):
232
+ sys.stderr.write(' ⛔ [%s] %s\n' % (f['kind'], f['detail']))
233
+ "
234
+ echo "cred-gate: see docs/cred-gate.md for remediation (iah-E08d)." >&2
235
+ fi
236
+ fi
237
+
238
+ [[ "$GATE_RESULT" == "PASS" ]] && exit 0 || exit 1
@@ -191,36 +191,138 @@ if [[ -z "$STATEMENT" ]]; then
191
191
  exit 1
192
192
  fi
193
193
 
194
- # --- OTel event (best-effort no-op if collector absent) ---
195
- # Fire agent.rollout.gate.evaluated per intent-eval-lab/000-docs/001-DR-RFC-...md.
196
- # We emit a single OTLP-shaped JSON line to stderr when AUDIT_HARNESS_OTEL=1
197
- # OR an OTEL_EXPORTER_OTLP_ENDPOINT is set. Real exporter wiring is consumer-side;
198
- # we emit a structured signal that any collector can scrape via stderr capture.
194
+ # --- OTel events (best-effort no-op if collector absent) ---
195
+ # The gate-decision event fires per the NORMATIVE runtime event taxonomy
196
+ # intent-eval-lab/000-docs/067-AT-SPEC-runtime-event-taxonomy-2026-06-12.md § 2.2
197
+ # (GOVERNANCE events, `gate.*`):
198
+ #
199
+ # 1. agent.rollout.gate.evaluated — observability signal fired at the
200
+ # start/observation of a gate evaluation. NON-NORMATIVE: 067-AT-SPEC closes
201
+ # the `gate.*` category and does NOT define a gate-evaluated event, so this
202
+ # carries the legacy raw gate identity + result for collectors that already
203
+ # scrape it. It is NOT a 067-pinned name and a future taxonomy extension may
204
+ # retire or rename it; nothing should pin to it. The normative signal is (2).
205
+ # 2. gate.decision.emitted (iah-E07b) — fired at the END of the gate
206
+ # evaluation. This is the NORMATIVE name from 067-AT-SPEC § 2.2: "a
207
+ # RolloutGate decision row is emitted under gate-result/v1". Payload per
208
+ # § 2.2: gate.name (string), gate.decision (enum pass|fail|advisory|error),
209
+ # gate.policy_ref (string). This is the one a ship-gate dashboard alerts on.
210
+ #
211
+ # ATTRIBUTE-SPELLING AUTHORITY (do NOT redefine here): the canonical attribute
212
+ # names are pinned by the kernel at
213
+ # intent-eval-core/schemas/v1/otel-attributes.yaml — OTel-idiomatic dotted
214
+ # lowercase (e.g. gate.decision). We spell every attribute to match that file.
215
+ # 067-AT-SPEC § 2.2 is the EVENT-NAME authority for gate.decision.emitted and its
216
+ # payload schema; the gate.decision enum {pass, fail, advisory, error} is the
217
+ # closed gate-result/v1 verdict enum (Blueprint B § 7.4 / kernel gate-result
218
+ # schema) — NOT the RolloutGateDecision ship/no_ship vocabulary.
219
+ #
220
+ # We emit OTLP-shaped JSON lines to stderr when AUDIT_HARNESS_OTEL=1 OR an
221
+ # OTEL_EXPORTER_OTLP_ENDPOINT is set. Real exporter wiring is consumer-side; we
222
+ # emit a structured signal any collector can scrape via stderr capture. The path
223
+ # is fully best-effort: a collector being absent is the no-op default, and a
224
+ # python failure (||) degrades to an empty line that is simply not printed —
225
+ # the gate's own exit status is never affected by OTel emission (iah-E07c).
199
226
  if [[ "${AUDIT_HARNESS_OTEL:-0}" == "1" ]] || [[ -n "${OTEL_EXPORTER_OTLP_ENDPOINT:-}" ]]; then
200
227
  # Compose the JSON via python so every attribute value is JSON-escaped.
201
228
  # printf-interpolating gate_id/result/runner into a JSON format string
202
229
  # emitted structurally invalid JSON whenever a value carried a double quote
203
230
  # (e.g. AUDIT_HARNESS_SIDE='ci"injection' flowing into gate_id).
204
- OTEL_LINE=$(GATE_JSON="$GATE_JSON" RUNNER="$RUNNER" COMMIT_SHA="$COMMIT_SHA" TIMESTAMP="$TIMESTAMP" \
231
+ OTEL_LINES=$(GATE_JSON="$GATE_JSON" RUNNER="$RUNNER" COMMIT_SHA="$COMMIT_SHA" TIMESTAMP="$TIMESTAMP" \
205
232
  python3 - <<'PY' 2>/dev/null || echo ""
206
233
  import json, os
207
234
  try:
208
235
  gate = json.loads(os.environ["GATE_JSON"])
209
236
  except (json.JSONDecodeError, ValueError):
210
237
  gate = {}
211
- print(json.dumps({
238
+
239
+ runner = os.environ["RUNNER"]
240
+ commit_sha = os.environ["COMMIT_SHA"]
241
+ timestamp = os.environ["TIMESTAMP"]
242
+ gate_id = str(gate.get("gate_id", ""))
243
+ # The canonical gate-result/v1 verdict field is gate_decision (lowercase enum,
244
+ # Blueprint B § 7.4); the legacy draft envelope used `result` (UPPERCASE). Read
245
+ # the canonical field first, fall back to the legacy field.
246
+ gate_decision_raw = str(gate.get("gate_decision", gate.get("result", "")))
247
+
248
+ # gate.name / gate.policy_ref per 067-AT-SPEC § 2.2 payload schema. The canonical
249
+ # envelope carries gate_name (kebab-case) + policy_ref; fall back to gate_id /
250
+ # policy_hash for legacy draft envelopes that predate Blueprint B § 7.4.
251
+ gate_name = str(gate.get("gate_name", gate_id))
252
+ policy_ref = str(gate.get("policy_ref", gate.get("policy_hash", "")))
253
+
254
+ # Map the inbound verdict to the closed gate.decision enum {pass, fail,
255
+ # advisory, error} (gate-result/v1 / kernel gate-result schema). This is the
256
+ # 067-AT-SPEC § 2.2 enum — NOT the RolloutGateDecision ship/no_ship vocabulary.
257
+ # Canonical lowercase values pass straight through; legacy UPPERCASE results map
258
+ # down; an unrecognized/missing verdict is `error` (the gate could not affirm a
259
+ # decision — an error condition, not a clean `fail`).
260
+ _DECISION_MAP = {
261
+ "pass": "pass",
262
+ "fail": "fail",
263
+ "advisory": "advisory",
264
+ "error": "error",
265
+ }
266
+ decision = _DECISION_MAP.get(gate_decision_raw.strip().lower(), "error")
267
+ # An advisory_severity hint on a non-fail/non-error row signals an advisory row
268
+ # even when the legacy `result` field only said PASS.
269
+ if decision in ("pass",) and gate.get("advisory_severity"):
270
+ decision = "advisory"
271
+
272
+ reasons = []
273
+ if decision == "pass":
274
+ reasons.append(f"gate '{gate_id}' decision: pass")
275
+ else:
276
+ reasons.append(
277
+ f"gate '{gate_id}' decision: {decision} "
278
+ f"(verdict={gate_decision_raw or 'NO_VERDICT'})"
279
+ )
280
+ fm = gate.get("failure_mode")
281
+ if fm:
282
+ reasons.append(f"failure_mode: {fm}")
283
+
284
+ # Event 1: agent.rollout.gate.evaluated (NON-NORMATIVE observability signal;
285
+ # unchanged shape — not a 067-AT-SPEC-pinned name, see header note).
286
+ evaluated = {
212
287
  "name": "agent.rollout.gate.evaluated",
213
288
  "attributes": {
214
- "gate.id": str(gate.get("gate_id", "")),
215
- "gate.result": str(gate.get("result", "")),
216
- "gate.runner": os.environ["RUNNER"],
217
- "gate.commit_sha": os.environ["COMMIT_SHA"],
289
+ "gate.id": gate_id,
290
+ "gate.result": gate_decision_raw,
291
+ "gate.runner": runner,
292
+ "gate.commit_sha": commit_sha,
293
+ },
294
+ "timestamp": timestamp,
295
+ }
296
+
297
+ # Event 2: gate.decision.emitted (iah-E07b) — NORMATIVE per 067-AT-SPEC § 2.2.
298
+ # Payload: gate.name (string) + gate.decision (enum pass|fail|advisory|error) +
299
+ # gate.policy_ref (string). The reasons / runner / commit_sha are additive
300
+ # diagnostic attributes carried for dashboards; they do not contradict the
301
+ # § 2.2 required payload.
302
+ decision_event = {
303
+ "name": "gate.decision.emitted",
304
+ "attributes": {
305
+ "gate.name": gate_name,
306
+ "gate.decision": decision,
307
+ "gate.policy_ref": policy_ref,
308
+ "gate.id": gate_id,
309
+ "gate.reasons": reasons,
310
+ "gate.runner": runner,
311
+ "gate.commit_sha": commit_sha,
218
312
  },
219
- "timestamp": os.environ["TIMESTAMP"],
220
- }, separators=(",", ":")))
313
+ "timestamp": timestamp,
314
+ }
315
+
316
+ for ev in (evaluated, decision_event):
317
+ print(json.dumps(ev, separators=(",", ":")))
221
318
  PY
222
319
  )
223
- [[ -n "$OTEL_LINE" ]] && printf '[OTEL] %s\n' "$OTEL_LINE" >&2
320
+ # Print each emitted OTLP line with the [OTEL] marker the collector scrapes.
321
+ if [[ -n "$OTEL_LINES" ]]; then
322
+ while IFS= read -r _otel_line; do
323
+ [[ -n "$_otel_line" ]] && printf '[OTEL] %s\n' "$_otel_line" >&2
324
+ done <<< "$OTEL_LINES"
325
+ fi
224
326
  fi
225
327
 
226
328
  # --- Sign + emit ---