devlyn-cli 2.9.3 → 2.10.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +3 -3
- package/CLAUDE.md +5 -5
- package/README.md +5 -5
- package/benchmark/auto-resolve/fixtures/SCHEMA.md +1 -1
- package/benchmark/auto-resolve/scripts/pair_evidence_contract.py +1 -0
- package/benchmark/auto-resolve/scripts/run-fixture.sh +11 -1
- package/benchmark/auto-resolve/scripts/test-benchmark-arg-parsing.sh +83 -1
- package/bin/devlyn.js +108 -15
- package/config/skills/_shared/adapters/README.md +2 -2
- package/config/skills/_shared/adapters/grok.md +99 -0
- package/config/skills/_shared/archive_run.py +35 -1
- package/config/skills/_shared/collect-codex-findings.py +231 -50
- package/config/skills/_shared/engine-doctor.sh +6 -3
- package/config/skills/_shared/engine-preflight.md +12 -5
- package/config/skills/_shared/expected.schema.json +7 -0
- package/config/skills/_shared/grok-anchor-guard.py +61 -0
- package/config/skills/_shared/phase-prompt-render.py +181 -0
- package/config/skills/_shared/resolve-bootstrap.py +630 -0
- package/config/skills/_shared/resolve-stop-hook.py +325 -0
- package/config/skills/_shared/spec-verify-check.py +308 -31
- package/config/skills/_shared/state-phase-write.py +2966 -27
- package/config/skills/_shared/terminal-claim-check.py +537 -0
- package/config/skills/_shared/verify-merge-findings.py +533 -16
- package/config/skills/devlyn:engines/SKILL.md +2 -2
- package/config/skills/devlyn:ideate/SKILL.md +1 -1
- package/config/skills/devlyn:queue/SKILL.md +12 -0
- package/config/skills/devlyn:resolve/SKILL.md +60 -78
- package/config/skills/devlyn:resolve/references/phases/build-gate.md +2 -2
- package/config/skills/devlyn:resolve/references/phases/implement.md +1 -1
- package/config/skills/devlyn:resolve/references/phases/plan.md +1 -5
- package/config/skills/devlyn:resolve/references/phases/probe-derive.md +4 -2
- package/config/skills/devlyn:resolve/references/phases/surface-close.md +34 -0
- package/config/skills/devlyn:resolve/references/phases/verify.md +70 -97
- package/config/skills/devlyn:resolve/references/state-schema.md +26 -19
- package/optional-skills/devlyn:reap/SKILL.md +1 -1
- package/optional-skills/devlyn:reap/scripts/reap.sh +25 -13
- package/optional-skills/devlyn:reap/scripts/scan.sh +15 -1
- package/package.json +1 -1
- package/scripts/lint-fixtures.sh +4 -1
- package/scripts/lint-skills.sh +269 -77
package/AGENTS.md
CHANGED
|
@@ -10,7 +10,7 @@ This contract serves one goal: any capable engine — Claude, GPT/Codex, or a fu
|
|
|
10
10
|
|
|
11
11
|
Seven rules govern every change. Cite them by name when a decision touches one.
|
|
12
12
|
|
|
13
|
-
1. **No workaround** — fix the root cause, never the symptom. No `any`, no `@ts-ignore`, no silent `catch`, no hardcoded fallback that hides a broken contract.
|
|
13
|
+
1. **No workaround** — fix the root cause, never the symptom. No `any`, no `@ts-ignore`, no silent `catch`, no hardcoded fallback that hides a broken contract. Explicit `--engine`, `--risk-probes`, and `--pair-verify` routes fail closed; automatic VERIFY pair is capability-gated and reports an unavailable-engine solo route.
|
|
14
14
|
2. **No overengineering** — smallest change that closes the goal. New abstractions require an observed failure mode they prevent. Subtractive-first: ask "what can I delete instead?" before writing anything new.
|
|
15
15
|
3. **No guesswork** — verify with the actual files, logs, diffs, and run output before forming conclusions. State the falsifiable prediction BEFORE the experiment; record raw results AFTER.
|
|
16
16
|
4. **Worldclass** — code that survives review at a non-trivial codebase. Zero CRITICAL, zero HIGH security/design findings on the shippable path.
|
|
@@ -36,7 +36,7 @@ ideate (optional) -> resolve -> ship
|
|
|
36
36
|
|
|
37
37
|
Each skill's `SKILL.md` is the source of truth for flags and workflow. Do not duplicate.
|
|
38
38
|
|
|
39
|
-
Engine roles: orchestrator = whichever CLI the user opened (this contract is symmetric with CLAUDE.md; measured status 2026-07-05: Claude Code + omp run the full phase-gated pipeline; in the iter-0061 minimal-repo trivial-add shape on this machine, Codex CLI also ran it when this AGENTS.md was present in the project (ordinary invocation, 4/4); without project AGENTS.md, the same shape silently skipped the pipeline (iter-0040 F6, 4/4). If you are Codex orchestrating `/devlyn:resolve`, the phase machinery is mandatory regardless of task size; if you will not run it, say so explicitly and stop — do not silently degrade to ad-hoc execution); executor (
|
|
39
|
+
Engine roles: orchestrator = whichever CLI the user opened (this contract is symmetric with CLAUDE.md; measured status 2026-07-05: Claude Code + omp run the full phase-gated pipeline; in the iter-0061 minimal-repo trivial-add shape on this machine, Codex CLI also ran it when this AGENTS.md was present in the project (ordinary invocation, 4/4); without project AGENTS.md, the same shape silently skipped the pipeline (iter-0040 F6, 4/4). If you are Codex orchestrating `/devlyn:resolve`, the phase machinery is mandatory regardless of task size; if you will not run it, say so explicitly and stop — do not silently degrade to ad-hoc execution); executor (IMPLEMENT/CLEANUP + primary VERIFY judge) defaults to `claude`, overridable per run with `--engine <name>` or durably via machine-local `.devlyn/engines.json` `{"executor": "<name>", "pair_judge_priority": ["<name>", ...]}`; PLAN is orchestrator-fixed and never inherits `--engine` or the executor pin; pair judge = first available OTHER engine, default for VERIFY whenever available; risk probes remain conditional and --no-pair opts out. Pins fail closed (`BLOCKED:<engine>-unavailable` / `BLOCKED:invalid-engine-config`); new engines plug in by shipping `_shared/adapters/<name>.md`. `/devlyn:engines` (no args) shows the role table + detected engines; `executor <name>` / `pair <name>,...` / `clear` manage the pins. The executor pin binds the orchestrator in plain conversation too, not only inside a skill run: when you would implement directly and executor is pinned to a non-default engine, route that work through the pin — via `/devlyn:resolve` (reads it at PHASE 0) or by delegating to that engine — instead of editing as `claude`; no pin / no file → unchanged (`claude`). The pair-judge pin stays pipeline-scoped.
|
|
40
40
|
|
|
41
41
|
Conversational handoff + loop engineering (default entry): the orchestrating model — not the user — invokes the skills. `/devlyn:queue` fronts the intent queue (`docs/specs/queue.md`): no args = status, `add <intent>`, `drain` = serial unattended drain per the contract below. Large agreed work is written to `docs/specs/<id>/spec.md` (always a spec file — the user's reviewed contract; free-form large assumes-and-logs instead of halting, zero-scope-signal goals still halt to ideate), summarized once for user review, then run hands-free via `--spec`. Outer loop per task: findings-backed verdicts (NEEDS_WORK, verify/build-gate exhaustion) get spec amendment + re-run, max 3 iterations; infrastructure / invalid-input / engine-availability / implement-empty BLOCKED verdicts surface immediately. Unattended queue drain (`docs/specs/queue.md`, strictly serial): assumptions may only take scope-narrowing, reversible, non-user-visible defaults — material ambiguity marks the item `[F] needs-review` and the drain continues.
|
|
42
42
|
|
|
@@ -81,7 +81,7 @@ No silent fallbacks.
|
|
|
81
81
|
- Fallbacks allowed only when widely accepted and harmless (CSS fallback fonts, CDN failover, image placeholders).
|
|
82
82
|
- Silent `catch` blocks are bugs.
|
|
83
83
|
- Logging is not user-visible error handling.
|
|
84
|
-
-
|
|
84
|
+
- Engine availability follows **No workaround** principle 1: explicit routes fail closed; automatic routes are capability-gated and report a solo skip when OTHER is unavailable. No fallback.
|
|
85
85
|
|
|
86
86
|
## Evidence Over Claim
|
|
87
87
|
|
package/CLAUDE.md
CHANGED
|
@@ -10,7 +10,7 @@ This contract serves one goal: any capable engine — Claude, GPT/Codex, or a fu
|
|
|
10
10
|
|
|
11
11
|
Seven rules govern every change. Cite them by name when a decision touches one.
|
|
12
12
|
|
|
13
|
-
1. **No workaround** — fix the root cause, never the symptom.
|
|
13
|
+
1. **No workaround** — fix the root cause, never the symptom. No `any`, no `@ts-ignore`, no silent `catch`, no hardcoded fallback that hides a broken contract. No config bypasses. Explicit `--engine`, `--risk-probes`, and `--pair-verify` routes fail closed when a required engine is unavailable. Automatic risk probes remain high-risk-gated; VERIFY pair is default-when-available. An unavailable OTHER engine routes automatic VERIFY solo with a reported skip. `--no-pair` and `--no-risk-probes` remain explicit opt-outs.
|
|
14
14
|
2. **No overengineering** — smallest change that closes the goal. New abstractions require an observed failure mode they prevent. Subtractive-first: ask "what can I delete instead?" before writing anything new.
|
|
15
15
|
3. **No guesswork** — verify with the actual files, logs, diffs, and run output before forming conclusions. State the falsifiable prediction BEFORE the experiment; record raw results AFTER. Retroactive prediction edits are dishonest.
|
|
16
16
|
4. **Worldclass** — code that survives review at a non-trivial codebase. Zero CRITICAL, zero HIGH security/design findings on the shippable path.
|
|
@@ -40,8 +40,8 @@ Each skill's `SKILL.md` is the source of truth for its flags and workflow — do
|
|
|
40
40
|
| Role | Default | Manual override |
|
|
41
41
|
|---|---|---|
|
|
42
42
|
| Orchestrator — conversation, handoff, loop driving | whichever CLI you open (contract is symmetric: CLAUDE.md ↔ AGENTS.md). Measured status 2026-07-05: Claude Code + omp run the full phase-gated pipeline; in the iter-0061 minimal-repo trivial-add shape on this machine, Codex CLI also ran it when the project carried the devlyn AGENTS.md (ordinary invocation, 4/4); without project AGENTS.md, the same shape silently skipped the pipeline (iter-0040 F6, 4/4) | switch CLIs; the file artifacts (spec/queue/state) carry over |
|
|
43
|
-
| Executor —
|
|
44
|
-
| Pair judge — VERIFY
|
|
43
|
+
| Executor — IMPLEMENT/CLEANUP + primary VERIFY judge; PLAN is orchestrator-fixed and never inherits `--engine` or an executor pin | `claude` | `--engine <name>` per run, or `/devlyn:engines executor <name>` (durable pin) |
|
|
44
|
+
| Pair judge — default for VERIFY; conditional for risk probes | first available OTHER engine (claude↔codex) | `/devlyn:engines pair <name>,...`; `--no-pair` opts out |
|
|
45
45
|
|
|
46
46
|
`/devlyn:engines` with no args shows the current role table, detected engines, and how to pin or clear — the pins live in `.devlyn/engines.json`.
|
|
47
47
|
|
|
@@ -158,14 +158,14 @@ A finding without one of these forms is excluded. Vague findings produce vague f
|
|
|
158
158
|
|
|
159
159
|
## Codex invocation
|
|
160
160
|
|
|
161
|
-
When `/devlyn:resolve` or `/devlyn:ideate` route a phase to Codex (`--engine codex
|
|
161
|
+
When `/devlyn:resolve` or `/devlyn:ideate` route a phase to Codex (`--engine codex`, default VERIFY pair, or conditional risk-probe routing), the wrapper-form contract lives in `config/skills/_shared/codex-config.md` (or `.claude/skills/_shared/codex-config.md` once installed). Omit `-m <model>` — the CLI's current flagship is used automatically. MCP is not in the loop. If Codex is required and unavailable, stop with `BLOCKED:codex-unavailable` and setup guidance.
|
|
162
162
|
|
|
163
163
|
## Working Mode
|
|
164
164
|
|
|
165
165
|
- **Checkpoint with TaskCreate / TaskUpdate.** Long investigations or multi-phase work: create tasks at start, mark completed as each one closes — don't batch.
|
|
166
166
|
- **Don't stop early on token-budget concerns.** Context auto-compacts; the model resumes after compaction. Run the work to a real stopping point.
|
|
167
167
|
- **Persist across context windows via disk.** `/devlyn:resolve` writes `.devlyn/pipeline.state.json` plus per-phase log/findings under `.devlyn/runs/<run_id>/`; for ad-hoc long work use `HANDOFF.md` and resume with `@HANDOFF.md continue`.
|
|
168
|
-
- **Parallelize independent tool calls; reserve `Agent` subagents for independent fan-out or isolated-context verification** — a single perspective is the default
|
|
168
|
+
- **Parallelize independent tool calls; reserve `Agent` subagents for independent fan-out or isolated-context verification** — a single perspective is the default outside VERIFY's dual-judge path.
|
|
169
169
|
|
|
170
170
|
## Skill Boundary Policy
|
|
171
171
|
|
package/README.md
CHANGED
|
@@ -106,10 +106,10 @@ Stack tasks to run back-to-back without supervision. `/devlyn:queue add "<intent
|
|
|
106
106
|
devlyn separates three engine roles:
|
|
107
107
|
|
|
108
108
|
- **Orchestrator** — the CLI you opened (Claude Code, Codex, or omp) that drives the conversation and loop. The contract is symmetric (`CLAUDE.md` ↔ `AGENTS.md`), so the same phase-gated pipeline runs whichever you launch; the file artifacts (spec, queue, state) carry over if you switch.
|
|
109
|
-
- **Executor** —
|
|
110
|
-
- **Pair judge** — the first available *other* engine, for
|
|
109
|
+
- **Executor** — IMPLEMENT / CLEANUP plus the primary VERIFY judge. Defaults to `claude`. PLAN is orchestrator-fixed and never inherits `--engine` or an executor pin.
|
|
110
|
+
- **Pair judge** — the first available *other* engine, default for VERIFY and conditional for risk probes.
|
|
111
111
|
|
|
112
|
-
`--engine claude` (default) is the canonical implementation surface for
|
|
112
|
+
`--engine claude` (default) is the canonical implementation surface for IMPLEMENT, BUILD_GATE, and CLEANUP; PLAN always runs on the orchestrator's native worker. VERIFY/JUDGE runs pair mode by default when the OTHER engine is available.
|
|
113
113
|
|
|
114
114
|
Pin roles durably with `/devlyn:engines` (no args shows the role table + detected engines; `executor <name>` / `pair <name>,...` / `clear` manage the pins, stored machine-local in `.devlyn/engines.json`). Pins fail closed: an unavailable pinned engine stops with `BLOCKED:<engine>-unavailable`, and a name with no `_shared/adapters/<name>.md` adapter stops with `BLOCKED:invalid-engine-config`. New engines plug in by shipping an adapter file — no skill changes.
|
|
115
115
|
|
|
@@ -119,7 +119,7 @@ Pin roles durably with `/devlyn:engines` (no args shows the role table + detecte
|
|
|
119
119
|
/devlyn:resolve "fix the auth bug" --engine codex # research-only
|
|
120
120
|
```
|
|
121
121
|
|
|
122
|
-
If Codex or Claude is absent when explicitly selected or
|
|
122
|
+
If Codex or Claude is absent when explicitly selected, or OTHER engine is absent under `--pair-verify`, the harness stops with `BLOCKED:<engine>-unavailable` and prints setup guidance. Automatic VERIFY absence is a reported solo route. Use `--no-pair` only when intentionally accepting solo VERIFY; use `--no-risk-probes` only when intentionally disabling automatic high-risk probes.
|
|
123
123
|
|
|
124
124
|
### Benchmark score runs
|
|
125
125
|
|
|
@@ -259,7 +259,7 @@ Selected during install. Run `npx devlyn-cli` again to add more.
|
|
|
259
259
|
|---|---|
|
|
260
260
|
| `playwright` | Playwright MCP — powers `/devlyn:resolve` BUILD_GATE browser tier (Chrome MCP → Playwright → curl fallback) |
|
|
261
261
|
|
|
262
|
-
> `--engine codex` and
|
|
262
|
+
> `--engine codex` and default-when-available VERIFY pair mode use the local `codex` CLI binary, not MCP. Install from https://platform.openai.com/docs/codex, run the current Codex auth/login flow, verify `codex --version`, then rerun.
|
|
263
263
|
|
|
264
264
|
</details>
|
|
265
265
|
|
|
@@ -123,7 +123,7 @@ Machine-readable acceptance criteria used by both `run-fixture.sh` verification
|
|
|
123
123
|
|
|
124
124
|
Fixtures with `metadata.json` `category: "high-risk"` must include at least
|
|
125
125
|
one resolve risk-trigger term in `metadata.intent` or `spec.md`, matching the
|
|
126
|
-
conditional
|
|
126
|
+
conditional risk-probe triggers used by `/devlyn:resolve`: security/auth,
|
|
127
127
|
money/pricing/tax/ledger, persistence/data mutation, idempotency/replay,
|
|
128
128
|
API/webhook/signature, allocation/scheduling/inventory/rollback/transaction,
|
|
129
129
|
priority, or output/response-shape contracts. This keeps future pair-evidence
|
|
@@ -744,8 +744,18 @@ verify = {"commands": [], "forbidden_pattern_hits": [], "deps_added": 0,
|
|
|
744
744
|
|
|
745
745
|
for vc in expected.get("verification_commands", []):
|
|
746
746
|
try:
|
|
747
|
+
timeout_sec = vc.get("timeout_sec", 60)
|
|
748
|
+
if (
|
|
749
|
+
isinstance(timeout_sec, bool)
|
|
750
|
+
or not isinstance(timeout_sec, int)
|
|
751
|
+
or not 1 <= timeout_sec <= 600
|
|
752
|
+
):
|
|
753
|
+
raise ValueError(
|
|
754
|
+
"timeout_sec must be an integer from 1 to 600 (bool is not allowed)"
|
|
755
|
+
)
|
|
747
756
|
proc = subprocess.run(vc["cmd"], cwd=work, shell=True, env=verify_env,
|
|
748
|
-
capture_output=True, text=True,
|
|
757
|
+
capture_output=True, text=True,
|
|
758
|
+
timeout=timeout_sec)
|
|
749
759
|
out = (proc.stdout or "") + (proc.stderr or "")
|
|
750
760
|
ok_exit = proc.returncode == vc.get("exit_code", 0)
|
|
751
761
|
ok_contains = all(s in out for s in vc.get("stdout_contains", []))
|
|
@@ -5,7 +5,7 @@ set -euo pipefail
|
|
|
5
5
|
ROOT="$(cd "$(dirname "$0")/../../.." && pwd)"
|
|
6
6
|
TMP="$(mktemp -d)"
|
|
7
7
|
BENCH_ROOT="$ROOT/benchmark/auto-resolve"
|
|
8
|
-
trap 'rm -rf "$TMP"; rm -rf "$BENCH_ROOT/results/arg-parse-command-test" "$BENCH_ROOT/results/arg-parse-discovery-test" "$BENCH_ROOT/results/arg-parse-shadow-suite-dry-run" "$BENCH_ROOT/results/arg-parse-shadow-cli-suite-dry-run" "$BENCH_ROOT/results/arg-parse-variant-path" "$BENCH_ROOT/results/arg-parse-headroom-cli-replay" "$BENCH_ROOT/results/arg-parse-pair-cli-replay" "$BENCH_ROOT/results/arg-parse-shadow-judge" "$BENCH_ROOT/results/arg-parse-opus-bad-mapping" "$BENCH_ROOT/results/arg-parse-opus-malformed-mapping" "$BENCH_ROOT/results/arg-parse-opus-malformed-score" "$BENCH_ROOT/results/arg-parse-opus-invalid-generated-score" "$BENCH_ROOT/results/arg-parse-opus-invalid-generated-dq" "$BENCH_ROOT/results/arg-parse-opus-summary-mapping" "$BENCH_ROOT/results/arg-parse-opus-summary-null-margin"; rm -rf /tmp/bench-arg-parse-variant-path-* /tmp/bench-arg-parse-headroom-cli-replay-*' EXIT
|
|
8
|
+
trap 'rm -rf "$TMP"; rm -rf "$BENCH_ROOT/results/arg-parse-command-test" "$BENCH_ROOT/results/arg-parse-discovery-test" "$BENCH_ROOT/results/arg-parse-shadow-suite-dry-run" "$BENCH_ROOT/results/arg-parse-shadow-cli-suite-dry-run" "$BENCH_ROOT/results/arg-parse-variant-path" "$BENCH_ROOT/results/arg-parse-timeout-budget" "$BENCH_ROOT/results/arg-parse-headroom-cli-replay" "$BENCH_ROOT/results/arg-parse-pair-cli-replay" "$BENCH_ROOT/results/arg-parse-shadow-judge" "$BENCH_ROOT/results/arg-parse-opus-bad-mapping" "$BENCH_ROOT/results/arg-parse-opus-malformed-mapping" "$BENCH_ROOT/results/arg-parse-opus-malformed-score" "$BENCH_ROOT/results/arg-parse-opus-invalid-generated-score" "$BENCH_ROOT/results/arg-parse-opus-invalid-generated-dq" "$BENCH_ROOT/results/arg-parse-opus-summary-mapping" "$BENCH_ROOT/results/arg-parse-opus-summary-null-margin" "$BENCH_ROOT/shadow-fixtures/arg-parse-timeout-budget"; rm -rf /tmp/bench-arg-parse-variant-path-* /tmp/bench-arg-parse-timeout-budget-* /tmp/bench-arg-parse-headroom-cli-replay-*' EXIT
|
|
9
9
|
|
|
10
10
|
expect_fail_contains() {
|
|
11
11
|
local name="$1"
|
|
@@ -849,6 +849,88 @@ if grep -Fq -- '--engine auto' \
|
|
|
849
849
|
exit 1
|
|
850
850
|
fi
|
|
851
851
|
mkdir -p "$BENCH_ROOT/shadow-fixtures"
|
|
852
|
+
rm -rf "$BENCH_ROOT/shadow-fixtures/arg-parse-timeout-budget" \
|
|
853
|
+
"$BENCH_ROOT/results/arg-parse-timeout-budget"
|
|
854
|
+
cp -R "$BENCH_ROOT/fixtures/F1-cli-trivial-flag" \
|
|
855
|
+
"$BENCH_ROOT/shadow-fixtures/arg-parse-timeout-budget"
|
|
856
|
+
python3 - "$BENCH_ROOT/shadow-fixtures/arg-parse-timeout-budget/expected.json" <<'PY'
|
|
857
|
+
import json
|
|
858
|
+
import pathlib
|
|
859
|
+
import sys
|
|
860
|
+
|
|
861
|
+
path = pathlib.Path(sys.argv[1])
|
|
862
|
+
expected = json.loads(path.read_text())
|
|
863
|
+
expected["verification_commands"] = [
|
|
864
|
+
{"cmd": "true"},
|
|
865
|
+
{"cmd": "true", "timeout_sec": 1},
|
|
866
|
+
{"cmd": "true", "timeout_sec": 600},
|
|
867
|
+
{"cmd": "true", "timeout_sec": True},
|
|
868
|
+
{"cmd": "true", "timeout_sec": 601},
|
|
869
|
+
{
|
|
870
|
+
"cmd": "python3 -c \"import time; time.sleep(2)\"",
|
|
871
|
+
"timeout_sec": 1,
|
|
872
|
+
},
|
|
873
|
+
{
|
|
874
|
+
"cmd": "python3 -c \"import time; time.sleep(1)\"",
|
|
875
|
+
"timeout_sec": 5,
|
|
876
|
+
},
|
|
877
|
+
]
|
|
878
|
+
path.write_text(json.dumps(expected, indent=2) + "\n")
|
|
879
|
+
PY
|
|
880
|
+
bash "$ROOT/benchmark/auto-resolve/scripts/run-fixture.sh" \
|
|
881
|
+
--fixture arg-parse-timeout-budget \
|
|
882
|
+
--arm bare \
|
|
883
|
+
--run-id arg-parse-timeout-budget \
|
|
884
|
+
--dry-run > "$TMP/timeout-budget-dry-run.out" 2>&1
|
|
885
|
+
python3 - "$BENCH_ROOT/results/arg-parse-timeout-budget/arg-parse-timeout-budget/bare/verify.json" <<'PY'
|
|
886
|
+
import json
|
|
887
|
+
import pathlib
|
|
888
|
+
import sys
|
|
889
|
+
|
|
890
|
+
commands = json.loads(pathlib.Path(sys.argv[1]).read_text())["commands"]
|
|
891
|
+
assert commands[0]["pass"] is True, commands
|
|
892
|
+
assert commands[1]["pass"] is True, commands
|
|
893
|
+
assert commands[2]["pass"] is True, commands
|
|
894
|
+
expected_error = (
|
|
895
|
+
"error:ValueError:timeout_sec must be an integer from 1 to 600 "
|
|
896
|
+
"(bool is not allowed)"
|
|
897
|
+
)
|
|
898
|
+
assert commands[3]["pass"] is False and commands[3]["reason"] == expected_error, commands
|
|
899
|
+
assert commands[4]["pass"] is False and commands[4]["reason"] == expected_error, commands
|
|
900
|
+
assert commands[5]["reason"] == "timeout", commands
|
|
901
|
+
assert commands[6]["pass"] is True, commands
|
|
902
|
+
PY
|
|
903
|
+
|
|
904
|
+
TIMEOUT_LINT_ROOT="$TMP/timeout-lint-fixtures"
|
|
905
|
+
mkdir -p "$TIMEOUT_LINT_ROOT"
|
|
906
|
+
cp -R "$BENCH_ROOT/fixtures/F1-cli-trivial-flag" \
|
|
907
|
+
"$TIMEOUT_LINT_ROOT/F1-cli-trivial-flag"
|
|
908
|
+
python3 - "$TIMEOUT_LINT_ROOT/F1-cli-trivial-flag/expected.json" <<'PY'
|
|
909
|
+
import json
|
|
910
|
+
import pathlib
|
|
911
|
+
import sys
|
|
912
|
+
|
|
913
|
+
path = pathlib.Path(sys.argv[1])
|
|
914
|
+
expected = json.loads(path.read_text())
|
|
915
|
+
expected["verification_commands"][0]["timeout_sec"] = 600
|
|
916
|
+
path.write_text(json.dumps(expected, indent=2) + "\n")
|
|
917
|
+
PY
|
|
918
|
+
DEVLYN_FIXTURES_DIR="$TIMEOUT_LINT_ROOT" DEVLYN_LINT_FIXTURES_NO_JSONSCHEMA=1 \
|
|
919
|
+
bash "$ROOT/scripts/lint-fixtures.sh" > "$TMP/timeout-lint-pass.out" 2>&1
|
|
920
|
+
python3 - "$TIMEOUT_LINT_ROOT/F1-cli-trivial-flag/expected.json" <<'PY'
|
|
921
|
+
import json
|
|
922
|
+
import pathlib
|
|
923
|
+
import sys
|
|
924
|
+
|
|
925
|
+
path = pathlib.Path(sys.argv[1])
|
|
926
|
+
expected = json.loads(path.read_text())
|
|
927
|
+
expected["verification_commands"][0]["timeout_sec"] = True
|
|
928
|
+
path.write_text(json.dumps(expected, indent=2) + "\n")
|
|
929
|
+
PY
|
|
930
|
+
expect_fail_contains timeout-lint-bool \
|
|
931
|
+
"timeout_sec must be an integer from 1 to 600" \
|
|
932
|
+
env DEVLYN_FIXTURES_DIR="$TIMEOUT_LINT_ROOT" DEVLYN_LINT_FIXTURES_NO_JSONSCHEMA=1 \
|
|
933
|
+
bash "$ROOT/scripts/lint-fixtures.sh"
|
|
852
934
|
rm -rf "$BENCH_ROOT/shadow-fixtures/arg-parse-nan-metadata" \
|
|
853
935
|
"$BENCH_ROOT/shadow-fixtures/arg-parse-nan-expected" \
|
|
854
936
|
"$BENCH_ROOT/results/arg-parse-nan-metadata" \
|
package/bin/devlyn.js
CHANGED
|
@@ -16,6 +16,7 @@ const PKG = require('../package.json');
|
|
|
16
16
|
// ui-*) stay Claude-core-only, matching the pre-existing Codex bundle.
|
|
17
17
|
const DEVLYN_CORE_SKILLS = ['devlyn:resolve', 'devlyn:ideate', 'devlyn:design-ui', 'devlyn:engines', 'devlyn:queue', '_shared'];
|
|
18
18
|
const DEVLYN_SKILL_DIR_STAMP = '__DEVLYN_SKILL_DIR__';
|
|
19
|
+
const DEVLYN_INSTALL_MARKER = '.devlyn-install.json';
|
|
19
20
|
|
|
20
21
|
// Cross-agent shared skills directory read by BOTH oh-my-pi and Pi. Verified
|
|
21
22
|
// from the omp binary's skill-provider strings ("skills from .agents/skills —
|
|
@@ -354,6 +355,39 @@ function copyRecursive(src, dest, baseDir) {
|
|
|
354
355
|
}
|
|
355
356
|
}
|
|
356
357
|
|
|
358
|
+
function clearInstallMarker(skillsDir) {
|
|
359
|
+
fs.rmSync(path.join(skillsDir, DEVLYN_INSTALL_MARKER), { force: true });
|
|
360
|
+
}
|
|
361
|
+
|
|
362
|
+
function assertCompleteSkillInstall(sourceSkillsDir, skillsDir, skillNames) {
|
|
363
|
+
const missing = skillNames.filter((skillName) =>
|
|
364
|
+
!fs.existsSync(path.join(sourceSkillsDir, skillName))
|
|
365
|
+
|| !fs.existsSync(path.join(skillsDir, skillName)));
|
|
366
|
+
if (missing.length > 0) {
|
|
367
|
+
throw new Error(`Incomplete devlyn skill install; missing: ${missing.join(', ')}`);
|
|
368
|
+
}
|
|
369
|
+
}
|
|
370
|
+
|
|
371
|
+
function writeInstallMarker(skillsDir) {
|
|
372
|
+
const markerPath = path.join(skillsDir, DEVLYN_INSTALL_MARKER);
|
|
373
|
+
const tempPath = `${markerPath}.${process.pid}.tmp`;
|
|
374
|
+
const marker = {
|
|
375
|
+
schemaVersion: 1,
|
|
376
|
+
package: PKG.name,
|
|
377
|
+
version: PKG.version,
|
|
378
|
+
};
|
|
379
|
+
try {
|
|
380
|
+
fs.writeFileSync(tempPath, JSON.stringify(marker, null, 2) + '\n', {
|
|
381
|
+
encoding: 'utf8',
|
|
382
|
+
flag: 'wx',
|
|
383
|
+
mode: 0o600,
|
|
384
|
+
});
|
|
385
|
+
fs.renameSync(tempPath, markerPath);
|
|
386
|
+
} finally {
|
|
387
|
+
fs.rmSync(tempPath, { force: true });
|
|
388
|
+
}
|
|
389
|
+
}
|
|
390
|
+
|
|
357
391
|
function shellDoubleQuoteDefault(value) {
|
|
358
392
|
if (value.includes('\n')) {
|
|
359
393
|
throw new Error(`Cannot stamp skill path with newline: ${value}`);
|
|
@@ -594,10 +628,10 @@ function installSkillsForCLI(cliKey) {
|
|
|
594
628
|
if (!cli || !cli.skillsDir || !cli.skillsToInstall) return 0;
|
|
595
629
|
|
|
596
630
|
const sourceSkillsDir = path.join(CONFIG_SOURCE, 'skills');
|
|
597
|
-
if (!fs.existsSync(sourceSkillsDir)) return 0;
|
|
598
631
|
if (!fs.existsSync(cli.skillsDir)) {
|
|
599
632
|
fs.mkdirSync(cli.skillsDir, { recursive: true });
|
|
600
633
|
}
|
|
634
|
+
clearInstallMarker(cli.skillsDir);
|
|
601
635
|
|
|
602
636
|
const removed = cleanupDeprecated(path.dirname(cli.skillsDir));
|
|
603
637
|
if (removed > 0) {
|
|
@@ -619,6 +653,8 @@ function installSkillsForCLI(cliKey) {
|
|
|
619
653
|
copied++;
|
|
620
654
|
log(` → ${cli.skillsDir.replace(os.homedir(), '~')}/${skillName}`, 'dim');
|
|
621
655
|
}
|
|
656
|
+
assertCompleteSkillInstall(sourceSkillsDir, cli.skillsDir, cli.skillsToInstall);
|
|
657
|
+
writeInstallMarker(cli.skillsDir);
|
|
622
658
|
return copied;
|
|
623
659
|
}
|
|
624
660
|
|
|
@@ -726,15 +762,20 @@ function installAgentsForAllDetected() {
|
|
|
726
762
|
// is chosen. Behavior is unchanged from the previous unconditional core install.
|
|
727
763
|
function installClaudeCore() {
|
|
728
764
|
const targetDir = getTargetDir();
|
|
765
|
+
const skillsDir = path.join(targetDir, 'skills');
|
|
729
766
|
log('\n📁 Installing Claude Code config to .claude/', 'green');
|
|
767
|
+
if (!fs.existsSync(skillsDir)) fs.mkdirSync(skillsDir, { recursive: true });
|
|
768
|
+
clearInstallMarker(skillsDir);
|
|
730
769
|
const refreshed = cleanManagedSkillDirs(
|
|
731
770
|
path.join(CONFIG_SOURCE, 'skills'),
|
|
732
|
-
|
|
771
|
+
skillsDir,
|
|
733
772
|
);
|
|
734
773
|
if (refreshed > 0) {
|
|
735
774
|
log(` 🔄 Refreshing ${refreshed} managed skill director${refreshed === 1 ? 'y' : 'ies'}`, 'dim');
|
|
736
775
|
}
|
|
737
776
|
copyRecursive(CONFIG_SOURCE, targetDir, targetDir);
|
|
777
|
+
assertCompleteSkillInstall(path.join(CONFIG_SOURCE, 'skills'), skillsDir, DEVLYN_CORE_SKILLS);
|
|
778
|
+
writeInstallMarker(skillsDir);
|
|
738
779
|
|
|
739
780
|
// Remove deprecated files from previous versions
|
|
740
781
|
const removed = cleanupDeprecated(targetDir);
|
|
@@ -751,17 +792,21 @@ function installClaudeCore() {
|
|
|
751
792
|
log(' → CLAUDE.md', 'dim');
|
|
752
793
|
}
|
|
753
794
|
|
|
754
|
-
//
|
|
795
|
+
// Keep installer-managed pipeline state and install metadata out of git.
|
|
755
796
|
const gitignorePath = path.join(process.cwd(), '.gitignore');
|
|
756
|
-
const
|
|
797
|
+
const gitignoreEntries = ['.devlyn/', '.claude/skills/.devlyn-install.json'];
|
|
757
798
|
let gitignoreContent = fs.existsSync(gitignorePath)
|
|
758
799
|
? fs.readFileSync(gitignorePath, 'utf8')
|
|
759
800
|
: '';
|
|
760
|
-
|
|
801
|
+
const gitignoreLines = gitignoreContent.split('\n').map((line) => line.trim());
|
|
802
|
+
const missingGitignoreEntries = gitignoreEntries.filter((entry) =>
|
|
803
|
+
!gitignoreLines.includes(entry) && !(entry === '.devlyn/' && gitignoreLines.includes('.devlyn')));
|
|
804
|
+
if (missingGitignoreEntries.length > 0) {
|
|
761
805
|
const prefix = gitignoreContent && !gitignoreContent.endsWith('\n') ? '\n' : '';
|
|
762
|
-
const
|
|
763
|
-
|
|
764
|
-
|
|
806
|
+
const hasManagedHeader = gitignoreLines.includes('# devlyn-cli pipeline state');
|
|
807
|
+
const header = hasManagedHeader ? '' : gitignoreContent ? '\n# devlyn-cli pipeline state\n' : '# devlyn-cli pipeline state\n';
|
|
808
|
+
fs.writeFileSync(gitignorePath, gitignoreContent + prefix + header + missingGitignoreEntries.join('\n') + '\n');
|
|
809
|
+
log(` → .gitignore (added ${missingGitignoreEntries.join(', ')})`, 'dim');
|
|
765
810
|
}
|
|
766
811
|
|
|
767
812
|
// Enable agent teams in project settings
|
|
@@ -770,14 +815,37 @@ function installClaudeCore() {
|
|
|
770
815
|
if (fs.existsSync(settingsPath)) {
|
|
771
816
|
try {
|
|
772
817
|
settings = JSON.parse(fs.readFileSync(settingsPath, 'utf8'));
|
|
773
|
-
} catch {
|
|
774
|
-
settings
|
|
818
|
+
} catch (error) {
|
|
819
|
+
throw new Error(`Cannot merge .claude/settings.json: ${error.message}`);
|
|
775
820
|
}
|
|
776
821
|
}
|
|
777
|
-
if (!settings
|
|
822
|
+
if (!settings || typeof settings !== 'object' || Array.isArray(settings)) {
|
|
823
|
+
throw new Error('Cannot merge .claude/settings.json: root must be a JSON object');
|
|
824
|
+
}
|
|
825
|
+
const hasOwnSetting = (key) => Object.prototype.hasOwnProperty.call(settings, key);
|
|
826
|
+
let settingsChanged = false;
|
|
827
|
+
if (!hasOwnSetting('env')) {
|
|
828
|
+
settings.env = {};
|
|
829
|
+
settingsChanged = true;
|
|
830
|
+
}
|
|
831
|
+
if (!settings.env || typeof settings.env !== 'object' || Array.isArray(settings.env)) {
|
|
832
|
+
throw new Error('Cannot merge .claude/settings.json: env must be a JSON object');
|
|
833
|
+
}
|
|
778
834
|
// Auto-allow pipeline state directory and common git commands so resolve doesn't prompt
|
|
779
|
-
if (!
|
|
780
|
-
|
|
835
|
+
if (!hasOwnSetting('permissions')) {
|
|
836
|
+
settings.permissions = {};
|
|
837
|
+
settingsChanged = true;
|
|
838
|
+
}
|
|
839
|
+
if (!settings.permissions || typeof settings.permissions !== 'object' || Array.isArray(settings.permissions)) {
|
|
840
|
+
throw new Error('Cannot merge .claude/settings.json: permissions must be a JSON object');
|
|
841
|
+
}
|
|
842
|
+
if (!Object.prototype.hasOwnProperty.call(settings.permissions, 'allow')) {
|
|
843
|
+
settings.permissions.allow = [];
|
|
844
|
+
settingsChanged = true;
|
|
845
|
+
}
|
|
846
|
+
if (!Array.isArray(settings.permissions.allow)) {
|
|
847
|
+
throw new Error('Cannot merge .claude/settings.json: permissions.allow must be an array');
|
|
848
|
+
}
|
|
781
849
|
const pipelinePermissions = [
|
|
782
850
|
'Write(.devlyn/**)',
|
|
783
851
|
'Edit(.devlyn/**)',
|
|
@@ -787,7 +855,6 @@ function installClaudeCore() {
|
|
|
787
855
|
'Bash(git status *)',
|
|
788
856
|
'Bash(git log *)',
|
|
789
857
|
];
|
|
790
|
-
let settingsChanged = false;
|
|
791
858
|
for (const perm of pipelinePermissions) {
|
|
792
859
|
if (!settings.permissions.allow.includes(perm)) {
|
|
793
860
|
settings.permissions.allow.push(perm);
|
|
@@ -798,9 +865,35 @@ function installClaudeCore() {
|
|
|
798
865
|
settings.env.CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS = '1';
|
|
799
866
|
settingsChanged = true;
|
|
800
867
|
}
|
|
868
|
+
if (!hasOwnSetting('hooks')) {
|
|
869
|
+
settings.hooks = {};
|
|
870
|
+
settingsChanged = true;
|
|
871
|
+
}
|
|
872
|
+
if (!settings.hooks || typeof settings.hooks !== 'object' || Array.isArray(settings.hooks)) {
|
|
873
|
+
throw new Error('Cannot merge .claude/settings.json: hooks must be a JSON object');
|
|
874
|
+
}
|
|
875
|
+
if (!Object.prototype.hasOwnProperty.call(settings.hooks, 'Stop')) {
|
|
876
|
+
settings.hooks.Stop = [];
|
|
877
|
+
settingsChanged = true;
|
|
878
|
+
}
|
|
879
|
+
if (!Array.isArray(settings.hooks.Stop)) {
|
|
880
|
+
throw new Error('Cannot merge .claude/settings.json: hooks.Stop must be an array');
|
|
881
|
+
}
|
|
882
|
+
const stopHookCommand = 'python3 "$CLAUDE_PROJECT_DIR/.claude/skills/_shared/resolve-stop-hook.py"';
|
|
883
|
+
const stopHookInstalled = settings.hooks.Stop.some((entry) => (
|
|
884
|
+
entry && Array.isArray(entry.hooks) && entry.hooks.some((hook) => (
|
|
885
|
+
hook && hook.type === 'command' && hook.command === stopHookCommand
|
|
886
|
+
))
|
|
887
|
+
));
|
|
888
|
+
if (!stopHookInstalled) {
|
|
889
|
+
settings.hooks.Stop.push({
|
|
890
|
+
hooks: [{ type: 'command', command: stopHookCommand, timeout: 30 }],
|
|
891
|
+
});
|
|
892
|
+
settingsChanged = true;
|
|
893
|
+
}
|
|
801
894
|
if (settingsChanged) {
|
|
802
895
|
fs.writeFileSync(settingsPath, JSON.stringify(settings, null, 2) + '\n');
|
|
803
|
-
log(' → settings.json (agent teams + pipeline permissions)', 'dim');
|
|
896
|
+
log(' → settings.json (agent teams + pipeline permissions + Stop hook)', 'dim');
|
|
804
897
|
}
|
|
805
898
|
|
|
806
899
|
// Configure global Claude Code settings (~/.claude/settings.json)
|
|
@@ -46,8 +46,8 @@ Keep each section to ≤ 8 lines. Adapters are deltas, not full prompts. If an a
|
|
|
46
46
|
|
|
47
47
|
Adapters may add two more sections beyond the base format:
|
|
48
48
|
|
|
49
|
-
- **`## Role eligibility`** — fixed ASCII fields, one per line: `executor: yes|no`, `pair_judge: yes|no`. Absent section = eligible for both roles (every adapter shipped before iter-0051). Present only when a backend is structurally unable
|
|
50
|
-
- **`## Invocation`** — for backends called over an API
|
|
49
|
+
- **`## Role eligibility`** — fixed ASCII fields, one per line: `executor: yes|no`, `pair_judge: yes|no`. Absent section = eligible for both roles (every adapter shipped before iter-0051). Present only when a backend is structurally unable or not certified for a role — e.g. a completion-only local model has no tool/file-edit access, so it declares `executor: no`. `_shared/engine-preflight.md` and `/devlyn:engines` read this section as a structured marker, not prose, so it validates mechanically.
|
|
50
|
+
- **`## Invocation`** — for backends called over an API or CLI backends with non-default invocation contracts: the concrete request shape (endpoint, JSON body, timeout) and the adapter-declared availability probe (`_shared/engine-preflight.md#role-resolution`'s "an adapter may declare a different probe" hook).
|
|
51
51
|
|
|
52
52
|
## When to add a new adapter
|
|
53
53
|
|
|
@@ -0,0 +1,99 @@
|
|
|
1
|
+
# Grok adapter
|
|
2
|
+
|
|
3
|
+
## Role eligibility
|
|
4
|
+
|
|
5
|
+
executor: no
|
|
6
|
+
pair_judge: yes
|
|
7
|
+
|
|
8
|
+
Certification status: the pair-judge seat is wired, reachable, and probe-capable, but not emission-certified; isolation has three independent invariants — tool callability, MCP connection/enumeration, and context injection — and only P-0080-C can certify them; the doctor cannot. Before this change, the shipped recipe connected 65 MCP tools and injected both an MCP reminder and a plugin-skills reminder into the judge's conversation; a durable `pair grok` pin remains gated on follow-up emission registration and the standing seat-fitness rule.
|
|
9
|
+
|
|
10
|
+
## Invocation
|
|
11
|
+
|
|
12
|
+
**Availability probe**: `command -v grok >/dev/null 2>&1`; record
|
|
13
|
+
`grok --version` as evidence. The probe is necessary, not sufficient: auth or
|
|
14
|
+
wrapper failures after it passes are the same fail-closed availability class as
|
|
15
|
+
a failed probe (`BLOCKED:grok-unavailable` for an explicit route; an automatic
|
|
16
|
+
escalation remains a reported solo skip).
|
|
17
|
+
|
|
18
|
+
```bash
|
|
19
|
+
# $ISO_HOME = ephemeral dir seeded with a copied auth.json + agent_id, and a
|
|
20
|
+
# config.toml containing:
|
|
21
|
+
# [plugins]
|
|
22
|
+
# disabled = ["telegram", "vercel"]
|
|
23
|
+
# enabled = []
|
|
24
|
+
#
|
|
25
|
+
# [skills]
|
|
26
|
+
# ignore = ["~/.agents", "~/.claude", "~/.cursor"]
|
|
27
|
+
# plus $ISO_HOME/hooks/anchor-guard.json. grok reads that file as JSON, not
|
|
28
|
+
# through a shell, so its "command" must carry an ALREADY-EXPANDED path:
|
|
29
|
+
# {"hooks":{"PreToolUse":[{"hooks":[{"type":"command",
|
|
30
|
+
# "command":"python3 /abs/path/to/_shared/grok-anchor-guard.py","timeout":10}]}]}}
|
|
31
|
+
# $NEUTRAL = empty dir outside any git repo
|
|
32
|
+
# $REPO = absolute repo root (redirects MUST be absolute — cwd is $NEUTRAL)
|
|
33
|
+
cd "$NEUTRAL"
|
|
34
|
+
HOME="$NEUTRAL" ZDOTDIR="$NEUTRAL" GROK_HOME="$ISO_HOME" \
|
|
35
|
+
DEVLYN_PROBE_ANCHOR="<the bare anchor, derived exactly as the --allow rule below>" \
|
|
36
|
+
GROK_CLAUDE_MCPS_ENABLED=false GROK_CURSOR_MCPS_ENABLED=false \
|
|
37
|
+
GROK_CLAUDE_SKILLS_ENABLED=false GROK_CURSOR_SKILLS_ENABLED=false \
|
|
38
|
+
GROK_CLAUDE_HOOKS_ENABLED=false GROK_CURSOR_HOOKS_ENABLED=false \
|
|
39
|
+
GROK_CLAUDE_AGENTS_ENABLED=false GROK_CLAUDE_RULES_ENABLED=false \
|
|
40
|
+
GROK_MEMORY=0 \
|
|
41
|
+
python3 "$DEVLYN_SHARED_DIR/run-bounded.py" 600 -- grok -p "<judge prompt>" \
|
|
42
|
+
--permission-mode dontAsk --no-memory \
|
|
43
|
+
--tools "read_file,grep,list_dir,run_terminal_cmd" \
|
|
44
|
+
--disallowed-tools "Agent,use_tool,search_tool" \
|
|
45
|
+
--allow 'Bash(<the bare anchor>)' \
|
|
46
|
+
--reasoning-effort medium \
|
|
47
|
+
--output-format json \
|
|
48
|
+
> "$REPO/.devlyn/grok-judge.stdout" 2> "$REPO/.devlyn/grok-judge.stderr"
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
Copy `auth.json` into `$ISO_HOME` for every run and seed its `config.toml` with
|
|
52
|
+
the shown `[plugins]` and `[skills] ignore` entries and its
|
|
53
|
+
`hooks/anchor-guard.json` with the shown `PreToolUse` entry. Before launch, create
|
|
54
|
+
`$NEUTRAL/.zshenv` containing `export HOME=<the operator's real home>`.
|
|
55
|
+
Both overrides are needed because grok scans `~/.claude.json` from `$HOME`,
|
|
56
|
+
while zsh reads rc files from `$ZDOTDIR`, which is exported in some environments.
|
|
57
|
+
`$NEUTRAL` and `$ISO_HOME` must be on paths that do not embed a project or repo identifier.
|
|
58
|
+
`--permission-mode plan` is forbidden headless because it silently blocks
|
|
59
|
+
all tools and returns an empty review; `--always-approve` is banned because it
|
|
60
|
+
auto-approves writes. `--tools` and `--disallowed-tools` are headless-only (the
|
|
61
|
+
TUI warns and ignores them); when both are present, the disallow list wins and
|
|
62
|
+
removes tools entirely rather than merely gating execution.
|
|
63
|
+
|
|
64
|
+
Never use `--json-schema` for the pair judge: measured runs degraded the
|
|
65
|
+
tool-use loop (2/3 made zero tool calls and fabricated findings), and a
|
|
66
|
+
`Cancelled` run emitted a PASS-shaped payload after its schema had failed.
|
|
67
|
+
|
|
68
|
+
The judge prompt must name the exact permitted command(s). Derive the prompt
|
|
69
|
+
anchor and allow-list string identically from one source, in this precedence:
|
|
70
|
+
the solo-headroom hypothesis's backticked observable command when present;
|
|
71
|
+
otherwise the backticked commands in the spec's `## Verification` bullets;
|
|
72
|
+
otherwise the repo's existing test/CLI runner. If none exists, the probe
|
|
73
|
+
obligation does not arise and a static review is correct for that spec.
|
|
74
|
+
The documented `Bash(<command>)` form matches the exact command or its prefix;
|
|
75
|
+
the trailing ` *` form fails a no-argument command.
|
|
76
|
+
Under `dontAsk`, every allow shape tested for a chained command — including an
|
|
77
|
+
exact full-string rule for the chain — returned `PermissionCancelled`, so the
|
|
78
|
+
mandatory dominance-loss anchor must run unchained with the allow rule scoped
|
|
79
|
+
to that bare anchor.
|
|
80
|
+
|
|
81
|
+
Grok's deny semantics are not Claude-parity: reaching the permission mode's
|
|
82
|
+
auto-deny silently truncates the review at exit 0. On the configured, measured
|
|
83
|
+
path the `PreToolUse` hook above keeps the judge from reaching it — the hook runs
|
|
84
|
+
at step 1, before the rules and before the mode policy, and returns the denial as
|
|
85
|
+
a model-visible `tool_result`.
|
|
86
|
+
It admits the bare anchor and anchor-plus-argv, and vetoes any character that
|
|
87
|
+
could start a second command, so `verify.md`'s bounded input variations survive
|
|
88
|
+
while chaining does not. The hook fails open, so the collector rejection remains
|
|
89
|
+
the fail-closed floor: treat it as `BLOCKED` for
|
|
90
|
+
`verify.pair.emission-contract`, never PASS. After a non-timeout spawn, run:
|
|
91
|
+
|
|
92
|
+
```bash
|
|
93
|
+
python3 "$DEVLYN_SHARED_DIR/collect-codex-findings.py" \
|
|
94
|
+
--devlyn-dir "$REPO/.devlyn" \
|
|
95
|
+
--stdout-file grok-judge.stdout
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
On exit 124, write `.devlyn/verify.pair.timeout.json` with
|
|
99
|
+
`{"engine": "grok", "budget_seconds": 600}` before merge.
|