devlyn-cli 2.7.0 → 2.8.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +9 -1
- package/CLAUDE.md +28 -11
- package/benchmark/auto-resolve/fixtures/F1-cli-trivial-flag/task.ko.txt +8 -0
- package/benchmark/auto-resolve/fixtures/retired/F34-cli-settle-inflight/NOTES.md +28 -0
- package/benchmark/auto-resolve/fixtures/retired/F34-cli-settle-inflight/RETIRED.md +15 -0
- package/benchmark/auto-resolve/fixtures/retired/F34-cli-settle-inflight/expected.json +57 -0
- package/benchmark/auto-resolve/fixtures/retired/F34-cli-settle-inflight/metadata.json +10 -0
- package/benchmark/auto-resolve/fixtures/retired/F34-cli-settle-inflight/setup.sh +50 -0
- package/benchmark/auto-resolve/fixtures/retired/F34-cli-settle-inflight/spec.md +59 -0
- package/benchmark/auto-resolve/fixtures/retired/F34-cli-settle-inflight/task.txt +9 -0
- package/benchmark/auto-resolve/fixtures/retired/F34-cli-settle-inflight/verifiers/ordered-settlement.js +88 -0
- package/benchmark/auto-resolve/fixtures/retired/F34-cli-settle-inflight/verifiers/reject-invalid.js +45 -0
- package/benchmark/auto-resolve/fixtures/retired/F35-cli-apply-journal/NOTES.md +30 -0
- package/benchmark/auto-resolve/fixtures/retired/F35-cli-apply-journal/RETIRED.md +16 -0
- package/benchmark/auto-resolve/fixtures/retired/F35-cli-apply-journal/expected.json +64 -0
- package/benchmark/auto-resolve/fixtures/retired/F35-cli-apply-journal/metadata.json +10 -0
- package/benchmark/auto-resolve/fixtures/retired/F35-cli-apply-journal/setup.sh +21 -0
- package/benchmark/auto-resolve/fixtures/retired/F35-cli-apply-journal/spec.md +58 -0
- package/benchmark/auto-resolve/fixtures/retired/F35-cli-apply-journal/task.txt +9 -0
- package/benchmark/auto-resolve/fixtures/retired/F35-cli-apply-journal/verifiers/dup-op.js +54 -0
- package/benchmark/auto-resolve/fixtures/retired/F35-cli-apply-journal/verifiers/priority-apply.js +82 -0
- package/benchmark/auto-resolve/fixtures/retired/F35-cli-apply-journal/verifiers/rollback.js +77 -0
- package/benchmark/auto-resolve/fixtures/retired/F36-cli-session-admission/NOTES.md +99 -0
- package/benchmark/auto-resolve/fixtures/retired/F36-cli-session-admission/RETIRED.md +18 -0
- package/benchmark/auto-resolve/fixtures/retired/F36-cli-session-admission/expected.json +64 -0
- package/benchmark/auto-resolve/fixtures/retired/F36-cli-session-admission/metadata.json +10 -0
- package/benchmark/auto-resolve/fixtures/retired/F36-cli-session-admission/setup.sh +4 -0
- package/benchmark/auto-resolve/fixtures/retired/F36-cli-session-admission/spec.md +61 -0
- package/benchmark/auto-resolve/fixtures/retired/F36-cli-session-admission/task.txt +9 -0
- package/benchmark/auto-resolve/fixtures/retired/F36-cli-session-admission/verifiers/correctness-small.js +135 -0
- package/benchmark/auto-resolve/fixtures/retired/F36-cli-session-admission/verifiers/scale.js +100 -0
- package/benchmark/auto-resolve/fixtures/retired/F36-cli-session-admission/verifiers/validation.js +75 -0
- package/benchmark/auto-resolve/fixtures/retired/F37-cli-rule-lookup/NOTES.md +93 -0
- package/benchmark/auto-resolve/fixtures/retired/F37-cli-rule-lookup/RETIRED.md +16 -0
- package/benchmark/auto-resolve/fixtures/retired/F37-cli-rule-lookup/expected.json +65 -0
- package/benchmark/auto-resolve/fixtures/retired/F37-cli-rule-lookup/metadata.json +10 -0
- package/benchmark/auto-resolve/fixtures/retired/F37-cli-rule-lookup/setup.sh +18 -0
- package/benchmark/auto-resolve/fixtures/retired/F37-cli-rule-lookup/spec.md +66 -0
- package/benchmark/auto-resolve/fixtures/retired/F37-cli-rule-lookup/task.txt +9 -0
- package/benchmark/auto-resolve/fixtures/retired/F37-cli-rule-lookup/verifiers/correctness-small.js +84 -0
- package/benchmark/auto-resolve/fixtures/retired/F37-cli-rule-lookup/verifiers/scale.js +184 -0
- package/benchmark/auto-resolve/fixtures/retired/F37-cli-rule-lookup/verifiers/validation.js +74 -0
- package/benchmark/auto-resolve/scripts/pair-rejected-fixtures.sh +12 -0
- package/benchmark/auto-resolve/scripts/run-fixture.sh +39 -1
- package/benchmark/auto-resolve/scripts/test-lint-fixtures.sh +1 -1
- package/bin/devlyn.js +42 -1
- package/config/skills/_shared/adapters/README.md +7 -0
- package/config/skills/_shared/adapters/claude.md +47 -0
- package/config/skills/_shared/adapters/omp.md +17 -0
- package/config/skills/_shared/archive_run.py +8 -2
- package/config/skills/_shared/codex-config.md +22 -6
- package/config/skills/_shared/engine-doctor.sh +107 -0
- package/config/skills/_shared/engine-preflight.md +18 -11
- package/config/skills/_shared/expected.schema.json +26 -0
- package/config/skills/_shared/runtime-principles.md +1 -1
- package/config/skills/_shared/spec-verify-check.py +773 -486
- package/config/skills/_shared/state-phase-write.py +374 -0
- package/config/skills/_shared/verify-merge-findings.py +177 -61
- package/config/skills/devlyn:engines/SKILL.md +47 -0
- package/config/skills/devlyn:ideate/SKILL.md +22 -3
- package/config/skills/devlyn:ideate/references/elicitation.md +2 -2
- package/config/skills/devlyn:ideate/references/from-spec-mode.md +2 -2
- package/config/skills/devlyn:ideate/references/project-mode.md +1 -1
- package/config/skills/devlyn:ideate/references/spec-template.md +15 -4
- package/config/skills/devlyn:queue/SKILL.md +26 -0
- package/config/skills/devlyn:resolve/SKILL.md +54 -24
- package/config/skills/devlyn:resolve/references/free-form-mode.md +1 -1
- package/config/skills/devlyn:resolve/references/phases/build-gate.md +2 -2
- package/config/skills/devlyn:resolve/references/phases/cleanup.md +1 -1
- package/config/skills/devlyn:resolve/references/phases/implement.md +1 -1
- package/config/skills/devlyn:resolve/references/phases/plan.md +3 -2
- package/config/skills/devlyn:resolve/references/phases/probe-derive.md +8 -0
- package/config/skills/devlyn:resolve/references/phases/verify.md +18 -9
- package/config/skills/devlyn:resolve/references/state-schema.md +10 -6
- package/optional-skills/devlyn:pencil-pull/SKILL.md +5 -0
- package/optional-skills/devlyn:pencil-push/SKILL.md +5 -0
- package/package.json +1 -1
- package/scripts/lint-skills.sh +48 -41
|
@@ -119,13 +119,19 @@ def finding_rank(finding: dict[str, Any]) -> int:
|
|
|
119
119
|
return 0
|
|
120
120
|
|
|
121
121
|
|
|
122
|
-
def read_findings(devlyn: pathlib.Path) -> tuple[list[dict[str, Any]], dict[str, str]]:
|
|
122
|
+
def read_findings(devlyn: pathlib.Path) -> tuple[list[dict[str, Any]], dict[str, str | None]]:
|
|
123
123
|
findings: list[dict[str, Any]] = []
|
|
124
|
-
source_verdicts = {source: "PASS" for source, _ in SOURCE_FILES}
|
|
124
|
+
source_verdicts: dict[str, str | None] = {source: "PASS" for source, _ in SOURCE_FILES}
|
|
125
|
+
# A verdict must come from a judge that ran: pair_judge stays null until
|
|
126
|
+
# spawn evidence exists (a pair findings file or pair stdout). verify.md's
|
|
127
|
+
# pair contract records null when no second agent is spawned.
|
|
128
|
+
source_verdicts["pair_judge"] = None
|
|
125
129
|
for source, name in SOURCE_FILES:
|
|
126
130
|
path = devlyn / name
|
|
127
131
|
if not path.is_file():
|
|
128
132
|
continue
|
|
133
|
+
if source_verdicts[source] is None:
|
|
134
|
+
source_verdicts[source] = "PASS"
|
|
129
135
|
with path.open(encoding="utf-8") as handle:
|
|
130
136
|
for line_no, line in enumerate(handle, 1):
|
|
131
137
|
raw = line.strip()
|
|
@@ -401,7 +407,7 @@ def is_command_like_backtick(value: str) -> bool:
|
|
|
401
407
|
|
|
402
408
|
def state_pair_trigger_reasons(
|
|
403
409
|
devlyn: pathlib.Path,
|
|
404
|
-
source_verdicts: dict[str, str],
|
|
410
|
+
source_verdicts: dict[str, str | None],
|
|
405
411
|
) -> list[str]:
|
|
406
412
|
state_path = devlyn / "pipeline.state.json"
|
|
407
413
|
if not state_path.is_file():
|
|
@@ -450,7 +456,7 @@ def state_pair_trigger_reasons(
|
|
|
450
456
|
|
|
451
457
|
def pair_trigger_missing_contract_violation(
|
|
452
458
|
devlyn: pathlib.Path,
|
|
453
|
-
source_verdicts: dict[str, str],
|
|
459
|
+
source_verdicts: dict[str, str | None],
|
|
454
460
|
) -> dict[str, Any] | None:
|
|
455
461
|
if rank(source_verdicts.get("mechanical")) >= 2 or rank(source_verdicts.get("judge")) >= 2:
|
|
456
462
|
return None
|
|
@@ -469,7 +475,7 @@ def pair_trigger_missing_contract_violation(
|
|
|
469
475
|
|
|
470
476
|
def pair_trigger_skip_contract_violation(
|
|
471
477
|
devlyn: pathlib.Path,
|
|
472
|
-
source_verdicts: dict[str, str],
|
|
478
|
+
source_verdicts: dict[str, str | None],
|
|
473
479
|
) -> dict[str, Any] | None:
|
|
474
480
|
state_path = devlyn / "pipeline.state.json"
|
|
475
481
|
if not state_path.is_file():
|
|
@@ -565,7 +571,7 @@ def pair_trigger_skip_contract_violation(
|
|
|
565
571
|
|
|
566
572
|
def pair_trigger_reason_completeness_violation(
|
|
567
573
|
devlyn: pathlib.Path,
|
|
568
|
-
source_verdicts: dict[str, str],
|
|
574
|
+
source_verdicts: dict[str, str | None],
|
|
569
575
|
) -> dict[str, Any] | None:
|
|
570
576
|
if rank(source_verdicts.get("mechanical")) >= 2 or rank(source_verdicts.get("judge")) >= 2:
|
|
571
577
|
return None
|
|
@@ -623,9 +629,12 @@ def pair_blocker(id_: str, message: str, file_: str | None = None) -> dict[str,
|
|
|
623
629
|
|
|
624
630
|
def detect_pair_stdout_contract_violations(
|
|
625
631
|
devlyn: pathlib.Path,
|
|
626
|
-
source_verdicts: dict[str, str],
|
|
632
|
+
source_verdicts: dict[str, str | None],
|
|
627
633
|
) -> list[dict[str, Any]]:
|
|
628
|
-
|
|
634
|
+
# *-judge.stdout: any engine's pair-judge capture (codex-judge.stdout,
|
|
635
|
+
# claude-judge.stdout — adapters/claude.md ## Invocation). The emission
|
|
636
|
+
# contract is engine-neutral.
|
|
637
|
+
stdout_paths = sorted(devlyn.glob("*-judge.stdout"))
|
|
629
638
|
flag_violation = pair_flag_contract_violation(devlyn)
|
|
630
639
|
if flag_violation is not None:
|
|
631
640
|
source_verdicts["pair_judge"] = "BLOCKED"
|
|
@@ -689,73 +698,76 @@ def detect_pair_stdout_contract_violations(
|
|
|
689
698
|
]
|
|
690
699
|
if has_pair_findings(devlyn):
|
|
691
700
|
return []
|
|
692
|
-
if not
|
|
701
|
+
if not stdout_paths:
|
|
693
702
|
if required:
|
|
694
703
|
source_verdicts["pair_judge"] = "BLOCKED"
|
|
695
704
|
return [
|
|
696
705
|
pair_blocker(
|
|
697
706
|
"verify-pair-required-output-missing",
|
|
698
|
-
"Pair-mode was required, but
|
|
699
|
-
"
|
|
707
|
+
"Pair-mode was required, but the pair-JUDGE produced no stdout or canonical findings file.",
|
|
708
|
+
"verify.pair.findings.jsonl",
|
|
700
709
|
)
|
|
701
710
|
]
|
|
702
711
|
return []
|
|
703
|
-
|
|
704
|
-
|
|
705
|
-
|
|
706
|
-
|
|
707
|
-
|
|
708
|
-
|
|
709
|
-
|
|
710
|
-
|
|
711
|
-
|
|
712
|
-
|
|
713
|
-
|
|
714
|
-
|
|
715
|
-
|
|
716
|
-
|
|
717
|
-
|
|
718
|
-
|
|
719
|
-
|
|
712
|
+
if source_verdicts["pair_judge"] is None:
|
|
713
|
+
source_verdicts["pair_judge"] = "PASS"
|
|
714
|
+
for stdout_path in stdout_paths:
|
|
715
|
+
raw_text = stdout_path.read_text(encoding="utf-8")
|
|
716
|
+
if not raw_text.strip():
|
|
717
|
+
source_verdicts["pair_judge"] = "BLOCKED"
|
|
718
|
+
return [
|
|
719
|
+
pair_blocker(
|
|
720
|
+
"verify-pair-empty-output",
|
|
721
|
+
f"pair-JUDGE stdout {stdout_path.name} was empty; the bounded contract requires a JSONL finding or PASS line.",
|
|
722
|
+
stdout_path.name,
|
|
723
|
+
)
|
|
724
|
+
]
|
|
725
|
+
has_jsonl_finding = False
|
|
726
|
+
has_nonpass_summary = False
|
|
727
|
+
for line in raw_text.splitlines():
|
|
728
|
+
raw = line.strip()
|
|
729
|
+
if not raw:
|
|
730
|
+
continue
|
|
731
|
+
if raw.startswith("# SUMMARY "):
|
|
732
|
+
try:
|
|
733
|
+
summary = loads_strict_json(raw.removeprefix("# SUMMARY ").strip())
|
|
734
|
+
except ValueError:
|
|
735
|
+
continue
|
|
736
|
+
if summary.get("verdict") in {"NEEDS_WORK", "FAIL", "BLOCKED"}:
|
|
737
|
+
has_nonpass_summary = True
|
|
738
|
+
continue
|
|
739
|
+
if raw.startswith("#"):
|
|
740
|
+
continue
|
|
720
741
|
try:
|
|
721
|
-
|
|
742
|
+
item = loads_strict_json(raw)
|
|
722
743
|
except ValueError:
|
|
723
744
|
continue
|
|
724
|
-
if
|
|
725
|
-
|
|
726
|
-
|
|
727
|
-
|
|
728
|
-
|
|
729
|
-
|
|
730
|
-
|
|
731
|
-
|
|
732
|
-
|
|
733
|
-
|
|
734
|
-
|
|
735
|
-
|
|
736
|
-
|
|
737
|
-
|
|
738
|
-
|
|
739
|
-
|
|
740
|
-
|
|
741
|
-
|
|
742
|
-
|
|
743
|
-
return [
|
|
744
|
-
pair_blocker(
|
|
745
|
-
"verify-pair-emission-contract-violated",
|
|
746
|
-
(
|
|
747
|
-
"Codex pair-JUDGE stdout contained findings or a non-PASS summary, "
|
|
748
|
-
"but the canonical pair findings JSONL file was empty."
|
|
749
|
-
),
|
|
750
|
-
"codex-judge.stdout",
|
|
751
|
-
)
|
|
752
|
-
]
|
|
745
|
+
if isinstance(item, dict) and str(item.get("severity") or "").upper() in {
|
|
746
|
+
"CRITICAL",
|
|
747
|
+
"HIGH",
|
|
748
|
+
"MEDIUM",
|
|
749
|
+
"LOW",
|
|
750
|
+
}:
|
|
751
|
+
has_jsonl_finding = True
|
|
752
|
+
if has_jsonl_finding or has_nonpass_summary:
|
|
753
|
+
source_verdicts["pair_judge"] = "BLOCKED"
|
|
754
|
+
return [
|
|
755
|
+
pair_blocker(
|
|
756
|
+
"verify-pair-emission-contract-violated",
|
|
757
|
+
(
|
|
758
|
+
f"pair-JUDGE stdout {stdout_path.name} contained findings or a non-PASS "
|
|
759
|
+
"summary, but the canonical pair findings JSONL file was empty."
|
|
760
|
+
),
|
|
761
|
+
stdout_path.name,
|
|
762
|
+
)
|
|
763
|
+
]
|
|
764
|
+
return []
|
|
753
765
|
|
|
754
766
|
|
|
755
767
|
def write_outputs(
|
|
756
768
|
devlyn: pathlib.Path,
|
|
757
769
|
findings: list[dict[str, Any]],
|
|
758
|
-
source_verdicts: dict[str, str],
|
|
770
|
+
source_verdicts: dict[str, str | None],
|
|
759
771
|
) -> dict[str, Any]:
|
|
760
772
|
merged_path = devlyn / "verify-merged.findings.jsonl"
|
|
761
773
|
summary_path = devlyn / "verify-merge.summary.json"
|
|
@@ -786,7 +798,18 @@ def write_state(devlyn: pathlib.Path, summary: dict[str, Any]) -> None:
|
|
|
786
798
|
verify = {}
|
|
787
799
|
phases["verify"] = verify
|
|
788
800
|
verify["verdict"] = summary["verdict"]
|
|
789
|
-
sub = verify.
|
|
801
|
+
sub = verify.get("sub_verdicts")
|
|
802
|
+
if sub is None:
|
|
803
|
+
# spawn (state-phase-write.py) always writes sub_verdicts: null as
|
|
804
|
+
# part of the per-round reset contract (state-schema.md#write-protocol)
|
|
805
|
+
# — legal, expected state before this function populates it.
|
|
806
|
+
# setdefault() would not replace an existing null, only an absent key.
|
|
807
|
+
sub = {}
|
|
808
|
+
verify["sub_verdicts"] = sub
|
|
809
|
+
elif not isinstance(sub, dict):
|
|
810
|
+
raise SystemExit(
|
|
811
|
+
f"error: phases.verify.sub_verdicts must be null or an object, got {type(sub).__name__}"
|
|
812
|
+
)
|
|
790
813
|
for source, source_verdict in summary["source_verdicts"].items():
|
|
791
814
|
if source in {"mechanical", "judge", "pair_judge"}:
|
|
792
815
|
sub[source] = source_verdict
|
|
@@ -801,6 +824,68 @@ def write_state(devlyn: pathlib.Path, summary: dict[str, Any]) -> None:
|
|
|
801
824
|
def self_test() -> int:
|
|
802
825
|
with tempfile.TemporaryDirectory() as tmp:
|
|
803
826
|
devlyn = pathlib.Path(tmp)
|
|
827
|
+
|
|
828
|
+
# state-phase-write.py's spawn always writes sub_verdicts: null (the
|
|
829
|
+
# per-round reset contract, state-schema.md#write-protocol) — this is
|
|
830
|
+
# the real shape write_state() sees on every VERIFY completion, not
|
|
831
|
+
# the pre-populated {} the other scenarios below seed for brevity.
|
|
832
|
+
(devlyn / "pipeline.state.json").write_text(
|
|
833
|
+
json.dumps({"phases": {"verify": {"verdict": None, "sub_verdicts": None}}}),
|
|
834
|
+
encoding="utf-8",
|
|
835
|
+
)
|
|
836
|
+
(devlyn / "verify.findings.jsonl").write_text("", encoding="utf-8")
|
|
837
|
+
findings, source_verdicts = read_findings(devlyn)
|
|
838
|
+
summary = write_outputs(devlyn, findings, source_verdicts)
|
|
839
|
+
write_state(devlyn, summary)
|
|
840
|
+
state = loads_strict_json((devlyn / "pipeline.state.json").read_text(encoding="utf-8"))
|
|
841
|
+
assert summary["verdict"] == "PASS", summary
|
|
842
|
+
assert state["phases"]["verify"]["sub_verdicts"] == {
|
|
843
|
+
"mechanical": "PASS", "judge": "PASS", "pair_judge": None,
|
|
844
|
+
}, state
|
|
845
|
+
|
|
846
|
+
# 2026-07-04 field bug (iter-0060 G1): an AUTO pair trigger skipped on
|
|
847
|
+
# OTHER-engine unavailability spawns no second judge — pair_judge must
|
|
848
|
+
# be recorded null, never a synthesized PASS.
|
|
849
|
+
(devlyn / "pipeline.state.json").write_text(
|
|
850
|
+
json.dumps({
|
|
851
|
+
"mode": "spec",
|
|
852
|
+
"phases": {
|
|
853
|
+
"verify": {
|
|
854
|
+
"verdict": None,
|
|
855
|
+
"sub_verdicts": None,
|
|
856
|
+
"pair_trigger": {
|
|
857
|
+
"eligible": False,
|
|
858
|
+
"reasons": [],
|
|
859
|
+
"skipped_reason": "auto_pair_other_engine_unavailable",
|
|
860
|
+
},
|
|
861
|
+
}
|
|
862
|
+
},
|
|
863
|
+
}),
|
|
864
|
+
encoding="utf-8",
|
|
865
|
+
)
|
|
866
|
+
findings, source_verdicts = read_findings(devlyn)
|
|
867
|
+
summary = write_outputs(devlyn, findings, source_verdicts)
|
|
868
|
+
write_state(devlyn, summary)
|
|
869
|
+
state = loads_strict_json((devlyn / "pipeline.state.json").read_text(encoding="utf-8"))
|
|
870
|
+
assert summary["verdict"] == "PASS", summary
|
|
871
|
+
assert summary["source_verdicts"]["pair_judge"] is None, summary
|
|
872
|
+
assert state["phases"]["verify"]["sub_verdicts"]["pair_judge"] is None, state
|
|
873
|
+
|
|
874
|
+
# A non-null, non-dict sub_verdicts is corrupted state, not a legal
|
|
875
|
+
# placeholder — write_state() must fail loud, not silently coerce it.
|
|
876
|
+
(devlyn / "pipeline.state.json").write_text(
|
|
877
|
+
json.dumps({"phases": {"verify": {"verdict": None, "sub_verdicts": "corrupt"}}}),
|
|
878
|
+
encoding="utf-8",
|
|
879
|
+
)
|
|
880
|
+
findings, source_verdicts = read_findings(devlyn)
|
|
881
|
+
summary = write_outputs(devlyn, findings, source_verdicts)
|
|
882
|
+
try:
|
|
883
|
+
write_state(devlyn, summary)
|
|
884
|
+
except SystemExit as e:
|
|
885
|
+
assert "sub_verdicts must be null or an object" in str(e), e
|
|
886
|
+
else:
|
|
887
|
+
raise AssertionError("write_state() must reject non-dict, non-null sub_verdicts")
|
|
888
|
+
|
|
804
889
|
(devlyn / "pipeline.state.json").write_text(
|
|
805
890
|
json.dumps({
|
|
806
891
|
"phases": {
|
|
@@ -869,8 +954,24 @@ def self_test() -> int:
|
|
|
869
954
|
state = loads_strict_json((devlyn / "pipeline.state.json").read_text(encoding="utf-8"))
|
|
870
955
|
assert summary["verdict"] == "BLOCKED", summary
|
|
871
956
|
assert state["phases"]["verify"]["sub_verdicts"]["pair_judge"] == "BLOCKED", state
|
|
872
|
-
|
|
873
957
|
(devlyn / "codex-judge.stdout").unlink()
|
|
958
|
+
|
|
959
|
+
# Engine-neutral stdout contract (iter-0060): a Claude pair-judge
|
|
960
|
+
# capture (claude-judge.stdout, adapters/claude.md ## Invocation)
|
|
961
|
+
# binds the same emission contract as the Codex one.
|
|
962
|
+
(devlyn / "claude-judge.stdout").write_text(
|
|
963
|
+
json.dumps({"id": "clj1", "severity": "HIGH"}) + "\n",
|
|
964
|
+
encoding="utf-8",
|
|
965
|
+
)
|
|
966
|
+
findings, source_verdicts = read_findings(devlyn)
|
|
967
|
+
summary = write_outputs(devlyn, findings, source_verdicts)
|
|
968
|
+
assert summary["verdict"] == "BLOCKED", summary
|
|
969
|
+
assert any(
|
|
970
|
+
finding.get("id") == "verify-pair-emission-contract-violated"
|
|
971
|
+
and "claude-judge.stdout" in finding.get("message", "")
|
|
972
|
+
for finding in findings
|
|
973
|
+
), findings
|
|
974
|
+
(devlyn / "claude-judge.stdout").unlink()
|
|
874
975
|
(devlyn / "pipeline.state.json").write_text(
|
|
875
976
|
json.dumps({
|
|
876
977
|
"phases": {
|
|
@@ -1672,6 +1773,21 @@ def self_test() -> int:
|
|
|
1672
1773
|
finding.get("id") == "verify-pair-trigger-reasons-unknown"
|
|
1673
1774
|
for finding in findings
|
|
1674
1775
|
), findings
|
|
1776
|
+
|
|
1777
|
+
# stdout-only spawn evidence: no pair findings file at all; a clean
|
|
1778
|
+
# PASS stdout from a claude judge promotes pair_judge null -> PASS.
|
|
1779
|
+
(devlyn / "verify.pair.findings.jsonl").unlink()
|
|
1780
|
+
(devlyn / "pipeline.state.json").write_text(
|
|
1781
|
+
json.dumps({"phases": {"verify": {"verdict": None, "sub_verdicts": None}}}),
|
|
1782
|
+
encoding="utf-8",
|
|
1783
|
+
)
|
|
1784
|
+
(devlyn / "claude-judge.stdout").write_text("PASS\n", encoding="utf-8")
|
|
1785
|
+
findings, source_verdicts = read_findings(devlyn)
|
|
1786
|
+
summary = write_outputs(devlyn, findings, source_verdicts)
|
|
1787
|
+
write_state(devlyn, summary)
|
|
1788
|
+
state = loads_strict_json((devlyn / "pipeline.state.json").read_text(encoding="utf-8"))
|
|
1789
|
+
assert summary["verdict"] == "PASS", summary
|
|
1790
|
+
assert state["phases"]["verify"]["sub_verdicts"]["pair_judge"] == "PASS", state
|
|
1675
1791
|
return 0
|
|
1676
1792
|
|
|
1677
1793
|
|
|
@@ -0,0 +1,47 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: devlyn:engines
|
|
3
|
+
description: Show and pin engine roles for the devlyn pipeline (executor / pair judge). Reads and writes machine-local .devlyn/engines.json with fail-closed validation. Use when the user asks which models are configured, wants to force a specific model for pipeline work ("수동모드", "engine 고정", "use codex to implement"), or wants to clear pins back to auto-detection.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
Utility front-end for the role-resolution contract in `_shared/engine-preflight.md#role-resolution`. It adds no semantics of its own: everything it writes is exactly what `/devlyn:resolve` PHASE 0 reads.
|
|
7
|
+
|
|
8
|
+
<args>
|
|
9
|
+
$ARGUMENTS
|
|
10
|
+
</args>
|
|
11
|
+
|
|
12
|
+
<runtime_paths>
|
|
13
|
+
Resolve shared scripts from this skill's installed directory, never from the project cwd, before step 2 below:
|
|
14
|
+
|
|
15
|
+
```bash
|
|
16
|
+
DEVLYN_SKILL_DIR="${CLAUDE_SKILL_DIR:-__DEVLYN_SKILL_DIR__}"
|
|
17
|
+
if [ "$DEVLYN_SKILL_DIR" = "__DEVLYN_SKILL_DIR__" ] || [ ! -d "$DEVLYN_SKILL_DIR/../_shared" ]; then
|
|
18
|
+
echo "BLOCKED:shared-dir-unresolved: $DEVLYN_SKILL_DIR/../_shared" >&2
|
|
19
|
+
exit 1
|
|
20
|
+
fi
|
|
21
|
+
DEVLYN_SHARED_DIR="$(cd "$DEVLYN_SKILL_DIR/../_shared" && pwd)"
|
|
22
|
+
```
|
|
23
|
+
</runtime_paths>
|
|
24
|
+
|
|
25
|
+
## No args — status + how to choose
|
|
26
|
+
|
|
27
|
+
1. Read `cwd/.devlyn/engines.json`. Absent → report "no pins — auto-detection active".
|
|
28
|
+
2. Run `bash "$DEVLYN_SHARED_DIR/engine-doctor.sh"` — read-only; never installs anything or writes `.devlyn/engines.json`. It detects, per target (`claude`, `codex`, `omp`, `pi`), whether its binary is present, whether `_shared/adapters/<name>.md` exists, and whether that makes it `pin_eligible` today. Its trailing line is either a pair-judge-diversity confirmation or a recommendation to add a second adapter-valid engine (`autoresearch/iterations/0045-model-arm-drift.md`: different model tiers hit different failure-mode blind spots, so VERIFY pair-judge/risk-probes need ≥2 to ever fire).
|
|
29
|
+
3. Print the role table. Executor's "Available engines" = rows where `pin_eligible: yes` and `role` includes `executor` (i.e. not `judge-only`). Pair judge's "Available engines" = rows where `pin_eligible: yes` and `role` includes `judge` (`executor+judge` or `judge-only`):
|
|
30
|
+
|
|
31
|
+
```
|
|
32
|
+
Role Resolved Source Available engines
|
|
33
|
+
orchestrator (this CLI) not configurable — switch CLIs to change
|
|
34
|
+
executor claude default claude ✓ codex ✓
|
|
35
|
+
pair judge codex binary rule (pin order with: pair <name>,...)
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
4. Show the script's full detection table plus its diversity/recommendation line beneath the role table.
|
|
39
|
+
5. End with one usage line per subcommand below.
|
|
40
|
+
|
|
41
|
+
## Subcommands
|
|
42
|
+
|
|
43
|
+
- `executor <name>` — pin the executor (PLAN/IMPLEMENT/CLEANUP + primary VERIFY judge). Refuse names with no `_shared/adapters/<name>.md`, or whose adapter's `## Role eligibility` section declares `executor: no` (judge-only backends), listing the valid names either way. If the engine's CLI is not currently available, still write the pin but warn: pins are promises — the run will stop with `BLOCKED:<name>-unavailable` until it is installed.
|
|
44
|
+
- `pair <name>[,<name>...]` — set `pair_judge_priority` (ordered; first adapter-valid, pair-judge-eligible, non-primary, available entry wins at VERIFY/risk-probe time). Refuse names with no `_shared/adapters/<name>.md`, or whose adapter declares `pair_judge: no`.
|
|
45
|
+
- `clear` — remove both pins; delete `.devlyn/engines.json` when nothing else remains in it.
|
|
46
|
+
|
|
47
|
+
Writes preserve any unrecognized keys already in the file. After every change, re-print the resolved role table. Never modify anything outside `.devlyn/engines.json`, and never launch a pipeline from this skill.
|
|
@@ -27,6 +27,25 @@ The user does not know context engineering. They will under-specify and over-ass
|
|
|
27
27
|
Read `_shared/runtime-principles.md` (Subtractive-first / Goal-locked / No-workaround / Evidence). The principles bind the spec content as well as your conversation. A spec that says "for future flexibility" is a Subtractive-first violation. A spec that asks for `try { ... } catch { return null }` is a No-workaround violation. AI flags these in elicitation, not after `/devlyn:resolve` has built them.
|
|
28
28
|
</harness_principles>
|
|
29
29
|
|
|
30
|
+
<runtime_paths>
|
|
31
|
+
Resolve shared scripts from this skill's installed directory, never from the project cwd:
|
|
32
|
+
|
|
33
|
+
```bash
|
|
34
|
+
DEVLYN_SKILL_DIR="${CLAUDE_SKILL_DIR:-__DEVLYN_SKILL_DIR__}"
|
|
35
|
+
if [ "$DEVLYN_SKILL_DIR" = "__DEVLYN_SKILL_DIR__" ] || [ ! -d "$DEVLYN_SKILL_DIR/../_shared" ]; then
|
|
36
|
+
echo "BLOCKED:shared-dir-unresolved: $DEVLYN_SKILL_DIR/../_shared" >&2
|
|
37
|
+
exit 1
|
|
38
|
+
fi
|
|
39
|
+
DEVLYN_SHARED_DIR="$(cd "$DEVLYN_SKILL_DIR/../_shared" && pwd)"
|
|
40
|
+
if [ ! -f "$DEVLYN_SHARED_DIR/spec-verify-check.py" ]; then
|
|
41
|
+
echo "BLOCKED:shared-dir-unresolved: $DEVLYN_SHARED_DIR/spec-verify-check.py" >&2
|
|
42
|
+
exit 1
|
|
43
|
+
fi
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
Use `python3 "$DEVLYN_SHARED_DIR/spec-verify-check.py"` for all spec validation commands. Claude Code supplies `CLAUDE_SKILL_DIR` by native render substitution; Codex/oh-my-pi installs receive an absolute copy-time stamp in the default branch. If the resolved skill directory is still the placeholder or `../_shared` is absent, stop with `BLOCKED:shared-dir-unresolved`.
|
|
47
|
+
</runtime_paths>
|
|
48
|
+
|
|
30
49
|
<engine_routing>
|
|
31
50
|
Default engine: Claude. The per-engine adapter from `_shared/adapters/<engine>.md` is prepended to the elicitation prompt so the model honors its own official prompt-engineering guidance during the Q&A.
|
|
32
51
|
</engine_routing>
|
|
@@ -86,13 +105,13 @@ Structural lint (inline check, no script needed):
|
|
|
86
105
|
- `## Context` non-empty (≥ 1 sentence).
|
|
87
106
|
- `## Requirements` has ≥ 1 `- [ ]` bullet.
|
|
88
107
|
- `## Out of Scope` present (may list "none" if truly nothing).
|
|
89
|
-
- `## Verification` has either ≥ 1 named command OR
|
|
108
|
+
- `## Verification` is preceded by a `<!-- devlyn:verification -->` sentinel and has either ≥ 1 named command OR sibling `spec.expected.json` declares `"pure_design": true`.
|
|
90
109
|
|
|
91
110
|
After lint passes:
|
|
92
111
|
1. Write `<spec-dir>/<id>-<slug>/spec.md` (the spec).
|
|
93
112
|
2. Generate `<spec-dir>/<id>-<slug>/spec.expected.json` from the spec's `## Verification` block + any `forbidden_patterns` / `required_files` / `forbidden_files` / `max_deps_added` the conversation surfaced.
|
|
94
|
-
3. Run `python3
|
|
95
|
-
4. Run `python3
|
|
113
|
+
3. Run `python3 "$DEVLYN_SHARED_DIR/spec-verify-check.py" --check <spec-path>` to validate the verification carrier shape, supported `complexity` frontmatter, and any present actionable solo-headroom hypothesis; if the spec uses a legacy inline `## Verification` JSON carrier, any solo-headroom hypothesis command must match that carrier's `verification_commands[].cmd`. If exit 2, fix the carrier/frontmatter/hypothesis and re-run.
|
|
114
|
+
4. Run `python3 "$DEVLYN_SHARED_DIR/spec-verify-check.py" --check-expected <expected-path>` to validate sibling `spec.expected.json` against `_shared/expected.schema.json` plus sibling spec `complexity` frontmatter and any present actionable solo-headroom hypothesis; if the spec has a solo-headroom hypothesis, its observable command must match `spec.expected.json.verification_commands[].cmd`. If exit 2, fix the JSON/frontmatter/hypothesis and re-run.
|
|
96
115
|
5. Print: `spec ready — /devlyn:resolve --spec <spec-path>`.
|
|
97
116
|
|
|
98
117
|
## PHASE 1Q: QUICK MODE
|
|
@@ -94,8 +94,8 @@ Before declaring the spec ready, verify structurally:
|
|
|
94
94
|
If the lint fails, fix the missing piece (ask one focused question if needed) before announcing.
|
|
95
95
|
|
|
96
96
|
After lint passes, run both mechanical checks:
|
|
97
|
-
1. `python3
|
|
98
|
-
2. `python3
|
|
97
|
+
1. `python3 "$DEVLYN_SHARED_DIR/spec-verify-check.py" --check <spec-path>` validates the spec's verification carrier shape, supported `complexity` frontmatter, and any present actionable solo-headroom hypothesis; if the spec uses a legacy inline `## Verification` JSON carrier, any solo-headroom hypothesis command must match that carrier's `verification_commands[].cmd`.
|
|
98
|
+
2. `python3 "$DEVLYN_SHARED_DIR/spec-verify-check.py" --check-expected <expected-path>` validates sibling `spec.expected.json` against `_shared/expected.schema.json` plus sibling spec `complexity` frontmatter and any present actionable solo-headroom hypothesis; if the spec has a solo-headroom hypothesis, its observable command must match `spec.expected.json.verification_commands[].cmd`.
|
|
99
99
|
|
|
100
100
|
If either exits 2: read the stderr message, fix the malformed carrier or JSON, and re-run the failed command. Both commands must exit 0 before ready.
|
|
101
101
|
</lint>
|
|
@@ -37,8 +37,8 @@ You must NOT:
|
|
|
37
37
|
3. For each missing/malformed piece: apply the smallest allowed fix.
|
|
38
38
|
4. Write the normalized spec. Default location: `<spec-dir>/<id>-<slug>/spec.md`. With `--in-place` flag: write to `<path>` directly (overwrites the original).
|
|
39
39
|
5. Generate or fix `spec.expected.json` per the rules above. Same dir as the spec.
|
|
40
|
-
6. Run `python3
|
|
41
|
-
7. Run `python3
|
|
40
|
+
6. Run `python3 "$DEVLYN_SHARED_DIR/spec-verify-check.py" --check <spec-path>` to validate the spec carrier and supported `complexity` frontmatter; if the spec uses a legacy inline `## Verification` JSON carrier, any solo-headroom hypothesis command must match that carrier's `verification_commands[].cmd`.
|
|
41
|
+
7. Run `python3 "$DEVLYN_SHARED_DIR/spec-verify-check.py" --check-expected <expected-path>` to validate sibling `spec.expected.json` plus sibling spec `complexity` frontmatter and any present solo-headroom hypothesis command against `spec.expected.json.verification_commands[].cmd`.
|
|
42
42
|
8. If lint still fails after allowed fixes (e.g. Requirements section is empty in the source), surface the issue and exit non-zero — do NOT invent Requirements.
|
|
43
43
|
9. If the preserved Requirements combine state mutation with ordering/priority,
|
|
44
44
|
idempotency, auth/error priority, or exact output shape but the Verification
|
|
@@ -82,7 +82,7 @@ Anything binding all features (e.g. "no new top-level dependencies", "all CLI ou
|
|
|
82
82
|
- `<spec-dir>/<id-N>/spec.md` for each feature (per `references/spec-template.md`).
|
|
83
83
|
- `<spec-dir>/<id-N>/spec.expected.json` for each feature (per `_shared/expected.schema.json`).
|
|
84
84
|
|
|
85
|
-
Each per-feature spec is structurally lint-validated using `python3
|
|
85
|
+
Each per-feature spec is structurally lint-validated using `python3 "$DEVLYN_SHARED_DIR/spec-verify-check.py" --check <spec-path>`, including supported `complexity` frontmatter and any present actionable solo-headroom hypothesis, and each sibling expected contract plus sibling spec `complexity` frontmatter and any present actionable solo-headroom hypothesis is validated using `python3 "$DEVLYN_SHARED_DIR/spec-verify-check.py" --check-expected <expected-path>`; if the spec has a solo-headroom hypothesis, its observable command must match `spec.expected.json.verification_commands[].cmd`.
|
|
86
86
|
|
|
87
87
|
Final announcement: `project ready — N specs at <spec-dir>/. Start with /devlyn:resolve --spec <first-spec-path>`.
|
|
88
88
|
</output>
|
|
@@ -64,9 +64,19 @@ If the spec is small enough that nothing meaningful is out of scope, write `- (n
|
|
|
64
64
|
|
|
65
65
|
## `## Verification`
|
|
66
66
|
|
|
67
|
+
Immediately precede the heading with a `<!-- devlyn:verification -->` sentinel
|
|
68
|
+
comment on its own line, directly above `## Verification` (no blank line
|
|
69
|
+
between them). This is the machine locator `spec-verify-check.py` uses to
|
|
70
|
+
find the section — it is language-neutral: the heading text after the
|
|
71
|
+
sentinel may be any human language (`## Verification`, `## 검증`, etc.); only
|
|
72
|
+
the sentinel itself is fixed ASCII and must never be translated.
|
|
73
|
+
|
|
67
74
|
Concrete commands or tests whose expected behavior names the success condition:
|
|
68
75
|
|
|
69
76
|
```markdown
|
|
77
|
+
<!-- devlyn:verification -->
|
|
78
|
+
## Verification
|
|
79
|
+
|
|
70
80
|
- `node bin/cli.js doctor` exits 0; output contains `doctor:`.
|
|
71
81
|
- `HOME=/nonexistent node bin/cli.js doctor` exits 1; output mentions `/nonexistent`.
|
|
72
82
|
- `node --test tests/cli.test.js` passes.
|
|
@@ -75,14 +85,14 @@ Concrete commands or tests whose expected behavior names the success condition:
|
|
|
75
85
|
|
|
76
86
|
The body of the section reads humanely; the machine-readable contract lives in `spec.expected.json` (sibling file). Each command in this section maps to one entry in `spec.expected.json.verification_commands`.
|
|
77
87
|
|
|
78
|
-
If all Requirements are pure-design (no observable runtime check), the body of this section is a single line
|
|
88
|
+
If all Requirements are pure-design (no observable runtime check), the body of this section is a single descriptive line (e.g. `- No runtime verification commands.`) and sibling `spec.expected.json` declares `"pure_design": true` instead of listing `verification_commands`.
|
|
79
89
|
|
|
80
90
|
## Sibling file: `spec.expected.json`
|
|
81
91
|
|
|
82
|
-
Schema: `_shared/expected.schema.json`. Required when Requirements have observable checks; optional when all Requirements are pure-design. Validate before announcing ready:
|
|
92
|
+
Schema: `_shared/expected.schema.json`. Required when Requirements have observable checks; optional when all Requirements are pure-design (declare `"pure_design": true` in that case — contradictory, and rejected, alongside a non-empty `verification_commands`). Validate before announcing ready:
|
|
83
93
|
|
|
84
94
|
```bash
|
|
85
|
-
python3
|
|
95
|
+
python3 "$DEVLYN_SHARED_DIR/spec-verify-check.py" --check-expected <expected-path>
|
|
86
96
|
```
|
|
87
97
|
|
|
88
98
|
Generated by ideate from the conversation:
|
|
@@ -90,6 +100,7 @@ Generated by ideate from the conversation:
|
|
|
90
100
|
- `forbidden_patterns` ← from Constraints that mention specific anti-patterns ("no silent catches", "no any types").
|
|
91
101
|
- `required_files` / `forbidden_files` ← from Out of Scope and Requirements.
|
|
92
102
|
- `max_deps_added` ← from Constraints (default 0 unless the spec licenses a new dep).
|
|
103
|
+
- `required_risk_probe_requirements` ← for specs where a Requirement combines ordering/priority, rollback, idempotency, auth/signature, concurrency, atomic-batch, or exact output shape, declare the risk categories PHASE 1.5 risk-probe derivation must cover directly, e.g. `[{"tag": "rollback_state", "derived_from": "<exact ## Verification bullet>"}]`. This replaces guessing the required categories from prose — declare them explicitly so the check works in any language. `tag` must be one of the risk-probe tags in `_shared/spec-verify-check.py:RISK_PROBE_TAGS`; `derived_from` must be an exact substring of `## Verification`. Omit entirely when the spec has no ordering/rollback/idempotency/auth/concurrency/atomic-batch/exact-shape risk.
|
|
93
104
|
|
|
94
105
|
## Quality bar (lint enforces structurally; ideate enforces substantively)
|
|
95
106
|
|
|
@@ -97,7 +108,7 @@ Structural (lint):
|
|
|
97
108
|
- Frontmatter valid + required fields present.
|
|
98
109
|
- All five H2 sections present.
|
|
99
110
|
- Requirements has ≥ 1 bullet.
|
|
100
|
-
- Verification has either commands or
|
|
111
|
+
- Verification has either commands or `spec.expected.json` declares `"pure_design": true`.
|
|
101
112
|
|
|
102
113
|
Substantive (ideate's job during elicitation):
|
|
103
114
|
- Each Requirement is independently verifiable.
|
|
@@ -0,0 +1,26 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: devlyn:queue
|
|
3
|
+
description: Manage and drain the intent queue (docs/specs/queue.md) — the backlog the loop-engineering contract runs on. Use when the user wants to stack work for later ("큐에 넣어줘", "queue this"), see what is queued, or start an unattended serial drain ("큐 드레인 시작", "밤새 돌려줘", "drain the queue") that takes each item through spec → /devlyn:resolve → verified done.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
Utility front-end for the intent-queue contract in the project instructions ("Intent queue" in CLAUDE.md / AGENTS.md — whichever this CLI loads). It adds no semantics of its own: the queue file is the API, and the drain loop follows the documented contract exactly — a runner (this skill, a long session, or a future devlyn-os daemon) is a replaceable driver over the same file.
|
|
7
|
+
|
|
8
|
+
<args>
|
|
9
|
+
$ARGUMENTS
|
|
10
|
+
</args>
|
|
11
|
+
|
|
12
|
+
## No args — status
|
|
13
|
+
|
|
14
|
+
Read `docs/specs/queue.md` (absent → report "queue empty — nothing staged" and how to add). Print pending `[ ]`, done `[x]`, and blocked `[F]` counts, the next item up, and one usage line per subcommand.
|
|
15
|
+
|
|
16
|
+
## Subcommands
|
|
17
|
+
|
|
18
|
+
- `add <intent text>` — append `- [ ] <intent>` to `docs/specs/queue.md` (create the file with its header if missing). If the intent came out of a conversation that already produced a spec, link it: `- [ ] (spec: docs/specs/<id>/spec.md) <intent>`.
|
|
19
|
+
- `drain` — serial drain per the project-instructions contract. For each pending item, in order:
|
|
20
|
+
1. Spec it if unspecced (the queue entry is the user's go-ahead). Unattended assumptions may only take scope-narrowing, reversible, non-user-visible defaults; material ambiguity (user-visible behavior, data/state semantics, new files/scripts/flags, implementation surface) → mark `[F] needs-review: <question>` and continue to the next item.
|
|
21
|
+
2. Run `/devlyn:resolve --spec <path>` hands-free.
|
|
22
|
+
3. Outer loop on the terminal verdict: PASS → mark `[x]`. Findings-backed verdicts (NEEDS_WORK, verify/build-gate exhaustion) → amend the spec (recorded in the spec file), re-run — at most 3 outer iterations. Infrastructure / invalid-input / engine-availability / implement-empty BLOCKED verdicts are not spec-amendable → mark `[F] <verdict>` immediately.
|
|
23
|
+
4. A blocked item never halts the queue; continue.
|
|
24
|
+
5. When the queue is drained (or the session must stop), emit the drain report: per-item verdict, every logged assumption, and the commit range produced.
|
|
25
|
+
|
|
26
|
+
Strictly SERIAL — one `/devlyn:resolve` run at a time, never parallel. Never invent queue items, never reorder them, and never delete an item — only mark `[x]` / `[F]`.
|