deepclause-pi 0.1.5 → 0.3.0

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "deepclause-pi",
-  "version": "0.1.5",
+  "version": "0.3.0",
   "description": "Pi-hosted runtime for DeepClause DML programs",
   "type": "module",
   "license": "MIT",

package/src/assets/AGENTS.md

@@ -52,6 +52,10 @@ Generated steps use `executor=dml` for contained model reasoning and `executor=p
 
 Contextual plans require explicit user execution with `/dc-run plans/<name>.dml` and confirmation. They cannot run through model-callable `dc_run`, because that would nest a pi agent turn inside the calling agent turn.
 
+### Change plans
+
+`/dc-plan <request> --change=<slug>` targets a change instead of `plans/`. The same planning turn first creates `changes/<slug>/` with normal file tools (proposal.md, one delta spec per capability under `specs/`, optional design.md), then commits `changes/<slug>/tasks.dml`: `plan_task/2` facts with `satisfies` (scenario ids) and declarative `checks`, plus the managed `plan_task_status/2` block. Every step must declare at least one check. Re-running the same change without `--update` fails once `tasks.dml` exists; `--update` (or a leading `update` keyword) regenerates it and resets every status to pending. Review it with `/dc-check <slug>` and execute it with `/dc-apply <slug>`.
+
 ## Program structure and arguments
 
 Pi passes zero to three positional **strings** to `agent_main`:
@@ -375,6 +379,84 @@ DML is most valuable when an application needs more structure than a prompt and
 
 Poor fits include long-running background services, high-frequency shell automation that would require many approval prompts, workflows needing unrestricted pi tools, secret handling, or durable state without an explicit workspace storage design.
 
+## Diagrams
+
+Pi can turn any DML file into a self-contained, offline Mermaid viewer. Ask for one in plain language:
+
+> "Make a presentation-grade diagram of .pi/deepclause/skills/my_skill.dml"
+> "Give me a specification-grade diagram of src/report.dml"
+
+Pi calls the `dc_diagram` model tool with the DML path and a grade:
+
+- **presentation** — about 8-12 nodes, plain language, headline numbers (slides and overviews).
+- **specification** — function names, task/tool roles, post-conditions (engineers).
+
+The tool extracts a deterministic Mermaid seed, has pi rewrite it in the chosen grade, validates the result, writes the viewer under `.pi/deepclause/diagrams/`, and opens it. The DML file may live anywhere (workspace-relative or absolute); only the generated viewer stays under `.pi/deepclause/`.
+
+Do not hand-write Mermaid for the user, and do not copy diagram tooling into the workspace. Regenerating a grade replaces only that grade's sidecar (`<name>.presentation.mmd` / `<name>.specification.mmd`).
+
+## Specs and deltas
+
+DeepClause keeps behaviour specs separate from executable plans:
+
+- `.pi/deepclause/specs/**/*.spec.md` — capability specs, the source of truth for behaviour.
+- `.pi/deepclause/changes/<slug>/specs/**/*.md` — change deltas.
+- `.pi/deepclause/lib/specs.dml` — the deterministic parser/validator used by the spec skills.
+- `.pi/deepclause/lib/apply.dml` — the task driver (verify, retry, status write-back).
+- `.pi/deepclause/skills/spec_validate.dml`, `spec_status.dml`, `spec_query.dml`, `spec_graph.dml`.
+
+`tasks.dml` uses `plan_task/2` and `plan_task_status/2` — **not** `task/2`, which collides with
+DML's built-in `task/N` predicate and will not unify after being read.
+
+Specs are plain Markdown and must describe **behaviour only** — no commands, file paths,
+library choices, or implementation plans; those belong in `design.md` or `tasks.dml`.
+Structure:
+
+- `## Purpose`
+- `### Requirement: <name>` followed by prose using SHALL / MUST / SHOULD
+- `#### Scenario: <name>` with `- **WHEN**` and `- **THEN**` (exactly four hashes)
+
+Delta files wrap requirements in `## ADDED Requirements`, `## MODIFIED Requirements`,
+`## REMOVED Requirements`, or `## RENAMED Requirements`. `MODIFIED` carries the full
+replacement requirement.
+
+Validate and inspect without spending model tokens:
+
+- `/dc-check` — parse and validate every spec and delta; reports 3-hash scenarios, missing
+  scenarios and duplicates with line numbers.
+- `/dc-run spec_status` — capability and delta inventory.
+- `/dc-run spec_query <capability>` — one capability's requirements and scenarios.
+- `/dc-run spec_graph capabilities|changes` — deterministic Mermaid graph.
+- `/dc-run spec_merge <change>` — preview the delta merge into `specs/` (read-only).
+- `/dc-run spec_coverage <change>` — which delta scenarios are covered by `tasks.dml`,
+  which tasks lack checks, and which `satisfies` ids are unknown. `/dc-check` reports the
+  same coverage and treats uncovered scenarios as errors once a `tasks.dml` exists.
+- `/dc-run spec_scaffold <change>` — print a draft `tasks.dml` with one task per delta
+  scenario (read-only; fill in executor, tools, expected and checks).
+- `/dc-run spec_apply <change> plan` — list the tasks and the approved verification commands
+  (read-only).
+- `/dc-apply <change>` — execute the remaining tasks, verify each task's declarative checks
+  (`exists`, `cmd`, `model`), retry with the failure feedback (up to 3 attempts), and rewrite the
+  `plan_task_status/2` block in `tasks.dml`. The `cmd(...)` set is approved once before the run;
+  each command runs through the allowlisted `dc_verify_run` tool. A git snapshot is recorded first
+  (in `change.json`, with `applyState`). If the run does not finish, the working tree and the
+  `done`/`failed` statuses are **preserved**: re-run `/dc-apply <change>` to resume from the
+  remaining tasks, or `/dc-apply <change> --abort` to discard the apply and restore the snapshot.
+  A dirty tree refuses a fresh snapshot, so the apply then proceeds without rollback.
+
+After `/dc-plan`, `/dc-apply` and `/dc-archive` leave uncommitted changes, the extension offers
+to commit them (`git add -A` with a suggested message) or reminds you to. A clean tree is what
+lets the next `/dc-apply` take a rollback snapshot; consider gitignoring
+`.pi/deepclause/diagrams/` and `.pi/deepclause/changes/*/change.json`.
+- `/dc-archive <change>` — show the preview, confirm, write the merged spec, then move the
+  change to `changes/archive/`. The DML step (`spec_archive.dml`) is marked `% Mutating: true`,
+  so `/dc-run` refuses it directly; always archive through `/dc-archive` so the merge is reviewed
+  first and the change folder is moved.
+- `dc_spec_graph` — pi tool that renders the capability/change graph in the diagram viewer.
+
+These skills are pure DML utilities: do not add model calls or runtime tools to them, and do
+not rewrite the parser by hand.
+
 ## Conservative editing rules
 
 When modifying an existing skill:

package/src/assets/apply.dml

@@ -0,0 +1,188 @@
+%%% apply.dml — deterministic task driver, consulted by spec_apply.dml.
+%%%
+%%% Reads changes/<change>/tasks.dml (plan_task/2 + plan_task_status/2),
+%%% executes the remaining tasks, verifies each task's checks, retries with
+%%% feedback, and rewrites the managed status block.
+%%%
+%%% Execution is delegated per task: executor=pi calls the pi_agent_step bridge;
+%%% executor=dml uses task/N with the current model. Checks are declarative:
+%%% exists(Path), cmd(Command), model(Question).
+
+:- consult('.pi/deepclause/lib/specs.dml').
+
+sp_change_dir(Change, Dir) :- atomic_list_concat([".pi/deepclause/changes/", Change], Dir).
+
+file_exists(Path) :-
+    (   atom_string(PathAtom, Path)
+    ->  true
+    ;   PathAtom = Path
+    ),
+    catch((open(PathAtom, read, Stream), close(Stream)), _, fail).
+
+remaining_tasks(Tasks, Statuses, Remaining) :-
+    findall(plan_task(Id, Props), (
+        member(plan_task(Id, Props), Tasks),
+        \+ sp_status_done(Statuses, Id)
+    ), Remaining).
+
+%% Snapshot/accept are optional: without the tools (or outside git) the apply just
+%% proceeds without rollback, and the report says so.
+take_snapshot(Snap) :-
+    (   catch(exec(dc_apply_snapshot, S), Err, S = error(Err)),
+        nonvar(S)
+    ->  Snap = S
+    ;   Snap = none
+    ).
+
+accept_apply(none) :- !.
+accept_apply(error(_)) :- !.
+accept_apply(_) :- catch(exec(dc_apply_accept, _), _, true).
+
+rollback_line(none, "rollback: unavailable (snapshot tool inactive or not a git repo)").
+rollback_line(error(Err), Line) :- !, format(string(Line), "rollback: unavailable (~w)", [Err]).
+rollback_line(Ref, Line) :-
+    format(string(Line), "rollback: snapshot ~w recorded (restored by /dc-apply on a non-OK run)", [Ref]).
+
+run_plan(Change, Max, Report) :-
+    sp_change_dir(Change, Dir),
+    atomic_list_concat([Dir, "/tasks.dml"], TasksPath),
+    sp_tasks(Dir, Tasks),
+    take_snapshot(Snap),
+    sp_task_statuses(Dir, Statuses0),
+    remaining_tasks(Tasks, Statuses0, Remaining),
+    length(Tasks, Total),
+    length(Remaining, RemainingCount),
+    format(string(Start), "apply ~w: ~w tasks, ~w remaining, max ~w attempts", [Change, Total, RemainingCount, Max]),
+    output(Start),
+    run_all(TasksPath, Change, Remaining, Max, Statuses0, Statuses),
+    sp_task_statuses(Dir, FinalStatuses),
+    findall(Id, (member(plan_task(Id, _), Tasks), \+ sp_status_done(FinalStatuses, Id)), Outstanding),
+    length(Outstanding, OutstandingCount),
+    Done is Total - OutstandingCount,
+    sp_count_repairs(FinalStatuses, Repairs),
+    format(string(Header), "apply ~w: ~w/~w tasks verified (~w repairs)", [Change, Done, Total, Repairs]),
+    (   OutstandingCount =:= 0
+    ->  accept_apply(Snap),
+        Result = ["status: OK"]
+    ;   findall(Line, (member(Id, Outstanding), format(string(Line), "  outstanding: ~w", [Id])), Lines),
+        Result = ["status: INCOMPLETE"|Lines]
+    ),
+    rollback_line(Snap, RollbackLine),
+    append([Header], Result, Body0),
+    append(Body0, [RollbackLine], Body),
+    atomics_to_string(Body, "\n", Report).
+
+run_all(_, _, [], _, Statuses, Statuses).
+run_all(TasksPath, Change, [plan_task(Id, Props)|Rest], Max, Statuses0, Statuses) :-
+    run_task(TasksPath, Statuses0, Change, Id, Props, Max, Statuses1),
+    run_all(TasksPath, Change, Rest, Max, Statuses1, Statuses).
+
+run_task(TasksPath, Statuses0, Change, Id, Props, Max, Statuses) :-
+    attempt(TasksPath, Statuses0, Change, Id, Props, 1, Max, none, Statuses).
+
+attempt(TasksPath, Statuses0, Change, Id, Props, N, Max, Feedback, Statuses) :-
+    N =< Max,
+    format(string(Progress), "task ~w attempt ~w/~w", [Id, N, Max]),
+    output(Progress),
+    execute(Change, Id, Props, Feedback, Summary),
+    verify(Props, Summary, Verdict),
+    (   Verdict = ok
+    ->  sp_set_status(Statuses0, Id, done(N), Statuses),
+        sp_write_statuses(TasksPath, Statuses),
+        format(string(Done), "task ~w verified on attempt ~w", [Id, N]),
+        output(Done)
+    ;   Verdict = fail(Evidence),
+        N1 is N + 1,
+        sp_set_status(Statuses0, Id, failed(N1, Evidence), Statuses1),
+        sp_write_statuses(TasksPath, Statuses1),
+        format(string(Msg), "task ~w failed verification: ~w", [Id, Evidence]),
+        output(Msg),
+        attempt(TasksPath, Statuses1, Change, Id, Props, N1, Max, Evidence, Statuses)
+    ).
+attempt(_, Statuses, _, Id, _, N, Max, _, Statuses) :-
+    N > Max,
+    format(string(Msg), "task ~w exhausted ~w attempts", [Id, Max]),
+    output(Msg).
+
+execute(Change, Id, Props, Feedback, Summary) :-
+    catch(get_dict(executor, Props, Executor), _, Executor = dml),
+    catch(get_dict(do, Props, Do), _, Do = ""),
+    catch(get_dict(expected, Props, Expected), _, Expected = ""),
+    build_instruction(Id, Do, Expected, Feedback, Instruction),
+    (   Executor == pi
+    ->  catch(get_dict(tools, Props, Tools), _, Tools = []),
+        exec(pi_agent_step(instruction: Instruction, tools: Tools, expected: Expected, skills: []), Summary)
+    ;   task(Instruction, string(Summary))
+    ).
+
+build_instruction(Id, Do, Expected, none, Instruction) :-
+    format(string(Instruction), "Task ~w. ~w~nExpected result: ~w", [Id, Do, Expected]).
+build_instruction(Id, Do, Expected, Feedback, Instruction) :-
+    Feedback \= none,
+    format(string(Instruction),
+        "Task ~w. ~w~nExpected result: ~w~n~nThe previous attempt failed verification: ~w~nFix only what is needed; do not redo the whole task.",
+        [Id, Do, Expected, Feedback]).
+
+verify(Props, Summary, Verdict) :-
+    catch(get_dict(checks, Props, Checks), _, Checks = []),
+    findall(Message, (member(Check, Checks), run_check(Check, Message), Message \= ok), Failures),
+    (   Summary == ""
+    ->  Verdict = fail("delegated step returned no summary")
+    ;   Failures == []
+    ->  Verdict = ok
+    ;   atomic_list_concat(Failures, "; ", Evidence),
+        Verdict = fail(Evidence)
+    ).
+
+run_check(exists(Path), Result) :-
+    (   file_exists(Path)
+    ->  Result = ok
+    ;   format(string(Result), "missing artifact ~w", [Path])
+    ).
+run_check(cmd(Command), Result) :- run_check(cmd(Command, 1), Result).
+run_check(cmd(Command, _), Result) :-
+    catch(exec(dc_verify_run(command: Command), Dict), Err, Dict = error(Err)),
+    (   Dict = error(Err)
+    ->  format(string(Result), "command error: ~w", [Err])
+    ;   get_dict(exitCode, Dict, Code),
+        (   Code =:= 0
+        ->  Result = ok
+        ;   catch(get_dict(stderr, Dict, ErrText), _, ErrText = ""),
+            normalize_space(string(Short), ErrText),
+            format(string(Result), "command failed: ~w (~w)", [Command, Short])
+        )
+    ).
+run_check(model(Question), Result) :-
+    format(string(Instruction), "Verify by inspection only; do not modify. ~w Reply PASS or FAIL with a short reason.", [Question]),
+    catch(exec(pi_agent_step(instruction: Instruction, tools: ["read"], expected: "PASS or FAIL", skills: []), Verdict), _, Verdict = ""),
+    (   sub_string(Verdict, _, _, _, "PASS")
+    ->  Result = ok
+    ;   format(string(Result), "model check failed: ~w", [Verdict])
+    ).
+
+check_command(cmd(Command), Command) :- !.
+check_command(cmd(Command, _), Command) :- !.
+
+plan_report(Change, Report) :-
+    sp_change_dir(Change, Dir),
+    sp_tasks(Dir, Tasks),
+    length(Tasks, Total),
+    findall(Line, (
+        member(plan_task(Id, Props), Tasks),
+        catch(get_dict(checks, Props, Checks), _, Checks = []),
+        catch(get_dict(do, Props, Do), _, Do = ""),
+        format(string(Line), "  task ~w: ~w  checks: ~w", [Id, Do, Checks])
+    ), Lines),
+    findall(Command, (
+        member(plan_task(_, Props), Tasks),
+        catch(get_dict(checks, Props, Checks), _, Checks = []),
+        member(Check, Checks),
+        check_command(Check, Command)
+    ), Commands0),
+    sort(Commands0, Commands),
+    findall(CommandLine, (member(Command, Commands), format(string(CommandLine), "command: ~w", [Command])), CommandLines),
+    format(string(Header), "apply plan ~w: ~w tasks", [Change, Total]),
+    append([Header], Lines, A),
+    append(A, [""], B),
+    append(B, CommandLines, Body),
+    atomics_to_string(Body, "\n", Report).

package/src/assets/spec_apply.dml

@@ -0,0 +1,20 @@
+% spec_apply — execute a change's tasks.dml with verification and bounded retries.
+% Mutating: true
+% Usage: /dc-run spec_apply <change>         (or /dc-apply <change>)
+%        /dc-run spec_apply <change> plan    (preview, read-only)
+:- consult('.pi/deepclause/lib/apply.dml').
+
+agent_main(Change, "plan") :-
+    plan_report(Change, Report),
+    answer(Report).
+
+agent_main(Change) :-
+    run_plan(Change, 3, Report),
+    answer(Report).
+
+agent_main(Change, _Mode) :-
+    run_plan(Change, 3, Report),
+    answer(Report).
+
+agent_main :-
+    answer("usage: /dc-run spec_apply <change> [plan]").

package/src/assets/spec_archive.dml

@@ -0,0 +1,11 @@
+% spec_archive — merge a change delta into specs/ and move the change to archive/.
+% Mutating: true
+% Usage: /dc-run spec_archive <change>
+:- consult('.pi/deepclause/lib/specs.dml').
+
+agent_main(Change) :-
+    sp_archive(Change, apply, Report),
+    answer(Report).
+
+agent_main :-
+    answer("usage: /dc-run spec_archive <change>").

package/src/assets/spec_coverage.dml

@@ -0,0 +1,26 @@
+% spec_coverage — scenario coverage of a change by its tasks.dml.
+% Usage: /dc-run spec_coverage <change>
+:- consult('.pi/deepclause/lib/specs.dml').
+
+agent_main(Change) :-
+    atomic_list_concat([".pi/deepclause/changes/", Change], ChangeDir),
+    sp_coverage(ChangeDir, Scenarios, Tasks, Uncovered, NoCheck, Orphan),
+    length(Scenarios, ScenarioCount),
+    length(Tasks, TaskCount),
+    length(Uncovered, UncoveredCount),
+    Covered is ScenarioCount - UncoveredCount,
+    format(string(Header), "scenario coverage ~w: ~w/~w covered, ~w tasks", [Change, Covered, ScenarioCount, TaskCount]),
+    (   Uncovered == []
+    ->  UncoveredLines = ["  all scenarios covered"]
+    ;   findall(Line, (member(Id, Uncovered), format(string(Line), "  uncovered: ~w", [Id])), UncoveredLines)
+    ),
+    findall(Line2, (member(Id, NoCheck), format(string(Line2), "  task without checks: ~w", [Id])), NoCheckLines),
+    findall(Line3, (member(Id, Orphan), format(string(Line3), "  satisfies unknown scenario: ~w", [Id])), OrphanLines),
+    append([Header], UncoveredLines, A),
+    append(A, NoCheckLines, B),
+    append(B, OrphanLines, Body),
+    atomics_to_string(Body, "\n", Report),
+    answer(Report).
+
+agent_main :-
+    answer("usage: /dc-run spec_coverage <change>").

package/src/assets/spec_graph.dml

@@ -0,0 +1,12 @@
+% spec_graph — deterministic Mermaid graphs of specs and changes.
+% Usage: /dc-run spec_graph [capabilities|changes]
+% Views: capabilities (default), changes.
+:- consult('.pi/deepclause/lib/specs.dml').
+
+agent_main :-
+    sp_graph(capabilities, Mermaid),
+    answer(Mermaid).
+
+agent_main(View) :-
+    sp_graph(View, Mermaid),
+    answer(Mermaid).

package/src/assets/spec_merge.dml

@@ -0,0 +1,10 @@
+% spec_merge — preview merging a change delta into specs/ (read-only).
+% Usage: /dc-run spec_merge <change>
+:- consult('.pi/deepclause/lib/specs.dml').
+
+agent_main(Change) :-
+    sp_archive(Change, plan, Report),
+    answer(Report).
+
+agent_main :-
+    answer("usage: /dc-run spec_merge <change>").

package/src/assets/spec_query.dml

@@ -0,0 +1,10 @@
+% spec_query — deterministic capability inspector.
+% Usage: /dc-run spec_query <capability>     e.g. /dc-run spec_query ui/theme
+:- consult('.pi/deepclause/lib/specs.dml').
+
+agent_main(Capability) :-
+    sp_query(Capability, Report),
+    answer(Report).
+
+agent_main :-
+    answer("usage: /dc-run spec_query <capability>   e.g. ui/theme").

package/src/assets/spec_scaffold.dml

@@ -0,0 +1,10 @@
+% spec_scaffold — draft a tasks.dml for a change from its delta scenarios (read-only).
+% Usage: /dc-run spec_scaffold <change>
+:- consult('.pi/deepclause/lib/specs.dml').
+
+agent_main(Change) :-
+    sp_scaffold(Change, Draft),
+    answer(Draft).
+
+agent_main :-
+    answer("usage: /dc-run spec_scaffold <change>").

package/src/assets/spec_status.dml

@@ -0,0 +1,7 @@
+% spec_status — deterministic spec/delta inventory.
+% Usage: /dc-run spec_status
+:- consult('.pi/deepclause/lib/specs.dml').
+
+agent_main :-
+    sp_status(Report),
+    answer(Report).

package/src/assets/spec_validate.dml

@@ -0,0 +1,9 @@
+% spec_validate — deterministic spec/delta validation.
+% Usage: /dc-run spec_validate
+% Pure DML: no model calls, no runtime tools. Reports structural errors with
+% line numbers (3-hash scenarios, missing scenarios, duplicates).
+:- consult('.pi/deepclause/lib/specs.dml').
+
+agent_main :-
+    sp_check_all(Report),
+    answer(Report).