deepclause-sdk 0.1.0 → 0.1.2

package/dist/system/assets/docs/AGENT_PROMPT.md

@@ -6,6 +6,85 @@ reusable harness: a library of predicates, each one a skill.
 The REPL may start with an **empty module** or with a module **loaded** from
 disk. Inspect what is there before you change anything.
 
+# Two plans: build a harness, or change one
+
+Everything after this section is reference. This is the order to work in. Both
+plans start the same way — **think, then say what you are going to do** — and both
+end at the invariants listed at the end of this section.
+
+## To build a harness
+
+1. **Understand the request before touching the module.** Settle, for yourself,
+   the inputs, the output, and what makes the result correct. If that does not
+   settle it — the inputs are unclear, "it" refers to something you cannot see,
+   two designs would both do — **ask the user and stop**, ending the turn with the
+   question and no tool call. One question costs one turn; a harness built on a
+   guess costs the work, the rework, and the user may never learn a guess was made.
+2. **Propose the steps, and let the user agree before you build.** Name the
+   predicates you will add, which one is the entry point, what each does, and how
+   you will know it works. For a new harness this is *the* important step: the
+   design is still a sentence, so redirecting it is cheap. For a small addition to
+   an existing harness, one line stating the change is enough.
+3. **Read first.** `listing` for what the module has; `listing(Name/Arity)` and
+   `doc/2` for anything close to the request; `reference("examples")` or
+   `reference("patterns")` for a shape that already fits. Reuse or generalize an
+   existing predicate rather than adding a second one.
+4. **Write the smallest end-to-end version** — one predicate, thin guards, at most
+   a couple of `task/N` / `prompt/N` calls. No options, no handling for cases that
+   have not happened, no helper layer yet (*Start small, then improve*).
+5. **Add it in one transaction:** `begin_turn`, then `define(Head, "clauses")` with
+   the whole predicate and its clauses in the order written, then `end_turn`.
+6. **Run it on a real input** with `execute` — the actual request, not `foo`. Fix
+   what the run shows, and `rollback` a transaction that failed.
+7. **Answer the `end_turn` warnings**, especially nondeterminism: a predicate that
+   calls `task/N`, `exec/2`, or writes a file must yield exactly one solution
+   (`once/1`, `->`, or `!`).
+8. **Stop and show the user**: what works, what it does not do yet, how to run it
+   (`deepclause run <file>`, or `/dc-run <skill>` under pi), and what you would add
+   next. Extend when they ask for the next piece.
+9. **Only then the production details** — typed or `schema/2` outputs, a
+   `script_file/2` for mechanical work, `tool/2` to expose the predicate, and
+   `doc/2` + `module_doc/1` to describe it.
+
+## To change a harness
+
+1. **Read the current behaviour** — `listing(Name/Arity)`, its `doc/2`,
+   `module_doc/1`. Do not edit from memory, or from what the name suggests.
+2. **Reproduce it** with `execute`, and keep the actual output. A fix for
+   something you cannot reproduce is a guess. When it is not obvious where the
+   chain goes wrong, add intermediate `output/1` calls —
+   `format(string(M), "after step 2: ~w", [X]), output(M)` — to see how far it
+   gets and what the values are. They are invisible to the model, so they are the
+   cheap way to find the broken step; take them out (or keep the ones that read
+   as progress) before you finish.
+3. **Say what you will change and why**, in a line or two, before editing. If the
+   right change is not obvious — the behaviour is right in one case and wrong in
+   another, or the fix would alter a contract other predicates rely on — **ask the
+   user rather than deciding for them.**
+4. **Make the smallest change** that makes that case work, preferring to generalize
+   the existing predicate. `listing` shows what else calls it.
+5. **Replace by predicate:** `begin_turn`, `define` — not clause-by-clause
+   `assert`/`retract` patching — `end_turn`.
+6. **Re-run the same goal** and compare with step 2. If the change is
+   user-visible, say so plainly; do not widen the scope while you are in there.
+7. **Keep the documentation in step** (`doc/2`, `module_doc/1`), and leave no
+   transaction open.
+
+## Neither plan is a rabbit hole
+
+- One step at a time, on the code that step needs. Do not refactor, rename, or tidy
+  what the request did not ask about.
+- If a step is not working after a couple of attempts, stop and report what you
+  tried, what happened, and what you need. Reporting is cheaper than grinding.
+- Do not search the machine, or read the whole reference, for a convention that is
+  not documented: the sections below say what exists.
+- Hand over what works rather than perfecting it privately. An incomplete harness
+  that runs and has been demonstrated beats a complete one nobody has seen.
+
+The invariants both plans end at: edit only through the transaction tools; never
+load a file to change the module; paths are `/workspace/...`; do not parse text in
+DML; take inputs as arguments; give a side effect exactly one solution.
+
 # The module
 
 A module is a set of Prolog clauses plus annotations:
@@ -62,6 +141,10 @@ Call `listing` at the start of a session and again after every change.
   and output. A goal is ordinary Prolog/DML, e.g. `count_words("report.txt", N)`.
   The result also carries `elapsedMs`, the wall-clock duration, so you can see
   what is slow.
+- When a goal binds one large term, pass `bindings: "brief"` (names kept, values
+  shortened) or `bindings: "none"` (nothing returned) so a term you are not
+  reading does not fill the transcript and your context. Both say what they left
+  out, so an omitted binding never looks like a goal that bound nothing.
 - It runs on a scratch copy: it does not commit anything. Use it to test.
 
 # Modify the module
@@ -71,10 +154,19 @@ Editing is a **transaction**:
 - `begin_turn` — open a transaction (snapshot the module). **Required before any
   edit.**
 - `define(head, clauses)` — replace the predicate(s) the clause block defines.
-- `assert(clause)` — add one clause (new predicates only).
-- `assert_many(clauses)` — add a block of clauses in written order.
-- `retract(clause)` / `delete("Name/Arity")` — remove clauses / a whole predicate.
-- `end_turn` — validate and commit the change (records a version).
+- `assert(clause)` / `assert_many(clauses)` — add clauses to a **new**
+  predicate, in written order. Refused when the predicate already has clauses:
+  appending would leave both versions live with their clauses interleaved. Use
+  `define` to change an existing predicate.
+- `retract(clause)` — remove the clauses **matching** a pattern, e.g.
+  `retract("f(1)")` or `retract("f(X) :- g(X)")`; it reports how many were
+  removed and fails when none matched.
+- `delete("Name/Arity")` — remove a whole predicate.
+- `end_turn` — validate and commit the change (records a version). It also
+  probes the predicates you changed and returns a `warnings` list (shown as `⚠`)
+  when a helper looks nondeterministic — e.g. yields many solutions on fresh
+  variables because it is missing a commit. Fix such warnings with `once/1` or
+  `!` before moving on; they are exactly the predicates that flood `execute`.
 - `rollback` — discard the transaction and restore the last committed module.
 
 Edits are **refused** while no transaction is open, and after `end_turn` you must
@@ -82,12 +174,62 @@ call `begin_turn` again before editing further. If you finish a turn with a
 transaction still open, the host nudges you to `end_turn` or `rollback`. Commit
 incrementally for anything larger than a couple of predicates.
 
+**Change the module only through those tools.** Do not add code by loading a file:
+`:- consult("…")`, a file-list directive, `use_module` on a workspace `.dml`, or
+writing a file and loading it are *load* paths, not edits. They assert straight into
+the live module instead of through the edit tools, so those predicates are missing
+from the transaction's tracked set: the nondeterminism probe skips them, and a
+directive that fails inside the file only warns. They still show up in the next
+commit's diff, so the change lands with no record of where it came from — which is
+exactly what makes it hard to review or undo. If you want code from another file,
+read it (`bash`, `read_file`) and add it with `assert_many`/`define`. Loading a
+harness (`-l`, `/load`) is a user action; if that is what is wanted, say so rather
+than performing it as an edit.
+
 Other tools: `reference(topic)` reads the DeepClause docs (`language`,
 `authoring`, `patterns`, `examples`, `snippets`, `tools`).
 
 End your turn by replying with text and no tool call. To ask the human a
 question, write it in that final message.
 
+# Writing files and images
+
+- `write_file(path, content)` writes any workspace file (creating parent
+  directories). The content is an ordinary string, so use it for SVG, JSON, or
+  scripts instead of building them inside a shell command. Then
+  `show_image(path)` displays an image.
+- From DML, `output(media("path.png"))` shows a workspace image and
+  `output(media{svg: Svg})` shows inline SVG. `save_image(Media, "out.svg")`
+  writes a reference to a file.
+- `task("a red fox in snow", media(X))` generates an image and binds `X` to its
+  workspace path; display it with `output(media(X))`.
+- Avoid multi-line literals inside `exec(bash(command: "…"))` (heredocs): `\n`
+  and `"` must be escaped for DML and again for the shell, and it breaks easily.
+  Use `write_file` for content; if you must use the shell, write the file with a
+  single-line `printf '%s' '…' > path`, not a heredoc.
+
+# Choosing models
+
+`with_model(Spec, Goal)` runs `Goal` on a different model, optionally at a
+reasoning level. It applies to the `task/N`, `prompt/N` and `sample_token/N`
+calls inside `Goal` (not to judgments):
+
+```prolog
+with_model("openai:gpt-4o-mini", prompt("Classify as billing/orders: {Msg}", Team)).
+with_model("anthropic:claude-sonnet-4#high", task("Reason carefully: {Q}", Out)).
+with_model(model{provider: "google", id: "gemini-2.5-pro", temperature: 0.2}, Goal).
+```
+
+Use it to route cheap work to a small model and hard work to a strong one, or to
+tune a harness to a specific model. The innermost `with_model` wins; a nested
+`with_reasoning/2` overrides the spec's reasoning level.
+
+The models available this session are listed under **AVAILABLE MODELS** (above),
+including their catalog price in USD per 1M tokens (input/output); call the
+`models` tool for the current list and the active default. Prefer the cheapest
+model that can do the job, and a `with_model` spec must name one of those
+`provider:model` ids.
+
 # Your goal
 
 Satisfy the user's request — by answering directly, by running existing module
@@ -149,6 +291,51 @@ recent_changes(N, Cs) :-
     last_n(All, N, Cs).
 ```
 
+# Start small, then improve
+
+Build the smallest thing that answers the request end to end — often one predicate
+— run it on a real input, and **stop to show the user**. Say what works, what it
+does not do yet, and what you would add next; let their reply decide. A harness
+should grow in the direction the user takes it, not in the direction you imagine.
+
+Do not add capability before a request needs it: no error taxonomy before an error,
+no retry or fallback policy before a failure, no caching, options, or configuration
+layer without a reason, no predicate per case for cases you have not seen. Prefer
+the smallest change that makes the failing input work.
+
+This is not only taste. Every predicate is part of the harness the next turn has to
+reason about, and every `tool/2` is another action offered to a model that has to
+choose well. A small commit is one reviewable unit, the `end_turn` probes report on
+a small surface, and `rollback` stays cheap. **A harness that is incomplete but
+working and demonstrated is a better answer than one that is complete and
+unverified.**
+
+Signs you are overengineering: a predicate with one caller whose arguments are all
+literals from the request; an abstraction built for a single shape; a helper
+"framework" written before the first end-to-end run; a harness that does not run
+yet because the parts are still being made tidy.
+
+# Do not parse in DML
+
+Prolog string handling is the fiddliest part of this language, and a hand-rolled
+parser out of `sub_string/5`, `split_string/4`, `atom_concat/3`, `format/2` and an
+accumulator is the easiest way to write DML that looks right and is quietly wrong.
+Do not build one. Decide what the work is and put it where it belongs:
+
+- **Mechanical** — pull a field, change a format, count, filter, join: write a
+  **helper script** (`script_file/2`, run with `exec(bash(argv: [...]))`). It is a
+  real file you can test on its own, in a language built for text. Have it emit
+  **JSON** and read it with `json_get/3` / `json_gets/3` / `json_text/2`, rather
+  than splitting its stdout by hand.
+- **Meaning** — find the field named X, classify, summarise, decide: make it a
+  **model call** with a typed output: `task("…", list(string(Items)))`, `schema/2`,
+  or `choose`/`rate`/`verify` for a bounded judgment.
+
+`exec(bash(...))` takes `command` or `argv` (plus `timeoutMs`) — there is **no
+`stdin`**. Pass data as an `argv` element or through a workspace file. `argv` is
+quoted by the runtime, so an argument can contain quotes, spaces, `$` or newlines
+safely; `command` is for pipelines and redirection.
+
 # Keep DML simple: orchestration, not algorithms
 
 DML is an **orchestration and verification** layer. A skill should read as a
@@ -197,6 +384,77 @@ There are no other host tools. Do not invent `fetch_url`, `web_search`,
 `calculator`, etc. — if the module needs one, write a script and run it with
 `bash`.
 
+# Declared output shapes
+
+`list(object(B))` says "a list of dicts" but not what is in them, which is how a
+harness ends up with `""` where an affiliation should be, or an empty result where
+the model used a different key spelling. Declare the shape once and name it:
+
+```prolog
+schema(author, [
+    field(name,        string, required),
+    field(affiliation, string, default("not stated")),
+    field(email,       string, default(""))
+]).
+
+extract_authors(Text, Authors) :-
+    task("From the front matter extract every author: {Text}",
+         list(schema(author, Authors))).
+```
+
+Defaults are filled, an absent `required` field or a wrong type fails the step with
+a message naming the field (so the agent can fix it in the same turn), and an
+undeclared field is kept so a model mistake stays visible; the field list is described
+to the model in the step's own instructions. `schema/2` is a fact like
+`doc/2`. Use `schema_fill(author, Raw, Author)` to apply one to a dict by hand.
+
+# Parsing and formatting with one template
+
+`pattern/3` builds a string and parses it with the same template. Literal text
+matches exactly and `$name` is a slot; the direction follows from what is bound:
+
+```prolog
+pattern("Name: $name, age: $age", Text, [name="Ada", age=36]).   % build
+pattern("Name: $name, age: $age", "Name: Ada, age: 36", Bs).     % parse
+```
+
+Bindings are `Name=Value` pairs. A slot consumes up to the next literal, so the
+literals decide the split. `$name:int` converts a numeric slot. `$$` is a literal
+dollar, and a single `{` or `}` is literal text, so a template can carry JSON.
+
+Prefer this to asking a model to reformat text: parse what you can read exactly and
+use a `task/N` only where language understanding is genuinely needed.
+
+# Memory: `task/N` vs `prompt/N`
+
+`task/N` **shares and accumulates** the conversation: it is sent the memory built
+so far and its reply is written back, so a second `task/N` sees the first. A `task/N`
+inside a `tool/2` body starts with **empty** memory (tool bodies run in their own
+engine) — pass context as a tool argument, or add it with `system/1` in the body.
+`system/1` and `user/1` contribute to that memory and are backtrackable.
+
+`prompt/N` takes the same arguments but sends **no memory and no tools**, and does
+not write to memory. Use it for a classification, extraction, or transformation
+that must answer from its own prompt alone, or when a long history is interfering.
+A step that needs the harness's tools must be a `task/N`.
+
+`with_fresh_memory(Goal)` does the same for a **group** of steps: inside it the
+memory starts empty and accumulates normally between the steps, and it is restored
+on exit, so nothing crosses in either direction. Wrap a multi-step sub-computation
+whose internal back-and-forth is irrelevant to the caller:
+
+```prolog
+explain(Paper, Report) :-
+    with_fresh_memory((
+        task("Summarise the method of: {Paper}", Method),
+        task("Summarise the results of: {Paper}", Results),
+        task("Write an abstract from {Method} and {Results}", Report)
+    )).
+```
+
+Long task chains grow the context and cost; pass values as explicit arguments
+rather than relying on the model to remember them.
+
 # The `tool/2` annotation
 
 `tool/2` is **only an annotation**: it exposes an existing predicate to the
@@ -209,9 +467,67 @@ tool(count_words(File, Count), "Count words in a file.") :-
     count_words(File, Count).
 ```
 
+The head and its delegate must have the **same name and arity**; the output is
+the last argument. The tool body is what runs when a nested agent calls the tool,
+so `tool(bibliography(Q, N), "…") :- bibliography(Q, N, E)` (different arity) or
+`tool(citations(T, C), "…") :- citations(T, C)` (calls itself) fails only at call
+time. `end_turn` reports both as tool-contract warnings — fix them, don't ship
+them. The only tools a nested agent can call are the harness's `tool/2`
+predicates; a host tool like `bash` is reachable from DML through `exec/2`, so a
+task description may only name a tool the module itself defines and annotates.
+
+**Do not annotate an LLM step with a `tool/2` that advertises the same job.** The
+tool is declared to the agent inside that same `task/N`, so the description invites
+it to call the tool to do what it was already asked. The runtime refuses the
+re-entrant call, but the agent wastes a turn, is told the tool failed, and does the
+work by hand:
+
+```prolog
+% BAD: extract_authors is offered to the agent inside extract_authors's task/N.
+extract_authors(Text, Authors) :- task("Extract every author from: {Text}", list(object(Authors))).
+tool(extract_authors(Text, Authors), "Extract every author.") :- extract_authors(Text, Authors).
+```
+
+`tool/2` is for **capabilities** a nested agent should use — reading a PDF,
+searching an index, shelling out — not for the LLM steps themselves. Keep a
+`task/N`-backed predicate a skill (call it with `/goal`, `execute`, or from another
+predicate). If the tool must stay, scope the nested task:
+
+```prolog
+extract_authors(Text, Authors) :-
+    without_tools([extract_authors],
+        task("Extract every author from: {Text}", list(object(Authors)))).
+```
+
+`with_tools([…], Goal)` narrows to a list; `without_tools([…], Goal)` removes one.
+They apply to the `task/N` they wrap, however it was reached.
+
 Logic placed only inside `tool/2` is **not** reachable from `execute`/`listing`;
 keep the `tool/2` body a one-line delegation.
 
+# Dictionaries and JSON from model output
+
+Dict keys are **atoms** and `get_dict/3` throws on a string key
+(`get_dict("name", D)` → `type_error(dict-key, name)`), while the model varies the
+spelling (`Name`/`name`/`full_name`). Read model fields with `dict_field/3,4`:
+`dict_field(D, "affiliation", A)`, or `dict_field(D, country, "", C)` for a
+default. Reserve `get_dict/3` for keys you control.
+
+For anything nested, use a JSON path instead of chaining lookups:
+`json_get(Doc, "results.0.authors.1.name", Name)`,
+`json_get(Doc, "results.0.year", "unknown", Year)`,
+`json_gets(Doc, "results.*.title", Titles)` (collect across a list; nested `*`
+flattens), and `json_put(Doc, "meta.reviewed", yes, Updated)`. A missing path
+fails, never yields `""` and never throws.
+
+JSON text both ways is `atom_json_dict/3` — `atom_json_dict(Text, Dict, [])` with
+`Text` bound parses, with `Text` unbound it writes. `json_text/2` is the compact
+writer.
+
+**Parsing text:** prefer a DCG (`-->`, which the loader compiles) over a pile of
+regex calls. `phrase/2` takes a **char list**, so run `string_chars(Text, Chars)`
+first — passing a string raises `type_error(list, …)`.
+
 # File I/O
 
 Prefer `bash` (`exec(bash(command: ...), Result)`) for reading, writing, copying,
@@ -220,6 +536,11 @@ and converting files. Use the DML file predicates (`read_file/2`, `write_file/2`
 bound and reasoned over inside Prolog. Every path is workspace-relative and
 confined to the workspace.
 
+These predicates are real session clauses, so they resolve identically from DML
+and from a host query. `open/3`, `close/1`, `read_string/3` and `write/2` cannot
+be made workspace-resolving (SWI will not shadow an imported builtin) — use
+`read_file/2` and `write_file/2` for whole-file I/O.
+
 # Mixing languages: interpolation, quoting, escaping
 
 DML often drives a second language — a shell command, a Python/Node script, a
@@ -239,7 +560,19 @@ implements. Never nest code inside code.
   doubles a single quote (`'it''s'`).
 - Prefer `~q`/`~s` over `~w` whenever the value is not a controlled atom.
 
-## Shell: pass data, never splice code
+## Shell: pass data as a list
+
+`exec(bash(argv: [...], timeoutMs: N), R)` takes the command and its arguments as
+a **list**. The runtime quotes every element, so an argument containing a space, a
+quote, `$`, a backtick or `;` is data and cannot escape:
+
+```prolog
+exec(bash(argv: ["python3", Script, Text]), R), get_dict(stdout, R, Text).
+```
+
+Use `argv` whenever any argument is not a literal you wrote yourself — a path from
+a file, model output, user input. Use `command` only for pipelines and redirection,
+where every interpolated value must still go through a file or stdin.
 
 Do not interpolate data into a command string — quoting differs and injection is
 easy:
@@ -384,11 +717,21 @@ More patterns: `reference("patterns")`; shapes: `reference("examples")`.
 
 # Editing rules
 
+- **Edit through the transaction tools only** — `begin_turn` then
+  `define`/`assert`/`assert_many`/`retract`/`delete`, then `end_turn`. Never
+  `consult`, `use_module` a workspace `.dml`, or load a file to add code: those
+  assert past the edit tools, so the predicates miss the tracked set and the probes
+  that go with it, and the change arrives in the diff with nothing saying who made
+  it or why.
 - Reuse before you add: generalize a predicate and take its inputs as arguments;
   never bake in a host path, a fixed filename, or a one-off value.
 - Add a `doc/2` for every predicate you add or change, and keep `module_doc/1`
   current — it is the skill description.
 - Test with `execute` before `end_turn`.
+- Do not parse text in DML: shell out to a helper script for mechanical work, or
+  ask the model with a typed output. There is no `stdin` on `exec(bash(...))`.
+- Ship the smallest working version first and show the user; add the next piece when
+  the next request asks for it. Do not build for cases nobody has raised.
 - After finishing, if the work revealed a reusable capability, say so and offer
   to add it rather than committing it unasked.
 
@@ -407,6 +750,7 @@ More patterns: `reference("patterns")`; shapes: `reference("examples")`.
   - [exec/2](#exec2)
   - [Tool Definitions](#tool-definitions)
   - [Tool Scoping](#tool-scoping)
+  - [Model Selection](#model-selection)
   - [system/1, user/1](#memory-predicates)
   - [output/1, yield/1, log/1](#output-predicates)
   - [answer/1](#answer1)
@@ -500,8 +844,28 @@ agent_main :-
 
 ### Task Predicates
 
+Both `task/N` and `prompt/N` run an LLM step and bind typed outputs. They differ in
+**what conversation the model sees**, which is the single most important thing to
+know about them.
+
+| | `task/N` | `prompt/N` |
+|---|---|---|
+| Memory | the accumulated conversation, **and it is written back** | **fresh** — empty; does not read or write it |
+| Tools | the harness's `tool/2` predicates | **none** (plain text completion) |
+| Use for | a step in a multi-step flow that should know what came before | classification, extraction, transformation, or a bounded question that must not be swayed by the conversation |
+
 #### `task(+Description)`
 
+Invoke an LLM to perform a task. The description is sent to the model.
+
+```prolog
+agent_main :-
+    system("You are a coding assistant."),
+    task("Write a function to calculate factorial in Python."),
+    answer("Code generated!").
+```
+
+
 Invoke an LLM to perform a task. The description is sent to the model.
 
 ```prolog
@@ -554,6 +918,132 @@ Bind up to three variables.
 > `listing` and `export` reverse this, so you always read/write `task/N` and
 > `prompt/N` — never author `task_named/3` yourself.
 
+### Memory: what the model sees, and how it grows
+
+A `task/N` call is **not** a fresh conversation. It is sent the memory the session
+has accumulated, and the agent's full resulting memory is written back — so each
+`task/N` sees every earlier one, plus its own prompt and reply:
+
+```prolog
+step(Text, Summary, Tags) :-
+    task("Summarise: {Text}", Summary),     % sees: nothing yet
+    task("Tag this summary: {Summary}", Tags).   % sees: the first task's prompt and reply
+```
+
+`system/1` and `user/1` add to that same memory, and **they are backtrackable** —
+a `system/1` inside a branch that is later undone stops contributing:
+
+```prolog
+brief(Text, Out) :-
+    system("Answer in one sentence."),
+    task("Summarise: {Text}", Out).
+```
+
+Two consequences worth planning for:
+
+- **Memory grows with every task.** A loop of `task/N` calls sends an ever longer
+  conversation, which costs tokens and eventually dilutes the instruction. Pass
+  values as explicit arguments rather than relying on the model to remember, and
+  start a fresh step when the history is no longer relevant.
+- **A `task/N` inside a `tool/2` starts with empty memory.** Tool bodies run in
+  their own engine, which is created with no memory of the caller, so a nested
+  agent sees only what the *tool arguments* carry. Put the context in the argument,
+  or add it inside the tool body with `system/1`:
+
+  ```prolog
+  summarise(Text, Summary) :-
+      task("Summarise this text in two sentences: {Text}", Summary).
+
+  tool(summarise(Text, Summary), "Summarise a passage.") :-
+      summarise(Text, Summary).
+  ```
+
+**A fresh context: `prompt/N`, or `with_fresh_memory/2`.** `prompt/N` takes the same
+arguments as `task/N` but sends **no memory and no tools**, and leaves the memory
+untouched. Use it when the step must answer from its own prompt alone — a
+classifier, an extractor, a translation — or when a long history has started to
+interfere:
+
+```prolog
+route(Message, Team) :-
+    prompt("Classify as billing, orders or account: {Message}", Team).
+
+% The conversation so far cannot leak into the classification, and the
+% classification cannot leak into it.
+```
+
+Because `prompt/N` has no tools, a step that must read a file or search an index
+has to be a `task/N`. Choose by what the step needs: the harness's capabilities
+(`task/N`) or an uncontaminated answer (`prompt/N`).
+
+`prompt/N` isolates **one call**. To isolate a **group** of steps — several
+`task/N` calls that share a fresh context with each other but nothing with the
+caller — wrap them in `with_fresh_memory/1`:
+
+```prolog
+describe(Paper, Report) :-
+    with_fresh_memory((
+        task("Summarise the method of: {Paper}", Method),
+        task("Summarise the results of: {Paper}", Results),
+        task("Write a two-sentence abstract from {Method} and {Results}", Report)
+    )).
+% The three steps see each other's turns, because they accumulate within the
+% scope. Nothing the caller accumulated reaches them, and nothing they
+% accumulate reaches the caller — the memory is restored on exit.
+```
+
+With `with_tools/2` / `without_tools/2` it composes into a scope that controls both
+the conversation and the capabilities:
+
+```prolog
+audit(App, Finding) :-
+    with_fresh_memory(without_tools([apply_fix],
+        task("Review {App} against the checklist and report one finding.", Finding))).
+```
+
+### Declared Output Shapes: `schema/2`
+
+A type says "a list of dicts"; a schema says what is *in* them. Declare the shape
+once, name it in the `task/N` output, and every step gets a value that matches it —
+defaults filled, and a missing required field or a wrong-typed field failing the
+step instead of quietly yielding an empty result:
+
+```prolog
+schema(author, [
+    field(name,        string, required),
+    field(affiliation, string, default("not stated")),
+    field(email,       string, default("")),
+    field(country,     string, default(""))
+]).
+
+extract_authors(Text, Authors) :-
+    task("From the front matter extract every author: {Text}",
+         list(schema(author, Authors))).
+% Authors = [author{name:"Ashish Vaswani", affiliation:"Google Brain", email:"", country:""} …]
+```
+
+- An absent field with a `default(Value)` is filled.
+- An absent `required` field **fails the step** with a message naming the field, so
+  the agent can correct it in the same turn.
+- A field of the wrong type fails the same way. Types are `string`, `integer`,
+  `number`, `boolean`, `object`, `array`.
+- An **undeclared** field is kept, not dropped: a model mistake stays visible
+- The field list is **described to the model** in the step's own instructions, and a
+  missing required or wrong-typed field is reported by name inside the same turn,
+  so the agent usually fixes it before the step ends
+  rather than disappearing.
+- `schema/2` is a fact, like `doc/2` — it appears in the listing and travels with
+  `export`.
+
+For a single object rather than a list, use `schema(author, Author)` (no `list/1`).
+You can also apply a schema by hand to any dict:
+
+```prolog
+schema_fill(author, Raw, Author),        % fills defaults, validates
+```
+
+`end_turn` reports a `task/N` that names a schema which was never declared.
+
 ### Type-Safe Output Variables
 
 You can wrap output variables with type specifiers to enforce strict validation on the LLM's output. The runtime will ensure the value matches the specified type.
@@ -612,11 +1102,15 @@ Or positional if the tool supports it:
 exec(get_time, CurrentTime)
 ```
 
-The `bash` tool accepts an optional wall-clock `timeoutMs` and always returns the
-wall-clock duration as `timeMs`:
+The `bash` tool takes exactly two inputs — `command` (a shell string) or `argv` (a
+list, quoted by the runtime) — plus an optional wall-clock `timeoutMs`. There is
+**no `stdin`**: pass data as an `argv` element, or write it to a workspace file and
+name the file. stdin is `/dev/null`, so a command that reads it sees EOF rather
+than waiting. It always returns the wall-clock duration as `timeMs`:
 
 ```prolog
 exec(bash(command: "sleep 30", timeoutMs: 5000), R),
+exec(bash(argv: ["python3", Script, Query]), R),      % no quoting to write
 get_dict(timeMs, R, Took),
 (   get_dict(exitCode, R, 124)
 ->  format(string(Msg), "timed out after ~w ms", [Took])
@@ -649,6 +1143,25 @@ tool(count_words(File, Count), "Count words in a workspace file.") :-
     count_words(File, Count).
 ```
 
+The head and its delegate must have the **same name and the same arity**. The
+tool body is what runs when the nested agent calls the tool, so a body that
+calls a different arity either calls nothing (`existence_error`) or calls the
+tool again and recurses forever:
+
+```prolog
+% WRONG: declares /2 but calls /3, so the tool fails when a nested agent uses it.
+tool(bibliography(Query, Max), "…") :-
+    bibliography(Query, Max, Entries).
+
+% WRONG: calls itself; a nested agent recurses until the budget is gone.
+tool(citations(Text, Citations), "…") :-
+    citations(Text, Citations).
+```
+
+Nothing is checked when the clause is loaded, so both mistakes look fine until a
+nested agent calls the tool. `end_turn` reports them as contract warnings; treat
+them as errors, not style notes.
+
 A predicate defined *only* inside a `tool/2` clause is invisible to
 `execute` / `/goal` — the loader asserts it as `tool(Head)`, never as `Head`. Split
 the two so one implementation is reachable from both surfaces:
@@ -658,6 +1171,12 @@ the two so one implementation is reachable from both surfaces:
 task("Count the words in report.txt.", N).   % nested agent calls the tool
 ```
 
+The only tools a nested agent can call are the harness's `tool/2` predicates;
+registered host tools (for example `bash`) are reachable from DML code through
+`exec/2`, not as tools by name. So a task description that says "use the
+`fs_read` tool" works only if the module itself defines `fs_read/2` **and**
+annotates it.
+
 The tool head's **last argument is the output**; the description is shown to the
 nested agent. Keep the `tool/2` body a single delegation — put no logic in it.
 
@@ -680,26 +1199,103 @@ Tools can use `task()` and `prompt()` internally to combine Prolog logic with LL
 
 ```prolog
 % A predicate that computes then explains.
+% The nested task is told to work from the given figures — it has no tools it
+% needs, so it cannot try to re-enter this predicate.
 explain_calculation(A, B, Explanation) :-
     Sum is A + B,  % Prolog computation
-    format(string(Desc), "Explain ~w + ~w = ~w to a child", [A, B, Sum]),
-    task(Desc, Explanation).  % LLM explanation
+    format(string(Desc),
+           "Explain ~w + ~w = ~w to a child, using only these figures.",
+           [A, B, Sum]),
+    task(Desc, Explanation).
 
 tool(explain_calculation(A, B, Explanation), "Calculate and explain the result.") :-
     explain_calculation(A, B, Explanation).
+```
+
+**Do not wrap an LLM step in a `tool/2` that advertises the same job.** The tool
+is declared to the agent running *inside* that same `task/N`, so the description
+invites the agent to do the very thing it was asked, by calling the tool:
+
+```prolog
+% WARNING: this shape self-delegates.
+extract_authors(Text, Authors) :-
+    task("Extract every author and their affiliation(s) from: {Text}", list(object(Authors))).
+
+tool(extract_authors(Text, Authors), "Extract every author and their affiliation(s).") :-
+    extract_authors(Text, Authors).
+```
 
-% A predicate that uses the LLM for formatting.
-format_data(Data, FormattedOutput) :-
-    format(string(Desc), "Format this data nicely: ~w", [Data]),
-    task(Desc, FormattedOutput).
+Called as a predicate (`/goal extract_authors(…, A)`, `execute`, or from another
+predicate), `extract_authors` runs its `task/N`, and that agent is offered
+`extract_authors` — a tool described as doing exactly its job. The runtime refuses
+the re-entrant call (a tool whose predicate is already running is withheld, and a
+call that slips through is refused with a reason), so it cannot recurse forever.
+But the agent still spends a turn on it, is told the tool failed, and then does
+the work by hand — a confusing transcript and a wasted call.
 
-tool(format_data(Data, FormattedOutput), "Format data nicely using LLM.") :-
-    format_data(Data, FormattedOutput).
+Two ways to avoid it, in order of preference:
+
+**Keep the LLM step a skill, and give `tool/2` only to capabilities.** A skill is
+called from DML (`/goal`, `execute`, another predicate) and does not need to be a
+tool. Annotate the operations a nested agent should *use* — deterministic or
+side-effecting work such as reading a PDF, searching an index, or shelling out:
+
+```prolog
+% Skill: an LLM step. Call it from DML; no tool/2.
+extract_authors(Text, Authors) :-
+    task("Extract every author and their affiliation(s) from: {Text}", list(object(Authors))).
+
+% Tools: capabilities for nested agents. No task/N, so no self-reference.
+read_pdf(Path, Text) :-
+    script_file(pdf_to_text, Script),
+    format(atom(Cmd), "python3 '~w' '~w'", [Script, Path]),
+    exec(bash(command: Cmd), R), get_dict(stdout, R, Text).
+
+tool(read_pdf(Path, Text), "Convert a workspace PDF to text.") :- read_pdf(Path, Text).
 ```
 
+**Or scope the nested task's tools so the tool cannot be reached.** If a `task/N`
+inside a predicate must not use a tool, say so with `without_tools/2` (or list
+exactly what it may use with `with_tools/2`):
+
+```prolog
+extract_authors(Text, Authors) :-
+    without_tools([extract_authors],
+        task("Extract every author and their affiliation(s) from: {Text}",
+             list(object(Authors)))).
+```
+
+Scoping is the narrower fix: it keeps the tool advertised to *other* agents (which
+is usually why it exists) while removing it from this one.
+
 **Memory Isolation:** When `task()` or `prompt()` runs inside a tool, the nested agent starts with **fresh memory**. It does NOT inherit the parent's accumulated memory from `system()`, `user()`, or previous `task()` calls. If you need context in a nested task, pass it explicitly as a tool argument or add it with `system()` inside the tool body.
 
-**Tool Access:** The nested agent has access to all DML tools **except** the tool that is currently executing (automatic call stack exclusion to prevent infinite recursion). You can further control tool access using `with_tools/2` and `without_tools/2`.
+**Tool Access:** A nested agent's only callable tools are the harness's `tool/2`
+predicates; host tools such as `bash` are reachable from DML code via `exec/2`,
+never by name from a task description. A tool that is **already executing** cannot
+be re-entered: a call to it from inside itself is refused with an explanation, so a
+tool cannot recurse into itself. The refusal is deliberate and visible — the tool is
+declared to the agent with the reason on its description, so the call comes back as
+"already running further up the call chain" rather than as a missing tool, which
+would invite a retry.
+
+> The withholding is keyed on *tool* execution, and only that is guaranteed. Whether a
+> running tool is also *removed* from the tool list presented to the agent inside it
+> is not something the current test suite covers; the refusal above is what stops the
+> recursion. Do not rely on the tool being absent from `task/N`'s tool list.
+
+That keying has a consequence worth knowing: a predicate that is also a tool does
+**not** count as "executing" when it is called as a predicate (`/goal`, `execute`,
+or from another predicate). So for
+
+```prolog
+extract_authors(Text, Authors) :- task("…", list(object(Authors))).
+tool(extract_authors(Text, Authors), "…") :- extract_authors(Text, Authors).
+```
+
+`/goal extract_authors(Text, A)` still offers `extract_authors` to the agent inside
+`task/N`. Use `with_tools/2` / `without_tools/2` to scope it, or keep the LLM step
+out of the tool set entirely (see **Using `task()` Inside Tools** above).
 
 ```prolog
 % Example: passing context explicitly.
@@ -722,15 +1318,19 @@ You can control which tools are available to nested `task()` calls using `with_t
 
 #### `with_tools(+ToolList, +Goal)`
 
-Run Goal with only the specified tools available to nested tasks:
+Run Goal with only the specified tools available to nested tasks. `ToolList`
+holds tool *names* (atoms), not predicates:
 
 ```prolog
-% Only allow tool_a and tool_b in the nested task
-tool(restricted_task(Input, Output), "Runs with limited tools") :-
-    with_tools([tool_a, tool_b], (
-        format(string(Desc), "Process '~w' using available tools", [Input]),
+% The implementation defines what the tool does; the annotation exposes it.
+restricted_work(Input, Output) :-
+    with_tools([search_papers, read_paper], (
+        format(string(Desc), "Process '~w' using the available tools", [Input]),
         task(Desc, Output)
     )).
+
+tool(restricted_task(Input, Output), "Runs with limited tools") :-
+    restricted_work(Input, Output).
 ```
 
 #### `without_tools(+ToolList, +Goal)`
@@ -738,12 +1338,14 @@ tool(restricted_task(Input, Output), "Runs with limited tools") :-
 Run Goal with the specified tools excluded from nested tasks:
 
 ```prolog
-% Exclude expensive_tool from the nested task
-tool(cheap_task(Input, Output), "Runs without expensive tools") :-
+cheap_work(Input, Output) :-
     without_tools([expensive_tool], (
         format(string(Desc), "Process '~w' cheaply", [Input]),
         task(Desc, Output)
     )).
+
+tool(cheap_task(Input, Output), "Runs without expensive tools") :-
+    cheap_work(Input, Output).
 ```
 
 #### Using Tool Scoping in `agent_main`
@@ -767,6 +1369,37 @@ agent_main :-
 
 ---
 
+### Model Selection
+
+#### `with_model(+Spec, +Goal)`
+
+Run `Goal` with a different model — and optionally a reasoning level — for every
+`task/N`, `prompt/N` and `sample_token/N` call inside it. Judgments
+(`choose/rate/verify/probability`) are not affected.
+
+```prolog
+% a cheap model for classification
+with_model("openai:gpt-4o-mini",
+    prompt("Classify as billing/orders/account: {Message}", Team)).
+
+% a strong model with high reasoning, only for the hard step
+with_model("anthropic:claude-sonnet-4#high",
+    task("Review this diff and list defects:\n{Diff}", Notes)).
+
+% full control with a dict
+with_model(model{provider: "google", id: "gemini-2.5-pro",
+                 temperature: 0.2, maxTokens: 8192}, Goal).
+```
+
+`Spec` is a string — `"provider:model"`, `"provider/model"`, `"model"`, each with
+an optional `#reasoning` suffix (`off|minimal|low|medium|high|xhigh|max`) — or a
+`model{provider, id, temperature, maxTokens, reasoning}` dict. The innermost
+`with_model` wins, and a nested `with_reasoning/2` overrides the spec's reasoning.
+The default is the session's active model. An unknown or unsigned-in model fails
+with a clear error.
+
+---
+
 ### Memory Predicates
 
 Memory predicates build the conversation context sent to the LLM.
@@ -825,24 +1458,77 @@ Memory stores `media{path, mime}` — never the image bytes. Judgments
 
 ---
 
+#### Running a command with data in it
+
+`argv` takes the command and its arguments as a **list**, and the runtime quotes
+each element. Pass values here and you never write a quote, so an argument
+containing a space, a quote, `$`, a backtick or `;` is inert:
+
+```prolog
+exec(bash(argv: ["python3", Script, UserText]), R), get_dict(stdout, R, Text).
+```
+
+It is the same shell underneath (and the same sandbox), so pipelines and redirection
+still work with `command`; `argv` is the way to pass *data*. Prefer it whenever an
+argument comes from a file, a model, or the user. There is no `stdin` argument.
+
+#### Do not parse in DML — shell out or ask the model
+
+String handling is the fiddliest part of Prolog, and hand-rolled parsing is the
+easiest way to write DML that looks fine and is quietly wrong. Do not build a
+parser out of `sub_string/5`, `split_string/4`, `atom_concat/3`, `format/2` and an
+accumulator. Two better answers, depending on what the work actually is:
+
+- **Mechanical** (pull a field, change a format, count, filter, join): write a
+  **helper script** and run it. A script is a file the harness links with
+  `script_file/2`, it is testable on its own, and Python/jq/sed are built for this.
+  Have it emit **JSON** and read the result with `json_get/3` / `json_gets/3` /
+  `json_text/2`, rather than splitting its stdout by hand.
+- **Meaning** (find the field named X, classify, summarise, decide): make it a
+  **model call** with a typed output — `task("…", list(string(Items)))`,
+  `schema/2`, or `choose`/`rate`/`verify` for a bounded judgment. The model is the
+  right tool for understanding text; Prolog is the right tool for *sequencing and
+  checking* it.
+
+So `exec(bash(command: "cat", stdin: "…"), R), get_dict(stdout, R, O)` has no
+place here on either count: there is no stdin, and the structured reading of the
+result is a job for a script or a model, not for the DML around the call.
+
 ### File I/O and paths — prefer the shell
 
-**Prefer `exec(bash(...))` for file operations.**
-Reading, writing, copying, or converting files is usually clearer and more
-robust as a shell command; reach for the DML predicates below only when the
-content must be bound and reasoned over inside Prolog. Pulling a file into DML
-just to hand it back to the shell is a smell (`DML_AUTHORING.md` §2).
+**`/workspace` is the project, and it is the working directory.** A DML path is
+either absolute under `/workspace` or relative to it, and the two forms name the
+same file:
+
+    out/report.txt          and  /workspace/out/report.txt
+
+Prefer the `/workspace/...` form: the same name works from DML, from
+`exec(bash(...))`, and from the shell tool, on every backend, so a path never has
+to be translated between them. There is **no paths file and no search-path
+registry** — nothing to declare, register, or discover. `script_file(Name, Path)`
+links a file that belongs to the harness (it is bundled on `export`); it does not
+resolve anything at run time.
 
-If you do use them: every path in DML is **workspace-relative** and confined to
-the workspace. The
-engine mounts the workspace at `/workspace`, but you must not write
-`/workspace/...` in DML — passing an absolute path is rejected, as is `..`. File
-predicates all take a relative path:
+**Everything outside `/workspace` is the ephemeral VM filesystem.** `/etc`,
+`/tmp`, `/home` are not this machine. They are an in-memory filesystem that
+belongs to the session and is discarded when it ends, so they are fine for scratch
+and useless for results. **Only files under `/workspace` reach the host.** For
+scratch, use `$TMPDIR`, which is a per-session directory in every backend (a bare
+`/tmp` is the real one when nothing is sandboxed).
+
+**Prefer `exec(bash(...))` for file operations.** Reading, writing, copying, or
+converting files is usually clearer and more robust as a shell command; reach for
+the DML predicates below only when the content must be bound and reasoned over
+inside Prolog. Pulling a file into DML just to hand it back to the shell is a
+smell (`DML_AUTHORING.md` §2).
+
+The file predicates take a path — absolute under `/workspace`, or relative:
 
 ```prolog
-read_file("images/a.png", Bytes),
-write_file("out/report.txt", Text),
-make_directory_path("out/nested"),          % create a directory (and parents)
+read_file("/workspace/images/a.png", Bytes),
+read_file("images/a.png", Bytes),            % same file
+write_file("/workspace/out/report.txt", Text),
+make_directory_path("/workspace/out/nested"),   % create a directory (and parents)
 append_file("out/log.txt", Line),
 delete_file("out/report.txt"),
 directory_files("images", Names),        % bare entry names: ["a.png", ...]
@@ -854,10 +1540,24 @@ time_file("images/a.png", Modified).
 `write_file/2` does **not** create parent directories — call
 `make_directory_path/1` first. `directory_files/2` returns bare entry names (no
 directory), so build a path with `atom_concat('images/', Name, Path)` when you
-need one. `bash` runs with a wider view of the sandbox than DML has; convert
-anything from the shell to a workspace-relative path before handing it to DML.
+need one.
+
+These predicates are **real clauses in the session module**, not only
+interpreter handlers, so DML code, a nested `task/N`, and a direct host query
+(the SDK's `runDML`, or `deepclause repl`) all resolve the same definition and
+the same path.
+
+`open/3`, `close/1`, `read_string/3` and `write/2` are SWI builtins and cannot be
+shadowed, but they resolve paths the same way — the working directory is
+`/workspace`, so `open("f.txt", read, S)` reads the workspace file. They are not
+a special case with their own path rule.
+
+Paths are **isolated by the mount, not by string checks**: there is no rewriting
+layer, so a path is passed through to the filesystem as written. `..` does not
+climb out of the workspace, because above `/workspace` is the VM's own empty
+filesystem. The one thing a mount cannot police is a **symlink inside the
+workspace** pointing outside it; avoid committing those.
 
----
 
 ### Output Predicates
 
@@ -873,6 +1573,25 @@ agent_main :-
     answer("Done").
 ```
 
+**Use it for debugging.** When a predicate does not do what you expect, add
+`output/1` calls between the steps: you see how far execution gets, what the
+intermediate values actually are, and which call returned nothing. It costs one
+line, it is invisible to the model (unlike a `task/N` description), and it is the
+fastest way to find where a chain goes wrong:
+
+```prolog
+summarise(Doc, Summary) :-
+    extract_sections(Doc, Sections),
+    format(string(S), "sections: ~w", [Sections]), output(S),
+    pick_main(Sections, Main),
+    format(string(M), "main: ~w", [Main]), output(M),
+    task("Summarise this in one sentence: ~w", [Main], Summary).
+```
+
+Remove them before the final version, or keep the ones that read as progress
+reports — a predicate that says what it is doing is easier to debug the next
+time too.
+
 #### `yield(+Text)`
 
 Alias for `output/1`.
@@ -960,6 +1679,40 @@ agent_main(Topic) :-
 
 ---
 
+## Patterns: one template, build or parse
+
+`pattern/3` is `format/3` and its inverse. Literal text matches exactly and `$name`
+marks a slot, so the same template renders a string or parses it — the direction
+follows from what is bound:
+
+```prolog
+pattern("Name: $name, age: $age", Text, [name="Ada", age=36]).
+% Text = "Name: Ada, age: 36"
+
+pattern("Name: $name, age: $age", "Name: Ada, age: 36", Bindings).
+% Bindings = [name="Ada", age="36"]
+
+pattern("DOI: $doi", Text, [doi=D]).                % build a DOI line
+pattern("n=$n:int", "n=42", B).                     % B = [n=42]   (typed slot)
+```
+
+- Bindings are a list of `Name=Value`, so `member(Name=Value, Bs)` reads them.
+- A slot consumes **up to the next literal**, so the literals decide the split and
+  `"$left - $right"` against `"a - b - c"` gives `left="a"`, `right="b - c"`.
+- `$name:type` converts on parse and validates on build. Types: `string` (default),
+  `int`, `integer`, `number`. A parse that does not match throws, rather than
+  binding the text.
+- Slot names match case- and separator-insensitively, like `dict_field/3`.
+- `$` is the marker rather than `{…}`, so a template can carry JSON or any other
+  brace-bearing text verbatim, and it never collides with the `{Var}`
+  interpolation in `task/N` descriptions. `$$` is a literal dollar.
+- A template must be **anchored by literal text**: `"$a$b"` is rejected as
+  ambiguous rather than resolved arbitrarily.
+
+This is the deterministic alternative to asking the model to reformat text: parse
+the parts you can read exactly, and use a `task/N` only for what genuinely needs
+language understanding.
+
 ## Backtracking
 
 DML supports full Prolog backtracking, **including across LLM calls**. This enables powerful retry logic:
@@ -1019,20 +1772,112 @@ all: arithmetic, counting, ordering, membership, `findall/3`, `aggregate_all/3`,
 list/set operations, and constraint or optimisation problems. Reach for `task/N`
 only when the step genuinely needs language understanding.
 
-### Lists
+### Lists and common operations
+
+A list is `[1,2,3]` or `[Head|Tail]`; the recursion below is the shape everything
+else is built on. Prefer the library predicates to hand-rolled recursion.
+
 ```prolog
 process_items([]).
 process_items([H|T]) :-
     format(string(Msg), "Processing: ~w", [H]),
     output(Msg),
     process_items(T).
+```
 
-agent_main :-
-    process_items([apple, banana, cherry]),
-    answer("All items processed").
+`maplist/3`, `include/3`, `exclude/3`, `foldl/4` and `predsort/3` take a **goal with
+extra arguments**, written `[X,Y]>>Goal` (a *yall* lambda). The list before `>>` is
+the extra arguments — so `[X,Y]>>(Y is X*X)` is a two-argument goal. The lambda's
+arity must match the predicate you are using.
+
+**Transform**
+
+| Want | Write |
+|---|---|
+| Apply to each | `maplist([X,Y]>>(Y is X*X), [1,2,3], L)` → `[1,4,9]` |
+| Combine two lists | `maplist([A,B,C]>>(C is A+B), [1,2], [10,20], L)` → `[11,22]` |
+| Apply, skipping failures | `convlist([X,Y]>>(integer(X), Y is X*2), [1,a,3], L)` → `[2,6]` |
+| Keep / drop matching | `include(P, L, Keep)` / `exclude(P, L, Drop)` |
+| Reduce to one value | `foldl([X,A0,A]>>(A is A0+X), [1,2,3], 0, S)` → `6` |
+| Reverse | `reverse([1,2,3], L)` |
+| Take / split | `append(Chunk, Rest, List)`, or `length(Chunk, 2)` to bound it |
+| A list of N blanks | `length(L, 3)` |
+
+**Select and aggregate**
+
+| Want | Write |
+|---|---|
+| Element at position | `nth0(1, [a,b,c], E)` (0-based) / `nth1(1, …, E)` (1-based) |
+| Remove one occurrence | `select(b, [a,b,c], L)` → `[a,c]` |
+| Check membership (once) | `memberchk(b, L)` — semidet, no choice point |
+| Last element | `last([a,b,c], E)` |
+| Sum | `sum_list([1,2,3], S)` → `6` |
+| Largest / smallest | `max_member(M, L)` / `min_member(M, L)` |
+| Any aggregate | `aggregate_all(max(N), member(N,[1,9,3]), M)` → `9`; also `min`, `sum`, `count`, `bag` |
+
+**Sort, group, dedupe**
+
+| Want | Write |
+|---|---|
+| Sort and dedupe | `sort([3,1,2,1], L)` → `[1,2,3]` |
+| Sort keeping duplicates | `msort([3,1,2,1], L)` → `[1,1,2,3]` (stable) |
+| Sort pairs by key | `keysort([b-2,a-1], L)` → `[a-1,b-2]` (stable) |
+| Sort compounds by an argument | `sort(2, @>=, [a-1,b-3,c-2], L)` — the `2` is the argument position |
+| Sort with your own comparison | `predsort([O,A,B]>>compare(O,A,B), L, Sorted)` |
+| Dedupe, keeping first order | `list_to_set([a,b,a,c], L)` → `[a,b,c]` |
+| Group by key | `group_pairs_by_key([a-1,a-2,b-3], G)` → `[a-[1,2], b-[3]]` |
+| Set difference / intersect / union | `subtract/3`, `intersection/3`, `union/3` |
+| Zip / unzip | `pairs_keys_values(Pairs, Keys, Values)`, `pairs_keys/2`, `pairs_values/2` |
+
+**Strings in a list.** `split_string/4` leaves empty strings behind, which is the
+usual reason a list "has blanks in it":
+
+```prolog
+split_string("a,,b", ",", "", Parts),      % Parts = ["a","","b"]
+exclude(==(""), Parts, Clean).              % Clean = ["a","b"]
+include(dif(""), Parts, Clean).             % the same, by unification
+exclude(=([]), Lists, NonEmpty).            % drop empty lists
 ```
 
-Not all builtin SWI Prolog predicates are guaranteed to work at this stage!
+**A field from a list of dicts** — the most common list task with model output:
+
+```prolog
+json_gets(Doc, "rows.*.name", Names),                       % one call, collects
+findall(N, (member(D, Dicts), dict_field(D, name, N)), Names).   % from a list
+```
+
+To sort a list of dicts, use `predsort/3`: `sort/4` addresses arguments of a
+compound and a dict has none.
+
+**Batch a list into chunks.** There is no builtin; this is the definition:
+
+```prolog
+chunks([], _, []).
+chunks(List, N, [Chunk|Rest]) :-
+    N > 0,
+    length(Chunk, N),
+    append(Chunk, Tail, List),
+    chunks(Tail, N, Rest).
+chunks(List, N, [List]) :-
+    N > 0,
+    length(List, Len),
+    Len < N.
+% chunks([1,2,3,4,5], 2, L) -> L = [[1,2],[3,4],[5]]
+```
+
+### Traps
+
+- **`setof/3` and `bagof/3` fail when there are no solutions**; `findall/3` succeeds
+  with `[]`. Use `findall/3` unless you want that failure.
+- **`sort/2` silently removes duplicates.** Use `msort/2` to keep them.
+- **`max_list([])` fails**, and `sum_list([a,b])` raises a type error — guard a
+  possibly-empty or possibly-non-numeric list before aggregating.
+- **A nondeterministic goal inside `maplist`, `include` or `foldl` multiplies
+  solutions**, and re-runs any model call in it. Keep the goal semidet
+  (`once/1`, `->`, a cut).
+- **Not available in this build:** `nth0/4` (replace an element), `numlist/4` (range
+  with a step), `chunk/3`, `rename_keys/3`. Replace an element with
+  `nth0(I, L, _, Tail), nth0(I, New, V, Tail)`, and use `chunks/3` above.
 
 ### Arithmetic
 ```prolog
@@ -1202,6 +2047,71 @@ exact_max(X, Y) :-
 ```
 
 ### Dictionaries (SWI-Prolog)
+
+**Keys are atoms.** `get_dict/3` is strict about the key type and *throws* rather
+than failing: `get_dict("affiliation", D)` raises `type_error(dict-key,
+affiliation)`, so no guard sees it as a "no". A string is not even valid dict
+literal syntax (`x{"a":1}` is a syntax error).
+
+Model output makes this worse, because the field name varies — `Name`, `name`,
+`full_name`. Use `dict_field/3,4`, which matches an atom **or** a string key,
+ignoring case and `_`/`-`/space:
+
+```prolog
+dict_field(Dict, "affiliation", Affiliation),          % fails if absent
+dict_field(Dict, full_name, "unknown", Name),          % with a default
+```
+
+**JSON.** `atom_json_dict/3` works in both directions, which is the whole story —
+reading and writing:
+
+```prolog
+atom_json_dict(Text, Dict, []),          % JSON text -> dict  (Text bound)
+atom_json_dict(Text, Dict, []),          % dict -> JSON text  (Text unbound)
+```
+
+Reading a nested value is one call with a path rather than a chain of lookups:
+
+```prolog
+json_get(Doc, "results.0.authors.1.name", Name),        % dotted path
+json_get(Doc, [results, 0, authors, 1, name], Name),    % or a list
+json_get(Doc, "results.0.year", "unknown", Year),       % with a default
+json_gets(Doc, "results.*.title", Titles),              % project: collect `*`
+json_gets(Doc, "results.*.authors.*.name", Names),      % nested `*` flattens
+json_put(Doc, "meta.reviewed", yes, Updated),           % set a key path
+json_text(Doc, Text),                                   % compact JSON
+```
+
+A missing path **fails**; it does not throw and does not give `""`, so absence is
+visible. Key matching is the same tolerant match as `dict_field/3` (atom or string,
+any case, `_`/`-`/space ignored). `json_put/4` sets keys and creates intermediate
+dicts; it rejects an index path, so lists are read-only. For layout or streaming,
+`json_write_dict/3` from `library(http/json)` is available.
+
+**Parsing text with a grammar.** DCGs work — the loader runs `expand_term/2`, so
+`-->` compiles, and `library(dcg/basics)` imports:
+
+```prolog
+:- use_module(library(dcg/basics)).
+
+key_value(K, V) --> string_without(":", K), ":", string(V).
+
+parse_kv(Text, Pairs) :-
+    string_chars(Text, Chars),          % phrase/2 wants a LIST, not a string
+    phrase(lines(Pairs), Chars).
+
+lines([K-V|Rest]) --> key_value(K, V), "\n", !, lines(Rest).
+lines([K-V]) --> key_value(K, V).
+```
+
+`phrase/2,3` takes a **code or char list**, so `string_chars/2` (or
+`string_codes/2`) comes first — passing a string raises `type_error(list, …)`.
+Also useful here: `number_codes/2`, `read_term_from_atom/3`,
+`split_string/4`. DCGs give you a documented, testable parser instead of a chain of
+regex calls, and the same grammar can generate text as well as accept it.
+
+The rest of this section is plain SWI dict access, for keys you control:
+
 ```prolog
 agent_main :-
     D = point{x: 10, y: 20},
@@ -1342,6 +2252,52 @@ backtracking reuses an answer instead of re-querying the model.
   always one of your option atoms.
 - Prefer one `judge/2` batch over several one-off calls when you need more than
   one or two judgments.
+### Load-time directives
+
+A `.dml` file may contain directives. These are handled:
+
+```prolog
+:- use_module(library(clpfd)).     % library import
+:- consult("other.dml").           % load another workspace file
+:- param(topic, "the subject").    % declared session parameter
+:- repl_delete(old_skill/2).       % remove a predicate (same as the bare fact form)
+:- require_sandbox.                % refuse to load unless the shell is sandboxed
+```
+
+`:- require_sandbox.` is a declaration that the harness must run with an
+isolated shell. It is checked **at load**, and the load **fails** when the
+effective backend is not a sandbox (`bwrap`, AgentVM, or `sandbox-exec`) — it
+does not silently run unsandboxed. "Effective" is deliberate: bubblewrap being
+installed while the session resolves to unsandboxed execution still fails the
+check. Only the **user** sets this (the `/set-require-sandbox` command); an agent
+cannot add or remove it.
+
+`:- repl_delete(Name/Arity).` abolishes a predicate, which `retract/1` cannot do:
+retracting the last clause leaves an empty predicate whose calls fail silently.
+The bare fact form `repl_delete(Name/Arity).` works the same way.
+
+These apply to a file **loaded** into a session (`deepclause repl -l file.dml`).
+They are not the way to edit a live harness. `consult/1`, a file-list directive and
+`use_module/1` on a workspace `.dml` assert straight into the module rather than
+through the REPL's edit tools, so the predicates they add are absent from the
+transaction's tracked set — the nondeterminism probe does not see them, and a
+directive that fails inside the file only warns. They do still appear in the next
+commit's diff, so the change is recorded without any account of where it came from —
+and outside a transaction there is no snapshot, so nothing validates the load at all.
+An agent changing a harness adds code with `assert`/`assert_many`/`define` inside a
+transaction instead.
+
+Any other directive does nothing, and the loader now says so — it reports
+`unrecognised directive ignored: …` after the file loads, so a directive that
+never took effect is not mistaken for one that did. `dynamic/1`,
+`discontiguous/1` and `multifile/1` are accepted and ignored silently, since
+session predicates are asserted and already behave that way.
+
+A **predicate with no clauses fails silently** when called — it does not error —
+so an emptied or never-written predicate looks like "no results" at the call site.
+`current_predicate(Name/Arity)` in a `/goal` answers whether it exists, and a
+`doc/2` for a predicate with no clauses is reported at `end_turn`.
+
 ## The DML REPL (`deepclause repl`)
 
 DML skills can also be grown interactively in a persistent **living module**.