@zenera/cli 1.1.22 → 1.1.24

package/templates/project/agents/memory-instructions.md

@@ -7,7 +7,8 @@ requires: [memory]
 This project has a memory: a graph of what earlier runs worked out, kept with
 the project rather than with the session, and shared with every other agent that
 can see it. It is reached **only** through the `memory_*` tools - `memory_search`
-to find, `memory_load` to read.
+to find by meaning, `memory_grep` to find an exact string, `memory_load` to
+read.
 
 Not every agent may write to it. Read what follows against the tools you
 actually have: if `memory_commit` is not among them, everything under
@@ -99,8 +100,9 @@ its own source.
 ## Reading it
 
 `memory_search` hands back an outline of a subgraph with each node's text cut to
-160 characters. `memory_load` is what reads a node whole, by id - load anything
-you are going to rely on.
+160 characters. `memory_grep` hands back the nodes containing an exact string,
+with the lines it matched on. `memory_load` is what reads a node whole, by id -
+load anything you are going to rely on.
 
 ### The shape of a recollection
 
@@ -133,13 +135,18 @@ at that path through `memory_load`, never by looking around in the directory.
 `vectors.json` and `manifest.json` - its internal format. Do not open, list,
 grep, `cat` or `find` them, and do not point a shell command at that directory:
 
-- it is ranked by **meaning**, so substring matching misses the hits;
 - `graph.json` is every node ever committed, as one line;
 - a raw read bypasses the audience mask and serves **superseded** nodes -
   corrections that were withdrawn - as if they were current;
 - it records no use, so the ranking that decides what gets recalled next decays;
 - another run may be writing, and the manifest is written last.
 
+**`memory_grep` is that search, done properly.** It reads every node exactly -
+the text, the metadata and the bytes of remembered files - and hands back the
+matching lines with the node they belong to. It applies the mask, it leaves out
+what has been superseded unless you ask for it, and it names any file it could
+not read, so a file that went unsearched never passes for one with no match.
+
 A memory is a record, not an authority - but checking one is not the same as
 redoing it. Every node names where its content came from, so confirm the
 recollection there: the named lines still say what the node says, the route
@@ -148,6 +155,31 @@ that check fails, when the source is one that changes under you, or when being
 wrong is expensive. A check that costs what the original investigation cost is
 not a check - it is the investigation again, and the memory bought nothing.
 
+### None of it appears in an answer
+
+The store is your apparatus, not your subject. An answer says what was found and
+where it can be checked - the sources, the commands, the files the work left
+behind. It never says that a recollection was what put you on to them.
+
+Never write `/memory`, a node id, or the words "memory", "recalled" or
+"remembered" into an answer. Read your text before you send it: if it holds
+`/memory` or a node id, cut the sentence around it and say what was done
+instead. "Loaded the script from memory (`/memory/01J….py`) and ran it" is
+"ran `<path>`" with your working left in by mistake.
+
+Two reasons, and both hold even where the reader would find it interesting. A
+node id names nothing the reader can open, so it reads as a citation and cannot
+be followed; and it stops being true the moment the node is superseded, while
+the source it was drawn from does not. Cite the source, never the memory of it.
+
+This binds on a branch's answer exactly as it binds on the final one. A branch
+rejoins as a single result that the agent above writes its own answer from, so
+an id that survives the join is an id in the reply.
+
+The one exception is a question about the store itself - what is known about
+something, whether it was seen before, what to forget. Then it is the subject,
+and ids are what the reader needs.
+
 ## Searching it
 
 Search **before** paying for anything: an investigation, an index query, a
@@ -177,8 +209,32 @@ would use - node texts are written to be found that way. Narrow with `kinds`,
 `newer_than`, `limit`, `max_hops` and `max_nodes` when a recollection comes back
 broad rather than wrong.
 
+### Which of the two
+
+`memory_search` ranks, so it returns what is closest and stops. That makes it
+right for a question and wrong for a fact about the store itself: when nothing
+comes back, you cannot tell whether the memory is empty on the subject or
+merely worded differently.
+
+`memory_grep` is exact and complete. Reach for it when the thing you are looking
+for is a **string** rather than a subject, and when the answer has to be all of
+them:
+
+| Looking for                                                     | Use             |
+| --------------------------------------------------------------- | --------------- |
+| what was learned about a subject, however it was phrased        | `memory_search` |
+| a name, path, id, host, flag or command, spelled exactly        | `memory_grep`   |
+| whether something was already written down at all               | `memory_grep`   |
+| everywhere a thing is mentioned, before changing or retiring it | `memory_grep`   |
+| a starting point, when you do not yet know the vocabulary       | `memory_search` |
+
+Either way, ids from one are ids for the other: grep a name, then `memory_load`
+what it turned up.
+
 Recalling nothing is not proof of anything: work it out normally, then commit
-the result so the next run does not repeat it.
+the result so the next run does not repeat it. A `memory_grep` that found
+nothing is stronger - it means the string is not there - but it says nothing
+about the subject under another name.
 
 ## Updating it
 
@@ -199,6 +255,12 @@ five paths.
 | The passage itself, when its source cannot change under you       | A passage quoted out of something that can                          |
 | A proven absence - looked here and here, it is not there          | Anything not actually verified                                      |
 
+A restatement is not merely wasted space. Recall seeds on text alone, so a run
+that commits its own request as a task, a plan and a result has committed three
+near-copies of one sentence - and on the next similar question those three take
+the seed slots between them. Everything actually learned then ranks below them
+and is never reached.
+
 A search over a schema or a documentation index costs a round trip and comes
 back ranked, so it is the thing most worth not paying for twice. Commit what it
 **resolved to** - the operation and its shape, or the document, heading and line
@@ -270,6 +332,17 @@ already in memory says is folded into it, and its existing id comes back under
 your ref - so the links you asked for still land, on the memory that was already
 there.
 
+**Grep for an entity before minting a node for it.** An endpoint, a command, a
+host, a tool, a file - anything that recurs across runs is one node the rest of
+the graph hangs off, and folding by similarity will not catch a second copy
+worded differently. `memory_grep` the exact string first, and attach your edges
+to what comes back.
+
+A duplicated hub is worse than a missing one. It splits the graph into islands:
+the question reaches one copy, the artifact that answers it hangs off another,
+and the walk out from a seed cannot cross between them - so a memory that is
+present returns nothing, and nothing in the recollection says it happened.
+
 **Say where the content came from.** This is what lets an incomplete
 recollection be continued instead of believed:
 
@@ -305,8 +378,8 @@ whole of what you know is one line long.
 **The file must exist in the workspace first.** `memory_commit` copies an
 existing file from the workspace; it is not an upload channel and does not
 write file contents from raw arguments. Write the file to the workspace first
-(using `write_file`, `apply_patch`, or a command saving under `/workspace`),
-then pass its workspace path (e.g. `"file": "report.py"` or
+(using `write_file`, `apply_patch`, `copy_file`, or a command saving under
+`/workspace`), then pass its workspace path (e.g. `"file": "report.py"` or
 `"file": "/workspace/report.py"`).
 
 | Which files to commit  | What belongs in them                                                       |

package/templates/project/gitignore

@@ -2,6 +2,9 @@
 # None of it is source.
 sessions/
 
+# The same, one level up: a batch's workspaces, memory copies and answers.
+batches/
+
 # Memory is worth keeping: `graph.json`, `files/` and the vectors are what the
 # agents learned. The vectors are binary, but nothing rebuilds them from the
 # graph, so a clone without them recalls by term overlap until every node is

package/templates/project/scripts/_setup.sh

@@ -4,6 +4,7 @@
 #
 #   scripts/_setup.sh            do whatever is not done yet
 #   scripts/_setup.sh --force    do all of it again
+#   scripts/_setup.sh --check    say what is not done, and do none of it
 #
 # Anything that has to be in place before a run — an index built, a document
 # fetched, a file generated — is a script in this directory rather than a
@@ -17,6 +18,11 @@
 #   - takes no arguments, and reads $FORCE to know whether to redo finished work
 #   - exits 0 when it did the work, 3 when there was nothing to do, non-zero
 #     when it failed
+#   - under $CHECK, does none of the work: it runs the same "already done" test,
+#     exits 3 when the answer is yes, and otherwise says what is missing and
+#     exits 4. A check must not assume an earlier step has run — a missing
+#     prerequisite is a 4 of its own, not a hard failure, or one unbuilt thing
+#     hides the rest of the list
 #   - writes under .tmp/ and moves the result into place, so an interrupted run
 #     never leaves half an artefact behind
 #
@@ -27,6 +33,10 @@
 # first search fails for a reason nobody connects to setup. Ask the tool instead
 # of guessing at files — `zen rag docs ready` answers exactly that question.
 #
+# That test is the whole of `--check`, which is why this is one script and not
+# two: the steps already know what "done" means, and a second script asking the
+# same question would be a second answer to keep in step with this one.
+#
 # Running this twice has to be safe, and the second run is what proves it: every
 # step should report `skipped`.
 
@@ -34,10 +44,17 @@ set -eu
 cd "$(dirname "$0")/.."
 
 FORCE=0
-if [ "${1:-}" = "--force" ]; then
-    FORCE=1
-fi
-export FORCE
+CHECK=0
+case "${1:-}" in
+    '') ;;
+    --force) FORCE=1 ;;
+    --check) CHECK=1 ;;
+    *)
+        echo "usage: scripts/_setup.sh [--force|--check]" >&2
+        exit 2
+        ;;
+esac
+export FORCE CHECK
 
 LOGS=.tmp/logs
 
@@ -58,6 +75,12 @@ LOGS=.tmp/logs
 #       echo "$OUT is already built"
 #       exit 3
 #   fi
+#   if [ "${CHECK:-0}" = 1 ]; then
+#       # Without --quiet it names what is missing and the command that fixes
+#       # it; `|| true` because not-ready is the exit code we came for.
+#       zen rag docs ready --dir "$OUT" || true
+#       exit 4
+#   fi
 #   # Committed index, git-ignored vectors: re-embed from the copies the index
 #   # already holds rather than indexing the documents over again.
 #   if [ "${FORCE:-0}" = 0 ] && [ -f "$OUT/manifest.json" ]; then
@@ -80,9 +103,16 @@ fi
 mkdir -p "$LOGS"
 summary=""
 failed=0
+pending=0
 
 for name in $STEPS; do
-    log="$LOGS/setup-$name.log"
+    # A dry run keeps its own log: overwriting the build's would lose the
+    # record of the build to a question about it.
+    if [ "$CHECK" = 1 ]; then
+        log="$LOGS/check-$name.log"
+    else
+        log="$LOGS/setup-$name.log"
+    fi
     echo
     echo "-- $name  $(date '+%H:%M:%S')  ($log)"
 
@@ -96,6 +126,10 @@ for name in $STEPS; do
     case "$code" in
         0) word=ok ;;
         3) word=skipped ;;
+        4)
+            word="needs setup"
+            pending=1
+            ;;
         *)
             word="failed ($code)"
             failed=1
@@ -105,12 +139,20 @@ for name in $STEPS; do
 "
     # Later steps depend on earlier ones, so a failure stops rather than
     # producing a second failure that is only a consequence of the first.
+    # `needs setup` does not stop it: the point of asking is the whole list.
     if [ "$failed" = 1 ]; then
         break
     fi
 done
 
 echo
-echo "setup:"
+if [ "$CHECK" = 1 ]; then
+    echo "setup check:"
+else
+    echo "setup:"
+fi
 printf '%s' "$summary"
-exit "$failed"
+if [ "$failed" = 1 ] || [ "$pending" = 1 ]; then
+    exit 1
+fi
+exit 0