@zenera/cli 1.1.9 → 1.1.11

package/templates/editor/.github/prompts/sync-with-spec.prompt.md

@@ -0,0 +1,202 @@
+---
+description: Bring the project into compliance with SPECIFICATION.md, and record every open question in SPECIFICATION-FEEDBACK.md.
+---
+
+Make this project match `SPECIFICATION.md`. The specification is the intent;
+the files are the implementation. Where they disagree, the specification wins —
+except where it is ambiguous, contradictory or impossible, which is what
+`SPECIFICATION-FEEDBACK.md` is for.
+
+## 1. Read before changing anything
+
+Read, in this order, and do not edit until all of it is read:
+
+1. `SPECIFICATION.md` — what the system is meant to be.
+2. `agents.yaml` — providers, models, embeddings, agents, tools, handoffs,
+   skills, sandbox.
+3. `INSTRUCTIONS.md` and every file under `agents/prompts/`.
+4. Every `agents/skills/*/SKILL.md`, plus what each skill folder ships.
+5. `assets/`, `sandbox/Dockerfile` and every script under `scripts/`, starting
+   with `scripts/_setup.sh`.
+6. `SPECIFICATION-FEEDBACK.md`, if it exists — do not re-raise something already
+   open there.
+
+Run `zen check` first and record what it said. A project that does not load is
+the first thing to fix.
+
+## 2. Build the difference
+
+Produce a difference list before touching a file. For every item in the
+specification, decide which of these it is:
+
+- **present and correct** — leave it alone
+- **missing** — the specification asks for it and nothing implements it
+- **divergent** — implemented, but not as specified
+- **extra** — implemented, and the specification does not ask for it
+- **unclear** — the specification does not say enough to implement it
+
+Check at least: the model and embedding ids and which agent uses which; that
+every agent named in the specification exists with that job and no second job;
+handoff edges and fork permissions; the tools each agent holds; every skill the
+specification implies; and every setup step, index or asset it depends on.
+
+Then build the difference the other way, because the list above can only find
+what the specification asked for and nothing implements. Read `INSTRUCTIONS.md`,
+each `agents/prompts/*.md` and each `SKILL.md` **line by line**, and for every
+line name the specification item it serves. A line serving none is an **extra** —
+the category covers a single sentence in a prompt, not only a whole file. Three
+things to look for while reading, none of which the forward pass can see:
+
+- **A line that contradicts another file.** One prompt against another, a prompt
+  against `INSTRUCTIONS.md`, or a prompt against `agents.yaml` — an instruction
+  to use a tool the agent does not hold, to hand to an agent that is not in its
+  `handoffs:`, or to load a skill outside its `allow:`.
+- **A line that asks for something the system already supplies.** A question put
+  to the user about a value that arrives from the environment, a file or an
+  earlier step is unanswerable, and the answer would be discarded.
+- **A line the agent cannot carry out** — naming a path it cannot reach, a tool
+  it was not granted, or an agent it cannot transfer to.
+
+These are bugs whatever the specification says: fix them in place and report each
+one. An extra that is merely unspecified is reported under `➕`, not removed.
+
+## 3. Change the smallest thing that closes the gap
+
+Apply the fixes in the order the copilot instructions prefer: a prompt or skill
+edit over a new skill, a new skill over a new agent, a tool grant over asking
+the model to compute or remember. Specifically:
+
+- A behaviour gap is a prompt or skill edit, not a new agent.
+- A fact, rate, threshold or command surface goes in a skill, never in a prompt.
+- Arithmetic or a fixed transformation goes in a script under the skill folder,
+  called by its absolute `/skills/<name>/…` path.
+- A capability gap is a `tools:` grant plus the prompt line that says when to
+  use it. A granted tool no prompt mentions is not implemented.
+- A toolchain the work always needs goes in `sandbox/Dockerfile`, not in a
+  prompt that installs it every run.
+- If the project has a `sandbox/Dockerfile`, pin `@zenera/cli` and `@zenera/rag`
+  in it to the version of `zen` in hand. Read that version with `zen --version`
+  and write it into the `npm install -g` line, so the CLI an agent runs inside
+  the container is the same one that built it. An unpinned name drifts away from
+  the host on the next image build; a stale pin is a mismatch nothing reports.
+
+Never delete an **extra** without saying so. Report it, say why the
+specification does not cover it, and leave it unless I tell you to remove it.
+
+Do not invent a value the specification does not state — a model id, a limit, a
+retry count, a file path. That is an entry in `SPECIFICATION-FEEDBACK.md`, not a
+guess.
+
+## 4. Every setup step is a script
+
+Anything the project needs done before it can run — building an index, fetching
+a document, generating a file, warming the sandbox image — is a shell script,
+not a paragraph of instructions for a human.
+
+- One step per file, under `scripts/`, named after the step:
+  `scripts/build-schema-index.sh`, `scripts/fetch-api-docs.sh`.
+- `scripts/_setup.sh` is the **only** entry point — the leading underscore is
+  what separates the runner from the steps it runs. It runs the steps in the
+  order they depend on each other and is the single command that initialises the
+  project. Consolidate any setup script that lives elsewhere into this shape,
+  including one left at the project root.
+- Every script starts `set -eu`, works from any working directory
+  (`cd "$(dirname "$0")/.."` first), needs no arguments, and exits non-zero on
+  failure so `scripts/_setup.sh` stops rather than continuing on a broken step.
+- **Re-entrancy is required.** Running `scripts/_setup.sh` a second time must be
+  safe. A step whose output is present and newer than its inputs prints that it
+  is up to date and returns 0; otherwise it redoes the work idempotently, writing
+  to a temporary path and moving it into place so an interrupted run never leaves
+  a half-built artefact behind. `scripts/_setup.sh --force` redoes everything.
+- **It must be watchable.** `scripts/_setup.sh` tees each step's output to
+  `.tmp/logs/setup-<step>.log`, prints a heartbeat line while a long step runs,
+  and finishes with one line per step: `ok`, `skipped`, or `failed`.
+- **Everything transient goes under `.tmp/`** — logs, scratch files, downloads,
+  test output, and the temporary path a step writes to before moving its
+  artefact into place. Nothing else in the tree is a scratch directory, and
+  `.tmp/` is git-ignored: add it to `.gitignore` if it is not there already.
+  Deleting `.tmp/` must leave the project runnable and `scripts/_setup.sh`
+  re-runnable.
+- Steps that need credentials run on the host, not in the sandbox. Read keys
+  from the environment or `.env`; never inline one into a script.
+- Anything the sandbox always needs belongs in `sandbox/Dockerfile`, not in a
+  setup script that installs it again on every run.
+
+## 5. Write `SPECIFICATION-FEEDBACK.md`
+
+Create or update `SPECIFICATION-FEEDBACK.md` at the project root whenever
+anything in the
+specification is unclear, contradictory, impossible, or wrong. If there is
+nothing to raise, say so in the chat and do not create the file.
+
+Group entries under these headings, and omit a heading that has no entries.
+The file is scanned before it is read, so keep the emoji on every heading:
+
+```markdown
+# Feedback on SPECIFICATION.md
+
+## 🛑 Blocking — cannot implement without an answer
+
+## ❓ Ambiguous — implemented one way, confirm the choice
+
+## ⚡ Contradictions
+
+## ✏️ Errors and typos
+
+## ➕ Out of scope — implemented but unspecified
+```
+
+Each entry is one bullet, opening with the emoji of the heading it sits under,
+then four parts in this order:
+
+- **What the specification says**, quoted, with the section it is in.
+- **Why it cannot be implemented as written** — the specific gap, not "unclear".
+- **What was done in the meantime**, or "nothing — blocked".
+- **The question**, phrased so a one-line answer unblocks it.
+
+Use no other emoji anywhere in the file — one per entry is what makes them
+scannable. Keep entries factual and short. Do not editorialise about the
+specification's quality, do not restate the runtime, and do not copy the whole
+section in.
+Append to the existing file rather than rewriting it; strike an entry only when
+this pass has actually resolved it, and say in the chat which ones you closed.
+
+## 6. Run `scripts/_setup.sh` and watch it finish
+
+After `zen check` passes, run `scripts/_setup.sh` yourself. A pass is not done
+until it has run to completion in this session.
+
+It is slow — an embedding index is minutes, not seconds — so start it in the
+background and follow the logs instead of waiting blind:
+
+- Tail `.tmp/logs/setup-*.log` and check the output artefact is still growing.
+- **Wait in a loop, never in one long sleep.** Sleep at most 30 seconds at a
+  time, then check the log and say what it shows, and repeat until the step
+  finishes. A single long block makes the run unwatchable and hides a failure
+  that happened in the first ten seconds.
+- Report progress as you go: which step is running, how long it has been going,
+  what the last log line said.
+- Do not kill a quiet step. Confirm the process is dead or the log and the
+  output have both stopped growing before calling it stuck.
+- On failure, read that step's log, fix the cause, and run `scripts/_setup.sh`
+  again — re-running is the fix path, not a reset. If the same step fails twice
+  for the same reason, stop and raise it in `SPECIFICATION-FEEDBACK.md`.
+- Prove re-entrancy: once it has succeeded, run `scripts/_setup.sh` once more and
+  check that every step reports `skipped` rather than rebuilding.
+
+## 7. Verify and report
+
+Run `zen check` again and fix whatever it names rather than working around it.
+It validates structure and says nothing about the prose, so work §9 of the
+copilot instructions as a checklist afterwards and report each group pass or
+fail. Then report in the chat:
+
+- what changed, file by file, and which specification item each change closes
+- every prompt or skill line that served no specification item, and what became
+  of it — corrected, or reported and left alone
+- the outcome of every `scripts/_setup.sh` step, and what each one produced
+- what is still open, with a pointer to its `SPECIFICATION-FEEDBACK.md` entry
+- anything you found that the specification does not cover at all
+
+Do not report a specification item as done unless the file that implements it
+exists, `zen check` passes, and `scripts/_setup.sh` completed every step it owns.

package/templates/editor/.github/skills/zen-cli/SKILL.md

@@ -29,6 +29,7 @@ zen version
 
 zen faker    <serve|build|cache> [spec...]
 zen rag      schema <index|search|list|grep|trace|show|stats> [spec...]
+zen rag      docs <index|search|list|grep|show|stats> [path...]
 ```
 
 ## Read the reference before answering
@@ -48,7 +49,7 @@ guess a flag, and do not read them all.
 | The container shell commands run in, images, `persist`              | [sandbox.md](./references/sandbox.md)   |
 | Run reports, trajectories, what a session directory holds           | [inspect.md](./references/inspect.md)   |
 | `zen faker` — a mock API from an OpenAPI/Swagger document           | [faker.md](./references/faker.md)       |
-| `zen rag` — searching an OpenAPI/Swagger document as a graph        | [rag.md](./references/rag.md)           |
+| `zen rag` — searching an API description, or a pile of markdown     | [rag.md](./references/rag.md)           |
 
 ## The short version
 

package/templates/editor/.github/skills/zen-cli/references/faker.md

@@ -91,17 +91,27 @@ Probes are synthetic on purpose — real request bodies never reach a prompt.
 
 ## The cache
 
-Under `~/.zenera/neo/faker/generators/<key>/`, one directory per operation. A
-generator that a model gave up on is remembered, so a hopeless operation is not
-re-asked on every request; a _transient_ failure — a 429, a dropped socket — is
-not, because it is about this minute rather than this operation.
-
-`zen faker cache clear` removes the generators and the container together. They
-have to go together: the container's name is a hash of its configuration, so
-deleting the directory alone would leave a stopped container bind-mounted onto a
+Generators live in the machine's shared cache, `~/.zenera/neo/cache/faker-generator/`,
+keyed by the operation's shape — so the same spec served from two directories is
+written once, and a spec edit produces a new key rather than a stale answer. A
+generator that a model gave up on is remembered for the life of the process, so
+a hopeless operation is not re-asked on every request; a _transient_ failure — a
+429, a dropped socket — is not, because it is about this minute rather than this
+operation.
+
+`~/.zenera/neo/faker/` is the container's workspace, and scratch: a cache hit is
+copied into it, because the container can only run what is under its mount.
+
+`zen faker cache ls` lists what has been generated. `zen faker cache clear`
+removes the generators, the workspace and the container together. They have to
+go together: the container's name is a hash of its configuration, so deleting
+the workspace alone would leave a stopped container bind-mounted onto a
 directory that no longer exists, and every generator would fail with
 `python3: can't open file '/workspace/generators/…/gen.py'`.
 
+`zen cache ls --kind faker-generator` reaches the same entries from the other
+side, along with everything else this machine has cached.
+
 ## Documents it accepts
 
 Swagger 2 and OpenAPI 3.0/3.1, JSON or YAML, `$ref`s resolved. Several documents

package/templates/editor/.github/skills/zen-cli/references/keys.md

@@ -25,8 +25,8 @@ keys; one of them is active.
 | `zen key show <ref> [--reveal]`   | Masked by default                       |
 | `zen key env [provider …]`        | Shell exports, for other tools          |
 
-`zen key add` also takes `--project` and `--location`, for a Vertex service
-account.
+`zen key add` also takes `--gcp-project` and `--gcp-location`, for a Vertex
+service account.
 
 ```
 zen key add openai                    # prompts, echo off
@@ -56,18 +56,18 @@ echo-off prompt and from nowhere else.
 
 A **service-account file** is absorbed into `~/.zenera/neo/keys/` and
 `GOOGLE_APPLICATION_CREDENTIALS` points at it. It wants a project too, from
-`--project`, or `GOOGLE_CLOUD_PROJECT`, or the `project_id` inside the file, and
-a region from `--location` (`global` otherwise). Application Default
+`--gcp-project`, or `GOOGLE_CLOUD_PROJECT`, or the `project_id` inside the file,
+and a region from `--gcp-location` (`global` otherwise). Application Default
 Credentials from `gcloud auth application-default login` work with no entry at
 all.
 
 An **express-mode key** is an ordinary secret under `VERTEX_API_KEY`, and
 addresses no project: a key and a project are alternatives, and sending both
-gets a `403` that mentions neither. `--project` and `--location` are therefore
-refused alongside a key.
+gets a `403` that mentions neither. `--gcp-project` and `--gcp-location` are
+therefore refused alongside a key.
 
 ```
-zen key add vertex --project acme-prod --location europe-west4
+zen key add vertex --gcp-project acme-prod --gcp-location europe-west4
 # paste a path  → the file shape
 # paste a key   → the express shape
 ```

package/templates/editor/.github/skills/zen-cli/references/rag.md

@@ -327,3 +327,107 @@ found and the endpoint to call is what is actually wanted.
 
 They share the group `schema`, so an agent takes them with `schema:*` in its
 `tools:`.
+
+# Document search — `zen rag docs`
+
+```
+zen rag docs <index|search|list|grep|show|stats> [args...]
+```
+
+The second subject of the same command. Where `schema` reads an API description
+as a graph, `docs` reads a pile of **markdown and plain text** and answers with
+the documents themselves: the passages that matched, quoted verbatim with their
+line numbers, and a marker wherever something between two of them was skipped.
+
+## `index`
+
+```
+zen rag docs index <path...> [--embedding <ref>] [-o <dir>] [--chunk-tokens <n>]
+```
+
+| Flag                 | Default     | Meaning                                                      |
+| -------------------- | ----------- | ------------------------------------------------------------ |
+| `--embedding <ref>`  | —           | Which embedder makes the vectors. Omit it to see the choices |
+| `-o`, `--out <dir>`  | `./docs-db` | Where the index goes; `$ZEN_DOCS_DB` if set                  |
+| `--batch <n>`        | `96`        | Texts per embedding request                                  |
+| `--chunk-tokens <n>` | `384`       | Target chunk size                                            |
+| `--quiet`            | —           | No narration                                                 |
+
+Paths may be files, directories or globs. `.md`, `.markdown`, `.txt` and
+`.text` are read; hidden directories and `node_modules` are not.
+
+Every document is **copied into the index**, and every quoted line is read back
+out of that copy — so the index is one portable thing and what it quotes is the
+document rather than a reconstruction of it. Each document is named by its path
+relative to the common root of everything indexed, which is what keeps
+`nsx_4.1.0/api/routing.md` and `nsx_4.2.0/api/routing.md` apart.
+
+```
+docs-db/
+manifest.json     written last; its presence means the index is complete
+outline.json      every heading and table, with the lines they cover
+sources/          the documents, verbatim
+lance/            one row per chunk: two texts, one vector, the filter columns
+```
+
+## `search`
+
+```
+zen rag docs search [text...] [-f <pattern>] [-s <section>] [--kind <k>]
+```
+
+| Flag                     | Default  | Meaning                                              |
+| ------------------------ | -------- | ---------------------------------------------------- |
+| `-f`, `--file <pattern>` | —        | Only documents whose name matches. Repeatable        |
+| `--exclude-file <p>`     | —        | Drop documents whose name matches                    |
+| `-s`, `--section <name>` | —        | Only under this heading, and what nests in it        |
+| `--kind <k>`             | —        | `paragraph`, `list`, `table`, `table_row`, `code`, … |
+| `--mode <m>`             | `hybrid` | `hybrid`, `vector` or `text` for exact wording       |
+| `--limit <n>`            | `8`      | Passages kept                                        |
+| `-B`, `-A <n>`           | `0`      | Extra lines quoted before and after each passage     |
+| `--max-lines <n>`        | `400`    | A ceiling on the whole answer                        |
+| `--exclude-id <id>`      | —        | Drop a passage already seen. Repeatable              |
+| `--hits`                 | —        | One line per passage instead of the text             |
+| `--interactive`          | —        | Prompt, search, narrow, search again                 |
+
+**Narrowing is the interface.** Nobody finds the right paragraph on the first
+ask; the second call is the same question inside one part of the tree.
+
+```
+zen rag docs search "how are rate limits counted"
+zen rag docs search --file "nsx_4.2.*/api/**" "rate limit for the users route"
+zen rag docs search --section "Rate limits" --kind table "requests per minute"
+zen rag docs search --mode text "X-RateLimit-Remaining"
+```
+
+A `--file` pattern with `*` or `?` is a glob over the whole document name and
+a substring otherwise.
+
+## `list`, `grep`, `show` — no embedder, no credential
+
+```
+zen rag docs list <files|sections|tables> [-f <pattern>] [-s <section>]
+zen rag docs grep <pattern> [-f <pattern>] [--regex] [--case-sensitive]
+zen rag docs show <file> [--section <name>] [--lines <from-to>]
+```
+
+`list sections` gives every heading with the lines it spans, which is how a
+section is named before it is searched. `grep` reports `found` as the true
+total even when `--limit` cuts the rows, so unlike a search it can answer
+whether a string appears at all. `show` prints a document, a named section, or
+a line range, verbatim.
+
+## Giving it to an agent
+
+`@zenera/rag/docs/tools` exports four tools in the group `docs`:
+
+| Tool          | For                                                             |
+| ------------- | --------------------------------------------------------------- |
+| `search_docs` | The search above, with the same narrowing parameters            |
+| `list_docs`   | The documents, their headings, or their tables — no ranking     |
+| `grep_docs`   | Every matching line, counted in full — the way to prove absence |
+| `read_docs`   | A section or a line range, verbatim and complete                |
+
+Only `search_docs` ranks. Every answer carries line numbers and `read_docs`
+takes them, which is the loop the subject exists for: find the passage, read
+around it, then edit the file it came from.