@azure-id/orc 1.4.2 → 1.6.0

package/templates/commands/orc-test.md

@@ -0,0 +1,66 @@
+---
+description: Run the tests against the real running system — happy path to security — and report only what was observed
+---
+
+Use the **orc-test** skill. Standalone — no plan, no build, no code written into
+your project.
+
+Every other ORC lane reasons about your system. This one **runs it**:
+
+> **A test that ran is a FACT; a test that was written is an OPINION.**
+
+ORC already has a lane that WRITES tests and never runs them (Phase 6.5,
+`test-generator/`). This is the mirror image, and the two never mix: `/orc-test`
+writes nothing into your test tree and runs everything.
+
+**It never edits the system it is testing.** If the server will not come up, it
+prints exactly what it ran and the tail of the output, and hands back. You fix
+it, and re-run for free — the same shape as `/orc-challenge`, and for the same
+reason: a session that just fixed the code cannot be trusted to measure it.
+
+**It never reports a result it did not observe.** Three verdicts and no fourth:
+`pass` (observed, matched), `fail` (observed, did not match), and `unknown` —
+not observed. `unknown` keeps its slot in the report and never becomes a pass.
+A flake is recorded, never retried away: there is no retry count, because if the
+same case answers differently twice, *that instability is the finding*.
+
+One pass:
+
+1. **Target** — `orc test init <slug>` asks what it must not guess: back end or
+   front end, local or remote, the base URL. On a remote target it also requires
+   `--authorized "<who authorized this, and where it is recorded>"`, stored
+   verbatim and reprinted at the head of every report. ORC cannot verify
+   authorization and does not pretend to — it makes the assertion impossible to
+   skip. Frozen, and never asked again.
+2. **Surface** — free, deterministic: an OpenAPI or GraphQL spec on disk, the
+   same spec at the target, route extraction by framework fingerprint, FE router
+   config. Anything ambiguous comes back as `unresolved[]` and is never guessed.
+   The **code-vs-live diff** is a first-class output: routes that answer live and
+   exist in no source file are zombie APIs.
+3. **Flow** — it digs the repo for how a request reaches your endpoint: the
+   route registration, the middleware in order, the handler, what it writes, what
+   must be true first, which fields name an object, and the login flow itself.
+4. **Environment** (local only) — it brings the system up and waits for health.
+   Five states, one next action each.
+5. **Cases** — the CLI derives the bulk for free: equivalence partitions,
+   boundary values, method and content-type negatives, stateful create → read →
+   update → delete sequences. An agent fills only what a schema cannot know.
+6. **Run** — paced, capped, and fenced to the frozen origin. A 429 is a RESULT,
+   not an error: it means rate limiting works. Every case leaves its exact
+   request, its exact response and a reproducible `curl`, with every credential
+   redacted **before the bytes reach disk**.
+7. **Report** — `orc/orc-test/<slug>/REPORT.md`, written for someone who does
+   not read code.
+
+The security tier is a **closed set** mapped to the OWASP API Security Top 10
+(2023), and it demonstrates a CONDITION rather than an extraction — confirming
+an exploit is your decision on your authority. With one identity, BOLA and BFLA
+report `UNCHECKABLE`, which keeps its slot and never becomes a pass.
+
+**The run folder is never staged.** Evidence contains real response bodies from
+a real system. The lane offers you the one `.gitignore` line; it does not edit
+`.gitignore` itself.
+
+Read the state back any time without this lane: `orc test status <slug>`.
+
+The slug to open or reopen (or nothing, and it will ask): $ARGUMENTS

package/templates/hooks/README.md

@@ -1,8 +1,11 @@
-# The ORC status line
+# The ORC hooks
 
 > This page is written in Simplified Technical English. Short sentences, one
 > idea each, plain words. See `bin/webui/i18n/TERMS.md` for the term list.
 
+This page covers the two hooks you can see. The STATUS LINE, which shows what
+is happening, and the READ GATE, which can stop an oversized read.
+
 ORC shows two lines at the bottom of your terminal. Claude Code draws them.
 ORC writes them.
 
@@ -258,6 +261,91 @@ it did not see.
 
 ---
 
+## The read gate
+
+This is a different hook. It is off when you install ORC.
+
+ORC has a rule about reading. The main session reads a file to FIND things.
+To UNDERSTAND a file, ORC sends an agent to read it and report back. The agent
+uses its own context, not yours. The rule was written in three guide files and
+nothing checked it. The read gate is the part that can say no.
+
+Turn it on like this:
+
+```
+orc config set read_gate warn     # it tells you, and still reads the file
+orc config set read_gate block    # it stops the read
+orc config set read_gate off      # the default
+```
+
+### What it does
+
+It looks at one thing: a `Read` of a whole file, by the main session, during an
+ORC run, when the file is 1000 lines or more.
+
+In `warn` it shows you a note and reads the file anyway. In `block` it stops the
+read and tells you three other ways to get what you need:
+
+- Read the file with `offset` and `limit`. This is never stopped.
+- Search the file first, then read that part.
+- Send an agent to read it. An agent's reads are never stopped.
+
+**A block always names another way.** A gate that only says no is a gate people
+turn off.
+
+### When it says nothing
+
+This list is the important part. The gate is quiet in all of these states, and
+each one is on purpose:
+
+| State | Why |
+|---|---|
+| An agent is reading | An agent must read a file in full before it edits it. If it could not, it would guess the old text and damage the file. |
+| `read_gate` is `off` | This is the default. With `off`, the hook does nothing at all. |
+| No ORC run is open | The gate is about ORC's own reading. It is not a rule for your session. Outside a run it never stops anything. |
+| You used `offset` or `limit` | This is the behaviour the gate wants. It can never stop it. |
+| The file is under 1000 lines | See the next part. |
+| The file is a build log, a test result, or `.jsonl` | ORC reads these to decide pass or fail. A cut-short failing build looks like a passing build. That is worse than any saving. |
+| The gate hit an error | It always lets the read through. Then it writes down what went wrong. |
+
+**The gate cannot see a file you read with a shell command** such as `cat` or
+`head`. It only sees the `Read` tool.
+
+### Why 1000 lines
+
+Sending an agent is not free. One real agent read cost about 13,000 tokens and
+about 76 seconds. A line in this project is about 55 characters. So a file must
+be near 1000 lines before sending an agent costs less than reading it yourself.
+
+Below that number, sending an agent costs more than it saves.
+
+You can change it:
+
+```
+orc config set read_gate_max_lines 500
+```
+
+### What it never does
+
+- It never stops an agent's read.
+- It never stops a read outside an ORC run.
+- It never stops a `Bash` command.
+- It never reads the file to you. It only counts the lines.
+- It never fails closed. If the hook breaks, your read still happens.
+
+### Where to look when it acts
+
+Every `warn` and every `block` writes one line in the run trace:
+
+```
+[070926 14:22:01.220] hook     READ-GATE block :: lines=2400 max=1000 file=big-plan.md
+```
+
+An allowed read writes nothing. `orc doctor` tells you if the gate is on but not
+wired, and if it ever had to let a read through because it could not judge it.
+
+---
+
 ## For maintainers
 
 - The hook is `orc-statusline.js`. `orc init` installs it and wires it into
@@ -285,3 +373,24 @@ it did not see.
 - The cache file is `.claude/orc/usage-session.json`. The hook reads it once and
   writes it once, after the text is ready. It stores raw numbers only — never a
   word like `fresh` or `STALE`, which is computed each time it is shown.
+
+### The read gate
+
+- The hook is `orc-read-gate.js`, on `PreToolUse` with matcher `Read`. `orc
+  init` wires it even though `read_gate` defaults to `off`, so arming it is a
+  config edit and never an install step somebody has to find.
+- **`off` is byte-identical to not having the hook**, and a test asserts it.
+- **`agent_id` is the only way to tell a subagent's read from the main
+  session's, and this was MEASURED, not assumed.** `PreToolUse` does fire
+  inside a dispatched subagent, and `session_id` and `transcript_path` are
+  identical in both. A gate written against either would block the full read an
+  executor must do before it edits, and a reconstructed `old_string` corrupts
+  files. Test for the PRESENCE of `agent_id`. Never test for the absence of
+  another key — that is not a positive statement about anything.
+- The threshold is measured, not borrowed. See `read_gate_max_lines`.
+- It fails open on every path, and each failure it can name writes
+  `.claude/orc/read-gate-fallback.json` for `orc doctor` to turn into a
+  sentence — only while the feature is armed.
+- It writes one trace line per `warn` and per `block`, never on an allow. That
+  is affordable here for a structural reason: the gate only acts while a run is
+  open, so a trace always exists.

package/templates/hooks/orc-read-gate.js

@@ -0,0 +1,279 @@
+#!/usr/bin/env node
+"use strict";
+
+/**
+ * ORC read gate — a Claude Code PreToolUse hook on `Read`.
+ *
+ * ORC's read discipline (`skills/_shared/read-ladder.md`, orc-doc hard rule 0,
+ * orc-quick line 21) says the ORCHESTRATOR reads to LOCATE and dispatches to
+ * UNDERSTAND. That rule was advisory prose in three places and enforced in
+ * none. This hook is the one layer that can refuse.
+ *
+ * It is DELIBERATELY the narrowest useful gate. Everything below is a state in
+ * which it says nothing at all, and every one of them is intentional:
+ *
+ *   - a SUBAGENT is reading            (measured: W1 — see `agent_id` below)
+ *   - `read_gate` is `off`             (the default)
+ *   - no ORC run is open
+ *   - the read is already targeted     (`offset` / `limit` — ladder step 3)
+ *   - the file is under the threshold
+ *   - the file is output a gate parses (ladder exception 2)
+ *   - `read_gate` is `warn`            (it says so, and still allows)
+ *
+ * THE SUBAGENT RULE IS LOAD-BEARING, AND IT IS MEASURED, NOT ASSUMED.
+ * `PreToolUse` DOES fire inside a dispatched subagent. `session_id` and
+ * `transcript_path` are IDENTICAL in both contexts — a gate written against
+ * either would treat an executor's read as the orchestrator's and block the
+ * full read that must precede an `Edit`, whose `old_string` cannot be
+ * reconstructed from an outline. That is a file-corruption path. The ONLY
+ * discriminator is `agent_id`, present in a subagent payload and absent in a
+ * main-session one. Test for its PRESENCE — never for the absence of some
+ * other key, which is not a positive assertion.
+ *
+ * Wiring (installed by `orc init` into .claude/settings.json):
+ *   hooks.PreToolUse[] { matcher: "Read", hooks:[{ type:"command",
+ *     command: 'node "<.claude>/hooks/orc-read-gate.js"' }] }
+ *
+ * Contract: read PreToolUse JSON from stdin. Exit 0 = allow. Exit 2 = block
+ * the tool call and show stderr to Claude, which relays it to the user.
+ *
+ * FAIL-OPEN, ALWAYS. A read gate that throws and blocks a read has broken the
+ * tool. Every failure path here exits 0, and the ones we can name RECORD
+ * themselves so `orc doctor` can turn them into a sentence — but only while
+ * the feature is armed. Precedent: orc-statusline.js is fail-silent, and the
+ * v1.3.0 statusline gate ladder is all fallbacks, each of which records itself.
+ */
+
+const fs = require("fs");
+const path = require("path");
+
+// .claude/hooks/orc-read-gate.js → CLAUDE_DIR = .. → PROJECT_ROOT = ../..
+const CLAUDE_DIR = path.join(__dirname, "..");
+const PROJECT_ROOT = path.join(CLAUDE_DIR, "..");
+
+// A run whose trace has been idle this long has ENDED. Same window as
+// orc-trace.js — one idea of "a run is open", not two.
+const STALE_MS = 6 * 60 * 60 * 1000;
+
+// Default line threshold. MEASURED, not copied.
+//
+// The number in the source material this pattern came from is 350, derived
+// from a 10-30s delegation round trip. ORC's is nothing like that: a Task
+// dispatch measured p50 76s / p90 188s (n=125), and one real read-only
+// dispatch cost 13,276 tokens to read a four-line file and return three words.
+// At the measured 55.2 chars/line across 316 main-session full reads, that
+// floor is ~962 lines; at the median file's 51.9 chars/line it is ~1024.
+// Break-even is therefore ~1000 lines, and BELOW it delegating a read costs
+// more than the read does. Copying 350 would delegate work whose overhead
+// exceeds its saving.
+const DEFAULT_THRESHOLD = 1000;
+
+// Output a gate PARSES is read whole — read-ladder exception 2. The smoke
+// gate, the TDD gate, the verifier and /orc-quick's build loop all decide red
+// vs green from these exact bytes, and a truncated red build reads GREEN.
+// That is worse than any token saving, so this list is deliberately generous:
+// a false ALLOW costs tokens, a false BLOCK costs correctness.
+const GATE_PARSED =
+  /(\.(log|tap|xml|lcov|junit)$|(^|[\\/])(coverage|test-results|npm-debug|yarn-error)([\\/]|$)|\.jsonl$)/i;
+
+// ---------------------------------------------------------------------------
+// Config — tolerant top-level YAML scalar read, matching orc-trace.js exactly.
+// The repo is zero-dep; there is no YAML parser.
+// ---------------------------------------------------------------------------
+function readConfigScalar(key) {
+  try {
+    const text = fs.readFileSync(path.join(CLAUDE_DIR, "orc.config.yaml"), "utf8");
+    const re = new RegExp("^" + key + "\\s*:\\s*(.+?)\\s*(?:#.*)?$", "m");
+    const m = text.match(re);
+    return m ? m[1].replace(/^['"]|['"]$/g, "").trim() : null;
+  } catch (_) {
+    return null;
+  }
+}
+
+function logDir() {
+  const rel = readConfigScalar("log_dir") || ".claude/orc/logs";
+  return path.isAbsolute(rel) ? rel : path.join(PROJECT_ROOT, rel);
+}
+
+// Is an ORC run open? `.current` names the run; the trace file it names must
+// exist and be fresh. A pointer naming a file that does not exist is
+// indistinguishable from a dangling one (v0.34.2), so honour a pointer whose
+// OWN mtime is fresh too — neither half depends on the other.
+function runIsOpen() {
+  const dir = logDir();
+  let name;
+  try {
+    name = fs.readFileSync(path.join(dir, ".current"), "utf8").trim();
+  } catch (_) {
+    return false;
+  }
+  if (!name) return false;
+  const now = Date.now();
+  try {
+    if (now - fs.statSync(path.join(dir, name)).mtimeMs < STALE_MS) return true;
+  } catch (_) {
+    /* trace file not created yet — fall through to the pointer's own mtime */
+  }
+  try {
+    return now - fs.statSync(path.join(dir, ".current")).mtimeMs < STALE_MS;
+  } catch (_) {
+    return false;
+  }
+}
+
+// D10 — A GATE DECISION THAT LEAVES NO LINE CANNOT BE COUNTED. Same rule as a
+// resume and a demotion, and it is CHEAP HERE FOR A STRUCTURAL REASON: the gate
+// only ever acts while a run is open (rung 2), so a trace to write into is
+// guaranteed to exist by the time we get here. There is no "a read happened
+// with no run" case to handle, because that read was already allowed.
+//
+// Hook-composed, like every other line orc-trace.js writes. Best effort: a line
+// that cannot be written must never take the gate down with it.
+function traceLine(verb, tail) {
+  try {
+    const dir = logDir();
+    const name = fs.readFileSync(path.join(dir, ".current"), "utf8").trim();
+    if (!name) return;
+    const d = new Date();
+    const p = (n, w) => String(n).padStart(w || 2, "0");
+    const stamp =
+      p(d.getDate()) + p(d.getMonth() + 1) + p(d.getFullYear() % 100) + " " +
+      p(d.getHours()) + ":" + p(d.getMinutes()) + ":" + p(d.getSeconds()) +
+      "." + p(d.getMilliseconds(), 3);
+    fs.appendFileSync(
+      path.join(dir, name),
+      `[${stamp}] ${"hook".padEnd(8)} ${verb}` + (tail ? ` :: ${tail}` : "") + "\n"
+    );
+  } catch (_) {}
+}
+
+// A fallback RECORDS ITSELF (v1.3.0's rule). Best effort by construction: a
+// record that cannot be written must never take the gate down with it.
+function recordFallback(rung, detail) {
+  try {
+    const dir = path.join(CLAUDE_DIR, "orc");
+    fs.mkdirSync(dir, { recursive: true });
+    fs.writeFileSync(
+      path.join(dir, "read-gate-fallback.json"),
+      JSON.stringify({ rung, detail: String(detail || "").slice(0, 300), at: Date.now() })
+    );
+  } catch (_) {}
+}
+
+// Count lines without holding the whole file in memory twice. Returns null when
+// the file cannot be measured — and null NEVER blocks (unknown is not "big").
+function countLines(file) {
+  try {
+    const st = fs.statSync(file);
+    if (!st.isFile()) return null;
+    // A file that cannot possibly reach the threshold is not worth reading.
+    // 1 byte/line is the floor, so bytes < threshold ⇒ lines < threshold.
+    const buf = fs.readFileSync(file);
+    let n = 1;
+    for (let i = 0; i < buf.length; i++) if (buf[i] === 10) n++;
+    return n;
+  } catch (_) {
+    return null;
+  }
+}
+
+let raw = "";
+process.stdin.on("data", (c) => (raw += c));
+process.stdin.on("end", () => {
+  // RUNG 0 — anything at all goes wrong: ALLOW. This wrapper is the whole
+  // fail-open guarantee; every `return` below it is an allow.
+  try {
+    gate();
+  } catch (e) {
+    recordFallback("threw", e && e.message);
+    process.exit(0);
+  }
+  process.exit(0);
+});
+
+function gate() {
+  let data;
+  try {
+    data = JSON.parse(raw || "{}");
+  } catch (e) {
+    recordFallback("unparseable-payload", e && e.message);
+    return; // allow
+  }
+
+  if (data.tool_name !== "Read") return;
+
+  // RUNG 0.5 — a SUBAGENT is reading. ALLOW, always, and this check comes
+  // before every other one. See the header: this is what keeps the gate off
+  // the pre-`Edit` full read that an executor must perform, and it is why the
+  // gate never needs to know a slice's declared_files.
+  if (data.agent_id != null) return;
+
+  const mode = String(readConfigScalar("read_gate") || "off").toLowerCase();
+  if (mode !== "warn" && mode !== "block") return; // `off` (the default), or garbage
+
+  // RUNG 2 — no ORC run is open. The gate constrains ORC's own reading; it is
+  // not a policy on the user's session. A read outside a run is not ORC's to
+  // refuse, and the gate says nothing. This is an HONEST LIMIT, not an
+  // oversight: it is stated in templates/hooks/README.md and in `orc doctor`.
+  if (!runIsOpen()) return;
+
+  const input = data.tool_input || {};
+  const file = String(input.file_path || "");
+  if (!file) return;
+
+  // RUNG 3 — already a targeted read. This IS ladder step 3; it is the
+  // behaviour the gate exists to encourage, so it can never be refused.
+  if (input.offset != null || input.limit != null) return;
+
+  // RUNG 5 — output a gate parses. Exception 2 is not a preference.
+  if (GATE_PARSED.test(file)) return;
+
+  let threshold = parseInt(readConfigScalar("read_gate_max_lines"), 10);
+  if (!Number.isFinite(threshold) || threshold < 1) threshold = DEFAULT_THRESHOLD;
+
+  const lines = countLines(file);
+  if (lines == null) {
+    // Unmeasurable — a directory, a missing file, a permissions error, a binary
+    // the Read tool will handle its own way. UNKNOWN IS NOT BIG.
+    recordFallback("unmeasurable", file);
+    return;
+  }
+  if (lines < threshold) return; // RUNG 4 — under the threshold
+
+  // RUNG 7 — `warn`: allow, and say so. `systemMessage` is shown to the user
+  // and is NOT added to model context, so the warning itself costs no tokens.
+  if (mode === "warn") {
+    traceLine("READ-GATE warn", `lines=${lines} max=${threshold} file=${path.basename(file)}`);
+    try {
+      process.stdout.write(
+        JSON.stringify({
+          systemMessage:
+            `📖 ORC read gate — ${path.basename(file)} is ${lines} lines ` +
+            `(threshold ${threshold}). Allowed: read_gate is 'warn'.\n` +
+            `   Cheaper: Read with offset/limit, or dispatch an agent to read it and report back.`,
+        })
+      );
+    } catch (_) {}
+    return;
+  }
+
+  // RUNG 8 — block, and NAME THE ALTERNATIVE. A block that only refuses
+  // teaches people to switch it off.
+  traceLine("READ-GATE block", `lines=${lines} max=${threshold} file=${path.basename(file)}`);
+  process.stderr.write(
+    `\n⛔ ORC read gate — ${path.basename(file)} is ${lines} lines, over the ` +
+      `${threshold}-line threshold.\n` +
+      `   Reading it whole spends the orchestrator's context on material a ` +
+      `worker could summarise.\n\n` +
+      `   Do one of these instead:\n` +
+      `     • Read with offset/limit — the ladder's targeted read, and always allowed.\n` +
+      `     • Grep for what you actually need, then read that range.\n` +
+      `     • Dispatch an agent to read it and report back (its reads are never gated).\n\n` +
+      `   If you genuinely need the whole file here: set read_gate to warn or off\n` +
+      `     orc config set read_gate warn\n` +
+      `   The gate is silent outside an ORC run, on targeted reads, on files a\n` +
+      `   gate parses, and on every read a subagent makes.\n`
+  );
+  process.exit(2);
+}