@azure-id/orc 1.4.2 → 1.6.0

package/CHANGELOG.md

@@ -10,6 +10,192 @@ Format: `### v<version> — <title> _(<date>)_`.
 
 ---
 
+### v1.6.0 - the rule that can finally say no _(2026-09-07)_
+
+**Still on the unscoped `orc` package?** Do this once first - your `orc upgrade`
+is the pre-v0.56.0 one and cannot install itself. Full detail in the CAUTION at
+the top of this file.
+
+- **Step 1 - release the command from the old package:** `npm uninstall -g orc`
+- **Step 2 - install the current package:** `npm i -g @azure-id/orc`
+- **Step 3 - re-apply it to your project:** `orc update`
+
+**Do not use `npm i -g -f`.** Full detail in v0.56.0 below.
+
+
+ORC's read discipline has always said the same thing in three places: the main
+session reads to **find**, and dispatches an agent to **understand**. It was
+prose in `_shared/read-ladder.md`, in `/orc-doc` hard rule 0, and on line 21 of
+`/orc-quick` - and **nothing checked any of it**.
+
+`orc-read-gate.js` is a `PreToolUse` hook on `Read` that can refuse. It ships
+**off**, and `off` is byte-identical to not having it - asserted by a test, not
+by intention.
+
+**The threshold is ORC's own number, measured.** The pattern this came from uses
+350 lines, derived from a 10-30 second delegation round trip. ORC's round trip
+is nothing like that: a dispatch measured **p50 76s, p90 188s** (n=125), and one
+real read-only dispatch cost **13,276 tokens** to read a four-line file. At the
+measured 55.2 chars/line across 316 sampled reads, break-even is **~1000 lines**.
+Copying 350 would delegate work whose overhead exceeds its saving.
+
+**`agent_id` is the only discriminator, and that was MEASURED, not assumed.**
+`PreToolUse` **does** fire inside a dispatched subagent - the assumption that
+hooks are session-level was wrong. `session_id` and `transcript_path` are
+**identical** in both contexts, so a gate written against either would block the
+full read an executor must perform before an `Edit`, and a reconstructed
+`old_string` corrupts files. The gate tests for the **presence** of `agent_id`,
+never for the absence of some other key, which asserts nothing.
+
+**Everything it stays silent on, and each one is deliberate:** any read by a
+subagent - outside an open ORC run - a targeted `offset`/`limit` read - a file
+under the threshold - build logs, test results and `.jsonl` that a gate parses
+whole, because a truncated red build reads **green** - and any error at all,
+because **a read gate that throws and blocks a read has broken the tool.**
+
+**A block always names the cheaper path.** A gate that only refuses is a gate
+people switch off. It names the targeted read, the agent dispatch, and the
+config key that turns it down.
+
+**Every `warn` and `block` writes one trace line** (`READ-GATE`), an allow
+writes none. That is affordable here for a structural reason: the gate only acts
+while a run is open, so a trace always exists. `orc doctor` gains
+`read-gate-unwired` and `read-gate-fallback`, both reported **only while the
+feature is armed** - a doctor that warns about the default is one people learn
+to ignore.
+
+**Two config keys**, both on the `SEED_EMPTY` allowlist with an empty `lanes[]`,
+because a hook has no lane and cannot resolve config: `read_gate`
+(`off`|`warn`|`block`, default `off`) and `read_gate_max_lines` (default 1000).
+
+**What this release deliberately does NOT do**, with the reasons recorded so
+nobody re-proposes them: no gate on `Bash` reads (`cat`/`head`/`tail`) - it is
+several times the false-positive surface, and the measurement says the `Read`
+tool is only about 7.5% of the actual read surface here - and no
+`context-reader` slot in `EXTRA_SLOTS`, because at a 1000-line threshold the
+addressable population is five reads across 239 sampled sessions, which is
+infrastructure for nothing.
+
+**An honest note on the measurement.** The audit behind this release found the
+existing prose is largely *working*: median full read is 85 lines, p90 is 288,
+and 46% of reads already use `offset`/`limit` without being told. Oversized
+reads are about **1% of price-weighted main-session ingest** at a generous upper
+bound. This hook is a guardrail on a road most runs already stay on - it is off
+by default for exactly that reason.
+
+---
+
+### v1.5.0 - the lane that runs the test _(2026-09-07)_
+
+**Still on the unscoped `orc` package?** Do this once first - your `orc upgrade`
+is the pre-v0.56.0 one and cannot install itself. Full detail in the CAUTION at
+the top of this file.
+
+- **Step 1 - release the command from the old package:** `npm uninstall -g orc`
+- **Step 2 - install the current package:** `npm i -g @azure-id/orc`
+- **Step 3 - re-apply it to your project:** `orc update`
+
+**Do not use `npm i -g -f`.** Full detail in v0.56.0 below.
+
+Every other testing surface in ORC **writes** tests. `/orc-test` **runs** them.
+
+It points at a running API - yours, on your laptop, or one you were given
+permission to touch - sends real requests, writes down what came back, and
+stops. The split is the whole design:
+
+> **The CLI executes and measures. The model designs and interprets.**
+> A model never sends a request. The CLI never decides what a response means.
+
+That split is what makes the report worth reading. A number in it was measured
+by a program. A sentence in it was written by a model that only ever saw
+evidence on disk.
+
+**Three rules hold it together**, each a registered contract:
+
+- **`a lane that reports a result it did not observe`** - three verdicts, and
+  `unknown` is the honest one. A case nobody ran is never a pass.
+- **`a lane that sends traffic to a target nobody authorized`** - the target is
+  frozen once, a remote one **requires** a written statement of who said yes,
+  and every request is fenced to one origin.
+- **`a lane that fixes the system under test`** - it reports and hands back. Not
+  to fix a boot failure, not to add a `data-testid`, not to relax a check.
+
+**The free pass runs first, and always.** `orc test surface` reads your
+repository and the target's own spec document for zero model tokens, and the
+**code-vs-live diff** falls out of it: a route that answers at the target and is
+in **no file in your repository** is undocumented surface nobody reviews and
+nobody patches. It is derived from the two spec documents and from nothing else
+- no route is ever called to produce it. With no spec at the target the diff
+reads **NOT MEASURED, with the reason**, because an empty list there would say
+"no shadow APIs", which is a claim nobody made.
+
+**The case matrix derives itself.** The tiers come out of the schema for free.
+What costs anything is a **gap** - what a schema cannot know, like what a valid
+email looks like in your business, or which order two calls have to happen in.
+That is the only part a model is paid for.
+
+**The security tier is a closed set** - the OWASP API Top 10 (2023), ten rows,
+never extended on the fly, and **all ten render in every state**. A category ORC
+could not measure says `UNCHECKABLE`, **keeps its slot**, carries its own
+reason, never becomes a pass, and never changes the exit code. Reporting an
+unmeasured category as clean is the single most damaging thing a security report
+can do, because somebody then ships on it. It **detects and never exploits**:
+every probe shows the condition and stops there.
+
+**When your app is broken, it stops.** `orc test env` finds the start command,
+sees the process is alive, reads the exact line in the log - and **will not open
+that file**. It names the environment values that are absent and writes a value
+for none of them, because a placeholder in a real environment is a
+credential-shaped lie and only you know the right one.
+
+**The runner is the only place in ORC that sends a request to a system it did
+not start**, so everything in it exists to make sure ORC's own run is never the
+incident. A red happy path **stops the ladder** - a 500 on every request will
+"prove" a dozen vulnerabilities that are one bug. The pace is the target's, and
+the CLI applies it. **A 429 is a result**: it means rate limiting works, so ORC
+records it and backs off rather than pushing through, which is how a staging
+scan becomes a lockout. Failed logins are capped per identity. **Redaction is
+structural** - every credential is replaced before the bytes reach disk, not in
+a review step afterwards.
+
+**A finding has to point at something.** One that cites evidence which does not
+resolve on disk is **dropped by name and counted**. Severity is ORC's; if the
+interpreting model disagrees, its word is kept beside ORC's and printed, so a
+disagreement is visible instead of invisible. **A flake is recorded, never
+retried away** - the instability is the finding.
+
+**The front end drives Playwright as a script**, not as an agent loop: a step an
+LLM took is not a step you can re-run. ORC **names** the install command and
+never runs it - it does not install into your project.
+
+**`orc ui` ▸ Test** renders all of it and works out none of it. Five tabs, a
+button for every free action, and **`orc test run` as a command you copy** -
+never a button. It costs no model tokens, so the usual rule would make it one;
+it sends real traffic, and a page that can start a scan against a live host is a
+page that can start one by accident.
+
+**The run folder is never staged.** It holds real response bodies from a real
+system - the most sensitive thing ORC writes to disk. `orc doctor` tells you if
+it is not git-ignored, and it will not edit `.gitignore` for you.
+
+Five config keys: `test_gate` (warn only - the payoff is knowing, not gating),
+`test_security_tier`, `test_max_rps`, `test_case_budget`, `test_ui_driver`. Five
+more were **refused**, with the reasons written down so nobody proposes them
+again - among them `test_retries`, because a flake is recorded rather than
+retried away, and `test_auto_fix`, which is this lane's premise inverted.
+
+**And one bug that was not in this lane.** Writing the tests isolated an
+intermittent failure that turned out to be real: `orc ui` compressed its API
+answers but not its **static files**, guarded by a comment reading *"No asset is
+over 64 KiB today"* that had been false for two releases. `js/panels/extra.js`
+is 138 KB. Over a Windows loopback socket a response that size loses its tail
+roughly one time in six - which in a browser is a panel script that stops in the
+middle of a function, with nothing in any log to say so. Both paths now share
+one encoder. **A stale comment can guard a live bug for longer than the bug
+would have survived alone.**
+
+---
+
 ### v1.4.2 - the panel that stops reloading, and the fields you could not read back _(2026-09-05)_
 
 **Still on the unscoped `orc` package?** Do this once first - your `orc upgrade`

package/README.md

@@ -7,14 +7,14 @@
 *Intake → analyze → plan → score → parallel subagents → review → verify → ship.*
 
 ![npm](https://img.shields.io/npm/v/%40azure-id%2Forc?style=for-the-badge&color=cb3837&logo=npm)
-![Version](https://img.shields.io/badge/version-1.4.2-blue.svg?style=for-the-badge)
+![Version](https://img.shields.io/badge/version-1.6.0-blue.svg?style=for-the-badge)
 ![License](https://img.shields.io/badge/license-MIT-green.svg?style=for-the-badge)
 ![Node](https://img.shields.io/badge/node-%3E%3D18-brightgreen.svg?style=for-the-badge)
 ![Claude Code](https://img.shields.io/badge/Claude_Code-Skills-purple.svg?style=for-the-badge)
 ![Dependencies](https://img.shields.io/badge/dependencies-zero-lightgrey.svg?style=for-the-badge)
 ![GitHub stars](https://img.shields.io/github/stars/azure-id/orc?style=for-the-badge&color=yellow)
 
-**Latest: v1.4.2** · updated 2026-09-05 · [full changelog](CHANGELOG.md)
+**Latest: v1.6.0** · updated 2026-09-07 · [full changelog](CHANGELOG.md)
 
 **On npm: [`@azure-id/orc`](https://www.npmjs.com/package/@azure-id/orc)** — `npm i -g @azure-id/orc`
 
@@ -287,6 +287,7 @@ ORC have terminal hook to see: Context Window %, 5 Hour usage %, Weekly usage %
 | **`/orc-challenge`** | Grades a **finished** artifact — a TSD, a PRD, an ADR, an API contract, a README, a module — against a goal **you** state, then **stops and makes you fix it in a different session**. ORC judges, you fix, ORC re-judges: **it never fixes what it judged**, because a session that just wrote the fix would grade its own homework. **And it never guesses what "good" means here.** | [see it](mock-run/orc-challenge.md) |
 | **`/orc-pact`** | The promises your project makes, and which are in doubt right now. Four states, all **computed on read**: HOLDING · DRIFTED · **UNCHECKABLE** (the honest state — never a failure) · BROKEN. It never invents a promise and never retires one for you. | [see it](mock-run/orc-pact.md) |
 | **`/orc-boundary`** | What ORC should **not** try here, and exactly what would change that. EXECUTE · ESCALATE · REFUSE, per area. **A REFUSE always names what would make it a yes.** It gates ORC's own dispatch, never your instructions. | [see it](mock-run/orc-boundary.md) |
+| **`/orc-test`** | **Runs** the test against a running system - yours, or one you were given permission to touch. Sends the requests, writes down what came back, and stops. **The CLI executes and measures; the model designs and interprets.** Three verdicts, and **`unknown` is the honest one**. The OWASP API Top 10 (2023) as a **closed set** where a category it could not measure keeps its slot and never becomes a pass. It never edits the system it is testing. | [see it](mock-run/orc-test.md) |
 | **`/orc-verify`** | Verifies only your git-modified changes: build, tests, diff sanity, findings on a P0–P3 ladder. Read-only. | [see it](templates/skills/orc-verify/examples/verify-mock.md) |
 | **`/orc-aftermath`** | Did what we shipped hold up? Graded from the repository's own future: churn, reverts, deleted tests, broken promises. **Churn is a signal, never a verdict**, and it never names a person. | [see it](mock-run/orc-aftermath.md) |
 | **`/orc-budget`** | What a run costs, in the unit you are billed in. A **token vector** — fresh input, cache write, cache read, output, never blended — shown four ways: tokens, dollars, percent of your 5-hour window, and context risk. Needs a plan, not a sentence. | [see it](mock-run/orc-budget.md) |
@@ -575,7 +576,7 @@ a current audit: [EVAL-REPORT.md](EVAL-REPORT.md).
 **Full history: [CHANGELOG.md](CHANGELOG.md)** — or `orc changelog`, which prints
 only what is newer than the version you have.
 
-### v1.4.2 - the panel that stops reloading, and the fields you could not read back _(2026-09-05)_
+### v1.6.0 - the rule that can finally say no _(2026-09-07)_
 
 **Still on the unscoped `orc` package?** Do this once first - your `orc upgrade`
 is the pre-v0.56.0 one and cannot install itself. Full detail in the CAUTION at
@@ -587,54 +588,66 @@ the top of this file.
 
 **Do not use `npm i -g -f`.** Full detail in v0.56.0 below.
 
-Four more defects in `orc ui` ▸ **CLI Hook Interface**. Three were the same
-defect wearing three faces.
-
-**Staging a change reloaded the whole panel** - every staged op re-entered the
-router, so four endpoints were refetched and everything was rebuilt from a
-skeleton. That made a staged change look like an applied one, and it left the
-open editor showing values from before the change you had just made: you picked
-a shape and nothing moved. Nothing is fetched now until something actually moves
-the disk; a staged op repaints from cache with **zero requests**, keeps your
-scroll position, and the open editor repaints with it - caret and all.
-
-**And the fields could not be read back.** `orc statusline show --json` emitted
-twelve of the twenty-four fields a part can carry, so `case`, `prefix`,
-`min_cols`, `precision` and seven more were written to disk correctly and then
-were invisible to the panel that wrote them. Every field ships now, plus
-`authored`, so a control can say **which values are yours** rather than
-inherited. `--json is not a summary`, found again.
-
-**Six parts per line, not five.** **The dropdown nobody could read** turned out
-to be a panel with no colours at all: `hookui.css` was written against four CSS tokens that do not
-exist, so every one of those declarations was silently dropped.
-
-**The preview is drawn as a terminal now** - a window of the stated width with a
-column ruler - and `orc statusline preview` takes `--theme` and `--glyphs` as a
-**render-only** override, so a new drawer shows the same bar under all four
-colour sets and all seven symbol sets. Picking a colour set from its name is a
-guess; picking it from its picture is a decision.
-
-**And the sixth part still reset the bar to the default**, because the cap was in
-three places: the hook re-checks the compiled file with a cheap shape guard of
-its own, and that guard held its own `5` — and counted spacers, which the
-validator never has. The cap now travels on the lock file and the compiler marks
-each structural item, so one cap and one counting rule serve all three readers.
-
-**And a name you typed sometimes went nowhere.** Seven of the thirty-five shapes
-draw a name; the other twenty-eight ignore it, which is the design — a bar with a
-word in front of it is a different shape. The Name box was offered on all
-thirty-five anyway. Each renderer now publishes the fields it **uses**, and the
-editor greys out what this shape ignores, keeping the slot and naming the shapes
-that do.
-
-**And it still blinked with the network tab empty:** `hkPaint` replaces every
-child, and the stylesheet faded each one in on a stagger. Entrance animations are
-off from the second paint onward.
-
-Full detail in [CHANGELOG.md](CHANGELOG.md).
 
----
+ORC's read discipline has always said the same thing in three places: the main
+session reads to **find**, and dispatches an agent to **understand**. It was
+prose in `_shared/read-ladder.md`, in `/orc-doc` hard rule 0, and on line 21 of
+`/orc-quick` - and **nothing checked any of it**.
+
+`orc-read-gate.js` is a `PreToolUse` hook on `Read` that can refuse. It ships
+**off**, and `off` is byte-identical to not having it - asserted by a test, not
+by intention.
+
+**The threshold is ORC's own number, measured.** The pattern this came from uses
+350 lines, derived from a 10-30 second delegation round trip. ORC's round trip
+is nothing like that: a dispatch measured **p50 76s, p90 188s** (n=125), and one
+real read-only dispatch cost **13,276 tokens** to read a four-line file. At the
+measured 55.2 chars/line across 316 sampled reads, break-even is **~1000 lines**.
+Copying 350 would delegate work whose overhead exceeds its saving.
+
+**`agent_id` is the only discriminator, and that was MEASURED, not assumed.**
+`PreToolUse` **does** fire inside a dispatched subagent - the assumption that
+hooks are session-level was wrong. `session_id` and `transcript_path` are
+**identical** in both contexts, so a gate written against either would block the
+full read an executor must perform before an `Edit`, and a reconstructed
+`old_string` corrupts files. The gate tests for the **presence** of `agent_id`,
+never for the absence of some other key, which asserts nothing.
+
+**Everything it stays silent on, and each one is deliberate:** any read by a
+subagent - outside an open ORC run - a targeted `offset`/`limit` read - a file
+under the threshold - build logs, test results and `.jsonl` that a gate parses
+whole, because a truncated red build reads **green** - and any error at all,
+because **a read gate that throws and blocks a read has broken the tool.**
+
+**A block always names the cheaper path.** A gate that only refuses is a gate
+people switch off. It names the targeted read, the agent dispatch, and the
+config key that turns it down.
+
+**Every `warn` and `block` writes one trace line** (`READ-GATE`), an allow
+writes none. That is affordable here for a structural reason: the gate only acts
+while a run is open, so a trace always exists. `orc doctor` gains
+`read-gate-unwired` and `read-gate-fallback`, both reported **only while the
+feature is armed** - a doctor that warns about the default is one people learn
+to ignore.
+
+**Two config keys**, both on the `SEED_EMPTY` allowlist with an empty `lanes[]`,
+because a hook has no lane and cannot resolve config: `read_gate`
+(`off`|`warn`|`block`, default `off`) and `read_gate_max_lines` (default 1000).
+
+**What this release deliberately does NOT do**, with the reasons recorded so
+nobody re-proposes them: no gate on `Bash` reads (`cat`/`head`/`tail`) - it is
+several times the false-positive surface, and the measurement says the `Read`
+tool is only about 7.5% of the actual read surface here - and no
+`context-reader` slot in `EXTRA_SLOTS`, because at a 1000-line threshold the
+addressable population is five reads across 239 sampled sessions, which is
+infrastructure for nothing.
+
+**An honest note on the measurement.** The audit behind this release found the
+existing prose is largely *working*: median full read is 85 lines, p90 is 288,
+and 46% of reads already use `offset`/`limit` without being told. Oversized
+reads are about **1% of price-weighted main-session ingest** at a generous upper
+bound. This hook is a guardrail on a road most runs already stay on - it is off
+by default for exactly that reason.
 
 ## Requirements