@agentsfleet/orly 0.10.5 → 0.10.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +6 -6
- package/README.md +26 -8
- package/audits/agents-md.md +7 -3
- package/audits/doc-read.sh +97 -16
- package/audits/spec-template.sh +30 -113
- package/audits/spec-template.ts +193 -0
- package/core/operating-model.md +6 -6
- package/dispatch/lifecycle.md +98 -42
- package/dispatch/verify.md +30 -48
- package/dispatch/write_spec.md +56 -82
- package/docs/ORLY_ARCHITECTURE.md +13 -3
- package/docs/RULE_ENFORCEMENT.md +2 -2
- package/docs/TEMPLATE.md +82 -77
- package/docs/VERIFY_TIERS.md +20 -26
- package/llms.txt +17 -12
- package/package.json +1 -1
- package/registry.json +4 -0
- package/skills/orly-spec-new/SKILL.md +3 -1
- package/skills/orly-write-integration-test/SKILL.md +1 -1
- package/skills/orly-write-unit-test/SKILL.md +1 -1
- package/src/cli.ts +14 -16
- package/src/criteria.test.ts +47 -6
- package/src/criteria.ts +30 -49
- package/src/criteria_spec.ts +53 -6
- package/src/criteria_support.ts +5 -4
- package/src/gates.test.ts +2 -2
- package/src/gates_spec.test.ts +40 -1
- package/src/gates_test_support.ts +18 -4
- package/src/install.ts +16 -81
- package/src/render.test.ts +5 -7
- package/src/setup.test.ts +116 -0
- package/src/spec_baseline.test.ts +96 -0
- package/src/spec_template.test.ts +109 -0
- package/src/spec_template_io.test.ts +97 -0
- package/src/validation.ts +9 -0
package/AGENTS.md
CHANGED
|
@@ -168,7 +168,7 @@ Guards fire pre-hoc regardless of lifecycle stage. Override: `<GATE>: SKIPPED pe
|
|
|
168
168
|
| Begin implementation OR branch carries spec changes in `pending/` | CHORE(open): `pending/`→`active/`, `Status: IN_PROGRESS` + `Branch:`, create worktree, commit on feature branch. **No code until 4 steps committed.** |
|
|
169
169
|
| Every commit during implementation | Mark completed Dimensions/Sections `DONE` same commit as the code. |
|
|
170
170
|
| All work complete, before PR | CHORE(close). |
|
|
171
|
-
|
|
|
171
|
+
| Final implementation commit, or explicit request to park a spec | CHORE(close) is mandatory next action; Section commits continue the stream. |
|
|
172
172
|
|
|
173
173
|
---
|
|
174
174
|
|
|
@@ -182,13 +182,13 @@ Non-trivial (full lifecycle) if it: touches >1 file · new abstraction · data m
|
|
|
182
182
|
|
|
183
183
|
**Anchor invariant — `orly gate` proves the boundary.** No PR opens unless every `orly gate pr` criterion is green or carries an `Orly-Override` trailer the user recorded with a reason.
|
|
184
184
|
|
|
185
|
-
**
|
|
185
|
+
**Gate ownership.** `work` runs the declared `conform` command at commit. `verify` runs documentation checks and non-test `verify.*` commands at push; it permits unfinished Sections. `pr` checks branch shape, exact upstream revision, spec completeness, and every declared `verify.*` command itself. `orly gate pr` is the command CHORE(close) runs; a push is never proof that tests ran. Unit and integration baselines are measured before the Pull Request against the recorded comparison revision. Full unit and integration suites run at the PR boundary, not at opening or every push. Integration and memory suites skip only on branches with no code. The authoritative sequence is `dispatch/lifecycle.md`; no spec means only spec criteria skip. A user-surface change without a docs change is red until docs land or an override says why not.
|
|
186
186
|
|
|
187
187
|
**LAND (after merge, or when the user confirms it merged):** pull the default branch, prune the merged worktree + branch, `make down` where defined.
|
|
188
188
|
|
|
189
189
|
### CHORE (open)
|
|
190
190
|
|
|
191
|
-
Spec `pending/`→`active/`; `Status: IN_PROGRESS`; `Branch:` set; **`Test Baseline:`
|
|
191
|
+
Spec `pending/`→`active/`; `Status: IN_PROGRESS`; `Branch:` set; **`Test Baseline:` declared pending; `Baseline revision:` records the full comparison commit** (unit and integration counts plus evidence are due before the Pull Request); committed. Worktree created, CWD inside (verify `pwd` + `git worktree list`), carrying the base checkout's uncommitted work in with it. No code until the 4 steps are committed.
|
|
192
192
|
|
|
193
193
|
### PLAN
|
|
194
194
|
|
|
@@ -197,7 +197,7 @@ Required: one-paragraph goal · explicit assumptions · file/task impact list ·
|
|
|
197
197
|
### EXECUTE
|
|
198
198
|
|
|
199
199
|
Read what the selected dispatch names — **sectioned, not whole-file**: scan headers (`grep -nE "^##+ "`), read the sections the diff shape touches, cite them.
|
|
200
|
-
The full `agentsfleet` trigger map lives in `docs/EXECUTE_DOC_READS.md`. At EXECUTE start read the **Rule-code gloss legend** of `docs/greptile-learnings/RULES.md` plus the sections the spec's "Applicable Rules" names — never the full file. On sub-task shape change, section-scan (`grep -n "^## "`) and re-read only the sections the new shape touches. DOC READ GATE emits one `📖 DOC READ: <path>` proof-line **per triggered document per turn** before the turn's first triggering edit, citing the applied section or a cited skip; auto-mode and prior-turn reads never excuse it. **Record it, don't just claim it:** `bash audits/doc-read.sh log <path
|
|
200
|
+
The full `agentsfleet` trigger map lives in `docs/EXECUTE_DOC_READS.md`. At EXECUTE start read the **Rule-code gloss legend** of `docs/greptile-learnings/RULES.md` plus the sections the spec's "Applicable Rules" names — never the full file. On sub-task shape change, section-scan (`grep -n "^## "`) and re-read only the sections the new shape touches. DOC READ GATE emits one `📖 DOC READ: <path>` proof-line **per triggered document per turn** before the turn's first triggering edit, citing the applied section or a cited skip; auto-mode and prior-turn reads never excuse it. **Record it, don't just claim it:** `bash audits/doc-read.sh log <path> "<section applied>"` — any runtime; Claude Code's Read hook does it for you; pre-commit compares the record to the staged diff. **Log each page as you read it, and cite the section**: a loop writes the same rows, and once did — nine façades in one second, `check` green, violations from a section nobody opened. `check` counts citations and names any bulk second.
|
|
201
201
|
|
|
202
202
|
Edit only approved scope; no opportunistic refactors. Stay in active worktree. Cross-repo writes to `~/Projects/docs/` need explicit per-session ask.
|
|
203
203
|
|
|
@@ -210,8 +210,8 @@ In `agentsfleet` this stage is `make harness-verify`; its output block and end-o
|
|
|
210
210
|
|
|
211
211
|
### VERIFY
|
|
212
212
|
|
|
213
|
-
**
|
|
214
|
-
The `agentsfleet` output block and both cadences live in `docs/VERIFY_TIERS.md`. **FIRST: `/orly-write-unit-test`** audits diff coverage. **LAST: the Test Delta row** compares against the
|
|
213
|
+
**Section proof and final verification.** Follow `dispatch/lifecycle.md` for command timing. Section checks prove the changed Dimensions; the final gate runs the full declared verification set. A package-scoped command never replaces a declared boundary command, and a Section result never satisfies a repository-wide "tests pass" claim.
|
|
214
|
+
The `agentsfleet` output block and both cadences live in `docs/VERIFY_TIERS.md`. **FIRST: `/orly-write-unit-test`** audits diff coverage. **LAST: the Test Delta row** compares against the recorded comparison revision, measured before the Pull Request; zero or negative unit growth on a code-adding diff needs justification or a return to EXECUTE. Paste memory-leak evidence into PR Session Notes or cite its Continuous Integration (CI) URL. After refactors, list newly dead code before removing it.
|
|
215
215
|
|
|
216
216
|
### REVIEW
|
|
217
217
|
|
package/README.md
CHANGED
|
@@ -61,15 +61,33 @@ Works with Claude Code, Codex, OpenCode, and Amp.
|
|
|
61
61
|
Run this inside the repository you want governed.
|
|
62
62
|
|
|
63
63
|
```bash
|
|
64
|
-
|
|
64
|
+
bun add -g @agentsfleet/orly
|
|
65
|
+
orly init
|
|
65
66
|
```
|
|
66
67
|
|
|
67
68
|
orly scans your source, detects your languages, and installs only the rules that apply.
|
|
68
69
|
|
|
69
70
|
> [!TIP]
|
|
70
|
-
> Try `
|
|
71
|
+
> Try `orly init --dry-run` first. It previews the generated rules and changes nothing.
|
|
71
72
|
|
|
72
|
-
|
|
73
|
+
The Git hooks use the installed `orly` executable. Keep it on `PATH`.
|
|
74
|
+
Complete any setup items printed by `init`, then run `orly doctor` and commit the generated files.
|
|
75
|
+
Teammates get the rules on clone and run `orly init` to install their hooks.
|
|
76
|
+
|
|
77
|
+
Declare `conform` and at least one `verify.*` command in `.oracle/orly.json`.
|
|
78
|
+
Orly runs your repository's commands; the verification name does not require a particular language or test runner.
|
|
79
|
+
For a documentation repository whose Makefile provides `test` and `lint`:
|
|
80
|
+
|
|
81
|
+
```json
|
|
82
|
+
"commands": {
|
|
83
|
+
"conform": [["make", "test"]],
|
|
84
|
+
"verify.docs": [["make", "lint"]]
|
|
85
|
+
}
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
Here `test` checks documentation during work, and `lint` validates the site and links before the pull request.
|
|
89
|
+
Markdown and Markdown JSX (MDX) repositories do not need application unit or integration suites.
|
|
90
|
+
`orly doctor` checks configuration and installed files; run the gates to verify the commands themselves.
|
|
73
91
|
|
|
74
92
|
---
|
|
75
93
|
|
|
@@ -120,8 +138,8 @@ Three checkpoints, all running your own declared commands:
|
|
|
120
138
|
| Checkpoint | When | Effect |
|
|
121
139
|
|---|---|---|
|
|
122
140
|
| `pre-commit` | every commit | stops at the first failing check |
|
|
123
|
-
| `pre-push` | every push |
|
|
124
|
-
| `orly gate pr` | opening the Pull Request |
|
|
141
|
+
| `pre-push` | every push | documentation and declared non-test verification checks |
|
|
142
|
+
| `orly gate pr` | opening the Pull Request | runs all declared verification commands and checks the branch and spec |
|
|
125
143
|
|
|
126
144
|
---
|
|
127
145
|
|
|
@@ -144,7 +162,7 @@ flowchart TB
|
|
|
144
162
|
subgraph active["⚙️ docs/v1/active/"]
|
|
145
163
|
direction TB
|
|
146
164
|
|
|
147
|
-
plan["PLAN<br/><small>understand spec ·
|
|
165
|
+
plan["PLAN<br/><small>understand spec · resolve decisions</small>"]
|
|
148
166
|
execute["EXECUTE<br/><small>make the change</small>"]
|
|
149
167
|
conform["CONFORM<br/><small>apply matching rule pages</small>"]
|
|
150
168
|
verify["VERIFY<br/><small>run deterministic gates</small>"]
|
|
@@ -171,7 +189,7 @@ flowchart TB
|
|
|
171
189
|
| Directory | Meaning |
|
|
172
190
|
|---|---|
|
|
173
191
|
| `docs/v1/pending/` | the spec is written and committed on `main` |
|
|
174
|
-
| `docs/v1/active/` | branch cut,
|
|
192
|
+
| `docs/v1/active/` | branch cut, comparison revision recorded, measurement pending; no code until it commits |
|
|
175
193
|
| `docs/v1/done/` | gates green, Pull Request opens |
|
|
176
194
|
|
|
177
195
|
Inside `active/`, work runs through the stages: **PLAN → EXECUTE → CONFORM → VERIFY → REVIEW → DOCUMENT → COMMIT**.
|
|
@@ -216,7 +234,7 @@ The last two files exist because installing a rules file is not the same as deli
|
|
|
216
234
|
| `orly update --with <pack>` | add an opt-in pack, recorded for every clone |
|
|
217
235
|
| `orly gate` | run your declared checks in order, stopping at the first failure |
|
|
218
236
|
| `orly override <criterion> --reason <why>` | record a gate exception as an empty commit that rides into the Pull Request |
|
|
219
|
-
| `orly doctor` |
|
|
237
|
+
| `orly doctor` | check installed files and required command configuration |
|
|
220
238
|
|
|
221
239
|
---
|
|
222
240
|
|
package/audits/agents-md.md
CHANGED
|
@@ -107,7 +107,7 @@ The questionnaire is organised by scenario. Each scenario corresponds to a momen
|
|
|
107
107
|
|---|---|---|
|
|
108
108
|
| 5.1 | Is the agent required to verify CWD is inside the active worktree before resuming (`pwd` + `git worktree list`)? | YES |
|
|
109
109
|
| 5.2 | Must the agent re-read RULES.md when sub-task shape changes (new layer/language/resume after break)? | YES |
|
|
110
|
-
| 5.3 | If the spec is in `active/`,
|
|
110
|
+
| 5.3 | If the spec is in `active/`, does CHORE(close) follow the final implementation commit or an explicit request to park, while Section commits continue implementation? | YES |
|
|
111
111
|
| 5.4 | If unexpected changes appear in files the agent is editing, must the agent stop and ask (not overwrite as stale)? | YES |
|
|
112
112
|
|
|
113
113
|
### Scenario 6 — Conformance and verification lifecycle
|
|
@@ -122,8 +122,11 @@ The questionnaire is organised by scenario. Each scenario corresponds to a momen
|
|
|
122
122
|
| 6.6 | Is at least one declared `verify.integration` run from clean state required per branch before ship-ready (where the repository declares one)? | YES |
|
|
123
123
|
| 6.7 | Are package-scoped runners (`bun run test`, `vitest <file>`, `zig build test` w/o integration) explicitly **not** verification? | YES |
|
|
124
124
|
| 6.8 | Must memleak evidence (last 3 lines verbatim) appear in PR Session Notes when touching `src/http/**` / `src/cmd/serve.zig` / allocator wiring? | YES |
|
|
125
|
-
| 6.9 | Does CHORE(open)
|
|
126
|
-
| 6.
|
|
125
|
+
| 6.9 | Does CHORE(open) DECLARE the `Test Baseline:` line in the spec header while measuring nothing, with the counts from the repository's declared `verify.*` commands landing before the Pull Request (a product pack may name a dedicated counter)? | YES |
|
|
126
|
+
| 6.9a | Does the baseline identify the full comparison commit, with matching run evidence or an isolated checkout, declared unit and integration counts, and a report reference? | YES |
|
|
127
|
+
| 6.9b | Does `orly gate pr` reject missing declared baseline counts, allow reasoned `n/a` only without code or baseline lanes, and run all declared verification commands independently of custom hooks? | YES |
|
|
128
|
+
| 6.9c | Does setup require `conform` and a named `verify.*` command, allowing `verify.docs` without application test suites while reporting incomplete configuration? | YES |
|
|
129
|
+
| 6.10 | Does VERIFY end with a Test Delta row (growth vs the branch point's baseline) plus a lacking-areas verdict, with zero/negative unit delta on a code-adding diff requiring justification or a return to EXECUTE? | YES |
|
|
127
130
|
| 6.11 | Do the lifecycle stage runbooks (CHORE(open/close) checklists, PLAN expansions, deferral quote format, pre-PR gates, LAND) resolve from `dispatch/lifecycle.md`, with AGENTS.md keeping each stage's binding essence? | YES |
|
|
128
131
|
|
|
129
132
|
### Scenario 7 — Review discipline before merge
|
|
@@ -233,6 +236,7 @@ The questionnaire is organised by scenario. Each scenario corresponds to a momen
|
|
|
233
236
|
| 18.1 | Must `gitleaks` pass before any `git commit` / `git push`? | YES |
|
|
234
237
|
| 18.2 | Is the rule "one worktree per active stream — no edits outside, no reads from siblings, merge only after REVIEW and the final commit" preserved? | YES |
|
|
235
238
|
| 18.3 | Are cross-worktree edits explicitly forbidden without explicit user approval? | YES |
|
|
239
|
+
| 18.4 | Does CHORE(open) MOVE the base checkout's uncommitted work into the new worktree — never copy it, naming what came across, and stopping on a conflicting `git stash pop` — so a stream is never cut from a dirty base that then reads as unowned drift? | YES |
|
|
236
240
|
|
|
237
241
|
### Scenario 19 — CONFORM combined audit
|
|
238
242
|
|
package/audits/doc-read.sh
CHANGED
|
@@ -1,14 +1,32 @@
|
|
|
1
1
|
#!/usr/bin/env bash
|
|
2
2
|
# doc-read.sh — turn the 📖 DOC READ proof-line into a set comparison.
|
|
3
3
|
#
|
|
4
|
-
# audits/doc-read.sh log <path> record that a file was read
|
|
5
|
-
# audits/doc-read.sh check
|
|
4
|
+
# audits/doc-read.sh log <path> [section] record that a file was read
|
|
5
|
+
# audits/doc-read.sh check staged source vs recorded reads
|
|
6
6
|
#
|
|
7
7
|
# The DOC READ GATE asks an agent to read the façade its edit triggers, and
|
|
8
8
|
# then asks the same agent whether it did. That is a claim about itself
|
|
9
9
|
# compared against nothing. `log` records the read and `check` compares the
|
|
10
10
|
# record against the façades the staged files trigger.
|
|
11
11
|
#
|
|
12
|
+
# `log` is still an assertion, and a loop can make nine of them in one second.
|
|
13
|
+
# That happened: an agent ran `log` over its whole trigger list before opening
|
|
14
|
+
# most of the pages, `check` reported green, and two rule violations reached the
|
|
15
|
+
# diff from a section it had skipped. Two things answer it, and neither can
|
|
16
|
+
# block, because neither is provable:
|
|
17
|
+
#
|
|
18
|
+
# - The optional `section` argument records WHAT the read applied — "§Error
|
|
19
|
+
# discipline: composed with #[from]". A loop cannot produce that line; only
|
|
20
|
+
# reading can. `check` reports how many triggered façades carry one.
|
|
21
|
+
# - Rows for different façades sharing one timestamp are reported as a bulk
|
|
22
|
+
# assertion. A read hook fires per read and spreads across seconds; a
|
|
23
|
+
# scripted loop lands them together. An agent that genuinely read three
|
|
24
|
+
# short pages inside one second is possible too, which is exactly why this
|
|
25
|
+
# is a number in the output and never a red.
|
|
26
|
+
#
|
|
27
|
+
# Partial mechanisation stated honestly beats a red that means nothing — the
|
|
28
|
+
# same argument the missing-record branch below already makes.
|
|
29
|
+
#
|
|
12
30
|
# NOTHING HERE MAY DEPEND ON AN AGENT RUNTIME. `log` is a shell command any
|
|
13
31
|
# agent can run; a runtime that also has a read hook (Claude Code today) can
|
|
14
32
|
# call it automatically, and a runtime that removes that feature tomorrow
|
|
@@ -37,7 +55,12 @@ source "$HERE/rule-ledger-lib.sh"
|
|
|
37
55
|
MODE_LOG="log"
|
|
38
56
|
MODE_CHECK="check"
|
|
39
57
|
LOG_RELATIVE_PATH="orly/doc-reads.jsonl"
|
|
40
|
-
USAGE="usage: $0 log <path> | $0 check"
|
|
58
|
+
USAGE="usage: $0 log <path> [section] | $0 check"
|
|
59
|
+
|
|
60
|
+
# How many façades recorded in one second read as a loop rather than as reading.
|
|
61
|
+
# Two is a plausible pair of short pages; three in the same second is a list
|
|
62
|
+
# being walked.
|
|
63
|
+
BULK_ASSERTION_THRESHOLD=3
|
|
41
64
|
|
|
42
65
|
if [[ -t 1 ]]; then G=$'\033[32m'; R=$'\033[31m'; Y=$'\033[33m'; X=$'\033[0m'
|
|
43
66
|
else G=''; R=''; Y=''; X=''; fi
|
|
@@ -65,7 +88,7 @@ repo_relative() {
|
|
|
65
88
|
# invocation is why two agents in two sessions cannot corrupt each other: the
|
|
66
89
|
# order rows land in does not change which paths the set contains.
|
|
67
90
|
run_log() {
|
|
68
|
-
local path="$1" relative log blob
|
|
91
|
+
local path="$1" section="${2:-}" relative log blob
|
|
69
92
|
relative="$(repo_relative "$path")" || return 0
|
|
70
93
|
[ -n "$relative" ] || return 0
|
|
71
94
|
log="$(read_log_path)" || return 0
|
|
@@ -75,7 +98,11 @@ run_log() {
|
|
|
75
98
|
# Recording it would let two failures compare equal and validate a read of a
|
|
76
99
|
# file that is no longer there.
|
|
77
100
|
[ -n "$blob" ] || return 0
|
|
78
|
-
|
|
101
|
+
# The section rides the SAME row rather than a second file: a citation that
|
|
102
|
+
# could be written without a read beside it would be a third assertion to
|
|
103
|
+
# reconcile, and the pair is what carries meaning.
|
|
104
|
+
printf '{"ts":%s,"path":"%s","blob":"%s","section":"%s"}\n' \
|
|
105
|
+
"$(date +%s)" "$(json_escape "$relative")" "$blob" "$(json_escape "$section")" >> "$log"
|
|
79
106
|
return 0
|
|
80
107
|
}
|
|
81
108
|
|
|
@@ -115,22 +142,43 @@ json_unescape() {
|
|
|
115
142
|
printf '%s' "$1" | sed -e 's/\\"/"/g' -e 's/\\\\/\\/g'
|
|
116
143
|
}
|
|
117
144
|
|
|
118
|
-
# Paths whose recorded read saw the content that is on disk now
|
|
119
|
-
#
|
|
120
|
-
# else, so it does not count — and
|
|
121
|
-
# no blob, cannot prove anything
|
|
145
|
+
# Paths whose recorded read saw the content that is on disk now, one
|
|
146
|
+
# `path<TAB>ts<TAB>section` row each. A row written against an older version of
|
|
147
|
+
# the document proves the agent read something else, so it does not count — and
|
|
148
|
+
# rows from before this milestone, which carry no blob, cannot prove anything
|
|
149
|
+
# and are ignored.
|
|
150
|
+
#
|
|
151
|
+
# The path is cut at the first `","blob":"` rather than by stripping a fixed
|
|
152
|
+
# tail: the row gained a trailing `section` field, and a suffix strip that
|
|
153
|
+
# spelled the old shape would have silently yielded a path that matches nothing
|
|
154
|
+
# — reading as "never read" for every document.
|
|
122
155
|
current_reads() {
|
|
123
|
-
local log="$1" line path blob
|
|
156
|
+
local log="$1" line path blob ts section
|
|
124
157
|
while IFS= read -r line; do
|
|
125
158
|
case "$line" in *'"blob":"'*) ;; *) continue ;; esac
|
|
126
|
-
blob="${line
|
|
159
|
+
blob="${line#*\"blob\":\"}"; blob="${blob%%\"*}"
|
|
127
160
|
[ -n "$blob" ] || continue
|
|
128
|
-
path="${line#*\"path\":\"}"; path="${path
|
|
161
|
+
path="${line#*\"path\":\"}"; path="${path%%\",\"blob\":\"*}"
|
|
129
162
|
path="$(json_unescape "$path")"
|
|
130
|
-
|
|
163
|
+
ts="${line#*\"ts\":}"; ts="${ts%%,*}"
|
|
164
|
+
section=""
|
|
165
|
+
case "$line" in
|
|
166
|
+
*'"section":"'*) section="${line#*\"section\":\"}"; section="${section%%\"*}" ;;
|
|
167
|
+
esac
|
|
168
|
+
[ "$blob" = "$(content_hash "$path")" ] &&
|
|
169
|
+
printf '%s\t%s\t%s\n' "$path" "$ts" "$(json_unescape "$section")"
|
|
131
170
|
done < "$log"
|
|
132
171
|
}
|
|
133
172
|
|
|
173
|
+
# Timestamps carrying BULK_ASSERTION_THRESHOLD or more distinct façades, one
|
|
174
|
+
# `ts<TAB>count` row each. See the header: reported, never blocking.
|
|
175
|
+
bulk_seconds() {
|
|
176
|
+
cut -f1,2 "$1" | sort -u | cut -f2 | sort | uniq -c |
|
|
177
|
+
while read -r count ts; do
|
|
178
|
+
[ "$count" -ge "$BULK_ASSERTION_THRESHOLD" ] && printf '%s\t%s\n' "$ts" "$count"
|
|
179
|
+
done
|
|
180
|
+
}
|
|
181
|
+
|
|
134
182
|
# File-scope so the EXIT trap can still see them: a trap fires after a
|
|
135
183
|
# function's locals are gone, and `set -u` turns that into a spurious error on
|
|
136
184
|
# the way out of a clean run.
|
|
@@ -164,18 +212,51 @@ run_check() {
|
|
|
164
212
|
current_reads "$log" | sort -u > "$recorded"
|
|
165
213
|
|
|
166
214
|
while IFS= read -r page; do
|
|
167
|
-
grep -
|
|
215
|
+
grep -qF "$(printf '%s\t' "$page")" "$recorded" && continue
|
|
168
216
|
printf ' %s🔴%s DOC READ: %s triggered by the staged diff, not read at its current content\n' "$R" "$X" "$page"
|
|
169
217
|
unread=1
|
|
170
218
|
done < "$expected"
|
|
171
219
|
[ "$unread" -eq 0 ] && printf ' %s🟢%s DOC READ: every triggered façade was read at its current content\n' "$G" "$X"
|
|
220
|
+
report_citations "$expected" "$recorded"
|
|
221
|
+
report_bulk "$recorded"
|
|
172
222
|
return "$unread"
|
|
173
223
|
}
|
|
174
224
|
|
|
225
|
+
# How many of the triggered façades carry a cited section. A record with one
|
|
226
|
+
# says what the read APPLIED; a record without says only that a read happened,
|
|
227
|
+
# which is the claim a loop can also make.
|
|
228
|
+
report_citations() {
|
|
229
|
+
local expected="$1" recorded="$2" page total=0 cited=0 section
|
|
230
|
+
while IFS= read -r page; do
|
|
231
|
+
total=$((total + 1))
|
|
232
|
+
section="$(grep -F "$(printf '%s\t' "$page")" "$recorded" | cut -f3 | grep -c '[^[:space:]]')"
|
|
233
|
+
[ "$section" -gt 0 ] && cited=$((cited + 1))
|
|
234
|
+
done < "$expected"
|
|
235
|
+
[ "$total" -eq 0 ] && return 0
|
|
236
|
+
if [ "$cited" -eq "$total" ]; then
|
|
237
|
+
printf ' %s🟢%s DOC READ: all %s triggered façade(s) cite the section applied\n' "$G" "$X" "$total"
|
|
238
|
+
return 0
|
|
239
|
+
fi
|
|
240
|
+
printf ' %s🟠%s DOC READ: %s of %s triggered façade(s) cite a section; the rest record\n' \
|
|
241
|
+
"$Y" "$X" "$cited" "$total"
|
|
242
|
+
printf ' only that a read happened — pass it: doc-read.sh log <path> "<section applied>"\n'
|
|
243
|
+
}
|
|
244
|
+
|
|
245
|
+
# Says when rows look like a list being walked rather than pages being read.
|
|
246
|
+
report_bulk() {
|
|
247
|
+
local recorded="$1" ts count
|
|
248
|
+
while IFS="$(printf '\t')" read -r ts count; do
|
|
249
|
+
[ -n "$ts" ] || continue
|
|
250
|
+
printf ' %s🟠%s DOC READ: %s façades recorded in one second (ts %s) — that is a bulk\n' \
|
|
251
|
+
"$Y" "$X" "$count" "$ts"
|
|
252
|
+
printf ' assertion, not %s reads; log each at the moment it is read\n' "$count"
|
|
253
|
+
done < <(bulk_seconds "$recorded")
|
|
254
|
+
}
|
|
255
|
+
|
|
175
256
|
case "${1:-}" in
|
|
176
257
|
"$MODE_LOG")
|
|
177
|
-
[ "$#" -eq 2 ] || { printf '%s\n' "$USAGE" >&2; exit 2; }
|
|
178
|
-
run_log "$2"
|
|
258
|
+
{ [ "$#" -eq 2 ] || [ "$#" -eq 3 ]; } || { printf '%s\n' "$USAGE" >&2; exit 2; }
|
|
259
|
+
run_log "$2" "${3:-}"
|
|
179
260
|
;;
|
|
180
261
|
"$MODE_CHECK")
|
|
181
262
|
[ "$#" -eq 1 ] || { printf '%s\n' "$USAGE" >&2; exit 2; }
|
package/audits/spec-template.sh
CHANGED
|
@@ -8,12 +8,12 @@
|
|
|
8
8
|
# 2. REQUIRED-PRESENT + NO-PLACEHOLDER (positive space) — the determinism
|
|
9
9
|
# sections the agent-facing template mandates (PR Intent, Applicable Gates,
|
|
10
10
|
# Prior-Art, Metrics, Decomposition, tiered Test Spec, Product Clarity,
|
|
11
|
-
# Discovery, …) must EXIST and
|
|
11
|
+
# Discovery, …) must EXIST and carry content: no known unfilled slots and
|
|
12
12
|
# no surviving "tpl:" guidance comments (template fill grammar). A spec
|
|
13
13
|
# that omits them forces the executing agent to guess intent. This is the
|
|
14
14
|
# half that makes a spec "built for the agent".
|
|
15
15
|
# 2b. DECLARED-COMMAND PARITY — a pending/active spec's Acceptance Rubric
|
|
16
|
-
# must quote the repository's declared conform + verify
|
|
16
|
+
# must quote the repository's declared conform + applicable verify.* commands
|
|
17
17
|
# (.oracle/orly.json) verbatim: the rubric and `orly gate` grade one
|
|
18
18
|
# boundary. Staged/file scope only; done/ specs stay historical.
|
|
19
19
|
#
|
|
@@ -25,7 +25,7 @@
|
|
|
25
25
|
# --all (default) pending+active specs only — current/in-flight work
|
|
26
26
|
# --include-done adds done/ specs to the scan (one-time sweep tool)
|
|
27
27
|
#
|
|
28
|
-
# Family 2 runs
|
|
28
|
+
# Family 2 runs in --staged and --file scope — the spec being authored/edited right
|
|
29
29
|
# now, which is exactly the agent's own output. The bulk scans (--all /
|
|
30
30
|
# --include-done) run Family 1 only, so they behave identically over the whole
|
|
31
31
|
# corpus and never break an existing spec. No legacy carve-out, no heuristics:
|
|
@@ -37,6 +37,7 @@
|
|
|
37
37
|
set -euo pipefail
|
|
38
38
|
|
|
39
39
|
MODE="${1:-${SCOPE:-all}}"
|
|
40
|
+
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
40
41
|
ROOT="$(git rev-parse --show-toplevel)"
|
|
41
42
|
cd "$ROOT"
|
|
42
43
|
|
|
@@ -116,93 +117,8 @@ PATTERNS=(
|
|
|
116
117
|
# Carve-outs — lines that look prohibited but are legitimate. Skip any match.
|
|
117
118
|
SAFE_LINES_RE='^\*\*Date:\*\*|^\s*```|<!--|^\s*//|"[0-9]+\s*(h|hour|day|min)"'
|
|
118
119
|
|
|
119
|
-
# ---------------------------------------------------------------------------
|
|
120
|
-
# Family 2 — Required determinism sections (agent-facing docs/TEMPLATE.md).
|
|
121
|
-
# Each entry: <heading ERE>:<human description>
|
|
122
|
-
# ---------------------------------------------------------------------------
|
|
123
|
-
REQUIRED_SECTIONS=(
|
|
124
|
-
'^#+ .*PR Intent:PR Intent & comprehension handshake'
|
|
125
|
-
'^#+ .*Applicable Rules:Applicable Rules'
|
|
126
|
-
'^#+ .*Applicable Gates:Applicable Gates'
|
|
127
|
-
'^#+ .*Overview:Overview'
|
|
128
|
-
'^#+ .*(Prior-Art|Reference Implementation):Prior-Art / Reference Implementations'
|
|
129
|
-
'^#+ .*Files Changed:Files Changed (blast radius)'
|
|
130
|
-
'^#+ .*(Decomposition|Alternatives):Decomposition & alternatives (patch vs refactor)'
|
|
131
|
-
'^#+ .*Sections:Sections (implementation slices)'
|
|
132
|
-
'^#+ .*Metrics.*Observability:Metrics & Observability'
|
|
133
|
-
'^#+ .*Interfaces:Interfaces'
|
|
134
|
-
'^#+ .*Failure Modes:Failure Modes'
|
|
135
|
-
'^#+ .*Invariants:Invariants'
|
|
136
|
-
'^#+ .*Test Specification:Test Specification'
|
|
137
|
-
'^#+ .*Acceptance (Criteria|Rubric):Acceptance Rubric (legacy heading Acceptance Criteria accepted)'
|
|
138
|
-
'^#+ .*Product Clarity:Product Clarity (authoring record)'
|
|
139
|
-
'^#+ .*Discovery:Discovery (consult log)'
|
|
140
|
-
)
|
|
141
|
-
|
|
142
|
-
# Template residue — strings that exist ONLY in the unfilled template. Their
|
|
143
|
-
# survival in a pending/active spec means the section was never filled. The
|
|
144
|
-
# "<!-- tpl:" marker is the template's fill grammar: every guidance comment is
|
|
145
|
-
# deleted at authoring, so any survivor is residue by construction.
|
|
146
|
-
PLACEHOLDER_SENTINELS=(
|
|
147
|
-
'<!-- tpl:'
|
|
148
|
-
'Title — testable, not vague'
|
|
149
|
-
'one-line reason}'
|
|
150
|
-
'Slice title}'
|
|
151
|
-
'path/to/file.ext'
|
|
152
|
-
'test_<short_name>'
|
|
153
|
-
'why this is the right pattern to mirror'
|
|
154
|
-
'{outcome the user can observe}'
|
|
155
|
-
'{exit 0 / substring / 0 matches}'
|
|
156
|
-
'path/to/spec_or_doc.md'
|
|
157
|
-
'{path/to/old_file.ext}'
|
|
158
|
-
'{old_symbol}'
|
|
159
|
-
'{one sentence that could be a test name}'
|
|
160
|
-
'{smallest verifiable behaviour}'
|
|
161
|
-
'{one-line behavioural claim}'
|
|
162
|
-
'{conform command from .oracle/orly.json'
|
|
163
|
-
'{verify.unit command from .oracle/orly.json'
|
|
164
|
-
"{gate from this repository's AGENTS.md dispatch index}"
|
|
165
|
-
)
|
|
166
|
-
|
|
167
|
-
# ---------------------------------------------------------------------------
|
|
168
|
-
# Family 2b — declared-command parity (TEMPLATE.md "Command source rule").
|
|
169
|
-
# A pending/active spec's Acceptance Rubric must quote the repository's
|
|
170
|
-
# declared conform + verify.unit commands VERBATIM — the same set `orly gate`
|
|
171
|
-
# runs — so the spec's ship gate and the mechanical PR gate grade one
|
|
172
|
-
# boundary. Reads .oracle/orly.json with bun (orly's own runtime). Done specs
|
|
173
|
-
# are historical records and are never retrofitted. No config, no bun, or no
|
|
174
|
-
# declared commands → NOTE and skip; this check never guesses.
|
|
175
|
-
# ---------------------------------------------------------------------------
|
|
176
|
-
scan_commands() {
|
|
177
|
-
local spec="$1" cfg=".oracle/orly.json" cmds
|
|
178
|
-
case "$spec" in
|
|
179
|
-
*/pending/*|*/active/*) ;;
|
|
180
|
-
*) return 0 ;;
|
|
181
|
-
esac
|
|
182
|
-
[[ -f "$cfg" ]] || { note "$spec — no $cfg; declared-command parity skipped"; return 0; }
|
|
183
|
-
command -v bun >/dev/null 2>&1 || { note "$spec — bun unavailable; declared-command parity skipped"; return 0; }
|
|
184
|
-
cmds="$(bun -e '
|
|
185
|
-
const cfg = await Bun.file(".oracle/orly.json").json().catch(() => null);
|
|
186
|
-
const commands = cfg && typeof cfg.commands === "object" && cfg.commands !== null ? cfg.commands : {};
|
|
187
|
-
for (const key of ["conform", "verify.unit"]) {
|
|
188
|
-
for (const argv of Array.isArray(commands[key]) ? commands[key] : []) {
|
|
189
|
-
if (Array.isArray(argv) && argv.length > 0) console.log(argv.join(" "));
|
|
190
|
-
}
|
|
191
|
-
}' 2>/dev/null || true)"
|
|
192
|
-
[[ -n "$cmds" ]] || { note "$spec — no declared conform/verify.unit commands; parity skipped"; return 0; }
|
|
193
|
-
local miss=0 cmd
|
|
194
|
-
while IFS= read -r cmd; do
|
|
195
|
-
[[ -z "$cmd" ]] && continue
|
|
196
|
-
grep -qF "$cmd" "$spec" && continue
|
|
197
|
-
fail "$spec — rubric misses declared command verbatim: \"$cmd\" (.oracle/orly.json — the set orly gate runs)"
|
|
198
|
-
miss=$((miss + 1))
|
|
199
|
-
done <<<"$cmds"
|
|
200
|
-
if [[ $miss -eq 0 ]]; then ok "$spec — rubric quotes the declared conform/verify.unit commands"; fi
|
|
201
|
-
return 0
|
|
202
|
-
}
|
|
203
|
-
|
|
204
120
|
scan_spec() {
|
|
205
|
-
local spec="$1"
|
|
121
|
+
local spec="$1" content_file="$2"
|
|
206
122
|
local hits=0
|
|
207
123
|
local pattern desc severity line content
|
|
208
124
|
while IFS=: read -r severity desc pattern; do
|
|
@@ -221,36 +137,37 @@ scan_spec() {
|
|
|
221
137
|
note "$spec:$line — $desc"
|
|
222
138
|
fi
|
|
223
139
|
hits=$((hits + 1))
|
|
224
|
-
done < <(grep -nE "$pattern" "$
|
|
140
|
+
done < <(grep -nE "$pattern" "$content_file" 2>/dev/null || true)
|
|
225
141
|
done < <(printf '%s\n' "${PATTERNS[@]}")
|
|
226
142
|
if [[ $hits -eq 0 ]]; then ok "$spec — no prohibited patterns"; fi
|
|
227
143
|
return 0
|
|
228
144
|
}
|
|
229
145
|
|
|
230
|
-
#
|
|
231
|
-
|
|
232
|
-
|
|
233
|
-
local spec="$1"
|
|
234
|
-
local miss=0 entry sec desc sentinel
|
|
235
|
-
for entry in "${REQUIRED_SECTIONS[@]}"; do
|
|
236
|
-
sec="${entry%%:*}"; desc="${entry#*:}"
|
|
237
|
-
grep -qE "$sec" "$spec" && continue
|
|
238
|
-
fail "$spec — missing required section: $desc"
|
|
239
|
-
miss=$((miss + 1))
|
|
240
|
-
done
|
|
241
|
-
for sentinel in "${PLACEHOLDER_SENTINELS[@]}"; do
|
|
242
|
-
grep -qF "$sentinel" "$spec" || continue
|
|
243
|
-
fail "$spec — unfilled template placeholder: \"$sentinel\""
|
|
244
|
-
miss=$((miss + 1))
|
|
245
|
-
done
|
|
246
|
-
if [[ $miss -eq 0 ]]; then ok "$spec — required sections present, no placeholders"; fi
|
|
247
|
-
return 0
|
|
248
|
-
}
|
|
249
|
-
|
|
146
|
+
# Staged checks grade the index, including files removed only from the worktree.
|
|
147
|
+
CONTENT_DIR="$(mktemp -d)"
|
|
148
|
+
trap 'rm -rf "$CONTENT_DIR"' EXIT
|
|
250
149
|
for spec in "${SPECS[@]}"; do
|
|
251
|
-
|
|
252
|
-
|
|
253
|
-
|
|
150
|
+
content_file="$spec"
|
|
151
|
+
case "$MODE" in
|
|
152
|
+
--staged|staged)
|
|
153
|
+
content_file="$CONTENT_DIR/spec.md"
|
|
154
|
+
if ! git show ":$spec" > "$content_file"; then
|
|
155
|
+
fail "$spec — cannot read staged content"
|
|
156
|
+
continue
|
|
157
|
+
fi
|
|
158
|
+
;;
|
|
159
|
+
esac
|
|
160
|
+
[[ -f "$content_file" ]] || { fail "$spec — cannot read spec"; continue; }
|
|
161
|
+
scan_spec "$spec" "$content_file"
|
|
162
|
+
case "$MODE" in
|
|
163
|
+
--staged|staged|--file|file)
|
|
164
|
+
if ! command -v bun >/dev/null 2>&1; then
|
|
165
|
+
fail "$spec — Bun is required for spec readiness checks"
|
|
166
|
+
elif ! bun "$SCRIPT_DIR/spec-template.ts" "$spec" "$content_file" "$MODE"; then
|
|
167
|
+
FAIL=1
|
|
168
|
+
fi
|
|
169
|
+
;;
|
|
170
|
+
esac
|
|
254
171
|
done
|
|
255
172
|
|
|
256
173
|
if [[ $FAIL -ne 0 ]]; then
|