@agentsfleet/orly 0.6.8 → 0.7.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.githooks/pre-commit +5 -1
- package/audits/agents-md.md +2 -2
- package/audits/doc-read.sh +14 -12
- package/audits/logging.sh +164 -11
- package/audits/rule-ledger-lib.sh +45 -1
- package/audits/rule-ledger.sh +15 -3
- package/dispatch/edit_rules.sh +41 -0
- package/dispatch/write_any.md +35 -5
- package/dispatch/write_go.md +2 -0
- package/dispatch/write_mdx.md +2 -0
- package/dispatch/write_python.md +2 -0
- package/dispatch/write_rust.md +2 -0
- package/dispatch/write_shell.md +2 -0
- package/docs/EXECUTE_DOC_READS.md +3 -0
- package/docs/LOGGING_STANDARD.md +60 -5
- package/docs/ORLY_ARCHITECTURE.md +19 -0
- package/docs/RULE_ENFORCEMENT.md +3 -3
- package/docs/greptile-learnings/RULES.md +7 -0
- package/package.json +1 -1
- package/packs/language/go/rules.md +2 -0
- package/packs/language/mdx/rules.md +2 -0
- package/packs/language/python/rules.md +2 -0
- package/packs/language/rust/rules.md +130 -0
- package/packs/language/shell/rules.md +2 -0
- package/registry.json +4 -0
- package/skills/orly-babysit-prs/SKILL.md +6 -0
- package/skills/orly-spec-new/SKILL.md +6 -0
- package/skills/orly-write-integration-test/SKILL.md +6 -0
- package/skills/orly-write-unit-test/SKILL.md +6 -0
- package/src/cli.test.ts +83 -2
- package/src/cli.ts +62 -18
- package/src/install.test.ts +3 -2
- package/src/install.ts +2 -2
- package/src/telemetry.test.ts +308 -0
- package/src/telemetry.ts +350 -0
- package/src/telemetry_sync.ts +134 -0
package/docs/LOGGING_STANDARD.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# LOGGING_STANDARD — wire format and discipline for structured logs
|
|
2
2
|
|
|
3
|
-
The contract between code that emits log records and the systems (humans, collectors, dashboards) that consume them. Cross-language: applies to every Zig, TypeScript, JavaScript, and shell call site that emits to `stderr`/`stdout`.
|
|
3
|
+
The contract between code that emits log records and the systems (humans, collectors, dashboards) that consume them. Cross-language: applies to every Zig, Rust, TypeScript, JavaScript, and shell call site that emits to `stderr`/`stdout`.
|
|
4
4
|
|
|
5
5
|
Pre-design rules, decisive defaults, and the anti-patterns each rule exists to prevent. Every rule has a one-line "why" so you can judge edge cases.
|
|
6
6
|
|
|
@@ -9,12 +9,13 @@ Pre-design rules, decisive defaults, and the anti-patterns each rule exists to p
|
|
|
9
9
|
Triggers on every `Edit`/`Write` that adds, removes, or changes a log emit:
|
|
10
10
|
|
|
11
11
|
- `*.zig` outside `vendor/`/`third_party/`/`.zig-cache/` — `std.log.*`, `std.debug.print`, `std.io.getStdErr().writer().print`, any helper in `src/lib/logging/`.
|
|
12
|
+
- `*.rs` under `rustd/` outside test and benchmark scopes — `tracing::{error,warn,info,debug,trace}!`, `println!`, `eprintln!`, and `dbg!`.
|
|
12
13
|
- `*.ts`/`*.tsx`/`*.js`/`*.jsx` outside `vendor/`/`node_modules/` — `console.*`, custom logger calls.
|
|
13
14
|
- `*.sh` outside generated directories — `echo`, `printf` to `&2`.
|
|
14
15
|
|
|
15
16
|
Out of scope (explicitly):
|
|
16
17
|
|
|
17
|
-
- Test-only diagnostic prints inside `*_test.zig`, `*.test.ts`, `*.spec.ts`. Tests render to humans, not collectors; gates ignore.
|
|
18
|
+
- Test-only diagnostic prints inside `*_test.zig`, Rust `tests/` and `benches/` trees or `#[cfg(test)]` items, `*.test.ts`, and `*.spec.ts`. Tests render to humans, not collectors; gates ignore.
|
|
18
19
|
- Build/release scripts — the harness's own `audits/*`, `audits/release-*`, and any governance-repo tooling — toolchain output, not application logs.
|
|
19
20
|
- Generated framework noise (Next.js startup banners, Bun runtime warnings) — out of our control.
|
|
20
21
|
|
|
@@ -123,6 +124,12 @@ a per-chunk callback: these flood collectors and drown the records that matter.
|
|
|
123
124
|
`debug` is not a lesser level, it is the level with a volume switch. A sweep that
|
|
124
125
|
reaps 4,000 workspaces logs each one at `debug` and its outcome at `info`.
|
|
125
126
|
|
|
127
|
+
Boundary pairing and per-iteration severity both apply to hot polls. A lease poll
|
|
128
|
+
that crosses Redis and Postgres on every iteration still emits its complete
|
|
129
|
+
`_started` and `_completed`/`_failed` pair, but emits both records at `debug`.
|
|
130
|
+
The pair satisfies rule 1; the level satisfies rule 3. A coarser poll-loop start,
|
|
131
|
+
stop, or aggregate outcome may remain `info`.
|
|
132
|
+
|
|
126
133
|
### `err` and `warn` are unconditional
|
|
127
134
|
|
|
128
135
|
> [UNENFORCED → "always emitted" is a runtime filter-configuration property; nothing greps for a suppressed err/warn path]
|
|
@@ -288,6 +295,48 @@ class AgentError extends Error {
|
|
|
288
295
|
|
|
289
296
|
Throw-style modules `throw new AgentError({...})`. Result-style modules return `{ ok: false, error: new AgentError({...}) }`. Render path picks human or JSON based on the runtime mode.
|
|
290
297
|
|
|
298
|
+
## §8A · Per-language binding — Rust (`rustd`)
|
|
299
|
+
|
|
300
|
+
> [DETERMINISTIC → LOG]
|
|
301
|
+
|
|
302
|
+
Rust uses the existing `tracing` crate and emits the same logical fields as §3.
|
|
303
|
+
Every production emit supplies an `event` field, explicitly or through tracing's
|
|
304
|
+
field shorthand, whose value is a snake_case `verb_noun`. Fields are structured
|
|
305
|
+
arguments, never positional message formatting.
|
|
306
|
+
|
|
307
|
+
> [JUDGMENT → RUST-STYLE]
|
|
308
|
+
|
|
309
|
+
Every `warn` or `error` that maps to a registry code supplies
|
|
310
|
+
`error_code = code` per §5. Hoist field expressions into locals before the macro
|
|
311
|
+
call. The log bridge duplicates field expressions, and LLVM coverage
|
|
312
|
+
instrumentation scores the dead copy; locals preserve evaluation and coverage
|
|
313
|
+
behavior. A trailing string message is §3's optional `msg` and is always the
|
|
314
|
+
last macro argument. Prefer fields alone when they already explain the event.
|
|
315
|
+
|
|
316
|
+
```rust
|
|
317
|
+
let code = error_code::INTERNAL_DB_QUERY.as_str();
|
|
318
|
+
let id = runner.as_str();
|
|
319
|
+
let reason = error.to_string();
|
|
320
|
+
let event = "runner_lease_failed";
|
|
321
|
+
tracing::warn!(error_code = code, runner_id = id, reason, event);
|
|
322
|
+
```
|
|
323
|
+
|
|
324
|
+
> [JUDGMENT → EVENT-COMPAT]
|
|
325
|
+
|
|
326
|
+
Ports preserve existing event-name bytes by default. A Rust replacement for a
|
|
327
|
+
Zig operation keeps the Zig event spelling so dashboards and alerts continue to
|
|
328
|
+
match. Renaming is a separate, intentional observability migration.
|
|
329
|
+
|
|
330
|
+
> [DETERMINISTIC → LOG]
|
|
331
|
+
|
|
332
|
+
**Anti-patterns flagged by `logging.sh`:**
|
|
333
|
+
|
|
334
|
+
| Pattern | Why banned | Fix |
|
|
335
|
+
|---|---|---|
|
|
336
|
+
| `println!`, `eprintln!`, or `dbg!` in non-test source | No level, event, or structured fields. | Convert to a `tracing` emit or delete. |
|
|
337
|
+
| `tracing::<level>!` without `event = ...` | Drops the stable §3 event key. | Add the byte-stable snake_case event field. |
|
|
338
|
+
| Positional formatting in a `tracing` message | Hides values inside text and defeats field queries. | Hoist values and emit named fields. |
|
|
339
|
+
|
|
291
340
|
## §9 · Pretty-printer (dev-only render variant)
|
|
292
341
|
|
|
293
342
|
In production: every record on the wire is logfmt. No exceptions.
|
|
@@ -330,10 +379,10 @@ Failure modes the audit script and reviewer must close. These are **not aspirati
|
|
|
330
379
|
|
|
331
380
|
| # | Rationalization | Closure |
|
|
332
381
|
|---|---|---|
|
|
333
|
-
| L1 | "Temporary debug print, I'll remove later" | `logging.sh` greps `std.debug.print` and `console.log` / `console.debug` / `console.info` in non-test source unconditionally. Found in commit → gate fails. No "temporary" carve-out. |
|
|
382
|
+
| L1 | "Temporary debug print, I'll remove later" | `logging.sh` greps `std.debug.print`, Rust `println!` / `eprintln!` / `dbg!`, and `console.log` / `console.debug` / `console.info` in non-test source unconditionally. Found in commit → gate fails. No "temporary" carve-out. |
|
|
334
383
|
| L2 | "`std.log.scoped` is fine, `obs.scoped` is just a wrapper" | `std.log.scoped` is **forbidden** in `src/**/*.zig` outside `src/lib/logging/`. Only `obs.scoped` is callable. Audit flags every `std.log.` call site. |
|
|
335
384
|
| L3 | "I added `error_code=UZ-NEW-001` — registry entry coming next commit" | The registry entry **must land in the same commit** as the first reference. `error-codes.sh` runs against the staged diff; missing entry = blocking. |
|
|
336
|
-
| L4 | "This per-iteration event matters for debugging — `info`-level" | Per-iteration and per-row paths are `debug`,
|
|
385
|
+
| L4 | "This per-iteration event matters for debugging — `info`-level" | Per-iteration and per-row paths are `debug`, including mandatory boundary pairs on hot polls. `debug` is the level with a volume switch. This closes the ONLY `info` prohibition — there is no allow-list of event names and none may be reintroduced (§4). The inverse rationalization is closed too: "this operation is obviously fine, no need to log it" does not survive §4 rule 1, which requires a `_started`/`_completed`\|`_failed` pair on every boundary-crossing operation. Reviewer checks the pair and the loop, never the spelling. |
|
|
337
386
|
| L5 | "Operator needs the full stack trace in `msg=`" | `msg=` capped at 300 chars; total fields per record capped at 15. Stack traces emit as a separate `event=stack_trace` record at `debug` level, correlated by `correlation_id`, not stuffed into `msg`. |
|
|
338
387
|
| L6 | "Embedded newlines because I copy-pasted output" | Audit greps for raw newline byte inside quoted logfmt values. Must be `\n` literal (two chars). |
|
|
339
388
|
| L7 | "Auto-mode is on, the gate block is ceremony" | **Auto-mode does NOT cover gate skips.** Skip without an explicit user-given override = automatic violation. No size threshold lets an edit bypass the gate. |
|
|
@@ -348,6 +397,9 @@ These are enforced by `logging.sh` (mechanical) and the dispatch façade (`dispa
|
|
|
348
397
|
|---|---|
|
|
349
398
|
| Free-form English log lines | Not greppable, not parseable, no event tag. |
|
|
350
399
|
| `std.debug.print` outside tests | No level, no scope, no fields — dev debugging that escaped to main. |
|
|
400
|
+
| Rust `println!`, `eprintln!`, or `dbg!` outside tests | No level, event, or structured fields — dev diagnostics that escaped to main. |
|
|
401
|
+
| Rust `tracing` emit without `event` | Drops the stable event key required by §3. |
|
|
402
|
+
| Positional formatting in a Rust `tracing` emit | Values become opaque message text instead of queryable fields. |
|
|
351
403
|
| `console.log` in `agentsfleet` source | Bypasses logger, breaks `--json` mode, violates `dispatch/write_ts_adhere_bun.md` §10. |
|
|
352
404
|
| Logging on hot per-iteration paths at `info` | Floods collectors. Use `debug` (gated off by default). |
|
|
353
405
|
| `error_code=` missing on `err`/`warn` mapping to registry codes | Breaks traceability; future operator can't link log to docs. |
|
|
@@ -371,6 +423,7 @@ immediately preceding the edit. Generic "scope creep" is not a valid reason —
|
|
|
371
423
|
- `dispatch/write_ts_adhere_bun.md` §10 — banned `console.log` in TS/JS source. Cross-referenced by `logging.sh`.
|
|
372
424
|
- `dispatch/write_ts_adhere_bun.md` §9 — module-level error style (throw vs Result) for `agentsfleet`.
|
|
373
425
|
- `dispatch/write_zig.md` — Zig discipline umbrella; this doc's §7 is the logging-specific layer.
|
|
426
|
+
- `dispatch/write_rust.md` — Rust discipline umbrella; this doc's §8A is the logging-specific layer.
|
|
374
427
|
- `LIFECYCLE_PATTERNS.md` — orthogonal: ownership/cleanup of structs, including allocator wiring for the thread-local log buffer.
|
|
375
428
|
- M42_002 redaction harness (`src/runner/engine/runner_progress.zig`) — secret-redaction precondition; this doc's §6 inherits.
|
|
376
429
|
- Universal rules (RULE UFS, RULE TGU, RULE PRI, RULE FLL, RULE ORP, RULE TST-NAM) live in `docs/greptile-learnings/RULES.md`.
|
|
@@ -388,10 +441,12 @@ its line covers the clauses under it until the next heading.
|
|
|
388
441
|
|
|
389
442
|
| Tag | Meaning | Where the verdict comes from |
|
|
390
443
|
|---|---|---|
|
|
391
|
-
| `[DETERMINISTIC → LOG]` | a script decides; no talking past it | `audits/logging.sh` —
|
|
444
|
+
| `[DETERMINISTIC → LOG]` | a script decides; no talking past it | `audits/logging.sh` — direct Zig/Rust/TypeScript diagnostics and malformed Rust tracing emits in non-test source block; missing `error_code=` on `std.log.{err,warn}` reports as INFO |
|
|
392
445
|
| `[JUDGMENT → BOUNDARY]` | is this operation boundary-crossing, and is its `_started`/`_completed`\|`_failed` pair complete | the agent at write time; the reviewer at `/review` |
|
|
393
446
|
| `[JUDGMENT → REDACT]` | is this value a secret, or the label of one | the agent at write time; the M42_002 harness covers only executor-child output |
|
|
394
447
|
| `[JUDGMENT → MSG-REVIEW]` | does this `msg=` carry a credential | the reviewer; no length or content check runs today |
|
|
395
448
|
| `[JUDGMENT → TS-STYLE]` | throw-style or Result-style for this module | `dispatch/write_ts_adhere_bun.md` §9, agent-decided |
|
|
449
|
+
| `[JUDGMENT → RUST-STYLE]` | does a Rust failure map to a registry code, and are field expressions safely hoisted | the agent at write time; the reviewer at `/review` |
|
|
450
|
+
| `[JUDGMENT → EVENT-COMPAT]` | does a language port preserve the event bytes consumed by existing dashboards | the agent compares old and new event constants; the reviewer confirms intentional renames |
|
|
396
451
|
| `[JUDGMENT → SECTION-SCAN]` | which sections of this file the current sub-task needs | the agent; `audits/doc-read.sh` records façade reads, not delegated-doc reads |
|
|
397
452
|
| `[UNENFORCED → reason]` | acknowledged prose; the reason states why no check exists | nothing — that is the point of the class |
|
|
@@ -122,6 +122,25 @@ surfaces:
|
|
|
122
122
|
|
|
123
123
|
Orly owns policy and invokes these commands. The repository owns what they do.
|
|
124
124
|
|
|
125
|
+
## Usage telemetry
|
|
126
|
+
|
|
127
|
+
Orly keeps usage telemetry off until a person enables anonymous collection in
|
|
128
|
+
an interactive run. Hooks, Continuous Integration (CI), and other automated
|
|
129
|
+
runs never ask for consent. Off writes no event and makes no network request.
|
|
130
|
+
|
|
131
|
+
Anonymous events enter a bounded outbound spool under the agentsfleet state
|
|
132
|
+
root. After each append, Orly launches a detached sync attempt and lets the
|
|
133
|
+
command exit. The sync sends at most 100 events and exits without a fetch when
|
|
134
|
+
another attempt ran within five minutes. A successful response removes the
|
|
135
|
+
acknowledged prefix. Spool maintenance drops records older than seven days and
|
|
136
|
+
keeps the file at or below 10 MiB, so an unavailable destination cannot create
|
|
137
|
+
an unbounded local log. Orly runs no telemetry daemon.
|
|
138
|
+
|
|
139
|
+
PostHog owns product-event storage and funnel analysis. Orly sends no source,
|
|
140
|
+
path, repository, branch, argument, output, environment, hostname, username, or
|
|
141
|
+
raw error value. A future Insights Fleet may query PostHog with a private key;
|
|
142
|
+
the Fleet is an analysis consumer and never the public ingestion boundary.
|
|
143
|
+
|
|
125
144
|
## Evidence
|
|
126
145
|
|
|
127
146
|
`orly verify --all --write-evidence` records the source commit and every check
|
package/docs/RULE_ENFORCEMENT.md
CHANGED
|
@@ -6,15 +6,15 @@ regenerates this file and fails on any difference. To update it, run
|
|
|
6
6
|
|
|
7
7
|
| Rule document | enforced by | judged | acknowledged | classified | trigger |
|
|
8
8
|
|---|---|--:|--:|--:|---|
|
|
9
|
-
| `docs/LOGGING_STANDARD.md` | `LOG` → `logging.sh` |
|
|
9
|
+
| `docs/LOGGING_STANDARD.md` | `LOG` → `logging.sh` | 11 | 13 | 28/28 | mechanical |
|
|
10
10
|
| `docs/REST_API_DESIGN_GUIDELINES.md` | — | 0 | 0 | 0/74 | mechanical |
|
|
11
11
|
| `docs/SCHEMA_CONVENTIONS.md` | — | 0 | 0 | 0/16 | mechanical |
|
|
12
12
|
| `docs/DOCUMENTATION_RULES.md` | — | 0 | 0 | 0/22 | latent |
|
|
13
13
|
| `docs/LIFECYCLE_PATTERNS.md` | — | 0 | 0 | 0/22 | mechanical |
|
|
14
14
|
| `docs/CHANGELOG_VOICE.md` | — | 0 | 0 | 0/3 | latent |
|
|
15
15
|
| `docs/VERIFY_TIERS.md` | — | 0 | 0 | 0/4 | uncited |
|
|
16
|
-
| `docs/greptile-learnings/RULES.md` | — | 0 | 0 | 0/
|
|
17
|
-
| **corpus** | | | | **
|
|
16
|
+
| `docs/greptile-learnings/RULES.md` | — | 0 | 0 | 0/114 | mechanical |
|
|
17
|
+
| **corpus** | | | | **28/283** | |
|
|
18
18
|
|
|
19
19
|
## How to read this
|
|
20
20
|
|
|
@@ -179,6 +179,13 @@ every code a dispatch `.sh` emits resolves to exactly one row here.
|
|
|
179
179
|
**Tags:** zig, testing, sql
|
|
180
180
|
**Example:** tested 0 for a column with CHECK >= 512.
|
|
181
181
|
|
|
182
|
+
## RULE TCF — A test that cannot fail is not a test
|
|
183
|
+
|
|
184
|
+
**Rule:** Before trusting a test, make it red — delete the clause, guard, or branch it covers and confirm it fails. Assert what the check FOUND (the count, the symbol, the line), never only its exit code. A test that survives removal of its own subject is repaired or deleted, never kept for coverage.
|
|
185
|
+
**Why:** A vacuous pass and a real pass look identical from the outside, so the suite reports green while the guarantee is already gone.
|
|
186
|
+
**Tags:** testing, review, all
|
|
187
|
+
**Example:** a tenant-scope property asserted absence only and passed with the scope clause deleted; a dispatch eval asserted an exit code while its glob pointed at a dead tree.
|
|
188
|
+
|
|
182
189
|
## RULE JCL — CLI JSON contract discipline
|
|
183
190
|
|
|
184
191
|
**Rule:** Use only stable error codes; UNKNOWN_COMMAND must name the unrecognized token; dual jsonMode guards need a comment.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@agentsfleet/orly",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.7.0",
|
|
4
4
|
"description": "Repository-scoped engineering harness: renders agent rules, materialises the gates that enforce them, and proves the PR boundary \u2014 for any coding agent.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"type": "module",
|
|
@@ -1,5 +1,7 @@
|
|
|
1
1
|
# Python authoring
|
|
2
2
|
|
|
3
|
+
<!-- oracle-scope: *.py -->
|
|
4
|
+
|
|
3
5
|
Use standard-library parsers for structured input. Keep resource ownership in
|
|
4
6
|
context managers. Validate data at parse boundaries and preserve specific
|
|
5
7
|
exception classes until the caller can decide how to respond.
|
|
@@ -1,5 +1,7 @@
|
|
|
1
1
|
# Rust authoring
|
|
2
2
|
|
|
3
|
+
<!-- oracle-scope: *.rs -->
|
|
4
|
+
|
|
3
5
|
Use ownership and borrowing to make resource lifetime visible. Keep `unsafe`
|
|
4
6
|
blocks small and state the invariant they rely on. Preserve error variants until
|
|
5
7
|
the caller has enough information to decide whether to retry or stop.
|
|
@@ -31,6 +33,134 @@ Local convention wins on conflict, but the divergence is named in the review
|
|
|
31
33
|
output. Refresh the copy when the user asks or a cited rule looks stale
|
|
32
34
|
(re-run the fetch; upstream updates continuously).
|
|
33
35
|
|
|
36
|
+
## Error discipline (RULE ERR-RS)
|
|
37
|
+
|
|
38
|
+
One error type per crate, and a `Result` alias beside it:
|
|
39
|
+
|
|
40
|
+
pub type Result<T, E = Error> = core::result::Result<T, E>;
|
|
41
|
+
|
|
42
|
+
Every fallible function in the crate returns `Result<T>`. The default type
|
|
43
|
+
parameter is what lets the few functions answering with a foreign error keep
|
|
44
|
+
the same spelling — `Result<T, VerifyError>` — instead of reaching for
|
|
45
|
+
`std::result::Result`. A reader must never have to check WHICH error a
|
|
46
|
+
signature returns to know it is this crate's.
|
|
47
|
+
|
|
48
|
+
**Compose with `From`; `?` does the lifting.** This is `M-FROM-ERROR` and is
|
|
49
|
+
read there, carve-out included; what the guideline leaves open is how the
|
|
50
|
+
variant is spelled, which locally is `#[from]`:
|
|
51
|
+
|
|
52
|
+
#[derive(Debug, thiserror::Error)]
|
|
53
|
+
pub enum BootFailure {
|
|
54
|
+
#[error(transparent)]
|
|
55
|
+
Environment(#[from] Refusal),
|
|
56
|
+
#[error("cannot boot: the database would not answer")]
|
|
57
|
+
Database(#[from] db::Error),
|
|
58
|
+
}
|
|
59
|
+
|
|
60
|
+
**`map_err` only to ADD context the call site alone knows** — a role tag, a
|
|
61
|
+
budget, an operation name; the same carve-out `M-FROM-ERROR` names. And never
|
|
62
|
+
this, which no guideline covers:
|
|
63
|
+
|
|
64
|
+
// WRONG: a lossy conversion wearing a conversion's clothes.
|
|
65
|
+
.map_err(|error| Mine::Database(error.to_string()))?
|
|
66
|
+
|
|
67
|
+
`to_string()` on the way INTO an error type destroys the `source()` chain. It
|
|
68
|
+
compiles, it reads fine, and it silently defeats every chain walker downstream.
|
|
69
|
+
|
|
70
|
+
**`source()` returns what caused you, never yourself.** An error whose
|
|
71
|
+
`Display` already renders its kind must not also return that kind as its
|
|
72
|
+
source, or a chain walker prints the same sentence twice before reaching
|
|
73
|
+
anything new:
|
|
74
|
+
|
|
75
|
+
// WRONG — Display is "[code] {kind}", so the kind is already printed.
|
|
76
|
+
fn source(&self) -> Option<&(dyn Error + 'static)> { Some(&self.kind) }
|
|
77
|
+
|
|
78
|
+
// RIGHT — skip ourselves, hand back what the kind wraps.
|
|
79
|
+
fn source(&self) -> Option<&(dyn Error + 'static)> {
|
|
80
|
+
std::error::Error::source(&self.kind)
|
|
81
|
+
}
|
|
82
|
+
|
|
83
|
+
**Not every error has a cause, and that is not a defect.** A variant holding
|
|
84
|
+
another error has one; a variant holding only data (`MissingUrl { knob }`) does
|
|
85
|
+
not — nothing *caused* an unset variable. A test asserting `source().is_some()`
|
|
86
|
+
for every variant is WRONG and forces authors to invent causes. Assert the real
|
|
87
|
+
invariant: where there IS a source, it is not a repeat of our own message.
|
|
88
|
+
|
|
89
|
+
**Divergence from the reference guideline, named as that section requires.**
|
|
90
|
+
`M-ERRORS-CANONICAL-STRUCTS` prescribes a situation-specific `struct` carrying
|
|
91
|
+
a `Backtrace` and lets a complex crate expose several error types;
|
|
92
|
+
`M-APP-ERROR` further lets an application crate skip its own type for
|
|
93
|
+
`anyhow`/`eyre`/`ohno::AppError`. This rule diverges on all three for one
|
|
94
|
+
reason: a `#[from]`-composed enum behind a single per-crate alias is what lets
|
|
95
|
+
`?` lift across a crate boundary with no call-site conversion, and an
|
|
96
|
+
application-level catch-all erases the variant the caller would have matched
|
|
97
|
+
on. A `Backtrace` inside a variant is welcome. A crate with two genuinely
|
|
98
|
+
unrelated fallible surfaces may carry a second type — with its own alias, under
|
|
99
|
+
these same rules. Cite this divergence in review rather than re-deriving it.
|
|
100
|
+
|
|
101
|
+
Prior art, not invention: `core_api` (multi-crate workspace, per-crate
|
|
102
|
+
`error.rs` with `pub type Result<T>` + flat `Error`, in production for years),
|
|
103
|
+
bun (`thiserror`, `#[from]`, `pub type Result<T, E = Error>`), habitat (one
|
|
104
|
+
payload-carrying `enum Error`, one alias).
|
|
105
|
+
|
|
106
|
+
## Functional design (RULE FN-RS)
|
|
107
|
+
|
|
108
|
+
Design the fallible path as a Result PIPELINE, never as a control-flow tree.
|
|
109
|
+
|
|
110
|
+
**Fallibility lives in the type, never in a sentinel.** A function that can
|
|
111
|
+
fail returns `Result<T>`; a value that can be absent returns `Option<T>`.
|
|
112
|
+
Never a bool-plus-out-parameter, never a magic value (-1, empty string, null
|
|
113
|
+
UUID), never a logged-and-swallowed error. If the caller cannot see the
|
|
114
|
+
failure in the signature, the design is wrong before the body is written.
|
|
115
|
+
|
|
116
|
+
**Compose with combinators; spend `match` only where arms genuinely differ.**
|
|
117
|
+
`?`, `map`, `and_then`, `ok_or_else`, `unwrap_or_else` say "transform on
|
|
118
|
+
success, pass failure through" in one line, where a nested match spends five
|
|
119
|
+
and gives every arm a chance to swallow the `Err`:
|
|
120
|
+
|
|
121
|
+
// WRONG: the failure path is hand-plumbed, twice.
|
|
122
|
+
let config = match load(path) {
|
|
123
|
+
Ok(raw) => match parse(&raw) {
|
|
124
|
+
Ok(c) => c,
|
|
125
|
+
Err(e) => return Err(e.into()),
|
|
126
|
+
},
|
|
127
|
+
Err(e) => return Err(e.into()),
|
|
128
|
+
};
|
|
129
|
+
|
|
130
|
+
// RIGHT: one pipeline; `?` and `From` (RULE ERR-RS) do the lifting.
|
|
131
|
+
let config = parse(&load(path)?)?;
|
|
132
|
+
|
|
133
|
+
A `match` earns its place when the arms carry different LOGIC — not when one
|
|
134
|
+
arm is a hand-written `?`.
|
|
135
|
+
|
|
136
|
+
**Make illegal states unrepresentable; parse, don't validate.** Convert raw
|
|
137
|
+
input into a type that can only hold valid states ONCE, at the boundary, and
|
|
138
|
+
pass that type inward. Two `Option` fields where exactly one is ever set is an
|
|
139
|
+
enum. A `state: String` compared against literals is an enum. A checked-then-
|
|
140
|
+
used raw value is a newtype whose constructor returns `Result` — that
|
|
141
|
+
constructor is `M-STRONG-TYPES-GUARD`, and picking the type family is
|
|
142
|
+
`M-STRONG-TYPES`. What those do not say is the consequence: interior code that
|
|
143
|
+
trusts its types needs no defensive re-checks — delete them.
|
|
144
|
+
|
|
145
|
+
**Expression-orientation: bind once.** `let x = if ... { a } else { b };` and
|
|
146
|
+
`let x = match ...` — never a mutable default that later branches overwrite.
|
|
147
|
+
A `mut` that exists only to simulate an expression is a bug surface with no
|
|
148
|
+
compensating power.
|
|
149
|
+
|
|
150
|
+
**`unwrap()` / `expect()` are for tests and for invariants already proved.**
|
|
151
|
+
`M-PANIC-ON-BUG` settles when a panic is right and `M-PANIC-MESSAGE` that it
|
|
152
|
+
must be helpful; the addition here is WHAT the message carries — the proof
|
|
153
|
+
("mutex never poisoned: no thread panics while holding it"). In production
|
|
154
|
+
code, an `unwrap()` without a written invariant is a `Result` that was never
|
|
155
|
+
designed.
|
|
156
|
+
|
|
157
|
+
Prior art, not invention: `core_api` — a multi-crate production workspace
|
|
158
|
+
where every crate with a fallible surface carries its own `error.rs` +
|
|
159
|
+
`Result` alias, and the fallible path is written as pipelines throughout
|
|
160
|
+
(573 `.map`, 463 `.and_then`, 164 `.ok_or_else` across the workspace). It has
|
|
161
|
+
run in production for years; its author attributes the absence of error-path
|
|
162
|
+
defects to exactly this design, and the maintenance record bears that out.
|
|
163
|
+
|
|
34
164
|
## Constant discipline (RULE UFS)
|
|
35
165
|
|
|
36
166
|
`audits/ufs.sh` reads `*.rs`. It did not until this pack said so, and the cost
|
package/registry.json
CHANGED
|
@@ -263,6 +263,10 @@
|
|
|
263
263
|
"source": "dispatch/edit_rules.md",
|
|
264
264
|
"target": "dispatch/edit_rules.md"
|
|
265
265
|
},
|
|
266
|
+
{
|
|
267
|
+
"source": "dispatch/edit_rules.sh",
|
|
268
|
+
"target": "dispatch/edit_rules.sh"
|
|
269
|
+
},
|
|
266
270
|
{
|
|
267
271
|
"source": "audits/agents-md.md",
|
|
268
272
|
"target": "audits/agents-md.md"
|
|
@@ -13,6 +13,12 @@ description: |
|
|
|
13
13
|
|
|
14
14
|
# orly-babysit-prs
|
|
15
15
|
|
|
16
|
+
At invocation start, record the consent-gated usage signal without blocking the skill:
|
|
17
|
+
|
|
18
|
+
```bash
|
|
19
|
+
command -v orly >/dev/null 2>&1 && ORLY_INVOCATION=skill orly skill-event orly-babysit-prs >/dev/null 2>&1 || true
|
|
20
|
+
```
|
|
21
|
+
|
|
16
22
|
Review bots (greptile on GitHub; greptile or another review bot on GitLab)
|
|
17
23
|
post auto-reviews **asynchronously**. They land as PR review comments / MR
|
|
18
24
|
discussion threads, not check runs, so `gh pr checks --watch` (or the glab
|
|
@@ -11,6 +11,12 @@ description: |
|
|
|
11
11
|
|
|
12
12
|
# orly-spec-new
|
|
13
13
|
|
|
14
|
+
At invocation start, record the consent-gated usage signal without blocking the skill:
|
|
15
|
+
|
|
16
|
+
```bash
|
|
17
|
+
command -v orly >/dev/null 2>&1 && ORLY_INVOCATION=skill orly skill-event orly-spec-new >/dev/null 2>&1 || true
|
|
18
|
+
```
|
|
19
|
+
|
|
14
20
|
A spec is the **rulebook** the executing agent (Orly) plans and ships from. This
|
|
15
21
|
skill makes me author one whose Pull Request (PR) lands **deterministic, review-clean, and
|
|
16
22
|
reported** — without a 20-questions loop. It is written for my reasoning, not as
|
|
@@ -11,6 +11,12 @@ description: >
|
|
|
11
11
|
|
|
12
12
|
# Write Integration Test
|
|
13
13
|
|
|
14
|
+
At invocation start, record the consent-gated usage signal without blocking the skill:
|
|
15
|
+
|
|
16
|
+
```bash
|
|
17
|
+
command -v orly >/dev/null 2>&1 && ORLY_INVOCATION=skill orly skill-event orly-write-integration-test >/dev/null 2>&1 || true
|
|
18
|
+
```
|
|
19
|
+
|
|
14
20
|
Service-layer tests that prove real wiring works under real failure modes. Sits between unit tests (`make test`) and browser E2E (gstack `/qa`, `/e2e-qa-playwright`).
|
|
15
21
|
|
|
16
22
|
> **What this guarantees / what it doesn't.** Integration tests don't prove "zero bugs
|
|
@@ -11,6 +11,12 @@ description: >
|
|
|
11
11
|
|
|
12
12
|
# Write Unit Test
|
|
13
13
|
|
|
14
|
+
At invocation start, record the consent-gated usage signal without blocking the skill:
|
|
15
|
+
|
|
16
|
+
```bash
|
|
17
|
+
command -v orly >/dev/null 2>&1 && ORLY_INVOCATION=skill orly skill-event orly-write-unit-test >/dev/null 2>&1 || true
|
|
18
|
+
```
|
|
19
|
+
|
|
14
20
|
Generate tests that catch bugs before they ship. Tests run *during* implementation, not after.
|
|
15
21
|
|
|
16
22
|
> **What this guarantees / what it doesn't.** No test suite proves "zero bugs in
|
package/src/cli.test.ts
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
import { describe, expect, test } from "bun:test";
|
|
2
|
-
import { existsSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";
|
|
2
|
+
import { existsSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";
|
|
3
3
|
import { tmpdir } from "node:os";
|
|
4
|
-
import { join, resolve } from "node:path";
|
|
4
|
+
import { dirname, join, resolve } from "node:path";
|
|
5
5
|
|
|
6
6
|
const ROOT = resolve(import.meta.dir, "..");
|
|
7
7
|
const COMMAND = resolve(ROOT, "bin/orly");
|
|
@@ -18,6 +18,19 @@ describe("orly command", () => {
|
|
|
18
18
|
expect(output).not.toContain("oracle-rules");
|
|
19
19
|
});
|
|
20
20
|
|
|
21
|
+
test("telemetry_help_names_tiers_storage_and_privacy", () => {
|
|
22
|
+
const result = Bun.spawnSync([COMMAND, "--help"], { cwd: ROOT, stdout: "pipe", stderr: "pipe" });
|
|
23
|
+
const output = result.stdout.toString();
|
|
24
|
+
|
|
25
|
+
expect(output).toContain("off by default");
|
|
26
|
+
expect(output).toContain("anonymous");
|
|
27
|
+
expect(output).toContain("~/.config/agentsfleet/orly/.orly.json");
|
|
28
|
+
expect(output).toContain("AGENTSFLEET_STATE_DIR");
|
|
29
|
+
expect(output).toContain("ORLY_TELEMETRY_OFF=1");
|
|
30
|
+
expect(output).toContain("packaged Orly skill");
|
|
31
|
+
expect(output).toContain("never collects source, paths, repositories, branches, arguments, prompts");
|
|
32
|
+
});
|
|
33
|
+
|
|
21
34
|
// Adversarial review found recordOptIn running before the --dry-run branch,
|
|
22
35
|
// so a preview created .oracle/orly.json — and on an installed repository
|
|
23
36
|
// permanently added the pack. A dry run changes nothing, opt-in included.
|
|
@@ -67,4 +80,72 @@ describe("orly command", () => {
|
|
|
67
80
|
|
|
68
81
|
expect(result.exitCode).toBe(0);
|
|
69
82
|
});
|
|
83
|
+
|
|
84
|
+
test("telemetry_failure_never_changes_command_result", () => {
|
|
85
|
+
const blockedRoot = join(mkdtempSync(join(tmpdir(), "orly-telemetry-blocked-")), "state-file");
|
|
86
|
+
writeFileSync(blockedRoot, "not a directory\n");
|
|
87
|
+
const enabled = { ...process.env, AGENTSFLEET_STATE_DIR: blockedRoot, ORLY_TELEMETRY: "anonymous" };
|
|
88
|
+
const off = { ...enabled, ORLY_TELEMETRY_OFF: "1" };
|
|
89
|
+
try {
|
|
90
|
+
const successWithFailure = Bun.spawnSync([COMMAND, "doctor"], { cwd: ROOT, env: enabled, stdout: "pipe", stderr: "pipe" });
|
|
91
|
+
const successWithOff = Bun.spawnSync([COMMAND, "doctor"], { cwd: ROOT, env: off, stdout: "pipe", stderr: "pipe" });
|
|
92
|
+
const errorWithFailure = Bun.spawnSync([COMMAND, "gate", "nope"], { cwd: ROOT, env: enabled, stdout: "pipe", stderr: "pipe" });
|
|
93
|
+
const errorWithOff = Bun.spawnSync([COMMAND, "gate", "nope"], { cwd: ROOT, env: off, stdout: "pipe", stderr: "pipe" });
|
|
94
|
+
|
|
95
|
+
expect(successWithFailure.exitCode).toBe(successWithOff.exitCode);
|
|
96
|
+
expect(successWithFailure.stdout.toString()).toBe(successWithOff.stdout.toString());
|
|
97
|
+
expect(errorWithFailure.exitCode).toBe(errorWithOff.exitCode);
|
|
98
|
+
expect(errorWithFailure.stderr.toString()).toBe(errorWithOff.stderr.toString());
|
|
99
|
+
} finally {
|
|
100
|
+
rmSync(dirname(blockedRoot), { recursive: true, force: true });
|
|
101
|
+
}
|
|
102
|
+
});
|
|
103
|
+
|
|
104
|
+
test("gate_command_records_its_selected_gate_and_failed_criterion", async () => {
|
|
105
|
+
const root = mkdtempSync(join(tmpdir(), "orly-telemetry-command-"));
|
|
106
|
+
const repo = join(root, "repo");
|
|
107
|
+
const stateRoot = join(root, "state");
|
|
108
|
+
Bun.spawnSync(["git", "init", "-q", repo]);
|
|
109
|
+
writeFileSync(join(repo, "README.md"), "fixture\n");
|
|
110
|
+
Bun.spawnSync(["git", "-C", repo, "add", "README.md"]);
|
|
111
|
+
Bun.spawnSync(["git", "-C", repo, "-c", "user.name=Orly Test", "-c", "user.email=orly@example.invalid", "commit", "-q", "-m", "test: seed"]);
|
|
112
|
+
Bun.spawnSync(["git", "-C", repo, "checkout", "-q", "-b", "feat/telemetry"]);
|
|
113
|
+
const env = {
|
|
114
|
+
...process.env,
|
|
115
|
+
AGENTSFLEET_STATE_DIR: stateRoot,
|
|
116
|
+
ORLY_TELEMETRY: "anonymous",
|
|
117
|
+
ORLY_TELEMETRY_OFF: "0",
|
|
118
|
+
ORLY_POSTHOG_HOST: "http://127.0.0.1:1",
|
|
119
|
+
};
|
|
120
|
+
try {
|
|
121
|
+
const result = Bun.spawnSync([COMMAND, "gate", "pr"], { cwd: repo, env, stdout: "pipe", stderr: "pipe" });
|
|
122
|
+
expect(result.exitCode).toBe(1);
|
|
123
|
+
const failedCriterion = result.stdout.toString().match(/🔴 ([^:]+):/)?.[1];
|
|
124
|
+
expect(failedCriterion).toBeDefined();
|
|
125
|
+
|
|
126
|
+
const spool = join(stateRoot, "orly/analytics/orly-usage.jsonl");
|
|
127
|
+
for (let attempt = 0; attempt < 20 && !existsSync(spool); attempt += 1) await Bun.sleep(10);
|
|
128
|
+
const event = JSON.parse(readFileSync(spool, "utf8"));
|
|
129
|
+
expect(event).toMatchObject({ command: "gate", gate: "pr", outcome: "error", failed_criterion: failedCriterion });
|
|
130
|
+
expect(event.duration_ms).toBeGreaterThanOrEqual(0);
|
|
131
|
+
} finally {
|
|
132
|
+
rmSync(root, { recursive: true, force: true });
|
|
133
|
+
}
|
|
134
|
+
});
|
|
135
|
+
|
|
136
|
+
test("skill_event_accepts_only_packaged_skills", () => {
|
|
137
|
+
const env = { ...process.env, ORLY_TELEMETRY_OFF: "1", ORLY_INVOCATION: "skill" };
|
|
138
|
+
const packagedSkills = ["orly-babysit-prs", "orly-spec-new", "orly-write-integration-test", "orly-write-unit-test"];
|
|
139
|
+
for (const skill of packagedSkills) {
|
|
140
|
+
const accepted = Bun.spawnSync([COMMAND, "skill-event", skill], { cwd: ROOT, env, stdout: "pipe", stderr: "pipe" });
|
|
141
|
+
expect(accepted.exitCode).toBe(0);
|
|
142
|
+
expect(accepted.stdout.toString()).toBe("");
|
|
143
|
+
expect(readFileSync(join(ROOT, "skills", skill, "SKILL.md"), "utf8")).toContain(`orly skill-event ${skill}`);
|
|
144
|
+
}
|
|
145
|
+
const rejected = Bun.spawnSync([COMMAND, "skill-event", "other-skill"], { cwd: ROOT, env, stdout: "pipe", stderr: "pipe" });
|
|
146
|
+
|
|
147
|
+
expect(rejected.exitCode).toBe(1);
|
|
148
|
+
expect(rejected.stderr.toString()).toContain("skill-event requires a packaged Orly skill name");
|
|
149
|
+
expect(Bun.spawnSync([COMMAND, "--help"], { cwd: ROOT, env, stdout: "pipe" }).stdout.toString()).not.toContain("skill-event");
|
|
150
|
+
});
|
|
70
151
|
});
|