@agentsfleet/orly 0.13.0 → 0.14.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -42,9 +42,19 @@ Works with Claude Code, Codex, OpenCode, and Amp.
42
42
 
43
43
  ---
44
44
 
45
- ## 0.13: measured judgment comparison
45
+ ## 0.14: retained Jev answers explained
46
46
 
47
- The 0.13.0 source candidate adds a deterministic examiner for unsupported completion claims.
47
+ The [offline diagnosis](https://github.com/agentsfleet/orly/blob/main/evals/judgments/comparison/diagnosis/report.md) explains all 188 retained TypeSafe Jev attempts.
48
+ It separates native-answer agreement, confidence withholding, grading disagreements and unavailable replies.
49
+ Each attempt links to its saved inputs and a bounded explanation; unknown provider causes stay unknown.
50
+
51
+ The diagnosis uses existing receipts and sends no new provider requests.
52
+ All five proposed checks remain evaluation-only while fixture meanings and independent adoption evidence need review.
53
+ These contributor records live in the source checkout; installed packages link to the same published source.
54
+
55
+ ## Measured judgment comparison
56
+
57
+ The source-checkout examiner checks unsupported completion claims.
48
58
  It freezes the six runtime questions and evaluates five proposed checks separately.
49
59
  Those checks cover obligations, evidence, production wiring, failure tests and finding resolution.
50
60
 
@@ -55,7 +65,9 @@ A successful run reports `passed: true`; seeded false completions remain failed
55
65
 
56
66
  A separate explicit `jev-run.ts --live` command measures TypeSafe Jev answers against the frozen expectations.
57
67
  Its retained report lists every measured answer, grading match, disagreement and unavailable response.
58
- Replay reproduces every result with zero new requests. Autonomous agent improvement remains unmeasured.
68
+ The saved replay reproduced every result with zero new requests at historical source revision [`751ee303`](https://github.com/agentsfleet/orly/tree/751ee303d0c97a403863f12f33adf8fbc20ff433).
69
+ Replay requires that revision's unchanged source inventory; changed bytes fail admission.
70
+ Autonomous agent improvement remains unmeasured.
59
71
  All five candidates remain evaluation-only; uncertain and failed replies stay in the report.
60
72
  An additional rehearsal uses the packed candidate and actual `agentsfleet` source in an isolated worktree.
61
73
  See the [evaluation design](docs/architecture/judgment-evaluation.md) for commands, evidence and limits.
@@ -65,7 +77,8 @@ The [0.12 report](evals/release/release-report.md) retains the earlier package a
65
77
  TypeSafe's Jev model supplies runtime advice through explicit `orly judge` requests; contributor measurement uses the separate live evaluator.
66
78
 
67
79
  See [releases](https://github.com/agentsfleet/orly/releases) for published versions.
68
- The install commands below fetch the published package; 0.13.0 publication is a separate owner action.
80
+ The install commands below fetch the published package.
81
+ The release workflow publishes 0.14.1 after its version change merges to `main`.
69
82
 
70
83
  ---
71
84
 
package/docs/CHANGELOG.md CHANGED
@@ -1,5 +1,20 @@
1
1
  # Changelog
2
2
 
3
+ ## 0.14.1 — 2026-10-11
4
+
5
+ The logging standard names the field for a second failure's code.
6
+
7
+ - **Role-prefixed codes** — §5 defines `cause_error_code`, `rollback_error_code` and `config_error_code`, and the `<role>_error_code` shape for new roles.
8
+ - **Rust binding** — §8A points a record that reports a second failure at §5.
9
+
10
+ ## 0.14.0 — 2026-10-08
11
+
12
+ Retained TypeSafe Jev measurements now carry per-attempt explanations and explicit adoption limits.
13
+
14
+ - **Diagnosis** — accounts for all 188 saved attempts and separates native-answer agreement from confidence withholding and grading.
15
+ - **Evidence** — preserves original replies and labels; names uncertain fixture meanings and unavailable provider explanations.
16
+ - **Guidance** — links contributors to the source diagnosis while keeping all five proposed checks evaluation-only.
17
+
3
18
  ## 0.13.0 — 2026-10-07
4
19
 
5
20
  A source-checkout examiner checks unsupported completion claims and grades actual TypeSafe Jev answers against frozen expectations.
@@ -184,6 +184,21 @@ Embed the code as a struct field on the logger call: `log.err("event_name", .{ .
184
184
 
185
185
  System-level failures with no domain meaning (e.g. raw `EACCES` from a syscall before we attribute it to a tenant operation) emit without `error_code`. The follow-up rule: if a syscall failure surfaces to a user, it gets attributed to a registry code at the boundary.
186
186
 
187
+ **One record, two failures.** Some records report a decision about another
188
+ failure: a gate that fails open, a rollback, a refused configuration. The
189
+ record's `error_code` names the decision. The failure behind it keeps its own
190
+ registry code in a field named for its role:
191
+
192
+ | Field | Carries the code of |
193
+ |---|---|
194
+ | `cause_error_code` | the failure the decision answers |
195
+ | `rollback_error_code` | a failure in the recovery that followed |
196
+ | `config_error_code` | the configuration read that could not be used |
197
+
198
+ A new role takes the same `<role>_error_code` shape. `error_code` always names
199
+ the record's own code, so never overwrite it with the cause's code. Never
200
+ embed a code inside a reason sentence either; the field carries it.
201
+
187
202
  ## §6 · PII / secret discipline
188
203
 
189
204
  > [UNENFORCED → broad principle; no scan covers every log call site in the codebase for credential-shaped values]
@@ -324,12 +339,14 @@ arguments, never positional message formatting.
324
339
 
325
340
  > [JUDGMENT → RUST-STYLE]
326
341
 
327
- Every `warn` or `error` that maps to a registry code supplies
328
- `error_code = code` per §5. Hoist field expressions into locals before the macro
329
- call. The log bridge duplicates field expressions, and LLVM coverage
330
- instrumentation scores the dead copy; locals preserve evaluation and coverage
331
- behavior. A trailing string message is §3's optional `msg` and is always the
332
- last macro argument. Prefer fields alone when they already explain the event.
342
+ Every `warn` or `error` that maps to a registry code supplies `error_code =
343
+ code` per §5. A record that reports a second failure adds its
344
+ `<role>_error_code` per §5, for example `cause_error_code`. Hoist field
345
+ expressions into locals before the macro call. The log bridge duplicates field
346
+ expressions, and LLVM coverage instrumentation scores the dead copy; locals
347
+ preserve evaluation and coverage behavior. A trailing string message is §3's
348
+ optional `msg` and is always the last macro argument. Prefer fields alone when
349
+ they already explain the event.
333
350
 
334
351
  ```rust
335
352
  let code = error_code::INTERNAL_DB_QUERY.as_str();
@@ -6,7 +6,7 @@ regenerates this file and fails on any difference. To update it, run
6
6
 
7
7
  | Rule document | enforced by | judged | acknowledged | classified | trigger |
8
8
  |---|---|--:|--:|--:|---|
9
- | `docs/LOGGING_STANDARD.md` | `LOG` → `logging.sh` | 10 | 13 | 28/28 | mechanical |
9
+ | `docs/LOGGING_STANDARD.md` | `LOG` → `logging.sh` | 10 | 13 | 30/30 | mechanical |
10
10
  | `docs/REST_API_DESIGN_GUIDELINES.md` | — | 0 | 0 | 0/67 | mechanical |
11
11
  | `docs/SCHEMA_CONVENTIONS.md` | — | 0 | 0 | 0/17 | mechanical |
12
12
  | `docs/DOCUMENTATION_RULES.md` | — | 0 | 0 | 0/22 | latent |
@@ -14,7 +14,7 @@ regenerates this file and fails on any difference. To update it, run
14
14
  | `docs/CHANGELOG_VOICE.md` | — | 0 | 0 | 0/3 | latent |
15
15
  | `docs/VERIFY_TIERS.md` | — | 0 | 0 | 0/4 | uncited |
16
16
  | `docs/greptile-learnings/RULES.md` | — | 0 | 0 | 0/115 | mechanical |
17
- | **corpus** | | | | **28/278** | |
17
+ | **corpus** | | | | **30/280** | |
18
18
 
19
19
  ## How to read this
20
20
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@agentsfleet/orly",
3
- "version": "0.13.0",
3
+ "version": "0.14.1",
4
4
  "description": "Repository-scoped engineering harness: renders agent rules, materialises the gates that enforce them, and proves the PR boundary \u2014 for any coding agent.",
5
5
  "license": "MIT",
6
6
  "type": "module",