@agentsfleet/orly 0.6.9 → 0.7.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/audits/agents-md.md +2 -2
- package/audits/logging.sh +164 -11
- package/dispatch/write_any.md +7 -5
- package/docs/LOGGING_STANDARD.md +60 -5
- package/docs/ORLY_ARCHITECTURE.md +19 -0
- package/docs/RULE_ENFORCEMENT.md +2 -2
- package/package.json +1 -1
- package/skills/orly-babysit-prs/SKILL.md +6 -0
- package/skills/orly-spec-new/SKILL.md +6 -0
- package/skills/orly-write-integration-test/SKILL.md +6 -0
- package/skills/orly-write-unit-test/SKILL.md +6 -0
- package/src/cli.test.ts +83 -2
- package/src/cli.ts +62 -18
- package/src/install.test.ts +3 -2
- package/src/install.ts +2 -2
- package/src/telemetry.test.ts +308 -0
- package/src/telemetry.ts +350 -0
- package/src/telemetry_sync.ts +134 -0
package/audits/agents-md.md
CHANGED
|
@@ -81,8 +81,8 @@ The questionnaire is organised by scenario. Each scenario corresponds to a momen
|
|
|
81
81
|
| 4.9 (Shell) | Must `gitleaks` pass before any commit/push? | YES |
|
|
82
82
|
| 4.10 (CI) | Are CI/CD edits (`.github/workflows/**`, deploy configs) **forbidden without explicit user approval** even in auto mode? | YES |
|
|
83
83
|
| 4.11 (Bun) | For `*.ts`/`*.tsx`/`*.js`/`*.jsx` edits, does AGENTS.md route to `dispatch/write_ts_adhere_bun.md` (TS FILE SHAPE DECISION at PLAN, const/import/Bun-primitive discipline)? | YES |
|
|
84
|
-
| 4.12 (Logging) | For every Edit/Write that adds/removes/changes a log emit (Zig `std.log.*`/`std.debug.print`/`obs.scoped`, TS/JS `console.*`/custom logger, shell `echo` to `&2`), does LOGGING GATE require reading `docs/LOGGING_STANDARD.md` and printing the per-edit gate block citing §3 (wire format) / §4 (severity) / §5 (error codes) / §6 (PII) / §10A (tightenings)? | YES |
|
|
85
|
-
| 4.13 (Logging) |
|
|
84
|
+
| 4.12 (Logging) | For every Edit/Write that adds/removes/changes a log emit (Zig `std.log.*`/`std.debug.print`/`obs.scoped`, Rust `tracing::*`/print macros under `rustd/`, TS/JS `console.*`/custom logger, shell `echo` to `&2`), does LOGGING GATE require reading `docs/LOGGING_STANDARD.md` and printing the per-edit gate block citing §3 (wire format) / §4 (severity) / §5 (error codes) / §6 (PII) / §7, §8, or §8A (language binding) / §10A (tightenings)? | YES |
|
|
85
|
+
| 4.13 (Logging) | Are `std.debug.print`, Rust `println!`/`eprintln!`/`dbg!`, Rust tracing without `event` or with positional formatting, and `console.log` in their non-test production scopes automatic blocking violations, with no "temporary" carve-out? | YES |
|
|
86
86
|
| 4.14 (Logging) | Must a new registry-scheme `error_code` reference (agentsfleet: `UZ-XXX-NNN`) land in the same commit as its registry entry (used-but-undeclared = blocking)? | YES |
|
|
87
87
|
| 4.15 (Lifecycle) | For every Edit/Write that adds/reshapes a lifecycle method in `*.zig` (`pub fn init|deinit|close|release|destroy|shutdown|dispose|free`) or an `errdefer`/`defer` adjacent to allocation, does LIFECYCLE GATE require reading `docs/LIFECYCLE_PATTERNS.md` and printing the per-edit gate block? | YES |
|
|
88
88
|
| 4.16 (Lifecycle) | Is `defer X.free(Y)` + `errdefer X.free(Y)` on the same allocation in the same scope a blocking violation? | YES |
|
package/audits/logging.sh
CHANGED
|
@@ -9,7 +9,7 @@
|
|
|
9
9
|
# - agentsfleet scope-prefix format from LOGGING_STANDARD §7 (the
|
|
10
10
|
# `log.scoped(...)` API path under `src/logging/`)
|
|
11
11
|
# - `UZ-XXX-NNN` as the `error_code=` substring per LOGGING_STANDARD §5
|
|
12
|
-
# - `src/` + `agentsfleet/src/` as the scan roots
|
|
12
|
+
# - `src/` + `rustd/` + `agentsfleet/src/` as the scan roots
|
|
13
13
|
# A parameterised version that reads prefix + scope-api path + scan
|
|
14
14
|
# roots from a per-project config is the right long-term shape; not
|
|
15
15
|
# done yet. Today, this script lives in dotfiles so the gate body and
|
|
@@ -18,6 +18,8 @@
|
|
|
18
18
|
# Two severity tiers:
|
|
19
19
|
# BLOCK — exits 1, must fix:
|
|
20
20
|
# - `std.debug.print(` in non-test source under src/.
|
|
21
|
+
# - `println!` / `eprintln!` / `dbg!`, missing `event`, or positional
|
|
22
|
+
# tracing formatting in non-test Rust source under rustd/.
|
|
21
23
|
# - `console.log/debug/info/warn/error` in agentsfleet/src outside tests.
|
|
22
24
|
# INFO — surfaced for reviewer/agent attention, doesn't block:
|
|
23
25
|
# - `std.log.scoped(...)` outside `src/logging/` (LOGGING_STANDARD §7
|
|
@@ -29,7 +31,7 @@
|
|
|
29
31
|
#
|
|
30
32
|
# Modes:
|
|
31
33
|
# --staged diff-scope: only files in `git diff --cached`
|
|
32
|
-
# --all (default) full src/ + agentsfleet/src/ scan
|
|
34
|
+
# --all (default) full src/ + rustd/ + agentsfleet/src/ scan
|
|
33
35
|
# --strict promote every INFO finding to BLOCK (post-migration use)
|
|
34
36
|
#
|
|
35
37
|
# Exits 0 clean, 1 on BLOCK findings.
|
|
@@ -79,6 +81,13 @@ is_test_zig() {
|
|
|
79
81
|
return 1
|
|
80
82
|
}
|
|
81
83
|
|
|
84
|
+
is_test_rust_path() {
|
|
85
|
+
case "$1" in
|
|
86
|
+
*/tests/*|*/benches/*) return 0 ;;
|
|
87
|
+
esac
|
|
88
|
+
return 1
|
|
89
|
+
}
|
|
90
|
+
|
|
82
91
|
# ---------------------------------------------------------------------------
|
|
83
92
|
# 1. Gather files in scope.
|
|
84
93
|
# ---------------------------------------------------------------------------
|
|
@@ -86,11 +95,12 @@ gather_paths() {
|
|
|
86
95
|
case "$MODE" in
|
|
87
96
|
--staged)
|
|
88
97
|
git diff --cached --name-only --diff-filter=ACMRT \
|
|
89
|
-
| grep -E '^(src/.*\.zig$|agentsfleet/src/.*\.(js|jsx|ts|tsx)$)' || true
|
|
98
|
+
| grep -E '^(src/.*\.zig$|rustd/.*\.rs$|agentsfleet/src/.*\.(js|jsx|ts|tsx)$)' || true
|
|
90
99
|
;;
|
|
91
100
|
--all)
|
|
92
|
-
find src -type f -name '*.zig' 2>/dev/null
|
|
93
|
-
find
|
|
101
|
+
find src -type f -name '*.zig' 2>/dev/null || true
|
|
102
|
+
find rustd -type f -name '*.rs' 2>/dev/null || true
|
|
103
|
+
find agentsfleet/src -type f \( -name '*.js' -o -name '*.jsx' -o -name '*.ts' -o -name '*.tsx' \) 2>/dev/null || true
|
|
94
104
|
;;
|
|
95
105
|
esac
|
|
96
106
|
}
|
|
@@ -105,11 +115,15 @@ fi
|
|
|
105
115
|
# previously ran 3–4 forks × N files; M70 perf pass batches them into
|
|
106
116
|
# single awk/grep passes.
|
|
107
117
|
zig_nontest=()
|
|
118
|
+
rust_nontest=()
|
|
108
119
|
js_nontest=()
|
|
109
120
|
for f in "${FILES[@]}"; do
|
|
110
121
|
if [[ "$f" == *.zig ]] && ! is_test_zig "$f"; then
|
|
111
122
|
zig_nontest+=("$f")
|
|
112
123
|
fi
|
|
124
|
+
if [[ "$f" == *.rs ]] && ! is_test_rust_path "$f"; then
|
|
125
|
+
rust_nontest+=("$f")
|
|
126
|
+
fi
|
|
113
127
|
case "$f" in
|
|
114
128
|
agentsfleet/src/*.test.*|agentsfleet/src/*.spec.*|agentsfleet/src/tests/*) ;;
|
|
115
129
|
agentsfleet/src/*.js|agentsfleet/src/*.jsx|agentsfleet/src/*.ts|agentsfleet/src/*.tsx)
|
|
@@ -140,7 +154,146 @@ if [[ ${#zig_nontest[@]} -gt 0 ]]; then
|
|
|
140
154
|
fi
|
|
141
155
|
|
|
142
156
|
# ---------------------------------------------------------------------------
|
|
143
|
-
# 3. BLOCKING:
|
|
157
|
+
# 3. BLOCKING: unstructured or event-less tracing in non-test Rust source.
|
|
158
|
+
# The awk tracker excludes complete items annotated with #[cfg(test)].
|
|
159
|
+
# ---------------------------------------------------------------------------
|
|
160
|
+
rust_direct_hits=0
|
|
161
|
+
rust_missing_event_hits=0
|
|
162
|
+
rust_positional_hits=0
|
|
163
|
+
if [[ ${#rust_nontest[@]} -gt 0 ]]; then
|
|
164
|
+
while IFS='|' read -r kind f ln; do
|
|
165
|
+
[[ -z "$kind" ]] && continue
|
|
166
|
+
case "$kind" in
|
|
167
|
+
direct)
|
|
168
|
+
fail "$f:$ln — direct Rust diagnostic macro in non-test source (LOGGING_STANDARD §8A)"
|
|
169
|
+
rust_direct_hits=$((rust_direct_hits + 1))
|
|
170
|
+
;;
|
|
171
|
+
event)
|
|
172
|
+
fail "$f:$ln — \`tracing\` emit without \`event = ...\` (LOGGING_STANDARD §8A)"
|
|
173
|
+
rust_missing_event_hits=$((rust_missing_event_hits + 1))
|
|
174
|
+
;;
|
|
175
|
+
positional)
|
|
176
|
+
fail "$f:$ln — positional formatting in \`tracing\` emit (LOGGING_STANDARD §8A)"
|
|
177
|
+
rust_positional_hits=$((rust_positional_hits + 1))
|
|
178
|
+
;;
|
|
179
|
+
esac
|
|
180
|
+
done < <(awk '
|
|
181
|
+
function count_char(text, char, copy, count) {
|
|
182
|
+
copy = text
|
|
183
|
+
count = gsub(char, "", copy)
|
|
184
|
+
return count
|
|
185
|
+
}
|
|
186
|
+
function code_only(text, output, cursor, char, escaped, quoted) {
|
|
187
|
+
output = ""
|
|
188
|
+
escaped = 0
|
|
189
|
+
quoted = 0
|
|
190
|
+
for (cursor = 1; cursor <= length(text); cursor++) {
|
|
191
|
+
char = substr(text, cursor, 1)
|
|
192
|
+
if (quoted) {
|
|
193
|
+
if (escaped) escaped = 0
|
|
194
|
+
else if (char == "\\") escaped = 1
|
|
195
|
+
else if (char == "\"") quoted = 0
|
|
196
|
+
continue
|
|
197
|
+
}
|
|
198
|
+
if (char == "\"") {
|
|
199
|
+
quoted = 1
|
|
200
|
+
continue
|
|
201
|
+
}
|
|
202
|
+
if (char == "/" && substr(text, cursor + 1, 1) == "/") break
|
|
203
|
+
output = output char
|
|
204
|
+
}
|
|
205
|
+
return output
|
|
206
|
+
}
|
|
207
|
+
function without_comment(text, output, cursor, char, escaped, quoted) {
|
|
208
|
+
output = ""
|
|
209
|
+
escaped = 0
|
|
210
|
+
quoted = 0
|
|
211
|
+
for (cursor = 1; cursor <= length(text); cursor++) {
|
|
212
|
+
char = substr(text, cursor, 1)
|
|
213
|
+
if (quoted) {
|
|
214
|
+
output = output char
|
|
215
|
+
if (escaped) escaped = 0
|
|
216
|
+
else if (char == "\\") escaped = 1
|
|
217
|
+
else if (char == "\"") quoted = 0
|
|
218
|
+
continue
|
|
219
|
+
}
|
|
220
|
+
if (char == "\"") {
|
|
221
|
+
quoted = 1
|
|
222
|
+
output = output char
|
|
223
|
+
continue
|
|
224
|
+
}
|
|
225
|
+
if (char == "/" && substr(text, cursor + 1, 1) == "/") break
|
|
226
|
+
output = output char
|
|
227
|
+
}
|
|
228
|
+
return output
|
|
229
|
+
}
|
|
230
|
+
function check_emit() {
|
|
231
|
+
if (emit_code !~ /event[[:space:]]*=/ && emit_code !~ /[(,][[:space:]]*event[[:space:]]*[,)]/)
|
|
232
|
+
printf "event|%s|%d\n", FILENAME, emit_line
|
|
233
|
+
if (emit_source ~ /"[^"\n]*\{[^"\n]*\}[^"\n]*"/)
|
|
234
|
+
printf "positional|%s|%d\n", FILENAME, emit_line
|
|
235
|
+
emit_code = ""
|
|
236
|
+
emit_source = ""
|
|
237
|
+
in_emit = 0
|
|
238
|
+
}
|
|
239
|
+
FNR == 1 {
|
|
240
|
+
depth = 0
|
|
241
|
+
test_depth = 0
|
|
242
|
+
cfg_test = 0
|
|
243
|
+
in_emit = 0
|
|
244
|
+
emit_parens = 0
|
|
245
|
+
emit_code = ""
|
|
246
|
+
emit_source = ""
|
|
247
|
+
}
|
|
248
|
+
{
|
|
249
|
+
code = code_only($0)
|
|
250
|
+
if (!in_emit && code ~ /^[[:space:]]*$/) next
|
|
251
|
+
opens = count_char(code, "\\{")
|
|
252
|
+
closes = count_char(code, "\\}")
|
|
253
|
+
if (test_depth > 0) {
|
|
254
|
+
depth += opens - closes
|
|
255
|
+
if (depth < test_depth) test_depth = 0
|
|
256
|
+
next
|
|
257
|
+
}
|
|
258
|
+
if (code ~ /^[[:space:]]*#\[cfg\(test\)\]/) {
|
|
259
|
+
if (opens > 0) {
|
|
260
|
+
test_depth = depth + 1
|
|
261
|
+
depth += opens - closes
|
|
262
|
+
if (depth < test_depth) test_depth = 0
|
|
263
|
+
} else cfg_test = 1
|
|
264
|
+
next
|
|
265
|
+
}
|
|
266
|
+
if (cfg_test) {
|
|
267
|
+
if (opens > 0) {
|
|
268
|
+
test_depth = depth + 1
|
|
269
|
+
depth += opens - closes
|
|
270
|
+
if (depth < test_depth) test_depth = 0
|
|
271
|
+
cfg_test = 0
|
|
272
|
+
} else if (code ~ /;/) cfg_test = 0
|
|
273
|
+
next
|
|
274
|
+
}
|
|
275
|
+
if (code ~ /(^|[^[:alnum:]_])(println|eprintln|dbg)!/)
|
|
276
|
+
printf "direct|%s|%d\n", FILENAME, FNR
|
|
277
|
+
if (!in_emit && match(code, /tracing::(error|warn|info|debug|trace)!/)) {
|
|
278
|
+
in_emit = 1
|
|
279
|
+
emit_line = FNR
|
|
280
|
+
emit_code = code
|
|
281
|
+
emit_source = without_comment($0)
|
|
282
|
+
macro = substr(code, RSTART)
|
|
283
|
+
emit_parens = count_char(macro, "\\(") - count_char(macro, "\\)")
|
|
284
|
+
} else if (in_emit) {
|
|
285
|
+
emit_code = emit_code " " code
|
|
286
|
+
emit_source = emit_source " " without_comment($0)
|
|
287
|
+
emit_parens += count_char(code, "\\(") - count_char(code, "\\)")
|
|
288
|
+
}
|
|
289
|
+
if (in_emit && emit_parens <= 0) check_emit()
|
|
290
|
+
depth += opens - closes
|
|
291
|
+
}
|
|
292
|
+
' "${rust_nontest[@]}")
|
|
293
|
+
fi
|
|
294
|
+
|
|
295
|
+
# ---------------------------------------------------------------------------
|
|
296
|
+
# 4. BLOCKING: console.log/debug/info/warn/error in agentsfleet/src non-test.
|
|
144
297
|
# ---------------------------------------------------------------------------
|
|
145
298
|
console_hits=0
|
|
146
299
|
if [[ ${#js_nontest[@]} -gt 0 ]]; then
|
|
@@ -155,7 +308,7 @@ if [[ ${#js_nontest[@]} -gt 0 ]]; then
|
|
|
155
308
|
fi
|
|
156
309
|
|
|
157
310
|
# ---------------------------------------------------------------------------
|
|
158
|
-
#
|
|
311
|
+
# 5. INFO: std.log.scoped outside src/logging/ (pre-migration to the named
|
|
159
312
|
# `log` module's scoped API). The audit no longer carves out src/auth/ —
|
|
160
313
|
# the named module is import-able from layer-isolated trees, so the
|
|
161
314
|
# portability exception is gone.
|
|
@@ -180,7 +333,7 @@ if [[ ${#scoped_eligible[@]} -gt 0 ]]; then
|
|
|
180
333
|
fi
|
|
181
334
|
|
|
182
335
|
# ---------------------------------------------------------------------------
|
|
183
|
-
#
|
|
336
|
+
# 6. INFO: err/warn logs without `error_code=` substring on the same line.
|
|
184
337
|
# Heuristic — captures the common case where an err/warn line should
|
|
185
338
|
# embed UZ-XXX-NNN per LOGGING_STANDARD §5.
|
|
186
339
|
# ---------------------------------------------------------------------------
|
|
@@ -198,16 +351,16 @@ if [[ ${#zig_nontest[@]} -gt 0 ]]; then
|
|
|
198
351
|
fi
|
|
199
352
|
|
|
200
353
|
# ---------------------------------------------------------------------------
|
|
201
|
-
#
|
|
354
|
+
# 7. Promote INFO to BLOCK in --strict mode.
|
|
202
355
|
# ---------------------------------------------------------------------------
|
|
203
356
|
if [[ $STRICT -eq 1 && $INFO_COUNT -gt 0 ]]; then
|
|
204
357
|
fail "--strict: $INFO_COUNT informational findings promoted to blocking"
|
|
205
358
|
fi
|
|
206
359
|
|
|
207
360
|
# ---------------------------------------------------------------------------
|
|
208
|
-
#
|
|
361
|
+
# 8. Verdict.
|
|
209
362
|
# ---------------------------------------------------------------------------
|
|
210
|
-
ok "scanned ${#FILES[@]} files; std.debug.print=$debug_print_hits console.*=$console_hits std.log.scoped=$scoped_hits missing-error_code=$missing_code_hits"
|
|
363
|
+
ok "scanned ${#FILES[@]} files; std.debug.print=$debug_print_hits rust-direct=$rust_direct_hits rust-missing-event=$rust_missing_event_hits rust-positional=$rust_positional_hits console.*=$console_hits std.log.scoped=$scoped_hits missing-error_code=$missing_code_hits"
|
|
211
364
|
if [[ $FAIL -ne 0 ]]; then
|
|
212
365
|
printf "\n🔴 LOGGING GATE: blocking violations. See dispatch/write_any.md (Logging Gate).\n" >&2
|
|
213
366
|
exit 1
|
package/dispatch/write_any.md
CHANGED
|
@@ -127,6 +127,7 @@ shebang and is the authoritative check.
|
|
|
127
127
|
**Triggers** — every `Edit`/`Write` that adds, removes, or changes a log emit:
|
|
128
128
|
|
|
129
129
|
- `*.zig` outside `vendor/`/`third_party/`/`.zig-cache/`/`*_test.zig` — `std.log.*`, `std.debug.print`, raw stderr writes, calls into the `log` named module (source: `src/logging/mod.zig`).
|
|
130
|
+
- `*.rs` under `rustd/` outside `tests/`, `benches/`, and `#[cfg(test)]` items — `tracing::*`, `println!`, `eprintln!`, and `dbg!`.
|
|
130
131
|
- `*.ts`/`*.tsx`/`*.js`/`*.jsx` outside `vendor/`/`node_modules/`/`*.test.*`/`*.spec.*` — `console.*`, custom logger calls.
|
|
131
132
|
- `*.sh` outside generated dirs — `echo`/`printf` to `&2`.
|
|
132
133
|
|
|
@@ -145,10 +146,11 @@ shebang and is the authoritative check.
|
|
|
145
146
|
| Pattern | Rule |
|
|
146
147
|
|---|---|
|
|
147
148
|
| New `logging.scoped(.tag)` call | Scope is a Zig enum literal — adding a new tag is freeform. `event` must be snake_case `verb_noun`. |
|
|
149
|
+
| New Rust `tracing::<level>!` call | `event = "verb_noun"` or the equivalent `event` field shorthand is mandatory; values are structured fields hoisted into locals. |
|
|
148
150
|
| New `err`/`warn` log mapping to a domain failure | registry-scheme `error_code=` field required (agentsfleet: `UZ-XXX-NNN`). Registry entry must land in same commit. |
|
|
149
151
|
| Per-iteration / hot-loop log | Use `debug` (hidden by default), not `info`. |
|
|
150
|
-
| `info` level | No allow-list — `info` is open (§4 rule 2). Two checks instead: a boundary-crossing operation needs its `_started`/`_completed`\|`_failed` pair on every exit path (rule 1), and a per-iteration path is `debug` (rule 3). |
|
|
151
|
-
| `console.log`/`std.debug.print
|
|
152
|
+
| `info` level | No allow-list — `info` is open (§4 rule 2). Two checks instead: a boundary-crossing operation needs its `_started`/`_completed`\|`_failed` pair on every exit path (rule 1), and a per-iteration path is `debug` (rule 3). Hot-poll boundary pairs remain complete but use `debug`. |
|
|
153
|
+
| `console.log`/`std.debug.print`/Rust print macro in non-test source | Forbidden. Convert to the structured logger or delete before commit. |
|
|
152
154
|
| `std.log.scoped` outside `src/logging/` | Migration target. The `log` named module's `scoped` API is the only non-test entry point; flip the file's alias and migrate every call site in the same commit. |
|
|
153
155
|
| `msg=` field | ≤ 300 chars. Stack traces emit as separate `event=stack_trace` debug record. |
|
|
154
156
|
| Multi-line values | Newlines must be `\n` literal (two chars), not raw newline byte. |
|
|
@@ -173,7 +175,7 @@ Full multi-line block fires when a sub-rule reports a violation:
|
|
|
173
175
|
|
|
174
176
|
```
|
|
175
177
|
LOGGING GATE: <file>
|
|
176
|
-
docs/LOGGING_STANDARD.md sections consulted: §3 (wire format), §4 (severity), §5 (error codes), §6 (PII), §7 (
|
|
178
|
+
docs/LOGGING_STANDARD.md sections consulted: §3 (wire format), §4 (severity), §5 (error codes), §6 (PII), §7 (Zig binding) | §8 (TS binding) | §8A (Rust binding), §10A (tightenings)
|
|
177
179
|
Wire format: <logfmt ✓ | violation: <where>>
|
|
178
180
|
Required keys: <ts_ms,level,scope,event present ✓ | violation: <missing>>
|
|
179
181
|
Severity choice: <within rules ✓ | violation: <e.g. info on per-iteration path>>
|
|
@@ -188,13 +190,13 @@ LOGGING GATE: <file>
|
|
|
188
190
|
|
|
189
191
|
> [DETERMINISTIC → LOG]
|
|
190
192
|
|
|
191
|
-
`logging.sh` walks the **full `src/` + `agentsfleet/src/` working tree
|
|
193
|
+
`logging.sh` walks the **full `src/` + `rustd/` + `agentsfleet/src/` working tree**. The staged mode reads the index, so a fix staged in pre-commit satisfies the check on the same hook run. `--staged` is preserved as an opt-in narrowing mode for iterative dev.
|
|
192
194
|
|
|
193
195
|
#### End-of-turn audit
|
|
194
196
|
|
|
195
197
|
> [DETERMINISTIC → LOG]
|
|
196
198
|
|
|
197
|
-
`audits/logging.sh` runs as part of `make lint`. It greps `std.log.*`, `console
|
|
199
|
+
`audits/logging.sh` runs as part of `make lint`. It greps `std.log.*`, Rust `tracing` and direct-print macros, `console.*`, and raw stderr writes. It does NOT see calls through the Zig `log` named module's `scoped()`, so a clean run is not evidence that §4/§5 hold everywhere. Mechanical today: direct Zig/Rust/TypeScript diagnostics, Rust's required `event` field and positional-format ban, `std.log.scoped` outside `src/lib/logging/`, and legacy Zig error-code hints. Reviewer-side: severity choice, entry/exit pairing, registry applicability, inline Rust expression hoisting, and Personally Identifiable Information (PII) spot-checks.
|
|
198
200
|
|
|
199
201
|
#### Family
|
|
200
202
|
|
package/docs/LOGGING_STANDARD.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# LOGGING_STANDARD — wire format and discipline for structured logs
|
|
2
2
|
|
|
3
|
-
The contract between code that emits log records and the systems (humans, collectors, dashboards) that consume them. Cross-language: applies to every Zig, TypeScript, JavaScript, and shell call site that emits to `stderr`/`stdout`.
|
|
3
|
+
The contract between code that emits log records and the systems (humans, collectors, dashboards) that consume them. Cross-language: applies to every Zig, Rust, TypeScript, JavaScript, and shell call site that emits to `stderr`/`stdout`.
|
|
4
4
|
|
|
5
5
|
Pre-design rules, decisive defaults, and the anti-patterns each rule exists to prevent. Every rule has a one-line "why" so you can judge edge cases.
|
|
6
6
|
|
|
@@ -9,12 +9,13 @@ Pre-design rules, decisive defaults, and the anti-patterns each rule exists to p
|
|
|
9
9
|
Triggers on every `Edit`/`Write` that adds, removes, or changes a log emit:
|
|
10
10
|
|
|
11
11
|
- `*.zig` outside `vendor/`/`third_party/`/`.zig-cache/` — `std.log.*`, `std.debug.print`, `std.io.getStdErr().writer().print`, any helper in `src/lib/logging/`.
|
|
12
|
+
- `*.rs` under `rustd/` outside test and benchmark scopes — `tracing::{error,warn,info,debug,trace}!`, `println!`, `eprintln!`, and `dbg!`.
|
|
12
13
|
- `*.ts`/`*.tsx`/`*.js`/`*.jsx` outside `vendor/`/`node_modules/` — `console.*`, custom logger calls.
|
|
13
14
|
- `*.sh` outside generated directories — `echo`, `printf` to `&2`.
|
|
14
15
|
|
|
15
16
|
Out of scope (explicitly):
|
|
16
17
|
|
|
17
|
-
- Test-only diagnostic prints inside `*_test.zig`, `*.test.ts`, `*.spec.ts`. Tests render to humans, not collectors; gates ignore.
|
|
18
|
+
- Test-only diagnostic prints inside `*_test.zig`, Rust `tests/` and `benches/` trees or `#[cfg(test)]` items, `*.test.ts`, and `*.spec.ts`. Tests render to humans, not collectors; gates ignore.
|
|
18
19
|
- Build/release scripts — the harness's own `audits/*`, `audits/release-*`, and any governance-repo tooling — toolchain output, not application logs.
|
|
19
20
|
- Generated framework noise (Next.js startup banners, Bun runtime warnings) — out of our control.
|
|
20
21
|
|
|
@@ -123,6 +124,12 @@ a per-chunk callback: these flood collectors and drown the records that matter.
|
|
|
123
124
|
`debug` is not a lesser level, it is the level with a volume switch. A sweep that
|
|
124
125
|
reaps 4,000 workspaces logs each one at `debug` and its outcome at `info`.
|
|
125
126
|
|
|
127
|
+
Boundary pairing and per-iteration severity both apply to hot polls. A lease poll
|
|
128
|
+
that crosses Redis and Postgres on every iteration still emits its complete
|
|
129
|
+
`_started` and `_completed`/`_failed` pair, but emits both records at `debug`.
|
|
130
|
+
The pair satisfies rule 1; the level satisfies rule 3. A coarser poll-loop start,
|
|
131
|
+
stop, or aggregate outcome may remain `info`.
|
|
132
|
+
|
|
126
133
|
### `err` and `warn` are unconditional
|
|
127
134
|
|
|
128
135
|
> [UNENFORCED → "always emitted" is a runtime filter-configuration property; nothing greps for a suppressed err/warn path]
|
|
@@ -288,6 +295,48 @@ class AgentError extends Error {
|
|
|
288
295
|
|
|
289
296
|
Throw-style modules `throw new AgentError({...})`. Result-style modules return `{ ok: false, error: new AgentError({...}) }`. Render path picks human or JSON based on the runtime mode.
|
|
290
297
|
|
|
298
|
+
## §8A · Per-language binding — Rust (`rustd`)
|
|
299
|
+
|
|
300
|
+
> [DETERMINISTIC → LOG]
|
|
301
|
+
|
|
302
|
+
Rust uses the existing `tracing` crate and emits the same logical fields as §3.
|
|
303
|
+
Every production emit supplies an `event` field, explicitly or through tracing's
|
|
304
|
+
field shorthand, whose value is a snake_case `verb_noun`. Fields are structured
|
|
305
|
+
arguments, never positional message formatting.
|
|
306
|
+
|
|
307
|
+
> [JUDGMENT → RUST-STYLE]
|
|
308
|
+
|
|
309
|
+
Every `warn` or `error` that maps to a registry code supplies
|
|
310
|
+
`error_code = code` per §5. Hoist field expressions into locals before the macro
|
|
311
|
+
call. The log bridge duplicates field expressions, and LLVM coverage
|
|
312
|
+
instrumentation scores the dead copy; locals preserve evaluation and coverage
|
|
313
|
+
behavior. A trailing string message is §3's optional `msg` and is always the
|
|
314
|
+
last macro argument. Prefer fields alone when they already explain the event.
|
|
315
|
+
|
|
316
|
+
```rust
|
|
317
|
+
let code = error_code::INTERNAL_DB_QUERY.as_str();
|
|
318
|
+
let id = runner.as_str();
|
|
319
|
+
let reason = error.to_string();
|
|
320
|
+
let event = "runner_lease_failed";
|
|
321
|
+
tracing::warn!(error_code = code, runner_id = id, reason, event);
|
|
322
|
+
```
|
|
323
|
+
|
|
324
|
+
> [JUDGMENT → EVENT-COMPAT]
|
|
325
|
+
|
|
326
|
+
Ports preserve existing event-name bytes by default. A Rust replacement for a
|
|
327
|
+
Zig operation keeps the Zig event spelling so dashboards and alerts continue to
|
|
328
|
+
match. Renaming is a separate, intentional observability migration.
|
|
329
|
+
|
|
330
|
+
> [DETERMINISTIC → LOG]
|
|
331
|
+
|
|
332
|
+
**Anti-patterns flagged by `logging.sh`:**
|
|
333
|
+
|
|
334
|
+
| Pattern | Why banned | Fix |
|
|
335
|
+
|---|---|---|
|
|
336
|
+
| `println!`, `eprintln!`, or `dbg!` in non-test source | No level, event, or structured fields. | Convert to a `tracing` emit or delete. |
|
|
337
|
+
| `tracing::<level>!` without `event = ...` | Drops the stable §3 event key. | Add the byte-stable snake_case event field. |
|
|
338
|
+
| Positional formatting in a `tracing` message | Hides values inside text and defeats field queries. | Hoist values and emit named fields. |
|
|
339
|
+
|
|
291
340
|
## §9 · Pretty-printer (dev-only render variant)
|
|
292
341
|
|
|
293
342
|
In production: every record on the wire is logfmt. No exceptions.
|
|
@@ -330,10 +379,10 @@ Failure modes the audit script and reviewer must close. These are **not aspirati
|
|
|
330
379
|
|
|
331
380
|
| # | Rationalization | Closure |
|
|
332
381
|
|---|---|---|
|
|
333
|
-
| L1 | "Temporary debug print, I'll remove later" | `logging.sh` greps `std.debug.print` and `console.log` / `console.debug` / `console.info` in non-test source unconditionally. Found in commit → gate fails. No "temporary" carve-out. |
|
|
382
|
+
| L1 | "Temporary debug print, I'll remove later" | `logging.sh` greps `std.debug.print`, Rust `println!` / `eprintln!` / `dbg!`, and `console.log` / `console.debug` / `console.info` in non-test source unconditionally. Found in commit → gate fails. No "temporary" carve-out. |
|
|
334
383
|
| L2 | "`std.log.scoped` is fine, `obs.scoped` is just a wrapper" | `std.log.scoped` is **forbidden** in `src/**/*.zig` outside `src/lib/logging/`. Only `obs.scoped` is callable. Audit flags every `std.log.` call site. |
|
|
335
384
|
| L3 | "I added `error_code=UZ-NEW-001` — registry entry coming next commit" | The registry entry **must land in the same commit** as the first reference. `error-codes.sh` runs against the staged diff; missing entry = blocking. |
|
|
336
|
-
| L4 | "This per-iteration event matters for debugging — `info`-level" | Per-iteration and per-row paths are `debug`,
|
|
385
|
+
| L4 | "This per-iteration event matters for debugging — `info`-level" | Per-iteration and per-row paths are `debug`, including mandatory boundary pairs on hot polls. `debug` is the level with a volume switch. This closes the ONLY `info` prohibition — there is no allow-list of event names and none may be reintroduced (§4). The inverse rationalization is closed too: "this operation is obviously fine, no need to log it" does not survive §4 rule 1, which requires a `_started`/`_completed`\|`_failed` pair on every boundary-crossing operation. Reviewer checks the pair and the loop, never the spelling. |
|
|
337
386
|
| L5 | "Operator needs the full stack trace in `msg=`" | `msg=` capped at 300 chars; total fields per record capped at 15. Stack traces emit as a separate `event=stack_trace` record at `debug` level, correlated by `correlation_id`, not stuffed into `msg`. |
|
|
338
387
|
| L6 | "Embedded newlines because I copy-pasted output" | Audit greps for raw newline byte inside quoted logfmt values. Must be `\n` literal (two chars). |
|
|
339
388
|
| L7 | "Auto-mode is on, the gate block is ceremony" | **Auto-mode does NOT cover gate skips.** Skip without an explicit user-given override = automatic violation. No size threshold lets an edit bypass the gate. |
|
|
@@ -348,6 +397,9 @@ These are enforced by `logging.sh` (mechanical) and the dispatch façade (`dispa
|
|
|
348
397
|
|---|---|
|
|
349
398
|
| Free-form English log lines | Not greppable, not parseable, no event tag. |
|
|
350
399
|
| `std.debug.print` outside tests | No level, no scope, no fields — dev debugging that escaped to main. |
|
|
400
|
+
| Rust `println!`, `eprintln!`, or `dbg!` outside tests | No level, event, or structured fields — dev diagnostics that escaped to main. |
|
|
401
|
+
| Rust `tracing` emit without `event` | Drops the stable event key required by §3. |
|
|
402
|
+
| Positional formatting in a Rust `tracing` emit | Values become opaque message text instead of queryable fields. |
|
|
351
403
|
| `console.log` in `agentsfleet` source | Bypasses logger, breaks `--json` mode, violates `dispatch/write_ts_adhere_bun.md` §10. |
|
|
352
404
|
| Logging on hot per-iteration paths at `info` | Floods collectors. Use `debug` (gated off by default). |
|
|
353
405
|
| `error_code=` missing on `err`/`warn` mapping to registry codes | Breaks traceability; future operator can't link log to docs. |
|
|
@@ -371,6 +423,7 @@ immediately preceding the edit. Generic "scope creep" is not a valid reason —
|
|
|
371
423
|
- `dispatch/write_ts_adhere_bun.md` §10 — banned `console.log` in TS/JS source. Cross-referenced by `logging.sh`.
|
|
372
424
|
- `dispatch/write_ts_adhere_bun.md` §9 — module-level error style (throw vs Result) for `agentsfleet`.
|
|
373
425
|
- `dispatch/write_zig.md` — Zig discipline umbrella; this doc's §7 is the logging-specific layer.
|
|
426
|
+
- `dispatch/write_rust.md` — Rust discipline umbrella; this doc's §8A is the logging-specific layer.
|
|
374
427
|
- `LIFECYCLE_PATTERNS.md` — orthogonal: ownership/cleanup of structs, including allocator wiring for the thread-local log buffer.
|
|
375
428
|
- M42_002 redaction harness (`src/runner/engine/runner_progress.zig`) — secret-redaction precondition; this doc's §6 inherits.
|
|
376
429
|
- Universal rules (RULE UFS, RULE TGU, RULE PRI, RULE FLL, RULE ORP, RULE TST-NAM) live in `docs/greptile-learnings/RULES.md`.
|
|
@@ -388,10 +441,12 @@ its line covers the clauses under it until the next heading.
|
|
|
388
441
|
|
|
389
442
|
| Tag | Meaning | Where the verdict comes from |
|
|
390
443
|
|---|---|---|
|
|
391
|
-
| `[DETERMINISTIC → LOG]` | a script decides; no talking past it | `audits/logging.sh` —
|
|
444
|
+
| `[DETERMINISTIC → LOG]` | a script decides; no talking past it | `audits/logging.sh` — direct Zig/Rust/TypeScript diagnostics and malformed Rust tracing emits in non-test source block; missing `error_code=` on `std.log.{err,warn}` reports as INFO |
|
|
392
445
|
| `[JUDGMENT → BOUNDARY]` | is this operation boundary-crossing, and is its `_started`/`_completed`\|`_failed` pair complete | the agent at write time; the reviewer at `/review` |
|
|
393
446
|
| `[JUDGMENT → REDACT]` | is this value a secret, or the label of one | the agent at write time; the M42_002 harness covers only executor-child output |
|
|
394
447
|
| `[JUDGMENT → MSG-REVIEW]` | does this `msg=` carry a credential | the reviewer; no length or content check runs today |
|
|
395
448
|
| `[JUDGMENT → TS-STYLE]` | throw-style or Result-style for this module | `dispatch/write_ts_adhere_bun.md` §9, agent-decided |
|
|
449
|
+
| `[JUDGMENT → RUST-STYLE]` | does a Rust failure map to a registry code, and are field expressions safely hoisted | the agent at write time; the reviewer at `/review` |
|
|
450
|
+
| `[JUDGMENT → EVENT-COMPAT]` | does a language port preserve the event bytes consumed by existing dashboards | the agent compares old and new event constants; the reviewer confirms intentional renames |
|
|
396
451
|
| `[JUDGMENT → SECTION-SCAN]` | which sections of this file the current sub-task needs | the agent; `audits/doc-read.sh` records façade reads, not delegated-doc reads |
|
|
397
452
|
| `[UNENFORCED → reason]` | acknowledged prose; the reason states why no check exists | nothing — that is the point of the class |
|
|
@@ -122,6 +122,25 @@ surfaces:
|
|
|
122
122
|
|
|
123
123
|
Orly owns policy and invokes these commands. The repository owns what they do.
|
|
124
124
|
|
|
125
|
+
## Usage telemetry
|
|
126
|
+
|
|
127
|
+
Orly keeps usage telemetry off until a person enables anonymous collection in
|
|
128
|
+
an interactive run. Hooks, Continuous Integration (CI), and other automated
|
|
129
|
+
runs never ask for consent. Off writes no event and makes no network request.
|
|
130
|
+
|
|
131
|
+
Anonymous events enter a bounded outbound spool under the agentsfleet state
|
|
132
|
+
root. After each append, Orly launches a detached sync attempt and lets the
|
|
133
|
+
command exit. The sync sends at most 100 events and exits without a fetch when
|
|
134
|
+
another attempt ran within five minutes. A successful response removes the
|
|
135
|
+
acknowledged prefix. Spool maintenance drops records older than seven days and
|
|
136
|
+
keeps the file at or below 10 MiB, so an unavailable destination cannot create
|
|
137
|
+
an unbounded local log. Orly runs no telemetry daemon.
|
|
138
|
+
|
|
139
|
+
PostHog owns product-event storage and funnel analysis. Orly sends no source,
|
|
140
|
+
path, repository, branch, argument, output, environment, hostname, username, or
|
|
141
|
+
raw error value. A future Insights Fleet may query PostHog with a private key;
|
|
142
|
+
the Fleet is an analysis consumer and never the public ingestion boundary.
|
|
143
|
+
|
|
125
144
|
## Evidence
|
|
126
145
|
|
|
127
146
|
`orly verify --all --write-evidence` records the source commit and every check
|
package/docs/RULE_ENFORCEMENT.md
CHANGED
|
@@ -6,7 +6,7 @@ regenerates this file and fails on any difference. To update it, run
|
|
|
6
6
|
|
|
7
7
|
| Rule document | enforced by | judged | acknowledged | classified | trigger |
|
|
8
8
|
|---|---|--:|--:|--:|---|
|
|
9
|
-
| `docs/LOGGING_STANDARD.md` | `LOG` → `logging.sh` |
|
|
9
|
+
| `docs/LOGGING_STANDARD.md` | `LOG` → `logging.sh` | 11 | 13 | 28/28 | mechanical |
|
|
10
10
|
| `docs/REST_API_DESIGN_GUIDELINES.md` | — | 0 | 0 | 0/74 | mechanical |
|
|
11
11
|
| `docs/SCHEMA_CONVENTIONS.md` | — | 0 | 0 | 0/16 | mechanical |
|
|
12
12
|
| `docs/DOCUMENTATION_RULES.md` | — | 0 | 0 | 0/22 | latent |
|
|
@@ -14,7 +14,7 @@ regenerates this file and fails on any difference. To update it, run
|
|
|
14
14
|
| `docs/CHANGELOG_VOICE.md` | — | 0 | 0 | 0/3 | latent |
|
|
15
15
|
| `docs/VERIFY_TIERS.md` | — | 0 | 0 | 0/4 | uncited |
|
|
16
16
|
| `docs/greptile-learnings/RULES.md` | — | 0 | 0 | 0/114 | mechanical |
|
|
17
|
-
| **corpus** | | | | **
|
|
17
|
+
| **corpus** | | | | **28/283** | |
|
|
18
18
|
|
|
19
19
|
## How to read this
|
|
20
20
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@agentsfleet/orly",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.7.0",
|
|
4
4
|
"description": "Repository-scoped engineering harness: renders agent rules, materialises the gates that enforce them, and proves the PR boundary \u2014 for any coding agent.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"type": "module",
|
|
@@ -13,6 +13,12 @@ description: |
|
|
|
13
13
|
|
|
14
14
|
# orly-babysit-prs
|
|
15
15
|
|
|
16
|
+
At invocation start, record the consent-gated usage signal without blocking the skill:
|
|
17
|
+
|
|
18
|
+
```bash
|
|
19
|
+
command -v orly >/dev/null 2>&1 && ORLY_INVOCATION=skill orly skill-event orly-babysit-prs >/dev/null 2>&1 || true
|
|
20
|
+
```
|
|
21
|
+
|
|
16
22
|
Review bots (greptile on GitHub; greptile or another review bot on GitLab)
|
|
17
23
|
post auto-reviews **asynchronously**. They land as PR review comments / MR
|
|
18
24
|
discussion threads, not check runs, so `gh pr checks --watch` (or the glab
|
|
@@ -11,6 +11,12 @@ description: |
|
|
|
11
11
|
|
|
12
12
|
# orly-spec-new
|
|
13
13
|
|
|
14
|
+
At invocation start, record the consent-gated usage signal without blocking the skill:
|
|
15
|
+
|
|
16
|
+
```bash
|
|
17
|
+
command -v orly >/dev/null 2>&1 && ORLY_INVOCATION=skill orly skill-event orly-spec-new >/dev/null 2>&1 || true
|
|
18
|
+
```
|
|
19
|
+
|
|
14
20
|
A spec is the **rulebook** the executing agent (Orly) plans and ships from. This
|
|
15
21
|
skill makes me author one whose Pull Request (PR) lands **deterministic, review-clean, and
|
|
16
22
|
reported** — without a 20-questions loop. It is written for my reasoning, not as
|
|
@@ -11,6 +11,12 @@ description: >
|
|
|
11
11
|
|
|
12
12
|
# Write Integration Test
|
|
13
13
|
|
|
14
|
+
At invocation start, record the consent-gated usage signal without blocking the skill:
|
|
15
|
+
|
|
16
|
+
```bash
|
|
17
|
+
command -v orly >/dev/null 2>&1 && ORLY_INVOCATION=skill orly skill-event orly-write-integration-test >/dev/null 2>&1 || true
|
|
18
|
+
```
|
|
19
|
+
|
|
14
20
|
Service-layer tests that prove real wiring works under real failure modes. Sits between unit tests (`make test`) and browser E2E (gstack `/qa`, `/e2e-qa-playwright`).
|
|
15
21
|
|
|
16
22
|
> **What this guarantees / what it doesn't.** Integration tests don't prove "zero bugs
|
|
@@ -11,6 +11,12 @@ description: >
|
|
|
11
11
|
|
|
12
12
|
# Write Unit Test
|
|
13
13
|
|
|
14
|
+
At invocation start, record the consent-gated usage signal without blocking the skill:
|
|
15
|
+
|
|
16
|
+
```bash
|
|
17
|
+
command -v orly >/dev/null 2>&1 && ORLY_INVOCATION=skill orly skill-event orly-write-unit-test >/dev/null 2>&1 || true
|
|
18
|
+
```
|
|
19
|
+
|
|
14
20
|
Generate tests that catch bugs before they ship. Tests run *during* implementation, not after.
|
|
15
21
|
|
|
16
22
|
> **What this guarantees / what it doesn't.** No test suite proves "zero bugs in
|