harnex 0.9.0 → 0.10.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +122 -0
- data/README.md +38 -38
- data/TECHNICAL.md +17 -15
- data/docs/codex-appserver.md +16 -13
- data/docs/configuration.md +11 -4
- data/docs/dispatch-telemetry.md +189 -112
- data/docs/events.md +2 -3
- data/guides/01_dispatch.md +29 -32
- data/guides/04_monitoring.md +4 -4
- data/lib/harnex/artifact_report.rb +455 -6
- data/lib/harnex/cli.rb +11 -3
- data/lib/harnex/commands/artifact_report.rb +8 -7
- data/lib/harnex/commands/doctor.rb +1 -1
- data/lib/harnex/commands/run.rb +22 -37
- data/lib/harnex/commands/status.rb +0 -2
- data/lib/harnex/commands/telemetry.rb +110 -0
- data/lib/harnex/commands/wait.rb +0 -1
- data/lib/harnex/config.rb +6 -2
- data/lib/harnex/core.rb +200 -14
- data/lib/harnex/dispatch_history.rb +3 -3
- data/lib/harnex/retention.rb +13 -4
- data/lib/harnex/runtime/session.rb +317 -160
- data/lib/harnex/telemetry_reconciler.rb +494 -0
- data/lib/harnex/terminal_status.rb +7 -20
- data/lib/harnex/version.rb +2 -2
- data/lib/harnex.rb +2 -0
- metadata +4 -2
data/docs/dispatch-telemetry.md
CHANGED
|
@@ -15,9 +15,8 @@ pre-v2 envelope-less summaries may coexist and remain readable/skippable.
|
|
|
15
15
|
|
|
16
16
|
```text
|
|
17
17
|
harnex run codex --meta '{"model":"gpt-5.3-codex","effort":"high","predicted":{"input_tokens":[200000,800000]}}'
|
|
18
|
-
harnex run
|
|
19
|
-
harnex artifact-report
|
|
20
|
-
harnex run pi --artifact-report .harnex/reports/pi-i-61.json --require-artifact-report --context 'Finalize $HARNEX_ARTIFACT_REPORT_PATH and validate it with --final'
|
|
18
|
+
harnex run pi --context 'Implement the task; Harnex writes the final receipt'
|
|
19
|
+
harnex run pi --artifact-report .harnex/receipts/pi-r-64.json --context 'Optionally write review claims to $HARNEX_ARTIFACT_CLAIMS_PATH'
|
|
21
20
|
harnex run pi --project-id harnex --queue-id queue-005 --entry-id SP-4 --phase implement --intent queue-work --require-attribution
|
|
22
21
|
harnex run pi --orchestration-run-id queue-005 --orchestration-generation-id gen-1 --orchestration-role worker
|
|
23
22
|
harnex orchestration report --dispatch .harnex/dispatch.jsonl --run-id queue-005 --json
|
|
@@ -25,23 +24,22 @@ harnex orchestration report --dispatch .harnex/dispatch.jsonl --run-id queue-005
|
|
|
25
24
|
|
|
26
25
|
- `--meta JSON` must be a JSON object. The parsed object is echoed verbatim on
|
|
27
26
|
the `started.meta` event.
|
|
28
|
-
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
`
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
- `harnex artifact-report
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
machine-readable diagnostics without echoing report payloads or transcripts.
|
|
27
|
+
- Every run allocates a harness-owned `harnex.artifact_report.v1` receipt path.
|
|
28
|
+
The default is a repo-keyed file under `~/.local/state/harnex/receipts/`; live status
|
|
29
|
+
and the end row expose it as `artifact_report_path` / `artifact_report.path`.
|
|
30
|
+
- `--artifact-report PATH` overrides that output destination.
|
|
31
|
+
`--validation-report PATH` remains a compatibility alias. The worker receives
|
|
32
|
+
the final path as `HARNEX_ARTIFACT_REPORT_PATH` /
|
|
33
|
+
`HARNEX_VALIDATION_REPORT_PATH`, but Harnex owns and overwrites that file.
|
|
34
|
+
- `HARNEX_ARTIFACT_CLAIMS_PATH` is a separate optional input for bounded
|
|
35
|
+
`summary`, `verdict`, and `P1`/`P2`/`P3` counts. Claims are never acceptance
|
|
36
|
+
evidence. Stale, malformed, or oversized claims are ignored.
|
|
37
|
+
- `--require-artifact-report` is retained for script compatibility and exports
|
|
38
|
+
`HARNEX_ARTIFACT_REPORT_REQUIRED=1`, but it no longer needs an explicit path
|
|
39
|
+
or model-authored report. Receipt write failure is fail-closed for every run.
|
|
40
|
+
- `harnex artifact-report validate PATH --final` validates the generated
|
|
41
|
+
receipt without echoing its contents. `init` and the old final-validation
|
|
42
|
+
contract remain available for legacy/manual v1 documents.
|
|
45
43
|
- `--project-id`, `--queue-id`, `--entry-id`, `--entry-title`, `--phase`,
|
|
46
44
|
`--tier`, `--issue`, `--plan`, `--intent`, `--model`, `--effort`,
|
|
47
45
|
`--parent-dispatch-id`, `--parent-attempt-id`, and `--attempt-kind` are
|
|
@@ -58,8 +56,9 @@ harnex orchestration report --dispatch .harnex/dispatch.jsonl --run-id queue-005
|
|
|
58
56
|
- `--require-attribution` fails before launch unless `project_id`, `phase`,
|
|
59
57
|
`intent`, and at least one of `queue_id` / `entry_id` / `issue` / `plan` are
|
|
60
58
|
present through first-class flags or `--meta`.
|
|
61
|
-
-
|
|
62
|
-
|
|
59
|
+
- The canonical stream is the only destination a dispatch writes telemetry to.
|
|
60
|
+
There is no second copy and no flag to request one; a flag asking for a
|
|
61
|
+
mirror file is rejected as unknown. See CHANGELOG for the removal note.
|
|
63
62
|
- The v2 `dispatch_end` combines the history envelope (`schema_version`,
|
|
64
63
|
`record_type`, id/status/timing fields) with all rich telemetry sections.
|
|
65
64
|
A default dispatch therefore adds exactly two rows, not separate thin and
|
|
@@ -67,15 +66,51 @@ harnex orchestration report --dispatch .harnex/dispatch.jsonl --run-id queue-005
|
|
|
67
66
|
|
|
68
67
|
Use `harnex history --json | jq .` for pipelines over the repo-local log.
|
|
69
68
|
|
|
69
|
+
## Canonical assertion and reconciliation
|
|
70
|
+
|
|
71
|
+
`harnex telemetry assert-canonical` is the read-only drift gate for the
|
|
72
|
+
canonical dispatch stream. Without sources it performs structural validation;
|
|
73
|
+
with explicit `--source PATH` inputs it also reports missing or conflicting rich
|
|
74
|
+
end rows and exits non-zero until the canonical stream is clean.
|
|
75
|
+
|
|
76
|
+
`harnex telemetry reconcile` runs the same analysis. It is a dry-run by default;
|
|
77
|
+
only `--apply` appends missing rich end rows, and then only after the canonical
|
|
78
|
+
stream and all source candidates have been parsed and conflict-checked. Apply
|
|
79
|
+
uses one append lock, rechecks under that lock, and is idempotent.
|
|
80
|
+
|
|
81
|
+
```text
|
|
82
|
+
harnex telemetry assert-canonical [--canonical PATH | --global] [--source PATH ...] [--json]
|
|
83
|
+
harnex telemetry reconcile [--canonical PATH | --global] --source PATH [--source PATH ...] [--apply] [--json]
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
The default canonical path is the repo-local `.harnex/dispatch.jsonl`, or the
|
|
87
|
+
global dispatch stream outside a git repo. `--canonical` and `--global` are
|
|
88
|
+
mutually exclusive. Sources are never discovered automatically: each `--source`
|
|
89
|
+
must be a file or directory. Directory scans consider regular `.json` and
|
|
90
|
+
`.jsonl` files, skip `.git`, symlinks, and the resolved canonical path, and
|
|
91
|
+
ignore unrelated JSON that does not match a rich Harnex dispatch end shape.
|
|
92
|
+
|
|
93
|
+
Mixed-era history is valid input. Legacy v1 thin rows, pre-v2 envelope-less rich
|
|
94
|
+
summaries, and v2 start/end rows may coexist. Open v2 starts are tolerated
|
|
95
|
+
because a running or interrupted dispatch may not have an end row yet. Identity
|
|
96
|
+
checks normalize equivalent timestamp offsets, and conflicts fail closed rather
|
|
97
|
+
than choosing a winner.
|
|
98
|
+
|
|
99
|
+
Reports are bounded and redacted: they contain counts, statuses, identities, and
|
|
100
|
+
path:line diagnostics, not raw telemetry payloads, prompts, claims, command
|
|
101
|
+
text, or rich sections. The commands never rewrite, delete, sort, migrate,
|
|
102
|
+
clean source files, reintroduce mirrors, or perform source discovery on their
|
|
103
|
+
own.
|
|
104
|
+
|
|
70
105
|
## Metadata and prediction contract
|
|
71
106
|
|
|
72
107
|
The v2 `dispatch_end` always has `meta`, `predicted`, `actual`, `agent`,
|
|
73
|
-
`usage`, `context`, `attribution`, `outcome`, `attempt`,
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
108
|
+
`usage`, `context`, `attribution`, `outcome`, `attempt`, `reliability`,
|
|
109
|
+
`artifact_report`, `receipt`, `observed`, and `validation` blocks. When queue
|
|
110
|
+
attribution fields are provided, harnex also adds a top-level `queue` block.
|
|
111
|
+
When orchestration fields are provided, harnex adds a top-level
|
|
112
|
+
`orchestration` block. A sanitized `claims` block is additive only when the
|
|
113
|
+
worker supplied one.
|
|
79
114
|
|
|
80
115
|
Harnex-owned `meta` fields are always populated when derivable: `id`,
|
|
81
116
|
`tmux_session`, `description`, `started_at`, `ended_at`, `harness`,
|
|
@@ -240,16 +275,14 @@ into this block.
|
|
|
240
275
|
|
|
241
276
|
`attribution.status` is `complete` when `project_id`, `phase`, `intent`, and a
|
|
242
277
|
work id are present; `partial` when any attribution is known but that contract
|
|
243
|
-
is incomplete; otherwise `missing`. `outcome`
|
|
244
|
-
|
|
245
|
-
|
|
246
|
-
`
|
|
247
|
-
`
|
|
248
|
-
|
|
249
|
-
|
|
250
|
-
|
|
251
|
-
contains final commit/path/LOC observations and does **not** claim those changes
|
|
252
|
-
prove authorship or semantic quality.
|
|
278
|
+
is incomplete; otherwise `missing`. `outcome.status` is derived from the
|
|
279
|
+
harness receipt: `accepted`, `rejected`, `no_change`, or `unknown`. Optional
|
|
280
|
+
worker claims cannot set it. Its additive `class` records the work verdict
|
|
281
|
+
(`completed_with_proof`, `completed_no_activity`, `report_invalid`,
|
|
282
|
+
`task_failed`, plus legacy classes retained in old rows), and `report_status`
|
|
283
|
+
is normally `accepted` or `rejected`. `source=harnex_observed_state` identifies
|
|
284
|
+
new receipts. The block also contains final commit/path/LOC observations; those
|
|
285
|
+
facts prove the delta, not semantic quality or human authorship.
|
|
253
286
|
|
|
254
287
|
Every end row has one Harnex-session `attempt`. Its random `id` is distinct
|
|
255
288
|
from operator-visible `run_id`; `parent_attempt_id` and `parent_dispatch_id`
|
|
@@ -307,95 +340,137 @@ accepted/rejected/blocked/unknown child outcomes deduplicated by work id,
|
|
|
307
340
|
primary usage/tool calls per accepted entry, and explicit `missing` /
|
|
308
341
|
`unsupported` statuses instead of treating absent telemetry as zero.
|
|
309
342
|
|
|
310
|
-
##
|
|
343
|
+
## Observed-state receipts and optional claims
|
|
344
|
+
|
|
345
|
+
Every dispatch receives a bounded receipt authored by Harnex, not by the
|
|
346
|
+
worker. The schema identifier remains `harnex.artifact_report.v1` so existing
|
|
347
|
+
`artifact-report validate --final` consumers keep one command and one result
|
|
348
|
+
contract. The additive `receipt.author=harnex` marker selects the observed-state
|
|
349
|
+
final contract.
|
|
311
350
|
|
|
312
|
-
|
|
313
|
-
proof to canonical human-readable artifacts (usually files under `koder/`). A
|
|
314
|
-
valid v1 report looks like:
|
|
351
|
+
A generated receipt contains:
|
|
315
352
|
|
|
316
353
|
```json
|
|
317
354
|
{
|
|
318
355
|
"schema": "harnex.artifact_report.v1",
|
|
319
356
|
"status": "pass",
|
|
357
|
+
"receipt": {
|
|
358
|
+
"version": 1,
|
|
359
|
+
"author": "harnex",
|
|
360
|
+
"generated_at": "2026-08-03T05:00:00Z",
|
|
361
|
+
"id": "cx-r-64",
|
|
362
|
+
"session_id": "8d8f3f07c8fc343d"
|
|
363
|
+
},
|
|
320
364
|
"outcome": {
|
|
321
|
-
"status": "
|
|
322
|
-
"summary": "
|
|
365
|
+
"status": "no_change",
|
|
366
|
+
"summary": "Harnex observed successful completion with no Git delta."
|
|
323
367
|
},
|
|
324
|
-
"canonical_artifacts": ["koder/issues/52_typed_artifact_validation_sidecars.md"],
|
|
325
368
|
"validation": {
|
|
326
369
|
"status": "pass",
|
|
327
370
|
"final_reported": true,
|
|
328
371
|
"commands": [
|
|
329
|
-
{ "cmd": "
|
|
372
|
+
{ "cmd": "git diff --check", "exit_code": 0, "status": "completed" }
|
|
330
373
|
]
|
|
331
374
|
},
|
|
332
|
-
"
|
|
333
|
-
{
|
|
334
|
-
"
|
|
335
|
-
"
|
|
336
|
-
"
|
|
337
|
-
"
|
|
338
|
-
"
|
|
339
|
-
|
|
340
|
-
|
|
375
|
+
"observed": {
|
|
376
|
+
"git": {
|
|
377
|
+
"status": "observed",
|
|
378
|
+
"start_sha": "0123456789abcdef0123456789abcdef01234567",
|
|
379
|
+
"end_sha": "0123456789abcdef0123456789abcdef01234567",
|
|
380
|
+
"branch": "main",
|
|
381
|
+
"changed_paths": [],
|
|
382
|
+
"loc_added": 0,
|
|
383
|
+
"loc_removed": 0,
|
|
384
|
+
"files_changed": 0,
|
|
385
|
+
"commits": 0
|
|
386
|
+
},
|
|
387
|
+
"commands": [
|
|
388
|
+
{ "cmd": "git diff --check", "exit_code": 0, "status": "completed" }
|
|
389
|
+
],
|
|
390
|
+
"command_observation": "observed",
|
|
391
|
+
"turn": {
|
|
392
|
+
"status": "completed",
|
|
393
|
+
"outcome_class": "completed_with_proof",
|
|
394
|
+
"task_complete": true,
|
|
395
|
+
"task_failed": false,
|
|
396
|
+
"accepted": true,
|
|
397
|
+
"exit_code": 0
|
|
398
|
+
},
|
|
399
|
+
"usage": { "status": "observed", "input_tokens": 100, "output_tokens": 20, "total_tokens": 120 }
|
|
400
|
+
}
|
|
341
401
|
}
|
|
342
402
|
```
|
|
343
403
|
|
|
344
|
-
|
|
345
|
-
|
|
404
|
+
`observed.git` is sufficient for a queue to distinguish commit proof
|
|
405
|
+
(`start_sha != end_sha`, commit count/path/LOC evidence) from an observed
|
|
406
|
+
`no_change` result. `start_dirty`, `end_dirty`, and `worktree_changed` expose
|
|
407
|
+
worktree caveats while unchanged pre-session dirt is excluded from the delta.
|
|
408
|
+
Codex app-server `commandExecution` items contribute
|
|
409
|
+
bounded command text, integer exit code, status, and optional duration. Other
|
|
410
|
+
transports currently report `command_observation: "unsupported"` rather than
|
|
411
|
+
guessing from prose or generic tool events. If the 256-KiB receipt cap requires
|
|
412
|
+
trimming, `commands_truncated` / `changed_paths_truncated` make that explicit;
|
|
413
|
+
full aggregate counts remain in the dispatch row. Usage carries the same
|
|
414
|
+
measured/missing/unsupported semantics as the dispatch
|
|
415
|
+
row and is refreshed at teardown when final adapter usage becomes available.
|
|
416
|
+
|
|
417
|
+
`validation.commands` mirrors all bounded observed command exits for legacy
|
|
418
|
+
readers. Its aggregate status is `not_run`, `pass`, or `fail`; a failed
|
|
419
|
+
exploratory command may precede a successful turn. For harness receipts,
|
|
420
|
+
`validate --final` therefore validates receipt authorship/shape, accepted turn,
|
|
421
|
+
and accepted/no-change outcome instead of letting an intermediate command exit
|
|
422
|
+
rewrite the harness verdict. Queue policy may impose a stricter command gate by
|
|
423
|
+
inspecting `observed.commands`.
|
|
424
|
+
|
|
425
|
+
Reviewers can write one optional input file at
|
|
426
|
+
`HARNEX_ARTIFACT_CLAIMS_PATH`:
|
|
346
427
|
|
|
347
|
-
```
|
|
348
|
-
|
|
349
|
-
|
|
350
|
-
|
|
351
|
-
|
|
428
|
+
```json
|
|
429
|
+
{
|
|
430
|
+
"claims": {
|
|
431
|
+
"summary": "Review complete; one P2 remains.",
|
|
432
|
+
"verdict": "changes_requested",
|
|
433
|
+
"findings": { "P1": 0, "P2": 1, "P3": 0 }
|
|
434
|
+
}
|
|
435
|
+
}
|
|
352
436
|
```
|
|
353
437
|
|
|
354
|
-
|
|
355
|
-
|
|
356
|
-
|
|
357
|
-
|
|
358
|
-
|
|
359
|
-
|
|
360
|
-
|
|
361
|
-
|
|
362
|
-
|
|
363
|
-
|
|
364
|
-
|
|
365
|
-
|
|
366
|
-
|
|
367
|
-
|
|
368
|
-
|
|
369
|
-
|
|
370
|
-
|
|
371
|
-
|
|
372
|
-
|
|
373
|
-
not change the wrapped process exit code. Strict mode evaluates final validation
|
|
374
|
-
before successful auto-stop/terminal acceptance and returns non-zero for those
|
|
375
|
-
defects, rejected outcomes, or a valid final report that was already present
|
|
376
|
-
and unchanged when the session started. Only the configured path is read;
|
|
377
|
-
report-shaped JSON in final prose is not scraped. The typed failure is exposed
|
|
378
|
-
before auto-stop through `task_failed`, `outcome.class`, and
|
|
379
|
-
`outcome.report_status` so `harnex watch --until done` returns non-zero.
|
|
380
|
-
|
|
381
|
-
Harnex does not copy large transcripts or replace plain-text `koder/` docs. The
|
|
382
|
-
sidecar is an evidence index for queue tooling; the canonical explanation should
|
|
383
|
-
remain in the referenced files.
|
|
438
|
+
Only those bounded fields are copied. Claims are informational and never affect
|
|
439
|
+
receipt validity, `outcome.status`, or the completion gate. The fingerprint of
|
|
440
|
+
a configured pre-existing file is used only to avoid ingesting stale claims;
|
|
441
|
+
Harnex always atomically replaces the final receipt, eliminating the old
|
|
442
|
+
missing/stale-proof acceptance race. Legacy workers that still write a full v1
|
|
443
|
+
file may contribute its outcome summary/status as advisory claims, but Harnex
|
|
444
|
+
replaces the document and ignores it for acceptance.
|
|
445
|
+
|
|
446
|
+
`artifact_report` in the dispatch row records final path, bytes, SHA-256,
|
|
447
|
+
schema, ingest status, report status, and author. `receipt`, `observed`, and
|
|
448
|
+
`validation` are compact top-level copies; `claims` appears only when present.
|
|
449
|
+
The default receipt path is outside the checkout so proof generation cannot
|
|
450
|
+
pollute the Git delta. An explicit path is supported when a queue requires one.
|
|
451
|
+
Receipt write/validation failure is typed `report_invalid` and fails closed.
|
|
452
|
+
Harnex never scrapes report-shaped final prose or copies full transcripts.
|
|
453
|
+
|
|
454
|
+
`harnex artifact-report init` still creates the older manual skeleton, and the
|
|
455
|
+
validator still accepts that legacy final contract. It is compatibility tooling,
|
|
456
|
+
not a required worker step for new dispatches.
|
|
384
457
|
|
|
385
458
|
## Autonomous completion gate
|
|
386
459
|
|
|
387
460
|
For Codex app-server runs launched with `--context`, provider turn completion
|
|
388
|
-
is not by itself accepted work completion. Harnex emits `task_complete` only
|
|
389
|
-
it has at least one structured command/tool/file-change item
|
|
390
|
-
|
|
391
|
-
`
|
|
392
|
-
normalizes
|
|
461
|
+
is not by itself accepted work completion. Harnex emits `task_complete` only
|
|
462
|
+
when it has at least one structured command/tool/file-change item or a Git
|
|
463
|
+
delta. If both are absent, it emits `task_failed` with
|
|
464
|
+
`outcome_class=completed_no_activity`, writes a rejected observed receipt, and
|
|
465
|
+
normalizes auto-stop to non-zero. This applies equally to
|
|
393
466
|
`service_tier=flex` and `service_tier=fast` and deliberately ignores final
|
|
394
|
-
answer text. Intentional no-op work
|
|
467
|
+
answer text and claims. Intentional no-op work must still perform observable
|
|
468
|
+
inspection/validation; a model assertion of `no_change` cannot self-approve.
|
|
395
469
|
|
|
396
|
-
PTY transports do not expose equivalent item metadata, so their
|
|
397
|
-
prompt-return auto-stop behavior remains.
|
|
398
|
-
|
|
470
|
+
PTY transports do not expose equivalent completion-item metadata, so their
|
|
471
|
+
existing prompt-return auto-stop behavior remains. Non-Codex transports label
|
|
472
|
+
command observation `unsupported`; Git and terminal state are still recorded
|
|
473
|
+
without guessing.
|
|
399
474
|
|
|
400
475
|
## Actuals
|
|
401
476
|
|
|
@@ -407,9 +482,11 @@ fields. Separately, Pi aggregates bounded `contextUsage` samples and Codex
|
|
|
407
482
|
aggregates `tokenUsage.last` plus `modelContextWindow`; neither source is
|
|
408
483
|
substituted with cumulative usage when active occupancy is unavailable.
|
|
409
484
|
|
|
410
|
-
Git actuals
|
|
411
|
-
|
|
412
|
-
|
|
485
|
+
Git actuals capture the start/end SHA plus committed, staged, unstaged, and
|
|
486
|
+
untracked changes relative to the worktree state observed at session start.
|
|
487
|
+
Unchanged pre-existing dirt is not credited to the worker, and harness-owned
|
|
488
|
+
dispatch/receipt paths are excluded. Git failures leave the corresponding
|
|
489
|
+
consolidated fields `null` and omit `git` events.
|
|
413
490
|
|
|
414
491
|
The `actual` block includes model/effort hints from `--meta`, duration, token
|
|
415
492
|
counts, `agent_session_id`, compatibility `cost_usd`, adapter transport, git
|
|
@@ -420,7 +497,7 @@ payloads, output/event volume measurements, and output/events log paths. New
|
|
|
420
497
|
additive attempt counters are `attempts_total`, `attempts_succeeded`,
|
|
421
498
|
`attempts_failed`, `retry_count`, `throttle_429_count`, `disconnect_count`, and
|
|
422
499
|
`fallback_triggered`. Throughput values are populated only for a
|
|
423
|
-
|
|
500
|
+
harness-accepted outcome: `throughput_tokens_per_s` and
|
|
424
501
|
`throughput_successes_per_h`; `retry_tax_pct` is `0.0` when no retry occurred
|
|
425
502
|
and `null` until a retry source can measure attributable wasted tokens.
|
|
426
503
|
|
|
@@ -446,16 +523,16 @@ jq -s 'map(select(.actual.attempts_total > 0))
|
|
|
446
523
|
|
|
447
524
|
## Exit taxonomy
|
|
448
525
|
|
|
449
|
-
- `success`: wrapped process exited `0` with task completion, accepted
|
|
450
|
-
|
|
451
|
-
- `failure`: wrapped process exited non-zero
|
|
452
|
-
|
|
526
|
+
- `success`: wrapped process exited `0` with task completion, accepted observed
|
|
527
|
+
receipt proof, or an adapter session summary.
|
|
528
|
+
- `failure`: wrapped process exited non-zero, the observed-activity gate emitted
|
|
529
|
+
`task_failed`, or Harnex could not write/validate the receipt.
|
|
453
530
|
- `timeout`: wrapped process exited with code `124`.
|
|
454
531
|
- `boot_failure`: JSON-RPC app-server exited within the startup window before a
|
|
455
532
|
turn was observed.
|
|
456
533
|
- `disconnected`: wrapped process exited `0` but no session summary was parsed.
|
|
457
534
|
|
|
458
|
-
Canonical dispatch-stream
|
|
459
|
-
|
|
460
|
-
|
|
535
|
+
Canonical dispatch-stream writes remain best-effort. Receipt
|
|
536
|
+
writes are different: proof-generation failure is fail-closed and changes the
|
|
537
|
+
work verdict. Repo phase allowlists and runtime-log retention are documented in
|
|
461
538
|
[configuration.md](configuration.md).
|
data/docs/events.md
CHANGED
|
@@ -113,9 +113,8 @@ New event types:
|
|
|
113
113
|
`sha` and `branch`; `phase: "end"` includes `sha`, `loc_added`,
|
|
114
114
|
`loc_removed`, `files_changed`, and `commits`.
|
|
115
115
|
- `summary`: emitted last before `exited`. `path` is the canonical tracked
|
|
116
|
-
dispatch stream
|
|
117
|
-
|
|
118
|
-
`timeout`, or `disconnected`.
|
|
116
|
+
dispatch stream — the only destination a dispatch record is written to.
|
|
117
|
+
`exit` is `success`, `failure`, `timeout`, or `disconnected`.
|
|
119
118
|
|
|
120
119
|
Example telemetry sequence:
|
|
121
120
|
|
data/guides/01_dispatch.md
CHANGED
|
@@ -22,9 +22,11 @@ Inside a harnex-managed session, these environment variables are available:
|
|
|
22
22
|
| `HARNEX_SESSION_REPO_ROOT` | Repo root for the session |
|
|
23
23
|
| `HARNEX_SESSION_ID` | Internal harnex instance ID |
|
|
24
24
|
| `HARNEX_SPAWNER_PANE` | Tmux pane ID of the invoker |
|
|
25
|
-
| `HARNEX_ARTIFACT_REPORT_PATH` | Absolute
|
|
26
|
-
| `
|
|
27
|
-
| `
|
|
25
|
+
| `HARNEX_ARTIFACT_REPORT_PATH` | Absolute harness-owned final receipt path |
|
|
26
|
+
| `HARNEX_ARTIFACT_CLAIMS_PATH` | Optional bounded worker-claims input path |
|
|
27
|
+
| `HARNEX_ARTIFACT_REPORT_SCHEMA` | Receipt schema identifier |
|
|
28
|
+
| `HARNEX_ARTIFACT_REPORT_MODE` | `observed_state` for harness-authored receipts |
|
|
29
|
+
| `HARNEX_ARTIFACT_REPORT_REQUIRED` | `1` when the compatibility strict flag was passed |
|
|
28
30
|
|
|
29
31
|
Use `harnex send`, `harnex status`, `harnex wait`, `harnex pane`, and
|
|
30
32
|
`harnex logs` to coordinate with peers. If you are not inside harnex,
|
|
@@ -69,8 +71,8 @@ harnex run pi --id pi-i-NN --tmux pi-i-NN \
|
|
|
69
71
|
For one-shot context dispatches that should clean themselves up, add
|
|
70
72
|
`--auto-stop`. It requires `--context` and does not keep the session alive for
|
|
71
73
|
later reuse. On Codex app-server, a turn launched from `--context` is accepted
|
|
72
|
-
only after structured command/tool activity
|
|
73
|
-
|
|
74
|
+
only after structured command/tool activity or a Git delta. Optional claims and
|
|
75
|
+
final prose cannot satisfy that observed-activity gate. A prose-only acknowledgment emits
|
|
74
76
|
`completed_no_activity`, is
|
|
75
77
|
visible as `task_failed` before teardown, and exits non-zero. This keeps
|
|
76
78
|
parallel orchestration compact without converting agent turn completion into
|
|
@@ -103,33 +105,29 @@ harnex run codex --cwd /tmp/leximaze_eval_run_001 \
|
|
|
103
105
|
`--root DIR` only overrides harnex's root attribution; it does not change the
|
|
104
106
|
child process cwd. Neither flag is a sandbox.
|
|
105
107
|
|
|
106
|
-
For queue closeout, ask workers to
|
|
107
|
-
|
|
108
|
-
|
|
108
|
+
For queue closeout, do not ask workers to author proof JSON. Harnex writes a
|
|
109
|
+
canonical receipt for every dispatch from the Git delta, structured command
|
|
110
|
+
exits, turn outcome, and usage it observed. The default receipt lives outside
|
|
111
|
+
the checkout under the Harnex state directory; pass `--artifact-report PATH`
|
|
112
|
+
only when a queue needs a fixed destination. Consumers can run:
|
|
109
113
|
|
|
110
114
|
```bash
|
|
111
|
-
harnex artifact-report
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
115
|
+
harnex artifact-report validate /path/from/the-dispatch-row.json --final
|
|
116
|
+
```
|
|
117
|
+
|
|
118
|
+
A worker may add review context by writing only a small block to
|
|
119
|
+
`$HARNEX_ARTIFACT_CLAIMS_PATH` before it completes:
|
|
120
|
+
|
|
121
|
+
```json
|
|
122
|
+
{"claims":{"summary":"Review complete","verdict":"changes_requested","findings":{"P1":0,"P2":1,"P3":0}}}
|
|
117
123
|
```
|
|
118
124
|
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
`
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
`validation.final_reported=true`.
|
|
126
|
-
|
|
127
|
-
Without `--require-artifact-report`, missing/malformed reports remain warning
|
|
128
|
-
telemetry. Strict mode fails closed for missing, malformed, unsupported,
|
|
129
|
-
oversized, schema-incomplete, rejected, or unchanged stale reports. JSON in the
|
|
130
|
-
agent's final prose does not count: only the configured sidecar path is read.
|
|
131
|
-
A fresh valid `no_change` report is the explicit proof path for intentional
|
|
132
|
-
no-delta work.
|
|
125
|
+
Claims are bounded and advisory. They never determine receipt validity, and
|
|
126
|
+
malformed/stale claims are ignored. `HARNEX_ARTIFACT_REPORT_PATH` is owned and
|
|
127
|
+
overwritten by Harnex; JSON printed in final prose is not scraped. The legacy
|
|
128
|
+
`artifact-report init` command and `--require-artifact-report` flag remain for
|
|
129
|
+
compatibility, but normal dispatches need neither worker JSON nor an explicit
|
|
130
|
+
receipt path.
|
|
133
131
|
|
|
134
132
|
Queue runners should pass first-class attribution so dispatch rows can be grouped
|
|
135
133
|
without path/id heuristics:
|
|
@@ -146,10 +144,9 @@ harnex run pi --id pi-i-NN --tmux pi-i-NN \
|
|
|
146
144
|
are present.
|
|
147
145
|
|
|
148
146
|
Every dispatch writes one `dispatch_start` and one rich v2 `dispatch_end` row
|
|
149
|
-
to the canonical repo/global dispatch stream
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
before spawn.
|
|
147
|
+
to the canonical repo/global dispatch stream, which is the only telemetry
|
|
148
|
+
destination. Repo `.harnex/config.json` can warn on or reject non-canonical
|
|
149
|
+
phase names before spawn.
|
|
153
150
|
|
|
154
151
|
Pi runs use structured RPC (`pi --mode rpc`). Pass Pi child flags after `--`
|
|
155
152
|
(e.g. `harnex run pi --context "..." -- --model anthropic/claude-sonnet-4-5 --thinking high`).
|
data/guides/04_monitoring.md
CHANGED
|
@@ -71,10 +71,10 @@ worktrees). `--attempt-kind review` is exempt: a completed parent may still
|
|
|
71
71
|
sit at a live prompt while its work is reviewed. For structured sessions (Pi RPC and Codex app-server),
|
|
72
72
|
`harnex wait --until task_complete` remains the exact accepted-turn fence.
|
|
73
73
|
Codex acknowledgment-only auto-stop turns are typed
|
|
74
|
-
`completed_no_activity` and fail this fence without transcript parsing.
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
verify the expected artifact or tests afterward.
|
|
74
|
+
`completed_no_activity` and fail this fence without transcript parsing. Harnex
|
|
75
|
+
writes the observed-state receipt before publishing `task_complete`; optional
|
|
76
|
+
worker claims never make an inactive turn pass. Harnex still does not judge
|
|
77
|
+
semantic quality, so verify the expected artifact or tests afterward.
|
|
78
78
|
|
|
79
79
|
## Completion Test
|
|
80
80
|
|