ai-wiki-toolkit-linux-arm64 0.1.31 → 0.1.35

package/README.md

@@ -1,7 +1,7 @@
 # ai-wiki-toolkit-linux-arm64
 
 This package contains the `aiwiki-toolkit` executable for `linux-arm64-glibc`.
-It is published as the platform-specific binary package for `ai-wiki-toolkit` `0.1.31`.
+It is published as the platform-specific binary package for `ai-wiki-toolkit` `0.1.35`.
 Most users should install `ai-wiki-toolkit` instead of using this package directly.
 
 ---
@@ -305,7 +305,7 @@ aiwiki-toolkit init
 - create a gitignored `.env.aiwiki` file for the current local actor identity
 - create starter indexes such as `ai-wiki/conventions/index.md`, `ai-wiki/review-patterns/index.md`, `ai-wiki/problems/index.md`, `ai-wiki/features/index.md`, `ai-wiki/trails/index.md`, `ai-wiki/work/index.md`, and `ai-wiki/people/<handle>/index.md`
 - create `ai-wiki/conventions/`, `ai-wiki/review-patterns/`, `ai-wiki/problems/`, `ai-wiki/features/`, `ai-wiki/work/`, `ai-wiki/people/<handle>/drafts/`, `ai-wiki/metrics/`, and repo/home `_toolkit/`
-- generate package-managed `_toolkit/index.md`, `_toolkit/workflows.md`, `_toolkit/catalog.json`, `_toolkit/schema/reuse-v1.md`, `_toolkit/schema/team-memory-v1.md`, `_toolkit/schema/work-v1.md`, `_toolkit/metrics/*.json`, and `_toolkit/work/*`
+- generate package-managed `_toolkit/index.md`, `_toolkit/workflows.md`, `_toolkit/catalog.json`, `_toolkit/schema/reuse-v1.md`, `_toolkit/schema/team-memory-v1.md`, `_toolkit/schema/work-v1.md`, `_toolkit/metrics/*`, and `_toolkit/work/*`
 - upsert a managed `.gitignore` block that ignores `.env.aiwiki`, AI wiki telemetry, and generated aggregate snapshots so routine agent use does not dirty `git status`
 - create or refresh package-owned `.agents/skills/ai-wiki-reuse-check/`, `.agents/skills/ai-wiki-update-check/`, `.agents/skills/ai-wiki-clarify-before-code/`, `.agents/skills/ai-wiki-capture-review-learning/`, and `.agents/skills/ai-wiki-consolidate-drafts/`
 - update `AGENT.md`, `AGENTS.md`, and/or `CLAUDE.md` with a short managed instruction block that points agents to `ai-wiki/_toolkit/system.md` when the repo contains `ai-wiki/`
@@ -338,7 +338,7 @@ If package-owned repo-local skill files already exist under `.agents/skills/ai-w
 
 `init` remains as a backward-compatible alias for `install`. The actual scaffold creation does not happen at package install time; it happens when you run `aiwiki-toolkit install` or `aiwiki-toolkit init` inside a git repository.
 
-To append one explicit knowledge-reuse observation and refresh managed aggregates:
+To append one explicit knowledge-reuse observation and refresh handle-scoped managed metrics:
 
 ```bash
 aiwiki-toolkit record-reuse \
@@ -352,7 +352,9 @@ aiwiki-toolkit record-reuse \
   --saved-seconds 45
 ```
 
-This appends to the user-owned `ai-wiki/metrics/reuse-events/<handle>.jsonl` shard and refreshes the package-managed aggregate views under `ai-wiki/_toolkit/metrics/`. The installer ignores both the shard and the generated aggregate views by default so these telemetry updates stay local.
+This appends to the user-owned `ai-wiki/metrics/reuse-events/<handle>.jsonl` shard and refreshes the handle-scoped generated views under `ai-wiki/_toolkit/metrics/by-handle/<handle>/`. The installer ignores both the shard and the generated aggregate views by default so these telemetry updates stay local.
+
+For post-task diagnosis, reuse events can also carry optional provenance such as `--session-id`, `--source-session-id`, `--source-task-id`, `--consulted-order`, `--signal-status candidate`, `--not-helpful-reason superseded_by_later_doc`, `--resolved-by-doc-id`, and `--superseded-by-doc-id`. Candidate not-helpful signals are review hints; confirmed outcomes still require explicit human or agent judgment.
 
 Only record user-owned AI wiki knowledge docs with `record-reuse`.
 
@@ -401,7 +403,7 @@ aiwiki-toolkit record-reuse-check \
   --check-outcome wiki_used
 ```
 
-This appends to the user-owned `ai-wiki/metrics/task-checks/<handle>.jsonl` shard and refreshes the package-managed aggregate views under `ai-wiki/_toolkit/metrics/`. The installer ignores both the shard and the generated aggregate views by default so these telemetry updates stay local.
+This appends to the user-owned `ai-wiki/metrics/task-checks/<handle>.jsonl` shard and refreshes the handle-scoped generated views under `ai-wiki/_toolkit/metrics/by-handle/<handle>/`. The installer ignores both the shard and the generated aggregate views by default so these telemetry updates stay local.
 
 Both metrics logs are sharded by handle under:
 
@@ -410,7 +412,7 @@ Both metrics logs are sharded by handle under:
 
 These logs are intended as local telemetry by default, not merge-heavy source files.
 
-If you need a fresh local telemetry and work snapshot, regenerate package-managed aggregate views such as `ai-wiki/_toolkit/catalog.json`, `ai-wiki/_toolkit/metrics/*.json`, or `ai-wiki/_toolkit/work/*` with:
+If you need a fresh local telemetry and work snapshot, regenerate package-managed aggregate views such as `ai-wiki/_toolkit/catalog.json`, `ai-wiki/_toolkit/metrics/*`, or `ai-wiki/_toolkit/work/*` with:
 
 ```bash
 aiwiki-toolkit refresh-metrics
@@ -424,7 +426,7 @@ aiwiki-toolkit diagnose memory --since 14d --handle your-handle
 aiwiki-toolkit diagnose memory --focus trial-error
 ```
 
-This writes regenerated local reports under `ai-wiki/_toolkit/diagnostics/` and prints the report to stdout. The report highlights high-ROI memory, noisy memory, stale or missing docs, conflict notes, missed-memory signals, and coverage gaps such as document reuse events that were never paired with a task-level reuse check. It does not edit user-owned AI wiki docs.
+This writes regenerated local reports under `ai-wiki/_toolkit/diagnostics/<handle-or-all>/` and prints the report to stdout. The report highlights high-ROI memory, noisy memory, stale or missing docs, conflict notes, missed-memory signals, and coverage gaps such as document reuse events that were never paired with a task-level reuse check. It does not edit user-owned AI wiki docs.
 
 Use `--focus trial-error` to generate a focused trial/error reduction report from existing
 AI wiki evidence. It summarizes material effects such as `avoided_retry`,
@@ -439,25 +441,142 @@ aiwiki-toolkit consolidate queue
 aiwiki-toolkit consolidate queue --since 14d --handle your-handle
 ```
 
-This writes regenerated local reports under `ai-wiki/_toolkit/consolidation/` and prints the queue to stdout. The queue suggests one action per draft cluster: keep, refine, promotion candidate, conflict, or supersession. It does not edit user-owned AI wiki docs or create shared conventions, review patterns, problems, features, or decisions; those still require human confirmation.
+This writes regenerated local reports under `ai-wiki/_toolkit/consolidation/<handle>/` and prints the queue to stdout. The queue suggests one action per draft cluster: keep, refine, promotion candidate, conflict, or supersession. It does not edit user-owned AI wiki docs or create shared conventions, review patterns, problems, features, or decisions; those still require human confirmation.
+
+To mark handle-local draft promotion candidates from confirmed-useful reuse evidence:
+
+```bash
+aiwiki-toolkit promote candidates --handle your-handle
+aiwiki-toolkit promote candidates --handle your-handle --apply
+```
+
+The default run is report-only. With `--apply`, a draft is marked only when it has more than three distinct resolved task IDs, no `not_helpful` reuse events, and an existing non-stale source draft. The command refreshes stable links in `ai-wiki/people/<handle>/index.md`; exact reuse counts stay in generated reports under `ai-wiki/_toolkit/reports/promotion-candidates/<handle>/`.
+
+To inspect referenced files and estimated time impact from local reuse evidence:
+
+```bash
+aiwiki-toolkit report usefulness --handle your-handle
+aiwiki-toolkit report usefulness --handle your-handle --format json
+```
+
+This writes `ai-wiki/_toolkit/reports/usefulness/<handle-or-all>/latest.md` and `.json`. It lists referenced files and sums resolved-event `estimated_savings`. Baseline/current/remaining durations are reported as `unknown` until task logs include explicit timing evidence.
+
+To generate a weekly local HTML review queue:
+
+```bash
+aiwiki-toolkit report weekly --handle your-handle
+aiwiki-toolkit report weekly --handle your-handle --if-due
+```
+
+This writes a static HTML review queue and JSON payload under `ai-wiki/_toolkit/reports/weekly/<handle>/<iso-week>/`, refreshes `latest.html` and `latest.json`, and records the last generated period in `ai-wiki/_toolkit/reports/weekly/<handle>/state.json`. The HTML page focuses only on items that need human judgment: promotion candidates, personal drafts that may need diagnosis, and not-helpful signals. Coverage, referenced-file, and other raw evidence remains in the JSON payload and supporting reports; saved-time estimates belong in impact-eval reports, not the weekly HTML view. Use `--if-due` from cron, launchd, or an agent workflow so the same ISO week is generated once; use `--force` for local testing before a release.
 
 To summarize first-attempt product impact from a captured eval run:
 
 ```bash
+aiwiki-toolkit eval impact families
+aiwiki-toolkit eval impact families --format json
+aiwiki-toolkit eval impact discover
+aiwiki-toolkit eval impact family show ownership_boundary
+aiwiki-toolkit eval impact family candidates
+aiwiki-toolkit eval impact family init --name retry_loop --from-candidate problems/retry-loop --baseline-ref HEAD^
+aiwiki-toolkit eval impact family draft --candidate problems_retry_loop --baseline-ref HEAD^
+aiwiki-toolkit eval impact family promote --candidate problems_retry_loop
+aiwiki-toolkit eval impact family promote --candidate problems_retry_loop --apply
+aiwiki-toolkit eval impact plan --family ownership_boundary
+aiwiki-toolkit eval impact plan --family ownership_boundary --format json
+aiwiki-toolkit eval impact prepare --family ownership_boundary
+aiwiki-toolkit eval impact prepare --family ownership_boundary --format json
+aiwiki-toolkit eval impact run --run-dir /path/to/eval-run --slot s01
+aiwiki-toolkit eval impact run --run-dir /path/to/eval-run --all-slots --score-policy command-exit
+aiwiki-toolkit eval impact run --run-dir /path/to/eval-run --all-slots --score-policy rubric --rubric evals/impact/rubrics/my-family.json
+aiwiki-toolkit eval impact benchmark --family ownership_boundary --score-policy command-exit
+aiwiki-toolkit eval impact schedule report --handle your-handle --candidate-max-items 25
+aiwiki-toolkit eval impact schedule run --family ownership_boundary --score-policy command-exit
+aiwiki-toolkit eval impact schedule run --all-runnable --if-due --score-policy rubric
+aiwiki-toolkit eval impact capture --run-dir /path/to/eval-run --slot s01 --prompt-level original --first-pass-success
+aiwiki-toolkit eval impact validate --run-dir /path/to/eval-run
+aiwiki-toolkit eval impact score --run-dir /path/to/eval-run --slot s01 --prompt-level original --label success
+aiwiki-toolkit eval impact manifest --run-dir /path/to/eval-run
+aiwiki-toolkit eval impact manifest --run-dir /path/to/eval-run --format json
 aiwiki-toolkit eval impact report --run-dir /path/to/eval-run
 aiwiki-toolkit eval impact report --run-dir /path/to/eval-run --format json
 aiwiki-toolkit eval impact summarize --run-dir /path/to/eval-run --run-dir /path/to/another-run
 aiwiki-toolkit eval impact summarize --runs-file evals/impact/runs.json
 ```
 
-This reads an existing run directory with `metadata.json`, result captures, optional
-`score.json` files, and optional `confounds.json`. It compares the run's primary variants,
-normally `no_aiwiki_workflow` versus `aiwiki_ambient_memory_workflow`, using first-attempt
-metrics only: `first_pass` captures count toward the signal, while `final` repair captures
-stay diagnostic. The command reports first-attempt success rate, average score, attempts, human
-nudges, changed files, untracked files, change-profile splits for project files versus AI wiki
-telemetry and user-owned wiki churn, and whether the run is ready for shareable causal claims.
-It does not run agents or mutate eval artifacts.
+Use `eval impact families` before running benchmarks. It discovers registered families from
+`evals/impact/families/*/spec.toml`, reports readiness, prompt and rubric presence, memory fixture
+counts, baseline refs, historical issues, and next commands. Use `eval impact family show <name>`
+for one family.
+
+Use `eval impact family candidates` to expose trial/error replay candidates from existing AI wiki
+telemetry. It layers over `diagnose memory --focus trial-error` and reports candidate readiness
+without writing user-owned AI wiki docs. Use `eval impact family init --from-candidate ...` only
+after confirming a source incident, baseline ref, prompt shape, and rubric direction; it creates a
+draft family scaffold under `evals/impact/`.
+
+Use `eval impact discover` for the continuous loop. It refreshes the managed candidate queue under
+`ai-wiki/_toolkit/evals/candidates/`, preserves first-seen/last-seen/seen-count state, and prints
+the next draft, promotion, and schedule commands. Use `eval impact family draft` to create managed
+candidate files under `ai-wiki/_toolkit/evals/drafts/<candidate>/` without registering a formal
+family. Use `eval impact family promote` as a report-only gate; add `--apply` only after the draft
+has a real baseline ref, prompt, and rubric and you want to write formal files under
+`evals/impact/`.
+
+Use `eval impact plan` to inspect the next run before creating workspaces or invoking agents. It
+reads `evals/impact/families/<family>/spec.toml` and prompt files, then reports the planned
+baseline ref, prompt hashes, workflow-primary variants, output paths, and script commands. The plan
+command does not mutate eval artifacts or call an agent.
+
+Use `eval impact prepare` to execute the planned setup only: it creates neutral slot workspaces,
+creates the run directory and metadata, and writes initial `manifest.json` and `manifest.md` files.
+It still does not call an agent.
+
+Use `eval impact run` to invoke Codex CLI against one neutral slot or all slots in an already
+prepared run. The command calls the repo-local slot runner, captures first-pass artifacts,
+optionally exports visible Codex sessions, validates confounds, applies an explicit score policy,
+and writes a report bundle under `<run-dir>/report_bundle/`. The default score policy is `none`.
+`--score-policy command-exit` is useful for smoke tests and execution-health automation, but it
+only scores Codex/save-result command completion; use manual or semantic scoring before making
+research-quality correctness claims.
+`--score-policy rubric` reads an `impact-eval-rubric-v1` JSON file, writes
+`rubric_judgment.json` next to each slot score, then writes the normal `score.json` artifact.
+Rubric criteria can inspect captured diffs, final messages, result fields, changed files, and
+untracked files.
+
+Use `eval impact benchmark` when you want one command to prepare a family and immediately run all
+slots. It wraps `prepare` plus `run`, then returns the prepared run directory, run result, validation
+status, scores, and report bundle.
+
+Use `eval impact schedule report` to generate a periodic benchmark dashboard under
+`ai-wiki/_toolkit/evals/reports/<period>/`. It combines registered families, the managed candidate
+queue, and the run index. Pass the same candidate filters you use for discovery, such as `--handle`,
+`--since`, and `--candidate-max-items`, so the scheduled report does not accidentally stale a
+larger queue with a narrower refresh. Use `eval impact schedule run --family <name>` or
+`--all-runnable` to run benchmarks, append `ai-wiki/_toolkit/evals/runs/index.json`, refresh the
+report, and record `ai-wiki/_toolkit/evals/schedule/state.json`. `--if-due` is intended for cron,
+launchd, or an agent workflow that should run at most once per period.
+
+Use `eval impact capture` after a manual first pass or repaired pass to save `result.json`, the
+workspace diff, status, head, and optional final-message artifact. It infers slot variant and
+workspace from `metadata.json` when possible. Use `eval impact validate` after exporting visible
+sessions to write `confounds.json`; missing exports are reported as critical confounds rather than
+silently accepted. Use `eval impact score` to write the manual `score.json` artifact for a slot.
+Each of these commands refreshes `manifest.json` and `manifest.md` so the run inventory stays
+current.
+
+The report and manifest commands read an existing run directory with `metadata.json`, result
+captures, optional `score.json` files, and optional `confounds.json`. The `eval impact report`
+command compares the run's primary variants, normally `no_aiwiki_workflow` versus
+`aiwiki_ambient_memory_workflow`, using first-attempt metrics only: `first_pass` captures count
+toward the signal, while `final` repair captures stay diagnostic. The command reports
+first-attempt success rate, average score, attempts, human nudges, changed files, untracked files,
+change-profile splits for project files versus AI wiki telemetry and user-owned wiki churn, and
+whether the run is ready for shareable causal claims. It does not run agents.
+
+Use `eval impact manifest` to audit run identity before interpreting scores. It reports the
+baseline ref, prompt hashes, model, reasoning effort, execution surface, slot-to-variant mapping,
+session export presence, confounds, and captured artifact paths.
 
 Use `eval impact summarize` to aggregate multiple captured runs into a product-level dashboard.
 It reports each family's primary outcome, product signal, shareability, success and score deltas,
@@ -516,7 +635,7 @@ Even with `--purge-user-docs --yes`, the shared home wiki under `~/ai-wiki/syste
 - `ai-wiki/index.md` is a repo-owned map and is not treated as a starter-drift upgrade target by `doctor`.
 - `ai-wiki/workflows.md` remains user-owned; package-managed workflow updates land in `ai-wiki/_toolkit/workflows.md` instead of rewriting the repo-owned file.
 - `.env.aiwiki` stores the current local actor identity in a managed block. It is gitignored and should not be committed.
-- `ai-wiki/metrics/reuse-events/<handle>.jsonl` and `ai-wiki/metrics/task-checks/<handle>.jsonl` are user-owned evidence data. `ai-wiki/work/events/<handle>.jsonl` is user-owned work state. Package-managed aggregate views are regenerated under `ai-wiki/_toolkit/metrics/` and `ai-wiki/_toolkit/work/`; memory diagnostics are generated under `ai-wiki/_toolkit/diagnostics/`; consolidation queues are generated under `ai-wiki/_toolkit/consolidation/`. The installer ignores those generated paths by default in `.gitignore`.
+- `ai-wiki/metrics/reuse-events/<handle>.jsonl` and `ai-wiki/metrics/task-checks/<handle>.jsonl` are user-owned evidence data. `ai-wiki/work/events/<handle>.jsonl` is user-owned work state. Package-managed aggregate views are regenerated under `ai-wiki/_toolkit/metrics/` and `ai-wiki/_toolkit/work/`; handle-scoped metrics are regenerated under `ai-wiki/_toolkit/metrics/by-handle/<handle>/`; memory diagnostics, consolidation queues, promotion reports, usefulness reports, and weekly reports are written under handle-scoped generated paths where they depend on a handle. The installer ignores those generated paths by default in `.gitignore`.
 - Legacy flat files such as `ai-wiki/metrics/reuse-events.jsonl` and `ai-wiki/metrics/task-checks.jsonl` are still read for compatibility, but new writes should use the handle-sharded layout.
 - `aiwiki-toolkit doctor --suggest-index-upgrade` prints suggested replacements for missing repo starter docs and repo-owned companion docs such as `ai-wiki/workflows.md`, but it does not overwrite them automatically.
 - Package-owned `.agents/skills/ai-wiki-reuse-check/**`, `.agents/skills/ai-wiki-update-check/**`, `.agents/skills/ai-wiki-clarify-before-code/**`, `.agents/skills/ai-wiki-capture-review-learning/**`, and `.agents/skills/ai-wiki-consolidate-drafts/**` are refreshed by `install` so package workflow updates reach existing repos.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ai-wiki-toolkit-linux-arm64",
-  "version": "0.1.31",
+  "version": "0.1.35",
   "description": "Platform binary package for ai-wiki-toolkit (linux-arm64-glibc).",
   "license": "MIT",
   "homepage": "https://github.com/BochengYin/ai-wiki-toolkit#readme",