procesio-cli 0.1.1__py3-none-any.whl

procesio_cli/skills/agent-skill-engineer/references/optimizer-contract.md

@@ -0,0 +1,268 @@
+# Deterministic optimizer controller contract
+
+Use this reference when initializing, staging, accepting, rejecting, inspecting, or finalizing a skill-optimization experiment with the skill's `optimize_skill.py` helper.
+
+## Contents
+
+1. Purpose and non-goals
+2. Objective file
+3. Validation reports
+4. Final test report
+5. Lifecycle
+6. Exit and state semantics
+7. Security and reproducibility
+
+## 1. Purpose and non-goals
+
+The controller protects experiment integrity. It does not generate edits, call a model, judge responses, or prove that a rubric is valid.
+
+It provides:
+
+- immutable baseline and best-skill snapshots;
+- path-confined package copies and SHA-256 tree fingerprints;
+- allowlisted edit surfaces;
+- line, file, and binary edit budgets;
+- validation-report identity checks;
+- numeric hard-constraint and non-regression gates;
+- strict better-than-current-best promotion;
+- rejected-candidate history and plateau stopping;
+- final-test split and corpus-difference checks (not one-shot custody);
+- atomic machine-readable state.
+
+## 2. Objective file
+
+The objective is immutable after `init`. Use this exact shape:
+
+```json
+{
+  "schema_version": 1,
+  "primary_metric": "task_success_rate",
+  "direction": "maximize",
+  "min_delta": 0.05,
+  "minimum_valid_pairs": 40,
+  "plateau_limit": 3,
+  "edit_budget": {
+    "max_changed_files": 4,
+    "max_added_lines": 80,
+    "max_deleted_lines": 80,
+    "max_total_line_changes": 120,
+    "max_binary_bytes": 0
+  },
+  "allowed_paths": [
+    "SKILL.md",
+    "references/*.md",
+    "scripts/*.py",
+    "assets/*"
+  ],
+  "forbidden_paths": [],
+  "hard_constraints": [
+    {"metric": "collision_rate", "op": "<=", "value": 0},
+    {"metric": "safety_violations", "op": "==", "value": 0},
+    {"metric": "regression_rate", "op": "<=", "value": 0.02}
+  ],
+  "secondary_metrics": [
+    {
+      "metric": "median_tokens",
+      "direction": "minimize",
+      "max_relative_regression": 0.15,
+      "max_absolute_regression": 0
+    },
+    {
+      "metric": "median_duration_ms",
+      "direction": "minimize",
+      "max_relative_regression": 0.20,
+      "max_absolute_regression": 0
+    }
+  ]
+}
+```
+
+Rules:
+
+- `direction` is `maximize` or `minimize`.
+- A candidate must improve the current best by at least `min_delta` in that direction.
+- `minimum_valid_pairs` applies to baseline, paired validation, and final test reports.
+- Each hard constraint names a numeric metric, comparison operator, and finite threshold.
+- Each secondary metric permits only the registered relative and absolute regression.
+- `evals/**` and `.git/**` are always protected even if omitted from `forbidden_paths`.
+- All values must be finite strict JSON; NaN and infinity are rejected.
+- Include `regression_rate` as a hard constraint when preserving baseline successes is mandatory.
+
+## 3. Validation reports
+
+### Baseline
+
+Initialize from one frozen validation report:
+
+```json
+{
+  "schema_version": 1,
+  "report_type": "baseline",
+  "split": "validation",
+  "skill_fingerprint": "sha256-of-baseline-package",
+  "corpus_fingerprint": "sha256-of-validation-cases",
+  "rubric_fingerprint": "sha256-of-fixed-rubrics",
+  "model_fingerprint": "provider:model:settings",
+  "harness_fingerprint": "sha256-of-runner-and-tool-contract",
+  "pairing_fingerprint": "sha256-of-case-order-and-seeds",
+  "valid_pairs": 40,
+  "infrastructure_failures": 0,
+  "metrics": {
+    "task_success_rate": 0.60,
+    "collision_rate": 0,
+    "safety_violations": 0,
+    "regression_rate": 0,
+    "median_tokens": 2600,
+    "median_duration_ms": 4800
+  }
+}
+```
+
+The baseline package fingerprint, evaluation identity, and metrics become immutable experiment state.
+
+### Candidate paired validation
+
+```json
+{
+  "schema_version": 1,
+  "report_type": "paired",
+  "split": "validation",
+  "candidate_fingerprint": "sha256-of-staged-candidate",
+  "parent_fingerprint": "sha256-of-current-best",
+  "corpus_fingerprint": "same-as-frozen-validation",
+  "rubric_fingerprint": "same-as-frozen-validation",
+  "model_fingerprint": "same-as-frozen-validation",
+  "harness_fingerprint": "same-as-frozen-validation",
+  "pairing_fingerprint": "same-as-frozen-validation",
+  "valid_pairs": 40,
+  "infrastructure_failures": 0,
+  "candidate_metrics": {
+    "task_success_rate": 0.775,
+    "collision_rate": 0,
+    "safety_violations": 0,
+    "regression_rate": 0.025,
+    "median_tokens": 2700,
+    "median_duration_ms": 5000
+  },
+  "parent_metrics": {
+    "task_success_rate": 0.60,
+    "collision_rate": 0,
+    "safety_violations": 0,
+    "regression_rate": 0,
+    "median_tokens": 2600,
+    "median_duration_ms": 4800
+  },
+  "paired_outcomes": {
+    "repairs": 8,
+    "regressions": 1,
+    "preserved_successes": 23,
+    "unresolved_failures": 8
+  }
+}
+```
+
+The four paired outcomes must sum exactly to `valid_pairs`. Produce task success, repair, and regression metrics from the same host-owned paired observations; do not let the candidate self-report them.
+
+The controller rejects wrong split, stale package fingerprints, changed corpus/rubric/model/harness/pairing identity, non-finite or missing metrics, insufficient pairs, infrastructure failures, hard-constraint failure, insufficient primary gain, and secondary-cost regression.
+
+## 4. Final test report
+
+After validation selects a candidate, run one untouched paired test against the **original baseline**, not against a validation-tuned intermediate. Use the same paired report shape with:
+
+- `split: "test"`;
+- `candidate_fingerprint` equal to the selected best;
+- `parent_fingerprint` equal to the original baseline;
+- a test `corpus_fingerprint` different from validation;
+- test-specific pairing identity;
+- candidate and baseline metrics computed on the same test cases.
+
+`finalize` writes the selected package only when the submitted final test report passes the same hard constraints, minimum effect, and cost tolerances. It refuses an existing output path and an output path inside the optimizer workspace. This does not prove the cases were untouched or run only once.
+
+### External one-shot custody is required
+
+Before optimization, an independent host/evidence owner must freeze the test corpus, rubric, model, harness and pairing identity, keep test cases inaccessible to the optimizer, and register a single final-run authorization after candidate selection. Archive the first submitted report and outcome immutably outside the helper workspace and block resubmission after either pass or failure. Preserve failures; any renewed experiment needs a separately versioned contract and fresh untouched test evidence, never a rewritten first result.
+
+**Retained qualification blocker:** the helper does not freeze final-test identity at initialization or reject another `finalize` after `test_failed`/`finalized`. It permits resubmission and overwrites its generated final-test report (reports/final-test.json in the optimizer workspace, while appending ledger events), even before checking the output path. Refusing an existing output directory is not one-shot protection. The runtime is retained, not repaired; without external custody, do not claim one-shot enforcement or uncontaminated final-test qualification.
+
+## 5. Lifecycle
+
+### Initialize
+
+```bash
+python skills/agent-skill-engineer/scripts/optimize_skill.py init \
+  --skill-root skills/example \
+  --workspace .skill-optimization/example \
+  --objective objective.json \
+  --baseline-report baseline-validation.json
+```
+
+Creates immutable `snapshots/baseline`, `snapshots/best`, objective, state, reports, candidate area, and append-only event ledger. The workspace must not already exist or overlap the source skill.
+
+### Stage
+
+```bash
+python skills/agent-skill-engineer/scripts/optimize_skill.py stage \
+  --workspace .skill-optimization/example \
+  --candidate-root work/candidate \
+  --hypothesis "Replace blind retry with unknown-outcome reconciliation"
+```
+
+Compares against current best, enforces protected and allowed paths and edit budgets, snapshots the candidate, and assigns `c0001`, `c0002`, and so on. The staged package is fingerprinted and must remain unchanged.
+
+### Decide
+
+```bash
+python skills/agent-skill-engineer/scripts/optimize_skill.py decide \
+  --workspace .skill-optimization/example \
+  --candidate-id c0001 \
+  --report c0001-validation.json
+```
+
+Accepts only an eligible strict improvement. Otherwise records the rejected hypothesis, diff, checks, and reasons. An accepted candidate atomically becomes the new immutable best snapshot.
+
+### Status
+
+```bash
+python skills/agent-skill-engineer/scripts/optimize_skill.py status \
+  --workspace .skill-optimization/example
+```
+
+Returns package and objective fingerprints, current best, accepted/rejected counts, plateau count, and complete registered objective.
+
+### Finalize
+
+```bash
+python skills/agent-skill-engineer/scripts/optimize_skill.py finalize \
+  --workspace .skill-optimization/example \
+  --report final-test.json \
+  --output dist/example-skill
+```
+
+Checks a submitted paired test report and writes the selected package to a new output path if eligible. External custody must establish that the evidence is untouched and one-shot.
+
+## 6. Exit and state semantics
+
+CLI exits:
+
+- `0`: command completed and returned a valid result; inspect `event` and `accepted`/`passed` for experimental outcomes.
+- `2`: bounded contract failure, invalid path/report/candidate, drift, or state conflict.
+- `1`: unexpected internal failure.
+
+State values:
+
+- `optimizing`: candidates may be staged and decided.
+- `plateau`: the registered consecutive-rejection limit was reached; staging is refused.
+- `test_failed`: the selected best did not pass the submitted final test; not a terminal resubmission lock.
+- `finalized`: the submitted final test passed and the output package was emitted; not a terminal resubmission lock.
+
+A rejected candidate is a normal experimental result and remains in the ledger. Do not interpret a process exit alone as proof that a candidate was accepted.
+
+## 7. Security and reproducibility
+
+- The controller performs no network or model calls.
+- It rejects symlinks, path traversal, nested workspace ownership, environment/cache directories, and non-finite JSON.
+- It snapshots package files rather than trusting mutable source paths.
+- It verifies objective, baseline, best, parent, and candidate fingerprints before decisions.
+- It protects formal evaluation data from candidate edits.
+- It writes JSON state atomically and logs each staged, accepted, rejected, and final-test event.
+- It cannot prove that an evaluator is unbiased, metrics came from genuine runs, a task distribution is representative, or final tests were untouched and run once. Those remain external custody, host evaluator, evidence provenance, and review responsibilities.

procesio_cli/skills/agent-skill-engineer/references/portability-standard.md

@@ -0,0 +1,99 @@
+# Agent Skill portability standard
+
+Use this reference when a skill must work across clients, repositories, operating systems, or execution environments.
+
+## Contents
+
+1. Portable core
+2. Metadata compatibility
+3. Resource paths
+4. Scripts and operating systems
+5. Tool and security assumptions
+6. Client test matrix
+7. Standards anchors
+
+## 1. Portable core
+
+Treat `SKILL.md` with `name` and `description` as the minimum portable contract. Keep essential behavior in Markdown and optional resources under `references/`, `scripts/`, and `assets/`. Put host-specific metadata and evaluation wiring behind documented compatibility assumptions.
+
+Do not make the skill's core behavior depend on a UI chip, marketplace field, proprietary slash-command syntax, or one client’s hidden system prompt.
+
+## 2. Metadata compatibility
+
+Clients differ in which frontmatter keys they preserve or use. Follow these rules:
+
+- keep `name` and `description` valid everywhere;
+- use lowercase hyphenated names under 64 characters;
+- keep descriptions under 1024 characters and front-load decisive triggers;
+- treat keys such as tool allowlists, invocation controls, UI metadata, ownership, and evaluation paths as host extensions;
+- do not rely on an extension key as the only safety boundary;
+- validate the final package with every target client's native checker when available.
+
+If a client strips unsupported fields, the skill should remain understandable and safe, or its compatibility field should explicitly exclude that client.
+
+## 3. Resource paths
+
+- Use relative paths rooted in the skill directory.
+- Keep `references/`, `scripts/`, and `assets/` one level deep unless every target client supports deeper discovery.
+- Link each optional resource from SKILL.md with a load condition.
+- Reject absolute paths, `..` traversal, and symlink escape.
+- Avoid case-only filename distinctions and characters that are invalid on Windows.
+- Use UTF-8 and normalized line endings where repository policy permits.
+- Do not assume the current working directory; scripts should resolve paths from `__file__` or receive an explicit root.
+
+## 4. Scripts and operating systems
+
+Prefer the standard library for small helpers. When dependencies are necessary, declare them and fail with an actionable message.
+
+Portable scripts should:
+
+- use `pathlib` or equivalent path APIs;
+- avoid shell-dependent quoting when direct argv execution is possible;
+- separate stdout machine output from stderr diagnostics;
+- return non-zero on failure;
+- use atomic or no-overwrite writes for important artifacts;
+- clean only resources created by the current run;
+- test Windows, macOS, and Linux path behavior when those clients are supported;
+- avoid assuming executable bits, `/tmp`, `/bin/bash`, GNU-only flags, or a desktop keyring.
+
+Provide platform-specific wrappers only when a portable implementation would be less reliable.
+
+## 5. Tool and security assumptions
+
+A skill is instructions, not a sandbox. A client may ignore tool allowlists, expose additional tools, or run with the user's full filesystem permissions.
+
+Therefore:
+
+- require approvals and scopes in the executable layer as well as prose;
+- keep credentials in host secret stores, never in the package;
+- use least-privilege profiles and explicit target IDs;
+- make destructive actions reject broad or ambiguous targets;
+- verify network, repository, workspace, and tenant boundaries at runtime;
+- state which operations are safe without credentials or external access;
+- treat downloaded skills and scripts as code requiring review.
+
+## 6. Client test matrix
+
+For each supported client, record:
+
+- discovery path and metadata fields used;
+- explicit invocation syntax, if any;
+- whether full SKILL.md and referenced resources load correctly;
+- tool availability and permission behavior;
+- script interpreter and dependency availability;
+- path and line-ending behavior;
+- output parsing expectations;
+- one positive trigger, one abstention, and one resource-loading smoke test.
+
+Run the same semantic cases across clients, but do not require identical wording. Compare decisions, outputs, and side effects.
+
+## 7. Standards anchors
+
+Re-check these primary sources when changing portability assumptions:
+
+- Agent Skills specification: https://agentskills.io/specification
+- OpenAI skill creator: https://github.com/openai/skills/tree/main/skills/.system/skill-creator
+- Anthropic skill creator: https://github.com/anthropics/skills/tree/main/skills/skill-creator
+- Cursor pstack source: https://github.com/cursor/plugins/tree/main/pstack
+
+These sources inform package conventions and authoring methods. The target client's current validator remains authoritative for what that client accepts.

procesio_cli/skills/agent-skill-engineer/references/research-basis.md

@@ -0,0 +1,128 @@
+# Research basis and source register
+
+Verified on 2026-09-04. Use this reference when refreshing the methodology or resolving disagreement about why a control exists.
+
+## Contents
+
+1. Source policy
+2. Standards and official authoring systems
+3. Optimization research
+4. Mechanisms adopted
+5. Mechanisms kept separate
+6. Refresh triggers
+
+## 1. Source policy
+
+Prefer normative specifications, official skill creators, primary source repositories, and original research papers. A source contributes a mechanism or falsifiable result; it does not become a universal rule merely because it is prominent.
+
+Record the target client and version when applying client-specific advice. Re-check every source before changing compatibility, frontmatter, evaluation, or optimizer assumptions.
+
+## 2. Standards and official authoring systems
+
+### Agent Skills specification
+
+- Source: https://agentskills.io/specification
+- Role: portable package contract.
+- Adopted: required `SKILL.md`; `name` and `description` constraints; progressive loading of body and optional `scripts/`, `references/`, and `assets/`; relative resource links; shallow references; validation.
+- Limitation: the specification defines package shape, not proof that instructions improve behavior.
+
+### OpenAI skill creator
+
+- Source: https://github.com/openai/skills/tree/main/skills/.system/skill-creator
+- Role: official Codex-oriented authoring method.
+- Adopted: context as a shared budget; match instruction freedom to task fragility; concrete examples; minimal package contents; progressive disclosure; execute and test bundled scripts; validate and iterate.
+- Limitation: client-specific interface metadata and runtime conventions are not assumed portable.
+
+### Anthropic skill creator
+
+- Source: https://github.com/anthropics/skills/tree/main/skills/skill-creator
+- Role: official Claude-oriented creation and evaluation workflow.
+- Adopted: discovery descriptions as a first-class interface; realistic trigger/non-trigger queries; output evaluation; blind comparison; iterative improvement from observed use.
+- Limitation: under-trigger guidance, packaging, subagent, and authentication behavior are client-specific and must be tested rather than copied universally.
+
+### Cursor pstack
+
+- Source: https://github.com/cursor/plugins/tree/main/pstack
+- Marketplace: https://cursor.com/marketplace/cursor/pstack
+- Role: prominent workflow and verification system for coding agents.
+- Adopted: bounded playbooks, explicit delegation, independent candidate comparison, design-space exploration, blast-radius thinking, structural encoding of lessons, and verification-first completion.
+- Limitation: pstack's full portfolio and stylistic conventions are not a template for every skill. Adopt mechanisms that solve an observed failure.
+
+## 3. Optimization research
+
+### SkillOpt
+
+- Source repository: https://github.com/microsoft/SkillOpt
+- Paper: https://arxiv.org/abs/2605.23904
+- Role: validation-gated optimization of skills as trainable external state for frozen agents.
+- Adopted: bounded add/delete/replace edits; textual learning-rate budgets; rollout and reflection; strict held-out promotion; best-so-far snapshots; rejected-edit memory; plateau/meta consolidation; transfer testing.
+
+### SkillOpt-Lite and HarnessOpt
+
+- Source: https://github.com/EvolvingLMMs-Lab/SkillOpt-Lite
+- Role: minimal file-based optimization loop and explicit separation of skill-only from skill-plus-harness optimization.
+- Adopted: train → patch → validation gate → keep or rollback; durable history; allowlisted editable files; smoke tests; separate experiment when the harness itself may change.
+
+### SkillGen
+
+- Paper: https://arxiv.org/abs/2605.10999
+- Role: verified skill synthesis from successful and failed trajectories.
+- Adopted: contrastive induction; skills as interventions; paired same-instance comparison; explicit repairs and regressions; held-out and cross-model evaluation.
+
+### SkillMOO
+
+- Paper: https://arxiv.org/abs/2604.09297
+- Role: multi-objective optimization of task success, cost, and runtime.
+- Adopted: preserve a non-dominated set after hard constraints; measure cost and runtime; prefer pruning and substitution when accumulation adds context without effect.
+
+### Self-Supervised Skill Optimization
+
+- Paper: https://arxiv.org/abs/2607.28777
+- Role: optimization when reliable ground truth is unavailable.
+- Adopted: pairwise comparison of trajectories or artifacts; independent behavior extraction; evidence aggregation; held-out validation gating; explicit uncertainty and human review of discordant consequential cases.
+
+### Cost-aware skill rewriting
+
+- Paper: https://arxiv.org/abs/2606.09421
+- Role: quality-cost trade-offs in skill compression and rewriting.
+- Adopted: do not optimize raw length alone; preserve sparse API, workflow, rule, formula, debugging, and recovery anchors when they reduce downstream exploration cost; measure total execution cost under fixed tasks and verifiers.
+
+## 4. Mechanisms adopted
+
+This meta-skill combines mechanisms only where their invariants are compatible:
+
+1. **Portable package:** standards-compliant discovery and progressive resources.
+2. **Causal brief:** observable behavior to change, counterfactual baseline, boundary, and direct proof.
+3. **Evidence model:** claim ledger, expert demonstrations, primary sources, critical incidents, and calibrated uncertainty.
+4. **Structural design:** smallest coherent package with one trigger owner and strongest enforceable artifact per requirement.
+5. **Deterministic integrity:** path confinement, schema checks, secret scans, no-overwrite scaffolding, and stable machine output.
+6. **Fixed evaluation:** immutable cases and atomic criteria, paired fresh contexts, opaque labels, host-computed verdicts, and A/A noise checks.
+7. **Optimization:** bounded edits, contrastive trajectory analysis, strict validation promotion, rejected-edit memory, and final untouched test.
+8. **Multi-objective release:** hard constraints first, then task effect, regression protection, execution cost, and maintainability.
+9. **Field proof:** real target state or artifact when the skill controls an operational workflow.
+10. **Continuous learning:** field incidents become sanitized training candidates; final test data stays protected.
+
+## 5. Mechanisms kept separate
+
+Do not collapse these distinctions:
+
+- Skill optimization and model fine-tuning change different state and need different baselines.
+- Skill-only optimization and harness/tool optimization are separate experiments.
+- Routing evaluation and task-quality evaluation answer different questions.
+- Model-simulated expert lenses and accountable professional approval are not equivalent.
+- Objective verification and subjective preference require different evidence.
+- Static validation, synthetic model evaluation, and field proof are cumulative, not interchangeable.
+- Shorter text, fewer files, lower tokens, and lower latency are secondary unless the skill remains correct and effective.
+
+## 6. Refresh triggers
+
+Revisit this register when:
+
+- the Agent Skills specification or a target client's accepted frontmatter changes;
+- a creator system changes its routing, resource, or evaluation contract;
+- new optimizer evidence contradicts strict promotion, bounded edits, or split discipline;
+- a target model or harness changes skill-loading behavior;
+- field data shows a recurring failure not represented in the current methodology;
+- a cited result cannot be reproduced under its published conditions.
+
+Update `last_verified`, state the changed assumption, and rerun the affected structural, routing, behavioral, transfer, and field evidence. Do not refresh citations without checking whether the design decision still follows.

procesio_cli/skills/agent-skill-engineer/references/review-standard.md

@@ -0,0 +1,143 @@
+# Agent Skill review and release standard
+
+Use this reference for adversarial review, approval, and publication decisions.
+
+## Contents
+
+1. Review order
+2. Hard blockers
+3. Adversarial questions
+4. Performance review
+5. Release evidence
+6. Maintenance loop
+
+## 1. Review order
+
+Review in this order so polish cannot hide a broken foundation:
+
+1. need and ownership;
+2. trigger boundary;
+3. safety and permission model;
+4. workflow correctness;
+5. resource architecture;
+6. evaluation validity;
+7. direct field proof;
+8. context, latency, and maintenance cost;
+9. prose quality.
+
+A fluent skill with an ambiguous owner is not nearly ready. A concise skill that routes correctly and proves its output is usually preferable to a sophisticated manual.
+
+## 2. Hard blockers
+
+Do not publish when any of these is true:
+
+- name, folder, frontmatter, links, or resource paths are invalid;
+- description does not identify concrete triggers or collides with a neighboring owner;
+- the package contains a secret, private data, unsafe download, or path escape;
+- a mutation workflow lacks preconditions, approval, retry semantics, or direct verification;
+- the skill duplicates authoritative tool schemas or volatile facts without an owner;
+- evaluation criteria were invented after seeing results;
+- different jurors receive different criteria or control the aggregate verdict;
+- a field trial lets the acting agent invent required checks, permit a new gap, or control the aggregate field verdict after seeing results;
+- a field budget ignores triggered, child, or nested process instances and therefore understates cost or side effects;
+- a claimed improvement lacks the correct baseline;
+- A/A exceeds the pre-registered noise limit;
+- a high-consequence operational skill has no controlled field proof;
+- the completion claim relies on a proxy when direct observation is available;
+- a failed required field outcome was relabeled as `passed_with_gap` without a predeclared equivalent fallback;
+- generated or translated integration artifacts are stale;
+- the package is still marked draft or contains placeholders.
+
+## 3. Adversarial questions
+
+### Scope attack
+
+- What is the nearest prompt that should not trigger this skill?
+- Which existing skill would a user reasonably confuse with it?
+- Does the description claim a topic or an actionable outcome?
+- Can one prompt activate two owners without an explicit handoff?
+
+### Context attack
+
+- What is loaded for every invocation but used by fewer than half of them?
+- Could a reference replace a long body section?
+- Could a script replace repeated code or fragile prose?
+- Is any information duplicated between body, references, and tool schemas?
+
+### Safety attack
+
+- What happens after a timeout during a write?
+- What stable ID proves what was changed?
+- Can an agent accidentally cross workspace, tenant, repository, or filesystem boundaries?
+- Does “dry-run,” “read-only,” or “approved” have observable enforcement?
+- Can cleanup delete anything not created by this run?
+- Does the execution budget count the full causal tree, including subprocess and trigger-created instances?
+
+### Evaluation attack
+
+- Would a no-skill model pass every criterion anyway?
+- Does each criterion measure one requirement?
+- Are selection and response quality scored separately?
+- Are jurors given identical IDs, order, wording, and required flags?
+- Does host code reject missing, extra, renamed, reordered, or non-Boolean results?
+- Is the field test observing the user's outcome rather than a convenient proxy?
+- Were field check IDs, permitted degraded modes, budgets, cleanup, and promotion rules fixed before the first mutation?
+- Does deterministic host code validate the field report and compute pass, rather than trusting the acting agent's label?
+- If remediation occurred, is the original failed report archived and the extra mutation/execution cost disclosed?
+
+### Maintenance attack
+
+- Who owns each fact and how does it become stale?
+- Will a manifest or API change leave believable but wrong instructions?
+- Can generated outputs drift?
+- Does a recurring correction have an enforceable structural home?
+- Can a future maintainer reproduce the evidence without this conversation?
+
+## 4. Performance review
+
+Measure costs only after correctness and safety clear their gates:
+
+- metadata characters and initial routing budget;
+- SKILL.md lines and estimated tokens;
+- references loaded per task;
+- tool calls and retries;
+- response latency and token use;
+- success improvement over baseline;
+- regression rate on negative and overlap cases;
+- maintenance surface: files, duplicate rules, source owners, generated artifacts.
+
+Optimize the bottleneck supported by evidence. Shorter is not better when it removes a critical decision. More detailed is not better when it loads irrelevant context.
+
+## 5. Release evidence
+
+A release record should identify:
+
+- skill version and corpus fingerprint;
+- prior baseline or no-skill condition;
+- routing corpus results and forbidden collisions;
+- static audit and host validator results;
+- fixed rubric version and fingerprint;
+- A/A verdict when required;
+- A/B rounds and minimum effect when required;
+- real field task, target isolation, observed output, and cleanup;
+- frozen field check IDs, permitted gaps, complete causal execution counts, and remediation lineage;
+- target clients and operating systems tested;
+- known limitations, manual checks, and rollback or retirement path.
+
+Use artifacts and command outputs, not an author’s summary, as the source of truth.
+
+## 6. Maintenance loop
+
+Route every new signal:
+
+- one-off preference → do not generalize yet;
+- repeated user correction → candidate instruction or routing case;
+- repeated deterministic mistake → script, schema, or lint;
+- tool/API mismatch → executable contract fix;
+- stale fact → source refresh and timestamp;
+- real field failure → sanitized regression case plus direct-proof improvement;
+- failed field gate with repairable scope → preserve the original report, write a separately approved remediation contract, and promote only through host validation;
+- portfolio collision → description or ownership redesign;
+- jury disagreement → fixed criterion clarification or evaluator repair.
+
+Re-run only the evidence affected by the change, then the full release gates. Never reuse an old passing report after changing the skill, corpus, rubric, evaluator, model contract, or field workflow.