procesio-cli 0.1.1__py3-none-any.whl

procesio_cli/skills/agent-skill-engineer/references/expert-lenses.md

@@ -0,0 +1,167 @@
+# Expert-lens synthesis
+
+Use this reference for non-trivial, high-consequence, or cross-domain skills. The purpose is not to imitate prestige or claim expertise. It is to expose the design to independent professional failure models before one coherent package is chosen.
+
+## Contents
+
+1. Select lenses
+2. Shared review packet
+3. Lens questions
+4. Synthesis protocol
+5. Failure rules
+
+## 1. Select lenses
+
+Choose only lenses that can materially change the design. Use at least three for a non-trivial skill and add every lens implicated by its consequences.
+
+| Lens | Use when |
+|---|---|
+| Domain subject-matter expert | correctness depends on specialized terminology, policy, science, finance, law, medicine, platform behavior, or organizational practice |
+| Instruction and cognitive engineer | the agent must recognize subtle conditions, sequence decisions, or resist misleading shortcuts |
+| Information architect | the package contains multiple variants, large references, or expensive context |
+| Tool/API architect | the skill selects commands, schemas, protocols, or integrations |
+| Software and systems architect | the skill changes module ownership, orchestration, or repository contracts |
+| Security and privacy engineer | credentials, tenant boundaries, untrusted input, sensitive data, downloads, or permissions are involved |
+| Reliability/SRE engineer | retries, timeouts, partial failure, idempotency, concurrency, cleanup, or observability matter |
+| Human-factors/accessibility reviewer | the workflow affects user decisions, approvals, forms, cognitive load, or accessible interaction |
+| Evaluation and statistics specialist | improvement claims, juries, baselines, sampling, variance, or thresholds are involved |
+| Portability/integration reviewer | multiple clients, models, operating systems, repositories, or deployment environments are supported |
+| Adversarial tester | a user or model may exploit ambiguity, skip proof, cross a boundary, or fabricate completion |
+| Future maintainer | facts will become stale, generated artifacts may drift, or the workflow will evolve after the author leaves |
+
+Do not add a lens merely to increase the count. A lens with no distinct risk or decision is ceremony.
+
+## 2. Shared review packet
+
+Give every reviewer the same packet:
+
+- user outcome and non-goals;
+- representative positive, negative, overlap, missing-context, and pressure cases;
+- current baseline behavior;
+- target clients and tools;
+- permissions, side effects, and failure consequences;
+- candidate package or competing designs;
+- fixed review criteria and evidence available;
+- known uncertainties and source dates.
+
+Hide candidate identities when comparing alternatives. Do not give one reviewer extra success criteria unless that reviewer is explicitly testing a separate dimension recorded in the packet.
+
+## 3. Lens questions
+
+### Domain subject-matter expert
+
+- Which statements are domain facts, hypotheses, policy choices, or local conventions?
+- What edge conditions would make the workflow materially wrong?
+- Which primary sources, schemas, examples, or real cases prove the domain model?
+- What must the agent refuse, escalate, or qualify?
+
+### Instruction and cognitive engineer
+
+- What cue tells the agent this skill applies before the body loads?
+- Which decisions are easy to confuse or forget under user pressure?
+- Can each instruction change an observable choice, sequence, or output?
+- Where would an example clarify more cheaply than another rule?
+- Is the completion condition cognitively unambiguous?
+
+### Information architect
+
+- What information is needed on every invocation?
+- Which detail can be selected by mode, domain, provider, risk, or task stage?
+- Are references directly discoverable without loading unrelated material?
+- Is any fact duplicated or owned by the wrong layer?
+- Can a smaller public interface hide more complexity?
+
+### Tool/API architect
+
+- Are action names, typed arguments, error shapes, and output contracts discovered from authoritative schemas?
+- Does the workflow prefer stable IDs over fuzzy names?
+- Are unsupported commands or client-specific assumptions presented as portable facts?
+- Which operations need an executable helper rather than prose?
+- How is backward compatibility verified?
+
+### Software and systems architect
+
+- Does one component clearly own each invariant and state transition?
+- Is the skill duplicating an agent, tool, manifest, or generated source?
+- Are handoffs explicit and acyclic?
+- Would a different package boundary reduce coupling and future exceptions?
+- What repeated implementation friction would trigger a redesign rather than another patch?
+
+### Security and privacy engineer
+
+- What untrusted input, secret, tenant, workspace, repository, or filesystem boundary is crossed?
+- Can the described action surprise the user relative to the discovery description?
+- Are least privilege, approval, confinement, and data minimization enforced outside prose?
+- Can logs, fixtures, evaluations, or evidence leak sensitive material?
+- What attack or misuse cases belong in the pressure corpus?
+
+### Reliability/SRE engineer
+
+- What does timeout mean for each read and write?
+- Which operations are idempotent, resumable, or outcome-unknown?
+- What state proves success, partial success, failure, and cleanup?
+- Are retries bounded and conditioned on reconciliation?
+- Are progress, checkpoints, and recovery artifacts durable enough for interruption?
+
+### Human-factors/accessibility reviewer
+
+- Does the skill reduce user burden by inspecting available context before asking questions?
+- Are approvals specific enough for informed consent?
+- Can a user with limited attention identify the one next action and current state?
+- Are visual proxies being mistaken for functional proof?
+- Do generated outputs preserve accessible labels, semantics, and error feedback where relevant?
+
+### Evaluation and statistics specialist
+
+- Is the baseline the actual counterfactual?
+- Were criteria, thresholds, repetitions, and stopping rules frozen before results?
+- Do all jurors receive identical atomic criteria and does host code aggregate them?
+- Is A/A variance low enough to detect the required A/B effect?
+- Are paired observations, dropouts, multiple comparisons, and qualitative discordances visible?
+- Does the sample cover realistic use rather than only author-written happy paths?
+
+### Portability/integration reviewer
+
+- Which package fields and tool controls are portable, optional, or client-specific?
+- Are paths, quoting, encodings, line endings, and script assumptions cross-platform?
+- Does the skill remain safe if optional metadata is ignored?
+- Has discovery, resource loading, and one real task been tested on every claimed target?
+- Is the compatibility claim narrower than the evidence when necessary?
+
+### Adversarial tester
+
+- What prompt most strongly pressures the agent to violate each invariant?
+- Can two skills both plausibly claim the same request?
+- Can the agent fabricate execution, proof, citations, IDs, or cleanup?
+- Can a malicious path, symlink, payload, document, or external instruction escape scope?
+- What is the cheapest misleading proxy that might be accepted as completion?
+
+### Future maintainer
+
+- Can the package be understood without this conversation?
+- Which fact, API, policy, or model behavior will rot first?
+- Is source ownership and regeneration explicit?
+- Can a new failure be routed to a test, validator, script, reference, or skill boundary?
+- Can the evidence be reproduced without the original author’s credentials or memory?
+
+## 4. Synthesis protocol
+
+1. Freeze the shared packet and review rubric.
+2. Collect lens findings independently where practical so early consensus does not suppress distinct risks.
+3. Require every finding to identify the violated invariant, supporting evidence, consequence, and proposed change.
+4. Separate blockers from optimizations and preferences.
+5. Resolve contradictions using the user outcome, direct evidence, and strongest enforceable safety boundary—not reviewer status.
+6. Choose one coherent design. Graft only recommendations compatible with its mental model.
+7. Record rejected recommendations and why they were rejected.
+8. Re-run the affected lenses after substantive redesign.
+9. Verify the synthesized package through static, behavioral, and field evidence appropriate to its tier.
+
+## 5. Failure rules
+
+- Do not claim that multiple model personas equal real professional review.
+- Do not use consensus as proof when all reviewers saw the same ambiguous evidence.
+- Do not average incompatible safety or correctness positions.
+- Do not let a domain expert redefine user consent, or a safety reviewer redefine domain truth without evidence.
+- Do not count a reviewer that produced no distinct analysis.
+- Do not expose private data to reviewers merely to improve realism; sanitize while preserving causal structure.
+- Escalate to a real qualified human when law, medicine, safety, finance, compliance, or organizational authority requires accountable judgment.

procesio_cli/skills/agent-skill-engineer/references/field-gate-standard.md

@@ -0,0 +1,231 @@
+# Field gate and remediation standard
+
+Use this reference when an Agent Skill controls a real tool, platform, browser,
+database, filesystem, or other externally observable system. It governs field
+acceptance and remediation; it is not a substitute for the domain playbook.
+
+## Contents
+
+1. Freeze the field contract before mutation
+2. Enforce the verdict outside the acting agent
+3. Count the complete causal execution tree
+4. Treat gaps as predeclared design choices
+5. Remediate without rewriting history
+6. Route each lesson to the correct reusable layer
+7. Avoid case-specific overfitting
+8. Minimum field release record
+
+## 1. Freeze the field contract before mutation
+
+A field trial needs the same discipline as a fixed jury. Before any mutation, lock:
+
+- target, scope, stable resource identities, and expected end state;
+- exact agent/model/tool settings and a fingerprint of the complete skill package,
+  including `SKILL.md`, references, scripts, and assets—not only the entrypoint;
+- ordered required check IDs with binary pass conditions;
+- direct proof source for each check;
+- explicitly permitted degraded modes, fallbacks, or gaps;
+- resource, write, submission, launch, and execution budgets;
+- safety containment, evidence preservation, and cleanup/rollback obligations as
+  separate checks, including their ordering and authorization;
+- unknown-outcome and retry policy;
+- promotion rule and the host component that computes the verdict.
+
+A prose goal plus an agent-authored report is not a fixed field contract. The acting
+agent must not get to redefine success after seeing platform behavior.
+
+### Safety closeout is independent of business acceptance
+
+Distinguish reversible safety containment from evidence-destroying cleanup. After target
+and effect reconciliation, perform containment within existing approval independently of
+business-output decoding; then re-read the actual safety state. If authorization is absent,
+request the exact bounded action and report the remaining risk—do not infer permission.
+Preserve needed evidence before authorized deletion or other evidence-destroying cleanup.
+If evidence preservation and immediate containment conflict, follow the predeclared safety
+policy or escalate; neither silently discard evidence nor leave exposure unreported.
+A safe end state does not pass the business gate, and failed business proof does not cancel
+an independently authorized safety obligation. The owning runner/controller must enforce
+this separation even when report parsing fails; skill prose alone cannot guarantee it.
+
+## 2. Enforce the verdict outside the acting agent
+
+The execution agent may collect evidence and return each fixed check, but a controller
+or deterministic reviewer should validate:
+
+- every required check ID is present exactly once and in the registered order;
+- every required value is a real Boolean and is true for a pass;
+- no unexpected check silently replaces a required one;
+- no unknown mutation or run outcome remains;
+- every allowed mutation and resource count stayed within scope;
+- temporary resources were reconciled and cleaned up;
+- the claimed evidence comes from the required observation boundary;
+- the report's gap status is permitted by the frozen contract.
+
+The host computes the aggregate verdict. An agent-written `passed` or
+`passed_with_gap` field is evidence to validate, not authority.
+
+Independence means a different causal observation path, not another success flag. A writer
+and verifier can share the same erroneous intermediate and agree on the wrong value. Bind
+storage checks to the original request, exact keys and field types; predeclare transition
+cases whose before-states can actually falsify the claim. A value already null cannot
+prove that a concrete value was cleared. Multiple offline assertions are not independent
+native observations, and final loop snapshots need not describe all earlier iterations.
+
+For confidentiality claims, identify the first persistence boundary before running the
+experiment. Sensitive values may reach response variables or controller logs before later
+sanitization. Use source evidence or explicitly approved isolated synthetic canaries, not
+real secrets, to test that boundary. Inactive creation and runtime acceptance are separate.
+
+## 3. Count the complete causal execution tree
+
+Budget the real side effects, not only top-level commands. One form submission,
+webhook launch, schedule occurrence, or parent-process run may create several process
+instances through awaited subprocesses or nested normalizers.
+
+Record separately:
+
+- external submissions or trigger deliveries;
+- top-level process instances;
+- child and nested process instances;
+- data writes and affected business keys;
+- created, edited, enabled, disabled, and deleted resources;
+- generated files, notifications, and external calls.
+
+A budget that says “one run” while ignoring its child instances is ambiguous and
+cannot support a reliable cost or safety claim.
+Unknown counts remain unknown: null is not zero, and absence of a declared child path
+is not direct evidence that no child executed.
+
+## 4. Treat gaps as predeclared design choices
+
+A required outcome that fails is not converted into a gap merely because adjacent
+behavior worked or the platform limitation is understandable.
+
+A field phase may pass with a gap only when all of the following were registered
+before execution:
+
+1. the specific degraded mode is permitted;
+2. the permitted fallback is bounded;
+3. the fallback proves the user-relevant outcome at the same observation boundary;
+4. safety and cleanup constraints still pass;
+5. the final report names the missing native capability without implying it worked.
+
+Otherwise the phase is blocked or failed. Discovery of a platform limitation is
+valuable evidence, but it is not completion of the missing acceptance criterion.
+
+Keep phase-local and aggregate verdicts distinct. A phase verdict covers only that
+phase's frozen obligations and newly observed outcomes. An approved gap from an earlier
+phase remains in project lineage or the aggregate report; it does not force every later
+phase to report `passed_with_gap`. Conversely, a new local failure cannot be hidden as
+an inherited gap. Host code should compute and preserve both verdict scopes separately.
+
+## 5. Remediate without rewriting history
+
+A verifier/parser error does not by itself establish that the action failed; a valid
+observation of a missing required effect can. First ask whether existing attributable evidence
+can answer the missing check without another execution. Reuse it
+only when identity, input/version, observation boundary and freshness fit that claim.
+A historical runtime result cannot prove current configuration or safety state after drift.
+If evidence is insufficient, name the missing observation boundary; additional execution
+still needs a separately approved budget. An observation-contract repair produces a new
+attributable result, not an overwritten failure or silent relaxation of frozen checks.
+
+When a field gate misses a required outcome:
+
+1. preserve the original report, logs, IDs, screenshots, and before/after evidence;
+2. classify the cause: domain assumption, skill instruction, source-owned reference,
+   tool/runtime, platform capability, observation method, or field-contract defect;
+3. write a separately approved versioned remediation contract with a narrower mutation
+   and execution budget;
+4. generate stable non-secret business keys before any retryable operation;
+5. re-read current state and save pre-edit snapshots;
+6. repair the smallest responsible layer;
+7. rerun only the missing acceptance claim, never a successful destructive claim for
+   cosmetic extra evidence;
+8. reconcile unknown outcomes before another submit, launch, write, or run;
+9. let deterministic host code validate remediation reports and promote the original
+   phase only after all required checks pass;
+10. archive the pre-remediation report and disclose the additional cost and mutations.
+
+Credential exposure is monotonic for the exposed credential. Deleting a file, masking
+a screenshot, or redacting a transcript copy does not make a completed
+`no_secret_exposure` check true. Preserve the incident, revoke or rotate the credential
+under a separately approved remediation, prove the old value is rejected and the new
+value is distributed only through approved boundaries, invalidate stale exports or
+reports, and then compute a new attributable promotion result.
+
+Never edit the old report in place to pretend the first attempt passed. Promotion is a
+new, attributable result built on preserved evidence.
+
+Separate approval recording, checkpoint eligibility and execution receipts. A controller
+must bind one approval to the exact phase, payload and reconciled before-state; recording
+approval before discovering an ineligible checkpoint does not establish a completed action
+or permission to replay another phase. Repeated manual archive/resume recipes belong in a
+source-owned state machine with adversarial tests, not ever-longer skill instructions.
+
+## 6. Route each lesson to the correct reusable layer
+
+Do not turn every field incident into more top-level skill prose.
+
+| Signal | Durable home |
+|---|---|
+| Stable platform/API semantic | Source-owned tool description or domain reference |
+| Repeated deterministic construction error | Builder, schema, validator, lint, or script |
+| General operational decision | Domain skill core or progressive reference |
+| Evaluation/field-gate integrity defect | Meta-skill standard and controller test |
+| Project-specific ID, title, payload, or workaround | Versioned field contract/evidence only |
+| One unexplained observation | Preserve as provisional evidence; do not generalize yet |
+
+Prefer a source-owned correction over duplication. Link to it from the skill when the
+agent must change a decision, but do not copy volatile schemas into several places.
+
+Separate repaired artifact proof from causal skill improvement. A successful typed-null
+workaround after simultaneous script/mapping/instruction edits demonstrates that bounded
+artifact outcome, not which skill edit caused it. Freeze sanitized incident cases in a
+separate development corpus; preserve formal holdouts and historical failures. A changed
+reference changes the skill package and needs fresh fingerprint-bound evaluation before
+new release or superiority claims. Do not relabel old field passes as evidence for new bytes.
+
+## 7. Avoid case-specific overfitting
+
+Before changing a reusable skill, ask:
+
+- Is the observation supported by a live schema, source, repeated trajectory, or a
+  controlled counterexample?
+- Does the proposed instruction change a recurring decision rather than narrate one
+  project's history?
+- Is this actually a skill issue, or can a tool/schema/controller enforce it better?
+- Can the rule be stated without workspace IDs, project titles, one-off evidence keys,
+  or exact remediation filenames?
+- Which baseline success could the new rule break?
+- What targeted regression case will prove the rule without making the full skill more
+  specific to this incident?
+
+A useful field lesson should become a small invariant, validator, or source-owned
+reference. The detailed incident remains in evidence, not in the always-loaded skill.
+
+Express the candidate rule as trigger → decision → limit. Add a structurally similar case
+in a different domain and a counterexample where it must not apply, alongside a preserved
+baseline success. If transfer requires the original field names or workaround, keep it
+local or provisional. These are development cases, not transfer-performance evidence;
+measured generalization still requires fresh held-out observations. Reject a new paragraph
+when an existing rule already covers the decision; repair routing or enforcement instead.
+
+## 8. Minimum field release record
+
+Record:
+
+- frozen contract/version and required check IDs;
+- complete skill-package fingerprint plus model/tool settings;
+- target and stable IDs, with secrets removed;
+- model/client/tool versions when they affect execution;
+- permitted gaps and actual gaps;
+- phase-local verdicts and the separately computed aggregate/project verdict;
+- exact mutation and full causal execution counts;
+- direct proof per required check;
+- unknown outcomes and reconciliation evidence;
+- temporary-resource cleanup;
+- credential exposures, revocation/rotation evidence, and stale-artifact invalidation;
+- original and remediated reports when remediation occurred;
+- source-owned documentation or code changes derived from the field evidence;
+- residual limits and claims deliberately not made.

procesio_cli/skills/agent-skill-engineer/references/optimization-protocol.md

@@ -0,0 +1,233 @@
+# Skill optimization protocol
+
+Use this reference when a skill already exists or when the initial design is strong enough to enter formal iterative optimization.
+
+## Contents
+
+1. Optimization contract
+2. Data and split discipline
+3. Outcome vector
+4. Trajectory collection
+5. Contrastive diagnosis
+6. Candidate generation
+7. Bounded textual learning rate
+8. Validation promotion
+9. Rejected-edit memory and plateau handling
+10. Statistical interpretation
+11. No-ground-truth evaluation
+12. Transfer and field proof
+13. Reproducible controller
+
+## 1. Optimization contract
+
+Treat the skill package as external trainable state for a frozen target agent. Keep the target model, tool surface, system instructions, generation settings, and evaluator fixed during one experiment unless the experiment explicitly studies transfer.
+
+Freeze before optimization:
+
+- immutable baseline package and fingerprint;
+- target task distribution and exclusions;
+- train, validation, test, and field ownership;
+- primary metric and direction;
+- hard correctness, safety, permission, privacy, and collision constraints;
+- secondary token, latency, tool-call, and maintenance costs;
+- minimum meaningful effect;
+- edit budget and allowed paths;
+- maximum rounds, plateau limit, and infrastructure retry policy;
+- fixed behavioral rubrics and evaluator version.
+
+Changing one of these creates a new experiment. Do not reinterpret an old run under a new contract.
+
+## 2. Data and split discipline
+
+Use four logically distinct evidence sets:
+
+| Set | Purpose | Permitted use |
+|---|---|---|
+| Train | discover failure modes and generate edit hypotheses | may be inspected repeatedly |
+| Validation | decide whether a candidate replaces the current best | scores may be used for promotion; case content should not drive hand-written patching |
+| Test | estimate final generalization after candidate selection | inspect once per final candidate; never optimize on it |
+| Field | verify real tools, artifacts, side effects, and user outcomes | controlled execution with approval and cleanup |
+
+Split by causal family rather than random text alone. Paraphrases, near-duplicate incidents, shared source documents, and variants of the same underlying task belong in one split. Keep a split manifest and fingerprint.
+
+Do not place rubrics, expected answers, evaluator rationales, or validation/test outcomes in the skill package. Do not regenerate the split until a new experiment version is declared.
+
+## 3. Outcome vector
+
+Use lexicographic evaluation rather than a single weighted score:
+
+1. **Eligibility:** structural validity, security, required permissions, hard correctness, no forbidden routing collisions, and mandatory field invariants.
+2. **Primary effect:** task success, direct user preference, or another registered outcome.
+3. **Non-regression:** preserve baseline successes and critical subgroups.
+4. **Efficiency:** token use, latency, tool calls, retries, and context loaded.
+5. **Maintainability:** package size, duplicate rules, source freshness, and extension cost.
+
+A candidate failing eligibility is rejected even if its average task score is high. Weights may summarize trade-offs among already eligible candidates; they must not compensate for a safety or correctness violation.
+
+For each paired item, classify the transition:
+
+| Baseline | Candidate | Classification |
+|---|---|---|
+| fail | pass | repair |
+| pass | fail | regression |
+| pass | pass | preserved success |
+| fail | fail | unresolved failure |
+
+Publish repair and regression counts, not only the net average.
+
+## 4. Trajectory collection
+
+For every run preserve a sanitized record of:
+
+- case and split identifiers;
+- package, rubric, evaluator, model, harness, and settings fingerprints;
+- selected skill and resources loaded;
+- response, tool calls, errors, retries, and direct observations;
+- criterion booleans and evidence spans;
+- token, latency, and cost measures when available;
+- infrastructure failures kept separate from task failures.
+
+Use fresh contexts. Interleave baseline and candidate runs through opaque labels so provider drift and load do not align with one variant. Checkpoint each completed observation.
+
+Collect successful trajectories as actively as failures. An optimizer that sees only failure examples tends to add rules that repair narrow cases while damaging concise successful behavior.
+
+## 5. Contrastive diagnosis
+
+Before editing, compare at least these groups:
+
+- baseline success versus baseline failure;
+- repaired failure versus unresolved failure;
+- preserved success versus candidate regression;
+- low-cost success versus high-cost success;
+- near-success versus hard failure.
+
+Extract the smallest repeated difference that could causally alter a decision. State one intervention hypothesis:
+
+```text
+When <observable cue>, the agent currently <failure behavior> because <supported mechanism>.
+Changing <specific instruction/resource/control> should produce <observable state> without breaking <named baseline successes>.
+```
+
+Reject diagnoses that merely restate the score, rely on one anecdote, or propose adding generic caution everywhere. Attribute tool, environment, observation, or evaluator defects to their own layer.
+
+## 6. Candidate generation
+
+For the first design or a major boundary failure, compare structurally different candidates. After a coherent base exists, generate local alternatives around one causal hypothesis.
+
+Candidate operations are:
+
+- **add:** introduce a missing discriminator, invariant, handoff, or proof requirement;
+- **delete:** remove misleading, redundant, over-broad, or costly content;
+- **replace:** substitute a more precise decision rule, example, resource route, or executable control;
+- **move:** shift conditional detail from the always-loaded body into a direct reference or script;
+- **split/merge:** redesign portfolio ownership only when routing evidence requires it.
+
+Prefer deletion, replacement, and structural enforcement over accumulation. Generate at least one minimal candidate. More candidates are useful only when the validation process accounts for the larger search.
+
+## 7. Bounded textual learning rate
+
+Every candidate proposal must declare:
+
+- parent fingerprint;
+- hypothesis and affected cases;
+- files permitted to change;
+- added, deleted, and total changed-line limits;
+- forbidden paths, especially formal evaluation data;
+- expected repair and protected successes;
+- rollback condition.
+
+Use a small budget after each round. If the required edit exceeds the budget, stop and classify the need as a redesign rather than silently expanding the patch.
+
+Never let the optimizer modify validation/test cases, rubrics, thresholds, evaluator code, or the result parser in the same experiment. Harness optimization is a separate experiment with its own immutable counterfactual.
+
+## 8. Validation promotion
+
+A candidate may replace the current best only when:
+
+1. package and security checks pass;
+2. report fingerprints match the candidate, parent, split, rubric, and evaluator;
+3. every hard constraint passes;
+4. primary validation performance strictly exceeds the registered threshold relative to the current best;
+5. protected subgroups and baseline-success retention meet their floors;
+6. secondary costs remain within registered tolerances;
+7. the run contains enough valid paired observations.
+
+Compare against the **current best**, not merely the original baseline. Copy the accepted package into a new immutable best snapshot. Never edit the best snapshot in place.
+
+A tie is not an improvement unless a pre-registered secondary objective resolves it. Do not lower the threshold after seeing a result.
+
+## 9. Rejected-edit memory and plateau handling
+
+Record every rejected candidate with:
+
+- parent and candidate fingerprints;
+- hypothesis and exact patch summary;
+- validation metrics and hard-constraint failures;
+- repaired and regressed case families;
+- rejection reason;
+- evidence that would justify revisiting the idea.
+
+Before generating a proposal, search this ledger for equivalent failed interventions. Reusing a rejected idea requires new evidence, not new wording.
+
+Stop when any condition holds:
+
+- plateau limit reached;
+- edit or model-call budget exhausted;
+- A/A evaluator noise exceeds the detectable effect;
+- validation gains come only from one overrepresented family;
+- complexity grows faster than task success;
+- remaining failures belong to the tool, data, environment, observation, or evaluator;
+- the final candidate cannot meet a hard constraint.
+
+A later meta-update may consolidate repeatedly accepted lessons into a shorter decision model. Validate the consolidation exactly like any other candidate.
+
+## 10. Statistical interpretation
+
+Use paired analysis because baseline and candidate see the same cases. Report sample size, missing pairs, repairs, regressions, and the paired effect. For binary task success, include a confidence interval for the paired difference and use a matched-pair test such as McNemar's test when inferential claims matter.
+
+Do not use significance as the only promotion rule. A tiny but statistically detectable gain may be operationally irrelevant; a large uncertain gain may need more repetitions. Pre-register both a minimum effect and an uncertainty rule appropriate to consequence.
+
+When searching many candidates, validation becomes progressively selected. Preserve an untouched final test and disclose candidate count and stopping behavior. Analyze critical subgroups separately rather than trusting a global mean to expose a rare safety regression.
+
+## 11. No-ground-truth evaluation
+
+When exact answers are unavailable:
+
+1. define observable behavioral attributes independently of candidate outputs;
+2. compare baseline and candidate responses under opaque labels;
+3. use multiple independent judgments or a calibrated evaluator where consequence justifies it;
+4. extract criterion-level evidence rather than one holistic score;
+5. gate promotion on held-out validation;
+6. send discordant, ambiguous, and high-consequence pairs to accountable human review.
+
+Self-preference by the same model that produced the answer is weak evidence. Pairwise preference is useful only when the preference question, evidence packet, and conflict handling are frozen.
+
+## 12. Transfer and field proof
+
+After final selection, test the untouched set and at least one adjacent model, client, or harness when portability is claimed. A skill that wins only on one optimizer-model combination may have learned harness-specific cues.
+
+Operational skills also require a controlled field task. Record before-state, exact target, approval, execution ID, direct outcome, cleanup, and restored state. Synthetic grades cannot substitute for the actual artifact or external system.
+
+## 13. Reproducible controller
+
+Use the skill's `optimize_skill.py` helper to enforce local experiment state:
+
+```bash
+python skills/agent-skill-engineer/scripts/optimize_skill.py init \
+  --skill-root skills/example \
+  --workspace .skill-optimization/example \
+  --objective objective.json \
+  --baseline-report baseline-validation.json
+
+python skills/agent-skill-engineer/scripts/optimize_skill.py stage \
+  --workspace .skill-optimization/example \
+  --candidate-root work/candidate \
+  --hypothesis "Replace the ambiguous retry rule with unknown-outcome reconciliation"
+
+python skills/agent-skill-engineer/scripts/optimize_skill.py decide \
+  --workspace .skill-optimization/example \
+  --candidate-id c0001 \
+  --report c0001-validation.json
+```
+
+The controller does not invent edits or score model responses. It checks snapshots, path allowlists, bounded diffs, report fingerprints and held-out promotion, and records rejected candidates. Final test separation needs the external one-shot custody in `references/optimizer-contract.md`: the helper permits final-report resubmission and overwrites, so it does not enforce an untouched single final test.