@hobin/developer 0.1.8 → 0.1.9

package/skills/verify/references/verifier-selection-and-pass-but-wrong.md

@@ -1,227 +1,187 @@
 # Verifier Selection And Pass-But-Wrong Risk
 
-Use this reference when a completion claim depends on several kinds of evidence,
-a passing check may not exercise the claim, or structural correctness matters in
-addition to visible behavior.
+Use this reference when a completion claim needs more than one evidence kind, a
+green check may be irrelevant, or visible behavior can pass while an important
+observer or structural guarantee degrades.
 
-## Contents
+## Judgment Spine
 
-- Claim Before Check
-- Evidence Map
-- Verifier Ladder
-- Execution Versus Relevance
-- Pass-But-Wrong Search
-- Test Design And Cost
-- Structural Degradation
-- Feedback Classification
-- Residual Risk And Stop Checks
-- Complete Evidence Example
+```text
+claim
+-> declared observer and boundary
+-> cheapest check that could falsify it
+-> actual execution and provenance
+-> plausible wrong implementation
+-> distinguishing evidence
+-> strongest supported claim + residual
+```
 
-## Claim Before Check
+A command result is an observation. Verification decides what claim that
+observation supports.
 
-State the claim before selecting a verifier. "Tests pass" is an observation;
-"the conversion rejects every impossible schedule variant" is a claim.
+## Split The Claim
 
-Split broad completion statements into independently checkable claims:
+Separate independently falsifiable claims:
 
-- product behavior;
+- user or caller behavior;
 - forbidden and boundary cases;
+- data multiplicity and preservation;
 - type or API contract;
-- transition and history behavior;
-- compatibility with the requested source or version;
-- representation or abstraction boundary;
-- operational property such as performance, ordering, or idempotency.
+- state transition and history;
+- representation or ownership boundary;
+- source/version compatibility;
+- operational properties such as order, effects, time, space, and idempotency.
 
-If a claim cannot be stated, return to specification or modeling rather than
-choosing a convenient command.
+If the intended claim cannot be stated, return to `specify` or `model`. Do not
+choose a convenient command first.
 
 ## Evidence Map
 
-For each claim record:
-
 ```text
-Claim: the narrow statement under review
-Verifier: command, inspection, fixture, or human acceptance
-Observation: what actually happened
-Relevance: why the observation bears on the claim
-Counterexample coverage: meaningful wrong shapes exercised or omitted
-Residual: uncertainty that remains
+Claim:
+Observer and admitted boundary:
+Verifier:
+Observation:
+Why it is relevant:
+Counterexample coverage:
+Source/version exercised:
+Residual:
 ```
 
-Evidence may be direct, supporting, weak, contradicted, unavailable, or
-human-owned. Do not merge unavailable evidence with passing evidence.
-
-## Verifier Ladder
-
-Prefer the cheapest verifier that can actually falsify the claim:
-
-| Verifier | Best fit |
-| --- | --- |
-| static inspection or focused diff | locality, wiring, ownership, stale code, representation boundary |
-| typecheck, lint, build, manifest check | syntax, imports, compile-time contracts, packaging |
-| focused example test | one concrete behavior or regression |
-| table or parameterized test | finite meaningful cases and boundaries |
-| property or metamorphic check | many examples share a stable rule |
-| integration or UI check | composition, rendering, persistence, external boundaries |
-| state-transition or concurrency fixture | history, ordering, retry, stale events, idempotency |
-| performance or resource measurement | runtime shape is part of the claim |
-| model, solver, or exhaustive check | small critical rule space justifies exhaustive reasoning |
-| human acceptance | the remaining question is policy, meaning, or visual judgment |
-
-Use a stronger verifier when a weaker one can pass under a plausible wrong
-implementation. Use a weaker verifier when it is the only practical evidence,
-but narrow the claim and record the residual.
-
-## Execution Versus Relevance
-
-A verifier execution proves only that the verifier ran and produced an
-observation. Check relevance separately:
-
-- Did the fixture reach the changed branch?
-- Did assertions inspect the product meaning or only absence of an exception?
-- Did the build use the source and version requested by the user?
-- Did a UI snapshot expose interaction and state changes or only initial pixels?
-- Did a broad suite include the new boundary case?
-- Did the manual inspection examine the actual generated artifact?
-
-A green but irrelevant verifier creates a `verifier-gap`; it does not support
-the broader claim.
-
-## Pass-But-Wrong Search
-
-For every consequential claim, imagine at least one implementation that passes
-current checks while violating the intended meaning. Look for:
-
-- missing branches whose fixtures never occur;
-- fallback/default behavior that silently selects product policy;
-- assertions that accept both the intended and an unintended value;
-- mocks that bypass the real boundary;
-- tests coupled to the implementation rather than the contract;
-- source/version mismatch;
-- current-state checks that miss invalid transitions;
-- correct result with wrong ordering, mutation, sharing, or complexity;
-- public API compatibility tested without real consumers.
-
-When a plausible shape remains cheap to test, add the smallest distinguishing
-fixture. Otherwise record it as residual risk instead of broadening the claim.
-
-## Test Design And Cost
+Classify evidence as direct, supporting, weak, contradicted, unavailable, or
+human-owned. Never merge unavailable evidence into a passing row.
 
-Tests are design evidence only when their boundary and maintenance cost remain
-useful. Prefer tests that expose public responsibility and product meaning over
-tests that echo private method structure.
+## Choose By Falsification Power
 
-Choose the unit by the claim:
-
-- focused examples for stable responsibility and boundary cases;
-- role or contract tests when several implementations must substitute;
-- integration tests for collaborations, creation, persistence, and wiring;
-- properties when many examples share a generatable rule;
-- stateful fixtures when history, retry, order, or identity matters.
-
-Do not require one test layer per class. Loose coupling may make a smaller unit
-economical to test, while simple forwarding or framework glue may be better
-covered by a larger behavior check. Fakes should remove irrelevant context while
-preserving the collaborator role; mocks that restate call internals create an
-echo chamber.
-
-Delete or reorganize tests only when the remaining evidence still supports the
-same claims. Redundancy is sometimes cheap insurance and sometimes obsolete
-context that prevents a design from moving. State which claim each retained test
-protects.
-
-## Structural Degradation
+| Claim shape | Cheapest relevant verifier | Typical gap |
+| --- | --- | --- |
+| wiring, ownership, stale code, representation leak | focused inspection or diff | runtime external behavior |
+| syntax, imports, compile-time API, package shape | typecheck, lint, build, packed artifact | dynamic input and real consumer |
+| one concrete behavior | focused example | unrepresented variants |
+| finite partitions and boundaries | table/parameterized check | invalid partition or omitted domain |
+| generatable stable rule | property/metamorphic check | generator excludes the wrong case |
+| collaboration, persistence, UI, adapter | integration or interaction check | environment variance |
+| retry, order, stale, concurrency | transition/trace fixture | production scheduling and scale |
+| time, space, allocation, distribution | measurement with stated model | different size/case/cost model |
+| bounded exhaustive relation | model/solver/proof evidence | assumptions, scope, encoding, toolchain |
+| product or visual policy | human acceptance | mechanical correctness only |
+
+Use a stronger verifier when a weaker one admits a plausible wrong shape. Use a
+weaker verifier when it is the only economical evidence, but narrow the claim.
+
+## Execution Is Not Relevance
+
+Ask:
+
+- Did the fixture reach the changed branch and meaningful case?
+- Did assertions observe product meaning or only absence of an exception?
+- Did the command use the requested source and version?
+- Did a UI check exercise interaction and state change or only initial pixels?
+- Did a migration check pass through the real persistence adapter?
+- Did a structural claim receive structural evidence rather than only equal
+  returned values?
+
+A green irrelevant verifier is a `verifier-gap`.
+
+## Counterexample Families
+
+Use the claim's observer to select a wrong shape. The table integrates the source
+calibrations by failure mechanism rather than by book.
+
+| Claim | Green proxy that can mislead | Plausible wrong implementation | Distinguishing check |
+| --- | --- | --- | --- |
+| transformation preserves domain values | expected output looks ordered | output drops duplicates or invents members | membership, no-additions, and multiplicity properties |
+| optional states preserve meaning | happy configured case passes | missing, null, empty, and default collapse | same absence shape with different discriminator/meaning |
+| property holds generally | sampled generated cases pass | generator omits one fixed boundary | inspect generator domain and add the distinguishing example |
+| implication/test is strong | assertion excludes every bad case | assertion also rejects every correct implementation or passes vacuously | witness a valid instance and one violating instance |
+| proof/model result establishes behavior | tool reports success | specification, arithmetic, scope, or assumptions omit product meaning | audit obligations, bounds, satisfiable witness, and observers |
+| relational/storage rule matches model | projected rows look right | endpoint, `NULL`, bag duplicates, or mutation path differs | runtime-specific endpoint/multiplicity/event checks |
+| numeric result is accurate | value is near a known answer | residual, location, bracket, or downstream error is wrong | name tolerance meaning and test each claimed relation |
+| arithmetic refactor preserves behavior | ordinary magnitudes pass | reassociation changes inexact result or effect order | adversarial magnitudes, reverse order, effects, and scale |
+| benchmark supports growth claim | sampled run is faster | input distribution or constant dominates; adversarial shape loses | state size/case/cost model and test relevant range |
+| database schema behavior is preserved | row content matches | labels or function-valued predicates changed | inspect schema and execute predicates/integrity checks |
+| temporal implementation is correct | each enum state is reachable | stale or duplicate event commits twice | ordered trace with stale, retry, duplicate, and concurrency |
+| abstraction is safe | public tests pass | callers still depend on fields, load order, or hidden default | leak search, fake variant, and real caller rewrite |
+| package/source compatibility holds | workspace tests pass | packed artifact or installed version lacks changed files | inspect tarball/fresh install and bind provenance |
+
+A verifier for one row does not support another. In particular, location,
+residual, representation, and downstream tolerances are not interchangeable.
+
+## Formal And Search Result Boundaries
+
+Keep sampled property evidence distinct from total specification and proof. Keep
+bounded model results relative to scope/configuration and require a satisfiable
+witness. Preserve exposed statuses:
 
-Behavior may pass while the change weakens the code's ability to preserve the
-same guarantee. Inspect for:
+```text
+satisfiable | unsatisfiable | unknown | timeout | error
+unbounded | feasible | proven-optimal
+```
 
-- duplicated invariant or absence/default policy;
-- representation details leaking through a caller contract;
-- generic abstraction without stable examples or a nameable purpose;
-- hidden state, history, ordering, or ownership;
-- vocabulary drifting away from the accepted model;
-- a local fix that makes the next accepted requirement harder or unsafe;
-- tests passing only because they mirror the new implementation.
+For logic/search work, distinguish no answer after complete exhaustion from
+bounded unknown, timeout, error, or divergence. A later answer may exist behind
+an infinite earlier branch.
 
-Structural degradation is evidence for structural observation or abstraction
-review. It does not automatically require refactoring now.
+## Test Design As Structural Evidence
 
-## Feedback Classification
+Choose test units by public responsibility and failure-localization value, not
+one file per class. Focused role tests suit stable contracts; integration evidence
+suits creation, collaboration, persistence, and wiring. A fake should preserve a
+collaborator role while removing irrelevant context. A mock that repeats private
+calls creates an echo chamber.
 
-Use a small, stable taxonomy:
-
-- `pass`: the evidence supports the narrow claim;
-- `implementation-fail`: relevant evidence contradicts the accepted claim;
-- `model-or-spec-fail`: evidence shows the accepted meaning or case model is
-  wrong or incomplete;
-- `verifier-gap`: the check is missing or irrelevant to the claim;
-- `blocked-environment`: the required source or environment was not exercised;
-- `pass-but-wrong-risk`: current evidence passes while a plausible wrong shape
-  remains;
-- `structural-degradation`: behavior passes but structure weakens a relevant
-  boundary or guarantee.
-
-Classification describes the observation. The caller decides whether to
-repair, remodel, add evidence, ask a human, accept residual risk, or continue.
+Redundancy can be independent domain evidence or obsolete context. Delete it only
+when the remaining tests still support the same claim. Hard-to-write tests are a
+structural signal, not automatic proof that production must be generalized.
 
-## Residual Risk And Stop Checks
+## Feedback And Stop
 
-Verification is complete enough when:
+Use:
 
-- every stated claim is supported, narrowed, or explicitly unverified;
-- each important forbidden or boundary case has evidence or a residual;
-- actual source and version match the requested target;
-- plausible pass-but-wrong shapes were attempted or recorded;
-- structural claims have more than behavioral evidence when needed;
-- human-owned acceptance is distinguished from mechanical checks;
-- another verifier would cost more than the information it is expected to add.
+- `pass`;
+- `implementation-fail`;
+- `model-or-spec-fail`;
+- `verifier-gap`;
+- `blocked-environment`;
+- `pass-but-wrong-risk`;
+- `structural-degradation`.
 
-Do not claim total correctness from a finite evidence set. State the strongest
-claim justified now.
+Verification is complete enough when every claim is supported, narrowed, or
+explicitly unverified; important wrong shapes have evidence or residuals; source
+and version match; and another check would cost more than its expected
+information.
 
-## Complete Evidence Example
+## Complete Artifact
 
 ```text
-Claim: recurring schedules without an end date remain valid.
-Verifier: focused table test with one-time, recurring-open, recurring-bounded,
-and invalid-interval rows.
-Observation: all rows produce the expected domain result.
-Relevance: each semantic variant crosses the new conversion boundary.
-Pass-but-wrong search: a mapper could still ignore timezone normalization.
-Residual: timezone behavior remains unverified and is not included in the
-supported claim.
+Claim/evidence matrix:
+Requested-versus-exercised provenance:
+Counterexample family and attempted wrong implementation:
+Distinguishing checks and observations:
+Structural degradation, if any:
+Strongest supported claim:
+Unverified claims and residual risk:
 ```
 
-Now attempt a wrong implementation that always emits `recurring-open` whenever
-`endDate` is absent. It passes the open-recurring row but also misclassifies a
-one-time schedule whose UI omitted an optional display field. Add a one-time row
-with the same absence shape but a different recurrence discriminator. This
-fixture fails the plausible wrong rule while remaining at the public conversion
-boundary.
-
-Map the broader completion claim:
-
-| Claim | Verifier | What it can still miss |
-| --- | --- | --- |
-| semantic variants convert correctly | focused table test | persisted legacy shapes not in the table |
-| public type accepts intended callers | typecheck plus real caller build | runtime external input |
-| legacy saved schedules remain readable | migration fixture through persistence adapter | unknown production corruption |
-| UI shows the converted meaning | interaction test or manual run | timezone/environment variance |
-| package contains the changed source | packed-tarball inspection and fresh install | registry propagation delay |
-
-If only the first row is executed, report only the first claim. The correct
-result is a narrower supported statement, not an unqualified “done.”
-
 ## Source Trace
 
-- Hillel Wayne, *Logic for Programmers*, version 0.14.0, May 4, 2026:
-  strong and weak tests, partial specifications, structural properties,
-  property-based testing, contracts, and proof limitations.
-- Sandi Metz, Katrina Owen, and TJ Stankus, *99 Bottles of OOP*, Second
-  Edition, version 2.2.2, 2024: chapters 2 and 9 on cost-effective tests,
-  avoiding implementation echo, choosing units, unit versus integration tests,
-  context independence, fakes, role verification, and obsolete-test removal.
-- Matthias Felleisen, Robert Bruce Findler, Matthew Flatt, and Shriram
-  Krishnamurthi, *How to Design Programs, Second Edition*, MIT Press, 2018:
-  representative examples and tests as successive constraints in a design
-  recipe.
+- Hillel Wayne, *Logic for Programmers*, v0.14.0:
+  Chapters 3-6 and 9-11, pp. 23-155, for logical/runtime preservation, partial
+  specifications, generated properties, proof limits, bounded models, and solver
+  result states. Recorded beta defects are excluded.
+- Sandi Metz, Katrina Owen, and TJ Stankus, *99 Bottles of OOP*, Second Edition,
+  v2.2.2, Chapters 2 and 9, pp. 23-50 and 226-268, for cost-effective tests,
+  public responsibility, integration versus unit evidence, fakes, role checks,
+  and obsolete context.
+- Matthias Felleisen et al., *How to Design Programs*, living build 9.2.0.3:
+  Prologue,
+  Chapter 18,
+  Chapter 23,
+  Chapter 25,
+  Chapter 27,
+  Chapter 28,
+  Chapter 29,
+  Intermezzo 4, and
+  Intermezzo 5
+  for progressive property strength, multiplicity, schema predicates, numeric
+  tolerances, order-sensitive arithmetic, and bounded performance evidence.

package/SOURCES.md

@@ -1,104 +0,0 @@
-# Source-to-Capability Trace
-
-Developer is organized around questions the agent must answer, not around books.
-This maintainer document records how source material is decomposed across leaf
-skills and the implementation path. It is not a Pi runtime resource or a required
-workflow.
-
-## Ownership Rule
-
-A source idea belongs where its observable job is performed:
-
-- product meaning and change pressure go to `specify`;
-- cases, rules, contracts, and transitions go to `model`;
-- implementable boundaries and collaborations go to `sketch`;
-- visible structural movement goes to `signal`;
-- candidate stability goes to `abstraction-review`;
-- timing goes to `schedule`;
-- names go to `naming-judgment`;
-- claim-to-evidence judgment goes to `verify`;
-- already-justified mutation goes to the `implementation` route.
-
-No source implies a mandatory route order. Several leaves may use different
-parts of the same source because they answer different questions.
-
-## Runtime Reference Quality
-
-`SKILL.md` owns discovery, the core question, output, completion, and conditional
-routing. A reference loaded through that route must be usable without reopening
-the original source or reconstructing unstated steps.
-
-Match detail to the reference's job:
-
-- a concept or design reference needs the core insight, input, derivation rule,
-  completed example, counterexample or failure diagnosis, output, and boundary;
-- a recipe card needs a trigger, input artifact, construction rule, output
-  artifact, observable stop, and repair direction;
-- a diagnostic table needs a symptom, evidence path, repair artifact, and
-  re-check;
-- a worked-example file needs enough context to show why the decision follows,
-  not only a polished final answer;
-- a mutation protocol needs an entry contract, safe intermediate states,
-  evidence rhythm, reroute conditions, and a stable landing.
-
-Split a reference when independent questions would otherwise make it expensive
-to load, but keep the central insight and one complete example in the primary
-reference. Splitting by chapter title or merely to shorten a file is not a
-capability boundary.
-
-## Capability Matrix
-
-| Source | Capability extracted | Primary owner | Supporting owner or execution path |
-| --- | --- | --- | --- |
-| *99 Bottles of OOP*, ch. 1-2 | simple concrete baseline, Shameless Green, cost-effective and intention-revealing tests | `verify` | `signal`, `schedule` |
-| *99 Bottles of OOP*, ch. 3-4 | real change pressure, point of attack, closest pair, smallest difference, horizontal movement | `signal` | `implementation` behavior-preserving protocol |
-| *99 Bottles of OOP*, ch. 4-5 | stable landings, responsibility-derived names, argument and data movement | `abstraction-review`, `naming-judgment` | `implementation` behavior-preserving protocol |
-| *99 Bottles of OOP*, ch. 5-8 | responsibility separation, messages, type transitions, polymorphism, dependency direction, object creation at the edge, factory tradeoffs | `sketch`, `abstraction-review` | `model`, `schedule` |
-| *99 Bottles of OOP*, ch. 9 | unit boundaries, context independence, role verification, obsolete-test removal | `verify` | `sketch` |
-| *How to Design Programs* | information analysis, data definitions, examples, templates, structural and generative recursion, accumulators, iterative refinement | `sketch` | `model`, `verify`, `signal`, `abstraction-review` |
-| *Structure and Interpretation of Computer Programs*, ch. 1-2 | procedural abstraction, process shape, higher-order composition, data abstraction, closure, conventional interfaces, multiple representations | `sketch`, `abstraction-review` | `model` |
-| *Structure and Interpretation of Computer Programs*, ch. 3 | state, identity, history, concurrency, streams, constraint propagation | `model`, `sketch` | `abstraction-review`, `verify` |
-| *Structure and Interpretation of Computer Programs*, ch. 4-5 | language/evaluator boundaries and explicit execution machinery | `sketch`, `abstraction-review` | imported only when a real DSL, interpreter, compiler, or runtime boundary exists |
-| *Logic for Programmers*, ch. 2-3 | predicates, sets, quantifiers, logical refactoring, runtime-semantic caveats | `model` | `verify` |
-| *Logic for Programmers*, ch. 4-6 | partial specifications, properties, contracts, replacement, proof limits | `model`, `verify` | `abstraction-review` |
-| *Logic for Programmers*, ch. 7-12 | relational constraints, decision tables, domain/time/system models, solvers, logic programming | `model` | `verify` |
-| *Elements of Clojure*: Names | narrow and consistent sense, honest effects and scope crossing | `naming-judgment` | `model` |
-| *Elements of Clojure*: Idioms | language-specific conventions and explicit operational semantics | project conventions or `implementation` | generalized only when the semantic lesson survives the language |
-| *Elements of Clojure*: Indirection | abstraction cost, module environment/model/interface/assumptions, principled versus adaptable systems | `sketch`, `abstraction-review` | `schedule` |
-| *Elements of Clojure*: Composition | units of computation and pull-transform-push process boundaries | `sketch` | `naming-judgment`, `verify` |
-| *Tidy First?* | separate behavior from structure, small tidyings, optionality, reversibility, first/after/later/never | `schedule` | `implementation` behavior-preserving protocol, `signal` |
-
-## Intentionally Not Imported As Universal Rules
-
-Source coverage does not mean copying every example or language idiom into every
-task. The following remain contextual:
-
-- Ruby class and factory shapes from *99 Bottles* are evidence about
-  responsibility and variation, not a destination for every design.
-- Scheme evaluators, register machines, and compiler details from SICP are used
-  only for tasks that actually expose those runtime or language boundaries.
-- Clojure-specific arities, option maps, atoms, macros, and interop idioms stay
-  subordinate to the target project's language and conventions.
-- Logic solvers, formal proof, TLA+, Prolog, and answer-set programming are
-  selected only when the modeled uncertainty and risk justify their cost.
-- HtDP recipes are adapted to existing repositories; they do not require adding
-  pedagogical comments, signatures, or templates to production code.
-
-These exclusions prevent a respected source from becoming a framework-shaped
-answer imposed on unrelated product work.
-
-## Reassembly Check
-
-When adding or revising a source-derived reference, record:
-
-```text
-Source location: book, edition/version, chapter or section
-Capability: the question or action the idea improves
-Owner: one leaf or the implementation path
-Boundary: what the owner must not absorb
-Observable effect: the artifact, decision, or safer execution behavior expected
-```
-
-Create a new leaf only when a recurring question has a distinct input, output,
-completion condition, and boundary that no existing owner can express. A new
-source, technique, or vocabulary is not by itself evidence for a new leaf.