pi-feature-dev 1.14.0 → 1.15.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +17 -0
- package/package.json +1 -1
- package/skills/deslop/SKILL.md +126 -0
- package/skills/deslop/agents/openai.yaml +7 -0
- package/skills/deslop/references/code-smells.md +80 -0
- package/skills/deslop/references/evidence-and-reachability.md +91 -0
- package/skills/deslop/references/scientific-code.md +70 -0
- package/skills/deslop/references/test-smells.md +111 -0
- package/skills/deslop/references/verification-and-trust.md +143 -0
package/README.md
CHANGED
|
@@ -14,6 +14,11 @@ feature work:
|
|
|
14
14
|
|
|
15
15
|
It does not require specific task-tracking, question, delegation, or review tools. If the current environment provides equivalent capabilities, use them; otherwise follow the same workflow directly in chat and with normal code tools.
|
|
16
16
|
|
|
17
|
+
The `deslop` skill audits or applies evidence-backed subtractive cleanup for
|
|
18
|
+
accumulated test bloat, circular verification, and defensive or fallback
|
|
19
|
+
machinery. Audit mode is read-only. Only an explicit `apply` request authorizes
|
|
20
|
+
edits.
|
|
21
|
+
|
|
17
22
|
The `plan-exec` skill executes implementation plan files task by task with
|
|
18
23
|
isolated workers, Git task commits, internal reviews, finalize, and a portable
|
|
19
24
|
run summary. It is portable across host agents that provide fresh-context
|
|
@@ -95,6 +100,7 @@ from:
|
|
|
95
100
|
- [anthropics/claude-code feature-dev plugin](https://github.com/anthropics/claude-code/tree/main/plugins/feature-dev)
|
|
96
101
|
- [mattpocock/skills batch-grill-me and domain-modeling skills](https://github.com/mattpocock/skills)
|
|
97
102
|
- [mattpocock/skills prototype](https://github.com/mattpocock/skills/tree/main/skills/engineering/prototype)
|
|
103
|
+
- [MrZoyo/deslop-GPT](https://github.com/MrZoyo/deslop-GPT/tree/main/skills/deslop)
|
|
98
104
|
- [danyuchn/asd-ste100-skill](https://github.com/danyuchn/asd-ste100-skill)
|
|
99
105
|
- [AminBlg/SimpleEnglish](https://github.com/AminBlg/SimpleEnglish/tree/main/skills/simple-english)
|
|
100
106
|
- [ayghri/i-have-adhd](https://github.com/ayghri/i-have-adhd)
|
|
@@ -115,6 +121,12 @@ Natural language also works when Pi's skill matcher triggers:
|
|
|
115
121
|
Use feature-dev to implement API rate limiting.
|
|
116
122
|
```
|
|
117
123
|
|
|
124
|
+
Audit accumulated complexity without editing files:
|
|
125
|
+
|
|
126
|
+
```text
|
|
127
|
+
/skill:deslop audit the current branch for test, verification, and fallback bloat.
|
|
128
|
+
```
|
|
129
|
+
|
|
118
130
|
Carry an incomplete creative brief into the work itself:
|
|
119
131
|
|
|
120
132
|
```text
|
|
@@ -212,6 +224,11 @@ pi-feature-dev/
|
|
|
212
224
|
├── creator-vibe/
|
|
213
225
|
│ ├── agents/openai.yaml
|
|
214
226
|
│ └── SKILL.md
|
|
227
|
+
├── deslop/
|
|
228
|
+
│ ├── agents/openai.yaml
|
|
229
|
+
│ ├── references/
|
|
230
|
+
│ ├── LICENSE.txt
|
|
231
|
+
│ └── SKILL.md
|
|
215
232
|
├── feature-dev/
|
|
216
233
|
│ ├── agents/openai.yaml
|
|
217
234
|
│ └── SKILL.md
|
package/package.json
CHANGED
|
@@ -0,0 +1,126 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: deslop
|
|
3
|
+
description: Audit or apply evidence-backed, test-first subtractive cleanup for accumulated agent-created test bloat, verification theater, and defensive or fallback bloat while preserving independent external behavior. Invoke explicitly for semantic simplification, not generic refactoring.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Deslop
|
|
7
|
+
|
|
8
|
+
Deslop removes complexity accumulated during repeated coding-agent implementation and correction cycles. It is a semantic cleanup policy, not a beautifier, dead-code sweeper, generic refactoring tool, or redesign assistant.
|
|
9
|
+
|
|
10
|
+
## Priority order
|
|
11
|
+
|
|
12
|
+
Work in this order. Do not let an easy dead-code deletion displace a higher-priority cluster.
|
|
13
|
+
|
|
14
|
+
1. **Test-suite bloat.** Treat tests as production code that can accumulate after every agent correction. Remove duplicate, self-referential, implementation-detail, and obsolete tests while retaining a minimum sufficient set of independent behavioral evidence.
|
|
15
|
+
2. **Verification theater.** Investigate checksums, receipts, manifests, validators, recomputation, and result envelopes whose producer and verifier share the same information and failure domain.
|
|
16
|
+
3. **Defensive and fallback bloat.** Investigate broad catches, catch-and-fallback paths, speculative compatibility branches, repeated validation, and recovery machinery that masks errors without a current contract.
|
|
17
|
+
|
|
18
|
+
Generic dead code, wrappers, abstractions, comments, and ordinary duplication are secondary. Touch them only when they belong to one of the three target clusters or have direct high-confidence evidence.
|
|
19
|
+
|
|
20
|
+
**Reduce test surface, not behavior surface.** Evidence that a test is redundant or accumulated test-suite bloat justifies deleting or consolidating that test; it is not independent evidence for changing the production behavior the test exercises. When test-suite bloat is the active target, keep distinct externally observable production semantics—including public success, rejection, error, edge-case, and supported compatibility behavior—outside the deletion target unless the production construct is separately justified for removal by another target cluster, or direct evidence from current requirements, real callers, specifications, or history establishes that the behavior is obsolete or incorrect. Do not reclassify tested behavior as defensive or validation bloat merely because deleting its test leaves nothing else requiring it; the test may be its clearest executable specification.
|
|
21
|
+
|
|
22
|
+
Adding tests is not the default response to cleanup. When production slop is deleted, tests whose only purpose is to protect that slop should normally be deleted in the same change.
|
|
23
|
+
|
|
24
|
+
## Test-first evidence pass
|
|
25
|
+
|
|
26
|
+
When tests are in scope, complete this pass before changing production behavior:
|
|
27
|
+
|
|
28
|
+
1. Inventory collected test nodes plus their fixtures, fakes, helper stacks, generated inputs, output targets, skips, and deselection rules.
|
|
29
|
+
2. Map each test to `current owner -> production branch -> observable result -> independent oracle -> failure domain`.
|
|
30
|
+
3. Group tests by failure domain. Consolidate inputs that reach the same branch and result; preserve separate rejection, safety, persistence, protocol, resource, and numerical semantics.
|
|
31
|
+
4. Choose the strongest surviving evidence root for each group: a public seam where possible, an independently specified low-level invariant where necessary, and one hermetic cross-layer root for each current delivery path.
|
|
32
|
+
5. Remove fixtures and test-only support with the tests they serve. Remove production code in that cluster only when separate caller, contract, history, and reachability evidence also justify it.
|
|
33
|
+
6. Re-collect and run the remaining suite. Record skips and deselections, and compare worktree state before and after tests that may generate files.
|
|
34
|
+
|
|
35
|
+
Do not use coverage, test count, or a test-created configuration as an owner. The output of this pass is a smaller evidence set with the same real failure domains, not a target number of tests.
|
|
36
|
+
|
|
37
|
+
### Test decision contrast
|
|
38
|
+
|
|
39
|
+
- **HIGH cleanup:** several tests reach the same public success branch; one asserts the exact externally visible result while the others only repeat construction, type, or non-empty checks. Keep the strongest behavioral root and remove the dominated tests plus support used only by them.
|
|
40
|
+
- **PRESERVE:** one test covers successful output and another covers externally visible rejection, safety, persistence, or supported compatibility behavior. They protect different results or failure domains; do not merge or delete one merely to reduce the count.
|
|
41
|
+
|
|
42
|
+
## Closed justification loops
|
|
43
|
+
|
|
44
|
+
Production code does not justify a test merely because the test exercises it. A test does not justify production code merely because the production code exists. Follow justification chains outward until they reach an independent evidence root.
|
|
45
|
+
|
|
46
|
+
A production/test pair is mutual-support slop only when the production construct has no independently meaningful externally observable purpose. A test of a distinct externally observable rejection, error, or supported compatibility behavior is not a closed justification loop merely because the test is its clearest executable specification. Internal checksum or receipt machinery and tests created only for that machinery can form such a loop, as can a speculative fallback and tests created only to exercise it.
|
|
47
|
+
|
|
48
|
+
If a fallback exists only because a test exercises it, and that test exists only because the fallback was added, neither member is independent evidence. The same closed justification loop can include checksum logic and checksum tests, receipts/manifests and validators, wrappers and wrapper-only tests, obsolete compatibility branches and their tests, or defensive validators and tests that only exercise them. Call this **mutual-support slop**.
|
|
49
|
+
|
|
50
|
+
Accept a dependency cluster only when its chain reaches a current user requirement, real external caller, public API contract, documented protocol or specification, security/trust boundary, persistence or corruption boundary, or scientific/numerical invariant. If the cluster only justifies itself, prefer deleting the whole cluster rather than preserving each member because another member depends on it.
|
|
51
|
+
|
|
52
|
+
A closed loop is a finding about evidence that is present, not evidence that is absent. Failing to find a caller, configuration, document, or history inside a small or isolated scope is missing evidence: it caps the finding at MEDIUM and does not make the cluster unreachable or HIGH. A compatibility branch with its own first-class positive test or current documentation explicitly naming it as supported behavior is a preservation root; removing that behavior requires positive evidence that the compatibility promise has ended. A branch reached only through a broad catch, with no test or document of its own, is not such a root.
|
|
53
|
+
|
|
54
|
+
Trace edges as well as nodes. Separately tested producers, readers, and consumers do not prove that a current production path connects them. Before removing a fixture, validator, registry entry, or wrapper that crosses layers, identify the current producer -> reader -> consumer path and preserve one hermetic integration root when that path carries real behavior.
|
|
55
|
+
|
|
56
|
+
## Modes and authorization
|
|
57
|
+
|
|
58
|
+
Interpret invocation from natural language; do not depend on a runtime-specific arguments variable.
|
|
59
|
+
|
|
60
|
+
- **Default or `audit`:** read-only. Report candidates, evidence, confidence, closed loops, and constructs to preserve. Do not intentionally modify repository-owned content; disable or redirect tool caches and generated outputs when practical.
|
|
61
|
+
- **`apply`:** modify files only within the established scope.
|
|
62
|
+
- **`tests`:** prioritize test signal and mutual-support slop. Without `apply`, remain read-only.
|
|
63
|
+
- **`deep`:** inspect repository-wide. Without `apply`, remain read-only; with `apply`, cleanup is allowed but redesign is not.
|
|
64
|
+
- **Explicit paths:** inspect and edit those paths plus the minimum callers, contracts, and tests needed to establish independence.
|
|
65
|
+
- **Current branch or no scope inside Git:** use the actual merge base and include staged, unstaged, and untracked work; never assume `main`.
|
|
66
|
+
|
|
67
|
+
Only `apply` authorizes edits. Do not fetch, reset, switch branches, stage, commit, push, or create backups unless explicitly requested.
|
|
68
|
+
|
|
69
|
+
## Establish evidence before editing
|
|
70
|
+
|
|
71
|
+
1. Read the active host's applicable project-instruction files, such as `AGENTS.md` or `CLAUDE.md`, plus repository conventions and the requested scope.
|
|
72
|
+
2. Inspect callers, tests, history, specifications, current configuration, registries, public readers, and documented verification commands.
|
|
73
|
+
3. Classify the evidence chain before trusting an existing test or fallback.
|
|
74
|
+
4. Put explicit current requirements first. A requirement or correction stated in the current task, or recorded in a current authoritative project document, overrides conflicting historical tests. Do not promote an inferred preference or the cleanup objective itself into a requirement.
|
|
75
|
+
5. A bug fix should normally replace incorrect behavior, not preserve it behind a fallback.
|
|
76
|
+
|
|
77
|
+
Prove reachability from a non-test producer such as an active config, external request, public CLI, persisted record, or hardware/runtime selection. A test-injected flag, synthetic future config, diagnostic command, or scripted dry-run does not by itself own a production branch.
|
|
78
|
+
|
|
79
|
+
Inside trusted code, use a **fail-visible bias**: allow unexpected failures to surface unless there is a concrete recovery, translation, cleanup, protocol, or compatibility contract. Broad `except Exception`, catch-and-fallback, catch-log-rethrow, speculative legacy fallbacks, and compatibility branches without independent evidence are high-priority investigation targets. Do not hide bugs in the name of robustness.
|
|
80
|
+
|
|
81
|
+
## Confidence and apply behavior
|
|
82
|
+
|
|
83
|
+
- **HIGH:** redundant, tautological, unreachable, self-justifying, or disconnected from a real contract after the evidence chain is resolved.
|
|
84
|
+
- **MEDIUM:** apparently unnecessary, but caller, history, compatibility, or boundary evidence is still missing.
|
|
85
|
+
- **LOW / PRESERVE BY DEFAULT:** security, authorization, concurrency, persistence, transactions, external protocols, supported compatibility, resource limits, and scientific invariants whose purpose may be outside the local file.
|
|
86
|
+
|
|
87
|
+
In apply mode:
|
|
88
|
+
|
|
89
|
+
- HIGH: delete or simplify once the evidence is resolved.
|
|
90
|
+
- MEDIUM: do not modify until the missing evidence question is resolved.
|
|
91
|
+
- LOW: preserve unless direct contrary evidence is established.
|
|
92
|
+
|
|
93
|
+
Apply authorization is permission to edit, not permission to resolve uncertainty in favor of deletion.
|
|
94
|
+
|
|
95
|
+
Read only the relevant references:
|
|
96
|
+
|
|
97
|
+
- [test-smells.md](references/test-smells.md) for accumulated test suites and test/production mutual-support clusters.
|
|
98
|
+
- [verification-and-trust.md](references/verification-and-trust.md) for checksum, receipt, manifest, provenance, and trust-boundary clusters.
|
|
99
|
+
- [code-smells.md](references/code-smells.md) for defensive, fallback, compatibility, wrapper, and abstraction candidates.
|
|
100
|
+
- [scientific-code.md](references/scientific-code.md) for numerical, simulation, ML, or engineering invariants.
|
|
101
|
+
- [evidence-and-reachability.md](references/evidence-and-reachability.md) when cleanup touches cross-layer fixtures, generated artifacts, active configuration, schema readers, registries/CLIs, or test hermeticity.
|
|
102
|
+
|
|
103
|
+
## Subtractive workflow
|
|
104
|
+
|
|
105
|
+
1. Complete the test-first evidence pass before production cleanup when tests are in scope.
|
|
106
|
+
2. Trace verification machinery as a cluster, not a function. Remove serialization, digest fields, envelopes, manifests, validators, recomputation, and tests together when no independent root remains.
|
|
107
|
+
3. Trace fallback branches to actual supported consumers and failure contracts. Prefer direct failure when the current contract says an operation should fail.
|
|
108
|
+
4. Delete tests that exist only to keep deleted production slop green. Add a replacement test only when deleting the old test would leave a real external behavior unprotected and an independent oracle exists.
|
|
109
|
+
5. Preserve real public, persistence, security, protocol, compatibility, resource, and scientific boundaries even when their code resembles a smell.
|
|
110
|
+
6. Treat declared authoritative inputs as required unless the protocol explicitly marks them optional. Missing and invalid authoritative artifacts should share the same visible failure semantics.
|
|
111
|
+
7. For a schema or identity change, enumerate every public reader, including CLIs, tools, visualizers, converters, and resume paths. Do not infer compatibility from a missing field or file.
|
|
112
|
+
8. Keep permanent tests hermetic: use repository-managed or test-created inputs, write to temporary outputs, and skip only when the test actually crosses the optional dependency boundary.
|
|
113
|
+
|
|
114
|
+
## Negative-change budget
|
|
115
|
+
|
|
116
|
+
Normally reduce structural surface area. New dependencies, abstractions, wrappers, compatibility layers, cryptographic/provenance machinery, and tests have a default budget of zero. New code is acceptable only when it preserves a real behavior while removing more accumulated slop. A small current-path integration root or a direct required-input check can be justified when cleanup exposes a real protection gap. If a cleanup adds substantial production or test lines, stop and reconsider.
|
|
117
|
+
|
|
118
|
+
In `deep apply`, exclude generated code, vendored dependencies, `third_party` trees, migration history, lockfiles, and externally generated snapshots or artifacts unless explicitly included or demonstrably repository-owned.
|
|
119
|
+
|
|
120
|
+
## Proportional verification
|
|
121
|
+
|
|
122
|
+
Run the narrowest existing checks after each meaningful semantic group and the repository's documented final checks once when feasible. Compare test collection before and after; zero surviving tests is a failure, and unexpected skips or deselections require explanation. When tests can generate files, compare the worktree before and after the suite so a green run cannot hide writes to tracked outputs. In read-only modes, use no-write options or temporary locations for caches and generated output when available, and report any incidental tool artifacts left behind. Verification should be independent of the change where possible. Do not create proof files, audit ledgers, checksum reports, or a new verification framework merely to validate a deletion. If a check cannot run without changing repository-owned content, do not run it in audit mode; state that plainly.
|
|
123
|
+
|
|
124
|
+
## Final report
|
|
125
|
+
|
|
126
|
+
Report the inspected scope; removed test, verification, and fallback clusters; independent evidence roots; preserved boundaries; tests removed or consolidated; before/after collection plus skips or deselections when available; checks actually run; approximate production/test size changes when useful; and uncertainty intentionally left untouched. Explicitly call out any closed justification loop that drove deletion.
|
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
interface:
|
|
2
|
+
display_name: "Deslop"
|
|
3
|
+
short_description: "Audit accumulated test, verification, and fallback bloat"
|
|
4
|
+
default_prompt: "Use $deslop to audit accumulated test bloat, verification theater, and defensive/fallback machinery before applying a subtractive cleanup."
|
|
5
|
+
|
|
6
|
+
policy:
|
|
7
|
+
allow_implicit_invocation: false
|
|
@@ -0,0 +1,80 @@
|
|
|
1
|
+
# Code Smells
|
|
2
|
+
|
|
3
|
+
Use this checklist for production code after test bloat and verification clusters have been inspected. A smell identifies where to investigate; caller, correction, and contract evidence decide whether to delete. Generic dead code and ordinary duplication are secondary unless they belong to a focused cluster.
|
|
4
|
+
|
|
5
|
+
| Smell | Signal | Evidence needed | Common false positives | Preferred simplification |
|
|
6
|
+
| --- | --- | --- | --- | --- |
|
|
7
|
+
| Repeated validation | `None`, type, enum, shape, or schema checks repeat after parsing | Trace the true input boundary and all direct callers | Independently callable API, mutation, persisted reload, distinct invariant | Validate once at the boundary |
|
|
8
|
+
| Phantom exception handling | Broad catches, catch/log/rethrow, impossible caught exception | Enumerate exceptions the operation can raise and required recovery | Cleanup, stable exception translation, documented retry policy | Remove the handler or narrow it to real recovery |
|
|
9
|
+
| Fallback proliferation | Default, legacy, and “safe” paths for states callers cannot produce | Trace configuration, supported environments, and selection paths | Availability requirements, rollout, heterogeneous deployments | Keep the required path and expose invalid state normally |
|
|
10
|
+
| Pass-through wrapper | Function only forwards arguments or renames a readable call | Confirm it adds no policy, lifecycle, stable seam, or representation change | Public facade, transaction boundary, instrumentation | Call the underlying operation directly |
|
|
11
|
+
| Wrapper tower | Adapter-to-adapter or DTO-to-equivalent-DTO chains | Follow data end to end and identify actual boundaries | Protocol translation, generated API surface | Collapse the chain at the nearest real boundary |
|
|
12
|
+
| Single-use helper | One private caller and a short obvious body | Check dynamic registration and whether the name encodes domain knowledge | Callback, recursion, framework hook, meaningful algorithm | Inline and delete helper-only tests |
|
|
13
|
+
| One-implementation abstraction | Interface, factory, strategy, registry, provider, or service with one real implementation | Inspect construction sites and supported extension commitments | External plugins, platform variants, real test seam | Use the concrete behavior directly |
|
|
14
|
+
| Premature utility | Generic helper unifies similar syntax through flags or callbacks | Decide whether callers share domain knowledge or only appearance | One authoritative business or protocol rule | Keep obvious local operations local |
|
|
15
|
+
| Repeated canonicalization | Normalizers run throughout one trusted call graph | Identify first non-canonical input and the consumer requiring canonical form | Signing format, Unicode rule, cache key, equality identity | Canonicalize once at the representation boundary |
|
|
16
|
+
| “Safe” wrapper type | `Validated`, `Canonical`, `Trusted`, or `Safe` type without a trust transition | Locate the representation or authority change it models | Capability type, protocol state, validated external input | Pass the ordinary typed value internally |
|
|
17
|
+
| Redundant copy or freeze | Each layer copies or freezes values without a mutation path | Trace aliases, ownership, concurrency, and external mutation | Views, caches, concurrency, API isolation promise | Keep one clear ownership boundary |
|
|
18
|
+
| Duplicate implementation | Near-identical functions or adapters accumulated across patches | Compare semantics, callers, failure modes, and ownership | Similar code with independent domain evolution | Delete the obsolete copy or use the existing direct path |
|
|
19
|
+
| Dead branch or stale flag | No call sites, impossible branch, fixed flag, overwritten value | Check reflection, registration, exports, config, and supported variants | Framework hooks, serialized names, CLI entry points | Delete the branch and its plumbing |
|
|
20
|
+
| Test-created reachability | A test mutates a gate, injects a future config, or calls a post-gate helper directly | Find an active config, request, persisted value, or runtime selector that reaches it | Current platform variants or staged rollouts with an owned work package | Delete the unreachable branch and its dedicated test |
|
|
21
|
+
| Compatibility residue | Alias, shim, or historical fallback has no supported consumer | Check support policy, releases, public API, and history | Third-party consumers, rolling upgrades, stored data | Delete only with affirmative end-of-support evidence |
|
|
22
|
+
| Product-surface loop | Registry entry, CLI command, diagnostic module, config, and tests only reference one another | Exclude the same cluster and locate a real caller, operator workflow, or external protocol | A documented and currently used public command | Delete the full inactive entry path, not one wrapper |
|
|
23
|
+
| Implicit schema default | Dataclass or parser defaults a missing identity/path/count to a historical value | Check current producers and stored-data support commitments | Truly optional presentation or enrichment fields | Make current identity explicit and fail on omission |
|
|
24
|
+
| Stale current documentation | PLAN, STATUS, architecture, or workflow text still promises a removed fallback or lists a completed fix as pending | Compare current callers/configs with authoritative documents | Clearly labeled history, migration guidance, or future proposals | Update the current claim while preserving labeled history |
|
|
25
|
+
| Comment noise | Text narrates syntax, types, or generic intent | Ask whether it contributes information not present in clear code | External constraint, workaround, invariant, tradeoff | Delete restatement; preserve concise “why” |
|
|
26
|
+
| Type workaround drift | Cast chains, ignores, and runtime probes conflict with surrounding conventions | Confirm current type contract and toolchain behavior | Broken third-party stubs, version-gated API | Express the real type directly and remove stale workarounds |
|
|
27
|
+
| Inferred transformation | Date, filename, project ID, or incidental metadata silently selects a data transform | Find the authoritative caller choice and historical failure evidence | Versioned protocol fields with explicit semantics | Replace guessing with an explicit option and record it |
|
|
28
|
+
|
|
29
|
+
## Fail-visible bias
|
|
30
|
+
|
|
31
|
+
Inside trusted code, unexpected failures should normally surface. Robustness is not a license to hide programming errors. Investigate these patterns first:
|
|
32
|
+
|
|
33
|
+
- broad `except Exception` around ordinary internal work;
|
|
34
|
+
- catch-log-rethrow that adds no stable translation, cleanup, or attribution;
|
|
35
|
+
- catch-and-fallback after a new path raises unexpectedly;
|
|
36
|
+
- `try new behavior; otherwise run old behavior` without a supported version or protocol signal;
|
|
37
|
+
- silent defaults after malformed or missing internal state;
|
|
38
|
+
- compatibility branches with no reachable supported consumer;
|
|
39
|
+
- repeated validation after a trusted boundary or after a typed representation is established;
|
|
40
|
+
- defensive branches whose only evidence is tests created for those branches.
|
|
41
|
+
|
|
42
|
+
For each branch, ask:
|
|
43
|
+
|
|
44
|
+
1. What exact failure is recoverable?
|
|
45
|
+
2. Does the current user requirement or protocol require recovery, translation, cleanup, or compatibility?
|
|
46
|
+
3. Can ordinary runtime failure expose an internal bug more usefully?
|
|
47
|
+
4. Is the fallback's test an independent contract, or part of a closed justification loop?
|
|
48
|
+
5. What supported consumer would break if this branch disappeared?
|
|
49
|
+
|
|
50
|
+
If the answer is only “be robust,” “future-proof,” or “keep old tests green,” prefer direct failure. A bug fix should replace incorrect behavior, not preserve it behind a fallback. Preserve narrow exception handling when it performs concrete cleanup, translates a stable public error, enforces a documented legacy protocol, or protects a real resource/transaction boundary.
|
|
51
|
+
|
|
52
|
+
## Defensive clusters, not isolated lines
|
|
53
|
+
|
|
54
|
+
Do not delete only the most visible helper while leaving its mutually supporting tests, validators, wrappers, or fallback branches. Follow the justification chain outward. If production code, a test, and a receipt/validator each exist only to justify one another, treat them as one deletion candidate. A cluster is preserved only when its chain reaches an independent user requirement, external caller, protocol, security boundary, persistence/corruption boundary, or scientific invariant.
|
|
55
|
+
|
|
56
|
+
## Deletion Evidence
|
|
57
|
+
|
|
58
|
+
Prefer concrete evidence such as:
|
|
59
|
+
|
|
60
|
+
- no static, dynamic, exported, or registered consumer;
|
|
61
|
+
- impossible state under the established typed contract;
|
|
62
|
+
- value already validated at the actual boundary;
|
|
63
|
+
- caught exception cannot originate in the protected operation;
|
|
64
|
+
- wrapper adds no observable behavior;
|
|
65
|
+
- fallback masks a programming or configuration error;
|
|
66
|
+
- duplicate path has an identified authoritative replacement;
|
|
67
|
+
- compatibility promise has explicitly ended;
|
|
68
|
+
- value is overwritten before observation.
|
|
69
|
+
|
|
70
|
+
Do not cite AI authorship as evidence.
|
|
71
|
+
|
|
72
|
+
## Abstraction Check
|
|
73
|
+
|
|
74
|
+
Before keeping an abstraction, ask what independent variation it contains today. “Could support another implementation later” is not enough. Before deleting one, check for external implementations, framework discovery, package commitments, platform variants, and tests that model a real seam rather than merely mock internals.
|
|
75
|
+
|
|
76
|
+
Do not replace removed abstraction with a differently named abstraction. A few duplicated obvious lines can be clearer than a shared layer that forces navigation across files.
|
|
77
|
+
|
|
78
|
+
A thin wrapper can still own a real boundary. Preserve one that bridges independently versioned formats, manages a temporary dataset or resource lifecycle, maintains a published import path, or translates a stable external contract. Thinness alone is not deletion evidence.
|
|
79
|
+
|
|
80
|
+
For active-config reachability and cross-layer ownership, also read [evidence-and-reachability.md](evidence-and-reachability.md).
|
|
@@ -0,0 +1,91 @@
|
|
|
1
|
+
# Evidence and Reachability
|
|
2
|
+
|
|
3
|
+
Cleanup decisions fail when the audit proves that individual pieces exist but never proves how current production reaches them. Use this reference when a deletion touches fixtures, generated artifacts, active configuration, readers, registries, CLIs, or documentation.
|
|
4
|
+
|
|
5
|
+
## Prove ownership from production inputs
|
|
6
|
+
|
|
7
|
+
Trace reachability forward from a non-test producer:
|
|
8
|
+
|
|
9
|
+
```text
|
|
10
|
+
active config / external request / public CLI / persisted record / runtime selection
|
|
11
|
+
-> producer
|
|
12
|
+
-> representation or boundary
|
|
13
|
+
-> reader
|
|
14
|
+
-> current consumer
|
|
15
|
+
-> observable result
|
|
16
|
+
```
|
|
17
|
+
|
|
18
|
+
A test can show that a branch executes; it cannot make the branch current. Synthetic flags, monkeypatched thresholds, future-only configs, diagnostic commands, and scripted dry-runs need a real work package or consumer before they own production behavior.
|
|
19
|
+
|
|
20
|
+
Registries and CLIs require the same test. A registry entry, command, config, diagnostic module, and their tests can form one closed product-surface loop. Exclude that cluster while searching for a caller. If nothing remains except archives or plans, delete the active chain together.
|
|
21
|
+
|
|
22
|
+
## Trace edges, not only nodes
|
|
23
|
+
|
|
24
|
+
Local unit coverage can stay green after a production path breaks:
|
|
25
|
+
|
|
26
|
+
```text
|
|
27
|
+
producer test passes reader test passes state-machine test passes
|
|
28
|
+
\ | /
|
|
29
|
+
no test proves the current identity crosses all three
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
Before retiring a cross-layer fixture, validator, wrapper, or package loader:
|
|
33
|
+
|
|
34
|
+
1. Map each behavior carried by the old path to its current owner.
|
|
35
|
+
2. Retire behavior with no current owner.
|
|
36
|
+
3. Move surviving behavior to the lowest stable public seam that still crosses the real boundary.
|
|
37
|
+
4. Keep one hermetic integration root for each current delivery path whose edges would otherwise be unprotected.
|
|
38
|
+
|
|
39
|
+
Do not preserve an obsolete fixture merely because it once carried useful behavior. Do not delete current behavior merely because its old carrier is obsolete.
|
|
40
|
+
|
|
41
|
+
## Keep permanent tests hermetic
|
|
42
|
+
|
|
43
|
+
Permanent test inputs should come from the repository, explicit managed resources, or data created by the test. A default test that reads an untracked `outputs/` directory, one-off experiment package, or developer-local path is not a reliable integration root. Delete an orphan test/support cluster or redesign a self-contained test from the surviving behavior.
|
|
44
|
+
|
|
45
|
+
Treat tracked assets as read-only during tests. A compiler may hide writes behind configuration, so search the resolved output path rather than only `write_text()` calls in test files. Redirect generated outputs to a temporary directory. When running the full suite, compare tracked and untracked worktree state before and after; identical generated bytes do not make overwriting user edits safe.
|
|
46
|
+
|
|
47
|
+
Match skips to actual dependency boundaries. A fake-only test should not skip because an unused simulator or SDK is absent. Keep a precise skip when the test imports the SDK, compiles a real model, encodes through the optional codec, or otherwise crosses that boundary.
|
|
48
|
+
|
|
49
|
+
Distinguish two kinds of dry-run:
|
|
50
|
+
|
|
51
|
+
- Parsing, assembly, and validation dry-runs may prove that inputs form a valid request without claiming execution capability.
|
|
52
|
+
- A script that invents observations, forces phases, or manufactures success cannot prove planner, hardware, or runtime capability. If its only owner is a permanent success test, remove both.
|
|
53
|
+
|
|
54
|
+
## Close authority and schema boundaries
|
|
55
|
+
|
|
56
|
+
Treat a declared authoritative dependency as required:
|
|
57
|
+
|
|
58
|
+
```text
|
|
59
|
+
package reference -> required artifact -> digest/schema check -> component data
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
Do not write `exists -> verify -> otherwise continue` unless absence is explicitly valid. A missing authority and a mismatched authority have the same failure meaning. Lower-level caches or sidecars cannot silently take over authority.
|
|
63
|
+
|
|
64
|
+
For schema changes, search by format and field names to enumerate all readers, not only imports of the main loader. Include library APIs, CLIs, tools, visualizers, conversion jobs, resume paths, and validators. Current readers reject unsupported versions consistently; explicitly scoped migration readers may continue to accept them.
|
|
65
|
+
|
|
66
|
+
For required identity, path, count, handedness, or source fields, check three layers:
|
|
67
|
+
|
|
68
|
+
1. the declared type has no historical default;
|
|
69
|
+
2. the parser requires the field instead of using `get(..., old_value)`;
|
|
70
|
+
3. validation rejects malformed current data.
|
|
71
|
+
|
|
72
|
+
Filesystem absence and missing keys do not select a legacy protocol. Compatibility requires an explicit version, producer, consumer, and retirement condition.
|
|
73
|
+
|
|
74
|
+
## Treat current documentation as product surface
|
|
75
|
+
|
|
76
|
+
Authoritative PLAN, STATUS, architecture, and workflow documents can keep a retired fallback alive or misstate an unfinished fix as complete. After cleanup, compare their current-path claims with callers and active configuration. Remove stale current claims while preserving clearly labeled history, migration notes, and future proposals.
|
|
77
|
+
|
|
78
|
+
## Decision checklist
|
|
79
|
+
|
|
80
|
+
Before applying a cross-layer deletion, answer:
|
|
81
|
+
|
|
82
|
+
1. Which non-test input reaches this branch today?
|
|
83
|
+
2. Which producer and consumer sit on opposite sides of the boundary?
|
|
84
|
+
3. Does one test protect the current edge, or only its endpoints?
|
|
85
|
+
4. Are every permanent input and output hermetic?
|
|
86
|
+
5. Is each skipped dependency actually invoked?
|
|
87
|
+
6. Is a missing artifact or field explicitly optional?
|
|
88
|
+
7. Which public readers enforce the current schema?
|
|
89
|
+
8. Do current docs and registries describe the same production path?
|
|
90
|
+
|
|
91
|
+
Unanswered ownership or boundary questions are MEDIUM evidence. Resolve them before deletion.
|
|
@@ -0,0 +1,70 @@
|
|
|
1
|
+
# Scientific and Numerical Code
|
|
2
|
+
|
|
3
|
+
Generic software robustness is not numerical correctness. In scientific, simulation, ML, and engineering code, preserve checks with a mathematical, physical, resource, or data-boundary reason; investigate enterprise-style hardening that has none.
|
|
4
|
+
|
|
5
|
+
## Decision Criteria
|
|
6
|
+
|
|
7
|
+
```text
|
|
8
|
+
Does the check enforce a documented mathematical, physical, convergence,
|
|
9
|
+
conditioning, tolerance, allocation, or external-data requirement?
|
|
10
|
+
|
|
|
11
|
+
+-- Yes -> preserve it and keep the reason visible
|
|
12
|
+
|
|
|
13
|
+
`-- No
|
|
14
|
+
`-- Does it detect a concrete independent failure or trust transition?
|
|
15
|
+
+-- Yes -> preserve if the check can actually detect that failure
|
|
16
|
+
`-- No -> likely generic hardening; simplify or delete
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
Prefer the ordinary path:
|
|
20
|
+
|
|
21
|
+
```text
|
|
22
|
+
typed caller input
|
|
23
|
+
-> direct readable numerical kernel
|
|
24
|
+
-> plain result
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
## Investigate Aggressively
|
|
28
|
+
|
|
29
|
+
- cryptographic identity or tamper receipts for in-memory arrays;
|
|
30
|
+
- `ResultEnvelope`, `VerifiedArray`, or canonical-array wrappers with no boundary;
|
|
31
|
+
- finite-real and positive-real validation repeated at every layer;
|
|
32
|
+
- generic stable-product, stable-ratio, or “safe arithmetic” frameworks added speculatively;
|
|
33
|
+
- arbitrary overflow or underflow guards disconnected from the project's numerical model;
|
|
34
|
+
- output-finiteness checks repeated after the responsible kernel already establishes the invariant;
|
|
35
|
+
- defensive copies of every returned array without aliasing risk;
|
|
36
|
+
- duplicate recomputation described as verification;
|
|
37
|
+
- schema, provenance, or evidence files produced and consumed within one experiment process;
|
|
38
|
+
- runtime type checks that restate typed internal APIs.
|
|
39
|
+
|
|
40
|
+
Use ordinary floating-point computation when that is the project's intended model. Do not impose exact arithmetic, universal finiteness, or cryptographic integrity merely because they sound safer.
|
|
41
|
+
|
|
42
|
+
## Preserve When Scientifically Meaningful
|
|
43
|
+
|
|
44
|
+
- solver convergence and termination criteria;
|
|
45
|
+
- physical domain constraints;
|
|
46
|
+
- covariance symmetry or positive-semidefiniteness when mathematically required;
|
|
47
|
+
- conditioning checks tied to algorithm validity;
|
|
48
|
+
- documented numerical tolerances and error budgets;
|
|
49
|
+
- known limiting cases and conservation laws;
|
|
50
|
+
- required event or phase ordering when final state alone can hide an invalid trajectory;
|
|
51
|
+
- explicit failure categories that preserve experiment denominators and diagnostic attribution;
|
|
52
|
+
- finite-result checks where downstream mathematics specifically requires finiteness;
|
|
53
|
+
- bounded allocation and resource constraints;
|
|
54
|
+
- checks preventing invalid numerical algorithms;
|
|
55
|
+
- validation of external datasets, models, checkpoints, and instrument files;
|
|
56
|
+
- provenance consumed independently for reproducibility or audit.
|
|
57
|
+
|
|
58
|
+
## Copies and Mutability
|
|
59
|
+
|
|
60
|
+
Before deleting a copy, trace array ownership, views, aliases, mutation sites, caches, concurrency, and library calls that may mutate inputs. A new array is not automatically untrusted; a view into shared mutable storage can still require isolation.
|
|
61
|
+
|
|
62
|
+
Before preserving immutability machinery, identify the reachable mutation that it prevents. “Results should be safe” is not enough.
|
|
63
|
+
|
|
64
|
+
## Numerical Verification
|
|
65
|
+
|
|
66
|
+
An analytical result, conservation law, independently derived implementation, calibrated reference dataset, or known limiting behavior can provide a real oracle. Repeating the same kernel with the same formula and inputs usually cannot.
|
|
67
|
+
|
|
68
|
+
Do not remove a numerical check merely because it resembles ordinary defensive validation. State its mathematical purpose first; delete only when no such purpose or independent failure class exists.
|
|
69
|
+
|
|
70
|
+
Do not collapse several scientifically distinct failures into one “result looks acceptable” assertion. A final target can be reached after an invalid event order, numerical instability, collision, limit violation, or rebound. Preserve separate checks when those outcomes change acceptance, the experiment denominator, or root-cause analysis.
|
|
@@ -0,0 +1,111 @@
|
|
|
1
|
+
# Test Smells
|
|
2
|
+
|
|
3
|
+
Tests are code and can be accumulated agent slop. Test signal matters more than test volume. Preserve the minimum sufficient set of tests that protects distinct externally meaningful behavior with an independent oracle.
|
|
4
|
+
|
|
5
|
+
## Start with the oracle
|
|
6
|
+
|
|
7
|
+
Ask where the expected result came from and what failure the test would expose.
|
|
8
|
+
|
|
9
|
+
```text
|
|
10
|
+
independent specification / known example / external caller / protocol fixture
|
|
11
|
+
-> meaningful behavior assertion
|
|
12
|
+
|
|
13
|
+
same helper / same algorithm / implementation-generated expected value / mock setup
|
|
14
|
+
-> self-referential test: delete, consolidate, or find a real oracle
|
|
15
|
+
```
|
|
16
|
+
|
|
17
|
+
An independently implemented cross-check is useful only when it can fail differently. Repeating an implementation with different names is not independence.
|
|
18
|
+
|
|
19
|
+
## Build an evidence map
|
|
20
|
+
|
|
21
|
+
For a non-trivial suite, make the reasoning inspectable before deleting nodes:
|
|
22
|
+
|
|
23
|
+
| Test or parameter group | Current owner | Production branch | Observable result | Oracle root | Failure domain | Stronger overlap |
|
|
24
|
+
| --- | --- | --- | --- | --- | --- | --- |
|
|
25
|
+
| one row per candidate | active config, caller, protocol, or none | actual branch reached | success, rejection, error, or state | independent source | what can fail differently | test that dominates it, if any |
|
|
26
|
+
|
|
27
|
+
This can remain working notes; do not add a permanent audit artifact merely to prove cleanup. The map prevents three common errors: merging tests that have different failure semantics, retaining permutations that reach the same branch and result, and deleting production behavior merely because its only test was redundant.
|
|
28
|
+
|
|
29
|
+
One test dominates another only when it protects the same owner, branch, result, and failure domain with an equally independent or stronger oracle. A public integration test does not automatically dominate a known numerical example, corruption test, or protocol rejection. Conversely, a long snapshot does not dominate a focused test merely because it asserts more fields.
|
|
30
|
+
|
|
31
|
+
## High-priority accumulated patterns
|
|
32
|
+
|
|
33
|
+
| Pattern | Suspicious signal | Preserve when | Preferred action |
|
|
34
|
+
| --- | --- | --- | --- |
|
|
35
|
+
| Duplicate behavior tests | Same input, path, and meaningful outcome with no distinct failure mode | Each test protects a separate contract or historical regression | Keep the strongest readable test |
|
|
36
|
+
| Successive regression-test accumulation | A new near-identical test was added after every minor agent/user correction | Each correction protects a genuinely different externally visible bug | Collapse to the minimum distinct regression set |
|
|
37
|
+
| Duplicate integration coverage | Several tests traverse the same public path and oracle | Different boundary, failure class, or supported environment is exercised | Remove redundant scenarios |
|
|
38
|
+
| Parameter-permutation bloat | Values vary without changing a branch, equivalence class, invariant, or risk | The permutation represents a real boundary or failure mode | Keep representative cases |
|
|
39
|
+
| Aggregate-plus-item duplicate | `all(...)`, count, or length restates stronger per-item behavior checks | The aggregate is a distinct collection-level contract | Delete the weaker assertion |
|
|
40
|
+
| Weak type/None/length/existence check | Assertion is dominated by a stronger behavior assertion in the same test | Presence, type, or cardinality is itself public contract | Remove or keep only the independent contract |
|
|
41
|
+
| Private-helper test | Test exists because an agent extracted a helper already covered through a public seam | Helper has a stable, independently specified algorithm | Delete with the helper or test publicly |
|
|
42
|
+
| Implementation-detail test | Private state, helper call, exact temporary object, or harmless ordering is asserted | Detail is a protocol, transaction, concurrency, lifecycle, or resource contract | Assert the meaningful seam |
|
|
43
|
+
| Exact call-count test | Count follows current implementation mechanics | Count controls billing, retry, batching, idempotency, or a documented limit | Assert external effect unless count is semantic |
|
|
44
|
+
| Exact sequencing test | Harmless refactor breaks the expected order | Order is required by protocol, locks, transactions, or cleanup lifecycle | Keep only semantic ordering |
|
|
45
|
+
| Mock-verifies-mock | Mock setup dictates output and verification repeats setup | Interaction with an unavailable boundary is the real contract | Reduce mocking and assert boundary behavior |
|
|
46
|
+
| Wrapper-only test | Test protects a pass-through wrapper with no policy or boundary | Wrapper is a published API, external integration, or lifecycle seam | Remove wrapper and test together |
|
|
47
|
+
| Defensive-machinery test | Test exists only for a validator, fallback, receipt, checksum, retry, or self-check that is itself suspect | Machinery has an independent surviving contract | Delete the mutual-support cluster |
|
|
48
|
+
| Implementation-derived expected value | Expected value is recomputed by the code under test or a copied algorithm | Independent reference data or mathematical invariant exists | Use the independent oracle or delete |
|
|
49
|
+
| Snapshot bloat | Large snapshots obscure a small behavioral signal | Rendered/serialized artifact is the externally reviewed contract | Replace with focused semantic assertions |
|
|
50
|
+
| Fixture-tautological test | Fixture setup guarantees the asserted state and no production behavior is invoked | Fixture state is an independently established integration precondition | Delete |
|
|
51
|
+
| Obsolete-behavior test | Test conflicts with an explicit current-task requirement or current authoritative project document | Supported consumers still require the old behavior | Replace the test with the corrected contract |
|
|
52
|
+
| Unmanaged-artifact test | Default tests read untracked `outputs/`, a one-off run directory, or a local formal package | Input is repository-managed, test-created, or an explicitly provisioned external resource | Delete the orphan test/support cluster or rebuild a self-contained behavioral test |
|
|
53
|
+
| Tracked-output test | A compiler or materializer reached through a test writes to a tracked target | The test redirects writes to a temporary path and treats tracked data as read-only input | Redirect the output; inspect deep builders, not only direct file writes |
|
|
54
|
+
| Synthetic-capability test | A production dry-run invents observations or success states and a test asserts that scripted success | Dry-run only parses, assembles, or validates without claiming the backend capability | Delete the fake capability path and its test |
|
|
55
|
+
| Fake-only dependency skip | A fake-backed test skips when an SDK or simulator it never calls is absent | The test actually imports, compiles, encodes, or calls the optional boundary | Remove the skip or use one consistent module-level boundary |
|
|
56
|
+
| Disconnected layer tests | Producer, loader, and consumer pass separately, but no test crosses the current identity through all three | A hermetic integration root already protects that production edge | Keep or create the smallest independent current-path root before retiring the old fixture |
|
|
57
|
+
| Coverage-owned branch test | A threshold mutation, future-only config, or direct post-gate helper call exists only to cover a branch | Active production input reaches the branch or an adjacent guard has distinct safety semantics | Remove the unreachable branch/test cluster after checking current configs and registries |
|
|
58
|
+
|
|
59
|
+
## Closed justification loops
|
|
60
|
+
|
|
61
|
+
Production code does not justify a test merely because the test exercises it. A test does not justify production code merely because the production code exists. Trace test -> production branch -> reason -> external evidence.
|
|
62
|
+
|
|
63
|
+
If a test exists only to keep a defensive branch green, and the branch exists only because that test was added, the pair is mutual-support slop. Apply the same rule to checksum tests and digest code, receipt tests and receipt validators, wrapper tests and wrappers, compatibility tests and obsolete branches, and validator tests and validators. Do not preserve one member merely because the other member depends on it; delete the closed cluster when no independent root remains.
|
|
64
|
+
|
|
65
|
+
Independent roots include an explicit current-task requirement, a current authoritative project document, real external caller, public API, documented protocol, security or trust boundary, persisted corruption boundary, scientific invariant, or a separately maintained reference dataset.
|
|
66
|
+
|
|
67
|
+
## Keep or delete
|
|
68
|
+
|
|
69
|
+
For every test, answer:
|
|
70
|
+
|
|
71
|
+
1. What distinct behavior or plausible regression does it protect?
|
|
72
|
+
2. Would it fail if that behavior regressed?
|
|
73
|
+
3. Is the oracle independent from the implementation and its support machinery?
|
|
74
|
+
4. Is stronger coverage already present?
|
|
75
|
+
5. Does it observe a public or meaningful boundary?
|
|
76
|
+
6. Would a harmless refactor break it while behavior stayed identical?
|
|
77
|
+
7. Was it added only after a local correction, without a new failure mode?
|
|
78
|
+
|
|
79
|
+
Delete or consolidate when no distinct regression remains. Preserve separate tests for distinct contracts, failure semantics, security properties, persistence/transaction behavior, concurrency, supported compatibility, resource limits, scientific invariants, known numerical results, and real prior regressions.
|
|
80
|
+
|
|
81
|
+
Do not optimize for test count, assertion count, branch coverage, or line coverage. A smaller suite with independent signal is better than a larger suite that merely documents its own implementation.
|
|
82
|
+
|
|
83
|
+
## Retire fixtures by behavior
|
|
84
|
+
|
|
85
|
+
An obsolete fixture and the behavior once reached through it are separate decisions. Map every fixture-backed test to a current owner. Delete behavior with no owner; move surviving behavior to the lowest stable public seam. Before removing a cross-layer fixture, confirm that one hermetic test still crosses the current producer, reader, and consumer. Green endpoint unit tests do not prove that edge.
|
|
86
|
+
|
|
87
|
+
Delete a fixture's private builders, fakes, compatibility fields, and helper stack when no surviving test or production path uses them. Do not restore an unmanaged experiment artifact or preserve a whole legacy package merely to keep one current assertion reachable.
|
|
88
|
+
|
|
89
|
+
## Adding tests during cleanup
|
|
90
|
+
|
|
91
|
+
Adding a test is not the default response. Add one only when:
|
|
92
|
+
|
|
93
|
+
- a real externally meaningful behavior is otherwise unprotected;
|
|
94
|
+
- the behavior is genuinely uncertain after reading callers, contracts, and history;
|
|
95
|
+
- a plausible regression would fail the test;
|
|
96
|
+
- the expected result has an independent source; and
|
|
97
|
+
- the test observes a meaningful seam rather than cleanup details.
|
|
98
|
+
|
|
99
|
+
When production slop is deleted, delete tests whose only purpose was to protect that slop in the same change. Do not add a replacement test merely because a function now lacks a unit test, coverage falls, or a model prefers symmetry. An explicit current-task requirement or current authoritative project document outranks a historical test that asserts obsolete or incorrect behavior; an inferred preference does not.
|
|
100
|
+
|
|
101
|
+
## Consolidation discipline
|
|
102
|
+
|
|
103
|
+
- Prefer one readable test per distinct behavior, not one per branch or helper.
|
|
104
|
+
- Parameterize only genuine equivalence classes; do not hide an opaque generated matrix.
|
|
105
|
+
- Keep scenario names and failure messages capable of localizing a real regression.
|
|
106
|
+
- Run the remaining suite after deletion; zero remaining tests is not a passing cleanup.
|
|
107
|
+
- Preserve low-level tests when low-level behavior itself is a stable contract, not merely because the code is private.
|
|
108
|
+
- Count collected nodes before and after, then inspect skips and deselections; a smaller reported total can hide lost execution.
|
|
109
|
+
- Fingerprint the worktree around suites that invoke compilers, materializers, exporters, or code generators.
|
|
110
|
+
|
|
111
|
+
For generated inputs, current configuration, and cross-layer path checks, also read [evidence-and-reachability.md](evidence-and-reachability.md).
|
|
@@ -0,0 +1,143 @@
|
|
|
1
|
+
# Verification and Trust
|
|
2
|
+
|
|
3
|
+
Verification adds value only when the verifier has information, authority, or a failure domain meaningfully independent from the producer. A checksum is not automatically a trust boundary, and a second function or test is not automatically an independent oracle.
|
|
4
|
+
|
|
5
|
+
## Closed justification loops
|
|
6
|
+
|
|
7
|
+
Trace the whole dependency cluster outward:
|
|
8
|
+
|
|
9
|
+
```text
|
|
10
|
+
production result
|
|
11
|
+
-> digest / receipt / manifest / validator
|
|
12
|
+
-> validator test
|
|
13
|
+
-> reason each member exists
|
|
14
|
+
-> independent evidence root?
|
|
15
|
+
```
|
|
16
|
+
|
|
17
|
+
Production code does not justify a test merely because the test exercises it. A test does not justify production code merely because the production code exists. If a checksum exists only because its test expects it, and that test exists only because the checksum was added, the pair is **mutual-support slop**. The same loop can contain serialization added only for hashing, digest fields, result envelopes, manifests, receipts, recomputation, wrappers, and wrapper-only tests.
|
|
18
|
+
|
|
19
|
+
Delete the whole closed cluster when its chain reaches no independent root. Independent roots include:
|
|
20
|
+
|
|
21
|
+
- a current user requirement or correction;
|
|
22
|
+
- a real external caller or public API contract;
|
|
23
|
+
- a documented wire protocol or specification;
|
|
24
|
+
- a separately controlled security key or trust authority;
|
|
25
|
+
- an independently supplied manifest, release digest, or content-addressed identity;
|
|
26
|
+
- a persistence/readback or corruption boundary;
|
|
27
|
+
- an independent consumer, regulator, reproducibility workflow, or operational system;
|
|
28
|
+
- a scientific or numerical invariant with an independently derived expected result.
|
|
29
|
+
|
|
30
|
+
Preserve one member only when the remaining cluster still serves such a root. Do not preserve every member because another member depends on it.
|
|
31
|
+
|
|
32
|
+
## Trust-boundary decision tree
|
|
33
|
+
|
|
34
|
+
```text
|
|
35
|
+
Does the value cross out of the current trusted execution domain?
|
|
36
|
+
|
|
|
37
|
+
+-- No
|
|
38
|
+
| `-- Is there concrete corruption, persistence, ownership, concurrency,
|
|
39
|
+
| or independent hardware/process failure to detect?
|
|
40
|
+
| +-- No -> verification is probably redundant
|
|
41
|
+
| `-- Yes -> preserve only the check that can detect that failure
|
|
42
|
+
|
|
|
43
|
+
`-- Yes
|
|
44
|
+
`-- Is the source independently controlled or untrusted?
|
|
45
|
+
+-- Yes -> parse / validate / decode once at the boundary
|
|
46
|
+
`-- No
|
|
47
|
+
`-- Does a protocol, specification, or consumer require it?
|
|
48
|
+
+-- Yes -> preserve the required check
|
|
49
|
+
`-- No -> identify the concrete failure model first
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
Public visibility alone does not create a trust boundary. Two services owned by the same producer can still be circular; one process can still cross a real failure domain by writing, publishing, reopening, decoding, or reading persisted output.
|
|
53
|
+
|
|
54
|
+
## Declared authority is not optional enrichment
|
|
55
|
+
|
|
56
|
+
When a config, package manifest, or sidecar declares an authoritative artifact, require and verify that artifact. A pattern such as `if path.exists(): verify(path)` silently changes a missing authority into compatibility mode. Missing and mismatched authoritative inputs should fail at the same boundary unless the protocol explicitly defines absence as valid.
|
|
57
|
+
|
|
58
|
+
Lower-level sidecars or caches do not replace a missing package-level authority merely because they remain internally consistent. Conversely, a thumbnail, annotation, or cache marked optional by the current protocol may be absent without failure. Determine required versus optional from the contract, not from filesystem presence.
|
|
59
|
+
|
|
60
|
+
## Schema and identity closure
|
|
61
|
+
|
|
62
|
+
A writer and its main loader do not complete a schema migration. Enumerate every public deserialize boundary: library API, CLI, `tools/`, visualizer, converter, resume/recovery path, and validator. Each current reader should reuse the canonical validator or enforce the same shared version contract. Keep an explicit migration reader only when old data remains supported.
|
|
63
|
+
|
|
64
|
+
For critical identity, source path, count, or handedness fields, align all three layers: type declaration, parser, and validation. A required annotation paired with `mapping.get(..., old_default)` remains an implicit compatibility path. Remove defaults that can reinterpret malformed current data as a historical format.
|
|
65
|
+
|
|
66
|
+
## Independence test
|
|
67
|
+
|
|
68
|
+
For every claimed verification, record:
|
|
69
|
+
|
|
70
|
+
1. **Producer:** who created the value?
|
|
71
|
+
2. **Verifier:** who checks it?
|
|
72
|
+
3. **Independent knowledge:** what does the verifier know that the producer did not generate?
|
|
73
|
+
4. **Independent authority:** is there a separate key, manifest, specification, identity, or consumer?
|
|
74
|
+
5. **Independent failure domain:** can producer and verifier fail differently?
|
|
75
|
+
6. **Action:** what meaningful response follows failure?
|
|
76
|
+
|
|
77
|
+
If all inputs, logic, authority, and failure modes are shared, the check is circular even when it uses cryptography or another process.
|
|
78
|
+
|
|
79
|
+
## SHA256 and checksum clusters
|
|
80
|
+
|
|
81
|
+
Treat SHA256/checksum machinery as a cluster, not as one function. Search for the entire chain:
|
|
82
|
+
|
|
83
|
+
```text
|
|
84
|
+
canonicalization / serialization added only for hashing
|
|
85
|
+
-> digest field or checksum helper
|
|
86
|
+
-> result envelope / manifest / receipt
|
|
87
|
+
-> recomputation or validator
|
|
88
|
+
-> tests and fixtures created only for those mechanisms
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
When there is no independent expected digest, identity authority, trust transition, persistence-corruption role, content-addressing role, or external consumer, investigate and remove the whole support chain. Strong deletion candidates include hashes of local arguments, parameter objects, internal arrays, result dataclasses, self-created manifests, self-issued receipts, and evidence consumed only by the same agent or process.
|
|
92
|
+
|
|
93
|
+
Do not call a local digest authentication merely because it is SHA256. A digest retained after secret redaction can be a legitimate non-secret correlation fingerprint when an independently recorded input or consumer uses it; that is provenance, not authentication, and its independent role must be explicit.
|
|
94
|
+
|
|
95
|
+
## Verification-theater patterns
|
|
96
|
+
|
|
97
|
+
### Recompute theater
|
|
98
|
+
|
|
99
|
+
```text
|
|
100
|
+
calculate result -> repeat the same calculation -> call it verified
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
Delete when both computations share algorithm, inputs, constants, and likely bugs. Preserve independently derived analytical checks, diversified implementations with a concrete safety purpose, and cheap invariants that detect a distinct failure class.
|
|
104
|
+
|
|
105
|
+
### Schema and envelope theater
|
|
106
|
+
|
|
107
|
+
```text
|
|
108
|
+
producer emits payload -> producer-owned schema mirrors payload -> producer validates it
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
Delete when no external protocol, persisted format, or independent consumer relies on the schema. Preserve wire contracts and independently maintained schemas.
|
|
112
|
+
|
|
113
|
+
### Receipt, manifest, and evidence theater
|
|
114
|
+
|
|
115
|
+
```text
|
|
116
|
+
operation -> evidence.json / receipt -> local validator -> local test
|
|
117
|
+
```
|
|
118
|
+
|
|
119
|
+
Delete ledgers and validators generated and consumed only by the same trusted workflow. Preserve records required by a separate authority, operational system, regulator, reproducibility workflow, transaction, idempotency boundary, or corruption detector.
|
|
120
|
+
|
|
121
|
+
### Signature theater
|
|
122
|
+
|
|
123
|
+
```text
|
|
124
|
+
generate local key -> sign local result -> verify with paired local key
|
|
125
|
+
```
|
|
126
|
+
|
|
127
|
+
Delete when no identity, key custody, distribution boundary, or independent trust anchor exists. Preserve signatures authenticating an external publisher, protocol peer, or independently controlled key.
|
|
128
|
+
|
|
129
|
+
## Persistence and readback
|
|
130
|
+
|
|
131
|
+
Reopening an encoded file, archive, media object, or persisted record can be genuinely independent when it detects truncation, partial publication, codec mismatch, schema loss, or corruption across a write/read boundary. Do not collapse that into circular in-memory recomputation. Conversely, reading a value back from the same memory representation without a distinct failure domain adds little.
|
|
132
|
+
|
|
133
|
+
## Defensive and fallback checks
|
|
134
|
+
|
|
135
|
+
Repeated null/type/shape checks, broad catches, catch-log-rethrow, catch-and-fallback, repeated normalization, and self-validating result objects are suspicious when they add no boundary or failure domain. Preserve checks that enforce a documented safety, resource, authorization, transaction, concurrency, persistence, protocol, or numerical invariant.
|
|
136
|
+
|
|
137
|
+
Prefer direct failure for unexpected internal errors. A narrow fallback for one documented missing field or supported legacy version is different from `except Exception: use_old_path()`. The former may be contractual while the compatibility window remains; the latter hides bugs unless independent evidence proves otherwise.
|
|
138
|
+
|
|
139
|
+
## Preserve by default
|
|
140
|
+
|
|
141
|
+
When evidence is incomplete, preserve security and authorization, real external protocols, supported compatibility, persistence and transactions, concurrency, resource limits, scientific invariants, independently supplied artifact verification, and content-addressed identity. Report the missing evidence instead of inventing a justification or deleting on aesthetic grounds.
|
|
142
|
+
|
|
143
|
+
For production reachability, public-reader enumeration, and documentation/config ownership, also read [evidence-and-reachability.md](evidence-and-reachability.md).
|