dsh-logicprobe 0.9.0 → 1.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.en-US.md +128 -66
- package/README.md +128 -66
- package/lib/baseline.js +142 -0
- package/lib/client.js +56 -1
- package/lib/compose-tool.js +1 -1
- package/lib/concurrency.js +14 -0
- package/lib/data-engine.js +42 -8
- package/lib/engine.js +3151 -2801
- package/lib/export-tool.js +27 -2
- package/lib/granularity.js +237 -0
- package/lib/index.js +63 -41
- package/lib/report-diff-tool.js +39 -0
- package/lib/structure-tool.js +64 -0
- package/lib/structure.js +648 -0
- package/lib/tool.js +3 -3
- package/lib/types/baseline.d.ts +69 -0
- package/lib/types/concurrency.d.ts +10 -0
- package/lib/types/data-engine.d.ts +13 -1
- package/lib/types/engine.d.ts +419 -277
- package/lib/types/granularity.d.ts +106 -0
- package/lib/types/index.d.ts +9 -8
- package/lib/types/report-diff-tool.d.ts +8 -0
- package/lib/types/structure-tool.d.ts +10 -0
- package/lib/types/structure.d.ts +156 -0
- package/lib/types/tool.d.ts +2 -2
- package/lib/types/uml.d.ts +89 -0
- package/lib/uml-tool.js +146 -104
- package/lib/uml.js +1748 -1544
- package/locale/en.json +6 -0
- package/locale/zh.json +6 -0
- package/package.json +16 -14
- package/skills/logicprobe/SKILL.md +84 -58
- package/skills/logicprobe/references/dsh-model-schema.md +102 -10
- package/skills/logicprobe/references/gap-routing-guide.md +37 -36
- package/skills/logicprobe/references/hash-spec.md +153 -0
- package/skills/logicprobe/references/logic-verification-guide.md +27 -250
- package/skills/logicprobe-concurrency/SKILL.md +43 -0
- package/skills/{logicprobe → logicprobe-concurrency}/references/concurrency-risk-guide.md +1 -1
- package/skills/logicprobe-datamodel/SKILL.md +5 -4
- package/skills/logicprobe-datamodel/references/data-model-guide.md +1 -1
- package/skills/logicprobe-datamodel/references/data-model-schema.md +1 -1
- package/skills/logicprobe-structure/SKILL.md +95 -0
- package/skills/logicprobe-structure/references/structure-review-guide.md +177 -0
- package/skills/logicprobe-uml/SKILL.md +57 -0
- package/skills/{logicprobe → logicprobe-uml}/references/uml-modeling-guide.md +121 -8
- package/src/baseline.ts +180 -0
- package/src/client.js +56 -1
- package/src/compose-tool.ts +47 -47
- package/src/concurrency.ts +23 -0
- package/src/data-engine.ts +51 -9
- package/src/engine.ts +503 -72
- package/src/export-tool.ts +73 -48
- package/src/granularity.ts +312 -0
- package/src/index.ts +63 -41
- package/src/report-diff-tool.ts +43 -0
- package/src/structure-tool.ts +68 -0
- package/src/structure.ts +724 -0
- package/src/tool.ts +3 -3
- package/src/uml-tool.ts +53 -10
- package/src/uml.ts +288 -24
- package/skills/logicprobe/references/logicprobe-engine.py +0 -4943
- package/skills/logicprobe/references/verification-harness.py +0 -959
- package/skills/logicprobe-datamodel/references/data-model-harness.py +0 -528
|
@@ -0,0 +1,153 @@
|
|
|
1
|
+
# Model Hash Specification
|
|
2
|
+
|
|
3
|
+
`modelHash` is how a report, an archive record and a CI comparison agree on *which*
|
|
4
|
+
model was verified. A hash nobody can recompute is decoration, so the algorithm, the
|
|
5
|
+
normalization rules and the excluded keys are published here — and pinned by golden
|
|
6
|
+
tests (`tests/engine/run.mjs`, and the parity test).
|
|
7
|
+
|
|
8
|
+
## Published specifications
|
|
9
|
+
|
|
10
|
+
| Spec | Since | Normalization |
|
|
11
|
+
|------|-------|---------------|
|
|
12
|
+
| `v1` | 1.0.0 | sorted keys, no insignificant whitespace, UTF-8 with non-ASCII preserved, **every `_`-prefixed key removed at every level** |
|
|
13
|
+
| `v0` | never published | the same normalization **without** metadata filtering — the pre-1.0 behaviour, kept only so an archived hash can be checked against the old engine |
|
|
14
|
+
|
|
15
|
+
`v0` was never a specification; it is the behaviour that existed before this document
|
|
16
|
+
did. Do not record new hashes with it.
|
|
17
|
+
|
|
18
|
+
For a model that carries no `_`-prefixed key, `v0` and `v1` produce the *identical*
|
|
19
|
+
hash. That is why adopting `v1` does not invalidate any hash recorded before it.
|
|
20
|
+
|
|
21
|
+
## The algorithm (`v1`)
|
|
22
|
+
|
|
23
|
+
1. Take the model as JSON.
|
|
24
|
+
2. Remove every key whose name starts with `_`, at every level — including objects
|
|
25
|
+
nested inside arrays.
|
|
26
|
+
3. Serialize deterministically:
|
|
27
|
+
- object keys sorted by code-unit order,
|
|
28
|
+
- tokens separated by `,` and `:` with no whitespace,
|
|
29
|
+
- strings escaped as JSON, non-ASCII preserved (`réussi` stays `réussi`, not `r\u00e9ussi`),
|
|
30
|
+
- numbers written the way JavaScript `JSON.stringify` writes them,
|
|
31
|
+
- `null`, `true`, `false` spelled as JSON.
|
|
32
|
+
4. `sha256` of the UTF-8 bytes, lower-case hex.
|
|
33
|
+
|
|
34
|
+
In other words, with `strip_metadata` removing `_`-prefixed keys at every level:
|
|
35
|
+
|
|
36
|
+
```text
|
|
37
|
+
sha256(json.dumps(strip_metadata(model), sort_keys=True, separators=(',', ':'), ensure_ascii=False))
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
The TypeScript engine does not call `json.dumps`; it implements the same serialization
|
|
41
|
+
directly (`stableStringify`), and the parity test compares the two engines
|
|
42
|
+
byte-for-byte on every fixture. The specification, not either implementation, is the
|
|
43
|
+
contract.
|
|
44
|
+
|
|
45
|
+
Nothing else is normalized: no key renaming, no default filling, no unit conversion,
|
|
46
|
+
no ordering of `states` or `transitions` arrays. Two models that differ only in array
|
|
47
|
+
order hash differently — that is intended, because the order is part of the input.
|
|
48
|
+
|
|
49
|
+
## Recomputing a hash
|
|
50
|
+
|
|
51
|
+
Both commands below reproduce the engine's value exactly.
|
|
52
|
+
|
|
53
|
+
```bash
|
|
54
|
+
# Node
|
|
55
|
+
node -e "const m=JSON.parse(require('fs').readFileSync('model.json','utf8'));const s=(v)=>Array.isArray(v)?v.map(s):v&&typeof v==='object'?Object.fromEntries(Object.keys(v).sort().filter((k)=>!k.startsWith('_')).map((k)=>[k,s(v[k])])):v;console.log(require('crypto').createHash('sha256').update(JSON.stringify(s(m))).digest('hex'))"
|
|
56
|
+
|
|
57
|
+
# Node.js
|
|
58
|
+
node -e "const c=require('node:crypto'),f=require('node:fs');const s=v=>Array.isArray(v)?'['+v.map(s).join(',')+']':(v&&typeof v==='object'?'{'+Object.keys(v).filter(k=>!k.startsWith('_')).sort().map(k=>JSON.stringify(k)+':'+s(v[k])).join(',')+'}':JSON.stringify(v));console.log(c.createHash('sha256').update(s(JSON.parse(f.readFileSync('model.json','utf8')))).digest('hex'))"
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
The reports carry the specification next to the value, e.g. `"hashSpec": "v1"`, so an
|
|
62
|
+
archive record does not need to guess:
|
|
63
|
+
|
|
64
|
+
```json
|
|
65
|
+
{ "hashSpec": "v1", "modelHash": "f583202888b827922ac512a312758fa2d8cf9055b102205ecad18f81a0a7d539" }
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
Round-trip reports (`logicprobe_uml action=review`, `roundTrip`) carry `hashSpec`
|
|
69
|
+
alongside `modelHash` and `parsedHash`, so a fidelity comparison is always
|
|
70
|
+
spec-qualified.
|
|
71
|
+
|
|
72
|
+
**Read `diffs`, not `ok`.** `roundTrip.ok` says only that the model was rendered and the
|
|
73
|
+
result read back; it is `true` while `modelHash` and `parsedHash` legitimately differ,
|
|
74
|
+
because the model hash covers what a diagram cannot carry (the `narrative` block, in
|
|
75
|
+
particular) and the parsed hash covers what the diagram actually says. Fidelity is
|
|
76
|
+
`roundTrip.diffs.length === 0`. Equal hashes are neither necessary nor sufficient for
|
|
77
|
+
fidelity, and the two must not be forced to agree.
|
|
78
|
+
|
|
79
|
+
Every report also aggregates its hashes under `hashes`, so a baseline diff or an archive
|
|
80
|
+
record reads one field instead of walking the report:
|
|
81
|
+
|
|
82
|
+
```json
|
|
83
|
+
{ "hashes": { "hashSpec": "v1", "modelHash": "f5832028…", "afterModelHash": "f5832028…" } }
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
A structure review hashes its inputs under the same rule but with its own keys:
|
|
87
|
+
`hashes.diagram` (sha256 of the diagram text) and `hashes.matrix` (sha256 of the
|
|
88
|
+
canonical — key-sorted — matrix JSON, so key order does not change the hash) when a
|
|
89
|
+
dependency matrix was supplied. A data-model report carries `hashes.modelHash`; the
|
|
90
|
+
data-model hash uses the same normalization but its schema is not covered by the
|
|
91
|
+
published specification.
|
|
92
|
+
|
|
93
|
+
A `diagram` hash is a **text** hash, not a semantic one: `sha256` over the exact
|
|
94
|
+
characters, so comments and `note` blocks change it, and so does a trailing newline.
|
|
95
|
+
Two renderings that mean the same thing hash differently when their text differs, and
|
|
96
|
+
that is intended — it is what makes "the diagram I reviewed" checkable. For one model
|
|
97
|
+
(`modelHash` `77a43a39822147707229638515cf8f0c6bef4b08635ea6197d2181fc6c26f0b4`) four
|
|
98
|
+
probe variants hash as:
|
|
99
|
+
|
|
100
|
+
| Diagram text | `hashes.diagram` |
|
|
101
|
+
|--------------|------------------|
|
|
102
|
+
| clean | `25ee1d0b803c9477d526d92c1dbb9059e408e282e515499dba06d14630286c8d` |
|
|
103
|
+
| one `%%` comment line | `8c41f046c954e1dc086eee8bd24c3d1d033b489840895520002c1e8120dda052` |
|
|
104
|
+
| one `'` comment line (PlantUML style) | `5552a91a73972edb71ea98f460fa6a393c6556bd1241461f0f29cd51531e9e56` |
|
|
105
|
+
| one multi-line `note … end note` block | `359182313f451713a937f5148328d238e8ea1941fbcef663c4c34e0a7f1bbcda` |
|
|
106
|
+
|
|
107
|
+
Compare diagram hashes only between identical byte strings: the same diagram handed
|
|
108
|
+
over without its trailing newline hashes differently (`8c41f046…` becomes
|
|
109
|
+
`c8f3a41b…`). That is the specification, not drift.
|
|
110
|
+
|
|
111
|
+
## Checking an archived hash
|
|
112
|
+
|
|
113
|
+
`--hash-spec` recomputes under a named specification; `--hash-check` answers whether a
|
|
114
|
+
recorded value belongs to any published specification *at all*:
|
|
115
|
+
|
|
116
|
+
```bash
|
|
117
|
+
# verify under a named spec
|
|
118
|
+
The repository CLI takes the same flag: `verify model.json --hash-spec v0`
|
|
119
|
+
|
|
120
|
+
# ask about a recorded hash: prints {checked, matches, published[], verdict, verdictReason}
|
|
121
|
+
The repository CLI reports which spec reproduces an archived hash: `verify model.json --hash-check 0000…deadbeef`
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
`--hash-check` exits `0` when a published spec reproduces the value and `2` when none
|
|
125
|
+
does, with the explicit answer `no published hash spec reproduces <hex>`. It never
|
|
126
|
+
guesses a spec to make a number match.
|
|
127
|
+
|
|
128
|
+
The in-process equivalents are `modelHash(model, spec)` (TypeScript,
|
|
129
|
+
`lib/engine.js`) and `model_hash(model, spec)` in the mirror, plus `hashPayload(model, spec)`
|
|
130
|
+
and `hash_payload(model, spec)` for the exact bytes that are hashed.
|
|
131
|
+
|
|
132
|
+
## When the hash changes
|
|
133
|
+
|
|
134
|
+
A change that alters the bytes alters the hash, and that is a compatibility event:
|
|
135
|
+
|
|
136
|
+
- Adding, removing or renaming an *excluded* key (`_`-prefixed): no hash change by
|
|
137
|
+
construction.
|
|
138
|
+
- Adding a declared model field that a model actually uses: that model's hash changes.
|
|
139
|
+
The change must be named in the release note, and it is covered by the golden tests
|
|
140
|
+
rather than discovered by a user comparing snapshots.
|
|
141
|
+
- Changing any normalization rule (separator, escaping, key order, exclusion set): a
|
|
142
|
+
**new** specification, never a silent change to `v1`. The old spec keeps its name and
|
|
143
|
+
its behaviour so archived hashes stay checkable.
|
|
144
|
+
|
|
145
|
+
`--hash-spec` changes the reported `modelHash` only. The `beforeModelHash` /
|
|
146
|
+
`afterModelHash` pair inside a `comparison` block is a diagnostic of one run and always
|
|
147
|
+
uses `v1`.
|
|
148
|
+
|
|
149
|
+
### Version history
|
|
150
|
+
|
|
151
|
+
| Release | Hash change | Affected models |
|
|
152
|
+
|---------|-------------|-----------------|
|
|
153
|
+
| 1.0.0 | `hashSpec: "v1"` published; `_`-prefixed keys are now excluded from the hash (and accepted by the schema) | **None.** Before 1.0.0 a model carrying `_` keys was rejected outright, so no accepted model's hash moves. `v0` reproduces the old behaviour for archived values, and the two specs are byte-identical for a model without `_` keys — pinned by golden tests on `happy-path.json` and `narrative-complete.json`. |
|
|
@@ -90,288 +90,67 @@ which wastes more time than no verification at all.
|
|
|
90
90
|
|
|
91
91
|
## Probe Design Patterns
|
|
92
92
|
|
|
93
|
+
Every probe below is implemented by `logicprobe_verify`. These patterns are here to read a finding, not to reimplement it.
|
|
94
|
+
|
|
93
95
|
### A1: Unexpected Event Injection
|
|
94
96
|
|
|
95
|
-
|
|
96
|
-
# For each state S, list all events defined anywhere in the machine
|
|
97
|
-
# For each event E not handled by S, test: what happens?
|
|
98
|
-
def unexpected_event_probe(states):
|
|
99
|
-
all_events = set()
|
|
100
|
-
for trans in states.values():
|
|
101
|
-
all_events.update(trans.keys())
|
|
102
|
-
|
|
103
|
-
findings = []
|
|
104
|
-
for state, transitions in states.items():
|
|
105
|
-
unhandled = all_events - set(transitions.keys())
|
|
106
|
-
if unhandled:
|
|
107
|
-
findings.append({
|
|
108
|
-
"state": state,
|
|
109
|
-
"unhandled_events": list(unhandled),
|
|
110
|
-
"risk": "Silent ignore or undefined behavior"
|
|
111
|
-
})
|
|
112
|
-
return findings
|
|
113
|
-
```
|
|
97
|
+
For every state, list the events declared anywhere in the machine that this state does not handle. An unhandled event is a silent ignore or undefined behaviour.
|
|
114
98
|
|
|
115
99
|
### A2: Race Interleaving
|
|
116
100
|
|
|
117
|
-
For pairs
|
|
118
|
-
|
|
119
|
-
```python
|
|
120
|
-
def race_interleaving(states, init, event_pairs):
|
|
121
|
-
# event_pairs = [("timeout", "ack"), ("error", "done"), ...]
|
|
122
|
-
findings = []
|
|
123
|
-
for e1, e2 in event_pairs:
|
|
124
|
-
# Path 1: e1 then e2
|
|
125
|
-
s1 = step(init, [e1, e2])
|
|
126
|
-
# Path 2: e2 then e1
|
|
127
|
-
s2 = step(init, [e2, e1])
|
|
128
|
-
if s1 != s2:
|
|
129
|
-
findings.append({
|
|
130
|
-
"events": (e1, e2),
|
|
131
|
-
"order_e1_e2_ends_in": s1,
|
|
132
|
-
"order_e2_e1_ends_in": s2,
|
|
133
|
-
"risk": "Order-dependent outcome not documented"
|
|
134
|
-
})
|
|
135
|
-
return findings
|
|
136
|
-
```
|
|
101
|
+
For event pairs that can arrive in the same tick, compare `e1` then `e2` against `e2` then `e1`. A different outcome is order dependence the design must state.
|
|
137
102
|
|
|
138
103
|
### A3: Order Permutation
|
|
139
104
|
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
```python
|
|
143
|
-
from itertools import permutations
|
|
144
|
-
|
|
145
|
-
def order_permutation_probe(states, init, events, invariant_check=None):
|
|
146
|
-
findings = []
|
|
147
|
-
terminals = set()
|
|
148
|
-
for perm in permutations(events):
|
|
149
|
-
final = step(init, list(perm))
|
|
150
|
-
terminals.add(final)
|
|
151
|
-
if invariant_check and not invariant_check(final):
|
|
152
|
-
findings.append({
|
|
153
|
-
"sequence": list(perm),
|
|
154
|
-
"final_state": final,
|
|
155
|
-
"violates": "invariant"
|
|
156
|
-
})
|
|
157
|
-
if len(terminals) > 1:
|
|
158
|
-
findings.append({
|
|
159
|
-
"terminal_states": list(terminals),
|
|
160
|
-
"risk": f"Same events produce {len(terminals)} different outcomes depending on order"
|
|
161
|
-
})
|
|
162
|
-
return findings
|
|
163
|
-
```
|
|
105
|
+
The same question at N events. Up to 5 events permute fully; beyond that the probe samples. More than one terminal state means the outcome depends on arrival order.
|
|
164
106
|
|
|
165
107
|
### A4: Pair Symmetry
|
|
166
108
|
|
|
167
|
-
|
|
168
|
-
# Define paired operations
|
|
169
|
-
PAIRS = [
|
|
170
|
-
("lock", "unlock"),
|
|
171
|
-
("start", "stop"),
|
|
172
|
-
("alloc", "free"),
|
|
173
|
-
("enable_irq", "disable_irq"),
|
|
174
|
-
("open", "close"),
|
|
175
|
-
]
|
|
176
|
-
|
|
177
|
-
def pair_symmetry_probe(states, init):
|
|
178
|
-
findings = []
|
|
179
|
-
for acquire, release in PAIRS:
|
|
180
|
-
if acquire not in all_events(states) and release not in all_events(states):
|
|
181
|
-
continue # This pair type is not used
|
|
182
|
-
# DFS from init: every path that calls acquire must eventually call release
|
|
183
|
-
# before reaching a terminal state (or acquire again)
|
|
184
|
-
unbalanced = find_unbalanced_paths(states, init, acquire, release)
|
|
185
|
-
if unbalanced:
|
|
186
|
-
findings.append({
|
|
187
|
-
"pair": (acquire, release),
|
|
188
|
-
"unbalanced_paths": unbalanced,
|
|
189
|
-
"risk": "Resource leak or deadlock"
|
|
190
|
-
})
|
|
191
|
-
return findings
|
|
192
|
-
```
|
|
109
|
+
For paired operations (lock/unlock, start/stop, alloc/free, enable_irq/disable_irq, open/close), search from the initial state for a path that acquires without releasing before it terminates or acquires again. That is a leak or a deadlock.
|
|
193
110
|
|
|
194
111
|
### A5: Boundary Blast
|
|
195
112
|
|
|
196
|
-
|
|
197
|
-
def boundary_blast(counters, timestamps):
|
|
198
|
-
findings = []
|
|
199
|
-
for name, val in counters.items():
|
|
200
|
-
test_values = [0, 1, val-1, val, val+1, 2**32-1, 2**32]
|
|
201
|
-
for tv in test_values:
|
|
202
|
-
if tv < 0 or tv > val:
|
|
203
|
-
findings.append({
|
|
204
|
-
"variable": name,
|
|
205
|
-
"value": tv,
|
|
206
|
-
"risk": f"Counter overflow/underflow at {tv} (max defined: {val})"
|
|
207
|
-
})
|
|
208
|
-
# Timestamp wraparound (millis() / sys_tick())
|
|
209
|
-
# On 32-bit ARM: wraparound at ~49.7 days for 1ms tick
|
|
210
|
-
for name, val in timestamps.items():
|
|
211
|
-
findings.append({
|
|
212
|
-
"variable": name,
|
|
213
|
-
"risk": f"Timestamp wraparound not handled — check elapsed_ms() / elapsed_ticks() pattern"
|
|
214
|
-
})
|
|
215
|
-
return findings
|
|
216
|
-
```
|
|
113
|
+
Test each counter at 0, 1, max-1, max and max+1. Flag overflow and underflow. Timestamps wrap too: a 32-bit millisecond clock wraps after about 49.7 days, so check every `elapsed_*()` pattern.
|
|
217
114
|
|
|
218
115
|
### A6: Resource Injection
|
|
219
116
|
|
|
220
|
-
|
|
221
|
-
RESOURCE_FAILURES = {
|
|
222
|
-
"malloc": "returns NULL",
|
|
223
|
-
"queue_send": "queue full",
|
|
224
|
-
"semaphore_take": "timeout",
|
|
225
|
-
"message_alloc": "pool exhausted",
|
|
226
|
-
}
|
|
227
|
-
|
|
228
|
-
def resource_injection_probe(states):
|
|
229
|
-
findings = []
|
|
230
|
-
for state, transitions in states.items():
|
|
231
|
-
for event in transitions:
|
|
232
|
-
for resource, failure in RESOURCE_FAILURES.items():
|
|
233
|
-
if resource in event.lower() or resource in state.lower():
|
|
234
|
-
# Simulate: what if this resource call fails in this state?
|
|
235
|
-
# Does the machine have a recovery transition?
|
|
236
|
-
has_recovery = any(
|
|
237
|
-
"error" in t.lower() or "fail" in t.lower() or "retry" in t.lower()
|
|
238
|
-
for t in transitions.values()
|
|
239
|
-
)
|
|
240
|
-
if not has_recovery:
|
|
241
|
-
findings.append({
|
|
242
|
-
"state": state,
|
|
243
|
-
"resource": resource,
|
|
244
|
-
"failure_mode": failure,
|
|
245
|
-
"risk": f"No recovery path if {resource} {failure} in state {state}"
|
|
246
|
-
})
|
|
247
|
-
return findings
|
|
248
|
-
```
|
|
117
|
+
For every state and transition, ask what happens when the resource call fails: `malloc` returns NULL, a queue is full, a semaphore times out, a pool is exhausted. No error, fail or retry transition means no recovery path.
|
|
249
118
|
|
|
250
119
|
### A7: Minimal Counter-Example
|
|
251
120
|
|
|
252
|
-
|
|
253
|
-
from collections import deque
|
|
254
|
-
|
|
255
|
-
def shortest_violating_path(states, init, invariant_check):
|
|
256
|
-
"""BFS to find the shortest event sequence that violates an invariant."""
|
|
257
|
-
queue = deque([(init, [])])
|
|
258
|
-
visited = set()
|
|
259
|
-
while queue:
|
|
260
|
-
state, path = queue.popleft()
|
|
261
|
-
if state in visited:
|
|
262
|
-
continue
|
|
263
|
-
visited.add(state)
|
|
264
|
-
|
|
265
|
-
if not invariant_check(state):
|
|
266
|
-
return path # Shortest violating path found
|
|
267
|
-
|
|
268
|
-
for event, next_state in states.get(state, {}).items():
|
|
269
|
-
queue.append((next_state, path + [event]))
|
|
270
|
-
|
|
271
|
-
return None # Invariant holds for all reachable states
|
|
272
|
-
```
|
|
273
|
-
|
|
274
|
-
That template decides one runtime state at a time, so it only fits the state-predicate kinds. A `leads-to`, `sequence`, `atomicity`, `budget` or `probability` invariant asserts a property of a whole run: reuse the search in its own section (A9-A13) as the witness finder, or A7 will report "all invariants hold" for a model where one of them fails.
|
|
121
|
+
Breadth-first search for the shortest event sequence that violates an invariant. It decides one runtime state at a time, so it fits the state-predicate kinds only. `leads-to`, `sequence`, `atomicity`, `budget` and `probability` assert a property of a whole run and use the search in their own sections (A9-A13).
|
|
275
122
|
|
|
276
123
|
### S8: Monotonic Variables
|
|
277
124
|
|
|
278
|
-
For counters
|
|
279
|
-
|
|
280
|
-
```python
|
|
281
|
-
MONOTONIC_VARS = [
|
|
282
|
-
{"name": "retry_count", "direction": "inc", "increase_events": ["retry"], "decrease_events": []},
|
|
283
|
-
]
|
|
284
|
-
```
|
|
125
|
+
For counters that must move one way only, list the events that change them. An event that moves against the declared direction is a finding.
|
|
285
126
|
|
|
286
127
|
### A8: Idempotent Replay
|
|
287
128
|
|
|
288
|
-
|
|
289
|
-
|
|
290
|
-
```python
|
|
291
|
-
IDEMPOTENT_EVENTS = {"retry", "sync", "webhook_delivery"}
|
|
292
|
-
```
|
|
129
|
+
Apply each retryable event twice from every reachable state. A second application that changes state, or cannot happen, is a finding.
|
|
293
130
|
|
|
294
131
|
### A9: Leads-To
|
|
295
132
|
|
|
296
|
-
For progress
|
|
297
|
-
|
|
298
|
-
```python
|
|
299
|
-
LEADS_TO = [
|
|
300
|
-
("MIGRATING", "DONE"),
|
|
301
|
-
# or a set of acceptable outcomes
|
|
302
|
-
("DRAINING", ["IDLE", "FAULT"]),
|
|
303
|
-
]
|
|
304
|
-
```
|
|
133
|
+
For a progress claim ("from MIGRATING, eventually DONE"), check every path from the source state. A path that dead-ends or loops before the target fails. A target set is still a universal claim: a branch that reaches none of its members fails.
|
|
305
134
|
|
|
306
135
|
### A10: Sequence Order
|
|
307
136
|
|
|
308
|
-
For ordered
|
|
309
|
-
|
|
310
|
-
```python
|
|
311
|
-
SEQUENCES = [
|
|
312
|
-
["backup", "modify", "commit"],
|
|
313
|
-
]
|
|
314
|
-
```
|
|
137
|
+
For an ordered sequence ("backup, then modify, then commit"), search for a path where a later event occurs before an earlier one.
|
|
315
138
|
|
|
316
139
|
### A11: Atomicity
|
|
317
140
|
|
|
318
|
-
For all-or-nothing groups, track whether
|
|
319
|
-
|
|
320
|
-
```python
|
|
321
|
-
ATOMIC_GROUPS = [
|
|
322
|
-
{"events": ["write"], "commit": "commit", "rollback": "rollback"},
|
|
323
|
-
]
|
|
324
|
-
```
|
|
141
|
+
For all-or-nothing groups, track whether the atomic event started and whether commit or rollback followed. Leaving the scope, or terminating, before commit or rollback is a violation.
|
|
325
142
|
|
|
326
143
|
### A12: Budget (Worst-Case Path Cost)
|
|
327
144
|
|
|
328
|
-
|
|
329
|
-
|
|
330
|
-
```python
|
|
331
|
-
TRANSITION_COSTS = { # (from, event): cost; absent entries count as 1
|
|
332
|
-
("BOOT", "calibrate_done"): 40,
|
|
333
|
-
("IDLE", "enable"): 5,
|
|
334
|
-
("RUN", "watchdog_expiry"): 15,
|
|
335
|
-
}
|
|
336
|
-
BUDGETS = [
|
|
337
|
-
{"id": "dispatch-budget", "description": "worst-case path within budget", "budget": 100},
|
|
338
|
-
]
|
|
339
|
-
```
|
|
340
|
-
|
|
341
|
-
Probe: find the shortest path from init whose accumulated cost exceeds the budget; report it verbatim. A reachable cycle with positive total cost means cost can grow without bound — flag it as a budget violation regardless of the declared budget.
|
|
342
|
-
|
|
343
|
-
In DSH (`logicprobe_verify`), `cost` is an optional field on transitions (absent = 1) and `budget` is an invariant kind; A12 runs automatically. Cost values are modeler-provided static labels — they verify the model against the declared budget, not the real WCET, which needs binary-level timing analysis.
|
|
145
|
+
Give each transition a `cost` and declare a `budget`. The probe reports the shortest path whose accumulated cost exceeds it. A reachable cycle with positive cost means cost grows without bound, which is a violation whatever the budget says. Costs are modeler labels: this checks the model against its budget, not real WCET.
|
|
344
146
|
|
|
345
147
|
### A13: Probability Reachability (DTMC)
|
|
346
148
|
|
|
347
|
-
|
|
348
|
-
|
|
349
|
-
```python
|
|
350
|
-
TRANSITION_WEIGHTS = { # (from_state, event): weight; absent entries count as 1
|
|
351
|
-
("RUN", "ok"): 9,
|
|
352
|
-
("RUN", "bad"): 1,
|
|
353
|
-
}
|
|
354
|
-
PROBABILITY_INVARIANTS = [
|
|
355
|
-
{"id": "reliability", "target": "SAFE", "op": ">=", "p": 0.9},
|
|
356
|
-
]
|
|
357
|
-
```
|
|
358
|
-
|
|
359
|
-
Probe: solve P(ever reaching `target`) from INIT by value iteration over the reachable absorbing chain; compare against the bound. In DSH (`logicprobe_verify`), `weight` on transitions plus a `probability` invariant runs A13 automatically. This is a qualitative DTMC check of the modeler's weights — real MTBF/failure-rate numbers need PRISM/Storm or fault-tree analysis.
|
|
149
|
+
Give each branch a relative `weight` and declare a `probability` bound. The probe solves P(ever reaching the target) by value iteration and compares it with the bound. Absent weight is 1; weight 0 never fires. This is a qualitative check of the weights, not a measured failure rate.
|
|
360
150
|
|
|
361
151
|
### A14: Deadline (Discrete Tick Clock)
|
|
362
152
|
|
|
363
|
-
|
|
364
|
-
|
|
365
|
-
```python
|
|
366
|
-
TICK_EVENTS = {"tick"}
|
|
367
|
-
STATE_MAX_TICKS = {"BUSY": 3}
|
|
368
|
-
```
|
|
369
|
-
|
|
370
|
-
Probe: explore residency — a `tick` step that keeps the machine resident past `maxTicks` is a deadline miss; report the over-residency path. In DSH (`logicprobe_verify`), state `maxTicks` plus top-level `tickEvents` run A14 automatically. This checks the model's declared deadlines, not real execution time (see Known Limitations below; hard real-time semantics route to UPPAAL).
|
|
371
|
-
|
|
372
|
-
### Standalone engine (non-DSH, JSON models)
|
|
373
|
-
|
|
374
|
-
`references/logicprobe-engine.py` is an exact Python mirror of the DSH tools: `verify model.json` runs all 22 checks + D1-D4 with a `--before-model`/optional `--state-mapping`; `compose m1.json m2.json ... --rendezvous a,b` runs C1/C2 composition; `export model.json --format uppaal|tla|prism|spin` reproduces the four exporters byte-for-byte. It reads the same LogicModelV1 JSON as DSH, so a model verified in one host verifies identically in the other (checked by tests/python/run.mjs).
|
|
153
|
+
Declare which events advance the clock (`tickEvents`) and cap per-state residency with `maxTicks`. A tick that keeps the machine resident past its cap is a deadline miss, and the probe reports the over-residency path. This checks declared deadlines, not execution time.
|
|
375
154
|
|
|
376
155
|
## Counter-Example Interpretation
|
|
377
156
|
|
|
@@ -409,7 +188,7 @@ When a probe finds a counter-example, classify it:
|
|
|
409
188
|
### What This Method CANNOT Detect
|
|
410
189
|
|
|
411
190
|
- **Implementation bugs**: The C code may have errors not present in the model
|
|
412
|
-
- **Timing-dependent bugs**:
|
|
191
|
+
- **Timing-dependent bugs**: the model does not simulate real-time constraints
|
|
413
192
|
- **Compiler/optimization issues**: Volatile omission, reordering, inlining effects
|
|
414
193
|
- **Hardware-specific behavior**: Memory-mapped I/O timing, DMA races, cache coherency
|
|
415
194
|
- **Undocumented behavior**: If the plan doesn't describe a transition, the model can't either
|
|
@@ -423,7 +202,7 @@ When a probe finds a counter-example, classify it:
|
|
|
423
202
|
|
|
424
203
|
### Model Fidelity Warning
|
|
425
204
|
|
|
426
|
-
The
|
|
205
|
+
The model is an APPROXIMATION. It models state transitions, not execution semantics. A model that passes all 22 checks (S1-S8 structural, A1-A14) means the plan's LOGIC is consistent — NOT that the implementation will work. Always follow logic verification with code-level review.
|
|
427
206
|
|
|
428
207
|
## Refactoring Verification
|
|
429
208
|
|
|
@@ -455,7 +234,7 @@ If the plan claims "simplification" but the numbers don't decrease, flag as UNSU
|
|
|
455
234
|
|
|
456
235
|
**Deadlock regression**: A refactoring that splits one state into two should not introduce a deadlock path that didn't exist before. Run S2 on AFTER and compare to BEFORE's deadlock report.
|
|
457
236
|
|
|
458
|
-
|
|
237
|
+
For before/after comparison, pass the earlier model as `beforeModel` and an optional `stateMapping`; the D1-D4 checks then run after the standard ones.
|
|
459
238
|
|
|
460
239
|
### Common Refactoring Bugs Caught by This Method
|
|
461
240
|
|
|
@@ -466,20 +245,20 @@ In the Python harness, set `BEFORE_STATES`, `BEFORE_INIT`, `BEFORE_TERMINALS`, a
|
|
|
466
245
|
|
|
467
246
|
## Manual Verification Mode
|
|
468
247
|
|
|
469
|
-
When
|
|
248
|
+
When no engine can run (air-gapped machine, locked-down host), execute each check manually. The agent performs the verification using its own reasoning — the methodology is identical, only the execution engine changes.
|
|
470
249
|
|
|
471
250
|
### Size Limit
|
|
472
251
|
|
|
473
|
-
Manual verification is reliable for state machines with **≤ 10 states and ≤ 30 transitions**. For larger machines, manual BFS and cycle detection become error-prone. If the machine exceeds this threshold and
|
|
252
|
+
Manual verification is reliable for state machines with **≤ 10 states and ≤ 30 transitions**. For larger machines, manual BFS and cycle detection become error-prone. If the machine exceeds this threshold and no engine can run, either:
|
|
474
253
|
|
|
475
254
|
- Decompose the machine into sub-machines and verify each independently, then check cross-machine contracts manually
|
|
476
|
-
- Flag the size limitation as a finding and recommend the
|
|
255
|
+
- Flag the size limitation as a finding and recommend running the engine offline from the repository
|
|
477
256
|
|
|
478
257
|
### Prerequisites
|
|
479
258
|
|
|
480
259
|
- Transition table and model narrative have been extracted and confirmed with the user
|
|
481
260
|
- You have the full table in context (from Phase 2 extraction step)
|
|
482
|
-
- **
|
|
261
|
+
- **Model validation**: when a host with an engine becomes available, run the extracted model there and compare its findings with the hand-checked ones.
|
|
483
262
|
|
|
484
263
|
### Phase 2a — Manual Structural Checks
|
|
485
264
|
|
|
@@ -515,7 +294,7 @@ Manual verification is reliable for state machines with **≤ 10 states and ≤
|
|
|
515
294
|
|
|
516
295
|
### Manual Mode Output Format
|
|
517
296
|
|
|
518
|
-
For each finding, output the
|
|
297
|
+
For each finding, output the engine structured format:
|
|
519
298
|
|
|
520
299
|
```text
|
|
521
300
|
[CHECK] S1 Reachability
|
|
@@ -537,9 +316,7 @@ Manual verification takes longer but produces identical-quality findings. The ke
|
|
|
537
316
|
2. Read plan → Phase 1 (enumerate claims)
|
|
538
317
|
3. Detect behavioral claims → trigger logic-primitive escalation
|
|
539
318
|
4. Load this guide
|
|
540
|
-
5.
|
|
541
|
-
6a. Python ≥ 3.6 → load verification-harness.py → fill in MODEL → run
|
|
542
|
-
6b. No Python → use Manual Verification Mode (see above) — execute each check step by step
|
|
319
|
+
5. Run the checks: the native tool when the host has it, otherwise Manual Verification Mode (see above), step by step
|
|
543
320
|
7. Extract model → show transition table → GET USER CONFIRMATION
|
|
544
321
|
8. Run Phase 2a (8 structural primitives) → log results
|
|
545
322
|
9. Run Phase 2b (14 adversarial probes) → log results
|
|
@@ -548,4 +325,4 @@ Manual verification takes longer but produces identical-quality findings. The ke
|
|
|
548
325
|
12. Include in Phase 5 (structured output)
|
|
549
326
|
```
|
|
550
327
|
|
|
551
|
-
Never skip
|
|
328
|
+
Never skip the check. A verified model based on wrong extraction is worse than no verification — it creates false confidence.
|
|
@@ -0,0 +1,43 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: logicprobe-concurrency
|
|
3
|
+
description: "Use when a design or plan claims concurrency safety (thread-safe, lock-free, no data race, ISR-safe) or names shared state, mutexes, atomics or interrupts. Mines the claims and routes each to dedicated verification (TSan, Helgrind, CBMC, TLA+); it does not prove concurrency safety. Doctrine: the logicprobe skill."
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Logic Probe — Concurrency Claims
|
|
7
|
+
|
|
8
|
+
This skill does **not** prove concurrency safety, and no report from it should ever be read as a proof. It mines a document, plan or code review for concurrency claims, marks them `UNVERIFIED`, and routes each one to the tool that can actually decide it.
|
|
9
|
+
|
|
10
|
+
The claim-verification doctrine it inherits — enumerate verifiable claims, cite `file:line`, read `verdict` and never `ok` — lives in the `logicprobe` skill. Load that one when the task is behavioural verification of a state machine rather than a judgement about concurrency claims.
|
|
11
|
+
|
|
12
|
+
## When This Skill Applies
|
|
13
|
+
|
|
14
|
+
**Only after confirming the target actually has concurrency requirements or behaviour** — multiple threads, async tasks, interrupts, shared state, or parallel execution. If the target is purely sequential, do not invoke concurrency mining; there is nothing to mine.
|
|
15
|
+
|
|
16
|
+
| The claim | How it is treated |
|
|
17
|
+
|---|---|
|
|
18
|
+
| Absolute: "thread-safe", "lock-free", "no data race", "interrupt-safe", "ISR-safe" | error / `UNVERIFIED` unless dedicated evidence is provided |
|
|
19
|
+
| Risk keyword: "race condition", "shared variable", "mutex", "atomic", "shared memory", "critical section", `ISR`, `IRQ`, `NMI`, `disable_irq` / `enable_irq` | warning — review whether the plan actually addresses it |
|
|
20
|
+
| Lock/unlock, alloc/free, start/stop pairing where ordering matters | routed to `logicprobe` model verification (A4 pair symmetry), which can decide balance inside the modelled machine |
|
|
21
|
+
|
|
22
|
+
In DSH, run the `logicprobe_concurrency_scan` tool. For manual review, follow [`references/concurrency-risk-guide.md`](references/concurrency-risk-guide.md).
|
|
23
|
+
|
|
24
|
+
## Routing: What Decides Each Claim
|
|
25
|
+
|
|
26
|
+
Mining a claim is not verifying it. Hand each one to a tool that can:
|
|
27
|
+
|
|
28
|
+
| Dimension | Tool that decides it |
|
|
29
|
+
|---|---|
|
|
30
|
+
| Data races, lock-ordering, lock-free algorithms | ThreadSanitizer, Helgrind, or a model checker over the real code (CBMC) |
|
|
31
|
+
| Interrupt/ISR races, critical sections | RTOS-aware interrupt analysis, or a TLA+ model of the ISR/task interleaving |
|
|
32
|
+
| Protocol-level interleaving of discrete events | `logicprobe_verify` (S1-S8 / A1-A14) and `logicprobe_compose_verify` (C1/C2) |
|
|
33
|
+
| Hard real time: deadlines, periods, preemption | Scheduling analysis (RMA/EDF) and a timing tool — not logicprobe |
|
|
34
|
+
| Probability or reliability claims | PRISM/Storm, fault-tree analysis |
|
|
35
|
+
|
|
36
|
+
The full routing table, including hard-real-time, hybrid-control and execution-cost dimensions, is [`../logicprobe/references/gap-routing-guide.md`](../logicprobe/references/gap-routing-guide.md). `logicprobe_verify` reports carry the same routing as informational `coverageNotes` on matching models.
|
|
37
|
+
|
|
38
|
+
## Rules for This Mode
|
|
39
|
+
|
|
40
|
+
1. **Mining, not proof.** Report a concurrency claim as `UNVERIFIED` with the routing target; never as passing.
|
|
41
|
+
2. **No absolute language in the answer.** "This is thread-safe" is exactly the claim being mined, not a result this skill can produce.
|
|
42
|
+
3. **One counterexample is enough to refute.** If a race is suspected, the deliverable is the interleaving that breaks it plus the harness that demonstrates it (TSan/Helgrind output), not an argument.
|
|
43
|
+
4. **Sequential targets are out of scope.** If nothing runs concurrently, say so and stop.
|
|
@@ -53,4 +53,4 @@ Manual checklist:
|
|
|
53
53
|
- General: TLA+, SPIN, Alloy
|
|
54
54
|
- Rust: loom, Shuttle
|
|
55
55
|
|
|
56
|
-
Other semantic dimensions (hard real time, hybrid control, probability/reliability, execution cost) route through the same pattern — see `references/gap-routing-guide.md`.
|
|
56
|
+
Other semantic dimensions (hard real time, hybrid control, probability/reliability, execution cost) route through the same pattern — see the full table in the `logicprobe` skill: `../logicprobe/references/gap-routing-guide.md`.
|
|
@@ -1,11 +1,11 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: logicprobe-datamodel
|
|
3
|
-
description: "Use when reviewing design
|
|
3
|
+
description: "Use when reviewing design docs, specs, plans or migrations claiming entities, fields, constraints, relationships, data invariants, or before/after data-model equivalence: schema changes, migration logic, or data guarantees ('all records must have X', 'no orphan rows', 'ids unique') want executable data-model verification (migration coverage, copy consistency, rollback symmetry, DD1-DD4 regression). State machines and protocols: logicprobe; diagrams: logicprobe-uml."
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# Logic Probe Data
|
|
7
7
|
|
|
8
|
-
|
|
8
|
+
An unverified data model fails later. Model the schema and the migration, run them, and it becomes clear. Check every data claim before you act on it.
|
|
9
9
|
|
|
10
10
|
## Methodology
|
|
11
11
|
|
|
@@ -44,8 +44,7 @@ Escalate immediately if the document contains ANY of:
|
|
|
44
44
|
```text
|
|
45
45
|
Document claims → Extract DataModelV1 → Runtime check:
|
|
46
46
|
├── DSH + `logicprobe_datamodel_verify` tool available → build DataModelV1 → call tool → structured report
|
|
47
|
-
|
|
48
|
-
└── No Python → Manual Verification Mode (references/data-model-guide.md)
|
|
47
|
+
└── No engine available → Manual Verification Mode (references/data-model-guide.md)
|
|
49
48
|
|
|
50
49
|
Refactoring variant:
|
|
51
50
|
Extract BEFORE DataModelV1 + AFTER DataModelV1
|
|
@@ -54,6 +53,8 @@ Refactoring variant:
|
|
|
54
53
|
→ Flag any invariant that held in BEFORE but fails in AFTER
|
|
55
54
|
```
|
|
56
55
|
|
|
56
|
+
The **DataModelV1 shape** is in [`references/data-model-schema.md`](references/data-model-schema.md): entities, fields, keys, relationships, invariants, boundary checks, and the optional before/mapping/copy/migration/backup blocks. Those field names are the tool contract. A hand-written model that disagrees with them is rejected, not guessed at.
|
|
57
|
+
|
|
57
58
|
## Checks
|
|
58
59
|
|
|
59
60
|
### DS — Data structure
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Data Model Manual Verification Mode
|
|
2
2
|
|
|
3
|
-
Use this when
|
|
3
|
+
Use this when the DSH `logicprobe_datamodel_verify` tool is not available. It mirrors the automated DS/DA/DD checks as a structured checklist.
|
|
4
4
|
|
|
5
5
|
## 1. Extract the DataModelV1 table
|
|
6
6
|
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# DataModelV1 — logicprobe-datamodel Schema
|
|
2
2
|
|
|
3
|
-
The `logicprobe_datamodel_verify` tool accepts a structured JSON data model. The engine runs DS/DA checks, and DD1-DD4 when `beforeModel` is supplied.
|
|
3
|
+
The `logicprobe_datamodel_verify` tool accepts a structured JSON data model. The engine runs DS/DA checks, and DD1-DD4 when `beforeModel` is supplied. The schema is host-agnostic: the native tool and the repository engine take the same JSON.
|
|
4
4
|
|
|
5
5
|
## Top-level model
|
|
6
6
|
|