@haaaiawd/skeinn 2.4.0 → 2.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +12 -0
- package/README.md +37 -5
- package/README.zh-CN.md +41 -8
- package/cli/bin/skeinn.js +20 -5
- package/cli/src/protocol.js +72 -7
- package/cli/src/store.js +484 -46
- package/design.md +36 -9
- package/docs/UX_FLOW.md +16 -4
- package/docs/eval/RELAY-001.md +115 -0
- package/package.json +1 -1
package/design.md
CHANGED
|
@@ -120,17 +120,44 @@ authority, risk, or material cost changes return to the conversation.
|
|
|
120
120
|
New Tasks must make design and capability applicability explicit: use `implements` and `capability_hooks`, or
|
|
121
121
|
record a concrete `design_exemption` / `capability_exemption`. Task start rejects an unclassified active horizon.
|
|
122
122
|
Completion verifies that every declared local `touches` path exists, and `skeinn check` detects later filesystem drift.
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
123
|
+
A Task may carry `kind: "spike"` for empirical probes that run during `shaping` (no `implements` required); spikes
|
|
124
|
+
produce measured facts, never deliverables, so a delivery Task cannot `depends_on` a spike — distill its findings
|
|
125
|
+
into Design/Capability or `skeinn record` instead. Keeper findings pin to a `scope` (a Task id, a `.skeinn/` doc
|
|
126
|
+
path, or `project`); `task start` automatically surfaces every open finding pinned to that Task or to the docs in
|
|
127
|
+
its `reads`/`implements`, gating on `blocking` and listing `minor` items as advisories to close with evidence.
|
|
128
|
+
|
|
129
|
+
### Launch Keeper and in-flight checkpoints
|
|
130
|
+
|
|
131
|
+
Keeper runs in two shapes, both built on two primitives: a per-artifact `reviewed_digests` hash map (PROJECT,
|
|
132
|
+
STRUCTURE, DECISIONS, every design doc, every capability file, every Task definition) and a scope-pinned
|
|
133
|
+
`findings` ledger. The **active horizon** — currently executable Tasks plus their 1–2 hop unblocked successors and
|
|
134
|
+
the `.skeinn` documents they read or implement — defines what deserves blocking scrutiny; `dirty` artifacts are
|
|
135
|
+
horizon members whose current digest differs from their last reviewed digest.
|
|
136
|
+
|
|
137
|
+
**Launch** is a one-time birth certificate at the shaping→building transition. A fresh Agent receives no prior
|
|
138
|
+
conversation, runs `skeinn context --keeper`, explains the whole, selects a first Task, navigates its design
|
|
139
|
+
documents and professional capabilities, and identifies concrete evidence. Every attempt binds a unique
|
|
129
140
|
fresh-thread `run_id` to the digest frozen by `project ready`. A passing record also carries an explicit independent
|
|
130
141
|
review attestation (`review.mode`, reviewer identity, and isolation evidence); known self-review cannot be recorded as
|
|
131
|
-
passed. This is an auditable host assertion, not cryptographic identity proof.
|
|
132
|
-
|
|
133
|
-
|
|
142
|
+
passed. This is an auditable host assertion, not cryptographic identity proof. Blocking findings inside the horizon
|
|
143
|
+
return `needs_revision`; minor-only results pass as `passed_with_advisories`, stamping the horizon into
|
|
144
|
+
`reviewed_digests` and pinning each finding to its scope instead of forcing another full round. Revision must change
|
|
145
|
+
project truth or Task definitions before a new digest and fresh Keeper run are allowed — and round 2+ is a
|
|
146
|
+
scope-bounded delta re-check: the Keeper sees only unclosed blocking findings plus the artifacts changed by the
|
|
147
|
+
repair, out-of-scope observations demote to advisories, and two consecutive rounds that fail to reduce blocking
|
|
148
|
+
count halt blind looping (unlockable with `--force-round --reason`). After Launch passes, `launch_completed` stays
|
|
149
|
+
permanent; a genuine direction pivot uses `skeinn keeper reset --reason`, which clears `reviewed_digests` and logs a
|
|
150
|
+
decision.
|
|
151
|
+
|
|
152
|
+
**Checkpoints** are the continuing legitimacy mechanism during `building` — not a recurring full review. They fire
|
|
153
|
+
on `dirty` horizon artifacts (designs edited mid-flight via `decision`, or distant documents whose Tasks just
|
|
154
|
+
entered the horizon and were never reviewed), and are surfaced by `skeinn check` when the set is non-empty; as a
|
|
155
|
+
fallback, completing five delivery Tasks since the last clean review raises a silent-drift watch even when no
|
|
156
|
+
declaration changed. `skeinn keeper checkpoint prompt` hands a fresh Agent only the incremental slice — dirty
|
|
157
|
+
artifacts, open findings, and the recent delivery Tasks whose `touches` should be diffed against spec — for three
|
|
158
|
+
jobs: spec-to-code drift reconcile (fix the code or record `skeinn decision`, never silent), newly-entering horizon
|
|
159
|
+
verification, and evidence/advisory closure. A `clean` verdict commits the horizon into `reviewed_digests` and
|
|
160
|
+
advances the drift watermark; `drift_detected` does not.
|
|
134
161
|
|
|
135
162
|
If the host lacks subagents, Skeinn supplies a one-sentence prompt for a new window. Explicit skip is possible
|
|
136
163
|
but must preserve a concrete reason and reduced confidence.
|
package/docs/UX_FLOW.md
CHANGED
|
@@ -110,14 +110,20 @@ a unique `run_id`, attempt number, and prepared digest.
|
|
|
110
110
|
Keeper tests whether it can explain the whole, navigate every necessary design system, locate distinct professional capabilities, select the correct first Task, find all context,
|
|
111
111
|
name the first concrete action, and reproduce completion evidence.
|
|
112
112
|
|
|
113
|
-
### Passed
|
|
113
|
+
### Passed / passed_with_advisories
|
|
114
114
|
|
|
115
115
|
A pass requires independent review provenance (`review.mode: "independent"`, reviewer identity, evidence) and
|
|
116
116
|
`closure_results` covering every finding still open across all earlier attempts. A changed digest alone never
|
|
117
117
|
closes a finding. When no separate Agent is available, the maintainer records an explicit
|
|
118
118
|
`skeinn keeper skip --reason` instead of simulating a pass.
|
|
119
119
|
|
|
120
|
-
|
|
120
|
+
Blocking findings inside the active horizon return `needs_revision`; minor-only findings pass as
|
|
121
|
+
`passed_with_advisories` — each is pinned to a `scope` (a Task id or `.skeinn/` doc path) and automatically
|
|
122
|
+
surfaces at `skeinn task start` when that Task or the docs it reads/implements are involved, closing via
|
|
123
|
+
`skeinn keeper resolve <id> --evidence`.
|
|
124
|
+
|
|
125
|
+
State becomes `build_ready`. Launch is a one-time birth certificate (`launch_completed`); it never reopens on
|
|
126
|
+
later edits — in-flight checkpoints carry legitimacy forward.
|
|
121
127
|
|
|
122
128
|
### Needs revision or blocked
|
|
123
129
|
|
|
@@ -126,6 +132,8 @@ as the highest-priority uncertainty.
|
|
|
126
132
|
|
|
127
133
|
The main Agent decides where the gap belongs, repairs everything it can, asks the human only for decisions they genuinely own,
|
|
128
134
|
changes the relevant disk truth, runs `skeinn project ready` again, and opens a new Keeper run with a new run_id and digest.
|
|
135
|
+
Round 2+ is a scope-bounded delta re-check: the Keeper sees only unclosed blocking findings plus artifacts changed by the
|
|
136
|
+
repair; two consecutive rounds that fail to reduce the blocking count halt the loop (unlockable via `--force-round --reason`).
|
|
129
137
|
|
|
130
138
|
Skeinn refuses an unchanged revision, a duplicate run_id, a result for the wrong digest, and a result written before ready.
|
|
131
139
|
|
|
@@ -149,8 +157,9 @@ and the Agent delivers the real result, evidence, remaining tradeoffs, and any u
|
|
|
149
157
|
|
|
150
158
|
New user information is first evaluated for impact. A local reversible implementation detail updates the active Task. A change
|
|
151
159
|
to the desired whole updates PROJECT.md and affected design documents, records a superseding decision, revises affected Task definitions, and blocks or reopens
|
|
152
|
-
the active Task as needed.
|
|
153
|
-
|
|
160
|
+
the active Task as needed. Design edits via `decision` and distant docs entering the horizon become `dirty` artifacts surfaced
|
|
161
|
+
by `skeinn check`; `skeinn keeper checkpoint prompt|record` runs a lightweight fresh review on just that slice (drift reconcile,
|
|
162
|
+
new-horizon verification, advisory closure), and five delivery Tasks since the last clean review raise a silent-drift watch.
|
|
154
163
|
|
|
155
164
|
## Recovery matrix
|
|
156
165
|
|
|
@@ -168,6 +177,9 @@ human authority govern later changes.
|
|
|
168
177
|
| Attempt to mark done without evidence | CLI rejects it |
|
|
169
178
|
| Evidence disproves an earlier done state | Reopen with a reason, then repair, re-verify, or block |
|
|
170
179
|
| Host cannot spawn Keeper | Ask the human to open a new window with one short prompt |
|
|
180
|
+
| Design/code drift suspected mid-build | `skeinn keeper checkpoint prompt` — fix code or record `skeinn decision`, never silent drift |
|
|
181
|
+
| Pinned blocking finding at task start | Close it with `skeinn keeper resolve <id> --evidence "<proof>"` |
|
|
182
|
+
| Full-direction pivot | `skeinn keeper reset --reason` clears reviewed digests and reopens Launch |
|
|
171
183
|
|
|
172
184
|
## Flow diagrams
|
|
173
185
|
|
|
@@ -0,0 +1,115 @@
|
|
|
1
|
+
# RELAY-001 Live Run Log — Evil Eval L2 on Skeinn 2.4.0
|
|
2
|
+
|
|
3
|
+
Model under test: DeepSeek-V4.1-Flash-1017 via AiPing (https://aiping.cn/api/v1)
|
|
4
|
+
Harness: opencode-ai 1.19.6 (`opencode run`, fresh session per baton = zero memory)
|
|
5
|
+
Human channel: unavailable (question tool denied in opencode.json)
|
|
6
|
+
Workspaces: D:\eval-run\baseline-workspace, D:\eval-run\skeinn-workspace
|
|
7
|
+
Skeinn state: D:\eval-run\skeinn-state (sidecar, --state-dir outside scored workspace)
|
|
8
|
+
|
|
9
|
+
## Harness incident (recorded for honesty)
|
|
10
|
+
|
|
11
|
+
- Skeinn baton-1 first attempt died after 13 tool calls: opencode auto-rejected
|
|
12
|
+
`external_directory` writes to the sidecar state dir. Fixed by granting
|
|
13
|
+
`external_directory: allow` (+ standard tool allows) in BOTH arms' opencode.json,
|
|
14
|
+
wiping skeinn-state + workspace, and re-running baton 1 clean.
|
|
15
|
+
- Baseline baton-1 had already completed under default perms; it never touched
|
|
16
|
+
external dirs, so material conditions were unaffected. Recorded as 1 harness
|
|
17
|
+
retry for the skeinn arm.
|
|
18
|
+
- WSL/Windows path split: runner passes /mnt/d paths to node fs and D:\ paths to
|
|
19
|
+
the Windows opencode.exe.
|
|
20
|
+
|
|
21
|
+
## Baton telemetry (from opencode step_finish events)
|
|
22
|
+
|
|
23
|
+
| arm | baton | wall_s | input | output | reasoning | tool_calls |
|
|
24
|
+
|-----|-------|--------|-------|--------|-----------|------------|
|
|
25
|
+
| baseline | 1 | 549 | 35,164 | 20,641 | 31,960 | ~46 |
|
|
26
|
+
| baseline | 2 | 820 | 61,506 | 27,373 | 39,728 | ~75 |
|
|
27
|
+
| baseline | 3 | 705 | 63,502 | 15,181 | 35,283 | ~53 |
|
|
28
|
+
| skeinn | 1 | 1,489 | 144,726 | 64,111 | 72,895 | ~114 |
|
|
29
|
+
| skeinn | 2 | 1,108 | 154,221 | 60,067 | 58,033 | ~112 |
|
|
30
|
+
| skeinn | 3 | 1,031 | 143,321 | 16,035 | 28,293 | ~82 |
|
|
31
|
+
| **baseline total** | | **2,074** | **160,172** | **63,195** | **106,971** | **~174** |
|
|
32
|
+
| **skeinn total** | | **3,628** | **442,268** | **140,213** | **159,221** | **~308** |
|
|
33
|
+
|
|
34
|
+
Estimated cost (≈¥0.5/M in, ¥2/M out+reasoning): baseline ~¥0.42, skeinn ~¥0.82.
|
|
35
|
+
Both well under the ¥20 budget cap.
|
|
36
|
+
|
|
37
|
+
## Baton outcomes (from agent self-reports)
|
|
38
|
+
|
|
39
|
+
- baseline-1: single-file index.html + 14 pytest tests + README.
|
|
40
|
+
- baseline-2: absorbed pivot (PBKDF2 200k + AES-256-GCM backup) + added local
|
|
41
|
+
history from README roadmap; 20 tests.
|
|
42
|
+
- baseline-3: edge hardening (IME Enter, dedup, timestamps, 320px overflow);
|
|
43
|
+
43 tests; VERIFICATION.md honest notes.
|
|
44
|
+
- skeinn-1: full shaping (4 design docs, 2 capability files, 4-task map) then
|
|
45
|
+
meeting-retro.html + 32 tests (16 real headless-Chrome) + README + artifacts.
|
|
46
|
+
Keeper recorded skipped (no fresh-agent harness available for the role).
|
|
47
|
+
- skeinn-2: pivot via 3 new tasks; PBKDF2 600k + AAD + 8-char floor, preview-
|
|
48
|
+
confirm-undo import, CSP connect-src 'none'; 43/43 tests; design docs updated.
|
|
49
|
+
- skeinn-3: found+fixed a real mid-session storage-failure persistence bug;
|
|
50
|
+
44 tests (23 real browser); Keeper attempt 1 needs_revision (4 doc gaps) →
|
|
51
|
+
fixed → attempt 2 PASSED; 8/8 tasks, check healthy.
|
|
52
|
+
|
|
53
|
+
## Hidden Oracle (objective assertions, run post-hoc by me)
|
|
54
|
+
|
|
55
|
+
- baseline: 8/8 PASS
|
|
56
|
+
- skeinn: 8/8 PASS
|
|
57
|
+
|
|
58
|
+
## Blind judging
|
|
59
|
+
|
|
60
|
+
- Packets sealed to D:\eval-run\X and D:\eval-run\Y via blind.mjs;
|
|
61
|
+
mapping-sealed.json written, unread until verdicts recorded.
|
|
62
|
+
- Judge pass 1 (X→Y order): fresh subagent, agent_id e70ac5d6
|
|
63
|
+
- Judge pass 2 (Y→X order): fresh subagent, agent_id 57985483
|
|
64
|
+
|
|
65
|
+
## Verdicts (recorded before unsealing)
|
|
66
|
+
|
|
67
|
+
Pass 1 — OVERALL_X_CLEAR_WIN
|
|
68
|
+
1. Intent fidelity: X_CLEAR_WIN (timer persists across refresh; preview+undo import)
|
|
69
|
+
2. Craft depth: X_CLEAR_WIN (AAD-bound GCM, 600k PBKDF2, error taxonomy, quarantine)
|
|
70
|
+
3. Cold-handoff: TIE
|
|
71
|
+
4. Oracle/proof credibility: TIE (8/8 both, honest blind spots both)
|
|
72
|
+
5. User burden: X_CLEAR_WIN (zero-dep node --test vs pytest+Playwright install)
|
|
73
|
+
6. Net value/RMB: Y_CLEAR_WIN (¥0.42 vs ¥0.82 for same oracle result)
|
|
74
|
+
|
|
75
|
+
Pass 2 — OVERALL_X_CLEAR_WIN
|
|
76
|
+
1. Intent fidelity: TIE
|
|
77
|
+
2. Craft depth: X_CLEAR_WIN
|
|
78
|
+
3. Cold-handoff: TIE
|
|
79
|
+
4. Oracle/proof: TIE
|
|
80
|
+
5. User burden: X_CLEAR_WIN
|
|
81
|
+
6. Net value/RMB: Y_CLEAR_WIN
|
|
82
|
+
|
|
83
|
+
No verdict flipped across orderings — all STABLE. Judge 1 flagged dim-2 as
|
|
84
|
+
"borderline but held"; judge 2 independently confirmed it.
|
|
85
|
+
|
|
86
|
+
## Unsealed mapping (read only after both verdicts recorded)
|
|
87
|
+
|
|
88
|
+
X = skeinn Y = baseline → FINAL: SKEINN wins the relay, stably, on craft
|
|
89
|
+
margins; baseline wins on cost efficiency and shipped more features (history).
|
|
90
|
+
|
|
91
|
+
## Judge-noted leakage (honesty)
|
|
92
|
+
|
|
93
|
+
Pass-2 judge noticed oracle.target paths ("skeinn-workspace"/"baseline-workspace")
|
|
94
|
+
inside judge-packet.json and ignored them for scoring. The packet should redact
|
|
95
|
+
target paths in a future harness revision — recorded as a methodology fix, not a
|
|
96
|
+
scoring factor.
|
|
97
|
+
|
|
98
|
+
## Post-hoc: did the .skeinn docs earn their tokens?
|
|
99
|
+
|
|
100
|
+
Judges never saw the state files (sidecar was excluded from packets by design),
|
|
101
|
+
so doc quality was not blind-scored — only its effects were. Post-hoc read:
|
|
102
|
+
|
|
103
|
+
- `capabilities/local-crypto-backup.md` (written by baton 2 under the pivot)
|
|
104
|
+
cites OWASP/MDN/W3C primary sources and, in its Reflection section, predicted
|
|
105
|
+
the export-while-running timer bug BEFORE implementation — that became the
|
|
106
|
+
`draftAtTimestamp` pause-on-export code judge 2 called "a sharp edge-semantic
|
|
107
|
+
call". Capability → design delta → shipped line: traceable.
|
|
108
|
+
- `capabilities/timeboxed-reflection.md` predicted the generic failure mode
|
|
109
|
+
(canonical went-well/went-badly columns) and its design delta produced the
|
|
110
|
+
non-generic prompt set ("哪里没被说透").
|
|
111
|
+
- `design/system.md` lists 8 alternatives including genuinely tempting ones
|
|
112
|
+
(IndexedDB, bundled Argon2id lib, stored password), rejected on real
|
|
113
|
+
constraints; `Applied capabilities` imports rules verbatim from the dossiers.
|
|
114
|
+
- Caveat: `external_researched` source URLs cannot be disk-verified as truly
|
|
115
|
+
fetched vs model recall; content matches the real documents.
|