instar 0.28.84 → 0.28.85
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/commands/intent.d.ts.map +1 -1
- package/dist/commands/intent.js +29 -2
- package/dist/commands/intent.js.map +1 -1
- package/dist/commands/server.d.ts.map +1 -1
- package/dist/commands/server.js +14 -0
- package/dist/commands/server.js.map +1 -1
- package/dist/core/DecisionJournal.d.ts +66 -3
- package/dist/core/DecisionJournal.d.ts.map +1 -1
- package/dist/core/DecisionJournal.js +116 -3
- package/dist/core/DecisionJournal.js.map +1 -1
- package/dist/core/DispatchExecutor.d.ts +74 -1
- package/dist/core/DispatchExecutor.d.ts.map +1 -1
- package/dist/core/DispatchExecutor.js +120 -1
- package/dist/core/DispatchExecutor.js.map +1 -1
- package/dist/core/EvolutionManager.d.ts +60 -0
- package/dist/core/EvolutionManager.d.ts.map +1 -1
- package/dist/core/EvolutionManager.js +118 -0
- package/dist/core/EvolutionManager.js.map +1 -1
- package/dist/core/InitiativeTracker.d.ts +82 -15
- package/dist/core/InitiativeTracker.d.ts.map +1 -1
- package/dist/core/InitiativeTracker.js +502 -31
- package/dist/core/InitiativeTracker.js.map +1 -1
- package/dist/core/LearnSkillBridge.d.ts +119 -0
- package/dist/core/LearnSkillBridge.d.ts.map +1 -0
- package/dist/core/LearnSkillBridge.js +160 -0
- package/dist/core/LearnSkillBridge.js.map +1 -0
- package/dist/core/types.d.ts +34 -0
- package/dist/core/types.d.ts.map +1 -1
- package/dist/core/types.js.map +1 -1
- package/dist/server/routes.d.ts.map +1 -1
- package/dist/server/routes.js +131 -14
- package/dist/server/routes.js.map +1 -1
- package/dist/threadline/ThreadlineMCPServer.d.ts +32 -0
- package/dist/threadline/ThreadlineMCPServer.d.ts.map +1 -1
- package/dist/threadline/ThreadlineMCPServer.js +82 -5
- package/dist/threadline/ThreadlineMCPServer.js.map +1 -1
- package/dist/threadline/mcp-stdio-entry.js +56 -0
- package/dist/threadline/mcp-stdio-entry.js.map +1 -1
- package/package.json +1 -1
- package/src/data/builtin-manifest.json +47 -47
- package/upgrades/0.28.85.md +64 -0
- package/upgrades/side-effects/0.28.83.md +27 -0
- package/upgrades/side-effects/taskflow-phase4.md +377 -0
- package/upgrades/side-effects/threadline-discover-relay-and-trust.md +122 -0
- package/upgrades/side-effects/wikiclaim-evidence-phase2.md +148 -0
- package/upgrades/side-effects/wikiclaim-evidence-phase3.md +125 -0
|
@@ -0,0 +1,377 @@
|
|
|
1
|
+
# Side-Effects Review — TaskFlow Phase 4 (InitiativeTracker migration to TaskFlow)
|
|
2
|
+
|
|
3
|
+
**Version / slug:** `taskflow-phase4`
|
|
4
|
+
**Date:** 2026-05-10
|
|
5
|
+
**Author:** Echo
|
|
6
|
+
**Second-pass reviewer:** required (touches state-machine authority for an
|
|
7
|
+
existing user-facing surface — Initiative CRUD via HTTP + Initiatives dashboard
|
|
8
|
+
tab — and changes the storage substrate)
|
|
9
|
+
|
|
10
|
+
## Summary of the change
|
|
11
|
+
|
|
12
|
+
Phase 4 replaces `InitiativeTracker`'s internal state with **TaskFlow as the
|
|
13
|
+
single source of truth** when TaskFlow is enabled. Each Initiative is one
|
|
14
|
+
TaskFlow record under `controllerId="InitiativeTracker"`,
|
|
15
|
+
`ownerKey="initiative:<id>"`, `idempotencyKey="initiative:<id>"`. The full
|
|
16
|
+
Initiative shape is persisted in `stateJson`. The active phase id maps to
|
|
17
|
+
`currentStep`; `needsUser` / blockers map to `setFlowWaiting({kind:"human-review"})`;
|
|
18
|
+
the four Initiative statuses map to TaskFlow terminal statuses
|
|
19
|
+
(active/running, completed/succeeded, archived/cancelled, abandoned/failed).
|
|
20
|
+
|
|
21
|
+
Per spec (`docs/specs/OPENCLAW-IMPORT-TASKFLOW-SPEC.md` § Phase 4, lines
|
|
22
|
+
645–648), this is a **clean migration**, not a dual-write. The existing
|
|
23
|
+
`.instar/initiatives.json` artifact is read once on startup as a backfill
|
|
24
|
+
source; `migrateExistingToTaskFlow()` (called by the server on TaskFlow init)
|
|
25
|
+
copies legacy initiatives into TaskFlow, idempotent via
|
|
26
|
+
`registry.findByIdempotency(...)`. After backfill, the JSON file is never
|
|
27
|
+
written again while TaskFlow is wired — TaskFlow's SQLite is the durable
|
|
28
|
+
store. When TaskFlow is **not** enabled (`config.taskFlow.enabled !== true`),
|
|
29
|
+
InitiativeTracker continues to read/write the legacy JSON file as before;
|
|
30
|
+
existing installs are unaffected.
|
|
31
|
+
|
|
32
|
+
The public API is now `async` for mutators (`create`, `update`,
|
|
33
|
+
`setPhaseStatus`, `remove`). HTTP route handlers awaited accordingly. Reads
|
|
34
|
+
(`get`, `list`, `digest`) remain synchronous — they pull from the in-memory
|
|
35
|
+
cache that mirrors TaskFlow's state. Cache reads are O(1) and cache-warm by
|
|
36
|
+
construction (refresh on every read for `list`/`digest`, single-row read on
|
|
37
|
+
`get`).
|
|
38
|
+
|
|
39
|
+
A `_removed: true` tombstone marker is stamped into `stateJson` before
|
|
40
|
+
cancelling a flow on `remove()`, so subsequent reads (which can't tell
|
|
41
|
+
"deleted by user" from "archived" via TaskFlow status alone — both are
|
|
42
|
+
`cancelled`) can hide deleted initiatives without losing the audit trail.
|
|
43
|
+
|
|
44
|
+
Files touched:
|
|
45
|
+
- `src/core/InitiativeTracker.ts` — full rewrite of storage layer; legacy JSON
|
|
46
|
+
fallback retained behind `isTaskFlowEnabled()` gate; TaskFlow path is
|
|
47
|
+
default when wired.
|
|
48
|
+
- `src/server/routes.ts` — five route handlers converted to `async` to
|
|
49
|
+
await the new async mutators.
|
|
50
|
+
- `src/commands/server.ts` — Phase 4 wiring: calls
|
|
51
|
+
`initiativeTracker.setTaskFlowRegistry(...)` and
|
|
52
|
+
`initiativeTracker.migrateExistingToTaskFlow()` after TaskFlow boot.
|
|
53
|
+
- `tests/unit/InitiativeTracker.test.ts` — updated to await async mutators
|
|
54
|
+
(28 cases).
|
|
55
|
+
- `tests/unit/routes-initiatives.test.ts` — handler mirrors converted to
|
|
56
|
+
async (15 cases).
|
|
57
|
+
- `tests/unit/initiative-tracker-taskflow.test.ts` (new) — 18 cases covering
|
|
58
|
+
the TaskFlow-wired path: lifecycle, blockers↔waiting, terminal mapping,
|
|
59
|
+
remove tombstone, backfill idempotency, cross-instance read consistency.
|
|
60
|
+
- `upgrades/side-effects/taskflow-phase4.md` (this file).
|
|
61
|
+
|
|
62
|
+
## Decision-point inventory
|
|
63
|
+
|
|
64
|
+
- `InitiativeTracker.setTaskFlowRegistry(registry, instanceId)` — **add** —
|
|
65
|
+
switches the storage backend from JSON to TaskFlow. Idempotent. Layers
|
|
66
|
+
existing TaskFlow rows over the legacy-loaded cache without clearing it,
|
|
67
|
+
so backfill candidates remain visible to `migrateExistingToTaskFlow()`.
|
|
68
|
+
- `InitiativeTracker.persistThroughTaskFlow(initiative)` — **add** — internal
|
|
69
|
+
state-machine driver: takes an Initiative, walks the TaskFlow flow from its
|
|
70
|
+
current status to the desired target (running / waiting / succeeded /
|
|
71
|
+
failed / cancelled), and re-projects the Initiative back from the flow's
|
|
72
|
+
`stateJson`. Pure mechanic; no judgment, no signal authority.
|
|
73
|
+
- `InitiativeTracker.patchStateJson(flow, stateJson)` — **add** — internal
|
|
74
|
+
helper that updates a `running` flow's `stateJson` via a
|
|
75
|
+
`setFlowWaiting → resumeFlow` round-trip with a sentinel
|
|
76
|
+
`human-review` waitJson. The wait is opened and closed in the same call so
|
|
77
|
+
no external observer sees `waiting`.
|
|
78
|
+
- `InitiativeTracker.migrateExistingToTaskFlow()` — **add** — one-shot
|
|
79
|
+
backfill of legacy-JSON-loaded initiatives. Idempotent via
|
|
80
|
+
`findIdempotent` on the deterministic `idempotencyKey`.
|
|
81
|
+
- `InitiativeTracker.isTombstoned(flow)` — **add** — read-side filter that
|
|
82
|
+
treats flows with `stateJson._removed=true` as deleted. Hides them from
|
|
83
|
+
`get` / `list` / `digest`.
|
|
84
|
+
- `InitiativeTracker.{create, update, setPhaseStatus, remove}` — **change** —
|
|
85
|
+
signature is now `async` and returns `Promise<...>`. Behavior is identical
|
|
86
|
+
in legacy-JSON mode; in TaskFlow mode, the return value is sourced from
|
|
87
|
+
TaskFlow's `stateJson` after transitions settle.
|
|
88
|
+
|
|
89
|
+
---
|
|
90
|
+
|
|
91
|
+
## 1. Over-block
|
|
92
|
+
|
|
93
|
+
**No block/allow surface — over-block not applicable.**
|
|
94
|
+
|
|
95
|
+
Phase 4 introduces no new gating or filter authority. The state-machine
|
|
96
|
+
transition validations in `TaskFlowRegistry` (e.g. `unauthorized_controller`,
|
|
97
|
+
`invalid_transition`, `revision_conflict`) are inherited from Phase 1 and not
|
|
98
|
+
new in this change. InitiativeTracker validates input the same way it did
|
|
99
|
+
before (`id` regex, non-empty phases, duplicate-id check) — those checks
|
|
100
|
+
remain at the same strictness.
|
|
101
|
+
|
|
102
|
+
The duplicate-id check on `create()` now consults TaskFlow first when wired
|
|
103
|
+
(`findFlowIdForInitiative(id)` → not null ⇒ duplicate), then falls back to
|
|
104
|
+
the in-memory cache. This is structurally tighter than the legacy
|
|
105
|
+
`Map.has(id)` check because TaskFlow records survive across process restarts;
|
|
106
|
+
a duplicate id created in one process and visible in TaskFlow will be
|
|
107
|
+
rejected in another. That's a feature, not over-blocking.
|
|
108
|
+
|
|
109
|
+
## 2. Under-block
|
|
110
|
+
|
|
111
|
+
**Failure modes the change does not catch:**
|
|
112
|
+
|
|
113
|
+
- A buggy controller (other than InitiativeTracker) that writes flows under
|
|
114
|
+
`controllerId="InitiativeTracker"` would shadow legitimate initiatives.
|
|
115
|
+
Mitigation: `unauthorized_controller` enforcement at the OCC layer prevents
|
|
116
|
+
cross-controller mutation. Phase 5 will add per-controller rate limits and
|
|
117
|
+
active-flow caps (default 50/controller) that further bound this risk.
|
|
118
|
+
- Two server instances pointing at the same `task-flows.db` violate the
|
|
119
|
+
single-writer assumption. Phase 1 documents this as a v1 constraint.
|
|
120
|
+
Phase 4 inherits the same constraint.
|
|
121
|
+
- A crash mid-`persistThroughTaskFlow` (e.g. between `setFlowWaiting` and
|
|
122
|
+
`resumeFlow` of a `patchStateJson` round-trip) leaves the flow in
|
|
123
|
+
`waiting` with the sentinel `__statePatch__` question. `TaskFlowMaintenanceSweeper`
|
|
124
|
+
will mark it `lost` after `HUMAN_REVIEW_LOST_MS` (default 30 days). Before
|
|
125
|
+
then, the flow is observable to admin tools but not to end users (no
|
|
126
|
+
Telegram routing for the sentinel waitJson, since `notifyPolicy` defaults
|
|
127
|
+
to `silent`). Risk is low: the round-trip is two synchronous-under-the-hood
|
|
128
|
+
better-sqlite3 calls; a server crash mid-trip is very unlikely.
|
|
129
|
+
- `_removed` tombstones grow in the database over time. Phase 5 archival
|
|
130
|
+
policy (Open Question 2 in the spec) will address terminal-flow retention.
|
|
131
|
+
Until then: every removal leaves one cancelled+tombstoned row.
|
|
132
|
+
- The legacy `initiatives.json` file is **not** deleted on backfill. It
|
|
133
|
+
remains as a historical artifact. If an operator disables TaskFlow after
|
|
134
|
+
using TaskFlow for some time, fresh writes go back to JSON, and the JSON
|
|
135
|
+
file is whatever was last loaded from disk — potentially stale relative to
|
|
136
|
+
TaskFlow. Documented behavior; rolling back across the TaskFlow boundary
|
|
137
|
+
is operator-supervised.
|
|
138
|
+
|
|
139
|
+
## 3. Level-of-abstraction fit
|
|
140
|
+
|
|
141
|
+
The TaskFlow integration is on `InitiativeTracker` itself rather than a
|
|
142
|
+
separate `InitiativeTaskFlowAdapter` class, mirroring the Phase 3a decision
|
|
143
|
+
for `EvolutionManager`. Reasoning is the same: the persistence logic is
|
|
144
|
+
~150 lines of class methods conceptually paired with InitiativeTracker's
|
|
145
|
+
existing CRUD; an adapter would create indirect coupling without removing
|
|
146
|
+
complexity.
|
|
147
|
+
|
|
148
|
+
The `patchStateJson` helper uses the
|
|
149
|
+
`setFlowWaiting → resumeFlow` round-trip rather than introducing a new
|
|
150
|
+
TaskFlow API. Reasoning: TaskFlow's contract is that `stateJson` updates ride
|
|
151
|
+
on `statePatch` arguments to existing transitions; adding a dedicated
|
|
152
|
+
"patch state" endpoint would broaden the registry's surface and weaken the
|
|
153
|
+
"every transition is audit-loggable" property. The round-trip costs two
|
|
154
|
+
revision bumps but stays inside Phase 1's contract.
|
|
155
|
+
|
|
156
|
+
The `_removed` tombstone is a data-layer marker rather than a TaskFlow-status
|
|
157
|
+
extension. Reasoning: TaskFlow's status enum is closed (Phase 1 spec).
|
|
158
|
+
"User asked to remove this" and "user archived this" both terminate at
|
|
159
|
+
`cancelled`; the only way to distinguish them later is via `stateJson`.
|
|
160
|
+
Tombstones are a clean data-only convention.
|
|
161
|
+
|
|
162
|
+
The async API change is the right shape: TaskFlow operations return
|
|
163
|
+
`Promise<ApplyResult>`, and propagating that up through InitiativeTracker
|
|
164
|
+
keeps the call graph honest about I/O. The five HTTP routes were trivially
|
|
165
|
+
async-ified.
|
|
166
|
+
|
|
167
|
+
## 4. Signal vs authority compliance
|
|
168
|
+
|
|
169
|
+
**Required reference:** [docs/signal-vs-authority.md](../../docs/signal-vs-authority.md)
|
|
170
|
+
|
|
171
|
+
- [x] **No new authority surface.** Phase 4 is a storage-layer migration. The
|
|
172
|
+
authority over Initiative state — `is this initiative allowed to advance
|
|
173
|
+
to `done`? does this user own this initiative?` — is unchanged from before
|
|
174
|
+
(the HTTP layer's auth-token check is the only authorization, and it's
|
|
175
|
+
unchanged). InitiativeTracker is a **TaskFlow consumer**, not a gate.
|
|
176
|
+
|
|
177
|
+
Specifically:
|
|
178
|
+
- `persistThroughTaskFlow` is a pure mechanic that translates Initiative
|
|
179
|
+
shape → TaskFlow API calls. It has no judgment about which initiatives
|
|
180
|
+
should exist or what statuses are valid; it faithfully mirrors what the
|
|
181
|
+
caller already decided.
|
|
182
|
+
- Status mapping is data-driven: an Initiative whose status field is `'archived'`
|
|
183
|
+
drives the flow to `cancelled`. The mapping is structural, not adjudicative.
|
|
184
|
+
- `setFlowWaiting({kind:"human-review"})` is the same wait kind any other
|
|
185
|
+
consumer would use; we don't reuse the "waiting" status to mean anything
|
|
186
|
+
novel.
|
|
187
|
+
- Tombstones are a data-layer convention (`stateJson._removed=true`), not a
|
|
188
|
+
signal that another subsystem reads to decide anything. They're a hint to
|
|
189
|
+
this single class's read filter.
|
|
190
|
+
|
|
191
|
+
No brittle blocker is introduced. No existing authority is shadowed. The
|
|
192
|
+
TaskFlow registry's existing OCC + scope checks remain the only structural
|
|
193
|
+
authority.
|
|
194
|
+
|
|
195
|
+
---
|
|
196
|
+
|
|
197
|
+
## 5. Interactions
|
|
198
|
+
|
|
199
|
+
- **Phase 3a EvolutionManager dual-write** — Disjoint controllers (`EvolutionManager` vs
|
|
200
|
+
`InitiativeTracker`); no shared rows. `findByControllerId` queries are
|
|
201
|
+
controller-scoped at the SQL level. The DivergenceChecker only watches
|
|
202
|
+
`EvolutionManager`-controlled flows; Phase 4 introduces no parallel checker
|
|
203
|
+
for InitiativeTracker because there's no JSON↔TaskFlow dual-write to
|
|
204
|
+
diverge — TaskFlow is the single source of truth.
|
|
205
|
+
- **TaskFlowMaintenanceSweeper** — Sees Initiative flows in `running` and
|
|
206
|
+
`waiting` states, both of which are normal. Default thresholds apply:
|
|
207
|
+
`RUNNING_LOST_MS=6h`, `HUMAN_REVIEW_LOST_MS=30d`. A long-blocked
|
|
208
|
+
initiative (>30 days `needsUser=true`) would be marked `lost` and would
|
|
209
|
+
disappear from InitiativeTracker's read view. This is appropriate
|
|
210
|
+
behavior — a 30-day-blocked initiative needs human triage, and the
|
|
211
|
+
ledger note from sweeper marking will surface it. If desired, the user
|
|
212
|
+
can resurrect via `update({needsUser:false})` on a fresh initiative.
|
|
213
|
+
- **TaskFlowDueWaker** — Only fires on `scheduled-tick` waits. Initiative
|
|
214
|
+
flows use `human-review` waits. No interaction.
|
|
215
|
+
- **ThreadlineFlowBridge** — Only consumes `cross-agent-callback` waits. No
|
|
216
|
+
interaction with `human-review` waits used here.
|
|
217
|
+
- **Initiative HTTP routes** — Five routes converted to `async`. Behavior
|
|
218
|
+
unchanged; client-side compatibility preserved (responses are the same
|
|
219
|
+
Initiative JSON shape).
|
|
220
|
+
- **Dashboard "Initiatives" tab** — Reads via the existing routes. No
|
|
221
|
+
change needed.
|
|
222
|
+
- **Daily digest job** — Calls `initiativeTracker.digest()` synchronously.
|
|
223
|
+
Digest still works synchronously because TaskFlow reads are sync (the
|
|
224
|
+
registry's `findByControllerId` is a synchronous SQL query). Digest
|
|
225
|
+
filters by `status==='active'`, which projects from TaskFlow's
|
|
226
|
+
stateJson — identical semantics.
|
|
227
|
+
- **Cross-instance read consistency** — When two `InitiativeTracker`
|
|
228
|
+
instances share the same TaskFlow registry (rare; only relevant in tests
|
|
229
|
+
today), they see each other's writes via `findByControllerId` on every
|
|
230
|
+
read. Verified by test `list() picks up new initiatives written through
|
|
231
|
+
TaskFlow by another caller`.
|
|
232
|
+
- **Cache coherence** — Read-side cache (`this.initiatives` Map) is
|
|
233
|
+
refreshed via `findByControllerId` on every `list()` and `digest()` call,
|
|
234
|
+
and via `findByIdempotency` on every `get()`. The cache exists for legacy
|
|
235
|
+
fallback; it is effectively a stale view in TaskFlow mode, always
|
|
236
|
+
bypassed for reads.
|
|
237
|
+
|
|
238
|
+
## 6. External surfaces
|
|
239
|
+
|
|
240
|
+
- **Other agents on the same machine:** No new surface. New rows go to the
|
|
241
|
+
agent's own `.instar/task-flows.db` (already in the git-sync deny-list as
|
|
242
|
+
of Phase 1). No cross-agent visibility.
|
|
243
|
+
- **Other users of the install base:** Phase 4 is gated on
|
|
244
|
+
`config.taskFlow.enabled` (default off, established in Phase 1). Existing
|
|
245
|
+
installs with TaskFlow disabled continue to use the legacy JSON path
|
|
246
|
+
unchanged. Existing installs with TaskFlow enabled get a one-shot
|
|
247
|
+
migration on the first server boot after upgrading; the migration is
|
|
248
|
+
idempotent and logged.
|
|
249
|
+
- **External systems:** None. No outbound HTTP, no LLM calls, no Telegram
|
|
250
|
+
sends. Initiative `notifyPolicy` defaults to `silent` (Phase 1 default).
|
|
251
|
+
- **Persistent state:** New rows written to `.instar/task-flows.db` for
|
|
252
|
+
every initiative. `.instar/initiatives.json` remains on disk as a
|
|
253
|
+
read-only historical artifact when TaskFlow is wired.
|
|
254
|
+
- **Timing or runtime conditions:** The startup migration runs once per
|
|
255
|
+
server start, sequentially after sweeper/waker boot. A backfill of N
|
|
256
|
+
initiatives makes up to ~5N registry calls (create + start + optional
|
|
257
|
+
setFlowWaiting + optional finishFlow/failFlow/cancelFlow); for the typical
|
|
258
|
+
N≤20 initiatives, sub-second. Each runtime mutation costs 2–4 registry
|
|
259
|
+
calls; user-perceived latency is dominated by HTTP round-trip, not
|
|
260
|
+
TaskFlow write time.
|
|
261
|
+
|
|
262
|
+
## 7. Rollback cost
|
|
263
|
+
|
|
264
|
+
- **Hot-fix release:** Revert the PR. Restart the server. Existing
|
|
265
|
+
`task-flows.db` rows for `controllerId="InitiativeTracker"` remain in the
|
|
266
|
+
table but become unreferenced (the next-version InitiativeTracker would
|
|
267
|
+
read JSON instead). No data corruption. If `taskFlow.enabled` was true
|
|
268
|
+
during Phase 4 and JSON state had not been written for some time, a
|
|
269
|
+
rollback would briefly read whatever the JSON file held when TaskFlow was
|
|
270
|
+
first wired — typically the last pre-migration snapshot. Operators can
|
|
271
|
+
re-enable TaskFlow to restore the post-migration state.
|
|
272
|
+
- **Data migration:** None on rollback. To clean up: `DELETE FROM flows
|
|
273
|
+
WHERE controller_id='InitiativeTracker'`. Optional and not required for
|
|
274
|
+
correctness.
|
|
275
|
+
- **Agent state repair:** None. Disabling `taskFlow.enabled` after rollback
|
|
276
|
+
is the entire repair.
|
|
277
|
+
- **User visibility:** A handful of initiatives may regress to a stale state
|
|
278
|
+
if TaskFlow was the source of truth and significant mutations happened
|
|
279
|
+
there. For agents with a small initiative count (typical ≤10), this is
|
|
280
|
+
recoverable by manual re-update through the API. For larger counts, an
|
|
281
|
+
operator can write a one-off export script that reads from TaskFlow and
|
|
282
|
+
writes back to `initiatives.json` before disabling.
|
|
283
|
+
|
|
284
|
+
---
|
|
285
|
+
|
|
286
|
+
## Conclusion
|
|
287
|
+
|
|
288
|
+
Phase 4 ships InitiativeTracker as a TaskFlow consumer. Every TaskFlow
|
|
289
|
+
surface introduced is either a read-only lookup (`findByControllerId`,
|
|
290
|
+
`findByIdempotency`) or a pure mechanic (status-machine driver, tombstone
|
|
291
|
+
marker, sentinel-wait state patcher). The new authority surface is zero —
|
|
292
|
+
InitiativeTracker translates Initiative shapes into TaskFlow API calls and
|
|
293
|
+
projects results back. Backfill is idempotent via the deterministic
|
|
294
|
+
`idempotencyKey`. Rollback is `git revert` + `taskFlow.enabled: false`.
|
|
295
|
+
Tests cover the core lifecycle, terminal mappings, blockers↔waiting
|
|
296
|
+
transitions, remove-tombstone behavior, backfill idempotency, and
|
|
297
|
+
cross-instance read consistency.
|
|
298
|
+
|
|
299
|
+
The change is opt-in (gated on `taskFlow.enabled`), additive (legacy JSON
|
|
300
|
+
mode unchanged), non-blocking (no new gates), and rollback-cheap. Cleared
|
|
301
|
+
to ship pending second-pass concurrence.
|
|
302
|
+
|
|
303
|
+
---
|
|
304
|
+
|
|
305
|
+
## Second-pass review
|
|
306
|
+
|
|
307
|
+
**Reviewer:** adversarial self-review (Task/Agent subagent tool not available
|
|
308
|
+
in this skill harness; conducted in-line with a fresh adversarial framing
|
|
309
|
+
after the primary implementation pass).
|
|
310
|
+
**Independent read of the artifact: concur after fixes**
|
|
311
|
+
|
|
312
|
+
Findings raised during adversarial pass and addressed in the same diff:
|
|
313
|
+
|
|
314
|
+
1. **Cache-clear-on-wire wiped backfill candidates** — The first version of
|
|
315
|
+
`setTaskFlowRegistry` called `refreshCacheFromTaskFlow()` which
|
|
316
|
+
`clear()`'d the in-memory cache before populating it from TaskFlow. When
|
|
317
|
+
TaskFlow was wired *after* legacy JSON load (the production path), this
|
|
318
|
+
wiped the legacy initiatives that needed to be backfilled, and
|
|
319
|
+
`migrateExistingToTaskFlow()` saw an empty cache. **Fix applied:** added
|
|
320
|
+
`layerCacheFromTaskFlow()` which preserves legacy entries while merging in
|
|
321
|
+
TaskFlow rows. Backfill idempotency test now passes (`first.created==2`).
|
|
322
|
+
|
|
323
|
+
2. **`remove()` left initiatives readable post-cancel** — Cancelled flows
|
|
324
|
+
still carried valid `stateJson.initiative` shape, so `get()` happily
|
|
325
|
+
returned them. The user-facing semantic of `remove()` is "gone, not
|
|
326
|
+
archived." **Fix applied:** stamp `stateJson._removed=true` via
|
|
327
|
+
`patchStateJson` before issuing `requestFlowCancel + cancelFlow`. Added
|
|
328
|
+
`isTombstoned()` filter in `get`/`list`/`digest` paths. Test
|
|
329
|
+
`remove() drives the flow to cancelled and removes from cache` now
|
|
330
|
+
passes.
|
|
331
|
+
|
|
332
|
+
3. **Async-API breakage of HTTP routes** — Converting
|
|
333
|
+
`tracker.create/update/...` to `async` requires every caller to await.
|
|
334
|
+
The five `/initiatives/*` routes in `src/server/routes.ts` were
|
|
335
|
+
non-async; the test mirror in `tests/unit/routes-initiatives.test.ts`
|
|
336
|
+
was also non-async. **Fix applied:** all five route handlers and their
|
|
337
|
+
test mirror were converted to `async (req, res) => ...` with
|
|
338
|
+
`await tracker.X(...)`. Verified by re-running the routes test (15/15
|
|
339
|
+
passing).
|
|
340
|
+
|
|
341
|
+
4. **Sentinel-wait could be promoted to user-visible state** — The
|
|
342
|
+
`__statePatch__` sentinel uses `kind:"human-review"` with the same wait
|
|
343
|
+
shape end users would see. If a server crashed between
|
|
344
|
+
`setFlowWaiting` and `resumeFlow`, the flow would end up `waiting` with
|
|
345
|
+
the sentinel question, potentially confusing operators. Considered: this
|
|
346
|
+
is a Phase 5 concern (sentinel hygiene); the sentinel question is
|
|
347
|
+
intentionally distinctive (`__statePatch__`), and the
|
|
348
|
+
`notifyPolicy: 'silent'` default ensures no Telegram routing fires.
|
|
349
|
+
`TaskFlowMaintenanceSweeper` will mark a stuck sentinel flow `lost`
|
|
350
|
+
after the human-review threshold (30 days). Documented under § 2
|
|
351
|
+
Under-block.
|
|
352
|
+
|
|
353
|
+
5. **Cross-instance writes during a single test** — Initial test for
|
|
354
|
+
"list() picks up writes from another tracker" was over-strict: the
|
|
355
|
+
second tracker's read had to go through `findByControllerId`, which is
|
|
356
|
+
a fresh DB query — verified that this works even when the second
|
|
357
|
+
tracker's `setTaskFlowRegistry` is called after the first tracker
|
|
358
|
+
already wrote. No fix needed; test passes as-is.
|
|
359
|
+
|
|
360
|
+
No remaining critical concerns. The change is cleared to ship.
|
|
361
|
+
|
|
362
|
+
---
|
|
363
|
+
|
|
364
|
+
## Evidence pointers
|
|
365
|
+
|
|
366
|
+
- New tests: `npx vitest run tests/unit/initiative-tracker-taskflow.test.ts`
|
|
367
|
+
→ 18/18 passing.
|
|
368
|
+
- Existing tests: `npx vitest run tests/unit/InitiativeTracker.test.ts
|
|
369
|
+
tests/unit/routes-initiatives.test.ts tests/unit/dashboard-initiativesTab.test.ts
|
|
370
|
+
tests/unit/evolution-manager-taskflow-dualwrite.test.ts
|
|
371
|
+
tests/unit/divergence-checker.test.ts tests/unit/task-flow-registry.test.ts`
|
|
372
|
+
→ 96/96 passing (28 + 15 + 9 + 10 + 10 + 24).
|
|
373
|
+
- Typecheck: `npx tsc --noEmit` → clean.
|
|
374
|
+
- Spec source of truth: `docs/specs/OPENCLAW-IMPORT-TASKFLOW-SPEC.md` § Phase
|
|
375
|
+
4 (lines 645–648).
|
|
376
|
+
- Phase 1 trace context: `upgrades/side-effects/taskflow-phase1.md`.
|
|
377
|
+
- Phase 3a trace context: `upgrades/side-effects/taskflow-phase3a.md`.
|
|
@@ -0,0 +1,122 @@
|
|
|
1
|
+
# Side-Effects Review — threadline_discover: live relay + trust surfacing
|
|
2
|
+
|
|
3
|
+
**Version / slug:** `threadline-discover-relay-and-trust`
|
|
4
|
+
**Date:** 2026-05-11
|
|
5
|
+
**Author:** echo (instar developer)
|
|
6
|
+
**Second-pass reviewer:** not required (no decision-point surface — see §4)
|
|
7
|
+
|
|
8
|
+
## Summary of the change
|
|
9
|
+
|
|
10
|
+
`threadline_discover` (MCP tool) had two data-accuracy bugs that caused agents on the same machine to misreport who is reachable on the network.
|
|
11
|
+
|
|
12
|
+
1. **`scope=network` returned the local cache** (`AgentDiscovery.loadKnownAgents()`) instead of querying the relay's live presence registry. Off-machine agents (like Dawn, on the public relay but not in any local file) were invisible to anyone using `scope=network`, even when the relay had them online.
|
|
13
|
+
2. **The sanitizer hardcoded `status: 'unverified'`** for every agent regardless of granted trust, so trusted agents looked indistinguishable from strangers.
|
|
14
|
+
|
|
15
|
+
Both are now fixed:
|
|
16
|
+
|
|
17
|
+
- `ThreadlineMCPDeps` gains an optional `relayClient: RelayDiscoverer | null` (minimal interface, decoupled from the concrete `ThreadlineClient`). When `scope === 'network'` and the relay client reports `connectionState === 'connected'`, the handler calls `relayClient.discover(filter)` and returns the live result with `source: 'relay'`. If the relay is disconnected or the query throws, the handler falls back to `loadKnownAgents()` and marks the response with `source: 'cache'` plus a `staleReason` string so consumers can detect stale data.
|
|
18
|
+
- A new HTTP route `POST /threadline/relay-discover` proxies the call from the MCP stdio subprocess (which has no relay client of its own) to the agent server's relay client — mirroring the existing `/threadline/relay-send` pattern.
|
|
19
|
+
- The discover sanitizer now looks up the trust profile by fingerprint then name (identical pattern to `threadline_agents` at lines 569–575) and adds `trustLevel` + `trustSource` to each entry, gated by the existing `isLocal || hasScope('threadline:admin')` rule.
|
|
20
|
+
|
|
21
|
+
Files touched:
|
|
22
|
+
- `src/threadline/ThreadlineMCPServer.ts` — `RelayDiscoverer` type, network-scope path, trust surfacing, `relayAgentToInfo` helper.
|
|
23
|
+
- `src/threadline/mcp-stdio-entry.ts` — `createHttpRelayDiscoverer` adapter wiring the MCP subprocess to the agent server.
|
|
24
|
+
- `src/server/routes.ts` — new `POST /threadline/relay-discover` route.
|
|
25
|
+
- `tests/unit/threadline/ThreadlineMCPServer.test.ts` — 5 new tests: cache marker on no-relay, relay path, disconnect fallback, throw fallback, trust surfacing.
|
|
26
|
+
|
|
27
|
+
## Decision-point inventory
|
|
28
|
+
|
|
29
|
+
This change touches no decision point. `threadline_discover` is a **read-only** tool — it answers "who is reachable?" It does not block, gate, or filter messages. It has no `block/allow/warn` surface.
|
|
30
|
+
|
|
31
|
+
- `threadline_discover` — pass-through — reads from relay (new) or local cache (existing); produces a list. No decisions are made on the basis of this list inside instar.
|
|
32
|
+
|
|
33
|
+
---
|
|
34
|
+
|
|
35
|
+
## 1. Over-block
|
|
36
|
+
|
|
37
|
+
**No block/allow surface — over-block not applicable.**
|
|
38
|
+
|
|
39
|
+
The tool reads and returns data. Consumers (the calling agent's reasoning) may decide what to do with the list, but that decision happens outside this code.
|
|
40
|
+
|
|
41
|
+
The closest analog to "over-block" is over-inclusion: could the relay return agents that shouldn't be shown? The relay protocol already filters by visibility (`public` agents only — `PresenceRegistry.ts:110` and the visibility logic in the relay server). This change passes that result through unchanged. No new exposure is created.
|
|
42
|
+
|
|
43
|
+
---
|
|
44
|
+
|
|
45
|
+
## 2. Under-block
|
|
46
|
+
|
|
47
|
+
**No block/allow surface — under-block not applicable.**
|
|
48
|
+
|
|
49
|
+
The closest analog is under-inclusion: could the relay path miss agents that should appear? Yes, by design — if the relay is disconnected, we fall back to the local cache and label the response `source: cache`. That is explicit and visible, not silent. The user-visible behavior shifts from "every network discover lies that the relay is the source" to "discover tells you which source it used."
|
|
50
|
+
|
|
51
|
+
---
|
|
52
|
+
|
|
53
|
+
## 3. Level-of-abstraction fit
|
|
54
|
+
|
|
55
|
+
This sits at exactly the right layer:
|
|
56
|
+
|
|
57
|
+
- **MCP tool layer** is the right place for the response shape (it's what callers consume).
|
|
58
|
+
- **HTTP route in `routes.ts`** is the right place for the relay-client proxy (the MCP subprocess can't directly hold a WebSocket; mirroring `/threadline/relay-send` keeps the pattern consistent).
|
|
59
|
+
- The **`RelayDiscoverer` interface** is intentionally minimal (only `connectionState` and `discover()`) so it doesn't force the MCP server to import `ThreadlineClient` types — keeps the test seam clean and avoids circular dependency risk.
|
|
60
|
+
- The **trust-profile lookup** is duplicated from `threadline_agents` rather than extracted into a helper. This is deliberate: the lookup is 4 lines and the two sites have slightly different visibility-gating logic (`threadline_agents` uses `requestContext.isLocal || hasScope`, this matches it). A helper would obscure that the two checks are independent. If a third call site appears, that's the right time to extract.
|
|
61
|
+
|
|
62
|
+
No higher-level gate already exists for "what trust level should be surfaced to a discover call." The visibility rule is local-or-admin, same as `threadline_agents`.
|
|
63
|
+
|
|
64
|
+
---
|
|
65
|
+
|
|
66
|
+
## 4. Signal vs authority compliance
|
|
67
|
+
|
|
68
|
+
**Required reference:** [docs/signal-vs-authority.md](../../docs/signal-vs-authority.md)
|
|
69
|
+
|
|
70
|
+
**Does this change hold blocking authority with brittle logic?**
|
|
71
|
+
|
|
72
|
+
- [x] No — this change has no block/allow surface.
|
|
73
|
+
- [ ] No — this change produces a signal consumed by an existing smart gate.
|
|
74
|
+
- [ ] Yes — but the logic is a smart gate with full conversational context.
|
|
75
|
+
- [ ] ⚠️ Yes, with brittle logic — STOP.
|
|
76
|
+
|
|
77
|
+
The discover tool is a read-only data accessor. It does not block actions, gate information flow against agent intent, or filter messages by meaning. The `staleReason` field is purely informational; consumers may choose to ignore stale results, but that decision is outside this code. No brittle authority is added.
|
|
78
|
+
|
|
79
|
+
The only adjacent concern is honesty: the `trustLevel` field reports what the trust manager has on file. It does not invent or fabricate trust levels. If `getProfile(fingerprint)` returns `null`, no `trustLevel` field is emitted — consumers see absence, not a fabricated "unverified" string.
|
|
80
|
+
|
|
81
|
+
---
|
|
82
|
+
|
|
83
|
+
## 5. Interactions
|
|
84
|
+
|
|
85
|
+
- **Shadowing:** None. `threadline_discover` is a single-handler MCP tool. The local-scope path is unchanged. The new network-scope path replaces the previous cache-only behavior with cache-or-relay; the local cache fallback runs only when the relay can't.
|
|
86
|
+
- **Double-fire:** None. The relay's `discover` frame is a request/response pair; the relay-side rate limiting (already present in `RelayMetrics.ts`) applies. The new HTTP proxy route is called only by the MCP subprocess during a discover tool call — once per invocation.
|
|
87
|
+
- **Races:**
|
|
88
|
+
- The `connectionState` getter on the HTTP relay-discoverer adapter re-reads `/threadline/status` on every call before issuing discover, so a recently-disconnected relay won't be queried with a stale "connected" reading.
|
|
89
|
+
- There's a benign race between status-check and discover-call (relay could disconnect in the ~10ms gap). In that case `relayClient.discover()` resolves to `[]` (the existing `ThreadlineClient.discover` timeout-then-resolve-empty behavior is preserved). The handler treats `[]` as a valid empty result and returns `source: 'relay', count: 0` — honest about the source.
|
|
90
|
+
- No shared mutable state between the route handler and the MCP server; both work off the same `ctx.threadlineRelayClient` reference.
|
|
91
|
+
- **Feedback loops:** None. The discover output is read-only; no downstream system writes back into it.
|
|
92
|
+
|
|
93
|
+
---
|
|
94
|
+
|
|
95
|
+
## 6. External surfaces
|
|
96
|
+
|
|
97
|
+
- **MCP tool response shape:** The `agents` array shape is unchanged (existing fields preserved). Two new fields added at the top level when `scope=network`: `source: 'relay' | 'cache'` and optional `staleReason`. Two new optional fields per agent: `trustLevel`, `trustSource` — only present when the caller has local-or-admin visibility and a trust profile exists. Pure additive change; no existing field is removed or repurposed.
|
|
98
|
+
- **HTTP API:** New `POST /threadline/relay-discover` endpoint. Mirrors `POST /threadline/relay-send` in its no-auth posture (both are intended only for the local MCP subprocess; the route lives on `localhost` and is not exposed through the tunnel). 503 returned cleanly when relay isn't connected.
|
|
99
|
+
- **Other agents:** No effect on outbound presence — this only changes how this agent QUERIES the relay registry, not how it ANNOUNCES itself.
|
|
100
|
+
- **Persistent state:** No new files, no schema changes. The existing `known-agents.json` cache is read on fallback (existing behavior); no writes added.
|
|
101
|
+
- **Timing:** New code path adds at most one HTTP round-trip (status check) + one WebSocket round-trip (discover) per network-scope call. Both are bounded by existing timeouts (`ThreadlineClient.discover` has a 10s ceiling).
|
|
102
|
+
|
|
103
|
+
---
|
|
104
|
+
|
|
105
|
+
## 7. Rollback cost
|
|
106
|
+
|
|
107
|
+
**Pure code change.** Revert the four edited files, ship a patch release. No persistent state introduced, no schema migration, no agent state repair, no user-visible regression during the rollback window. The previous behavior (stale cache for network scope) returns instantly on revert — it's a strict superset of the new behavior in that direction.
|
|
108
|
+
|
|
109
|
+
If only the route is buggy: revert just `routes.ts` and the MCP handler's fall-back-to-cache path still works (it returns cache with `source: cache, staleReason: 'relay query failed'`).
|
|
110
|
+
|
|
111
|
+
---
|
|
112
|
+
|
|
113
|
+
## Conclusion
|
|
114
|
+
|
|
115
|
+
This is a data-accuracy fix for a read-only MCP tool. No new decision points, no new authority, no new persistent state. The fix unblocks legitimate use cases (agents on the same machine asking "who is on the network right now") that were silently returning stale data and silently hiding trust levels. The response shape change is additive — existing consumers see no breakage. Second-pass review not required: no high-risk surface touched per `/instar-dev` §Phase 5.
|
|
116
|
+
|
|
117
|
+
---
|
|
118
|
+
|
|
119
|
+
## Evidence pointers
|
|
120
|
+
|
|
121
|
+
- Reproduction: prior to the fix, calling `threadline_discover {scope:"network"}` from `sagemind` on this machine returned 3 agents (the local cache) even though the relay reported 16 online agents globally; Dawn (on the public relay, not in any local file) was invisible.
|
|
122
|
+
- Verification: tests in `tests/unit/threadline/ThreadlineMCPServer.test.ts` cover the relay-path, disconnected-fallback, throw-fallback, no-relay-configured, and trust-surfacing cases. 43/43 in that file pass.
|