@parall/agent-core 1.44.0 → 1.46.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/channel-capability.d.ts +2 -0
- package/dist/channel-capability.d.ts.map +1 -1
- package/dist/channel-capability.js +15 -0
- package/dist/dispatch-adapter.d.ts +9 -0
- package/dist/dispatch-adapter.d.ts.map +1 -1
- package/dist/event-format.d.ts.map +1 -1
- package/dist/event-format.js +27 -7
- package/dist/fork-session-finalizer.d.ts +65 -0
- package/dist/fork-session-finalizer.d.ts.map +1 -0
- package/dist/fork-session-finalizer.js +70 -0
- package/dist/gateway-base.d.ts +55 -0
- package/dist/gateway-base.d.ts.map +1 -1
- package/dist/gateway-base.js +640 -263
- package/dist/gateway-lane-flow.d.ts +75 -5
- package/dist/gateway-lane-flow.d.ts.map +1 -1
- package/dist/gateway-lane-flow.js +240 -18
- package/dist/http-keepalive.d.ts +4 -0
- package/dist/http-keepalive.d.ts.map +1 -0
- package/dist/http-keepalive.js +33 -0
- package/dist/index.d.ts +2 -1
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +7 -1
- package/dist/lane-ledger.d.ts +8 -0
- package/dist/lane-ledger.d.ts.map +1 -1
- package/dist/lane-ledger.js +14 -0
- package/dist/session-lifecycle.d.ts +198 -0
- package/dist/session-lifecycle.d.ts.map +1 -0
- package/dist/session-lifecycle.js +446 -0
- package/dist/skills/parall-clips.d.ts +1 -1
- package/dist/skills/parall-clips.d.ts.map +1 -1
- package/dist/skills/parall-clips.js +3 -0
- package/dist/skills/parall-schedules.d.ts +1 -1
- package/dist/skills/parall-schedules.d.ts.map +1 -1
- package/dist/skills/parall-schedules.js +1 -1
- package/dist/skills/parall-tasks.d.ts +1 -1
- package/dist/skills/parall-tasks.d.ts.map +1 -1
- package/dist/skills/parall-tasks.js +21 -5
- package/dist/step-persister.d.ts +66 -0
- package/dist/step-persister.d.ts.map +1 -0
- package/dist/step-persister.js +116 -0
- package/dist/step-retry-queue.d.ts +91 -0
- package/dist/step-retry-queue.d.ts.map +1 -0
- package/dist/step-retry-queue.js +259 -0
- package/dist/types.d.ts +1 -1
- package/dist/types.d.ts.map +1 -1
- package/package.json +3 -2
- package/src/channel-capability.ts +16 -0
- package/src/dispatch-adapter.ts +10 -0
- package/src/event-format.ts +27 -7
- package/src/fork-session-finalizer.ts +122 -0
- package/src/gateway-base.ts +747 -331
- package/src/gateway-lane-flow.ts +275 -18
- package/src/http-keepalive.ts +36 -0
- package/src/index.ts +7 -1
- package/src/lane-ledger.ts +14 -0
- package/src/session-lifecycle.ts +552 -0
- package/src/skills/parall-clips.ts +3 -0
- package/src/skills/parall-schedules.ts +1 -1
- package/src/skills/parall-tasks.ts +21 -5
- package/src/step-persister.ts +161 -0
- package/src/step-retry-queue.ts +296 -0
- package/src/types.ts +2 -1
|
@@ -0,0 +1,552 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* SessionLifecycleCoordinator — the single entry point for session status
|
|
3
|
+
* writes (active / idle / closed).
|
|
4
|
+
*
|
|
5
|
+
* SCOPE — this is a **process-local mitigation**, not a distributed fence.
|
|
6
|
+
* It serializes and orders the lifecycle writes THIS process issues. It does
|
|
7
|
+
* NOT order across processes, and it cannot fix a write that times out
|
|
8
|
+
* client-side but lands server-side later, nor an old turn's step retry
|
|
9
|
+
* landing during a new turn. The structural fix is a server-side turn fence
|
|
10
|
+
* (design tracked in #1879 — durable step outbox + turn fence); until that
|
|
11
|
+
* ships, this closes the in-process interleavings that produced the
|
|
12
|
+
* 2026-07-13 repro.
|
|
13
|
+
*
|
|
14
|
+
* The gateway used to fire active and idle PATCHes independently
|
|
15
|
+
* (fire-and-forget + detached retries). Two independent HTTP requests have
|
|
16
|
+
* no ordering: turn 1's in-flight idle could land AFTER turn 2's active and
|
|
17
|
+
* demote a running session — an epoch check on retry TIMERS cannot cancel a
|
|
18
|
+
* request already on the wire. The coordinator closes that class by being a
|
|
19
|
+
* serialized desired-state reconciler:
|
|
20
|
+
*
|
|
21
|
+
* - Per session it stores only the LATEST desired state plus a monotonic
|
|
22
|
+
* generation (bumped by beginTurn and closeSession).
|
|
23
|
+
* - At most ONE lifecycle write is in flight per session; when it settles
|
|
24
|
+
* the loop re-reads the latest desired state and reconciles — so a
|
|
25
|
+
* begin(turn N+1), or a closeSession, that arrives while an earlier write
|
|
26
|
+
* is in flight simply waits for it to settle and is written AFTER it.
|
|
27
|
+
* - finishTurn carries the TurnHandle from its beginTurn; a stale handle
|
|
28
|
+
* (generation superseded) is ignored outright.
|
|
29
|
+
* - Retries re-run the reconcile loop against the LATEST desired state —
|
|
30
|
+
* they never replay a captured command, so a pending idle retry is
|
|
31
|
+
* naturally superseded by a new turn.
|
|
32
|
+
* - Errors are CLASSIFIED (ports, defaults are conservative): transient →
|
|
33
|
+
* bounded backoff; permanent (4xx) → warn + abandon that desired state (a
|
|
34
|
+
* later turn retries with a fresh generation); session-stale (the server
|
|
35
|
+
* considers the session terminal) → drop the session, never write again.
|
|
36
|
+
* - beginTurn / closeSession resolve once one write attempt covering their
|
|
37
|
+
* generation has SETTLED (success or failure — bounded by the write's own
|
|
38
|
+
* timeout, plus at most one in-flight predecessor). The gateway awaits
|
|
39
|
+
* beginTurn before persisting the turn's first AgentStep, so a reused idle
|
|
40
|
+
* session is reconciled to active before steps arrive and the server's
|
|
41
|
+
* presence guard cannot swallow the turn's activity. A failed active write
|
|
42
|
+
* warns and lets the step flow continue.
|
|
43
|
+
* - closeSession's first settle carries a STRUCTURED outcome (`closed` /
|
|
44
|
+
* `stale` / `retrying` / `permanent_failure` / `disposed`) plus a
|
|
45
|
+
* `terminal` promise that resolves when the close reaches a terminal fate
|
|
46
|
+
* — a caller holding cleanup behind the close (ForkSessionFinalizer) can
|
|
47
|
+
* keep ownership through `retrying` instead of mistaking "one attempt
|
|
48
|
+
* settled" for "close completed".
|
|
49
|
+
* - Entries are RECLAIMED (no unbounded Map growth on long-lived daemons):
|
|
50
|
+
* dropSession removes the entry immediately when idle, or at the end of an
|
|
51
|
+
* in-flight write; a `closed` that landed OR failed permanently reclaims
|
|
52
|
+
* too — the permanent failure is not hidden by the reclaim, it is reported
|
|
53
|
+
* through the close's outcome/terminal before the entry goes away.
|
|
54
|
+
* - dispose() is absorbing: every later public command is a no-op that never
|
|
55
|
+
* re-creates an entry, issues a write, or arms a timer.
|
|
56
|
+
*
|
|
57
|
+
* The write port is infrastructure (ParallClient.updateAgentSession);
|
|
58
|
+
* retries, generations, error classification and the state machine live here.
|
|
59
|
+
*/
|
|
60
|
+
|
|
61
|
+
export type SessionStatusPayload =
|
|
62
|
+
| { status: 'active'; trigger_message_id?: string }
|
|
63
|
+
| { status: 'idle' }
|
|
64
|
+
| { status: 'closed' };
|
|
65
|
+
|
|
66
|
+
export type TurnHandle = { sessionId: string; generation: number };
|
|
67
|
+
|
|
68
|
+
/**
|
|
69
|
+
* Terminal fate of a closeSession: `closed` — the write landed; `stale` —
|
|
70
|
+
* the server already considers the session terminal (equivalent outcome for
|
|
71
|
+
* the caller); `permanent_failure` — a 4xx or retry exhaustion; the close
|
|
72
|
+
* will never be written and the server-side session may be left non-closed;
|
|
73
|
+
* `disposed` — the coordinator shut down before the close settled.
|
|
74
|
+
*/
|
|
75
|
+
export type SessionCloseOutcome = 'closed' | 'stale' | 'permanent_failure' | 'disposed';
|
|
76
|
+
|
|
77
|
+
/** First-settle view: `retrying` means a transient failure settled the first
|
|
78
|
+
* bounded attempt and background reconciliation continues — await `terminal`
|
|
79
|
+
* for the eventual fate. */
|
|
80
|
+
export type SessionCloseSettle = SessionCloseOutcome | 'retrying';
|
|
81
|
+
|
|
82
|
+
export type SessionCloseResult = {
|
|
83
|
+
/** Outcome of the first bounded settle (one write attempt covering the close). */
|
|
84
|
+
outcome: SessionCloseSettle;
|
|
85
|
+
/**
|
|
86
|
+
* Resolves when the close reaches a terminal state. Already resolved (to
|
|
87
|
+
* `outcome`) unless `outcome === 'retrying'`.
|
|
88
|
+
*/
|
|
89
|
+
terminal: Promise<SessionCloseOutcome>;
|
|
90
|
+
};
|
|
91
|
+
|
|
92
|
+
/** How one write attempt settled — the value first-settle waiters receive. */
|
|
93
|
+
type WriteSettle = 'ok' | 'retry' | 'permanent' | 'stale' | 'dropped' | 'disposed';
|
|
94
|
+
|
|
95
|
+
export type SessionLifecycleScheduler = {
|
|
96
|
+
setTimeout(fn: () => void, ms: number): unknown;
|
|
97
|
+
clearTimeout(timer: unknown): void;
|
|
98
|
+
};
|
|
99
|
+
|
|
100
|
+
export type SessionLifecycleOpts = {
|
|
101
|
+
write: (sessionId: string, payload: SessionStatusPayload) => Promise<void>;
|
|
102
|
+
/**
|
|
103
|
+
* Transient failure worth retrying (timeout / network / 5xx). Anything
|
|
104
|
+
* else is treated as permanent: warn + abandon (no retry loop against a
|
|
105
|
+
* 400/401/403/404). Default: nothing is retryable — callers that want
|
|
106
|
+
* retries must classify.
|
|
107
|
+
*/
|
|
108
|
+
isRetryable?: (err: unknown) => boolean;
|
|
109
|
+
/**
|
|
110
|
+
* The server considers this session terminal (409 SESSION_NOT_LIVE /
|
|
111
|
+
* INVALID_TRANSITION). The session is dropped — no further lifecycle
|
|
112
|
+
* writes, ever. Default: never stale.
|
|
113
|
+
*/
|
|
114
|
+
isSessionStale?: (err: unknown) => boolean;
|
|
115
|
+
log?: { warn: (msg: string) => void };
|
|
116
|
+
/** Test port; default wraps global timers and unrefs them. */
|
|
117
|
+
scheduler?: SessionLifecycleScheduler;
|
|
118
|
+
/** Reconcile retry backoff. Default 5s→15s→45s→135s→405s (~10 min). */
|
|
119
|
+
retryDelaysMs?: number[];
|
|
120
|
+
};
|
|
121
|
+
|
|
122
|
+
type DesiredState = 'active' | 'idle' | 'closed';
|
|
123
|
+
|
|
124
|
+
type StateStamp = { state: DesiredState; generation: number };
|
|
125
|
+
|
|
126
|
+
type Entry = {
|
|
127
|
+
desired: DesiredState;
|
|
128
|
+
generation: number;
|
|
129
|
+
triggerMessageId?: string;
|
|
130
|
+
writing: boolean;
|
|
131
|
+
/** Last successfully written state. */
|
|
132
|
+
written?: StateStamp;
|
|
133
|
+
/** Permanently failed state — not retried until the desired state moves. */
|
|
134
|
+
abandoned?: StateStamp;
|
|
135
|
+
retryTimer?: unknown;
|
|
136
|
+
retryAttempt: number;
|
|
137
|
+
waiters: Array<{ generation: number; resolve: (settle: WriteSettle) => void }>;
|
|
138
|
+
/** closeSession terminal-outcome waiters — resolved exactly once each. */
|
|
139
|
+
closeWaiters: Array<{ generation: number; resolve: (outcome: SessionCloseOutcome) => void }>;
|
|
140
|
+
dropped: boolean;
|
|
141
|
+
};
|
|
142
|
+
|
|
143
|
+
const DEFAULT_RETRY_DELAYS_MS = [5_000, 15_000, 45_000, 135_000, 405_000];
|
|
144
|
+
|
|
145
|
+
const defaultScheduler: SessionLifecycleScheduler = {
|
|
146
|
+
setTimeout(fn, ms) {
|
|
147
|
+
const timer = setTimeout(fn, ms);
|
|
148
|
+
(timer as NodeJS.Timeout).unref?.();
|
|
149
|
+
return timer;
|
|
150
|
+
},
|
|
151
|
+
clearTimeout(timer) {
|
|
152
|
+
clearTimeout(timer as NodeJS.Timeout);
|
|
153
|
+
},
|
|
154
|
+
};
|
|
155
|
+
|
|
156
|
+
export class SessionLifecycleCoordinator {
|
|
157
|
+
private readonly sessions = new Map<string, Entry>();
|
|
158
|
+
private readonly scheduler: SessionLifecycleScheduler;
|
|
159
|
+
private readonly retryDelays: number[];
|
|
160
|
+
// Generations are coordinator-global and strictly increasing, so a handle
|
|
161
|
+
// from before an entry was reclaimed can never match a REBUILT entry for
|
|
162
|
+
// the same session id (per-entry counters restart at 1 — an ABA hazard
|
|
163
|
+
// where a stale finish would demote a fresh turn).
|
|
164
|
+
private nextGeneration = 0;
|
|
165
|
+
private disposed = false;
|
|
166
|
+
|
|
167
|
+
constructor(private readonly opts: SessionLifecycleOpts) {
|
|
168
|
+
this.scheduler = opts.scheduler ?? defaultScheduler;
|
|
169
|
+
this.retryDelays = opts.retryDelaysMs?.length ? opts.retryDelaysMs : DEFAULT_RETRY_DELAYS_MS;
|
|
170
|
+
}
|
|
171
|
+
|
|
172
|
+
/** Sessions currently tracked — reclaimed entries are gone (leak probe). */
|
|
173
|
+
trackedSessions(): number {
|
|
174
|
+
return this.sessions.size;
|
|
175
|
+
}
|
|
176
|
+
|
|
177
|
+
/**
|
|
178
|
+
* Declare a new turn. Resolves once one write attempt covering this turn's
|
|
179
|
+
* generation has settled (bounded: at most one in-flight predecessor plus
|
|
180
|
+
* one active write, each bounded by the write port's own timeout). Always
|
|
181
|
+
* resolves — a failed active write is warned and reconciled in the
|
|
182
|
+
* background, never blocks the caller.
|
|
183
|
+
*/
|
|
184
|
+
async beginTurn(sessionId: string, triggerMessageId?: string): Promise<TurnHandle> {
|
|
185
|
+
// Dispose is absorbing: never touch the map (an upsert here would
|
|
186
|
+
// re-create an entry nothing will ever reclaim), never write.
|
|
187
|
+
// Generation 0 can't match any live entry, so the handle is inert.
|
|
188
|
+
if (this.disposed) return { sessionId, generation: 0 };
|
|
189
|
+
const entry = this.upsert(sessionId);
|
|
190
|
+
entry.desired = 'active';
|
|
191
|
+
entry.triggerMessageId = triggerMessageId;
|
|
192
|
+
const generation = entry.generation;
|
|
193
|
+
const settled = this.waitFor(entry, generation);
|
|
194
|
+
this.pump(sessionId);
|
|
195
|
+
await settled;
|
|
196
|
+
return { sessionId, generation };
|
|
197
|
+
}
|
|
198
|
+
|
|
199
|
+
/**
|
|
200
|
+
* Declare the turn finished. A stale handle (a newer turn — or a close —
|
|
201
|
+
* has bumped this session's generation) is ignored: finish(turn N) can
|
|
202
|
+
* never override begin(turn N+1). Reconciliation runs detached.
|
|
203
|
+
*/
|
|
204
|
+
finishTurn(handle: TurnHandle): void {
|
|
205
|
+
if (this.disposed) return;
|
|
206
|
+
const entry = this.sessions.get(handle.sessionId);
|
|
207
|
+
if (!entry || entry.dropped || entry.generation !== handle.generation) return;
|
|
208
|
+
if (entry.desired === 'closed') return;
|
|
209
|
+
entry.desired = 'idle';
|
|
210
|
+
entry.retryAttempt = 0;
|
|
211
|
+
this.cancelRetry(entry);
|
|
212
|
+
this.pump(handle.sessionId);
|
|
213
|
+
}
|
|
214
|
+
|
|
215
|
+
/**
|
|
216
|
+
* Terminal transition (fork teardown). Serialized behind any in-flight
|
|
217
|
+
* write — a `closed` can never race the turn's idle — and reclaimed once
|
|
218
|
+
* settled. Resolves after one bounded settle with a STRUCTURED outcome:
|
|
219
|
+
* `closed` / `stale` / `permanent_failure` are terminal; `retrying` means
|
|
220
|
+
* background reconciliation continues — the caller keeps ownership of the
|
|
221
|
+
* session until `terminal` resolves. A close is never silently "done":
|
|
222
|
+
* every fate is observable.
|
|
223
|
+
*/
|
|
224
|
+
async closeSession(sessionId: string): Promise<SessionCloseResult> {
|
|
225
|
+
if (this.disposed) {
|
|
226
|
+
return { outcome: 'disposed', terminal: Promise.resolve('disposed') };
|
|
227
|
+
}
|
|
228
|
+
const entry = this.upsert(sessionId);
|
|
229
|
+
entry.desired = 'closed';
|
|
230
|
+
entry.triggerMessageId = undefined;
|
|
231
|
+
const generation = entry.generation;
|
|
232
|
+
const terminal = new Promise<SessionCloseOutcome>((resolve) => {
|
|
233
|
+
entry.closeWaiters.push({ generation, resolve });
|
|
234
|
+
});
|
|
235
|
+
const settled = this.waitFor(entry, generation);
|
|
236
|
+
this.pump(sessionId);
|
|
237
|
+
const settle = await settled;
|
|
238
|
+
return { outcome: closeSettleFor(settle), terminal };
|
|
239
|
+
}
|
|
240
|
+
|
|
241
|
+
/**
|
|
242
|
+
* Session is dead (stale / superseded) — no further lifecycle writes, and
|
|
243
|
+
* the entry is reclaimed (immediately when idle; at the end of an in-flight
|
|
244
|
+
* write otherwise, unless a new turn revives it in the meantime).
|
|
245
|
+
*/
|
|
246
|
+
dropSession(sessionId: string): void {
|
|
247
|
+
const entry = this.sessions.get(sessionId);
|
|
248
|
+
if (!entry) return;
|
|
249
|
+
entry.dropped = true;
|
|
250
|
+
this.cancelRetry(entry);
|
|
251
|
+
this.resolveWaiters(entry, Number.POSITIVE_INFINITY, 'dropped');
|
|
252
|
+
this.reclaim(sessionId, entry);
|
|
253
|
+
}
|
|
254
|
+
|
|
255
|
+
/** Every tracked session is dead (server closed them all — new_session). */
|
|
256
|
+
dropAllSessions(): void {
|
|
257
|
+
for (const sessionId of [...this.sessions.keys()]) this.dropSession(sessionId);
|
|
258
|
+
}
|
|
259
|
+
|
|
260
|
+
/**
|
|
261
|
+
* One best-effort reconcile pass over unsettled sessions within a hard
|
|
262
|
+
* deadline (shutdown). Returns the number still unsettled.
|
|
263
|
+
*/
|
|
264
|
+
async flush(deadlineMs: number): Promise<number> {
|
|
265
|
+
const deadline = Date.now() + deadlineMs;
|
|
266
|
+
for (const [sessionId, entry] of [...this.sessions.entries()]) {
|
|
267
|
+
if (this.disposed) break;
|
|
268
|
+
if (entry.dropped || this.isSettled(entry)) continue;
|
|
269
|
+
this.cancelRetry(entry);
|
|
270
|
+
entry.retryAttempt = 0;
|
|
271
|
+
const timeLeft = deadline - Date.now();
|
|
272
|
+
if (timeLeft <= 0) break;
|
|
273
|
+
const settled = this.waitFor(entry, entry.generation);
|
|
274
|
+
this.pump(sessionId);
|
|
275
|
+
await raceWithDeadline(settled, timeLeft);
|
|
276
|
+
}
|
|
277
|
+
let remaining = 0;
|
|
278
|
+
for (const entry of this.sessions.values()) {
|
|
279
|
+
if (!entry.dropped && !this.isSettled(entry)) remaining += 1;
|
|
280
|
+
}
|
|
281
|
+
return remaining;
|
|
282
|
+
}
|
|
283
|
+
|
|
284
|
+
dispose(): void {
|
|
285
|
+
this.disposed = true;
|
|
286
|
+
for (const entry of this.sessions.values()) {
|
|
287
|
+
this.cancelRetry(entry);
|
|
288
|
+
this.resolveWaiters(entry, Number.POSITIVE_INFINITY, 'disposed');
|
|
289
|
+
this.settleCloseTerminal(entry, 'disposed');
|
|
290
|
+
}
|
|
291
|
+
this.sessions.clear();
|
|
292
|
+
}
|
|
293
|
+
|
|
294
|
+
// --- internals ----------------------------------------------------------
|
|
295
|
+
|
|
296
|
+
/**
|
|
297
|
+
* Get-or-create the entry and take the next generation. A dropped entry is
|
|
298
|
+
* REUSED (not replaced): its `writing` flag keeps the single-writer chain
|
|
299
|
+
* intact if a pre-drop write is still on the wire, and the generation stays
|
|
300
|
+
* monotonic across the drop boundary.
|
|
301
|
+
*/
|
|
302
|
+
private upsert(sessionId: string): Entry {
|
|
303
|
+
let entry = this.sessions.get(sessionId);
|
|
304
|
+
if (!entry) {
|
|
305
|
+
entry = {
|
|
306
|
+
desired: 'active',
|
|
307
|
+
generation: 0,
|
|
308
|
+
writing: false,
|
|
309
|
+
retryAttempt: 0,
|
|
310
|
+
waiters: [],
|
|
311
|
+
closeWaiters: [],
|
|
312
|
+
dropped: false,
|
|
313
|
+
};
|
|
314
|
+
this.sessions.set(sessionId, entry);
|
|
315
|
+
}
|
|
316
|
+
entry.dropped = false;
|
|
317
|
+
this.nextGeneration += 1;
|
|
318
|
+
entry.generation = this.nextGeneration;
|
|
319
|
+
entry.retryAttempt = 0;
|
|
320
|
+
this.cancelRetry(entry);
|
|
321
|
+
return entry;
|
|
322
|
+
}
|
|
323
|
+
|
|
324
|
+
private waitFor(entry: Entry, generation: number): Promise<WriteSettle> {
|
|
325
|
+
return new Promise<WriteSettle>((resolve) => {
|
|
326
|
+
entry.waiters.push({ generation, resolve });
|
|
327
|
+
});
|
|
328
|
+
}
|
|
329
|
+
|
|
330
|
+
private isReconciled(entry: Entry): boolean {
|
|
331
|
+
return (
|
|
332
|
+
entry.written !== undefined &&
|
|
333
|
+
entry.written.state === entry.desired &&
|
|
334
|
+
entry.written.generation === entry.generation
|
|
335
|
+
);
|
|
336
|
+
}
|
|
337
|
+
|
|
338
|
+
private isAbandoned(entry: Entry): boolean {
|
|
339
|
+
return (
|
|
340
|
+
entry.abandoned !== undefined &&
|
|
341
|
+
entry.abandoned.state === entry.desired &&
|
|
342
|
+
entry.abandoned.generation === entry.generation
|
|
343
|
+
);
|
|
344
|
+
}
|
|
345
|
+
|
|
346
|
+
/** Nothing more to do for the current desired state (written or given up). */
|
|
347
|
+
private isSettled(entry: Entry): boolean {
|
|
348
|
+
return this.isReconciled(entry) || this.isAbandoned(entry);
|
|
349
|
+
}
|
|
350
|
+
|
|
351
|
+
private pump(sessionId: string): void {
|
|
352
|
+
const entry = this.sessions.get(sessionId);
|
|
353
|
+
if (!entry || entry.writing || entry.dropped || this.disposed) return;
|
|
354
|
+
if (this.isSettled(entry)) {
|
|
355
|
+
this.resolveWaiters(entry, entry.generation, this.isReconciled(entry) ? 'ok' : 'permanent');
|
|
356
|
+
this.reclaim(sessionId, entry);
|
|
357
|
+
return;
|
|
358
|
+
}
|
|
359
|
+
entry.writing = true;
|
|
360
|
+
void this.writeLoop(sessionId, entry);
|
|
361
|
+
}
|
|
362
|
+
|
|
363
|
+
private async writeLoop(sessionId: string, entry: Entry): Promise<void> {
|
|
364
|
+
try {
|
|
365
|
+
while (!entry.dropped && !this.disposed && !this.isSettled(entry)) {
|
|
366
|
+
const snapshot: StateStamp & { trigger?: string } = {
|
|
367
|
+
state: entry.desired,
|
|
368
|
+
generation: entry.generation,
|
|
369
|
+
trigger: entry.triggerMessageId,
|
|
370
|
+
};
|
|
371
|
+
let outcome: 'ok' | 'retry' | 'permanent' | 'stale' = 'ok';
|
|
372
|
+
try {
|
|
373
|
+
await this.opts.write(sessionId, payloadFor(snapshot));
|
|
374
|
+
entry.written = { state: snapshot.state, generation: snapshot.generation };
|
|
375
|
+
entry.retryAttempt = 0;
|
|
376
|
+
if (snapshot.state === 'closed') {
|
|
377
|
+
this.settleCloseTerminal(entry, 'closed', snapshot.generation);
|
|
378
|
+
}
|
|
379
|
+
} catch (err) {
|
|
380
|
+
outcome = this.classify(err);
|
|
381
|
+
this.opts.log?.warn(
|
|
382
|
+
`session ${sessionId} ${snapshot.state} write failed (${outcome}, gen ${snapshot.generation}): ${String(err)}`,
|
|
383
|
+
);
|
|
384
|
+
}
|
|
385
|
+
// A settle — success or failure — resolves every waiter this
|
|
386
|
+
// snapshot covers (the bounded beginTurn / closeSession contract).
|
|
387
|
+
this.resolveWaiters(entry, snapshot.generation, outcome);
|
|
388
|
+
|
|
389
|
+
if (outcome === 'stale') {
|
|
390
|
+
// Terminal server-side: never write to this session again.
|
|
391
|
+
entry.dropped = true;
|
|
392
|
+
this.cancelRetry(entry);
|
|
393
|
+
this.settleCloseTerminal(entry, 'stale');
|
|
394
|
+
this.resolveWaiters(entry, Number.POSITIVE_INFINITY, 'stale');
|
|
395
|
+
return;
|
|
396
|
+
}
|
|
397
|
+
const desiredUnchanged =
|
|
398
|
+
snapshot.generation === entry.generation && snapshot.state === entry.desired;
|
|
399
|
+
if (outcome === 'permanent' && desiredUnchanged) {
|
|
400
|
+
// No retry loop against a 4xx — a later turn (fresh generation)
|
|
401
|
+
// un-abandons this session automatically.
|
|
402
|
+
entry.abandoned = { state: snapshot.state, generation: snapshot.generation };
|
|
403
|
+
if (snapshot.state === 'closed') {
|
|
404
|
+
this.settleCloseTerminal(entry, 'permanent_failure', snapshot.generation);
|
|
405
|
+
}
|
|
406
|
+
return;
|
|
407
|
+
}
|
|
408
|
+
if (outcome === 'retry' && desiredUnchanged && !entry.dropped && !this.disposed) {
|
|
409
|
+
// The retry re-enters pump and replays the LATEST desired state,
|
|
410
|
+
// never this snapshot.
|
|
411
|
+
this.scheduleRetry(sessionId, entry);
|
|
412
|
+
return;
|
|
413
|
+
}
|
|
414
|
+
// Success, or the desired state advanced while writing: loop and
|
|
415
|
+
// reconcile the fresh state immediately.
|
|
416
|
+
}
|
|
417
|
+
} finally {
|
|
418
|
+
entry.writing = false;
|
|
419
|
+
this.reclaim(sessionId, entry);
|
|
420
|
+
}
|
|
421
|
+
}
|
|
422
|
+
|
|
423
|
+
private classify(err: unknown): 'retry' | 'permanent' | 'stale' {
|
|
424
|
+
if (this.opts.isSessionStale?.(err)) return 'stale';
|
|
425
|
+
if (this.opts.isRetryable?.(err)) return 'retry';
|
|
426
|
+
return 'permanent';
|
|
427
|
+
}
|
|
428
|
+
|
|
429
|
+
/**
|
|
430
|
+
* Drop the entry when it can never write again: dropped (and no write on
|
|
431
|
+
* the wire), a terminal `closed` that landed, or a `closed` that failed
|
|
432
|
+
* permanently (its fate has been reported via closeSession's outcome — an
|
|
433
|
+
* entry kept around would be an unowned leak, not a recovery path). Called
|
|
434
|
+
* after every write settles and from dropSession — this is what keeps the
|
|
435
|
+
* Map bounded on a long-lived daemon churning fork sessions.
|
|
436
|
+
*/
|
|
437
|
+
private reclaim(sessionId: string, entry: Entry): void {
|
|
438
|
+
if (this.sessions.get(sessionId) !== entry) return;
|
|
439
|
+
if (entry.writing) return; // the write's finally will re-run this
|
|
440
|
+
const closedLanded =
|
|
441
|
+
entry.desired === 'closed' &&
|
|
442
|
+
entry.written?.state === 'closed' &&
|
|
443
|
+
entry.written.generation === entry.generation;
|
|
444
|
+
const closedAbandoned =
|
|
445
|
+
entry.desired === 'closed' &&
|
|
446
|
+
entry.abandoned?.state === 'closed' &&
|
|
447
|
+
entry.abandoned.generation === entry.generation;
|
|
448
|
+
if (!entry.dropped && !closedLanded && !closedAbandoned) return;
|
|
449
|
+
this.cancelRetry(entry);
|
|
450
|
+
this.settleCloseTerminal(
|
|
451
|
+
entry,
|
|
452
|
+
closedLanded ? 'closed' : entry.dropped ? 'stale' : 'permanent_failure',
|
|
453
|
+
);
|
|
454
|
+
this.resolveWaiters(
|
|
455
|
+
entry,
|
|
456
|
+
Number.POSITIVE_INFINITY,
|
|
457
|
+
closedLanded ? 'ok' : entry.dropped ? 'dropped' : 'permanent',
|
|
458
|
+
);
|
|
459
|
+
this.sessions.delete(sessionId);
|
|
460
|
+
}
|
|
461
|
+
|
|
462
|
+
private scheduleRetry(sessionId: string, entry: Entry): void {
|
|
463
|
+
if (entry.retryTimer !== undefined) return;
|
|
464
|
+
if (entry.retryAttempt >= this.retryDelays.length) {
|
|
465
|
+
this.opts.log?.warn(
|
|
466
|
+
`session ${sessionId} lifecycle reconcile giving up after ${entry.retryAttempt} retries (desired=${entry.desired})`,
|
|
467
|
+
);
|
|
468
|
+
entry.abandoned = { state: entry.desired, generation: entry.generation };
|
|
469
|
+
if (entry.desired === 'closed') {
|
|
470
|
+
this.settleCloseTerminal(entry, 'permanent_failure');
|
|
471
|
+
}
|
|
472
|
+
return;
|
|
473
|
+
}
|
|
474
|
+
const delay = this.retryDelays[entry.retryAttempt] ?? 5_000;
|
|
475
|
+
entry.retryAttempt += 1;
|
|
476
|
+
entry.retryTimer = this.scheduler.setTimeout(() => {
|
|
477
|
+
entry.retryTimer = undefined;
|
|
478
|
+
this.pump(sessionId);
|
|
479
|
+
}, delay);
|
|
480
|
+
}
|
|
481
|
+
|
|
482
|
+
private cancelRetry(entry: Entry): void {
|
|
483
|
+
if (entry.retryTimer !== undefined) {
|
|
484
|
+
this.scheduler.clearTimeout(entry.retryTimer);
|
|
485
|
+
entry.retryTimer = undefined;
|
|
486
|
+
}
|
|
487
|
+
}
|
|
488
|
+
|
|
489
|
+
private resolveWaiters(entry: Entry, upToGeneration: number, settle: WriteSettle): void {
|
|
490
|
+
if (entry.waiters.length === 0) return;
|
|
491
|
+
const remaining: Entry['waiters'] = [];
|
|
492
|
+
for (const waiter of entry.waiters) {
|
|
493
|
+
if (waiter.generation <= upToGeneration) waiter.resolve(settle);
|
|
494
|
+
else remaining.push(waiter);
|
|
495
|
+
}
|
|
496
|
+
entry.waiters = remaining;
|
|
497
|
+
}
|
|
498
|
+
|
|
499
|
+
/** Resolve closeSession terminal waiters covered by this settle, once. */
|
|
500
|
+
private settleCloseTerminal(
|
|
501
|
+
entry: Entry,
|
|
502
|
+
outcome: SessionCloseOutcome,
|
|
503
|
+
upToGeneration = Number.POSITIVE_INFINITY,
|
|
504
|
+
): void {
|
|
505
|
+
if (entry.closeWaiters.length === 0) return;
|
|
506
|
+
const remaining: Entry['closeWaiters'] = [];
|
|
507
|
+
for (const waiter of entry.closeWaiters) {
|
|
508
|
+
if (waiter.generation <= upToGeneration) waiter.resolve(outcome);
|
|
509
|
+
else remaining.push(waiter);
|
|
510
|
+
}
|
|
511
|
+
entry.closeWaiters = remaining;
|
|
512
|
+
}
|
|
513
|
+
}
|
|
514
|
+
|
|
515
|
+
/** Map a first-settle write outcome onto the closeSession vocabulary. */
|
|
516
|
+
function closeSettleFor(settle: WriteSettle): SessionCloseSettle {
|
|
517
|
+
switch (settle) {
|
|
518
|
+
case 'ok':
|
|
519
|
+
return 'closed';
|
|
520
|
+
case 'retry':
|
|
521
|
+
return 'retrying';
|
|
522
|
+
case 'stale':
|
|
523
|
+
case 'dropped':
|
|
524
|
+
return 'stale';
|
|
525
|
+
case 'disposed':
|
|
526
|
+
return 'disposed';
|
|
527
|
+
default:
|
|
528
|
+
return 'permanent_failure';
|
|
529
|
+
}
|
|
530
|
+
}
|
|
531
|
+
|
|
532
|
+
function payloadFor(snapshot: StateStamp & { trigger?: string }): SessionStatusPayload {
|
|
533
|
+
if (snapshot.state === 'active') {
|
|
534
|
+
return {
|
|
535
|
+
status: 'active',
|
|
536
|
+
...(snapshot.trigger ? { trigger_message_id: snapshot.trigger } : {}),
|
|
537
|
+
};
|
|
538
|
+
}
|
|
539
|
+
return snapshot.state === 'idle' ? { status: 'idle' } : { status: 'closed' };
|
|
540
|
+
}
|
|
541
|
+
|
|
542
|
+
async function raceWithDeadline<T>(work: Promise<T>, ms: number): Promise<T | 'timeout'> {
|
|
543
|
+
let timer: NodeJS.Timeout | undefined;
|
|
544
|
+
const timeout = new Promise<'timeout'>((resolve) => {
|
|
545
|
+
timer = setTimeout(() => resolve('timeout'), Math.max(0, ms));
|
|
546
|
+
});
|
|
547
|
+
try {
|
|
548
|
+
return await Promise.race([work, timeout]);
|
|
549
|
+
} finally {
|
|
550
|
+
clearTimeout(timer);
|
|
551
|
+
}
|
|
552
|
+
}
|
|
@@ -26,6 +26,9 @@ Results are JSON on stdout; failures print an error.
|
|
|
26
26
|
clip's owner or an admin to bind it — do not retry or work around it.
|
|
27
27
|
- If the executing runtime is offline or the call times out, report that
|
|
28
28
|
plainly; do not queue, and never fabricate a result for a run that errored.
|
|
29
|
+
- Hosted browser activation is handled by the CLI: it waits (bounded) while a
|
|
30
|
+
cold hosted browser starts, so if the invoke still fails, report the error —
|
|
31
|
+
do not blind-retry in a loop.
|
|
29
32
|
- A clip may act through a person's real logged-in account — outward,
|
|
30
33
|
irreversible, or spending actions (post, order, delete, pay) get the same
|
|
31
34
|
caution as any shared-state change: confirm when intent isn't explicit.
|
|
@@ -35,7 +35,7 @@ parall schedules create \\
|
|
|
35
35
|
--run-at <FUTURE_RFC3339_TIME>
|
|
36
36
|
\`\`\`
|
|
37
37
|
|
|
38
|
-
\`--target-ids\` is who receives the fire (usually yourself when you're self-scheduling; another agent or human when delegating). \`--attached-to-uri\` optionally anchors the schedule to a task / chat / project / wiki page — when that resource is archived or deleted, the schedule auto-cancels (\`
|
|
38
|
+
\`--target-ids\` is who receives the fire (usually yourself when you're self-scheduling; another agent or human when delegating). \`--attached-to-uri\` optionally anchors the schedule to a task / chat / project / wiki page — when that resource is archived or deleted, the schedule auto-cancels (\`status_reason=attached_gone\`). A schedule whose agent targets all sit on a terminated machine is auto-paused by the platform (\`status_reason=attendee_unreachable\`) instead of firing into a void; resuming a recurring schedule while the machine is still terminated just pauses it again on the next slot (a one-shot resumed past its catch-up window instead follows the normal missed semantics and completes).
|
|
39
39
|
|
|
40
40
|
### Reminders for someone else
|
|
41
41
|
|
|
@@ -34,16 +34,32 @@ parall tasks list --assignee-id prll://usr_xxx # first page only (default 20)
|
|
|
34
34
|
parall tasks subtasks prll://tsk_xxx # children of a single parent task
|
|
35
35
|
|
|
36
36
|
# Create a task (add --parent-id to make it a SUBTASK of another task)
|
|
37
|
-
parall tasks create --title "Task title" [--assignee-id prll://usr_xxx] [--parent-id prll://tsk_xxx] [--project-id prll://prj_xxx]
|
|
37
|
+
parall tasks create --title "Task title" [--assignee-id prll://usr_xxx] [--parent-id prll://tsk_xxx] [--project-id prll://prj_xxx] [--due-date 2026-08-01]
|
|
38
38
|
|
|
39
|
-
# Update task status
|
|
40
|
-
|
|
41
|
-
parall tasks update prll://tsk_xxx --status
|
|
39
|
+
# Update task status — add --placement end so the task lands at the end of
|
|
40
|
+
# its NEW status column (a bare --status keeps the old column's sort_order)
|
|
41
|
+
parall tasks update prll://tsk_xxx --status in_progress --placement end
|
|
42
|
+
parall tasks update prll://tsk_xxx --status done --placement end
|
|
43
|
+
|
|
44
|
+
# Due date — a plain YYYY-MM-DD date (no timestamps); "none" clears it
|
|
45
|
+
parall tasks update prll://tsk_xxx --due-date 2026-08-01
|
|
46
|
+
parall tasks update prll://tsk_xxx --due-date none
|
|
47
|
+
|
|
48
|
+
# Move a task to the end of its status column
|
|
49
|
+
parall tasks update prll://tsk_xxx --placement end
|
|
42
50
|
|
|
43
51
|
# Add a comment
|
|
44
52
|
parall tasks comments add prll://tsk_xxx --body "Progress update..."
|
|
45
53
|
\`\`\`
|
|
46
54
|
|
|
55
|
+
Ordering: to append a task to the end of a status column, always use
|
|
56
|
+
\`--placement end\` — the server resolves the position atomically. This
|
|
57
|
+
includes status changes: a bare \`--status\` keeps the task's old
|
|
58
|
+
\`sort_order\`, which may collide inside the new column. Do NOT compute a
|
|
59
|
+
\`sort_order\` value yourself from listed tasks (your view may be stale or
|
|
60
|
+
partial). \`--sort-order\` is only for pinpoint insertion between two cards
|
|
61
|
+
you just listed, and it cannot be combined with \`--placement\`.
|
|
62
|
+
|
|
47
63
|
Subtasks are just tasks with a parent: create one with \`tasks create --parent-id\`,
|
|
48
64
|
re-parent with \`tasks update --parent-id\`, list a parent's children with
|
|
49
65
|
\`tasks subtasks\`. \`tasks list\` without \`--parent-id\` already returns both
|
|
@@ -78,7 +94,7 @@ watcher.
|
|
|
78
94
|
When you receive \`[Event: task.assigned]\`:
|
|
79
95
|
|
|
80
96
|
1. Acknowledge with a comment: \`tasks comments add prll://tsk_xxx --body "On it"\`
|
|
81
|
-
2. Update status: \`tasks update prll://tsk_xxx --status in_progress\`
|
|
97
|
+
2. Update status: \`tasks update prll://tsk_xxx --status in_progress --placement end\`
|
|
82
98
|
3. Do the work
|
|
83
99
|
4. Report results in a comment. If a gate remains — review, merge, deploy,
|
|
84
100
|
requester acceptance — set \`in_review\` and name the gate; set \`done\`
|