@gotcos/glasses-server 6.36.18 → 6.36.21
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +144 -0
- package/package.json +1 -1
- package/server/index.ts +23 -0
- package/server/lib/fence-liveness.ts +126 -0
- package/server/lib/message-era-reset.ts +37 -28
- package/server/lib/message-era.ts +32 -3
- package/server/lib/query-job-runtime.ts +1 -1
- package/server/lib/thread-turn-queue.ts +34 -1
- package/server/routes/agent-session-bindings.ts +55 -6
- package/server/routes/query.ts +1 -1
- package/server/routes/thread-turn-queue.ts +1 -0
- package/server/scripts/reset-message-era.ts +4 -3
package/CHANGELOG.md
CHANGED
|
@@ -1,3 +1,147 @@
|
|
|
1
|
+
## 6.36.21
|
|
2
|
+
|
|
3
|
+
**Fence liveness: the one thing about a fence a machine can actually establish.**
|
|
4
|
+
|
|
5
|
+
`GET /agent-sessions/fences` and the release preview now carry a `liveness`
|
|
6
|
+
aggregate -- `state` (`running` / `none_running` / `unknown`) plus counts. It
|
|
7
|
+
answers "is a child COS spawned for this turn still writing", which is the only
|
|
8
|
+
way releasing is unsafe for a reason a machine can see: admit a new turn while an
|
|
9
|
+
old child is still writing and two writers interleave in one transcript.
|
|
10
|
+
|
|
11
|
+
IT IS NOT A SAFETY VERDICT and no surface may render it as one. Whether the
|
|
12
|
+
ambiguous turn landed stays unknowable. Two automatic fence resolvers were
|
|
13
|
+
designed and both rejected (thread-fence-store.ts:40) because the dominant shape
|
|
14
|
+
is `timeout`, where the child had ~21 minutes to run tool calls before SIGKILL --
|
|
15
|
+
and the only fence this system has ever produced was exactly that shape. A
|
|
16
|
+
classifier built on n=1 would be guessing with a confident face.
|
|
17
|
+
|
|
18
|
+
The module enforces the three traps FenceRecord warns about rather than repeating
|
|
19
|
+
them: a pid is matched against its RECORDED start, because the OS recycles pids;
|
|
20
|
+
an empty or missing spawn list resolves to `unknown`, never `none_running`,
|
|
21
|
+
because "nothing was recorded" is not "nothing ran"; and any probe that throws or
|
|
22
|
+
returns something unparseable resolves toward `unknown`. A live child outranks an
|
|
23
|
+
unreadable probe.
|
|
24
|
+
|
|
25
|
+
Only the aggregate crosses the wire. No pid, no spawn list, and the rest of the
|
|
26
|
+
evidence block stays disk-only -- widening that is a deliberate contract change,
|
|
27
|
+
not a side effect of this.
|
|
28
|
+
|
|
29
|
+
`listFences` now REQUIRES the probe. It was optional, and the resulting
|
|
30
|
+
`deps === undefined` branch was unreachable from the route: a mutation flipping
|
|
31
|
+
its default to `none_running` left all 262 route tests green. An unreached line
|
|
32
|
+
that returns a confident answer is worse than no line.
|
|
33
|
+
|
|
34
|
+
|
|
35
|
+
**A turn queued against a fenced thread was thrown away in about two minutes.**
|
|
36
|
+
|
|
37
|
+
The thread-turn queue is wired before the agent-session-bindings router, and that
|
|
38
|
+
router built its own `TargetGuard`. So the queue's occupancy gate could not see a
|
|
39
|
+
fence at all: a fenced target reported attachable, `drainDecision` returned
|
|
40
|
+
'deliver', the loopback attach refused `native_target_fenced`, and the refusal
|
|
41
|
+
spent one of five delivery attempts. Five 20-second ticks retire a turn -- long
|
|
42
|
+
before anyone could reach the Mac, and the only fence this system has produced sat
|
|
43
|
+
for roughly 40 hours because clearing it required a terminal.
|
|
44
|
+
|
|
45
|
+
This is the SAME failure `native_target_busy` hit before the binding was made
|
|
46
|
+
visible to occupancy, and the note recording that fix is three lines above the
|
|
47
|
+
change. Fences never got the same treatment.
|
|
48
|
+
|
|
49
|
+
The server now owns one `TargetGuard`, constructed above the queue and handed to
|
|
50
|
+
the router, and occupancy consults it. A fenced target holds, and a hold spends
|
|
51
|
+
nothing.
|
|
52
|
+
|
|
53
|
+
`native_target_fenced` is now queueable -- the one entry in that set cleared by a
|
|
54
|
+
person rather than a clock. The previous reasoning (waiting cannot resolve "may or
|
|
55
|
+
may not have been delivered") was correct for a world where releasing needed a
|
|
56
|
+
terminal. COS Control 0.5.63 ships a Release button that runs, so the wait ends on
|
|
57
|
+
a real event. Queueing does not weaken anything: delivery re-enters the attach
|
|
58
|
+
route and re-runs the fence check.
|
|
59
|
+
|
|
60
|
+
Fence-held turns get `FENCE_HELD_TURN_TTL_MS` (72h) instead of the ordinary 6h.
|
|
61
|
+
Six hours is right for a busy thread; it is wrong for a state that ends when a
|
|
62
|
+
person looks. It still expires, and it still says so.
|
|
63
|
+
|
|
64
|
+
Two existing tests were retargeted rather than deleted, both with the reasoning
|
|
65
|
+
kept: the one asserting a fence is NOT queueable, and the one using a fence as its
|
|
66
|
+
example of a refusal that can never clear -- that rule is unchanged and now uses a
|
|
67
|
+
structurally permanent reason as its witness.
|
|
68
|
+
|
|
69
|
+
Requires COS Control 0.5.63 for the Release button to actually work.
|
|
70
|
+
|
|
71
|
+
Suite 3003 / 213 files, tsc 0. The wiring and liveness guards are mutation-verified: removing
|
|
72
|
+
the fence check from occupancy, and moving the guard below the queue, each fail
|
|
73
|
+
the assertion written for them.
|
|
74
|
+
|
|
75
|
+
## 6.36.20
|
|
76
|
+
|
|
77
|
+
Follow-up to 6.36.19, which was never published. QA found three things in it.
|
|
78
|
+
|
|
79
|
+
**The reset CLI mutated production archives just by importing.**
|
|
80
|
+
`message-era-reset.ts` imported `endSession`/`getActiveSessions` from
|
|
81
|
+
`conversation.js` and used neither -- but that module's scope runs
|
|
82
|
+
`loadFromDisk()` and a boot `runDailyArchiveMirror()`. In the one-shot CLI that
|
|
83
|
+
is a second process concurrently loading and rewriting the archives the live
|
|
84
|
+
server owns, and `appendToArchive` appends rather than upserts, so it can
|
|
85
|
+
duplicate prior-day chats. The import is gone, and a test now asserts it stays
|
|
86
|
+
gone. The previous canary injected an `archiveAndRelease` and asserted it was
|
|
87
|
+
never called, which only ever caught a regression routed through the callback --
|
|
88
|
+
the code it replaced called `endSession()` directly as its default. `sessions`
|
|
89
|
+
and `archiveAndRelease` are deleted from the input type rather than left as a
|
|
90
|
+
silent no-op for a future caller.
|
|
91
|
+
|
|
92
|
+
**A corrupt `message-era.json` was indistinguishable from a reset.**
|
|
93
|
+
`currentMessageEraState` fell back to `legacy` on missing OR invalid content.
|
|
94
|
+
Reverting to `legacy` after a real era existed is the worst available answer: it
|
|
95
|
+
re-reads every era-stamped exchange as a PREVIOUS era, hiding it from
|
|
96
|
+
`/all-messages`, while promoting unstamped ones to current -- and to a client it
|
|
97
|
+
looks exactly like a reset nobody asked for. Missing still means `legacy`, which
|
|
98
|
+
is correct for a first upgrade whose exchanges carry no stamp at all. Corrupt
|
|
99
|
+
now rotates explicitly, says so on stderr, and degrades to `legacy` without
|
|
100
|
+
caching only if the replacement cannot be persisted.
|
|
101
|
+
|
|
102
|
+
**The header described a code path that had been removed.** It still said
|
|
103
|
+
rotation stops "if an archive write fails". There is no archive write. It now
|
|
104
|
+
also records what 6.36.19 quietly changed: this path does NOT archive, and the
|
|
105
|
+
daily mirror skips today by design, so a same-day copy exists only after an
|
|
106
|
+
explicit session end or `POST /api/archive/now`.
|
|
107
|
+
|
|
108
|
+
The 409 copy on both query paths no longer tells the wearer to reopen for a
|
|
109
|
+
"fresh message list" -- as of app 6.8.423 the cards stay.
|
|
110
|
+
|
|
111
|
+
Requires app 6.8.423. Suite 2976 / 209 files, tsc 0, measured on a CLEAN
|
|
112
|
+
checkout of this commit rather than a working tree carrying another session's
|
|
113
|
+
uncommitted files -- the 6.36.19 note quoted 2981/211 from a contaminated tree,
|
|
114
|
+
and that tree's untracked code would have shipped in the tarball.
|
|
115
|
+
|
|
116
|
+
The conversation-import canary is mutation-verified: reintroducing the import
|
|
117
|
+
fails it. One unidentified test failed on a single clean-tree run and did not
|
|
118
|
+
recur across six further runs; it is not attributed to this change and it is
|
|
119
|
+
recorded here rather than rounded down to "clean".
|
|
120
|
+
|
|
121
|
+
## 6.36.19
|
|
122
|
+
- **Resetting the spoken message count no longer ends the conversation.**
|
|
123
|
+
`resetLiveMessageEra` used to `endSession` every live session before rotating
|
|
124
|
+
the era, so "reset the message count" also emptied CHAT and killed the thread
|
|
125
|
+
the wearer was in the middle of. It now rotates the era and nothing else.
|
|
126
|
+
- Numbers stay unique because they were never bare ints: a number is
|
|
127
|
+
`{messageEra, globalMsgNum}`. The era rotates, the current-era ceiling
|
|
128
|
+
restarts at 0, and leftover cards keep their old era and their old number.
|
|
129
|
+
Lookup already prefers the current era (`message-ref.ts:231-234`) and the
|
|
130
|
+
counter is already era-scoped (`message-ref.ts:242-251`), so "message N" stays
|
|
131
|
+
unambiguous without a second ID scheme.
|
|
132
|
+
- `archived` stays in the response and is now always `0`. Callers read the field,
|
|
133
|
+
so it is kept — but copy that reports it must stop claiming sessions were
|
|
134
|
+
archived. The `archive_failed` 503 path is **gone**, not relocated: there is no
|
|
135
|
+
archive step left to fail.
|
|
136
|
+
- Tests inverted to match. A supplied `archiveAndRelease` is now asserted
|
|
137
|
+
*never to be called*, and a refusing one can no longer block the rotation.
|
|
138
|
+
Mutation-verified: reintroducing the release loop fails the suite.
|
|
139
|
+
|
|
140
|
+
**Ship gate:** do not POST `/api/message-era/reset` from any surface until app
|
|
141
|
+
6.8.422 is also live. Server-first plus the old companion still wipes local
|
|
142
|
+
messages, and `/sessions/today/all-messages` is era-filtered so recover cannot
|
|
143
|
+
bring the leftover cards back.
|
|
144
|
+
|
|
1
145
|
## 6.36.18
|
|
2
146
|
- **Meetings list now carries voice-assignment tags.** Each row includes
|
|
3
147
|
`voiceReview` from the sidecar head (`speakers[]`) plus whether a human
|
package/package.json
CHANGED
package/server/index.ts
CHANGED
|
@@ -24,10 +24,12 @@ import { createAttachedTurnStream } from './lib/session-stream-producer.js'
|
|
|
24
24
|
import { claudeSessionsRouter } from './routes/claude-sessions.js'
|
|
25
25
|
import {
|
|
26
26
|
createAgentSessionBindingsRouter,
|
|
27
|
+
TargetGuard,
|
|
27
28
|
|
|
28
29
|
threadAttachEnabled,
|
|
29
30
|
} from './routes/agent-session-bindings.js'
|
|
30
31
|
import { AgentSessionBindingRegistry } from './lib/agent-session-binding-registry.js'
|
|
32
|
+
import { targetKey } from './lib/agent-session-binding-store.js'
|
|
31
33
|
import { cosSpawnedPids } from './lib/agent-session-ownership-store.js'
|
|
32
34
|
import { buildOccupancyProbes, realOccupancyDirs } from './lib/occupancy-probes.js'
|
|
33
35
|
import { realAttachedWorkspaceDeps, resolveAttachedWorkspace } from './lib/attached-workspace.js'
|
|
@@ -516,6 +518,18 @@ app.use('/api', claudeSessionsRouter)
|
|
|
516
518
|
// accounting; a second copy of that sequence in a background worker is a second place
|
|
517
519
|
// for the gate to drift. One request per delivered turn, at a handful of turns a day,
|
|
518
520
|
// buys the guarantee that the queue CANNOT weaken the gate even by accident.
|
|
521
|
+
// ONE guard, owned here, because the thread-turn queue below is wired BEFORE the
|
|
522
|
+
// bindings router and still has to see a fence. When the router made its own, the
|
|
523
|
+
// queue read an empty one: a fenced target reported attachable, the drainer
|
|
524
|
+
// ATTEMPTED, the attach route refused `native_target_fenced`, and five of those
|
|
525
|
+
// retired the queued turn in about two minutes. That is the identical failure the
|
|
526
|
+
// binding note in `occupancy` below records, one layer over.
|
|
527
|
+
const sharedTargetGuard = new TargetGuard(
|
|
528
|
+
process.env.COS_THREAD_FENCE_DURABLE === '1'
|
|
529
|
+
? { load: readFences, save: writeFences }
|
|
530
|
+
: null,
|
|
531
|
+
)
|
|
532
|
+
|
|
519
533
|
if (threadAttachEnabled()) {
|
|
520
534
|
const queueDeps = {
|
|
521
535
|
// TWO GATES, ONE ANSWER. `threadOccupancy` sees FOREIGN holders -- another app on
|
|
@@ -544,6 +558,12 @@ if (threadAttachEnabled()) {
|
|
|
544
558
|
// (`blocksTarget`), so an expired or terminal one correctly reads as free.
|
|
545
559
|
const holder = agentSessionBindingRegistry.getByThread(provider, threadId, Date.now())
|
|
546
560
|
if (holder !== null) return { attachable: false, reason: 'native_target_busy' }
|
|
561
|
+
// THE FENCE, for the same reason the binding is here and not one layer down.
|
|
562
|
+
// This is ADVISORY: the attach route stays the authority that refuses. Saying
|
|
563
|
+
// it here only means the drainer holds instead of spending an attempt, and a
|
|
564
|
+
// turn can wait out a fence that a person will clear.
|
|
565
|
+
const fenced = sharedTargetGuard.fencedReason(targetKey(provider, threadId))
|
|
566
|
+
if (fenced !== null) return { attachable: false, reason: fenced }
|
|
547
567
|
return { attachable: true, reason: null }
|
|
548
568
|
} catch {
|
|
549
569
|
// A throwing probe is not an open door.
|
|
@@ -597,6 +617,9 @@ app.use('/api', createAgentSessionBindingsRouter({
|
|
|
597
617
|
// POST /agent-sessions/fences/release work either way — which already removes
|
|
598
618
|
// the restart from the recovery path — and this flag flips on once COS Control
|
|
599
619
|
// has a Fences card.
|
|
620
|
+
// The instance the queue's `occupancy` already reads. Constructing a second one
|
|
621
|
+
// here would give the queue a permanently empty fence view.
|
|
622
|
+
guard: sharedTargetGuard,
|
|
600
623
|
fencePersistence: process.env.COS_THREAD_FENCE_DURABLE === '1'
|
|
601
624
|
? { load: readFences, save: writeFences }
|
|
602
625
|
: undefined,
|
|
@@ -0,0 +1,126 @@
|
|
|
1
|
+
// Is any child COS spawned for a fenced turn STILL RUNNING?
|
|
2
|
+
//
|
|
3
|
+
// WHAT THIS IS NOT. It is not a safety verdict and it must never be presented as
|
|
4
|
+
// one. Two plans for an automatic fence resolver were designed and both rejected
|
|
5
|
+
// (thread-fence-store.ts:40), and the reasoning still holds: the dominant fence
|
|
6
|
+
// shape is `timeout`, where the child had a ~21 minute budget to run tool calls
|
|
7
|
+
// before SIGKILL, and nothing observable afterwards distinguishes "the turn landed"
|
|
8
|
+
// from "it did not". This module answers a narrower question that IS mechanically
|
|
9
|
+
// knowable, and leaves delivery to the person.
|
|
10
|
+
//
|
|
11
|
+
// WHY THE NARROW QUESTION IS WORTH ANSWERING. A still-running child is the one way
|
|
12
|
+
// releasing a fence is unsafe for a reason a machine can see: admit a new turn while
|
|
13
|
+
// an old child is still writing and two writers interleave in one transcript.
|
|
14
|
+
// Everything else about a fence is judgement.
|
|
15
|
+
//
|
|
16
|
+
// PID ALONE IS NOT AN IDENTITY. `spawns` records a MEASURED start for each pid
|
|
17
|
+
// precisely because the OS recycles pids; a live pid whose start time does not match
|
|
18
|
+
// is a different process and says nothing about ours. FenceRecord's own comment says
|
|
19
|
+
// so, and this module is where that warning is enforced rather than repeated.
|
|
20
|
+
//
|
|
21
|
+
// EMPTY IS NOT ABSENCE. An empty or missing `spawns` list means no child was ever
|
|
22
|
+
// RECORDED, which is not the same as no child having run -- FenceRecord says no
|
|
23
|
+
// resolver may read it as "nothing landed". It resolves to `unknown`, never to
|
|
24
|
+
// `none_running`.
|
|
25
|
+
//
|
|
26
|
+
// FAIL CLOSED. Anything unmeasurable resolves toward `unknown`. The caller is a
|
|
27
|
+
// human deciding whether to release; an over-confident `none_running` is worse than
|
|
28
|
+
// admitting the probe could not see.
|
|
29
|
+
|
|
30
|
+
/** Recorded spawn: a pid and the start COS measured when it created the child. */
|
|
31
|
+
export interface RecordedSpawn {
|
|
32
|
+
pid: number
|
|
33
|
+
startMs: number
|
|
34
|
+
}
|
|
35
|
+
|
|
36
|
+
export type FenceLivenessState =
|
|
37
|
+
/** Every recorded child is provably gone. */
|
|
38
|
+
| 'none_running'
|
|
39
|
+
/** At least one recorded child is alive AND is identity-matched to ours. */
|
|
40
|
+
| 'running'
|
|
41
|
+
/** Nothing was recorded, or at least one probe could not answer. */
|
|
42
|
+
| 'unknown'
|
|
43
|
+
|
|
44
|
+
export interface FenceLiveness {
|
|
45
|
+
state: FenceLivenessState
|
|
46
|
+
/** Spawns examined. Zero means nothing was recorded. */
|
|
47
|
+
recorded: number
|
|
48
|
+
/** Alive and identity-matched: the reason a release is mechanically unsafe. */
|
|
49
|
+
running: number
|
|
50
|
+
/** Alive but started too far from the recorded time -- a RECYCLED pid, not ours.
|
|
51
|
+
* Counted, never treated as ours, and never treated as evidence about our child. */
|
|
52
|
+
recycled: number
|
|
53
|
+
/** Probes that threw or returned something unusable. */
|
|
54
|
+
unverifiable: number
|
|
55
|
+
}
|
|
56
|
+
|
|
57
|
+
export interface FenceLivenessDeps {
|
|
58
|
+
/** Epoch ms at which `pid` started, or null when it is not running.
|
|
59
|
+
* MAY THROW; a throwing probe is not an answer and resolves to `unknown`. */
|
|
60
|
+
pidStartMs: (pid: number) => number | null
|
|
61
|
+
}
|
|
62
|
+
|
|
63
|
+
/**
|
|
64
|
+
* Clock skew between COS's own `Date.now()` at spawn time and the start `ps`
|
|
65
|
+
* reports, which has one-second resolution. Two seconds is comfortably wider than
|
|
66
|
+
* that rounding and far narrower than any realistic pid-recycling window.
|
|
67
|
+
*/
|
|
68
|
+
export const START_MATCH_TOLERANCE_MS = 2_000
|
|
69
|
+
|
|
70
|
+
export function fenceLiveness(
|
|
71
|
+
spawns: ReadonlyArray<RecordedSpawn> | undefined | null,
|
|
72
|
+
deps: FenceLivenessDeps,
|
|
73
|
+
toleranceMs: number = START_MATCH_TOLERANCE_MS,
|
|
74
|
+
): FenceLiveness {
|
|
75
|
+
const rows = Array.isArray(spawns) ? spawns : []
|
|
76
|
+
const out: FenceLiveness = {
|
|
77
|
+
state: 'unknown', recorded: rows.length, running: 0, recycled: 0, unverifiable: 0,
|
|
78
|
+
}
|
|
79
|
+
|
|
80
|
+
// Nothing recorded. NOT 'none_running' -- see the header.
|
|
81
|
+
if (rows.length === 0) return out
|
|
82
|
+
|
|
83
|
+
for (const row of rows) {
|
|
84
|
+
if (!row || !Number.isSafeInteger(row.pid) || row.pid <= 0
|
|
85
|
+
|| !Number.isFinite(row.startMs)) {
|
|
86
|
+
out.unverifiable += 1
|
|
87
|
+
continue
|
|
88
|
+
}
|
|
89
|
+
let started: number | null
|
|
90
|
+
try {
|
|
91
|
+
started = deps.pidStartMs(row.pid)
|
|
92
|
+
} catch {
|
|
93
|
+
out.unverifiable += 1
|
|
94
|
+
continue
|
|
95
|
+
}
|
|
96
|
+
if (started === null) continue // gone: the ordinary, good case
|
|
97
|
+
if (!Number.isFinite(started)) { out.unverifiable += 1; continue }
|
|
98
|
+
if (Math.abs(started - row.startMs) <= toleranceMs) out.running += 1
|
|
99
|
+
else out.recycled += 1 // someone else's process wearing our pid
|
|
100
|
+
}
|
|
101
|
+
|
|
102
|
+
// A live child outranks an unreadable probe: it is the one thing we are sure of.
|
|
103
|
+
if (out.running > 0) out.state = 'running'
|
|
104
|
+
else if (out.unverifiable > 0) out.state = 'unknown'
|
|
105
|
+
else out.state = 'none_running'
|
|
106
|
+
return out
|
|
107
|
+
}
|
|
108
|
+
|
|
109
|
+
/** Real probe. `ps -o lstart=` is the only start-time keyword macOS ps offers --
|
|
110
|
+
* `etimes` is not a valid keyword here, which is why this parses a date rather
|
|
111
|
+
* than reading elapsed seconds. */
|
|
112
|
+
export function makePidStartProbe(
|
|
113
|
+
run: (pid: number) => string | null,
|
|
114
|
+
): (pid: number) => number | null {
|
|
115
|
+
return (pid: number) => {
|
|
116
|
+
const raw = run(pid)
|
|
117
|
+
if (raw === null) return null
|
|
118
|
+
const text = raw.trim()
|
|
119
|
+
if (!text) return null
|
|
120
|
+
const parsed = Date.parse(text)
|
|
121
|
+
// Unparseable is NOT "not running" -- throw so the caller counts it
|
|
122
|
+
// unverifiable rather than silently reading a live child as gone.
|
|
123
|
+
if (!Number.isFinite(parsed)) throw new Error(`unparseable process start: ${text}`)
|
|
124
|
+
return parsed
|
|
125
|
+
}
|
|
126
|
+
}
|
|
@@ -1,15 +1,23 @@
|
|
|
1
|
-
//
|
|
2
|
-
//
|
|
3
|
-
//
|
|
4
|
-
//
|
|
5
|
-
|
|
6
|
-
|
|
1
|
+
// Rotate the short-number namespace. Live sessions are NOT ended and nothing
|
|
2
|
+
// is archived here: the companion keeps every card, so the reset is a numbering
|
|
3
|
+
// change rather than a history change.
|
|
4
|
+
//
|
|
5
|
+
// Do not rotate the era if a query is in flight.
|
|
6
|
+
//
|
|
7
|
+
// NOTE: this file deliberately imports nothing from ./conversation.js. Importing
|
|
8
|
+
// it runs its module scope -- loadFromDisk() and a boot runDailyArchiveMirror()
|
|
9
|
+
// -- which in the one-shot CLI means a second process concurrently loading and
|
|
10
|
+
// rewriting the same archives the live server owns. appendToArchive appends
|
|
11
|
+
// rather than upserts, so that duplicates prior-day chats.
|
|
12
|
+
//
|
|
13
|
+
// Today's exchanges are NOT archived by this path. The daily mirror skips
|
|
14
|
+
// today by design (conversation.ts: `sessionDay >= todayLocal`), so a same-day
|
|
15
|
+
// copy exists only after an explicit session end or POST /api/archive/now.
|
|
7
16
|
import {
|
|
8
17
|
createMessageEra,
|
|
9
18
|
currentMessageEraState,
|
|
10
19
|
type MessageEraState,
|
|
11
20
|
} from './message-era.js'
|
|
12
|
-
import type { SessionToArchive } from './archive.js'
|
|
13
21
|
|
|
14
22
|
export class MessageEraResetError extends Error {
|
|
15
23
|
readonly code: string
|
|
@@ -37,8 +45,6 @@ export interface MessageEraResetInput {
|
|
|
37
45
|
now?: number
|
|
38
46
|
activeRuns?: number
|
|
39
47
|
shuttingDown?: boolean
|
|
40
|
-
sessions?: SessionToArchive[]
|
|
41
|
-
archiveAndRelease?: (session: SessionToArchive) => Promise<boolean>
|
|
42
48
|
}
|
|
43
49
|
|
|
44
50
|
async function resolveJobHealth(input: MessageEraResetInput): Promise<{ activeRuns: number; shuttingDown: boolean }> {
|
|
@@ -79,26 +85,29 @@ export async function resetLiveMessageEra(input: MessageEraResetInput): Promise<
|
|
|
79
85
|
}
|
|
80
86
|
|
|
81
87
|
const previous = currentMessageEraState()
|
|
82
|
-
const sessions = input.sessions ?? getActiveSessions()
|
|
83
|
-
const archiveAndRelease = input.archiveAndRelease ?? (async (session: SessionToArchive) => {
|
|
84
|
-
const result = await endSession(session.id)
|
|
85
|
-
if (!result) return true
|
|
86
|
-
if (result.exchangeCount > 0 && !result.logged) return false
|
|
87
|
-
return true
|
|
88
|
-
})
|
|
89
88
|
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
89
|
+
// 6.36.19 — rotate the era, do NOT end the thread.
|
|
90
|
+
//
|
|
91
|
+
// This used to endSession() every live session before rotating, which is what
|
|
92
|
+
// made "reset the message count" also empty CHAT and kill the conversation the
|
|
93
|
+
// wearer was in the middle of. Miles wants the opposite shape: the next message
|
|
94
|
+
// is #1, the old cards keep their numbers, and the thread survives.
|
|
95
|
+
//
|
|
96
|
+
// Numbers stay unique because they are {messageEra, globalMsgNum}, not a bare
|
|
97
|
+
// int: the era rotates, the current-era ceiling restarts at 0, and leftover
|
|
98
|
+
// cards keep their old era. Lookup already prefers the current era
|
|
99
|
+
// (message-ref.ts:231-234) and the counter is already era-scoped
|
|
100
|
+
// (message-ref.ts:242-251), so nothing downstream needs to change to keep
|
|
101
|
+
// "message N" unambiguous.
|
|
102
|
+
//
|
|
103
|
+
// `archived` stays in the result and is now always 0. Control and the app read
|
|
104
|
+
// it, so removing the field would break them; the copy that reports it has to
|
|
105
|
+
// stop claiming sessions were archived.
|
|
106
|
+
//
|
|
107
|
+
// Deliberately no archive step here. There is nothing to release, so there is
|
|
108
|
+
// no 503 archive_failed path any more — that failure mode is gone rather than
|
|
109
|
+
// relocated. The day-archive mirror already retains history.
|
|
110
|
+
const archived = 0
|
|
102
111
|
|
|
103
112
|
const next: MessageEraState = createMessageEra(input.now ?? Date.now())
|
|
104
113
|
return {
|
|
@@ -55,9 +55,38 @@ export function currentMessageEraState(): MessageEraState {
|
|
|
55
55
|
cachedMtimeMs = mtimeMs
|
|
56
56
|
return cached
|
|
57
57
|
}
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
58
|
+
// Missing is not corrupt. A legacy install upgrading for the first time has
|
|
59
|
+
// no file AND no era stamps on its exchanges, and only LEGACY_MESSAGE_ERA
|
|
60
|
+
// classifies an unstamped exchange as current. Rotating here would file every
|
|
61
|
+
// one of them under a previous era and empty the chat on upgrade.
|
|
62
|
+
if (loaded.status === 'missing') {
|
|
63
|
+
cached = { v: 1, era: LEGACY_MESSAGE_ERA, startedAt: 0 }
|
|
64
|
+
cachedMtimeMs = mtimeMs
|
|
65
|
+
return cached
|
|
66
|
+
}
|
|
67
|
+
|
|
68
|
+
// Corrupt, or present-but-invalid: the file existed, so a real era almost
|
|
69
|
+
// certainly did too, and its value is now unrecoverable. Reverting to legacy
|
|
70
|
+
// would be the worst available answer -- it re-reads every era-stamped
|
|
71
|
+
// exchange as a PREVIOUS era (hiding it from /all-messages) while promoting
|
|
72
|
+
// unstamped ones to current, and it is indistinguishable from a reset nobody
|
|
73
|
+
// asked for. Rotate explicitly and say so, so the state is self-consistent
|
|
74
|
+
// and the event is diagnosable.
|
|
75
|
+
console.error(
|
|
76
|
+
`[message-era] ${MESSAGE_ERA_FILE} was unreadable (status=${loaded.status}); `
|
|
77
|
+
+ 'rotating to a fresh era. Older messages stay in day archives and remain '
|
|
78
|
+
+ 'reachable by session, but not by short number.',
|
|
79
|
+
)
|
|
80
|
+
try {
|
|
81
|
+
return createMessageEra()
|
|
82
|
+
} catch (err) {
|
|
83
|
+
// Read-only or full data dir: degrade to legacy rather than crash the
|
|
84
|
+
// server, but never cache it -- the next call retries the rotation.
|
|
85
|
+
console.error('[message-era] could not persist the replacement era:', err)
|
|
86
|
+
cached = null
|
|
87
|
+
cachedMtimeMs = Number.NaN
|
|
88
|
+
return { v: 1, era: LEGACY_MESSAGE_ERA, startedAt: 0 }
|
|
89
|
+
}
|
|
61
90
|
}
|
|
62
91
|
|
|
63
92
|
export function currentMessageEra(): string {
|
|
@@ -82,7 +82,7 @@ export async function preparePublicDurableQueryAdmission(raw: unknown): Promise<
|
|
|
82
82
|
throw new QueryJobAdmissionPreparationError(
|
|
83
83
|
409,
|
|
84
84
|
'message_era_mismatch',
|
|
85
|
-
'Reopen or update COS Glasses
|
|
85
|
+
'Reopen or update COS Glasses. Your cards stay; the next message is #1.',
|
|
86
86
|
)
|
|
87
87
|
}
|
|
88
88
|
try {
|
|
@@ -94,6 +94,21 @@ export interface QueuedThreadTurn {
|
|
|
94
94
|
*/
|
|
95
95
|
export const QUEUED_TURN_TTL_MS = 6 * 60 * 60 * 1000
|
|
96
96
|
|
|
97
|
+
/**
|
|
98
|
+
* TTL for a turn held by a FENCE specifically.
|
|
99
|
+
*
|
|
100
|
+
* Six hours is right for a busy thread: the holder finishes or it never will. A
|
|
101
|
+
* fence is different in kind -- it ends when a PERSON looks, and the only fence this
|
|
102
|
+
* system has ever produced sat for about 40 hours before anyone could clear it,
|
|
103
|
+
* because clearing it needed a terminal. Expiring at six would have thrown the turn
|
|
104
|
+
* away roughly seven times over while the user was still waiting for a working
|
|
105
|
+
* Release button.
|
|
106
|
+
*
|
|
107
|
+
* It still expires. A turn that outlives this is one nobody is coming back for, and
|
|
108
|
+
* silently holding it forever is worse than telling the truth about losing it.
|
|
109
|
+
*/
|
|
110
|
+
export const FENCE_HELD_TURN_TTL_MS = 72 * 60 * 60 * 1000
|
|
111
|
+
|
|
97
112
|
/** Waiting turns per thread. Small: this is a queue, not a backlog. */
|
|
98
113
|
export const MAX_QUEUED_PER_THREAD = 8
|
|
99
114
|
|
|
@@ -141,6 +156,18 @@ const QUEUEABLE_REFUSALS: ReadonlySet<string> = new Set([
|
|
|
141
156
|
'unverifiable_process_start',
|
|
142
157
|
'unverifiable_liveness_socket',
|
|
143
158
|
'probe_failed',
|
|
159
|
+
// A FENCE IS CLEARED BY A PERSON, NOT BY A CLOCK -- the one exception in this set.
|
|
160
|
+
//
|
|
161
|
+
// It is queueable because "COS sent a turn and never learned whether it arrived"
|
|
162
|
+
// is precisely the state a queued turn should outlive: the operator checks, releases,
|
|
163
|
+
// and the turn lands. Releasing kicks the drain, so the wait ends on a real event.
|
|
164
|
+
//
|
|
165
|
+
// It is ALSO why the fence had to become visible to `occupancy` (index.ts). Without
|
|
166
|
+
// that, a fenced target reads attachable, the drainer ATTEMPTS, the attach route
|
|
167
|
+
// refuses, and five of those retire the turn in about two minutes -- the identical
|
|
168
|
+
// failure the binding comment above records, and it would have made queue-and-release
|
|
169
|
+
// look like it worked while silently dropping the turn.
|
|
170
|
+
'native_target_fenced',
|
|
144
171
|
])
|
|
145
172
|
|
|
146
173
|
/** Can a turn refused for this reason be parked, or must it refuse now? */
|
|
@@ -159,6 +186,9 @@ export interface DrainObservation {
|
|
|
159
186
|
turnEnded: boolean
|
|
160
187
|
/** The 30s transcript clock. `idle` is the backstop when no terminal record lands. */
|
|
161
188
|
activity: 'working' | 'idle' | 'unknown'
|
|
189
|
+
/** The gate's reason when `attachable` is false. Carried ONLY so a fence -- the one
|
|
190
|
+
* hold a clock cannot end -- can be given a longer life than a busy thread. */
|
|
191
|
+
reason?: string | null
|
|
162
192
|
}
|
|
163
193
|
|
|
164
194
|
export type DrainDecision = 'deliver' | 'hold' | 'expire' | 'give_up'
|
|
@@ -178,7 +208,10 @@ export function drainDecision(
|
|
|
178
208
|
): DrainDecision {
|
|
179
209
|
if (turn.status !== 'waiting') return 'hold'
|
|
180
210
|
if (!Number.isFinite(now) || !Number.isFinite(turn.queuedAt)) return 'hold'
|
|
181
|
-
|
|
211
|
+
// A fence is cleared by a person, so it gets the longer clock. Everything else --
|
|
212
|
+
// including a fence we could not name -- keeps the ordinary one.
|
|
213
|
+
const effectiveTtl = seen.reason === 'native_target_fenced' ? FENCE_HELD_TURN_TTL_MS : ttlMs
|
|
214
|
+
if (now - turn.queuedAt >= effectiveTtl) return 'expire'
|
|
182
215
|
if (turn.attempts >= MAX_DELIVERY_ATTEMPTS) return 'give_up'
|
|
183
216
|
// THE GATE, unweakened. Everything below is about WHEN, never about whether.
|
|
184
217
|
if (!seen.attachable) return 'hold'
|
|
@@ -95,6 +95,8 @@
|
|
|
95
95
|
// and answer 400. They never treat an unparsed body as an empty one.
|
|
96
96
|
|
|
97
97
|
import type { FenceRecord } from '../lib/thread-fence-store.js'
|
|
98
|
+
import { execFileSync } from 'node:child_process'
|
|
99
|
+
import { fenceLiveness, makePidStartProbe, type FenceLiveness, type FenceLivenessDeps } from '../lib/fence-liveness.js'
|
|
98
100
|
import { Router, type Request, type Response } from 'express'
|
|
99
101
|
import { createHash, randomUUID } from 'node:crypto'
|
|
100
102
|
import {
|
|
@@ -271,6 +273,11 @@ export type AttachedTurnResult =
|
|
|
271
273
|
| { ok: boolean; delivery: 'not_attempted' | 'aborted' | 'ambiguous' | 'delivered' }
|
|
272
274
|
|
|
273
275
|
export interface AgentSessionBindingsDeps {
|
|
276
|
+
/** Shared fence state. Omit and the router owns a private one, which is correct
|
|
277
|
+
* for tests and wrong for the server -- see the wiring note at its use site. */
|
|
278
|
+
guard?: TargetGuard
|
|
279
|
+
/** Process-start probe behind the fence liveness aggregate. */
|
|
280
|
+
liveness?: FenceLivenessDeps
|
|
274
281
|
/**
|
|
275
282
|
* Durable fence storage. OPTIONAL, and omitting it is what keeps the existing
|
|
276
283
|
* suite in memory: a test that silently began writing the real data home would
|
|
@@ -889,7 +896,14 @@ export interface FencePersistence {
|
|
|
889
896
|
* the real data home would leak fences between cases and into the running server,
|
|
890
897
|
* so the suite runs with `null` and stays in memory.
|
|
891
898
|
*/
|
|
892
|
-
|
|
899
|
+
/** The fence question, narrowed for callers that must not touch anything else.
|
|
900
|
+
* `occupancy` in index.ts reads this so a fenced target holds instead of burning
|
|
901
|
+
* a delivery attempt; the ATTACH route remains the authority that refuses. */
|
|
902
|
+
export interface TargetFenceView {
|
|
903
|
+
fencedReason(targetKey: string): WriteRefusal | null
|
|
904
|
+
}
|
|
905
|
+
|
|
906
|
+
export class TargetGuard {
|
|
893
907
|
/** targetKey -> turnId of the single COS turn allowed to be in flight. */
|
|
894
908
|
private readonly claims = new Map<string, string>()
|
|
895
909
|
/** targetKey -> the fence record. DURABLE as of 6.36.10: persistence is injected
|
|
@@ -994,8 +1008,18 @@ class TargetGuard {
|
|
|
994
1008
|
}
|
|
995
1009
|
|
|
996
1010
|
/** Every fence, REDACTED for the wire: the raw targetKey embeds the private
|
|
997
|
-
* native thread id, so callers address a fence by its deterministic digest.
|
|
998
|
-
|
|
1011
|
+
* native thread id, so callers address a fence by its deterministic digest.
|
|
1012
|
+
*
|
|
1013
|
+
* `liveness` is the ONLY evidence field that crosses, and it crosses as an
|
|
1014
|
+
* aggregate: a state plus counts, never a pid. It answers "is a child from this
|
|
1015
|
+
* turn still writing", which is the one thing about a fence a machine can
|
|
1016
|
+
* actually establish. It is NOT a safety verdict and the UI must not render it
|
|
1017
|
+
* as one -- whether the ambiguous turn landed stays unknowable, which is why two
|
|
1018
|
+
* automatic resolvers were rejected (thread-fence-store.ts:40).
|
|
1019
|
+
*
|
|
1020
|
+
* The rest of the evidence block stays disk-only. Widening that is a deliberate
|
|
1021
|
+
* contract change, not a side effect of adding this. */
|
|
1022
|
+
listFences(deps: FenceLivenessDeps): Array<{ target: string; provider: string; reason: string; headBefore: string | null; turnId: string; fencedAt: number; liveness: FenceLiveness }> {
|
|
999
1023
|
return [...this.fences.values()].map(row => ({
|
|
1000
1024
|
target: opaqueRevision(row.targetKey),
|
|
1001
1025
|
provider: row.provider,
|
|
@@ -1003,6 +1027,12 @@ class TargetGuard {
|
|
|
1003
1027
|
headBefore: row.headBefore,
|
|
1004
1028
|
turnId: row.turnId,
|
|
1005
1029
|
fencedAt: row.fencedAt,
|
|
1030
|
+
// REQUIRED, not optional. An optional probe meant a `deps === undefined`
|
|
1031
|
+
// branch that the route could never take -- a mutation flipping it to
|
|
1032
|
+
// `none_running` left the whole 262-test suite green, which is the
|
|
1033
|
+
// definition of an unreached line. A caller with no probe must construct a
|
|
1034
|
+
// deliberate one rather than fall into a default answer.
|
|
1035
|
+
liveness: fenceLiveness(row.spawns, deps),
|
|
1006
1036
|
}))
|
|
1007
1037
|
}
|
|
1008
1038
|
|
|
@@ -1287,7 +1317,23 @@ export function createAgentSessionBindingsRouter(deps: AgentSessionBindingsDeps)
|
|
|
1287
1317
|
const canListBindings = bindingDepsUsable(deps)
|
|
1288
1318
|
const canWriteBindings = bindingWriteDepsUsable(deps)
|
|
1289
1319
|
const detect = deps?.occupancy ?? threadOccupancy
|
|
1290
|
-
|
|
1320
|
+
// INJECTABLE. index.ts owns the instance because the thread-turn queue is wired
|
|
1321
|
+
// BEFORE this router and must be able to see a fence; a second guard created here
|
|
1322
|
+
// would be a disconnected copy, and the queue would go on reading an empty one.
|
|
1323
|
+
const guard = deps.guard ?? new TargetGuard(deps.fencePersistence ?? null)
|
|
1324
|
+
// `ps -o lstart=` is the only start-time keyword macOS ps offers. Injectable so a
|
|
1325
|
+
// test drives recycled and unreadable pids without spawning real processes.
|
|
1326
|
+
const livenessDeps: FenceLivenessDeps = deps.liveness ?? {
|
|
1327
|
+
pidStartMs: makePidStartProbe((pid) => {
|
|
1328
|
+
try {
|
|
1329
|
+
return execFileSync('ps', ['-p', String(pid), '-o', 'lstart='], {
|
|
1330
|
+
encoding: 'utf8', timeout: 2_000,
|
|
1331
|
+
})
|
|
1332
|
+
} catch {
|
|
1333
|
+
return null // not running, or ps refused the pid
|
|
1334
|
+
}
|
|
1335
|
+
}),
|
|
1336
|
+
}
|
|
1291
1337
|
const ownership = deps?.ownership ?? { record: recordCosSpawn, release: releaseCosSpawn }
|
|
1292
1338
|
// One per router. Injectable so the follow-on (attach accepting a `forkRef`)
|
|
1293
1339
|
// shares this instance rather than standing up a second, disconnected one.
|
|
@@ -1383,7 +1429,7 @@ export function createAgentSessionBindingsRouter(deps: AgentSessionBindingsDeps)
|
|
|
1383
1429
|
// `degraded` is reported, never inferred: a memory-only fence set behaves
|
|
1384
1430
|
// identically to a durable one until the process restarts, so a silent
|
|
1385
1431
|
// fallback would be indistinguishable from working.
|
|
1386
|
-
res.json({ fences: guard.listFences(), degraded: guard.degraded() })
|
|
1432
|
+
res.json({ fences: guard.listFences(livenessDeps), degraded: guard.degraded() })
|
|
1387
1433
|
})
|
|
1388
1434
|
|
|
1389
1435
|
router.post('/agent-sessions/fences/release', (req, res) => {
|
|
@@ -1400,7 +1446,10 @@ export function createAgentSessionBindingsRouter(deps: AgentSessionBindingsDeps)
|
|
|
1400
1446
|
// The API token is shared by the phone, the lens and every COS agent
|
|
1401
1447
|
// session, so nothing is structurally prevented from asserting `confirm`.
|
|
1402
1448
|
// It stops an accidental release, not an automated one.
|
|
1403
|
-
|
|
1449
|
+
// The preview carries the liveness aggregate too: this is the exact moment a
|
|
1450
|
+
// person decides, and "a child from this turn is still running" is the one
|
|
1451
|
+
// thing here a machine can tell them.
|
|
1452
|
+
const preview = guard.listFences(livenessDeps).find(f => f.target === target) ?? null
|
|
1404
1453
|
res.status(400).json({ released: false, reason: 'confirmation_required', preview })
|
|
1405
1454
|
return
|
|
1406
1455
|
}
|
package/server/routes/query.ts
CHANGED
|
@@ -60,7 +60,7 @@ queryRouter.post('/query', async (req, res) => {
|
|
|
60
60
|
maintenanceLease.release()
|
|
61
61
|
return res.status(409).json({
|
|
62
62
|
error: 'message_era_mismatch',
|
|
63
|
-
detail: 'Reopen or update COS Glasses
|
|
63
|
+
detail: 'Reopen or update COS Glasses. Your cards stay; the next message is #1.',
|
|
64
64
|
era: activeMessageEra,
|
|
65
65
|
})
|
|
66
66
|
}
|
|
@@ -113,6 +113,7 @@ export async function drainThread(
|
|
|
113
113
|
attachable: gate.attachable,
|
|
114
114
|
turnEnded: deps.turnEnded(provider, threadId),
|
|
115
115
|
activity: deps.activity(provider, threadId),
|
|
116
|
+
reason: gate.reason,
|
|
116
117
|
}
|
|
117
118
|
const decision = drainDecision(turn, seen, deps.now())
|
|
118
119
|
|
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
#!/usr/bin/env tsx
|
|
2
|
-
//
|
|
3
|
-
//
|
|
2
|
+
// Creates a fresh short-number namespace. Live sessions are NOT ended and
|
|
3
|
+
// nothing is archived -- the companion keeps every card. Disk mtime is enough
|
|
4
|
+
// — no server restart.
|
|
4
5
|
//
|
|
5
6
|
// npx tsx server/scripts/reset-message-era.ts --confirm
|
|
6
7
|
|
|
@@ -22,7 +23,7 @@ console.log(JSON.stringify({
|
|
|
22
23
|
restartRequired: false,
|
|
23
24
|
verify: 'GET /api/message-counter should return { max: 0, era: "<era above>" }',
|
|
24
25
|
nextSteps: [
|
|
25
|
-
'Phone:
|
|
26
|
+
'Phone: reopen the companion — cards stay, the next message is #1',
|
|
26
27
|
'Send a test message — expect #1',
|
|
27
28
|
],
|
|
28
29
|
}, null, 2))
|