@threadcrumbs/common 0.1.4 → 0.1.5

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/NOTICES.md CHANGED
@@ -31,3 +31,99 @@ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
31
31
  OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
32
32
  SOFTWARE.
33
33
  ```
34
+
35
+ ### Fonts
36
+
37
+ `ui/fonts/` includes these fonts as .woff2 files, from the @fontsource packages. Each is used under the SIL Open Font License, Version 1.1, which follows the list.
38
+
39
+ - Inter and Inter Tight: Copyright 2016 and 2022 The Inter Project Authors (https://github.com/rsms/inter)
40
+ - JetBrains Mono: Copyright 2020 The JetBrains Mono Project Authors (https://github.com/JetBrains/JetBrainsMono)
41
+
42
+ ```text
43
+ SIL OPEN FONT LICENSE Version 1.1 - 26 February 2007
44
+ -----------------------------------------------------------
45
+
46
+ PREAMBLE
47
+ The goals of the Open Font License (OFL) are to stimulate worldwide
48
+ development of collaborative font projects, to support the font creation
49
+ efforts of academic and linguistic communities, and to provide a free and
50
+ open framework in which fonts may be shared and improved in partnership
51
+ with others.
52
+
53
+ The OFL allows the licensed fonts to be used, studied, modified and
54
+ redistributed freely as long as they are not sold by themselves. The
55
+ fonts, including any derivative works, can be bundled, embedded,
56
+ redistributed and/or sold with any software provided that any reserved
57
+ names are not used by derivative works. The fonts and derivatives,
58
+ however, cannot be released under any other type of license. The
59
+ requirement for fonts to remain under this license does not apply
60
+ to any document created using the fonts or their derivatives.
61
+
62
+ DEFINITIONS
63
+ "Font Software" refers to the set of files released by the Copyright
64
+ Holder(s) under this license and clearly marked as such. This may
65
+ include source files, build scripts and documentation.
66
+
67
+ "Reserved Font Name" refers to any names specified as such after the
68
+ copyright statement(s).
69
+
70
+ "Original Version" refers to the collection of Font Software components as
71
+ distributed by the Copyright Holder(s).
72
+
73
+ "Modified Version" refers to any derivative made by adding to, deleting,
74
+ or substituting -- in part or in whole -- any of the components of the
75
+ Original Version, by changing formats or by porting the Font Software to a
76
+ new environment.
77
+
78
+ "Author" refers to any designer, engineer, programmer, technical
79
+ writer or other person who contributed to the Font Software.
80
+
81
+ PERMISSION & CONDITIONS
82
+ Permission is hereby granted, free of charge, to any person obtaining
83
+ a copy of the Font Software, to use, study, copy, merge, embed, modify,
84
+ redistribute, and sell modified and unmodified copies of the Font
85
+ Software, subject to the following conditions:
86
+
87
+ 1) Neither the Font Software nor any of its individual components,
88
+ in Original or Modified Versions, may be sold by itself.
89
+
90
+ 2) Original or Modified Versions of the Font Software may be bundled,
91
+ redistributed and/or sold with any software, provided that each copy
92
+ contains the above copyright notice and this license. These can be
93
+ included either as stand-alone text files, human-readable headers or
94
+ in the appropriate machine-readable metadata fields within text or
95
+ binary files as long as those fields can be easily viewed by the user.
96
+
97
+ 3) No Modified Version of the Font Software may use the Reserved Font
98
+ Name(s) unless explicit written permission is granted by the corresponding
99
+ Copyright Holder. This restriction only applies to the primary font name as
100
+ presented to the users.
101
+
102
+ 4) The name(s) of the Copyright Holder(s) or the Author(s) of the Font
103
+ Software shall not be used to promote, endorse or advertise any
104
+ Modified Version, except to acknowledge the contribution(s) of the
105
+ Copyright Holder(s) and the Author(s) or with their explicit written
106
+ permission.
107
+
108
+ 5) The Font Software, modified or unmodified, in part or in whole,
109
+ must be distributed entirely under this license, and must not be
110
+ distributed under any other license. The requirement for fonts to
111
+ remain under this license does not apply to any document created
112
+ using the Font Software.
113
+
114
+ TERMINATION
115
+ This license becomes null and void if any of the above conditions are
116
+ not met.
117
+
118
+ DISCLAIMER
119
+ THE FONT SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
120
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTIES OF
121
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT
122
+ OF COPYRIGHT, PATENT, TRADEMARK, OR OTHER RIGHT. IN NO EVENT SHALL THE
123
+ COPYRIGHT HOLDER BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
124
+ INCLUDING ANY GENERAL, SPECIAL, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL
125
+ DAMAGES, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
126
+ FROM, OUT OF THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM
127
+ OTHER DEALINGS IN THE FONT SOFTWARE.
128
+
129
+ ```
package/README.md CHANGED
@@ -19,14 +19,21 @@ need it.
19
19
  | `@threadcrumbs/common/launch` | `present(result, via)`: carries out a harness adapter's answer to "open this thread" by fronting the window of a session already running, handing a deep link to the OS, or running the harness's CLI in a new terminal. When nothing on the machine answers the link, it tries the adapter's `fallback` links, such as Claude Code's thread in VS Code, and failing those returns `{ ok: false, error, missing: { name, download, scheme, also }, terminal }`: the app to install, where to get it, other apps that would do, and whether `via: 'terminal'` would work instead. Also `launch(target)` and `resolveFolder(dir)`. Threadgauge uses it, and Threadnauts still has its own copy |
20
20
  | `@threadcrumbs/common/license` | The Threadcrumbs license both apps check offline and the key service signs: `createLicenseKeys`, `publicKeyOf`, `licenseClaims`, `signLicense`, `verifyLicense`, and the unlock rule, `paidThrough` and `unlocks`, plus `renewalDue`. Web APIs only, so the same file runs in a Cloudflare Worker. The format and rules are in [DECISIONS.md](DECISIONS.md#licenses-are-signed-by-threadcrumbs-and-checked-offline). `LICENSE_PUBLIC_KEYS` holds the production keys every app trusts, and never a sandbox key |
21
21
  | `@threadcrumbs/common/license/store` | Where the license lives on this machine, shared by every app: `licenseStore()` returns `device()`, `current()`, `save(token)`, and `clear()`, over `license.json` in `threadcrumbsDataDir()`. `current().at` is the time to test unlocks at, and `current()` writes nothing on a machine where no one has entered a key. `trustedKeys({ development })` adds a sandbox key in a development build, and `cloudUrl({ development })` is the key service's address: `https://threadcrumbs.io/api`, or the local Worker at `http://localhost:8787/api` in a development build. Node only, so the Worker never loads it |
22
+ | `@threadcrumbs/common/ui/theme.css` | The Threadcrumbs look as CSS variables: the Quieter backgrounds, lines, and text levels, Threadcrumbs orange, the six status colors and their plates, `--warn`, added and removed, the chart and gauge colors, focus, type sizes, spacing, radii, and the default `--sans`, `--cond`, and `--mono` stacks. Every element keeps one x-height (`font-size-adjust: 0.51`) |
23
+ | `@threadcrumbs/common/ui/fonts.css` | `@font-face` rules for Inter, Inter Tight, and JetBrains Mono, Latin and Latin Extended, upright, with their `.woff2` files in `ui/fonts/`. A bundler copies the files into the app, so they load from its own origin. `tools/ui-fonts.mjs` writes both from the `@fontsource` packages |
24
+ | `@threadcrumbs/common/ui/status` | `STATUSES` (each status's key, name, color, and plate, in the order both apps pin), `STATUS_KEYS`, `URGENCY_ORDER` (for the status row and the waffle), `LIVE_STATUSES`, `statusOf`, the status icons as SVG in `LAMP_ICONS`, and `lampHTML(status)`. `ui/status.css` colors each `[data-st]` and animates the lamps, and reduced motion stops them |
25
+ | `@threadcrumbs/common/ui/icons` | `ICO(name)`, the line icons (organization, folder, worktree, branch, model, key, and one per tool kind); `effortBars` and `effortName`; `harnessMark` (CC, Cx, and the others); and, for tool calls, `KINDS`, `kindOf`, `kindTip`, `kindIconHTML`, `toolKinds`, and `mcpServer` |
26
+ | `@threadcrumbs/common/ui/where` | `whereHTML(thread)`, the organization and repo chips, `whereFullHTML(thread)` with the worktree and branch, and `repoName(name, org)`, which drops a package scope the organization repeats. `ui/where.css` styles the chips, the harness initials, the effort bars, and the tool kind icons |
27
+ | `@threadcrumbs/common/ui/gauge` | `contextGauge({ pct, level }, size)`, the context gauge as SVG, with `levelOf(pct)` (amber from 70%, red from 90%) and `levelClass`. `ui/gauge.css` draws it and gives a picture with a tip a slightly larger hover area (`.i-wrap`) |
22
28
 
23
29
  `launch/` is the only part of this package that starts processes, and `license/store.mjs` is the
24
- only part that writes a file. The readers and `orgs/` import neither.
30
+ only part that writes a file. The readers and `orgs/` import neither. `ui/` is plain functions
31
+ that return markup, plus CSS and fonts, for a page to draw; it touches no DOM and does no I/O.
25
32
 
26
33
  These are likely to follow, each when an app first needs it from here:
27
34
  - **Suite discovery:** the `/api/about` identity format and the port search.
28
35
  - **Local-server safety:** the Host and Origin checks and the security policy.
29
- - **Design tokens:** the palette and status colors.
36
+ - **More shared UI:** the list's lamps row, group heading, and thread row, then the Conversation.
30
37
 
31
38
  ## Using it from an app
32
39
 
@@ -1,6 +1,6 @@
1
1
  import fsp from 'node:fs/promises'
2
2
  import { requestCost } from './anthropic-prices.mjs'
3
- import { emptyFixed, emptySizes, setFixed, sizeCall, sizeCompaction, sizePrompt, sizeReply, sizeRequest, sizeResult, sizeResumed } from './context-breakdown.mjs'
3
+ import { FIXED_PARTS, emptyFixed, emptySizes, setFixed, sizeCall, sizeCompaction, sizePrompt, sizeReply, sizeRequest, sizeResult, sizeResumed } from './context-breakdown.mjs'
4
4
  import { listFiles, parseJsonLine, readNewLines } from './fsutil.mjs'
5
5
  import {
6
6
  DETAIL_LIMITS,
@@ -25,6 +25,7 @@ import {
25
25
  settleCalls,
26
26
  startCall,
27
27
  textList,
28
+ wordsFull,
28
29
  toolFile,
29
30
  toolInput,
30
31
  toolTarget,
@@ -34,6 +35,7 @@ import { mcpServer } from './tool-kinds.mjs'
34
35
  const HEAD_BYTES = 192 * 1024
35
36
  const TAIL_BYTES = 64 * 1024
36
37
  const TAIL_LIMIT = 4 * 1024 * 1024
38
+ const WORDS_TAIL_BYTES = 512 * 1024
37
39
  const NOISE = /<([a-z][\w-]*)(?:\s[^>]*)?>[\s\S]*?<\/\1>/g
38
40
 
39
41
  const AGENT_TOOLS = new Set(['Agent', 'Task'])
@@ -42,6 +44,7 @@ const AWAIT_LIMIT = 20
42
44
  const PEER_BUSY = new Set(['busy', 'waiting'])
43
45
 
44
46
  const MID_TURN = Object.freeze({ handedBack: false, failed: false })
47
+ const INTERRUPTED = /^\[Request interrupted by user/
45
48
 
46
49
  const number = (v) => (Number.isFinite(Number(v)) ? Number(v) : 0)
47
50
  const text = (v) => (typeof v === 'string' ? v : '')
@@ -72,7 +75,7 @@ export async function readLiveSessions(dir, isAlive = pidAlive) {
72
75
  return live
73
76
  }
74
77
 
75
- export function transcriptReader({ skim = false, detail = false, words = false, sizes = false } = {}) {
78
+ export function transcriptReader({ skim = false, detail = false, words = false, sizes = false, tail = false } = {}) {
76
79
  const files = new Map()
77
80
 
78
81
  async function advance(entry, file, size) {
@@ -80,7 +83,9 @@ export function transcriptReader({ skim = false, detail = false, words = false,
80
83
  const state = entry.state
81
84
  if (size === state.offset) return state
82
85
  try {
83
- if (skim && state.offset === 0 && size > HEAD_BYTES + TAIL_BYTES) {
86
+ if (tail && state.offset === 0 && size > WORDS_TAIL_BYTES) {
87
+ await tailInto(state, file, size, () => emptyState(entry.sidechain, detail, words, sizes))
88
+ } else if (skim && state.offset === 0 && size > HEAD_BYTES + TAIL_BYTES) {
84
89
  await skimInto(state, file, size)
85
90
  } else {
86
91
  const { lines, carry } = await readNewLines(file, state.offset, size, state.carry)
@@ -117,6 +122,19 @@ export function transcriptReader({ skim = false, detail = false, words = false,
117
122
  return { read, sweep }
118
123
  }
119
124
 
125
+ async function tailInto(state, file, size, fresh) {
126
+ for (let bytes = WORDS_TAIL_BYTES; ; bytes *= 4) {
127
+ const from = Math.max(0, size - bytes)
128
+ const { lines, carry } = await readNewLines(file, from, size)
129
+ const trial = fresh()
130
+ for (const line of from > 0 ? lines.slice(1) : lines) applyLine(trial, line)
131
+ if (from === 0 || wordsFull(trial.words)) {
132
+ Object.assign(state, trial, { offset: size, carry })
133
+ return
134
+ }
135
+ }
136
+ }
137
+
120
138
  async function skimInto(state, file, size) {
121
139
  const head = await readNewLines(file, 0, HEAD_BYTES)
122
140
  const headEnd = HEAD_BYTES - head.carry.length
@@ -253,27 +271,31 @@ function applySystem(detail, record, at) {
253
271
  }
254
272
  }
255
273
 
256
- const emptySent = () => ({ system: 0, tools: [], deferred: new Set(), instructions: new Map(), skills: new Map(), skillsOther: 0, memory: new Map(), agents: 0 })
274
+ const emptySent = () => ({ system: 0, tools: [], deferred: new Set(), instructions: new Map(), skills: new Map(), skillsOther: 0, memory: new Map(), agents: 0, seen: new Set() })
257
275
 
258
276
  function applySent(state, record) {
259
277
  const sent = state.sent
260
278
  const attachment = record.attachment
261
279
  if (record.type === 'system' && record.subtype === 'compact_boundary') {
262
280
  sizeCompaction(state.sizes)
263
- state.sent = { ...emptySent(), system: sent.system, tools: sent.tools }
281
+ state.sent = { ...emptySent(), system: sent.system, tools: sent.tools, seen: sent.seen }
264
282
  return
265
283
  }
266
284
  if (record.type === 'api-request-shape') {
267
285
  const shape = record.shape || {}
286
+ sent.seen.add('shape')
268
287
  sent.system = (Array.isArray(shape.system) ? shape.system : []).reduce((n, part) => n + text(part?.text).length, 0)
269
288
  sent.tools = (Array.isArray(shape.tools) ? shape.tools : []).map((tool) => ({ name: text(tool?.name), chars: JSON.stringify(tool ?? null).length }))
270
289
  } else if (attachment?.type === 'deferred_tools_delta') {
290
+ sent.seen.add('mcpTools')
271
291
  for (const name of [...listOf(attachment.addedNames), ...listOf(attachment.readdedNames)]) sent.deferred.add(text(name))
272
292
  for (const name of listOf(attachment.removedNames)) sent.deferred.delete(text(name))
273
293
  } else if (attachment?.type === 'mcp_instructions_delta') {
294
+ sent.seen.add('mcpTools')
274
295
  listOf(attachment.addedNames).forEach((name, i) => sent.instructions.set(text(name), text(listOf(attachment.addedBlocks)[i]).length))
275
296
  for (const name of listOf(attachment.removedNames)) sent.instructions.delete(text(name))
276
297
  } else if (attachment?.type === 'skill_listing') {
298
+ sent.seen.add('skills')
277
299
  if (attachment.isInitial !== false) {
278
300
  sent.skills = new Map()
279
301
  sent.skillsOther = 0
@@ -285,6 +307,7 @@ function applySent(state, record) {
285
307
  else sent.skillsOther += line.length + 1
286
308
  }
287
309
  } else if (attachment?.type === 'instructions') {
310
+ sent.seen.add('memory')
288
311
  for (const file of listOf(attachment.files)) if (text(file?.path)) sent.memory.set(text(file.path), text(file.content).length)
289
312
  } else if (attachment?.type === 'agent_listing_delta') {
290
313
  sent.agents = (attachment.isInitial ? 0 : sent.agents) + listOf(attachment.addedLines).reduce((n, line) => n + text(line).length + 1, 0)
@@ -319,6 +342,8 @@ function fixedOf(sent) {
319
342
  fixed.mcp = new Map([...servers].map(([name, s]) => [name, { tools: s.names.size || null, chars: s.chars }]))
320
343
  fixed.skills = new Map(sent.skills)
321
344
  fixed.memory = new Map(sent.memory)
345
+ fixed.complete = sent.seen.has('shape')
346
+ fixed.recorded = new Set(fixed.complete ? FIXED_PARTS : ['system', ...[...sent.seen].filter((part) => part !== 'shape')])
322
347
  return fixed
323
348
  }
324
349
 
@@ -344,6 +369,13 @@ function applyUser(state, record, at) {
344
369
  if (!failed) countEdits(state, result, call)
345
370
  }
346
371
  }
372
+ if (INTERRUPTED.test(firstText(content).trim())) {
373
+ state.open.clear()
374
+ state.ending = { handedBack: true, failed: false }
375
+ state.handedBackAt = at
376
+ if (state.detail) settleCalls(state.detail.activity)
377
+ return
378
+ }
347
379
  if (!state.ending || state.ending.handedBack) state.turnStartedAt = at
348
380
  state.ending = MID_TURN
349
381
  const notice = record.isMeta ? IDLE_NOTICE.exec(firstText(content)) : null
@@ -4,8 +4,10 @@ const add = (map, key, n) => map.set(key, (map.get(key) || 0) + n)
4
4
  const sum = (values) => [...values].reduce((total, n) => total + n, 0)
5
5
  const noPending = () => ({ prompts: 0, replies: 0, tools: new Map() })
6
6
 
7
+ export const FIXED_PARTS = ['system', 'builtInTools', 'mcpTools', 'skills', 'memory']
8
+
7
9
  export function emptyFixed() {
8
- return { system: 0, builtInTools: 0, mcp: new Map(), skills: new Map(), memory: new Map() }
10
+ return { system: 0, builtInTools: 0, mcp: new Map(), skills: new Map(), memory: new Map(), recorded: new Set(), complete: false }
9
11
  }
10
12
 
11
13
  export function emptySizes() {
@@ -58,8 +60,8 @@ export function sizeCompaction(sizes) {
58
60
  sizes.callNames.clear()
59
61
  }
60
62
 
61
- const fixedChars = (fixed) =>
62
- fixed.system + fixed.builtInTools + sum([...fixed.mcp.values()].map((m) => m.chars)) + sum(fixed.skills.values()) + sum(fixed.memory.values())
63
+ const otherChars = (fixed) => fixed.builtInTools + sum([...fixed.mcp.values()].map((m) => m.chars)) + sum(fixed.skills.values()) + sum(fixed.memory.values())
64
+ const fixedChars = (fixed) => fixed.system + otherChars(fixed)
63
65
 
64
66
  function spread(weights, pending, tokens) {
65
67
  const chars = pending.prompts + pending.replies + sum(pending.tools.values())
@@ -74,9 +76,13 @@ function spread(weights, pending, tokens) {
74
76
 
75
77
  const counted = (sizes) => (sizes.first && !sizes.resumed ? sizes.first.input - sizes.first.prompt : 0)
76
78
 
77
- function ratio(sizes) {
78
- const fixed = counted(sizes)
79
- return fixed > 0 && sizes.first.chars ? fixed / sizes.first.chars : 1 / CHARS_PER_TOKEN
79
+ const measuredText = (sizes) => Boolean(sizes.first?.complete && sizes.first.chars && counted(sizes) > 0)
80
+
81
+ const ratio = (sizes) => (measuredText(sizes) ? counted(sizes) / sizes.first.chars : 1 / CHARS_PER_TOKEN)
82
+
83
+ function fixedNow(sizes) {
84
+ if (measuredText(sizes)) return Math.round(fixedChars(sizes.fixed) * ratio(sizes))
85
+ return counted(sizes) + Math.round((otherChars(sizes.fixed) - sizes.first.other) / CHARS_PER_TOKEN)
80
86
  }
81
87
 
82
88
  export function sizeRequest(sizes, input, at) {
@@ -84,11 +90,10 @@ export function sizeRequest(sizes, input, at) {
84
90
  if (at) sizes.lastAt = at
85
91
  if (sizes.lastInput == null) {
86
92
  if (!sizes.firstPrompt) sizes.resumed = true
87
- sizes.first = { input, prompt: Math.round(sizes.firstPrompt / CHARS_PER_TOKEN), chars: fixedChars(sizes.fixed) }
93
+ sizes.first = { input, prompt: Math.round(sizes.firstPrompt / CHARS_PER_TOKEN), chars: fixedChars(sizes.fixed), other: otherChars(sizes.fixed), complete: sizes.fixed.complete }
88
94
  spread(sizes.weights, sizes.pending, Math.min(input, sizes.first.prompt))
89
95
  } else if (sizes.compacted) {
90
- const fixed = sizes.first.chars ? Math.round(fixedChars(sizes.fixed) * ratio(sizes)) : counted(sizes)
91
- sizes.weights = { prompts: 0, replies: 0, summary: Math.max(0, input - fixed), tools: new Map() }
96
+ sizes.weights = { prompts: 0, replies: 0, summary: Math.max(0, input - fixedNow(sizes)), tools: new Map() }
92
97
  } else if (input > sizes.lastInput) {
93
98
  spread(sizes.weights, sizes.pending, input - sizes.lastInput)
94
99
  }
@@ -129,21 +134,24 @@ export function contextBreakdown(sizes, { total, limit = null, reserved = null }
129
134
  const system = Math.round(fixedChars(sizes.fixed) / CHARS_PER_TOKEN)
130
135
  return system > 0 && system < total ? twoParts(base, sizes.weights, system) : null
131
136
  }
132
- if (!sizes.first.chars) return twoParts(base, sizes.weights, fixed)
137
+ const { recorded } = sizes.fixed
138
+ if (!recorded.size || !sizes.first.chars) return twoParts(base, sizes.weights, fixed)
133
139
  const tokens = (n) => Math.round(n * ratio(sizes))
134
- const { system, builtInTools, mcp, skills, memory } = sizes.fixed
140
+ const { builtInTools, mcp, skills, memory } = sizes.fixed
135
141
  const servers = [...mcp].map(([name, m]) => ({ name, tools: m.tools ?? null, tokens: tokens(m.chars) })).filter((m) => m.tokens > 0)
136
142
  const skillItems = [...skills].map(([name, n]) => ({ name, tokens: tokens(n) })).filter((s) => s.tokens > 0)
137
143
  const memoryItems = [...memory].map(([path, n]) => ({ path, tokens: tokens(n) })).filter((m) => m.tokens > 0)
138
- const parts = [
139
- { id: 'system', tokens: tokens(system), items: [] },
144
+ const others = [
140
145
  { id: 'builtInTools', tokens: tokens(builtInTools), items: [] },
141
146
  { id: 'mcpTools', tokens: sum(servers.map((m) => m.tokens)), items: servers.sort((a, b) => b.tokens - a.tokens) },
142
147
  { id: 'skills', tokens: sum(skillItems.map((s) => s.tokens)), items: skillItems.sort((a, b) => b.tokens - a.tokens) },
143
148
  { id: 'memory', tokens: sum(memoryItems.map((m) => m.tokens)), items: memoryItems },
144
- ].filter((p) => p.tokens > 0)
145
- const fixedNow = sum(parts.map((p) => p.tokens))
146
- if (fixedNow >= total) return twoParts(base, sizes.weights, fixed)
147
- const messages = total - fixedNow
149
+ ].filter((p) => recorded.has(p.id))
150
+ const system = measuredText(sizes) ? tokens(sizes.fixed.system) : fixedNow(sizes) - sum(others.map((p) => p.tokens))
151
+ if (system < 0) return twoParts(base, sizes.weights, fixed)
152
+ const parts = [{ id: 'system', tokens: system, items: [] }, ...others]
153
+ const fixedTotal = sum(parts.map((p) => p.tokens))
154
+ if (fixedTotal >= total) return twoParts(base, sizes.weights, fixed)
155
+ const messages = total - fixedTotal
148
156
  return { ...base, parts: [{ id: 'messages', tokens: messages, items: messageItems(sizes.weights, messages) }, ...parts] }
149
157
  }
@@ -1,13 +1,27 @@
1
1
  import { parseJsonLine, readNewLines } from './fsutil.mjs'
2
2
 
3
- export function jsonlFollower({ empty, apply }) {
3
+ export function jsonlFollower({ empty, apply, tail = null }) {
4
4
  const files = new Map()
5
5
 
6
+ async function readTail(file, size) {
7
+ for (let bytes = tail.bytes; ; bytes *= 4) {
8
+ const from = Math.max(0, size - bytes)
9
+ const { lines, carry } = await readNewLines(file, from, size)
10
+ const state = { ...empty(), offset: size, carry }
11
+ for (const line of from > 0 ? lines.slice(1) : lines) {
12
+ const record = parseJsonLine(line)
13
+ if (record) apply(state, record)
14
+ }
15
+ if (from === 0 || tail.enough(state)) return state
16
+ }
17
+ }
18
+
6
19
  async function advance(entry, file, size) {
7
20
  if (!entry.state || size < entry.state.offset) entry.state = { ...empty(), offset: 0, carry: Buffer.alloc(0) }
8
21
  const state = entry.state
9
22
  if (size === state.offset) return state
10
23
  try {
24
+ if (tail && state.offset === 0 && size > tail.bytes) return (entry.state = await readTail(file, size))
11
25
  const { lines, carry } = await readNewLines(file, state.offset, size, state.carry)
12
26
  state.offset = size
13
27
  state.carry = carry
@@ -20,6 +20,7 @@ import {
20
20
  openTool,
21
21
  planItems,
22
22
  planSummary,
23
+ relativePath,
23
24
  sampleList,
24
25
  samplePoint,
25
26
  settleCalls,
@@ -130,9 +131,10 @@ export function opencodeStore({ id, name, dbPath, watchDirs, limitsFile, continu
130
131
  return sessions
131
132
  }
132
133
 
133
- async function sessionDetail(key) {
134
+ async function sessionDetail(key, { filesOnly = false } = {}) {
134
135
  const prefix = `${id}:`
135
136
  const sessionId = String(key).startsWith(prefix) ? String(key).slice(prefix.length) : ''
137
+ if (filesOnly) return sessionId ? { id: `${id}:${sessionId}`, sessionId, files: facts.get(`v1:${sessionId}`)?.value.files ?? null } : null
136
138
  const file = sessionId ? await dbPath() : ''
137
139
  if (!file) return null
138
140
  const context = { ...options, facts, seen: new Set(), byModel: new Map(), at: now() }
@@ -517,6 +519,7 @@ function legacyTurn(db, row, context) {
517
519
  )
518
520
  let last = null
519
521
  let lastAssistant = ''
522
+ const changed = new Set()
520
523
  for (const message of db.prepare('SELECT id, time_created, data FROM message WHERE session_id = ? ORDER BY time_created, rowid').all(row.id)) {
521
524
  const data = parse(message.data)
522
525
  last = data
@@ -531,6 +534,7 @@ function legacyTurn(db, row, context) {
531
534
  for (const diff of data.summary.diffs) {
532
535
  turn.lines.added += number(diff?.additions)
533
536
  turn.lines.removed += number(diff?.deletions)
537
+ if (text(diff?.file)) changed.add(relativePath(text(diff.file), text(row.directory)))
534
538
  }
535
539
  }
536
540
  continue
@@ -546,6 +550,7 @@ function legacyTurn(db, row, context) {
546
550
  samplePoint(turn.contextSample, turn.context)
547
551
  }
548
552
  }
553
+ turn.files = changed.size || !context.liveSummary ? [...changed].map((file) => ({ path: file })) : null
549
554
  const usage = usageFromColumns(row)
550
555
  if (usage) {
551
556
  turn.tokens = usage.tokens
@@ -633,12 +638,14 @@ function currentSample(db, sessionId, before) {
633
638
 
634
639
  function currentTurn(db, row, parts, context) {
635
640
  const before = context.facts.get(`v2:${row.id}`)?.value
636
- const turn = { busy: false, failed: false, prompts: 0, firstPrompt: '', model: '', provider: '', effort: '', tokens: null, cost: null, context: 0, tool: null, plan: null, turnStartedAt: 0, ...currentSample(db, row.id, before) }
641
+ const sampled = currentSample(db, row.id, before)
642
+ const kept = sampled.contextSample === before?.contextSample ? before : null
643
+ const turn = { busy: false, failed: false, prompts: 0, firstPrompt: '', model: '', provider: '', effort: '', tokens: null, cost: null, context: 0, tool: null, plan: null, turnStartedAt: 0, ...sampled }
637
644
  turn.prompts = number(db.prepare(`SELECT COUNT(*) AS n FROM session_message WHERE session_id = ? AND type = 'user'`).get(row.id)?.n)
638
645
  const lastUser = db.prepare(`SELECT time_created FROM session_message WHERE session_id = ? AND type = 'user' ORDER BY seq DESC LIMIT 1`).get(row.id)
639
646
  turn.turnStartedAt = number(row.time_busy) || number(lastUser?.time_created)
640
647
  turn.firstPrompt = text(parse(db.prepare(`SELECT data FROM session_message WHERE session_id = ? AND type = 'user' ORDER BY seq ASC LIMIT 1`).get(row.id)?.data).text).trim()
641
- const assistant = db.prepare(`SELECT id, data FROM session_message WHERE session_id = ? AND type = 'assistant' ORDER BY seq DESC LIMIT 1`).get(row.id)
648
+ const assistant = db.prepare(`SELECT id, seq, data FROM session_message WHERE session_id = ? AND type = 'assistant' ORDER BY seq DESC LIMIT 1`).get(row.id)
642
649
  const latest = parse(assistant?.data)
643
650
  turn.model = text(latest.model?.id)
644
651
  turn.provider = text(latest.model?.providerID)
@@ -660,36 +667,68 @@ function currentTurn(db, row, parts, context) {
660
667
  const outcome = text(row.idle_outcome) || (last?.type === 'idle' ? text(parse(last.data).outcome) : '')
661
668
  turn.failed = !busy && outcome === 'failed'
662
669
  turn.errorText = turn.failed && last?.type === 'idle' ? text(parse(last.data).error?.message) : ''
663
- const tools = currentTools(db, row.id, assistant, parts, text(row.directory))
670
+ const tools = currentTools(db, row.id, assistant, parts, text(row.directory), kept)
671
+ turn.planSeq = tools.planSeq
672
+ turn.olderTodos = tools.olderTodos
673
+ turn.planRow = tools.planRow
674
+ turn.rowTodos = tools.rowTodos
664
675
  if (busy) turn.tool = tools.running
665
676
  turn.plan = tools.todos
666
677
  return turn
667
678
  }
668
679
 
669
- function currentTools(db, sessionId, assistant, parts, cwd) {
680
+ const completedTodos = (parts) => {
681
+ const found = parts.find((p) => p?.name === TODO_TOOL && p.state?.status === 'completed' && Array.isArray(p.state?.input?.todos))
682
+ return found ? found.state.input.todos.map((t) => ({ content: text(t?.content), status: text(t?.status) })) : null
683
+ }
684
+
685
+ function rowTodos(db, sessionId, assistant, before) {
686
+ const newest = text(assistant?.id)
687
+ const from = before?.planRow ?? -1
688
+ let older = before?.rowTodos ?? null
689
+ const rows = db
690
+ .prepare(`SELECT data FROM session_part WHERE session_id = ? AND type = 'tool' AND rowid > ? AND message_id != ? AND data LIKE ? ORDER BY rowid DESC`)
691
+ .all(sessionId, from, newest, `%"${TODO_TOOL}"%`)
692
+ for (const row of rows) {
693
+ const found = completedTodos([parse(row.data)])
694
+ if (found) {
695
+ older = found
696
+ break
697
+ }
698
+ }
699
+ const top = db.prepare('SELECT MAX(rowid) AS row FROM session_part WHERE session_id = ? AND message_id != ?').get(sessionId, newest)
700
+ return { rowTodos: older, planRow: top?.row == null ? from : number(top.row) }
701
+ }
702
+
703
+ function inlineTodos(db, sessionId, assistant, before) {
704
+ const newest = number(assistant?.seq)
705
+ let older = before?.olderTodos ?? null
706
+ const rows = db
707
+ .prepare(`SELECT data FROM session_message WHERE session_id = ? AND type = 'assistant' AND seq > ? AND seq < ? AND data LIKE ? ORDER BY seq DESC`)
708
+ .all(sessionId, before?.planSeq ?? -1, newest, `%"${TODO_TOOL}"%`)
709
+ for (const row of rows) {
710
+ const found = completedTodos(list(parse(row.data).content).filter((c) => c?.type === 'tool').reverse())
711
+ if (found) {
712
+ older = found
713
+ break
714
+ }
715
+ }
716
+ return { olderTodos: older, planSeq: assistant ? newest - 1 : before?.planSeq ?? -1 }
717
+ }
718
+
719
+ function currentTools(db, sessionId, assistant, parts, cwd, before = null) {
670
720
  const result = { running: null, todos: null }
671
721
  let latest = []
672
- let recent = []
673
722
  if (parts && assistant) {
674
- latest = db.prepare(`SELECT data FROM session_part WHERE session_id = ? AND message_id = ? AND type = 'tool'`).all(sessionId, assistant.id).map((p) => parse(p.data))
675
- }
676
- if (parts) {
677
- recent = db
678
- .prepare(`SELECT data FROM session_part WHERE session_id = ? AND type = 'tool' AND data LIKE ? ORDER BY rowid DESC LIMIT 20`)
679
- .all(sessionId, `%"${TODO_TOOL}"%`)
680
- .map((p) => parse(p.data))
723
+ latest = db.prepare(`SELECT data FROM session_part WHERE session_id = ? AND message_id = ? AND type = 'tool' ORDER BY rowid`).all(sessionId, assistant.id).map((p) => parse(p.data))
681
724
  }
682
725
  if (!latest.length && assistant) latest = list(parse(assistant.data).content).filter((c) => c?.type === 'tool')
683
- if (!recent.length) {
684
- recent = db
685
- .prepare(`SELECT data FROM session_message WHERE session_id = ? AND type = 'assistant' AND data LIKE ? ORDER BY seq DESC LIMIT 20`)
686
- .all(sessionId, `%"${TODO_TOOL}"%`)
687
- .flatMap((m) => list(parse(m.data).content).filter((c) => c?.type === 'tool').reverse())
688
- }
689
726
  for (const part of latest) {
690
727
  if (OPEN_TOOL.has(part.state?.status)) result.running = openTool(text(part.name), number(part.time?.ran) || number(part.time?.created), part.state?.input, cwd)
691
728
  }
692
- const todos = recent.find((p) => p.name === TODO_TOOL && p.state?.status === 'completed' && Array.isArray(p.state?.input?.todos))
693
- if (todos) result.todos = todos.state.input.todos.map((t) => ({ content: text(t?.content), status: text(t?.status) }))
729
+ const rows = parts ? rowTodos(db, sessionId, assistant, before) : { rowTodos: null, planRow: -1 }
730
+ const inline = inlineTodos(db, sessionId, assistant, before)
731
+ Object.assign(result, rows, inline)
732
+ result.todos = completedTodos([...latest].reverse()) ?? rows.rowTodos ?? inline.olderTodos
694
733
  return result
695
734
  }
@@ -160,6 +160,8 @@ export function keepText(list, kind, { id = null, at, text: value }) {
160
160
  return keepNewest(list, { id, at, text: body }, max)
161
161
  }
162
162
 
163
+ export const wordsFull = (words) => words.replies.length >= DETAIL_LIMITS.replies && words.thoughts.length >= DETAIL_LIMITS.thoughts
164
+
163
165
  export const textList = (list) => (list ? list.map(({ at, text: value }) => ({ at, text: value })) : null)
164
166
 
165
167
  export function joinedText(blocks, type, field = type) {
@@ -219,6 +221,8 @@ export function fileList(files, cwd) {
219
221
  .sort((a, b) => b.added + b.removed - (a.added + a.removed) || a.path.localeCompare(b.path))
220
222
  }
221
223
 
224
+ export const filePaths = (files, cwd) => [...files.values()].map((f) => ({ path: relativePath(f.path, cwd) }))
225
+
222
226
  export const emptyActivity = () => ({ calls: [], tools: new Map() })
223
227
 
224
228
  export function startCall(activity, { id = '', name, target = '', input = null, path = null, line = null, startedAt, running = true }) {
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@threadcrumbs/common",
3
- "version": "0.1.4",
3
+ "version": "0.1.5",
4
4
  "type": "module",
5
5
  "description": "Code the Threadcrumbs apps share, starting with the readers for each coding-agent harness.",
6
6
  "license": "SEE LICENSE IN LICENSE.md",
@@ -25,7 +25,16 @@
25
25
  "./license": "./license/index.mjs",
26
26
  "./license/store": "./license/store.mjs",
27
27
  "./orgs": "./orgs/index.mjs",
28
- "./lib/*": "./lib/*.mjs"
28
+ "./lib/*": "./lib/*.mjs",
29
+ "./ui/status": "./ui/status.mjs",
30
+ "./ui/status.css": "./ui/status.css",
31
+ "./ui/icons": "./ui/icons.mjs",
32
+ "./ui/where": "./ui/where.mjs",
33
+ "./ui/where.css": "./ui/where.css",
34
+ "./ui/gauge": "./ui/gauge.mjs",
35
+ "./ui/gauge.css": "./ui/gauge.css",
36
+ "./ui/theme.css": "./ui/theme.css",
37
+ "./ui/fonts.css": "./ui/fonts.css"
29
38
  },
30
39
  "files": [
31
40
  "NOTICES.md",
@@ -34,9 +43,15 @@
34
43
  "license/",
35
44
  "lib/",
36
45
  "orgs/",
37
- "telemetry/"
46
+ "telemetry/",
47
+ "ui/"
38
48
  ],
39
49
  "scripts": {
40
50
  "test": "node --test \"test/**/*.test.mjs\""
51
+ },
52
+ "devDependencies": {
53
+ "@fontsource-variable/inter": "^5.3.0",
54
+ "@fontsource-variable/inter-tight": "^5.3.0",
55
+ "@fontsource-variable/jetbrains-mono": "^5.3.0"
41
56
  }
42
57
  }
@@ -158,6 +158,15 @@ Again, a `null` field means the harness doesn't record it. The shared helpers fo
158
158
  (`startCall`, `endCall`, `toolTarget`, `keepNewest`, `fileList`, and the limits) are in
159
159
  `../lib/telemetry-fields.mjs`.
160
160
 
161
+ `sessionDetail(id, { filesOnly: true })` returns only `{ id, sessionId, files }`, with each file
162
+ as `{ path }`, from what the session scan already holds. It's for finding other threads that
163
+ changed the same files, so it never opens the readers that serve an open thread.
164
+
165
+ The readers that serve an open thread (thoughts and replies for Claude Code, Codex, Cursor, and
166
+ Antigravity) are kept for the last three sessions opened. Claude Code's and Codex's read the
167
+ transcript from its end first, in growing steps, until they have the newest 50 replies, and then
168
+ follow it forwards.
169
+
161
170
  ### Context breakdown
162
171
 
163
172
  `contextBreakdown` splits the context into parts, so an app can show what fills it.
@@ -170,8 +179,8 @@ Again, a `null` field means the harness doesn't record it. The shared helpers fo
170
179
  them aren't read, so an app should call it the default. Free space is `limit - total -
171
180
  reserved`.
172
181
  - **`parts`** are `[{ id, tokens, items }]` in this order: `messages`, `system`,
173
- `builtInTools`, `mcpTools`, `skills`, `memory`. A part a harness doesn't let us size is left
174
- out, and its tokens stay in `system`.
182
+ `builtInTools`, `mcpTools`, `skills`, `memory`. A part the harness records is sent even at 0
183
+ tokens; one it doesn't record is left out, and its tokens stay in `system`.
175
184
  - `messages` items: `{ id: 'prompts' }`, `{ id: 'replies' }`, `{ id: 'summary' }` (from the
176
185
  last compaction), and `{ id: 'tool', name }` for each tool's calls and results.
177
186
  - `mcpTools` items: `{ name, tools }` per server, `tools` a count or `null`.
@@ -183,6 +192,11 @@ Again, a `null` field means the harness doesn't record it. The shared helpers fo
183
192
  - That gives a characters-per-token ratio for the text of each fixed part (system prompt,
184
193
  tool definitions, MCP servers, skills, memory files), so each part is its text length at
185
194
  that ratio. A part added later counts at the same ratio.
195
+ - The ratio is used only when the system prompt and tool definitions were recorded, as in
196
+ Claude Code's `api-request-shape`. Otherwise it would spread their unrecorded text over the
197
+ parts that were, so each recorded part is its text at 4 characters a token, and `system` is
198
+ the rest of the fixed count. That's always the case for Codex, whose built-in tool
199
+ definitions aren't written.
186
200
  - `messages` is the rest of the total. Each request's growth over the one before is split
187
201
  among the prompts, replies, and tool calls and results since then, by text length. After a
188
202
  compaction, the next request's growth over the fixed parts is the summary.
@@ -200,8 +214,8 @@ Again, a `null` field means the harness doesn't record it. The shared helpers fo
200
214
  - Codex: the rollout's base instructions and developer messages, its AGENTS.md messages, and
201
215
  its `additional_tools` items. Its built-in tool definitions aren't written, so they're in
202
216
  `system`.
203
- - **Read only for an open thread,** by the same second reader as thoughts and replies. It keeps
204
- numbers and names, never the text.
217
+ - **Its sizes are kept by the session scan,** which already reads every record, as numbers and
218
+ names only, never the text, so opening a thread doesn't read its transcript again.
205
219
 
206
220
  ### Tool kinds
207
221
 
@@ -252,9 +252,10 @@ export function antigravity({
252
252
  return sessions
253
253
  }
254
254
 
255
- async function sessionDetail(key) {
255
+ async function sessionDetail(key, { filesOnly = false } = {}) {
256
256
  const id = String(key).replace(/^antigravity:/, '')
257
257
  if (!known.has(id)) return null
258
+ if (filesOnly) return { id: `antigravity:${id}`, sessionId: id, files: null }
258
259
  const sessionDir = path.join(brainDir, id)
259
260
  const found = await transcriptOf(sessionDir)
260
261
  const state = (found && (await transcripts.read(found.file, found.stat).catch(() => null))) || emptyTranscript()