typeclaw 0.46.0 → 0.46.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (52) hide show
  1. package/README.md +1 -1
  2. package/package.json +2 -2
  3. package/src/agent/multimodal/looker.ts +11 -1
  4. package/src/agent/network/safe-http.ts +17 -8
  5. package/src/agent/plugin-tools.ts +1 -3
  6. package/src/agent/tool-file-safety.ts +396 -136
  7. package/src/agent/tools/webfetch/fetch.ts +22 -2
  8. package/src/agent/tools-without-local-file-operands.ts +9 -6
  9. package/src/bundled-plugins/backup/README.md +23 -10
  10. package/src/bundled-plugins/backup/git-auth.ts +6 -4
  11. package/src/bundled-plugins/backup/index.ts +83 -43
  12. package/src/bundled-plugins/backup/runner.ts +83 -0
  13. package/src/bundled-plugins/github-cli-auth/git-askpass.ts +74 -26
  14. package/src/bundled-plugins/github-cli-auth/git-command.ts +224 -36
  15. package/src/bundled-plugins/github-cli-auth/graphql-auth-nudge.ts +9 -9
  16. package/src/bundled-plugins/github-cli-auth/index.ts +110 -31
  17. package/src/bundled-plugins/github-cli-auth/token-class.ts +4 -6
  18. package/src/bundled-plugins/memory/README.md +9 -0
  19. package/src/bundled-plugins/memory/vector/embed-threads.ts +36 -0
  20. package/src/bundled-plugins/memory/vector/embedder.ts +19 -1
  21. package/src/bundled-plugins/researcher/write-report.ts +3 -3
  22. package/src/bundled-plugins/security/policies/private-surface-read.ts +610 -50
  23. package/src/bundled-plugins/security/policies/ssrf.ts +119 -34
  24. package/src/channels/adapters/github/inbound.ts +11 -0
  25. package/src/channels/adapters/github/index.ts +48 -54
  26. package/src/channels/adapters/github/reconcile-cooldown-store.ts +178 -0
  27. package/src/channels/adapters/github/reconcile-open-prs.ts +31 -0
  28. package/src/channels/adapters/slack-bot-edit.ts +85 -20
  29. package/src/channels/adapters/slack-bot.ts +7 -2
  30. package/src/channels/adapters/slack-format.ts +245 -0
  31. package/src/channels/github-token-bridge.ts +2 -3
  32. package/src/container/docker-app.ts +42 -13
  33. package/src/container/start.ts +13 -0
  34. package/src/cron/bridge.ts +9 -2
  35. package/src/doctor/checks.ts +5 -2
  36. package/src/git/bound-gc-config.ts +82 -0
  37. package/src/hostd/spawn.ts +133 -47
  38. package/src/init/dockerfile.ts +13 -0
  39. package/src/inspect/replay.ts +21 -9
  40. package/src/inspect/session-list.ts +71 -33
  41. package/src/mcp/tools.ts +1 -0
  42. package/src/network/internal-destinations.ts +267 -0
  43. package/src/plugin/types.ts +3 -2
  44. package/src/sandbox/env-exposure.ts +18 -15
  45. package/src/sandbox/policy.ts +5 -3
  46. package/src/sandbox/privileged-runtime.ts +1 -0
  47. package/src/skills/typeclaw-config/SKILL.md +22 -6
  48. package/src/skills/typeclaw-permissions/SKILL.md +22 -9
  49. package/src/skills/typeclaw-plugins/SKILL.md +37 -0
  50. package/src/usage/scan.ts +11 -2
  51. package/typeclaw.schema.json +5 -0
  52. package/src/git/secret-history.ts +0 -371
@@ -160,6 +160,15 @@ The embedding model (`Xenova/multilingual-e5-base`) caps inputs at **512 tokens*
160
160
  1. **Deterministic bound at embed time (primary, always on).** `embed()` calls `boundEmbeddableText` on every input before the tokenizer sees it, trimming to a character budget derived from the same conservative per-script token estimate (CJK counted 1:1, Latin ~3.5 chars/token; the tokenizer is never loaded, preserving the embedder's lazy-load). The cut is now explicit and owned, not a hidden tokenizer side effect, and the leading heading/belief sentence — the load-bearing retrieval signal — always survives because it comes first. This guarantees no silent loss even for inputs nothing ever rewrites (raw legacy prose, long queries).
161
161
  2. **Compaction routed to dreaming (remediation).** Bounding keeps retrieval correct but the bounded tail is still not embedded. The real fix is to make the shard fit. On each run the dreaming handler computes which topic shards are over budget (measuring `topicPassage(...).text` — the same citation-stripped string the embedder bounds — so the flag matches what is actually truncated) and renders an "Over the embedding budget" table into the subagent's prompt, alongside the strength-signal table. Dreaming rule 8 instructs it to compact those shards into the one-belief-sentence form **while preserving every `fragments:`/`superseded:` citation** — shrinking prose, never dropping a citation (a dropped citation would trip the citation-superset revert, leaving the shard both reverted and still over budget). The signal is gated on the vector index (`memory/.vectors/index.db`) existing; before the first index exists, the table is suppressed. The `over_budget=N` count rides the `[dreaming] done` line.
162
162
 
163
+ ## Embedder resource tuning
164
+
165
+ The embedding model is a single onnxruntime-node session shared process-wide (`vector/embedder.ts`, `getEmbedder()` singleton). Two session options bound its memory footprint, since the container runs with no docker `--cpus` limit:
166
+
167
+ - **Adaptive intra-op threads** (`vector/embed-threads.ts`). ORT's default (`intraOpNumThreads: 0`) sizes the shared session's intra-op thread pool to one worker per visible host core, each holding its own activation workspace for the singleton's lifetime. `resolveEmbedThreadCount()` caps this at half the visible cores, clamped to `[1, 4]`. The pool is session-level and shared — the transformers.js Node path calls `session.run()` directly (concurrent `Run()` is allowed; the serialization only applies to the web backend), so concurrent embeds contend for it. Capping trades some potential intra-op parallelism (per embed and across overlapping embeds) for RSS bounded by the cap rather than by host core count; for this workload of short per-turn query embeds that trade is worth it.
168
+ - **CPU mem arena off** (`enableCpuMemArena: false`). ORT's arena never returns memory to the OS; for the intermittent per-turn query embed it would inflate steady-state RSS. Disabling frees per-embed at a negligible per-call malloc cost.
169
+
170
+ Neither is configurable — the adaptive default is intended to be correct on any host.
171
+
163
172
  ## How `session.idle` works
164
173
 
165
174
  Core fires `session.idle` immediately after every `session.prompt()` completion. The plugin owns the debounce: a `Map<sessionId, Timeout>` reset on every event. When the timer fires, the plugin spawns `memory-logger` for that session — unless the min-delta gate suppresses the spawn (see below).
@@ -0,0 +1,36 @@
1
+ import { availableParallelism, cpus } from 'node:os'
2
+
3
+ // The container is launched with no docker `--cpus` limit, so onnxruntime-node's
4
+ // default `intraOpNumThreads` (0 = auto) sizes the shared session's intra-op
5
+ // thread pool to one worker PER visible host core — each with its own activation
6
+ // workspace, held for the life of the embedder singleton. On an 11-core host
7
+ // that is ~11 thread-local buffers, a real contributor to the container's RSS.
8
+ // The pool is session-level and shared: the transformers.js Node path calls
9
+ // `session.run()` directly (concurrent `Run()` is allowed — serialization only
10
+ // applies to the web backend), so concurrent embeds from overlapping turns
11
+ // contend for it. Capping is a deliberate trade: it caps intra-op parallelism
12
+ // (per embed and across overlapping embeds) in exchange for RSS bounded by the
13
+ // cap instead of by host core count. The workload is short per-turn query embeds
14
+ // (tens of ms each), so bounding memory is the priority over peak parallelism.
15
+ export const DEFAULT_MAX_EMBED_THREADS = 4
16
+
17
+ // Halve the visible cores (leaves headroom for the event loop and concurrent
18
+ // sessions), then clamp into [1, cap]. `min 1` handles a single-core box; the
19
+ // cap stops the many-core blowup.
20
+ export function resolveEmbedThreadCount(input?: { cap?: number; parallelism?: number }): number {
21
+ const cap = input?.cap ?? DEFAULT_MAX_EMBED_THREADS
22
+ const parallelism = input?.parallelism ?? detectParallelism()
23
+ const halved = Math.floor(parallelism / 2)
24
+ return Math.min(Math.max(halved, 1), cap)
25
+ }
26
+
27
+ function detectParallelism(): number {
28
+ try {
29
+ const n = availableParallelism()
30
+ if (Number.isInteger(n) && n >= 1) return n
31
+ } catch {
32
+ // availableParallelism is Node 18.14+/20+; fall back to cpus() on older runtimes.
33
+ }
34
+ const count = cpus().length
35
+ return count >= 1 ? count : 1
36
+ }
@@ -16,6 +16,7 @@ import {
16
16
  import { getResolvedTransformersVersion } from '@/models/transformers-version'
17
17
 
18
18
  import { homeRoot } from '../../../hostd/paths'
19
+ import { resolveEmbedThreadCount } from './embed-threads'
19
20
  import { type BoundedText, boundEmbeddableText, MAX_MODEL_TOKENS } from './truncation'
20
21
 
21
22
  // Re-exported for the vector subsystem's existing imports. The canonical
@@ -99,7 +100,24 @@ export class Embedder {
99
100
  // miss, or silently loads a stale variant. Fails loudly with a refresh hint.
100
101
  await verifyModelCache()
101
102
  configureTransformers(env)
102
- const extractor = await pipeline('feature-extraction', MODEL_NAME, { local_files_only: true, dtype: MODEL_DTYPE })
103
+ const extractor = await pipeline('feature-extraction', MODEL_NAME, {
104
+ local_files_only: true,
105
+ dtype: MODEL_DTYPE,
106
+ // onnxruntime-node session tuning (transformers.js forwards session_options
107
+ // straight into the native InferenceSession). Two memory levers:
108
+ // - intraOpNumThreads: capped adaptively instead of ORT's default (0 =
109
+ // one worker per visible host core), each of which holds its own
110
+ // activation workspace for the singleton's lifetime.
111
+ // - enableCpuMemArena: off — ORT's arena never returns memory to the OS,
112
+ // so for the intermittent per-turn query embed it inflates steady-state
113
+ // RSS with no benefit. Disabling frees per-embed, at a small per-call
114
+ // malloc cost that is negligible next to the ~tens-of-ms forward pass.
115
+ session_options: {
116
+ intraOpNumThreads: resolveEmbedThreadCount(),
117
+ interOpNumThreads: 1,
118
+ enableCpuMemArena: false,
119
+ },
120
+ })
103
121
  return new Embedder(extractor)
104
122
  }
105
123
 
@@ -31,8 +31,8 @@ const writtenBySession = new Map<string, true>()
31
31
  // - the parent dir's realpath must equal the real workspace/public dir, which
32
32
  // blocks `workspace -> /agent/.env` style symlink escapes that a lexical
33
33
  // check would follow,
34
- // - the file is created with O_EXCL, so an existing file or a planted
35
- // final-path symlink is rejected rather than clobbered or followed,
34
+ // - the runtime descriptor-anchors the parent, then the file is created with
35
+ // O_EXCL, so an existing file or planted final-path symlink is rejected,
36
36
  // - a second write in the same session is rejected (one report per spawn),
37
37
  // - the schema is strict, so an `acknowledgeGuards` field is rejected, not
38
38
  // silently stripped.
@@ -52,7 +52,7 @@ Write to \`public/\` instead of \`workspace/\` when your resolved role lacks \`f
52
52
  .describe('Absolute path: <agent>/workspace/research-<slug>.md or <agent>/public/research-<slug>.md'),
53
53
  content: z.string().describe('The full markdown report body.'),
54
54
  }),
55
- fileOperands: { output: ['path'] },
55
+ fileOperands: { create: ['path'] },
56
56
  async execute(args: WriteReportArgs, ctx: ToolContext) {
57
57
  if (writtenBySession.has(ctx.sessionId)) {
58
58
  throw new Error('A report has already been written for this session. You may write exactly one report.')