@yahyahammad/venastine 0.2.0 → 0.3.8

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (129) hide show
  1. package/.env.example +19 -14
  2. package/CONFIG_ARCHITECTURE.md +2002 -0
  3. package/NOTICE +1 -1
  4. package/README.md +233 -94
  5. package/agents/builtin/build.md +73 -0
  6. package/agents/builtin/general.md +80 -0
  7. package/agents/builtin/plan.md +23 -5
  8. package/agents/builtin/test.md +65 -0
  9. package/agents/builtin/writer.md +67 -0
  10. package/agents/manager.py +261 -195
  11. package/agents/subagent_tool.py +277 -43
  12. package/agents/tui_commands.py +7 -12
  13. package/bin/venastine.mjs +528 -382
  14. package/bin/venastine.py +71 -0
  15. package/cli_commands.py +835 -0
  16. package/config.py +95 -1198
  17. package/config.yaml +1090 -0
  18. package/config_edit.py +1301 -0
  19. package/config_schema.py +966 -0
  20. package/config_update.py +749 -0
  21. package/core/agent_activity.py +158 -20
  22. package/core/ansi.py +452 -0
  23. package/core/approval.py +27 -5
  24. package/core/call_view.py +364 -0
  25. package/core/cancel.py +121 -0
  26. package/core/chatgpt_auth.py +747 -0
  27. package/core/claims_view.py +107 -0
  28. package/core/client.py +1238 -82
  29. package/core/commands.py +454 -0
  30. package/core/compaction.py +830 -66
  31. package/core/config_loader.py +516 -97
  32. package/core/connect.py +686 -0
  33. package/core/events.py +68 -6
  34. package/core/file_mentions.py +280 -0
  35. package/core/interaction.py +87 -4
  36. package/core/line_pattern.py +128 -0
  37. package/core/loop.py +1516 -440
  38. package/core/memory.py +120 -8
  39. package/core/pipeline_models.py +23 -9
  40. package/core/provider_errors.py +234 -0
  41. package/core/reasoning/confidence_scoring.py +70 -54
  42. package/core/reasoning/json_retry.py +19 -1
  43. package/core/reasoning/orchestrator.py +232 -53
  44. package/core/reasoning/output_writer.py +90 -31
  45. package/core/reasoning/payload_validation.py +133 -7
  46. package/core/reasoning/pipeline_storage.py +427 -361
  47. package/core/reasoning/review.py +659 -615
  48. package/core/reasoning/scholar.py +38 -14
  49. package/core/reasoning/source_corpus.py +110 -4
  50. package/core/replay.py +158 -26
  51. package/core/session.py +36 -12
  52. package/core/session_view.py +259 -0
  53. package/core/session_wake.py +259 -0
  54. package/core/shell_sessions.py +1553 -0
  55. package/core/workspace_trust.py +1 -1
  56. package/credentials.py +308 -180
  57. package/database.py +82 -1
  58. package/{PRIVACY.md → docs/PRIVACY.md} +19 -7
  59. package/docs/SECURITY.md +108 -0
  60. package/docs/THIRD_PARTY_NOTICES.md +148 -0
  61. package/env_secrets.py +118 -77
  62. package/json_store.py +236 -16
  63. package/logging_setup.py +121 -11
  64. package/main.py +1671 -199
  65. package/mcp_client/client.py +257 -23
  66. package/mcp_client/config.py +484 -379
  67. package/mcp_client/registration.py +163 -2
  68. package/memories/manager.py +15 -0
  69. package/memories/tui_commands.py +7 -12
  70. package/package.json +234 -4
  71. package/project_init/config_files.py +4 -0
  72. package/project_init/generator.py +201 -37
  73. package/project_init/manifest.py +418 -406
  74. package/project_init/tui_commands.py +7 -4
  75. package/prompts/revalidate.md +1 -1
  76. package/prompts/revise.md +17 -15
  77. package/prompts/system_prompts.py +64 -10
  78. package/providers.json.example +94 -92
  79. package/pyproject.toml +51 -2
  80. package/requirements.txt +252 -29
  81. package/safety/policy_enforcement.py +353 -17
  82. package/security/askpass.py +419 -0
  83. package/security/capability.py +37 -7
  84. package/security/confine.py +484 -0
  85. package/security/hosted.py +182 -0
  86. package/security/permissions.py +54 -7
  87. package/security/posture.py +124 -12
  88. package/security/protected_paths.py +198 -6
  89. package/security/sandbox.py +3262 -139
  90. package/security/secrets.py +855 -0
  91. package/skills/tui_commands.py +3 -4
  92. package/storage.py +368 -35
  93. package/tools/base.py +465 -305
  94. package/tools/builtin/_math_common.py +306 -16
  95. package/tools/builtin/_net_common.py +267 -61
  96. package/tools/builtin/arxiv.py +681 -223
  97. package/tools/builtin/ask_user.py +231 -213
  98. package/tools/builtin/discrete_math.py +35 -4
  99. package/tools/builtin/fetch_url.py +239 -21
  100. package/tools/builtin/file_ops.py +341 -33
  101. package/tools/builtin/geometry.py +184 -148
  102. package/tools/builtin/get_time.py +2 -2
  103. package/tools/builtin/linear_algebra.py +66 -23
  104. package/tools/builtin/load_skill.py +10 -0
  105. package/tools/builtin/logic.py +171 -12
  106. package/tools/builtin/pin.py +46 -12
  107. package/tools/builtin/probability_stats.py +224 -228
  108. package/tools/builtin/project_docs.py +80 -21
  109. package/tools/builtin/remember.py +21 -4
  110. package/tools/builtin/shell.py +586 -67
  111. package/tools/builtin/shell_sessions.py +613 -0
  112. package/tools/builtin/symbolic_math.py +80 -10
  113. package/tools/builtin/todo.py +5 -3
  114. package/tools/builtin/web_search.py +329 -151
  115. package/tools/context.py +23 -0
  116. package/tools/isolation.py +387 -314
  117. package/tools/registry.py +347 -30
  118. package/tui/app.py +4645 -438
  119. package/tui/app.tcss +185 -2
  120. package/tui/commands.py +10 -154
  121. package/tui/diffs.py +286 -265
  122. package/tui/hosted_driver.py +102 -0
  123. package/tui/markdown.py +60 -10
  124. package/tui/meters.py +103 -7
  125. package/tui/screens.py +346 -92
  126. package/tui/themes.py +59 -19
  127. package/tui/widgets.py +1575 -111
  128. package/SECURITY.md +0 -88
  129. package/THIRD_PARTY_NOTICES.md +0 -100
@@ -0,0 +1,2002 @@
1
+ # CONFIG_ARCHITECTURE.md
2
+
3
+ **Why every value in `config.yaml` is what it is.**
4
+
5
+ `config.yaml` holds the harness's tunable values and says, in a line or two
6
+ per key, what each one *does*. This file is the other half: the argument. Most
7
+ of these numbers were chosen against a specific measurement or a specific
8
+ incident, several were reversed once, and a few are load-bearing in ways that
9
+ are invisible from the value alone. **Read the entry for a key before changing
10
+ it.** More than one of these has been "corrected" back to a value it was
11
+ deliberately moved away from.
12
+
13
+ The prose here was `config.py`'s, from the years when the values and their
14
+ history lived in one 1,210-line module of which 788 lines were comment. It was
15
+ carried over verbatim. Section headings are `config.py`'s own; the entry for
16
+ each key is named by its **`config.yaml` key**, so a key you are editing leads
17
+ straight here.
18
+
19
+ **Two things about that port, both worth knowing before you trust a
20
+ sentence here.** First, it is no longer verbatim in five places, each
21
+ marked and dated: the `tool_compute_timeout_s` arithmetic described a
22
+ value the code has not had for some time, three entries told the reader to
23
+ set something in `config.py` where an edit is now silently overwritten at
24
+ import, and one described a `settings.json` key that does not exist. The
25
+ rest is unchanged, so a diff against `config.py` before the migration is
26
+ still the way to check it. Second, **this file is hand-maintained from
27
+ here.** It was produced once by a migration script that was not kept;
28
+ nothing regenerates it, so a value's rationale moves when someone moves
29
+ it.
30
+
31
+ ## How configuration is laid out
32
+
33
+ | File | Holds | Tier |
34
+ |---|---|---|
35
+ | `config.yaml` | every tunable value | harness, one location |
36
+ | `config_schema.py` | the schema, the environment overrides, the loader | -- |
37
+ | `config.py` | the same values as module attributes, for the code | -- |
38
+ | `~/.config/venastine/settings.json` | the user's overrides | user |
39
+ | `<project>/.venastine/settings.json` | a project's overrides, if trusted | project |
40
+
41
+ `settings.json` is merged across tiers by `core/config_loader.py`, and for the
42
+ keys it knows a **project's file beats the user's** (D29) because it is what
43
+ you authored versus what arrived with a directory you cloned. That precedence
44
+ is the reason for the two rules below.
45
+
46
+ ### One location, and no override variable
47
+
48
+ `config.yaml` resolves from `config_schema.__file__`, beside `config.py`. There
49
+ is no user-tier copy, no project-tier copy, and **no environment variable that
50
+ redirects it**. That absence is a security property rather than an oversight.
51
+
52
+ `settings.json` refuses `shell_approval_mode`, `ensemble_models`,
53
+ `critic_model`, `embedder_model` and `research.granted_tools` **by name** --
54
+ R12, E2, G7 and SQ7, each argued in `ROADMAP_v2.md` -- and the argument is
55
+ always the same shape: a project's `settings.json` beats the user's and arrives
56
+ with a clone, so a key there would let a cloned repository set this harness's
57
+ approval posture or point its research passes at a provider you never chose. A
58
+ file that ships inside the harness, in exactly one place, carries none of that
59
+ risk. An `AGENT_CONFIG_FILE` variable would hand it straight back, and it would
60
+ look like the obvious sibling of `AGENT_ENV_FILE` and `APP_DB_PATH`. It is not:
61
+ those redirect *state*, and this is a shipped asset.
62
+
63
+ ### The one location survives an npm update (batch 86)
64
+
65
+ `config.yaml` ships inside the package, and **npm replaces a package directory wholesale** -- measured with `npm pack` and two installs into a scratch prefix: an edited `max_tokens: 31337` came back as the shipped `16000`, and the new version's `max_iterations` default arrived with it. So the rule above had a cost nobody had paid attention to: every value an npm user had chosen was discarded at each update.
66
+
67
+ The fix is a merge, **not a second location**. `config_update.py` keeps the pristine copy of the installed version and a mirror of the user's own file in `~/.config/venastine/config-state/`, and after an update it re-applies the difference onto whatever npm has just written. The live document is still the one inside the harness, still resolved from `__file__`, still with no variable that redirects it -- the state directory holds HISTORY, which is state, and state has always been redirectable here. A reader who finds a `config.yaml` under the home directory and concludes the user tier is back has found the mirror, which nothing reads at startup.
68
+
69
+ Two consequences worth stating. Keys the user never touched adopt the new version's defaults, and keys the new version adds arrive at theirs -- the merge starts FROM the shipped file, so that half needs no code. And the nine AUTHORITY keys are restored like any other, each one named in the report the launch prints: an update must neither silently loosen nor silently tighten the posture, and the file surviving is the status quo.
70
+
71
+ ### Read once, at startup
72
+
73
+ `config.py` imports `config_schema`, which reads the file, folds the
74
+ environment into the raw document, and validates the result — **once**, in
75
+ that order. The environment goes in *before* validation so that there is one
76
+ model with the overrides already in it: applied afterwards they would reach
77
+ only the namespace `config.py` publishes, and `config_schema.current()` would
78
+ quietly disagree with `config.<NAME>` about any key that has a variable.
79
+ Nothing re-reads the file. `security/posture.py`
80
+ depends on that: `config` binding at import is what makes
81
+ `os.environ[...] = ...` unable to move the security posture, where anything
82
+ reading `os.environ` at call time would follow the mutation (UN1, and batch
83
+ 37's asymmetry). `tools/builtin/shell.py` validates the approval mode at its
84
+ own import, so the values must be final before it runs.
85
+
86
+ Editing `config.yaml` while the harness is running therefore changes nothing
87
+ until the next launch.
88
+
89
+ ### The keys marked AUTHORITY
90
+
91
+ Nine keys decide what the harness may do to its host, or name a provider it
92
+ sends your content to. Nothing *unattended* can change them -- not a tool
93
+ call, not a settings file, not an environment variable.
94
+ `config_schema.HARNESS_AUTHORITY_KEYS` is the machine-readable list, and a
95
+ test holds it against `security/posture.Posture`'s fields and the five
96
+ by-name `settings.json` rejections. (Four ROADMAP markers, five keys:
97
+ `critic_model` and `embedder_model` share SQ7.)
98
+
99
+ **Batch 84 gave `/config` a route to them, behind a confirmation.** Until
100
+ then the rule was absolute: no in-session surface at all, a slash command
101
+ included. What changed is the reading of *why*. The `settings.json`
102
+ rejections argue from a file that arrives with a directory you cloned, and
103
+ the environment argument from a variable a subprocess can set; neither
104
+ describes a person typing at the prompt, who is the harness operator and is
105
+ the one `config.yaml` has always been edited by. The model cannot reach the
106
+ route -- `tui/commands.dispatch` is called from the prompt's submit handler
107
+ and nowhere else.
108
+
109
+ **What the file tools do here, stated accurately** (corrected in batch 87).
110
+ This said "the file tools refuse the harness install tree", and they do not:
111
+ `security/protected_paths.PROTECTED_SEGMENTS` is `{".venastine"}`, so
112
+ `file_ops._protected_path_error` denies a path carrying that segment and
113
+ returns None for `<install>/config.yaml`. The control that actually holds is
114
+ weaker and still real — `write` and `edit` ship **denied** in
115
+ `tool_permissions` and cannot be enabled at runtime, and `protected_paths`
116
+ makes a workspace that overlaps the harness tree a *startup error*, so the
117
+ install tree is never inside the workspace and a write there is
118
+ approval-gated rather than auto-approved. `protected_paths.py` says why the
119
+ difference is worth a paragraph: a half-understood control is worse than
120
+ none.
121
+
122
+ The compensating control is that the confirmation says what the key
123
+ PERMITS, not "are you sure": `config_edit.AUTHORITY_EFFECT` carries one
124
+ sentence per key and a test fails if a key has none. And the change still
125
+ does not take effect in the running process -- it is written to the file
126
+ and applied by relaunching, so the frozen posture is never edited under a
127
+ session that has already read it.
128
+
129
+ `ensemble_mode` is deliberately *not* one of them. It is a mode, persistable in
130
+ `settings.json`, and the worst it can do is spend more of a provider you
131
+ already chose -- the line `core/config_loader.py` draws between authority and
132
+ cost.
133
+
134
+ ### Two values with no key in the file
135
+
136
+ Both are derived in `config_schema.derived_values()` because neither is a static
137
+ default, and giving either a key would change behaviour:
138
+
139
+ - **`OUTPUT_DIR`** is `AGENT_OUTPUT_DIR`, else `<AGENT_WORKSPACE or ".">/output`
140
+ -- one expression with no branch in it. Note the `"."` default here against
141
+ `workspace_dir`'s `"./workspace"`: an unset variable still means `./output`
142
+ rather than `./workspace/output`. A key would have to be ignored whenever
143
+ `AGENT_WORKSPACE` is named, which is the branch this form exists to avoid.
144
+ - **`WORKSPACE_DIR_EXPLICIT`** answers "was this workspace named as the project?",
145
+ and since batch 111 it has **two inputs, OR'd**: the **presence** of
146
+ `AGENT_WORKSPACE`, never its value, and the `workspace_is_project` key. The
147
+ presence rule is why the variable cannot be a value test -- the default
148
+ `./workspace` is a subdirectory of the launch directory, so comparing values
149
+ would move the project one level down for everyone who set nothing, and YAML
150
+ cannot express "was this variable named?". The key exists because that left a
151
+ person editing `config.yaml` with no way to say it at all (TECHNICAL_DEBT 28),
152
+ which cost them a refused launch telling them to set a variable in a session
153
+ where they had just set the key. Still live from batch 107: an unset variable
154
+ and an undeclared key make the LAUNCH DIRECTORY the project, and
155
+ `protected_paths.check_project` refuses that at startup when it is the install
156
+ tree (TECHNICAL_DEBT 26).
157
+
158
+ ### The environment variables
159
+
160
+ `AGENT_MODEL`, `APP_DB_PATH`, `AGENT_WORKSPACE`, `AGENT_SHELL` and
161
+ `AGENT_SANDBOX_IMAGE` each override one key; `AGENT_OUTPUT_DIR` and
162
+ `AGENT_WORKSPACE` feed the two derived values above.
163
+ `config_schema.ENV_OVERRIDES` is the table, and `HARNESS_ENV_VARS` the full
164
+ list, so neither has to be maintained by hand anywhere else. An override is
165
+ folded into the document before validation, so it goes through the same type
166
+ check the file's own value does.
167
+
168
+ Note that there is **no variable for `config.yaml`'s own path** — see *One
169
+ location, and no override variable* above. That is the one absence in this
170
+ list which is a decision rather than an omission.
171
+
172
+
173
+ ## Contents
174
+
175
+ - [Model and loop](#model-and-loop)
176
+ - [Loop control](#loop-control)
177
+ - [Subagents (ROADMAP_v2 §18)](#subagents-roadmap_v2-18)
178
+ - [Deep research pipeline](#deep-research-pipeline)
179
+ - [Spend cap (batch 27, #4)](#spend-cap-batch-27-4)
180
+ - [Ensemble mode (ROADMAP §10, as redesigned by its revisit)](#ensemble-mode-roadmap-10-as-redesigned-by-its-revisit)
181
+ - [Sampling-parameter support (ROADMAP_v2 §16 prerequisite)](#sampling-parameter-support-roadmap_v2-16-prerequisite)
182
+ - [Critic-model routing (ROADMAP §11)](#critic-model-routing-roadmap-11)
183
+ - [Embedder routing (ROADMAP_v2 §45, SQ2/SQ7)](#embedder-routing-roadmap_v2-45-sq2sq7)
184
+ - [Source authority: domain classes (ROADMAP_v2 §45, SQ4)](#source-authority-domain-classes-roadmap_v2-45-sq4)
185
+ - [The sources/ artifact (ROADMAP_v2 §45, SQ8)](#the-sources-artifact-roadmap_v2-45-sq8)
186
+ - [Scholarly authority: OpenAlex (ROADMAP_v2 §45, SQ5/SQ9)](#scholarly-authority-openalex-roadmap_v2-45-sq5sq9)
187
+ - [Reasoning effort (ROADMAP_v2 §16; default changed batch 25, #139)](#reasoning-effort-roadmap_v2-16-default-changed-batch-25-139)
188
+ - [Database](#database)
189
+ - [Output artifacts](#output-artifacts)
190
+ - [File-ops workspace (ROADMAP §6)](#file-ops-workspace-roadmap-6)
191
+ - [/init (ROADMAP_v2 §24)](#init-roadmap_v2-24)
192
+ - [MCP teardown (ROADMAP_v2 §37 F6, #64)](#mcp-teardown-roadmap_v2-37-f6-64)
193
+ - [Shell / sandbox (ROADMAP §7)](#shell-sandbox-roadmap-7)
194
+ - [Output redaction (#167/#49, batch 20)](#output-redaction-16749-batch-20)
195
+ - [Compaction (ROADMAP_v2 §21)](#compaction-roadmap_v2-21)
196
+ - [Catalog text (ROADMAP_v2 §32, A5)](#catalog-text-roadmap_v2-32-a5)
197
+ - [Summaries, references and the checklist (ROADMAP_v2 §21c / §23)](#summaries-references-and-the-checklist-roadmap_v2-21c-23)
198
+ - [Context windows (ROADMAP_v2 §21 M1, as amended by batches 30 and 44)](#context-windows-roadmap_v2-21-m1-as-amended-by-batches-30-and-44)
199
+ - [Tool permissions and approvals (ROADMAP_v2 §15, D24)](#tool-permissions-and-approvals-roadmap_v2-15-d24)
200
+
201
+ ---
202
+
203
+ ## Model and loop
204
+
205
+ ### `model_name`
206
+
207
+ --- Model / loop settings -- these were missing; call_model, RunAgentLoop,
208
+ and database.py all depend on them ---
209
+
210
+ ### `max_tokens`
211
+
212
+ Per-call output ceiling. Raised from 4096 in ROADMAP_v2 §16: on current
213
+ Anthropic models max_tokens caps THINKING PLUS RESPONSE TEXT together, so
214
+ 4096 truncated answers mid-sentence as soon as reasoning effort was
215
+ enabled. There is no default cumulative budget for it to interact with any
216
+ more (batch 27, #4): spend is uncapped unless settings.json
217
+ max_token_budget says otherwise.
218
+
219
+ ## Loop control
220
+
221
+ ### `max_iterations`
222
+
223
+ Raised from 20 in batch 16 (#45): this is THE default step ceiling -- the
224
+ `or config.MAX_ITERATIONS` fallback at every agent-shaped call site, the
225
+ value an invalid `max_steps:` frontmatter field is repaired to, and the
226
+ ceiling of every chat turn and research pass that does not name its own.
227
+
228
+ ### `model_call_max_retries`
229
+
230
+ Batch 91. Nothing retried a model call before this, and the SDKs cover less
231
+ than they appear to: openai and anthropic retry a request that fails BEFORE
232
+ its stream opens (408/409/429/5xx, connection errors, twice each), but an
233
+ error sent inside a stream that opened with 200 is raised straight up by
234
+ both, and google-genai retries nothing at all unless told to (1.0.0,
235
+ 2.29.0 since batch 172, and 2.32.0 since batch 234, measured on a 503:
236
+ with no `retry_options` it makes one attempt).
237
+ OpenRouter reports an
238
+ overloaded upstream exactly that way -- "The service is temporarily
239
+ unavailable", "Upstream idle timeout exceeded" -- which failed four nested
240
+ subagents in the run that prompted this.
241
+
242
+ Two, so a call gets three attempts. The SDK retries are KEPT (owner
243
+ decision) and happen inside each attempt, so the worst case is nine HTTP
244
+ requests for one call; turning the SDKs' off would lose their retry-after
245
+ handling for the statuses they already cover well. `core/provider_errors.py`
246
+ decides what qualifies -- temporary provider errors only, never a bad
247
+ request, an auth failure or a harness bug -- and `core/loop.py` decides
248
+ WHEN: a run whose output reaches no screen is retried whatever it had
249
+ streamed, a run someone is watching only before its first delta, because a
250
+ transcript row cannot be taken back.
251
+
252
+ ### `model_call_retry_base_delay_s`
253
+
254
+ Batch 91. Three seconds, then six, with a quarter either way of jitter.
255
+ Deliberately slower than the SDKs' half-second start (owner decision): these
256
+ retries sit ABOVE the SDK's own, so by the time one fires the provider has
257
+ already refused several requests in quick succession, and a rate limit met
258
+ with more of the same only extends itself. The jitter is for siblings: the
259
+ failures that prompted this came in pairs from one parallel spawn batch, and
260
+ without it they would retry in the same instant. A server's retry-after can
261
+ lengthen a wait, never shorten it, up to 60 seconds.
262
+
263
+ ## Subagents (ROADMAP_v2 §18)
264
+
265
+ ### `subagent_max_depth`
266
+
267
+ Maximum spawn_subagent nesting. The counter lives on
268
+ ToolContext.subagent_depth; this is the value it is checked against (C3).
269
+
270
+ ### `subagent_max_parallel`
271
+
272
+ §47 slice 8 (NA10). How many calls of one response may run at once, for
273
+ tools whose ToolSpec declares `parallel`. A ceiling, not a target: a
274
+ response naming fewer runs fewer, and a response naming more runs them in
275
+ waves of this size. 1 makes the parallel path behave exactly like the
276
+ sequential one, which is the escape hatch if concurrency is ever suspect.
277
+
278
+ Three because that is the case the design keeps citing -- one turn
279
+ spawning `explore` three times -- and because three children at depth 1,
280
+ each able to nest to SUBAGENT_MAX_DEPTH, is a bounded worst case for the
281
+ serialised approval queue and for the SQLite writers.
282
+
283
+ ## Deep research pipeline
284
+
285
+ ### `max_pipeline_retries`
286
+
287
+ max revise/re-validate loop iterations per claim before fallback
288
+
289
+ ### `max_json_retries`
290
+
291
+ max corrective follow-up attempts when a pass returns malformed JSON
292
+
293
+ (total attempts per pass = MAX_JSON_RETRIES + 1). See ROADMAP §3.
294
+
295
+ ## Spend cap (batch 27, #4)
296
+
297
+ There is NO default spend ceiling any more. The loop's cumulative
298
+ input+output counter is a BILLING meter -- the prompt is re-sent and
299
+ re-billed on every step of a tool-using turn, so it grows quadratically
300
+ in steps, which is correct as billing and meaningless as a size limit.
301
+ Reading it as "how large may a thread get" is the defect TECHNICAL_DEBT
302
+ item 9 recorded; size reasoning reads ModelResponse.turn_new_tokens and
303
+ memory.last_input_tokens now, and compaction keys off measured context
304
+ size (it never keyed off this number).
305
+
306
+ A user who wants a hard ceiling sets settings.json `max_token_budget`
307
+ (int tokens per _run() invocation -- one user turn, one research pass,
308
+ one /init run). core/config_loader.spend_cap() resolves it, and the
309
+ headroom advisory in effective_compaction() speaks only when a cap is
310
+ configured. Hard limits at the provider remain the stronger instrument:
311
+ they see the same bill and cannot be forgotten by a code path. The three
312
+ constants this file used to carry (MAX_TOKEN_BUDGET 250k,
313
+ RESEARCH_PASS_TOKEN_BUDGET 1M for passes, INIT_TOKEN_BUDGET 1M) died
314
+ here -- the two 1M envelopes existed only because the 250k chat value
315
+ was misread as bounding pass context (the Pass-1-with-14-tool-calls
316
+ incident in DEVLOG), and with no default cap there is nothing to work
317
+ around.
318
+
319
+ Deferred for now (core sequential pipeline only, per current scope):
320
+ ```text
321
+ (none remaining -- ensemble_mode/ensemble_n built in ROADMAP §10,
322
+ critic_model built in ROADMAP §11)
323
+ ```
324
+
325
+ ## Ensemble mode (ROADMAP §10, as redesigned by its revisit)
326
+
327
+ ### `ensemble_mode`
328
+
329
+ Run Pass 1 once per entry below, then extract the union of claims across
330
+ candidates, and send a factual claim at most half of them asserted to Pass
331
+ 6a/6b to be checked (ROADMAP_v3 §90). Off by default.
332
+
333
+ DIVERSITY COMES FROM DIFFERENT MODELS, NOT DIFFERENT SAMPLING (E1). §10
334
+ originally raised `temperature` on N runs of one model. That could not work
335
+ on current Anthropic models, which reject sampling parameters outright (see
336
+ MODELS_REJECTING_SAMPLING_PARAMS), so the section was built, documented as
337
+ working, and could not execute against this harness's own default model.
338
+ ENSEMBLE_TEMPERATURE is deleted rather than repaired: it was an ABSOLUTE
339
+ value being used as though it were a raise, so what "1.0" meant differed
340
+ per provider, and so did how much diversity a run actually got.
341
+
342
+ The deeper reason is §11's, verbatim: "a model checking its own output for
343
+ errors shares that model's blind spots." N samples of one model agree most
344
+ confidently on that model's systematic errors, which is exactly where a
345
+ research harness needs agreement to mean something. Different models do
346
+ not share blind spots, so their agreement is real evidence.
347
+
348
+ N is len(ENSEMBLE_MODELS) -- derived, never configured separately (E3). The
349
+ denominator of a confidence score must not be able to disagree with the
350
+ roster that produced it.
351
+
352
+ `config.yaml` ONLY, deliberately -- there is no settings.json key for
353
+ this, following `critic_model` (E2). Trusting a cloned repo already lets it pick
354
+ the provider and multiply pipeline cost; a project-tier list of N providers
355
+ is that same grant multiplied by N.
356
+
357
+ Fewer than two DISTINCT (provider_name, model) pairs is refused, not
358
+ tolerated: one model cannot disagree with itself, so every claim would
359
+ score maximal consistency and Pass 5 would read that as confidence. That
360
+ is the original defect, and repeating the same entry N times is the way to
361
+ recreate it through this config.
362
+
363
+ ### `ensemble_models`
364
+
365
+ Example, as it is written in `config.yaml` -- a sequence of mappings,
366
+ not the Python assignment this example used to show:
367
+
368
+ ```yaml
369
+ ensemble_models:
370
+ - {provider_name: ANTHROPIC, model: claude-opus-5}
371
+ - {provider_name: OPENAI, model: gpt-5.1}
372
+ - {provider_name: GOOGLE, model: gemini-2.5-pro}
373
+ ```
374
+
375
+ ## Sampling-parameter support (ROADMAP_v2 §16 prerequisite)
376
+
377
+ ### `models_rejecting_sampling_params`
378
+
379
+ Models that reject temperature/top_p/top_k. Current Anthropic models
380
+ removed these parameters outright (any value returns HTTP 400); Sonnet 5
381
+ rejects non-default values. Steering is expected to happen through
382
+ prompting and MODEL_EFFORT_LEVELS instead.
383
+
384
+ A static table rather than a capability query: the Models API capability
385
+ tree does not report sampling support, so unlike effort levels (which ARE
386
+ queryable, see MODEL_EFFORT_LEVELS) there is nothing to ask. Same posture
387
+ as §21's MODEL_CONTEXT_WINDOWS -- honest about being incomplete, and
388
+ incompleteness is safe here because the failure is a loud 400, not a
389
+ silent wrong answer.
390
+
391
+ EXCEPT FOR A MODEL THAT IGNORES THEM, which is why two Google names are
392
+ listed (ROADMAP_v3 §87, A82). Gemini 3.6 Flash and 3.5 Flash-Lite ignore
393
+ the parameters: no error, no effect. Google says every later Gemini will do
394
+ the same and then refuse them with a 400, so a new Gemini belongs here when
395
+ it ships. OpenAI's reasoning models are NOT listed: core/client.py drops a
396
+ temperature on OPENAI whenever a reasoning effort other than "none" is
397
+ sent, which is OpenAI's documented rule and needs no names.
398
+
399
+ ON ANTHROPIC THE LIST IS NOT CONSULTED, since batch 234 (A82, amended).
400
+ anthropic 2.x sends API version 2026-10-21, which refuses all three
401
+ parameters on every model, even at the default, so core/client.py drops a
402
+ temperature there on every model. The claude names stay, and the five
403
+ 5.x-generation ones were added (Fable 5.1, Mythos 5.1, Opus 5.5, Sonnet
404
+ 5.5, Haiku 5.5), for Claude reached by its bare name through an
405
+ OpenAI-compatible gateway, which still sends whatever it is given.
406
+
407
+ ## Critic-model routing (ROADMAP §11)
408
+
409
+ ### `critic_model`
410
+
411
+ Route the critic/grounding passes (3a, 3b, 6c) to a different model than
412
+ the generator, so a model isn't checking its own output for errors.
413
+ None means every pass uses the same provider/model — no special routing.
414
+ Example to enable: {"provider_name": "OPENAI", "model": "gpt-5.1"}
415
+
416
+ ## Embedder routing (ROADMAP_v2 §45, SQ2/SQ7)
417
+
418
+ ### `embedder_model`
419
+
420
+ The model that turns a claim and a source passage into vectors, so
421
+ `similarity_score` is a cosine rather than a self-report. None -- the
422
+ default -- means the pipeline warns once at launch and falls back to the
423
+ grounding model's own number, under source_grounding.md's anchored
424
+ rubric.
425
+
426
+ `config.yaml` ONLY, with no settings.json key, following `critic_model`
427
+ and `ensemble_models` (E2): choosing a provider is a grant, and this one sends
428
+ claim text and fetched page text to whoever is named. `/embedder` writes
429
+ to the user-tier store in core/pipeline_models.py, which OUTRANKS this.
430
+ Example: {"provider_name": "OPENAI", "model": "text-embedding-3-small"}
431
+
432
+ ### `similarity_calibration`
433
+
434
+ Asymmetric embedders score materially worse without their prefixes and
435
+ produce NO ERROR AT ALL when they are missing, which is why this is a
436
+ table rather than something inferred. Keyed by a substring of the model
437
+ slug, longest match wins; a model matching nothing gets no prefix, which
438
+ is correct for every symmetric embedder (OpenAI's, Cohere's, Voyage's).
439
+
440
+ `task_type` is Google's equivalent and is ignored by every other
441
+ provider. Incomplete on purpose: an unlisted asymmetric model scores
442
+ lower than it should, which is visible in the artifact as a low cosine
443
+ on a source that plainly matches -- and adding a row is the fix.
444
+ Raw cosine is NOT a 0-1 score, and treating it as one is the quiet way
445
+ to publish a wrong number. Unrelated text sits near 0.1 on some models
446
+ and above 0.7 on others, so the same passage would score "irrelevant" on
447
+ one embedder and "supports the claim" on another. This maps the useful
448
+ band onto 0-1: below `floor` is noise, above `ceiling` is as close as
449
+ that model gets.
450
+
451
+ Keyed by a substring of the model slug, longest match wins; anything
452
+ unlisted takes DEFAULT_SIMILARITY_CALIBRATION. The RAW cosine is always
453
+ recorded beside the calibrated score, so a bad row here is visible in
454
+ the artifact and fixable after the fact rather than baked in.
455
+
456
+ ### `default_similarity_calibration`
457
+
458
+ ### `embedder_prefixes`
459
+
460
+ ## Source authority: domain classes (ROADMAP_v2 §45, SQ4)
461
+
462
+ ### `domain_authority_classes`
463
+
464
+ What a source is worth before anything is known about the specific page.
465
+ Two tables, deliberately: a CLASS carries the number, and a SUFFIX names
466
+ which class a host belongs to. A user who thinks preprints deserve more
467
+ edits one number; a user adding their field's own trusted domain edits a
468
+ membership. One combined {suffix: score} table would make the first of
469
+ those a find-and-replace across a hundred rows.
470
+
471
+ THE SIGNAL IS A RESTRICTED REGISTRY, NOT A FAMILIAR SUFFIX. `.gov`,
472
+ `.mil`, `.int` and `.edu` gate registration -- you cannot buy one -- and
473
+ so do `gov.uk`, `ac.uk`, `gc.ca` and their equivalents worldwide, which
474
+ is why source_scoring derives the `<kind>.<cc>` forms rather than
475
+ enumerating them here. `.org` gates nothing at all and never has, so it
476
+ sits barely above `.com`; treating it as trustworthy is the single most
477
+ common version of this mistake.
478
+
479
+ INCOMPLETE ON PURPOSE, and safe when incomplete: an unlisted host takes
480
+ DEFAULT_DOMAIN_AUTHORITY, which is the generic-commercial number. The
481
+ failure mode is a good source scored ordinary, never a bad source scored
482
+ authoritative, and the model's bounded adjustment (below) is the route
483
+ for the first.
484
+
485
+ **Notes on individual entries.**
486
+
487
+ .gov .mil .int .edu, gov.uk, ac.uk...
488
+ WHO, UN, World Bank, OECD, EU
489
+ IETF, W3C, ISO, NIST, IEC
490
+ not peer reviewed; §45 SQ5 refines this
491
+ encyclopaedias -- a pointer to a source
492
+ an unrestricted registry, small bump
493
+
494
+ ### `default_domain_authority`
495
+
496
+ == generic_commercial, for an unknown host
497
+
498
+ ### `domain_authority_suffixes`
499
+
500
+ host suffix -> class. Longest suffix wins, so `blogs.nature.com` can be
501
+ listed separately from `nature.com` if it ever needs to be.
502
+
503
+ **Notes on individual entries.**
504
+
505
+ Restricted registries. The bare TLDs; the `<kind>.<cc>` second-level
506
+ forms are derived in source_scoring.py rather than listed.
507
+ Intergovernmental bodies, whose hosts are ordinary .org/.int names.
508
+ Standards bodies.
509
+ Peer-reviewed publishers and indexes.
510
+ Preprint servers and repositories -- not peer reviewed.
511
+ News organisations with a correction policy and a masthead. An
512
+ ALLOWLIST, because "news site" is not a property of a domain suffix
513
+ and the alternative is scoring every .com that publishes articles.
514
+ Tertiary references -- they summarise sources rather than being one.
515
+ Blog and newsletter platforms: the host says nothing about the author.
516
+ Forums and Q&A. Often correct, never attributable.
517
+ Social media. Speculation and unattributed claims travel here fastest,
518
+ which is the whole reason a claim needs grounding elsewhere.
519
+ Aggregators that republish without attribution, and image boards with
520
+ no text to ground anything in. The measured case: a Pinterest pin
521
+ scored similarity 1.0 on a Nobel Prize claim.
522
+
523
+ ### `authority_adjustment_cap`
524
+
525
+ How far the model may move a computed authority, in either direction
526
+ (§45, SQ4). Small on purpose: it is a correction for what a domain
527
+ cannot see -- a primary source on an ordinary host, an opinion column on
528
+ a masthead -- not a second opinion about the number. An adjustment
529
+ arriving without a reason is discarded rather than clamped.
530
+
531
+ ## The sources/ artifact (ROADMAP_v2 §45, SQ8)
532
+
533
+ ### `persist_source_text`
534
+
535
+ Whether output/<run_id>/sources/ keeps the TEXT each cited page served,
536
+ or only its hash, size and provenance.
537
+
538
+ On by default, because it is what makes a similarity score reproducible
539
+ after the run -- and the directory output_writer has created since §12
540
+ was described by fetch_url.py's own comment for just as long. Everything
541
+ written there has been through redact_output_text on entry to the
542
+ corpus.
543
+
544
+ ROADMAP_v3 §52 (RB1) is what makes that sentence true again for a paged
545
+ document, and it was measured false first. §51 cut a page out of the
546
+ body before the redactor saw it, so a credential straddling character
547
+ 5,000 matched nothing in either page and PG7's append rejoined it here
548
+ -- reproduced end to end into artifact_entries(), which is exactly what
549
+ this directory is written from. fetch_url redacts the whole body before
550
+ it cuts now, so the halves never exist.
551
+
552
+ It is new data at rest, so it has a switch and PRIVACY.md says what it
553
+ holds. Turning it off costs the audit trail, not the scores: the corpus
554
+ still exists in memory for the run that computes them.
555
+
556
+ ## Scholarly authority: OpenAlex (ROADMAP_v2 §45, SQ5/SQ9)
557
+
558
+ ### `scholar_lookup`
559
+
560
+ OFF BY DEFAULT. This is the first network call in this harness made by
561
+ something that is not a registered tool, so it bypasses the approval
562
+ registry and granted_calls.json -- and an out-of-the-box run must make
563
+ no call the user did not ask for. With it off, a paper takes its
564
+ preprint-server or publisher domain class and nothing else.
565
+
566
+ ### `scholar_api_url`
567
+
568
+ ### `scholar_timeout_s`
569
+
570
+ ### `scholar_max_retries`
571
+
572
+ ### `scholar_cache_ttl_s`
573
+
574
+ ### `scholar_mailto`
575
+
576
+ OpenAlex asks for a contact address to put a caller in its "polite pool"
577
+ (higher rate limits). EMPTY BY DEFAULT AND NEVER DERIVED: an address is
578
+ personal data, and sending one the user did not type -- from a git
579
+ config, an environment variable, anywhere -- is not a rate-limit
580
+ optimisation, it is an unasked-for disclosure.
581
+
582
+ ### `scholar_venue_weight`
583
+
584
+ How a paper's score is composed. Renormalised over whichever terms are
585
+ actually available, so a paper too young to have a citation percentile
586
+ is not scored as though it had a bad one.
587
+
588
+ ### `scholar_citation_weight`
589
+
590
+ ### `scholar_author_weight`
591
+
592
+ ### `scholar_venue_credit`
593
+
594
+ Venue credit. A preprint scores lower than a comparable published paper
595
+ -- generally, not strictly, which is why it is a weight and not a gate.
596
+
597
+ **Notes on individual entries.**
598
+
599
+ arXiv, bioRxiv, institutional repositories
600
+
601
+ ### `scholar_h_saturation`
602
+
603
+ max h-index across a paper's authors, log-saturated. 60 is a full career
604
+ at the top of most fields; the curve is deliberately flat above it,
605
+ because the difference between 60 and 90 says more about field size and
606
+ career length than about this paper.
607
+
608
+ ### `scholar_min_cohort_size`
609
+
610
+ Below this many works, a field-and-year cohort is too small for a
611
+ percentile to mean anything and the window widens instead.
612
+
613
+ ### `scholar_min_citation_age_days`
614
+
615
+ Below this age, citation counts are noise: 72% of computer-science
616
+ preprints from 2024 have zero citations, so a three-week-old paper's
617
+ zero says nothing at all. The term is DROPPED and the remaining weights
618
+ renormalise, rather than being scored as low impact.
619
+
620
+ ### `scholar_retracted_authority`
621
+
622
+ A retracted paper is not evidence, whatever its venue or citation count
623
+ -- and a heavily cited retraction is the most dangerous shape there is.
624
+
625
+ ## Reasoning effort (ROADMAP_v2 §16; default changed batch 25, #139)
626
+
627
+ ### `default_effort`
628
+
629
+ The default effort level requested when the user has not chosen one.
630
+
631
+ This shipped as None -- "send nothing", the one universally safe value,
632
+ because reasoning_effort is rejected by OpenAI-compatible NON-reasoning
633
+ models. Batch 25 changed it to "high" by owner decision: the pipeline
634
+ now consumes the setting (#139), so None meant the mode this project
635
+ exists for ran at the provider's own default forever, invisibly. The
636
+ risk the old note named is managed in MODEL_EFFORT_LEVELS rather than
637
+ by silence: known non-reasoning models are listed there with an EMPTY
638
+ level set, which effort_for() treats as authoritative and drops the
639
+ level cleanly instead of sending a parameter that would 400.
640
+
641
+ ### `model_effort_levels`
642
+
643
+ Fallback effort levels for providers whose APIs expose no capability
644
+ endpoint (every OpenAI-compatible provider, and Google). ANTHROPIC is NOT
645
+ listed here on purpose: its Models API reports per-model effort support,
646
+ so client.py queries it and new Anthropic models need no entry.
647
+
648
+ An entry whose value is an EMPTY list means "this model takes no effort
649
+ parameter at all" and is authoritative: effort_for() DROPS the requested
650
+ level with a naming warning instead of sending it. That is the safety
651
+ mechanism behind DEFAULT_EFFORT="high" -- without these entries, the
652
+ fallback for an unknown model ASSUMES ["low","medium","high"] and the
653
+ default high would reach gpt-4o-class endpoints as a 400 on every call.
654
+ Incomplete on purpose (same posture as MODELS_REJECTING_SAMPLING_PARAMS):
655
+ an unlisted non-reasoning endpoint still fails loud, and the fix is one
656
+ line here or `--effort auto`.
657
+
658
+ ### `default_effort_levels`
659
+
660
+ ### `google_thinking_budgets`
661
+
662
+ Google expresses reasoning depth as an integer token budget rather than an
663
+ enum, so a level has to be mapped to a number at the boundary. -1 is the
664
+ SDK's "decide dynamically" sentinel.
665
+
666
+ Live since batch 172 (ROADMAP_v3 §82). Until then this map was never sent:
667
+ the google-genai pin, 1.0.0, had no `thinking_budget`, so Google reported
668
+ no levels. The pin has it since 2.29.0 (2.32.0 since batch 234). Measured on Gemini 3: every budget here,
669
+ and 0, was accepted by gemini-3.1-flash-lite. Gemini 3's native control
670
+ is `thinking_level`; this map sends budgets, which it still takes. A Gemini
671
+ model with no `model_effort_levels` entry gets `default_effort_levels` and a
672
+ WARNING, like any other unlisted model.
673
+
674
+ ## Database
675
+
676
+ ### `db_path`
677
+
678
+ Must not be empty (#213). An empty path was accepted until then and
679
+ opened SQLite's anonymous temporary database, so every thread and memory
680
+ vanished at exit without a word.
681
+
682
+ ## Output artifacts
683
+
684
+ ### `output_dir`
685
+
686
+ BESIDE THE WORK, which since batch 44 means beside the WORKSPACE when one
687
+ was named. "State is global, artifacts are local" is the distribution
688
+ split, and the local half is only true if "local" tracks where the work
689
+ is: with AGENT_WORKSPACE pointing at a project, a run launched from the
690
+ harness checkout wrote its reports into the harness. Reading the env var
691
+ rather than WORKSPACE_DIR below, and defaulting it to ".", so this stays
692
+ one expression with no branch in it -- an unset AGENT_WORKSPACE gives
693
+ ./output exactly as before, not ./workspace/output.
694
+
695
+ ## File-ops workspace (ROADMAP §6)
696
+
697
+ ### `workspace_dir`
698
+
699
+ An empty one is refused by name at startup, by
700
+ `protected_paths.check_workspace`, which answers both routes to it -- this
701
+ key and an empty `AGENT_WORKSPACE`. The schema deliberately does not
702
+ refuse it too (#213): it would answer the variable's case first, as an
703
+ error about this file.
704
+
705
+ ### `workspace_is_project`
706
+
707
+ Whether the workspace above is also the PROJECT (batch 111,
708
+ TECHNICAL_DEBT 28). Shipped `false`.
709
+
710
+ THE SECOND OF TWO ROUTES TO ONE ANSWER, and it exists because the first
711
+ one is an environment variable. `AGENT_WORKSPACE` has decided this since
712
+ batch 44, by its presence; someone configuring the harness through
713
+ `config.yaml` alone had no way to say it. What they got instead was the
714
+ workspace they asked for and a project of `os.getcwd()` -- and, launched
715
+ from the install tree, a refusal telling them to set a variable in the
716
+ same breath as the file where they had just named the directory they
717
+ meant. The refusal's own last paragraph used to say so.
718
+
719
+ THE TWO OPTIONS THE DEBT ENTRY OFFERED WERE BOTH WORSE, and the second
720
+ for a reason the entry did not have. "Any value in the file counts" is
721
+ always true, because the shipped file writes `workspace_dir: ./workspace`
722
+ out in full -- so it moves the project for every install nobody has
723
+ edited. "Differs from the shipped default" needs a shipped default to
724
+ compare against, and `HarnessConfig` has none: every field is required
725
+ and comes from the file, by design, so the comparison would need either
726
+ the first schema default in the codebase or a second hand-written copy
727
+ of `"./workspace"` -- the exact staleness `shipped_defaults()` exists to
728
+ prevent. It also quietly makes `./workspace` un-nameable.
729
+
730
+ NOT AN AUTHORITY KEY, deliberately. An authority key is one no
731
+ unattended surface reaches, and `ENV_OVERRIDES` and
732
+ `HARNESS_AUTHORITY_KEYS` are disjoint for that reason
733
+ (`test_no_authority_key_has_an_environment_override`). `AGENT_WORKSPACE`
734
+ already produces this exact effect through an ordinary override, so
735
+ gating the file key would put the confirmation on the louder of the two
736
+ routes and leave the quieter one open. It is also strictly less
737
+ dangerous than `workspace_dir` itself, which is ungated and carries the
738
+ sharper hazard named above it.
739
+
740
+ ### `workspace_dir_explicit`
741
+
742
+ Whether the workspace was NAMED as the project, as opposed to defaulted
743
+ (batch 44; a second input added in batch 111).
744
+
745
+ `cfg.workspace_is_project or "AGENT_WORKSPACE" in os.environ`. The
746
+ variable half is its PRESENCE, never its value: the default
747
+ "./workspace" is a subdirectory of wherever you launched, so a value
748
+ test would make `./workspace` the project for everyone who never set
749
+ anything -- which is the whole population this must not disturb. main()
750
+ reads this to decide the project path; the decision is there, because
751
+ the config layer holds plain values and this is one.
752
+
753
+ OR, with no precedence between the halves. Both say the same thing about
754
+ the same directory, so there is no conflict to resolve, and inventing a
755
+ winner would mean a user who did both getting less than a user who did
756
+ either. main() never learns there are two routes -- that is the point of
757
+ computing the OR here.
758
+
759
+ Batch 107 made the UNNAMED branch a refusal rather than a silent
760
+ fallback: with nothing naming a workspace the project is os.getcwd(), and
761
+ protected_paths.check_project() refuses that when it is the harness's own
762
+ install tree. Naming the harness had always been refused by
763
+ check_workspace; the implicit route reached the same state by saying
764
+ nothing. Batch 111 closed what that left open (TECHNICAL_DEBT 28) and the
765
+ refusal now offers three ways out rather than two and a dead end.
766
+
767
+ Note that this value has NO `config.yaml` key of its own -- it is computed
768
+ in `config_schema.derived_values(cfg)`, because half of it is a question
769
+ about the environment that YAML cannot ask. `workspace_is_project` is an
770
+ input to it, not a key for it. There is nothing here to edit.
771
+
772
+ ### `max_file_size_bytes`
773
+
774
+ 25 MB — hard reject before opening
775
+
776
+ ### `max_read_lines`
777
+
778
+ max lines per read call
779
+
780
+ ### `max_read_chars`
781
+
782
+ max chars per read call
783
+
784
+ It is also the bound on an MCP tool's whole result (ROADMAP_v3 §84, A79,
785
+ batch 225). The server's text is cut at it with a note giving the size it
786
+ had, a structured payload is kept whole if it fits in what the text left
787
+ and dropped with a note if not, and a word the cut splits goes too, so the
788
+ redactor never sees half a credential. One key and not a second one,
789
+ because the compaction headroom check (`compaction_min_headroom_tokens`,
790
+ below) measures one read against this number, and an MCP result is a read
791
+ for that purpose.
792
+
793
+ ### `max_image_bytes`
794
+
795
+ ROADMAP_v3 §81 (IM7). The largest image file `view_image` will attach:
796
+ 3,932,160 bytes, which is 3.75 MiB.
797
+
798
+ The number is derived, not chosen. An image travels as base64, which is 4/3
799
+ the size of the file, so 3.75 MiB of file is 5 MiB on the wire. 5 MiB is
800
+ Anthropic's per-image limit, the smallest of the wires §81 delivers to;
801
+ OpenAI's is larger, and Google's limit (a wire since batch 173) is on the whole
802
+ request rather than per image. A cap at 5 MiB of FILE would put a 6.7 MiB payload
803
+ on the wire, and Anthropic would reject it with a 400 that names neither the
804
+ file nor this key.
805
+
806
+ It is also a cost bound, separate from `max_file_size_bytes` (25 MB). That
807
+ key limits what a tool may operate on; this one limits what is resent with
808
+ every request for as long as the image stays in the conversation's view.
809
+ Raising it toward the 25 MB general limit would not be consistent, it would
810
+ break: the provider would reject the request.
811
+
812
+ ## /init (ROADMAP_v2 §24)
813
+
814
+ ### `initializer_agent`
815
+
816
+ ### `init_read_chars`
817
+
818
+ Three bounds, each doing a different job (I7). /init is a tool-heavy loop
819
+ over documentation that can run to hundreds of kilobytes -- this repo's own
820
+ root markdown is 721KB, with DEVLOG.md alone at 226KB.
821
+
822
+ INIT_READ_CHARS is well below MAX_READ_CHARS because of TECHNICAL_DEBT
823
+ item 9 (now closed, batch 27): the billing meter re-counts the WHOLE
824
+ prompt on every step, so each 50KB read is re-billed for every step that
825
+ follows it. 20KB keeps a dozen reads affordable and still returns a
826
+ useful span of a document per call. It stays even though the meter no
827
+ longer caps anything by default -- it bounds per-read VOLUME, which is a
828
+ different axis from spend and from the number of reads.
829
+
830
+ ### `init_max_steps`
831
+
832
+ The number of reads, which neither budget nor read size bounds: a model
833
+ can spend an unbounded number of small reads inside any ceiling.
834
+
835
+ ### `init_max_hub_chars`
836
+
837
+ Batch 221 (#226, TECHNICAL_DEBT 72). The existing hub was the one read on
838
+ /init's path with no bound at all: measured, a 5.4 MB `AGENTS.md` became a
839
+ 5.4 MB model task and a 5.4 MB diff in the transcript. A cap with a notice
840
+ ("the first N characters are shown") was rejected, because the initializer's
841
+ reply REPLACES the whole file -- a model shown the head would write the hub
842
+ back without its tail. So past this size /init stops before any model call
843
+ and says to trim the file or edit it by hand. 1,000,000 characters is about
844
+ three times this repository's own 347 KB hub and past what most model
845
+ windows hold anyway, so an ordinary hand-grown hub never meets it. The owner
846
+ chose the shape and the number.
847
+
848
+ ### `init_diff_shown_lines`
849
+
850
+ The other half of #226: below the cap a hub can still produce a diff of
851
+ tens of thousands of lines, and printing it whole is the transcript freeze
852
+ S15 removed. Only the PRINT is cut, with a line saying how many lines were
853
+ not shown; the whole diff still decides whether anything changed, and the
854
+ write is still the whole proposal. 2,000 lines is enough to judge a rewrite
855
+ by -- this repository's own hub is about 3,300 lines.
856
+
857
+ ## MCP teardown (ROADMAP_v2 §37 F6, #64)
858
+
859
+ ### `teardown_budget_s`
860
+
861
+ ONE shared wall-clock budget for the whole goodbye -- polite close,
862
+ force-cancel, loop-thread join -- replacing three sequential 15-second
863
+ waits whose worst case hung a quitting harness for ~45s, Ctrl+C
864
+ included. Servers still alive when the budget expires are named in a
865
+ WARNING; their child processes may outlive this session. The value
866
+ lives with the other harness bounds and is re-exported by
867
+ mcp_client/client.py beside its sibling timeouts.
868
+
869
+ ## MCP results
870
+
871
+ ### `mcp_max_event_bytes`
872
+
873
+ 16 MiB (16,777,216), the owner's number (batch 208). mcp 2.3.0 reads
874
+ each Streamable HTTP server-sent event whole and fails the call when one
875
+ exceeds `max_sse_event_size`; 2.2.0, on the httpx2 it installed, had no
876
+ limit at all. A result a server STREAMS is one event, so for those this is
877
+ the memory one result can take before the harness sees it -- a bound on
878
+ the transport, NOT on what reaches the model. That one is `max_read_chars`,
879
+ applied to the whole result since batch 225 (A79).
880
+
881
+ WHICH RESULTS IT COVERS, measured rather than read (batch 208). A server
882
+ may answer a request as SSE or as plain JSON. mcp 2.3.0's own server,
883
+ negotiating 2026-07-28, answered every request as `application/json`, and
884
+ the client reads that body whole with no limit: 4 MB and 40 MB results
885
+ both arrived, at this ceiling and at 1 MiB. So this key bounds streamed
886
+ answers only. Through `mcp_client` against a server streaming every answer
887
+ (an in-process mock on 2025-06-18): a 2 MB result passed at 16 MiB and
888
+ failed at 1 MiB with "MCP call failed: SSE stream failed: Server-sent event
889
+ exceeded the 1048576 byte limit."; 16,000,000 characters passed at 16 MiB
890
+ and 20,000,000 failed naming 16777216.
891
+
892
+ Why not the SDK's 1 MiB: a screenshot is 0.2-3 MB, base64 adds a third,
893
+ and a tool that returns one would fail outright, text included -- this
894
+ harness replaces an MCP image with a note and keeps the text, but only if
895
+ the event arrives. 16 MiB fits an image at Claude's 5 MB per-image limit
896
+ (~6.7 MB encoded) with headroom while a broken or hostile server still
897
+ costs a bounded amount per event. Uploads are unaffected: this bounds what
898
+ a server sends back.
899
+
900
+ Read per connection by `mcp_client/client.py`, which passes it on BOTH
901
+ Streamable HTTP branches, because a bare URL takes the SDK's default. The
902
+ legacy "sse" transport keeps the SDK's fixed 1 MiB: `sse_client` takes no
903
+ size. An ordinary key rather than harness authority, like
904
+ `max_read_chars`: it is a resource bound, not a permission.
905
+
906
+ ## Shell / sandbox (ROADMAP §7)
907
+
908
+ ### `shell_binary`
909
+
910
+ auto-detect if empty
911
+
912
+ ### `allow_insecure_sandbox_fallback`
913
+
914
+ explicitly enable subprocess fallback
915
+
916
+ ### `auto_approve_sandbox_fallback`
917
+
918
+ auto-approve fallback runs (no per-run prompt)
919
+
920
+ ### `allow_wsl_backend`
921
+
922
+ ROADMAP_v3 §49, slice 3 (SS35). Whether the agent may ask for a command
923
+ to run in a WSL distribution instead of the container. Ships `false`, and
924
+ is part of the frozen posture -- so no tool call, `settings.json` at
925
+ either tier, environment variable or slash command can move it.
926
+
927
+ It is off by default because WSL is not a weaker container, it is no
928
+ container at all. Measured on the machine this was built on:
929
+
930
+ ```text
931
+ the harness's own config.yaml writable from the distribution
932
+ ~/.config (and .config/venastine) writable
933
+ memory / CPU / process limits none -- they belong to the WSL VM
934
+ network always on, not withholdable
935
+ Windows interop survives `env -i`; it is binfmt_misc,
936
+ not PATH, so a command there can run
937
+ cmd.exe with the user's authority
938
+ ```
939
+
940
+ So the harness treats it as UNCONTAINED for every tier (SS1): under
941
+ `tiered` and `contained` only a read-only command whose every argument is
942
+ inside the workspace runs unasked, and a command naming a protected path
943
+ segment is REFUSED rather than asked about (SS40) -- because the
944
+ read-only bind over `.venastine/` that makes "ask" tolerable on the
945
+ container route does not exist here, and a write would succeed and be
946
+ noticed only at the next launch.
947
+
948
+ Two questions, two owners, which is why this is a config key and the
949
+ backend choice is a tool parameter: whether this machine may run agent
950
+ commands outside a container is the user's decision, and which command
951
+ needs a Linux userland is the agent's. A process-wide switch would answer
952
+ both at once and give up the container for every command in order to
953
+ serve one.
954
+
955
+ ### `wsl_distro`
956
+
957
+ Which distribution the backend runs in. Empty means the first name
958
+ `wsl.exe -l -q` lists, which is WSL's own default.
959
+
960
+ Never settable by the agent (SS36): a distribution is a fact about the
961
+ machine rather than about a command, and the list on a typical developer
962
+ box includes `docker-desktop`, which is Docker's own internal
963
+ distribution and not a place to run anything. A configured name is
964
+ matched case-insensitively against the installed list and refused with
965
+ that list if it is absent -- rather than passed through, because
966
+ `wsl.exe -d NoSuchDistro` exits `4294967295` and writes its complaint to
967
+ *stdout* in UTF-16, so an unvalidated name fails in a way that reads as
968
+ the command failing rather than the configuration being wrong.
969
+
970
+ The listing itself is decoded as UTF-16LE for the same measured reason:
971
+ read as UTF-8 it is a NUL-riddled string that matches no name at all, so
972
+ a probe that guessed the encoding would report "no distribution
973
+ installed" on a machine with three.
974
+
975
+ ### `allow_ssh_backend`
976
+
977
+ ROADMAP_v3 §49, slice 4 (SS46). Whether the agent may ask for a command to
978
+ run on the one remote host configured below, instead of the container.
979
+ Ships `false`, and is part of the frozen posture -- so no tool call,
980
+ `settings.json` at either tier, environment variable or slash command can
981
+ move it.
982
+
983
+ It is off by default because a remote host is not a weaker sandbox; it is
984
+ a machine this harness cannot look at:
985
+
986
+ ```text
987
+ isolation none
988
+ memory / CPU / process limits none -- they belong to the remote
989
+ network the remote's, always on
990
+ what this harness can inspect nothing: no path, no file, no mount
991
+ credentials yours, not ones this harness issued
992
+ ```
993
+
994
+ **NOTHING runs there without asking you (SS48)** -- the one rule that
995
+ makes this backend different from the other three. Every other backend
996
+ runs a read-only in-workspace command unasked; this one asks about `ls`.
997
+ The reason is not caution: the check that decides whether a command's
998
+ arguments stay inside the workspace resolves them against the LOCAL
999
+ filesystem, so on a remote it is answering a true question about the wrong
1000
+ machine. Slice 3 (SS45) is the evidence for what such a check is worth --
1001
+ it did not fail loudly, it approved confidently. A second reason stands on
1002
+ its own: `ssh` has no `execve` path, so the shell-free argv that the
1003
+ auto-approved tier is defined by does not exist on this route at all.
1004
+
1005
+ `never` still means never, because SS1 already said so and this slice
1006
+ widened no mode.
1007
+
1008
+ ### `ssh_host`, `ssh_user`, `ssh_port`
1009
+
1010
+ The one host the backend reaches. The agent says `backend: "ssh"` and gets
1011
+ THIS host; it never names one. SS35's split, unchanged: which machine this
1012
+ harness may reach is the user's question, which command needs it is the
1013
+ agent's. A named map of hosts is a later addition that this shape does not
1014
+ block.
1015
+
1016
+ `ssh_port` is a whole number from 1 to 65535 (#213). It was a lax `int`
1017
+ until then, so `true` was read as port 1 and `99999` reached the SSH call
1018
+ before anything said no.
1019
+
1020
+ ### `ssh_identity_file`
1021
+
1022
+ The private key to authenticate with. Empty offers whatever a running
1023
+ `ssh-agent` holds.
1024
+
1025
+ **This harness never asks for, holds or stores a passphrase or a password
1026
+ (SS47).** If the key is encrypted, load it into an agent with `ssh-add`
1027
+ first; the connection is made with `BatchMode=yes` and
1028
+ `PreferredAuthentications=publickey`, so `ssh` can never prompt. That flag
1029
+ is load-bearing rather than tidy -- measured, without it an encrypted key
1030
+ with no agent waits twenty seconds and is still waiting, trying to prompt
1031
+ on a terminal this process does not own. With it the same call fails in
1032
+ 0.12 s and the refusal says `ssh-add`.
1033
+
1034
+ `IdentitiesOnly=yes` is passed only WHEN a key is configured: with no key
1035
+ it would rule out the agent, which is the one way to use an encrypted key
1036
+ without this harness ever seeing the passphrase.
1037
+
1038
+ ### `ssh_remote_workspace`
1039
+
1040
+ Where a command runs on the remote. **Required** when the backend is on,
1041
+ and deliberately not defaulted to the login home (SS52): a command that
1042
+ runs somewhere nobody named is the failure this backend is most able to
1043
+ cause, and nothing here can see where it landed afterwards. `ssh` has no
1044
+ `--cd`, so the command is prefixed with `cd -- <dir> ||` and a failure is
1045
+ recognised by BOTH a chosen exit code and a marker on stderr -- either
1046
+ alone is guessable, since 125 is inside `timeout`'s own vocabulary and a
1047
+ marker is a string any command may print.
1048
+
1049
+ Note the trap this key names: `write` saves files LOCALLY while the shell
1050
+ runs remotely, so a file the agent just wrote is not in this directory.
1051
+
1052
+ ### `ssh_host_key`
1053
+
1054
+ The remote's host key, PINNED. Run `ssh-keyscan -p <port> <host>` and
1055
+ paste its output; the `#` banner line it prints above the key is ignored,
1056
+ so pasting the whole thing works. The harness writes it to a file it owns
1057
+ and passes `UserKnownHostsFile` with `StrictHostKeyChecking=yes`, never
1058
+ touching `~/.ssh/known_hosts` -- its trust set is exactly what you pinned.
1059
+
1060
+ A key that does not match, or no key at all, REFUSES (SS49). There is no
1061
+ trust-on-first-use prompt, for two reasons: a first-use question cannot be
1062
+ answered in a headless run, and the case actually worth refusing is a key
1063
+ that CHANGES, which a prompt teaches people to click through.
1064
+
1065
+ The fingerprint is not configured -- it is a hash, and `known_hosts` needs
1066
+ the key. It is DERIVED and shown, so you can verify it out of band once.
1067
+
1068
+ ### `ssh_binary`
1069
+
1070
+ Which `ssh` program to run. Empty prefers the operating system's own
1071
+ (`%SYSTEMROOT%\System32\OpenSSH\ssh.exe` on Windows) and falls back to
1072
+ `PATH`.
1073
+
1074
+ Not `PATH` first, and that is measured rather than defensive: on the
1075
+ machine this was built, `PATH` resolves to a git-for-Windows MSYS2 build
1076
+ while Windows ships its own, and the two are NOT interchangeable. They
1077
+ agree on every happy path and disagree on FAILURES -- a refused connection
1078
+ is `ssh: connect to host ... Connection refused` from one and `banner
1079
+ exchange: Connection to UNKNOWN port -1: Connection refused` from the
1080
+ other -- and a failure message is the half a refusal is built out of. It
1081
+ is SS38's GnuWin32 `cat` on a third route.
1082
+
1083
+ ### `shell_approval_mode`
1084
+
1085
+ ROADMAP_v2 §28 (G3). WHICH shell commands need a human to say yes.
1086
+
1087
+ ```text
1088
+ "always" every command is asked about, whatever it does.
1089
+ "tiered" the classifier decides -- see security/capability.py for
1090
+ the one rule, and security/sandbox.py:classify_command for
1091
+ how a command is measured against it. Since §48 (CE1) only
1092
+ a read-only INERT command runs unasked: anything that may
1093
+ run code asks, contained or not.
1094
+ "contained" §48 (CE2): the rule "tiered" had before §48. A command the
1095
+ container confines runs unasked unless it gets network,
1096
+ code included; reads outside the workspace and network
1097
+ commands still ask.
1098
+ "never" nothing is ever asked about.
1099
+ ```
1100
+
1101
+ `auto_approve_sandbox_fallback` is honoured under "tiered" and
1102
+ "contained" (§48, CE5) -- the one written exception to "code asks" under
1103
+ "tiered" -- and never under "always", whose check returns before the
1104
+ opt-in is read (CE6).
1105
+
1106
+ There is no list of interpreters behind "tiered", deliberately. `python
1107
+ -c` is the obvious case, but `pytest` runs a conftest.py, `make` runs a
1108
+ Makefile, and `"python"`, `python3.13` and `echo x | python` are the same
1109
+ call to a shell -- a list would be a parser whose misses auto-approve
1110
+ (G2). Every command that is not INERT may run code, and that is the
1111
+ answer the classifier gives.
1112
+
1113
+ This is the gate. Before §28 the gate was ToolApprovals.shell, and
1114
+ `_shell_approval_check` looked like a five-layer policy underneath it --
1115
+ but four of those layers could only ever return False on the shipped
1116
+ flags, so the whole thing was a pass-through for one boolean and the
1117
+ harness had exactly two settings: ask about everything, or ask about
1118
+ nothing. "tiered" is the rung that docstring already claimed existed.
1119
+
1120
+ "never" is that second setting, preserved deliberately and now NAMED.
1121
+ It is the pre-§28 `ToolApprovals.shell = False` behaviour exactly: an
1122
+ inert command runs on the HOST with unrestricted arguments, so `cat
1123
+ ~/.aws/credentials` returns your keys to the model. You reach that by
1124
+ writing "never", not by switching off a field that reads like "stop
1125
+ nagging me" -- which is the actual fix for audit #157.
1126
+ The legal set lives with the code that ENFORCES it --
1127
+ security/capability.APPROVAL_MODES, which validate_mode() reads. A second
1128
+ copy here was read by nothing but a test asserting its literal, so the two
1129
+ could have disagreed and only the unenforced one would have been wrong.
1130
+
1131
+ ### `sandbox_docker_image`
1132
+
1133
+ The image both runtimes run: Docker, or Podman when Docker cannot run a
1134
+ container (ROADMAP_v3 §49, SS22). The key kept its name when Podman
1135
+ arrived, because config_update.py carries a user's own value across an
1136
+ npm update BY KEY, and a renamed key would silently drop it.
1137
+
1138
+ FULLY QUALIFIED since batch 93 (SS23), and the reason is Podman's: an
1139
+ unqualified name is resolved through `registries.conf`, and with no
1140
+ search registries and no short-name alias -- measured on a stock Ubuntu
1141
+ Podman, whose `shortnames.conf` happens to alias `python` -- a name it
1142
+ cannot resolve fails without a terminal to ask on. `docker.io/library/`
1143
+ is where Docker resolves the short name anyway. A value you set yourself
1144
+ is carried as written.
1145
+
1146
+ A TAG, NOT AN IDENTITY (ROADMAP_v2 §46, EP7): the resolved image ID is
1147
+ logged once per process, and two machines -- or two runtimes on one --
1148
+ pulling the same tag on different days get different images.
1149
+
1150
+ ### `sandbox_timeout_seconds`
1151
+
1152
+ ### `sandbox_memory_mb`
1153
+
1154
+ ### `sandbox_cpu_seconds`
1155
+
1156
+ ### `sandbox_max_pids`
1157
+
1158
+ ### `shell_session_timeout_cap_s`
1159
+
1160
+ ROADMAP_v3 §49 (SS12, SS13). The longest a background or monitor session
1161
+ may run, whatever the agent asks for. 3600 because the owner's own example
1162
+ was an hour, and it is also the ceiling on how long a misbehaving agent can
1163
+ hold the input block (SS2) with one session.
1164
+
1165
+ A value above the cap is CLAMPED, not refused, and the started result says
1166
+ so. Refusing would make the agent guess numbers until one fits, and the cap
1167
+ is not a secret. The clamp happens in one place, the session manager, and
1168
+ is enforced twice: the harness kills at the timeout, and coreutils `timeout`
1169
+ inside the container ends it `SESSION_TIMEOUT_MARGIN_S` later if the harness
1170
+ is no longer there to.
1171
+
1172
+ ### `shell_session_max_live`
1173
+
1174
+ ROADMAP_v3 §49 (SS12). Live background and monitor sessions, counted
1175
+ process-wide across the main agent and every subagent -- a subagent does not
1176
+ get a quota of its own, or three parallel children could each fill one. The
1177
+ classic one-shot `shell` is not counted: it holds no slot past its own call.
1178
+
1179
+ 4, because each is a container under `sandbox_memory_mb` (2048), so the
1180
+ ceiling this implies is 8 GB of container memory. At the cap a start is
1181
+ refused with the list of live sessions, never queued: a queued session would
1182
+ start at a time nobody chose.
1183
+
1184
+ ### `shell_session_max_consecutive_wakes`
1185
+
1186
+ ROADMAP_v3 §49 (SS2). How many wake turns may run in a row with no message
1187
+ from the user. Every wake is a model call nobody typed, and while sessions
1188
+ are live the prompt is blocked, so an agent that starts a new session on
1189
+ every wake would otherwise hold the session and spend tokens indefinitely.
1190
+ Past the limit, waking stops, the plain prompt unblocks, and the results are
1191
+ delivered with the user's next message; a user message resets the count.
1192
+
1193
+ 10 is a ceiling on an unattended loop, not an estimate of a normal one. A
1194
+ subagent has no user to reset it, so it gets one final wake and returns
1195
+ (SS20).
1196
+
1197
+ ### `shell_session_output_head_chars`
1198
+
1199
+ ### `shell_session_output_tail_chars`
1200
+
1201
+ ROADMAP_v3 §49 (SS12). How much of a session's output is kept: the first
1202
+ 10 000 characters and the most recent 190 000, IN MEMORY ONLY. Never on disk,
1203
+ because program output can hold secrets and redaction is pattern-based, so a
1204
+ file would be a durable copy of whatever the patterns miss. The dropped
1205
+ middle is reported as a gap when the agent pages through the output.
1206
+
1207
+ The head is what shows how a run STARTED -- the command's own banner, the
1208
+ first error of a cascade -- which a tail-only buffer loses on exactly the long
1209
+ runs sessions exist for. The split is uneven because the end of a run is what
1210
+ is usually asked about.
1211
+
1212
+ ## Output redaction (#167/#49, batch 20)
1213
+
1214
+ ### `redact_tool_outputs`
1215
+
1216
+ The PERMANENT master switch for pattern-based redaction of what leaves a
1217
+ tool: vendor token substitution and credential-shape redaction, applied
1218
+ by check_output_policy() to every tool result and by param_digest() to
1219
+ what the shells display of an argument. On by default; a user whose
1220
+ workflow needs the model to see real values (debugging their own
1221
+ credentials, say) turns it off here, or per-run with
1222
+ VENASTINE_REDACT_OFF in the environment -- see redaction_enabled().
1223
+
1224
+ DELIBERATELY NOT a settings.json key (decision recorded in DEVLOG,
1225
+ batch 20): project tier beats user tier there, so a cloned repo could
1226
+ ship `.venastine/settings.json` switching the scrubbing off behind a
1227
+ trust prompt nobody reads -- the exact shape G7 rejected
1228
+ shell_approval_mode for and R12 rejected research.granted_tools for.
1229
+ Permanent means editing this line.
1230
+
1231
+ NEVER covers three things, whatever this is set to: check_input_policy's
1232
+ refusals (a denial is legible, not destructive); check_output_policy's
1233
+ depth-cap substitution (fail-closed structure bound -- making it optional
1234
+ would recreate the deterministic bypass its comment forbids); and
1235
+ logging_setup.py's formatter redaction (the second sink keeps its own
1236
+ guard).
1237
+
1238
+ ### `tool_compute_timeout_s`
1239
+
1240
+ ROADMAP_v2 §31 (H9). The wall clock on a BUDGET_COMPUTE tool call --
1241
+ the six math tools, which are pure functions of their params and have
1242
+ nothing bounding them from the inside. dispatch runs them in a
1243
+ killable subprocess under this budget (tools/isolation.py).
1244
+
1245
+ 20s is chosen against measurement, not taste. The slowest LEGITIMATE
1246
+ call found is `symbolic_math series order=1000`, so the headroom is the
1247
+ ratio between that call and this number; the runaways it exists for do
1248
+ not return at all. Ten passes each burning a full budget is 200s rather
1249
+ than forever, which is the trade being made.
1250
+
1251
+ *(Re-measured in batch 143, and the value is again unchanged. That call
1252
+ is 9.92-10.19s warm on Windows / Python 3.13.13 / sympy 1.13.3 and
1253
+ 11.14s through the real transport -- roughly 2x headroom, not the 6x this
1254
+ paragraph claimed from a 3.44s reading. The ratio is the thing that
1255
+ moved: a timing taken on another machine or another sympy is not a
1256
+ number to restate bare, which is why it is now named as a ratio and
1257
+ measured where it is quoted. The call itself is refused on SIZE from
1258
+ batch 143 anyway -- 1,185,090 characters against
1259
+ `config.MAX_READ_CHARS` -- so it is no longer the worst case this budget
1260
+ has to cover, and its output was never measured when it was chosen.)*
1261
+
1262
+ *(Corrected in batch 83. This paragraph was carried over saying 15s,
1263
+ roughly 4x and 150s -- all three consistent with a 15 the value has not
1264
+ been for some time. `README.md` said 15 in three more places against its
1265
+ own table's 20. The value is unchanged; the arithmetic now describes it.)*
1266
+
1267
+ ONE number, not one per tool: there is no evidence any two math tools
1268
+ want different answers, and a per-tool budget makes each a judgement
1269
+ call at the registration site. It sits beside SANDBOX_TIMEOUT_SECONDS
1270
+ because that is the constant a reader would compare it against.
1271
+
1272
+ ### `network_allowed_commands`
1273
+
1274
+ WARNING: commands matching these words get network access inside the
1275
+ sandbox. A compromised package or script can exfiltrate data or
1276
+ download payloads. Only add commands you trust.
1277
+
1278
+ ### `inert_commands`
1279
+
1280
+ WARNING: a word added here is a promise that the program neither writes
1281
+ nor runs code -- under `tiered` it is the ONLY tier that runs unasked
1282
+ (§48, CE1), so an entry that can execute its arguments or a file (`find
1283
+ -exec`, `sort --compress-program`, `tar --to-command`) is an unprompted
1284
+ code path. An inert command runs in the container when Docker or Podman is up and
1285
+ on the host when it is not (§46, EP5); one whose argument reaches outside
1286
+ the workspace is HOST_READ, runs on the host, and always asks. The risk of
1287
+ the host path is information disclosure, not modification.
1288
+
1289
+ ### `max_granted_tool_calls`
1290
+
1291
+ ROADMAP_v2 §25 (R6). Ceiling on how many PRE-GRANTED tool calls one
1292
+ research run may make without being asked again. Counts granted calls
1293
+ only -- web_search, arxiv_search and every math tool are approval-free
1294
+ and never touch it, so this is not a cap on tool use.
1295
+
1296
+ Pre-flight authorization trades a per-call decision for one up-front
1297
+ decision, and the thing it gives up is the natural bound on how many
1298
+ times the granted tool runs. Set generously: the ceiling exists to stop
1299
+ an injected loop spending the whole authorization unattended, not to
1300
+ ration ordinary work across ten passes.
1301
+
1302
+ ### `attended_approval_timeout_s`
1303
+
1304
+ ROADMAP_v2 §25 (R9). How long an attended research run waits for a human
1305
+ to answer one approval prompt before DENYING that call and carrying on.
1306
+
1307
+ Denial is the safe direction, and continuing is the useful one: walking
1308
+ away from an attended run degrades it rather than wedging it, so nine
1309
+ completed passes are not lost to one unanswered question. Generous by
1310
+ default because the whole point of the mode is that someone is there.
1311
+
1312
+ ### `subagent_review`
1313
+
1314
+ ROADMAP_v2 §20 (D9). Whether a finished research run gets a reviewer pass
1315
+ over its own output. Off by default: it is one more model call plus a
1316
+ re-synthesis whenever a correction is accepted, and it needs someone
1317
+ present to consent to anything it changes.
1318
+
1319
+ ### `max_review_findings`
1320
+
1321
+ §20 (V4). Ceiling on how many findings one review may put to the user.
1322
+ Consent fatigue is the failure mode a mutating review stage has and a
1323
+ read-only one does not: the fiftieth prompt gets answered differently
1324
+ from the first, and an injected "correction" only needs one reflexive
1325
+ yes. Findings past the cap are dropped and the drop is TRACED -- a
1326
+ silent truncation would read as "the reviewer found twelve things".
1327
+
1328
+ ### `max_review_refinements`
1329
+
1330
+ §20 (V5). How many times one finding may be sent back for refinement.
1331
+ A human asks for each round, so this is not a runaway guard -- it bounds
1332
+ the case where the reviewer keeps producing a proposal that misses the
1333
+ same point, which is when the honest answer is to reject it.
1334
+
1335
+ ## Compaction (ROADMAP_v2 §21)
1336
+
1337
+ ### `compaction_trigger_fraction`
1338
+
1339
+ D27: these are DEFAULTS, not settings. Every value is a starting point
1340
+ chosen to be reasonable and explicitly not claimed to be right, and the
1341
+ compaction block as a whole is overridable through the existing settings
1342
+ mechanism -- `config.yaml` default -> user
1343
+ ~/.config/venastine/settings.json -> trusted project
1344
+ .venastine/settings.json -> a per-invocation `/compact --strength N`.
1345
+ The architecture is what's locked; the numbers are expected to move once
1346
+ there are real long threads to look at.
1347
+
1348
+ **THIS KEY IS NOT ONE OF THE OVERRIDABLE ONES.** The paragraph above is
1349
+ D27's about the compaction SECTION, and the seven keys a `settings.json`
1350
+ `compaction` block actually accepts are `strength`, `keep_recent_tokens`,
1351
+ `trigger_tokens`, `warning_margin_tokens`, `keep_recent_turns`,
1352
+ `strategy` and `max_retries` -- `core/config_loader._KNOWN_COMPACTION`.
1353
+ `trigger_fraction` is not among them and unknown nested keys RAISE, so
1354
+ writing `compaction: {trigger_fraction: 0.9}` is a startup error rather
1355
+ than an override. `README.md`'s compaction table lists the seven.
1356
+
1357
+ How much of the model's context window a thread may fill before
1358
+ compaction folds it. `thresholds()` reaches it through
1359
+ `derived_trigger()`, which is `context_limit()` minus `headroom_for()` --
1360
+ so it is 900k on a 1M model and 180k on a 200k one.
1361
+
1362
+ **BATCH 119 MADE IT A CLAMPED HEADROOM RATHER THAN A BARE SHARE, and
1363
+ raised it from 0.85 to 0.90.** What the fraction really names is the
1364
+ REMAINDER: the room the step after the trigger has to fit inside, because
1365
+ nothing recovers from a prompt that overflows the window -- there is no
1366
+ context-overflow handler anywhere in `core/`, so the provider raises, the
1367
+ loop retries the same oversized prompt three times, and the turn dies.
1368
+
1369
+ A bare share gets that wrong at both ends, for the mirror image of the
1370
+ reason batch 44 gave against a flat margin. Batch 44 was right that 40k is
1371
+ a rounding error under 1M and strangling under 64k. A share has the
1372
+ opposite defect, because what the headroom must contain does not scale
1373
+ with the window at all: one `max_read_chars` result is 12,500 proxy tokens
1374
+ whether the model holds 64k or a million, and one measured `arxiv_search`
1375
+ result was 29,249. 0.10 of 1M is 100k; 0.10 of 64k is 6,553, which one
1376
+ ordinary file read overflows.
1377
+
1378
+ So `compaction_min_headroom_tokens` and `compaction_max_headroom_fraction`
1379
+ clamp it, and the floor binds below roughly 160k and nowhere above:
1380
+
1381
+ | window | 0.85 (pre-119) | bare 0.90 | shipped | which clamp |
1382
+ |---|---|---|---|---|
1383
+ | 32,768 | 27,852 | 29,491 | 24,576 | cap |
1384
+ | 65,536 | 55,705 | 58,982 | 49,536 | floor |
1385
+ | 131,072 | 111,411 | 117,964 | 115,072 | floor |
1386
+ | 262,144 | 222,822 | 235,929 | 235,929 | fraction |
1387
+ | 1,000,000 | 850,000 | 900,000 | 900,000 | fraction |
1388
+
1389
+ The headroom is computed as `window - int(fraction * window)` rather than
1390
+ `(1 - fraction) * window`, so the unclamped case reproduces the fraction
1391
+ exactly. The two differ by the truncation, and truncating the headroom
1392
+ down put the trigger one token ABOVE `int(fraction * window)` -- harmless
1393
+ as a number, not harmless as an identity, since every table above would
1394
+ have been off by one with nothing to point at.
1395
+
1396
+ ### `compaction_min_headroom_tokens`
1397
+
1398
+ The floor on the headroom above. Sized from `max_read_chars` (50,000
1399
+ chars = 12,500 proxy tokens, the largest a single bounded read returns)
1400
+ plus the assistant turn that asked for it. It binds under a ~160k window
1401
+ and nowhere above one, which is the whole point: a 262k or 1M thread gets
1402
+ `compaction_trigger_fraction` exactly as written, and a 64k local model
1403
+ keeps a usable margin instead of the 6,553 tokens a bare share leaves it.
1404
+
1405
+ Not every tool respects `max_read_chars` -- `arxiv_search` does not, and
1406
+ one of its results measured 29,249 proxy tokens (TECHNICAL_DEBT 41). This
1407
+ floor does not cover that, and above ~160k the fraction leaves more room
1408
+ than it anyway; what it covers is the bounded case at the bottom of the
1409
+ roster, where nothing else would.
1410
+
1411
+ Both halves of that stay true and the ITEM is closed (batch 126).
1412
+ `arxiv_search` still does not consult `max_read_chars` -- it should not, a
1413
+ character cap over an assembled dict cuts it mid-structure -- but it now
1414
+ carries its own bounds, per field and on the wire, so the row that sized
1415
+ this floor can no longer occur. The floor's derivation is unchanged: it was
1416
+ sized from `max_read_chars` plus the turn that asked for it, never from
1417
+ this row. What the row did was prove the fraction had to be CLAMPED, and
1418
+ that argument does not depend on the tool staying unbounded.
1419
+
1420
+ ### `compaction_max_headroom_fraction`
1421
+
1422
+ The cap on the headroom, and it is load-bearing rather than defensive:
1423
+ 16,000 tokens of a 32,768-token window is half of it, and an unclamped
1424
+ floor under a small window derives a trigger at or below zero -- or one
1425
+ under `keep_recent_tokens`, where `effective_compaction()` raises on every
1426
+ evaluation and `should_compact()` runs at the top of every step. This is
1427
+ the clamp that keeps the bottom of the range working at all.
1428
+
1429
+ BATCH 44 REVERSED M1, AND THE REASON M1 GAVE HAS EXPIRED. M1 rejected a
1430
+ window-derived trigger because "a window-derived trigger sits at a size
1431
+ the thread can never reach in working order" -- and the arithmetic
1432
+ behind that was about MAX_TOKEN_BUDGET, then a default spend ceiling of
1433
+ 250k: the prompt is re-sent on every step of a tool-using turn, so a
1434
+ thread at 160k got one response and no tool calls before the cap ended
1435
+ the turn. Batch 27 DELETED that default ceiling. There is no cap for a
1436
+ large trigger to collide with any more, so the premise is gone and the
1437
+ flat 40k outlived it -- compacting a 1M-window thread at 4% of its
1438
+ window, spending a model call and losing fidelity to solve a problem
1439
+ the model did not have.
1440
+
1441
+ THE COST OF THE REVERSAL, STATED. MODEL_CONTEXT_WINDOWS and the provider
1442
+ query decide when every thread folds now, where under M1 a wrong entry
1443
+ only mistimed a research backstop. That is why core/model_windows.py
1444
+ exists (the user's own answer, outranking both) and why the fallback
1445
+ still WARNS. A fraction rather than `window - margin`: a fixed margin
1446
+ that is right for a 1M model leaves a 64k model no working room at all,
1447
+ and one that is right for 64k never fires on 1M.
1448
+
1449
+ ### `compaction_trigger_tokens`
1450
+
1451
+ The trigger used when NO MODEL IS IN SCOPE -- and the default
1452
+ settings.json's `compaction.trigger_tokens` overrides.
1453
+
1454
+ One caller has no (provider, model) to derive a window from:
1455
+ compaction.pin_measurements, which computes #89's cap as
1456
+ PIN_MAX_TRIGGER_FRACTION x the trigger from a thread id alone. Left at
1457
+ 40_000 deliberately, so the pin cap is exactly what it was before batch
1458
+ 44 -- this batch changes when a thread compacts, not how much of one a
1459
+ single pin may freeze, and a cap that grew with the window would let one
1460
+ `pin` call freeze 425k tokens of a 1M-window thread.
1461
+
1462
+ ### `compaction_warning_margin_tokens`
1463
+
1464
+ Fires this many tokens BEFORE the trigger, so there is room to do
1465
+ something about it -- pin a message, wrap up a thought, run /compact at a
1466
+ natural break.
1467
+
1468
+ STILL ABSOLUTE after batch 44, and that is a decision rather than an
1469
+ oversight. What this buys is a HUMAN'S REACTION TIME, measured in turns,
1470
+ and a turn does not get bigger because the window did: 8k is roughly a
1471
+ turn or two of notice on a 64k model and on a 1M one alike. As a
1472
+ fraction of the new trigger it would be 128k on a 1M model -- a warning
1473
+ that fires while the thread is at 72% and has nothing to worry about
1474
+ yet. §21 singles this out as the one value whose wrong setting
1475
+ is not self-correcting: too small and it is an alert with no time
1476
+ attached to it. Validated to be strictly less than the trigger.
1477
+
1478
+ ### `compaction_keep_recent_tokens`
1479
+
1480
+ The most recent tokens that always stay verbatim, never compacted.
1481
+
1482
+ ABSOLUTE, like the warning margin above and for the same kind of reason:
1483
+ this is an amount of CONVERSATION to keep intact, not a share of a
1484
+ window. It is also the weaker of the two floors in practice -- M5 keeps
1485
+ whichever protects more, and the three-turn floor below almost always
1486
+ protects more than 4k on the long threads that reach the trigger at all.
1487
+ Left where it was so batch 44 changes when a thread folds and not what
1488
+ survives the fold.
1489
+
1490
+ ### `compaction_keep_recent_turns`
1491
+
1492
+ M5: a FLOOR in turns, on top of the token floor above, and the two
1493
+ compose -- whichever protects more wins, plus the current turn, which is
1494
+ never foldable at all. A single tool-heavy turn can consume the whole
1495
+ keep-token budget by itself, leaving the immediately preceding exchange
1496
+ summarized; a follow-up like "no, the other one" then has no referent.
1497
+
1498
+ ### `pin_max_trigger_fraction`
1499
+
1500
+ #89 (batch 16). The most one `pin` call may protect, as a fraction of
1501
+ the trigger above. A pin is a PERMANENT FLOOR under the derived view:
1502
+ pinned rows re-enter it verbatim forever (M9), nothing folds them, and
1503
+ before this cap one ungated call with last_n=40 could put a thread
1504
+ permanently past the trigger with no unpin to undo it. 0.5 leaves room
1505
+ for a real working session while making "floor the trigger by yourself"
1506
+ structurally impossible. Enforced at the tool with a REFUSAL that states
1507
+ the cap and the current share -- never silently trimmed (M15's rule: a
1508
+ pin asked for more protection than allowed must not quietly deliver less).
1509
+
1510
+ ### `compaction_strength`
1511
+
1512
+ 1-5, mapping to the target compression ratios below.
1513
+
1514
+ ### `compaction_max_retries`
1515
+
1516
+ Bounded corrective retries when the compactor misses its target ratio.
1517
+
1518
+ ### `compaction_target_ratios`
1519
+
1520
+ §21: strength is an objectively measurable target, not a qualitative
1521
+ instruction. "Be more aggressive" behaves differently across models with
1522
+ different summarization tendencies and no amount of prompting fixes that;
1523
+ a ratio is checkable after the fact. Qualitative guidance still belongs
1524
+ in the compactor's prompt, shaping WHAT survives within the budget.
1525
+
1526
+ ### `compaction_ratio_tolerance`
1527
+
1528
+ How far outside the target a summary may land before it is sent back.
1529
+ Under-shooting is not an error: a summary tighter than asked for has
1530
+ already done the job, and rejecting it would spend a second call to make
1531
+ the context bigger.
1532
+
1533
+ ### `compaction_strategy`
1534
+
1535
+ M2 as amended by batch 16 (#90, owner decision). Two strategies:
1536
+
1537
+ ```text
1538
+ "chain" summarizes the PREVIOUS SUMMARY plus whatever followed it.
1539
+ Constant cost per compaction forever -- the input never
1540
+ grows past summary-plus-tail -- at the price of loss that
1541
+ compounds over a long-lived thread (a summary of a summary
1542
+ of a ...).
1543
+ "rederive" summarizes the ORIGINAL messages every time, so exactly
1544
+ one summarization step always sits between an original
1545
+ message and what the model sees. Fidelity-optimal; input
1546
+ grows with the covered span.
1547
+ ```
1548
+
1549
+ The DEFAULT is "chain", reversing M2's original choice -- an explicit
1550
+ owner decision recorded in DEVLOG (batch 16): rederive's whole-span
1551
+ input made every compaction the most expensive call of the turn, and
1552
+ cost compounds on exactly the threads that trigger most. Whatever is
1553
+ configured here, BOTH strategies fall back to chain automatically when
1554
+ a span outgrows one call (built at last, after three documents promised
1555
+ it for two sections), saying so in a WARNING -- and truncate the oldest
1556
+ material with a stated truncation when even chain cannot fit, because
1557
+ an oversized send is never the right failure direction.
1558
+
1559
+ ### `compaction_strategies`
1560
+
1561
+ ### `max_injected_memories`
1562
+
1563
+ ROADMAP_v2 §21b (M14). Ceiling on how many durable memories reach one
1564
+ prompt. Newest first, and the truncation is stated in the fragment the
1565
+ model reads rather than only in a log.
1566
+
1567
+ Every in-scope memory would otherwise enter every turn forever, which is
1568
+ the unbounded context growth §21 exists to fight -- a memory feature that
1569
+ quietly reintroduces the problem compaction solves would be a poor trade.
1570
+ Recency is the ordering because it is the only staleness signal available
1571
+ without asking a model, and /forget is how a still-relevant old memory
1572
+ gets kept ahead of the cap.
1573
+
1574
+ ## Catalog text (ROADMAP_v2 §32, A5)
1575
+
1576
+ ### `max_catalog_text_chars`
1577
+
1578
+ The cap on a skill's or an agent's `name` and `description` -- the
1579
+ two strings that go from a .md file straight into the system prompt
1580
+ of every run in a project, with no tool call and no further consent
1581
+ (#131).
1582
+
1583
+ 300 is headroom rather than a squeeze: the longest description this
1584
+ project ships is `compactor` at 171 characters, and all eight are
1585
+ single-line. A project whose description does not fit in twice the
1586
+ length of ours is writing instructions, not a summary, which is
1587
+ exactly the thing the cap is for.
1588
+
1589
+ THE NEWLINE COLLAPSE MATTERS MORE THAN THE NUMBER, and they are one
1590
+ constant because they are one rule -- see config_loader._catalog_text.
1591
+ Same posture as tools/builtin/arxiv.MAX_SUMMARY_CHARS (600) and
1592
+ web_search.MAX_SNIPPET_CHARS (300): model-facing text from a source
1593
+ this project did not write is bounded where it is produced.
1594
+
1595
+ ROADMAP_v3 §51 (PG8) left both caps exactly where they are and made
1596
+ them SAY when they bit. A value cut at a cap and a value that happens
1597
+ to be that long were indistinguishable, so "the paper does not mention
1598
+ X" and "the first 600 characters do not" read the same to the model.
1599
+ The flag is the fix; raising the cap is not, for the reason above.
1600
+
1601
+ ## Summaries, references and the checklist (ROADMAP_v2 §21c / §23)
1602
+
1603
+ ### `compactor_agent`
1604
+
1605
+ The agent that does the summarizing. §21: this is the shape of an agent
1606
+ call -- a system prompt, a task, a judgment-based output -- so it runs
1607
+ through the same RunAgentLoop as everything else rather than needing its
1608
+ own LLM-calling path, and that is also what gives it real autonomy over
1609
+ WHAT to preserve. Mechanical truncation cannot exercise judgment at all.
1610
+
1611
+ ### `summary_target_chars`
1612
+
1613
+ ROADMAP_v2 §21c. How long a whole-thread summary may be, in characters.
1614
+
1615
+ ABSOLUTE, not a COMPACTION_TARGET_RATIOS entry, and the difference is the
1616
+ consumer rather than the summarizer. A fold's summary REPLACES the span it
1617
+ came from, so scaling with that span is right: a bigger fold earns a bigger
1618
+ summary and the thread still shrinks. §21c's summary is injected as a prompt
1619
+ tier present on every turn of the REFERENCING thread, where nothing it
1620
+ replaces bounds it -- a 500KB thread at strength 3 would put 75KB into every
1621
+ call indefinitely.
1622
+
1623
+ A thread whose rendered text already fits this is stored verbatim and costs
1624
+ no model call at all.
1625
+
1626
+ ### `max_injected_refs`
1627
+
1628
+ §21c. How many referenced threads may be attached to one thread at once.
1629
+
1630
+ Small on purpose, and the cap REFUSES rather than dropping the oldest: a
1631
+ reference is something a person chose by name, and silently discarding one
1632
+ to make room for another is exactly what §21b's "removal is by id, never by
1633
+ substring" rule exists to prevent. The count and the cap are stated in the
1634
+ fragment the model reads, per M14's no-silent-caps rule.
1635
+
1636
+ ### `todo_statuses`
1637
+
1638
+ §23 slice 2. The todo list's vocabulary and its ceiling.
1639
+
1640
+ A plain tuple rather than an Enum, for storage.py's THREAD_KIND_* reason: a
1641
+ list persisted under an older vocabulary reads back as data rather than
1642
+ raising, and the tool can then say which status it did not recognise.
1643
+
1644
+ ### `max_todo_items`
1645
+
1646
+ The list is injected into every turn's system prompt, so its length is a
1647
+ per-call cost for as long as the thread lives. REFUSES rather than
1648
+ truncating, MAX_INJECTED_REFS' rule: a checklist silently cut to 50 would
1649
+ have the model believe it had recorded work it has now forgotten, which is
1650
+ worse than being told to write a shorter list.
1651
+
1652
+ ### `compaction_pipeline_backstop_tokens`
1653
+
1654
+ M6. A research pass is headless and unattended, and each one already
1655
+ returns a distillation, so routine compaction there would spend on a
1656
+ judgment call nobody is watching. Passes compact only when approaching
1657
+ the model's actual context window -- a safety net against a hard provider
1658
+ error mid-pipeline, not a working-set policy.
1659
+
1660
+ ## Context windows (ROADMAP_v2 §21 M1, as amended by batches 30 and 44)
1661
+
1662
+ ### `model_context_windows`
1663
+
1664
+ The backstop's source, the summarizer's input budget, and M1's remaining
1665
+ use for a window table.
1666
+
1667
+ NOW A FALLBACK, NOT THE ONLY SOURCE. This said "deliberately NOT a
1668
+ per-provider API query", on the grounds that the roster of fifteen had no
1669
+ uniform endpoint and a query would need fifteen adapters to avoid a
1670
+ fallback constant they would all still need. The premise was measured
1671
+ against the PINNED SDKs and did not survive:
1672
+
1673
+ ```text
1674
+ anthropic 0.116.0 ModelInfo.max_input_tokens -- on the SAME object
1675
+ core/client.py already retrieves to read
1676
+ capabilities.effort. The answer was in hand and
1677
+ being dropped; reading it costs no extra call.
1678
+ google-genai 2.29.0 types.Model.input_token_limit, one field (as 1.0.0).
1679
+ openai 2.45.0 Model is declared extra="allow", so a provider's
1680
+ own field survives onto .model_extra. That makes
1681
+ the OpenAI-compatible side ONE alias reader --
1682
+ context_window (Groq), context_length (Together,
1683
+ OpenRouter, Cohere), max_context_length (Mistral)
1684
+ -- rather than thirteen adapters.
1685
+ ```
1686
+
1687
+ So it is two attribute reads and one sniff, not fifteen adapters. The
1688
+ query lives in core/client.py:context_window_for, beside the effort query
1689
+ it mirrors; core/compaction.py:context_limit prefers it and falls back
1690
+ here. (The old note also corrected itself on a provider count from the
1691
+ dead APICredentials class -- audit #23. Fifteen is right; the count was
1692
+ never what was wrong with the argument.)
1693
+
1694
+ THE CONCLUSION OUTLIVED THE PREMISE, which is why this table stays: OpenAI,
1695
+ DeepSeek and Perplexity answer /models with id/created/object/owned_by and
1696
+ nothing else, and Grok, Fireworks, Qwen and Z.AI publish no model-metadata
1697
+ endpoint at all. A fallback constant is still needed at the end of it.
1698
+
1699
+ KEYS ARE STORED NORMALIZED -- no date suffix, no `vendor/` or `region.`
1700
+ prefix. core/compaction.py:_normalized reduces an incoming id to this form
1701
+ before looking it up, so a dated key here would be UNREACHABLE. That is
1702
+ also the bug this fixed: matching was exact, so `claude-sonnet-5-20260724`
1703
+ missed a million-token entry and silently took the default.
1704
+
1705
+ AN INCOMPLETE ENTRY IS EXPENSIVE NOW. Under M1 it cost a mistimed
1706
+ research backstop and a mistimed summarizer input budget, because the
1707
+ working-set trigger was a flat number that never consulted this. Batch 44
1708
+ made that trigger a FRACTION of what context_limit() returns, so a wrong
1709
+ number here mistimes compaction on every thread -- too high and the
1710
+ thread never folds and the provider raises a hard context-limit error
1711
+ instead. core/model_windows.py is the user's answer to that, and /window
1712
+ is how they give it; this table is what answers before they do.
1713
+
1714
+ **Notes on individual entries.**
1715
+
1716
+ 1M IS THE ORDINARY WINDOW ON THESE, not a beta.
1717
+
1718
+ This block used to carry the opposite claim -- that 1M was gated
1719
+ behind a `context-1m-2025-08-07` header this harness does not send,
1720
+ so an ordinary account got 200_000 and these entries "want to be
1721
+ 200_000". That was true of the Sonnet 4 era and was never true of
1722
+ the models listed here; checked against the current model reference
1723
+ in batch 44, the whole 4.6-and-later family is 1M and Haiku 4.5 is
1724
+ 200K. The values were right and the comment was stale, which is the
1725
+ dangerous combination now that the trigger reads them: a reader
1726
+ following the old note would have "corrected" five entries by a
1727
+ factor of five and compacted every Claude thread at a fifth of the
1728
+ window.
1729
+ Dateless, per the normalization note above: this was keyed
1730
+ "claude-haiku-4-5-20251001" while its five siblings were dateless,
1731
+ which is the inconsistency that exposed the exact-match bug.
1732
+ Alibaba markets "1M"; the documented input limit is 983,616, and
1733
+ drops to that figure specifically when thinking is enabled. The
1734
+ SMALLER number is the right one to record: this feeds the pipeline
1735
+ compaction backstop (context_limit - COMPACTION_PIPELINE_BACKSTOP_
1736
+ TOKENS), where erring low compacts slightly early and erring high
1737
+ means a hard context-limit error from the provider mid-run.
1738
+
1739
+ ### `default_context_window`
1740
+
1741
+ Logged at WARNING ONCE PER MODEL, per §21 -- a wrong-by-default window
1742
+ means compaction and the backstop both fire at the wrong time in
1743
+ whichever direction the guess is wrong, and silent guessing turns a
1744
+ tuning problem into a mystery. Once per model rather than once per use,
1745
+ because context_limit() is on should_compact()'s path and that runs at
1746
+ the top of every step of every turn; the dedup set lives in
1747
+ core/compaction.py. A long session therefore says this once and is then
1748
+ silent, which is the trade -- the line is actionable exactly once, and
1749
+ since batch 44 it names the action (/window, which remembers).
1750
+
1751
+ 256_000 rather than 200_000 (batch 44): the floor a hosted agentic model
1752
+ ships with today. Below that is essentially always a self-hosted
1753
+ deployment, whose operator knows the number and can say it once with
1754
+ /window -- which is the case this default cannot serve and does not try
1755
+ to. Erring low here would compact every unknown model early forever to
1756
+ protect the rarer case that has an owner who can answer.
1757
+
1758
+ ## Connecting a model (ROADMAP_v3 §79)
1759
+
1760
+ ### `hf_oauth_client_id`
1761
+
1762
+ ROADMAP_v3 §79 (HD5). The Hugging Face OAuth app `/connect huggingface` signs
1763
+ in through. The default is the project's own app, registered by the owner on
1764
+ 2026-10-20, so the sign-in works in every install and every hosted deployment
1765
+ without anyone registering another. Another app's id signs in through that app
1766
+ instead; empty hides the sign-in and leaves a pasted token working.
1767
+
1768
+ **A client id is public, and that is what makes it a `config.yaml` value.** The
1769
+ app is registered with NO secret (Hugging Face calls this a public app), with
1770
+ the device code flow enabled, so nothing about it needs hiding: it names which
1771
+ app asks, and the person approving the request sees that name on Hugging
1772
+ Face's own page. The scopes live on the app's registration (`openid profile
1773
+ inference-api`), not here, so editing this value can change WHICH app asks but
1774
+ not what it may do.
1775
+
1776
+ **Not an authority key.** It cannot widen a tool permission or an approval, and
1777
+ `settings.json` cannot set it at all, like every other key in this file.
1778
+
1779
+ ## Tool permissions and approvals (ROADMAP_v2 §15, D24)
1780
+
1781
+ ### `tool_permissions`
1782
+
1783
+ APICredentials was here and is deleted (audit #23). It was dead -- nothing
1784
+ constructed it, imported it or type-hinted against it -- and its guidance
1785
+ contradicted the mechanism that is real: it told the reader to put the key
1786
+ in an environment variable and write the VARIABLE'S NAME here, while
1787
+ credentials.save_credentials stores the key value directly in
1788
+ providers.json. Anyone following it ended up with the literal string
1789
+ "MY_KEY_VAR" as their API key.
1790
+
1791
+ Credentials flow through credentials.py (providers.json) and env_secrets.py
1792
+ (.env) -- the deliberate two-mechanism split D19 describes, which
1793
+ env_secrets.py documents at length on the other side. The provider roster
1794
+ lives in providers.json.example, beside the file it describes.
1795
+
1796
+ #### `tool_permissions.fetch_url` -- ships `true`
1797
+
1798
+ ROADMAP_v2 §15/D24: fetch_url was registered and documented as working
1799
+ but had no field here, so getattr(..., False) denied every call since
1800
+ the tool was added. Allowed by default, matching web_search -- it is
1801
+ already constrained by policy_enforcement's blocked-domain list and
1802
+ its output goes through secret redaction like every other tool's.
1803
+
1804
+ #### `tool_permissions.shell_background` / `shell_monitor` / `shell_sessions` / `shell_output` / `shell_kill` -- ship `false`
1805
+
1806
+ ROADMAP_v3 §49 (SS11). The five background-session tools, shipped disabled
1807
+ for `shell`'s reason and not a weaker one: a session IS a shell command, and
1808
+ one that keeps running after the call that started it has been answered.
1809
+ Enabling them without `shell` is allowed and coherent -- a session is
1810
+ classified, approved and routed by exactly the same rules -- but a session
1811
+ also holds a container for as long as its timeout, blocks the user's prompt
1812
+ while it runs, and can wake the agent, so it is more authority rather than
1813
+ less.
1814
+
1815
+ The two that START one are approval-gated by `shell_approval_mode` alone
1816
+ (SS16) and have no `tool_approvals` key, exactly like `shell`; the three
1817
+ that list, read and stop need no approval, because they act only on sessions
1818
+ this conversation already had approved. All five are additionally hidden
1819
+ wherever nothing would ever wake the run -- a research pass, or any run
1820
+ outside a chat turn or a subagent (SS17).
1821
+
1822
+ #### `tool_permissions.spawn_subagent` -- ships `true`
1823
+
1824
+ §18: spawning is allowed by default (model autonomy, D6) and needs no
1825
+ approval -- the spawned run's own tools are each gated by policy,
1826
+ intersection-capped by the parent's context (C6).
1827
+
1828
+ #### `tool_permissions.pin` -- ships `true`
1829
+
1830
+ §21/D26. Thread-scoped, reversible (batch 16 gave that word a
1831
+ mechanism: `unpin`), and it takes effect inside a conversation the
1832
+ user is watching -- a wrong pin costs some context budget and nothing
1833
+ else.
1834
+
1835
+ #### `tool_permissions.unpin` -- ships `true`
1836
+
1837
+ §21/D26, restored by batch 16 (#89). D26's premise called a pin
1838
+ reversible while nothing anywhere unpinned one; unpin is the other
1839
+ half of making the premise true rather than aspirational. Same axis,
1840
+ same gating posture, same reasons.
1841
+
1842
+ #### `tool_permissions.remember` -- ships `true`
1843
+
1844
+ §21b. Allowed everywhere; whether it can actually RUN is decided
1845
+ by the approval gate below plus §13's headless rule.
1846
+
1847
+ #### `tool_permissions.read_project_doc` -- ships `true`
1848
+
1849
+ §24 (I2). Allowed, unlike `read`, because it is confined to the
1850
+ project directory by realpath and to documentation/manifest files by
1851
+ an allowlist -- see tools/builtin/project_docs.py for what that
1852
+ deliberately excludes and why the narrow set is load-bearing.
1853
+
1854
+ #### `tool_permissions.write_project_doc` -- ships `true`
1855
+
1856
+ §24 (I1). Allowed, unlike `write`, because it takes a document NAME
1857
+ from a fixed allowlist rather than a path: the destination is derived,
1858
+ so the tool cannot be aimed anywhere else. The gate is below.
1859
+
1860
+ #### `tool_permissions.ask_user` -- ships `true`
1861
+
1862
+ §23 slice 2. Allowed everywhere, and ungated below (J12): whether it
1863
+ can actually reach a person is decided by whether the run has a
1864
+ response channel, which the tool itself checks.
1865
+
1866
+ #### `tool_permissions.todo_write` -- ships `true`
1867
+
1868
+ §23 slice 2. Thread-scoped, reversible, and it asks nobody -- so it is
1869
+ allowed and ungated for `pin`'s reasons (J9).
1870
+
1871
+ ### `tool_approvals`
1872
+
1873
+ #### `tool_approvals.fetch_url` -- ships `false`
1874
+
1875
+ No approval: a tool with approval=True is unusable wherever there is
1876
+ no permission_channel to answer the prompt (the CLI chat loop and
1877
+ every research pass), so gating fetch_url would leave it denied in
1878
+ exactly the places the grounding passes need it -- the same outcome
1879
+ as the D24 bug, with a different error string. See DEVLOG §15.
1880
+
1881
+ #### `tool_approvals.shell` -- removed, not shipped
1882
+
1883
+ There is deliberately no such key. Shell approval is governed solely by
1884
+ `shell_approval_mode` above: the field this section used to describe was
1885
+ the RATCHET (`True` forced `always` whatever the mode said), and every
1886
+ outcome it could produce is an outcome the mode already names -- so a
1887
+ second switch could only ever agree with the gate or surprise someone who
1888
+ set the two differently. An agent's `approval_overrides` keeps its own
1889
+ one-way power over shell through the context layer (D14); what is gone is
1890
+ only the global duplicate. An old file still carrying the key is refused
1891
+ at startup naming it (unknown keys are refused, always), and the npm-update
1892
+ merge drops it as "not a setting in this version" while keeping every
1893
+ other edit.
1894
+
1895
+ #### `tool_approvals.shell_background` / `shell_monitor` -- removed, not shipped
1896
+
1897
+ ROADMAP_v3 §49 (SS16). There are deliberately no such keys, for the reason
1898
+ `tool_approvals.shell` has none: a start tool runs a shell command, so
1899
+ `shell_approval_mode` above decides it, and a second switch here could only
1900
+ agree with the gate or surprise whoever set the two differently. They are
1901
+ registered with shell's own approval check -- by object identity, asserted
1902
+ at import in `tools/registry.py` -- so the exemption cannot outlive the gate
1903
+ that justifies it.
1904
+
1905
+ #### `tool_approvals.shell_sessions` / `shell_output` / `shell_kill` -- ship `false`
1906
+
1907
+ ROADMAP_v3 §49 (SS11): UNGATED, on the axis that separates `pin` from
1908
+ `remember`. Listing sessions, reading one's output and stopping one act
1909
+ only on sessions this conversation started, each of which was already
1910
+ approved as a command; none of them starts anything or reaches outside the
1911
+ thread. Gating them would also make them invisible exactly where they are
1912
+ needed, since an approval-gated tool is not advertised where nothing can
1913
+ ask (§13) -- an agent able to start a session but not to read or stop it is
1914
+ the worse failure.
1915
+
1916
+ #### `tool_approvals.spawn_subagent` -- ships `true`
1917
+
1918
+ Approving a spawn is the §18 subagent sign-off: it authorises the
1919
+ child's whole approval-gated tool set for the rest of the turn, so
1920
+ delegation itself has to be the thing approved. Consequence, named
1921
+ rather than discovered: the headless callability filter drops
1922
+ spawn_subagent anywhere nothing can grant the approval -- CLI chat,
1923
+ and any research run that is neither attended nor granting. The
1924
+ once-per-run notice names it.
1925
+
1926
+ §25 R4 declares it GRANT_NEVER (R13, on the ToolSpec): it can never
1927
+ be PRE-granted, in a research run OR at §18's sign-off, because
1928
+ approving a spawn hands the child a whole gated set and one
1929
+ launch-time tick would compound into unbounded delegated authority
1930
+ across ten unattended passes. #67 is why "or at the sign-off" is in
1931
+ that sentence: R4's argument is ABOUT the sign-off, and the
1932
+ exclusion had never reached it. An ATTENDED run can still approve a
1933
+ spawn live,
1934
+ which is a per-call human decision and exactly what the gate is for.
1935
+ The child then runs headless with nothing granted, since
1936
+ spawn_subagent forwards a grant only alongside a permission_channel.
1937
+
1938
+ #### `tool_approvals.pin` -- ships `false`
1939
+
1940
+ §21/D26: pin does NOT require approval, and the asymmetry with
1941
+ remember (§21b) is the decision. They differ on the axis that already
1942
+ separates `read` from `write` -- pin is thread-scoped and reversible,
1943
+ while remember writes something that outlives the thread and silently
1944
+ shapes conversations the user has not started yet.
1945
+
1946
+ Consequence, named rather than discovered: this is what lets pin work
1947
+ on the CLI and inside a research pass, where an approval-gated tool
1948
+ is not merely denied but not advertised at all (§13).
1949
+
1950
+ #### `tool_approvals.unpin` -- ships `false`
1951
+
1952
+ §21/D26, batch 16 (#89). Ungated for pin's reasons, and now genuinely
1953
+ symmetric: releasing protection costs context budget in a
1954
+ conversation the user is watching, exactly like applying it.
1955
+
1956
+ #### `tool_approvals.remember` -- ships `true`
1957
+
1958
+ §21b/D26, and the asymmetry with pin above IS the decision. A
1959
+ memory outlives its thread and silently shapes conversations the
1960
+ user has not started yet, so it is persistent and invisible at the
1961
+ moment it matters -- the same axis that separates write from read.
1962
+
1963
+ Consequence: unreachable on any headless path, which includes every
1964
+ research pass. That is deliberate (§21's consequence 1) and is
1965
+ reinforced by the tool's GRANT_NEVER policy (M17, carried by R13),
1966
+ because a grant would otherwise route around it.
1967
+
1968
+ #### `tool_approvals.read_project_doc` -- ships `false`
1969
+
1970
+ §24 (I2): no approval. Reading the project's own documentation, at the
1971
+ user's explicit request, on the files the manifest already listed to
1972
+ them, is not an escalation -- and a gate here would mean a dozen
1973
+ prompts for one /init, which is the shape of consent that gets clicked
1974
+ through. The confinement in project_docs.py is the control.
1975
+
1976
+ #### `tool_approvals.write_project_doc` -- ships `true`
1977
+
1978
+ §24 (I1): gated, on the same axis that separates `read` from `write`.
1979
+ This writes a file into the user's project that is then injected into
1980
+ future prompts, and it invalidates the D17 trust hash as it goes.
1981
+ /init supplies the approval itself, as the yes to a rendered diff --
1982
+ so the gate is what makes "no silent overwrite" (§24 AC2) structural
1983
+ rather than a promise the command makes about itself.
1984
+
1985
+ #### `tool_approvals.ask_user` -- ships `false`
1986
+
1987
+ §23 slice 2 (J12): UNGATED. Two reasons, and the second is the one
1988
+ that decides it. Gating would mean approving a prompt in order to be
1989
+ shown a prompt -- but worse, §13 does not merely deny a gated tool
1990
+ where nothing can ask, it stops ADVERTISING it. Gated, this tool would
1991
+ be invisible in every headless run; §24 AC2 requires it be visible,
1992
+ called, and answered with a denial the model can work around. Asking
1993
+ a person is not an action that needs authorising -- it is the least
1994
+ unilateral thing a run can do.
1995
+
1996
+ #### `tool_approvals.todo_write` -- ships `false`
1997
+
1998
+ §23 slice 2 (J9): UNGATED, on the axis that already separates `pin`
1999
+ from `remember`. A todo list is thread-scoped and reversible, and
2000
+ rewriting it costs some prompt budget inside a conversation the user
2001
+ can see. Gating it would also make it invisible in every research pass
2002
+ (§13), which is where a checklist across ten passes helps most.