@ponythewhite/base-context 1.0.11 → 1.0.13

package/docs/compaction.md

@@ -35,16 +35,67 @@ or loss from a summary alone.
 
 ### When It Triggers
 
-Auto-compaction triggers when:
+Auto-compaction uses 90% of the model context window as its default soft target:
 
 ```
-contextTokens > contextWindow - reserveTokens
+softTarget = floor(0.9 * contextWindow)
+threshold = min(contextWindow - reserveTokens, max(softTarget, fixedContextTokens + 4 * keepRecentTokens))
+contextTokens > threshold
 ```
 
-By default, `reserveTokens` is 16384 tokens (configurable in `~/.base-context/settings.json` or `<project-dir>/.base-context/settings.json`). This leaves room for the LLM's response.
+With default `keepRecentTokens` of 20000 and `reserveTokens` of 16384, before
+fixed-context headroom raises the target, the thresholds are 244800 for a
+272000-token model, 111616 for a 128000-token model, and 47616 for a 64000-token
+model. The model-window reserve can therefore trigger compaction before 90%.
+The model-window ceiling always applies.
+
+`fixedContextTokens` estimates current system instructions, tool schemas, the
+current TaskFrame and latest harness snapshot. It does not count all historical
+messages or obsolete snapshots as fixed. Reserving room above this context avoids
+repeated ineffective summaries when required instructions already exceed the
+soft target. These are local estimates, not exact provider token counts.
+
+Set `compaction.targetTokens` to a positive safe integer to replace the soft
+target; required-context headroom can still raise a numeric target. Set
+`"model-limit"` to use exactly `contextWindow - reserveTokens` instead.
+Configure these settings in `~/.base-context/settings.json` or
+`<project-dir>/.base-context/settings.json`.
+
+This is a compaction heuristic, not a strict request cap, a validated backend
+limit, or a promise of optimal token use. Earlier summaries trade shorter replay
+for summary calls and possible rereads. Required instructions and replay
+dependencies are not clipped to meet this target. Explicit SDK request-token
+budgets remain separate; see [context management](context-management.md#model-aware-budgets).
 
 You can also trigger manually with `/compact [instructions]`, where optional instructions focus the summary — for example `/compact focus on the auth refactor, remember the exact migration command`. The instructions are passed to the summarization prompt with high priority, persisted on the `CompactionEntry`, and shown on the `[compaction]` message in the TUI.
 
+### Agent-Requested Compaction
+
+The agent can request compaction before the automatic threshold, including when
+using Astra. The bundled `compact` skill is enabled by default and runs from the
+Python REPL:
+
+```python
+await compact.status()
+await compact.run("keep the remaining plan and exact test names")
+```
+
+`status()` reports `tokens`, `context_window`, `percent`, and `scheduled`. Usage
+can be unknown immediately after compaction. `run()` schedules compaction at the
+next turn boundary, not in the middle of the Python cell. It returns
+`{"scheduled": True}` when accepted, or `{"scheduled": False, "reason": ...}`
+when no turn is active or there is nothing to compact yet. Optional instructions
+focus the summary. Repeating the request before the boundary updates them.
+Interrupted tool work can continue after the checkpoint; this does not start new
+work after an otherwise completed turn.
+
+This request does not depend on the automatic threshold or `compaction.enabled`.
+It requires context optimization to be on. `compaction.agentCallable: false`
+disables the skill, and disabling Python or bundled skills can remove its normal
+entry point. Each recursive agent uses its own session's compaction path; children
+inherit the parent's compact-skill availability. There is no Astra-specific gate.
+A request compacts the requesting session, not the whole agent tree.
+
 ### How It Works
 
 1. **Find cut point**: Walk backwards from newest message, accumulating token estimates until `keepRecentTokens` (default 20k, configurable in `~/.base-context/settings.json` or `<project-dir>/.base-context/settings.json`) is reached
@@ -432,9 +483,11 @@ Configure compaction in `~/.base-context/settings.json` or `<project-dir>/.base-
 | `enabled` | `true` | Enable auto-compaction |
 | `reserveTokens` | `16384` | Headroom used by the compaction threshold |
 | `keepRecentTokens` | `20000` | Estimated recent-token target for the retained tail |
+| `targetTokens` | 90% of the model context window | Positive safe integer to override the soft target, or `"model-limit"` for the model-window threshold; fixed-context headroom and the model reserve still apply as described above |
+| `agentCallable` | `true` | Expose the `compact` skill so the agent can request earlier compaction |
 
 Disable automatic compaction with `"compaction": { "enabled": false }`. Manual
-`/compact` remains available while `context.mode` is `"on"`. Setting `context.mode`
+`/compact` and agent-requested compaction remain available while `context.mode` is `"on"`. Setting `context.mode`
 to `"off"` disables context optimization, including manual compaction; logging,
 recovery, limits and other non-optimization ownership remain active.
 

package/docs/context-management.md

@@ -82,7 +82,29 @@ Profiles identify the API, provider, endpoint, final model, context allowance, o
 
 In supported native Responses/Codex selection paths, historical assistant literals can become optional at accepted boundaries. Users, the latest assistant, TaskFrames, summaries, recovery, and required dependencies remain mandatory. If that set cannot fit, the request can refuse. Unsupported media, opaque layouts, or missing contracts also remain explicit limits.
 
-See [SDK request-token profiles](sdk.md#explicit-request-token-budget-profiles) for the exact scope and configuration. Ordinary [compaction](compaction.md) and its `reserveTokens`/`keepRecentTokens` settings are separate.
+See [SDK request-token profiles](sdk.md#explicit-request-token-budget-profiles) for the exact scope and configuration.
+
+Ordinary [compaction](compaction.md) uses a separate soft target of 90% of the
+model context window: `floor(0.9 * contextWindow)`. The actual trigger also leaves
+`4 * keepRecentTokens` above estimated fixed context, then caps the result at
+`contextWindow - reserveTokens`. Fixed context includes current system
+instructions, tool schemas, the current TaskFrame and latest harness snapshot,
+not every historical message. This avoids repeated ineffective summaries of
+noncompactable instructions.
+
+With small fixed context, default thresholds are 244800 estimated tokens for a
+272000-token model, 111616 for a 128000-token model, and 47616 for a 64000-token
+model. The model-window reserve can trigger compaction before 90%. Set
+`compaction.targetTokens` to a positive safe integer to replace the soft target
+(still subject to fixed-context headroom), or `"model-limit"` to use exactly
+`contextWindow - reserveTokens`. Required replay groups are not clipped to meet
+this target. Summaries can add calls and rereads; this is not a strict request
+cap, an optimality guarantee, or evidence of a deployment's actual limit.
+
+The agent can request an earlier checkpoint from the Python REPL with
+`await compact.run()`, independently of the automatic threshold. It can inspect
+usage with `await compact.status()`. This is enabled by default for all models,
+including Astra; see [agent-requested compaction](compaction.md#agent-requested-compaction).
 
 ## Stable context epochs
 

package/docs/rlm.md

@@ -101,7 +101,7 @@ await agent_message.send(
 
 #### Child handles and lifecycle
 
-An admission handle contains `rlm_child_id`, `name`, `session_dir`, and `model`. Child usage is attributed to the parent session while remaining distinguishable in context-tree reporting.
+An admission handle contains `rlm_child_id`, `name`, `session_dir`, and `model`. Child usage is attributed to the parent session while remaining distinguishable in context-tree reporting. Native child usage reporting continues after passivation and rehydration. `/context` separately totals saved request receipts for the captured family; parent attribution records are not added to those totals. See [token usage and status](usage.md#token-usage-and-status) for missing-data, catalog-estimate, and goal-budget scope.
 
 The parent-scoped child registry survives compaction, kernel restart, and parent restoration:
 

package/docs/settings.md

@@ -94,6 +94,8 @@ Private download manifests require `version` and `package` (or `packageName`) se
 | `compaction.enabled` | boolean | `true` | Enable auto-compaction |
 | `compaction.reserveTokens` | number | `16384` | Tokens reserved for LLM response |
 | `compaction.keepRecentTokens` | number | `20000` | Recent tokens to keep (not summarized) |
+| `compaction.targetTokens` | positive integer or `"model-limit"` | 90% of the model context window | Override the soft compaction target, still capped by the model window minus reserve |
+| `compaction.agentCallable` | boolean | `true` | Expose the `compact` skill so the agent can request compaction before the automatic threshold |
 | `compaction.model` | object | Current main model and effort | Explicit summary model: `provider`, `modelId`, and `thinkingLevel` are all required |
 
 ```json
@@ -107,6 +109,25 @@ Private download manifests require `version` and `package` (or `packageName`) se
 ```
 
 
+Without `targetTokens`, the soft target is 90% of the model context window:
+`floor(0.9 * contextWindow)`. The trigger also leaves `4 * keepRecentTokens` above
+an estimate of current fixed instructions, tool schemas, TaskFrame and latest
+harness snapshot. It is capped at `contextWindow - reserveTokens`. With small
+fixed context, default thresholds are 244800 for a 272000-token model, 111616 for a
+128000-token model, and 47616 for a 64000-token model. The model-window reserve can
+therefore trigger compaction before 90%. A positive safe integer replaces the
+soft target, but required-context headroom can raise it; `"model-limit"` uses
+exactly `contextWindow - reserveTokens`. This is a summary heuristic, not strict
+provider-request admission. See [compaction](compaction.md#when-it-triggers) and
+[model-aware budgets](context-management.md#model-aware-budgets).
+
+With the default `compaction.agentCallable: true`, the agent can use
+`await compact.status()` and `await compact.run(instructions=None)` in the Python
+REPL. This model-independent path includes Astra and can request compaction before
+the automatic threshold. It also works with `compaction.enabled: false`, but new
+requests require `context.mode: "on"`. Requests run at a turn boundary, not
+mid-cell. See [agent-requested compaction](compaction.md#agent-requested-compaction).
+
 Set `compaction.model` to choose the model and effort for manual, automatic and
 model-requested compaction summaries. For example, when this exact model/route is
 configured and budgeted:

package/docs/usage.md

@@ -49,7 +49,7 @@ Type `/` in the editor to open command completion. Extensions can register custo
 | `/new` | Start a new session |
 | `/name <name>` | Set session display name |
 | `/session` | Show session file, ID, and message counts |
-| `/usage`, `/context` | Show the parent and subagent context, token, and cost breakdown |
+| `/usage`, `/context` | Show captured-family request usage, context, and separate goal-budget scope |
 | `/tree` | Jump to any point in the session and continue from there |
 | `/fork` | Create a new session from a previous user message |
 | `/clone` | Duplicate the current active branch into a new session |
@@ -73,6 +73,29 @@ The value is saved as the global `rlmMaxSubagents` preference, so it survives re
 
 Lowering the limit never kills or passivates existing agents. They keep running or remain idle. Only new spawns/admissions are blocked until the live count is below the limit. This slash command is separate from `base-context agents`, which lists agents.
 
+## Token usage and status
+
+`/context` (also `/usage`) separates generation usage from the goal budget:
+
+- When saved provider-attempt receipts are available, it shows each captured agent's own usage by purpose: main work, child work, compaction, refinement, status, and other recorded generation calls. The captured-family total counts each source once. It does not add assistant or child-attribution projections on top of receipts.
+- Processed tokens equal total input (including cache) plus output. Cached input is not added again. Uncached input, cache read/write, and output are also shown separately.
+- Missing or partial usage, unsettled attempts, unavailable prices, and agents without receipts are explicit. Missing values are not zero. The captured family may not include every historical child or request.
+- Dollar amounts are catalog estimates from recorded rates, not provider invoices. Unavailable prices are not treated as free usage.
+- Older sessions or daemons without receipt data use a labeled legacy conversation projection. It can omit auxiliary calls and is not whole-family provider spend.
+- The goal budget remains **root successful main uncached input + output only**. Cached input, child work, and auxiliary calls do not consume that counter.
+
+Dashboard summaries do not request inference again when their bounded input is unchanged. Changed working input and retries are coalesced with a 60-second minimum interval and a short settle debounce. An observed working-to-idle transition can request a final summary without waiting for that interval. Local activity, queues, and terminal errors remain visible without a new summary call. The selected status model is unchanged.
+
+## Harness refinement
+
+`/refine` updates the editable continual harness. Manual and automatic refinement keep their existing scheduling and scope rules.
+
+Static harness instructions stay in the system prompt. Fresh compact entries and selected error-fix advice enter the next owned model request as a saved **Continual Harness Snapshot**, rather than rewriting the system prefix. Direct kernel harness edits use the same path. Unchanged snapshots are not appended again.
+
+The latest snapshot supersedes older snapshots, including deletions, rollbacks, and empty state. Entries remain advice below system instructions and the current task. Resume and native context epochs use the retained source records; if compaction removes the snapshot, the next request adds the current state again. Snapshot rows are hidden in the normal UI but remain model-visible.
+
+This preserves fresh advice while reducing prefix changes. It does not guarantee a provider cache hit or reduce refinement frequency.
+
 ## Message Queue
 
 You can submit messages while the agent is still working:

package/package.json

@@ -1,6 +1,6 @@
 {
 	"name": "@ponythewhite/base-context",
-	"version": "1.0.11",
+	"version": "1.0.13",
 	"description": "Synerise base-context: a coding and research agent with durable context and a persistent Python REPL",
 	"type": "module",
 	"bin": {
@@ -42,9 +42,9 @@
 	},
 	"dependencies": {
 		"@agentclientprotocol/sdk": "^1.3.0",
-		"@ponythewhite/base-context-agent": "1.0.11",
-		"@ponythewhite/base-context-ai": "1.0.11",
-		"@ponythewhite/base-context-tui": "1.0.11",
+		"@ponythewhite/base-context-agent": "1.0.13",
+		"@ponythewhite/base-context-ai": "1.0.13",
+		"@ponythewhite/base-context-tui": "1.0.13",
 		"@silvia-odwyer/photon-node": "^0.3.4",
 		"chalk": "^5.5.0",
 		"cli-highlight": "^2.1.11",