dsh-context-compression-improved 0.5.2 → 0.5.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (28) hide show
  1. package/.gitattributes +1 -0
  2. package/CHANGELOG.ja.md +144 -119
  3. package/CHANGELOG.ko.md +143 -118
  4. package/CHANGELOG.md +278 -250
  5. package/CHANGELOG.zh.md +131 -109
  6. package/docs/installation.md +103 -103
  7. package/docs/installation.zh.md +100 -100
  8. package/package.json +1 -1
  9. package/packages/selector/cordis.patch.yml +5 -6
  10. package/packages/selector/src/client/EstimatorControls.tsx +277 -277
  11. package/packages/selector/src/client/locales.ts +196 -196
  12. package/packages/selector/src/index.ts +463 -463
  13. package/packages/selector/src/pruner/state.ts +50 -50
  14. package/packages/selector/src/pruner.ts +2402 -2402
  15. package/packages/selector/src/runtime/tokenpilot/advisor-prompt.ts +188 -188
  16. package/packages/selector/src/runtime/tokenpilot/advisor-state.ts +149 -149
  17. package/packages/selector/src/runtime/tokenpilot/advisor.ts +419 -419
  18. package/packages/selector/src/runtime/tokenpilot/benefit.ts +200 -200
  19. package/packages/selector/tests/advisor-report.host.spec.ts +223 -223
  20. package/packages/selector/tests/public/package-contract.client.spec.ts +20 -0
  21. package/packages/selector/tests/runtime/advice-never-withholds.host.spec.ts +232 -232
  22. package/packages/selector/tests/runtime/advisor-invariant.spec.ts +272 -272
  23. package/packages/selector/tests/runtime/advisor.spec.ts +226 -226
  24. package/packages/selector/tests/runtime/char-basis.spec.ts +30 -30
  25. package/packages/selector/tests/runtime/deprecated-preset-options.spec.ts +96 -96
  26. package/packages/selector/tests/runtime/tokenpilot/benefit.spec.ts +217 -217
  27. package/packages/selector/tests/settings-seat.client.spec.ts +29 -4
  28. package/scripts/toolclass-corpus-replay.mjs +281 -281
package/CHANGELOG.md CHANGED
@@ -1,250 +1,278 @@
1
- # Changelog
2
-
3
- All notable changes use this file. The project follows semantic versioning after `0.1.0`.
4
-
5
- ## 0.5.2 - 2026-09-20
6
-
7
- ### Changed
8
-
9
- - The human-gated review pipeline (beta) is **retired**. Its semantics are replaced by
10
- advice: the benefit model still prices every pass as ONE merged mutation, but the band it
11
- computes (`profitable` / `high-impact` / `slow-payback` / `unpriceable` / `not-worth-it`)
12
- is now published as a `reduction-advice` audit record and snapshotted onto the advisor
13
- report route it never withholds, delays, or rewrites a reduction. The gate contradicted
14
- the feature's own requirement (a reduction must never block automatic processing) and, with
15
- the shipped defaults (`reviewMode` on plus a 4,000-token high-impact threshold against an
16
- 8,192-token fresh trigger), diverted 100% of a fresh batch into human review, leaving the
17
- automatic path effectively off for anyone who opted in. The advice thresholds are now
18
- module constants `0.1`, high-impact `4,000` tokens): nothing acts on them, so they are
19
- no longer settings.
20
- - Removed with the gate: the review queue and its `storageDomain` adapter, the process-wide
21
- registry, the `review-queue` / `review-decide` HTTP routes and the `reviewQueueRoute`
22
- deployment flag, the `shell.overlay` client panel, the `reviewMode` /
23
- `reviewTimeoutTurns` / `cacheHitDiscountAlpha` / `reviewHighImpactTokens` settings keys,
24
- and the `review-outcome` audit kind. The four settings keys stay ACCEPTED and IGNORED by
25
- both decoders an existing document (the live one carries `reviewMode: false`) still loads
26
- and still renders its settings card — and never reach the resolved policy. The read-only
27
- `GET .../advisor-report` route additionally serves `lastAdvice`.
28
-
29
- ### Added
30
-
31
- - Regression pins for the retirement: a host-integration spec drives the exact settings that
32
- used to divert everything (`reviewMode: true`, `reviewHighImpactTokens: 1`) and asserts the
33
- fresh batch LANDS, described by a `high-impact` advice record; a deprecation-contract spec
34
- pins accept-and-ignore for the retired keys on both the runtime parser and the browser
35
- decoder.
36
-
37
- ### Rollback
38
-
39
- - Reinstall the last release that still ships the gate: `npm dist-tag add
40
- dsh-context-compression-improved@0.5.1 dsh-0.1.5 --registry https://registry.npmjs.org/`
41
- then `dsh plugin --profile web add dsh-context-compression-improved@0.5.1`.
42
-
43
- ## 0.5.1 - 2026-09-20
44
-
45
- ### Fixed
46
-
47
- - Concurrent preset-overlay composition of one identity no longer fails on Windows.
48
- Publication is now serialized per destination path, and when the atomic rename still
49
- loses the race the destination is confirmed to already carry this staging file's
50
- `{mtimeMs, size}` standing key before the publish reports success. Windows `MoveFileEx`
51
- reports that lost race as `EPERM`/`EBUSY` where POSIX `rename` simply replaces the
52
- destination, which made `standingKeyFor()` throw during concurrent session start. A
53
- destination that does not match still fails loudly, so a silently reused generation
54
- stays forbidden.
55
- - Release-gate and test repairs that had been hiding this and other pre-existing red
56
- gates: stale identifiers and a retired audit reason in the packed smokes, a stale
57
- client-inject expectation, and Windows-only spawn traps (`git` and multi-line
58
- `node -e` scripts were routed through `cmd.exe`, which rewrote their arguments).
59
-
60
- ## 0.5.0 - 2026-09-20
61
-
62
- ### Added
63
-
64
- - Advisory relevance advisor (statistics and suggestions only, default off): at every turn
65
- boundary a fire-and-forget pass summarizes the session's tail-task semantics from the
66
- most recent `todo/write` event (falling back to recent user text), incrementally scores
67
- historical tool-result candidates for content-and-comment relevance against the current
68
- task, and computes a prefix-decay figure (character-pressure-weighted mean relevance).
69
- Low-relevance old segments are marked `recertified` as suggestions for later
70
- history-aggressiveness decisions — nothing this round consumes them, and advisor output
71
- can never suppress, delay, or rewrite any reduction that would land (pinned by a
72
- dedicated invariant test). Configure through the `presetOptions.advisor*` settings keys
73
- (`advisorMode` `''|'host'|'direct'`, default `''`; the direct channel reuses the
74
- estimator endpoint; `SideChannel` gained an optional overrides parameter so the
75
- estimator's transport is shared without sharing its configuration). Observability: new
76
- `advisor-outcome` audit records (content-free, one per phase: summary / scoring / decay)
77
- and a read-only `GET .../advisor-report?sessionId=` HTTP route (opt-in deployment flag
78
- `advisorReportRoute`, mirroring the review routes). Client UI is intentionally absent
79
- this round.
80
-
81
- ## 0.4.0 - 2026-09-20
82
-
83
- ### Fixed
84
-
85
- - The plugin no longer depends on the routed model id: every planning gate now decides on
86
- the character basis (Unicode code points via `characterPressure` / `pressureCost`) instead
87
- of exact tokenizer counts, so a route without a bundled tokenizer (the live
88
- `deepseek-flash`) lands rewrites again instead of silently skipping. Token thresholds keep
89
- their names and values (converted at the documented 4.0 chars/token convention, frozen
90
- profile baseline untouched); token figures become telemetry, labelled honestly by the new
91
- `measurementBasis` field on rewrite audit records (`exact-tokenizer` vs `characters`,
92
- with `tokenizerId: 'characters'` / `tokenizerRevision: 'chars-per-token-4.0'` when derived).
93
- Rollback: `git revert 7a1972a` restores the exact-tokenizer gates as a single unit; confirm
94
- no later change re-introduced a model-id split that relies on this commit first.
95
-
96
- ### Added
97
-
98
- - Batch-level benefit pricing (R1): a review pass is priced as ONE merged mutation — the
99
- tail KV-cache refill penalty is paid once per batch instead of per candidate, so real
100
- batches (5×50k with a 64k tail) reach the auto band instead of all dropping.
101
- - Original-event line mapping (R9a): terminal normalization returns folded lines that each
102
- carry their 1-based original-event line number; `retrieve` reads raw events, so printed
103
- ranges resolve to the right lines.
104
- - Document skeleton and universal prose keep (R8/R8b): structured documents keep headings,
105
- section first/last lines, list starts and table headers; every other non-code text keeps
106
- head AND tail with an R9 line-range marker (prose was previously head-only truncated).
107
- - Two-tier search folding (R10, fixes D8): a lossless per-file L1 locator plus a
108
- water-filled L2 content quota.
109
- - Non-adjacent frequency folding (R11): separated exact repeats (up to 8.37% of large
110
- results) fold to first occurrence plus one counted marker.
111
- - Long-string placeholders (R12): base64/hex/UUID blobs become length summaries with a
112
- 16-char recognition prefix.
113
- - Two-stage HTML reduction (R13, fixes D9): `html-slim` then `html-skeleton`, line-aligned
114
- so original line numbers survive.
115
- - R9b anchors: contiguous masks cite original line ranges, scatter masks report
116
- `lines 1-N scanned, K kept`, and every retrieve hint carries a pasteable
117
- `{"ref":…,"start_line":N,"max_lines":80}`.
118
- - Document census: omitted-document summaries list section headings instead of a constant
119
- `0 error, 0 warn, N info` histogram.
120
-
121
- ## 0.3.1 - 2026-09-18
122
-
123
- ### Changed
124
-
125
- - The runtime package is merged into the selector package: one install brings the whole
126
- stack, the repository root is the install surface (`name`, `main`, `types`, `exports` with
127
- `./pruner` and `./invariant`, `dependencies`, `dsh`), and the toolchain, scripts and CI were
128
- swept to the single package. The verified estimator-catalog registration (dual prefix, guarded
129
- two-channel activation, per-request service resolution, visible lifecycle lines) was replayed
130
- onto this line with a host-side guard; the settings schema that `ab2175a` had downgraded to
131
- `z.any()` is restored, so the daily Custom defaults are published again.
132
- - Adapt to the official DeepSeek Harness `v0.1.5-rc.2` on this branch. All `@deepseek-ai/dsh-*` dev dependencies and the pinned e2e host set move from `0.1.1-rc.2` to `0.1.5-rc.2` (cordis `4.0.2`, schemastery `3.18.2`), including the new split packages (`dsh-session-projection`, `dsh-session-persistence`, `dsh-atomic-write`, `dsh-home-paths`, `dsh-sandbox`, and related) and the `dsh-client-store` client stack.
133
- - Surface replace operations now use the v3 `startSeq`/`endSeq` shape with branded `SessionSeq` values; `compaction/prune` manifests keep the durable `start`/`end` fields. Events are resolved from surface nodes by seq lookup instead of array indexing.
134
- - The client bundle no longer imports the removed `@deepseek-ai/dsh-client-runtime`: settings types now come from `@deepseek-ai/dsh-client-ui-settings` and the session hooks merge from `@deepseek-ai/dsh-client-ui-session`. `engines.dsh >=0.1.5-alpha.1 <0.2.0-0` is declared in both package manifests and `dsh.plugin.json`.
135
- - Harness 0.1.5 no longer exposes the session `agentPreset` to the browser, so the client can no longer detect Minimal-only sessions; the selector stays selectable and the old unavailable banner is unreachable.
136
- - Test batteries updated for 0.1.5 semantics: cordis plugin starts require `.await()`, the Token Meter requires a mounted `SessionProjectionRegistry`, assistant events carry `stream: []`, and settings namespaces are plain strings.
137
-
138
- ### Fixed
139
-
140
- - The estimator card no longer demands an API key on the Harness host channel. Selecting the host channel shows the live provider/model dropdowns, names the route that would actually run (explicit override, else the session default), and renders neither a key field nor a second manual model input: the base URL, the model text field, and the write-only key belong to the direct endpoint channel alone.
141
- - `presetOptions` writes are path-addressed. Writing the whole section replaced it, so touching any second estimator field (a provider, a model, an endpoint) deleted `estimatorMode` and every sibling override — silently switching the estimator back off while the panel still reported a successful save. Each field now writes only itself, `undefined` clears exactly the field it names, and the confirmation read validates the same field set instead of the mode alone.
142
- - New coverage: `packages/selector/tests/preset-options-write.client.spec.ts` (path-scoped writes, sibling preservation, explicit clears, no-op patches, uncommitted-write reporting) and `packages/selector/tests/estimator-channel.client.spec.tsx` (per-channel fields, catalog dropdowns, manual fallback).
143
-
144
-
145
- ### Added
146
-
147
- - New `tokenpilot-inspired` profile: a TokenPilot-paper-inspired capability matrix layered on the Balanced thresholds, selected explicitly from the settings UI; every pre-existing profile keeps a byte-identical resolved policy (enforced by a captured-baseline golden test).
148
- - Byte-identical repeated tool-result dedup: an oversized repeat is replaced with a pointer to the first occurrence's append-only original event (`dedupe-pointer`), with a per-session SHA-256 index (2,048-entry insertion-order eviction, hash+seq metadata only).
149
- - No-net-savings guard: replacements whose text is not smaller than the original are rejected even when the exact tokenizer reports a token saving.
150
- - Recovery exemption: recovery-tool output is permanently exempt from every reduction pass via a unified per-session exemption set, preventing compress-restore oscillation.
151
- - Auto Compact summary locator: after `compaction/end`, the landed summary checkpoint gains an Exact Sources block (shadowed seq range, spill files, touched files) so summarized-away details stay recoverable; skipped when it would locate nothing concrete.
152
- - Read-state semantics: a historical read whose file was later mutated is `superseded` and takes the small whole-result placeholder; optional error/warn/info clustering of omitted lines is appended to historical placeholders.
153
- - Optional residual-utility estimator (three channels: off / Harness host model / direct OpenAI-compatible endpoint) with per-session exponential backoff, strict timeout, advisory-only verdicts consumed by the next pressure pass, and numeric-only `estimator-outcome` audits. The estimator card appears only while the new profile is selected; the API key is write-only in settings and never enters the frozen policy, audits, or logs.
154
- - New audit records: `summary-locator` and `estimator-outcome`; the rewrite record covers dedup via the `dedupe-pointer` reducer. Audit field allowlists are unchanged.
155
- - Simplified Chinese and English copy for the new profile and estimator card; unit and golden coverage under `packages/runtime/tests/tokenpilot/`.
156
-
157
- - Orthogonal code-skeleton compression gate (`codeSkeleton.enabled`, default off): the first exposure of an oversized fresh source-code tool result can keep an imports-and-declarations skeleton with bodies elided and error lines preserved, falling back to the original head pruning. The gate is independent of every profile and gated on exact tokenizer measurement.
158
- - Settings-UI toggle for the gate in the selector settings section, with Simplified Chinese and English copy.
159
- - Browser/runtime decode parity for the new section, confirm-on-write contract tests for `saveCodeSkeleton`, and a full-document parity matrix extension.
160
-
161
- ### Changed
162
-
163
- - Added an ESLint flat-config baseline (`pnpm lint`, enforced in CI) and a `pnpm test:watch` TDD loop; removed dead imports and hardened two error paths surfaced by the lint baseline.
164
- - This repository is now maintained as an improved fork of `WilliamShi666/dsh-context-compression-selector`; documentation ships in English, Simplified Chinese, Japanese, and Korean.
165
-
166
- ## 0.1.0 - 2026-09-03
167
-
168
- ### Added
169
-
170
- - Stable release of DeepSeek V4 Flash Vision tokenizer integration for `deepseek-v4-flash-vision-exp`, including exact text counting and bounded image-token estimates.
171
- - User-configurable model-driven Auto Compact threshold in the selector settings section.
172
- - Auto Compact threshold linkage for each standard profile's History / micro-compact watermarks and related compression parameters.
173
-
174
- ### Changed
175
-
176
- - The threshold editor now uses one direct numeric input; the slider and fixed quick-value buttons were removed.
177
- - Runtime session-event access supports both the established Harness `events` accessor and the newer `snapshotEvents()` public API.
178
-
179
- ## 0.1.0-beta.4 - 2026-09-02
180
-
181
- ### Fixed
182
-
183
- - Support the official DeepSeek Harness `dsh-v0.1.2-alpha.5` public API while retaining compatibility with the existing `0.1.1-rc.2` peer range. The plugin now owns the two small immutable-value helpers that the newer Harness no longer exports, and uses the same public `context-compression` namespace literal accepted by both Settings implementations. No Harness core code is modified.
184
-
185
- ## 0.1.0-beta.3 - 2026-09-01
186
-
187
- ### Scope
188
-
189
- This is a staged release. Exact **text-class** token counting for `deepseek-v4-flash-vision-exp`, best-effort bounded **vision-class image** estimates, and the Auto Compact threshold/UI/audit work are delivered. Exact image measurement remains **BLOCKED upstream**: the current measurement seam exposes neither the adapter's projected request-image dimensions nor the absolute serialized position, so estimates cannot be promoted to `exact-tokenizer`.
190
-
191
- ### Added
192
-
193
- - DeepSeek V4 Flash Vision support for `deepseek-v4-flash-vision-exp` via a separately bundled official tokenizer pinned at `deepseek-ai/DeepSeek-V4-Flash-Vision-Exp` revision `6821d6ad3681a4b137b066b76094fa82ebd0a380`. Text, reasoning, tool-call arguments, and pure-text tool results are counted exactly; image-bearing tool-result candidates stay fail-open.
194
- - Vision image-token arithmetic ported line-by-line from the official `inference/image_processor.py` (patch size 14, downsample 3, 384-token cap, min pixels 147456, 8:1 aspect clamp, and position-dependent alignment padding), validated against golden fixtures generated by executing the official Python implementation. Valid intrinsic dimensions now produce `tokenizer-estimate` at the midpoint of all four alignment residues with a 384-token per-image upper bound; malformed or unevaluable dimensions use a documented 256-token fallback. Mixed text/image surfaces aggregate exact text and estimated images without promoting them to exact.
195
- - `autoCompact.thresholdPercent` setting (default 80, integer 50–90, step 1) with one shared validation contract across the settings UI, the persisted schema, and the runtime resolver. The editor lives inside the context-compression selector settings section.
196
- - Standard-profile History linkage to the Auto Compact watermark: `A = floor(C × a)` rescales the History trigger, minimum reclaim, and recent-token tail; `D = floor(A × 0.875)` replaces the fixed 0.7 capacity-pressure ratio as the micro-compact last-chance gate; one batch must justify its cache break by pulling the complete request back below the deadline. Defaults at 80% reproduce the previous numbers exactly.
197
- - The preset overlay writes the saved threshold into the generated `compaction-basic` composition as `thresholdRatio` (with `retainRatio` pinned at 0.16) and, from the same read, into the plugin runtime's deployment config as `autoCompactThresholdPercent`, so one standing generation never runs Auto Compact and micro compact on two different thresholds. Any generation-identity change — threshold, source, or module paths, including equal-length ones — produces a new standing composition generation. Deterministic identity-derived stamps use an 8-hex whole-second window; equal-prefix identities can collide in that first window on a coarse filesystem, so the overlay observes the staging file's real `mtimeMs+size` key and escalates to later hash windows before the atomic rename. Content, permissions, and the final unique stamp are complete before publication; already-running sessions keep their frozen policy.
198
- - `policy-resolved` audits now record the Auto Compact coordination facts (threshold percent, `A`, `D`, parameter source — including `deployment-override`/`mixed` when deployment config replaces linked History watermarks), the routed provider/model, and the bundled tokenizer identity.
199
- - Persisted settings reject present-but-invalid sections (`profile: null`, `custom: null`, own-property `undefined`) before any schema default can absorb them; a malformed stored document freezes the session losslessly (`profile: off`, audited as `settingsInvalidFallback: lossless-off`) instead of silently enabling the lossy Balanced default. The browser decoder applies the same rule and canonicalizes legacy Custom v1/v2 documents to the same v3 document the runtime resolver produces.
200
- - The History planner returns a discriminated outcome, and `component-evaluation` audits distinguish the full skip taxonomy: `below-profile-trigger`, `below-micro-deadline`, `exact-tokenizer-unavailable`, `no-safe-candidates` (only recovery-tool output or already-cleared results), `protected-working-set` (everything inside the protected tail), `insufficient-reclaim` and `cannot-reach-deadline-target` (with the reached/required token numbers), `adaptive-cost-rejected`, and `recovery-tool-unavailable`.
201
-
202
- ### Known limitations
203
-
204
- - Images never claim exact counts. The official expansion depends on the absolute prompt position (system prompt, chat-template framing, adapter image handles) and on the adapter's final request-image projection (including per-route `imagePixelBudget`/`imageDetail` overrides and byte-cap reprojection), none of which is exposed through the current measurement seam. Intrinsic/default estimates may therefore differ materially from provider accounting. Upstream capability requests remain projected request-image dimensions and the absolute serialized position exposed to token-meter extensions.
205
- - History skips the whole batch whenever any tool-result candidate lacks an exact count, including image-bearing candidates, even though sibling text candidates are individually exact.
206
- - Custom remains manual token mode; its History parameters do not follow the Auto Compact watermark.
207
- - A vision token breakdown UI is not shipped; image estimates and the intrinsic alignment diagnostic are available on the measured token view, while lossy rewrite proofs still require exact counts.
208
-
209
- ### Deferred
210
-
211
- - Audit `modality` field and the tokenizer artifact SHA-256 inside audit records (the audits already carry the routed provider/model and tokenizer identity).
212
- - Publishing the (now complete) runtime skip-reason taxonomy as a user-facing documentation table.
213
- - Custom-profile display of the A/D watermarks and an above-D warning; Custom remains fully manual.
214
- - Vision token breakdown UI and promotion of image estimates to exact measurement.
215
-
216
- ## 0.1.0-beta.2 - 2026-08-28
217
-
218
- ### Fixed
219
-
220
- - Resolve the official DeepSeek V4 Flash tokenizer route so Fresh and Aggregate can evaluate tool results for the supported V4 models.
221
- - Run Cache Strict History at the real request boundary once its configured capacity-pressure condition is met; trigger the capacity condition at 70% routed-context utilization.
222
- - Disable Harness-native head/middle/tail tool-result pruning whenever a selector profile is active, leaving the selector as the sole tool-result compactor.
223
-
224
- ### Changed
225
-
226
- - Protect the newest 10 agent tool calls and a 64,000-token tool-result tail window before History/microcompact rewrites older results.
227
-
228
- ## 0.1.0-beta.1 - 2026-08-27
229
-
230
- ### Added
231
-
232
- - One-install DeepSeek Harness Product Bundle backed by a separate exact-version runtime package.
233
- - Web profile selector with preset-stable settings and an explicit built-in Minimal exception.
234
- - Fresh, Aggregate, routine/capacity-aware History, Native tool-result pruning, and default-off Custom TailTrim.
235
- - Standard-event TailTrim protocol using `compaction/prune` plus recoverable `user/message` replacement.
236
- - Plugin-owned `context_compression_retrieve` recovery tool.
237
- - Structured, content-free policy, evaluation, rewrite, failure, and Native auto-compact audit records.
238
- - Pinned official DeepSeek V4 tokenizer assets with runtime SHA-256 validation and upstream license.
239
- - Public-API component E2E, preset/Minimal, and parent/fork/spawn cache-prefix regression tests.
240
-
241
- ### Compatibility
242
-
243
- - Verified against DeepSeek Harness `dsh-v0.1.1-rc.2` public packages.
244
- - Exact tokenizer mapping is currently limited to `deepseek-v4-flash` and `deepseek-v4-pro`.
245
-
246
- ### Known limitations
247
-
248
- - Adaptive ordinary History fails closed when public request-level route/cache evidence is incomplete; capacity pressure remains a separate safety override.
249
- - Cache-prefix tests prove native fork inheritance and identical serialized prefixes, not a provider-specific cache allocation or a guaranteed DeepSeek cache hit.
250
- - Settings snapshots and first-exposure decisions are process-local to the mounted runtime.
1
+ # Changelog
2
+
3
+ All notable changes use this file. The project follows semantic versioning after `0.1.0`.
4
+
5
+ ## 0.5.3 - 2026-09-20
6
+
7
+ ### Fixed
8
+
9
+ - The Bundle patch no longer sets the retired review route flag. `reviewQueueRoute` was
10
+ removed from the plugin's Config schema when the review gate was retired, but it stayed in
11
+ `packages/selector/cordis.patch.yml`, where it advertised a route that can never register.
12
+ Hosts tolerate the unknown key today (the plugin loads and serves verified on a live host),
13
+ so this is a stale-config defect rather than a break, but it would break on any host that
14
+ validates plugin config strictly.
15
+ - Generated artifacts are pinned to LF (`packages/selector/lib/** text eol=lf`). With
16
+ `core.autocrlf=true` every checkout rewrote the committed `lib/**` to CRLF, so any branch
17
+ switch or merge left the whole artifact directory reported as modified with EOL-only
18
+ differences. Nothing was ever committed from that state, but it made every tree look dirty
19
+ and it would hide a real artifact change.
20
+
21
+ ### Added
22
+
23
+ - A negative-control contract pin on the Bundle patch: it fails when a retired config key is
24
+ set in the patch (asserted on key-setting lines, so the explanatory comment may still name
25
+ the key), and it requires the live `estimatorCatalogRoute` flag to stay wired.
26
+
27
+ ### Tests
28
+
29
+ - The client seat contract also pins the older-host degradation: a host that does not declare
30
+ the seat rejects the registration at the slot boundary, and `apply()` must swallow it and
31
+ warn instead of taking every settings entry down with it.
32
+
33
+ ## 0.5.2 - 2026-09-20
34
+
35
+ ### Changed
36
+
37
+ - The human-gated review pipeline (beta) is **retired**. Its semantics are replaced by
38
+ advice: the benefit model still prices every pass as ONE merged mutation, but the band it
39
+ computes (`profitable` / `high-impact` / `slow-payback` / `unpriceable` / `not-worth-it`)
40
+ is now published as a `reduction-advice` audit record and snapshotted onto the advisor
41
+ report route it never withholds, delays, or rewrites a reduction. The gate contradicted
42
+ the feature's own requirement (a reduction must never block automatic processing) and, with
43
+ the shipped defaults (`reviewMode` on plus a 4,000-token high-impact threshold against an
44
+ 8,192-token fresh trigger), diverted 100% of a fresh batch into human review, leaving the
45
+ automatic path effectively off for anyone who opted in. The advice thresholds are now
46
+ module constants (α `0.1`, high-impact `4,000` tokens): nothing acts on them, so they are
47
+ no longer settings.
48
+ - Removed with the gate: the review queue and its `storageDomain` adapter, the process-wide
49
+ registry, the `review-queue` / `review-decide` HTTP routes and the `reviewQueueRoute`
50
+ deployment flag, the `shell.overlay` client panel, the `reviewMode` /
51
+ `reviewTimeoutTurns` / `cacheHitDiscountAlpha` / `reviewHighImpactTokens` settings keys,
52
+ and the `review-outcome` audit kind. The four settings keys stay ACCEPTED and IGNORED by
53
+ both decoders an existing document (the live one carries `reviewMode: false`) still loads
54
+ and still renders its settings card — and never reach the resolved policy. The read-only
55
+ `GET .../advisor-report` route additionally serves `lastAdvice`.
56
+
57
+ ### Added
58
+
59
+ - Regression pins for the retirement: a host-integration spec drives the exact settings that
60
+ used to divert everything (`reviewMode: true`, `reviewHighImpactTokens: 1`) and asserts the
61
+ fresh batch LANDS, described by a `high-impact` advice record; a deprecation-contract spec
62
+ pins accept-and-ignore for the retired keys on both the runtime parser and the browser
63
+ decoder.
64
+
65
+ ### Rollback
66
+
67
+ - Reinstall the last release that still ships the gate: `npm dist-tag add
68
+ dsh-context-compression-improved@0.5.1 dsh-0.1.5 --registry https://registry.npmjs.org/`
69
+ then `dsh plugin --profile web add dsh-context-compression-improved@0.5.1`.
70
+
71
+ ## 0.5.1 - 2026-09-20
72
+
73
+ ### Fixed
74
+
75
+ - Concurrent preset-overlay composition of one identity no longer fails on Windows.
76
+ Publication is now serialized per destination path, and when the atomic rename still
77
+ loses the race the destination is confirmed to already carry this staging file's
78
+ `{mtimeMs, size}` standing key before the publish reports success. Windows `MoveFileEx`
79
+ reports that lost race as `EPERM`/`EBUSY` where POSIX `rename` simply replaces the
80
+ destination, which made `standingKeyFor()` throw during concurrent session start. A
81
+ destination that does not match still fails loudly, so a silently reused generation
82
+ stays forbidden.
83
+ - Release-gate and test repairs that had been hiding this and other pre-existing red
84
+ gates: stale identifiers and a retired audit reason in the packed smokes, a stale
85
+ client-inject expectation, and Windows-only spawn traps (`git` and multi-line
86
+ `node -e` scripts were routed through `cmd.exe`, which rewrote their arguments).
87
+
88
+ ## 0.5.0 - 2026-09-20
89
+
90
+ ### Added
91
+
92
+ - Advisory relevance advisor (statistics and suggestions only, default off): at every turn
93
+ boundary a fire-and-forget pass summarizes the session's tail-task semantics from the
94
+ most recent `todo/write` event (falling back to recent user text), incrementally scores
95
+ historical tool-result candidates for content-and-comment relevance against the current
96
+ task, and computes a prefix-decay figure (character-pressure-weighted mean relevance).
97
+ Low-relevance old segments are marked `recertified` as suggestions for later
98
+ history-aggressiveness decisions nothing this round consumes them, and advisor output
99
+ can never suppress, delay, or rewrite any reduction that would land (pinned by a
100
+ dedicated invariant test). Configure through the `presetOptions.advisor*` settings keys
101
+ (`advisorMode` `''|'host'|'direct'`, default `''`; the direct channel reuses the
102
+ estimator endpoint; `SideChannel` gained an optional overrides parameter so the
103
+ estimator's transport is shared without sharing its configuration). Observability: new
104
+ `advisor-outcome` audit records (content-free, one per phase: summary / scoring / decay)
105
+ and a read-only `GET .../advisor-report?sessionId=` HTTP route (opt-in deployment flag
106
+ `advisorReportRoute`, mirroring the review routes). Client UI is intentionally absent
107
+ this round.
108
+
109
+ ## 0.4.0 - 2026-09-20
110
+
111
+ ### Fixed
112
+
113
+ - The plugin no longer depends on the routed model id: every planning gate now decides on
114
+ the character basis (Unicode code points via `characterPressure` / `pressureCost`) instead
115
+ of exact tokenizer counts, so a route without a bundled tokenizer (the live
116
+ `deepseek-flash`) lands rewrites again instead of silently skipping. Token thresholds keep
117
+ their names and values (converted at the documented 4.0 chars/token convention, frozen
118
+ profile baseline untouched); token figures become telemetry, labelled honestly by the new
119
+ `measurementBasis` field on rewrite audit records (`exact-tokenizer` vs `characters`,
120
+ with `tokenizerId: 'characters'` / `tokenizerRevision: 'chars-per-token-4.0'` when derived).
121
+ Rollback: `git revert 7a1972a` restores the exact-tokenizer gates as a single unit; confirm
122
+ no later change re-introduced a model-id split that relies on this commit first.
123
+
124
+ ### Added
125
+
126
+ - Batch-level benefit pricing (R1): a review pass is priced as ONE merged mutation the
127
+ tail KV-cache refill penalty is paid once per batch instead of per candidate, so real
128
+ batches (5×50k with a 64k tail) reach the auto band instead of all dropping.
129
+ - Original-event line mapping (R9a): terminal normalization returns folded lines that each
130
+ carry their 1-based original-event line number; `retrieve` reads raw events, so printed
131
+ ranges resolve to the right lines.
132
+ - Document skeleton and universal prose keep (R8/R8b): structured documents keep headings,
133
+ section first/last lines, list starts and table headers; every other non-code text keeps
134
+ head AND tail with an R9 line-range marker (prose was previously head-only truncated).
135
+ - Two-tier search folding (R10, fixes D8): a lossless per-file L1 locator plus a
136
+ water-filled L2 content quota.
137
+ - Non-adjacent frequency folding (R11): separated exact repeats (up to 8.37% of large
138
+ results) fold to first occurrence plus one counted marker.
139
+ - Long-string placeholders (R12): base64/hex/UUID blobs become length summaries with a
140
+ 16-char recognition prefix.
141
+ - Two-stage HTML reduction (R13, fixes D9): `html-slim` then `html-skeleton`, line-aligned
142
+ so original line numbers survive.
143
+ - R9b anchors: contiguous masks cite original line ranges, scatter masks report
144
+ `lines 1-N scanned, K kept`, and every retrieve hint carries a pasteable
145
+ `{"ref":…,"start_line":N,"max_lines":80}`.
146
+ - Document census: omitted-document summaries list section headings instead of a constant
147
+ `0 error, 0 warn, N info` histogram.
148
+
149
+ ## 0.3.1 - 2026-09-18
150
+
151
+ ### Changed
152
+
153
+ - The runtime package is merged into the selector package: one install brings the whole
154
+ stack, the repository root is the install surface (`name`, `main`, `types`, `exports` with
155
+ `./pruner` and `./invariant`, `dependencies`, `dsh`), and the toolchain, scripts and CI were
156
+ swept to the single package. The verified estimator-catalog registration (dual prefix, guarded
157
+ two-channel activation, per-request service resolution, visible lifecycle lines) was replayed
158
+ onto this line with a host-side guard; the settings schema that `ab2175a` had downgraded to
159
+ `z.any()` is restored, so the daily Custom defaults are published again.
160
+ - Adapt to the official DeepSeek Harness `v0.1.5-rc.2` on this branch. All `@deepseek-ai/dsh-*` dev dependencies and the pinned e2e host set move from `0.1.1-rc.2` to `0.1.5-rc.2` (cordis `4.0.2`, schemastery `3.18.2`), including the new split packages (`dsh-session-projection`, `dsh-session-persistence`, `dsh-atomic-write`, `dsh-home-paths`, `dsh-sandbox`, and related) and the `dsh-client-store` client stack.
161
+ - Surface replace operations now use the v3 `startSeq`/`endSeq` shape with branded `SessionSeq` values; `compaction/prune` manifests keep the durable `start`/`end` fields. Events are resolved from surface nodes by seq lookup instead of array indexing.
162
+ - The client bundle no longer imports the removed `@deepseek-ai/dsh-client-runtime`: settings types now come from `@deepseek-ai/dsh-client-ui-settings` and the session hooks merge from `@deepseek-ai/dsh-client-ui-session`. `engines.dsh >=0.1.5-alpha.1 <0.2.0-0` is declared in both package manifests and `dsh.plugin.json`.
163
+ - Harness 0.1.5 no longer exposes the session `agentPreset` to the browser, so the client can no longer detect Minimal-only sessions; the selector stays selectable and the old unavailable banner is unreachable.
164
+ - Test batteries updated for 0.1.5 semantics: cordis plugin starts require `.await()`, the Token Meter requires a mounted `SessionProjectionRegistry`, assistant events carry `stream: []`, and settings namespaces are plain strings.
165
+
166
+ ### Fixed
167
+
168
+ - The estimator card no longer demands an API key on the Harness host channel. Selecting the host channel shows the live provider/model dropdowns, names the route that would actually run (explicit override, else the session default), and renders neither a key field nor a second manual model input: the base URL, the model text field, and the write-only key belong to the direct endpoint channel alone.
169
+ - `presetOptions` writes are path-addressed. Writing the whole section replaced it, so touching any second estimator field (a provider, a model, an endpoint) deleted `estimatorMode` and every sibling override — silently switching the estimator back off while the panel still reported a successful save. Each field now writes only itself, `undefined` clears exactly the field it names, and the confirmation read validates the same field set instead of the mode alone.
170
+ - New coverage: `packages/selector/tests/preset-options-write.client.spec.ts` (path-scoped writes, sibling preservation, explicit clears, no-op patches, uncommitted-write reporting) and `packages/selector/tests/estimator-channel.client.spec.tsx` (per-channel fields, catalog dropdowns, manual fallback).
171
+
172
+
173
+ ### Added
174
+
175
+ - New `tokenpilot-inspired` profile: a TokenPilot-paper-inspired capability matrix layered on the Balanced thresholds, selected explicitly from the settings UI; every pre-existing profile keeps a byte-identical resolved policy (enforced by a captured-baseline golden test).
176
+ - Byte-identical repeated tool-result dedup: an oversized repeat is replaced with a pointer to the first occurrence's append-only original event (`dedupe-pointer`), with a per-session SHA-256 index (2,048-entry insertion-order eviction, hash+seq metadata only).
177
+ - No-net-savings guard: replacements whose text is not smaller than the original are rejected even when the exact tokenizer reports a token saving.
178
+ - Recovery exemption: recovery-tool output is permanently exempt from every reduction pass via a unified per-session exemption set, preventing compress-restore oscillation.
179
+ - Auto Compact summary locator: after `compaction/end`, the landed summary checkpoint gains an Exact Sources block (shadowed seq range, spill files, touched files) so summarized-away details stay recoverable; skipped when it would locate nothing concrete.
180
+ - Read-state semantics: a historical read whose file was later mutated is `superseded` and takes the small whole-result placeholder; optional error/warn/info clustering of omitted lines is appended to historical placeholders.
181
+ - Optional residual-utility estimator (three channels: off / Harness host model / direct OpenAI-compatible endpoint) with per-session exponential backoff, strict timeout, advisory-only verdicts consumed by the next pressure pass, and numeric-only `estimator-outcome` audits. The estimator card appears only while the new profile is selected; the API key is write-only in settings and never enters the frozen policy, audits, or logs.
182
+ - New audit records: `summary-locator` and `estimator-outcome`; the rewrite record covers dedup via the `dedupe-pointer` reducer. Audit field allowlists are unchanged.
183
+ - Simplified Chinese and English copy for the new profile and estimator card; unit and golden coverage under `packages/runtime/tests/tokenpilot/`.
184
+
185
+ - Orthogonal code-skeleton compression gate (`codeSkeleton.enabled`, default off): the first exposure of an oversized fresh source-code tool result can keep an imports-and-declarations skeleton with bodies elided and error lines preserved, falling back to the original head pruning. The gate is independent of every profile and gated on exact tokenizer measurement.
186
+ - Settings-UI toggle for the gate in the selector settings section, with Simplified Chinese and English copy.
187
+ - Browser/runtime decode parity for the new section, confirm-on-write contract tests for `saveCodeSkeleton`, and a full-document parity matrix extension.
188
+
189
+ ### Changed
190
+
191
+ - Added an ESLint flat-config baseline (`pnpm lint`, enforced in CI) and a `pnpm test:watch` TDD loop; removed dead imports and hardened two error paths surfaced by the lint baseline.
192
+ - This repository is now maintained as an improved fork of `WilliamShi666/dsh-context-compression-selector`; documentation ships in English, Simplified Chinese, Japanese, and Korean.
193
+
194
+ ## 0.1.0 - 2026-09-03
195
+
196
+ ### Added
197
+
198
+ - Stable release of DeepSeek V4 Flash Vision tokenizer integration for `deepseek-v4-flash-vision-exp`, including exact text counting and bounded image-token estimates.
199
+ - User-configurable model-driven Auto Compact threshold in the selector settings section.
200
+ - Auto Compact threshold linkage for each standard profile's History / micro-compact watermarks and related compression parameters.
201
+
202
+ ### Changed
203
+
204
+ - The threshold editor now uses one direct numeric input; the slider and fixed quick-value buttons were removed.
205
+ - Runtime session-event access supports both the established Harness `events` accessor and the newer `snapshotEvents()` public API.
206
+
207
+ ## 0.1.0-beta.4 - 2026-09-02
208
+
209
+ ### Fixed
210
+
211
+ - Support the official DeepSeek Harness `dsh-v0.1.2-alpha.5` public API while retaining compatibility with the existing `0.1.1-rc.2` peer range. The plugin now owns the two small immutable-value helpers that the newer Harness no longer exports, and uses the same public `context-compression` namespace literal accepted by both Settings implementations. No Harness core code is modified.
212
+
213
+ ## 0.1.0-beta.3 - 2026-09-01
214
+
215
+ ### Scope
216
+
217
+ This is a staged release. Exact **text-class** token counting for `deepseek-v4-flash-vision-exp`, best-effort bounded **vision-class image** estimates, and the Auto Compact threshold/UI/audit work are delivered. Exact image measurement remains **BLOCKED upstream**: the current measurement seam exposes neither the adapter's projected request-image dimensions nor the absolute serialized position, so estimates cannot be promoted to `exact-tokenizer`.
218
+
219
+ ### Added
220
+
221
+ - DeepSeek V4 Flash Vision support for `deepseek-v4-flash-vision-exp` via a separately bundled official tokenizer pinned at `deepseek-ai/DeepSeek-V4-Flash-Vision-Exp` revision `6821d6ad3681a4b137b066b76094fa82ebd0a380`. Text, reasoning, tool-call arguments, and pure-text tool results are counted exactly; image-bearing tool-result candidates stay fail-open.
222
+ - Vision image-token arithmetic ported line-by-line from the official `inference/image_processor.py` (patch size 14, downsample 3, 384-token cap, min pixels 147456, 8:1 aspect clamp, and position-dependent alignment padding), validated against golden fixtures generated by executing the official Python implementation. Valid intrinsic dimensions now produce `tokenizer-estimate` at the midpoint of all four alignment residues with a 384-token per-image upper bound; malformed or unevaluable dimensions use a documented 256-token fallback. Mixed text/image surfaces aggregate exact text and estimated images without promoting them to exact.
223
+ - `autoCompact.thresholdPercent` setting (default 80, integer 50–90, step 1) with one shared validation contract across the settings UI, the persisted schema, and the runtime resolver. The editor lives inside the context-compression selector settings section.
224
+ - Standard-profile History linkage to the Auto Compact watermark: `A = floor(C × a)` rescales the History trigger, minimum reclaim, and recent-token tail; `D = floor(A × 0.875)` replaces the fixed 0.7 capacity-pressure ratio as the micro-compact last-chance gate; one batch must justify its cache break by pulling the complete request back below the deadline. Defaults at 80% reproduce the previous numbers exactly.
225
+ - The preset overlay writes the saved threshold into the generated `compaction-basic` composition as `thresholdRatio` (with `retainRatio` pinned at 0.16) and, from the same read, into the plugin runtime's deployment config as `autoCompactThresholdPercent`, so one standing generation never runs Auto Compact and micro compact on two different thresholds. Any generation-identity change — threshold, source, or module paths, including equal-length ones — produces a new standing composition generation. Deterministic identity-derived stamps use an 8-hex whole-second window; equal-prefix identities can collide in that first window on a coarse filesystem, so the overlay observes the staging file's real `mtimeMs+size` key and escalates to later hash windows before the atomic rename. Content, permissions, and the final unique stamp are complete before publication; already-running sessions keep their frozen policy.
226
+ - `policy-resolved` audits now record the Auto Compact coordination facts (threshold percent, `A`, `D`, parameter source — including `deployment-override`/`mixed` when deployment config replaces linked History watermarks), the routed provider/model, and the bundled tokenizer identity.
227
+ - Persisted settings reject present-but-invalid sections (`profile: null`, `custom: null`, own-property `undefined`) before any schema default can absorb them; a malformed stored document freezes the session losslessly (`profile: off`, audited as `settingsInvalidFallback: lossless-off`) instead of silently enabling the lossy Balanced default. The browser decoder applies the same rule and canonicalizes legacy Custom v1/v2 documents to the same v3 document the runtime resolver produces.
228
+ - The History planner returns a discriminated outcome, and `component-evaluation` audits distinguish the full skip taxonomy: `below-profile-trigger`, `below-micro-deadline`, `exact-tokenizer-unavailable`, `no-safe-candidates` (only recovery-tool output or already-cleared results), `protected-working-set` (everything inside the protected tail), `insufficient-reclaim` and `cannot-reach-deadline-target` (with the reached/required token numbers), `adaptive-cost-rejected`, and `recovery-tool-unavailable`.
229
+
230
+ ### Known limitations
231
+
232
+ - Images never claim exact counts. The official expansion depends on the absolute prompt position (system prompt, chat-template framing, adapter image handles) and on the adapter's final request-image projection (including per-route `imagePixelBudget`/`imageDetail` overrides and byte-cap reprojection), none of which is exposed through the current measurement seam. Intrinsic/default estimates may therefore differ materially from provider accounting. Upstream capability requests remain projected request-image dimensions and the absolute serialized position exposed to token-meter extensions.
233
+ - History skips the whole batch whenever any tool-result candidate lacks an exact count, including image-bearing candidates, even though sibling text candidates are individually exact.
234
+ - Custom remains manual token mode; its History parameters do not follow the Auto Compact watermark.
235
+ - A vision token breakdown UI is not shipped; image estimates and the intrinsic alignment diagnostic are available on the measured token view, while lossy rewrite proofs still require exact counts.
236
+
237
+ ### Deferred
238
+
239
+ - Audit `modality` field and the tokenizer artifact SHA-256 inside audit records (the audits already carry the routed provider/model and tokenizer identity).
240
+ - Publishing the (now complete) runtime skip-reason taxonomy as a user-facing documentation table.
241
+ - Custom-profile display of the A/D watermarks and an above-D warning; Custom remains fully manual.
242
+ - Vision token breakdown UI and promotion of image estimates to exact measurement.
243
+
244
+ ## 0.1.0-beta.2 - 2026-08-28
245
+
246
+ ### Fixed
247
+
248
+ - Resolve the official DeepSeek V4 Flash tokenizer route so Fresh and Aggregate can evaluate tool results for the supported V4 models.
249
+ - Run Cache Strict History at the real request boundary once its configured capacity-pressure condition is met; trigger the capacity condition at 70% routed-context utilization.
250
+ - Disable Harness-native head/middle/tail tool-result pruning whenever a selector profile is active, leaving the selector as the sole tool-result compactor.
251
+
252
+ ### Changed
253
+
254
+ - Protect the newest 10 agent tool calls and a 64,000-token tool-result tail window before History/microcompact rewrites older results.
255
+
256
+ ## 0.1.0-beta.1 - 2026-08-27
257
+
258
+ ### Added
259
+
260
+ - One-install DeepSeek Harness Product Bundle backed by a separate exact-version runtime package.
261
+ - Web profile selector with preset-stable settings and an explicit built-in Minimal exception.
262
+ - Fresh, Aggregate, routine/capacity-aware History, Native tool-result pruning, and default-off Custom TailTrim.
263
+ - Standard-event TailTrim protocol using `compaction/prune` plus recoverable `user/message` replacement.
264
+ - Plugin-owned `context_compression_retrieve` recovery tool.
265
+ - Structured, content-free policy, evaluation, rewrite, failure, and Native auto-compact audit records.
266
+ - Pinned official DeepSeek V4 tokenizer assets with runtime SHA-256 validation and upstream license.
267
+ - Public-API component E2E, preset/Minimal, and parent/fork/spawn cache-prefix regression tests.
268
+
269
+ ### Compatibility
270
+
271
+ - Verified against DeepSeek Harness `dsh-v0.1.1-rc.2` public packages.
272
+ - Exact tokenizer mapping is currently limited to `deepseek-v4-flash` and `deepseek-v4-pro`.
273
+
274
+ ### Known limitations
275
+
276
+ - Adaptive ordinary History fails closed when public request-level route/cache evidence is incomplete; capacity pressure remains a separate safety override.
277
+ - Cache-prefix tests prove native fork inheritance and identical serialized prefixes, not a provider-specific cache allocation or a guaranteed DeepSeek cache hit.
278
+ - Settings snapshots and first-exposure decisions are process-local to the mounted runtime.