dsh-jev-tools-continued 0.0.0-stage → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.en.md +449 -0
- package/CHANGELOG.i18n.yaml +13 -0
- package/CHANGELOG.md +357 -0
- package/LICENSE +21 -0
- package/README.en.md +210 -0
- package/README.i18n.yaml +13 -0
- package/README.md +209 -2
- package/client/client.js +755 -0
- package/cordis.patch.yml +30 -0
- package/docs/banner.png +0 -0
- package/docs/dev-workflow.md +414 -0
- package/docs/s0-trigger-rate.md +168 -0
- package/icon.svg +16 -0
- package/lib/backends/jev.d.ts +69 -0
- package/lib/backends/jev.d.ts.map +1 -0
- package/lib/backends/jev.js +151 -0
- package/lib/backends/jev.js.map +1 -0
- package/lib/backends/types.d.ts +104 -0
- package/lib/backends/types.d.ts.map +1 -0
- package/lib/backends/types.js +12 -0
- package/lib/backends/types.js.map +1 -0
- package/lib/budget.d.ts +60 -0
- package/lib/budget.d.ts.map +1 -0
- package/lib/budget.js +70 -0
- package/lib/budget.js.map +1 -0
- package/lib/calibrate.d.ts +162 -0
- package/lib/calibrate.d.ts.map +1 -0
- package/lib/calibrate.js +269 -0
- package/lib/calibrate.js.map +1 -0
- package/lib/config.d.ts +261 -0
- package/lib/config.d.ts.map +1 -0
- package/lib/config.js +194 -0
- package/lib/config.js.map +1 -0
- package/lib/context-cache.d.ts +59 -0
- package/lib/context-cache.d.ts.map +1 -0
- package/lib/context-cache.js +116 -0
- package/lib/context-cache.js.map +1 -0
- package/lib/credentials.d.ts +32 -0
- package/lib/credentials.d.ts.map +1 -0
- package/lib/credentials.js +44 -0
- package/lib/credentials.js.map +1 -0
- package/lib/degrade.d.ts +36 -0
- package/lib/degrade.d.ts.map +1 -0
- package/lib/degrade.js +41 -0
- package/lib/degrade.js.map +1 -0
- package/lib/features/prune.d.ts +116 -0
- package/lib/features/prune.d.ts.map +1 -0
- package/lib/features/prune.js +574 -0
- package/lib/features/prune.js.map +1 -0
- package/lib/features/screen.d.ts +112 -0
- package/lib/features/screen.d.ts.map +1 -0
- package/lib/features/screen.js +130 -0
- package/lib/features/screen.js.map +1 -0
- package/lib/features/skill-suggest.d.ts +68 -0
- package/lib/features/skill-suggest.d.ts.map +1 -0
- package/lib/features/skill-suggest.js +283 -0
- package/lib/features/skill-suggest.js.map +1 -0
- package/lib/host.d.ts +280 -0
- package/lib/host.d.ts.map +1 -0
- package/lib/host.js +15 -0
- package/lib/host.js.map +1 -0
- package/lib/http.d.ts +71 -0
- package/lib/http.d.ts.map +1 -0
- package/lib/http.js +162 -0
- package/lib/http.js.map +1 -0
- package/lib/i18n.d.ts +372 -0
- package/lib/i18n.d.ts.map +1 -0
- package/lib/i18n.js +254 -0
- package/lib/i18n.js.map +1 -0
- package/lib/index.d.ts +44 -0
- package/lib/index.d.ts.map +1 -0
- package/lib/index.js +268 -0
- package/lib/index.js.map +1 -0
- package/lib/keys.d.ts +90 -0
- package/lib/keys.d.ts.map +1 -0
- package/lib/keys.js +131 -0
- package/lib/keys.js.map +1 -0
- package/lib/ledger-domain.d.ts +169 -0
- package/lib/ledger-domain.d.ts.map +1 -0
- package/lib/ledger-domain.js +338 -0
- package/lib/ledger-domain.js.map +1 -0
- package/lib/ledger.d.ts +243 -0
- package/lib/ledger.d.ts.map +1 -0
- package/lib/ledger.js +239 -0
- package/lib/ledger.js.map +1 -0
- package/lib/memo.d.ts +32 -0
- package/lib/memo.d.ts.map +1 -0
- package/lib/memo.js +62 -0
- package/lib/memo.js.map +1 -0
- package/lib/notify.d.ts +78 -0
- package/lib/notify.d.ts.map +1 -0
- package/lib/notify.js +122 -0
- package/lib/notify.js.map +1 -0
- package/lib/proxy.d.ts +40 -0
- package/lib/proxy.d.ts.map +1 -0
- package/lib/proxy.js +136 -0
- package/lib/proxy.js.map +1 -0
- package/lib/request.d.ts +99 -0
- package/lib/request.d.ts.map +1 -0
- package/lib/request.js +168 -0
- package/lib/request.js.map +1 -0
- package/lib/source.d.ts +46 -0
- package/lib/source.d.ts.map +1 -0
- package/lib/source.js +35 -0
- package/lib/source.js.map +1 -0
- package/lib/status.d.ts +42 -0
- package/lib/status.d.ts.map +1 -0
- package/lib/status.js +214 -0
- package/lib/status.js.map +1 -0
- package/lib/tokens.d.ts +26 -0
- package/lib/tokens.d.ts.map +1 -0
- package/lib/tokens.js +57 -0
- package/lib/tokens.js.map +1 -0
- package/lib/tools/jev-ask.d.ts +44 -0
- package/lib/tools/jev-ask.d.ts.map +1 -0
- package/lib/tools/jev-ask.js +208 -0
- package/lib/tools/jev-ask.js.map +1 -0
- package/lib/tools/jev-gate.d.ts +79 -0
- package/lib/tools/jev-gate.d.ts.map +1 -0
- package/lib/tools/jev-gate.js +412 -0
- package/lib/tools/jev-gate.js.map +1 -0
- package/package.json +169 -4
package/CHANGELOG.en.md
ADDED
|
@@ -0,0 +1,449 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
English | [中文](CHANGELOG.md)
|
|
4
|
+
|
|
5
|
+
Notable changes to `dsh-jev-tools`, in the [Keep a Changelog](https://keepachangelog.com/en/1.1.0/)
|
|
6
|
+
format, with versions following [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
|
+
This project is pre-1.0: a minor version may contain a breaking change, and the
|
|
8
|
+
`Removed`/`Changed` sections below will say so when it does.
|
|
9
|
+
|
|
10
|
+
## Version status
|
|
11
|
+
|
|
12
|
+
| Version | Date | State | Summary |
|
|
13
|
+
|---|---|---|---|
|
|
14
|
+
| `0.2.0` | 2026-11-19 | **unreleased** | The network exit is switchable (`proxy`), and several keys can be configured with a switch once one is spent. |
|
|
15
|
+
| `0.1.13` | 2026-09-30 | **published** | The plugin icon: a checklist that becomes a decision path. |
|
|
16
|
+
| `0.1.12` | 2026-09-30 | **published** | The settings card finally reads and writes its own settings: mount row id, `.volatile()` markers, the `configForms` seam, and idempotent resolution, fixed together. |
|
|
17
|
+
| `0.1.11` | 2026-09-30 | **published** | The judgment endpoint and model are finally editable in the configuration card: both settings have existed since 0.1.6, and the card had never drawn them. |
|
|
18
|
+
| `0.1.10` | 2026-09-25 | **published** | The ledger records baseline coverage; a net gain is no longer reported without a measured baseline. |
|
|
19
|
+
| `0.1.9` | 2026-09-25 | **published** | An injected notice used to fail the whole turn; the source kind now names its producer, as format v4 requires. |
|
|
20
|
+
| `0.1.8` | 2026-09-22 | **published** | Skill suggestion had **never fired**, now fixed; the two tools' quota and refusal reasons are no longer dead or silent. |
|
|
21
|
+
| `0.1.7` | 2026-09-22 | **published** | Structural failures are no longer silent; cost is visible; calibration gains AUC and a threshold sweep; CI and tag-driven releases. |
|
|
22
|
+
| `0.1.6` | 2026-09-22 | **published** | The judgment endpoint is configurable (`baseUrl`); `/jev-status` reports it. |
|
|
23
|
+
| `0.1.5` | 2026-09-21 | **published** | Fixes the Workshop manifest's adapter field; the capabilities are the same as `0.1.4`. |
|
|
24
|
+
| `0.1.4` | 2026-09-21 | **published** | Adds the OMDSH Workshop manifest; the capabilities are the same as `0.1.3`. |
|
|
25
|
+
| `0.1.3` | 2026-09-21 | **published** | README rewrite; the capabilities are the same as `0.1.2`. |
|
|
26
|
+
| `0.1.2` | 2026-09-21 | **published** | Source installs work; the capabilities are the same as `0.1.1`. |
|
|
27
|
+
| `0.1.1` | 2026-09-21 | **published** | Documentation corrections; the capabilities are the same as `0.1.0`. |
|
|
28
|
+
| `0.1.0` | 2026-09-20 | **published** | The first release, containing everything described below. |
|
|
29
|
+
|
|
30
|
+
Published on npm: `npm i dsh-jev-tools`. It can also be installed from the repository checkout.
|
|
31
|
+
|
|
32
|
+
## [0.2.0] — 2026-11-19
|
|
33
|
+
|
|
34
|
+
### Added
|
|
35
|
+
|
|
36
|
+
- **A network-exit setting (`proxy`).** Empty connects directly; `system` uses the operating
|
|
37
|
+
system's proxy (on Windows the `ProxyEnable` / `ProxyServer` values under
|
|
38
|
+
`HKCU\Software\Microsoft\Windows\CurrentVersion\Internet Settings`, elsewhere `HTTPS_PROXY`);
|
|
39
|
+
an address such as `http://127.0.0.1:7897` is used as given. Underneath it is an undici
|
|
40
|
+
`ProxyAgent` passed to `fetch` as its `dispatcher` — Node's `fetch` neither reads the WinINET
|
|
41
|
+
registry nor inherits a proxy configured outside the process. **Why it is needed:** Jev decides
|
|
42
|
+
availability by exit IP, so a blocked region gets a `403 RegionError` on every direct call, and
|
|
43
|
+
because this plugin is fail-open that looks like nothing happened at all (`/jev-status` only
|
|
44
|
+
shows a run of `skip.server`).
|
|
45
|
+
- **Several keys (`apiKeyEnv` may list multiple variable names, comma-separated).** A session stays
|
|
46
|
+
on one key (the pin in `src/keys.ts`) and moves to the next only when that key answers `401` or
|
|
47
|
+
`429` (quota spent). A `403`, a `5xx` or a network failure is identical on every key, so switching
|
|
48
|
+
there would only multiply the same failure. With more than one key, `/jev-status` adds a
|
|
49
|
+
"key pool: n/m usable in this session" line.
|
|
50
|
+
- **Dependency on `undici`**, only for `ProxyAgent`; when it is missing the module falls back to a
|
|
51
|
+
direct connection and the plugin still mounts.
|
|
52
|
+
- **The settings card draws all seven.** It used to draw six, leaving `proxy` and the multi-key
|
|
53
|
+
reference to the settings page; the reference input, the network-exit input and the per-key
|
|
54
|
+
paired name-plus-key rows now all live on the card instead of a comma-separated string field:
|
|
55
|
+
an add button grows a row, each row renames inline in the same save, and each row's remove
|
|
56
|
+
button shrinks the list (one reference always stays), with `test/client.test.ts` and
|
|
57
|
+
`test/bundle-contract.test.ts` locking reachability.
|
|
58
|
+
|
|
59
|
+
## [0.1.13] — 2026-09-30
|
|
60
|
+
|
|
61
|
+
### Added
|
|
62
|
+
|
|
63
|
+
- **The plugin icon: a checklist that becomes a decision path.** `package.json` gains
|
|
64
|
+
`icon: "icon.svg"` — one crossed-out row, one checked row, and the checked row running on as a
|
|
65
|
+
cyan→violet arrow: the rejected one stops where it is, the accepted one becomes the path. The
|
|
66
|
+
Plugins page shows it on package cards, detail pages and plugin rows: the Host's
|
|
67
|
+
`readPluginMeta()` (`@deepseek-ai/dsh-app-boot`) reads `package.json.icon`, maps the extension to
|
|
68
|
+
a MIME type and hands the page an `image/svg+xml;base64` data URL as `<img src>`, falling back to
|
|
69
|
+
the default pinwheel only when the field is absent or the image will not decode. `icon.svg` is
|
|
70
|
+
listed in `files` as well: `files` is an allowlist, so omitting it ships the field without the
|
|
71
|
+
file and the installed plugin shows no icon. The colours are fixed values, because an `<img>`
|
|
72
|
+
gets no `currentColor`, so one file serves both the light and the dark theme.
|
|
73
|
+
|
|
74
|
+
## [0.1.12] — 2026-09-30
|
|
75
|
+
|
|
76
|
+
### Fixed
|
|
77
|
+
|
|
78
|
+
- **The configuration card could not read its own settings, so all three toggles were disabled and
|
|
79
|
+
blank and both endpoint fields were read-only.** Two causes stacked: (1) DSH publishes a bundle's
|
|
80
|
+
settings namespace under its **mount row id** (`SettingsForms.describe()` reports
|
|
81
|
+
`entry.options.id`, and a write resolves the same way), while the card (the occupant of
|
|
82
|
+
`plugins.bundle.config`) and the plugin page's config seat are keyed by the **package name**; and
|
|
83
|
+
(2) the page publishes only fields marked `.volatile()` — `volatileForm()` returns nothing for a
|
|
84
|
+
schema with no volatile field, so the namespace never appears in `describe()` at all. With the row
|
|
85
|
+
id `jev-tools` and not one `.volatile()` in the schema, the card could read nothing from either
|
|
86
|
+
side, with no error printed anywhere. That is the whole of "the settings page cannot change
|
|
87
|
+
anything" from 0.1.6 to 0.1.11 (issue #3). The mount row id is now the package name and the six
|
|
88
|
+
fields the card reads and writes are marked `.volatile()`; `test/bundle-contract.test.ts` pins the
|
|
89
|
+
package name, the row id, the namespace the card asks for, and the editability of those six fields
|
|
90
|
+
together.
|
|
91
|
+
- **The `settings.installSection` call is gone.** DSH 0.2.0-rc.2's `settings` service has no such
|
|
92
|
+
method, so every mount threw a `TypeError` that the plugin's own catch turned into a warning — a
|
|
93
|
+
warning that reported a failed registration as merely unavailable. The page is generated from the
|
|
94
|
+
entry's exported Config schema anyway, and a write restarts that entry's fiber so `apply`
|
|
95
|
+
re-resolves the config: "a toggle takes effect immediately" never depended on a registration
|
|
96
|
+
callback.
|
|
97
|
+
- **A namespace that is not published is now named.** The card used to draw an empty form that looked
|
|
98
|
+
exactly like "nothing is configured"; it now names the namespace it was looking for, and the file
|
|
99
|
+
that decides it.
|
|
100
|
+
|
|
101
|
+
### Changed
|
|
102
|
+
|
|
103
|
+
- **Configuration is resolved through the standard-schema path, and volatile markers are unwrapped.**
|
|
104
|
+
`.volatile()` wraps a marked field's resolved value in a marker object (the writer uses it to know
|
|
105
|
+
which keys to persist), so the config is resolved through `Config['~standard'].validate(value)` —
|
|
106
|
+
the very path the Loader takes — and then unwrapped recursively, leaving `model` and `baseUrl` as
|
|
107
|
+
the strings they denote. `@deepseek-ai/schemastery` moves to `^3.18.4` (the version that provides
|
|
108
|
+
`.volatile()`, and the one the host itself ships), and `@deepseek-ai/cosmokit` is added for
|
|
109
|
+
`isVolatile`.
|
|
110
|
+
- **The card now reads and writes through the browser's one settings seam.** Both directions go
|
|
111
|
+
through `ctx.configForms.get(namespace)` — the same seam the four official settings pages use:
|
|
112
|
+
reads take the shared mirror's snapshot, writes submit an operation list (`{ op, path, value }`)
|
|
113
|
+
with the revision read before editing. It used to call `remote.settings` directly, and the
|
|
114
|
+
browser's single `settings.describe()` reader is the settings UI itself; a third-party card never
|
|
115
|
+
gets it — so **the host could publish the namespace and the card would still read nothing**. The
|
|
116
|
+
diagnostic now lists the namespaces the host actually serves.
|
|
117
|
+
- **Resolution has to be idempotent.** The Loader hands `apply` the config **it already resolved** —
|
|
118
|
+
volatile markers included — so the input is unwrapped before it is validated. Without that,
|
|
119
|
+
`enabled` was validated as an object and threw, and any throw inside `apply` fails the whole
|
|
120
|
+
fiber: the plugin shows as failed and serves no settings namespace at all, which looks exactly
|
|
121
|
+
like "the card cannot read its own settings". `test/config.test.ts` now pins it: the Loader-shaped
|
|
122
|
+
input must resolve to the same values as the raw one.
|
|
123
|
+
|
|
124
|
+
## [0.1.11] — 2026-09-30
|
|
125
|
+
|
|
126
|
+
### Fixed
|
|
127
|
+
|
|
128
|
+
- **The judgment endpoint is finally editable in the UI.** `baseUrl` and `model` have been host-side
|
|
129
|
+
settings since 0.1.6, and the backend reads their live values on every judgment, but the
|
|
130
|
+
configuration card drew only the key input and three toggles — and the `plugins.bundle.config` seat
|
|
131
|
+
that card occupies is **single-occupant per package name**, so a field the card does not draw is a
|
|
132
|
+
field the page cannot reach. Both settings were therefore declared, wired and documented as
|
|
133
|
+
"editable on the settings page" while in fact being editable only by hand-editing the composition
|
|
134
|
+
row and restarting the host. Evidence: the reporter of issue #3 described exactly that — wanting to
|
|
135
|
+
point Jev at OpenRouter and finding it could only be set at startup. The card now carries a
|
|
136
|
+
"Judgment endpoint" section with the service address and the model, staged and written on an
|
|
137
|
+
explicit save; the address must start with `http(s)://` and be non-empty or the save is blocked
|
|
138
|
+
with a reason; and only genuinely changed fields are written, so looking at the card is not an
|
|
139
|
+
override.
|
|
140
|
+
- **Two lines of copy stopped lying about the destination.** The privacy note hardcoded
|
|
141
|
+
`api.typesafe.ai`, which became wrong the moment the endpoint was made configurable; it now names
|
|
142
|
+
the host actually in force. The key link follows the same value: an OpenRouter endpoint gets
|
|
143
|
+
`openrouter.ai/settings/keys`, everything else keeps the TypeSafe console.
|
|
144
|
+
|
|
145
|
+
## [0.1.10] — 2026-09-25
|
|
146
|
+
|
|
147
|
+
### Added
|
|
148
|
+
|
|
149
|
+
- **Baseline coverage in the ledger.** Every prune judgment now records whether the deterministic
|
|
150
|
+
baseline was really measured: content answers are stored as `baselineKeptTokens`; a `null` answer
|
|
151
|
+
(the payload is inside DSH's own budget) is stored as the **full original token count** — a
|
|
152
|
+
measurement that the baseline removes nothing, not missing data; an absent service or a thrown
|
|
153
|
+
call is stored as `baselineUnavailable: no-service | error`. The totals gain `baselineMeasured`
|
|
154
|
+
and `baselineUnmeasured`. Evidence: on 2026-09-25 two prune judgments were indistinguishable in
|
|
155
|
+
the old ledger — a headless composition measured `baselineKeptTokens: 4441.5`, while a web
|
|
156
|
+
composition recorded `no-service` (the Web surface mounts the deterministic pruner behind an
|
|
157
|
+
agent preset).
|
|
158
|
+
|
|
159
|
+
### Changed
|
|
160
|
+
|
|
161
|
+
- **A net gain is no longer reported without a baseline.** `/jev-status` prints the coverage next
|
|
162
|
+
to the increment, and with `baselineMeasured` at 0 it reports only the removal and says the net
|
|
163
|
+
gain has no data. The old totals computed `baselineKept = entry.baselineKeptTokens ?? original`,
|
|
164
|
+
which read "never measured" as "the baseline removes nothing" — making the net gain equal to
|
|
165
|
+
everything the plugin removed: an assumption, not a result.
|
|
166
|
+
- Two README statements are now accurate: pruning keeps **scattered relevant segments** rather than
|
|
167
|
+
cutting one contiguous middle block, and the net increment is stated with its coverage premise.
|
|
168
|
+
|
|
169
|
+
## [0.1.9] — 2026-09-25
|
|
170
|
+
|
|
171
|
+
### Fixed
|
|
172
|
+
|
|
173
|
+
- **An injected notice failed the whole turn.** Since 0.1.7 the harness writes sessions in format
|
|
174
|
+
v4, which has **no shared `plugin` source kind**: each message states **who** produced it, and the
|
|
175
|
+
retired `{ kind: 'plugin', plugin: … }` wrapper is refused by the writer — it does not drop that
|
|
176
|
+
one message, it ends the turn with `format v4 message requires a producer-owned source kind`
|
|
177
|
+
("this turn failed" in the UI). All four notice lines hit it: `prune`, `screen`, `suggest`, and
|
|
178
|
+
the degrade notice. Evidence: on 2026-09-24 two sessions stopped mid-step at the exact moment a
|
|
179
|
+
notice was injected — `f87f8e58` at its prune judgment (23:55:53, 5376 → 3187 tokens) and
|
|
180
|
+
`5f45b2e2` at its screen judgment (23:56:51) — the log ends on `tool/call` with no `turn/end`;
|
|
181
|
+
reproducing it with the harness 0.1.7-rc.1 `assertV4MessageSources`: `kind: 'plugin'` throws,
|
|
182
|
+
`kind: 'plugin:dsh-jev-tools'` passes.
|
|
183
|
+
|
|
184
|
+
### Changed
|
|
185
|
+
|
|
186
|
+
- **One module owns the notice source.** `src/source.ts` builds it for every capability, and the
|
|
187
|
+
kind it writes — `plugin:dsh-jev-tools` — is the identity the harness's own v3→v4 migration
|
|
188
|
+
records for this plugin's historical notices, so notices written before the upgrade and notices
|
|
189
|
+
written today stay under one kind instead of splitting in two.
|
|
190
|
+
|
|
191
|
+
## [0.1.8] — 2026-09-22
|
|
192
|
+
|
|
193
|
+
### Fixed
|
|
194
|
+
|
|
195
|
+
- **Skill suggestion had never fired.** The only defect here where **a whole capability did nothing and
|
|
196
|
+
nothing outside could tell.** `skills.list()` passed no `scope`, and the registry reads the **global
|
|
197
|
+
layer alone** when `scope` is omitted — while a plugin mounted by an agent preset registers into that
|
|
198
|
+
**preset's layer**. A session holding twenty-odd skills was therefore invisible to the plugin, the
|
|
199
|
+
catalog stayed under the 15 floor, and every turn returned silently. Evidence: before the fix,
|
|
200
|
+
**227 ledger rows across 6 sessions contained not one `suggest` row**, against 218 for `prune` and 9
|
|
201
|
+
for `screen`; after it, the first `judged` row appeared and a suggestion reached the session.
|
|
202
|
+
- **`jev_ask` / `jev_gate` could be called three times per session.** Both called
|
|
203
|
+
`budget.tryConsume(agentId, -1)`: `-1` is a **pseudo-turn that never advances**, and the ceiling passed
|
|
204
|
+
in is `prune.perTurnLimit` — a per-turn value — so `turns.get(-1)` only ever grew. The fourth call
|
|
205
|
+
onwards was refused **permanently**. Measured: three successes, then three `skip: "budget-turn"` rows.
|
|
206
|
+
Explicit calls now take a separate `tryConsumeSession()` bounded only by `sessionCallLimit`: the
|
|
207
|
+
per-turn ceiling exists to bound the latency an **automatic** capability adds, which has no rationale
|
|
208
|
+
for a call that is asking for one answer.
|
|
209
|
+
- **A refusal reached the session as a placeholder.** The renderer reads only `report`, and `declined()`
|
|
210
|
+
returned `{ problem, message, detail }` without it — so a missing key, an invalid request, an exhausted
|
|
211
|
+
quota and a network failure were indistinguishable in the session: `(no judgment)` (and
|
|
212
|
+
`(no gate verdict)` for the gate). That contradicts what the README promises — the price of fail-open
|
|
213
|
+
is that these failures are visible **only** in the session.
|
|
214
|
+
- **Refusal rows lost the session id.** The failure path hardcoded `agentId: ''` while the success path
|
|
215
|
+
recorded the real one, so half of a tool's ledger could not be attributed.
|
|
216
|
+
- **The refusal text named the wrong ceiling.** It claimed "this session allows at most 200", while the
|
|
217
|
+
check actually performed was the per-turn one. The text now describes the ceiling that stopped it.
|
|
218
|
+
- **An abstention re-judged on every later step of the same turn.** The turn was latched only once a
|
|
219
|
+
notice was actually produced, so an abstention — the normal outcome on a vague turn — latched nothing
|
|
220
|
+
and every subsequent step re-sent the judgment. Measured: **three user turns produced four judgment
|
|
221
|
+
rows**, each spending the per-turn allowance `prune` shares. The latch is now set before the request
|
|
222
|
+
goes out, and the latch hit writes no row: **one `suggest` row per turn** is the invariant.
|
|
223
|
+
|
|
224
|
+
### Added
|
|
225
|
+
|
|
226
|
+
- **Every skill-suggestion decline is now recorded.** Three skip reasons are added: `no-skills`,
|
|
227
|
+
`catalog-unavailable`, `catalog-too-small`. Until now all eight early returns were silent, so "never
|
|
228
|
+
ran" and "ran and declined every time" were **indistinguishable from outside** — the one question the
|
|
229
|
+
ledger exists to answer. Rows are deduplicated by (turn, reason): `agent/pre-step` fires on every step,
|
|
230
|
+
and without that the same line would flush the 1000-row ledger window.
|
|
231
|
+
- **Skill suggestion now judges the latest user message** (falling back to the last three when it is too
|
|
232
|
+
short). That is a **semantic** choice, not an accuracy gain: a suggestion answers "what is being asked
|
|
233
|
+
now", while a three-message window answers "what is this session doing" — pruning's question. A
|
|
234
|
+
controlled rerun (same criteria, only the task text changed) showed the window merely **diluting** the
|
|
235
|
+
distribution: winner 0.510 → 0.470, top-two margin 0.24 → 0.16, **winner unchanged**.
|
|
236
|
+
|
|
237
|
+
### Changed
|
|
238
|
+
|
|
239
|
+
- The README's quota wording is now accurate: `sessionCallLimit` names both tools, and
|
|
240
|
+
`prune.perTurnLimit` says it is **shared by prune, screen and suggest**. The data-boundary table gains
|
|
241
|
+
`jev_ask` and `jev_gate` — both send caller-supplied text off the machine, and the section had omitted
|
|
242
|
+
them.
|
|
243
|
+
- The poster and its render script moved out of `docs/` (to `poster/` at the repository root): the
|
|
244
|
+
`files` allowlist covers all of `docs/`, and 3 MB of marketing artwork has no reason to reach every
|
|
245
|
+
`npm i`.
|
|
246
|
+
|
|
247
|
+
## [0.1.7] — 2026-09-22
|
|
248
|
+
|
|
249
|
+
### Added
|
|
250
|
+
|
|
251
|
+
- **Structural failures are no longer silent.** With no key configured, or a key the endpoint refuses
|
|
252
|
+
(401), the plugin says so **once in the session** — once per session, per reason, de-duplicated
|
|
253
|
+
through the session history. Fail-open is the right rule (a failed judgment must not fail the task),
|
|
254
|
+
and its price is that neither failure is reported anywhere else: a mistyped `baseUrl` or a key issued
|
|
255
|
+
for a different host looks exactly like a plugin doing nothing. The notice reaches the model as well
|
|
256
|
+
as the reader, so a model stops trusting a pruning layer that has gone blind.
|
|
257
|
+
- **Spend is visible.** The ledger gains a cumulative input-token counter (input is billed, output is
|
|
258
|
+
free), and both `/jev-status` and `npm run measure -- --ledger` show it in dollars at `$0.042` per
|
|
259
|
+
million. Until now `estimateCostUsd()` had no caller outside its unit test, and the measurement script
|
|
260
|
+
carried its own second copy of the price.
|
|
261
|
+
- **The calibration report gains AUC and a threshold sweep.** `npm run measure -- samples.jsonl` now
|
|
262
|
+
answers two questions it did not before: ROC AUC (can the model separate the classes at all — a
|
|
263
|
+
different question from whether it is calibrated) and a 0.05–0.95 threshold table with
|
|
264
|
+
precision/recall/F1 and the best operating point. That is the number a gate needs, and it complements
|
|
265
|
+
the existing ECE/Brier.
|
|
266
|
+
- **CI and release automation.** `.github/workflows/ci.yml` (ubuntu + windows × node 24: `npm ci` →
|
|
267
|
+
`npm test` → tarball check) and `.github/workflows/release.yml` (a **tag is the only thing that
|
|
268
|
+
publishes**: the tag is checked against the version in `package.json`, the same gates run, and the
|
|
269
|
+
publish goes through npm trusted publishing with no long-lived token). Together with
|
|
270
|
+
`scripts/check-tarball.mjs`, which asserts the published tarball carries no local state
|
|
271
|
+
(`.npm-cache/`, `trigger-rate.json`, …) and is not missing anything a consumer needs — an allowlist is
|
|
272
|
+
a control only for as long as nobody widens it.
|
|
273
|
+
|
|
274
|
+
### Changed
|
|
275
|
+
|
|
276
|
+
- The judgment price is declared once (`USD_PER_MTOK` in `src/request.ts`); the measurement script no
|
|
277
|
+
longer keeps its own copy.
|
|
278
|
+
|
|
279
|
+
## [0.1.6] — 2026-09-22
|
|
280
|
+
|
|
281
|
+
### Added
|
|
282
|
+
|
|
283
|
+
- **The judgment endpoint is configurable** (`baseUrl`, default `https://api.typesafe.ai`). The change is
|
|
284
|
+
small because it fixes a **declared-but-never-wired** gap: `JevBackendOptions.baseUrl` already existed
|
|
285
|
+
and `createJevBackend` already read its address from it, but `JevSettings` had no such field, so the
|
|
286
|
+
settings page offered no way to set one and the three construction points in `apply()` had nothing to
|
|
287
|
+
pass — the pipeline could only ever reach the default host. A hardcoded endpoint is what blocks
|
|
288
|
+
**self-hosted Jev-compatible servers** and deployments that put **a gateway in front**. The value now
|
|
289
|
+
travels from the settings (or the bundle row's `config:`) into all three backend construction points;
|
|
290
|
+
give it a bare host and the plugin still appends `/v1/systemone`. Like the key, the endpoint is read
|
|
291
|
+
per operation, so a change needs no restart.
|
|
292
|
+
- **`/jev-status` gains a "Judgment endpoint" line.** With the endpoint configurable, where the content
|
|
293
|
+
goes stops being a constant — and a wrong one looks exactly like a plugin doing nothing. That report is
|
|
294
|
+
the one place built to answer that question.
|
|
295
|
+
|
|
296
|
+
### Changed
|
|
297
|
+
|
|
298
|
+
- The default endpoint is now declared in exactly one place (`src/config.ts`) and imported by the
|
|
299
|
+
backend instead of being held twice, so the setting's default and the backend's fallback cannot drift
|
|
300
|
+
into two different hosts.
|
|
301
|
+
|
|
302
|
+
## [0.1.5] — 2026-09-21
|
|
303
|
+
|
|
304
|
+
### Fixed
|
|
305
|
+
|
|
306
|
+
- `dshWorkshop.install.adapter` changed from `harness-cordis` to `third-party`: the OMDSH Workshop protocol mapping pairs `harness-cordis` with the `third-party` adapter, and using the protocol name as the adapter is rejected by its validator.
|
|
307
|
+
|
|
308
|
+
## [0.1.4] — 2026-09-21
|
|
309
|
+
|
|
310
|
+
### Added
|
|
311
|
+
|
|
312
|
+
- `package.json` gains a `dshWorkshop` manifest (`omdsh-workshop-package/v1`) declaring the integration protocol, install mode, lifecycle, structured permissions, compatible releases, and one observable capability, for the intake of [OMDSH Hub](https://github.com/omdsh-dev/dsh-hub-workshop). It is metadata for the catalog and changes no behaviour inside DSH.
|
|
313
|
+
|
|
314
|
+
## [0.1.3] — 2026-09-21
|
|
315
|
+
|
|
316
|
+
### Changed
|
|
317
|
+
|
|
318
|
+
- Rewrote the README into a shorter structure: capabilities, install, configuration, data boundary,
|
|
319
|
+
settings, troubleshooting, known limitations, ledger, development. The long tutorial-style
|
|
320
|
+
introduction to Jev and the roadmap section are gone; no key fact was dropped.
|
|
321
|
+
|
|
322
|
+
### Fixed
|
|
323
|
+
|
|
324
|
+
- The header comment in `cordis.patch.yml` listed only some of the capabilities; it now lists all
|
|
325
|
+
of them.
|
|
326
|
+
|
|
327
|
+
## [0.1.2] — 2026-09-21
|
|
328
|
+
|
|
329
|
+
### Fixed
|
|
330
|
+
|
|
331
|
+
- Installing from the GitHub source now builds automatically: `package.json` gained a `prepare`
|
|
332
|
+
script. Before this a source install succeeded, but `lib/` is a build output and is not in the
|
|
333
|
+
repository, so there was no entry file to load. Installing from npm is unaffected.
|
|
334
|
+
|
|
335
|
+
## [0.1.1] — 2026-09-21
|
|
336
|
+
|
|
337
|
+
### Fixed
|
|
338
|
+
|
|
339
|
+
- The README and CHANGELOG statements about release status now match reality (they previously
|
|
340
|
+
said "not published"), and the README gained npm-version and license badges.
|
|
341
|
+
- The stale "only two capabilities" wording in the README now states the capabilities that
|
|
342
|
+
actually ship.
|
|
343
|
+
|
|
344
|
+
## [0.1.0] — 2026-09-20
|
|
345
|
+
|
|
346
|
+
### Added
|
|
347
|
+
|
|
348
|
+
- **Semantic pruning of oversized tool results** (`tools/post-execute`), for
|
|
349
|
+
`read` / `grep` / `glob` / `web_fetch` / `web_search`. Jev judges each segment
|
|
350
|
+
for relevance to the current request and drops the irrelevant ones, leaving a
|
|
351
|
+
visible notice. It **ranks rather than thresholds**, keeps deterministic
|
|
352
|
+
head/tail floors, and is **pure fail-open**. Measured on the real API:
|
|
353
|
+
`read: 4613 → 2624 tokens`.
|
|
354
|
+
- **Shadow mode** (`prune.shadow`): judge and record as normal, then change
|
|
355
|
+
nothing — each payload reports what pruning *would* have removed. It exists so
|
|
356
|
+
the trade-off can be inspected before it is trusted, and its records are
|
|
357
|
+
excluded from the saved-token totals because nothing was saved.
|
|
358
|
+
- **Injection screening** for fetched content (`web_fetch` / `web_search`):
|
|
359
|
+
judges the probability that a page carries instructions addressed to an AI and
|
|
360
|
+
attaches a warning. **Advisory only** — it never blocks a call and never
|
|
361
|
+
rewrites content. It runs below the pruning floor, because the most dangerous
|
|
362
|
+
page is a short one, and it rides along in the pruning request when there is
|
|
363
|
+
one to join.
|
|
364
|
+
- **Skill suggestion** (`agent/pre-step`): at most one suggestion per turn,
|
|
365
|
+
silent below the confidence floor, advisory in the strict sense — it never
|
|
366
|
+
rejects a step.
|
|
367
|
+
- **`jev_ask`** — a model-facing tool for any typed question (`noul` / `choice` /
|
|
368
|
+
`score`), with validation of every documented vendor limit before the content
|
|
369
|
+
leaves the machine.
|
|
370
|
+
- **`jev_gate`** — a model-facing tool that checks a delivery before it is called
|
|
371
|
+
done: one verdict per completion claim (`verified` / `contradicted` /
|
|
372
|
+
`not_addressed`) against the evidence actually supplied, plus one action for
|
|
373
|
+
the whole delivery (`auto` / `review` / `escalate`). This is the one capability
|
|
374
|
+
that **inverts the plugin's fail-open rule**: every unclear path resolves to
|
|
375
|
+
`escalate`, because failing open for a gate means failing to "approved".
|
|
376
|
+
- **Judgment ledger**, metadata only, with cumulative counters kept separately
|
|
377
|
+
from the bounded record set so eviction cannot shrink them. Records what the
|
|
378
|
+
deterministic baseline would have kept alongside what semantic selection kept,
|
|
379
|
+
which makes the increment a measured number rather than an argument. Persisted
|
|
380
|
+
through `storageDomain` when the profile provides it, in memory when it does
|
|
381
|
+
not.
|
|
382
|
+
- **`/jev-status`** — enabled state, key source, judgment counts, ledger location,
|
|
383
|
+
and the reason for every single skip.
|
|
384
|
+
- **Bilingual output** for everything a person reads: the settings card follows
|
|
385
|
+
the DSH interface language, and session notices, `/jev-status` and tool reports
|
|
386
|
+
follow the conversation.
|
|
387
|
+
- **Measurement tooling**: `scripts/trigger-rate.ts` (trigger rate from local
|
|
388
|
+
session logs; no key, no network) and `scripts/measure.ts` (labelled-set
|
|
389
|
+
calibration, or the ledger's own increment report).
|
|
390
|
+
- **Test coverage at every seam**: the pruning pipeline and every skip guard,
|
|
391
|
+
the retry/backoff matrix, request validation, credentials policy, ledger
|
|
392
|
+
persistence and restart continuity, screening, the gate's decision matrix, and
|
|
393
|
+
the architectural layering (which is enforced by a test rather than by
|
|
394
|
+
convention).
|
|
395
|
+
|
|
396
|
+
### Defaults, and why
|
|
397
|
+
|
|
398
|
+
Every one of these came from a measurement over 72 real sessions and 4,653 real
|
|
399
|
+
tool results, not from a preference. `docs/s0-trigger-rate.md` holds the numbers.
|
|
400
|
+
|
|
401
|
+
| Setting | Default | Basis |
|
|
402
|
+
|---|---|---|
|
|
403
|
+
| `prune.minTokens` | `2000` | Latency is governed by the quota, not the threshold; of the oversized results, this keeps 31.9% of the available saving |
|
|
404
|
+
| `prune.perTurnLimit` | `3` | Uncapped, the worst turn added 8.1 s; capped at 3, 0.9 s |
|
|
405
|
+
| `prune.toolAllowlist` | `read` `grep` `glob` `web_fetch` `web_search` | Covers 76% of oversized results; **`pwsh` is deliberately absent** |
|
|
406
|
+
| `prune.headLines` / `tailLines` | `40` / `40` | Deterministic floors; not negotiable |
|
|
407
|
+
| `prune.keepHigh` | `0.5` | A segment this relevant is kept unconditionally |
|
|
408
|
+
| `prune.minKeepRatio` | `0.2` | Never reduce a result to a shell |
|
|
409
|
+
| `prune.minSaving` | `0.15` | Below this, the distortion is not worth the tokens |
|
|
410
|
+
| `prune.minTaskChars` | `12` | Judging relevance against "carry on" produces noise |
|
|
411
|
+
| `screen.minTokens` | `300` | Below this there is not enough text to carry an instruction |
|
|
412
|
+
| `screen.threshold` | `0.75` | Injection probability at which the warning is attached |
|
|
413
|
+
| `suggest.minCatalogSize` | `15` | Measured catalog: min 27, median 29 |
|
|
414
|
+
| `suggest.minConfidence` | `0.3` | Below this, no suggestion is injected at all |
|
|
415
|
+
| `sessionCallLimit` | `200` | The hard ceiling on both cost and content leaving the machine |
|
|
416
|
+
|
|
417
|
+
### Known limitations
|
|
418
|
+
|
|
419
|
+
- **No accuracy figure exists.** The only quality evidence is 8/8 on an 8-item
|
|
420
|
+
self-authored Chinese three-way set, and **no comparison against the host model
|
|
421
|
+
was ever run**. The project's own plan asked for that comparison and it was not
|
|
422
|
+
done.
|
|
423
|
+
- **Probabilities are not calibrated.** Measured: they saturate at `1.000` on easy
|
|
424
|
+
inputs and run under-confident on hard ones. Rank with them; never read them as
|
|
425
|
+
an accuracy figure, and do not gate on a fixed confidence threshold in the easy
|
|
426
|
+
regime.
|
|
427
|
+
- **Calibration is a ready layer, not a working loop.** `src/calibrate.ts` has no
|
|
428
|
+
runtime caller: fitting a curve needs a probability and a later observable
|
|
429
|
+
outcome per judgment, and `noul` answers carry no confidence field while
|
|
430
|
+
nothing reports whether a pruned segment turned out to be needed.
|
|
431
|
+
- **`/jev-status` and skill suggestion have never been observed in a live
|
|
432
|
+
session.** Both have logic-level tests; neither has been seen working in the
|
|
433
|
+
harness.
|
|
434
|
+
- **The completion gate is a judgment, not proof.** It runs no tests and applies
|
|
435
|
+
no patch, it caps input (truncation forbids `auto`), and its field isolation is
|
|
436
|
+
instruction-level only — a prompt injection inside a diff is mitigated by
|
|
437
|
+
wording, not by a hard boundary.
|
|
438
|
+
- **Two harness processes sharing one storage root** make the cumulative counters
|
|
439
|
+
last-write-wins; the domain facility's single-open guarantee is per process.
|
|
440
|
+
|
|
441
|
+
## Compatibility
|
|
442
|
+
|
|
443
|
+
| Subject | Requirement |
|
|
444
|
+
|---|---|
|
|
445
|
+
| DSH release | `0.1.6-alpha.2`, declared in `package.json` under `dsh.compatibility` |
|
|
446
|
+
| Node.js | `>= 20` (`engines`); developed and tested on 24 |
|
|
447
|
+
| Required DSH services | None. Every service is soft-injected, so a profile missing one still mounts the plugin and degrades visibly |
|
|
448
|
+
| Optional DSH services | `credentials`, `settings`, `tools`, `commands`, `skills`, `storageDomain`, `toolResultPruner` |
|
|
449
|
+
| Package dependencies | `@deepseek-ai/schemastery` only. DSH packages are optional peer dependencies: the published ones lag the deployment badly, so their types are deliberately not imported |
|
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
# Consistency record for a pair of documents. Both files listed below must say the
|
|
2
|
+
# same thing; these are the git blob hashes of each side at the last confirmed-
|
|
3
|
+
# consistent state, so editing one side without re-recording goes red in
|
|
4
|
+
# `test/docs.test.ts`.
|
|
5
|
+
#
|
|
6
|
+
# After editing either side, bring the other along and re-record the pair:
|
|
7
|
+
#
|
|
8
|
+
# git hash-object CHANGELOG.md CHANGELOG.en.md
|
|
9
|
+
#
|
|
10
|
+
# That test recomputes these hashes with `node:crypto` and also checks that the two
|
|
11
|
+
# sides keep the same structure.
|
|
12
|
+
CHANGELOG.md: 087f88e2ac8bb30bddb8208717f9c6ca5b5e019f
|
|
13
|
+
CHANGELOG.en.md: b6442ceb9a3328e81a1f9bd4d8904e26c3a295fc
|