dsh-decision-tools 0.0.0-stage → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +10 -0
- package/LICENSE +21 -0
- package/README.md +234 -2
- package/README.zh.md +235 -0
- package/THIRD_PARTY_NOTICES +16 -0
- package/assets/decision-usage/SKILL.md +71 -0
- package/cordis.patch.yml +49 -0
- package/docs/architecture-current.md +349 -0
- package/docs/threshold-calibration.md +112 -0
- package/docs/use-cases.md +182 -0
- package/lib/browser.js +1014 -0
- package/lib/client.js +352 -0
- package/lib/config-bus.js +133 -0
- package/lib/configs.js +58 -0
- package/lib/decision-entry.js +15 -0
- package/lib/deliver.js +44 -0
- package/lib/density.js +88 -0
- package/lib/escalate.js +328 -0
- package/lib/event-log.js +159 -0
- package/lib/index.js +75 -0
- package/lib/monitor-entry.js +15 -0
- package/lib/monitor-tool.js +648 -0
- package/lib/monitor.js +1232 -0
- package/lib/notify.js +209 -0
- package/lib/ollama.js +199 -0
- package/lib/parameters.js +183 -0
- package/lib/pipe.js +525 -0
- package/lib/plan.js +73 -0
- package/lib/render.js +193 -0
- package/lib/scan.js +151 -0
- package/lib/skill.js +88 -0
- package/lib/source.js +362 -0
- package/lib/tool.js +1331 -0
- package/lib/validate.js +934 -0
- package/lib/wire.js +76 -0
- package/package.json +85 -4
- package/scripts/link-peers.mjs +55 -0
- package/scripts/list-models.mjs +50 -0
- package/scripts/rebuild-node-modules.mjs +109 -0
package/CHANGELOG.md
ADDED
|
@@ -0,0 +1,10 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
## 0.3.0 — 2026-10-19
|
|
4
|
+
|
|
5
|
+
First public release.
|
|
6
|
+
|
|
7
|
+
- `decision`: batch and mixed typed judgements (1–64 items or questions), inline/file/glob/tool material, a shared rubric sent once, model ladder and escalation, coverage/cost/notes.
|
|
8
|
+
- `monitor`: probability-conditioned polling over job/file/glob/command/events, `once`/`continuous` modes, `countMode`, `notifyEvery`, EP, and streaming job output.
|
|
9
|
+
- Four-row plugin layout (core / decision / monitor / skill) with GUI Configure pages and the bundled `decision-usage` skill.
|
|
10
|
+
- Release packaging: repository/homepage/bugs metadata, docs and scripts in the npm tarball, npm and link install instructions, CI workflow.
|
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Dinosaur_MC
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
CHANGED
|
@@ -1,3 +1,235 @@
|
|
|
1
|
-
#
|
|
1
|
+
# dsh-decision-tools
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
**Ask a local model for probabilities, not prose.** Two DSH tools turn bounded judgements into cheap, repeatable calls:
|
|
4
|
+
|
|
5
|
+
| Tool | What it answers | Typical use |
|
|
6
|
+
|---|---|---|
|
|
7
|
+
| `decision` | One criterion over 1–64 items, or up to 64 different questions — in **one** call | Filter a list, classify tickets, route requests, rank candidates, "does this log mean failure?" |
|
|
8
|
+
| `monitor` | Re-samples one source until your criterion holds | Wait for a build to finish, a line to appear, a job to fail |
|
|
9
|
+
|
|
10
|
+
Judgements run on **your own Ollama** (`/v1/systemone`). Nothing goes to a cloud provider, and the material never has to enter your conversation.
|
|
11
|
+
|
|
12
|
+
## Why
|
|
13
|
+
|
|
14
|
+
- **Cheap by design** — the rubric is sent **once**, and one call judges up to 64 items. Looping one call per item is the expensive anti-pattern.
|
|
15
|
+
- **Material stays out of your context** — pass a workspace file, a glob, or another tool's raw output; you only read back the answers.
|
|
16
|
+
- **Honest output** — every answer is a probability plus `confidence`, partial work is reported in `coverage`, and `notes` says what the plugin did.
|
|
17
|
+
|
|
18
|
+
## Requirements
|
|
19
|
+
|
|
20
|
+
- DSH desktop with this plugin enabled (see Install).
|
|
21
|
+
- A local Ollama serving the decision model — default `tev1:0.8b`, override per call with `model`.
|
|
22
|
+
- No API key; the judgement itself makes no outbound call.
|
|
23
|
+
|
|
24
|
+
## Install
|
|
25
|
+
|
|
26
|
+
From npm, once the package is published:
|
|
27
|
+
|
|
28
|
+
```sh
|
|
29
|
+
dsh plugin --profile web add dsh-decision-tools
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
For development from a checkout, link it into the profile:
|
|
33
|
+
|
|
34
|
+
```sh
|
|
35
|
+
dsh plugin --profile web add link:/absolute/path/to/dsh-decision-tools
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
Then confirm the tools are visible:
|
|
39
|
+
|
|
40
|
+
```
|
|
41
|
+
cordis_inspect_query(platform="host", provider="Tool", method="listTools")
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
The companion skill `decision-usage` is registered with the tools and documents every parameter; it loads on demand.
|
|
45
|
+
|
|
46
|
+
### Using a different local model
|
|
47
|
+
|
|
48
|
+
The shipped defaults (`baseUrl` = `http://localhost:11434`, `defaultModel` = `tev1:0.8b`, and the four-rung `ladder`) are a **snapshot**. Any Ollama model that answers the System One endpoint works:
|
|
49
|
+
|
|
50
|
+
- pin one for a call — `model: "<name>"` (disables capacity picking and escalation);
|
|
51
|
+
- change the session default — `defaultModel`;
|
|
52
|
+
- point at another host/port — `baseUrl`;
|
|
53
|
+
- control escalation — `ladder: [{ model, numCtx }]` (see Configuration).
|
|
54
|
+
|
|
55
|
+
To see what is actually installed, and get a paste-ready ladder:
|
|
56
|
+
|
|
57
|
+
```
|
|
58
|
+
node scripts/list-models.mjs # name · context · parameters · size
|
|
59
|
+
node scripts/list-models.mjs --json # { models, ladder }
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
`numCtx` comes from `/api/show`'s declared context length — an upper bound; the runtime `num_ctx` can be smaller.
|
|
63
|
+
|
|
64
|
+
## Quick start
|
|
65
|
+
|
|
66
|
+
```jsonc
|
|
67
|
+
// 1) Filter a list: one rubric, N items
|
|
68
|
+
{ "material": "<the list or its context>", "rubric": "Is this a bug report?",
|
|
69
|
+
"items": ["app crashes on save", "please add dark mode"], "type": "noul" }
|
|
70
|
+
|
|
71
|
+
// 2) Ask different questions about one file
|
|
72
|
+
{ "material": { "file": { "path": "build.log", "startLine": 1, "endLine": 200 } },
|
|
73
|
+
"questions": [
|
|
74
|
+
{ "name": "ok", "type": "noul", "instructions": "Did the build succeed?" },
|
|
75
|
+
{ "name": "why", "type": "choice", "instructions": "What failed?",
|
|
76
|
+
"criteria": { "compile": "compile error", "test": "a test failed", "env": "missing dependency" } }
|
|
77
|
+
] }
|
|
78
|
+
|
|
79
|
+
// 3) Judge another tool's output without reading it
|
|
80
|
+
{ "material": { "tool": { "name": "git_bash", "arguments": { "command": "npm test" } } },
|
|
81
|
+
"rubric": "Did the test suite pass?", "items": ["the test suite output"], "type": "noul" }
|
|
82
|
+
|
|
83
|
+
// 4) Wait for a background job, and get a notice on a hit
|
|
84
|
+
{ "source": { "kind": "job", "id": "<jobId>" },
|
|
85
|
+
"rubric": "Did the job finish successfully?", "background": true, "notify": "inbox" }
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
## `decision`: one call, one mode
|
|
89
|
+
|
|
90
|
+
Give `material` plus **exactly one** mode:
|
|
91
|
+
|
|
92
|
+
| Mode | Shape | Meaning |
|
|
93
|
+
|---|---|---|
|
|
94
|
+
| Batch | `rubric` + `items` + `type` (+ `criteria`) | One shared criterion, judged independently per item |
|
|
95
|
+
| Mixed | `questions` | Up to 64 different questions about the same material |
|
|
96
|
+
|
|
97
|
+
| Parameter | Required | Notes |
|
|
98
|
+
|---|---|---|
|
|
99
|
+
| `material` | yes | Inline string, `{file}`, `{glob}`, or `{tool}` |
|
|
100
|
+
| `rubric` | batch | The one shared criterion; the wire layer repeats it for the model |
|
|
101
|
+
| `items` | batch | 1–64 self-contained items; plain strings are valid |
|
|
102
|
+
| `questions` | mixed | 1–64 `{name, type, instructions, criteria?}` |
|
|
103
|
+
| `type` | batch | `choice` / `noul` / `score` (mixed: per question) |
|
|
104
|
+
| `criteria` | choice/score | Labels once: top-level in batch, per question in mixed |
|
|
105
|
+
| `depth` | no | `quick` (no escalation/batching), `normal` (default), `deep` (may chunk) |
|
|
106
|
+
| `model` | no | Pin a model; disables capacity choice and escalation |
|
|
107
|
+
|
|
108
|
+
**Material forms**: inline text · `{ file: { path, startLine?, endLine? } }` · `{ glob: { pattern, limit? } }` · `{ tool: { name, arguments? } }`. A tool's raw output never enters your context; the full text is spilled and returned as `material.spillPath`. `tool` cannot be combined with `file`/`glob`, and the plugin's own tools are refused.
|
|
109
|
+
|
|
110
|
+
**Read the result**: `results[]` (`id`, `type`, `value`, `probabilities?`, `confidence?`) · `coverage` (`requested`/`answered`/`batches`/`truncated`/`truncatedBy`/`failed`/`aborted`) · `cost` (`model`/`inputTokens`/`bodyBytes`/`wallMs`/`escalated`/`attempts`) · `notes` (first line = model-selection decision).
|
|
111
|
+
|
|
112
|
+
`noul` returns a **boolean** `value`; its P(true) is in `probabilities.true` (0.475 = 47.5% yes). `score` is on the 0..N-1 scale; `choice` returns the chosen label. `confidence` is concentration, **not accuracy** — when `notes` flags low-confidence items, read `probabilities` instead of the label.
|
|
113
|
+
|
|
114
|
+
## `monitor`: wait for an event
|
|
115
|
+
|
|
116
|
+
Samples one source on an interval, asks the same rubric each time, and counts hits. `once` returns when `countThreshold` is reached (or the deadline expires); `continuous` runs to the deadline and returns the accumulated `count`. It **never** kills, retries, or cancels anything.
|
|
117
|
+
|
|
118
|
+
| Source | Keys | Reads |
|
|
119
|
+
|---|---|---|
|
|
120
|
+
| `job` | `id?`, `select?` (`newest`/`oldest`), `jobKind?` | A background job's output (default: newest of this session) |
|
|
121
|
+
| `file` | `path`, `startLine?`, `endLine?` | Workspace-relative file |
|
|
122
|
+
| `glob` | `pattern`, `limit?` | Matched files, concatenated |
|
|
123
|
+
| `command` | `argv` | Re-runs a host command through the plugin's upstream pipe |
|
|
124
|
+
| `events` | `types`, `limit?` | Job-event ring buffer (falls back to the session's current job list) |
|
|
125
|
+
|
|
126
|
+
Parameters: `description` (required; job description; label = source.kind + description) · `source` (required) · `rubric` (required) · `type` (`noul` only) · `criteria` (`noul` optional false/true descriptions) · `hitThreshold` (0.5) · `tailBytes` (4096, max 65536) · `mode` (`once`|`continuous`) · `countMode` (`cumulative`|`consecutive`, default cumulative) · `notifyEvery` (continuous, default 1) · `intervalMs` (5000) · `timeoutMs` (120000) · `countThreshold` (1; once only) · `background` · `notify` · `coalesce` · `maxNotificationsPerMinute` (6).
|
|
127
|
+
|
|
128
|
+
- **Empty samples**: `file`/`glob` with empty content return immediately (`samples: 1`, `emptySource: true`, no model call); `job`/`events`/`command` empty samples count as P=0 and keep waiting for the event.
|
|
129
|
+
- **Details**: the host `finished [...]` shows `hit/P|EP/dur` (once) or `trigger_num/EP/dur` (continuous); the job label is `source.kind + description`. For once, `EP = Σ(hit P)/countThreshold` (missing hits count as 0) and a failed once shows `1-EP`; for continuous, EP is the mean of per-trigger P (a trigger P = Σ(window hit P)/notifyEvery; 0 triggers ⇒ 0). Durations are ms. A boolean-only verdict (`{hit:true}` without P) counts as 0.00, so detail/notice/stream never show `-` for a hit. `description` is folded to one line before it becomes the job label. Permanent source errors (invalid descriptor, outside root, explicit job id missing, file too large) fail fast; transient read errors are retried.
|
|
130
|
+
- **Background**: `background: true` returns a `jobId`; `job_output` streams progress lines (config/hit/trigger/summary) and is consumed on read. `notify: "inbox"` posts a one-line notice **only at a trigger** (once: threshold; continuous: every `notifyEvery` hits) and **requires** background. Job status: abort ⇒ `killed`; all probes errored (no valid verdict) ⇒ `failed`; timeout / valid verdict ⇒ `completed`. Abort interrupts an in-flight `decide` and `sleep`, so `killed` is prompt.
|
|
131
|
+
- **`command` argv**: `argv[0]` picks the interpreter (`pwsh`, `git_bash`/`bash`/`sh`/`zsh`); interpreter flags are stripped; **quote arguments containing spaces yourself**.
|
|
132
|
+
- **`events.types`** is validated against a fixed vocabulary (`job.registered`, `job.stopping`, `job.output`, `job.progress`, `job.settled`, `job.completed`, `job.failed`, `job.killed`, `job.removed`, `job.running`, `job.unknown`); a typo fails as `source_invalid`.
|
|
133
|
+
|
|
134
|
+
## Configuration
|
|
135
|
+
|
|
136
|
+
The plugin is four rows: a core row, a decision row, a monitor row and a skill row. Each row has its own switch in the Plugins page; the core/decision/monitor rows also have their own Configure page. GUI-editable fields are declared .volatile() and apply to the next call without reloading the row. toolName / monitorToolName / ladder / spillDirCap are registration-time or advanced YAML fields: edit them in cordis.patch.yml and reload that row.
|
|
137
|
+
|
|
138
|
+
### Core row — decision-tools-plugin (shared model & runtime)
|
|
139
|
+
|
|
140
|
+
| Key | Default | Meaning |
|
|
141
|
+
|---|---|---|
|
|
142
|
+
| `baseUrl` | http://localhost:11434 | Ollama endpoint; path prefixes are preserved; GUI-editable. |
|
|
143
|
+
| `defaultModel` | tev1:0.8b | Model used when a tool row has no modelOverride; GUI-editable. |
|
|
144
|
+
| `timeoutMs` | 120000 | Per-request budget, including cold model load; GUI-editable. |
|
|
145
|
+
| `keepAlive` | unset | Passed as keep_alive; unset uses the server default; GUI-editable. |
|
|
146
|
+
| `apiKey` | unset | Optional Bearer token for remote / System One-compatible endpoints; GUI-editable. |
|
|
147
|
+
| `modelsPath` | /api/tags | Model discovery path; use `/v1/models` for OpenAI-compatible endpoints; GUI-editable. |
|
|
148
|
+
| `systemOnePath` | v1/systemone | Judgement endpoint path relative to baseUrl; GUI-editable. |
|
|
149
|
+
| `modelOrder` | unset | Comma-separated model priority; `model@numCtx` sets context; unknown models use 8192; GUI-editable. |
|
|
150
|
+
| `escalationEnabled` | true | Allow automatic escalation on context overflow; GUI-editable. |
|
|
151
|
+
| `ladder` | unset (DEFAULT_LADDER) | Ordered {model,numCtx}[]; YAML only; reload the core row after editing. |
|
|
152
|
+
| `toolName` | decision | Model-visible decision tool name; reload the decision row after editing. |
|
|
153
|
+
| `monitorToolName` | monitor | Model-visible monitor tool name; reload the monitor row and the decision row after editing (both snapshot the name). |
|
|
154
|
+
| `spillDirCap` | unset (16 MiB) | Byte cap for the plugin's spill directory; YAML only. |
|
|
155
|
+
|
|
156
|
+
### decision row — decision-tools-decision
|
|
157
|
+
|
|
158
|
+
| Key | Default | Meaning |
|
|
159
|
+
|---|---|---|
|
|
160
|
+
| `modelOverride` | unset (core default) | Blank uses the core defaultModel; GUI-editable. |
|
|
161
|
+
| `defaultDepth` | normal | quick / normal / deep; used when the call omits depth; GUI-editable. |
|
|
162
|
+
| `bodyWarning` | true | false suppresses size/text-shape notes; GUI-editable. |
|
|
163
|
+
| `softBodyBytes` | unset (4096) | Body size above which notes[] carries a size note; 0 warns every call; GUI-editable. |
|
|
164
|
+
| `minConfidence` | unset (0.25) | choice/score confidence below this (or missing) is named in notes[]; 0 disables; GUI-editable. |
|
|
165
|
+
| `maxChoiceOptions` | unset (255) | Upper bound on choice options; GUI-editable. |
|
|
166
|
+
| `escalationMaxAttempts` | 3 | Extra escalation attempts after the first call; `deep` adds 2; blank = 3; GUI-editable. |
|
|
167
|
+
| `scanFirstChunkBytes` | unset (4096) | First scan chunk byte budget; GUI-editable. |
|
|
168
|
+
| `scanMaxChunks` | unset (32) | Maximum chunks per scan; GUI-editable. |
|
|
169
|
+
| `scanMaxTotalBytes` | unset (4 MiB) | Cumulative byte cap per scan; GUI-editable. |
|
|
170
|
+
| `scanDeadlineMs` | unset (180000) | Wall-clock budget per scan; GUI-editable. |
|
|
171
|
+
| `scanAggregate` | none | none / any; any adds a provisional cross-chunk max probability; GUI-editable. |
|
|
172
|
+
|
|
173
|
+
### monitor row — decision-tools-monitor
|
|
174
|
+
|
|
175
|
+
| Key | Default | Meaning |
|
|
176
|
+
|---|---|---|
|
|
177
|
+
| `modelOverride` | unset (core default) | Blank uses the core defaultModel; GUI-editable. |
|
|
178
|
+
| `minProbeIntervalMs` | unset (1000) | Lower bound for the monitor probe interval; GUI-editable. |
|
|
179
|
+
| `notifyCoalesceWindowMs` | unset (5000) | Same-kind notice coalescing window; 0 disables coalescing; GUI-editable. |
|
|
180
|
+
| `maxNotificationsPerMinute` | unset (6) | Per-minute notice cap; GUI-editable. |
|
|
181
|
+
|
|
182
|
+
Override an existing row with a bare id and no insert; fields go under config. config is replaced wholesale, never deep-merged, so restate every field the row needs.
|
|
183
|
+
|
|
184
|
+
- id: decision-tools-plugin
|
|
185
|
+
config:
|
|
186
|
+
baseUrl: 'http://localhost:11434'
|
|
187
|
+
defaultModel: 'tev1:0.8b'
|
|
188
|
+
timeoutMs: 180000
|
|
189
|
+
keepAlive: '30m'
|
|
190
|
+
- id: decision-tools-decision
|
|
191
|
+
config:
|
|
192
|
+
modelOverride: 'tev1:0.8b'
|
|
193
|
+
defaultDepth: normal
|
|
194
|
+
- id: decision-tools-monitor
|
|
195
|
+
config:
|
|
196
|
+
modelOverride: 'nimble:latest'
|
|
197
|
+
|
|
198
|
+
The three Config schemas live in lib/configs.js.
|
|
199
|
+
|
|
200
|
+
## Troubleshooting
|
|
201
|
+
|
|
202
|
+
| Symptom | Likely cause | Fix |
|
|
203
|
+
|---|---|---|
|
|
204
|
+
| `context_overflow` | Material + questions exceed the model's window | Use `depth: "deep"` (chunking), smaller material, or fewer questions |
|
|
205
|
+
| Tool not listed | Plugin not enabled in the profile | Link it into the profile and reload |
|
|
206
|
+
| `ERR_MODULE_NOT_FOUND` after a DSH upgrade | Workspace `node_modules` still points at the old install | Run `scripts/rebuild-node-modules.mjs` inside DSH, then re-run the tests |
|
|
207
|
+
| `monitor` gives `hit:false` with `errors: 0` | Source was empty, or `types` matched nothing | Check `emptySource`; `notes` states the reason |
|
|
208
|
+
| `monitor` gives `errors: N > 0` | Reads or judgements failed | Inspect the error text; failed probes are also rendered |
|
|
209
|
+
| `source.kind=events` fails | Event source unavailable in this composition | Falls back to the session's current job list when possible |
|
|
210
|
+
| `invalid_request` / `mode_invalid` | Malformed arguments | The message lists legal values; `mode_invalid` = mode/count/shape, `invalid_request` = a field |
|
|
211
|
+
|
|
212
|
+
## Limits and safety
|
|
213
|
+
|
|
214
|
+
- These are **probabilities, not proof**. Never treat one as correctness evidence, and never let it be the sole basis for a pass/allow decision — that needs artifact-level evidence.
|
|
215
|
+
- The tool must **not** decide routing, and must **not** be the basis of any **fail-open or fail-closed** policy; those belong to deterministic rules.
|
|
216
|
+
- `tev1` returns only probabilities and no text — use a generative model when you need a human-readable rationale.
|
|
217
|
+
- Raw tool output is spilled to a plugin-owned temp directory (16 MiB cap, `spillDirCap`); the path is returned in the result.
|
|
218
|
+
- Judgements are local; the plugin makes no cloud calls of its own.
|
|
219
|
+
|
|
220
|
+
## Development
|
|
221
|
+
|
|
222
|
+
```
|
|
223
|
+
node --test test/*.test.mjs # full suite
|
|
224
|
+
node --check lib/*.js # syntax gate
|
|
225
|
+
```
|
|
226
|
+
|
|
227
|
+
After editing `lib/`, reload the plugin in DSH before drawing conclusions from real calls: the host runs the last loaded snapshot.
|
|
228
|
+
|
|
229
|
+
## Licence
|
|
230
|
+
|
|
231
|
+
MIT — see [LICENSE](./LICENSE).
|
|
232
|
+
|
|
233
|
+
## No compatibility layer
|
|
234
|
+
|
|
235
|
+
This is a one-time structural upgrade, not a version transition. Old row ids (`decision-tools-tool`) and old config keys (`decisionEnabled` / `monitorEnabled`) are not accepted; use the four rows and three Config tables above. See [docs/architecture-current.md](./docs/architecture-current.md) for the as-is contract and [docs/use-cases.md](./docs/use-cases.md) for worked examples.
|
package/README.zh.md
ADDED
|
@@ -0,0 +1,235 @@
|
|
|
1
|
+
# dsh-decision-tools
|
|
2
|
+
|
|
3
|
+
**向本地模型要概率,而不是要文字。** 两个 DSH 工具,把有界判定变成便宜、可重复的调用:
|
|
4
|
+
|
|
5
|
+
| 工具 | 回答什么 | 典型用法 |
|
|
6
|
+
|---|---|---|
|
|
7
|
+
| `decision` | 一份判据判 1–64 项,或一次问最多 64 个不同问题 | 筛列表、分类工单、路由请求、排序候选、「这段日志是不是失败?」 |
|
|
8
|
+
| `monitor` | 反复采样同一个源,直到判据成立 | 等构建结束、等某行出现、等作业失败 |
|
|
9
|
+
|
|
10
|
+
判定跑在**你自己的 Ollama**(`/v1/systemone`)上:不经过云服务,材料也不必进入对话上下文。
|
|
11
|
+
|
|
12
|
+
## 为什么值得用
|
|
13
|
+
|
|
14
|
+
- **天然省钱**:判据只发**一次**,一次调用判最多 64 项;逐项循环才是昂贵的反模式。
|
|
15
|
+
- **材料不进上下文**:传工作区文件、glob 或另一个工具的原始输出,你只读回答案。
|
|
16
|
+
- **输出诚实**:每个答案是概率 + `confidence`;部分完成写进 `coverage`,插件做了什么写在 `notes`。
|
|
17
|
+
|
|
18
|
+
## 环境要求
|
|
19
|
+
|
|
20
|
+
- 已启用本插件的 DSH 桌面版(见「安装」)。
|
|
21
|
+
- 本地 Ollama 提供 decision 模型 —— 默认 `tev1:0.8b`,可用 `model` 单次覆盖。
|
|
22
|
+
- 无需 API Key;判定本身不出网。
|
|
23
|
+
|
|
24
|
+
## 安装
|
|
25
|
+
|
|
26
|
+
npm 发布后:
|
|
27
|
+
|
|
28
|
+
```sh
|
|
29
|
+
dsh plugin --profile web add dsh-decision-tools
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
开发期从工作区链接:
|
|
33
|
+
|
|
34
|
+
```sh
|
|
35
|
+
dsh plugin --profile web add link:/absolute/path/to/dsh-decision-tools
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
然后确认工具可见:
|
|
39
|
+
|
|
40
|
+
```
|
|
41
|
+
cordis_inspect_query(platform="host", provider="Tool", method="listTools")
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
随包技能 `decision-usage` 与工具一起注册,写全了每个参数,按需加载。
|
|
45
|
+
|
|
46
|
+
### 换用本机其它模型
|
|
47
|
+
|
|
48
|
+
默认值(`baseUrl` = `http://localhost:11434`、`defaultModel` = `tev1:0.8b`、四条 `ladder`)都只是**快照**。任何能应答 System One 端点的 Ollama 模型都能用:
|
|
49
|
+
|
|
50
|
+
- 单次指定 —— `model: "<名称>"`(同时关闭容量选择与升档);
|
|
51
|
+
- 改会话默认 —— `defaultModel`;
|
|
52
|
+
- 换主机/端口 —— `baseUrl`;
|
|
53
|
+
- 调升档阶梯 —— `ladder: [{ model, numCtx }]`(见 Configuration)。
|
|
54
|
+
|
|
55
|
+
想知道本机实际装了哪些、并拿到可直接粘贴的 ladder:
|
|
56
|
+
|
|
57
|
+
```
|
|
58
|
+
node scripts/list-models.mjs # 名称 · 上下文 · 参数量 · 大小
|
|
59
|
+
node scripts/list-models.mjs --json # { models, ladder }
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
`numCtx` 取自 `/api/show` 的**声明**上下文长度(上限),运行时 `num_ctx` 可能更小。
|
|
63
|
+
|
|
64
|
+
## 快速开始
|
|
65
|
+
|
|
66
|
+
```jsonc
|
|
67
|
+
// 1) 筛列表:一份判据判 N 项
|
|
68
|
+
{ "material": "<列表或其上下文>", "rubric": "This is a bug report?",
|
|
69
|
+
"items": ["app crashes on save", "please add dark mode"], "type": "noul" }
|
|
70
|
+
|
|
71
|
+
// 2) 对同一个文件问不同的问题
|
|
72
|
+
{ "material": { "file": { "path": "build.log", "startLine": 1, "endLine": 200 } },
|
|
73
|
+
"questions": [
|
|
74
|
+
{ "name": "ok", "type": "noul", "instructions": "Did the build succeed?" },
|
|
75
|
+
{ "name": "why", "type": "choice", "instructions": "What failed?",
|
|
76
|
+
"criteria": { "compile": "compile error", "test": "a test failed", "env": "missing dependency" } }
|
|
77
|
+
] }
|
|
78
|
+
|
|
79
|
+
// 3) 判另一个工具的输出,而不把它读进上下文
|
|
80
|
+
{ "material": { "tool": { "name": "git_bash", "arguments": { "command": "npm test" } } },
|
|
81
|
+
"rubric": "Did the test suite pass?", "items": ["the test suite output"], "type": "noul" }
|
|
82
|
+
|
|
83
|
+
// 4) 等后台作业结束,并让插件通知你
|
|
84
|
+
{ "source": { "kind": "job", "id": "<jobId>" },
|
|
85
|
+
"rubric": "Did the job finish successfully?", "background": true, "notify": "inbox" }
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
## `decision`:一次调用,一种模式
|
|
89
|
+
|
|
90
|
+
给 `material`,再加**恰好一种**模式:
|
|
91
|
+
|
|
92
|
+
| 模式 | 形状 | 含义 |
|
|
93
|
+
|---|---|---|
|
|
94
|
+
| 批量 | `rubric` + `items` + `type`(+ `criteria`) | 一份共享判据,逐项独立判定 |
|
|
95
|
+
| 混合 | `questions` | 对同一份材料问最多 64 个不同问题 |
|
|
96
|
+
|
|
97
|
+
| 参数 | 必填 | 说明 |
|
|
98
|
+
|---|---|---|
|
|
99
|
+
| `material` | 是 | 内联字符串 / `{file}` / `{glob}` / `{tool}` |
|
|
100
|
+
| `rubric` | 批量 | 唯一共享判据;线格式会替模型重复它 |
|
|
101
|
+
| `items` | 批量 | 1–64 个自包含项;直接传字符串也合法 |
|
|
102
|
+
| `questions` | 混合 | 1–64 个 `{name, type, instructions, criteria?}` |
|
|
103
|
+
| `type` | 批量 | `choice` / `noul` / `score`(混合模式逐题给) |
|
|
104
|
+
| `criteria` | choice/score | 标签集只写一次:批量走顶层,混合走每题 |
|
|
105
|
+
| `depth` | 否 | `quick`(不升档不分批)、`normal`(默认)、`deep`(可切块) |
|
|
106
|
+
| `model` | 否 | 固定模型;关闭容量选择与升档 |
|
|
107
|
+
|
|
108
|
+
**材料四形态**:内联文本 · `{ file: { path, startLine?, endLine? } }` · `{ glob: { pattern, limit? } }` · `{ tool: { name, arguments? } }`。工具输出**不进上下文**,全文落盘、路径回传为 `material.spillPath`;`tool` 不能与 `file`/`glob` 同时给,插件自身工具会被拒。
|
|
109
|
+
|
|
110
|
+
**读结果**:`results[]`(`id`/`type`/`value`/`probabilities?`/`confidence?`)· `coverage`(`requested`/`answered`/`batches`/`truncated`/`truncatedBy`/`failed`/`aborted`)· `cost`(`model`/`inputTokens`/`bodyBytes`/`wallMs`/`escalated`/`attempts`)· `notes`(首行是选档结论)。
|
|
111
|
+
|
|
112
|
+
`noul` 返回**布尔** `value`;P(true) 在 `probabilities.true` 里(0.475 = 47.5% 是)。`score` 落在 0..N-1 尺度;`choice` 返回所选标签。`confidence` 是分布集中度,**不是正确率** —— 当 `notes` 点名低置信项时,改读 `probabilities` 而不是标签。
|
|
113
|
+
|
|
114
|
+
## `monitor`:等一个事件
|
|
115
|
+
|
|
116
|
+
按间隔采样同一个源、每次问同一个判据、累计命中。`once` 达到 `countThreshold`(或 deadline)返回;`continuous` 跑到 deadline/cancel,返回累积 `count`。它**绝不** kill、重试或取消任何东西。
|
|
117
|
+
|
|
118
|
+
| 采样源 | 允许的键 | 读什么 |
|
|
119
|
+
|---|---|---|
|
|
120
|
+
| `job` | `id?`、`select?`(`newest`/`oldest`)、`jobKind?` | 后台作业输出(缺省看本会话最新) |
|
|
121
|
+
| `file` | `path`、`startLine?`、`endLine?` | 工作区相对文件 |
|
|
122
|
+
| `glob` | `pattern`、`limit?` | 命中文件拼接 |
|
|
123
|
+
| `command` | `argv` | 复用插件上游管道重跑宿主命令 |
|
|
124
|
+
| `events` | `types`、`limit?` | 作业事件环形缓冲(无缓冲时退到当前作业列表) |
|
|
125
|
+
|
|
126
|
+
参数:`description`(必填;作业 label = `source.kind` + description)· `source`(必填)· `rubric`(必填)· `type`(仅 `noul`)· `criteria` · `hitThreshold`(0.5)· `tailBytes`(4096,上限 65536)· `mode`(`once`|`continuous`)· `countMode`(`cumulative`|`consecutive`,默认 cumulative)· `notifyEvery`(continuous,默认 1)· `intervalMs`(5000)· `timeoutMs`(120000)· `countThreshold`(1;仅 once)· `background` · `notify` · `coalesce` · `maxNotificationsPerMinute`(6)。
|
|
127
|
+
|
|
128
|
+
- **空样本**:`file`/`glob` 为空时立即返回(`samples: 1`、`emptySource: true`,不调模型);`job`/`events`/`command` 的空样本按 P=0 记一次未命中并继续等待事件。
|
|
129
|
+
- **后台**:`background: true` 返回 `jobId`;`job_output` 流式读过程行(config/hit/trigger/summary,读一次即消耗)。`notify: "inbox"` **只在 trigger** 发一行通知(once 达标 / continuous 每 `notifyEvery` 次命中),且**必须**是后台调用。作业状态:abort ⇒ `killed`;全部探测 errors(无有效 verdict)⇒ `failed`;timeout/有有效 verdict ⇒ `completed`。
|
|
130
|
+
- **detail**:`finished [...]` 对 once 为 `hit/P|EP/dur`,continuous 为 `trigger_num/EP/dur`;EP:once = Σ(hit P)/N(N=countThreshold,不足补 0;0 hit ⇒ 0);continuous = 各 trigger P 均值(trigger P = 窗口 hit P / notifyEvery;0 trigger ⇒ 0)。once `hit=false` 取 1-EP;单位 ms。布尔型 verdict 无 P 时按 0.00(detail/通知/流都不出现 `-`);`description` 折叠为单行;永久源错误(描述符/越界/显式作业 id 不存在/文件超限)当场失败,瞬态读取错误继续重试;abort 会打断 in-flight decide/sleep。作业 label = `source.kind + description`。
|
|
131
|
+
- **`command` 的 argv**:`argv[0]` 选解释器(`pwsh`、`git_bash`/`bash`/`sh`/`zsh`);解释器标志会被剥掉;**带空格的参数要自己加引号**。
|
|
132
|
+
- **`events.types`** 按固定词表校验(`job.registered`、`job.stopping`、`job.output`、`job.progress`、`job.settled`、`job.completed`、`job.failed`、`job.killed`、`job.removed`、`job.running`、`job.unknown`),拼错报 `source_invalid`。
|
|
133
|
+
|
|
134
|
+
## Configuration
|
|
135
|
+
|
|
136
|
+
插件由四行组成:核心行、decision 行、monitor 行与 skill 行。每行在 Plugins 页都有自己的开关;核心/decision/monitor 三行还各有自己的 Configure 页。GUI 可改字段声明为 .volatile(),保存后下一次调用生效,无需重载行。toolName / monitorToolName / ladder / spillDirCap 是注册期或高级 YAML 项:在 cordis.patch.yml 里改,然后重载对应行。
|
|
137
|
+
|
|
138
|
+
### 核心行 — decision-tools-plugin(共享模型与运行时)
|
|
139
|
+
|
|
140
|
+
| Key | Default | Meaning |
|
|
141
|
+
|---|---|---|
|
|
142
|
+
| `baseUrl` | http://localhost:11434 | Ollama 地址;路径前缀会被保留;GUI 可改。 |
|
|
143
|
+
| `defaultModel` | tev1:0.8b | 工具行没有 modelOverride 时使用的模型;GUI 可改。 |
|
|
144
|
+
| `timeoutMs` | 120000 | 单次请求预算,含冷加载;GUI 可改。 |
|
|
145
|
+
| `keepAlive` | unset | 透传为 keep_alive;未设用服务端默认;GUI 可改。 |
|
|
146
|
+
| `apiKey` | 未设 | 可选;作为 Bearer token 发给远程 / System One 兼容接口;GUI 可改。 |
|
|
147
|
+
| `modelsPath` | /api/tags | 模型发现路径;OpenAI 兼容接口可用 `/v1/models`;GUI 可改。 |
|
|
148
|
+
| `systemOnePath` | v1/systemone | 判定接口路径(相对 baseUrl);GUI 可改。 |
|
|
149
|
+
| `modelOrder` | 未设 | 逗号分隔的模型优先级;`model@numCtx` 可指定上下文;未知模型按 8192;GUI 可改。 |
|
|
150
|
+
| `escalationEnabled` | true | 上下文溢出时允许自动升档;GUI 可改。 |
|
|
151
|
+
| `ladder` | unset (DEFAULT_LADDER) | 有序的 {model,numCtx}[];仅 YAML;改后重载核心行。 |
|
|
152
|
+
| `toolName` | decision | decision 工具的模型可见名;改后重载 decision 行。 |
|
|
153
|
+
| `monitorToolName` | monitor | monitor 工具的模型可见名;改后重载 monitor 行与 decision 行(两行都快照该名)。 |
|
|
154
|
+
| `spillDirCap` | unset (16 MiB) | 插件转储目录的字节上限;仅 YAML。 |
|
|
155
|
+
|
|
156
|
+
### decision 行 — decision-tools-decision
|
|
157
|
+
|
|
158
|
+
| Key | Default | Meaning |
|
|
159
|
+
|---|---|---|
|
|
160
|
+
| `modelOverride` | unset (core default) | 留空使用核心行的 defaultModel;GUI 可改。 |
|
|
161
|
+
| `defaultDepth` | normal | quick / normal / deep;调用省略 depth 时使用;GUI 可改。 |
|
|
162
|
+
| `bodyWarning` | true | false 时不再返回体积/文本形状提示;GUI 可改。 |
|
|
163
|
+
| `softBodyBytes` | unset (4096) | 超过该字节数会在 notes[] 带体积提示;0 表示每次都告警;GUI 可改。 |
|
|
164
|
+
| `minConfidence` | unset (0.25) | confidence 低于该值(或缺失)的 choice/score 项会被点名;0 关闭;GUI 可改。 |
|
|
165
|
+
| `maxChoiceOptions` | unset (255) | choice 选项数上限;GUI 可改。 |
|
|
166
|
+
| `escalationMaxAttempts` | 3 | 额外升档次数(不含首次;deep +2;留空 = 3);GUI 可改。 |
|
|
167
|
+
| `scanFirstChunkBytes` | unset (4096) | 首块字节预算;GUI 可改。 |
|
|
168
|
+
| `scanMaxChunks` | unset (32) | 单次扫描的块数上限;GUI 可改。 |
|
|
169
|
+
| `scanMaxTotalBytes` | unset (4 MiB) | 单次扫描的累计字节上限;GUI 可改。 |
|
|
170
|
+
| `scanDeadlineMs` | unset (180000) | 单次扫描的墙钟预算;GUI 可改。 |
|
|
171
|
+
| `scanAggregate` | none | none / any;any 会给跨块答案加一个暂定的最大概率;GUI 可改。 |
|
|
172
|
+
|
|
173
|
+
### monitor 行 — decision-tools-monitor
|
|
174
|
+
|
|
175
|
+
| Key | Default | Meaning |
|
|
176
|
+
|---|---|---|
|
|
177
|
+
| `modelOverride` | unset (core default) | 留空使用核心行的 defaultModel;GUI 可改。 |
|
|
178
|
+
| `minProbeIntervalMs` | unset (1000) | monitor 探测间隔下限;GUI 可改。 |
|
|
179
|
+
| `notifyCoalesceWindowMs` | unset (5000) | 同类通知合并窗口;0 表示不合并;GUI 可改。 |
|
|
180
|
+
| `maxNotificationsPerMinute` | unset (6) | 每分钟通知上限;GUI 可改。 |
|
|
181
|
+
|
|
182
|
+
覆盖既有行要用裸 id、不带 insert,字段写在 config 键下。config 是整块替换、不做深合并,所以覆盖时必须重述该行用到的全部字段。
|
|
183
|
+
|
|
184
|
+
- id: decision-tools-plugin
|
|
185
|
+
config:
|
|
186
|
+
baseUrl: 'http://localhost:11434'
|
|
187
|
+
defaultModel: 'tev1:0.8b'
|
|
188
|
+
timeoutMs: 180000
|
|
189
|
+
keepAlive: '30m'
|
|
190
|
+
- id: decision-tools-decision
|
|
191
|
+
config:
|
|
192
|
+
modelOverride: 'tev1:0.8b'
|
|
193
|
+
defaultDepth: normal
|
|
194
|
+
- id: decision-tools-monitor
|
|
195
|
+
config:
|
|
196
|
+
modelOverride: 'nimble:latest'
|
|
197
|
+
|
|
198
|
+
三条 Config schema 在 lib/configs.js。
|
|
199
|
+
|
|
200
|
+
## 常见问题
|
|
201
|
+
|
|
202
|
+
| 现象 | 可能原因 | 处理 |
|
|
203
|
+
|---|---|---|
|
|
204
|
+
| `context_overflow` | 材料 + 问题超出模型窗口 | 用 `depth: "deep"` 切块、减小材料或减少问题数 |
|
|
205
|
+
| 工具列表里没有 | 插件未在 profile 启用 | 链接进 profile 后重载 |
|
|
206
|
+
| DSH 升级后 `ERR_MODULE_NOT_FOUND` | 工作区 `node_modules` 仍指向旧安装 | 在 DSH 内跑 `scripts/rebuild-node-modules.mjs`,再复跑测试 |
|
|
207
|
+
| `monitor` 返回 `hit:false` 且 `errors:0` | 源为空,或 `types` 没匹配到 | 看 `emptySource`;`notes` 会写原因 |
|
|
208
|
+
| `monitor` 返回 `errors: N > 0` | 读取或判定失败 | 看错误正文,渲染结果里也有 failed probes |
|
|
209
|
+
| `source.kind=events` 报错 | 该组合下事件源不可用 | 能回落时会退到本会话当前作业列表 |
|
|
210
|
+
| `invalid_request` / `mode_invalid` | 参数不合法 | 报错列出合法值;`mode_invalid` = 模式/数量/形态,`invalid_request` = 字段 |
|
|
211
|
+
|
|
212
|
+
## 边界与安全
|
|
213
|
+
|
|
214
|
+
- 这些是**概率,不是证据**:不得当作正确性证据,也不得单独作为放行/拒绝的依据(那需要产物级证据)。
|
|
215
|
+
- 工具**不得**决定路由,也**不得**作为任何 **fail-open / fail-closed** 策略的依据 —— 那属于确定性规则。
|
|
216
|
+
- `tev1` 只返回概率、不返回文本;需要可读理由时换生成类模型。
|
|
217
|
+
- 工具原始输出落到插件自建的临时目录(总上限 16 MiB,`spillDirCap`),路径随结果回传。
|
|
218
|
+
- 判定全程本地,插件自身不发起云调用。
|
|
219
|
+
|
|
220
|
+
## 开发
|
|
221
|
+
|
|
222
|
+
```
|
|
223
|
+
node --test test/*.test.mjs # 全量套件
|
|
224
|
+
node --check lib/*.js # 语法门
|
|
225
|
+
```
|
|
226
|
+
|
|
227
|
+
改过 `lib/` 后,先在 DSH 里重载插件再下结论:宿主跑的是最后一次加载的快照。
|
|
228
|
+
|
|
229
|
+
## Licence
|
|
230
|
+
|
|
231
|
+
MIT —— 见 [LICENSE](./LICENSE)。
|
|
232
|
+
|
|
233
|
+
## 无兼容层
|
|
234
|
+
|
|
235
|
+
这是一次性结构升级,不是版本过渡。旧行 id(`decision-tools-tool`)与旧配置键(`decisionEnabled` / `monitorEnabled`)不再被接受;使用上面的四行与三张 Config 表。契约见 [docs/architecture-current.md](./docs/architecture-current.md),用例见 [docs/use-cases.md](./docs/use-cases.md)。
|
|
@@ -0,0 +1,16 @@
|
|
|
1
|
+
dsh-decision-tools — third-party notices
|
|
2
|
+
========================================
|
|
3
|
+
|
|
4
|
+
Runtime peer dependencies (provided by the DSH host, not bundled here):
|
|
5
|
+
|
|
6
|
+
@deepseek-ai/cordis — MIT
|
|
7
|
+
@deepseek-ai/dsh-skill — MIT
|
|
8
|
+
@deepseek-ai/dsh-tools — MIT
|
|
9
|
+
@deepseek-ai/schemastery — MIT
|
|
10
|
+
|
|
11
|
+
No third-party source code is vendored into this package. The plugin talks to a
|
|
12
|
+
locally running Ollama server over its HTTP API (POST /v1/systemone); no Ollama
|
|
13
|
+
code is included or required at build time.
|
|
14
|
+
|
|
15
|
+
Model weights are NOT included. The decision models (tev1, nimble, clef, ...) are
|
|
16
|
+
downloaded separately through Ollama and carry their own licences.
|
|
@@ -0,0 +1,71 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: decision-usage
|
|
3
|
+
description: Use for probability judgements on local material — `decision` (routing, classify/filter/rank, relevance, yes-no, 1–N scoring; inline/file/glob/tool input) and `monitor` (wait for a background job, file, command output or job event; polling, not a scheduler). Both return probabilities, not prose.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Using the local decision model (`decision`) and `monitor`
|
|
7
|
+
|
|
8
|
+
A **local** System One model returns probabilities for a bounded question — routing, classification, filtering, relevance scoring. The plugin registers two tools: `decision` (one-shot judgement) and `monitor` (repeated probing until an event happens).
|
|
9
|
+
|
|
10
|
+
## 1. `decision`: one call, one mode
|
|
11
|
+
|
|
12
|
+
Give `material` plus exactly one mode; **giving both is an error**:
|
|
13
|
+
|
|
14
|
+
- `rubric` + `items` + `type` — one shared rubric judges N items independently. Write the rubric **once**.
|
|
15
|
+
- `questions` — M different questions on the same material: `{ name, type, instructions, criteria? }`.
|
|
16
|
+
|
|
17
|
+
Ask up to 64 items/questions in ONE call; **NEVER loop one call per item** (cost ≈ count × body). `type` is `choice`/`noul`/`score`. `noul` returns a BOOLEAN `value`; its P(true) is in `probabilities.true` (0.475 = 47.5% chance of yes). `score` is on the 0..N-1 level scale, not normalised. For `choice`/`score` write the label set **once**: top-level `criteria` in batch mode, per-question in mixed mode. `noul` takes optional `{ false, true }` strings.
|
|
18
|
+
|
|
19
|
+
Each `item` is the complete short string or object to judge; the plugin never infers what it refers to in `material`.
|
|
20
|
+
|
|
21
|
+
Other parameters: `depth`, `model` (see §3). Invalid calls fail with a stable `kind` (`mode_invalid` for mode/count/shape problems, `invalid_request` for field-level ones, `material_*`/`source_*`/`upstream_*` for the rest) — read the message, it lists the legal values.
|
|
22
|
+
|
|
23
|
+
## 2. `material`: inline, file, glob, or tool
|
|
24
|
+
|
|
25
|
+
- a string (inline material);
|
|
26
|
+
- `{ file: { path, startLine?, endLine? } }` — `path` is workspace-root-relative (absolute, `..`, escaping symlinks refused); ranges are 1-based inclusive;
|
|
27
|
+
- `{ glob: { pattern, limit? } }`;
|
|
28
|
+
- `{ tool: { name, arguments? } }` — judge a host tool's **raw output**: it never enters your context, full text at `material.spillPath`; not with `file`/`glob` (`material_conflict`); the plugin's own tools are refused (`upstream_self_recursion`).
|
|
29
|
+
|
|
30
|
+
A file over the cap **fails**; a glob over the total cap is **truncated** (never complete). `material` reports the source, byte counts and `truncated`; `coverage.truncated`/`truncatedBy` mirror it.
|
|
31
|
+
|
|
32
|
+
## 3. `depth`: how much work the plugin may do
|
|
33
|
+
|
|
34
|
+
- `quick` — default model only; over budget it **fails**. `normal` (default) — smallest fitting model, escalates on overflow. Batching never changes `results[]`: one entry per item. `deep` — starts bigger, may chunk the material.
|
|
35
|
+
|
|
36
|
+
An explicit `model` disables capacity choice/escalation. Escalation switches to a different base model, so thresholds calibrated for one may not carry over (`cost.escalated`). Any Ollama model answering System One works: `model` pins a call, `defaultModel` changes the session default, `ladder` controls escalation, `node scripts/list-models.mjs` lists installed models.
|
|
37
|
+
|
|
38
|
+
## 4. Read `results`, `coverage`, `cost`, `notes`
|
|
39
|
+
|
|
40
|
+
- `results[]` — `{ id, type, value, probabilities?, confidence? }`, one entry per item/question in order; `value` = chosen label (`choice`/`score`) or boolean (`noul`).
|
|
41
|
+
- `coverage` — `requested`/`answered`/`batches`/`truncated`/`truncatedBy?`/`failed`/`aborted`: partial results are data, not failure.
|
|
42
|
+
- `cost` — `model`/`inputTokens`/`bodyBytes`/`wallMs`/`escalated`/`attempts[]`. `notes[]` — the plugin's observations verbatim (size, batching, low-confidence names).
|
|
43
|
+
|
|
44
|
+
`confidence` is the concentration of the distribution (1 - H/ln N); it is NOT accuracy. A `notes[]` line naming low-confidence items is a FLAG, not a filter: unnamed items are NOT thereby endorsed; read `probabilities` and escalate that question to your own reasoning.
|
|
45
|
+
|
|
46
|
+
In `deep` the material may be chunked: a per-chunk maximum is a locator, not a document-level probability.
|
|
47
|
+
|
|
48
|
+
## 5. `monitor`: wait for an event
|
|
49
|
+
|
|
50
|
+
`monitor` samples one source on an interval, asks the same `rubric`, counts hits. `once` returns at `countThreshold` (or deadline); `continuous` runs to deadline/cancel and returns accumulated `count` (no `hit`). Notices only at a **trigger** (once: threshold; continuous: every `notifyEvery` hits).
|
|
51
|
+
|
|
52
|
+
`description`=job description (folded to one line); label=source.kind+description. `job_output` streams ISO lines: `config …`, `hit=true P=…` per hit (consecutive miss only on hit→miss edge), `trigger_num=…` (N>1), `summary …`. `finished [...]`: once `hit/P|EP/dur` (or `hit/EP/dur` if threshold>1), continuous `trigger_num/EP/dur`; hit bool; ms; once EP=Σ(hit P)/N (pad 0); continuous EP=mean(trigger P), trigger P=Σ(window hit P)/notifyEvery, 0 trigger⇒0; once failure 1-EP. Boolean-only verdict has no P ⇒ 0.00 (detail/notice/stream).
|
|
53
|
+
|
|
54
|
+
Parameters: `description` (required; job label) · `source` (required) · `rubric` (required) · `type` (`noul` only) · `criteria` · `hitThreshold` (default 0.5) · `tailBytes` (default 4096, max 65536) · `mode` (`once`/`continuous`) · `countMode` (`cumulative`/`consecutive`; default cumulative) · `notifyEvery` (continuous, default 1) · `intervalMs` (5000) · `timeoutMs` (120000) · `countThreshold` (`once` return threshold, default 1; `continuous` ignores it) · `background` · `notify` · `coalesce` · `maxNotificationsPerMinute` (default 6).
|
|
55
|
+
|
|
56
|
+
`source.kind` (each kind accepts only its own keys):
|
|
57
|
+
|
|
58
|
+
- `job` — `{ id?, select?, jobKind? }`: tail a background job (default: newest of this session);
|
|
59
|
+
- `file` — `{ path, startLine?, endLine? }`; `glob` — `{ pattern, limit? }`;
|
|
60
|
+
- `command` — `{ argv }`: re-runs a host command through the plugin's upstream pipe; `argv[0]` picks the interpreter (`pwsh`/`git_bash`/`bash`/`sh`/`zsh`), interpreter flags are stripped, **quote arguments containing spaces yourself**;
|
|
61
|
+
- `events` — `{ types, limit? }`: reads the job-event ring buffer (`job.registered`/`job.stopping`/`job.output`/`job.progress`/`job.settled`/`job.completed`/`job.failed`/`job.killed`/`job.removed`/`job.running`/`job.unknown`; typos fail as `source_invalid`), falling back to the session's current job list.
|
|
62
|
+
|
|
63
|
+
Empty: `file`/`glob` ⇒ immediate (`samples:1`, `emptySource:true`, no model call); `job`/`events`/`command` ⇒ P=0, keep waiting. Permanent source errors (bad descriptor/outside root/explicit missing job id/file too large) fail fast; transient errors retry.
|
|
64
|
+
|
|
65
|
+
`background:true` returns `jobId`; `job_output` streams progress (consumed on read); abort=>killed, all-error=>failed; abort interrupts in-flight decide/sleep. `notify:'inbox'` posts a one-line notice only at a trigger, only with `background:true`; `coalesce`/`maxNotificationsPerMinute` bound them.
|
|
66
|
+
|
|
67
|
+
Output: `once` returns `hit`; `continuous` returns `count` (accumulated hits) and `streak` instead of `hit`. Both carry `description`/`ep`/`triggers`/`hitProbability`/`countThreshold`/`countMode`/`notifyEvery`/`probability`/`evidence` plus `samples`/`hitSamples`/`errors`/`timedOut`/`elapsedMs`/`mode`/`source`/`clampNotes`/`notes`. Failed probes count in `errors`; a hit is a probability, not proof.
|
|
68
|
+
|
|
69
|
+
## 6. Hard boundaries
|
|
70
|
+
|
|
71
|
+
Probabilities, not proof: never correctness evidence, never the sole basis for a pass/allow (that needs artifact-level evidence), never the basis of any **fail-open or fail-closed** policy (deterministic rules own that). tev1 returns only probabilities and no text — for a human-readable rationale use a generative model.
|