@vllm-sr/dsh 0.4.1 → 0.5.0

package/README.md

@@ -11,8 +11,8 @@ agent orders, is the work really done, which team should get each of these 230 t
 - **System One**, the decision model, decides: `vllm-sr/d3-ultra`. It answers typed questions (choice, yes/no, score,
   box) with probabilities, from one forward pass per question instead of generated text.
 
-Both come from [System Duo](https://app.vllm-sr.ai) with one free key, and the plugin gets that key for you: System One
-answers at `https://app.vllm-sr.ai/v1/systemone`, and `vllm-sr/auto` at `https://app.vllm-sr.ai/v1/chat/completions`.
+Both take a free [System Duo](https://app.vllm-sr.ai) key, which the plugin gets for you: System One answers at `https://app.vllm-sr.ai/v1/systemone`,
+and `vllm-sr/auto` at `https://app.vllm-sr.ai/v1/chat/completions`.
 
 ## Quick start
 
@@ -26,9 +26,10 @@ The first line installs DeepSeek Harness (0.2.1-rc.5 or a later 0.2 release, on
 `dsh plugin` uses to install plugins; without pnpm, `dsh plugin add` stops with "pnpm was not found". On a machine
 with git, the package can also come straight from GitHub: `dsh plugin --profile web add github:Xunzhuo/dsh-system-one-plugin`.
 
-On its first start the plugin gets one free System Duo key, with the same small proof of work the site's "Get your
-API key" button runs in the browser, and saves it in DeepSeek Harness's credential store and in `~/.dsh/.env` (readable
-by you only). A notice in the session says so. `vllm-sr/auto` becomes the default model; choose another model
+On its first start the plugin gets a free System Duo key, with the same small proof of work the site's "Get your API
+key" button runs in the browser, and saves it in DeepSeek Harness's credential store and in `~/.dsh/.env` (readable
+by you only). A line under your first request says so: "Free API key created for app.vllm-sr.ai, saved in
+~/.dsh/.env", or why the site refused one. `vllm-sr/auto` becomes the default model; choose another model
 in the app and System One keeps working beside it.
 
 Headless:
@@ -56,6 +57,22 @@ of your profile patch stay as they are.
 | Done-check | `agent/turn-stopping` | complete?, checked?, waiting for you? | an unfinished turn goes back once or twice to finish and verify |
 | Tool | `system_one` | any rubric the LLM writes, over a file of items or images | one result row per item (JSONL or CSV) and a summary |
 
+### Span answers (vllm-sr/vela3-turbo)
+
+`vllm-sr/vela3-turbo` also answers span questions: which words of a text carry a label, with code-point offsets and a
+probability each. Once the Decision API lists `span` for it (`spans: auto`, the default), four reflexes point at words
+instead of whole passages:
+
+- **Shield**: in a passage it flagged, the sentences that give orders go, and the text around them stays. The passage
+  is scored again without them, and a cut that leaves an injection behind falls back to removing lines.
+- **Guard**: a call it flags names the shell command that does the damage, in its card and its question.
+- **Done-check**: words of the final answer that nothing the turn read supports (preset `halu`) are marked, and
+  System Two goes back to re-verify just those.
+- **Privacy** (`privacy: true`, off by default): personal data in tool results (preset `pii`, 17 labels) becomes its
+  label, as `[EMAIL_ADDRESS]`, before System Two reads it: the nine sensitive labels by default (names, e-mail and
+  street addresses, phone, card, bank and ID numbers, IP addresses), while dates, organizations and places pass. A
+  small card shows what was masked, by label only.
+
 ## See it in the Web UI
 
 In `dsh web`, System One shows up in the conversation itself, where it acted, marked with an ember **System One**
@@ -70,11 +87,16 @@ badge that carries the decision's latency:
 - **Done-check**: a card after the answer: done, waiting on you, or sent back to finish.
 - **`system_one` tool**: its row shows the batch: items, questions, each question's answer distribution and the
   results file.
+- **Free key**: a **System Duo** line under your first request when the plugin created a key, or could not.
+- **Spans** (vllm-sr/vela3-turbo): the words a span answer marked, in the card that used them: removed injections
+  struck through, a command's dangerous part, unsupported claims of the answer underlined, and masked personal data as
+  label chips in a **Privacy** card.
 
 Routine checks (an allowed call, a clean result) are one line and fold with the turn's work. Cards stay visible when
 the work folds, so a finished turn still shows what System One decided. Parallel calls in one step share one line or
 card, and every decision pops in with a short animation the first time it appears (none with reduced motion). Each
-element shows the Decision API's own answer: the probabilities, the model and the measured latency.
+element shows the Decision API's own answer: the probabilities, the model (in bold, as `vllm-sr/d3-ultra`) and the
+measured latency.
 
 Set `systemOneUi` to `compact` for one line per decision, or to `off` to hide them.
 
@@ -85,21 +107,26 @@ attributes that say what it shows:
 
 | Attribute | Values |
 | --- | --- |
-| `data-system-one` | `card`, `compact` (one line) or `tool` (the `system_one` row) |
-| `data-system-one-reflex` | `triage`, `guard`, `shield`, `done` or `tool` |
-| `data-system-one-verdict` | triage `brief`; guard `allow`, `ask` or `deny`; shield `clean`, `removed` or `flagged`; done-check `stop` or `continue`; tool `running` or `done`; `error` when System One did not answer |
+| `data-system-one` | `card`, `compact` (one line), `tool` (the `system_one` row) or `notice` (the free-key line) |
+| `data-system-one-reflex` | `triage`, `guard`, `shield`, `privacy`, `done`, `tool` or `key` |
+| `data-system-one-verdict` | triage `brief`; guard `allow`, `ask` or `deny`; shield `clean`, `removed` or `flagged`; privacy `masked`; done-check `stop` or `continue`; tool `running` or `done`; key `created` or `failed`; `error` when System One did not answer |
 | `data-system-one-outcome` | guard, once you answer its approval: `allowed`, `rejected`, `cancelled` or `failed` |
-| `data-system-one-why` | done-check: `complete`, `waiting`, `unfinished` or `unchecked` |
+| `data-system-one-why` | done-check: `complete`, `waiting`, `unfinished`, `unchecked` or `unsupported` |
 | `data-system-one-tone` | its colour: `ok` (green), `info` (ember), `warn` (amber), `bad` (red) or `muted` |
 | `data-system-one-count` | how many decisions it holds (a batch: how many items) |
 | `data-system-one-turn`, `data-system-one-step` | where it acted (the step for guard and shield) |
+| `data-system-one-model` | the model that answered, as `vllm-sr/d3-ultra` |
+| `data-system-one-span-model`, `data-system-one-spans` | the span model that marked words, as `vllm-sr/vela3-turbo`, and how many |
 
 Inside a root, `data-system-one-part` marks `badge` (the ember **System One** pill), `latency`, `reflex`, `summary`,
 `model`, `head`, `toggle` (the button of a one-line element), `rows`, `verdict`, `outcome`, `subject`, `facts`,
-`note` and `more`, and in the batch row also `stack`, `legend`, `failed`, `results` and `raw`. Each number is a
-`data-system-one-fact`: `kind`, `effort`, `unclear` and `risky` (triage), `risk`, `leaks` and `on-task` (guard),
-`passages` and `injection` (shield), `complete`, `checked` and `waiting` (done-check). Decision rows carry
-`data-system-one-decision` and `data-system-one-call`, batch questions `data-system-one-question`. For example:
+`note` and `more`, in the batch row also `stack`, `legend`, `failed`, `results` and `raw`, and in the free-key line
+`badge` (the **System Duo** pill), `summary` and `links`. Span answers add `excerpts`, each `excerpt`, every marked
+`span` (with `data-system-one-span-label` and `data-system-one-probability`) and, in a Privacy card, the `labels`. Each
+number is a `data-system-one-fact`: `kind`, `effort`, `unclear` and `risky` (triage), `risk`, `leaks` and `on-task`
+(guard), `passages`, `injection` and `spans` (shield), `masked` (privacy), `complete`, `checked`, `waiting` and
+`unsupported` (done-check). Decision rows carry `data-system-one-decision` and `data-system-one-call`, batch questions
+`data-system-one-question`. For example:
 
 | Element | Selector |
 | --- | --- |
@@ -113,11 +140,17 @@ Inside a root, `data-system-one-part` marks `badge` (the ember **System One** pi
 | The ember badge, and its latency | `[data-system-one-part="badge"]`, `[data-system-one-part="latency"]` |
 | "Dangerous", and "You rejected it" | `[data-system-one-fact="risk"]`, `[data-system-one-part="outcome"]` |
 | The reason | `[data-system-one-part="note"]` |
+| The model's name (bold) | `[data-system-one-part="model"]` |
+| Free key created | `[data-system-one="notice"][data-system-one-verdict="created"]` |
+| The words a span answer marked | `[data-system-one-part="span"]` (`[data-system-one-span-label="INJECTION"]`, `"DANGEROUS"`, `"unsupported"`, a PII label) |
+| Privacy card | `[data-system-one-reflex="privacy"][data-system-one-verdict="masked"]` |
+| Done-check that sent claims back | `[data-system-one-reflex="done"][data-system-one-why="unsupported"]` |
 
 ## Your data
 
 System One sends what it decides about: the user's request (clipped), each tool call it reviews, the passages of tool
-results it screens, the end of the turn, and the items given to the `system_one` tool. System Two receives the
+results it screens, the end of the turn (with what the turn read, for the done-check's span answer), and the items
+given to the `system_one` tool. With `privacy: true`, the results of `privacyTools` go to System One whole. System Two receives the
 conversation as any LLM provider does. System Duo logs requests to improve the services and to train models; read its
 [terms](https://app.vllm-sr.ai/terms) and [privacy](https://app.vllm-sr.ai/privacy) pages before you use it with
 private code or data. Each reflex can be turned off.
@@ -151,11 +184,25 @@ Every field has a default. Set any of them on the `system-one` row of a profile
     tool: true
     trace: ~/.dsh/system-one/trace.jsonl   # or off
     systemOneUi: card              # how Chat shows System One: card, compact (one line each) or off
+    spans: auto                    # span answers: auto (once spanModel lists span), on or off
+    spanModel: vllm-sr/vela3-turbo
+    shieldSpans: true              # remove injected words, not lines
+    guardSpans: true               # name the dangerous part of a flagged call
+    doneHalu: true                 # send unsupported claims back
+    doneHaluAt: 0.6                # the lowest probability of an unsupported word that counts
+    privacy: false                 # mask personal data before System Two reads it
+    privacyAt: 0.5                 # the lowest probability of a personal-data span that is masked
+    privacyLabels: [PERSON, EMAIL_ADDRESS, PHONE_NUMBER, STREET_ADDRESS, CREDIT_CARD, IBAN_CODE, US_SSN, US_DRIVER_LICENSE, IP_ADDRESS]
+    privacyTools: [web_fetch, bash, read]
 ```
 
 `systemOneUi` applies to decisions made after a restart with the new value; decisions already in a session keep the
 form they were recorded with. The tool's custom row is drawn for the default `toolName`, `system_one`.
 
+With `privacy: true`, System Two sees `[LABEL]` placeholders instead of personal data in the results of
+`privacyTools`, so it cannot copy a real value into a file it writes from what it read; it is told to ask you
+instead. Span records keep labels and offsets of personal data, never the data itself.
+
 The key comes from the credential reference `VLLM_SR_API_KEY` (`llmKeyEnv`), which both systems use. A separate
 System One key in `DECISION_API_KEY` (`apiKeyEnv`), as plugin 0.2 kept it, still wins for System One; a 0.2 key for
 `vllm-sr/auto` keeps working too.

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@vllm-sr/dsh",
-  "version": "0.4.1",
-  "description": "System Duo for DeepSeek Harness. Thinking, fast and slow: System One decision models guard, shield, triage and check your agent around System Two (vllm-sr/auto), with one free app.vllm-sr.ai key.",
+  "version": "0.5.0",
+  "description": "System Duo for DeepSeek Harness. Thinking, fast and slow: System One decision models guard, shield, triage and check your agent around System Two (vllm-sr/auto), with a free app.vllm-sr.ai key.",
   "keywords": [
     "deepseek-harness",
     "dsh",

package/src/client.js

@@ -156,8 +156,8 @@ export function createDecisionClient(options) {
    * @param {AbortSignal} [request.signal]
    * @param {number} [request.timeoutMs] - per-call deadline (default: the client's)
    * @param {number} [request.retries] - per-call retries (default: the client's)
-   * @returns {Promise<{answers: object[], ms: number, model: string, inputTokens: number, requestId?: string}>}
-   *   answers in input order
+   * @returns {Promise<{answers: object[], spans: object[], thresholds: object[], ms: number, model: string, inputTokens: number, requestId?: string}>}
+   *   answers, span answers and the thresholds applied, in input order
    */
   async function ask({ model, questions, inputs, signal, timeoutMs: callTimeoutMs, retries: callRetries }) {
     if (inputs.length === 0 || inputs.length > ITEMS_PER_REQUEST) {
@@ -176,10 +176,10 @@ export function createDecisionClient(options) {
     const started = performance.now()
     try {
       const { json, requestId } = await post('/v1/decisions', body, signal, { timeoutMs: callTimeoutMs, retries: callRetries })
-      const answers = [json.answers, ...rest.map((_, index) => json.states?.[`i${index + 1}`]?.answers ?? {})]
+      const each = field => [json[field] ?? {}, ...rest.map((_, index) => json.states?.[`i${index + 1}`]?.[field] ?? {})]
       let inputTokens = json.usage?.input_tokens ?? 0
       for (const entry of Object.values(json.states ?? {})) inputTokens += entry.usage?.input_tokens ?? 0
-      return { answers, ms: performance.now() - started, model: json.model ?? model, inputTokens, requestId }
+      return { answers: each('answers'), spans: each('spans'), thresholds: each('thresholds'), ms: performance.now() - started, model: json.model ?? model, inputTokens, requestId }
     } finally {
       release()
     }
@@ -190,7 +190,15 @@ export function createDecisionClient(options) {
     send('/api/info', 'GET', undefined, {}, AbortSignal.timeout(timeoutMs)).catch(() => {})
   }
 
-  return { ask, warm, hasKey: async () => Boolean(await apiKey()), close: () => agent.destroy() }
+  /** The published models' cards (no key needed): `[{id, question_types, presets}]`. */
+  async function models(signal) {
+    const response = await send('/v1/systemone/models', 'GET', undefined, { accept: 'application/json' }, signal ?? AbortSignal.timeout(timeoutMs))
+    if (response.status !== 200) throw new DecisionError(`Decision API ${response.status} listing models`, response.status)
+    const cards = JSON.parse(response.text)?.data
+    return Array.isArray(cards) ? cards : []
+  }
+
+  return { ask, warm, models, hasKey: async () => Boolean(await apiKey()), close: () => agent.destroy() }
 }
 
 /** Probability that a noul answer is yes, or undefined when the question failed. */