@qvac/harness 0.2.6 → 0.2.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/skills/gh-stack/SKILL.md +24 -24
- package/.claude/skills/gh-stack/references/commands.md +2 -2
- package/.claude/skills/js-house-style/SKILL.md +89 -68
- package/.claude/skills/kvcache/SKILL.md +7 -7
- package/.claude/skills/pr-create/SKILL.md +58 -15
- package/.claude/skills/pr-review/SKILL.md +83 -14
- package/CLAUDE.md +3 -0
- package/README.md +26 -4
- package/dist/harness.bundle +5 -4
- package/dist-lib/lib/docx/index.d.ts +5 -0
- package/dist-lib/lib/docx/index.js +49 -0
- package/dist-lib/lib/tools/exec/adapters/python/pyodide.js +2 -0
- package/dist-lib/lib/tools/exec/adapters/python/wheels.js +2 -1
- package/dist-lib/lib/tools/exec/adapters/python/worker/client.js +1 -1
- package/docs/decisions/2026-05-29-kvcache-tool-turns.md +75 -0
- package/docs/decisions/2026-06-02-kvcache-tool-turns-reenabled.md +127 -0
- package/docs/decisions/2026-06-23-android-cpu-llm-defaults.md +126 -0
- package/docs/decisions/2026-07-29-assistant-owned-storage-layout.md +88 -0
- package/docs/decisions/2026-08-04-sidecar-orphan-reaping.md +79 -0
- package/docs/decisions/2026-08-11-pyodide-worker-thread.md +68 -0
- package/docs/decisions/2026-08-17-harness-hook-pipeline.md +84 -0
- package/docs/decisions/2026-08-17-harness-outbound-egress-guard.md +176 -0
- package/docs/decisions/2026-08-20-chat-prompt-prefix-is-stable.md +95 -0
- package/docs/decisions/2026-08-20-kvcache-static-placement.md +109 -0
- package/docs/decisions/2026-08-21-vendored-python-wheels.md +85 -0
- package/docs/decisions/2026-08-26-exec-is-not-gated-by-skills.md +64 -0
- package/docs/decisions/2026-08-27-writable-skills-overlay.md +56 -0
- package/docs/decisions/2026-09-01-skill-operation-manifests.md +53 -0
- package/docs/decisions/2026-09-01-usage-breakdown-is-measured.md +78 -0
- package/docs/decisions/2026-09-03-usage-breakdown-reported-by-the-harness.md +136 -0
- package/docs/decisions/INDEX.md +38 -0
- package/docs/decisions/TEMPLATE.md +32 -0
- package/package.json +6 -2
|
@@ -0,0 +1,56 @@
|
|
|
1
|
+
---
|
|
2
|
+
id: ADR-2026-08-27-writable-skills-overlay
|
|
3
|
+
status: active
|
|
4
|
+
introduced_by: PR-1857
|
|
5
|
+
supersedes: []
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# A user-added skill layers over the bundled base and is trusted only by shadow
|
|
9
|
+
|
|
10
|
+
Skills load from two roots. The bundled base is the read-only, version-pinned set
|
|
11
|
+
shipped with the app. Over it sits a writable, per-user **overlay** at
|
|
12
|
+
`<sdk-home>/skills`, resolved in `lib/tools/skills/overlay-dir.ts`
|
|
13
|
+
from the `QVAC_CONFIG_PATH` env var that `configureSdkStorage` publishes on both
|
|
14
|
+
runtimes — read via env rather than importing from `assistant`, which would be a
|
|
15
|
+
circular dependency, and needing no new caller argument, so desktop and mobile
|
|
16
|
+
provide it just by configuring storage. `loadSkillCatalog` in
|
|
17
|
+
`lib/tools/skills/catalog.ts` reads both roots and merges them.
|
|
18
|
+
|
|
19
|
+
## Decision
|
|
20
|
+
|
|
21
|
+
**The merge key is the folder name, and the overlay wins.** A skill's identity is
|
|
22
|
+
its folder, not its front-matter `name` — because the `skill` tool reads a skill by
|
|
23
|
+
folder path (`/<name>/SKILL.md`, `lib/tools/skill.ts`). Keying the
|
|
24
|
+
catalog on the folder makes the merge key and the on-disk read lookup the same key,
|
|
25
|
+
so they can never disagree. (Keying on `meta.name` did not: an overlay `gmail/`
|
|
26
|
+
declaring `name: pirate` would file itself as a _new_, untrusted `pirate` entry
|
|
27
|
+
while its files still shadowed the trusted base `gmail` at read time — so loading
|
|
28
|
+
`gmail` served the overlay body under the base's grants, bypassing the merge below.
|
|
29
|
+
A divergent `meta.name` is now ignored.) An overlay folder matching a bundled folder
|
|
30
|
+
_shadows_ it; any other folder is a _new_ skill.
|
|
31
|
+
|
|
32
|
+
**A new overlay skill is untrusted.** A skill's front matter is a grant: loading it
|
|
33
|
+
unlocks tools, widens the credential egress allow-list, and classifies MCP methods
|
|
34
|
+
as reads. A file anyone can drop on disk must not do those things silently. So a
|
|
35
|
+
new overlay skill is excluded from `credentialGrants`
|
|
36
|
+
(`lib/tools/skills/credential-grants.ts`) and from `mcpReadMethods`
|
|
37
|
+
(`catalog.ts`) — it still loads and runs, but its http/exec calls go through the
|
|
38
|
+
normal per-call approval gate, and its declared reads count as writes. This is the
|
|
39
|
+
`origin: 'overlay'` marker, which is local-only and never crosses the wire.
|
|
40
|
+
|
|
41
|
+
**A shadow overrides instructions, never capability.** Editing a first-party skill
|
|
42
|
+
is the point of the overlay, but the edit must not be able to re-arm it. `mergeOverlay`
|
|
43
|
+
takes the overlay copy's body and display metadata but pins its `allowList`,
|
|
44
|
+
`credentials`, `mcpReads`, and `platform` to the trusted base. So an edit can't add
|
|
45
|
+
an egress host, can't break the skill's auth by dropping a field, and can't claim an
|
|
46
|
+
OS the base doesn't support — and a shadow is _not_ marked `origin: 'overlay'`,
|
|
47
|
+
because its capability surface is the base's.
|
|
48
|
+
|
|
49
|
+
**An undeclared platform means "here", not "nowhere".** The base treats a skill with
|
|
50
|
+
no `platform` as supported nowhere, because every bundled skill lists its platforms
|
|
51
|
+
explicitly. A user dropping a bare `SKILL.md` (e.g. one from the Claude or Codex
|
|
52
|
+
skills repos) does not know that incantation, and requiring it would mean every
|
|
53
|
+
such skill shows as "not available on this device". So a _new_ overlay skill with no
|
|
54
|
+
declared platform defaults to every platform the app targets; the other gates
|
|
55
|
+
(missing tools, model capability) still apply, so this only lifts the OS gate. This
|
|
56
|
+
is what makes "drop a folder and it works" true.
|
|
@@ -0,0 +1,53 @@
|
|
|
1
|
+
---
|
|
2
|
+
id: ADR-2026-09-01-skill-operation-manifests
|
|
3
|
+
status: active
|
|
4
|
+
introduced_by: PR-000
|
|
5
|
+
supersedes: []
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# A provider write is a compiled skill operation, not a shaping hook in `http_request`
|
|
9
|
+
|
|
10
|
+
## Decision
|
|
11
|
+
|
|
12
|
+
A skill declares its write operations in a sibling `operations.json` manifest. The harness compiles
|
|
13
|
+
each entry into a typed tool (`gmail_send`, `gmail_draft`, `calendar_create_meet_event`,
|
|
14
|
+
`calendar_add_meet`) that validates arguments, builds the provider payload, and delegates the call to
|
|
15
|
+
`http_request`. `http_request` itself carries no provider knowledge: the per-host request/response
|
|
16
|
+
shaping registry is removed, and an oversized JSON response is reduced by a generic per-node pruner.
|
|
17
|
+
|
|
18
|
+
A compiled tool is owned by the skill whose bundled manifest defined it. The tool gate matches on the
|
|
19
|
+
owning skill, not the tool name, so a sibling skill or a writable-overlay skill that declares the same
|
|
20
|
+
name in `tools:` cannot expose it. A writable overlay never contributes operations of its own; editing
|
|
21
|
+
a bundled skill keeps the base manifest's tools, exactly as it keeps the base credentials and
|
|
22
|
+
allow-list.
|
|
23
|
+
|
|
24
|
+
## Context
|
|
25
|
+
|
|
26
|
+
Gmail, Calendar, and Docs each needed request shaping (RFC 2822 assembly, base64url, conference
|
|
27
|
+
requests) and response shaping. That knowledge lived inside the generic `http_request` tool as a
|
|
28
|
+
host-keyed registry, so every new provider widened a shared tool, and the model still had to write raw
|
|
29
|
+
URLs and bodies for a write it could not verify.
|
|
30
|
+
|
|
31
|
+
Shaping also meant the response cap was provider-specific. A provider with no registry entry got a raw
|
|
32
|
+
character slice, which cuts JSON mid-token and drops shallow identifiers the model needs next.
|
|
33
|
+
|
|
34
|
+
## Alternatives Considered
|
|
35
|
+
|
|
36
|
+
- **Keep the shaping registry, add a provider per file.** Rejected: provider coupling in a generic
|
|
37
|
+
tool, and it never gives the model a typed schema for the write.
|
|
38
|
+
- **Let an overlay skill ship its own `operations.json`.** Rejected: a user-editable file would mint a
|
|
39
|
+
tool with the base skill's credentials — privilege escalation by rename.
|
|
40
|
+
|
|
41
|
+
## Consequences
|
|
42
|
+
|
|
43
|
+
- A new provider write is a manifest entry plus a builder, never an `http_request` change.
|
|
44
|
+
- Every operation tool is `externalWrite`, non-auto-approved, and skill-gated; it passes through the
|
|
45
|
+
same egress guard, private-IP check, and approval path as a raw `http_request`.
|
|
46
|
+
- Response truncation is uniform: parseable JSON in, parseable JSON out, for every host.
|
|
47
|
+
- The manifest describes one HTTP call. Multi-step or paginated flows stay in skill instructions.
|
|
48
|
+
- Revisit if a provider needs a write whose payload cannot be expressed by a named builder.
|
|
49
|
+
|
|
50
|
+
## When Agents Should Read This
|
|
51
|
+
|
|
52
|
+
Read this when changing the paths, workflows, runtime boundaries, or platform
|
|
53
|
+
behavior listed for this ADR in `docs/decisions/INDEX.md`.
|
|
@@ -0,0 +1,78 @@
|
|
|
1
|
+
---
|
|
2
|
+
id: ADR-2026-09-01-usage-breakdown-is-measured
|
|
3
|
+
status: superseded
|
|
4
|
+
introduced_by: PR-1953
|
|
5
|
+
superseded_by: ADR-2026-09-03-usage-breakdown-reported-by-the-harness
|
|
6
|
+
supersedes: []
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
# The Usage breakdown divides a measured total, it never estimates tokens
|
|
10
|
+
|
|
11
|
+
## Decision
|
|
12
|
+
|
|
13
|
+
Only the Usage modal's total is a token count: `cacheTokens` from the turn's SDK
|
|
14
|
+
metrics. Every row below it is that total split by character share, at a
|
|
15
|
+
chars-per-token ratio fitted per chat against the SDK's own count on the first
|
|
16
|
+
turn that reports one. No row converts characters to tokens with a hardcoded
|
|
17
|
+
ratio, and the fit is never recomputed on a later turn.
|
|
18
|
+
|
|
19
|
+
Two costs the model receives but we do not author are declared as measured
|
|
20
|
+
constants rather than absorbed: the chat template's tool preamble
|
|
21
|
+
(`FRAMING_CHARS_BY_DIALECT`, read through `toolBlockFramingChars`) and one
|
|
22
|
+
exchange's fixed overhead (`TURN_OVERHEAD_TOKENS`, charged to the turn via the
|
|
23
|
+
fit's denominator). Only the first is keyed by dialect — a template either has a
|
|
24
|
+
tool preamble or has none, whereas every template wraps its turns, so declaring
|
|
25
|
+
zero overhead would be further from the truth than Qwen's number is elsewhere.
|
|
26
|
+
|
|
27
|
+
## Context
|
|
28
|
+
|
|
29
|
+
The SDK exposes no tokenizer — `@qvac/sdk` 0.18.2 ships tokenization only for
|
|
30
|
+
VLA, whose docs tell consumers to tokenize their own input — so a real count is
|
|
31
|
+
unavailable and `cacheTokens` is the only ground truth.
|
|
32
|
+
|
|
33
|
+
A single hardcoded ratio cannot work. Measured on Qwen3.5-4B, real density is
|
|
34
|
+
3.17 chars-per-token for a prompt with no tools and 4.23 for a seven-tool
|
|
35
|
+
prompt, because tool schemas are dense JSON and the rest is prose. The retired
|
|
36
|
+
3.5 constant therefore overshot the measured total, and the scaling fallback
|
|
37
|
+
squeezed every row to fit — which is why the rows appeared to grow between turns
|
|
38
|
+
while Conversation collapsed to single digits.
|
|
39
|
+
|
|
40
|
+
Character counting is also blind to two real costs: the template's own tool
|
|
41
|
+
section, and per-turn role wrappers plus the harness time annotation. Left
|
|
42
|
+
unattributed they are spread across the rows that do have characters, which
|
|
43
|
+
overstated a tool-less agent's System Prompt by 47%.
|
|
44
|
+
|
|
45
|
+
## Alternatives Considered
|
|
46
|
+
|
|
47
|
+
- **Better hardcoded ratios per segment.** Rejected: it replaces one invented
|
|
48
|
+
constant with three, all model-specific, and all silently wrong on a new model.
|
|
49
|
+
- **A two-point fit** using the delta between the first two turns to measure the
|
|
50
|
+
per-turn cost directly. Correct and constant-free, but it reports nothing until
|
|
51
|
+
a chat has two turns.
|
|
52
|
+
- **Keying the framing globally rather than by dialect.** Rejected: dialect is
|
|
53
|
+
the axis the template varies on, and an unmeasured model should declare zero
|
|
54
|
+
rather than borrow another template's number.
|
|
55
|
+
|
|
56
|
+
## Consequences
|
|
57
|
+
|
|
58
|
+
- The rows sum exactly to the measured total on every turn, and the prompt rows
|
|
59
|
+
hold still across turns of one chat. Both are load-bearing: QA asserts them.
|
|
60
|
+
- Getting a constant wrong is cosmetic. It shifts the split between rows and can
|
|
61
|
+
never move the total, the sum, or the flatness — so a stale value degrades
|
|
62
|
+
silently, and only a real-model measurement can catch it.
|
|
63
|
+
- The estimator must keep mirroring the production selection functions
|
|
64
|
+
(`permittedTools`, `enabledTools`, `skillIsActive`); a drift test pins this.
|
|
65
|
+
- The fit's numerator must keep pace with what the cache holds. Tool calls and
|
|
66
|
+
their output are re-sent and counted; generated `thinking` is not, and
|
|
67
|
+
`cacheTokens` excludes it (measured: 408 generated tokens against 162 reported).
|
|
68
|
+
- Re-measure a constant by diffing the addon's `PROMPT_SEND` dump against
|
|
69
|
+
`systemPromptChars` + `toolsChars`. Drive it from `@qvac/core → examples/chat-tui.mjs`
|
|
70
|
+
over a pty with `QVAC_STORAGE` pointed at a scratch dir, and read `N ctx` from
|
|
71
|
+
the footer.
|
|
72
|
+
- Revisit if the SDK ever exposes a token-count or prompt-render op: that would
|
|
73
|
+
retire the fit and both constants.
|
|
74
|
+
|
|
75
|
+
## When Agents Should Read This
|
|
76
|
+
|
|
77
|
+
Read this when changing the paths, workflows, runtime boundaries, or platform
|
|
78
|
+
behavior listed for this ADR in `docs/decisions/INDEX.md`.
|
|
@@ -0,0 +1,136 @@
|
|
|
1
|
+
---
|
|
2
|
+
id: ADR-2026-09-03-usage-breakdown-reported-by-the-harness
|
|
3
|
+
status: active
|
|
4
|
+
introduced_by: PR-2067
|
|
5
|
+
supersedes: [ADR-2026-09-01-usage-breakdown-is-measured]
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# The device that ran the turn reports the Usage breakdown; the app only renders it
|
|
9
|
+
|
|
10
|
+
## Decision
|
|
11
|
+
|
|
12
|
+
The Usage modal's rows come from the run. The harness attributes the prompt it
|
|
13
|
+
just assembled — `systemPrompt`, `agentInstructions`, `activeSkills`, `tools` —
|
|
14
|
+
measured as `.length` on the strings it sent, converts each to tokens at the
|
|
15
|
+
measured density **for that kind of text**, and ships the result inside the
|
|
16
|
+
existing `metrics` JSON of the terminal run status
|
|
17
|
+
(`lib/capabilities/usage-breakdown.ts`).
|
|
18
|
+
|
|
19
|
+
A prompt row is never a share of the measured total. It is its own characters at
|
|
20
|
+
its own density, so it holds still across every turn of a chat while the
|
|
21
|
+
conversation grows — the property QA asks for. `conversation` is what is left of
|
|
22
|
+
`cacheTokens` once the prompt rows are subtracted, which is also where the costs
|
|
23
|
+
no segment authors end up: the template's per-message wrappers (~28 tokens a
|
|
24
|
+
message on Qwen3.5) and the reply the run just generated, which the cache holds.
|
|
25
|
+
|
|
26
|
+
`app-runtime` re-derives nothing. `buildUsageSegments` reads the newest reported
|
|
27
|
+
breakdown off the messages and maps it to labels and colors. Conversation is the
|
|
28
|
+
remainder against the live `usedTokens`, never a reported number, so occupancy
|
|
29
|
+
the report cannot know about lands on the turns rather than skewing a prompt row.
|
|
30
|
+
|
|
31
|
+
Three constants per template, keyed off `detectToolDialect`: prose chars/token
|
|
32
|
+
(the prompt body, the bio, the skills list), schema chars/token (the serialized
|
|
33
|
+
SDK function schemas), and the tool block's token count. Measured on
|
|
34
|
+
Qwen3.5-2B-Q4_K_M as `{ prose: 4.38, schema: 4.08, toolBlockTokens: 189 }` by
|
|
35
|
+
least squares over the addon's own `cacheTokens` across 14 tool/skill/bio
|
|
36
|
+
configurations x 3 turns — 42 samples, leave-one-configuration-out worst error
|
|
37
|
+
5.9%. An unmeasured template borrows these rather than reporting zeros.
|
|
38
|
+
|
|
39
|
+
Two things the previous design needed are gone: the per-chat chars-per-token fit
|
|
40
|
+
(`usage-calibration.ts`) and the app's copy of the selection gates
|
|
41
|
+
(`usage-segment-system-prompt.ts`, and with it the drift test that policed the
|
|
42
|
+
copy). `TURN_OVERHEAD_TOKENS` is gone too, and so is the 935-char framing
|
|
43
|
+
constant — the tool block is a fixed string, so it is declared in tokens.
|
|
44
|
+
|
|
45
|
+
The row for attachment filenames is retired. Those labels are composed into the
|
|
46
|
+
user turn by the assistant layer, so they are conversation characters; the app
|
|
47
|
+
was counting raw filenames that never appear in the prompt verbatim.
|
|
48
|
+
|
|
49
|
+
## Context
|
|
50
|
+
|
|
51
|
+
Every row except the total was re-derived app-side by re-running
|
|
52
|
+
`permittedTools`, `enabledTools` and `skillIsActive` against a device tool
|
|
53
|
+
report and a skill catalog. The mirror was only ever as current as the last
|
|
54
|
+
person to remember it existed, and because the fit normalizes the known
|
|
55
|
+
characters onto the measured total, an input that goes missing does not read as
|
|
56
|
+
zero in place — it silently reweights every other row.
|
|
57
|
+
|
|
58
|
+
That failure shipped. After PR-1960 the six provider skills declared tools
|
|
59
|
+
compiled from `operations.json`, so on any host that had not wired
|
|
60
|
+
`compileOperationTools` (the mobile worklet until QVAC-24473; the in-process
|
|
61
|
+
path still) `isToolSupported` marked them unavailable, the estimator dropped
|
|
62
|
+
them, their tool grants went with them, and the System Prompt row absorbed the
|
|
63
|
+
entire window: 2,361 of 2,616 tokens, with Active Skills and Tools at 0.
|
|
64
|
+
|
|
65
|
+
No tokenizer is exposed — `@qvac/sdk` 0.18.2 ships tokenization only for VLA,
|
|
66
|
+
whose docs tell consumers to tokenize their own input — so `cacheTokens` remains
|
|
67
|
+
the only ground truth and the rows remain shares of it. What changed is where
|
|
68
|
+
the characters are counted: at the point the prompt is built, where being wrong
|
|
69
|
+
is not possible without also sending a different prompt.
|
|
70
|
+
|
|
71
|
+
## Alternatives Considered
|
|
72
|
+
|
|
73
|
+
- **Keep the mirror, add a stronger drift test.** Rejected: the drift test runs
|
|
74
|
+
against a stubbed completion, so it cannot see inputs that arrive over the
|
|
75
|
+
wire — exactly the class that broke. It also grows a second maintenance
|
|
76
|
+
surface for every prompt change.
|
|
77
|
+
- **Report chars, fit in the app.** The fit needs the first metrics-bearing turn
|
|
78
|
+
of the chat, whose prompt the harness can no longer reconstruct. Rejected
|
|
79
|
+
anyway: per-turn attribution against that turn's own measurement needs no fit,
|
|
80
|
+
and the prompt rows stay flat regardless (the shares move together).
|
|
81
|
+
- **Report tokens per segment.** Requires a tokenizer we do not have. This is
|
|
82
|
+
the shape to move to when the SDK grows one: the harness swaps its char counts
|
|
83
|
+
for tokenize calls and nothing downstream changes.
|
|
84
|
+
- **One density for the whole prompt** (what shipped first, and what QA caught).
|
|
85
|
+
Rejected on measurement: with rows taken as shares of the total, a chat whose
|
|
86
|
+
tool schemas are dense JSON and whose turns are prose re-weights every row as
|
|
87
|
+
the mix changes. Three "hi" turns moved Tools 3,166 -> 3,252 -> 3,268 with an
|
|
88
|
+
unchanged tool set, and fitting one density across the matrix above is off by
|
|
89
|
+
up to 46% against six per-kind constants' 1.7%.
|
|
90
|
+
- **Keep the attachment-filenames row, reported by the assistant layer.** Held
|
|
91
|
+
back: it would put a third layer in the attribution path for a row that reads
|
|
92
|
+
zero in every chat we have looked at.
|
|
93
|
+
|
|
94
|
+
## Consequences
|
|
95
|
+
|
|
96
|
+
- A prompt-shaping change cannot desync the breakdown. There is nothing to keep
|
|
97
|
+
in sync: the numbers are measured off the prompt that was sent.
|
|
98
|
+
- A delegated chat is attributed by the device that ran it, so the observer no
|
|
99
|
+
longer needs the host's tool catalog or skill names to price the window.
|
|
100
|
+
Everything that fed the estimator therefore loses its only reader — the
|
|
101
|
+
reported tool catalog (`agentToolCatalog`, `selectToolCatalog`,
|
|
102
|
+
`selectActiveAgentHostToolCatalog`, the `skill.toolCatalog` slice and the
|
|
103
|
+
`coreV2.tools({})` call that filled it) and the app-side fs-grant mirror
|
|
104
|
+
(`agent.actor.fs-grants.ts`, its actions, its reducer branches,
|
|
105
|
+
`state.agent.fsGrants`, its selectors and `AgentFsGrants`/`NO_FS_GRANTS`).
|
|
106
|
+
PR-2068 deletes them. The real grant plumbing in `@qvac/core → lib/fs-grants.ts`
|
|
107
|
+
stays either way — that is what gives the harness its roots.
|
|
108
|
+
- The rows still sum exactly to the measured total, and the prompt rows still
|
|
109
|
+
hold still across turns of one chat. Both are asserted by QA (QA-723).
|
|
110
|
+
- The app parses only the four prompt rows out of the report. The run also sends
|
|
111
|
+
its own `conversation` and `totalTokens`, which stay useful in a log or a
|
|
112
|
+
debug view, but requiring them here would turn a harness that trimmed either
|
|
113
|
+
one into no breakdown at all.
|
|
114
|
+
- A turn recorded before this change, or run by an older peer, carries no
|
|
115
|
+
breakdown: the modal shows the window on Conversation and holds the System
|
|
116
|
+
Prompt row back rather than inventing a split. There is no estimator fallback
|
|
117
|
+
on purpose — a stale mirror is the defect this removes.
|
|
118
|
+
- `promptChars` on the reported tool catalog now has no reader, and neither does
|
|
119
|
+
the `toolCatalog` field on a device row (still parsed in
|
|
120
|
+
`device/core-v2-device.ts`, still listed in the Core v2 compatibility matrix).
|
|
121
|
+
Both stay until one schema bump retires them together.
|
|
122
|
+
- Re-measure the densities by driving `Harness.completion` headlessly with the
|
|
123
|
+
real SDK (no TUI or pty needed: it takes `model`, `agent` and `messages`
|
|
124
|
+
directly), reading the char set the harness logs beside the `cacheTokens` the
|
|
125
|
+
addon reports, and fitting the four terms by least squares. **One fresh SDK
|
|
126
|
+
home per configuration, one configuration per process**: the SDK persists cache
|
|
127
|
+
state under its home, and `cacheTokens` counts that whole occupancy, so a reused
|
|
128
|
+
home reports the same prompt as 152, 219 or 469 tokens. That also means a
|
|
129
|
+
device's total can include cache another chat left behind — a separate question
|
|
130
|
+
from this attribution, and the reason `conversation` is the remainder rather
|
|
131
|
+
than a reported number.
|
|
132
|
+
|
|
133
|
+
## When Agents Should Read This
|
|
134
|
+
|
|
135
|
+
Read this when changing the paths, workflows, runtime boundaries, or platform
|
|
136
|
+
behavior listed for this ADR in `docs/decisions/INDEX.md`.
|
|
@@ -0,0 +1,38 @@
|
|
|
1
|
+
# Harness Architecture Decision Records
|
|
2
|
+
|
|
3
|
+
Decision records for `@qvac/harness`: the inference engine and the tool runtime
|
|
4
|
+
skills execute in. They answer HOW we chose to build something, when the reason
|
|
5
|
+
is not obvious from the code or the PR that introduced it.
|
|
6
|
+
|
|
7
|
+
Read this index first. Open a record only when the touched paths, area, or
|
|
8
|
+
keywords match the task at hand.
|
|
9
|
+
|
|
10
|
+
Paths are relative to this repository. A path prefixed `@qvac/core →`,
|
|
11
|
+
`@qvac/sdk →` or `qvac-app →` lives in another repository and is context, not a
|
|
12
|
+
local file. `node scripts/docs-audit.cjs` checks this directory.
|
|
13
|
+
|
|
14
|
+
## Active Decisions
|
|
15
|
+
|
|
16
|
+
| Status | Area | Paths | Decision | Keywords |
|
|
17
|
+
| ------ | ------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
18
|
+
| active | assistant, harness, skills, security | `lib/tools/operations/manifest.ts`, `lib/tools/operations/compile.ts`, `lib/tools/operations/builders.ts`, `lib/tools/http-request.ts`, `lib/tools/http-request/truncate-response.ts`, `lib/capabilities/tool-selection.ts`, `skills/gmail/operations.json`, `skills/google-calendar/operations.json` | [A provider write is a compiled skill operation, not a shaping hook in `http_request`](./2026-09-01-skill-operation-manifests.md) | `operations.json`, `compileOperationTools`, `parseOperationsManifest`, `OperationSpec`, `BUILDERS`, `gmail_send`, `gmail_draft`, `calendar_create_meet_event`, `calendar_add_meet`, `owningSkill`, `grantedOwners`, `host shaping`, `shapeRequest`, `truncateResponseBody`, `fitJson`, `MAX_RESPONSE_CHARS` |
|
|
19
|
+
| active | assistant, harness, skills, security | `lib/tools/skills/overlay-dir.ts`, `lib/tools/skills/catalog.ts`, `lib/tools/skills/credential-grants.ts`, `lib/tools/skill.ts` | [A user-added skill layers over the bundled base and is trusted only by shadow](./2026-08-27-writable-skills-overlay.md) | `skills overlay`, `writable overlay`, `resolveSkillsOverlayDir`, `loadSkillCatalog`, `mergeOverlay`, `origin overlay`, `shadow`, `credentialGrants`, `mcpReadMethods`, `OVERLAY_DEFAULT_PLATFORMS`, `platform gate`, `untrusted skill`, `user-added skill`, `not available on this device` |
|
|
20
|
+
| active | assistant, harness, security, product | `lib/tools/exec.ts`, `lib/tools/exec/scope.ts`, `lib/capabilities/completion.ts`, `lib/capabilities/tool-selection.ts`, `lib/tools/skill.ts` | [A skill no longer decides what `exec` may run; approval does](./2026-08-26-exec-is-not-gated-by-skills.md) | `exec`, `skillGated`, `scopeRequired`, `grantingSkills`, `grantedScopes`, `toolScopes`, `checkScopedCommand`, `checkPythonScope`, `sanitizeCommand`, `locked until you load the skill`, `binary allowlist`, `RESTRICTED`, `approval is the boundary`, `PERM-7`, `PERM-10` |
|
|
21
|
+
| active | assistant, harness, runtime, inference | `lib/capabilities/prompt-prefix.ts`, `lib/capabilities/completion.ts`, `lib/tools/skills/mention.ts` | [A chat's prompt prefix changes only when the user changes something](./2026-08-20-chat-prompt-prefix-is-stable.md) | `promptPrefixHash`, `prompt prefix`, `round start`, `cacheTokens collapse`, `CACHE_INIT`, `KV cache re-prime`, `skillIsActive`, `SkillActivation`, `activeSkills`, `hasActiveSkills`, `loadedSkillsFrom`, `skillBlocked`, `blockedSkillMessage`, `skills section`, `turn-scoped activation`, `SKILL-3` |
|
|
22
|
+
| active | assistant, harness, runtime | `lib/hooks/**`, `lib/hooks/built-in/arg-repair.ts`, `lib/capabilities/completion.ts` | [Harness hooks fold a whole round before any call runs, and fail closed](./2026-08-17-harness-hook-pipeline.md) | `HarnessHook`, `HookPipeline`, `harnessHooks`, `runHooks`, `preToolCall`, `postToolCall`, `runStart`, `runEnd`, `HookRejection`, `guidance`, `round fold`, `deep clone`, `fail closed`, `tool executed; output withheld`, `abandoned`, `ArgRepairHook`, `arg repair`, `jsonrepair`, `parameters` |
|
|
23
|
+
| active | assistant, storage, desktop, mobile | `@qvac/core → lib/storage.ts`, `@qvac/core → lib/image-cache-dir.ts`, `@qvac/core → lib/sdk-storage.ts`, `qvac-app → desktop/electron/main.ts`, `qvac-app → mobile/worker/assistant-worklet.mjs` | [Assistant-owned storage separates shared models from private KV](./2026-07-29-assistant-owned-storage-layout.md) | `bare-storage`, `models`, `~/.qvac/models`, `SDK shared cache`, `image cache`, `KV cache`, `sdk-home`, `release channel`, `QVAC_MODEL_CACHE_PATH`, `Bare.argv[0]`, `SNAP_USER_COMMON`, `no migration` |
|
|
24
|
+
| active | runtime, inference, harness | `lib/models/model-pool.ts`, `lib/context-window.ts`, `@qvac/core → lib/assistant.ts`, `lib/capabilities/completion.ts` | [Static tool placement, no sliding window](./2026-08-20-kvcache-static-placement.md) | `kvCache`, `prompt cache`, `static placement`, `toolsMode`, `n_discarded`, `sliding window`, `savedCount`, `cold prime`, `modelId re-key`, `cache reset`, `overflow ladder` |
|
|
25
|
+
| active | runtime, inference, mobile, harness | `qvac-app → mobile/lib/android-llm-device.ts`, `qvac-app → mobile/lib/core-v2-harness-config.ts`, `index.ts`, `lib/models/kv-cache.ts`, `lib/models/opencl-cache.ts` | [Android LLM defaults to CPU except Tensor-era Pixels](./2026-06-23-android-cpu-llm-defaults.md) | `Android`, `Pixel 9`, `Pixel 10`, `Pixel Tablet`, `Pixel Fold`, `Adreno`, `Vulkan`, `OpenCL`, `openclCacheDir`, `kernel cache`, `backendDevice`, `Snapdragon`, `CPU inference`, `GPU inference`, `KV cache`, `q4_0`, `cache-type-k`, `cache-type-v`, `gpu_layers`, `transcriptionDevice`, `parakeet`, `useGPU`, `empty transcript` |
|
|
26
|
+
| active | runtime, harness, mobile, desktop | `lib/tools/exec/adapters/python/**`, `lib/tools/exec/scope.ts` | [Pyodide runs in a worker thread with a shared interrupt buffer](./2026-08-11-pyodide-worker-thread.md) | `Pyodide`, `python worker`, `Bare.Thread`, `bare-channel`, `SharedArrayBuffer`, `setInterruptBuffer`, `KeyboardInterrupt`, `terminate`, `join`, `stowed bundle`, `QVAC_PYODIDE_WORKER_ENTRY`, `QVAC_PYODIDE_TEST_LOADER`, `exec`, `in-process fallback` |
|
|
27
|
+
| active | runtime, harness, desktop, build, security | `wheels/manifest.json`, `lib/tools/exec/adapters/python/wheels.ts`, `scripts/vendor-python-wheels.mjs`, `qvac-app → desktop/electron.vite.config.ts` | [Python packages ship with the app, pinned by hash](./2026-08-21-vendored-python-wheels.md) | `python wheels`, `micropip`, `packageCacheDir`, `emfs:`, `deps=False`, `vendored wheels`, `offline`, `PyPI`, `sha256`, `supply chain`, `QVAC_PYTHON_WHEELS_DIR`, `fpdf2`, `pypdf`, `openpyxl`, `python-docx`, `python-pptx`, `cryptography` |
|
|
28
|
+
| active | desktop, harness, runtime | `qvac-app → desktop/electron/assistant.ts`, `qvac-app → desktop/electron/assistant-sidecar-lifecycle.ts`, `qvac-app → desktop/electron/assistant-sidecar-process.ts`, `worker/harness-sidecar.mjs` | [The host reaps the sidecar's process group; the harness cannot reap itself](./2026-08-04-sidecar-orphan-reaping.md) | `orphan`, `process group`, `detached`, `SIGKILL`, `reparent to init`, `taskkill`, `Job Object`, `libuv`, `KILL_ON_JOB_CLOSE`, `orphan watchdog`, `os.ppid`, `model memory leak`, `killStowChildren` |
|
|
29
|
+
| active | harness, security, runtime, mobile, desktop | `lib/hooks/built-in/egress.ts`, `lib/hooks/built-in/skill-catalog.ts`, `lib/hooks/registry.ts`, `lib/tools/url-guard.ts`, `lib/tools/private-ip.ts`, `lib/dns-lookup.ts`, `lib/validating-lookup.ts`, `lib/no-redirect-fetch-bare.ts`, `lib/no-redirect-fetch-node.ts`, `lib/helpers/spawn-path.ts`, `lib/tools/http-request.ts`, `lib/tools/mcp-call.ts`, `lib/tools/exec.ts`, `lib/tools/skill.ts`, `lib/tools/fs/scope.ts` | [Harness tool egress and spawn environments are allowlists, not filters](./2026-08-17-harness-outbound-egress-guard.md) | `SSRF`, `resolveOutboundUrl`, `private-ip`, `isPrivateOrBlocked`, `DNS rebinding`, `validating lookup`, `resolveHost`, `lookupHost`, `bare-dns`, `metadata endpoint`, `redirect hop`, `EXEC_ENV_ALLOWLIST`, `spawn env allowlist`, `workdir`, `resolveKey`, `granted roots`, `deniedRoots`, `canonicalDenyRoots`, `deny outranks grant`, `storage root`, `skill catalog`, `fail closed` |
|
|
30
|
+
| active | harness, app-runtime, desktop, mobile, chat | `lib/capabilities/usage-breakdown.ts`, `lib/capabilities/completion.ts`, `qvac-app → app-runtime/src/chat/usage-segment.ts`, `qvac-app → app-runtime/src/chat/chunk-to-turn.ts`, `qvac-app → app-runtime/src/agent/**`, `qvac-app → app-runtime/src/skill/skill.actor.catalog.ts` | [The device that ran the turn reports the Usage breakdown; the app only renders it](./2026-09-03-usage-breakdown-reported-by-the-harness.md) | `Usage modal`, `usage segments`, `usageBreakdown`, `promptUsageParts`, `buildUsageSegments`, `cacheTokens`, `charsPerToken`, `FRAMING_CHARS_BY_DIALECT`, `toolBlockFramingChars`, `detectToolDialect`, `chat template framing`, `PROMPT_SEND`, `context used`, `no tokenizer`, `metrics json` |
|
|
31
|
+
|
|
32
|
+
## Superseded Decisions
|
|
33
|
+
|
|
34
|
+
| Status | Area | Paths | Decision | Keywords |
|
|
35
|
+
| ---------- | ---- | ----- | ---------------------------------------------------------------------------------------------------------------------- | -------- |
|
|
36
|
+
| superseded | | | [KV-cache session is disabled for tool-using completions](./2026-05-29-kvcache-tool-turns.md) | |
|
|
37
|
+
| superseded | | | [KV-cache session is re-enabled for tool-using completions](./2026-06-02-kvcache-tool-turns-reenabled.md) | |
|
|
38
|
+
| superseded | | | [The Usage breakdown divides a measured total, it never estimates tokens](./2026-09-01-usage-breakdown-is-measured.md) | |
|
|
@@ -0,0 +1,32 @@
|
|
|
1
|
+
---
|
|
2
|
+
id: ADR-YYYY-MM-DD-short-name
|
|
3
|
+
status: active
|
|
4
|
+
introduced_by: PR-000
|
|
5
|
+
supersedes: []
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# Short Decision Title
|
|
9
|
+
|
|
10
|
+
## Decision
|
|
11
|
+
|
|
12
|
+
State the durable technical choice in one or two short paragraphs.
|
|
13
|
+
|
|
14
|
+
## Context
|
|
15
|
+
|
|
16
|
+
Explain the constraints or trade-offs that made this decision necessary.
|
|
17
|
+
|
|
18
|
+
## Alternatives Considered
|
|
19
|
+
|
|
20
|
+
Optional. Include only rejected approaches that future maintainers or agents
|
|
21
|
+
are likely to rediscover.
|
|
22
|
+
|
|
23
|
+
## Consequences
|
|
24
|
+
|
|
25
|
+
- What future changes must preserve.
|
|
26
|
+
- What this deliberately does not solve.
|
|
27
|
+
- What should trigger revisiting the decision.
|
|
28
|
+
|
|
29
|
+
## When Agents Should Read This
|
|
30
|
+
|
|
31
|
+
Read this when changing the paths, workflows, runtime boundaries, or platform
|
|
32
|
+
behavior listed for this ADR in `docs/decisions/INDEX.md`.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@qvac/harness",
|
|
3
|
-
"version": "0.2.
|
|
3
|
+
"version": "0.2.7",
|
|
4
4
|
"type": "module",
|
|
5
5
|
"description": "An AI harness implementing generate(context) via @qvac/sdk. Knows nothing about workbench, hyperdb, or chunks — just inference in, events out.",
|
|
6
6
|
"exports": {
|
|
@@ -8,6 +8,10 @@
|
|
|
8
8
|
"types": "./dist-lib/index.d.ts",
|
|
9
9
|
"default": "./dist-lib/index.js"
|
|
10
10
|
},
|
|
11
|
+
"./docx": {
|
|
12
|
+
"types": "./dist-lib/lib/docx/index.d.ts",
|
|
13
|
+
"default": "./dist-lib/lib/docx/index.js"
|
|
14
|
+
},
|
|
11
15
|
"./lib/dns-lookup.ts": {
|
|
12
16
|
"types": "./dist-lib/lib/dns-lookup.d.ts",
|
|
13
17
|
"default": "./dist-lib/lib/dns-lookup.js"
|
|
@@ -539,7 +543,7 @@
|
|
|
539
543
|
},
|
|
540
544
|
"scripts": {
|
|
541
545
|
"build:schema": "bare schema/build.ts",
|
|
542
|
-
"test": "npm run test:node && npm run test:bare && npm run test:addons",
|
|
546
|
+
"test": "npm run vendor:wheels && npm run test:node && npm run test:bare && npm run test:addons",
|
|
543
547
|
"test:node": "QVAC_NO_SKILL_BIN_PROBE=1 brittle-node test/all.ts | prettytap",
|
|
544
548
|
"test:bare": "QVAC_NO_SKILL_BIN_PROBE=1 brittle-bare test/all.ts | prettytap",
|
|
545
549
|
"format": "prettier . --write",
|