dsh-codex-style-computer-use 1.3.1 → 2.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (46) hide show
  1. package/README.ar.md +234 -0
  2. package/README.de.md +234 -0
  3. package/README.es.md +264 -0
  4. package/README.fa.md +234 -0
  5. package/README.fr.md +243 -0
  6. package/README.md +72 -17
  7. package/README.pt.md +234 -0
  8. package/README.ru.md +259 -0
  9. package/README.zh.md +66 -16
  10. package/icon.svg +20 -4
  11. package/lib/audit.js +7 -0
  12. package/lib/batch.js +14 -19
  13. package/lib/bin/attention-worker.exe +0 -0
  14. package/lib/bin/attention-worker.exe.json +1 -1
  15. package/lib/compact-feedback.js +175 -0
  16. package/lib/continuation.js +1 -1
  17. package/lib/core/bin/dsh-computer-use-worker.exe +0 -0
  18. package/lib/core/bin/dsh-computer-use-worker.exe.json +2 -2
  19. package/lib/core/index.js +403 -136
  20. package/lib/core/native-runtime.js +17 -0
  21. package/lib/core/worker.cs +2160 -229
  22. package/lib/feedback-routes.json +11 -0
  23. package/lib/feedback.js +49 -16
  24. package/lib/image-output.js +87 -17
  25. package/lib/index.js +25 -7
  26. package/lib/key-command.js +37 -0
  27. package/lib/launch.js +67 -0
  28. package/lib/media-contract.js +43 -0
  29. package/lib/observations.js +267 -13
  30. package/lib/policy.js +29 -9
  31. package/lib/tool-contract.js +56 -0
  32. package/lib/tools.js +203 -52
  33. package/lib/verification-region.js +17 -0
  34. package/mcp/package.json +2 -1
  35. package/mcp/src/execution.js +8 -5
  36. package/mcp/src/index.js +16 -267
  37. package/mcp/src/tools.js +271 -0
  38. package/package.json +6 -1
  39. package/scripts/check-behavior.mjs +83 -0
  40. package/scripts/measure-tool-tokens.mjs +21 -0
  41. package/scripts/verify-package.mjs +9 -0
  42. package/skills/computer-use/SKILL.md +55 -43
  43. package/skills/computer-use/references/lifecycle.md +11 -7
  44. package/skills/computer-use/references/maintenance.md +9 -5
  45. package/skills/computer-use/references/precision.md +39 -14
  46. package/skills/computer-use/references/progressive-control.md +10 -4
@@ -4,7 +4,7 @@ Read this when interpreting a delta, recovering a replaced UIA control, or diagn
4
4
 
5
5
  ## Tool visibility
6
6
 
7
- With progressiveTools enabled, idle Agents see computer_use_activate, computer_ctrl and computer_ask. A successful computer-use skill load, receipt acknowledgement or activation exposes the 18 existing tools to that Agent; the activation entry remains available too. Other Agents and children activate independently. The operating policy, skill receipt, cycle ownership and human brakes continue to govern execution. Capability activation itself performs no desktop call. A legacy host can set progressiveTools:false to retain the original global tool surface.
7
+ With progressiveTools enabled, idle Agents see computer_use_activate, computer_ctrl and computer_ask. A successful computer-use skill load, receipt acknowledgement or activation exposes the 19 desktop tools to that Agent; the activation entry remains available too. Other Agents and children activate independently. The operating policy, skill receipt, cycle ownership and human brakes continue to govern execution. Capability activation itself performs no desktop call. A legacy host can set progressiveTools:false to expose the desktop tools globally.
8
8
 
9
9
  ## Native target recovery
10
10
 
@@ -13,11 +13,11 @@ With progressiveTools enabled, idle Agents see computer_use_activate, computer_c
13
13
  2. Select the intended result using its visible label, context and task. Pass its opaque target to computer_uia_act {target:"<returned handle>",action:"invoke"}.
14
14
  3. If the native control was replaced while the task/context remains valid, rebind:true allows a new bounded scan. A unique AutomationId must retain the recorded semantics and ancestors. If that identifier disappeared, a unique name/role/class/pattern/ancestor match may be used. A changed meaning, duplicate identity, ambiguous semantics, incomplete scan, changed window title/geometry/process or expired handle refuses before action.
15
15
 
16
- Targets are scoped to this Agent's retained observations and the native worker's cache (512 identities; five minutes from observation). Handles cannot survive worker restart. No coordinates are guessed during UIA recovery. This feature applies to UIA pattern actions; M-mark clicks remain snapshot/geometry checked and still require refreshing after scrolling/navigation. A recovered control is not proof of the business outcome: inspect readback or query the relevant state.
16
+ Targets are scoped to this Agent's retained observations and the native worker's cache (512 identities; five minutes from observation). Worker replacement or human takeover invalidates actionable targets; retained historical snapshots can still be read with `targetsValid:false`. No coordinates are guessed during UIA recovery. This feature applies to UIA pattern actions; M-mark clicks remain snapshot/geometry checked and still require refreshing after scrolling/navigation. A recovered control is not proof of the business outcome: inspect readback or query the relevant state.
17
17
 
18
18
  ## Incremental queries and full retrieval
19
19
 
20
- The first query returns mode:"full" and observation:"O…". Repeat every original query parameter and add since:"O…" for a fresh comparison. Changes are returned in delta.elements for computer_uia, or delta.windows/delta.uiaFlat for computer_state.
20
+ When retention succeeds, the first query returns mode:"full" and observation:"O…". Repeat every original query parameter and add since:"O…" for a fresh comparison. Changes are returned in delta.elements for computer_uia, or delta.windows/delta.uiaFlat for computer_state. Resource pressure can return the current full value without an observation ID; do not invent a retrievable snapshot.
21
21
 
22
22
  - added and changed contain {key,element}, with the complete current element, not just a changed field.
23
23
  - removed contains identity keys. Delete those records; replace changed records; insert added records.
@@ -25,6 +25,12 @@ The first query returns mode:"full" and observation:"O…". Repeat every origina
25
25
  - An element key is its target handle when supplied, otherwise uia:<pid>:<runtimeId>. A window key is window:<pid>:<hwnd>.
26
26
  - Non-array metadata is always the current metadata. The next observation ID identifies the reconstructed complete state and can be the next since base.
27
27
 
28
- The result contains no hidden duplicate full element array. The plugin keeps the complete provider result internally for both full and diff outputs, including all returned fields. computer_uia {snapshot:"O…"} or computer_state {snapshot:"O…"} retrieves that exact retained full snapshot. This retrieval is historical and performs no capture. The bounded cache holds 64 observations per Agent; expired/missing snapshots report unavailable. Request a fresh query to replace them. A missing since base instead automatically returns a fresh full result with resetReason.
28
+ The result contains no hidden duplicate full element array. Retained snapshots preserve all provider-returned fields. `computer_uia {snapshot:"O…"}` or `computer_state {snapshot:"O…"}` retrieves the retained full value without another capture; it returns `snapshotHistorical:true` and the original `capturedAt`. Compact action feedback snapshots use `computer_state`. Explicit `offset` (nonnegative integer) and `limit` (1–80) page the main arrays: elements, uiaFlat, windows and ancestors. `page.fields` reports totals, returned counts and next offsets. A paged response omits the duplicate full `uiaRoot` and reports it in `page.omittedFields`; it does not mutate the cached snapshot. Omit pagination for full retrieval.
29
+
30
+ Each store indexes at most 64 observations and retains at most 4 MiB of JSON across Agents, with Agent-isolated access. Snapshots expire after 24 hours and may be evicted earlier. Opt-in archived feedback uses the dedicated `observation-cache` directory, limited to 64 managed snapshots and 32 MiB total, also with a 24-hour TTL. Lazy cleanup runs on saves and periodically on reads; it needs no persistent timer and handles old managed files after process restart. It only removes manifest-owned, verified files, never arbitrary files or symlinks. Expired/missing snapshots report unavailable; a missing `since` base instead returns a fresh full result with `resetReason`.
31
+
32
+ Disk failure returns `archive.status:"memory-only"` when a bounded in-memory snapshot remains available. Disk-full, unavailable storage and capacity failures do not repeat input. If memory is under pressure or the value cannot fit, the current full result is returned without a fake observation ID, with `archive.status:"uncached"`, `reason:"memory-pressure"`, `fallback:"inline"`. Existing disk history can still be explicitly read under pressure. Automatic compact feedback that cannot retain a retrievable snapshot returns its already-read raw evidence as `observation.inlineEvidence` and marks `feedback.detail:"full"`; this fallback is exempt from the normal 6 KiB extra-text budget and performs no extra query.
33
+
34
+ `computer_ctrl {action:"clear_cache"}` clears this Agent's retained observations and recorded owned cache files without opening a desktop cycle. It reports `cleared`, `filesRemoved`, `bytesFreed` and `retainedOnFailure`; an indeterminate deletion count is reported as unknown rather than success. It does not clear another Agent's cache, screenshots, attachments or user documents. Deletion failures prevent unbounded new disk writes when the quota is full.
29
35
 
30
36
  Truncated observations, missing/duplicate element identities and scope changes use a full response. The plugin also chooses full when a delta would be larger. This is a bounded observation of what the UIA provider returned; it does not claim that a truncated UIA tree contains every control in the application.