deadbyte-mcp 0.11.3 → 0.13.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/MANIFEST.SHA256 +126 -54
- package/R41-PARENT.json +15 -0
- package/README.txt +5 -4
- package/acceptance/r40/desktop-live-drift.mjs +106 -0
- package/acceptance/r40/desktop-live-ground.mjs +110 -0
- package/acceptance/r40/desktop-live-input.mjs +128 -0
- package/acceptance/r40/desktop-live-observe.mjs +100 -0
- package/acceptance/r40/desktop-live-uia.mjs +121 -0
- package/bin/appcontainer-stage.obj +0 -0
- package/bin/bootstrap-advapi32.obj +0 -0
- package/bin/bootstrap-exitcode.obj +0 -0
- package/bin/bootstrap-kernel32.obj +0 -0
- package/bin/child-control-probe.obj +0 -0
- package/bin/child-control-stage.obj +0 -0
- package/bin/contained-reverse-worker.obj +0 -0
- package/bin/contained-transform-worker.obj +0 -0
- package/bin/containment-probe.obj +0 -0
- package/bin/deadbyte-contain.obj +0 -0
- package/bin/deadbyte-desktop-input-r40.exe +0 -0
- package/bin/deadbyte-desktop-input-r40.obj +0 -0
- package/bin/deadbyte-desktop-observer-r40.exe +0 -0
- package/bin/deadbyte-desktop-observer-r40.obj +0 -0
- package/bin/deadbyte-desktop-uia-r40.exe +0 -0
- package/bin/deadbyte-desktop-uia-r40.obj +0 -0
- package/bin/deadbyte-exec.obj +0 -0
- package/bin/deadbyte-process-host.obj +0 -0
- package/bin/deadbyte-tunnel-host.obj +0 -0
- package/controller/README.TXT +7 -5
- package/controller/deadbyte-controller.config.psd1.example +3 -0
- package/controller/deadbyte-controller.ps1 +149 -4
- package/controller/deadbyte-desktop-policy.json +41 -0
- package/controller/deadbyte-process-policy.json +4 -4
- package/docs/ARCHITECTURE.md +6 -1
- package/docs/superpowers/specs/2026-09-19-r40-desktop-computer-use-design.md +214 -0
- package/docs/superpowers/specs/2026-09-20-r41-deterministic-memory-design.md +132 -0
- package/native/deadbyte-desktop-input.cpp +455 -0
- package/native/deadbyte-desktop-observer.cpp +253 -0
- package/native/deadbyte-desktop-uia.cpp +307 -0
- package/package.json +12 -8
- package/proof/CONTAINMENT-BUILD.txt +6 -6
- package/scripts/build-containment.ps1 +24 -14
- package/scripts/build-desktop-r40.ps1 +112 -0
- package/scripts/deferred-slot-operation-r40.mjs +126 -0
- package/scripts/deferred-slot-operation-r41.mjs +127 -0
- package/scripts/final-closure-r40.mjs +114 -0
- package/scripts/final-closure-r41.mjs +116 -0
- package/scripts/final-closure-verify-r40.mjs +212 -0
- package/scripts/final-closure-verify-r41.mjs +215 -0
- package/scripts/gate-windows-r40.ps1 +43 -0
- package/scripts/gate-windows-r41.ps1 +48 -0
- package/scripts/init-desktop-policy-r40.ps1 +85 -0
- package/scripts/r40-compact-ledger-probe.mjs +4 -0
- package/scripts/release-parity-r40.mjs +212 -0
- package/scripts/release-parity-r41.mjs +221 -0
- package/scripts/release-parity-tests-r40.ps1 +9 -0
- package/scripts/release-parity-tests-r41.ps1 +9 -0
- package/scripts/verify-memory-r41.mjs +88 -0
- package/scripts/verify-r41-parent.mjs +61 -0
- package/scripts/windows-gate-evidence-r40.mjs +52 -0
- package/scripts/windows-gate-evidence-r41.mjs +53 -0
- package/src/autonomous-context-engine.mjs +3 -1
- package/src/autonomous-planner-contracts.mjs +19 -1
- package/src/autonomous-planner.mjs +24 -0
- package/src/autonomous-runtime.mjs +164 -25
- package/src/capability-ledger.mjs +6 -1
- package/src/compact-gateway-mcp-tools.mjs +14 -2
- package/src/deadbyte-cli.mjs +72 -44
- package/src/desktop-contract.mjs +263 -0
- package/src/desktop-input-r40.mjs +148 -0
- package/src/desktop-mcp-tools-r40.mjs +126 -0
- package/src/desktop-memory-r40.mjs +272 -0
- package/src/desktop-observer-r40.mjs +231 -0
- package/src/desktop-policy-r40.mjs +110 -0
- package/src/desktop-runtime-r40.mjs +318 -0
- package/src/desktop-uia-r40.mjs +123 -0
- package/src/mcp-server.mjs +87 -19
- package/src/memory-r41.mjs +692 -0
- package/src/r40-compact-disarmed-probe.mjs +71 -0
- package/src/r40-full-surface-probe.mjs +34 -0
- package/src/release-generation.mjs +38 -5
- package/src/remote-console.mjs +41 -0
- package/src/remote-live-log.mjs +208 -45
- package/src/remote-mcp-desktop-r40-smoke-client.mjs +70 -0
- package/src/remote-mcp-r40-parity-client.mjs +200 -0
- package/src/remote-mcp-smoke-client.mjs +20 -6
- package/src/remote-result-journal.mjs +10 -2
- package/src/runtime-update.mjs +55 -17
- package/src/tool-audit.mjs +29 -2
- package/src/version.mjs +1 -1
- package/test/autonomous-capability-r36.test.mjs +43 -1
- package/test/autonomous-desktop-r40.test.mjs +188 -0
- package/test/capability-ledger-r36.test.mjs +1 -0
- package/test/cli-entrypoint.test.mjs +1 -1
- package/test/compact-catalog-r36.test.mjs +2 -2
- package/test/compact-gateway-r36.test.mjs +56 -4
- package/test/compact-max-catalog-r40.test.mjs +95 -0
- package/test/controller-desktop-r40.test.mjs +86 -0
- package/test/core.test.mjs +1 -1
- package/test/desktop-adversarial-r40.test.mjs +62 -0
- package/test/desktop-contract-r40.test.mjs +200 -0
- package/test/desktop-input-r40.test.mjs +63 -0
- package/test/desktop-mcp-tools-r40.test.mjs +61 -0
- package/test/desktop-memory-r40.test.mjs +141 -0
- package/test/desktop-observer-r40.test.mjs +119 -0
- package/test/desktop-policy-r40.test.mjs +111 -0
- package/test/desktop-runtime-r40.test.mjs +278 -0
- package/test/desktop-uia-r40.test.mjs +42 -0
- package/test/full-max-catalog-r40.test.mjs +91 -0
- package/test/helpers/autonomous-r34-fixture.mjs +3 -3
- package/test/memory-autonomous-integration-r41.test.mjs +31 -0
- package/test/memory-history-r41.test.mjs +126 -0
- package/test/memory-projections-r41.test.mjs +92 -0
- package/test/memory-retrieval-r41.test.mjs +124 -0
- package/test/memory-snapshot-r41.test.mjs +106 -0
- package/test/r33-closeout-regression.test.mjs +7 -7
- package/test/r33-finalization.test.mjs +15 -15
- package/test/r37-remote-tool-call-timeline.test.mjs +6 -5
- package/test/r40-finalization.test.mjs +118 -0
- package/test/r41-finalization.test.mjs +38 -0
- package/test/release-generation.test.mjs +41 -9
- package/test/release-version.test.mjs +20 -61
- package/test/remote-console-r40.test.mjs +38 -0
- package/test/remote-live-log-r37.test.mjs +89 -32
- package/test/remote-r24-catalog.test.mjs +12 -3
- package/test/remote-result-journal.test.mjs +15 -0
- package/test/remote-session-cli.test.mjs +69 -46
- package/test/runtime-self-update.test.mjs +44 -0
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
[CmdletBinding()]
|
|
3
3
|
param(
|
|
4
4
|
[Parameter(Position=0)]
|
|
5
|
-
[ValidateSet('menu','start','resume','park','stop','restart','status','url','test','fulltest','toggle','clean','machinetest','armprocess','disarmprocess','processstatus','armhost','disarmhost','hoststatus','hosttest','armcoding','disarmcoding','codingstatus','codingtest','armautonomous','disarmautonomous','autonomousstatus','autonomoustest','supervisorstart','supervisorstop','supervisorstatus','supervisorrecover')]
|
|
5
|
+
[ValidateSet('menu','start','resume','park','stop','restart','status','url','test','fulltest','toggle','clean','machinetest','armprocess','disarmprocess','processstatus','armhost','disarmhost','hoststatus','hosttest','armcoding','disarmcoding','codingstatus','codingtest','armautonomous','disarmautonomous','autonomousstatus','autonomoustest','supervisorstart','supervisorstop','supervisorstatus','supervisorrecover','armdesktop','disarmdesktop','desktopstatus')]
|
|
6
6
|
[string]$Command = 'menu'
|
|
7
7
|
)
|
|
8
8
|
|
|
@@ -197,6 +197,19 @@ function Get-Config {
|
|
|
197
197
|
}
|
|
198
198
|
if ([bool]$cfg.ObservationRuntimeEnabled -and -not [bool]$cfg.MachineRuntimeEnabled) { Fail 'ObservationRuntimeEnabled requires MachineRuntimeEnabled' }
|
|
199
199
|
if ([bool]$cfg.ObservationRuntimeEnabled -and -not [bool]$cfg.CodingRuntimeEnabled) { Fail 'ObservationRuntimeEnabled requires CodingRuntimeEnabled' }
|
|
200
|
+
if ($cfg.ContainsKey('DesktopRuntimeEnabled')) {
|
|
201
|
+
if ($cfg.DesktopRuntimeEnabled -isnot [bool]) { Fail 'DesktopRuntimeEnabled must be boolean' }
|
|
202
|
+
} else {
|
|
203
|
+
$cfg['DesktopRuntimeEnabled'] = $false
|
|
204
|
+
}
|
|
205
|
+
if ($cfg.ContainsKey('DesktopPolicyFile') -and -not [string]::IsNullOrWhiteSpace([string]$cfg.DesktopPolicyFile)) {
|
|
206
|
+
$desktopPolicy = [System.IO.Path]::GetFullPath([string]$cfg.DesktopPolicyFile)
|
|
207
|
+
if (-not (Test-Path -LiteralPath $desktopPolicy -PathType Leaf)) { Fail "DesktopPolicyFile missing: $desktopPolicy" }
|
|
208
|
+
$cfg['DesktopPolicyFile'] = $desktopPolicy
|
|
209
|
+
} else {
|
|
210
|
+
$cfg['DesktopPolicyFile'] = ''
|
|
211
|
+
}
|
|
212
|
+
if ([bool]$cfg.DesktopRuntimeEnabled -and [string]::IsNullOrWhiteSpace([string]$cfg.DesktopPolicyFile)) { Fail 'DesktopRuntimeEnabled requires DesktopPolicyFile' }
|
|
200
213
|
if ($cfg.ContainsKey('AutonomousRuntimeEnabled')) {
|
|
201
214
|
if ($cfg.AutonomousRuntimeEnabled -isnot [bool]) { Fail 'AutonomousRuntimeEnabled must be boolean' }
|
|
202
215
|
} else {
|
|
@@ -420,6 +433,10 @@ function Start-OwnedComponent([string]$Name, $Spec, $Cfg = $null) {
|
|
|
420
433
|
$previousMachineFsRuntimeEnv = $env:DEADBYTE_MACHINE_FS_RUNTIME
|
|
421
434
|
$hadObservationRuntimeEnv = Test-Path Env:DEADBYTE_OBSERVATION_RUNTIME
|
|
422
435
|
$previousObservationRuntimeEnv = $env:DEADBYTE_OBSERVATION_RUNTIME
|
|
436
|
+
$hadDesktopRuntimeEnv = Test-Path Env:DEADBYTE_DESKTOP_RUNTIME
|
|
437
|
+
$previousDesktopRuntimeEnv = $env:DEADBYTE_DESKTOP_RUNTIME
|
|
438
|
+
$hadDesktopPolicyEnv = Test-Path Env:DEADBYTE_DESKTOP_POLICY
|
|
439
|
+
$previousDesktopPolicyEnv = $env:DEADBYTE_DESKTOP_POLICY
|
|
423
440
|
$hadCodingPolicyEnv = Test-Path Env:DEADBYTE_CODING_POLICY
|
|
424
441
|
$previousCodingPolicyEnv = $env:DEADBYTE_CODING_POLICY
|
|
425
442
|
$hadSemanticConfigEnv = Test-Path Env:DEADBYTE_SEMANTIC_LSP_CONFIG
|
|
@@ -453,6 +470,12 @@ function Start-OwnedComponent([string]$Name, $Spec, $Cfg = $null) {
|
|
|
453
470
|
$env:DEADBYTE_CODING_RUNTIME = if ([bool]$Cfg.CodingRuntimeEnabled) { '1' } else { '0' }
|
|
454
471
|
$env:DEADBYTE_MACHINE_FS_RUNTIME = if ([bool]$Cfg.MachineFsRuntimeEnabled) { '1' } else { '0' }
|
|
455
472
|
$env:DEADBYTE_OBSERVATION_RUNTIME = if ([bool]$Cfg.ObservationRuntimeEnabled) { '1' } else { '0' }
|
|
473
|
+
$env:DEADBYTE_DESKTOP_RUNTIME = if ([bool]$Cfg.DesktopRuntimeEnabled) { '1' } else { '0' }
|
|
474
|
+
if (-not [string]::IsNullOrWhiteSpace([string]$Cfg.DesktopPolicyFile)) {
|
|
475
|
+
$env:DEADBYTE_DESKTOP_POLICY = [string]$Cfg.DesktopPolicyFile
|
|
476
|
+
} else {
|
|
477
|
+
Remove-Item Env:DEADBYTE_DESKTOP_POLICY -ErrorAction SilentlyContinue
|
|
478
|
+
}
|
|
456
479
|
if (-not [string]::IsNullOrWhiteSpace([string]$Cfg.CodingPolicyFile)) {
|
|
457
480
|
$env:DEADBYTE_CODING_POLICY = [string]$Cfg.CodingPolicyFile
|
|
458
481
|
}
|
|
@@ -481,6 +504,8 @@ function Start-OwnedComponent([string]$Name, $Spec, $Cfg = $null) {
|
|
|
481
504
|
Remove-Item Env:DEADBYTE_PROCESS_POLICY -ErrorAction SilentlyContinue
|
|
482
505
|
Remove-Item Env:DEADBYTE_MACHINE_FS_RUNTIME -ErrorAction SilentlyContinue
|
|
483
506
|
Remove-Item Env:DEADBYTE_OBSERVATION_RUNTIME -ErrorAction SilentlyContinue
|
|
507
|
+
Remove-Item Env:DEADBYTE_DESKTOP_RUNTIME -ErrorAction SilentlyContinue
|
|
508
|
+
Remove-Item Env:DEADBYTE_DESKTOP_POLICY -ErrorAction SilentlyContinue
|
|
484
509
|
Remove-Item Env:DEADBYTE_SEMANTIC_LSP_CONFIG -ErrorAction SilentlyContinue
|
|
485
510
|
Remove-Item Env:DEADBYTE_SEMANTIC_LSP_CONFIG_SHA256 -ErrorAction SilentlyContinue
|
|
486
511
|
Remove-Item Env:DEADBYTE_AUTONOMOUS_SUPERVISOR_MODE -ErrorAction SilentlyContinue
|
|
@@ -518,6 +543,10 @@ function Start-OwnedComponent([string]$Name, $Spec, $Cfg = $null) {
|
|
|
518
543
|
else { Remove-Item Env:DEADBYTE_MACHINE_FS_RUNTIME -ErrorAction SilentlyContinue }
|
|
519
544
|
if ($hadObservationRuntimeEnv) { $env:DEADBYTE_OBSERVATION_RUNTIME = $previousObservationRuntimeEnv }
|
|
520
545
|
else { Remove-Item Env:DEADBYTE_OBSERVATION_RUNTIME -ErrorAction SilentlyContinue }
|
|
546
|
+
if ($hadDesktopRuntimeEnv) { $env:DEADBYTE_DESKTOP_RUNTIME = $previousDesktopRuntimeEnv }
|
|
547
|
+
else { Remove-Item Env:DEADBYTE_DESKTOP_RUNTIME -ErrorAction SilentlyContinue }
|
|
548
|
+
if ($hadDesktopPolicyEnv) { $env:DEADBYTE_DESKTOP_POLICY = $previousDesktopPolicyEnv }
|
|
549
|
+
else { Remove-Item Env:DEADBYTE_DESKTOP_POLICY -ErrorAction SilentlyContinue }
|
|
521
550
|
if ($hadCodingPolicyEnv) { $env:DEADBYTE_CODING_POLICY = $previousCodingPolicyEnv }
|
|
522
551
|
else { Remove-Item Env:DEADBYTE_CODING_POLICY -ErrorAction SilentlyContinue }
|
|
523
552
|
if ($hadSemanticConfigEnv) { $env:DEADBYTE_SEMANTIC_LSP_CONFIG = $previousSemanticLspConfigEnv }
|
|
@@ -737,6 +766,7 @@ function Start-Deadbyte($Cfg) {
|
|
|
737
766
|
}
|
|
738
767
|
|
|
739
768
|
function Park-Deadbyte($Cfg) {
|
|
769
|
+
Disarm-Desktop $Cfg -Quiet
|
|
740
770
|
Stop-OwnedComponent 'Bridge'
|
|
741
771
|
$s = Get-ObservedState $Cfg
|
|
742
772
|
if ($s.State -notin @('PARKED','OFFLINE')) { Fail "park gate failed: observed $($s.State)" }
|
|
@@ -744,6 +774,7 @@ function Park-Deadbyte($Cfg) {
|
|
|
744
774
|
}
|
|
745
775
|
|
|
746
776
|
function Stop-Deadbyte($Cfg) {
|
|
777
|
+
Disarm-Desktop $Cfg -Quiet
|
|
747
778
|
Stop-OwnedComponent 'Bridge'
|
|
748
779
|
Stop-OwnedComponent 'Cloud'
|
|
749
780
|
Remove-Item -LiteralPath (Join-Path $RuntimeDir 'current-origin.txt') -Force -ErrorAction SilentlyContinue
|
|
@@ -992,6 +1023,107 @@ function Show-AutonomousStatus($Cfg) {
|
|
|
992
1023
|
return $a
|
|
993
1024
|
}
|
|
994
1025
|
|
|
1026
|
+
function Get-DesktopPolicyMetadata($Cfg) {
|
|
1027
|
+
if ([string]::IsNullOrWhiteSpace([string]$Cfg.DesktopPolicyFile)) { Fail 'desktop plane is not configured' }
|
|
1028
|
+
$policyPath = [string]$Cfg.DesktopPolicyFile
|
|
1029
|
+
try { $policy = Get-Content -LiteralPath $policyPath -Raw -Encoding UTF8 | ConvertFrom-Json } catch { Fail "invalid desktop policy JSON: $policyPath" }
|
|
1030
|
+
if ($policy.schema -ne 'deadbyte.desktop-policy.v1' -or $policy.enabled -ne $true) { Fail 'desktop policy schema/enabled mismatch' }
|
|
1031
|
+
if ([string]::IsNullOrWhiteSpace([string]$policy.grant_file)) { Fail 'desktop policy grant_file missing' }
|
|
1032
|
+
if ([string]::IsNullOrWhiteSpace([string]$policy.evidence_dir)) { Fail 'desktop policy evidence_dir missing' }
|
|
1033
|
+
$grantFile = [System.IO.Path]::GetFullPath([string]$policy.grant_file)
|
|
1034
|
+
$evidenceDir = [System.IO.Path]::GetFullPath([string]$policy.evidence_dir)
|
|
1035
|
+
$runtimeReal = [System.IO.Path]::GetFullPath($RuntimeDir).TrimEnd('\') + '\'
|
|
1036
|
+
if (-not $grantFile.StartsWith($runtimeReal, [System.StringComparison]::OrdinalIgnoreCase)) { Fail 'desktop policy grant_file must live under controller runtime directory' }
|
|
1037
|
+
if (-not ($evidenceDir.TrimEnd('\') + '\').StartsWith($runtimeReal, [System.StringComparison]::OrdinalIgnoreCase)) { Fail 'desktop policy evidence_dir must live under controller runtime directory' }
|
|
1038
|
+
$knownActions = @('pointer_move','click_primary','click_secondary','double_click_primary','scroll','key_chord','text_input')
|
|
1039
|
+
$actionKinds = @($policy.allowed_actions)
|
|
1040
|
+
$uniqueActions = @($actionKinds | Sort-Object -Unique)
|
|
1041
|
+
if ($actionKinds.Count -lt 1 -or $actionKinds.Count -ne $uniqueActions.Count -or @($actionKinds | Where-Object { $_ -notin $knownActions }).Count -gt 0) { Fail 'desktop policy allowed_actions invalid' }
|
|
1042
|
+
if (-not $policy.observer -or [string]::IsNullOrWhiteSpace([string]$policy.observer.executable) -or [string]$policy.observer.sha256 -cnotmatch '^[0-9a-f]{64}$') { Fail 'desktop observer metadata invalid' }
|
|
1043
|
+
if (-not $policy.input -or [string]::IsNullOrWhiteSpace([string]$policy.input.executable) -or [string]$policy.input.sha256 -cnotmatch '^[0-9a-f]{64}$') { Fail 'desktop input metadata invalid' }
|
|
1044
|
+
if (-not $policy.grounder -or [string]::IsNullOrWhiteSpace([string]$policy.grounder.executable) -or [string]$policy.grounder.sha256 -cnotmatch '^[0-9a-f]{64}$') { Fail 'desktop grounder metadata invalid' }
|
|
1045
|
+
$observerPath = [System.IO.Path]::GetFullPath([string]$policy.observer.executable)
|
|
1046
|
+
$inputPath = [System.IO.Path]::GetFullPath([string]$policy.input.executable)
|
|
1047
|
+
$grounderPath = [System.IO.Path]::GetFullPath([string]$policy.grounder.executable)
|
|
1048
|
+
foreach ($provider in @(
|
|
1049
|
+
[pscustomobject]@{ Name='observer'; Path=$observerPath; Sha256=([string]$policy.observer.sha256).ToLowerInvariant() },
|
|
1050
|
+
[pscustomobject]@{ Name='input'; Path=$inputPath; Sha256=([string]$policy.input.sha256).ToLowerInvariant() },
|
|
1051
|
+
[pscustomobject]@{ Name='grounder'; Path=$grounderPath; Sha256=([string]$policy.grounder.sha256).ToLowerInvariant() }
|
|
1052
|
+
)) {
|
|
1053
|
+
if (-not (Test-Path -LiteralPath $provider.Path -PathType Leaf)) { Fail "desktop $($provider.Name) executable missing: $($provider.Path)" }
|
|
1054
|
+
$item = Get-Item -LiteralPath $provider.Path -Force
|
|
1055
|
+
if (($item.Attributes -band [System.IO.FileAttributes]::ReparsePoint) -ne 0) { Fail "desktop $($provider.Name) executable must not be a reparse point" }
|
|
1056
|
+
$actual = Get-ExecutableHash $provider.Path
|
|
1057
|
+
if ($actual -ne $provider.Sha256) { Fail "desktop $($provider.Name) executable SHA-256 mismatch: expected $($provider.Sha256), found $actual" }
|
|
1058
|
+
}
|
|
1059
|
+
[pscustomobject]@{
|
|
1060
|
+
Path=$policyPath; Sha256=Get-ExecutableHash $policyPath; GrantFile=$grantFile; EvidenceDir=$evidenceDir;
|
|
1061
|
+
ActionKinds=$actionKinds; ObserverPath=$observerPath; ObserverSha256=([string]$policy.observer.sha256).ToLowerInvariant();
|
|
1062
|
+
InputPath=$inputPath; InputSha256=([string]$policy.input.sha256).ToLowerInvariant();
|
|
1063
|
+
GrounderPath=$grounderPath; GrounderSha256=([string]$policy.grounder.sha256).ToLowerInvariant()
|
|
1064
|
+
}
|
|
1065
|
+
}
|
|
1066
|
+
|
|
1067
|
+
function Get-DesktopGrantFileForRevoke($Cfg) {
|
|
1068
|
+
if ([string]::IsNullOrWhiteSpace([string]$Cfg.DesktopPolicyFile)) { return '' }
|
|
1069
|
+
$policyPath = [string]$Cfg.DesktopPolicyFile
|
|
1070
|
+
try { $policy = Get-Content -LiteralPath $policyPath -Raw -Encoding UTF8 | ConvertFrom-Json } catch { Fail "invalid desktop policy JSON during revoke: $policyPath" }
|
|
1071
|
+
if ([string]::IsNullOrWhiteSpace([string]$policy.grant_file)) { Fail 'desktop policy grant_file missing during revoke' }
|
|
1072
|
+
$grantFile = [System.IO.Path]::GetFullPath([string]$policy.grant_file)
|
|
1073
|
+
$runtimeReal = [System.IO.Path]::GetFullPath($RuntimeDir).TrimEnd('\') + '\'
|
|
1074
|
+
if (-not $grantFile.StartsWith($runtimeReal, [System.StringComparison]::OrdinalIgnoreCase)) { Fail 'desktop policy grant_file must live under controller runtime directory' }
|
|
1075
|
+
return $grantFile
|
|
1076
|
+
}
|
|
1077
|
+
|
|
1078
|
+
function Get-DesktopGrantStatus($Cfg) {
|
|
1079
|
+
if ([string]::IsNullOrWhiteSpace([string]$Cfg.DesktopPolicyFile)) { return [pscustomobject]@{ Configured=$false; Armed=$false; Reason='no-policy'; ActionKinds=@() } }
|
|
1080
|
+
try { $meta = Get-DesktopPolicyMetadata $Cfg } catch { return [pscustomobject]@{ Configured=$true; Armed=$false; Reason=$_.Exception.Message; ActionKinds=@() } }
|
|
1081
|
+
if (-not (Test-Path -LiteralPath $meta.GrantFile -PathType Leaf)) { return [pscustomobject]@{ Configured=$true; Armed=$false; Reason='disarmed'; ActionKinds=@() } }
|
|
1082
|
+
try { $grant = Get-Content -LiteralPath $meta.GrantFile -Raw -Encoding UTF8 | ConvertFrom-Json } catch { return [pscustomobject]@{ Configured=$true; Armed=$false; Reason='invalid-grant'; ActionKinds=@() } }
|
|
1083
|
+
if ($grant.schema -ne 'deadbyte.desktop-grant.v1') { return [pscustomobject]@{ Configured=$true; Armed=$false; Reason='grant-schema'; ActionKinds=@() } }
|
|
1084
|
+
if ([string]$grant.policy_sha256 -ne [string]$meta.Sha256) { return [pscustomobject]@{ Configured=$true; Armed=$false; Reason='policy-digest'; ActionKinds=@() } }
|
|
1085
|
+
if ([string]$grant.instance_nonce -cnotmatch '^[0-9a-f]{64}$') { return [pscustomobject]@{ Configured=$true; Armed=$false; Reason='nonce-invalid'; ActionKinds=@() } }
|
|
1086
|
+
$actions = @($grant.action_kinds)
|
|
1087
|
+
$unique = @($actions | Sort-Object -Unique)
|
|
1088
|
+
if ($actions.Count -lt 1 -or $actions.Count -ne $unique.Count) { return [pscustomobject]@{ Configured=$true; Armed=$false; Reason='actions-invalid'; ActionKinds=@() } }
|
|
1089
|
+
if (@($actions | Where-Object { $_ -notin $meta.ActionKinds }).Count -gt 0) { return [pscustomobject]@{ Configured=$true; Armed=$false; Reason='actions-outside-policy'; ActionKinds=@() } }
|
|
1090
|
+
return [pscustomobject]@{ Configured=$true; Armed=$true; Reason='standing-grant'; ActionKinds=$actions }
|
|
1091
|
+
}
|
|
1092
|
+
|
|
1093
|
+
function Arm-Desktop($Cfg) {
|
|
1094
|
+
$meta = Get-DesktopPolicyMetadata $Cfg
|
|
1095
|
+
Ensure-Directory $RuntimeDir
|
|
1096
|
+
Ensure-Directory $meta.EvidenceDir
|
|
1097
|
+
$nonceBytes = New-Object byte[] 32
|
|
1098
|
+
$rng = [System.Security.Cryptography.RandomNumberGenerator]::Create()
|
|
1099
|
+
try { $rng.GetBytes($nonceBytes) } finally { $rng.Dispose() }
|
|
1100
|
+
$nonce = -join ($nonceBytes | ForEach-Object { $_.ToString('x2') })
|
|
1101
|
+
$body = [ordered]@{
|
|
1102
|
+
schema = 'deadbyte.desktop-grant.v1'
|
|
1103
|
+
policy_sha256 = [string]$meta.Sha256
|
|
1104
|
+
instance_nonce = $nonce
|
|
1105
|
+
action_kinds = @($meta.ActionKinds)
|
|
1106
|
+
granted_at_utc = [DateTime]::UtcNow.ToString('o')
|
|
1107
|
+
}
|
|
1108
|
+
$tmp = $meta.GrantFile + '.tmp'
|
|
1109
|
+
$body | ConvertTo-Json -Depth 6 | Set-Content -LiteralPath $tmp -Encoding UTF8
|
|
1110
|
+
Move-Item -LiteralPath $tmp -Destination $meta.GrantFile -Force
|
|
1111
|
+
Write-Host ("DESKTOP PLANE ENABLED UNTIL EXPLICIT REVOKE [{0}]" -f ($meta.ActionKinds -join ','))
|
|
1112
|
+
}
|
|
1113
|
+
|
|
1114
|
+
function Disarm-Desktop($Cfg, [switch]$Quiet) {
|
|
1115
|
+
$grantFile = Get-DesktopGrantFileForRevoke $Cfg
|
|
1116
|
+
if ([string]::IsNullOrWhiteSpace([string]$grantFile)) { return }
|
|
1117
|
+
Remove-Item -LiteralPath $grantFile -Force -ErrorAction SilentlyContinue
|
|
1118
|
+
if (-not $Quiet) { Write-Host 'DESKTOP PLANE DISARMED' }
|
|
1119
|
+
}
|
|
1120
|
+
|
|
1121
|
+
function Show-DesktopStatus($Cfg) {
|
|
1122
|
+
$d = Get-DesktopGrantStatus $Cfg
|
|
1123
|
+
Write-Host ("Desktop plane : {0} [{1}]" -f $(if ($d.Armed) { 'ARMED' } elseif ($d.Configured) { 'DISARMED' } else { 'UNCONFIGURED' }), $d.Reason)
|
|
1124
|
+
return $d
|
|
1125
|
+
}
|
|
1126
|
+
|
|
995
1127
|
function Get-AutonomousSupervisorStatus($Cfg) {
|
|
996
1128
|
if (-not [bool]$Cfg.AutonomousSupervisorEnabled) { return [pscustomobject]@{ Configured=$false; Running=$false; Reason='disabled'; Owned=$null } }
|
|
997
1129
|
$owned = Test-OwnedInstance 'Supervisor'
|
|
@@ -1022,6 +1154,7 @@ function Show-AutonomousSupervisorStatus($Cfg) {
|
|
|
1022
1154
|
|
|
1023
1155
|
function Recover-AutonomousSupervisor($Cfg) {
|
|
1024
1156
|
if (-not [bool]$Cfg.AutonomousSupervisorEnabled) { Fail 'autonomous supervisor recovery requires AutonomousSupervisorEnabled' }
|
|
1157
|
+
Disarm-Desktop $Cfg -Quiet
|
|
1025
1158
|
Start-Deadbyte $Cfg | Out-Null
|
|
1026
1159
|
try { Disarm-Autonomous $Cfg -Quiet } catch {}
|
|
1027
1160
|
try { Disarm-Coding $Cfg } catch {}
|
|
@@ -1032,11 +1165,12 @@ function Recover-AutonomousSupervisor($Cfg) {
|
|
|
1032
1165
|
$process = Get-ProcessGrantStatus $Cfg
|
|
1033
1166
|
$coding = Get-CodingLeaseStatus $Cfg
|
|
1034
1167
|
$autonomous = Get-AutonomousLeaseStatus $Cfg
|
|
1168
|
+
$desktop = Get-DesktopGrantStatus $Cfg
|
|
1035
1169
|
$hostStatus = Get-HostArmStatus $Cfg
|
|
1036
|
-
if ($process.Armed -or $coding.Armed -or $autonomous.Armed -or $hostStatus.Armed) { Fail 'supervisor recovery final authority is not fully disarmed' }
|
|
1170
|
+
if ($process.Armed -or $coding.Armed -or $autonomous.Armed -or $desktop.Armed -or $hostStatus.Armed) { Fail 'supervisor recovery final authority is not fully disarmed' }
|
|
1037
1171
|
$supervisor = Start-AutonomousSupervisor $Cfg
|
|
1038
1172
|
Write-Host 'AUTONOMOUS SUPERVISOR RECOVERY PASS: runtime ONLINE; privileged planes DISARMED.'
|
|
1039
|
-
return [pscustomobject]@{ Runtime=$runtime; Supervisor=$supervisor; Process=$process; Coding=$coding; Autonomous=$autonomous; Host=$hostStatus }
|
|
1173
|
+
return [pscustomobject]@{ Runtime=$runtime; Supervisor=$supervisor; Process=$process; Coding=$coding; Autonomous=$autonomous; Desktop=$desktop; Host=$hostStatus }
|
|
1040
1174
|
}
|
|
1041
1175
|
|
|
1042
1176
|
function Show-State($Cfg) {
|
|
@@ -1055,11 +1189,13 @@ function Show-State($Cfg) {
|
|
|
1055
1189
|
Write-Host ("Coding runtime: {0}" -f $(if ([bool]$Cfg.CodingRuntimeEnabled) { 'ENABLED' } else { 'DISABLED' }))
|
|
1056
1190
|
Write-Host ("Machine FS rt : {0}" -f $(if ([bool]$Cfg.MachineFsRuntimeEnabled) { 'ENABLED' } else { 'DISABLED' }))
|
|
1057
1191
|
Write-Host ("Observation rt: {0}" -f $(if ([bool]$Cfg.ObservationRuntimeEnabled) { 'ENABLED' } else { 'DISABLED' }))
|
|
1192
|
+
Write-Host ("Desktop runtime: {0}" -f $(if ([bool]$Cfg.DesktopRuntimeEnabled) { 'ENABLED' } else { 'DISABLED' }))
|
|
1058
1193
|
Write-Host ("Autonomous rt : {0}" -f $(if ([bool]$Cfg.AutonomousRuntimeEnabled) { 'ENABLED' } else { 'DISABLED' }))
|
|
1059
1194
|
Show-AutonomousSupervisorStatus $Cfg | Out-Null
|
|
1060
1195
|
Show-ProcessStatus $Cfg | Out-Null
|
|
1061
1196
|
Show-CodingStatus $Cfg | Out-Null
|
|
1062
1197
|
Show-AutonomousStatus $Cfg | Out-Null
|
|
1198
|
+
Show-DesktopStatus $Cfg | Out-Null
|
|
1063
1199
|
Show-HostStatus $Cfg | Out-Null
|
|
1064
1200
|
return $s
|
|
1065
1201
|
}
|
|
@@ -1171,6 +1307,9 @@ function Run-Command([string]$Name) {
|
|
|
1171
1307
|
'disarmautonomous' { Disarm-Autonomous $cfg; Show-AutonomousStatus $cfg | Out-Null }
|
|
1172
1308
|
'autonomousstatus' { Show-AutonomousStatus $cfg | Out-Null }
|
|
1173
1309
|
'autonomoustest' { Invoke-AutonomousSmoke $cfg }
|
|
1310
|
+
'armdesktop' { Arm-Desktop $cfg; Show-DesktopStatus $cfg | Out-Null }
|
|
1311
|
+
'disarmdesktop' { Disarm-Desktop $cfg; Show-DesktopStatus $cfg | Out-Null }
|
|
1312
|
+
'desktopstatus' { Show-DesktopStatus $cfg | Out-Null }
|
|
1174
1313
|
'supervisorstart' { Start-AutonomousSupervisor $cfg | Out-Null; Show-AutonomousSupervisorStatus $cfg | Out-Null }
|
|
1175
1314
|
'supervisorstop' { Stop-AutonomousSupervisor $cfg | Out-Null; Show-AutonomousSupervisorStatus $cfg | Out-Null }
|
|
1176
1315
|
'supervisorstatus' { Show-AutonomousSupervisorStatus $cfg | Out-Null }
|
|
@@ -1197,7 +1336,7 @@ while ($true) {
|
|
|
1197
1336
|
Clear-Host
|
|
1198
1337
|
$cfg = Get-Config
|
|
1199
1338
|
Assert-ControllerRootBound $cfg
|
|
1200
|
-
Write-Host 'DEADBYTE MCP ON-DEMAND CONTROLLER V0.
|
|
1339
|
+
Write-Host 'DEADBYTE MCP ON-DEMAND CONTROLLER V0.13.0 / R41'
|
|
1201
1340
|
Write-Host '======================================'
|
|
1202
1341
|
Show-State $cfg | Out-Null
|
|
1203
1342
|
Write-Host ''
|
|
@@ -1225,6 +1364,9 @@ while ($true) {
|
|
|
1225
1364
|
Write-Host '[22] Enable process plane (standing grant)'
|
|
1226
1365
|
Write-Host '[23] Revoke process plane'
|
|
1227
1366
|
Write-Host '[24] Process plane status'
|
|
1367
|
+
Write-Host '[25] Enable desktop plane (standing grant; never auto-armed)'
|
|
1368
|
+
Write-Host '[26] Revoke desktop plane'
|
|
1369
|
+
Write-Host '[27] Desktop plane status'
|
|
1228
1370
|
Write-Host '[0] Exit controller (runtime stays as-is)'
|
|
1229
1371
|
$choice = Read-Host 'Select'
|
|
1230
1372
|
try {
|
|
@@ -1253,6 +1395,9 @@ while ($true) {
|
|
|
1253
1395
|
'22' { Run-Command 'armprocess' }
|
|
1254
1396
|
'23' { Run-Command 'disarmprocess' }
|
|
1255
1397
|
'24' { Run-Command 'processstatus' }
|
|
1398
|
+
'25' { Run-Command 'armdesktop' }
|
|
1399
|
+
'26' { Run-Command 'disarmdesktop' }
|
|
1400
|
+
'27' { Run-Command 'desktopstatus' }
|
|
1256
1401
|
'0' { exit 0 }
|
|
1257
1402
|
default { Write-Warning 'invalid selection' }
|
|
1258
1403
|
}
|
|
@@ -0,0 +1,41 @@
|
|
|
1
|
+
{
|
|
2
|
+
"schema": "deadbyte.desktop-policy.v1",
|
|
3
|
+
"enabled": true,
|
|
4
|
+
"grant_file": "D:\\Projects\\deadbyte-mcp-r41-memory-dev\\controller\\runtime\\desktop-grant.json",
|
|
5
|
+
"evidence_dir": "D:\\Projects\\deadbyte-mcp-r41-memory-dev\\controller\\runtime\\desktop-evidence",
|
|
6
|
+
"observer": {
|
|
7
|
+
"executable": "D:\\Projects\\deadbyte-mcp-r41-memory-dev\\bin\\deadbyte-desktop-observer-r40.exe",
|
|
8
|
+
"sha256": "32671ef2b7decd68c25fcabfe064086e4b3f83a8e9d3558b9d6de3a2510c6dab",
|
|
9
|
+
"backend": "gdi-bitblt-fallback"
|
|
10
|
+
},
|
|
11
|
+
"input": {
|
|
12
|
+
"executable": "D:\\Projects\\deadbyte-mcp-r41-memory-dev\\bin\\deadbyte-desktop-input-r40.exe",
|
|
13
|
+
"sha256": "0b967fd0ef599cbb9882b96ea16db7261f086dbcccae631b56f1bc2445a8ab5b"
|
|
14
|
+
},
|
|
15
|
+
"grounder": {
|
|
16
|
+
"executable": "D:\\Projects\\deadbyte-mcp-r41-memory-dev\\bin\\deadbyte-desktop-uia-r40.exe",
|
|
17
|
+
"sha256": "19ff558218fe8437dc04f50a8848756a8a83a84f73f464057562d77c7e19fb75"
|
|
18
|
+
},
|
|
19
|
+
"allowed_input_desktops": [
|
|
20
|
+
"Default"
|
|
21
|
+
],
|
|
22
|
+
"allowed_actions": [
|
|
23
|
+
"pointer_move",
|
|
24
|
+
"click_primary",
|
|
25
|
+
"click_secondary",
|
|
26
|
+
"double_click_primary",
|
|
27
|
+
"scroll",
|
|
28
|
+
"key_chord",
|
|
29
|
+
"text_input"
|
|
30
|
+
],
|
|
31
|
+
"limits": {
|
|
32
|
+
"max_frame_age_ms": 2000,
|
|
33
|
+
"max_frame_bytes": 134217728,
|
|
34
|
+
"capture_timeout_ms": 10000,
|
|
35
|
+
"input_timeout_ms": 5000,
|
|
36
|
+
"ground_timeout_ms": 5000,
|
|
37
|
+
"post_observe_timeout_ms": 10000,
|
|
38
|
+
"max_preview_bytes": 1048576,
|
|
39
|
+
"max_text_bytes": 4096
|
|
40
|
+
}
|
|
41
|
+
}
|
|
@@ -1,15 +1,15 @@
|
|
|
1
1
|
{
|
|
2
2
|
"schema": "deadbyte.process-policy.v1",
|
|
3
3
|
"enabled": true,
|
|
4
|
-
"grant_file": "D:\\Projects\\deadbyte-mcp-
|
|
5
|
-
"state_dir": "D:\\Projects\\deadbyte-mcp-
|
|
4
|
+
"grant_file": "D:\\Projects\\deadbyte-mcp-r41-memory-dev\\controller\\runtime\\process-grant.json",
|
|
5
|
+
"state_dir": "D:\\Projects\\deadbyte-mcp-r41-memory-dev\\controller\\runtime\\process-state",
|
|
6
6
|
"native_host": {
|
|
7
|
-
"path": "D:\\Projects\\deadbyte-mcp-
|
|
7
|
+
"path": "D:\\Projects\\deadbyte-mcp-r41-memory-dev\\bin\\deadbyte-process-host.exe",
|
|
8
8
|
"sha256": "59ea2026c39a1266c0d645a4cf39d7955ac2cf905f7cda812f546dfc9de636fb"
|
|
9
9
|
},
|
|
10
10
|
"roots": {
|
|
11
11
|
"core": {
|
|
12
|
-
"path": "D:\\Projects\\deadbyte-mcp-
|
|
12
|
+
"path": "D:\\Projects\\deadbyte-mcp-r41-memory-dev",
|
|
13
13
|
"deny_paths": [
|
|
14
14
|
"node_modules",
|
|
15
15
|
"controller/runtime",
|
package/docs/ARCHITECTURE.md
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
# DEADBYTE MCP V0.
|
|
1
|
+
# DEADBYTE MCP V0.13.0 / R41 ARCHITECTURE
|
|
2
2
|
|
|
3
3
|
```mermaid
|
|
4
4
|
flowchart TD
|
|
@@ -14,6 +14,11 @@ flowchart TD
|
|
|
14
14
|
D --> MF[Optional Machine FS 8 Tools]
|
|
15
15
|
D --> OB[Optional Observation 6 Tools]
|
|
16
16
|
D --> AA[Optional Autonomous 12 Tools]
|
|
17
|
+
D --> DT[Optional Desktop 4 Tools]
|
|
18
|
+
DT --> DOBS[Native Windows Observation + UIA Grounding]
|
|
19
|
+
DOBS --> DGUARD[Frame/Epoch/Topology/Foreground Guard]
|
|
20
|
+
DGUARD --> DINPUT[Separate Desktop Standing Grant + SendInput]
|
|
21
|
+
DINPUT --> DPOST[Signed Post-Observation + Effect Chain]
|
|
17
22
|
E --> H[Workspace Path Policy]
|
|
18
23
|
E --> I[Contained Operation Policy]
|
|
19
24
|
I --> J[Private Import]
|
|
@@ -0,0 +1,214 @@
|
|
|
1
|
+
# R40 DESKTOP COMPUTER-USE DESIGN
|
|
2
|
+
|
|
3
|
+
Status: R40-T1 implementation baseline
|
|
4
|
+
Date: 2026-09-19
|
|
5
|
+
Predecessor: PUBLIC_FINAL v0.11.3 / R39
|
|
6
|
+
Development root only. The sealed v0.11.3 release slot is not modified.
|
|
7
|
+
|
|
8
|
+
## Goal
|
|
9
|
+
|
|
10
|
+
Add real Windows desktop computer-use without weakening DEADBYTE's authority/evidence model.
|
|
11
|
+
|
|
12
|
+
Canonical chain:
|
|
13
|
+
|
|
14
|
+
CAPABILITY -> AUTHORITY -> INTENT -> PRE-OBSERVATION -> GUARDED EFFECT -> POST-OBSERVATION -> EVIDENCE -> ACCEPTANCE
|
|
15
|
+
|
|
16
|
+
A screenshot is evidence, not authority. A coordinate is not a target unless it is bound to a fresh frame, desktop epoch, display topology, and (when required) foreground window identity.
|
|
17
|
+
|
|
18
|
+
## Research inputs
|
|
19
|
+
|
|
20
|
+
The R40 design studies techniques rather than copying an upstream runtime.
|
|
21
|
+
|
|
22
|
+
- Simular Agent-S: screen-grounded computer use with mouse/keyboard actions and iterative desktop observation.
|
|
23
|
+
- SWE-agent / mini-SWE-agent lineage: narrow Agent-Computer Interfaces that make environment interaction explicit and model-friendly.
|
|
24
|
+
- Desktop Commander / Remote Desktop Commander: persistent device sessions, remote machine process/file operations, and foreground device-agent lifecycle.
|
|
25
|
+
- Serena: semantic coding remains the preferred path for source changes; desktop control is a fallback for GUI-only work, not a replacement for semantic tools.
|
|
26
|
+
- Letta: persistent state should retain compact durable facts and verified trajectory summaries instead of unbounded raw context.
|
|
27
|
+
- Guardrails: structured action validation and fail/repair behavior before side effects.
|
|
28
|
+
- OpenManus: tool-mediated general-agent/browser workflows are useful as orchestration references; DEADBYTE keeps its own authority and evidence chain.
|
|
29
|
+
- Windows platform authority remains Microsoft APIs. No upstream GUI-agent Python automation runtime becomes production authority.
|
|
30
|
+
|
|
31
|
+
## Non-goals
|
|
32
|
+
|
|
33
|
+
- No arbitrary Python/Bash execution for GUI control.
|
|
34
|
+
- No OCR dependency in the authority path.
|
|
35
|
+
- No image-only claim that an action succeeded.
|
|
36
|
+
- No host-plane arm as a shortcut.
|
|
37
|
+
- No Firecracker claim: DEADBYTE remains Windows-native; AppContainer/Job Object containment is not a microVM.
|
|
38
|
+
- No public R40 tool until native observation, guarded input, post-observation, and adversarial acceptance all pass.
|
|
39
|
+
|
|
40
|
+
## R40 construction ladder
|
|
41
|
+
|
|
42
|
+
### T1 — Desktop action contract (this change)
|
|
43
|
+
|
|
44
|
+
Pure deterministic contract layer:
|
|
45
|
+
- desktop epoch binding;
|
|
46
|
+
- virtual-desktop bounds;
|
|
47
|
+
- active-window binding;
|
|
48
|
+
- frame hash and topology hash;
|
|
49
|
+
- maximum frame age;
|
|
50
|
+
- coordinate bounds;
|
|
51
|
+
- explicit action intent;
|
|
52
|
+
- stale-frame rejection;
|
|
53
|
+
- post-observation binding;
|
|
54
|
+
- no success inference from visual change alone.
|
|
55
|
+
|
|
56
|
+
No input is generated in T1.
|
|
57
|
+
|
|
58
|
+
### T2 — Windows native observer
|
|
59
|
+
|
|
60
|
+
Implement a fixed native helper owned by the release:
|
|
61
|
+
- enumerate monitor topology;
|
|
62
|
+
- capture virtual desktop frame;
|
|
63
|
+
- report foreground HWND/PID;
|
|
64
|
+
- report frame dimensions and DPI/topology data;
|
|
65
|
+
- emit exact binary frame bytes plus metadata hash;
|
|
66
|
+
- bounded capture time and frame size;
|
|
67
|
+
- no caller-selected executable;
|
|
68
|
+
- native helper SHA pinned in release manifest.
|
|
69
|
+
|
|
70
|
+
Primary implementation candidates:
|
|
71
|
+
1. Windows Graphics Capture where production-compatible;
|
|
72
|
+
2. Desktop Duplication for monitor capture where appropriate;
|
|
73
|
+
3. GDI only as a compatibility fallback, never as unlabelled equivalent evidence.
|
|
74
|
+
|
|
75
|
+
T2 acceptance includes multi-monitor coordinates, DPI scaling, minimized/locked/secure desktop behavior, and capture-failure semantics.
|
|
76
|
+
|
|
77
|
+
### T3 — Guarded input plane
|
|
78
|
+
|
|
79
|
+
Add a separate Desktop authority grant. It is not Coding, Process, Autonomous, or Host authority.
|
|
80
|
+
|
|
81
|
+
Allowed actions are fixed:
|
|
82
|
+
- pointer move;
|
|
83
|
+
- primary/secondary click;
|
|
84
|
+
- double click;
|
|
85
|
+
- bounded wheel scroll;
|
|
86
|
+
- key chord from allowlisted key vocabulary;
|
|
87
|
+
- bounded UTF-8 text insertion.
|
|
88
|
+
|
|
89
|
+
Every mutating action requires:
|
|
90
|
+
- explicit Desktop standing grant;
|
|
91
|
+
- exact pre-frame id;
|
|
92
|
+
- desktop epoch;
|
|
93
|
+
- display topology hash;
|
|
94
|
+
- optional required foreground-window binding;
|
|
95
|
+
- frame age within policy;
|
|
96
|
+
- coordinate inside captured virtual desktop;
|
|
97
|
+
- fresh pre-observation immediately before effect.
|
|
98
|
+
|
|
99
|
+
Windows input implementation will use fixed native APIs (for example SendInput) behind a pinned worker. Caller cannot inject arbitrary Win32 messages or choose an executable.
|
|
100
|
+
|
|
101
|
+
### T4 — Post-observe + stale-frame/recovery
|
|
102
|
+
|
|
103
|
+
Every Desktop effect must produce:
|
|
104
|
+
- action receipt;
|
|
105
|
+
- post-frame receipt;
|
|
106
|
+
- before/action/after chain hash;
|
|
107
|
+
- change_observed boolean;
|
|
108
|
+
- foreground-window transition;
|
|
109
|
+
- effect-ledger binding.
|
|
110
|
+
|
|
111
|
+
A changed image is NOT equivalent to task success.
|
|
112
|
+
An unchanged image is NOT automatically failure.
|
|
113
|
+
The planner receives observed facts and decides/replans through the existing autonomous release/verification model.
|
|
114
|
+
|
|
115
|
+
A frame becomes unusable when any required binding changes:
|
|
116
|
+
- desktop epoch;
|
|
117
|
+
- topology hash;
|
|
118
|
+
- expected foreground window;
|
|
119
|
+
- frame identity;
|
|
120
|
+
- age budget.
|
|
121
|
+
|
|
122
|
+
### T5 — Grounding and target selection
|
|
123
|
+
|
|
124
|
+
Grounding may produce candidate regions, but it never directly generates input.
|
|
125
|
+
Candidate target contract:
|
|
126
|
+
- target rectangle;
|
|
127
|
+
- confidence;
|
|
128
|
+
- source kind (visual, UI Automation, accessibility, deterministic geometry);
|
|
129
|
+
- frame id;
|
|
130
|
+
- optional semantic label hash;
|
|
131
|
+
- optional HWND/PID binding.
|
|
132
|
+
|
|
133
|
+
R40 should prefer deterministic Windows UI Automation/accessibility targets when available and use visual grounding for applications that do not expose usable semantics.
|
|
134
|
+
|
|
135
|
+
### T6 — Durable desktop memory
|
|
136
|
+
|
|
137
|
+
Borrow the useful stateful-agent principle without importing an external memory runtime.
|
|
138
|
+
|
|
139
|
+
Persist only authenticated summaries:
|
|
140
|
+
- stable application/window identifiers;
|
|
141
|
+
- verified successful navigation facts;
|
|
142
|
+
- rejected/stale target fingerprints;
|
|
143
|
+
- compact recent action trajectory;
|
|
144
|
+
- last verified desktop epoch/topology.
|
|
145
|
+
|
|
146
|
+
Do not persist raw screenshots forever by default.
|
|
147
|
+
Memory cannot grant authority.
|
|
148
|
+
Memory cannot override a fresh observation.
|
|
149
|
+
Stale memory is planner context only.
|
|
150
|
+
|
|
151
|
+
### T7 — Autonomous desktop loop
|
|
152
|
+
|
|
153
|
+
Extend autonomous action kinds with desktop effects only after T2-T6 pass.
|
|
154
|
+
|
|
155
|
+
Loop:
|
|
156
|
+
OBSERVE -> GROUND -> PLAN -> GUARD -> ACT -> POST-OBSERVE -> VERIFY -> REPLAN
|
|
157
|
+
|
|
158
|
+
Loop guard additionally blocks:
|
|
159
|
+
- repeated clicks on the same target/frame family;
|
|
160
|
+
- repeated no-change effects;
|
|
161
|
+
- focus thrashing;
|
|
162
|
+
- stale-frame retries;
|
|
163
|
+
- destructive action escalation without explicit policy.
|
|
164
|
+
|
|
165
|
+
### T8 — Public MCP surface
|
|
166
|
+
|
|
167
|
+
Target compact gateways, names provisional:
|
|
168
|
+
- desktop_observe
|
|
169
|
+
- desktop_target
|
|
170
|
+
- desktop_action
|
|
171
|
+
- desktop_status
|
|
172
|
+
|
|
173
|
+
Keep the public compact surface within an explicitly revised ceiling. Do not silently grow the R39 41-tool contract.
|
|
174
|
+
|
|
175
|
+
### T9 — Acceptance
|
|
176
|
+
|
|
177
|
+
Required adversarial proof:
|
|
178
|
+
- stale frame denied;
|
|
179
|
+
- wrong epoch denied;
|
|
180
|
+
- wrong topology denied;
|
|
181
|
+
- wrong foreground window denied;
|
|
182
|
+
- off-screen coordinate denied;
|
|
183
|
+
- integer overflow/negative coordinate denied;
|
|
184
|
+
- post-observation required;
|
|
185
|
+
- locked/secure desktop fails closed;
|
|
186
|
+
- capture failure blocks input;
|
|
187
|
+
- helper substitution detected;
|
|
188
|
+
- receipt substitution detected;
|
|
189
|
+
- interrupted action reconciles without blind replay;
|
|
190
|
+
- Host remains DISARMED throughout;
|
|
191
|
+
- old R39 coding/process/autonomous parity remains green.
|
|
192
|
+
|
|
193
|
+
## Authority decision
|
|
194
|
+
|
|
195
|
+
Desktop mutation gets its own standing grant.
|
|
196
|
+
|
|
197
|
+
Observation does not require Desktop mutation authority.
|
|
198
|
+
Desktop mutation authority does not imply Host authority.
|
|
199
|
+
Autonomous authority may request Desktop effects only when Desktop authority is separately armed.
|
|
200
|
+
Revoking Desktop authority cancels further GUI effects but preserves signed observations/evidence.
|
|
201
|
+
|
|
202
|
+
## Completion definition
|
|
203
|
+
|
|
204
|
+
R40 is not complete when it can move the mouse.
|
|
205
|
+
|
|
206
|
+
R40 is complete only when a fresh public consumer can:
|
|
207
|
+
1. observe a Windows desktop;
|
|
208
|
+
2. bind a target to fresh evidence;
|
|
209
|
+
3. perform an authorized input effect;
|
|
210
|
+
4. post-observe;
|
|
211
|
+
5. reject stale/replayed evidence;
|
|
212
|
+
6. recover/replan after interruption;
|
|
213
|
+
7. verify signed evidence;
|
|
214
|
+
8. do all of the above with Host DISARMED.
|