minimal-agent 0.10.0 → 0.12.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +11 -3
- package/package.json +52 -5
- package/plugins/computer_use/.claude-plugin/plugin.json +6 -0
- package/plugins/computer_use/README.md +94 -0
- package/plugins/computer_use/app/_template.md +142 -0
- package/plugins/computer_use/app/chrome.md +165 -0
- package/plugins/computer_use/app/finder.md +76 -0
- package/plugins/computer_use/app/macos-system-settings.md +81 -0
- package/plugins/computer_use/app/microsoft-excel.md +80 -0
- package/plugins/computer_use/app/microsoft-powerpoint.md +80 -0
- package/plugins/computer_use/app/microsoft-word.md +79 -0
- package/plugins/computer_use/app/wechat.md +413 -0
- package/plugins/computer_use/app/weread.md +319 -0
- package/plugins/computer_use/commands/computeruse.md +25 -0
- package/plugins/computer_use/commands/cu.md +9 -0
- package/plugins/computer_use/plugin.js +26 -0
- package/plugins/computer_use/plugin.ts +30 -0
- package/plugins/computer_use/src/capture.js +90 -0
- package/plugins/computer_use/src/capture.ts +104 -0
- package/plugins/computer_use/src/commands.js +91 -0
- package/plugins/computer_use/src/commands.ts +103 -0
- package/plugins/computer_use/src/daemon.js +134 -0
- package/plugins/computer_use/src/daemon.ts +161 -0
- package/plugins/computer_use/src/manuals.js +135 -0
- package/plugins/computer_use/src/manuals.ts +159 -0
- package/plugins/computer_use/src/verify.js +168 -0
- package/plugins/computer_use/src/verify.ts +219 -0
- package/plugins/computer_use/test/commands.test.ts +73 -0
- package/plugins/cron-runner/plugin.ts +38 -0
- package/plugins/cron-runner/src/commands.ts +373 -0
- package/plugins/cron-runner/src/cronTime.ts +155 -0
- package/plugins/cron-runner/src/lock.ts +79 -0
- package/plugins/cron-runner/src/runs.ts +99 -0
- package/plugins/cron-runner/src/scheduleEngine.ts +76 -0
- package/plugins/cron-runner/src/scheduler.ts +376 -0
- package/plugins/cron-runner/src/schema.ts +139 -0
- package/plugins/cron-runner/src/service.ts +193 -0
- package/plugins/cron-runner/src/store.ts +333 -0
- package/plugins/cron-runner/src/types.ts +141 -0
- package/plugins/cron-runner/test/cronTime.test.ts +41 -0
- package/plugins/cron-runner/test/plugin-integration.test.ts +47 -0
- package/plugins/cron-runner/test/scheduleEngine.test.ts +41 -0
- package/plugins/cron-runner/test/service.test.ts +76 -0
- package/plugins/cron-runner/test/store-and-scheduler.test.ts +501 -0
- package/plugins/workflow-runner/plugin.js +56 -2
- package/plugins/workflow-runner/plugin.ts +113 -0
- package/plugins/workflow-runner/src/agentCommon.js +43 -4
- package/plugins/workflow-runner/src/agentCommon.ts +435 -0
- package/plugins/workflow-runner/src/controlFlow/branch.js +40 -0
- package/plugins/workflow-runner/src/controlFlow/branch.ts +46 -0
- package/plugins/workflow-runner/src/controlFlow/loop.js +211 -0
- package/plugins/workflow-runner/src/controlFlow/loop.ts +248 -0
- package/plugins/workflow-runner/src/controlFlow/pause.js +129 -0
- package/plugins/workflow-runner/src/controlFlow/pause.ts +141 -0
- package/plugins/workflow-runner/src/controlFlow/signal.js +21 -0
- package/plugins/workflow-runner/src/controlFlow/signal.ts +29 -0
- package/plugins/workflow-runner/src/controlFlow/types.js +18 -0
- package/plugins/workflow-runner/src/controlFlow/types.ts +63 -0
- package/plugins/workflow-runner/src/dslManifest.ts +127 -0
- package/plugins/workflow-runner/src/expressions.ts +622 -0
- package/plugins/workflow-runner/src/index.js +13 -4
- package/plugins/workflow-runner/src/index.ts +338 -0
- package/plugins/workflow-runner/src/jsonPayload.ts +84 -0
- package/plugins/workflow-runner/src/loader.js +23 -0
- package/plugins/workflow-runner/src/loader.ts +721 -0
- package/plugins/workflow-runner/src/multimodal.js +2 -1
- package/plugins/workflow-runner/src/multimodal.ts +64 -0
- package/plugins/workflow-runner/src/recovery.ts +149 -0
- package/plugins/workflow-runner/src/runner.js +75 -659
- package/plugins/workflow-runner/src/runner.ts +604 -0
- package/plugins/workflow-runner/src/stepExecutors/agent.ts +234 -0
- package/plugins/workflow-runner/src/stepExecutors/assert.ts +50 -0
- package/plugins/workflow-runner/src/stepExecutors/echo.js +31 -0
- package/plugins/workflow-runner/src/stepExecutors/echo.ts +38 -0
- package/plugins/workflow-runner/src/stepExecutors/llm.ts +133 -0
- package/plugins/workflow-runner/src/stepExecutors/set.js +49 -0
- package/plugins/workflow-runner/src/stepExecutors/set.ts +56 -0
- package/plugins/workflow-runner/src/stepExecutors/skill.js +5 -6
- package/plugins/workflow-runner/src/stepExecutors/skill.ts +140 -0
- package/plugins/workflow-runner/src/stepExecutors/tool.js +80 -8
- package/plugins/workflow-runner/src/stepExecutors/tool.ts +256 -0
- package/plugins/workflow-runner/src/stepExecutors/verify.ts +68 -0
- package/plugins/workflow-runner/src/stepPipeline.js +277 -0
- package/plugins/workflow-runner/src/stepPipeline.ts +366 -0
- package/plugins/workflow-runner/src/taskIdentity.ts +405 -0
- package/plugins/workflow-runner/src/toolRuntime.js +48 -0
- package/plugins/workflow-runner/src/toolRuntime.ts +64 -0
- package/plugins/workflow-runner/src/types.js +37 -1
- package/plugins/workflow-runner/src/types.ts +602 -0
- package/plugins/workflow-runner/src/workflowState.ts +733 -0
- package/plugins/workflow-runner/test/agent.test.ts +587 -0
- package/plugins/workflow-runner/test/agentLlmError.test.ts +373 -0
- package/plugins/workflow-runner/test/bashBlacklist.sweep.test.ts +65 -0
- package/plugins/workflow-runner/test/captureMisbind.test.ts +99 -0
- package/plugins/workflow-runner/test/cli.e2e.test.ts +147 -0
- package/plugins/workflow-runner/test/context-files.test.ts +320 -0
- package/plugins/workflow-runner/test/e2e.test.ts +370 -0
- package/plugins/workflow-runner/test/engineRobustness.test.ts +226 -0
- package/plugins/workflow-runner/test/expressions.test.ts +397 -0
- package/plugins/workflow-runner/test/foundation.test.ts +126 -0
- package/plugins/workflow-runner/test/graceful.test.ts +139 -0
- package/plugins/workflow-runner/test/greedyInputs.load.test.ts +47 -0
- package/plugins/workflow-runner/test/hostCapability.test.ts +108 -0
- package/plugins/workflow-runner/test/illustrate.load.test.ts +183 -0
- package/plugins/workflow-runner/test/jsonPayload.test.ts +83 -0
- package/plugins/workflow-runner/test/loader.test.ts +617 -0
- package/plugins/workflow-runner/test/maxTurnsRetryDelay.test.ts +33 -0
- package/plugins/workflow-runner/test/multimodal.test.ts +342 -0
- package/plugins/workflow-runner/test/output-schema.test.ts +213 -0
- package/plugins/workflow-runner/test/parseWorkflowArgs.test.ts +87 -0
- package/plugins/workflow-runner/test/pluginRunner.isolation.test.ts +269 -0
- package/plugins/workflow-runner/test/ppt.load.test.ts +1220 -0
- package/plugins/workflow-runner/test/ralphLoop.e2e.test.ts +220 -0
- package/plugins/workflow-runner/test/recovery.test.ts +265 -0
- package/plugins/workflow-runner/test/runner.test.ts +911 -0
- package/plugins/workflow-runner/test/runnerPreflight.test.ts +379 -0
- package/plugins/workflow-runner/test/skillTurnDone.test.ts +71 -0
- package/plugins/workflow-runner/test/stateLifecycle.test.ts +150 -0
- package/plugins/workflow-runner/test/stepPipeline.test.ts +114 -0
- package/plugins/workflow-runner/test/taskArbitration.test.ts +338 -0
- package/plugins/workflow-runner/test/taskIdentity.test.ts +237 -0
- package/plugins/workflow-runner/test/taskLifecycle.test.ts +270 -0
- package/plugins/workflow-runner/test/toolArtifactOverwrite.test.ts +221 -0
- package/plugins/workflow-runner/test/toolCapabilityBoundary.test.ts +29 -0
- package/plugins/workflow-runner/test/toolRuntimeIntegration.test.ts +168 -0
- package/plugins/workflow-runner/test/webppt.load.test.ts +250 -0
- package/plugins/workflow-runner/test/word.load.test.ts +208 -0
- package/plugins/workflow-runner/test/workflows.smoke.test.ts +76 -0
- package/plugins/workflow-runner/test/yamlGuardConditions.test.ts +207 -0
- package/skills/browser-automation/SKILL.md +9 -19
- package/skills/web-crawler/SKILL.md +9 -11
- package/src/askBridge.ts +249 -0
- package/src/bootstrap/cwdArg.js +10 -3
- package/src/bootstrap/cwdArg.ts +31 -0
- package/src/bootstrap/workingDir.ts +48 -0
- package/src/cli/args.js +70 -7
- package/src/cli/args.ts +146 -0
- package/src/cli/computerUse.js +185 -0
- package/src/cli/computerUse.ts +221 -0
- package/src/cli/configMenu.js +274 -202
- package/src/cli/configMenu.tsx +521 -0
- package/src/cli/memoryAccess.js +45 -0
- package/src/cli/memoryAccess.ts +53 -0
- package/src/cli/print.js +119 -5
- package/src/cli/print.ts +597 -0
- package/src/cli/quickModelSetup.js +60 -0
- package/src/cli/quickModelSetup.tsx +96 -0
- package/src/cli/streamJson.js +80 -4
- package/src/cli/streamJson.ts +433 -0
- package/src/cli/wizards/advisorWizard.js +1 -0
- package/src/cli/wizards/advisorWizard.tsx +195 -0
- package/src/cli/wizards/agentRuntimeWizard.js +84 -0
- package/src/cli/wizards/agentRuntimeWizard.tsx +134 -0
- package/src/cli/wizards/browserWizard.js +55 -0
- package/src/cli/wizards/browserWizard.tsx +77 -0
- package/src/cli/wizards/contextAdvisorWizard.js +84 -0
- package/src/cli/wizards/contextAdvisorWizard.tsx +124 -0
- package/src/cli/wizards/fieldForm.js +7 -4
- package/src/cli/wizards/fieldForm.tsx +250 -0
- package/src/cli/wizards/imageWizard.js +4 -5
- package/src/cli/wizards/imageWizard.tsx +96 -0
- package/src/cli/wizards/openRouterWizard.tsx +48 -0
- package/src/cli/wizards/providerPresets.js +1 -8
- package/src/cli/wizards/providerPresets.ts +106 -0
- package/src/cli/wizards/providerWizard.js +152 -205
- package/src/cli/wizards/providerWizard.tsx +266 -0
- package/src/cli/wizards/searchWizard.tsx +47 -0
- package/src/cli/wizards/ttsWizard.js +4 -3
- package/src/cli/wizards/ttsWizard.tsx +61 -0
- package/src/config/configDefaults.js +25 -0
- package/src/config/configDefaults.ts +33 -0
- package/src/config/configDiagnostics.js +117 -0
- package/src/config/configDiagnostics.ts +166 -0
- package/src/config/configFile.js +52 -32
- package/src/config/configFile.ts +633 -0
- package/src/config/configManagement.js +126 -0
- package/src/config/configManagement.ts +159 -0
- package/src/config/configService.js +66 -11
- package/src/config/configService.ts +554 -0
- package/src/config/numericLimits.js +61 -0
- package/src/config/numericLimits.ts +80 -0
- package/src/config.ts +109 -0
- package/src/context/agentHome.ts +11 -0
- package/src/context/archive.ts +144 -0
- package/src/context/budgetPlan.js +54 -0
- package/src/context/budgetPlan.ts +88 -0
- package/src/context/compact.js +179 -106
- package/src/context/compact.ts +584 -0
- package/src/context/contextBundle.js +119 -0
- package/src/context/contextBundle.ts +176 -0
- package/src/context/contextLock.ts +84 -0
- package/src/context/microCompactLite.ts +218 -0
- package/src/context/persistContext.ts +177 -0
- package/src/context/reactiveCompact.js +19 -4
- package/src/context/reactiveCompact.ts +240 -0
- package/src/context/recentDirs.ts +42 -0
- package/src/context/sessionPath.ts +206 -0
- package/src/context/sessionRegistry.js +8 -2
- package/src/context/sessionRegistry.ts +414 -0
- package/src/context/snipCompact.ts +163 -0
- package/src/context/tokenCounter.ts +106 -0
- package/src/context/workspaceIndex.js +2 -2
- package/src/context/workspaceIndex.ts +751 -0
- package/src/entry.compiled.tsx +20 -0
- package/src/harness/claimValidation.js +9 -0
- package/src/harness/claimValidation.ts +22 -0
- package/src/harness/evidenceLedger.js +92 -0
- package/src/harness/evidenceLedger.ts +96 -0
- package/src/harness/stallGuard.js +60 -0
- package/src/harness/stallGuard.ts +81 -0
- package/src/humanIntervention.js +129 -0
- package/src/humanIntervention.ts +216 -0
- package/src/infra/verification.ts +295 -0
- package/src/irreversible/rules.js +216 -0
- package/src/irreversible/rules.ts +261 -0
- package/src/llm/client.js +40 -3
- package/src/llm/client.ts +621 -0
- package/src/llm/promptCache.js +27 -0
- package/src/llm/promptCache.ts +41 -0
- package/src/llm/providers/anthropicFormat.js +41 -5
- package/src/llm/providers/anthropicFormat.ts +290 -0
- package/src/llm/providers/openaiFormat.js +186 -8
- package/src/llm/providers/openaiFormat.ts +428 -0
- package/src/loop.js +614 -63
- package/src/loop.ts +1454 -0
- package/src/main.js +67 -10
- package/src/main.tsx +344 -0
- package/src/memory/codec.js +204 -0
- package/src/memory/codec.ts +249 -0
- package/src/memory/constants.js +34 -0
- package/src/memory/constants.ts +50 -0
- package/src/memory/episodicMemory.js +684 -0
- package/src/memory/episodicMemory.ts +815 -0
- package/src/memory/fileStore.js +148 -0
- package/src/memory/fileStore.ts +184 -0
- package/src/memory/memoryService.js +570 -0
- package/src/memory/memoryService.ts +859 -0
- package/src/memory/migration.js +128 -0
- package/src/memory/migration.ts +160 -0
- package/src/memory/repository.js +789 -0
- package/src/memory/repository.ts +893 -0
- package/src/memory/retrieval.js +108 -0
- package/src/memory/retrieval.ts +145 -0
- package/src/memory/searchIndex.js +107 -0
- package/src/memory/searchIndex.ts +153 -0
- package/src/memory/topicView.js +82 -0
- package/src/memory/topicView.ts +95 -0
- package/src/memory/turnJournal.js +95 -0
- package/src/memory/turnJournal.ts +120 -0
- package/src/memory/types.js +2 -0
- package/src/memory/types.ts +186 -0
- package/src/messageQueue.ts +123 -0
- package/src/multimodal.js +128 -46
- package/src/multimodal.ts +349 -0
- package/src/plugin-sdk.js +21 -1
- package/src/plugin-sdk.ts +134 -0
- package/src/plugins/builtinPluginIds.js +47 -0
- package/src/plugins/builtinPluginIds.ts +51 -0
- package/src/plugins/builtinRegistry.ts +43 -0
- package/src/plugins/commandRouter.js +40 -26
- package/src/plugins/commandRouter.ts +261 -0
- package/src/plugins/hookEngine.ts +341 -0
- package/src/plugins/pluginApi.ts +133 -0
- package/src/plugins/pluginLifecycle.ts +34 -0
- package/src/plugins/pluginLoader.ts +147 -0
- package/src/plugins/pluginManifest.js +127 -0
- package/src/plugins/pluginManifest.ts +152 -0
- package/src/plugins/pluginRunner.js +47 -2
- package/src/plugins/pluginRunner.ts +207 -0
- package/src/plugins/transcript.ts +208 -0
- package/src/prompts/lint.js +169 -0
- package/src/prompts/lint.ts +220 -0
- package/src/prompts/pluginList.js +32 -16
- package/src/prompts/pluginList.ts +86 -0
- package/src/prompts/policyRegistry.js +318 -0
- package/src/prompts/policyRegistry.ts +358 -0
- package/src/prompts/projectInstructions.js +24 -0
- package/src/prompts/projectInstructions.ts +81 -0
- package/src/prompts/promptAssembly.js +311 -202
- package/src/prompts/promptAssembly.ts +510 -0
- package/src/prompts/skillList.js +106 -45
- package/src/prompts/skillList.ts +198 -0
- package/src/prompts/system.js +31 -6
- package/src/prompts/system.ts +81 -0
- package/src/runtime/agentRuntimeContext.js +53 -0
- package/src/runtime/agentRuntimeContext.ts +112 -0
- package/src/session/runAgentTurn.js +71 -12
- package/src/session/runAgentTurn.ts +181 -0
- package/src/session/runTurn.js +29 -0
- package/src/session/runTurn.ts +160 -0
- package/src/session/runtimeStore.js +149 -0
- package/src/session/runtimeStore.ts +197 -0
- package/src/session/sessionState.js +79 -3
- package/src/session/sessionState.ts +134 -0
- package/src/toolGate.ts +50 -0
- package/src/tools/advisor/advisor.js +76 -15
- package/src/tools/advisor/advisor.ts +512 -0
- package/src/tools/agent/agent.js +245 -29
- package/src/tools/agent/agent.ts +837 -0
- package/src/tools/agent/agentTaskService.js +60 -0
- package/src/tools/agent/agentTaskService.ts +113 -0
- package/src/tools/agent/agentTypes.js +20 -8
- package/src/tools/agent/agentTypes.ts +94 -0
- package/src/tools/agent/coordinator.ts +111 -0
- package/src/tools/agent/worktree.ts +296 -0
- package/src/tools/ask/ask.js +17 -0
- package/src/tools/ask/ask.ts +122 -0
- package/src/tools/bash/attachImage.js +223 -0
- package/src/tools/bash/attachImage.ts +281 -0
- package/src/tools/bash/bash.js +132 -14
- package/src/tools/bash/bash.ts +642 -0
- package/src/tools/bash/semantics.ts +103 -0
- package/src/tools/bash/warnings.js +17 -11
- package/src/tools/bash/warnings.ts +117 -0
- package/src/tools/capabilities.js +42 -0
- package/src/tools/capabilities.ts +69 -0
- package/src/tools/computeruse/accessibilityPermission.js +261 -0
- package/src/tools/computeruse/accessibilityPermission.ts +294 -0
- package/src/tools/computeruse/axSearch.js +96 -0
- package/src/tools/computeruse/axSearch.ts +120 -0
- package/src/tools/computeruse/axSnap.js +272 -0
- package/src/tools/computeruse/axSnap.ts +349 -0
- package/src/tools/computeruse/capabilityLabels.js +95 -0
- package/src/tools/computeruse/capabilityLabels.ts +115 -0
- package/src/tools/computeruse/computerUseService.js +2235 -0
- package/src/tools/computeruse/computerUseService.ts +2686 -0
- package/src/tools/computeruse/computeruse.js +2742 -0
- package/src/tools/computeruse/computeruse.ts +3265 -0
- package/src/tools/computeruse/diagnosticTrace.js +154 -0
- package/src/tools/computeruse/diagnosticTrace.ts +195 -0
- package/src/tools/computeruse/diagnostics.js +369 -0
- package/src/tools/computeruse/diagnostics.ts +453 -0
- package/src/tools/computeruse/grants.js +53 -0
- package/src/tools/computeruse/grants.ts +70 -0
- package/src/tools/computeruse/humanBoundary.js +159 -0
- package/src/tools/computeruse/humanBoundary.ts +199 -0
- package/src/tools/computeruse/intervention.js +77 -0
- package/src/tools/computeruse/intervention.ts +110 -0
- package/src/tools/computeruse/manualSchema.js +709 -0
- package/src/tools/computeruse/manualSchema.ts +916 -0
- package/src/tools/computeruse/observationImages.js +705 -0
- package/src/tools/computeruse/observationImages.ts +889 -0
- package/src/tools/computeruse/protocol.js +60 -0
- package/src/tools/computeruse/protocol.ts +515 -0
- package/src/tools/computeruse/telemetryLog.js +57 -0
- package/src/tools/computeruse/telemetryLog.ts +67 -0
- package/src/tools/computeruse/vision/appGuide.js +325 -0
- package/src/tools/computeruse/vision/appGuide.ts +482 -0
- package/src/tools/computeruse/vision/focus.js +291 -0
- package/src/tools/computeruse/vision/focus.ts +382 -0
- package/src/tools/computeruse/vision/frames.js +248 -0
- package/src/tools/computeruse/vision/frames.ts +329 -0
- package/src/tools/computeruse/vision/geometry.js +131 -0
- package/src/tools/computeruse/vision/geometry.ts +163 -0
- package/src/tools/computeruse/vision/imaging.js +201 -0
- package/src/tools/computeruse/vision/imaging.ts +281 -0
- package/src/tools/computeruse/vision/png.js +274 -0
- package/src/tools/computeruse/vision/png.ts +317 -0
- package/src/tools/computeruse/vision/raster.js +320 -0
- package/src/tools/computeruse/vision/raster.ts +408 -0
- package/src/tools/computeruse/vision/settle.js +131 -0
- package/src/tools/computeruse/vision/settle.ts +220 -0
- package/src/tools/computeruse/vision/telemetry.js +196 -0
- package/src/tools/computeruse/vision/telemetry.ts +241 -0
- package/src/tools/computeruse/visionOcr.js +115 -0
- package/src/tools/computeruse/visionOcr.ts +128 -0
- package/src/tools/computeruse/waitFor.js +237 -0
- package/src/tools/computeruse/waitFor.ts +291 -0
- package/src/tools/download/download.js +10 -6
- package/src/tools/download/download.ts +187 -0
- package/src/tools/edit/edit.js +30 -18
- package/src/tools/edit/edit.ts +349 -0
- package/src/tools/edit/multi-edit.js +25 -14
- package/src/tools/edit/multi-edit.ts +223 -0
- package/src/tools/effectiveToolSet.js +73 -0
- package/src/tools/effectiveToolSet.ts +91 -0
- package/src/tools/genimage/genimage.ts +257 -0
- package/src/tools/genspeech/genspeech.ts +83 -0
- package/src/tools/glob/glob.js +176 -29
- package/src/tools/glob/glob.ts +283 -0
- package/src/tools/grep/grep.js +61 -19
- package/src/tools/grep/grep.ts +239 -0
- package/src/tools/grep/rgPath.ts +206 -0
- package/src/tools/index.js +99 -5
- package/src/tools/index.ts +293 -0
- package/src/tools/lsp/config.js +113 -0
- package/src/tools/lsp/config.ts +113 -0
- package/src/tools/lsp/lsp.js +256 -0
- package/src/tools/lsp/lsp.ts +274 -0
- package/src/tools/lsp/service.js +303 -0
- package/src/tools/lsp/service.ts +296 -0
- package/src/tools/memory/memory.js +520 -0
- package/src/tools/memory/memory.ts +625 -0
- package/src/tools/memory/policy.js +53 -0
- package/src/tools/memory/policy.ts +72 -0
- package/src/tools/notebook/notebookEdit.js +114 -0
- package/src/tools/notebook/notebookEdit.ts +126 -0
- package/src/tools/powershell/powershell.js +210 -0
- package/src/tools/powershell/powershell.ts +239 -0
- package/src/tools/presentresult/presentresult.js +145 -0
- package/src/tools/presentresult/presentresult.ts +156 -0
- package/src/tools/read/read.js +371 -163
- package/src/tools/read/read.ts +558 -0
- package/src/tools/shared/diffSummary.js +44 -0
- package/src/tools/shared/diffSummary.ts +63 -0
- package/src/tools/shared/fileActivity.js +51 -0
- package/src/tools/shared/fileActivity.ts +66 -0
- package/src/tools/shared/fileMutationBus.js +13 -0
- package/src/tools/shared/fileMutationBus.ts +14 -0
- package/src/tools/shared/fileState.js +73 -65
- package/src/tools/shared/fileState.ts +145 -0
- package/src/tools/shared/fileUtils.js +81 -4
- package/src/tools/shared/fileUtils.ts +504 -0
- package/src/tools/shared/killTree.ts +50 -0
- package/src/tools/shared/permanentDeleteGuard.js +26 -0
- package/src/tools/shared/permanentDeleteGuard.ts +34 -0
- package/src/tools/shared/schemas.ts +20 -0
- package/src/tools/shared/toolOutputStore.js +97 -0
- package/src/tools/shared/toolOutputStore.ts +110 -0
- package/src/tools/skill/skill.js +86 -0
- package/src/tools/skill/skill.ts +94 -0
- package/src/tools/snip/snip.js +67 -0
- package/src/tools/snip/snip.ts +80 -0
- package/src/tools/tasks/backgroundTaskService.js +275 -0
- package/src/tools/tasks/backgroundTaskService.ts +369 -0
- package/src/tools/tasks/processEvidence.js +24 -0
- package/src/tools/tasks/processEvidence.ts +34 -0
- package/src/tools/tasks/taskManagement.js +163 -0
- package/src/tools/tasks/taskManagement.ts +180 -0
- package/src/tools/tasks/taskOutput.js +113 -0
- package/src/tools/tasks/taskOutput.ts +145 -0
- package/src/tools/tasks/taskStop.js +74 -0
- package/src/tools/tasks/taskStop.ts +99 -0
- package/src/tools/tasks/taskStore.js +46 -0
- package/src/tools/tasks/taskStore.ts +77 -0
- package/src/tools/terminal/terminalCapture.js +167 -0
- package/src/tools/terminal/terminalCapture.ts +179 -0
- package/src/tools/trash/trash.js +233 -0
- package/src/tools/trash/trash.ts +283 -0
- package/src/tools/types.js +1 -1
- package/src/tools/types.ts +24 -0
- package/src/tools/webbrowser/agentBrowserPath.ts +286 -0
- package/src/tools/webbrowser/browser.js +214 -11
- package/src/tools/webbrowser/browser.ts +293 -0
- package/src/tools/webbrowser/browserControl.js +153 -48
- package/src/tools/webbrowser/browserControl.ts +262 -0
- package/src/tools/webbrowser/browserDaemon.js +147 -39
- package/src/tools/webbrowser/browserDaemon.ts +270 -0
- package/src/tools/webbrowser/browserOutcome.js +273 -0
- package/src/tools/webbrowser/browserOutcome.ts +306 -0
- package/src/tools/webbrowser/browserRunRegistry.js +150 -0
- package/src/tools/webbrowser/browserRunRegistry.ts +179 -0
- package/src/tools/webbrowser/browserService.js +20 -6
- package/src/tools/webbrowser/browserService.ts +56 -0
- package/src/tools/webbrowser/browserStream.js +30 -1
- package/src/tools/webbrowser/browserStream.ts +216 -0
- package/src/tools/webbrowser/browserTabs.js +143 -0
- package/src/tools/webbrowser/browserTabs.ts +210 -0
- package/src/tools/webbrowser/webbrowser.js +831 -150
- package/src/tools/webbrowser/webbrowser.ts +1537 -0
- package/src/tools/webfetch/preapproved.ts +274 -0
- package/src/tools/webfetch/webfetch.js +24 -23
- package/src/tools/webfetch/webfetch.ts +383 -0
- package/src/tools/websearch/websearch.js +12 -11
- package/src/tools/websearch/websearch.ts +292 -0
- package/src/tools/worktree/sessionWorktree.js +128 -0
- package/src/tools/worktree/sessionWorktree.ts +158 -0
- package/src/tools/write/write.js +27 -44
- package/src/tools/write/write.ts +136 -0
- package/src/types.ts +960 -0
- package/src/ui/App.js +216 -12
- package/src/ui/App.tsx +879 -0
- package/src/ui/AskQuestion.tsx +300 -0
- package/src/ui/InputBox.js +77 -1
- package/src/ui/InputBox.tsx +567 -0
- package/src/ui/LiveArea.tsx +344 -0
- package/src/ui/MessageList.js +27 -6
- package/src/ui/MessageList.tsx +559 -0
- package/src/ui/ModelPicker.tsx +107 -0
- package/src/ui/QueuedMessagesBanner.tsx +51 -0
- package/src/ui/Root.js +6 -8
- package/src/ui/Root.tsx +100 -0
- package/src/ui/SessionPicker.tsx +169 -0
- package/src/ui/StatusLine.js +22 -1
- package/src/ui/StatusLine.tsx +145 -0
- package/src/ui/WorkspacePicker.tsx +84 -0
- package/src/ui/advisorCommand.js +54 -0
- package/src/ui/advisorCommand.ts +63 -0
- package/src/ui/computerCommand.js +19 -0
- package/src/ui/computerCommand.ts +35 -0
- package/src/ui/figures.ts +45 -0
- package/src/ui/hooks/askSlot.ts +112 -0
- package/src/ui/hooks/useChat.js +271 -7
- package/src/ui/hooks/useChat.ts +1578 -0
- package/src/ui/hooks/useInputHistory.ts +235 -0
- package/src/ui/hooks/usePasteHandler.ts +220 -0
- package/src/ui/hooks/useTerminalSize.ts +41 -0
- package/src/ui/hooks/useTextBuffer.ts +108 -0
- package/src/ui/hooks/useTokenUsage.ts +44 -0
- package/src/ui/liveViewport.ts +189 -0
- package/src/ui/markdown.ts +230 -0
- package/src/ui/textBuffer.ts +239 -0
- package/src/ui/theme.ts +35 -0
- package/src/utils/clipboardImage.js +358 -0
- package/src/utils/clipboardImage.ts +420 -0
- package/src/utils/debugLog.ts +50 -0
- package/src/utils/greenRoot.ts +35 -0
- package/src/utils/packageRoot.ts +40 -0
- package/src/utils/pathContains.js +28 -0
- package/src/utils/pathContains.ts +31 -0
- package/src/utils/resourcePaths.ts +71 -0
- package/src/utils/stripThink.ts +49 -0
- package/src/utils/zodToJson.ts +73 -0
- package/src/web/browserBridge.js +365 -66
- package/src/web/browserBridge.ts +672 -0
- package/src/web/protocol.ts +1804 -0
- package/vendor/computer-use/arm64-darwin/cu-helperd +0 -0
- package/workflows/ppt.yaml +13 -2
- package/workflows/wechat-reading-shots.yaml +315 -0
- package/workflows/word.yaml +11 -1
- package/skills/docx/scripts/office/helpers/__pycache__/__init__.cpython-314.pyc +0 -0
- package/skills/docx/scripts/office/helpers/__pycache__/merge_runs.cpython-314.pyc +0 -0
- package/skills/docx/scripts/office/helpers/__pycache__/simplify_redlines.cpython-314.pyc +0 -0
- package/skills/docx/scripts/office/validators/__pycache__/__init__.cpython-314.pyc +0 -0
- package/skills/docx/scripts/office/validators/__pycache__/base.cpython-314.pyc +0 -0
- package/skills/docx/scripts/office/validators/__pycache__/docx.cpython-314.pyc +0 -0
- package/skills/docx/scripts/office/validators/__pycache__/pptx.cpython-314.pyc +0 -0
- package/skills/docx/scripts/office/validators/__pycache__/redlining.cpython-314.pyc +0 -0
- package/skills/image-stats/scripts/__pycache__/image_stats.cpython-314.pyc +0 -0
- package/skills/image-stats/scripts/__pycache__/image_stats_test.cpython-314.pyc +0 -0
- package/skills/ui-design/scripts/__pycache__/assemble_material.cpython-314.pyc +0 -0
- package/skills/ui-design/scripts/__pycache__/assemble_material_test.cpython-314.pyc +0 -0
- package/skills/ui-design/scripts/__pycache__/chart_sanity_check.cpython-314.pyc +0 -0
- package/skills/ui-design/scripts/__pycache__/check_language_policy.cpython-314.pyc +0 -0
- package/skills/ui-design/scripts/__pycache__/check_ui_artifact.cpython-314.pyc +0 -0
- package/skills/ui-design/scripts/__pycache__/check_ui_artifact_pptx_test.cpython-314.pyc +0 -0
- package/skills/ui-design/scripts/__pycache__/deck_pages_check.cpython-314.pyc +0 -0
- package/skills/ui-design/scripts/__pycache__/deck_pages_check_test.cpython-314.pyc +0 -0
- package/skills/ui-design/scripts/__pycache__/deck_visual_style_check.cpython-314.pyc +0 -0
- package/skills/ui-design/scripts/__pycache__/deck_visual_style_check_test.cpython-314.pyc +0 -0
- package/skills/ui-design/scripts/__pycache__/design_budget_check.cpython-314.pyc +0 -0
- package/skills/ui-design/scripts/__pycache__/design_budget_check_test.cpython-314.pyc +0 -0
- package/skills/ui-design/scripts/__pycache__/design_repair_hints.cpython-314.pyc +0 -0
- package/skills/ui-design/scripts/__pycache__/design_repair_hints_test.cpython-314.pyc +0 -0
- package/skills/ui-design/scripts/__pycache__/design_style_check.cpython-314.pyc +0 -0
- package/skills/ui-design/scripts/__pycache__/design_style_check_test.cpython-314.pyc +0 -0
- package/skills/ui-design/scripts/__pycache__/draft_coverage_check.cpython-314.pyc +0 -0
- package/skills/ui-design/scripts/__pycache__/extract_source_assets.cpython-314.pyc +0 -0
- package/skills/ui-design/scripts/__pycache__/extract_source_assets_test.cpython-314.pyc +0 -0
- package/skills/ui-design/scripts/__pycache__/generated_image_review_collect.cpython-314.pyc +0 -0
- package/skills/ui-design/scripts/__pycache__/generated_image_review_collect_test.cpython-314.pyc +0 -0
- package/skills/ui-design/scripts/__pycache__/image_plan_lint.cpython-314.pyc +0 -0
- package/skills/ui-design/scripts/__pycache__/image_plan_lint_test.cpython-314.pyc +0 -0
- package/skills/ui-design/scripts/__pycache__/image_prompt_rewrite.cpython-314.pyc +0 -0
- package/skills/ui-design/scripts/__pycache__/image_prompt_rewrite_test.cpython-314.pyc +0 -0
- package/skills/ui-design/scripts/__pycache__/layout_balance.cpython-314.pyc +0 -0
- package/skills/ui-design/scripts/__pycache__/layout_balance_test.cpython-314.pyc +0 -0
- package/skills/ui-design/scripts/__pycache__/normalize_design_json.cpython-314.pyc +0 -0
- package/skills/ui-design/scripts/__pycache__/normalize_design_json_test.cpython-314.pyc +0 -0
- package/skills/ui-design/scripts/__pycache__/plan_source_layout.cpython-314.pyc +0 -0
- package/skills/ui-design/scripts/__pycache__/plan_source_layout_test.cpython-314.pyc +0 -0
- package/skills/ui-design/scripts/__pycache__/resolve_source_hint.cpython-314.pyc +0 -0
- package/skills/ui-design/scripts/__pycache__/resolve_source_hint_test.cpython-314.pyc +0 -0
- package/skills/ui-design/scripts/__pycache__/resolve_source_input.cpython-314.pyc +0 -0
- package/skills/ui-design/scripts/__pycache__/resolve_source_input_test.cpython-314.pyc +0 -0
- package/skills/ui-design/scripts/__pycache__/scaffold_ui_artifact.cpython-314.pyc +0 -0
- package/skills/ui-design/scripts/__pycache__/source_usage_check.cpython-314.pyc +0 -0
- package/skills/ui-design/scripts/__pycache__/source_usage_check_test.cpython-314.pyc +0 -0
- package/skills/ui-design/scripts/__pycache__/text_readability_pixel.cpython-314.pyc +0 -0
- package/skills/ui-design/scripts/__pycache__/visual_quality_gate.cpython-314.pyc +0 -0
- package/skills/ui-design/scripts/__pycache__/visual_quality_gate_test.cpython-314.pyc +0 -0
- /package/skills/ai-image/scripts/{generate_test.mjs → generate.selftest.mjs} +0 -0
- /package/skills/ai-image/scripts/{generate_from_plan_test.mjs → generate_from_plan.selftest.mjs} +0 -0
|
@@ -0,0 +1,413 @@
|
|
|
1
|
+
---
|
|
2
|
+
schemaVersion: 2
|
|
3
|
+
guideVersion: 4
|
|
4
|
+
status: layout-verified
|
|
5
|
+
platforms: [macOS, Windows]
|
|
6
|
+
match:
|
|
7
|
+
bundleId: [com.tencent.xinWeChat]
|
|
8
|
+
processName: [WeChat, 微信, WeChat.exe, Weixin.exe]
|
|
9
|
+
displayName: [WeChat, 微信]
|
|
10
|
+
irreversible:
|
|
11
|
+
critical:
|
|
12
|
+
- intent:payment-or-transfer
|
|
13
|
+
- intent:remove-contact
|
|
14
|
+
- intent:block-user
|
|
15
|
+
- intent:dissolve-group
|
|
16
|
+
- intent:leave-group-permanently
|
|
17
|
+
- intent:delete-account
|
|
18
|
+
- intent:clear-chat-history
|
|
19
|
+
normal:
|
|
20
|
+
- intent:send-message
|
|
21
|
+
- intent:send-file
|
|
22
|
+
- intent:send-media
|
|
23
|
+
- intent:create-group
|
|
24
|
+
- intent:add-friend
|
|
25
|
+
- intent:add-emoji
|
|
26
|
+
- intent:recall-message
|
|
27
|
+
- intent:set-mute
|
|
28
|
+
- intent:set-top
|
|
29
|
+
- intent:set-background
|
|
30
|
+
- intent:set-nickname
|
|
31
|
+
- intent:share-chat
|
|
32
|
+
verifiedRange:
|
|
33
|
+
macOS: WeChat 4.x identity and main-window layout
|
|
34
|
+
windows: not-verified
|
|
35
|
+
lastVerifiedAt: 2026-07-29
|
|
36
|
+
|
|
37
|
+
# ---- schema v2 结构化部分 ----
|
|
38
|
+
#
|
|
39
|
+
# ⚠️ 微信是**自绘应用**:无障碍树几乎为空(本仓库实测确认)。所以这里刻意
|
|
40
|
+
# **不写 rect、不写 ax 锚点** —— 写了就是编的,而编出来的锚点比没有更糟:
|
|
41
|
+
# 模型会拿它当事实去点。分区只声明"存在且大致在哪一侧",真实坐标必须
|
|
42
|
+
# 每次从当次观测里读。
|
|
43
|
+
|
|
44
|
+
# 这个应用专属的「该叫人来」信号。内核只认识跨应用通用的那些(密码框、验证码、
|
|
45
|
+
# 二维码登录…),微信的登录小窗**不暴露「扫码登录」四个字** —— AX 里只有
|
|
46
|
+
# 「二维码」和「仅传输文件」两个控件,必须两个一起出现才算数:单独一个「二维码」
|
|
47
|
+
# 很可能只是聊天里别人发的图,据此判定会把正常聊天误报成需要人工介入。
|
|
48
|
+
# ⚠️ 这段以前是写死在内核 humanBoundary.ts 里的一段 bundleId 判断。通用内核里
|
|
49
|
+
# 出现具体应用的 bundleId = 多支持一个应用就要改一次内核,与「drop 一份说明书
|
|
50
|
+
# 即扩展能力」的架构相反,因此搬到这里。
|
|
51
|
+
humanBoundary:
|
|
52
|
+
- reason: qr_login
|
|
53
|
+
summary: 需要你扫码完成微信登录
|
|
54
|
+
allOf:
|
|
55
|
+
- 二维码
|
|
56
|
+
- 仅传输文件
|
|
57
|
+
|
|
58
|
+
layout:
|
|
59
|
+
- id: nav_rail
|
|
60
|
+
desc: 最左侧**窄图标栏**(纯图标无文字:头像在最上,下面是聊天/通讯录/收藏等)
|
|
61
|
+
verified: true
|
|
62
|
+
- id: conversation_list
|
|
63
|
+
desc: 第二栏会话列表;顶部一行是搜索框 + 右侧「+」新建按钮,下方按最近活跃排序
|
|
64
|
+
verified: true
|
|
65
|
+
- id: chat_header
|
|
66
|
+
desc: 右侧聊天区顶部,显示当前聊天对象名称;群聊在名称后带括号成员数(如「童年 (7)」)
|
|
67
|
+
verified: true
|
|
68
|
+
- id: message_area
|
|
69
|
+
desc: 消息气泡区,全自绘、无控件信息,只能看图;自己发的在右、别人的在左带头像
|
|
70
|
+
verified: true
|
|
71
|
+
- id: compose_area
|
|
72
|
+
desc: 底部输入区;左下工具栏依次可见表情、扩展、文件夹、截图下拉与麦克风;⚠️ 注意:右下角的麦克风/语音图标不是发送按钮,微信 Mac 版默认无独立发送按钮,靠键盘 Enter 发送
|
|
73
|
+
verified: true
|
|
74
|
+
- id: chat_actions
|
|
75
|
+
desc: 聊天区右上角有消息/会话操作下拉、电话和更多菜单;具体图标会随单聊、群聊和版本变化
|
|
76
|
+
verified: true
|
|
77
|
+
|
|
78
|
+
controls: []
|
|
79
|
+
|
|
80
|
+
flows:
|
|
81
|
+
activate_wechat:
|
|
82
|
+
goal: 从其他应用切回微信主窗口
|
|
83
|
+
verified: false
|
|
84
|
+
steps:
|
|
85
|
+
- do: 优先用 activate_app 按 Bundle ID com.tencent.xinWeChat 激活微信
|
|
86
|
+
- do: observe 确认前台应用和三栏主窗口;若只出现登录、通话或小程序窗口,不继续操作聊天
|
|
87
|
+
- do: 仅在原生激活失败时,视觉定位 Dock 中绿色双气泡微信图标并单击,再重新 observe
|
|
88
|
+
pitfalls:
|
|
89
|
+
- Agent 操作期间宿主窗口必须隐藏且不可聚焦;若 KakadeAI、终端或其他用户窗口成为前台,立即暂停并交还用户,不能循环激活微信
|
|
90
|
+
- Dock 红色数字只是未读数量,不能用来确认当前打开了哪个会话
|
|
91
|
+
- Dock 图标下方黑点只表示应用正在运行,不表示主窗口已获得焦点
|
|
92
|
+
- Dock 可能自动隐藏、位于屏幕左侧或右侧,不能使用截图里的固定坐标
|
|
93
|
+
|
|
94
|
+
browse_conversations:
|
|
95
|
+
goal: 查看左侧会话列表,找到目标会话
|
|
96
|
+
verified: false
|
|
97
|
+
steps:
|
|
98
|
+
- do: 确认微信在前台且主窗口可见
|
|
99
|
+
precondition: foreground_app_is com.tencent.xinWeChat
|
|
100
|
+
- do: observe 一次,从截图里读出会话列表的实际位置与条目
|
|
101
|
+
- do: 需要下翻时在会话列表区域 scroll,每次翻动后重新 observe
|
|
102
|
+
pitfalls:
|
|
103
|
+
- 会话列表按活跃度实时重排,上一次观测的第 N 行下一秒可能已经是别人
|
|
104
|
+
- 未读红点与置顶会改变视觉顺序,不要用"第几个"来指代某个会话
|
|
105
|
+
|
|
106
|
+
search:
|
|
107
|
+
goal: 搜索联系人、群聊或聊天记录
|
|
108
|
+
verified: false
|
|
109
|
+
steps:
|
|
110
|
+
- do: 确认微信在前台
|
|
111
|
+
precondition: foreground_app_is com.tencent.xinWeChat
|
|
112
|
+
- do: 点击会话列表顶部的搜索框,observe 确认光标进去了
|
|
113
|
+
- do: |
|
|
114
|
+
按通用输入框规范,检查并清空残留内容后再输入关键词。
|
|
115
|
+
⛔ **原样输入联系人的中文名,绝对不要转成拼音或英文。**
|
|
116
|
+
微信搜索是字面匹配 —— 搜「王磊」能找到,搜「wang lei」什么都搜不到。
|
|
117
|
+
- do: 输入关键词后 observe 看结果分组(联系人/群聊/聊天记录分开列出)
|
|
118
|
+
- do: 点选目标条目,再 observe 确认右侧聊天区标题变成了预期对象
|
|
119
|
+
pitfalls:
|
|
120
|
+
- 搜索结果混排联系人、群聊和聊天记录三类,点错类别会打开完全不同的东西
|
|
121
|
+
- 同名联系人与同名群极常见,只靠名字选中就是在赌 —— 但「唯一同名联系人 + 若干群/聊天记录」
|
|
122
|
+
**不是**歧义:只有联系人分区里出现两条以上完全同名才需要停下问用户
|
|
123
|
+
- ⛔ **中文名不要自己转拼音。** 实测事故:要搜「王磊」,模型在输入框里改成了
|
|
124
|
+
「wang lei」,结果零结果、整个任务卡死。中文由工具负责送达:系统装着输入法时
|
|
125
|
+
它会**自动整串走剪贴板粘贴**并还原你原来的剪贴板(回执会说明走了哪条路),
|
|
126
|
+
不需要你换写法去绕 —— 换写法只会把一个工具已经处理好的问题变成一个搜不到的问题
|
|
127
|
+
|
|
128
|
+
send_message:
|
|
129
|
+
goal: 给指定联系人或群发一条文字消息
|
|
130
|
+
verified: false
|
|
131
|
+
steps:
|
|
132
|
+
- do: |
|
|
133
|
+
⛔ **必须在本次任务里亲自打开目标会话,不许沿用「打开着的那个」。**
|
|
134
|
+
即使微信当前已经停在某个会话上,也**必须**先走一遍 search 流程搜到目标联系人
|
|
135
|
+
并点开 —— 除非你在**本次任务的前几步里亲手点开的就是它**。
|
|
136
|
+
「界面上正开着一个聊天窗」不构成「它就是收件人」的任何证据。
|
|
137
|
+
- do: |
|
|
138
|
+
canonical observe,**逐字核对聊天区左上角的标题** = 目标联系人姓名。
|
|
139
|
+
截图与屏幕 1:1(引擎已把桌面切到 1920×1080,见 `ensure_model_friendly_resolution`),
|
|
140
|
+
姓名是可以看清的 —— 认真看,不要「觉得像」就算数:
|
|
141
|
+
「王鹏」和「王磊」只差一个字,发错无法挽回(真实事故)。
|
|
142
|
+
标题不是目标、或确实看不清 → 停下重新走 search,**绝对不许先打字**。
|
|
143
|
+
⚠️ 窗口标题栏永远是「微信」,不含聊天对象名 —— 引擎的 precondition
|
|
144
|
+
校验不了收件人,这一步只能靠你看截图,它是发错人的**唯一**防线。
|
|
145
|
+
precondition: foreground_app_is com.tencent.xinWeChat
|
|
146
|
+
- do: |
|
|
147
|
+
判断目标是否唯一。**只有真歧义才停下问用户**:
|
|
148
|
+
· 唯一命中 → 「联系人」分区里**只有一条**姓名与目标完全相同 → 直接选它,不要问。
|
|
149
|
+
同时出现的**群聊**、以及「包含:xxx」这类聊天记录匹配,**不算**同名冲突 ——
|
|
150
|
+
它们是别的类别,不是另一个同名的人。
|
|
151
|
+
· 真歧义 → 「联系人」分区里有**两条或以上**姓名完全相同 → 这时才停下,
|
|
152
|
+
让用户提供头像、备注名、微信号或共同群等可区分信息。
|
|
153
|
+
⚠️ 把「有个群名里含这个名字」当成同名冲突而停下,是实测发生过的事故:
|
|
154
|
+
搜「王磊」已经命中唯一联系人并高亮在「最常使用」第一条,模型却因为下面有个
|
|
155
|
+
「HOSSKY—AI+XR 包含:A 王磊」的群,判定无法确认对象,整个任务放弃。
|
|
156
|
+
- do: 点击底部输入区,observe 确认焦点已在输入框
|
|
157
|
+
- do: 按通用输入框规范,检查并清空残留内容(发新消息必须清空)
|
|
158
|
+
- do: 输入内容后 observe,逐字核对屏幕上显示的文字与要发的一致,确认输入法候选框已消失
|
|
159
|
+
- do: 确认当前用户设置采用哪个发送键;发送快捷键可被用户改,不能猜 Enter 或 Cmd+Enter;⚠️ macOS 微信 4.x 桌面版默认没有独立的发送按钮:输入框右下角是语音/麦克风图标不是发送按钮,默认按 Enter 发送,用户可改为 Cmd+Enter
|
|
160
|
+
- do: 发送
|
|
161
|
+
irreversible: true
|
|
162
|
+
# actionClass 是「这一步的前提该管哪次动作」的唯一桥梁:模型自报
|
|
163
|
+
# actionClass:intent:send-message 时,下面这条 precondition 才会被检查。
|
|
164
|
+
# 不写就等于这条前提永远不生效(见 manualSchema::unmetPreconditionsFor)。
|
|
165
|
+
actionClass: intent:send-message
|
|
166
|
+
precondition: foreground_app_is com.tencent.xinWeChat
|
|
167
|
+
- do: |
|
|
168
|
+
canonical observe:确认聊天标题仍是目标对象,且新消息出现在本人一侧(右侧气泡)。
|
|
169
|
+
⛔ 报告结果时**只能写你在截图上真看到的那个标题**。没核对过就写「已发给 XX」=
|
|
170
|
+
编造 —— 实测发生过(发给了王鹏,报告说发给了王磊)。无法确认时只报告状态,
|
|
171
|
+
禁止重复发送。
|
|
172
|
+
pitfalls:
|
|
173
|
+
- 发送快捷键可被用户改(Enter 或 Cmd+Enter),不能假定哪个是发送、哪个是换行
|
|
174
|
+
- ⚠️ macOS 微信 4.x 桌面版默认没有独立的发送按钮!输入框右下角是语音/麦克风图标,不是发送按钮,不要点击它!
|
|
175
|
+
- 输入法在合成中时按发送键只会上屏候选词,必须先确认候选框已消失
|
|
176
|
+
- 微信消息发出后 2 分钟内才可撤回,且对方仍会看到"撤回了一条消息"
|
|
177
|
+
- ⛔ **发错对象无法挽回(撤回只有 2 分钟,且对方仍看得到「撤回了一条消息」)。**
|
|
178
|
+
实测事故(真机 trace + 截图为证):任务是「给王磊发多运动」,微信当时停在**王鹏**的
|
|
179
|
+
会话上,模型 observe 之后**没有搜索、没有核对标题**,直接 click 输入框 → type → 回车,
|
|
180
|
+
消息发给了王鹏;然后它向用户报告「已成功给联系人**王磊**发送消息」。
|
|
181
|
+
整条链只有 4 个动作,搜索联系人那一整步被跳过了。
|
|
182
|
+
⚠️ 当时的根因是**物理上看不清**:桌面还是 4K,截图缩小约 3.7 倍,姓名只剩约 8 像素高,
|
|
183
|
+
裁出来放大 6 倍看第二个字完全糊成一团 —— 那张图里**不含**区分这两个字的信息。
|
|
184
|
+
现已根治:引擎在每次 observe 前把桌面切到 1920×1080 并 1:1 送图(见
|
|
185
|
+
`macos.rs::ensure_model_friendly_resolution`)。但**「不许沿用打开着的会话」这条依然成立** ——
|
|
186
|
+
看得清不等于看过,跳过搜索直接打字仍然是发错人的路。
|
|
187
|
+
两个教训:① 「屏幕上开着一个会话」≠「它是收件人」;
|
|
188
|
+
② **没核对过就不许在完成报告里写收件人名字** —— 那是编的,不是观察到的
|
|
189
|
+
- 登录、二维码、风险提示、验证码或微信异常退出都属于人工介入边界;停止自动动作,不关闭提示、不重复启动、不尝试绕过
|
|
190
|
+
|
|
191
|
+
group_chat:
|
|
192
|
+
goal: 在群聊里交互(看成员、@ 某人、查看群公告)
|
|
193
|
+
verified: false
|
|
194
|
+
steps:
|
|
195
|
+
- do: 打开目标群会话并 observe 确认标题与成员数
|
|
196
|
+
precondition: foreground_app_is com.tencent.xinWeChat
|
|
197
|
+
- do: 需要看成员或群设置时,点聊天区右上角进入群信息面板,再 observe
|
|
198
|
+
- do: 需要 @ 某人时在输入区输入 @ 触发成员候选框,observe 后从列表里选
|
|
199
|
+
pitfalls:
|
|
200
|
+
- 输入 @ 会弹出成员候选浮层,此时直接按发送键会选中候选而不是发消息
|
|
201
|
+
- 群成员昵称可被本人改动,同一个人在不同群显示不同名字
|
|
202
|
+
|
|
203
|
+
create_group:
|
|
204
|
+
goal: 创建一个新群聊
|
|
205
|
+
verified: false
|
|
206
|
+
steps:
|
|
207
|
+
- do: 确认微信在前台
|
|
208
|
+
precondition: foreground_app_is com.tencent.xinWeChat
|
|
209
|
+
- do: 从会话列表顶部的新建入口发起群聊,observe 确认选人面板已打开
|
|
210
|
+
- do: 逐个搜索并勾选成员,每勾一个后 observe 核对已选列表
|
|
211
|
+
- do: 确认创建
|
|
212
|
+
irreversible: true
|
|
213
|
+
actionClass: intent:create-group
|
|
214
|
+
precondition: foreground_app_is com.tencent.xinWeChat
|
|
215
|
+
pitfalls:
|
|
216
|
+
- 建群会立刻给所有被选中的人发出通知,选错人就是打扰到了真人
|
|
217
|
+
- 三人及以上才成群;只选一个人会直接开成单聊而不是群
|
|
218
|
+
|
|
219
|
+
add_friend:
|
|
220
|
+
goal: 添加好友
|
|
221
|
+
verified: false
|
|
222
|
+
steps:
|
|
223
|
+
- do: 确认微信在前台
|
|
224
|
+
precondition: foreground_app_is com.tencent.xinWeChat
|
|
225
|
+
- do: 在搜索框输入微信号/手机号,observe 确认搜索结果确实是目标本人
|
|
226
|
+
- do: 打开其资料页,observe 核对头像、微信号、地区等可见信息
|
|
227
|
+
- do: 发送好友申请
|
|
228
|
+
irreversible: true
|
|
229
|
+
actionClass: intent:add-friend
|
|
230
|
+
precondition: foreground_app_is com.tencent.xinWeChat
|
|
231
|
+
pitfalls:
|
|
232
|
+
- 好友申请一旦发出无法撤回,对方会收到通知
|
|
233
|
+
- 搜索手机号会把该号码暴露给对方(对方能看到"通过手机号搜索添加")
|
|
234
|
+
- 加错人属于隐私事故,不属于"重试一次"能解决的错误
|
|
235
|
+
|
|
236
|
+
read_chat_history:
|
|
237
|
+
goal: 向上翻阅当前会话的历史消息(只读)
|
|
238
|
+
verified: false
|
|
239
|
+
steps:
|
|
240
|
+
- do: 先用 search 流程打开目标会话,observe 核对聊天区顶部标题确实是目标对象
|
|
241
|
+
precondition: foreground_app_is com.tencent.xinWeChat
|
|
242
|
+
- do: 用 scroll 翻页,x/y 必须框在 **message_area 中心**(聊天气泡区中间,
|
|
243
|
+
大约屏幕横向 55%~75%、纵向 30%~60% 一带),direction 用 up。
|
|
244
|
+
系统会先把鼠标移到该点再滚轮,所以坐标必须落在消息区里面,
|
|
245
|
+
落在窗口边缘或别的应用上就滚错地方了。
|
|
246
|
+
- do: 每滚一次都重新 observe,对比气泡内容确认**真的往上翻了**;
|
|
247
|
+
回执里的变化率看的是「目标区域」,别拿全屏动静当数
|
|
248
|
+
- do: 若回执提示目标区域没有变化,改用键盘:先 click 一下 message_area 中心让它获得焦点,
|
|
249
|
+
再用 key 发 pageup(整页上翻)。键盘事件走焦点控件,不依赖鼠标位置,
|
|
250
|
+
在微信这种自绘应用上常常比滚轮更稳。需要一次到底可用 home。
|
|
251
|
+
- do: 读到足够条数或已到最早一条(顶部不再出现新内容)就停止,按屏幕可见内容如实汇总
|
|
252
|
+
pitfalls:
|
|
253
|
+
- ⛔ **绝对不要用 drag 拖滚动条,也不要去点右侧那条细边**。微信的滚动条是隐藏的、
|
|
254
|
+
只有几像素宽;拖偏或点偏就落在消息气泡上 —— 那不是滚动,是**框选文本**,
|
|
255
|
+
属于误操作,而且看起来还像"成功了"。要翻页只有两条路:scroll + 坐标,或 click + pageup。
|
|
256
|
+
- message_area 全自绘、无障碍树为空:翻没翻动**只能靠对比截图里的气泡内容**判断,
|
|
257
|
+
不能靠"动作返回成功"
|
|
258
|
+
- 滚轮事件投递给**鼠标所在**的视图。坐标给到左侧会话列表,滚的就是会话列表 ——
|
|
259
|
+
实测发生过:连滚多次,右侧聊天记录一行没动、左侧列表滚了一大截
|
|
260
|
+
- 历史很长时不要一次滚太多再 observe:跨度太大会跳过中间内容,宁可小步多滚
|
|
261
|
+
- 只读流程:全程不得点击输入区、发送键或任何会改动会话的入口
|
|
262
|
+
|
|
263
|
+
read_contact_profile:
|
|
264
|
+
goal: 读取好友资料
|
|
265
|
+
verified: false
|
|
266
|
+
steps:
|
|
267
|
+
- do: 用 search 流程找到该联系人,或从会话中点开对方头像
|
|
268
|
+
precondition: foreground_app_is com.tencent.xinWeChat
|
|
269
|
+
- do: observe 资料页,只读取屏幕上可见的信息
|
|
270
|
+
pitfalls:
|
|
271
|
+
- 资料页只读;不要点"设置备注和标签"之类会改动对方条目的入口
|
|
272
|
+
- 备注名与昵称是两个不同的东西,报告给用户时要说清读到的是哪个
|
|
273
|
+
|
|
274
|
+
transfer_files:
|
|
275
|
+
goal: 收发文件
|
|
276
|
+
verified: false
|
|
277
|
+
steps:
|
|
278
|
+
- do: 打开目标会话并核对标题
|
|
279
|
+
precondition: foreground_app_is com.tencent.xinWeChat
|
|
280
|
+
- do: 发送时用 file_drop 把文件送进输入区,observe 确认输入区出现了文件卡片
|
|
281
|
+
- do: 确认发送
|
|
282
|
+
irreversible: true
|
|
283
|
+
actionClass: intent:send-file
|
|
284
|
+
precondition: foreground_app_is com.tencent.xinWeChat
|
|
285
|
+
- do: 接收时点消息里的文件,observe 确认下载完成后再去读本地路径
|
|
286
|
+
pitfalls:
|
|
287
|
+
- file_drop 不是真拖拽(macOS 不允许合成),走剪贴板或文件面板;必须 observe 核实文件真的到了
|
|
288
|
+
- 大文件与受限类型会被拒绝,错误提示只出现在界面上,动作本身不会报错
|
|
289
|
+
- 文件发出后同样不可撤回
|
|
290
|
+
|
|
291
|
+
verification:
|
|
292
|
+
environment: macOS / WeChat 4.x, 2048×1024 main-window screenshot plus macOS Dock screenshot
|
|
293
|
+
verifiedAt: 2026-07-31
|
|
294
|
+
# 只确认截图直接证明的分区、可见图标与相对位置,不含坐标和可点击性。
|
|
295
|
+
verifiedItems: [identity, layout, visible-controls, dock-appearance]
|
|
296
|
+
# ★ 流程步骤仍是公开帮助文档的转写,**没有一条实机跑过**。
|
|
297
|
+
# 引擎注入时会带"未实机验证"标注,别把它当成已验证的操作序列。
|
|
298
|
+
unverifiedItems: [accessibility-anchors, flows, windows-layout]
|
|
299
|
+
---
|
|
300
|
+
|
|
301
|
+
# 微信 / WeChat App Guide
|
|
302
|
+
|
|
303
|
+
## 稳定身份
|
|
304
|
+
|
|
305
|
+
### macOS
|
|
306
|
+
|
|
307
|
+
- Bundle ID:`com.tencent.xinWeChat`
|
|
308
|
+
- 进程名候选:`WeChat`、`微信`
|
|
309
|
+
- 常见显示名:`WeChat`、`微信`
|
|
310
|
+
|
|
311
|
+
### Windows
|
|
312
|
+
|
|
313
|
+
- 可执行文件名候选:`WeChat.exe`、`Weixin.exe`
|
|
314
|
+
- Windows 身份尚未实机验证,不能仅凭显示名判定版本或进程。
|
|
315
|
+
|
|
316
|
+
## AX / UIA 可发现线索
|
|
317
|
+
|
|
318
|
+
- **macOS AX 候选线索**:主窗口、搜索、会话列表、聊天标题、消息列表、输入区和工具栏应从当前 AX 快照确认。
|
|
319
|
+
- **Windows UIA 候选线索**:主窗口、搜索、会话列表、消息区域、编辑区和附件入口应从当前 UIA 树确认。
|
|
320
|
+
- **焦点通道可用层级**:macOS 与 Windows 均未实机验证。
|
|
321
|
+
- 发送快捷键可由用户设置改变,因此不能把 Enter 或组合键固定声明为发送动作。
|
|
322
|
+
|
|
323
|
+
## 常见自绘区
|
|
324
|
+
|
|
325
|
+
- 消息气泡、表情/贴纸、图片与视频预览、语音消息、小程序、通话和二维码区域可能只有部分 Accessibility 信息,需视觉兜底;尚未实机验证。
|
|
326
|
+
|
|
327
|
+
## 风险动作与不可逆动作
|
|
328
|
+
|
|
329
|
+
- 发送文字、文件、图片、视频或名片。
|
|
330
|
+
- 撤回消息、清空聊天记录、删除会话或移除联系人。
|
|
331
|
+
- 发起付款、转账、红包或任何金融动作。
|
|
332
|
+
- 对外发送前必须重新核对当前聊天对象和待发送内容;群名与联系人重名不能仅靠标题判断。
|
|
333
|
+
|
|
334
|
+
## 弹窗与恢复
|
|
335
|
+
|
|
336
|
+
| 弹窗或状态 | 可观察线索 | 安全恢复出口 |
|
|
337
|
+
|---|---|---|
|
|
338
|
+
| 扫码登录、验证码或设备确认 | 登录窗口、二维码或验证提示 | 交还用户完成,不读取或转发验证码 |
|
|
339
|
+
| 文件过大、类型受限或上传失败 | 发送前/发送中的错误提示 | 保留原文件,不重复发送,报告失败状态 |
|
|
340
|
+
| 联系人或群聊重名 | 搜索结果中出现多个同名项 | 停止选择,要求用户提供可区分身份 |
|
|
341
|
+
| 更新、通话或小程序窗口抢占前台 | 新窗口、浮层或全屏内容出现 | 关闭或退出后重新确认主窗口与聊天对象 |
|
|
342
|
+
| KakadeAI 或其他普通窗口抢占前台 | 前台 Bundle ID 不再是微信 | 自动接管并等待用户;不得由 Agent 循环激活微信 |
|
|
343
|
+
| 用户真实鼠标或键盘输入 | Helper 报告 human_input_detected | 撤销输入租约,等待用户点击“完成并继续”,随后 canonical observe |
|
|
344
|
+
| 风险提示或微信异常退出 | 安全提示、登录页、进程退出 | 立即停止本次自动化并保留现场,不自动重启或重试 |
|
|
345
|
+
| 网络断开或消息未送达 | 连接状态、失败标记或重试提示 | 不自动重复发送,先确认服务恢复与消息状态 |
|
|
346
|
+
|
|
347
|
+
## 可替代 API
|
|
348
|
+
|
|
349
|
+
- 无已验证的个人微信桌面消息官方自动化 API。
|
|
350
|
+
- 企业微信或组织系统应优先使用其正式开放接口,但不能把企业接口权限外推到个人微信。
|
|
351
|
+
- 本机文件选择和内容校验优先使用文件工具;最终发送仍属于微信内的外部动作。
|
|
352
|
+
|
|
353
|
+
## 界面布局
|
|
354
|
+
|
|
355
|
+
主窗口是自左向右三段:**最左竖排功能栏 → 会话列表 → 聊天区**。聊天区自身
|
|
356
|
+
再分三段:顶部标题(当前聊天对象)、中间消息气泡区、底部输入区。
|
|
357
|
+
|
|
358
|
+
这三段的**存在与相对位置**是稳定的,**具体坐标不是** —— 窗口可拉伸、可全屏、
|
|
359
|
+
可隐藏侧栏。所以 frontmatter 的 `layout` 只声明分区语义,不声明 rect:
|
|
360
|
+
每次动手前的坐标只能来自当次 observe。
|
|
361
|
+
|
|
362
|
+
聊天区顶部标题是发送前最可靠的对象核对点;左侧选中态只能作为辅助证据。
|
|
363
|
+
截图中群聊标题采用“名称 (成员数)”形式,例如“童年 (7)”。发送前必须重新
|
|
364
|
+
observe 核对标题,不能沿用上一个动作前的判断。
|
|
365
|
+
|
|
366
|
+
## 截图确认的 macOS 4.x 控件
|
|
367
|
+
|
|
368
|
+
- 会话列表顶部是搜索框,右侧是圆形“+”入口。
|
|
369
|
+
- 聊天区右上角依次可见会话操作下拉、电话和更多菜单;单聊与群聊可能不同。
|
|
370
|
+
- 输入区左下可见表情、扩展、文件夹、截图下拉和麦克风;右下有独立语音/播放图标。
|
|
371
|
+
- 左侧功能栏与会话列表会显示未读红点;红点与列表顺序都不是联系人身份。
|
|
372
|
+
- Dock 中微信是绿色双气泡图标,红色角标表示未读,图标下黑点表示正在运行。
|
|
373
|
+
|
|
374
|
+
这些是视觉外观证据,不是稳定坐标或 AX 标识。窗口缩放、侧栏宽度、显示器比例、
|
|
375
|
+
Dock 位置和微信升级都会改变坐标;每次操作仍应以当次 observe 为准。
|
|
376
|
+
|
|
377
|
+
## 典型流程速查
|
|
378
|
+
|
|
379
|
+
frontmatter 的 `flows` 是权威版本(引擎会直接注入给你,不必来这里读)。
|
|
380
|
+
这一节只补充两条贯穿所有流程的原则:
|
|
381
|
+
|
|
382
|
+
1. **每个动作之后 observe。** 微信没有可用的控件树,动作的退出码只说明
|
|
383
|
+
"事件发出去了",说明不了界面变成了什么样。
|
|
384
|
+
2. **不可逆动作前重新核对对象。** 从打开会话到按下发送之间,界面可能因为
|
|
385
|
+
新消息、来电弹窗、别的应用抢前台而变化。上一次核对的结论不能跨动作复用。
|
|
386
|
+
|
|
387
|
+
## 实测记录
|
|
388
|
+
|
|
389
|
+
> **这一节是 agent 的自我完善区。** 说明书与实况不符时(写着的分区找不到、
|
|
390
|
+
> 流程步骤对不上、precondition 反复失败),把**观察到的事实**追加到这里,
|
|
391
|
+
> 一条一行,带日期与应用版本。
|
|
392
|
+
>
|
|
393
|
+
> 只写观察到的,不写推测。确认稳定后由人提升进 frontmatter 并标
|
|
394
|
+
> `verified: true` —— 单次观察不能直接升级为永久事实。
|
|
395
|
+
|
|
396
|
+
- 2026-07-30 · WeChat 4.x / macOS 26.5.2 · 一次真实观测 + 一张屏幕截图:
|
|
397
|
+
- **AX 树整棵只有 19 个节点**(`axNodes:19`,且 `truncated:false` —— 不是被截断,
|
|
398
|
+
是微信真的只暴露这么点)。这坐实了头注那句「自绘应用 AX 基本是空的」:
|
|
399
|
+
`wait_for_element` / `textTarget` / `focused_role_is` 在微信上**都用不了**,
|
|
400
|
+
只有 `wait_for_window` 和看图可用。
|
|
401
|
+
- 布局确认为三栏:窄图标栏 → 会话列表(顶部搜索框 + 右侧「+」)→ 聊天区
|
|
402
|
+
(标题 / 气泡区 / 输入区)。群聊标题形如「童年 (7)」,括号内是成员数。
|
|
403
|
+
- 观测耗时 411ms,backend=native,`frames:1`(采样器刚起来时只有一帧)。
|
|
404
|
+
- ⚠️ 这次观测**没有走完任何一条 flow** —— 上面 `flows` 仍全部未验证。
|
|
405
|
+
|
|
406
|
+
## 已验证范围
|
|
407
|
+
|
|
408
|
+
- **验证环境**:macOS 26.5.2,WeChat 4.1.11。
|
|
409
|
+
- **已验证**:应用包 Bundle ID 和版本元数据。
|
|
410
|
+
- **仍需验证**:macOS AX 层级、Windows 身份/UIA、搜索、会话选择、发送、附件、登录和网络恢复。
|
|
411
|
+
- **frontmatter 里的 `layout` 与 `flows` 全部未实机验证** —— 它们是把面向人的
|
|
412
|
+
帮助文档转写成面向 agent 的形式,不是跑出来的结论。
|
|
413
|
+
- 不声明 Windows 版本或 WeChat 4.1.11 的交互流程已受支持。
|