@salesforce/afv-skills 1.36.0 → 1.38.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/package.json +1 -1
- package/skills/agentforce-generate/README.md +20 -3
- package/skills/agentforce-generate/SKILL.md +253 -440
- package/skills/agentforce-generate/assets/agent-spec-template.md +6 -2
- package/skills/agentforce-generate/assets/agents/order-service.agent +16 -12
- package/skills/agentforce-generate/assets/agents/production-faq.agent +4 -4
- package/skills/agentforce-generate/assets/agents/router-first.agent +5 -5
- package/skills/agentforce-generate/assets/agents/template-single-subagent.agent +4 -4
- package/skills/agentforce-generate/assets/agents/verification-gate.agent +8 -6
- package/skills/agentforce-generate/assets/agents/voice-knowledge-grounded.agent +9 -9
- package/skills/agentforce-generate/assets/agents/voice-service-agent.agent +7 -7
- package/skills/agentforce-generate/assets/patterns/README.md +3 -3
- package/skills/agentforce-generate/assets/patterns/action-callbacks.agent +5 -5
- package/skills/agentforce-generate/assets/patterns/advanced-input-bindings.agent +6 -8
- package/skills/agentforce-generate/assets/patterns/bidirectional-routing.agent +11 -13
- package/skills/agentforce-generate/assets/patterns/critical-input-collection.agent +11 -16
- package/skills/agentforce-generate/assets/patterns/lifecycle-events.agent +2 -3
- package/skills/agentforce-generate/assets/patterns/llm-controlled-actions.agent +6 -7
- package/skills/agentforce-generate/assets/patterns/open-gate-routing.agent +3 -3
- package/skills/agentforce-generate/assets/patterns/prompt-template-action.agent +14 -19
- package/skills/agentforce-generate/assets/patterns/system-instruction-overrides.agent +11 -20
- package/skills/agentforce-generate/references/actions-reference.md +2 -2
- package/skills/agentforce-generate/references/agent-audit-and-repair.md +135 -0
- package/skills/agentforce-generate/references/agent-audit-candidate-verification.md +160 -0
- package/skills/agentforce-generate/references/agent-audit-diagnostic-catalog.md +156 -0
- package/skills/agentforce-generate/references/agent-audit-diagnostics-actions-state.md +176 -0
- package/skills/agentforce-generate/references/agent-audit-diagnostics-architecture-evaluation.md +68 -0
- package/skills/agentforce-generate/references/agent-audit-diagnostics-instructions-routing.md +283 -0
- package/skills/agentforce-generate/references/agent-audit-evaluation-loop.md +191 -0
- package/skills/agentforce-generate/references/agent-audit-repair-report.md +143 -0
- package/skills/agentforce-generate/references/agent-audit-scope-path-review.md +180 -0
- package/skills/agentforce-generate/references/agent-design-and-spec-creation.md +105 -59
- package/skills/agentforce-generate/references/agent-script-core-language.md +144 -61
- package/skills/agentforce-generate/references/agent-subagent-map-diagrams.md +33 -23
- package/skills/agentforce-generate/references/agent-validation-and-debugging.md +37 -11
- package/skills/agentforce-generate/references/agentscript-toolchain.md +112 -0
- package/skills/agentforce-generate/references/architecture-patterns.md +81 -18
- package/skills/agentforce-generate/references/common-control-flow-pitfalls.md +255 -0
- package/skills/agentforce-generate/references/control-flow-actions-sequencing.md +198 -0
- package/skills/agentforce-generate/references/control-flow-lifecycle-side-effects.md +87 -0
- package/skills/agentforce-generate/references/examples.md +22 -22
- package/skills/agentforce-generate/references/instruction-resolution.md +123 -83
- package/skills/agentforce-generate/references/known-issues.md +1 -2
- package/skills/agentforce-generate/references/optimization-pattern-1-data-flow.md +4 -0
- package/skills/agentforce-generate/references/optimization-pattern-2-deterministic-logic.md +33 -0
- package/skills/agentforce-generate/references/optimization-pattern-3-reference-syntax.md +32 -3
- package/skills/agentforce-generate/references/optimization-pattern-4-escalation.md +2 -2
- package/skills/agentforce-generate/references/patterns-by-requirement.md +3 -1
- package/skills/agentforce-generate/references/posture-and-determinism.md +103 -22
- package/skills/agentforce-generate/references/reference-map.md +19 -3
- package/skills/agentforce-generate/references/scoring-rubric.md +1 -1
- package/skills/agentforce-generate/references/voice-latency-heuristics.md +4 -0
- package/skills/agentforce-generate/references/voice-modality-reference.md +4 -4
- package/skills/agentforce-generate/references/zen-of-agentscript.md +139 -23
- package/skills/agentforce-generate/scripts/agentscript-sdk-loader.mjs +133 -0
- package/skills/agentforce-generate/scripts/index-agent.mjs +141 -0
- package/skills/agentforce-generate/scripts/setup-agentscript-sdk.mjs +337 -0
- package/skills/automation-sandbox-post-copy-config-generate/SKILL.md +1 -1
- package/skills/automation-sandbox-post-copy-configure/SKILL.md +433 -0
- package/skills/automation-sandbox-post-copy-configure/assets/api_request_templates.json +89 -0
- package/skills/automation-sandbox-post-copy-configure/examples/sample_config_input.json +40 -0
- package/skills/automation-sandbox-post-copy-configure/examples/sample_execution_summary.md +79 -0
- package/skills/automation-sandbox-post-copy-configure/references/api_endpoints.md +273 -0
- package/skills/automation-sandbox-post-copy-configure/references/authentication.md +94 -0
- package/skills/automation-sandbox-post-copy-configure/references/execution_phasing.md +93 -0
- package/skills/automation-sandbox-post-copy-configure/references/rules_gotchas.md +35 -0
- package/skills/automation-sandbox-post-copy-configure/scripts/classify-patch-result.mjs +88 -0
- package/skills/automation-sandbox-post-copy-configure/scripts/map-metadata-key.mjs +98 -0
- package/skills/automation-sandbox-post-copy-configure/scripts/plan-phases.mjs +98 -0
- package/skills/automation-sandbox-post-copy-configure/scripts/resolve-target-org.mjs +56 -0
- package/skills/design-systems-slds-validate/SKILL.md +14 -13
- package/skills/dx-code-analyzer-custom-rule-create/examples/xpath-examples.md +1 -1
- package/skills/dx-code-analyzer-custom-rule-create/references/xpath-patterns-security.md +1 -1
- package/skills/dx-org-devhub-configure/SKILL.md +268 -0
- package/skills/dx-org-devhub-configure/examples/status-output.md +39 -0
- package/skills/dx-org-devhub-configure/scripts/devhub.sh +650 -0
- package/skills/dx-org-devhub-configure/scripts/test-devhub.sh +116 -0
- package/skills/dx-org-manage/SKILL.md +135 -42
- package/skills/dx-org-manage/assets/derive-alias.sh +95 -0
- package/skills/dx-org-manage/assets/scratch-def.seed.json +6 -0
- package/skills/dx-org-manage/examples/README.md +1 -1
- package/skills/dx-org-manage/examples/scratch-orgs/delete_output.json +8 -0
- package/skills/dx-org-manage/examples/scratch-orgs/display_output.json +23 -0
- package/skills/dx-org-manage/examples/scratch-orgs/list_output.json +58 -0
- package/skills/dx-org-manage/examples/scratch-orgs/resume_output.json +38 -0
- package/skills/dx-org-manage/examples/scratch-orgs/success_definition_file.json +1 -1
- package/skills/dx-org-manage/examples/scratch-orgs/success_edition.json +1 -1
- package/skills/dx-org-manage/examples/scratch-orgs/success_shape.json +41 -0
- package/skills/dx-org-manage/examples/scratch-orgs/success_snapshot.json +1 -1
- package/skills/dx-org-manage/examples/snapshots/error_output.json +3 -3
- package/skills/dx-org-manage/references/creating-scratch-org.md +2 -2
- package/skills/dx-org-manage/references/creating-snapshot.md +1 -2
- package/skills/dx-org-manage/references/definition_file_options.md +24 -0
- package/skills/dx-org-manage/references/edition_types.md +10 -8
- package/skills/dx-org-manage/references/opening-org.md +11 -12
- package/skills/dx-org-manage/references/scratch-org-create.md +303 -0
- package/skills/dx-org-manage/references/scratch-org-operations.md +135 -0
- package/skills/experience-aura-lwc-migrate/SKILL.md +120 -0
- package/skills/experience-aura-lwc-migrate/references/aura-api-expert.md +170 -0
- package/skills/experience-aura-lwc-migrate/references/aura-data-expert.md +172 -0
- package/skills/experience-aura-lwc-migrate/references/aura-migration-guidelines.md +299 -0
- package/skills/experience-aura-lwc-migrate/references/aura-prd-framework.md +79 -0
- package/skills/experience-aura-lwc-migrate/references/aura-redundant-code-expert.md +24 -0
- package/skills/experience-aura-lwc-migrate/references/aura-reference-expert.md +140 -0
- package/skills/experience-aura-lwc-migrate/references/aura-resolver-expert.md +115 -0
- package/skills/experience-aura-lwc-migrate/references/aura-slots-expert.md +67 -0
- package/skills/experience-aura-lwc-migrate/references/aura-style-expert.md +62 -0
- package/skills/experience-aura-lwc-migrate/references/aura-to-lwc-completeness-checklist.md +188 -0
- package/skills/experience-aura-lwc-migrate/references/aura-values-expert.md +67 -0
- package/skills/experience-content-media-search/SKILL.md +17 -12
- package/skills/experience-content-media-stock-image-search/SKILL.md +192 -0
- package/skills/experience-content-media-stock-image-search/scripts/download-stock-image.py +102 -0
- package/skills/experience-lds-best-practices-apply/SKILL.md +245 -0
- package/skills/experience-lds-best-practices-apply/references/adapter-apis.md +1640 -0
- package/skills/experience-lds-best-practices-apply/references/lds-data-consistency.md +126 -0
- package/skills/experience-lds-best-practices-apply/references/lds-expert.md +429 -0
- package/skills/experience-lds-best-practices-apply/references/lds-referential-integrity.md +322 -0
- package/skills/experience-lds-best-practices-apply/references/wire-adapter-types.md +1511 -0
- package/skills/experience-lds-graphql-generate/SKILL.md +222 -0
- package/skills/experience-lds-graphql-generate/references/generation-guide.md +236 -0
- package/skills/experience-lds-graphql-generate/references/generation-mutation.md +277 -0
- package/skills/experience-lds-graphql-generate/references/generation-query.md +237 -0
- package/skills/experience-lds-graphql-generate/scripts/fetch-lds-graphql-schema.sh +230 -0
- package/skills/experience-lds-graphql-generate/scripts/test-lds-graphql-query.sh +128 -0
- package/skills/experience-lwc-accessibility-validate/SKILL.md +112 -0
- package/skills/experience-lwc-accessibility-validate/references/reviewers/sc-1-1-1-non-text-content.md +88 -0
- package/skills/experience-lwc-accessibility-validate/references/reviewers/sc-1-3-1-i-lists.md +52 -0
- package/skills/experience-lwc-accessibility-validate/references/reviewers/sc-1-3-1-ii-tables.md +106 -0
- package/skills/experience-lwc-accessibility-validate/references/reviewers/sc-1-3-1-iii-form-labels.md +78 -0
- package/skills/experience-lwc-accessibility-validate/references/reviewers/sc-1-3-1-iv-regions.md +26 -0
- package/skills/experience-lwc-accessibility-validate/references/reviewers/sc-1-3-1-v-groups.md +66 -0
- package/skills/experience-lwc-accessibility-validate/references/reviewers/sc-1-3-5-identify-input.md +70 -0
- package/skills/experience-lwc-accessibility-validate/references/reviewers/sc-1-4-3-contrast.md +115 -0
- package/skills/experience-lwc-accessibility-validate/references/reviewers/sc-2-1-1-keyboard.md +47 -0
- package/skills/experience-lwc-accessibility-validate/references/reviewers/sc-2-4-4-link-purpose.md +39 -0
- package/skills/experience-lwc-accessibility-validate/references/reviewers/sc-2-4-6-headings-labels.md +50 -0
- package/skills/experience-lwc-accessibility-validate/references/reviewers/sc-2-5-1-pointer-gestures.md +54 -0
- package/skills/experience-lwc-accessibility-validate/references/reviewers/sc-2-5-2-pointer-cancellation.md +49 -0
- package/skills/experience-lwc-accessibility-validate/references/reviewers/sc-2-5-3-label-in-name.md +77 -0
- package/skills/experience-lwc-accessibility-validate/references/reviewers/sc-2-5-7-dragging-movement.md +45 -0
- package/skills/experience-lwc-accessibility-validate/references/reviewers/sc-3-2-1-on-focus.md +55 -0
- package/skills/experience-lwc-accessibility-validate/references/reviewers/sc-3-2-2-on-input.md +51 -0
- package/skills/experience-lwc-accessibility-validate/references/reviewers/sc-3-3-1-error-identification.md +84 -0
- package/skills/experience-lwc-accessibility-validate/references/reviewers/sc-3-3-2-labels-instructions.md +50 -0
- package/skills/experience-lwc-accessibility-validate/references/reviewers/sc-3-3-3-error-suggestion.md +64 -0
- package/skills/experience-lwc-accessibility-validate/references/reviewers/sc-4-1-2-i-name.md +105 -0
- package/skills/experience-lwc-accessibility-validate/references/reviewers/sc-4-1-2-ii-role.md +99 -0
- package/skills/experience-lwc-accessibility-validate/references/reviewers/sc-4-1-2-iii-value.md +97 -0
- package/skills/experience-lwc-accessibility-validate/references/vision/sc-1-1-1-non-text-content.md +83 -0
- package/skills/experience-lwc-accessibility-validate/references/vision/sc-1-4-1-use-of-color.md +62 -0
- package/skills/experience-lwc-accessibility-validate/references/vision/sc-1-4-10-resize-reflow.md +18 -0
- package/skills/experience-lwc-accessibility-validate/references/vision/sc-1-4-11-non-text-contrast.md +43 -0
- package/skills/experience-lwc-accessibility-validate/references/vision/sc-1-4-3-contrast.md +25 -0
- package/skills/experience-lwc-accessibility-validate/scripts/contrast-ratio.py +153 -0
- package/skills/experience-lwc-design-generate/SKILL.md +261 -0
- package/skills/experience-lwc-design-generate/references/figma-to-prd-blueprint.md +66 -0
- package/skills/experience-lwc-design-generate/references/prd-analysis-template.md +17 -0
- package/skills/experience-lwc-design-generate/scripts/check-component-name.sh +75 -0
- package/skills/experience-lwc-design-generate/scripts/detect-project-tools.sh +97 -0
- package/skills/experience-lwc-runtime-observe/SKILL.md +244 -0
- package/skills/experience-lwc-runtime-observe/examples/component-preview-and-dom.md +41 -0
- package/skills/experience-lwc-runtime-observe/scripts/extract-dom.sh +79 -0
- package/skills/experience-lwc-runtime-observe/scripts/open-frontdoor.sh +46 -0
- package/skills/experience-lwc-runtime-observe/scripts/verify-toolchain.sh +48 -0
- package/skills/experience-lwc-security-validate/SKILL.md +151 -0
- package/skills/experience-lwc-security-validate/examples/review-report.md +6 -0
- package/skills/experience-lwc-security-validate/examples/score-report.sarif.json +32 -0
- package/skills/experience-lwc-security-validate/references/lws-security-expert.md +986 -0
- package/skills/experience-lwc-security-validate/references/security-analysis.md +611 -0
- package/skills/experience-lwc-security-validate/scripts/check-lwc-import.sh +166 -0
- package/skills/experience-lwc-security-validate/scripts/validate-sarif.sh +131 -0
- package/skills/experience-lwr-site-generate/SKILL.md +1 -2
- package/skills/experience-ui-bundle-2gp-deploy/SKILL.md +454 -0
- package/skills/experience-ui-bundle-2gp-deploy/assets/CustomApplication.app-meta.xml +12 -0
- package/skills/experience-ui-bundle-2gp-deploy/assets/PermissionSet.permissionset-meta.xml +9 -0
- package/skills/experience-ui-bundle-2gp-deploy/scripts/find-bundle-package-dir.sh +53 -0
- package/skills/experience-ui-bundle-agentforce-client-generate/SKILL.md +104 -26
- package/skills/experience-ui-bundle-agentforce-client-generate/references/constraints.md +35 -24
- package/skills/experience-ui-bundle-agentforce-client-generate/references/examples.md +92 -1
- package/skills/experience-ui-bundle-agentforce-client-generate/references/style-tokens.md +2 -0
- package/skills/experience-ui-bundle-agentforce-client-generate/references/troubleshooting.md +14 -1
- package/skills/experience-ui-bundle-agentforce-client-generate/scripts/detect-framework.sh +67 -0
- package/skills/experience-ui-bundle-deploy/SKILL.md +83 -14
- package/skills/experience-ui-bundle-deploy/assets/Communities.settings-meta.xml +19 -0
- package/skills/experience-ui-bundle-deploy/assets/org-setup.config.template.json +5 -0
- package/skills/experience-ui-bundle-deploy/assets/social-login-auth-providers.apex +158 -0
- package/skills/experience-ui-bundle-deploy/references/config-scaffold.md +13 -1
- package/skills/experience-ui-bundle-deploy/references/social-login.md +179 -0
- package/skills/experience-ui-bundle-frontend-generate/SKILL.md +14 -0
- package/skills/experience-ui-bundle-mfa-configure/SKILL.md +6 -5
- package/skills/experience-ui-bundle-mfa-configure/references/social-login.md +15 -7
- package/skills/experience-ui-bundle-salesforce-data-access/SKILL.md +41 -5
- package/skills/experience-ui-bundle-salesforce-data-access/references/caching.md +10 -22
- package/skills/experience-ui-bundle-salesforce-data-access/references/graphql-hand-authoring.md +4 -19
- package/skills/experience-ui-bundle-salesforce-data-access/references/migration.md +5 -0
- package/skills/experience-ui-bundle-salesforce-data-access/references/sdk-api.md +33 -150
- package/skills/mobile-platform-native-capabilities-integrate/references/nfc.md +35 -0
- package/skills/platform-apex-generate/SKILL.md +8 -7
- package/skills/platform-apex-test-generate/SKILL.md +3 -1
- package/skills/platform-apex-test-run/SKILL.md +9 -8
- package/skills/platform-custom-field-generate/SKILL.md +7 -1
- package/skills/platform-lightning-app-coordinate/SKILL.md +3 -3
- package/skills/platform-lightning-type-widget-coordinate/SKILL.md +1 -1
- package/skills/platform-lightning-type-widget-coordinate/examples/existing-lightning-type-with-widget-prompt.md +1 -1
- package/skills/platform-lightning-type-widget-coordinate/examples/new-lightning-type-with-widget-prompt.md +1 -1
- package/skills/platform-lightning-type-widget-coordinate/references/validation-gates.md +3 -3
- package/skills/platform-mcp-tool-widget-coordinate/SKILL.md +43 -67
- package/skills/platform-mcp-tool-widget-coordinate/examples/action-name-source-prompt.md +1 -2
- package/skills/platform-mcp-tool-widget-coordinate/examples/apex-invocable-source-prompt.md +2 -3
- package/skills/platform-mcp-tool-widget-coordinate/examples/nested-object-list-source-prompt.md +163 -0
- package/skills/platform-mcp-tool-widget-coordinate/examples/nested-object-single-source-prompt.md +198 -0
- package/skills/platform-mcp-tool-widget-coordinate/examples/pasted-tool-output-prompt.md +0 -1
- package/skills/platform-mcp-tool-widget-coordinate/references/build-plan-format.md +17 -11
- package/skills/platform-mcp-tool-widget-coordinate/references/mcp-tool-output-discovery.md +48 -15
- package/skills/platform-mcp-tool-widget-coordinate/references/two-clt-modeling.md +73 -10
- package/skills/platform-mcp-tool-widget-coordinate/references/validation-gates.md +48 -8
- package/skills/platform-policy-rule-generate/SKILL.md +22 -12
- package/skills/platform-policy-rule-generate/references/deploy-errors.md +1 -1
- package/skills/platform-policy-rule-generate/references/policy-schema-full.md +8 -29
- package/skills/platform-policy-rule-generate/references/templates-advanced.md +1 -1
- package/skills/platform-report-generate/SKILL.md +1 -1
- package/skills/platform-sharing-owd-configure/SKILL.md +20 -8
- package/skills/platform-sharing-owd-configure/references/access_levels.md +14 -1
- package/skills/platform-sharing-rules-generate/SKILL.md +67 -35
- package/skills/platform-sharing-rules-generate/examples/create-cases.md +16 -17
- package/skills/platform-sharing-rules-generate/examples/delete-cases.md +34 -79
- package/skills/platform-sharing-rules-generate/examples/edit-cases.md +13 -26
- package/skills/platform-sharing-rules-generate/scripts/count-remaining-rules.sh +33 -0
- package/skills/platform-value-set-generate/SKILL.md +6 -2
- package/skills/platform-widget-generate/SKILL.md +3 -5
- package/skills/platform-widget-generate/examples/conditional.json +18 -13
- package/skills/platform-widget-generate/examples/list-with-foreach.json +2 -2
- package/skills/platform-widget-generate/examples/single-object.json +2 -2
- package/skills/platform-widget-generate/references/schema-from-lightning-type.md +27 -6
- package/skills/platform-widget-generate/references/widget-bundle-layout.md +4 -3
- package/skills/service-itsm-agentic-setup-cmdb-access-assign/SKILL.md +287 -0
- package/skills/service-itsm-agentic-setup-cmdb-access-assign/references/mcp-invocation.md +260 -0
- package/skills/service-itsm-agentic-setup-cmdb-bundle-deploy/SKILL.md +252 -0
- package/skills/service-itsm-agentic-setup-cmdb-bundle-deploy/references/mcp-invocation.md +204 -0
- package/skills/service-itsm-agentic-setup-cmdb-configure/SKILL.md +259 -0
- package/skills/service-itsm-agentic-setup-cmdb-configure/references/mcp-invocation.md +188 -0
- package/skills/service-itsm-agentic-setup-cmdb-coordinate/SKILL.md +197 -0
- package/skills/service-itsm-agentic-setup-cmdb-coordinate/examples/output-templates.md +77 -0
- package/skills/service-itsm-agentic-setup-cmdb-discovery-configure/SKILL.md +316 -0
- package/skills/service-itsm-agentic-setup-cmdb-discovery-configure/references/mcp-invocation.md +221 -0
- package/skills/service-itsm-incident-priority-configure/SKILL.md +168 -0
- package/skills/service-itsm-incident-priority-configure/examples/matrix-operations.md +200 -0
- package/skills/service-itsm-incident-priority-configure/examples/render-matrix.md +54 -0
- package/skills/service-itsm-incident-priority-configure/examples/seed-full-matrix.md +52 -0
- package/skills/service-itsm-incident-priority-configure/references/sf-cli-invocation.md +264 -0
- package/skills/platform-mcp-tool-widget-coordinate/examples/nested-object-source-prompt.md +0 -191
- package/skills/platform-policy-rule-generate/references/fixtures-index.md +0 -29
|
@@ -0,0 +1,135 @@
|
|
|
1
|
+
# Audit and Repair Existing Agents
|
|
2
|
+
|
|
3
|
+
Repair an existing AgentScript agent without replacing its intended behavior
|
|
4
|
+
with generic preferences.
|
|
5
|
+
|
|
6
|
+
Org-backed validation requires an Agentforce-enabled org and Salesforce CLI.
|
|
7
|
+
A bounded static review can proceed without org access, but must not be
|
|
8
|
+
reported as compiler or runtime validation.
|
|
9
|
+
|
|
10
|
+
## Contents
|
|
11
|
+
|
|
12
|
+
- [Operating Contract](#operating-contract)
|
|
13
|
+
- [Surface Preservation Gate](#surface-preservation-gate)
|
|
14
|
+
- [Related Skill Boundaries](#related-skill-boundaries)
|
|
15
|
+
- [Workflow references](#workflow-references)
|
|
16
|
+
|
|
17
|
+
## Operating Contract
|
|
18
|
+
|
|
19
|
+
1. **Use cases define correctness.** A checklist can reveal risk; it cannot
|
|
20
|
+
decide what the agent should do. Reconstruct the agent's intended use cases
|
|
21
|
+
before recommending or making behavioral changes.
|
|
22
|
+
2. **Diagnose before editing.** Record the baseline, exact evidence, affected
|
|
23
|
+
use cases, and smallest credible fix first.
|
|
24
|
+
3. **Make findings actionable.** Every finding must include a source location,
|
|
25
|
+
runtime consequence, affected use case, proposed change, and verification
|
|
26
|
+
method. Omit unsupported style opinions.
|
|
27
|
+
4. **Prefer the smallest repair.** Do not redesign the agent, add subagents,
|
|
28
|
+
add persistent state, or add universal ambiguity, off-topic, or human-help
|
|
29
|
+
behavior unless the use cases require it.
|
|
30
|
+
5. **Evaluate the control tradeoff.** Do not mechanically make every condition
|
|
31
|
+
deterministic or leave every condition to the model. More control improves
|
|
32
|
+
ordering and repeatability but costs flexibility, state, and maintenance.
|
|
33
|
+
More model latitude improves interpretation and recovery but makes behavior
|
|
34
|
+
probabilistic. Protect requirements whose failure cost warrants runtime
|
|
35
|
+
control; preserve model judgment where flexibility is valuable. When the
|
|
36
|
+
model needs a runtime value, inject it explicitly.
|
|
37
|
+
6. **Choose the intervention level.** After diagnosis, assess Surface,
|
|
38
|
+
Structural, and Rewrite; recommend the smallest sufficient level, explain
|
|
39
|
+
the alternatives and tradeoffs, and obtain user choice before Structural or
|
|
40
|
+
Rewrite work. Never broaden the repair silently.
|
|
41
|
+
7. **Compile locally, then validate against the target org.** Run the bundled
|
|
42
|
+
local compiler first. When an authenticated org is available, also use
|
|
43
|
+
`sf agent validate authoring-bundle --json` for target-org language
|
|
44
|
+
validation. Never claim language validity from a regex, a home-grown parser,
|
|
45
|
+
or prompt inspection. Treat diagnostics as evidence, not an optimization
|
|
46
|
+
score. Do not silence a diagnostic by violating documented AgentScript
|
|
47
|
+
syntax, removing intentional initialization, or changing behavior. Treat
|
|
48
|
+
reference lists and examples as guidance rather than exhaustive schemas.
|
|
49
|
+
When references, examples, the selected compiler, and the target validator
|
|
50
|
+
disagree, preserve an accepted existing construct, report the discrepancy,
|
|
51
|
+
and validate against the intended deployment target instead of deleting the
|
|
52
|
+
construct by inference.
|
|
53
|
+
8. **Compare like with like.** Run the same use cases and evaluators against
|
|
54
|
+
the unchanged baseline and candidate. Label them explicitly. For Structural
|
|
55
|
+
or Rewrite candidates, account for every material removal of a subagent,
|
|
56
|
+
route, experience-specific branch, action, default, or user-visible
|
|
57
|
+
response before claiming capabilities were preserved.
|
|
58
|
+
9. **Distinguish availability, invocation, execution, and effect.** A good
|
|
59
|
+
response or tool name does not prove that an external side effect occurred.
|
|
60
|
+
10. **Calibrate every claim.** Distinguish language/compiler contracts,
|
|
61
|
+
runtime-source behavior, observed trace behavior, empirical heuristics, and
|
|
62
|
+
authoring recommendations. Do not turn a single trace, runtime version, or
|
|
63
|
+
preferred design into a universal rule. Use causal or categorical wording
|
|
64
|
+
only when the evidence supports it. Compilation alone supports “compiles”
|
|
65
|
+
or “candidate for review.” Use “safe,” “behavior-preserving,” “ready to
|
|
66
|
+
ship,” or equivalent language only when relevant behavior has also been
|
|
67
|
+
evaluated; otherwise state what remains untested.
|
|
68
|
+
11. **Do not release.** This workflow authorizes local edits and proportionate
|
|
69
|
+
validation, not deployment, publication, activation, production execution,
|
|
70
|
+
or live consequential actions.
|
|
71
|
+
12. **Deliver useful work before exhaustive work.** Create a durable audit
|
|
72
|
+
ledger or report shell after the initial scan, rank supported findings by
|
|
73
|
+
observable harm, and update the artifact as evidence or repairs land. Do
|
|
74
|
+
not make a complete runtime theory, complete reference review, or complete
|
|
75
|
+
rewrite a prerequisite for the first useful deliverable.
|
|
76
|
+
13. **Bound investigation by decisions.** Follow a reference, source path, or
|
|
77
|
+
trace only when it can change a named finding, repair, or verification
|
|
78
|
+
decision. If a primary source and one relevant corroborating source do not
|
|
79
|
+
resolve a non-safety-critical semantic question, record the uncertainty
|
|
80
|
+
and proceed. If the uncertainty blocks a safe repair, deliver the ranked
|
|
81
|
+
findings and blocker instead of continuing an open-ended investigation.
|
|
82
|
+
14. **Carry the contract into delegated work.** Give each delegated pass a
|
|
83
|
+
bounded artifact or cause group, priority order, lookup-only references,
|
|
84
|
+
and explicit output path. Require the first checkpoint after its initial
|
|
85
|
+
scan and a useful partial report when blocked. Do not hand a delegate a
|
|
86
|
+
broad mandatory reading list or an all-or-nothing final deliverable.
|
|
87
|
+
|
|
88
|
+
## Surface Preservation Gate
|
|
89
|
+
|
|
90
|
+
For a Surface-only request, use the original artifact as the byte-preserving
|
|
91
|
+
baseline:
|
|
92
|
+
|
|
93
|
+
1. List accepted findings before editing. Each must identify the original
|
|
94
|
+
source text, evidence, concrete consequence, and exact intended edit.
|
|
95
|
+
2. Do not treat a draft as a generation task. Missing recommended fields,
|
|
96
|
+
optional messages, preferred formatting, or equivalent control-flow forms
|
|
97
|
+
are not repair findings by themselves.
|
|
98
|
+
3. Edit the existing bundle in place. Preserve its directory, API name,
|
|
99
|
+
filenames, metadata, and equivalent control-flow form unless an accepted
|
|
100
|
+
finding specifically requires changing one of them. Do not emit a renamed
|
|
101
|
+
candidate bundle for an ordinary repair.
|
|
102
|
+
4. Apply only edits on the accepted list. Do not improve tone, complete a
|
|
103
|
+
schema, reorder blocks, or normalize style opportunistically.
|
|
104
|
+
5. Diff the candidate against the baseline. Revert every hunk that cannot be
|
|
105
|
+
mapped one-to-one to an accepted finding. If a hunk contains both required
|
|
106
|
+
and unrelated cleanup, narrow it before delivery.
|
|
107
|
+
6. Validation is evidence, not permission to widen the repair. Do not change
|
|
108
|
+
block-scalar style, normalize equivalent control flow, or migrate otherwise
|
|
109
|
+
preserved metadata merely so an org validator can run; record the
|
|
110
|
+
validation limit and keep the unrelated bytes unchanged.
|
|
111
|
+
|
|
112
|
+
This gate does not prevent evidence-backed Surface fixes. It prevents an
|
|
113
|
+
authoring preference from silently expanding a bounded repair.
|
|
114
|
+
|
|
115
|
+
Use [Diagnostic Catalog](agent-audit-diagnostic-catalog.md) as a lookup after
|
|
116
|
+
the initial scan; do not preload it. Consult the relevant section of
|
|
117
|
+
[Evaluation Loop](agent-audit-evaluation-loop.md) before recording a baseline,
|
|
118
|
+
applying a repair, or accepting a candidate; do not preload sections for later
|
|
119
|
+
steps.
|
|
120
|
+
|
|
121
|
+
## Related Skill Boundaries
|
|
122
|
+
|
|
123
|
+
- Use the parent skill's create or modify task domain for a new agent or one
|
|
124
|
+
already-specified edit.
|
|
125
|
+
- Use `agentforce-observe` when production session or trace evidence is the
|
|
126
|
+
primary input.
|
|
127
|
+
- Use `agentforce-test` when the task is only to author or run a predefined
|
|
128
|
+
functional or security test suite.
|
|
129
|
+
|
|
130
|
+
## Workflow references
|
|
131
|
+
|
|
132
|
+
Read both workflow references in order for a full audit or repair:
|
|
133
|
+
|
|
134
|
+
1. [Audit Scope and Path Review](agent-audit-scope-path-review.md) — establish scope, scale large audits, reconstruct use cases, and inspect reachable paths.
|
|
135
|
+
2. [Audit Repair and Report](agent-audit-repair-report.md) — select the intervention level, preserve a baseline, repair, evaluate, and report.
|
|
@@ -0,0 +1,160 @@
|
|
|
1
|
+
# AgentScript Audit Candidate Verification
|
|
2
|
+
|
|
3
|
+
Continue here after freezing the comparison, recording the baseline, and
|
|
4
|
+
applying one coherent repair.
|
|
5
|
+
|
|
6
|
+
## Contents
|
|
7
|
+
|
|
8
|
+
- [Validate the candidate](#5-validate-the-candidate)
|
|
9
|
+
- [Inspect more than the final text](#6-inspect-more-than-the-final-text)
|
|
10
|
+
- [Evaluate reasoning boundaries](#7-evaluate-reasoning-boundaries)
|
|
11
|
+
- [Decide whether to keep the repair](#8-decide-whether-to-keep-the-repair)
|
|
12
|
+
- [Handle regressions](#9-handle-regressions)
|
|
13
|
+
- [Report limitations](#10-report-limitations)
|
|
14
|
+
|
|
15
|
+
## 5. Validate the candidate
|
|
16
|
+
|
|
17
|
+
Run in this order:
|
|
18
|
+
|
|
19
|
+
1. repository checks;
|
|
20
|
+
2. `sf agent validate authoring-bundle --json`;
|
|
21
|
+
3. the new regression case;
|
|
22
|
+
4. affected existing cases;
|
|
23
|
+
5. representative unaffected canaries;
|
|
24
|
+
6. the complete frozen matrix before final acceptance.
|
|
25
|
+
|
|
26
|
+
Use the same model, runtime, action mode, evaluator, and test data as baseline.
|
|
27
|
+
|
|
28
|
+
## 6. Inspect more than the final text
|
|
29
|
+
|
|
30
|
+
A response can look correct while the agent used the wrong mechanism. Evaluate:
|
|
31
|
+
|
|
32
|
+
```text
|
|
33
|
+
configured
|
|
34
|
+
-> available
|
|
35
|
+
-> invoked
|
|
36
|
+
-> executed
|
|
37
|
+
-> returned
|
|
38
|
+
-> stored
|
|
39
|
+
-> transitioned
|
|
40
|
+
-> effected
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
Examples:
|
|
44
|
+
|
|
45
|
+
- “I transferred you” does not prove `@utils.escalate` executed.
|
|
46
|
+
- A tool invocation does not prove its external write succeeded.
|
|
47
|
+
- A raw JSON result does not prove derived boolean state was stored.
|
|
48
|
+
- A `checked=True` flag does not prove every branch input is usable.
|
|
49
|
+
- A self-transition can prove a new reasoning iteration began, but not that the
|
|
50
|
+
eventual external action effected its target.
|
|
51
|
+
|
|
52
|
+
## 7. Evaluate reasoning boundaries
|
|
53
|
+
|
|
54
|
+
When a repair introduces a reasoning boundary, inspect both sides.
|
|
55
|
+
|
|
56
|
+
Before the boundary:
|
|
57
|
+
|
|
58
|
+
- the producer runs once;
|
|
59
|
+
- the raw result is stored;
|
|
60
|
+
- completion remains false;
|
|
61
|
+
- no downstream branch reads default derived fields;
|
|
62
|
+
- the transition or stage exit is guarded against repetition.
|
|
63
|
+
|
|
64
|
+
After the boundary:
|
|
65
|
+
|
|
66
|
+
- the effective prompt is rebuilt from the stored result;
|
|
67
|
+
- only result-processing guidance is active;
|
|
68
|
+
- the model-visible raw value is explicitly injected when needed;
|
|
69
|
+
- one grouped state-update action requests all related fields and completion;
|
|
70
|
+
- success or failure actions are unavailable until the grouped update has
|
|
71
|
+
produced the state required by their gates.
|
|
72
|
+
|
|
73
|
+
A grouped state-update call expresses one semantic state change. Keep
|
|
74
|
+
downstream actions unavailable until their complete required state is present;
|
|
75
|
+
the grouping does not make model parsing deterministic.
|
|
76
|
+
|
|
77
|
+
For a guarded self-transition, add regression cases for:
|
|
78
|
+
|
|
79
|
+
- successful re-entry;
|
|
80
|
+
- failed or malformed result;
|
|
81
|
+
- no infinite loop;
|
|
82
|
+
- second independent request after reset;
|
|
83
|
+
- stale prior result not reused.
|
|
84
|
+
|
|
85
|
+
Treat the self-transition as an explicit phase boundary, not as evidence that
|
|
86
|
+
looping is generally desirable.
|
|
87
|
+
|
|
88
|
+
## 8. Decide whether to keep the repair
|
|
89
|
+
|
|
90
|
+
Keep the change only when:
|
|
91
|
+
|
|
92
|
+
- the target case improves;
|
|
93
|
+
- no critical or high-severity case regresses;
|
|
94
|
+
- relevant compile diagnostics are no worse;
|
|
95
|
+
- unaffected canaries are no worse;
|
|
96
|
+
- no new unauthorized action becomes available;
|
|
97
|
+
- no simulated result is presented as proof of a live effect;
|
|
98
|
+
- the candidate stays within the selected intervention level;
|
|
99
|
+
- every material removal in Structural or Rewrite work is preserved elsewhere
|
|
100
|
+
or recorded as an accepted behavior change.
|
|
101
|
+
|
|
102
|
+
If results vary, run repeated trials with the same setup and report the
|
|
103
|
+
distribution. Do not hide variance in an average.
|
|
104
|
+
|
|
105
|
+
## 9. Handle regressions
|
|
106
|
+
|
|
107
|
+
When a candidate regresses:
|
|
108
|
+
|
|
109
|
+
1. identify the smallest repair group responsible;
|
|
110
|
+
2. narrow or revert that group;
|
|
111
|
+
3. keep the evaluator frozen;
|
|
112
|
+
4. rerun structural checks;
|
|
113
|
+
5. rerun the affected case and all previously passing regression cases.
|
|
114
|
+
|
|
115
|
+
Do not:
|
|
116
|
+
|
|
117
|
+
- weaken an expected outcome after seeing the candidate fail;
|
|
118
|
+
- delete a failing case without showing it is outside the contract;
|
|
119
|
+
- change from live to simulated actions to obtain a pass;
|
|
120
|
+
- combine unrelated cleanup with a behavioral repair;
|
|
121
|
+
- declare improvement from aggregate score while a critical case regresses.
|
|
122
|
+
|
|
123
|
+
Stop and report a blocker when evaluation requires unavailable org access,
|
|
124
|
+
missing action implementations, production-only side effects, or a material
|
|
125
|
+
product-policy decision.
|
|
126
|
+
|
|
127
|
+
For a large agent, use bounded repair batches:
|
|
128
|
+
|
|
129
|
+
```text
|
|
130
|
+
indexed first pass
|
|
131
|
+
-> ranked ledger checkpoint
|
|
132
|
+
-> select highest-impact supported cause group
|
|
133
|
+
-> one small, coherent cause group
|
|
134
|
+
-> full compiler check
|
|
135
|
+
-> affected regression cases
|
|
136
|
+
-> unaffected canary cases
|
|
137
|
+
-> keep, narrow, or revert
|
|
138
|
+
-> update the durable report
|
|
139
|
+
-> next ranked group
|
|
140
|
+
-> final cross-node check and frozen-matrix regression
|
|
141
|
+
```
|
|
142
|
+
|
|
143
|
+
## 10. Report limitations
|
|
144
|
+
|
|
145
|
+
State explicitly:
|
|
146
|
+
|
|
147
|
+
- artifact revisions compared;
|
|
148
|
+
- explicit versus inferred cases;
|
|
149
|
+
- compiler and Salesforce CLI version;
|
|
150
|
+
- simulated versus live cases;
|
|
151
|
+
- whether external effects were independently verified;
|
|
152
|
+
- branches that could not be executed;
|
|
153
|
+
- runtime, model, test-data, or evaluator differences;
|
|
154
|
+
- assessed intervention levels, recommendation, user choice, and whether the
|
|
155
|
+
candidate remained within that scope.
|
|
156
|
+
|
|
157
|
+
Use “not evaluated” instead of “passed” when evidence is unavailable.
|
|
158
|
+
Compilation alone supports “compiles” or “candidate for review.” Use “safe,”
|
|
159
|
+
“behavior-preserving,” “ready to ship,” or equivalent language only when the
|
|
160
|
+
relevant behavior has been evaluated; otherwise state what remains untested.
|
|
@@ -0,0 +1,156 @@
|
|
|
1
|
+
# AgentScript Diagnostic Catalog
|
|
2
|
+
|
|
3
|
+
Use this catalog after reconstructing the agent's use cases. A pattern is a
|
|
4
|
+
finding only when it has a source location, reachable consequence, affected
|
|
5
|
+
use case, minimal fix, and verification method.
|
|
6
|
+
|
|
7
|
+
This catalog is a lookup index, not a required cover-to-cover review. Start
|
|
8
|
+
with the artifact and its use cases, consult only categories that can confirm
|
|
9
|
+
or reject a named observation, and return to the audit ledger after each
|
|
10
|
+
relevant section.
|
|
11
|
+
|
|
12
|
+
## Contents
|
|
13
|
+
|
|
14
|
+
- [Post-diagnosis intervention levels](#post-diagnosis-intervention-levels)
|
|
15
|
+
- [Control posture and tradeoffs](#control-posture-and-tradeoffs)
|
|
16
|
+
- [Claim calibration](#claim-calibration)
|
|
17
|
+
- [Focused diagnostic references](#focused-diagnostic-references)
|
|
18
|
+
|
|
19
|
+
## Post-diagnosis intervention levels
|
|
20
|
+
|
|
21
|
+
Assess all three levels after ranking findings. Mark each level **warranted** or
|
|
22
|
+
**not warranted** from evidence, then recommend the smallest level that can
|
|
23
|
+
resolve the accepted findings.
|
|
24
|
+
|
|
25
|
+
### Surface
|
|
26
|
+
|
|
27
|
+
Use for local bugs, typos with runtime impact, simple best-practice violations,
|
|
28
|
+
and one-line or similarly bounded fixes. It may correct a predicate, action
|
|
29
|
+
description, variable reference, indentation error, reset, or isolated prompt
|
|
30
|
+
instruction without reorganizing the flow or changing its architecture.
|
|
31
|
+
|
|
32
|
+
Choose Surface when the intended structure and contracts remain sound. State
|
|
33
|
+
which deeper problems, if any, it intentionally leaves unresolved.
|
|
34
|
+
|
|
35
|
+
Surface is not schema completion or style normalization. Preserve valid
|
|
36
|
+
optional metadata and semantically equivalent control-flow shapes. Absence
|
|
37
|
+
from an abbreviated reference list or example does not establish that an
|
|
38
|
+
existing field is unsupported; use the selected compiler, target validator,
|
|
39
|
+
or concrete runtime evidence. Likewise, do not add a missing field without a
|
|
40
|
+
diagnostic and evidence for the correct value; a guessed value can change
|
|
41
|
+
deployment semantics even when it compiles.
|
|
42
|
+
|
|
43
|
+
### Structural
|
|
44
|
+
|
|
45
|
+
Use for limited reorganization or basic structure changes when local edits
|
|
46
|
+
cannot make sequencing, ownership, routing, or lifecycle behavior reliable. It
|
|
47
|
+
may reorder or regroup existing logic, clarify a phase boundary, consolidate
|
|
48
|
+
duplicated rules, or make a bounded state lifecycle explicit.
|
|
49
|
+
|
|
50
|
+
Classify by behavioral coupling, not line count. Several individually small
|
|
51
|
+
edits are Structural when they must change multiple producers, consumers, and
|
|
52
|
+
reset paths that jointly encode one phase. In particular, when overlapping
|
|
53
|
+
flags and a stage value describe the same lifecycle, local patches that leave
|
|
54
|
+
contradictory combinations or unclear ownership do not make the design
|
|
55
|
+
reliable. Consolidating that lifecycle is a Structural change. A single
|
|
56
|
+
missing producer, gate, or reset can still be Surface when the surrounding
|
|
57
|
+
state model remains coherent.
|
|
58
|
+
|
|
59
|
+
Do not choose Surface merely because it fixes the highest-severity symptoms.
|
|
60
|
+
If an accepted shared root cause still creates competing duties on the repaired
|
|
61
|
+
happy path, Surface is an incomplete stabilization, not the smallest sufficient
|
|
62
|
+
final repair. Recommend Structural and present Surface only as that bounded
|
|
63
|
+
stopgap.
|
|
64
|
+
|
|
65
|
+
Structural work preserves the agent's objectives and overall architecture. It
|
|
66
|
+
costs more regression testing because several paths or prompt-resolution
|
|
67
|
+
boundaries may change.
|
|
68
|
+
|
|
69
|
+
For a cross-turn lifecycle reorganization, verify normal continuation,
|
|
70
|
+
correction before commitment, cancellation or intent change, and reset or a
|
|
71
|
+
second independent request. These cases test the state ownership; they do not
|
|
72
|
+
justify adding persistent state when the flow does not otherwise need it.
|
|
73
|
+
|
|
74
|
+
### Rewrite
|
|
75
|
+
|
|
76
|
+
Use for a total redesign only when evidence shows the existing architecture
|
|
77
|
+
cannot safely or maintainably satisfy the intended use cases. It may replace
|
|
78
|
+
routing, state, subagent boundaries, or action contracts and therefore requires
|
|
79
|
+
a complete frozen-matrix regression and contract review.
|
|
80
|
+
|
|
81
|
+
Do not recommend Rewrite merely because the file is large or unfamiliar.
|
|
82
|
+
Explain why Surface and Structural changes are insufficient and identify the
|
|
83
|
+
behavioral and migration risks.
|
|
84
|
+
|
|
85
|
+
### Choice and scope control
|
|
86
|
+
|
|
87
|
+
For each level, report:
|
|
88
|
+
|
|
89
|
+
- findings and use cases it resolves;
|
|
90
|
+
- findings it leaves unresolved;
|
|
91
|
+
- expected diff and architecture scope;
|
|
92
|
+
- regression risk and evaluation burden;
|
|
93
|
+
- compatibility or migration tradeoffs.
|
|
94
|
+
|
|
95
|
+
Recommend one level, but let the user choose. A general request to fix findings
|
|
96
|
+
defaults to Surface. Obtain explicit user choice before Structural or Rewrite,
|
|
97
|
+
unless the user already selected that level in the request. Do not mix levels
|
|
98
|
+
silently; if the chosen level is insufficient, stop and request a broader
|
|
99
|
+
choice.
|
|
100
|
+
|
|
101
|
+
## Control posture and tradeoffs
|
|
102
|
+
|
|
103
|
+
Judge each flow on a spectrum from prompt-led to mixed to scripted. Do not
|
|
104
|
+
award determinism merely for being deterministic, and do not flag a staged
|
|
105
|
+
flow merely for having stages.
|
|
106
|
+
|
|
107
|
+
More runtime control improves ordering, repeatability, auditability, and
|
|
108
|
+
protection of consequential effects. It also adds state, lifecycle cases,
|
|
109
|
+
maintenance, and rigidity when the user corrects themselves, digresses, or
|
|
110
|
+
changes intent. More model latitude provides natural interpretation and
|
|
111
|
+
recovery, but exact action choice and ordering remain probabilistic.
|
|
112
|
+
|
|
113
|
+
For each disputed decision, record:
|
|
114
|
+
|
|
115
|
+
- the observable cost if the model makes the wrong or reordered choice;
|
|
116
|
+
- the conversational flexibility lost by locking the choice;
|
|
117
|
+
- whether a mixed design can let the model interpret intent while runtime
|
|
118
|
+
gates only the consequence;
|
|
119
|
+
- evidence from requirements, evaluations, or traces that justifies changing
|
|
120
|
+
the current posture.
|
|
121
|
+
|
|
122
|
+
Recommend more control only when its reliability benefit exceeds its
|
|
123
|
+
flexibility and lifecycle cost. Recommend less control only when doing so
|
|
124
|
+
preserves the required invariants. A stage that spans turns needs the recovery
|
|
125
|
+
paths relevant to its use cases; a short-lived guard does not automatically
|
|
126
|
+
need correction, cancellation, retry, and expiry machinery.
|
|
127
|
+
|
|
128
|
+
## Claim calibration
|
|
129
|
+
|
|
130
|
+
Classify the basis of each finding before choosing its wording:
|
|
131
|
+
|
|
132
|
+
| Basis | What it supports |
|
|
133
|
+
|---|---|
|
|
134
|
+
| Compiler or language contract | A statement scoped to the validated language/version |
|
|
135
|
+
| Runtime source and tests | A statement scoped to the inspected runtime/version |
|
|
136
|
+
| Repeated trace or evaluation evidence | An empirical reliability claim for the tested configuration |
|
|
137
|
+
| Single trace | What happened in that session, not a universal causal rule |
|
|
138
|
+
| Design analysis | A recommendation with explicit tradeoffs |
|
|
139
|
+
| No direct evidence | A hypothesis to verify, not a finding |
|
|
140
|
+
|
|
141
|
+
Search categorical terms such as `always`, `never`, `every`, `cannot`,
|
|
142
|
+
`guarantees`, `ends the turn`, and `must` in the draft report and proposed
|
|
143
|
+
guidance. Keep them only when the cited contract or invariant is equally
|
|
144
|
+
categorical. State uncertainty and version scope where relevant.
|
|
145
|
+
|
|
146
|
+
Do not infer causation from sequence alone. For example, a response after a
|
|
147
|
+
state update does not prove the state update ended the turn, and a workaround
|
|
148
|
+
that improved one trace does not prove why it worked.
|
|
149
|
+
|
|
150
|
+
## Focused diagnostic references
|
|
151
|
+
|
|
152
|
+
Load only the category needed for a supported finding:
|
|
153
|
+
|
|
154
|
+
- [Instruction and Routing Diagnostics](agent-audit-diagnostics-instructions-routing.md) — language validity, instruction resolution, variable visibility, prompt pseudo-code, user-facing behavior, routing, and HyperClassifier fit.
|
|
155
|
+
- [Action and State Diagnostics](agent-audit-diagnostics-actions-state.md) — action surfaces, output contracts, state lifecycle, turn sequencing, transitions, authority, and side effects.
|
|
156
|
+
- [Architecture and Evaluation Diagnostics](agent-audit-diagnostics-architecture-evaluation.md) — architecture density, evaluation integrity, and non-findings that should not trigger edits.
|
|
@@ -0,0 +1,176 @@
|
|
|
1
|
+
# AgentScript Action and State Diagnostics
|
|
2
|
+
|
|
3
|
+
Use these categories only after the audit identifies a relevant action,
|
|
4
|
+
contract, state, sequencing, transition, or authority concern.
|
|
5
|
+
|
|
6
|
+
## Contents
|
|
7
|
+
|
|
8
|
+
- [Action surface and contracts](#action-surface-and-contracts)
|
|
9
|
+
- [Outputs and trusted decisions](#outputs-and-trusted-decisions)
|
|
10
|
+
- [State and lifecycle](#state-and-lifecycle)
|
|
11
|
+
- [Turn sequencing](#turn-sequencing)
|
|
12
|
+
- [Transitions and message continuity](#transitions-and-message-continuity)
|
|
13
|
+
- [Authority and side effects](#authority-and-side-effects)
|
|
14
|
+
|
|
15
|
+
## Action surface and contracts
|
|
16
|
+
|
|
17
|
+
Check:
|
|
18
|
+
|
|
19
|
+
- an action is referenced but undefined, or defined but unreachable;
|
|
20
|
+
- availability is broader than the use case;
|
|
21
|
+
- an action remains available after success and can repeat;
|
|
22
|
+
- descriptions overpromise capability or conflict with eligibility;
|
|
23
|
+
- required inputs have no conversation, variable, or literal producer;
|
|
24
|
+
- outputs are declared but not captured, or captured but never consumed;
|
|
25
|
+
- implementation inputs and outputs disagree with the `.agent` contract;
|
|
26
|
+
- a utility action is treated as though it returned action outputs.
|
|
27
|
+
|
|
28
|
+
Fix:
|
|
29
|
+
|
|
30
|
+
- Gate availability with trusted state.
|
|
31
|
+
- Return typed outputs and bind them to named consumers.
|
|
32
|
+
- Align the contract with the real implementation.
|
|
33
|
+
- Remove dead actions only after proving they serve no intended use case.
|
|
34
|
+
|
|
35
|
+
Evaluate availability, invocation, execution, output, and effect separately.
|
|
36
|
+
Include negative cases where the action must be absent.
|
|
37
|
+
|
|
38
|
+
## Outputs and trusted decisions
|
|
39
|
+
|
|
40
|
+
Check:
|
|
41
|
+
|
|
42
|
+
- raw JSON or display text controls authorization, eligibility, routing, or
|
|
43
|
+
success;
|
|
44
|
+
- the model is asked to parse a value that should be typed;
|
|
45
|
+
- a “checked” flag becomes true before every downstream value is stored;
|
|
46
|
+
- stale structured values survive a new raw result;
|
|
47
|
+
- the prompt assumes stored state is visible without explicit injection;
|
|
48
|
+
- a `|` block is assumed to pause deterministic resolution so the model can act
|
|
49
|
+
before the next `set` or condition resolves.
|
|
50
|
+
|
|
51
|
+
Fix:
|
|
52
|
+
|
|
53
|
+
- Prefer typed outputs.
|
|
54
|
+
- Normalize raw output deterministically before setting completion.
|
|
55
|
+
- Request related result fields as one grouped semantic state update when
|
|
56
|
+
possible.
|
|
57
|
+
- If model-driven normalization is unavoidable, create an explicit reasoning
|
|
58
|
+
boundary: first produce and store the raw result, then rebuild reasoning from
|
|
59
|
+
that state and store all derived fields plus completion together.
|
|
60
|
+
- Treat a guarded self-transition as a last-resort phase boundary when a
|
|
61
|
+
deterministic producer resolves before a required model-selected
|
|
62
|
+
normalization call can occur, and no clearer supported stage or typed result
|
|
63
|
+
is available. Document its entry and exit guards.
|
|
64
|
+
|
|
65
|
+
Apply this two-phase rule:
|
|
66
|
+
|
|
67
|
+
```text
|
|
68
|
+
1. Resolve deterministic run, set, and if statements and assemble all | text.
|
|
69
|
+
2. Send the completed prompt to the model, which may call a tool or respond.
|
|
70
|
+
```
|
|
71
|
+
|
|
72
|
+
Evaluate:
|
|
73
|
+
|
|
74
|
+
- Test true, false, malformed, missing, and stale-result cases.
|
|
75
|
+
- Confirm the branch reads the current result, not a default.
|
|
76
|
+
- Inspect the effective prompt before and after the reasoning boundary.
|
|
77
|
+
|
|
78
|
+
## State and lifecycle
|
|
79
|
+
|
|
80
|
+
Check:
|
|
81
|
+
|
|
82
|
+
- state merely remembers dialogue that remains available in conversation
|
|
83
|
+
history and has no runtime consumer;
|
|
84
|
+
- `current_step`, `question_asked`, or similar variables imitate a workflow
|
|
85
|
+
engine without deterministic consumers;
|
|
86
|
+
- several booleans and a phase variable encode the same workflow position;
|
|
87
|
+
- reachable combinations of state have no coherent meaning;
|
|
88
|
+
- completion means attempted or initiated rather than effected;
|
|
89
|
+
- request-scoped state leaks into a later request;
|
|
90
|
+
- reset, retry, correction, cancellation, logout, or expiry semantics are
|
|
91
|
+
missing;
|
|
92
|
+
- `before_reasoning` unconditionally erases a legitimate prior-turn value;
|
|
93
|
+
- `after_reasoning` overwrites or advances state despite failure.
|
|
94
|
+
|
|
95
|
+
Fix:
|
|
96
|
+
|
|
97
|
+
- Remove state only when its control benefit does not justify its flexibility
|
|
98
|
+
and lifecycle cost.
|
|
99
|
+
- Prefer one explicit phase value over several overlapping asking/answered
|
|
100
|
+
latches when deterministic sequencing is truly required.
|
|
101
|
+
- Name state after the evidence it represents.
|
|
102
|
+
- Give every mutable variable a producer, consumer, and lifecycle
|
|
103
|
+
proportionate to how long it lives and what it controls.
|
|
104
|
+
|
|
105
|
+
Select the second-request, retry, cancellation, correction, and expiry paths
|
|
106
|
+
that can affect the changed state before it stops mattering. Assert state
|
|
107
|
+
transitions, not just response wording.
|
|
108
|
+
|
|
109
|
+
## Turn sequencing
|
|
110
|
+
|
|
111
|
+
Check:
|
|
112
|
+
|
|
113
|
+
- prompt text asks the model to call an action, while later deterministic
|
|
114
|
+
conditions assume that action has already run;
|
|
115
|
+
- one prompt asks a question and consumes the future answer immediately;
|
|
116
|
+
- several model-selected tools are required in an exact order;
|
|
117
|
+
- the model can return text instead of invoking a required action;
|
|
118
|
+
- post-action and first-entry instructions can resolve together because their
|
|
119
|
+
predicates are not mutually exclusive and no control boundary separates
|
|
120
|
+
them.
|
|
121
|
+
|
|
122
|
+
Fix:
|
|
123
|
+
|
|
124
|
+
- Give each branch one next outcome.
|
|
125
|
+
- Bind current-turn inputs directly to the real action when persistence is not
|
|
126
|
+
required.
|
|
127
|
+
- Use deterministic chaining, an explicit reasoning boundary, or one
|
|
128
|
+
purpose-built action that owns the sequence when order protects correctness.
|
|
129
|
+
- Do not describe `@utils.setVariables` as turn-ending or as guaranteeing
|
|
130
|
+
another reasoning iteration. It is a model-selected state-update tool whose
|
|
131
|
+
call updates state but does not itself define a turn boundary. The risk is
|
|
132
|
+
assuming that prompt text has already caused the call before the prompt is
|
|
133
|
+
sent.
|
|
134
|
+
|
|
135
|
+
Evaluate the actual multi-turn sequence and inspect each reasoning iteration.
|
|
136
|
+
|
|
137
|
+
## Transitions and message continuity
|
|
138
|
+
|
|
139
|
+
Check:
|
|
140
|
+
|
|
141
|
+
- the target subagent reprocesses the message that completed the source flow;
|
|
142
|
+
- transition state is incomplete for the target's arrival path;
|
|
143
|
+
- both source and target respond or perform the same work;
|
|
144
|
+
- a flow cannot return or exit when supported use cases require it;
|
|
145
|
+
- a generic “stay” rule traps genuine topic switches.
|
|
146
|
+
|
|
147
|
+
Fix:
|
|
148
|
+
|
|
149
|
+
- Define one owner for the arrival turn.
|
|
150
|
+
- Pass explicit state only when the target needs it.
|
|
151
|
+
- Make continuation and exit predicates reflect real use cases, not exhaustive
|
|
152
|
+
phrase lists.
|
|
153
|
+
|
|
154
|
+
Evaluate the entry, continuation, cancellation, pivot, and return cases that
|
|
155
|
+
the supported use cases can reach.
|
|
156
|
+
|
|
157
|
+
## Authority and side effects
|
|
158
|
+
|
|
159
|
+
Check:
|
|
160
|
+
|
|
161
|
+
- a consequential action is gated by model-inferred or user-claimed authority;
|
|
162
|
+
- confirmation exists only in prose;
|
|
163
|
+
- two actions both appear to perform the same external effect;
|
|
164
|
+
- idempotency is absent for repeatable writes, transfers, messages, or charges;
|
|
165
|
+
- completion is set before a typed success result;
|
|
166
|
+
- simulated execution is treated as proof of a live effect.
|
|
167
|
+
|
|
168
|
+
Fix:
|
|
169
|
+
|
|
170
|
+
- Gate on trusted authorization and confirmation evidence.
|
|
171
|
+
- Choose one side-effect owner.
|
|
172
|
+
- Use idempotency keys or returned identifiers where supported.
|
|
173
|
+
- Claim only the strongest effect the runtime can verify.
|
|
174
|
+
|
|
175
|
+
Evaluate unauthorized, unconfirmed, duplicate, failure, timeout, and retry
|
|
176
|
+
cases. Verify the external record only when live execution is authorized.
|