mustflow 2.116.3 → 2.117.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/package.json +1 -1
- package/templates/default/i18n.toml +145 -7
- package/templates/default/locales/en/.mustflow/skills/INDEX.md +121 -11
- package/templates/default/locales/en/.mustflow/skills/agent-eval-integrity-review/SKILL.md +62 -34
- package/templates/default/locales/en/.mustflow/skills/agent-execution-control-review/SKILL.md +102 -31
- package/templates/default/locales/en/.mustflow/skills/agent-memory-context-governance-review/SKILL.md +163 -0
- package/templates/default/locales/en/.mustflow/skills/agent-planning-recovery-review/SKILL.md +180 -0
- package/templates/default/locales/en/.mustflow/skills/agent-release-bundle-rollout-review/SKILL.md +181 -0
- package/templates/default/locales/en/.mustflow/skills/agent-runtime-isolation-review/SKILL.md +196 -0
- package/templates/default/locales/en/.mustflow/skills/agent-runtime-multi-worker-review/SKILL.md +180 -0
- package/templates/default/locales/en/.mustflow/skills/automation-investment-case-review/SKILL.md +173 -0
- package/templates/default/locales/en/.mustflow/skills/client-platform-strategy-review/SKILL.md +236 -0
- package/templates/default/locales/en/.mustflow/skills/credit-ledger-integrity-review/SKILL.md +8 -4
- package/templates/default/locales/en/.mustflow/skills/credit-monetization-integrity-review/SKILL.md +283 -0
- package/templates/default/locales/en/.mustflow/skills/desktop-commercial-distribution-review/SKILL.md +225 -0
- package/templates/default/locales/en/.mustflow/skills/external-prompt-injection-defense/SKILL.md +49 -3
- package/templates/default/locales/en/.mustflow/skills/freemium-ad-monetization-review/SKILL.md +196 -0
- package/templates/default/locales/en/.mustflow/skills/game-economy-monetization-review/SKILL.md +208 -0
- package/templates/default/locales/en/.mustflow/skills/game-liveops-commerce-integrity-review/SKILL.md +237 -0
- package/templates/default/locales/en/.mustflow/skills/growth-distribution-integrity-review/SKILL.md +247 -0
- package/templates/default/locales/en/.mustflow/skills/idempotency-integrity-review/SKILL.md +20 -2
- package/templates/default/locales/en/.mustflow/skills/llm-model-routing-integrity-review/SKILL.md +183 -0
- package/templates/default/locales/en/.mustflow/skills/llm-product-monetization-review/SKILL.md +311 -0
- package/templates/default/locales/en/.mustflow/skills/llm-token-cost-control-review/SKILL.md +15 -1
- package/templates/default/locales/en/.mustflow/skills/localization-market-expansion-review/SKILL.md +224 -0
- package/templates/default/locales/en/.mustflow/skills/multi-agent-work-coordination/SKILL.md +7 -2
- package/templates/default/locales/en/.mustflow/skills/pricing-model-integrity-review/SKILL.md +288 -0
- package/templates/default/locales/en/.mustflow/skills/product-engagement-retention-review/SKILL.md +234 -0
- package/templates/default/locales/en/.mustflow/skills/product-onboarding-activation-review/SKILL.md +269 -0
- package/templates/default/locales/en/.mustflow/skills/product-portfolio-integrity-review/SKILL.md +233 -0
- package/templates/default/locales/en/.mustflow/skills/prompt-contract-quality-review/SKILL.md +1 -0
- package/templates/default/locales/en/.mustflow/skills/referral-incentive-integrity-review/SKILL.md +206 -0
- package/templates/default/locales/en/.mustflow/skills/retry-policy-integrity-review/SKILL.md +45 -2
- package/templates/default/locales/en/.mustflow/skills/routes.toml +329 -7
- package/templates/default/locales/en/.mustflow/skills/service-portfolio-capital-allocation-review/SKILL.md +245 -0
- package/templates/default/locales/en/.mustflow/skills/subscription-retention-profit-review/SKILL.md +217 -0
- package/templates/default/manifest.toml +86 -1
|
@@ -0,0 +1,196 @@
|
|
|
1
|
+
---
|
|
2
|
+
mustflow_doc: skill.agent-runtime-isolation-review
|
|
3
|
+
locale: en
|
|
4
|
+
canonical: true
|
|
5
|
+
revision: 1
|
|
6
|
+
lifecycle: mustflow-owned
|
|
7
|
+
authority: procedure
|
|
8
|
+
name: agent-runtime-isolation-review
|
|
9
|
+
description: Apply this skill when a long-running agent system must keep durable workflow state outside short-lived workers, split work across capability-specific queues, run model-directed shell, browser, file, package, or code activity in ephemeral sandboxes, inject narrow temporary credentials, broker production effects, and choose isolation by blast radius and maximum tolerable loss.
|
|
10
|
+
metadata:
|
|
11
|
+
mustflow_schema: "1"
|
|
12
|
+
mustflow_kind: procedure
|
|
13
|
+
pack_id: mustflow.core
|
|
14
|
+
skill_id: mustflow.core.agent-runtime-isolation-review
|
|
15
|
+
command_intents:
|
|
16
|
+
- changes_status
|
|
17
|
+
- changes_diff_summary
|
|
18
|
+
- lint
|
|
19
|
+
- build
|
|
20
|
+
- test_related
|
|
21
|
+
- test
|
|
22
|
+
- docs_validate_fast
|
|
23
|
+
- test_release
|
|
24
|
+
- mustflow_check
|
|
25
|
+
---
|
|
26
|
+
|
|
27
|
+
# Agent Runtime Isolation Review
|
|
28
|
+
|
|
29
|
+
<!-- mustflow-section: purpose -->
|
|
30
|
+
## Purpose
|
|
31
|
+
|
|
32
|
+
Keep state long-lived while keeping execution processes and authority short-lived. Separate the
|
|
33
|
+
trusted control plane from model-directed compute so a worker crash, poisoned context, dependency,
|
|
34
|
+
shell command, browser page, or credential misuse cannot inherit the lifetime and blast radius of
|
|
35
|
+
the whole workflow.
|
|
36
|
+
|
|
37
|
+
<!-- mustflow-section: use-when -->
|
|
38
|
+
## Use When
|
|
39
|
+
|
|
40
|
+
- An agent runtime executes shell commands, code, package operations, browsers, files, previews,
|
|
41
|
+
data transformations, external tools, or untrusted content.
|
|
42
|
+
- A logical task lasts longer than one process, container, sandbox, model context, credential, queue
|
|
43
|
+
lease, or worker allocation.
|
|
44
|
+
- The design chooses between a long-lived process and durable workflow plus queued activities,
|
|
45
|
+
shared workers and per-task sandboxes, containers and stronger isolation, or direct production
|
|
46
|
+
credentials and a policy broker.
|
|
47
|
+
- Isolation cost, startup cost, state rehydration, task granularity, retry loss, incident frequency,
|
|
48
|
+
blast radius, or maximum tolerable loss affects architecture.
|
|
49
|
+
|
|
50
|
+
<!-- mustflow-section: do-not-use-when -->
|
|
51
|
+
## Do Not Use When
|
|
52
|
+
|
|
53
|
+
- The main problem is generic durable workflow state, callbacks, timers, or compensation with no
|
|
54
|
+
model-directed execution plane; use `durable-workflow-orchestration`.
|
|
55
|
+
- The main problem is broker acknowledgement, redelivery, ordering, DLQ, or worker loss; use
|
|
56
|
+
`queue-processing-integrity-review`.
|
|
57
|
+
- The main problem is child-process timeout, output, argv, environment, or termination inside one
|
|
58
|
+
trusted execution boundary; use `process-execution-safety`.
|
|
59
|
+
- The main problem is deciding whether multiple runtime agents outperform one; use
|
|
60
|
+
`agent-runtime-multi-worker-review`.
|
|
61
|
+
|
|
62
|
+
<!-- mustflow-section: required-inputs -->
|
|
63
|
+
## Required Inputs
|
|
64
|
+
|
|
65
|
+
- Control-plane ledger: workflow state, event history, plan, policy, approvals, identities, budgets,
|
|
66
|
+
audit, billing, recovery, artifact registry, and which trusted service owns each item.
|
|
67
|
+
- Execution-plane ledger: worker image and identity, sandbox type, lifetime, task input, mounts,
|
|
68
|
+
writable paths, output artifacts, processes, ports, packages, browser, network, and cleanup proof.
|
|
69
|
+
- Queue ledger: capability class, admission, concurrency, priority, lease, heartbeat, timeout, retry,
|
|
70
|
+
cancellation, backpressure, worker pool, and production-mutation separation.
|
|
71
|
+
- Credential ledger: issuer, subject, tenant, resource, effect, audience, expiry, call and cost limit,
|
|
72
|
+
injection path, revocation, persistence rule, and broker boundary.
|
|
73
|
+
- Task-boundary ledger: independently verifiable outcome, serialized state, artifact references,
|
|
74
|
+
rehydration cost, failure and restart rate, idempotency, unknown outcome, compensation, and
|
|
75
|
+
checkpoint or split decision.
|
|
76
|
+
- Isolation ledger: public versus private inputs, untrusted code or content, secret access, host and
|
|
77
|
+
network exposure, production authority, candidate isolation levels, direct and operating cost,
|
|
78
|
+
incident likelihood, expected loss, maximum loss, and organizational survival limit.
|
|
79
|
+
|
|
80
|
+
<!-- mustflow-section: preconditions -->
|
|
81
|
+
## Preconditions
|
|
82
|
+
|
|
83
|
+
- Identify the durable source of truth before shortening worker lifetime. Process memory, a browser
|
|
84
|
+
profile, a prompt transcript, or a sandbox filesystem is not durable workflow authority by itself.
|
|
85
|
+
- Classify every input, mount, credential, network target, artifact, and external effect before
|
|
86
|
+
selecting an isolation level.
|
|
87
|
+
- Refresh provider sandbox, runtime, pricing, persistence, credential, and network behavior before
|
|
88
|
+
embedding exact claims. Do not copy example costs, durations, or checkpoint formulas into policy.
|
|
89
|
+
- Keep command execution under `.mustflow/config/commands.toml`; this skill does not authorize live
|
|
90
|
+
workers, sandboxes, packages, browsers, cloud mutations, or production credentials.
|
|
91
|
+
|
|
92
|
+
<!-- mustflow-section: allowed-edits -->
|
|
93
|
+
## Allowed Edits
|
|
94
|
+
|
|
95
|
+
- Add or refine control and execution plane separation, durable state, task queues, worker admission,
|
|
96
|
+
sandbox manifests, mounts, output directories, ephemeral credentials, network policy, artifact
|
|
97
|
+
validation, policy brokers, idempotent commit, cleanup, tests, docs, routes, and templates.
|
|
98
|
+
- Replace persistent all-purpose workers with capability-specific pools or per-task sandboxes when
|
|
99
|
+
risk and operating evidence support the split.
|
|
100
|
+
- Do not expose host-wide sockets, root authority, user home directories, long-lived cloud keys,
|
|
101
|
+
production database administration, or broad filesystem and network access to model-directed
|
|
102
|
+
compute as a convenience shortcut.
|
|
103
|
+
- Do not treat a container boundary alone as authorization, tenant isolation, production approval,
|
|
104
|
+
or protection against every host escape.
|
|
105
|
+
|
|
106
|
+
<!-- mustflow-section: procedure -->
|
|
107
|
+
## Procedure
|
|
108
|
+
|
|
109
|
+
1. Separate logical lifetime from process lifetime. Keep one durable workflow identity, event or
|
|
110
|
+
state history, approvals, budgets, and effect ledger across processes. Rebuild transient clients,
|
|
111
|
+
prompts, caches, browsers, and sandboxes for each admitted activity.
|
|
112
|
+
2. Keep the control plane trusted. Let it own the agent loop, model routing, policy, approvals,
|
|
113
|
+
identity, audit, billing, recovery, and authoritative run state. Put model-directed file, shell,
|
|
114
|
+
package, browser, and preview work in an execution plane with narrower authority.
|
|
115
|
+
3. Choose activity boundaries by recoverability, not arbitrary time or one model turn. A good unit
|
|
116
|
+
produces an independently verifiable artifact or state transition, can start from serialized
|
|
117
|
+
state and references, is safe to retry or reconcile, and does not repeat completed external
|
|
118
|
+
effects after failure.
|
|
119
|
+
4. Balance unit size from local evidence. Count sandbox startup, serialization, queue, cache warm-up,
|
|
120
|
+
and rehydration cost against expected lost work, timeout, context contamination, and failure
|
|
121
|
+
recovery. Use measured restart and rehydration behavior; do not impose a universal duration.
|
|
122
|
+
5. Route activities by capability. Separate read-only browsing, private-data reading, CPU build,
|
|
123
|
+
GPU work, package execution, and production mutation so concurrency, priority, network, image,
|
|
124
|
+
budget, and retry policy can differ. Do not let one general queue imply every capability.
|
|
125
|
+
6. Build a minimal sandbox manifest. Mount only required inputs, default them to read-only, provide a
|
|
126
|
+
distinct bounded output area, use an allowlisted environment, declare packages and ports, and
|
|
127
|
+
exclude host caches, homes, sockets, credentials, and unrelated repositories.
|
|
128
|
+
7. Keep credentials short-lived and task-scoped. Inject them at runtime from a trusted broker, bind
|
|
129
|
+
tenant, resource, effect, audience, expiry, and budget, prevent persistence in prompts, files,
|
|
130
|
+
snapshots, logs, images, and artifacts, and revoke them when the task ends or policy changes.
|
|
131
|
+
8. Deny network by default where practical. Allow only declared package mirrors, data sources, APIs,
|
|
132
|
+
callbacks, or artifact stores. Apply DNS, redirect, private-address, credential-forwarding, and
|
|
133
|
+
egress policy at a trusted boundary rather than asking the model to comply.
|
|
134
|
+
9. Validate artifacts before export. Check schema, hashes, provenance, sensitive content, executable
|
|
135
|
+
files, links, permissions, ownership, and expected output path before moving results into trusted
|
|
136
|
+
storage or a later workflow stage.
|
|
137
|
+
10. Broker high-impact effects outside the sandbox. Let the worker propose a normalized change; let
|
|
138
|
+
a trusted policy service re-evaluate current identity, target, state, approval, idempotency, and
|
|
139
|
+
capability before executing or handing the action to a human.
|
|
140
|
+
11. Make queued execution replay-safe. Persist intent before dispatch, keep effect identity stable,
|
|
141
|
+
use idempotency keys or reconciliation, heartbeat genuinely long work, fence stale workers, and
|
|
142
|
+
preserve UNKNOWN outcomes. Route broker settlement details to
|
|
143
|
+
`queue-processing-integrity-review` and workflow recovery to
|
|
144
|
+
`durable-workflow-orchestration`.
|
|
145
|
+
12. Choose isolation from blast radius and maximum loss, not compute price alone. Compare direct and
|
|
146
|
+
fixed isolation cost with expected incident reduction, but exclude any level whose worst
|
|
147
|
+
credible loss exceeds the organization's declared survival limit.
|
|
148
|
+
13. Match isolation to workload. Public read-only tasks with no secrets or host value may use a
|
|
149
|
+
lighter boundary; private repositories, untrusted pages, arbitrary packages, shell, browser, or
|
|
150
|
+
customer data need stronger task isolation; production IAM, payments, DNS, and destructive
|
|
151
|
+
infrastructure need a separate account, project, tenant, or brokered control plane as applicable.
|
|
152
|
+
14. Verify teardown. Stop process trees, revoke credentials, close ports, expire leases, preserve
|
|
153
|
+
bounded diagnostics, export only admitted artifacts, and record whether filesystem, network,
|
|
154
|
+
mount, snapshot, and secret cleanup was confirmed or remains uncertain.
|
|
155
|
+
15. Test containment and recovery. Cover worker crash before and after effects, stale lease,
|
|
156
|
+
duplicate delivery, unknown outcome, poisoned input, forbidden mount, blocked egress, expired
|
|
157
|
+
credential, secret-in-artifact rejection, sandbox escape signal, artifact tampering, cleanup
|
|
158
|
+
failure, and resume in a fresh worker.
|
|
159
|
+
|
|
160
|
+
<!-- mustflow-section: postconditions -->
|
|
161
|
+
## Postconditions
|
|
162
|
+
|
|
163
|
+
- Durable workflow truth survives worker and sandbox loss without depending on process memory.
|
|
164
|
+
- Model-directed compute receives only task-scoped mounts, network, credentials, capabilities, and
|
|
165
|
+
lifetime.
|
|
166
|
+
- High-impact external effects are revalidated and committed by a trusted broker with idempotency or
|
|
167
|
+
reconciliation.
|
|
168
|
+
- Isolation choice accounts for blast radius and maximum tolerable loss as well as direct cost.
|
|
169
|
+
|
|
170
|
+
<!-- mustflow-section: verification -->
|
|
171
|
+
## Verification
|
|
172
|
+
|
|
173
|
+
Use configured oneshot command intents when available: `changes_status`, `changes_diff_summary`,
|
|
174
|
+
`lint`, `build`, `test_related`, `test`, `docs_validate_fast`, `test_release`, and `mustflow_check`.
|
|
175
|
+
Do not infer raw container, VM, queue, browser, cloud, package, network, or production commands.
|
|
176
|
+
|
|
177
|
+
<!-- mustflow-section: failure-handling -->
|
|
178
|
+
## Failure Handling
|
|
179
|
+
|
|
180
|
+
- If durable truth lives only in one process or sandbox, stop claiming resumability.
|
|
181
|
+
- If credential, mount, network, or output scope cannot be bounded, keep the task manual or move it
|
|
182
|
+
to a stronger isolated environment.
|
|
183
|
+
- If an external effect lacks idempotency or outcome reconciliation, disable automatic replay and
|
|
184
|
+
preserve manual-review state.
|
|
185
|
+
- If cleanup cannot be confirmed, quarantine the environment and revoke authority before reuse.
|
|
186
|
+
|
|
187
|
+
<!-- mustflow-section: output-format -->
|
|
188
|
+
## Output Format
|
|
189
|
+
|
|
190
|
+
- Control and execution plane boundary
|
|
191
|
+
- Durable state, activity, queue, worker, and artifact decisions
|
|
192
|
+
- Sandbox, mount, network, credential, and cleanup contract
|
|
193
|
+
- Policy broker, idempotency, reconciliation, and high-impact effect boundary
|
|
194
|
+
- Isolation cost, blast radius, maximum loss, and survival-gate findings
|
|
195
|
+
- Command intents run and skipped checks
|
|
196
|
+
- Remaining agent-runtime isolation risk
|
package/templates/default/locales/en/.mustflow/skills/agent-runtime-multi-worker-review/SKILL.md
ADDED
|
@@ -0,0 +1,180 @@
|
|
|
1
|
+
---
|
|
2
|
+
mustflow_doc: skill.agent-runtime-multi-worker-review
|
|
3
|
+
locale: en
|
|
4
|
+
canonical: true
|
|
5
|
+
revision: 1
|
|
6
|
+
lifecycle: mustflow-owned
|
|
7
|
+
authority: procedure
|
|
8
|
+
name: agent-runtime-multi-worker-review
|
|
9
|
+
description: Apply this skill when a production LLM agent runtime may use several agents or workers and must decide whether independent parallelism justifies coordination cost, then define orchestrator-worker topology, specialized roles, capability separation, artifact ownership, independent evidence, correlated-error controls, bounded fan-out, and central verification.
|
|
10
|
+
metadata:
|
|
11
|
+
mustflow_schema: "1"
|
|
12
|
+
mustflow_kind: procedure
|
|
13
|
+
pack_id: mustflow.core
|
|
14
|
+
skill_id: mustflow.core.agent-runtime-multi-worker-review
|
|
15
|
+
command_intents:
|
|
16
|
+
- changes_status
|
|
17
|
+
- changes_diff_summary
|
|
18
|
+
- lint
|
|
19
|
+
- build
|
|
20
|
+
- test_related
|
|
21
|
+
- test
|
|
22
|
+
- docs_validate_fast
|
|
23
|
+
- test_release
|
|
24
|
+
- mustflow_check
|
|
25
|
+
---
|
|
26
|
+
|
|
27
|
+
# Agent Runtime Multi-Worker Review
|
|
28
|
+
|
|
29
|
+
<!-- mustflow-section: purpose -->
|
|
30
|
+
## Purpose
|
|
31
|
+
|
|
32
|
+
Use multiple runtime agents only when independently solvable work, parallel context capacity, or
|
|
33
|
+
specialized tools create more accepted value than planning, communication, duplication, ownership,
|
|
34
|
+
and correlated-error cost. Keep the default architecture single-agent until evidence supports the
|
|
35
|
+
extra topology.
|
|
36
|
+
|
|
37
|
+
<!-- mustflow-section: use-when -->
|
|
38
|
+
## Use When
|
|
39
|
+
|
|
40
|
+
- A production agent runtime adds lead agents, managers, worker agents, peer agents, debate,
|
|
41
|
+
independent reviewers, candidate generators, judges, or distributed research workers.
|
|
42
|
+
- A task may split by source, customer, document set, package, hypothesis, candidate solution, or
|
|
43
|
+
another independently verifiable work unit.
|
|
44
|
+
- The design must choose worker count, topology, role diversity, task ownership, communication,
|
|
45
|
+
artifact handoff, result aggregation, verification, or stop conditions.
|
|
46
|
+
- A claim says multi-agent execution improves quality, coverage, throughput, latency, or reliability
|
|
47
|
+
enough to justify extra tokens, infrastructure, coordination, and failure surface.
|
|
48
|
+
|
|
49
|
+
<!-- mustflow-section: do-not-use-when -->
|
|
50
|
+
## Do Not Use When
|
|
51
|
+
|
|
52
|
+
- Several coding agents coordinate files, worktrees, commands, and merges for one repository task;
|
|
53
|
+
use `multi-agent-work-coordination`.
|
|
54
|
+
- One runtime parent already has a justified child set and the main risk is join, cancellation,
|
|
55
|
+
deadlines, cleanup, or late results; use `structured-concurrency-supervision-review`.
|
|
56
|
+
- The main risk is durable resume, queues, short-lived workers, or sandbox isolation; use
|
|
57
|
+
`agent-runtime-isolation-review`.
|
|
58
|
+
- The main risk is tool authority, approval, external effects, or one agent's autonomy envelope; use
|
|
59
|
+
`agent-execution-control-review`.
|
|
60
|
+
|
|
61
|
+
<!-- mustflow-section: required-inputs -->
|
|
62
|
+
## Required Inputs
|
|
63
|
+
|
|
64
|
+
- Work-unit ledger: total work, independent units, dependency graph, shared context, expected output,
|
|
65
|
+
independent validator, merge rule, and serial critical path.
|
|
66
|
+
- Benefit ledger: single-agent baseline, coverage, quality, latency, throughput, task value, backlog,
|
|
67
|
+
and accepted-outcome improvement attributable to parallel workers rather than extra total compute.
|
|
68
|
+
- Coordination ledger: planning, delegation, communication, context transfer, duplicate work,
|
|
69
|
+
conflict resolution, merge, verification, supervision, and fixed operating cost.
|
|
70
|
+
- Topology ledger: orchestrator, workers, reviewers, verifier, allowed communication edges, worker
|
|
71
|
+
admission rule, fan-out and depth budgets, wait policy, cancellation, and terminal owner.
|
|
72
|
+
- Role ledger: unique information, tools, model, search strategy, permissions, evaluation criterion,
|
|
73
|
+
artifact owner, and forbidden overlap for each role.
|
|
74
|
+
- Correlation ledger: shared model, prompt, source, memory, retrieval, tool, evaluator, failure domain,
|
|
75
|
+
and how supposedly independent conclusions remain correlated.
|
|
76
|
+
|
|
77
|
+
<!-- mustflow-section: preconditions -->
|
|
78
|
+
## Preconditions
|
|
79
|
+
|
|
80
|
+
- Prove that at least one meaningful work unit can be completed and validated without continuous
|
|
81
|
+
access to another worker's hidden context.
|
|
82
|
+
- Establish a single-agent baseline under a comparable total-compute and outcome contract before
|
|
83
|
+
attributing a gain to topology.
|
|
84
|
+
- Do not copy benchmark worker counts, token multipliers, break-even sizes, or universal formulas
|
|
85
|
+
into runtime policy. Calibrate from the product's task distribution and operating evidence.
|
|
86
|
+
- Keep command execution under `.mustflow/config/commands.toml`; this skill does not authorize live
|
|
87
|
+
worker spawning, model calls, production traffic, or background agent loops.
|
|
88
|
+
|
|
89
|
+
<!-- mustflow-section: allowed-edits -->
|
|
90
|
+
## Allowed Edits
|
|
91
|
+
|
|
92
|
+
- Add or refine worker admission, orchestrator-worker topology, delegation schemas, role and
|
|
93
|
+
capability boundaries, artifact stores, central verification, independent review, correlation
|
|
94
|
+
metrics, fan-out and depth budgets, cancellation, tests, docs, route metadata, and templates.
|
|
95
|
+
- Replace peer-to-peer debate with bounded independent work and one aggregation owner when debate
|
|
96
|
+
does not provide unique evidence.
|
|
97
|
+
- Do not add workers merely because a task is difficult, long, or uses many tokens.
|
|
98
|
+
- Do not let several agents jointly own one authoritative artifact or external effect.
|
|
99
|
+
|
|
100
|
+
<!-- mustflow-section: procedure -->
|
|
101
|
+
## Procedure
|
|
102
|
+
|
|
103
|
+
1. Start from the single-agent baseline. Record accepted outcomes, latency, cost, failures, and
|
|
104
|
+
context limits under the same task mix and total-compute accounting used for the candidate.
|
|
105
|
+
2. Decompose by independently completable and verifiable work units, not by arbitrary prompt turns.
|
|
106
|
+
Reject parallelization when most steps share one evolving context, mutate the same authority, or
|
|
107
|
+
depend on subtle upstream judgment.
|
|
108
|
+
3. Model the break-even qualitatively and quantitatively from local evidence. Count parallel quality
|
|
109
|
+
or time value, then subtract planning, communication, duplication, ownership misses, conflict,
|
|
110
|
+
verification, retry, and operating cost. If marginal benefit is non-positive, keep one agent.
|
|
111
|
+
4. Prefer a star topology. Let one orchestrator issue bounded work orders, let workers return
|
|
112
|
+
independent artifacts or structured findings, and let one verifier or merge owner accept or
|
|
113
|
+
reject them. Avoid all-to-all conversation whose message count and repeated context grow rapidly.
|
|
114
|
+
5. Give roles distinct information or capability. Separate sources, tools, models, search strategies,
|
|
115
|
+
permissions, hypotheses, or evaluation criteria. Multiple copies of one prompt and source set are
|
|
116
|
+
correlated samples, not independent experts.
|
|
117
|
+
6. Define one owner per artifact and effect. Workers may propose or review; the named owner admits
|
|
118
|
+
the result, resolves conflict, and owns terminal state. Consensus does not erase accountability.
|
|
119
|
+
7. Preserve independent judgment before aggregation. Do not reveal peer answers to reviewers before
|
|
120
|
+
their initial assessment when independence matters. Record shared failure domains and avoid
|
|
121
|
+
treating majority vote as strong evidence when errors are correlated.
|
|
122
|
+
8. Use durable artifacts, not transcript relays. Store source-backed findings, patches, datasets,
|
|
123
|
+
reports, hashes, or typed records under stable references; pass lightweight references and
|
|
124
|
+
summaries to the orchestrator to reduce telephone-game loss and context duplication.
|
|
125
|
+
9. Bound admission from task evidence. Set active worker, queued worker, depth, tool-call, token,
|
|
126
|
+
cost, and wall-clock budgets from expected independent units and resource capacity. Stop adding
|
|
127
|
+
workers when remaining units overlap, coordination dominates, or the verifier becomes the
|
|
128
|
+
bottleneck.
|
|
129
|
+
10. Make completion policy explicit. Choose all-required, quorum, first-valid, best-of-N under an
|
|
130
|
+
external validator, or partial result. Define timeout, cancellation, late-result rejection,
|
|
131
|
+
missing-worker behavior, and which failures can preserve useful artifacts.
|
|
132
|
+
11. Keep capabilities task-scoped and attenuation-only. A worker receives only the sources, tools,
|
|
133
|
+
resources, effects, budget, and lifetime needed for its work unit. Worker failure must not trigger
|
|
134
|
+
automatic privilege escalation.
|
|
135
|
+
12. Verify centrally with deterministic checks first. Use tests, schemas, source agreement,
|
|
136
|
+
database state, checksums, invariants, or other independent evidence before an LLM judge. Keep
|
|
137
|
+
creator, reviewer, and final authority distinct for high-impact outcomes.
|
|
138
|
+
13. Evaluate topology against the baseline. Attribute gains separately to extra compute, parallelism,
|
|
139
|
+
role specialization, tool diversity, and verifier quality. Measure duplication, handoff loss,
|
|
140
|
+
correlated failures, verifier load, orphan work, and cost per accepted outcome.
|
|
141
|
+
14. Route runtime lifetime and sandbox design to `agent-runtime-isolation-review`; route child join
|
|
142
|
+
and cancellation mechanics to `structured-concurrency-supervision-review`; route outcome and
|
|
143
|
+
trajectory grading to `agent-eval-integrity-review`.
|
|
144
|
+
|
|
145
|
+
<!-- mustflow-section: postconditions -->
|
|
146
|
+
## Postconditions
|
|
147
|
+
|
|
148
|
+
- Multi-worker execution is enabled only for independently solvable and verifiable units with a
|
|
149
|
+
measured advantage over a comparable single-agent baseline.
|
|
150
|
+
- Communication is bounded, artifact ownership is singular, and correlated votes are not presented
|
|
151
|
+
as independent proof.
|
|
152
|
+
- Fan-out, depth, capability, cost, wait, cancellation, and terminal ownership are explicit.
|
|
153
|
+
- One central owner verifies and admits worker results.
|
|
154
|
+
|
|
155
|
+
<!-- mustflow-section: verification -->
|
|
156
|
+
## Verification
|
|
157
|
+
|
|
158
|
+
Use configured oneshot command intents when available: `changes_status`, `changes_diff_summary`,
|
|
159
|
+
`lint`, `build`, `test_related`, `test`, `docs_validate_fast`, `test_release`, and `mustflow_check`.
|
|
160
|
+
Do not infer raw worker, model, queue, load, or production commands.
|
|
161
|
+
|
|
162
|
+
<!-- mustflow-section: failure-handling -->
|
|
163
|
+
## Failure Handling
|
|
164
|
+
|
|
165
|
+
- If work units cannot be independently validated, serialize them under one owner.
|
|
166
|
+
- If worker roles differ only by name, reduce the worker set or redesign the evidence split.
|
|
167
|
+
- If communication or verification cost dominates the measured benefit, return to the single-agent
|
|
168
|
+
baseline.
|
|
169
|
+
- If artifact or effect ownership is ambiguous, stop admission until one terminal owner is named.
|
|
170
|
+
|
|
171
|
+
<!-- mustflow-section: output-format -->
|
|
172
|
+
## Output Format
|
|
173
|
+
|
|
174
|
+
- Single-agent baseline and multi-worker decision
|
|
175
|
+
- Independent work units, dependencies, and validators
|
|
176
|
+
- Topology, roles, capabilities, communication, and artifact ownership
|
|
177
|
+
- Correlation, fan-out, wait, cancellation, and terminal policies
|
|
178
|
+
- Cost-per-accepted-outcome and attribution findings
|
|
179
|
+
- Command intents run and skipped checks
|
|
180
|
+
- Remaining runtime multi-worker risk
|
package/templates/default/locales/en/.mustflow/skills/automation-investment-case-review/SKILL.md
ADDED
|
@@ -0,0 +1,173 @@
|
|
|
1
|
+
---
|
|
2
|
+
mustflow_doc: skill.automation-investment-case-review
|
|
3
|
+
locale: en
|
|
4
|
+
canonical: true
|
|
5
|
+
revision: 1
|
|
6
|
+
lifecycle: mustflow-owned
|
|
7
|
+
authority: procedure
|
|
8
|
+
name: automation-investment-case-review
|
|
9
|
+
description: Apply this skill when deciding whether an AI agent, workflow automation, internal tool, or operational integration is economically worth building, expanding, replacing, or retiring by comparing expected cost per accepted outcome, human alternatives, supervision, failure recovery, maintenance, break-even volume, throughput value, effective lifetime, NPV, and independent safety gates.
|
|
10
|
+
metadata:
|
|
11
|
+
mustflow_schema: "1"
|
|
12
|
+
mustflow_kind: procedure
|
|
13
|
+
pack_id: mustflow.core
|
|
14
|
+
skill_id: mustflow.core.automation-investment-case-review
|
|
15
|
+
command_intents:
|
|
16
|
+
- changes_status
|
|
17
|
+
- changes_diff_summary
|
|
18
|
+
- docs_validate_fast
|
|
19
|
+
- test_release
|
|
20
|
+
- mustflow_check
|
|
21
|
+
---
|
|
22
|
+
|
|
23
|
+
# Automation Investment Case Review
|
|
24
|
+
|
|
25
|
+
<!-- mustflow-section: purpose -->
|
|
26
|
+
## Purpose
|
|
27
|
+
|
|
28
|
+
Decide whether automation produces cheaper accepted outcomes, useful capacity, or strategic value
|
|
29
|
+
after supervision, failure recovery, maintenance, redesign, and safety costs are included. Prevent a
|
|
30
|
+
cheap model call or impressive throughput demo from being mistaken for a viable operating system.
|
|
31
|
+
|
|
32
|
+
<!-- mustflow-section: use-when -->
|
|
33
|
+
## Use When
|
|
34
|
+
|
|
35
|
+
- A team asks whether to build, buy, expand, replace, or retire an AI agent, workflow automation,
|
|
36
|
+
integration, internal tool, or operational pipeline.
|
|
37
|
+
- A proposal compares automation cost with human work, headcount, outsourcing, queue delay,
|
|
38
|
+
conversion, revenue, SLA, error cost, or throughput.
|
|
39
|
+
- The decision needs expected cost per accepted outcome, break-even volume, implementation and
|
|
40
|
+
maintenance cost, effective lifetime, scenario analysis, discounting, or NPV.
|
|
41
|
+
- A claimed saving excludes supervision, retries, manual review, incident response, rejected output,
|
|
42
|
+
redesign, vendor changes, security, compliance, or rollback cost.
|
|
43
|
+
|
|
44
|
+
<!-- mustflow-section: do-not-use-when -->
|
|
45
|
+
## Do Not Use When
|
|
46
|
+
|
|
47
|
+
- The task only optimizes LLM token spend, prompt caching, model routing, or per-request inference
|
|
48
|
+
cost inside an already-approved product; use `llm-token-cost-control-review`.
|
|
49
|
+
- The task only analyzes cloud infrastructure spend; use `cloud-cost-guardrail-review`.
|
|
50
|
+
- The task only selects a vendor, framework, runtime, or platform; use
|
|
51
|
+
`technology-stack-selection` and use this skill only for the investment model.
|
|
52
|
+
- The task only evaluates technical agent quality or safety; use the matching agent, eval, security,
|
|
53
|
+
privacy, or rollout skill. Economic return never overrides a failed safety gate.
|
|
54
|
+
|
|
55
|
+
<!-- mustflow-section: required-inputs -->
|
|
56
|
+
## Required Inputs
|
|
57
|
+
|
|
58
|
+
- Outcome contract: one accepted business outcome, rejection or correction rule, quality floor,
|
|
59
|
+
latency or SLA, volume, demand pattern, current backlog, and value owner.
|
|
60
|
+
- Automation cost ledger: model, tool, API, compute, storage, network, observability, licenses,
|
|
61
|
+
implementation, integration, data preparation, eval, security, compliance, supervision, manual
|
|
62
|
+
review, failure recovery, incident response, support, maintenance, retraining, migration, and exit.
|
|
63
|
+
- Human comparator: labor time and loaded cost, tooling, management, training, queue delay, error and
|
|
64
|
+
rework rates, review, escalation, coverage, turnover, and capacity constraints.
|
|
65
|
+
- Reliability ledger: success, accepted without correction, accepted after correction, rejected,
|
|
66
|
+
silent failure, duplicate or harmful effect, rollback or compensation, incident probability, and
|
|
67
|
+
cost distribution.
|
|
68
|
+
- Investment assumptions: up-front cost, recurring cost, volume scenarios, adoption ramp, effective
|
|
69
|
+
lifetime before redesign, discount rate, residual value, opportunity cost, and confidence range.
|
|
70
|
+
- Safety gate: legal, privacy, security, authorization, irreversibility, human-oversight, audit, and
|
|
71
|
+
rollback requirements that must pass independently of ROI.
|
|
72
|
+
|
|
73
|
+
<!-- mustflow-section: preconditions -->
|
|
74
|
+
## Preconditions
|
|
75
|
+
|
|
76
|
+
- Define accepted outcome and the current alternative before comparing model or vendor prices.
|
|
77
|
+
- Refresh price, wage, contract, volume, error, and lifecycle assumptions from current authoritative
|
|
78
|
+
sources when the decision is current or material.
|
|
79
|
+
- Label unknowns and ranges. Do not turn a vendor benchmark, example number, or one pilot into a
|
|
80
|
+
universal production rate.
|
|
81
|
+
- Keep command execution under `.mustflow/config/commands.toml`; this skill does not authorize
|
|
82
|
+
purchases, contracts, staffing changes, production rollout, or releases.
|
|
83
|
+
|
|
84
|
+
<!-- mustflow-section: allowed-edits -->
|
|
85
|
+
## Allowed Edits
|
|
86
|
+
|
|
87
|
+
- Add or refine decision records, assumptions, cost models, scenario tables, break-even analysis,
|
|
88
|
+
NPV models, evidence links, sensitivity analysis, safety gates, tests for calculation logic, docs,
|
|
89
|
+
route metadata, and synchronized templates.
|
|
90
|
+
- Separate measured, quoted, estimated, inferred, and unknown values.
|
|
91
|
+
- Do not embed universal numeric thresholds, vendor prices, wage rates, error rates, discount rates,
|
|
92
|
+
or useful-life assumptions in reusable policy.
|
|
93
|
+
- Do not convert throughput into value unless demand, queue, revenue, SLA, risk, or another binding
|
|
94
|
+
constraint makes extra completed work valuable.
|
|
95
|
+
|
|
96
|
+
<!-- mustflow-section: procedure -->
|
|
97
|
+
## Procedure
|
|
98
|
+
|
|
99
|
+
1. Define one accepted outcome. Count the business result that passes the quality and safety bar,
|
|
100
|
+
not a model response, tool call, attempted task, or superficially completed workflow.
|
|
101
|
+
2. Model automation expected variable cost per accepted outcome. Include ordinary execution,
|
|
102
|
+
supervision, review, retries, rejected work, failure recovery, rollback or compensation, and the
|
|
103
|
+
probability-weighted tail of incidents. Preserve distributions where rare failures dominate.
|
|
104
|
+
3. Add fixed and step-fixed costs. Include design, implementation, integration, data work, evals,
|
|
105
|
+
security, compliance, rollout, training, change management, monitoring, maintenance, vendor and
|
|
106
|
+
model migrations, and decommissioning.
|
|
107
|
+
4. Build the human or current-system comparator on the same outcome contract. Include loaded labor,
|
|
108
|
+
tools, review, rework, delay, error cost, escalation, management, training, and capacity limits.
|
|
109
|
+
Do not compare automation's perfect-path variable cost with a human's fully loaded cost.
|
|
110
|
+
5. Separate substitution, assistance, and new capability. An assistant may save minutes without
|
|
111
|
+
removing a role; a new service may create demand rather than replace cost; a constrained queue may
|
|
112
|
+
make throughput valuable even when unit cost is higher.
|
|
113
|
+
6. Value throughput only when a binding constraint exists. Tie added capacity to observed backlog,
|
|
114
|
+
avoided SLA penalties, faster cash collection, additional conversion or revenue, reduced risk,
|
|
115
|
+
or another evidence-backed outcome. Otherwise report it as unused headroom.
|
|
116
|
+
7. Calculate break-even with ranges. Compare fixed investment with the expected per-accepted-outcome
|
|
117
|
+
advantage across conservative, base, and optimistic volume, quality, supervision, and failure
|
|
118
|
+
scenarios. Report when the denominator is zero, negative, or too uncertain for a meaningful
|
|
119
|
+
break-even point.
|
|
120
|
+
8. Use effective lifetime, not accounting optimism. Estimate how long the automation remains useful
|
|
121
|
+
before a product redesign, policy change, provider change, data drift, integration replacement,
|
|
122
|
+
or maintenance burden requires material reinvestment.
|
|
123
|
+
9. Discount scenario cash flows when timing matters. Show NPV and payback under explicit assumptions,
|
|
124
|
+
including ramp-up, delayed benefits, recurring maintenance, redesign, migration, and exit cost.
|
|
125
|
+
10. Run sensitivity and threshold analysis. Identify which assumptions can reverse the decision:
|
|
126
|
+
accepted-outcome rate, review time, incident cost, volume, demand value, maintenance burden,
|
|
127
|
+
vendor pricing, lifetime, or discount rate. Ask for evidence on those first.
|
|
128
|
+
11. Keep safety as a hard independent gate. Reject, narrow, or keep human execution when the design
|
|
129
|
+
cannot meet authorization, privacy, legal, irreversible-effect, audit, reconciliation, or
|
|
130
|
+
rollback requirements, even if the expected financial return is positive.
|
|
131
|
+
12. Choose a reversible next step. Prefer a bounded pilot or shadow measurement that resolves the
|
|
132
|
+
largest decision uncertainty without creating an irreversible operational dependency. Define
|
|
133
|
+
the evidence required to expand, redesign, pause, or stop.
|
|
134
|
+
|
|
135
|
+
<!-- mustflow-section: postconditions -->
|
|
136
|
+
## Postconditions
|
|
137
|
+
|
|
138
|
+
- Automation and the current alternative use the same accepted-outcome definition.
|
|
139
|
+
- Variable, fixed, supervision, failure, maintenance, redesign, and exit costs are visible.
|
|
140
|
+
- Break-even, throughput value, effective lifetime, and NPV use explicit ranges and evidence levels.
|
|
141
|
+
- Safety and legal feasibility remain hard gates independent of financial return.
|
|
142
|
+
|
|
143
|
+
<!-- mustflow-section: verification -->
|
|
144
|
+
## Verification
|
|
145
|
+
|
|
146
|
+
Use configured oneshot command intents when available: `changes_status`, `changes_diff_summary`,
|
|
147
|
+
`docs_validate_fast`, `test_release`, and `mustflow_check`. Use narrower configured calculation,
|
|
148
|
+
data-quality, or decision-record checks when available. Do not infer purchasing, billing, staffing,
|
|
149
|
+
production, deployment, or vendor commands.
|
|
150
|
+
|
|
151
|
+
<!-- mustflow-section: failure-handling -->
|
|
152
|
+
## Failure Handling
|
|
153
|
+
|
|
154
|
+
- If accepted outcome, comparator, volume, or safety gate is missing, report the case as
|
|
155
|
+
non-decision-ready instead of manufacturing a return.
|
|
156
|
+
- If one uncertain assumption dominates the result, provide the threshold at which the decision
|
|
157
|
+
changes and propose the smallest measurement that can resolve it.
|
|
158
|
+
- If benefits depend on unproven demand or queue value, separate operational capacity from financial
|
|
159
|
+
benefit.
|
|
160
|
+
- If a positive case requires ignoring rare catastrophic loss, keep the tail risk explicit and hand
|
|
161
|
+
the safety decision to the named authority.
|
|
162
|
+
|
|
163
|
+
<!-- mustflow-section: output-format -->
|
|
164
|
+
## Output Format
|
|
165
|
+
|
|
166
|
+
- Accepted outcome and current comparator
|
|
167
|
+
- Fixed, variable, supervision, failure, maintenance, redesign, and exit costs
|
|
168
|
+
- Volume, quality, demand, lifetime, and discount assumptions with evidence levels
|
|
169
|
+
- Break-even, payback, NPV, sensitivity, and throughput-value findings
|
|
170
|
+
- Independent safety-gate result
|
|
171
|
+
- Reversible next experiment and stop or expansion criteria
|
|
172
|
+
- Command intents run and skipped checks
|
|
173
|
+
- Remaining automation-investment uncertainty
|