jumpstart-mode 1.1.13 → 1.1.14

package/.jumpstart/skills/agent-workflow-state-machine/references/state-machine-design-guide.md

@@ -0,0 +1,77 @@
+# State Machine Design Guide
+
+Use this guide to model agentic workflows as explicit operational state machines.
+
+## Modeling principle
+
+Model stable business-operational states, not individual low-level function calls.
+
+Good state names:
+
+- `Planning`
+- `Validation`
+- `AwaitingHumanApproval`
+
+Weak state names:
+
+- `CallLLM`
+- `RunFunctionA`
+- `IfStatementCheck`
+
+## Recommended process
+
+1. Define workflow start and terminal outcomes.
+2. Identify stable intermediate states.
+3. Identify events that trigger transitions.
+4. Add guard conditions per transition.
+5. Define side effects and emitted events.
+6. Add retries, timeouts, and compensation paths.
+7. Add human checkpoints where risk requires approval.
+8. Verify all states are reachable and no unsafe bypass exists.
+
+## Transition template
+
+Use this structure for each transition:
+
+```yaml
+from_state: Validation
+event_trigger: validation_passed
+guard_condition: schema_valid == true and confidence >= 0.8
+action: prepare_publish_payload
+to_state: AwaitingHumanApproval
+idempotency_rule: event_id is processed at most once
+audit_event: transition.validation_to_approval
+```
+
+## Guard condition categories
+
+- Data quality guards: completeness, freshness, schema validity
+- Policy guards: compliance policy check, DLP result, restricted content filters
+- Trust guards: confidence score, contradiction checks, citation coverage
+- Authorization guards: role, permission, scope of action
+- Risk guards: cost ceiling, blast radius class, environment tier
+
+## Approval checkpoint design
+
+At each checkpoint, define:
+
+- approver role
+- approval SLA
+- decision options (`approve`, `reject`, `request_rework`)
+- timeout behavior
+- evidence packet required for decision
+
+## Reliability patterns
+
+- Retry transient failures with exponential backoff and jitter.
+- Route repeated failures to `Recovery` or `AwaitingHumanReview`.
+- Use circuit-breakers for unstable tools.
+- Use compensation actions for partial commits.
+
+## Validation checklist
+
+- No non-terminal dead-end states
+- No transition without guard conditions
+- No high-risk transition without required approval
+- No infinite retry loops
+- Terminal states reachable from at least one path

package/.jumpstart/skills/agent-workflow-state-machine/templates/agent-workflow-state-machine.mmd

@@ -0,0 +1,42 @@
+stateDiagram-v2
+    [*] --> Intake
+    Intake --> ContextPreparation: intake_validated
+    Intake --> Failed: intake_invalid
+
+    ContextPreparation --> Planning: context_ready
+    ContextPreparation --> Recovery: context_fetch_failed
+
+    Planning --> ToolExecution: plan_approved
+    Planning --> AwaitingHumanApproval: high_risk_plan
+    Planning --> Failed: plan_invalid
+
+    ToolExecution --> Validation: tool_run_complete
+    ToolExecution --> Recovery: transient_tool_error
+    ToolExecution --> Failed: nonrecoverable_tool_error
+
+    Validation --> AwaitingHumanApproval: requires_human_gate
+    Validation --> CommitOrPublish: validation_passed
+    Validation --> Rework: validation_failed_retryable
+    Validation --> Failed: validation_failed_nonretryable
+
+    Rework --> Planning: rework_complete
+    Rework --> Failed: rework_exhausted
+
+    AwaitingHumanApproval --> CommitOrPublish: approved
+    AwaitingHumanApproval --> Rework: changes_requested
+    AwaitingHumanApproval --> Canceled: rejected
+    AwaitingHumanApproval --> Escalation: sla_expired
+
+    Escalation --> AwaitingHumanApproval: reassigned
+    Escalation --> Canceled: escalation_rejected
+
+    CommitOrPublish --> Completed: commit_success
+    CommitOrPublish --> Recovery: partial_commit
+    CommitOrPublish --> Failed: commit_failed
+
+    Recovery --> Planning: recovered
+    Recovery --> Failed: recovery_exhausted
+
+    Completed --> [*]
+    Failed --> [*]
+    Canceled --> [*]

package/.jumpstart/skills/agent-workflow-state-machine/templates/agent-workflow-state-machine.yaml

@@ -0,0 +1,113 @@
+machine_name: example-agent-workflow
+version: 1.0.0
+initial_state: Intake
+terminal_states:
+  - Completed
+  - Failed
+  - Canceled
+
+states:
+  - name: Intake
+    purpose: Capture request and classify intent, risk, and scope.
+    allowed_actions:
+      - parse_request
+      - classify_risk
+      - validate_required_inputs
+    entry_criteria:
+      - request_received
+    exit_criteria:
+      - required_inputs_present
+    timeout_seconds: 300
+    retry_policy:
+      max_attempts: 1
+      strategy: none
+
+  - name: Planning
+    purpose: Build execution plan and select tools.
+    allowed_actions:
+      - generate_plan
+      - map_plan_to_tools
+      - run_preflight_checks
+    entry_criteria:
+      - context_prepared
+    exit_criteria:
+      - plan_validated
+    timeout_seconds: 600
+    retry_policy:
+      max_attempts: 2
+      strategy: fixed
+      delay_seconds: 10
+
+  - name: AwaitingHumanApproval
+    purpose: Hold for human decision before high-impact action.
+    allowed_actions:
+      - present_evidence_packet
+      - capture_decision
+    entry_criteria:
+      - approval_required
+    exit_criteria:
+      - decision_recorded
+    timeout_seconds: 1800
+    retry_policy:
+      max_attempts: 0
+      strategy: none
+
+transitions:
+  - id: t1
+    from_state: Intake
+    event_trigger: intake_validated
+    guard_condition: required_inputs_present == true
+    action: emit_transition_event
+    to_state: Planning
+    idempotency_rule: event_id_unique
+    audit_event: transition.intake_to_planning
+
+  - id: t2
+    from_state: Planning
+    event_trigger: high_risk_plan
+    guard_condition: risk_class in ["high", "critical"]
+    action: route_to_human_gate
+    to_state: AwaitingHumanApproval
+    idempotency_rule: checkpoint_once_per_run
+    audit_event: transition.planning_to_approval
+
+  - id: t3
+    from_state: AwaitingHumanApproval
+    event_trigger: approved
+    guard_condition: approver_role in ["DeliveryLead", "ComplianceReviewer"]
+    action: unlock_commit_path
+    to_state: CommitOrPublish
+    idempotency_rule: approval_decision_id_unique
+    audit_event: transition.approval_to_commit
+
+retry_defaults:
+  transient_errors:
+    max_attempts: 3
+    strategy: exponential
+    base_delay_seconds: 2
+    jitter: true
+  validation_errors:
+    max_attempts: 2
+    strategy: fixed
+    delay_seconds: 5
+
+approval_defaults:
+  decision_options:
+    - approve
+    - reject
+    - request_rework
+  sla_minutes: 60
+  timeout_target_state: Escalation
+
+observability:
+  required_events:
+    - state_entered
+    - state_exited
+    - transition_executed
+    - retry_attempted
+    - approval_decision_recorded
+  required_metrics:
+    - time_in_state_seconds
+    - transition_latency_seconds
+    - retries_per_state
+    - approval_wait_seconds

package/.jumpstart/skills/agent-workflow-state-machine/templates/human-approval-matrix.md

@@ -0,0 +1,15 @@
+# Human Approval Matrix
+
+Use this matrix to define approval checkpoints for high-impact transitions.
+
+| checkpoint_id | state_or_transition | risk_class | trigger | approver_role | evidence_required | decision_options | decision_sla_minutes | approve_target_state | reject_target_state | timeout_target_state |
+| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
+| ap-1 | Planning -> AwaitingHumanApproval | high | high_risk_plan | DeliveryLead | plan summary, risk score, policy checks | approve/reject/request_rework | 60 | CommitOrPublish | Rework | Escalation |
+| ap-2 | Validation -> AwaitingHumanApproval | critical | external_communication_required | ComplianceReviewer | output payload, citations, DLP scan | approve/reject/request_rework | 30 | CommitOrPublish | Rework | Escalation |
+| ap-3 | CommitOrPublish | critical | production_write_action | ProductOwner | change set, rollback plan, blast radius | approve/reject | 30 | Completed | Canceled | Escalation |
+
+## Notes
+
+- `request_rework` should route to a non-terminal state with explicit remediation steps.
+- Approver roles should map to named owners in runbooks.
+- SLA timeout must never bypass required approval for high-risk actions.

package/.jumpstart/skills/agent-workflow-state-machine/templates/state-transition-table.csv

@@ -0,0 +1,9 @@
+transition_id,from_state,event_trigger,guard_condition,action,to_state,retry_policy,timeout_seconds,idempotency_rule,audit_event,owner_role
+t1,Intake,intake_validated,required_inputs_present == true,prepare_context,ContextPreparation,none,300,event_id_unique,transition.intake_to_context,WorkflowEngine
+t2,ContextPreparation,context_ready,context_quality_score >= 0.8,create_plan,Planning,fixed:2x10s,600,event_id_unique,transition.context_to_planning,WorkflowEngine
+t3,Planning,high_risk_plan,risk_class in ["high" "critical"],request_human_approval,AwaitingHumanApproval,none,1800,checkpoint_once_per_run,transition.planning_to_approval,DeliveryLead
+t4,AwaitingHumanApproval,approved,approver_role in ["DeliveryLead" "ComplianceReviewer"],unlock_commit_path,CommitOrPublish,none,3600,approval_decision_id_unique,transition.approval_to_commit,Approver
+t5,AwaitingHumanApproval,rejected,true,record_rejection,Canceled,none,3600,approval_decision_id_unique,transition.approval_to_canceled,Approver
+t6,ToolExecution,transient_tool_error,retry_count < 3,retry_tool_call,ToolExecution,exponential:3,120,tool_call_id_unique,retry.tool_execution,WorkflowEngine
+t7,ToolExecution,nonrecoverable_tool_error,true,route_to_recovery,Recovery,none,120,event_id_unique,transition.tool_to_recovery,WorkflowEngine
+t8,Recovery,recovery_exhausted,recovery_attempts >= 2,mark_failed,Failed,none,300,event_id_unique,transition.recovery_to_failed,WorkflowEngine

package/.jumpstart/skills/ai-cost-model-designer/SKILL.md

@@ -0,0 +1,100 @@
+---
+name: ai-cost-model-designer
+description: 'Build a defensible per-call, per-user, per-workflow cost model for an AI build with caching, routing, and budget guardrails — usable in proposals, finance reviews, and post-launch operations. Use this whenever the user is sizing AI cost, building a TCO, asking "how much will this cost", "what does this cost per user", "model the OpenAI bill", or "set a budget guardrail".'
+argument-hint: 'Provide the use case, expected volume, model(s) under consideration, prompt/context sizing, tools and storage, and any caching or routing options.'
+---
+
+# AI Cost Model Designer
+
+## Positioning
+
+AI cost surprises kill pilots. The bill arrives only after the volume is real. Use this skill to model cost at three grains — per call, per user, per workflow — with the levers visible (model choice, caching, routing, prompt size, tool count, storage, retrieval) and budget guardrails specified before launch.
+
+A good model lets the team:
+
+1. Defend the proposal price and the production budget with arithmetic, not vibes.
+2. See which lever changes the bill the most.
+3. Detect cost anomalies before finance does.
+4. Compare alternatives (smaller model, caching, routing) on apples-to-apples.
+
+## When To Use This
+
+- A proposal or SOW needs a defensible cost line.
+- A pilot's bill is surprising leadership; need to model and explain it.
+- Choosing between models, frameworks, or platforms with different cost shapes.
+- Designing cost guardrails (budget caps, per-user quotas, alerting) for production.
+
+## When Not To Use This
+
+- Do not use this in place of [`model-selection-advisor`](../model-selection-advisor/SKILL.md) — selection drives cost; model first.
+- Do not use this in place of [`bottom-up-estimation`](../bottom-up-estimation/SKILL.md) for delivery effort estimates.
+- Do not use this for non-AI infra costs alone.
+
+## The Cost Components
+
+| Component | Driver |
+|---|---|
+| Model input tokens | Prompt size × calls; cache hit rate |
+| Model output tokens | Output length × calls |
+| Reasoning / extended-thinking tokens | Reasoning model class; slice routing |
+| Embedding tokens | Corpus size × re-embeds + per-query embedding |
+| Reranker calls | Top-k pre-rerank × call rate |
+| Tool calls | Per-tool price (search APIs, document AI, etc.) |
+| Vector store | Index size, queries, hosting tier |
+| Object/blob storage | Corpus size, traces, audit logs |
+| Compute (orchestration, agents) | Function/container hours |
+| Observability / eval / red-team | Monitoring stack, judge calls |
+| Human review (HITL) | Reviewer hours per item × items |
+| Egress / network | Region transfers |
+
+See [references/cost-lever-catalog.md](./references/cost-lever-catalog.md) for sizing assumptions and knobs.
+
+## Workflow
+
+1. Define the unit: per call, per user-month, per workflow execution. Model all three when relevant.
+2. Build the per-call cost from token math and tool/storage costs. Use platform-published prices; pin date.
+3. Layer in volume: calls/day, peak QPS, calls/user/month.
+4. Apply caching, routing, and reuse assumptions; show with and without.
+5. Add HITL costs where applicable.
+6. Output a sensitivity table on the top three levers.
+7. Recommend budget guardrails: per-user quotas, per-tenant ceilings, daily/weekly burn alerts, circuit breakers.
+8. Capture in [templates/cost-model.md](./templates/cost-model.md) and a spreadsheet.
+9. Pair with the [observability-design-for-ai](../observability-design-for-ai/SKILL.md) cost attribution to verify the model in production.
+
+## Output Structure
+
+1. Use case and volume assumptions.
+2. Per-call cost build-up.
+3. Per-user and per-workflow cost.
+4. Caching, routing, and reuse impact.
+5. Sensitivity analysis (top 3 levers).
+6. Budget guardrails (quotas, alerts, circuit breakers).
+7. Pricing date and refresh trigger.
+8. Risks and assumptions.
+
+A worked output is in [examples/example-support-copilot-cost-model.md](./examples/example-support-copilot-cost-model.md).
+
+## Common Pitfalls
+
+- **One average cost.** Hides the long tail; a few power users drive the bill.
+- **Forgetting reasoning tokens.** Reasoning/extended-thinking models charge for invisible thought.
+- **Static prices.** Vendor prices change; pin the date and set a refresh trigger.
+- **Ignoring HITL.** Human review is often the largest line item.
+- **No guardrails.** Without per-user quotas and burn alerts, runaway loops bankrupt the pilot.
+- **Cache assumed.** Cache hit rate is empirical; assume modestly until measured.
+- **Single-model assumption.** Routing changes the math materially.
+
+## Quality Bar
+
+- Per-call math is shown, not just totals.
+- Sensitivity is reported on the top three levers.
+- Guardrails are specified with thresholds.
+- Pricing has a date and a refresh trigger.
+
+## Related Skills
+
+- `model-selection-advisor` — drives cost via model choice.
+- `latency-budget-designer` — pairs with cost; smaller/faster usually cheaper.
+- `bottom-up-estimation` — delivery effort estimate.
+- `observability-design-for-ai` — production cost attribution.
+- `roi-calculator-builder` — rolls cost into ROI.

package/.jumpstart/skills/ai-cost-model-designer/evals/evals.json

@@ -0,0 +1,31 @@
+{
+  "skill_name": "ai-cost-model-designer",
+  "evals": [
+    {
+      "id": 1,
+      "prompt": "Build a cost model for a knowledge-grounded employee assistant: 20k users, ~8 calls/user/month, Azure OpenAI GPT-4o-mini default with GPT-4o on hard slices, hybrid AI Search index of ~5GB, 5% sampled human review. Include guardrails and sensitivity.",
+      "expected_output": "A cost model with per-call build-up, scenarios with caching/routing, per-user cost, sensitivity, and guardrails.",
+      "expectations": [
+        "Builds per-call cost from token math with input/output/embedding/tool/storage components.",
+        "Pins a pricing date and a refresh trigger.",
+        "Models scenarios with and without prompt caching and with and without routing, showing the delta.",
+        "Reports per-user-month cost and monthly total at the stated volume.",
+        "Includes a sensitivity table on the top three levers.",
+        "Specifies guardrails (per-user cap, tenant budget alerts, timeouts, circuit breaker, retry cap).",
+        "Adds a HITL line item for the 5% sampled review."
+      ]
+    },
+    {
+      "id": 2,
+      "prompt": "Our agentic copilot is burning $1,800/day in OpenAI costs in pilot — 10x the model. Help us find where the burn is and what guardrails to install in 48 hours.",
+      "expected_output": "A diagnosis and a guardrail set focused on stopping the burn fast.",
+      "expectations": [
+        "Identifies likely causes: agent loops, retry storms, large context per call, tool-call explosion, missing prompt cache, no per-user cap, reasoning model on easy slices.",
+        "Recommends immediate guardrails: per-user/per-tenant cap, max-iterations on agent loops, retry cap with backoff, circuit breaker, timeout.",
+        "Recommends fast-impact changes: prompt-cache enablement, router for easy slices, output cap, retrieval k reduction.",
+        "Calls out the observability needed to attribute cost (per user, prompt_version, model) and how to wire it quickly.",
+        "Sequences the actions by impact and elapsed time."
+      ]
+    }
+  ]
+}

package/.jumpstart/skills/ai-cost-model-designer/examples/example-support-copilot-cost-model.md

@@ -0,0 +1,51 @@
+# Example: Customer Support Copilot Cost Model
+
+## Pricing date
+- 2026-04-15. Refresh on Azure OpenAI quarterly price update.
+
+## Volume assumptions
+- 50,000 calls/day; peak 25 QPS; ~10 calls/user/month; 150,000 active users.
+
+## Per-call build-up (baseline GPT-4o)
+| Component | Units | Price | Cost |
+|---|---|---|---|
+| Input tokens | 4,000 | $0.0025 / 1k | $0.0100 |
+| Output tokens | 400 | $0.0100 / 1k | $0.0040 |
+| Embedding (per query) | 100 | $0.00013 / 1k | $0.0000 |
+| Reranker (semantic) | 1 call | included | $0.0000 |
+| Tool calls (CRM lookup) | 0.7 avg | $0.001 | $0.0007 |
+| Storage prorated | — | — | $0.0001 |
+| **Total / call** | | | **~$0.0148** |
+
+## Caching, routing, reuse
+| Scenario | Per-call | Monthly @ 1.5M calls | Δ |
+|---|---|---|---|
+| Baseline (GPT-4o) | $0.0148 | $22,200 | 0 |
+| Prompt cache 50% | $0.0098 | $14,700 | -34% |
+| Router (mini default, 4o for hard 25%) | $0.0073 | $10,950 | -51% |
+| Combined cache + router | $0.0058 | $8,700 | -61% |
+
+## Per-user
+- Combined: ~$0.058/user/month at 10 calls.
+
+## Sensitivity
+| Lever | Range | Cost range |
+|---|---|---|
+| Calls/user/month | 5–25 | $4,350 – $21,750 |
+| Output length | 200–800 | $7,200 – $11,400 |
+| Cache hit rate | 0–70% | $7,800 – $22,200 |
+
+## Guardrails
+- 50 calls/user/day cap with friendly throttle.
+- Tenant budget = monthly forecast × 1.3; alerts at 80%/95%.
+- p95 timeout 8s; agent max 4 iterations.
+- Circuit breaker: hourly burn > 5% monthly budget → throttle to mini-only.
+- Retry cap 2 with backoff.
+
+## HITL
+- 5% sampled review × 1.5M = 75k items × 30s avg × $1.50/min = $56k/month at peak. Treat as production cost line.
+
+## Risks and assumptions
+- Cache hit assumes prompt-prefix discipline; verify after week 2.
+- Router assumes ≤ 25% hard slice; eval gate enforces.
+- Vendor price change > 10% triggers rebuild.

package/.jumpstart/skills/ai-cost-model-designer/references/cost-components.md

@@ -0,0 +1,94 @@
+# Cost Components
+
+Canonical cost taxonomy for AI systems. Use as a checklist — not every component applies to every system.
+
+## Model Inference Costs
+
+### Input Tokens
+- Charged per 1K or 1M tokens depending on provider.
+- Includes: system prompt, user message, retrieved context, few-shot examples, tool schemas.
+- Optimization: prompt compression, context pruning, smaller system prompts, semantic caching.
+
+### Output Tokens
+- Typically 2–4x more expensive than input tokens.
+- Includes: generated response, tool-call arguments, structured output.
+- Optimization: constrained generation, shorter responses, schema enforcement.
+
+### Cached Input Tokens (where supported)
+- Azure OpenAI and Anthropic offer prompt caching at reduced rates.
+- Optimization: structure prompts so static portions (system prompt, few-shot) are cacheable.
+
+## Embedding Costs
+
+- Per-token cost for embedding generation.
+- Consider: initial bulk embedding, incremental updates, query-time embedding.
+- Optimization: batch embedding, embedding caching for repeated queries.
+
+## Retrieval / Vector DB Costs
+
+- Storage: per-GB for vectors + metadata.
+- Compute: per-query search cost (RU, vCore, or query-based pricing).
+- Scaling: cost at 1M, 10M, 100M vectors.
+- Optimization: tiered storage, TTL on stale embeddings, hybrid search reducing vector scan.
+
+## Tool/API Call Costs
+
+- External API calls (per-call, per-record pricing).
+- Internal service compute triggered by agent actions.
+- Include: authentication overhead, retry calls, webhook delivery.
+
+## Compute Costs
+
+- Orchestration runtime (Azure Functions, AKS, App Service).
+- Batch processing jobs (reindexing, eval runs, fine-tuning).
+- Optimization: serverless for spiky workloads, reserved instances for steady-state.
+
+## Storage Costs
+
+- Document storage (source corpus).
+- Conversation history / memory store.
+- Eval datasets and logs.
+- Optimization: tiered storage, retention policies, compression.
+
+## Egress Costs
+
+- Data transfer between regions.
+- Cross-cloud data movement.
+- Often forgotten; significant at scale.
+
+## Observability Costs
+
+- Log ingestion (per-GB).
+- Trace storage.
+- Dashboard compute.
+- Typically 5–15% of total system cost.
+
+## Optimization Levers
+
+| Lever | Saves | Tradeoff |
+|-------|-------|----------|
+| Semantic caching | 20–60% on repeated queries | Stale results, cache invalidation complexity |
+| Model routing (small for easy, large for hard) | 30–50% on model costs | Classification overhead, quality risk on edge cases |
+| Prompt compression | 10–30% on input tokens | Potential quality loss |
+| Batching | Reduced per-call overhead | Latency increase |
+| TTL on embeddings | Storage cost reduction | Stale retrieval risk |
+| Output length limits | Output token savings | Truncation risk |
+| Fine-tuned small model | 80–90% vs. frontier | Training cost, maintenance burden |
+
+## Guardrail Patterns
+
+- **Per-call budget:** reject or downgrade calls exceeding token threshold.
+- **Per-user daily cap:** prevent abuse or runaway usage.
+- **Per-workflow ceiling:** total cost for an end-to-end execution.
+- **Alert thresholds:** 50%, 80%, 95% of monthly budget with escalation path.
+- **Circuit breaker:** halt agent execution when cost anomaly detected.
+
+## Unit Economics Template
+
+| Metric | Value |
+|--------|-------|
+| Cost per workflow execution | $X.XX |
+| Cost per user per month | $X.XX |
+| Cost per transaction | $X.XX |
+| Break-even volume | N transactions/month |
+| Marginal cost at 10x | $X.XX per additional unit |

package/.jumpstart/skills/ai-cost-model-designer/references/cost-lever-catalog.md

@@ -0,0 +1,44 @@
+# Cost Lever Catalog
+
+## Token math (per call)
+```
+input_cost  = (prompt_tokens × input_price) × (1 - cache_hit_rate × cache_discount)
+output_cost = output_tokens × output_price
+reasoning_cost = reasoning_tokens × reasoning_price   # for o-series, extended thinking
+embedding_cost = (query_tokens × embed_price) + amortized_corpus_embed
+tool_cost   = sum(tool_calls × tool_price)
+storage     = (vector_store_tier + blob_storage) prorated per call
+total_call  = input + output + reasoning + embedding + tool + storage
+```
+
+## Sizing rules of thumb (verify against production)
+
+- Prompt overhead per call: 800–2,500 tokens (system + policy + few-shot + retrieved).
+- Retrieved context: 1,500–6,000 tokens after reranking.
+- Output: 150–800 tokens for assistant turns; up to 4,000 for summaries.
+- Reasoning models can spend 5–20× the visible output in hidden tokens; verify.
+- Embedding: ~1 token / 4 chars; corpus is one-time + delta on refresh.
+
+## Levers and impact
+
+| Lever | Typical impact | Caveats |
+|---|---|---|
+| Prompt cache | -30 to -70% on input tokens | Provider/region support; prompt prefix discipline |
+| Smaller model on easy slices | -50 to -90% on routed calls | Eval gate must hold |
+| Tighter retrieval (k, reranker) | -20 to -40% input | Recall risk |
+| Output length cap | -10 to -30% output | UX impact |
+| Tool-call reduction | varies | Sometimes adds prompt cost |
+| Self-hosted open-weights | -50 to -80% per token; +ops cost | GPU capacity, eval/safety burden |
+| Batching / async | latency tradeoff | Not for interactive |
+
+## Budget guardrails
+
+- Per-user daily token cap, with graceful refusal.
+- Per-tenant monthly budget with 80% / 95% alerts.
+- Per-call timeout and max-iterations on agentic loops.
+- Circuit breaker: if hourly burn > X% of monthly budget, throttle.
+- Retry caps with exponential backoff to avoid loop costs.
+
+## Verification post-launch
+- Compare modeled vs. actual cost weekly; tune.
+- Tag every call with feature, prompt_version, model — drives the tuning data.

package/.jumpstart/skills/ai-cost-model-designer/templates/cost-model.md

@@ -0,0 +1,60 @@
+# Cost Model — `<system>`
+
+## Pricing date
+- Captured: YYYY-MM-DD
+- Source: vendor public pricing
+- Refresh trigger: any vendor price change OR quarterly
+
+## Volume assumptions
+- Calls/day:
+- Peak QPS:
+- Calls/user/month:
+- Active users:
+
+## Per-call build-up
+| Component | Tokens / units | Unit price | Cost |
+|---|---|---|---|
+| Input tokens | | | |
+| Cached input | | | |
+| Output tokens | | | |
+| Reasoning tokens | | | |
+| Embedding | | | |
+| Reranker | | | |
+| Tool calls | | | |
+| Storage prorated | | | |
+| **Total / call** | | | |
+
+## Per-user / per-workflow
+- Per-user-month:
+- Per-workflow execution:
+
+## Caching, routing, reuse
+| Scenario | Per-call cost | Monthly total | Δ vs. baseline |
+|---|---|---|---|
+| Baseline (no cache, single model) | | | 0 |
+| With prompt cache | | | |
+| With router (small for easy, frontier for hard) | | | |
+| Combined | | | |
+
+## Sensitivity (top 3 levers)
+| Lever | Range | Cost range |
+|---|---|---|
+
+## Guardrails
+- Per-user quota:
+- Per-tenant budget + alert thresholds:
+- Per-call timeout + max-iterations:
+- Circuit breaker rule:
+- Retry cap:
+
+## HITL
+- Reviewer minutes per item:
+- Items per period:
+- Loaded reviewer cost / hour:
+- Total HITL cost / period:
+
+## Risks and assumptions
+- Cache hit rate assumption:
+- Routing accuracy assumption:
+- Volume forecast confidence:
+- Vendor price stability: