jumpstart-mode 1.1.13 → 1.1.14

package/.github/skills/pre-deployment-gotcha-review/references/gotcha-taxonomy.md

@@ -0,0 +1,292 @@
+# Gotcha Taxonomy
+
+Complete taxonomy of gotcha patterns organized by the ten required review categories. Use this reference when performing category sweeps to ensure no common pattern is missed.
+
+---
+
+## 1. Identity Gotchas
+
+### Provider and Protocol
+- SSO named but provider not identified (Ping, Okta, Entra ID, Auth0, custom LDAP)
+- Protocol not specified (OIDC, SAML 2.0, WS-Federation)
+- Multiple identity providers in scope but federation not designed
+- B2B vs B2C identity requirements conflated
+
+### MFA and Conditional Access
+- MFA required but not tested in non-production
+- Conditional access policies block unattended automation
+- Step-up authentication needed for sensitive operations but not designed
+- MFA enrollment flow not included in user onboarding
+
+### Claims and Roles
+- Claims mapping undefined between IdP and application
+- Group-to-role synchronization not designed
+- Custom claims require IdP configuration changes with separate approval
+- Role definitions exist in multiple systems with no single source of truth
+
+### Service Accounts and Non-Human Identity
+- Service accounts prohibited by policy but needed for background jobs
+- Agent/bot identity not supported by current IdP configuration
+- Certificate-based auth required but PKI process unknown
+- Workload identity federation not available in target environment
+
+### App Registration and Onboarding
+- App registration requires security review with multi-week SLA
+- Redirect URIs need approval for each environment
+- Consent grants require admin approval
+- Token lifetime policies conflict with session requirements
+
+---
+
+## 2. Authorization Gotchas
+
+### Role and Permission Design
+- Frontend route guards mistaken for real authorization
+- Authorization enforced only at UI layer, not API or data layer
+- Admin users get broader access than intended by design
+- Break-glass roles not defined for emergency access
+
+### Data-Level Security
+- Row-level security needed but database does not support it
+- Document-level permissions not preserved after transformation
+- Field-level security needed for PII/PHI columns
+- Tenant isolation assumed but not enforced at data layer
+
+### Approval and Workflow
+- Approval workflows lack separation of duties
+- Same user can submit and approve
+- Delegation model not defined for when approvers are unavailable
+- Override and audit trail requirements not specified
+
+### Permission Lifecycle
+- No process for permission review or recertification
+- Orphaned permissions accumulate when users change roles
+- Permission changes do not propagate to downstream caches
+- "Internal only" treated as substitute for authorization
+
+---
+
+## 3. API Gateway and Integration Gotchas
+
+### Gateway Presence
+- Apigee or other gateway sits in front of APIs but proposal assumes direct access
+- Gateway onboarding takes weeks and is not in timeline
+- Multiple gateways in path (e.g., Apigee + Azure API Management)
+- Gateway version or policy set differs between environments
+
+### API Contracts and Availability
+- Integration assumes API exists but only UI access is available
+- Required fields not exposed by current API version
+- API contracts unstable or undocumented
+- No OpenAPI spec available
+
+### Security and Policy
+- mTLS required but certificate provisioning not planned
+- API keys require rotation and the rotation process is manual
+- Rate limits or quotas will throttle the expected load
+- Request/response transformations handled in gateway but not documented
+
+### Onboarding Process
+- Developer portal registration required with separate approval
+- Sandbox and production APIs behave differently
+- API onboarding requires separate security review
+- Existing APIs have breaking changes planned
+
+---
+
+## 4. Legacy On-Prem Access Gotchas
+
+### Network Connectivity
+- Data exists but is not reachable from cloud environment
+- VPN, ExpressRoute, or Private Link required but not provisioned
+- Firewall rules require change request with multi-week approval
+- Proxy rules block required traffic patterns
+
+### System Access
+- Application has no API — only accessible through UI
+- RPA assumed because no programmatic access exists
+- Named user account required (no service account support)
+- Database reads require DBA approval and production change window
+
+### Data Extraction
+- Data extracts are manual processes run by specific individuals
+- Batch windows limit when data can be read
+- Legacy systems cannot handle real-time call patterns
+- "Read-only" queries still require production change approval
+
+### Performance and Stability
+- Legacy system performance degrades under additional load
+- No read replica available for query offloading
+- System maintenance windows conflict with AI processing schedule
+- Retry patterns could overwhelm legacy system
+
+---
+
+## 5. RAG Document Identity and ACL Gotchas
+
+### Document Identity
+- File path used as identity but changes on rename/move
+- Document title used as identity causing collisions
+- No durable source object ID (OID) defined
+- Version ID not preserved — cannot distinguish document versions
+- Content hash not stored for deduplication or change detection
+
+### Chunk Lineage
+- Chunks have no durable parent document ID
+- Chunk boundaries break semantic units
+- Overlapping chunks create duplicate retrieval
+- No mechanism to rebuild chunks when source changes
+- Chunk metadata does not include position/page information
+
+### Permission Preservation
+- ACLs not copied into vector index metadata
+- Permissions checked only at ingestion time, not retrieval time
+- Permission changes in source system do not propagate to index
+- Deleted documents remain indexed and retrievable
+- Group membership changes not reflected in retrieval filtering
+
+### Citation Integrity
+- Citations reveal document names users should not see
+- Citations cannot resolve back to source document for verification
+- Citation URIs break when documents move
+- No permission check on citation link access
+
+---
+
+## 6. Data Readiness Gotchas
+
+### Quality and Completeness
+- Demo data is clean but production data is not
+- Missing values in critical fields
+- Duplicates with no deduplication strategy
+- Data freshness unknown or stale
+
+### Ownership and Access
+- Data ownership unclear or disputed
+- Data cannot legally be used for the stated AI purpose
+- Data cannot leave source system due to policy
+- Multiple conflicting sources claimed as "single source of truth"
+
+### Labeling and Ground Truth
+- No labeled data exists for supervised learning
+- No ground truth exists for evaluation
+- Labels are inconsistent or contradictory
+- Historical decisions used as labels but contain bias
+
+### Sensitivity and Classification
+- PII, PHI, PCI data present but not classified
+- ITAR or export-controlled data in scope
+- Data residency requirements conflict with processing location
+- Retention policies undefined for AI-generated derivatives
+
+---
+
+## 7. AI, Agent, and Model Risk Gotchas
+
+### Behavioral Boundaries
+- AI described as "helper" but influences operational decisions
+- No defined behavior for when the AI should say "I don't know"
+- Agent tools can perform actions without human approval
+- Model confidence thresholds not defined for different risk tiers
+
+### Evaluation and Monitoring
+- No evaluation dataset exists
+- No telemetry to explain bad decisions after the fact
+- No drift monitoring for model or data distribution changes
+- Evaluation metrics not defined (faithfulness, groundedness, task success)
+
+### Security
+- Prompt injection from untrusted retrieved documents not addressed
+- Data exfiltration through crafted prompts not mitigated
+- Model outputs logged without sensitivity filtering
+- System prompt or tool descriptions exposed through prompt leakage
+
+### Human-in-the-Loop
+- No approval gate for high-risk AI decisions
+- No escalation path when AI confidence is low
+- No override mechanism for incorrect AI outputs
+- No feedback loop to capture and learn from corrections
+
+---
+
+## 8. Security, Compliance, and Legal Gotchas
+
+### Data Handling
+- Sensitive prompts logged without redaction
+- Model provider retention settings unknown or unfavorable
+- AI outputs create regulated records (HIPAA, SOX, GDPR)
+- Audit logs omit model version or source evidence
+
+### Compliance
+- Legal/compliance review assumed but not scheduled
+- Data residency conflicts with hosting or model provider location
+- Cross-border data transfer implications not assessed
+- EU AI Act risk classification not performed
+
+### Vendor and Model Terms
+- Model provider terms restrict commercial use of outputs
+- Fine-tuning data may be used for model improvement by provider
+- Indemnification unclear for AI-generated content
+- Liability for AI errors not allocated in contract
+
+### Key and Secret Management
+- Secrets hardcoded or stored in configuration files
+- No key rotation schedule defined
+- API keys shared across environments
+- Encryption at rest and in transit not confirmed for all data paths
+
+---
+
+## 9. Operational Readiness Gotchas
+
+### Support Model
+- MVP has no support owner defined
+- No runbook exists for common failure scenarios
+- No on-call rotation for AI-specific incidents
+- Original developer is the only person who understands the system
+
+### Monitoring and Alerting
+- No alerting for failed jobs or degraded quality
+- No cost monitoring or budget alerts
+- No model performance dashboards
+- Log aggregation not configured
+
+### Change Management
+- Prompt changes not governed or versioned
+- Knowledge source updates have no review process
+- Model version changes deployed without evaluation
+- No rollback path for bad deployments
+
+### Ongoing Maintenance
+- No one owns re-indexing failures
+- No schedule for knowledge base refresh
+- Secret rotation not automated
+- No process for user feedback review
+
+---
+
+## 10. Delivery, Scope, and Commercial Risk Gotchas
+
+### Scope Boundaries
+- "Simple integration" hides weeks of access and onboarding work
+- UAT depends on users who are not assigned or available
+- Security review timeline not included in project plan
+- Production data access not available during build phase
+
+### Estimation
+- Scope is not bounded enough for fixed-fee pricing
+- Integration complexity underestimated
+- Access approval lead times not factored into timeline
+- Environment provisioning assumed instant
+
+### Acceptance
+- Acceptance criteria are subjective or unmeasurable
+- POC described like a demo but stakeholders expect production readiness
+- "Done" not defined for AI outputs (what accuracy is acceptable?)
+- User acceptance testing plan not defined
+
+### Commercial
+- Change request process not established
+- Out-of-scope items not explicitly listed
+- Client responsibilities not enumerated
+- Hypercare and transition period not discussed

package/.github/skills/pre-deployment-gotcha-review/references/identity-auth-mfa.md

@@ -0,0 +1,94 @@
+# Identity, Authentication, and MFA — Deep Dive
+
+## Why This Category Matters
+
+Identity integration is one of the most frequently underestimated aspects of enterprise AI delivery. A document that says "users will log in with SSO" is making a statement that could hide weeks of IAM work including app registration, claims mapping, protocol negotiation, MFA testing, conditional access evaluation, and security review.
+
+---
+
+## Questions the Review Must Ask
+
+### Identity Provider
+- Which identity provider? (Ping, Okta, Entra ID, Auth0, Cognito, custom LDAP, ADFS)
+- Is there more than one IdP in scope (e.g., internal users on Entra, partners on Ping)?
+- Is identity federated across multiple directories?
+- Is the IdP managed by the client, a third party, or shared services?
+
+### Protocol
+- Is OIDC or SAML 2.0 required?
+- Is WS-Federation still in use?
+- Does the chosen framework/platform support the required protocol?
+- Are there protocol version requirements?
+
+### App Registration
+- Is app registration required with the IdP?
+- Who approves app registrations?
+- What is the SLA for app registration approval?
+- Are redirect URIs per-environment?
+- Are wildcard redirect URIs allowed?
+- Is there a developer self-service portal or is it all manual?
+
+### Claims and Tokens
+- What claims are available in the token?
+- Are custom claims needed?
+- Who configures custom claims in the IdP?
+- Are group memberships available as claims?
+- What is the maximum token size (group claim explosion)?
+- Are roles in the token, a database, or both?
+
+### MFA
+- Is MFA required for all users?
+- Is MFA required for all environments (including dev/test)?
+- Which MFA methods are supported (push, TOTP, FIDO2, SMS)?
+- Is step-up authentication required for sensitive operations?
+- Does MFA break automation or service-to-service calls?
+
+### Conditional Access
+- Are conditional access policies applied?
+- Do they restrict by device, location, network, or risk level?
+- Do they affect service accounts or daemon apps?
+- Are there exceptions for automation scenarios?
+- Is the conditional access policy documented or must it be discovered?
+
+### Service Accounts and Non-Human Identity
+- Are service accounts needed for background processing?
+- Are service accounts permitted by security policy?
+- Is managed identity or workload identity federation available?
+- Are certificates required for service authentication?
+- What is the certificate provisioning and rotation process?
+- Is client credentials flow supported?
+
+### Delegated vs Application Permissions
+- Are users acting as themselves (delegated) or is the app acting on its own (application)?
+- Does the downstream API support both permission models?
+- Are consent grants required?
+- Who can grant admin consent?
+
+---
+
+## Common Gotcha Patterns
+
+### The "SSO is simple" trap
+The document says "SSO integration" but the client uses Ping Federate with custom claims, SAML assertions that require specific attribute mappings, and a security review process with a 4-week SLA.
+
+### The MFA automation blocker
+Background jobs or daemon services cannot complete MFA challenges. The architecture assumes user credentials can be used by automation.
+
+### The conditional access surprise
+The AI application runs in a cloud environment. Conditional access policies require requests to originate from a corporate network, blocking cloud-hosted services.
+
+### The group explosion
+The user has 200+ group memberships. Token size exceeds limits. Claims must be queried from Graph API at runtime instead of embedded in tokens.
+
+### The multi-IdP federation gap
+Internal users authenticate with Entra ID, but partner users authenticate with Ping. The application must handle both with different claims schemas and token formats.
+
+---
+
+## Mitigation Patterns
+
+1. **IAM Spike** — Schedule a dedicated spike before build to validate: IdP, protocol, claims, app registration process, MFA behavior, conditional access impact, and role mapping.
+2. **Auth Prototype** — Build a minimal app that completes the full auth flow in the target environment before committing to scope.
+3. **Security Review Lead Time** — Include IAM security review in the project timeline with appropriate buffer (typically 2-6 weeks).
+4. **Service Account Strategy** — Define the non-human identity approach early: managed identity, workload identity federation, certificates, or client credentials.
+5. **Claims Contract** — Document the expected claims from the IdP as a contract the application depends on.

package/.github/skills/pre-deployment-gotcha-review/references/legacy-onprem-access.md

@@ -0,0 +1,92 @@
+# Legacy On-Prem Data and Application Access — Deep Dive
+
+## Why This Category Matters
+
+Many AI initiatives require data or system access that lives on-premises, behind corporate firewalls, in legacy databases, or in applications that predate modern API design. These systems often have no programmatic access, strict change control, limited availability windows, and complex approval chains that are invisible to cloud-first architecture designs.
+
+---
+
+## Questions the Review Must Ask
+
+### Network Connectivity
+- Is the source system reachable from the target compute environment?
+- Is VPN, ExpressRoute, AWS Direct Connect, or Private Link required?
+- Are these already provisioned or must they be requested?
+- Are firewall rule changes needed?
+- Who approves firewall changes and what is the SLA?
+- Are proxy servers in the path?
+- Does the network path support the required bandwidth?
+
+### System Type
+- What is the system? (SQL Server, Oracle, DB2, AS/400, mainframe, SAP, custom app)
+- How old is it?
+- Is it actively maintained?
+- Is there documentation?
+- Who are the subject matter experts?
+- Is the system shared across business units?
+
+### Access Method
+- Is there an API? (REST, SOAP, ODBC, JDBC, file-based, messaging)
+- If no API, how is data currently accessed? (UI-only, batch export, report)
+- Can a new API be created? Who would build it? What timeline?
+- Is direct database access allowed?
+- Are there stored procedures or views available?
+- Is there an ETL process that already extracts this data?
+
+### Credentials and Permissions
+- What credentials are needed? (Windows auth, SQL auth, certificate, API key)
+- Are service accounts available for this system?
+- Who grants access?
+- Is access role-based or named-user?
+- Can one credential read all needed data or are multiple needed?
+- Are there separate credentials for different data domains?
+
+### Data Extraction Constraints
+- Are there batch windows when queries must run?
+- Are there performance constraints on the source system?
+- Is there a read replica or reporting database?
+- Can the system handle concurrent queries from an AI workload?
+- Are there data volume limits on extracts?
+- Is change data capture (CDC) available?
+- What is the data freshness requirement vs. what is achievable?
+
+### Change Control
+- Is the source system under strict change control?
+- Do new queries require a change request?
+- Is there a DBA review process?
+- Are production changes restricted to maintenance windows?
+- Can non-production environments be used for development?
+- Do non-production environments have realistic data?
+
+---
+
+## Common Gotcha Patterns
+
+### The unreachable data
+The architecture diagram shows a cloud AI agent querying an on-prem database, but there is no network path, no VPN, and no Private Link provisioned.
+
+### The UI-only system
+The "source system" is a legacy application with no API. The only way to access data is through a thick client or browser-based UI. RPA is silently assumed.
+
+### The named-user trap
+The legacy system requires named user accounts. Service accounts are prohibited. Every query is attributed to a human who must be logged in.
+
+### The batch-only access
+Real-time access is assumed, but the system only supports nightly batch extracts. Data will always be 24+ hours stale.
+
+### The performance cliff
+Adding AI query load to a legacy system degrades performance for existing users. The DBA refuses to allow additional connections.
+
+### The change control wall
+Even a "read-only" query against production requires a change request, DBA review, testing, and a production deployment window — 4-6 weeks minimum.
+
+---
+
+## Mitigation Patterns
+
+1. **Connectivity Validation** — Prove network path from AI compute to source system before any other work. A simple TCP test can save weeks.
+2. **Access Method Discovery** — Spend time in discovery confirming the actual access method (API, database, batch file, UI-only) rather than assuming.
+3. **Read Replica Strategy** — If direct production access is risky, identify or request a read replica or reporting database.
+4. **Data Pipeline First** — Build the data extraction pipeline as the first sprint item. If access is harder than expected, surface it early.
+5. **Legacy System SME Engagement** — Identify and engage the legacy system SME early. They hold institutional knowledge that is not documented anywhere.
+6. **Fallback to Batch** — If real-time access is not feasible, design for batch ingestion with appropriate staleness SLAs rather than blocking on impossible real-time access.

package/.github/skills/pre-deployment-gotcha-review/references/mcp-extension-guide.md

@@ -0,0 +1,102 @@
+# MCP Extension Guide
+
+## Overview
+
+When performing a pre-deployment gotcha review, findings often depend on current implementation behavior of specific platforms, libraries, SDKs, or tools. Rather than relying on potentially stale knowledge, use available MCP (Model Context Protocol) tools to fetch current documentation before making recommendations.
+
+---
+
+## When to Use MCP Tools
+
+Use MCP-assisted lookup when a finding or mitigation involves:
+
+- A specific library version's behavior (e.g., "does LangChain support permission-aware retrieval?")
+- A platform's current configuration options (e.g., "does Azure AI Search support per-document ACL filtering?")
+- An SDK's authentication patterns (e.g., "how does MSAL handle conditional access challenges?")
+- A vendor tool's policy or limitation (e.g., "what are Apigee's default rate limits?")
+- A framework's integration approach (e.g., "how does Semantic Kernel connect to vector stores?")
+
+Do NOT use MCP lookup for:
+- General architectural principles
+- Common security best practices
+- Standard delivery methodology questions
+- Questions the document itself should answer
+
+---
+
+## Context7 Usage
+
+Context7 provides up-to-date library and framework documentation. Use it when reviewing gotchas tied to:
+
+### Frontend / Backend Frameworks
+- React, Vue, Next.js, Nuxt
+- FastAPI, Flask, Django
+- .NET, ASP.NET Core, Blazor
+- Express, NestJS
+
+### AI / ML Frameworks
+- LangChain, LlamaIndex
+- Semantic Kernel, Microsoft.Extensions.AI
+- OpenAI SDK (Python, .NET, JS)
+- Azure AI SDK
+- Hugging Face Transformers
+
+### Vector Stores and Search
+- Azure AI Search
+- pgvector / PostgreSQL
+- Pinecone, Weaviate, Qdrant
+- ChromaDB, Milvus
+
+### Identity and Auth Libraries
+- MSAL (Microsoft Authentication Library)
+- Passport.js, next-auth
+- OAuth2/OIDC libraries
+- SAML libraries (e.g., node-saml, python-saml)
+
+### Cloud SDKs
+- Azure SDKs (@azure/*)
+- AWS SDKs (boto3, @aws-sdk/*)
+- Google Cloud SDKs
+
+### API Gateway
+- Apigee libraries and tooling
+- Azure API Management
+- Kong configuration
+
+---
+
+## MCP Citation Requirements
+
+When MCP tools are used during review:
+
+1. **Cite separately** — MCP-sourced information should be cited separately from source document citations
+2. **Label the source** — Indicate that the information came from current documentation (e.g., "Per current Azure AI Search documentation...")
+3. **Version awareness** — Note if the behavior is version-specific
+4. **Freshness caveat** — If the documentation is older than 6 months, note this
+
+---
+
+## Example Usage Patterns
+
+### Pattern 1: Validating an integration assumption
+The document says "we will use Azure AI Search for RAG." The review asks whether Azure AI Search supports the permission model described.
+→ Use Context7 to fetch current Azure AI Search documentation on security trimming and access control.
+
+### Pattern 2: Checking SDK compatibility
+The document specifies a technology stack. A finding questions whether the chosen OIDC library supports the required flow.
+→ Use Context7 to fetch current library documentation on supported flows and configuration.
+
+### Pattern 3: Verifying rate limits
+The document proposes high-volume API calls. A finding questions whether the target service can handle the load.
+→ Use the relevant MCP tool to check current documentation on rate limits and quotas.
+
+---
+
+## Fallback When MCP Is Not Available
+
+If MCP tools are not configured or not available:
+
+1. Note that the recommendation would benefit from current documentation verification
+2. Label the finding confidence as "Medium" rather than "High" for implementation-specific claims
+3. Recommend that the team verify with current documentation as a mitigation step
+4. Do not make specific version or configuration recommendations without current documentation

package/.github/skills/pre-deployment-gotcha-review/references/operational-readiness.md

@@ -0,0 +1,102 @@
+# Operational Readiness — Deep Dive
+
+## Why This Category Matters
+
+AI systems require operational support that goes beyond traditional software. Models drift, prompts need updating, knowledge bases go stale, costs can spike unexpectedly, and AI-specific incidents (hallucination events, data exposure, prompt injection attacks) require specialized response. If no one owns operations after MVP, the system will degrade silently.
+
+---
+
+## Questions the Review Must Ask
+
+### Support Ownership
+- Who supports this system after MVP delivery?
+- Is there a defined L1/L2/L3 support model?
+- Who is on-call for AI-specific incidents?
+- Is the support team trained on AI-specific failure modes?
+- Is there a handoff plan from the delivery team to operations?
+- Is hypercare defined (scope, duration, team)?
+
+### Runbooks and Procedures
+- Do runbooks exist for common failure scenarios?
+- Is there a runbook for: model service outage, degraded quality, cost spike, prompt injection detected, source system down, re-indexing failure?
+- Are runbooks tested/rehearsed?
+- Who updates runbooks when the system changes?
+
+### Monitoring and Alerting
+- What is monitored? (Availability, latency, error rate, quality, cost)
+- Are there AI-specific quality metrics? (Faithfulness, groundedness, rejection rate)
+- Are alerts configured for critical conditions?
+- Who receives alerts?
+- What are the SLAs for alert response?
+- Is there a dashboard for operational health?
+
+### Incident Response
+- What constitutes an AI incident? (Hallucination causing harm, data exposure, unauthorized action, prompt injection success)
+- Is there an incident severity classification for AI events?
+- Is there a communication plan for AI incidents?
+- Who makes the decision to take the AI offline?
+- Is there a post-incident review process?
+
+### Rollback and Recovery
+- Can the system be rolled back to a previous version?
+- Can prompts be rolled back independently of code?
+- Can the knowledge base be rolled back?
+- What is the rollback time?
+- Is there a "kill switch" to disable AI and fall back to manual process?
+- How is rollback tested?
+
+### Cost Management
+- Who monitors AI costs (token usage, API calls, compute)?
+- Are there cost alerts or budget limits?
+- Is there a cost anomaly detection mechanism?
+- What happens when budget is exceeded?
+- Are there cost optimization strategies in place (caching, routing, model selection)?
+- Is cost per transaction tracked?
+
+### Knowledge Base and Prompt Management
+- Who owns the knowledge base content?
+- How is the knowledge base refreshed?
+- Who updates prompts when requirements change?
+- Is there a review process for prompt changes?
+- Are prompt changes versioned and traceable?
+- Is there a staging environment for testing prompt changes?
+
+### Secrets and Credential Management
+- Are secrets rotated on schedule?
+- Is rotation automated or manual?
+- What happens when a secret expires unexpectedly?
+- Are there alerts for expiring certificates/keys?
+- Who is responsible for credential lifecycle?
+
+---
+
+## Common Gotcha Patterns
+
+### The developer-dependent system
+The system works only when the original developer is watching it. When they go on vacation, issues accumulate unaddressed.
+
+### The unmonitored degradation
+Quality degrades slowly (data drift, model drift, knowledge staleness). No one notices until users complain loudly enough to escalate.
+
+### The prompt change without review
+Someone updates a prompt to fix one user's complaint. The change breaks behavior for other users. No review process, no rollback, no versioning.
+
+### The runaway costs
+A change in user behavior or a prompt loop causes token consumption to spike 10x. No cost alert exists. The invoice arrives at month end.
+
+### The re-indexing failure nobody owns
+The nightly re-indexing job fails. Source documents are updated but the AI still returns stale information. No one owns the reconciliation process.
+
+### The missing kill switch
+The AI starts producing harmful outputs (hallucinated legal advice, wrong medical information). There is no fast way to disable it and fall back to a manual process.
+
+---
+
+## Mitigation Patterns
+
+1. **Operations Handoff Plan** — Define the operational support model during delivery, not after. Include: owner, team, SLAs, tools, access, and training.
+2. **Day-2 Runbook** — Write runbooks for the top 5 failure scenarios during build. Test them before go-live.
+3. **Quality Monitoring** — Implement automated quality sampling (random sample of outputs evaluated against criteria) from day one.
+4. **Cost Guards** — Set budget alerts, per-transaction cost limits, and automatic throttling before deployment.
+5. **Kill Switch** — Implement a feature flag or configuration that can disable AI features instantly, falling back to a non-AI path.
+6. **Prompt Governance** — Version all prompts, require review for changes, test in staging, and maintain rollback capability.