mindforge-cc 10.0.3 → 11.0.0

package/.mindforge/skills/event-driven-architecture/SKILL.md

@@ -0,0 +1,162 @@
+---
+name: event-driven-architecture
+version: 1.0.0
+min_mindforge_version: 0.1.0
+status: stable
+triggers: event driven architecture, event bus, pub sub pattern, event schema design, ordering guarantee, exactly once delivery, dead letter topic, event sourcing integration, event catalog, event versioning strategy, event replay strategy, event consumer group
+---
+
+# Skill — Event-Driven Architecture
+
+## When this skill activates
+Any task involving event bus design, pub/sub patterns, message ordering,
+delivery guarantees, dead letter handling, or event schema evolution.
+
+## Mandatory actions when this skill is active
+
+### Before writing any code
+1. Classify event types (domain, integration, or command events).
+2. Define delivery guarantees required for each event stream.
+3. Design the event schema with forward/backward compatibility in mind.
+
+### During implementation
+- Make all consumers idempotent (safe to process same event multiple times).
+- Implement dead letter topic handling with alerting.
+- Use partition keys to maintain ordering where required.
+
+### After implementation
+- Register events in the event catalog with schema and owner.
+- Add consumer lag monitoring.
+- Document retry and failure handling in ARCHITECTURE.md.
+
+## Event Types
+
+### Domain Events
+- Facts about what happened in a bounded context.
+- Named in past tense: `OrderPlaced`, `PaymentProcessed`, `UserRegistered`.
+- Owned by the producing domain — consumers must adapt.
+- Immutable once published.
+
+### Integration Events
+- Cross-boundary communication between services.
+- May be transformed from domain events (different schema, less detail).
+- Published on shared event bus (Kafka, SNS, EventBridge).
+
+### Command Events
+- Request for action (not a fact).
+- Named as imperative: `ProcessPayment`, `SendNotification`.
+- Exactly one consumer expected to handle.
+- Requires acknowledgment/response.
+
+## Delivery Guarantees
+
+### At-Most-Once
+- Fire and forget. No retries.
+- Use for: metrics, analytics, non-critical notifications.
+- Risk: message loss on failure.
+
+### At-Least-Once (Recommended Default)
+- Retry until acknowledged.
+- Consumers MUST be idempotent.
+- Use for: most business events.
+- Risk: duplicate processing (mitigated by idempotency).
+
+### Exactly-Once (Expensive)
+- Requires transactional outbox + deduplication.
+- Use for: financial transactions, inventory changes.
+- Implementation: idempotency key + processed event log.
+
+## Ordering Guarantees
+
+### Per-Partition Ordering
+- Events with the same partition key are ordered.
+- Partition key = entity ID (e.g., order_id, user_id).
+- Different entities may be processed out of order (acceptable).
+
+### Global Ordering
+- Extremely expensive — single partition = no parallelism.
+- Almost never needed — design around per-entity ordering instead.
+
+### Kafka Partition Key Design
+```
+topic: order-events
+partition_key: order_id
+result: all events for order-123 arrive in sequence
+```
+
+## Schema Evolution
+
+### Compatibility Modes (Avro/Protobuf)
+- **Backward compatible**: new schema can read old data (add optional fields).
+- **Forward compatible**: old schema can read new data (ignore unknown fields).
+- **Full compatible**: both directions (safest, most restrictive).
+
+### Rules for Safe Evolution
+- Adding optional fields: always safe.
+- Removing fields: only if no consumers depend on them.
+- Renaming fields: treat as remove + add (breaking).
+- Changing field types: always breaking.
+
+### Schema Registry
+- Central registry of all event schemas with version history.
+- Validates compatibility before allowing schema updates.
+- Consumers reference schema by ID (embedded in message header).
+
+## Consumer Groups
+
+### Competing Consumers (Scaling Pattern)
+- Multiple instances in same group share the load.
+- Each message processed by exactly one instance.
+- Use for: order processing, notification sending.
+- Scale by adding more consumers (up to partition count).
+
+### Broadcasting (Fan-Out Pattern)
+- Each consumer group gets every message.
+- Use for: audit logging, cache invalidation, analytics.
+- Different groups process independently at their own pace.
+
+## Dead Letter Topics (DLT)
+
+### Flow
+```
+message → consumer → FAIL → retry (3x with backoff) → FAIL → DLT → alert
+```
+
+### Requirements
+- Every consumer MUST have a DLT configured.
+- DLT messages retain full context (original message + error + attempt count).
+- Alert on first DLT message (don't silently accumulate).
+- Manual resolution workflow: inspect → fix → replay or discard.
+
+### Retry Strategy
+- Attempt 1: immediate.
+- Attempt 2: 1 second delay.
+- Attempt 3: 10 second delay.
+- After 3 failures: route to DLT.
+
+## Event Catalog
+
+Every event in the system must be registered:
+
+| Field | Description |
+|-------|-------------|
+| Event name | `OrderPlaced` |
+| Schema version | `v3` |
+| Owner (team) | Order Service team |
+| Producers | order-service |
+| Consumers | notification-svc, analytics-svc, fulfillment-svc |
+| Partition key | order_id |
+| Delivery guarantee | at-least-once |
+| Retention | 7 days |
+
+## Self-check before task completion
+
+Before marking a task done when this skill was active:
+
+- [ ] Did I read the full SKILL.md before starting? (Not just the triggers)
+- [ ] Are all consumers idempotent?
+- [ ] Is ordering guaranteed per entity via partition keys?
+- [ ] Is dead letter topic configured with alerting?
+- [ ] Are event schemas registered in the catalog?
+- [ ] Is schema evolution backward-compatible?
+- [ ] Are consumer groups configured correctly (competing vs broadcasting)?

package/.mindforge/skills/experiment-design/SKILL.md

@@ -0,0 +1,139 @@
+---
+name: experiment-design
+version: 1.0.0
+min_mindforge_version: 10.0.4
+status: stable
+triggers: experiment design, A/B testing architecture, statistical significance, sample size calculator, guardrail metric, experiment lifecycle, hypothesis testing, control variant, experiment analysis, metric sensitivity, experiment duration, randomization unit
+---
+
+# Skill — Experiment Design (Rigorous A/B Testing Architecture)
+
+## When this skill activates
+When designing, planning, or analyzing A/B tests, multivariate experiments, or
+any controlled experiment that requires statistical rigor. Use for feature rollout
+decisions, conversion optimization, pricing tests, or any scenario where you need
+to measure the causal impact of a change.
+
+Core principle: **Hypothesis-first** — never launch an experiment without a written
+hypothesis that specifies the expected effect direction, magnitude, and mechanism.
+
+## Mandatory actions when this skill is active
+
+### Before experiment begins
+
+1. **Write the hypothesis in structured format:**
+   ```
+   If we [change X],
+   then [metric Y] will [improve/degrade] by [Z amount]
+   because [causal mechanism].
+   ```
+   - The hypothesis must be falsifiable
+   - The expected effect size must be realistic (based on prior data or industry benchmarks)
+   - The causal mechanism must be articulated (not just "it will be better")
+
+2. **Calculate sample size:**
+   ```
+   Inputs:
+   - Baseline conversion rate (current metric value)
+   - Minimum Detectable Effect (MDE): smallest improvement worth detecting
+   - Statistical significance level (alpha): typically 0.05
+   - Statistical power (1-beta): typically 0.80
+   - Number of variants (control + treatments)
+
+   Output:
+   - Required sample size per variant
+   - Estimated duration = required_N / daily_traffic_per_variant
+   ```
+
+   Rules:
+   - MDE should be the smallest PRACTICALLY significant effect (not just statistically significant)
+   - If duration > 8 weeks: increase MDE or find higher-traffic surface
+   - Never compromise on power — underpowered experiments waste everyone's time
+
+3. **Define guardrail metrics:**
+   ```json
+   {
+     "primary_metric": "conversion_rate",
+     "secondary_metrics": ["revenue_per_user", "engagement_time"],
+     "guardrail_metrics": [
+       {"name": "page_load_time_p95", "threshold": "+200ms", "action": "stop"},
+       {"name": "error_rate", "threshold": "+0.5%", "action": "stop"},
+       {"name": "revenue_per_session", "threshold": "-2%", "action": "alert"}
+     ]
+   }
+   ```
+   - Guardrails are metrics that MUST NOT degrade beyond threshold
+   - Violation of a guardrail = experiment stopped regardless of primary metric
+   - Always include: performance, error rate, and revenue as guardrails
+
+4. **Choose randomization unit:**
+   - **User-level**: Default for most experiments (consistent experience across sessions)
+   - **Session-level**: For UI experiments where cross-session contamination is acceptable
+   - **Page-level**: Only for layout experiments with no carryover effects
+   - **Device-level**: When logged-out users are significant traffic
+   - Rule: randomization unit >= analysis unit (never analyze at user level if randomized at page level)
+
+### During experiment
+
+1. **Minimum duration rules:**
+   - Run for at least 1 full business cycle (typically 7 days minimum)
+   - Recommended: 2 full weeks to capture weekday/weekend variation
+   - NEVER stop early because results "look significant" (peeking problem)
+   - If using sequential testing: define stopping rules BEFORE launch
+
+2. **Monitoring protocol:**
+   - Check guardrail metrics daily
+   - Do NOT check primary metric significance until planned end date
+   - If peeking is necessary: use group sequential methods with alpha spending
+   - Log any system issues that may contaminate results (outages, bugs, other launches)
+
+3. **Sample Ratio Mismatch (SRM) check:**
+   - Verify variant assignment is balanced (chi-square test, p < 0.001 = SRM)
+   - SRM invalidates the experiment — do not trust results
+   - Common causes: bot filtering, redirect failures, bucketing bugs
+
+### After experiment (analysis)
+
+1. **Statistical analysis checklist:**
+   - [ ] Confirm no SRM
+   - [ ] Check primary metric: p-value < 0.05 AND confidence interval excludes 0
+   - [ ] Check practical significance: is the effect size large enough to matter?
+   - [ ] Check guardrail metrics: no violations
+   - [ ] Check segment consistency: does the effect hold across key segments?
+   - [ ] Check novelty/primacy effects: is the effect stable over time?
+
+2. **Decision framework:**
+   ```
+   IF p < 0.05 AND practical significance AND no guardrail violations:
+     → SHIP (roll out to 100%)
+   IF p < 0.05 BUT guardrail violation:
+     → ITERATE (fix guardrail issue, re-run)
+   IF p >= 0.05 AND confidence interval includes meaningful effects:
+     → EXTEND (underpowered, run longer or increase traffic)
+   IF p >= 0.05 AND confidence interval excludes meaningful effects:
+     → KILL (the change doesn't work, move on)
+   ```
+
+3. **Document the result:**
+   ```markdown
+   ## Experiment Result: [name]
+   - Hypothesis: [statement]
+   - Duration: [days] | Sample: [N per variant]
+   - Primary metric: [baseline] → [variant] ([+/-X%], p=[value])
+   - Guardrails: [all clear / violations]
+   - Decision: SHIP / ITERATE / EXTEND / KILL
+   - Learning: [what did we learn about user behavior?]
+   ```
+
+## Self-check before task completion
+
+Before marking a task done when this skill was active:
+
+- [ ] Did I write a structured hypothesis with expected effect size and mechanism?
+- [ ] Did I calculate required sample size based on MDE and baseline?
+- [ ] Did I define guardrail metrics with explicit thresholds?
+- [ ] Did I choose an appropriate randomization unit?
+- [ ] Did I set minimum duration (>= 1 business cycle)?
+- [ ] Did I plan for the peeking problem (no early stopping without sequential testing)?
+- [ ] Did I document the decision framework (ship/iterate/extend/kill)?
+- [ ] Is the experiment design reproducible by another engineer?

package/.mindforge/skills/experiment-platform/SKILL.md

@@ -0,0 +1,43 @@
+---
+name: experiment-platform
+version: 1.0.0
+min_mindforge_version: 10.6.0
+status: stable
+triggers: experiment platform design, experimentation infrastructure, statistical rigor experiment, guardrail metric design, experiment velocity, feature flag experiment, experiment analysis automation, sample size calculation, multi-variant testing, experiment platform lifecycle, experiment review process, sequential testing
+compose: experiment-design
+---
+
+# Skill — Experiment Platform
+
+## When this skill activates
+This skill activates when building experimentation infrastructure, implementing statistical testing frameworks, or designing A/B testing platforms. Use when organizations need to scale testing velocity while maintaining statistical rigor.
+
+## Mandatory actions when this skill is active
+
+### Before writing any code
+1. Define experiment framework components: randomization service, exposure logging, metric computation pipeline, and statistical analysis engine
+2. Establish statistical rigor standards: minimum sample size, power (80%+), significance level (5%), minimum detectable effect, and multiple comparison corrections
+3. Design guardrail metrics framework: business health (revenue, retention), user experience (latency, errors), and ecosystem health (partner impact)
+4. Plan experiment lifecycle states: draft, review, running, paused, completed, archived with transition criteria and approval gates
+
+### During implementation
+- Implement consistent randomization using stable hashing (user_id + experiment_id) ensuring users see same variant across sessions
+- Build exposure logging capturing: timestamp, user_id, experiment_id, variant, context for accurate sample size and covariate adjustment
+- Create metric computation pipeline with: numerator/denominator structure, winsorization for outliers, delta method for ratios, bootstrap for confidence intervals
+- Design sequential testing capability for early stopping: alpha spending functions, futility boundaries, and minimum runtime requirements
+- Implement stratified analysis for heterogeneous treatment effects: by platform, user segment, geography with interaction effect testing
+- Build automated guardrail checks: alert on significant negative movement in critical metrics with experiment auto-pause capability
+- Create experiment metadata repository: hypothesis, success criteria, related experiments, learnings, and decision outcome for institutional knowledge
+
+### After implementation
+- Generate automated experiment scorecards: primary metric movement, guardrail status, statistical significance, practical significance, recommendation
+- Build experiment catalog with search and discovery: hypothesis library, metric glossary, analysis templates, and historical results
+- Create experimentation health dashboard: velocity (experiments/week), quality (statistical power distribution), impact (significant wins), and coverage (features tested)
+- Document statistical methodology: test selection, variance reduction techniques, multiple comparison approach, and sequential testing procedures
+
+## Self-check before task completion
+- [ ] Randomization service ensures stable assignment and balanced allocation across variants
+- [ ] Exposure logging captures all necessary context for accurate analysis and debugging
+- [ ] Statistical analysis engine implements proper corrections for multiple comparisons and peeking
+- [ ] Guardrail metrics monitored automatically with alerting and experiment pause capability
+- [ ] Experiment lifecycle enforces minimum runtime and sample size before declaring results

package/.mindforge/skills/feature-engineering/SKILL.md

@@ -0,0 +1,42 @@
+---
+name: feature-engineering
+version: 1.0.0
+min_mindforge_version: 10.6.0
+status: stable
+triggers: ML feature engineering workflow, feature selection method, feature transformation, feature importance analysis, automated feature discovery, feature scaling normalization, feature interaction, temporal feature extraction, text feature engineering, categorical encoding strategy, feature validation, domain feature creation
+---
+
+# Skill — Feature Engineering
+
+## When this skill activates
+This skill activates when building ML pipelines that require feature creation, transformation, or selection. Use when designing feature stores, implementing automated feature discovery, or optimizing model input representation.
+
+## Mandatory actions when this skill is active
+
+### Before writing any code
+1. Conduct exploratory data analysis to understand feature distributions, missing patterns, correlations, and domain-specific relationships
+2. Define feature engineering strategy: target encoding risks, temporal leakage prevention, train-test split boundaries, and cross-validation approach
+3. Document business logic for derived features with domain expert validation and interpretability requirements
+4. Establish feature quality metrics: null rates, cardinality, stability over time, and correlation with target variable
+
+### During implementation
+- Implement feature transformations within sklearn Pipelines or similar frameworks to prevent train-test leakage
+- Use robust scaling methods appropriate to distribution (StandardScaler for normal, RobustScaler for outliers, quantile for non-parametric)
+- Create temporal features with proper lag handling: rolling windows, exponential smoothing, seasonal decomposition, time-since-event
+- Encode categorical variables with strategy matching cardinality (one-hot <10 categories, target encoding >50, embeddings for high-cardinality)
+- Generate interaction features guided by domain knowledge and feature importance: polynomial, ratio, difference, product features
+- Handle missing values explicitly with strategy documented: imputation (mean/median/mode), indicator variables, or model-based imputation
+- Validate feature importance using multiple methods: permutation importance, SHAP values, and univariate tests to identify top contributors
+
+### After implementation
+- Create feature documentation with schema definitions, transformation logic, expected ranges, and update frequency
+- Build feature monitoring dashboards tracking distribution drift, missing rate changes, and correlation stability over time
+- Generate feature store integration with versioning, metadata tracking, and point-in-time correctness for temporal joins
+- Validate feature pipeline performance: transformation latency, memory usage, and batch vs online serving consistency
+
+## Self-check before task completion
+- [ ] All features are computed within transformation pipelines to prevent train-test leakage
+- [ ] Feature importance analysis identifies top 20 contributors with interpretable business meaning
+- [ ] Temporal features respect time boundaries and use only historically available information
+- [ ] Feature documentation includes transformation logic, expected distributions, and monitoring thresholds
+- [ ] Feature validation tests confirm stability across different time periods and data segments

package/.mindforge/skills/feature-flag-management/SKILL.md

@@ -0,0 +1,183 @@
+---
+name: feature-flag-management
+version: 1.0.0
+min_mindforge_version: 0.1.0
+status: stable
+triggers: feature flag, flag lifecycle, targeting rules, percentage rollout, kill switch, flag debt, openfeature, flag evaluation, flag cleanup, flag dependency, experiment flag, operational flag
+---
+
+# Skill — Feature Flag Management
+
+## When this skill activates
+Any task involving feature flag implementation, progressive rollouts, kill
+switches, flag lifecycle management, or flag debt cleanup.
+
+## Mandatory actions when this skill is active
+
+### Before writing any code
+1. Classify the flag type (release, experiment, operational, permission).
+2. Define the flag lifecycle including planned cleanup date.
+3. Determine targeting rules (who sees what).
+
+### During implementation
+- Set expiration date on every flag (max 90 days for release flags).
+- Implement clean fallback behavior for both flag states.
+- Use server-side evaluation for security-sensitive flags.
+
+### After implementation
+- Register flag in the flag inventory with owner and cleanup date.
+- Add monitoring for flag evaluation metrics.
+- Schedule cleanup task for after 100% rollout.
+
+## Flag Types
+
+### Release Flags
+- Purpose: ship code behind a flag, enable when ready.
+- Lifecycle: short-lived (days to weeks).
+- Example: `enable_new_checkout_flow`
+- Cleanup: remove after 100% rollout is stable (max 90 days).
+
+### Experiment Flags
+- Purpose: A/B testing with controlled variants.
+- Lifecycle: duration of experiment (2-8 weeks).
+- Example: `experiment_pricing_page_v2`
+- Cleanup: remove after experiment concludes and winner is chosen.
+
+### Operational Flags (Kill Switches)
+- Purpose: disable features under load or during incidents.
+- Lifecycle: permanent (always present in codebase).
+- Example: `kill_switch_recommendations_engine`
+- Cleanup: never — these are infrastructure.
+
+### Permission Flags (Entitlements)
+- Purpose: gate features by user plan/tier.
+- Lifecycle: permanent (tied to business model).
+- Example: `entitlement_advanced_analytics`
+- Cleanup: only when business model changes.
+
+## Flag Lifecycle
+
+```
+CREATE → DEVELOP → TEST → ROLLOUT → STABLE → CLEANUP
+```
+
+### Create
+- Register flag with name, type, owner, description, planned cleanup date.
+- Default state: OFF (new flags start disabled).
+
+### Develop
+- Code both paths (flag ON and flag OFF).
+- Both paths must be tested and functional.
+
+### Test
+- Enable in staging/QA environments.
+- Test both variants thoroughly.
+- Verify fallback behavior when flag service is unavailable.
+
+### Rollout
+- Progressive: 1% → 5% → 25% → 50% → 100%.
+- Monitor error rates and performance at each stage.
+- Hold at each percentage for minimum observation period.
+
+### Stable
+- Flag at 100% for > 7 days with no issues.
+- Old code path is dead code.
+
+### Cleanup
+- Remove flag evaluation from code.
+- Remove dead code path (the OFF variant).
+- Delete flag from management system.
+- Remove from flag inventory.
+
+## Targeting Rules
+
+### User Attributes
+```json
+{
+  "flag": "new_dashboard",
+  "rules": [
+    {"attribute": "plan", "operator": "in", "values": ["enterprise", "pro"]},
+    {"attribute": "country", "operator": "equals", "value": "US"}
+  ]
+}
+```
+
+### Segments
+- Define reusable user groups: "beta_users", "internal_team", "enterprise_customers".
+- Target flags at segments rather than individual attributes.
+- Segments update independently of flag rules.
+
+### Percentage Rollout
+- Hash user_id to determine bucket (consistent — same user always in same bucket).
+- Increase percentage gradually: 1% → 5% → 25% → 50% → 100%.
+- Sticky: user in 5% group stays in 25% group (no flip-flopping).
+
+### Geographic Targeting
+- Enable by region/country for compliance or staged global rollout.
+- Example: launch in US first, then EU, then APAC.
+
+## Flag Evaluation
+
+### Server-Side (Preferred for Security)
+- Flag evaluated on server, client receives result.
+- Source code and flag rules not exposed to client.
+- Use for: payment features, access control, security-sensitive logic.
+
+### Client-Side (Better Performance)
+- Flag rules sent to client, evaluated locally.
+- Faster (no network round-trip for each evaluation).
+- Use for: UI changes, non-sensitive features.
+- Risk: rules visible in client bundle.
+
+### Default Values
+- Always provide a sensible default if flag service is unreachable.
+- Release flags default to OFF (safe — old behavior).
+- Kill switches default to ON (safe — feature stays enabled).
+
+## Flag Debt Management
+
+### Indicators of Flag Debt
+- Flags older than 90 days still in codebase.
+- Flags at 100% rollout not cleaned up.
+- Nested flag dependencies (flag A depends on flag B).
+- Dead code paths that will never execute.
+
+### Prevention
+- Automated reminders: alert owner when flag exceeds planned cleanup date.
+- Flag inventory review: monthly audit of all active flags.
+- CI check: warn on PR if modifying code with expired flags.
+- Metric: track total active flag count as a health metric.
+
+### Cleanup Process
+1. Verify flag is at 100% and stable for > 7 days.
+2. Remove flag evaluation from code.
+3. Remove dead code path.
+4. Delete flag from management system.
+5. Update tests (remove flag-variant test cases).
+
+## OpenFeature (Vendor-Neutral Standard)
+
+### Why OpenFeature
+- Standard SDK interface regardless of flag provider.
+- Switch providers (LaunchDarkly, Flagsmith, custom) without code changes.
+- Community-driven specification with official SDKs.
+
+### Usage Pattern
+```typescript
+import { OpenFeature } from '@openfeature/js-sdk';
+
+const client = OpenFeature.getClient();
+const showNewUI = await client.getBooleanValue('new_ui', false, context);
+```
+
+## Self-check before task completion
+
+Before marking a task done when this skill was active:
+
+- [ ] Did I read the full SKILL.md before starting? (Not just the triggers)
+- [ ] Does every flag have a type classification?
+- [ ] Does every release/experiment flag have a cleanup date?
+- [ ] Are both code paths (ON and OFF) tested?
+- [ ] Is there a sensible default for flag service unavailability?
+- [ ] Is the flag registered in the inventory with owner?
+- [ ] Is server-side evaluation used for security-sensitive flags?