mindforge-cc 10.0.2 → 10.7.0

package/.mindforge/skills/infrastructure-as-code/SKILL.md

@@ -0,0 +1,98 @@
+---
+name: infrastructure-as-code
+version: 1.0.0
+min_mindforge_version: 10.0.7
+status: stable
+triggers: infrastructure as code, terraform pattern, pulumi pattern, state management iac, module design iac, drift detection, plan apply workflow, remote backend, workspace isolation, iac best practice, declarative infrastructure, resource provisioning pattern
+---
+
+# Infrastructure as Code
+
+## When this skill activates
+
+This skill activates when the user is designing, implementing, or troubleshooting
+infrastructure-as-code patterns. This includes Terraform/OpenTofu module design,
+Pulumi program structure, state management strategy, drift detection workflows,
+plan/apply pipelines, workspace isolation for multi-environment deployments, and
+general IaC best practices for declarative infrastructure provisioning.
+
+## Mandatory actions
+
+### Before
+
+1. Identify the IaC tool in use (Terraform, OpenTofu, Pulumi, CloudFormation, Bicep).
+2. Determine the target cloud provider(s) and existing state backend configuration.
+3. Assess current module structure and versioning strategy.
+4. Check for existing CI/CD pipeline integration (plan on PR, apply on merge).
+5. Identify secrets management approach (Vault, SOPS, AWS Secrets Manager).
+
+### During
+
+**Declarative over Imperative:**
+- Always prefer declarative resource definitions over procedural scripts.
+- Express desired end-state; let the provider handle ordering and dependencies.
+- Use `depends_on` only when implicit dependency detection fails.
+
+**State Management:**
+- Remote backends are mandatory for team environments (S3+DynamoDB locking, Terraform Cloud, GCS+locking).
+- Never commit `.tfstate` files to version control.
+- Enable state encryption at rest.
+- Use state locking to prevent concurrent modifications.
+- Implement state backup/versioning via backend configuration.
+
+**Module Design:**
+- Single responsibility: one module = one logical resource group.
+- Version all modules with semantic versioning (pin in consumers).
+- Define clear input/output contracts via `variables.tf` and `outputs.tf`.
+- Use composition (modules calling modules) over monolithic configurations.
+- Document module interfaces with descriptions on every variable and output.
+
+**Plan/Apply Workflow:**
+- ALWAYS run `plan` first and review the diff before applying.
+- Automate plan output on pull requests (comment the diff).
+- Require human approval gate before `apply` in production.
+- Use `-target` sparingly — it creates state drift risk.
+- Save plan files (`-out=plan.tfplan`) for deterministic applies.
+
+**Drift Detection:**
+- Schedule periodic plan-only runs to detect configuration drift.
+- Alert on any detected drift (resources modified outside IaC).
+- Reconcile drift by either importing changes or reverting manual modifications.
+- Use `terraform refresh` cautiously — it updates state but not code.
+
+**Workspace Isolation:**
+- Separate state files per environment (dev/staging/prod).
+- Use Terraform workspaces OR separate root modules per environment.
+- Prefer separate root modules for production isolation (stronger blast radius containment).
+- Environment-specific variables via `.tfvars` files or workspace-aware variable lookups.
+
+**Secrets Handling:**
+- NEVER store secrets in state files or variable defaults.
+- Use data sources to fetch secrets from Vault/SOPS at plan time.
+- Mark sensitive outputs with `sensitive = true`.
+- Encrypt state backend at rest and in transit.
+
+**Testing:**
+- Use Terratest or `terraform validate` + `terraform plan` in CI.
+- Write integration tests that provision real infrastructure in ephemeral accounts.
+- Validate plan output against policy (OPA/Sentinel/Conftest).
+- Test module interfaces with example configurations.
+
+### After
+
+1. Verify `terraform plan` shows expected changes (no surprises).
+2. Confirm state backend is accessible and lock is released.
+3. Validate outputs match expected resource identifiers.
+4. Run compliance/policy checks against the final plan.
+5. Document any manual steps required outside IaC scope.
+
+## Self-check before task completion
+
+- [ ] All infrastructure is defined declaratively (no imperative scripts for provisioning).
+- [ ] State is stored remotely with locking and encryption enabled.
+- [ ] Modules follow single responsibility and are versioned.
+- [ ] Plan/apply workflow is enforced (no direct applies without review).
+- [ ] Secrets are not hardcoded in configurations or state.
+- [ ] Drift detection mechanism is in place or recommended.
+- [ ] Workspace isolation separates environments with independent state.
+- [ ] Testing strategy covers plan validation and policy compliance.

package/.mindforge/skills/instinct-clustering/SKILL.md

@@ -0,0 +1,190 @@
+---
+name: instinct-clustering
+version: 1.0.0
+min_mindforge_version: 10.0.5
+status: stable
+triggers: instinct cluster, cluster instincts, group instincts, batch promote, instinct groups, pattern clusters, behavior clusters, auto-group, merge instincts, combine instincts, related patterns, instinct merge
+---
+
+# Skill — Instinct Clustering (Auto-Group & Batch Promote Instincts)
+
+## When this skill activates
+
+When the instinct store has accumulated enough patterns that manual promotion to
+skills becomes impractical. Use to automatically identify groups of related instincts,
+deduplicate redundant entries, and batch-promote coherent clusters into full skills.
+Converts organic behavioral patterns into structured, reusable skill definitions.
+
+Core principle: **Emergent structure** — let clusters form from the data rather than
+imposing categories top-down. Instincts that repeatedly co-occur reveal natural
+skill boundaries.
+
+## Mandatory actions when this skill is active
+
+### Before clustering begins
+
+1. **Load and filter instinct store:**
+   - Read `instinct-store.jsonl` (or configured instinct storage path)
+   - Filter to active instincts only (`status == "active"`)
+   - Exclude instincts with `confidence < 0.3` (too uncertain to cluster)
+   - Record total count: `N active instincts loaded`
+
+2. **Validate minimum viable dataset:**
+   - Minimum 10 active instincts required for meaningful clustering
+   - If < 10: report "insufficient instincts for clustering" and exit
+   - If 10-20: expect 1-3 clusters
+   - If 20-50: expect 3-8 clusters
+   - If 50+: expect 5-15 clusters
+
+3. **Understand instinct schema:**
+   ```json
+   {
+     "id": "inst-uuid",
+     "observation": "what was observed (natural language)",
+     "behavior": "what action to take (imperative)",
+     "tags": ["tag1", "tag2", "tag3"],
+     "confidence": 0.0-1.0,
+     "frequency": 0,
+     "project_scope": "global" | "project-name",
+     "status": "active" | "promoted" | "deprecated",
+     "created_at": "ISO-8601",
+     "last_triggered": "ISO-8601"
+   }
+   ```
+
+### During clustering
+
+**Step 1 — Deduplication pass:**
+- For each pair of instincts, compute observation word overlap:
+  - Tokenize observations (lowercase, remove stop words)
+  - Overlap = |intersection| / |union| (jaccard on word sets)
+- If overlap > 0.8: merge the pair (keep the one with higher confidence)
+- Log all merges: `"Merged inst-X into inst-Y (overlap: Z%)""`
+- Record: `D duplicates removed, M instincts remaining`
+
+**Step 2 — Tag overlap calculation:**
+- For each remaining pair of instincts, compute tag jaccard similarity:
+  ```
+  jaccard(A, B) = |A.tags intersection B.tags| / |A.tags union B.tags|
+  ```
+- Build a similarity matrix: instincts (rows) x instincts (columns)
+- Store as adjacency list for efficiency (only pairs with jaccard > 0.3)
+
+**Step 3 — Cluster formation:**
+- Group instincts where:
+  - Tag jaccard > 0.5 (strong tag overlap)
+  - AND shared `project_scope` (both global, or both same project)
+- Use single-linkage clustering: if A is similar to B and B is similar to C,
+  then {A, B, C} form a cluster (even if A and C are not directly similar)
+- Cap cluster size at 10 instincts (larger clusters should be split)
+
+**Step 4 — Filter clusters:**
+- Minimum cluster size: 3 instincts (smaller groups are not worth promoting)
+- Average confidence across cluster members must be > 0.7
+- At least 2 distinct observations in the cluster (not all near-duplicates)
+- Discard clusters that fail any filter
+
+**Step 5 — Skill generation per cluster:**
+- **Name**: derived from the 2-3 most common tags across cluster members
+- **Triggers**: extracted from observation keywords (deduplicated, natural phrasing)
+- **Behavior**: combine all instinct `behavior` fields in logical order:
+  1. Sort by frequency (most-triggered instincts first)
+  2. Remove redundant steps
+  3. Organize into Before/During/After structure
+- **Self-check**: synthesize from the cluster's collective edge cases
+
+Generated skill structure:
+```markdown
+---
+name: [derived-name]
+version: 1.0.0
+min_mindforge_version: 10.0.5
+status: stable
+triggers: [combined trigger keywords]
+---
+
+# Skill — [Name]
+
+## When this skill activates
+[Synthesized from cluster observations]
+
+## Mandatory actions when this skill is active
+### Before [derived from behaviors]
+### During [derived from behaviors]
+### After [derived from behaviors]
+
+## Self-check before task completion
+[Derived from cluster edge cases]
+```
+
+**Step 6 — Conflict detection:**
+- Check generated triggers against MANIFEST.md
+- If jaccard > 0.3 with existing skill triggers: flag conflict
+- Resolution options: merge into existing skill, rename triggers, or keep separate with justification
+
+### After clustering
+
+1. **Present clusters for user review:**
+   ```
+   ## Cluster Report
+
+   ### Cluster 1: [proposed-name]
+   - Instincts: [count]
+   - Avg confidence: [score]
+   - Members: [list of instinct IDs with one-line observations]
+   - Proposed skill name: [name]
+   - Proposed triggers: [list]
+
+   ### Cluster 2: ...
+
+   ## Unclustered Instincts
+   [Instincts that didn't fit any cluster — may need more data]
+   ```
+
+2. **On user approval:**
+   - Create SKILL.md in `.mindforge/skills/[name]/`
+   - Register in MANIFEST.md
+   - Mark all clustered instincts as `status: "promoted"`
+   - Add `promoted_to: "[skill-name]"` field to each promoted instinct
+   - Append promotion event to audit log
+
+3. **On user rejection or modification:**
+   - Apply requested changes to cluster membership or skill definition
+   - Re-validate filters (size >= 3, confidence > 0.7)
+   - Re-check for trigger conflicts
+   - Present revised version for approval
+
+4. **Generate CLUSTER-REPORT.md:**
+   ```markdown
+   ## Instinct Clustering Report — [date]
+
+   **Input:** N active instincts
+   **After dedup:** M instincts (D removed)
+   **Clusters formed:** C clusters
+   **Clusters passing filters:** F clusters
+   **Instincts promoted:** P
+   **Instincts unclustered:** U
+
+   ### Cluster Details
+   [table: name, size, avg_confidence, proposed_skill, status]
+
+   ### Deduplication Log
+   [list of merged instinct pairs with overlap scores]
+
+   ### Recommendations
+   - [instincts close to clustering threshold — collect more data]
+   - [potential trigger conflicts to monitor]
+   ```
+
+## Self-check before task completion
+
+Before marking an instinct clustering task done:
+
+- [ ] Did I deduplicate before clustering (overlap > 0.8 merged)?
+- [ ] Did I verify minimum cluster size (3+ instincts per cluster)?
+- [ ] Did I check average confidence > 0.7 for each cluster?
+- [ ] Did I combine behaviors coherently (sorted by frequency, no redundancy)?
+- [ ] Did I check for trigger conflicts against MANIFEST.md?
+- [ ] Did I present clusters to the user for approval before creating skills?
+- [ ] Did I mark promoted instincts with `status: "promoted"`?
+- [ ] Did I generate CLUSTER-REPORT.md with full statistics?

package/.mindforge/skills/internal-developer-platform/SKILL.md

@@ -0,0 +1,51 @@
+---
+name: internal-developer-platform
+version: 1.0.0
+min_mindforge_version: 10.7.0
+status: stable
+triggers: internal developer platform, golden path design, developer portal architecture, service catalog platform, platform engineering, IDP design, backstage implementation, developer self-service, platform team scope, developer experience platform, service template, platform API design
+---
+
+# Skill — Internal Developer Platform
+
+## When this skill activates
+
+This skill activates when the user is designing, building, or evolving an Internal Developer Platform (IDP). This includes defining golden paths, building developer portals, creating service catalogs, implementing platform engineering practices, defining platform team scope, building self-service capabilities, and designing platform APIs for developer consumption.
+
+## Mandatory actions when this skill is active
+
+### Before writing any code
+
+1. Map the current developer journey from idea to production, identifying friction points and toil hotspots.
+2. Define the platform boundary: what belongs in the platform vs what belongs in application teams.
+3. Identify the personas using the platform (backend devs, frontend devs, data engineers, SREs) and their distinct needs.
+4. Assess existing tooling sprawl and identify consolidation opportunities (reduce tool count by 40-60% is typical).
+5. Establish platform success metrics (time-to-first-deploy, cognitive load score, toil hours saved, adoption rate).
+
+### During implementation
+
+- **Golden Paths:** Create opinionated, paved roads for common tasks (new service, new API, new data pipeline). Each golden path should reduce a multi-day task to under 30 minutes. Include defaults that work for 80% of cases and escape hatches for the 20%.
+- **Service Catalog:** Maintain a living registry of all services, APIs, data stores, and infrastructure with ownership, dependencies, and health status. Use Backstage, OpsLevel, or ServiceCatalog.io. Each entry must have: owner, on-call, docs, runbooks, and SLOs.
+- **Developer Portal:** Single entry point for all platform capabilities. Include: service catalog, golden paths, documentation, status page, internal API marketplace, cost dashboard. Portal must be searchable and load in under 2 seconds.
+- **Self-Service Capabilities:** Enable developers to provision environments, request resources, deploy services, configure monitoring, and manage secrets without filing tickets. Each self-service action should complete in under 5 minutes.
+- **Platform APIs:** Design platform APIs as products. Use OpenAPI specs, provide SDKs, maintain changelogs, version properly. Platform APIs should have 99.9% uptime SLOs (higher than most application services).
+- **Avoid Paving Cowpaths:** Don't automate broken processes. Simplify first, then automate. If a process requires 12 approval steps, fix the process before building the platform.
+- **Platform as a Product:** Treat internal developers as customers. Collect feedback, measure satisfaction (NPS or CSAT), prioritize features by impact. Platform teams should spend 20% of time on user research.
+
+### After implementation
+
+- Verify each golden path reduces task time by at least 70% compared to manual execution.
+- Confirm the service catalog is auto-populated and stays in sync with production reality (via discovery, not manual updates).
+- Validate that the developer portal has search functionality and surfaces the most-used capabilities within 2 clicks.
+- Ensure platform APIs have monitoring, alerting, and SLOs defined and tracked.
+- Check that self-service actions have audit logs and cost attribution.
+
+## Self-check before task completion
+
+- [ ] Golden paths exist for the top 5 most common developer tasks.
+- [ ] Service catalog includes all production services with ownership and dependencies.
+- [ ] Developer portal is the single source of truth and loads in under 2 seconds.
+- [ ] Platform success metrics are defined, instrumented, and tracked.
+- [ ] Platform APIs have OpenAPI specs, SDKs, and 99.9% uptime SLOs.
+- [ ] Self-service capabilities complete in under 5 minutes and include audit logs.
+- [ ] Platform team spends 20% of time on user research and feedback collection.

package/.mindforge/skills/iot-platform/SKILL.md

@@ -0,0 +1,41 @@
+---
+name: iot-platform
+version: 1.0.0
+min_mindforge_version: 10.2.0
+status: stable
+triggers: IoT platform architecture, device management system, telemetry ingestion pipeline, firmware OTA update, device twin, IoT edge gateway, sensor data processing, IoT device provisioning, industrial IoT, MQTT broker design, device lifecycle management, IoT data pipeline
+compose: edge-computing
+---
+
+# Skill — IoT Platform
+
+## When this skill activates
+This skill activates when building IoT device management systems, telemetry ingestion pipelines, firmware over-the-air (OTA) update mechanisms, device twin/shadow architectures, edge-to-cloud data flows, MQTT brokers, device provisioning, or industrial IoT platforms.
+
+## Mandatory actions when this skill is active
+
+### Before writing any code
+1. Design device lifecycle management: provisioning (certificate issuance, device registration, initial config push) → active (telemetry streaming, command execution) → maintenance (firmware updates, config changes) → decommissioning (certificate revocation, data archival), with state machine transitions and audit trails
+2. Model telemetry ingestion architecture: devices publish MQTT messages (QoS 1 for critical, QoS 0 for high-frequency) → broker buffers (HiveMQ, AWS IoT Core) → stream processor (Kafka, Kinesis) → time-series database (InfluxDB, TimescaleDB) → analytics/alerting, with backpressure handling and dead letter queues
+3. Map device twin/shadow pattern: desired state (cloud wants device at firmware v2.1, config A) vs reported state (device currently at v2.0, config B) → delta detection → command sent to device → device applies change → reports new state → convergence detected
+
+### During implementation
+- Implement MQTT broker with scalability: support 100K+ concurrent connections, message throughput (10K msg/sec), QoS 0/1/2 support (at-most-once, at-least-once, exactly-once), retained messages (last known state), last will and testament (detect disconnections), topic ACLs (device can only publish to devices/{device_id}/telemetry)
+- Build device provisioning flow: device boots with factory cert → requests device-specific cert from provisioning service (using CSR) → service validates manufacturing batch, issues cert signed by CA, registers device in registry → device stores cert in secure element (TPM, TEE) → uses cert for future MQTT authentication
+- Design firmware OTA update with safety: cloud creates firmware campaign (target devices by tag, rollout percentage), device polls for updates (or push notification), downloads firmware chunk-by-chunk (resumable, checksum verified), stores in secondary partition, validates signature (RSA/ECDSA), swaps partitions on next boot, rollback on boot failure (watchdog timer, 3 attempts)
+- Implement device twin with conflict resolution: cloud updates desired state (twin document with version), device polls for changes, detects delta, applies change, reports new state, cloud reconciles (last-write-wins or version vector), handle offline devices (queue commands, apply on reconnect up to 7 days)
+- Build telemetry pipeline with windowing: ingest raw sensor data (temperature, pressure, vibration at 1 Hz) → stream processor aggregates (1-minute windows: avg, min, max, stddev) → anomaly detection (Z-score >3 triggers alert) → store aggregates (reduce storage 60x), retain raw data for 7 days (hot storage), archive aggregates for 5 years (cold storage)
+
+### After implementation
+- Validate device connectivity resilience: simulate network failures (disconnect device mid-message), verify QoS 1 retries (message delivered after reconnect), test persistent sessions (broker retains subscriptions), measure reconnect time (<5s for 95th percentile), verify backoff exponential retry (1s, 2s, 4s, 8s, max 60s)
+- Test firmware OTA reliability: simulate power loss during download (resume from last chunk), corrupt firmware signature (device rejects), partial update failure (automatic rollback to previous version), measure update success rate (>99% for targeted devices), track update latency (time from campaign start to device running new firmware)
+- Execute telemetry load testing: simulate 10K devices sending data at 1 Hz (10K msg/sec), measure broker latency (p99 <100ms), stream processor lag (<1s behind real-time), database write throughput (handle burst traffic 10x normal), verify no message loss (QoS 1 delivery confirmed)
+
+## Self-check before task completion
+- [ ] Device lifecycle managed: provisioning (cert issuance, registration), active (telemetry/commands), maintenance (OTA updates), decommissioning (cert revocation)
+- [ ] MQTT broker scalable: 100K+ concurrent connections, QoS 0/1/2 support, topic ACLs, retained messages, LWT for disconnect detection
+- [ ] Device provisioning secure: factory cert → CSR → device-specific cert (signed by CA), stored in secure element (TPM/TEE), cert-based MQTT auth
+- [ ] Firmware OTA robust: resumable downloads (chunk-by-chunk), signature validation (RSA/ECDSA), dual-partition swap, automatic rollback on failure
+- [ ] Device twin functional: desired vs reported state, delta detection, command queuing for offline devices (7 day retention), conflict resolution
+- [ ] Telemetry pipeline optimized: windowed aggregation (1-min windows), anomaly detection (Z-score >3), hot/cold storage tiering (7 days raw, 5 years aggregates)
+- [ ] Connectivity resilient: QoS 1 retries, persistent sessions, exponential backoff (1s to 60s), reconnect time <5s for 95th percentile