mindforge-cc 10.0.3 → 11.0.0

package/.mindforge/skills/k8s-deployment/SKILL.md

@@ -0,0 +1,358 @@
+---
+name: k8s-deployment
+version: 1.0.0
+min_mindforge_version: 10.0.6
+status: stable
+triggers: kubernetes deployment, helm chart, rolling update, HPA autoscaling, pod disruption budget, network policy, resource quota, liveness probe, readiness probe, ingress controller, k8s manifest, container orchestration
+---
+
+# Skill — Kubernetes Deployment
+
+## When this skill activates
+Any task involving Kubernetes deployments: writing or modifying manifests, Helm charts,
+configuring autoscaling, probes, network policies, ingress, resource management,
+or deployment strategies for containerized workloads.
+
+## Mandatory actions when this skill is active
+
+### Before writing any code
+1. Identify the deployment requirements:
+   - Target environment (dev/staging/production)
+   - Availability requirements (SLA target, max acceptable downtime)
+   - Scale expectations (baseline replicas, peak load multiplier)
+   - Network exposure (internal only, public ingress, specific CIDR allowlists)
+2. Check existing cluster state:
+   ```bash
+   kubectl get nodes -o wide          # Cluster capacity
+   kubectl top nodes                   # Current resource usage
+   kubectl get namespaces              # Available namespaces
+   kubectl get resourcequotas -A       # Existing quotas
+   ```
+3. Determine if Helm or raw manifests are appropriate:
+   - **Helm**: Multiple environments, parameterized configs, community chart availability.
+   - **Raw manifests + Kustomize**: Simpler apps, GitOps with ArgoCD/Flux, overlay-based config.
+
+### During implementation
+
+#### Deployment Strategies
+- **RollingUpdate** (default, recommended for most services):
+  ```yaml
+  apiVersion: apps/v1
+  kind: Deployment
+  metadata:
+    name: my-service
+  spec:
+    replicas: 3
+    strategy:
+      type: RollingUpdate
+      rollingUpdate:
+        maxSurge: 1          # Max extra pods during update
+        maxUnavailable: 0    # Zero downtime: never kill before new is ready
+    selector:
+      matchLabels:
+        app: my-service
+    template:
+      metadata:
+        labels:
+          app: my-service
+      spec:
+        containers:
+        - name: my-service
+          image: registry.example.com/my-service:v1.2.3
+          # ... rest of spec
+  ```
+- **Recreate** (only for stateful apps that cannot run two versions simultaneously):
+  ```yaml
+  strategy:
+    type: Recreate
+  ```
+- **Blue/Green** (via service selector swap or Argo Rollouts):
+  Deploy new version as separate deployment, switch service selector when healthy.
+- **Canary** (via Argo Rollouts or Istio traffic splitting):
+  Route percentage of traffic to new version, promote or rollback based on metrics.
+
+#### Helm Chart Structure
+```
+my-chart/
+  Chart.yaml          # Chart metadata (name, version, appVersion)
+  values.yaml         # Default configuration values
+  values-staging.yaml # Environment-specific overrides
+  values-prod.yaml
+  templates/
+    _helpers.tpl      # Template helpers and labels
+    deployment.yaml
+    service.yaml
+    hpa.yaml
+    ingress.yaml
+    configmap.yaml
+    secret.yaml       # Reference to external secrets, not raw values
+    pdb.yaml
+    networkpolicy.yaml
+    serviceaccount.yaml
+  tests/
+    test-connection.yaml
+```
+
+- **Chart.yaml** must include:
+  ```yaml
+  apiVersion: v2
+  name: my-service
+  version: 1.0.0       # Chart version (bump on chart changes)
+  appVersion: "1.2.3"  # Application version (matches container tag)
+  ```
+- **values.yaml** conventions:
+  ```yaml
+  replicaCount: 3
+  image:
+    repository: registry.example.com/my-service
+    tag: ""            # Overridden per environment
+    pullPolicy: IfNotPresent
+  resources:
+    requests:
+      cpu: 100m
+      memory: 128Mi
+    limits:
+      cpu: 500m
+      memory: 512Mi
+  ```
+
+#### HPA (Horizontal Pod Autoscaler)
+```yaml
+apiVersion: autoscaling/v2
+kind: HorizontalPodAutoscaler
+metadata:
+  name: my-service
+spec:
+  scaleTargetRef:
+    apiVersion: apps/v1
+    kind: Deployment
+    name: my-service
+  minReplicas: 3         # Never go below 3 for production
+  maxReplicas: 20        # Cap to prevent runaway scaling
+  metrics:
+  - type: Resource
+    resource:
+      name: cpu
+      target:
+        type: Utilization
+        averageUtilization: 70    # Scale up at 70% CPU
+  - type: Resource
+    resource:
+      name: memory
+      target:
+        type: Utilization
+        averageUtilization: 80
+  behavior:
+    scaleDown:
+      stabilizationWindowSeconds: 300   # Wait 5min before scaling down
+      policies:
+      - type: Percent
+        value: 25
+        periodSeconds: 60               # Max 25% reduction per minute
+    scaleUp:
+      stabilizationWindowSeconds: 30
+      policies:
+      - type: Percent
+        value: 100
+        periodSeconds: 30               # Can double pods in 30s
+```
+
+- Custom metrics (requests per second, queue depth) via Prometheus adapter:
+  ```yaml
+  - type: Pods
+    pods:
+      metric:
+        name: http_requests_per_second
+      target:
+        type: AverageValue
+        averageValue: "1000"
+  ```
+
+#### Pod Disruption Budget (PDB)
+```yaml
+apiVersion: policy/v1
+kind: PodDisruptionBudget
+metadata:
+  name: my-service-pdb
+spec:
+  minAvailable: 2        # Always keep at least 2 pods running
+  # OR: maxUnavailable: 1  # At most 1 pod can be down
+  selector:
+    matchLabels:
+      app: my-service
+```
+- **Always create a PDB for production workloads.** Without one, cluster upgrades and
+  node drains can take down all pods simultaneously.
+- Rule of thumb: `minAvailable` = `replicas - 1` or use `maxUnavailable: 1`.
+
+#### Probes (Health Checks)
+```yaml
+containers:
+- name: my-service
+  livenessProbe:           # Is the process alive? Restart if failing.
+    httpGet:
+      path: /healthz
+      port: 8080
+    initialDelaySeconds: 15
+    periodSeconds: 10
+    failureThreshold: 3    # 3 failures = restart
+    timeoutSeconds: 3
+  readinessProbe:          # Is it ready for traffic? Remove from LB if failing.
+    httpGet:
+      path: /readyz
+      port: 8080
+    initialDelaySeconds: 5
+    periodSeconds: 5
+    failureThreshold: 2
+    timeoutSeconds: 3
+  startupProbe:            # For slow starters. Disables liveness/readiness until passing.
+    httpGet:
+      path: /healthz
+      port: 8080
+    initialDelaySeconds: 0
+    periodSeconds: 5
+    failureThreshold: 30   # 30 * 5s = 150s max startup time
+    timeoutSeconds: 3
+```
+- **liveness**: "Is the process stuck?" Triggers container restart. Keep simple (not DB-dependent).
+- **readiness**: "Can it serve requests?" Controls load balancer membership. Can check dependencies.
+- **startup**: Use for apps that take > 30s to initialize (JVM warmup, large model loading).
+
+#### Resource Requests and Limits
+```yaml
+resources:
+  requests:              # Scheduling guarantee (must be available)
+    cpu: 100m           # 0.1 CPU cores
+    memory: 256Mi       # 256 MiB RAM
+  limits:               # Hard ceiling (OOMKilled if exceeded for memory)
+    cpu: 1000m          # 1 CPU core (throttled, not killed)
+    memory: 512Mi       # OOMKilled if exceeded
+```
+- **Requests**: Set to observed p50 usage. Cluster scheduler uses this for placement.
+- **Limits**: Set to observed p99 + 20% headroom. Too tight = OOMKills. Too loose = noisy neighbors.
+- **CPU limits debate**: Some teams remove CPU limits (use only requests) to avoid throttling.
+  This is acceptable if the cluster has sufficient headroom and resource quotas protect namespaces.
+- Always set memory limits (OOM without limits can crash the node).
+
+#### Network Policies
+```yaml
+apiVersion: networking.k8s.io/v1
+kind: NetworkPolicy
+metadata:
+  name: default-deny-all
+  namespace: my-namespace
+spec:
+  podSelector: {}          # Applies to all pods in namespace
+  policyTypes:
+  - Ingress
+  - Egress
+---
+apiVersion: networking.k8s.io/v1
+kind: NetworkPolicy
+metadata:
+  name: allow-my-service-ingress
+  namespace: my-namespace
+spec:
+  podSelector:
+    matchLabels:
+      app: my-service
+  policyTypes:
+  - Ingress
+  ingress:
+  - from:
+    - namespaceSelector:
+        matchLabels:
+          name: ingress-nginx
+    ports:
+    - protocol: TCP
+      port: 8080
+```
+- **Default deny first**, then allow specific traffic paths.
+- Minimum for production: deny all ingress/egress, then whitelist:
+  1. Ingress controller to service pods.
+  2. Service pods to database pods.
+  3. Egress to external APIs (specific IPs/CIDRs if possible).
+
+#### Ingress Configuration
+```yaml
+apiVersion: networking.k8s.io/v1
+kind: Ingress
+metadata:
+  name: my-service-ingress
+  annotations:
+    cert-manager.io/cluster-issuer: letsencrypt-prod
+    nginx.ingress.kubernetes.io/rate-limit: "100"
+    nginx.ingress.kubernetes.io/rate-limit-window: "1m"
+spec:
+  ingressClassName: nginx
+  tls:
+  - hosts:
+    - api.example.com
+    secretName: api-tls-cert
+  rules:
+  - host: api.example.com
+    http:
+      paths:
+      - path: /
+        pathType: Prefix
+        backend:
+          service:
+            name: my-service
+            port:
+              number: 80
+```
+- Always configure TLS termination (use cert-manager for automatic certificate management).
+- Add rate limiting annotations to prevent abuse.
+- Use path-based routing to split traffic to different services under one domain.
+
+### After implementation
+1. Validate manifests before applying:
+   ```bash
+   helm template my-chart ./chart -f values-prod.yaml | kubectl apply --dry-run=client -f -
+   kubectl diff -f manifest.yaml    # Show what would change
+   ```
+2. Verify rollout health:
+   ```bash
+   kubectl rollout status deployment/my-service --timeout=300s
+   kubectl get pods -l app=my-service -o wide
+   kubectl top pods -l app=my-service
+   ```
+3. Test probes manually:
+   ```bash
+   kubectl exec -it <pod> -- curl -s localhost:8080/healthz
+   kubectl exec -it <pod> -- curl -s localhost:8080/readyz
+   ```
+4. Verify network policies:
+   ```bash
+   # From a test pod, confirm blocked traffic is actually blocked
+   kubectl run test --rm -it --image=busybox -- wget -qO- --timeout=3 http://my-service:8080
+   ```
+5. Test PDB during a drain:
+   ```bash
+   kubectl drain <node> --ignore-daemonsets --dry-run=client
+   ```
+
+## Common mistakes to flag
+
+- No PDB on production deployments (cluster upgrades will cause downtime).
+- Liveness probe checks database connectivity (cascading restarts on DB issues).
+- No resource limits (one pod can starve the entire node).
+- `latest` tag in production (non-reproducible deployments).
+- Secrets in ConfigMaps or values.yaml (use sealed-secrets, external-secrets, or vault).
+- No network policies (all pods can communicate with all other pods by default).
+- HPA and VPA both active on the same resource (they conflict).
+
+## Self-check before task completion
+
+Before marking a task done when this skill was active:
+
+- [ ] All manifests pass `kubectl apply --dry-run=client`.
+- [ ] Deployment has both readiness and liveness probes configured.
+- [ ] Resource requests and memory limits are set on all containers.
+- [ ] PDB exists for production deployments (minAvailable or maxUnavailable).
+- [ ] Network policies enforce least-privilege communication.
+- [ ] No secrets stored in plain text in manifests or values files.
+- [ ] Image tags are pinned to specific versions (not `latest`).
+- [ ] HPA configured with appropriate min/max and scale-down stabilization.
+- [ ] Ingress has TLS termination configured.
+- [ ] Rollout tested with `kubectl rollout status`.

package/.mindforge/skills/knowledge-graphs/SKILL.md

@@ -0,0 +1,56 @@
+---
+name: knowledge-graphs
+version: 1.0.0
+min_mindforge_version: 10.5.0
+status: stable
+triggers: knowledge graph design, ontology architecture, graph database modeling, entity resolution system, knowledge extraction pipeline, graph schema design, triple store, semantic relationship, knowledge base construction, graph traversal pattern, entity linking, knowledge graph embedding
+compose:
+---
+
+# Knowledge Graphs & Ontology Design
+
+## When this skill activates
+
+This skill activates when building knowledge graphs, designing ontologies, implementing entity resolution systems, extracting structured knowledge from unstructured text, or querying graph databases. It applies to any system that represents complex relationships between entities for reasoning, search, or recommendation.
+
+## Mandatory actions when this skill is active
+
+### Before writing any code
+
+1. **Define ontology structure** — Identify entity types (Person, Organization, Product, Event), relationship types (works_for, located_in, purchased, happened_at), and attributes (name, date, value). Document cardinality: one-to-one, one-to-many, many-to-many. Ontology is the schema for your knowledge graph.
+2. **Select graph database** — Choose based on scale (thousands vs. billions of nodes), query patterns (simple lookups vs. multi-hop traversals), and ecosystem: Neo4j (property graph, Cypher), Amazon Neptune (supports RDF and property graphs), RDFox (reasoning and inference), or embedded graphs (NetworkX for prototypes). Benchmark query performance on your schema.
+3. **Design entity resolution strategy** — Entities from multiple sources must be deduplicated (same person with different names, same product with different IDs). Define resolution rules: exact name match, fuzzy string matching (Levenshtein distance), embedding similarity, or human-in-the-loop confirmation. Test resolution accuracy on labeled data.
+4. **Establish schema versioning** — Ontologies evolve as domain understanding improves. Version the schema with semantic versioning (v1.0, v1.1). Define migration paths for schema changes (add new relationship types, rename attributes). Ensure backward compatibility or provide migration scripts.
+
+### During implementation
+
+- **Extract entities and relationships from text** — Use named entity recognition (NER) for entity extraction and relation extraction models for relationships. Validate that extraction recall is high (>80% of entities are found) and precision is acceptable (>70% of extracted entities are correct). Fine-tune models on domain-specific data.
+- **Implement entity linking** — Link extracted entities to canonical entities in the graph. Use fuzzy matching, embedding similarity, or knowledge base APIs (Wikidata, DBpedia). Handle ambiguity: "Apple" could be a fruit, a company, or a person's nickname. Disambiguate using context (surrounding words, known relationships).
+- **Design relationship inference rules** — Add implicit relationships via rules: if A works_for B and B is_part_of C, then A works_for C (transitive closure). Use graph query languages (Cypher, SPARQL) or inference engines (RDFox, Pellet). Validate that inferred relationships are logically correct.
+- **Normalize entity attributes** — Canonicalize names (lowercase, remove punctuation), dates (ISO 8601), and values (currency conversion). Inconsistent attributes break queries: "Microsoft" vs. "microsoft" vs. "Microsoft Corp." should resolve to the same entity.
+- **Implement graph traversal optimizations** — Avoid Cartesian product explosions in multi-hop queries. Use query hints (index usage, join order), limit traversal depth (max 3 hops for most use cases), and cache frequent subgraph patterns. Measure query latency and optimize slow queries.
+- **Version entities and relationships** — Track temporal validity: relationships have start and end dates (person worked at company from 2020 to 2023). Implement bitemporal modeling if you need to track both valid time (when the fact was true in the real world) and transaction time (when the fact was recorded in the database).
+
+### After implementation
+
+- **Validate graph completeness** — Measure coverage: % of entities from source data that are present in the graph, % of relationships that are captured. Incomplete graphs produce incorrect query results. Identify missing entities and relationships, then backfill.
+- **Test query correctness** — Create a test suite of queries with known ground-truth answers. Validate that queries return expected results. Common failure modes: missing relationships, incorrect cardinality, transitive closure errors.
+- **Measure query performance** — Benchmark query latency under realistic load. Target: simple lookups <10ms, multi-hop traversals <100ms. If slower, optimize indexes, limit traversal depth, or denormalize hot paths (precompute frequent traversals).
+- **Audit for duplicate entities** — Run entity resolution on the entire graph post-construction. Identify entities that should be merged (similar names, same attributes). Merge duplicates and redirect relationships to canonical entities.
+
+## Self-check before task completion
+
+- [ ] Ontology defines entity types, relationship types, attributes, and cardinality constraints
+- [ ] Graph database is selected and benchmarked on schema-specific query patterns
+- [ ] Entity resolution strategy is defined and tested on labeled data (precision/recall metrics)
+- [ ] Schema versioning is implemented with migration paths for schema changes
+- [ ] Entity extraction achieves >80% recall and >70% precision on domain data
+- [ ] Entity linking disambiguates entities using context and canonical knowledge bases
+- [ ] Relationship inference rules are implemented and validated for logical correctness
+- [ ] Entity attributes are normalized (names, dates, values) for consistent queries
+- [ ] Graph traversal queries are optimized with indexes, depth limits, and subgraph caching
+- [ ] Temporal validity is tracked with start/end dates for time-sensitive relationships
+- [ ] Graph completeness is measured (% entities and relationships captured from source data)
+- [ ] Query correctness is validated with ground-truth test suite
+- [ ] Query latency is benchmarked (simple lookups <10ms, multi-hop <100ms)
+- [ ] Duplicate entities are audited and merged with canonical entity resolution

package/.mindforge/skills/knowledge-sharing-systems/SKILL.md

@@ -0,0 +1,112 @@
+---
+name: knowledge-sharing-systems
+version: 1.0.0
+min_mindforge_version: 10.1.0
+status: stable
+triggers: knowledge sharing, documentation culture, brown bag session, tech talk, decision log, tribal knowledge, bus factor, knowledge base design, knowledge transfer system, institutional memory, expertise sharing, learning organization
+---
+
+# Knowledge Sharing Systems
+
+## When this skill activates
+
+This skill activates when designing, evaluating, or improving how knowledge flows
+within an engineering organization. It addresses bus factor reduction, documentation
+culture, decision logging, and systematic approaches to converting tribal knowledge
+into institutional memory.
+
+## Mandatory actions when this skill is active
+
+### Before
+
+1. **Assess current state** — Where does knowledge currently live? (People's heads,
+   Slack threads, outdated wikis, code comments, nowhere?) Map the gaps.
+2. **Identify bus factor risks** — Which components/systems have only 1-2 people who
+   understand them? These are critical vulnerabilities.
+3. **Understand failure modes** — What happens when the knowledge holder is unavailable?
+   How long does it take a new person to become productive in each area?
+
+### During
+
+4. **Categorize knowledge types and appropriate capture methods:**
+
+   - **Explicit knowledge** (can be written down):
+     - Architecture Decision Records (ADRs) for non-obvious choices
+     - API documentation generated from code
+     - Runbooks for operational procedures
+     - README files for project context and setup
+     - Design documents for complex features
+
+   - **Tacit knowledge** (hard to articulate, best transferred person-to-person):
+     - Pair programming sessions (regular rotation)
+     - Shadowing on-call rotations
+     - Code review as teaching (explain the WHY in review comments)
+     - Mob programming for complex problems
+     - Recorded debugging sessions showing thought process
+
+   - **Tribal knowledge** (exists only in collective memory, needs active capture):
+     - "Why does this system do X?" interviews with long-tenured engineers
+     - Archaeological code tours (walk through old systems, document context)
+     - Decision archaeology (reconstruct rationale for old choices)
+     - Oral history capture before people leave
+
+5. **Implement knowledge sharing systems:**
+
+   - **Decision logs** — Lightweight ADRs for all non-obvious choices. Template:
+     Context, Decision, Consequences, Status. Write at decision time, not after.
+     Low ceremony, high value.
+
+   - **Tech talks** — Bi-weekly internal presentations (30 min max). Rotate presenters.
+     Record all sessions. Topics: recent incidents, new technologies, deep dives
+     into system internals, lessons learned.
+
+   - **Onboarding paths** — Curated reading lists per domain/team. Progressive
+     complexity (week 1: overview, week 2: deep dive, week 3: contribute). Include
+     "who to ask" for each topic.
+
+   - **Brown bag sessions** — Informal lunch-and-learn. Low pressure, high exploration.
+     Can be external topics, book discussions, or show-and-tell of side projects.
+
+   - **Documentation sprints** — Dedicate one sprint per quarter to documentation
+     catch-up. Treat docs as first-class deliverables, not afterthoughts.
+
+6. **Bus factor reduction strategies:**
+   - Pair programming rotation — No one works alone on critical systems for >2 weeks.
+   - Cross-team shadowing — Engineers spend time embedded in other teams quarterly.
+   - Documented decisions — If only one person knows WHY a decision was made, it is
+     organizational debt.
+   - Recorded walkthroughs — Screen-record explanations of complex systems. 10-minute
+     videos are more accessible than 50-page docs.
+   - Code ownership rotation — Periodically reassign code review duties to spread
+     understanding.
+
+7. **Metrics to track:**
+   - **Bus factor per component** — Number of people who can independently maintain
+     each critical system. Target: minimum 3.
+   - **Time-to-productive** — How long until a new joiner can ship independently?
+     Track trend over time.
+   - **Documentation freshness** — When was each critical doc last updated? Flag
+     anything >6 months stale.
+   - **Knowledge sharing participation** — Who presents tech talks, who reviews across
+     teams, who pairs with new joiners?
+
+### After
+
+8. **Establish cadence** — Knowledge sharing is not a one-time project. Set recurring
+   schedules for tech talks, documentation reviews, and bus factor assessments.
+9. **Reward sharing** — Include knowledge sharing in performance reviews. Recognize
+   people who write great docs, mentor others, or reduce bus factor.
+10. **Audit quarterly** — Review bus factor scores, onboarding feedback, and
+    documentation freshness. Invest where gaps persist.
+
+## Self-check before task completion
+
+- [ ] Bus factor assessed for all critical systems (target: minimum 3 per component)
+- [ ] Knowledge types categorized with appropriate capture methods
+- [ ] Decision log system implemented (lightweight ADRs at minimum)
+- [ ] Onboarding path documented for each team/domain
+- [ ] Regular cadence established for tech talks or equivalent
+- [ ] Documentation freshness tracked with staleness alerts
+- [ ] Time-to-productive measured and trending downward
+- [ ] Knowledge sharing recognized in performance evaluation
+- [ ] Tribal knowledge capture plan for single-points-of-knowledge