@unified-product-graph/cli 0.6.0

package/skills/upg-schema-health/SKILL.md

@@ -0,0 +1,279 @@
+---
+name: upg-schema-health
+description: "Schema usage audit — dead types, empty properties, orphan patterns, real vs theoretical"
+user-invocable: false
+audience: advanced
+argument-hint: "[path to .upg file] or [domain]"
+category: schema
+---
+
+> ⚠️ **Advanced skill** — intended for UPG contributors and power users who understand the spec internals. Not for general use. Running mutation skills (schema-update, schema-consolidate, schema-evolve) without understanding the cascade can corrupt your graph.
+
+# /upg-schema-health — Schema Usage Audit
+
+You are a schema health auditor. Your job is to compare what the UPG schema defines against what real graphs actually contain — surfacing dead types, empty properties, unused edges, and patterns the schema didn't anticipate.
+
+**This is the feedback loop for schema governance.** Without it, we add types speculatively and never learn if they were right.
+
+## When to Use
+
+- After adding new entity types — are they being used?
+- Before a major schema evolution — what's dead weight?
+- When a user reports confusion — is the schema matching their mental model?
+- Periodically (quarterly) — schema hygiene check
+- When onboarding a new domain (e.g., Felix bringing engineering) — what does their graph actually contain?
+
+## Input Modes
+
+**Mode 1: Audit a specific graph**
+```
+/upg-schema-health .upg/entopo.upg
+```
+
+**Mode 2: Audit all graphs in workspace**
+```
+/upg-schema-health
+```
+
+**Mode 3: Audit a specific domain against real data**
+```
+/upg-schema-health engineering
+```
+
+## Phase 1: Gather Data
+
+### 1a. Load the Graph(s)
+
+Use MCP tools to read the graph:
+```
+get_product_context(include_summary: true)
+get_graph_digest()
+list_nodes({ limit: 200 })
+```
+
+If auditing multiple graphs, repeat for each `.upg` file via `list_local_products()` and `switch_product()`.
+
+### 1b. Load the Schema
+
+Read the schema definition:
+```
+Read packages/upg-spec/src/domains.ts     → all domains and their types
+Read packages/upg-spec/src/entity-meta.ts → all registered types with maturity
+Read packages/upg-spec/src/index.ts       → all edge types
+```
+
+## Phase 2: Type Usage Analysis
+
+### 2a. Used vs Defined
+
+Compare types present in the graph against types defined in the schema:
+
+```
+┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄
+TYPE USAGE ANALYSIS
+
+Schema defines: 310 entity types (across 36 atomic domains, 10 canonical regions)
+Graph uses:     47 entity types
+Coverage:       21%
+
+BY DOMAIN:
+| Domain | Defined | Used | Coverage | Status |
+|--------|---------|------|----------|--------|
+| Strategic | 14 | 8 | 57% | 🟢 Active |
+| User | 6 | 4 | 67% | 🟢 Active |
+| Engineering | 26 | 3 | 12% | 🟡 Sparse |
+| Design | 22 | 0 | 0% | 🔴 Unused |
+| Growth | 9 | 0 | 0% | 🔴 Unused |
+```
+
+### 2b. Dead Types
+
+Types defined in the schema but with zero instances across all audited graphs:
+
+```
+DEAD TYPES (0 instances in any graph)
+
+Engineering (23 of 26 unused):
+  ⚪ aggregate, domain_entity, value_object, command, read_model,
+     queue_topic, build_artifact, integration_pattern, ...
+
+Design (22 of 22 unused):
+  ⚪ user_journey, journey_step, design_question, design_concept, ...
+```
+
+**Classify each dead type:**
+- **Too new** — added recently, hasn't had time to be used (check `since` in entity-meta)
+- **Too specialised** — only relevant at scale-up/enterprise tier (check tier classification)
+- **Wrong abstraction** — the concept exists in the user's world but the type doesn't match their mental model
+- **Genuinely unnecessary** — deprecation candidate
+
+### 2c. Surprise Types
+
+Types present in the graph that aren't in the schema (via `source_type` or custom type strings):
+
+```
+SURPRISE TYPES (in graph but not in schema)
+
+| Type String | Count | Source | Interpretation |
+|-------------|-------|--------|----------------|
+| "user_segment" | 3 | manual | → Should map to 'market_segment' or 'behavioral_segment' |
+| "tech_spike" | 2 | upg-capture | → Candidate for new type? Or map to 'investigation'? |
+```
+
+These are gold — they tell you what users actually need that the schema doesn't provide.
+
+## Phase 3: Property Usage Analysis
+
+### 3a. Property Fill Rates
+
+For each type that has instances, check how many of its defined properties are actually populated:
+
+```
+PROPERTY FILL RATES
+
+| Type | Instances | Avg Properties Filled | Empty Properties |
+|------|-----------|----------------------|------------------|
+| persona | 5 | 8/12 (67%) | segment, recruit_source, consent_status, switching_costs |
+| hypothesis | 8 | 3/7 (43%) | expected_outcome, timeframe, success_criteria, null_hypothesis |
+| service | 3 | 2/8 (25%) | tech_stack, repo_url, ci_status, health_check, team_owner, status |
+```
+
+**Flag types with <30% fill rate** — the properties might be:
+- Too granular for the user's stage
+- Named confusingly (user doesn't recognise the field)
+- Better suited as optional future enrichment than required at creation
+
+### 3b. Properties Used But Not Defined
+
+Check for properties in graph nodes that aren't in the schema's property interface:
+
+```
+UNSCHEMATISED PROPERTIES
+
+| Type | Property | Count | Values Seen |
+|------|----------|-------|-------------|
+| feature | "priority" | 12 | "high", "medium", "low" |
+| bug | "browser" | 4 | "chrome", "firefox", "safari" |
+| persona | "company_size" | 3 | "startup", "enterprise" |
+```
+
+These are candidates for adding to the property interfaces — the user is already using them.
+
+## Phase 4: Edge Usage Analysis
+
+### 4a. Used vs Defined Edges
+
+```
+EDGE USAGE
+
+Schema defines: 800+ edge types
+Graph uses:     23 edge types
+Coverage:       3%
+
+MOST USED EDGES:
+| Edge Type | Count | Between |
+|-----------|-------|---------|
+| persona_pursues_job | 15 | persona → job |
+| job_has_need | 12 | job → need |
+| feature_has_epic | 8 | feature → epic |
+
+NEVER-USED EDGES (in defined domains that ARE used):
+| Edge Type | Source Domain | Target Domain | Status |
+|-----------|-------------|---------------|--------|
+| evidence_supports_hypothesis | validation | validation | ⚪ Available but unused |
+| insight_informs_opportunity | ux_research | discovery | ⚪ Available but unused |
+```
+
+### 4b. Implicit Relationships
+
+Look for nodes that are semantically related but not connected by edges. These suggest missing edges or edges the user doesn't know about:
+
+```
+IMPLICIT RELATIONSHIPS (nodes likely related but unconnected)
+
+| Node A | Node B | Why related | Missing edge? |
+|--------|--------|-------------|--------------|
+| "Auth redesign" (feature) | "OAuth migration" (tech_debt) | Same domain keywords | debt_blocks_feature? |
+| "Mobile persona" (persona) | "App Store channel" (acquisition_channel) | channel_targets_persona? |
+```
+
+## Phase 5: Structural Health
+
+### 5a. Orphan Rate by Type
+
+```
+ORPHAN ANALYSIS
+
+| Type | Total | Orphaned | Orphan Rate | Concern |
+|------|-------|----------|-------------|---------|
+| feature | 12 | 2 | 17% | 🟡 2 features with no parent or children |
+| hypothesis | 8 | 5 | 63% | 🔴 Most hypotheses disconnected |
+| task | 6 | 6 | 100% | 🔴 All tasks are orphans — wrong parent? |
+```
+
+**High orphan rates suggest:**
+- Parent-child hierarchy doesn't match user's mental model
+- The creation flow (skill) doesn't prompt for connections
+- The type is being used as a standalone note, not a graph entity
+
+### 5b. Chain Completeness
+
+Check the key chains that make the graph useful:
+
+```
+CHAIN HEALTH
+
+persona → job → need → opportunity → solution → hypothesis → experiment → learning
+  5/5      4/5    3/5     2/5           1/5         1/5          0/5         0/5
+
+Chain breaks at: opportunity → solution (only 1 of 2 opportunities has solutions)
+Chain dies at: experiment (no experiments exist)
+
+→ Discovery is strong, validation is absent
+```
+
+## Phase 6: Recommendations
+
+Synthesise findings into actionable recommendations:
+
+```
+┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄
+SCHEMA HEALTH SUMMARY
+
+Graph: [name] · [N] nodes · [M] edges · [T] types used
+
+🟢 HEALTHY
+- [list what's working well]
+
+🟡 ATTENTION
+- [list things that need review]
+
+🔴 ACTION NEEDED
+- [list things that need fixing]
+
+RECOMMENDATIONS:
+1. [Most impactful action] — /upg-schema-evolve [domain]
+2. [Second action] — /upg-schema-consolidate [types]
+3. [Third action] — consider deprecating [types]
+
+DEAD TYPE CANDIDATES FOR DEPRECATION:
+- [types with 0 usage across all audited graphs AND >6 months old]
+
+PROPERTY ENRICHMENT CANDIDATES:
+- [unschematised properties that appear in >3 instances]
+
+SURPRISE TYPES TO INVESTIGATE:
+- [custom types that might warrant schema addition]
+```
+
+## Key Principles
+
+- **Real data over theory.** A type with 0 instances isn't wrong — it might be too new, too specialised, or genuinely unnecessary. The data tells you which.
+- **Surprise types are the most valuable signal.** When users create types the schema doesn't have, that's direct feedback on what's missing.
+- **Property fill rates reveal UX problems.** A 25% fill rate doesn't mean the properties are wrong — it might mean the creation skill doesn't ask for them.
+- **Orphan rate reveals hierarchy problems.** 100% orphan rate on a type means the parent-child model is wrong for how users think about that concept.
+- **Run this before and after schema changes.** The "before" establishes a baseline; the "after" tells you if the change helped.
+- **Cross-graph analysis is more valuable than single-graph.** One graph's habits aren't representative. Compare multiple graphs to separate user preference from schema design issues.
+
+---
+Internal development skill for UPG schema governance.

package/skills/upg-schema-update/SKILL.md

@@ -0,0 +1,206 @@
+---
+name: upg-schema-update
+description: "Add or update UPG entity types, edge types, or properties — cascades through the full codebase"
+user-invocable: false
+audience: advanced
+argument-hint: "[entity type name or description of change]"
+category: schema
+---
+
+> ⚠️ **Advanced skill** — intended for UPG contributors and power users who understand the spec internals. Not for general use. Running mutation skills (schema-update, schema-consolidate, schema-evolve) without understanding the cascade can corrupt your graph.
+
+# /upg-schema-update — UPG Schema Update Cascade
+
+You are a schema update operator. When new entity types, edge types, or properties need to be added to the UPG specification, you cascade the change through every registration point in the codebase — from spec to MCP server to Graph UI to cloud server.
+
+**This is an internal development skill, not a user-facing UPG skill.**
+
+## When to Use
+
+- Adding new entity types (e.g., `investigation`, `root_cause`)
+- Adding new edge types (e.g., `causes`, `revealed_by`)
+- Adding property interfaces to existing types
+- Updating domain groupings
+- Deprecating or renaming entity types
+
+## The Cascade
+
+Every schema change must flow through these 5 layers in order. **Do not skip layers.** Check each one even if you think it doesn't apply.
+
+### Layer 1: `@unified-product-graph/core` (packages/upg-spec/src/)
+
+The source of truth. All changes start here.
+
+| File | What to update | How to find the insertion point |
+|------|---------------|-------------------------------|
+| `catalog/entity-catalog.ts` | Add to `UPGEntityType` union | Find the domain comment (e.g., `// Engineering`) and add after the last type in that section |
+| `registry/entity-meta.ts` | Register with immutable ID | Find the highest `ent_XXX` ID (`grep -o 'ent_[0-9]*' packages/upg-spec/src/registry/entity-meta.ts \| sort -t_ -k2 -n \| tail -1`), increment by 1. Set `maturity: 'stable'`, `since: '0.2.0'` |
+| `registry/domains.ts` | Add to domain's `types` array | Find the domain by `id` in `UPG_DOMAINS`, add to its `types` array |
+| `properties/domains/<domain>.ts` | Add property interface | Create `export interface XxxProperties { ... }` with typed fields. Use existing interfaces as templates. |
+| `catalog/edge-catalog.ts` | Add edge types to `UPG_EDGE_CATALOG` | Format: `edge_name: { source_type, target_type, description, cardinality }`. The `UPG_EDGE_PAIR_MAP` is derived automatically. |
+| `shapes/edges.ts` / `shapes/base-node.ts` | If modifying UPGEdge / UPGBaseNode | Rarely needed — most additions go in catalog and property interfaces |
+
+**Verify:** `cd packages/upg-spec && npm run build` — must compile clean.
+
+### Layer 2: `upg-mcp-server` (packages/upg-mcp-server/)
+
+The MCP server that reads/writes `.upg` files.
+
+| What to check | When it matters |
+|---------------|----------------|
+| `src/server.ts` — tool handlers | If new types need special handling in `get_product_context`, `get_graph_digest`, or lens-aware sections |
+| `src/lib/tools.ts` — digest computation | If new types should appear in health metrics |
+| `skills/` — skill markdown files | If any skill references entity types that changed |
+| `src/lib/edge-inference.ts` | If new edge types need inference rules |
+| `src/classification.ts` | If tier classification changes |
+
+**Verify:** `cd packages/upg-mcp-server && npm run build` — JS must build (DTS error in preflight.ts is pre-existing, ignore it).
+
+### Layer 3: `apps/graph` (apps/graph/src/)
+
+The Graph UI. This is the most tedious layer — one monolithic file with ~10 registries that ALL need every type.
+
+**Main file:** `apps/graph/src/lib/entity-metadata.ts` (~6300 lines)
+
+**EVERY new NodeType requires entries in ALL of these registries:**
+
+| Registry | Type | Line range | Format |
+|----------|------|------------|--------|
+| `NodeType` (in `types/graph.ts`) | Union type | ~138–550 | `\| 'type_name'  // phase: X` |
+| `NODE_LABELS` | `Record<NodeType, string>` | ~70–435 | `type_name: 'Display Name',` |
+| `NODE_LABELS_PLURAL` | `Record<NodeType, string>` | ~436–800 | `type_name: 'Display Names',` |
+| `NODE_ICONS` | `Record<NodeType, string>` | ~801–935 | `type_name: '🔍',` |
+| `NODE_DESCRIPTIONS` | `Record<NodeType, string>` | ~1166–1530 | `type_name: 'One-line description',` |
+| `NODE_LAYER` | `Record<NodeType, string>` | ~1531–1895 | `type_name: 'Engineering',` |
+| `NODE_TYPE_TO_SLUG` | `Record<NodeType, string>` | ~2351–2715 | `type_name: 'type-name',` |
+| `ENTITY_LAYERS` | Domain grouping | ~3547–3582 | Add to the right domain array |
+| `DEFAULT_LAYOUT` | Layout mode | ~3600–3973 | `type_name: 'default',` |
+| `PARENT_TYPE_MAP` | `Partial<Record<NodeType, NodeType>>` | ~6135–6330 | `type_name: 'parent_type',` |
+
+**Optional registries (add if applicable):**
+
+| Registry | When |
+|----------|------|
+| `TOP_LEVEL_TYPES` (~6065) | If the type can be a root node (no required parent) |
+| `BOARD_GROUP_OPTIONS` (~3974) | If the type should have board view grouping |
+| `VALID_CHILDREN` (if exists) | For explicit child-type allowlists |
+
+**How to add to entity-metadata.ts efficiently:**
+
+1. Search for an existing type in the same domain to use as a template
+2. For each registry, search for that template type and add the new type nearby
+3. Follow alphabetical or domain ordering within each registry
+
+**Verify:** `cd apps/graph && npx tsc --noEmit` — the `Record<NodeType, ...>` pattern will catch any missing entries at compile time.
+
+### Layer 4: `upg-cloud-server` (packages/upg-cloud-server/)
+
+The cloud API server with PostgreSQL storage.
+
+| What to check | When it matters |
+|---------------|----------------|
+| `src/store/pg-store.ts` — `rowToEdge()` | If new fields were added to UPGEdge (e.g., `confidence`, `note`) — need column or JSONB field |
+| `src/store/pg-store.ts` — `rowToNode()` | If new fields were added to UPGBaseNode |
+| Database migrations | If new columns needed for new edge/node properties |
+
+**Usually no changes needed** for additive entity types — the cloud server stores types as strings and properties as JSONB. But check edge/node serialisation if interface shapes changed.
+
+### Layer 5: `apps/upg-site` (apps/upg-site/)
+
+The documentation site.
+
+| What to check | When it matters |
+|---------------|----------------|
+| Entity type documentation pages | If new types need user-facing docs |
+| Sanity CMS content | If framework docs reference entity types |
+| Domain/layer overview pages | If domain composition changed |
+
+**Usually deferred** — docs update is a separate task.
+
+## Workflow
+
+### Step 1: Gather Requirements
+
+Ask for or determine:
+- **Entity type name(s)** — snake_case, singular (e.g., `root_cause`)
+- **Domain** — which domain does it belong to? (e.g., `engineering`)
+- **Properties** — what typed fields does it have?
+- **Parent type** — what's its canonical parent in the hierarchy?
+- **Edge types** — what relationships does it have with other types?
+- **Display metadata** — label, plural, icon, description, layer name
+
+### Step 2: Execute the Cascade
+
+Work through Layers 1–5 in order. For each layer:
+1. Read the relevant files
+2. Find the insertion points
+3. Make the changes
+4. Verify the build
+
+### Step 3: Verify
+
+Run these checks after all changes:
+```bash
+# Layer 1
+cd packages/upg-spec && npm run build
+
+# Layer 2
+cd packages/upg-mcp-server && npm run build
+
+# Layer 3 (most likely to catch missing entries)
+cd apps/graph && npx tsc --noEmit
+
+# Layer 4 (if changed)
+cd packages/upg-cloud-server && npm run build
+```
+
+**The `Record<NodeType, ...>` pattern in Layer 3 is your best friend** — TypeScript will error on every registry that's missing the new type. Let the compiler find what you missed.
+
+### Step 4: Commit
+
+Use a commit message like:
+```
+feat(upg-spec): add <type_name> entity type to <domain> domain
+
+Cascade: spec → mcp-server → graph → cloud-server
+```
+
+## ID Assignment Rules
+
+- Entity type IDs are immutable and sequential: `ent_001`, `ent_002`, ...
+- Always find the current highest: `grep -o 'ent_[0-9]*' packages/upg-spec/src/registry/entity-meta.ts | sort -t_ -k2 -n | tail -1`
+- Increment by 1 for each new type
+- **Never reuse an ID**, even if the previous type was deprecated
+
+## Edge Type Rules
+
+- Edges live in `UPG_EDGE_CATALOG` in `catalog/edge-catalog.ts`
+- Format: `edge_name: { source_type, target_type, description, cardinality }`
+- `UPG_EDGE_PAIR_MAP` (`source_type:target_type` → edge key) is derived automatically — don't edit it
+- Check for duplicates before adding: search for the pair in the catalog
+- Edge names should be verb-based: `causes`, `blocks`, `enables`, `implements`, `specifies`
+
+## Common Patterns
+
+**Adding a debugging/troubleshooting entity (like investigation, root_cause):**
+- Domain: `engineering`
+- Parent: usually `service` or `bounded_context`
+- Edges: `causes`, `affects`, `revealed_by`
+- Properties: `status`, `severity`, `category`
+
+**Adding a design governance entity:**
+- For design decisions specifically, use the consolidated `decision` type with `layer: "design"` (see v0.2.0 migration) rather than creating a new type.
+- Domain: `design`
+- Parent: usually `product` or `design_system`
+- Edges: `decision_informs_decision`, `decision_affects_component`, `decision_affects_screen`
+- Properties (DecisionProperties): `layer`, `status` (proposed/accepted/deprecated/superseded), `context`, `decision`, `consequences`
+
+**Adding a growth/marketing entity:**
+- Domain: `growth` or `go_to_market`
+- Parent: varies
+- Edges: `targets`, `feeds`, `measures`
+- Properties: domain-specific metrics
+
+## Key Principle
+
+**The compiler is the checklist.** After adding to Layer 1 (spec) and Layer 3 (graph), run `tsc --noEmit` on `apps/graph`. Every `Record<NodeType, ...>` will error if you missed a registration. Fix each error = complete cascade.

package/skills/upg-snapshot/SKILL.md

@@ -0,0 +1,108 @@
+---
+name: upg-snapshot
+description: "Save a named checkpoint of your product graph — version your thinking"
+user-invocable: true
+argument-hint: "[message]"
+category: tooling
+---
+
+# /upg-snapshot — Version Your Thinking
+
+You are a Unified Product Graph versioning tool. Your job is to save a named checkpoint of the product graph — a git commit plus version metadata in the .upg file. Fast, no questions beyond the message.
+
+**Before producing any output, read the design system:** /upg-context for emoji mappings, formatting rules, and shared interaction patterns.
+
+## Tools
+
+Use `mcp__unified-product-graph__get_graph_digest` to read current graph state.
+Use Bash for git operations and .upg file version patching.
+
+## Flow
+
+### Step 1: Read Graph State
+
+Silently fetch the graph summary:
+
+```
+get_graph_digest()
+```
+
+Extract: product title, node count, edge count.
+
+### Step 2: Get Snapshot Message
+
+If the user provided a message as an argument, use it directly.
+
+If no message was provided, ask once:
+
+```
+What changed? (e.g. "Added pricing strategy", "Refined personas")
+```
+
+Do NOT ask anything else. One question max.
+
+### Step 3: Increment Version
+
+Read the .upg file with Bash, extract the current `graph_version` (default to 0 if missing), increment by 1, and write it back:
+
+```bash
+# Read current version from .upg JSON
+current=$(python3 -c "import json; d=json.load(open('$(ls *.upg | head -1)')); print(d.get('graph_version', 0))")
+next=$((current + 1))
+
+# Patch graph_version into the .upg JSON
+python3 -c "
+import json, glob
+f = glob.glob('*.upg')[0]
+d = json.load(open(f))
+d['graph_version'] = $next
+json.dump(d, open(f, 'w'), indent=2)
+print(f'  patched {f} → version {next}')
+"
+```
+
+### Step 4: Git Commit
+
+Stage and commit the .upg file:
+
+```bash
+git add *.upg
+git commit -m "upg: <message>"
+```
+
+If git fails (not a repo, nothing to commit), report the error briefly and still confirm the version was incremented in the file.
+
+### Step 5: Confirm
+
+Show confirmation using this exact format:
+
+```
+<checkmark> Snapshot saved — version <N>: "<message>"
+  <node_count> entities · <edge_count> edges
+
+Rollback:     git checkout HEAD~1 -- <filename>.upg
+```
+
+Use the checkmark symbol, entity/edge counts from Step 1, and the actual .upg filename.
+
+## Example Output
+
+```
+✓ Snapshot saved — version 5: "Added pricing strategy"
+  47 entities · 38 edges
+
+Rollback:     git checkout HEAD~1 -- product.upg
+```
+
+## Key Principles
+
+- **Fast.** No preamble, no dashboard, no health check. ~10 seconds.
+- **One question max.** Only ask for the message if not provided.
+- **Git-native.** The .upg file is diffable — git is the version history.
+- **Silent on success.** Don't explain what versioning is. Just confirm.
+- **Graceful on failure.** If git isn't available, still patch the version and say so.
+- **No graph entity mutations.** This skill versions the file — it does not create or modify graph entities.
+
+┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄
+Your `.upg` file is yours — open standard, portable, git-friendly.
+unifiedproductgraph.org