structured-context 0.9.0

package/docs/rules.md

@@ -0,0 +1,120 @@
+# Executable Rules
+
+Rules are JSONata expressions in schema metadata (`$metadata.rules`). They run after structural JSON Schema validation and let you enforce cross-node checks and workflow constraints.
+
+For metadata structure and composition behavior, see [docs/schemas.md](schemas.md).
+
+## Rule shape
+
+Rules are a flat array.
+
+```json5
+"rules": [
+  {
+    "id": "active-outcome-count",
+    "category": "workflow",
+    "description": "Only one outcome should be active at a time",
+    "scope": "global",
+    "check": "$count(nodes[resolvedType='outcome' and status='active']) <= 1"
+  }
+]
+```
+
+| Field | Required | Description |
+|---|---|---|
+| `id` | yes | Unique rule identifier |
+| `category` | yes | `validation` \| `coherence` \| `workflow` \| `best-practice` |
+| `description` | yes | Human-readable rule intent |
+| `check` | yes | JSONata expression; must evaluate to `true` |
+| `type` | no | Restrict rule to nodes of this `resolvedType` |
+| `scope` | no | Use `"global"` to evaluate once for the whole space |
+| `override` | no | Only for merge conflicts: allows later duplicate `id` to replace earlier |
+
+## Categories
+
+Categories label violations for reporting. They do not change expression execution semantics.
+
+| Category | Typical use |
+|---|---|
+| `validation` | Hard correctness constraints |
+| `coherence` | Cross-node consistency checks |
+| `workflow` | Process/operating-discipline checks |
+| `best-practice` | Advisory quality checks |
+
+## Evaluation model
+
+- Rules with `scope: "global"` run once.
+- Other rules run per applicable node.
+- If `type` is set, only nodes with matching `resolvedType` are evaluated.
+
+## Expression context
+
+Each evaluation receives:
+
+| Variable | Description |
+|---|---|
+| `nodes` | All nodes in the space |
+| `current` | Current node for this evaluation |
+| `parent` | First resolved parent node (if any) |
+| `parents` | All resolved parent nodes (if any) |
+
+Useful resolved fields on nodes:
+- `resolvedType`
+- `resolvedParentTitle`
+- `resolvedParentTitles`
+
+Prefer `resolvedType` over raw `type` so aliases are handled correctly.
+
+## Predicate scoping (`$$`)
+
+Inside `nodes[...]`, bare names refer to each candidate node. Use `$$` to reference outer variables:
+
+```jsonata
+$count(nodes[resolvedParentTitle=$$.current.title and resolvedType='solution'])
+```
+
+## Rule imports (`$ref`)
+
+`$metadata.rules` can include `$ref` entries that import:
+- one rule
+- a rule-set object with `rules: []`
+
+Example:
+
+```json5
+"rules": [
+  { "$ref": "sctx://rule-pack#/$defs/workflowRule" },
+  { "$ref": "sctx://rule-pack#/$defs/coreRuleSet" }
+]
+```
+
+Imported entries are normalized into the same flat runtime list.
+
+## Merge and conflict behavior
+
+When metadata is composed across `$ref`:
+- Rules are merged by `id`.
+- Different payloads for the same `id` are an error by default.
+- A later rule may replace an earlier one only with `override: true`.
+
+Example override:
+
+```json5
+{
+  "id": "active-outcome-count",
+  "override": true,
+  "category": "workflow",
+  "description": "Require exactly one active outcome",
+  "scope": "global",
+  "check": "$count(nodes[resolvedType='outcome' and status='active']) = 1"
+}
+```
+
+## Common patterns
+
+```jsonata
+$exists(current.metric) = true
+$count(current.sources) >= 1
+$count(nodes[resolvedType='outcome' and status='active']) <= 1
+current.status != 'active' or $exists(parent) = false or parent.status = 'active'
+```

package/docs/schemas.md

@@ -0,0 +1,340 @@
+# Schemas
+
+This document explains schema usage, metadata shape, and composition semantics in `structured-context`.
+
+## Overview
+
+A **schema** defines the valid structure for nodes in a `space`: entity types, field constraints, hierarchy behavior, type aliases, and executable rules.
+
+`structured-context` uses JSON Schema Draft-07 plus a custom top-level `$metadata` keyword.
+
+## Selecting a schema
+
+Set `schema` in the space config entry:
+
+```json
+{
+  "name": "my-space",
+  "path": "/path/to/space",
+  "schema": "schemas/strict_ost.json"
+}
+```
+
+Resolution order: space `schema` > global `schema` > bundled `schemas/general.json`.
+
+## Bundled schemas
+
+### `general.json` (default)
+
+Flexible planning schema spanning strategy + OST-like flow.
+
+Main types:
+- `vision`
+- `mission`
+- `goal` (alias: `outcome`)
+- `opportunity`
+- `solution`
+- `experiment` (aliases: `assumption_test`, `test`)
+
+### `strict_ost.json`
+
+Canonical 4-level OST structure.
+
+Main types:
+- `outcome`
+- `opportunity`
+- `solution`
+- `assumption_test`
+
+This schema composes shared structural defs and strict metadata/rules from partials.
+
+## Custom format annotations
+
+`structured-context` registers the following `format` annotations beyond standard JSON Schema. All apply to `string` properties and are validated at schema validation time.
+
+| Format | Validates | Example |
+|--------|-----------|---------|
+| `date` | ISO 8601 date (`YYYY-MM-DD`) | `"2026-03-31"` |
+| `path` | Non-empty filesystem path — absolute, relative, or a plain name | `"notes"`, `"./subdir/file.md"`, `"/abs/path"` |
+| `wikilink` | Obsidian wikilink syntax (`[[...]]`) | `"[[Parent Node]]"` |
+
+`path` and `wikilink` are also available as shared `$ref` definitions in `_sctx_base.json`:
+
+```json
+{ "$ref": "sctx://_sctx_base#/$defs/wikilink" }
+```
+
+Using `format` directly is more concise when the full definition isn't needed:
+
+```json
+{ "type": "string", "format": "wikilink" }
+```
+
+### Date coercion
+
+YAML parsers (gray-matter, js-yaml) coerce unquoted ISO dates to JavaScript `Date` objects:
+
+```yaml
+published_date: 2026-03-31   # parsed as a Date object by gray-matter
+```
+
+The markdown plugin automatically coerces `Date` objects to `YYYY-MM-DD` strings before validation, so unquoted dates in frontmatter and embedded YAML blocks work correctly with `format: "date"` fields.
+
+## Metadata dialect
+
+Schemas use this metaschema URL:
+
+- `https://raw.githubusercontent.com/mindsocket/structured-context/main/schemas/generated/_structured_context_schema_meta.json`
+
+Top-level metadata shape:
+
+```json5
+{
+  "$metadata": {
+    "hierarchy": {
+      "levels": [
+        "outcome",
+        { "type": "opportunity", "selfRef": true },
+        "solution",
+        "assumption_test"
+      ],
+      "allowSkipLevels": false
+    },
+    "aliases": {
+      "experiment": "assumption_test"
+    },
+    "rules": [
+      {
+        "id": "active-outcome-count",
+        "category": "workflow",
+        "description": "Only one outcome should be active at a time",
+        "scope": "global",
+        "check": "$count(nodes[resolvedType='outcome' and status='active']) <= 1"
+      }
+    ]
+  }
+}
+```
+
+### `$metadata` fields
+
+| Field | Type | Notes |
+|---|---|---|
+| `hierarchy` | object | Optional per provider; at most one provider may define it after composition |
+| `hierarchy.levels` | `(string \| HierarchyLevel)[]` | Ordered root→leaf types |
+| `hierarchy.allowSkipLevels` | `boolean` | Optional; allows parent to be any ancestor level |
+| `relationships` | `Relationship[]` | Optional; defines related node links outside the primary hierarchy |
+| `aliases` | `Record<string, string>` | Optional type alias map |
+| `rules` | `Rule[]` | Optional flat rule array |
+
+### Relationships
+
+Relationships define links between node types that are not part of the primary structural hierarchy. They are handled during parsing and template generation.
+
+| Field | Type | Default | Description |
+|---|---|---|---|
+| `parent` | `string` | **Required** | The parent's canonical type name |
+| `type` | `string` | **Required** | The child's canonical type name |
+| `field` | `string` | `"parent"` | The frontmatter field that holds the wikilink(s) for this relationship |
+| `fieldOn` | `string` | `"child"` | `"child"` (child holds a link to parent) or `"parent"` (parent holds an array of child links) |
+| `format` | `string` | `"page"` | Hint for `template-sync`: `"table"`, `"list"`, or `"heading"` |
+| `matchers` | `string[]` | `[]` | Heading text to match (strings or `/regex/`). Case-insensitive. |
+| `embeddedTemplateFields` | `string[]` | `[]` | Field names to include in templates when `format` is `"table"` |
+| `multiple` | `boolean` | `true` | Whether multiple children are expected |
+
+**`fieldOn: "child"` (default)** — child node has a field pointing to its parent. Embedded parsing sets this field on each child node; validation checks that it resolves to a node of the declared parent type.
+
+**`fieldOn: "parent"` — parent-side array** — the parent node has an array field (`field`) holding wikilinks to child nodes. When `field` is required, it must be specified. Embedded parsing appends `[[Child]]` entries to the parent node's field array; validation checks that each entry resolves to a node of the declared child type.
+
+**Example — child-side (default):**
+
+```json5
+"relationships": [
+  {
+    "parent": "opportunity",
+    "type": "assumption",
+    "format": "table",
+    "matchers": ["Assumptions", "/assum.*/"],
+    "embeddedTemplateFields": ["assumption", "status", "confidence"]
+  }
+]
+```
+
+**Example — parent-side array:**
+
+```json5
+"relationships": [
+  {
+    "parent": "activity",
+    "type": "task",
+    "field": "tasks",
+    "fieldOn": "parent",
+    "format": "list",
+    "matchers": ["Tasks"],
+    "multiple": true
+  }
+]
+```
+
+With this configuration, embedded task items under an Activity's "Tasks" heading populate `activity.tasks` as `[[Task Title]]` wikilinks, and validation confirms each entry resolves to a `task` node.
+
+`HierarchyLevel` options:
+
+| Option | Default | Meaning |
+|---|---|---|
+| `type` | required | Canonical type name |
+| `field` | `"parent"` | Frontmatter field holding wikilink(s) |
+| `fieldOn` | `"child"` | `"parent"` means the parent points to children |
+| `multiple` | `false` | Field contains array of wikilinks |
+| `selfRef` | `false` | Allows same-type parent |
+| `selfRefField` | _undefined_ | Separate field for same-type parent links |
+| `format` | _undefined_ | Embedding hint: `"list"`, `"table"`, or `"heading"` — enables hierarchy embedding when set |
+| `matchers` | _undefined_ | Heading text patterns (strings or `/regex/`) to detect embedding sections |
+| `embeddedTemplateFields` | _undefined_ | Column/field names for table stubs generated by `template-sync` |
+
+String shorthand (`"goal"`) normalizes to:
+`{ "type": "goal", "field": "parent", "fieldOn": "child", "multiple": false, "selfRef": false }`.
+
+### Hierarchy embedding
+
+When a hierarchy level has `format` and `matchers`, it participates in **hierarchy embedding** — the same section-based parsing used for relationships. Two patterns are supported:
+
+**Child-level embedding** — a typed-page heading signals that following content should produce nodes of the child type (next level in hierarchy):
+
+```json5
+"levels": [
+  "goal",
+  {
+    "type": "opportunity",
+    "field": "parent",
+    "fieldOn": "child",
+    "format": "list",
+    "matchers": ["Opportunities", "User Opportunities"]
+  }
+]
+```
+
+With this config, a goal page with `### Opportunities` followed by a list creates opportunity nodes without explicit `[type:: opportunity]` annotations.
+
+**Bare wikilinks in embedded lists** — when a list item is a bare wikilink (`- [[Node Title]]`) inside an embedding section, it populates a field on the parent without creating a new node:
+
+```markdown
+### tool
+- [[Zephyr]]        ← populates activity.tools = ["[[Zephyr]]"]
+- New Custom Tool   ← creates a new tool node
+```
+
+This requires `fieldOn: "parent"` (the parent node holds the array field).
+
+**Parent-level references** (`fieldOn: "child"`, child holds the field) — a heading matching the immediate parent type lets a node reference its parents by wikilink:
+
+```json5
+"levels": [
+  { "type": "capability", "field": "parent", "fieldOn": "child", "multiple": false },
+  {
+    "type": "application",
+    "field": "capabilities",   // application nodes have a capabilities field
+    "fieldOn": "child",
+    "multiple": true
+  }
+]
+```
+
+With this config, an application page may include `### capability` with wikilink items to populate `application.capabilities`:
+
+```markdown
+## My Application ^application1
+### capability
+- [[Task Management]]    ← populates application.capabilities
+### tool
+- [[Zephyr]]             ← populates application.tools (fieldOn: parent)
+```
+
+## Composition and merge semantics
+
+Metadata is composed across the `$ref` graph with deterministic behavior:
+
+1. Traverse external `$ref` graph in DFS order.
+2. Apply root schema metadata last.
+
+Merge rules:
+- `hierarchy`: zero or one provider allowed. Multiple providers error.
+- `aliases`: shallow merged; later provider wins per key.
+- `rules`: merged by `id`.
+- Duplicate rule `id` with different payload errors by default.
+- A later rule may replace an earlier one only with `"override": true`.
+
+When no provider defines `hierarchy`, hierarchy-based behavior is disabled (`show` tree shape, hierarchy validation, parent-edge checks). `space_on_a_page` parsing still requires hierarchy and will error without it.
+
+### Rule imports via `$ref`
+
+Inside `$metadata.rules`, entries can be inline rules or `$ref` imports:
+
+```json5
+"rules": [
+  { "$ref": "sctx://my-pack#/$defs/workflowRule" },
+  { "$ref": "sctx://my-pack#/$defs/ruleSet" }
+]
+```
+
+Import targets may be:
+- a single rule object
+- an object containing `rules: []`
+
+Imported rules are normalized into one executable flat list before validation.
+
+### Override example
+
+```json5
+{
+  "$metadata": {
+    "rules": [
+      {
+        "id": "active-outcome-count",
+        "override": true,
+        "category": "workflow",
+        "description": "Require exactly one active outcome",
+        "scope": "global",
+        "check": "$count(nodes[resolvedType='outcome' and status='active']) = 1"
+      }
+    ]
+  }
+}
+```
+
+## Partials and `$ref`
+
+- Files starting with `_` are auto-loaded partials.
+- Both bundled partials and local schema-directory partials are registered.
+- Local partial `$id` values must not collide with bundled IDs.
+- `$ref` resolution is transitive across files.
+- Partials with no `$metadata` should prefer `$schema: "http://json-schema.org/draft-07/schema#"` so they validate standalone as plain JSON Schema fragments.
+
+## Editor expectations
+
+Use the shipped metaschema URL in `$schema` for best cross-tool behavior.
+
+Notes:
+- Custom `$id` values like `sctx://...` are still supported by the CLI registry.
+- Some generic editors may not resolve custom URI schemes for `$ref`; CLI behavior is authoritative.
+- Do not rely on editor-only mappings for runtime correctness.
+
+## Breaking migration checklist (legacy -> current)
+
+For schemas migrating from older metadata structure:
+
+1. Move any legacy metadata from `$defs._metadata` to top-level `$metadata`.
+2. Convert `hierarchy` array to `hierarchy.levels` object shape.
+3. Move `allowSkipLevels` under `hierarchy`.
+4. Convert grouped rule containers to flat `rules[]` with per-rule `category`.
+5. If duplicate rule IDs are intentional, mark later rules with `override: true`.
+6. Re-run `bunx structured-context schemas show --space <name>` and `validate` to confirm merged metadata/rules.
+
+## JSON5 support
+
+Schema files are parsed as JSON5 (`//` comments and trailing commas are allowed).
+
+## Further reading
+
+- [Executable Rules](rules.md)
+- [JSON Schema](https://json-schema.org/)

package/package.json

@@ -0,0 +1,69 @@
+{
+  "name": "structured-context",
+  "version": "0.9.0",
+  "description": "Tools for working with Opportunity Solution Tree structures and other product management and strategy frameworks",
+  "module": "./dist/index.js",
+  "type": "module",
+  "license": "MIT",
+  "bin": {
+    "sctx": "dist/index.js"
+  },
+  "files": [
+    "dist/",
+    "schemas/",
+    "docs/concepts.md",
+    "docs/config.md",
+    "docs/schemas.md",
+    "docs/rules.md"
+  ],
+  "exports": {
+    ".": "./dist/index.js",
+    "./plugin-api": {
+      "types": "./dist/plugin-api.d.ts",
+      "default": "./dist/plugin-api.js"
+    }
+  },
+  "scripts": {
+    "clean": "rm -rf dist",
+    "generate:schema-meta": "bun run scripts/generate-schema-meta.ts",
+    "typecheck": "tsc --noEmit",
+    "build": "bun run clean && bun run generate:schema-meta && tsc -p tsconfig.build.json && cp -r schemas dist/ && chmod +x dist/index.js",
+    "prepublishOnly": "bun run build",
+    "validate": "bun run src/index.ts validate",
+    "diagram": "bun run src/index.ts diagram",
+    "test": "bun test tests/",
+    "test:smoke": "bun test smoke/",
+    "test:hook": "bun test hook-test/unit/",
+    "test:hook:e2e": "bun test hook-test/hooks.test.ts",
+    "test:all": "bun test",
+    "lint": "biome check",
+    "lint:fix": "biome check --write",
+    "preversion": "bun run generate:schema-meta && bun run lint:fix && bun run test",
+    "version": "git add -A",
+    "postversion": "git push --follow-tags"
+  },
+  "devDependencies": {
+    "@anthropic-ai/claude-agent-sdk": "^0.2.92",
+    "@biomejs/biome": "^2.4.10",
+    "@types/bun": "latest",
+    "@types/js-yaml": "^4.0.9",
+    "json-schema-to-ts": "^3.1.1",
+    "lefthook": "^2.1.4"
+  },
+  "peerDependencies": {
+    "typescript": "^5"
+  },
+  "dependencies": {
+    "ajv": "^8.18.0",
+    "chokidar": "^5.0.0",
+    "commander": "^14.0.3",
+    "gray-matter": "^4.0.3",
+    "js-yaml": "^4.1.1",
+    "jsonata": "^2.1.0",
+    "mdast-util-to-string": "^4.0.0",
+    "remark-gfm": "^4.0.1",
+    "remark-parse": "^11.0.0",
+    "unified": "^11.0.5",
+    "unist-util-visit": "^5.1.0"
+  }
+}

package/schemas/_ost_strict.json

@@ -0,0 +1,81 @@
+{
+  "$schema": "https://raw.githubusercontent.com/mindsocket/structured-context/main/schemas/generated/_structured_context_schema_meta.json",
+  "$id": "sctx://_ost_strict",
+  // Shared definitions for the strict OST (Opportunity Solution Tree) schema.
+  // This schema follows Teresa Torres' canonical 4-level structure as described in
+  // "Continuous Discovery Habits" (2021) and at producttalk.org.
+  "$metadata": {
+    "hierarchy": {
+      "levels": ["outcome", { "type": "opportunity", "selfRef": true }, "solution", "assumption_test"]
+    },
+    "rules": [
+      {
+        "id": "active-outcome-count",
+        "category": "workflow",
+        "description": "Only one outcome should be active at a time",
+        "scope": "global",
+        "check": "$count(nodes[resolvedType='outcome' and status='active']) <= 1"
+      },
+      {
+        "id": "active-opportunity-count",
+        "category": "workflow",
+        "description": "Only one target opportunity should be active at a time",
+        "scope": "global",
+        "check": "$count(nodes[resolvedType='opportunity' and status='active']) <= 1"
+      },
+      {
+        "id": "active-node-parent-active",
+        "category": "workflow",
+        "description": "An active node's parent should also be active",
+        "check": "current.status != 'active' or $exists(parent) = false or parent.status = 'active'"
+      },
+      {
+        "id": "solution-quantity",
+        "category": "best-practice",
+        "description": "Explore multiple candidate solutions (aim for at least three) for the target opportunity",
+        "type": "opportunity",
+        "check": "(current.status != 'exploring' and current.status != 'active') or $count(nodes[resolvedParentTitle=$$.current.title and resolvedType='solution']) >= 3"
+      }
+    ]
+  },
+  "$defs": {
+    "outcomeProps": {
+      "type": "object",
+      "description": "Properties for an Outcome node (product metric, not vision/mission)",
+      "properties": {
+        "metric": {
+          "type": "string",
+          "description": "The specific product metric to move (e.g., 'Increase % of first-time users who reach the aha moment')"
+        }
+      },
+      "required": ["metric"]
+    },
+    "opportunityProps": {
+      "type": "object",
+      "description": "Properties for an Opportunity, emphasizing research grounding",
+      "properties": {
+        "source": {
+          "type": "string",
+          "description": "Customer research source that grounded this opportunity (e.g., 'Interview with Jane, 2024-03-15')"
+        }
+      },
+      "required": ["source"]
+    },
+    "assumptionTestProps": {
+      "type": "object",
+      "description": "Properties for an Assumption Test (the fourth level of the OST)",
+      "properties": {
+        "assumption": {
+          "type": "string",
+          "description": "The specific belief being tested"
+        },
+        "category": {
+          "type": "string",
+          "enum": ["desirability", "viability", "feasibility", "usability", "ethical"],
+          "description": "Assumption type categories"
+        }
+      },
+      "required": ["assumption"]
+    }
+  }
+}

package/schemas/_sctx_base.json

@@ -0,0 +1,72 @@
+{
+  "$schema": "http://json-schema.org/draft-07/schema#",
+  "$id": "sctx://_sctx_base",
+  "description": "Shared definitions for structured-context schemas",
+  "$defs": {
+    "title": {
+      "type": "string",
+      "description": "Title of the node"
+    },
+    "content": {
+      "type": "string",
+      "description": "Body content"
+    },
+    "summary": {
+      "type": "string",
+      "description": "Short summary"
+    },
+    "status": {
+      "type": "string",
+      "enum": ["identified", "wondering", "exploring", "active", "paused", "completed", "archived"]
+    },
+    "priority": {
+      "type": "string",
+      "enum": ["p1", "p2", "p3", "p4"]
+    },
+    "assessment": {
+      "type": "integer",
+      "minimum": 1,
+      "maximum": 5,
+      "description": "Assessment score from 1-5"
+    },
+    "wikilink": {
+      "type": "string",
+      "pattern": "^\\[\\[.+\\]\\]$",
+      "description": "An Obsidian wikilink, e.g. [[Parent Node]]"
+    },
+    "baseNodeProps": {
+      "type": "object",
+      "properties": {
+        "title": { "$ref": "#/$defs/title" },
+        "content": { "$ref": "#/$defs/content" },
+        "tags": {
+          "type": "array",
+          "items": { "type": "string" },
+          "description": "Categorization tags"
+        }
+      },
+      "required": ["title"]
+    },
+    "ostEntityProps": {
+      "type": "object",
+      "properties": {
+        "status": { "$ref": "#/$defs/status" },
+        "summary": { "$ref": "#/$defs/summary" },
+        "status_tweet": {
+          "type": "string",
+          "description": "Succinct free-text status summary"
+        }
+      },
+      "required": ["status"]
+    },
+    "parentNodeProps": {
+      "type": "object",
+      "properties": {
+        "parent": {
+          "$ref": "#/$defs/wikilink",
+          "description": "Parent node"
+        }
+      }
+    }
+  }
+}