@x12i/memorix-docs 1.0.1 → 1.1.0

package/files/use-cases/intelligence-propose-install.md

@@ -0,0 +1,163 @@
+# Intelligence propose → human install
+
+> **Use case id:** `intelligence-propose-install`
+> **Goal:** Profile and propose metadata, then install only via deliberate human action.
+> **Audiences:** developers
+> **Tags:** intelligence, metadata
+
+## Reading path
+
+1. **Structural Intelligence** (developers) → chapters: `2-deterministic-profile-first`, `3-propose-from-evidence`, `4-human-install-the-only-write-path`, `15-anti-patterns`
+1. **Metadata, Agents, Abstracts & Mappings** (developers) → chapters: `9-install-and-publish-never-auto`
+
+## From: Structural Intelligence — 2. Deterministic profile first
+
+Profiling is **structural and deterministic**. Feed it sample records (fixtures or bounded reads) and get evidence — not vibes.
+
+What a profile typically reports:
+
+| Signal | Why it matters |
+|---|---|
+| Paths & value types | Candidate field maps / content shapes |
+| Null vs missing | Optional vs required / mapping quirks |
+| Uniqueness | Identifier / key candidates |
+| Cardinality | One-vs-many content / relationship hints |
+| Overlap across samples | Abstract union / shared shape candidates |
+
+Air-gapped rule: heuristics must work with **no model credentials**. If AI is unavailable, profiling and proposal scaffolding still run; AI-only enrichment is skipped or marked insufficient.
+
+### HTTP — profile
+
+```http
+POST /api/intelligence/profile
+x-memorix-org-id: g12ops…
+x-memorix-agent-ids: ops
+content-type: application/json
+```
+
+Body (conceptual): scoped sample set + label (e.g. `employees`, `work-items`). Response: structural profile artifact — **not** an install.
+
+Acceptance orgs: fresh `g12ops*` only — never live customer orgs as profiling playgrounds that might leak into install habits.
+
+---
+
+## From: Structural Intelligence — 3. Propose from evidence
+
+A proposal is a **draft metadata pack fragment** (or diff) tied to profile evidence. Outcomes you must handle:
+
+| Outcome | Meaning | Next step |
+|---|---|---|
+| Evidence-backed suggestion | Enough structure to propose types/maps/… | Human review |
+| Insufficient evidence | Samples too thin / conflicting | Gather more fixtures; do not install |
+| Rejected by validation | Draft fails metadata/format rules | Fix samples or proposal rules; do not install |
+
+```ts
+const proposal = createProposal(profile, {
+  scope: { orgId: "g12ops…", agentIds: ["ops"] },
+  basePackId: "ops", // compare against approved ops when relevant
+});
+```
+
+### HTTP — proposals & helpers
+
+| Route | Role |
+|---|---|
+| `POST /api/intelligence/suggest/people` | Focused people/abstract-oriented suggestions |
+| `POST /api/intelligence/proposals` | Create / list proposal artifacts |
+| `POST /api/intelligence/compare` | Diff draft vs approved pack |
+| `POST /api/intelligence/export-draft` | Export a draft for review / offline inspect |
+
+None of these routes install metadata. If a host “Apply” button skips `/api/metadata/install` with `confirm: true` (or Studio’s explicit install), it is a product bug.
+
+---
+
+## From: Structural Intelligence — 4. Human install (the only write path)
+
+After review:
+
+```bash
+ORG=g12opsintel01
+H=(-H "x-memorix-org-id: $ORG" -H "x-memorix-agent-ids: ops")
+
+# Review proposal in Studio Design or via compare/export
+curl -s "${H[@]}" -X POST "$SERVICE/api/intelligence/compare" \
+  -H "content-type: application/json" \
+  -d '{"proposalId":"…","againstPackId":"ops"}'
+
+# Explicit install — separate family
+curl -s "${H[@]}" -X POST "$SERVICE/api/metadata/install" \
+  -H "content-type: application/json" \
+  -d '{"packId":"ops","confirm":true}'
+```
+
+Rules:
+
+1. **Never** auto-install on proposal create.
+2. **Never** treat AI confidence as permission to publish.
+3. Prefer installing into a fresh `g12ops*` first; promote only after acceptance.
+4. Reject thin proposals — insufficient evidence is a successful, honest outcome.
+
+Studio path: Design → intelligence proposal → review evidence → **Install / publish** deliberately (builders twin).
+
+---
+
+## From: Structural Intelligence — 15. Anti-patterns
+
+| Anti-pattern | Why it fails | Do this instead |
+|---|---|---|
+| Auto-install on propose | Silent production metadata drift | Explicit install with confirm |
+| Treat AI confidence as epistemic truth | Wrong gate | Evidence + human review |
+| Intelligence writes snapshots / links | Wrong package boundary | Pipelines / relationship engine |
+| Require live AI for CI | Blocks air-gapped gates | Deterministic fixtures |
+| Profile on live customer org “for fun” | Risk + habit | Fresh `g12ops*` |
+| Hide insufficient-evidence | Forces bad installs | Surface as first-class outcome |
+| Merge propose + install in one host API | Skips review | Two calls, two UX steps |
+| Invent abstract collections from suggestions | Violates virtual abstracts | Bindings only after install |
+| Use proposal text as record payload | Wrong layer | Metadata vs data plane |
+
+---
+
+## From: Metadata, Agents, Abstracts & Mappings — 9. Install and publish — never auto
+
+Hard product behavior: **Intelligence may propose metadata. Install/publish is always an explicit human or API step.**
+
+![Explicit metadata installation](../screenshots/19-explicit-metadata-installation.png)
+
+### Why never auto-install
+
+- Metadata changes change what engines will run next.
+- Accidental install on a shared org is a production incident.
+- Proposals can be wrong, incomplete, or unsafe — review is the product.
+
+### Routes
+
+| Route | Meaning |
+|---|---|
+| `POST /api/metadata/validate` | Check a pack without making it live |
+| `POST /api/metadata/install` | Install into the org metadata DB for the scoped agents |
+| `POST /api/metadata/publish` | Publish / promote a validated set per product policy |
+| `GET /api/metadata/packs` | List known packs |
+
+`@x12i/memorix-metadata-runtime` is the in-process helper behind those routes. Host apps that only talk HTTP never need to import it.
+
+### Ops acceptance flow
+
+1. Start fresh `g12ops*` + `agentIds: ["ops"]`.
+2. Validate the ops pack (or the subset you are changing).
+3. Install explicitly.
+4. Fetch `/api/metadata/effective` and confirm `People`, `work-items`, relationship definitions, sources.
+5. Only then run Memory / pipelines / relationship verify.
+
+If an intelligence proposal looks wrong, **reject it**. Do not install “to see what happens” on a shared org.
+
+Full propose / profile / artifact flow (how proposals are produced and reviewed): [Structural Intelligence](../../10-intelligence/developers/BOOK.md). This book only owns the **install boundary**: proposals are never live until an explicit install/publish.
+
+---
+
+## Also see
+
+- **Data Access Patterns** (`04-data-access-patterns`) — Decision matrix — best / acceptable / forbidden access methods for real client tasks.
+
+---
+
+_Generated use-case pack for agents and humans. See `agent-manifest.json` for discovery._

package/files/use-cases/knowx-claims-review.md

@@ -0,0 +1,396 @@
+# Build KnowX claims with human review
+
+> **Use case id:** `knowx-claims-review`
+> **Goal:** Model NODE/EDGE claims with provenance and run the fixture path without live AI.
+> **Audiences:** developers
+> **Tags:** knowx, provenance
+
+## Reading path
+
+1. **Building KnowX** (developers) → chapters: `1-core-architecture`, `2-mandatory-provenance-and-sourcespan`, `5-modeling-edge-assertions-claims`, `6-human-review-via-siblings`, `10-full-runnable-fixture-no-live-ai`
+
+## From: Building KnowX — 1. Core architecture
+
+Three rules:
+
+1. **Business subjects wear KnowX as a facet.**  
+   An employee, work item, or procedure becomes graph-visible by adding `contentType: "knowx"` on the **same** `recordId`.
+
+2. **Claims are first-class subjects.**  
+   An assertion between subjects gets its **own** `recordId` and object type (e.g. `assertions`) that also uses `contentType: "knowx"` with `data.kind: "EDGE"`.
+
+3. **Traversal is lightweight.**  
+   Heavy KnowX JSON is for audit. Fast graph UI reads materialized four-field relationship links — never copied payloads.
+
+```text
+Business subject (recordId)
+  ├─ snapshots / raw / workflow / …   ← ordinary Memorix content
+  └─ knowx                            ← NODE facet (epistemic + provenance)
+
+Assertion subject (its own recordId)
+  ├─ knowx                            ← EDGE kind + domain refs + evidence
+  ├─ workflow                         ← human review assignment (sibling)
+  └─ confirmations                    ← accept/reject decisions (sibling)
+
+Accepted EDGE knowx
+  └─ materialize → 4-field RelationshipLink (navigation only)
+```
+
+Same `recordId` ⇒ composition. Different `recordId` values ⇒ relationships. KnowX does not invent a parallel identity system.
+
+### One KnowXEntity (not Node/Edge types)
+
+| Layer | Name | Stored? |
+|---|---|---|
+| Content type | `knowx` | Yes |
+| Payload discriminator | `data.kind`: `NODE` \| `EDGE` | Inside `.data` |
+| Abstract projection | `KnowXEntity` | Virtual only |
+| Forbidden | `KnowXNode` / `KnowXEdge` types or collections | — |
+
+`KnowXEntity` bindings union concrete object types that may carry `knowx` (`employees`, `work-items`, `assertions`, …). Callers read the abstract; writers always target concrete object types.
+
+---
+
+## From: Building KnowX — 2. Mandatory provenance and SourceSpan
+
+Every KnowX `.data` includes:
+
+```ts
+type SourceSpan =
+  | { jsonPath: string }
+  | { lineStart: number; lineEnd: number }
+  | { byteStart: number; byteEnd: number };
+
+type KnowxProvenance = {
+  sourceClass: "measured" | "inferred" | "asserted" | "imported";
+  sources: Array<{
+    docId: string;
+    docType: string;
+    docHash: string; // sha-256 hex of exact source bytes/JSON
+    span: SourceSpan;
+    retrievedAt: string; // ISO-8601
+  }>;
+  extraction: {
+    method: string; // e.g. "deterministic_fixture", "api_fetch"
+    extractorId: string;
+    confidence: number; // 0..1 extraction confidence — NOT epistemic state
+  };
+};
+```
+
+Rules:
+
+- `docHash` hashes the **exact** source document bytes (or canonical JSON) that Memory landed — never a mutated copy.
+- `span` points at the region that justified this claim.
+- Missing provenance ⇒ reject the write in product validation / pipeline `validate` step.
+
+---
+
+## From: Building KnowX — 5. Modeling EDGE assertions (claims)
+
+An edge such as *employee-10 is assigned to wi-100* is not an employee and not a work item. It is a claim with its own `recordId`.
+
+Dedicated object type: `assertions` (category typically `knowledge` or `entity` per pack — pick one and keep it stable).
+
+```json
+{
+  "format": "memorix-record/2",
+  "recordId": "ASSERTION-99",
+  "revision": 1,
+  "objectType": "assertions",
+  "contentType": "knowx",
+  "dataCategory": "knowledge",
+  "concept": {
+    "title": "employee-10 assigned to wi-100",
+    "identifiers": {
+      "primary": {
+        "kind": "assertionId",
+        "value": "ASSERTION-99",
+        "issuer": "knowx-pipeline"
+      }
+    }
+  },
+  "data": {
+    "kind": "EDGE",
+    "label": "assigned-to",
+    "refs": {
+      "from": "employee-10",
+      "to": "wi-100",
+      "relationshipDefinitionId": "assigned-to",
+      "role": "assignee"
+    },
+    "provenance": {
+      "sourceClass": "inferred",
+      "sources": [
+        {
+          "docId": "fixture:activity-log-001",
+          "docType": "system_event",
+          "docHash": "a1b2c3d4e5f6g7h8i9j0k1l2m3n4o5p6q7r8s9t0u1v2w3x4y5z6a7b8c9d0e1f2",
+          "span": { "lineStart": 12, "lineEnd": 12 },
+          "retrievedAt": "2026-07-22T10:00:00.000Z"
+        }
+      ],
+      "extraction": {
+        "method": "deterministic_fixture",
+        "extractorId": "knowx-ops-fixture-v1",
+        "confidence": 0.85
+      }
+    },
+    "epistemic": {
+      "state": "hypothesis",
+      "confirmation": "pending",
+      "confidenceScore": 0.85,
+      "lastTransition": {
+        "from": "gap",
+        "to": "hypothesis",
+        "reason": "fixture extraction",
+        "triggeredBy": "pipeline",
+        "at": "2026-07-22T10:05:00.000Z"
+      }
+    }
+  },
+  "_system": {
+    "state": "active",
+    "createdAt": "2026-07-22T10:05:00.000Z",
+    "modifiedAt": "2026-07-22T10:05:00.000Z"
+  }
+}
+```
+
+### `refs` as domain semantics
+
+| Ref key | Role |
+|---|---|
+| `from` / `to` | Endpoint subjects of the claim |
+| `relationshipDefinitionId` | Which Memorix relationship definition this claim intends to materialize |
+| `role` / custom keys | Domain-specific claim structure |
+
+These refs describe **what the claim means**. Graph navigation still requires materialization (section 7). Do not walk `refs` as a substitute for `/api/relationships/query`.
+
+### Reification
+
+Because `ASSERTION-99` is a subject, another EDGE can set `refs.from: "ASSERTION-99"`. Claims about claims stay ordinary Memorix subjects.
+
+---
+
+## From: Building KnowX — 6. Human review via siblings
+
+Do not stuff review UI state into provenance. Use composition siblings on the assertion `recordId`.
+
+### Workflow sibling
+
+```json
+{
+  "format": "memorix-record/2",
+  "recordId": "ASSERTION-99",
+  "revision": 1,
+  "objectType": "assertions",
+  "contentType": "workflow",
+  "dataCategory": "knowledge",
+  "concept": {
+    "title": "Review workflow for ASSERTION-99",
+    "identifiers": {
+      "primary": { "kind": "assertionId", "value": "ASSERTION-99", "issuer": "knowx-pipeline" }
+    }
+  },
+  "data": {
+    "state": "pending-review",
+    "assignedTo": ["ops-reviewers"]
+  }
+}
+```
+
+Use `PUT /api/workflow` with `objectType=assertions` and exact `expectedRevision` (book `09`).
+
+### Confirmations sibling
+
+```json
+{
+  "format": "memorix-record/2",
+  "recordId": "ASSERTION-99",
+  "revision": 1,
+  "objectType": "assertions",
+  "contentType": "confirmations",
+  "dataCategory": "knowledge",
+  "concept": {
+    "title": "Confirmation for ASSERTION-99",
+    "identifiers": {
+      "primary": { "kind": "assertionId", "value": "ASSERTION-99", "issuer": "knowx-pipeline" }
+    }
+  },
+  "data": {
+    "confirmationType": "human-review",
+    "decision": "accept",
+    "confirmedBy": "ops-user-1",
+    "note": "Matches work-item assignee in snapshots"
+  }
+}
+```
+
+After accept, pipeline issues a **revision-safe** patch on `assertions-knowx`: `epistemic.state: "fact"`, `confirmation: "human"`.
+
+---
+
+## From: Building KnowX — 10. Full runnable fixture (no live AI)
+
+This fixture is enough to accept KnowX **extraction + draft shapes** on an isolated org without model credentials. It is implemented in `@x12i/memorix-ops-pack` and covered by package tests (`knowx-fixture.test.ts`, `index.test.ts`).
+
+### 10.1 What is shipped vs documented
+
+| Slice | Status |
+|---|---|
+| Metadata: `knowx` CT, `assertions` OT, `KnowXEntity` abstract + bindings | **Shipped** in ops-pack |
+| `fixtures/knowx-activity-log.jsonl` | **Shipped** |
+| `extractKnowxFromActivityLog` / service id `knowx-ops-fixture-v1` | **Shipped** + unit tested |
+| Pack pipeline `knowx-ops-e2e` (`analyze` → `write-record`) | **Shipped** (pack authoring) |
+| Live AI extractor | **Not shipped** — do not require for acceptance |
+| Studio NODE/EDGE screenshots | May still be placeholders — see shot list |
+| Human accept → relationship materialize | Product policy (same ports as other content); run after drafts are written |
+
+### 10.2 Scope
+
+```text
+orgId:     g12opsknowx1          # example; generate via isolation helpers
+agentIds:  ["ops"]
+```
+
+### 10.3 Source document (land as raw)
+
+Path: `memorix-ops-pack/fixtures/knowx-activity-log.jsonl` (exact bytes hashed → `docHash`):
+
+```text
+{"line":12,"actorId":"employee-10","workItemId":"wi-100","verb":"assigned"}
+```
+
+```ts
+import { readFileSync } from "node:fs";
+import {
+  extractKnowxFromActivityLog,
+  sha256Hex,
+} from "@x12i/memorix-ops-pack";
+
+const bytes = readFileSync("…/fixtures/knowx-activity-log.jsonl");
+const { docHash, drafts } = extractKnowxFromActivityLog(bytes);
+// docHash === sha256Hex(bytes)
+```
+
+### 10.4 Deterministic extractor service
+
+Pack `services[]` + runtime bridge:
+
+```json
+{
+  "id": "knowx-ops-fixture-v1",
+  "capabilities": ["analyze"],
+  "invocation": { "kind": "worker", "operation": "knowx-extract-fixture" }
+}
+```
+
+```json
+{
+  "serviceId": "knowx-ops-fixture-v1",
+  "kind": "worker",
+  "capabilities": ["analyze"]
+}
+```
+
+Behavior (implemented):
+
+1. Read exact source bytes.
+2. Parse the fixture line (`line`, `actorId`, `workItemId`, `verb`).
+3. Emit NODE drafts for `employee-10` / `wi-100`.
+4. Emit EDGE draft `ASSERTION-99` with mandatory provenance (`span.lineStart/lineEnd`, `docHash`, extraction confidence `0.85`).
+5. Never call an external model.
+
+Wire the function into an `analyze` `PortHandler` (or call it from a worker that uses `@x12i/core-service`). Pipeline `write-record` persists drafts — the extractor itself does not open Mongo.
+
+### 10.5 Pipeline definition
+
+Pack authoring (`knowx-ops-e2e`):
+
+```json
+{
+  "id": "knowx-ops-e2e",
+  "steps": [
+    { "id": "extract", "kind": "analyze", "serviceId": "knowx-ops-fixture-v1" },
+    { "id": "write-knowx", "kind": "write-record", "contentType": "knowx" }
+  ]
+}
+```
+
+Runtime v2 shape (after bind/compile):
+
+```json
+{
+  "pipelineId": "knowx-ops-e2e",
+  "version": "1",
+  "services": [
+    { "serviceId": "knowx-ops-fixture-v1", "kind": "worker", "capabilities": ["analyze"] }
+  ],
+  "steps": [
+    {
+      "stepId": "extract",
+      "stepKind": "analyze",
+      "serviceId": "knowx-ops-fixture-v1",
+      "outputKey": "knowxDrafts"
+    },
+    {
+      "stepId": "write-knowx",
+      "stepKind": "write-record",
+      "dependsOn": ["extract"],
+      "portConfig": { "contentType": "knowx" },
+      "outputKey": "knowxWrites"
+    }
+  ]
+}
+```
+
+Optional Memory pull / `persist-raw` steps can land the JSONL first; the fixture extractor only needs the exact bytes (file or raw DataRef).
+
+### 10.6 Acceptance checklist (runnable)
+
+```bash
+# From memorix-ops-pack:
+npm test
+# covers: metadata KnowXEntity/assertions/knowx, fixture extract, docHash, epistemic ≠ extraction confidence
+```
+
+HTTP smoke after install (when service + live ports are up):
+
+```bash
+# Install ops-pack into fresh g12ops* + agentIds=["ops"]
+# Start pipeline knowx-ops-e2e (or write drafts via data API from extractKnowxFromActivityLog output)
+
+curl -sS "${HDR[@]}" \
+  "$SERVICE/api/data/content?objectType=work-items&contentType=knowx&id=wi-100"
+
+curl -sS "${HDR[@]}" \
+  "$SERVICE/api/data/content?objectType=assertions&contentType=knowx&id=ASSERTION-99"
+
+curl -sS "${HDR[@]}" \
+  "$SERVICE/api/data/abstract?abstractType=KnowXEntity&id=ASSERTION-99"
+```
+
+Verify:
+
+- [ ] `epistemic.state` and `provenance.extraction.confidence` both present and distinct
+- [ ] `docHash` matches `sha256` of exact fixture bytes
+- [ ] `KnowXEntity` read works; writing to `KnowXEntity` as objectType fails
+- [ ] EDGE starts as `hypothesis` until human confirmations + revision-safe patch to `fact`
+- [ ] After accept, materialize four-field links (relationship engine) — do not copy knowx JSON onto neighbors
+- [ ] `format === "memorix-record/2"` on all KnowX rows
+- [ ] No `associated*` keys on any content root
+- [ ] Re-run extract does not invent duplicate EDGE ids (`ASSERTION-99` stable)
+- [ ] No live AI credentials required
+- [ ] Org matches `g12ops*`; agents `["ops"]`
+
+---
+
+## Also see
+
+- **Relationships & Memory** (`06-relationships-and-memory`) — Exact Memory ingest and relationship engine links — never associated* payloads on records.
+- **Building Pipelines** (`08-pipelines`) — Pipeline definition, step kinds, AI fail-closed, operations, and the complete ops tutorial.
+
+---
+
+_Generated use-case pack for agents and humans. See `agent-manifest.json` for discovery._