@x12i/memorix-docs 1.0.2 → 1.1.0

package/files/use-cases/author-pipeline-e2e.md

@@ -0,0 +1,339 @@
+# Author and run a pipeline end-to-end
+
+> **Use case id:** `author-pipeline-e2e`
+> **Goal:** Define steps, execute under g12ops*, and observe runs/artifacts.
+> **Audiences:** developers
+> **Tags:** pipelines, ops
+
+## Reading path
+
+1. **Building Pipelines** (developers) → chapters: `1-pipeline-definition`, `2-execution-model`, `4-step-types-one-by-one`, `13-complete-ops-tutorial`
+
+## From: Building Pipelines — 1. Pipeline definition
+
+### Identity
+
+- Stable pipeline metadata ID (`pipelineId`) belongs to an agent (e.g. `ops`).
+- Inheritance and child replacement follow multi-agent metadata rules.
+- Pipeline ID is **not** a run ID.
+- Metadata does not hold runtime state; fingerprints and build evidence live on the run.
+- Do not invent unsupported metadata-version machinery beyond what the runtime captures.
+
+### Scope
+
+Every run uses:
+
+```ts
+{
+  orgId: string;      // e.g. g12ops7a3c — complete data universe
+  agentIds: string[]; // e.g. ["ops"] — effective metadata only
+}
+```
+
+Scope is captured on the run. Every step receives the same canonical scope. Steps must not reinterpret scope. `agentIds` does not filter organization data.
+
+```http
+x-memorix-org-id: g12ops7a3c
+x-memorix-agent-ids: ops
+```
+
+### Definition fields (runtime)
+
+Core runtime shape (`PipelineDefinition` in `@x12i/memorix-pipeline`):
+
+```ts
+type PipelineDefinition = {
+  pipelineId: string;
+  version: string;
+  description?: string;
+  steps: StepDefinition[];
+  services?: ServiceRef[];
+  schedules?: ScheduleDefinition[];
+  retryPolicy?: RetryPolicy;
+  dataRefThresholdBytes?: number; // default 1_048_576
+  metadata?: Record<string, unknown>;
+};
+
+type StepDefinition = {
+  stepId: string;
+  stepKind: StepKind;
+  label?: string;
+  dependsOn?: string[];
+  inputs?: DataRefExpr[];
+  outputKey?: string;
+  retryPolicy?: RetryPolicy;
+  idempotencyKeyExpr?: string; // required for mutating steps
+  port?: string;
+  portConfig?: unknown;
+  serviceId?: string;
+  batchSize?: number;
+  atomicBatch?: boolean; // opt-in; rejected unless adapter supports
+  timeoutMs?: number;
+  disabled?: boolean;
+  inputSchema?: unknown;
+  outputSchema?: unknown;
+};
+```
+
+Agent-pack metadata may use a lighter authoring shape (`id`, `name`, `trigger`, `mode`, `steps[]` with `kind` / `serviceId` / `mappingId`). The runtime compiles or binds that into the executable definition. Always validate packs with `@x12i/memorix-metadata` before install.
+
+### Inputs and outputs
+
+**Inputs:** small inline JSON, DataRef/artifact, record/content address, list/query, abstract reference, relationship reference, Memory source/pull reference, captured context, batch manifest, workflow reference, external trigger payload.
+
+**Outputs:** inline result, DataRef, record address/revision, relationship run result, partial batch manifest, external action receipt, workflow/analysis/decision/narrative references, diagnostics.
+
+Prefer references to stored results over copying large payloads into the run ledger. Default DataRef threshold: **1 MiB**.
+
+### DAG and dependencies
+
+Rules:
+
+- Unique `stepId`s
+- `dependsOn` edges; topological execution
+- Fan-out / fan-in and parallelism for independent branches
+- Cycle rejection; unreachable steps rejected at validate
+- Output-to-input via `inputs` DataRef expressions (`fromStep`, `key`, `path`)
+- Disabled steps skipped
+
+Loops are not a graph feature. Repeated work uses batches, retries, schedules, or an explicit redesign — not hidden cycles.
+
+### Control flow (supported)
+
+Sequential execution · parallel independent branches · per-item batch work · short-circuit on terminal failure · continue-on-item-failure for batches · optional/disabled steps · cancellation.
+
+**Unsupported as global features:** cross-service distributed transactions, hidden graph cycles, silent compensation without an explicit step. Atomic batch is **opt-in** and only when the backing port declares support.
+
+---
+
+## From: Building Pipelines — 2. Execution model
+
+### Run, step, attempt
+
+```text
+Pipeline definition
+  → Run          (one execution)
+      → Step     (one node in the definition)
+          → Attempt  (one try)
+```
+
+Retries create **attempts**; they do not duplicate logical steps.
+
+### Run statuses
+
+| Status | Meaning | Terminal? |
+|---|---|---|
+| `pending` | Queued | No |
+| `running` | Active | No |
+| `succeeded` | All required work ok | Yes |
+| `failed` | Terminal failure | Yes |
+| `partial` | Explicit partial success (e.g. batch) | Yes |
+| `cancelled` | Operator/runtime cancel | Yes |
+
+### Step statuses
+
+`pending` · `running` · `succeeded` · `failed` · `skipped`
+
+### Durable production runtime
+
+Production requires durable run ledger, durable artifact store, real canonical ports, real service bindings, and explicit production mode.
+
+Production rejects: missing ledger, missing artifact store, required missing ports, stub services without opt-in, in-memory operational fallback presented as prod, unknown step kinds, invalid definitions.
+
+Development stubs require explicit opt-in.
+
+### At-least-once execution
+
+A mutating step may be attempted again. Services and mutation ports must be idempotent. A successful external effect may occur before the runtime sees the response — retries must not create duplicate records, links, or external actions.
+
+### Idempotency
+
+| Concern | Mechanism |
+|---|---|
+| Memory pulls | Cursor + content-hash dedup |
+| Raw persistence | Content hash / absent-or-revision |
+| Record writes | Precondition + idempotency key |
+| Relationship writes | Deterministic `relationshipId` |
+| Mapping | Pure replay |
+| Analysis / decision / narrative writes | Discriminator + revision / key |
+| External actions | Mandatory key + receipt reuse |
+| AI jobs | Captured input; one accepted mutation |
+| Batch items | Per-item status (`already_applied`) |
+| Artifacts | Content hash |
+
+Mutating step kinds include: `persist-raw`, `write-record`, `relationship-write`, `workflow`, `narrative`. These require `idempotencyKeyExpr`.
+
+### Retry
+
+Default policy: `maxAttempts: 3`, `backoffMs: 1000`, `backoffMultiplier: 2`, `maxBackoffMs: 60000`.
+
+Classify errors as retryable vs terminal. Support automatic retry, manual retry of one failed step, and retry of failed batch items only. Each attempt is recorded.
+
+### Resume
+
+If the runtime process stops, the durable ledger determines completed steps. Succeeded upstream work is not rerun unnecessarily. Failed/incomplete work resumes with captured context and stable artifact references.
+
+### Partial success
+
+Examples: record write succeeded + relationship write failed; valid batch items + invalid items. Per-item ledger; retry failed portion only; final run status may be `partial`; artifact holds the partial manifest.
+
+### Atomicity
+
+Default: **no** global pipeline transaction. Successful prior steps remain successful. Atomic batch is opt-in only when supported — otherwise rejected before execution. Do not claim cross-service global atomicity.
+
+### Cancellation
+
+Cancel queued work or active attempts. Respect remote-service limits. If an external side effect may already have occurred, record uncertainty honestly. Retain artifacts per retention policy.
+
+---
+
+## From: Building Pipelines — 4. Step types, one by one
+
+Registered `StepKind` values:
+
+`memory-pull` · `persist-raw` · `map-normalize` · `validate` · `identity-dedupe` · `write-record` · `relationship-discover` · `relationship-write` · `compose` · `verify` · `workflow` · `enrich` · `analyze` · `decide` · `narrative`
+
+Behavioral steps may reference `serviceId` for custom/AI work. Generic invoke is expressed through these kinds (and pack-level `invoke-service` where metadata uses that authoring alias) — not a second runtime.
+
+### 4.1 `memory-pull`
+
+**Purpose:** Invoke one configured Memory source.
+
+Cover: scheduled/on-demand, knowledge vs context mode, `sourceId`, workRef, cursor, pull/run reference, quarantine, context expiry, dedup, exact raw payload. Credentials via `credentialRef` only.
+
+Memory owns raw landing end-to-end for the pull; `persist-raw` is used when the graph needs an explicit handoff step in front of later ports — do not document two competing “ways to land raw.”
+
+**Ops:** Pull procedures from `ops-source` into `g12ops*`; land raw for **PROC-2**.
+
+### 4.2 `persist-raw`
+
+**Purpose:** Persist normalized Memory handoff as `<objectType>-raw` with exact provider `.data`, provenance, absent/revision behavior, dedup hash, directory registration — when the plan separates pull from persist.
+
+**Anti-pattern:** Decorating raw `.data` or promoting to snapshot inside this step.
+
+### 4.3 `map-normalize`
+
+**Purpose:** Execute the shared mapping AST (`@x12i/memorix-mapping`).
+
+Source mapping, registered transforms, no arbitrary JavaScript, no I/O, deterministic replay. Mapping errors are structured. Prefer this over inventing a service for renames/shapes.
+
+### 4.4 `validate`
+
+**Purpose:** Validate data/drafts/contracts.
+
+Distinguish V2 format validation, metadata validation, mapping output validation, service output validation, and domain validation services. Output structured errors — not message-only failures.
+
+### 4.5 `identity-dedupe`
+
+**Purpose:** Establish or resolve identity safely.
+
+`recordId`, external identifiers, record directory, Memory content hash, duplicate identity conflict, semantic duplicate *candidates*. No silent merge. No invented record IDs.
+
+### 4.6 `write-record`
+
+**Purpose:** Canonical writer — `add` | `upsert` | `patch` | `replace` | `remove`.
+
+Absent/revision preconditions, discriminator addressing, lifecycle, stale conflicts, directory behavior, idempotency, partial batch writes. This is how drafts become stored subject content.
+
+### 4.7 `relationship-discover`
+
+**Purpose:** Compile/evaluate relationship metadata and calculate expected candidates.
+
+Abstract endpoint unions, deterministic operators, registered resolvers, provider raw references, bounded discovery plans, dry-run, diagnostics. Discovery-only does not mutate links.
+
+### 4.8 `relationship-write`
+
+**Purpose:** Materialize candidates through the relationship engine.
+
+Deterministic `relationshipId`, four-field link, idempotent writes, category routing, stale-link removal where required. No copied payload. No manual arbitrary links. Partial failure/retry supported.
+
+### 4.9 `compose`
+
+**Purpose:** Retrieve selected sibling content for one `recordId`.
+
+Selected content types, arrays per type, discriminated instances. No flattening. No relationship definition. Prefer captured composition for retry-sensitive downstream AI/analysis.
+
+### 4.10 `enrich`
+
+**Purpose:** Deterministic or service-based enrichment.
+
+Preferred flow: capture → service/mapping → validate → `write-record`. Do not hide uncontrolled persistence inside enrich.
+
+**Ops:** Enrich **WI-100** priority via `ops-completion` / enrich service.
+
+### 4.11 `analyze`
+
+**Purpose:** Produce analysis content/artifacts.
+
+Captured context, deterministic or AI service, required `analysisType` discriminator, output validation, DataRef, sibling V2 persistence through writer, repeated-run idempotency.
+
+### 4.12 `decide`
+
+**Purpose:** Produce decision/recommendation content with `decisionType`.
+
+Consume analysis inputs; keep rationale as domain content where appropriate; operational provenance separate. A decision does **not** perform an external action — that is a later action service step.
+
+### 4.13 `narrative`
+
+**Purpose:** Produce narrative content or artifact.
+
+Current/captured context, relationship expansion, deterministic/AI service, validation, content versus artifact. No legacy narrative mutation path. Mutating when it persists — requires idempotency key.
+
+### 4.14 `workflow`
+
+**Purpose:** Read/update workflow sibling content or invoke workflow actions.
+
+Exact revision, composition, workflow action service, state transition, retries. No separate workflow persistence model.
+
+### 4.15 `verify`
+
+**Purpose:** Explicit verification — relationship verify, data/directory checks, output postconditions — selected by configuration. Do not use one vague verify to hide unrelated behaviors.
+
+### External actions and intelligence
+
+External actions are **service invocations** (action role) from enrich/workflow/generic service steps with declared side effects — not a separate hidden step kind unless your pack aliases one. Structural intelligence is a service/pipeline pattern: deterministic profiling first, proposal artifacts, optional AI, **no** metadata auto-install, **no** data/link mutation inside the proposal step.
+
+---
+
+## From: Building Pipelines — 13. Complete ops tutorial
+
+End-to-end non-cyber drill. Use a **fresh** `g12ops*` org every acceptance run.
+
+1. **Define procedure source** in ops metadata (`ops-source`).
+2. **Define connector** (Memory collect; `credentialRef` only).
+3. **Define knowledge-ingestion pipeline** (`memory-pull` → raw → map → validate → write → relationships).
+4. **Pull procedures**; confirm raw for **PROC-2**.
+5. **Inspect raw** in Explore / data API (`procedures-raw`).
+6. **Promote** selected procedure snapshot via map/validate/`write-record` — never automatic.
+7. **Pull work-item context** for **WI-100** (context mode as configured).
+8. **Normalize** employee/contractor data (mapping and/or normalize service).
+9. **List People** abstract (employees + contractors including **Alex**).
+10. **Materialize assignments** (relationship discover + write).
+11. **Create work-item ↔ procedure relationship** (WI-100 ↔ PROC-2).
+12. **Run enrichment** (priority completion / enrich service).
+13. **Produce analysis** (`analyze` + `analysisType`).
+14. **Produce decision** (`decide` + `decisionType`).
+15. **Generate narrative** (recorded AI adapter acceptable).
+16. **Update workflow** sibling with exact revision.
+17. **Trigger controlled relationship failure** (e.g. break a definition temporarily).
+18. **Retry relationship step only**; confirm records not duplicated.
+19. **Restart runtime** during another run.
+20. **Resume** without repeating completed work.
+21. **Inspect captured context** artifacts vs current data.
+22. **Inspect artifacts** (hashes, sizes, step association).
+23. **Inspect final records/links** in Explore — four-field links, no copied payloads.
+24. **Run scheduled** invocation; confirm schedule history.
+25. **Run on-demand** invocation with explicit input; confirm distinct `runId`.
+
+Tutorial companion narrative: [documentation/tutorials/04-memory-and-pipeline.md](../../../documentation/tutorials/04-memory-and-pipeline.md).
+
+---
+
+## Also see
+
+- **Building Services** (`07-services`) — When to build a service, handlers, pipeline invoke, and core-service / ports-manager / api-live-view.
+- **Agent Packs & Ops** (`11-agent-packs-and-ops`) — Pack layout, harness, g12ops* acceptance, custom kinds, golden V2 validation.
+
+---
+
+_Generated use-case pack for agents and humans. See `agent-manifest.json` for discovery._

package/files/use-cases/build-callable-service.md

@@ -0,0 +1,347 @@
+# Build a service Memorix pipelines invoke
+
+> **Use case id:** `build-callable-service`
+> **Goal:** Decide you need a service, define it, implement handlers, bind, and invoke from a step.
+> **Audiences:** developers
+> **Tags:** services, pipelines
+
+## Reading path
+
+1. **Building Services** (developers) → chapters: `1-do-i-need-a-service`, `2-one-model-roles-catalog`, `3-define-the-service`, `4-implement-the-handler`, `7-invoke-from-a-pipeline`
+
+## From: Building Services — 1. Do I need a service?
+
+### Decision guide
+
+| Prefer | When |
+|---|---|
+| **Mapping** (`@x12i/memorix-mapping`) | Pure deterministic field transform; no I/O; expressible in the shared AST |
+| **Memory** (`@x12i/memorix-memory`) | External source material must land exact raw first (`*-raw`) |
+| **Relationship metadata** | Connecting different `recordId`s with deterministic rules/operators |
+| **Composition** | Same `recordId`, different content types (siblings) |
+| **Pipeline** | Ordered, scheduled, retryable, multi-step work with run evidence |
+| **Service** | Custom code, remote I/O, specialized compute, reusable external action, or model execution |
+
+### Decision tree
+
+```text
+Is it a pure deterministic field transformation?
+├─ Yes → mapping (@x12i/memorix-mapping)
+└─ No
+
+Is it external source material that must land raw?
+├─ Yes → Memory source + connector (collect role)
+└─ No
+
+Is it connecting different recordIds?
+├─ Expressible by metadata rules → relationship engine
+├─ Needs custom resolution → resolver service + relationship engine
+└─ Same recordId → composition (siblings), not a relationship service
+
+Is it several ordered / retryable operations?
+├─ Yes → pipeline (may invoke services as steps)
+└─ No
+
+Does it require custom computation, I/O, or AI?
+└─ Service
+```
+
+See also [diagrams/decision-tree.md](../diagrams/decision-tree.md).
+
+### Things that are not services
+
+Object types, content types, mappings, abstract bindings, relationship definitions, schedules, pipelines, workflow content, the record writer/reader, the relationship store, the artifact store, and the metadata catalog are **platform surfaces**. A service may *use* them through canonical contracts; it must not reimplement them.
+
+---
+
+## From: Building Services — 2. One model — roles catalog
+
+Service “types” are **roles** (capabilities) on one model. A single service declares one or more capabilities; pipelines and operators still see one `serviceId`.
+
+| Role | Capability examples | Purpose |
+|---|---|---|
+| **Collect** | `collect`, `connector`, `source-pull` | Read material from an external source for Memory |
+| **Enrich** | `enrich`, `normalize`, `completion`, `lookup` | Compute or fetch additional information for an existing subject |
+| **Resolve** | `resolve`, `identity`, `dedupe`, `relationship-candidates` | Produce identity or relationship *candidates* — engine writes links |
+| **Action** | `action`, `notify`, `external-command`, `workflow-action` | Deliberate external side effect with idempotency |
+| **AI job** | `ai`, `extract`, `classify`, `summarize`, `narrative`, `analyze` | Model call behind the single AI adapter |
+
+Ops-pack examples use capabilities such as `normalize` and `completion` on services `ops-normalize` and `ops-completion` — same model, different roles.
+
+### Side-effect classifications
+
+| Classification | Retry safe? | Idempotency key | Typical evidence |
+|---|---|---|---|
+| Pure / read-only | Yes | Optional | Input/output hashes |
+| External read | Usually yes | Recommended | Source receipt, cursor |
+| Deterministic computation | Yes | Optional | Output artifact |
+| Platform draft producer | Yes (draft only) | Recommended | Draft + validation result |
+| External action | Only with key | **Mandatory** | External receipt |
+| AI / model call | Re-callable; output may differ | Capture input; one accepted write | Request/response artifacts |
+
+---
+
+## From: Building Services — 3. Define the service
+
+### Metadata shape
+
+Exact logical contract (`ServiceDefinition` in `@x12i/memorix-metadata`):
+
+```ts
+type ServiceIoPort = {
+  id: string;
+  schemaRef?: string;   // logical schema reference — not a runtime payload
+  description?: string;
+};
+
+type ServiceInvocation = {
+  kind: "http" | "worker" | "internal";
+  operation?: string;
+};
+
+type ServiceDefinition = {
+  id: string;
+  name: string;
+  description?: string;
+  capabilities: string[];
+  inputs?: ServiceIoPort[];
+  outputs?: ServiceIoPort[];
+  invocation?: ServiceInvocation;
+};
+```
+
+Example — enrich WI-100 with an external priority scorer:
+
+```json
+{
+  "id": "ops-wi-priority-enrich",
+  "name": "Work-item priority enricher",
+  "description": "Returns a patch plan for work-item snapshot priority from an external scorer",
+  "capabilities": ["enrich", "lookup"],
+  "inputs": [
+    { "id": "snapshot", "schemaRef": "memorix-record/snapshots" }
+  ],
+  "outputs": [
+    { "id": "patch-plan", "schemaRef": "memorix-patch-plan/1" }
+  ],
+  "invocation": { "kind": "worker", "operation": "enrichPriority" }
+}
+```
+
+### Identity rules
+
+- Stable `id` is the logical name pipelines reference (`serviceId`).
+- Agent ownership and inheritance follow the same multi-agent resolution as other kinds.
+- Child agents may replace a service definition; unrelated agents must not silently collide.
+- Service ID is **not** an endpoint URL.
+- Metadata has no mutable runtime state; build/version belongs in run evidence.
+
+### Input contract
+
+Prefer references over copying large records into pipeline definitions or ledgers:
+
+| Form | Use when |
+|---|---|
+| Small inline JSON | Tiny control values |
+| DataRef / artifact | Large payloads, batches, model responses |
+| Concrete record reference | Address by `(objectType, contentType, recordId)` + revision |
+| Abstract projection reference | Read through virtual union (People), write concrete later |
+| Captured context reference | Exact revision frozen for the run |
+| Source / pull reference | Memory work |
+| Relationship definition reference | Resolver / discovery helpers |
+| Batch reference | Per-item manifests |
+
+Every input is schema-validated. Prefer captured revisions for retry-sensitive work.
+
+### Output contract
+
+| Form | Meaning |
+|---|---|
+| Inline result | Small structured result |
+| Artifact / DataRef | Large or binary output |
+| Record draft / patch plan | Candidate content — **not** yet stored |
+| Validation / mapping result | Structured issues |
+| Relationship candidate result | Candidates for the relationship engine |
+| Analysis / decision / narrative draft | Sibling content drafts with discriminators |
+| External-action receipt | Proof of side effect |
+| Diagnostics / metrics | Operational only |
+| Retryable / non-retryable error | Machine-readable codes |
+
+A service output is **not** automatically persisted as subject content. The pipeline validates and persists through canonical ports.
+
+Forbidden in metadata: secret values, deployment URLs as identity, fake “always succeed” stubs presented as production contracts.
+
+---
+
+## From: Building Services — 4. Implement the handler
+
+### Lifecycle
+
+1. Identify the capability.
+2. Confirm it should be a service (decision guide).
+3. Select roles / capabilities.
+4. Define input/output schemas in pack metadata (`ServiceDefinition`).
+5. Declare side effects and errors.
+6. Implement a **`PortHandler`** for the canonical pipeline **`StepKind`** that will run the work (usually `enrich` / `analyze` / `decide` / `narrative`).
+7. Register a runtime **`ServiceRef`** (`serviceId` + `kind`) — from pack bridge, pipeline `services[]`, or env defaults.
+8. Add service metadata to the agent pack (`ops`) as `services[]` with pack field `id`.
+9. Reference it from a pipeline step: `stepKind` selects the handler; optional `serviceId` must resolve in the registry.
+10. Validate / install metadata.
+11. Run conformance tests.
+12. Deploy; execute through the pipeline; observe runs / attempts / artifacts.
+
+### Real invocation contract (pipeline v2)
+
+Handlers are **not** free-standing “service handlers.” They are **step-kind port handlers**:
+
+```ts
+type StepKind =
+  | "memory-pull" | "persist-raw" | "map-normalize" | "validate"
+  | "identity-dedupe" | "write-record" | "relationship-discover"
+  | "relationship-write" | "compose" | "verify" | "workflow"
+  | "enrich" | "analyze" | "decide" | "narrative";
+
+type StepExecutionRequest = {
+  scope: { orgId: string; agentIds: string[] };
+  runId: string;
+  stepId: string;
+  attempt: number;
+  stepType: StepKind;
+  idempotencyKey?: string;
+  deadline?: string;
+  inputs: Record<string, unknown>;
+  capturedContextRef?: /* DataRef */;
+  config?: unknown;           // from step.portConfig
+};
+
+type StepExecutionResult = {
+  status: "succeeded" | "failed" | "partial";
+  output?: unknown;
+  itemOutcomes?: /* BatchItemOutcome[] */;
+  diagnostics?: Record<string, unknown>;
+  mutationRefs?: unknown[];
+  retryable?: boolean;        // failed + retryable ≈ safe to attempt again
+  error?: { code: string; message: string };
+};
+
+type PortHandler = (req: StepExecutionRequest) => Promise<StepExecutionResult>;
+```
+
+Package: `@x12i/memorix-pipeline` (`createLivePortRegistry`, `createPortInvoker`, `createServiceRegistry`).
+
+**Critical:** `serviceId` on a step does **not** select which function runs. Execution is always:
+
+```text
+step.stepKind → PortRegistry[stepKind] → PortHandler(req)
+```
+
+If `step.serviceId` is set, the runtime **resolves** that id in the service registry (fail closed if missing), then still invokes the **stepKind** port.
+
+Do not invent private Mongo handles. Use the adapters the live port registry already wires (`@x12i/memorix-data`, relationships, Memory).
+
+### Result contract
+
+Every result is machine-readable — callers must not parse free-text messages:
+
+| `status` | Meaning |
+|---|---|
+| `succeeded` | Full success |
+| `partial` | Batch: some items ok (`itemOutcomes`) |
+| `failed` | Failure — set `retryable: true` when safe to attempt again; omit or `false` when terminal |
+
+Timeouts, cancellations, and skips are expressed as `failed` (+ diagnostics / error codes) or by the pipeline runtime — there is no separate port-status enum for `timed_out` / `cancelled` / `skipped`.
+
+### Pack definition versus runtime binding
+
+| Layer | Shape | Role |
+|---|---|---|
+| Pack metadata | `ServiceDefinition`: `{ id, name, capabilities, inputs?, outputs?, invocation? }` | Declarative catalog (`services[]` in agent pack) |
+| Runtime registry | `ServiceRef`: `{ serviceId, kind, endpoint?, credentialRef?, … }` | Deploy binding |
+| Bridge | `servicesFromMetadata` | Maps pack `id` → `serviceId`, `invocation.kind` → `kind` (default `internal`) |
+
+```ts
+// Pack (ops-pack metadata/services.json)
+{ id: "ops-enrich-priority", capabilities: ["enrich"], invocation: { kind: "worker" } }
+
+// Runtime after bridge
+{ serviceId: "ops-enrich-priority", kind: "worker", capabilities: ["enrich"] }
+```
+
+### In-process versus remote
+
+| | In-process | Remote |
+|---|---|---|
+| Use when | Trusted, low-latency, deterministic platform code | Separate process, vendor API, isolation |
+| Binding | Live `PortHandler` on the step kind + `ServiceRef` | `ServiceRef.endpoint` + HTTP worker process |
+| Credentials | `credentialRef` → runtime vault | Same — never in metadata |
+| Large I/O | DataRef / artifacts | Same |
+| Failure | Process-local errors → attempt ledger | Unavailable → `failed` + `retryable`; missing binding → fail closed in prod |
+
+Helper `createServiceInvokePorts().invokeRemote` POSTs `JSON.stringify(req.inputs)` to `endpoint` and expects HTTP 200 + JSON. It is **not** a first-class `StepKind` — wire it from a custom `enrich`/`analyze`/… port when you need remote dispatch. Prefer `@x12i/core-service` on the worker process (see HTTP process compliance below).
+
+Production rejects missing bindings and silent stub handlers. Development stubs require **explicit** opt-in (`mode: "dev"` + `allowStubPorts: true`) and clear labeling.
+
+---
+
+## From: Building Services — 7. Invoke from a pipeline
+
+Preferred split:
+
+```text
+read / capture
+  → StepKind port (enrich / analyze / …)  [optional serviceId resolve]
+  → validate result
+  → write-record or relationship-write
+```
+
+Avoid a service that hides computation and persistence in one opaque call. External-action services are the exception — the side effect is their declared purpose.
+
+### Adding a service to a pipeline (ops recipe)
+
+1. Add `services[]` entry to the agent pack (`id`, `capabilities`, `invocation`) — see `memorix-ops-pack/metadata/services.json` (`ops-enrich-priority`).
+2. Install / resolve effective metadata for `g12ops*` + `agentIds: ["ops"]`.
+3. Bridge pack services → `ServiceRef`s (`servicesFromMetadata`) when constructing the pipeline runtime.
+4. Implement or override the **`enrich`** (or other) live port so the handler returns a **draft / patch plan**, not a silent Mongo write.
+5. Author a v2 pipeline step:
+   ```ts
+   {
+     stepId: "enrich",
+     stepKind: "enrich",          // selects PortHandler
+     serviceId: "ops-enrich-priority", // must resolve in registry
+     portConfig: { /* optional */ },
+   }
+   ```
+   Pack authoring may use the same `serviceId` with kind `enrich` (see `ops-enrich-priority` pipeline in ops-pack). Pack kinds are not always identical to v2 `StepKind` — the platform compile/bind path must produce a valid v2 definition.
+6. Add a separate mutation step (`write-record` / `relationship-write`) that applies the draft with revision preconditions.
+7. Start the run; inspect `/api/pipelines` attempts and artifacts (or Studio Operate).
+
+There is **no** v2 `StepKind` named `invoke-service`. Older pack sketches that used `kind: "invoke-service"` are not executable until rewritten to a canonical step kind.
+
+### Partial batches
+
+- Per-item input and per-item success/failure.
+- Retry only failed items.
+- No false global atomicity unless a port explicitly declares atomic batch support.
+- Output artifact holds the partial manifest.
+
+### Retry matrix (summary)
+
+| Kind | Repeat same invocation? |
+|---|---|
+| Pure calculation | Yes |
+| External read | Yes (watch rate limits / cursors) |
+| Enrichment draft | Yes; one accepted write |
+| AI call | Yes; outputs may differ; capture input; one accepted mutation |
+| External action | Only with idempotency key + receipt reuse |
+| Relationship candidates | Yes; engine write remains idempotent |
+| Artifact processing | Yes if content-addressed |
+
+---
+
+## Also see
+
+- **Building Pipelines** (`08-pipelines`) — Pipeline definition, step kinds, AI fail-closed, operations, and the complete ops tutorial.
+
+---
+
+_Generated use-case pack for agents and humans. See `agent-manifest.json` for discovery._