@aionis/substrate 0.1.0

package/docs/RUNTIME_MAPPING.md

@@ -0,0 +1,69 @@
+# Aionis Runtime to Substrate Mapping
+
+This document maps current Aionis Runtime product capabilities onto the Substrate contract. It is a design map only. It does not imply Runtime is already using Substrate.
+
+## Product Loop Mapping
+
+| Runtime surface | Substrate primitive | Notes |
+|---|---|---|
+| `observe` | `memory.node.upsert`, `memory.relation.upsert`, `memory.feedback.recorded` | Runtime observations become durable evidence events. Execution memory, ordinary memory, feedback, and trace pointers are all nodes or events. |
+| `guide` | `compileContext` | Runtime guide is a governed context compile: `use_now`, `inspect_before_use`, `do_not_use`, and `rehydrate`. |
+| `feedback` | `memory.feedback.recorded`, optional `memory.lifecycle.transition` | Feedback is evidence first. Authority changes require substrate-visible transitions. |
+| `forget` | `memory.lifecycle.transition`, `requires_payload` relation | Forgetting is suppress/archive/rehydrate/revalidate state, not physical deletion. |
+| `measure` | `memory.decision.recorded`, feedback and transition events | Measure reads decision traces and evidence history. It does not silently mutate memory. |
+
+## Runtime Data Mapping
+
+| Current Runtime concept | Substrate representation |
+|---|---|
+| Execution memory | `AionisMemoryNode(kind="execution")` |
+| Workflow/procedure memory | `AionisMemoryNode(kind="procedure")` |
+| Ordinary facts/preferences | `kind="fact"` / `kind="preference"` |
+| Claim ledger item | `kind="claim"` |
+| Handoff or trace pointer | `kind="trace_pointer"` with `payloadRef` |
+| Lifecycle relation | `AionisRelation(kind="supersedes" | "contradicts" | "invalidates")` |
+| Rehydrate hook | `lifecycle="rehydrate_required"` or `requires_payload` relation |
+| Suppression/archive | `memory.lifecycle.transition` |
+| Decision trace | `memory.decision.recorded` |
+| Outcome attribution | `memory.feedback.recorded` |
+
+Runtime product snapshot imports also preserve two agent-facing boundaries:
+
+- execution outcome contracts can override weak textual trust. A succeeded
+  execution observation such as `passed_solution` becomes active/trusted
+  substrate memory, while a failed branch or failed verifier becomes
+  blocked/rejected memory.
+- audit-only nodes marked `not_agent_facing` are skipped by the agent-facing
+  snapshot importer. Guide exposure ledgers and measure receipts remain Runtime
+  audit evidence; they must not become rehydrate hooks in compiled Agent context.
+
+## Admission Boundary
+
+Runtime currently owns the richer product policy. Substrate owns the minimum storage-level contract:
+
+- authoritative active memory may be directly used.
+- unknown/candidate memory must be inspected.
+- rejected/suppressed/retired memory must not be used.
+- archived/payload-required memory must be a rehydrate hook.
+- supersede/contradict/invalidate relations can block direct use.
+
+This separation matters:
+
+- Runtime can evolve admission policy without changing the storage event model.
+- Substrate can remain deterministic and auditable.
+- Future adapters can be tested against the same contract.
+
+## Safe Integration Sequence
+
+1. Keep Substrate independent.
+2. Export a small read-only Runtime snapshot into Substrate.
+3. Compare Runtime guide buckets with Substrate compiled buckets.
+4. Add experimental dual-write outside Runtime core.
+5. Only after repeated parity runs, consider a Runtime adapter.
+
+## Explicit Non-goals
+
+- Do not replace Runtime SQLite/Zvec yet.
+- Do not add benchmark-specific rules to Substrate.
+- Do not make Substrate responsible for LLM reasoning.
+- Do not let Substrate output source-code repair procedures.

package/docs/RUNTIME_REFERENCE_CORPUS.md

@@ -0,0 +1,129 @@
+# Runtime Reference Corpus Parity
+
+`runtime-reference-corpus` checks whether exported Runtime `agent_context` or
+`memory_decision_trace` JSON can be tied back to real Runtime Lite SQLite data.
+
+It is intentionally stricter than demo validation:
+
+- references are parsed for concrete memory ids only;
+- Runtime scopes are discovered from `lite_memory_nodes`;
+- a reference is matched to a scope only when memory ids overlap;
+- unmatched references stay in `unmatched_reference_reports` and are not counted
+  as parity evidence.
+
+## Run
+
+```bash
+npm run check:runtime-reference-corpus -- \
+  --source-root /Volumes/ziel/AionisRuntime-focused/.tmp \
+  --reference-root /Volumes/ziel/AionisRuntime-focused/docs/examples \
+  --max-source-files all \
+  --max-scopes-per-file 20 \
+  --max-scopes 100 \
+  --max-references all
+```
+
+The default output is written under `reports/runtime-reference-corpus-*/summary.json`.
+
+## Report fields
+
+- `matched_references`: reference JSON files with at least `min_overlap` ids
+  found in a Runtime SQLite scope.
+- `unmatched_references`: reference JSON files that had no Runtime scope overlap,
+  or contained no recognizable Runtime admission surfaces.
+- `exact_matches`: matched references where Substrate `compileContext` buckets
+  exactly match Runtime reference buckets.
+- `partial_matches`: matched references where both sides were available but at
+  least one bucket differed.
+
+This tool does not prove Runtime product behavior by itself. It proves whether a
+reference artifact is a real, traceable Runtime parity artifact or only an
+unmatched demo/export file.
+
+## Current focused Runtime check
+
+On 2026-06-25, the tool was run against:
+
+```bash
+npm run check:runtime-reference-corpus -- \
+  --source-root /Volumes/ziel/AionisRuntime-focused \
+  --reference-root /Volumes/ziel/AionisRuntime-focused/docs/examples \
+  --max-source-files all \
+  --max-scopes all \
+  --max-scopes-per-file 100 \
+  --max-references all
+```
+
+Result: 30 SQLite files were discovered, 14 contained Runtime Lite scopes, 228
+candidate scopes were scanned, 36 reference JSON files were scanned, and 4 files
+contained extractable Runtime admission surfaces. None of those 4 reference files
+shared memory ids with the scanned Runtime SQLite scopes.
+
+That means the current `docs/examples` files are useful product examples, but
+they are not current real Runtime reference parity artifacts for the local
+SQLite corpus. A future real parity run should export guide/measure JSON from
+the same Runtime database snapshot that is supplied as `--source-root`.
+
+## Same-source reference fixture
+
+Use `make:runtime-product-reference` to create a same-source artifact pair:
+
+1. start `AionisRuntime-focused` as a local Lite Runtime;
+2. force its write/replay SQLite files into the output directory;
+3. run a real `/v1/observe -> /v1/guide -> /v1/feedback -> /v1/measure`
+   product loop through the Runtime SDK;
+4. write `reference.json` containing the returned `agent_context` and
+   `memory_decision_trace`;
+5. run `check:runtime-reference-corpus` against that exact SQLite/reference pair.
+
+```bash
+npm run make:runtime-product-reference -- \
+  --runtime-root /Volumes/ziel/AionisRuntime-focused \
+  --scenario-count 4
+```
+
+Outputs are written to `reports/runtime-product-reference-*/`:
+
+- `runtime-write.sqlite`
+- `runtime-replay.sqlite`
+- `references/*.json`
+- `scenario-summaries/*.json`
+- `parity-summary.json`
+- `run-summary.json`
+
+The default `--scenario-count` is `1` for quick smoke checks. Use
+`--scenario-count 4` to exercise the current built-in same-source corpus:
+active route, schema migration, context compiler, and feedback attribution.
+Every scenario is written through the focused Runtime SDK into the same Runtime
+SQLite file, then matched back to its own exported Runtime guide/measure
+reference by concrete memory id overlap.
+
+This command is intentionally outside Runtime core. It creates a traceable
+reference artifact for Substrate parity; it does not mutate the focused Runtime
+repository and does not use `docs/examples` as evidence.
+
+## Current same-source boundary
+
+The built-in same-source product fixture currently validates the execution
+continuity path that Runtime itself exposes from the same Lite SQLite file:
+
+- accepted execution state enters `use_now`;
+- failed execution branches enter `do_not_use`;
+- measure/debug/audit records are retained as audit evidence and excluded from
+  agent-facing Substrate import.
+
+It does not claim full four-bucket Runtime parity. In particular,
+`inspect_before_use` and `rehydrate` have separate Runtime product paths:
+
+- ordinary/candidate memory admission is surfaced through planning recall and
+  external candidate governance;
+- archived/payload evidence becomes a rehydration hook only when the Runtime
+  guide surface returns that hook.
+
+Substrate already supports those buckets at the store contract level, and the
+contract benchmark covers them. The same-source Runtime reference corpus should
+only count `inspect_before_use` or `rehydrate` when the focused Runtime exports
+those exact memory ids in `agent_context` or `memory_decision_trace`. Do not
+force extra local SQLite rows into those buckets just to increase apparent
+coverage; that would turn parity into a runner-specific assertion instead of a
+Runtime reference.

package/docs/RUNTIME_SNAPSHOT_CORPUS.md

@@ -0,0 +1,81 @@
+# Runtime Snapshot Corpus Validation
+
+The corpus checker validates whether Aionis Substrate can represent real Aionis Runtime Lite SQLite snapshots without mutating Runtime data.
+
+It is a read-only validation path:
+
+1. scan Runtime SQLite files;
+2. select Runtime scopes;
+3. import each scope into a temporary Substrate SQLite store;
+4. compile Substrate context;
+5. aggregate import coverage, bucket counts, warnings, and failures.
+
+It does not replace Runtime storage and does not install dual-write.
+
+## CLI
+
+```bash
+npm run check:runtime-corpus -- \
+  --root /path/to/AionisRuntime-focused/.tmp \
+  --max-files all \
+  --max-scopes 100 \
+  --max-scopes-per-file 20 \
+  --min-nodes 1 \
+  --max-per-bucket 50
+```
+
+The command writes a report under `reports/runtime-snapshot-corpus-*` unless `--output` is supplied.
+
+## Report Fields
+
+- `discovered_sqlite_files`: SQLite-like files found under the root.
+- `runtime_sqlite_files`: files with Runtime `lite_memory_nodes`.
+- `candidate_scopes`: scopes selected from Runtime files before global truncation.
+- `attempted_scopes`: scopes actually imported.
+- `passed_scopes`: scopes imported and compiled successfully.
+- `failed_scopes`: scopes that failed import or compile.
+- `total_nodes_imported`: imported Runtime nodes.
+- `total_warnings`: scan and import warnings.
+- `scope_reports`: per-scope import and bucket summary.
+
+## Interpretation
+
+This checker proves storage-contract coverage, not Runtime policy superiority.
+
+Good results mean:
+
+- Runtime SQLite sources can be opened read-only;
+- Runtime memory nodes can be mapped into Substrate nodes;
+- Substrate can compile governed context for real scopes;
+- warnings and failures are visible.
+
+Bad results should be treated as mapping or substrate-contract evidence, not as a reason to mutate Aionis Runtime core.
+
+## Local Validation Snapshot
+
+Latest local run:
+
+```bash
+npm run check:runtime-corpus -- \
+  --root /Volumes/ziel/AionisRuntime-focused/.tmp \
+  --max-files all \
+  --max-scopes 100 \
+  --max-scopes-per-file 20 \
+  --min-nodes 1 \
+  --max-per-bucket 50
+```
+
+Result on 2026-06-25:
+
+- discovered SQLite files: 30
+- Runtime SQLite files: 14
+- candidate scopes: 128
+- attempted scopes: 100
+- passed scopes: 100
+- failed scopes: 0
+- imported Runtime nodes: 8,948
+- warnings: 0
+
+Local report:
+
+`reports/runtime-snapshot-corpus-2026-06-25T11-54-32-014Z/summary.json`

package/docs/RUNTIME_SNAPSHOT_IMPORT.md

@@ -0,0 +1,120 @@
+# Runtime Lite Snapshot Import
+
+This importer reads an existing Aionis Runtime Lite SQLite database and writes a one-way snapshot into an independent Aionis Substrate store.
+
+It is intentionally not a Runtime integration path:
+
+- source Runtime SQLite is opened read-only;
+- no Runtime table is created, altered, or updated;
+- no dual-write is installed;
+- no Runtime guide/admission rule is changed;
+- no benchmark or host-specific behavior is encoded.
+
+The importer exists to answer one engineering question:
+
+> Can the Substrate contract represent Runtime memory evidence, relations, feedback, and decision traces without polluting the focused Runtime?
+
+## Source Tables
+
+The importer currently understands the focused Runtime Lite write-store tables:
+
+- `lite_memory_nodes`
+- `lite_memory_edges`
+- `lite_memory_execution_native_index`
+- `lite_memory_rule_feedback`
+- `lite_memory_execution_decisions`
+
+`lite_memory_nodes` is required. Other tables are optional; missing optional tables are treated as empty.
+
+## Mapping
+
+Runtime nodes become Substrate memory nodes:
+
+- `id`, `scope`, `title`, `text_summary`, `confidence`, agent/team ownership, and Runtime metadata are preserved.
+- `slots_json` and execution-native index fields are preserved under `metadata.runtime_slots` and `metadata.runtime_execution_index`.
+- execution target files are collected from `slots_json` and `lite_memory_execution_native_index`.
+- Runtime raw/evidence references become `payloadRef`.
+
+Admission state is mapped conservatively:
+
+- explicit active/trusted/current/workflow-anchor evidence can enter `use_now`;
+- candidate/advisory/unknown evidence enters `inspect_before_use`;
+- archived/cold payload evidence enters `rehydrate`;
+- `supersedes`, `contradicts`, and `invalidates` relations block direct use through `do_not_use`.
+
+Runtime edges become Substrate relations:
+
+- `supersedes`, `contradicts`, `invalidates`, `requires_payload`, `derived_from`, or `supports`.
+- relation metadata preserves Runtime edge type, weight, decay, commit id, and metadata JSON.
+
+Rule feedback becomes Substrate feedback when the referenced rule node was imported.
+
+Execution decisions become Substrate decision traces when their referenced source rules were imported.
+
+## CLI
+
+SQLite target:
+
+```bash
+node scripts/import-runtime-snapshot.ts \
+  --source /path/to/aionis-runtime-lite.sqlite \
+  --target /tmp/aionis-substrate.sqlite \
+  --adapter sqlite \
+  --scope repo-a
+```
+
+File target:
+
+```bash
+node scripts/import-runtime-snapshot.ts \
+  --source /path/to/aionis-runtime-lite.sqlite \
+  --target /tmp/aionis-substrate-file-store \
+  --adapter file
+```
+
+The command prints a JSON summary with imported/skipped counts and warnings.
+
+## Parity Checker
+
+The parity checker imports a Runtime Lite SQLite snapshot into an isolated SQLite Substrate store, compiles a Substrate context, and optionally compares its bucket ids with a Runtime guide/measure JSON file.
+
+Smoke mode, without Runtime reference JSON:
+
+```bash
+npm run check:runtime-snapshot -- \
+  --source /path/to/aionis-runtime-lite.sqlite \
+  --scope repo-a \
+  --output /tmp/runtime-snapshot-smoke.json
+```
+
+Parity mode, with a Runtime response containing `agent_context` and/or `memory_decision_trace`:
+
+```bash
+npm run check:runtime-snapshot -- \
+  --source /path/to/aionis-runtime-lite.sqlite \
+  --scope repo-a \
+  --reference /path/to/runtime-guide-or-measure.json \
+  --output /tmp/runtime-snapshot-parity.json
+```
+
+The checker compares:
+
+- `use_now`
+- `inspect_before_use`
+- `do_not_use`
+- `rehydrate`
+
+The Runtime reference may be a direct `agent_context`, a direct `memory_decision_trace`, or a larger JSON object containing either surface.
+
+When no reference is supplied, the report is still useful: it gives import coverage, skipped rows, warnings, and Substrate bucket counts for a real Runtime DB.
+
+## Product Boundary
+
+This is a read-only bridge for validation. It is not a replacement for Runtime Lite SQLite, Zvec, AIFS, or the product facade.
+
+The safe sequence remains:
+
+1. import Runtime snapshots into an isolated Substrate store;
+2. compare admission buckets and decision traces outside Runtime;
+3. run parity and negative-control tests;
+4. only then decide whether any Runtime adapter is worth productizing.

package/docs/STORE_CONTRACT.md

@@ -0,0 +1,181 @@
+# Aionis Store Contract
+
+This document defines the first Aionis Substrate boundary. It is intentionally about memory semantics, not about a storage engine.
+
+## Non-goals
+
+- It is not a general database.
+- It is not a vector database.
+- It is not an Agent framework.
+- It is not connected to Aionis Runtime yet.
+- It must not encode task-specific repair rules or benchmark fixtures.
+
+## Core Model
+
+### Event Log
+
+Every durable mutation is recorded as an append-only event:
+
+- `memory.node.upsert`
+- `memory.lifecycle.transition`
+- `memory.relation.upsert`
+- `memory.feedback.recorded`
+- `memory.decision.recorded`
+- `substrate.checkpoint.created`
+
+The event log is the evidence trail. Read models may be rebuilt from it or maintained alongside it, but they are not the authority boundary.
+
+`substrate.checkpoint.created` is a physical compaction event. It contains a checksum-covered snapshot of the current substrate state plus metadata about the covered event history. It does not suppress, archive, delete, or promote memory by itself.
+
+### Store Schema Version
+
+Every store has a substrate schema version. The current schema version is `1`.
+
+Adapters must expose schema metadata through `getStoreInfo`:
+
+- adapter kind
+- schema version
+- last event sequence
+- event count
+
+The SQLite adapter persists schema metadata in `substrate_metadata` and mirrors the same version into SQLite `user_version`. Opening a store with a newer unsupported schema must fail before any mutation occurs.
+
+The file adapter writes the same schema version into `snapshot.json`. The append-only event log remains the durable evidence source; the snapshot schema is the derived read-model format.
+
+### Backup Boundary
+
+Backups export the append-only event log plus schema metadata and a SHA-256 checksum over the canonical event list.
+
+Restore must verify the backup before writing a target store. Restored stores preserve original event ids and sequence numbers, then rebuild derived read models.
+
+Payload files referenced by `payloadRef` are not embedded in the Substrate backup. They remain external artifacts and need their own retention policy.
+
+### Checkpoint Compaction
+
+Adapters may compact a long event log into a `substrate.checkpoint.created` event.
+
+The checkpoint must preserve:
+
+- covered event count;
+- covered last sequence;
+- SHA-256 checksum of the covered event list;
+- current memory nodes;
+- current relations;
+- current feedback records;
+- current decision traces.
+
+After compaction, the physical log sequence restarts at the checkpoint event. Future writes continue after that checkpoint. This is a storage-maintenance boundary only; it must not change admission buckets or lifecycle state.
+
+### Memory Node
+
+A memory node is a governed memory object:
+
+- `scope`
+- `kind`
+- `summary`
+- `lifecycle`
+- `authority`
+- `confidence`
+- optional target files
+- optional payload pointer
+- optional owner identity
+
+The store treats execution memory, procedure memory, facts, preferences, claims, feedback, and trace pointers as first-class node kinds. It does not flatten all memory into free text.
+
+### Search Contract
+
+`searchNodes` is a read-only substrate query over memory nodes.
+
+It must:
+
+- require an explicit `scope`;
+- keep all results scoped;
+- support exact filters for kind, lifecycle, authority, target file, owner identity, confidence, and update time;
+- provide deterministic lexical query scoring over node id, title, summary, target files, payload pointer, owner ids, and primitive metadata;
+- return scored results with inspectable reason codes;
+- avoid writing decision events or mutating lifecycle state.
+
+Search is not admission. It may find candidate evidence, but it must not decide whether memory can influence the next Agent turn. Governed prompt surfaces are produced by `compileContext`.
+
+### Relation Graph
+
+Relations connect memory evidence:
+
+- `supports`
+- `derived_from`
+- `supersedes`
+- `contradicts`
+- `invalidates`
+- `requires_payload`
+
+Relations are scoped. A relation in one scope must never affect admission in another scope.
+
+### Lifecycle
+
+Forgetting is represented as state, not deletion.
+
+Allowed lifecycle states:
+
+- `active`
+- `candidate`
+- `contested`
+- `suppressed`
+- `archived`
+- `retired`
+- `blocked`
+- `rehydrate_required`
+
+The store must keep the old evidence and express changes through lifecycle transitions and relations.
+
+## Admission Contract
+
+The store compiles memory into four buckets:
+
+- `use_now`: directly usable context.
+- `inspect_before_use`: relevant but not authoritative.
+- `do_not_use`: blocked, superseded, contradicted, invalidated, rejected, suppressed, retired, or unsafe.
+- `rehydrate`: payload exists but should not be stuffed into the prompt until requested.
+
+The current deterministic admission baseline is deliberately conservative:
+
+- active + trusted/verified memory may enter `use_now`.
+- candidate/contested/advisory/unknown memory enters `inspect_before_use`.
+- suppressed/retired/blocked/rejected memory enters `do_not_use`.
+- archived or payload-required memory enters `rehydrate`.
+- high-confidence supersede/contradict/invalidate relations block direct use.
+- payload-required relations route to `rehydrate`.
+
+This is not the final Aionis Runtime admission policy. It is the substrate-level minimum contract.
+
+## Decision Trace
+
+Every compiled context must include a decision trace:
+
+- which memory id was routed
+- what bucket it entered
+- which reason code caused the decision
+- which relation caused the decision when applicable
+
+The trace is for audit/debug/measure. It must not mutate admission by itself.
+
+`previewContext` is the side-effect-free admission preview. It returns the same bucket and reason-code shape as `compileContext`, but it must not append events or insert decision rows.
+
+`compileContext` is intentionally not a pure read. It records `memory.decision.recorded` so every exported context has an auditable receipt. Tools that need a side-effect-free view must use `previewContext` instead of treating `compileContext` as read-only.
+
+## Adapter Requirements
+
+Every adapter must satisfy the same observable contract:
+
+1. Persist append-only events.
+2. Preserve scope isolation.
+3. Keep lifecycle transitions instead of deleting memory.
+4. Compile the same admission buckets from the same node/relation state.
+5. Return the same scoped search results from the same node state.
+6. Record decision traces as events.
+7. Reopen cleanly and recover the same read model.
+8. If compaction is supported, compact only through a validated checkpoint event.
+
+Current adapters:
+
+- `openFileAionisSubstrate`: append-only JSONL plus derived JSON snapshot.
+- `openSqliteAionisSubstrate`: SQLite event table plus structured read model tables.

package/examples/basic/README.md

@@ -0,0 +1,19 @@
+# Basic Example
+
+This example runs the smallest useful Substrate loop:
+
+1. open a file-backed store;
+2. write current and stale execution memories;
+3. link the stale memory with a `supersedes` relation;
+4. preview context without writing a receipt;
+5. compile context with a `memory.decision.recorded` receipt;
+6. record positive feedback;
+7. search deterministic memory candidates;
+8. export and verify a backup;
+9. open a SQLite store and compile a rehydrate hook.
+
+Run it from the repository root:
+
+```bash
+npm run example:basic
+```