@aionis/substrate 0.1.0 → 0.1.2

package/docs/CLI.md

@@ -0,0 +1,183 @@
+# CLI
+
+`@aionis/substrate` publishes a small CLI for local store operations, validation, and sidecar integration work.
+
+It is intentionally narrow:
+
+- it inspects, previews, backs up, restores, and compacts Substrate stores;
+- it imports Runtime Lite SQLite snapshots into separate Substrate stores;
+- it runs read-only checks over existing Runtime evidence;
+- it writes reports to local files;
+- it does not start Aionis Runtime unless you explicitly use the separate repository script `check:runtime-dual-write`;
+- it does not mutate Runtime source code or replace Runtime storage.
+
+## Install
+
+Run without installing permanently:
+
+```bash
+npx @aionis/substrate --help
+```
+
+Install into a project:
+
+```bash
+npm install @aionis/substrate
+npx aionis-substrate --help
+```
+
+The CLI requires Node 24+.
+
+## Store Commands
+
+All store commands use explicit adapter and path arguments:
+
+```bash
+npx aionis-substrate inspect --adapter sqlite --path ./substrate.sqlite --scope repo-a
+npx aionis-substrate preview-context --adapter sqlite --path ./substrate.sqlite --scope repo-a --query "continue runtime work"
+npx aionis-substrate backup --adapter sqlite --path ./substrate.sqlite --output ./substrate-backup.json
+npx aionis-substrate restore --adapter sqlite --path ./restored.sqlite --input ./substrate-backup.json
+npx aionis-substrate compact --adapter sqlite --path ./substrate.sqlite
+```
+
+Use `--adapter file` with a directory path for the file-backed adapter:
+
+```bash
+npx aionis-substrate inspect --adapter file --path ./substrate-store --scope repo-a
+```
+
+### Inspect
+
+`inspect` prints store metadata. With `--scope`, it also prints scoped counts and memory-node summaries:
+
+```bash
+npx aionis-substrate inspect \
+  --adapter sqlite \
+  --path ./substrate.sqlite \
+  --scope repo-a
+```
+
+The report contract is `aionis_substrate_inspect_report_v1`.
+
+### Preview Context
+
+`preview-context` compiles the governed buckets without writing a `memory.decision.recorded` receipt:
+
+```bash
+npx aionis-substrate preview-context \
+  --adapter sqlite \
+  --path ./substrate.sqlite \
+  --scope repo-a \
+  --query "continue the current route" \
+  --max-per-bucket 8
+```
+
+The report includes `read_only: true` when event counts and sequence numbers are unchanged.
+
+### Backup and Restore
+
+`backup` writes a checksum-covered event backup:
+
+```bash
+npx aionis-substrate backup \
+  --adapter sqlite \
+  --path ./substrate.sqlite \
+  --output ./substrate-backup.json
+```
+
+`restore` verifies the backup before writing an empty target:
+
+```bash
+npx aionis-substrate restore \
+  --adapter sqlite \
+  --path ./restored.sqlite \
+  --input ./substrate-backup.json
+```
+
+Use `--overwrite` only when replacing an existing restore target is intentional.
+
+### Compact
+
+`compact` rewrites the event history into one checkpoint event without changing governed state:
+
+```bash
+npx aionis-substrate compact \
+  --adapter sqlite \
+  --path ./substrate.sqlite
+```
+
+## Runtime Snapshot Import
+
+Use `import-runtime-snapshot` to copy Runtime Lite SQLite evidence into a separate Substrate store:
+
+```bash
+npx aionis-substrate import-runtime-snapshot \
+  --source /path/to/aionis-runtime-lite.sqlite \
+  --target ./substrate.sqlite \
+  --adapter sqlite \
+  --scope repo-a
+```
+
+The Runtime source is opened read-only. The target is a Substrate store owned by this command.
+
+## Sidecar Check
+
+Use `sidecar` when you already have Runtime Lite SQLite evidence and want to check whether Substrate can mirror the governed context surface from outside the Runtime boundary.
+
+Snapshot parity:
+
+```bash
+npx aionis-substrate sidecar \
+  --source /path/to/aionis-runtime-lite.sqlite \
+  --scope repo-a \
+  --reference /path/to/runtime-guide-or-measure.json \
+  --output reports/runtime-sidecar/summary.json
+```
+
+Reference corpus parity:
+
+```bash
+npx aionis-substrate sidecar \
+  --source-root /path/to/runtime-sqlite-root \
+  --reference-root /path/to/runtime-reference-root \
+  --max-source-files all \
+  --max-scopes all \
+  --max-scopes-per-file 100 \
+  --max-references all
+```
+
+Combined report:
+
+```bash
+npx aionis-substrate sidecar \
+  --source /path/to/aionis-runtime-lite.sqlite \
+  --scope repo-a \
+  --reference /path/to/runtime-guide-or-measure.json \
+  --source-root /path/to/runtime-sqlite-root \
+  --reference-root /path/to/runtime-reference-root \
+  --output reports/runtime-sidecar/summary.json
+```
+
+The report contract is `aionis_runtime_sidecar_check_report_v1`.
+
+## What Passing Means
+
+Passing snapshot parity means one Runtime SQLite scope can be imported into an isolated Substrate store and compiled into matching governed buckets.
+
+Passing reference corpus parity means exported Runtime guide/measure JSON is traceable to real Runtime memory ids in the same source corpus.
+
+Passing these checks does not mean Substrate has become the full Runtime policy engine. Runtime still owns richer product policy; Substrate owns durable evidence, lifecycle state, relations, feedback, and the minimum governed context contract.
+
+## Common Failures
+
+`no_matched_reference`
+
+The reference files were scanned, but none of their memory ids overlap the discovered Runtime SQLite scopes. Point `--reference-root` at guide/measure outputs produced from the same Runtime data as `--source-root`.
+
+`snapshot_parity failed`
+
+The imported Substrate buckets do not match the supplied Runtime guide/measure surface. Inspect the generated `summary.json` before changing code; the mismatch may be a scope, reference, or fixture problem.
+
+`ExperimentalWarning: SQLite is an experimental feature`
+
+Node 24 currently marks `node:sqlite` as experimental. This is expected for the current embedded SQLite adapter.

package/docs/RUNTIME_SIDECAR_STABILIZATION.md

@@ -0,0 +1,96 @@
+# Runtime Sidecar Stabilization
+
+This page defines the Substrate sidecar validation path for Aionis Runtime.
+
+The sidecar path is intentionally external:
+
+- it does not mutate `AionisRuntime-focused` source code;
+- it does not replace Runtime Lite SQLite, Zvec, AIFS, or product storage;
+- it does not install a production dual-write adapter;
+- it only proves that Substrate can mirror, import, and inspect Runtime evidence from outside the Runtime boundary.
+
+## Sidecar Gates
+
+### Gate 1: Read-Only Snapshot Parity
+
+Import one Runtime Lite SQLite snapshot into an isolated Substrate store and compile context for one scope.
+
+```bash
+npm run check:runtime-sidecar -- \
+  --source /path/to/aionis-runtime-lite.sqlite \
+  --scope repo-a \
+  --reference /path/to/runtime-guide-or-measure.json
+```
+
+Without `--reference`, this is an import smoke: Runtime SQLite is opened read-only and Substrate writes to an isolated target.
+
+With `--reference`, the command compares the four governed buckets:
+
+- `use_now`
+- `inspect_before_use`
+- `do_not_use`
+- `rehydrate`
+
+### Gate 2: Same-Source Reference Corpus
+
+Scan Runtime SQLite files and Runtime guide/measure JSON references, then count only references that share concrete memory ids with a discovered Runtime scope.
+
+```bash
+npm run check:runtime-sidecar -- \
+  --source-root /path/to/runtime-sqlite-root \
+  --reference-root /path/to/runtime-reference-root \
+  --max-source-files all \
+  --max-scopes all \
+  --max-scopes-per-file 100 \
+  --max-references all
+```
+
+This gate rejects demo-only evidence: a reference JSON must overlap real Runtime memory ids to count.
+
+### Gate 3: Real Runtime Dual-Write Sidecar
+
+This stage starts focused Runtime with isolated Lite SQLite files, calls the public Runtime SDK loop, mirrors the observed memory ids and outcomes into a separate Substrate SQLite store, and compares Runtime guide surfaces against Substrate compiled context.
+
+```bash
+npm run check:runtime-dual-write -- \
+  --runtime-root /Volumes/ziel/AionisRuntime-focused \
+  --generated-count 8 \
+  --chain-probe-count 4 \
+  --concurrency 4
+```
+
+This command remains separate from `check:runtime-sidecar` because it starts a Runtime process. It must be explicit.
+
+## Combined Read-Only Report
+
+`check:runtime-sidecar` can run Gate 1 and Gate 2 in one report:
+
+```bash
+npm run check:runtime-sidecar -- \
+  --source /path/to/aionis-runtime-lite.sqlite \
+  --scope repo-a \
+  --reference /path/to/runtime-guide-or-measure.json \
+  --source-root /path/to/runtime-sqlite-root \
+  --reference-root /path/to/runtime-reference-root \
+  --output reports/runtime-sidecar-manual/summary.json
+```
+
+The report contract is `aionis_runtime_sidecar_check_report_v1`.
+
+It records:
+
+- requested stages;
+- pass/fail summary per stage;
+- snapshot import coverage and parity;
+- reference corpus matched/unmatched counts;
+- notes confirming that Runtime source and Runtime storage were not replaced.
+
+## Interpretation
+
+Passing Gate 1 means Substrate can represent one Runtime SQLite scope as governed substrate state.
+
+Passing Gate 2 means exported Runtime references are traceable to real Runtime SQLite memory ids, not only documentation examples.
+
+Passing Gate 3 means an external host can mirror a small real Runtime execution loop into Substrate and preserve the same admission surface after reopen.
+
+None of these gates mean Substrate has become the full Aionis Runtime policy engine. Runtime owns richer product policy; Substrate owns the durable evidence and minimum governed context contract.

package/docs/STORE_CONTRACT.md

@@ -40,6 +40,8 @@ Adapters must expose schema metadata through `getStoreInfo`:
 
 The SQLite adapter persists schema metadata in `substrate_metadata` and mirrors the same version into SQLite `user_version`. Opening a store with a newer unsupported schema must fail before any mutation occurs.
 
+SQLite schema changes are applied through an explicit adapter migration registry. Applied migrations are recorded in `substrate_schema_migrations`. The registry must stay contiguous and end at the current substrate schema version, and a tampered migration name must cause open to fail.
+
 The file adapter writes the same schema version into `snapshot.json`. The append-only event log remains the durable evidence source; the snapshot schema is the derived read-model format.
 
 ### Backup Boundary
@@ -48,6 +50,8 @@ Backups export the append-only event log plus schema metadata and a SHA-256 chec
 
 Restore must verify the backup before writing a target store. Restored stores preserve original event ids and sequence numbers, then rebuild derived read models.
 
+Backup verification must replay event reference integrity, including lifecycle targets, relation endpoints, feedback memory ids, and decision-trace memory ids.
+
 Payload files referenced by `payloadRef` are not embedded in the Substrate backup. They remain external artifacts and need their own retention policy.
 
 ### Checkpoint Compaction
@@ -156,12 +160,22 @@ Every compiled context must include a decision trace:
 - which reason code caused the decision
 - which relation caused the decision when applicable
 
+Every decision entry must reference a memory node that exists in the same scope as the trace. Decision traces are receipts over known substrate evidence; they cannot introduce orphan memory ids.
+
 The trace is for audit/debug/measure. It must not mutate admission by itself.
 
 `previewContext` is the side-effect-free admission preview. It returns the same bucket and reason-code shape as `compileContext`, but it must not append events or insert decision rows.
 
 `compileContext` is intentionally not a pure read. It records `memory.decision.recorded` so every exported context has an auditable receipt. Tools that need a side-effect-free view must use `previewContext` instead of treating `compileContext` as read-only.
 
+Decision and evidence records must also be inspectable without creating new events:
+
+- `listRelations(scope, memoryId?)`
+- `listFeedback({ scope, memoryId? })`
+- `listDecisions(scope)`
+
+These read APIs are for audit, debug, and measure surfaces. They do not compile context and do not mutate lifecycle, relation, feedback, decision, or checkpoint state.
+
 ## Adapter Requirements
 
 Every adapter must satisfy the same observable contract:
@@ -172,8 +186,10 @@ Every adapter must satisfy the same observable contract:
 4. Compile the same admission buckets from the same node/relation state.
 5. Return the same scoped search results from the same node state.
 6. Record decision traces as events.
-7. Reopen cleanly and recover the same read model.
-8. If compaction is supported, compact only through a validated checkpoint event.
+7. Return the same scoped audit reads from the same relation, feedback, and decision state.
+8. Reopen cleanly and recover the same read model.
+9. Reject corrupt lifecycle, relation, feedback, decision, and checkpoint references before persisting invalid events.
+10. If compaction is supported, compact only through a validated checkpoint event.
 
 Current adapters:
 

package/docs/V0_2_ROADMAP.md

@@ -0,0 +1,108 @@
+# Aionis Substrate v0.2 Roadmap
+
+This roadmap freezes the intended scope for the next Substrate release.
+
+Substrate remains an independent storage-contract layer. It is not the full Aionis Runtime policy engine, not a vector database, and not an Agent framework.
+
+## v0.2 Principles
+
+- Keep append-only evidence as the source of truth.
+- Keep lifecycle, relation, feedback, and decision receipts inspectable.
+- Preserve file and SQLite adapter parity for every public API.
+- Add product hardening only where it improves durability, auditability, or Runtime sidecar integration.
+- Do not encode task-specific benchmark fixtures or one-off Runtime policies in Substrate.
+
+## Included
+
+### 1. Migration Scaffold
+
+Add a small schema-migration boundary for future SQLite changes:
+
+- explicit migration registry;
+- current schema guard remains strict;
+- migration tests for already-created stores;
+- no best-effort reads of unsupported future schemas.
+
+Initial implementation status: SQLite now has a registry-backed v1 migration ledger. Future schema changes should add migrations through that registry instead of editing open-time schema creation ad hoc.
+
+### 2. Read API Hardening
+
+Expose scoped audit reads that do not mutate the event log:
+
+- `listRelations(scope, memoryId?)`
+- `listFeedback({ scope, memoryId? })`
+- `listDecisions(scope)`
+
+These APIs are for audit/debug/measure surfaces and adapter parity checks. They do not replace `compileContext`, and they do not make admission decisions.
+
+### 3. Durability Negative Tests
+
+Extend failure-mode coverage:
+
+- corrupt event references;
+- interrupted writes;
+- relation and feedback edge cases after compaction;
+- checkpoint reopen after mixed read/write traffic.
+
+Initial implementation status: decision trace references are validated like relation and feedback references. File and SQLite adapters reject missing decision targets before persisting events or rows; backup verification rejects checksum-valid orphan decision receipts; checkpoint reopen rejects corrupt decision references; post-checkpoint invalid relation, feedback, and decision writes stay atomic.
+
+### 4. Runtime Sidecar Stabilization
+
+Keep Runtime experiments isolated:
+
+- read-only Runtime snapshot import remains separate from Runtime source;
+- dual-write sidecar continues to write into a separate Substrate store;
+- no replacement of Aionis Runtime storage in v0.2.
+
+Initial implementation status: `check:runtime-sidecar` now combines read-only Runtime snapshot parity and same-source reference corpus parity into a single report contract. Real Runtime dual-write remains an explicit separate gate through `check:runtime-dual-write` because it starts focused Runtime.
+
+### 5. Product CLI and Docs
+
+Make the substrate boundary easier to consume without widening policy scope:
+
+- publish a package CLI entrypoint for read-only sidecar checks;
+- document install, minimal API usage, and sidecar reports separately;
+- keep repository-only Runtime process experiments explicit and separate.
+
+Initial implementation status: the package exposes `aionis-substrate sidecar` for read-only snapshot/reference checks and store commands for inspect, preview-context, backup, restore, compact, and Runtime snapshot import. These commands do not start Runtime, mutate Runtime storage, or implement Runtime admission policy.
+
+## Excluded
+
+- Vector search, ANN, embeddings, or semantic recall.
+- Full Aionis Runtime admission policy.
+- LLM-as-judge or model-generated lifecycle policy.
+- Agent orchestration or external Agent harnesses.
+- SaaS tenancy, auth, billing, or cloud service behavior.
+- Replacing `AionisRuntime-focused` storage.
+
+## Release Gates
+
+Before v0.2 can be tagged:
+
+- `npm run typecheck`
+- `npm test`
+- `npm run bench:contract`
+- `npm run check:release`
+- `npm run check:scale -- --nodes 10000 --scopes 10 --relations 2000 --feedback 1000`
+- adapter parity tests for every new public API;
+- package install smoke from tarball;
+- published registry smoke after release.
+
+## v0.1 Baseline
+
+v0.1 already provides:
+
+- file and SQLite adapters;
+- append-only event log;
+- governed context buckets;
+- decision receipts;
+- read-only context preview;
+- backup and restore;
+- checkpoint compaction;
+- deterministic scoped search;
+- Runtime snapshot import;
+- external admission parity;
+- isolated Runtime dual-write sidecar experiments;
+- npm package and registry smoke checks.
+
+v0.2 should harden this substrate boundary instead of broadening the product surface.

package/package.json

@@ -1,10 +1,13 @@
 {
   "name": "@aionis/substrate",
-  "version": "0.1.0",
-  "description": "Aionis-native memory and execution-state substrate experiment.",
+  "version": "0.1.2",
+  "description": "Durable governed memory substrate for Aionis execution state.",
   "type": "module",
   "main": "./dist/index.js",
   "types": "./dist/index.d.ts",
+  "bin": {
+    "aionis-substrate": "dist/cli.js"
+  },
   "exports": {
     ".": {
       "types": "./dist/index.d.ts",
@@ -30,11 +33,14 @@
     "check:runtime-snapshot": "node scripts/runtime-snapshot-parity.ts",
     "check:runtime-corpus": "node scripts/runtime-snapshot-corpus.ts",
     "check:runtime-reference-corpus": "node scripts/runtime-reference-corpus.ts",
+    "check:runtime-sidecar": "node scripts/runtime-sidecar-check.ts",
     "make:runtime-product-reference": "node scripts/runtime-product-reference-fixture.ts",
     "check:external-admission-parity": "node scripts/external-admission-parity.ts",
     "check:runtime-dual-write": "node scripts/runtime-dual-write-experiment.ts",
     "check:pack": "npm run build && node scripts/verify-package.ts",
     "check:install-smoke": "npm run build && node scripts/install-smoke.ts",
+    "check:registry-install": "node scripts/registry-install-smoke.ts",
+    "check:published-runtime-smoke": "node scripts/published-runtime-smoke.ts",
     "check:scale": "node scripts/scale-test.ts",
     "example:basic": "npm run build && node examples/basic/index.mjs",
     "check:release": "npm run typecheck && npm test && npm run bench:contract && npm run example:basic && npm run check:pack && npm run check:install-smoke"