igniter 0.5.0 → 0.5.2

data/docs/guide/store-adapters.md

@@ -0,0 +1,183 @@
+# Store Adapters
+
+This guide describes execution snapshot stores: persistence for contract
+executions, pending nodes, and resume flows.
+
+Do not confuse this with the newer contract persistence target, where ordinary
+contracts may declare record, workflow, credential-local, or signal persistence
+as part of their own capability. That direction is tracked in
+[Contract Persistence Target Plan](../dev/contract-persistence-target-plan.md).
+
+Igniter ships with reference execution stores for:
+
+- memory
+- file
+- sqlite
+- ActiveRecord-style persistence
+- Redis-style persistence
+
+All stores implement the same protocol:
+
+| Method | Description |
+|--------|-------------|
+| `save(snapshot, correlation: nil, graph: nil)` | Persist a snapshot; build secondary indexes for query |
+| `fetch(execution_id)` | Load a snapshot by ID; raises on missing |
+| `delete(execution_id)` | Remove a snapshot and clean up indexes |
+| `exist?(execution_id)` | Check existence without raising |
+| `find_by_correlation(graph:, correlation:)` | Find execution_id by correlation hash |
+| `list_all(graph: nil)` | All execution_ids, optionally filtered by graph name |
+| `list_pending(graph: nil)` | Execution_ids that have at least one node in `:pending` state |
+
+## Memory Store
+
+Useful for tests and single-process flows.
+
+```ruby
+Igniter.execution_store = Igniter::Runtime::Stores::MemoryStore.new
+```
+
+## File Store
+
+Useful for local development and smoke-testing worker flows.
+
+```ruby
+Igniter.execution_store = Igniter::Runtime::Stores::FileStore.new(
+  root: Rails.root.join("tmp/igniter_executions")
+)
+```
+
+## SQLite Store
+
+Useful for single-node application profiles that want durable local execution
+snapshots without pulling in Redis or ActiveRecord.
+
+```ruby
+Igniter.execution_store = Igniter::Runtime::Stores::SQLiteStore.new(
+  path: "var/igniter_executions.sqlite3"
+)
+```
+
+`sqlite3` now ships as a standard Igniter dependency, so this path works out of
+the box in the default stack and gem installation shape.
+
+The store keeps one row per execution snapshot and secondary indexes for:
+
+- `graph`
+- `pending`
+- `(graph, correlation_json)`
+
+## ActiveRecord Store
+
+Expected record shape:
+
+- one unique `execution_id` column
+- one text/json column for serialized snapshot payload
+
+Example model:
+
+```ruby
+class IgniterExecutionSnapshot < ApplicationRecord
+  validates :execution_id, presence: true, uniqueness: true
+end
+```
+
+Example migration:
+
+```ruby
+class CreateIgniterExecutionSnapshots < ActiveRecord::Migration[7.1]
+  def change
+    create_table :igniter_execution_snapshots do |t|
+      t.string :execution_id, null: false
+      t.jsonb :snapshot_json, null: false, default: {}
+      t.timestamps
+    end
+
+    add_index :igniter_execution_snapshots, :execution_id, unique: true
+  end
+end
+```
+
+Store configuration:
+
+```ruby
+Igniter.execution_store = Igniter::Runtime::Stores::ActiveRecordStore.new(
+  record_class: IgniterExecutionSnapshot,
+  execution_id_column: :execution_id,
+  snapshot_column: :snapshot_json
+)
+```
+
+## Redis Store
+
+`RedisStore` maintains secondary indexes so it can answer all query API calls efficiently:
+
+| Key pattern | Redis type | Purpose |
+|-------------|-----------|---------|
+| `{ns}:{execution_id}` | String | Serialized snapshot JSON |
+| `{ns}:all` | Set | All execution IDs |
+| `{ns}:graph:{name}` | Set | Execution IDs for one graph |
+| `{ns}:corr:{graph}` | Hash | `JSON(sorted_correlation)` → execution_id |
+
+The client must support these Redis commands:
+`set`, `get`, `del`, `exists?`, `sadd`, `srem`, `smembers`, `hset`, `hget`.
+
+The standard [`redis` gem](https://github.com/redis/redis-rb) satisfies this interface.
+
+```ruby
+redis = Redis.new(url: ENV.fetch("REDIS_URL"))
+
+Igniter.execution_store = Igniter::Runtime::Stores::RedisStore.new(
+  redis: redis,
+  namespace: "igniter:executions"   # optional, default shown
+)
+```
+
+**Query API:**
+
+```ruby
+store = Igniter::Runtime::Stores::RedisStore.new(redis: redis)
+
+# Find a pending execution by correlation
+execution_id = store.find_by_correlation(
+  graph: "OrderContract",
+  correlation: { order_id: "o-42" }
+)
+
+# List all executions for a graph
+ids = store.list_all(graph: "OrderContract")
+
+# List only pending executions (O(n) scan — acceptable for moderate volumes)
+pending_ids = store.list_pending(graph: "OrderContract")
+```
+
+## Worker Flow
+
+Store-backed contracts should declare:
+
+```ruby
+class AsyncPricingContract < Igniter::Contract
+  run_with runner: :store
+end
+```
+
+Producer side:
+
+```ruby
+contract = AsyncPricingContract.new(order_total: 100)
+deferred = contract.result.gross_total
+
+execution_id = contract.execution.events.execution_id
+token = deferred.token
+```
+
+Worker side:
+
+```ruby
+AsyncPricingContract.resume_from_store(
+  execution_id,
+  token: token,
+  value: 150
+)
+```
+
+If the resumed execution finishes successfully, the current `StoreRunner` deletes the persisted snapshot automatically.

data/docs/research/README.md

@@ -0,0 +1,131 @@
+# Research
+
+Compact research staging for ideas that may shape Igniter's architecture but are
+not accepted public API yet.
+
+Use this directory for governance protocols, long-horizon decisions, and cross-cutting
+design notes. Keep implementation snapshots and package specs in their respective
+`packages/<pkg>/docs/` directories once shipped. See **Rotation Policy** below.
+
+## Index
+
+### Protocols & Governance
+- [Horizon Protocol](./horizon-protocol.md) - working protocol, planning frame,
+  architect/agent roles, and Igniter Lang guardrails.
+- [Vision Handoff Protocol](./vision-handoff-protocol.md) - compact briefing
+  format for giving agents a large horizon with a narrow executable slice.
+- [Project Status Horizon Report](./project-status-horizon-report.md) -
+  whole-project status, verification, strategic insights, and horizon ideas.
+
+### Persistence Design
+- [Contract Persistence Landing Zone](./contract-persistence-landing-zone.md) -
+  placement decision for app-local proof, staged extraction, and future package
+  naming.
+- [Contract Persistence Organic Model](./contract-persistence-organic-model.md) -
+  horizon model for Store/History as future graph-native concepts.
+- [Igniter Lang Convergence Report](./igniter-lang-convergence-report.md) -
+  public synthesis of the private Igniter-Lang research against current Ledger,
+  Ledger Client, and Durable Model direction.
+
+### Wizard Type Spec
+- [Wizard Type Spec Research Request](./wizard-type-spec-research-request.md) -
+  handoff request for spec lineage, materialization, export, and migration
+  analysis.
+- [Wizard Type Spec Architecture](./wizard-type-spec-architecture.md) -
+  research response defining the canonical spec model, export rules, and
+  materializer boundary.
+
+## Moved to Playgrounds
+
+Planning/roadmap docs for work that is now fully shipped. Kept for historical
+reference in `playgrounds/docs/research/`.
+
+| Document | Playgrounds Location |
+|----------|---------------------|
+| Contract Persistence Development Track | [playgrounds/docs/research/](../../playgrounds/docs/research/contract-persistence-development-track.md) |
+| Contract Persistence Roadmap | [playgrounds/docs/research/](../../playgrounds/docs/research/contract-persistence-roadmap.md) |
+
+## Rotated to Package Docs
+
+Implementation snapshots that shipped are kept in their package `docs/` directories:
+
+| Document | New Location |
+|----------|-------------|
+| Igniter Ledger Open Protocol | [packages/igniter-ledger/docs/open-protocol.md](../../packages/igniter-ledger/docs/open-protocol.md) |
+| Contract-Native Store POC spec | [packages/igniter-ledger/docs/poc-specification.md](../../packages/igniter-ledger/docs/poc-specification.md) |
+| Contract-Native Store Server Model | [packages/igniter-ledger/docs/server-model.md](../../packages/igniter-ledger/docs/server-model.md) |
+| Contract Persistence Relations | [packages/igniter-ledger/docs/relations-specification.md](../../packages/igniter-ledger/docs/relations-specification.md) |
+| Companion / Store Convergence | [packages/igniter-ledger/docs/companion-convergence.md](../../packages/igniter-ledger/docs/companion-convergence.md) |
+| Igniter Ledger Progress Summary | [packages/igniter-ledger/docs/progress.md](../../packages/igniter-ledger/docs/progress.md) |
+| Contract-Native Store Research (iterations) | [packages/igniter-ledger/docs/research/store-iterations.md](../../packages/igniter-ledger/docs/research/store-iterations.md) |
+| Contract-Native Store Sync Hub (iterations) | [packages/igniter-ledger/docs/research/sync-hub-iterations.md](../../packages/igniter-ledger/docs/research/sync-hub-iterations.md) |
+| Durable Model Current Status Summary | [packages/igniter-durable-model/docs/current-status.md](../../packages/igniter-durable-model/docs/current-status.md) |
+| Durable Model Persistence App Status | [packages/igniter-durable-model/docs/app-status.md](../../packages/igniter-durable-model/docs/app-status.md) |
+| Durable Model Persistence Manifest Glossary | [packages/igniter-durable-model/docs/manifest-glossary.md](../../packages/igniter-durable-model/docs/manifest-glossary.md) |
+| Durable Model Contract Performance Signal | [packages/igniter-durable-model/docs/performance.md](../../packages/igniter-durable-model/docs/performance.md) |
+| Inter-Agent Compact Handoff (convergence audit) | [docs/store/convergence-audit.md](../store/convergence-audit.md) |
+
+## Read First
+
+Before changing architecture or reviewing an agent proposal, read:
+
+1. [Current Runtime Snapshot](../dev/current-runtime-snapshot.md)
+2. [Package Map](../dev/package-map.md)
+3. [Module System](../dev/module-system.md)
+4. [Igniter Lang Foundation](../guide/igniter-lang-foundation.md)
+5. [Horizon Protocol](./horizon-protocol.md)
+6. [Vision Handoff Protocol](./vision-handoff-protocol.md)
+7. [Project Status Horizon Report](./project-status-horizon-report.md)
+8. [Contract Persistence Landing Zone](./contract-persistence-landing-zone.md)
+9. [Contract Persistence Organic Model](./contract-persistence-organic-model.md)
+10. [Wizard Type Spec Research Request](./wizard-type-spec-research-request.md)
+11. [Wizard Type Spec Architecture](./wizard-type-spec-architecture.md)
+
+## Current Research State
+
+Status date: 2026-05-02.
+
+- Active package family: `igniter-contracts`, `igniter-extensions`,
+  `igniter-embed`, `igniter-application`, `igniter-ai`, `igniter-agents`,
+  `igniter-hub`, `igniter-web`, `igniter-cluster`, `igniter-mcp-adapter`.
+- `Igniter::Lang` is additive and report-only: Ruby backend wrapper,
+  immutable descriptors (`History`, `BiHistory`, `OLAPPoint`, `Forecast`),
+  `VerificationReport`, and `MetadataManifest`.
+- Lang metadata is declared, not enforced. `return_type`, `deadline`, `wcet`,
+  descriptors, stores, and invariants must graduate through reports before
+  runtime checks.
+- AI and agents have first package slices: provider-neutral AI envelopes and
+  minimal single-turn agent state. Tool contracts, memory, handoff, and human
+  gates remain future package work.
+- Companion is the strongest product pressure for persistence and agents. Its
+  app-local proof now covers records, histories, projections, command mutation
+  intents, typed effect intent descriptors, relation manifests, relation health diagnostics,
+  registry/readiness/manifest, `WizardTypeSpec`, static materialization planning,
+  and a review-only materializer/approval audit vertical.
+- `igniter-ledger` has shipped Phase 1 (LedgerServer, NetworkBackend, TCP/Unix transport,
+  graceful drain, stats, PID, CLI) and Phase 2 (SubscriptionRegistry reactive push).
+- `igniter-ledger` now has an Open Protocol proposal: descriptor/fact/receipt/query
+  waist for non-Igniter clients as well as Igniter contracts.
+- `igniter-durable-model` is now the typed Record/History facade over
+  `igniter-ledger`, carrying Companion manifest pressure toward package-level
+  Store/History experiments.
+- Cluster owns distributed placement, routing, ownership, health, remediation,
+  and mesh attempts. Contracts remain local-first and host-agnostic.
+
+## Rotation Policy
+
+When a research document's subject is fully implemented and shipped:
+1. `git mv` it to `packages/<pkg>/docs/` (or `docs/store/` for cross-package convergence records)
+2. Add a row to the **Rotated to Package Docs** table above
+3. Remove it from the **Index** and **Read First** list
+4. Keep governance docs (protocols, long-horizon decisions, landing zones, roadmaps) here permanently
+
+## Compression Rule
+
+Each research document should preserve:
+
+- one claim
+- the boundary it changes
+- the evidence or source pressure
+- the smallest reversible next move
+- the reasons not to ship it yet

data/docs/research/contract-persistence-landing-zone.md

@@ -0,0 +1,126 @@
+# Contract Persistence Landing Zone
+
+Status date: 2026-04-29.
+Scope: research placement decision. Not a package commitment or public API.
+
+## Claim
+
+Contract persistence now deserves a named landing path, but not an immediate
+package split.
+
+The Companion proof is no longer just CRUD convenience. It shows a recursive
+system shape: contracts declare durable capabilities, inspect their own
+infrastructure, produce materializer review packets, lower approvals into
+history append intents, and expose audit read models for the process that may
+later materialize more contracts.
+
+That fractal signal is strong evidence for the direction. It is still extraction
+pressure, not an API-promotion trigger.
+
+## Decision
+
+Keep the next iteration app-local in Companion.
+
+When extraction starts, split by responsibility:
+
+- `igniter-extensions`: optional contract vocabulary and report-only
+  descriptors over `igniter-contracts`
+- `igniter-application`: host/runtime binding, adapters, setup/readiness
+  surfaces, explicit mutation boundaries, materializer review flows
+- future `igniter-persistence`: stable durable capability package after repeated
+  app pressure and a smaller adapter/manifest contract
+
+Do not create `igniter-data` yet. The name is too broad: it can mean dataflow,
+analytics, datasets, ETL, tables, events, or persistence. The sharper future
+name is `igniter-persistence`, because the capability is durable `Store[T]`,
+append-only `History[T]`, typed relations, commands, materialization, and audit.
+
+Do not put this in `igniter-contracts` now. The kernel should stay
+host-agnostic; persistence remains optional vocabulary and app-boundary
+behavior until the lowerings are clear.
+
+## Landing Map
+
+`igniter-extensions` owns:
+
+- `persist` / `history` as optional Ruby product sugar
+- future `Store[T]` / `History[T]` / `Relation[...]` descriptors
+- field/index/scope/command metadata descriptors
+- relation manifest shape
+- operation-intent metadata shape
+- report-only validation and diagnostics
+
+`igniter-application` owns:
+
+- capability registry/factory
+- app-local record/history bindings
+- adapter wiring and backend choice
+- explicit write boundary
+- setup/readiness/materializer surfaces
+- approval/attempt receipt persistence
+- application lifecycle around generated or materialized files
+
+Future `igniter-persistence` owns only after stabilization:
+
+- adapter contract
+- migration/change-plan contract
+- durable capability runtime APIs
+- relation enforcement policy
+- index/partition/placement metadata
+- cross-app persistence test kit
+
+## Maturity Ladder
+
+1. Companion app-local proof: current state.
+2. Companion internal stabilization: freeze manifest vocabulary, operation
+   algebra, relation semantics, approval lifecycle, and status packets.
+3. Split-shaped extraction without package promise: isolate extension-shaped and
+   application-shaped modules inside the app or example boundary.
+4. Optional shared packs: move descriptors to `igniter-extensions` and host
+   bindings to `igniter-application` only when another app/example repeats the
+   pressure.
+5. `igniter-persistence`: create only when adapter semantics, migrations,
+   relation policy, and Store/History lowerings have stable tests.
+6. `Igniter::Lang`: lower `persist` to `Store[T]`, `history` to `History[T]`,
+   and relation manifests to typed relation descriptors.
+
+## Next Slice
+
+Best next move:
+
+- keep Companion as the proving ground
+- stabilize a canonical manifest schema with `schema_version` and
+  `storage.shape`
+- keep `persist`/`history` aliases for current app-local compatibility
+- make operation algebra and relation metadata boring and explicit
+- keep materializer execution blocked behind human approval and audited
+  app-boundary writes
+- avoid SQL planners, migrations, FK generation, cascade semantics, and relation
+  enforcement until the descriptors stop moving
+
+Acceptance:
+
+- a new agent can read the manifest and know what is durable, append-only,
+  projected, related, command-driven, blocked, or approved
+- no setup/read endpoint mutates durable state
+- all mutation still flows through explicit command intent plus app boundary
+- all terms still lower cleanly to future `Store[T]` / `History[T]`
+
+## Handoff
+
+```text
+[Architect Supervisor / Codex]
+Track: docs/research/contract-persistence-landing-zone.md
+Status: persistence has a reserved landing path, not a new package yet.
+[D] Keep proof app-local in Companion for the next stabilization slice.
+[D] First extraction, when justified, is split: contract vocabulary to
+igniter-extensions; host/runtime/adapters/materializer boundary to
+igniter-application.
+[D] Reserve igniter-persistence as the future package name; avoid igniter-data.
+[R] Do not promote persistence DSL into igniter-contracts from this proof alone.
+[R] Treat fractality as a positive architecture signal, not a promotion trigger.
+[S] Preserve persist -> Store[T], history -> History[T], and typed relation
+lowerings.
+Next: stabilize manifest vocabulary, operation algebra, relation semantics, and
+approval/materializer lifecycle while staying app-local.
+```