igniter 0.5.1 → 0.5.2

data/docs/research/horizon-protocol.md

@@ -0,0 +1,180 @@
+# Horizon Protocol
+
+Status: active research protocol. Not a stable API, package plan, or release
+promise.
+
+Mission: take one step beyond the horizon while preserving the ability to build
+Igniter Lang, distributed contracts, agent-native interfaces, and compact human
+tools without turning research energy into accidental runtime commitments.
+
+## Axioms
+
+- The contract graph is the kernel. Grammar, UI, agents, storage, and cluster
+  behavior must lower to explicit graph semantics or a clearly owned layer above
+  it.
+- Ruby DSL is the reference implementation for Lang. A standalone grammar waits
+  for stable DSL signals from real applications.
+- Metadata precedes enforcement. The ladder is: declare -> report -> warn ->
+  reject/execute.
+- Type descriptors imply capability requirements. `Store[T]`, `History[T]`,
+  `BiHistory[T]`, `OLAPPoint[T,dims]`, and `Forecast[T]` should feed manifests
+  before they become schedulers, adapters, or parsers.
+- Package ownership beats conceptual elegance. If an idea cannot name its owner,
+  it stays research.
+- Human sugar and agent-clean forms must stay equivalent. Every compact DSL form
+  needs an explicit object/configuration representation.
+- Interaction and handoff require policy, evidence, and receipt. Do not turn an
+  affordance into autonomous execution by implication.
+- Public docs are cache; private research is archive. Promote only compressed,
+  current decisions.
+
+## Planning Loop
+
+Use four planning horizons:
+
+- Horizon: years; preserves optionality and names non-negotiable invariants.
+- Track: weeks; package or doctrine direction with ownership and acceptance.
+- Slice: one to three days; narrow proof with tests or review artifact.
+- Patch: hours; code or document delta with verification.
+
+Every active slice should fit this frame:
+
+```text
+Aim:
+Boundary:
+Current evidence:
+Smallest move:
+Lang impact: opens | preserves | narrows | closes
+Reversibility:
+Verification:
+Next owner:
+```
+
+`Lang impact: closes` requires explicit architect acceptance.
+
+## Architect Frame
+
+The architect reviews every serious proposal through nine lenses:
+
+- Semantics: what meaning enters the contract graph?
+- Types: what can be declared, inferred, checked, or lowered?
+- Time: is this point state, history, bitemporal state, forecast, or deadline?
+- Data: who owns durable shape, query shape, lineage, and materialization?
+- Agency: who acts, under what policy, with what trace and receipt?
+- Distribution: what is local, placed, routed, replicated, or eventually merged?
+- Interface: what is human-facing sugar, agent-facing structure, or transport?
+- Game theory: what incentives or failure modes appear when actors optimize?
+- Constraints: what bottleneck, invariant, or irreversibility dominates?
+
+Architect output should be terse:
+
+```text
+[D] Decision:
+[R] Rule:
+[S] Current status:
+[H] Cold history pointer:
+Risk:
+Next slice:
+```
+
+## Agent Roles
+
+Architect:
+
+- owns invariants, package placement, and future-option preservation
+- accepts, narrows, rejects, or defers proposals
+- names what must remain report-only
+
+Researcher:
+
+- explores private and public context
+- returns compressed claims, assumptions, deltas, and non-goals
+- does not create runtime code or public APIs from research-only material
+
+Implementer:
+
+- changes the smallest owned surface
+- keeps app-local pressure app-local until repeated proof exists
+- includes tests, examples, or smoke evidence
+- reports changed files and verification
+
+Reviewer:
+
+- checks boundary leaks, future-option loss, weak legacy preservation, missing
+  tests, and documentation inflation
+- treats `igniter-lang` compatibility as an architectural invariant
+
+Product Pressure Agent:
+
+- uses Companion and examples to discover real friction
+- may add narrow app-local sugar
+- records repeated pressure before asking for package promotion
+
+## Igniter Lang Decision Filter
+
+Green moves:
+
+- add immutable descriptors, manifests, reports, or read-only diagnostics
+- make backend seams more explicit without changing runtime behavior
+- keep new semantics representable as explicit objects before sugar
+- lower `persist`-style ergonomics toward `Store[T]` or `History[T]`
+- add tests that prove existing execution remains unchanged
+
+Yellow moves:
+
+- add a DSL keyword before two real examples need it
+- enforce a field that was previously report-only
+- introduce storage behavior without a requirements manifest
+- add agent memory, handoff, tools, or gates without policy and receipts
+- make web/application surfaces inspect or execute each other's internals
+
+Red moves:
+
+- standalone grammar or `.il` parser before the Ruby DSL friction log stabilizes
+- distributed routing, placement, or replication inside `igniter-contracts`
+- provider-specific AI calls inside application examples
+- web rendering that owns agent runtime logic
+- hidden boot mutation or package auto-loading that changes lower-layer meaning
+- preserving weak transitional structure only because it already exists
+
+## Review Checklist For Other Agents
+
+Ask these before accepting another agent's work:
+
+- Owner: which package, app, or doc lane owns the change?
+- Lowering: can the shape lower to contracts, manifest entries, or an owned
+  higher layer?
+- Lang: does it open, preserve, narrow, or close the path to Igniter Lang?
+- Enforcement: is the change declare/report/warn/reject/execute, and is that
+  phase justified?
+- Dual form: is there both explicit agent-clean structure and optional human
+  sugar?
+- Evidence: what spec, example, smoke, receipt, or report proves the boundary?
+- Compression: did the docs replace stale state instead of adding replay?
+- Privacy: did private research stay private unless deliberately compressed?
+
+## Near-Term Horizon Map
+
+Now:
+
+- keep `Igniter::Lang` report-only and additive
+- use Companion as app-local pressure for persistence, records, histories, and
+  agent value
+- keep AI provider execution in `igniter-ai`, agent state in `igniter-agents`,
+  application wiring in `igniter-application`, and rendering in `igniter-web`
+
+Next:
+
+- extend metadata manifests toward stores and invariant coverage
+- stabilize contracts-first tool calls, human gates, and handoff receipts for
+  agents
+- harvest repeated Companion persistence shapes into an owned package slice
+- maintain a Ruby DSL friction log before grammar work
+
+Later:
+
+- formal AST shared by DSL and future parser
+- enforced invariants and type refinements for the decidable fragment
+- `Store[T]` / `History[T]` lowering into storage requirement manifests
+- OLAP/fanout planning owned above the contracts kernel
+- Rust/certification backend only after real-time or formal-export pressure

data/docs/research/igniter-lang-convergence-report.md

@@ -0,0 +1,338 @@
+# Igniter Lang Convergence Report
+
+Status: public research synthesis
+Date: 2026-05-04
+Source horizon: archived `playgrounds/docs/experts/igniter-lang`
+Current evidence: `igniter-contracts`, `igniter-ledger`, `igniter-ledger-client`,
+and `igniter-durable-model`
+
+## Claim
+
+Igniter is not moving toward a separate language by first designing syntax.
+It is moving toward a contract-native platform by proving semantics inside the
+Ruby DSL first.
+
+The strongest direction from the old Igniter-Lang research still holds:
+
+```text
+contract graph
+-> typed descriptors
+-> durable model shapes
+-> access paths and time
+-> materialization as contracts
+-> cluster placement and execution later
+```
+
+The current implementation has validated the first durable lane:
+
+```text
+Record        -> Store[T]
+History       -> History[T]
+scope         -> query/access path
+receipt       -> normalized mutation result
+manifest      -> generated model class
+LedgerClient  -> remote protocol boundary
+Ledger        -> current fact engine / WAL / changefeed
+```
+
+This supports the package identity decision:
+
+```text
+packages/igniter-durable-model   package name
+Igniter::DurableModel            canonical Ruby namespace
+```
+
+`Companion` should remain an app/product/example name. The package is becoming
+Igniter's typed durable model layer.
+
+## How We Are Moving
+
+### 1. Ruby DSL First, Grammar Later
+
+The old implementation track said the Ruby DSL should be the reference
+implementation and grammar should wait until semantics are stable.
+
+Current movement matches this.
+
+Evidence:
+
+- core contracts still run through the Ruby compiler/runtime
+- `Igniter::Lang` descriptors are report-only
+- `History`, `BiHistory`, `OLAPPoint`, and `Forecast` exist as metadata before
+  runtime enforcement
+- Ledger and Durable Model work are proving real pressure before a new grammar
+
+Decision:
+
+Do not design `.il` syntax yet. Keep extracting stable semantics from Ruby usage.
+
+### 2. Durable Model, Not Database Abstraction
+
+The persistence research framed persistence as a type/access-pattern property:
+`Store[T]`, `History[T]`, `OLAPPoint[T]`, saga logs, caches, and rule sets each
+declare their required physical behavior.
+
+Current movement partially validates this:
+
+- `igniter-ledger` owns hot fact physics: immutable facts, WAL, causation,
+  query, changefeed, retention, compaction, and server delivery
+- the current `igniter-durable-model` package owns typed application ergonomics:
+  Record, History, scopes, receipts, and manifest-generated model classes
+- `igniter-ledger-client` owns protocol and transport only
+
+Decision:
+
+Promote the package concept as **Durable Model**, not as a database wrapper and
+not as `LedgerClient`.
+
+### 3. History First, OLAP Later
+
+The OLAP research makes a strong claim:
+
+```text
+History[T] == OLAPPoint[T, { time: DateTime }]
+```
+
+Current movement is starting from the simpler 1D form:
+
+- append-only History
+- replay
+- partition keys
+- time-travel reads
+- changefeed
+- retention/compaction boundary
+
+This is the right order. History is the smallest useful proof of the larger
+OLAP model.
+
+Decision:
+
+Keep OLAP as a horizon, but continue building facts/history/compaction first.
+Future OLAP should grow from proven History segments, summaries, projections,
+and boundary/compaction receipts.
+
+### 4. Invariants as Reports Before Enforcement
+
+The invariant research says invariants are global contracts and refinement-type
+claims, not just local guards.
+
+Current movement follows the safe version:
+
+- setup/readiness/health packets are report-only
+- storage plans, migration plans, field/type plans, relation type plans, access
+  path plans, and effect intent plans are non-executing descriptors
+- materializer approval remains review-only and grants no capabilities
+
+Decision:
+
+Do not rush invariant enforcement. First make invariant-like reports observable,
+stable, and useful. Enforcement should graduate only after the report vocabulary
+is boring and repeated.
+
+### 5. Materialization as a Contract
+
+The old research repeatedly treats materialization as a contract: operational
+facts feed analytical or static artifacts through a declared, inspectable graph.
+
+Current movement strongly matches this:
+
+- `WizardTypeSpec` keeps dynamic specs durable
+- static materialization is modeled as plan/parity/preflight/runbook/receipt
+- materializer attempts and approvals are auditable
+- no file/git/test/restart capability is granted by read endpoints
+
+Decision:
+
+Keep materialization as "print to code plus receipts", not live dynamic runtime
+magic. A future materializer should be a capability-gated contract/agent that
+prints static contracts, runs checks, and records evidence.
+
+### 6. Reactive Delivery Is the Next Practical Bridge
+
+The temporal and persistence research both point toward causal/as-of reads,
+events, and reactive state.
+
+Current movement has the engine-side ingredients:
+
+- Ledger changefeed
+- SSE `/v1/events`
+- replay cursors
+- async fan-out
+- observability and diagnostics
+
+Current Durable Model movement is now connecting application ergonomics:
+
+- client-backed `scope` works through `LedgerClient#query`
+- next slice is client-backed `on_scope` through Ledger Client events
+
+Decision:
+
+Treat reactive scopes as the near-term bridge from facts to application value.
+This gives ordinary apps immediate use while preserving the long-term temporal
+model.
+
+## Valuable Ideas To Carry Forward
+
+### Durable Shape Taxonomy
+
+Keep this as the north star:
+
+```text
+Store[T]       record-like point reads/writes
+History[T]     append-only log, replay, time ranges
+BiHistory[T]   corrections and valid/transaction time
+OLAPPoint[T]   multi-dimensional analytical value
+WorkflowLog    await/saga execution state
+Cache[T]       ephemeral/coalesced derived value
+RuleSet[T]     versioned rules and policy
+```
+
+Do not implement all at once. Use the taxonomy to avoid painting `Store[T]`,
+`History[T]`, projections, receipts, and adapter storage into one object model.
+
+### Type-Directed Storage Requirements
+
+Keep the idea that the model can emit requirements:
+
+```text
+shape
+access pattern
+consistency need
+partition strategy
+retention/compaction policy
+materialization strategy
+```
+
+Current manifest work is the first public-safe version of this.
+
+### `as_of` as a First-Class Read Semantics
+
+`as_of` should remain central. It is already useful in Ledger reads and Durable
+Model scopes, and later it connects to temporal rules, bitemporal history, and
+causal consistency.
+
+### Compaction/Boundary as Semantic Compression
+
+The recent boundary/container work maps well to old HistorySegment thinking:
+
+```text
+raw facts
+-> sealed boundary/container
+-> derived summaries/checks/projections
+-> optional purge after references and policy allow it
+```
+
+This is the practical path toward long-lived ledgers that do not grow without
+limit.
+
+### Conflict Resolution Shares Vocabulary With Rules
+
+The old idea that write conflicts and rule conflicts both use `combines:` is
+still valuable. Keep it as a design guardrail for future concurrent writes,
+OLAP rollups, and multi-writer histories.
+
+### Property Models and Agents
+
+The property-model research is still early, but one idea is useful now:
+
+```text
+ontology/spec -> generated contract/model -> verified runtime behavior
+```
+
+This aligns with `WizardTypeSpec`, generated Record/History classes, and future
+agent-assisted materialization. LLMs should author specs and proposals first,
+then a gated materializer turns accepted specs into static code.
+
+### Probabilistic Precomputation as Later Optimization
+
+The approximate/precompute research is valuable but not immediate.
+
+Carry forward only the principle:
+
+```text
+consumer precision should drive computation cost
+```
+
+Near-term equivalents are cache TTL, projection freshness, summary confidence,
+and approximate analytical reads. Do not add probabilistic runtime semantics yet.
+
+## Ideas To Defer
+
+These remain good research but should not steer near-term package design:
+
+- standalone grammar and parser
+- full property-model synthesis
+- formal invariant verifier
+- Rust backend for contracts
+- OLAPPoint runtime
+- probabilistic abstract domains
+- cluster-wide causal clocks and vector-clock reads
+- automated placement/fanout compiler
+- cross-store transactions
+
+Each should return only after a small Ruby/package proof creates repeated
+pressure.
+
+## Package Direction
+
+Recommended target map:
+
+```text
+core igniter / igniter-contracts
+  contract graph, compiler, runtime, effects, diagnostics
+
+packages/igniter-ledger
+  hot durable fact engine:
+  facts, WAL, causation, query, changefeed, compaction, server
+
+packages/igniter-ledger-client
+  protocol and transports:
+  write/read/append/replay/query/subscribe, HTTP/SSE/object dispatch
+
+packages/igniter-durable-model
+  typed durable application model layer:
+  Record, History, scopes, receipts, manifest-generated classes,
+  later entity/OLAP/workflow-state vocabulary
+
+examples/application/companion
+  product/app proof:
+  UI, setup packets, materializer playground, user-facing experiments
+```
+
+Rename path:
+
+1. Finish the active Ledger Client subscription slice.
+2. Add `Igniter::DurableModel` namespace in the current package.
+3. Keep `Igniter::Companion` aliases temporarily.
+4. Update docs/examples to use Durable Model.
+5. Rename package/gem directory once tests are stable.
+
+## Smallest Next Moves
+
+1. Land client-backed `on_scope` over Ledger Client events.
+2. Start the `Igniter::DurableModel` namespace shim.
+3. Rename package docs from Companion identity to Durable Model identity.
+4. Define the first public Durable Model README around `Record`, `History`,
+   `Store`, `scope`, `receipt`, and `client:`.
+5. Keep OLAP/time-machine/property synthesis as horizon notes, not implementation
+   tracks.
+
+## Public Boundary
+
+This report is not a public API promise.
+
+Accepted public direction:
+
+- Ruby DSL proves semantics before grammar.
+- Durable models are separate from Ledger engine and Ledger Client transport.
+- Ledger is the current hot fact substrate.
+- Durable Model is the application-facing typed layer.
+
+Not accepted yet:
+
+- stable `igniter-durable-model` gem API
+- `Store[T]` / `History[T]` as core language syntax
+- OLAPPoint runtime
+- invariant enforcement
+- automatic materializer write capabilities
+- standalone Igniter language grammar