@its-not-rocket-science/ananke 0.1.69 → 0.5.4

package/docs/wire-protocol.md

@@ -1,209 +1,140 @@
-# Ananke — Wire Protocol & Save Format
-
-This document specifies how Ananke state is serialised for persistence, replay, and
-network transport.  All formats are deterministic: the same simulation state always
-produces the same bytes.
-
----
-
-## 1. Concepts
-
-| Term | Meaning |
-|------|---------|
-| **Snapshot** | A serialised `WorldState` — complete enough to resume simulation |
-| **Replay** | An initial snapshot + a sequence of command frames |
-| **Diff** | A compact binary diff between two consecutive snapshots (CE-9) |
-| **Wire message** | A single unit transmitted between host and client over the network |
-| **Q value** | A fixed-point integer scaled by `SCALE.Q = 10 000` (e.g. `q(0.75) = 7500`) |
-
----
-
-## 2. JSON Snapshot Format
-
-JSON is the recommended format for long-term save files and editor tooling.
-
-### 2.1 Deterministic key ordering
-
-When computing hash-checks across clients, keys must appear in insertion order.
-The canonical TypeScript implementation (`JSON.stringify`) preserves insertion
-order for string keys.  Third-party deserializers must preserve or sort keys
-identically.
-
-### 2.2 Q values
-
-All `Q`-typed fields are serialised as plain integers.  Do **not** divide by
-`SCALE.Q` before saving — the raw integer is the canonical representation.
-
-```json
-{ "fearQ": 7500 }    // correct — q(0.75)
-{ "fearQ": 0.75 }    // WRONG — will cause precision loss and replay divergence
-```
-
-### 2.3 Maps
-
-JavaScript `Map` instances do not serialise to JSON automatically.  Ananke
-serialises `Map<K, V>` as an array of `[K, V]` pairs:
-
-```json
-{ "__nutritionAccum": 0 }
-```
-
-> Note: `__nutritionAccum` was simplified to a scalar in v0.1.  If a `Map`
-> field is added in a future version, its pairs will use the array format above.
-
-### 2.4 Version stamping
-
-Always call `stampSnapshot(world, "world")` before persisting.  This adds
-`_ananke_version` and `_schema` fields that enable forward migration:
-
-```typescript
-import { stampSnapshot } from "@its-not-rocket-science/ananke/schema";
-// or: import { stampSnapshot } from "@ananke/core";   (when published)
-
-const save = JSON.stringify(stampSnapshot(world, "world"), null, 2);
-```
-
-### 2.5 JSON Schema files
-
-Canonical schemas ship with the package:
-
-| File | Validates |
-|------|-----------|
-| `schema/world.schema.json` | `WorldState` snapshots |
-| `schema/replay.schema.json` | `Replay` objects |
-
-Use `validateSnapshot(raw)` from `@its-not-rocket-science/ananke/schema` to
-check conformance programmatically before calling `stepWorld`.
-
----
-
-## 3. Binary Diff Format
-
-For tick-to-tick state synchronisation (multiplayer, streaming), use the binary
-diff format implemented in `src/snapshot.ts`.
-
-### 3.1 Encoding
-
-```
-[magic: "ANKD" (4 bytes)] [version: 1 (u8)] [payload: tag-value stream]
-```
-
-Tag values:
-
-| Tag | Byte | Encodes |
-|-----|------|---------|
-| NULL | 0x00 | `null` |
-| TRUE | 0x01 | `true` |
-| FALSE | 0x02 | `false` |
-| UINT8 | 0x10 | Unsigned integer 0–255 |
-| INT32 | 0x11 | Signed 32-bit integer (big-endian) |
-| FLOAT64 | 0x12 | IEEE 754 double (big-endian) — use only for non-Q floats |
-| STRING | 0x20 | Length-prefixed UTF-8 |
-| ARRAY | 0x30 | Length-prefixed sequence of tag-value items |
-| OBJECT | 0x40 | Length-prefixed sequence of (string key, tag-value) pairs |
-
-### 3.2 Usage
-
-```typescript
-import { diffWorldState, packDiff, unpackDiff, applyDiff } from "@its-not-rocket-science/ananke";
-
-// Sender
-const diff   = diffWorldState(prevState, nextState);
-const bytes  = packDiff(diff);
-socket.send(bytes);
-
-// Receiver
-const diff2  = unpackDiff(bytes);
-const state2 = applyDiff(prevState, diff2);
-```
-
-### 3.3 Determinism guarantee
-
-A diff produced from identical states must produce identical bytes.  Do not
-include wall-clock timestamps or random nonces in diff payloads.
-
----
-
-## 4. Multiplayer Message Protocol
-
-For lockstep multiplayer, hosts exchange command frames rather than full state.
-
-### 4.1 Message types
-
-| `kind` | Direction | Payload |
-|--------|-----------|---------|
-| `"cmd"` | Client → Server | `{ tick, commands: Command[] }` |
-| `"ack"` | Server → Client | `{ tick, stateHash: number }` |
-| `"resync"` | Server → Client | `{ tick, snapshot: WorldState }` |
-| `"hash_mismatch"` | Server → Client | `{ tick, expected: number, got: number }` |
-
-### 4.2 State hash
-
-Use the built-in tick counter and entity count as a cheap hash for divergence
-detection:
-
-```typescript
-function stateHash(world: WorldState): number {
-  return world.tick * 0x10000 + (world.entities.length & 0xFFFF);
-}
-```
-
-A full structural hash is more robust but expensive; use it only on resync.
-
-### 4.3 Lockstep loop
-
-```
-┌──────────────────────────────────────────────────────────┐
-│ Client                       Server                      │
-│                                                          │
-│ collect commands ──── cmd ──► apply to authoritative     │
-│                               state                      │
-│                   ◄── ack ─── broadcast stateHash        │
-│ verify hash                                              │
-│ if mismatch ─── resync req ─► send full snapshot        │
-│                ◄── resync ──                             │
-│ restore snapshot                                         │
-└──────────────────────────────────────────────────────────┘
-```
-
-### 4.4 Transport encoding
-
-Use JSON for development and debugging.  For production, encode wire messages
-as CBOR (RFC 8949) or MessagePack for ~30% size reduction.  The message
-structure is identical; only the outer encoding changes.
-
----
-
-## 5. Save File Recommendations
-
-| Scenario | Format | Compression |
-|----------|--------|-------------|
-| Development / debugging | JSON (pretty-printed) | none |
-| Production saves | JSON (compact) | gzip or zstd |
-| Network sync (full state) | JSON or CBOR | none (already compact) |
-| Network sync (incremental) | Binary diff (`packDiff`) | none |
-| Replay archives | JSON replay schema | zstd |
-
----
-
-## 6. Migration
-
-Load a save and bring it to the current schema version before simulating:
-
-```typescript
-import {
-  migrateWorld, validateSnapshot, stampSnapshot,
-} from "@its-not-rocket-science/ananke/schema";
-
-function loadSave(json: string): WorldState {
-  const raw      = JSON.parse(json) as Record<string, unknown>;
-  const migrated = migrateWorld(raw);          // no-op until 0.2 is released
-  const errors   = validateSnapshot(migrated);
-  if (errors.length > 0) {
-    throw new Error(`Invalid save: ${errors.map(e => `${e.path}: ${e.message}`).join("; ")}`);
-  }
-  return migrated as WorldState;
-}
-```
-
-See `docs/migration-monolith-to-modular.md` for package-level migration guidance.
+# Ananke Wire Protocol & Schema Formats
+
+This page documents only what is currently shipped in code. It avoids treating roadmap ideas as current protocol guarantees.
+
+For maintainer promises/non-promises and pinning recommendations, see `docs/support-boundaries.md`.
+
+Status badge legend:
+- 🟢 **Implemented + public**: implemented and exported on documented package entrypoints.
+- 🟡 **Partial**: implemented API exists, but completeness/coverage is intentionally limited.
+- 🟠 **Implemented + internal**: implemented and exported only on internal/advanced surfaces.
+- 🔵 **Planned**: not shipped as a canonical helper/module.
+
+## Compatibility table (current)
+
+| Surface | Status | Current contract |
+|---|---|---|
+| Save format | 🟢 Implemented + public | JSON snapshots stamped with `_ananke_version`/`_schema` via `stampSnapshot` on `@its-not-rocket-science/ananke/schema` (and `./schema-migration` alias). |
+| Replay format | 🟢 Implemented + public | `Replay` JSON via `serializeReplay` / `deserializeReplay` on root entrypoint `@its-not-rocket-science/ananke`. |
+| Schema migration | 🟡 Partial | `migrateWorld` + `registerMigration` are shipped; only explicitly registered migration edges run (no built-in universal chain). |
+| Binary diff | 🟠 Implemented + internal | `diffWorldState` / `packDiff` / `unpackDiff` / `applyDiff` are shipped on `@its-not-rocket-science/ananke/tier3` (advanced/internal surface). |
+| Lockstep messages | 🔵 Planned | No canonical exported lockstep message envelope/type module. Hosts define their own message schema today. |
+
+Versioning note: production hosts should pin exact patch versions for schema migration and any non-root protocol helpers.
+
+## 1) Save snapshots (JSON)
+
+**Status:** 🟢 Implemented + public
+
+Shipped helpers:
+- `stampSnapshot(snapshot, schemaKind)`
+- `validateSnapshot(snapshot)`
+- `detectVersion(snapshot)`
+- `isValidSnapshot(snapshot)`
+
+Entrypoints:
+- `@its-not-rocket-science/ananke/schema`
+- `@its-not-rocket-science/ananke/schema-migration` (alias)
+
+Example:
+
+```ts
+import { stampSnapshot, validateSnapshot } from "@its-not-rocket-science/ananke/schema";
+
+const world = {} as Parameters<typeof stampSnapshot>[0];
+const stamped = stampSnapshot(world, "world");
+const errors = validateSnapshot(stamped);
+if (errors.length > 0) throw new Error("invalid snapshot");
+const json = JSON.stringify(stamped);
+```
+
+Notes:
+- `validateSnapshot` validates core structural fields required by simulation, and permits extra host fields.
+- Canonical schema files are shipped in `schema/world.schema.json` and `schema/replay.schema.json`.
+
+## 2) Replay format
+
+**Status:** 🟢 Implemented + public
+
+Shipped helpers (root entrypoint):
+- `ReplayRecorder`
+- `replayTo`
+- `serializeReplay`
+- `deserializeReplay`
+
+Example:
+
+```ts
+import { ReplayRecorder, serializeReplay, deserializeReplay, replayTo } from "@its-not-rocket-science/ananke";
+
+const world = {} as ConstructorParameters<typeof ReplayRecorder>[0];
+const ctx = {} as Parameters<typeof replayTo>[2];
+const recorder = new ReplayRecorder(world);
+// ... record frames while stepping
+const json = serializeReplay(recorder.toReplay());
+const replay = deserializeReplay(json);
+const worldAt100 = replayTo(replay, 100, ctx);
+```
+
+Notes:
+- Replay JSON includes custom Map handling used by serializer/reviver.
+- This is the shipped deterministic replay contract.
+
+## 3) Schema migration
+
+**Status:** 🟡 Partial
+
+Shipped helpers:
+- `registerMigration(fromVersion, toVersion, fn)`
+- `migrateWorld(snapshot, toVersion?)`
+
+Behavior today:
+- If snapshot version equals target version, `migrateWorld` returns unchanged.
+- If no migration is registered for the requested edge, `migrateWorld` throws.
+- Legacy snapshots without `_ananke_version` are treated as `"0.0"`.
+
+This is a real API, but migration coverage is only as complete as registered migration paths.
+
+## 4) Binary diff format
+
+**Status:** 🟠 Implemented + internal
+
+Shipped helpers (Tier 3 surface):
+- `diffWorldState(prev, next)`
+- `packDiff(diff)`
+- `unpackDiff(bytes)`
+- `applyDiff(base, diff)`
+
+Entrypoint:
+- `@its-not-rocket-science/ananke/tier3`
+
+Scope:
+- Useful for incremental state transport/storage.
+- Not part of Tier-1 root stability contract.
+
+## 5) Lockstep message protocol
+
+**Status:** 🔵 Planned
+
+Current code provides deterministic primitives (`stepWorld`, replay, and `hashWorldState` in netcode), but **does not** provide a canonical exported lockstep message protocol module with fixed message kinds/envelopes.
+
+Roadmap/proposed protocol details have been moved to `docs/planned-protocol-work.md`.
+
+## 6) Helper/path audit (docs ↔ code)
+
+| Item | Exists in code | Exported | Classification |
+|---|---|---|---|
+| `stampSnapshot` (`src/schema-migration.ts`) | yes | `./schema`, `./schema-migration` | implemented + public |
+| `validateSnapshot` (`src/schema-migration.ts`) | yes | `./schema`, `./schema-migration` | implemented + public |
+| `migrateWorld` (`src/schema-migration.ts`) | yes | `./schema`, `./schema-migration` | partial |
+| `registerMigration` (`src/schema-migration.ts`) | yes | `./schema`, `./schema-migration` | implemented + public |
+| `serializeReplay` (`src/replay.ts`) | yes | root `.` | implemented + public |
+| `deserializeReplay` (`src/replay.ts`) | yes | root `.` | implemented + public |
+| `diffWorldState` (`src/snapshot.ts`) | yes | `./tier3` | implemented + internal |
+| `packDiff` (`src/snapshot.ts`) | yes | `./tier3` | implemented + internal |
+| `unpackDiff` (`src/snapshot.ts`) | yes | `./tier3` | implemented + internal |
+| `applyDiff` (`src/snapshot.ts`) | yes | `./tier3` | implemented + internal |
+| `schema/world.schema.json` | yes | package file artifact | implemented + public artifact |
+| `schema/replay.schema.json` | yes | package file artifact | implemented + public artifact |
+| Canonical lockstep message module | no | n/a | planned |
+

package/docs/world-evolution-backend.md

@@ -0,0 +1,200 @@
+# World Evolution Backend (Host Integration)
+
+## Purpose
+
+`world-evolution-backend` is an additive, deterministic composition layer for host platforms (for example OpenWorldBuilder) that need reproducible world-scale progression runs.
+
+It does **not** replace existing systems. It orchestrates existing modules and emits host-facing outputs:
+
+- final world snapshot
+- per-step timeline/event log
+- summary metrics
+- optional deltas/checkpoints
+
+## Stable import path
+
+Preferred additive subpath export:
+
+```ts
+import {
+  runWorldEvolution,
+  listAvailableWorldEvolutionProfiles,
+} from "@its-not-rocket-science/ananke/world-evolution-engine";
+```
+
+Compatibility alias (still supported):
+
+```ts
+import { runWorldEvolution } from "@its-not-rocket-science/ananke/world-evolution-backend";
+```
+
+No Tier-1 root exports were changed.
+
+## Host flow
+
+1. Build an initial `WorldEvolutionSnapshot` (canon state).
+2. Pick a profile (`minimal_world_history`, `polity_dynamics`, `conflict_heavy`, `climate_and_migration`, `full_world_evolution`) and optionally layer deterministic host overrides on top.
+3. Run `runWorldEvolution({ snapshot, steps, ... })`.
+4. Read:
+   - `finalSnapshot`
+   - `timeline`
+   - `metrics`
+   - optional `deltas`/`checkpoints`
+
+## Canon vs derived timelines
+
+- `initialSnapshot` is treated as canonical input.
+- simulation mutations occur on an isolated runtime clone.
+- `finalSnapshot` and timeline artifacts are derived outputs.
+
+This keeps lore/canon source state separate from generated timeline projections.
+
+## Composition architecture (additive)
+
+The backend composes existing systems in deterministic order per step:
+
+1. **Polity core**: `stepPolityDay` (trade, war, morale/stability baseline)
+2. **Governance**: `stepGovernanceCooldown`, `stepGovernanceStability`, `computeGovernanceModifiers`
+3. **Diplomacy**: treaty strength/expiry progression
+4. **Trade routes**: efficiency step + `applyDailyTrade`
+5. **Migration**: `resolveMigration` + `applyMigrationFlows`
+6. **Climate**: deterministic `generateClimateEvent`, aggregate effects, lifecycle stepping
+7. **Epidemic**: `stepEpidemic` + death pressure application
+
+All sequencing is deterministic and order-stable (sorted IDs, no `Math.random`).
+
+## Deterministic profile presets
+
+Profiles are explicit additive controls over existing modules and all share the same explicit pipeline order:
+
+1. `polity`
+2. `governance`
+3. `diplomacy`
+4. `trade`
+5. `migration`
+6. `climate`
+7. `epidemic`
+
+Preset intent:
+
+- `minimal_world_history`: only core polity day stepping.
+- `polity_dynamics`: polity + governance + diplomacy + trade.
+- `conflict_heavy`: polity/governance/diplomacy/trade/migration with no climate/epidemic load.
+- `climate_and_migration`: polity + migration + climate + epidemic pressure.
+- `full_world_evolution`: all currently integrated world-scale subsystems enabled.
+
+Legacy IDs (`balanced`, `resilience`, `expansion`) are kept as deterministic aliases to `full_world_evolution` for compatibility.
+
+### Profile matrix (explicit ruleset behavior)
+
+| Profile | Polity | Governance | Diplomacy | Trade | Migration | Climate | Epidemic | Intended host use |
+| --- | --- | --- | --- | --- | --- | --- | --- | --- |
+| `minimal_world_history` | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | Lowest-cost long-horizon history baselines |
+| `polity_dynamics` | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | Political/economic simulation without environmental load |
+| `conflict_heavy` | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | War/mobility-driven scenarios with reduced overhead |
+| `climate_and_migration` | ✅ | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ | Environmental stress, displacement, and disease pressure studies |
+| `full_world_evolution` | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | Full-stack world simulation for maximal systemic richness |
+
+All presets preserve the same deterministic subsystem execution order and only enable/disable or tune existing mechanics.
+
+## Host overrides layered on profile
+
+Hosts can provide:
+
+- `profileId` to select a preset
+- `ruleOverrides` to override specific fields deterministically
+
+Overrides are applied additively to the resolved profile; no random behavior is introduced.
+
+Example:
+
+```ts
+import {
+  resolveWorldEvolutionProfile,
+  runWorldEvolution,
+} from "@its-not-rocket-science/ananke/world-evolution-backend";
+
+const snapshot = {} as any;
+const baseProfile = resolveWorldEvolutionProfile("polity_dynamics");
+
+const result = runWorldEvolution({
+  snapshot,
+  steps: 90,
+  profile: {
+    ...baseProfile,
+    governanceStabilityDaysPerStep: 2,
+  },
+});
+```
+
+The resolved profile remains deterministic for a fixed snapshot/seed + override object.
+
+## Profile comparison example
+
+See `examples/world-evolution-profiles-comparison.ts` for a side-by-side run comparing multiple profiles over the same input state and seed.
+
+## Determinism notes
+
+- Uses fixed-point arithmetic and existing deterministic world primitives.
+- Iteration order is normalized by IDs before applying updates.
+- same snapshot + same profile + same step count => byte-for-byte equal result object.
+
+## Minimal API
+
+- `createWorldEvolutionSnapshot(snapshot)`
+- `runWorldEvolution(request)`
+- `listAvailableWorldEvolutionProfiles()`
+- plus profile helpers/types via the same subpath.
+
+
+
+## Host timeline/event layer for readable world history
+
+For UI-facing history rendering, use the presentation-oriented event stream builder:
+
+```ts
+import {
+  runWorldEvolution,
+  buildEvolutionTimeline,
+  sortTimelineEventsBySignificance,
+} from "@its-not-rocket-science/ananke/world-evolution-backend";
+import type { WorldEvolutionSnapshot } from "@its-not-rocket-science/ananke/world-evolution-backend";
+
+declare const snapshot: WorldEvolutionSnapshot;
+
+const result = runWorldEvolution({ snapshot, steps: 180, profileId: "full_world_evolution" });
+const timeline = buildEvolutionTimeline(result, { includeSummaryText: true });
+const highlights = sortTimelineEventsBySignificance(timeline).slice(0, 10);
+```
+
+`buildEvolutionTimeline` converts subsystem output into a **stable chronological stream** with:
+
+- deterministic `id` and `hash`
+- event categories (`polity`, `migration`, `conflict`, `diplomacy`, `economy`, `disease`, `climate`, `infrastructure`, `governance`, `mythology_culture`)
+- `severity` / `significance` scoring for host ranking
+- optional summary text for direct UI rendering
+- fact references (`factRefs`) pointing back to originating step/subsystem facts
+
+### Raw engine state vs presentation-oriented timeline events
+
+- **Raw engine state** (`finalSnapshot`, step metrics, per-step subsystem arrays):
+  - canonical simulation outputs
+  - optimized for correctness/replay/integrity
+  - rich but not directly UI-friendly
+
+- **Presentation timeline events** (`buildEvolutionTimeline` output):
+  - additive read-only projection layer
+  - normalized shape for host feeds, journals, and map overlays
+  - stable ordering + identity for caching/diffing in external platforms
+  - preserves links back to raw subsystem facts through `factRefs`
+
+This separation lets hosts keep simulation truth in snapshots while rendering an approachable “world history” layer for players.
+
+## Session orchestration layer
+
+For hosts that need a session-oriented API (`createEvolutionSession`, `stepEvolution`, serialization helpers), use `@its-not-rocket-science/ananke/world-evolution` and see `docs/world-evolution-orchestration.md`.
+
+
+## Host canonical schema
+
+For external platform integrators, see `docs/host-world-evolution-schema.md` and `schema/world-evolution-input.schema.json`.