@git-stunts/git-warp 10.8.0 → 11.3.3

package/README.md

@@ -5,12 +5,14 @@
 [![npm version](https://badge.fury.io/js/%40git-stunts%2Fgit-warp.svg)](https://www.npmjs.com/package/@git-stunts/git-warp)
 
 <p align="center">
-  <img src="hero.gif" alt="git-warp CLI demo" width="600">
+  <img src="docs/images/hero.gif" alt="git-warp CLI demo" width="600">
 </p>
 
-A multi-writer graph database that uses Git commits as its storage substrate. Graph state is stored as commits pointing to the empty tree (`4b825dc...`), making the data invisible to normal Git workflows while inheriting Git's content-addressing, cryptographic integrity, and distributed replication.
+## The Core Idea
 
-Writers collaborate without coordination using CRDTs (OR-Set for nodes/edges, LWW registers for properties). Every writer maintains an independent patch chain; materialization deterministically merges all writers into a single consistent view.
+**git-warp** is a graph database that doesn't need a database server. It stores all its data inside a Git repository by abusing a clever trick: every piece of data is a Git commit that points to the **empty tree** — a special object that exists in every Git repo. Because the commits don't reference any actual files, they're completely invisible to normal Git operations like `git log`, `git diff`, or `git status`. Your codebase stays untouched, but there's a full graph database living alongside it.
+
+Writers collaborate without coordination using CRDTs (Conflict-free Replicated Data Types) that guarantee deterministic convergence regardless of what order the patches arrive in.
 
 ```bash
 npm install @git-stunts/git-warp @git-stunts/plumbing
@@ -31,19 +33,18 @@ const graph = await WarpGraph.open({
   persistence,
   graphName: 'demo',
   writerId: 'writer-1',
-  autoMaterialize: true,  // auto-materialize on query
 });
 
-// Write data using the patch builder
-await (await graph.createPatch())
-  .addNode('user:alice')
-  .setProperty('user:alice', 'name', 'Alice')
-  .setProperty('user:alice', 'role', 'admin')
-  .addNode('user:bob')
-  .setProperty('user:bob', 'name', 'Bob')
-  .addEdge('user:alice', 'user:bob', 'manages')
-  .setEdgeProperty('user:alice', 'user:bob', 'manages', 'since', '2024')
-  .commit();
+// Write data — single await with graph.patch()
+await graph.patch(p => {
+  p.addNode('user:alice')
+    .setProperty('user:alice', 'name', 'Alice')
+    .setProperty('user:alice', 'role', 'admin')
+    .addNode('user:bob')
+    .setProperty('user:bob', 'name', 'Bob')
+    .addEdge('user:alice', 'user:bob', 'manages')
+    .setEdgeProperty('user:alice', 'user:bob', 'manages', 'since', '2024');
+});
 
 // Query the graph
 const result = await graph.query()
@@ -54,15 +55,21 @@ const result = await graph.query()
 
 ## How It Works
 
-Each writer creates **patches**: atomic batches of graph operations (add/remove nodes, add/remove edges, set properties). Patches are serialized as CBOR-encoded Git commit messages pointing to the empty tree, forming a per-writer chain under `refs/warp/<graphName>/writers/<writerId>`.
+### The Multi-Writer Problem (and How It's Solved)
+
+Multiple people (or machines, or processes) can write to the same graph **simultaneously, without any coordination**. There's no central server, no locking, no "wait your turn."
+
+Each writer maintains their own independent chain of **patches** — atomic batches of operations like "add this node, set this property, create this edge." These patches are stored as Git commits under refs like `refs/warp/<graphName>/writers/<writerId>`.
 
-**Materialization** replays all patches from all writers, applying CRDT merge semantics:
+When you want to read the graph, you **materialize** — which means replaying all patches from all writers and merging them into a single consistent view. The specific CRDT rules are:
 
-- **Nodes and edges** use an Observed-Remove Set (OR-Set). An add wins over a concurrent remove unless the remove has observed the specific add event.
-- **Properties** use Last-Write-Wins (LWW) registers, ordered by Lamport timestamp, then writer ID, then patch SHA.
-- **Version vectors** track causality across writers, ensuring deterministic convergence regardless of patch arrival order.
+- **Nodes and edges** use an OR-Set (Observed-Remove Set). If Alice adds a node and Bob concurrently deletes it, the add wins — unless Bob's delete specifically observed Alice's add. This is the "add wins over concurrent remove" principle.
+- **Properties** use LWW (Last-Writer-Wins) registers. If two writers set the same property at the same time, the one with the higher Lamport timestamp wins. Ties are broken by writer ID (lexicographic), then by patch SHA.
+- **Version vectors** track causality across writers so the system knows which operations are concurrent vs. causally ordered.
 
-**Checkpoints** snapshot materialized state into a single commit for fast incremental recovery. Subsequent materializations only need to replay patches created after the checkpoint.
+Every operation gets a unique **EventId** — `(lamport, writerId, patchSha, opIndex)` — which creates a total ordering that makes merge results identical no matter which machine runs them.
+
+**Checkpoints** snapshot the materialized state into a single commit for fast incremental recovery. Subsequent materializations only need to replay patches created after the checkpoint.
 
 ## Multi-Writer Collaboration
 
@@ -76,10 +83,9 @@ const graphA = await WarpGraph.open({
   writerId: 'alice',
 });
 
-await (await graphA.createPatch())
-  .addNode('doc:1')
-  .setProperty('doc:1', 'title', 'Draft')
-  .commit();
+await graphA.patch(p => {
+  p.addNode('doc:1').setProperty('doc:1', 'title', 'Draft');
+});
 
 // Writer B (on machine B)
 const graphB = await WarpGraph.open({
@@ -88,10 +94,9 @@ const graphB = await WarpGraph.open({
   writerId: 'bob',
 });
 
-await (await graphB.createPatch())
-  .addNode('doc:2')
-  .setProperty('doc:2', 'title', 'Notes')
-  .commit();
+await graphB.patch(p => {
+  p.addNode('doc:2').setProperty('doc:2', 'title', 'Notes');
+});
 
 // After git push/pull, materialize merges both writers
 const state = await graphA.materialize();
@@ -120,13 +125,11 @@ await graphA.syncWith(graphB);
 
 ## Querying
 
-Query methods require materialized state. Either call `materialize()` first, or pass `autoMaterialize: true` to `WarpGraph.open()` to handle this automatically.
+Query methods auto-materialize by default. Just open a graph and start querying:
 
 ### Simple Queries
 
 ```javascript
-await graph.materialize();
-
 await graph.getNodes();                              // ['user:alice', 'user:bob']
 await graph.hasNode('user:alice');                   // true
 await graph.getNodeProps('user:alice');              // Map { 'name' => 'Alice', 'role' => 'admin' }
@@ -492,9 +495,27 @@ npm run test:deno       # Deno: API integration tests
 npm run test:matrix     # All runtimes in parallel
 ```
 
+## When git-warp is Most Useful
+
+- **Distributed configuration management.** A fleet of servers each writing their own state into a shared graph without a central database.
+- **Offline-first field applications.** Collecting data in the field with no connectivity; merging cleanly when back online.
+- **Collaborative knowledge bases.** Researchers curating nodes and relationships independently.
+- **Git-native issue/project tracking.** Embedding a full project graph directly in the repo.
+- **Audit-critical systems.** Tamper-evident records with cryptographic proof (via Audit Receipts).
+- **IoT sensor networks.** Sensors logging readings and relationships, syncing when bandwidth allows.
+- **Game world state.** Modders independently adding content that composes without a central manager.
+
+## When NOT to Use It
+
+- **High-throughput transactional workloads.** If you need thousands of writes per second with immediate consistency, use Postgres or Redis.
+- **Large binary or blob storage.** Data lives in Git commit messages (default cap 1 MB). Use object storage for images or videos.
+- **Sub-millisecond read latency.** Materialization has overhead. Use an in-memory database for real-time gaming physics or HFT.
+- **Simple key-value storage.** If you don't have relationships or need traversals, a graph database is overkill.
+- **Non-Git environments.** The value proposition depends on Git infrastructure (push/pull, content-addressing).
+
 ## AIΩN Foundations Series
 
-This package is the reference implementation of WARP (Worldline Algebra for Recursive Provenance) graphs as described in the AIΩN Foundations Series. The papers define WARP graphs as a minimal recursive state object ([Paper I](https://doi.org/10.5281/zenodo.17908005)), equip them with deterministic tick-based operational semantics ([Paper II](https://doi.org/10.5281/zenodo.17934512)), develop computational holography, provenance payloads, and prefix forks ([Paper III](https://doi.org/10.5281/zenodo.17963669)), and introduce observer geometry with rulial distance and temporal logic ([Paper IV](https://doi.org/10.5281/zenodo.18038297)). This codebase implements the core data structures and multi-writer collaboration protocol described in those papers.
+This package is the reference implementation of WARP (Worldline Algebra for Recursive Provenance) graphs as described in the AIΩN Foundations Series. The papers formalize the graph as a minimal recursive state object ([Paper I](https://doi.org/10.5281/zenodo.17908005)), equip it with deterministic tick-based semantics ([Paper II](https://doi.org/10.5281/zenodo.17934512)), develop computational holography and provenance payloads ([Paper III](https://doi.org/10.5281/zenodo.17963669)), and introduce observer geometry with the translation cost metric ([Paper IV](https://doi.org/10.5281/zenodo.18038297)).
 
 ## License
 

package/SECURITY.md

@@ -113,6 +113,70 @@ await graph.syncWith('http://peer:3000', {
 });
 ```
 
+## Dependency Risk Assessment
+
+| Package | Type | Risk | Notes |
+|---|---|---|---|
+| `@git-stunts/plumbing` | Runtime | Low | Internal package, spawns git processes with strict whitelist |
+| `@git-stunts/alfred` | Runtime | Low | Retry/backoff utility, no I/O |
+| `@git-stunts/trailer-codec` | Runtime | Low | Pure string encoding, no I/O |
+| `cbor-x` | Runtime | Medium | Binary parser processing untrusted sync payloads; mitigated by body size limit in HttpSyncServer (4 MB default) |
+| `roaring` | Runtime | Medium | Native C++ bindings (N-API); largest attack surface but sandboxed by N-API boundary, no network-facing input |
+| `zod` | Runtime | Low | Schema validation, pure JS |
+| `elkjs` | Runtime (lazy) | Low | ELK layout engine, pure JS, lazy-loaded only for `--view` |
+| `chalk` | CLI-only | Negligible | Terminal coloring, no security surface |
+| `boxen` | CLI-only | Negligible | Terminal box drawing |
+| `cli-table3` | CLI-only | Negligible | Terminal table rendering |
+| `string-width` | CLI-only | Negligible | String measurement |
+| `strip-ansi` | Inlined | Negligible | ANSI escape removal; inlined into `src/visualization/utils/ansi.js` since v10.1.2, no longer a direct dependency |
+| `open` | Transitive | Low | Opens URLs in browser; transitive dependency only, invoked by `--view` flag |
+
+## Accepted Risks
+
+| Risk | Severity | Owner | Expiry | Mitigation |
+|---|---|---|---|---|
+| `roaring` native bindings could have memory-safety bugs | Medium | @jross | 2026-08-01 | N-API sandbox; no untrusted input reaches bitmap code directly |
+| `cbor-x` parser handles untrusted sync payloads | Medium | @jross | 2026-08-01 | 4 MB body size limit in HttpSyncServer; schema validation post-parse |
+| Nonce cache lost on restart allows replay within clock skew window | Low | @jross | 2026-08-01 | 5-minute TTL; recommend TLS in production |
+| No rate limiting on sync endpoint | Low | @jross | 2026-08-01 | Deploy behind reverse proxy with rate limiting |
+
+## Threat Model Boundaries
+
+| Category | In Scope | Out of Scope |
+|---|---|---|
+| **Data integrity** | CRDT convergence, CAS ref updates, patch ordering | Filesystem corruption, disk failure |
+| **Authentication** | HMAC-SHA256 sync auth, key rotation, replay protection | User identity management, OAuth/OIDC |
+| **Authorization** | Writer whitelist (per-graph allowed writers) | Role-based access control, per-node ACLs |
+| **Transport** | Body signing, tamper detection | TLS termination (delegated to reverse proxy) |
+| **Availability** | Body size limits, streaming for large graphs | DDoS protection, rate limiting (delegated to infrastructure) |
+| **Supply chain** | `npm audit` in CI, pinned dependencies via lockfile | Runtime SCA scanning, SBOM generation |
+
+## Writer Authorization
+
+The sync server supports an optional writer whitelist that restricts which writer IDs can submit patches through the HTTP sync protocol.
+
+### Configuration
+
+```js
+await graph.serve({
+  port: 3000,
+  httpPort: new NodeHttpAdapter(),
+  auth: {
+    keys: { default: 'your-shared-secret' },
+    mode: 'enforce',
+  },
+  allowedWriters: ['alice', 'bob', 'node-1'],
+});
+```
+
+### Rules
+
+- Writer ID matching is **case-sensitive** and follows the pattern `[A-Za-z0-9._-]+` (1–64 characters)
+- Writer IDs are validated at server startup; invalid IDs throw immediately
+- When `allowedWriters` is not set (default), all authenticated writers are permitted
+- When set, sync requests containing patches from unlisted writers receive HTTP 403 (`FORBIDDEN_WRITER`)
+- The `forbiddenWriterRejects` metric tracks rejected requests
+
 ## Reporting a Vulnerability
 
 If you discover a security vulnerability, please send an e-mail to [james@flyingrobots.dev](mailto:james@flyingrobots.dev).

package/bin/cli/commands/check.js

@@ -0,0 +1,168 @@
+import HealthCheckService from '../../../src/domain/services/HealthCheckService.js';
+import ClockAdapter from '../../../src/infrastructure/adapters/ClockAdapter.js';
+import { buildCheckpointRef, buildCoverageRef } from '../../../src/domain/utils/RefLayout.js';
+import { EXIT_CODES } from '../infrastructure.js';
+import { openGraph, applyCursorCeiling, emitCursorWarning, readCheckpointDate, createHookInstaller } from '../shared.js';
+
+/** @typedef {import('../types.js').CliOptions} CliOptions */
+/** @typedef {import('../types.js').Persistence} Persistence */
+/** @typedef {import('../types.js').WarpGraphInstance} WarpGraphInstance */
+
+/** @param {Persistence} persistence */
+async function getHealth(persistence) {
+  const clock = ClockAdapter.global();
+  const healthService = new HealthCheckService({ persistence: /** @type {import('../../../src/domain/types/WarpPersistence.js').CorePersistence} */ (/** @type {unknown} */ (persistence)), clock });
+  return await healthService.getHealth();
+}
+
+/** @param {WarpGraphInstance} graph */
+async function getGcMetrics(graph) {
+  await graph.materialize();
+  return graph.getGCMetrics();
+}
+
+/** @param {WarpGraphInstance} graph */
+async function collectWriterHeads(graph) {
+  const frontier = await graph.getFrontier();
+  return [...frontier.entries()]
+    .sort(([a], [b]) => a.localeCompare(b))
+    .map(([writerId, sha]) => ({ writerId, sha }));
+}
+
+/**
+ * @param {Persistence} persistence
+ * @param {string} graphName
+ */
+async function loadCheckpointInfo(persistence, graphName) {
+  const checkpointRef = buildCheckpointRef(graphName);
+  const checkpointSha = await persistence.readRef(checkpointRef);
+  const checkpointDate = await readCheckpointDate(persistence, checkpointSha);
+  const checkpointAgeSeconds = computeAgeSeconds(checkpointDate);
+
+  return {
+    ref: checkpointRef,
+    sha: checkpointSha || null,
+    date: checkpointDate,
+    ageSeconds: checkpointAgeSeconds,
+  };
+}
+
+/** @param {string|null} checkpointDate */
+function computeAgeSeconds(checkpointDate) {
+  if (!checkpointDate) {
+    return null;
+  }
+  const parsed = Date.parse(checkpointDate);
+  if (Number.isNaN(parsed)) {
+    return null;
+  }
+  return Math.max(0, Math.floor((Date.now() - parsed) / 1000));
+}
+
+/**
+ * @param {Persistence} persistence
+ * @param {string} graphName
+ * @param {Array<{writerId: string, sha: string}>} writerHeads
+ */
+async function loadCoverageInfo(persistence, graphName, writerHeads) {
+  const coverageRef = buildCoverageRef(graphName);
+  const coverageSha = await persistence.readRef(coverageRef);
+  const missingWriters = coverageSha
+    ? await findMissingWriters(persistence, writerHeads, coverageSha)
+    : [];
+
+  return {
+    ref: coverageRef,
+    sha: coverageSha || null,
+    missingWriters: missingWriters.sort(),
+  };
+}
+
+/**
+ * @param {Persistence} persistence
+ * @param {Array<{writerId: string, sha: string}>} writerHeads
+ * @param {string} coverageSha
+ */
+async function findMissingWriters(persistence, writerHeads, coverageSha) {
+  const missing = [];
+  for (const head of writerHeads) {
+    const reachable = await persistence.isAncestor(head.sha, coverageSha);
+    if (!reachable) {
+      missing.push(head.writerId);
+    }
+  }
+  return missing;
+}
+
+/**
+ * @param {{repo: string, graphName: string, health: *, checkpoint: *, writerHeads: Array<{writerId: string, sha: string}>, coverage: *, gcMetrics: *, hook: *|null, status: *|null}} params
+ */
+function buildCheckPayload({
+  repo,
+  graphName,
+  health,
+  checkpoint,
+  writerHeads,
+  coverage,
+  gcMetrics,
+  hook,
+  status,
+}) {
+  return {
+    repo,
+    graph: graphName,
+    health,
+    checkpoint,
+    writers: {
+      count: writerHeads.length,
+      heads: writerHeads,
+    },
+    coverage,
+    gc: gcMetrics,
+    hook: hook || null,
+    status: status || null,
+  };
+}
+
+/** @param {string} repoPath */
+function getHookStatusForCheck(repoPath) {
+  try {
+    const installer = createHookInstaller();
+    return installer.getHookStatus(repoPath);
+  } catch {
+    return null;
+  }
+}
+
+/**
+ * Handles the `check` command: reports graph health, GC, and hook status.
+ * @param {{options: CliOptions}} params
+ * @returns {Promise<{payload: unknown, exitCode: number}>}
+ */
+export default async function handleCheck({ options }) {
+  const { graph, graphName, persistence } = await openGraph(options);
+  const cursorInfo = await applyCursorCeiling(graph, persistence, graphName);
+  emitCursorWarning(cursorInfo, null);
+  const health = await getHealth(persistence);
+  const gcMetrics = await getGcMetrics(graph);
+  const status = await graph.status();
+  const writerHeads = await collectWriterHeads(graph);
+  const checkpoint = await loadCheckpointInfo(persistence, graphName);
+  const coverage = await loadCoverageInfo(persistence, graphName, writerHeads);
+  const hook = getHookStatusForCheck(options.repo);
+
+  return {
+    payload: buildCheckPayload({
+      repo: options.repo,
+      graphName,
+      health,
+      checkpoint,
+      writerHeads,
+      coverage,
+      gcMetrics,
+      hook,
+      status,
+    }),
+    exitCode: EXIT_CODES.OK,
+  };
+}