@shapeshift-labs/frontier-swarm 0.1.0 → 0.3.0

package/README.md

@@ -36,7 +36,7 @@ The published Frontier package family is generated from one shared package catal
 - [`@shapeshift-labs/frontier-workflow`](https://www.npmjs.com/package/@shapeshift-labs/frontier-workflow): Serializable durable workflow/process manifests for Frontier apps, including steps, waits, approvals, timers, retries, expected patches, compensation, records, timelines, and registry graph output.
 - [`@shapeshift-labs/frontier-worker`](https://www.npmjs.com/package/@shapeshift-labs/frontier-worker): Serializable worker and edge task descriptors for Frontier apps, including queues, idempotency keys, retry and timeout policy, declared reads/writes/effects, snapshots, patch outputs, produced assets, execution records, logs, trace links, proof hashes, dedupe indexes, and registry graph output.
 - [`@shapeshift-labs/frontier-queue`](https://www.npmjs.com/package/@shapeshift-labs/frontier-queue): Serializable durable queue state, leases, retries, dedupe keys, patch-carrying jobs, dead-letter records, replay evidence, and queue inspection for Frontier apps.
-- [`@shapeshift-labs/frontier-swarm-codex`](https://www.npmjs.com/package/@shapeshift-labs/frontier-swarm-codex): Node Codex CLI adapter for Frontier swarm plans, including prompt rendering, worktree and snapshot workspaces, Codex argument construction, JSONL capture, verification commands, and result artifacts.
+- [`@shapeshift-labs/frontier-swarm-codex`](https://www.npmjs.com/package/@shapeshift-labs/frontier-swarm-codex): Node Codex CLI adapter for Frontier swarm plans, including prompt rendering, worktree and snapshot workspaces, Codex argument compatibility, JSONL capture, verification commands, pid-backed stop, collect buckets, merge bundles, and result artifacts.
 - [`@shapeshift-labs/frontier-kv`](https://www.npmjs.com/package/@shapeshift-labs/frontier-kv): Serializable in-memory key/value state for Frontier apps, including TTL, versioned compare-and-set, batched patch mutations, scans, watchers, snapshots, JSONL event evidence, and replay verification.
 - [`@shapeshift-labs/frontier-kv-locks`](https://www.npmjs.com/package/@shapeshift-labs/frontier-kv-locks): Lease-style lock records on top of Frontier KV, including acquire, renew, release, fencing tokens, expiration, owner evidence, and replayable lock events.
 - [`@shapeshift-labs/frontier-kv-rate-limit`](https://www.npmjs.com/package/@shapeshift-labs/frontier-kv-rate-limit): Patch-native rate limit buckets for Frontier KV, including fixed windows, sliding windows, token buckets, deterministic refill, consume evidence, and reset records.
@@ -203,6 +203,55 @@ const tasks = defineSwarmTasks([{
 const plan = createSwarmPlan(manifest, tasks, { limit: 4 });
 ```
 
+## 1000-Agent Control Plane
+
+Large swarms need a control plane, not just a flat worker loop. `frontier-swarm` now exports deterministic data helpers for that layer:
+
+```ts
+import {
+  checkSwarmBudget,
+  createSwarmArtifactIndex,
+  createSwarmEventStream,
+  createSwarmLeases,
+  createSwarmMergePlan,
+  createSwarmQueueSnapshot,
+  createSwarmReviewPlan,
+  createSwarmSchedule,
+  routeSwarmEventToMailboxes
+} from '@shapeshift-labs/frontier-swarm';
+
+const schedule = createSwarmSchedule({
+  plan,
+  maxReadyJobs: 100,
+  maxConcurrencyKeyConcurrency: { 'runtime-state': 1 },
+  maxComputeConcurrency: { deep: 40 }
+});
+
+const leases = createSwarmLeases({
+  schedule,
+  workerId: 'worker-a',
+  leaseMs: 15 * 60 * 1000,
+  count: 10
+});
+```
+
+The scale APIs are runtime-neutral and serializable:
+
+- dependency DAGs are compiled into `plan.graph`,
+- `createSwarmSchedule` returns ready, blocked, running, completed, and failed jobs under lane/compute/contention limits,
+- `createSwarmLeases` gives workers expiring leases with fencing tokens,
+- lane/task `capabilities` and `resourceRequirements` can reserve browser work with lower concurrency, port pools, profile directories, and explicit capability checks,
+- `createSwarmQueueSnapshot` and run checkpoints give durable queue/run-store adapters a stable serialization shape,
+- `createSwarmEventStream` and `routeSwarmEventToMailboxes` route global, lane, task, and worker events into coordinator-facing JSONL streams,
+- `checkSwarmBudget` records token/cost/time/retry budget decisions,
+- `createSwarmArtifactIndex` groups evidence, timelines, logs, and produced files,
+- `createSwarmReviewPlan` samples or requires reviewer assignments,
+- `createSwarmMergePlan` blocks jobs with failed checks, required reviews, ownership violations, or conflicting changed paths,
+- job results include merge-readiness classification: `discovery-only`, `patch-candidate`, `verified-patch`, `rejected`, or `blocked`,
+- `ownershipRegions` allow hot files to be split into semantic regions such as `content.docs.*` or `adminSettings.quota.*`; merge conflict detection prefers changed regions when workers report them,
+- `createSwarmMergeBundle` builds a compact worker `merge.json` shape with touched owned files, patch path, evidence, verification, queue items satisfied, risk, and disposition,
+- `decomposeSwarmFeature` creates an initial task queue for feature work across lanes.
+
 ## Hierarchical Compute
 
 Higher swarm layers can choose compute for lower layers without binding the core package to Codex or any other runtime. Compute resolution is deterministic:
@@ -220,7 +269,18 @@ That lets a parent swarm route implementation jobs to a deep model while evidenc
 - `defineSwarmManifest`, `createSwarmManifest`
 - `defineSwarmTasks`
 - `compileSwarm`, `validateSwarmManifest`
+- `createSwarmTaskSelection`
 - `createSwarmPlan`, `createSwarmRun`
+- `createSwarmSchedule`, `createSwarmLeases`
+- `createSwarmQueueSnapshot`, `createSwarmRunCheckpoint`
+- `createSwarmEventStream`, `createSwarmMailbox`, `routeSwarmEventToMailboxes`
+- `checkSwarmBudget`
+- `createSwarmArtifactIndex`
+- `createSwarmReviewPlan`, `createSwarmMergePlan`
+- `createSwarmMergeBundle`
+- `classifySwarmMergeReadiness`, `classifySwarmMergeDisposition`
+- `resolveSwarmChangedRegions`
+- `decomposeSwarmFeature`
 - `recordSwarmEvent`, `completeSwarmJob`
 - `resolveSwarmCompute`
 - `checkSwarmOwnership`, `matchesGlob`
@@ -235,7 +295,7 @@ Run the package-local benchmark:
 npm run bench
 ```
 
-The benchmark writes `benchmarks/results/frontier-swarm-package-bench-latest.json` when run from the monorepo. These are Frontier-only package measurements for plan creation, manifest validation, hierarchical compute resolution, ownership checks, run creation, JSONL, and proof hashing.
+The benchmark writes `benchmarks/results/frontier-swarm-package-bench-latest.json` when run from the monorepo. These are Frontier-only package measurements for plan creation, manifest validation, hierarchical compute resolution, ownership checks, scheduling/leases, queue snapshots, merge bundle creation, event routing, run checkpoints, JSONL, and proof hashing.
 
 ## Source Repository
 

package/benchmarks/package-bench.mjs

@@ -5,12 +5,19 @@ import { fileURLToPath } from 'node:url';
 import {
   checkSwarmOwnership,
   createSwarmManifest,
+  createSwarmEventStream,
+  createSwarmLeases,
+  createSwarmMergeBundle,
   createSwarmPlan,
   createSwarmProof,
+  createSwarmQueueSnapshot,
   createSwarmRun,
+  createSwarmRunCheckpoint,
+  createSwarmSchedule,
   decodeSwarmJsonl,
   defineSwarmTasks,
   encodeSwarmJsonl,
+  routeSwarmEventToMailboxes,
   resolveSwarmCompute,
   validateSwarmManifest
 } from '../dist/index.js';
@@ -31,6 +38,9 @@ let plan = createSwarmPlan(manifest, tasks, { limit: 64 });
 let run = createSwarmRun({ plan });
 let jsonl = encodeSwarmJsonl([plan, run]);
 let cursor = 0;
+let schedule = createSwarmSchedule({ plan, maxReadyJobs: 128 });
+let leases = createSwarmLeases({ schedule, workerId: 'bench-worker', now: 1000, leaseMs: 60000, count: 16 });
+let eventStream = createSwarmEventStream({ runId: 'bench', root: 'agent-runs/bench/streams', lanes: manifest.lanes });
 
 const rows = [
   measure('create-plan-' + taskCount, 8, () => {
@@ -49,7 +59,31 @@ const rows = [
     return jsonl.length;
   }),
   measure('jsonl-decode', 16, () => decodeSwarmJsonl(jsonl).length),
-  measure('proof', 16, () => createSwarmProof(plan).hash.length)
+  measure('proof', 16, () => createSwarmProof(plan).hash.length),
+  measure('schedule-lease-' + taskCount, 8, () => {
+    schedule = createSwarmSchedule({ plan, maxReadyJobs: 128, maxComputeConcurrency: { fast: 64, deep: 32 } });
+    leases = createSwarmLeases({ schedule, workerId: 'bench-worker', now: 1000 + cursor++, leaseMs: 60000, count: 16 });
+    return schedule.ready.length + leases.length;
+  }),
+  measure('queue-snapshot-' + taskCount, 8, () => {
+    const snapshot = createSwarmQueueSnapshot({ plan, run, leases, generatedAt: 2000 + cursor++ });
+    return snapshot.summary.jobCount + snapshot.summary.leaseCount;
+  }),
+  measure('run-checkpoint-' + taskCount, 16, () => createSwarmRunCheckpoint({ run, sequence: cursor++ }).hash.length),
+  measure('merge-bundle-' + taskCount, 32, () => createSwarmMergeBundle({
+    job: plan.jobs[cursor % plan.jobs.length],
+    result: {
+      jobId: plan.jobs[cursor++ % plan.jobs.length].id,
+      status: 'completed',
+      changedPaths: ['src/runtime/file.ts'],
+      verification: [{ status: 0 }]
+    },
+    patchPath: 'agent-runs/bench/changes.patch'
+  }).id.length),
+  measure('event-route-' + taskCount, 64, () => {
+    eventStream = createSwarmEventStream({ runId: 'bench', root: 'agent-runs/bench/streams', lanes: manifest.lanes });
+    return routeSwarmEventToMailboxes(eventStream, { type: 'agent.evidence', jobId: plan.jobs[cursor++ % plan.jobs.length].id, lane: 'runtime' }).length;
+  })
 ];
 
 const report = {