owenloop 0.2.1 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/docs/design.md CHANGED
@@ -24,9 +24,53 @@ function rather than a feature bolted beside it.
24
24
  - **§2.2 Task / lease** — the claimable unit of work-in-flight. One per
25
25
  `(step, key)`; `key` is `""` for plain/reduce/collection firings and the
26
26
  element path for a map firing.
27
+
28
+ **Lease lifecycle.** Claiming a firing writes `claimedAt` on the task and opens
29
+ a run. A lease's liveness is judged on read (at `reap`/`tick`/claim time), never
30
+ on a timer:
31
+ - **Reap TTL (the anchor rule).** A lease is fresh while
32
+ `now - max(claimedAt, heartbeatAt) <= ttl`, where `ttl` is the effective reap
33
+ TTL (engine `reapTtlMs`, default 2h; per-step `reapTtl:` overrides). A run that
34
+ heartbeats within the TTL is **never** reaped by this rule, however long it
35
+ runs — heartbeats extend the lease indefinitely. A run that goes silent past
36
+ the TTL is stale, reaped back to `idle` (its `attempts` bumps), and re-claimable
37
+ by another worker; the old run's next `green`/`close` then fails the commit CAS
38
+ (§12.2) with a stale-lease error.
39
+ - **Opt-in max-lease cap.** There is **no default cap on total lease lifetime.**
40
+ An operator may set one — engine `maxLeaseMs` or per-step `maxLease:` (per-step
41
+ overrides the engine option) — as a runaway backstop: it bounds total lifetime
42
+ from the original `claimedAt` regardless of heartbeats, so past
43
+ `claimedAt + maxLease` even a still-beating lease is reaped. The cap re-anchors
44
+ to the new `claimedAt` on re-claim. **Tradeoff, stated plainly:** a cap can reap
45
+ a healthy, still-beating job, after which another worker claims the same order —
46
+ duplicate execution is the price of the backstop. Leave it unset unless you
47
+ specifically want that bound; heartbeat liveness is already the reap signal.
48
+ - **Reap reasons.** When a reap clears a lease it records why, so the two failure
49
+ modes are distinguishable: `heartbeat-lost` (the anchor rule lapsed — the job
50
+ went silent) vs. `max-lease-exceeded` (a configured cap expired a still-beating
51
+ lease). When both bounds lapse, `heartbeat-lost` wins (a dead job is not reported
52
+ as cap-killed). See the CLI `reap` command (docs/cli.md) — or the `ReapReason`
53
+ type in `engine.ts` — for the full reason set.
27
54
  - **§2.3 Run** — the audit/budget record created when a task is claimed; holds the
28
55
  claim's input **fingerprint** for the commit CAS.
29
56
 
57
+ ### §2.4 Persistence history
58
+
59
+ SQLite keeps `artifact` as the compact current-state projection used by the
60
+ scheduler. In addition, every newly produced positive version is captured once
61
+ in immutable `artifact_version` (payload, fingerprint, producer, initial
62
+ acceptance, and time), while `artifact_event` is an append-only, version-addressed
63
+ lifecycle stream. Re-persisting a semantically identical artifact state appends no
64
+ event: change-detection is structural (a canonical, key-order-independent
65
+ comparison of the artifact's semantic fields), so `artifact_event` stays a true
66
+ audit stream rather than growing on order-only or no-op rewrites.
67
+ `Store.getArtifactHistory(workflow, path)` is deliberately a
68
+ narrow, lazy read; normal artifact/status listing does not load it. Intentional
69
+ artifact or workflow deletion also removes its history, so active instances retain
70
+ history but cleanup leaves no orphan audit rows. Upgrading legacy databases can
71
+ backfill the existing reason entries, but cannot recover payloads overwritten
72
+ before this schema existed; durable payload history begins after migration.
73
+
30
74
  ## §3 The firing rule
31
75
 
32
76
  A step's eligibility depends on its consume mode:
@@ -517,10 +561,10 @@ Shape rules:
517
561
  When the engine spawns a child instance, it passes `producedBy: { parentWf, parentPath }` to `createInstance`, which persists it via the store. The coordinate serves three duties:
518
562
 
519
563
  1. **Re-attach on reap**: when a child run is reaped, the engine re-attaches via the stored link.
520
- 2. **Reverse lookup**: `store.findChildByParent(parentWf, parentPath)` — the never-duplicate guard.
564
+ 2. **Reverse lookup**: `store.findChildByParent(parentWf, parentPath)` — the re-attach lookup (returns the *oldest* matching child deterministically, `ORDER BY created_at, id`; the never-duplicate *guarantee* lives in the atomic spawn plus the unique index, §23.6.9).
521
565
  3. **Cascade-up anchor**: the engine reads `producedBy` to propagate the child's outcome to the parent.
522
566
 
523
- **Storage**: two nullable columns on the `workflow` table — `produced_by_wf TEXT` and `produced_by_path TEXT` (both null for a top-level instance). Two columns (not a JSON blob) because the reverse lookup `(parentWf, parentPath) → child` must be SQL-indexable. The index `workflow_produced_by ON workflow(produced_by_wf, produced_by_path)` makes the lookup O(1). Added by the additive migration in `store.migrate()` (schema version 3).
567
+ **Storage**: two nullable columns on the `workflow` table — `produced_by_wf TEXT` and `produced_by_path TEXT` (both null for a top-level instance). Two columns (not a JSON blob) because the reverse lookup `(parentWf, parentPath) → child` must be SQL-indexable. The index `workflow_produced_by ON workflow(produced_by_wf, produced_by_path)` makes the lookup O(1). Added by the additive migration in `store.migrate()` (schema version 3). A second, **partial UNIQUE** index `workflow_produced_by_unique` on the same two columns (schema version 8, `WHERE` both coords are `NOT NULL` so top-level instances with NULL coords never conflict) makes a duplicate child physically impossible; it is created only when no legacy duplicates already exist (see §23.6.9).
524
568
 
525
569
  ### §23.3 calls: steps are machine-handled
526
570
 
@@ -547,10 +591,10 @@ The engine ships `maintainCalls` in `engine.ts` — the engine-internal method t
547
591
  Called at the top of every parent `tick` (outside any transaction), after `provideInput` on the parent (so a newly-supplied human input is immediately re-provided to any mapped child), and as a cascade-up prompt after child progress. For each `calls:` step in the parent def:
548
592
 
549
593
  1. **Gate check**: `gateStems = Object.values(callsInputs)` (parent artifact names wired to child inputs). Gate is ready when every gate stem is green.
550
- 2. **Re-attach guard**: `findChildByParent(parentWf, callsPath)` — spawn only when no child exists (`undefined`). This prevents duplicate children across crashes and re-ticks. A second guard sits alongside it (§23.6.7 F2): if no child exists AND the parent calls artifact is already `rejected` on a schema refusal, skip the spawn attempt entirely while the gate stems' fingerprint is unchanged from the one stamped at refusal time — a moved gate is the only thing worth retrying against.
551
- 3. **Spawn**: if gate is ready and no child, `createInstance(step.calls, { producedBy, provide: gateValues })`. The parent calls: artifact stays `owed`. §23.6.7 (F2): a child input-schema refusal here is caught narrowly (`SchemaRefusalError` only — a genuine bug still throws) and recorded as a debt on the parent calls artifact instead of crashing the tick.
594
+ 2. **Re-attach guard**: `findChildByParent(parentWf, callsPath)` — spawn only when no child exists (`undefined`). This re-attaches (rather than re-spawns) across crashes and sequential re-ticks; it does NOT by itself stop *concurrent* ticks in separate processes, which can each read `undefined` here before either inserts — that race is closed by the atomic spawn (§23.6.9). A second guard sits alongside it (§23.6.7 F2): if no child exists AND the parent calls artifact is already `rejected` on a schema refusal, skip the spawn attempt entirely while the gate stems' fingerprint is unchanged from the one stamped at refusal time — a moved gate is the only thing worth retrying against.
595
+ 3. **Provision**: if gate is ready, `provisionCallsChild(parentWf, step, callsStem, gateStems)` — a single `BEGIN IMMEDIATE` (§23.6.9) that re-reads the parent snapshot inside the write lock, then finds-or-creates the child AND syncs every mapped `callsInputs` value from that same fresh snapshot. It absorbs both the old spawn and the old re-provide (step 5) into one atomic read-verify-write, so a child is never seeded or re-provided from a snapshot older than its committing tx. The pre-reads above (gate, `findChildByParent`, F2 fingerprint, depth) are OPTIMISTIC — they decide only *whether* to attempt; the helper re-verifies in-tx and aborts silently (returns `null`, nothing written) if the gate re-armed or an F2 debt landed since. Instance/`settled` events fire only when this call created the child; a `commit`/`provide` fires for each input it re-wrote. The parent calls: artifact stays `owed`. §23.6.7 (F2): a child input-schema refusal here (seed or re-provide) is caught narrowly at the call site (`SchemaRefusalError` only — a genuine bug still throws) and recorded as a debt on the parent calls artifact instead of crashing the tick.
552
596
  4. **Outcome read**: read the child's declared `outputs:` artifact (exactly one, validated at load time). If it is green, machine-green the parent's calls: artifact.
553
- 5. **Re-provide**: for each `callsInputs` mapping, if the parent's value differs (deep-equal) from what the child holds, `provideInput(child, inputName, newValue)`. The child re-runs internally. §23.6.7 (F2): same narrow schema-refusal handling as spawn — a child-illegal re-provided value becomes a debt on the parent calls artifact, not a thrown cascade.
597
+ 5. **Re-provide**: folded into step 3's atomic provision — `provisionCallsChild` syncs every changed `callsInputs` mapping (deep-equal check) inside the same tx that finds-or-creates the child, so the re-provide can no longer race a concurrent parent advance across a separate `provideInput` cascade. The child re-runs internally on each changed input. §23.6.7 (F2): same narrow schema-refusal handling as spawn — a child-illegal re-provided value becomes a debt on the parent calls artifact, not a thrown cascade.
554
598
  6. **Machine-green**: set parent calls: artifact to `acceptance: 'green'`, `version + 1`, `value = child outcome value`, `fingerprint = computeFingerprint(parentArts, gateStems) plus the child-outcome version pin` (§23.6.8, F4) — but only when the child outcome's version has moved past whatever version is currently pinned on the parent artifact. Then `settle(parentWf)` so downstream (teardown) fires. Do NOT set `terminal` — the calls: artifact must be re-armable if gate inputs move.
555
599
  7. **Re-arm on child working**: if the child's outcome is no longer green (e.g. re-provide re-armed it) but the parent calls: artifact is green, re-arm the parent calls: artifact to `owed`. This handles gate re-arm correctly even though `deliver` step has `consumes: []` (the pure cascade cannot detect fingerprint mismatch for calls: steps). This re-arm is unconditional on the pin — the pin only gates re-greening, never gates reopening to `owed`.
556
600
 
@@ -572,13 +616,15 @@ The machine-green fingerprint covers `gateStems` (the parent artifacts wired int
572
616
 
573
617
  #### §23.6.6 Transaction composition
574
618
 
575
- `maintainCalls` runs OUTSIDE any open `store.tx()`. Each mutating action (spawn via `createInstance`, re-provide via `provideInput`, machine-green) opens its own `store.tx()`. No nested transactions — node:sqlite does not support nested `BEGIN IMMEDIATE`.
619
+ `maintainCalls` runs OUTSIDE any open `store.tx()`. Each mutating action (provision — spawn + re-provide — via `provisionCallsChild`, machine-green via `publishCallsGreen`, re-arm via `rearmCallsGreen`) opens its own `store.tx()`. No nested transactions — node:sqlite does not support nested `BEGIN IMMEDIATE`.
620
+
621
+ Each mutating action does its **read-verify-write in ONE `BEGIN IMMEDIATE`**, not just its write. `maintainCalls`'s pre-reads (the outcome/gate/pin reads before STEP 6, the reverse-lookup before spawn) are an OPTIMISTIC gate that decides only *whether* to attempt the action — never what to commit; every value the write depends on is re-read and re-verified inside the same transaction. Under WAL this closes the cross-connection interleave where a second connection advances the parent gate (to a newer version) or re-arms the child between the stale pre-read and the write: without it, a stale child value could be published under a fresh gate fingerprint, and the same tick would issue downstream work off that stale value. `publishCallsGreen` re-reads the parent artifact, child, child outcome and gate in-tx, verifies the child is still green, the gate still ready, the F4 pin not yet passed, and the child inputs still consistent with the gate the fingerprint will claim, then computes the fingerprint from that one snapshot and writes; any mismatch aborts (returns `false`, nothing written — no debt, no reasons, no events) and defers to the next tick. `rearmCallsGreen` is the mirror guard (re-arm only if the parent is still green AND the child outcome still not green in-tx). The `commit`/`settled` events fire only when the helper actually wrote, so an aborted pass is silent. Child provisioning follows the same single-`BEGIN IMMEDIATE` shape (§23.6.9): `provisionCallsChild` re-reads the parent snapshot in-tx, then finds-or-creates the child and syncs its inputs from that snapshot — the reverse-lookup read, the insert, and every input write are one atomic step, so a child is never created (spawn) or re-written (re-provide) from a snapshot older than its committing tx. This is the creation-side twin of the machine-green fix above.
576
622
 
577
623
  #### §23.6.7 F2 — child input-schema refusal is a debt, not a thrown tick
578
624
 
579
625
  A `calls:` step wires parent artifact values into the child's inputs (`callsInputs`), but the parent's own schema for that artifact (if any) can be looser than — or entirely absent from — the child's declared input schema. Before this fix, `createInstance`'s seed-provide validation and `provideInput`'s validation threw a bare `Error` on a schema mismatch; because `maintainCalls` runs at the top of every `tick()` with no catch, one such value made every subsequent `tick(parent)` throw — a permanent crash loop with no debt or stall visible in `status()`.
580
626
 
581
- The fix: `createInstance`/`provideInput` now throw a dedicated `SchemaRefusalError` (carrying the offending input name and schema issues) instead of a bare `Error` on a schema mismatch. `maintainCalls`'s STEP 3 (spawn) and STEP 5 (re-provide) each catch `SchemaRefusalError` narrowly — any other error still propagates, since that's a genuine bug, not an expected refusal. On catch, the parent calls artifact is recorded as a debt with the same shape `green()` uses for its own schema-reject branch: `acceptance: 'rejected'`, `schemaRejects + 1`, a `validation`-kind reasons entry naming the child input and the schema issues. The tick then proceeds instead of throwing.
627
+ The fix: `createInstance`/`provideInput` now throw a dedicated `SchemaRefusalError` (carrying the offending input name and schema issues) instead of a bare `Error` on a schema mismatch. `maintainCalls`'s STEP 3 provision — `provisionCallsChild`, which now covers both spawn and re-provide — surfaces the refusal (from the child seed or an input sync, rolling the whole provision tx back) and the call site catches `SchemaRefusalError` narrowly — any other error still propagates, since that's a genuine bug, not an expected refusal. On catch, the parent calls artifact is recorded as a debt with the same shape `green()` uses for its own schema-reject branch: `acceptance: 'rejected'`, `schemaRejects + 1`, a `validation`-kind reasons entry naming the child input and the schema issues. The tick then proceeds instead of throwing.
582
628
 
583
629
  While the parent calls artifact sits `rejected` on a schema refusal and the gate stems' fingerprint (stamped at refusal time) is unchanged, `maintainCalls` does not re-attempt the spawn (STEP 2's guard) nor re-green from a rejected state (STEP 6 short-circuits on `acceptance === 'rejected'`) — re-trying the same illegal value every tick would just refuse again. A human `retry` clears the counter and re-arms to `owed`; once the parent value is fixed (the gate's fingerprint moves), the next tick spawns/re-provides normally.
584
630
 
@@ -593,7 +639,7 @@ Before this fix, a consumer's `reject()` (or a human `skip()`) on a `calls:`-pro
593
639
  - Resolve the child via `store.findChildByParent(parentWf, callsPath)`. If no child was ever spawned, refuse the reject (`cannot reject '<path>': no child instance has been spawned yet`) — consistent with the existing verb-guard rule that a verdict needs a built version.
594
640
  - The CHILD's declared outcome artifact goes `acceptance: 'rejected'`, `judgmentRejects + 1`, with a reasons entry carrying the verdict text and `by: 'parent:<by>'` — the `parent:` prefix marks the entry as an engine-forwarded verdict, not a direct one, in the child's own audit thread. The forward is engine-internal: authority was already checked against the PARENT def (`assertAuthority` runs before the calls-detection branch); the child's own `assertAuthority` is never consulted for this call.
595
641
  - The PARENT calls artifact is reopened to `owed` (never left `rejected`) with a `reopen`/`structural` reasons entry, and its fingerprint is stamped with the rejected child outcome's version (the pin — §below). "Owed, waiting on the child to rebuild" is the honest status, and it lets STEP 6's existing mirror logic pick the new value up unmodified once the child recovers.
596
- - The child's own producer re-arms on the normal knock-back loop — the verdict text rides the child producer's `owes` thread on its next firing, exactly like any other rejected artifact. Driving that re-arm requires ticking the CHILD instance (see the sweeping note below); `rejectCallsArtifact` also prompts `maintainCalls(parentWf)` so the parent's own `owed`/pin state is visible immediately, but it does not — and cannot, from the parent side alone — cause the child's producer step to fire.
642
+ - The child's own producer re-arms on the normal knock-back loop — the verdict text rides the child producer's `owes` thread on its next firing, exactly like any other rejected artifact. Driving that re-arm means firing the CHILD instance's own tick; a deep `tick(parentWf)` does this for you by descending into the child (see the sweeping note below), and `rejectCallsArtifact` also prompts `maintainCalls(parentWf)` so the parent's own `owed`/pin state is visible immediately. What re-arms the child's producer step is the child's own tick, not the parent mirror — the deep descent is simply how a single parent tick reaches it.
597
643
  - If the child keeps failing, `judgmentRejects` accrues on the CHILD artifact and the child stalls at its own `maxAttempts` — the liveness bound lands on the instance actually doing the work, not on the parent's pass-through artifact.
598
644
 
599
645
  **Version-pinned mirror.** A `calls:` step declares `consumes: []`, so the parent artifact's own fingerprint machinery never sees its real input — the child outcome, which lives in another instance entirely. The fix threads a synthetic key, `Engine.CHILD_OUTCOME_PIN_KEY` (`'__child_outcome_version__'`), through the parent calls artifact's `fingerprint` map alongside the gate stems, holding the child outcome version the parent artifact currently rests on:
@@ -610,7 +656,24 @@ This gives the two orderings the settled design calls for:
610
656
 
611
657
  The `M2B-REARM` branch (§23.6.1 step 7 — child un-greened while parent stays green) is unaffected by the pin: it only checks `isGreen(childOutcomeArt)`, so it still reopens the parent to `owed` regardless of what's pinned. The pin governs only the re-green transition (`owed`/`rejected` → `green`), never the green → `owed` reopen.
612
658
 
613
- **Sweeping note.** A propagated reject re-arms work in the CHILD instance, not the parent — `tick(parentWf)` re-runs `maintainCalls` (the mirror), but the child's own re-firing needs `tick(childWf)`. A driver/conductor that only ticks parents (never sweeping children) will see the parent sit `owed` indefinitely with no order ever appearing, because the actual work order is a child-instance concern. Drivers are expected to sweep every live instance (parent and child alike) — `store.listChildrenByParent`/`findChildByParent` give the reverse index needed to discover children from a parent id. The propagated reject is visible for such a sweep via the CHILD's own `status(childWf).debts` (the rejected outcome artifact appears there with `acceptance: 'rejected'`), independent of whether the sweep follows `producedBy` links or just iterates every known workflow id.
659
+ **Deep tick — descend into `calls:` children from the parent tick.** `tick(parentWf)` is **deep by default**: after the parent's own tx settles, it walks every live `calls:` child and ticks each one in its own tx, folding the children's orders, reaps, deferrals, and `dueAt` up into the parent's `TickResult`. One `tick(rootWf)` therefore drives the whole tree — the caller no longer has to discover children and tick them itself. This is what makes a propagated reject (§23.6.8 above) resolve on a plain parent sweep: the reject re-arms work in the CHILD instance, and the deep descent is what fires the child's own tick so its producer step actually gets an order.
660
+
661
+ - **Descent condition.** The parent descends into a child step only when all three hold: (1) the step's **gate is green** (same readiness `maintainCalls` uses to spawn/mirror — see `callsGateReady`), (2) a **child instance exists** (`findChildByParent`), and (3) the parent's `calls:` artifact for that step is specifically **`owed`** — an outstanding producer debt, not merely non-green. `owed` is the sole live debt: it covers the fresh spawn, the F4 propagated-reject reopen (§23.6.8), and a gate-move re-arm. A `skipped`/`rejected`/`retracted` mirror is a dead or refused branch and is **not** descended (issuing child work for it would resurrect a settled decision); a re-armed gate with a still-green mirror, or a step whose child was never spawned, is likewise skipped — descent chases only real, outstanding child work. `callsDescendTargets(parentWf, def)` computes this set.
662
+ - **Own-tx-per-frame.** Descent runs *outside* the parent's transaction (`store.tx` is non-re-entrant — `BEGIN IMMEDIATE` would deadlock on itself). Each child frame opens its own tx via its own `tickInternal` call. The descent condition above is an OPTIMISTIC pre-read (computed outside any tx); the child frame carries the `calls:` edge (`parentEdge`) and, inside its own claim tx, re-verifies the parent gate + child-input consistency — the same predicate `publishCallsGreen` uses — before issuing any order, so a concurrent parent advance that superseded the parent value between the pre-read and the claim issues nothing and skips the subtree. This is the descent-path twin of the in-tx read-verify-write invariant §23.6.6 establishes for the publish/provision paths. The recursion carries a `visited` set so a diamond (two parents reaching the same child) or an accidental cycle ticks each instance at most once.
663
+ - **Aggregation.** Child orders are concatenated onto the parent's; `reaped` sums; `dueAt` takes the minimum across the tree so the caller's next-wake is the earliest across all levels. Each folded `DeferredFiring` carries a `workflow` field naming the instance it belongs to — **absent means the ticked root, present means a descendant** — so a caller can still tell which instance a deferral came from. The single `now` is threaded through the whole descent, so a reap TTL that trips at the parent's clock trips consistently for children in the same sweep.
664
+ - **Opt-out.** `tick(parentWf, { deep: false })` (CLI `--shallow`) ticks only that one instance — no descent, orders all carry the root's `workflow`. Use it to drive a single instance deliberately (tests, targeted retries); the default deep tick is what production drivers want.
665
+
666
+ **Status child-summary.** `status(parentWf)` enriches each `calls:`-debt entry with a `child: ChildStatusSummary` (`{ workflow, def, done, stalled, debts }`) when a child has been spawned for that step. `stalled` is true when the child (or, recursively, a grandchild on an unpaid `calls:` path) has any stalled debt — a worker that hit `maxAttempts` with no green outcome. This lets a conductor see, from the parent alone, that a `calls:` debt is blocked on a stuck child without separately walking into the child's own `status`. Like `failedRuns`/`attempts`, the field is engine-populated cross-instance state; `model.ts`'s pure single-instance `workflowStatus` never sets it.
667
+
668
+ **Discovering children directly.** A driver can still enumerate children itself — `store.listChildrenByParent`/`findChildByParent` give the reverse index from a parent id — and a propagated reject remains visible via the CHILD's own `status(childWf).debts` (the rejected outcome artifact appears there with `acceptance: 'rejected'`). But with deep tick as the default, a driver that sweeps only roots already reaches all live descendants; explicit child enumeration is for inspection and for the `--shallow` single-instance path, not a requirement for liveness.
669
+
670
+ #### §23.6.9 REL-5 / C2 — atomic fresh-snapshot child provision (never two children, never a stale seed)
671
+
672
+ Before REL-5, spawn read `findChildByParent` outside any transaction and then called `createInstance`, which opened its own transaction — a check-then-insert split across two tx boundaries and backed only by a *non-unique* reverse index. Two driver ticks (even in separate processes) could each observe "no child" and each insert one, leaving duplicate children for the same parent coordinate. REL-5 made the check and the insert one atomic step. C2 then widened that same tx to also carry the parent-snapshot re-read and the input sync, closing the creation-side twin of the C1 publish race (§23.6.6): a child seeded, or an existing child re-provided, from a stale parent value while a concurrent tick advanced the gate.
673
+
674
+ - **`provisionCallsChild(parentWf, step, callsStem, gateStems, now)`** (REL-5's `spawnChildIfAbsent`, widened by C2) wraps a fresh parent-snapshot re-read, `findChildByParent`, the find-or-create insert, AND the sync of every mapped `callsInputs` value in a *single* `BEGIN IMMEDIATE`. The first tick to acquire the write lock inserts the child; any other serializes behind it, re-reads the committed row, and re-attaches. Because the parent snapshot is re-read inside the write lock, a child is never seeded (spawn) or re-written (re-provide) from a value older than its committing tx — the fresh in-tx read IS the guard, so "never regress a child input" needs no version comparison. It returns `{ childId, created, provided }`, or `null` on a silent in-tx abort (the gate re-armed, or an F2 debt landed, since the optimistic pre-read). The caller fires `instance`/`settled` only when `created` is true and a `commit`/`provide` for each name in `provided`, so a re-attach with no input change stays silent — matching the pre-REL-5/pre-C2 per-action semantics. A `SchemaRefusalError` (a seed or re-provided parent value illegal per the child's input schema) rolls the whole tx back with no orphan row and no partial input sync, preserving the §23.6.7 F2 contract.
675
+ - **Partial UNIQUE index `workflow_produced_by_unique`** (schema version 8, on `(produced_by_wf, produced_by_path)` where both are `NOT NULL`) is the physical backstop that makes a duplicate impossible for any future writer path; `provisionCallsChild` also catches the unique-constraint error and retries the whole tx once — the retry finds the winner in-tx and syncs its inputs — as a belt-and-suspenders defense (unreachable under node:sqlite's write-lock-serialized connections).
676
+ - **Legacy duplicates are tolerated, not deleted.** If a pre-v8 database already holds duplicate children from the old race, the unique index is *not* created (that would fail) and no data is removed — the migration skips index creation while any duplicate exists (a cheap aggregate re-checked on every open) and relies on the atomic spawn to prevent new ones. Once the operator removes the legacy duplicates, the next open creates the index. Meanwhile `findChildByParent` returns the *oldest* matching child (`ORDER BY created_at, id LIMIT 1`) so every reader converges on the same winner rather than an arbitrary row.
614
677
 
615
678
  ## §24 Artifact judges (`judges:`)
616
679
 
package/docs/embedding.md CHANGED
@@ -25,10 +25,12 @@ const { engine, store } = createEngine({
25
25
 
26
26
  | option | meaning |
27
27
  |-------------|---------|
28
- | `db` | SQLite path. `':memory:'` for an ephemeral store (great in tests). Defaults to `.owenloop/state.db`; parent dirs are created for a file path. |
29
- | `defs` | In-memory definitions as a `Map<string, WorkflowDef>` or an array of `WorkflowDef` (de-duped by name). Takes precedence over `defsDir`. |
28
+ | `db` | SQLite path. `':memory:'` for an ephemeral store (great in tests). Defaults to `.owenloop/state.db`; parent dirs are created for a file path. When the **default** path is used, a symlinked `.owenloop` directory — and a symlinked `state.db` file (or its SQLite `-wal`/`-shm`/`-journal` sidecars) inside a real `.owenloop` — is refused (filesystem-isolation guard against a hostile checkout redirecting state writes); an explicit `db` path is the caller's responsibility and is created/opened as-is. |
29
+ | `defs` | In-memory definitions as a `Map<string, WorkflowDef>` or an array of `WorkflowDef` (de-duped by name). Takes precedence over `defsDir`. Validated as a whole before use — see [In-memory definitions](#in-memory-definitions). |
30
30
  | `defsDir` | Directory of `*.yaml` definitions, loaded via `loadDefs`. A missing dir yields no defs (lenient, like the CLI), not an error. |
31
31
  | `reapTtlMs` | Forwarded to the `Engine` — the stranded-lease reap TTL. |
32
+ | `maxLeaseMs` | Forwarded to the `Engine` — an opt-in hard cap on total lease lifetime (per-step `maxLease:` overrides). Unset (default): no cap; heartbeats extend a lease indefinitely. Set it only as a runaway backstop — it can reap a healthy, still-beating job. |
33
+ | `maxCallDepth` | Forwarded to the `Engine` — the hard cap on `calls:` composition depth (root instance = depth 0). Defaults to 64. Defense in depth against a `calls:` cycle; only relevant when you hand-wire a custom `DefResolver` that construction-time validation can't inspect. |
32
34
  | `onEvent` | A push-style observer registered at construction (equivalent to `engine.subscribe`). See [Events](#events). |
33
35
  | `onListenerError` | Where a throwing listener's error is routed (default: swallowed). |
34
36
 
@@ -50,6 +52,11 @@ const engine = new Engine(store, (name) => {
50
52
  });
51
53
  ```
52
54
 
55
+ `loadDefs` already validates what it reads from disk. If you build a def set by
56
+ hand and want the same whole-set checks the factory applies (cross-def `calls:`
57
+ validity and cycle detection, not just per-def `validateDef`), run it through
58
+ the exported `finalizeDefs(rawMap)` before wiring the resolver over it.
59
+
53
60
  ## The worker step
54
61
 
55
62
  The shape is the same as a CLI wiring — `tick` to pull orders, run them, then
@@ -73,13 +80,36 @@ for (const order of orders) {
73
80
  const status = engine.status(wf); // { done, debts, eligible, blocked, inFlight }
74
81
  ```
75
82
 
76
- Each `order` is self-contained: `prompt`, `consumes` (the captured green
77
- inputs), `owes` (the owed outputs + their accumulated reason threads), plus
83
+ Each `order` is self-contained: `workflow` (the instance it belongs to — see
84
+ deep tick below), `prompt`, `consumes` (the captured green inputs), `owes` (the
85
+ owed outputs + their accumulated reason threads), plus
78
86
  `inputs`/`outputs`/`model`/`worker`/`command`/`spec`. A consumer rejecting an
79
87
  upstream artifact is just `engine.reject(wf, path, by, text)`; the forward
80
88
  cascade and stall liveness behave exactly as they do under the CLI, because
81
89
  it's the same engine.
82
90
 
91
+ `engine.tick(wf)` is **deep by default**: after ticking `wf` it descends into
92
+ every live `calls:` child (recursively), folding their orders, `reaped`, folded
93
+ `deferred` (each descendant's deferral tagged with its `workflow`; a root-frame
94
+ deferral stays unstamped), and the tree-minimum `dueAt`
95
+ into the one `TickResult`. So an order in the list may belong to a child, not
96
+ `wf` — commit (`green`/`emit`/`seal`/`reject`) and `close` against
97
+ `order.workflow`, not the id you passed to `tick`. Pass `engine.tick(wf, { deep:
98
+ false })` to tick just that one instance (every order then carries `wf`). The
99
+ childless example above never triggers descent, so `order.workflow === wf`
100
+ throughout; it matters once a step declares `calls:`.
101
+
102
+ **Migrating from shallow ticking.** Because a single deep `engine.tick(root)`
103
+ can advance *many* workflow instances in one call, an embedder moving off the
104
+ one-workflow-per-tick assumption must key any per-workflow logic —
105
+ checkpointing, batching, an external transaction, a progress counter — on
106
+ `order.workflow`, not on the id it passed to `tick`. Don't assume every order in
107
+ one `TickResult` belongs to the instance you ticked; each order carries its own
108
+ `order.workflow`, and a folded deferral is stamped with a descendant's
109
+ `workflow` (absent means the root you ticked). `engine.tick(wf, { deep: false })`
110
+ (CLI `--shallow`) stays the deliberate single-instance escape hatch when you
111
+ want exactly the old behavior.
112
+
83
113
  `order.worker` is where dispatch-by-executor-type earns its keep in an
84
114
  embedder: rather than every order going to the same LLM-driving `runYourWorker`,
85
115
  branch on it before deciding how to run the order.
@@ -156,9 +186,17 @@ The event union (exported as `EngineEvent`):
156
186
  commits (and the cascade has already settled inside that tx), so a listener
157
187
  that calls `status`/`tick`/`green` observes fully-committed, settled state —
158
188
  there is no open transaction to corrupt. A state-changing verb fires its
159
- specific `commit`/`instance` event followed by a `settled`; `close` fires only
160
- `closed` (it releases a lease, it doesn't touch artifact state); `tick` fires
161
- nothing (it hands you orders directly).
189
+ specific `commit`/`instance` event followed by a `settled`. `close` fires only
190
+ its `closed` lifecycle event and `tick` fires nothing of its own — `close`
191
+ otherwise just releases a lease and `tick` just hands you orders — but either
192
+ can advance `calls:` composition as a side
193
+ effect, and that composition fires events: a deep `tick` that spawns a child
194
+ emits `instance` + `settled`, and one that machine-greens or re-arms a parent
195
+ `calls:` artifact from a child's outcome emits `commit` + `settled` on the
196
+ parent; a `close` whose run belongs to a `calls:` child prompts that same
197
+ parent maintenance, so it too can emit `commit`/`settled` on the parent. Only
198
+ for an instance with no `calls:` parent or children does `close` fire just
199
+ `closed` and `tick` fire nothing.
162
200
  - **Synchronous, registration order.** The engine is single-writer and
163
201
  synchronous; listeners fire in the order they subscribed, on the calling
164
202
  thread. This is an in-process observer, not an async/cross-process bus —
@@ -188,6 +226,14 @@ const research = parseDef(/* a WorkflowDef-shaped object or YAML you parsed */);
188
226
  const { engine } = createEngine({ db: ':memory:', defs: [research] });
189
227
  ```
190
228
 
229
+ An in-memory `defs` set is validated as a whole — exactly like the filesystem
230
+ `defsDir` loader — before the engine is returned: `calls:` targets must exist,
231
+ `callsInputs` keys must be valid, a called def must have exactly one child
232
+ output, each def passes `validateDef`, and the set is checked for `calls:`
233
+ cycles. An invalid set **throws `DefError`** instead of silently registering.
234
+ The returned `defs` is a validated *copy*, so mutating the `Map`/array you
235
+ passed after construction no longer changes resolution.
236
+
191
237
  ## Lifecycle & concurrency
192
238
 
193
239
  - **Long-lived.** Create the `engine`/`store` once per database and reuse them;
@@ -216,6 +262,40 @@ all the shared types (`Order`, `CommitResult`, `WorkflowStatus`, `WorkflowDef`,
216
262
  `EngineEvent`, `EngineListener`, …). For most hosts, `createEngine` + the engine methods + the `Order` /
217
263
  `CommitResult` / `WorkflowStatus` types are the whole surface you need.
218
264
 
265
+ The index also exports a **read-only credential surface** for hosts that need
266
+ to read a stored hub credential through owenloop's own store logic instead of
267
+ duplicating it: `readStoredCredential(origin, opts)` returns the `Credential`
268
+ for a hub origin **and slot** (or `null` if none is stored), using the same
269
+ backend selection as the CLI — macOS Keychain, or the `0600` credentials file
270
+ elsewhere / with `OWENLOOP_NO_KEYCHAIN=1` (see
271
+ [where the credential lands](cli.md#hub-login--connect--push--logout)). The
272
+ backend is chosen once and a keychain-backed read never falls through to the
273
+ file; a corrupt keychain entry reads as absent. The origin is normalized with
274
+ the exported `normalizeOrigin` (invalid or plaintext-remote origins throw,
275
+ exactly as in the CLI), and `ReadStoredCredentialOpts` lets tests and hosts
276
+ inject `env` and a `Keychain` backend — whose `get`/`set`/`delete` now take
277
+ `(service, account)`, the account being the slot.
278
+
279
+ `opts` is required, and carries the slot selector: `{ principal: 'human' }` or
280
+ `{ principal: 'agent', account?: string }` (`account` defaults to `'default'`,
281
+ and is only meaningful for `'agent'`). The companion exports `credentialSlot`
282
+ (selector → the canonical slot string `human` / `agent:<account>`, throwing on
283
+ an invalid account name) and `keychainServiceFor` (normalized origin → the
284
+ keychain service name) let a host address the same slots the CLI does without
285
+ re-deriving the keying. Slots never fall back to one another: an empty
286
+ `agent:ci` reads as `null` even when `human` is populated. The function never
287
+ logs or echoes the secret, and there is deliberately no write or delete companion on the public
288
+ surface — `login`/`logout` remain the only way to store or remove a credential.
289
+
290
+ One behaviour to handle: when the user has set `OWENLOOP_CREDENTIAL_COMMAND` (an
291
+ external command that supplies the credential, ahead of both stores — see
292
+ [where the credential lands](cli.md#hub-login--connect--push--logout)),
293
+ `readStoredCredential` consults that command instead of any store, and a failing
294
+ command **throws** rather than returning `null`. So in that mode `null` still
295
+ means "no credential", but an exception means "the configured command could not
296
+ produce one" — surface its message rather than treating it as an empty slot,
297
+ because falling back to a local store is exactly what the setting forbids.
298
+
219
299
  `SUPPORTED_ENGINE_VERSION` is the programmatic form of the design.md §27
220
300
  engine-version contract — the highest `engine:` a def may declare and still load.
221
301
  Tooling can preflight-check a def's `engine:` field against it before handing the
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "owenloop",
3
- "version": "0.2.1",
3
+ "version": "0.4.0",
4
4
  "description": "Deterministic rails for agentic workflows. Declare steps and dependencies; the engine guarantees order, redoes what changes invalidate, and stops what keeps failing — instead of hoping the agent follows through.",
5
5
  "license": "Apache-2.0",
6
6
  "author": "typicalday",