@memberjunction/task-graph 6.1.0-edge.1 → 6.1.0-edge.3

package/LICENSE

@@ -1,7 +1,183 @@
-ISC License
+Business Source License 1.1
 
-Copyright (c) 2023 MemberJunction
+License text copyright (c) 2024 MariaDB plc, All Rights Reserved.
+"Business Source License" is a trademark of MariaDB plc.
 
-Permission to use, copy, modify, and/or distribute this software for any purpose with or without fee is hereby granted, provided that the above copyright notice and this permission notice appear in all copies.
+-----------------------------------------------------------------------------
 
-THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+Parameters
+
+Licensor:             Blue Cypress, Inc.
+
+Licensed Work:        MemberJunction.
+                      The Licensed Work is (c) 2023-2026 Blue Cypress, Inc.
+
+Additional Use Grant: Subject to the terms of this License, Licensor grants
+                      you the following additional rights to make Production
+                      Use of the Licensed Work.
+
+                      1. Internal Use
+
+                      You may make production use of the Licensed Work for
+                      your own internal business or organizational operations.
+
+                      2. Nonprofit Use
+
+                      If you are a Nonprofit, you may make production use of
+                      the Licensed Work for the operations and activities of
+                      your Organizational Family.
+
+                      3. MemberJunction Certified Program Use
+
+                      If you are authorized by Licensor under the
+                      MemberJunction Certified Program to provide professional
+                      services using the Licensed Work, you may make
+                      production use of the Licensed Work in providing such
+                      professional services to a client, provided that:
+
+                      (a) the Licensed Work is deployed in, and the applicable
+                      production use occurs within, an environment owned,
+                      leased, licensed, subscribed to, or otherwise controlled
+                      by that client; and
+
+                      (b) the production use is for that client's own internal
+                      business or organizational operations or is otherwise
+                      independently permitted to that client under this
+                      Additional Use Grant.
+
+                      4. Definitions Applicable to the Additional Use Grant
+
+                      "Affiliate" means, with respect to a specified Person,
+                      any other Person that directly or indirectly Controls,
+                      is Controlled by, or is under common Control with such
+                      specified Person.
+
+                      "Control" (including the terms "Controls," "Controlled
+                      by," and "under common Control with") means the direct
+                      or indirect possession of the power to direct or cause
+                      the direction of the management and policies of a
+                      Person, whether through ownership of voting interests,
+                      by contract, or otherwise.
+
+                      "Organizational Family" means, with respect to a Person,
+                      (a) such Person and its Affiliates, and (b) any
+                      nonprofit organization, governmental entity, chapter,
+                      division, local affiliate, regional affiliate, state
+                      affiliate, national affiliate, or other entity that is
+                      formally affiliated with such Person through governing
+                      documents, a charter, bylaws, a membership agreement, or
+                      another written organizational instrument, and is
+                      recognized under such documents as part of the same
+                      organizational structure.
+
+                      "Nonprofit" means a Person recognized by the Internal
+                      Revenue Service as exempt from federal income taxation
+                      under Section 501(c)(3), 501(c)(4), 501(c)(5), or
+                      501(c)(6) of the Internal Revenue Code, or a foreign
+                      organization recognized under substantially equivalent
+                      laws.
+
+                      A Person claiming eligibility as a Nonprofit shall, upon
+                      Licensor's reasonable request, provide documentation
+                      reasonably sufficient to demonstrate that it qualifies
+                      as a Nonprofit. If such Person materially misrepresents,
+                      or is unable to demonstrate, its qualification as a
+                      Nonprofit, the rights granted to such Person under
+                      Section 2 of this Additional Use Grant shall terminate.
+
+                      "MemberJunction Certified Program" means Licensor's
+                      then-current program for certifying and authorizing a
+                      Person to provide professional services using the
+                      Licensed Work.
+
+                      "Person" means any individual, corporation, limited
+                      liability company, partnership, association, nonprofit
+                      organization, governmental entity, or other legal or
+                      organizational entity.
+
+Change Date:          Four (4) years from the date the Licensed Work is first
+                      made available.
+
+Change License:       MIT License.
+
+For information about alternative licensing arrangements for the Licensed
+Work, please contact Blue Cypress, Inc.
+
+-----------------------------------------------------------------------------
+
+Terms
+
+The Licensor hereby grants you the right to copy, modify, create derivative
+works, redistribute, and make non-production use of the Licensed Work. The
+Licensor may make an Additional Use Grant, above, permitting limited
+production use.
+
+Effective on the Change Date, or the fourth anniversary of the first publicly
+available distribution of a specific version of the Licensed Work under this
+License, whichever comes first, the Licensor hereby grants you rights under
+the terms of the Change License, and the rights granted in the paragraph
+above terminate.
+
+If your use of the Licensed Work does not comply with the requirements
+currently in effect as described in this License, you must purchase a
+commercial license from the Licensor, its affiliated entities, or authorized
+resellers, or you must refrain from using the Licensed Work.
+
+All copies of the original and modified Licensed Work, and derivative works
+of the Licensed Work, are subject to this License. This License applies
+separately for each version of the Licensed Work and the Change Date may vary
+for each version of the Licensed Work released by Licensor.
+
+You must conspicuously display this License on each original or modified copy
+of the Licensed Work. If you receive the Licensed Work in original or
+modified form from a third party, the terms and conditions set forth in this
+License apply to your use of that work.
+
+Any use of the Licensed Work in violation of this License will automatically
+terminate your rights under this License for the current and all other
+versions of the Licensed Work.
+
+This License does not grant you any right in any trademark or logo of
+Licensor or its affiliates (provided that you may use a trademark or logo of
+Licensor as expressly required by this License).
+
+TO THE EXTENT PERMITTED BY APPLICABLE LAW, THE LICENSED WORK IS PROVIDED ON
+AN "AS IS" BASIS. LICENSOR HEREBY DISCLAIMS ALL WARRANTIES AND CONDITIONS,
+EXPRESS OR IMPLIED, INCLUDING (WITHOUT LIMITATION) WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NON-INFRINGEMENT, AND
+TITLE.
+
+MariaDB hereby grants you permission to use this License's text to license
+your works, and to refer to it using the trademark "Business Source License",
+as long as you comply with the Covenants of Licensor below.
+
+-----------------------------------------------------------------------------
+
+Covenants of Licensor
+
+In consideration of the right to use this License's text and the "Business
+Source License" name and trademark, Licensor covenants to MariaDB, and to all
+other recipients of the licensed work to be provided by Licensor:
+
+1. To specify as the Change License the GPL Version 2.0 or any later version,
+   or a license that is compatible with GPL Version 2.0 or a later version,
+   where "compatible" means that software provided under the Change License
+   can be included in a program with software provided under GPL Version 2.0
+   or a later version. Licensor may specify additional Change Licenses without
+   limitation.
+
+2. To either: (a) specify an additional grant of rights to use that does not
+   impose any additional restriction on the right granted in this License, as
+   the Additional Use Grant; or (b) insert the text "None".
+
+3. To specify a Change Date.
+
+4. Not to modify this License in any other way.
+
+-----------------------------------------------------------------------------
+
+Notice
+
+The Business Source License (this document, or the "License") is not an Open
+Source license. However, the Licensed Work will eventually be made available
+under an Open Source License, as stated in this License.

package/README.md

@@ -0,0 +1,214 @@
+# @memberjunction/task-graph
+
+Durable execution of task graphs: submission, the dispatcher, the claim protocol, and the runners
+that turn a node into work.
+
+> **New to workflows?** Start with the [Workflows and Task Graphs
+> Guide](../../guides/WORKFLOW_AND_TASK_GRAPH_GUIDE.md), which covers what a workflow is, when to
+> use one, and every rule that decides what happens next. To **step a live run**, see the
+> [Workflow Debugger Guide](../../guides/WORKFLOW_DEBUGGER_GUIDE.md). This README is the package tour.
+
+---
+
+## What this package is for
+
+A task graph is work that has to **outlive the thing that asked for it**. Before durable execution,
+a multi-step agent plan lived inside one agent run: a page reload lost it, a server restart orphaned
+it, and no channel other than the one that started it could see it.
+
+This package is the other half of that split. Something else *produces* a graph; this package makes
+it durable and runs it.
+
+```mermaid
+graph TB
+    subgraph Producers["Producers — anyone"]
+        P1["Flow agent<br/><i>compiled</i>"]
+        P2["Loop agent<br/><i>emitted</i>"]
+        P3["Entity action"]
+        P4["TaskGraph.Submit<br/><i>remote op</i>"]
+    end
+
+    P1 & P2 & P3 & P4 --> SVC["<b>TaskGraphService.Submit</b><br/>validate · resolve · persist"]
+    SVC --> ROWS["Task + TaskDependency"]
+    ROWS --> DISP["<b>TaskGraphDispatcher</b><br/>poll · claim · execute · settle"]
+
+    DISP --> R1["TaskAgentRunner"]
+    DISP --> R2["TaskActionRunner"]
+    DISP --> R3["TaskLoopExecutor"]
+    DISP --> R4["a person"]
+
+    style SVC fill:#7c5295,stroke:#563a6b,color:#fff
+    style ROWS fill:#2d8659,stroke:#1a5c3a,color:#fff
+    style DISP fill:#b8762f,stroke:#8a5722,color:#fff
+```
+
+**Submission never waits for execution.** `Submit` returns as soon as the graph is durable. That
+split is what makes the engine invocation-agnostic: an agent, a scheduled job, a Slack message and a
+manual UI all call the same method, and whichever dispatcher instance is running picks the work up.
+
+---
+
+## What it deliberately does not do
+
+**It does not decide graph semantics.** Eligibility, failure propagation, parent rollup, skip
+cascade, exclusive-group resolution and stall detection all come from pure, dependency-free
+functions in [`@memberjunction/ai-core-plus`](../AI/CorePlus). That is not tidiness — it is what
+stops the in-run executor and the durable executor from drifting apart. Neither owns the rules.
+
+**It does not import MJServer.** Provider minting and agent execution arrive as injected seams
+(`ProviderFactory`, `TaskAgentRunner`, `TaskActionRunner`), so the dependency runs
+MJServer → task-graph and never the reverse. That is also what keeps the dispatcher unit-testable
+without standing up the agent framework.
+
+---
+
+## The pieces
+
+| File | Responsibility |
+|---|---|
+| `TaskGraphService.ts` | Validate a spec, resolve names to IDs, write parent + children + edges. The only write path. |
+| `TaskGraphDispatcher.ts` | Poll, claim, execute, evaluate edges, propagate, roll up, settle, credit cost. |
+| `TaskClaimStore.ts` | The atomic claim protocol — guarded writes so two instances cannot run the same task. |
+| `TaskLoopExecutor.ts` | `ForEach` / `While` semantics: bounds, ordering, concurrency, delay, failure. |
+| `DispatcherConditionEvaluator.ts` | Edge conditions, over the superset context both dialects can read. |
+| `TaskGraphSubmitterImpl.ts` | Registers the durable submitter under the `ClassFactory` seam. |
+| `operations/` | The `TaskGraph.*` remote operations. |
+| `debug-state.ts` | Durable `$.debug` bag — pause, breakpoints, step allowance, `skipBreakpointTaskID`, edge overrides. Pure claim-gate. |
+| `task-graph-kick.ts` | Process-local kick set. `Start()` registers; `Submit` pokes every running dispatcher so the first pass is immediate. |
+
+---
+
+## Claiming: why a task runs exactly once
+
+Multiple dispatcher instances poll the same table. Correctness comes from a **guarded write**, not
+from coordination: claiming is an update that only succeeds if the row still looks the way the
+claimer expects.
+
+```mermaid
+sequenceDiagram
+    participant A as Instance A
+    participant B as Instance B
+    participant DB as Task row
+
+    A->>DB: claim if unclaimed / expired
+    B->>DB: claim if unclaimed / expired
+    DB-->>A: 1 row updated ✅
+    DB-->>B: 0 rows updated ❌
+    Note over B: defers — no lock, no retry storm
+    loop while running
+        A->>DB: heartbeat (extends ClaimExpiresAt)
+    end
+    A->>DB: complete IF still owned by A
+```
+
+If A dies, its claim expires and reconciliation makes the task claimable again. If A finishes but
+the row changed underneath it (cancelled, reassigned, reclaimed), the guarded completion refuses and
+A defers — overwriting would undo a newer, deliberate decision.
+
+---
+
+## Runners are seams, not implementations
+
+| Seam | Implemented by | Absent means |
+|---|---|---|
+| `TaskAgentRunner` | `TaskGraphAgentRunner` (MJServer) | agent nodes cannot run here |
+| `TaskActionRunner` | `TaskGraphActionRunner` (MJServer) | action nodes stay `Pending`, **not** Failed |
+| `ProviderFactory` | `TaskGraphProviderFactory` (MJServer) | required |
+| `TaskContinuationDeliverer` | `TaskGraphContinuationDeliverer` (MJServer) | outcomes are recorded but not announced |
+| `TaskGraphObserver` | the frame resolver (MJServer) | nobody is watching; behaviour identical |
+
+**A host with no runner is limited, not broken.** "Nobody here can run this" is not the same as
+"this ran and did not work", so those tasks stay visible and claimable by an instance that can.
+
+> **An Agent node starts a brand-new `AIAgentRun`** — a root run, linked from `Task.AgentRunID`
+> rather than nested under the submitting run. The relationship is expressed by the Task row.
+
+---
+
+## Starting a dispatcher
+
+MJAPI does this for you (`StartTaskGraphDispatcher`). For a custom host:
+
+```typescript
+import { TaskGraphDispatcher, LoadTaskGraphOperations } from '@memberjunction/task-graph';
+
+LoadTaskGraphOperations();   // registers the remote operations + the durable submitter
+
+const dispatcher = new TaskGraphDispatcher(
+    providerFactory,
+    agentRunner,
+    contextUser,
+    { InstanceID: `worker-${process.pid}` },
+    continuationDeliverer,   // optional
+    observer,                // optional
+    actionRunner,            // optional
+);
+await dispatcher.Start();
+```
+
+Without `LoadTaskGraphOperations()` nothing registers the submitter, and
+`GetTaskGraphSubmitter()` returns `null` — which callers must report rather than swallow.
+
+`Start()` registers a kick and fires one immediately. Do not wait for the first `setInterval`
+tick — that is a 5s dead pause after Debug/Run. Submit cannot import the dispatcher instance
+(submit and execute are separate halves), so it calls `KickTaskGraphDispatchers()` instead.
+
+---
+
+## Debugging a live graph
+
+Every control is a **gate on claiming**, never on running. State lives under `$.debug` on the
+parent task's `InputPayload`. The debugger UI and the dispatcher never talk directly — they
+rendezvous on the row. See [`debug-state.ts`](src/debug-state.ts) and the
+[Workflow Debugger Guide](../../guides/WORKFLOW_DEBUGGER_GUIDE.md).
+
+- **Start-paused.** Submit writes `paused: true` before any dispatcher has seen the graph.
+- **Breakpoints.** The claim gate treats an armed, eligible task like a hold. Resume from a
+  breakpoint stamps `skipBreakpointTaskID` so that one claim is allowed through; without it the
+  next poll re-hits the same Pending row. `ForEach` is one task — one Continue runs the loop.
+- **Edge overrides.** Authored `'true'` / `'false'` on a `TaskDependency` id, honored by every
+  instance, survive a restart.
+- **Kick.** After Submit (and on `Start()`), `KickTaskGraphDispatchers()` runs a pass now.
+
+The Angular drop-in is `<mj-task-graph-debugger>` in
+[`@memberjunction/ng-task-graph-editor`](../Angular/Generic/task-graph-editor/README.md).
+
+---
+
+## Cost rollup
+
+When a graph settles, the dispatcher credits its spending back to the run that submitted it, writing
+the `…Rollup` columns on `AIAgentRun`:
+
+- `TotalCost` — what the submitting run itself spent. **Never rewritten here.**
+- `TotalCostRollup` — that run plus everything it caused.
+
+This cannot happen during the run: a submitting run *ends at submission*, so at the moment it
+computes its own totals the graph has not spent anything yet. See the guide's [cost
+section](../../guides/WORKFLOW_AND_TASK_GRAPH_GUIDE.md#cost-and-tokens-the-seam).
+
+---
+
+## Testing
+
+```bash
+cd packages/TaskGraph && pnpm test
+```
+
+Unit tests cover the pure pieces — loop semantics, dispatchable kinds, configuration persistence,
+parent metadata. The dispatcher driving **real rows against SQL Server** is covered by the `IT74`
+bundle in `@memberjunction/integration-test-suite`, which uses a stub agent runner so it stays in
+the deterministic tier: no model calls, no tokens, real claim protocol, real condition evaluator,
+real rollup.
+
+---
+
+## Related
+
+- [Workflows and Task Graphs Guide](../../guides/WORKFLOW_AND_TASK_GRAPH_GUIDE.md) — start here
+- [Workflow Debugger Guide](../../guides/WORKFLOW_DEBUGGER_GUIDE.md) — step a live run
+- [`@memberjunction/ng-task-graph-editor`](../Angular/Generic/task-graph-editor/README.md) — canvas and drop-in debugger
+- [`@memberjunction/ai-core-plus`](../AI/CorePlus) — the spec, validator, compiler, pure algorithms,
+  payload mapping and layout
+- [`@memberjunction/ai-agents`](../AI/Agents) — the agent framework and `FlowAgentType`
+- [`packages/Actions/CLAUDE.md`](../Actions/CLAUDE.md) — Actions as boundaries