mcp-event-intelligence 0.4.0 → 0.5.0

package/CHANGELOG.md

@@ -1,5 +1,64 @@
 # Changelog
 
+## [0.5.0] - 2026-09-30
+
+### Pattern AST v2 / advanced CEP
+
+- add additive recursive Pattern AST v2 while keeping legacy composite triggers executable;
+- add nested `allOf` / `anyOf` / `sequence`, explicit `next` / `followedBy` / `followedByAny` contiguity, repeat/optional quantifiers and greedy/lazy matching;
+- add bounded negative patterns (`notNext`, `notFollowedBy`) and first-class calendar/absence/debounce/rate/distinct temporal nodes;
+- add within/sliding/tumbling/hopping/session/count windows;
+- add cross-event arithmetic comparisons plus startsWith/endsWith/regex/notIn/between/isNull/type predicates;
+- add sum/avg/min/max/count/countDistinct/first/last/nth/stddev/percentile aggregates and derived-event measures;
+- add changed/changedFrom/increasedBy/decreasedBy/delta/percentChange/stableFor/crossesAbove/crossesBelow state operators;
+- add partitioned durable CEP buffers, overlap/after-match selection policies, bounded candidate budgets and bounded event-time retention;
+- add vendor-neutral first-class `semantic` nodes behind `SemanticEvaluator`; TypeSafe Jev remains optional and explicit;
+- enforce per-node semantic timeout, trigger-level semantic-evaluation budgets and audit every semantic decision, including rejects;
+- validate Pattern v2 refs, partition dimensions, durations, payload paths and derived measures against live source contracts;
+- add `compileLegacyTriggerToPatternV2()` for compatibility/migration and differential verification;
+- expose the complete v2 language through `describeTriggerLanguage()` / `trigger_language_describe`;
+- add end-to-end coverage for advanced predicates, quantifiers, contiguity, windows, arithmetic, state, semantic evaluation, partition isolation, negative deadlines and derived measures.
+
+### Event-time correctness
+
+- preserve host `receivedAt` separately from provider `occurredAt` through correlation evidence, derived lineage and activation hydration;
+- sequence and window evaluation no longer depend on delivery order;
+- partial matches are selected by compatible event-time span and can be re-anchored when an earlier event arrives late;
+- add explicit `eventTime.allowedLatenessMs` watermarks for bounded late-event tolerance;
+- events older than the watermark may finish retained compatible partials but cannot seed stale windows;
+- late events outside `withinMs` are isolated instead of creating false composite matches.
+
+### Agent authoring language
+
+- add a versioned, self-describing trigger operator registry;
+- add embedded `host.describeTriggerLanguage()` / scoped discovery;
+- add non-mutating MCP `trigger_language_describe` with category/operator filtering;
+- make the MCP `trigger_plan` tool reuse the package's canonical `TriggerPlanInputSchema` instead of maintaining a duplicate input schema;
+- expose all canonical temporal operators through `TriggerPlanInput.temporal`;
+- validate temporal event references and `distinct.path` against live source schemas before compilation;
+- add registry/schema parity tests so supported operators cannot silently drift out of the agent-visible catalog.
+
+### Trigger composition
+
+- add explicit `conditionOnly: true` programs with no runtime target, continuation or derived output;
+- agent-friendly planning can compile condition-only programs;
+- identical active `(connection, event, arguments)` clauses share one upstream MCP EventSubscription with observable local fan-out count.
+
+### MCP Events compatibility
+
+- external draft assumptions are centralized behind a versioned compatibility profile instead of being spread through the runtime;
+- the current default profile remains the 2026-09-25 experimental extension snapshot;
+- host-owned/embedded integration remains the primary architecture; standalone/provider-native paths are compatibility surfaces, not a new deployment requirement.
+
+
+### Compatibility
+
+- Pattern AST v2 is additive; legacy composite/temporal triggers remain executable unchanged;
+- the internal persisted Event Intelligence protocol/schema identifiers remain on the v0.1 compatibility line;
+- `compileLegacyTriggerToPatternV2()` is available for migration/differential verification without forcing existing persisted triggers to rewrite;
+- MCP Events remains experimental and isolated behind the versioned compatibility profile.
+
+
 ## [0.4.0] - 2026-09-25
 
 Current MCP Events draft alignment.

package/README.md

@@ -4,7 +4,7 @@
 
 [![CI](https://github.com/sarooo17/event-intelligence/actions/workflows/ci.yml/badge.svg)](https://github.com/sarooo17/event-intelligence/actions/workflows/ci.yml)
 [![npm](https://img.shields.io/npm/v/mcp-event-intelligence.svg)](https://www.npmjs.com/package/mcp-event-intelligence)
-[![MCP Registry](https://img.shields.io/badge/MCP%20Registry-v0.4.0-5b5bd6)](https://registry.modelcontextprotocol.io/?q=io.github.sarooo17%2Fevent-intelligence)
+[![MCP Registry](https://img.shields.io/badge/MCP%20Registry-v0.5.0-5b5bd6)](https://registry.modelcontextprotocol.io/?q=io.github.sarooo17%2Fevent-intelligence)
 [![License](https://img.shields.io/badge/license-Apache--2.0-blue.svg)](LICENSE)
 
 MCP Event Intelligence is an experimental event runtime for agents that need to react to **future conditions over multiple event sources** without keeping an LLM or agent loop alive.
@@ -19,7 +19,7 @@ An agent can express an intent such as:
 
 > When this PR is merged, the production deploy succeeds, and no error is observed for 10 minutes, wake this task and review the release.
 
-Event Intelligence persists that continuation independently of the model, waits for the world to satisfy it, and wakes the host only when necessary.
+Event Intelligence persists the future condition independently of the model, waits for the world to satisfy it, and wakes the host only when the condition has an activation target. Conditions may also exist without an agent continuation; higher-level facts are composed separately through derived-event triggers.
 
 ## Embed it in an existing agent host
 
@@ -59,7 +59,17 @@ Event Intelligence enumerates the host registry automatically. GitHub, Gmail, pr
 
 ### Agent-first trigger flow
 
-Agents no longer need to construct the low-level trigger DSL directly for common cases. Ask EI to compile an agent-friendly plan against the event sources that are actually available:
+Agents do not need to construct the low-level trigger DSL directly. The public authoring flow is self-describing: discover the live event sources, inspect the trigger language only when needed, then ask EI to compile an agent-friendly plan against the sources that are actually available:
+
+```js
+const temporalLanguage = ei.describeTriggerLanguage({
+  category: 'temporal',
+});
+
+// returns the supported temporal operators, required fields and examples
+```
+
+The same catalog is available through the optional MCP control plane as `trigger_language_describe`. This keeps operator discovery out of the agent's permanent prompt/context and makes new operators discoverable on demand.
 
 ```js
 const plan = await ei.planTrigger({
@@ -91,9 +101,78 @@ await ei.triggerControl.createTrigger({
 });
 ```
 
-`planTrigger()` is deterministic. It does not call a model. It resolves event names to live source/server IDs, validates predicate paths against advertised payload schemas, compiles `all` / `any` / `sequence` / `count`, and returns the canonical trigger definition plus the required connection IDs.
+`planTrigger()` is deterministic. It does not call a model. It resolves event names to live source/server IDs, validates predicate paths against advertised payload schemas, compiles `all` / `any` / `sequence` / `count`, accepts the canonical temporal operator set (`calendar`, `absence`, `not`, `unless`, `after`, `until`, `debounce`, `threshold`, `rate`, `distinct`), and returns the canonical trigger definition plus the required connection IDs.
+
+The canonical trigger definition remains the engine IR/advanced API. Agents should normally author through `planTrigger()` / `trigger_plan`, using `describeTriggerLanguage()` / `trigger_language_describe` for operator discovery.
+
+### Pattern AST v2: advanced CEP
+
+For advanced event programs, `TriggerPlanInput.patternV2` is the preferred authoring surface. Event declarations still describe discovered MCP event sources once; the recursive AST refers to those aliases and composes them without duplicating transport/provider configuration.
+
+```js
+const plan = await ei.planTrigger({
+  events: [
+    { id: 'order', event: 'erpnext.sales_order.created' },
+    { id: 'failure', event: 'payments.failed' },
+    { id: 'ticket', event: 'support.ticket.created' },
+  ],
+  withinMs: 30 * 60 * 1000,
+  patternV2: {
+    version: '2',
+    partitionBy: [{
+      key: 'customer',
+      fields: [
+        { ref: 'order', path: 'customer' },
+        { ref: 'failure', path: 'customer' },
+        { ref: 'ticket', path: 'customer' },
+      ],
+    }],
+    root: {
+      kind: 'aggregate',
+      function: 'sum',
+      ref: 'order',
+      path: 'grand_total',
+      op: 'gte',
+      value: 10000,
+      child: {
+        kind: 'sequence',
+        contiguity: 'followedBy',
+        children: [
+          { kind: 'event', ref: 'order' },
+          {
+            kind: 'repeat',
+            child: { kind: 'event', ref: 'failure' },
+            min: 2,
+            max: 5,
+            mode: 'greedy',
+          },
+          {
+            kind: 'optional',
+            child: { kind: 'event', ref: 'ticket' },
+            mode: 'greedy',
+          },
+        ],
+      },
+    },
+  },
+  target: {
+    runtime: 'agent',
+    kind: 'conversation',
+    id: 'chat-42',
+  },
+  continuation: {
+    instruction: 'Review this customer payment-risk pattern.',
+  },
+});
+```
+
+Pattern AST v2 adds nested boolean/sequence patterns, strict and relaxed contiguity, repeat/optional quantifiers, negative look-ahead, streaming windows, cross-event comparisons, aggregates, state-transition operators, partitioning, overlap/after-match policies and deterministic derived measures.
+
+The runtime is **deterministic first**. A first-class `semantic` node is available only when structural data cannot express the condition reliably. It is vendor-neutral and calls the configured `SemanticEvaluator`; the bundled TypeSafe Jev adapter is optional when `TYPESAFE_API_KEY` is present. Semantic calls are explicit, bounded, timed out and audit-recorded; ordinary CEP execution requires no model API.
 
-The persisted `continuation` answers a separate question from the trigger condition: **what should the agent do after the future condition becomes true?**
+Legacy `expression + temporal + correlation` triggers remain supported. `compileLegacyTriggerToPatternV2()` provides an additive compatibility path rather than forcing stored v1 triggers to migrate synchronously.
+
+The persisted `continuation` answers a separate question from the trigger condition: **what should the agent do after the future condition becomes true?** A trigger can instead set `conditionOnly: true` and persist only the condition state, with no runtime target or continuation.
 
 The wire wake remains deliberately small and reference-only. Embedded hosts also receive an Activation Envelope as the second wake argument. The same envelope can be reconstructed later:
 
@@ -215,6 +294,7 @@ production ERP connector itself.
 - `events/poll` with server-directed `nextPollMs`, nullable cursors and bounded page draining;
 - host-owned push and webhook delivery adapters feeding the same EventOccurrence pipeline;
 - persistent opaque cursors and per-subscription delivery state;
+- shared upstream subscriptions fan out to every active trigger using the same `(connection, event, arguments)` tuple;
 - per-scope source/cursor isolation for shared hosts;
 - single-flight polling per connection plus bounded `hasMore` batch draining;
 - automatic event-source registration;
@@ -244,7 +324,7 @@ Poll, push and webhook are treated only as delivery mechanisms. All three normal
 - event-source discovery;
 - agent-friendly deterministic trigger planning/compilation;
 - MCP subscription `arguments` validated against each source `inputSchema`, kept distinct from EI payload predicates;
-- `eq`, `neq`, `contains`, `in`, `exists`, `gt`, `gte`, `lt`, `lte` predicates;
+- structured predicates including `eq`, `neq`, `contains`, `startsWith`, `endsWith`, `regex`, `in`, `notIn`, `between`, `exists`, `isNull`, `type`, `gt`, `gte`, `lt`, `lte`;
 - persisted continuation contracts separated from trigger conditions;
 - Activation Envelope hydration with matched event evidence;
 - embedded wake callbacks receive `(packet, activation)`;
@@ -306,10 +386,12 @@ This project does **not** propose a replacement for MCP Events and does not clai
 
 ## AI and API keys
 
-Event Intelligence has one optional AI boundary: **semantic correlation**.
+Event Intelligence has one optional AI boundary: **explicit semantic evaluation**.
 
-- `TYPESAFE_API_KEY` enables the bundled TypeSafe Jev evaluator when a trigger explicitly requests semantic correlation.
+- Pattern AST v2 exposes a vendor-neutral `semantic` node; legacy `semanticCorrelation` is the compatibility form.
+- `TYPESAFE_API_KEY` enables the bundled TypeSafe Jev evaluator when a trigger explicitly requests semantic evaluation.
 - embedded hosts may inject a compatible `semanticEvaluator` instead.
+- deterministic predicates, patterns, windows, aggregations, state changes, deadlines and derived measures never require an AI evaluator.
 - there is no bundled OpenAI planner and no OpenAI API dependency.
 
 The agent/harness is already responsible for natural-language reasoning. It can use `trigger_plan` / `planTrigger()` to compile common requests deterministically against discovered source schemas, or submit a canonical trigger definition directly for advanced cases. Deterministic correlation, temporal logic, persistence, derived events and wake delivery require no model API.
@@ -334,7 +416,9 @@ The v0.4 acceptance suite verifies:
 
 A separate live regression also verified real GitHub webhook ingress into the MCP EventOccurrence / composite fan-in path.
 
-## Standalone reference service
+## Reference/debug service
+
+The HTTP service remains useful for local development, conformance work and provider-native regression tests. It is **not** the primary integration model and does not replace the embedded host-owned package path above.
 
 ### Requirements
 
@@ -362,13 +446,13 @@ The standalone service supports manual/provider-native event ingress. It does no
 ### Docker
 
 ```bash
-docker build -t mcp-event-intelligence:0.4.0 .
+docker build -t mcp-event-intelligence:0.5.0 .
 
 docker run --rm \
   -p 3000:3000 \
   -v mcp-event-intelligence-data:/data \
   -e SERVICE_AUTH_TOKEN="$(openssl rand -hex 32)" \
-  mcp-event-intelligence:0.4.0
+  mcp-event-intelligence:0.5.0
 ```
 
 ## Optional MCP control plane
@@ -414,7 +498,11 @@ The detailed execution model, clocks, lifecycle, persistence and trust boundarie
 
 ### Event-time vs processing-time
 
-Normal event windows use event `occurredAt`.
+Normal event windows use event `occurredAt`; MCP ingress also preserves host `receivedAt` when available. Delivery order does not define event order.
+
+Out-of-order events are matched against compatible event-time windows. A late event may re-anchor a partial match to an earlier `occurredAt` only when the complete event-time span still fits `withinMs`; otherwise it starts a separate partial window and cannot create a false sequence.
+
+Triggers may set `eventTime.allowedLatenessMs` (default `0`). EI maintains an event-time watermark per trigger/correlation stream: old partial windows are retained until their expiry falls behind `maxObservedEventTime - allowedLatenessMs`. An event older than the watermark can still complete a retained compatible partial, but cannot seed a new stale window.
 
 Absence/deadline progression uses Event Intelligence processing time. This separation is explicit and tested.
 
@@ -432,7 +520,7 @@ It uses stable wake IDs, persisted delivery state, atomic claim leases, bounded
 
 A derived event says **what became true at a point in history**.
 
-v0.3 intentionally does not implement a mutable current-state/facts database.
+Event Intelligence intentionally does not implement a mutable current-state/facts database.
 
 ## Security
 
@@ -457,7 +545,7 @@ io.github.sarooo17/event-intelligence
 
 ## Project status
 
-**v0.4.x reference implementation / experimental.**
+**v0.5.x reference implementation / experimental.**
 
 The architecture is implemented and exercised end-to-end. Storage is now injectable and scoped, while the bundled JSONL backend remains a single-process reference implementation. Remaining work is primarily production database adapters/HA validation, scale benchmarks and upstream feedback.
 

package/SECURITY.md

@@ -2,12 +2,12 @@
 
 ## Supported versions
 
-The public package is pre-1.0. Security fixes are supported on the current 0.3.x line.
+The public package is pre-1.0. Security fixes are supported on the current 0.5.x line.
 
 | Version | Supported |
 | --- | --- |
-| 0.3.x | Yes |
-| < 0.3 | No |
+| 0.5.x | Yes |
+| < 0.5 | No |
 
 ## Reporting a vulnerability
 
@@ -26,7 +26,7 @@ Useful reports include:
 
 ## Security boundaries
 
-Event Intelligence 0.3.x is an experimental reference implementation with scoped multi-tenant host support. It is not a turnkey hardened hosted service.
+Event Intelligence 0.5.x is an experimental reference implementation with scoped multi-tenant host support. It is not a turnkey hardened hosted service.
 
 Important defaults and constraints:
 

package/bin/mcp-event-intelligence.mjs

@@ -4,14 +4,14 @@ const args = process.argv.slice(2);
 const flags = new Set(args);
 
 if (flags.has('--version') || flags.has('-v')) {
-  console.log('0.4.0');
+  console.log('0.5.0');
   process.exit(0);
 }
 
 if (flags.has('--help') || flags.has('-h')) {
-  console.log(`MCP Event Intelligence v0.4
+  console.log(`MCP Event Intelligence v0.5
 
-Durable temporal event intelligence for sleeping agents.
+MCP-native complex event processing for durable agents.
 
 Usage:
   mcp-event-intelligence          Start the HTTP reference service

package/dist/src/composite/engine.d.ts

@@ -19,8 +19,14 @@ export declare class CompositeTriggerEngine {
     markFired(matchId: string, wakeId: string): Promise<TriggerMatchRecord>;
     markDerivedEmitted(matchId: string, derivedEventId: string): Promise<TriggerMatchRecord>;
     private ingestTimerEvent;
+    private patternPartitionKey;
+    private patternBufferId;
+    private patternDecisionRecord;
+    private applyPatternV2;
+    private evaluatePatternV2Buffer;
     private applyEvent;
     private finalizeEligible;
+    private branchMatch;
     private newMatch;
     private latestMatch;
 }