stateful-mcp 0.1.1

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 stateful-mcp contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,111 @@
+# Stateful MCP Middleware Suite
+
+A modular, version-controlled [Model Context Protocol (MCP)](https://modelcontextprotocol.io) middleware suite that helps LLMs manage stateful workflows incrementally across conversational turns — filters, objects, event logs, and a terminology dictionary — optimizing token consumption and avoiding huge payloads.
+
+Each service is exposed as an MCP server over stdio and launched via the monolith router.
+
+---
+
+## Install
+
+```bash
+npm install -g stateful-mcp      # or: npm install stateful-mcp
+```
+
+> **Runtime:** Bun (>= 1.0) is required to run the server, because the SQLite storage adapter uses `bun:sqlite`. The Memory and PostgreSQL adapters work under any supported runtime; only the SQLite adapter needs Bun.
+
+---
+
+## Quick start
+
+```bash
+# Launch a service (reads config from ./config by default)
+SERVICE_TYPE=filter stateful-mcp
+
+# Or point at an explicit config directory
+stateful-mcp --config-dir /path/to/my-config
+SERVICE_TYPE=object stateful-mcp -c /path/to/my-config
+```
+
+The binary maps to the monolith router in `index.ts`. `SERVICE_TYPE` selects the service: `filter`, `object`, `dictionary`, `log`, or `event` (defaults to `filter`).
+
+---
+
+## Configuration
+
+Configuration lives in the **config directory** and is split by concern:
+
+| File | Purpose |
+|------|---------|
+| `config/tools.config.json` | Tool schemas, validation engines, and query backends. |
+| `config/storage.config.json` | Storage backends (memory / SQLite / PostgreSQL) for session & persistent state, auto-compression thresholds, and `pagination_limits`. |
+| `config/about.config.json` | Markdown docs served by the `*_about` / `*_examples` developer-guidance tools (optional). |
+
+### Where config is read from
+
+`loadMiddlewareConfig` resolves the config directory in this order:
+
+1. `--config-dir <path>` / `-c <path>` / `--config-dir=<path>` (CLI flag)
+2. `STATEFUL_MCP_CONFIG_DIR` (environment variable)
+3. `process.cwd()` (fallback)
+
+Ship your own `config/` and point the package at it — you do **not** rely on the publisher's demo config. A single-file `filter.config.json` (or `config.json`) at the config root is also supported.
+
+### Environment variables
+
+| Variable | Required | Purpose |
+|----------|----------|---------|
+| `SERVICE_TYPE` | no | Which service to start (`filter` default). |
+| `STATEFUL_MCP_CONFIG_DIR` | no | Directory containing `config/`. |
+| `LOG_SERVICE_SECRET` | no | 32-byte HMAC key for stateless `log_next` page tokens. Random if unset. |
+| `WORKSPACE_ID` | no | Fallback dictionary workspace id (default `global`). |
+
+---
+
+## Optional: Native DataFrame query engine
+
+The `dataframe` query engine runs DuckDB SQL natively over CSV, Parquet, or JSONL files.
+
+---
+
+## Building from source
+
+```bash
+bun install
+npm run build         # bundles dist/index.js (bun build) + emits .d.ts + copies assets
+```
+
+`prepublishOnly` runs `npm run build`, so `npm publish` always ships a built package.
+
+---
+
+## Services
+
+1. **Filter Service** — build multi-clause database queries incrementally (VCS-style rule DAG).
+2. **Object Service** — manage structured JSON objects with stateful delta tracking and templating.
+3. **Dictionary Service** — normalize abbreviations to standardized concept identifiers.
+4. **Event Service** — version-controlled append-only logs with branching, LCA merging, and conflict resolution.
+5. **Log Service** — stateless, paginated traversal over filter/object history using HMAC-signed page tokens.
+6. **Form Service** — manage stateful interactive forms with branching logic, step validations, and back-navigation staleness tracking.
+
+Each service exposes `*_about` / `*_examples` developer-guidance tools. LLMs should consult them when handling complex state.
+
+See `docs/` for the full reference (filter, object, form, event, dictionary, log, config, pipeline, browser).
+
+---
+
+## Client-Side & Browser Storage
+
+For web/frontend environments, the suite provides native client-side storage repositories:
+*   **LocalStorage**: Simple web storage for session/persistent state.
+*   **IndexedDB**: Non-blocking asynchronous store for larger state histories.
+
+See [docs/browser.md](docs/browser.md) for details on integrating client-side adapters.
+
+---
+
+## Compliance Test Suite
+
+Verify that custom third-party storage adapters strictly comply with the stateful consistency and isolation contract:
+*   Exposes `runStoreComplianceTests`, `runFormStoreComplianceTests`, `runObjectStoreComplianceTests`, and `runEventStoreComplianceTests` via `"stateful-mcp/adapters/compliance"`.
+*   Asserts uniqueness, routing, isolation, TTL, and VCS compaction correctness.

package/config/about/dictionary.md

@@ -0,0 +1,8 @@
+# Dictionary Service: Strategy & Guidelines
+
+The Dictionary Service normalizes clinical or business terms to their authoritative coordinate mappings.
+
+## Core Rules
+* **Normalize Early**: Always resolve vernacular terminology (e.g. abbreviations, aliases) to formal concept codes before using them in filter conditions or object field values.
+* **Explore**: Use `dictionary_find` to discover terms and synonyms.
+* **Scope Resolution**: Personal aliases shadow global configurations. Rely on the resolved priority outputs.

package/config/about/event.md

@@ -0,0 +1,8 @@
+# Event Service: Strategy & Guidelines
+
+The Event Service models structured append-only event logs (e.g. observation flows, order histories) via version-controlled commit DAG chains.
+
+## Multi-turn Conflict Management
+* **Merge Conflicts**: When merging parallel updates on identical event IDs, a merge conflict is raised.
+* **Inspect & Resolve**: Call `event_merge_inspect` to view conflicting values, and resolve them step-by-step using `event_merge_resolve` strategies.
+* **Commit**: Call `event_merge_commit` only when all conflicts are resolved.

package/config/about/filter.md

@@ -0,0 +1,9 @@
+# Filter Service: Strategy & Guidelines
+
+The Filter Service builds database queries incrementally. It is designed to save context window tokens and allow interactive redirection by the user.
+
+## Intentional Strategy
+* **Incremental Construction**: Do NOT call `filter_add` with all conditions at once. Emit one or two conditions, inspect, and continue. This allows the user to correct you before you waste compute/tokens.
+* **Inspect before Execution**: Always call `filter_inspect` before executing a query to ensure you are executing the correct filter state.
+* **Compression**: Call `filter_compress` (or let auto-compression trigger) to flatten long linear version chains and keep execution performance high.
+* **Branching**: Use the version chain to branch when a user changes their mind rather than rebuilding the filter from scratch.

package/config/about/form.md

@@ -0,0 +1,9 @@
+# Form Service: Strategy & Guidelines
+
+The Form Service manages dynamically branched questionnaires and survey forms. It operates as a stateful graph engine where next questions are resolved dynamically on-the-fly based on prior answers and skip actions.
+
+## Rules of Engagement
+* **Incremental Answering**: Call `form_answer` or `form_skip` after each response. The returned `next_questions` list will tell you which question(s) are now active and need to be asked.
+* **Back-Navigation**: If the user wants to revise a previous choice, call `form_back` with the target question. The engine preserves subsequent answers where possible but flags them as `stale` if the branching path shifts.
+* **Skipping Optional Fields**: Non-required questions can be explicitly bypassed using `form_skip`.
+* **Resolve Final State**: Once the form indicates `complete: true`, call `form_resolve` to consolidate all active answers, automatically filtering out any stale answers from inactive branching paths.

package/config/about/log.md

@@ -0,0 +1,9 @@
+# Log Service: Strategy & Guidelines
+
+The Log Service is a read-only time machine over Filter and Object history. It is designed for inspection, audit, and explanation — never for mutation.
+
+## Intentional Strategy
+* **Use it to explain lineage**: When a user asks "how did this filter/object get to this state?", open a log traversal and walk back, summarizing each checkpoint's delta.
+* **Page, don't dump**: Always pass a modest `limit` to `log_open` and follow `next_page_token` with `log_next`. Large histories should be consumed incrementally to protect the context window.
+* **Pair with the owning service**: A `delta`/`rules` entry from the Log Service tells you *what changed* at a checkpoint. To see the *full* state at that point, call `filter_get_filter` / `object_get` on the returned `id`.
+* **Remember it is ephemeral**: The Log Service only sees checkpoints still present in the session store. After `filter_gc` / `object_gc`, older history is gone — traverse before pruning.

package/config/about/middleware.md

@@ -0,0 +1,18 @@
+# Middleware Architecture & Coordination
+
+This middleware exposes four stateful services designed to help you construct complex query filters, validate objects, standardise vocabulary, and track structured log events incrementally.
+
+## How the Services Relate
+
+```mermaid
+graph TD
+  A[Dictionary Service] -->|Normalize Vernacular| B[Filter / Object / Event]
+  C[Filter Service] -->|Build Structured Queries| E[Database Execution]
+  D[Object Service] -->|Validate Structured Inputs| F[Tool Arguments]
+  G[Event Service] -->|Track Mutation Log Stream| H[VCS DAG commits]
+```
+
+1. **Dictionary Service**: Enforces ontology and resolves user shorthand (e.g. "heart attack" -> "myocardial infarction") before values are injected into filters, objects, or tool functions that require domain-specific vocabulary.
+2. **Filter Service**: Allows you to incrementally construct complex query conditions without repeatedly sending the whole filter definition.
+3. **Object Service**: Validates structured object data against schemas, using templates to minimise network traffic.
+4. **Event Service**: Offers a version-controlled, append-only log array representing changes over time with branching and symmetric merging.

package/config/about/object.md

@@ -0,0 +1,8 @@
+# Object Service: Strategy & Guidelines
+
+The Object Service manages structured JSON objects (such as orders or SOAP notes) using a stateful delta model.
+
+## Rules of Engagement
+* **Template Reuse**: Always call `object_from_saved` first to see if there is an existing object template that you can reuse, saving token overhead.
+* **Property set as you go**: Set fields using `object_patch` incrementally as you learn values from the user, rather than calling it with a massive payload on turn one.
+* **Validation Check**: Always call `object_validate` to inspect errors and get a clean diff before finalizing.

package/config/about.config.json

@@ -0,0 +1,37 @@
+{
+  "about_and_examples": {
+    "middleware_about": [
+      { "_type": "file", "path": "config/about/middleware.md" }
+    ],
+    "filter_about": [
+      { "_type": "file", "path": "config/about/filter.md" }
+    ],
+    "filter_examples": [
+      { "_type": "file", "path": "config/examples/filter.md" }
+    ],
+    "object_about": [
+      { "_type": "file", "path": "config/about/object.md" }
+    ],
+    "object_examples": [
+      { "_type": "file", "path": "config/examples/object.md" }
+    ],
+    "dictionary_about": [
+      { "_type": "file", "path": "config/about/dictionary.md" }
+    ],
+    "dictionary_examples": [
+      { "_type": "file", "path": "config/examples/dictionary.md" }
+    ],
+    "event_about": [
+      { "_type": "file", "path": "config/about/event.md" }
+    ],
+    "event_examples": [
+      { "_type": "file", "path": "config/examples/event.md" }
+    ],
+    "log_about": [
+      { "_type": "file", "path": "config/about/log.md" }
+    ],
+    "log_examples": [
+      { "_type": "file", "path": "config/examples/log.md" }
+    ]
+  }
+}

package/config/examples/dictionary.md

@@ -0,0 +1,9 @@
+# Dictionary Service Worked Example
+
+```
+User: Search for patient notes mentioning a heart attack.
+LLM: Calling `dictionary_resolve(term="heart attack", namespace="CLINICAL")` -> returns "CLINICAL::MYOCARDIAL_INFARCTION"
+LLM: Calling `filter_init(table="clinical_notes", session_id="s1")` -> returns "filter_1"
+LLM: Calling `filter_add(filter_id="filter_1", condition={field: "diagnosis", operator: "equals", value: "CLINICAL::MYOCARDIAL_INFARCTION"})`
+```
+This demonstrates resolving vernacular to a formal coordinate identifier before query execution.

package/config/examples/event.md

@@ -0,0 +1,10 @@
+# Event Service Worked Example
+
+```
+User: Merge stream A and B.
+LLM: Calling `event_merge(session_id="s1", source_ids_or_aliases=["stream_a", "stream_b"], target_id_or_alias="main")` -> returns status: "conflict", merge_session_id: "merge_123"
+LLM: Calling `event_merge_inspect(merge_session_id="merge_123")` -> returns conflicting event ID: "ev_1"
+LLM: Calling `event_merge_resolve(merge_session_id="merge_123", event_id="ev_1", resolution={strategy: "accept_source", source_id: "stream_b"})` -> returns new merge session ID: "merge_124"
+LLM: Calling `event_merge_commit(merge_session_id="merge_124", session_id="s1")` -> returns committed ID "main"
+```
+This shows the step-by-step resolution of conflicts before merge completion.

package/config/examples/filter.md

@@ -0,0 +1,13 @@
+# Filter Service Worked Example
+
+Here is a worked example demonstrating incremental filter construction:
+
+```
+User: Show me active customers who spent more than $100.
+LLM: Calling `filter_init(table="customers", session_id="s1")` -> returns "filter_1"
+LLM: Calling `filter_add(filter_id="filter_1", condition={field: "status", operator: "equals", value: "active"})` -> returns "filter_2"
+LLM: Calling `filter_add(filter_id="filter_2", condition={field: "spend", operator: "greater_than", value: 100})` -> returns "filter_3"
+LLM: Calling `filter_inspect(filter_id="filter_3")` -> returns compiled filter
+LLM: Calling `filter_execute(filter_id="filter_3")` -> returns data
+```
+Notice that the LLM builds the filter incrementally, ensuring correctness at each step.

package/config/examples/form.md

@@ -0,0 +1,23 @@
+# Form Service Worked Example
+
+```
+User: Let's start the patient intake form.
+LLM: Calling `form_init(schema_name="patient_intake", session_id="s1")` -> returns "form_1" (next questions: ["q_name"])
+LLM: What is your full name?
+User: John Doe
+LLM: Calling `form_answer(form_id="form_1", question_id="q_name", value="John Doe", session_id="s1")` -> returns "form_2" (next questions: ["q_age"])
+LLM: What is your age?
+User: 30
+LLM: Calling `form_answer(form_id="form_2", question_id="q_age", value=30, session_id="s1")` -> returns "form_3" (next questions: ["q_smoke"])
+LLM: Do you currently smoke?
+User: Yes
+LLM: Calling `form_answer(form_id="form_3", question_id="q_smoke", value=true, session_id="s1")` -> returns "form_4" (next questions: ["q_smoke_frequency"])
+LLM: How many cigarettes per day do you smoke?
+User: 5
+LLM: Calling `form_answer(form_id="form_4", question_id="q_smoke_frequency", value=5, session_id="s1")` -> returns "form_5" (next questions: ["q_alcohol"])
+LLM: Describe your alcohol usage (optional).
+User: None, please skip.
+LLM: Calling `form_skip(form_id="form_5", question_id="q_alcohol", session_id="s1")` -> returns "form_6" (complete: true)
+LLM: Calling `form_resolve(form_id="form_6", session_id="s1")` -> returns fully compiled form answers
+```
+This shows linear answering, branching based on `q_smoke === true`, optional question skipping, and resolution.

package/config/examples/log.md

@@ -0,0 +1,14 @@
+# Log Service Worked Example
+
+```
+User: Walk me through how this object evolved.
+LLM: Calling `log_open(type="object", session_id="s1", id_or_alias="obj_tip", limit=3)`
+  -> returns entries: [obj_tip delta, obj_prev delta, obj_base delta], next_page_token="tok_1", has_more=true
+LLM: (summarize the three deltas to the user)
+User: And before that?
+LLM: Calling `log_next(page_token="tok_1")`
+  -> returns entries: [obj_older delta, obj_seed delta], next_page_token=null, has_more=false
+LLM: (root reached — traversal complete)
+```
+
+This demonstrates stateless, paginated traversal of an object's version chain from tip back to its seed checkpoint.

package/config/examples/object.md

@@ -0,0 +1,12 @@
+# Object Service Worked Example
+
+```
+User: Create a new prescription order for Patient 123.
+LLM: Calling `object_init(schema_name="prescription", session_id="s1")` -> returns "obj_1"
+LLM: Calling `object_patch(object_id="obj_1", patch={patient_id: "123"})` -> returns "obj_2"
+User: Set the medication to Amoxicillin and dose to 500mg.
+LLM: Calling `object_patch(object_id="obj_2", patch={medication: "Amoxicillin", dose: "500mg"})` -> returns "obj_3"
+LLM: Calling `object_validate(object_id="obj_3")` -> returns validation success
+LLM: Calling `object_resolve(object_id="obj_3")` -> returns fully compiled object
+```
+This demonstrates updating properties incrementally across dialogue turns.

package/config/storage.config.json

@@ -0,0 +1,39 @@
+{
+  "version": 1,
+  "filter_session_state": {
+    "_type": "adapter",
+    "name": "memory"
+  },
+  "filter_persistent_state": {
+    "global": {
+      "_type": "adapter",
+      "name": "memory"
+    },
+    "user": {
+      "_type": "adapter",
+      "name": "memory"
+    }
+  },
+  "object_session_state": {
+    "_type": "adapter",
+    "name": "memory"
+  },
+  "object_persistent_state": {
+    "global": {
+      "_type": "adapter",
+      "name": "memory"
+    },
+    "user": {
+      "_type": "adapter",
+      "name": "memory"
+    }
+  },
+  "dictionary_state": {
+    "_type": "adapter",
+    "name": "memory"
+  },
+  "dictionary_resolver": {
+    "_type": "adapter",
+    "name": "memory"
+  }
+}

package/config/tools.config.json

@@ -0,0 +1,14 @@
+{
+  "tools": {
+    "browse_catalog": {
+      "schema": {
+        "_type": "file",
+        "path": "./config/tools/browse_catalog.schema.json"
+      },
+      "engine": {
+        "_type": "adapter",
+        "name": "memory-engine"
+      }
+    }
+  }
+}

package/dist/adapters/engines/dataframe-bridge.py

@@ -0,0 +1,89 @@
+import sys
+import json
+import traceback
+
+try:
+    import pandas as pd
+    import duckdb
+except ImportError:
+    # Print error to stderr so the TS process knows imports failed
+    print("Error: Missing 'pandas' or 'duckdb' library.", file=sys.stderr)
+    sys.exit(1)
+
+# Keep references to loaded dataframes in local scope
+dfs = {}
+
+def handle_load(params):
+    source_file = params.get("source_file")
+    df_name = params.get("dataframe_name", "df")
+    
+    if not source_file:
+        return {"success": False, "error": "Missing 'source_file' parameter"}
+
+    try:
+        if source_file.endswith(".csv"):
+            df = pd.read_csv(source_file)
+        elif source_file.endswith(".parquet"):
+            df = pd.read_parquet(source_file)
+        elif source_file.endswith(".json") or source_file.endswith(".jsonl"):
+            df = pd.read_json(source_file, lines=source_file.endswith(".jsonl"))
+        else:
+            return {"success": False, "error": f"Unsupported file format: {source_file}"}
+            
+        dfs[df_name] = df
+        # Register the DataFrame as a view in DuckDB
+        duckdb.register(df_name, df)
+        return {"success": True, "rows": len(df)}
+    except Exception as e:
+        return {"success": False, "error": str(e), "traceback": traceback.format_exc()}
+
+def handle_query(params):
+    sql = params.get("sql")
+    params_list = params.get("params", [])
+    if not sql:
+        return {"success": False, "error": "Missing 'sql' parameter"}
+        
+    try:
+        # Run DuckDB SQL query directly over the registered Pandas DataFrames
+        res = duckdb.execute(sql, params_list).df()
+        # Convert to records JSON representation
+        records = res.to_dict(orient="records")
+        # Handle nan/inf float serialization issues by converting to None
+        cleaned = json.loads(json.dumps(records, default=str))
+        return {"success": True, "data": cleaned}
+    except Exception as e:
+        return {"success": False, "error": str(e), "traceback": traceback.format_exc()}
+
+def main():
+    # Loop reading JSON commands from stdin
+    while True:
+        line = sys.stdin.readline()
+        if not line:
+            break
+        if not line.strip():
+            continue
+        try:
+            cmd = json.loads(line)
+            action = cmd.get("action")
+            
+            if action == "load":
+                res = handle_load(cmd)
+            elif action == "query":
+                res = handle_query(cmd)
+            elif action == "ping":
+                res = {"success": True, "pong": True}
+            else:
+                res = {"success": False, "error": f"Unknown action: {action}"}
+                
+            if "id" in cmd:
+                res["id"] = cmd["id"]
+                
+            print(json.dumps(res))
+            sys.stdout.flush()
+        except Exception as e:
+            err_res = {"success": False, "error": f"Malformed input: {str(e)}"}
+            print(json.dumps(err_res))
+            sys.stdout.flush()
+
+if __name__ == "__main__":
+    main()

package/dist/index.d.ts

@@ -0,0 +1,3 @@
+#!/usr/bin/env bun
+export {};
+//# sourceMappingURL=index.d.ts.map

package/dist/index.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../index.ts"],"names":[],"mappings":""}