@ema.co/mcp-toolkit 2026.1.27 → 2026.1.28-2

package/docs/validation-contract.md

@@ -0,0 +1,397 @@
+# Validation Contract - Go to TypeScript Compatibility
+
+**Date**: 2026-01-27  
+**Purpose**: Define exact semantics for TypeScript validator to match Go validator behavior
+
+---
+
+## Core Validation Algorithm
+
+### Overview
+
+The Go validator performs **static validation** by:
+1. Enumerating all possible execution paths in a workflow
+2. Validating each path for named results compliance
+3. Reporting errors for paths that fail validation
+
+### Key Principle
+
+**Static validation only applies to workflows with named results.**
+- If `workflow.Proto.NamedResults` is empty, validation is skipped
+- This is intentional - workflows without named results don't need path validation
+
+---
+
+## Path Enumeration Algorithm
+
+### Step 1: Create Initial Session
+
+```go
+initialSession, err := CreateValidationWorkflowSession(ctx, workflow, logger)
+```
+
+Creates a validation session with:
+- Dummy workflow inputs (type-aware)
+- Dummy widget configs (type-aware)
+- Empty action states
+- Static validation strategy
+
+### Step 2: Process Session Queue
+
+Main loop:
+```go
+sessionsToProcess := []ValidationWorkflowSession{initialSession}
+for len(sessionsToProcess) > 0 {
+    session := sessionsToProcess[0]
+    sessionsToProcess = sessionsToProcess[1:]
+    
+    result, err := session.RunValidation(ctx)
+    // Handle result...
+}
+```
+
+### Step 3: Handle Branching
+
+When `result.Outcome == OutcomeBranch`:
+
+1. **Identify Branching Action**: Action with enumerable output (enum or boolean)
+2. **Get Possible Values**: All enum options or [true, false] for boolean
+3. **Fork Session**: Create one session per possible value
+4. **Plug Dummy Values**: 
+   - For branching output: Use chosen value
+   - For other outputs: Generate type-aware dummy values
+5. **Mark Action Complete**: Mark branching action as completed in forked session
+6. **Queue New Sessions**: Add all forked sessions to queue
+
+**Branching Actions**:
+- Categorizers (enum output: category)
+- Boolean outputs (true/false)
+- Any action with enumerable output
+
+**Non-Branching Actions**:
+- Generate type-aware dummy outputs for all outputs
+- Mark action as completed
+- Continue traversal
+
+### Step 4: Handle Completion
+
+When `result.Outcome == OutcomeCompleted`:
+
+1. **Track Completed Path**:
+   - Completed actions (topological order)
+   - Final outputs produced
+   - Path identifier (branching action + value, or last action)
+
+2. **Validate Named Results**:
+   - Check if all named results are produced
+   - Check for multiple writers
+   - Check for response/abstain
+   - Apply custom validators (persona-type specific)
+
+3. **Store Path Info**: Add to `completedSessionMap`
+
+### Step 5: Cleanup
+
+Remove incomplete sessions (those that only branched but never completed).
+
+---
+
+## Validation Error Types
+
+### 1. RESULT_ERROR_TYPE_REQUIRED_OUTPUT_NOT_PRODUCED
+
+**Definition**: A named result is not produced on at least one execution path.
+
+**Detection**:
+- For each named result in `workflow.Proto.NamedResults`
+- For each completed path in `completedSessionMap`
+- Check if named result is in `FinalOutputs` of that path
+- If missing on any path → error
+
+**Path Identification**:
+- If path branched: `{actionName: branching_action, outputName: category, outputValue: chosen_value}`
+- If path didn't branch: `{actionName: last_completed_action}`
+
+**Example**:
+```
+Named result: "response_with_sources"
+Path "Billing" (from categorizer, value="Billing") does not produce it
+→ Error: RESULT_ERROR_TYPE_REQUIRED_OUTPUT_NOT_PRODUCED
+```
+
+### 2. RESULT_ERROR_TYPE_MULTIPLE_WRITERS
+
+**Definition**: Multiple actions produce the same named result on the same execution path.
+
+**Detection**:
+- For each named result
+- For each completed path
+- Count how many actions produce that result
+- If count > 1 on any path → error
+
+**Note**: This may be intentional (aggregation), but Go validator flags it as error.
+
+**Path Identification**: Same as above.
+
+**Example**:
+```
+Named result: "response_with_sources"
+Path "Billing" has both "respond_1" and "respond_2" producing it
+→ Error: RESULT_ERROR_TYPE_MULTIPLE_WRITERS
+```
+
+### 3. RESULT_ERROR_TYPE_MISSING_RESPONSE_OR_ABSTAIN_REASON
+
+**Definition**: An execution path does not produce a response and does not have an abstain reason.
+
+**Detection**:
+- For each completed path
+- Check if path produces any response output:
+  - `respond_with_sources.response_with_sources`
+  - `call_llm.llm_output`
+  - `fixed_response.response`
+  - Other response-like outputs
+- If no response AND no abstain reason → error
+
+**Abstain Reason**: TBD - need to check how this is defined in Go code.
+
+**Path Identification**: Same as above.
+
+**Example**:
+```
+Path "Billing" ends at "search" node
+No response node connected
+No abstain reason specified
+→ Error: RESULT_ERROR_TYPE_MISSING_RESPONSE_OR_ABSTAIN_REASON
+```
+
+### 4. RESULT_ERROR_TYPE_CATEGORY_HIERARCHY_VIOLATION
+
+**Definition**: Categorizer category structure violates hierarchy rules.
+
+**Detection**: TBD - need to analyze `named_result_agent_assist_validator.go`
+
+**Path Identification**: Same as above.
+
+**Example**: TBD
+
+---
+
+## Path Identification Rules
+
+### Rule 1: Branching Path Identification
+
+If path was created by branching action:
+```
+{
+  actionName: "<branching_action_name>",
+  outputName: "<branching_output_name>",
+  outputValue: "<chosen_value>"
+}
+```
+
+**Example**:
+```
+{
+  actionName: "chat_categorizer",
+  outputName: "category",
+  outputValue: "Billing"
+}
+```
+
+### Rule 2: Non-Branching Path Identification
+
+If path had no branching action:
+```
+{
+  actionName: "<last_completed_action_name>"
+}
+```
+
+**Example**:
+```
+{
+  actionName: "respond_with_sources"
+}
+```
+
+### Rule 3: Empty Path
+
+If no actions were executed (shouldn't happen, but handle gracefully):
+```
+{
+  actionName: ""
+}
+```
+
+---
+
+## Dummy Value Generation
+
+### For Workflow Inputs
+
+- Generate type-aware dummy values based on input type
+- Use `util.CreateTypeAwareDummyValue(type, enumTypes)`
+- Validate type compatibility
+
+### For Widget Configs
+
+- Generate type-aware dummy values based on widget type
+- Use same utility function
+- Validate type compatibility
+
+### For Action Outputs (Non-Branching)
+
+- Generate type-aware dummy values for all outputs
+- Use `getDummyValueForActionOutput(outputDef, enumTypes)`
+- Based on output type definition
+
+### For Action Outputs (Branching)
+
+- For branching output: Use chosen value (enum option or boolean)
+- For other outputs: Generate type-aware dummy values
+
+---
+
+## Custom Validators
+
+### Interface
+
+```go
+type CustomNamedResultValidator interface {
+    Validate(workflow *WorkflowGraph, pathInfo *completedPathInfo) error
+}
+```
+
+### Behavior
+
+- Called after general named result validations pass
+- Only called if `!pathInfo.NamedResultsValidationsFailed`
+- Can add persona-type specific rules
+- Example: `named_result_agent_assist_validator.go` for Agent Assist personas
+
+### Implementation Note
+
+- TypeScript validator should support plugin system for custom validators
+- For now, focus on the 4 core error types
+- Custom validators can be added later
+
+---
+
+## Error Message Format (Go)
+
+### Structure
+
+```go
+type WorkflowError_NamedResultError struct {
+    ErrorType pb.WorkflowError_ResultErrorType
+    NamedResultName string
+    PathIdentifier *WorkflowError_ActionOutput
+    Message string
+}
+```
+
+### Path Identifier
+
+```go
+type WorkflowError_ActionOutput struct {
+    ActionName string
+    OutputName string  // Optional, only for branching
+    OutputValue string  // Optional, only for branching
+}
+```
+
+### Message Format
+
+TBD - need to analyze actual error messages from Go code.
+
+---
+
+## TypeScript Compatibility Requirements
+
+### Must Match
+
+1. **Error Types**: All 4 error types must be detected
+2. **Path Enumeration**: Same algorithm (topological sort + DFS + forking)
+3. **Path Identification**: Same format (actionName, outputName, outputValue)
+4. **Validation Logic**: Same checks (required outputs, multiple writers, etc.)
+
+### Can Differ
+
+1. **Error Message Format**: Can be enhanced (what/why/how)
+2. **Performance**: Can optimize (caching, early pruning)
+3. **Additional Validations**: Can add more (but document as extensions)
+
+---
+
+## Test Cases
+
+### Test Case 1: Simple Linear Workflow
+
+```
+trigger → action1 → action2 → respond → WORKFLOW_OUTPUT
+Named result: "response"
+
+Expected: ✅ Valid (all paths produce response)
+```
+
+### Test Case 2: Categorizer with Missing Response
+
+```
+trigger → categorizer → [Billing branch] → search → (no response)
+Named result: "response"
+
+Expected: ❌ Error: RESULT_ERROR_TYPE_REQUIRED_OUTPUT_NOT_PRODUCED
+Path: {actionName: "categorizer", outputName: "category", outputValue: "Billing"}
+```
+
+### Test Case 3: Multiple Writers
+
+```
+trigger → categorizer → [Billing] → respond1 → WORKFLOW_OUTPUT
+                      → [Billing] → respond2 → WORKFLOW_OUTPUT
+Named result: "response" (both respond1 and respond2 produce it)
+
+Expected: ❌ Error: RESULT_ERROR_TYPE_MULTIPLE_WRITERS
+Path: {actionName: "categorizer", outputName: "category", outputValue: "Billing"}
+```
+
+### Test Case 4: Missing Response/Abstain
+
+```
+trigger → categorizer → [Billing] → search → (ends, no response, no abstain)
+Named result: "response"
+
+Expected: ❌ Error: RESULT_ERROR_TYPE_MISSING_RESPONSE_OR_ABSTAIN_REASON
+Path: {actionName: "categorizer", outputName: "category", outputValue: "Billing"}
+```
+
+---
+
+## Performance Considerations
+
+### Timeout Limits
+
+- Target: <100ms for typical workflows
+- Max paths: Prevent exponential explosion (suggest 1000 paths max)
+- Early pruning: Stop if path is obviously invalid
+
+### Optimization Opportunities
+
+1. **Path Pruning**: If path already missing required output, don't continue
+2. **Caching**: Cache validation results for unchanged workflows
+3. **Incremental**: Only validate changed paths (future enhancement)
+
+---
+
+## Open Questions
+
+1. **Abstain Reason**: How is abstain reason defined/stored?
+2. **Category Hierarchy**: What are the exact hierarchy rules?
+3. **Custom Validators**: What persona types have custom validators?
+4. **Error Messages**: What's the exact Go error message format?
+
+---
+
+**Status**: ✅ Contract defined  
+**Next**: Analyze Go code for missing details, then implement TypeScript validator

package/docs/validation-error-format.md

@@ -0,0 +1,326 @@
+# Validation Error Format Specification
+
+**Date**: 2026-01-27  
+**Purpose**: Define enhanced error message format for TypeScript validator
+
+---
+
+## Design Principles
+
+1. **Actionable**: Users must know what to fix
+2. **Clear**: Explain what's wrong and why
+3. **Specific**: Point to exact location (node, edge, path)
+4. **Helpful**: Provide remediation steps
+5. **Referenceable**: Link to rule documentation
+
+---
+
+## Error Response Structure
+
+### TypeScript Interface
+
+```typescript
+export interface ValidationError {
+  // Identification
+  type: ValidationErrorType;
+  rule_id: string;
+  severity: "critical" | "warning" | "info";
+  
+  // Location
+  location: {
+    node_id?: string;
+    edge_id?: string;
+    path?: string[];  // Execution path: ["trigger", "categorizer", "billing_branch", "search"]
+    category?: string; // Category name if categorizer-related
+    named_result?: string; // Named result name if applicable
+  };
+  
+  // Explanation (Enhanced)
+  what: string;      // What is wrong (specific)
+  why: string;       // Why it's wrong (explanation)
+  how_to_fix: string; // How to fix (actionable steps)
+  
+  // Reference
+  rule_reference: string; // URI to rule: "ema://validation/rules#required_output_all_paths"
+  
+  // Examples (Optional)
+  examples?: {
+    bad: string;     // Bad pattern example
+    good: string;    // Good pattern example
+  };
+}
+
+export interface ValidationResult {
+  valid: boolean;
+  errors: ValidationError[];
+  warnings: ValidationError[];
+  info: ValidationError[];
+  summary: {
+    total_paths: number;
+    valid_paths: number;
+    invalid_paths: number;
+    errors_by_type: Record<ValidationErrorType, number>;
+  };
+}
+
+export type ValidationErrorType =
+  | "required_output_not_produced"
+  | "multiple_writers"
+  | "missing_response_or_abstain_reason"
+  | "category_hierarchy_violation";
+```
+
+---
+
+## Error Type Specifications
+
+### 1. required_output_not_produced
+
+**Rule ID**: `required_output_all_paths`
+
+**What**:
+```
+"Path '{path_name}' does not produce required output '{named_result}'"
+```
+
+**Why**:
+```
+"All execution paths from trigger must produce all named results. The path '{path_name}' ends at '{last_node}' without producing '{named_result}'."
+```
+
+**How to Fix**:
+```
+"Add a node that produces '{named_result}' to the '{path_name}' path. Connect the node's '{output_name}' output to WORKFLOW_OUTPUT via resultMappings."
+```
+
+**Example**:
+```json
+{
+  "type": "required_output_not_produced",
+  "rule_id": "required_output_all_paths",
+  "severity": "critical",
+  "location": {
+    "path": ["trigger", "categorizer", "billing_branch", "search"],
+    "category": "Billing",
+    "named_result": "response_with_sources"
+  },
+  "what": "Path 'Billing' does not produce required output 'response_with_sources'",
+  "why": "All execution paths from trigger must produce all named results. The path 'Billing' ends at 'search' node without producing 'response_with_sources'.",
+  "how_to_fix": "Add a response node (respond_with_sources, call_llm, or fixed_response) after the 'search' node in the 'Billing' branch. Connect the response node's output to WORKFLOW_OUTPUT via resultMappings.",
+  "rule_reference": "ema://validation/rules#required_output_all_paths",
+  "examples": {
+    "bad": "trigger → categorizer → [Billing] → search → (no response)",
+    "good": "trigger → categorizer → [Billing] → search → respond_with_sources → WORKFLOW_OUTPUT"
+  }
+}
+```
+
+---
+
+### 2. multiple_writers
+
+**Rule ID**: `single_writer_per_output`
+
+**What**:
+```
+"Multiple nodes produce '{named_result}' on path '{path_name}'"
+```
+
+**Why**:
+```
+"A single named result can only have one producer per execution path. The path '{path_name}' has both '{node1}' and '{node2}' producing '{named_result}'."
+```
+
+**How to Fix**:
+```
+"Ensure only one node produces '{named_result}' on the '{path_name}' path. Either remove one of the producers or use different named results for each node."
+```
+
+**Example**:
+```json
+{
+  "type": "multiple_writers",
+  "rule_id": "single_writer_per_output",
+  "severity": "critical",
+  "location": {
+    "path": ["trigger", "categorizer", "billing_branch"],
+    "category": "Billing",
+    "named_result": "response_with_sources"
+  },
+  "what": "Multiple nodes produce 'response_with_sources' on path 'Billing'",
+  "why": "A single named result can only have one producer per execution path. The path 'Billing' has both 'respond_billing' and 'respond_general' producing 'response_with_sources'.",
+  "how_to_fix": "Ensure only one node produces 'response_with_sources' on the 'Billing' path. Either remove 'respond_general' or use a different named result for it.",
+  "rule_reference": "ema://validation/rules#single_writer_per_output"
+}
+```
+
+---
+
+### 3. missing_response_or_abstain_reason
+
+**Rule ID**: `response_or_abstain_required`
+
+**What**:
+```
+"Path '{path_name}' does not produce a response and has no abstain reason"
+```
+
+**Why**:
+```
+"Every execution path must either produce a user-visible response or explicitly abstain with a reason. The path '{path_name}' ends without a response node and no abstain reason is specified."
+```
+
+**How to Fix**:
+```
+"Either add a response node (respond_with_sources, call_llm, or fixed_response) to the '{path_name}' path, or add an abstain reason explaining why this path doesn't respond."
+```
+
+**Example**:
+```json
+{
+  "type": "missing_response_or_abstain_reason",
+  "rule_id": "response_or_abstain_required",
+  "severity": "critical",
+  "location": {
+    "path": ["trigger", "categorizer", "billing_branch", "search"],
+    "category": "Billing"
+  },
+  "what": "Path 'Billing' does not produce a response and has no abstain reason",
+  "why": "Every execution path must either produce a user-visible response or explicitly abstain with a reason. The path 'Billing' ends at 'search' node without a response node and no abstain reason is specified.",
+  "how_to_fix": "Either add a response node (respond_with_sources, call_llm, or fixed_response) after 'search' in the 'Billing' branch, or add an abstain reason to the workflow configuration.",
+  "rule_reference": "ema://validation/rules#response_or_abstain_required"
+}
+```
+
+---
+
+### 4. category_hierarchy_violation
+
+**Rule ID**: `category_hierarchy_valid`
+
+**What**:
+```
+"Category hierarchy violation in categorizer '{categorizer_name}'"
+```
+
+**Why**:
+```
+TBD - Need to analyze category hierarchy rules from Go validator
+```
+
+**How to Fix**:
+```
+TBD - Need to analyze category hierarchy rules
+```
+
+**Example**:
+```json
+{
+  "type": "category_hierarchy_violation",
+  "rule_id": "category_hierarchy_valid",
+  "severity": "critical",
+  "location": {
+    "node_id": "chat_categorizer",
+    "category": "Billing"
+  },
+  "what": "Category hierarchy violation in categorizer 'chat_categorizer'",
+  "why": "TBD",
+  "how_to_fix": "TBD",
+  "rule_reference": "ema://validation/rules#category_hierarchy_valid"
+}
+```
+
+---
+
+## Path Format
+
+### Path Representation
+
+Paths are represented as arrays of node IDs in execution order:
+
+```typescript
+path: ["trigger", "categorizer", "billing_branch", "search", "respond_with_sources"]
+```
+
+### Path Naming
+
+For branching paths, use category/value name:
+```
+"Billing"  // From categorizer, value="Billing"
+"Fallback" // From categorizer, value="Fallback"
+"true"     // From boolean output, value=true
+```
+
+For non-branching paths, use last action name:
+```
+"respond_with_sources"  // Last action in path
+```
+
+---
+
+## Rule Reference Format
+
+### URI Structure
+
+```
+ema://validation/rules#{rule_id}
+```
+
+### Examples
+
+- `ema://validation/rules#required_output_all_paths`
+- `ema://validation/rules#single_writer_per_output`
+- `ema://validation/rules#response_or_abstain_required`
+- `ema://validation/rules#category_hierarchy_valid`
+
+---
+
+## Error Message Templates
+
+### Template Variables
+
+- `{path_name}` - Human-readable path name
+- `{named_result}` - Named result name
+- `{node_id}` - Node identifier
+- `{last_node}` - Last node in path
+- `{output_name}` - Output name
+- `{category}` - Category name
+
+### Template Functions
+
+- `formatPath(path: string[]): string` - Format path array to readable string
+- `formatNode(nodeId: string): string` - Format node ID to display name
+- `formatCategory(category: string): string` - Format category name
+
+---
+
+## Compatibility with Go Validator
+
+### Must Match
+
+- Error types (4 types)
+- Path identification format
+- Validation logic
+
+### Can Enhance
+
+- Error message format (what/why/how)
+- Additional context (examples, references)
+- Severity levels (critical/warning/info)
+
+---
+
+## Implementation Checklist
+
+- [ ] Define TypeScript interfaces
+- [ ] Create error message generators for each error type
+- [ ] Implement path formatting utilities
+- [ ] Add rule reference generation
+- [ ] Create example generators (good vs bad)
+- [ ] Test error message clarity
+- [ ] Validate against Go validator error format
+
+---
+
+**Status**: ✅ Format designed  
+**Next**: Implement error message generators