@ema.co/mcp-toolkit 2026.2.5 → 2026.2.13

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Potentially problematic release.


This version of @ema.co/mcp-toolkit might be problematic. Click here for more details.

Files changed (44) hide show
  1. package/.context/public/guides/dashboard-operations.md +63 -0
  2. package/.context/public/guides/workflow-builder-patterns.md +708 -0
  3. package/LICENSE +29 -21
  4. package/README.md +58 -35
  5. package/dist/mcp/domain/proto-constraints.js +284 -0
  6. package/dist/mcp/domain/structural-rules.js +8 -0
  7. package/dist/mcp/domain/validation-rules.js +102 -15
  8. package/dist/mcp/domain/workflow-graph-optimizer.js +235 -0
  9. package/dist/mcp/domain/workflow-graph-transforms.js +808 -0
  10. package/dist/mcp/domain/workflow-graph.js +376 -0
  11. package/dist/mcp/domain/workflow-optimizer.js +10 -4
  12. package/dist/mcp/guidance.js +45 -2
  13. package/dist/mcp/handlers/feedback/index.js +139 -0
  14. package/dist/mcp/handlers/feedback/store.js +262 -0
  15. package/dist/mcp/handlers/workflow/index.js +12 -11
  16. package/dist/mcp/handlers/workflow/optimize.js +73 -33
  17. package/dist/mcp/knowledge.js +87 -36
  18. package/dist/mcp/resources.js +393 -17
  19. package/dist/mcp/server.js +38 -4
  20. package/dist/mcp/tools.js +89 -2
  21. package/dist/sdk/generated/deprecated-actions.js +182 -96
  22. package/dist/sdk/generated/proto-fields.js +2 -1
  23. package/dist/sdk/generated/protos/service/agent_qa/v1/agent_qa_pb.js +460 -21
  24. package/dist/sdk/generated/protos/service/auth/v1/auth_pb.js +11 -1
  25. package/dist/sdk/generated/protos/service/dataingest/v1/dataingest_pb.js +173 -66
  26. package/dist/sdk/generated/protos/service/feedback/v1/feedback_pb.js +43 -1
  27. package/dist/sdk/generated/protos/service/llmservice/v1/llmservice_pb.js +26 -21
  28. package/dist/sdk/generated/protos/service/persona/v1/persona_config_pb.js +100 -89
  29. package/dist/sdk/generated/protos/service/persona/v1/persona_pb.js +126 -116
  30. package/dist/sdk/generated/protos/service/persona/v1/shared_widgets/widget_types_pb.js +33 -1
  31. package/dist/sdk/generated/protos/service/persona/v1/voicebot_widgets/widget_types_pb.js +60 -11
  32. package/dist/sdk/generated/protos/service/tenant/v1/tenant_pb.js +1 -1
  33. package/dist/sdk/generated/protos/service/user/v1/user_pb.js +1 -1
  34. package/dist/sdk/generated/protos/service/utils/v1/agent_qa_pb.js +35 -0
  35. package/dist/sdk/generated/protos/service/workflows/v1/action_registry_pb.js +1 -1
  36. package/dist/sdk/generated/protos/service/workflows/v1/action_type_pb.js +6 -1
  37. package/dist/sdk/generated/protos/service/workflows/v1/chatbot_pb.js +106 -11
  38. package/dist/sdk/generated/protos/service/workflows/v1/common_forms_pb.js +1 -1
  39. package/dist/sdk/generated/protos/service/workflows/v1/coordinator_pb.js +1 -1
  40. package/dist/sdk/generated/protos/service/workflows/v1/external_actions_pb.js +31 -1
  41. package/dist/sdk/generated/protos/service/workflows/v1/well_known_pb.js +5 -1
  42. package/dist/sdk/generated/protos/service/workflows/v1/workflow_pb.js +1 -1
  43. package/dist/sdk/generated/protos/util/tracking_metadata_pb.js +1 -1
  44. package/package.json +2 -2
@@ -0,0 +1,708 @@
1
+ ---
2
+ title: "Workflow Builder Patterns & Node Reference"
3
+ date: 2026-02-05
4
+ audience: MCP users, workflow builders, auto-builder agents
5
+ ---
6
+
7
+ # Workflow Builder Patterns & Node Reference
8
+
9
+ Practical reference for building Ema workflows. Covers input/output semantics for each node type,
10
+ data types flowing between nodes, entity extraction best practices, and common wiring patterns.
11
+
12
+ Extracted from real auto-builder sessions and validated against platform behavior.
13
+
14
+ ---
15
+
16
+ ## Data Types Between Nodes
17
+
18
+ Four core data types flow between workflow nodes:
19
+
20
+ | Type | Code Constant | Description | Example |
21
+ | ------------------ | ----------------------------------- | ------------------------------------------------- | ---------------------------------------------- |
22
+ | **Plain Text** | `WELL_KNOWN_TYPE_TEXT_WITH_SOURCES` | Text with optional citation metadata | User's message, LLM response, formatted output |
23
+ | **Conversation** | `WELL_KNOWN_TYPE_CHAT_CONVERSATION` | Structured message history (role + content pairs) | Full chat thread for context |
24
+ | **Search Results** | `WELL_KNOWN_TYPE_SEARCH_RESULT` | Retrieved document chunks with citations | KB search results with source metadata |
25
+ | **Enum** | `WELL_KNOWN_TYPE_ENUM` | Category/classification signal for routing | `category::Schedule Appointment` |
26
+ | **Document** | `WELL_KNOWN_TYPE_DOCUMENT` | Uploaded file content | PDF, DOCX for extraction |
27
+ | **Any** | `WELL_KNOWN_TYPE_ANY` | Untyped — needs intermediary for type-safe wiring | entity_extraction output |
28
+
29
+ **Critical rule**: Types are NOT interchangeable. `CHAT_CONVERSATION` into a `TEXT_WITH_SOURCES` input
30
+ causes type mismatch errors. Use converter nodes (`conversation_to_search_query`) when needed.
31
+
32
+ ---
33
+
34
+ ## Input Semantics by Context
35
+
36
+ The same input name means different things depending on the node:
37
+
38
+ | Input Name | In Search Nodes | In Respond Nodes | In Extract Nodes | In Categorizers |
39
+ | ---------------- | ---------------------- | ------------------------- | ---------------------- | ------------------------------- |
40
+ | `query` | Search term to look up | User's question to answer | Source text to analyze | N/A |
41
+ | `conversation` | N/A | N/A (use named_inputs) | N/A | Full history for classification |
42
+ | `trigger_when` | N/A | "Should I run?" | "Should I run?" | N/A |
43
+ | `named_inputs_*` | N/A | Additional context | Additional context | N/A |
44
+
45
+ **Universal mental model:**
46
+
47
+ - `query` = "What should I process?"
48
+ - `conversation` = "What's the full context?"
49
+ - `trigger_when` = "Should I run at all?"
50
+ - `named_inputs_*` = "Extra context" (search results, current message, etc.)
51
+
52
+ ---
53
+
54
+ ## Node Type Reference
55
+
56
+ ### Chat Trigger (`chat_trigger`)
57
+
58
+ | | |
59
+ | -------------- | ------------------------------------------------------------------------- |
60
+ | **Purpose** | Entry point for chat workflows |
61
+ | **Inputs** | None (system event) |
62
+ | **Outputs** | `user_query` (TEXT_WITH_SOURCES), `chat_conversation` (CHAT_CONVERSATION) |
63
+ | **Pairs with** | Intent routers, search nodes, LLM nodes |
64
+
65
+ ### Intent Router (`chat_categorizer`)
66
+
67
+ | | |
68
+ | -------------- | ------------------------------------------------------------------------- |
69
+ | **Purpose** | Classify conversation into intent categories for routing |
70
+ | **Inputs** | `conversation` (CHAT_CONVERSATION) — must be full history, NOT user_query |
71
+ | **Outputs** | `category` (ENUM) — one per configured category |
72
+ | **Pairs with** | Multiple gated respond/action nodes, fallback |
73
+ | **Critical** | Always include Fallback category. Every category needs a handler. |
74
+
75
+ ### Knowledge Search (`search/v2`)
76
+
77
+ | | |
78
+ | -------------- | ----------------------------------------------------------------------------- |
79
+ | **Purpose** | Retrieve relevant documents from uploaded knowledge base |
80
+ | **Inputs** | `query` (TEXT_WITH_SOURCES), `datastore_configs` (from widget) |
81
+ | **Outputs** | `search_results` (SEARCH_RESULT) |
82
+ | **Pairs with** | Respond nodes (for grounded answers), extract nodes (for grounded extraction) |
83
+ | **Critical** | NOT an LLM node — do NOT include `model_config`. Data must be uploaded first. |
84
+
85
+ ### Respond (call_llm, respond_with_sources)
86
+
87
+ | | |
88
+ | -------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- |
89
+ | **Purpose** | Generate natural language response using LLM |
90
+ | **Inputs** | `query` (TEXT_WITH_SOURCES), `named_inputs_Conversation` (CHAT_CONVERSATION), `named_inputs_Search_Results` (SEARCH_RESULT), `trigger_when` (ENUM) |
91
+ | **Outputs** | `response_with_sources` (TEXT_WITH_SOURCES) |
92
+ | **Pairs with** | Search nodes (for RAG), categorizers (for gating), WORKFLOW_OUTPUT |
93
+ | **Critical** | Must wire to WORKFLOW_OUTPUT or response is silently lost |
94
+
95
+ ### Extract (call_llm configured for extraction)
96
+
97
+ | | |
98
+ | -------------- | ------------------------------------------------------------------------------------------------------ |
99
+ | **Purpose** | Extract specific entities (emails, dates, IDs) from text |
100
+ | **Inputs** | `query` (TEXT_WITH_SOURCES), `named_inputs_Current_Message` (TEXT_WITH_SOURCES), `trigger_when` (ENUM) |
101
+ | **Outputs** | Extracted data (TEXT_WITH_SOURCES) |
102
+ | **Pairs with** | External actions, json_mapper, send_email |
103
+ | **Critical** | Scope inputs tightly — use only `user_query` unless entity spans multiple turns |
104
+
105
+ ### Fixed Response (`fixed_response`)
106
+
107
+ | | |
108
+ | -------------- | --------------------------------------------------------------------------- |
109
+ | **Purpose** | Return a static predefined message (fallback, compliance notice) |
110
+ | **Inputs** | `trigger_when` (ENUM), `named_inputs_*` for template variables |
111
+ | **Outputs** | `fixed_response_with_sources` (TEXT_WITH_SOURCES) |
112
+ | **Pairs with** | Categorizers (as fallback), type conversion (template with `{{variables}}`) |
113
+
114
+ ### External Action Caller (`external_action_caller`)
115
+
116
+ | | |
117
+ | -------------- | ----------------------------------------------------------------------------------- |
118
+ | **Purpose** | Call external APIs/tools (ServiceNow, Salesforce, calendars) |
119
+ | **Inputs** | `query` (TEXT_WITH_SOURCES), `conversation` (CHAT_CONVERSATION), tool configuration |
120
+ | **Outputs** | `tool_execution_result` (TEXT_WITH_SOURCES) |
121
+ | **Pairs with** | Entity extractors (for parameters), respond nodes (for result formatting) |
122
+
123
+ ### Document Trigger (`document_trigger`)
124
+
125
+ | | |
126
+ | -------------- | ----------------------------------------------------------------------------- |
127
+ | **Purpose** | Entry point for dashboard workflows (file upload per row) |
128
+ | **Inputs** | None (system event — triggered when a row is created/file uploaded) |
129
+ | **Outputs** | `document_content` (DOCUMENT — uploaded file), `row_data` (ANY — column values) |
130
+ | **Pairs with** | entity_extraction_with_documents, call_llm, search |
131
+ | **Critical** | Dashboard personas only. Each row triggers one workflow execution. |
132
+
133
+ ### Send Email Agent (`send_email_agent`)
134
+
135
+ | | |
136
+ | -------------- | -------------------------------------------------------------------------------------- |
137
+ | **Purpose** | Send email with specified recipient, subject, and body |
138
+ | **Inputs** | `email_to` (TEXT_WITH_SOURCES), `email_subject` (TEXT_WITH_SOURCES), `email_body` (TEXT_WITH_SOURCES) |
139
+ | **Outputs** | `confirmation` (TEXT_WITH_SOURCES — send result) |
140
+ | **Pairs with** | fixed_response (for type conversion), json_mapper (field extraction) |
141
+ | **Critical** | Inputs must be TEXT_WITH_SOURCES — use intermediary chain from ANY-typed sources. Enable HITL via `disable_human_interaction: false` for approval. |
142
+
143
+ ### Live Web Search (`live_web_search`)
144
+
145
+ | | |
146
+ | -------------- | ----------------------------------------------------------------------------- |
147
+ | **Purpose** | Real-time web search for current information |
148
+ | **Inputs** | `query` (TEXT_WITH_SOURCES) |
149
+ | **Outputs** | `web_search_results` (SEARCH_RESULT) |
150
+ | **Pairs with** | combine_search_results (to merge with KB search), respond nodes |
151
+ | **Critical** | No data upload needed (searches the web). Use for current events, external info not in KB. |
152
+
153
+ ### Combine Search Results (`combine_search_results`)
154
+
155
+ | | |
156
+ | -------------- | ----------------------------------------------------------------------------- |
157
+ | **Purpose** | Merge results from multiple search sources with deduplication |
158
+ | **Inputs** | `search_results_1` (SEARCH_RESULT), `search_results_2` (SEARCH_RESULT) |
159
+ | **Outputs** | `combined_results` (SEARCH_RESULT) |
160
+ | **Pairs with** | search/v2 + live_web_search, or any two search sources |
161
+
162
+ ### Conversation to Search Query (`conversation_to_search_query`)
163
+
164
+ | | |
165
+ | -------------- | ----------------------------------------------------------------------------- |
166
+ | **Purpose** | Convert multi-turn conversation to a search-optimized query |
167
+ | **Inputs** | `conversation` (CHAT_CONVERSATION) |
168
+ | **Outputs** | `summarized_conversation` (TEXT_WITH_SOURCES) |
169
+ | **Pairs with** | search/v2 (provides search query from conversation context) |
170
+ | **Critical** | Required for multi-turn chat search. Direct CHAT_CONVERSATION → search causes type mismatch. |
171
+
172
+ ### Response Validator (`response_validator`)
173
+
174
+ | | |
175
+ | -------------- | ----------------------------------------------------------------------------- |
176
+ | **Purpose** | Validate LLM output against quality/compliance criteria |
177
+ | **Inputs** | `reference_query` (TEXT_WITH_SOURCES), `response_to_validate` (TEXT_WITH_SOURCES) |
178
+ | **Outputs** | `abstain_reason` (TEXT_WITH_SOURCES — reason for rejection, empty if valid) |
179
+ | **Pairs with** | call_llm (validates output), abstain_action (handles rejection) |
180
+
181
+ ### Abstain Action (`abstain_action`)
182
+
183
+ | | |
184
+ | -------------- | ----------------------------------------------------------------------------- |
185
+ | **Purpose** | Provide a safe decline response when AI should not answer |
186
+ | **Inputs** | `abstain_reason` (TEXT_WITH_SOURCES — from response_validator) |
187
+ | **Outputs** | `abstain_reason` (TEXT_WITH_SOURCES — decline message) |
188
+ | **Pairs with** | response_validator (receives rejection reason) |
189
+
190
+ ### Rule Validation with Documents (`rule_validation_with_documents`)
191
+
192
+ | | |
193
+ | -------------- | ----------------------------------------------------------------------------- |
194
+ | **Purpose** | Check extracted data against business rules (compliance, thresholds) |
195
+ | **Inputs** | `primary_docs` (DOCUMENT), `map_of_extracted_columns` (ANY) |
196
+ | **Outputs** | `ruleset_output` (ANY — validation results) |
197
+ | **Pairs with** | entity_extraction (provides data to check), call_llm (summarizes results) |
198
+ | **Critical** | Rules configured in UI settings panel, not via workflow_def inputs. |
199
+
200
+ ### Entity Extraction with Documents (`entity_extraction_with_documents`)
201
+
202
+ | | |
203
+ | -------------- | --------------------------------------------------------------------------------------------------- |
204
+ | **Purpose** | Extract structured entities grounded in provided documents |
205
+ | **Inputs** | `documents` (DOCUMENT), extraction column config |
206
+ | **Outputs** | `extraction_columns` (ANY — structured extraction results, needs intermediary for type-safe wiring) |
207
+ | **Pairs with** | json_mapper (field extraction), rule_validation (compliance), send_email (via intermediary) |
208
+ | **Critical** | Output type is ANY — needs intermediary before send_email inputs |
209
+
210
+ ### JSON Mapper (`json_mapper`)
211
+
212
+ | | |
213
+ | -------------- | ------------------------------------------------------------------------------------------- |
214
+ | **Purpose** | Extract specific fields from JSON/structured data into individual outputs |
215
+ | **Inputs** | `input_json` (ANY), mapping rules |
216
+ | **Outputs** | `output_json` per mapped field |
217
+ | **Pairs with** | entity_extraction (field extraction), fixed_response (type conversion to TEXT_WITH_SOURCES) |
218
+
219
+ ---
220
+
221
+ ## Entity Extraction Best Practices
222
+
223
+ Ranked by reliability:
224
+
225
+ ### 1. json_mapper with Schema (Best for known fields)
226
+
227
+ One-step extraction when you know the exact fields and types.
228
+
229
+ ```
230
+ entity_extraction → json_mapper(rules=[{fieldName: "email", path: "email_address"}])
231
+ ```
232
+
233
+ ### 2. entity_extraction_with_documents (Best for grounded extraction)
234
+
235
+ When extracted values MUST come from provided documents (not hallucinated).
236
+
237
+ ```
238
+ document_trigger → entity_extraction_with_documents(extraction_columns=[...])
239
+ ```
240
+
241
+ ### 3. Two-step: Extract then Normalize
242
+
243
+ When you need high recall first, then strict formatting.
244
+
245
+ ```
246
+ call_llm(extract) → json_mapper(normalize dates, format phones, dedupe)
247
+ ```
248
+
249
+ ### 4. call_llm with Extraction Prompt (Most flexible)
250
+
251
+ When extraction logic is complex or requires reasoning.
252
+
253
+ ```
254
+ call_llm(prompt="Extract the patient email from this message")
255
+ ```
256
+
257
+ **Validation rule**: For critical fields (emails, phone numbers, order IDs), add deterministic
258
+ validation as a separate step — regex/rules via `rule_validation_with_documents` or `response_validator`.
259
+
260
+ **Scoping rule**: Pass only the minimum context needed:
261
+
262
+ - Extracting from current message? Use `trigger.user_query` only
263
+ - Extracting from documents? Use search results or uploaded docs
264
+ - Entity might span multiple turns? Include conversation history
265
+
266
+ ---
267
+
268
+ ## Common Wiring Patterns
269
+
270
+ ### Intent Routing with Shared Search
271
+
272
+ Search runs once, results shared across all gated respond nodes:
273
+
274
+ ```
275
+ chat_trigger ─┬─→ chat_categorizer
276
+
277
+ └─→ search ─→ [results shared to all respond nodes]
278
+ ├─→ respond_A (runIf: category==A) ─→ WORKFLOW_OUTPUT
279
+ ├─→ respond_B (runIf: category==B) ─→ WORKFLOW_OUTPUT
280
+ └─→ fallback (runIf: category==Fallback) ─→ WORKFLOW_OUTPUT
281
+ ```
282
+
283
+ **Critical**: ALL respond branches must wire to WORKFLOW_OUTPUT.
284
+
285
+ ### Consolidated Intent Response
286
+
287
+ Single respond node handles all intents, reducing duplication:
288
+
289
+ ```
290
+ chat_trigger ─┬─→ chat_categorizer ─→ call_llm.named_inputs_Intent
291
+ ├─→ search ─→ call_llm.named_inputs_Search_Results
292
+ └─→ call_llm.query
293
+ call_llm.response ─→ WORKFLOW_OUTPUT (runIf: != Fallback)
294
+ fallback ─→ WORKFLOW_OUTPUT (runIf: == Fallback)
295
+ ```
296
+
297
+ **Use when**: All intents share similar prompts, search sources, and safety constraints.
298
+
299
+ ### Externalized Instructions via KB
300
+
301
+ Instructions live in uploaded documents, not hardcoded prompts:
302
+
303
+ ```
304
+ Upload: "Scheduling Policy.pdf", "Tone Guide.pdf" to KB
305
+
306
+ chat_trigger → conversation_to_search_query → search → respond_with_sources → WORKFLOW_OUTPUT
307
+ (converts CHAT_CONVERSATION (retrieves instructions (grounded response
308
+ to TEXT_WITH_SOURCES query) + knowledge docs) with citations)
309
+ ```
310
+
311
+ **Benefit**: Change behavior by updating documents, not workflow structure.
312
+
313
+ ---
314
+
315
+ ## Extraction Chain: entity_extraction → json_mapper → fixed_response
316
+
317
+ When extracting structured data from documents and using it downstream (e.g., sending an email), nodes must be chained in a specific order due to type constraints.
318
+
319
+ ### The Chain
320
+
321
+ ```
322
+ entity_extraction_with_documents → json_mapper → fixed_response → send_email_agent
323
+ (ANY output) (field decomposition) (type conversion) (TEXT_WITH_SOURCES inputs)
324
+ ```
325
+
326
+ **Why this order?**
327
+
328
+ 1. **entity_extraction** outputs `ANY` type (structured JSON) -- cannot be wired directly to send_email inputs which expect `TEXT_WITH_SOURCES`
329
+ 2. **json_mapper** decomposes the structured JSON into individual fields -- still `ANY` type per field
330
+ 3. **fixed_response** converts each field to `TEXT_WITH_SOURCES` using template variables (`{{to}}`, `{{subject}}`)
331
+ 4. **send_email_agent** receives properly typed `TEXT_WITH_SOURCES` inputs
332
+
333
+ ### Wiring Detail
334
+
335
+ ```
336
+ entity_extraction.extraction_columns → json_mapper.input_json
337
+ json_mapper.output_json → fixed_response_to.named_inputs_Extracted_Data (template: "{{to}}")
338
+ json_mapper.output_json → fixed_response_subj.named_inputs_Extracted_Data (template: "{{subject}}")
339
+ json_mapper.output_json → fixed_response_body.named_inputs_Extracted_Data (template: "{{body}}")
340
+ fixed_response_to.response → send_email_agent.email_to
341
+ fixed_response_subj.response → send_email_agent.email_subject
342
+ fixed_response_body.response → send_email_agent.email_body
343
+ ```
344
+
345
+ You need one `fixed_response` per email field (to, subject, body).
346
+
347
+ ### When to Use json_mapper vs entity_extraction
348
+
349
+ | Scenario | Use | Why |
350
+ |---|---|---|
351
+ | Extract from uploaded documents | `entity_extraction_with_documents` | Grounded in source documents, structured schema |
352
+ | Decompose structured JSON into fields | `json_mapper` | Field-level access from ANY-typed data |
353
+ | Extract from conversation text | `call_llm` with extraction prompt | Flexible, handles reasoning |
354
+ | Format fields for typed inputs | `fixed_response` with template | Converts ANY → TEXT_WITH_SOURCES |
355
+
356
+ ### Alternative: custom_agent → json_mapper
357
+
358
+ When using `custom_agent` instead of `entity_extraction`, you **must** configure `output_fields` or use strict JSON-only prompting. Without this, `custom_agent` returns JSON as a string blob in `response_with_sources`, and `json_mapper` fails to parse it.
359
+
360
+ **Recommended**: Define `output_fields` on custom_agent (same extraction column format as entity_extraction).
361
+
362
+ See `ema://rules/json-output-patterns` for the full custom_agent + json_mapper pattern.
363
+
364
+ ---
365
+
366
+ ## Search Configuration
367
+
368
+ ### search/v2 vs search/v0
369
+
370
+ Always use **search/v2** (not deprecated search/v0). The key difference is `datastore_configs` input.
371
+
372
+ ### datastore_configs Wiring
373
+
374
+ The `datastore_configs` input connects search to the persona's uploaded data sources via widget configuration:
375
+
376
+ ```json
377
+ "datastore_configs": {
378
+ "multiBinding": {
379
+ "elements": [{
380
+ "widgetConfig": { "widgetName": "fileUpload" }
381
+ }]
382
+ }
383
+ }
384
+ ```
385
+
386
+ - `widgetName` maps to the persona widget that holds data sources
387
+ - Default widget: `fileUpload` (standard KB upload widget)
388
+ - For personas with multiple upload widgets: use the specific widget name (e.g., `v822`, `upload1`)
389
+
390
+ ### Search Input Rules
391
+
392
+ | Input Source | When to Use | Why |
393
+ |---|---|---|
394
+ | `trigger.user_query` | Simple single-turn search | Direct text query |
395
+ | `conversation_to_search_query.summarized_conversation` | Multi-turn chat | Converts conversation history to search-optimized query |
396
+ | `trigger.chat_conversation` | **Never for search** | Wrong type (CHAT_CONVERSATION vs TEXT_WITH_SOURCES) |
397
+
398
+ ### Multi-Source Search
399
+
400
+ Combine local KB search with web search using `combine_search_results`:
401
+
402
+ ```
403
+ conversation_to_search_query ─┬─→ search (local KB)
404
+ └─→ live_web_search (real-time web)
405
+ ├─→ combine_search_results.search_results_1
406
+ └─→ combine_search_results.search_results_2
407
+ → respond_with_sources.search_results
408
+ ```
409
+
410
+ ### Critical: Upload Data Before Deploying Search Workflows
411
+
412
+ Search returns empty results without uploaded documents. Always:
413
+ 1. Upload: `persona(id='...', data={method:'upload', path:'/path/to/doc.pdf'})`
414
+ 2. Verify: `persona(id='...', data={method:'stats'})` → check `success` count > 0
415
+ 3. Then deploy the workflow
416
+
417
+ ---
418
+
419
+ ## LLM Node Configuration (call_llm)
420
+
421
+ ### named_inputs Convention
422
+
423
+ `call_llm` accepts additional context via `named_inputs` using the suffix pattern `named_inputs_<Descriptive_Name>`:
424
+
425
+ | Named Input | Type | Purpose |
426
+ |---|---|---|
427
+ | `named_inputs_Search_Results` | SEARCH_RESULT | KB search results for RAG |
428
+ | `named_inputs_Conversation` | CHAT_CONVERSATION | Full conversation history |
429
+ | `named_inputs_Intent` | ENUM | Detected category from categorizer |
430
+ | `named_inputs_Current_Message` | TEXT_WITH_SOURCES | Current user message for extraction |
431
+ | `named_inputs_Tool_Result` | TEXT_WITH_SOURCES | External action output |
432
+
433
+ `named_inputs` accepts **ANY** type -- this is how you pass CHAT_CONVERSATION and SEARCH_RESULT into LLM nodes.
434
+
435
+ ### Temperature Guidelines
436
+
437
+ | Use Case | Temperature | Why |
438
+ |---|---|---|
439
+ | Document generation | 0.3-0.5 | Consistent formatting, predictable output |
440
+ | Entity extraction | 0.0-0.3 | Accuracy over creativity |
441
+ | General Q&A / chat | 0.5-0.7 | Balanced creativity and accuracy |
442
+ | Creative writing | 0.7-1.0 | More varied, creative output |
443
+
444
+ ### Avoiding Duplicate LLM Nodes
445
+
446
+ If multiple `call_llm` nodes share the same inputs (query, search_results, conversation) and differ only by `trigger_when` gate, **consolidate** them:
447
+
448
+ 1. Remove duplicate respond nodes
449
+ 2. Create one `call_llm` that always runs (or runs when not Fallback)
450
+ 3. Wire `categorizer.category → call_llm.named_inputs_Intent`
451
+ 4. Update prompt: "Based on the detected intent ({{Intent}}), respond accordingly"
452
+ 5. Wire single `call_llm.response_with_sources → WORKFLOW_OUTPUT`
453
+
454
+ Keep separate nodes **only** when intents require different tools, search sources, or safety constraints.
455
+
456
+ ### Scoping Extraction Inputs
457
+
458
+ When using `call_llm` for entity extraction, scope inputs tightly:
459
+
460
+ - Extracting from current message? → `trigger.user_query` only
461
+ - Extracting from documents? → `search.search_results` (intentional)
462
+ - Entity might span multiple turns? → Include `chat_conversation`
463
+
464
+ Passing full search_results to extraction prompts causes hallucinated entities from irrelevant context.
465
+
466
+ ---
467
+
468
+ ## Categorizer Patterns
469
+
470
+ ### Basic Rules
471
+
472
+ - **Input**: `chat_categorizer` MUST receive `chat_conversation` (not `user_query`) for accurate multi-turn classification
473
+ - **Fallback**: ALWAYS include a Fallback category
474
+ - **Handlers**: Every category must have at least one node with a matching `runIf` condition
475
+ - **typeArguments**: Categorizer must have `typeArguments.categories` pointing to the enum type -- empty `typeArguments` causes deploy failure
476
+
477
+ ### runIf Condition Format
478
+
479
+ ```json
480
+ {
481
+ "lhs": {
482
+ "actionOutput": { "actionName": "chat_categorizer", "output": "category" },
483
+ "autoDetectedBinding": false
484
+ },
485
+ "operator": 1,
486
+ "rhs": {
487
+ "inline": { "enumValue": "Market_Impact" },
488
+ "autoDetectedBinding": false
489
+ }
490
+ }
491
+ ```
492
+
493
+ **Important**: Use `category` as the output name and compare to `enumValue` directly. Do NOT use `category_<Name>` format.
494
+
495
+ ### runIf Operator Values
496
+
497
+ | Operator | Meaning | Use Case |
498
+ |---|---|---|
499
+ | `1` | Equals (`==`) | Route to handler when category matches |
500
+ | `2` | Not equals (`!=`) | Run for all categories except one (e.g., `!= Fallback`) |
501
+
502
+ For OR conditions (e.g., "run for Sales OR General"), use operator `2` with Fallback: `category != Fallback` -- this runs the node for all non-fallback categories.
503
+
504
+ ### Defining enumTypes
505
+
506
+ Categories must be defined in the `workflow_def.enumTypes` array:
507
+
508
+ ```json
509
+ "enumTypes": [{
510
+ "name": { "name": "intent_categories", "namespaces": [] },
511
+ "options": [
512
+ { "name": "Sales", "description": "Sales inquiries and product questions" },
513
+ { "name": "Support", "description": "Technical support and ticket creation" },
514
+ { "name": "General", "description": "General questions" },
515
+ { "name": "Fallback", "description": "Unclear or unmatched intents" }
516
+ ]
517
+ }]
518
+ ```
519
+
520
+ Then the categorizer's `typeArguments` references this enum:
521
+
522
+ ```json
523
+ "typeArguments": {
524
+ "categories": {
525
+ "enumType": { "name": { "name": "intent_categories", "namespaces": [] } },
526
+ "isList": false
527
+ }
528
+ }
529
+ ```
530
+
531
+ ### text_categorizer vs chat_categorizer
532
+
533
+ | Categorizer | Input Type | When to Use |
534
+ |---|---|---|
535
+ | `chat_categorizer` | CHAT_CONVERSATION | Routing conversations (most common) |
536
+ | `text_categorizer/v1` | named_inputs (multiBinding) | Routing based on text content, not conversation |
537
+
538
+ `text_categorizer/v0` is deprecated -- use v1 with `named_inputs`.
539
+
540
+ ### Nested Categorizers
541
+
542
+ For complex routing with sub-intents, chain categorizers:
543
+
544
+ ```
545
+ chat_categorizer (Level 1: HR, IT, General, Fallback)
546
+ └─→ text_categorizer (Level 2 for HR: Benefits, Leave, Payroll, Fallback)
547
+ ```
548
+
549
+ Use clear precedence -- first categorizer routes broadly, second categorizer handles sub-intent.
550
+
551
+ ---
552
+
553
+ ## Error Handling & Validation Patterns
554
+
555
+ ### response_validator + abstain_action
556
+
557
+ For guardrails on LLM output:
558
+
559
+ ```
560
+ call_llm.response → response_validator ─→ abstain_action (if validation fails)
561
+ └─→ WORKFLOW_OUTPUT (if validation passes)
562
+ ```
563
+
564
+ - `response_validator` checks generated response against reference query for quality/compliance
565
+ - `abstain_action` provides a safe decline message when AI should not answer
566
+
567
+ ### Graceful Degradation
568
+
569
+ When search returns no results or external APIs fail:
570
+
571
+ ```
572
+ search → call_llm (runIf: search has results)
573
+ fixed_response (runIf: search empty) → "I don't have enough information to answer that."
574
+ ```
575
+
576
+ ### Choosing the Right Validator
577
+
578
+ | Validator | Input Type | Use Case |
579
+ |---|---|---|
580
+ | `response_validator` | TEXT_WITH_SOURCES | Validate LLM-generated text (quality, compliance, hallucination) |
581
+ | `rule_validation_with_documents` | ANY | Validate extracted data against business rules (thresholds, required fields) |
582
+
583
+ For **extraction workflows**, use `rule_validation_with_documents` since it accepts ANY-typed extraction output directly. `response_validator` only works with TEXT_WITH_SOURCES (LLM responses), so you'd need the full intermediary chain (json_mapper → fixed_response) before it.
584
+
585
+ ### rule_validation_with_documents
586
+
587
+ For business rule compliance checking:
588
+
589
+ ```
590
+ entity_extraction → rule_validation_with_documents → call_llm (summarize results)
591
+ ```
592
+
593
+ Rules are configured in the UI settings panel, not via `workflow_def` inputs.
594
+
595
+ ---
596
+
597
+ ## HITL (Human-in-the-Loop) Patterns
598
+
599
+ ### How HITL Works
600
+
601
+ HITL is a **flag on specific action nodes**, NOT a standalone workflow node.
602
+
603
+ ```json
604
+ {
605
+ "name": "send_email",
606
+ "action": { "name": "send_email_agent" },
607
+ "disable_human_interaction": false
608
+ }
609
+ ```
610
+
611
+ **Counter-intuitive naming:**
612
+ - `disable_human_interaction: false` → HITL **ON** (requires approval)
613
+ - `disable_human_interaction: true` → HITL **OFF** (auto-proceeds)
614
+ - Omitted → defaults to false (HITL ON)
615
+
616
+ ### When to Enable HITL
617
+
618
+ | Action | Enable HITL? | Why |
619
+ |---|---|---|
620
+ | `send_email_agent` | Yes, if user requests approval | External communication |
621
+ | `external_action_caller` | Yes, for destructive operations | Side effects (create ticket, update CRM) |
622
+ | `call_llm` | Usually no | No external side effects |
623
+ | `search` | No | Read-only operation |
624
+
625
+ ### Legacy general_hitl Nodes
626
+
627
+ Existing workflows may have standalone `general_hitl` nodes -- these still function, but **do not add new ones**. Use the flag pattern instead.
628
+
629
+ ---
630
+
631
+ ## Dashboard Output Columns
632
+
633
+ For dashboard personas, every workflow output mapped in `resultMappings` becomes a column in the dashboard UI.
634
+
635
+ ### resultMappings Example
636
+
637
+ ```json
638
+ "results": {
639
+ "entity_extraction.recipient_email": {
640
+ "actionName": "entity_extraction",
641
+ "outputName": "recipient_email"
642
+ },
643
+ "entity_extraction.subject_line": {
644
+ "actionName": "entity_extraction",
645
+ "outputName": "subject_line"
646
+ },
647
+ "entity_extraction.amount": {
648
+ "actionName": "entity_extraction",
649
+ "outputName": "amount"
650
+ }
651
+ }
652
+ ```
653
+
654
+ Each entry creates one visible dashboard column. The key (e.g., `entity_extraction.recipient_email`) becomes the column display name.
655
+
656
+ ### Key Rules
657
+
658
+ - **Missing mapping = invisible column**: If you forget to map an output, it won't appear in the dashboard
659
+ - **Column ordering**: Determined by the order of entries in `results` JSON. To reorder: remove all entries, re-add in desired order
660
+ - **Input columns**: Come from `document_trigger` -- `document_content` for files, `row_data` for text/number inputs
661
+
662
+ ### Wiring document_trigger to entity_extraction
663
+
664
+ ```
665
+ document_trigger.document_content → entity_extraction_with_documents.documents
666
+ ```
667
+
668
+ For additional column data from the dashboard row:
669
+ ```
670
+ document_trigger.row_data → call_llm.named_inputs_Row_Data
671
+ ```
672
+
673
+ See `.context/public/guides/dashboard-operations.md` for full dashboard column details including nested structures.
674
+
675
+ ## Extraction Column Groups (Nested Structures)
676
+
677
+ Entity extraction supports grouped/nested columns using `dataType: 5` (OBJECT). A group column has sub-columns defined in `value.objectValue.values`. This appears in the UI as a parent with expandable children.
678
+
679
+ See `ema://rules/extraction-column-format` for the full API shape including nested examples.
680
+
681
+ ## Anti-Pattern Quick Reference
682
+
683
+ | Anti-Pattern | Symptom | Fix |
684
+ | -------------------------------- | ---------------------------------------------------- | ---------------------------------------------------------- |
685
+ | Partial output wiring | Some intents return no response | Wire ALL gated respond nodes to WORKFLOW_OUTPUT |
686
+ | Duplicate identical LLM nodes | Multiple nodes with same inputs, different gates | Consolidate to one node + category as named_input |
687
+ | Overscoped extraction | Extractions hallucinate entities from search results | Scope to `user_query` only |
688
+ | Hardcoded instructions | Must redeploy workflow to change behavior | Upload instructions as KB documents |
689
+ | Missing fallback | Unrecognized intents silently fail | Always include Fallback category |
690
+ | Text content as email recipient | Email sends fail or go to garbage addresses | Use entity_extraction → json_mapper → fixed_response chain |
691
+ | Missing dashboard output columns | Node output not visible in dashboard | Add resultMapping entry for every desired output column |
692
+ | Wrong column order in dashboard | Columns appear in unexpected order | Remove and re-add resultMappings in desired order |
693
+ | entity_extraction direct to email | Type mismatch (ANY vs TEXT_WITH_SOURCES) | Use json_mapper + fixed_response intermediary chain |
694
+ | json_mapper without upstream data | json_mapper receives no structured input | Wire from entity_extraction, custom_agent, or other JSON source |
695
+ | Search without uploaded data | Search returns empty results | Upload documents before deploying search workflows |
696
+ | Redundant search nodes | Multiple searches with same query | Single search node, share results via named_inputs |
697
+ | Sequential LLM calls | Unnecessary latency and incoherence | Single call_llm with comprehensive instructions |
698
+ | Categorizer without typeArguments | Deploy failure | Add typeArguments.categories pointing to enumType |
699
+
700
+ ---
701
+
702
+ ## References
703
+
704
+ - `src/mcp/domain/validation-rules.ts` — Anti-pattern definitions
705
+ - `src/mcp/domain/structural-rules.ts` — Structural invariants
706
+ - `src/mcp/knowledge.ts` — Workflow patterns and agent catalog
707
+ - `.context/public/guides/email-patterns.md` — Email wiring patterns
708
+ - `.context/public/guides/dashboard-operations.md` — Dashboard-specific patterns