@ema.co/mcp-toolkit 2026.2.5 → 2026.2.13
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Potentially problematic release.
This version of @ema.co/mcp-toolkit might be problematic. Click here for more details.
- package/.context/public/guides/dashboard-operations.md +63 -0
- package/.context/public/guides/workflow-builder-patterns.md +708 -0
- package/LICENSE +29 -21
- package/README.md +58 -35
- package/dist/mcp/domain/proto-constraints.js +284 -0
- package/dist/mcp/domain/structural-rules.js +8 -0
- package/dist/mcp/domain/validation-rules.js +102 -15
- package/dist/mcp/domain/workflow-graph-optimizer.js +235 -0
- package/dist/mcp/domain/workflow-graph-transforms.js +808 -0
- package/dist/mcp/domain/workflow-graph.js +376 -0
- package/dist/mcp/domain/workflow-optimizer.js +10 -4
- package/dist/mcp/guidance.js +45 -2
- package/dist/mcp/handlers/feedback/index.js +139 -0
- package/dist/mcp/handlers/feedback/store.js +262 -0
- package/dist/mcp/handlers/workflow/index.js +12 -11
- package/dist/mcp/handlers/workflow/optimize.js +73 -33
- package/dist/mcp/knowledge.js +87 -36
- package/dist/mcp/resources.js +393 -17
- package/dist/mcp/server.js +38 -4
- package/dist/mcp/tools.js +89 -2
- package/dist/sdk/generated/deprecated-actions.js +182 -96
- package/dist/sdk/generated/proto-fields.js +2 -1
- package/dist/sdk/generated/protos/service/agent_qa/v1/agent_qa_pb.js +460 -21
- package/dist/sdk/generated/protos/service/auth/v1/auth_pb.js +11 -1
- package/dist/sdk/generated/protos/service/dataingest/v1/dataingest_pb.js +173 -66
- package/dist/sdk/generated/protos/service/feedback/v1/feedback_pb.js +43 -1
- package/dist/sdk/generated/protos/service/llmservice/v1/llmservice_pb.js +26 -21
- package/dist/sdk/generated/protos/service/persona/v1/persona_config_pb.js +100 -89
- package/dist/sdk/generated/protos/service/persona/v1/persona_pb.js +126 -116
- package/dist/sdk/generated/protos/service/persona/v1/shared_widgets/widget_types_pb.js +33 -1
- package/dist/sdk/generated/protos/service/persona/v1/voicebot_widgets/widget_types_pb.js +60 -11
- package/dist/sdk/generated/protos/service/tenant/v1/tenant_pb.js +1 -1
- package/dist/sdk/generated/protos/service/user/v1/user_pb.js +1 -1
- package/dist/sdk/generated/protos/service/utils/v1/agent_qa_pb.js +35 -0
- package/dist/sdk/generated/protos/service/workflows/v1/action_registry_pb.js +1 -1
- package/dist/sdk/generated/protos/service/workflows/v1/action_type_pb.js +6 -1
- package/dist/sdk/generated/protos/service/workflows/v1/chatbot_pb.js +106 -11
- package/dist/sdk/generated/protos/service/workflows/v1/common_forms_pb.js +1 -1
- package/dist/sdk/generated/protos/service/workflows/v1/coordinator_pb.js +1 -1
- package/dist/sdk/generated/protos/service/workflows/v1/external_actions_pb.js +31 -1
- package/dist/sdk/generated/protos/service/workflows/v1/well_known_pb.js +5 -1
- package/dist/sdk/generated/protos/service/workflows/v1/workflow_pb.js +1 -1
- package/dist/sdk/generated/protos/util/tracking_metadata_pb.js +1 -1
- package/package.json +2 -2
|
@@ -0,0 +1,708 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: "Workflow Builder Patterns & Node Reference"
|
|
3
|
+
date: 2026-02-05
|
|
4
|
+
audience: MCP users, workflow builders, auto-builder agents
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# Workflow Builder Patterns & Node Reference
|
|
8
|
+
|
|
9
|
+
Practical reference for building Ema workflows. Covers input/output semantics for each node type,
|
|
10
|
+
data types flowing between nodes, entity extraction best practices, and common wiring patterns.
|
|
11
|
+
|
|
12
|
+
Extracted from real auto-builder sessions and validated against platform behavior.
|
|
13
|
+
|
|
14
|
+
---
|
|
15
|
+
|
|
16
|
+
## Data Types Between Nodes
|
|
17
|
+
|
|
18
|
+
Four core data types flow between workflow nodes:
|
|
19
|
+
|
|
20
|
+
| Type | Code Constant | Description | Example |
|
|
21
|
+
| ------------------ | ----------------------------------- | ------------------------------------------------- | ---------------------------------------------- |
|
|
22
|
+
| **Plain Text** | `WELL_KNOWN_TYPE_TEXT_WITH_SOURCES` | Text with optional citation metadata | User's message, LLM response, formatted output |
|
|
23
|
+
| **Conversation** | `WELL_KNOWN_TYPE_CHAT_CONVERSATION` | Structured message history (role + content pairs) | Full chat thread for context |
|
|
24
|
+
| **Search Results** | `WELL_KNOWN_TYPE_SEARCH_RESULT` | Retrieved document chunks with citations | KB search results with source metadata |
|
|
25
|
+
| **Enum** | `WELL_KNOWN_TYPE_ENUM` | Category/classification signal for routing | `category::Schedule Appointment` |
|
|
26
|
+
| **Document** | `WELL_KNOWN_TYPE_DOCUMENT` | Uploaded file content | PDF, DOCX for extraction |
|
|
27
|
+
| **Any** | `WELL_KNOWN_TYPE_ANY` | Untyped — needs intermediary for type-safe wiring | entity_extraction output |
|
|
28
|
+
|
|
29
|
+
**Critical rule**: Types are NOT interchangeable. `CHAT_CONVERSATION` into a `TEXT_WITH_SOURCES` input
|
|
30
|
+
causes type mismatch errors. Use converter nodes (`conversation_to_search_query`) when needed.
|
|
31
|
+
|
|
32
|
+
---
|
|
33
|
+
|
|
34
|
+
## Input Semantics by Context
|
|
35
|
+
|
|
36
|
+
The same input name means different things depending on the node:
|
|
37
|
+
|
|
38
|
+
| Input Name | In Search Nodes | In Respond Nodes | In Extract Nodes | In Categorizers |
|
|
39
|
+
| ---------------- | ---------------------- | ------------------------- | ---------------------- | ------------------------------- |
|
|
40
|
+
| `query` | Search term to look up | User's question to answer | Source text to analyze | N/A |
|
|
41
|
+
| `conversation` | N/A | N/A (use named_inputs) | N/A | Full history for classification |
|
|
42
|
+
| `trigger_when` | N/A | "Should I run?" | "Should I run?" | N/A |
|
|
43
|
+
| `named_inputs_*` | N/A | Additional context | Additional context | N/A |
|
|
44
|
+
|
|
45
|
+
**Universal mental model:**
|
|
46
|
+
|
|
47
|
+
- `query` = "What should I process?"
|
|
48
|
+
- `conversation` = "What's the full context?"
|
|
49
|
+
- `trigger_when` = "Should I run at all?"
|
|
50
|
+
- `named_inputs_*` = "Extra context" (search results, current message, etc.)
|
|
51
|
+
|
|
52
|
+
---
|
|
53
|
+
|
|
54
|
+
## Node Type Reference
|
|
55
|
+
|
|
56
|
+
### Chat Trigger (`chat_trigger`)
|
|
57
|
+
|
|
58
|
+
| | |
|
|
59
|
+
| -------------- | ------------------------------------------------------------------------- |
|
|
60
|
+
| **Purpose** | Entry point for chat workflows |
|
|
61
|
+
| **Inputs** | None (system event) |
|
|
62
|
+
| **Outputs** | `user_query` (TEXT_WITH_SOURCES), `chat_conversation` (CHAT_CONVERSATION) |
|
|
63
|
+
| **Pairs with** | Intent routers, search nodes, LLM nodes |
|
|
64
|
+
|
|
65
|
+
### Intent Router (`chat_categorizer`)
|
|
66
|
+
|
|
67
|
+
| | |
|
|
68
|
+
| -------------- | ------------------------------------------------------------------------- |
|
|
69
|
+
| **Purpose** | Classify conversation into intent categories for routing |
|
|
70
|
+
| **Inputs** | `conversation` (CHAT_CONVERSATION) — must be full history, NOT user_query |
|
|
71
|
+
| **Outputs** | `category` (ENUM) — one per configured category |
|
|
72
|
+
| **Pairs with** | Multiple gated respond/action nodes, fallback |
|
|
73
|
+
| **Critical** | Always include Fallback category. Every category needs a handler. |
|
|
74
|
+
|
|
75
|
+
### Knowledge Search (`search/v2`)
|
|
76
|
+
|
|
77
|
+
| | |
|
|
78
|
+
| -------------- | ----------------------------------------------------------------------------- |
|
|
79
|
+
| **Purpose** | Retrieve relevant documents from uploaded knowledge base |
|
|
80
|
+
| **Inputs** | `query` (TEXT_WITH_SOURCES), `datastore_configs` (from widget) |
|
|
81
|
+
| **Outputs** | `search_results` (SEARCH_RESULT) |
|
|
82
|
+
| **Pairs with** | Respond nodes (for grounded answers), extract nodes (for grounded extraction) |
|
|
83
|
+
| **Critical** | NOT an LLM node — do NOT include `model_config`. Data must be uploaded first. |
|
|
84
|
+
|
|
85
|
+
### Respond (call_llm, respond_with_sources)
|
|
86
|
+
|
|
87
|
+
| | |
|
|
88
|
+
| -------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
89
|
+
| **Purpose** | Generate natural language response using LLM |
|
|
90
|
+
| **Inputs** | `query` (TEXT_WITH_SOURCES), `named_inputs_Conversation` (CHAT_CONVERSATION), `named_inputs_Search_Results` (SEARCH_RESULT), `trigger_when` (ENUM) |
|
|
91
|
+
| **Outputs** | `response_with_sources` (TEXT_WITH_SOURCES) |
|
|
92
|
+
| **Pairs with** | Search nodes (for RAG), categorizers (for gating), WORKFLOW_OUTPUT |
|
|
93
|
+
| **Critical** | Must wire to WORKFLOW_OUTPUT or response is silently lost |
|
|
94
|
+
|
|
95
|
+
### Extract (call_llm configured for extraction)
|
|
96
|
+
|
|
97
|
+
| | |
|
|
98
|
+
| -------------- | ------------------------------------------------------------------------------------------------------ |
|
|
99
|
+
| **Purpose** | Extract specific entities (emails, dates, IDs) from text |
|
|
100
|
+
| **Inputs** | `query` (TEXT_WITH_SOURCES), `named_inputs_Current_Message` (TEXT_WITH_SOURCES), `trigger_when` (ENUM) |
|
|
101
|
+
| **Outputs** | Extracted data (TEXT_WITH_SOURCES) |
|
|
102
|
+
| **Pairs with** | External actions, json_mapper, send_email |
|
|
103
|
+
| **Critical** | Scope inputs tightly — use only `user_query` unless entity spans multiple turns |
|
|
104
|
+
|
|
105
|
+
### Fixed Response (`fixed_response`)
|
|
106
|
+
|
|
107
|
+
| | |
|
|
108
|
+
| -------------- | --------------------------------------------------------------------------- |
|
|
109
|
+
| **Purpose** | Return a static predefined message (fallback, compliance notice) |
|
|
110
|
+
| **Inputs** | `trigger_when` (ENUM), `named_inputs_*` for template variables |
|
|
111
|
+
| **Outputs** | `fixed_response_with_sources` (TEXT_WITH_SOURCES) |
|
|
112
|
+
| **Pairs with** | Categorizers (as fallback), type conversion (template with `{{variables}}`) |
|
|
113
|
+
|
|
114
|
+
### External Action Caller (`external_action_caller`)
|
|
115
|
+
|
|
116
|
+
| | |
|
|
117
|
+
| -------------- | ----------------------------------------------------------------------------------- |
|
|
118
|
+
| **Purpose** | Call external APIs/tools (ServiceNow, Salesforce, calendars) |
|
|
119
|
+
| **Inputs** | `query` (TEXT_WITH_SOURCES), `conversation` (CHAT_CONVERSATION), tool configuration |
|
|
120
|
+
| **Outputs** | `tool_execution_result` (TEXT_WITH_SOURCES) |
|
|
121
|
+
| **Pairs with** | Entity extractors (for parameters), respond nodes (for result formatting) |
|
|
122
|
+
|
|
123
|
+
### Document Trigger (`document_trigger`)
|
|
124
|
+
|
|
125
|
+
| | |
|
|
126
|
+
| -------------- | ----------------------------------------------------------------------------- |
|
|
127
|
+
| **Purpose** | Entry point for dashboard workflows (file upload per row) |
|
|
128
|
+
| **Inputs** | None (system event — triggered when a row is created/file uploaded) |
|
|
129
|
+
| **Outputs** | `document_content` (DOCUMENT — uploaded file), `row_data` (ANY — column values) |
|
|
130
|
+
| **Pairs with** | entity_extraction_with_documents, call_llm, search |
|
|
131
|
+
| **Critical** | Dashboard personas only. Each row triggers one workflow execution. |
|
|
132
|
+
|
|
133
|
+
### Send Email Agent (`send_email_agent`)
|
|
134
|
+
|
|
135
|
+
| | |
|
|
136
|
+
| -------------- | -------------------------------------------------------------------------------------- |
|
|
137
|
+
| **Purpose** | Send email with specified recipient, subject, and body |
|
|
138
|
+
| **Inputs** | `email_to` (TEXT_WITH_SOURCES), `email_subject` (TEXT_WITH_SOURCES), `email_body` (TEXT_WITH_SOURCES) |
|
|
139
|
+
| **Outputs** | `confirmation` (TEXT_WITH_SOURCES — send result) |
|
|
140
|
+
| **Pairs with** | fixed_response (for type conversion), json_mapper (field extraction) |
|
|
141
|
+
| **Critical** | Inputs must be TEXT_WITH_SOURCES — use intermediary chain from ANY-typed sources. Enable HITL via `disable_human_interaction: false` for approval. |
|
|
142
|
+
|
|
143
|
+
### Live Web Search (`live_web_search`)
|
|
144
|
+
|
|
145
|
+
| | |
|
|
146
|
+
| -------------- | ----------------------------------------------------------------------------- |
|
|
147
|
+
| **Purpose** | Real-time web search for current information |
|
|
148
|
+
| **Inputs** | `query` (TEXT_WITH_SOURCES) |
|
|
149
|
+
| **Outputs** | `web_search_results` (SEARCH_RESULT) |
|
|
150
|
+
| **Pairs with** | combine_search_results (to merge with KB search), respond nodes |
|
|
151
|
+
| **Critical** | No data upload needed (searches the web). Use for current events, external info not in KB. |
|
|
152
|
+
|
|
153
|
+
### Combine Search Results (`combine_search_results`)
|
|
154
|
+
|
|
155
|
+
| | |
|
|
156
|
+
| -------------- | ----------------------------------------------------------------------------- |
|
|
157
|
+
| **Purpose** | Merge results from multiple search sources with deduplication |
|
|
158
|
+
| **Inputs** | `search_results_1` (SEARCH_RESULT), `search_results_2` (SEARCH_RESULT) |
|
|
159
|
+
| **Outputs** | `combined_results` (SEARCH_RESULT) |
|
|
160
|
+
| **Pairs with** | search/v2 + live_web_search, or any two search sources |
|
|
161
|
+
|
|
162
|
+
### Conversation to Search Query (`conversation_to_search_query`)
|
|
163
|
+
|
|
164
|
+
| | |
|
|
165
|
+
| -------------- | ----------------------------------------------------------------------------- |
|
|
166
|
+
| **Purpose** | Convert multi-turn conversation to a search-optimized query |
|
|
167
|
+
| **Inputs** | `conversation` (CHAT_CONVERSATION) |
|
|
168
|
+
| **Outputs** | `summarized_conversation` (TEXT_WITH_SOURCES) |
|
|
169
|
+
| **Pairs with** | search/v2 (provides search query from conversation context) |
|
|
170
|
+
| **Critical** | Required for multi-turn chat search. Direct CHAT_CONVERSATION → search causes type mismatch. |
|
|
171
|
+
|
|
172
|
+
### Response Validator (`response_validator`)
|
|
173
|
+
|
|
174
|
+
| | |
|
|
175
|
+
| -------------- | ----------------------------------------------------------------------------- |
|
|
176
|
+
| **Purpose** | Validate LLM output against quality/compliance criteria |
|
|
177
|
+
| **Inputs** | `reference_query` (TEXT_WITH_SOURCES), `response_to_validate` (TEXT_WITH_SOURCES) |
|
|
178
|
+
| **Outputs** | `abstain_reason` (TEXT_WITH_SOURCES — reason for rejection, empty if valid) |
|
|
179
|
+
| **Pairs with** | call_llm (validates output), abstain_action (handles rejection) |
|
|
180
|
+
|
|
181
|
+
### Abstain Action (`abstain_action`)
|
|
182
|
+
|
|
183
|
+
| | |
|
|
184
|
+
| -------------- | ----------------------------------------------------------------------------- |
|
|
185
|
+
| **Purpose** | Provide a safe decline response when AI should not answer |
|
|
186
|
+
| **Inputs** | `abstain_reason` (TEXT_WITH_SOURCES — from response_validator) |
|
|
187
|
+
| **Outputs** | `abstain_reason` (TEXT_WITH_SOURCES — decline message) |
|
|
188
|
+
| **Pairs with** | response_validator (receives rejection reason) |
|
|
189
|
+
|
|
190
|
+
### Rule Validation with Documents (`rule_validation_with_documents`)
|
|
191
|
+
|
|
192
|
+
| | |
|
|
193
|
+
| -------------- | ----------------------------------------------------------------------------- |
|
|
194
|
+
| **Purpose** | Check extracted data against business rules (compliance, thresholds) |
|
|
195
|
+
| **Inputs** | `primary_docs` (DOCUMENT), `map_of_extracted_columns` (ANY) |
|
|
196
|
+
| **Outputs** | `ruleset_output` (ANY — validation results) |
|
|
197
|
+
| **Pairs with** | entity_extraction (provides data to check), call_llm (summarizes results) |
|
|
198
|
+
| **Critical** | Rules configured in UI settings panel, not via workflow_def inputs. |
|
|
199
|
+
|
|
200
|
+
### Entity Extraction with Documents (`entity_extraction_with_documents`)
|
|
201
|
+
|
|
202
|
+
| | |
|
|
203
|
+
| -------------- | --------------------------------------------------------------------------------------------------- |
|
|
204
|
+
| **Purpose** | Extract structured entities grounded in provided documents |
|
|
205
|
+
| **Inputs** | `documents` (DOCUMENT), extraction column config |
|
|
206
|
+
| **Outputs** | `extraction_columns` (ANY — structured extraction results, needs intermediary for type-safe wiring) |
|
|
207
|
+
| **Pairs with** | json_mapper (field extraction), rule_validation (compliance), send_email (via intermediary) |
|
|
208
|
+
| **Critical** | Output type is ANY — needs intermediary before send_email inputs |
|
|
209
|
+
|
|
210
|
+
### JSON Mapper (`json_mapper`)
|
|
211
|
+
|
|
212
|
+
| | |
|
|
213
|
+
| -------------- | ------------------------------------------------------------------------------------------- |
|
|
214
|
+
| **Purpose** | Extract specific fields from JSON/structured data into individual outputs |
|
|
215
|
+
| **Inputs** | `input_json` (ANY), mapping rules |
|
|
216
|
+
| **Outputs** | `output_json` per mapped field |
|
|
217
|
+
| **Pairs with** | entity_extraction (field extraction), fixed_response (type conversion to TEXT_WITH_SOURCES) |
|
|
218
|
+
|
|
219
|
+
---
|
|
220
|
+
|
|
221
|
+
## Entity Extraction Best Practices
|
|
222
|
+
|
|
223
|
+
Ranked by reliability:
|
|
224
|
+
|
|
225
|
+
### 1. json_mapper with Schema (Best for known fields)
|
|
226
|
+
|
|
227
|
+
One-step extraction when you know the exact fields and types.
|
|
228
|
+
|
|
229
|
+
```
|
|
230
|
+
entity_extraction → json_mapper(rules=[{fieldName: "email", path: "email_address"}])
|
|
231
|
+
```
|
|
232
|
+
|
|
233
|
+
### 2. entity_extraction_with_documents (Best for grounded extraction)
|
|
234
|
+
|
|
235
|
+
When extracted values MUST come from provided documents (not hallucinated).
|
|
236
|
+
|
|
237
|
+
```
|
|
238
|
+
document_trigger → entity_extraction_with_documents(extraction_columns=[...])
|
|
239
|
+
```
|
|
240
|
+
|
|
241
|
+
### 3. Two-step: Extract then Normalize
|
|
242
|
+
|
|
243
|
+
When you need high recall first, then strict formatting.
|
|
244
|
+
|
|
245
|
+
```
|
|
246
|
+
call_llm(extract) → json_mapper(normalize dates, format phones, dedupe)
|
|
247
|
+
```
|
|
248
|
+
|
|
249
|
+
### 4. call_llm with Extraction Prompt (Most flexible)
|
|
250
|
+
|
|
251
|
+
When extraction logic is complex or requires reasoning.
|
|
252
|
+
|
|
253
|
+
```
|
|
254
|
+
call_llm(prompt="Extract the patient email from this message")
|
|
255
|
+
```
|
|
256
|
+
|
|
257
|
+
**Validation rule**: For critical fields (emails, phone numbers, order IDs), add deterministic
|
|
258
|
+
validation as a separate step — regex/rules via `rule_validation_with_documents` or `response_validator`.
|
|
259
|
+
|
|
260
|
+
**Scoping rule**: Pass only the minimum context needed:
|
|
261
|
+
|
|
262
|
+
- Extracting from current message? Use `trigger.user_query` only
|
|
263
|
+
- Extracting from documents? Use search results or uploaded docs
|
|
264
|
+
- Entity might span multiple turns? Include conversation history
|
|
265
|
+
|
|
266
|
+
---
|
|
267
|
+
|
|
268
|
+
## Common Wiring Patterns
|
|
269
|
+
|
|
270
|
+
### Intent Routing with Shared Search
|
|
271
|
+
|
|
272
|
+
Search runs once, results shared across all gated respond nodes:
|
|
273
|
+
|
|
274
|
+
```
|
|
275
|
+
chat_trigger ─┬─→ chat_categorizer
|
|
276
|
+
│
|
|
277
|
+
└─→ search ─→ [results shared to all respond nodes]
|
|
278
|
+
├─→ respond_A (runIf: category==A) ─→ WORKFLOW_OUTPUT
|
|
279
|
+
├─→ respond_B (runIf: category==B) ─→ WORKFLOW_OUTPUT
|
|
280
|
+
└─→ fallback (runIf: category==Fallback) ─→ WORKFLOW_OUTPUT
|
|
281
|
+
```
|
|
282
|
+
|
|
283
|
+
**Critical**: ALL respond branches must wire to WORKFLOW_OUTPUT.
|
|
284
|
+
|
|
285
|
+
### Consolidated Intent Response
|
|
286
|
+
|
|
287
|
+
Single respond node handles all intents, reducing duplication:
|
|
288
|
+
|
|
289
|
+
```
|
|
290
|
+
chat_trigger ─┬─→ chat_categorizer ─→ call_llm.named_inputs_Intent
|
|
291
|
+
├─→ search ─→ call_llm.named_inputs_Search_Results
|
|
292
|
+
└─→ call_llm.query
|
|
293
|
+
call_llm.response ─→ WORKFLOW_OUTPUT (runIf: != Fallback)
|
|
294
|
+
fallback ─→ WORKFLOW_OUTPUT (runIf: == Fallback)
|
|
295
|
+
```
|
|
296
|
+
|
|
297
|
+
**Use when**: All intents share similar prompts, search sources, and safety constraints.
|
|
298
|
+
|
|
299
|
+
### Externalized Instructions via KB
|
|
300
|
+
|
|
301
|
+
Instructions live in uploaded documents, not hardcoded prompts:
|
|
302
|
+
|
|
303
|
+
```
|
|
304
|
+
Upload: "Scheduling Policy.pdf", "Tone Guide.pdf" to KB
|
|
305
|
+
↓
|
|
306
|
+
chat_trigger → conversation_to_search_query → search → respond_with_sources → WORKFLOW_OUTPUT
|
|
307
|
+
(converts CHAT_CONVERSATION (retrieves instructions (grounded response
|
|
308
|
+
to TEXT_WITH_SOURCES query) + knowledge docs) with citations)
|
|
309
|
+
```
|
|
310
|
+
|
|
311
|
+
**Benefit**: Change behavior by updating documents, not workflow structure.
|
|
312
|
+
|
|
313
|
+
---
|
|
314
|
+
|
|
315
|
+
## Extraction Chain: entity_extraction → json_mapper → fixed_response
|
|
316
|
+
|
|
317
|
+
When extracting structured data from documents and using it downstream (e.g., sending an email), nodes must be chained in a specific order due to type constraints.
|
|
318
|
+
|
|
319
|
+
### The Chain
|
|
320
|
+
|
|
321
|
+
```
|
|
322
|
+
entity_extraction_with_documents → json_mapper → fixed_response → send_email_agent
|
|
323
|
+
(ANY output) (field decomposition) (type conversion) (TEXT_WITH_SOURCES inputs)
|
|
324
|
+
```
|
|
325
|
+
|
|
326
|
+
**Why this order?**
|
|
327
|
+
|
|
328
|
+
1. **entity_extraction** outputs `ANY` type (structured JSON) -- cannot be wired directly to send_email inputs which expect `TEXT_WITH_SOURCES`
|
|
329
|
+
2. **json_mapper** decomposes the structured JSON into individual fields -- still `ANY` type per field
|
|
330
|
+
3. **fixed_response** converts each field to `TEXT_WITH_SOURCES` using template variables (`{{to}}`, `{{subject}}`)
|
|
331
|
+
4. **send_email_agent** receives properly typed `TEXT_WITH_SOURCES` inputs
|
|
332
|
+
|
|
333
|
+
### Wiring Detail
|
|
334
|
+
|
|
335
|
+
```
|
|
336
|
+
entity_extraction.extraction_columns → json_mapper.input_json
|
|
337
|
+
json_mapper.output_json → fixed_response_to.named_inputs_Extracted_Data (template: "{{to}}")
|
|
338
|
+
json_mapper.output_json → fixed_response_subj.named_inputs_Extracted_Data (template: "{{subject}}")
|
|
339
|
+
json_mapper.output_json → fixed_response_body.named_inputs_Extracted_Data (template: "{{body}}")
|
|
340
|
+
fixed_response_to.response → send_email_agent.email_to
|
|
341
|
+
fixed_response_subj.response → send_email_agent.email_subject
|
|
342
|
+
fixed_response_body.response → send_email_agent.email_body
|
|
343
|
+
```
|
|
344
|
+
|
|
345
|
+
You need one `fixed_response` per email field (to, subject, body).
|
|
346
|
+
|
|
347
|
+
### When to Use json_mapper vs entity_extraction
|
|
348
|
+
|
|
349
|
+
| Scenario | Use | Why |
|
|
350
|
+
|---|---|---|
|
|
351
|
+
| Extract from uploaded documents | `entity_extraction_with_documents` | Grounded in source documents, structured schema |
|
|
352
|
+
| Decompose structured JSON into fields | `json_mapper` | Field-level access from ANY-typed data |
|
|
353
|
+
| Extract from conversation text | `call_llm` with extraction prompt | Flexible, handles reasoning |
|
|
354
|
+
| Format fields for typed inputs | `fixed_response` with template | Converts ANY → TEXT_WITH_SOURCES |
|
|
355
|
+
|
|
356
|
+
### Alternative: custom_agent → json_mapper
|
|
357
|
+
|
|
358
|
+
When using `custom_agent` instead of `entity_extraction`, you **must** configure `output_fields` or use strict JSON-only prompting. Without this, `custom_agent` returns JSON as a string blob in `response_with_sources`, and `json_mapper` fails to parse it.
|
|
359
|
+
|
|
360
|
+
**Recommended**: Define `output_fields` on custom_agent (same extraction column format as entity_extraction).
|
|
361
|
+
|
|
362
|
+
See `ema://rules/json-output-patterns` for the full custom_agent + json_mapper pattern.
|
|
363
|
+
|
|
364
|
+
---
|
|
365
|
+
|
|
366
|
+
## Search Configuration
|
|
367
|
+
|
|
368
|
+
### search/v2 vs search/v0
|
|
369
|
+
|
|
370
|
+
Always use **search/v2** (not deprecated search/v0). The key difference is `datastore_configs` input.
|
|
371
|
+
|
|
372
|
+
### datastore_configs Wiring
|
|
373
|
+
|
|
374
|
+
The `datastore_configs` input connects search to the persona's uploaded data sources via widget configuration:
|
|
375
|
+
|
|
376
|
+
```json
|
|
377
|
+
"datastore_configs": {
|
|
378
|
+
"multiBinding": {
|
|
379
|
+
"elements": [{
|
|
380
|
+
"widgetConfig": { "widgetName": "fileUpload" }
|
|
381
|
+
}]
|
|
382
|
+
}
|
|
383
|
+
}
|
|
384
|
+
```
|
|
385
|
+
|
|
386
|
+
- `widgetName` maps to the persona widget that holds data sources
|
|
387
|
+
- Default widget: `fileUpload` (standard KB upload widget)
|
|
388
|
+
- For personas with multiple upload widgets: use the specific widget name (e.g., `v822`, `upload1`)
|
|
389
|
+
|
|
390
|
+
### Search Input Rules
|
|
391
|
+
|
|
392
|
+
| Input Source | When to Use | Why |
|
|
393
|
+
|---|---|---|
|
|
394
|
+
| `trigger.user_query` | Simple single-turn search | Direct text query |
|
|
395
|
+
| `conversation_to_search_query.summarized_conversation` | Multi-turn chat | Converts conversation history to search-optimized query |
|
|
396
|
+
| `trigger.chat_conversation` | **Never for search** | Wrong type (CHAT_CONVERSATION vs TEXT_WITH_SOURCES) |
|
|
397
|
+
|
|
398
|
+
### Multi-Source Search
|
|
399
|
+
|
|
400
|
+
Combine local KB search with web search using `combine_search_results`:
|
|
401
|
+
|
|
402
|
+
```
|
|
403
|
+
conversation_to_search_query ─┬─→ search (local KB)
|
|
404
|
+
└─→ live_web_search (real-time web)
|
|
405
|
+
├─→ combine_search_results.search_results_1
|
|
406
|
+
└─→ combine_search_results.search_results_2
|
|
407
|
+
→ respond_with_sources.search_results
|
|
408
|
+
```
|
|
409
|
+
|
|
410
|
+
### Critical: Upload Data Before Deploying Search Workflows
|
|
411
|
+
|
|
412
|
+
Search returns empty results without uploaded documents. Always:
|
|
413
|
+
1. Upload: `persona(id='...', data={method:'upload', path:'/path/to/doc.pdf'})`
|
|
414
|
+
2. Verify: `persona(id='...', data={method:'stats'})` → check `success` count > 0
|
|
415
|
+
3. Then deploy the workflow
|
|
416
|
+
|
|
417
|
+
---
|
|
418
|
+
|
|
419
|
+
## LLM Node Configuration (call_llm)
|
|
420
|
+
|
|
421
|
+
### named_inputs Convention
|
|
422
|
+
|
|
423
|
+
`call_llm` accepts additional context via `named_inputs` using the suffix pattern `named_inputs_<Descriptive_Name>`:
|
|
424
|
+
|
|
425
|
+
| Named Input | Type | Purpose |
|
|
426
|
+
|---|---|---|
|
|
427
|
+
| `named_inputs_Search_Results` | SEARCH_RESULT | KB search results for RAG |
|
|
428
|
+
| `named_inputs_Conversation` | CHAT_CONVERSATION | Full conversation history |
|
|
429
|
+
| `named_inputs_Intent` | ENUM | Detected category from categorizer |
|
|
430
|
+
| `named_inputs_Current_Message` | TEXT_WITH_SOURCES | Current user message for extraction |
|
|
431
|
+
| `named_inputs_Tool_Result` | TEXT_WITH_SOURCES | External action output |
|
|
432
|
+
|
|
433
|
+
`named_inputs` accepts **ANY** type -- this is how you pass CHAT_CONVERSATION and SEARCH_RESULT into LLM nodes.
|
|
434
|
+
|
|
435
|
+
### Temperature Guidelines
|
|
436
|
+
|
|
437
|
+
| Use Case | Temperature | Why |
|
|
438
|
+
|---|---|---|
|
|
439
|
+
| Document generation | 0.3-0.5 | Consistent formatting, predictable output |
|
|
440
|
+
| Entity extraction | 0.0-0.3 | Accuracy over creativity |
|
|
441
|
+
| General Q&A / chat | 0.5-0.7 | Balanced creativity and accuracy |
|
|
442
|
+
| Creative writing | 0.7-1.0 | More varied, creative output |
|
|
443
|
+
|
|
444
|
+
### Avoiding Duplicate LLM Nodes
|
|
445
|
+
|
|
446
|
+
If multiple `call_llm` nodes share the same inputs (query, search_results, conversation) and differ only by `trigger_when` gate, **consolidate** them:
|
|
447
|
+
|
|
448
|
+
1. Remove duplicate respond nodes
|
|
449
|
+
2. Create one `call_llm` that always runs (or runs when not Fallback)
|
|
450
|
+
3. Wire `categorizer.category → call_llm.named_inputs_Intent`
|
|
451
|
+
4. Update prompt: "Based on the detected intent ({{Intent}}), respond accordingly"
|
|
452
|
+
5. Wire single `call_llm.response_with_sources → WORKFLOW_OUTPUT`
|
|
453
|
+
|
|
454
|
+
Keep separate nodes **only** when intents require different tools, search sources, or safety constraints.
|
|
455
|
+
|
|
456
|
+
### Scoping Extraction Inputs
|
|
457
|
+
|
|
458
|
+
When using `call_llm` for entity extraction, scope inputs tightly:
|
|
459
|
+
|
|
460
|
+
- Extracting from current message? → `trigger.user_query` only
|
|
461
|
+
- Extracting from documents? → `search.search_results` (intentional)
|
|
462
|
+
- Entity might span multiple turns? → Include `chat_conversation`
|
|
463
|
+
|
|
464
|
+
Passing full search_results to extraction prompts causes hallucinated entities from irrelevant context.
|
|
465
|
+
|
|
466
|
+
---
|
|
467
|
+
|
|
468
|
+
## Categorizer Patterns
|
|
469
|
+
|
|
470
|
+
### Basic Rules
|
|
471
|
+
|
|
472
|
+
- **Input**: `chat_categorizer` MUST receive `chat_conversation` (not `user_query`) for accurate multi-turn classification
|
|
473
|
+
- **Fallback**: ALWAYS include a Fallback category
|
|
474
|
+
- **Handlers**: Every category must have at least one node with a matching `runIf` condition
|
|
475
|
+
- **typeArguments**: Categorizer must have `typeArguments.categories` pointing to the enum type -- empty `typeArguments` causes deploy failure
|
|
476
|
+
|
|
477
|
+
### runIf Condition Format
|
|
478
|
+
|
|
479
|
+
```json
|
|
480
|
+
{
|
|
481
|
+
"lhs": {
|
|
482
|
+
"actionOutput": { "actionName": "chat_categorizer", "output": "category" },
|
|
483
|
+
"autoDetectedBinding": false
|
|
484
|
+
},
|
|
485
|
+
"operator": 1,
|
|
486
|
+
"rhs": {
|
|
487
|
+
"inline": { "enumValue": "Market_Impact" },
|
|
488
|
+
"autoDetectedBinding": false
|
|
489
|
+
}
|
|
490
|
+
}
|
|
491
|
+
```
|
|
492
|
+
|
|
493
|
+
**Important**: Use `category` as the output name and compare to `enumValue` directly. Do NOT use `category_<Name>` format.
|
|
494
|
+
|
|
495
|
+
### runIf Operator Values
|
|
496
|
+
|
|
497
|
+
| Operator | Meaning | Use Case |
|
|
498
|
+
|---|---|---|
|
|
499
|
+
| `1` | Equals (`==`) | Route to handler when category matches |
|
|
500
|
+
| `2` | Not equals (`!=`) | Run for all categories except one (e.g., `!= Fallback`) |
|
|
501
|
+
|
|
502
|
+
For OR conditions (e.g., "run for Sales OR General"), use operator `2` with Fallback: `category != Fallback` -- this runs the node for all non-fallback categories.
|
|
503
|
+
|
|
504
|
+
### Defining enumTypes
|
|
505
|
+
|
|
506
|
+
Categories must be defined in the `workflow_def.enumTypes` array:
|
|
507
|
+
|
|
508
|
+
```json
|
|
509
|
+
"enumTypes": [{
|
|
510
|
+
"name": { "name": "intent_categories", "namespaces": [] },
|
|
511
|
+
"options": [
|
|
512
|
+
{ "name": "Sales", "description": "Sales inquiries and product questions" },
|
|
513
|
+
{ "name": "Support", "description": "Technical support and ticket creation" },
|
|
514
|
+
{ "name": "General", "description": "General questions" },
|
|
515
|
+
{ "name": "Fallback", "description": "Unclear or unmatched intents" }
|
|
516
|
+
]
|
|
517
|
+
}]
|
|
518
|
+
```
|
|
519
|
+
|
|
520
|
+
Then the categorizer's `typeArguments` references this enum:
|
|
521
|
+
|
|
522
|
+
```json
|
|
523
|
+
"typeArguments": {
|
|
524
|
+
"categories": {
|
|
525
|
+
"enumType": { "name": { "name": "intent_categories", "namespaces": [] } },
|
|
526
|
+
"isList": false
|
|
527
|
+
}
|
|
528
|
+
}
|
|
529
|
+
```
|
|
530
|
+
|
|
531
|
+
### text_categorizer vs chat_categorizer
|
|
532
|
+
|
|
533
|
+
| Categorizer | Input Type | When to Use |
|
|
534
|
+
|---|---|---|
|
|
535
|
+
| `chat_categorizer` | CHAT_CONVERSATION | Routing conversations (most common) |
|
|
536
|
+
| `text_categorizer/v1` | named_inputs (multiBinding) | Routing based on text content, not conversation |
|
|
537
|
+
|
|
538
|
+
`text_categorizer/v0` is deprecated -- use v1 with `named_inputs`.
|
|
539
|
+
|
|
540
|
+
### Nested Categorizers
|
|
541
|
+
|
|
542
|
+
For complex routing with sub-intents, chain categorizers:
|
|
543
|
+
|
|
544
|
+
```
|
|
545
|
+
chat_categorizer (Level 1: HR, IT, General, Fallback)
|
|
546
|
+
└─→ text_categorizer (Level 2 for HR: Benefits, Leave, Payroll, Fallback)
|
|
547
|
+
```
|
|
548
|
+
|
|
549
|
+
Use clear precedence -- first categorizer routes broadly, second categorizer handles sub-intent.
|
|
550
|
+
|
|
551
|
+
---
|
|
552
|
+
|
|
553
|
+
## Error Handling & Validation Patterns
|
|
554
|
+
|
|
555
|
+
### response_validator + abstain_action
|
|
556
|
+
|
|
557
|
+
For guardrails on LLM output:
|
|
558
|
+
|
|
559
|
+
```
|
|
560
|
+
call_llm.response → response_validator ─→ abstain_action (if validation fails)
|
|
561
|
+
└─→ WORKFLOW_OUTPUT (if validation passes)
|
|
562
|
+
```
|
|
563
|
+
|
|
564
|
+
- `response_validator` checks generated response against reference query for quality/compliance
|
|
565
|
+
- `abstain_action` provides a safe decline message when AI should not answer
|
|
566
|
+
|
|
567
|
+
### Graceful Degradation
|
|
568
|
+
|
|
569
|
+
When search returns no results or external APIs fail:
|
|
570
|
+
|
|
571
|
+
```
|
|
572
|
+
search → call_llm (runIf: search has results)
|
|
573
|
+
fixed_response (runIf: search empty) → "I don't have enough information to answer that."
|
|
574
|
+
```
|
|
575
|
+
|
|
576
|
+
### Choosing the Right Validator
|
|
577
|
+
|
|
578
|
+
| Validator | Input Type | Use Case |
|
|
579
|
+
|---|---|---|
|
|
580
|
+
| `response_validator` | TEXT_WITH_SOURCES | Validate LLM-generated text (quality, compliance, hallucination) |
|
|
581
|
+
| `rule_validation_with_documents` | ANY | Validate extracted data against business rules (thresholds, required fields) |
|
|
582
|
+
|
|
583
|
+
For **extraction workflows**, use `rule_validation_with_documents` since it accepts ANY-typed extraction output directly. `response_validator` only works with TEXT_WITH_SOURCES (LLM responses), so you'd need the full intermediary chain (json_mapper → fixed_response) before it.
|
|
584
|
+
|
|
585
|
+
### rule_validation_with_documents
|
|
586
|
+
|
|
587
|
+
For business rule compliance checking:
|
|
588
|
+
|
|
589
|
+
```
|
|
590
|
+
entity_extraction → rule_validation_with_documents → call_llm (summarize results)
|
|
591
|
+
```
|
|
592
|
+
|
|
593
|
+
Rules are configured in the UI settings panel, not via `workflow_def` inputs.
|
|
594
|
+
|
|
595
|
+
---
|
|
596
|
+
|
|
597
|
+
## HITL (Human-in-the-Loop) Patterns
|
|
598
|
+
|
|
599
|
+
### How HITL Works
|
|
600
|
+
|
|
601
|
+
HITL is a **flag on specific action nodes**, NOT a standalone workflow node.
|
|
602
|
+
|
|
603
|
+
```json
|
|
604
|
+
{
|
|
605
|
+
"name": "send_email",
|
|
606
|
+
"action": { "name": "send_email_agent" },
|
|
607
|
+
"disable_human_interaction": false
|
|
608
|
+
}
|
|
609
|
+
```
|
|
610
|
+
|
|
611
|
+
**Counter-intuitive naming:**
|
|
612
|
+
- `disable_human_interaction: false` → HITL **ON** (requires approval)
|
|
613
|
+
- `disable_human_interaction: true` → HITL **OFF** (auto-proceeds)
|
|
614
|
+
- Omitted → defaults to false (HITL ON)
|
|
615
|
+
|
|
616
|
+
### When to Enable HITL
|
|
617
|
+
|
|
618
|
+
| Action | Enable HITL? | Why |
|
|
619
|
+
|---|---|---|
|
|
620
|
+
| `send_email_agent` | Yes, if user requests approval | External communication |
|
|
621
|
+
| `external_action_caller` | Yes, for destructive operations | Side effects (create ticket, update CRM) |
|
|
622
|
+
| `call_llm` | Usually no | No external side effects |
|
|
623
|
+
| `search` | No | Read-only operation |
|
|
624
|
+
|
|
625
|
+
### Legacy general_hitl Nodes
|
|
626
|
+
|
|
627
|
+
Existing workflows may have standalone `general_hitl` nodes -- these still function, but **do not add new ones**. Use the flag pattern instead.
|
|
628
|
+
|
|
629
|
+
---
|
|
630
|
+
|
|
631
|
+
## Dashboard Output Columns
|
|
632
|
+
|
|
633
|
+
For dashboard personas, every workflow output mapped in `resultMappings` becomes a column in the dashboard UI.
|
|
634
|
+
|
|
635
|
+
### resultMappings Example
|
|
636
|
+
|
|
637
|
+
```json
|
|
638
|
+
"results": {
|
|
639
|
+
"entity_extraction.recipient_email": {
|
|
640
|
+
"actionName": "entity_extraction",
|
|
641
|
+
"outputName": "recipient_email"
|
|
642
|
+
},
|
|
643
|
+
"entity_extraction.subject_line": {
|
|
644
|
+
"actionName": "entity_extraction",
|
|
645
|
+
"outputName": "subject_line"
|
|
646
|
+
},
|
|
647
|
+
"entity_extraction.amount": {
|
|
648
|
+
"actionName": "entity_extraction",
|
|
649
|
+
"outputName": "amount"
|
|
650
|
+
}
|
|
651
|
+
}
|
|
652
|
+
```
|
|
653
|
+
|
|
654
|
+
Each entry creates one visible dashboard column. The key (e.g., `entity_extraction.recipient_email`) becomes the column display name.
|
|
655
|
+
|
|
656
|
+
### Key Rules
|
|
657
|
+
|
|
658
|
+
- **Missing mapping = invisible column**: If you forget to map an output, it won't appear in the dashboard
|
|
659
|
+
- **Column ordering**: Determined by the order of entries in `results` JSON. To reorder: remove all entries, re-add in desired order
|
|
660
|
+
- **Input columns**: Come from `document_trigger` -- `document_content` for files, `row_data` for text/number inputs
|
|
661
|
+
|
|
662
|
+
### Wiring document_trigger to entity_extraction
|
|
663
|
+
|
|
664
|
+
```
|
|
665
|
+
document_trigger.document_content → entity_extraction_with_documents.documents
|
|
666
|
+
```
|
|
667
|
+
|
|
668
|
+
For additional column data from the dashboard row:
|
|
669
|
+
```
|
|
670
|
+
document_trigger.row_data → call_llm.named_inputs_Row_Data
|
|
671
|
+
```
|
|
672
|
+
|
|
673
|
+
See `.context/public/guides/dashboard-operations.md` for full dashboard column details including nested structures.
|
|
674
|
+
|
|
675
|
+
## Extraction Column Groups (Nested Structures)
|
|
676
|
+
|
|
677
|
+
Entity extraction supports grouped/nested columns using `dataType: 5` (OBJECT). A group column has sub-columns defined in `value.objectValue.values`. This appears in the UI as a parent with expandable children.
|
|
678
|
+
|
|
679
|
+
See `ema://rules/extraction-column-format` for the full API shape including nested examples.
|
|
680
|
+
|
|
681
|
+
## Anti-Pattern Quick Reference
|
|
682
|
+
|
|
683
|
+
| Anti-Pattern | Symptom | Fix |
|
|
684
|
+
| -------------------------------- | ---------------------------------------------------- | ---------------------------------------------------------- |
|
|
685
|
+
| Partial output wiring | Some intents return no response | Wire ALL gated respond nodes to WORKFLOW_OUTPUT |
|
|
686
|
+
| Duplicate identical LLM nodes | Multiple nodes with same inputs, different gates | Consolidate to one node + category as named_input |
|
|
687
|
+
| Overscoped extraction | Extractions hallucinate entities from search results | Scope to `user_query` only |
|
|
688
|
+
| Hardcoded instructions | Must redeploy workflow to change behavior | Upload instructions as KB documents |
|
|
689
|
+
| Missing fallback | Unrecognized intents silently fail | Always include Fallback category |
|
|
690
|
+
| Text content as email recipient | Email sends fail or go to garbage addresses | Use entity_extraction → json_mapper → fixed_response chain |
|
|
691
|
+
| Missing dashboard output columns | Node output not visible in dashboard | Add resultMapping entry for every desired output column |
|
|
692
|
+
| Wrong column order in dashboard | Columns appear in unexpected order | Remove and re-add resultMappings in desired order |
|
|
693
|
+
| entity_extraction direct to email | Type mismatch (ANY vs TEXT_WITH_SOURCES) | Use json_mapper + fixed_response intermediary chain |
|
|
694
|
+
| json_mapper without upstream data | json_mapper receives no structured input | Wire from entity_extraction, custom_agent, or other JSON source |
|
|
695
|
+
| Search without uploaded data | Search returns empty results | Upload documents before deploying search workflows |
|
|
696
|
+
| Redundant search nodes | Multiple searches with same query | Single search node, share results via named_inputs |
|
|
697
|
+
| Sequential LLM calls | Unnecessary latency and incoherence | Single call_llm with comprehensive instructions |
|
|
698
|
+
| Categorizer without typeArguments | Deploy failure | Add typeArguments.categories pointing to enumType |
|
|
699
|
+
|
|
700
|
+
---
|
|
701
|
+
|
|
702
|
+
## References
|
|
703
|
+
|
|
704
|
+
- `src/mcp/domain/validation-rules.ts` — Anti-pattern definitions
|
|
705
|
+
- `src/mcp/domain/structural-rules.ts` — Structural invariants
|
|
706
|
+
- `src/mcp/knowledge.ts` — Workflow patterns and agent catalog
|
|
707
|
+
- `.context/public/guides/email-patterns.md` — Email wiring patterns
|
|
708
|
+
- `.context/public/guides/dashboard-operations.md` — Dashboard-specific patterns
|