fastworkflow 3.1.2__tar.gz → 3.2.0__tar.gz

{fastworkflow-3.1.2 → fastworkflow-3.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: fastworkflow
-Version: 3.1.2
+Version: 3.2.0
 Summary: A framework for rapidly building large-scale, deterministic, interactive workflows with a fault-tolerant, conversational UX
 License: Apache-2.0
 Keywords: fastworkflow,ai,workflow,llm,openai
@@ -167,6 +167,7 @@ fastWorkflow was benchmarked on [Tau Bench](https://github.com/sierra-research/t
 - [Quick Start: run an example in 5 minutes](#quick-start-run-an-example-in-5-minutes)
 - [AI-enable your own app (without restructuring it)](#ai-enable-your-own-app-without-restructuring-it)
 - [How complex workflows scale: context hierarchies](#how-complex-workflows-scale-context-hierarchies)
+- [Chat with it and debug it: `run_chatbot`](#chat-with-it-and-debug-it-run_chatbot)
 - [Production deployment](#production-deployment)
 - [Developer FAQ](#developer-faq)
 - [Key concepts (going deeper)](#key-concepts-going-deeper)
@@ -238,10 +239,10 @@ class OrderService:
 
 ### Recommended: let a coding agent wrap it for you
 
-The fastest path for a non-trivial app is the **[integrate-chat-agent](./fastworkflow/docs/integrate-chat-agent) skill** with Cursor or Claude Code:
+The fastest path for a non-trivial app is the **[integrate-chat-agent](./fastworkflow/skills_for_coding_fastworkflows/integrate-chat-agent) skill** with Cursor or Claude Code:
 
 ```text
-Open fastworkflow/docs/integrate-chat-agent/SKILL.md
+Open fastworkflow/skills_for_coding_fastworkflows/integrate-chat-agent/SKILL.md
 Prompt: "Integrate a fastWorkflow chat agent for OrderService in orders.py"
 ```
 
@@ -352,6 +353,88 @@ This is what lets small models stay accurate as your app grows — and what keep
 
 ---
 
+## Chat with it and debug it: `run_chatbot`
+
+When a turn does the wrong thing, the question is almost always *which stage* got it wrong: did intent detection pick the wrong command, did parameter extraction mis-fill an argument, or did the command itself fail? `run_chatbot` answers that without a debugger.
+
+Every run records what it did to a per-workflow SQLite database, `observability.sqlite3`, under `FASTWORKFLOW_STATE_ROOT/workflows/<workflow-id>/`. `run_chatbot` is a local browser UI over the workflow: a chat client for driving new turns, and a trace viewer for everything that already happened.
+
+```sh
+fastworkflow run_chatbot
+```
+
+One command opens the workflow picker, then starts the selected workflow's FastAPI server for you (loopback-only, stopped when the chatbot exits), picks a free port, and connects the chat — no server URL, token, channel, workflow path, or env-file CLI arguments to fill in. Env files are auto-detected in the workflow directory and, for bundled workflows, the shared `examples/` directory. If either is missing, choose existing files in the browser or create owner-only workflow-local copies from the bundled templates.
+
+```
+fastWorkflow Chatbot
+  pick a workflow in the browser (bundled examples
+  and local folders are listed; you can browse anywhere).
+
+  Open in your browser:
+
+    http://127.0.0.1:8901/?token=Bg4-uLpLdETJAsYbGzXbfRQdAEhitS6p8podPXmK25g
+```
+
+Use that printed URL. It carries a one-off token, and requests without it are refused — so one you leave running is not readable by anything else on the machine. Pass `--server-port` to skip spawning and use a FastAPI server already running on that port. The page opens with a workflow picker: bundled examples plus a directory browser, with a **Switch workflow** button to change later.
+
+### Debug mode: what a turn actually did
+
+Debug mode nests turns directly beneath their conversations (with a separate conversation-less group when needed), and opens any turn into a span tree — the nested record of the stages that ran, each with its duration, status, and attributes:
+
+```
+fw.turn                        the whole logical turn (stable across ask_user suspensions)
+├── fw.planner.plan            the agent's plan for the turn (fw.planner.replan on a re-plan)
+│   └── fw.llm.call            the planner's LLM request/response
+└── fw.agent.execute           the ReAct loop as a whole (attempts, final answer)
+    └── fw.agent.step          one reasoning step: thought, chosen tool, observation
+        ├── fw.llm.call        the step's reasoning LLM call (+ provider-native reasoning)
+        └── fw.agent.tool_call the agent invoking a command
+            └── fw.command.execute   resolution + your business logic running
+                ├── fw.nlu.intent            which matching layer decided, classifier
+                │                            confidence vs threshold, candidates on ambiguity
+                └── fw.nlu.param_extraction  extraction method, missing/invalid fields,
+                    │                        db_lookup outcomes, validation-hook verdict
+                    └── fw.llm.call          the LLM extraction call, when one ran
+fw.ask_user                    a clarifying question, and how long the user took
+
+(Deterministic "/"-mode turns skip the planner/agent layers: fw.command.execute
+sits directly under fw.turn.)
+```
+
+The diagnosis lives in the attributes of `fw.command.execute`. It records `raw_command` (the natural-language command the agent sent), `command_name` (what intent detection *resolved* it to), the extracted `parameters`, the `response_text`, and `success`. Comparing the first two is the whole trick:
+
+- **`raw_command` was reasonable but `command_name` is the wrong command** → an intent-detection miss. Add seed utterances for that command and retrain.
+- **`command_name` is right but `parameters` are wrong** → a parameter-extraction miss. Strengthen the `Field(description=…, examples=[…])` in that command's Signature.
+- **Both right, span status is `error`** → the bug is in your own code, and `response_text` usually says how.
+
+Two turn-level fields are worth knowing because they are deliberately independent. `status` is the turn's lifecycle (`completed`, `failed`, …) and `success` means every command in it succeeded. A turn that is `completed` but not `success` is the case worth hunting: a command failed and the agent wrote a confident answer over it.
+
+Coding agents can run this diagnosis directly against the database: the wheel-shipped [`debug-workflow-conversations`](fastworkflow/skills_for_coding_fastworkflows/debug-workflow-conversations/SKILL.md) skill carries the failure-triage decision tree, the full schema and span-attribute contract, and the routing table from each diagnosis to the fastWorkflow feature (and companion skill) that fixes it.
+
+The **Health** view reports the background writer's state and the database size, which is how you tell "no spans recorded" apart from "spans dropped under load."
+
+### Chat: drive the workflow from the browser
+
+The Chat tab is connected the moment the workflow's server is ready (the first start loads models, so give it a moment — the page tells you what it is waiting on). Every turn is recorded like any other, so you can send a message and immediately open its span tree via its **view trace** link or the Debug tab.
+
+Messages go to `/invoke_agent`; prefix one with `/` to force deterministic execution via `/invoke_assistant`, the same convention as the CLI prompt. **New conversation** in the header archives the current thread and starts a fresh one, like the CLI's `//new`. Sessions are single-developer by design: the chatbot manages one fixed private channel (`chatbot`), so there is nothing multi-user to configure and your history is in one place across launches.
+
+There is deliberately no box for a startup command or a per-session context: the session is driven by what you type, so pre-seeding it from a chat form makes no sense. Those stay launch-time decisions — `run_fastapi_mcp --startup_command`, or the `/initialize` request fields if you are calling the API yourself.
+
+A spawned server always binds `127.0.0.1`, has CORS pinned to loopback origins, and is stopped when the chatbot exits (SIGTERM included). It runs with unsigned dev JWTs — fine for a loopback-only dev server whose tokens the chatbot mints itself; pass `--expect-encrypted-jwt` to require signed tokens instead (then paste one in the chat's Advanced panel). Already have a server running? Launch with `--server-port <port>` and connect via the Advanced panel.
+
+Insights distillation (`fastworkflow run --generate_insights`) stays CLI-only by design, as do bind address and env-file choices for externally managed servers. The full capability comparison is in [`docs/run_chatbot_cli_parity.md`](docs/run_chatbot_cli_parity.md).
+
+### Retention and erasure
+
+Trace data grows. Spans and artifacts are pruned beyond a retention horizon; **turn records and conversations are never deleted by a default prune**, so history survives.
+
+Use **Clear conversations** in Debug mode for an explicit, confirmed reset. It removes conversation labels, turns, spans, artifacts, feedback, and legacy per-channel conversation files. Training runs, writer diagnostics, and monotonic conversation counters survive, so identities are never reused after a clear.
+
+To turn recording off entirely, set `FW_OBSERVABILITY=0`.
+
+---
+
 ## Production deployment
 
 ### Pattern 1 — host it as a FastAPI service (recommended)
@@ -488,7 +571,7 @@ Deep-dive articles:
 
 ## Architecture overview
 
-fastWorkflow separates **build-time**, **train-time**, and **run-time**. At build-time you create a command interface from your code (recommended via the [integrate-chat-agent](./fastworkflow/docs/integrate-chat-agent) skill). `train` builds the NLP models; `run` executes the workflow. Your existing code is never modified — fastWorkflow sits as a layer on top.
+fastWorkflow separates **build-time**, **train-time**, and **run-time**. At build-time you create a command interface from your code (recommended via the [integrate-chat-agent](./fastworkflow/skills_for_coding_fastworkflows/integrate-chat-agent) skill). `train` builds the NLP models; `run` executes the workflow. Your existing code is never modified — fastWorkflow sits as a layer on top.
 
 ```mermaid
 graph LR
@@ -601,6 +684,12 @@ fastworkflow run <workflow_dir> <env_file> <passwords_file> --assistant   # dete
 fastworkflow run <workflow_dir> <env_file> <passwords_file> \
   --startup_command "your command" --keep_alive False
 
+# Browser chatbot + observability viewer (`run` is also spelled `run_cli`);
+# workflow, env files, and trace maintenance are chosen in the browser
+fastworkflow run_chatbot
+fastworkflow run_chatbot --server-port 8000        # pin the spawned server's port
+fastworkflow run_chatbot --expect-encrypted-jwt    # require signed tokens
+
 # Host as a FastAPI/MCP service
 python -m fastworkflow.run_fastapi_mcp --workflow_path ./wf --port 8000
 ```
@@ -630,6 +719,10 @@ Two files per workflow (templates ship with `fastworkflow examples fetch`).
 | `LITELLM_PROXY_API_BASE` | LiteLLM Proxy URL | with `litellm_proxy/` models | *not set* |
 | `INTENT_DETECTION_TINY_MODEL` | HF id for the small intent model | `train` (optional) | `google/bert_uncased_L-4_H-128_A-2` |
 | `INTENT_DETECTION_LARGE_MODEL` | HF id for the large intent model | `train` (optional) | `distilbert-base-uncased` |
+| `FW_OBSERVABILITY` | Master switch for trace recording. `0` disables it. On by default for `run`/`run_fastapi_mcp`; off by default when embedding the core as a library | Optional | `1` (fastWorkflow entry points) |
+| `FW_OBS_RETENTION_DAYS` | Age beyond which the automatic prune (run at recorder startup) drops spans/artifacts (turn records are exempt) | Optional | `30` |
+| `FW_OBS_DB_MAX_BYTES` | Size cap; the automatic prune evicts oldest spans first while over it | Optional | `1073741824` (1 GiB) |
+| `FW_OBS_CAPTURE_TRACEBACKS` | Persist exception tracebacks as artifacts. Off by default because tracebacks can carry sensitive values | Optional | `0` |
 
 ### `fastworkflow.passwords.env`
 
@@ -668,6 +761,8 @@ When a model uses the `litellm_proxy/` prefix, the per-role keys are ignored and
 
 > **Commands not recognized** — a command module with an import/syntax error won't load and won't appear as an intent. Check your `_commands/*.py` files.
 
+> **The agent did something unexpected** — open the chatbot (`fastworkflow run_chatbot`, then pick the workflow) and read the turn's span tree. It shows which command intent detection chose, what parameters were extracted, and whether your command failed — usually enough to identify the stage at fault before reaching for a debugger.
+
 > [!tip]
 > To debug command files, set up a VSCode `launch.json` with `justMyCode: false`, add breakpoints, and run in debug mode.
 

{fastworkflow-3.1.2 → fastworkflow-3.2.0}/README.md

@@ -113,6 +113,7 @@ fastWorkflow was benchmarked on [Tau Bench](https://github.com/sierra-research/t
 - [Quick Start: run an example in 5 minutes](#quick-start-run-an-example-in-5-minutes)
 - [AI-enable your own app (without restructuring it)](#ai-enable-your-own-app-without-restructuring-it)
 - [How complex workflows scale: context hierarchies](#how-complex-workflows-scale-context-hierarchies)
+- [Chat with it and debug it: `run_chatbot`](#chat-with-it-and-debug-it-run_chatbot)
 - [Production deployment](#production-deployment)
 - [Developer FAQ](#developer-faq)
 - [Key concepts (going deeper)](#key-concepts-going-deeper)
@@ -184,10 +185,10 @@ class OrderService:
 
 ### Recommended: let a coding agent wrap it for you
 
-The fastest path for a non-trivial app is the **[integrate-chat-agent](./fastworkflow/docs/integrate-chat-agent) skill** with Cursor or Claude Code:
+The fastest path for a non-trivial app is the **[integrate-chat-agent](./fastworkflow/skills_for_coding_fastworkflows/integrate-chat-agent) skill** with Cursor or Claude Code:
 
 ```text
-Open fastworkflow/docs/integrate-chat-agent/SKILL.md
+Open fastworkflow/skills_for_coding_fastworkflows/integrate-chat-agent/SKILL.md
 Prompt: "Integrate a fastWorkflow chat agent for OrderService in orders.py"
 ```
 
@@ -298,6 +299,88 @@ This is what lets small models stay accurate as your app grows — and what keep
 
 ---
 
+## Chat with it and debug it: `run_chatbot`
+
+When a turn does the wrong thing, the question is almost always *which stage* got it wrong: did intent detection pick the wrong command, did parameter extraction mis-fill an argument, or did the command itself fail? `run_chatbot` answers that without a debugger.
+
+Every run records what it did to a per-workflow SQLite database, `observability.sqlite3`, under `FASTWORKFLOW_STATE_ROOT/workflows/<workflow-id>/`. `run_chatbot` is a local browser UI over the workflow: a chat client for driving new turns, and a trace viewer for everything that already happened.
+
+```sh
+fastworkflow run_chatbot
+```
+
+One command opens the workflow picker, then starts the selected workflow's FastAPI server for you (loopback-only, stopped when the chatbot exits), picks a free port, and connects the chat — no server URL, token, channel, workflow path, or env-file CLI arguments to fill in. Env files are auto-detected in the workflow directory and, for bundled workflows, the shared `examples/` directory. If either is missing, choose existing files in the browser or create owner-only workflow-local copies from the bundled templates.
+
+```
+fastWorkflow Chatbot
+  pick a workflow in the browser (bundled examples
+  and local folders are listed; you can browse anywhere).
+
+  Open in your browser:
+
+    http://127.0.0.1:8901/?token=Bg4-uLpLdETJAsYbGzXbfRQdAEhitS6p8podPXmK25g
+```
+
+Use that printed URL. It carries a one-off token, and requests without it are refused — so one you leave running is not readable by anything else on the machine. Pass `--server-port` to skip spawning and use a FastAPI server already running on that port. The page opens with a workflow picker: bundled examples plus a directory browser, with a **Switch workflow** button to change later.
+
+### Debug mode: what a turn actually did
+
+Debug mode nests turns directly beneath their conversations (with a separate conversation-less group when needed), and opens any turn into a span tree — the nested record of the stages that ran, each with its duration, status, and attributes:
+
+```
+fw.turn                        the whole logical turn (stable across ask_user suspensions)
+├── fw.planner.plan            the agent's plan for the turn (fw.planner.replan on a re-plan)
+│   └── fw.llm.call            the planner's LLM request/response
+└── fw.agent.execute           the ReAct loop as a whole (attempts, final answer)
+    └── fw.agent.step          one reasoning step: thought, chosen tool, observation
+        ├── fw.llm.call        the step's reasoning LLM call (+ provider-native reasoning)
+        └── fw.agent.tool_call the agent invoking a command
+            └── fw.command.execute   resolution + your business logic running
+                ├── fw.nlu.intent            which matching layer decided, classifier
+                │                            confidence vs threshold, candidates on ambiguity
+                └── fw.nlu.param_extraction  extraction method, missing/invalid fields,
+                    │                        db_lookup outcomes, validation-hook verdict
+                    └── fw.llm.call          the LLM extraction call, when one ran
+fw.ask_user                    a clarifying question, and how long the user took
+
+(Deterministic "/"-mode turns skip the planner/agent layers: fw.command.execute
+sits directly under fw.turn.)
+```
+
+The diagnosis lives in the attributes of `fw.command.execute`. It records `raw_command` (the natural-language command the agent sent), `command_name` (what intent detection *resolved* it to), the extracted `parameters`, the `response_text`, and `success`. Comparing the first two is the whole trick:
+
+- **`raw_command` was reasonable but `command_name` is the wrong command** → an intent-detection miss. Add seed utterances for that command and retrain.
+- **`command_name` is right but `parameters` are wrong** → a parameter-extraction miss. Strengthen the `Field(description=…, examples=[…])` in that command's Signature.
+- **Both right, span status is `error`** → the bug is in your own code, and `response_text` usually says how.
+
+Two turn-level fields are worth knowing because they are deliberately independent. `status` is the turn's lifecycle (`completed`, `failed`, …) and `success` means every command in it succeeded. A turn that is `completed` but not `success` is the case worth hunting: a command failed and the agent wrote a confident answer over it.
+
+Coding agents can run this diagnosis directly against the database: the wheel-shipped [`debug-workflow-conversations`](fastworkflow/skills_for_coding_fastworkflows/debug-workflow-conversations/SKILL.md) skill carries the failure-triage decision tree, the full schema and span-attribute contract, and the routing table from each diagnosis to the fastWorkflow feature (and companion skill) that fixes it.
+
+The **Health** view reports the background writer's state and the database size, which is how you tell "no spans recorded" apart from "spans dropped under load."
+
+### Chat: drive the workflow from the browser
+
+The Chat tab is connected the moment the workflow's server is ready (the first start loads models, so give it a moment — the page tells you what it is waiting on). Every turn is recorded like any other, so you can send a message and immediately open its span tree via its **view trace** link or the Debug tab.
+
+Messages go to `/invoke_agent`; prefix one with `/` to force deterministic execution via `/invoke_assistant`, the same convention as the CLI prompt. **New conversation** in the header archives the current thread and starts a fresh one, like the CLI's `//new`. Sessions are single-developer by design: the chatbot manages one fixed private channel (`chatbot`), so there is nothing multi-user to configure and your history is in one place across launches.
+
+There is deliberately no box for a startup command or a per-session context: the session is driven by what you type, so pre-seeding it from a chat form makes no sense. Those stay launch-time decisions — `run_fastapi_mcp --startup_command`, or the `/initialize` request fields if you are calling the API yourself.
+
+A spawned server always binds `127.0.0.1`, has CORS pinned to loopback origins, and is stopped when the chatbot exits (SIGTERM included). It runs with unsigned dev JWTs — fine for a loopback-only dev server whose tokens the chatbot mints itself; pass `--expect-encrypted-jwt` to require signed tokens instead (then paste one in the chat's Advanced panel). Already have a server running? Launch with `--server-port <port>` and connect via the Advanced panel.
+
+Insights distillation (`fastworkflow run --generate_insights`) stays CLI-only by design, as do bind address and env-file choices for externally managed servers. The full capability comparison is in [`docs/run_chatbot_cli_parity.md`](docs/run_chatbot_cli_parity.md).
+
+### Retention and erasure
+
+Trace data grows. Spans and artifacts are pruned beyond a retention horizon; **turn records and conversations are never deleted by a default prune**, so history survives.
+
+Use **Clear conversations** in Debug mode for an explicit, confirmed reset. It removes conversation labels, turns, spans, artifacts, feedback, and legacy per-channel conversation files. Training runs, writer diagnostics, and monotonic conversation counters survive, so identities are never reused after a clear.
+
+To turn recording off entirely, set `FW_OBSERVABILITY=0`.
+
+---
+
 ## Production deployment
 
 ### Pattern 1 — host it as a FastAPI service (recommended)
@@ -434,7 +517,7 @@ Deep-dive articles:
 
 ## Architecture overview
 
-fastWorkflow separates **build-time**, **train-time**, and **run-time**. At build-time you create a command interface from your code (recommended via the [integrate-chat-agent](./fastworkflow/docs/integrate-chat-agent) skill). `train` builds the NLP models; `run` executes the workflow. Your existing code is never modified — fastWorkflow sits as a layer on top.
+fastWorkflow separates **build-time**, **train-time**, and **run-time**. At build-time you create a command interface from your code (recommended via the [integrate-chat-agent](./fastworkflow/skills_for_coding_fastworkflows/integrate-chat-agent) skill). `train` builds the NLP models; `run` executes the workflow. Your existing code is never modified — fastWorkflow sits as a layer on top.
 
 ```mermaid
 graph LR
@@ -547,6 +630,12 @@ fastworkflow run <workflow_dir> <env_file> <passwords_file> --assistant   # dete
 fastworkflow run <workflow_dir> <env_file> <passwords_file> \
   --startup_command "your command" --keep_alive False
 
+# Browser chatbot + observability viewer (`run` is also spelled `run_cli`);
+# workflow, env files, and trace maintenance are chosen in the browser
+fastworkflow run_chatbot
+fastworkflow run_chatbot --server-port 8000        # pin the spawned server's port
+fastworkflow run_chatbot --expect-encrypted-jwt    # require signed tokens
+
 # Host as a FastAPI/MCP service
 python -m fastworkflow.run_fastapi_mcp --workflow_path ./wf --port 8000
 ```
@@ -576,6 +665,10 @@ Two files per workflow (templates ship with `fastworkflow examples fetch`).
 | `LITELLM_PROXY_API_BASE` | LiteLLM Proxy URL | with `litellm_proxy/` models | *not set* |
 | `INTENT_DETECTION_TINY_MODEL` | HF id for the small intent model | `train` (optional) | `google/bert_uncased_L-4_H-128_A-2` |
 | `INTENT_DETECTION_LARGE_MODEL` | HF id for the large intent model | `train` (optional) | `distilbert-base-uncased` |
+| `FW_OBSERVABILITY` | Master switch for trace recording. `0` disables it. On by default for `run`/`run_fastapi_mcp`; off by default when embedding the core as a library | Optional | `1` (fastWorkflow entry points) |
+| `FW_OBS_RETENTION_DAYS` | Age beyond which the automatic prune (run at recorder startup) drops spans/artifacts (turn records are exempt) | Optional | `30` |
+| `FW_OBS_DB_MAX_BYTES` | Size cap; the automatic prune evicts oldest spans first while over it | Optional | `1073741824` (1 GiB) |
+| `FW_OBS_CAPTURE_TRACEBACKS` | Persist exception tracebacks as artifacts. Off by default because tracebacks can carry sensitive values | Optional | `0` |
 
 ### `fastworkflow.passwords.env`
 
@@ -614,6 +707,8 @@ When a model uses the `litellm_proxy/` prefix, the per-role keys are ignored and
 
 > **Commands not recognized** — a command module with an import/syntax error won't load and won't appear as an intent. Check your `_commands/*.py` files.
 
+> **The agent did something unexpected** — open the chatbot (`fastworkflow run_chatbot`, then pick the workflow) and read the turn's span tree. It shows which command intent detection chose, what parameters were extracted, and whether your command failed — usually enough to identify the stage at fault before reaching for a debugger.
+
 > [!tip]
 > To debug command files, set up a VSCode `launch.json` with `justMyCode: false`, add breakpoints, and run in debug mode.
 

{fastworkflow-3.1.2 → fastworkflow-3.2.0}/fastworkflow/__init__.py

@@ -63,6 +63,9 @@ class CommandTraceEvent:
     response_text: str | None
     success: bool | None
     timestamp_ms: int
+    # Logical-turn correlation key (additive, observability design §3.1 [X7]).
+    # None only for events emitted outside any logical turn.
+    turn_key: str | None = None
 
 class CommandOutput(BaseModel):
     """The result of one command execution.
@@ -343,3 +346,17 @@ _turn_types_namespace = {
 }
 TurnOutput.model_rebuild(_types_namespace=_turn_types_namespace)
 TurnResult.model_rebuild(_types_namespace=_turn_types_namespace)
+
+# Observability sinks (fastworkflow.tracing / fastworkflow.metrics, both
+# stdlib-only). Embedders implement TraceSink/MetricsSink and wire them via
+# WorkflowExecutionContext (observability design §3.1).
+from fastworkflow.tracing import (
+    Span as Span,
+    TraceSink as TraceSink,
+    NoOpTraceSink as NoOpTraceSink,
+)
+from fastworkflow.metrics import (
+    MetricsSink as MetricsSink,
+    NoOpMetricsSink as NoOpMetricsSink,
+    LoggingMetricsSink as LoggingMetricsSink,
+)

{fastworkflow-3.1.2 → fastworkflow-3.2.0}/fastworkflow/_workflows/command_metadata_extraction/intent_detection.py

@@ -7,7 +7,7 @@ from pydantic import BaseModel
 
 import fastworkflow
 from fastworkflow.utils.logging import logger
-from fastworkflow import NLUPipelineStage
+from fastworkflow import NLUPipelineStage, tracing
 from fastworkflow.cache_matching import cache_match, store_utterance_cache
 from fastworkflow.kvstore import KVStore
 from fastworkflow.model_pipeline_training import (
@@ -40,6 +40,60 @@ class CommandNamePrediction:
         self.path = self._get_cache_path_cache(self.convo_path)
 
     def predict(self, command_context_name: str, command: str, nlu_pipeline_stage: NLUPipelineStage) -> "CommandNamePrediction.Output":
+        """Predict, wrapped in a ``fw.nlu.intent`` span (D3 as amended).
+
+        One span per prediction attempt — the wildcard command's parent-chain
+        walk calls this once per context, and each attempt is recorded with
+        the context it ran against. The span carries which matching layer
+        decided (exact prefix / fuzzy pre-match / embedding cache /
+        classifier), the classifier's confidence and threshold when it ran,
+        and the candidate set on an ambiguity. Emission never affects the
+        prediction: the helpers no-op without a bound host/sink.
+        """
+        host = tracing.current_host()
+        span = tracing.start_span(
+            host,
+            tracing.SPAN_NLU_INTENT,
+            attributes={
+                "context": command_context_name,
+                "stage": nlu_pipeline_stage.name,
+                "utterance": command,
+            },
+        )
+        nlu_trace: dict = {}
+        try:
+            output = self._predict_impl(
+                command_context_name, command, nlu_pipeline_stage, nlu_trace
+            )
+        except BaseException:
+            tracing.end_span(
+                host, span, status=tracing.STATUS_ERROR, attributes=nlu_trace
+            )
+            raise
+        tracing.end_span(
+            host,
+            span,
+            status=tracing.STATUS_OK,
+            attributes={
+                **nlu_trace,
+                "command_name": output.command_name,
+                "is_cme_command": output.is_cme_command,
+                "ambiguous": output.error_msg is not None,
+                # None = no local prediction; the caller walks up the context
+                # chain (or files a misunderstanding) — exactly the routing
+                # signal a debugging agent needs.
+                "resolved": output.command_name is not None,
+            },
+        )
+        return output
+
+    def _predict_impl(
+        self,
+        command_context_name: str,
+        command: str,
+        nlu_pipeline_stage: NLUPipelineStage,
+        nlu_trace: dict,
+    ) -> "CommandNamePrediction.Output":
         # sourcery skip: extract-duplicate-method
 
         model_artifact_path = f"{self.app_workflow_folderpath}/___command_info/{command_context_name}"
@@ -109,6 +163,7 @@ class CommandNamePrediction:
         if normalized_command_name in command_name_dict:
             command_name = normalized_command_name
             command = command.replace(f"{tentative_command_name}", "").strip().replace("  ", " ")
+            nlu_trace["matcher_layer"] = "exact_prefix"
         else:
             # Use Levenshtein distance for fuzzy matching with the full command part after @
             # No match is ([], None), never (None, None) — len() here is safe.
@@ -133,16 +188,24 @@ class CommandNamePrediction:
                     f"utterance '{command}' in context '{command_context_name}'. "
                     "Deferring to the classifier instead of picking one."
                 )
+                nlu_trace["fuzzy_prematch_tie"] = [str(c) for c in best_matched_commands]
             elif best_matched_commands:
                 command_name = best_matched_commands[0]
+                nlu_trace["matcher_layer"] = "fuzzy_prematch"
 
         if nlu_pipeline_stage == NLUPipelineStage.INTENT_DETECTION:
             if not command_name:
                 if cache_result := cache_match(self.path, command, modelpipeline, 0.85):
                     command_name = cache_result
+                    nlu_trace["matcher_layer"] = "embedding_cache"
+                    nlu_trace["cache_similarity_threshold"] = 0.85
                 else:
-                    predictions=command_router.predict(command)
+                    predictions, classifier_details = (
+                        command_router.predict_with_details(command)
+                    )
                     # predictions = majority_vote_predictions(command_router, command)
+                    nlu_trace["matcher_layer"] = "classifier"
+                    nlu_trace["classifier"] = classifier_details
 
                     if len(predictions)==1:
                         command_name = predictions[0].split('/')[-1]
@@ -158,11 +221,15 @@ class CommandNamePrediction:
                                 "will be prompted with the local candidates only and the "
                                 "'this belongs to an ancestor context' signal is dropped."
                             )
+                            nlu_trace["escalation_labels_discarded"] = [
+                                str(label) for label in escalation_signals
+                            ]
 
                         error_msg = self._formulate_ambiguous_command_error_message(
                             predictions, "run_as_agent" in self.app_workflow.context)
 
                         # Store suggested commands
+                        nlu_trace["candidates"] = [str(p) for p in predictions]
                         self._store_suggested_commands(self.path, predictions, 1)
                         return CommandNamePrediction.Output(error_msg=error_msg)
 
@@ -171,6 +238,7 @@ class CommandNamePrediction:
             NLUPipelineStage.INTENT_MISUNDERSTANDING_CLARIFICATION
         ) and not command_name:
             command_name = "what can i do?"
+            nlu_trace["matcher_layer"] = "clarification_default"
 
         fully_qualified_command_name = self.resolve_fully_qualified_command_name(
             command_name, command_name_dict)

{fastworkflow-3.1.2 → fastworkflow-3.2.0}/fastworkflow/_workflows/command_metadata_extraction/parameter_extraction.py

@@ -10,7 +10,7 @@ from pydantic_core import PydanticUndefined
 
 import fastworkflow
 from fastworkflow.utils.logging import logger
-from fastworkflow import ModuleType
+from fastworkflow import ModuleType, tracing
 
 from fastworkflow.utils.signatures import InputForParamExtraction
 
@@ -58,6 +58,43 @@ class ParameterExtraction:
         self.command = command
 
     def extract(self) -> "ParameterExtraction.Output":
+        """Extract, wrapped in a ``fw.nlu.param_extraction`` span (D3 as
+        amended). The span records the extraction method (stored-merge /
+        xml-regex / llm), whether this is a NOT_FOUND retry round, and the
+        STRUCTURED validation outcome from ``validate_parameters`` —
+        missing/invalid fields, per-field db_lookup events, and the
+        validate_extracted_parameters verdict. A failed extraction is a
+        conversational state, not a span error: status stays ok and
+        ``parameters_valid`` carries the signal; only an exception marks the
+        span error. Emission never affects extraction (no-op without a host).
+        """
+        host = tracing.current_host()
+        span = tracing.start_span(
+            host,
+            tracing.SPAN_NLU_PARAM_EXTRACTION,
+            command_name=self.command_name,
+            attributes={"command_name": self.command_name},
+        )
+        diagnostics: dict = {}
+        try:
+            output = self._extract_impl(diagnostics)
+        except BaseException:
+            tracing.end_span(
+                host, span, status=tracing.STATUS_ERROR, attributes=diagnostics
+            )
+            raise
+        tracing.end_span(
+            host,
+            span,
+            status=tracing.STATUS_OK,
+            attributes={
+                **diagnostics,
+                "parameters_valid": output.parameters_are_valid,
+            },
+        )
+        return output
+
+    def _extract_impl(self, diagnostics: dict) -> "ParameterExtraction.Output":
         app_workflow_folderpath = self.app_workflow.folderpath
         app_command_routing_definition = fastworkflow.RoutingRegistry.get_definition(app_workflow_folderpath)
 
@@ -78,8 +115,10 @@ class ParameterExtraction:
             self.command)
 
         # If we have missing fields (in parameter extraction error state), try to apply the command directly
+        diagnostics["retry_round"] = bool(stored_params)
         if stored_params:
             new_params = self._extract_and_merge_missing_parameters(stored_params, self.command)
+            diagnostics["extraction_method"] = "stored_merge"
         else:
             # Check if we're in agentic mode (not assistant mode command)
             is_agentic_mode = (
@@ -91,6 +130,7 @@ class ParameterExtraction:
             if is_agentic_mode:
                 # Try regex-based extraction first in agentic mode
                 new_params = self._extract_parameters_from_xml(self.command, command_parameters_class)
+                diagnostics["extraction_method"] = "xml_regex"
 
                 # If regex extraction fails, fall back to LLM-based extraction
                 if new_params is None:
@@ -98,16 +138,19 @@ class ParameterExtraction:
                         command_parameters_class,
                         self.command_name,
                         app_workflow_folderpath)
+                    diagnostics["extraction_method"] = "llm"
             else:
                 # Use LLM-based extraction for assistant mode
                 new_params = input_for_param_extraction.extract_parameters(
                     command_parameters_class,
                     self.command_name,
                     app_workflow_folderpath)
+                diagnostics["extraction_method"] = "llm"
 
         is_valid, error_msg, suggestions, missing_invalid_fields = \
             input_for_param_extraction.validate_parameters(
-            self.app_workflow, self.command_name, new_params
+            self.app_workflow, self.command_name, new_params,
+            diagnostics=diagnostics
         )
 
         # Set all the missing and invalid fields to appropriate sentinel values before storing

{fastworkflow-3.1.2 → fastworkflow-3.2.0}/fastworkflow/chat_session.py

@@ -117,9 +117,15 @@ class ChatSession:
         """
         self._core = WorkflowExecutionContext(
             run_as_agent=run_as_agent,
-            mirror_action_log_to_file=True,
             generate_insights=generate_insights,
         )
+        # CLI identity [R17]: a synthetic per-session channel so CLI turns are
+        # attributable in the observability store (conversation ids are minted
+        # by the store from Phase 2).
+        from datetime import datetime, timezone
+        self._core.bind_observability_identity(
+            channel_id=f"cli:{datetime.now(timezone.utc).strftime('%Y%m%dT%H%M%S.%f')}Z"
+        )
 
         # Create queues for user messages and command outputs (CLI transport)
         self._user_message_queue = Queue()
@@ -371,8 +377,10 @@ class ChatSession:
             # Handle startup command/action
             if self._startup_command:
                 last_output = self._core._execute_message(self._startup_command)
+                self._core.finalize_turn_for_observability(last_output)
             elif self._startup_action:
                 last_output = self._core.process_action(self._startup_action)
+                self._core.finalize_turn_for_observability(last_output)
 
             while (
                 not self.workflow_is_complete or self._keep_alive
@@ -384,6 +392,9 @@ class ChatSession:
                         last_output = self._core.process_action(message)
                     else:
                         last_output = self._core._execute_message(message)
+                    # Emit the turn record/root-span close for observability;
+                    # the CLI transport itself still rides the queues.
+                    self._core.finalize_turn_for_observability(last_output)
 
                 except Empty:
                     continue