fastworkflow 2.15.7__tar.gz → 2.15.9__tar.gz

{fastworkflow-2.15.7 → fastworkflow-2.15.9}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: fastworkflow
-Version: 2.15.7
+Version: 2.15.9
 Summary: A framework for rapidly building large-scale, deterministic, interactive workflows with a fault-tolerant, conversational UX
 License: Apache-2.0
 Keywords: fastworkflow,ai,workflow,llm,openai
@@ -50,7 +50,7 @@ While [DSPy](https://dspy.ai) ([Why DSPy](https://x.com/lateinteraction/status/1
 
 ### Why fastWorkflow?
 
-- ✅ **Unlimited Tool Scaling**: fastworkflow can scale to an unlimited number of tools
+- ✅ **Unlimited Tool Scaling**: fastworkflow organizes tools into context hierarchies so use any number of tools without sacrificing performance or efficiency
 - ✅ **Cost-Effective Performance**: fastWorkFlow with small, free models can match the quality of large expensive models
 - ✅ **Reliable Tool Execution**: fastworkflow validation pipeline virtually eliminates incorrect tool calling or parameter extraction, ensuring a reliable tool response
 - ✅ **Adaptive Learning**: 1-shot learning from intent detection mistakes. It learns your conversational vocabulary as you interact with it
@@ -202,14 +202,14 @@ LITELLM_API_KEY_PLANNER=your-mistral-api-key
 LITELLM_API_KEY_AGENT=your-mistral-api-key
 ```
 
-You can get a free API key from [Mistral AI](https://mistral.ai) - the example is configured to use the `mistral-small-latest` model which is available on their free tier.
+You can get a free API key from [Mistral AI](https://mistral.ai) for the mistral small model. Or a free API key from [OpenRouter](https://openrouter.ai/openai/gpt-oss-20b:free) for the GPT-OSS-20B:free model. You can use different models for different LLM roles in the same workflow if you wish.
 
 ### Step 3: Train the Example
 
 Train the intent-detection models for the workflow:
 
 ```sh
-fastworkflow examples train hello_world
+fastworkflow train ./examples/hello_world ./examples/fastworkflow.env ./examples/fastworkflow.passwords.env
 ```
 
 This step builds the NLP models that help the workflow understand user commands.
@@ -219,7 +219,7 @@ This step builds the NLP models that help the workflow understand user commands.
 Once training is complete, run the interactive assistant:
 
 ```sh
-fastworkflow examples run hello_world
+fastworkflow run ./examples/hello_world ./examples/fastworkflow.env ./examples/fastworkflow.passwords.env
 ```
 
 You will be greeted with a `User >` prompt. Try it out by asking "what can you do?" or "add 49 + 51"!
@@ -240,12 +240,6 @@ fastworkflow examples list
 
 # Fetch an example to your local directory
 fastworkflow examples fetch <example_name>
-
-# Train an example workflow
-fastworkflow examples train <example_name>
-
-# Run an example workflow
-fastworkflow examples run <example_name>
 ```
 
 ### Workflow Operations
@@ -261,11 +255,8 @@ fastworkflow train <workflow_dir> <env_file> <passwords_file>
 fastworkflow run <workflow_dir> <env_file> <passwords_file>
 ```
 
-To run a workflow in agentic mode, add the `--run_as_agent` flag:
-
-```sh
-fastworkflow run <workflow_dir> <env_file> <passwords_file> --run_as_agent
-```
+> [!tip]
+> **Deterministic execution:** Prefix a natural language command with `/` to execute it deterministically (non‑agentic) during an interactive run.
 
 Each command has additional options that can be viewed with the `--help` flag:
 

{fastworkflow-2.15.7 → fastworkflow-2.15.9}/README.md

@@ -17,7 +17,7 @@ While [DSPy](https://dspy.ai) ([Why DSPy](https://x.com/lateinteraction/status/1
 
 ### Why fastWorkflow?
 
-- ✅ **Unlimited Tool Scaling**: fastworkflow can scale to an unlimited number of tools
+- ✅ **Unlimited Tool Scaling**: fastworkflow organizes tools into context hierarchies so use any number of tools without sacrificing performance or efficiency
 - ✅ **Cost-Effective Performance**: fastWorkFlow with small, free models can match the quality of large expensive models
 - ✅ **Reliable Tool Execution**: fastworkflow validation pipeline virtually eliminates incorrect tool calling or parameter extraction, ensuring a reliable tool response
 - ✅ **Adaptive Learning**: 1-shot learning from intent detection mistakes. It learns your conversational vocabulary as you interact with it
@@ -169,14 +169,14 @@ LITELLM_API_KEY_PLANNER=your-mistral-api-key
 LITELLM_API_KEY_AGENT=your-mistral-api-key
 ```
 
-You can get a free API key from [Mistral AI](https://mistral.ai) - the example is configured to use the `mistral-small-latest` model which is available on their free tier.
+You can get a free API key from [Mistral AI](https://mistral.ai) for the mistral small model. Or a free API key from [OpenRouter](https://openrouter.ai/openai/gpt-oss-20b:free) for the GPT-OSS-20B:free model. You can use different models for different LLM roles in the same workflow if you wish.
 
 ### Step 3: Train the Example
 
 Train the intent-detection models for the workflow:
 
 ```sh
-fastworkflow examples train hello_world
+fastworkflow train ./examples/hello_world ./examples/fastworkflow.env ./examples/fastworkflow.passwords.env
 ```
 
 This step builds the NLP models that help the workflow understand user commands.
@@ -186,7 +186,7 @@ This step builds the NLP models that help the workflow understand user commands.
 Once training is complete, run the interactive assistant:
 
 ```sh
-fastworkflow examples run hello_world
+fastworkflow run ./examples/hello_world ./examples/fastworkflow.env ./examples/fastworkflow.passwords.env
 ```
 
 You will be greeted with a `User >` prompt. Try it out by asking "what can you do?" or "add 49 + 51"!
@@ -207,12 +207,6 @@ fastworkflow examples list
 
 # Fetch an example to your local directory
 fastworkflow examples fetch <example_name>
-
-# Train an example workflow
-fastworkflow examples train <example_name>
-
-# Run an example workflow
-fastworkflow examples run <example_name>
 ```
 
 ### Workflow Operations
@@ -228,11 +222,8 @@ fastworkflow train <workflow_dir> <env_file> <passwords_file>
 fastworkflow run <workflow_dir> <env_file> <passwords_file>
 ```
 
-To run a workflow in agentic mode, add the `--run_as_agent` flag:
-
-```sh
-fastworkflow run <workflow_dir> <env_file> <passwords_file> --run_as_agent
-```
+> [!tip]
+> **Deterministic execution:** Prefix a natural language command with `/` to execute it deterministically (non‑agentic) during an interactive run.
 
 Each command has additional options that can be viewed with the `--help` flag:
 

fastworkflow-2.15.9/fastworkflow/_workflows/command_metadata_extraction/_commands/wildcard.py

@@ -0,0 +1,163 @@
+import fastworkflow
+from fastworkflow import Action, CommandOutput, CommandResponse, NLUPipelineStage
+from fastworkflow.command_executor import CommandExecutor
+
+from ..intent_detection import CommandNamePrediction
+from ..parameter_extraction import ParameterExtraction
+
+
+class Signature:
+    plain_utterances = [
+        "3",
+        "france",
+        "16.7,.002",
+        "John Doe, 56, 281-995-6423",
+        "/path/to/my/object",
+        "id=3636",
+        "25.73 and Howard St",
+    ]
+
+    @staticmethod
+    def generate_utterances(workflow: fastworkflow.Workflow, command_name: str) -> list[str]:
+        return [
+            command_name.split('/')[-1].lower().replace('_', ' ')
+        ] + Signature.plain_utterances
+
+
+class ResponseGenerator:
+    def __call__(
+        self, 
+        workflow: fastworkflow.Workflow, 
+        command: str,
+    ) -> CommandOutput:  # sourcery skip: hoist-if-from-if
+        app_workflow = workflow.context["app_workflow"]   # type: fastworkflow.Workflow
+        cmd_ctxt_obj_name = app_workflow.current_command_context_name
+        nlu_pipeline_stage = workflow.context.get(
+            "NLU_Pipeline_Stage", 
+            NLUPipelineStage.INTENT_DETECTION)
+
+        predictor = CommandNamePrediction(workflow)           
+        cnp_output = predictor.predict(cmd_ctxt_obj_name, command, nlu_pipeline_stage)
+
+        if cnp_output.error_msg:
+            workflow_context = workflow.context
+            workflow_context["NLU_Pipeline_Stage"] = NLUPipelineStage.INTENT_AMBIGUITY_CLARIFICATION
+            workflow_context["command"] = command
+            workflow.context = workflow_context
+            return CommandOutput(
+                command_responses=[
+                    CommandResponse(
+                        response=(
+                            f"Ambiguous intent error for command '{command}'\n"
+                            f"{cnp_output.error_msg}"
+                        ),
+                        success=False
+                    )
+                ]
+            )
+        else:
+            if nlu_pipeline_stage == NLUPipelineStage.INTENT_DETECTION and \
+                cnp_output.command_name != 'ErrorCorrection/you_misunderstood':
+                workflow_context = workflow.context
+                workflow_context["command"] = command
+                workflow.context = workflow_context
+        
+        if cnp_output.is_cme_command:
+            workflow_context = workflow.context
+            if cnp_output.command_name == 'ErrorCorrection/you_misunderstood':
+                workflow_context["NLU_Pipeline_Stage"] = NLUPipelineStage.INTENT_MISUNDERSTANDING_CLARIFICATION
+                workflow_context["command"] = command
+            else:
+                workflow.end_command_processing()
+            workflow.context = workflow_context
+
+            startup_action = Action(
+                command_name=cnp_output.command_name,
+                command=command,
+            )
+            command_output = CommandExecutor.perform_action(workflow, startup_action)
+            command_output.command_responses[0].artifacts["command_handled"] = True     
+            # Set the additional attributes
+            command_output.command_name = cnp_output.command_name
+            return command_output
+        
+        if nlu_pipeline_stage in {
+                NLUPipelineStage.INTENT_DETECTION,
+                NLUPipelineStage.INTENT_AMBIGUITY_CLARIFICATION,
+                NLUPipelineStage.INTENT_MISUNDERSTANDING_CLARIFICATION
+            }:
+            app_workflow.command_context_for_response_generation = \
+                app_workflow.current_command_context
+
+            if cnp_output.command_name is None:
+                while not cnp_output.command_name and \
+                    app_workflow.command_context_for_response_generation is not None and \
+                        not app_workflow.is_command_context_for_response_generation_root:
+                    app_workflow.command_context_for_response_generation = \
+                        app_workflow.get_parent(app_workflow.command_context_for_response_generation)
+                    cnp_output = predictor.predict(
+                        fastworkflow.Workflow.get_command_context_name(app_workflow.command_context_for_response_generation), 
+                        command, nlu_pipeline_stage)
+            
+                if cnp_output.command_name is None:
+                    if nlu_pipeline_stage == NLUPipelineStage.INTENT_DETECTION:
+                        # out of scope commands
+                        workflow_context = workflow.context
+                        workflow_context["NLU_Pipeline_Stage"] = \
+                            NLUPipelineStage.INTENT_MISUNDERSTANDING_CLARIFICATION
+                        workflow_context["command"] = command
+                        workflow.context = workflow_context
+
+                        startup_action = Action(
+                            command_name='ErrorCorrection/you_misunderstood',
+                            command=command,
+                        )
+                        command_output = CommandExecutor.perform_action(workflow, startup_action)
+                        command_output.command_responses[0].artifacts["command_handled"] = True
+                        return command_output
+
+                    return CommandOutput(
+                        command_responses=[
+                            CommandResponse(
+                                response=cnp_output.error_msg,
+                                success=False
+                            )
+                        ]
+                    )
+
+            # move to the parameter extraction stage
+            workflow_context = workflow.context
+            workflow_context["NLU_Pipeline_Stage"] = NLUPipelineStage.PARAMETER_EXTRACTION
+            workflow.context = workflow_context
+
+        command_name = cnp_output.command_name
+        extractor = ParameterExtraction(workflow, app_workflow, command_name, command)
+        pe_output = extractor.extract()
+        if not pe_output.parameters_are_valid:
+            return CommandOutput(
+                command_name = command_name,
+                command_responses=[
+                    CommandResponse(
+                        response=(
+                            f"PARAMETER EXTRACTION ERROR FOR COMMAND '{command_name}'\n"
+                            f"{pe_output.error_msg}"
+                        ),
+                        success=False
+                    )
+                ]
+            )
+
+        workflow.end_command_processing()
+
+        return CommandOutput(
+            command_responses=[
+                CommandResponse(
+                    response="",
+                    artifacts={
+                        "command": command,
+                        "command_name": command_name,
+                        "cmd_parameters": pe_output.cmd_parameters,
+                    },
+                )
+            ]
+        ) 

fastworkflow-2.15.9/fastworkflow/_workflows/command_metadata_extraction/intent_detection.py

@@ -0,0 +1,360 @@
+from typing import Optional
+import os
+from collections import Counter
+from concurrent.futures import ThreadPoolExecutor, as_completed
+
+from pydantic import BaseModel
+from speedict import Rdict
+
+import fastworkflow
+from fastworkflow.utils.logging import logger
+from fastworkflow import NLUPipelineStage
+from fastworkflow.cache_matching import cache_match, store_utterance_cache
+from fastworkflow.model_pipeline_training import (
+    CommandRouter
+)
+
+from fastworkflow.utils.fuzzy_match import find_best_matches
+
+
+class CommandNamePrediction:
+    class Output(BaseModel):
+        command_name: Optional[str] = None
+        error_msg: Optional[str] = None
+        is_cme_command: bool = False
+
+    def __init__(self, cme_workflow: fastworkflow.Workflow):
+        self.cme_workflow = cme_workflow
+        self.app_workflow = cme_workflow.context["app_workflow"]
+        self.app_workflow_folderpath = self.app_workflow.folderpath
+        self.app_workflow_id = self.app_workflow.id
+
+        self.convo_path = os.path.join(self.app_workflow_folderpath, "___convo_info")
+        self.cache_path = self._get_cache_path(self.app_workflow_id, self.convo_path)
+        self.path = self._get_cache_path_cache(self.convo_path)
+
+    def predict(self, command_context_name: str, command: str, nlu_pipeline_stage: NLUPipelineStage) -> "CommandNamePrediction.Output":
+        # sourcery skip: extract-duplicate-method
+
+        model_artifact_path = f"{self.app_workflow_folderpath}/___command_info/{command_context_name}"
+        command_router = CommandRouter(model_artifact_path)
+
+        # Re-use the already-built ModelPipeline attached to the router
+        # instead of instantiating a fresh one.  This avoids reloading HF
+        # checkpoints and transferring tensors each time we see a new
+        # message for the same context.
+        modelpipeline = command_router.modelpipeline
+
+        crd = fastworkflow.RoutingRegistry.get_definition(
+            self.cme_workflow.folderpath)
+        cme_command_names = crd.get_command_names('IntentDetection')
+
+        valid_command_names = set()
+        if nlu_pipeline_stage == NLUPipelineStage.INTENT_AMBIGUITY_CLARIFICATION:
+            valid_command_names = self._get_suggested_commands(self.path)
+        elif nlu_pipeline_stage in (
+                NLUPipelineStage.INTENT_DETECTION, NLUPipelineStage.INTENT_MISUNDERSTANDING_CLARIFICATION):
+            app_crd = fastworkflow.RoutingRegistry.get_definition(
+                self.app_workflow_folderpath)
+            valid_command_names = (
+                set(cme_command_names) | 
+                set(app_crd.get_command_names(command_context_name))
+            )
+
+        command_name_dict = {
+            fully_qualified_command_name.split('/')[-1]: fully_qualified_command_name 
+            for fully_qualified_command_name in valid_command_names
+        }
+
+        if nlu_pipeline_stage == NLUPipelineStage.INTENT_AMBIGUITY_CLARIFICATION:
+            # what_can_i_do is special in INTENT_AMBIGUITY_CLARIFICATION
+            # We will not predict, just match plain utterances with exact or fuzzy match
+            command_name_dict |= {
+                plain_utterance: 'IntentDetection/what_can_i_do'
+                for plain_utterance in crd.command_directory.map_command_2_utterance_metadata[
+                    'IntentDetection/what_can_i_do'
+                ].plain_utterances
+            }
+
+        if nlu_pipeline_stage != NLUPipelineStage.INTENT_DETECTION:
+            # abort is special. 
+            # We will not predict, just match plain utterances with exact or fuzzy match
+            command_name_dict |= {
+                plain_utterance: 'ErrorCorrection/abort'
+                for plain_utterance in crd.command_directory.map_command_2_utterance_metadata[
+                    'ErrorCorrection/abort'
+                ].plain_utterances
+            }
+
+        if nlu_pipeline_stage != NLUPipelineStage.INTENT_MISUNDERSTANDING_CLARIFICATION:
+            # you_misunderstood is special. 
+            # We will not predict, just match plain utterances with exact or fuzzy match
+            command_name_dict |= {
+                plain_utterance: 'ErrorCorrection/you_misunderstood'
+                for plain_utterance in crd.command_directory.map_command_2_utterance_metadata[
+                    'ErrorCorrection/you_misunderstood'
+                ].plain_utterances
+            }
+
+        # See if the command starts with a command name followed by a space
+        tentative_command_name = command.split(" ", 1)[0]
+        normalized_command_name = tentative_command_name.lower()
+        command_name = None
+        if normalized_command_name in command_name_dict:
+            command_name = normalized_command_name
+            command = command.replace(f"{tentative_command_name}", "").strip().replace("  ", " ")
+        else:
+            # Use Levenshtein distance for fuzzy matching with the full command part after @
+            best_matched_commands, _ = find_best_matches(
+                command.replace(" ", "_"),
+                command_name_dict.keys(),
+                threshold=0.3  # Adjust threshold as needed
+            )
+            if best_matched_commands:
+                command_name = best_matched_commands[0]
+
+        if nlu_pipeline_stage == NLUPipelineStage.INTENT_DETECTION:
+            if not command_name:
+                if cache_result := cache_match(self.path, command, modelpipeline, 0.85):
+                    command_name = cache_result
+                else:
+                    predictions=command_router.predict(command)
+                    # predictions = majority_vote_predictions(command_router, command)
+
+                    if len(predictions)==1:
+                        command_name = predictions[0].split('/')[-1]
+                    else:
+                        # If confidence is low, treat as ambiguous command (type 1)
+                        error_msg = self._formulate_ambiguous_command_error_message(
+                            predictions, "run_as_agent" in self.app_workflow.context)
+
+                        # Store suggested commands
+                        self._store_suggested_commands(self.path, predictions, 1)
+                        return CommandNamePrediction.Output(error_msg=error_msg)
+
+        elif nlu_pipeline_stage in (
+            NLUPipelineStage.INTENT_AMBIGUITY_CLARIFICATION,
+            NLUPipelineStage.INTENT_MISUNDERSTANDING_CLARIFICATION
+        ) and not command_name:
+            command_name = "what_can_i_do"
+
+        if not command_name or command_name == "wildcard":
+            fully_qualified_command_name=None
+            is_cme_command=False
+        else:
+            fully_qualified_command_name = command_name_dict[command_name]
+            is_cme_command=(
+                fully_qualified_command_name in cme_command_names or 
+                fully_qualified_command_name in crd.get_command_names('ErrorCorrection')
+            )
+
+        if (
+            nlu_pipeline_stage
+            in (
+                NLUPipelineStage.INTENT_AMBIGUITY_CLARIFICATION,
+                NLUPipelineStage.INTENT_MISUNDERSTANDING_CLARIFICATION,
+            )
+            and not fully_qualified_command_name.endswith('abort')
+            and not fully_qualified_command_name.endswith('what_can_i_do')
+            and not fully_qualified_command_name.endswith('you_misunderstood')
+        ):
+            command = self.cme_workflow.context["command"]
+            store_utterance_cache(self.path, command, command_name, modelpipeline)
+
+        return CommandNamePrediction.Output(
+            command_name=fully_qualified_command_name,
+            is_cme_command=is_cme_command
+        )
+
+    @staticmethod
+    def _get_cache_path(workflow_id, convo_path):
+        """
+        Generate cache file path based on workflow ID
+        """
+        base_dir = convo_path
+        # Create directory if it doesn't exist
+        os.makedirs(base_dir, exist_ok=True)
+        return os.path.join(base_dir, f"{workflow_id}.db")
+
+    @staticmethod
+    def _get_cache_path_cache(convo_path):
+        """
+        Generate cache file path based on workflow ID
+        """
+        base_dir = convo_path
+        # Create directory if it doesn't exist
+        os.makedirs(base_dir, exist_ok=True)
+        return os.path.join(base_dir, "cache.db")
+
+    # Store the suggested commands with the flag type
+    @staticmethod
+    def _store_suggested_commands(cache_path, command_list, flag_type):
+        """
+        Store the list of suggested commands for the constrained selection
+
+        Args:
+            cache_path: Path to the cache database
+            command_list: List of suggested commands
+            flag_type: Type of constraint (1=ambiguous, 2=misclassified)
+        """
+        db = Rdict(cache_path)
+        try:
+            db["suggested_commands"] = command_list
+            db["flag_type"] = flag_type
+        finally:
+            db.close()
+
+    # Get the suggested commands
+    @staticmethod
+    def _get_suggested_commands(cache_path):
+        """
+        Get the list of suggested commands for the constrained selection
+        """
+        db = Rdict(cache_path)
+        try:
+            return db.get("suggested_commands", [])
+        finally:
+            db.close()
+
+    @staticmethod
+    def _get_count(cache_path):
+        db = Rdict(cache_path)
+        try:
+            return db.get("utterance_count", 0)  # Default to 0 if key doesn't exist
+        finally:
+            db.close()
+
+    @staticmethod
+    def _print_db_contents(cache_path):
+        db = Rdict(cache_path)
+        try:
+            print("All keys in database:", list(db.keys()))
+            for key in db.keys():
+                print(f"Key: {key}, Value: {db[key]}")
+        finally:
+            db.close()
+
+    @staticmethod
+    def _store_utterance(cache_path, utterance, label):
+        """
+        Store utterance in existing or new database
+        Returns: The utterance count used
+        """
+        # Open the database (creates if doesn't exist)
+        db = Rdict(cache_path)
+
+        try:
+            # Get existing counter or initialize to 0
+            utterance_count = db.get("utterance_count", 0)
+
+            # Create and store the utterance entry
+            utterance_data = {
+                "utterance": utterance,
+                "label": label
+            }
+
+            db[utterance_count] = utterance_data
+
+            # Increment and store the counter
+            utterance_count += 1
+            db["utterance_count"] = utterance_count
+
+            return utterance_count - 1  # Return the count used for this utterance
+
+        finally:
+            # Always close the database
+            db.close()
+
+    # Function to read from database
+    @staticmethod
+    def _read_utterance(cache_path, utterance_id):
+        """
+        Read a specific utterance from the database
+        """
+        db = Rdict(cache_path)
+        try:
+            return db.get(utterance_id)['utterance']
+        finally:
+            db.close()
+
+    @staticmethod
+    def _formulate_ambiguous_command_error_message(
+        route_choice_list: list[str], run_as_agent: bool) -> str:
+        command_list = (
+            "\n".join([
+                f"{route_choice.split('/')[-1].lower()}"
+                for route_choice in route_choice_list if route_choice != 'wildcard'
+            ])
+        )
+
+        return (
+            "The command is ambiguous. "
+            + (
+                "Choose the correct command name from these possible options and update your command:\n"
+                if run_as_agent
+                else "Please choose a command name from these possible options:\n"
+            )
+            + f"{command_list}\n\nor type 'what can i do' to see all commands\n"
+            + ("or type 'abort' to cancel" if run_as_agent else '')
+        )
+
+
+# TODO - generation is deterministic. They all return the same answer
+# TODO - Need 'temperature' for intent detection pipeline
+def majority_vote_predictions(command_router, command: str, n_predictions: int = 5) -> list[str]:
+    """
+    Generate N prediction sets in parallel and return the set that wins the majority vote.
+    
+    This function improves prediction reliability by running multiple parallel predictions
+    and selecting the most common result through majority voting. This helps reduce
+    the impact of random variations in model predictions.
+    
+    Args:
+        command_router: The CommandRouter instance to use for predictions
+        command: The input command string
+        n_predictions: Number of parallel predictions to generate (default: 5)
+                      Can be configured via N_PARALLEL_PREDICTIONS environment variable
+        
+    Returns:
+        The prediction set that received the majority vote. Falls back to a single
+        prediction if all parallel predictions fail.
+        
+    Note:
+        Uses ThreadPoolExecutor with max_workers limited to min(n_predictions, 10)
+        to avoid overwhelming the system with too many concurrent threads.
+    """
+    def get_single_prediction():
+        """Helper function to get a single prediction"""
+        return command_router.predict(command)
+    
+    # Generate N predictions in parallel
+    prediction_sets = []
+    with ThreadPoolExecutor(max_workers=min(n_predictions, 10)) as executor:
+        # Submit all prediction tasks
+        futures = [executor.submit(get_single_prediction) for _ in range(n_predictions)]
+        
+        # Collect results as they complete
+        for future in as_completed(futures):
+            try:
+                prediction_set = future.result()
+                prediction_sets.append(prediction_set)
+            except Exception as e:
+                logger.warning(f"Prediction failed: {e}")
+                # Continue with other predictions even if one fails
+    
+    if not prediction_sets:
+        # Fallback to single prediction if all parallel predictions failed
+        logger.warning("All parallel predictions failed, falling back to single prediction")
+        return command_router.predict(command)
+    
+    # Convert lists to tuples so they can be hashed and counted
+    prediction_tuples = [tuple(sorted(pred_set)) for pred_set in prediction_sets]
+    
+    # Count occurrences of each unique prediction set
+    vote_counts = Counter(prediction_tuples)
+    
+    # Get the prediction set with the most votes
+    winning_tuple = vote_counts.most_common(1)[0][0]
+    
+    # Convert back to list and return
+    return list(winning_tuple)