fastworkflow 2.17.26__tar.gz → 2.17.27__tar.gz

{fastworkflow-2.17.26 → fastworkflow-2.17.27}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: fastworkflow
-Version: 2.17.26
+Version: 2.17.27
 Summary: A framework for rapidly building large-scale, deterministic, interactive workflows with a fault-tolerant, conversational UX
 License: Apache-2.0
 Keywords: fastworkflow,ai,workflow,llm,openai
@@ -13,24 +13,31 @@ Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
 Provides-Extra: fastapi
 Provides-Extra: training
+Requires-Dist: aiohttp (>=3.13.3)
 Requires-Dist: colorama (>=0.4.6,<0.5.0)
 Requires-Dist: datasets (>=4.0.0,<5.0.0) ; extra == "training"
 Requires-Dist: dspy (>=3.0.1,<4.0.0)
 Requires-Dist: fastapi (>=0.120.1) ; extra == "fastapi"
 Requires-Dist: fastapi-mcp (>=0.4.0,<0.5.0) ; extra == "fastapi"
+Requires-Dist: filelock (>=3.20.1)
 Requires-Dist: libcst (>=1.8.2,<2.0.0)
-Requires-Dist: litellm[proxy] (>=1.80.5,<2.0.0)
+Requires-Dist: litellm[proxy] (>=1.81.4,<2.0.0)
 Requires-Dist: mmh3 (>=5.1.0,<6.0.0)
 Requires-Dist: openai (>=2.8.0)
+Requires-Dist: orjson (>=3.9.15)
 Requires-Dist: prompt_toolkit (>=3.0.43,<4.0.0)
+Requires-Dist: pyasn1 (>=0.6.2)
 Requires-Dist: pydantic (>=2.9.2,<3.0.0)
+Requires-Dist: pyjwt[crypto] (>=2.4.0) ; extra == "fastapi"
+Requires-Dist: pynacl (>=1.6.2)
 Requires-Dist: python-dotenv (>=1.0.1,<2.0.0)
-Requires-Dist: python-jose[cryptography] (>=3.3.0,<4.0.0) ; extra == "fastapi"
 Requires-Dist: python-levenshtein (>=0.27.1,<0.28.0)
 Requires-Dist: scikit-learn (>=1.6.1,<2.0.0)
 Requires-Dist: speedict (>=0.3.12,<0.4.0)
+Requires-Dist: starlette (>=0.49.1)
 Requires-Dist: torch (>=2.7.1,<3.0.0)
 Requires-Dist: transformers (>=4.48.2,<5.0.0)
+Requires-Dist: urllib3 (>=2.6.0)
 Requires-Dist: uvicorn (>=0.31.1,<0.32.0) ; extra == "fastapi"
 Project-URL: homepage, https://github.com/radiantlogicinc/fastworkflow
 Project-URL: repository, https://github.com/radiantlogicinc/fastworkflow
@@ -606,6 +613,7 @@ This single command will generate the `greet.py` command, `get_properties` and `
 | `LLM_PLANNER` | LiteLLM model string for the agent's task planner | `run` (agent mode) | `mistral/mistral-small-latest` |
 | `LLM_AGENT` | LiteLLM model string for the DSPy agent | `run` (agent mode) | `mistral/mistral-small-latest` |
 | `LLM_CONVERSATION_STORE` | LiteLLM model string for conversation topic/summary generation | FastAPI service | `mistral/mistral-small-latest` |
+| `LITELLM_PROXY_API_BASE` | URL of your LiteLLM Proxy server | When using `litellm_proxy/` models | *not set* |
 | `NOT_FOUND` | Placeholder value for missing parameters during extraction | Always | `"NOT_FOUND"` |
 | `MISSING_INFORMATION_ERRMSG` | Error message prefix for missing parameters | Always | `"Missing required..."` |
 | `INVALID_INFORMATION_ERRMSG` | Error message prefix for invalid parameters | Always | `"Invalid information..."` |
@@ -620,10 +628,35 @@ This single command will generate the `greet.py` command, `get_properties` and `
 | `LITELLM_API_KEY_PLANNER`| API key for the `LLM_PLANNER` model | `run` (agent mode) | *required* |
 | `LITELLM_API_KEY_AGENT`| API key for the `LLM_AGENT` model | `run` (agent mode) | *required* |
 | `LITELLM_API_KEY_CONVERSATION_STORE`| API key for the `LLM_CONVERSATION_STORE` model | FastAPI service | *required* |
+| `LITELLM_PROXY_API_KEY`| Shared API key for authenticating with LiteLLM Proxy | When using `litellm_proxy/` models | *optional* |
 
 > [!tip]
 > The example workflows are configured to use Mistral's models by default. You can get a free API key from [Mistral AI](https://mistral.ai) that works with the `mistral-small-latest` model.
 
+### Using LiteLLM Proxy
+
+FastWorkflow supports routing LLM calls through a [LiteLLM Proxy](https://docs.litellm.ai/docs/simple_proxy) server. This is useful when you want to:
+- Centralize API key management
+- Use a unified endpoint for multiple LLM providers
+- Route requests through a corporate proxy with custom configurations
+
+To use LiteLLM Proxy, set your model strings to use the `litellm_proxy/` prefix and configure the proxy URL:
+
+```
+# In fastworkflow.env - use the litellm_proxy/ prefix for model names
+LLM_AGENT=litellm_proxy/bedrock_mistral_large_2407
+LLM_PARAM_EXTRACTION=litellm_proxy/bedrock_mistral_large_2407
+LITELLM_PROXY_API_BASE=http://127.0.0.1:4000
+
+# In fastworkflow.passwords.env - shared key for proxy authentication
+LITELLM_PROXY_API_KEY=your-proxy-api-key
+```
+
+The model name after `litellm_proxy/` (e.g., `bedrock_mistral_large_2407`) is passed to your proxy server, which routes it to the actual provider based on its configuration.
+
+> [!note]
+> When using `litellm_proxy/` models, the per-role API keys (`LITELLM_API_KEY_*`) are ignored. All proxied calls use the shared `LITELLM_PROXY_API_KEY` instead. You can mix proxied and direct models - only models with the `litellm_proxy/` prefix are routed through the proxy.
+
 ---
 
 ## Troubleshooting / FAQ

{fastworkflow-2.17.26 → fastworkflow-2.17.27}/README.md

@@ -568,6 +568,7 @@ This single command will generate the `greet.py` command, `get_properties` and `
 | `LLM_PLANNER` | LiteLLM model string for the agent's task planner | `run` (agent mode) | `mistral/mistral-small-latest` |
 | `LLM_AGENT` | LiteLLM model string for the DSPy agent | `run` (agent mode) | `mistral/mistral-small-latest` |
 | `LLM_CONVERSATION_STORE` | LiteLLM model string for conversation topic/summary generation | FastAPI service | `mistral/mistral-small-latest` |
+| `LITELLM_PROXY_API_BASE` | URL of your LiteLLM Proxy server | When using `litellm_proxy/` models | *not set* |
 | `NOT_FOUND` | Placeholder value for missing parameters during extraction | Always | `"NOT_FOUND"` |
 | `MISSING_INFORMATION_ERRMSG` | Error message prefix for missing parameters | Always | `"Missing required..."` |
 | `INVALID_INFORMATION_ERRMSG` | Error message prefix for invalid parameters | Always | `"Invalid information..."` |
@@ -582,10 +583,35 @@ This single command will generate the `greet.py` command, `get_properties` and `
 | `LITELLM_API_KEY_PLANNER`| API key for the `LLM_PLANNER` model | `run` (agent mode) | *required* |
 | `LITELLM_API_KEY_AGENT`| API key for the `LLM_AGENT` model | `run` (agent mode) | *required* |
 | `LITELLM_API_KEY_CONVERSATION_STORE`| API key for the `LLM_CONVERSATION_STORE` model | FastAPI service | *required* |
+| `LITELLM_PROXY_API_KEY`| Shared API key for authenticating with LiteLLM Proxy | When using `litellm_proxy/` models | *optional* |
 
 > [!tip]
 > The example workflows are configured to use Mistral's models by default. You can get a free API key from [Mistral AI](https://mistral.ai) that works with the `mistral-small-latest` model.
 
+### Using LiteLLM Proxy
+
+FastWorkflow supports routing LLM calls through a [LiteLLM Proxy](https://docs.litellm.ai/docs/simple_proxy) server. This is useful when you want to:
+- Centralize API key management
+- Use a unified endpoint for multiple LLM providers
+- Route requests through a corporate proxy with custom configurations
+
+To use LiteLLM Proxy, set your model strings to use the `litellm_proxy/` prefix and configure the proxy URL:
+
+```
+# In fastworkflow.env - use the litellm_proxy/ prefix for model names
+LLM_AGENT=litellm_proxy/bedrock_mistral_large_2407
+LLM_PARAM_EXTRACTION=litellm_proxy/bedrock_mistral_large_2407
+LITELLM_PROXY_API_BASE=http://127.0.0.1:4000
+
+# In fastworkflow.passwords.env - shared key for proxy authentication
+LITELLM_PROXY_API_KEY=your-proxy-api-key
+```
+
+The model name after `litellm_proxy/` (e.g., `bedrock_mistral_large_2407`) is passed to your proxy server, which routes it to the actual provider based on its configuration.
+
+> [!note]
+> When using `litellm_proxy/` models, the per-role API keys (`LITELLM_API_KEY_*`) are ignored. All proxied calls use the shared `LITELLM_PROXY_API_KEY` instead. You can mix proxied and direct models - only models with the `litellm_proxy/` prefix are routed through the proxy.
+
 ---
 
 ## Troubleshooting / FAQ

fastworkflow-2.17.27/fastworkflow/examples/fastworkflow.env

@@ -0,0 +1,35 @@
+# ============================================================================
+# LLM Model Configuration
+# ============================================================================
+# Use direct provider model strings (e.g., mistral/mistral-small-latest)
+# or LiteLLM Proxy model strings (e.g., litellm_proxy/your_model_name)
+LLM_SYNDATA_GEN=mistral/mistral-small-latest
+LLM_PARAM_EXTRACTION=mistral/mistral-small-latest
+LLM_RESPONSE_GEN=mistral/mistral-small-latest
+LLM_PLANNER=mistral/mistral-small-latest
+LLM_AGENT=mistral/mistral-small-latest
+LLM_CONVERSATION_STORE=mistral/mistral-small-latest
+
+# ============================================================================
+# LiteLLM Proxy Configuration (Optional)
+# ============================================================================
+# To route LLM calls through a LiteLLM Proxy, set the model strings above to
+# use the litellm_proxy/ prefix and configure the proxy URL below.
+# Example:
+#   LLM_AGENT=litellm_proxy/bedrock_mistral_large_2407
+#   LITELLM_PROXY_API_BASE=http://127.0.0.1:4000
+# The proxy API key should be set in fastworkflow.passwords.env
+# LITELLM_PROXY_API_BASE=http://127.0.0.1:4000
+
+# ============================================================================
+# Workflow Configuration
+# ============================================================================
+SPEEDDICT_FOLDERNAME=___workflow_contexts
+SYNTHETIC_UTTERANCE_GEN_NUMOF_PERSONAS=4
+SYNTHETIC_UTTERANCE_GEN_UTTERANCES_PER_PERSONA=5
+SYNTHETIC_UTTERANCE_GEN_PERSONAS_PER_BATCH=1
+MISSING_INFORMATION_ERRMSG="Missing parameter values: "
+INVALID_INFORMATION_ERRMSG="Invalid parameter values: "
+NOT_FOUND="NOT_FOUND"
+INVALID="INVALID"
+PARAMETER_EXTRACTION_ERROR_MSG="Error in parameter extraction: {error}"

fastworkflow-2.17.27/fastworkflow/examples/fastworkflow.passwords.env

@@ -0,0 +1,20 @@
+# ============================================================================
+# Direct Provider API Keys
+# ============================================================================
+# Tested with Mistral Small 3.1. A bigger model will produce better results, obviously
+# These keys are used when LLM_* variables use direct provider model strings
+# (e.g., mistral/mistral-small-latest, openai/gpt-4, etc.)
+LITELLM_API_KEY_SYNDATA_GEN=<API KEY for synthetic data generation model>
+LITELLM_API_KEY_PARAM_EXTRACTION=<API KEY for parameter extraction model>
+LITELLM_API_KEY_RESPONSE_GEN=<API KEY for response generation model>
+LITELLM_API_KEY_PLANNER=<API KEY for the agent's task planner model>
+LITELLM_API_KEY_AGENT=<API KEY for the agent model>
+LITELLM_API_KEY_CONVERSATION_STORE=<API KEY for conversation topic/summary generation model>
+
+# ============================================================================
+# LiteLLM Proxy API Key (Optional)
+# ============================================================================
+# When using litellm_proxy/ model strings, this shared key is used for all
+# proxied calls. The per-role keys above are ignored for proxied models.
+# Leave commented if your proxy doesn't require authentication.
+# LITELLM_PROXY_API_KEY=<API KEY for LiteLLM Proxy authentication>

{fastworkflow-2.17.26 → fastworkflow-2.17.27}/fastworkflow/run_fastapi_mcp/__main__.py

@@ -28,15 +28,16 @@ from contextlib import asynccontextmanager
 import argparse
 
 import uvicorn
-from jose import JWTError
+from jwt.exceptions import PyJWTError as JWTError
 from dotenv import dotenv_values
 
 import fastworkflow
 from fastworkflow.utils.logging import logger
 
-from fastapi import FastAPI, HTTPException, status, Depends, Header
+from fastapi import FastAPI, HTTPException, status, Depends, Header, Request
 from fastapi.middleware.cors import CORSMiddleware
 from fastapi.responses import HTMLResponse, JSONResponse, StreamingResponse
+from starlette.middleware.base import BaseHTTPMiddleware
 
 from .mcp_specific import setup_mcp
 from .utils import (
@@ -76,6 +77,85 @@ from .conversation_store import (
 )
 
  
+# ============================================================================
+# Probe Logging Filter Middleware
+# ============================================================================
+
+# Paths that should not be logged unless they return non-200 status
+PROBE_PATHS = {"/probes/healthz", "/probes/readyz"}
+
+
+class ProbeLoggingFilterMiddleware(BaseHTTPMiddleware):
+    """
+    Middleware to suppress logging for Kubernetes probe endpoints.
+    
+    Probe endpoints (/probes/healthz, /probes/readyz) are called frequently by
+    Kubernetes and would generate excessive logs. This middleware only logs
+    probe requests when they return a non-200 status code.
+    """
+    
+    async def dispatch(self, request: Request, call_next):
+        response = await call_next(request)
+        
+        # Only log probe endpoints if they return non-200 status
+        if request.url.path in PROBE_PATHS and response.status_code != 200:
+            logger.warning(
+                f"Probe {request.url.path} returned status {response.status_code}"
+            )
+        
+        return response
+
+
+# ============================================================================
+# Readiness State Tracking
+# ============================================================================
+
+class ReadinessState:
+    """
+    Tracks the readiness state of the application.
+    
+    The application is considered ready when set_ready(True) is called,
+    typically after successful initialization in the lifespan startup.
+    
+    Additional debug attributes (is_initialized, workflow_path_valid) are
+    retained for production debugging but do not control readiness.
+    """
+    
+    def __init__(self):
+        self._is_ready = False
+        # Debug attributes - do not control readiness, used for diagnostics
+        self._is_initialized = False
+        self._workflow_path_valid = False
+    
+    def set_ready(self, value: bool = True):
+        """Set the main readiness state. Called after successful initialization."""
+        self._is_ready = value
+    
+    def set_initialized(self, value: bool = True):
+        """Mark FastWorkflow as initialized (for debugging/diagnostics)."""
+        self._is_initialized = value
+    
+    def set_workflow_path_valid(self, value: bool = True):
+        """Mark workflow path as validated (for debugging/diagnostics)."""
+        self._workflow_path_valid = value
+    
+    def is_ready(self) -> bool:
+        """Check if the application is ready to serve traffic."""
+        return self._is_ready
+    
+    def get_status(self) -> dict:
+        """Get detailed readiness status for debugging."""
+        return {
+            "ready": self._is_ready,
+            "fastworkflow_initialized": self._is_initialized,
+            "workflow_path_valid": self._workflow_path_valid
+        }
+
+
+# Global readiness state
+readiness_state = ReadinessState()
+
+
 # ============================================================================
 # Session Management
 # ============================================================================
@@ -162,6 +242,17 @@ async def lifespan(_app: FastAPI):
         
         # Configure JWT verification mode based on CLI parameter
         set_jwt_verification_mode(ARGS.expect_encrypted_jwt)
+        
+        # Mark FastWorkflow as initialized for readiness probe
+        readiness_state.set_initialized(True)
+        
+        # Validate workflow path for readiness probe
+        if ARGS.workflow_path and os.path.exists(ARGS.workflow_path):
+            readiness_state.set_workflow_path_valid(True)
+            logger.info(f"Workflow path validated: {ARGS.workflow_path}")
+        else:
+            logger.warning(f"Workflow path not valid or not found: {ARGS.workflow_path}")
+            readiness_state.set_workflow_path_valid(False)
 
     async def _active_turn_channel_ids() -> list[str]:
         active: list[str] = []
@@ -213,6 +304,9 @@ async def lifespan(_app: FastAPI):
 
     try:
         initialize_fastworkflow_on_startup()
+        # Mark application as ready to accept traffic
+        readiness_state.set_ready(True)
+        logger.info("Application ready to accept traffic")
         yield
     finally:
         logger.info("FastWorkflow FastAPI service shutting down...")
@@ -275,8 +369,8 @@ def custom_openapi():
     
     # Apply security globally to all endpoints except public ones
     for path, path_item in openapi_schema["paths"].items():
-        # Skip endpoints that don't require authentication
-        if path in ["/initialize", "/refresh_token", "/", "/admin/dump_all_conversations", "/admin/generate_mcp_token"]:
+        # Skip endpoints that don't require authentication (including probe endpoints)
+        if path in ["/initialize", "/refresh_token", "/", "/admin/dump_all_conversations", "/admin/generate_mcp_token", "/probes/healthz", "/probes/readyz"]:
             continue
         for method in path_item:
             if method in ["get", "post", "put", "delete", "patch"] and "security" not in path_item[method]:
@@ -296,6 +390,9 @@ app.add_middleware(
     allow_headers=["*"],
 )
 
+# Probe logging filter middleware - suppresses logs for successful probe requests
+app.add_middleware(ProbeLoggingFilterMiddleware)
+
 # ============================================================================
 # Endpoints
 # ============================================================================
@@ -317,6 +414,91 @@ async def root():
     """
 
 
+# ============================================================================
+# Kubernetes Probe Endpoints
+# ============================================================================
+
+@app.get(
+    "/probes/healthz",
+    operation_id="liveness_probe",
+    status_code=status.HTTP_200_OK,
+    responses={
+        200: {"description": "Application is alive and running"},
+        503: {"description": "Application is unhealthy"}
+    },
+    tags=["probes"]
+)
+async def liveness_probe() -> dict:
+    """
+    Liveness probe endpoint for Kubernetes.
+    
+    Determines whether the container is still running. If this probe fails,
+    Kubernetes will restart the container.
+    
+    This endpoint checks basic application health:
+    - The FastAPI application is responsive
+    - The event loop is processing requests
+    
+    This endpoint is not logged unless it returns a non-200 status code
+    to avoid excessive logging from frequent Kubernetes health checks.
+    
+    Returns:
+        200 OK: {"status": "alive"} - Application is running normally
+        503 Service Unavailable: Application is unhealthy
+    """
+    # Basic liveness check - if we can respond, we're alive
+    # The application is considered "live" if it can process HTTP requests
+    return {"status": "alive"}
+
+
+@app.get(
+    "/probes/readyz",
+    operation_id="readiness_probe",
+    status_code=status.HTTP_200_OK,
+    responses={
+        200: {"description": "Application is ready to accept traffic"},
+        503: {"description": "Application is not ready to accept traffic"}
+    },
+    tags=["probes"]
+)
+async def readiness_probe() -> JSONResponse:
+    """
+    Readiness probe endpoint for Kubernetes.
+    
+    Checks whether the container is ready to accept traffic. Kubernetes only
+    routes traffic to containers that pass the readiness check.
+    
+    This endpoint verifies:
+    - FastWorkflow has been initialized
+    - The configured workflow path is valid and accessible
+    
+    This endpoint is not logged unless it returns a non-200 status code
+    to avoid excessive logging from frequent Kubernetes health checks.
+    
+    Returns:
+        200 OK: {"status": "ready", "checks": {...}} - Ready to accept traffic
+        503 Service Unavailable: {"status": "not_ready", "checks": {...}} - Not ready
+    """
+    status_info = readiness_state.get_status()
+    
+    if readiness_state.is_ready():
+        return JSONResponse(
+            status_code=status.HTTP_200_OK,
+            content={
+                "status": "ready",
+                "checks": status_info
+            }
+        )
+    else:
+        return JSONResponse(
+            status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
+            content={
+                "status": "not_ready",
+                "checks": status_info
+            }
+        )
+
+
 @app.post(
     "/initialize",
     operation_id="rest_initialize",

{fastworkflow-2.17.26 → fastworkflow-2.17.27}/fastworkflow/run_fastapi_mcp/jwt_manager.py

@@ -9,8 +9,8 @@ import os
 from datetime import datetime, timedelta, timezone
 from typing import Optional
 
-from jose import JWTError, jwt
-from jose.constants import ALGORITHMS
+import jwt
+from jwt.exceptions import PyJWTError as JWTError
 from cryptography.hazmat.primitives import serialization
 from cryptography.hazmat.primitives.asymmetric import rsa
 from cryptography.hazmat.backends import default_backend
@@ -19,7 +19,7 @@ from fastworkflow.utils.logging import logger
 
 
 # JWT Configuration (can be made configurable via env vars)
-JWT_ALGORITHM = ALGORITHMS.RS256
+JWT_ALGORITHM = "RS256"
 JWT_ACCESS_TOKEN_EXPIRE_MINUTES = 60  # 1 hour
 JWT_REFRESH_TOKEN_EXPIRE_DAYS = 30  # 30 days
 JWT_ISSUER = "fastworkflow-api"
@@ -274,7 +274,7 @@ def verify_token(token: str, expected_type: str = "access") -> dict:
         # Trusted network mode: decode without verification (accepts both unsigned and signed tokens)
         try:
             # Use unverified decoding - works for any JWT regardless of algorithm or signing
-            payload = jwt.get_unverified_claims(token)
+            payload = jwt.decode(token, options={"verify_signature": False})
         except Exception as e:
             logger.warning(f"Token decoding failed: {e}")
             raise JWTError(f"Failed to decode token: {e}") from e
@@ -332,7 +332,7 @@ def get_token_expiry(token: str) -> Optional[datetime]:
     """
     try:
         # Decode without verification (just to inspect claims)
-        payload = jwt.get_unverified_claims(token)
+        payload = jwt.decode(token, options={"verify_signature": False})
         if exp_timestamp := payload.get("exp"):
             return datetime.fromtimestamp(exp_timestamp, tz=timezone.utc)
     except Exception as e:

{fastworkflow-2.17.26 → fastworkflow-2.17.27}/fastworkflow/run_fastapi_mcp/utils.py

@@ -7,7 +7,7 @@ from typing import Any, Optional
 
 from fastapi import HTTPException, status, Depends
 from fastapi.security import HTTPBearer, HTTPAuthorizationCredentials
-from jose import JWTError
+from jwt.exceptions import PyJWTError as JWTError
 from pydantic import BaseModel, field_validator
 
 import fastworkflow

{fastworkflow-2.17.26 → fastworkflow-2.17.27}/fastworkflow/utils/dspy_utils.py

@@ -6,11 +6,65 @@ import fastworkflow
 from fastworkflow.utils.logging import logger
 
 def get_lm(model_env_var: str, api_key_env_var: Optional[str] = None, **kwargs):
-    """get the dspy lm object"""
+    """
+    Get the dspy LM object.
+    
+    Supports LiteLLM Proxy routing: if the model string starts with 'litellm_proxy/',
+    the call is routed through the LiteLLM Proxy using LITELLM_PROXY_API_BASE and
+    LITELLM_PROXY_API_KEY environment variables.
+    
+    Args:
+        model_env_var: Name of the environment variable containing the model string
+                       (e.g., 'LLM_AGENT', 'LLM_PARAM_EXTRACTION').
+        api_key_env_var: Name of the environment variable containing the API key
+                         for direct provider calls. Ignored for litellm_proxy/ models.
+        **kwargs: Additional keyword arguments passed to dspy.LM().
+    
+    Returns:
+        dspy.LM: Configured language model instance.
+    
+    Raises:
+        ValueError: If model is not set, or if using litellm_proxy/ without
+                    LITELLM_PROXY_API_BASE configured.
+    
+    Example:
+        # Direct provider call (existing behavior):
+        # LLM_AGENT=mistral/mistral-small-latest
+        # LITELLM_API_KEY_AGENT=sk-...
+        lm = get_lm("LLM_AGENT", "LITELLM_API_KEY_AGENT")
+        
+        # LiteLLM Proxy call:
+        # LLM_AGENT=litellm_proxy/bedrock_mistral_large_2407
+        # LITELLM_PROXY_API_BASE=http://127.0.0.1:4000
+        # LITELLM_PROXY_API_KEY=proxy-key-...
+        lm = get_lm("LLM_AGENT", "LITELLM_API_KEY_AGENT")  # api_key_env_var is ignored for proxy
+    """
     model = fastworkflow.get_env_var(model_env_var)
     if not model:
-        logger.critical(f"Critical Error:DSPy Language Model not provided. Set {model_env_var} environment variable.")
+        logger.critical(f"Critical Error: DSPy Language Model not provided. Set {model_env_var} environment variable.")
         raise ValueError(f"DSPy Language Model not provided. Set {model_env_var} environment variable.")
+    
+    # Check if this is a LiteLLM Proxy call
+    if model.startswith("litellm_proxy/"):
+        # Route through LiteLLM Proxy
+        proxy_api_base = fastworkflow.get_env_var("LITELLM_PROXY_API_BASE")
+        if not proxy_api_base:
+            raise ValueError(
+                f"Model '{model}' uses litellm_proxy/ prefix but LITELLM_PROXY_API_BASE is not set. "
+                "Set LITELLM_PROXY_API_BASE to your LiteLLM Proxy URL (e.g., http://127.0.0.1:4000)."
+            )
+        
+        # Get optional proxy API key (allows no-auth proxies when empty/not set)
+        proxy_api_key = fastworkflow.get_env_var("LITELLM_PROXY_API_KEY", default=None)
+        
+        logger.debug(f"Routing {model_env_var} through LiteLLM Proxy at {proxy_api_base}")
+        
+        if proxy_api_key:
+            return dspy.LM(model=model, api_base=proxy_api_base, api_key=proxy_api_key, **kwargs)
+        else:
+            return dspy.LM(model=model, api_base=proxy_api_base, **kwargs)
+    
+    # Direct provider call (existing behavior)
     api_key = fastworkflow.get_env_var(api_key_env_var) if api_key_env_var else None
     return dspy.LM(model=model, api_key=api_key, **kwargs) if api_key else dspy.LM(model=model, **kwargs)
 

{fastworkflow-2.17.26 → fastworkflow-2.17.27}/fastworkflow/utils/logging.py

@@ -47,7 +47,7 @@ class FormatterNs(logging.Formatter):
 
 logging.setLogRecordFactory(LogRecordNs)
 
-LOG_FORMAT = "%(asctime)s - %(levelname)s - %(filename)s-%(funcName)s - %(message)s"
+LOG_FORMAT = "%(levelname)s:     %(message)s - %(asctime)s - %(filename)s-%(funcName)s"
 log_formatter = FormatterNs(LOG_FORMAT)
 
 if log_level := get_env_variable("LOG_LEVEL", "INFO"):
@@ -98,7 +98,7 @@ pytest_assertion_logger.propagate = (
 pytest_assertion_logger.handlers.clear()
 ch = logging.StreamHandler()
 ch.setLevel(logging.DEBUG)
-ch.setFormatter(FormatterNs("%(asctime)s - %(levelname)s - %(message)s"))
+ch.setFormatter(FormatterNs("%(levelname)s:     %(message)s - %(asctime)s"))
 pytest_assertion_logger.addHandler(ch)
 
 logging.getLogger("dspy").setLevel(logging.ERROR)

{fastworkflow-2.17.26 → fastworkflow-2.17.27}/fastworkflow/workflow_agent.py

@@ -12,7 +12,7 @@ from fastworkflow.utils.logging import logger
 from fastworkflow.utils import dspy_utils
 from fastworkflow.command_metadata_api import CommandMetadataAPI
 from fastworkflow.utils.react import fastWorkflowReAct
-
+from fastworkflow.utils.chat_adapter import CommandsSystemPreludeAdapter
 
 class WorkflowAgentSignature(dspy.Signature):
     """
@@ -110,8 +110,7 @@ def _execute_workflow_query(command: str, chat_session_obj: fastworkflow.ChatSes
 
     # Handle intent ambiguity clarification state with specialized agent
     if nlu_stage == fastworkflow.NLUPipelineStage.INTENT_AMBIGUITY_CLARIFICATION:
-        if intent_agent := chat_session_obj.intent_clarification_agent:
-            from fastworkflow.utils.chat_adapter import CommandsSystemPreludeAdapter    
+        if intent_agent := chat_session_obj.intent_clarification_agent: 
             # Use CommandsSystemPreludeAdapter specifically for workflow agent calls
             agent_adapter = CommandsSystemPreludeAdapter()
 
@@ -321,8 +320,7 @@ def build_query_with_next_steps(user_query: str,
         Avoid specifying 'ask user' because 9 times out of 10, you can find the information via available commands. 
         """
         user_query: str = dspy.InputField()
-        available_commands: list[str] = dspy.InputField()
-        next_steps: list[str] = dspy.OutputField(desc="task descriptions as short sentences")
+        next_steps: str = dspy.OutputField(desc="task descriptions as a numbered list of short sentences separated by line breaks")
 
     class TaskPlannerWithTrajectoryAndAgentInputsSignature(dspy.Signature):
         """
@@ -333,8 +331,7 @@ def build_query_with_next_steps(user_query: str,
         agent_inputs: dict = dspy.InputField()
         agent_trajectory: dict = dspy.InputField()
         user_response: str = dspy.InputField()
-        available_commands: list[str] = dspy.InputField()
-        next_steps: list[str] = dspy.OutputField(desc="task descriptions as short sentences")
+        next_steps: str = dspy.OutputField(desc="task descriptions as a numbered list of short sentences separated by line breaks")
 
     current_workflow = chat_session_obj.get_active_workflow()
     available_commands = CommandMetadataAPI.get_command_display_text(
@@ -344,26 +341,28 @@ def build_query_with_next_steps(user_query: str,
     )
 
     planner_lm = dspy_utils.get_lm("LLM_PLANNER", "LITELLM_API_KEY_PLANNER")
-    with dspy.context(lm=planner_lm):
+    agent_adapter = CommandsSystemPreludeAdapter()
+    with dspy.context(lm=planner_lm, adapter=agent_adapter):
         if with_agent_inputs_and_trajectory:
             workflow_tool_agent = chat_session_obj.workflow_tool_agent
             task_planner_func = dspy.ChainOfThought(TaskPlannerWithTrajectoryAndAgentInputsSignature)
+            cleaned_agent_inputs = {k: v for k, v in workflow_tool_agent.inputs.items() if k != "available_commands"}
             prediction = task_planner_func(
-                agent_inputs = workflow_tool_agent.inputs,
+                agent_inputs = cleaned_agent_inputs,
                 agent_trajectory = workflow_tool_agent.current_trajectory,
                 user_response = user_query,
-                available_commands=available_commands)
+                available_commands=available_commands) # Note that this is not part of the signature. It is extra metadata that will be picked up by the CommandsSystemPreludeAdapter
         else:
             task_planner_func = dspy.ChainOfThought(TaskPlannerSignature)
             prediction = task_planner_func(
                 user_query=user_query,
-                available_commands=available_commands)
+                available_commands=available_commands) # Note that this is not part of the signature. It is extra metadata that will be picked up by the CommandsSystemPreludeAdapter
 
         if not prediction.next_steps:
             return user_query
 
-        steps_list = '\n'.join([f'{i + 1}. {task}' for i, task in enumerate(prediction.next_steps)])
-        user_query_and_next_steps = f"{user_query}\n\nExecute these next steps:\n{steps_list}"
+        steps_formatted = " ".join(prediction.next_steps.split())
+        user_query_and_next_steps = f"{user_query}\n\nExecute these next steps:\n{steps_formatted}"
         return (
             f'User Query:\n{user_query_and_next_steps}'
             if with_agent_inputs_and_trajectory else