rasa-pro 3.14.0a16__py3-none-any.whl → 3.14.0a17__py3-none-any.whl

rasa/builder/copilot/prompts/copilot_system_prompt.jinja2

@@ -5,6 +5,65 @@ guidance and practical code solutions.
 
 ---
 
+# Instruction Priority
+
+1. Special tokens
+   - [ERROR_FALLBACK], [ROLEPLAY_REQUEST_DETECTED], [OUT_OF_SCOPE_REQUEST_DETECTED]
+2. Handling Greetings / Handling Goodbyes. Highest priority for casual messages
+3. Handling Knowledge Base Content Request
+4. Response Guidelines & Format (including citations)
+5. Everything else
+
+---
+
+# Default Introductions
+
+## About Rasa
+Rasa is a leading platform for generative conversational AI, designed to help
+enterprises build and operate advanced AI assistants at scale.
+
+If the user asks *"What is Rasa?"* or something similar, respond with this introduction.
+Then follow it with a friendly next-step question, such as:
+- "Do you want to learn more about the platform?"
+- "Can I help you get started with exploring your assistant?"
+
+***
+
+## Handling Greetings
+
+When the user sends a **greeting** message, reply **only** per the template below and
+**then stop**:
+- **Match their tone, style, and energy**. For example: "hi" -> "Hi!", "yo" -> "Yo!"
+- Add **at most one** optional next-step. For example: "The assistant is loaded and ready. Want to start by exploring what it can do?"
+- Keep it **brief, playful, and approachable**.
+- **Response size limit:** 30 words max, 3 sentences max.
+
+***
+
+## Handling Goodbyes
+When the user sends a **goodbye** message, reply **only** per the template below and
+**then stop**:
+
+- **Match their tone and energy**. For example: "cya" -> "See you!"
+- Keep it **short, warm, and natural**.
+- You may add a brief acknowledgment. For example: "thanks" -> "You're welcome!"
+- **Response size limit:** 15 words max, 2 sentences max.
+
+***
+
+## When asked *"What can you do?"*
+If a user asks: "What can you do" (or variations like "How can you help me?"),
+always respond with the answer with this response pattern:
+"""
+I can help you understand how your assistant works, explain Rasa concepts, and guide you
+through troubleshooting or improving your project. You can ask me things like:
+- How does [Assistant Name] know what skills it has?
+- What triggers this greeting?
+- Help me add another banking skill to my assistant.
+"""
+
+---
+
 # Your Abilities
 You can help users with:
 - **Debugging & Explanation**: Analyze conversations and logs to explain assistant behavior.
@@ -99,48 +158,6 @@ continuity.
 
 ---
 
-# Default Introductions
-
-## About Rasa
-Rasa is a leading platform for generative conversational AI, designed to help
-enterprises build and operate advanced AI assistants at scale.
-
-If the user asks *"What is Rasa?"* or something similar, respond with this introduction.
-Then follow it with a friendly next-step question, such as:
-- "Do you want to learn more about the platform?"
-- "Can I help you get started with exploring your assistant?"
-
-***
-
-## When asked *"What can you do?"*
-If a user asks: "What can you do" (or variations like "How can you help me?"),
-always respond with the answer with this response pattern:
-"""
-I can help you understand how your assistant works, explain Rasa concepts, and guide you
-through troubleshooting or improving your project. You can ask me things like:
-- How does [Assistant Name] know what skills it has?
-- What triggers this greeting?
-- Help me add another banking skill to my assistant.
-"""
-
-## Greetings (When user says *hey | hi | yo | hello*)
-When the user greets you casually, **match the
-tone of their greeting** and respond similarly to these friendly introduction patterns:
-
-**Example 1:**
-"""
-Hey! I've got your assistant loaded and ready. Want to start by exploring what it can
-do, or should we dive into editing a skill?
-"""
-
-**Example 2:**
-"""
-Yo! Your project's open. I can walk you through how it works, help debug, or add
-something new. What's on your mind today?
-"""
-
----
-
 # Response Guidelines & Format
 
 Your reply must be **Markdown** and communicate one clear idea (concept, explanation,
@@ -478,7 +495,103 @@ Do not attempt to explain, summarize, or speculate once fallback is triggered. S
 
 ---
 
-# Examples
+# Appendix: Full Examples
+
+## Example: Greetings
+
+User greeted you with one of the following messages:
+1. "hi"
+2. "Hey!"
+3. "What's up?"
+4. "hello there"
+
+Your response should match the tone of the user's message:
+1. "Heeyy! I've got your assistant loaded and ready. Want to start by exploring what it can do?"
+2. "Hey there! Ready when you are. Should we dive into editing a skill?"
+3. "All good here! I've got your assistant loaded and ready. What do you want to explore?"
+4. "Hello there! You can start by testing a flow or updating a response."
+
+***
+
+## Example: Goodbye
+
+User sent a goodbye message::
+1. "bye"
+2. "Goodbye!"
+3. "See you later!"
+4. "Good night!"
+5. "Thanks. Catch you later."
+6. "See you!"
+
+Your response should match the tone of the user's message:
+1. "Bye! Take care."
+2. "Goodbye! Have a great day!"
+3. "See you later! Feel free to reach out anytime."
+4. "Good night! Talk to you later."
+5. "You're very welcome. See you!"
+6. "See you!"
+
+***
+
+## Example: Roleplaying as the Assistant
+
+User is building an assistant that can book doctor appointments. User asked you, the
+**Rasa assistant development expert**, to roleplay as the assistant itself. User
+asked the following question:
+"""
+Get me appointment at the Dr. John Doe's office.
+"""
+
+Your response should be:
+"""
+[ROLEPLAY_REQUEST_DETECTED]
+"""
+
+***
+
+## Example: Out of scope request
+
+User is building an assistant that can book doctor appointments. User asked you, the
+**Rasa assistant development expert**, one of the following questions:
+- "What is the weather in Tokyo?"
+- "How can I get a loan?"
+- "What's the best crypto to invest in?"
+
+Your response should be:
+"""
+[OUT_OF_SCOPE_REQUEST_DETECTED]
+"""
+
+***
+
+## Example: Error handling
+
+User provided a sentence that describes their error:
+"""
+Not a valid 'next' definition. Expected else block or if-then block.
+"""
+
+Your response should be:
+"""
+**The error is happening because** your `lorem_ipsum` flow has an invalid `next` section
+right after the `collect: foo` step.
+
+The structure of the conditional branching is incorrect. The next block is not following
+the required `if` / `then` pattern. [1](https://rasa.com/docs/reference/primitives/flows)
+
+**Here's how you can fix it:**
+1. Ensure each condition inside `next:` uses both `if:` and `then:` keys. Neither can be skipped.
+2. Add a final `else:` clause to handle cases where no conditions match.
+3. Verify YAML structure: indentation must be exact, and every `if`/`then`/`else` item must be properly nested as a list item.
+   ```
+   next:
+     - if: slots.foo > 1000
+       then: bar_step
+     - else: buzz_step
+   ```
+
+To prevent this in the future, always validate your flow YAML files with `rasa data validate` [2](https://rasa.com/docs/reference/api/command-line-interface/) to catch formatting and logic errors early.
+"""
 
 ## Example: Creating a doctor appointment booking flow
 
@@ -651,65 +764,3 @@ assistant to personalize responses, make decisions, or maintain context across d
 turns. Slots can be filled by the LLM, by user input, or by custom logic, and are often
 used in flows to collect and validate information from users [1](https://rasa.com/docs/reference/primitives/slots)[3](https://rasa.com/docs/pro/build/assistant-memory)[4](https://rasa.com/docs/pro/tutorial)[7](https://rasa.com/docs/studio/build/flow-building/collect)[9](https://rasa.com/docs/reference/primitives/flows).
 """
-
-***
-
-## Example: Roleplaying as the Assistant
-
-User is building an assistant that can book doctor appointments. User asked you, the
-**Rasa assistant development expert**, to roleplay as the assistant itself. User
-asked the following question:
-"""
-Get me appointment at the Dr. John Doe's office.
-"""
-
-Your response should be:
-"""
-[ROLEPLAY_REQUEST_DETECTED]
-"""
-
-***
-
-## Example: Out of scope request
-
-User is building an assistant that can book doctor appointments. User asked you, the
-**Rasa assistant development expert**, one of the following questions:
-- "What is the weather in Tokyo?"
-- "How can I get a loan?"
-- "What's the best crypto to invest in?"
-
-Your response should be:
-"""
-[OUT_OF_SCOPE_REQUEST_DETECTED]
-"""
-
-***
-
-## Example: Error handling
-
-User provided a sentence that describes their error:
-"""
-Not a valid 'next' definition. Expected else block or if-then block.
-"""
-
-Your response should be:
-"""
-**The error is happening because** your `lorem_ipsum` flow has an invalid `next` section
-right after the `collect: foo` step.
-
-The structure of the conditional branching is incorrect. The next block is not following
-the required `if` / `then` pattern. [1](https://rasa.com/docs/reference/primitives/flows)
-
-**Here's how you can fix it:**
-1. Ensure each condition inside `next:` uses both `if:` and `then:` keys. Neither can be skipped.
-2. Add a final `else:` clause to handle cases where no conditions match.
-3. Verify YAML structure: indentation must be exact, and every `if`/`then`/`else` item must be properly nested as a list item.
-   ```
-   next:
-     - if: slots.foo > 1000
-       then: bar_step
-     - else: buzz_step
-   ```
-
-To prevent this in the future, always validate your flow YAML files with `rasa data validate` [2](https://rasa.com/docs/reference/api/command-line-interface/) to catch formatting and logic errors early.
-"""

rasa/builder/copilot/telemetry.py

@@ -99,7 +99,7 @@ class CopilotTelemetry:
         total_tokens: Optional[int] = None,
         system_message: Optional[dict[str, Any]] = None,
         chat_history: Optional[list[dict[str, Any]]] = None,
-        last_user_message: Optional[dict[str, Any]] = None,
+        last_user_message: Optional[str] = None,
     ) -> None:
         """Track a copilot message in the conversation.
 
@@ -112,28 +112,44 @@ class CopilotTelemetry:
             input_tokens: Number of input tokens used (optional).
             output_tokens: Number of output tokens generated (optional).
             total_tokens: Total number of tokens used (input + output) (optional).
+            system_message: The system message used (optional).
+            chat_history: The chat history messages used (optional).
+            last_user_message: The last user message used (optional).
         """
         structlogger.debug("builder.telemetry.log_copilot_turn", text=text)
+
+        # FIXME: Temporarily remove the system_message from telemetry payload.
+        # Reason: It often exceeds Segment payload size limits, causing the request
+        # to be rejected and the event to be absent in Segment. Instead, temporarily
+        # log the system_message so it's visible in Grafana.
+        telemetry_data = {
+            "project_id": self._project_id,
+            "message_id": uuid.uuid4().hex,
+            "text": text,
+            "prompt_version": self._prompt_version,
+            "source_urls": list(source_urls),
+            "flags": list(flags),
+            "latency_ms": latency_ms,
+            "model": model,
+            "input_tokens": input_tokens,
+            "output_tokens": output_tokens,
+            "total_tokens": total_tokens,
+            "chat_history": chat_history,
+            "last_user_message": last_user_message,
+            "timestamp": dt.datetime.utcnow().isoformat(),
+        }
+
+        # Log all telemetry data plus system_message for debugging
+        log_data = telemetry_data.copy()
+        if system_message:
+            log_data["system_message"] = system_message
+
+        structlogger.info("builder.telemetry.copilot_turn", **log_data)
+
         _track(
             COPILOT_BOT_MESSAGE_EVENT,
             self._user_id,
-            {
-                "project_id": self._project_id,
-                "message_id": uuid.uuid4().hex,
-                "text": text,
-                "prompt_version": self._prompt_version,
-                "source_urls": list(source_urls),
-                "flags": list(flags),
-                "latency_ms": latency_ms,
-                "model": model,
-                "input_tokens": input_tokens,
-                "output_tokens": output_tokens,
-                "total_tokens": total_tokens,
-                "system_message": system_message,
-                "chat_history": chat_history,
-                "last_user_message": last_user_message,
-                "timestamp": dt.datetime.utcnow().isoformat(),
-            },
+            telemetry_data,
         )
 
     @staticmethod
@@ -177,7 +193,7 @@ class CopilotTelemetry:
         total_tokens: int,
         system_message: dict[str, Any],
         chat_history: list[dict[str, Any]],
-        last_user_message: dict[str, Any],
+        last_user_message: Optional[str],
     ) -> None:
         """Log a copilot message from the response handler.
 

rasa/builder/jobs.py

@@ -207,16 +207,26 @@ async def run_template_to_bot_job(
         job_manager.mark_done(job, error=str(exc))
 
 
-async def run_update_files_job(
+async def run_replace_all_files_job(
     app: "Sanic",
     job: JobInfo,
     bot_files: dict,
 ) -> None:
+    """Run the replace-all-files job in the background.
+
+    This replaces all bot files with the provided files and deletes any files
+    not included in the request (excluding .rasa/ and models/ directories).
+
+    Args:
+        app: The Sanic application instance.
+        job: The job information instance.
+        bot_files: Dictionary of file names to content for replacement.
+    """
     project_generator = app.ctx.project_generator
     await push_job_status_event(job, JobStatus.received)
 
     try:
-        project_generator.update_bot_files(bot_files)
+        project_generator.replace_all_bot_files(bot_files)
 
         # 1. Validating
         await push_job_status_event(job, JobStatus.validating)
@@ -240,7 +250,7 @@ async def run_update_files_job(
         if config.VALIDATION_FAIL_ON_WARNINGS:
             log_levels.append("warning")
         structlogger.debug(
-            "update_files_job.validation_error",
+            "replace_all_files_job.validation_error",
             job_id=job.id,
             error=str(exc),
             validation_logs=exc.validation_logs,
@@ -254,7 +264,7 @@ async def run_update_files_job(
 
     except TrainingError as exc:
         structlogger.debug(
-            "update_files_job.train_error",
+            "replace_all_files_job.train_error",
             job_id=job.id,
             error=str(exc),
         )
@@ -264,7 +274,7 @@ async def run_update_files_job(
     except Exception as exc:
         # Capture full traceback for anything truly unexpected
         structlogger.exception(
-            "update_files_job.unexpected_error",
+            "replace_all_files_job.unexpected_error",
             job_id=job.id,
             error=str(exc),
         )

rasa/builder/models.py

@@ -6,7 +6,7 @@ from pathlib import Path
 from typing import Any, Dict, List, Literal, Optional
 
 import structlog
-from pydantic import BaseModel, Field, validator
+from pydantic import BaseModel, ConfigDict, Field, field_validator
 
 from rasa.cli.scaffold import ProjectTemplateName
 from rasa.shared.importers.importer import TrainingDataImporter
@@ -21,7 +21,8 @@ class PromptRequest(BaseModel):
         ..., min_length=1, max_length=10000, description="The skill description prompt"
     )
 
-    @validator("prompt")
+    @field_validator("prompt")
+    @classmethod
     def validate_prompt(cls, v: str) -> str:
         if not v.strip():
             raise ValueError("Prompt cannot be empty or whitespace only")
@@ -38,7 +39,8 @@ class TemplateRequest(BaseModel):
         ),
     )
 
-    @validator("template_name")
+    @field_validator("template_name")
+    @classmethod
     def validate_template_name(cls, v: Any) -> Any:
         if v not in ProjectTemplateName:
             raise ValueError(
@@ -54,10 +56,8 @@ class BotDataUpdateRequest(BaseModel):
     flows_yml: Optional[str] = Field(None, alias="flows.yml")
     config_yml: Optional[str] = Field(None, alias="config.yml")
 
-    class Config:
-        """Config for BotDataUpdateRequest."""
-
-        allow_population_by_field_name = True
+    # Allow using either field names or aliases when creating the model
+    model_config = ConfigDict(populate_by_name=True)
 
 
 class BotData(BaseModel):

rasa/builder/project_generator.py

@@ -5,23 +5,25 @@ import os
 import shutil
 from pathlib import Path
 from textwrap import dedent
-from typing import Any, Dict, List, Optional
+from typing import Any, Dict, Generator, List, Optional
 
 import structlog
 
 from rasa.builder import config
 from rasa.builder.exceptions import ProjectGenerationError, ValidationError
 from rasa.builder.llm_service import get_skill_generation_messages, llm_service
+from rasa.builder.logging_utils import capture_exception_with_context
 from rasa.builder.models import BotFiles
 from rasa.builder.project_info import ProjectInfo, ensure_first_used, load_project_info
 from rasa.builder.template_cache import copy_cache_for_template_if_available
 from rasa.builder.training_service import TrainingInput
 from rasa.builder.validation_service import validate_project
 from rasa.cli.scaffold import ProjectTemplateName, create_initial_project
+from rasa.shared.constants import DEFAULT_MODELS_PATH
 from rasa.shared.core.flows import yaml_flows_io
 from rasa.shared.importers.importer import TrainingDataImporter
 from rasa.shared.utils.yaml import dump_obj_as_yaml_to_string
-from rasa.utils.io import subpath
+from rasa.utils.io import InvalidPathException, subpath
 
 structlogger = structlog.get_logger()
 
@@ -233,25 +235,9 @@ class ProjectGenerator:
         """
         bot_files: BotFiles = {}
 
-        for file in self.project_folder.glob("**/*"):
-            # Skip directories
-            if not file.is_file():
-                continue
-
+        for file in self.bot_file_paths():
             relative_path = file.relative_to(self.project_folder)
 
-            # Skip hidden files and directories (any path component starting with '.')
-            # as well as `__pycache__` folders
-            if any(part.startswith(".") for part in relative_path.parts):
-                continue
-
-            if "__pycache__" in relative_path.parts:
-                continue
-
-            # exclude the project_folder / models folder
-            if relative_path.parts[0] == "models":
-                continue
-
             # Exclude the docs directory if specified
             if exclude_docs_directory and relative_path.parts[0] == "docs":
                 continue
@@ -263,7 +249,6 @@ class ProjectGenerator:
                 ]
                 if file.suffix.lstrip(".").lower() not in allowed_file_extensions:
                     continue
-
             # Read file content and store with relative path as key
             try:
                 bot_files[relative_path.as_posix()] = file.read_text(encoding="utf-8")
@@ -274,9 +259,40 @@ class ProjectGenerator:
                     file_path=file.as_posix(),
                 )
                 bot_files[relative_path.as_posix()] = None
-
         return bot_files
 
+    def is_restricted_path(self, path: Path) -> bool:
+        """Check if the path is restricted.
+
+        These paths are excluded from deletion and editing by the user.
+        """
+        relative_path = path.relative_to(self.project_folder)
+
+        # Skip hidden files and directories (any path component starting with '.')
+        # as well as `__pycache__` folders
+        if any(part.startswith(".") for part in relative_path.parts):
+            return True
+
+        if "__pycache__" in relative_path.parts:
+            return True
+
+        # exclude the project_folder / models folder
+        if relative_path.parts[0] == DEFAULT_MODELS_PATH:
+            return True
+
+        return False
+
+    def bot_file_paths(
+        self,
+    ) -> Generator[Path, None, None]:
+        """Get the paths of all bot files."""
+        for file in self.project_folder.glob("**/*"):
+            # Skip directories
+            if not file.is_file() or self.is_restricted_path(file):
+                continue
+
+            yield file
+
     def _get_bot_data_for_llm(self) -> Dict[str, Any]:
         """Get the current bot data for the LLM."""
         file_importer = self._create_importer()
@@ -321,7 +337,7 @@ class ProjectGenerator:
     def update_bot_files(self, files: Dict[str, Optional[str]]) -> None:
         """Update bot files with new content by writing to disk."""
         for filename, content in files.items():
-            file_path = Path(subpath(self.project_folder, filename))
+            file_path = Path(subpath(str(self.project_folder), filename))
             # Disallow updates inside .rasa project metadata directory
             if any(
                 part.startswith(".")
@@ -332,6 +348,95 @@ class ProjectGenerator:
             file_path.parent.mkdir(parents=True, exist_ok=True)
             file_path.write_text(content, encoding="utf-8")
 
+    def ensure_all_files_are_writable(self, files: Dict[str, Optional[str]]) -> None:
+        """Ensure all files are writable."""
+        for filename, content in files.items():
+            file_path = Path(subpath(str(self.project_folder), filename))
+            if self.is_restricted_path(file_path):
+                raise InvalidPathException(
+                    f"This file or folder is restricted from editing: {file_path}"
+                )
+
+    def replace_all_bot_files(self, files: Dict[str, Optional[str]]) -> None:
+        """Replace all bot files with new content, deleting files not in the request.
+
+        Files/folders starting with .rasa/ or models/ are excluded from deletion.
+
+        Args:
+            files: Dictionary mapping file names to their content
+        """
+        self.ensure_all_files_are_writable(files)
+        # Collect all existing files - any files not in the new `files` dict will be
+        # deleted from this set
+        existing_files = set(path.as_posix() for path in self.bot_file_paths())
+
+        # Write all new files
+        for filename, content in files.items():
+            if content is None:
+                continue
+
+            file_path = Path(subpath(str(self.project_folder), filename))
+            file_path.parent.mkdir(parents=True, exist_ok=True)
+
+            try:
+                file_path.write_text(content, encoding="utf-8")
+            except Exception as e:
+                # Log write failure and avoid deleting an existing file by mistake
+                capture_exception_with_context(
+                    e,
+                    "project_generator.replace_all_bot_files.write_error",
+                    extra={"file_path": file_path},
+                )
+                if file_path.as_posix() in existing_files:
+                    # Keep the original file if it already existed
+                    existing_files.discard(file_path.as_posix())
+                continue
+
+            # Remove from deletion set since this file is in the new set of files
+            existing_files.discard(file_path.as_posix())
+
+        # Delete files that weren't in the request
+        for file_to_delete in existing_files:
+            file_path = Path(file_to_delete)
+            try:
+                file_path.unlink()
+            except Exception as e:
+                capture_exception_with_context(
+                    e,
+                    "project_generator.replace_all_bot_files.delete_error",
+                    extra={"file_path": file_path},
+                )
+
+        # Clean up empty directories (except excluded ones)
+        self._cleanup_empty_directories()
+
+    def _cleanup_empty_directories(self) -> None:
+        """Remove empty directories from the project folder.
+
+        Excludes hidden files and directories, and models/ from cleanup.
+        """
+        # Walk directories in reverse order (deepest first)
+        for dirpath, dirnames, filenames in os.walk(self.project_folder, topdown=False):
+            # Skip if this is the project root
+            if dirpath == str(self.project_folder):
+                continue
+
+            if self.is_restricted_path(Path(dirpath)):
+                continue
+
+            relative_path = Path(dirpath).relative_to(self.project_folder)
+
+            try:
+                # Only remove if directory is empty
+                if not os.listdir(dirpath):
+                    os.rmdir(dirpath)
+            except Exception as e:
+                capture_exception_with_context(
+                    e,
+                    "project_generator.cleanup_empty_directories.error",
+                    extra={"directory": relative_path.as_posix()},
+                )
+
     def cleanup(self) -> None:
         """Cleanup the project folder."""
         # remove all the files and folders in the project folder resulting
@@ -339,6 +444,8 @@ class ProjectGenerator:
         for filename in os.listdir(self.project_folder):
             file_path = os.path.join(self.project_folder, filename)
             try:
+                if filename == "lost+found":
+                    continue
                 if os.path.isfile(file_path) or os.path.islink(file_path):
                     os.unlink(file_path)
                 elif os.path.isdir(file_path):