PraisonAI 0.0.50__tar.gz → 0.0.53__tar.gz

{praisonai-0.0.50 → praisonai-0.0.53}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: PraisonAI
-Version: 0.0.50
+Version: 0.0.53
 Summary: PraisonAI application combines AutoGen and CrewAI or similar frameworks into a low-code solution for building and managing multi-agent LLM systems, focusing on simplicity, customization, and efficient human-agent collaboration.
 Author: Mervin Praison
 Requires-Python: >=3.10,<3.13
@@ -24,6 +24,7 @@ Requires-Dist: chainlit (>=1.1.301,<2.0.0) ; extra == "ui" or extra == "chat" or
 Requires-Dist: crewai (>=0.32.0)
 Requires-Dist: flask (>=3.0.0) ; extra == "api"
 Requires-Dist: gradio (>=4.26.0) ; extra == "gradio"
+Requires-Dist: greenlet (>=3.0.3) ; extra == "code"
 Requires-Dist: langchain-anthropic (>=0.1.13) ; extra == "anthropic"
 Requires-Dist: langchain-cohere (>=0.1.4) ; extra == "cohere"
 Requires-Dist: langchain-google-genai (>=1.0.4) ; extra == "google"
@@ -45,6 +46,13 @@ Description-Content-Type: text/markdown
     <img alt="PraisonAI Logo" src="docs/images/praisonai-logo-black-large.png">
   </picture>
 </p>
+
+<p align="center">
+<a href="https://github.com/MervinPraison/PraisonAI"><img src="https://static.pepy.tech/badge/PraisonAI" alt="Total Downloads"></a>
+<a href="https://github.com/MervinPraison/PraisonAI"><img src="https://img.shields.io/github/v/release/MervinPraison/PraisonAI" alt="Latest Stable Version"></a>
+<a href="https://github.com/MervinPraison/PraisonAI"><img src="https://img.shields.io/badge/License-MIT-yellow.svg" alt="License"></a>
+</p>
+
 <div align="center">
 
 # Praison AI
@@ -53,12 +61,13 @@ Description-Content-Type: text/markdown
 
 Praison AI, leveraging both AutoGen and CrewAI or any other agent framework, represents a low-code, centralised framework designed to simplify the creation and orchestration of multi-agent systems for various LLM applications, emphasizing ease of use, customization, and human-agent interaction.
 
-|  | Cookbook | Open in Colab |
-| --- | --- | --- |
-| Basic | PraisonAI | <a target="_blank" href="https://colab.research.google.com/github/MervinPraison/PraisonAI/blob/main/cookbooks/praisonai-googlecolab.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a> |
+|               | Cookbook        | Open in Colab                                                                                                                                                                                                                                  |
+| ------------- | --------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| Basic         | PraisonAI       | <a target="_blank" href="https://colab.research.google.com/github/MervinPraison/PraisonAI/blob/main/cookbooks/praisonai-googlecolab.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a>       |
 | Include Tools | PraisonAI Tools | <a target="_blank" href="https://colab.research.google.com/github/MervinPraison/PraisonAI/blob/main/cookbooks/praisonai-tools-googlecolab.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a> |
 
 ## TL;DR
+
 ```bash
 pip install praisonai
 export OPENAI_API_KEY="Enter your API key"
@@ -66,6 +75,14 @@ praisonai --init create a movie script about dog in moon
 praisonai
 ```
 
+## Different User Interfaces:
+
+| Interface | Description | URL |
+|---|---|---|
+| **UI** | Multi Agents such as CrewAI or AutoGen | [https://docs.praison.ai/ui/ui](https://docs.praison.ai/ui/ui) |
+| **Chat** | Chat with 100+ LLMs, single AI Agent | [https://docs.praison.ai/ui/chat](https://docs.praison.ai/ui/chat) |
+| **Code** | Chat with entire Codebase, single AI Agent | [https://docs.praison.ai/ui/code](https://docs.praison.ai/ui/code) |
+
 ## Table of Contents
 
 - [Installation](#installation)
@@ -97,13 +114,14 @@ export OPENAI_API_KEY="Enter your API key"
 Generate your OPENAI API KEY from here: https://platform.openai.com/api-keys
 
 Note: You can use other providers such as Ollama, Mistral ... etc. Details are provided at the bottom.
-    
+
 ```bash
 praisonai --init create a movie script about dog in moon
 ```
+
 This will automatically create agents.yaml file in the current directory.
 
-### To initialse with a specific agent framework (Optional):
+### To initialise with a specific agent framework (Optional):
 
 ```bash
 praisonai --framework autogen --init create movie script about cat in mars
@@ -115,8 +133,8 @@ praisonai --framework autogen --init create movie script about cat in mars
 praisonai
 ```
 
-or 
-    
+or
+
 ```bash
 python -m praisonai
 ```
@@ -135,6 +153,14 @@ praisonai --auto create a movie script about Dog in Moon
 
 ## User Interface
 
+## PraisonAI User Interfaces:
+
+| Interface | Description                                | URL                                                                   |
+| --------- | ------------------------------------------ | --------------------------------------------------------------------- |
+| **UI**    | Multi Agents such as CrewAI or AutoGen     | [https://docs.praisonai.com/ui/ui](https://docs.praison.ai/ui/ui)     |
+| **Chat**  | Chat with 100+ LLMs, single AI Agent       | [https://docs.praisonai.com/ui/chat](https://docs.praison.ai/ui/chat) |
+| **Code**  | Chat with entire Codebase, single AI Agent | [https://docs.praisonai.com/ui/code](https://docs.praison.ai/ui/code) |
+
 ```bash
 pip install -U "praisonai[ui]"
 export OPENAI_API_KEY="Enter your API key"
@@ -143,7 +169,7 @@ export CHAINLIT_AUTH_SECRET=xxxxxxxx
 praisonai ui
 ```
 
-or 
+or
 
 ```
 python -m praisonai ui
@@ -151,7 +177,7 @@ python -m praisonai ui
 
 ## Praison AI Chat
 
-* https://docs.praison.ai/chat/
+- https://docs.praison.ai/chat/
 
 ```bash
 pip install "praisonai[chat]"
@@ -161,10 +187,11 @@ praisonai chat
 
 ## Create Custom Tools
 
-* https://docs.praison.ai/tools/custom/
+- https://docs.praison.ai/tools/custom/
 
 ### Step 1: Pre-requisite to Create a Custom Tool
-`agents.yaml` file should be present in the current directory. 
+
+`agents.yaml` file should be present in the current directory.
 
 If it doesn't exist, create it by running the command `praisonai --init research about the latest AI News and prepare a detailed report`.
 
@@ -201,14 +228,15 @@ roles:
     role: Research Analyst
     tasks:
       gather_data:
-        description: Conduct in-depth research on the latest AI News trends from reputable
+        description:
+          Conduct in-depth research on the latest AI News trends from reputable
           sources.
         expected_output: Comprehensive report on current AI News trends.
     tools:
-    - InternetSearchTool
+      - InternetSearchTool
 ```
 
-## Agents Playbook 
+## Agents Playbook
 
 ### Simple Playbook Example
 
@@ -217,23 +245,23 @@ framework: crewai
 topic: Artificial Intelligence
 roles:
   screenwriter:
-    backstory: 'Skilled in crafting scripts with engaging dialogue about {topic}.'
+    backstory: "Skilled in crafting scripts with engaging dialogue about {topic}."
     goal: Create scripts from concepts.
     role: Screenwriter
     tasks:
       scriptwriting_task:
-        description: 'Develop scripts with compelling characters and dialogue about {topic}.'
-        expected_output: 'Complete script ready for production.'
+        description: "Develop scripts with compelling characters and dialogue about {topic}."
+        expected_output: "Complete script ready for production."
 ```
 
 ## Use 100+ Models
 
-* https://docs.praison.ai/models/
+- https://docs.praison.ai/models/
 
 ## Include praisonai package in your project
 
-* https://docs.praison.ai/developers/wrapper
-* https://docs.praison.ai/developers/wrapper-tools/
+- https://docs.praison.ai/developers/wrapper
+- https://docs.praison.ai/developers/wrapper-tools/
 
 ## Option 1: Using RAW YAML
 
@@ -268,8 +296,7 @@ print(result)
 
 ## Option 2: Using separate agents.yaml file
 
-
-Note: Please create agents.yaml file before hand. 
+Note: Please create agents.yaml file before hand.
 
 ```python
 from praisonai import PraisonAI
@@ -285,16 +312,19 @@ if __name__ == "__main__":
 ## Commands to Install Dependencies:
 
 1. **Install all dependencies, including dev dependencies:**
+
    ```sh
    poetry install
    ```
 
 2. **Install only documentation dependencies:**
+
    ```sh
    poetry install --with docs
    ```
 
 3. **Install only test dependencies:**
+
    ```sh
    poetry install --with test
    ```
@@ -319,3 +349,8 @@ This configuration ensures that your development dependencies are correctly cate
 ## Star History
 
 [![Star History Chart](https://api.star-history.com/svg?repos=MervinPraison/PraisonAI&type=Date)](https://docs.praison.ai)
+
+## License
+
+Praison AI is an open-sourced software licensed under the **[MIT license](https://opensource.org/licenses/MIT)**.
+

{praisonai-0.0.50 → praisonai-0.0.53}/README.md

@@ -5,6 +5,13 @@
     <img alt="PraisonAI Logo" src="docs/images/praisonai-logo-black-large.png">
   </picture>
 </p>
+
+<p align="center">
+<a href="https://github.com/MervinPraison/PraisonAI"><img src="https://static.pepy.tech/badge/PraisonAI" alt="Total Downloads"></a>
+<a href="https://github.com/MervinPraison/PraisonAI"><img src="https://img.shields.io/github/v/release/MervinPraison/PraisonAI" alt="Latest Stable Version"></a>
+<a href="https://github.com/MervinPraison/PraisonAI"><img src="https://img.shields.io/badge/License-MIT-yellow.svg" alt="License"></a>
+</p>
+
 <div align="center">
 
 # Praison AI
@@ -13,12 +20,13 @@
 
 Praison AI, leveraging both AutoGen and CrewAI or any other agent framework, represents a low-code, centralised framework designed to simplify the creation and orchestration of multi-agent systems for various LLM applications, emphasizing ease of use, customization, and human-agent interaction.
 
-|  | Cookbook | Open in Colab |
-| --- | --- | --- |
-| Basic | PraisonAI | <a target="_blank" href="https://colab.research.google.com/github/MervinPraison/PraisonAI/blob/main/cookbooks/praisonai-googlecolab.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a> |
+|               | Cookbook        | Open in Colab                                                                                                                                                                                                                                  |
+| ------------- | --------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| Basic         | PraisonAI       | <a target="_blank" href="https://colab.research.google.com/github/MervinPraison/PraisonAI/blob/main/cookbooks/praisonai-googlecolab.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a>       |
 | Include Tools | PraisonAI Tools | <a target="_blank" href="https://colab.research.google.com/github/MervinPraison/PraisonAI/blob/main/cookbooks/praisonai-tools-googlecolab.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a> |
 
 ## TL;DR
+
 ```bash
 pip install praisonai
 export OPENAI_API_KEY="Enter your API key"
@@ -26,6 +34,14 @@ praisonai --init create a movie script about dog in moon
 praisonai
 ```
 
+## Different User Interfaces:
+
+| Interface | Description | URL |
+|---|---|---|
+| **UI** | Multi Agents such as CrewAI or AutoGen | [https://docs.praison.ai/ui/ui](https://docs.praison.ai/ui/ui) |
+| **Chat** | Chat with 100+ LLMs, single AI Agent | [https://docs.praison.ai/ui/chat](https://docs.praison.ai/ui/chat) |
+| **Code** | Chat with entire Codebase, single AI Agent | [https://docs.praison.ai/ui/code](https://docs.praison.ai/ui/code) |
+
 ## Table of Contents
 
 - [Installation](#installation)
@@ -57,13 +73,14 @@ export OPENAI_API_KEY="Enter your API key"
 Generate your OPENAI API KEY from here: https://platform.openai.com/api-keys
 
 Note: You can use other providers such as Ollama, Mistral ... etc. Details are provided at the bottom.
-    
+
 ```bash
 praisonai --init create a movie script about dog in moon
 ```
+
 This will automatically create agents.yaml file in the current directory.
 
-### To initialse with a specific agent framework (Optional):
+### To initialise with a specific agent framework (Optional):
 
 ```bash
 praisonai --framework autogen --init create movie script about cat in mars
@@ -75,8 +92,8 @@ praisonai --framework autogen --init create movie script about cat in mars
 praisonai
 ```
 
-or 
-    
+or
+
 ```bash
 python -m praisonai
 ```
@@ -95,6 +112,14 @@ praisonai --auto create a movie script about Dog in Moon
 
 ## User Interface
 
+## PraisonAI User Interfaces:
+
+| Interface | Description                                | URL                                                                   |
+| --------- | ------------------------------------------ | --------------------------------------------------------------------- |
+| **UI**    | Multi Agents such as CrewAI or AutoGen     | [https://docs.praisonai.com/ui/ui](https://docs.praison.ai/ui/ui)     |
+| **Chat**  | Chat with 100+ LLMs, single AI Agent       | [https://docs.praisonai.com/ui/chat](https://docs.praison.ai/ui/chat) |
+| **Code**  | Chat with entire Codebase, single AI Agent | [https://docs.praisonai.com/ui/code](https://docs.praison.ai/ui/code) |
+
 ```bash
 pip install -U "praisonai[ui]"
 export OPENAI_API_KEY="Enter your API key"
@@ -103,7 +128,7 @@ export CHAINLIT_AUTH_SECRET=xxxxxxxx
 praisonai ui
 ```
 
-or 
+or
 
 ```
 python -m praisonai ui
@@ -111,7 +136,7 @@ python -m praisonai ui
 
 ## Praison AI Chat
 
-* https://docs.praison.ai/chat/
+- https://docs.praison.ai/chat/
 
 ```bash
 pip install "praisonai[chat]"
@@ -121,10 +146,11 @@ praisonai chat
 
 ## Create Custom Tools
 
-* https://docs.praison.ai/tools/custom/
+- https://docs.praison.ai/tools/custom/
 
 ### Step 1: Pre-requisite to Create a Custom Tool
-`agents.yaml` file should be present in the current directory. 
+
+`agents.yaml` file should be present in the current directory.
 
 If it doesn't exist, create it by running the command `praisonai --init research about the latest AI News and prepare a detailed report`.
 
@@ -161,14 +187,15 @@ roles:
     role: Research Analyst
     tasks:
       gather_data:
-        description: Conduct in-depth research on the latest AI News trends from reputable
+        description:
+          Conduct in-depth research on the latest AI News trends from reputable
           sources.
         expected_output: Comprehensive report on current AI News trends.
     tools:
-    - InternetSearchTool
+      - InternetSearchTool
 ```
 
-## Agents Playbook 
+## Agents Playbook
 
 ### Simple Playbook Example
 
@@ -177,23 +204,23 @@ framework: crewai
 topic: Artificial Intelligence
 roles:
   screenwriter:
-    backstory: 'Skilled in crafting scripts with engaging dialogue about {topic}.'
+    backstory: "Skilled in crafting scripts with engaging dialogue about {topic}."
     goal: Create scripts from concepts.
     role: Screenwriter
     tasks:
       scriptwriting_task:
-        description: 'Develop scripts with compelling characters and dialogue about {topic}.'
-        expected_output: 'Complete script ready for production.'
+        description: "Develop scripts with compelling characters and dialogue about {topic}."
+        expected_output: "Complete script ready for production."
 ```
 
 ## Use 100+ Models
 
-* https://docs.praison.ai/models/
+- https://docs.praison.ai/models/
 
 ## Include praisonai package in your project
 
-* https://docs.praison.ai/developers/wrapper
-* https://docs.praison.ai/developers/wrapper-tools/
+- https://docs.praison.ai/developers/wrapper
+- https://docs.praison.ai/developers/wrapper-tools/
 
 ## Option 1: Using RAW YAML
 
@@ -228,8 +255,7 @@ print(result)
 
 ## Option 2: Using separate agents.yaml file
 
-
-Note: Please create agents.yaml file before hand. 
+Note: Please create agents.yaml file before hand.
 
 ```python
 from praisonai import PraisonAI
@@ -245,16 +271,19 @@ if __name__ == "__main__":
 ## Commands to Install Dependencies:
 
 1. **Install all dependencies, including dev dependencies:**
+
    ```sh
    poetry install
    ```
 
 2. **Install only documentation dependencies:**
+
    ```sh
    poetry install --with docs
    ```
 
 3. **Install only test dependencies:**
+
    ```sh
    poetry install --with test
    ```
@@ -278,4 +307,8 @@ This configuration ensures that your development dependencies are correctly cate
 
 ## Star History
 
-[![Star History Chart](https://api.star-history.com/svg?repos=MervinPraison/PraisonAI&type=Date)](https://docs.praison.ai)
+[![Star History Chart](https://api.star-history.com/svg?repos=MervinPraison/PraisonAI&type=Date)](https://docs.praison.ai)
+
+## License
+
+Praison AI is an open-sourced software licensed under the **[MIT license](https://opensource.org/licenses/MIT)**.

{praisonai-0.0.50 → praisonai-0.0.53}/praisonai/deploy.py

@@ -56,7 +56,7 @@ class CloudDeployer:
             file.write("FROM python:3.11-slim\n")
             file.write("WORKDIR /app\n")
             file.write("COPY . .\n")
-            file.write("RUN pip install flask praisonai==0.0.50 gunicorn markdown\n")
+            file.write("RUN pip install flask praisonai==0.0.53 gunicorn markdown\n")
             file.write("EXPOSE 8080\n")
             file.write('CMD ["gunicorn", "-b", "0.0.0.0:8080", "api:app"]\n')
             

praisonai-0.0.53/praisonai/ui/context.py

@@ -0,0 +1,201 @@
+import os
+import fnmatch
+import re
+import yaml
+from pathlib import Path
+import logging
+
+# Set up logging
+logger = logging.getLogger(__name__)
+log_level = os.getenv("LOGLEVEL", "INFO").upper()
+logger.handlers = []
+
+# Set up logging to console
+console_handler = logging.StreamHandler()
+console_handler.setLevel(log_level)
+console_formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
+console_handler.setFormatter(console_formatter)
+logger.addHandler(console_handler)
+
+# Set the logging level for the logger
+logger.setLevel(log_level)
+
+class ContextGatherer:
+    def __init__(self, directory='.', output_file='context.txt',
+                 relevant_extensions=None, max_file_size=1_000_000, max_tokens=900000):
+        self.directory = directory
+        self.output_file = output_file
+        self.relevant_extensions = relevant_extensions or [
+            '.py', '.js', '.ts', '.java', '.rb', '.php', '.pl', '.pm', '.c', '.h',
+            '.cpp', '.hpp', '.cs', '.vb', '.swift', '.kt', '.m', '.mm', '.go', '.rs',
+            '.hs', '.r', '.lua', '.sh', '.bat', '.clj', '.scala', '.erl', '.ex',
+            '.ml', '.fs', '.groovy', '.jsm', '.jsx', '.tsx', '.yaml'
+        ]
+        self.max_file_size = max_file_size
+        self.max_tokens = int(os.getenv("PRAISONAI_MAX_TOKENS", max_tokens))
+        self.ignore_patterns = self.get_ignore_patterns()
+
+    def get_ignore_patterns(self):
+        """
+        Loads ignore patterns from various sources, prioritizing them in 
+        the following order:
+            1. .praisonignore
+            2. settings.yaml (under code.ignore_files)
+            3. PRAISONAI_IGNORE_FILES environment variable
+            4. .gitignore
+            5. Default patterns
+        """
+        ignore_patterns = []
+
+        def load_from_file(filepath):
+            if os.path.exists(filepath):
+                with open(filepath, 'r') as f:
+                    ignore_patterns.extend(
+                        line.strip() for line in f 
+                        if line.strip() and not line.startswith('#')
+                    )
+
+        # 1. Load from .praisonignore
+        load_from_file(os.path.join(self.directory, '.praisonignore'))
+
+        # 2. Load from settings.yaml
+        settings_path = os.path.join(self.directory, 'settings.yaml')
+        if os.path.exists(settings_path):
+            with open(settings_path, 'r') as f:
+                settings = yaml.safe_load(f)
+                if 'code' in settings and 'ignore_files' in settings['code']:
+                    ignore_patterns.extend(settings['code']['ignore_files'])
+
+        # 3. Load from environment variable
+        ignore_files_env = os.getenv("PRAISONAI_IGNORE_FILES")
+        if ignore_files_env:
+            ignore_patterns.extend(ignore_files_env.split(","))
+
+        # 4. Load from .gitignore
+        load_from_file(os.path.join(self.directory, '.gitignore'))
+
+        # 5. Default patterns (only if no patterns loaded from above sources)
+        if not ignore_patterns:
+            ignore_patterns = [
+                ".*", "*.pyc", "__pycache__", ".git", ".gitignore", ".vscode",
+                ".idea", ".DS_Store", "*.lock", "*.pyc", ".env", "docs", "tests", 
+                "test", "tmp", "temp", "*.txt", "*.md", "*.json", "*.csv", "*.tsv",
+                "public", "*.sql", "*.sqlite", "*.db", "*.db3", "*.sqlite3", 
+                "*.log", "*.zip", "*.gz", "*.tar", "*.rar", "*.7z", "*.pdf", 
+                "*.jpg", "*.jpeg", "*.png", "*.gif", "*.svg", "cookbooks", 
+                "assets", "__pycache__", "dist", "build", "node_modules", "venv"
+            ]
+            logger.debug(f"Using default ignore patterns: {ignore_patterns}")
+
+        # Modify patterns to match directories and add leading '*' if necessary
+        modified_ignore_patterns = [
+            '*' + pattern if not pattern.startswith('.') and not pattern.startswith('*') else pattern
+            for pattern in ignore_patterns
+        ]
+        logger.debug(f"Final ignore patterns: {modified_ignore_patterns}")
+        return modified_ignore_patterns
+
+    def should_ignore(self, file_path):
+        """
+        Check if a file or directory should be ignored based on patterns.
+        Handles both file names and directory names for more comprehensive filtering.
+        """
+        relative_path = os.path.relpath(file_path, self.directory)
+        if relative_path.startswith('.'):
+            return True
+        for pattern in self.ignore_patterns:
+            if fnmatch.fnmatch(relative_path, pattern) or \
+               fnmatch.fnmatch(os.path.basename(file_path), pattern):
+                return True
+        return False
+
+    def is_relevant_file(self, file_path):
+        """Determine if a file is relevant for the context."""
+        return os.path.isfile(file_path) and \
+               os.path.getsize(file_path) <= self.max_file_size and \
+               any(file_path.endswith(ext) for ext in self.relevant_extensions)
+
+    def gather_context(self):
+        """Gather context from relevant files, respecting ignore patterns."""
+        context = []
+        total_files = 0
+        processed_files = 0
+
+        for root, dirs, files in os.walk(self.directory):
+            total_files += len(files) 
+            dirs[:] = [d for d in dirs if not self.should_ignore(os.path.join(root, d))]
+            for file in files:
+                file_path = os.path.join(root, file)
+                if not self.should_ignore(file_path) and self.is_relevant_file(file_path):
+                    try:
+                        with open(file_path, 'r', encoding='utf-8') as f:
+                            content = f.read()
+                            context.append(f"File: {file_path}\n\n{content}\n\n{'='*50}\n")
+                    except Exception as e:
+                        logger.error(f"Error reading {file_path}: {e}")
+                processed_files += 1
+                print(f"\rProcessed {processed_files}/{total_files} files", end="", flush=True)
+        print()  # New line after progress indicator
+        return '\n'.join(context)
+
+    def count_tokens(self, text):
+        """Count tokens using a simple whitespace-based tokenizer."""
+        return len(text.split()) 
+
+    def truncate_context(self, context):
+        """Truncate context to stay within the token limit."""
+        tokens = context.split()
+        if len(tokens) > self.max_tokens:
+            truncated_context = ' '.join(tokens[:self.max_tokens])
+            logger.warning("Context truncated due to token limit.")
+            return truncated_context
+        return context
+
+    def save_context(self, context):
+        """Save the gathered context to a file."""
+        with open(self.output_file, 'w', encoding='utf-8') as f:
+            f.write(context)
+
+    def get_context_tree(self):
+        """Generate a formatted tree structure of included files and folders."""
+        tree = []
+        start_dir = Path(self.directory)
+
+        def add_to_tree(path, prefix=''):
+            contents = sorted(path.iterdir())
+            pointers = [('└── ' if i == len(contents) - 1 else '├── ') for i in range(len(contents))]
+            for pointer, item in zip(pointers, contents):
+                # Use should_ignore for consistency
+                if self.should_ignore(item):
+                    continue
+
+                rel_path = item.relative_to(start_dir)
+                tree.append(f"{prefix}{pointer}{rel_path}")
+
+                if item.is_dir():
+                    add_to_tree(item, prefix + ('    ' if pointer == '└── ' else '│   '))
+
+        add_to_tree(start_dir)
+        return '\n'.join(tree)
+
+    def run(self):
+        """Execute the context gathering, truncation, and reporting."""
+        context = self.gather_context()
+        context = self.truncate_context(context)
+        token_count = self.count_tokens(context)
+        print(f"Context gathered successfully.")
+        print(f"Total number of tokens (estimated): {token_count}")
+        # self.save_context(context)
+        context_tree = self.get_context_tree()
+        logger.debug(f"Context tree:\n{context_tree}")
+        return context, token_count, context_tree
+
+def main():
+    gatherer = ContextGatherer()
+    context, token_count, context_tree = gatherer.run()
+    print(f"\nThe context contains approximately {token_count} tokens.")
+    print("First 500 characters of context:")
+    print(context[:500] + "...")
+
+if __name__ == "__main__":
+    main()

{praisonai-0.0.50 → praisonai-0.0.53}/pyproject.toml

@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "PraisonAI"
-version = "0.0.50"
+version = "0.0.53"
 description = "PraisonAI application combines AutoGen and CrewAI or similar frameworks into a low-code solution for building and managing multi-agent LLM systems, focusing on simplicity, customization, and efficient human-agent collaboration."
 authors = ["Mervin Praison"]
 license = ""
@@ -31,6 +31,7 @@ langchain-openai = {version = ">=0.1.7", optional = true}
 langchain-cohere = {version = ">=0.1.4", optional = true}
 litellm = {version = ">=1.41.8", optional = true}
 aiosqlite= {version = ">=0.20.0", optional = true}
+greenlet = {version = ">=3.0.3", optional = true}
 
 [tool.poetry.group.docs.dependencies]
 mkdocs = "*"
@@ -79,6 +80,7 @@ langchain-openai = ">=0.1.7"
 langchain-cohere = ">=0.1.4"
 litellm = ">=1.41.8"
 aiosqlite = ">=0.20.0"
+greenlet = ">=3.0.3"
 
 [build-system]
 requires = ["poetry-core"]
@@ -98,4 +100,4 @@ openai = ["langchain-openai"]
 anthropic = ["langchain-anthropic"]
 cohere = ["langchain-cohere"]
 chat = ["chainlit", "litellm"]
-code = ["chainlit", "litellm", "aiosqlite"]
+code = ["chainlit", "litellm", "aiosqlite", "greenlet"]

praisonai-0.0.50/praisonai/ui/context.py

@@ -1,172 +0,0 @@
-import os
-import fnmatch
-import re
-import yaml
-import logging
-# Set up logging
-logger = logging.getLogger(__name__)
-log_level = os.getenv("LOGLEVEL", "INFO").upper()
-logger.handlers = []
-
-# Set up logging to console
-console_handler = logging.StreamHandler()
-console_handler.setLevel(log_level)
-console_formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
-console_handler.setFormatter(console_formatter)
-logger.addHandler(console_handler)
-
-# Set the logging level for the logger
-logger.setLevel(log_level)
-
-class ContextGatherer:
-    def __init__(self, directory='.', output_file='context.txt',
-                 relevant_extensions=None, max_file_size=1_000_000, max_tokens=128000):
-        self.directory = directory
-        self.output_file = output_file
-        self.relevant_extensions = relevant_extensions or ['.py']
-        self.max_file_size = max_file_size
-        self.max_tokens = int(os.getenv("PRAISONAI_MAX_TOKENS", max_tokens))
-        self.ignore_patterns = self.get_ignore_patterns()
-
-    def get_ignore_patterns(self):
-        """Read .gitignore file and return ignore patterns."""
-        
-        settings_path = os.path.join(self.directory, 'settings.yaml')
-        if os.path.exists(settings_path):
-            with open(settings_path, 'r') as f:
-                settings = yaml.safe_load(f)
-                if 'code' in settings and 'ignore_files' in settings['code']:
-                    logger.debug(f"Ignored settings.yaml files: {settings['code']['ignore_files']}")
-                    return settings['code']['ignore_files']
-                
-        # If settings.yaml doesn't exist, get from env variable
-        ignore_files_env = os.getenv("PRAISONAI_IGNORE_FILES")
-        if ignore_files_env:
-            logger.debug(f"Ignored PRAISONAI_IGNORE_FILES ENV files: {ignore_files_env}")
-            return ignore_files_env.split(",")
-                
-        default_patterns = [".*", "*.pyc", "__pycache__", ".git", ".gitignore", ".vscode",
-                            ".idea", ".DS_Store", "*.lock", "*.pyc", ".env",
-                            "docs", "tests", "test", "tmp", "temp", 
-                            "*.txt", "*.md", "*.json", "*.csv", "*.tsv","public",
-                            "*.sql", "*.sqlite", "*.db", "*.db3", "*.sqlite3", "*.log", "*.zip", "*.gz",
-                            "*.tar", "*.rar", "*.7z", "*.pdf", "*.jpg", "*.jpeg", "*.png", "*.gif", "*.svg",
-                            "cookbooks", "assets", "__pycache__", "dist", "build", "node_modules", "venv",]
-        gitignore_path = os.path.join(self.directory, '.gitignore')
-        if os.path.exists(gitignore_path):
-            with open(gitignore_path, 'r') as f:
-                gitignore_patterns = [line.strip() for line in f if line.strip() and not line.startswith('#')]
-            logger.debug(f"Ignored gitignore and default files: {ignore_files_env}")
-            return list(set(default_patterns + gitignore_patterns))
-        return default_patterns
-
-    def should_ignore(self, file_path):
-        """Check if a file should be ignored based on patterns."""
-        relative_path = os.path.relpath(file_path, self.directory)
-        if relative_path.startswith('.'):
-            return True
-        for pattern in self.ignore_patterns:
-            if fnmatch.fnmatch(relative_path, pattern):
-                return True
-        return False
-
-    def is_relevant_file(self, file_path):
-        """Determine if a file is relevant for the context."""
-        if os.path.getsize(file_path) > self.max_file_size:
-            return False
-        return any(file_path.endswith(ext) for ext in self.relevant_extensions)
-
-    def gather_context(self):
-        """Gather context from relevant files in the directory."""
-        context = []
-        total_files = sum(len(files) for _, _, files in os.walk(self.directory))
-        processed_files = 0
-
-        for root, dirs, files in os.walk(self.directory):
-            dirs[:] = [d for d in dirs if not self.should_ignore(os.path.join(root, d))]
-            for file in files:
-                file_path = os.path.join(root, file)
-                if not self.should_ignore(file_path) and self.is_relevant_file(file_path):
-                    try:
-                        with open(file_path, 'r', encoding='utf-8') as f:
-                            content = f.read()
-                            context.append(f"File: {file_path}\n\n{content}\n\n{'='*50}\n")
-                    except Exception as e:
-                        print(f"Error reading {file_path}: {e}")
-                processed_files += 1
-                print(f"\rProcessed {processed_files}/{total_files} files", end="", flush=True)
-        print()  # New line after progress indicator
-        return '\n'.join(context)
-
-    def count_tokens(self, text):
-        """Count the number of tokens in the given text using a simple tokenizer."""
-        # Split on whitespace and punctuation
-        tokens = re.findall(r'\b\w+\b|[^\w\s]', text)
-        return len(tokens)
-
-    def truncate_context(self, context):
-        """Truncate context to fit within the specified token limit."""
-        tokens = re.findall(r'\b\w+\b|[^\w\s]', context)
-        if len(tokens) > self.max_tokens:
-            truncated_tokens = tokens[:self.max_tokens]
-            return ' '.join(truncated_tokens)
-        return context
-
-    def save_context(self, context):
-        """Save the gathered context to a file."""
-        with open(self.output_file, 'w', encoding='utf-8') as f:
-            f.write(context)
-            
-    def get_context_tree(self):
-        """Generate a formatted tree structure of the folder, including only relevant files."""
-        tree = []
-        start_dir = os.path.abspath(self.directory)
-        
-        def add_to_tree(path, prefix=''):
-            contents = sorted(os.listdir(path))
-            pointers = [('└── ' if i == len(contents) - 1 else '├── ') for i in range(len(contents))]
-            for pointer, name in zip(pointers, contents):
-                full_path = os.path.join(path, name)
-                if self.should_ignore(full_path):
-                    continue
-                
-                rel_path = os.path.relpath(full_path, start_dir)
-                tree.append(f"{prefix}{pointer}{name}")
-                
-                if os.path.isdir(full_path):
-                    add_to_tree(full_path, prefix + ('    ' if pointer == '└── ' else '│   '))
-                elif self.is_relevant_file(full_path):
-                    continue  # We've already added the file to the tree
-
-        add_to_tree(start_dir)
-        return '\n'.join(tree)
-
-    def run(self):
-        """Run the context gathering process and return the context and token count."""
-        context = self.gather_context()
-        context = self.truncate_context(context)
-        token_count = self.count_tokens(context)
-        print(f"Context gathered successfully.")
-        print(f"Total number of tokens (estimated): {token_count}")
-        # self.save_context(context)
-        context_tree = self.get_context_tree()
-        print("\nContext Tree Structure:")
-        print(context_tree)
-        
-        return context, token_count, context_tree
-
-def main():
-    gatherer = ContextGatherer(
-        directory='.',
-        output_file='context.txt',
-        relevant_extensions=['.py'],
-        max_file_size=500_000,  # 500KB
-        max_tokens=60000
-    )
-    context, token_count, context_tree = gatherer.run()
-    print(f"\nThe context contains approximately {token_count} tokens.")
-    print("First 500 characters of context:")
-    print(context[:500] + "...")
-
-if __name__ == "__main__":
-    main()