ws-bom-robot-app 0.0.79__tar.gz → 0.0.81__tar.gz

{ws_bom_robot_app-0.0.79/ws_bom_robot_app.egg-info → ws_bom_robot_app-0.0.81}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: ws_bom_robot_app
-Version: 0.0.79
+Version: 0.0.81
 Summary: A FastAPI application serving ws bom/robot/llm platform ai.
 Home-page: https://github.com/websolutespa/bom
 Author: Websolute Spa
@@ -83,18 +83,30 @@ from ws_bom_robot_app import main
 app = main.app
 ```
 
-FIll `.env` with the following code:
-
-```env
-#robot_env=local/development/production
-robot_env=local
-robot_user='[user]'
-robot_password='[pwd]'
-robot_data_folder='./.data'
-robot_cms_auth='[auth]'
-robot_cms_host='https://[DOMAIN]'
-robot_cms_db_folder=llmVectorDb
-robot_cms_files_folder=llmKbFile
+Create a `.env` file in the root directory with the following configuration:
+
+```properties
+# robot configuration
+robot_env=development
+robot_user=your_username
+USER_AGENT=ws-bom-robot-app
+
+# cms (bowl) configuration
+robot_cms_host='http://localhost:4000'
+robot_cms_auth='users API-Key your-api-key-here'
+
+# llm providers: fill one or more of these with your API keys
+DEEPSEEK_API_KEY="your-deepseek-api-key"
+OPENAI_API_KEY="your-openai-api-key"
+GOOGLE_API_KEY="your-google-api-key"
+ANTHROPIC_API_KEY="your-anthropic-api-key"
+GROQ_API_KEY="your-groq-api-key"
+# ibm
+WATSONX_URL="https://eu-gb.ml.cloud.ibm.com"
+WATSONX_APIKEY="your-watsonx-api-key"
+WATSONX_PROJECTID="your-watsonx-project-id"
+# gvertex: ensure to mount the file in docker
+GOOGLE_APPLICATION_CREDENTIALS="./.data/secrets/google-credentials.json" 
 ```
 
 ## 🚀 Run the app
@@ -125,38 +137,39 @@ robot_cms_files_folder=llmKbFile
 - [swagger](http://localhost:6001/docs)
 - [redoc](http://localhost:6001/redoc)
 
-### 💬 multimodal chat
+---
 
-The multimodal message allows users to interact with the application using both text and media files. 
-`robot` accept multimodal input in a uniform way, regarding the llm provider used. Can also be used the llm/model specific input format.
+## 🐳 Docker
 
-- simple message
+dockerize base image
 
-```json
-{
-  "role": "user",
-  "content": "What is the capital of France?"
-}
+```pwsh
+<# cpu #>
+docker build -f Dockerfile-robot-base-cpu -t ghcr.io/websolutespa/ws-bom-robot-base:cpu .
+docker push ghcr.io/websolutespa/ws-bom-robot-base:cpu
+<# gpu #>
+docker build -f Dockerfile-robot-base-gpu -t ghcr.io/websolutespa/ws-bom-robot-base:gpu .
+docker push ghcr.io/websolutespa/ws-bom-robot-base:gpu
 ```
 
-- multimodal message
-
-```json
-{
-  "role": "user",
-  "content": [
-    {"type": "text", "text": "Read carefully all the attachments, analize the content and provide a summary for each one:"},        
-    {"type": "image", "url": "https://www.example.com/image/foo.jpg"},        
-    {"type": "file", "url": "https://www.example.com/pdf/bar.pdf"},   
-    {"type": "file", "url": "data:plain/text;base64,CiAgICAgIF9fX19fCiAgICAgLyAgIC..."}, # base64 encoded file     
-    {"type": "media", "mime_type": "plain/text", "data": "CiAgICAgIF9fX19fCiAgICAgLyAgIC..."} # google/gemini specific input format    
-  ]          
-}
+dockerize app
+
+```pwsh
+docker build -f Dockerfile -t ws-bom-robot-app .
+docker run --rm --name ws-bom-robot-app -d --env-file .env -p 6001:6001 ws-bom-robot-app
+```
+
+docker run mounted to src (dev mode)
+
+```pwsh
+docker run --rm --name ws-bom-robot-app-src -d --env-file .env -v "$(pwd)/ws_bom_robot_app:/app/ws_bom_robot_app" -v "$(pwd)/.data:/app/.data" -v "$(pwd)/tmp:/tmp"  -p 6001:6001 ws-bom-robot-app fastapi dev ./ws_bom_robot_app/main.py --host 0.0.0.0 --port 6001
 ```
 
 ---
 
-## 🔖 Windows requirements  
+## 🔖 Windows requirements (for RAG functionality only)
+
+> ⚠️ While it's strongly recommended to use a docker container for development, you can run the app on Windows with the following requirements
 
 ### libmagic (mandatory)
 
@@ -267,7 +280,7 @@ prospector ./ws_bom_robot_app -t dodgy -t bandit
 prospector ./ws_bom_robot_app -t pyroma
 ```
 
-lauch pytest
+#### 🧪 run tests
 
 ```pwsh
 !py -m pip install -U pytest pytest-asyncio pytest-mock pytest-cov pyclean
@@ -278,36 +291,12 @@ pytest --cov=ws_bom_robot_app --log-cli-level=info
 # pytest --cov=ws_bom_robot_app --log-cli-level=info ./tests/app/llm/vector_store/db
 ```
 
-launch debugger
+#### 🐞 start debugger
 
 ```pwsh
 streamlit run debugger.py --server.port 8051
 ```
 
-dockerize base image
-
-```pwsh
-<# cpu #>
-docker build -f Dockerfile-robot-base-cpu -t ghcr.io/websolutespa/ws-bom-robot-base:cpu .
-docker push ghcr.io/websolutespa/ws-bom-robot-base:cpu
-<# gpu #>
-docker build -f Dockerfile-robot-base-gpu -t ghcr.io/websolutespa/ws-bom-robot-base:gpu .
-docker push ghcr.io/websolutespa/ws-bom-robot-base:gpu
-```
-
-dockerize app
-
-```pwsh
-docker build -f Dockerfile -t ws-bom-robot-app .
-docker run --rm --name ws-bom-robot-app -d -p 6001:6001 ws-bom-robot-app
-```
-
-docker run mounted to src
-
-```pwsh
-docker run --rm --name ws-bom-robot-app-src -d -v "$(pwd)/ws_bom_robot_app:/app/ws_bom_robot_app" -v "$(pwd)/.data:/app/.data" -v "$(pwd)/tmp:/tmp"  -p 6001:6001 ws-bom-robot-app
-```
-
 ### ✈️ publish
 
 - [testpypi](https://test.pypi.org/project/ws-bom-robot-app/)

{ws_bom_robot_app-0.0.79 → ws_bom_robot_app-0.0.81}/README.md

@@ -17,18 +17,30 @@ from ws_bom_robot_app import main
 app = main.app
 ```
 
-FIll `.env` with the following code:
-
-```env
-#robot_env=local/development/production
-robot_env=local
-robot_user='[user]'
-robot_password='[pwd]'
-robot_data_folder='./.data'
-robot_cms_auth='[auth]'
-robot_cms_host='https://[DOMAIN]'
-robot_cms_db_folder=llmVectorDb
-robot_cms_files_folder=llmKbFile
+Create a `.env` file in the root directory with the following configuration:
+
+```properties
+# robot configuration
+robot_env=development
+robot_user=your_username
+USER_AGENT=ws-bom-robot-app
+
+# cms (bowl) configuration
+robot_cms_host='http://localhost:4000'
+robot_cms_auth='users API-Key your-api-key-here'
+
+# llm providers: fill one or more of these with your API keys
+DEEPSEEK_API_KEY="your-deepseek-api-key"
+OPENAI_API_KEY="your-openai-api-key"
+GOOGLE_API_KEY="your-google-api-key"
+ANTHROPIC_API_KEY="your-anthropic-api-key"
+GROQ_API_KEY="your-groq-api-key"
+# ibm
+WATSONX_URL="https://eu-gb.ml.cloud.ibm.com"
+WATSONX_APIKEY="your-watsonx-api-key"
+WATSONX_PROJECTID="your-watsonx-project-id"
+# gvertex: ensure to mount the file in docker
+GOOGLE_APPLICATION_CREDENTIALS="./.data/secrets/google-credentials.json" 
 ```
 
 ## 🚀 Run the app
@@ -59,38 +71,39 @@ robot_cms_files_folder=llmKbFile
 - [swagger](http://localhost:6001/docs)
 - [redoc](http://localhost:6001/redoc)
 
-### 💬 multimodal chat
+---
 
-The multimodal message allows users to interact with the application using both text and media files. 
-`robot` accept multimodal input in a uniform way, regarding the llm provider used. Can also be used the llm/model specific input format.
+## 🐳 Docker
 
-- simple message
+dockerize base image
 
-```json
-{
-  "role": "user",
-  "content": "What is the capital of France?"
-}
+```pwsh
+<# cpu #>
+docker build -f Dockerfile-robot-base-cpu -t ghcr.io/websolutespa/ws-bom-robot-base:cpu .
+docker push ghcr.io/websolutespa/ws-bom-robot-base:cpu
+<# gpu #>
+docker build -f Dockerfile-robot-base-gpu -t ghcr.io/websolutespa/ws-bom-robot-base:gpu .
+docker push ghcr.io/websolutespa/ws-bom-robot-base:gpu
 ```
 
-- multimodal message
-
-```json
-{
-  "role": "user",
-  "content": [
-    {"type": "text", "text": "Read carefully all the attachments, analize the content and provide a summary for each one:"},        
-    {"type": "image", "url": "https://www.example.com/image/foo.jpg"},        
-    {"type": "file", "url": "https://www.example.com/pdf/bar.pdf"},   
-    {"type": "file", "url": "data:plain/text;base64,CiAgICAgIF9fX19fCiAgICAgLyAgIC..."}, # base64 encoded file     
-    {"type": "media", "mime_type": "plain/text", "data": "CiAgICAgIF9fX19fCiAgICAgLyAgIC..."} # google/gemini specific input format    
-  ]          
-}
+dockerize app
+
+```pwsh
+docker build -f Dockerfile -t ws-bom-robot-app .
+docker run --rm --name ws-bom-robot-app -d --env-file .env -p 6001:6001 ws-bom-robot-app
+```
+
+docker run mounted to src (dev mode)
+
+```pwsh
+docker run --rm --name ws-bom-robot-app-src -d --env-file .env -v "$(pwd)/ws_bom_robot_app:/app/ws_bom_robot_app" -v "$(pwd)/.data:/app/.data" -v "$(pwd)/tmp:/tmp"  -p 6001:6001 ws-bom-robot-app fastapi dev ./ws_bom_robot_app/main.py --host 0.0.0.0 --port 6001
 ```
 
 ---
 
-## 🔖 Windows requirements  
+## 🔖 Windows requirements (for RAG functionality only)
+
+> ⚠️ While it's strongly recommended to use a docker container for development, you can run the app on Windows with the following requirements
 
 ### libmagic (mandatory)
 
@@ -201,7 +214,7 @@ prospector ./ws_bom_robot_app -t dodgy -t bandit
 prospector ./ws_bom_robot_app -t pyroma
 ```
 
-lauch pytest
+#### 🧪 run tests
 
 ```pwsh
 !py -m pip install -U pytest pytest-asyncio pytest-mock pytest-cov pyclean
@@ -212,36 +225,12 @@ pytest --cov=ws_bom_robot_app --log-cli-level=info
 # pytest --cov=ws_bom_robot_app --log-cli-level=info ./tests/app/llm/vector_store/db
 ```
 
-launch debugger
+#### 🐞 start debugger
 
 ```pwsh
 streamlit run debugger.py --server.port 8051
 ```
 
-dockerize base image
-
-```pwsh
-<# cpu #>
-docker build -f Dockerfile-robot-base-cpu -t ghcr.io/websolutespa/ws-bom-robot-base:cpu .
-docker push ghcr.io/websolutespa/ws-bom-robot-base:cpu
-<# gpu #>
-docker build -f Dockerfile-robot-base-gpu -t ghcr.io/websolutespa/ws-bom-robot-base:gpu .
-docker push ghcr.io/websolutespa/ws-bom-robot-base:gpu
-```
-
-dockerize app
-
-```pwsh
-docker build -f Dockerfile -t ws-bom-robot-app .
-docker run --rm --name ws-bom-robot-app -d -p 6001:6001 ws-bom-robot-app
-```
-
-docker run mounted to src
-
-```pwsh
-docker run --rm --name ws-bom-robot-app-src -d -v "$(pwd)/ws_bom_robot_app:/app/ws_bom_robot_app" -v "$(pwd)/.data:/app/.data" -v "$(pwd)/tmp:/tmp"  -p 6001:6001 ws-bom-robot-app
-```
-
 ### ✈️ publish
 
 - [testpypi](https://test.pypi.org/project/ws-bom-robot-app/)

{ws_bom_robot_app-0.0.79 → ws_bom_robot_app-0.0.81}/setup.py

@@ -4,7 +4,7 @@ _requirements = [line.split('#')[0].strip() for line in open("requirements.txt")
 
 setup(
     name="ws_bom_robot_app",
-    version="0.0.79",
+    version="0.0.81",
     description="A FastAPI application serving ws bom/robot/llm platform ai.",
     long_description=open("README.md", encoding='utf-8').read(),
     long_description_content_type="text/markdown",

{ws_bom_robot_app-0.0.79 → ws_bom_robot_app-0.0.81}/ws_bom_robot_app/config.py

@@ -32,7 +32,7 @@ class Settings(BaseSettings):
     WATSONX_URL: str = ''
     WATSONX_APIKEY: str = ''
     WATSONX_PROJECTID: str = ''
-    NEBULY_API_URL: str =''
+    NEBULY_API_URL: str ='https://backend.nebuly.com/'
     GOOGLE_APPLICATION_CREDENTIALS: str = '' # path to google credentials iam file, e.d. ./.secrets/google-credentials.json
     model_config = ConfigDict(
         env_file='./.env',

{ws_bom_robot_app-0.0.79 → ws_bom_robot_app-0.0.81}/ws_bom_robot_app/llm/agent_description.py

@@ -1,123 +1,123 @@
-import json, requests, re
-from typing import Any
-from abc import ABC, abstractmethod
-from langchain_core.prompts import ChatPromptTemplate
-from langchain_core.messages import AIMessage
-from langchain_core.runnables import RunnableSerializable
-from langchain_core.runnables import RunnableLambda
-from bs4 import BeautifulSoup
-from ws_bom_robot_app.llm.models.api import LlmRules
-from ws_bom_robot_app.llm.providers.llm_manager import LlmInterface
-from ws_bom_robot_app.llm.utils.agent import get_rules
-
-# SafeDict helper class
-class SafeDict(dict):
-    def __missing__(self, key):
-        return ''
-
-# Strategy Interface
-class AgentDescriptorStrategy(ABC):
-    @abstractmethod
-    def enrich_prompt(self, prompt: str, input: dict) -> str:
-        pass
-
-    @abstractmethod
-    def rule_input(self, input: dict) -> str:
-        pass
-
-# Concrete Strategy for Default Agent
-class DefaultAgentDescriptor(AgentDescriptorStrategy):
-    def enrich_prompt(self, prompt: str, input: dict) -> str:
-        # Default enrichment logic (could be minimal or no-op)
-        return prompt.format_map(SafeDict(input))
-
-    def rule_input(self, input: dict) -> str:
-        return input.get('content', "")
-
-# Concrete Strategy for URL2Text Agent
-class URL2TextAgentDescriptor(AgentDescriptorStrategy):
-    def enrich_prompt(self, prompt: str, input: dict) -> str:
-        input["context"] = self._get_page_text(input)
-        return prompt.format_map(SafeDict(input))
-
-    def rule_input(self, input: dict) -> str:
-        return input.get('context', "")
-
-    def _get_page_text(self, input: dict) -> str:
-        url = input.get("content", "")
-        exclusions = input.get("exclude", {})
-        response = requests.get(url)
-        response.raise_for_status()
-        soup = BeautifulSoup(response.content, 'html5lib')
-        classes_to_exclude = exclusions.get("classes", [])
-        ids_to_exclude = exclusions.get("ids", [])
-        for class_name in classes_to_exclude:
-            for element in soup.find_all(class_=class_name):
-                element.extract()
-        for id_name in ids_to_exclude:
-            for element in soup.find_all(id=id_name):
-                element.extract()
-        for script in soup(["script", "noscript", "style", "head", "footer", "iframe"]):
-            script.extract()
-        return re.sub(' +', ' ', soup.get_text())
-
-
-class AgentDescriptor:
-    # Dictionary to hold all agent strategies
-    _list: dict[str,AgentDescriptorStrategy] = {
-        "default": DefaultAgentDescriptor(),
-        "url2text": URL2TextAgentDescriptor(),
-    }
-
-    # Functions to manage strategies
-    @staticmethod
-    def add_strategy(name: str, strategy: AgentDescriptorStrategy):
-        """_summary_
-            add a new strategy to the dictionary
-        Args:
-            name (str): name of the strategy, in lowercase
-            strategy (AgentDescriptorStrategy): class implementing the strategy
-        Examples:
-            AgentDescriptor.add_strategy("custom_agent_descriptor", CustomAgentDescriptor())
-        """
-        AgentDescriptor._list[name.lower()] = strategy
-
-    @staticmethod
-    def get_strategy(name: str) -> AgentDescriptorStrategy:
-        return AgentDescriptor._list.get(name.lower(), DefaultAgentDescriptor())
-
-    def __init__(self, llm: LlmInterface, prompt: str, mode: str, rules: LlmRules = None):
-        self.__prompt = prompt
-        self.__llm = llm
-        self.rules= rules
-        self.strategy = self.get_strategy(mode)  # Selects the strategy from the dictionary
-
-    async def __create_prompt(self, input_dict: dict):
-        input_data = json.loads(input_dict.get("input", {}))
-        system = self.strategy.enrich_prompt(self.__prompt, input_data)
-        if self.rules:
-            rule_input = self.strategy.rule_input(input_data)
-            rules_prompt = await get_rules(self.__llm.get_embeddings(), self.rules, rule_input)
-            system += rules_prompt
-        return ChatPromptTemplate.from_messages(
-            [
-                ("system", system),
-                ("user", input_data.get("content", ""))
-            ]
-        )
-
-    def __create_agent_descriptor(self, content) -> RunnableSerializable[Any, Any]:
-        content = json.loads(content)
-        agent = (
-            {
-                "input": lambda x: x["input"],
-            }
-            | RunnableLambda(self.__create_prompt)
-            | self.__llm.get_llm()
-        )
-        return agent
-
-    async def run_agent(self, content) -> Any:
-        agent_descriptor = self.__create_agent_descriptor(content)
-        response: AIMessage = await agent_descriptor.ainvoke({"input": content})
-        return response
+import json, requests, re
+from typing import Any
+from abc import ABC, abstractmethod
+from langchain_core.prompts import ChatPromptTemplate
+from langchain_core.messages import AIMessage
+from langchain_core.runnables import RunnableSerializable
+from langchain_core.runnables import RunnableLambda
+from bs4 import BeautifulSoup
+from ws_bom_robot_app.llm.models.api import LlmRules
+from ws_bom_robot_app.llm.providers.llm_manager import LlmInterface
+from ws_bom_robot_app.llm.utils.agent import get_rules
+
+# SafeDict helper class
+class SafeDict(dict):
+    def __missing__(self, key):
+        return ''
+
+# Strategy Interface
+class AgentDescriptorStrategy(ABC):
+    @abstractmethod
+    def enrich_prompt(self, prompt: str, input: dict) -> str:
+        pass
+
+    @abstractmethod
+    def rule_input(self, input: dict) -> str:
+        pass
+
+# Concrete Strategy for Default Agent
+class DefaultAgentDescriptor(AgentDescriptorStrategy):
+    def enrich_prompt(self, prompt: str, input: dict) -> str:
+        # Default enrichment logic (could be minimal or no-op)
+        return prompt.format_map(SafeDict(input))
+
+    def rule_input(self, input: dict) -> str:
+        return input.get('content', "")
+
+# Concrete Strategy for URL2Text Agent
+class URL2TextAgentDescriptor(AgentDescriptorStrategy):
+    def enrich_prompt(self, prompt: str, input: dict) -> str:
+        input["context"] = self._get_page_text(input)
+        return prompt.format_map(SafeDict(input))
+
+    def rule_input(self, input: dict) -> str:
+        return input.get('context', "")
+
+    def _get_page_text(self, input: dict) -> str:
+        url = input.get("content", "")
+        exclusions = input.get("exclude", {})
+        response = requests.get(url)
+        response.raise_for_status()
+        soup = BeautifulSoup(response.content, 'html5lib')
+        classes_to_exclude = exclusions.get("classes", [])
+        ids_to_exclude = exclusions.get("ids", [])
+        for class_name in classes_to_exclude:
+            for element in soup.find_all(class_=class_name):
+                element.extract()
+        for id_name in ids_to_exclude:
+            for element in soup.find_all(id=id_name):
+                element.extract()
+        for script in soup(["script", "noscript", "style", "head", "footer", "iframe"]):
+            script.extract()
+        return re.sub(' +', ' ', soup.get_text())
+
+
+class AgentDescriptor:
+    # Dictionary to hold all agent strategies
+    _list: dict[str,AgentDescriptorStrategy] = {
+        "default": DefaultAgentDescriptor(),
+        "url2text": URL2TextAgentDescriptor(),
+    }
+
+    # Functions to manage strategies
+    @staticmethod
+    def add_strategy(name: str, strategy: AgentDescriptorStrategy):
+        """_summary_
+            add a new strategy to the dictionary
+        Args:
+            name (str): name of the strategy, in lowercase
+            strategy (AgentDescriptorStrategy): class implementing the strategy
+        Examples:
+            AgentDescriptor.add_strategy("custom_agent_descriptor", CustomAgentDescriptor())
+        """
+        AgentDescriptor._list[name.lower()] = strategy
+
+    @staticmethod
+    def get_strategy(name: str) -> AgentDescriptorStrategy:
+        return AgentDescriptor._list.get(name.lower(), DefaultAgentDescriptor())
+
+    def __init__(self, llm: LlmInterface, prompt: str, mode: str, rules: LlmRules = None):
+        self.__prompt = prompt
+        self.__llm = llm
+        self.rules= rules
+        self.strategy = self.get_strategy(mode)  # Selects the strategy from the dictionary
+
+    async def __create_prompt(self, input_dict: dict):
+        input_data = json.loads(input_dict.get("input", {}))
+        system = self.strategy.enrich_prompt(self.__prompt, input_data)
+        if self.rules:
+            rule_input = self.strategy.rule_input(input_data)
+            rules_prompt = await get_rules(self.__llm.get_embeddings(), self.rules, rule_input)
+            system += rules_prompt
+        return ChatPromptTemplate.from_messages(
+            [
+                ("system", system),
+                ("user", input_data.get("content", ""))
+            ]
+        )
+
+    def __create_agent_descriptor(self, content) -> RunnableSerializable[Any, Any]:
+        content = json.loads(content)
+        agent = (
+            {
+                "input": lambda x: x["input"],
+            }
+            | RunnableLambda(self.__create_prompt)
+            | self.__llm.get_llm()
+        )
+        return agent
+
+    async def run_agent(self, content) -> Any:
+        agent_descriptor = self.__create_agent_descriptor(content)
+        response: AIMessage = await agent_descriptor.ainvoke({"input": content})
+        return response