ragaai-catalyst 2.0.6b1__tar.gz → 2.0.7b1__tar.gz

{ragaai_catalyst-2.0.6b1/ragaai_catalyst.egg-info → ragaai_catalyst-2.0.7b1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: ragaai_catalyst
-Version: 2.0.6b1
+Version: 2.0.7b1
 Summary: RAGA AI CATALYST
 Author-email: Kiran Scaria <kiran.scaria@raga.ai>, Kedar Gaikwad <kedar.gaikwad@raga.ai>, Dushyant Mahajan <dushyant.mahajan@raga.ai>, Siddhartha Kosti <siddhartha.kosti@raga.ai>, Ritika Goel <ritika.goel@raga.ai>, Vijay Chaurasia <vijay.chaurasia@raga.ai>
 Requires-Python: >=3.9
@@ -28,6 +28,7 @@ Requires-Dist: litellm==1.51.1
 Requires-Dist: tenacity==8.3.0
 Requires-Dist: tqdm>=4.66.5
 Requires-Dist: llama-index==0.10.0
+Requires-Dist: pyopenssl==24.2.1
 Provides-Extra: dev
 Requires-Dist: pytest; extra == "dev"
 Requires-Dist: pytest-cov; extra == "dev"
@@ -38,7 +39,7 @@ Requires-Dist: flake8; extra == "dev"
 
 # RagaAI Catalyst
 
-RagaAI Catalyst is a powerful tool for managing and optimizing LLM projects. It provides functionalities for project management, trace recording, and experiment management, allowing you to fine-tune and evaluate your LLM applications effectively.
+RagaAI Catalyst is a comprehensive platform designed to enhance the management and optimization of LLM projects. It offers a wide range of features, including project management, dataset management, evaluation management, trace management, prompt management, synthetic data generation, and guardrail management. These functionalities enable you to efficiently evaluate, and safeguard your LLM applications.
 
 ## Table of Contents
 
@@ -53,6 +54,7 @@ RagaAI Catalyst is a powerful tool for managing and optimizing LLM projects. It
     - [Trace Management](#trace-management)
     - [Prompt Management](#prompt-management)
     - [Synthetic Data Generation](#synthetic-data-generation)
+    - [Guardrail Management](#guardrail-management)
 
 ## Installation
 
@@ -144,6 +146,7 @@ evaluation = Evaluation(
 evaluation.list_metrics()
 
 # Add metrics to the experiment
+
 schema_mapping={
     'Query': 'prompt',
     'response': 'response',
@@ -201,8 +204,12 @@ tracer = Tracer(
 
 # Your code here
 
+
 # Stop the trace recording
 tracer.stop()
+
+# Get upload status
+tracer.get_upload_status()
 ```
 
 
@@ -237,7 +244,7 @@ print("variable:",variable)
 prompt_content = prompt.get_prompt_content()
 print("prompt_content:", prompt_content)
 
-# Compile a prompt with variables
+# Compile the prompt with variables
 compiled_prompt = prompt.compile(query="What's the weather?", context="sunny", llm_response="It's sunny today")
 print("Compiled prompt:", compiled_prompt)
 
@@ -280,7 +287,9 @@ sdg = SyntheticDataGeneration()
 text = sdg.process_document(input_data="file_path")
 
 # Generate results
-result = sdg.generate_qna(text, question_type ='simple',model_config={"provider":"openai","model":"gpt-4o-mini"},n=20)
+result = sdg.generate_qna(text, question_type ='complex',model_config={"provider":"openai","model":"openai/gpt-3.5-turbo"},n=5)
+
+print(result.head())
 
 # Get supported Q&A types
 sdg.get_supported_qna()
@@ -291,5 +300,87 @@ sdg.get_supported_providers()
 
 
 
+### Guardrail Management
+
+```py
+from ragaai_catalyst import GuardrailsManager
+
+# Initialize Guardrails Manager
+gdm = GuardrailsManager(project_name=project_name)
+
+# Get list of Guardrails available
+guardrails_list = gdm.list_guardrails()
+print('guardrails_list:', guardrails_list)
+
+# Get list of fail condition for guardrails
+fail_conditions = gdm.list_fail_condition()
+print('fail_conditions;', fail_conditions)
+
+#Get list of deployment ids
+deployment_list = gdm.list_deployment_ids()
+print('deployment_list:', deployment_list)
+
+# Get specific deployment id with guardrails information
+deployment_id_detail = gdm.get_deployment(17)
+print('deployment_id_detail:', deployment_id_detail)
+
+# Add guardrails to a deployment id
+guardrails_config = {"guardrailFailConditions": ["FAIL"],
+                     "deploymentFailCondition": "ALL_FAIL",
+                     "alternateResponse": "Your alternate response"}
+
+guardrails = [
+    {
+      "displayName": "Response_Evaluator",
+      "name": "Response Evaluator",
+      "config":{
+          "mappings": [{
+                        "schemaName": "Text",
+                        "variableName": "Response"
+                    }],
+          "params": {
+                    "isActive": {"value": False},
+                    "isHighRisk": {"value": True},
+                    "threshold": {"eq": 0},
+                    "competitors": {"value": ["Google","Amazon"]}
+                }
+      }
+    },
+    {
+      "displayName": "Regex_Check",
+      "name": "Regex Check",
+      "config":{
+          "mappings": [{
+                        "schemaName": "Text",
+                        "variableName": "Response"
+                    }],
+          "params":{
+              "isActive": {"value": False},
+              "isHighRisk": {"value": True},
+              "threshold": {"lt1": 1}
+          }
+      }
+    }
+]
+
+gdm.add_guardrails(deployment_id, guardrails, guardrails_config)
 
 
+# Import GuardExecutor
+from ragaai_catalyst import GuardExecutor
+
+# Initialise GuardExecutor with required params and Evaluate
+executor = GuardExecutor(deployment_id,gdm,field_map={'context':'document'})
+
+
+message={'role':'user',
+         'content':'What is the capital of France'
+        }
+prompt_params={'document':' France'}
+
+model_params = {'temperature':.7,'model':'gpt-4o-mini'}
+llm_caller = 'litellm'
+
+executor([message],prompt_params,model_params,llm_caller)
+
+```

{ragaai_catalyst-2.0.6b1 → ragaai_catalyst-2.0.7b1}/README.md

@@ -1,6 +1,6 @@
 # RagaAI Catalyst
 
-RagaAI Catalyst is a powerful tool for managing and optimizing LLM projects. It provides functionalities for project management, trace recording, and experiment management, allowing you to fine-tune and evaluate your LLM applications effectively.
+RagaAI Catalyst is a comprehensive platform designed to enhance the management and optimization of LLM projects. It offers a wide range of features, including project management, dataset management, evaluation management, trace management, prompt management, synthetic data generation, and guardrail management. These functionalities enable you to efficiently evaluate, and safeguard your LLM applications.
 
 ## Table of Contents
 
@@ -15,6 +15,7 @@ RagaAI Catalyst is a powerful tool for managing and optimizing LLM projects. It
     - [Trace Management](#trace-management)
     - [Prompt Management](#prompt-management)
     - [Synthetic Data Generation](#synthetic-data-generation)
+    - [Guardrail Management](#guardrail-management)
 
 ## Installation
 
@@ -106,6 +107,7 @@ evaluation = Evaluation(
 evaluation.list_metrics()
 
 # Add metrics to the experiment
+
 schema_mapping={
     'Query': 'prompt',
     'response': 'response',
@@ -163,8 +165,12 @@ tracer = Tracer(
 
 # Your code here
 
+
 # Stop the trace recording
 tracer.stop()
+
+# Get upload status
+tracer.get_upload_status()
 ```
 
 
@@ -199,7 +205,7 @@ print("variable:",variable)
 prompt_content = prompt.get_prompt_content()
 print("prompt_content:", prompt_content)
 
-# Compile a prompt with variables
+# Compile the prompt with variables
 compiled_prompt = prompt.compile(query="What's the weather?", context="sunny", llm_response="It's sunny today")
 print("Compiled prompt:", compiled_prompt)
 
@@ -242,7 +248,9 @@ sdg = SyntheticDataGeneration()
 text = sdg.process_document(input_data="file_path")
 
 # Generate results
-result = sdg.generate_qna(text, question_type ='simple',model_config={"provider":"openai","model":"gpt-4o-mini"},n=20)
+result = sdg.generate_qna(text, question_type ='complex',model_config={"provider":"openai","model":"openai/gpt-3.5-turbo"},n=5)
+
+print(result.head())
 
 # Get supported Q&A types
 sdg.get_supported_qna()
@@ -253,5 +261,87 @@ sdg.get_supported_providers()
 
 
 
+### Guardrail Management
+
+```py
+from ragaai_catalyst import GuardrailsManager
+
+# Initialize Guardrails Manager
+gdm = GuardrailsManager(project_name=project_name)
+
+# Get list of Guardrails available
+guardrails_list = gdm.list_guardrails()
+print('guardrails_list:', guardrails_list)
+
+# Get list of fail condition for guardrails
+fail_conditions = gdm.list_fail_condition()
+print('fail_conditions;', fail_conditions)
+
+#Get list of deployment ids
+deployment_list = gdm.list_deployment_ids()
+print('deployment_list:', deployment_list)
+
+# Get specific deployment id with guardrails information
+deployment_id_detail = gdm.get_deployment(17)
+print('deployment_id_detail:', deployment_id_detail)
+
+# Add guardrails to a deployment id
+guardrails_config = {"guardrailFailConditions": ["FAIL"],
+                     "deploymentFailCondition": "ALL_FAIL",
+                     "alternateResponse": "Your alternate response"}
+
+guardrails = [
+    {
+      "displayName": "Response_Evaluator",
+      "name": "Response Evaluator",
+      "config":{
+          "mappings": [{
+                        "schemaName": "Text",
+                        "variableName": "Response"
+                    }],
+          "params": {
+                    "isActive": {"value": False},
+                    "isHighRisk": {"value": True},
+                    "threshold": {"eq": 0},
+                    "competitors": {"value": ["Google","Amazon"]}
+                }
+      }
+    },
+    {
+      "displayName": "Regex_Check",
+      "name": "Regex Check",
+      "config":{
+          "mappings": [{
+                        "schemaName": "Text",
+                        "variableName": "Response"
+                    }],
+          "params":{
+              "isActive": {"value": False},
+              "isHighRisk": {"value": True},
+              "threshold": {"lt1": 1}
+          }
+      }
+    }
+]
+
+gdm.add_guardrails(deployment_id, guardrails, guardrails_config)
+
+
+# Import GuardExecutor
+from ragaai_catalyst import GuardExecutor
+
+# Initialise GuardExecutor with required params and Evaluate
+executor = GuardExecutor(deployment_id,gdm,field_map={'context':'document'})
+
+
+message={'role':'user',
+         'content':'What is the capital of France'
+        }
+prompt_params={'document':' France'}
+
+model_params = {'temperature':.7,'model':'gpt-4o-mini'}
+llm_caller = 'litellm'
 
+executor([message],prompt_params,model_params,llm_caller)
 
+```

{ragaai_catalyst-2.0.6b1 → ragaai_catalyst-2.0.7b1}/pyproject.toml

@@ -8,7 +8,7 @@ description = "RAGA AI CATALYST"
 readme = "README.md"
 requires-python = ">=3.9"
 # license = {file = "LICENSE"}
-version = "2.0.6.beta.1"
+version = "2.0.7.beta.1"
 authors = [
     {name = "Kiran Scaria", email = "kiran.scaria@raga.ai"},
     {name = "Kedar Gaikwad", email = "kedar.gaikwad@raga.ai"},
@@ -41,7 +41,8 @@ dependencies = [
     "litellm==1.51.1",  
     "tenacity==8.3.0",
     "tqdm>=4.66.5",
-    "llama-index==0.10.0"
+    "llama-index==0.10.0",
+    "pyopenssl==24.2.1"
 ]
 
 [project.optional-dependencies]

{ragaai_catalyst-2.0.6b1 → ragaai_catalyst-2.0.7b1}/ragaai_catalyst/dataset.py

@@ -16,7 +16,7 @@ class Dataset:
 
     def __init__(self, project_name):
         self.project_name = project_name
-        self.num_projects = 100
+        self.num_projects = 99999
         Dataset.BASE_URL = (
             os.getenv("RAGAAI_CATALYST_BASE_URL")
             if os.getenv("RAGAAI_CATALYST_BASE_URL")

{ragaai_catalyst-2.0.6b1 → ragaai_catalyst-2.0.7b1}/ragaai_catalyst/evaluation.py

@@ -16,7 +16,7 @@ class Evaluation:
         self.base_url = f"{RagaAICatalyst.BASE_URL}"
         self.timeout = 10
         self.jobId = None
-        self.num_projects=100
+        self.num_projects=99999
 
         try:
             response = requests.get(

{ragaai_catalyst-2.0.6b1 → ragaai_catalyst-2.0.7b1}/ragaai_catalyst/guardrails_manager.py

@@ -13,7 +13,7 @@ class GuardrailsManager:
         """
         self.project_name = project_name
         self.timeout = 10
-        self.num_projects = 100
+        self.num_projects = 99999
         self.deployment_name = "NA"
         self.deployment_id = "NA"
         self.base_url = f"{RagaAICatalyst.BASE_URL}"
@@ -31,7 +31,7 @@ class GuardrailsManager:
         :return: A tuple containing a list of project names and a list of dictionaries with project IDs and names.
         """
         headers = {'Authorization': f'Bearer {os.getenv("RAGAAI_CATALYST_TOKEN")}'}
-        response = requests.request("GET", f"{self.base_url}/v2/llm/projects?size=12&page=0", headers=headers, timeout=self.timeout)
+        response = requests.request("GET", f"{self.base_url}/v2/llm/projects?size={self.num_projects}", headers=headers, timeout=self.timeout)
         project_content = response.json()["data"]["content"]
         list_project = [_["name"] for _ in project_content]
         project_name_with_id = [{"id": _["id"], "name": _["name"]} for _ in project_content]

{ragaai_catalyst-2.0.6b1 → ragaai_catalyst-2.0.7b1}/ragaai_catalyst/internal_api_completion.py

@@ -36,7 +36,7 @@ def api_completion(messages, model_config, kwargs):
                     raise ValueError(response["error"]["message"])
                 else:
                     result=  response["choices"][0]["message"]["content"]
-                    response1 = result.replace('\n', '')
+                    response1 = result.replace('\n', '').replace('```json','').replace('```', '').strip()
                     try:
                         json_data = json.loads(response1)
                         df = pd.DataFrame(json_data)

{ragaai_catalyst-2.0.6b1 → ragaai_catalyst-2.0.7b1}/ragaai_catalyst/prompt_manager.py

@@ -23,7 +23,7 @@ class PromptManager:
         self.project_name = project_name
         self.base_url = f"{RagaAICatalyst.BASE_URL}/playground/prompt"
         self.timeout = 10
-        self.size = 100 #Number of projects to fetch
+        self.size = 99999 #Number of projects to fetch
 
         try:
             response = requests.get(
@@ -432,7 +432,12 @@ class PromptObject:
         Returns:
             dict: A dictionary of parameters found in the prompt text.
         """
-        parameters = {param["name"]: self._convert_value(param["value"], param["type"]) for param in self.parameters}
+        parameters = {}
+        for param in self.parameters:
+            if "value" in param:
+                parameters[param["name"]] = self._convert_value(param["value"], param["type"])
+            else:
+                parameters[param["name"]] = ""
         parameters["model"] = self.model
         return parameters    
     

{ragaai_catalyst-2.0.6b1 → ragaai_catalyst-2.0.7b1}/ragaai_catalyst/ragaai_catalyst.py

@@ -287,7 +287,7 @@ class RagaAICatalyst:
     def get_project_id(self, project_name):
         pass
 
-    def list_projects(self, num_projects=100):
+    def list_projects(self, num_projects=99999):
         """
         Retrieves a list of projects with the specified number of projects.
 

{ragaai_catalyst-2.0.6b1 → ragaai_catalyst-2.0.7b1}/ragaai_catalyst/synthetic_data_generation.py

@@ -289,6 +289,7 @@ class SyntheticDataGeneration:
 
         # Extract the content from the response
         content = response.choices[0].message.content
+        content = content.replace('\n', '').replace('```json','').replace('```', '').strip()
 
         # Clean the response if needed (remove any prefix before the JSON list)
         list_start_index = content.find('[')

{ragaai_catalyst-2.0.6b1 → ragaai_catalyst-2.0.7b1/ragaai_catalyst.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: ragaai_catalyst
-Version: 2.0.6b1
+Version: 2.0.7b1
 Summary: RAGA AI CATALYST
 Author-email: Kiran Scaria <kiran.scaria@raga.ai>, Kedar Gaikwad <kedar.gaikwad@raga.ai>, Dushyant Mahajan <dushyant.mahajan@raga.ai>, Siddhartha Kosti <siddhartha.kosti@raga.ai>, Ritika Goel <ritika.goel@raga.ai>, Vijay Chaurasia <vijay.chaurasia@raga.ai>
 Requires-Python: >=3.9
@@ -28,6 +28,7 @@ Requires-Dist: litellm==1.51.1
 Requires-Dist: tenacity==8.3.0
 Requires-Dist: tqdm>=4.66.5
 Requires-Dist: llama-index==0.10.0
+Requires-Dist: pyopenssl==24.2.1
 Provides-Extra: dev
 Requires-Dist: pytest; extra == "dev"
 Requires-Dist: pytest-cov; extra == "dev"
@@ -38,7 +39,7 @@ Requires-Dist: flake8; extra == "dev"
 
 # RagaAI Catalyst
 
-RagaAI Catalyst is a powerful tool for managing and optimizing LLM projects. It provides functionalities for project management, trace recording, and experiment management, allowing you to fine-tune and evaluate your LLM applications effectively.
+RagaAI Catalyst is a comprehensive platform designed to enhance the management and optimization of LLM projects. It offers a wide range of features, including project management, dataset management, evaluation management, trace management, prompt management, synthetic data generation, and guardrail management. These functionalities enable you to efficiently evaluate, and safeguard your LLM applications.
 
 ## Table of Contents
 
@@ -53,6 +54,7 @@ RagaAI Catalyst is a powerful tool for managing and optimizing LLM projects. It
     - [Trace Management](#trace-management)
     - [Prompt Management](#prompt-management)
     - [Synthetic Data Generation](#synthetic-data-generation)
+    - [Guardrail Management](#guardrail-management)
 
 ## Installation
 
@@ -144,6 +146,7 @@ evaluation = Evaluation(
 evaluation.list_metrics()
 
 # Add metrics to the experiment
+
 schema_mapping={
     'Query': 'prompt',
     'response': 'response',
@@ -201,8 +204,12 @@ tracer = Tracer(
 
 # Your code here
 
+
 # Stop the trace recording
 tracer.stop()
+
+# Get upload status
+tracer.get_upload_status()
 ```
 
 
@@ -237,7 +244,7 @@ print("variable:",variable)
 prompt_content = prompt.get_prompt_content()
 print("prompt_content:", prompt_content)
 
-# Compile a prompt with variables
+# Compile the prompt with variables
 compiled_prompt = prompt.compile(query="What's the weather?", context="sunny", llm_response="It's sunny today")
 print("Compiled prompt:", compiled_prompt)
 
@@ -280,7 +287,9 @@ sdg = SyntheticDataGeneration()
 text = sdg.process_document(input_data="file_path")
 
 # Generate results
-result = sdg.generate_qna(text, question_type ='simple',model_config={"provider":"openai","model":"gpt-4o-mini"},n=20)
+result = sdg.generate_qna(text, question_type ='complex',model_config={"provider":"openai","model":"openai/gpt-3.5-turbo"},n=5)
+
+print(result.head())
 
 # Get supported Q&A types
 sdg.get_supported_qna()
@@ -291,5 +300,87 @@ sdg.get_supported_providers()
 
 
 
+### Guardrail Management
+
+```py
+from ragaai_catalyst import GuardrailsManager
+
+# Initialize Guardrails Manager
+gdm = GuardrailsManager(project_name=project_name)
+
+# Get list of Guardrails available
+guardrails_list = gdm.list_guardrails()
+print('guardrails_list:', guardrails_list)
+
+# Get list of fail condition for guardrails
+fail_conditions = gdm.list_fail_condition()
+print('fail_conditions;', fail_conditions)
+
+#Get list of deployment ids
+deployment_list = gdm.list_deployment_ids()
+print('deployment_list:', deployment_list)
+
+# Get specific deployment id with guardrails information
+deployment_id_detail = gdm.get_deployment(17)
+print('deployment_id_detail:', deployment_id_detail)
+
+# Add guardrails to a deployment id
+guardrails_config = {"guardrailFailConditions": ["FAIL"],
+                     "deploymentFailCondition": "ALL_FAIL",
+                     "alternateResponse": "Your alternate response"}
+
+guardrails = [
+    {
+      "displayName": "Response_Evaluator",
+      "name": "Response Evaluator",
+      "config":{
+          "mappings": [{
+                        "schemaName": "Text",
+                        "variableName": "Response"
+                    }],
+          "params": {
+                    "isActive": {"value": False},
+                    "isHighRisk": {"value": True},
+                    "threshold": {"eq": 0},
+                    "competitors": {"value": ["Google","Amazon"]}
+                }
+      }
+    },
+    {
+      "displayName": "Regex_Check",
+      "name": "Regex Check",
+      "config":{
+          "mappings": [{
+                        "schemaName": "Text",
+                        "variableName": "Response"
+                    }],
+          "params":{
+              "isActive": {"value": False},
+              "isHighRisk": {"value": True},
+              "threshold": {"lt1": 1}
+          }
+      }
+    }
+]
+
+gdm.add_guardrails(deployment_id, guardrails, guardrails_config)
 
 
+# Import GuardExecutor
+from ragaai_catalyst import GuardExecutor
+
+# Initialise GuardExecutor with required params and Evaluate
+executor = GuardExecutor(deployment_id,gdm,field_map={'context':'document'})
+
+
+message={'role':'user',
+         'content':'What is the capital of France'
+        }
+prompt_params={'document':' France'}
+
+model_params = {'temperature':.7,'model':'gpt-4o-mini'}
+llm_caller = 'litellm'
+
+executor([message],prompt_params,model_params,llm_caller)
+
+```

{ragaai_catalyst-2.0.6b1 → ragaai_catalyst-2.0.7b1}/ragaai_catalyst.egg-info/requires.txt

@@ -21,6 +21,7 @@ litellm==1.51.1
 tenacity==8.3.0
 tqdm>=4.66.5
 llama-index==0.10.0
+pyopenssl==24.2.1
 
 [dev]
 pytest

{ragaai_catalyst-2.0.6b1 → ragaai_catalyst-2.0.7b1}/requirements.txt

@@ -1,5 +1,5 @@
 toml==0.10.2
-aiohttp==3.9.5
+aiohttp==3.10.2
 opentelemetry-api==1.25.0
 opentelemetry-sdk==1.25.0
 opentelemetry-instrumentation==0.46b0