hana-ai 1.0.25070700__tar.gz → 1.0.25070700.post2__tar.gz

{hana_ai-1.0.25070700 → hana_ai-1.0.25070700.post2}/INTRODUCTION.md

@@ -21,12 +21,13 @@ The generative AI toolkit for SAP HANA Cloud provides the following key capabili
 This conversational agent (agents.hanaml_agent_with_memory.HANAMLAgentWithMemory class), aids to streamline development of SAP HANA Cloud forecasting and machine learning models, it's re-using the provided library of HANA ai-tools (tools.toolkit.HANAMLToolkit class) and is based on the Langchain agent framework. Trained models and artifacts are persisted using the hana_ml model storage class that helps manage the model version.
 ```python
 from hana_ai.tools.toolkit import HANAMLToolkit
+from hana_ai.agents.hanaml_rag_agent import HANAMLRAGAgent
+
 tools = HANAMLToolkit(cc, used_tools='all').get_tools()
+chatbot = HANAMLRAGAgent(llm=llm, tools=tools, verbose=True, vector_store_type="hanadb", hana_connection_context=cc)
 
-from hana_ai.agents.hanaml_agent_with_memory import HANAMLAgentWithMemory
-chatbot = HANAMLAgentWithMemory(llm=llm, toos=tools, session_id='hana_ai_test', n_messages=10)
 ```
-<img src="./img/chatbotwithtoolkit.png" alt="image" width="800" height="auto">
+<img src="./doc/image/chatbotwithtoolkit.png" alt="image" width="800" height="auto">
 
 ## Library of tools for HANA-ML 
 Provided AI-tools for streamlining usage of HANA ML functions in context of the conversational agent.
@@ -88,7 +89,7 @@ code_tool.set_vectordb(self.vectordb)
 ```python
 hana_vec.query("AutoML classification", top_n=1)
 ```
-![alt](./img/code_template.png)
+![alt](./doc/image/code_template.png)
 
 ### Working with multiple Vector Stores
 Union of multiple vector stores is possible
@@ -116,14 +117,14 @@ sdf = SmartDataFrame(hana_df)
 sdf.configure(tools=[code_tool], llm=llm)
 sdf.ask("Show the samples of the dataset", verbose=True)
 ```
-![alt](./img/smartdf_ask.png)
+![alt](./doc/image/smartdf_ask.png)
 
 ```python
 new_df = sdf.transform("Get first two rows", verbose=True)
 new_df.collect()
 ```
-![alt](./img/smartdf_transform.png)
-![alt](./img/smartdf_res.png)
+![alt](./doc/image/smartdf_transform.png)
+![alt](./doc/image/smartdf_res.png)
 
 
 

{hana_ai-1.0.25070700 → hana_ai-1.0.25070700.post2}/MANIFEST.in

@@ -7,4 +7,3 @@ include CONTRIBUTING.md
 include src/hana_ai/vectorstore/knowledge_base/python_knowledge/*
 include src/hana_ai/vectorstore/knowledge_base/sql_knowledge/*
 include src/hana_ai/agents/scenario_knowledge_base/*
-include img/*

{hana_ai-1.0.25070700/src/hana_ai.egg-info → hana_ai-1.0.25070700.post2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: hana_ai
-Version: 1.0.25070700
+Version: 1.0.25070700.post2
 Summary: Generative AI Toolkit for SAP HANA Cloud
 Author: SAP
 License: Apache-2.0
@@ -25,7 +25,6 @@ Requires-Dist: pydantic
 Requires-Dist: pydantic-core
 Requires-Dist: cryptography
 Requires-Dist: ipython
-Requires-Dist: sentence_transformers
 Requires-Dist: deprecated
 Dynamic: license-file
 
@@ -35,7 +34,7 @@ Dynamic: license-file
 
 ## About this project
 
-Generative AI Client for SAP HANA Cloud is an extension of the existing HANA ML Python client library, mainly focusing on GenAI and related use cases. It includes many leading-edge GenAI related open source libraries and provides seamless integration with HANA ML, HANA vector engine, and other SAP GenAI Hub SDK, see our [Introduction](https://github.com/SAP/generative-ai-toolkit-for-sap-hana-cloud/blob/main/INTRODUCTION.md), [Notebook](https://github.com/SAP/generative-ai-toolkit-for-sap-hana-cloud/blob/main/nutest/testscripts/demo/e2e_scenarios/time_series_forecast_scenario_sales_refunds.ipynb) and [Documentation](https://sap.github.io/generative-ai-toolkit-for-sap-hana-cloud/).
+Generative AI Client for SAP HANA Cloud is an extension of the existing HANA ML Python client library, mainly focusing on GenAI and related use cases. It includes many leading-edge GenAI related open source libraries and provides seamless integration with HANA ML, HANA vector engine, and other SAP GenAI Hub SDK, see our [Introduction](https://github.com/SAP/generative-ai-toolkit-for-sap-hana-cloud/blob/main/INTRODUCTION.md), [Notebook](https://github.com/SAP/generative-ai-toolkit-for-sap-hana-cloud/blob/main/nutest/testscripts/demo/e2e_scenarios/time_series_forecast_scenario_sales_refunds_new.ipynb) and [Documentation](https://sap.github.io/generative-ai-toolkit-for-sap-hana-cloud/).
 
 ## Requirements and Setup
 

{hana_ai-1.0.25070700 → hana_ai-1.0.25070700.post2}/README.md

@@ -4,7 +4,7 @@
 
 ## About this project
 
-Generative AI Client for SAP HANA Cloud is an extension of the existing HANA ML Python client library, mainly focusing on GenAI and related use cases. It includes many leading-edge GenAI related open source libraries and provides seamless integration with HANA ML, HANA vector engine, and other SAP GenAI Hub SDK, see our [Introduction](https://github.com/SAP/generative-ai-toolkit-for-sap-hana-cloud/blob/main/INTRODUCTION.md), [Notebook](https://github.com/SAP/generative-ai-toolkit-for-sap-hana-cloud/blob/main/nutest/testscripts/demo/e2e_scenarios/time_series_forecast_scenario_sales_refunds.ipynb) and [Documentation](https://sap.github.io/generative-ai-toolkit-for-sap-hana-cloud/).
+Generative AI Client for SAP HANA Cloud is an extension of the existing HANA ML Python client library, mainly focusing on GenAI and related use cases. It includes many leading-edge GenAI related open source libraries and provides seamless integration with HANA ML, HANA vector engine, and other SAP GenAI Hub SDK, see our [Introduction](https://github.com/SAP/generative-ai-toolkit-for-sap-hana-cloud/blob/main/INTRODUCTION.md), [Notebook](https://github.com/SAP/generative-ai-toolkit-for-sap-hana-cloud/blob/main/nutest/testscripts/demo/e2e_scenarios/time_series_forecast_scenario_sales_refunds_new.ipynb) and [Documentation](https://sap.github.io/generative-ai-toolkit-for-sap-hana-cloud/).
 
 ## Requirements and Setup
 

{hana_ai-1.0.25070700 → hana_ai-1.0.25070700.post2}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "hana_ai"
-version = "1.0.25070700"
+version = "1.0.25070700.post2"
 authors = [
     {name = "SAP"}
 ]

{hana_ai-1.0.25070700 → hana_ai-1.0.25070700.post2}/requirements.txt

@@ -11,5 +11,4 @@ pydantic
 pydantic-core
 cryptography
 ipython
-sentence_transformers
 deprecated

{hana_ai-1.0.25070700 → hana_ai-1.0.25070700.post2}/src/hana_ai/__init__.py

@@ -2,4 +2,4 @@
 hana_ai is a Python package for AI/ML related utilities.
 """
 
-__version__ = "1.0.25070700"
+__version__ = "1.0.25070700.post2"

{hana_ai-1.0.25070700 → hana_ai-1.0.25070700.post2}/src/hana_ai/agents/hanaml_rag_agent.py

@@ -26,7 +26,23 @@ from langchain_community.chat_message_histories import SQLChatMessageHistory
 from langchain_community.vectorstores import FAISS
 from langchain_community.vectorstores.hanavector import HanaDB
 
-from sentence_transformers import CrossEncoder
+try:
+    from sentence_transformers import CrossEncoder
+except ImportError:
+    class CrossEncoder:
+        """
+        Dummy CrossEncoder class for compatibility when sentence_transformers is not installed.
+        This class simulates the behavior of a cross-encoder by returning zeros for all predictions.
+        This is useful for testing purposes or when the actual model is not available.
+        """
+        def __init__(self, model_name):
+            self.model_name = model_name
+        def predict(self, pairs):
+            """
+            Simulate the prediction of similarity scores for pairs of texts.
+            """
+            # Dummy implementation, returns zeros
+            return [0.0 for _ in pairs]
 from hana_ai.agents.utilities import _check_generated_cap_for_bas, _inspect_python_code
 from hana_ai.vectorstore.embedding_service import GenAIHubEmbeddings
 
@@ -153,6 +169,21 @@ class HANAMLRAGAgent:
             Similarity score threshold for retrieval.
 
             Defaults to 0.5.
+        vector_store_type : str
+            Type of vector store to use for long-term memory. Options are 'faiss' or 'hanadb'.
+
+            Defaults to 'faiss'.
+        hana_connection_context : Any
+            HANA connection context for HANA DB vector store. Required if vector_store_type is 'hanadb'.
+
+        hana_vector_table : str
+            Name of the HANA vector table to use for long-term memory.
+
+            Defaults to "HANA_AI_CHAT_HISTORY".
+        drop_existing_hana_vector_table : bool
+            Whether to drop the existing HANA vector table before creating a new one.
+
+            Defaults to False.
         verbose : bool
             Whether to enable verbose logging.
 
@@ -202,7 +233,14 @@ class HANAMLRAGAgent:
         self._initialize_vectorstore()
 
     def delete_message_long_term_store(self, message_id) -> None:
-        """Delete a specific message by its ID"""
+        """
+        Delete a specific message by its 
+
+        Parameters
+        ----------
+        message_id : str
+            The ID of the message to delete from long-term memory.
+        """
         long_term_store = self.long_term_store
         try:
             long_term_store._create_table_if_not_exists()
@@ -316,6 +354,9 @@ class HANAMLRAGAgent:
         # Clean up oldest memories if needed
         self._forget_old_memories()
 
+    def _forget_past_messages_in_hana_db(self, timestamp: str):
+        self.vectorstore.delete(filter={"timestamp": {"$lte": timestamp}})
+
     def _forget_old_memories(self):
         """Remove oldest memories when storage limit is exceeded"""
         if len(self.long_term_store.messages) > self.long_term_memory_limit:
@@ -329,83 +370,61 @@ class HANAMLRAGAgent:
                     key=lambda msg: msg.metadata.get('timestamp', '1970-01-01')
                 )
                 # Delete oldest messages from SQL store
+                last_timestamp = '1970-01-01'
                 for msg in sorted_messages[:num_to_remove]:
                     self.delete_message_long_term_store(msg.id)
+                    last_timestamp = msg.metadata.get('timestamp', last_timestamp)
                 logger.info("Deleted %s oldest memories from SQL store", num_to_remove)
             except Exception as e:
                 logger.error("Error deleting from SQL store: %s", str(e))
-            # 2. Rebuild vectorstore from remaining messages
-            try:
-                # Get remaining messages after deletion
-                remaining_messages = self.long_term_store.messages
-                # Skip if no messages left
-                if not remaining_messages:
-                    if self.vectorstore_type.lower() == "faiss":
+            # 2. Rebuild vectorstore from remaining messages in Faiss
+            if self.vectorstore_type.lower() == "faiss":
+                try:
+                    # Get remaining messages after deletion
+                    remaining_messages = self.long_term_store.messages
+                    # Skip if no messages left
+                    if not remaining_messages:
                         self.vectorstore = FAISS.from_texts(
                             [""],
                             embedding=self.embeddings
                         )
-                    else:
-                        self.hana_connection_context.drop_table(self.hana_vector_table)
-                        self.vectorstore = HanaDB.from_texts(
-                            texts=[""],
-                            embedding=self.embeddings,
-                            connection=self.hana_connection_context.connection,
-                            table_name=self.hana_vector_table
-                        )
-                    return
-                # Recreate documents from remaining messages
-                splitter = RecursiveCharacterTextSplitter(
-                    chunk_size=self.chunk_size,
-                    chunk_overlap=self.chunk_overlap,
-                )
-                documents = []
-                for i in range(0, len(remaining_messages), 2):
-                    if i+1 >= len(remaining_messages):
-                        break  # Skip incomplete pairs
-                    user_msg = remaining_messages[i]
-                    ai_msg = remaining_messages[i+1]
-                    # Create combined document
-                    text = f"User: {user_msg.content}\nAssistant: {ai_msg.content}"
-                    metadata = {"timestamp": user_msg.metadata.get('timestamp', '')}
-                    # Split and add to documents
-                    docs = splitter.create_documents(
-                        texts=[text],
-                        metadatas=[metadata]
+                        return
+                    # Recreate documents from remaining messages
+                    splitter = RecursiveCharacterTextSplitter(
+                        chunk_size=self.chunk_size,
+                        chunk_overlap=self.chunk_overlap,
                     )
-                    documents.extend(docs)
-                # Rebuild vectorstore
-                if self.vectorstore_type.lower() == "faiss":
+                    documents = []
+                    for i in range(0, len(remaining_messages), 2):
+                        if i+1 >= len(remaining_messages):
+                            break  # Skip incomplete pairs
+                        user_msg = remaining_messages[i]
+                        ai_msg = remaining_messages[i+1]
+                        # Create combined document
+                        text = f"User: {user_msg.content}\nAssistant: {ai_msg.content}"
+                        metadata = {"timestamp": user_msg.metadata.get('timestamp', '')}
+                        # Split and add to documents
+                        docs = splitter.create_documents(
+                            texts=[text],
+                            metadatas=[metadata]
+                        )
+                        documents.extend(docs)
+                    # Rebuild vectorstore
                     self.vectorstore = FAISS.from_documents(
                         documents,
                         self.embeddings
                     )
                     self.vectorstore.save_local(self.vectorstore_path)
-                else:
-                    self.hana_connection_context.drop_table(self.hana_vector_table)
-                    self.vectorstore = HanaDB.from_documents(
-                        documents,
-                        self.embeddings,
-                        connection=self.hana_connection_context.connection,
-                        table_name=self.hana_vector_table
-                    )
-                logger.info("Rebuilt vectorstore with %s documents", len(documents))
-            except Exception as e:
-                logger.error("Error rebuilding vectorstore: %s", str(e))
-                # Fallback to empty vectorstore
-                if self.vectorstore_type.lower() == "faiss":
+                    logger.info("Rebuilt vectorstore with %s documents", len(documents))
+                except Exception as e:
+                    logger.error("Error rebuilding vectorstore: %s", str(e))
+                    # Fallback to empty vectorstore
                     self.vectorstore = FAISS.from_texts(
                         [""],
                         embedding=self.embeddings
                     )
-                else:
-                    self.hana_connection_context.drop_table(self.hana_vector_table)
-                    self.vectorstore = HanaDB.from_texts(
-                        texts=[""],
-                        embedding=self.embeddings,
-                        connection=self.hana_connection_context.connection,
-                        table_name=self.hana_vector_table
-                    )
+            else:
+                self._forget_past_messages_in_hana_db(last_timestamp)
 
     def _retrieve_relevant_memories(self, query: str) -> List[str]:
         """Retrieve relevant memories using RAG with reranking"""

{hana_ai-1.0.25070700 → hana_ai-1.0.25070700.post2/src/hana_ai.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: hana_ai
-Version: 1.0.25070700
+Version: 1.0.25070700.post2
 Summary: Generative AI Toolkit for SAP HANA Cloud
 Author: SAP
 License: Apache-2.0
@@ -25,7 +25,6 @@ Requires-Dist: pydantic
 Requires-Dist: pydantic-core
 Requires-Dist: cryptography
 Requires-Dist: ipython
-Requires-Dist: sentence_transformers
 Requires-Dist: deprecated
 Dynamic: license-file
 
@@ -35,7 +34,7 @@ Dynamic: license-file
 
 ## About this project
 
-Generative AI Client for SAP HANA Cloud is an extension of the existing HANA ML Python client library, mainly focusing on GenAI and related use cases. It includes many leading-edge GenAI related open source libraries and provides seamless integration with HANA ML, HANA vector engine, and other SAP GenAI Hub SDK, see our [Introduction](https://github.com/SAP/generative-ai-toolkit-for-sap-hana-cloud/blob/main/INTRODUCTION.md), [Notebook](https://github.com/SAP/generative-ai-toolkit-for-sap-hana-cloud/blob/main/nutest/testscripts/demo/e2e_scenarios/time_series_forecast_scenario_sales_refunds.ipynb) and [Documentation](https://sap.github.io/generative-ai-toolkit-for-sap-hana-cloud/).
+Generative AI Client for SAP HANA Cloud is an extension of the existing HANA ML Python client library, mainly focusing on GenAI and related use cases. It includes many leading-edge GenAI related open source libraries and provides seamless integration with HANA ML, HANA vector engine, and other SAP GenAI Hub SDK, see our [Introduction](https://github.com/SAP/generative-ai-toolkit-for-sap-hana-cloud/blob/main/INTRODUCTION.md), [Notebook](https://github.com/SAP/generative-ai-toolkit-for-sap-hana-cloud/blob/main/nutest/testscripts/demo/e2e_scenarios/time_series_forecast_scenario_sales_refunds_new.ipynb) and [Documentation](https://sap.github.io/generative-ai-toolkit-for-sap-hana-cloud/).
 
 ## Requirements and Setup
 

{hana_ai-1.0.25070700 → hana_ai-1.0.25070700.post2}/src/hana_ai.egg-info/SOURCES.txt

@@ -5,20 +5,6 @@ MANIFEST.in
 README.md
 pyproject.toml
 requirements.txt
-img/agent.png
-img/chatbotwithtoolkit.png
-img/code_template.png
-img/crag.png
-img/dataset_report.png
-img/embeddings.png
-img/embeddings_pal.png
-img/hana_embeddings.png
-img/kg.png
-img/pal_embeddings0.png
-img/pal_embeddings1.png
-img/smartdf_ask.png
-img/smartdf_res.png
-img/smartdf_transform.png
 src/hana_ai/__init__.py
 src/hana_ai/smart_dataframe.py
 src/hana_ai/utility.py

{hana_ai-1.0.25070700 → hana_ai-1.0.25070700.post2}/src/hana_ai.egg-info/requires.txt

@@ -11,5 +11,4 @@ pydantic
 pydantic-core
 cryptography
 ipython
-sentence_transformers
 deprecated