sunholo 0.62.18__tar.gz → 0.64.3__tar.gz

{sunholo-0.62.18/sunholo.egg-info → sunholo-0.64.3}/PKG-INFO

@@ -1,9 +1,9 @@
 Metadata-Version: 2.1
 Name: sunholo
-Version: 0.62.18
+Version: 0.64.3
 Summary: Large Language Model DevOps - a package to help deploy LLMs to the Cloud.
 Home-page: https://github.com/sunholo-data/sunholo-py
-Download-URL: https://github.com/sunholo-data/sunholo-py/archive/refs/tags/v0.62.18.tar.gz
+Download-URL: https://github.com/sunholo-data/sunholo-py/archive/refs/tags/v0.64.3.tar.gz
 Author: Holosun ApS
 Author-email: multivac@sunholo.com
 License: Apache License, Version 2.0
@@ -19,6 +19,7 @@ Classifier: Programming Language :: Python :: 3.12
 Description-Content-Type: text/markdown
 License-File: LICENSE.txt
 Requires-Dist: google-auth
+Requires-Dist: tenacity==8.3.0
 Requires-Dist: langchain
 Requires-Dist: langchain_experimental
 Requires-Dist: langchain-community
@@ -111,54 +112,21 @@ This is the Sunholo Python project, a comprehensive toolkit for working with lan
 
 Please refer to the website for full documentation at https://dev.sunholo.com/
 
-# sunholo-py
-
-(draft release https://pypi.org/project/sunholo/ )
-## Table of Contents
-- [Agents](#agents)
-- [Archive](#archive)
-- [Bots](#bots)
-- [Chunker](#chunker)
-- [Components](#components)
-- [Database](#database)
-- [Embedder](#embedder)
-- [PubSub](#pubsub)
-- [QnA](#qna)
-- [Streaming](#streaming)
-- [Summarise](#summarise)
-- [Utils](#utils)
+## Demos
 
+Using https://github.com/charmbracelet/vhs
 
 ```sh
-pip install sunholo
+vhs record > cassette.tape
 ```
 
-A python library to enable LLMOps within cloud environments
-
-`sunholo` provides utilities to help manage LLM operations on Google Cloud Platform at first, but it is hoped that making it open source will help it support other clouds in the future.  A lot of the functionality is not Google Cloud Platform specific, so still may be helpful.
-
-It is derived from the Edmonbrain project, the original blog post you can read here: https://code.markedmondson.me/running-llms-on-gcp/ and owes a lot to Langchain ( https://github.com/langchain-ai/langchain )
-
-The package includes:
+Then make gif:
 
-* `agents/` - functions for working with agents, including easy flask apps, parsing chat history and dispatching requests to different agent endpoints
-* `archive/` - functions to record all Q&A activity to BigQuery via PubSub
-* `bots/` - functions for special cases regarding frontend bots such as GChat, Web Apps, Discord and Slack
-* `chunker/` - functions to slice up documents for sending into vector stores
-* `components/` - functions to help configure which LLM, prompt, vectorstore or document retriever you will use based on a yaml config file
-* `database/` - database setup functions and SQL to run on those sources such as Supabase
-* `embedder/` - functions to send chunks into embedding vector stores
-* `pubsub/` - use of PubSub for a message queue between components
-* `qna/` - utilities for running agents such as retry strats and parsing of output/input
-* `streaming/` - creation of streaming responses from LLM bots
-* `summarise/` - creation of summaries of large documents
-* `utils/` - reading configuration files, Google Cloud Platform metadata
-
-## Configuration
+```sh
+vhs docs/tapes/config-list.tape
+```
 
-The library uses the config specifications that some examples are given in the `config/` folder.
 
-When using the functions, make sure to have the `config/` folder in the root of where your application is running (usually `$HOME/config`)
 
 ```
    Copyright [2024] [Holosun ApS]

sunholo-0.64.3/README.md

@@ -0,0 +1,37 @@
+## Introduction
+This is the Sunholo Python project, a comprehensive toolkit for working with language models and vector stores on Google Cloud Platform. It provides a wide range of functionalities and utilities to facilitate the development and deployment of language model applications.
+
+Please refer to the website for full documentation at https://dev.sunholo.com/
+
+## Demos
+
+Using https://github.com/charmbracelet/vhs
+
+```sh
+vhs record > cassette.tape
+```
+
+Then make gif:
+
+```sh
+vhs docs/tapes/config-list.tape
+```
+
+
+
+```
+   Copyright [2024] [Holosun ApS]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.
+```
+

{sunholo-0.62.18 → sunholo-0.64.3}/setup.py

@@ -1,7 +1,7 @@
 from setuptools import setup, find_packages
 
 # Define your base version
-version = '0.62.18'
+version = '0.64.3'
 
 setup(
     name='sunholo',
@@ -29,6 +29,7 @@ setup(
     install_requires=[
         # Base dependencies
         "google-auth", # to check if on gcp
+        "tenacity==8.3.0",
         "langchain",
         "langchain_experimental",
         "langchain-community",

{sunholo-0.62.18 → sunholo-0.64.3}/sunholo/agents/flask/qna_routes.py

@@ -16,6 +16,7 @@
 import json
 import traceback
 import datetime
+import uuid
 
 from ...agents import extract_chat_history, handle_special_commands
 from ...qna.parsers import parse_output
@@ -23,7 +24,11 @@ from ...streaming import start_streaming_chat
 from ...archive import archive_qa
 from ...logging import log
 from ...utils.config import load_config
- 
+from ...utils.version import sunholo_version
+import os
+from ...gcs.add_file import add_file_to_gcs
+
+
 try:
     from flask import request, jsonify, Response
 except ImportError:
@@ -34,6 +39,7 @@ try:
 except ImportError:
     pass    
 
+
 def register_qna_routes(app, stream_interpreter, vac_interpreter):
 
     @app.route("/")
@@ -155,9 +161,114 @@ def register_qna_routes(app, stream_interpreter, vac_interpreter):
             span.end(output=jsonify(bot_output))
             trace.update(output=jsonify(bot_output)) 
 
+        # {'answer': 'output'}
         return jsonify(bot_output)
 
-    # Any other QNA related routes can be added here
+    @app.route('/openai/v1/chat/completions', methods=['POST'])
+    def openai_compatible_endpoint():
+        data = request.get_json()
+        model = data.pop('model', None)
+        messages = data.pop('messages', None)
+        chat_history = data.pop('chat_history', None)
+        stream = data.pop('stream', False)
+
+        if not messages:
+            return jsonify({"error": "No messages provided"}), 400
+
+        user_message = next((msg['content'] for msg in messages if msg['role'] == 'user'), None)
+        if not user_message:
+            return jsonify({"error": "No user message provided"}), 400
+
+        all_input = {
+            "user_input": user_message,
+            "chat_history": chat_history,
+            "kwargs": data
+        }
+
+        observed_stream_interpreter = observe()(stream_interpreter)
+
+
+        response_id = str(uuid.uuid4())
+        
+        def generate_response_content():
+            for chunk in start_streaming_chat(question=user_message,
+                                            vector_name=model,
+                                            qna_func=observed_stream_interpreter,
+                                            chat_history=all_input["chat_history"],
+                                            wait_time=all_input.get("stream_wait_time", 1),
+                                            timeout=all_input.get("stream_timeout", 60),
+                                            **all_input["kwargs"]
+                                            ):
+                if isinstance(chunk, dict) and 'content' in chunk:
+                    openai_chunk = {
+                        "id": response_id,
+                        "object": "chat.completion.chunk",
+                        "created": int(datetime.time()),
+                        "model": model,
+                        "system_fingerprint": sunholo_version(),
+                        "choices": [{
+                            "index": 0,
+                            "delta": {"content": chunk['content']},
+                            "logprobs": None,
+                            "finish_reason": None
+                        }]
+                    }
+                    yield json.dumps(openai_chunk) + "\n"
+
+            final_chunk = {
+                "id": response_id,
+                "object": "chat.completion.chunk",
+                "created": int(datetime.time()),
+                "model": model,
+                "system_fingerprint": sunholo_version(),
+                "choices": [{
+                    "index": 0,
+                    "delta": {},
+                    "logprobs": None,
+                    "finish_reason": "stop"
+                }]
+            }
+            yield json.dumps(final_chunk) + "\n"
+
+        if stream:
+            return Response(generate_response_content(), content_type='text/plain; charset=utf-8')
+
+        try:
+            bot_output = observed_stream_interpreter(
+                question=user_message,
+                vector_name=model,
+                chat_history=all_input["chat_history"],
+                **all_input["kwargs"]
+            )
+            bot_output = parse_output(bot_output)
+
+            openai_response = {
+                "id": response_id,
+                "object": "chat.completion",
+                "created": int(datetime.time()),
+                "model": model,
+                "system_fingerprint": sunholo_version(),
+                "choices": [{
+                    "index": 0,
+                    "message": {
+                        "role": "assistant",
+                        "content": bot_output.get('answer', ''),
+                    },
+                    "logprobs": None,
+                    "finish_reason": "stop"
+                }],
+                "usage": {
+                    "prompt_tokens": len(user_message.split()),
+                    "completion_tokens": len(bot_output.get('answer', '').split()),
+                    "total_tokens": len(user_message.split()) + len(bot_output.get('answer', '').split())
+                }
+            }
+
+            return jsonify(openai_response)
+
+        except Exception as err:
+            return jsonify({'error': f'QNA_ERROR: An error occurred: {str(err)} traceback: {traceback.format_exc()}'}), 500
+
 
 def create_langfuse_trace(request, vector_name):
     try:
@@ -172,10 +283,8 @@ def create_langfuse_trace(request, vector_name):
     session_id = request.headers.get("X-Session-ID")
     message_source = request.headers.get("X-Message-Source")
 
-    # can't import tags yet via CallbackHandler
-    from importlib.metadata import version
-    package_version = version('sunholo')
-    tags = [f"sunholo-v{package_version}"]
+    package_version = sunholo_version()
+    tags = [package_version]
     if message_source:
         tags.append(message_source)
 
@@ -191,8 +300,28 @@ def prep_vac(request, vector_name):
     #trace = create_langfuse_trace(request, vector_name)
     trace = None
     span = None
-    data = request.get_json()
+
+    if request.content_type.startswith('application/json'):
+        data = request.get_json()
+    elif request.content_type.startswith('multipart/form-data'):
+        data = request.form.to_dict()
+        if 'file' in request.files:
+            file = request.files['file']
+            if file.filename != '':
+                log.info(f"Found file: {file.filename} to upload to GCS")
+                try:
+                    image_uri, mime_type = handle_file_upload(file, vector_name)
+                    data["image_uri"] = image_uri
+                    data["mime"] = mime_type
+                except Exception as e:
+                    return jsonify({'error': str(e), 'traceback': traceback.format_exc()}), 500
+            else:
+                return jsonify({"error": "No file selected"}), 400
+    else:
+        return jsonify({"error": "Unsupported content type"}), 400
+
     log.info(f"vac/{vector_name} got data: {data}")
+
     config, _ = load_config("config/llm_config.yaml")
     vac_configs = config.get("vac")
     if vac_configs:
@@ -213,7 +342,7 @@ def prep_vac(request, vector_name):
                  'vector_name': vector_name, 
                  'chat_history': paired_messages, 
                  'stream_wait_time': stream_wait_time,
-                 'stream_timeout':stream_timeout,
+                 'stream_timeout': stream_timeout,
                  'kwargs': data}
 
     if trace:
@@ -233,4 +362,14 @@ def prep_vac(request, vector_name):
         "command_response": command_response,
         "all_input": all_input,
         "vac_config": vac_config
-    }
+    }
+
+
+def handle_file_upload(file, vector_name):
+    try:
+        file.save(file.filename)
+        image_uri = add_file_to_gcs(file.filename, vector_name)
+        os.remove(file.filename)  # Clean up the saved file
+        return image_uri, file.mimetype
+    except Exception as e:
+        raise Exception(f'File upload failed: {str(e)}')

{sunholo-0.62.18 → sunholo-0.64.3}/sunholo/auth/run.py

@@ -53,26 +53,23 @@ def get_header(vector_name) -> Optional[dict]:
     else:
         run_url = "http://127.0.0.1:8080"
 
-    if "http://" in run_url:
-        return None
-    else:
-        # Append ID Token to make authenticated requests to Cloud Run services
-        frame = inspect.currentframe()
-        caller_frame = frame.f_back if frame is not None else None  # One level up in the stack
+    # Append ID Token to make authenticated requests to Cloud Run services
+    frame = inspect.currentframe()
+    caller_frame = frame.f_back if frame is not None else None  # One level up in the stack
+    deets = {
+        'message': 'Authenticating for run_url',
+        'run_url': run_url
+    }
+    if caller_frame:
         deets = {
-            'message': 'Authenticating for run_url',
-            'run_url': run_url
-        }
-        if caller_frame:
-            deets = {
-                    'message': 'Authenticating for run_url',
-                    'file': caller_frame.f_code.co_filename,
-                    'line': str(caller_frame.f_lineno),  
-                    'function': caller_frame.f_code.co_name,
-                    'run_url': run_url
-                }
-        log.info(f"Authenticating for run_url {run_url} from {caller_frame.f_code.co_name}")
-        id_token = get_id_token(run_url)
-        headers = {"Authorization": f"Bearer {id_token}"}
-        #log.info(f"id_token {id_token}")
-        return headers
+                'message': 'Authenticating for run_url',
+                'file': caller_frame.f_code.co_filename,
+                'line': str(caller_frame.f_lineno),  
+                'function': caller_frame.f_code.co_name,
+                'run_url': run_url
+            }
+    log.info(f"Authenticating for run_url {run_url} from {caller_frame.f_code.co_name}")
+    id_token = get_id_token(run_url)
+    headers = {"Authorization": f"Bearer {id_token}"}
+    #log.info(f"id_token {id_token}")
+    return headers

{sunholo-0.62.18 → sunholo-0.64.3}/sunholo/cli/chat_vac.py

@@ -149,46 +149,72 @@ def headless_mode(service_url, service_name, user_input, chat_history=None, stre
     user_id = generate_user_id()
     session_id = str(uuid.uuid4())
 
-    def stream_response():
-        generate = generate_proxy_stream(
-                send_to_qa,
-                user_input,
-                vector_name=service_name,
-                chat_history=chat_history,
-                generate_f_output=lambda x: x,  # Replace with actual processing function
-                stream_wait_time=0.5,
-                stream_timeout=120,
-                message_author=user_id,
-                #TODO: populate these
-                image_url=None,
-                source_filters=None,
-                search_kwargs=None,
-                private_docs=None,
-                whole_document=False,
-                source_filters_and_or=False,
-                # system kwargs
-                configurable={
-                    "vector_name": service_name,
-                },
-                user_id=user_id,
-                session_id=session_id, 
-                message_source="cli",
-                override_endpoint=service_url
-        )
-        for part in generate():
-            yield part
+    if not stream:
+        vac_response = send_to_qa(user_input,
+            vector_name=service_name,
+            chat_history=chat_history,
+            message_author=user_id,
+            #TODO: populate these
+            image_url=None,
+            source_filters=None,
+            search_kwargs=None,
+            private_docs=None,
+            whole_document=False,
+            source_filters_and_or=False,
+            # system kwargs
+            configurable={
+                "vector_name": service_name,
+            },
+            user_id=user_id,
+            session_id=session_id, 
+            message_source="cli",
+            override_endpoint=service_url)
+        
+        # ensures {'answer': answer}
+        answer = parse_output(vac_response)
+        
+        console.print(answer.get('answer'))
+    else:
+        def stream_response():
+            generate = generate_proxy_stream(
+                    send_to_qa,
+                    user_input,
+                    vector_name=service_name,
+                    chat_history=chat_history,
+                    generate_f_output=lambda x: x,  # Replace with actual processing function
+                    stream_wait_time=0.5,
+                    stream_timeout=120,
+                    message_author=user_id,
+                    #TODO: populate these
+                    image_url=None,
+                    source_filters=None,
+                    search_kwargs=None,
+                    private_docs=None,
+                    whole_document=False,
+                    source_filters_and_or=False,
+                    # system kwargs
+                    configurable={
+                        "vector_name": service_name,
+                    },
+                    user_id=user_id,
+                    session_id=session_id, 
+                    message_source="cli",
+                    override_endpoint=service_url
+            )
+            for part in generate():
+                yield part
 
-    vac_response = ""
+        answer = ""
 
-    for token in stream_response():
-        if isinstance(token, bytes):
-            token = token.decode('utf-8')
-        print(token, end='', flush=True)
-        vac_response += token
+        for token in stream_response():
+            if isinstance(token, bytes):
+                token = token.decode('utf-8')
+            print(token, end='', flush=True)
+            answer += token
 
-    if vac_response:
+    if answer:
         chat_history.append({"name": "Human", "content": user_input})
-        chat_history.append({"name": "AI", "content": vac_response})
+        chat_history.append({"name": "AI", "content": answer})
     print()  # For new line after streaming ends
 
     return chat_history

{sunholo-0.62.18 → sunholo-0.64.3}/sunholo/langfuse/callback.py

@@ -7,6 +7,8 @@ try:
 except ImportError:
     CallbackHandler = None
 
+from ..utils.version import sunholo_version
+
 def create_langfuse_callback(**kwargs):
 
     if not CallbackHandler:
@@ -40,10 +42,7 @@ def add_langfuse_tracing(
     session_id = request.headers.get("X-Session-ID")
     message_source = request.headers.get("X-Message-Source")
 
-    # can't import tags yet via CallbackHandler
-    from importlib.metadata import version
-    package_version = version('sunholo')
-    tags = [f"sunholo-v{package_version}"]
+    tags = [sunholo_version()]
     if message_source:
         tags.append(message_source)
 

sunholo-0.64.3/sunholo/utils/version.py

@@ -0,0 +1,3 @@
+def sunholo_version():
+    from importlib.metadata import version
+    return f"sunholo-{version('sunholo')}"

{sunholo-0.62.18 → sunholo-0.64.3/sunholo.egg-info}/PKG-INFO

@@ -1,9 +1,9 @@
 Metadata-Version: 2.1
 Name: sunholo
-Version: 0.62.18
+Version: 0.64.3
 Summary: Large Language Model DevOps - a package to help deploy LLMs to the Cloud.
 Home-page: https://github.com/sunholo-data/sunholo-py
-Download-URL: https://github.com/sunholo-data/sunholo-py/archive/refs/tags/v0.62.18.tar.gz
+Download-URL: https://github.com/sunholo-data/sunholo-py/archive/refs/tags/v0.64.3.tar.gz
 Author: Holosun ApS
 Author-email: multivac@sunholo.com
 License: Apache License, Version 2.0
@@ -19,6 +19,7 @@ Classifier: Programming Language :: Python :: 3.12
 Description-Content-Type: text/markdown
 License-File: LICENSE.txt
 Requires-Dist: google-auth
+Requires-Dist: tenacity==8.3.0
 Requires-Dist: langchain
 Requires-Dist: langchain_experimental
 Requires-Dist: langchain-community
@@ -111,54 +112,21 @@ This is the Sunholo Python project, a comprehensive toolkit for working with lan
 
 Please refer to the website for full documentation at https://dev.sunholo.com/
 
-# sunholo-py
-
-(draft release https://pypi.org/project/sunholo/ )
-## Table of Contents
-- [Agents](#agents)
-- [Archive](#archive)
-- [Bots](#bots)
-- [Chunker](#chunker)
-- [Components](#components)
-- [Database](#database)
-- [Embedder](#embedder)
-- [PubSub](#pubsub)
-- [QnA](#qna)
-- [Streaming](#streaming)
-- [Summarise](#summarise)
-- [Utils](#utils)
+## Demos
 
+Using https://github.com/charmbracelet/vhs
 
 ```sh
-pip install sunholo
+vhs record > cassette.tape
 ```
 
-A python library to enable LLMOps within cloud environments
-
-`sunholo` provides utilities to help manage LLM operations on Google Cloud Platform at first, but it is hoped that making it open source will help it support other clouds in the future.  A lot of the functionality is not Google Cloud Platform specific, so still may be helpful.
-
-It is derived from the Edmonbrain project, the original blog post you can read here: https://code.markedmondson.me/running-llms-on-gcp/ and owes a lot to Langchain ( https://github.com/langchain-ai/langchain )
-
-The package includes:
+Then make gif:
 
-* `agents/` - functions for working with agents, including easy flask apps, parsing chat history and dispatching requests to different agent endpoints
-* `archive/` - functions to record all Q&A activity to BigQuery via PubSub
-* `bots/` - functions for special cases regarding frontend bots such as GChat, Web Apps, Discord and Slack
-* `chunker/` - functions to slice up documents for sending into vector stores
-* `components/` - functions to help configure which LLM, prompt, vectorstore or document retriever you will use based on a yaml config file
-* `database/` - database setup functions and SQL to run on those sources such as Supabase
-* `embedder/` - functions to send chunks into embedding vector stores
-* `pubsub/` - use of PubSub for a message queue between components
-* `qna/` - utilities for running agents such as retry strats and parsing of output/input
-* `streaming/` - creation of streaming responses from LLM bots
-* `summarise/` - creation of summaries of large documents
-* `utils/` - reading configuration files, Google Cloud Platform metadata
-
-## Configuration
+```sh
+vhs docs/tapes/config-list.tape
+```
 
-The library uses the config specifications that some examples are given in the `config/` folder.
 
-When using the functions, make sure to have the `config/` folder in the root of where your application is running (usually `$HOME/config`)
 
 ```
    Copyright [2024] [Holosun ApS]

{sunholo-0.62.18 → sunholo-0.64.3}/sunholo.egg-info/SOURCES.txt

@@ -108,10 +108,12 @@ sunholo/utils/gcp_project.py
 sunholo/utils/parsers.py
 sunholo/utils/timedelta.py
 sunholo/utils/user_ids.py
+sunholo/utils/version.py
 sunholo/vertex/__init__.py
 sunholo/vertex/init.py
 sunholo/vertex/memory_tools.py
 sunholo/vertex/safety.py
 tests/test_chat_history.py
+tests/test_chunker.py
 tests/test_config.py
 tests/test_dispatch_to_qa.py

{sunholo-0.62.18 → sunholo-0.64.3}/sunholo.egg-info/requires.txt

@@ -1,4 +1,5 @@
 google-auth
+tenacity==8.3.0
 langchain
 langchain_experimental
 langchain-community

sunholo-0.64.3/tests/test_chunker.py

@@ -0,0 +1,23 @@
+import pytest
+from unittest.mock import patch, MagicMock
+from sunholo.chunker.data_to_embed_pubsub import data_to_embed_pubsub
+
+# Mock external calls within the function
+@patch('sunholo.chunker.data_to_embed_pubsub.process_pubsub_message', return_value=({}, {}, 'test_vector'))
+@patch('sunholo.chunker.data_to_embed_pubsub.process_chunker_data', return_value='processed_data')
+def test_data_to_embed_pubsub(mock_process_chunker_data, mock_process_pubsub_message):
+    # Test the function with various inputs including edge cases
+    assert data_to_embed_pubsub({}) == 'processed_data'
+    assert data_to_embed_pubsub({'key': 'value'}) == 'processed_data'
+    mock_process_pubsub_message.assert_called()
+    mock_process_chunker_data.assert_called()
+
+    # Ensure tests are self-contained and do not require external dependencies
+    mock_process_pubsub_message = MagicMock(return_value=({}, {}, 'test_vector'))
+    mock_process_chunker_data = MagicMock(return_value='processed_data')
+    assert data_to_embed_pubsub({'key': 'value'}) == 'processed_data'
+
+    # Validate the function's output against expected results
+    expected_output = 'processed_data'
+    actual_output = data_to_embed_pubsub({'key': 'value'})
+    assert actual_output == expected_output, f"Expected {expected_output}, got {actual_output}"

{sunholo-0.62.18 → sunholo-0.64.3}/tests/test_dispatch_to_qa.py

@@ -8,13 +8,19 @@ from sunholo.agents.dispatch_to_qa import prep_request_payload
     # Additional test cases to cover all logic paths
     ("", [], "empty_input_vector", False, "http://example.com/invoke", {"user_input": "", "chat_history": [], "vector_name": "empty_input_vector"}),
     ("Valid input", ["Chat history present"], "history_vector", False, "http://example.com/invoke", {"user_input": "Valid input", "chat_history": ["Chat history present"], "vector_name": "history_vector"}),
-    ("Stream request", [], "stream_vector", True, "http://example.com/stream", {"user_input": "Stream request", "chat_history": [], "vector_name": "stream_vector"})
+    ("Stream request", [], "stream_vector", True, "http://example.com/stream", {"user_input": "Stream request", "chat_history": [], "vector_name": "stream_vector"}),
+    # New test cases for prep_request_payload function
+    ("User input with override endpoint", [], "override_vector", False, "http://override.com/invoke", {"user_input": "User input with override endpoint", "chat_history": [], "vector_name": "override_vector"}),
+    ("Stream with chat history and override endpoint", ["Chat history for override"], "override_stream_vector", True, "http://override.com/stream", {"user_input": "Stream with chat history and override endpoint", "chat_history": ["Chat history for override"], "vector_name": "override_stream_vector"})
 ])
 @patch('sunholo.agents.dispatch_to_qa.load_config_key')
 @patch('sunholo.agents.dispatch_to_qa.route_endpoint')
 def test_prep_request_payload(mock_route_endpoint, mock_load_config_key, user_input, chat_history, vector_name, stream, expected_endpoint, expected_payload):
     mock_route_endpoint.return_value = {"invoke": "http://example.com/invoke", "stream": "http://example.com/stream"}
     mock_load_config_key.side_effect = lambda key, vector_name=None, kind=None: "my_vector" if key == "vector_name" else None
+    # Mocking for override endpoint scenario
+    if vector_name.startswith("override"):
+        mock_route_endpoint.return_value = {"invoke": "http://override.com/invoke", "stream": "http://override.com/stream"}
 
     endpoint, payload = prep_request_payload(user_input, chat_history, vector_name, stream)
 

sunholo-0.62.18/README.md

@@ -1,70 +0,0 @@
-## Introduction
-This is the Sunholo Python project, a comprehensive toolkit for working with language models and vector stores on Google Cloud Platform. It provides a wide range of functionalities and utilities to facilitate the development and deployment of language model applications.
-
-Please refer to the website for full documentation at https://dev.sunholo.com/
-
-# sunholo-py
-
-(draft release https://pypi.org/project/sunholo/ )
-## Table of Contents
-- [Agents](#agents)
-- [Archive](#archive)
-- [Bots](#bots)
-- [Chunker](#chunker)
-- [Components](#components)
-- [Database](#database)
-- [Embedder](#embedder)
-- [PubSub](#pubsub)
-- [QnA](#qna)
-- [Streaming](#streaming)
-- [Summarise](#summarise)
-- [Utils](#utils)
-
-
-```sh
-pip install sunholo
-```
-
-A python library to enable LLMOps within cloud environments
-
-`sunholo` provides utilities to help manage LLM operations on Google Cloud Platform at first, but it is hoped that making it open source will help it support other clouds in the future.  A lot of the functionality is not Google Cloud Platform specific, so still may be helpful.
-
-It is derived from the Edmonbrain project, the original blog post you can read here: https://code.markedmondson.me/running-llms-on-gcp/ and owes a lot to Langchain ( https://github.com/langchain-ai/langchain )
-
-The package includes:
-
-* `agents/` - functions for working with agents, including easy flask apps, parsing chat history and dispatching requests to different agent endpoints
-* `archive/` - functions to record all Q&A activity to BigQuery via PubSub
-* `bots/` - functions for special cases regarding frontend bots such as GChat, Web Apps, Discord and Slack
-* `chunker/` - functions to slice up documents for sending into vector stores
-* `components/` - functions to help configure which LLM, prompt, vectorstore or document retriever you will use based on a yaml config file
-* `database/` - database setup functions and SQL to run on those sources such as Supabase
-* `embedder/` - functions to send chunks into embedding vector stores
-* `pubsub/` - use of PubSub for a message queue between components
-* `qna/` - utilities for running agents such as retry strats and parsing of output/input
-* `streaming/` - creation of streaming responses from LLM bots
-* `summarise/` - creation of summaries of large documents
-* `utils/` - reading configuration files, Google Cloud Platform metadata
-
-## Configuration
-
-The library uses the config specifications that some examples are given in the `config/` folder.
-
-When using the functions, make sure to have the `config/` folder in the root of where your application is running (usually `$HOME/config`)
-
-```
-   Copyright [2024] [Holosun ApS]
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-```
-