monocle-apptrace 0.1.1__tar.gz → 0.3.0b1__tar.gz

{monocle_apptrace-0.1.1 → monocle_apptrace-0.3.0b1}/.gitignore

@@ -12,4 +12,5 @@ traces.txt
 dist
 .tox
 .DS_Store
-Pipfile**
+Pipfile**
+launch.json

monocle_apptrace-0.3.0b1/CHANGELOG.md

@@ -0,0 +1,49 @@
+## Version 0.3.0b1 (2024-12-10)
+
+- Add dev dependency for Mistral AI integration ([#81](https://github.com/monocle2ai/monocle/pull/81))
+- Add VectorStore deployment URL capture support ([#80](https://github.com/monocle2ai/monocle/pull/80))  
+- Clean up cloud exporter implementation ([#79](https://github.com/monocle2ai/monocle/pull/79))
+- Capture inference span input/output events attributes ([#77](https://github.com/monocle2ai/monocle/pull/77))
+- Add release automation workflows ([#76](https://github.com/monocle2ai/monocle/pull/76))
+- Fix gaps in Monocle SDK implementation ([#72](https://github.com/monocle2ai/monocle/pull/72))  
+- Add kwargs and return value handling in Accessor ([#71](https://github.com/monocle2ai/monocle/pull/71))
+- Update workflow name formatting ([#69](https://github.com/monocle2ai/monocle/pull/69))
+- Implement Haystack metamodel support ([#68](https://github.com/monocle2ai/monocle/pull/68))
+
+## Version 0.2.0 (2024-12-05)
+
+## 0.2.0 (Oct 22, 2024)
+
+- Ndjson format for S3 and Blob exporters ([#61](https://github.com/monocle2ai/monocle/pull/61))
+- Set monocle exporter from env setting ([#60](https://github.com/monocle2ai/monocle/pull/60))
+- Update workflow name and type with new format ([#59](https://github.com/monocle2ai/monocle/pull/59))
+- Updated async and custom output processor testcase for metamodel([#58](https://github.com/monocle2ai/monocle/pull/58))
+- Build okahu exporter and added test cases for okahu exporte ([#56](https://github.com/monocle2ai/monocle/pull/56))
+- Handle exception in span wrappers([#52](https://github.com/monocle2ai/monocle/pull/52))
+- Metamodel entity changes ([#51](https://github.com/monocle2ai/monocle/pull/51)), ([#54](https://github.com/monocle2ai/monocle/pull/54))
+- Error handling for llm_endpoint and tags ([#50](https://github.com/monocle2ai/monocle/pull/50))
+- Context_output for vector store retriever ([#48](https://github.com/monocle2ai/monocle/pull/48))
+- Direct exporter - AWS S3 ([#42](https://github.com/monocle2ai/monocle/pull/42))
+- Direct Exporter - Blob store ([#41](https://github.com/monocle2ai/monocle/pull/41))
+- Initial metamodel definition ([#39](https://github.com/monocle2ai/monocle/pull/39))
+- Improvement in vectorstore traces ([#38](https://github.com/monocle2ai/monocle/pull/38))
+- Update key for session context field in attributes ([#34](https://github.com/monocle2ai/monocle/pull/34))
+
+
+## 0.1.0 (Aug 27, 2024)
+
+- Fixed LlamaIndex tracing bugs ([#32](https://github.com/monocle2ai/monocle/pull/32))
+- Added support to add AWS cloud infra attributes ([#29](https://github.com/monocle2ai/monocle/pull/29))
+- Added support to add Azure cloud infra attributes ([#23](https://github.com/monocle2ai/monocle/pull/23))
+- Added support for adding provider name in LLM span in traces ([#22](https://github.com/monocle2ai/monocle/pull/22))
+- Added a default file span exporter ([#21](https://github.com/monocle2ai/monocle/pull/21))
+- Moved input and output context and prompts from attributes to events ([#15](https://github.com/monocle2ai/monocle/pull/15))
+
+
+
+
+
+
+## 0.0.1 (Jul 17, 2024)
+
+- First monocle release

{monocle_apptrace-0.1.1 → monocle_apptrace-0.3.0b1}/Monocle_User_Guide.md

@@ -1,4 +1,4 @@
-#Monocle User Guide
+# Monocle User Guide
 
 ## Monocle Concepts
 ### Traces
@@ -13,21 +13,42 @@ It’s typically the workflow code components of an application that generate th
 ``` 
     > pip install monocle_apptrace
 ```
+
+- For Azure support (to upload traces to Azure), install with the azure extra:
+```
+    > pip install monocle_apptrace[azure]
+```
+
+- For AWS support (to upload traces to AWS), install with the aws extra:
+```
+    > pip install monocle_apptrace[aws]
+```
+
 - You can locally build and install Monocle library from source
 ```
-> pip install .
+    > pip install .
 ```
 - Install the optional test dependencies listed against dev in pyproject.toml in editable mode
 ```
-> pip install -e ".[dev]"
+    > pip install -e ".[dev]"
 ```
 
+
 ## Using Monocle with your application to generate traces
 ### Enable Monocle tracing
 You need to import monocle package and invoke the API ``setup_monocle_telemetry(workflow=<workflow-name>)`` to enable the tracing. The 'workflow-name' is what you define to identify the give application workflow, for example "customer-chatbot". Monocle trace will include this name in every trace. The trace output will include a list of spans in the traces. You can print the output on the console or send it to an HTTP endpoint.
 
 ### Using Monocle's out of box support of genAI technology components
 Monocle community has done the hard work of figuring out what to trace and how to extract relevant details from multiple genAI technology components. For example, if you have a python app coded using LlamaIndex and using models hostsed in OpenAI, Monocle can seamlessly trace your app. All you need to do enable Monocle tracing.
+
+### Using Monocle's Support for Adding Custom Attributes
+Monocle provides users with the ability to add custom attributes to various spans, such as inference and retrieval spans, by utilizing the output processor within its metamodel. This feature allows for dynamic attribute assignment through lambda functions, which operate on an arguments dictionary.
+The arguments dictionary contains key-value pairs that can be used to compute custom attributes. The dictionary includes the following components: 
+```python
+arguments = {"instance":instance, "args":args, "kwargs":kwargs, "output":return_value}
+```
+By leveraging this dictionary, users can define custom attributes for spans, enabling the integration of additional context and information into the tracing process. The lambda functions used in the attributes field can access and process these values to enrich the span with relevant custom data.
+
 #### Example - Enable Monocle tracing in your application
 ```python
 from monocle_apptrace.instrumentor import setup_monocle_telemetry
@@ -48,7 +69,7 @@ chain.invoke({"number":2})
 # Request callbacks: Finally, let's use the request `callbacks` to achieve the same result
 chain = LLMChain(llm=llm, prompt=prompt)
 chain.invoke({"number":2}, {"callbacks":[handler]})
-    
+
 ```
 
 ### Accessing monocle trace
@@ -63,6 +84,12 @@ setup_monocle_telemetry(workflow_name = "simple_math_app",
 ```
 To print the trace on the console, use ```ConsoleSpanExporter()``` instead of ```FileSpanExporter()```
 
+For Azure:
+    Install the Azure support as shown in the setup section, then use  ```AzureBlobSpanExporter()``` to upload the traces to Azure. 
+
+For AWS:
+    Install the AWS support as shown in the setup section, then use  ```S3SpanExporter()``` to upload the traces to an S3 bucket.
+ 
 ### Leveraging Monocle's extensibility to handle customization 
 When the out of box features from app frameworks are not sufficent, the app developers have to add custom code. For example, if you are extending a LLM class in LlamaIndex to use a model hosted in NVIDIA Triton. This new class is not know to Monocle. You can specify this new class method part of Monocle enabling API and it will be able to trace it.
 

{monocle_apptrace-0.1.1 → monocle_apptrace-0.3.0b1}/PKG-INFO

@@ -1,11 +1,9 @@
 Metadata-Version: 2.3
 Name: monocle_apptrace
-Version: 0.1.1
+Version: 0.3.0b1
 Summary: package with monocle genAI tracing
 Project-URL: Homepage, https://github.com/monocle2ai/monocle
 Project-URL: Issues, https://github.com/monocle2ai/monocle/issues
-License-File: LICENSE
-License-File: NOTICE
 Classifier: License :: OSI Approved :: MIT License
 Classifier: Operating System :: OS Independent
 Classifier: Programming Language :: Python :: 3
@@ -15,17 +13,24 @@ Requires-Dist: opentelemetry-instrumentation
 Requires-Dist: opentelemetry-sdk>=1.21.0
 Requires-Dist: requests
 Requires-Dist: wrapt>=1.14.0
+Provides-Extra: aws
+Requires-Dist: boto3==1.35.19; extra == 'aws'
+Provides-Extra: azure
+Requires-Dist: azure-storage-blob==12.22.0; extra == 'azure'
 Provides-Extra: dev
 Requires-Dist: datasets==2.20.0; extra == 'dev'
 Requires-Dist: faiss-cpu==1.8.0; extra == 'dev'
 Requires-Dist: instructorembedding==1.0.1; extra == 'dev'
 Requires-Dist: langchain-chroma==0.1.1; extra == 'dev'
 Requires-Dist: langchain-community==0.2.5; extra == 'dev'
+Requires-Dist: langchain-mistralai==0.1.13; extra == 'dev'
 Requires-Dist: langchain-openai==0.1.8; extra == 'dev'
 Requires-Dist: langchain==0.2.5; extra == 'dev'
 Requires-Dist: llama-index-embeddings-huggingface==0.2.0; extra == 'dev'
+Requires-Dist: llama-index-llms-mistralai==0.1.20; extra == 'dev'
 Requires-Dist: llama-index-vector-stores-chroma==0.1.9; extra == 'dev'
 Requires-Dist: llama-index==0.10.30; extra == 'dev'
+Requires-Dist: mistral-haystack==0.0.2; extra == 'dev'
 Requires-Dist: numpy==1.26.4; extra == 'dev'
 Requires-Dist: parameterized==0.9.0; extra == 'dev'
 Requires-Dist: pytest==8.0.0; extra == 'dev'

{monocle_apptrace-0.1.1 → monocle_apptrace-0.3.0b1}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "monocle_apptrace"
-version = "0.1.1"
+version = "0.3.0b1"
 authors = []
 description = "package with monocle genAI tracing"
 readme = "README.md"
@@ -43,7 +43,18 @@ dev = [
   'llama-index==0.10.30',
   'llama-index-embeddings-huggingface==0.2.0',
   'llama-index-vector-stores-chroma==0.1.9',
-  'parameterized==0.9.0'
+  'parameterized==0.9.0',
+  'llama-index-llms-mistralai==0.1.20',
+  'langchain-mistralai==0.1.13',
+  'mistral-haystack==0.0.2'
+]
+
+azure = [
+    'azure-storage-blob==12.22.0',
+]
+
+aws = [
+    'boto3==1.35.19',
 ]
 
 [project.urls]

monocle_apptrace-0.3.0b1/src/monocle_apptrace/botocore/__init__.py

@@ -0,0 +1,9 @@
+import os
+from monocle_apptrace.utils import get_wrapper_methods_config
+
+parent_dir = os.path.abspath(os.path.join(os.path.dirname(__file__), '..'))
+BOTOCORE_METHODS = get_wrapper_methods_config(
+    wrapper_methods_config_path=os.path.join(parent_dir, 'metamodel', 'maps', 'botocore_methods.json'),
+    attributes_config_base_path=os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
+
+

monocle_apptrace-0.3.0b1/src/monocle_apptrace/constants.py

@@ -0,0 +1,36 @@
+# Azure environment constants
+AZURE_ML_ENDPOINT_ENV_NAME = "AZUREML_ENTRY_SCRIPT"
+AZURE_FUNCTION_WORKER_ENV_NAME = "FUNCTIONS_WORKER_RUNTIME"
+AZURE_APP_SERVICE_ENV_NAME = "WEBSITE_SITE_NAME"
+AWS_LAMBDA_ENV_NAME = "AWS_LAMBDA_RUNTIME_API"
+GITHUB_CODESPACE_ENV_NAME = "CODESPACES"
+
+AWS_LAMBDA_FUNCTION_IDENTIFIER_ENV_NAME = "AWS_LAMBDA_FUNCTION_NAME"
+AZURE_FUNCTION_IDENTIFIER_ENV_NAME = "WEBSITE_SITE_NAME"
+AZURE_APP_SERVICE_IDENTIFIER_ENV_NAME = "WEBSITE_DEPLOYMENT_ID"
+GITHUB_CODESPACE_IDENTIFIER_ENV_NAME = "GITHUB_REPOSITORY"
+# Azure naming reference can be found here
+# https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/ready/azure-best-practices/resource-abbreviations
+AZURE_FUNCTION_NAME = "azure.func"
+AZURE_APP_SERVICE_NAME = "azure.asp"
+AZURE_ML_SERVICE_NAME = "azure.mlw"
+AWS_LAMBDA_SERVICE_NAME = "aws.lambda"
+GITHUB_CODESPACE_SERVICE_NAME = "github_codespace"
+
+# Env variables to identify infra service type
+service_type_map = {
+    AZURE_ML_ENDPOINT_ENV_NAME: AZURE_ML_SERVICE_NAME,
+    AZURE_APP_SERVICE_ENV_NAME: AZURE_APP_SERVICE_NAME,
+    AZURE_FUNCTION_WORKER_ENV_NAME: AZURE_FUNCTION_NAME,
+    AWS_LAMBDA_ENV_NAME: AWS_LAMBDA_SERVICE_NAME,
+    GITHUB_CODESPACE_ENV_NAME: GITHUB_CODESPACE_SERVICE_NAME
+}
+
+# Env variables to identify infra service name
+service_name_map = {
+    AZURE_APP_SERVICE_NAME: AZURE_APP_SERVICE_IDENTIFIER_ENV_NAME,
+    AZURE_FUNCTION_NAME: AZURE_FUNCTION_IDENTIFIER_ENV_NAME,
+    AZURE_ML_SERVICE_NAME: AZURE_ML_ENDPOINT_ENV_NAME,
+    AWS_LAMBDA_SERVICE_NAME: AWS_LAMBDA_FUNCTION_IDENTIFIER_ENV_NAME,
+    GITHUB_CODESPACE_SERVICE_NAME: GITHUB_CODESPACE_IDENTIFIER_ENV_NAME
+}

monocle_apptrace-0.3.0b1/src/monocle_apptrace/exporters/aws/s3_exporter.py

@@ -0,0 +1,158 @@
+import os
+import time
+import random
+import datetime
+import logging
+import asyncio
+import boto3
+from botocore.exceptions import ClientError
+from botocore.exceptions import (
+    BotoCoreError,
+    ConnectionClosedError,
+    ConnectTimeoutError,
+    EndpointConnectionError,
+    ReadTimeoutError,
+)
+from opentelemetry.sdk.trace import ReadableSpan
+from opentelemetry.sdk.trace.export import SpanExporter, SpanExportResult
+from monocle_apptrace.exporters.base_exporter import SpanExporterBase
+from typing import Sequence
+import json
+logger = logging.getLogger(__name__)
+
+class S3SpanExporter(SpanExporterBase):
+    def __init__(self, bucket_name=None, region_name=None):
+        super().__init__()
+        # Use environment variables if credentials are not provided
+        DEFAULT_FILE_PREFIX = "monocle_trace_"
+        DEFAULT_TIME_FORMAT = "%Y-%m-%d_%H.%M.%S"
+        self.max_batch_size = 500
+        self.export_interval = 1
+        self.s3_client = boto3.client(
+            's3',
+            aws_access_key_id=os.getenv('AWS_ACCESS_KEY_ID'),
+            aws_secret_access_key=os.getenv('AWS_SECRET_ACCESS_KEY'),
+            region_name=region_name,
+        )
+        self.bucket_name = bucket_name or os.getenv('MONOCLE_S3_BUCKET_NAME','default-bucket')
+        self.file_prefix = DEFAULT_FILE_PREFIX
+        self.time_format = DEFAULT_TIME_FORMAT
+        self.export_queue = []
+        self.last_export_time = time.time()
+
+        # Check if bucket exists or create it
+        if not self.__bucket_exists(self.bucket_name):
+            try:
+                self.s3_client.create_bucket(
+                Bucket=self.bucket_name,
+                CreateBucketConfiguration={'LocationConstraint': region_name}
+                 )
+                logger.info(f"Bucket {self.bucket_name} created successfully.")
+            except ClientError as e:
+                logger.error(f"Error creating bucket {self.bucket_name}: {e}")
+                raise e
+
+    def __bucket_exists(self, bucket_name):
+        try:
+            # Check if the bucket exists by calling head_bucket
+            self.s3_client.head_bucket(Bucket=bucket_name)
+            return True
+        except ClientError as e:
+            error_code = e.response['Error']['Code']
+            if error_code == '404':
+                # Bucket not found
+                logger.error(f"Bucket {bucket_name} does not exist (404).")
+                return False
+            elif error_code == '403':
+                # Permission denied
+                logger.error(f"Access to bucket {bucket_name} is forbidden (403).")
+                raise PermissionError(f"Access to bucket {bucket_name} is forbidden.")
+            elif error_code == '400':
+                # Bad request or malformed input
+                logger.error(f"Bad request for bucket {bucket_name} (400).")
+                raise ValueError(f"Bad request for bucket {bucket_name}.")
+            else:
+                # Other client errors
+                logger.error(f"Unexpected error when accessing bucket {bucket_name}: {e}")
+                raise e
+        except TypeError as e:
+            # Handle TypeError separately
+            logger.error(f"Type error while checking bucket existence: {e}")
+            raise e
+
+    def export(self, spans: Sequence[ReadableSpan]) -> SpanExportResult:
+        """Synchronous export method that internally handles async logic."""
+        try:
+            # Run the asynchronous export logic in an event loop
+            asyncio.run(self.__export_async(spans))
+            return SpanExportResult.SUCCESS
+        except Exception as e:
+            logger.error(f"Error exporting spans: {e}")
+            return SpanExportResult.FAILURE
+
+    async def __export_async(self, spans: Sequence[ReadableSpan]) -> SpanExportResult:
+        try:
+            # Add spans to the export queue
+            for span in spans:
+                self.export_queue.append(span)
+                # If the queue reaches MAX_BATCH_SIZE, export the spans
+                if len(self.export_queue) >= self.max_batch_size:
+                    await self.__export_spans()
+
+            # Check if it's time to force a flush
+            current_time = time.time()
+            if current_time - self.last_export_time >= self.export_interval:
+                await self.__export_spans()  # Export spans if time interval has passed
+                self.last_export_time = current_time  # Reset the last export time
+
+            return SpanExportResult.SUCCESS
+        except Exception as e:
+            logger.error(f"Error exporting spans: {e}")
+            return SpanExportResult.FAILURE
+
+    def __serialize_spans(self, spans: Sequence[ReadableSpan]) -> str:
+        try:
+            # Serialize spans to JSON or any other format you prefer
+            valid_json_list = []
+            for span in spans:
+                try:
+                    valid_json_list.append(span.to_json(indent=0).replace("\n", ""))
+                except json.JSONDecodeError as e:
+                    logger.warning(f"Invalid JSON format in span data: {span.context.span_id}. Error: {e}")
+                    continue
+            ndjson_data = "\n".join(valid_json_list) + "\n"
+            return ndjson_data
+        except Exception as e:
+            logger.warning(f"Error serializing spans: {e}")
+
+
+    async def __export_spans(self):
+        if len(self.export_queue) == 0:
+            return
+
+        # Take a batch of spans from the queue
+        batch_to_export = self.export_queue[:self.max_batch_size]
+        serialized_data = self.__serialize_spans(batch_to_export)
+        self.export_queue = self.export_queue[self.max_batch_size:]
+        try:
+            self.__upload_to_s3(serialized_data)
+        except Exception as e:
+            logger.error(f"Failed to upload span batch: {e}")
+
+    @SpanExporterBase.retry_with_backoff(exceptions=(EndpointConnectionError, ConnectionClosedError, ReadTimeoutError, ConnectTimeoutError))
+    def __upload_to_s3(self, span_data_batch: str):
+        current_time = datetime.datetime.now().strftime(self.time_format)
+        file_name = f"{self.file_prefix}{current_time}.ndjson"
+        self.s3_client.put_object(
+            Bucket=self.bucket_name,
+            Key=file_name,
+            Body=span_data_batch
+        )
+        logger.info(f"Span batch uploaded to AWS S3 as {file_name}.")
+
+    async def force_flush(self, timeout_millis: int = 30000) -> bool:
+        await self.__export_spans()  # Export any remaining spans in the queue
+        return True
+
+    def shutdown(self) -> None:
+        logger.info("S3SpanExporter has been shut down.")

monocle_apptrace-0.3.0b1/src/monocle_apptrace/exporters/azure/blob_exporter.py

@@ -0,0 +1,125 @@
+import os
+import time
+import random
+import datetime
+import logging
+import asyncio
+from azure.storage.blob import BlobServiceClient, BlobClient, ContainerClient
+from azure.core.exceptions import ResourceNotFoundError, ClientAuthenticationError, ServiceRequestError
+from opentelemetry.sdk.trace import ReadableSpan
+from opentelemetry.sdk.trace.export import SpanExporter, SpanExportResult
+from typing import Sequence
+from monocle_apptrace.exporters.base_exporter import SpanExporterBase
+import json
+logger = logging.getLogger(__name__)
+
+class AzureBlobSpanExporter(SpanExporterBase):
+    def __init__(self, connection_string=None, container_name=None):
+        super().__init__()
+        DEFAULT_FILE_PREFIX = "monocle_trace_"
+        DEFAULT_TIME_FORMAT = "%Y-%m-%d_%H.%M.%S"
+        self.max_batch_size = 500
+        self.export_interval = 1
+        # Use default values if none are provided
+        if not connection_string:
+            connection_string = os.getenv('MONOCLE_BLOB_CONNECTION_STRING')
+            if not connection_string:
+                raise ValueError("Azure Storage connection string is not provided or set in environment variables.")
+
+        if not container_name:
+            container_name = os.getenv('MONOCLE_BLOB_CONTAINER_NAME', 'default-container')
+
+        self.blob_service_client = BlobServiceClient.from_connection_string(connection_string)
+        self.container_name = container_name
+        self.file_prefix = DEFAULT_FILE_PREFIX
+        self.time_format = DEFAULT_TIME_FORMAT
+
+        # Check if container exists or create it
+        if not self.__container_exists(container_name):
+            try:
+                self.blob_service_client.create_container(container_name)
+                logger.info(f"Container {container_name} created successfully.")
+            except Exception as e:
+                logger.error(f"Error creating container {container_name}: {e}")
+                raise e
+
+    def __container_exists(self, container_name):
+        try:
+            container_client = self.blob_service_client.get_container_client(container_name)
+            container_client.get_container_properties()
+            return True
+        except ResourceNotFoundError:
+            logger.error(f"Container {container_name} not found (404).")
+            return False
+        except ClientAuthenticationError:
+            logger.error(f"Access to container {container_name} is forbidden (403).")
+            raise PermissionError(f"Access to container {container_name} is forbidden.")
+        except Exception as e:
+            logger.error(f"Unexpected error when checking if container {container_name} exists: {e}")
+            raise e
+
+    def export(self, spans: Sequence[ReadableSpan]) -> SpanExportResult:
+        """Synchronous export method that internally handles async logic."""
+        try:
+            # Run the asynchronous export logic in an event loop
+            asyncio.run(self._export_async(spans))
+            return SpanExportResult.SUCCESS
+        except Exception as e:
+            logger.error(f"Error exporting spans: {e}")
+            return SpanExportResult.FAILURE
+
+    async def _export_async(self, spans: Sequence[ReadableSpan]):
+        """The actual async export logic is run here."""
+        # Add spans to the export queue
+        for span in spans:
+            self.export_queue.append(span)
+            if len(self.export_queue) >= self.max_batch_size:
+                await self.__export_spans()
+
+        # Force a flush if the interval has passed
+        current_time = time.time()
+        if current_time - self.last_export_time >= self.export_interval:
+            await self.__export_spans()
+            self.last_export_time = current_time
+
+    def __serialize_spans(self, spans: Sequence[ReadableSpan]) -> str:
+        try:
+            valid_json_list = []
+            for span in spans:
+                try:
+                    valid_json_list.append(span.to_json(indent=0).replace("\n", ""))
+                except json.JSONDecodeError as e:
+                    logger.warning(f"Invalid JSON format in span data: {span.context.span_id}. Error: {e}")
+                    continue
+
+            ndjson_data = "\n".join(valid_json_list) + "\n"
+            return ndjson_data
+        except Exception as e:
+            logger.warning(f"Error serializing spans: {e}")
+
+    async def __export_spans(self):
+        if len(self.export_queue) == 0:
+            return
+
+        batch_to_export = self.export_queue[:self.max_batch_size]
+        serialized_data = self.__serialize_spans(batch_to_export)
+        self.export_queue = self.export_queue[self.max_batch_size:]
+        try:
+            self.__upload_to_blob(serialized_data)
+        except Exception as e:
+            logger.error(f"Failed to upload span batch: {e}")
+
+    @SpanExporterBase.retry_with_backoff(exceptions=(ResourceNotFoundError, ClientAuthenticationError, ServiceRequestError))
+    def __upload_to_blob(self, span_data_batch: str):
+        current_time = datetime.datetime.now().strftime(self.time_format)
+        file_name = f"{self.file_prefix}{current_time}.ndjson"
+        blob_client = self.blob_service_client.get_blob_client(container=self.container_name, blob=file_name)
+        blob_client.upload_blob(span_data_batch, overwrite=True)
+        logger.info(f"Span batch uploaded to Azure Blob Storage as {file_name}.")
+
+    async def force_flush(self, timeout_millis: int = 30000) -> bool:
+        await self.__export_spans()
+        return True
+
+    def shutdown(self) -> None:
+        logger.info("AzureBlobSpanExporter has been shut down.")

monocle_apptrace-0.3.0b1/src/monocle_apptrace/exporters/base_exporter.py

@@ -0,0 +1,48 @@
+import time
+import random
+import logging
+from abc import ABC, abstractmethod
+from opentelemetry.sdk.trace import ReadableSpan
+from opentelemetry.sdk.trace.export import SpanExporter, SpanExportResult
+from typing import Sequence
+import asyncio
+
+logger = logging.getLogger(__name__)
+
+class SpanExporterBase(ABC):
+    def __init__(self):
+        self.backoff_factor = 2
+        self.max_retries = 10
+        self.export_queue = []
+        self.last_export_time = time.time()
+
+    @abstractmethod
+    async def export(self, spans: Sequence[ReadableSpan]) -> SpanExportResult:
+        pass
+
+    @abstractmethod
+    async def force_flush(self, timeout_millis: int = 30000) -> bool:
+        pass
+
+    def shutdown(self) -> None:
+        pass
+
+    @staticmethod
+    def retry_with_backoff(retries=3, backoff_in_seconds=1, max_backoff_in_seconds=32, exceptions=(Exception,)):
+        def decorator(func):
+            def wrapper(*args, **kwargs):
+                attempt = 0
+                while attempt < retries:
+                    try:
+                        return func(*args, **kwargs)
+                    except exceptions as e:
+                        attempt += 1
+                        sleep_time = min(max_backoff_in_seconds, backoff_in_seconds * (2 ** (attempt - 1)))
+                        sleep_time = sleep_time * (1 + random.uniform(-0.1, 0.1))  # Add jitter
+                        logger.warning(f"Network connectivity error, Attempt {attempt} failed: {e}. Retrying in {sleep_time:.2f} seconds...")
+                        time.sleep(sleep_time)
+                raise Exception(f"Failed after {retries} attempts")
+
+            return wrapper
+
+        return decorator

monocle_apptrace-0.3.0b1/src/monocle_apptrace/exporters/exporter_processor.py

@@ -0,0 +1,19 @@
+from abc import ABC, abstractmethod
+import logging
+from typing import Callable
+
+logger = logging.getLogger(__name__)
+
+class ExportTaskProcessor(ABC):
+
+    @abstractmethod
+    def start(self):
+        return
+
+    @abstractmethod
+    def stop(self):
+        return
+
+    @abstractmethod
+    def queue_task(self, async_task: Callable[[Callable, any], any] = None, args: any = None):
+        return

monocle_apptrace-0.3.0b1/src/monocle_apptrace/exporters/monocle_exporters.py

@@ -0,0 +1,27 @@
+from typing import Dict, Any
+import os, warnings
+from importlib import import_module
+from opentelemetry.sdk.trace.export import SpanExporter, ConsoleSpanExporter
+from monocle_apptrace.exporters.file_exporter import FileSpanExporter
+
+monocle_exporters:Dict[str, Any] = {
+    "s3": {"module": "monocle_apptrace.exporters.aws.s3_exporter", "class": "S3SpanExporter"},
+    "blob": {"module":"monocle_apptrace.exporters.azure.blob_exporter", "class": "AzureBlobSpanExporter"},
+    "okahu": {"module":"monocle_apptrace.exporters.okahu.okahu_exporter", "class": "OkahuSpanExporter"},
+    "file": {"module":"monocle_apptrace.exporters.file_exporter", "class": "FileSpanExporter"}
+}
+
+def get_monocle_exporter() -> SpanExporter:
+    exporter_name = os.environ.get("MONOCLE_EXPORTER", "file")
+    try:
+        exporter_class_path  = monocle_exporters[exporter_name]
+    except Exception as ex:
+        warnings.warn(f"Unsupported Monocle span exporter setting {exporter_name}, using default FileSpanExporter.")
+        return FileSpanExporter()
+    try:
+        exporter_module = import_module(exporter_class_path.get("module"))
+        exporter_class = getattr(exporter_module, exporter_class_path.get("class"))
+        return exporter_class()
+    except Exception as ex:
+        warnings.warn(f"Unable to set Monocle span exporter to {exporter_name}, error {ex}. Using ConsoleSpanExporter")
+        return ConsoleSpanExporter()