monocle-apptrace 0.1.1__py3-none-any.whl → 0.3.0__py3-none-any.whl

monocle_apptrace/metamodel/spans/README.md

@@ -1,121 +0,0 @@
-# Monocle Span format
-Monocle generates [traces](../../../../Monocle_User_Guide.md#traces) which comprises of [spans](../../../../Monocle_User_Guide.md#spans). Note that Monocle trace is [OpenTelemetry format](https://opentelemetry.io/docs/concepts/signals/traces/) compatible. Each span is essentially a step in the execution that interacts with one of more GenAI technology components. This document explains the [span format](./span_format.json) that Monocle generates for GenAI application tracing.
-
-Per the OpenTelemetry convention, each span contains an attribute section and event section. In Monocle generated trace, the attribute sections includes details of GenAI entities used in the span. The event section includes the input, output and metadata related to the execution of that span.
-
-## Attributes 
-The attribute sections includes details of GenAI entities used in the span. For each entity used in the span in includes the entity name and entity type. For every type of entity, there are required and optional attributes listed below.
-### Json format
-```json
-    attributes: 
-        "span.type": "Monocle-span-type",
-        "entity.count": "count-of-entities",
-    
-        "entity.<index>.name": "Monocle-Entity-name",
-        "entity.<index>.type": "MonocleEntity.<entity-type>"
-        ...
-```
-The ```entity.count``` indicates total number of entities used in the given span. For each entity, the details are captured in ```entity.<index>.X```. For example, 
-```json
-    "attributes": {
-        "span.type": "Inference",
-        "entity.count": 2,
-        "entity.1.name": "AzureOpenAI",
-        "entity.1.type": "Inference.Azure_oai",
-        "entity.2.name": "gpt-35-turbo",
-        "entity.2.type": "Model.LLM",
-        "entity.2.model_name": "gpt-35-turbo",
-```
-
-### Entity type specific attributes
-#### MonocleEntity.Workflow
-| Name | Description | Values | Required |
-| - | - | - | - |
-| name | Entity name generated by Monocle | Name String | Required |
-| type | Monocle Entity type | MonocleEntity.Workflow | Required |
-| optional-attribute | Additional attribute specific to entity |  | Optional |
-
-### MonocleEntity.Model
-| Name | Description | Values | Required |
-| - | - | - | - |
-| name | Entity name generated by Monocle | Name String | Required |
-| type | Monocle Entity type | MonocleEntity.Model | Required |
-| model_name | Name of model | String | Required |
-| optional-attribute | Additional attribute specific to entity |  | Optional |
-
-### MonocleEntity.AppHosting
-| Name | Description | Values | Required |
-| - | - | - | - |
-| name | Entity name generated by Monocle | Name String | Required |
-| type | Monocle Entity type | MonocleEntity.AppHosting | Required |
-| optional-attribute | Additional attribute specific to entity |  | Optional |
-
-### MonocleEntity.Inference
-| Name | Description | Values | Required |
-| - | - | - | - |
-| name | Entity name generated by Monocle | Name String | Required |
-| type | Monocle Entity type | MonocleEntity.Inference | Required |
-| optional-attribute | Additional attribute specific to entity |  | Optional |
-
-### MonocleEntity.VectorDB
-| Name | Description | Values | Required |
-| - | - | - | - |
-| name | Entity name generated by Monocle | Name String | Required |
-| type | Monocle Entity type | MonocleEntity.VectorDB | Required |
-| optional-attribute | Additional attribute specific to entity |  | Optional |
-
-## Events
-The event section includes the input, output and metadata generated by that span execution. For each type of span, there are required and option input, output and metadata items listed below. If there's no data genearated in the space, the events will be an empty array.
-
-### Json format
-```json
-    "events" : [
-        {
-            "name": "data.input",
-            "timestamp": "UTC timestamp",
-            "attributes": {
-                "input_attribute": "value"
-           }
-        },
-        {
-            "name": "data.output",
-            "timestamp": "UTC timestamp",
-            "attributes": {
-                "output_attribute": "value"
-            }
-        },
-        {
-            "name": "metadata",
-            "timestamp": "UTC timestamp",
-            "attributes": {
-                "metadata_attribute": "value"
-            }
-        }
-    ]
-```
-
-## Span types and events
-The ```span.type``` captured in ```attributes``` section of the span dectates the format of the ```events```
-### SpanType.Retrieval
-| Name | Description | Values | Required |
-| - | - | - | - |
-| name | event name  | data.input or data.output or metadata | Required |
-| timestamp | timestap when the event occurred | UTC timestamp | Required |
-| attributes | input/output/metadata attributes generated in span | Dictionary | Required |
-
-### SpanType.Inference
-| Name | Description | Values | Required |
-| - | - | - | - |
-| name | event name  | data.input or data.output or metadata | Required |
-| timestamp | timestap when the event occurred | UTC timestamp | Required |
-| attributes | input/output/metadata attributes generated in span | Dictionary | Required |
-
-### SpanType.Workflow
-| Name | Description | Values | Required |
-| - | - | - | - |
-| name | event name  | data.input or data.output or metadata | Required |
-| timestamp | timestap when the event occurred | UTC timestamp | Required |
-| attributes | input/output/metadata attributes generated in span | Dictionary | Required |
-
-### SpanType.Internal
-Events will be empty

monocle_apptrace/metamodel/spans/span_example.json

@@ -1,140 +0,0 @@
-{
-    "name": "llamaindex.retrieve",
-    "context": {
-        "trace_id": "0x93cd0bf865b3ffcc3cf9c075dc3e3797",
-        "span_id": "0x5d3f839e900bda24",
-        "trace_state": "[]"
-    },
-    "kind": "SpanKind.CLIENT",
-    "parent_id": "0x7a63d63e42ccac60",
-    "start_time": "2024-09-09T14:38:45.237182Z",
-    "end_time": "2024-09-09T14:38:45.620112Z",
-    "status": {
-        "status_code": "OK"
-    },
-    "attributes": {
-        "span.type": "Retrieval", 
-        "entity.count": 2,
-        "entity.1.name": "ChromaVectorStore",
-        "entity.1.type": "vectorstore.chroma",
-        "entity.1.embedding-model-name": "BAAI/bge-small-en-v1.5",
-        "entity.2.name": "BAAI/bge-small-en-v1.5",
-        "entity.2.type": "model.embedding",
-        "entity.2.model_name": "BAAI/bge-small-en-v1.5"
-    },
-    "events": [
-        {
-            "name": "data.input",
-            "timestamp": "timestamp",
-            "attributes": {
-                "context_input": "question: What is an americano?"
-            }
-        },
-        {
-            "name": "data.output",
-            "timestamp": "timestamp",
-            "attributes": {
-                "context_output": "Coffee is a hot drink made from the roasted and ground seeds (coffee beans) of a tropical shrub\nA latte consists of one or more shots of espresso, served in a glass (or sometimes a cup), into which hot steamed milk is added\nAmericano is a type of coffee drink prepared by diluting an espresso shot with hot water at a 1:3 to 1:4 ratio, resulting in a drink that retains the complex flavors of espresso, but in a lighter way"
-            }
-        }
-
-    ],
-    "links": [],
-    "resource": {
-        "attributes": {
-            "service.name": "coffee-bot"
-        },
-        "schema_url": ""
-    }
-},
-{
-    "name": "llamaindex.openai",
-    "context": {
-        "trace_id": "0x93cd0bf865b3ffcc3cf9c075dc3e3797",
-        "span_id": "0x8b6363e1937a4d7b",
-        "trace_state": "[]"
-    },
-    "kind": "SpanKind.CLIENT",
-    "parent_id": "0x7a63d63e42ccac60",
-    "start_time": "2024-09-09T14:38:45.622174Z",
-    "end_time": "2024-09-09T14:38:46.514120Z",
-    "status": {
-        "status_code": "OK"
-    },
-    "attributes": {
-        "span.type": "inference",
-        "entity.count": 2,
-        "entity.1.name": "AzureOpenAI",
-        "entity.1.type": "inference.azure_oai",
-        "entity.1.provider_name": "openai.azure.com",
-        "entity.1.deployment": "kshitiz-gpt",
-        "entity.1.inference_endpoint": "https://okahu-openai-dev.openai.azure.com/",
-
-        "entity.2.name": "gpt-35-turbo",
-        "entity.2.type": "model.llm",
-        "entity.2.model_name": "gpt-35-turbo"
-    },
-    "events": [
-        {
-            "name": "data.input",
-            "timestamp": "timestamp",
-            "attributes": {
-                "question": "What is an americano?",
-            }
-        },
-        {
-            "name": "data.output",
-            "timestamp": "timestamp",
-            "attributes": {
-                "response": "An americano is a type of coffee drink that is made by diluting an espresso shot with hot water at a 1:3 to 1:4 ratio, resulting in a drink that retains the complex flavors of espresso, but in a lighter way.",
-            }
-        },
-        {
-            "name": "metadata",
-            "timestamp": "timestamp",
-            "attributes": {
-                "temperature": 0.1,
-                "completion_tokens": 52,
-                "prompt_tokens": 233,
-                "total_tokens": 285
-            }
-        }
-    ],
-    "links": [],
-    "resource": {
-        "attributes": {
-            "service.name": "coffee-bot"
-        },
-        "schema_url": ""
-    }
-}
-{
-    "name": "llamaindex.query",
-    "context": {
-        "trace_id": "0x93cd0bf865b3ffcc3cf9c075dc3e3797",
-        "span_id": "0x7a63d63e42ccac60",
-        "trace_state": "[]"
-    },
-    "kind": "SpanKind.CLIENT",
-    "parent_id": null,
-    "start_time": "2024-09-09T14:38:45.236627Z",
-    "end_time": "2024-09-09T14:38:46.514442Z",
-    "status": {
-        "status_code": "OK"
-    },
-    "attributes": {
-        "span.type": "workflow",
-        "entity.count": 1,
-        "entity.1.name": "coffee-bot",
-        "entity.1.type": "workflow.llama_index"
-    },
-    "events": [
-    ],
-    "links": [],
-    "resource": {
-        "attributes": {
-            "service.name": "coffee-bot"
-        },
-        "schema_url": ""
-    }
-}

monocle_apptrace/metamodel/spans/span_format.json

@@ -1,55 +0,0 @@
-{
-    "name": "span-name",
-    "context": {
-        "trace_id": "trace-id",
-        "span_id": "span-id",
-        "trace_state": "[]"
-    },
-    "kind": "SpanKind.CLIENT",
-    "parent_id": "parent-id or None (for root span)",
-    "start_time": "UTC timestamp",
-    "end_time": "UTC timestamp",
-    "status": {
-        "status_code": "OK or Error"
-    },
-    "attributes": {
-        "description": "List of AI component entities used in this span, eg Model, Inference hosting service. Needs to be one of the supported entity types.",
-    
-        "span.type": "Monocle-span-type",
-        "entity.count": "count-of-entities",
-    
-        "entity.<index>.name": "Monocle-Entity-name",
-        "entity.<index>.type": "Monocle-Entity-Type",
-        "entity.<index>.<attribute>": "Value"
-    },
-    "events" : [
-        {
-            "name": "data.input",
-            "timestamp": "UTC timestamp",
-            "attributes": {
-                "input_attribute": "value"
-           }
-        },
-        {
-            "name": "data.output",
-            "timestamp": "UTC timestamp",
-            "attributes": {
-                "output_attribute": "value"
-            }
-        },
-        {
-            "name": "metadata",
-            "timestamp": "UTC timestamp",
-            "attributes": {
-                "metadata_attribute": "value"
-            }
-        }
-    ],
-    "links": [],
-    "resource": {
-        "attributes": {
-            "service.name": "top-workflow-name"
-        },
-        "schema_url": ""
-    }
-}

monocle_apptrace/utils.py

@@ -1,93 +0,0 @@
-import logging
-import json
-from importlib import import_module
-import os
-from opentelemetry.trace import Span
-from monocle_apptrace.constants import azure_service_map, aws_service_map
-
-embedding_model_context = {}
-
-def set_span_attribute(span, name, value):
-    if value is not None:
-        if value != "":
-            span.set_attribute(name, value)
-
-def dont_throw(func):
-    """
-    A decorator that wraps the passed in function and logs exceptions instead of throwing them.
-
-    @param func: The function to wrap
-    @return: The wrapper function
-    """
-    # Obtain a logger specific to the function's module
-    logger = logging.getLogger(func.__module__)
-    # pylint: disable=inconsistent-return-statements
-    def wrapper(*args, **kwargs):
-        try:
-            return func(*args, **kwargs)
-        except Exception as ex:
-            logger.warning("Failed to execute %s, error: %s", func.__name__, str(ex))
-    return wrapper
-
-def with_tracer_wrapper(func):
-    """Helper for providing tracer for wrapper functions."""
-
-    def _with_tracer(tracer, to_wrap):
-        def wrapper(wrapped, instance, args, kwargs):
-            return func(tracer, to_wrap, wrapped, instance, args, kwargs)
-
-        return wrapper
-
-    return _with_tracer
-
-def resolve_from_alias(my_map, alias):
-    """Find a alias that is not none from list of aliases"""
-
-    for i in alias:
-        if i in my_map.keys():
-            return my_map[i]
-    return None
-
-def load_wrapper_from_config(config_file_path: str, module_name: str = None):
-    wrapper_methods = []
-    with open(config_file_path, encoding='UTF-8') as config_file:
-        json_data = json.load(config_file)
-        wrapper_methods = json_data["wrapper_methods"]
-        for wrapper_method in wrapper_methods:
-            wrapper_method["wrapper"] = get_wrapper_method(
-                wrapper_method["wrapper_package"], wrapper_method["wrapper_method"])
-            if "span_name_getter_method" in wrapper_method :
-                wrapper_method["span_name_getter"] = get_wrapper_method(
-                    wrapper_method["span_name_getter_package"],
-                    wrapper_method["span_name_getter_method"])
-        return wrapper_methods
-
-def get_wrapper_method(package_name: str, method_name: str):
-    wrapper_module = import_module("monocle_apptrace." + package_name)
-    return getattr(wrapper_module, method_name)
-
-def update_span_with_infra_name(span: Span, span_key: str):
-    for key,val  in azure_service_map.items():
-        if key in os.environ:
-            span.set_attribute(span_key, val)
-    for key,val  in aws_service_map.items():
-        if key in os.environ:
-            span.set_attribute(span_key, val)
-
-
-def set_embedding_model(model_name: str):
-    """
-    Sets the embedding model in the global context.
-
-    @param model_name: The name of the embedding model to set
-    """
-    embedding_model_context['embedding_model'] = model_name
-
-
-def get_embedding_model() -> str:
-    """
-    Retrieves the embedding model from the global context.
-
-    @return: The name of the embedding model, or 'unknown' if not set
-    """
-    return embedding_model_context.get('embedding_model', 'unknown')