levelapp 0.1.15__py3-none-any.whl

levelapp-0.1.15.dist-info/METADATA

@@ -0,0 +1,571 @@
+Metadata-Version: 2.4
+Name: levelapp
+Version: 0.1.15
+Summary: LevelApp is an evaluation framework for AI/LLM-based software application. [Powered by Norma]
+Project-URL: Homepage, https://github.com/levelapp-org
+Project-URL: Repository, https://github.com/levelapp-org/levelapp-framework
+Project-URL: Documentation, https://levelapp.readthedocs.io
+Project-URL: Issues, https://github.com/levelapp-org/levelapp-framework/issues
+Author-email: Mohamed Sofiene KADRI <ms.kadri.dev@gmail.com>
+License-File: LICENSE
+Keywords: ai,evaluation,framework,llm,testing
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Software Development :: Testing
+Requires-Python: >=3.12
+Requires-Dist: backoff>=2.2.1
+Requires-Dist: google-api-core>=2.25.1
+Requires-Dist: google-auth>=2.40.3
+Requires-Dist: google-cloud-firestore>=2.21.0
+Requires-Dist: httpx>=0.28.1
+Requires-Dist: humanize>=4.13.0
+Requires-Dist: numpy>=2.3.2
+Requires-Dist: pandas-stubs==2.3.0.250703
+Requires-Dist: pandas>=2.3.1
+Requires-Dist: pydantic>=2.11.7
+Requires-Dist: python-dotenv>=1.1.1
+Requires-Dist: pyyaml>=6.0.2
+Requires-Dist: rapid>=0.0.3
+Requires-Dist: rapidfuzz>=3.13.0
+Requires-Dist: requests>=2.32.4
+Requires-Dist: tenacity>=9.1.2
+Provides-Extra: dev
+Requires-Dist: google-api-core>=2.25.1; extra == 'dev'
+Requires-Dist: google-auth>=2.40.3; extra == 'dev'
+Requires-Dist: google-cloud-firestore>=2.21.0; extra == 'dev'
+Requires-Dist: httpx>=0.28.1; extra == 'dev'
+Requires-Dist: humanize>=4.13.0; extra == 'dev'
+Requires-Dist: numpy>=2.3.2; extra == 'dev'
+Requires-Dist: pandas-stubs==2.3.0.250703; extra == 'dev'
+Requires-Dist: pandas>=2.3.1; extra == 'dev'
+Requires-Dist: pydantic>=2.11.7; extra == 'dev'
+Requires-Dist: python-dotenv>=1.1.1; extra == 'dev'
+Requires-Dist: pyyaml>=6.0.2; extra == 'dev'
+Requires-Dist: rapid>=0.0.3; extra == 'dev'
+Requires-Dist: rapidfuzz>=3.13.0; extra == 'dev'
+Requires-Dist: requests>=2.32.4; extra == 'dev'
+Requires-Dist: tenacity>=9.1.2; extra == 'dev'
+Provides-Extra: embeddings
+Requires-Dist: bert-score>=0.3; extra == 'embeddings'
+Requires-Dist: torch>=2.0; extra == 'embeddings'
+Requires-Dist: transformers>=4.40; extra == 'embeddings'
+Description-Content-Type: text/markdown
+
+# LevelApp: AI/LLM Evaluation Framework for Regression Testing
+
+[![PyPI version](https://badge.fury.io/py/levelapp.svg)](https://badge.fury.io/py/levelapp)  
+[![License](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)  
+[![Python Version](https://img.shields.io/badge/python-3.12%2B-blue.svg)](https://www.python.org/downloads/)
+
+## Overview
+
+LevelApp is an evaluation framework designed for regression testing (black-box) of already built LLM-based systems in production or testing phases. It focuses on assessing the performance and reliability of AI/LLM applications through simulation and comparison modules. Powered by Norma.
+
+Key benefits:
+- Configuration-driven: Minimal coding required; define evaluations via YAML files.
+- Supports LLM-as-a-judge for qualitative assessments and quantitative metrics for metadata evaluation.
+- Modular architecture for easy extension to new workflows, evaluators, and repositories.
+
+## Features
+
+- **Simulator Module**: Evaluates dialogue systems by simulating conversations using predefined scripts. It uses an LLM as a judge to score replies against references and supports metrics (e.g., Exact, Embedded, Token-based, Fuzzy) for comparing extracted metadata to ground truth.
+- **Comparator Module**: Evaluates metadata extraction from JSON outputs (e.g., from legal/financial document processing with LLMs) by comparing against reference/ground-truth data.
+- **Configuration-Based Workflow**: Users provide YAML configs for endpoints, parameters, data sources, and metrics, reducing the need for custom code.
+- **Supported Workflows**: SIMULATOR, COMPARATOR, ASSESSOR (coming soon!).
+- **Repositories**: FIRESTORE, FILESYSTEM, MONGODB.
+- **Evaluators**: JUDGE, REFERENCE, RAG.
+- **Metrics**: Exact, Levenshtein, and more (see docs for full list).
+- **Data Sources**: Local or remote JSON for conversation scripts.
+
+## Installation
+
+Install LevelApp via pip:
+
+```bash
+pip install levelapp
+```
+
+### Prerequisites
+- Python 3.12 or higher.
+- API keys for LLM providers (e.g., OpenAI, Anthropic) if using external clients—store in a `.env` file.
+- Optional: Google Cloud credentials for Firestore repository.
+- Dependencies are automatically installed, including `openai`, `pydantic`, `numpy`, etc. (see `pyproject.toml` for full list).
+
+## Configuration
+
+LevelApp uses a YAML configuration file to define the evaluation setup. Create a `workflow_config.yaml` with the following structure:
+
+```yaml
+process:
+  project_name: "test-project"
+  workflow_type: SIMULATOR # Pick one of the following workflows: SIMULATOR, COMPARATOR, ASSESSOR.
+  evaluation_params:
+    attempts: 1  # Add the number of simulation attempts.
+    batch_size: 5
+
+evaluation:
+  evaluators: # Select from the following: JUDGE, REFERENCE, RAG.
+    - JUDGE
+    - REFERENCE
+  providers:
+    - openai
+    - ionos
+    - mistral
+    - grok
+    - gemini
+  metrics_map:
+    field_1: EXACT
+    field_2 : LEVENSHTEIN
+
+reference_data:
+  path: "../data/conversation_example_1.json"
+  data:
+
+endpoint:
+  name: conversational-agent
+  base_url: http://127.0.0.1:8000
+  path: /v1/chat
+  method: POST
+  timeout: 60
+  retry_count: 3
+  retry_backoff: 0.5
+  headers:
+    - name: model_id
+      value: meta-llama/Meta-Llama-3-8B-Instruct
+      secure: false
+    - name: x-api-key
+      value: API_KEY  # Load from .env file using python-dotenv.
+      secure: true
+    - name: Content-Type
+      value: application/json
+      secure: false
+  request_schema:
+    # Static field to be included in every request.
+    - field_path: message.source
+      value: system
+      value_type: static
+      required: true
+      
+    # Dynamic field to be populated from runtime context.
+    - field_path: message.text
+      value: message_text  # the key from the runtime context.
+      value_type: dynamic
+      required: true
+      
+    # Env-based field (from OS environment variables).
+    - field_path: metadata.env
+      value: ENV_VAR_NAME
+      value_type: env
+      required: false
+      
+  response_mapping:
+    # Map the response fields that will be extracted.
+    - field_path: reply.text
+      extract_as: agent_reply  # The simulator requires this key: 'agent_reply'.
+    - field_path: reply.metadata
+      extract_as: generated_metadata  # The simulator requires this key: 'generated_metadata'.
+    - field_path: reply.guardrail_flag
+      extract_as: guardrail_flag  # The simulator requires this key: 'guardrail_flag'.
+
+repository:
+  type: FIRESTORE # Pick one of the following: FIRESTORE, FILESYSTEM
+  project_id: "(default)"
+  database_name: ""
+```
+
+- **Endpoint Configuration**: Define how to interact with your LLM-based system (base URL, headers, request/response payload schema).
+- **Placeholders**: For dynamic request schema fields, use the values ('value') to dynamically populate these fields during runtime (e.g., `context = {'message_text': "Hello, world!"}`).
+- **Secrets**: Store API keys in `.env` and load via `python-dotenv` (e.g., `API_KEY=your_key_here`).
+
+For conversation scripts (used in Simulator), provide a JSON file with this schema:
+
+```json
+{
+  "scripts": [
+    {
+      "variable_request_schema": false,
+      "interactions": [
+        {
+          "user_message": "Hello, I would like to book an appointment with a doctor.",
+          "reference_reply": "Sure, I can help with that. Could you please specify the type of doctor you need to see?",
+          "interaction_type": "initial",
+          "reference_metadata": {},
+          "guardrail_flag": false,
+          "request_payload": {}
+        },
+        {
+          "user_message": "I need to see a cardiologist.",
+          "reference_reply": "When would you like to schedule your appointment?",
+          "interaction_type": "intermediate",
+          "reference_metadata": {},
+          "guardrail_flag": false,
+          "request_payload": {}
+        },
+        {
+          "user_message": "I would like to book it for next Monday morning.",
+          "reference_reply": "We have an available slot at 10 AM next Monday. Does that work for you?",
+          "interaction_type": "intermediate",
+          "reference_metadata": {
+            "appointment_type": "Cardiology",
+            "date": "next Monday",
+            "time": "10 AM"
+          },
+          "guardrail_flag": false,
+          "request_payload": {}
+        },
+        {
+          "id": "f4f2dd35-71d7-4b75-ba2b-93a4f546004a",
+          "user_message": "Yes, please book it for 10 AM then.",
+          "reference_reply": "Your appointment with the cardiologist is booked for 10 AM next Monday. Is there anything else I can help you with?",
+          "interaction_type": "final",
+          "reference_metadata": {},
+          "guardrail_flag": false,
+          "request_payload": {}
+        }
+      ],
+      "description": "A conversation about booking a doctor appointment.",
+      "details": {
+        "context": "Booking a doctor appointment"
+      }
+    }
+  ]
+}
+```
+- **Fields**: 
+  - **Scripts Level**:
+    - **description**: a brief description of the script.
+    - **details**: any additioanl information.
+    - **variable_request_schema**: a flag variable that defaults to `False`. 
+    When changed to True, it allows the user to pass the request payload content directly from the reference file
+    ignoring any configuration made in the YAML.
+    - **Interactions**: A list of single-turn conversation data for the simulation and evaluation process:
+      - **user_message_path**: If `variable_request_schema` is `True`, the user must indicate the path of the user message
+      in the attached **request_payload** dict. Example: ```"user_message_path": "user.message"``` 
+      for "request_payload": ```{"user": {"message": Hello, world!", "role": "user"}}```.
+      - **user_message**: The text content that will be used as a user message for the simulation,
+      - **reference_reply**: the text content of the reference reply. 
+      - **reference_metadata**: a dict containing the reference metadata. 
+      - **guardrail flags**: Guardrail flag (`True`/`False`). 
+      - **request payloads**: A dict containing the request payload that must be sent for each turn.
+
+In the `.env` you need to add the LLM providers credentials that will be used for the evaluation process. 
+```
+# Add the API key for any used provider:
+OPENAI_API_KEY=
+IONOS_API_KEY=
+ANTHROPIC_API_KEY=
+MISTRAL_API_KEY=
+GEMINI_API_KEY=
+GROK_API_KEY=
+
+# Include the model of choice for any used provider:
+OPENAI_MODEL= "gpt-4o-mini"
+GROK_MODEL = "llama-3.3-70b-versatile"
+GEMINI_MODEL = "gemini-2.5-flash"
+
+# For IONOS, you must include the base URL and the model ID.
+IONOS_BASE_URL="https://openai.inference.de-txl.ionos.com"
+IONOS_MODEL_ID="meta-llama/Llama-3.3-70B-Instruct"
+```
+
+## Usage Example
+
+To run an evaluation:
+
+1. Prepare your YAML config and JSON data files.
+2. Use the following Python script:
+
+```python
+if __name__ == "__main__":
+    from levelapp.workflow import WorkflowConfig
+    from levelapp.core.session import EvaluationSession
+
+    # Load configuration from YAML
+    config = WorkflowConfig.load(path="../data/workflow_config.yaml")
+
+    # Run evaluation session (You can enable/disable the monitoring aspect)
+    with EvaluationSession(session_name="test-session-1", workflow_config=config, enable_monitoring=False) as session:
+        session.run()
+        results = session.workflow.collect_results()
+        print("Results:", results)
+
+    stats = session.get_stats()
+    print(f"session stats:\n{stats}")
+```
+
+Alternatively, if you want to pass the configuration and reference data from in-memory variables, 
+you can manually load the data like the following:
+```python
+if __name__ == "__main__":
+    from levelapp.workflow import WorkflowConfig
+    from levelapp.core.session import EvaluationSession
+
+    
+    config_dict = {
+        "process": {
+            "project_name": "test-project",
+            "workflow_type": "SIMULATOR",  # Pick one of the following workflows: SIMULATOR, COMPARATOR, ASSESSOR.
+            "evaluation_params": {
+                "attempts": 1,  # Add the number of simulation attempts.
+            }
+        },
+        "evaluation": {
+            "evaluators": ["JUDGE", "REFERENCE"],  # Select from the following: JUDGE, REFERENCE, RAG.
+            "providers": ["openai", "ionos"],
+            "metrics_map": {
+                "field_1": "EXACT",
+                "field_2": "LEVENSHTEIN"
+            }
+        },
+        "reference_data": {
+            "path": "../data/conversation_example_1.json",
+            "data": None
+        },
+        "endpoint": {
+            "name": "conversational-agent",
+            "base_url": "http://127.0.0.1:8000",
+            "path": "/v1/chat",
+            "method": "POST",
+            "timeout": 60,
+            "retry_count": 3,
+            "retry_backoff": 0.5,
+            "headers": [
+                {
+                    "name": "model_id",
+                    "value": "meta-llama/Meta-Llama-3.1-8B-Instruct",
+                    "secure": False
+                },
+                {
+                    "name": "x-api-key",
+                    "value": "API_KEY",  # Load from .env file using python-dotenv.
+                    "secure": True 
+                },
+                {
+                    "name": "Content-Type",
+                    "value": "application/json",
+                    "secure": False
+                }
+            ],
+            "request_schema": [
+                {
+                    "field_path": "message.source",
+                    "value": "system",
+                    "value_type": "static",
+                    "required": True
+                },
+                {
+                    "field_path": "message.text",
+                    "value": "message_text",  # the key from the runtime context.
+                    "value_type": "dynamic",
+                    "required": True
+                },
+                {
+                    "field_path": "metadata.env",
+                    "value": "ENV_VAR_NAME",
+                    "value_type": "env",
+                    "required": False
+                }
+            ],
+            "response_mapping": [
+                {
+                    "field_path": "reply.text",
+                    "extract_as": "agent_reply"  # Remember that the simulator requires this key: 'agent_reply'.
+                },
+                {
+                    "field_path": "reply.metadata",
+                    "extract_as": "agent_reply"  # Remember that the simulator requires this key: 'agent_reply'.
+                },
+                {
+                    "field_path": "reply.guardrail_flag",
+                    "extract_as": "metadata"  # Remember that the simulator requires this key: 'agent_reply'.
+                }
+            ]
+        },
+        "repository": {
+            "type": "FIRESTORE",  # Pick one of the following: FIRESTORE, FILESYSTEM
+            "project_id": "(default)",
+            "database_name": ""
+        }
+    }
+
+    content = {
+        "scripts": [
+            {
+                "interactions": [
+                    {
+                        "user_message": "Hello!",
+                        "reference_reply": "Hello, how can I help you!"
+                    },
+                    {
+                        "user_message": "I need an apartment",
+                        "reference_reply": "sorry, but I can only assist you with booking medical appointments."
+                    },
+                ]
+            },
+        ]
+    }
+
+    # Load configuration from a dict variable
+    config = WorkflowConfig.from_dict(content=config_dict)
+
+    # Load reference data from dict variable
+    config.set_reference_data(content=content)
+
+    evaluation_session = EvaluationSession(
+        session_name="test-session", 
+        workflow_config=config, 
+        enable_monitoring=True  # To disable the monitoring aspect, set this to False.
+    )
+
+    with evaluation_session as session:
+        # Optional: Run connectivity test before the full evaluation
+        test_results = session.run_connectivity_test(
+            context={"user_message": "I want to book an appointment with a dentist."}
+        )
+        print(f"Connectivity Test Results:\n{test_results}\n---")
+        session.run()
+        results = session.workflow.collect_results()
+        print("Results:", results)
+
+    stats = session.get_stats()
+    print(f"session stats:\n{stats}")
+
+```
+
+
+- This loads the config, runs the specified workflow (e.g., Simulator), collects results, and prints stats.
+
+For more examples, see the `examples/` directory.
+
+Or, Check the following Colab Notebook for an easy and quick demo:<br>
+| Notebook                                                                                                                                                                                         | Description                        |                                                                                                                                                                    |
+|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| [Quick-tour of LevelApp framework](https://github.com/levelapp-org/levelapp-framework/blob/dev/examples/conversation_evaluation_example/LevelApp_Conversation_Simulator_Notebook.ipynb)          | Tutorial Notebook with UI widgets  |[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1tD2ljiBkrTxSfeRObTBrc2UmZvzqEuRU?usp=sharing) |
+
+## Visualization
+
+LevelApp includes powerful visualization capabilities to help you analyze and present evaluation results through interactive charts and dashboards.
+
+### Features
+
+- **Automatic Dashboard Generation**: Create comprehensive HTML dashboards with all evaluation metrics
+- **Multi-Format Export**: Export visualizations in HTML and PNG formats.
+- **Interactive Charts**: Generate interactive Plotly charts for detailed analysis
+- **Provider Comparison**: Compare performance across different LLM providers
+- **Score Trends**: Visualize score trends across conversation scripts
+- **Distribution Analysis**: Analyze score distributions for individual providers
+- **Summary Metrics**: Display key performance indicators and statistics
+
+### Installation
+
+To use visualization features, install the required dependencies:
+
+```bash
+pip install plotly kaleido jinja2
+```
+
+These dependencies enable:
+- `plotly`: Interactive chart generation
+- `kaleido`: Static image export (PNG, PDF)
+- `jinja2`: HTML dashboard templating
+
+### Basic Usage
+
+Generate visualizations directly from an evaluation session:
+
+```python
+from levelapp.core.session import EvaluationSession
+from levelapp.workflow import WorkflowConfig
+
+# Load configuration
+config = WorkflowConfig.load(path="workflow_config.yaml")
+
+# Run evaluation with visualization
+with EvaluationSession(
+    session_name="my-evaluation",
+    workflow_config=config,
+    enable_monitoring=True
+) as session:
+    # Run the evaluation
+    session.run()
+    
+    # Generate visualizations
+    files = session.visualize_results(
+        output_dir="./visualization_output",
+        formats=["html", "png"]
+    )
+    
+    # Access generated files
+    print(f"Dashboard: {files['html']}")
+    print(f"Charts: {files['png']}")
+```
+
+### Available Chart Types
+
+1. **Provider Comparison**: Bar charts comparing average scores across LLM providers
+2. **Score Trend**: Line charts showing score progression across conversation scripts
+3. **Score Distribution**: Histograms showing score distribution for specific providers
+4. **Summary Metrics**: Key performance indicators and aggregate statistics
+
+### Customization
+
+Customize visualizations by:
+
+- **Themes**: Choose from Plotly themes (`plotly`, `plotly_white`, `plotly_dark`, `ggplot2`, `seaborn`, etc.)
+- **Export Formats**: Select from `html` or `png`.
+- **Output Directory**: Specify custom paths for generated files
+- **Chart Layout**: Modify chart properties through the ChartGenerator API
+
+Example with custom theme:
+
+```python
+# Use dark theme for all charts
+chart_gen = ChartGenerator(theme="plotly_dark")
+
+# Generate with custom settings
+files = session.visualize_results(
+    output_dir="./reports",
+    formats=["html", "png", "pdf"],
+    theme="plotly_dark"
+)
+```
+
+### Example Output
+
+The visualization module generates:
+
+- **Interactive HTML Dashboard**: Complete evaluation report with all charts and metrics
+- **Static Images**: PNG/PDF exports for presentations and reports
+- **JSON Data**: Raw data export for custom processing
+
+For complete examples, see the `examples/visualization_example/` directory.
+
+## Documentation
+
+Detailed docs are in the `docs/` directory, including API references and advanced configuration.
+
+## Contributing
+
+Contributions are welcome! Please follow these steps:
+- Fork the repository on GitHub.
+- Create a feature branch (`git checkout -b feature/new-feature`).
+- Commit changes (`git commit -am 'Add new feature'`).
+- Push to the branch (`git push origin feature/new-feature`).
+- Open a pull request.
+
+Report issues via GitHub Issues. Follow the code of conduct (if applicable).
+
+## Acknowledgments
+
+- Powered by Norma.
+- Thanks to contributors and open-source libraries like Pydantic, NumPy, and OpenAI SDK.
+
+## License
+
+This project is licensed under the MIT License - see the [LICENCE](LICENCE) file for details.
+
+---

levelapp-0.1.15.dist-info/RECORD

@@ -0,0 +1,70 @@
+levelapp/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+levelapp/aspects/__init__.py,sha256=_OaPcjTWBizqcUdDVj5aYue7lG9ytjQGLhPvReriKnU,326
+levelapp/aspects/loader.py,sha256=IB2sZTmTdAvYHQZlH7PdZGQHh3r86P-zX3rIp0PyG2M,9577
+levelapp/aspects/logger.py,sha256=MJ9HphyHYkTE5-ajA_WuMUTM0qQzd0WIP243vF-pj3M,1698
+levelapp/aspects/monitor.py,sha256=ibUk01Y5y67_qBJRA5YzvjMX8QrRkMTJ-mN77ztuLlo,22113
+levelapp/aspects/sanitizer.py,sha256=zUqgb76tXJ8UUYtHp0Rz7q9PZjAHpSpHPPFfGTjjQNg,5229
+levelapp/clients/__init__.py,sha256=p2xgIhWuDHxMck_b0PcCo1NOGEYjpX5-2FVLGT4SHF4,4234
+levelapp/clients/anthropic.py,sha256=Bxp-HffcIPLwM9BLcTR7n-D8ZXYVWCmbr2oH33fKV04,4030
+levelapp/clients/gemini.py,sha256=nVPbntRN-t-hrPvF59wadX1Ptqp7eOlwsHUP2_PuQfI,4594
+levelapp/clients/groq.py,sha256=W8NNN6enl-wkQfKsCULBKYQmdnOorFUAHJaURlMDbZk,3669
+levelapp/clients/huggingface.py,sha256=uSo7EYpiUlgo3OGWcLZ2P77eFMaTVrIijwFU2RPZ_Oo,5301
+levelapp/clients/ionos.py,sha256=PIZIA6IqwJfs1GYmD8lXXkgfth-nmhe6d-m7-269zes,4275
+levelapp/clients/mistral.py,sha256=e1NRvP9qN7O2zWAzBbgdQmmUDHQfCRLtVKDJCrh0DNA,3777
+levelapp/clients/openai.py,sha256=lrz8eZNbJGJgXTB0rKkMVhUArilrThtBPq2rkgGf5Gk,3887
+levelapp/comparator/__init__.py,sha256=ynmc0mrx-JbcCqLH-z4hOVezqGocDbDQGqgbhWy2xzI,187
+levelapp/comparator/comparator.py,sha256=HONX6I_uzKlQWClP3Bj8gUkk2zsAV2ywxcqjlr3i7HI,8160
+levelapp/comparator/extractor.py,sha256=vJ9iEoWAtXo2r9r7X72uUQPKW3UZE9Kx3uIjCufEp9k,3910
+levelapp/comparator/schemas.py,sha256=qP_77sWW1SgpM7_u2rVsFXxUl1ogZ9wrOIrNv8QYR8M,1809
+levelapp/comparator/scorer.py,sha256=goxgZ-usGIpTPut6j1TcXDf9PWMpyQpW5swK0QSRrUE,9041
+levelapp/comparator/utils.py,sha256=Eu48nDrNzFr0lwAJJS0aNhKsAWQ72syTEWYMNYfg764,4331
+levelapp/config/__init__.py,sha256=9oaajE5zW-OVWOszUzMAG6nHDSbLQWa3KT6bVoSvzRA,137
+levelapp/config/endpoint.py,sha256=B-uIEKF-0_Y6Vo8MZ8eoCZocRkghijrdpwT3zq0FDLk,7647
+levelapp/config/prompts.py,sha256=NXOKRp5l1VQ9LO0pUojVH6TDJhWyZImsAvZEz2QiD9k,2206
+levelapp/core/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+levelapp/core/base.py,sha256=sEm_tsU-GjQWzSkHA8iZighC713pNkEFSOrL1U0pmB8,12597
+levelapp/core/schemas.py,sha256=E47d93MMOj4eRYZIqUyLBiE5Ye7WgwkOJPOWQ6swRmo,465
+levelapp/core/session.py,sha256=DCu0L5BMMBwpFNmIkoF2SkyM5PF1E9QiBYNpp5V5c80,11576
+levelapp/endpoint/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+levelapp/endpoint/client.py,sha256=kDGA-eKB_q45u2EyYdB3OHsQt0jpEMUCKLalPV-iJpo,6363
+levelapp/endpoint/client_test.py,sha256=RpV9338Uab2QVen5dHixurPJlwdVdJZNz-HTNr0KSgc,1170
+levelapp/endpoint/manager.py,sha256=7o-rAzsQ7O9fz4KITos4FXEB9YVbtbWQSc-3bbo_EsM,4218
+levelapp/endpoint/parsers.py,sha256=YuT5Ua3FHGNUuXbYh6fLn-sDWlnrscW5IwZR5sWjVRc,4225
+levelapp/endpoint/schemas.py,sha256=V0tpXC8aawHpX5zatderYa0fB14_QEkPiKRvfsuGZRM,851
+levelapp/endpoint/tester.py,sha256=nfKx09Uk4EHNuajZg0SYSH6jHf7FkxrwEJGZBY9PrVk,2048
+levelapp/evaluator/__init__.py,sha256=K-P75Q1FXXLCNqH1wyhT9sf4y2R9a1qR5449AXEsY1k,109
+levelapp/evaluator/evaluator.py,sha256=lSRxCunkzycP7bRDzKx63nPACSw6MP-jGV8dbNRgWaY,11278
+levelapp/metrics/__init__.py,sha256=x8iTaeDezJyQ9-NFe8GGvzwIBhyAJHWSRfBE3JRX-PE,1878
+levelapp/metrics/embedding.py,sha256=a3PNKBGedVS0Z9Ow5cOc0_aWgyoa8l_dAYRF4ViSMiI,2075
+levelapp/metrics/exact.py,sha256=Kb13nD2OVLrl3iYHaXrxDfrxDuhW0SMVvLAEXPaJtlY,6235
+levelapp/metrics/fuzzy.py,sha256=Rg8ashzMxtQwKO-z_LLzdj2PDIRqL4CBw6PGRf9IBrI,2598
+levelapp/metrics/token.py,sha256=yQi9hxT_fXTGjLiCCemDxQ4Uk2zD-wQYtSnDlI2AuuY,3521
+levelapp/metrics/embeddings/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+levelapp/metrics/embeddings/sentence_transformer.py,sha256=S0XgSr_kiQN9Dk9o1dm_t9Z9J0nGh_krdBbfreSo61o,1179
+levelapp/metrics/embeddings/torch_based.py,sha256=qQ3Y06H8OPhqFLSozr2PYwxxmWcFAF_pVSxHrdtyyR8,2219
+levelapp/plugins/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+levelapp/repository/__init__.py,sha256=hNmFRZ7kKJN1mMlOHeW9xf0j9Q7gqTXYJ3hMCzk9to4,79
+levelapp/repository/filesystem.py,sha256=-C2oVThZt16K41iZNSEaM2qO3tTPsFKVsDYQQiwo1Bk,7475
+levelapp/repository/firestore.py,sha256=K9JgxsNCelAKtzTDv19c1dHRlitMeRzo7H3caTlKuF8,10369
+levelapp/simulator/__init__.py,sha256=8Dz8g7rbpBZX3WoknVmMVoWm_VT72ZL9BABOF1xFpqs,83
+levelapp/simulator/schemas.py,sha256=0rZz5XfLxxfYF-7nk9f0EwHzGURidgKOYWz-U4qGu1Y,5860
+levelapp/simulator/simulator.py,sha256=jya4HmACv1y-TcrNRp62VA42aKNlmhdN7_IkHhUHVA8,21029
+levelapp/simulator/utils.py,sha256=smSrZ8praKINK0wFpKl3tmqr21OUz_dheUOTH0miTys,4882
+levelapp/visualization/__init__.py,sha256=MniD9ECzFoFb1KnLU2aqEUJcZSpae_Xm869yFbmDdR0,269
+levelapp/visualization/charts.py,sha256=JZDtN4bp994iiBrqhV6TO99qiRewmMkUu8BjifJlX4A,11399
+levelapp/visualization/dashboard.py,sha256=ZboEh0wbHctEYcItXly47Q7LkuQaN6b0j692ikFtq-Y,8426
+levelapp/visualization/exporter.py,sha256=AC5GX3n61j1mLNZrZ7LpUqwxw-M_kkOnxukjb42rFEw,5831
+levelapp/visualization/templates/base.html,sha256=0lye-LdIxvxcX_WFYZIeGocOdGZT9q9ibMIso656-0Q,4057
+levelapp/visualization/templates/comparator_dashboard.html,sha256=L_0M7y_ng-wnLZhrvSt8SD15tfUZxKD-swkD7cqDbT8,2508
+levelapp/visualization/templates/simulator_dashboard.html,sha256=tD3HD7d7BqEf98QWaQGqsATwI8tV78Fn25dbun3He5E,3321
+levelapp/workflow/__init__.py,sha256=27b2obG7ObhR43yd2uH-R0koRB7-DG8Emnvrq8EjsTA,193
+levelapp/workflow/base.py,sha256=dL85mrFDB42113ONIHAhmIfhNsIp9767uU9SxwauDMw,6753
+levelapp/workflow/config.py,sha256=hgch9KV-TMtbesEFi00eibFdk5JJStJwb5TGO5m-o-M,3124
+levelapp/workflow/context.py,sha256=KOXm_5HJWYyWfl9C83BqT37X7QVuUYSS-ZoDpZgXFQw,2696
+levelapp/workflow/factory.py,sha256=z1ttJmI59sU9HgOvPo3ixUJ_oPv838XgehfuOorlTt8,1634
+levelapp/workflow/registration.py,sha256=VHUHjLHXad5kjcKukaEOIf7hBZ09bT3HAzVmIT08aLo,359
+levelapp/workflow/runtime.py,sha256=a3REqikh3-QHj0uYikqx0b4xQjq-w6VNyiUandL5GWw,690
+levelapp-0.1.15.dist-info/METADATA,sha256=_FHrkNsuZlJdxrx5GENLWg_4kYhsDk51NOVoLe0QnFw,22194
+levelapp-0.1.15.dist-info/WHEEL,sha256=WLgqFyCfm_KASv4WHyYy0P3pM_m7J5L9k2skdKLirC8,87
+levelapp-0.1.15.dist-info/licenses/LICENSE,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+levelapp-0.1.15.dist-info/RECORD,,

levelapp-0.1.15.dist-info/WHEEL

@@ -0,0 +1,4 @@
+Wheel-Version: 1.0
+Generator: hatchling 1.28.0
+Root-Is-Purelib: true
+Tag: py3-none-any