veris-ai 1.3.0__py3-none-any.whl → 1.4.0__py3-none-any.whl
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Potentially problematic release.
This version of veris-ai might be problematic. Click here for more details.
- veris_ai/README.md +75 -0
- veris_ai/jaeger_interface/README.md +55 -91
- veris_ai/logging.py +116 -0
- veris_ai/tool_mock.py +75 -9
- veris_ai/utils.py +3 -0
- veris_ai-1.4.0.dist-info/METADATA +223 -0
- veris_ai-1.4.0.dist-info/RECORD +15 -0
- veris_ai-1.3.0.dist-info/METADATA +0 -457
- veris_ai-1.3.0.dist-info/RECORD +0 -13
- {veris_ai-1.3.0.dist-info → veris_ai-1.4.0.dist-info}/WHEEL +0 -0
- {veris_ai-1.3.0.dist-info → veris_ai-1.4.0.dist-info}/licenses/LICENSE +0 -0
veris_ai/README.md
ADDED
|
@@ -0,0 +1,75 @@
|
|
|
1
|
+
# Veris AI Module Architecture
|
|
2
|
+
|
|
3
|
+
This module contains the core implementation of the Veris AI Python SDK. Each component focuses on a specific aspect of tool mocking, tracing, and MCP integration.
|
|
4
|
+
|
|
5
|
+
## Quick Reference
|
|
6
|
+
|
|
7
|
+
**Purpose**: Core SDK implementation with modular architecture
|
|
8
|
+
**Entry Point**: [`__init__.py`](__init__.py) handles lazy imports and public API exports
|
|
9
|
+
**Source of Truth**: Individual module files contain implementation details
|
|
10
|
+
|
|
11
|
+
## Module Overview
|
|
12
|
+
|
|
13
|
+
**Semantic Tag**: `core-modules`
|
|
14
|
+
|
|
15
|
+
| Module | Purpose | Key Classes/Functions | Lines |
|
|
16
|
+
|--------|---------|----------------------|-------|
|
|
17
|
+
| [`tool_mock.py`](tool_mock.py) | Function mocking & FastAPI MCP | `VerisSDK`, `@mock`, `@stub` | 327 |
|
|
18
|
+
| [`braintrust_tracing.py`](braintrust_tracing.py) | Dual tracing instrumentation | `instrument()` | 283 |
|
|
19
|
+
| [`utils.py`](utils.py) | Type utilities & JSON schema | `extract_json_schema()` | 272 |
|
|
20
|
+
| [`logging.py`](logging.py) | Logging configuration | `setup_logging()` | 116 |
|
|
21
|
+
| [`models.py`](models.py) | Data models | Type definitions | 12 |
|
|
22
|
+
| [`jaeger_interface/`](jaeger_interface/) | Jaeger Query API wrapper | `JaegerClient` | See module README |
|
|
23
|
+
|
|
24
|
+
## Core Workflows
|
|
25
|
+
|
|
26
|
+
**Semantic Tag**: `implementation-flows`
|
|
27
|
+
|
|
28
|
+
### Mock Flow
|
|
29
|
+
1. **Decoration**: `@veris.mock()` captures function metadata
|
|
30
|
+
2. **Environment Check**: `ENV=simulation` determines behavior
|
|
31
|
+
3. **API Call**: POST to `{VERIS_ENDPOINT_URL}/api/v2/tool_mock`
|
|
32
|
+
4. **Type Conversion**: Response converted using `extract_json_schema()`
|
|
33
|
+
|
|
34
|
+
**Implementation**: [`tool_mock.py:200-250`](tool_mock.py)
|
|
35
|
+
|
|
36
|
+
### Spy Flow
|
|
37
|
+
1. **Pre-execution Logging**: Call details sent to `/api/v2/log_tool_call`
|
|
38
|
+
2. **Function Execution**: Original function runs normally
|
|
39
|
+
3. **Post-execution Logging**: Response sent to `/api/v2/log_tool_response`
|
|
40
|
+
|
|
41
|
+
**Implementation**: [`tool_mock.py:250-300`](tool_mock.py)
|
|
42
|
+
|
|
43
|
+
### Tracing Flow
|
|
44
|
+
1. **Dual Setup**: Braintrust + OpenTelemetry instrumentation
|
|
45
|
+
2. **Session Tagging**: Bearer tokens → session IDs
|
|
46
|
+
3. **Span Attribution**: All operations tagged with `veris.session_id`
|
|
47
|
+
|
|
48
|
+
**Implementation**: [`braintrust_tracing.py:50-150`](braintrust_tracing.py)
|
|
49
|
+
|
|
50
|
+
## Configuration
|
|
51
|
+
|
|
52
|
+
**Semantic Tag**: `module-config`
|
|
53
|
+
|
|
54
|
+
Environment variables are processed in [`tool_mock.py`](tool_mock.py):
|
|
55
|
+
|
|
56
|
+
- `VERIS_ENDPOINT_URL`: Mock server endpoint
|
|
57
|
+
- `VERIS_MOCK_TIMEOUT`: Request timeout (default: 90s)
|
|
58
|
+
- `ENV`: Set to `"simulation"` for mock mode
|
|
59
|
+
- `VERIS_SERVICE_NAME`: Tracing service identifier
|
|
60
|
+
- `VERIS_OTLP_ENDPOINT`: OpenTelemetry collector endpoint
|
|
61
|
+
|
|
62
|
+
## Development Notes
|
|
63
|
+
|
|
64
|
+
**Semantic Tag**: `development-patterns`
|
|
65
|
+
|
|
66
|
+
- **Lazy Imports**: [`__init__.py`](__init__.py) minimizes startup dependencies
|
|
67
|
+
- **Type Safety**: Extensive use of Pydantic models and type hints
|
|
68
|
+
- **Error Handling**: Comprehensive exception handling with timeouts
|
|
69
|
+
- **Testing**: Module-specific tests in [`../tests/`](../tests/)
|
|
70
|
+
|
|
71
|
+
**Architecture Principle**: Each module is self-contained with minimal cross-dependencies, enabling selective imports and reduced memory footprint.
|
|
72
|
+
|
|
73
|
+
---
|
|
74
|
+
|
|
75
|
+
**Parent Documentation**: See [main README](../../README.md) for installation and usage patterns.
|
|
@@ -1,137 +1,101 @@
|
|
|
1
1
|
# Jaeger Interface
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
[Jaeger Query Service](https://www.jaegertracing.io/docs/) HTTP API so
|
|
5
|
-
that you can **search for and retrieve traces** directly from Python
|
|
6
|
-
with minimal boilerplate. It also provides **client-side span filtering**
|
|
7
|
-
capabilities for more granular control over the returned data.
|
|
3
|
+
Typed Python wrapper for the Jaeger Query Service HTTP API with client-side span filtering capabilities.
|
|
8
4
|
|
|
9
|
-
|
|
10
|
-
> dependencies) and uses *pydantic* for full type-safety.
|
|
5
|
+
## Quick Reference
|
|
11
6
|
|
|
12
|
-
|
|
7
|
+
**Purpose**: Search and retrieve traces from Jaeger with minimal boilerplate
|
|
8
|
+
**Core Component**: [`JaegerClient`](client.py) class with `search()` and `get_trace()` methods
|
|
9
|
+
**Dependencies**: Uses `requests` and `pydantic` (included in base SDK)
|
|
10
|
+
**Compatibility**: Jaeger v1.x REST endpoints, OpenSearch storage backends
|
|
13
11
|
|
|
14
12
|
## Installation
|
|
15
13
|
|
|
16
|
-
|
|
17
|
-
requirements, so **no additional dependencies are required**.
|
|
14
|
+
**Semantic Tag**: `jaeger-setup`
|
|
18
15
|
|
|
16
|
+
No additional dependencies required - included with base `veris-ai` package:
|
|
19
17
|
```bash
|
|
20
18
|
pip install veris-ai
|
|
21
19
|
```
|
|
22
20
|
|
|
23
21
|
---
|
|
24
22
|
|
|
25
|
-
##
|
|
23
|
+
## Basic Usage
|
|
24
|
+
|
|
25
|
+
**Semantic Tag**: `jaeger-client-usage`
|
|
26
26
|
|
|
27
27
|
```python
|
|
28
28
|
from veris_ai.jaeger_interface import JaegerClient
|
|
29
|
-
|
|
30
|
-
from veris_ai.jaeger_interface.models import Trace
|
|
31
|
-
# Replace with the URL of your Jaeger Query Service instance
|
|
29
|
+
|
|
32
30
|
client = JaegerClient("http://localhost:16686")
|
|
33
31
|
|
|
34
|
-
#
|
|
35
|
-
|
|
32
|
+
# Search traces with filtering
|
|
33
|
+
traces = client.search(
|
|
36
34
|
service="veris-agent",
|
|
37
35
|
limit=10,
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
span_tags={"bt.metadata.model":"gpt-4.1-2025-04-14"}
|
|
36
|
+
tags={"veris.session_id": "session-123"},
|
|
37
|
+
span_tags={"http.status_code": 500}
|
|
41
38
|
)
|
|
42
39
|
|
|
43
|
-
#
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
# Guard clause
|
|
48
|
-
if not resp or not resp.data:
|
|
49
|
-
print("No data found")
|
|
50
|
-
exit(1)
|
|
51
|
-
|
|
52
|
-
# Print trace ids
|
|
53
|
-
for trace in resp.data:
|
|
54
|
-
if isinstance(trace, Trace):
|
|
55
|
-
print("TRACE ID:", trace.traceID, len(trace.spans), "spans")
|
|
56
|
-
|
|
57
|
-
# --- 2. Retrieve a specific trace -------------------------------------
|
|
58
|
-
if isinstance(resp.data, list):
|
|
59
|
-
trace_id = resp.data[0].traceID
|
|
60
|
-
else:
|
|
61
|
-
trace_id = resp.data.traceID
|
|
62
|
-
|
|
63
|
-
detailed = client.get_trace(trace_id)
|
|
64
|
-
# save detailed to json
|
|
65
|
-
with open("detailed.json", "w") as f:
|
|
66
|
-
f.write(detailed.model_dump_json(indent=2))
|
|
40
|
+
# Retrieve specific trace
|
|
41
|
+
if traces.data:
|
|
42
|
+
detailed = client.get_trace(traces.data[0].traceID)
|
|
67
43
|
```
|
|
68
44
|
|
|
45
|
+
**Data Models**: See [`models.py`](models.py) for `Trace`, `Span`, and response type definitions.
|
|
46
|
+
|
|
69
47
|
---
|
|
70
48
|
|
|
71
49
|
## API Reference
|
|
72
50
|
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
| Method | Description |
|
|
76
|
-
| -------- | ----------- |
|
|
77
|
-
| `search(service, *, limit=None, tags=None, operation=None, span_tags=None, **kwargs) -> SearchResponse` | Search for traces with optional span-level filtering. |
|
|
78
|
-
| `get_trace(trace_id: str) -> GetTraceResponse` | Fetch a single trace by ID (wrapper around `/api/traces/{id}`). |
|
|
79
|
-
|
|
80
|
-
### `search()` Parameters
|
|
81
|
-
|
|
82
|
-
The `search()` method now uses a flattened parameter structure:
|
|
51
|
+
**Semantic Tag**: `jaeger-api-methods`
|
|
83
52
|
|
|
84
|
-
|
|
85
|
-
| --------- | ---- | ----------- |
|
|
86
|
-
| `service` | `str` | Service name to search for. Optional - if not provided, searches across all services. |
|
|
87
|
-
| `limit` | `int` | Maximum number of traces to return. |
|
|
88
|
-
| `tags` | `Dict[str, Any]` | Trace-level tag filters (AND logic). A trace must have a span matching ALL tags. |
|
|
89
|
-
| `operation` | `str` | Filter by operation name. |
|
|
90
|
-
| `span_tags` | `Dict[str, Any]` | Span-level tag filters (OR logic). Returns only spans matching ANY of these tags. |
|
|
91
|
-
| `span_operations` | `List[str]` | Span-level operation name filters (OR logic). Returns only spans matching ANY of these operations. |
|
|
92
|
-
| `**kwargs` | `Any` | Additional parameters passed directly to Jaeger API. |
|
|
53
|
+
### Core Methods
|
|
93
54
|
|
|
94
|
-
|
|
55
|
+
| Method | Purpose | Returns |
|
|
56
|
+
|--------|---------|---------|
|
|
57
|
+
| `search(service, **filters)` | Search traces with optional filtering | `SearchResponse` |
|
|
58
|
+
| `get_trace(trace_id)` | Retrieve single trace by ID | `GetTraceResponse` |
|
|
95
59
|
|
|
96
|
-
|
|
60
|
+
**Implementation**: See [`client.py`](client.py) for method signatures and error handling.
|
|
97
61
|
|
|
98
|
-
|
|
99
|
-
- Sent directly to Jaeger API
|
|
100
|
-
- Uses AND logic: all tag key-value pairs must match on a single span
|
|
101
|
-
- Efficient server-side filtering
|
|
62
|
+
### Filtering Strategy
|
|
102
63
|
|
|
103
|
-
|
|
104
|
-
- Applied client-side after retrieving traces
|
|
105
|
-
- Uses OR logic: spans matching ANY of the provided tags are included
|
|
106
|
-
- Traces with no matching spans are excluded from results
|
|
107
|
-
- Useful for finding spans with specific characteristics across different traces
|
|
108
|
-
|
|
109
|
-
### Example: Combining Filters
|
|
64
|
+
**Semantic Tag**: `filtering-logic`
|
|
110
65
|
|
|
111
66
|
```python
|
|
112
|
-
#
|
|
113
|
-
|
|
67
|
+
# Server-side filtering (efficient)
|
|
68
|
+
traces = client.search(
|
|
69
|
+
service="my-service",
|
|
70
|
+
tags={"error": "true"}, # AND logic: trace must match ALL tags
|
|
71
|
+
operation="specific_op"
|
|
72
|
+
)
|
|
73
|
+
|
|
74
|
+
# Client-side filtering (granular)
|
|
114
75
|
traces = client.search(
|
|
115
|
-
service="my-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
"http.status_code": 500,
|
|
119
|
-
"http.status_code": 503,
|
|
120
|
-
"db.error": "connection_timeout"
|
|
121
|
-
} # Show only spans with these specific errors
|
|
76
|
+
service="my-service",
|
|
77
|
+
span_tags={"http.status_code": [500, 503]}, # OR logic: ANY span match
|
|
78
|
+
span_operations=["db_query", "api_call"]
|
|
122
79
|
)
|
|
123
80
|
```
|
|
124
81
|
|
|
82
|
+
**Filter Types**:
|
|
83
|
+
- **`tags`**: Trace-level filters (server-side, AND logic)
|
|
84
|
+
- **`span_tags`**: Span-level filters (client-side, OR logic)
|
|
85
|
+
- **`span_operations`**: Operation name filters (client-side, OR logic)
|
|
86
|
+
|
|
125
87
|
---
|
|
126
88
|
|
|
127
|
-
##
|
|
89
|
+
## Architecture
|
|
128
90
|
|
|
129
|
-
|
|
130
|
-
backed by **OpenSearch** storage the same endpoints apply. Should you
|
|
131
|
-
need API v3 support feel free to open an issue or contribution—thanks!
|
|
91
|
+
**Semantic Tag**: `jaeger-architecture`
|
|
132
92
|
|
|
133
|
-
|
|
93
|
+
- **Client Implementation**: [`client.py`](client.py) - HTTP requests to Jaeger API
|
|
94
|
+
- **Data Models**: [`models.py`](models.py) - Pydantic models for type safety
|
|
95
|
+
- **Compatibility**: Jaeger v1.x REST endpoints, OpenSearch backends
|
|
134
96
|
|
|
135
|
-
|
|
97
|
+
**Design Principle**: Thin wrapper maintaining Jaeger's native API structure while adding client-side span filtering capabilities.
|
|
98
|
+
|
|
99
|
+
---
|
|
136
100
|
|
|
137
|
-
|
|
101
|
+
**Parent Documentation**: See [module README](../README.md) for integration with other SDK components.
|
veris_ai/logging.py
ADDED
|
@@ -0,0 +1,116 @@
|
|
|
1
|
+
"""Logging utilities for VERIS tool calls and responses."""
|
|
2
|
+
|
|
3
|
+
import json
|
|
4
|
+
import logging
|
|
5
|
+
import os
|
|
6
|
+
from typing import Any
|
|
7
|
+
|
|
8
|
+
import httpx
|
|
9
|
+
|
|
10
|
+
logger = logging.getLogger(__name__)
|
|
11
|
+
|
|
12
|
+
|
|
13
|
+
async def log_tool_call_async(
|
|
14
|
+
session_id: str,
|
|
15
|
+
function_name: str,
|
|
16
|
+
parameters: dict[str, Any],
|
|
17
|
+
docstring: str,
|
|
18
|
+
) -> None:
|
|
19
|
+
"""Log tool call asynchronously to the VERIS logging endpoint."""
|
|
20
|
+
base_url = os.getenv("VERIS_ENDPOINT_URL")
|
|
21
|
+
if not base_url:
|
|
22
|
+
logger.warning("VERIS_ENDPOINT_URL not set, skipping tool call logging")
|
|
23
|
+
return
|
|
24
|
+
|
|
25
|
+
endpoint = f"{base_url}/api/v2/simulations/{session_id}/log_tool_call"
|
|
26
|
+
payload = {
|
|
27
|
+
"function_name": function_name,
|
|
28
|
+
"parameters": parameters,
|
|
29
|
+
"docstring": docstring,
|
|
30
|
+
}
|
|
31
|
+
|
|
32
|
+
timeout = float(os.getenv("VERIS_MOCK_TIMEOUT", "90.0"))
|
|
33
|
+
|
|
34
|
+
try:
|
|
35
|
+
async with httpx.AsyncClient(timeout=timeout) as client:
|
|
36
|
+
response = await client.post(endpoint, json=payload)
|
|
37
|
+
response.raise_for_status()
|
|
38
|
+
logger.debug(f"Tool call logged for {function_name}")
|
|
39
|
+
except Exception as e:
|
|
40
|
+
logger.warning(f"Failed to log tool call for {function_name}: {e}")
|
|
41
|
+
|
|
42
|
+
|
|
43
|
+
def log_tool_call_sync(
|
|
44
|
+
session_id: str,
|
|
45
|
+
function_name: str,
|
|
46
|
+
parameters: dict[str, Any],
|
|
47
|
+
docstring: str,
|
|
48
|
+
) -> None:
|
|
49
|
+
"""Log tool call synchronously to the VERIS logging endpoint."""
|
|
50
|
+
base_url = os.getenv("VERIS_ENDPOINT_URL")
|
|
51
|
+
if not base_url:
|
|
52
|
+
logger.warning("VERIS_ENDPOINT_URL not set, skipping tool call logging")
|
|
53
|
+
return
|
|
54
|
+
|
|
55
|
+
endpoint = f"{base_url}/api/v2/simulations/{session_id}/log_tool_call"
|
|
56
|
+
payload = {
|
|
57
|
+
"function_name": function_name,
|
|
58
|
+
"parameters": parameters,
|
|
59
|
+
"docstring": docstring,
|
|
60
|
+
}
|
|
61
|
+
|
|
62
|
+
timeout = float(os.getenv("VERIS_MOCK_TIMEOUT", "90.0"))
|
|
63
|
+
|
|
64
|
+
try:
|
|
65
|
+
with httpx.Client(timeout=timeout) as client:
|
|
66
|
+
response = client.post(endpoint, json=payload)
|
|
67
|
+
response.raise_for_status()
|
|
68
|
+
logger.debug(f"Tool call logged for {function_name}")
|
|
69
|
+
except Exception as e:
|
|
70
|
+
logger.warning(f"Failed to log tool call for {function_name}: {e}")
|
|
71
|
+
|
|
72
|
+
|
|
73
|
+
async def log_tool_response_async(session_id: str, response: object) -> None:
|
|
74
|
+
"""Log tool response asynchronously to the VERIS logging endpoint."""
|
|
75
|
+
base_url = os.getenv("VERIS_ENDPOINT_URL")
|
|
76
|
+
if not base_url:
|
|
77
|
+
logger.warning("VERIS_ENDPOINT_URL not set, skipping tool response logging")
|
|
78
|
+
return
|
|
79
|
+
|
|
80
|
+
endpoint = f"{base_url}/api/v2/simulations/{session_id}/log_tool_response"
|
|
81
|
+
payload = {
|
|
82
|
+
"response": json.dumps(response, default=str),
|
|
83
|
+
}
|
|
84
|
+
|
|
85
|
+
timeout = float(os.getenv("VERIS_MOCK_TIMEOUT", "90.0"))
|
|
86
|
+
|
|
87
|
+
try:
|
|
88
|
+
async with httpx.AsyncClient(timeout=timeout) as client:
|
|
89
|
+
http_response = await client.post(endpoint, json=payload)
|
|
90
|
+
http_response.raise_for_status()
|
|
91
|
+
logger.debug("Tool response logged")
|
|
92
|
+
except Exception as e:
|
|
93
|
+
logger.warning(f"Failed to log tool response: {e}")
|
|
94
|
+
|
|
95
|
+
|
|
96
|
+
def log_tool_response_sync(session_id: str, response: object) -> None:
|
|
97
|
+
"""Log tool response synchronously to the VERIS logging endpoint."""
|
|
98
|
+
base_url = os.getenv("VERIS_ENDPOINT_URL")
|
|
99
|
+
if not base_url:
|
|
100
|
+
logger.warning("VERIS_ENDPOINT_URL not set, skipping tool response logging")
|
|
101
|
+
return
|
|
102
|
+
|
|
103
|
+
endpoint = f"{base_url}/api/v2/simulations/{session_id}/log_tool_response"
|
|
104
|
+
payload = {
|
|
105
|
+
"response": json.dumps(response, default=str),
|
|
106
|
+
}
|
|
107
|
+
|
|
108
|
+
timeout = float(os.getenv("VERIS_MOCK_TIMEOUT", "90.0"))
|
|
109
|
+
|
|
110
|
+
try:
|
|
111
|
+
with httpx.Client(timeout=timeout) as client:
|
|
112
|
+
http_response = client.post(endpoint, json=payload)
|
|
113
|
+
http_response.raise_for_status()
|
|
114
|
+
logger.debug("Tool response logged")
|
|
115
|
+
except Exception as e:
|
|
116
|
+
logger.warning(f"Failed to log tool response: {e}")
|
veris_ai/tool_mock.py
CHANGED
|
@@ -15,6 +15,12 @@ from typing import (
|
|
|
15
15
|
|
|
16
16
|
import httpx
|
|
17
17
|
|
|
18
|
+
from veris_ai.logging import (
|
|
19
|
+
log_tool_call_async,
|
|
20
|
+
log_tool_call_sync,
|
|
21
|
+
log_tool_response_async,
|
|
22
|
+
log_tool_response_sync,
|
|
23
|
+
)
|
|
18
24
|
from veris_ai.models import ResponseExpectation
|
|
19
25
|
from veris_ai.utils import convert_to_type, extract_json_schema
|
|
20
26
|
|
|
@@ -94,7 +100,7 @@ class VerisSDK:
|
|
|
94
100
|
|
|
95
101
|
def mock( # noqa: C901, PLR0915
|
|
96
102
|
self,
|
|
97
|
-
mode: Literal["tool", "function"] = "tool",
|
|
103
|
+
mode: Literal["tool", "function", "spy"] = "tool",
|
|
98
104
|
expects_response: bool | None = None,
|
|
99
105
|
cache_response: bool | None = None,
|
|
100
106
|
) -> Callable:
|
|
@@ -165,10 +171,40 @@ class VerisSDK:
|
|
|
165
171
|
if not self.session_id:
|
|
166
172
|
# If not in simulation mode, execute the original function
|
|
167
173
|
return await func(*args, **kwargs)
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
|
|
174
|
+
|
|
175
|
+
# Handle spy mode - execute original function and log
|
|
176
|
+
if mode == "spy":
|
|
177
|
+
logger.info(f"Spying on function: {func.__name__}")
|
|
178
|
+
|
|
179
|
+
# Log the tool call
|
|
180
|
+
sig = inspect.signature(func)
|
|
181
|
+
bound_args = sig.bind(*args, **kwargs)
|
|
182
|
+
bound_args.apply_defaults()
|
|
183
|
+
_ = bound_args.arguments.pop("ctx", None)
|
|
184
|
+
_ = bound_args.arguments.pop("self", None)
|
|
185
|
+
_ = bound_args.arguments.pop("cls", None)
|
|
186
|
+
|
|
187
|
+
await log_tool_call_async(
|
|
188
|
+
session_id=self.session_id,
|
|
189
|
+
function_name=func.__name__,
|
|
190
|
+
parameters=bound_args.arguments,
|
|
191
|
+
docstring=inspect.getdoc(func) or "",
|
|
192
|
+
)
|
|
193
|
+
|
|
194
|
+
# Execute the original function
|
|
195
|
+
result = await func(*args, **kwargs)
|
|
196
|
+
|
|
197
|
+
# Log the response
|
|
198
|
+
await log_tool_response_async(session_id=self.session_id, response=result)
|
|
199
|
+
|
|
200
|
+
return result
|
|
201
|
+
|
|
202
|
+
# Regular mock mode
|
|
203
|
+
base_url = os.getenv("VERIS_ENDPOINT_URL")
|
|
204
|
+
if not base_url:
|
|
205
|
+
error_msg = "VERIS_ENDPOINT_URL environment variable is not set"
|
|
171
206
|
raise ValueError(error_msg)
|
|
207
|
+
endpoint = f"{base_url.rstrip('/')}/api/v2/tool_mock"
|
|
172
208
|
# Default timeout of 30 seconds
|
|
173
209
|
timeout = float(os.getenv("VERIS_MOCK_TIMEOUT", "90.0"))
|
|
174
210
|
|
|
@@ -197,10 +233,40 @@ class VerisSDK:
|
|
|
197
233
|
if not self.session_id:
|
|
198
234
|
# If not in simulation mode, execute the original function
|
|
199
235
|
return func(*args, **kwargs)
|
|
200
|
-
|
|
201
|
-
|
|
202
|
-
|
|
236
|
+
|
|
237
|
+
# Handle spy mode - execute original function and log
|
|
238
|
+
if mode == "spy":
|
|
239
|
+
logger.info(f"Spying on function: {func.__name__}")
|
|
240
|
+
|
|
241
|
+
# Log the tool call
|
|
242
|
+
sig = inspect.signature(func)
|
|
243
|
+
bound_args = sig.bind(*args, **kwargs)
|
|
244
|
+
bound_args.apply_defaults()
|
|
245
|
+
_ = bound_args.arguments.pop("ctx", None)
|
|
246
|
+
_ = bound_args.arguments.pop("self", None)
|
|
247
|
+
_ = bound_args.arguments.pop("cls", None)
|
|
248
|
+
|
|
249
|
+
log_tool_call_sync(
|
|
250
|
+
session_id=self.session_id,
|
|
251
|
+
function_name=func.__name__,
|
|
252
|
+
parameters=bound_args.arguments,
|
|
253
|
+
docstring=inspect.getdoc(func) or "",
|
|
254
|
+
)
|
|
255
|
+
|
|
256
|
+
# Execute the original function
|
|
257
|
+
result = func(*args, **kwargs)
|
|
258
|
+
|
|
259
|
+
# Log the response
|
|
260
|
+
log_tool_response_sync(session_id=self.session_id, response=result)
|
|
261
|
+
|
|
262
|
+
return result
|
|
263
|
+
|
|
264
|
+
# Regular mock mode
|
|
265
|
+
base_url = os.getenv("VERIS_ENDPOINT_URL")
|
|
266
|
+
if not base_url:
|
|
267
|
+
error_msg = "VERIS_ENDPOINT_URL environment variable is not set"
|
|
203
268
|
raise ValueError(error_msg)
|
|
269
|
+
endpoint = f"{base_url.rstrip('/')}/api/v2/tool_mock"
|
|
204
270
|
# Default timeout of 30 seconds
|
|
205
271
|
timeout = float(os.getenv("VERIS_MOCK_TIMEOUT", "90.0"))
|
|
206
272
|
|
|
@@ -240,7 +306,7 @@ class VerisSDK:
|
|
|
240
306
|
if not self.session_id:
|
|
241
307
|
# If not in simulation mode, execute the original function
|
|
242
308
|
return await func(*args, **kwargs)
|
|
243
|
-
logger.info(f"
|
|
309
|
+
logger.info(f"Stubbing function: {func.__name__}")
|
|
244
310
|
return return_value
|
|
245
311
|
|
|
246
312
|
@wraps(func)
|
|
@@ -248,7 +314,7 @@ class VerisSDK:
|
|
|
248
314
|
if not self.session_id:
|
|
249
315
|
# If not in simulation mode, execute the original function
|
|
250
316
|
return func(*args, **kwargs)
|
|
251
|
-
logger.info(f"
|
|
317
|
+
logger.info(f"Stubbing function: {func.__name__}")
|
|
252
318
|
return return_value
|
|
253
319
|
|
|
254
320
|
# Return the appropriate wrapper based on whether the function is async
|
veris_ai/utils.py
CHANGED
|
@@ -0,0 +1,223 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: veris-ai
|
|
3
|
+
Version: 1.4.0
|
|
4
|
+
Summary: A Python package for Veris AI tools
|
|
5
|
+
Project-URL: Homepage, https://github.com/veris-ai/veris-python-sdk
|
|
6
|
+
Project-URL: Bug Tracker, https://github.com/veris-ai/veris-python-sdk/issues
|
|
7
|
+
Author-email: Mehdi Jamei <mehdi@veris.ai>
|
|
8
|
+
License-Expression: MIT
|
|
9
|
+
License-File: LICENSE
|
|
10
|
+
Requires-Python: >=3.11
|
|
11
|
+
Requires-Dist: httpx>=0.24.0
|
|
12
|
+
Requires-Dist: pydantic>=2.0.0
|
|
13
|
+
Requires-Dist: requests>=2.31.0
|
|
14
|
+
Provides-Extra: dev
|
|
15
|
+
Requires-Dist: black>=23.7.0; extra == 'dev'
|
|
16
|
+
Requires-Dist: mypy>=1.5.1; extra == 'dev'
|
|
17
|
+
Requires-Dist: openai-agents>=0.0.1; extra == 'dev'
|
|
18
|
+
Requires-Dist: pre-commit>=3.3.3; extra == 'dev'
|
|
19
|
+
Requires-Dist: pytest-asyncio>=0.21.1; extra == 'dev'
|
|
20
|
+
Requires-Dist: pytest-cov>=4.1.0; extra == 'dev'
|
|
21
|
+
Requires-Dist: pytest>=7.4.0; extra == 'dev'
|
|
22
|
+
Requires-Dist: ruff>=0.11.4; extra == 'dev'
|
|
23
|
+
Provides-Extra: fastapi
|
|
24
|
+
Requires-Dist: fastapi; extra == 'fastapi'
|
|
25
|
+
Requires-Dist: fastapi-mcp; extra == 'fastapi'
|
|
26
|
+
Provides-Extra: instrument
|
|
27
|
+
Requires-Dist: braintrust; extra == 'instrument'
|
|
28
|
+
Requires-Dist: opentelemetry-api; extra == 'instrument'
|
|
29
|
+
Requires-Dist: opentelemetry-exporter-otlp; extra == 'instrument'
|
|
30
|
+
Requires-Dist: opentelemetry-exporter-otlp-proto-common; extra == 'instrument'
|
|
31
|
+
Requires-Dist: opentelemetry-exporter-otlp-proto-grpc; extra == 'instrument'
|
|
32
|
+
Requires-Dist: opentelemetry-exporter-otlp-proto-http; extra == 'instrument'
|
|
33
|
+
Requires-Dist: opentelemetry-sdk; extra == 'instrument'
|
|
34
|
+
Requires-Dist: wrapt; extra == 'instrument'
|
|
35
|
+
Description-Content-Type: text/markdown
|
|
36
|
+
|
|
37
|
+
# Veris AI Python SDK
|
|
38
|
+
|
|
39
|
+
A Python package for Veris AI tools with simulation capabilities and FastAPI MCP (Model Context Protocol) integration.
|
|
40
|
+
|
|
41
|
+
## Quick Reference
|
|
42
|
+
|
|
43
|
+
**Purpose**: Tool mocking, tracing, and FastAPI MCP integration for AI agent development
|
|
44
|
+
**Core Components**: [`tool_mock`](#function-mocking) • [`jaeger_interface`](#jaeger-trace-interface) • [`braintrust_tracing`](#tracing-integration) • [`fastapi_mcp`](#fastapi-mcp-integration)
|
|
45
|
+
**Deep Dive**: [`Module Architecture`](src/veris_ai/README.md) • [`Testing Guide`](tests/README.md) • [`Usage Examples`](examples/README.md)
|
|
46
|
+
**Source of Truth**: Implementation details in [`src/veris_ai/`](src/veris_ai/) source code
|
|
47
|
+
|
|
48
|
+
## Installation
|
|
49
|
+
|
|
50
|
+
```bash
|
|
51
|
+
# Base package
|
|
52
|
+
uv add veris-ai
|
|
53
|
+
|
|
54
|
+
# With optional extras
|
|
55
|
+
uv add "veris-ai[dev,fastapi,instrument]"
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
**Installation Profiles**:
|
|
59
|
+
- `dev`: Development tools (ruff, pytest, mypy)
|
|
60
|
+
- `fastapi`: FastAPI MCP integration
|
|
61
|
+
- `instrument`: Braintrust/OpenTelemetry tracing
|
|
62
|
+
|
|
63
|
+
## Import Patterns
|
|
64
|
+
|
|
65
|
+
**Semantic Tag**: `import-patterns`
|
|
66
|
+
|
|
67
|
+
```python
|
|
68
|
+
# Core imports (base dependencies only)
|
|
69
|
+
from veris_ai import veris, JaegerClient
|
|
70
|
+
|
|
71
|
+
# Optional features (require extras)
|
|
72
|
+
from veris_ai import braintrust_tracing # Requires [instrument]
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
**Complete Import Strategies**: See [`examples/README.md`](examples/README.md) for five different import approaches, conditional features, and integration patterns.
|
|
76
|
+
|
|
77
|
+
## Configuration
|
|
78
|
+
|
|
79
|
+
**Semantic Tag**: `environment-config`
|
|
80
|
+
|
|
81
|
+
| Variable | Purpose | Default |
|
|
82
|
+
|----------|---------|---------|
|
|
83
|
+
| `VERIS_ENDPOINT_URL` | Mock server endpoint | *Required* |
|
|
84
|
+
| `VERIS_MOCK_TIMEOUT` | Request timeout (seconds) | `90.0` |
|
|
85
|
+
| `ENV` | Set to `"simulation"` for mock mode | Production |
|
|
86
|
+
| `VERIS_SERVICE_NAME` | Tracing service name | Auto-detected |
|
|
87
|
+
| `VERIS_OTLP_ENDPOINT` | OpenTelemetry collector | *Required for tracing* |
|
|
88
|
+
|
|
89
|
+
**Configuration Details**: See [`src/veris_ai/tool_mock.py`](src/veris_ai/tool_mock.py) for environment handling logic.
|
|
90
|
+
|
|
91
|
+
## Tracing Integration
|
|
92
|
+
|
|
93
|
+
**Semantic Tag**: `distributed-tracing`
|
|
94
|
+
|
|
95
|
+
Parallel tracing to Braintrust and Jaeger/OpenTelemetry for monitoring and evaluation.
|
|
96
|
+
|
|
97
|
+
```python
|
|
98
|
+
from veris_ai import braintrust_tracing
|
|
99
|
+
|
|
100
|
+
# Enable dual tracing
|
|
101
|
+
braintrust_tracing.instrument(service_name="my-service", otlp_endpoint="http://localhost:4317")
|
|
102
|
+
```
|
|
103
|
+
|
|
104
|
+
**Session Management**: Automatic session ID extraction from bearer tokens. Manual session control via `veris.set_session_id()` and `veris.clear_session_id()`.
|
|
105
|
+
|
|
106
|
+
**Implementation Details**: See [`src/veris_ai/braintrust_tracing.py`](src/veris_ai/braintrust_tracing.py) for instrumentation logic.
|
|
107
|
+
|
|
108
|
+
## Function Mocking
|
|
109
|
+
|
|
110
|
+
**Semantic Tag**: `tool-mocking`
|
|
111
|
+
|
|
112
|
+
### Core Decorators
|
|
113
|
+
|
|
114
|
+
```python
|
|
115
|
+
from veris_ai import veris
|
|
116
|
+
|
|
117
|
+
# Mock mode: Returns simulated responses in ENV=simulation
|
|
118
|
+
@veris.mock()
|
|
119
|
+
async def your_function(param1: str, param2: int) -> dict:
|
|
120
|
+
"""Function documentation for LLM context."""
|
|
121
|
+
return {"result": "actual implementation"}
|
|
122
|
+
|
|
123
|
+
# Spy mode: Executes function but logs calls/responses
|
|
124
|
+
@veris.mock(mode="spy")
|
|
125
|
+
async def monitored_function(data: str) -> dict:
|
|
126
|
+
return process_data(data)
|
|
127
|
+
|
|
128
|
+
# Stub mode: Returns fixed value in simulation
|
|
129
|
+
@veris.stub(return_value={"status": "success"})
|
|
130
|
+
async def get_data() -> dict:
|
|
131
|
+
return await fetch_from_api()
|
|
132
|
+
```
|
|
133
|
+
|
|
134
|
+
**Behavior**: In simulation mode, decorators intercept calls to mock endpoints. In production, functions execute normally.
|
|
135
|
+
|
|
136
|
+
**Implementation**: See [`src/veris_ai/tool_mock.py`](src/veris_ai/tool_mock.py) for decorator logic and API integration.
|
|
137
|
+
|
|
138
|
+
## FastAPI MCP Integration
|
|
139
|
+
|
|
140
|
+
**Semantic Tag**: `fastapi-mcp`
|
|
141
|
+
|
|
142
|
+
Expose FastAPI endpoints as MCP tools for AI agent consumption.
|
|
143
|
+
|
|
144
|
+
```python
|
|
145
|
+
from fastapi import FastAPI
|
|
146
|
+
from veris_ai import veris
|
|
147
|
+
|
|
148
|
+
app = FastAPI()
|
|
149
|
+
|
|
150
|
+
# Enable MCP integration
|
|
151
|
+
veris.set_fastapi_mcp(
|
|
152
|
+
fastapi=app,
|
|
153
|
+
name="My API Server",
|
|
154
|
+
include_operations=["get_users", "create_user"],
|
|
155
|
+
exclude_tags=["internal"]
|
|
156
|
+
)
|
|
157
|
+
```
|
|
158
|
+
|
|
159
|
+
**Key Features**:
|
|
160
|
+
- **Automatic schema conversion**: FastAPI OpenAPI → MCP tool definitions
|
|
161
|
+
- **Session management**: Bearer token → session ID mapping
|
|
162
|
+
- **Filtering**: Include/exclude operations and tags
|
|
163
|
+
- **Authentication**: OAuth2 integration
|
|
164
|
+
|
|
165
|
+
**Configuration Reference**: See function signature in [`src/veris_ai/tool_mock.py`](src/veris_ai/tool_mock.py) for all `set_fastapi_mcp()` parameters.
|
|
166
|
+
|
|
167
|
+
## Utility Functions
|
|
168
|
+
|
|
169
|
+
**Semantic Tag**: `json-schema-utils`
|
|
170
|
+
|
|
171
|
+
```python
|
|
172
|
+
from veris_ai.utils import extract_json_schema
|
|
173
|
+
|
|
174
|
+
# Schema extraction from types
|
|
175
|
+
user_schema = extract_json_schema(User) # Pydantic models
|
|
176
|
+
list_schema = extract_json_schema(List[str]) # Generics
|
|
177
|
+
```
|
|
178
|
+
|
|
179
|
+
**Supported Types**: Built-in types, generics (List, Dict, Union), Pydantic models, TypedDict, forward references.
|
|
180
|
+
|
|
181
|
+
**Implementation**: See [`src/veris_ai/utils.py`](src/veris_ai/utils.py) for type conversion logic.
|
|
182
|
+
|
|
183
|
+
## Development
|
|
184
|
+
|
|
185
|
+
**Semantic Tag**: `development-setup`
|
|
186
|
+
|
|
187
|
+
**Requirements**: Python 3.11+, `uv` package manager
|
|
188
|
+
|
|
189
|
+
```bash
|
|
190
|
+
# Install with dev dependencies
|
|
191
|
+
uv add "veris-ai[dev]"
|
|
192
|
+
|
|
193
|
+
# Quality checks
|
|
194
|
+
ruff check --fix . # Lint and format
|
|
195
|
+
pytest --cov=veris_ai # Test with coverage
|
|
196
|
+
```
|
|
197
|
+
|
|
198
|
+
**Testing & Architecture**: See [`tests/README.md`](tests/README.md) for test structure, fixtures, and coverage strategies. See [`src/veris_ai/README.md`](src/veris_ai/README.md) for module architecture and implementation flows.
|
|
199
|
+
|
|
200
|
+
## Module Architecture
|
|
201
|
+
|
|
202
|
+
**Semantic Tag**: `module-architecture`
|
|
203
|
+
|
|
204
|
+
**Core Modules**: `tool_mock` (mocking), `jaeger_interface` (trace queries), `braintrust_tracing` (dual tracing), `utils` (schema conversion)
|
|
205
|
+
|
|
206
|
+
**Complete Architecture**: See [`src/veris_ai/README.md`](src/veris_ai/README.md) for module overview, implementation flows, and configuration details.
|
|
207
|
+
|
|
208
|
+
## Jaeger Trace Interface
|
|
209
|
+
|
|
210
|
+
**Semantic Tag**: `jaeger-query-api`
|
|
211
|
+
|
|
212
|
+
```python
|
|
213
|
+
from veris_ai.jaeger_interface import JaegerClient
|
|
214
|
+
|
|
215
|
+
client = JaegerClient("http://localhost:16686")
|
|
216
|
+
traces = client.search(service="veris-agent", tags={"error": "true"})
|
|
217
|
+
```
|
|
218
|
+
|
|
219
|
+
**Complete Guide**: See [`src/veris_ai/jaeger_interface/README.md`](src/veris_ai/jaeger_interface/README.md) for API reference, filtering strategies, and architecture details.
|
|
220
|
+
|
|
221
|
+
---
|
|
222
|
+
|
|
223
|
+
**License**: MIT License - see [LICENSE](LICENSE) file for details.
|
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
veris_ai/README.md,sha256=q0Qn6gu85HWU189SmyOygY_mkNj0l10wlAVy9kj4udo,3120
|
|
2
|
+
veris_ai/__init__.py,sha256=Vp5yf9ZRchw0mmPwrRuNebI5cb2NGwpJGfr41l86q1U,1177
|
|
3
|
+
veris_ai/braintrust_tracing.py,sha256=0i-HR6IuK3-Q5ujMjT1FojxESRZLgqEvJ44bfJfDHaw,11938
|
|
4
|
+
veris_ai/logging.py,sha256=2855E8s_k6yTnzFK10EQR4WQngQk3VuWFPJW-MYiw3k,3837
|
|
5
|
+
veris_ai/models.py,sha256=6HINPxNFCakCVPcyEbUswWkXwb2K4lF0A8g8EvTMal4,213
|
|
6
|
+
veris_ai/tool_mock.py,sha256=9rb4c8OZEg6hGmyJlmgFSmbBDa7L6_XzadvtVrGy9xs,13198
|
|
7
|
+
veris_ai/utils.py,sha256=aqFFNuNiBehil6874nOHtU7G_bWHbFpVuubcz2AIx6I,9267
|
|
8
|
+
veris_ai/jaeger_interface/README.md,sha256=kd9rKcE5xf3EyNaiHu0tjn-0oES9sfaK6Ih-OhhTyCM,2821
|
|
9
|
+
veris_ai/jaeger_interface/__init__.py,sha256=d873a0zq3eUYU2Y77MtdjCwIARjAsAP7WDqGXDMWpYs,1158
|
|
10
|
+
veris_ai/jaeger_interface/client.py,sha256=yJrh86wRR0Dk3Gq12DId99WogcMIVbL0QQFqVSevvlE,8772
|
|
11
|
+
veris_ai/jaeger_interface/models.py,sha256=e64VV6IvOEFuzRUgvDAMQFyOZMRb56I-PUPZLBZ3rX0,1864
|
|
12
|
+
veris_ai-1.4.0.dist-info/METADATA,sha256=yq1IbvlQ7qME6BdFz3tQ9-b0x9Oblcu2OJNUGS3iNVE,7702
|
|
13
|
+
veris_ai-1.4.0.dist-info/WHEEL,sha256=qtCwoSJWgHk21S1Kb4ihdzI2rlJ1ZKaIurTj_ngOhyQ,87
|
|
14
|
+
veris_ai-1.4.0.dist-info/licenses/LICENSE,sha256=2g4i20atAgtD5einaKzhQrIB-JrPhyQgD3bC0wkHcCI,1065
|
|
15
|
+
veris_ai-1.4.0.dist-info/RECORD,,
|
|
@@ -1,457 +0,0 @@
|
|
|
1
|
-
Metadata-Version: 2.4
|
|
2
|
-
Name: veris-ai
|
|
3
|
-
Version: 1.3.0
|
|
4
|
-
Summary: A Python package for Veris AI tools
|
|
5
|
-
Project-URL: Homepage, https://github.com/veris-ai/veris-python-sdk
|
|
6
|
-
Project-URL: Bug Tracker, https://github.com/veris-ai/veris-python-sdk/issues
|
|
7
|
-
Author-email: Mehdi Jamei <mehdi@veris.ai>
|
|
8
|
-
License-Expression: MIT
|
|
9
|
-
License-File: LICENSE
|
|
10
|
-
Requires-Python: >=3.11
|
|
11
|
-
Requires-Dist: httpx>=0.24.0
|
|
12
|
-
Requires-Dist: pydantic>=2.0.0
|
|
13
|
-
Requires-Dist: requests>=2.31.0
|
|
14
|
-
Provides-Extra: dev
|
|
15
|
-
Requires-Dist: black>=23.7.0; extra == 'dev'
|
|
16
|
-
Requires-Dist: mypy>=1.5.1; extra == 'dev'
|
|
17
|
-
Requires-Dist: openai-agents>=0.0.1; extra == 'dev'
|
|
18
|
-
Requires-Dist: pre-commit>=3.3.3; extra == 'dev'
|
|
19
|
-
Requires-Dist: pytest-asyncio>=0.21.1; extra == 'dev'
|
|
20
|
-
Requires-Dist: pytest-cov>=4.1.0; extra == 'dev'
|
|
21
|
-
Requires-Dist: pytest>=7.4.0; extra == 'dev'
|
|
22
|
-
Requires-Dist: ruff>=0.11.4; extra == 'dev'
|
|
23
|
-
Provides-Extra: fastapi
|
|
24
|
-
Requires-Dist: fastapi; extra == 'fastapi'
|
|
25
|
-
Requires-Dist: fastapi-mcp; extra == 'fastapi'
|
|
26
|
-
Provides-Extra: instrument
|
|
27
|
-
Requires-Dist: braintrust; extra == 'instrument'
|
|
28
|
-
Requires-Dist: opentelemetry-api; extra == 'instrument'
|
|
29
|
-
Requires-Dist: opentelemetry-exporter-otlp; extra == 'instrument'
|
|
30
|
-
Requires-Dist: opentelemetry-exporter-otlp-proto-common; extra == 'instrument'
|
|
31
|
-
Requires-Dist: opentelemetry-exporter-otlp-proto-grpc; extra == 'instrument'
|
|
32
|
-
Requires-Dist: opentelemetry-exporter-otlp-proto-http; extra == 'instrument'
|
|
33
|
-
Requires-Dist: opentelemetry-sdk; extra == 'instrument'
|
|
34
|
-
Requires-Dist: wrapt; extra == 'instrument'
|
|
35
|
-
Description-Content-Type: text/markdown
|
|
36
|
-
|
|
37
|
-
# Veris AI Python SDK
|
|
38
|
-
|
|
39
|
-
A Python package for Veris AI tools with simulation capabilities and FastAPI MCP (Model Context Protocol) integration.
|
|
40
|
-
|
|
41
|
-
## Installation
|
|
42
|
-
|
|
43
|
-
You can install the package using `uv`:
|
|
44
|
-
|
|
45
|
-
```bash
|
|
46
|
-
# Install the package
|
|
47
|
-
uv add veris-ai
|
|
48
|
-
|
|
49
|
-
# Install with development dependencies
|
|
50
|
-
uv add "veris-ai[dev]"
|
|
51
|
-
|
|
52
|
-
# Install with FastAPI MCP integration
|
|
53
|
-
uv add "veris-ai[fastapi]"
|
|
54
|
-
|
|
55
|
-
# Install with tracing/instrumentation support
|
|
56
|
-
uv add "veris-ai[instrument]"
|
|
57
|
-
```
|
|
58
|
-
|
|
59
|
-
## Import Options
|
|
60
|
-
|
|
61
|
-
The SDK supports flexible import patterns to minimize dependencies:
|
|
62
|
-
|
|
63
|
-
### Default Imports (Base Dependencies Only)
|
|
64
|
-
|
|
65
|
-
```python
|
|
66
|
-
# These imports only require base dependencies (httpx, pydantic, requests)
|
|
67
|
-
from veris_ai import veris, JaegerClient
|
|
68
|
-
```
|
|
69
|
-
|
|
70
|
-
### Optional Imports (Require Extra Dependencies)
|
|
71
|
-
|
|
72
|
-
```python
|
|
73
|
-
# The instrument function requires the 'instrument' extra
|
|
74
|
-
# Install with: pip install veris-ai[instrument] or uv add "veris-ai[instrument]"
|
|
75
|
-
from veris_ai import instrument # Lazy-loaded, will error if deps missing
|
|
76
|
-
```
|
|
77
|
-
|
|
78
|
-
### Direct Submodule Imports
|
|
79
|
-
|
|
80
|
-
For maximum control over dependencies, you can import directly from submodules:
|
|
81
|
-
|
|
82
|
-
```python
|
|
83
|
-
# Import only what you need
|
|
84
|
-
from veris_ai.tool_mock import veris
|
|
85
|
-
from veris_ai.jaeger_interface import JaegerClient
|
|
86
|
-
from veris_ai.braintrust_tracing import instrument # Requires [instrument] extra
|
|
87
|
-
```
|
|
88
|
-
|
|
89
|
-
## Environment Setup
|
|
90
|
-
|
|
91
|
-
The package requires the following environment variables:
|
|
92
|
-
|
|
93
|
-
```bash
|
|
94
|
-
# Required: URL for the mock endpoint
|
|
95
|
-
VERIS_MOCK_ENDPOINT_URL=http://your-mock-endpoint.com
|
|
96
|
-
|
|
97
|
-
# Optional: Timeout in seconds (default: 30.0)
|
|
98
|
-
VERIS_MOCK_TIMEOUT=30.0
|
|
99
|
-
|
|
100
|
-
# Optional: Set to "simulation" to enable mock mode
|
|
101
|
-
ENV=simulation
|
|
102
|
-
```
|
|
103
|
-
|
|
104
|
-
## Tracing Integration
|
|
105
|
-
|
|
106
|
-
The SDK provides seamless integration with Braintrust and Jaeger/OpenTelemetry for distributed tracing.
|
|
107
|
-
|
|
108
|
-
### Setting up Tracing
|
|
109
|
-
|
|
110
|
-
```python
|
|
111
|
-
from veris_ai import instrument
|
|
112
|
-
|
|
113
|
-
# Initialize tracing instrumentation
|
|
114
|
-
instrument(
|
|
115
|
-
service_name="my-service", # or via VERIS_SERVICE_NAME env var
|
|
116
|
-
otlp_endpoint="http://localhost:4317" # or via VERIS_OTLP_ENDPOINT env var
|
|
117
|
-
)
|
|
118
|
-
```
|
|
119
|
-
|
|
120
|
-
### Session ID Tracking
|
|
121
|
-
|
|
122
|
-
When using the SDK with FastAPI MCP integration, session IDs are automatically embedded in all traces:
|
|
123
|
-
|
|
124
|
-
1. **Automatic Session Extraction**: When a request includes an Authorization header with a bearer token, the token is automatically used as the session ID.
|
|
125
|
-
|
|
126
|
-
2. **Trace Attribution**: All spans created during a request will include the `veris.session_id` attribute, allowing you to:
|
|
127
|
-
- Filter traces by session in Jaeger
|
|
128
|
-
- Correlate all operations within a single user session
|
|
129
|
-
- Debug issues specific to a particular session
|
|
130
|
-
|
|
131
|
-
3. **Example**:
|
|
132
|
-
```python
|
|
133
|
-
# FastAPI app with MCP integration
|
|
134
|
-
from fastapi import FastAPI
|
|
135
|
-
from veris_ai import veris, instrument
|
|
136
|
-
|
|
137
|
-
app = FastAPI()
|
|
138
|
-
|
|
139
|
-
# Set up tracing
|
|
140
|
-
instrument()
|
|
141
|
-
|
|
142
|
-
# Set up FastAPI MCP with session handling
|
|
143
|
-
veris.set_fastapi_mcp(
|
|
144
|
-
fastapi=app,
|
|
145
|
-
name="My API Server"
|
|
146
|
-
)
|
|
147
|
-
|
|
148
|
-
# Now all traces will automatically include session IDs from bearer tokens
|
|
149
|
-
```
|
|
150
|
-
|
|
151
|
-
4. **Manual Session Management**: You can also manually set session IDs:
|
|
152
|
-
```python
|
|
153
|
-
veris.set_session_id("custom-session-123")
|
|
154
|
-
# All subsequent operations will be tagged with this session ID
|
|
155
|
-
veris.clear_session_id() # Clear when done
|
|
156
|
-
```
|
|
157
|
-
|
|
158
|
-
The session ID tracking works seamlessly with both the mock decorators and the tracing system, providing end-to-end visibility of user sessions across your application.
|
|
159
|
-
|
|
160
|
-
## Python Version
|
|
161
|
-
|
|
162
|
-
This project requires Python 3.11 or higher. We use [pyenv](https://github.com/pyenv/pyenv) for Python version management.
|
|
163
|
-
|
|
164
|
-
To set up the correct Python version:
|
|
165
|
-
|
|
166
|
-
```bash
|
|
167
|
-
# Install Python 3.11.0 using pyenv
|
|
168
|
-
pyenv install 3.11.0
|
|
169
|
-
|
|
170
|
-
# Set the local Python version for this project
|
|
171
|
-
pyenv local 3.11.0
|
|
172
|
-
```
|
|
173
|
-
|
|
174
|
-
## Usage
|
|
175
|
-
|
|
176
|
-
```python
|
|
177
|
-
from veris_ai import veris
|
|
178
|
-
|
|
179
|
-
@veris.mock()
|
|
180
|
-
async def your_function(param1: str, param2: int) -> dict:
|
|
181
|
-
"""
|
|
182
|
-
Your function documentation here.
|
|
183
|
-
|
|
184
|
-
Args:
|
|
185
|
-
param1: Description of param1
|
|
186
|
-
param2: Description of param2
|
|
187
|
-
|
|
188
|
-
Returns:
|
|
189
|
-
A dictionary containing the results
|
|
190
|
-
"""
|
|
191
|
-
# Your implementation here
|
|
192
|
-
return {"result": "actual implementation"}
|
|
193
|
-
```
|
|
194
|
-
|
|
195
|
-
When `ENV=simulation` is set, the decorator will:
|
|
196
|
-
1. Capture the function signature, type hints, and docstring
|
|
197
|
-
2. Send this information to the mock endpoint
|
|
198
|
-
3. Convert the mock response to the expected return type
|
|
199
|
-
4. Return the mock result
|
|
200
|
-
|
|
201
|
-
When not in simulation mode, the original function will be executed normally.
|
|
202
|
-
|
|
203
|
-
### Stub Decorator
|
|
204
|
-
|
|
205
|
-
For simple test scenarios where you want to return a fixed value in simulation mode, use the `@veris.stub()` decorator:
|
|
206
|
-
|
|
207
|
-
```python
|
|
208
|
-
from veris_ai import veris
|
|
209
|
-
|
|
210
|
-
@veris.stub(return_value={"status": "success", "data": [1, 2, 3]})
|
|
211
|
-
async def get_data() -> dict:
|
|
212
|
-
"""Get some data from external service."""
|
|
213
|
-
# This implementation is only called in production mode
|
|
214
|
-
return await fetch_from_api()
|
|
215
|
-
|
|
216
|
-
# In simulation mode: always returns {"status": "success", "data": [1, 2, 3]}
|
|
217
|
-
# In production mode: calls fetch_from_api()
|
|
218
|
-
```
|
|
219
|
-
|
|
220
|
-
The stub decorator is useful for:
|
|
221
|
-
- Quick prototyping with fixed responses
|
|
222
|
-
- Testing with predictable data
|
|
223
|
-
- Bypassing external dependencies in development
|
|
224
|
-
|
|
225
|
-
## FastAPI MCP Integration
|
|
226
|
-
|
|
227
|
-
The SDK provides integration with FastAPI applications through the Model Context Protocol (MCP). This allows your FastAPI endpoints to be exposed as MCP tools that can be used by AI agents.
|
|
228
|
-
|
|
229
|
-
```python
|
|
230
|
-
from fastapi import FastAPI
|
|
231
|
-
from veris_ai import veris
|
|
232
|
-
|
|
233
|
-
app = FastAPI()
|
|
234
|
-
|
|
235
|
-
# Set up FastAPI MCP with automatic session handling
|
|
236
|
-
veris.set_fastapi_mcp(
|
|
237
|
-
fastapi=app,
|
|
238
|
-
name="My API Server",
|
|
239
|
-
description="My FastAPI application exposed as MCP tools",
|
|
240
|
-
describe_all_responses=True,
|
|
241
|
-
include_operations=["get_users", "create_user"],
|
|
242
|
-
exclude_tags=["internal"]
|
|
243
|
-
)
|
|
244
|
-
|
|
245
|
-
# The MCP server is now available at veris.fastapi_mcp
|
|
246
|
-
mcp_server = veris.fastapi_mcp
|
|
247
|
-
```
|
|
248
|
-
|
|
249
|
-
### Configuration Options
|
|
250
|
-
|
|
251
|
-
The `set_fastapi_mcp()` method accepts the following parameters:
|
|
252
|
-
|
|
253
|
-
- **fastapi** (required): Your FastAPI application instance
|
|
254
|
-
- **name**: Custom name for the MCP server (defaults to app.title)
|
|
255
|
-
- **description**: Custom description (defaults to app.description)
|
|
256
|
-
- **describe_all_responses**: Whether to include all response schemas in tool descriptions
|
|
257
|
-
- **describe_full_response_schema**: Whether to include full JSON schema for responses
|
|
258
|
-
- **http_client**: Optional custom httpx.AsyncClient instance
|
|
259
|
-
- **include_operations**: List of operation IDs to include as MCP tools
|
|
260
|
-
- **exclude_operations**: List of operation IDs to exclude (can't use with include_operations)
|
|
261
|
-
- **include_tags**: List of tags to include as MCP tools
|
|
262
|
-
- **exclude_tags**: List of tags to exclude (can't use with include_tags)
|
|
263
|
-
- **auth_config**: Optional FastAPI MCP AuthConfig for custom authentication
|
|
264
|
-
|
|
265
|
-
### Session Management
|
|
266
|
-
|
|
267
|
-
The SDK automatically handles session management through OAuth2 authentication:
|
|
268
|
-
- Session IDs are extracted from bearer tokens
|
|
269
|
-
- Context is maintained across API calls
|
|
270
|
-
- Sessions can be managed programmatically:
|
|
271
|
-
|
|
272
|
-
```python
|
|
273
|
-
# Get current session ID
|
|
274
|
-
session_id = veris.session_id
|
|
275
|
-
|
|
276
|
-
# Set a new session ID
|
|
277
|
-
veris.set_session_id("new-session-123")
|
|
278
|
-
|
|
279
|
-
# Clear the session
|
|
280
|
-
veris.clear_session_id()
|
|
281
|
-
```
|
|
282
|
-
|
|
283
|
-
## Braintrust & Jaeger Tracing
|
|
284
|
-
|
|
285
|
-
The SDK includes a non-invasive instrumentation helper for sending traces to both Braintrust and a Jaeger-compatible OpenTelemetry (OTEL) collector simultaneously. This allows you to leverage Braintrust's powerful monitoring and evaluation tools while also storing and visualizing traces in your own Jaeger instance.
|
|
286
|
-
|
|
287
|
-
### Usage
|
|
288
|
-
|
|
289
|
-
To enable parallel tracing, simply import the `braintrust_tracing` module and call the `instrument()` function at the beginning of your application's lifecycle.
|
|
290
|
-
|
|
291
|
-
```python
|
|
292
|
-
from veris_ai import braintrust_tracing
|
|
293
|
-
|
|
294
|
-
# Enable Braintrust and Jaeger tracing
|
|
295
|
-
braintrust_tracing.instrument()
|
|
296
|
-
```
|
|
297
|
-
|
|
298
|
-
### Configuration
|
|
299
|
-
|
|
300
|
-
The `instrument()` function is configured through environment variables:
|
|
301
|
-
|
|
302
|
-
- **`VERIS_SERVICE_NAME`** (required): The name of your service as it will appear in Jaeger.
|
|
303
|
-
- **`VERIS_OTLP_ENDPOINT`** (required): The gRPC endpoint of your Jaeger OTLP collector (e.g., `http://host.docker.internal:4317`).
|
|
304
|
-
|
|
305
|
-
When initialized, the SDK automatically patches the `agents` library to send traces to both systems without any further code changes required.
|
|
306
|
-
|
|
307
|
-
## Utility Functions
|
|
308
|
-
|
|
309
|
-
### extract_json_schema
|
|
310
|
-
|
|
311
|
-
The SDK provides a utility function to extract JSON schemas from Python types and Pydantic models:
|
|
312
|
-
|
|
313
|
-
```python
|
|
314
|
-
from veris_ai.utils import extract_json_schema
|
|
315
|
-
from pydantic import BaseModel
|
|
316
|
-
from typing import List, Optional
|
|
317
|
-
|
|
318
|
-
# Extract schema from built-in types
|
|
319
|
-
int_schema = extract_json_schema(int)
|
|
320
|
-
# {"type": "integer"}
|
|
321
|
-
|
|
322
|
-
list_schema = extract_json_schema(List[str])
|
|
323
|
-
# {"type": "array", "items": {"type": "string"}}
|
|
324
|
-
|
|
325
|
-
# Extract schema from Pydantic models
|
|
326
|
-
class User(BaseModel):
|
|
327
|
-
name: str
|
|
328
|
-
age: int
|
|
329
|
-
email: Optional[str] = None
|
|
330
|
-
|
|
331
|
-
user_schema = extract_json_schema(User)
|
|
332
|
-
# Returns full JSON schema with properties, required fields, etc.
|
|
333
|
-
|
|
334
|
-
# Extract schema from TypedDict
|
|
335
|
-
from typing import TypedDict
|
|
336
|
-
|
|
337
|
-
class Config(TypedDict):
|
|
338
|
-
host: str
|
|
339
|
-
port: int
|
|
340
|
-
debug: bool
|
|
341
|
-
|
|
342
|
-
config_schema = extract_json_schema(Config)
|
|
343
|
-
# {"type": "object", "properties": {...}, "required": ["host", "port", "debug"]}
|
|
344
|
-
```
|
|
345
|
-
|
|
346
|
-
This function supports:
|
|
347
|
-
- Built-in types (str, int, float, bool, None)
|
|
348
|
-
- Generic types (List, Dict, Union, Optional, Literal)
|
|
349
|
-
- Pydantic models
|
|
350
|
-
- TypedDict classes
|
|
351
|
-
- Forward references and complex nested types
|
|
352
|
-
|
|
353
|
-
## Project Structure
|
|
354
|
-
|
|
355
|
-
```
|
|
356
|
-
src/veris_ai/
|
|
357
|
-
├── __init__.py # Main entry point, exports the veris instance
|
|
358
|
-
├── tool_mock.py # VerisSDK class with @veris.mock() decorator
|
|
359
|
-
└── utils.py # Type conversion and JSON schema utilities
|
|
360
|
-
|
|
361
|
-
tests/
|
|
362
|
-
├── conftest.py # Pytest fixtures
|
|
363
|
-
├── test_tool_mock.py # Tests for VerisSDK functionality
|
|
364
|
-
└── test_utils.py # Tests for type conversion and schema extraction
|
|
365
|
-
```
|
|
366
|
-
|
|
367
|
-
## Development
|
|
368
|
-
|
|
369
|
-
This project uses `pyproject.toml` for dependency management and `uv` for package installation.
|
|
370
|
-
|
|
371
|
-
### Development Dependencies
|
|
372
|
-
|
|
373
|
-
To install the package with development dependencies:
|
|
374
|
-
|
|
375
|
-
```bash
|
|
376
|
-
uv add "veris-ai[dev]"
|
|
377
|
-
```
|
|
378
|
-
|
|
379
|
-
This will install the following development tools:
|
|
380
|
-
- **Ruff**: Fast Python linter
|
|
381
|
-
- **pytest**: Testing framework
|
|
382
|
-
- **pytest-asyncio**: Async support for pytest
|
|
383
|
-
- **pytest-cov**: Coverage reporting for pytest
|
|
384
|
-
- **black**: Code formatter
|
|
385
|
-
- **mypy**: Static type checker
|
|
386
|
-
- **pre-commit**: Git hooks for code quality
|
|
387
|
-
|
|
388
|
-
### Code Quality
|
|
389
|
-
|
|
390
|
-
This project uses [Ruff](https://github.com/charliermarsh/ruff) for linting and code quality checks. Ruff is a fast Python linter written in Rust.
|
|
391
|
-
|
|
392
|
-
To run Ruff:
|
|
393
|
-
|
|
394
|
-
```bash
|
|
395
|
-
# Lint code
|
|
396
|
-
ruff check .
|
|
397
|
-
|
|
398
|
-
# Auto-fix linting issues
|
|
399
|
-
ruff check --fix .
|
|
400
|
-
|
|
401
|
-
# Format code
|
|
402
|
-
ruff format .
|
|
403
|
-
|
|
404
|
-
# Check formatting only
|
|
405
|
-
ruff format --check .
|
|
406
|
-
```
|
|
407
|
-
|
|
408
|
-
The Ruff configuration is defined in `pyproject.toml` under the `[tool.ruff]` section.
|
|
409
|
-
|
|
410
|
-
### Running Tests
|
|
411
|
-
|
|
412
|
-
```bash
|
|
413
|
-
# Run all tests with coverage
|
|
414
|
-
pytest tests/ --cov=veris_ai --cov-report=xml --cov-report=term-missing
|
|
415
|
-
|
|
416
|
-
# Run specific test file
|
|
417
|
-
pytest tests/test_tool_mock.py
|
|
418
|
-
|
|
419
|
-
# Run tests with verbose output
|
|
420
|
-
pytest -v tests/
|
|
421
|
-
```
|
|
422
|
-
|
|
423
|
-
### Type Checking
|
|
424
|
-
|
|
425
|
-
```bash
|
|
426
|
-
# Run static type checking
|
|
427
|
-
mypy src/veris_ai tests
|
|
428
|
-
```
|
|
429
|
-
|
|
430
|
-
## License
|
|
431
|
-
|
|
432
|
-
This project is licensed under the MIT License - see the LICENSE file for details.
|
|
433
|
-
|
|
434
|
-
## Jaeger Trace Interface
|
|
435
|
-
|
|
436
|
-
A lightweight, fully-typed wrapper around the Jaeger **Query Service** HTTP API lives under `veris_ai.jaeger_interface`.
|
|
437
|
-
It allows you to **search and retrieve traces** from both Jaeger's default storage back-ends as well as **OpenSearch**, with additional **client-side span filtering** capabilities.
|
|
438
|
-
|
|
439
|
-
```python
|
|
440
|
-
from veris_ai.jaeger_interface import JaegerClient
|
|
441
|
-
|
|
442
|
-
client = JaegerClient("http://localhost:16686")
|
|
443
|
-
|
|
444
|
-
# Search with trace-level filters (server-side)
|
|
445
|
-
traces = client.search(service="veris-agent", limit=2, tags={"error": "true"})
|
|
446
|
-
|
|
447
|
-
# Search with span-level filters (client-side, OR logic)
|
|
448
|
-
filtered = client.search(
|
|
449
|
-
service="veris-agent",
|
|
450
|
-
limit=10,
|
|
451
|
-
span_tags={"http.status_code": 500, "db.error": "timeout"}
|
|
452
|
-
)
|
|
453
|
-
|
|
454
|
-
first_trace = client.get_trace(traces.data[0].traceID)
|
|
455
|
-
```
|
|
456
|
-
|
|
457
|
-
See `src/veris_ai/jaeger_interface/README.md` for a complete walkthrough and API reference.
|
veris_ai-1.3.0.dist-info/RECORD
DELETED
|
@@ -1,13 +0,0 @@
|
|
|
1
|
-
veris_ai/__init__.py,sha256=Vp5yf9ZRchw0mmPwrRuNebI5cb2NGwpJGfr41l86q1U,1177
|
|
2
|
-
veris_ai/braintrust_tracing.py,sha256=0i-HR6IuK3-Q5ujMjT1FojxESRZLgqEvJ44bfJfDHaw,11938
|
|
3
|
-
veris_ai/models.py,sha256=6HINPxNFCakCVPcyEbUswWkXwb2K4lF0A8g8EvTMal4,213
|
|
4
|
-
veris_ai/tool_mock.py,sha256=U4xjINEZbtjutb_Vtr_So1ENNrU9kDROSyNUU5RL1Jc,10610
|
|
5
|
-
veris_ai/utils.py,sha256=3R2J9ko5t1UATiF1R6Hox9IPbtUz59EHsMDFg1-i7sk,9208
|
|
6
|
-
veris_ai/jaeger_interface/README.md,sha256=te5z3MWLqd2ECVV-a0MImBwTKRgQuuSuDB3bwIIsHi0,4437
|
|
7
|
-
veris_ai/jaeger_interface/__init__.py,sha256=d873a0zq3eUYU2Y77MtdjCwIARjAsAP7WDqGXDMWpYs,1158
|
|
8
|
-
veris_ai/jaeger_interface/client.py,sha256=yJrh86wRR0Dk3Gq12DId99WogcMIVbL0QQFqVSevvlE,8772
|
|
9
|
-
veris_ai/jaeger_interface/models.py,sha256=e64VV6IvOEFuzRUgvDAMQFyOZMRb56I-PUPZLBZ3rX0,1864
|
|
10
|
-
veris_ai-1.3.0.dist-info/METADATA,sha256=FD5DuzpYTZKjDmsPOb897HIMdOzS4l1UNTKh_68d29Y,13832
|
|
11
|
-
veris_ai-1.3.0.dist-info/WHEEL,sha256=qtCwoSJWgHk21S1Kb4ihdzI2rlJ1ZKaIurTj_ngOhyQ,87
|
|
12
|
-
veris_ai-1.3.0.dist-info/licenses/LICENSE,sha256=2g4i20atAgtD5einaKzhQrIB-JrPhyQgD3bC0wkHcCI,1065
|
|
13
|
-
veris_ai-1.3.0.dist-info/RECORD,,
|
|
File without changes
|
|
File without changes
|