arize_toolkit 0.1.0rc1__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- arize_toolkit-0.1.0rc1/LICENSE +21 -0
- arize_toolkit-0.1.0rc1/PKG-INFO +139 -0
- arize_toolkit-0.1.0rc1/README.md +112 -0
- arize_toolkit-0.1.0rc1/arize_toolkit/__init__.py +3 -0
- arize_toolkit-0.1.0rc1/arize_toolkit/client.py +1909 -0
- arize_toolkit-0.1.0rc1/arize_toolkit/constants.py +2 -0
- arize_toolkit-0.1.0rc1/arize_toolkit/exceptions.py +47 -0
- arize_toolkit-0.1.0rc1/arize_toolkit/model_managers.py +149 -0
- arize_toolkit-0.1.0rc1/arize_toolkit/models.py +368 -0
- arize_toolkit-0.1.0rc1/arize_toolkit/queries/__init__.py +0 -0
- arize_toolkit-0.1.0rc1/arize_toolkit/queries/basequery.py +126 -0
- arize_toolkit-0.1.0rc1/arize_toolkit/queries/custom_metric_queries.py +292 -0
- arize_toolkit-0.1.0rc1/arize_toolkit/queries/language_models.py +378 -0
- arize_toolkit-0.1.0rc1/arize_toolkit/queries/model_queries.py +267 -0
- arize_toolkit-0.1.0rc1/arize_toolkit/queries/monitor_queries.py +275 -0
- arize_toolkit-0.1.0rc1/arize_toolkit/queries/space_queries.py +68 -0
- arize_toolkit-0.1.0rc1/arize_toolkit/types.py +541 -0
- arize_toolkit-0.1.0rc1/arize_toolkit/utils.py +437 -0
- arize_toolkit-0.1.0rc1/pyproject.toml +42 -0
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2025 Duncan McKinnon
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
@@ -0,0 +1,139 @@
|
|
|
1
|
+
Metadata-Version: 2.3
|
|
2
|
+
Name: arize_toolkit
|
|
3
|
+
Version: 0.1.0rc1
|
|
4
|
+
Summary: A library to interact with Arize AI APIs
|
|
5
|
+
License: MIT
|
|
6
|
+
Keywords: arize,observability,monitoring,graphql
|
|
7
|
+
Author: Duncan McKinnon
|
|
8
|
+
Author-email: dmckinnon@arize.com
|
|
9
|
+
Requires-Python: >=3.9,<3.13
|
|
10
|
+
Classifier: Intended Audience :: Developers
|
|
11
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
12
|
+
Classifier: Programming Language :: Python :: 3
|
|
13
|
+
Classifier: Programming Language :: Python :: 3.9
|
|
14
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
15
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
16
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
17
|
+
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
|
|
18
|
+
Classifier: Topic :: Software Development :: Libraries :: Python Modules
|
|
19
|
+
Requires-Dist: gql (>=3.0.0,<4.0)
|
|
20
|
+
Requires-Dist: pandas (>=0.25.3,<3.0)
|
|
21
|
+
Requires-Dist: pydantic (>=2.0,<2.1)
|
|
22
|
+
Requires-Dist: requests (>=2.28.1,<3.0)
|
|
23
|
+
Requires-Dist: requests-toolbelt (>=1.0.0)
|
|
24
|
+
Project-URL: Homepage, https://www.arize.com
|
|
25
|
+
Description-Content-Type: text/markdown
|
|
26
|
+
|
|
27
|
+
[](https://github.com/duncankmckinnon/arize_toolkit/actions/workflows/test.yml)
|
|
28
|
+
[](https://github.com/duncankmckinnon/arize_toolkit/actions/workflows/docs.yml)
|
|
29
|
+
[](https://github.com/duncankmckinnon/arize_toolkit/actions/workflows/publish.yml)
|
|
30
|
+
# Arize Toolkit
|
|
31
|
+
|
|
32
|
+
Welcome to the documentation for Arize Toolkit, a Python client for interacting with the Arize AI API.
|
|
33
|
+
|
|
34
|
+
Check out the [latest docs](https://duncankmckinnon.github.io/arize_toolkit) for the latest features and information.
|
|
35
|
+
|
|
36
|
+
## Overview
|
|
37
|
+
|
|
38
|
+
Arize Toolkit is a set of tools packaged as a Python client that lets you easily interact with Arize AI APIs.
|
|
39
|
+
Here's a quick overview of the main features in the current release:
|
|
40
|
+
|
|
41
|
+
- Access and manage models
|
|
42
|
+
- Retrieve performance metrics over a time period
|
|
43
|
+
- Retrieve inference volume over a time period
|
|
44
|
+
- Create, copy, and manage custom metrics
|
|
45
|
+
- Create, copy, and manage monitors and alerting
|
|
46
|
+
- Work with LLM features like prompts and annotations
|
|
47
|
+
|
|
48
|
+
## Installation
|
|
49
|
+
|
|
50
|
+
```bash
|
|
51
|
+
pip install arize_toolkit
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
## Client Setup
|
|
55
|
+
The `Client` class is the entrypoint for interacting with the toolkit. It provides maintains the connection information for making requests to the Arize APIs, and offers a wide range of operations for interacting with models, monitors, dashboards, and more.
|
|
56
|
+
|
|
57
|
+
### API Key
|
|
58
|
+
To create a client, you need to provide your Arize API key. Use this reference to [get your API key](https://docs.arize.com/arize/reference/authentication-and-security/api-keys) from the Arize UI.
|
|
59
|
+
|
|
60
|
+

|
|
61
|
+
|
|
62
|
+
### Organization and Space
|
|
63
|
+
|
|
64
|
+
You will also need to provide an `organization` name and `space` name. To give some context, models are scoped to a space, and the space is scoped to an organization. These can be found by navigating to the Arize UI and looking at the upper left path in the `Projects & Models` page. They will be in the format `organization/space`.
|
|
65
|
+
|
|
66
|
+
For the example below, the organization is `Demo Models` and the space is `Demo Model Manager`.
|
|
67
|
+
|
|
68
|
+

|
|
69
|
+
|
|
70
|
+
### For On Prem deployments
|
|
71
|
+
|
|
72
|
+
For SaaS users, the default API endpoint is always going to be `https://api.arize.com`.
|
|
73
|
+
If you are using an on prem deployment of Arize, you will need to provide the `api_url` parameter.
|
|
74
|
+
This parameters should just be the base url of your Arize instance.
|
|
75
|
+
|
|
76
|
+
### Sleep Time
|
|
77
|
+
The `Client` class can be configured to wait a certain amount of time between requests. This is useful to avoid rate limiting.
|
|
78
|
+
The default sleep time is 0 seconds. When instantiating the client, you can set the `sleep_time` parameter to the number of seconds to wait between requests. This can save time if you are making a large number or requests or getting a large amount of data. If you start seeing rate limit errors, you can increase this value. There is a helper function in the client to update the sleep time in line:
|
|
79
|
+
|
|
80
|
+
#### `set_sleep_time`
|
|
81
|
+
```python
|
|
82
|
+
from arize_toolkit import Client
|
|
83
|
+
|
|
84
|
+
client: Client = client.set_sleep_time(sleep_time: int)
|
|
85
|
+
```
|
|
86
|
+
This will update the sleep time for the existing client in line.
|
|
87
|
+
|
|
88
|
+
* **Parameters**
|
|
89
|
+
* `sleep_time` – The number of seconds to wait between requests
|
|
90
|
+
|
|
91
|
+
* **Returns**
|
|
92
|
+
* `Client` – The updated client
|
|
93
|
+
|
|
94
|
+
* **Example**
|
|
95
|
+
```python
|
|
96
|
+
from arize_toolkit import Client
|
|
97
|
+
|
|
98
|
+
client = Client(..., sleep_time=0)
|
|
99
|
+
|
|
100
|
+
# Update the sleep time in line
|
|
101
|
+
lots_of_models = client.set_sleep_time(10).get_all_models()
|
|
102
|
+
```
|
|
103
|
+
|
|
104
|
+
### Switching Spaces and Organizations
|
|
105
|
+
|
|
106
|
+
The `Client` class can be configured to switch to a different space or a space in a different organization. This is useful if you are working with multiple spaces or if you need to switch to a different space on the fly without having to create a new client.
|
|
107
|
+
|
|
108
|
+
#### `set_space`
|
|
109
|
+
```python
|
|
110
|
+
space_url: str = client.set_space(space: str, organization: Optional[str] = None)
|
|
111
|
+
```
|
|
112
|
+
This will update the space and organization for the existing client. It will also return the full url of the new space, which is comprised of the organization and space ids.
|
|
113
|
+
|
|
114
|
+
* **Parameters**
|
|
115
|
+
* `space` – The name of the space to switch to
|
|
116
|
+
* `organization` – *Optional* The name of the organization to switch to if different from the current organization
|
|
117
|
+
|
|
118
|
+
* **Returns**
|
|
119
|
+
* `str` – The full url of the new space
|
|
120
|
+
|
|
121
|
+
* **Example**
|
|
122
|
+
```python
|
|
123
|
+
client = Client(..., space="my_space")
|
|
124
|
+
|
|
125
|
+
# get the current space url
|
|
126
|
+
current_space_url = client.space_url
|
|
127
|
+
|
|
128
|
+
# Get all models in the current space
|
|
129
|
+
current_space_models = client.get_all_models()
|
|
130
|
+
|
|
131
|
+
# Switch to a different space
|
|
132
|
+
other_space_url = client.set_space(space="my_other_space", organization="my_other_organization")
|
|
133
|
+
|
|
134
|
+
# Get all models in the new space
|
|
135
|
+
other_space_models = client.get_all_models()
|
|
136
|
+
```
|
|
137
|
+
|
|
138
|
+
|
|
139
|
+
|
|
@@ -0,0 +1,112 @@
|
|
|
1
|
+
[](https://github.com/duncankmckinnon/arize_toolkit/actions/workflows/test.yml)
|
|
2
|
+
[](https://github.com/duncankmckinnon/arize_toolkit/actions/workflows/docs.yml)
|
|
3
|
+
[](https://github.com/duncankmckinnon/arize_toolkit/actions/workflows/publish.yml)
|
|
4
|
+
# Arize Toolkit
|
|
5
|
+
|
|
6
|
+
Welcome to the documentation for Arize Toolkit, a Python client for interacting with the Arize AI API.
|
|
7
|
+
|
|
8
|
+
Check out the [latest docs](https://duncankmckinnon.github.io/arize_toolkit) for the latest features and information.
|
|
9
|
+
|
|
10
|
+
## Overview
|
|
11
|
+
|
|
12
|
+
Arize Toolkit is a set of tools packaged as a Python client that lets you easily interact with Arize AI APIs.
|
|
13
|
+
Here's a quick overview of the main features in the current release:
|
|
14
|
+
|
|
15
|
+
- Access and manage models
|
|
16
|
+
- Retrieve performance metrics over a time period
|
|
17
|
+
- Retrieve inference volume over a time period
|
|
18
|
+
- Create, copy, and manage custom metrics
|
|
19
|
+
- Create, copy, and manage monitors and alerting
|
|
20
|
+
- Work with LLM features like prompts and annotations
|
|
21
|
+
|
|
22
|
+
## Installation
|
|
23
|
+
|
|
24
|
+
```bash
|
|
25
|
+
pip install arize_toolkit
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
## Client Setup
|
|
29
|
+
The `Client` class is the entrypoint for interacting with the toolkit. It provides maintains the connection information for making requests to the Arize APIs, and offers a wide range of operations for interacting with models, monitors, dashboards, and more.
|
|
30
|
+
|
|
31
|
+
### API Key
|
|
32
|
+
To create a client, you need to provide your Arize API key. Use this reference to [get your API key](https://docs.arize.com/arize/reference/authentication-and-security/api-keys) from the Arize UI.
|
|
33
|
+
|
|
34
|
+

|
|
35
|
+
|
|
36
|
+
### Organization and Space
|
|
37
|
+
|
|
38
|
+
You will also need to provide an `organization` name and `space` name. To give some context, models are scoped to a space, and the space is scoped to an organization. These can be found by navigating to the Arize UI and looking at the upper left path in the `Projects & Models` page. They will be in the format `organization/space`.
|
|
39
|
+
|
|
40
|
+
For the example below, the organization is `Demo Models` and the space is `Demo Model Manager`.
|
|
41
|
+
|
|
42
|
+

|
|
43
|
+
|
|
44
|
+
### For On Prem deployments
|
|
45
|
+
|
|
46
|
+
For SaaS users, the default API endpoint is always going to be `https://api.arize.com`.
|
|
47
|
+
If you are using an on prem deployment of Arize, you will need to provide the `api_url` parameter.
|
|
48
|
+
This parameters should just be the base url of your Arize instance.
|
|
49
|
+
|
|
50
|
+
### Sleep Time
|
|
51
|
+
The `Client` class can be configured to wait a certain amount of time between requests. This is useful to avoid rate limiting.
|
|
52
|
+
The default sleep time is 0 seconds. When instantiating the client, you can set the `sleep_time` parameter to the number of seconds to wait between requests. This can save time if you are making a large number or requests or getting a large amount of data. If you start seeing rate limit errors, you can increase this value. There is a helper function in the client to update the sleep time in line:
|
|
53
|
+
|
|
54
|
+
#### `set_sleep_time`
|
|
55
|
+
```python
|
|
56
|
+
from arize_toolkit import Client
|
|
57
|
+
|
|
58
|
+
client: Client = client.set_sleep_time(sleep_time: int)
|
|
59
|
+
```
|
|
60
|
+
This will update the sleep time for the existing client in line.
|
|
61
|
+
|
|
62
|
+
* **Parameters**
|
|
63
|
+
* `sleep_time` – The number of seconds to wait between requests
|
|
64
|
+
|
|
65
|
+
* **Returns**
|
|
66
|
+
* `Client` – The updated client
|
|
67
|
+
|
|
68
|
+
* **Example**
|
|
69
|
+
```python
|
|
70
|
+
from arize_toolkit import Client
|
|
71
|
+
|
|
72
|
+
client = Client(..., sleep_time=0)
|
|
73
|
+
|
|
74
|
+
# Update the sleep time in line
|
|
75
|
+
lots_of_models = client.set_sleep_time(10).get_all_models()
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
### Switching Spaces and Organizations
|
|
79
|
+
|
|
80
|
+
The `Client` class can be configured to switch to a different space or a space in a different organization. This is useful if you are working with multiple spaces or if you need to switch to a different space on the fly without having to create a new client.
|
|
81
|
+
|
|
82
|
+
#### `set_space`
|
|
83
|
+
```python
|
|
84
|
+
space_url: str = client.set_space(space: str, organization: Optional[str] = None)
|
|
85
|
+
```
|
|
86
|
+
This will update the space and organization for the existing client. It will also return the full url of the new space, which is comprised of the organization and space ids.
|
|
87
|
+
|
|
88
|
+
* **Parameters**
|
|
89
|
+
* `space` – The name of the space to switch to
|
|
90
|
+
* `organization` – *Optional* The name of the organization to switch to if different from the current organization
|
|
91
|
+
|
|
92
|
+
* **Returns**
|
|
93
|
+
* `str` – The full url of the new space
|
|
94
|
+
|
|
95
|
+
* **Example**
|
|
96
|
+
```python
|
|
97
|
+
client = Client(..., space="my_space")
|
|
98
|
+
|
|
99
|
+
# get the current space url
|
|
100
|
+
current_space_url = client.space_url
|
|
101
|
+
|
|
102
|
+
# Get all models in the current space
|
|
103
|
+
current_space_models = client.get_all_models()
|
|
104
|
+
|
|
105
|
+
# Switch to a different space
|
|
106
|
+
other_space_url = client.set_space(space="my_other_space", organization="my_other_organization")
|
|
107
|
+
|
|
108
|
+
# Get all models in the new space
|
|
109
|
+
other_space_models = client.get_all_models()
|
|
110
|
+
```
|
|
111
|
+
|
|
112
|
+
|