eval-studio-client 1.0.3__py3-none-any.whl → 1.1.0__py3-none-any.whl

eval_studio_client/api/api_client.py

@@ -1,7 +1,7 @@
 # coding: utf-8
 
 """
-    ai/h2o/eval_studio/v1/collection.proto
+    ai/h2o/eval_studio/v1/insight.proto
 
     No description provided (generated by Openapi Generator https://github.com/openapitools/openapi-generator)
 

eval_studio_client/api/configuration.py

@@ -1,7 +1,7 @@
 # coding: utf-8
 
 """
-    ai/h2o/eval_studio/v1/collection.proto
+    ai/h2o/eval_studio/v1/insight.proto
 
     No description provided (generated by Openapi Generator https://github.com/openapitools/openapi-generator)
 

eval_studio_client/api/docs/AdversarialInputsServiceApi.md

@@ -0,0 +1,78 @@
+# eval_studio_client.api.AdversarialInputsServiceApi
+
+All URIs are relative to *http://localhost*
+
+Method | HTTP request | Description
+------------- | ------------- | -------------
+[**adversarial_inputs_service_test_adversarial_inputs_robustness**](AdversarialInputsServiceApi.md#adversarial_inputs_service_test_adversarial_inputs_robustness) | **POST** /v1/{test}:testAdversarialInputsRobustness | 
+
+
+# **adversarial_inputs_service_test_adversarial_inputs_robustness**
+> V1Operation adversarial_inputs_service_test_adversarial_inputs_robustness(test, body)
+
+
+
+### Example
+
+
+```python
+import eval_studio_client.api
+from eval_studio_client.api.models.adversarial_inputs_service_test_adversarial_inputs_robustness_request import AdversarialInputsServiceTestAdversarialInputsRobustnessRequest
+from eval_studio_client.api.models.v1_operation import V1Operation
+from eval_studio_client.api.rest import ApiException
+from pprint import pprint
+
+# Defining the host is optional and defaults to http://localhost
+# See configuration.py for a list of all supported configuration parameters.
+configuration = eval_studio_client.api.Configuration(
+    host = "http://localhost"
+)
+
+
+# Enter a context with an instance of the API client
+with eval_studio_client.api.ApiClient(configuration) as api_client:
+    # Create an instance of the API class
+    api_instance = eval_studio_client.api.AdversarialInputsServiceApi(api_client)
+    test = 'test_example' # str | Required. The Test to which the adversarial inputs TestCases will be added.
+    body = eval_studio_client.api.AdversarialInputsServiceTestAdversarialInputsRobustnessRequest() # AdversarialInputsServiceTestAdversarialInputsRobustnessRequest | 
+
+    try:
+        api_response = api_instance.adversarial_inputs_service_test_adversarial_inputs_robustness(test, body)
+        print("The response of AdversarialInputsServiceApi->adversarial_inputs_service_test_adversarial_inputs_robustness:\n")
+        pprint(api_response)
+    except Exception as e:
+        print("Exception when calling AdversarialInputsServiceApi->adversarial_inputs_service_test_adversarial_inputs_robustness: %s\n" % e)
+```
+
+
+
+### Parameters
+
+
+Name | Type | Description  | Notes
+------------- | ------------- | ------------- | -------------
+ **test** | **str**| Required. The Test to which the adversarial inputs TestCases will be added. | 
+ **body** | [**AdversarialInputsServiceTestAdversarialInputsRobustnessRequest**](AdversarialInputsServiceTestAdversarialInputsRobustnessRequest.md)|  | 
+
+### Return type
+
+[**V1Operation**](V1Operation.md)
+
+### Authorization
+
+No authorization required
+
+### HTTP request headers
+
+ - **Content-Type**: application/json
+ - **Accept**: application/json
+
+### HTTP response details
+
+| Status code | Description | Response headers |
+|-------------|-------------|------------------|
+**200** | A successful response. |  -  |
+**0** | An unexpected error response. |  -  |
+
+[[Back to top]](#) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to Model list]](../README.md#documentation-for-models) [[Back to README]](../README.md)
+

eval_studio_client/api/docs/AdversarialInputsServiceTestAdversarialInputsRobustnessRequest.md

@@ -0,0 +1,45 @@
+# AdversarialInputsServiceTestAdversarialInputsRobustnessRequest
+
+
+## Properties
+
+Name | Type | Description | Notes
+------------ | ------------- | ------------- | -------------
+**operation** | **str** | Required. The Operation processing adversarial inputs robustness testing. | [optional] 
+**generator_input_types** | [**List[V1TestCasesGenerator]**](V1TestCasesGenerator.md) | Optional. The list of adversarial input types to generate. | [optional] 
+**generator_document_urls** | **List[str]** | Required. The document URLs which were used to generate the baseline TestCases. | [optional] 
+**generator_model** | [**V1Model**](V1Model.md) |  | [optional] 
+**generator_base_llm_model** | **str** | Required. Base LLM model to use for generation of adversarial the prompts. | [optional] 
+**generator_count** | **int** | Required. The number of adversarial TestCases to generate. | [optional] 
+**generator_topics** | **List[str]** | Optional. Topics to generate questions for. If not specified, use document summarization as topic generation. | [optional] 
+**generator_chunks** | **List[str]** | Optional. The list of chunks to use for generation. If set, the Documents assigned to the Test and h2ogpte_collection_id are ignored. | [optional] 
+**generator_h2ogpte_collection_id** | **str** | Optional. ID of the h2oGPTe collection to use. If provided, documents referenced by Test and any specified chunks are ignored. This field is required if Test does not reference any documents and no chunks are provided. If this field is left empty, a temporary collection will be created. | [optional] 
+**evaluator_identifiers** | **List[str]** | Required. Evaluator identifiers to use for the model evaluation using the adversarial inputs. | [optional] 
+**evaluators_parameters** | **Dict[str, str]** | Optional. Additional evaluators configuration, for all the evaluators used in the evaluation. Key is the evaluator identifier, and the value is a JSON string containing the configuration dictionary. | [optional] 
+**model** | [**V1Model**](V1Model.md) |  | [optional] 
+**base_llm_model** | **str** | Required. Base LLM model to be evaluated using the adversarial inputs. | [optional] 
+**model_parameters** | **str** | Optional. Parameters overrides for the Model host in JSON format. | [optional] 
+**default_h2ogpte_model** | [**V1Model**](V1Model.md) |  | [optional] 
+**baseline_eval** | **str** | Required. Baseline evaluation name. | [optional] 
+**baseline_metrics** | [**Dict[str, V1MetricScores]**](V1MetricScores.md) | Required. Map of baseline metrics from the evaluator to the metric scores for the evaluator. | [optional] 
+
+## Example
+
+```python
+from eval_studio_client.api.models.adversarial_inputs_service_test_adversarial_inputs_robustness_request import AdversarialInputsServiceTestAdversarialInputsRobustnessRequest
+
+# TODO update the JSON string below
+json = "{}"
+# create an instance of AdversarialInputsServiceTestAdversarialInputsRobustnessRequest from a JSON string
+adversarial_inputs_service_test_adversarial_inputs_robustness_request_instance = AdversarialInputsServiceTestAdversarialInputsRobustnessRequest.from_json(json)
+# print the JSON string representation of the object
+print(AdversarialInputsServiceTestAdversarialInputsRobustnessRequest.to_json())
+
+# convert the object into a dict
+adversarial_inputs_service_test_adversarial_inputs_robustness_request_dict = adversarial_inputs_service_test_adversarial_inputs_robustness_request_instance.to_dict()
+# create an instance of AdversarialInputsServiceTestAdversarialInputsRobustnessRequest from a dict
+adversarial_inputs_service_test_adversarial_inputs_robustness_request_from_dict = AdversarialInputsServiceTestAdversarialInputsRobustnessRequest.from_dict(adversarial_inputs_service_test_adversarial_inputs_robustness_request_dict)
+```
+[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)
+
+

eval_studio_client/api/docs/DashboardServiceApi.md

@@ -413,7 +413,7 @@ No authorization required
 [[Back to top]](#) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to Model list]](../README.md#documentation-for-models) [[Back to README]](../README.md)
 
 # **dashboard_service_list_most_recent_dashboards**
-> V1ListMostRecentDashboardsResponse dashboard_service_list_most_recent_dashboards(limit=limit)
+> V1ListMostRecentDashboardsResponse dashboard_service_list_most_recent_dashboards(limit=limit, filter=filter)
 
 
 
@@ -438,9 +438,10 @@ with eval_studio_client.api.ApiClient(configuration) as api_client:
     # Create an instance of the API class
     api_instance = eval_studio_client.api.DashboardServiceApi(api_client)
     limit = 56 # int | Optional. The max number of the most recent Dashboards to retrieve. Use -1 to retrieve all. Defaults to 3. (optional)
+    filter = 'filter_example' # str | Optional. If specified, only leaderboards matching the filter will be returned. Attempts to implement AIP-160 (https://aip.dev/160), although not all fields, operators and features are supported.  Supported fields: - model   - only '=' operator is supported, e.g. \"model = \\\"models/<UUID>\\\"\" (optional)
 
     try:
-        api_response = api_instance.dashboard_service_list_most_recent_dashboards(limit=limit)
+        api_response = api_instance.dashboard_service_list_most_recent_dashboards(limit=limit, filter=filter)
         print("The response of DashboardServiceApi->dashboard_service_list_most_recent_dashboards:\n")
         pprint(api_response)
     except Exception as e:
@@ -455,6 +456,7 @@ with eval_studio_client.api.ApiClient(configuration) as api_client:
 Name | Type | Description  | Notes
 ------------- | ------------- | ------------- | -------------
  **limit** | **int**| Optional. The max number of the most recent Dashboards to retrieve. Use -1 to retrieve all. Defaults to 3. | [optional] 
+ **filter** | **str**| Optional. If specified, only leaderboards matching the filter will be returned. Attempts to implement AIP-160 (https://aip.dev/160), although not all fields, operators and features are supported.  Supported fields: - model   - only &#39;&#x3D;&#39; operator is supported, e.g. \&quot;model &#x3D; \\\&quot;models/&lt;UUID&gt;\\\&quot;\&quot; | [optional] 
 
 ### Return type
 

eval_studio_client/api/docs/HumanCalibrationServiceApi.md

@@ -0,0 +1,77 @@
+# eval_studio_client.api.HumanCalibrationServiceApi
+
+All URIs are relative to *http://localhost*
+
+Method | HTTP request | Description
+------------- | ------------- | -------------
+[**human_calibration_service_estimate_threshold**](HumanCalibrationServiceApi.md#human_calibration_service_estimate_threshold) | **POST** /v1:estimateThreshold | EstimateThreshold runs a threshold estimation process based on human labeling of randomly sampled test-cases.
+
+
+# **human_calibration_service_estimate_threshold**
+> V1Operation human_calibration_service_estimate_threshold(body)
+
+EstimateThreshold runs a threshold estimation process based on human labeling of randomly sampled test-cases.
+
+### Example
+
+
+```python
+import eval_studio_client.api
+from eval_studio_client.api.models.v1_estimate_threshold_request import V1EstimateThresholdRequest
+from eval_studio_client.api.models.v1_operation import V1Operation
+from eval_studio_client.api.rest import ApiException
+from pprint import pprint
+
+# Defining the host is optional and defaults to http://localhost
+# See configuration.py for a list of all supported configuration parameters.
+configuration = eval_studio_client.api.Configuration(
+    host = "http://localhost"
+)
+
+
+# Enter a context with an instance of the API client
+with eval_studio_client.api.ApiClient(configuration) as api_client:
+    # Create an instance of the API class
+    api_instance = eval_studio_client.api.HumanCalibrationServiceApi(api_client)
+    body = eval_studio_client.api.V1EstimateThresholdRequest() # V1EstimateThresholdRequest | 
+
+    try:
+        # EstimateThreshold runs a threshold estimation process based on human labeling of randomly sampled test-cases.
+        api_response = api_instance.human_calibration_service_estimate_threshold(body)
+        print("The response of HumanCalibrationServiceApi->human_calibration_service_estimate_threshold:\n")
+        pprint(api_response)
+    except Exception as e:
+        print("Exception when calling HumanCalibrationServiceApi->human_calibration_service_estimate_threshold: %s\n" % e)
+```
+
+
+
+### Parameters
+
+
+Name | Type | Description  | Notes
+------------- | ------------- | ------------- | -------------
+ **body** | [**V1EstimateThresholdRequest**](V1EstimateThresholdRequest.md)|  | 
+
+### Return type
+
+[**V1Operation**](V1Operation.md)
+
+### Authorization
+
+No authorization required
+
+### HTTP request headers
+
+ - **Content-Type**: application/json
+ - **Accept**: application/json
+
+### HTTP response details
+
+| Status code | Description | Response headers |
+|-------------|-------------|------------------|
+**200** | A successful response. |  -  |
+**0** | An unexpected error response. |  -  |
+
+[[Back to top]](#) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to Model list]](../README.md#documentation-for-models) [[Back to README]](../README.md)
+

eval_studio_client/api/docs/LeaderboardReportServiceApi.md

@@ -0,0 +1,75 @@
+# eval_studio_client.api.LeaderboardReportServiceApi
+
+All URIs are relative to *http://localhost*
+
+Method | HTTP request | Description
+------------- | ------------- | -------------
+[**leaderboard_report_service_get_leaderboard_report**](LeaderboardReportServiceApi.md#leaderboard_report_service_get_leaderboard_report) | **GET** /v1/{name_3} | 
+
+
+# **leaderboard_report_service_get_leaderboard_report**
+> V1GetLeaderboardReportResponse leaderboard_report_service_get_leaderboard_report(name_3)
+
+
+
+### Example
+
+
+```python
+import eval_studio_client.api
+from eval_studio_client.api.models.v1_get_leaderboard_report_response import V1GetLeaderboardReportResponse
+from eval_studio_client.api.rest import ApiException
+from pprint import pprint
+
+# Defining the host is optional and defaults to http://localhost
+# See configuration.py for a list of all supported configuration parameters.
+configuration = eval_studio_client.api.Configuration(
+    host = "http://localhost"
+)
+
+
+# Enter a context with an instance of the API client
+with eval_studio_client.api.ApiClient(configuration) as api_client:
+    # Create an instance of the API class
+    api_instance = eval_studio_client.api.LeaderboardReportServiceApi(api_client)
+    name_3 = 'name_3_example' # str | Required. The name of the Leaderboard to retrieve.
+
+    try:
+        api_response = api_instance.leaderboard_report_service_get_leaderboard_report(name_3)
+        print("The response of LeaderboardReportServiceApi->leaderboard_report_service_get_leaderboard_report:\n")
+        pprint(api_response)
+    except Exception as e:
+        print("Exception when calling LeaderboardReportServiceApi->leaderboard_report_service_get_leaderboard_report: %s\n" % e)
+```
+
+
+
+### Parameters
+
+
+Name | Type | Description  | Notes
+------------- | ------------- | ------------- | -------------
+ **name_3** | **str**| Required. The name of the Leaderboard to retrieve. | 
+
+### Return type
+
+[**V1GetLeaderboardReportResponse**](V1GetLeaderboardReportResponse.md)
+
+### Authorization
+
+No authorization required
+
+### HTTP request headers
+
+ - **Content-Type**: Not defined
+ - **Accept**: application/json
+
+### HTTP response details
+
+| Status code | Description | Response headers |
+|-------------|-------------|------------------|
+**200** | A successful response. |  -  |
+**0** | An unexpected error response. |  -  |
+
+[[Back to top]](#) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to Model list]](../README.md#documentation-for-models) [[Back to README]](../README.md)
+

eval_studio_client/api/docs/LeaderboardServiceApi.md

@@ -11,7 +11,7 @@ Method | HTTP request | Description
 [**leaderboard_service_create_leaderboard**](LeaderboardServiceApi.md#leaderboard_service_create_leaderboard) | **POST** /v1/leaderboards | 
 [**leaderboard_service_create_leaderboard_without_cache**](LeaderboardServiceApi.md#leaderboard_service_create_leaderboard_without_cache) | **POST** /v1/leaderboards:withoutCache | 
 [**leaderboard_service_delete_leaderboard**](LeaderboardServiceApi.md#leaderboard_service_delete_leaderboard) | **DELETE** /v1/{name_3} | 
-[**leaderboard_service_get_leaderboard**](LeaderboardServiceApi.md#leaderboard_service_get_leaderboard) | **GET** /v1/{name_3} | 
+[**leaderboard_service_get_leaderboard**](LeaderboardServiceApi.md#leaderboard_service_get_leaderboard) | **GET** /v1/{name_4} | 
 [**leaderboard_service_import_leaderboard**](LeaderboardServiceApi.md#leaderboard_service_import_leaderboard) | **POST** /v1/leaderboards:import | 
 [**leaderboard_service_list_leaderboards**](LeaderboardServiceApi.md#leaderboard_service_list_leaderboards) | **GET** /v1/leaderboards | 
 [**leaderboard_service_list_most_recent_leaderboards**](LeaderboardServiceApi.md#leaderboard_service_list_most_recent_leaderboards) | **GET** /v1/leaderboards:mostRecent | 
@@ -488,7 +488,7 @@ No authorization required
 [[Back to top]](#) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to Model list]](../README.md#documentation-for-models) [[Back to README]](../README.md)
 
 # **leaderboard_service_get_leaderboard**
-> V1GetLeaderboardResponse leaderboard_service_get_leaderboard(name_3)
+> V1GetLeaderboardResponse leaderboard_service_get_leaderboard(name_4)
 
 
 
@@ -512,10 +512,10 @@ configuration = eval_studio_client.api.Configuration(
 with eval_studio_client.api.ApiClient(configuration) as api_client:
     # Create an instance of the API class
     api_instance = eval_studio_client.api.LeaderboardServiceApi(api_client)
-    name_3 = 'name_3_example' # str | Required. The name of the Leaderboard to retrieve.
+    name_4 = 'name_4_example' # str | Required. The name of the Leaderboard to retrieve.
 
     try:
-        api_response = api_instance.leaderboard_service_get_leaderboard(name_3)
+        api_response = api_instance.leaderboard_service_get_leaderboard(name_4)
         print("The response of LeaderboardServiceApi->leaderboard_service_get_leaderboard:\n")
         pprint(api_response)
     except Exception as e:
@@ -529,7 +529,7 @@ with eval_studio_client.api.ApiClient(configuration) as api_client:
 
 Name | Type | Description  | Notes
 ------------- | ------------- | ------------- | -------------
- **name_3** | **str**| Required. The name of the Leaderboard to retrieve. | 
+ **name_4** | **str**| Required. The name of the Leaderboard to retrieve. | 
 
 ### Return type
 

eval_studio_client/api/docs/ModelServiceApi.md

@@ -9,7 +9,7 @@ Method | HTTP request | Description
 [**model_service_check_base_models**](ModelServiceApi.md#model_service_check_base_models) | **GET** /v1/models:check_base_models | 
 [**model_service_create_model**](ModelServiceApi.md#model_service_create_model) | **POST** /v1/models | 
 [**model_service_delete_model**](ModelServiceApi.md#model_service_delete_model) | **DELETE** /v1/{name_4} | 
-[**model_service_get_model**](ModelServiceApi.md#model_service_get_model) | **GET** /v1/{name_4} | 
+[**model_service_get_model**](ModelServiceApi.md#model_service_get_model) | **GET** /v1/{name_5} | 
 [**model_service_list_base_models**](ModelServiceApi.md#model_service_list_base_models) | **GET** /v1/models:base_models | 
 [**model_service_list_model_collections**](ModelServiceApi.md#model_service_list_model_collections) | **GET** /v1/models:collections | 
 [**model_service_list_models**](ModelServiceApi.md#model_service_list_models) | **GET** /v1/models | 
@@ -350,7 +350,7 @@ No authorization required
 [[Back to top]](#) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to Model list]](../README.md#documentation-for-models) [[Back to README]](../README.md)
 
 # **model_service_get_model**
-> V1GetModelResponse model_service_get_model(name_4)
+> V1GetModelResponse model_service_get_model(name_5)
 
 
 
@@ -374,10 +374,10 @@ configuration = eval_studio_client.api.Configuration(
 with eval_studio_client.api.ApiClient(configuration) as api_client:
     # Create an instance of the API class
     api_instance = eval_studio_client.api.ModelServiceApi(api_client)
-    name_4 = 'name_4_example' # str | Required. The name of the Model to retrieve.
+    name_5 = 'name_5_example' # str | Required. The name of the Model to retrieve.
 
     try:
-        api_response = api_instance.model_service_get_model(name_4)
+        api_response = api_instance.model_service_get_model(name_5)
         print("The response of ModelServiceApi->model_service_get_model:\n")
         pprint(api_response)
     except Exception as e:
@@ -391,7 +391,7 @@ with eval_studio_client.api.ApiClient(configuration) as api_client:
 
 Name | Type | Description  | Notes
 ------------- | ------------- | ------------- | -------------
- **name_4** | **str**| Required. The name of the Model to retrieve. | 
+ **name_5** | **str**| Required. The name of the Model to retrieve. | 
 
 ### Return type
 

eval_studio_client/api/docs/OperationServiceApi.md

@@ -4,13 +4,80 @@ All URIs are relative to *http://localhost*
 
 Method | HTTP request | Description
 ------------- | ------------- | -------------
+[**operation_service_abort_operation**](OperationServiceApi.md#operation_service_abort_operation) | **POST** /v1/{name}:abort | 
 [**operation_service_batch_get_operations**](OperationServiceApi.md#operation_service_batch_get_operations) | **GET** /v1/operations:batchGet | 
 [**operation_service_finalize_operation**](OperationServiceApi.md#operation_service_finalize_operation) | **PATCH** /v1/{operation.name}:finalize | 
-[**operation_service_get_operation**](OperationServiceApi.md#operation_service_get_operation) | **GET** /v1/{name_5} | 
+[**operation_service_get_operation**](OperationServiceApi.md#operation_service_get_operation) | **GET** /v1/{name_6} | 
 [**operation_service_list_operations**](OperationServiceApi.md#operation_service_list_operations) | **GET** /v1/operations | 
 [**operation_service_update_operation**](OperationServiceApi.md#operation_service_update_operation) | **PATCH** /v1/{operation.name} | 
 
 
+# **operation_service_abort_operation**
+> V1AbortOperationResponse operation_service_abort_operation(name)
+
+
+
+### Example
+
+
+```python
+import eval_studio_client.api
+from eval_studio_client.api.models.v1_abort_operation_response import V1AbortOperationResponse
+from eval_studio_client.api.rest import ApiException
+from pprint import pprint
+
+# Defining the host is optional and defaults to http://localhost
+# See configuration.py for a list of all supported configuration parameters.
+configuration = eval_studio_client.api.Configuration(
+    host = "http://localhost"
+)
+
+
+# Enter a context with an instance of the API client
+with eval_studio_client.api.ApiClient(configuration) as api_client:
+    # Create an instance of the API class
+    api_instance = eval_studio_client.api.OperationServiceApi(api_client)
+    name = 'name_example' # str | Required. The name of the Operation to abort.
+
+    try:
+        api_response = api_instance.operation_service_abort_operation(name)
+        print("The response of OperationServiceApi->operation_service_abort_operation:\n")
+        pprint(api_response)
+    except Exception as e:
+        print("Exception when calling OperationServiceApi->operation_service_abort_operation: %s\n" % e)
+```
+
+
+
+### Parameters
+
+
+Name | Type | Description  | Notes
+------------- | ------------- | ------------- | -------------
+ **name** | **str**| Required. The name of the Operation to abort. | 
+
+### Return type
+
+[**V1AbortOperationResponse**](V1AbortOperationResponse.md)
+
+### Authorization
+
+No authorization required
+
+### HTTP request headers
+
+ - **Content-Type**: Not defined
+ - **Accept**: application/json
+
+### HTTP response details
+
+| Status code | Description | Response headers |
+|-------------|-------------|------------------|
+**200** | A successful response. |  -  |
+**0** | An unexpected error response. |  -  |
+
+[[Back to top]](#) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to Model list]](../README.md#documentation-for-models) [[Back to README]](../README.md)
+
 # **operation_service_batch_get_operations**
 > V1BatchGetOperationsResponse operation_service_batch_get_operations(names=names)
 
@@ -147,7 +214,7 @@ No authorization required
 [[Back to top]](#) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to Model list]](../README.md#documentation-for-models) [[Back to README]](../README.md)
 
 # **operation_service_get_operation**
-> V1GetOperationResponse operation_service_get_operation(name_5)
+> V1GetOperationResponse operation_service_get_operation(name_6)
 
 
 
@@ -171,10 +238,10 @@ configuration = eval_studio_client.api.Configuration(
 with eval_studio_client.api.ApiClient(configuration) as api_client:
     # Create an instance of the API class
     api_instance = eval_studio_client.api.OperationServiceApi(api_client)
-    name_5 = 'name_5_example' # str | Required. The name of the Operation to retrieve.
+    name_6 = 'name_6_example' # str | Required. The name of the Operation to retrieve.
 
     try:
-        api_response = api_instance.operation_service_get_operation(name_5)
+        api_response = api_instance.operation_service_get_operation(name_6)
         print("The response of OperationServiceApi->operation_service_get_operation:\n")
         pprint(api_response)
     except Exception as e:
@@ -188,7 +255,7 @@ with eval_studio_client.api.ApiClient(configuration) as api_client:
 
 Name | Type | Description  | Notes
 ------------- | ------------- | ------------- | -------------
- **name_5** | **str**| Required. The name of the Operation to retrieve. | 
+ **name_6** | **str**| Required. The name of the Operation to retrieve. | 
 
 ### Return type
 

eval_studio_client/api/docs/PerturbationServiceCreatePerturbationRequest.md

@@ -8,6 +8,7 @@ Name | Type | Description | Notes
 **perturbator_configurations** | [**List[V1PerturbatorConfiguration]**](V1PerturbatorConfiguration.md) | Required. PerturbatorConfiguration to apply to the parent Test. | [optional] 
 **test_cases** | [**List[V1TestCase]**](V1TestCase.md) | Required. List of test cases to perturbate. These are the test cases from the parent test.  TODO: breaks https://google.aip.dev/144 | [optional] 
 **test_case_relationships** | [**List[V1TestCaseRelationship]**](V1TestCaseRelationship.md) | Optional. List of relationships between test cases. | [optional] 
+**default_h2ogpte_model** | [**V1Model**](V1Model.md) |  | [optional] 
 
 ## Example
 

eval_studio_client/api/docs/PerturbatorServiceApi.md

@@ -4,12 +4,12 @@ All URIs are relative to *http://localhost*
 
 Method | HTTP request | Description
 ------------- | ------------- | -------------
-[**perturbator_service_get_perturbator**](PerturbatorServiceApi.md#perturbator_service_get_perturbator) | **GET** /v1/{name_6} | 
+[**perturbator_service_get_perturbator**](PerturbatorServiceApi.md#perturbator_service_get_perturbator) | **GET** /v1/{name_7} | 
 [**perturbator_service_list_perturbators**](PerturbatorServiceApi.md#perturbator_service_list_perturbators) | **GET** /v1/perturbators | 
 
 
 # **perturbator_service_get_perturbator**
-> V1GetPerturbatorResponse perturbator_service_get_perturbator(name_6)
+> V1GetPerturbatorResponse perturbator_service_get_perturbator(name_7)
 
 
 
@@ -33,10 +33,10 @@ configuration = eval_studio_client.api.Configuration(
 with eval_studio_client.api.ApiClient(configuration) as api_client:
     # Create an instance of the API class
     api_instance = eval_studio_client.api.PerturbatorServiceApi(api_client)
-    name_6 = 'name_6_example' # str | Required. The name of the Perturbator to retrieve.
+    name_7 = 'name_7_example' # str | Required. The name of the Perturbator to retrieve.
 
     try:
-        api_response = api_instance.perturbator_service_get_perturbator(name_6)
+        api_response = api_instance.perturbator_service_get_perturbator(name_7)
         print("The response of PerturbatorServiceApi->perturbator_service_get_perturbator:\n")
         pprint(api_response)
     except Exception as e:
@@ -50,7 +50,7 @@ with eval_studio_client.api.ApiClient(configuration) as api_client:
 
 Name | Type | Description  | Notes
 ------------- | ------------- | ------------- | -------------
- **name_6** | **str**| Required. The name of the Perturbator to retrieve. | 
+ **name_7** | **str**| Required. The name of the Perturbator to retrieve. | 
 
 ### Return type
 
@@ -75,7 +75,7 @@ No authorization required
 [[Back to top]](#) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to Model list]](../README.md#documentation-for-models) [[Back to README]](../README.md)
 
 # **perturbator_service_list_perturbators**
-> V1ListPerturbatorsResponse perturbator_service_list_perturbators()
+> V1ListPerturbatorsResponse perturbator_service_list_perturbators(default_h2ogpte_model_name=default_h2ogpte_model_name, default_h2ogpte_model_create_time=default_h2ogpte_model_create_time, default_h2ogpte_model_creator=default_h2ogpte_model_creator, default_h2ogpte_model_update_time=default_h2ogpte_model_update_time, default_h2ogpte_model_updater=default_h2ogpte_model_updater, default_h2ogpte_model_delete_time=default_h2ogpte_model_delete_time, default_h2ogpte_model_deleter=default_h2ogpte_model_deleter, default_h2ogpte_model_display_name=default_h2ogpte_model_display_name, default_h2ogpte_model_description=default_h2ogpte_model_description, default_h2ogpte_model_url=default_h2ogpte_model_url, default_h2ogpte_model_api_key=default_h2ogpte_model_api_key, default_h2ogpte_model_type=default_h2ogpte_model_type, default_h2ogpte_model_parameters=default_h2ogpte_model_parameters, default_h2ogpte_model_demo=default_h2ogpte_model_demo)
 
 
 
@@ -99,9 +99,23 @@ configuration = eval_studio_client.api.Configuration(
 with eval_studio_client.api.ApiClient(configuration) as api_client:
     # Create an instance of the API class
     api_instance = eval_studio_client.api.PerturbatorServiceApi(api_client)
+    default_h2ogpte_model_name = 'default_h2ogpte_model_name_example' # str | Output only. Name of the Model resource. e.g.: \"models/<UUID>\" (optional)
+    default_h2ogpte_model_create_time = '2013-10-20T19:20:30+01:00' # datetime | Output only. Timestamp when the Model was created. (optional)
+    default_h2ogpte_model_creator = 'default_h2ogpte_model_creator_example' # str | Output only. Name of the user or service that requested creation of the Model. (optional)
+    default_h2ogpte_model_update_time = '2013-10-20T19:20:30+01:00' # datetime | Output only. Optional. Timestamp when the Model was last updated. (optional)
+    default_h2ogpte_model_updater = 'default_h2ogpte_model_updater_example' # str | Output only. Optional. Name of the user or service that requested update of the Model. (optional)
+    default_h2ogpte_model_delete_time = '2013-10-20T19:20:30+01:00' # datetime | Output only. Optional. Set when the Model is deleted. When set Model should be considered as deleted. (optional)
+    default_h2ogpte_model_deleter = 'default_h2ogpte_model_deleter_example' # str | Output only. Optional. Name of the user or service that requested deletion of the Model. (optional)
+    default_h2ogpte_model_display_name = 'default_h2ogpte_model_display_name_example' # str | Human readable name of the Model. (optional)
+    default_h2ogpte_model_description = 'default_h2ogpte_model_description_example' # str | Optional. Arbitrary description of the Model. (optional)
+    default_h2ogpte_model_url = 'default_h2ogpte_model_url_example' # str | Optional. Immutable. Absolute URL to the Model. (optional)
+    default_h2ogpte_model_api_key = 'default_h2ogpte_model_api_key_example' # str | Optional. API key used to access the Model. Not set for read calls (i.e. get, list) by public clients (front-end). Set only for internal (server-to-worker) communication. (optional)
+    default_h2ogpte_model_type = 'MODEL_TYPE_UNSPECIFIED' # str | Immutable. Type of this Model.   - MODEL_TYPE_UNSPECIFIED: Unspecified type.  - MODEL_TYPE_H2OGPTE_RAG: h2oGPTe RAG.  - MODEL_TYPE_OPENAI_RAG: OpenAI Assistant RAG.  - MODEL_TYPE_H2OGPTE_LLM: h2oGPTe LLM.  - MODEL_TYPE_H2OGPT_LLM: h2oGPT LLM.  - MODEL_TYPE_OPENAI_CHAT: OpenAI chat.  - MODEL_TYPE_AZURE_OPENAI_CHAT: Microsoft Azure hosted OpenAI Chat.  - MODEL_TYPE_OPENAI_API_CHAT: OpenAI API chat.  - MODEL_TYPE_H2OLLMOPS: H2O LLMOps.  - MODEL_TYPE_OLLAMA: Ollama.  - MODEL_TYPE_AMAZON_BEDROCK: Amazon Bedrock. (optional) (default to 'MODEL_TYPE_UNSPECIFIED')
+    default_h2ogpte_model_parameters = 'default_h2ogpte_model_parameters_example' # str | Optional. Model specific parameters in JSON format. (optional)
+    default_h2ogpte_model_demo = True # bool | Output only. Whether the Model is a demo resource or not. Demo resources are read only. (optional)
 
     try:
-        api_response = api_instance.perturbator_service_list_perturbators()
+        api_response = api_instance.perturbator_service_list_perturbators(default_h2ogpte_model_name=default_h2ogpte_model_name, default_h2ogpte_model_create_time=default_h2ogpte_model_create_time, default_h2ogpte_model_creator=default_h2ogpte_model_creator, default_h2ogpte_model_update_time=default_h2ogpte_model_update_time, default_h2ogpte_model_updater=default_h2ogpte_model_updater, default_h2ogpte_model_delete_time=default_h2ogpte_model_delete_time, default_h2ogpte_model_deleter=default_h2ogpte_model_deleter, default_h2ogpte_model_display_name=default_h2ogpte_model_display_name, default_h2ogpte_model_description=default_h2ogpte_model_description, default_h2ogpte_model_url=default_h2ogpte_model_url, default_h2ogpte_model_api_key=default_h2ogpte_model_api_key, default_h2ogpte_model_type=default_h2ogpte_model_type, default_h2ogpte_model_parameters=default_h2ogpte_model_parameters, default_h2ogpte_model_demo=default_h2ogpte_model_demo)
         print("The response of PerturbatorServiceApi->perturbator_service_list_perturbators:\n")
         pprint(api_response)
     except Exception as e:
@@ -112,7 +126,23 @@ with eval_studio_client.api.ApiClient(configuration) as api_client:
 
 ### Parameters
 
-This endpoint does not need any parameter.
+
+Name | Type | Description  | Notes
+------------- | ------------- | ------------- | -------------
+ **default_h2ogpte_model_name** | **str**| Output only. Name of the Model resource. e.g.: \&quot;models/&lt;UUID&gt;\&quot; | [optional] 
+ **default_h2ogpte_model_create_time** | **datetime**| Output only. Timestamp when the Model was created. | [optional] 
+ **default_h2ogpte_model_creator** | **str**| Output only. Name of the user or service that requested creation of the Model. | [optional] 
+ **default_h2ogpte_model_update_time** | **datetime**| Output only. Optional. Timestamp when the Model was last updated. | [optional] 
+ **default_h2ogpte_model_updater** | **str**| Output only. Optional. Name of the user or service that requested update of the Model. | [optional] 
+ **default_h2ogpte_model_delete_time** | **datetime**| Output only. Optional. Set when the Model is deleted. When set Model should be considered as deleted. | [optional] 
+ **default_h2ogpte_model_deleter** | **str**| Output only. Optional. Name of the user or service that requested deletion of the Model. | [optional] 
+ **default_h2ogpte_model_display_name** | **str**| Human readable name of the Model. | [optional] 
+ **default_h2ogpte_model_description** | **str**| Optional. Arbitrary description of the Model. | [optional] 
+ **default_h2ogpte_model_url** | **str**| Optional. Immutable. Absolute URL to the Model. | [optional] 
+ **default_h2ogpte_model_api_key** | **str**| Optional. API key used to access the Model. Not set for read calls (i.e. get, list) by public clients (front-end). Set only for internal (server-to-worker) communication. | [optional] 
+ **default_h2ogpte_model_type** | **str**| Immutable. Type of this Model.   - MODEL_TYPE_UNSPECIFIED: Unspecified type.  - MODEL_TYPE_H2OGPTE_RAG: h2oGPTe RAG.  - MODEL_TYPE_OPENAI_RAG: OpenAI Assistant RAG.  - MODEL_TYPE_H2OGPTE_LLM: h2oGPTe LLM.  - MODEL_TYPE_H2OGPT_LLM: h2oGPT LLM.  - MODEL_TYPE_OPENAI_CHAT: OpenAI chat.  - MODEL_TYPE_AZURE_OPENAI_CHAT: Microsoft Azure hosted OpenAI Chat.  - MODEL_TYPE_OPENAI_API_CHAT: OpenAI API chat.  - MODEL_TYPE_H2OLLMOPS: H2O LLMOps.  - MODEL_TYPE_OLLAMA: Ollama.  - MODEL_TYPE_AMAZON_BEDROCK: Amazon Bedrock. | [optional] [default to &#39;MODEL_TYPE_UNSPECIFIED&#39;]
+ **default_h2ogpte_model_parameters** | **str**| Optional. Model specific parameters in JSON format. | [optional] 
+ **default_h2ogpte_model_demo** | **bool**| Output only. Whether the Model is a demo resource or not. Demo resources are read only. | [optional] 
 
 ### Return type
 

eval_studio_client/api/docs/PromptGenerationServiceAutoGeneratePromptsRequest.md

@@ -9,9 +9,11 @@ Name | Type | Description | Notes
 **model** | [**V1Model**](V1Model.md) |  | [optional] 
 **count** | **int** | Required. The number of TestCases to generate. | [optional] 
 **base_llm_model** | **str** | Required. Base LLM model to use for generating the prompts. | [optional] 
-**document_urls** | **List[str]** | Optional. The list of document URLs. The document URL might be a managed document URL or a public URL. | [optional] 
-**generators** | [**List[V1TestCasesGenerator]**](V1TestCasesGenerator.md) | Optional. Topics to generate TestCases for. If not specified, all topics are selected. | [optional] 
+**document_urls** | [**V1RepeatedString**](V1RepeatedString.md) |  | [optional] 
+**chunks** | [**V1RepeatedContext**](V1RepeatedContext.md) |  | [optional] 
+**generators** | [**List[V1TestCasesGenerator]**](V1TestCasesGenerator.md) | Optional. Type of questions to generate TestCases for. If not specified, all types of questions are selected. | [optional] 
 **h2ogpte_collection_id** | **str** | Optional. The ID of the h2oGPTe collection to use. If empty, new temporary collection will be created. | [optional] 
+**topics** | **List[str]** | Optional. Topics to generate questions for. If not specified, use document summarization as topic generation. | [optional] 
 
 ## Example