traigent-schema 4.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- traigent_schema-4.1.0/.gitignore +97 -0
- traigent_schema-4.1.0/CHANGELOG.md +204 -0
- traigent_schema-4.1.0/LICENSE +661 -0
- traigent_schema-4.1.0/NOTICE +19 -0
- traigent_schema-4.1.0/PKG-INFO +234 -0
- traigent_schema-4.1.0/README.md +199 -0
- traigent_schema-4.1.0/pyproject.toml +84 -0
- traigent_schema-4.1.0/traigent_schema/__init__.py +37 -0
- traigent_schema-4.1.0/traigent_schema/analytics_validators.py +226 -0
- traigent_schema-4.1.0/traigent_schema/schemas/agents/agent_deployment_schema.json +221 -0
- traigent_schema-4.1.0/traigent_schema/schemas/agents/agent_response_schema.json +188 -0
- traigent_schema-4.1.0/traigent_schema/schemas/agents/agent_schema.json +152 -0
- traigent_schema-4.1.0/traigent_schema/schemas/agents/agent_types_schema.json +31 -0
- traigent_schema-4.1.0/traigent_schema/schemas/agents/agents_endpoints.json +232 -0
- traigent_schema-4.1.0/traigent_schema/schemas/agents/model_parameters_schema.json +120 -0
- traigent_schema-4.1.0/traigent_schema/schemas/agents/model_schema.json +224 -0
- traigent_schema-4.1.0/traigent_schema/schemas/agents/retriever_schema.json +125 -0
- traigent_schema-4.1.0/traigent_schema/schemas/analytics/analytics_endpoints.json +116 -0
- traigent_schema-4.1.0/traigent_schema/schemas/analytics/dataset_quality_schema.json +159 -0
- traigent_schema-4.1.0/traigent_schema/schemas/analytics/example_score_schema.json +124 -0
- traigent_schema-4.1.0/traigent_schema/schemas/analytics/scoring_job_status_schema.json +87 -0
- traigent_schema-4.1.0/traigent_schema/schemas/datasets/dataset_schema.json +12 -0
- traigent_schema-4.1.0/traigent_schema/schemas/datasets/datasets_endpoints.json +123 -0
- traigent_schema-4.1.0/traigent_schema/schemas/datasets/evaluation_set_schema.json +247 -0
- traigent_schema-4.1.0/traigent_schema/schemas/datasets/evaluator_config_schema.json +131 -0
- traigent_schema-4.1.0/traigent_schema/schemas/datasets/example_set_schema.json +69 -0
- traigent_schema-4.1.0/traigent_schema/schemas/datasets/generator_config_schema.json +128 -0
- traigent_schema-4.1.0/traigent_schema/schemas/evaluation/configuration_run_schema.json +182 -0
- traigent_schema-4.1.0/traigent_schema/schemas/evaluation/evaluation_request_schema.json +39 -0
- traigent_schema-4.1.0/traigent_schema/schemas/evaluation/evaluation_results_schema.json +135 -0
- traigent_schema-4.1.0/traigent_schema/schemas/evaluation/evaluation_schema.json +137 -0
- traigent_schema-4.1.0/traigent_schema/schemas/evaluation/experiment_run_schema.json +517 -0
- traigent_schema-4.1.0/traigent_schema/schemas/evaluation/experiment_schema.json +246 -0
- traigent_schema-4.1.0/traigent_schema/schemas/execution/dataset_storage_schema.json +233 -0
- traigent_schema-4.1.0/traigent_schema/schemas/execution/execution_endpoints.json +458 -0
- traigent_schema-4.1.0/traigent_schema/schemas/execution/execution_mode_schema.json +171 -0
- traigent_schema-4.1.0/traigent_schema/schemas/execution/hybrid_session_schema.json +183 -0
- traigent_schema-4.1.0/traigent_schema/schemas/execution/metric_submission_schema.json +135 -0
- traigent_schema-4.1.0/traigent_schema/schemas/execution/saas_execution_schema.json +318 -0
- traigent_schema-4.1.0/traigent_schema/schemas/execution/service_registration_schema.json +64 -0
- traigent_schema-4.1.0/traigent_schema/schemas/execution/workflow_metadata_schema.json +140 -0
- traigent_schema-4.1.0/traigent_schema/schemas/measures/measure_schema.json +249 -0
- traigent_schema-4.1.0/traigent_schema/schemas/measures/measures_endpoints.json +95 -0
- traigent_schema-4.1.0/traigent_schema/schemas/measures/score_schema.json +278 -0
- traigent_schema-4.1.0/traigent_schema/schemas/mep.json +26 -0
- traigent_schema-4.1.0/traigent_schema/schemas/mep_endpoints.json +51 -0
- traigent_schema-4.1.0/traigent_schema/schemas/observability/annotation_queue_create_request_schema.json +27 -0
- traigent_schema-4.1.0/traigent_schema/schemas/observability/annotation_queue_item_complete_result_schema.json +16 -0
- traigent_schema-4.1.0/traigent_schema/schemas/observability/annotation_queue_item_create_request_schema.json +33 -0
- traigent_schema-4.1.0/traigent_schema/schemas/observability/annotation_queue_item_create_result_schema.json +16 -0
- traigent_schema-4.1.0/traigent_schema/schemas/observability/annotation_queue_item_list_response_schema.json +15 -0
- traigent_schema-4.1.0/traigent_schema/schemas/observability/annotation_queue_item_schema.json +41 -0
- traigent_schema-4.1.0/traigent_schema/schemas/observability/annotation_queue_list_response_schema.json +15 -0
- traigent_schema-4.1.0/traigent_schema/schemas/observability/annotation_queue_schema.json +35 -0
- traigent_schema-4.1.0/traigent_schema/schemas/observability/annotation_queue_update_request_schema.json +16 -0
- traigent_schema-4.1.0/traigent_schema/schemas/observability/correlation_ids_schema.json +12 -0
- traigent_schema-4.1.0/traigent_schema/schemas/observability/evaluator_backfill_result_schema.json +30 -0
- traigent_schema-4.1.0/traigent_schema/schemas/observability/evaluator_definition_schema.json +47 -0
- traigent_schema-4.1.0/traigent_schema/schemas/observability/evaluator_execute_request_schema.json +21 -0
- traigent_schema-4.1.0/traigent_schema/schemas/observability/evaluator_list_response_schema.json +15 -0
- traigent_schema-4.1.0/traigent_schema/schemas/observability/evaluator_run_list_response_schema.json +15 -0
- traigent_schema-4.1.0/traigent_schema/schemas/observability/evaluator_run_schema.json +47 -0
- traigent_schema-4.1.0/traigent_schema/schemas/observability/evaluator_update_request_schema.json +20 -0
- traigent_schema-4.1.0/traigent_schema/schemas/observability/observability_endpoints.json +703 -0
- traigent_schema-4.1.0/traigent_schema/schemas/observability/observability_ingest_request_schema.json +16 -0
- traigent_schema-4.1.0/traigent_schema/schemas/observability/observation_schema.json +33 -0
- traigent_schema-4.1.0/traigent_schema/schemas/observability/prompt_link_schema.json +20 -0
- traigent_schema-4.1.0/traigent_schema/schemas/observability/prompt_reference_schema.json +18 -0
- traigent_schema-4.1.0/traigent_schema/schemas/observability/review_measure_summary_schema.json +28 -0
- traigent_schema-4.1.0/traigent_schema/schemas/observability/review_score_create_request_schema.json +103 -0
- traigent_schema-4.1.0/traigent_schema/schemas/observability/review_score_list_response_schema.json +22 -0
- traigent_schema-4.1.0/traigent_schema/schemas/observability/review_score_schema.json +48 -0
- traigent_schema-4.1.0/traigent_schema/schemas/observability/session_list_response_schema.json +17 -0
- traigent_schema-4.1.0/traigent_schema/schemas/observability/session_schema.json +21 -0
- traigent_schema-4.1.0/traigent_schema/schemas/observability/trace_collaboration_state_schema.json +13 -0
- traigent_schema-4.1.0/traigent_schema/schemas/observability/trace_collaboration_update_request_schema.json +15 -0
- traigent_schema-4.1.0/traigent_schema/schemas/observability/trace_comment_create_request_schema.json +11 -0
- traigent_schema-4.1.0/traigent_schema/schemas/observability/trace_comment_schema.json +17 -0
- traigent_schema-4.1.0/traigent_schema/schemas/observability/trace_comments_response_schema.json +15 -0
- traigent_schema-4.1.0/traigent_schema/schemas/observability/trace_feedback_response_schema.json +8 -0
- traigent_schema-4.1.0/traigent_schema/schemas/observability/trace_feedback_schema.json +18 -0
- traigent_schema-4.1.0/traigent_schema/schemas/observability/trace_feedback_upsert_request_schema.json +13 -0
- traigent_schema-4.1.0/traigent_schema/schemas/observability/trace_list_response_schema.json +17 -0
- traigent_schema-4.1.0/traigent_schema/schemas/observability/trace_observations_response_schema.json +15 -0
- traigent_schema-4.1.0/traigent_schema/schemas/observability/trace_schema.json +41 -0
- traigent_schema-4.1.0/traigent_schema/schemas/observability/trace_score_summary_schema.json +33 -0
- traigent_schema-4.1.0/traigent_schema/schemas/observability/user_summary_schema.json +21 -0
- traigent_schema-4.1.0/traigent_schema/schemas/optimization/config_space_schema.json +105 -0
- traigent_schema-4.1.0/traigent_schema/schemas/optimization/exploration_config_schema.json +95 -0
- traigent_schema-4.1.0/traigent_schema/schemas/optimization/objective_definition_schema.json +71 -0
- traigent_schema-4.1.0/traigent_schema/schemas/optimization/optimization_endpoints.json +280 -0
- traigent_schema-4.1.0/traigent_schema/schemas/optimization/promotion_policy_schema.json +82 -0
- traigent_schema-4.1.0/traigent_schema/schemas/optimization/tunable_schema.json +50 -0
- traigent_schema-4.1.0/traigent_schema/schemas/optimization/tvar_definition_schema.json +83 -0
- traigent_schema-4.1.0/traigent_schema/schemas/optimization/typed_constraints_schema.json +26 -0
- traigent_schema-4.1.0/traigent_schema/schemas/pagination_schema.json +46 -0
- traigent_schema-4.1.0/traigent_schema/schemas/planned_projects_endpoints.json +865 -0
- traigent_schema-4.1.0/traigent_schema/schemas/planner/planner_draft_schema.json +52 -0
- traigent_schema-4.1.0/traigent_schema/schemas/projects/project_context_error_schema.json +28 -0
- traigent_schema-4.1.0/traigent_schema/schemas/projects/project_create_request_schema.json +30 -0
- traigent_schema-4.1.0/traigent_schema/schemas/projects/project_export_job_list_response_schema.json +46 -0
- traigent_schema-4.1.0/traigent_schema/schemas/projects/project_export_job_schema.json +92 -0
- traigent_schema-4.1.0/traigent_schema/schemas/projects/project_export_policy_schema.json +43 -0
- traigent_schema-4.1.0/traigent_schema/schemas/projects/project_export_policy_update_request_schema.json +14 -0
- traigent_schema-4.1.0/traigent_schema/schemas/projects/project_list_response_schema.json +19 -0
- traigent_schema-4.1.0/traigent_schema/schemas/projects/project_membership_create_request_schema.json +25 -0
- traigent_schema-4.1.0/traigent_schema/schemas/projects/project_membership_list_response_schema.json +21 -0
- traigent_schema-4.1.0/traigent_schema/schemas/projects/project_membership_schema.json +71 -0
- traigent_schema-4.1.0/traigent_schema/schemas/projects/project_membership_update_request_schema.json +21 -0
- traigent_schema-4.1.0/traigent_schema/schemas/projects/project_rate_limit_policy_schema.json +59 -0
- traigent_schema-4.1.0/traigent_schema/schemas/projects/project_rate_limit_policy_update_request_schema.json +27 -0
- traigent_schema-4.1.0/traigent_schema/schemas/projects/project_retention_policy_schema.json +51 -0
- traigent_schema-4.1.0/traigent_schema/schemas/projects/project_retention_policy_update_request_schema.json +21 -0
- traigent_schema-4.1.0/traigent_schema/schemas/projects/project_schema.json +67 -0
- traigent_schema-4.1.0/traigent_schema/schemas/projects/project_scoped_analytics_summary_schema.json +246 -0
- traigent_schema-4.1.0/traigent_schema/schemas/projects/project_scoped_analytics_trend_schema.json +100 -0
- traigent_schema-4.1.0/traigent_schema/schemas/projects/project_scoped_core_metrics_overview_schema.json +81 -0
- traigent_schema-4.1.0/traigent_schema/schemas/projects/project_scoped_evaluator_quality_dashboard_schema.json +125 -0
- traigent_schema-4.1.0/traigent_schema/schemas/projects/project_scoped_experiment_trend_schema.json +95 -0
- traigent_schema-4.1.0/traigent_schema/schemas/projects/project_scoped_fine_tuning_export_query_schema.json +27 -0
- traigent_schema-4.1.0/traigent_schema/schemas/projects/project_scoped_fine_tuning_manifest_schema.json +106 -0
- traigent_schema-4.1.0/traigent_schema/schemas/projects/project_scoped_measure_distribution_schema.json +94 -0
- traigent_schema-4.1.0/traigent_schema/schemas/projects/project_scoped_observability_summary_dashboard_schema.json +132 -0
- traigent_schema-4.1.0/traigent_schema/schemas/projects/project_scoped_optimization_overview_dashboard_schema.json +146 -0
- traigent_schema-4.1.0/traigent_schema/schemas/projects/project_scoped_pricing_catalog_schema.json +119 -0
- traigent_schema-4.1.0/traigent_schema/schemas/projects/project_scoped_project_usage_dashboard_schema.json +122 -0
- traigent_schema-4.1.0/traigent_schema/schemas/projects/project_update_request_schema.json +29 -0
- traigent_schema-4.1.0/traigent_schema/schemas/prompts/chat_prompt_message_schema.json +13 -0
- traigent_schema-4.1.0/traigent_schema/schemas/prompts/prompt_analytics_schema.json +24 -0
- traigent_schema-4.1.0/traigent_schema/schemas/prompts/prompt_create_request_schema.json +82 -0
- traigent_schema-4.1.0/traigent_schema/schemas/prompts/prompt_detail_schema.json +18 -0
- traigent_schema-4.1.0/traigent_schema/schemas/prompts/prompt_labels_update_request_schema.json +21 -0
- traigent_schema-4.1.0/traigent_schema/schemas/prompts/prompt_playground_request_schema.json +34 -0
- traigent_schema-4.1.0/traigent_schema/schemas/prompts/prompt_playground_result_schema.json +29 -0
- traigent_schema-4.1.0/traigent_schema/schemas/prompts/prompt_summary_schema.json +28 -0
- traigent_schema-4.1.0/traigent_schema/schemas/prompts/prompt_version_create_request_schema.json +36 -0
- traigent_schema-4.1.0/traigent_schema/schemas/prompts/prompt_version_schema.json +33 -0
- traigent_schema-4.1.0/traigent_schema/schemas/prompts/prompts_endpoints.json +226 -0
- traigent_schema-4.1.0/traigent_schema/schemas/prompts/resolved_prompt_schema.json +17 -0
- traigent_schema-4.1.0/traigent_schema/schemas/results/comparison_schema.json +218 -0
- traigent_schema-4.1.0/traigent_schema/schemas/results/report_request_schema.json +277 -0
- traigent_schema-4.1.0/traigent_schema/schemas/results/report_schema.json +277 -0
- traigent_schema-4.1.0/traigent_schema/schemas/results/results_endpoints.json +90 -0
- traigent_schema-4.1.0/traigent_schema/schemas/results/visualization_request_schema.json +228 -0
- traigent_schema-4.1.0/traigent_schema/schemas/results/visualization_schema.json +273 -0
- traigent_schema-4.1.0/traigent_schema/schemas/sdk_tuning_endpoints.json +21 -0
- traigent_schema-4.1.0/traigent_schema/schemas/status_schema.json +105 -0
- traigent_schema-4.1.0/traigent_schema/utils.py +129 -0
- traigent_schema-4.1.0/traigent_schema/validator.py +268 -0
- traigent_schema-4.1.0/traigent_schema/version.py +3 -0
|
@@ -0,0 +1,97 @@
|
|
|
1
|
+
# Byte-compiled / optimized / DLL files
|
|
2
|
+
__pycache__/
|
|
3
|
+
*.py[cod]
|
|
4
|
+
*$py.class
|
|
5
|
+
|
|
6
|
+
# C extensions
|
|
7
|
+
*.so
|
|
8
|
+
|
|
9
|
+
# Distribution / packaging
|
|
10
|
+
.Python
|
|
11
|
+
build/
|
|
12
|
+
develop-eggs/
|
|
13
|
+
dist/
|
|
14
|
+
downloads/
|
|
15
|
+
eggs/
|
|
16
|
+
.eggs/
|
|
17
|
+
lib/
|
|
18
|
+
lib64/
|
|
19
|
+
parts/
|
|
20
|
+
sdist/
|
|
21
|
+
var/
|
|
22
|
+
wheels/
|
|
23
|
+
share/python-wheels/
|
|
24
|
+
*.egg-info/
|
|
25
|
+
.installed.cfg
|
|
26
|
+
*.egg
|
|
27
|
+
MANIFEST
|
|
28
|
+
|
|
29
|
+
# PyInstaller
|
|
30
|
+
*.manifest
|
|
31
|
+
*.spec
|
|
32
|
+
|
|
33
|
+
# Installer logs
|
|
34
|
+
pip-log.txt
|
|
35
|
+
pip-delete-this-directory.txt
|
|
36
|
+
|
|
37
|
+
# Unit test / coverage reports
|
|
38
|
+
htmlcov/
|
|
39
|
+
.tox/
|
|
40
|
+
.nox/
|
|
41
|
+
.coverage
|
|
42
|
+
.coverage.*
|
|
43
|
+
.cache
|
|
44
|
+
nosetests.xml
|
|
45
|
+
coverage.xml
|
|
46
|
+
*.cover
|
|
47
|
+
*.py,cover
|
|
48
|
+
.hypothesis/
|
|
49
|
+
.pytest_cache/
|
|
50
|
+
|
|
51
|
+
# Translations
|
|
52
|
+
*.mo
|
|
53
|
+
*.pot
|
|
54
|
+
|
|
55
|
+
# Environments
|
|
56
|
+
.env
|
|
57
|
+
.venv
|
|
58
|
+
env/
|
|
59
|
+
venv/
|
|
60
|
+
ENV/
|
|
61
|
+
env.bak/
|
|
62
|
+
venv.bak/
|
|
63
|
+
|
|
64
|
+
# IDEs
|
|
65
|
+
.idea/
|
|
66
|
+
.vscode/
|
|
67
|
+
*.swp
|
|
68
|
+
*.swo
|
|
69
|
+
*~
|
|
70
|
+
|
|
71
|
+
# Jupyter Notebook
|
|
72
|
+
.ipynb_checkpoints
|
|
73
|
+
|
|
74
|
+
# pyenv
|
|
75
|
+
.python-version
|
|
76
|
+
|
|
77
|
+
# mypy
|
|
78
|
+
.mypy_cache/
|
|
79
|
+
.dmypy.json
|
|
80
|
+
dmypy.json
|
|
81
|
+
|
|
82
|
+
# Ruff
|
|
83
|
+
.ruff_cache/
|
|
84
|
+
|
|
85
|
+
# OS files
|
|
86
|
+
.DS_Store
|
|
87
|
+
Thumbs.db
|
|
88
|
+
|
|
89
|
+
# Project specific
|
|
90
|
+
*.log
|
|
91
|
+
.claude/
|
|
92
|
+
AGENTS.md
|
|
93
|
+
CLAUDE.md
|
|
94
|
+
|
|
95
|
+
# SonarQube
|
|
96
|
+
.scannerwork/
|
|
97
|
+
sonar-project.local.properties
|
|
@@ -0,0 +1,204 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to this project will be documented in this file.
|
|
4
|
+
|
|
5
|
+
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
|
6
|
+
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
|
+
|
|
8
|
+
## [4.1.0] - 2026-05-08
|
|
9
|
+
|
|
10
|
+
### Added
|
|
11
|
+
- Added `parity/python-js-sdk.json` as the authoritative Python/JavaScript SDK
|
|
12
|
+
parity manifest, including release-order and refresh-cadence policy.
|
|
13
|
+
- Added manifest classifications for workflow-cost parity symbols used by the
|
|
14
|
+
Python and JavaScript SDKs.
|
|
15
|
+
- Added CI coverage that validates Python root-symbol classifications and
|
|
16
|
+
required JavaScript export coverage against the parity manifest.
|
|
17
|
+
- Added `planner/planner_draft_schema.json` for planner draft contracts.
|
|
18
|
+
|
|
19
|
+
### Changed
|
|
20
|
+
- Published the repository as the public source for `traigent-schema` contracts.
|
|
21
|
+
- Removed local-only agent instruction files and Sonar workspace configuration
|
|
22
|
+
from version control, and ignored those local files going forward.
|
|
23
|
+
- Clarified that `planned_projects` is a pre-release contract surface that may
|
|
24
|
+
change without a major-version bump until it graduates.
|
|
25
|
+
|
|
26
|
+
## [4.0.0] - 2026-03-28
|
|
27
|
+
|
|
28
|
+
### Changed
|
|
29
|
+
- Split endpoint catalogs into three explicit contract roots:
|
|
30
|
+
- `mep_endpoints.json` is now the canonical current-backend contract
|
|
31
|
+
- `sdk_tuning_endpoints.json` is the canonical direct-tuning contract for SDK session and hybrid flows
|
|
32
|
+
- `planned_projects_endpoints.json` preserves the `/api/v1beta/projects/...` surface as planned/experimental
|
|
33
|
+
- `SchemaValidator` now accepts `contract="backend" | "sdk_tuning" | "planned_projects"`, while `get_openapi_path()` remains a backward-compatible alias for the backend contract root.
|
|
34
|
+
- Canonical backend endpoint modules now use explicit `/api/v1/...` paths and only reference current backend route families.
|
|
35
|
+
|
|
36
|
+
### Removed
|
|
37
|
+
- Planned project-scoped routes and legacy tunables/config-space routes are no longer part of the default backend contract.
|
|
38
|
+
|
|
39
|
+
### Fixed
|
|
40
|
+
- Request validation now supports inline OpenAPI request schemas in addition to `$ref`-based mappings, which keeps split contract files simpler without losing validator coverage.
|
|
41
|
+
|
|
42
|
+
## [3.2.2] - 2026-03-28
|
|
43
|
+
|
|
44
|
+
### Changed
|
|
45
|
+
- Relicensed open-source releases of `traigent-schema` to AGPL-3.0-only to align
|
|
46
|
+
with the Traigent SDK release model.
|
|
47
|
+
- Added a release-oriented publish workflow with TestPyPI/PyPI verification.
|
|
48
|
+
- CI now validates build artifacts by building the wheel/sdist, running
|
|
49
|
+
`twine check`, and importing the built wheel in a clean virtual environment.
|
|
50
|
+
|
|
51
|
+
### Fixed
|
|
52
|
+
- README release metadata now matches the package version and describes the
|
|
53
|
+
repository as the shared contract source for backend, SDK, and frontend
|
|
54
|
+
consumers.
|
|
55
|
+
|
|
56
|
+
## [3.2.1] - 2026-03-23
|
|
57
|
+
|
|
58
|
+
### Fixed
|
|
59
|
+
- `GET /api/v1/experiment-runs/runs/{run_id}/traces` no longer requires the legacy `trials` and `trials_pagination` fields alongside the standardized `trials_page` envelope.
|
|
60
|
+
- Added a stable `$id` to `pagination_schema.json` so registry-backed cross-file `$ref` resolution works for paginated response schemas.
|
|
61
|
+
- `SchemaValidator` now loads request mappings from endpoint modules declared in `mep_endpoints.json`, restoring request validation for modular contracts such as agents and optimization.
|
|
62
|
+
- Added the optimization endpoint module to the root `mep_endpoints.json` catalog for discoverability.
|
|
63
|
+
|
|
64
|
+
## [3.2.0] - 2026-03-05
|
|
65
|
+
|
|
66
|
+
### Added
|
|
67
|
+
- Shared pagination contract schema:
|
|
68
|
+
- `pagination_schema.json`: canonical pagination metadata (`page`, `per_page`, `total`, `total_pages`, `has_next`, `has_prev`)
|
|
69
|
+
- New backend-aligned endpoint contracts in `execution_endpoints.json`:
|
|
70
|
+
- `GET /api/v1/keys/accessible-resources`
|
|
71
|
+
- `GET /api/v1/experiment-runs/runs/{run_id}/traces`
|
|
72
|
+
- `GET /api/v1/traces/{trace_id}`
|
|
73
|
+
|
|
74
|
+
### Changed
|
|
75
|
+
- **BREAKING**: Standardized paginated list response envelope for target endpoint modules:
|
|
76
|
+
- `agents/agents_endpoints.json`
|
|
77
|
+
- `measures/measures_endpoints.json`
|
|
78
|
+
- `execution/execution_endpoints.json`
|
|
79
|
+
- `evaluation/evaluation_endpoints.json`
|
|
80
|
+
- **BREAKING**: List endpoints in those modules now model:
|
|
81
|
+
- `success`, `message`, `data.items`, `data.pagination`
|
|
82
|
+
- instead of legacy `data: []` and `count/total` top-level conventions.
|
|
83
|
+
- **BREAKING**: Query pagination parameters were aligned from legacy `limit/offset` to `page/per_page` in the updated list endpoints.
|
|
84
|
+
|
|
85
|
+
### Migration Guide (FE/SDK/Backend Consumers)
|
|
86
|
+
- Request update:
|
|
87
|
+
- Replace `limit`/`offset` with `page`/`per_page`.
|
|
88
|
+
- Response parsing update:
|
|
89
|
+
- **Before**: read list payload from `data` (array), and pagination-like totals from top-level fields.
|
|
90
|
+
- **After**: read list payload from `data.items` and pagination metadata from `data.pagination`.
|
|
91
|
+
- Backward compatibility note:
|
|
92
|
+
- Legacy list contracts are not modeled in this version for the updated endpoints.
|
|
93
|
+
- Consumers should migrate parsing logic to the standardized paginated envelope.
|
|
94
|
+
|
|
95
|
+
## [3.1.0] - 2024-01-17
|
|
96
|
+
|
|
97
|
+
### Added
|
|
98
|
+
- **Analytics Schemas**: New schemas for example and dataset quality scoring
|
|
99
|
+
- `example_score_schema.json`: Per-example quality scores (11 score fields)
|
|
100
|
+
- `dataset_quality_schema.json`: Dataset-level metrics and recommendations
|
|
101
|
+
- `scoring_job_status_schema.json`: Async scoring job status tracking
|
|
102
|
+
- `analytics_endpoints.json`: OpenAPI spec for 4 new endpoints
|
|
103
|
+
- **AnalyticsValidator**: New validation utility class for analytics schemas
|
|
104
|
+
- Comprehensive test suite for analytics schemas (50+ test cases)
|
|
105
|
+
|
|
106
|
+
### Changed
|
|
107
|
+
|
|
108
|
+
- **BREAKING**: Nested Example Metrics Structure (`configuration_run_schema.json`):
|
|
109
|
+
- `measures` array now uses `ExampleMetrics` structure instead of flat `MeasureResults`
|
|
110
|
+
- New structure: `{"example_id": "ex_...", "metrics": {"score": 0.85, ...}}`
|
|
111
|
+
- This prevents `example_id` from being stripped during numeric-only validation
|
|
112
|
+
- Added `maxItems: 1000` constraint on measures array
|
|
113
|
+
- **Workflow Metadata Schema Enhancements** (`workflow_metadata_schema.json`):
|
|
114
|
+
- `AgentCostBreakdown`: Added `agent_id` field (required), renamed `model` to `model_used`, added `total_tokens` field (required)
|
|
115
|
+
- `WorkflowMetadata`: `workflow_name` is now required (was nullable), added `total_tokens`, `total_input_cost`, `total_output_cost` fields (all required)
|
|
116
|
+
|
|
117
|
+
### Added (Validators)
|
|
118
|
+
|
|
119
|
+
- `validate_example_metrics()` method in `AnalyticsValidator` for nested format validation
|
|
120
|
+
- Test suite for `ExampleMetrics` validation (15 test cases)
|
|
121
|
+
|
|
122
|
+
### Migration Guide (Nested Measures)
|
|
123
|
+
|
|
124
|
+
**Before (flat format - v3.0.x)**:
|
|
125
|
+
|
|
126
|
+
```json
|
|
127
|
+
{
|
|
128
|
+
"measures": [
|
|
129
|
+
{"example_id": "ex_abc123_0", "score": 0.85, "cost": 0.05}
|
|
130
|
+
]
|
|
131
|
+
}
|
|
132
|
+
```
|
|
133
|
+
|
|
134
|
+
**After (nested format - v3.1.0)**:
|
|
135
|
+
|
|
136
|
+
```json
|
|
137
|
+
{
|
|
138
|
+
"measures": [
|
|
139
|
+
{
|
|
140
|
+
"example_id": "ex_abc123_0",
|
|
141
|
+
"metrics": {"score": 0.85, "cost": 0.05}
|
|
142
|
+
}
|
|
143
|
+
]
|
|
144
|
+
}
|
|
145
|
+
```
|
|
146
|
+
|
|
147
|
+
## [3.0.0] - 2024-01-15
|
|
148
|
+
|
|
149
|
+
### Added
|
|
150
|
+
- **Workflow Metadata**: New schema for multi-agent workflow cost tracking
|
|
151
|
+
- `workflow_metadata_schema.json`: Tracks execution time, token usage, and costs per agent
|
|
152
|
+
- `AgentCostBreakdown` definition for per-agent metrics
|
|
153
|
+
- `WorkflowMetadata` definition for aggregate workflow metrics
|
|
154
|
+
- `workflow_metadata` field in `configuration_run_schema.json` for SDK integration
|
|
155
|
+
|
|
156
|
+
### Changed
|
|
157
|
+
- **BREAKING**: `MeasureResults` in `configuration_run_schema.json` now enforces:
|
|
158
|
+
- Numeric values only (`["number", "null"]`) - string/boolean no longer accepted
|
|
159
|
+
- Maximum 50 keys per result (`maxProperties: 50`)
|
|
160
|
+
- **BREAKING**: `MetricResults` in `metric_submission_schema.json` aligned with MeasureResults:
|
|
161
|
+
- Numeric values only (`["number", "null"]`)
|
|
162
|
+
- Maximum 50 keys per submission (`maxProperties: 50`)
|
|
163
|
+
- Non-numeric values should be placed in `metadata` field instead
|
|
164
|
+
|
|
165
|
+
### Fixed
|
|
166
|
+
- Removed conflicting `$id` values from `configuration_run_schema.json` and `workflow_metadata_schema.json` to enable proper `$ref` resolution
|
|
167
|
+
- `workflow_metadata_schema.json` root type now correctly validates only `WorkflowMetadata` (not `AgentCostBreakdown`)
|
|
168
|
+
|
|
169
|
+
### Migration Guide
|
|
170
|
+
|
|
171
|
+
#### MeasureResults Contract Change
|
|
172
|
+
If your SDK or backend previously submitted string/boolean measure values:
|
|
173
|
+
|
|
174
|
+
**Before (v2.x)**:
|
|
175
|
+
```python
|
|
176
|
+
measures = {
|
|
177
|
+
"accuracy": 0.95,
|
|
178
|
+
"status": "passed", # String - no longer valid
|
|
179
|
+
"passed": True # Boolean - no longer valid
|
|
180
|
+
}
|
|
181
|
+
```
|
|
182
|
+
|
|
183
|
+
**After (v3.x)**:
|
|
184
|
+
```python
|
|
185
|
+
measures = {
|
|
186
|
+
"accuracy": 0.95,
|
|
187
|
+
"pass_rate": 1.0 # Numeric representation
|
|
188
|
+
}
|
|
189
|
+
# Move non-numeric data to metadata
|
|
190
|
+
metadata = {
|
|
191
|
+
"status": "passed",
|
|
192
|
+
"details": {"passed": True}
|
|
193
|
+
}
|
|
194
|
+
```
|
|
195
|
+
|
|
196
|
+
#### Max Keys Limit
|
|
197
|
+
Ensure measure submissions have no more than 50 keys. If you need more:
|
|
198
|
+
1. Aggregate related measures
|
|
199
|
+
2. Use metadata for non-score information
|
|
200
|
+
3. Split into multiple submissions if necessary
|
|
201
|
+
|
|
202
|
+
## [2.x.x] - Previous Releases
|
|
203
|
+
|
|
204
|
+
See git history for changes prior to v3.0.0.
|