agentevals-cli 0.9.12__tar.gz → 0.10.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (736) hide show
  1. agentevals_cli-0.10.0/.github/workflows/kagent-contract-policy.yml +26 -0
  2. agentevals_cli-0.10.0/.github/workflows/kagent-contract.yml +143 -0
  3. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/CONTRIBUTING.md +17 -25
  4. agentevals_cli-0.10.0/PKG-INFO +309 -0
  5. agentevals_cli-0.10.0/README.md +267 -0
  6. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/charts/agentevals/templates/rbac.yaml +3 -0
  7. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/charts/agentevals/values.yaml +15 -7
  8. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/docs/custom-evaluators.md +37 -99
  9. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/docs/eval-set-format.md +24 -0
  10. agentevals_cli-0.10.0/docs/opentelemetry-pipeline.md +174 -0
  11. agentevals_cli-0.10.0/docs/otel-compatibility.md +122 -0
  12. agentevals_cli-0.10.0/examples/README.md +138 -0
  13. agentevals_cli-0.10.0/examples/dice_agent/main.py +113 -0
  14. agentevals_cli-0.10.0/examples/dice_agent/test_streaming.py +44 -0
  15. agentevals_cli-0.10.0/examples/kubernetes/README.md +198 -0
  16. agentevals_cli-0.10.0/examples/kubernetes/agentevals-values.yaml +8 -0
  17. agentevals_cli-0.10.0/examples/kubernetes/agents.yaml +54 -0
  18. agentevals_cli-0.10.0/examples/kubernetes/otel-collector.yaml +140 -0
  19. agentevals_cli-0.10.0/examples/langchain_agent/README.md +40 -0
  20. agentevals_cli-0.10.0/examples/langchain_agent/main.py +104 -0
  21. agentevals_cli-0.10.0/examples/langchain_agent/test_streaming.py +44 -0
  22. agentevals_cli-0.10.0/examples/strands_agent/main.py +88 -0
  23. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/examples/zero-code-examples/adk/run.py +4 -0
  24. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/examples/zero-code-examples/langchain/run.py +4 -0
  25. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/examples/zero-code-examples/ollama/run.py +4 -0
  26. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/examples/zero-code-examples/openai-agents/requirements.txt +1 -1
  27. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/examples/zero-code-examples/openai-agents/run.py +4 -0
  28. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/examples/zero-code-examples/pydantic-ai/run.py +4 -0
  29. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/examples/zero-code-examples/strands/run.py +4 -0
  30. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/packages/evaluator-sdk-py/src/agentevals_evaluator_sdk/types.py +7 -2
  31. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/pyproject.toml +8 -8
  32. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/_protocol.py +7 -2
  33. agentevals_cli-0.9.12/src/agentevals/_static/assets/index-D-a2xOzq.js → agentevals_cli-0.10.0/src/agentevals/_static/assets/index-DLtCJiNw.js +80 -81
  34. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/_static/index.html +1 -1
  35. agentevals_cli-0.10.0/src/agentevals/adk_bridge.py +144 -0
  36. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/api/app.py +19 -12
  37. agentevals_cli-0.10.0/src/agentevals/api/debug_routes.py +270 -0
  38. agentevals_cli-0.10.0/src/agentevals/api/dependencies.py +28 -0
  39. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/api/models.py +10 -47
  40. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/api/otlp_app.py +6 -2
  41. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/api/otlp_grpc.py +38 -18
  42. agentevals_cli-0.10.0/src/agentevals/api/otlp_http.py +348 -0
  43. agentevals_cli-0.10.0/src/agentevals/api/otlp_processing.py +156 -0
  44. agentevals_cli-0.10.0/src/agentevals/api/otlp_routes.py +59 -0
  45. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/api/routes.py +105 -78
  46. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/api/streaming_routes.py +47 -138
  47. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/builtin_metrics.py +10 -8
  48. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/cli.py +64 -6
  49. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/config.py +67 -45
  50. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/custom_evaluators.py +126 -230
  51. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/eval_config_loader.py +14 -2
  52. agentevals_cli-0.10.0/src/agentevals/evaluation_events.py +352 -0
  53. agentevals_cli-0.10.0/src/agentevals/evaluator/resolver.py +138 -0
  54. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/evaluator/sources.py +20 -7
  55. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/evaluator/templates.py +5 -2
  56. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/evaluator/venv.py +51 -3
  57. agentevals_cli-0.10.0/src/agentevals/genai/__init__.py +1 -0
  58. agentevals_cli-0.10.0/src/agentevals/genai/extract.py +766 -0
  59. agentevals_cli-0.10.0/src/agentevals/genai/grouping.py +70 -0
  60. agentevals_cli-0.10.0/src/agentevals/genai/matching.py +106 -0
  61. agentevals_cli-0.10.0/src/agentevals/genai/messages.py +264 -0
  62. agentevals_cli-0.10.0/src/agentevals/genai/model.py +117 -0
  63. agentevals_cli-0.10.0/src/agentevals/genai/overlay.py +407 -0
  64. agentevals_cli-0.10.0/src/agentevals/genai/routing.py +62 -0
  65. agentevals_cli-0.10.0/src/agentevals/genai/semconv.py +105 -0
  66. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/loader/__init__.py +10 -1
  67. agentevals_cli-0.10.0/src/agentevals/loader/auto.py +147 -0
  68. agentevals_cli-0.10.0/src/agentevals/loader/jaeger.py +175 -0
  69. agentevals_cli-0.10.0/src/agentevals/loader/otlp.py +89 -0
  70. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/mcp_server.py +27 -39
  71. agentevals_cli-0.10.0/src/agentevals/otel/__init__.py +1 -0
  72. agentevals_cli-0.10.0/src/agentevals/otel/decode.py +678 -0
  73. agentevals_cli-0.10.0/src/agentevals/otel/encode.py +190 -0
  74. agentevals_cli-0.10.0/src/agentevals/otel/identity.py +30 -0
  75. agentevals_cli-0.10.0/src/agentevals/otel/model.py +350 -0
  76. agentevals_cli-0.10.0/src/agentevals/otel/sdk_export.py +352 -0
  77. agentevals_cli-0.10.0/src/agentevals/otel/store.py +948 -0
  78. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/run/fetcher.py +5 -24
  79. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/run/result_builder.py +10 -6
  80. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/run/worker.py +5 -3
  81. agentevals_cli-0.10.0/src/agentevals/runner.py +349 -0
  82. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/sdk.py +92 -143
  83. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/storage/models.py +1 -1
  84. agentevals_cli-0.10.0/src/agentevals/storage/postgres/migrations/000002_drop_openai_eval.down.sql +6 -0
  85. agentevals_cli-0.10.0/src/agentevals/storage/postgres/migrations/000002_drop_openai_eval.up.sql +9 -0
  86. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/storage/postgres/migrator.py +7 -1
  87. agentevals_cli-0.10.0/src/agentevals/streaming/__init__.py +67 -0
  88. agentevals_cli-0.10.0/src/agentevals/streaming/manager.py +582 -0
  89. agentevals_cli-0.10.0/src/agentevals/trace_metrics.py +164 -0
  90. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/tests/api/test_evaluate_persistence.py +1 -1
  91. agentevals_cli-0.10.0/tests/api/test_live_routes.py +234 -0
  92. agentevals_cli-0.10.0/tests/contract/kagent/README.md +27 -0
  93. agentevals_cli-0.10.0/tests/contract/kagent/check.sh +59 -0
  94. agentevals_cli-0.10.0/tests/contract/kagent/policy/agentevals.rego +123 -0
  95. agentevals_cli-0.10.0/tests/contract/kagent/registries/main-2cc3ffae/a2a.yaml +64 -0
  96. agentevals_cli-0.10.0/tests/contract/kagent/registries/main-2cc3ffae/imports.yaml +22 -0
  97. agentevals_cli-0.10.0/tests/contract/kagent/registries/main-2cc3ffae/kagent.yaml +114 -0
  98. agentevals_cli-0.10.0/tests/contract/kagent/registries/main-2cc3ffae/manifest.yaml +12 -0
  99. agentevals_cli-0.10.0/tests/contract/kagent/registries/main-2cc3ffae/signals.yaml +247 -0
  100. agentevals_cli-0.10.0/tests/contract/kagent/registries/pr3121-412ef54e/a2a.yaml +64 -0
  101. agentevals_cli-0.10.0/tests/contract/kagent/registries/pr3121-412ef54e/imports.yaml +22 -0
  102. agentevals_cli-0.10.0/tests/contract/kagent/registries/pr3121-412ef54e/kagent.yaml +136 -0
  103. agentevals_cli-0.10.0/tests/contract/kagent/registries/pr3121-412ef54e/manifest.yaml +12 -0
  104. agentevals_cli-0.10.0/tests/contract/kagent/registries/pr3121-412ef54e/signals.yaml +363 -0
  105. agentevals_cli-0.10.0/tests/contract/kagent/registries/v1.0.0-alpha9/a2a.yaml +64 -0
  106. agentevals_cli-0.10.0/tests/contract/kagent/registries/v1.0.0-alpha9/kagent.yaml +114 -0
  107. agentevals_cli-0.10.0/tests/contract/kagent/registries/v1.0.0-alpha9/manifest.yaml +11 -0
  108. agentevals_cli-0.10.0/tests/contract/kagent/registries/v1.0.0-alpha9/signals.yaml +196 -0
  109. agentevals_cli-0.10.0/tests/contract/kagent/test.sh +37 -0
  110. agentevals_cli-0.10.0/tests/contract/kagent/testdata/agentevals_contract_gone/manifest.yaml +8 -0
  111. agentevals_cli-0.10.0/tests/contract/kagent/testdata/agentevals_contract_gone/model.yaml +11 -0
  112. agentevals_cli-0.10.0/tests/contract/kagent/testdata/agentevals_contract_missing_attribute/manifest.yaml +5 -0
  113. agentevals_cli-0.10.0/tests/contract/kagent/testdata/agentevals_contract_missing_attribute/model.yaml +25 -0
  114. agentevals_cli-0.10.0/tests/contract/kagent/testdata/agentevals_contract_requirement_floor/manifest.yaml +8 -0
  115. agentevals_cli-0.10.0/tests/contract/kagent/testdata/agentevals_contract_requirement_floor/model.yaml +14 -0
  116. agentevals_cli-0.10.0/tests/contract/kagent/weaver.sh +15 -0
  117. agentevals_cli-0.10.0/tests/fixtures/otlp/MANIFEST.md +69 -0
  118. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_default/S1.json +398 -0
  119. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_default/S1.truth.json +28 -0
  120. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_default/S2.json +1220 -0
  121. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_default/S2.truth.json +56 -0
  122. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_default/S3.json +1648 -0
  123. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_default/S3.truth.json +68 -0
  124. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_default/S4a.json +1257 -0
  125. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_default/S4a.truth.json +52 -0
  126. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_default/S4b.json +1081 -0
  127. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_default/S4b.truth.json +40 -0
  128. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_default/S5.json +928 -0
  129. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_default/S5.truth.json +55 -0
  130. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_default/S6a.json +612 -0
  131. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_default/S6a.truth.json +39 -0
  132. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_default/S6b.json +791 -0
  133. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_default/S6b.truth.json +40 -0
  134. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_default_contrib/S1.json +465 -0
  135. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_default_contrib/S1.truth.json +29 -0
  136. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_default_contrib/S2.json +1299 -0
  137. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_default_contrib/S2.truth.json +57 -0
  138. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_default_contrib/S3.json +1733 -0
  139. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_default_contrib/S3.truth.json +69 -0
  140. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_default_contrib/S4a.json +1336 -0
  141. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_default_contrib/S4a.truth.json +52 -0
  142. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_default_contrib/S4b.json +1160 -0
  143. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_default_contrib/S4b.truth.json +40 -0
  144. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_default_contrib/S5.json +1001 -0
  145. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_default_contrib/S5.truth.json +56 -0
  146. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_default_contrib/S6a.json +679 -0
  147. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_default_contrib/S6a.truth.json +40 -0
  148. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_default_contrib/S6b.json +864 -0
  149. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_default_contrib/S6b.truth.json +41 -0
  150. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_event_stable/S1.json +560 -0
  151. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_event_stable/S1.truth.json +28 -0
  152. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_event_stable/S2.json +2693 -0
  153. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_event_stable/S2.truth.json +56 -0
  154. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_event_stable/S3.json +3499 -0
  155. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_event_stable/S3.truth.json +68 -0
  156. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_event_stable/S4a.json +2628 -0
  157. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_event_stable/S4a.truth.json +52 -0
  158. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_event_stable/S4b.json +1837 -0
  159. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_event_stable/S4b.truth.json +40 -0
  160. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_event_stable/S5.json +1828 -0
  161. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_event_stable/S5.truth.json +55 -0
  162. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_event_stable/S6a.json +807 -0
  163. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_event_stable/S6a.truth.json +39 -0
  164. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_event_stable/S6b.json +1385 -0
  165. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_event_stable/S6b.truth.json +40 -0
  166. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_event_stable_contrib/S1.json +621 -0
  167. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_event_stable_contrib/S1.truth.json +29 -0
  168. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_event_stable_contrib/S2.json +2718 -0
  169. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_event_stable_contrib/S2.truth.json +57 -0
  170. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_event_stable_contrib/S3.json +3488 -0
  171. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_event_stable_contrib/S3.truth.json +69 -0
  172. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_event_stable_contrib/S4a.json +2653 -0
  173. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_event_stable_contrib/S4a.truth.json +52 -0
  174. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_event_stable_contrib/S4b.json +1886 -0
  175. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_event_stable_contrib/S4b.truth.json +40 -0
  176. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_event_stable_contrib/S5.json +1877 -0
  177. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_event_stable_contrib/S5.truth.json +56 -0
  178. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_event_stable_contrib/S6a.json +868 -0
  179. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_event_stable_contrib/S6a.truth.json +40 -0
  180. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_event_stable_contrib/S6b.json +1434 -0
  181. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_event_stable_contrib/S6b.truth.json +41 -0
  182. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_nolegacy/S1.json +398 -0
  183. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_nolegacy/S1.truth.json +29 -0
  184. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_nolegacy/S2.json +1220 -0
  185. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_nolegacy/S2.truth.json +44 -0
  186. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_nolegacy/S3.json +1648 -0
  187. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_nolegacy/S3.truth.json +63 -0
  188. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_nolegacy/S4a.json +1257 -0
  189. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_nolegacy/S4a.truth.json +46 -0
  190. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_nolegacy/S4b.json +1081 -0
  191. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_nolegacy/S4b.truth.json +39 -0
  192. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_nolegacy/S5.json +928 -0
  193. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_nolegacy/S5.truth.json +44 -0
  194. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_nolegacy/S6a.json +612 -0
  195. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_nolegacy/S6a.truth.json +38 -0
  196. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_nolegacy/S6b.json +791 -0
  197. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v1_nolegacy/S6b.truth.json +37 -0
  198. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_default/S1.json +418 -0
  199. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_default/S1.truth.json +28 -0
  200. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_default/S2.json +1240 -0
  201. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_default/S2.truth.json +56 -0
  202. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_default/S3.json +1708 -0
  203. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_default/S3.truth.json +68 -0
  204. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_default/S4a.json +1277 -0
  205. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_default/S4a.truth.json +52 -0
  206. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_default/S4b.json +1127 -0
  207. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_default/S4b.truth.json +40 -0
  208. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_default/S5.json +948 -0
  209. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_default/S5.truth.json +55 -0
  210. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_default/S6a.json +632 -0
  211. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_default/S6a.truth.json +39 -0
  212. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_default/S6b.json +811 -0
  213. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_default/S6b.truth.json +40 -0
  214. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_event_only/S1.json +517 -0
  215. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_event_only/S1.truth.json +28 -0
  216. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_event_only/S2.json +2392 -0
  217. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_event_only/S2.truth.json +56 -0
  218. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_event_only/S3.json +2749 -0
  219. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_event_only/S3.truth.json +68 -0
  220. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_event_only/S4a.json +2300 -0
  221. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_event_only/S4a.truth.json +52 -0
  222. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_event_only/S4b.json +1673 -0
  223. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_event_only/S4b.truth.json +40 -0
  224. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_event_only/S5.json +1554 -0
  225. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_event_only/S5.truth.json +55 -0
  226. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_event_only/S6a.json +848 -0
  227. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_event_only/S6a.truth.json +39 -0
  228. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_event_only/S6b.json +1282 -0
  229. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_event_only/S6b.truth.json +40 -0
  230. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_event_only_nolegacy/S1.json +517 -0
  231. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_event_only_nolegacy/S1.truth.json +28 -0
  232. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_event_only_nolegacy/S2.json +2392 -0
  233. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_event_only_nolegacy/S2.truth.json +56 -0
  234. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_event_only_nolegacy/S3.json +2749 -0
  235. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_event_only_nolegacy/S3.truth.json +68 -0
  236. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_event_only_nolegacy/S4a.json +2300 -0
  237. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_event_only_nolegacy/S4a.truth.json +52 -0
  238. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_event_only_nolegacy/S4b.json +1673 -0
  239. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_event_only_nolegacy/S4b.truth.json +40 -0
  240. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_event_only_nolegacy/S5.json +1554 -0
  241. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_event_only_nolegacy/S5.truth.json +55 -0
  242. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_event_only_nolegacy/S6a.json +848 -0
  243. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_event_only_nolegacy/S6a.truth.json +39 -0
  244. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_event_only_nolegacy/S6b.json +1282 -0
  245. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_event_only_nolegacy/S6b.truth.json +40 -0
  246. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_no_content/S1.json +376 -0
  247. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_no_content/S1.truth.json +28 -0
  248. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_no_content/S2.json +1159 -0
  249. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_no_content/S2.truth.json +56 -0
  250. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_no_content/S3.json +1384 -0
  251. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_no_content/S3.truth.json +68 -0
  252. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_no_content/S4a.json +1169 -0
  253. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_no_content/S4a.truth.json +52 -0
  254. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_no_content/S4b.json +1031 -0
  255. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_no_content/S4b.truth.json +40 -0
  256. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_no_content/S5.json +894 -0
  257. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_no_content/S5.truth.json +55 -0
  258. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_no_content/S6a.json +632 -0
  259. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_no_content/S6a.truth.json +39 -0
  260. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_no_content/S6b.json +757 -0
  261. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_no_content/S6b.truth.json +40 -0
  262. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_and_event/S1.json +541 -0
  263. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_and_event/S1.truth.json +28 -0
  264. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_and_event/S2.json +2446 -0
  265. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_and_event/S2.truth.json +56 -0
  266. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_and_event/S3.json +2821 -0
  267. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_and_event/S3.truth.json +68 -0
  268. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_and_event/S4a.json +2354 -0
  269. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_and_event/S4a.truth.json +52 -0
  270. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_and_event/S4b.json +1733 -0
  271. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_and_event/S4b.truth.json +40 -0
  272. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_and_event/S5.json +1590 -0
  273. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_and_event/S5.truth.json +55 -0
  274. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_and_event/S6a.json +866 -0
  275. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_and_event/S6a.truth.json +39 -0
  276. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_and_event/S6b.json +1318 -0
  277. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_and_event/S6b.truth.json +40 -0
  278. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only/S1.json +400 -0
  279. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only/S1.truth.json +28 -0
  280. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only/S2.json +1213 -0
  281. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only/S2.truth.json +56 -0
  282. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only/S3.json +1456 -0
  283. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only/S3.truth.json +68 -0
  284. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only/S4a.json +1223 -0
  285. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only/S4a.truth.json +52 -0
  286. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only/S4b.json +1091 -0
  287. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only/S4b.truth.json +40 -0
  288. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only/S5.json +930 -0
  289. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only/S5.truth.json +55 -0
  290. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only/S6a.json +650 -0
  291. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only/S6a.truth.json +39 -0
  292. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only/S6b.json +793 -0
  293. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only/S6b.truth.json +40 -0
  294. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only_contrib/S1.json +461 -0
  295. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only_contrib/S1.truth.json +29 -0
  296. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only_contrib/S2.json +1292 -0
  297. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only_contrib/S2.truth.json +57 -0
  298. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only_contrib/S3.json +1541 -0
  299. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only_contrib/S3.truth.json +69 -0
  300. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only_contrib/S4a.json +1302 -0
  301. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only_contrib/S4a.truth.json +52 -0
  302. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only_contrib/S4b.json +1164 -0
  303. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only_contrib/S4b.truth.json +40 -0
  304. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only_contrib/S5.json +1003 -0
  305. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only_contrib/S5.truth.json +56 -0
  306. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only_contrib/S6a.json +717 -0
  307. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only_contrib/S6a.truth.json +40 -0
  308. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only_contrib/S6b.json +866 -0
  309. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only_contrib/S6b.truth.json +41 -0
  310. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only_nolegacy/S1.json +400 -0
  311. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only_nolegacy/S1.truth.json +28 -0
  312. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only_nolegacy/S2.json +1213 -0
  313. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only_nolegacy/S2.truth.json +56 -0
  314. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only_nolegacy/S3.json +1456 -0
  315. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only_nolegacy/S3.truth.json +68 -0
  316. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only_nolegacy/S4a.json +1223 -0
  317. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only_nolegacy/S4a.truth.json +52 -0
  318. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only_nolegacy/S4b.json +1091 -0
  319. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only_nolegacy/S4b.truth.json +40 -0
  320. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only_nolegacy/S5.json +930 -0
  321. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only_nolegacy/S5.truth.json +55 -0
  322. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only_nolegacy/S6a.json +650 -0
  323. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only_nolegacy/S6a.truth.json +39 -0
  324. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only_nolegacy/S6b.json +793 -0
  325. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2_span_only_nolegacy/S6b.truth.json +40 -0
  326. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_event_only/S1.json +438 -0
  327. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_event_only/S1.truth.json +28 -0
  328. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_event_only/S2.json +2107 -0
  329. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_event_only/S2.truth.json +56 -0
  330. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_event_only/S3.json +2409 -0
  331. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_event_only/S3.truth.json +68 -0
  332. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_event_only/S4a.json +2015 -0
  333. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_event_only/S4a.truth.json +52 -0
  334. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_event_only/S4b.json +1412 -0
  335. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_event_only/S4b.truth.json +40 -0
  336. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_event_only/S5.json +1348 -0
  337. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_event_only/S5.truth.json +55 -0
  338. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_event_only/S6a.json +745 -0
  339. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_event_only/S6a.truth.json +39 -0
  340. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_event_only/S6b.json +1100 -0
  341. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_event_only/S6b.truth.json +40 -0
  342. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_no_content/S1.json +297 -0
  343. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_no_content/S1.truth.json +29 -0
  344. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_no_content/S2.json +874 -0
  345. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_no_content/S2.truth.json +44 -0
  346. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_no_content/S3.json +1044 -0
  347. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_no_content/S3.truth.json +63 -0
  348. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_no_content/S4a.json +884 -0
  349. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_no_content/S4a.truth.json +46 -0
  350. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_no_content/S4b.json +770 -0
  351. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_no_content/S4b.truth.json +39 -0
  352. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_no_content/S5.json +688 -0
  353. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_no_content/S5.truth.json +44 -0
  354. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_no_content/S6a.json +529 -0
  355. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_no_content/S6a.truth.json +38 -0
  356. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_no_content/S6b.json +575 -0
  357. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_no_content/S6b.truth.json +37 -0
  358. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_span_only/S1.json +321 -0
  359. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_span_only/S1.truth.json +28 -0
  360. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_span_only/S2.json +952 -0
  361. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_span_only/S2.truth.json +56 -0
  362. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_span_only/S3.json +1128 -0
  363. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_span_only/S3.truth.json +68 -0
  364. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_span_only/S4a.json +962 -0
  365. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_span_only/S4a.truth.json +52 -0
  366. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_span_only/S4b.json +842 -0
  367. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_span_only/S4b.truth.json +40 -0
  368. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_span_only/S5.json +748 -0
  369. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_span_only/S5.truth.json +55 -0
  370. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_span_only/S6a.json +553 -0
  371. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_span_only/S6a.truth.json +39 -0
  372. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_span_only/S6b.json +623 -0
  373. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_span_only/S6b.truth.json +40 -0
  374. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_span_only_contrib/S1.json +382 -0
  375. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_span_only_contrib/S1.truth.json +29 -0
  376. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_span_only_contrib/S2.json +1031 -0
  377. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_span_only_contrib/S2.truth.json +57 -0
  378. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_span_only_contrib/S3.json +1213 -0
  379. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_span_only_contrib/S3.truth.json +69 -0
  380. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_span_only_contrib/S4a.json +1041 -0
  381. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_span_only_contrib/S4a.truth.json +52 -0
  382. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_span_only_contrib/S4b.json +915 -0
  383. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_span_only_contrib/S4b.truth.json +40 -0
  384. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_span_only_contrib/S5.json +821 -0
  385. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_span_only_contrib/S5.truth.json +56 -0
  386. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_span_only_contrib/S6a.json +620 -0
  387. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_span_only_contrib/S6a.truth.json +40 -0
  388. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_span_only_contrib/S6b.json +696 -0
  389. agentevals_cli-0.10.0/tests/fixtures/otlp/adk_py/v2proj_span_only_contrib/S6b.truth.json +41 -0
  390. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/adk_cycle_poisons_batch.json +162 -0
  391. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/adk_cycle_poisons_batch.truth.json +30 -0
  392. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/adk_go_main.json +289 -0
  393. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/adk_go_main.truth.json +43 -0
  394. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/content_capture_off.json +188 -0
  395. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/content_capture_off.truth.json +36 -0
  396. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/deprecated_aliases.json +95 -0
  397. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/deprecated_aliases.truth.json +26 -0
  398. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/details_event_log_debug.json +163 -0
  399. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/details_event_log_debug.truth.json +30 -0
  400. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/duplicate_span_ids.json +186 -0
  401. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/duplicate_span_ids.truth.json +28 -0
  402. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/ext_eval_eventName.json +162 -0
  403. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/ext_eval_eventName.truth.json +36 -0
  404. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/ext_eval_event_name_attr.json +167 -0
  405. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/ext_eval_event_name_attr.truth.json +36 -0
  406. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/ext_eval_otel_event_name.json +167 -0
  407. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/ext_eval_otel_event_name.truth.json +36 -0
  408. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/ext_eval_span_event.json +132 -0
  409. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/ext_eval_span_event.truth.json +36 -0
  410. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/failed_inference_then_retry.json +151 -0
  411. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/failed_inference_then_retry.truth.json +28 -0
  412. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/mcp_tool_call.json +315 -0
  413. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/mcp_tool_call.truth.json +44 -0
  414. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/openai_agents_handoff.json +308 -0
  415. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/openai_agents_handoff.truth.json +50 -0
  416. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/openai_v2_logs_no_span_id_ambiguous.json +569 -0
  417. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/openai_v2_logs_no_span_id_ambiguous.truth.json +26 -0
  418. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/openai_v2_logs_no_span_id_single.json +244 -0
  419. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/openai_v2_logs_no_span_id_single.truth.json +26 -0
  420. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/openai_v2_logs_span_id.json +537 -0
  421. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/openai_v2_logs_span_id.truth.json +40 -0
  422. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/remote_parent_root.json +108 -0
  423. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/remote_parent_root.truth.json +28 -0
  424. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/self_parent.json +159 -0
  425. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/self_parent.truth.json +39 -0
  426. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/strands_span_events.json +274 -0
  427. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/strands_span_events.truth.json +42 -0
  428. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/three_trace_conversation.json +292 -0
  429. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/three_trace_conversation.truth.json +55 -0
  430. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/tool_error_no_error_type.json +192 -0
  431. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/tool_error_no_error_type.truth.json +38 -0
  432. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/two_span_cycle.json +212 -0
  433. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/two_span_cycle.truth.json +39 -0
  434. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/upstream_new_parts.json +113 -0
  435. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/upstream_new_parts.truth.json +27 -0
  436. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/util_genai_openai_tool_loop.json +214 -0
  437. agentevals_cli-0.10.0/tests/fixtures/otlp/hand/util_genai_openai_tool_loop.truth.json +41 -0
  438. agentevals_cli-0.10.0/tests/fixtures/otlp/kagent/v1.0.0-alpha9/README.md +24 -0
  439. agentevals_cli-0.10.0/tests/fixtures/otlp/kagent/v1.0.0-alpha9/adk_go_1turn.json +1 -0
  440. agentevals_cli-0.10.0/tests/fixtures/otlp/kagent/v1.0.0-alpha9/adk_go_1turn.truth.json +60 -0
  441. agentevals_cli-0.10.0/tests/fixtures/otlp/kagent/v1.0.0-alpha9/adk_go_2turn.json +1 -0
  442. agentevals_cli-0.10.0/tests/fixtures/otlp/kagent/v1.0.0-alpha9/adk_go_2turn.truth.json +90 -0
  443. agentevals_cli-0.10.0/tests/fixtures/otlp/kagent/v1.0.0-alpha9/claude_native_1turn_subagent.json +1 -0
  444. agentevals_cli-0.10.0/tests/fixtures/otlp/kagent/v1.0.0-alpha9/claude_native_1turn_subagent.truth.json +30 -0
  445. agentevals_cli-0.10.0/tests/fixtures/otlp/kagent/v1.0.0-alpha9/claude_native_2turn.json +1 -0
  446. agentevals_cli-0.10.0/tests/fixtures/otlp/kagent/v1.0.0-alpha9/claude_native_2turn.truth.json +44 -0
  447. agentevals_cli-0.10.0/tests/fixtures/otlp/kagent/v1.0.0-alpha9/claude_recipe_2turn.json +1 -0
  448. agentevals_cli-0.10.0/tests/fixtures/otlp/kagent/v1.0.0-alpha9/claude_recipe_2turn.truth.json +116 -0
  449. agentevals_cli-0.10.0/tests/fixtures/otlp/samples/helm.json +5333 -0
  450. agentevals_cli-0.10.0/tests/fixtures/otlp/samples/helm.truth.json +44 -0
  451. agentevals_cli-0.10.0/tests/fixtures/otlp/samples/helm_2.json +3708 -0
  452. agentevals_cli-0.10.0/tests/fixtures/otlp/samples/helm_2.truth.json +28 -0
  453. agentevals_cli-0.10.0/tests/fixtures/otlp/samples/helm_3.json +7182 -0
  454. agentevals_cli-0.10.0/tests/fixtures/otlp/samples/helm_3.truth.json +47 -0
  455. agentevals_cli-0.10.0/tests/fixtures/otlp/samples/k8s.json +3908 -0
  456. agentevals_cli-0.10.0/tests/fixtures/otlp/samples/k8s.truth.json +28 -0
  457. agentevals_cli-0.10.0/tests/fixtures/otlp/samples/tempo_export_with_batches.json +1 -0
  458. agentevals_cli-0.10.0/tests/fixtures/otlp/samples/tempo_export_with_batches.truth.json +44 -0
  459. agentevals_cli-0.10.0/tests/genai/test_continuations.py +119 -0
  460. agentevals_cli-0.10.0/tests/genai/test_corpus.py +305 -0
  461. agentevals_cli-0.10.0/tests/genai/test_failed_turns.py +73 -0
  462. agentevals_cli-0.10.0/tests/genai/test_grouping.py +24 -0
  463. agentevals_cli-0.10.0/tests/genai/test_legacy_chat.py +48 -0
  464. agentevals_cli-0.10.0/tests/genai/test_tools.py +137 -0
  465. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/tests/integration/conftest.py +48 -26
  466. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/tests/integration/test_live_agents.py +16 -17
  467. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/tests/integration/test_otlp_grpc_receiver.py +4 -5
  468. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/tests/integration/test_session_grouping.py +49 -44
  469. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/tests/integration/test_timing_stress.py +19 -18
  470. agentevals_cli-0.10.0/tests/otel/builders.py +185 -0
  471. agentevals_cli-0.10.0/tests/otel/test_emit.py +316 -0
  472. agentevals_cli-0.10.0/tests/otel/test_store.py +385 -0
  473. agentevals_cli-0.10.0/tests/run/__init__.py +0 -0
  474. agentevals_cli-0.10.0/tests/storage/__init__.py +0 -0
  475. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/tests/storage/test_migrator.py +3 -3
  476. agentevals_cli-0.10.0/tests/streaming/__init__.py +0 -0
  477. agentevals_cli-0.10.0/tests/streaming/test_live.py +224 -0
  478. agentevals_cli-0.10.0/tests/streaming/test_session_export.py +69 -0
  479. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/tests/test_api.py +119 -101
  480. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/tests/test_builtin_metrics.py +3 -3
  481. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/tests/test_cli.py +1 -1
  482. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/tests/test_jaeger_loader.py +14 -45
  483. agentevals_cli-0.10.0/tests/test_layering.py +37 -0
  484. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/tests/test_loader_auto.py +20 -3
  485. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/tests/test_mcp_server.py +26 -0
  486. agentevals_cli-0.10.0/tests/test_otlp_http_protocol.py +807 -0
  487. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/tests/test_otlp_loader.py +74 -73
  488. agentevals_cli-0.10.0/tests/test_otlp_receiver.py +985 -0
  489. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/tests/test_runner.py +92 -119
  490. agentevals_cli-0.10.0/tests/test_sdk.py +460 -0
  491. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/tests/test_trace_metrics.py +64 -70
  492. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/api/client.ts +13 -2
  493. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/inspector/InspectorView.tsx +4 -27
  494. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/streaming/LiveConversationPanel.tsx +3 -1
  495. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/streaming/LiveMessage.tsx +18 -0
  496. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/streaming/LiveStreamingView.tsx +75 -16
  497. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/streaming/SessionCard.tsx +70 -23
  498. agentevals_cli-0.10.0/ui/src/components/streaming/turnStatus.ts +31 -0
  499. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/upload/TraceEditorDrawer.tsx +1 -1
  500. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/config.ts +0 -7
  501. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/context/TraceProvider.tsx +7 -2
  502. agentevals_cli-0.10.0/ui/src/lib/trace-helpers.ts +2 -0
  503. agentevals_cli-0.10.0/ui/src/lib/trace-patcher.ts +352 -0
  504. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/lib/types.ts +40 -10
  505. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/lib/utils.ts +13 -19
  506. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/uv.lock +79 -73
  507. agentevals_cli-0.9.12/PKG-INFO +0 -483
  508. agentevals_cli-0.9.12/README.md +0 -442
  509. agentevals_cli-0.9.12/docs/otel-compatibility.md +0 -191
  510. agentevals_cli-0.9.12/docs/streaming.md +0 -322
  511. agentevals_cli-0.9.12/examples/README.md +0 -251
  512. agentevals_cli-0.9.12/examples/custom_evaluators/eval_config_openai_eval.yaml +0 -18
  513. agentevals_cli-0.9.12/examples/dice_agent/main.py +0 -139
  514. agentevals_cli-0.9.12/examples/dice_agent/test_streaming.py +0 -63
  515. agentevals_cli-0.9.12/examples/kubernetes/README.md +0 -256
  516. agentevals_cli-0.9.12/examples/langchain_agent/README.md +0 -232
  517. agentevals_cli-0.9.12/examples/langchain_agent/main.py +0 -180
  518. agentevals_cli-0.9.12/examples/langchain_agent/test_streaming.py +0 -79
  519. agentevals_cli-0.9.12/examples/strands_agent/main.py +0 -138
  520. agentevals_cli-0.9.12/src/agentevals/api/debug_routes.py +0 -262
  521. agentevals_cli-0.9.12/src/agentevals/api/dependencies.py +0 -28
  522. agentevals_cli-0.9.12/src/agentevals/api/otlp_processing.py +0 -335
  523. agentevals_cli-0.9.12/src/agentevals/api/otlp_routes.py +0 -69
  524. agentevals_cli-0.9.12/src/agentevals/converter.py +0 -319
  525. agentevals_cli-0.9.12/src/agentevals/evaluator/resolver.py +0 -70
  526. agentevals_cli-0.9.12/src/agentevals/extraction.py +0 -689
  527. agentevals_cli-0.9.12/src/agentevals/genai_converter.py +0 -477
  528. agentevals_cli-0.9.12/src/agentevals/loader/auto.py +0 -108
  529. agentevals_cli-0.9.12/src/agentevals/loader/base.py +0 -53
  530. agentevals_cli-0.9.12/src/agentevals/loader/jaeger.py +0 -112
  531. agentevals_cli-0.9.12/src/agentevals/loader/otlp.py +0 -252
  532. agentevals_cli-0.9.12/src/agentevals/openai_eval_backend.py +0 -267
  533. agentevals_cli-0.9.12/src/agentevals/otlp_anyvalue.py +0 -118
  534. agentevals_cli-0.9.12/src/agentevals/runner.py +0 -357
  535. agentevals_cli-0.9.12/src/agentevals/streaming/__init__.py +0 -120
  536. agentevals_cli-0.9.12/src/agentevals/streaming/incremental_processor.py +0 -341
  537. agentevals_cli-0.9.12/src/agentevals/streaming/processor.py +0 -288
  538. agentevals_cli-0.9.12/src/agentevals/streaming/ws_server.py +0 -926
  539. agentevals_cli-0.9.12/src/agentevals/trace_attrs.py +0 -116
  540. agentevals_cli-0.9.12/src/agentevals/trace_metrics.py +0 -245
  541. agentevals_cli-0.9.12/src/agentevals/utils/genai_messages.py +0 -142
  542. agentevals_cli-0.9.12/src/agentevals/utils/log_enrichment.py +0 -192
  543. agentevals_cli-0.9.12/tests/test_converter.py +0 -567
  544. agentevals_cli-0.9.12/tests/test_extraction.py +0 -1119
  545. agentevals_cli-0.9.12/tests/test_genai_converter.py +0 -992
  546. agentevals_cli-0.9.12/tests/test_incremental_processor.py +0 -26
  547. agentevals_cli-0.9.12/tests/test_log_enrichment.py +0 -214
  548. agentevals_cli-0.9.12/tests/test_openai_eval_backend.py +0 -117
  549. agentevals_cli-0.9.12/tests/test_otlp_receiver.py +0 -1826
  550. agentevals_cli-0.9.12/tests/test_sdk.py +0 -472
  551. agentevals_cli-0.9.12/ui/src/lib/trace-helpers.ts +0 -73
  552. agentevals_cli-0.9.12/ui/src/lib/trace-patcher.ts +0 -328
  553. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/.claude/skills/eval/SKILL.md +0 -0
  554. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/.claude/skills/eval/evals/evals.json +0 -0
  555. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/.claude/skills/inspect/SKILL.md +0 -0
  556. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/.claude/skills/inspect/evals/evals.json +0 -0
  557. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/.dockerignore +0 -0
  558. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/.github/ISSUE_TEMPLATE/bug_report.yml +0 -0
  559. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/.github/ISSUE_TEMPLATE/config.yml +0 -0
  560. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/.github/ISSUE_TEMPLATE/feature_request.yml +0 -0
  561. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/.github/dependabot.yml +0 -0
  562. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/.github/workflows/ci.yml +0 -0
  563. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/.github/workflows/publish-evaluator-sdk.yml +0 -0
  564. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/.github/workflows/release.yml +0 -0
  565. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/.gitignore +0 -0
  566. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/.mcp.json +0 -0
  567. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/DEVELOPMENT.md +0 -0
  568. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/Dockerfile +0 -0
  569. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/LICENSE +0 -0
  570. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/Makefile +0 -0
  571. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/charts/agentevals/Chart.yaml +0 -0
  572. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/charts/agentevals/templates/NOTES.txt +0 -0
  573. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/charts/agentevals/templates/_helpers.tpl +0 -0
  574. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/charts/agentevals/templates/deployment.yaml +0 -0
  575. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/charts/agentevals/templates/postgresql-secret.yaml +0 -0
  576. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/charts/agentevals/templates/postgresql.yaml +0 -0
  577. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/charts/agentevals/templates/service.yaml +0 -0
  578. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/charts/agentevals/templates/serviceaccount.yaml +0 -0
  579. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/docs/assets/logo-color-on-transparent.svg +0 -0
  580. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/docs/assets/logo-color.png +0 -0
  581. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/docs/assets/logo-dark-on-transparent.svg +0 -0
  582. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/docs/run-history.md +0 -0
  583. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/examples/custom_evaluators/eval_config.yaml +0 -0
  584. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/examples/custom_evaluators/response_quality.py +0 -0
  585. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/examples/custom_evaluators/tool_call_checker.py +0 -0
  586. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/examples/custom_sink/README.md +0 -0
  587. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/examples/custom_sink/agentevals_example_custom_sink/__init__.py +0 -0
  588. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/examples/custom_sink/agentevals_example_custom_sink/sink.py +0 -0
  589. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/examples/custom_sink/pyproject.toml +0 -0
  590. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/examples/dice_agent/README.md +0 -0
  591. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/examples/dice_agent/agent.py +0 -0
  592. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/examples/dice_agent/eval_set.json +0 -0
  593. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/examples/langchain_agent/agent.py +0 -0
  594. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/examples/langchain_agent/eval_set.json +0 -0
  595. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/examples/langchain_agent/requirements.txt +0 -0
  596. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/examples/sdk_example/async_example.py +0 -0
  597. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/examples/sdk_example/context_manager_example.py +0 -0
  598. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/examples/sdk_example/decorator_example.py +0 -0
  599. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/examples/sdk_example/requirements.txt +0 -0
  600. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/examples/strands_agent/agent.py +0 -0
  601. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/examples/strands_agent/eval_set.json +0 -0
  602. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/examples/strands_agent/requirements.txt +0 -0
  603. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/examples/zero-code-examples/adk/requirements.txt +0 -0
  604. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/examples/zero-code-examples/langchain/requirements.txt +0 -0
  605. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/examples/zero-code-examples/ollama/requirements.txt +0 -0
  606. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/examples/zero-code-examples/pydantic-ai/requirements.txt +0 -0
  607. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/examples/zero-code-examples/strands/requirements.txt +0 -0
  608. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/flake.lock +0 -0
  609. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/flake.nix +0 -0
  610. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/packages/evaluator-sdk-py/README.md +0 -0
  611. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/packages/evaluator-sdk-py/pyproject.toml +0 -0
  612. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/packages/evaluator-sdk-py/src/agentevals_evaluator_sdk/__init__.py +0 -0
  613. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/packages/evaluator-sdk-py/src/agentevals_evaluator_sdk/decorator.py +0 -0
  614. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/samples/eval_set_helm.json +0 -0
  615. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/samples/evalset_helm_3_2026-02-23.json +0 -0
  616. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/samples/evalset_k8s_2026-02-20.json +0 -0
  617. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/samples/helm.json +0 -0
  618. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/samples/helm_2.json +0 -0
  619. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/samples/helm_3.json +0 -0
  620. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/samples/k8s.json +0 -0
  621. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/samples/tempo_export_with_batches.json +0 -0
  622. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/__init__.py +0 -0
  623. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/_static/assets/index-BqibLiHO.css +0 -0
  624. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/_static/logo.svg +0 -0
  625. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/_static/vite.svg +0 -0
  626. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/api/__init__.py +0 -0
  627. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/api/runs_routes.py +0 -0
  628. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/evaluator/__init__.py +0 -0
  629. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/output.py +0 -0
  630. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/resolvers/__init__.py +0 -0
  631. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/resolvers/kubernetes.py +0 -0
  632. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/run/__init__.py +0 -0
  633. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/run/service.py +0 -0
  634. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/run/sinks.py +0 -0
  635. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/storage/__init__.py +0 -0
  636. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/storage/config.py +0 -0
  637. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/storage/postgres/__init__.py +0 -0
  638. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/storage/postgres/migrations/000001_init.down.sql +0 -0
  639. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/storage/postgres/migrations/000001_init.up.sql +0 -0
  640. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/storage/postgres/pool.py +0 -0
  641. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/storage/repos/__init__.py +0 -0
  642. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/storage/repos/memory.py +0 -0
  643. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/storage/repos/postgres.py +0 -0
  644. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/streaming/exports.py +0 -0
  645. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/streaming/session.py +0 -0
  646. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/utils/__init__.py +0 -0
  647. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/src/agentevals/utils/log_buffer.py +0 -0
  648. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/tests/api/__init__.py +0 -0
  649. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/tests/api/test_runs_routes.py +0 -0
  650. {agentevals_cli-0.9.12/tests/integration → agentevals_cli-0.10.0/tests/genai}/__init__.py +0 -0
  651. {agentevals_cli-0.9.12/tests/resolvers → agentevals_cli-0.10.0/tests/integration}/__init__.py +0 -0
  652. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/tests/integration/test_evaluation_pipeline.py +0 -0
  653. {agentevals_cli-0.9.12/tests/run → agentevals_cli-0.10.0/tests/otel}/__init__.py +0 -0
  654. {agentevals_cli-0.9.12/tests/storage → agentevals_cli-0.10.0/tests/resolvers}/__init__.py +0 -0
  655. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/tests/resolvers/test_kubernetes.py +0 -0
  656. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/tests/resolvers/test_registry.py +0 -0
  657. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/tests/run/test_fetcher.py +0 -0
  658. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/tests/run/test_result_builder.py +0 -0
  659. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/tests/run/test_service.py +0 -0
  660. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/tests/run/test_sinks.py +0 -0
  661. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/tests/storage/test_config.py +0 -0
  662. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/tests/storage/test_memory_repos.py +0 -0
  663. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/tests/storage/test_models.py +0 -0
  664. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/tests/test_credential_injection.py +0 -0
  665. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/tests/test_eval_config_loader.py +0 -0
  666. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/tests/test_output.py +0 -0
  667. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/tests/test_protocol.py +0 -0
  668. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/.gitignore +0 -0
  669. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/README.md +0 -0
  670. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/eslint.config.js +0 -0
  671. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/index.html +0 -0
  672. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/package-lock.json +0 -0
  673. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/package.json +0 -0
  674. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/public/logo.svg +0 -0
  675. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/public/vite.svg +0 -0
  676. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/App.css +0 -0
  677. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/App.tsx +0 -0
  678. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/assets/react.svg +0 -0
  679. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/annotation-queue/AnnotationDetailPanel.tsx +0 -0
  680. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/annotation-queue/AnnotationQueueView.tsx +0 -0
  681. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/annotation-queue/AnnotationTable.tsx +0 -0
  682. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/bug-report/BugReportModal.tsx +0 -0
  683. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/builder/BuilderHeader.tsx +0 -0
  684. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/builder/BuilderView.tsx +0 -0
  685. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/builder/EvalCaseCard.tsx +0 -0
  686. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/builder/EvalCasesList.tsx +0 -0
  687. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/builder/InvocationEditor.tsx +0 -0
  688. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/builder/JsonPreview.tsx +0 -0
  689. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/builder/MetadataEditor.tsx +0 -0
  690. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/builder/TraceUploadZone.tsx +0 -0
  691. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/builder/index.ts +0 -0
  692. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/dashboard/DashboardView.tsx +0 -0
  693. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/dashboard/MetricScoreCard.tsx +0 -0
  694. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/dashboard/PerformanceCard.tsx +0 -0
  695. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/dashboard/PerformanceCharts.tsx +0 -0
  696. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/dashboard/SummaryStats.tsx +0 -0
  697. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/dashboard/TraceCard.tsx +0 -0
  698. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/dashboard/TraceTable.tsx +0 -0
  699. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/inspector/ComparisonPanel.tsx +0 -0
  700. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/inspector/DataSection.tsx +0 -0
  701. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/inspector/InspectorHeader.tsx +0 -0
  702. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/inspector/InspectorLayout.tsx +0 -0
  703. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/inspector/InvocationCard.tsx +0 -0
  704. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/inspector/InvocationSummaryPanel.tsx +0 -0
  705. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/inspector/MetricResultsSection.tsx +0 -0
  706. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/inspector/MetricsComparisonSection.tsx +0 -0
  707. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/inspector/PerformanceSection.tsx +0 -0
  708. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/inspector/ToolCallList.tsx +0 -0
  709. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/inspector/TrajectoryComparisonDetails.tsx +0 -0
  710. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/runs/PassRateTrendChart.tsx +0 -0
  711. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/runs/PerMetricTrendChart.tsx +0 -0
  712. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/runs/RunDetailView.tsx +0 -0
  713. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/runs/RunsHistoryTable.tsx +0 -0
  714. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/runs/RunsView.tsx +0 -0
  715. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/runs/runHistory.ts +0 -0
  716. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/sidebar/Sidebar.tsx +0 -0
  717. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/streaming/SessionMetadata.tsx +0 -0
  718. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/upload/EvalSetEditorDrawer.tsx +0 -0
  719. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/upload/FileDropZone.tsx +0 -0
  720. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/upload/MetricSelector.tsx +0 -0
  721. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/upload/RawJsonPreview.tsx +0 -0
  722. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/upload/UploadView.tsx +0 -0
  723. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/components/welcome/WelcomeView.tsx +0 -0
  724. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/context/TraceContext.tsx +0 -0
  725. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/index.css +0 -0
  726. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/lib/console-capture.ts +0 -0
  727. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/lib/eval-config.ts +0 -0
  728. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/lib/evalset-builder.ts +0 -0
  729. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/lib/network-capture.ts +0 -0
  730. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/lib/trace-loader.ts +0 -0
  731. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/lib/trace-metadata.ts +0 -0
  732. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/src/main.tsx +0 -0
  733. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/tsconfig.app.json +0 -0
  734. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/tsconfig.json +0 -0
  735. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/tsconfig.node.json +0 -0
  736. {agentevals_cli-0.9.12 → agentevals_cli-0.10.0}/ui/vite.config.ts +0 -0
@@ -0,0 +1,26 @@
1
+ name: kagent contract policy
2
+
3
+ # Tests the kagent contract policy against its local registries and negative
4
+ # cases when the policy or its tooling changes. It never fetches kagent itself.
5
+ on:
6
+ pull_request:
7
+ branches: [main]
8
+ paths:
9
+ - "tests/contract/kagent/**"
10
+ - ".github/workflows/kagent-contract.yml"
11
+ - ".github/workflows/kagent-contract-policy.yml"
12
+
13
+ permissions:
14
+ contents: read
15
+
16
+ jobs:
17
+ selftest:
18
+ runs-on: ubuntu-latest
19
+ timeout-minutes: 20
20
+ steps:
21
+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
22
+ with:
23
+ persist-credentials: false
24
+
25
+ - name: Self test the contract policy
26
+ run: tests/contract/kagent/test.sh
@@ -0,0 +1,143 @@
1
+ name: kagent contract
2
+
3
+ # Checks every day, with Weaver, that kagent's telemetry registry still has what
4
+ # agentevals reads from kagent spans, and keeps one labelled issue up to date
5
+ # while it does not. It never runs on pull requests: kagent changes on its own
6
+ # schedule. See tests/contract/kagent/.
7
+ on:
8
+ schedule:
9
+ - cron: "23 5 * * *"
10
+ workflow_dispatch:
11
+
12
+ permissions: {}
13
+
14
+ concurrency:
15
+ group: kagent-contract
16
+ cancel-in-progress: false
17
+
18
+ defaults:
19
+ run:
20
+ shell: bash
21
+
22
+ env:
23
+ ISSUE_LABEL: kagent-contract
24
+
25
+ jobs:
26
+ check:
27
+ runs-on: ubuntu-latest
28
+ timeout-minutes: 30
29
+ permissions:
30
+ contents: read
31
+ steps:
32
+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
33
+ with:
34
+ persist-credentials: false
35
+
36
+ - name: Self test the contract policy
37
+ run: tests/contract/kagent/test.sh
38
+
39
+ # The release tag is fetched and used in one step, and check.sh validates it
40
+ # before use, so nothing fetched passes through GITHUB_ENV or an expression.
41
+ - name: Check kagent main and the newest release
42
+ env:
43
+ GH_TOKEN: ${{ github.token }}
44
+ run: |
45
+ release="$(gh release view --repo kagent-dev/kagent --json tagName --jq .tagName)"
46
+ tests/contract/kagent/check.sh report.md main "$release"
47
+
48
+ - name: Summarize
49
+ if: always()
50
+ continue-on-error: true
51
+ run: |
52
+ {
53
+ if [ -f report.md ]; then cat report.md; fi
54
+ echo
55
+ echo "## kagent main against the committed snapshot"
56
+ echo
57
+ tests/contract/kagent/weaver.sh registry diff --v2 --diff-format markdown \
58
+ -r "https://github.com/kagent-dev/kagent.git@main[telemetry/registry]" \
59
+ --baseline-registry /contract/registries/main-2cc3ffae
60
+ } >> "$GITHUB_STEP_SUMMARY"
61
+
62
+ - name: Upload the report
63
+ if: always()
64
+ uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
65
+ with:
66
+ name: kagent-contract-report
67
+ path: report.md
68
+ if-no-files-found: ignore
69
+
70
+ # The issue body is fixed text plus the report, whose fetched content sits in a
71
+ # code block without backticks. It is always passed as a file, never through the shell.
72
+ report-failure:
73
+ needs: check
74
+ if: failure()
75
+ runs-on: ubuntu-latest
76
+ timeout-minutes: 10
77
+ permissions:
78
+ issues: write
79
+ steps:
80
+ - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
81
+ continue-on-error: true
82
+ with:
83
+ name: kagent-contract-report
84
+
85
+ - name: Open or update the issue
86
+ env:
87
+ GH_TOKEN: ${{ github.token }}
88
+ GH_REPO: ${{ github.repository }}
89
+ RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
90
+ run: |
91
+ set -euo pipefail
92
+ {
93
+ echo "The daily kagent contract check failed: $RUN_URL"
94
+ echo
95
+ if [ -f report.md ]; then
96
+ cat report.md
97
+ else
98
+ echo "The check failed before it wrote a report (for example in the self test). See the run log."
99
+ echo
100
+ echo "<!-- kagent-contract-report: none -->"
101
+ fi
102
+ } > body.md
103
+ # The report appends its marker last, so the last match is the real one.
104
+ marker="$(grep -o 'kagent-contract-report: [0-9a-f]*' body.md | tail -n 1)"
105
+
106
+ number="$(gh issue list --label "$ISSUE_LABEL" --state open --json number --jq '.[0].number // empty')"
107
+ if [ -z "$number" ]; then
108
+ gh label create "$ISSUE_LABEL" --color B60205 --force \
109
+ --description "The daily check of kagent's telemetry registry failed"
110
+ gh issue create --title "kagent telemetry contract check failed" \
111
+ --label "$ISSUE_LABEL" --body-file body.md
112
+ exit 0
113
+ fi
114
+
115
+ # Only comment when the report changed since the last update. Comments
116
+ # from people are ignored, so nobody can hide a change by quoting a marker.
117
+ latest="$(gh issue view "$number" --json body,comments \
118
+ --jq '([.comments[] | select(.author.login == "github-actions")] | last | .body) // .body')"
119
+ if grep -qF "$marker" <<<"$latest"; then
120
+ echo "Issue #$number already has this report."
121
+ else
122
+ gh issue comment "$number" --body-file body.md
123
+ fi
124
+
125
+ close-resolved:
126
+ needs: check
127
+ if: success()
128
+ runs-on: ubuntu-latest
129
+ timeout-minutes: 10
130
+ permissions:
131
+ issues: write
132
+ steps:
133
+ - name: Close the issue once the check passes again
134
+ env:
135
+ GH_TOKEN: ${{ github.token }}
136
+ GH_REPO: ${{ github.repository }}
137
+ RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
138
+ run: |
139
+ set -euo pipefail
140
+ number="$(gh issue list --label "$ISSUE_LABEL" --state open --json number --jq '.[0].number // empty')"
141
+ if [ -n "$number" ]; then
142
+ gh issue close "$number" --comment "The kagent contract check passes again: $RUN_URL"
143
+ fi
@@ -150,34 +150,26 @@ docs/ # Documentation
150
150
 
151
151
  ## Trace Processing Architecture
152
152
 
153
- agentevals converts OTel traces from agent frameworks into a common `Invocation` format for evaluation. If you're adding support for a new framework or changing how we extract data from spans, this section will help you find your way around.
154
-
155
- ### Key Modules
153
+ Telemetry flows through four layers. Each is plain Python with no ADK dependency except the bridge.
156
154
 
157
155
  | Module | What it does |
158
156
  |--------|--------------|
159
- | `trace_attrs.py` | Single source of truth for OTel attribute key constants (`OTEL_GENAI_*` for standard semconv, `ADK_*` for Google ADK) |
160
- | `extraction.py` | Shared extraction functions, span classifiers, and the `TraceFormatExtractor` protocol with `AdkExtractor` / `GenAIExtractor` |
161
- | `converter.py` | Batch conversion orchestration, turns ADK traces into `Invocation` objects |
162
- | `genai_converter.py` | Batch conversion for GenAI semconv traces (single-turn and multi-turn) |
163
- | `streaming/incremental_processor.py` | Real-time span processing for the live UI, uses the same shared extraction functions |
164
- | `utils/log_enrichment.py` | Reconstructs `gen_ai.input/output.messages` from OTel log records into span attributes |
165
-
166
- ### Adding a new attribute constant
167
-
168
- Add it to `trace_attrs.py` and import from there. Don't use hardcoded attribute key strings elsewhere.
169
-
170
- ### Adding or modifying extraction logic
171
-
172
- The extraction functions in `extraction.py` accept flat `dict[str, Any]` attribute maps. This means they work with both `Span`-based batch converters (via `span.tags`) and the raw OTLP dict incremental processor. When extracting data, check ADK-specific attributes first (they contain richer data), then fall back to GenAI semconv.
173
-
174
- ### Supporting a new trace format
175
-
176
- 1. Add a new `TraceFormatExtractor` implementation in `extraction.py` with `detect()`, `find_invocation_spans()`, `find_llm_spans_in()`, `find_tool_spans_in()`, and `classify_span()`
177
- 2. Register it in the `_EXTRACTORS` list. Order matters here: more specific formats should come first so they get detected before the generic GenAI fallback
178
- 3. If the format introduces new attribute keys, add them to `trace_attrs.py`
179
- 4. If you need conversion logic that the shared extraction functions don't cover, add a dedicated converter module (see `genai_converter.py` for an example)
180
- 5. Add tests to `tests/test_extraction.py` for detection and span classification
157
+ | `otel/model.py`, `otel/decode.py`, `otel/encode.py` | Lossless OTLP envelopes (spans, logs, resources, scopes) decoded from protobuf, JSON or Jaeger files, and encoded back |
158
+ | `genai/overlay.py`, `genai/semconv.py` | Read only view of a span in current GenAI conventions, with fallbacks for deprecated keys, events, ADK and OpenLLMetry attributes |
159
+ | `genai/extract.py`, `genai/model.py` | Turns, logical model calls and tool calls extracted from traces (`extract_conversation`) |
160
+ | `genai/matching.py`, `genai/grouping.py` | Grouping traces into conversations and matching them to eval cases |
161
+ | `adk_bridge.py` | The only place that converts to ADK types (eval sets, built in metrics) |
162
+ | `otel/store.py`, `streaming/manager.py` | Live sessions: routing mechanics, limits, completion, recompute and UI updates |
163
+ | `genai/routing.py` | What the live store routes on: session keys, which traces and logs to keep, agentevals' own records |
164
+ | `evaluation_events.py` | Evaluation results as `gen_ai.evaluation.result` events |
165
+
166
+ Imports point down only: `otel/` imports nothing from `genai/`, and neither imports the runner, the evaluators or `evaluation_events.py`.
167
+
168
+ ### Supporting a new producer
169
+
170
+ 1. Record a few real traces and add them as fixtures with the turns you expect.
171
+ 2. If the producer uses standard GenAI attributes, it should already work. Check with `agentevals run`.
172
+ 3. If it uses its own attribute names, add a fallback in `genai/overlay.py`. Never change span classification for one producer; it keys on `gen_ai.operation.name`.
181
173
 
182
174
  ### Adding an SDK example
183
175
 
@@ -0,0 +1,309 @@
1
+ Metadata-Version: 2.5
2
+ Name: agentevals-cli
3
+ Version: 0.10.0
4
+ Summary: Standalone framework to evaluate agent correctness based on portable OpenTelemetry traces
5
+ License-File: LICENSE
6
+ Requires-Python: >=3.11
7
+ Requires-Dist: aiohttp>=3.14.3
8
+ Requires-Dist: authlib>=1.7.2
9
+ Requires-Dist: click>=8.3.3
10
+ Requires-Dist: fastapi>=0.115.0
11
+ Requires-Dist: google-adk[eval]<3,>=2.5.0
12
+ Requires-Dist: googleapis-common-protos>=1.72.0
13
+ Requires-Dist: grpcio>=1.66
14
+ Requires-Dist: httpx>=0.27.0
15
+ Requires-Dist: idna>=3.18
16
+ Requires-Dist: litellm>=1.85.7
17
+ Requires-Dist: nltk>=3.10.2
18
+ Requires-Dist: opentelemetry-exporter-otlp-proto-grpc>=1.38
19
+ Requires-Dist: opentelemetry-exporter-otlp-proto-http>=1.38
20
+ Requires-Dist: opentelemetry-proto>=1.36.0
21
+ Requires-Dist: opentelemetry-sdk>=1.38
22
+ Requires-Dist: pyasn1>=0.6.4
23
+ Requires-Dist: pygments>=2.20.0
24
+ Requires-Dist: python-dotenv>=1.2.2
25
+ Requires-Dist: python-multipart>=0.0.32
26
+ Requires-Dist: pyyaml>=6.0
27
+ Requires-Dist: requests>=2.34.2
28
+ Requires-Dist: tabulate>=0.9.0
29
+ Requires-Dist: urllib3>=2.7.0
30
+ Requires-Dist: uvicorn[standard]>=0.32.0
31
+ Provides-Extra: kubernetes
32
+ Requires-Dist: kubernetes>=36.0.0; extra == 'kubernetes'
33
+ Provides-Extra: live
34
+ Requires-Dist: httpx>=0.27.0; extra == 'live'
35
+ Requires-Dist: mcp<2,>=1.28.1; extra == 'live'
36
+ Requires-Dist: pydantic-settings>=2.15.0; extra == 'live'
37
+ Requires-Dist: pyjwt>=2.13.0; extra == 'live'
38
+ Provides-Extra: postgres
39
+ Requires-Dist: asyncpg>=0.30.0; extra == 'postgres'
40
+ Provides-Extra: streaming
41
+ Description-Content-Type: text/markdown
42
+
43
+ <p align="center">
44
+ <picture>
45
+ <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/agentevals-dev/agentevals/main/docs/assets/logo-color-on-transparent.svg">
46
+ <source media="(prefers-color-scheme: light)" srcset="https://raw.githubusercontent.com/agentevals-dev/agentevals/main/docs/assets/logo-dark-on-transparent.svg">
47
+ <img src="https://raw.githubusercontent.com/agentevals-dev/agentevals/main/docs/assets/logo-color-on-transparent.svg" alt="agentevals" width="420" />
48
+ </picture>
49
+ </p>
50
+
51
+ <h1 align="center">OpenTelemetry-native agent evaluation</h1>
52
+
53
+ <p align="center">
54
+ Send OTLP traces. Score what your agent did. Get the results back as OpenTelemetry GenAI eval events.<br>
55
+ No SDK, no reruns, no platform to host.
56
+ </p>
57
+
58
+ <p align="center">
59
+ <a href="https://github.com/agentevals-dev/agentevals/stargazers"><img src="https://img.shields.io/github/stars/agentevals-dev/agentevals?style=social" alt="GitHub Stars"></a>
60
+ &nbsp;
61
+ <a href="https://discord.gg/cpveEn8Ah2"><img src="https://img.shields.io/discord/1435836734666707190?label=Discord&logo=discord&logoColor=white&color=5865F2" alt="Discord"></a>
62
+ &nbsp;
63
+ <a href="https://github.com/agentevals-dev/agentevals/releases"><img src="https://img.shields.io/github/v/release/agentevals-dev/agentevals?label=Release" alt="Release"></a>
64
+ &nbsp;
65
+ <a href="https://github.com/agentevals-dev/agentevals/blob/main/LICENSE"><img src="https://img.shields.io/badge/License-Apache%202.0-green.svg" alt="License"></a>
66
+ &nbsp;
67
+ <a href="https://pypi.org/project/agentevals-cli/"><img src="https://img.shields.io/pypi/v/agentevals-cli?label=PyPI&color=blue" alt="PyPI"></a>
68
+ </p>
69
+
70
+ <p align="center">
71
+ <a href="#install">Install</a> · <a href="#quick-start">Quick start</a> · <a href="#docs">Docs</a> · <a href="CONTRIBUTING.md">Contributing</a> · <a href="https://discord.gg/cpveEn8Ah2">Discord</a>
72
+ </p>
73
+
74
+ ---
75
+
76
+ ## What is agentevals?
77
+
78
+ agentevals scores AI agent behavior from the OpenTelemetry telemetry your agent already produces. It is an OTLP destination like any tracing backend: point an exporter or a Collector at it, and it turns GenAI spans and log events into turns, tool calls, tokens and answers, then scores them against a golden eval set. Results can flow back into your pipeline as `gen_ai.evaluation.result` events.
79
+
80
+ * **Standard telemetry in.** OTLP over HTTP and gRPC, live or from files. Any producer that follows the [OpenTelemetry GenAI semantic conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/) works: Google ADK, Strands, LangChain, OpenAI Agents SDK, Pydantic AI and the official OpenTelemetry GenAI instrumentations.
81
+ * **Standard events out.** One `gen_ai.evaluation.result` log event per metric and turn, parented to the evaluated span, ready for your backend to query, alert on or turn into metrics.
82
+ * **No reruns.** Scoring reads recorded telemetry, so an evaluation costs no agent tokens and can be repeated on the same run.
83
+ * **Gates and judges.** Tool trajectory and response matching for deterministic pass/fail, LLM judges and rubrics for quality, custom evaluators in any language.
84
+ * **Local first.** One `pip install`. CLI for CI, web UI for inspection, MCP server for assistants, Helm chart for Kubernetes.
85
+
86
+ ```mermaid
87
+ flowchart LR
88
+ A[Agent with OTel instrumentation] -->|OTLP| C[OpenTelemetry Collector]
89
+ C --> B[Your tracing backend]
90
+ C --> E[agentevals]
91
+ E -->|gen_ai.evaluation.result| C
92
+ ```
93
+
94
+ The Collector is optional. An agent can export straight to agentevals.
95
+
96
+ > [!IMPORTANT]
97
+ > This project is under active development. Expect breaking changes.
98
+
99
+ ## Install
100
+
101
+ ```bash
102
+ pip install agentevals-cli # CLI, REST API, OTLP receivers, embedded web UI
103
+ pip install "agentevals-cli[live]" # adds the MCP server
104
+ ```
105
+
106
+ From a clone: `uv sync`, then use `uv run agentevals`. See [DEVELOPMENT.md](DEVELOPMENT.md).
107
+
108
+ ## Quick start
109
+
110
+ ### Score recorded traces
111
+
112
+ `samples/` holds traces from a Kubernetes Helm agent and eval sets that describe the expected behavior.
113
+
114
+ ```bash
115
+ agentevals run samples/helm.json \
116
+ --eval-set samples/eval_set_helm.json \
117
+ -m tool_trajectory_avg_score
118
+ ```
119
+
120
+ The eval set expects a call to `helm_list_releases`, and the trace has one:
121
+
122
+ ```
123
+ Trace: 3e289017fe03ffd7c4145316d2eb3d0d
124
+ Invocations: 1
125
+ Metric Score Status Per-Invocation Time
126
+ ------ ------------------------- ------- -------- ---------------- ------
127
+ [PASS] tool_trajectory_avg_score 1 PASSED 1 5ms
128
+ ```
129
+
130
+ A trace from a session that never called the tool fails, and the output says why:
131
+
132
+ ```bash
133
+ agentevals run samples/k8s.json \
134
+ --eval-set samples/eval_set_helm.json \
135
+ -m tool_trajectory_avg_score
136
+ ```
137
+
138
+ ```
139
+ [FAIL] tool_trajectory_avg_score 0 FAILED 0 1ms
140
+ Invocation 1 trajectory mismatch:
141
+ Expected:
142
+ - helm_list_releases({})
143
+ Actual:
144
+ (none)
145
+ ```
146
+
147
+ Add `-m response_match_score` to check the final answer as well, `--output json` for machine readable results, and `--emit-otel` to send the results into your pipeline.
148
+
149
+ ### Evaluate a live agent
150
+
151
+ ```bash
152
+ # Terminal 1
153
+ agentevals serve --dev
154
+
155
+ # Terminal 2
156
+ export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318
157
+ export OTEL_RESOURCE_ATTRIBUTES="agentevals.session_name=my-agent,service.instance.id=$(uuidgen)"
158
+ python your_agent.py
159
+ ```
160
+
161
+ Sessions appear in the UI at `http://localhost:8001` as the agent runs, with tool calls, inputs and outputs, and can be evaluated there. For gRPC exporters use port 4317 with `OTEL_EXPORTER_OTLP_PROTOCOL=grpc`.
162
+
163
+ Sessions are grouped by `agentevals.session_name`, then `gen_ai.conversation.id`, then `session.id`. `service.instance.id` tells a rerun from the next turn. Set `agentevals.eval_set_id` to associate a session with an eval set.
164
+
165
+ Per producer settings, such as turning on message content capture, are in [OpenTelemetry Compatibility](docs/otel-compatibility.md). Working setups for each framework are in [examples/zero-code-examples/](examples/zero-code-examples/).
166
+
167
+ ### Send results back to your pipeline
168
+
169
+ ```bash
170
+ agentevals run trace.json --eval-set eval_set.json -m tool_trajectory_avg_score --emit-otel
171
+ ```
172
+
173
+ The exporter is configured by the standard `OTEL_EXPORTER_OTLP_*` variables. `agentevals serve --emit-otel` does the same for evaluations started from the UI and the API. Collector recipes for forwarding telemetry to agentevals and for routing the results are in [OpenTelemetry Pipelines](docs/opentelemetry-pipeline.md).
174
+
175
+ ## What agentevals reads
176
+
177
+ Turns come from `invoke_agent` spans, tool calls from `execute_tool` spans, tokens and the model from model call spans, and messages from `gen_ai.input.messages` and `gen_ai.output.messages` as span attributes, span events or log events. Google ADK's `gcp.vertex.agent.*` attributes are read as well. Files can be OTLP JSON, Jaeger JSON or Collector `file` exporter output. Details, including what is ignored and the receiver limits: [OpenTelemetry Compatibility](docs/otel-compatibility.md).
178
+
179
+ ## SDK (optional)
180
+
181
+ The Python SDK wraps the OTel boilerplate in a session when you want the session name, eval set and run id set from code:
182
+
183
+ ```python
184
+ from agentevals import AgentEvals
185
+
186
+ app = AgentEvals()
187
+
188
+ with app.session(eval_set_id="my-eval"):
189
+ agent.invoke("Roll a 20-sided die for me")
190
+ ```
191
+
192
+ Spans and GenAI log events produced inside the session go to agentevals over OTLP. Your own exporters are untouched, and the SDK never reads `OTEL_EXPORTER_OTLP_*`, so credentials for another backend are not sent to agentevals. Endpoint: `endpoint=` or `AGENTEVALS_OTLP_ENDPOINT`, default `http://localhost:4318`. See [examples/sdk_example/](examples/sdk_example/).
193
+
194
+ ## Custom evaluators
195
+
196
+ An evaluator is any program that reads JSON from stdin and writes a score to stdout. Scaffold one and reference it next to the built in metrics:
197
+
198
+ ```bash
199
+ agentevals evaluator init my_evaluator
200
+ ```
201
+
202
+ ```yaml
203
+ evaluators:
204
+ - name: tool_trajectory_avg_score
205
+ type: builtin
206
+ - name: my_evaluator
207
+ type: code
208
+ path: ./evaluators/my_evaluator.py
209
+ threshold: 0.7
210
+ ```
211
+
212
+ `type: remote` pulls community evaluators from GitHub. Protocol, SDK helpers and runtimes: [Custom Evaluators](docs/custom-evaluators.md).
213
+
214
+ ## Run it
215
+
216
+ **CLI.** `agentevals run --help` lists everything: multiple traces, `--group-by trace|conversation`, `--trajectory-match-type`, judge model selection and eval config files. `agentevals evaluator list` shows built in and community evaluators.
217
+
218
+ **Web UI.** `agentevals serve` on `http://localhost:8001`. Upload traces and eval sets, pick evaluators, inspect span trees, watch live sessions. API docs at `/docs`.
219
+
220
+ **MCP server.** `agentevals mcp` exposes `list_metrics`, `evaluate_traces`, `list_sessions`, `summarize_session` and `evaluate_sessions` to MCP clients. A `.mcp.json` at the repo root lets Claude Code pick it up, and the `/eval` and `/inspect` skills in `.claude/skills/` build on it.
221
+
222
+ **Docker.**
223
+
224
+ ```bash
225
+ docker build -t agentevals .
226
+ docker run -p 8001:8001 -p 4317:4317 -p 4318:4318 agentevals
227
+ ```
228
+
229
+ | Port | Purpose |
230
+ |------|---------|
231
+ | 8001 | Web UI and REST API |
232
+ | 4317 | OTLP gRPC receiver (traces and logs) |
233
+ | 4318 | OTLP HTTP receiver (traces and logs) |
234
+ | 8080 | MCP (Streamable HTTP) |
235
+
236
+ **Helm.**
237
+
238
+ ```bash
239
+ helm install agentevals oci://ghcr.io/agentevals-dev/agentevals/helm/agentevals
240
+ ```
241
+
242
+ The chart runs in memory by default. `--set storage.backend=postgres` with a bundled or external Postgres enables the async `/api/runs` pipeline and the Run History view. That storage layer is a preview and its schema may change between minor versions. See [Run History](docs/run-history.md) and the [Kubernetes example](examples/kubernetes/README.md) for a full walkthrough with kagent and an OpenTelemetry Collector.
243
+
244
+ ## Examples
245
+
246
+ | Example | What it shows |
247
+ |---------|---------------|
248
+ | [ADK](examples/zero-code-examples/adk/) | Google ADK agent, standard OTLP export |
249
+ | [LangChain](examples/zero-code-examples/langchain/) | LangChain agent, OTLP traces and logs |
250
+ | [Strands](examples/zero-code-examples/strands/) | Strands agent, standard OTLP export |
251
+ | [OpenAI Agents](examples/zero-code-examples/openai-agents/) | OpenAI Agents SDK, standard OTLP export |
252
+ | [Pydantic AI](examples/zero-code-examples/pydantic-ai/) | Pydantic AI agent, standard OTLP export |
253
+ | [Ollama](examples/zero-code-examples/ollama/) | LangChain with a local model |
254
+ | [SDK sessions](examples/README.md#sdk-integration) | LangChain, Strands and ADK agents through `AgentEvals` sessions |
255
+ | [Kubernetes](examples/kubernetes/) | agentevals with kagent and an OpenTelemetry Collector |
256
+
257
+ ## Docs
258
+
259
+ | Guide | Description |
260
+ |-------|-------------|
261
+ | [OpenTelemetry Compatibility](docs/otel-compatibility.md) | What to send, per framework setup, sessions, receiver limits |
262
+ | [OpenTelemetry Pipelines](docs/opentelemetry-pipeline.md) | Collector recipes and evaluation results as OpenTelemetry events |
263
+ | [Eval Set Format](docs/eval-set-format.md) | Schema and examples for golden eval sets |
264
+ | [Custom Evaluators](docs/custom-evaluators.md) | Write scoring logic in any language |
265
+ | [Run History](docs/run-history.md) | Persist evaluations to Postgres and trend them in the UI |
266
+
267
+ ## Development
268
+
269
+ ```bash
270
+ uv run pytest # tests
271
+ uv run agentevals serve --dev # backend
272
+ cd ui && npm run dev # frontend, separate terminal
273
+ ```
274
+
275
+ See [DEVELOPMENT.md](DEVELOPMENT.md) and [CONTRIBUTING.md](CONTRIBUTING.md).
276
+
277
+ ## FAQ
278
+
279
+ **Do I need a database or any infrastructure?**
280
+
281
+ No. The CLI evaluates trace files from disk. The server keeps live sessions in memory. Postgres is optional and only backs run history.
282
+
283
+ **Does the CLI need a running server?**
284
+
285
+ No. `agentevals run` works offline. The server is for the web UI, live OTLP sessions and the MCP tools that read sessions.
286
+
287
+ **Can I use it in CI?**
288
+
289
+ Yes. Pass trace files and an eval set, set a threshold, and let the exit code gate the deployment. `--output json` gives machine readable results.
290
+
291
+ **Can I plug it into an existing OpenTelemetry pipeline?**
292
+
293
+ Yes. agentevals accepts OTLP over HTTP (`http/protobuf` and `http/json`, gzip included) and gRPC, so a default Collector exporter works. With `--emit-otel` it sends evaluation results back as `gen_ai.evaluation.result` events. See [OpenTelemetry Pipelines](docs/opentelemetry-pipeline.md).
294
+
295
+ **Can I evaluate Claude Code, Codex or OpenCode?**
296
+
297
+ Partly. They do not emit GenAI semantic convention spans, so agentevals sees a turn only when something wraps it in one, as kagent does for its Claude Code harness. A Collector recipe can then map Claude Code's own spans to tool and model calls, see the [Kubernetes example](examples/kubernetes/README.md#claude-code-workaround). agentevals is built for instrumented agents whose success is measurable through tool trajectories and responses, not for scoring long coding sessions end to end.
298
+
299
+ **How is this different from LangSmith, Langfuse, Opik or Bedrock AgentCore evaluation?**
300
+
301
+ Those run your agent again against a dataset, or need their own SDK and storage before anything can be scored. agentevals scores the OpenTelemetry telemetry you already have, from any instrumentation, without rerunning the agent and without a platform to host.
302
+
303
+ **How does this compare to ADK's evaluations?**
304
+
305
+ ADK eval runs the agent and scores it in one step. agentevals only scores, from telemetry, and uses ADK's metrics and eval set format to do it. Point an ADK agent at agentevals and you get live inspection and scoring without changing the agent.
306
+
307
+ ## Acknowledgements
308
+
309
+ The built in metrics and the eval set format come from [Google's Agent Development Kit](https://github.com/google/adk-python), licensed under [Apache 2.0](https://github.com/google/adk-python/blob/main/LICENSE) like agentevals. Thanks to the ADK team and contributors.