agentevals-cli 0.10.0__tar.gz → 0.11.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- agentevals_cli-0.11.0/.gitattributes +5 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/.github/workflows/ci.yml +35 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/.github/workflows/release.yml +27 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/DEVELOPMENT.md +15 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/Dockerfile +1 -1
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/Makefile +38 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/PKG-INFO +25 -3
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/README.md +18 -1
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/charts/agentevals/templates/NOTES.txt +4 -1
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/charts/agentevals/templates/deployment.yaml +16 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/charts/agentevals/templates/service.yaml +7 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/charts/agentevals/values.yaml +18 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/docs/custom-evaluators.md +15 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/docs/eval-set-format.md +47 -0
- agentevals_cli-0.11.0/docs/grpc-api.md +71 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/flake.nix +3 -0
- agentevals_cli-0.11.0/gen/go/README.md +10 -0
- agentevals_cli-0.11.0/gen/go/agentevals/api/v1alpha1/common.pb.go +501 -0
- agentevals_cli-0.11.0/gen/go/agentevals/api/v1alpha1/conversion.pb.go +263 -0
- agentevals_cli-0.11.0/gen/go/agentevals/api/v1alpha1/conversion_grpc.pb.go +123 -0
- agentevals_cli-0.11.0/gen/go/agentevals/api/v1alpha1/evaluation.pb.go +1845 -0
- agentevals_cli-0.11.0/gen/go/agentevals/api/v1alpha1/evaluation_grpc.pb.go +169 -0
- agentevals_cli-0.11.0/gen/go/agentevals/api/v1alpha1/runs.pb.go +1794 -0
- agentevals_cli-0.11.0/gen/go/agentevals/api/v1alpha1/runs_grpc.pb.go +331 -0
- agentevals_cli-0.11.0/gen/go/agentevals/api/v1alpha1/sessions.pb.go +552 -0
- agentevals_cli-0.11.0/gen/go/agentevals/api/v1alpha1/sessions_grpc.pb.go +211 -0
- agentevals_cli-0.11.0/gen/go/agentevals/api/v1alpha1/system.pb.go +555 -0
- agentevals_cli-0.11.0/gen/go/agentevals/api/v1alpha1/system_grpc.pb.go +159 -0
- agentevals_cli-0.11.0/gen/go/agentevals/api/v1alpha1/views.pb.go +1491 -0
- agentevals_cli-0.11.0/gen/go/go.mod +16 -0
- agentevals_cli-0.11.0/gen/go/go.sum +38 -0
- agentevals_cli-0.11.0/proto/agentevals/api/v1alpha1/common.proto +64 -0
- agentevals_cli-0.11.0/proto/agentevals/api/v1alpha1/conversion.proto +33 -0
- agentevals_cli-0.11.0/proto/agentevals/api/v1alpha1/evaluation.proto +218 -0
- agentevals_cli-0.11.0/proto/agentevals/api/v1alpha1/runs.proto +201 -0
- agentevals_cli-0.11.0/proto/agentevals/api/v1alpha1/sessions.proto +59 -0
- agentevals_cli-0.11.0/proto/agentevals/api/v1alpha1/system.proto +57 -0
- agentevals_cli-0.11.0/proto/agentevals/api/v1alpha1/views.proto +213 -0
- agentevals_cli-0.11.0/proto/buf.gen.yaml +25 -0
- agentevals_cli-0.11.0/proto/buf.lock +6 -0
- agentevals_cli-0.11.0/proto/buf.yaml +12 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/pyproject.toml +9 -1
- agentevals_cli-0.11.0/src/agentevals/_static/assets/index-ibYF9EKo.js +349 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/_static/index.html +1 -1
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/adk_bridge.py +12 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/api/app.py +44 -16
- agentevals_cli-0.11.0/src/agentevals/api/evaluation.py +193 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/api/models.py +0 -23
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/api/otlp_grpc.py +6 -1
- agentevals_cli-0.11.0/src/agentevals/api/redact.py +77 -0
- agentevals_cli-0.11.0/src/agentevals/api/routes.py +648 -0
- agentevals_cli-0.11.0/src/agentevals/api/rpc/__init__.py +1 -0
- agentevals_cli-0.11.0/src/agentevals/api/rpc/convert.py +431 -0
- agentevals_cli-0.11.0/src/agentevals/api/rpc/policy.py +152 -0
- agentevals_cli-0.11.0/src/agentevals/api/rpc/server.py +100 -0
- agentevals_cli-0.11.0/src/agentevals/api/rpc/services.py +401 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/api/runs_routes.py +6 -5
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/api/streaming_routes.py +1 -3
- agentevals_cli-0.11.0/src/agentevals/api/v1alpha1/common_pb2.py +51 -0
- agentevals_cli-0.11.0/src/agentevals/api/v1alpha1/common_pb2.pyi +72 -0
- agentevals_cli-0.11.0/src/agentevals/api/v1alpha1/common_pb2_grpc.py +4 -0
- agentevals_cli-0.11.0/src/agentevals/api/v1alpha1/conversion_pb2.py +46 -0
- agentevals_cli-0.11.0/src/agentevals/api/v1alpha1/conversion_pb2.pyi +34 -0
- agentevals_cli-0.11.0/src/agentevals/api/v1alpha1/conversion_pb2_grpc.py +78 -0
- agentevals_cli-0.11.0/src/agentevals/api/v1alpha1/evaluation_pb2.py +83 -0
- agentevals_cli-0.11.0/src/agentevals/api/v1alpha1/evaluation_pb2.pyi +269 -0
- agentevals_cli-0.11.0/src/agentevals/api/v1alpha1/evaluation_pb2_grpc.py +123 -0
- agentevals_cli-0.11.0/src/agentevals/api/v1alpha1/runs_pb2.py +95 -0
- agentevals_cli-0.11.0/src/agentevals/api/v1alpha1/runs_pb2.pyi +274 -0
- agentevals_cli-0.11.0/src/agentevals/api/v1alpha1/runs_pb2_grpc.py +303 -0
- agentevals_cli-0.11.0/src/agentevals/api/v1alpha1/sessions_pb2.py +55 -0
- agentevals_cli-0.11.0/src/agentevals/api/v1alpha1/sessions_pb2.pyi +63 -0
- agentevals_cli-0.11.0/src/agentevals/api/v1alpha1/sessions_pb2_grpc.py +171 -0
- agentevals_cli-0.11.0/src/agentevals/api/v1alpha1/system_pb2.py +52 -0
- agentevals_cli-0.11.0/src/agentevals/api/v1alpha1/system_pb2.pyi +79 -0
- agentevals_cli-0.11.0/src/agentevals/api/v1alpha1/system_pb2_grpc.py +120 -0
- agentevals_cli-0.11.0/src/agentevals/api/v1alpha1/views_pb2.py +64 -0
- agentevals_cli-0.11.0/src/agentevals/api/v1alpha1/views_pb2.pyi +247 -0
- agentevals_cli-0.11.0/src/agentevals/api/v1alpha1/views_pb2_grpc.py +4 -0
- agentevals_cli-0.11.0/src/agentevals/api/views.py +341 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/builtin_metrics.py +175 -12
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/cli.py +42 -31
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/config.py +55 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/custom_evaluators.py +3 -1
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/evaluation_events.py +6 -58
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/genai/extract.py +47 -16
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/genai/model.py +31 -7
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/genai/routing.py +11 -2
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/loader/__init__.py +2 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/loader/auto.py +8 -3
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/mcp_server.py +1 -1
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/otel/decode.py +1 -1
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/otel/identity.py +4 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/otel/store.py +13 -4
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/resolvers/kubernetes.py +1 -2
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/run/fetcher.py +12 -1
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/run/service.py +4 -1
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/run/sinks.py +12 -6
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/runner.py +1 -1
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/storage/postgres/migrations/000001_init.up.sql +3 -3
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/storage/repos/__init__.py +2 -2
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/storage/repos/postgres.py +16 -1
- agentevals_cli-0.11.0/src/agentevals/streaming/manager.py +450 -0
- agentevals_cli-0.11.0/src/agentevals/telemetry.py +110 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/trace_metrics.py +5 -5
- agentevals_cli-0.11.0/tests/api/test_api_tracing.py +95 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/api/test_evaluate_persistence.py +18 -2
- agentevals_cli-0.11.0/tests/api/test_grpc_api.py +741 -0
- agentevals_cli-0.11.0/tests/api/test_redact.py +78 -0
- agentevals_cli-0.11.0/tests/api/test_rpc_convert.py +202 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/api/test_runs_routes.py +76 -1
- agentevals_cli-0.11.0/tests/api/test_views.py +232 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/adk_cycle_poisons_batch.truth.json +2 -2
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/adk_go_main.truth.json +1 -1
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/openai_agents_handoff.truth.json +1 -1
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/self_parent.truth.json +1 -1
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/strands_span_events.truth.json +1 -1
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/two_span_cycle.truth.json +2 -2
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/upstream_new_parts.truth.json +1 -1
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/samples/tempo_export_with_batches.truth.json +1 -1
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/genai/test_continuations.py +2 -2
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/genai/test_corpus.py +3 -3
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/genai/test_failed_turns.py +9 -7
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/genai/test_tools.py +2 -2
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/integration/conftest.py +1 -1
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/integration/test_live_agents.py +18 -16
- agentevals_cli-0.11.0/tests/otel/test_own_spans.py +47 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/resolvers/test_kubernetes.py +4 -3
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/run/test_fetcher.py +36 -1
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/run/test_result_builder.py +1 -1
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/run/test_service.py +1 -1
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/run/test_sinks.py +38 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/storage/test_memory_repos.py +1 -1
- agentevals_cli-0.11.0/tests/storage/test_postgres_repos.py +136 -0
- agentevals_cli-0.11.0/tests/streaming/test_live.py +105 -0
- agentevals_cli-0.11.0/tests/streaming/test_watchers.py +122 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/test_api.py +29 -39
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/test_builtin_metrics.py +9 -1
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/test_otlp_loader.py +1 -1
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/test_otlp_receiver.py +50 -43
- agentevals_cli-0.11.0/tests/test_rubrics.py +441 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/.gitignore +1 -1
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/eslint.config.js +1 -1
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/package-lock.json +261 -234
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/package.json +1 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/api/client.ts +0 -67
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/annotation-queue/AnnotationDetailPanel.tsx +10 -72
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/annotation-queue/AnnotationQueueView.tsx +2 -2
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/annotation-queue/AnnotationTable.tsx +16 -45
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/bug-report/BugReportModal.tsx +3 -3
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/dashboard/TraceCard.tsx +2 -2
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/dashboard/TraceTable.tsx +5 -5
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/inspector/InvocationSummaryPanel.tsx +2 -2
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/inspector/MetricsComparisonSection.tsx +2 -2
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/inspector/ToolCallList.tsx +5 -2
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/runs/RunDetailView.tsx +2 -1
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/runs/RunsView.tsx +15 -14
- agentevals_cli-0.11.0/ui/src/components/streaming/LiveConversationPanel.tsx +92 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/streaming/LiveMessage.tsx +12 -10
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/streaming/LiveStreamingView.tsx +62 -363
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/streaming/SessionCard.tsx +35 -231
- agentevals_cli-0.11.0/ui/src/components/streaming/SessionMetadata.tsx +88 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/upload/EvalSetEditorDrawer.tsx +22 -8
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/upload/TraceEditorDrawer.tsx +17 -13
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/config.ts +0 -6
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/context/TraceContext.tsx +5 -4
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/context/TraceProvider.tsx +16 -18
- agentevals_cli-0.11.0/ui/src/generated/agentevals/api/v1alpha1/common_pb.d.ts +212 -0
- agentevals_cli-0.11.0/ui/src/generated/agentevals/api/v1alpha1/common_pb.js +75 -0
- agentevals_cli-0.11.0/ui/src/generated/agentevals/api/v1alpha1/conversion_pb.d.ts +100 -0
- agentevals_cli-0.11.0/ui/src/generated/agentevals/api/v1alpha1/conversion_pb.js +42 -0
- agentevals_cli-0.11.0/ui/src/generated/agentevals/api/v1alpha1/evaluation_pb.d.ts +814 -0
- agentevals_cli-0.11.0/ui/src/generated/agentevals/api/v1alpha1/evaluation_pb.js +170 -0
- agentevals_cli-0.11.0/ui/src/generated/agentevals/api/v1alpha1/runs_pb.d.ts +785 -0
- agentevals_cli-0.11.0/ui/src/generated/agentevals/api/v1alpha1/runs_pb.js +205 -0
- agentevals_cli-0.11.0/ui/src/generated/agentevals/api/v1alpha1/sessions_pb.d.ts +212 -0
- agentevals_cli-0.11.0/ui/src/generated/agentevals/api/v1alpha1/sessions_pb.js +79 -0
- agentevals_cli-0.11.0/ui/src/generated/agentevals/api/v1alpha1/system_pb.d.ts +229 -0
- agentevals_cli-0.11.0/ui/src/generated/agentevals/api/v1alpha1/system_pb.js +68 -0
- agentevals_cli-0.11.0/ui/src/generated/agentevals/api/v1alpha1/views_pb.d.ts +737 -0
- agentevals_cli-0.11.0/ui/src/generated/agentevals/api/v1alpha1/views_pb.js +109 -0
- agentevals_cli-0.11.0/ui/src/generated/opentelemetry/proto/common/v1/common_pb.d.ts +278 -0
- agentevals_cli-0.11.0/ui/src/generated/opentelemetry/proto/common/v1/common_pb.js +68 -0
- agentevals_cli-0.11.0/ui/src/generated/opentelemetry/proto/logs/v1/logs_pb.d.ts +461 -0
- agentevals_cli-0.11.0/ui/src/generated/opentelemetry/proto/logs/v1/logs_pb.js +91 -0
- agentevals_cli-0.11.0/ui/src/generated/opentelemetry/proto/resource/v1/resource_pb.d.ts +69 -0
- agentevals_cli-0.11.0/ui/src/generated/opentelemetry/proto/resource/v1/resource_pb.js +34 -0
- agentevals_cli-0.11.0/ui/src/generated/opentelemetry/proto/trace/v1/trace_pb.d.ts +646 -0
- agentevals_cli-0.11.0/ui/src/generated/opentelemetry/proto/trace/v1/trace_pb.js +134 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/lib/console-capture.ts +2 -2
- agentevals_cli-0.11.0/ui/src/lib/liveSessions.ts +77 -0
- agentevals_cli-0.11.0/ui/src/lib/toolPairing.ts +22 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/lib/trace-loader.ts +55 -44
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/lib/trace-patcher.ts +39 -24
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/lib/types.ts +17 -66
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/lib/utils.ts +4 -4
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/uv.lock +52 -1
- agentevals_cli-0.10.0/src/agentevals/_static/assets/index-DLtCJiNw.js +0 -348
- agentevals_cli-0.10.0/src/agentevals/api/routes.py +0 -1029
- agentevals_cli-0.10.0/src/agentevals/streaming/manager.py +0 -582
- agentevals_cli-0.10.0/tests/streaming/test_live.py +0 -224
- agentevals_cli-0.10.0/ui/src/components/streaming/LiveConversationPanel.tsx +0 -125
- agentevals_cli-0.10.0/ui/src/components/streaming/SessionMetadata.tsx +0 -95
- agentevals_cli-0.10.0/ui/src/components/streaming/turnStatus.ts +0 -31
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/.claude/skills/eval/SKILL.md +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/.claude/skills/eval/evals/evals.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/.claude/skills/inspect/SKILL.md +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/.claude/skills/inspect/evals/evals.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/.dockerignore +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/.github/ISSUE_TEMPLATE/bug_report.yml +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/.github/ISSUE_TEMPLATE/config.yml +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/.github/ISSUE_TEMPLATE/feature_request.yml +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/.github/dependabot.yml +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/.github/workflows/kagent-contract-policy.yml +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/.github/workflows/kagent-contract.yml +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/.github/workflows/publish-evaluator-sdk.yml +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/.gitignore +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/.mcp.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/CONTRIBUTING.md +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/LICENSE +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/charts/agentevals/Chart.yaml +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/charts/agentevals/templates/_helpers.tpl +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/charts/agentevals/templates/postgresql-secret.yaml +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/charts/agentevals/templates/postgresql.yaml +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/charts/agentevals/templates/rbac.yaml +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/charts/agentevals/templates/serviceaccount.yaml +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/docs/assets/logo-color-on-transparent.svg +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/docs/assets/logo-color.png +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/docs/assets/logo-dark-on-transparent.svg +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/docs/opentelemetry-pipeline.md +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/docs/otel-compatibility.md +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/docs/run-history.md +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/examples/README.md +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/examples/custom_evaluators/eval_config.yaml +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/examples/custom_evaluators/response_quality.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/examples/custom_evaluators/tool_call_checker.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/examples/custom_sink/README.md +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/examples/custom_sink/agentevals_example_custom_sink/__init__.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/examples/custom_sink/agentevals_example_custom_sink/sink.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/examples/custom_sink/pyproject.toml +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/examples/dice_agent/README.md +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/examples/dice_agent/agent.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/examples/dice_agent/eval_set.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/examples/dice_agent/main.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/examples/dice_agent/test_streaming.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/examples/kubernetes/README.md +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/examples/kubernetes/agentevals-values.yaml +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/examples/kubernetes/agents.yaml +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/examples/kubernetes/otel-collector.yaml +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/examples/langchain_agent/README.md +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/examples/langchain_agent/agent.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/examples/langchain_agent/eval_set.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/examples/langchain_agent/main.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/examples/langchain_agent/requirements.txt +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/examples/langchain_agent/test_streaming.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/examples/sdk_example/async_example.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/examples/sdk_example/context_manager_example.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/examples/sdk_example/decorator_example.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/examples/sdk_example/requirements.txt +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/examples/strands_agent/agent.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/examples/strands_agent/eval_set.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/examples/strands_agent/main.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/examples/strands_agent/requirements.txt +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/examples/zero-code-examples/adk/requirements.txt +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/examples/zero-code-examples/adk/run.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/examples/zero-code-examples/langchain/requirements.txt +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/examples/zero-code-examples/langchain/run.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/examples/zero-code-examples/ollama/requirements.txt +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/examples/zero-code-examples/ollama/run.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/examples/zero-code-examples/openai-agents/requirements.txt +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/examples/zero-code-examples/openai-agents/run.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/examples/zero-code-examples/pydantic-ai/requirements.txt +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/examples/zero-code-examples/pydantic-ai/run.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/examples/zero-code-examples/strands/requirements.txt +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/examples/zero-code-examples/strands/run.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/flake.lock +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/packages/evaluator-sdk-py/README.md +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/packages/evaluator-sdk-py/pyproject.toml +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/packages/evaluator-sdk-py/src/agentevals_evaluator_sdk/__init__.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/packages/evaluator-sdk-py/src/agentevals_evaluator_sdk/decorator.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/packages/evaluator-sdk-py/src/agentevals_evaluator_sdk/types.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/samples/eval_set_helm.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/samples/evalset_helm_3_2026-02-23.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/samples/evalset_k8s_2026-02-20.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/samples/helm.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/samples/helm_2.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/samples/helm_3.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/samples/k8s.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/samples/tempo_export_with_batches.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/__init__.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/_protocol.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/_static/assets/index-BqibLiHO.css +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/_static/logo.svg +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/_static/vite.svg +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/api/__init__.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/api/debug_routes.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/api/dependencies.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/api/otlp_app.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/api/otlp_http.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/api/otlp_processing.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/api/otlp_routes.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/eval_config_loader.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/evaluator/__init__.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/evaluator/resolver.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/evaluator/sources.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/evaluator/templates.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/evaluator/venv.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/genai/__init__.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/genai/grouping.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/genai/matching.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/genai/messages.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/genai/overlay.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/genai/semconv.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/loader/jaeger.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/loader/otlp.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/otel/__init__.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/otel/encode.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/otel/model.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/otel/sdk_export.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/output.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/resolvers/__init__.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/run/__init__.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/run/result_builder.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/run/worker.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/sdk.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/storage/__init__.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/storage/config.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/storage/models.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/storage/postgres/__init__.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/storage/postgres/migrations/000001_init.down.sql +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/storage/postgres/migrations/000002_drop_openai_eval.down.sql +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/storage/postgres/migrations/000002_drop_openai_eval.up.sql +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/storage/postgres/migrator.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/storage/postgres/pool.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/storage/repos/memory.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/streaming/__init__.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/streaming/exports.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/streaming/session.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/utils/__init__.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/src/agentevals/utils/log_buffer.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/api/__init__.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/api/test_live_routes.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/contract/kagent/README.md +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/contract/kagent/check.sh +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/contract/kagent/policy/agentevals.rego +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/contract/kagent/registries/main-2cc3ffae/a2a.yaml +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/contract/kagent/registries/main-2cc3ffae/imports.yaml +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/contract/kagent/registries/main-2cc3ffae/kagent.yaml +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/contract/kagent/registries/main-2cc3ffae/manifest.yaml +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/contract/kagent/registries/main-2cc3ffae/signals.yaml +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/contract/kagent/registries/pr3121-412ef54e/a2a.yaml +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/contract/kagent/registries/pr3121-412ef54e/imports.yaml +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/contract/kagent/registries/pr3121-412ef54e/kagent.yaml +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/contract/kagent/registries/pr3121-412ef54e/manifest.yaml +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/contract/kagent/registries/pr3121-412ef54e/signals.yaml +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/contract/kagent/registries/v1.0.0-alpha9/a2a.yaml +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/contract/kagent/registries/v1.0.0-alpha9/kagent.yaml +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/contract/kagent/registries/v1.0.0-alpha9/manifest.yaml +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/contract/kagent/registries/v1.0.0-alpha9/signals.yaml +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/contract/kagent/test.sh +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/contract/kagent/testdata/agentevals_contract_gone/manifest.yaml +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/contract/kagent/testdata/agentevals_contract_gone/model.yaml +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/contract/kagent/testdata/agentevals_contract_missing_attribute/manifest.yaml +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/contract/kagent/testdata/agentevals_contract_missing_attribute/model.yaml +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/contract/kagent/testdata/agentevals_contract_requirement_floor/manifest.yaml +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/contract/kagent/testdata/agentevals_contract_requirement_floor/model.yaml +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/contract/kagent/weaver.sh +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/MANIFEST.md +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_default/S1.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_default/S1.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_default/S2.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_default/S2.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_default/S3.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_default/S3.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_default/S4a.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_default/S4a.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_default/S4b.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_default/S4b.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_default/S5.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_default/S5.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_default/S6a.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_default/S6a.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_default/S6b.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_default/S6b.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_default_contrib/S1.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_default_contrib/S1.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_default_contrib/S2.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_default_contrib/S2.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_default_contrib/S3.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_default_contrib/S3.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_default_contrib/S4a.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_default_contrib/S4a.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_default_contrib/S4b.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_default_contrib/S4b.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_default_contrib/S5.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_default_contrib/S5.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_default_contrib/S6a.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_default_contrib/S6a.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_default_contrib/S6b.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_default_contrib/S6b.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_event_stable/S1.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_event_stable/S1.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_event_stable/S2.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_event_stable/S2.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_event_stable/S3.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_event_stable/S3.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_event_stable/S4a.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_event_stable/S4a.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_event_stable/S4b.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_event_stable/S4b.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_event_stable/S5.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_event_stable/S5.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_event_stable/S6a.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_event_stable/S6a.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_event_stable/S6b.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_event_stable/S6b.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_event_stable_contrib/S1.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_event_stable_contrib/S1.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_event_stable_contrib/S2.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_event_stable_contrib/S2.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_event_stable_contrib/S3.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_event_stable_contrib/S3.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_event_stable_contrib/S4a.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_event_stable_contrib/S4a.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_event_stable_contrib/S4b.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_event_stable_contrib/S4b.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_event_stable_contrib/S5.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_event_stable_contrib/S5.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_event_stable_contrib/S6a.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_event_stable_contrib/S6a.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_event_stable_contrib/S6b.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_event_stable_contrib/S6b.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_nolegacy/S1.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_nolegacy/S1.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_nolegacy/S2.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_nolegacy/S2.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_nolegacy/S3.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_nolegacy/S3.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_nolegacy/S4a.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_nolegacy/S4a.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_nolegacy/S4b.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_nolegacy/S4b.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_nolegacy/S5.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_nolegacy/S5.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_nolegacy/S6a.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_nolegacy/S6a.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_nolegacy/S6b.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v1_nolegacy/S6b.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_default/S1.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_default/S1.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_default/S2.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_default/S2.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_default/S3.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_default/S3.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_default/S4a.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_default/S4a.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_default/S4b.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_default/S4b.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_default/S5.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_default/S5.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_default/S6a.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_default/S6a.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_default/S6b.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_default/S6b.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_event_only/S1.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_event_only/S1.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_event_only/S2.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_event_only/S2.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_event_only/S3.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_event_only/S3.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_event_only/S4a.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_event_only/S4a.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_event_only/S4b.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_event_only/S4b.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_event_only/S5.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_event_only/S5.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_event_only/S6a.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_event_only/S6a.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_event_only/S6b.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_event_only/S6b.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_event_only_nolegacy/S1.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_event_only_nolegacy/S1.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_event_only_nolegacy/S2.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_event_only_nolegacy/S2.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_event_only_nolegacy/S3.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_event_only_nolegacy/S3.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_event_only_nolegacy/S4a.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_event_only_nolegacy/S4a.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_event_only_nolegacy/S4b.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_event_only_nolegacy/S4b.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_event_only_nolegacy/S5.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_event_only_nolegacy/S5.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_event_only_nolegacy/S6a.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_event_only_nolegacy/S6a.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_event_only_nolegacy/S6b.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_event_only_nolegacy/S6b.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_no_content/S1.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_no_content/S1.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_no_content/S2.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_no_content/S2.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_no_content/S3.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_no_content/S3.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_no_content/S4a.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_no_content/S4a.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_no_content/S4b.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_no_content/S4b.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_no_content/S5.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_no_content/S5.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_no_content/S6a.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_no_content/S6a.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_no_content/S6b.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_no_content/S6b.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_and_event/S1.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_and_event/S1.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_and_event/S2.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_and_event/S2.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_and_event/S3.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_and_event/S3.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_and_event/S4a.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_and_event/S4a.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_and_event/S4b.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_and_event/S4b.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_and_event/S5.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_and_event/S5.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_and_event/S6a.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_and_event/S6a.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_and_event/S6b.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_and_event/S6b.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only/S1.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only/S1.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only/S2.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only/S2.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only/S3.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only/S3.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only/S4a.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only/S4a.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only/S4b.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only/S4b.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only/S5.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only/S5.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only/S6a.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only/S6a.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only/S6b.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only/S6b.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only_contrib/S1.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only_contrib/S1.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only_contrib/S2.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only_contrib/S2.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only_contrib/S3.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only_contrib/S3.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only_contrib/S4a.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only_contrib/S4a.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only_contrib/S4b.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only_contrib/S4b.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only_contrib/S5.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only_contrib/S5.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only_contrib/S6a.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only_contrib/S6a.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only_contrib/S6b.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only_contrib/S6b.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only_nolegacy/S1.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only_nolegacy/S1.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only_nolegacy/S2.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only_nolegacy/S2.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only_nolegacy/S3.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only_nolegacy/S3.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only_nolegacy/S4a.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only_nolegacy/S4a.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only_nolegacy/S4b.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only_nolegacy/S4b.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only_nolegacy/S5.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only_nolegacy/S5.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only_nolegacy/S6a.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only_nolegacy/S6a.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only_nolegacy/S6b.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2_span_only_nolegacy/S6b.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_event_only/S1.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_event_only/S1.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_event_only/S2.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_event_only/S2.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_event_only/S3.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_event_only/S3.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_event_only/S4a.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_event_only/S4a.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_event_only/S4b.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_event_only/S4b.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_event_only/S5.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_event_only/S5.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_event_only/S6a.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_event_only/S6a.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_event_only/S6b.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_event_only/S6b.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_no_content/S1.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_no_content/S1.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_no_content/S2.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_no_content/S2.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_no_content/S3.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_no_content/S3.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_no_content/S4a.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_no_content/S4a.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_no_content/S4b.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_no_content/S4b.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_no_content/S5.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_no_content/S5.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_no_content/S6a.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_no_content/S6a.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_no_content/S6b.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_no_content/S6b.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_span_only/S1.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_span_only/S1.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_span_only/S2.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_span_only/S2.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_span_only/S3.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_span_only/S3.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_span_only/S4a.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_span_only/S4a.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_span_only/S4b.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_span_only/S4b.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_span_only/S5.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_span_only/S5.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_span_only/S6a.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_span_only/S6a.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_span_only/S6b.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_span_only/S6b.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_span_only_contrib/S1.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_span_only_contrib/S1.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_span_only_contrib/S2.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_span_only_contrib/S2.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_span_only_contrib/S3.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_span_only_contrib/S3.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_span_only_contrib/S4a.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_span_only_contrib/S4a.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_span_only_contrib/S4b.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_span_only_contrib/S4b.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_span_only_contrib/S5.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_span_only_contrib/S5.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_span_only_contrib/S6a.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_span_only_contrib/S6a.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_span_only_contrib/S6b.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/adk_py/v2proj_span_only_contrib/S6b.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/adk_cycle_poisons_batch.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/adk_go_main.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/content_capture_off.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/content_capture_off.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/deprecated_aliases.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/deprecated_aliases.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/details_event_log_debug.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/details_event_log_debug.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/duplicate_span_ids.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/duplicate_span_ids.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/ext_eval_eventName.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/ext_eval_eventName.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/ext_eval_event_name_attr.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/ext_eval_event_name_attr.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/ext_eval_otel_event_name.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/ext_eval_otel_event_name.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/ext_eval_span_event.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/ext_eval_span_event.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/failed_inference_then_retry.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/failed_inference_then_retry.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/mcp_tool_call.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/mcp_tool_call.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/openai_agents_handoff.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/openai_v2_logs_no_span_id_ambiguous.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/openai_v2_logs_no_span_id_ambiguous.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/openai_v2_logs_no_span_id_single.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/openai_v2_logs_no_span_id_single.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/openai_v2_logs_span_id.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/openai_v2_logs_span_id.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/remote_parent_root.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/remote_parent_root.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/self_parent.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/strands_span_events.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/three_trace_conversation.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/three_trace_conversation.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/tool_error_no_error_type.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/tool_error_no_error_type.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/two_span_cycle.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/upstream_new_parts.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/util_genai_openai_tool_loop.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/hand/util_genai_openai_tool_loop.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/kagent/v1.0.0-alpha9/README.md +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/kagent/v1.0.0-alpha9/adk_go_1turn.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/kagent/v1.0.0-alpha9/adk_go_1turn.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/kagent/v1.0.0-alpha9/adk_go_2turn.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/kagent/v1.0.0-alpha9/adk_go_2turn.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/kagent/v1.0.0-alpha9/claude_native_1turn_subagent.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/kagent/v1.0.0-alpha9/claude_native_1turn_subagent.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/kagent/v1.0.0-alpha9/claude_native_2turn.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/kagent/v1.0.0-alpha9/claude_native_2turn.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/kagent/v1.0.0-alpha9/claude_recipe_2turn.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/kagent/v1.0.0-alpha9/claude_recipe_2turn.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/samples/helm.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/samples/helm.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/samples/helm_2.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/samples/helm_2.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/samples/helm_3.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/samples/helm_3.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/samples/k8s.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/samples/k8s.truth.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/fixtures/otlp/samples/tempo_export_with_batches.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/genai/__init__.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/genai/test_grouping.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/genai/test_legacy_chat.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/integration/__init__.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/integration/test_evaluation_pipeline.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/integration/test_otlp_grpc_receiver.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/integration/test_session_grouping.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/integration/test_timing_stress.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/otel/__init__.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/otel/builders.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/otel/test_emit.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/otel/test_store.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/resolvers/__init__.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/resolvers/test_registry.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/run/__init__.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/storage/__init__.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/storage/test_config.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/storage/test_migrator.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/storage/test_models.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/streaming/__init__.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/streaming/test_session_export.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/test_cli.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/test_credential_injection.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/test_eval_config_loader.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/test_jaeger_loader.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/test_layering.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/test_loader_auto.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/test_mcp_server.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/test_otlp_http_protocol.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/test_output.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/test_protocol.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/test_runner.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/test_sdk.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/tests/test_trace_metrics.py +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/README.md +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/index.html +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/public/logo.svg +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/public/vite.svg +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/App.css +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/App.tsx +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/assets/react.svg +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/builder/BuilderHeader.tsx +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/builder/BuilderView.tsx +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/builder/EvalCaseCard.tsx +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/builder/EvalCasesList.tsx +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/builder/InvocationEditor.tsx +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/builder/JsonPreview.tsx +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/builder/MetadataEditor.tsx +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/builder/TraceUploadZone.tsx +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/builder/index.ts +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/dashboard/DashboardView.tsx +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/dashboard/MetricScoreCard.tsx +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/dashboard/PerformanceCard.tsx +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/dashboard/PerformanceCharts.tsx +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/dashboard/SummaryStats.tsx +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/inspector/ComparisonPanel.tsx +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/inspector/DataSection.tsx +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/inspector/InspectorHeader.tsx +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/inspector/InspectorLayout.tsx +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/inspector/InspectorView.tsx +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/inspector/InvocationCard.tsx +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/inspector/MetricResultsSection.tsx +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/inspector/PerformanceSection.tsx +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/inspector/TrajectoryComparisonDetails.tsx +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/runs/PassRateTrendChart.tsx +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/runs/PerMetricTrendChart.tsx +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/runs/RunsHistoryTable.tsx +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/runs/runHistory.ts +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/sidebar/Sidebar.tsx +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/upload/FileDropZone.tsx +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/upload/MetricSelector.tsx +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/upload/RawJsonPreview.tsx +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/upload/UploadView.tsx +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/components/welcome/WelcomeView.tsx +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/index.css +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/lib/eval-config.ts +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/lib/evalset-builder.ts +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/lib/network-capture.ts +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/lib/trace-helpers.ts +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/lib/trace-metadata.ts +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/src/main.tsx +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/tsconfig.app.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/tsconfig.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/tsconfig.node.json +0 -0
- {agentevals_cli-0.10.0 → agentevals_cli-0.11.0}/ui/vite.config.ts +0 -0
|
@@ -59,6 +59,41 @@ jobs:
|
|
|
59
59
|
- name: Build UI
|
|
60
60
|
run: make build-ui
|
|
61
61
|
|
|
62
|
+
- name: Lint UI
|
|
63
|
+
working-directory: ui
|
|
64
|
+
run: npm run lint
|
|
65
|
+
|
|
66
|
+
proto:
|
|
67
|
+
name: Protobuf Contract Check
|
|
68
|
+
runs-on: ubuntu-latest
|
|
69
|
+
steps:
|
|
70
|
+
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
71
|
+
|
|
72
|
+
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
|
|
73
|
+
with:
|
|
74
|
+
node-version: '22'
|
|
75
|
+
|
|
76
|
+
- uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
|
|
77
|
+
with:
|
|
78
|
+
go-version-file: gen/go/go.mod
|
|
79
|
+
cache-dependency-path: gen/go/go.sum
|
|
80
|
+
|
|
81
|
+
# proto-breaking compares against the local main branch.
|
|
82
|
+
- name: Fetch main
|
|
83
|
+
run: git fetch --no-tags --depth=1 origin main:main || true
|
|
84
|
+
|
|
85
|
+
- name: Lint, regenerate and check for drift
|
|
86
|
+
run: make proto-check
|
|
87
|
+
|
|
88
|
+
- name: Breaking changes
|
|
89
|
+
run: make proto-breaking
|
|
90
|
+
|
|
91
|
+
- name: Build the Go module
|
|
92
|
+
working-directory: gen/go
|
|
93
|
+
run: |
|
|
94
|
+
go build ./...
|
|
95
|
+
go vet ./...
|
|
96
|
+
|
|
62
97
|
docker:
|
|
63
98
|
runs-on: ubuntu-latest
|
|
64
99
|
permissions:
|
|
@@ -93,6 +93,33 @@ jobs:
|
|
|
93
93
|
files: dist/**/*.whl
|
|
94
94
|
generate_release_notes: true
|
|
95
95
|
|
|
96
|
+
# Go resolves github.com/agentevals-dev/agentevals/gen/go@vX.Y.Z from the tag gen/go/vX.Y.Z.
|
|
97
|
+
tag-go-module:
|
|
98
|
+
needs: github-release
|
|
99
|
+
runs-on: ubuntu-latest
|
|
100
|
+
permissions:
|
|
101
|
+
contents: write
|
|
102
|
+
steps:
|
|
103
|
+
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
104
|
+
with:
|
|
105
|
+
ref: ${{ github.event.inputs.tag || github.ref_name }}
|
|
106
|
+
fetch-depth: 0
|
|
107
|
+
|
|
108
|
+
- name: Tag the generated Go module
|
|
109
|
+
run: |
|
|
110
|
+
if ! [[ "$TAG" =~ ^v[0-9]+\.[0-9]+\.[0-9]+([-+][0-9A-Za-z.-]+)?$ ]]; then
|
|
111
|
+
echo "Not a semantic version tag: $TAG"
|
|
112
|
+
exit 1
|
|
113
|
+
fi
|
|
114
|
+
if git rev-parse -q --verify "refs/tags/gen/go/$TAG" >/dev/null; then
|
|
115
|
+
echo "gen/go/$TAG already exists"
|
|
116
|
+
exit 0
|
|
117
|
+
fi
|
|
118
|
+
git tag "gen/go/$TAG" "$TAG^{commit}"
|
|
119
|
+
git push origin "gen/go/$TAG"
|
|
120
|
+
env:
|
|
121
|
+
TAG: ${{ github.event.inputs.tag || github.ref_name }}
|
|
122
|
+
|
|
96
123
|
push-docker:
|
|
97
124
|
needs: github-release
|
|
98
125
|
runs-on: ubuntu-latest
|
|
@@ -65,6 +65,19 @@ make build-ui # build React app only → ui/dist/
|
|
|
65
65
|
|
|
66
66
|
Both `build` and `build-bundle` produce `dist/agentevals-*.whl` with the same package name and version. The difference is that `build-bundle` embeds `ui/dist/` as `agentevals/_static/` inside the wheel. The hatchling `artifacts` config ensures the gitignored `_static/` directory is included.
|
|
67
67
|
|
|
68
|
+
### API contract
|
|
69
|
+
|
|
70
|
+
The gRPC API is defined in `proto/`. The generated Go, Python and TypeScript code is committed, so only changes to `.proto` files need these:
|
|
71
|
+
|
|
72
|
+
```bash
|
|
73
|
+
make proto-lint # buf lint
|
|
74
|
+
make proto-generate # regenerate gen/go, src/agentevals/api/v1alpha1 and ui/src/generated
|
|
75
|
+
make proto-check # lint, regenerate and fail when committed code is out of date (CI)
|
|
76
|
+
make proto-breaking # compare with main for breaking changes (CI)
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
`buf` runs through `npx`, so Node is enough. Building the Go module (`cd gen/go && go build ./...`) needs Go, which the Nix dev shell provides.
|
|
80
|
+
|
|
68
81
|
### Testing
|
|
69
82
|
|
|
70
83
|
```bash
|
|
@@ -151,6 +164,8 @@ To release a new Nix derivation, update `flake.nix` with the new version and reb
|
|
|
151
164
|
```
|
|
152
165
|
4. Alternatively, trigger manually from **GitHub → Actions → Release → Run workflow** and enter the tag
|
|
153
166
|
|
|
167
|
+
The workflow also tags the same commit `gen/go/vX.Y.Z`, the version Go clients get with `go get github.com/agentevals-dev/agentevals/gen/go@vX.Y.Z`.
|
|
168
|
+
|
|
154
169
|
The workflow (`.github/workflows/release.yml`) runs `make release`, which builds the wheel twice (once without UI, once with embedded UI) into separate subdirectories:
|
|
155
170
|
|
|
156
171
|
```
|
|
@@ -52,6 +52,6 @@ USER app
|
|
|
52
52
|
ENV PATH="/app/.venv/bin:$PATH"
|
|
53
53
|
ENV AGENTEVALS_SERVER_URL=http://127.0.0.1:8001
|
|
54
54
|
|
|
55
|
-
EXPOSE 8001 4318 4317 8080
|
|
55
|
+
EXPOSE 8001 4318 4317 8002 8080
|
|
56
56
|
|
|
57
57
|
CMD ["agentevals", "serve", "--host", "0.0.0.0", "--port", "8001", "--otlp-http-port", "4318", "--otlp-grpc-port", "4317", "--mcp-port", "8080"]
|
|
@@ -143,3 +143,41 @@ helm-package: helm-cleanup
|
|
|
143
143
|
.PHONY: helm-publish
|
|
144
144
|
helm-publish: helm-package
|
|
145
145
|
helm push "$(HELM_DIST_FOLDER)/agentevals-$(HELM_CHART_VERSION).tgz" "$(HELM_CHART_OCI_URL)"
|
|
146
|
+
|
|
147
|
+
# Protobuf contract. buf runs through npx so no Go toolchain is needed; regenerating spends
|
|
148
|
+
# Buf Schema Registry remote plugin requests, which are rate limited.
|
|
149
|
+
BUF ?= npx --yes @bufbuild/buf@1.72.0
|
|
150
|
+
PROTO_BREAKING_BRANCH ?= main
|
|
151
|
+
PROTO_GENERATED_PATHS := gen/go/agentevals src/agentevals/api/v1alpha1 ui/src/generated
|
|
152
|
+
|
|
153
|
+
.PHONY: proto-lint
|
|
154
|
+
proto-lint:
|
|
155
|
+
cd proto && $(BUF) lint
|
|
156
|
+
|
|
157
|
+
.PHONY: proto-generate
|
|
158
|
+
proto-generate:
|
|
159
|
+
rm -rf $(PROTO_GENERATED_PATHS)
|
|
160
|
+
cd proto && $(BUF) generate
|
|
161
|
+
|
|
162
|
+
.PHONY: proto-breaking
|
|
163
|
+
proto-breaking:
|
|
164
|
+
@if git cat-file -e "$(PROTO_BREAKING_BRANCH):proto/buf.yaml" 2>/dev/null; then \
|
|
165
|
+
$(BUF) breaking proto --against ".git#branch=$(PROTO_BREAKING_BRANCH),subdir=proto"; \
|
|
166
|
+
else \
|
|
167
|
+
echo "No protobuf module on $(PROTO_BREAKING_BRANCH); skipping the breaking check"; \
|
|
168
|
+
fi
|
|
169
|
+
|
|
170
|
+
.PHONY: proto-check
|
|
171
|
+
proto-check: proto-lint proto-generate
|
|
172
|
+
@if test -n "$$(git status --porcelain -- $(PROTO_GENERATED_PATHS))"; then \
|
|
173
|
+
echo "Generated protobuf files are out of date:"; \
|
|
174
|
+
git status --short -- $(PROTO_GENERATED_PATHS); \
|
|
175
|
+
exit 1; \
|
|
176
|
+
fi
|
|
177
|
+
@# git status skips ignored files, so a .gitignore rule could keep a generated file out of the repo unnoticed.
|
|
178
|
+
@ignored="$$(git ls-files --others --ignored --exclude-standard -- $(PROTO_GENERATED_PATHS) | grep -v /__pycache__/)"; \
|
|
179
|
+
if test -n "$$ignored"; then \
|
|
180
|
+
echo "Generated protobuf files are ignored by a .gitignore rule:"; \
|
|
181
|
+
echo "$$ignored"; \
|
|
182
|
+
exit 1; \
|
|
183
|
+
fi
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.5
|
|
2
2
|
Name: agentevals-cli
|
|
3
|
-
Version: 0.
|
|
3
|
+
Version: 0.11.0
|
|
4
4
|
Summary: Standalone framework to evaluate agent correctness based on portable OpenTelemetry traces
|
|
5
5
|
License-File: LICENSE
|
|
6
6
|
Requires-Python: >=3.11
|
|
@@ -10,15 +10,20 @@ Requires-Dist: click>=8.3.3
|
|
|
10
10
|
Requires-Dist: fastapi>=0.115.0
|
|
11
11
|
Requires-Dist: google-adk[eval]<3,>=2.5.0
|
|
12
12
|
Requires-Dist: googleapis-common-protos>=1.72.0
|
|
13
|
-
Requires-Dist: grpcio>=1.
|
|
13
|
+
Requires-Dist: grpcio-health-checking>=1.78
|
|
14
|
+
Requires-Dist: grpcio-reflection>=1.78
|
|
15
|
+
Requires-Dist: grpcio-status>=1.78
|
|
16
|
+
Requires-Dist: grpcio>=1.78
|
|
14
17
|
Requires-Dist: httpx>=0.27.0
|
|
15
18
|
Requires-Dist: idna>=3.18
|
|
16
19
|
Requires-Dist: litellm>=1.85.7
|
|
17
20
|
Requires-Dist: nltk>=3.10.2
|
|
18
21
|
Requires-Dist: opentelemetry-exporter-otlp-proto-grpc>=1.38
|
|
19
22
|
Requires-Dist: opentelemetry-exporter-otlp-proto-http>=1.38
|
|
23
|
+
Requires-Dist: opentelemetry-instrumentation-grpc>=0.59b0
|
|
20
24
|
Requires-Dist: opentelemetry-proto>=1.36.0
|
|
21
25
|
Requires-Dist: opentelemetry-sdk>=1.38
|
|
26
|
+
Requires-Dist: protobuf<7,>=6.33.5
|
|
22
27
|
Requires-Dist: pyasn1>=0.6.4
|
|
23
28
|
Requires-Dist: pygments>=2.20.0
|
|
24
29
|
Requires-Dist: python-dotenv>=1.2.2
|
|
@@ -209,6 +214,19 @@ evaluators:
|
|
|
209
214
|
threshold: 0.7
|
|
210
215
|
```
|
|
211
216
|
|
|
217
|
+
Rubric metrics accept named properties in the evaluator config:
|
|
218
|
+
|
|
219
|
+
```yaml
|
|
220
|
+
evaluators:
|
|
221
|
+
- name: rubric_based_final_response_quality_v1
|
|
222
|
+
type: builtin
|
|
223
|
+
rubrics:
|
|
224
|
+
- id: latest_correction
|
|
225
|
+
text: The response uses the user's latest correction.
|
|
226
|
+
```
|
|
227
|
+
|
|
228
|
+
Rubrics can also come from a matched eval case or its invocations; see [Rubrics](docs/eval-set-format.md#rubrics).
|
|
229
|
+
|
|
212
230
|
`type: remote` pulls community evaluators from GitHub. Protocol, SDK helpers and runtimes: [Custom Evaluators](docs/custom-evaluators.md).
|
|
213
231
|
|
|
214
232
|
## Run it
|
|
@@ -217,18 +235,21 @@ evaluators:
|
|
|
217
235
|
|
|
218
236
|
**Web UI.** `agentevals serve` on `http://localhost:8001`. Upload traces and eval sets, pick evaluators, inspect span trees, watch live sessions. API docs at `/docs`.
|
|
219
237
|
|
|
238
|
+
**gRPC API.** `agentevals serve` also serves a gRPC API on port 8002 for programs: sessions, evaluation, runs and trace conversion, with generated clients for Go and Python. See [gRPC API](docs/grpc-api.md).
|
|
239
|
+
|
|
220
240
|
**MCP server.** `agentevals mcp` exposes `list_metrics`, `evaluate_traces`, `list_sessions`, `summarize_session` and `evaluate_sessions` to MCP clients. A `.mcp.json` at the repo root lets Claude Code pick it up, and the `/eval` and `/inspect` skills in `.claude/skills/` build on it.
|
|
221
241
|
|
|
222
242
|
**Docker.**
|
|
223
243
|
|
|
224
244
|
```bash
|
|
225
245
|
docker build -t agentevals .
|
|
226
|
-
docker run -p 8001:8001 -p 4317:4317 -p 4318:4318 agentevals
|
|
246
|
+
docker run -p 8001:8001 -p 8002:8002 -p 4317:4317 -p 4318:4318 agentevals
|
|
227
247
|
```
|
|
228
248
|
|
|
229
249
|
| Port | Purpose |
|
|
230
250
|
|------|---------|
|
|
231
251
|
| 8001 | Web UI and REST API |
|
|
252
|
+
| 8002 | gRPC API |
|
|
232
253
|
| 4317 | OTLP gRPC receiver (traces and logs) |
|
|
233
254
|
| 4318 | OTLP HTTP receiver (traces and logs) |
|
|
234
255
|
| 8080 | MCP (Streamable HTTP) |
|
|
@@ -263,6 +284,7 @@ The chart runs in memory by default. `--set storage.backend=postgres` with a bun
|
|
|
263
284
|
| [Eval Set Format](docs/eval-set-format.md) | Schema and examples for golden eval sets |
|
|
264
285
|
| [Custom Evaluators](docs/custom-evaluators.md) | Write scoring logic in any language |
|
|
265
286
|
| [Run History](docs/run-history.md) | Persist evaluations to Postgres and trend them in the UI |
|
|
287
|
+
| [gRPC API](docs/grpc-api.md) | Services, generated clients, configuration and security of the gRPC API |
|
|
266
288
|
|
|
267
289
|
## Development
|
|
268
290
|
|
|
@@ -167,6 +167,19 @@ evaluators:
|
|
|
167
167
|
threshold: 0.7
|
|
168
168
|
```
|
|
169
169
|
|
|
170
|
+
Rubric metrics accept named properties in the evaluator config:
|
|
171
|
+
|
|
172
|
+
```yaml
|
|
173
|
+
evaluators:
|
|
174
|
+
- name: rubric_based_final_response_quality_v1
|
|
175
|
+
type: builtin
|
|
176
|
+
rubrics:
|
|
177
|
+
- id: latest_correction
|
|
178
|
+
text: The response uses the user's latest correction.
|
|
179
|
+
```
|
|
180
|
+
|
|
181
|
+
Rubrics can also come from a matched eval case or its invocations; see [Rubrics](docs/eval-set-format.md#rubrics).
|
|
182
|
+
|
|
170
183
|
`type: remote` pulls community evaluators from GitHub. Protocol, SDK helpers and runtimes: [Custom Evaluators](docs/custom-evaluators.md).
|
|
171
184
|
|
|
172
185
|
## Run it
|
|
@@ -175,18 +188,21 @@ evaluators:
|
|
|
175
188
|
|
|
176
189
|
**Web UI.** `agentevals serve` on `http://localhost:8001`. Upload traces and eval sets, pick evaluators, inspect span trees, watch live sessions. API docs at `/docs`.
|
|
177
190
|
|
|
191
|
+
**gRPC API.** `agentevals serve` also serves a gRPC API on port 8002 for programs: sessions, evaluation, runs and trace conversion, with generated clients for Go and Python. See [gRPC API](docs/grpc-api.md).
|
|
192
|
+
|
|
178
193
|
**MCP server.** `agentevals mcp` exposes `list_metrics`, `evaluate_traces`, `list_sessions`, `summarize_session` and `evaluate_sessions` to MCP clients. A `.mcp.json` at the repo root lets Claude Code pick it up, and the `/eval` and `/inspect` skills in `.claude/skills/` build on it.
|
|
179
194
|
|
|
180
195
|
**Docker.**
|
|
181
196
|
|
|
182
197
|
```bash
|
|
183
198
|
docker build -t agentevals .
|
|
184
|
-
docker run -p 8001:8001 -p 4317:4317 -p 4318:4318 agentevals
|
|
199
|
+
docker run -p 8001:8001 -p 8002:8002 -p 4317:4317 -p 4318:4318 agentevals
|
|
185
200
|
```
|
|
186
201
|
|
|
187
202
|
| Port | Purpose |
|
|
188
203
|
|------|---------|
|
|
189
204
|
| 8001 | Web UI and REST API |
|
|
205
|
+
| 8002 | gRPC API |
|
|
190
206
|
| 4317 | OTLP gRPC receiver (traces and logs) |
|
|
191
207
|
| 4318 | OTLP HTTP receiver (traces and logs) |
|
|
192
208
|
| 8080 | MCP (Streamable HTTP) |
|
|
@@ -221,6 +237,7 @@ The chart runs in memory by default. `--set storage.backend=postgres` with a bun
|
|
|
221
237
|
| [Eval Set Format](docs/eval-set-format.md) | Schema and examples for golden eval sets |
|
|
222
238
|
| [Custom Evaluators](docs/custom-evaluators.md) | Write scoring logic in any language |
|
|
223
239
|
| [Run History](docs/run-history.md) | Persist evaluations to Postgres and trend them in the UI |
|
|
240
|
+
| [gRPC API](docs/grpc-api.md) | Services, generated clients, configuration and security of the gRPC API |
|
|
224
241
|
|
|
225
242
|
## Development
|
|
226
243
|
|
|
@@ -2,8 +2,11 @@
|
|
|
2
2
|
2. OTLP HTTP receiver: port {{ .Values.service.otlpHttp.port }} (OTEL_EXPORTER_OTLP_ENDPOINT=http://<service>:{{ .Values.service.otlpHttp.port }}).
|
|
3
3
|
3. OTLP gRPC receiver: port {{ .Values.service.otlpGrpc.port }} (OTEL_EXPORTER_OTLP_ENDPOINT=<service>:{{ .Values.service.otlpGrpc.port }}, OTEL_EXPORTER_OTLP_PROTOCOL=grpc).
|
|
4
4
|
4. MCP (Streamable HTTP): port {{ .Values.service.mcp.port }}, path /mcp (e.g. http://<service>:{{ .Values.service.mcp.port }}/mcp).
|
|
5
|
+
{{- if .Values.grpcApi.enabled }}
|
|
6
|
+
5. gRPC API (agentevals.api.v1alpha1): port {{ .Values.service.grpcApi.port }} (Service port name: grpc-api). Expose it only to trusted callers.
|
|
7
|
+
{{- end }}
|
|
5
8
|
{{- if .Values.ephemeralVolume.enabled }}
|
|
6
|
-
5. An emptyDir is mounted at /tmp with HOME=/tmp/agentevals-home (ephemeral; lost on pod restart). Set ephemeralVolume.enabled=false and readOnlyRootFilesystem=false if you need a writable root without this mount.
|
|
9
|
+
{{ ternary 6 5 .Values.grpcApi.enabled }}. An emptyDir is mounted at /tmp with HOME=/tmp/agentevals-home (ephemeral; lost on pod restart). Set ephemeralVolume.enabled=false and readOnlyRootFilesystem=false if you need a writable root without this mount.
|
|
7
10
|
{{- end }}
|
|
8
11
|
|
|
9
12
|
Get the Service URL:
|
|
@@ -64,6 +64,12 @@ spec:
|
|
|
64
64
|
env:
|
|
65
65
|
- name: AGENTEVALS_SERVER_URL
|
|
66
66
|
value: "http://127.0.0.1:{{ .Values.service.http.containerPort }}"
|
|
67
|
+
- name: AGENTEVALS_API_GRPC_PORT
|
|
68
|
+
value: {{ ternary .Values.service.grpcApi.containerPort 0 .Values.grpcApi.enabled | quote }}
|
|
69
|
+
{{- if and .Values.grpcApi.enabled .Values.grpcApi.reflection }}
|
|
70
|
+
- name: AGENTEVALS_GRPC_REFLECTION
|
|
71
|
+
value: "true"
|
|
72
|
+
{{- end }}
|
|
67
73
|
{{- if .Values.ephemeralVolume.enabled }}
|
|
68
74
|
- name: TMPDIR
|
|
69
75
|
value: "/tmp"
|
|
@@ -117,6 +123,11 @@ spec:
|
|
|
117
123
|
- name: mcp
|
|
118
124
|
containerPort: {{ .Values.service.mcp.containerPort }}
|
|
119
125
|
protocol: TCP
|
|
126
|
+
{{- if .Values.grpcApi.enabled }}
|
|
127
|
+
- name: grpc-api
|
|
128
|
+
containerPort: {{ .Values.service.grpcApi.containerPort }}
|
|
129
|
+
protocol: TCP
|
|
130
|
+
{{- end }}
|
|
120
131
|
resources:
|
|
121
132
|
{{- toYaml .Values.resources | nindent 12 }}
|
|
122
133
|
securityContext:
|
|
@@ -133,9 +144,14 @@ spec:
|
|
|
133
144
|
periodSeconds: 10
|
|
134
145
|
timeoutSeconds: 5
|
|
135
146
|
readinessProbe:
|
|
147
|
+
{{- if and .Values.grpcApi.enabled .Values.grpcApi.grpcReadinessProbe }}
|
|
148
|
+
grpc:
|
|
149
|
+
port: {{ .Values.service.grpcApi.containerPort }}
|
|
150
|
+
{{- else }}
|
|
136
151
|
httpGet:
|
|
137
152
|
path: /api/health
|
|
138
153
|
port: http
|
|
154
|
+
{{- end }}
|
|
139
155
|
initialDelaySeconds: 5
|
|
140
156
|
periodSeconds: 10
|
|
141
157
|
livenessProbe:
|
|
@@ -24,5 +24,12 @@ spec:
|
|
|
24
24
|
port: {{ .Values.service.mcp.port }}
|
|
25
25
|
targetPort: mcp
|
|
26
26
|
protocol: TCP
|
|
27
|
+
{{- if .Values.grpcApi.enabled }}
|
|
28
|
+
- name: grpc-api
|
|
29
|
+
port: {{ .Values.service.grpcApi.port }}
|
|
30
|
+
targetPort: grpc-api
|
|
31
|
+
protocol: TCP
|
|
32
|
+
appProtocol: grpc
|
|
33
|
+
{{- end }}
|
|
27
34
|
selector:
|
|
28
35
|
{{- include "agentevals.app.selectorLabels" . | nindent 4 }}
|
|
@@ -141,6 +141,24 @@ service:
|
|
|
141
141
|
mcp:
|
|
142
142
|
port: 8080
|
|
143
143
|
containerPort: 8080
|
|
144
|
+
# -- gRPC API port (agentevals.api.v1alpha1)
|
|
145
|
+
grpcApi:
|
|
146
|
+
port: 8002
|
|
147
|
+
containerPort: 8002
|
|
148
|
+
|
|
149
|
+
# ==============================================================================
|
|
150
|
+
# gRPC API
|
|
151
|
+
# ==============================================================================
|
|
152
|
+
|
|
153
|
+
grpcApi:
|
|
154
|
+
# -- Serve the gRPC API next to the HTTP API. Every caller that can reach the port is
|
|
155
|
+
# trusted, so expose it only to trusted callers.
|
|
156
|
+
enabled: true
|
|
157
|
+
# -- Turn on gRPC server reflection, for tools such as grpcurl
|
|
158
|
+
reflection: false
|
|
159
|
+
# -- Check readiness with the gRPC health service on the gRPC API port instead of HTTP
|
|
160
|
+
# /api/health
|
|
161
|
+
grpcReadinessProbe: false
|
|
144
162
|
|
|
145
163
|
# ==============================================================================
|
|
146
164
|
# Resources
|
|
@@ -89,6 +89,21 @@ agentevals run traces/my_trace.json \
|
|
|
89
89
|
|
|
90
90
|
Each evaluator entry in the `evaluators` list uses the following fields. The `type` field determines which other fields are valid.
|
|
91
91
|
|
|
92
|
+
### `type: builtin` (ADK metrics)
|
|
93
|
+
|
|
94
|
+
| Field | Required | Default | Description |
|
|
95
|
+
|---|---|---|---|
|
|
96
|
+
| `name` | yes | | The metric, for example `tool_trajectory_avg_score` or `rubric_based_final_response_quality_v1` |
|
|
97
|
+
| `type` | yes | | `builtin` |
|
|
98
|
+
| `threshold` | no | `0.5` | Score at or above this value means PASSED |
|
|
99
|
+
| `judge_model` | no | ADK default | Judge model for LLM-backed metrics |
|
|
100
|
+
| `trajectory_match_type` | no | `EXACT` | `EXACT`, `IN_ORDER` or `ANY_ORDER`, for `tool_trajectory_avg_score` |
|
|
101
|
+
| `rubrics` | for `rubric_based_final_response_quality_v1` and `rubric_based_tool_use_quality_v1` | | A list of `{id, text}` mappings, or plain strings (given positional ids `rubric_0`, `rubric_1`, ...). Ids must be unique. Config rubrics have no `type` field. Other metrics reject `rubrics` |
|
|
102
|
+
| `credential_ref` | no | | Name of a credential the judge key is resolved from (API runs) |
|
|
103
|
+
| `judge_base_url` | no | | Base URL of an OpenAI-compatible judge endpoint |
|
|
104
|
+
|
|
105
|
+
Each of these metrics requires an applicable rubric for each evaluated invocation, from the config, the matched eval case or that invocation. Its result carries `details.per_invocation[].rubric_scores` and `details.overall_rubric_scores`, one entry per rubric id with its score and the judge's rationale. Untyped case/invocation rubrics are listed in `details.warnings`; other types are ignored as in ADK, so response and tool-use properties can share one eval set. Config rubrics are not filtered by type. Invalid or empty effective rubric sets report an error before any judge call. Scores, aggregation and explanations follow ADK.
|
|
106
|
+
|
|
92
107
|
### `type: code` (local scripts)
|
|
93
108
|
|
|
94
109
|
| Field | Required | Default | Description |
|
|
@@ -176,6 +176,53 @@ An eval case can have multiple invocations to represent a conversation. Each inv
|
|
|
176
176
|
| `rubrics` | list[Rubric] | no | Scoring rubrics for this specific invocation |
|
|
177
177
|
| `creation_timestamp` | float | no | Unix timestamp |
|
|
178
178
|
|
|
179
|
+
### Rubrics
|
|
180
|
+
|
|
181
|
+
`rubrics` on an eval case apply to every invocation of that case; `rubrics` on an invocation apply to that invocation alone. Each rubric is an ADK `Rubric`: `rubric_id`, `rubric_content.text_property`, and `type`. Response-quality rubrics use `FINAL_RESPONSE_QUALITY`; tool-use rubrics use `TOOL_USE_QUALITY`. As in ADK, each metric ignores properties of another type. Untyped case/invocation properties are ignored with their ids and locations listed in `details.warnings`. The runner preserves authored types. Config-level `{id, text}` rubrics are not filtered by type. Effective rubric ids must be distinct across the metric, case and invocation. A clash or an invocation with no applicable rubrics reports an error before any judge call.
|
|
182
|
+
|
|
183
|
+
The `rubric_based_*` metrics grade each invocation on its own, with that invocation's prompt, steps and final response, so a case-level rubric is judged once per turn. In a multi-turn case a rubric that describes the end state ("produces the completed draft") is therefore judged against the intermediate turns as well, where it fails, and lowers the case's mean. Write such a rubric on the final invocation instead, and keep config/case-level rubrics for properties every turn must have. Every evaluated turn still needs an applicable property. A terminal response may be a clarification request when that is the expected outcome of the case. Earlier conversational context and a dedicated final-turn expectations format are separate from this per-invocation behavior.
|
|
184
|
+
|
|
185
|
+
```json
|
|
186
|
+
{
|
|
187
|
+
"eval_id": "booking-clarification-and-completion",
|
|
188
|
+
"rubrics": [
|
|
189
|
+
{
|
|
190
|
+
"rubric_id": "clear-outcome",
|
|
191
|
+
"type": "FINAL_RESPONSE_QUALITY",
|
|
192
|
+
"rubric_content": {"text_property": "Clearly distinguishes a clarification question from a completed draft."}
|
|
193
|
+
}
|
|
194
|
+
],
|
|
195
|
+
"conversation": [
|
|
196
|
+
{
|
|
197
|
+
"invocation_id": "clarification",
|
|
198
|
+
"user_content": {"role": "user", "parts": [{"text": "Draft a booking for Paris."}]},
|
|
199
|
+
"final_response": {"role": "model", "parts": [{"text": "What date should the booking use?"}]},
|
|
200
|
+
"rubrics": [
|
|
201
|
+
{
|
|
202
|
+
"rubric_id": "asks-date",
|
|
203
|
+
"type": "FINAL_RESPONSE_QUALITY",
|
|
204
|
+
"rubric_content": {"text_property": "Asks for the missing date without claiming a completed booking."}
|
|
205
|
+
}
|
|
206
|
+
]
|
|
207
|
+
},
|
|
208
|
+
{
|
|
209
|
+
"invocation_id": "completion",
|
|
210
|
+
"user_content": {"role": "user", "parts": [{"text": "Friday; actually use Lyon."}]},
|
|
211
|
+
"final_response": {"role": "model", "parts": [{"text": "Draft booking for Lyon on Friday."}]},
|
|
212
|
+
"rubrics": [
|
|
213
|
+
{
|
|
214
|
+
"rubric_id": "uses-latest-destination",
|
|
215
|
+
"type": "FINAL_RESPONSE_QUALITY",
|
|
216
|
+
"rubric_content": {"text_property": "Uses Lyon and Friday in the completed draft."}
|
|
217
|
+
}
|
|
218
|
+
]
|
|
219
|
+
}
|
|
220
|
+
]
|
|
221
|
+
}
|
|
222
|
+
```
|
|
223
|
+
|
|
224
|
+
Here `clear-outcome` applies to both invocations, `asks-date` only to the clarification, and `uses-latest-destination` only to the completed draft. A case intended to stop at clarification can contain just the first invocation and its properties; it does not need to require a completed answer. This uses the existing invocation rubric field without adding a new scope option or changing ADK semantics.
|
|
225
|
+
|
|
179
226
|
### Content
|
|
180
227
|
|
|
181
228
|
Uses the Google GenAI `Content` format:
|
|
@@ -0,0 +1,71 @@
|
|
|
1
|
+
# gRPC API
|
|
2
|
+
|
|
3
|
+
`agentevals serve` serves a gRPC API on port 8002, next to the web UI and REST API on 8001. It is the API for programs: other services, scripts and CI. It speaks native gRPC over HTTP/2, so browsers cannot call it; the web UI uses the REST API and an SSE stream that carry the same messages as JSON.
|
|
4
|
+
|
|
5
|
+
The contract lives in [`proto/agentevals/api/v1alpha1`](../proto/agentevals/api/v1alpha1). `v1alpha1` can still change; CI checks every change for compatibility.
|
|
6
|
+
|
|
7
|
+
## Services
|
|
8
|
+
|
|
9
|
+
| Service | Methods |
|
|
10
|
+
|---------|---------|
|
|
11
|
+
| `SessionService` | `ListSessions`, `GetSession`, `WatchSessions` (every current session, then each change) |
|
|
12
|
+
| `EvaluationService` | `ListMetrics`, `EvaluateTraces` (streams progress and results) |
|
|
13
|
+
| `RunService` | `SubmitRun`, `GetRun`, `ListRuns`, `CancelRun`, `ListRunResults`, `WatchRun`. Needs the Postgres backend, see [Run History](run-history.md) |
|
|
14
|
+
| `ConversionService` | `ConvertTraces` (trace files to conversations and a golden eval set) |
|
|
15
|
+
| `SystemService` | `GetVersion`, `GetConfig` |
|
|
16
|
+
|
|
17
|
+
The server also implements the standard `grpc.health.v1.Health` service, so Kubernetes `grpc` probes and load balancers can check it.
|
|
18
|
+
|
|
19
|
+
Sessions and conversations come back as views: turns with their user text, tool calls paired with their results, model calls, token usage and status, already worked out by the server. Each field names the OpenTelemetry attribute it comes from. A token count that was not recorded is left unset, never reported as 0.
|
|
20
|
+
|
|
21
|
+
Traces go in as OTLP protobuf (`TracesData` and `LogsData`) or as files (OTLP/JSON, OTLP/JSONL or Jaeger JSON), the same formats the REST API takes.
|
|
22
|
+
|
|
23
|
+
## Calling it
|
|
24
|
+
|
|
25
|
+
With reflection on, `grpcurl` can list and call everything:
|
|
26
|
+
|
|
27
|
+
```bash
|
|
28
|
+
AGENTEVALS_GRPC_REFLECTION=true uv run agentevals serve --dev
|
|
29
|
+
grpcurl -plaintext localhost:8002 list
|
|
30
|
+
grpcurl -plaintext localhost:8002 agentevals.api.v1alpha1.SessionService/ListSessions
|
|
31
|
+
grpcurl -plaintext -d '{"includeTurns": true}' localhost:8002 agentevals.api.v1alpha1.SessionService/WatchSessions
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
Reflection is off by default. Without it, pass the protos with `-import-path` and `-proto`.
|
|
35
|
+
|
|
36
|
+
Generated clients:
|
|
37
|
+
|
|
38
|
+
* **Go:** `go get github.com/agentevals-dev/agentevals/gen/go@vX.Y.Z`, then import `github.com/agentevals-dev/agentevals/gen/go/agentevals/api/v1alpha1`.
|
|
39
|
+
* **Python:** the `agentevals-cli` package ships the stubs as `agentevals.api.v1alpha1`.
|
|
40
|
+
* **TypeScript:** generate from the protos with `buf` and `protoc-gen-es`, as `ui/src/generated` does.
|
|
41
|
+
|
|
42
|
+
Errors use standard gRPC status codes with a `google.rpc.ErrorInfo` detail whose `reason` is stable, for example `STORAGE_NOT_CONFIGURED` or `RUN_SPEC_CONFLICT`. Internal errors return `INTERNAL` with no details; the server log has them.
|
|
43
|
+
|
|
44
|
+
## Configuration
|
|
45
|
+
|
|
46
|
+
| Setting | Purpose |
|
|
47
|
+
|---------|---------|
|
|
48
|
+
| `--api-grpc-port` or `AGENTEVALS_API_GRPC_PORT` | Port, default `8002`; `0` turns the API off |
|
|
49
|
+
| `AGENTEVALS_GRPC_REFLECTION=true` | Turn on server reflection |
|
|
50
|
+
| `AGENTEVALS_API_TRACES=true` | Export traces of the API server itself (see below) |
|
|
51
|
+
|
|
52
|
+
Messages can be up to 16 MiB. Uploaded trace documents are limited to 10 MiB each, as in the REST API.
|
|
53
|
+
|
|
54
|
+
The Helm chart has the same switches under `grpcApi`: `enabled`, `reflection`, and `grpcReadinessProbe` to check readiness with the gRPC health service. The Service exposes the port as `grpc-api`.
|
|
55
|
+
|
|
56
|
+
## Tracing the API server
|
|
57
|
+
|
|
58
|
+
agentevals receives telemetry, so it exports none of its own by default. With `AGENTEVALS_API_TRACES=true` it traces every API call with the OpenTelemetry gRPC instrumentation, using the standard exporter variables:
|
|
59
|
+
|
|
60
|
+
```bash
|
|
61
|
+
AGENTEVALS_API_TRACES=true OTEL_EXPORTER_OTLP_ENDPOINT=http://collector:4318 uv run agentevals serve
|
|
62
|
+
AGENTEVALS_API_TRACES=true OTEL_TRACES_EXPORTER=console uv run agentevals serve
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
`OTEL_SDK_DISABLED=true` or `OTEL_TRACES_EXPORTER=none` turn it off again. Health checks and reflection are not traced. If these traces are sent to agentevals itself, it drops them, so they never show up as sessions.
|
|
66
|
+
|
|
67
|
+
## Security
|
|
68
|
+
|
|
69
|
+
The API has no authentication. Any caller that can reach the port can read sessions, run evaluations with the server's credentials, and submit runs. Expose port 8002 only to trusted callers, for example inside a cluster behind a NetworkPolicy, and put authentication in front of it when you need it.
|
|
70
|
+
|
|
71
|
+
On a laptop `agentevals serve` listens on all interfaces. Pass `--host 127.0.0.1` to keep every port local.
|
|
@@ -0,0 +1,10 @@
|
|
|
1
|
+
# agentevals Go API client
|
|
2
|
+
|
|
3
|
+
Generated Go types and gRPC clients for the `agentevals.api.v1alpha1` API.
|
|
4
|
+
|
|
5
|
+
```go
|
|
6
|
+
import apiv1alpha1 "github.com/agentevals-dev/agentevals/gen/go/agentevals/api/v1alpha1"
|
|
7
|
+
```
|
|
8
|
+
|
|
9
|
+
The code is generated from `proto/` by `make proto-generate` and must not be edited by hand.
|
|
10
|
+
Releases are tagged `gen/go/vX.Y.Z`, matching the agentevals version.
|