bigquery-agent-analytics 0.3.4__tar.gz → 0.4.0__tar.gz

{bigquery_agent_analytics-0.3.4 → bigquery_agent_analytics-0.4.0}/.github/workflows/ci.yml

@@ -27,7 +27,7 @@ concurrency:
 jobs:
   format:
     name: Format check
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-22.04
     steps:
       - uses: actions/checkout@v4
 
@@ -51,11 +51,11 @@ jobs:
         run: |
           python scripts/generate_colab_from_codelab.py --check \
             docs/codelabs/periodic_materialization.md \
-            examples/codelab/periodic_materialization/colab_notebook.ipynb
+            examples/context_graph/codelab/colab_notebook.ipynb
 
   test:
     name: Test (Python ${{ matrix.python-version }})
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-22.04
     strategy:
       fail-fast: false
       matrix:
@@ -75,7 +75,7 @@ jobs:
 
   build:
     name: Build package
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-22.04
     needs: [format, test]
     steps:
       - uses: actions/checkout@v4

{bigquery_agent_analytics-0.3.4 → bigquery_agent_analytics-0.4.0}/.gitignore

@@ -14,13 +14,13 @@ env/
 .adk/
 uv.lock
 .env
+/.idea/
 
 # Script outputs
 scripts/reports/
 
 # Example run artifacts
 examples/*/reports/
-examples/*/reports_*/
 examples/*/trials_*/
 scripts/**/*.log
 examples/**/*.log

{bigquery_agent_analytics-0.3.4 → bigquery_agent_analytics-0.4.0}/CHANGELOG.md

@@ -7,6 +7,41 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.4.0] - 2026-06-18
+
+### Release highlights
+
+Richer analytics out of the box. A new `retail-returns` seed scenario gives
+token-usage and latency demos real data to query the moment you seed a
+dataset, the SDK's type surfaces now recognize the ADK 2.0 event vocabulary
+end to end, and the quality report graduates from two pass/fail metrics to a
+diagnostic view that tells you *why* a session failed and *what to fix*.
+
+### Added
+
+- **`retail-returns` seed-events scenario** — `bqaa seed-events --scenario
+  retail-returns` generates a multi-agent refund/exchange trace (intake-triage,
+  fraud-abuse, quality-defect) with `LLM_REQUEST`/`LLM_RESPONSE` token-usage
+  and latency telemetry, so token/latency analytics and the `v_llm_response`
+  view's `usage_*_tokens` / `total_ms` / `ttft_ms` columns are non-empty
+  immediately. Emits one terminal `AGENT_COMPLETED` per session, deterministic
+  outcome buckets (surfaced via `session_outcome_counts`), product-quality
+  feedback text, and `legacy_crm_db` diagnostic errors for filterable demo
+  queries. Output is deterministic for a fixed `(seed, now)`; existing
+  `decision` / `decision-realistic` scenarios are unchanged and byte-identical.
+- **ADK 2.0 event types across the SDK type surfaces** — the four
+  #293-shipped event types are registered with full typed views and the two
+  workflow-boundary types as header-only, across every consumer type surface,
+  and `TOOL_COMPLETED` is extended with the long-running pair keys.
+- **`SystemEvaluator`** as the preferred name for deterministic/code-defined
+  metrics. `CodeEvaluator` is kept as a backward-compatible alias; calling
+  `CodeEvaluator()` now emits `evaluator_name="system_evaluator"`.
+- **Expanded quality report** — adds quality dimensions, correction analysis,
+  execution traces, golden-Q&A grounding, and version filtering, so a failing
+  session shows why it failed, whether the answer is correct, and what to fix
+  (beyond the original `response_usefulness` / `task_grounding` pass/fail
+  metrics).
+
 ## [0.3.4] - 2026-06-10
 
 ### Release highlights

{bigquery_agent_analytics-0.3.4 → bigquery_agent_analytics-0.4.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: bigquery-agent-analytics
-Version: 0.3.4
+Version: 0.4.0
 Summary: SDK for analyzing and evaluating agent traces stored in BigQuery.
 Author: Google LLC
 License-Expression: Apache-2.0
@@ -62,7 +62,7 @@ Description-Content-Type: text/markdown
 An open-source Python SDK for analyzing, evaluating, and curating agent traces
 stored in BigQuery. Built on top of the
 [BigQuery Agent Analytics](https://adk.dev/integrations/bigquery-agent-analytics/), it provides
-a consumption-layer toolkit for agent observability, analysis, evaluation, and advanced capabilities like context graph at scale.
+a consumption-layer toolkit for agent observability, analysis, evaluation, and advanced capabilities like the Agent Context Graph — extracting decision traces from your agent's context graph — at scale.
 
 ## Overview
 
@@ -94,8 +94,7 @@ regressions — all through BigQuery SQL or Python.
 - Categorical (Hatteras-style) evaluation via BigFrames
 
 **Advanced Analytics**
-- Context Graph — property graph linking traces to business entities with GQL traversal
-- YAML-driven ontology extraction and materialization
+- Agent Context Graph — extract decision traces from your agent's context graph: the requests an agent handled, the options it weighed, and the outcomes it committed, materialized into a queryable BigQuery property graph (GQL traversal, scheduled refresh via `bqaa context-graph`)
 - Long-horizon cross-session memory
 - Multi-stage agent insights pipeline
 - Drift detection for golden vs production question distributions
@@ -155,11 +154,53 @@ trace.render()
 See [SDK.md](SDK.md) for the full API walkthrough with code examples for every
 feature.
 
+### Try it: extract decision traces (Agent Context Graph, ~10 minutes)
+
+Deploy a context graph, seed sample agent events, extract the decision
+traces, and query one in GQL — entirely from your terminal:
+
+```bash
+export PROJECT_ID="your-project" DATASET="agent_analytics_demo"
+gcloud config set project "$PROJECT_ID"
+bq --location=US mk --dataset "$PROJECT_ID:$DATASET"
+
+# 1. Deploy the context graph (one-time DDL: tables, then the property graph).
+cd examples/context_graph/codelab
+envsubst < table_ddl.sql      | bq query --use_legacy_sql=false
+envsubst < property_graph.sql | bq query --use_legacy_sql=false
+
+# 2. Seed five sample agent sessions into agent_events.
+bqaa seed-events --project-id "$PROJECT_ID" --dataset-id "$DATASET" --sessions 5
+
+# 3. Extract decision traces from the deployed graph
+#    (read back via INFORMATION_SCHEMA.PROPERTY_GRAPHS — no SQL file passed).
+bqaa context-graph --project-id "$PROJECT_ID" --dataset-id "$DATASET" \
+    --graph agent_decisions_graph --lookback-hours 24 --format json
+
+# 4. Query a decision trace: what did the agent weigh, and how did it resolve?
+bq query --use_legacy_sql=false "
+SELECT * FROM GRAPH_TABLE(
+  $DATASET.agent_decisions_graph
+  MATCH (req:DecisionRequest)-[eo:evaluatesOption]->(opt:DecisionOption),
+        (req)-[ri:resultedIn]->(out:DecisionOutcome)
+  COLUMNS (req.request_text AS question, opt.option_label AS considered,
+           out.status AS outcome, out.rationale AS rationale))"
+```
+
+Expect `"ok": true` with 5 sessions materialized, and fifteen GQL rows — three
+options weighed per request, each with the committed outcome and rationale.
+The [Agent Context Graph codelab](docs/codelabs/periodic_materialization.md) is the
+guided version of these steps (plus backfill and production scheduling), and
+[`examples/context_graph/`](examples/context_graph/) is the worked example
+with a runnable ADK agent.
+
 ## Documentation
 
 | Resource | Description |
 |----------|-------------|
 | [SDK Feature Reference](SDK.md) | Complete API walkthrough with working code examples |
+| [Agent Context Graph Codelab](docs/codelabs/periodic_materialization.md) | Extract decision traces from your agent's context graph, end to end (~35 min) |
+| [Scheduled Deploy Runbook](docs/guides/scheduled-context-graph-deploy.md) | Keep the context graph fresh on a Cloud Run + Cloud Scheduler cron |
 | [Design Documents](docs/README.md) | Architecture decisions and design rationale |
 | [Examples](examples/README.md) | Notebooks, SQL scripts, and demos |
 | [Deployment Guides](deploy/README.md) | Four deployment surfaces for Google Cloud |
@@ -178,7 +219,7 @@ src/bigquery_agent_analytics/
 │   └── formatter.py               # Output formatting (json/text/table)
 │
 ├── Evaluation
-│   ├── evaluators.py              # CodeEvaluator + LLMAsJudge
+│   ├── evaluators.py              # SystemEvaluator + LLMAsJudge
 │   ├── trace_evaluator.py         # Trajectory matching & replay
 │   ├── multi_trial.py             # Multi-trial runner + pass@k
 │   ├── grader_pipeline.py         # Grader composition pipeline
@@ -194,16 +235,12 @@ src/bigquery_agent_analytics/
 ├── Analytics
 │   ├── insights.py                # Multi-stage insights pipeline
 │   ├── feedback.py                # Drift detection & question distribution
-│   ├── context_graph.py           # Property Graph: BizNode extraction, GQL
 │   └── memory_service.py          # Long-horizon agent memory
 │
-├── Ontology
-│   ├── ontology_models.py         # Pydantic models for ontology schema
-│   ├── ontology_schema_compiler.py# YAML → compiled schema
-│   ├── ontology_graph.py          # Ontology graph construction
-│   ├── ontology_materializer.py   # Graph materialization to BigQuery
-│   ├── ontology_property_graph.py # Property graph operations
-│   └── ontology_orchestrator.py   # End-to-end ontology pipeline
+├── Agent Context Graph
+│   ├── context_graph.py           # Decision-trace extraction & GQL traversal
+│   ├── materialize_window.py      # Scheduled materialization (bqaa context-graph)
+│   └── property_graph_spec.py     # Derive the spec from your deployed property graph
 │
 └── CLI & Deploy
     ├── cli.py                     # CLI entry point (bq-agent-sdk)

{bigquery_agent_analytics-0.3.4 → bigquery_agent_analytics-0.4.0}/README.md

@@ -7,7 +7,7 @@
 An open-source Python SDK for analyzing, evaluating, and curating agent traces
 stored in BigQuery. Built on top of the
 [BigQuery Agent Analytics](https://adk.dev/integrations/bigquery-agent-analytics/), it provides
-a consumption-layer toolkit for agent observability, analysis, evaluation, and advanced capabilities like context graph at scale.
+a consumption-layer toolkit for agent observability, analysis, evaluation, and advanced capabilities like the Agent Context Graph — extracting decision traces from your agent's context graph — at scale.
 
 ## Overview
 
@@ -39,8 +39,7 @@ regressions — all through BigQuery SQL or Python.
 - Categorical (Hatteras-style) evaluation via BigFrames
 
 **Advanced Analytics**
-- Context Graph — property graph linking traces to business entities with GQL traversal
-- YAML-driven ontology extraction and materialization
+- Agent Context Graph — extract decision traces from your agent's context graph: the requests an agent handled, the options it weighed, and the outcomes it committed, materialized into a queryable BigQuery property graph (GQL traversal, scheduled refresh via `bqaa context-graph`)
 - Long-horizon cross-session memory
 - Multi-stage agent insights pipeline
 - Drift detection for golden vs production question distributions
@@ -100,11 +99,53 @@ trace.render()
 See [SDK.md](SDK.md) for the full API walkthrough with code examples for every
 feature.
 
+### Try it: extract decision traces (Agent Context Graph, ~10 minutes)
+
+Deploy a context graph, seed sample agent events, extract the decision
+traces, and query one in GQL — entirely from your terminal:
+
+```bash
+export PROJECT_ID="your-project" DATASET="agent_analytics_demo"
+gcloud config set project "$PROJECT_ID"
+bq --location=US mk --dataset "$PROJECT_ID:$DATASET"
+
+# 1. Deploy the context graph (one-time DDL: tables, then the property graph).
+cd examples/context_graph/codelab
+envsubst < table_ddl.sql      | bq query --use_legacy_sql=false
+envsubst < property_graph.sql | bq query --use_legacy_sql=false
+
+# 2. Seed five sample agent sessions into agent_events.
+bqaa seed-events --project-id "$PROJECT_ID" --dataset-id "$DATASET" --sessions 5
+
+# 3. Extract decision traces from the deployed graph
+#    (read back via INFORMATION_SCHEMA.PROPERTY_GRAPHS — no SQL file passed).
+bqaa context-graph --project-id "$PROJECT_ID" --dataset-id "$DATASET" \
+    --graph agent_decisions_graph --lookback-hours 24 --format json
+
+# 4. Query a decision trace: what did the agent weigh, and how did it resolve?
+bq query --use_legacy_sql=false "
+SELECT * FROM GRAPH_TABLE(
+  $DATASET.agent_decisions_graph
+  MATCH (req:DecisionRequest)-[eo:evaluatesOption]->(opt:DecisionOption),
+        (req)-[ri:resultedIn]->(out:DecisionOutcome)
+  COLUMNS (req.request_text AS question, opt.option_label AS considered,
+           out.status AS outcome, out.rationale AS rationale))"
+```
+
+Expect `"ok": true` with 5 sessions materialized, and fifteen GQL rows — three
+options weighed per request, each with the committed outcome and rationale.
+The [Agent Context Graph codelab](docs/codelabs/periodic_materialization.md) is the
+guided version of these steps (plus backfill and production scheduling), and
+[`examples/context_graph/`](examples/context_graph/) is the worked example
+with a runnable ADK agent.
+
 ## Documentation
 
 | Resource | Description |
 |----------|-------------|
 | [SDK Feature Reference](SDK.md) | Complete API walkthrough with working code examples |
+| [Agent Context Graph Codelab](docs/codelabs/periodic_materialization.md) | Extract decision traces from your agent's context graph, end to end (~35 min) |
+| [Scheduled Deploy Runbook](docs/guides/scheduled-context-graph-deploy.md) | Keep the context graph fresh on a Cloud Run + Cloud Scheduler cron |
 | [Design Documents](docs/README.md) | Architecture decisions and design rationale |
 | [Examples](examples/README.md) | Notebooks, SQL scripts, and demos |
 | [Deployment Guides](deploy/README.md) | Four deployment surfaces for Google Cloud |
@@ -123,7 +164,7 @@ src/bigquery_agent_analytics/
 │   └── formatter.py               # Output formatting (json/text/table)
 │
 ├── Evaluation
-│   ├── evaluators.py              # CodeEvaluator + LLMAsJudge
+│   ├── evaluators.py              # SystemEvaluator + LLMAsJudge
 │   ├── trace_evaluator.py         # Trajectory matching & replay
 │   ├── multi_trial.py             # Multi-trial runner + pass@k
 │   ├── grader_pipeline.py         # Grader composition pipeline
@@ -139,16 +180,12 @@ src/bigquery_agent_analytics/
 ├── Analytics
 │   ├── insights.py                # Multi-stage insights pipeline
 │   ├── feedback.py                # Drift detection & question distribution
-│   ├── context_graph.py           # Property Graph: BizNode extraction, GQL
 │   └── memory_service.py          # Long-horizon agent memory
 │
-├── Ontology
-│   ├── ontology_models.py         # Pydantic models for ontology schema
-│   ├── ontology_schema_compiler.py# YAML → compiled schema
-│   ├── ontology_graph.py          # Ontology graph construction
-│   ├── ontology_materializer.py   # Graph materialization to BigQuery
-│   ├── ontology_property_graph.py # Property graph operations
-│   └── ontology_orchestrator.py   # End-to-end ontology pipeline
+├── Agent Context Graph
+│   ├── context_graph.py           # Decision-trace extraction & GQL traversal
+│   ├── materialize_window.py      # Scheduled materialization (bqaa context-graph)
+│   └── property_graph_spec.py     # Derive the spec from your deployed property graph
 │
 └── CLI & Deploy
     ├── cli.py                     # CLI entry point (bq-agent-sdk)

{bigquery_agent_analytics-0.3.4 → bigquery_agent_analytics-0.4.0}/SDK.md

@@ -112,32 +112,32 @@ traces = client.list_traces(
 
 ## 3. Code-Based Evaluation (Deterministic Metrics)
 
-`CodeEvaluator` runs deterministic, code-defined metric functions against session summaries. Each metric returns a score between 0.0 and 1.0.
+`SystemEvaluator` runs deterministic, code-defined metric functions against session summaries. Each metric returns a score between 0.0 and 1.0.
 
 ### Pre-Built Evaluators
 
 The SDK ships with seven ready-to-use evaluators:
 
 ```python
-from bigquery_agent_analytics import CodeEvaluator
+from bigquery_agent_analytics import SystemEvaluator
 
-# Latency: fails when average latency exceeds the budget
-evaluator = CodeEvaluator.latency(threshold_ms=5000)
+# Latency: fails when average latency exceeds the threshold
+evaluator = SystemEvaluator.latency(threshold_ms=5000)
 
-# Turn count: fails when sessions use too many back-and-forth turns
-evaluator = CodeEvaluator.turn_count(max_turns=10)
+# Turn count: fails when session turns exceed the max turns
+evaluator = SystemEvaluator.turn_count(max_turns=10)
 
-# Error rate: fails on high tool error rates
-evaluator = CodeEvaluator.error_rate(max_error_rate=0.1)
+# Error rate: fails when tool error rate exceeds the max error rate
+evaluator = SystemEvaluator.error_rate(max_error_rate=0.1)
 
 # Token efficiency: checks total token usage stays within budget
-evaluator = CodeEvaluator.token_efficiency(max_tokens=50000)
+evaluator = SystemEvaluator.token_efficiency(max_tokens=50000)
 
 # Context cache hit rate: checks repeated prompt-prefix reuse
-evaluator = CodeEvaluator.context_cache_hit_rate(min_hit_rate=0.5)
+evaluator = SystemEvaluator.context_cache_hit_rate(min_hit_rate=0.5)
 
 # Cost per session: checks estimated USD cost stays under budget
-evaluator = CodeEvaluator.cost_per_session(
+evaluator = SystemEvaluator.cost_per_session(
     max_cost_usd=1.0,
     input_cost_per_1k=0.00025,
     output_cost_per_1k=0.00125,
@@ -173,7 +173,7 @@ Define your own metric functions and chain multiple metrics together:
 
 ```python
 evaluator = (
-    CodeEvaluator(name="my_quality_check")
+    SystemEvaluator(name="my_quality_check")
     .add_metric(
         name="latency",
         fn=lambda s: 1.0 - min(s.get("avg_latency_ms", 0) / 5000, 1.0),
@@ -216,7 +216,7 @@ Run evaluation across all sessions matching a filter:
 from bigquery_agent_analytics import TraceFilter
 
 report = client.evaluate(
-    evaluator=CodeEvaluator.latency(threshold_ms=3000),
+    evaluator=SystemEvaluator.latency(threshold_ms=3000),
     filters=TraceFilter(agent_id="my_agent"),
 )
 
@@ -561,7 +561,7 @@ pass_pow_k = compute_pass_pow_k(num_trials=10, num_passed=8)  # ~0.107
 
 ## 7. Grader Composition Pipeline
 
-Combine multiple evaluators (`CodeEvaluator` + `LLMAsJudge` + custom functions) into a single aggregated verdict using configurable scoring strategies.
+Combine multiple evaluators (`SystemEvaluator` + `LLMAsJudge` + custom functions) into a single aggregated verdict using configurable scoring strategies.
 
 ### Scoring Strategies
 
@@ -575,7 +575,7 @@ Combine multiple evaluators (`CodeEvaluator` + `LLMAsJudge` + custom functions)
 
 ```python
 from bigquery_agent_analytics import (
-    CodeEvaluator, GraderPipeline, LLMAsJudge,
+    SystemEvaluator, GraderPipeline, LLMAsJudge,
     WeightedStrategy, GraderResult,
 )
 
@@ -588,8 +588,8 @@ pipeline = (
         },
         threshold=0.6,
     ))
-    .add_code_grader(CodeEvaluator.latency(threshold_ms=5000), weight=0.2)
-    .add_code_grader(CodeEvaluator.cost_per_session(max_cost_usd=0.50), weight=0.1)
+    .add_system_grader(SystemEvaluator.latency(threshold_ms=5000), weight=0.2)
+    .add_system_grader(SystemEvaluator.cost_per_session(max_cost_usd=0.50), weight=0.1)
     .add_llm_grader(LLMAsJudge.correctness(threshold=0.7), weight=0.7)
 )
 
@@ -618,8 +618,8 @@ from bigquery_agent_analytics import BinaryStrategy
 
 pipeline = (
     GraderPipeline(BinaryStrategy())
-    .add_code_grader(CodeEvaluator.latency(threshold_ms=3000))
-    .add_code_grader(CodeEvaluator.error_rate(max_error_rate=0.05))
+    .add_system_grader(SystemEvaluator.latency(threshold_ms=3000))
+    .add_system_grader(SystemEvaluator.error_rate(max_error_rate=0.05))
     .add_llm_grader(LLMAsJudge.hallucination(threshold=0.8))
 )
 
@@ -649,7 +649,7 @@ def business_rules_grader(context):
 
 pipeline = (
     GraderPipeline(BinaryStrategy())
-    .add_code_grader(CodeEvaluator.latency())
+    .add_system_grader(SystemEvaluator.latency())
     .add_custom_grader("business_rules", business_rules_grader)
 )
 ```
@@ -1292,6 +1292,25 @@ vm.create_view("LLM_REQUEST")
 print(vm.get_view_sql("TOOL_COMPLETED"))
 ```
 
+#### ADK 2.0 event types
+
+The plugin emits additional event types under ADK 2.0, each with a typed view:
+`AGENT_TRANSFER` (`from_agent`, `to_agent`), `EVENT_COMPACTION`
+(`start_timestamp`, `end_timestamp`, `compacted_content`),
+`AGENT_STATE_CHECKPOINT` (`agent_state`, `agent_state_type`, `end_of_agent`;
+`agent_state_type` is a `JSON_TYPE` discriminator that distinguishes a missing
+state key from an explicit JSON null), and `TOOL_PAUSED`
+(`function_call_id`, `pause_kind`). `WORKFLOW_NODE_STARTING` /
+`WORKFLOW_NODE_COMPLETED` are registered with header-only views for now; their
+typed columns follow the workflow-boundary derivation work.
+
+The `TOOL_COMPLETED` view also exposes the long-running pair keys
+`function_call_id`, `pause_kind`, and `pause_orphan`. These are populated only
+on the resume row that pairs with a `TOOL_PAUSED`; on ordinary (non-long-running)
+tool completions they are **null**. `pause_orphan` is reserved for the pause
+registry and stays null until that ships — treat a null `pause_orphan` as
+"not yet determined", not as "not an orphan".
+
 ---
 
 ## 17. Categorical Evaluation & Real-Time Dashboards
@@ -1505,9 +1524,9 @@ See [`examples/categorical_dashboard.sql`](examples/categorical_dashboard.sql) f
 
 ---
 
-## 18. Context Graph (Property Graph for Agentic Ads)
+## 18. Agent Context Graph (Decision-Trace Extraction)
 
-The **Context Graph** module builds a BigQuery Property Graph that cross-links technical execution traces (TechNodes) with business-domain entities (BizNodes). It enables GQL-based trace reconstruction, causal reasoning, and world-change detection for long-running agent tasks.
+The **Agent Context Graph** module extracts decision traces from your agent's context graph: it builds a BigQuery property graph that cross-links technical execution traces (TechNodes) with the business-domain entities the agent reasoned over (BizNodes), so you can reconstruct any decision — the request, the options weighed, the outcome committed — with GQL traversal, causal reasoning, and world-change detection for long-running agent tasks.
 
 ### Architecture: 4-Pillar Property Graph
 
@@ -1530,7 +1549,7 @@ The **Context Graph** module builds a BigQuery Property Graph that cross-links t
                                        └────────────────────┘
 ```
 
-### Initialize the Context Graph Manager
+### Initialize the Agent Context Graph Manager
 
 ```python
 from bigquery_agent_analytics import ContextGraphManager, ContextGraphConfig
@@ -1558,7 +1577,7 @@ cgm = ContextGraphManager(
 
 ### End-to-End Pipeline
 
-Build the full Context Graph in one call:
+Build the full Agent Context Graph in one call:
 
 ```python
 results = cgm.build_context_graph(
@@ -2057,7 +2076,7 @@ bigquery_agent_analytics/
 │   Core
 │   ├── client.py              ← High-level SDK entry point
 │   ├── trace.py               ← Trace/Span reconstruction & DAG rendering
-│   └── evaluators.py          ← CodeEvaluator + LLMAsJudge + SQL templates
+│   └── evaluators.py          ← SystemEvaluator + LLMAsJudge + SQL templates
 │
 │   Evaluation Harness
 │   ├── trace_evaluator.py     ← BigQueryTraceEvaluator, trajectory matching, replay
@@ -2075,8 +2094,8 @@ bigquery_agent_analytics/
 │   ├── memory_service.py      ← Long-horizon agent memory (requires google-adk)
 │   └── bigframes_evaluator.py ← BigFrames DataFrame evaluator (optional)
 │
-│   Context Graph
-│   └── context_graph.py       ← Property Graph, BizNode extraction, GQL, world-change
+│   Agent Context Graph
+│   └── context_graph.py       ← Decision-trace extraction, GQL, world-change
 │
 │   CLI & Interfaces
 │   ├── cli.py                 ← typer CLI (bq-agent-sdk)

{bigquery_agent_analytics-0.3.4 → bigquery_agent_analytics-0.4.0}/deploy/remote_function/dispatch.py

@@ -25,9 +25,9 @@ import json
 from typing import Any
 
 from bigquery_agent_analytics import Client
-from bigquery_agent_analytics import CodeEvaluator
 from bigquery_agent_analytics import LLMAsJudge
 from bigquery_agent_analytics import serialize
+from bigquery_agent_analytics import SystemEvaluator
 from bigquery_agent_analytics import TraceFilter
 from bigquery_agent_analytics._deploy_runtime import resolve_client_options
 
@@ -137,7 +137,7 @@ def build_filters(params):
 
 
 def build_evaluator(params):
-  """Build CodeEvaluator from params dict."""
+  """Build SystemEvaluator from params dict."""
   metric = params.get("metric", "latency")
   threshold = params.get("threshold")
   fail_on_missing_telemetry = _bool_param(
@@ -145,35 +145,35 @@ def build_evaluator(params):
   )
 
   factories_with_t = {
-      "latency": lambda t: CodeEvaluator.latency(threshold_ms=t),
-      "error_rate": lambda t: CodeEvaluator.error_rate(
+      "latency": lambda t: SystemEvaluator.latency(threshold_ms=t),
+      "error_rate": lambda t: SystemEvaluator.error_rate(
           max_error_rate=t,
       ),
-      "turn_count": lambda t: CodeEvaluator.turn_count(
+      "turn_count": lambda t: SystemEvaluator.turn_count(
           max_turns=int(t),
       ),
-      "token_efficiency": lambda t: CodeEvaluator.token_efficiency(
+      "token_efficiency": lambda t: SystemEvaluator.token_efficiency(
           max_tokens=int(t),
       ),
-      "ttft": lambda t: CodeEvaluator.ttft(threshold_ms=t),
-      "cost": lambda t: CodeEvaluator.cost_per_session(
+      "ttft": lambda t: SystemEvaluator.ttft(threshold_ms=t),
+      "cost": lambda t: SystemEvaluator.cost_per_session(
           max_cost_usd=t,
       ),
   }
   factories_default = {
-      "latency": CodeEvaluator.latency,
-      "error_rate": CodeEvaluator.error_rate,
-      "turn_count": CodeEvaluator.turn_count,
-      "token_efficiency": CodeEvaluator.token_efficiency,
-      "ttft": CodeEvaluator.ttft,
-      "cost": CodeEvaluator.cost_per_session,
+      "latency": SystemEvaluator.latency,
+      "error_rate": SystemEvaluator.error_rate,
+      "turn_count": SystemEvaluator.turn_count,
+      "token_efficiency": SystemEvaluator.token_efficiency,
+      "ttft": SystemEvaluator.ttft,
+      "cost": SystemEvaluator.cost_per_session,
   }
 
   if metric == "context_cache_hit_rate":
     kwargs = {"fail_on_missing_telemetry": fail_on_missing_telemetry}
     if threshold is not None:
       kwargs["min_hit_rate"] = threshold
-    return CodeEvaluator.context_cache_hit_rate(**kwargs)
+    return SystemEvaluator.context_cache_hit_rate(**kwargs)
 
   if metric not in factories_with_t:
     raise ValueError(f"Unknown metric: {metric!r}")

{bigquery_agent_analytics-0.3.4 → bigquery_agent_analytics-0.4.0}/docs/README.md

@@ -16,19 +16,35 @@ architecture, rationale, and implementation plans behind key SDK features.
 |----------|-------------|
 | [hatteras_evaluation.md](hatteras_evaluation.md) | Hatteras-style categorical evaluation design |
 
-## Context & Ontology
+## Agent Context Graph
+
+Extract decision traces from your agent's context graph — the requests an
+agent handled, the options it weighed, and the outcomes it committed —
+materialized into a queryable BigQuery property graph.
 
 | Document | Description |
 |----------|-------------|
+| [codelabs/periodic_materialization.md](codelabs/periodic_materialization.md) | **Start here.** Deploy a graph, seed events, materialize with `bqaa context-graph --graph`, query decision traces in GQL |
+| [guides/scheduled-context-graph-deploy.md](guides/scheduled-context-graph-deploy.md) | Take the deployed graph to a scheduled Cloud Run + Cloud Scheduler production deploy |
+| [guides/conversational-analytics-first.md](guides/conversational-analytics-first.md) | Ask the decision graph questions in plain English before dropping to GQL |
 | [context_graph_v2_design.md](context_graph_v2_design.md) | Property Graph V2 design |
 | [context_graph_v3_design.md](context_graph_v3_design.md) | Property Graph V3 with GQL and world-change detection |
+
+## Ontology & Binding Internals (advanced)
+
+Internal machinery behind the Agent Context Graph's extraction spec. Most users
+never touch these — `bqaa context-graph --graph` derives everything from the
+deployed property graph.
+
+| Document | Description |
+|----------|-------------|
 | [ontology_graph_v4_design.md](ontology_graph_v4_design.md) | YAML-driven ontology extraction and materialization |
 | [ontology_graph_v5_design.md](ontology_graph_v5_design.md) | V5: TTL import, mixed extraction, temporal lineage |
 | [learning_ontology_and_context_graph.md](learning_ontology_and_context_graph.md) | Learning guide for ontology and context graph |
 | [implementation_plan_concept_index_runtime.md](implementation_plan_concept_index_runtime.md) | Phased implementation plan for concept index + runtime entity resolution (issue #58) |
 | [ontology_runtime_reader.md](ontology_runtime_reader.md) | Ontology runtime reader (issue #58 reader follow-on to PR #92). `OntologyRuntime` loads ontology + binding + optional concept-index lookup. `EntityResolver` Protocol + two reference impls: `ExactEntityResolver` (in-memory) + `LabelSynonymResolver` (BQ-backed). `ConceptIndexLookup` is fingerprint-strict: eager `verify()` at construction + every `lookup_*` query includes `WHERE compile_fingerprint = @expected_fp` as defense in depth. Stable failure codes: `FingerprintMismatchError`, `MetaTableMissingError`, `MetaTableEmptyError`, `MetaTableMultipleRowsError`. `table_id` validated at construction (same regex discipline as Phase C's bundle mirror); `verify()` always re-queries (no cache); SKOS traversal helpers (`in_scheme`, `broader`, `narrower`, `related`) + `relationships_by_name` (tuple, never singular) reflect #58's traversal-first contract. NO embedding / LLM / fuzzy in this slice — those are explicit non-goals; future PRs can implement the Protocol without changing the runtime surface. |
 
-## Ontology Reference
+### Component reference
 
 | Document | Description |
 |----------|-------------|
@@ -61,7 +77,7 @@ architecture, rationale, and implementation plans behind key SDK features.
 
 | Document | Description |
 |----------|-------------|
-| [Context Graph periodic materialization playbook](../examples/context_graph/periodic_materialization/README.md) | Customer deployment path for keeping the MAKO context graph fresh on a schedule: local dry-run, Cloud Run Job + Cloud Scheduler deploy with `--smoke`, IAM matrix, schedule guidance, JSON log shape, Cloud Monitoring alert filters, state-table inspection, cleanup, and troubleshooting. |
+| [Agent Context Graph periodic materialization playbook](../examples/context_graph/periodic_materialization/README.md) | Customer deployment path for keeping the MAKO context graph fresh on a schedule: local dry-run, Cloud Run Job + Cloud Scheduler deploy with `--smoke`, IAM matrix, schedule guidance, JSON log shape, Cloud Monitoring alert filters, state-table inspection, cleanup, and troubleshooting. |
 | [proposal_bigquery_agent_cli.md](proposal_bigquery_agent_cli.md) | CLI proposal and command design |
 | [python_udf_support_design.md](python_udf_support_design.md) | BigQuery Python UDF architecture |
 | [remote_function_rationale.md](remote_function_rationale.md) | Cloud Run remote function rationale |