PyPI - google-cloud-db-context-engineering - Versions diffs - 0.7.2__tar.gz → 0.7.3__tar.gz - Mend

google-cloud-db-context-engineering 0.7.2tar.gz → 0.7.3tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Files changed (31) hide show

{google_cloud_db_context_engineering-0.7.2/src/google_cloud_db_context_engineering.egg-info → google_cloud_db_context_engineering-0.7.3}/PKG-INFO RENAMED Viewed

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: google-cloud-db-context-engineering
-Version: 0.7.2
+Version: 0.7.3
 Summary: A FastMCP server for generating natural language to SQL templates from database schemas.
 Requires-Python: >=3.12
 Description-Content-Type: text/markdown
@@ -16,19 +16,17 @@ Requires-Dist: pytest; extra == "test"
 Requires-Dist: pytest-asyncio; extra == "test"
 Dynamic: license-file
-This is not an officially supported Google product. This project is not eligible for the [Google Open Source Software Vulnerability Rewards Program](https://bughunters.google.com/open-source-security), [Google Cloud Platform/SecOps Terms of Service](https://cloud.google.com/terms), [How Gemini for Google Cloud uses your data](https://cloud.google.com/gemini/docs/discover/data-governance). This tool is provided "as is" without warranty of any kind. Users are solely responsible for understanding and managing the tool's interaction with their databases. Use of this tool constitutes acceptance of all risks associated with database access, reading, usage, and modifications.
 # Context Engineering Agent
-The **Context Engineering Agent** is an AI coding agent plugin designed to run in developer agent harnesses (such as Claude Code, Antigravity, or Gemini CLI). It generates, evaluates, and iteratively tunes tailored context artifacts (`ContextSets` comprising `Templates`, `Facets`, and `Value Searches`) to enrich database schemas for **Gemini Data Analytics's data agent developer platform tools**, supporting both **relational SQL** and **Graph Query Language (GQL)** across [AlloyDB](https://docs.cloud.google.com/gemini/data-agents/querydata/alloydb/data-agent-overview), Cloud SQL ([PostgreSQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-postgres/data-agent-overview) / [MySQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-mysql/data-agent-overview)), and [Cloud Spanner (GoogleSQL & Spanner Graph)](https://docs.cloud.google.com/gemini/data-agents/querydata/spanner/data-agent-overview).
+The **Context Engineering Agent** is an AI coding agent plugin designed to run in developer agent harnesses (such as Claude Code, Antigravity, or Gemini CLI). It generates, evaluates, and iteratively tunes tailored context artifacts (`ContextSets` comprising `Templates`, `Facets`, and `Value Searches`) to enrich database schemas for **Gemini Data Analytics's data agent developer platform tools**, supporting both **relational SQL** and **Graph Query Language (GQL)** across [AlloyDB](https://docs.cloud.google.com/gemini/data-agents/querydata/alloydb/data-agent-overview), Cloud SQL ([PostgreSQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-postgres/data-agent-overview) / [MySQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-mysql/data-agent-overview)), and [Cloud Spanner (GoogleSQL, Spanner Graph, and PostgreSQL)](https://docs.cloud.google.com/gemini/data-agents/querydata/spanner/data-agent-overview).
 ---
 ## Why Context Engineering?
-When building data agents and natural language analytics interfaces, accurately translating user intent into database queries—whether relational SQL, pure GQL, or hybrid graph queries—is critical.
+When building data agents and natural language analytics interfaces, accurately translating user intent into database queries—whether relational SQL (PostgreSQL, GoogleSQL, MySQL), pure GQL, or hybrid graph queries—is critical.
-As outlined in **Build Context with Context Engineering Agent** ([AlloyDB](https://docs.cloud.google.com/gemini/data-agents/querydata/alloydb/build-context-gemini-cli) | Cloud SQL: [PostgreSQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-postgres/build-context-gemini-cli) / [MySQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-mysql/build-context-gemini-cli) | [Spanner (GoogleSQL)](https://docs.cloud.google.com/gemini/data-agents/querydata/spanner/build-context-gemini-cli)), by optimizing a `ContextSet` to match your application's expected query stream, the **QueryData API** acts as a data agent tool capable of achieving **~100% NL-to-SQL/GQL translation accuracy with low latency**.
+As outlined in **Build Context with Context Engineering Agent** ([AlloyDB](https://docs.cloud.google.com/gemini/data-agents/querydata/alloydb/build-context-gemini-cli) | Cloud SQL: [PostgreSQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-postgres/build-context-gemini-cli) / [MySQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-mysql/build-context-gemini-cli) | [Spanner](https://docs.cloud.google.com/gemini/data-agents/querydata/spanner/build-context-gemini-cli)), by optimizing a `ContextSet` to match your application's expected query stream, the **QueryData API** acts as a data agent tool capable of achieving **~100% NL-to-SQL/GQL translation accuracy with low latency**.
 ---
@@ -40,7 +38,7 @@ A `ContextSet` is the central artifact generated and managed by the agent, conta
 * **Facets**: Reusable, modular query fragments (e.g., parameterized `WHERE` clauses, specialized join filters, or graph `MATCH` traversal patterns) linked to domain vocabulary.
 * **Value Searches**: Specialized mapping queries that dynamically resolve user-supplied values (e.g., *"Lndn"*) to database records (*"London"*) via the capabilities of the underlying database, such as embedding search, AI operators, or trigram search on relational and graph property tables.
-For full schema details, structure specifications, and dialect-specific JSON representations of `ContextSets`, see the official **Context Sets Overview** ([AlloyDB](https://docs.cloud.google.com/gemini/data-agents/querydata/alloydb/context-sets-overview) | Cloud SQL: [PostgreSQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-postgres/context-sets-overview) / [MySQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-mysql/context-sets-overview) | [Spanner (GoogleSQL)](https://docs.cloud.google.com/gemini/data-agents/querydata/spanner/context-sets-overview)).
+For full schema details, structure specifications, and dialect-specific JSON representations of `ContextSets`, see the official **Context Sets Overview** ([AlloyDB](https://docs.cloud.google.com/gemini/data-agents/querydata/alloydb/context-sets-overview) | Cloud SQL: [PostgreSQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-postgres/context-sets-overview) / [MySQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-mysql/context-sets-overview) | [Spanner](https://docs.cloud.google.com/gemini/data-agents/querydata/spanner/context-sets-overview)).
 ---
@@ -49,13 +47,13 @@ For full schema details, structure specifications, and dialect-specific JSON rep
 Before getting started, prepare your GCP environment, required APIs (Data Analytics API, Gemini for Google Cloud API, Dataplex Universal Catalog API), IAM permissions, and database Data API settings.
 Follow the step-by-step setup guide in the official documentation:
-👉 **Prepare Your Environment**: ([AlloyDB](https://docs.cloud.google.com/gemini/data-agents/querydata/alloydb/build-context-gemini-cli#prepare-your-environment) | Cloud SQL: [PostgreSQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-postgres/build-context-gemini-cli#prepare-your-environment) / [MySQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-mysql/build-context-gemini-cli#prepare-your-environment) | [Spanner (GoogleSQL)](https://docs.cloud.google.com/gemini/data-agents/querydata/spanner/build-context-gemini-cli#prepare-your-environment))
+👉 **Prepare Your Environment**: ([AlloyDB](https://docs.cloud.google.com/gemini/data-agents/querydata/alloydb/build-context-gemini-cli#prepare-your-environment) | Cloud SQL: [PostgreSQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-postgres/build-context-gemini-cli#prepare-your-environment) / [MySQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-mysql/build-context-gemini-cli#prepare-your-environment) | [Spanner](https://docs.cloud.google.com/gemini/data-agents/querydata/spanner/build-context-gemini-cli#prepare-your-environment))
 ---
 ## Primary Workflow Phases
-The agent enables you to craft an optimized context for QueryData API through three primary phases:
+The agent enables you to craft an optimized context for QueryData API through three primary phases. Depending on your needs, you may also toggle the agent to skip phases. For example, if you have a dataset already, you can skip directly to context optimization "optimize context using dataset in \<file\>."
 ### Phase 1: Artifact Ingestion
 *Why it matters: Without broader context on the application's goals and scope, AI models generate sterile queries based solely on database column names, missing how your users actually ask for information.*
@@ -80,8 +78,7 @@ The optimization loop creates an initial `ContextSet` and then iteratively refin
 1. **Bootstrap**: Generate an initial baseline context.
 2. **Evaluate**: Measure context effectiveness against a golden dataset.
 3. **Hill-Climbing**: Perform gap analysis on failures and generate automated fixes.
-4. **Iterate**: Apply the improved context and re-run evaluation to continuously improve quality.
-5. **Final Validation** (Optional): Verify mutations against a separated test set to ensure generalization and prevent overfitting.
+4. **Iterate**: Apply the improved context and re-run evaluation to continuously improve quality until we reach an optimal point.
 *Note: While there is a typical ordering for these CUJs, the agent is flexible in how you want to execute. You can run the full pipeline end-to-end, trigger any individual phase, or ask for targeted changes to the `ContextSet`.*

{google_cloud_db_context_engineering-0.7.2 → google_cloud_db_context_engineering-0.7.3}/README.md RENAMED Viewed

@@ -1,16 +1,14 @@
-This is not an officially supported Google product. This project is not eligible for the [Google Open Source Software Vulnerability Rewards Program](https://bughunters.google.com/open-source-security), [Google Cloud Platform/SecOps Terms of Service](https://cloud.google.com/terms), [How Gemini for Google Cloud uses your data](https://cloud.google.com/gemini/docs/discover/data-governance). This tool is provided "as is" without warranty of any kind. Users are solely responsible for understanding and managing the tool's interaction with their databases. Use of this tool constitutes acceptance of all risks associated with database access, reading, usage, and modifications.
 # Context Engineering Agent
-The **Context Engineering Agent** is an AI coding agent plugin designed to run in developer agent harnesses (such as Claude Code, Antigravity, or Gemini CLI). It generates, evaluates, and iteratively tunes tailored context artifacts (`ContextSets` comprising `Templates`, `Facets`, and `Value Searches`) to enrich database schemas for **Gemini Data Analytics's data agent developer platform tools**, supporting both **relational SQL** and **Graph Query Language (GQL)** across [AlloyDB](https://docs.cloud.google.com/gemini/data-agents/querydata/alloydb/data-agent-overview), Cloud SQL ([PostgreSQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-postgres/data-agent-overview) / [MySQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-mysql/data-agent-overview)), and [Cloud Spanner (GoogleSQL & Spanner Graph)](https://docs.cloud.google.com/gemini/data-agents/querydata/spanner/data-agent-overview).
+The **Context Engineering Agent** is an AI coding agent plugin designed to run in developer agent harnesses (such as Claude Code, Antigravity, or Gemini CLI). It generates, evaluates, and iteratively tunes tailored context artifacts (`ContextSets` comprising `Templates`, `Facets`, and `Value Searches`) to enrich database schemas for **Gemini Data Analytics's data agent developer platform tools**, supporting both **relational SQL** and **Graph Query Language (GQL)** across [AlloyDB](https://docs.cloud.google.com/gemini/data-agents/querydata/alloydb/data-agent-overview), Cloud SQL ([PostgreSQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-postgres/data-agent-overview) / [MySQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-mysql/data-agent-overview)), and [Cloud Spanner (GoogleSQL, Spanner Graph, and PostgreSQL)](https://docs.cloud.google.com/gemini/data-agents/querydata/spanner/data-agent-overview).
 ---
 ## Why Context Engineering?
-When building data agents and natural language analytics interfaces, accurately translating user intent into database queries—whether relational SQL, pure GQL, or hybrid graph queries—is critical.
+When building data agents and natural language analytics interfaces, accurately translating user intent into database queries—whether relational SQL (PostgreSQL, GoogleSQL, MySQL), pure GQL, or hybrid graph queries—is critical.
-As outlined in **Build Context with Context Engineering Agent** ([AlloyDB](https://docs.cloud.google.com/gemini/data-agents/querydata/alloydb/build-context-gemini-cli) | Cloud SQL: [PostgreSQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-postgres/build-context-gemini-cli) / [MySQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-mysql/build-context-gemini-cli) | [Spanner (GoogleSQL)](https://docs.cloud.google.com/gemini/data-agents/querydata/spanner/build-context-gemini-cli)), by optimizing a `ContextSet` to match your application's expected query stream, the **QueryData API** acts as a data agent tool capable of achieving **~100% NL-to-SQL/GQL translation accuracy with low latency**.
+As outlined in **Build Context with Context Engineering Agent** ([AlloyDB](https://docs.cloud.google.com/gemini/data-agents/querydata/alloydb/build-context-gemini-cli) | Cloud SQL: [PostgreSQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-postgres/build-context-gemini-cli) / [MySQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-mysql/build-context-gemini-cli) | [Spanner](https://docs.cloud.google.com/gemini/data-agents/querydata/spanner/build-context-gemini-cli)), by optimizing a `ContextSet` to match your application's expected query stream, the **QueryData API** acts as a data agent tool capable of achieving **~100% NL-to-SQL/GQL translation accuracy with low latency**.
 ---
@@ -22,7 +20,7 @@ A `ContextSet` is the central artifact generated and managed by the agent, conta
 * **Facets**: Reusable, modular query fragments (e.g., parameterized `WHERE` clauses, specialized join filters, or graph `MATCH` traversal patterns) linked to domain vocabulary.
 * **Value Searches**: Specialized mapping queries that dynamically resolve user-supplied values (e.g., *"Lndn"*) to database records (*"London"*) via the capabilities of the underlying database, such as embedding search, AI operators, or trigram search on relational and graph property tables.
-For full schema details, structure specifications, and dialect-specific JSON representations of `ContextSets`, see the official **Context Sets Overview** ([AlloyDB](https://docs.cloud.google.com/gemini/data-agents/querydata/alloydb/context-sets-overview) | Cloud SQL: [PostgreSQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-postgres/context-sets-overview) / [MySQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-mysql/context-sets-overview) | [Spanner (GoogleSQL)](https://docs.cloud.google.com/gemini/data-agents/querydata/spanner/context-sets-overview)).
+For full schema details, structure specifications, and dialect-specific JSON representations of `ContextSets`, see the official **Context Sets Overview** ([AlloyDB](https://docs.cloud.google.com/gemini/data-agents/querydata/alloydb/context-sets-overview) | Cloud SQL: [PostgreSQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-postgres/context-sets-overview) / [MySQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-mysql/context-sets-overview) | [Spanner](https://docs.cloud.google.com/gemini/data-agents/querydata/spanner/context-sets-overview)).
 ---
@@ -31,13 +29,13 @@ For full schema details, structure specifications, and dialect-specific JSON rep
 Before getting started, prepare your GCP environment, required APIs (Data Analytics API, Gemini for Google Cloud API, Dataplex Universal Catalog API), IAM permissions, and database Data API settings.
 Follow the step-by-step setup guide in the official documentation:
-👉 **Prepare Your Environment**: ([AlloyDB](https://docs.cloud.google.com/gemini/data-agents/querydata/alloydb/build-context-gemini-cli#prepare-your-environment) | Cloud SQL: [PostgreSQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-postgres/build-context-gemini-cli#prepare-your-environment) / [MySQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-mysql/build-context-gemini-cli#prepare-your-environment) | [Spanner (GoogleSQL)](https://docs.cloud.google.com/gemini/data-agents/querydata/spanner/build-context-gemini-cli#prepare-your-environment))
+👉 **Prepare Your Environment**: ([AlloyDB](https://docs.cloud.google.com/gemini/data-agents/querydata/alloydb/build-context-gemini-cli#prepare-your-environment) | Cloud SQL: [PostgreSQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-postgres/build-context-gemini-cli#prepare-your-environment) / [MySQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-mysql/build-context-gemini-cli#prepare-your-environment) | [Spanner](https://docs.cloud.google.com/gemini/data-agents/querydata/spanner/build-context-gemini-cli#prepare-your-environment))
 ---
 ## Primary Workflow Phases
-The agent enables you to craft an optimized context for QueryData API through three primary phases:
+The agent enables you to craft an optimized context for QueryData API through three primary phases. Depending on your needs, you may also toggle the agent to skip phases. For example, if you have a dataset already, you can skip directly to context optimization "optimize context using dataset in \<file\>."
 ### Phase 1: Artifact Ingestion
 *Why it matters: Without broader context on the application's goals and scope, AI models generate sterile queries based solely on database column names, missing how your users actually ask for information.*
@@ -62,8 +60,7 @@ The optimization loop creates an initial `ContextSet` and then iteratively refin
 1. **Bootstrap**: Generate an initial baseline context.
 2. **Evaluate**: Measure context effectiveness against a golden dataset.
 3. **Hill-Climbing**: Perform gap analysis on failures and generate automated fixes.
-4. **Iterate**: Apply the improved context and re-run evaluation to continuously improve quality.
-5. **Final Validation** (Optional): Verify mutations against a separated test set to ensure generalization and prevent overfitting.
+4. **Iterate**: Apply the improved context and re-run evaluation to continuously improve quality until we reach an optimal point.
 *Note: While there is a typical ordering for these CUJs, the agent is flexible in how you want to execute. You can run the full pipeline end-to-end, trigger any individual phase, or ask for targeted changes to the `ContextSet`.*

{google_cloud_db_context_engineering-0.7.2 → google_cloud_db_context_engineering-0.7.3}/pyproject.toml RENAMED Viewed

@@ -1,6 +1,6 @@
 [project]
 name = "google-cloud-db-context-engineering"
-version = "0.7.2"
+version = "0.7.3"
 description = "A FastMCP server for generating natural language to SQL templates from database schemas."
 readme = "README.md"
 requires-python = ">=3.12"
@@ -46,7 +46,7 @@ dev = [
 [tool.db-context-engineering]
 toolbox_version = "1.4.0"
-evalbench_version = "1.12.0"
+evalbench_version = "1.17.0"
 [tool.ruff]
 line-length = 88

{google_cloud_db_context_engineering-0.7.2 → google_cloud_db_context_engineering-0.7.3}/src/google/cloud/db_context_enrichment/common/context_mutator.py RENAMED Viewed

@@ -48,7 +48,7 @@ def mutate_context_set(file_path: str, mutations: list[Mutation]) -> None:
         context_set = context.ContextSet()
     else:
         try:
-            with open(file_path) as f:
+            with open(file_path, encoding="utf-8") as f:
                 raw_data = json.load(f)
             context_set = context.ContextSet.model_validate(raw_data)
         except ValidationError as e:
@@ -126,7 +126,7 @@ def mutate_context_set(file_path: str, mutations: list[Mutation]) -> None:
     # 4. Save validated ContextSet
     try:
         os.makedirs(os.path.dirname(os.path.abspath(file_path)), exist_ok=True)
-        with open(file_path, "w") as f:
+        with open(file_path, "w", encoding="utf-8") as f:
             f.write(context_set.model_dump_json(indent=2, exclude_none=True))
     except OSError as e:
         raise RuntimeError(f"Error saving ContextSet to {file_path}: {e}") from e

google_cloud_db_context_engineering-0.7.3/src/google/cloud/db_context_enrichment/common/context_validator.py ADDED Viewed

@@ -0,0 +1,188 @@
+import json
+from typing import Any
+from pydantic import ValidationError
+from google.cloud.db_context_enrichment.model import context
+_ATTR_TO_TYPE = {
+    "templates": "template",
+    "facets": "facet",
+    "value_searches": "value_search",
+}
+# ContextSet accepts camelCase aliases (Context Store returns them on download)
+# and the deprecated "fragments" name. Map them back to the canonical attribute
+# so every check below only has to handle one spelling.
+_ALIAS_TO_ATTR = {
+    "valueSearches": "value_searches",
+    "fragments": "facets",
+}
+def validate_context_set(file_path: str) -> dict[str, Any]:
+    """Validate a ContextSet file and return a structured report of issues.
+    Always returns a dict of the shape:
+      {
+        "valid": bool,
+        "issues": [
+          {
+            "location": {"type": str, "index": int} | None,
+            "message": str,
+          },
+          ...
+        ],
+      }
+    File access failures (missing path, permission denied, path is a directory,
+    etc.) are surfaced as a single issue rather than raised.
+    """
+    try:
+        with open(file_path, encoding="utf-8") as f:
+            text = f.read()
+    except OSError as e:
+        return {
+            "valid": False,
+            "issues": [
+                _make_issue(f"Could not read file {file_path}: {type(e).__name__}: {e}")
+            ],
+        }
+    if text.strip() == "":
+        return {"valid": True, "issues": []}
+    try:
+        raw = json.loads(text)
+    except json.JSONDecodeError as e:
+        snippet = e.doc[max(0, e.pos - 30) : e.pos + 30]
+        return {
+            "valid": False,
+            "issues": [_make_issue(f"File is not valid JSON: {e}. Near: {snippet!r}")],
+        }
+    if not isinstance(raw, dict):
+        return {
+            "valid": False,
+            "issues": [_make_issue("Top-level value must be a JSON object")],
+        }
+    raw = _normalize_aliases(raw)
+    issues: list[dict[str, Any]] = []
+    issues.extend(_check_pydantic(raw))
+    issues.extend(_check_duplicates(raw))
+    issues.extend(_check_value_search_value_param(raw))
+    return {"valid": len(issues) == 0, "issues": issues}
+def _make_issue(message: str, location: dict[str, Any] | None = None) -> dict[str, Any]:
+    return {"location": location, "message": message}
+def _normalize_aliases(raw: dict[str, Any]) -> dict[str, Any]:
+    """Rewrite alias keys to their canonical ContextSet attribute names."""
+    return {_ALIAS_TO_ATTR.get(key, key): value for key, value in raw.items()}
+def _check_pydantic(raw: dict[str, Any]) -> list[dict[str, Any]]:
+    issues: list[dict[str, Any]] = []
+    try:
+        context.ContextSet.model_validate(raw)
+        return issues
+    except ValidationError as e:
+        for err in e.errors():
+            loc = err.get("loc", ())
+            msg = err.get("msg", "validation error")
+            location: dict[str, Any] | None = None
+            descriptor = ""
+            if len(loc) >= 1 and loc[0] in _ATTR_TO_TYPE:
+                item_type = _ATTR_TO_TYPE[loc[0]]
+                if len(loc) >= 2 and isinstance(loc[1], int):
+                    item_index = loc[1]
+                    location = {"type": item_type, "index": item_index}
+                    items = raw.get(loc[0])
+                    if isinstance(items, list) and 0 <= item_index < len(items):
+                        descriptor = _describe(item_type, items[item_index])
+            field_path = ".".join(str(p) for p in loc) if loc else ""
+            parts = [msg]
+            if field_path:
+                parts.append(f"at {field_path}")
+            if descriptor:
+                parts.append(descriptor)
+            full_msg = parts[0] + (
+                " (" + "; ".join(parts[1:]) + ")" if len(parts) > 1 else ""
+            )
+            issues.append(_make_issue(full_msg, location=location))
+    return issues
+def _check_duplicates(raw: dict[str, Any]) -> list[dict[str, Any]]:
+    issues: list[dict[str, Any]] = []
+    for attr, item_type in _ATTR_TO_TYPE.items():
+        items = raw.get(attr)
+        if not isinstance(items, list):
+            continue
+        seen: dict[str, int] = {}
+        for idx, item in enumerate(items):
+            try:
+                key = _canonical(item)
+            except (TypeError, ValueError):
+                continue
+            if key in seen:
+                descriptor = _describe(item_type, item)
+                suffix = f" ({descriptor})" if descriptor else ""
+                issues.append(
+                    _make_issue(
+                        f"Exact duplicate of {item_type} at index {seen[key]}{suffix}",
+                        location={"type": item_type, "index": idx},
+                    )
+                )
+            else:
+                seen[key] = idx
+    return issues
+def _check_value_search_value_param(raw: dict[str, Any]) -> list[dict[str, Any]]:
+    issues: list[dict[str, Any]] = []
+    items = raw.get("value_searches")
+    if not isinstance(items, list):
+        return issues
+    for idx, item in enumerate(items):
+        if not isinstance(item, dict):
+            continue
+        query = item.get("query")
+        if isinstance(query, str) and "$value" not in query:
+            descriptor = _describe("value_search", item)
+            suffix = f" ({descriptor})" if descriptor else ""
+            issues.append(
+                _make_issue(
+                    f"value_search query must reference the $value parameter{suffix}",
+                    location={"type": "value_search", "index": idx},
+                )
+            )
+    return issues
+def _describe(item_type: str, item: Any) -> str:
+    """Short human-readable identifier for an item, e.g. "intent: 'active users'"."""
+    if not isinstance(item, dict):
+        return ""
+    if item_type == "template":
+        nl = item.get("nl_query")
+        return f"nl_query: {nl!r}" if nl else ""
+    if item_type == "facet":
+        intent = item.get("intent")
+        return f"intent: {intent!r}" if intent else ""
+    if item_type == "value_search":
+        ct = item.get("concept_type")
+        return f"concept_type: {ct!r}" if ct else ""
+    return ""
+def _canonical(obj: Any) -> str:
+    return json.dumps(obj, sort_keys=True, ensure_ascii=False)

{google_cloud_db_context_engineering-0.7.2 → google_cloud_db_context_engineering-0.7.3}/src/google/cloud/db_context_enrichment/evaluate/db_generators/spanner.py RENAMED Viewed

@@ -9,10 +9,10 @@ class SpannerConfigGenerator(BaseDBConfigGenerator):
     """
     Dedicated generator mapping properties to explicit Spanner configuration
     topologies utilized by both EvalBench binaries and GDA Context objects.
+    Supports both GoogleSQL and PostgreSQL dialects.
     """
     SOURCE_TYPE = "spanner"
-    DIALECT = "spanner_gsql"
     REQUIRED_FIELDS = BaseDBConfigGenerator.REQUIRED_FIELDS | {
         "project",
         "instance",
@@ -25,6 +25,40 @@ class SpannerConfigGenerator(BaseDBConfigGenerator):
         self.instance = params.get("instance")
         self.database = params.get("database")
+        raw_dialect = (
+            params.get("dialect")
+            or params.get("engine")
+            or params.get("database_dialect")
+        )
+        if not raw_dialect and params.get("type") in ("spanner-postgres", "spanner-pg"):
+            raw_dialect = "POSTGRESQL"
+        if raw_dialect:
+            normalized = str(raw_dialect).strip().lower().replace("-", "_")
+            if normalized in (
+                "postgresql",
+                "postgres",
+                "spanner_pg",
+                "pg",
+                "spanner_postgres",
+            ):
+                self.engine = "POSTGRESQL"
+                self._dialect = "spanner_pg"
+            elif normalized in ("google_sql", "googlesql", "spanner_gsql", "gsql"):
+                self.engine = "GOOGLE_SQL"
+                self._dialect = "spanner_gsql"
+            else:
+                raise ValueError(
+                    f"Unsupported Spanner dialect/engine: '{raw_dialect}'. Must be 'GOOGLE_SQL' or 'POSTGRESQL'."
+                )
+        else:
+            self.engine = "GOOGLE_SQL"
+            self._dialect = "spanner_gsql"
+    @property
+    def DIALECT(self) -> str:
+        return self._dialect
     def generate_db_config(self) -> str:
         db_type = "spanner"
         db_path = f"projects/{self.project}/instances/{self.instance}/databases/{self.database}"
@@ -44,12 +78,16 @@ class SpannerConfigGenerator(BaseDBConfigGenerator):
     def build_datasource_reference(self, context_set_id: str) -> dict[str, Any]:
         database_ref: dict[str, Any] = {
-            "engine": "GOOGLE_SQL",
+            "engine": self.engine,
             "project_id": self.project,
             "instance_id": self.instance,
             "database_id": self.database,
         }
         if graph_ids := self.params.get("graph_ids"):
+            if self.engine == "POSTGRESQL":
+                raise ValueError(
+                    "graph_ids is not supported for Spanner PostgreSQL dialect"
+                )
             if not isinstance(graph_ids, list) or not all(
                 isinstance(g, str) for g in graph_ids
             ):

{google_cloud_db_context_engineering-0.7.2 → google_cloud_db_context_engineering-0.7.3}/src/google/cloud/db_context_enrichment/evaluate/evaluate_generator.py RENAMED Viewed

@@ -76,10 +76,10 @@ def _extract_toolbox_params(
         with open(toolbox_config_path) as f:
             content = f.read()
             interpolated = _interpolate_env_vars(content)
-            docs = yaml.safe_load_all(interpolated)
+            docs = [doc for doc in yaml.safe_load_all(interpolated) if doc]
+            source_doc = None
             for doc in docs:
-                if not doc:
-                    continue
                 if (
                     doc.get("kind") == "source"
                     and doc.get("name") == toolbox_source_name
@@ -88,11 +88,24 @@ def _extract_toolbox_params(
                         raise ValueError(
                             f"Selected source '{toolbox_source_name}' is missing the 'type' field."
                         )
-                    return doc
+                    source_doc = doc
+                    break
+            if not source_doc:
+                raise ValueError(
+                    f"Could not find a 'kind: source' named '{toolbox_source_name}' in {toolbox_config_path}"
+                )
+            # For Spanner sources, state.md is the authoritative single source of truth for graph_ids
+            # (QueryData API requires explicit graph_ids in model_config.yaml, whereas tools.yaml
+            # only configures MCP Toolbox runtime tools and parameters).
+            if source_doc.get("type") == "spanner":
+                state_md_dir = os.path.dirname(toolbox_config_path)
+                state_md_path = os.path.join(state_md_dir, "state.md")
+                if graph_ids := _parse_graph_ids_from_state_md(state_md_path):
+                    source_doc["graph_ids"] = graph_ids
-            raise ValueError(
-                f"Could not find a 'kind: source' named '{toolbox_source_name}' in {toolbox_config_path}"
-            )
+            return source_doc
     except FileNotFoundError:
         raise ValueError(f"Config file not found: {toolbox_config_path}")
@@ -104,6 +117,72 @@ def _extract_toolbox_params(
         raise ValueError(f"Failed to parse {toolbox_config_path} as YAML: {e}")
+def _parse_graph_ids_from_state_md(state_md_path: str) -> list[str] | None:
+    """Parses graph_ids from state.md.
+    state.md is the authoritative source of truth for the database and graph scope
+    because QueryData API requires explicit graph_ids in model_config.yaml to evaluate
+    property graphs, whereas tools.yaml only configures MCP Toolbox runtime tools.
+    """
+    if not os.path.exists(state_md_path):
+        return None
+    with open(state_md_path, encoding="utf-8") as f:
+        content = f.read()
+    match = re.search(
+        r"(?:^[ \t]*[-*][ \t]*)?\*\*Graph\s+Ids?:?\*\*:?[ \t]*([^\n]*)",
+        content,
+        re.MULTILINE | re.IGNORECASE,
+    )
+    if not match:
+        return None
+    val_str = match.group(1).split("#")[0].strip()
+    # If empty on the same line, check for multiline sub-bullets
+    if not val_str:
+        after_match = content[match.end() :]
+        bullet_items = []
+        for line in after_match.splitlines():
+            line_stripped = line.strip()
+            if not line_stripped:
+                continue
+            if line_stripped.startswith(("-", "*")) and not re.match(
+                r"^[-*]\s*\*\*", line_stripped
+            ):
+                item = line_stripped.lstrip("-* ").split("#")[0].strip().strip("'\"`")
+                if item and item.lower() not in (
+                    "none",
+                    "n/a",
+                    "null",
+                    "nil",
+                    "-",
+                ):
+                    bullet_items.append(item)
+            elif (
+                line_stripped.startswith("#")
+                or line_stripped.startswith("- **")
+                or line_stripped.startswith("* **")
+            ):
+                break
+            else:
+                break
+        return bullet_items if bullet_items else None
+    # Handle explicit empty / none indicators
+    if val_str.lower() in ("none", "n/a", "null", "nil", "-", "[]", ""):
+        return None
+    if val_str.startswith("[") and val_str.endswith("]"):
+        val_str = val_str[1:-1]
+    graphs = [
+        g.strip().strip("'\"`")
+        for g in val_str.split(",")
+        if g.strip().strip("'\"`")
+        and g.strip().strip("'\"`").lower() not in ("none", "n/a", "null", "nil", "-")
+    ]
+    return graphs if graphs else None
 def _interpolate_env_vars(raw_yaml: str) -> str:
     """Replaces ${ENV_NAME} or ${ENV_NAME:default_value} with environment variables."""
     # Matches ${VAR_NAME} or ${VAR_NAME:fallback}
@@ -133,6 +212,8 @@ def _get_db_generator(params: dict[str, Any]) -> BaseDBConfigGenerator:
         PostgresConfigGenerator.SOURCE_TYPE: PostgresConfigGenerator,
         MySQLConfigGenerator.SOURCE_TYPE: MySQLConfigGenerator,
         SpannerConfigGenerator.SOURCE_TYPE: SpannerConfigGenerator,
+        "spanner-postgres": SpannerConfigGenerator,
+        "spanner-pg": SpannerConfigGenerator,
     }
     if source_type not in generators:

{google_cloud_db_context_engineering-0.7.2 → google_cloud_db_context_engineering-0.7.3}/src/google/cloud/db_context_enrichment/main.py RENAMED Viewed

@@ -6,6 +6,7 @@ from fastmcp import FastMCP
 from google.cloud.db_context_enrichment.common import (
     context_mutator,
     context_store_client,
+    context_validator,
 )
 from google.cloud.db_context_enrichment.dataset import dataset_generator
 from google.cloud.db_context_enrichment.evaluate import (
@@ -258,6 +259,30 @@ def mutate_context_set(
         return f"Error applying mutations: {str(e)}"
+@mcp.tool
+def validate_context_set(file_path: str) -> str:
+    """
+    Validate a ContextSet JSON file for structural and convention issues. Reports issues only — does not fix them. The caller (agent) is expected to apply fixes via `mutate_context_set`, then re-run validation until `valid` is true.
+    Args:
+        file_path: Absolute path to the ContextSet file.
+    Returns:
+        A JSON string of the shape:
+          {
+            "valid": bool,
+            "issues": [
+              {
+                "location": {"type": "template" | "facet" | "value_search", "index": int} | null,
+                "message": str
+              },
+              ...
+            ]
+          }
+    """
+    return json.dumps(context_validator.validate_context_set(file_path), indent=2)
 @mcp.tool
 async def read_evaluation_result(
     run_folder_path: str, offset: int = 0, batch_size: int = 10

{google_cloud_db_context_engineering-0.7.2 → google_cloud_db_context_engineering-0.7.3/src/google_cloud_db_context_engineering.egg-info}/PKG-INFO RENAMED Viewed

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: google-cloud-db-context-engineering
-Version: 0.7.2
+Version: 0.7.3
 Summary: A FastMCP server for generating natural language to SQL templates from database schemas.
 Requires-Python: >=3.12
 Description-Content-Type: text/markdown
@@ -16,19 +16,17 @@ Requires-Dist: pytest; extra == "test"
 Requires-Dist: pytest-asyncio; extra == "test"
 Dynamic: license-file
-This is not an officially supported Google product. This project is not eligible for the [Google Open Source Software Vulnerability Rewards Program](https://bughunters.google.com/open-source-security), [Google Cloud Platform/SecOps Terms of Service](https://cloud.google.com/terms), [How Gemini for Google Cloud uses your data](https://cloud.google.com/gemini/docs/discover/data-governance). This tool is provided "as is" without warranty of any kind. Users are solely responsible for understanding and managing the tool's interaction with their databases. Use of this tool constitutes acceptance of all risks associated with database access, reading, usage, and modifications.
 # Context Engineering Agent
-The **Context Engineering Agent** is an AI coding agent plugin designed to run in developer agent harnesses (such as Claude Code, Antigravity, or Gemini CLI). It generates, evaluates, and iteratively tunes tailored context artifacts (`ContextSets` comprising `Templates`, `Facets`, and `Value Searches`) to enrich database schemas for **Gemini Data Analytics's data agent developer platform tools**, supporting both **relational SQL** and **Graph Query Language (GQL)** across [AlloyDB](https://docs.cloud.google.com/gemini/data-agents/querydata/alloydb/data-agent-overview), Cloud SQL ([PostgreSQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-postgres/data-agent-overview) / [MySQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-mysql/data-agent-overview)), and [Cloud Spanner (GoogleSQL & Spanner Graph)](https://docs.cloud.google.com/gemini/data-agents/querydata/spanner/data-agent-overview).
+The **Context Engineering Agent** is an AI coding agent plugin designed to run in developer agent harnesses (such as Claude Code, Antigravity, or Gemini CLI). It generates, evaluates, and iteratively tunes tailored context artifacts (`ContextSets` comprising `Templates`, `Facets`, and `Value Searches`) to enrich database schemas for **Gemini Data Analytics's data agent developer platform tools**, supporting both **relational SQL** and **Graph Query Language (GQL)** across [AlloyDB](https://docs.cloud.google.com/gemini/data-agents/querydata/alloydb/data-agent-overview), Cloud SQL ([PostgreSQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-postgres/data-agent-overview) / [MySQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-mysql/data-agent-overview)), and [Cloud Spanner (GoogleSQL, Spanner Graph, and PostgreSQL)](https://docs.cloud.google.com/gemini/data-agents/querydata/spanner/data-agent-overview).
 ---
 ## Why Context Engineering?
-When building data agents and natural language analytics interfaces, accurately translating user intent into database queries—whether relational SQL, pure GQL, or hybrid graph queries—is critical.
+When building data agents and natural language analytics interfaces, accurately translating user intent into database queries—whether relational SQL (PostgreSQL, GoogleSQL, MySQL), pure GQL, or hybrid graph queries—is critical.
-As outlined in **Build Context with Context Engineering Agent** ([AlloyDB](https://docs.cloud.google.com/gemini/data-agents/querydata/alloydb/build-context-gemini-cli) | Cloud SQL: [PostgreSQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-postgres/build-context-gemini-cli) / [MySQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-mysql/build-context-gemini-cli) | [Spanner (GoogleSQL)](https://docs.cloud.google.com/gemini/data-agents/querydata/spanner/build-context-gemini-cli)), by optimizing a `ContextSet` to match your application's expected query stream, the **QueryData API** acts as a data agent tool capable of achieving **~100% NL-to-SQL/GQL translation accuracy with low latency**.
+As outlined in **Build Context with Context Engineering Agent** ([AlloyDB](https://docs.cloud.google.com/gemini/data-agents/querydata/alloydb/build-context-gemini-cli) | Cloud SQL: [PostgreSQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-postgres/build-context-gemini-cli) / [MySQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-mysql/build-context-gemini-cli) | [Spanner](https://docs.cloud.google.com/gemini/data-agents/querydata/spanner/build-context-gemini-cli)), by optimizing a `ContextSet` to match your application's expected query stream, the **QueryData API** acts as a data agent tool capable of achieving **~100% NL-to-SQL/GQL translation accuracy with low latency**.
 ---
@@ -40,7 +38,7 @@ A `ContextSet` is the central artifact generated and managed by the agent, conta
 * **Facets**: Reusable, modular query fragments (e.g., parameterized `WHERE` clauses, specialized join filters, or graph `MATCH` traversal patterns) linked to domain vocabulary.
 * **Value Searches**: Specialized mapping queries that dynamically resolve user-supplied values (e.g., *"Lndn"*) to database records (*"London"*) via the capabilities of the underlying database, such as embedding search, AI operators, or trigram search on relational and graph property tables.
-For full schema details, structure specifications, and dialect-specific JSON representations of `ContextSets`, see the official **Context Sets Overview** ([AlloyDB](https://docs.cloud.google.com/gemini/data-agents/querydata/alloydb/context-sets-overview) | Cloud SQL: [PostgreSQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-postgres/context-sets-overview) / [MySQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-mysql/context-sets-overview) | [Spanner (GoogleSQL)](https://docs.cloud.google.com/gemini/data-agents/querydata/spanner/context-sets-overview)).
+For full schema details, structure specifications, and dialect-specific JSON representations of `ContextSets`, see the official **Context Sets Overview** ([AlloyDB](https://docs.cloud.google.com/gemini/data-agents/querydata/alloydb/context-sets-overview) | Cloud SQL: [PostgreSQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-postgres/context-sets-overview) / [MySQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-mysql/context-sets-overview) | [Spanner](https://docs.cloud.google.com/gemini/data-agents/querydata/spanner/context-sets-overview)).
 ---
@@ -49,13 +47,13 @@ For full schema details, structure specifications, and dialect-specific JSON rep
 Before getting started, prepare your GCP environment, required APIs (Data Analytics API, Gemini for Google Cloud API, Dataplex Universal Catalog API), IAM permissions, and database Data API settings.
 Follow the step-by-step setup guide in the official documentation:
-👉 **Prepare Your Environment**: ([AlloyDB](https://docs.cloud.google.com/gemini/data-agents/querydata/alloydb/build-context-gemini-cli#prepare-your-environment) | Cloud SQL: [PostgreSQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-postgres/build-context-gemini-cli#prepare-your-environment) / [MySQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-mysql/build-context-gemini-cli#prepare-your-environment) | [Spanner (GoogleSQL)](https://docs.cloud.google.com/gemini/data-agents/querydata/spanner/build-context-gemini-cli#prepare-your-environment))
+👉 **Prepare Your Environment**: ([AlloyDB](https://docs.cloud.google.com/gemini/data-agents/querydata/alloydb/build-context-gemini-cli#prepare-your-environment) | Cloud SQL: [PostgreSQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-postgres/build-context-gemini-cli#prepare-your-environment) / [MySQL](https://docs.cloud.google.com/gemini/data-agents/querydata/sql-mysql/build-context-gemini-cli#prepare-your-environment) | [Spanner](https://docs.cloud.google.com/gemini/data-agents/querydata/spanner/build-context-gemini-cli#prepare-your-environment))
 ---
 ## Primary Workflow Phases
-The agent enables you to craft an optimized context for QueryData API through three primary phases:
+The agent enables you to craft an optimized context for QueryData API through three primary phases. Depending on your needs, you may also toggle the agent to skip phases. For example, if you have a dataset already, you can skip directly to context optimization "optimize context using dataset in \<file\>."
 ### Phase 1: Artifact Ingestion
 *Why it matters: Without broader context on the application's goals and scope, AI models generate sterile queries based solely on database column names, missing how your users actually ask for information.*
@@ -80,8 +78,7 @@ The optimization loop creates an initial `ContextSet` and then iteratively refin
 1. **Bootstrap**: Generate an initial baseline context.
 2. **Evaluate**: Measure context effectiveness against a golden dataset.
 3. **Hill-Climbing**: Perform gap analysis on failures and generate automated fixes.
-4. **Iterate**: Apply the improved context and re-run evaluation to continuously improve quality.
-5. **Final Validation** (Optional): Verify mutations against a separated test set to ensure generalization and prevent overfitting.
+4. **Iterate**: Apply the improved context and re-run evaluation to continuously improve quality until we reach an optimal point.
 *Note: While there is a typical ordering for these CUJs, the agent is flexible in how you want to execute. You can run the full pipeline end-to-end, trigger any individual phase, or ask for targeted changes to the `ContextSet`.*

{google_cloud_db_context_engineering-0.7.2 → google_cloud_db_context_engineering-0.7.3}/src/google_cloud_db_context_engineering.egg-info/SOURCES.txt RENAMED Viewed

@@ -7,6 +7,7 @@ src/google/cloud/db_context_enrichment/common/__init__.py
 src/google/cloud/db_context_enrichment/common/config.py
 src/google/cloud/db_context_enrichment/common/context_mutator.py
 src/google/cloud/db_context_enrichment/common/context_store_client.py
+src/google/cloud/db_context_enrichment/common/context_validator.py
 src/google/cloud/db_context_enrichment/dataset/__init__.py
 src/google/cloud/db_context_enrichment/dataset/dataset_generator.py
 src/google/cloud/db_context_enrichment/evaluate/__init__.py