@1aboveio/skills 1.0.0 → 1.0.2

package/skills/app-dev/data-catalog/SKILL.md

@@ -1,7 +1,7 @@
 ---
 name: data-catalog
 description: "Column definitions for warehouse tables, written inside the Spark/PySpark or SQL script that builds each table (CATALOG_TABLE / CATALOG_FIELDS, or a `-- @field` comment block), with a generated markdown catalog in docs/catalog/ and a CI check that fails a PR whose catalog is stale. Use when writing or changing a table script that adds, renames, removes, or changes the meaning of an output column or feature; when asked what a warehouse column or feature means; to set a repo up ('init the data catalog', 'generate the data dictionary', '字段字典', 'add catalog CI'); or to audit existing SQL or Spark scripts for undocumented columns. NOT for implementing or tuning the Spark job itself (pyspark), DAG wiring (airflow), or general workflow YAML (github-actions)."
-release-version: 0.36.0
+release-version: 0.39.0
 ---
 **Updates:** Once/session run `node <skill-base-dir>/release-check.mjs`.
 # Data catalog
@@ -42,6 +42,7 @@ Never document tables during setup unless the task asks for it.
 | CI says the catalog is stale | `python3 <engine> generate`, commit the pages |
 | Before committing a table-script change, or reviewing one | `python3 <engine> drift` (`--base origin/main` for a whole branch) |
 | `Field`/`Table` arguments, the SQL block, config keys, exit codes | [references/format.md](references/format.md) |
+| Does the table's data match its definitions? | the data-quality skill |
 
 `<engine>` is `scripts/data-catalog/data_catalog.py` in the repo, or
 `<skill-base-dir>/scripts/data_catalog.py` before the repo has a copy. When
@@ -68,7 +69,8 @@ Never document tables during setup unless the task asks for it.
    Example: adding `refund_cnt_30d` to the SELECT means adding its `Field` in the
    same commit, then running `generate`.
 2. **Each row says only what is particular to its column, in business words.**
-   Shared filters, window anchor and currency go once in the table notes; the row
+   Shared filters, window anchor and currency go once in the table notes, in every
+   language; the row
    gives what one value measures, its unit, and 0/null meaning; `source` gives
    lineage in one line. Follow the eleven principles in
    [references/field-definitions.md](references/field-definitions.md).
@@ -178,16 +180,18 @@ script is clean when its columns were `unknown`.
 
 ## Backfill one table
 
-1. Pick the script that builds the table from the audit. Its backfill, migrate, and
-   isolated twins share the definitions and need none of their own.
+1. Pick the script that builds the table from the audit. Backfill, migration, and
+   copy scripts that write the same table share its definitions and need none of
+   their own.
 2. Read the SQL or DataFrame code for every output column. Reuse an existing
    hand-written dictionary as a draft, and check every row against the code.
 3. Add `CATALOG_TABLE` and `CATALOG_FIELDS` in contract order. Keep the literal
    contract while you write.
 4. Run `generate` until it passes, then derive the contract and run the repo's
    schema tests (rule 5).
-5. Rerun the audit for that script, and fix `relative-definition` and
-   `thin-definition` findings.
+5. Rerun the audit for that script and fix its writing findings (`repeated-text`,
+   `code-in-description`, `long-definition`, `thin-definition`,
+   `relative-definition`), or leave one only when the text really belongs in that row.
 6. Commit the script and pages together. Replace the old hand-written dictionary
    with a link to the generated page.
 

package/skills/app-dev/data-catalog/references/data-catalog-conventions.md

@@ -7,7 +7,8 @@ table. `docs/catalog/` is generated from those definitions; never edit it by han
 
 **Where definitions live**
 
-- Python: `CATALOG_TABLE = Table("schema.table", grain=..., <language>=...)` and
+- Python: `CATALOG_TABLE = Table("schema.table", <language>=..., grain_<language>=...,
+  notes_<language>=(...))` (grain and notes in every language) and
   `CATALOG_FIELDS = (Field("column", <language>=..., source=..., window=..., unit=...), ...)`,
   imported `from catalog_fields import Field, Table`. Use plain string literals.
 - SQL: a comment block above the first statement: `-- @table schema.table`, then one
@@ -30,7 +31,7 @@ table. `docs/catalog/` is generated from those definitions; never edit it by han
 
 1. Say each fact once, at the widest level where it is true. Filters, the window
    anchor, currency, time zone, terms many columns use ("personal", "platform") and
-   list cleaning rules go in the table's `grain` or `notes`; a row says only what
+   list cleaning rules go in the table's grain or notes; a row says only what
    differs for that column.
 2. The description is meaning in business words: what one value measures, its unit
    and form (a ratio says 0–1 or percent; a time says date or timestamp and its
@@ -51,7 +52,8 @@ table. `docs/catalog/` is generated from those definitions; never edit it by han
    collections.", not "0 when none".
 9. Never restate the column name or write "same as above".
 10. Cover every case: what the column holds for each kind of row, including when it
-    is empty; list an enumerated column's values; say so if it is always empty today.
+    is empty; list an enumerated column's values in the text and in `values=(...)`;
+    say so if it is always empty today.
 11. Write related columns as a set: a count and its amount, a value and its missing
     flag, a legacy column and its replacement use the same scope and wording, and any
     difference is stated in both rows.

package/skills/app-dev/data-catalog/references/field-definitions.md

@@ -8,7 +8,7 @@ column that says only what is particular to that column.
 
 1. **Say each fact once, at the widest level where it is true.** A filter, window
    anchor, currency rule, or row-inclusion rule that holds for every column goes in
-   the table's `grain` or `notes`, never in each row. So does the meaning of a term
+   the table's grain or notes, never in each row. So does the meaning of a term
    many columns use ("personal", "platform", "own name") and how lists are cleaned
    (case, sorting, empty items). A row carries only what differs for that column.
    Bad (every row): `"Posted collections only, test customers excluded, [dt-7d, dt), excluding dt; ..."`
@@ -50,7 +50,8 @@ column that says only what is particular to that column.
    `"同上，30 天窗口"` tell the reader nothing.
 10. **Cover every case.** Say what the column holds for every kind of row (each
     transaction type, each branch of the code), including when it is empty. List
-    the values an enumerated column can take, with a Chinese gloss for English codes.
+    the values an enumerated column can take, with a Chinese gloss for English codes,
+    and put the same list in `values=(...)` so the data can be checked against it.
     A column that is always empty or constant today says so.
     Bad: `"换汇交易为 COMPLETED，内部转账为空。"` (collections and payouts unsaid)
     Good: `"换汇交易为 COMPLETED；收款、付款和内部转账为空。"`
@@ -66,10 +67,10 @@ column that says only what is particular to that column.
 
 | Fact | Goes in |
 |---|---|
-| What one row is, which rows exist | `Table.grain` |
+| What one row is, which rows exist | `Table` `grain_<lang>` |
 | What the table is for, its main source | `Table` language text |
-| Filters, window anchor, currency, time zone shared by all columns | `Table.notes` |
-| Meaning of a term many columns use, list cleaning rules | `Table.notes` |
+| Filters, window anchor, currency, time zone shared by all columns | `Table` `notes_<lang>` |
+| Meaning of a term many columns use, list cleaning rules | `Table` `notes_<lang>` |
 | What this column measures, its unit in words, 0/null meaning | `Field` language text |
 | Upstream column, aggregation, filters only this column adds | `Field.source` |
 | Exact interval / unit code | `Field.window` / `Field.unit` |
@@ -81,15 +82,22 @@ Table:
 ```python
 CATALOG_TABLE = Table(
     "dwd.customer_daily",
-    grain="One row per (dt, cust_id) for each customer with a posted collection before dt.",
     en="Daily customer collection metrics for risk features.",
     zh="客户每日收款指标，供风险特征使用。",
-    notes=(
+    grain_en="One row per (dt, cust_id) for each customer with a posted collection before dt.",
+    grain_zh="每个 (dt, cust_id) 一行；只有 dt 之前有已入账收款的客户才有行。",
+    notes_en=(
         "All metrics count posted collections (status 'POSTED') from non-test customers.",
         "Windows end the day before dt: 'last 7 days' is [dt-7d, dt).",
         "Amounts are converted to USD at the transaction date's rate; collections with no rate are left out of every metric.",
         "Every metric is 0, not empty, if the customer had no collections in its window.",
     ),
+    notes_zh=(
+        "所有指标只计非测试客户的已入账（status 为 'POSTED'）收款。",
+        "窗口截至 dt 前一天：「近 7 天」指 [dt-7d, dt)，含 dt-7d，不含 dt。",
+        "金额按交易日汇率换算为美元；若某笔收款当天没有汇率，则所有指标都不计这笔收款。",
+        "若客户在窗口内没有收款，则指标为 0，而不是空值。",
+    ),
 )
 ```
 

package/skills/app-dev/data-catalog/references/format.md

@@ -7,13 +7,18 @@ from catalog_fields import Field, Table
 
 CATALOG_TABLE = Table(
     "dwd.customer_daily",
-    grain="One row per (dt, cust_id) for each customer with a posted transaction that day.",
     en="Daily customer collection rollup used by risk features.",
     zh="客户每日收款汇总，供风险特征使用。",
-    notes=(
+    grain_en="One row per (dt, cust_id) for each customer with a posted transaction that day.",
+    grain_zh="每个 (dt, cust_id) 一行；只有当天有已入账交易的客户才有行。",
+    notes_en=(
         "All metrics count posted collections (status 'POSTED').",
         "Amounts are converted to USD at the transaction date's rate.",
     ),
+    notes_zh=(
+        "所有指标只计已入账（status 为 'POSTED'）的收款。",
+        "金额按交易日汇率换算为美元。",
+    ),
 )
 
 KEYS = (
@@ -36,8 +41,13 @@ CATALOG_FIELDS = (
 PUBLISHED_COLUMNS = tuple(f.name for f in CATALOG_FIELDS)
 ```
 
-- `Field(name, <one keyword per language>, source, window=None, unit=None)`.
-- `Table(name, grain, <one keyword per language>, notes=())`.
+- `Field(name, <one keyword per language>, source, window=None, unit=None, values=())`.
+  `values` lists the values an enumerated column can take, as distinct strings
+  (`values=("是", "否", "无法判定")`); the data-quality skill checks the data against it.
+  Never use it on identifiers or free text.
+- `Table(name, <one keyword per language>, grain_<lang>=..., notes_<lang>=(...))`: a
+  grain for every language, and the same number of notes in every language (notes are
+  optional).
 - Pass only the name positionally; everything else by keyword.
 - The language keywords come from `languages` in `data-catalog.json`. `init`
   regenerates `catalog_fields.py` with exactly those, so a missing language is a
@@ -49,10 +59,12 @@ PUBLISHED_COLUMNS = tuple(f.name for f in CATALOG_FIELDS)
 
 ```sql
 -- @table ads.merchant_monthly
---   grain: One row per (month, merchant_id).
+--   grain_en: One row per (month, merchant_id).
+--   grain_zh: 每个 (month, merchant_id) 一行。
 --   en: Monthly settled volume per merchant.
 --   zh: 商户月度结算金额。
---   notes: Built on the 1st of each month; Counts settled rows (status 'SETTLED') only; Excludes test merchants
+--   notes_en: Built on the 1st of each month; Counts settled rows (status 'SETTLED') only; Excludes test merchants
+--   notes_zh: 每月 1 日构建；只计已结算（status 为 'SETTLED'）的记录；不含测试商户
 -- @field merchant_id
 --   en: Merchant that settled the volume; key of dim_merchant.
 --   zh: 结算该金额的商户，dim_merchant 主键。
@@ -68,9 +80,10 @@ SELECT ...
 ```
 
 - A directive line is `-- @table <name>` or `-- @field <name>`.
-- A key line is exactly two spaces after `-- `, then `key: value`.
+- A key line is exactly two spaces after `-- `, then `key: value`. `values:` takes a
+  list split on `;` or `；`.
 - A continuation line has four or more spaces and joins the previous key with a space.
-- `notes` on `@table` splits on `;`.
+- `notes_<lang>` on `@table` splits on `;` or `；`.
 - The block ends at the first non-comment line. One `@table` per file.
 
 ## data-catalog.json

package/skills/app-dev/data-catalog/scripts/data_catalog.py

@@ -49,7 +49,7 @@ DEFAULT_CONFIG = {
     "tableNames": [],
     "schemaNames": [],
 }
-FIELD_EXTRAS = ("source", "window", "unit")
+FIELD_EXTRAS = ("source", "window", "unit", "values")
 LANGUAGE_LABELS = {"en": "English", "zh": "中文", "zh_hant": "繁體中文", "ja": "日本語", "ko": "한국어"}
 LANGUAGE_RE = re.compile(r"^[a-z]{2}(_[a-z]+)?$")
 SKIP_DIRS = {"node_modules", "__pycache__", "venv"}
@@ -151,15 +151,16 @@ class FieldDoc:
     window: str | None
     unit: str | None
     line: int
+    values: tuple = ()   # the values an enumerated column can take; checked by data-quality
 
 
 @dataclass
 class TableDoc:
     name: str
     script: str
-    grain: str
+    grain: dict   # language -> what one row is
     text: dict
-    notes: list
+    notes: dict   # language -> [shared rules]
     fields: list
     line: int
     contracts: dict = field(default_factory=dict)  # name -> list[str] | "derived" | "unchecked"
@@ -258,26 +259,66 @@ def build_field(values: dict, languages: list, where: str, line: int) -> FieldDo
                            f"languages configured: {languages}")
     text = {lang: require_text(values, lang, where, line, f"Field {name!r}") for lang in languages}
     source = require_text(values, "source", where, line, f"Field {name!r}")
-    return FieldDoc(name, text, source, (values.get("window") or None), (values.get("unit") or None), line)
+    allowed = values.get("values") or ()
+    if isinstance(allowed, str):
+        allowed = tuple(v.strip() for v in re.split(r"[;；]", allowed))
+    if not isinstance(allowed, tuple) or not all(isinstance(v, str) and v != "" for v in allowed):
+        raise CatalogError(f"{where}:{line}: Field {name!r}: values must be a tuple of non-empty strings")
+    if len(set(allowed)) != len(allowed):
+        raise CatalogError(f"{where}:{line}: Field {name!r}: values lists a value twice")
+    return FieldDoc(name, text, source, (values.get("window") or None), (values.get("unit") or None), line, allowed)
 
 
 def build_table(values: dict, fields: list, languages: list, where: str, line: int, script: str) -> TableDoc:
     name = require_text(values, "name", where, line, "Table")
-    known = {"name", "grain", "notes", *languages}
+    if "grain" in values or "notes" in values:
+        keys = ", ".join(f"grain_{lang}=..., notes_{lang}=(...)" for lang in languages)
+        raise CatalogError(f"{where}:{line}: Table {name!r}: grain and notes are written per language: {keys}")
+    known = {"name", *languages, *(f"grain_{lang}" for lang in languages), *(f"notes_{lang}" for lang in languages)}
     unknown = sorted(set(values) - known)
     if unknown:
         raise CatalogError(f"{where}:{line}: Table {name!r} has unknown argument(s) {unknown}")
-    grain = require_text(values, "grain", where, line, f"Table {name!r}")
     text = {lang: require_text(values, lang, where, line, f"Table {name!r}") for lang in languages}
-    notes = values.get("notes") or ()
-    if isinstance(notes, str):
-        notes = (notes,)
-    return TableDoc(name, script, grain, text, [n.strip() for n in notes if n and n.strip()], fields, line)
+    grain = {lang: require_text(values, f"grain_{lang}", where, line, f"Table {name!r}") for lang in languages}
+    notes = {}
+    for lang in languages:
+        raw = values.get(f"notes_{lang}") or ()
+        raw = (raw,) if isinstance(raw, str) else raw
+        notes[lang] = [n.strip() for n in raw if n and n.strip()]
+    counts = {lang: len(n) for lang, n in notes.items()}
+    if len(set(counts.values())) > 1:
+        raise CatalogError(f"{where}:{line}: Table {name!r}: every language needs the same notes, got {counts}")
+    return TableDoc(name, script, grain, text, notes, fields, line)
+
+
+def struct_fields(node: ast.AST, names: dict):
+    """Column names of a literal StructType([StructField("name", ...), ...]), or None."""
+    if call_name(node) != "StructType" or not node.args:
+        return None
+    try:
+        items = resolve_sequence(node.args[0], names, "", frozenset())
+    except CatalogError:
+        return None
+    out = []
+    for item in items:
+        if call_name(item) != "StructField" or not item.args:
+            return None
+        try:
+            value = eval_text(item.args[0], names, "")
+        except CatalogError:
+            return None
+        if not isinstance(value, str):
+            return None
+        out.append(value)
+    return out
 
 
 def contract_state(node: ast.AST, names: dict):
     if any(isinstance(n, ast.Name) and n.id == FIELDS_VAR for n in ast.walk(node)):
         return "derived"
+    columns = struct_fields(node, names)
+    if columns is not None:
+        return columns
     try:
         items = resolve_sequence(node, names, "", frozenset())
         values = [eval_text(item, names, "") for item in items]
@@ -363,14 +404,12 @@ def parse_sql(text: str, where: str, languages: list) -> TableDoc | None:
     fields = []
     for kind, name, values, line, _ in entries:
         if kind == "field":
-            if "notes" in values:
-                raise CatalogError(f"{where}:{line}: `notes` belongs on @table, not @field")
+            if any(k.startswith(("notes", "grain")) for k in values):
+                raise CatalogError(f"{where}:{line}: `grain_*` and `notes_*` belong on @table, not @field")
             fields.append(build_field({"name": name, **values}, languages, where, line))
     _, name, values, line, _ = tables[0]
-    notes = values.pop("notes", None)
-    values = {"name": name, **values}
-    if notes:
-        values["notes"] = tuple(n.strip() for n in notes.split(";"))
+    values = {"name": name, **{k: tuple(n.strip() for n in re.split(r"[;；]", v)) if k.startswith("notes_") else v
+                               for k, v in values.items()}}
     return build_table(values, fields, languages, where, line, where)
 
 
@@ -467,6 +506,9 @@ def cell(text: str | None) -> str:
     return text.replace("\\", "\\\\").replace("|", "\\|").replace("\r\n", "\n").replace("\n", "<br>")
 
 
+SECTION_LABELS = {"en": ("Grain:", "Notes:"), "zh": ("粒度：", "说明："), "zh_hant": ("粒度：", "說明：")}
+
+
 def label(lang: str) -> str:
     return LANGUAGE_LABELS.get(lang, lang)
 
@@ -496,28 +538,33 @@ def render_table(table: TableDoc, root: Path, config: dict) -> str:
     langs = config["languages"]
     out = [GENERATED_MARK, "", f"# `{table.name}`", ""]
     out.append(f"- **Script:** [{table.script}]({link_to(root, config['output'], table.script)})")
-    out.append(f"- **Grain:** {cell(table.grain)}")
     out.append(f"- **Fields:** {len(table.fields)}")
     out.append("")
     for lang in langs:
-        out.append(f"**{label(lang)}:** {cell(table.text[lang])}" if len(langs) > 1 else cell(table.text[lang]))
-        out.append("")
-    if table.notes:
-        out.append("**Notes:**")
-        out.append("")
-        out.extend(f"- {cell(note)}" for note in table.notes)
+        grain_label, notes_label = SECTION_LABELS.get(lang, SECTION_LABELS["en"])
+        if len(langs) > 1:
+            out += [f"## {label(lang)}", ""]
+        out += [cell(table.text[lang]), "", f"- **{grain_label}** {cell(table.grain[lang])}"]
+        if table.notes[lang]:
+            out.append(f"- **{notes_label}**")
+            out.extend(f"  - {cell(note)}" for note in table.notes[lang])
         out.append("")
+    if len(langs) > 1:
+        out += ["## Fields", ""]
     has_window = any(f.window for f in table.fields)
     has_unit = any(f.unit for f in table.fields)
+    has_values = any(f.values for f in table.fields)
     head = ["Field", *(label(lang) for lang in langs), "Source"]
     head += ["Window"] if has_window else []
     head += ["Unit"] if has_unit else []
+    head += ["Values"] if has_values else []
     out.append("| " + " | ".join(head) + " |")
     out.append("|" + "---|" * len(head))
     for f in table.fields:
         row = [f"`{f.name}`", *(cell(f.text[lang]) for lang in langs), cell(f.source)]
         row += [cell(f.window)] if has_window else []
         row += [cell(f.unit)] if has_unit else []
+        row += [cell(", ".join(f"`{v}`" for v in f.values))] if has_values else []
         out.append("| " + " | ".join(row) + " |")
     return "\n".join(out) + "\n"
 
@@ -539,7 +586,7 @@ def render_index(tables: list, root: Path, config: dict) -> str:
     out += ["| Table | Summary | Grain | Fields | Script |", "|---|---|---|---|---|"]
     for t in tables:
         script = f"[{t.script}]({link_to(root, config['output'], t.script)})"
-        out.append(f"| [`{t.name}`]({page_name(t)}) | {cell(t.text[lang])} | {cell(t.grain)} | {len(t.fields)} | {script} |")
+        out.append(f"| [`{t.name}`]({page_name(t)}) | {cell(t.text[lang])} | {cell(t.grain[lang])} | {len(t.fields)} | {script} |")
     return "\n".join(out) + "\n"
 
 
@@ -1019,11 +1066,13 @@ def preview(items: list, limit: int = 12) -> str:
     return shown + (f", … (+{len(items) - limit})" if len(items) > limit else "")
 
 
-def cmd_audit(root: Path, globs: list, as_json: bool, verbose: bool, table_names: list, schema_names: list) -> int:
+def cmd_audit(root: Path, globs: list, as_json: bool, verbose: bool, table_names: list, schema_names: list,
+              contracts: list) -> int:
     config_path = root / CONFIG_FILE
     config = load_config(root) if config_path.exists() else dict(DEFAULT_CONFIG)
     config = {**config, "tableNames": [*config["tableNames"], *table_names],
-              "schemaNames": [*config["schemaNames"], *schema_names]}
+              "schemaNames": [*config["schemaNames"], *schema_names],
+              "contracts": [*config["contracts"], *contracts]}
     if globs:
         patterns, exclude = globs, config.get("exclude", [])
     elif config_path.exists():
@@ -1137,8 +1186,9 @@ def code_without_metadata(text: str, suffix: str) -> str:
 def definitions(table: TableDoc | None) -> tuple[dict, tuple]:
     if table is None:
         return {}, ()
-    fields = {f.name: (tuple(sorted(f.text.items())), f.source, f.window, f.unit) for f in table.fields}
-    return fields, (table.grain, tuple(sorted(table.text.items())), tuple(table.notes))
+    fields = {f.name: (tuple(sorted(f.text.items())), f.source, f.window, f.unit, f.values) for f in table.fields}
+    notes = tuple((lang, tuple(n)) for lang, n in sorted(table.notes.items()))
+    return fields, (tuple(sorted(table.grain.items())), tuple(sorted(table.text.items())), notes)
 
 
 def drift_items(root: Path, config: dict, base: str) -> list:
@@ -1212,12 +1262,14 @@ def cmd_drift(root: Path, base: str) -> int:
 
 def render_fields_module(languages: list) -> str:
     lang_fields = "\n".join(f"    {lang}: str" for lang in languages)
+    grain_fields = "\n".join(f"    grain_{lang}: str" for lang in languages)
+    notes_fields = "\n".join(f"    notes_{lang}: tuple[str, ...] = ()" for lang in languages)
     return f'''"""Column metadata types for the data catalog.
 
 Generated by data_catalog.py init from data-catalog.json; rerun init after changing
 `languages` there, never edit this file. Table scripts declare:
 
-    CATALOG_TABLE = Table("schema.table", grain=..., {", ".join(f"{lang}=..." for lang in languages)})
+    CATALOG_TABLE = Table("schema.table", {", ".join(f"{lang}=..., grain_{lang}=..." for lang in languages)})
     {FIELDS_VAR} = (Field("column", {", ".join(f"{lang}=..." for lang in languages)}, source=...), ...)
     PUBLISHED_COLUMNS = tuple(f.name for f in {FIELDS_VAR})
 """
@@ -1233,14 +1285,15 @@ class Field:
     source: str
     window: str | None = None
     unit: str | None = None
+    values: tuple[str, ...] = ()
 
 
 @dataclass(frozen=True)
 class Table:
     name: str
-    grain: str
 {lang_fields}
-    notes: tuple[str, ...] = ()
+{grain_fields}
+{notes_fields}
 '''
 
 
@@ -1367,6 +1420,8 @@ def main(argv: list | None = None) -> int:
                          help="extra variable/attribute whose string value is the table written; repeatable")
     p_audit.add_argument("--schema-name", action="append", default=[], metavar="NAME",
                          help="extra variable/attribute holding that table's schema; repeatable")
+    p_audit.add_argument("--contract", action="append", default=[], metavar="NAME",
+                         help="extra column-contract name (tuple of names or StructType); repeatable")
     p_drift = sub.add_parser("drift", help="local check: code changed since a git base while its definitions did not")
     p_drift.add_argument("--base", default="HEAD", help="git ref to compare with (default HEAD: uncommitted work)")
     sub.add_parser("version", help="print the engine identity")
@@ -1382,7 +1437,7 @@ def main(argv: list | None = None) -> int:
             return cmd_generate(root, args.check)
         if args.command == "drift":
             return cmd_drift(root, args.base)
-        return cmd_audit(root, args.globs, args.json, args.verbose, args.table_name, args.schema_name)
+        return cmd_audit(root, args.globs, args.json, args.verbose, args.table_name, args.schema_name, args.contract)
     except CatalogError as exc:
         print(f"data-catalog: {exc}", file=sys.stderr)
         return 2

package/skills/app-dev/data-catalog/scripts/test_data_catalog.py

@@ -25,10 +25,12 @@ WINDOW_NOTE = "[dt-7d, dt)"
 
 CATALOG_TABLE = Table(
     "dwd.customer_daily",
-    grain="one row per (dt, cust_id)",
     en="Daily customer collection rollup.",
     zh="客户每日收款汇总。",
-    notes=("Amounts in USD via dim_forex.",),
+    grain_en="one row per (dt, cust_id)",
+    grain_zh="每个 (dt, cust_id) 一行",
+    notes_en=("Amounts in USD via dim_forex.",),
+    notes_zh=("金额经 dim_forex 换算为 USD。",),
 )
 KEYS = (
     Field("dt", en="Snapshot date.", zh="快照日期。", source="Run parameter `dt`."),
@@ -50,10 +52,12 @@ PUBLISHED_COLUMNS = tuple(f.name for f in CATALOG_FIELDS)
 
 SQL_SCRIPT = """\
 -- @table ads.merchant_monthly
---   grain: one row per (month, merchant_id)
 --   en: Monthly merchant volume.
 --   zh: 商户月度交易量。
---   notes: Built monthly; Excludes test merchants
+--   grain_en: one row per (month, merchant_id)
+--   grain_zh: 每个 (month, merchant_id) 一行
+--   notes_en: Built monthly; Excludes test merchants
+--   notes_zh: 每月构建；不含测试商户
 -- @field merchant_id
 --   en: Merchant id.
 --   zh: 商户 ID。
@@ -111,12 +115,30 @@ class GenerateTest(unittest.TestCase):
         self.assertIn("| `volume_usd` | Settled volume in the month. | 当月结算金额。 | "
                       "dwd_settlement settled rows, summed amount_usd | USD |", sql_page)
         self.assertNotIn("Window", sql_page)
-        self.assertIn("- Excludes test merchants", sql_page)
+        self.assertIn("  - Excludes test merchants", sql_page)
+        self.assertIn("## 中文", sql_page)
+        self.assertIn("- **粒度：** 每个 (month, merchant_id) 一行", sql_page)
+        self.assertIn("  - 不含测试商户", sql_page)
         index = (root / "docs/catalog/README.md").read_text(encoding="utf-8")
         self.assertLess(index.index("ads.merchant_monthly"), index.index("dwd.customer_daily"))
         self.assertIn("| [`dwd.customer_daily`](dwd.customer_daily.md) | Daily customer collection rollup. | "
                       "one row per (dt, cust_id) | 2 |", index)
 
+    def test_values_render_and_must_be_distinct_strings(self):
+        script = PY_SCRIPT.replace('source="Run parameter `dt`."),', 'source="Run parameter `dt`.", values=("a", "b")),')
+        root = make_repo({"jobs/a.py": script}, BILINGUAL)
+        self.assertEqual(run(root, "generate")[0], 0)
+        page = (root / "docs/catalog/dwd.customer_daily.md").read_text(encoding="utf-8")
+        self.assertIn("| Unit | Values |", page)
+        self.assertIn("| `a`, `b` |", page)
+        twice = script.replace('values=("a", "b")', 'values=("a", "a")')
+        root = make_repo({"jobs/a.py": twice}, BILINGUAL)
+        self.assertIn("values lists a value twice", run(root, "generate")[2])
+        sql = SQL_SCRIPT.replace("--   source: dim_merchant.merchant_id", "--   source: dim_merchant.merchant_id\n--   values: m1；m2")
+        root = make_repo({"sql/m.sql": sql}, BILINGUAL)
+        self.assertEqual(run(root, "generate")[0], 0)
+        self.assertIn("`m1`, `m2`", (root / "docs/catalog/ads.merchant_monthly.md").read_text(encoding="utf-8"))
+
     def test_check_passes_when_current_and_fails_after_a_definition_changes(self):
         root = make_repo({"jobs/dwd/customer_daily.py": PY_SCRIPT}, BILINGUAL)
         self.assertEqual(run(root, "generate")[0], 0)
@@ -192,6 +214,18 @@ SELECT dt, SUM(amt) AS coll_amt_7d_usd, COUNT(*) AS coll_cnt_7d FROM dwd.txn GRO
         root = make_repo({"jobs/a.py": imported}, BILINGUAL)
         self.assertIn("imported lists cannot be read", run(root, "generate")[2])
 
+    def test_table_grain_and_notes_are_per_language_with_matching_notes(self):
+        old = PY_SCRIPT.replace('grain_en="one row per (dt, cust_id)",', 'grain="one row per (dt, cust_id)",')
+        root = make_repo({"jobs/a.py": old}, BILINGUAL)
+        code, _, err = run(root, "generate")
+        self.assertEqual(code, 1)
+        self.assertIn("grain and notes are written per language: grain_en=..., notes_en=(...), grain_zh=...", err)
+        uneven = PY_SCRIPT.replace('notes_zh=("金额经 dim_forex 换算为 USD。",),', 'notes_zh=(),')
+        root = make_repo({"jobs/a.py": uneven}, BILINGUAL)
+        code, _, err = run(root, "generate")
+        self.assertEqual(code, 1)
+        self.assertIn("every language needs the same notes, got {'en': 1, 'zh': 0}", err)
+
     def test_same_table_in_two_scripts_is_an_error(self):
         root = make_repo({"jobs/a.py": PY_SCRIPT, "jobs/b.py": PY_SCRIPT}, BILINGUAL)
         code, _, err = run(root, "generate")
@@ -238,6 +272,18 @@ class AuditTest(unittest.TestCase):
                          "writes dwd.undocumented but carries no CATALOG_TABLE/CATALOG_FIELDS (or -- @table block); "
                          "also written by jobs/backfill_b.py (put the metadata in the script that builds the table)")
 
+    def test_a_structtype_contract_is_read_for_its_column_names(self):
+        script = '''
+        L1A_PUBLISHED_SCHEMA = StructType([
+            StructField("dt", DateType(), False),
+            StructField("cust_id", StringType(), False),
+        ])
+        TARGET_TABLE = "dwd.flow"
+        '''
+        root = make_repo({"jobs/flow.py": script})
+        [entry] = json.loads(run(root, "audit", "--json", "--contract", "L1A_PUBLISHED_SCHEMA", "jobs/*.py")[1])["scripts"]
+        self.assertEqual((entry["columns"], entry["columnsFrom"]), (["dt", "cust_id"], "L1A_PUBLISHED_SCHEMA"))
+
     def test_class_attributes_name_the_table_with_their_schema(self):
         script = '''
         class OrdersEtl(Etl):
@@ -311,7 +357,7 @@ class AuditTest(unittest.TestCase):
                 ("c_amt", " ".join(["word"] * 31), "近 7 天收款总额。", "sum of amount"),
             ])
         script = f'''
-CATALOG_TABLE = Table("dwd.t", grain="One row per id.", en="Table.", zh="表。")
+CATALOG_TABLE = Table("dwd.t", en="Table.", zh="表。", grain_en="One row per id.", grain_zh="每个 id 一行。")
 CATALOG_FIELDS = (
 {rows}
 )
@@ -329,8 +375,9 @@ CATALOG_FIELDS = (
 
     def test_shared_vocabulary_and_plain_units_are_not_flagged(self):
         script = '''
-CATALOG_TABLE = Table("dwd.t", grain="One row per id.", en="Table.", zh="表。",
-                      notes=("All metrics count posted collections.", "Every metric is 0 if there were no collections."))
+CATALOG_TABLE = Table("dwd.t", en="Table.", zh="表。", grain_en="One row per id.", grain_zh="每个 id 一行。",
+                      notes_en=("All metrics count posted collections.", "Every metric is 0 if there were no collections."),
+                      notes_zh=("所有指标只计已入账收款。", "若窗口内无收款，则指标为 0。"))
 CATALOG_FIELDS = (
     Field("a", en="Number of collections in the last 7 days.", zh="近 7 天收款笔数。",
           source="count of dwd.transaction rows", window="[dt-7d, dt)"),
@@ -390,9 +437,9 @@ class ReviewFixTest(unittest.TestCase):
         self.assertIn("no column contract and no readable SELECT writing ads.merchant_monthly", err)
 
     def test_tables_mapping_to_one_page_or_the_index_are_errors(self):
-        a = PY_SCRIPT.replace('"dwd.customer_daily",\n    grain', '"ads.a_b",\n    grain')
-        b = PY_SCRIPT.replace('"dwd.customer_daily",\n    grain', '"ads.a b",\n    grain')
-        readme = PY_SCRIPT.replace('"dwd.customer_daily",\n    grain', '"README",\n    grain')
+        a = PY_SCRIPT.replace('"dwd.customer_daily",\n    en=', '"ads.a_b",\n    en=')
+        b = PY_SCRIPT.replace('"dwd.customer_daily",\n    en=', '"ads.a b",\n    en=')
+        readme = PY_SCRIPT.replace('"dwd.customer_daily",\n    en=', '"README",\n    en=')
         root = make_repo({"jobs/a.py": a, "jobs/b.py": b, "jobs/c.py": readme}, BILINGUAL)
         code, _, err = run(root, "generate")
         self.assertEqual(code, 1)

package/skills/app-dev/data-quality/LICENSE

@@ -0,0 +1,3 @@
+MIT License
+
+Copyright (c) 2026 1AboveIO