@thinkingai/ae-cli 6.1.18 → 6.1.20

package/skills/ae-analysis/references/property_create.md

@@ -13,6 +13,8 @@ ae-cli analysis-meta property create --dry-run
 
 Capability id: `metadata.property.create`.
 
+Authorization requires the single project function permission `editSuperMeta` with the `metadata:write` scope. In the zh-CN permission UI, this permission is labeled `元数据管理 > 编辑`; the corresponding English label is `Metadata Management > Edit`. This is the same project permission used by `metadata.super_metadata.batch_create`. If either command returns `PROJECT_PERMISSION_DENIED`, do not retry with a different payload and do not describe the two capability IDs as two separate permissions. Ask a project administrator to grant this shared project permission to the current identity.
+
 Input sends `project_id`, `table_type`, `payload`.
 
 Output is a successful gateway envelope with no business data. Read back with `property get` using the same table type.

package/skills/ae-analysis/references/super_metadata_batch_create.md

@@ -12,6 +12,8 @@ ae-cli analysis-meta super-metadata batch-create --project-id <project_id> --eve
 
 Capability id: `metadata.super_metadata.batch_create`.
 
+Authorization requires the single project function permission `editSuperMeta` with the `metadata:write` scope. In the zh-CN permission UI, this permission is labeled `元数据管理 > 编辑`; the corresponding English label is `Metadata Management > Edit`. This is the same project permission used by `metadata.property.create`. If either command returns `PROJECT_PERMISSION_DENIED`, do not retry with a different payload and do not describe the two capability IDs as two separate permissions. Ask a project administrator to grant this shared project permission to the current identity.
+
 Input sends `project_id` plus any non-empty JSON arrays among `events`, `event_properties`, and `user_properties`. Use snake_case object fields exactly as documented by the common-service schema:
 
 - Event items: `event_name`, optional `event_desc`, optional `remark`, optional `super_event_prop_names`.

package/skills/ae-analysis/references/user_cluster_models.md

@@ -7,6 +7,8 @@ Top-level variants:
 - condition: `{"type":"condition","conditions":{"relation":"and|or","items":[...]}}`
 - SQL: `{"type":"sql","sql":"...","params":[{"name":"partdate","type":"part_date","recent_day":"1-7"}]}`
 
+For `user-cluster update`, `type` must match the existing cluster type. The update command cannot change a cluster between condition and SQL, and it cannot change the cluster's analysis entity.
+
 For SQL clusters, quote Trino special identifiers with double quotes, for example `{"type":"sql","sql":"SELECT \"#user_id\" FROM v_user_1 WHERE vip_level >= 3"}`. This applies to identifiers containing `#`, `$`, `@`, spaces, or punctuation; single quotes are string literals.
 
 If a SQL cluster reads an event table, include a predicate on the quoted `"$part_date"` date-partition column; the backend rejects event-table SQL without it. This does not apply to the user-table example above.

package/skills/ae-analysis/references/user_cluster_update.md

@@ -2,18 +2,22 @@
 
 Update a condition or SQL user cluster. Discover the exact `cluster_name` first.
 
-Do not use it for ID-file membership replacement or to create a missing cluster. Supplying `--definition-request` automatically starts recomputation after the definition is updated; do not call `user-cluster refresh` afterward. Updating only `--display-name` or `--remark` does not recompute. `--auto-refresh-cron` changes an existing enabled auto-refresh schedule and does not enable auto refresh. A successful update means the definition was saved, not that the new result is complete. Poll `user-cluster get` until `progress=100` and `refresh_end_time` is not older than `update_time` before using `users_num` or querying members.
+This is a `high-risk-write`. First run the final command with `--dry-run`, summarize the exact cluster and fields that will change, and wait for explicit user confirmation. Then run the unchanged command with `--yes`. Do not use it for ID-file membership replacement, to create a missing cluster, to change its analysis entity, or to change it between condition and SQL types.
+
+Supplying `--definition-request` automatically starts recomputation after the definition is updated; do not call `user-cluster refresh` afterward. Updating only `--display-name` or `--remark` does not recompute. `--auto-refresh-cron` changes an existing enabled auto-refresh schedule and does not enable auto refresh. A successful update means the definition was saved, not that the new result is complete. Poll `user-cluster get` until `progress=100` and `refresh_end_time` is not older than `update_time` before using `users_num` or querying members.
 
 The response distinguishes both paths. A definition update returns `computation.triggered_automatically=true`, `result_freshness.is_stale=true`, and normally `next_action=poll_get` with an exact capability/input pair. A display-name/remark-only update returns `computation.status=not_triggered`, `result_freshness.status=fresh`, and `next_action=none`.
 
-Flags: `--project-id`, `--cluster-name` required. Optional: `--display-name`, `--definition-request`, `--authenticated-only`, `--remark`, `--zone-offset`, `--auto-refresh-cron`. The cluster type comes from `definition_request.type` when the definition changes.
+Flags: `--project-id`, `--cluster-name` required. Optional: `--display-name`, `--definition-request`, `--authenticated-only`, `--remark`, `--zone-offset`, `--auto-refresh-cron`. There is no `--entity-id` update flag. When the definition changes, `definition_request.type` must match the existing cluster type; it selects the definition variant but does not change the saved type.
 
 `display_name` is at most 80 characters and `remark` is at most 400 characters. The CLI rejects violations before dispatch. `cluster_name` is an existing exact identifier and cannot be renamed by update.
 
 Read `user_cluster_models.md` before changing the definition. The backend validates and compiles `definition_request` inside update and refuses to modify the cluster if clarification is required. Condition definitions are saved as mixed-condition clusters.
 
 ```bash
-ae-cli analysis user-cluster update --project-id <project_id> --cluster-name retained_users --display-name "Retained Users v2"
+ae-cli analysis user-cluster update --project-id <project_id> --cluster-name retained_users --display-name "Retained Users v2" --dry-run
+
+ae-cli analysis user-cluster update --project-id <project_id> --cluster-name retained_users --display-name "Retained Users v2" --yes
 
-ae-cli analysis user-cluster update --project-id <project_id> --cluster-name retained_users --auto-refresh-cron '0 30 2 * * ? *'
+ae-cli analysis user-cluster update --project-id <project_id> --cluster-name retained_users --auto-refresh-cron '0 30 2 * * ? *' --dry-run
 ```

package/skills/ae-analysis/references/user_tag_create.md

@@ -6,14 +6,42 @@ Do not use it for uploaded-ID tags; use `user-tag create-id`. A successful creat
 
 The response reports this directly: `computation.triggered_automatically=true`, `computation.status=submitted`, and `result_freshness.is_stale=true`. Follow `next_action`; when it is `poll_get`, invoke `next_capability_id` with the exact `next_input` returned by the command.
 
-Flags: `--project-id`, `--tag-name`, `--display-name`, `--definition-request` required. Optional: `--authenticated-only`, `--zone-offset`, `--entity-id`. The tag type comes from `definition_request.type`.
+Flags: `--project-id`, `--tag-name`, `--display-name`, `--definition-request` required. Optional: `--authenticated-only`, `--zone-offset`, `--entity-id`, `--enable-auto-refresh`, `--auto-refresh-schedule`, `--auto-refresh-cron`. The tag type comes from `definition_request.type`.
 
 `tag_name` is a machine identifier: 1-80 characters, starts with a letter, and contains only letters, digits, or underscores. `display_name` is 1-80 characters. The CLI rejects violations before dispatch.
 
-Read `user_tag_models.md` before constructing `--definition-request`. Create does not accept `--remark`; set it later with `user-tag update` when needed.
+Read `user_tag_models.md` before constructing `--definition-request`. Dynamic first/last ranges use semantic `time_range` values such as `{"mode":"recent","unit":"month","value":1}` for this month or `{"mode":"start_to_today","start_time":"2026-07-01"}` for a fixed start date through today. Create does not accept `--remark`; set it later with `user-tag update` when needed.
 
 The backend validates and compiles the definition inside the create operation; if metadata is ambiguous or missing, creation fails without creating the tag.
 
 ```bash
 ae-cli analysis user-tag create --project-id <project_id> --tag-name high_value --display-name "High Value" --definition-request '{"type":"condition","condition_values":[{"value":"high","events":[{"event":"pay","operator":"gte","value":3,"aggregation":"count","time_range":{"mode":"recent","unit":"day","value":30}}]}]}'
 ```
+
+First/last tag for this month:
+
+```bash
+ae-cli analysis user-tag create --project-id <project_id> --tag-name latest_platform_this_month --display-name "Latest Platform This Month" --definition-request '{"type":"first_last","first_last":{"event":"login","occurrence":"last","property":"platform","time_range":{"mode":"recent","unit":"month","value":1}}}'
+```
+
+Periodic refresh can be configured in this same create operation. It is separate from the initial computation. Omitted scheduling flags leave periodic refresh disabled; supplying a schedule enables it. `--enable-auto-refresh true` requires a schedule on create. Do not combine `--enable-auto-refresh false` with a schedule, or pass both schedule forms.
+
+Append one of these options to the create command:
+
+```bash
+# Every day at 02:30 in the tag timezone
+--auto-refresh-schedule '{"frequency":"daily","time":"02:30"}'
+
+# Monday and Sunday at 09:00 (ISO weekdays: 1=Monday, 7=Sunday)
+--auto-refresh-schedule '{"frequency":"weekly","time":"09:00","weekdays":[1,7]}'
+
+# The 1st and 15th of each month at 06:00
+--auto-refresh-schedule '{"frequency":"monthly","time":"06:00","month_days":[1,15]}'
+
+# Custom Quartz schedule, including multiple executions per day
+--auto-refresh-cron '0 0/30 8-18 * * ? *'
+```
+
+`time` uses 24-hour `HH:mm`. `month_days` accepts 1-31; dates absent from a month are skipped. Weekly/monthly schedules require their respective day array; other frequencies reject those fields. The structured form preserves the page's daily/weekly/monthly frequency selection. Cron uses the page's custom schedule mode and preserves every cron field.
+
+Schedules use the tag timezone. Use the existing `--zone-offset` only with an offset supported by the project; omit it to use the project's default behavior. Inspect `user-tag get` for `enable_auto_refresh` (1=enabled, 0=disabled), `scheduler_ui_config`, and `cluster_zone_offset` after creation.

package/skills/ae-analysis/references/user_tag_models.md

@@ -24,18 +24,32 @@ Top-level `type` is exactly one of `condition`, `metric`, `first_last`, or `sql`
 
 ## Metric tag
 
-Required: `event`, `aggregation`. `property`, `time_range`, and `filters` are optional. Filters support only event properties and user properties. A string `field` is an event property; use `{name,type:"user_property"}` for a user property.
+Required: `event`, `aggregation`. `property`, `percentile`, `time_range`, and `filters` are optional. Filters support only event properties and user properties. A string `field` is an event property; use `{name,type:"user_property"}` for a user property.
+
+Use `aggregation=percentile` with a numeric event `property` and pass `percentile`. Supported percentile values match the page controls: `5`, `10`, `20`, `25`, `30`, `40`, `60`, `70`, `75`, `80`, `90`, `95`, and `99`. The `percentile` field is required for percentile aggregation and is rejected for every other aggregation.
 
 ```json
 {"type":"metric","metric":{"event":"pay","aggregation":"sum","property":"amount","time_range":{"mode":"previous","unit":"day","value":30},"filters":{"relation":"and","items":[{"field":"channel","operator":"eq","values":["app"]},{"field":{"name":"country","type":"user_property"},"operator":"eq","values":["US"]}]}}}
 ```
 
+Percentile example:
+
+```json
+{"type":"metric","metric":{"event":"pay","aggregation":"percentile","property":"amount","percentile":90}}
+```
+
 ## First/last tag
 
-Required: `event`, `occurrence=first|last`, and exactly one value source: `calculation` or `property`. `time_range` and `filters` are optional. Filters support only event properties and user properties. A string `field` is an event property; use `{name,type:"user_property"}` for a user property. Supplying neither or both value sources is rejected before execution.
+Required: `event`, `occurrence=first|last`, and exactly one value source: `calculation` or `property`. `time_range` and `filters` are optional. Filters support only event properties and user properties. A string `field` is an event property; use `{name,type:"user_property"}` for a user property. Supplying neither or both value sources is rejected before execution. Use the semantic time mappings in [`audience_models.md`](audience_models.md) for dynamic ranges such as today, this month, or a fixed start date through today.
+
+```json
+{"type":"first_last","first_last":{"event":"login","occurrence":"last","property":"platform","time_range":{"mode":"recent","unit":"month","value":1},"filters":{"relation":"and","items":[{"field":{"name":"country","type":"user_property"},"operator":"eq","values":["US"]}]}}}
+```
+
+From a fixed date through today:
 
 ```json
-{"type":"first_last","first_last":{"event":"login","occurrence":"last","property":"platform","time_range":{"mode":"recent","unit":"day","value":30},"filters":{"relation":"and","items":[{"field":{"name":"country","type":"user_property"},"operator":"eq","values":["US"]}]}}}
+{"type":"first_last","first_last":{"event":"login","occurrence":"first","calculation":"specific_time","time_range":{"mode":"start_to_today","start_time":"2026-07-01"}}}
 ```
 
 ## SQL tag

package/skills/ae-analysis/references/user_tag_update.md

@@ -2,11 +2,11 @@
 
 Update a user tag. Discover the exact `tag_name` first.
 
-Do not use it for ID-file value replacement or to create a missing tag. Supplying `--definition-request` automatically starts recomputation after the definition is updated; do not call `user-tag refresh` afterward. Updating only `--display-name` or `--remark` does not recompute. `--auto-refresh-cron` changes an existing enabled auto-refresh schedule and does not enable auto refresh. A successful update means the definition was saved, not that the new result is complete. Record the current `refresh_time` before updating, then poll `user-tag get` until `progress=100` and `refresh_time` advances before using `users_num` or querying members.
+Do not use it for ID-file value replacement or to create a missing tag. Supplying `--definition-request` automatically starts recomputation after the definition is updated; do not call `user-tag refresh` afterward. Updating only `--display-name` or `--remark` does not recompute. A schedule-only update changes periodic refresh without starting immediate recomputation. `--auto-refresh-cron` or `--auto-refresh-schedule` can enable periodic refresh directly, including on a previously disabled tag. A successful update means the definition was saved, not that the new result is complete. Record the current `refresh_time` before updating, then poll `user-tag get` until `progress=100` and `refresh_time` advances before using `users_num` or querying members.
 
 The response distinguishes both paths. A definition update returns `computation.triggered_automatically=true`, `result_freshness.is_stale=true`, and normally `next_action=poll_get` with an exact capability/input pair. A display-name/remark-only update returns `computation.status=not_triggered`, `result_freshness.status=fresh`, and `next_action=none`.
 
-Flags: `--project-id`, `--tag-name` required. Optional: `--display-name`, `--definition-request`, `--authenticated-only`, `--remark`, `--zone-offset`, `--auto-refresh-cron`. The tag type comes from `definition_request.type` when the definition changes.
+Flags: `--project-id`, `--tag-name` required. Optional: `--display-name`, `--definition-request`, `--authenticated-only`, `--remark`, `--zone-offset`, `--enable-auto-refresh`, `--auto-refresh-schedule`, `--auto-refresh-cron`. The tag type comes from `definition_request.type` when the definition changes.
 
 `display_name` is at most 80 characters and `remark` is at most 400 characters. The CLI rejects violations before dispatch. `tag_name` is an existing exact identifier and cannot be renamed by update.
 
@@ -17,3 +17,15 @@ ae-cli analysis user-tag update --project-id <project_id> --tag-name user_level
 
 ae-cli analysis user-tag update --project-id <project_id> --tag-name user_level --auto-refresh-cron '0 30 2 * * ? *'
 ```
+
+Omitting scheduling flags preserves the existing enable state and schedule, including during a definition update. Omitting `--zone-offset` preserves the saved tag timezone. `--enable-auto-refresh true` can reuse an existing saved plan; if none exists, provide a new schedule. `--enable-auto-refresh false` disables periodic refresh. Do not combine false with a schedule or pass both schedule forms. Structured schedule fields and timezone rules are described in [user_tag_create.md](user_tag_create.md).
+
+```bash
+# Enable daily refresh in one update
+ae-cli analysis user-tag update --project-id <project_id> --tag-name user_level --auto-refresh-schedule '{"frequency":"daily","time":"02:30"}'
+
+# Disable periodic refresh
+ae-cli analysis user-tag update --project-id <project_id> --tag-name user_level --enable-auto-refresh false
+```
+
+Read `user-tag get` after updating to verify `enable_auto_refresh` (1=enabled, 0=disabled), `scheduler_ui_config`, and `cluster_zone_offset`. Schedule-only updates return `computation.status=not_triggered` and `next_action=none`.

package/skills/ae-data-integration/SKILL.md

@@ -17,6 +17,7 @@ Two entrances lead here: the AE Agent dialog (attach / plus-button upload) and `
 - Do not invent account IDs, distinct IDs, event times, event names, projects, APPIDs, receivers, or timezones.
 - `value_mapping` and `random_pool` are explicit user decisions. Never invent them.
 - Never auto-fill a missing time for `track`/`track_*` rows. A missing time on user-profile rows may be filled with the current time only by setting `missing_time: 'now'` and only after the user explicitly confirms it.
+- The mapping's fill-in options (`missing_time: 'now'`, `account_id_value`/`distinct_id_value`, `random_pool`, `exclude_columns`, `value_mapping`) record a decision the user has already made; they are never to make a validation failure disappear. When `convert` quarantines rows, present the failure first — the error `code`, the row count, its share of the total, and which events or time ranges are affected — and state the consequence in the user's terms ("3,000 events would carry a time that is not when they happened"). Only after the user has seen that may one of these options be set. When the evidence is insufficient, report the batch as not passed and pending customer data; never shrink the upload scope to manufacture a pass.
 - Do not read or send an AE access token or CLI token to `/sync_json`. The receiver request uses only APPID and UE data.
 - Never execute `data-integration upload` until the user has seen the target, mapping, valid/quarantined counts, batches, and dry-run and has explicitly confirmed that upload.
 - A blocked manifest requires a second, explicit clean-subset decision. Never add `--allow-clean-subset` implicitly.
@@ -35,6 +36,7 @@ Use this skill when the user wants to bring a **local data file** (CSV/TSV/TXT/J
 | Generate / upload a project-level tracking plan (source material is PRD / chat / template / code; deliverable is a real platform tracking plan) | ae-generate-tracking-plan |
 | Upload documents / URLs to a knowledge base | ae-kb |
 | Reports / dashboards / queries / governance on data already in AE | ae-analysis |
+| Dimension / dictionary data (a stable-entity lookup — city / product / device) to load as a dimension table bound to a property | ae-metadata |
 
 This skill also produces a tracking-plan draft (`source_type: data`) as a governance prerequisite; that draft is an input to ae-generate-tracking-plan, not a substitute for its five-phase platform plan.
 
@@ -42,7 +44,7 @@ This skill also produces a tracking-plan draft (`source_type: data`) as a govern
 
 Walk the four submodules in order. Each submodule is its own reference; follow it and come back here for the next step.
 
-1. **Source — business identification.** Read [references/source-inspect.md](references/source-inspect.md). Profile every file fully, infer its business meaning using business-doc / user-prompt priors, then pick a branch via [references/ue-routing.md](references/ue-routing.md).
+1. **Source — business identification.** Read [references/source-inspect.md](references/source-inspect.md). Profile every file fully, infer its business meaning using business-doc / user-prompt priors, then pick a branch via [references/ue-routing.md](references/ue-routing.md): UE ingestion, dimension routing ([references/dimension-routing.md](references/dimension-routing.md)), or local analysis.
 2. **Reuse check.** If the profile is `ue_eligible`, read [references/reuse.md](references/reuse.md) and match the recommended mapping against the handoff index. `reuse` searches the current directory's `.ae-cli/data-integration/` upward, then `~/.ae-cli/data-integration/`, so a package written elsewhere is still found. A match proposes a frozen package; after one explicit confirmation, run the returned `transform.mjs` command and jump to Sink (step 5). No match → continue.
 3. **Tracking plan.** Read [references/tracking-plan.md](references/tracking-plan.md). The plan is generated from the mapping (`plan --mapping`), so confirm the recommended mapping's key system fields with the user first — `mode`, `#account_id`/`#distinct_id`, `#time` + timezone, `#event_name`, `#ip`/`#uuid` (see [references/transform.md](references/transform.md) steps 1–5) — then generate the event/property plan and get a single explicit confirmation from the user before touching data. The plan is a separate, required deliverable from the transform mapping: a user who supplies a column→field mapping directly has **not** completed this step, so build the plan from the confirmed mapping anyway. `user_set` still requires a plan (no events; every property becomes a user property). This step runs for **every** file: a second or later file merges its new events and properties into the existing project plan (tracking-plan.md step 4) — an existing plan is never a reason to skip it.
 4. **Transform.** Read [references/transform.md](references/transform.md). Map columns to AE system fields and properties, convert, and quarantine dirty rows per [references/ue-mapping.md](references/ue-mapping.md).

package/skills/ae-data-integration/references/dimension-routing.md

@@ -0,0 +1,36 @@
+# Dimension routing
+
+Use this reference after [ue-routing.md](ue-routing.md) has classified the file as dimension data — a stable-entity lookup with no row-level identity or event time. It covers the handoff to ae-metadata; this skill does not ingest dimension data itself.
+
+## What dimension data is
+
+Judge by content, never by file extension — CSV / TSV / TXT / JSON / JSONL / XLS / XLSX can all be dimension data. The classification signals live in [ue-routing.md](ue-routing.md): no row-level identity, no row-level event time, finite entity enumeration, and a join key shared with event data.
+
+Dimension data is not the only file that fails UE prerequisites. Aggregates, pivot tables, and cumulative snapshots also lack identity/time, but those are local-analysis material, not dictionaries. The tell is the entity shape and the join key: a dimension table maps one entity code to its attributes (`city_code` → name / level), while an aggregate summarizes many rows into one measure.
+
+Low confidence is a proposal, never a silent decision — ask the user instead of routing automatically.
+
+## Handoff to ae-metadata
+
+Extract the dimension sheet / file to CSV, then hand the following commands to ae-metadata. The sequence lists entry points only; full flags live in ae-metadata's references.
+
+Bind prerequisite (confirm before creating the table):
+
+- The table binds to an existing property (`property_name` + `property_scope` = user or event). That property usually appears once event/user data is uploaded first (e.g. events carrying `city_code`), so dimension binding is a second-phase action after data lands.
+- If the target property does not exist yet, create it first via ae-analysis metadata or a tracking plan. Never invent a property name.
+
+Entry command sequence:
+
+```bash
+# 1. Extract the dimension sheet/file to CSV (metadata upload accepts CSV only,
+#    purpose data_table.csv)
+ae-cli analysis input-file upload --project-id <id> --purpose data_table.csv --file <dim.csv>
+# 2. Create + bind in one step (or split into csv-write + bind-existing)
+ae-cli metadata property create-and-bind-csv-dimension-table --project-id <id> \
+  --property-name <p> --property-scope user|event --input-file-id ifile_xxx
+# 3. Later dictionary changes (add/update/delete):
+ae-cli metadata data-table csv-write --operation incremental_update|replace_update \
+  --data-table-id <id> --input-file-id ifile_xxx
+```
+
+Binding model: AE attaches a dimension table to a user/event property, turning it into a dict property whose values join through the table's key column to expand `--dict-columns`. See ae-metadata's dimension-table reference for the full flags.

package/skills/ae-data-integration/references/error-handling.md

@@ -52,7 +52,7 @@ the user decide — never to guess an encoding or structure and retry silently.
 | `LOCAL_DATA_INPUT_INVALID` | Generic parse failure (malformed CSV/TSV/JSON/XLS) | Verify encoding and structure, then retry without changing the source |
 | `LOCAL_DATA_JSONL_INVALID` | A JSONL line is not valid JSON (`location.record` names the line) | Point at the offending record |
 | `LOCAL_DATA_JSON_ROOT_INVALID` | JSON root is not an object or array | Check the file's top-level shape |
-| `LOCAL_DATA_XLSX_INVALID` | Workbook metadata missing / no readable sheets / worksheet entry missing | Re-export the workbook |
+| `LOCAL_DATA_XLSX_INVALID` | Workbook metadata missing / no readable sheets / worksheet entry missing | Re-export the workbook (sheet recognition accepts `<sheet>` and namespaced `<x:sheet>` alike, so this is a real metadata gap) |
 | `LOCAL_DATA_SET_NOT_FOUND` / `LOCAL_DATA_SET_REQUIRED` | Sheet or JSON Path not found / ambiguous | Ask which `--data-set` to use |
 
 A parse error is never a reason to change the mapping or the tracking plan. Report the code and
@@ -81,6 +81,59 @@ Write failures (disk full `ENOSPC`, permission denied `EACCES`) do not hang the
 output streams fail with a clear `Failed to write "<path>"` message telling the user to check disk
 space and directory permissions. Fix the environment, then re-run into a fresh output directory.
 
+## File-level data-quality severity
+
+The classes above are per row or per field. The user also needs one verdict for the file as a whole,
+and its signals arrive scattered across the manifest and stderr. Grade them before reporting so the
+same file gets the same verdict no matter who reports it.
+
+| Severity | Signal | Handling |
+| --- | --- | --- |
+| Critical | Established with the user to be a cumulative snapshot or an aggregate report (see [ue-routing.md](ue-routing.md)); a required identity or time column empty for the whole file | Do not upload. Resolve with the user first |
+| High | `manifest.output.invalid_records` is a large share of the row count; inspect reported `leading_title_rows` or `header_signal` and the user has not yet said whether those rows are a title/banner; inspect reported `xlsx_structure.merged_covered_cells` and the user has not yet said whether the merged label belongs on the rows the block covers; `summary_rows` was reported and the user has not yet said whether those rows are totals or real records; `duplicate_keys` was reported and the user has not yet said whether the repeated rows are separate observations | Upload is allowed, but name the item explicitly in the confirmation gate and in the completion response |
+| Medium | `manifest.output.skipped_fields`; ragged delimited rows; `manifest.output.flatten_misses`; `manifest.output.unreadable_cells`; `xlsx_structure.hidden_rows` or `hidden_columns` read as data | Report the counts; the gate is unchanged |
+| Low | `manifest.output.lan_ip_records` | Report once |
+
+Rules:
+
+- Severity is reported, never silently applied. A Critical finding stops the pipeline and is stated
+  as a data finding, not as a program error.
+- Grade only what the run actually emitted. Every signal that belongs at High or Critical above now
+  has a detector, so grade it from what the run reported — never tell the user the tool checked
+  something it did not.
+- `unreadable_cells` counts XLSX cells that carried no value the tool may use, grouped by cause:
+  `formula_no_cached_value` (the file stores a formula but not the result Excel last computed),
+  `error_value` (`#N/A`, `#DIV/0!`, …), `unreadable_object` (an unrecognized cell shape). The cells
+  read as missing and their rows are kept, so the record count says nothing about them — a column
+  that is empty in AE while the spreadsheet looks full is this. The tool never evaluates a formula
+  and never guesses a result; the fix is to recalculate and re-export in Excel, or to export values
+  instead of formulas. `inspect` reports the same counts before conversion.
+- `xlsx_structure` records worksheet layout the rows themselves cannot carry: `merged_covered_cells`
+  counts cells that are empty only because a merged block covers them (Excel shows the value on the
+  block's first row), `hidden_rows` and `hidden_columns` name what the worksheet hides. The default
+  read changes none of it, so these counts describe what was uploaded: a column mostly missing in AE
+  while the spreadsheet looks full is the first of them. `merged_cells_filled` and
+  `excluded_hidden_rows` say what the run did about it, which is only ever what the user asked for
+  via `--fill-merged-cells` / `--exclude-hidden-rows` (mapping: `fill_merged_cells` /
+  `exclude_hidden_rows`). XLSX only — a legacy `.xls` workbook is not scanned.
+- `summary_rows` names rows that read as a summary line rather than an observation, by `row` (the
+  data-row ordinal) and `signals`: `total_label` (a cell reads as `合计` / `总计` / `小计` / `汇总` / `Total` /
+  `Subtotal`) and `column_total` (a number equal to the total of its column's other rows). The rows
+  were converted like any other, so this is a finding about what was uploaded: one fabricated event
+  whose amount is the whole group's, and a column whose reported `sum` is twice its real total. There
+  is no flag that drops a data row, because a row labelled `合计` is sometimes a real record; the fix
+  is to remove it from the source file or re-export without it. Any format, `.xls` included.
+- `duplicate_keys` names rows the source repeated under the same business key, by `key_columns` (the
+  columns compared), `duplicate_groups`, `extra_rows` (surplus records an upload would carry), and
+  `groups` with `count`, the data-row `rows`, and a `key_hash` prefix — never the key's own values.
+  Nothing was removed: a repeat is sometimes a real pair of records, two order lines in the same
+  checkout second, and AE appends accepted events with no way to un-send one, so ask the user whether
+  the rows are separate observations; if not, have them remove the rows from the source file. Values
+  are compared as written, so a repeat spelled two ways is missed, and `tracking_truncated` means
+  distinct keys outran the scan's budget and there may be more. Any format, `.xls` included.
+- A large quarantine share has no fixed threshold. State the ratio and the dominant error `code`,
+  and let the user judge.
+
 ## Cross-cutting rules
 
 - Classify first, act second. Match on `code`, not on message text.

package/skills/ae-data-integration/references/local-analysis.md

@@ -1,5 +1,7 @@
 # Local analysis
 
+Dimension / dictionary data (a stable-entity lookup with a join key) is not local-analysis material — it routes to ae-metadata as a dimension table. See [ue-routing.md](ue-routing.md) and [dimension-routing.md](dimension-routing.md).
+
 Keep the source on the local machine. Generated scripts and reports belong under `.ae-cli/data-integration/runs/<run-id>/` with restrictive permissions.
 Set the directory to `0700` and generated scripts/reports to `0600`.
 

package/skills/ae-data-integration/references/source-inspect.md

@@ -10,6 +10,8 @@ Confirm:
 
 Accept one or more CSV, TSV, TXT, JSON, JSONL (NDJSON), XLS, or XLSX files. CSV/TSV/TXT/JSON/JSONL/XLSX have no hard size limit; files over 1 GB print a stderr warning with an estimated processing time and suggest splitting. XLS over 100 MB prints a memory-risk warning (the legacy parser loads the whole workbook, roughly 5-10x file size); XLS over 1 GB is still rejected — convert it to XLSX or split it first.
 
+XLSX worksheet recognition matches `<sheet>` and namespaced `<x:sheet>` alike: some cleaning/export tools rewrite the default OOXML namespace as a prefix, and the workbook reads the same either way. The worksheet *row* nodes themselves are still read unprefixed, so a workbook whose row data is also namespaced falls back to CSV.
+
 ## Inspect without exposing raw values
 
 Before the full inspection (which streams and profiles the entire file and can take
@@ -39,16 +41,30 @@ If `selection_required=true`, show only the Sheet/JSON Path candidates and ask t
 ae-cli data-integration inspect --input-file '<path>' --data-set '<candidate-id>' --source-timezone '<iana-timezone>'
 ```
 
-Report row/column counts, field types, missing/unique/time-parse ratios, UE eligibility, mapping confidence, and warnings. Samples are bounded (up to 5 distinct, truncated) — summarize, never paste them. ID-like columns (`id`, `*_id`, `*_key`, `*_code`, `*_no`, `*_num`) stay `string` even when every value is numeric; JSON-encoded object/array values inside CSV cells are recognized as `object`/`list`, not `string`. IP- and UUID-named columns are additionally checked against their value specs: inspect warns how many non-empty values are invalid IPv4/IPv6, private/LAN IPs, or non-UUID strings, so the user can decide whether to map them as `ip_field`/`uuid_field`. Read [UE routing](ue-routing.md) before choosing a branch.
+Report row/column counts, field types, missing/unique/time-parse ratios, UE eligibility, mapping confidence, and warnings. Samples are bounded (up to 5 distinct, truncated) — summarize, never paste them. ID-like columns (`id`, `*_id`, `*_key`, `*_code`, `*_no`, `*_num`) stay `string` even when every value is numeric; JSON-encoded object/array values inside CSV cells are recognized as `object`/`list`, not `string`. IP- and UUID-named columns are additionally checked against their value specs: inspect warns how many non-empty values are invalid IPv4/IPv6, private/LAN IPs, or non-UUID strings, so the user can decide whether to map them as `ip_field`/`uuid_field`. Excel columns whose cells carry a date number format infer as `datetime` and are named in a warning (see **Excel date cells** below). Read [UE routing](ue-routing.md) before choosing a branch.
 
 A stderr `Warning: … column count different from the header row …` means the CSV/TSV has ragged rows (extra fields dropped, missing fields treated as empty); report it as a data-quality signal. For how every pipeline failure — abnormal data, parse errors, and program errors — is classified and handled, see [error handling](error-handling.md).
 
+**Value frequency and numeric distribution.** A distinct count says how many different values a column holds, not whether they are worth uploading. Two per-column fields answer that, and both appear in `inspect` output only — never in the convert manifest:
+
+- `value_frequency` — the 10 most frequent values as `{value, count, ratio}`, values truncated the way `samples` are and counted after truncation. It is reported only for a column whose distinct values all fit the tracked budget (200), so the counts are exact and complete when present; a column past the budget reports nothing here and its `unique_count` is the field to read instead. Use it to separate an enum from free text: a `渠道` column with three values is a property worth uploading, and its listed values are also what a `value_mapping` decision is made from. A single value covering every row usually means an export artifact, not data — propose `exclude_columns` and let the user decide.
+- `numeric_summary` — `count`, `min`, `max`, `sum`, `mean`, `p25`, `median`, `p75` for a column that inferred as `number`. `count` is the values that read as numbers, which is below the column's non-missing count when the column is mixed. `count`/`min`/`max`/`sum`/`mean` are always exact; the quantiles come from a bounded sample on large columns and then `quantiles_approximate: true` says so. Use it to check the magnitude before it is locked into an AE property: a mean far below the maximum on a monotonically climbing column is the signature of a cumulative snapshot rather than a per-row measure (see [UE routing](ue-routing.md)), an amount whose values are 100× the expected size is a 分/元 unit mismatch, and a column that is entirely one number carries no signal.
+
+Summarize both — report the shape of the distribution and the names of the values, and do not paste the whole table into the conversation. They are read out of the customer's file like `samples` are.
+
 ## Advanced input
 
 - **Headerless files** — inspect auto-detects a missing header row on CSV/TSV and reports `no_headers: true` with a `header_detection` verdict and `auto_headers` placeholders (`col_1..col_N`); the first row is already treated as data. `--headerless` forces the same behavior without detection. Never keep the `col_1..col_N` placeholders — they carry no business meaning. For each column, read its bounded samples and inferred type and propose a meaningful name, present every proposal to the user (column position, sample summary, suggested name), and let the user confirm or rename each one; record the confirmed names in the mapping's `headers` field. When the user already knows the names, re-run inspect with `--headers 'col1,col2,...'` so the recommended mapping carries them.
+- **Title rows above the header** — an exported report often puts a caption in the first cell (`2026年3月销售明细`) and the real header row underneath. Read as-is, the caption becomes the file's only column name, every real column name is lost, and the header row is counted as a data row. Inspect reports the suspected rows under `leading_title_rows` (row ordinals and non-empty cell counts only, never the cell text) and warns — but it does **not** change what it read: the first row was still used as the header, because Excel exports legitimately carry numeric header rows (`2024`, `2025`) that look like data, and there is no flag that puts a header row back once it has been treated as data. So the report is a question for the user. When they confirm those rows are a title or banner, re-run inspect with `--skip-rows N` (N is exactly the last ordinal listed); the rerun reads the real header row, reports `skipped_rows`, and carries `skip_rows` into the recommended mapping so `convert` reads the same rows inspect profiled. On XLSX, inspect additionally reports `header_signal` when the row it used as the header looks like data — same rule: reported, not applied; resolve it with `--headers` or `--headerless`. This covers CSV/TSV and XLSX; a legacy `.xls` workbook accepts `--skip-rows` but is not scanned for title rows.
+- **Summary and total rows** — an exported report ends with a 合计 row, and a grouped one repeats 小计 after every group. Those rows are not observations: uploaded, each becomes an event that never happened whose amount is the whole group's revenue, and profiled, it doubles the column's `sum` and turns its `max` into the total. Nothing in the row itself says so, so inspect flags them under `summary_rows` and warns. Each entry carries `row` (the data-row ordinal, the same numbering `invalid.rows.jsonl` and `--salvage-from` use) and `signals`: `total_label` when a cell reads as a total label (`合计` / `总计` / `小计` / `汇总` as a prefix, `Total` / `Subtotal` / `Sum` as the whole cell), naming the column in `label_column` but never the cell text; `column_total` when a number on that row equals the total of its column's other rows, listing those columns in `total_columns`. A row can raise one signal or both — a labelled group subtotal holds its group's total, not the column's, so only the label fires. Nothing is removed and nothing is changed: `convert` writes these rows as records, and every number reported for their columns counts them, which is exactly why the finding has to be read. There is no flag that drops a data row, because a row labelled `合计` is sometimes a real business record; when the user confirms a row is a total, ask them to remove it from the source file or re-export without it, then inspect again. `convert` repeats the finding in `manifest.output.summary_rows`, since by upload time a subtotal row with a plausible identity and time is indistinguishable from data. This covers every format, `.xls` included — the check reads rows, not worksheet structure.
+- **Repeated business keys** — a customer re-exports a report whose range overlaps the last export, or pastes two sheets together, and the same observation arrives twice. Uploaded, each repeat is a second event: that user's revenue doubles, every funnel counts them twice, and AE appends accepted events with no way to un-send one — so the only place this is fixable is before the upload. Inspect compares each row's business key against the rows before it and reports repeats under `duplicate_keys`, plus a warning. The report carries `key_columns` (the columns actually compared — always read it, since the key is what the finding means), `checked_rows`, `duplicate_groups`, `extra_rows` (surplus records an upload would carry), and `groups`, each with `count`, the data-row `rows` (the numbering `invalid.rows.jsonl` and `--salvage-from` use), and a `key_hash` prefix that distinguishes groups without revealing values — the key's own text is never reported. `groups_truncated` / `rows_truncated` mean the list is bounded, not that the counts are; `tracking_truncated` means distinct keys outran the scan's budget and there may be more. The key comes from the mapping's identity, time, and event-name columns on `convert`, and from column-name matching on `inspect`; a single column is never a key, so a file with no recognizable time column is not scanned at all (identity alone would call every returning user's second row a repeat). Two limits to state when reporting: values are compared as written, so `2026-03-01 10:00:00` and `2026/03/01 10:00:00` are two different keys and a repeat spelled two ways is missed; and a source's own unique key (an order id) is not compared on unless the mapping names it as identity, time, or event. Nothing is removed — a repeat is sometimes a real pair of records, two order lines in the same checkout second — so ask the user whether the rows are separate observations, and if they are not, have them remove the rows from the source file and inspect again. `convert` repeats the finding in `manifest.output.duplicate_keys`, because by upload time both copies are ordinary valid records; it describes the whole source file even on a `--salvage-from` run, since every valid row of that file ends up in AE. This covers every format, `.xls` included — the check reads rows, not worksheet structure.
 - **TSV / TXT** — `.tsv` and `.tab` use a tab delimiter with no quoting convention; `.txt` and unknown extensions are content-sniffed into CSV, TSV, or NDJSON.
 - **Encoding** — text files are auto-detected (UTF-8, GBK, GB2312, Big5, and others); no flag is needed.
-- **Excel sheets** — `--merge-sheets` streams every worksheet in file order instead of a single selected sheet; otherwise ask which sheet/`--data-set` to use. Inspect also reports `header_consistency` (`all_same` or `different`) across a workbook's sheets, with `header_details` listing each sheet's header row when they differ; prefer `--merge-sheets` only when headers match.
+- **Excel date cells** — a cell whose number format is a date or date+time is read as the wall-clock timestamp shown in Excel, not as the Excel serial number stored behind it, so the column infers as `datetime` and can serve as the time field. Inspect lists every such column in a warning. Treat that warning as a question to the user, not as a note: the same column profiled as `number` before this behavior existed, so if any part of this file was already sent to AE, the property may have been received as a number and its type is now locked — it cannot be changed to datetime, and the column has to be re-sent under a new property name. Ask whether the column was uploaded before, and only map it once the user answers. Elapsed-duration formats (`[h]:mm:ss` and the equivalent built-ins) are durations rather than points in time and stay `number`.
+- **Excel formula cells** — a spreadsheet stores a formula and, next to it, the result Excel last computed. That cached result is the value: it is read normally, including a result of `0` or `""`, which are real values and not blanks. This tool never evaluates a formula and never guesses a result, so a cell holding a formula the file never computed has nothing to upload; it is read as missing and counted, as is an Excel error value (`#N/A`, `#DIV/0!`, …). Inspect reports the counts per column in a warning and `convert` repeats them in `manifest.output.unreadable_cells`. Report them: the rows are kept and the record count is unchanged, so this is the only explanation for a column that is empty in AE while the spreadsheet looks full. When a column that matters reads as missing this way, ask the user to recalculate and re-export in Excel, or to export values instead of formulas, before uploading. This covers XLSX; a legacy `.xls` workbook goes through a different parser and is not counted here.
+- **Merged cells, hidden rows, and hidden columns** — a sheet maintained by hand merges a label down the rows it covers (`区域` spanning one region's block). Excel keeps that value on the block's first row only and stores every row below it as an empty cell, so a column that looks full on screen arrives mostly missing, and the AE property built from it would be empty for most events. The same worksheet may also hide a row inside a data block or hide a whole column. None of this travels with a row, so inspect scans the worksheet structure separately and reports it under `xlsx_structure`: `merged_ranges` with `merged_range_samples` (references such as `A3:A5`, never cell text), `merged_covered_cells` per column, `hidden_rows` with `hidden_row_samples` (source row numbers as Excel numbers them), and `hidden_columns` by header name. The default read is unchanged, so the report is a question for the user, and each answer is a flag: `--fill-merged-cells` copies each block's value into the cells its own range covers — bounded to the range, never overwriting a value that is there and never inventing one when the block's own cell is empty, so it is not a forward fill; `--exclude-hidden-rows` leaves hidden rows out. Neither is on by default: those cells really are empty in the file, and a row hidden inside a data block may still be real data — unlike a hidden *worksheet* (below), which is excluded by default. Hidden columns have no flag at all; when the user confirms one is not data, list it in the mapping's `exclude_columns`. Both flags are carried into the recommended mapping as `fill_merged_cells` / `exclude_hidden_rows`, which is what makes `convert` read the rows inspect profiled — `convert` has no read flags of its own — and `convert` repeats the findings in `manifest.output.xlsx_structure`, the only record of a layout the converted rows no longer show. This covers XLSX; a legacy `.xls` workbook is not scanned, so ask the user about merged labels and hidden rows there instead of trusting silence.
+- **Hidden worksheets** — a worksheet hidden in the workbook is left out of the `--data-set` candidates and out of `--merge-sheets`, because a sheet the file does not show is usually scratch space or a superseded draft rather than rows anyone meant to upload, but a hidden sheet can also be a dimension / dictionary table that is meant to be loaded, and that goes through dimension routing instead of being dismissed as scratch. Inspect lists each one under `excluded_sheets` (with `reason: hidden`); report those names to the user, since they are the only explanation for a row count lower than the workbook appears to hold. Their headers are also left out of `header_consistency`, so a stale hidden draft cannot make a mergeable workbook look ragged. A hidden sheet stays readable when the user names it in `--data-set` — the command then warns on stderr that the selected sheet is hidden. Only pass a hidden sheet after the user says that is what they want. When *every* worksheet is hidden there is no candidate left, and inspect fails with `LOCAL_DATA_ALL_DATA_SETS_HIDDEN` whose hint lists the hidden sheets; treat that as a question about which sheet holds the real data, not as an unreadable file. This detection covers XLSX only: a legacy `.xls` workbook's sheet list is unfiltered, so a hidden sheet there still appears as a candidate and is still merged — for `.xls`, ask the user to confirm the sheet list instead of trusting it.
+- **Excel sheets** — `--merge-sheets` streams every visible worksheet in file order instead of a single selected sheet; otherwise ask which sheet/`--data-set` to use. Inspect also reports `header_consistency` (`all_same` or `different`) across a workbook's sheets, with `header_details` listing each sheet's header row when they differ; prefer `--merge-sheets` only when headers match. Matching headers establish a shared structure, not disjoint rows: a detail sheet and a summary sheet, or `1月` and `1月修订版`, usually carry identical headers and would be merged and reported twice over. Before merging, confirm with the user that the sources are mutually exclusive partitions (one month per sheet, no overlap) rather than overlapping, revised, or derived views of the same rows, and show each sheet's row count and time coverage range in that confirmation so an overlap is visible. The same rule applies to repeated `--input-file`.
 - **Multi-file type conflicts** — when the same column has different inferred types across files, present each conflict and resolve with `--type-resolutions` on `convert` (see [transform](transform.md)).
 
 ## Nested flattening (NDJSON/JSON records and JSON-encoded CSV/TSV/Excel cells)

package/skills/ae-data-integration/references/tracking-plan.md

@@ -10,7 +10,7 @@ Generate and confirm the event/property plan **before** any transform or upload.
 ## Sub-steps
 
 1. **Event-model decision** — reuse the UE routing result: single-table single-event `track`, single-table multi-event (event-name column), `user_set`, or `mixed`. The agent may propose splitting one table into several events (for example an ad table into `ad_show`/`ad_click` by `campaign_type`); that proposal must be confirmed by the user in the confirmation gate.
-2. **Column → property draft** — confirm the recommended mapping's key system fields with the user **before** drafting: `mode` (`#type`), `#account_id`/`#distinct_id` (ask together; at least one is required — a `user_id` column can be either an anonymous or a login ID and only the user knows), `#time` + source timezone + `#zone_offset`, `#event_name` (track only; the event column or a reviewed `default_event_name`), and `#ip`/`#uuid` when the data has such a column. The exact questions and the never-infer-from-a-column-name-alone rule are [transform.md](transform.md) steps 1–5; run them here. The plan is generated from this mapping, so never draft from an unconfirmed mapping. With the system fields settled, map the remaining columns to event and/or user properties (the mapping `mode` decides). Common event properties — project-level super properties attached to every event — are defined by `ae-generate-tracking-plan`, not this import path. Name events and properties in snake_case and fill **every** `display_name`, `desc`, and `event_tag` (events also carry `event_desc`). Infer all three from field names, value distribution, samples, and business-doc / prompt priors — never leave them empty: `desc`/`event_desc` state what the item means in plain language (language follows the user), and `event_tag` picks the closest category from the canonical tag list (see the `event_tag` appendix in `../../ae-generate-tracking-plan/references/business-dimension-mapping.md`). When you cannot infer a `desc` or `event_tag`, mark it pending and ask the user for it inside the confirmation gate. Infer types (`number` / `bool` / `datetime` / enum) the same way; CSV defaults to `string`. Columns that stay uncertain or conflicting are marked pending and asked only inside the confirmation gate.
+2. **Column → property draft** — confirm the recommended mapping's key system fields with the user **before** drafting: `mode` (`#type`), `#account_id`/`#distinct_id` (ask together; at least one is required — a `user_id` column can be either an anonymous or a login ID and only the user knows), `#time` + source timezone + `#zone_offset`, `#event_name` (track only; the event column or a reviewed `default_event_name`), and `#ip`/`#uuid` when the data has such a column. The exact questions and the never-infer-from-a-column-name-alone rule are [transform.md](transform.md) steps 1–5; run them here. The plan is generated from this mapping, so never draft from an unconfirmed mapping. With the system fields settled, map the remaining columns to event and/or user properties (the mapping `mode` decides). Common event properties — project-level super properties attached to every event — are defined by `ae-generate-tracking-plan`, not this import path. Name events and properties in snake_case by default (keep an uppercase event name only when the user asks to preserve it; property names stay lowercase-only) and fill **every** `display_name`, `desc`, and `event_tag` (events also carry `event_desc`). Infer all three from field names, value distribution, samples, and business-doc / prompt priors — never leave them empty: `desc`/`event_desc` state what the item means in plain language (language follows the user), and `event_tag` picks the closest category from the canonical tag list (see the `event_tag` appendix in `../../ae-generate-tracking-plan/references/business-dimension-mapping.md`). When you cannot infer a `desc` or `event_tag`, mark it pending and ask the user for it inside the confirmation gate. Infer types (`number` / `bool` / `datetime` / enum) the same way; CSV defaults to `string`. Columns that stay uncertain or conflicting are marked pending and asked only inside the confirmation gate.
 3. **Confirmation gate (single, one pass)** — present the concrete plan, never a counts-only summary: the confirmed key system-field mapping (`mode`, `#account_id`/`#distinct_id`, `#time` + source timezone, `#event_name`; `#ip`/`#uuid` when present), then a full event table (one row per event: `event_name` + `event_tag`/`event_desc` + the properties attached to it), then a full property table (one row per property: source column → target AE name → type → `display_name`/`desc`, uncertain types highlighted; a kept-whole `object`/`array_row` lists its `parent.child` sub-properties next to the parent), plus field scope (default: plan fields only, with a full-import switch) and unrecognized/dirty-data handling. For a multi-sheet workbook, group the property table by sheet so each sheet's source columns are visible. The user answers once with ok or edits (renames, types, identity/time/event columns, add/drop columns).
 4. **Merge with the existing plan** — fetch the project's current tracking plan; same-name property type conflicts are severe, same-name events are advisory; decide append vs replace. This runs for **every** file, not just the first: when the project already has a plan (an earlier file or run), diff this file's events and properties against it and put the additions — new events, new properties, new object sub-properties from flattening — in the confirmation gate. An existing plan is never a reason to skip this step; only when every addition is already present may you skip the merge, and even then state and confirm that fact with the user.
 5. **Persist the plan** — `.ae-cli/data-integration/draft.json` → `.ae-cli/data-integration/draft.xlsx` → upload with `sdk_integration_mode=none`.

package/skills/ae-data-integration/references/transform.md

@@ -17,7 +17,7 @@ Confirm the system fields with the user before touching properties. These are th
 4. **`#event_name` (track only).** Confirm the event-name column, or a reviewed `default_event_name` when no column exists.
 5. **`#ip` / `#uuid` (optional).** Ask only when the data has an IP- or UUID-like column; map it via `ip_field`/`uuid_field`, otherwise skip. `#ip` is event data only and must be a valid IPv4/IPv6 address (a private/LAN IP is kept but reported — AE cannot geolocate it); `#uuid` must be a standard 36-character UUID. A value that violates the spec is dropped from that row only (`INVALID_IP` / `INVALID_UUID`) — the row itself is kept. The program never auto-generates a `#uuid`.
 
-When an `#event_name`, `#account_id`, or `#distinct_id` column's values do not satisfy AE naming rules (pure Chinese, uppercase, spaces), do not stop — scan the distinct values, list them to the user, and ask for one AE-name replacement each; record the pairs in `value_mapping` (see [UE mapping](ue-mapping.md)). A value with no matching key keeps its original text and fails validation, so confirm every distinct value is covered or excluded. The same mechanism applies to a property column via that entry's own `value_mapping`.
+When an `#event_name`, `#account_id`, or `#distinct_id` column's values do not satisfy AE naming rules (pure Chinese, spaces), do not stop — scan the distinct values, list them to the user, and ask for one AE-name replacement each; record the pairs in `value_mapping` (see [UE mapping](ue-mapping.md)). Event names are lowercased by default; keep an uppercase event name only when the user asks to preserve it — leave a legal uppercase value unmapped so it passes through, or map an illegal value to an uppercase target (e.g. `购买` → `Purchase`); see the casing rules in [UE mapping](ue-mapping.md). A value with no matching key keeps its original text and fails validation, so confirm every distinct value is covered or excluded. The same mechanism applies to a property column via that entry's own `value_mapping` — property names stay lowercase-only, so uppercase property values still need `value_mapping`.
 
 Then confirm the property set with the user before saving the mapping. Present every property as a readable table — one row per property with source column, target AE name, and type — never dump the raw mapping JSON at the user. Whether the rows are event or user properties follows `mode`: `track` → **event properties**, `user_set` (or another profile mode) → **user properties**, `mixed` → the same set applies to both event and user rows.
 
@@ -48,7 +48,9 @@ ae-cli data-integration convert \
   --output-dir '.ae-cli/data-integration/runs/<run-id>'
 ```
 
-The command never modifies the source. Inspect `manifest.json`; summarize valid and quarantined counts and the block reason. If `manifest.output.skipped_fields` is present, tell the user how many `#ip`/`#uuid` values were invalid and dropped (the rows were otherwise kept); if `manifest.output.lan_ip_records` is present, tell the user that many `#ip` values are private/LAN addresses that AE cannot geolocate. Neither blocks the manifest. If `manifest.output.flatten_misses` is present — or a stderr `Warning: flatten rule "X" did not materialize for N row(s).` fires — a `flatten_rules` path missed some rows: re-check the dot path against the source shape (or confirm the column is legitimately optional); the rows are otherwise kept. A stderr `Warning: … column count different from the header row …` means ragged CSV/TSV rows were tolerated (extra fields dropped, missing fields treated as empty) — surface it as a data-quality note. A blocked manifest whose reason is `The source contained no data rows.` means the file had zero data rows; do not re-run the same command on it. Do not expose rows from `invalid.rows.jsonl` unless the user specifically asks to inspect the local quarantine. For the full failure taxonomy and how to respond, see [error handling](error-handling.md).
+Merging several sources into one run assumes they are mutually exclusive partitions of the same data set. Confirm that with the user before converting — matching headers do not rule out a detail/summary pair or an original/revised pair — and present each source's row count and time coverage range so an overlap is visible. The same applies to `--merge-sheets` (see [source-inspect.md](source-inspect.md)).
+
+The command never modifies the source. Inspect `manifest.json`; summarize valid and quarantined counts and the block reason, and check the conservation equation first: `manifest.output.source_rows` must equal `valid_records + invalid_records` — a mismatch means rows were dropped or duplicated between the source and the output, so report the three numbers to the user and do not upload until it is explained. A salvage run's `source_rows` is only the rows it re-processed, not the whole file. If `manifest.output.skipped_fields` is present, tell the user how many `#ip`/`#uuid` values were invalid and dropped (the rows were otherwise kept); if `manifest.output.lan_ip_records` is present, tell the user that many `#ip` values are private/LAN addresses that AE cannot geolocate. Neither blocks the manifest. If `manifest.output.flatten_misses` is present — or a stderr `Warning: flatten rule "X" did not materialize for N row(s).` fires — a `flatten_rules` path missed some rows: re-check the dot path against the source shape (or confirm the column is legitimately optional); the rows are otherwise kept. A stderr `Warning: … column count different from the header row …` means ragged CSV/TSV rows were tolerated (extra fields dropped, missing fields treated as empty) — surface it as a data-quality note. A blocked manifest whose reason is `The source contained no data rows.` means the file had zero data rows; do not re-run the same command on it. Do not expose rows from `invalid.rows.jsonl` unless the user specifically asks to inspect the local quarantine. For the full failure taxonomy and how to respond, see [error handling](error-handling.md).
 
 ## Re-report only the failed rows (salvage loop)
 

package/skills/ae-data-integration/references/ue-mapping.md

@@ -66,12 +66,14 @@ Each mapped system field carries a value spec, enforced at both inspect (warning
 | Field | Value spec | On violation (convert) |
 | --- | --- | --- |
 | `#account_id` / `#distinct_id` | Non-empty string, at most 128 characters | Row error `MISSING_USER_ID` (absent) / `USER_ID_TOO_LONG` (>128) |
-| `#event_name` | `^[a-z][a-z0-9_]{0,49}$` (lowercase snake_case, letter-leading, ≤50 chars) | Row error `INVALID_EVENT_NAME` |
+| `#event_name` | `^[A-Za-z][A-Za-z0-9_]{0,49}$` (letter-leading letters/digits/underscore, ≤50 chars; lowercase by default, uppercase kept only on user request) | Row error `INVALID_EVENT_NAME` |
 | `#time` | One of the supported formats, within 3 years back / 3 days forward | Row error `INVALID_TIME` / `TIME_OUT_OF_RANGE` |
 | `#ip` | Valid IPv4 or IPv6. Event data only | Field skip `INVALID_IP`; a private/LAN IP is kept and reported — AE cannot geolocate it |
 | `#zone_offset` | Integer -12..14 (or an IANA name for `zone_offset_value`). Event data only | Row error `INVALID_ZONE_OFFSET` |
 | `#uuid` | Standard 36-character UUID (`xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx`). Both data kinds | Field skip `INVALID_UUID` |
 
+Event-name case is decided entirely by the mapping, never by the CLI — the CLI performs no case conversion and has no flag for it. The name AE receives is the source `event_name_field` value (or `default_event_name` when there is no event column), optionally replaced by a `value_mapping.event_name` target; `data-integration plan --event-name` passes each name through unchanged. Lowercase is the default: lowercase the name when writing the mapping (a `value_mapping.event_name` entry such as `Purchase` → `purchase`, or a lowercase `default_event_name`). To keep uppercase, skip that lowercasing: leave a legal uppercase source value unmapped so it passes through, write an uppercase `value_mapping.event_name` target only when the source value is not itself a legal name (e.g. `购买` → `Purchase`), or write an uppercase `default_event_name` / `--event-name`.
+
 A field skip is not a row failure: the row is kept with its other fields, and the count is reported in `manifest.output.skipped_fields` so the agent tells the user at the end. `#uuid` is never auto-generated — it comes only from a mapped source column.
 
 Mapping keys: `account_id_field`, `distinct_id_field`, `time.field`, `event_name_field` (or a reviewed `default_event_name`), `ip_field`, `uuid_field`. Inspect surfaces every identity-shaped column in `identity_candidates` (name, `account`/`distinct` kind, unique and missing ratios) so the agent can present all candidates for confirmation. When the required identity column is absent, use an explicit `account_id_value`/`distinct_id_value` placeholder or `random_pool` — a user decision, never invented.
@@ -147,7 +149,8 @@ Values with an explicit offset or `Z` are parsed by the JavaScript `Date` constr
 - Identity values remain strings and are at most 128 characters.
 - Source timezone is an IANA name derived from user/project context.
 - `#zone_offset`, when set, is a whole-hour integer in -12..14 (or an IANA name/column that resolves to one) and is emitted inside `properties`, never at the top level.
-- Event/property names are lowercase snake_case, begin with a letter, and are at most 50 characters.
+- Event names begin with a letter and are at most 50 characters; lowercase is the default, uppercase is kept only when the user asks to preserve it.
+- Property names are lowercase snake_case (letter-leading, at most 50 characters).
 - Target property names are unique and do not collide with UE system fields.
 - Types are one of `string`, `number`, `boolean`, `datetime`, `list`, `object`, or `array_row`.
 - Text is at most 2 KB; numbers stay within -9E15..9E15.

package/skills/ae-data-integration/references/ue-routing.md

@@ -20,7 +20,45 @@ Classification order:
 5. Rows that mix track and user-profile facts in one file use `mixed` with a `record_type_field`; require explicit review.
 6. Low-confidence output is a proposal, never silent approval.
 
-Aggregated metrics, pivot tables, cross-tabs, model outputs, free-form documents, and records without real identity/time should normally use local analysis.
+Aggregated metrics, pivot tables, cross-tabs, model outputs, and free-form documents should normally use local analysis. Records without real identity/time are checked against dimension routing next and fall to local analysis only if they are not a stable-entity lookup.
+
+### Time coverage is not native granularity
+
+A parseable time column establishes only when the rows are stamped, not what period each metric
+covers. A daily report and a cumulative snapshot both look like one row per user per point in time,
+so they satisfy every condition above and are then ingested as per-period events — inflating totals
+in a way that stays invisible in ratios, because numerator and denominator scale together.
+
+Native granularity must come from the user, a data dictionary, or a complete period structure in the
+data itself. Do not infer it from the file name, the first/last date, the interval between rows, or
+the row count; none of those is evidence.
+
+When the data carries numeric columns and any of the following holds, ask the user to state whether
+each row's value is the amount that occurred in that period or the total accumulated up to that
+point, and do not proceed until they answer:
+
+- Paired start/end time columns (`start_date`/`end_date`, `period_begin`/`period_end`).
+- Values for one identity that never decrease over time.
+- Column names carrying a to-date sense (`cumulative`, `total`, `ltv`, `累计`, `总`).
+
+An unanswered question, a cumulative snapshot, or overlapping periods route to local analysis
+instead.
+
+## Route to dimension table
+
+A dimension / dictionary table describes stable entities (city, product, device): a lookup that maps an entity code to its attributes. It has no row-level identity or event time, so it fails the UE prerequisites above, but it is not local-analysis material either — it belongs in AE as a dimension table bound to a property.
+
+Classification order (UE first, dimension second, local analysis last):
+
+1. Satisfy the UE must-holds above → UE ingestion wins; never route an identity/time-bearing file here.
+2. Fail the UE prerequisites **and** match most of these dimension signals → dimension routing:
+   - No row-level identity: no `account_id` / `distinct_id` column. A `code` / `id` / `no` key is an entity code, not a user identity.
+   - No row-level event time: no `#time` column. If time exists, it is an effective / expiry interval, not an event occurrence.
+   - Finite enumeration: few rows, each describing one entity's attributes (code → name / level), not facts accumulating over time.
+   - A join key: a column shared with event data (`city_code`, `sku_id`, `device_id`) whose values are descriptive attributes, not measures.
+3. Otherwise → local analysis.
+
+See [references/dimension-routing.md](references/dimension-routing.md) for the handoff.
 
 ## Route to local analysis
 
@@ -29,6 +67,7 @@ Choose local analysis when:
 - The user wants insights, not project ingestion.
 - UE identity or time prerequisites are missing.
 - Each row is an aggregate rather than a user/event record.
+- The rows are a cumulative snapshot, or their native granularity could not be established.
 - Conversion would invent semantics or discard important structure.
 - The user declines an uncertain mapping or destination.