@thinkingai/ae-cli 6.1.28 → 6.1.29
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +24 -4
- package/README.zh.md +24 -4
- package/bin/ae-cli.js +5 -0
- package/bin/runtime-capabilities.json +5 -0
- package/cli-installation-guide.md +24 -3
- package/dist/{auth-NKD5Z5BQ.js → auth-OBQ5KVQM.js} +30 -12
- package/dist/{automation-K32V2NKP.js → automation-NAR5DDGL.js} +6 -6
- package/dist/{capability-4CGSL7PK.js → capability-LFSVNI5B.js} +54 -26
- package/dist/chunk-2UW3Y463.js +2289 -0
- package/dist/{chunk-QW3R6O7L.js → chunk-3SDHJOCR.js} +57 -15
- package/dist/chunk-6HAUVWOQ.js +132 -0
- package/dist/{chunk-KHS64JZQ.js → chunk-72NJKS7O.js} +4 -4
- package/dist/{chunk-UUD5RB36.js → chunk-7JAVT5TK.js} +3 -3
- package/dist/{chunk-GTPWDLTE.js → chunk-CVCTFLSO.js} +35 -14
- package/dist/{chunk-P7KL7IOH.js → chunk-H3K5UGYC.js} +2 -2
- package/dist/{chunk-P6VFCH3P.js → chunk-IN7KKL6K.js} +2 -2
- package/dist/{chunk-T44M3PLZ.js → chunk-JEUXDY3B.js} +1 -1
- package/dist/{chunk-LMCYSVVA.js → chunk-JNVVJ273.js} +51 -37
- package/dist/{chunk-JJJIP2XQ.js → chunk-N6XBLRB7.js} +2 -2
- package/dist/{chunk-QSAR3GHY.js → chunk-OAWPT7VP.js} +2 -2
- package/dist/{chunk-UF244C5C.js → chunk-PWXDWMCW.js} +7 -7
- package/dist/{chunk-2IYIYSUO.js → chunk-QVRIDN42.js} +2 -2
- package/dist/chunk-QZLRHGVH.js +222 -0
- package/dist/{chunk-GI5BMEZO.js → chunk-QZUVLFNR.js} +1 -1
- package/dist/{chunk-2UWS5CUP.js → chunk-RJHGDWSC.js} +3 -3
- package/dist/{chunk-UJ7UPRXK.js → chunk-VCZE3IWO.js} +10 -8
- package/dist/{chunk-X7DRLE62.js → chunk-YVG6QD3V.js} +1 -1
- package/dist/{chunk-ZW4VMY5H.js → chunk-Z3Q2AEIN.js} +152 -98
- package/dist/{community-report-client-S2K3A3Q6.js → community-report-client-TJVCLZ3K.js} +1 -1
- package/dist/compat-check-CII6B74E.js +32 -0
- package/dist/{config-ASSAAIOI.js → config-YZ6O2UQ2.js} +38 -6
- package/dist/{context-6RWLVOM5.js → context-F6FCGTF3.js} +4 -4
- package/dist/{data-integration-EXSUPUDZ.js → data-integration-6UAZO4GG.js} +207 -24
- package/dist/{evaluation-5YT4QLCL.js → evaluation-4LO2O4AF.js} +4 -4
- package/dist/index.js +297 -70
- package/dist/{local-data-upload-client-7HCPGIS3.js → local-data-upload-client-KCMG6J62.js} +1 -1
- package/dist/{memory-VCYLVPMI.js → memory-HHPX3NF2.js} +6 -6
- package/dist/{metadata-YGI5M6OF.js → metadata-TIFY24LW.js} +16 -8
- package/dist/{model-DPJHSD73.js → model-7UHGWYVR.js} +4 -4
- package/dist/{personal-semantic-preference-76TBKAPY.js → personal-semantic-preference-WKJEGPTM.js} +7 -6
- package/dist/{project-semantic-XOEBO2Q5.js → project-semantic-EIQVZNHW.js} +11 -10
- package/dist/{sync-7PP6KQ75.js → sync-RV3OPCTI.js} +8 -7
- package/dist/{te-agent-PJVAPKBD.js → te-agent-5ENVZJUH.js} +11 -7
- package/dist/{te-analysis-OX62G5VD.js → te-analysis-USIK6ZZG.js} +185 -33
- package/dist/{te-community-QN3WMPNT.js → te-community-WE2UU4XV.js} +720 -228
- package/dist/{te-dataops-UKD2XN3E.js → te-dataops-XNMO7MBF.js} +6 -5
- package/dist/{te-engage-X3DIXPRA.js → te-engage-6C3HCFQR.js} +13 -12
- package/dist/{te-experiment-MEIIK3DP.js → te-experiment-LVTRI3QK.js} +55 -21
- package/dist/{te-kb-73EWDFDF.js → te-kb-TUPGMVBY.js} +409 -85
- package/dist/{te-system-IE7NCKC3.js → te-system-YM657FKO.js} +6 -6
- package/dist/{te-team-MGQ5CNRW.js → te-team-G4VEJJNL.js} +7 -6
- package/dist/{things-4JIBCA55.js → things-22KQWC23.js} +7 -7
- package/dist/{update-AX66IFMJ.js → update-XX4ILU7X.js} +11 -7
- package/dist/{workspace-Q5NMEKEN.js → workspace-LYDAP647.js} +6 -6
- package/package.json +17 -7
- package/skills/ae-agent/references/resource-package.md +4 -0
- package/skills/ae-analysis/SKILL.md +14 -10
- package/skills/ae-analysis/references/agent_review_preflight.md +52 -11
- package/skills/ae-analysis/references/agent_review_priorities_comparisons.md +5 -3
- package/skills/ae-analysis/references/agent_review_submit_to_page.md +57 -25
- package/skills/ae-analysis/references/command_index.md +7 -3
- package/skills/ae-analysis/references/governance_recommendation_export.md +31 -17
- package/skills/ae-analysis/references/governance_recommendation_run_update.md +13 -0
- package/skills/ae-analysis/references/project_semantic_knowledge_wiki.md +9 -1
- package/skills/ae-analysis/references/scenario_append.md +6 -0
- package/skills/ae-analysis/references/scenario_begin.md +11 -0
- package/skills/ae-analysis/references/scenario_collect.md +10 -2
- package/skills/ae-analysis/references/scenario_evaluation.md +6 -18
- package/skills/ae-analysis/references/scenario_get.md +15 -0
- package/skills/ae-analysis/references/scenario_list.md +64 -0
- package/skills/ae-analysis/scripts/project-semantic-knowledge-wiki/company-kb.mjs +14 -1
- package/skills/ae-community/SKILL.md +77 -27
- package/skills/ae-community/references/community-analyzing-theme-comment.md +10 -8
- package/skills/ae-community/references/community-data-quality.md +68 -0
- package/skills/ae-community/references/community-query-evidence.md +85 -0
- package/skills/ae-community/references/community-scenario-capabilities.md +137 -0
- package/skills/ae-community/references/community-weekly-report.md +6 -3
- package/skills/ae-community/references/get_channel_info.md +7 -4
- package/skills/ae-community/references/get_comment_tag_analysis.md +3 -3
- package/skills/ae-community/references/get_comments_summary.md +4 -4
- package/skills/ae-community/references/get_corpus_tags.md +3 -3
- package/skills/ae-community/references/get_daily_summary.md +3 -3
- package/skills/ae-community/references/get_hot_topics.md +3 -3
- package/skills/ae-community/references/get_livestream_analysis.md +3 -3
- package/skills/ae-community/references/get_livestream_detail.md +4 -4
- package/skills/ae-community/references/get_livestream_list.md +3 -3
- package/skills/ae-community/references/get_livestream_overview.md +3 -3
- package/skills/ae-community/references/get_livestream_room_metrics.md +3 -3
- package/skills/ae-community/references/get_livestream_rooms.md +3 -3
- package/skills/ae-community/references/get_overview_metrics.md +3 -3
- package/skills/ae-community/references/get_post_detail.md +4 -4
- package/skills/ae-community/references/get_risk_content.md +5 -5
- package/skills/ae-community/references/get_sentiment_overview.md +4 -4
- package/skills/ae-community/references/get_tag_trends.md +3 -3
- package/skills/ae-community/references/get_topic_detail.md +3 -3
- package/skills/ae-community/references/list_sources.md +28 -0
- package/skills/ae-community/references/list_spaces.md +39 -0
- package/skills/ae-community/references/search_posts.md +21 -7
- package/skills/ae-data-integration/SKILL.md +6 -4
- package/skills/ae-data-integration/references/error-handling.md +1 -0
- package/skills/ae-data-integration/references/handoff.md +1 -1
- package/skills/ae-data-integration/references/sink-upload.md +11 -0
- package/skills/ae-data-integration/references/source-inspect.md +2 -1
- package/skills/ae-data-integration/references/transform.md +1 -1
- package/skills/ae-data-integration/references/ue-mapping.md +1 -1
- package/skills/ae-experiment/SKILL.md +7 -1
- package/skills/ae-experiment/references/manage_experiment.md +18 -1
- package/skills/ae-experiment/references/query_experiment_detail.md +7 -0
- package/skills/ae-experiment/references/save_experiment.md +76 -2
- package/skills/ae-experiment/references/save_submit_experiment.md +7 -0
- package/skills/ae-kb/SKILL.md +33 -6
- package/skills/ae-kb/references/publication.md +81 -0
- package/skills/ae-kb/references/query-workflow.md +14 -1
- package/skills/ae-metadata/SKILL.md +2 -2
- package/skills/ae-metadata/references/metadata_data_table_get.md +35 -0
- package/dist/chunk-ELCR235M.js +0 -1429
|
@@ -9,8 +9,8 @@ Mapping command: `ae-cli community +search_posts`
|
|
|
9
9
|
|
|
10
10
|
| Flag | Type | Required | Description |
|
|
11
11
|
|------|------|------|------|
|
|
12
|
-
| `--space-id` |
|
|
13
|
-
| `--game-id` |
|
|
12
|
+
| `--space-id` | int64 | yes | Community space ID returned by `community spaces list` |
|
|
13
|
+
| `--game-id` | int64 | no | Deprecated compatibility alias; omit it, or make it equal to `--space-id` |
|
|
14
14
|
| `--channel-id-list` | string | No | List of channel IDs, comma separated |
|
|
15
15
|
| `--keywords` | string | No | List of keywords, comma separated |
|
|
16
16
|
| `--start-time` | string | Yes | Minimum content post time, format yyyy-MM-dd |
|
|
@@ -20,6 +20,8 @@ Mapping command: `ae-cli community +search_posts`
|
|
|
20
20
|
| `--search-mode` | number | No | Search mode: 0 word (default), 1 exact match, 2 author names |
|
|
21
21
|
| `--search-word` | string | No | Full-text search term, in conjunction with searchMode |
|
|
22
22
|
| `--sentiment-types` | string | No | Emotion type: 0 negative, 1 positive, 2 neutral, comma separated |
|
|
23
|
+
| `--topic-id-list` | string | No | Topic IDs returned by `+get_hot_topics`, comma separated |
|
|
24
|
+
| `--tag-filters` | JSON | No | Corpus tag filter array using `tagName`, `tagCode`, `inferenceType`, `calcuSymbol`, and optional `ftv` |
|
|
23
25
|
| `--order-by` | number | No | Sort: 0 time desc, 1 weight desc, 2 time asc, 3 replies desc, 4 heat desc |
|
|
24
26
|
| `--page-num` | number | No | Page from 1 |
|
|
25
27
|
| `--page-size` | number | No | Number of bars per page |
|
|
@@ -27,21 +29,33 @@ Mapping command: `ae-cli community +search_posts`
|
|
|
27
29
|
Examples
|
|
28
30
|
|
|
29
31
|
```bash
|
|
30
|
-
Basic search
|
|
32
|
+
# Basic search
|
|
31
33
|
ae-cli community +search_posts \
|
|
32
|
-
--space-id 1
|
|
34
|
+
--space-id 1 \
|
|
33
35
|
--start-time 2024-01-01 --end-time 2024-01-07
|
|
34
36
|
|
|
35
37
|
# Filter by channel and sentiment
|
|
36
38
|
ae-cli community +search_posts \
|
|
37
|
-
--space-id 1
|
|
39
|
+
--space-id 1 \
|
|
38
40
|
--start-time 2024-01-01 --end-time 2024-01-07 \
|
|
39
41
|
--channel-id-list 1,2 \
|
|
40
42
|
--sentiment-types 0,1
|
|
41
43
|
|
|
42
|
-
Pagination
|
|
44
|
+
# Pagination query
|
|
43
45
|
ae-cli community +search_posts \
|
|
44
|
-
--space-id 1
|
|
46
|
+
--space-id 1 \
|
|
45
47
|
--start-time 2024-01-01 --end-time 2024-01-07 \
|
|
46
48
|
--page-num 1 --page-size 20
|
|
49
|
+
|
|
50
|
+
# Filter by topic and corpus tag
|
|
51
|
+
ae-cli community +search_posts \
|
|
52
|
+
--space-id 1 \
|
|
53
|
+
--start-time 2024-01-01 --end-time 2024-01-07 \
|
|
54
|
+
--topic-id-list topic-1,topic-2 \
|
|
55
|
+
--tag-filters '[{"tagName":"Character","tagCode":"hero","inferenceType":"label","calcuSymbol":"C04"}]'
|
|
47
56
|
```
|
|
57
|
+
|
|
58
|
+
`inferenceType` must be `label` or `score` and should be copied from `+get_corpus_tags`.
|
|
59
|
+
Label filters support `C04` (has value) and `C05` (no value). Score filters support
|
|
60
|
+
`C00`, `C01`, `C02`, `C020`, `C03`, `C030`, `C04`, `C05`, and `C06`; comparisons other
|
|
61
|
+
than `C04`/`C05` require an `ftv` JSON array, and `C06` requires at least two values.
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: ae-data-integration
|
|
3
|
-
description: "Bring local CSV, TSV, TXT, JSON, JSONL (NDJSON), XLS, and
|
|
3
|
+
description: "Bring local CSV, TSV, TXT, JSON, JSONL (NDJSON), XLS, XLSX, and Parquet files — and Feishu sheet / Bitable (多维表格) URLs — into AE end-to-end: identify the source's business meaning, generate and confirm a tracking plan, transform rows into UE records, and upload. Also supports privacy-preserving local analysis and handing a small file to AE Agent. Use whenever a user wants to import offline/local data or a Feishu sheet / Bitable into AE, or analyze a file without uploading it. Trigger words: 本地数据导入 / 离线数据 / 数据文件 / 文件导入 / 文件上报 / CSV 导入 / Excel 导入 / TSV 导入 / JSON 导入 / Parquet 导入 / 飞书表格导入 / 在线表格导入 / 飞书多维表格导入 / 多维表格导入 / 在线多维表格导入 / 导入到 AE / 导入到 ThinkingAI / local data import / import local file / parquet import / import Feishu sheet / import Feishu Bitable."
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# AE Data Integration
|
|
@@ -11,7 +11,7 @@ When remaining work is outside this skill's scope, or a necessary prerequisite n
|
|
|
11
11
|
|
|
12
12
|
Turn local/offline files and Feishu sheet / Bitable (多维表格) URLs into AE data through one fixed pipeline of four submodules: **Source → Tracking plan → Transform → Sink**. A file or sheet is never uploaded merely because it is present: its business meaning is understood, confirmed once by a human, and only then ingested. The tracking plan is generated and confirmed **before** ingestion (data governance shift-left) — see [references/tracking-plan.md](references/tracking-plan.md).
|
|
13
13
|
|
|
14
|
-
Two entrances lead here: the AE Agent dialog (attach / plus-button upload) and `ae-cli`. Two sink paths exist: RESTful
|
|
14
|
+
Two entrances lead here: the AE Agent dialog (attach / plus-button upload) and `ae-cli`. Two sink paths exist: RESTful `/sync_json` upload for one-time/small loads, and file / LogBus2 delivery (convert-only — writes `valid.ue.jsonl` to a watched directory, never uploads) for recurring/high-volume loads. Which path applies is always the user's choice, never a default. Source and Sink are pluggable — adding one does not change the main pipeline.
|
|
15
15
|
|
|
16
16
|
## Mandatory safety rules
|
|
17
17
|
|
|
@@ -31,7 +31,7 @@ Two entrances lead here: the AE Agent dialog (attach / plus-button upload) and `
|
|
|
31
31
|
|
|
32
32
|
## When to use / When NOT to use
|
|
33
33
|
|
|
34
|
-
Use this skill when the user wants to bring a **local data file** (CSV/TSV/TXT/JSON/JSONL/XLS/XLSX) or a **Feishu sheet / Bitable (多维表格) URL** into AE, or analyze a local file without uploading.
|
|
34
|
+
Use this skill when the user wants to bring a **local data file** (CSV/TSV/TXT/JSON/JSONL/XLS/XLSX/Parquet) or a **Feishu sheet / Bitable (多维表格) URL** into AE, or analyze a local file without uploading.
|
|
35
35
|
|
|
36
36
|
| User intent | Use this instead |
|
|
37
37
|
| --- | --- |
|
|
@@ -53,7 +53,9 @@ Walk the four submodules in order. Each submodule is its own reference; follow i
|
|
|
53
53
|
2. **Reuse check.** If the profile is `ue_eligible`, read [references/reuse.md](references/reuse.md) and match the recommended mapping against the handoff index. `reuse` searches the current directory's `.ae-cli/data-integration/` upward, then `~/.ae-cli/data-integration/`, so a package written elsewhere is still found. A match proposes a frozen package; after one explicit confirmation, run the returned `transform.mjs` command and jump to Sink (step 5). No match → continue.
|
|
54
54
|
3. **Tracking plan.** Read [references/tracking-plan.md](references/tracking-plan.md). The plan is generated from the mapping (`plan --mapping`), so confirm the recommended mapping's key system fields with the user first — `mode`, `#account_id`/`#distinct_id`, `#time` + timezone, `#event_name`, `#ip`/`#uuid` (see [references/transform.md](references/transform.md) steps 1–5) — then generate the event/property plan and get a single explicit confirmation from the user before touching data. The plan is a separate, required deliverable from the transform mapping: a user who supplies a column→field mapping directly has **not** completed this step, so build the plan from the confirmed mapping anyway. `user_set` still requires a plan (no events; every property becomes a user property). This step runs for **every** file: a second or later file merges its new events and properties into the existing project plan (tracking-plan.md step 4) — an existing plan is never a reason to skip it.
|
|
55
55
|
4. **Transform.** Read [references/transform.md](references/transform.md). Map columns to AE system fields and properties, convert, and quarantine dirty rows per [references/ue-mapping.md](references/ue-mapping.md).
|
|
56
|
-
5. **Sink —
|
|
56
|
+
5. **Sink — choose the path first, then act.** Read [references/sink-upload.md](references/sink-upload.md). Before resolving any destination, ask the user which sink path this data takes — never default one silently (the user handing you an endpoint/APPID is not a sink decision):
|
|
57
|
+
- **RESTful `/sync_json` upload** — one-time / small loads. Resolve the destination, dry-run, confirm, then upload per [references/sync-json-upload.md](references/sync-json-upload.md). `receiver_accepted` is not persistence: after a ~1-minute ingestion delay, verify the data landed with ae-cli (`tracking live-data list` / `tracking ingest summary` / `tracking ingest-error list`) rather than telling the user to check the console.
|
|
58
|
+
- **File / LogBus2 mode — JSONL only, never uploads** — recurring / high-volume loads (e.g. daily parquet exports). Run `convert` (or the handoff `bin/run.sh`) so `valid.ue.jsonl` lands in the directory LogBus2 watches — never `upload.sh`, which posts to the RESTful receiver. Use the output directory confirmed in step 4. The same rows must not go through both paths: RESTful upload plus LogBus2 pickup double-ingests.
|
|
57
59
|
6. **Handoff.** Read [references/handoff.md](references/handoff.md). Export the reusable package (pipeline descriptor + frozen mappings + stage executors + docs) and a shareable zip; in the completion response, state the absolute zip path, the package directory, and the one-line way to run the next same-shape file.
|
|
58
60
|
|
|
59
61
|
## Error handling
|
|
@@ -50,6 +50,7 @@ the user decide — never to guess an encoding or structure and retry silently.
|
|
|
50
50
|
| `LOCAL_DATA_INPUT_NOT_FOUND` | Path is not a readable file | Ask for the correct path |
|
|
51
51
|
| `LOCAL_DATA_FILE_TOO_LARGE` | XLS over 1 GB | Convert to XLSX or split the workbook |
|
|
52
52
|
| `LOCAL_DATA_INPUT_INVALID` | Generic parse failure (malformed CSV/TSV/JSON/XLS) | Verify encoding and structure, then retry without changing the source |
|
|
53
|
+
| `LOCAL_DATA_PARQUET_INVALID` | File cannot be read as Parquet | Verify the file is a valid Parquet file and retry without changing the source |
|
|
53
54
|
| `LOCAL_DATA_JSONL_INVALID` | A JSONL line is not valid JSON (`location.record` names the line) | Point at the offending record |
|
|
54
55
|
| `LOCAL_DATA_JSON_ROOT_INVALID` | JSON root is not an object or array | Check the file's top-level shape |
|
|
55
56
|
| `LOCAL_DATA_XLSX_INVALID` | Workbook metadata missing / no readable sheets / worksheet entry missing | Re-export the workbook (sheet recognition accepts `<sheet>` and namespaced `<x:sheet>` alike, so this is a real metadata gap) |
|
|
@@ -48,7 +48,7 @@ A shareable archive is written next to the package root: `<parent>/ae-data-integ
|
|
|
48
48
|
|
|
49
49
|
## Pipeline descriptor
|
|
50
50
|
|
|
51
|
-
`pipeline.json` declares the three stages and their types. Only `source: local_file` and `sink: restful_sync_json` are implemented today; the `type` fields reserve logbus / datax / mysql for later phases. `bin/run.sh` and `bin/upload.sh` read the descriptor and dispatch each stage by its `type` to `ae-cli data-integration inspect / convert / upload` — they are executors, not a second runtime engine.
|
|
51
|
+
`pipeline.json` declares the three stages and their types. Only `source: local_file` and `sink: restful_sync_json` are implemented today; the `type` fields reserve logbus / datax / mysql for later phases. `bin/run.sh` and `bin/upload.sh` read the descriptor and dispatch each stage by its `type` to `ae-cli data-integration inspect / convert / upload` — they are executors, not a second runtime engine. A Parquet source runs the same `local_file` path: `bin/run.sh` converts it to `valid.ue.jsonl` on disk. When the destination is LogBus2 (or any file-watch sink), hand the runs directory to it and **never run `bin/upload.sh`**, which posts to the RESTful receiver.
|
|
52
52
|
|
|
53
53
|
## Recorded destination
|
|
54
54
|
|
|
@@ -1,5 +1,16 @@
|
|
|
1
1
|
# Sink — resolve destination and upload
|
|
2
2
|
|
|
3
|
+
## Choose the sink path first
|
|
4
|
+
|
|
5
|
+
Two sinks exist, and the choice is the user's — never a default:
|
|
6
|
+
|
|
7
|
+
| Sink | When | Deliverable |
|
|
8
|
+
| --- | --- | --- |
|
|
9
|
+
| RESTful `/sync_json` | one-time / small load | uploads rows to the receiver directly |
|
|
10
|
+
| File / LogBus2 | recurring / high-volume (daily/weekly exports) | `valid.ue.jsonl` written to a watched directory; **never uploads** |
|
|
11
|
+
|
|
12
|
+
Ask the user which one applies before resolving any destination. The user supplying an endpoint/APPID does not settle it — a file that will recur usually belongs on LogBus2, and the same rows must not go through both paths. **File / LogBus2 mode skips the rest of this file:** run `convert` with `--output-dir` pointed at the directory LogBus2 watches, then stop — no endpoint, no APPID, no `upload`.
|
|
13
|
+
|
|
3
14
|
## Resolve the destination
|
|
4
15
|
|
|
5
16
|
Use the current AE CLI project commands; never guess IDs:
|
|
@@ -8,7 +8,7 @@ Confirm:
|
|
|
8
8
|
2. The desired outcome: AE ingestion, local analysis, or help deciding.
|
|
9
9
|
3. For ingestion, the target environment/project if already known.
|
|
10
10
|
|
|
11
|
-
Accept one or more CSV, TSV, TXT, JSON, JSONL (NDJSON), XLS, or
|
|
11
|
+
Accept one or more CSV, TSV, TXT, JSON, JSONL (NDJSON), XLS, XLSX, or Parquet files. CSV/TSV/TXT/JSON/JSONL/XLSX/Parquet have no hard size limit; files over 1 GB print a stderr warning with an estimated processing time and suggest splitting. XLS over 100 MB prints a memory-risk warning (the legacy parser loads the whole workbook, roughly 5-10x file size); XLS over 1 GB is still rejected — convert it to XLSX or split it first.
|
|
12
12
|
|
|
13
13
|
XLSX worksheet recognition matches `<sheet>` and namespaced `<x:sheet>` alike: some cleaning/export tools rewrite the default OOXML namespace as a prefix, and the workbook reads the same either way. The worksheet *row* nodes themselves are still read unprefixed, so a workbook whose row data is also namespaced falls back to CSV.
|
|
14
14
|
|
|
@@ -59,6 +59,7 @@ Summarize both — report the shape of the distribution and the names of the val
|
|
|
59
59
|
- **Summary and total rows** — an exported report ends with a 合计 row, and a grouped one repeats 小计 after every group. Those rows are not observations: uploaded, each becomes an event that never happened whose amount is the whole group's revenue, and profiled, it doubles the column's `sum` and turns its `max` into the total. Nothing in the row itself says so, so inspect flags them under `summary_rows` and warns. Each entry carries `row` (the data-row ordinal, the same numbering `invalid.rows.jsonl` and `--salvage-from` use) and `signals`: `total_label` when a cell reads as a total label (`合计` / `总计` / `小计` / `汇总` as a prefix, `Total` / `Subtotal` / `Sum` as the whole cell), naming the column in `label_column` but never the cell text; `column_total` when a number on that row equals the total of its column's other rows, listing those columns in `total_columns`. A row can raise one signal or both — a labelled group subtotal holds its group's total, not the column's, so only the label fires. Nothing is removed and nothing is changed: `convert` writes these rows as records, and every number reported for their columns counts them, which is exactly why the finding has to be read. There is no flag that drops a data row, because a row labelled `合计` is sometimes a real business record; when the user confirms a row is a total, ask them to remove it from the source file or re-export without it, then inspect again. `convert` repeats the finding in `manifest.output.summary_rows`, since by upload time a subtotal row with a plausible identity and time is indistinguishable from data. This covers every format, `.xls` included — the check reads rows, not worksheet structure.
|
|
60
60
|
- **Repeated business keys** — a customer re-exports a report whose range overlaps the last export, or pastes two sheets together, and the same observation arrives twice. Uploaded, each repeat is a second event: that user's revenue doubles, every funnel counts them twice, and AE appends accepted events with no way to un-send one — so the only place this is fixable is before the upload. Inspect compares each row's business key against the rows before it and reports repeats under `duplicate_keys`, plus a warning. The report carries `key_columns` (the columns actually compared — always read it, since the key is what the finding means), `checked_rows`, `duplicate_groups`, `extra_rows` (surplus records an upload would carry), and `groups`, each with `count`, the data-row `rows` (the numbering `invalid.rows.jsonl` and `--salvage-from` use), and a `key_hash` prefix that distinguishes groups without revealing values — the key's own text is never reported. `groups_truncated` / `rows_truncated` mean the list is bounded, not that the counts are; `tracking_truncated` means distinct keys outran the scan's budget and there may be more. The key comes from the mapping's identity, time, and event-name columns on `convert`, and from column-name matching on `inspect`; a single column is never a key, so a file with no recognizable time column is not scanned at all (identity alone would call every returning user's second row a repeat). Two limits to state when reporting: values are compared as written, so `2026-03-01 10:00:00` and `2026/03/01 10:00:00` are two different keys and a repeat spelled two ways is missed; and a source's own unique key (an order id) is not compared on unless the mapping names it as identity, time, or event. Nothing is removed — a repeat is sometimes a real pair of records, two order lines in the same checkout second — so ask the user whether the rows are separate observations, and if they are not, have them remove the rows from the source file and inspect again. `convert` repeats the finding in `manifest.output.duplicate_keys`, because by upload time both copies are ordinary valid records; it describes the whole source file even on a `--salvage-from` run, since every valid row of that file ends up in AE. This covers every format, `.xls` included — the check reads rows, not worksheet structure.
|
|
61
61
|
- **TSV / TXT** — `.tsv` and `.tab` use a tab delimiter with no quoting convention; `.txt` and unknown extensions are content-sniffed into CSV, TSV, or NDJSON.
|
|
62
|
+
- **Parquet** — `.parquet` / `.pq` files read through the schema (column names come from the schema, there is no header row), so header/title-row handling never applies. 64-bit integer columns arrive as decimal strings (values beyond 2^53 cannot be a JS number), decimals are rendered with their scale, and timestamp columns follow the file's own semantics: a UTC-adjusted timestamp (`isAdjustedToUTC=true`) is an absolute instant, a naive one (`isAdjustedToUTC=false`, e.g. Trino/Spark wall clocks) is read as a wall-clock string that `source_timezone` then applies. Nested list/struct/map columns are kept nested (inferred `list`/`object`), not flattened. **Directories are not accepted** — `--input-file` takes file paths only, so a partitioned dataset (a directory of `part-*.parquet`) must be expanded into its `.parquet` file list first, then passed as repeated `--input-file` values in a stable order (sort them) so row order is deterministic. **Many files, one profile:** when every input is Parquet and there is more than one file, inspect fully profiles only the first file and verifies the rest against it by footer (column names + Arrow types), returning a `schema_checked` shape with the representative profile plus each file's `row_count` / `size_bytes` — schemas are confirmed identical, not values, so a UTF8 column whose values look numeric/datetime/JSON can still infer differently in another file (rare for one export pipeline; a schema mismatch falls back to profiling every file). **Size:** a Parquet file over 1 GB warns about memory (the reader decodes the whole file), so large files should be split by row group or partition.
|
|
62
63
|
- **Encoding** — text files are auto-detected (UTF-8, GBK, GB2312, Big5, and others); no flag is needed.
|
|
63
64
|
- **Excel date cells** — a cell whose number format is a date or date+time is read as the wall-clock timestamp shown in Excel, not as the Excel serial number stored behind it, so the column infers as `datetime` and can serve as the time field. Inspect lists every such column in a warning. Treat that warning as a question to the user, not as a note: the same column profiled as `number` before this behavior existed, so if any part of this file was already sent to AE, the property may have been received as a number and its type is now locked — it cannot be changed to datetime, and the column has to be re-sent under a new property name. Ask whether the column was uploaded before, and only map it once the user answers. Elapsed-duration formats (`[h]:mm:ss` and the equivalent built-ins) are durations rather than points in time and stay `number`.
|
|
64
65
|
- **Excel formula cells** — a spreadsheet stores a formula and, next to it, the result Excel last computed. That cached result is the value: it is read normally, including a result of `0` or `""`, which are real values and not blanks. This tool never evaluates a formula and never guesses a result, so a cell holding a formula the file never computed has nothing to upload; it is read as missing and counted, as is an Excel error value (`#N/A`, `#DIV/0!`, …). Inspect reports the counts per column in a warning and `convert` repeats them in `manifest.output.unreadable_cells`. Report them: the rows are kept and the record count is unchanged, so this is the only explanation for a column that is empty in AE while the spreadsheet looks full. When a column that matters reads as missing this way, ask the user to recalculate and re-export in Excel, or to export values instead of formulas, before uploading. This covers XLSX; a legacy `.xls` workbook goes through a different parser and is not counted here.
|
|
@@ -29,7 +29,7 @@ Walk the table item by item:
|
|
|
29
29
|
|
|
30
30
|
Before saving, present the complete mapping for a final sign-off on one grouped page — system fields (`#type`/mode, `#account_id`, `#distinct_id`, `#time` + source timezone, `#zone_offset`, `#event_name`, `#ip`/`#uuid`), then the property table (source column → target AE name → type), then excluded columns — and wait for an explicit yes. Never dump the raw mapping JSON at the user.
|
|
31
31
|
|
|
32
|
-
Save the reviewed `ae-data-integration-mapping/v1` JSON to `.ae-cli/data-integration/mapping.json` (project workspace). Convert into a new output directory:
|
|
32
|
+
Save the reviewed `ae-data-integration-mapping/v1` JSON to `.ae-cli/data-integration/mapping.json` (project workspace). Ask the user where the output should land before converting — never pick it silently. Offer the default `.ae-cli/data-integration/runs/<run-id>`; for file / LogBus2 delivery the `--output-dir` must be the directory LogBus2 watches. Convert into a new output directory:
|
|
33
33
|
|
|
34
34
|
```bash
|
|
35
35
|
ae-cli data-integration convert \
|
|
@@ -27,7 +27,7 @@ Required structure:
|
|
|
27
27
|
}
|
|
28
28
|
```
|
|
29
29
|
|
|
30
|
-
Use `distinct_id_field` when no account field exists. `mixed` requires `record_type_field`. Track requires `event_name_field` or a reviewed `default_event_name`. `source.format` is one of `csv`, `tsv`, `json`, `jsonl`, `xls`, `xlsx`. For a multi-file template mapping, `source.sha256` may be the wildcard `*`.
|
|
30
|
+
Use `distinct_id_field` when no account field exists. `mixed` requires `record_type_field`. Track requires `event_name_field` or a reviewed `default_event_name`. `source.format` is one of `csv`, `tsv`, `json`, `jsonl`, `xls`, `xlsx`, `parquet`. For a multi-file template mapping, `source.sha256` may be the wildcard `*`.
|
|
31
31
|
|
|
32
32
|
## Record types
|
|
33
33
|
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: ae-experiment
|
|
3
|
-
version: 1.0.
|
|
3
|
+
version: 1.0.5
|
|
4
4
|
description: "Use when managing Atlas AB or external experiments, traffic layers, Features, metrics, buckets, and experiment reports through ae-cli"
|
|
5
5
|
---
|
|
6
6
|
|
|
@@ -65,6 +65,12 @@ Naming and response boundary:
|
|
|
65
65
|
5. Check readiness with `experiment experiment ready-check`.
|
|
66
66
|
6. For a non-mutex traffic layer, run `experiment experiment conflict-check` before submit (needs `feature_key_list` from context or `experiment get`).
|
|
67
67
|
7. Move status with `experiment experiment manage`.
|
|
68
|
+
For a scheduled Feature start, save `startTime` (or use `--start-time`), then
|
|
69
|
+
submit with `--target-status pending`. Saving a draft does not submit it; `running`
|
|
70
|
+
starts immediately. Read [Feature timing](references/save_experiment.md#feature-timing)
|
|
71
|
+
before configuring a schedule or starting it early.
|
|
72
|
+
To resume `paused` to `running`, call `manage` directly and verify the status
|
|
73
|
+
with `get`; do not use the pre-launch `ready-check` as a resume prerequisite.
|
|
68
74
|
8. Query reports with `experiment report summary`, `experiment report sample-size`, and `experiment report metric-trend`.
|
|
69
75
|
|
|
70
76
|
If an experiment save returns `error_code: METRIC_NOT_FOUND`, list metrics for the same project. Create and verify the metric before retrying; never retry with another invented ID. Metric deletion returns `error_code: METRIC_IN_USE` while an active experiment binding exists.
|
|
@@ -12,6 +12,23 @@ Flags:
|
|
|
12
12
|
- `--target-status`: One of `draft`, `testing`, `pending`, `running`, `paused`, `ended`, `archive`.
|
|
13
13
|
- `--remark`: Optional operation remark.
|
|
14
14
|
|
|
15
|
-
When target status is `pending`, `testing`, or `running`, Hermes performs a ready
|
|
15
|
+
When target status is `pending`, `testing`, or `running`, Hermes performs a ready
|
|
16
|
+
check before changing status, except when resuming `paused` to `running`.
|
|
17
|
+
Resume uses the shared status transition service, which checks permissions,
|
|
18
|
+
deadline, and audience readiness and re-enables the Feature rules and RCC sync.
|
|
19
|
+
It preserves the first start time and the existing day-based or fixed deadline.
|
|
20
|
+
Do not run the standalone `ready-check` as a prerequisite for resuming a paused
|
|
21
|
+
experiment; that command accepts only pre-launch states.
|
|
22
|
+
|
|
23
|
+
For Feature scheduling, save `startTime` first and use `pending` to submit. The
|
|
24
|
+
server checks submitted experiments every half hour. `running` starts immediately;
|
|
25
|
+
read fresh detail and disclose `start_time_adjustment_required=true` before an
|
|
26
|
+
early manual start. No extra confirmation field is required. See
|
|
27
|
+
[Feature timing](save_experiment.md#feature-timing) for lifecycle and deadline rules.
|
|
16
28
|
|
|
17
29
|
Response shape: the operation or blocking readiness result is in `data.result`, with object keys recursively converted to snake_case.
|
|
30
|
+
|
|
31
|
+
An outer `ok=true` does not mean the status changed when `data.result.valid=false`.
|
|
32
|
+
Report the returned issues and read `experiment experiment get` to confirm the
|
|
33
|
+
actual status. A successful resume normally returns `data.result=true`; do not
|
|
34
|
+
report the experiment as running until an independent read confirms it.
|
|
@@ -12,6 +12,13 @@ Flags:
|
|
|
12
12
|
|
|
13
13
|
Response shape: the experiment is in `data.item`, with recursively snake_case keys such as `exp_id` and `feature_key_list`.
|
|
14
14
|
|
|
15
|
+
Feature timing is exposed as `start_time`, `end_time`, and `exp_cycle` (including
|
|
16
|
+
`cycle_type`, `fixed_end_time`, and planned `end_time`). The top-level `end_time`
|
|
17
|
+
records actual completion. Check `scheduled_start_error` for an expired pending
|
|
18
|
+
window and `start_time_adjustment_required` before starting a scheduled experiment
|
|
19
|
+
early. Use returned `status` to verify execution; a successful save or submission
|
|
20
|
+
does not mean the experiment is running.
|
|
21
|
+
|
|
15
22
|
When targeting contains reversible QP, it is returned as
|
|
16
23
|
`targeting.definition_request` with `targeting.definition_status=AVAILABLE`.
|
|
17
24
|
An absent definition is `NOT_APPLICABLE`; a historical unsupported definition is
|
|
@@ -3,12 +3,14 @@
|
|
|
3
3
|
Progressively create or patch an experiment draft.
|
|
4
4
|
|
|
5
5
|
```bash
|
|
6
|
-
ae-cli experiment experiment save --project-id <id> --req '<json>'
|
|
6
|
+
ae-cli experiment experiment save --project-id <id> --req '<json>' [--start-time <timestamp>] [--fixed-end-time <timestamp>]
|
|
7
7
|
```
|
|
8
8
|
|
|
9
9
|
Flags:
|
|
10
10
|
- `--project-id`, `-p`: Project ID.
|
|
11
11
|
- `--req`: Experiment save request JSON object.
|
|
12
|
+
- `--start-time`: Optional Feature start timestamp in ISO-8601 format with timezone offset. Overrides `req.startTime`.
|
|
13
|
+
- `--fixed-end-time`: Optional Feature deadline in the same format. Sets `req.expCycle.cycleType=time` and overrides `req.expCycle.fixedEndTime`, preserving other supplied cycle fields. These timing flags are for Feature experiments; external experiments use `external-experiment save-submit`.
|
|
12
14
|
|
|
13
15
|
## Contract
|
|
14
16
|
|
|
@@ -64,6 +66,7 @@ Optional while drafting:
|
|
|
64
66
|
- `isExpSharing`
|
|
65
67
|
- `bucketId`
|
|
66
68
|
- `expCycle`
|
|
69
|
+
- `startTime`
|
|
67
70
|
- `groups`
|
|
68
71
|
- `featureKeyList`
|
|
69
72
|
- `metrics`
|
|
@@ -101,7 +104,7 @@ ae-cli experiment experiment save --project-id 1 --req '{"expId":"exp_123","allo
|
|
|
101
104
|
- `isExpSharing=0` uses exclusive traffic with `bucketId`; select the bucket with `experiment bucket list`.
|
|
102
105
|
- `allocation` (experiment-level) is the experiment's share of the traffic layer. It is optional during draft construction, but before readiness it must be greater than `0` and no more than `100`. **Integer only** (for example `20`); decimals such as `20.5` are rejected at the CLI interface.
|
|
103
106
|
- `groups[].allocation` (group-level): **integer only; sum must equal `100` exactly**. Example for 3 groups: `34 + 33 + 33 = 100`. Decimals such as `33.33` are rejected at the CLI interface before save.
|
|
104
|
-
- `expCycle.cycleType` can be `day` or `
|
|
107
|
+
- `expCycle.cycleType` can be `day`, `sample`, or `time`; when `cycleType=day`, `dayNum` is `1..90`. `time` requires `fixedEndTime`. Blank cycle defaults to day + 30.
|
|
105
108
|
- Before readiness, `groups` must be non-empty, contain exactly one control group (`isControl=1`), group allocations must sum to exactly `100`, and each `expGroupValue` must be a non-empty JSON string array.
|
|
106
109
|
- Feature experiments must bind `featureKeyList` before readiness. `featureKeyList` contains Feature key strings, not Feature objects. The MCP currently supports one `featureKey` for feature experiments.
|
|
107
110
|
- Metrics use `metricRole=primary|secondary|guardrail`; the internal `observation` role is currently unavailable for saves. Before readiness at least one primary metric is required.
|
|
@@ -142,6 +145,61 @@ Semantic audience example:
|
|
|
142
145
|
}
|
|
143
146
|
```
|
|
144
147
|
|
|
148
|
+
## Feature Timing
|
|
149
|
+
|
|
150
|
+
Feature start time is optional. Use ISO-8601 timestamps with an explicit offset,
|
|
151
|
+
for example `2026-11-01T16:00:00+08:00`. Prefer minutes `00` or `30` to align
|
|
152
|
+
with the server's half-hour scheduling checks. The CLI forwards timestamps;
|
|
153
|
+
the server clears seconds and milliseconds on configured Feature start times.
|
|
154
|
+
|
|
155
|
+
Save timing on a new draft or patch an existing experiment:
|
|
156
|
+
|
|
157
|
+
```bash
|
|
158
|
+
ae-cli experiment experiment save --project-id 1 --req '{"expName":"Scheduled CTA"}' --start-time '2026-11-01T16:00:00+08:00' --fixed-end-time '2026-11-08T16:00:00+08:00' --dry-run
|
|
159
|
+
ae-cli experiment experiment save --project-id 1 --req '{"expId":"exp_123"}' --start-time '2026-11-01T16:00:00+08:00' --fixed-end-time '2026-11-08T16:00:00+08:00'
|
|
160
|
+
```
|
|
161
|
+
|
|
162
|
+
The second command produces the same native request as:
|
|
163
|
+
|
|
164
|
+
```json
|
|
165
|
+
{"expId":"exp_123","startTime":"2026-11-01T16:00:00+08:00","expCycle":{"cycleType":"time","fixedEndTime":"2026-11-08T16:00:00+08:00"}}
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
Saving does not submit or start the experiment. Complete the remaining Feature,
|
|
169
|
+
group, traffic, and metric configuration, check readiness, and check traffic
|
|
170
|
+
conflicts for a non-mutex layer before submitting:
|
|
171
|
+
|
|
172
|
+
```bash
|
|
173
|
+
ae-cli experiment experiment ready-check --project-id 1 --exp-id exp_123
|
|
174
|
+
ae-cli experiment experiment manage --project-id 1 --exp-id exp_123 --target-status pending
|
|
175
|
+
ae-cli experiment experiment get --project-id 1 --exp-id exp_123
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
Only submitted `pending` Feature experiments with a configured start time are
|
|
179
|
+
eligible for automatic start. A draft never starts automatically; a pending
|
|
180
|
+
experiment without a start time waits for manual start. The server checks every
|
|
181
|
+
half hour, so a successful submission is not evidence that the experiment is
|
|
182
|
+
already running. Verify `data.item.status`, `start_time`, and
|
|
183
|
+
`scheduled_start_error` with `get` (or `list`). An expired pending window remains
|
|
184
|
+
pending and reports an error; fix the timing instead of reporting a start.
|
|
185
|
+
|
|
186
|
+
Timing changes follow server lifecycle rules:
|
|
187
|
+
|
|
188
|
+
- Start time can change while draft or pending. Omission or `null` preserves a
|
|
189
|
+
saved start time; neither clears it. After the first run, start time is locked.
|
|
190
|
+
- `day` ends after start plus `dayNum` times 24 hours. `time` uses the absolute
|
|
191
|
+
`expCycle.fixedEndTime`, which must be later than start. The top-level `endTime`
|
|
192
|
+
is the actual end time, not the Feature deadline configuration.
|
|
193
|
+
- Submitted end conditions must keep their cycle type and may only be extended.
|
|
194
|
+
The server validates existing state and the effective time window.
|
|
195
|
+
- Manual `running` starts immediately. Read fresh detail first and disclose any
|
|
196
|
+
`start_time_adjustment_required=true` to the user: an early start uses current
|
|
197
|
+
server time, recalculates a day-based deadline, and keeps a fixed deadline.
|
|
198
|
+
Do not add an extra confirmation field to the request.
|
|
199
|
+
- Pause/resume keeps the original start and deadline. Running and paused
|
|
200
|
+
experiments are checked for expiry every half hour; deadlines between checks
|
|
201
|
+
are handled by a subsequent check.
|
|
202
|
+
|
|
145
203
|
## Payload Examples
|
|
146
204
|
|
|
147
205
|
Patch a Feature binding. `featureKeyList` is a string array:
|
|
@@ -168,6 +226,22 @@ Patch metrics. `metrics` is an object array:
|
|
|
168
226
|
ae-cli experiment experiment save --project-id 2 --req '{"expId":"e024","metrics":[{"metricId":"payment","metricRole":"primary"}]}'
|
|
169
227
|
```
|
|
170
228
|
|
|
229
|
+
## Timing Validation Errors
|
|
230
|
+
|
|
231
|
+
Atlas business time checks return `error.type=validation`, a specific code,
|
|
232
|
+
`error.details.field`, and a correction hint. Examples:
|
|
233
|
+
|
|
234
|
+
| Code | Field | Correction |
|
|
235
|
+
| --- | --- | --- |
|
|
236
|
+
| `EXPERIMENT_END_TIME_INVALID` | `req.expCycle.fixedEndTime` (related: `req.startTime`) | Make the deadline strictly later than the start; submission or start also requires an unexpired window. |
|
|
237
|
+
| `EXPERIMENT_END_TIME_REQUIRED` | `req.expCycle.fixedEndTime` | Supply a timestamp with timezone offset when `cycleType=time`. |
|
|
238
|
+
|
|
239
|
+
Fix the reported field before retrying. The error preserves `meta.invocation_id`
|
|
240
|
+
for investigation. These checks do not save the invalid configuration; use `get`
|
|
241
|
+
to verify the existing timing. Older backends may still return
|
|
242
|
+
`CAPABILITY_EXECUTION_FAILED`; changing CLI arguments cannot repair a missing
|
|
243
|
+
server-side exception translator.
|
|
244
|
+
|
|
171
245
|
## Dry-run Limitation
|
|
172
246
|
|
|
173
247
|
`--dry-run` calls the Capability Gateway preview endpoint. It validates the outer input contract but does not guarantee that every nested DTO business rule will pass execution.
|
|
@@ -14,3 +14,10 @@ internal `targetConfig` field or hand-write `Cxx`/`Axxx` codes.
|
|
|
14
14
|
Response shape: `data.result`, with object keys recursively converted to snake_case.
|
|
15
15
|
|
|
16
16
|
Prefer `experiment experiment save` for progressive draft construction.
|
|
17
|
+
|
|
18
|
+
Feature requests may include optional `req.startTime` and
|
|
19
|
+
`req.expCycle={"cycleType":"time","fixedEndTime":"2026-11-08T16:00:00+08:00"}`.
|
|
20
|
+
Use ISO-8601 timestamps with timezone offsets. Submitting a Feature draft
|
|
21
|
+
places it in `pending`; with a start time, the server checks it every half hour
|
|
22
|
+
for automatic start. See [Feature timing](save_experiment.md#feature-timing) for
|
|
23
|
+
the draft, submission, manual-start, and deadline boundaries.
|
package/skills/ae-kb/SKILL.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: ae-kb
|
|
3
|
-
version: 1.0.
|
|
3
|
+
version: 1.0.8
|
|
4
4
|
description: 'AE knowledge base CLI manual for creating, importing read-only compiled snapshots, querying, LLM-powered ask, listing accessible knowledge bases and their sources, deterministic index/grep/read retrieval, checking status, ZIP source upload and directory management, raw child-file reading, revision-checked child updates and deletion, compiling, schema generation, URL sources, source deletion, ordinary text source creation, full reads and revision-checked editing, file/ZIP source draft replacement and restoration, and knowledge base deletion. Use when the user asks to manage AE (Agentic Engine) knowledge bases, import a compiled Markdown ZIP snapshot, upload documents or URLs to a knowledge base, query knowledge, ask knowledge bases with an LLM, list accessible knowledge bases or source metadata, inspect knowledge base indexes, search knowledge base pages, read a specific knowledge base page, check knowledge base status, generate schemas, compile knowledge, remove sources, create or edit text sources, or delete a knowledge base. To choose which knowledge base is worth searching, use the ae-kb-discovery skill first; this skill runs the retrieval once a target is chosen. Must use ae-cli kb commands and must not guess knowledge base names, scopes, page paths, source IDs, source display names, JSON payload shapes, or URL formats.'
|
|
5
5
|
---
|
|
6
6
|
|
|
@@ -17,6 +17,10 @@ ae-cli kb +<command> [options]
|
|
|
17
17
|
ae-cli kb source <create-text|read-text|replace|restore|preview|commit|cancel|check-updates|check-url-updates> [options]
|
|
18
18
|
```
|
|
19
19
|
|
|
20
|
+
## Formal publication and pinned reads
|
|
21
|
+
|
|
22
|
+
Read [Publication and read contexts](references/publication.md) for company approval, personal auto-publication, candidate inspection and changed-only approval previews (`kb candidate changes`), explicit published queries, or consistent multi-command retrieval. Compile success can mean a candidate awaiting approval; use `kb publication get` to verify formal publication. Never treat the working build status as proof that the formal version changed.
|
|
23
|
+
|
|
20
24
|
## Global Rules
|
|
21
25
|
|
|
22
26
|
- Use this skill for AE knowledge base tasks: create, import a compiled snapshot, query, ask with LLM, list accessible knowledge bases and their sources, inspect indexes, grep pages, read pages, check status, upload sources, add URL sources, generate schema, compile, remove source files, and delete knowledge bases.
|
|
@@ -41,7 +45,8 @@ ae-cli kb source <create-text|read-text|replace|restore|preview|commit|cancel|ch
|
|
|
41
45
|
| `source check-url-updates` | write | Submit an asynchronous check for one compiled custom URL source; success means accepted only. |
|
|
42
46
|
| `source check-updates` | write | Check one compiled Feishu source with the scheduled detector; mark changes without compiling. |
|
|
43
47
|
| `+ask` | read | LLM-powered Q&A over knowledge bases; for multi-page synthesis or multi-hop questions. |
|
|
44
|
-
| `+ask-status` | read | Query
|
|
48
|
+
| `+ask-status` | read | Query an ask execution, or revisit its original versioned citation with `--source-index`. |
|
|
49
|
+
| `project-target get` | read | Resolve the authoritative company project semantic target by project ID, including tag conflicts, disabled and restricted states. |
|
|
45
50
|
| `+list` | read | List accessible knowledge bases filtered by buildStatus (default: compiled). |
|
|
46
51
|
| `+list-sources` | read | List source metadata for one knowledge base so exact source identifiers can be discovered safely. |
|
|
47
52
|
| `+index` | read | List accessible knowledge bases and their `index.md` navigation maps. |
|
|
@@ -59,6 +64,26 @@ ae-cli kb source <create-text|read-text|replace|restore|preview|commit|cancel|ch
|
|
|
59
64
|
| `+rm-source` | high-risk-write | Delete one source from a knowledge base by stable ID; exact display name is legacy compatibility only. |
|
|
60
65
|
| `+remove` | write | Delete an entire knowledge base. |
|
|
61
66
|
|
|
67
|
+
## Project Semantic Target
|
|
68
|
+
|
|
69
|
+
```bash
|
|
70
|
+
ae-cli kb project-target get --project-id <project_id> --host <host>
|
|
71
|
+
ae-cli kb +new --scope company --name <name> --project-id <project_id> --purpose project-semantic
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
Use `project-target get` for project semantic maintenance; `+list` remains a visible catalog and cannot prove that a project target is missing. The server verifies project access and considers all undeleted company candidates bound to that project, without filtering by enabled state or build state. One candidate is reused for legacy compatibility. With several candidates, exactly one fixed business tag `项目语义` selects the target; zero returns `multiple/untagged`, several returns `multiple/tag_conflict`. With no binding, the authoritative exact name `<project_name> 项目语义知识库` without another project binding is the legacy fallback, using the same tag rule.
|
|
75
|
+
|
|
76
|
+
`available` returns the exact target identity and access capabilities. `restricted`, `disabled`, and `multiple` stop maintenance; never select a different visible KB or create a replacement. Only `missing` permits creation. A query/auth failure or HTTP 404 from an older server is terminal; do not infer absence. Existing explicit target IDs, including stored automation targets, must not be silently switched.
|
|
77
|
+
|
|
78
|
+
`+new --purpose project-semantic` is optional and company-only, requires one valid project ID, presets the fixed tag on both client and server, and rechecks the target under the server KB write lock. User tags are preserved within the existing total limit of two; overflow fails explicitly. A 409 means refresh the target, not retry with a different name. Ordinary creation without `--purpose` retains its existing behavior. The tag classifies purpose and never grants access. Deploy the supporting Agent API before using the new CLI/Skill path; old CLI requests remain accepted by the new server.
|
|
79
|
+
|
|
80
|
+
Transition status: transitional
|
|
81
|
+
Owning module: te-claude Project Semantic Knowledge Base API
|
|
82
|
+
Current transport: authenticated Agent REST via `kbApi`
|
|
83
|
+
Gateway target: TBD (project semantic target resolution and purpose-aware creation)
|
|
84
|
+
Review after: 2027-01-27
|
|
85
|
+
Exit condition: migrate when the KB Gateway equivalent exists, preserving target identity, project isolation, tag and state semantics.
|
|
86
|
+
|
|
62
87
|
## Compilation Usage Logs
|
|
63
88
|
|
|
64
89
|
```bash
|
|
@@ -81,7 +106,7 @@ Exit condition: migrate to the Gateway equivalent when available, preserving typ
|
|
|
81
106
|
|
|
82
107
|
## Published Version History
|
|
83
108
|
|
|
84
|
-
Use the nine version commands described in [`references/versions.md`](references/versions.md) to inspect immutable published history, compare source changes, and explicitly roll back an earlier version as a new publication. These commands manage history;
|
|
109
|
+
Use the nine version commands described in [`references/versions.md`](references/versions.md) to inspect immutable published history, compare source changes, and explicitly roll back an earlier version as a new publication. These commands manage history. Ordinary queries resolve formal published content; Agent/Ask executions and explicit read contexts keep the formal versions captured at the start of that execution or context. Use the returned `readView` to identify the content actually read.
|
|
85
110
|
|
|
86
111
|
| Command | Risk | Purpose |
|
|
87
112
|
| --- | --- | --- |
|
|
@@ -250,7 +275,7 @@ ae-cli kb +schema --name engineering-handbook --scope company
|
|
|
250
275
|
|
|
251
276
|
Use `--force` only when `+status` reports `schema_generating` and the user explicitly wants to replace the current generation attempt. The replacement may consume additional tokens.
|
|
252
277
|
|
|
253
|
-
Schema and Compile accept the same optional model reference. Prefer the model record `id` returned by `ae-cli agent +list-models` (`Model.id`). Historical `modelId` and the unambiguous `modelId::scope` form remain compatible. A model `displayName` is presentation text, not a stable reference.
|
|
278
|
+
Schema and Compile accept the same optional model reference. By default, omit `--model` / `--model-id` and keep the existing KB default selection. Preserve an explicit choice from the user or task configuration. Never infer a model reference from the assistant's self-description. `company-auto` means the Company Auto routing policy, not a physical model; use the canonical reference `virtual:company-auto:v1` for compatibility with older servers. Prefer the model record `id` returned by `ae-cli agent +list-models` (`Model.id`). Historical `modelId` and the unambiguous `modelId::scope` form remain compatible. A model `displayName` is presentation text, not a stable reference.
|
|
254
279
|
|
|
255
280
|
To add one-time guidance for this generation without changing stored knowledge base metadata, pass `--custom-instructions`. The server trims the value, treats whitespace-only input as absent, and accepts up to 10,000 Unicode characters. Do not include secrets or credentials.
|
|
256
281
|
|
|
@@ -258,7 +283,6 @@ To add one-time guidance for this generation without changing stored knowledge b
|
|
|
258
283
|
ae-cli kb +schema \
|
|
259
284
|
--name engineering-handbook \
|
|
260
285
|
--scope company \
|
|
261
|
-
--model <model-ref> \
|
|
262
286
|
--custom-instructions "Prioritize troubleshooting workflows and preserve command examples"
|
|
263
287
|
```
|
|
264
288
|
|
|
@@ -267,7 +291,7 @@ Use `--dry-run` to inspect the request body before sending it. While generation
|
|
|
267
291
|
Compile after sources and schema are ready:
|
|
268
292
|
|
|
269
293
|
```bash
|
|
270
|
-
ae-cli kb +compile --name engineering-handbook --scope company --mode incremental
|
|
294
|
+
ae-cli kb +compile --name engineering-handbook --scope company --mode incremental
|
|
271
295
|
```
|
|
272
296
|
|
|
273
297
|
Valid compile modes are `incremental` and `full`; default is `incremental`.
|
|
@@ -440,6 +464,9 @@ ae-cli kb +ask-status --execution-id <id>
|
|
|
440
464
|
```
|
|
441
465
|
|
|
442
466
|
- `--execution-id`: required. The execution ID returned by `+ask` submission.
|
|
467
|
+
- `--source-index`: optional zero-based `sourceIndex` from `sources[].reference`. Returns that saved page's `content`, `path`, and public `readView`, at the original published version even after a new publication. Uses your current user permissions; query credentials are not revived. Legacy or old sources without an immutable version return `KB_REFERENCE_UNVERSIONED`; do not replace this with a fresh `+read` and call it the original citation.
|
|
468
|
+
- Completed Ask sources preserve `{knowledgeBaseKey, path, readView?, reference?}`. Keep `readView.versionId/versionNumber` when citing them. To revisit: `ae-cli kb +ask-status --execution-id <reference.executionId> --source-index <reference.sourceIndex>`.
|
|
469
|
+
- Historical source reads are unavailable inside an execution that requires a fixed KB context (`KB_READ_CONTEXT_REFERENCE_FORBIDDEN`). Do not bypass this by removing execution context or using an ordinary token. Revisit outside that execution with current user credentials. Ordinary `+ask-status` polling without `--source-index` remains available.
|
|
443
470
|
- Output: Returns one successful CLI envelope snapshot. For a failed execution, the stable server code remains at `data.error.code`: `{"ok":true,"data":{"executionId":"...","status":"failed","error":{"code":"retrieval_error","message":"..."}}}`. Does not poll.
|
|
444
471
|
|
|
445
472
|
### `+list`
|
|
@@ -0,0 +1,81 @@
|
|
|
1
|
+
# Publication and read contexts
|
|
2
|
+
|
|
3
|
+
Source-managed personal KBs publish automatically after a successful compile. Company KBs produce an immutable candidate and automatically submit `kb.version.publish@1`; an authorized current content editor (including the requester) approves through the existing approval workflow. Editing working sources after compilation does not change the candidate. Approval publishes that frozen candidate without overwriting working edits. No feature switch is required.
|
|
4
|
+
|
|
5
|
+
## Inspect and manage
|
|
6
|
+
|
|
7
|
+
| Command | Purpose | Risk |
|
|
8
|
+
| --- | --- | --- |
|
|
9
|
+
| `ae-cli kb publication get` | Formal version and candidate summary | read |
|
|
10
|
+
| `ae-cli kb candidate get` | Candidate metadata | read |
|
|
11
|
+
| `ae-cli kb candidate changes` | Changed sources, Wiki and Index; optional bounded text diff | read |
|
|
12
|
+
| `ae-cli kb candidate read` | Immutable candidate file | read |
|
|
13
|
+
| `ae-cli kb candidate discard` | Close a confirmed failed candidate | high-risk-write |
|
|
14
|
+
| `ae-cli kb +rm-source` | Delete a source, optionally cancelling its pending candidate | high-risk-write |
|
|
15
|
+
| `ae-cli kb read-context create` | Pin authorized formal views | read |
|
|
16
|
+
| `ae-cli kb read-context release` | Release the current user's context | write |
|
|
17
|
+
|
|
18
|
+
- `ae-cli kb publication get --name <verified-name> [--scope personal|company]`: formal version and active candidate summary.
|
|
19
|
+
- `ae-cli kb candidate get --name <verified-name> [--scope ...] --candidate <id>`: candidate metadata.
|
|
20
|
+
- `ae-cli kb candidate read --name <verified-name> [--scope ...] --candidate <id> --path <verified-path>`: immutable candidate index.md, CLAUDE.md or wiki Markdown. Candidate preview requires current content-write permission; it is not a formal query.
|
|
21
|
+
- `ae-cli kb candidate discard --name <verified-name> [--scope ...] --candidate <id> --yes`: `high-risk-write`; discard a candidate after a confirmed retryable publication failure. Pending requests use approval cancel. A running or uncertain publication cannot be discarded.
|
|
22
|
+
- When source deletion conflicts with a pending candidate, explicitly combine `ae-cli kb +rm-source --name <verified-name> --id <source-id> --cancel-candidate --expected-candidate-id <id> --yes`. This is `high-risk-write`; both flags are required. A changed candidate causes a conflict; inspect again instead of silently deleting it.
|
|
23
|
+
|
|
24
|
+
## Review compilation changes
|
|
25
|
+
|
|
26
|
+
```bash
|
|
27
|
+
ae-cli kb candidate changes --name 'Verified KB' --scope company --candidate '<candidateId>'
|
|
28
|
+
ae-cli kb candidate changes --name 'Verified KB' --scope company --candidate '<candidateId>' --entry-id '<entryId>'
|
|
29
|
+
ae-cli kb candidate changes --name 'Verified KB' --scope company --candidate '<candidateId>' --entry-id '<entryId>' --file '<returned-member-path>' --context 3
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
The list returns `candidateId`, the fixed `baseline`, `status`, `firstPublication`, `entries` and `summary`. Only changed sources, Wiki pages and Index are included; compilation rules and historical comparison are outside this preview. Copy entry IDs and member paths from actual responses. `--file` requires `--entry-id`; `--context` is 0 (default, changes only) or 3 (adjacent unchanged lines).
|
|
33
|
+
|
|
34
|
+
Entry details include changed source `fields`, changed `files`, `selectedFile`, `preview` and bounded `hunks`. The first changed file is selected by default; choose other returned members explicitly. Binary files return metadata; text over 1 MB or a complex diff has an explicit preview status. These states are not proof of unchanged contents. A legacy publication without a reconstructible frozen baseline returns `status=baseline_unavailable` and `summary=null`; missing/corrupt snapshots, forbidden access and invalid selections fail explicitly. Never substitute current working files. Successful first publication is shown entirely as added. The endpoint requires the same current content-write permission as candidate reads. Existing `candidate get/read` calls remain unchanged. Unsupported older Hosts return an error; do not fall back to comparing mutable content.
|
|
35
|
+
|
|
36
|
+
`--dry-run` shows the GET path and validated selectors without remote access. This new command uses the same Transitional admission and migration exit below.
|
|
37
|
+
|
|
38
|
+
## Query a stable formal view
|
|
39
|
+
|
|
40
|
+
`ae-cli kb +list --view published` creates a temporary context, queries, then releases it. Old Hosts without context support fail explicitly. The legacy `kb +list` default remains `buildStatus=compiled`; published/contextual list omits only this implicit filter. Explicit `--build-status` still applies.
|
|
41
|
+
|
|
42
|
+
For several related reads:
|
|
43
|
+
|
|
44
|
+
```bash
|
|
45
|
+
ae-cli kb read-context create --sources '[{"scope":"company","name":"Verified KB"}]'
|
|
46
|
+
ae-cli kb +index --name 'Verified KB' --scope company --read-context '<readContextId>'
|
|
47
|
+
ae-cli kb +read --name 'Verified KB' --scope company --path 'wiki/verified.md' --read-context '<readContextId>'
|
|
48
|
+
ae-cli kb read-context release --id '<readContextId>'
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
Create accepts optional `--sources` array of `{scope,name}` and `--locale zh|en|ja|ko`. Omission discovers currently authorized formal KBs. Contexts last 15 minutes and bind list/index/grep/read to the selected formal versions. Sources, permissions, KB lifecycle and actor identity remain checked on every read. Release is owner-only and idempotent for an expired context. Do not manufacture version IDs, storage keys or context IDs.
|
|
52
|
+
|
|
53
|
+
For an expired or mismatched context, stop the sequence. A new explicit sequence may create a new context, but must not mix its results with prior context results. New snapshot-mode KBs with no published version are unpublished; an inconsistent formal pointer fails rather than falling back to working files.
|
|
54
|
+
|
|
55
|
+
Inside an Agent/Ask execution the runtime supplies a private execution context. The CLI automatically uses its scoped credential for the four query commands; do not read, print, copy or replace this credential. Missing/expired/mismatched runtime credentials fail closed, including on 401. Management commands continue using normal CLI authorization. Runtime capability negotiation is protocol validation, not a user feature toggle.
|
|
56
|
+
|
|
57
|
+
## Copy and export
|
|
58
|
+
|
|
59
|
+
A new company-copy approval freezes the current personal formal version when its Effect first executes. Retries reuse the same version and target identity. Later source changes do not alter the copy.
|
|
60
|
+
|
|
61
|
+
`ae-cli agent resource-package export --view working|published ...` defaults to working. Working exports preserve editable sources/schema and omit stale generated wiki. Published exports use one frozen formal snapshot and matching source metadata. Export retries reuse the frozen dependency closure. Editable imports start with pending sources and no published version; read-only compiled imports keep their established serving behavior.
|
|
62
|
+
|
|
63
|
+
## Command admission
|
|
64
|
+
|
|
65
|
+
- Classification: Transitional.
|
|
66
|
+
- Owning module: te-claude knowledge-base publication and query-context management; CLI owner: te-cli te-kb.
|
|
67
|
+
- Current transport: authenticated External REST through `kbApi`; runtime query calls additionally require the scoped context credential and header.
|
|
68
|
+
- Gateway target: an equivalent KB publication/candidate/read-context capability with the same current-editor checks, immutable candidates, idempotency, and fail-closed query binding; no equivalent is currently registered.
|
|
69
|
+
- Review after: 2026-11-25.
|
|
70
|
+
- Exit condition: when Gateway provides these semantics, migrate value-adding typed commands to L2 and simple wrappers to L3. Preserve existing command compatibility during migration.
|
|
71
|
+
- Write safety: candidate discard and combined source deletion require explicit `--yes`, current write permission and candidate identity checks. Context release is owner-scoped and idempotent; request preview is local and does not claim remote validation.
|
|
72
|
+
|
|
73
|
+
### Large change sets and first publication
|
|
74
|
+
|
|
75
|
+
Use `kb candidate changes --group sources|wiki|index --search <name-or-path> --change added|modified|deleted --offset 0 --limit 50` for a bounded directory page. `summary` and `groups` describe all changes; `page.total` describes entries matching the filters. Omitting these flags preserves the existing full-list call. A detail request uses `--entry-id`; `--file-search`, `--file-offset` and `--file-limit` page the changed members. `--file` selects an exact changed member independent of member pagination.
|
|
76
|
+
|
|
77
|
+
The first successful source-managed KB compilation now publishes automatically, including company KBs. Later company updates require approval. Empty version pointers alone do not prove a first publication: legacy formal content and historical versions still require review. Existing eligible first-publication pending requests are withdrawn by the server with a system audit and the same frozen candidate is published; do not approve or create another compilation to force this transition. Already decided requests are not converted. `publication get` and `candidate get` remain read-only and expose the current publication state; an automatically published candidate is still readable by its known ID.
|
|
78
|
+
|
|
79
|
+
Text and Markdown members return changed hunks. Small, unambiguous CSV/TSV changes may additionally include `table` containing only changed rows. ZIP and URL/Feishu sources expose changed frozen members; PDF, Office, images, invalid UTF-8 and oversized content explicitly return metadata/limit status rather than a fabricated textual diff. No source URL is fetched during preview.
|
|
80
|
+
|
|
81
|
+
Compatibility: deploy the matching server before using new paging flags. Old CLI requests continue to work on the new server. New flags against older servers fail explicitly with `KB_CANDIDATE_PAGING_UNSUPPORTED` when `page` or `filePage` is missing; the CLI never treats an unfiltered old response as a filtered result.
|
|
@@ -140,7 +140,20 @@ A revision or access-context change requires revalidation, not silent reuse.
|
|
|
140
140
|
5. **Assess coverage, then answer or iterate.** Map the user's question into
|
|
141
141
|
subquestions and check each one against the sections you actually read. If a
|
|
142
142
|
subquestion is covered, answer with citations (knowledge base + page path +
|
|
143
|
-
section
|
|
143
|
+
section, plus the returned `readView.versionId/versionNumber` for snapshots).
|
|
144
|
+
For `version_pinned` results, describe consistency within that captured view.
|
|
145
|
+
Unchanged `versionId/readIdentity` proves that the reads used the same snapshot;
|
|
146
|
+
it does not establish whether a newer publication or rollback occurred outside
|
|
147
|
+
the execution. Claims about the latest formal version require separate current
|
|
148
|
+
publication evidence obtained through an authorized management workflow.
|
|
149
|
+
Preserve version identity with the reference; a fresh read of the same path is
|
|
150
|
+
not a historical citation read. For completed Ask sources with `reference`, use
|
|
151
|
+
`ae-cli kb +ask-status --execution-id <reference.executionId> --source-index <reference.sourceIndex>`
|
|
152
|
+
to reopen the original version under current permissions. An unversioned source
|
|
153
|
+
has no guaranteed historical snapshot; do not label its current content historical.
|
|
154
|
+
Historical source reads are rejected inside a required fixed-context execution;
|
|
155
|
+
never remove that context or fall back to an ordinary token to bypass its KB set.
|
|
156
|
+
If a gap remains, go back to step 2 with another set of `--paths`
|
|
144
157
|
**copied** from current navigation, or return to discovery for the next candidate.
|
|
145
158
|
For a knowledge-only question, missing evidence after the allowed search
|
|
146
159
|
means "not covered by the knowledge base" — do not attach a confident fact
|