@josephyan/qingflow-cli 1.1.4 → 1.1.5

package/skills/qingflow-cli/reference/record/analysis/analysis-patterns.md

@@ -0,0 +1,112 @@
+# Analysis Patterns
+
+## Canonical Sequence
+
+1. `app_get`
+2. `record_browse_schema_get`
+3. decide metric intent
+4. choose `record_access.columns / where / order_by`
+5. `record_access`
+6. Python over every returned CSV shard
+7. optional `record_list` or `record_get` only for sample/detail verification
+
+Metric intent must be one of:
+
+- `count`
+- `sum`
+- `avg`
+- `distinct_count`
+- ratio with numerator and denominator
+- sorted ranking
+- time trend
+- period comparison
+
+## Distribution
+
+1. Fetch grouping field and filter fields.
+2. Run the field-quality profile for the grouping field.
+3. If the field passes quality gates, group by the readable field-id anchored column such as `项目状态__field_343283094`.
+4. Count rows and calculate share from the sum of counts.
+5. Report top groups plus total row count.
+
+If the grouping field is ambiguous, ask the user to choose from a short candidate list.
+
+## Dimension Selection
+
+When the user asks for a semantic bucket such as `板块`, `模块`, `业务线`, or `来源`, inspect candidate fields and choose the most reliable one:
+
+1. Match schema titles to the user's wording.
+2. Fetch candidate fields together if they are cheap.
+3. Profile `blank_rate`, period coverage, and distinct count.
+4. Prefer the candidate with clear semantics and usable coverage.
+5. If the literal field is sparse, downgrade it to `已填写样本观察` and use the nearest reliable fallback for the main conclusion.
+
+Example: if `缺陷所属模块` is mostly empty but `缺陷所属平台` and `所属产品` are complete, use platform/product for the main conclusion and state that module-level analysis is limited.
+
+Quality gates:
+
+- Overall `blank_rate > 0.4`: not a primary conclusion dimension.
+- Any compared period `blank_rate > 0.8`: not valid for period comparison.
+- High-cardinality description/id fields are not dimensions unless the user explicitly asks for record-level ranking.
+
+## Ratio / Conversion / Penetration
+
+1. Define numerator and denominator in plain language.
+2. Fetch both populations with compatible scope.
+3. Compute ratio in Python.
+4. Report `numerator / denominator = percentage`.
+
+If denominator is missing or scope differs, do not call the result a rate.
+
+## Average / Sum
+
+1. Fetch grouping field and numeric metric field.
+2. Convert the metric column with `pd.to_numeric(errors="coerce")`.
+3. Report count, sum, and average together when useful.
+4. State how blanks/non-numeric values were handled if material.
+
+## Ranking
+
+1. Build the metric in Python.
+2. Sort explicitly.
+3. Report Top N with metric values.
+4. Do not infer ranking from unsorted sample rows.
+
+## Trend
+
+1. Choose a date/time field from `suggested_time_fields`.
+2. Convert relative phrases into exact date ranges.
+3. Fetch the date field and metrics.
+4. Bucket in pandas by day/week/month/quarter/year.
+5. Report both absolute values and changes.
+
+## Same-Period Comparison
+
+For `今年5月 vs 去年5月`:
+
+1. Use the same date field for both periods.
+2. Fetch the full combined date range or two separate compatible ranges.
+3. Apply identical business filters.
+4. Compute absolute delta and percentage delta.
+5. State both periods explicitly.
+
+## Sample Inspection
+
+Use `record_list` only after the aggregate result is complete, and only for:
+
+- representative examples
+- checking surprising categories
+- manually inspecting records behind a bucket
+
+Never use `record_list` alone for final averages, shares, rankings, trends, or distributions.
+
+## Ambiguous Field Recovery
+
+If the exact field is unclear:
+
+1. inspect `record_browse_schema_get.fields`
+2. use titles and suggested fields
+3. if one candidate is clearly dominant, proceed
+4. otherwise ask the user to confirm
+
+Do not retry tools with guessed field names.

package/skills/qingflow-cli/reference/record/analysis/business-context.md

@@ -0,0 +1,74 @@
+# Business Context
+
+Use this when analysis depends on organization, aliases, ownership, stage semantics, or user-provided business definitions.
+
+## When To Check Context
+
+Check for business context when the request mentions:
+
+- department / team / group / region
+- owner / assignee / sales rep / partner
+- stage / status / funnel / conversion
+- product line / business line
+- same-period comparison
+- "北斗部门", "SMB", "伙伴", or any named internal scope
+
+## Mapping Rules
+
+Use explicit mappings in this order:
+
+1. the user's message in the current thread
+2. attached or local business context files
+3. schema-visible fields and sample records
+4. short clarification to the user
+
+Do not infer hidden org hierarchy from memory. If the mapping changes the denominator or grouping, state it in the final answer.
+
+Example:
+
+```python
+dept_map = {
+    "烈焰组": "北斗部门",
+    "飓风组": "北斗部门",
+}
+df["部门口径"] = df["field_40"].replace(dept_map)
+```
+
+Final wording:
+
+```text
+部门口径：将「烈焰组」「飓风组」合并计入「北斗部门」。
+```
+
+## Ratio Definitions
+
+Before computing rates, define:
+
+- numerator
+- denominator
+- time range
+- grouping dimension
+- exclusions
+
+If any part is ambiguous, ask. Do not rename a count as a rate.
+
+## Time Scope
+
+Normalize relative dates to exact dates before calling `record_access`.
+
+Examples:
+
+- `今年5月` -> `2026-05-01` to `2026-05-31` when current year is 2026
+- `去年同期` -> same month range in the previous year
+- `最近一个完整自然月` -> previous calendar month, not the last 30 days
+
+## Cross-App Reconciliation
+
+If the analysis needs multiple apps:
+
+1. run the standard sequence per app
+2. keep each dataset's scope and completeness separately
+3. join in Python only on explicit ids or trusted business keys
+4. disclose join keys and unmatched records
+
+If no reliable key exists, report the gap instead of forcing a join.

package/skills/qingflow-cli/reference/record/analysis/confidence-reporting.md

@@ -0,0 +1,69 @@
+# Confidence Reporting
+
+## Full Conclusion Gate
+
+Use `全量可信结论` only when:
+
+- `record_browse_schema_get` was used
+- data came from `record_access`
+- all returned CSV shards were read in Python
+- `record_access.complete=true`
+- `record_access.truncated=false`
+- `record_access.safe_for_final_conclusion=true`
+- metric definitions are complete
+- denominator exists for every ratio
+- time fields and date ranges are explicit
+- primary grouping dimensions pass field-quality gates
+
+## Initial Observation Gate
+
+Use `初步观察` when:
+
+- `record_access.status=needs_scope`
+- `record_access.status=partial`
+- `record_access.complete=false`
+- `record_access.truncated=true`
+- `record_access.safe_for_final_conclusion=false`
+- evidence came from `record_list`
+- scope or saved view filter is unverified
+
+## Anti-Mixing Rule
+
+Do not combine full CSV-derived totals and sample-only rows in one sentence.
+
+Correct split:
+
+- full totals/distributions: `全量可信结论`
+- illustrative examples: `样本观察`
+
+## Semantic Gate
+
+Even with `safe_for_final_conclusion=true`, downgrade if:
+
+- metric definition is incomplete
+- denominator was not queried
+- conclusion mentions trend but no time field was used
+- conclusion mentions volume but no count was computed
+- grouping depends on unconfirmed business aliases
+- custom view scope is not verified
+- primary grouping field has high missingness or poor period coverage
+
+## Partial Disclosure
+
+If only part of the user request is complete:
+
+- say which parts are complete
+- say which parts are unresolved
+- do not collapse into one all-clear conclusion
+
+## Compact Disclosure Template
+
+```text
+可信度：全量可信 / 初步观察
+数据完整性：complete=..., truncated=..., safe_for_final_conclusion=...
+字段质量：primary dimension blank_rate=..., period coverage=...
+取数字段：...
+时间范围：...
+业务口径：...
+限制：...
+```

package/skills/qingflow-cli/reference/record/analysis/data-access-playbook.md

@@ -0,0 +1,106 @@
+# Data Access Playbook
+
+This file is the operational state machine for `record_access`.
+
+## Required Sequence
+
+1. `app_get`
+2. choose `view_id` from `accessible_views`
+3. `record_browse_schema_get(app_key, view_id)`
+4. build `record_access.columns / where / order_by` from field ids
+5. run `record_access`
+6. read every CSV shard with Python
+
+Do not call `record_access` with field titles, guessed ids, page controls, row limits, or profile.
+CSV columns are readable and field-id anchored, for example `项目状态__field_343283094`; do not look for extra `schema.json` or README files.
+
+## Finding `record_access`
+
+`record_access` can be available as an MCP tool or as a CLI subcommand. If the MCP surface does not show a top-level `record_access` tool, look under the Qingflow CLI record group before choosing any other path:
+
+```bash
+qingflow record --help
+qingflow record access --help
+```
+
+The CLI call is:
+
+```bash
+qingflow record access \
+  --app-key APP_KEY \
+  --view-id VIEW_ID \
+  --columns-file columns.json \
+  --where-file where.json \
+  --order-by-file order_by.json \
+  --json
+```
+
+This is the same analysis path: it writes CSV shards and metadata for Python. Do not replace it with list browsing, export, QingBI, or aggregate helpers just because the MCP tool is not visible.
+
+## Request Patterns
+
+### Count or distribution
+
+Fetch the grouping field and any time/business filter field.
+
+```json
+{
+  "app_key": "APP_KEY",
+  "view_id": "system:all",
+  "columns": [{ "field_id": 18 }],
+  "where": [{ "field_id": 2, "op": "between", "value": ["2026-05-01", "2026-05-31"] }],
+  "order_by": []
+}
+```
+
+### Trend
+
+Fetch the date/time field plus metric fields.
+
+```json
+{
+  "app_key": "APP_KEY",
+  "view_id": "system:all",
+  "columns": [{ "field_id": 2 }, { "field_id": 18 }],
+  "where": [{ "field_id": 2, "op": "between", "value": ["2026-01-01", "2026-12-31"] }],
+  "order_by": [{ "field_id": 2, "direction": "asc" }]
+}
+```
+
+### Ratio
+
+If numerator and denominator use different filters, run separate `record_access` calls. Only compute the ratio after both source datasets are complete and compatible.
+
+## Status Decisions
+
+| Status | Meaning | Agent action |
+|---|---|---|
+| `success` + `safe_for_final_conclusion=true` | Full retrieved scope is reliable | Give final conclusion |
+| `needs_scope` | Tool refused large unbounded scan, no CSV | Ask for scope or retry with explicit period/business filter |
+| `partial` | Some CSV files written, but not full data | Give only subset observation |
+| `complete=false` | Not all requested data is available | Do not present full-population conclusion |
+| `truncated=true` | Tool had to stop before full scope | Disclose and narrow scope |
+
+## `needs_scope` Recovery
+
+Use the returned `scope` object:
+
+- `reported_total`: explain why scope is needed
+- `suggested_time_fields`: choose likely date fields
+- `recommended_where_examples`: reuse if they match the user request
+
+If the user already provided a concrete month/quarter/year, retry with that period. If no business boundary is available, ask one short clarification.
+
+## `partial` Recovery
+
+You may read the files, but must label output as partial:
+
+- say which files/rows were analyzed
+- do not use `全部`, `所有`, `整体`, or `全量`
+- suggest narrowing time or business scope before final conclusion
+
+## View Scope
+
+For custom views, the result is scoped to that saved view. If `verification.view_filter_verified=false`, disclose that the saved-filter scope could not be fully verified.
+
+For board/gantt views, switch to a table-style view or `system:all` plus explicit filters.

package/skills/qingflow-cli/reference/record/analysis/pandas-recipes.md

@@ -0,0 +1,172 @@
+# Pandas Recipes
+
+Use Python to read returned CSV shards. Never paste raw CSV into the model context.
+
+## Load All Shards
+
+```python
+import pandas as pd
+
+files = [
+    "/absolute/path/records-0001.csv",
+    # include every record_access.files[].local_path
+]
+
+frames = [pd.read_csv(path, dtype=str, keep_default_na=False) for path in files]
+df = pd.concat(frames, ignore_index=True) if frames else pd.DataFrame()
+
+# Columns are readable and field-id anchored, e.g. 项目状态__field_343283094.
+fields = []  # optionally paste record_access.fields here if you need field-id metadata
+field_by_id = {int(item["field_id"]): item for item in fields if "field_id" in item}
+title_by_col = {item["column_name"]: item["title"] for item in fields if item.get("column_name")}
+```
+
+## Field Quality Profile
+
+Run this before choosing final grouping dimensions.
+
+```python
+def field_quality(frame: pd.DataFrame, *, date_col: str | None = None) -> pd.DataFrame:
+    rows = []
+    for col in frame.columns:
+        blank = frame[col].astype(str).eq("")
+        item = {
+            "column": col,
+            "row_count": len(frame),
+            "blank_count": int(blank.sum()),
+            "blank_rate": float(blank.mean()) if len(frame) else 0.0,
+            "distinct_count": int(frame[col].replace("", pd.NA).nunique(dropna=True)),
+        }
+        rows.append(item)
+    quality = pd.DataFrame(rows).sort_values(["blank_rate", "distinct_count"], ascending=[False, False])
+    if date_col and date_col in frame.columns:
+        tmp = frame.copy()
+        tmp["_period"] = pd.to_datetime(tmp[date_col], errors="coerce").dt.to_period("M").astype(str)
+        period_quality = []
+        for col in frame.columns:
+            if col == date_col:
+                continue
+            by_period = tmp.groupby("_period")[col].apply(lambda s: float(s.astype(str).eq("").mean()))
+            period_quality.append({"column": col, "max_period_blank_rate": float(by_period.max()) if len(by_period) else 0.0})
+        quality = quality.merge(pd.DataFrame(period_quality), on="column", how="left")
+    return quality
+```
+
+Quality gates:
+
+- `blank_rate > 0.4`: do not use as the primary conclusion dimension.
+- `max_period_blank_rate > 0.8`: do not use for period comparison.
+- Very high `distinct_count` fields are usually identifiers or descriptions, not grouping dimensions.
+- High-missing dimensions may still be reported as `已填写样本观察`.
+
+## Column Selection
+
+Prefer exact readable CSV columns. Use suffix matching only when you need to address a field id programmatically.
+
+```python
+def col_by_field_id(frame, field_id: int) -> str:
+    suffix = f"__field_{field_id}"
+    matches = [col for col in frame.columns if col.endswith(suffix)]
+    if not matches:
+        raise KeyError(f"field_id not in CSV: {field_id}")
+    return matches[0]
+```
+
+## Count Distribution
+
+```python
+col = "项目状态__field_18"
+quality = field_quality(df)
+blank_rate = quality.loc[quality["column"].eq(col), "blank_rate"].iloc[0]
+if blank_rate > 0.4:
+    print(f"Use only as filled-sample observation: {col} blank_rate={blank_rate:.1%}")
+dist = (
+    df[col]
+    .replace("", pd.NA)
+    .fillna("未填写")
+    .value_counts(dropna=False)
+    .rename_axis("group")
+    .reset_index(name="count")
+)
+dist["share"] = dist["count"] / dist["count"].sum()
+```
+
+## Numeric Aggregation
+
+```python
+group_col = "项目状态__field_18"
+amount_col = "金额__field_25"
+tmp = df.copy()
+tmp[amount_col] = (
+    tmp[amount_col]
+    .str.replace(",", "", regex=False)
+    .str.replace("￥", "", regex=False)
+    .pipe(pd.to_numeric, errors="coerce")
+)
+summary = (
+    tmp.groupby(group_col, dropna=False)[amount_col]
+    .agg(count="count", total="sum", avg="mean")
+    .reset_index()
+    .sort_values("total", ascending=False)
+)
+```
+
+## Date Trend
+
+```python
+date_col = "申请时间__field_2"
+tmp = df.copy()
+tmp[date_col] = pd.to_datetime(tmp[date_col], errors="coerce")
+tmp = tmp.dropna(subset=[date_col])
+tmp["month"] = tmp[date_col].dt.to_period("M").astype(str)
+trend = tmp.groupby("month").size().reset_index(name="count")
+```
+
+## Year-Over-Year Month Comparison
+
+```python
+date_col = "申请时间__field_2"
+tmp = df.copy()
+tmp[date_col] = pd.to_datetime(tmp[date_col], errors="coerce")
+tmp = tmp.dropna(subset=[date_col])
+tmp["year"] = tmp[date_col].dt.year
+tmp["month"] = tmp[date_col].dt.month
+monthly = tmp.groupby(["year", "month"]).size().reset_index(name="count")
+```
+
+## Ratio
+
+```python
+numerator = len(df[df["项目状态__field_18"].eq("已成交")])
+denominator = len(df)
+ratio = numerator / denominator if denominator else None
+```
+
+Always report numerator and denominator.
+
+## Multi-Select Cells
+
+If values are serialized with delimiters, inspect samples first. For simple comma-separated values:
+
+```python
+col = "标签__field_30"
+exploded = (
+    df.assign(_value=df[col].str.split(","))
+    .explode("_value")
+)
+exploded["_value"] = exploded["_value"].str.strip()
+multi_dist = exploded["_value"].value_counts().reset_index(name="count")
+```
+
+## Business Mapping
+
+```python
+mapping = {
+    "烈焰组": "北斗部门",
+    "飓风组": "北斗部门",
+}
+department_col = "部门__field_40"
+df["department_normalized"] = df[department_col].replace(mapping)
+```
+
+State the mapping in the final answer.

package/skills/qingflow-cli/reference/record/analysis/report-format.md

@@ -0,0 +1,76 @@
+# Report Format
+
+Use this for user-facing analysis reports.
+
+## Short Answer
+
+```text
+结论：
+- ...
+
+关键数据：
+- 指标 A：...
+- 指标 B：...
+
+口径与范围：
+- 应用 / 视图：...
+- 时间范围：...
+- 字段：...
+- 字段质量：...
+- 业务映射：...
+- 数据完整性：...
+
+限制：
+- ...
+```
+
+## Detailed Report
+
+```text
+1. 分析范围
+   - app / view
+   - time range
+   - filters
+   - rows analyzed
+
+2. 核心结论
+   - concrete numbers first
+   - no vague adjectives without numbers
+
+3. 分项数据
+   - distribution / trend / ranking tables
+   - percentages with numerator and denominator
+
+4. 解释与建议
+   - separate facts from hypotheses
+
+5. 口径与可信度
+   - fields used
+   - field-quality gates and downgraded dimensions
+   - mapping rules
+   - completeness
+   - partial or unverified scope warnings
+```
+
+## Wording Rules
+
+- Use `全量可信结论` only when the accessed scope is complete and safe.
+- Use `初步观察` for partial or unverified data.
+- Do not say `全部`, `所有`, `整体`, or `完整` when `safe_for_final_conclusion=false`.
+- For ratios, always show `numerator / denominator`.
+- For comparisons, show both periods' absolute values and the delta.
+
+## Comparison Template
+
+```text
+今年5月 vs 去年5月：
+- 记录数：今年 X，去年 Y，变化 +Z（+P%）
+- 金额：今年 X，去年 Y，变化 +Z（+P%）
+- 结构变化：...
+
+口径：
+- 时间字段：...
+- 部门字段：...
+- 部门映射：...
+- 数据完整性：...
+```

package/skills/qingflow-cli/reference/record/insert/README.md

@@ -0,0 +1,75 @@
+# Qingflow CLI Record Insert
+
+## Default Path
+
+`record schema insert -> record insert --items-file -> optional record get/readback`
+
+Default to batch-shaped insert. A single new record is `items` with one row.
+
+## Core Tools
+
+- `qingflow record schema insert`
+- `qingflow record insert --items-file`
+- `qingflow record member-candidates`
+- `qingflow record department-candidates`
+- file upload command when attachments are required
+
+## Working Rules
+
+1. Start with `record schema insert`
+2. Read `required_fields`, `optional_fields`, `runtime_linked_required_fields`, and `payload_template`
+3. Inside every field bucket, read field-level `linkage` first when present; it is the canonical static hint for linked visibility, reference-driven auto fill, or formula-driven fields
+4. Inside `optional_fields`, pay special attention to any field with `may_become_required=true`; these are writable fields that can become required when linked visibility or option-driven rules activate
+5. Build `items` as `[{"fields": {...}}]`, where each `fields` map uses field titles from the insert schema
+6. Treat `runtime_linked_required_fields` as required-but-not-directly-writable runtime/upstream dependencies, not as fields to hand-fill blindly
+7. For `linkage.kind=logic_visibility`, read `sources` as upstream trigger fields and treat `role=manual_input_after_activation` as "fill this only after the upstream condition is satisfied"
+8. For `linkage.kind=reference_fill`, prefer filling the source field first; treat target fields with `role=auto_fill_preferred` or `auto_fill_only` as reference-driven outputs rather than blind manual inputs
+9. For `linkage.kind=formula_fill`, treat the field as formula/default-auto-fill driven unless the user explicitly asks to override it and the field is still writable
+10. If insert succeeds and single-record detail/readback matters, prefer `record get`; use `record list` only for batch row-shaped normalized readback
+11. Keep subtable payloads under the parent field as a row array
+12. Member / department / relation fields may accept natural strings directly, such as `"张三"`, `"直销部"`, or `"海军军医大学"`; do not pre-query ids by default
+13. If the write returns `status="needs_confirmation"`, stop and surface the candidates
+14. Retry failed rows only with explicit ids / objects after the user confirms
+15. Keep `verify_write=true` for production inserts
+16. If post-write detail context matters, read `record get` fields, `media_assets.items[].local_path`, `file_assets.items[].local_path`, `file_assets.items[].extraction.text_path`, and `semantic_context`; `record get` follows the frontend storage cookie redirect path for Qingflow attachments, so prefer local paths over remote URLs and do not expect legacy `data.normalized_record`
+17. Treat nested schema shape as guidance, not a brittle contract; do not hard-code transient implementation details like optional nested `field_id` shape when composing inserts
+18. For `partial_success`, read `created_record_ids`, then repair only the failed `items[].row_number` using `failed_fields`; never retry the whole batch after any row has `write_executed=true`
+
+## Field Notes
+
+- `searchable_fields` on relation fields defines the backend-native searchable columns
+- `accepts_natural_input=true` means the field may accept a natural string before explicit id fallback
+- `may_become_required=true` means the field is writable now, but may turn required after linked visibility or option rules activate
+- `linkage.kind=logic_visibility` means the field is statically tied to linked visibility or option-driven rules
+- `linkage.kind=reference_fill` means the field participates in reference-based auto fill or default matching logic
+- `linkage.kind=formula_fill` means the field usually comes from formula/default auto-fill logic
+- `linkage.sources` lists the upstream field titles that influence the current field
+- `linkage.affects_fields` lists downstream field titles that may change when this field changes
+- `linkage.role=auto_fill_only` means "normally do not hand-fill this unless the product explicitly requires it"
+- `requires_upload=true` means upload the file first, then write the returned value
+- `failed_fields[].next_action` tells the next repair step for that row
+
+## CLI Pattern
+
+Use a JSON array file:
+
+```bash
+qingflow record insert --app-key APP_KEY --items-file records.json --json
+```
+
+`records.json`:
+
+```json
+[
+  { "fields": { "客户名称": "测试客户", "负责人": "张三" } }
+]
+```
+
+## Do Not
+
+- Do not skip `record schema insert`
+- Do not invent missing required fields
+- Do not flatten subtable leaf fields to the top level
+- Do not pre-query or silently guess member / department / relation ids when a natural string is enough
+- Do not retry a whole batch after `created_record_ids` is non-empty
+- Do not bind logic to a transient nested schema serialization detail when the field title and parent table already identify the legal payload shape