@josephyan/qingflow-app-user-mcp 1.0.5 → 1.0.7

package/README.md

@@ -3,13 +3,13 @@
 Install:
 
 ```bash
-npm install @josephyan/qingflow-app-user-mcp@1.0.5
+npm install @josephyan/qingflow-app-user-mcp@1.0.7
 ```
 
 Run:
 
 ```bash
-npx -y -p @josephyan/qingflow-app-user-mcp@1.0.5 qingflow-app-user-mcp
+npx -y -p @josephyan/qingflow-app-user-mcp@1.0.7 qingflow-app-user-mcp
 ```
 
 Environment:

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@josephyan/qingflow-app-user-mcp",
-  "version": "1.0.5",
+  "version": "1.0.7",
   "description": "Operational end-user MCP for Qingflow records, tasks, comments, and directory workflows.",
   "license": "MIT",
   "type": "module",

package/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "qingflow-mcp"
-version = "1.0.5"
+version = "1.0.7"
 description = "User-authenticated MCP server for Qingflow"
 readme = "README.md"
 license = "MIT"

package/skills/qingflow-record-analysis/SKILL.md

@@ -73,9 +73,9 @@ Use `record_access` to fetch detail rows into local CSV shards. It does not anal
 }
 ```
 
-Then run Python against every `files[].local_path`. CSV columns are stable: `record_id`, then `field_<field_id>`. Use `fields[]` metadata to map titles and types.
+Then run Python against every `files[].local_path`. CSV columns are stable: `record_id`, then `field_<field_id>`. Use `fields[]` plus `metadata_files.schema` / `metadata_files.readme` in the same output directory to map titles and types, especially when reusing the CSV later.
 
-- Never ask for `page`, `page_size`, `limit`, or `max_rows`; `record_access` owns paging internally.
+- Never ask for `page`, `page_size`, `limit`, or `max_rows`; `record_access` owns paging internally and follows the backend's native paging capability.
 - If multiple CSV files are returned, read them all.
 - If `complete=false` or `safe_for_final_conclusion=false`, downgrade the answer and disclose the limitation.
 - `record_export_direct` is only for explicit export/download/Excel requests, not default analysis.

package/skills/qingflow-record-analysis/references/analysis-gotchas.md

@@ -89,11 +89,12 @@ Examples of the right recovery question:
 
 ## Do not try to control paging manually
 
-`record_access` and `record_analyze` hide paging and scan budget on purpose.
+`record_access` hides paging and follows the backend's native paging capability. `record_analyze` hides paging and scan budget on purpose.
 
 - Do not invent `page_size`
 - Do not invent `requested_pages`
 - Do not invent `scan_max_pages`
+- Do not rename CSV columns before analysis; use `metadata_files.schema` / `metadata_files.readme` to interpret `field_<id>` columns.
 - Do not invent `auto_expand_pages`
 - Do not invent `max_rows`
 

package/skills/qingflow-record-analysis/references/analysis-patterns.md

@@ -28,9 +28,10 @@ Result reading order:
 1. `record_access.complete`
 2. `record_access.safe_for_final_conclusion`
 3. `record_access.files[].local_path`
-4. Python outputs
-5. `record_access.fields`
-6. `record_access.warnings`
+4. `record_access.metadata_files.schema`
+5. Python outputs
+6. `record_access.fields`
+7. `record_access.warnings`
 
 Treat `record_browse_schema_get` as the browse-schema source of truth. Missing fields are permission boundaries, not invitations to guess hidden ids.
 

package/src/qingflow_mcp/response_trim.py

@@ -455,11 +455,13 @@ def _trim_record_access(payload: JSONObject) -> None:
         "app_key",
         "view_id",
         "format",
+        "local_dir",
         "row_count",
         "complete",
         "truncated",
         "safe_for_final_conclusion",
         "files",
+        "metadata_files",
         "fields",
         "warnings",
         "verification",

package/src/qingflow_mcp/server_app_user.py

@@ -74,7 +74,7 @@ Inside `optional_fields`, any field with `may_become_required=true` is still wri
 
 Prefer `view_id` entries from `accessible_views` where `analysis_supported=true`.
 
-Use `record_access` to write local CSV shard files, then use Python to compute counts, rankings, ratios, trends, and final conclusions. `record_access` does not return bulk `items`; read `files[].local_path`.
+Use `record_access` to write local CSV shard files, then use Python to compute counts, rankings, ratios, trends, and final conclusions. `record_access` does not return bulk `items`; read `files[].local_path`. Use `fields[]` and the same-directory `metadata_files.schema` / `metadata_files.readme` to map stable `field_<id>` CSV columns back to titles and types.
 Use `chart_get` only when the user provides a report URL / chart_id or explicitly asks to read an existing report. Do not use QingBI as the default analysis route.
 
 Use this data-access DSL shape:

package/src/qingflow_mcp/tools/record_tools.py

@@ -32,9 +32,8 @@ from .directory_tools import _directory_has_more, _directory_items
 
 DEFAULT_QUERY_PAGE_SIZE = 50
 DEFAULT_LIST_PAGE_SIZE = 200
+BACKEND_RECORD_ACCESS_PAGE_SIZE = 1000
 DEFAULT_RECORD_ACCESS_SHARD_ROWS = 5000
-DEFAULT_RECORD_ACCESS_HARD_ROWS = 50000
-DEFAULT_RECORD_ACCESS_TIMEOUT_SECONDS = 60
 DEFAULT_ANALYSIS_PAGE_SIZE = 1000
 DEFAULT_SCAN_MAX_PAGES = 10
 DEFAULT_ANALYSIS_SCAN_MAX_PAGES = 100
@@ -1870,7 +1869,6 @@ class RecordTools(ToolBase):
                 primary_search_que_ids = primary_search_que_ids or None
             match_rules = self._resolve_match_rules(context, filters, index)
             sort_rules = self._resolve_sorts(sorts, index)
-            dept_member_cache: dict[int, set[int]] = {}
             used_list_type: int | None = None
             fallback_list_types = (
                 [view_route.list_type if view_route.list_type is not None else DEFAULT_RECORD_LIST_TYPE]
@@ -1934,17 +1932,9 @@ class RecordTools(ToolBase):
             current_page = 1
             has_more = False
             reported_total: int | None = None
-            limit_reached = False
-            timeout_reached = False
-            started_at = time.perf_counter()
+            created_at = datetime.now(UTC).isoformat()
             try:
                 while True:
-                    if row_count >= DEFAULT_RECORD_ACCESS_HARD_ROWS:
-                        limit_reached = True
-                        break
-                    if time.perf_counter() - started_at >= DEFAULT_RECORD_ACCESS_TIMEOUT_SECONDS:
-                        timeout_reached = True
-                        break
                     if used_list_type is None:
                         last_error: QingflowApiError | None = None
                         page: JSONObject | None = None
@@ -1955,7 +1945,7 @@ class RecordTools(ToolBase):
                                     app_key=app_key,
                                     view_selection=view_selection,
                                     page_num=current_page,
-                                    page_size=DEFAULT_LIST_PAGE_SIZE,
+                                    page_size=BACKEND_RECORD_ACCESS_PAGE_SIZE,
                                     query_key=None,
                                     match_rules=match_rules,
                                     sorts=sort_rules,
@@ -1982,7 +1972,7 @@ class RecordTools(ToolBase):
                             app_key=app_key,
                             view_selection=view_selection,
                             page_num=current_page,
-                            page_size=DEFAULT_LIST_PAGE_SIZE,
+                            page_size=BACKEND_RECORD_ACCESS_PAGE_SIZE,
                             query_key=None,
                             match_rules=match_rules,
                             sorts=sort_rules,
@@ -1992,25 +1982,15 @@ class RecordTools(ToolBase):
                     page_rows = page.get("list")
                     items = page_rows if isinstance(page_rows, list) else []
                     if reported_total is None:
-                        reported_total = _effective_total(page, DEFAULT_LIST_PAGE_SIZE)
-                    has_more = _page_has_more(page, current_page, DEFAULT_LIST_PAGE_SIZE, len(items))
+                        reported_total = _effective_total(page, BACKEND_RECORD_ACCESS_PAGE_SIZE)
+                    has_more = _page_has_more(page, current_page, BACKEND_RECORD_ACCESS_PAGE_SIZE, len(items))
                     page_apply_order: list[int] = []
                     page_answer_map: dict[int, list[JSONValue]] = {}
                     for item in items:
-                        if row_count + len(page_apply_order) >= DEFAULT_RECORD_ACCESS_HARD_ROWS:
-                            limit_reached = True
-                            break
                         if not isinstance(item, dict):
                             continue
                         answers = item.get("answers")
                         answer_list = answers if isinstance(answers, list) else []
-                        if not self._matches_view_selection(
-                            context,
-                            answer_list,
-                            view_selection=view_selection,
-                            dept_member_cache=dept_member_cache,
-                        ):
-                            continue
                         apply_id = _coerce_count(item.get("applyId")) or _coerce_count(item.get("id"))
                         if apply_id is None:
                             continue
@@ -2023,7 +2003,7 @@ class RecordTools(ToolBase):
                                 app_key=app_key,
                                 view_selection=view_selection,
                                 page_num=current_page,
-                                page_size=DEFAULT_LIST_PAGE_SIZE,
+                                page_size=BACKEND_RECORD_ACCESS_PAGE_SIZE,
                                 query_key=None,
                                 match_rules=match_rules,
                                 sorts=sort_rules,
@@ -2045,11 +2025,8 @@ class RecordTools(ToolBase):
                                     cast(list[JSONValue], extra_answer_list),
                                 )
                     for apply_id in page_apply_order:
-                        if row_count >= DEFAULT_RECORD_ACCESS_HARD_ROWS:
-                            limit_reached = True
-                            break
                         write_record(apply_id, page_answer_map.get(apply_id, []))
-                    if limit_reached or not has_more:
+                    if not has_more:
                         break
                     current_page += 1
             finally:
@@ -2069,24 +2046,29 @@ class RecordTools(ToolBase):
                         ),
                     }
                 )
-            if limit_reached:
-                warnings.append(
-                    {
-                        "code": "RECORD_ACCESS_INTERNAL_LIMIT_REACHED",
-                        "message": "record_access reached its internal row limit before all pages were fetched.",
-                    }
-                )
-            if timeout_reached:
-                warnings.append(
-                    {
-                        "code": "RECORD_ACCESS_TIMEOUT",
-                        "message": "record_access reached its internal time limit before all pages were fetched.",
-                    }
-                )
-            complete = not has_more and not limit_reached and not timeout_reached
+            complete = not has_more
             safe_for_final_conclusion = complete and not any(
                 warning.get("code") == "CUSTOM_VIEW_FILTER_UNVERIFIED" for warning in warnings
             )
+            fields_payload = [_record_access_field_payload(field) for field in selected_fields]
+            verification_payload: JSONObject = {
+                **_view_filter_verification_payload(view_route),
+                "reported_total": reported_total,
+                "list_type_used": used_list_type,
+            }
+            metadata_files = _write_record_access_metadata_files(
+                run_dir=run_dir,
+                created_at=created_at,
+                app_key=app_key,
+                view_route=view_route,
+                row_count=row_count,
+                complete=complete,
+                safe_for_final_conclusion=safe_for_final_conclusion,
+                files=files,
+                fields=fields_payload,
+                warnings=warnings,
+                verification=verification_payload,
+            )
             return {
                 "profile": profile,
                 "ws_id": session_profile.selected_ws_id,
@@ -2095,18 +2077,16 @@ class RecordTools(ToolBase):
                 "app_key": app_key,
                 "view_id": view_route.view_id,
                 "format": "csv",
+                "local_dir": str(run_dir),
                 "row_count": row_count,
                 "complete": complete,
                 "truncated": not complete,
                 "safe_for_final_conclusion": safe_for_final_conclusion,
                 "files": files,
-                "fields": [_record_access_field_payload(field) for field in selected_fields],
+                "metadata_files": metadata_files,
+                "fields": fields_payload,
                 "warnings": warnings,
-                "verification": {
-                    **_view_filter_verification_payload(view_route),
-                    "reported_total": reported_total,
-                    "list_type_used": used_list_type,
-                },
+                "verification": verification_payload,
                 "request_route": self._request_route_payload(context),
             }
 
@@ -5594,9 +5574,8 @@ class RecordTools(ToolBase):
         source_pages: list[int] = []
         result_amount: int | None = None
         has_more = False
-        dept_member_cache: dict[int, set[int]] = {}
         view_selection = resolved_view.view_selection
-        local_filtering = bool(filters) or bool(view_selection is not None and view_selection.conditions)
+        local_filtering = bool(filters)
         group_stats: dict[tuple[tuple[str, object], ...], JSONObject] = {}
         overall_metrics = self._initialize_metric_states(metrics)
         matched_rows = 0
@@ -5640,13 +5619,6 @@ class RecordTools(ToolBase):
                     continue
                 answers = item.get("answers")
                 answer_list = answers if isinstance(answers, list) else []
-                if not self._matches_view_selection(
-                    context,
-                    answer_list,
-                    view_selection=view_selection,
-                    dept_member_cache=dept_member_cache,
-                ):
-                    continue
                 if not self._matches_analyze_filters(answer_list, filters):
                     continue
                 matched_rows += 1
@@ -6732,7 +6704,6 @@ class RecordTools(ToolBase):
                 if view_selection is not None
                 else self._get_field_index(profile, context, app_key, force_refresh=False)
             )
-            dept_member_cache: dict[int, set[int]] = {}
             result = self._search_page(
                 context,
                 app_key=app_key,
@@ -6747,17 +6718,7 @@ class RecordTools(ToolBase):
             )
             rows = result.get("list")
             raw_rows = rows if isinstance(rows, list) else []
-            filtered_rows = [
-                item
-                for item in raw_rows
-                if isinstance(item, dict)
-                and self._matches_view_selection(
-                    context,
-                    item.get("answers") if isinstance(item.get("answers"), list) else [],
-                    view_selection=view_selection,
-                    dept_member_cache=dept_member_cache,
-                )
-            ]
+            filtered_rows = [item for item in raw_rows if isinstance(item, dict)]
             if isinstance(rows, list):
                 result = dict(result)
                 result["list"] = filtered_rows
@@ -7007,7 +6968,6 @@ class RecordTools(ToolBase):
         def runner(session_profile, context):
             index = self._get_field_index(profile, context, app_key, force_refresh=False)
             view_selection = self._resolve_view_selection(profile, context, app_key, view_key=view_key, view_name=view_name)
-            dept_member_cache: dict[int, set[int]] = {}
             resolved_column_cap = _bounded_column_limit(
                 max_columns,
                 default_limit=MAX_LIST_COLUMN_LIMIT,
@@ -7113,13 +7073,6 @@ class RecordTools(ToolBase):
                         continue
                     answers = item.get("answers")
                     answer_list = answers if isinstance(answers, list) else []
-                    if not self._matches_view_selection(
-                        context,
-                        answer_list,
-                        view_selection=view_selection,
-                        dept_member_cache=dept_member_cache,
-                    ):
-                        continue
                     matched_records += 1
                     apply_id = _coerce_count(item.get("applyId")) or _coerce_count(item.get("id"))
                     row = _build_flat_row(answer_list, selected_fields_from_primary, apply_id=apply_id)
@@ -11568,6 +11521,133 @@ def _record_access_field_payload(field: FormField) -> JSONObject:
     }
 
 
+def _write_record_access_metadata_files(
+    *,
+    run_dir: Path,
+    created_at: str,
+    app_key: str,
+    view_route: AccessibleViewRoute,
+    row_count: int,
+    complete: bool,
+    safe_for_final_conclusion: bool,
+    files: list[JSONObject],
+    fields: list[JSONObject],
+    warnings: list[JSONObject],
+    verification: JSONObject,
+) -> JSONObject:
+    schema_path = run_dir / "schema.json"
+    readme_path = run_dir / "README.md"
+    schema_payload: JSONObject = {
+        "created_at": created_at,
+        "app_key": app_key,
+        "view_id": view_route.view_id,
+        "view_name": view_route.name,
+        "view_type": view_route.view_type,
+        "format": "csv",
+        "row_count": row_count,
+        "complete": complete,
+        "truncated": not complete,
+        "safe_for_final_conclusion": safe_for_final_conclusion,
+        "files": files,
+        "fields": fields,
+        "warnings": warnings,
+        "verification": verification,
+        "csv_columns": ["record_id"] + [str(field["column_name"]) for field in fields if field.get("column_name")],
+    }
+    schema_path.write_text(json.dumps(schema_payload, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
+    readme_path.write_text(
+        _record_access_readme(
+            created_at=created_at,
+            app_key=app_key,
+            view_route=view_route,
+            row_count=row_count,
+            complete=complete,
+            safe_for_final_conclusion=safe_for_final_conclusion,
+            files=files,
+            fields=fields,
+            warnings=warnings,
+        ),
+        encoding="utf-8",
+    )
+    return {
+        "schema": str(schema_path),
+        "readme": str(readme_path),
+    }
+
+
+def _record_access_readme(
+    *,
+    created_at: str,
+    app_key: str,
+    view_route: AccessibleViewRoute,
+    row_count: int,
+    complete: bool,
+    safe_for_final_conclusion: bool,
+    files: list[JSONObject],
+    fields: list[JSONObject],
+    warnings: list[JSONObject],
+) -> str:
+    lines = [
+        "# Qingflow Record Access",
+        "",
+        f"- Created at: {created_at}",
+        f"- App key: `{app_key}`",
+        f"- View: {view_route.name} (`{view_route.view_id}`)",
+        f"- Format: CSV",
+        f"- Rows: {row_count}",
+        f"- Complete: {str(complete).lower()}",
+        f"- Safe for final conclusion: {str(safe_for_final_conclusion).lower()}",
+        "",
+        "## Data Files",
+        "",
+    ]
+    if files:
+        for file_info in files:
+            local_path = _normalize_optional_text(file_info.get("local_path")) or ""
+            part = file_info.get("part")
+            file_rows = file_info.get("row_count")
+            lines.append(f"- `{Path(local_path).name}`: part {part}, {file_rows} rows")
+    else:
+        lines.append("- No CSV shard files were written because the query returned 0 rows.")
+    lines.extend(
+        [
+            "",
+            "## Columns",
+            "",
+            "| CSV column | Field ID | Title | Type |",
+            "|---|---:|---|---|",
+            "| `record_id` | - | Record ID | record_id |",
+        ]
+    )
+    for field in fields:
+        column_name = _markdown_escape(str(field.get("column_name", "")))
+        field_id = field.get("field_id", "")
+        title = _markdown_escape(str(field.get("title", "")))
+        field_type = _markdown_escape(str(field.get("type", "")))
+        lines.append(f"| `{column_name}` | {field_id} | {title} | {field_type} |")
+    if warnings:
+        lines.extend(["", "## Warnings", ""])
+        for warning in warnings:
+            code = _normalize_optional_text(warning.get("code")) or "WARNING"
+            message = _normalize_optional_text(warning.get("message")) or ""
+            lines.append(f"- `{code}`: {message}")
+    lines.extend(
+        [
+            "",
+            "## Usage Notes",
+            "",
+            "- Use `records-*.csv` for Python or pandas processing; column names are stable and field-id based.",
+            "- Use `schema.json` or this README to map `field_<field_id>` columns back to Qingflow field titles and types.",
+            "",
+        ]
+    )
+    return "\n".join(lines)
+
+
+def _markdown_escape(value: str) -> str:
+    return value.replace("\\", "\\\\").replace("|", "\\|").replace("\n", " ")
+
+
 def _record_access_field_type(field: FormField) -> str:
     if field.que_type in DATE_QUE_TYPES:
         return "datetime"