@josephyan/qingflow-app-builder-mcp 0.2.0-beta.12 → 0.2.0-beta.14
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +2 -2
- package/package.json +1 -1
- package/pyproject.toml +1 -1
- package/skills/qingflow-app-builder/SKILL.md +49 -15
- package/skills/qingflow-app-builder/references/create-app.md +36 -0
- package/skills/qingflow-app-builder/references/gotchas.md +8 -0
- package/skills/qingflow-app-builder/references/tool-selection.md +16 -0
- package/skills/qingflow-app-builder/references/update-flow.md +17 -4
- package/src/qingflow_mcp/__init__.py +1 -1
- package/src/qingflow_mcp/builder_facade/service.py +71 -0
- package/src/qingflow_mcp/tools/record_tools.py +344 -7
package/README.md
CHANGED
|
@@ -3,13 +3,13 @@
|
|
|
3
3
|
Install:
|
|
4
4
|
|
|
5
5
|
```bash
|
|
6
|
-
npm install @josephyan/qingflow-app-builder-mcp@0.2.0-beta.
|
|
6
|
+
npm install @josephyan/qingflow-app-builder-mcp@0.2.0-beta.14
|
|
7
7
|
```
|
|
8
8
|
|
|
9
9
|
Run:
|
|
10
10
|
|
|
11
11
|
```bash
|
|
12
|
-
npx -y -p @josephyan/qingflow-app-builder-mcp@0.2.0-beta.
|
|
12
|
+
npx -y -p @josephyan/qingflow-app-builder-mcp@0.2.0-beta.14 qingflow-app-builder-mcp
|
|
13
13
|
```
|
|
14
14
|
|
|
15
15
|
Environment:
|
package/package.json
CHANGED
package/pyproject.toml
CHANGED
|
@@ -17,6 +17,29 @@ Before any write or verification flow, identify whether the task targets `test`
|
|
|
17
17
|
|
|
18
18
|
Pick the smallest tool layer that can finish the task.
|
|
19
19
|
|
|
20
|
+
## Mental Model
|
|
21
|
+
|
|
22
|
+
Model builder requests in four layers. Do not flatten them.
|
|
23
|
+
|
|
24
|
+
- `package`: the solution container or app bundle, for example “研发项目管理” or “费控管理系统”
|
|
25
|
+
- `app`: one form/app inside that package, for example “项目”, “需求”, “任务”, “缺陷”, “团队”
|
|
26
|
+
- `field`: one field inside one app
|
|
27
|
+
- `relation`: a field that links one app to another app
|
|
28
|
+
|
|
29
|
+
Interpret user intent with this hierarchy:
|
|
30
|
+
|
|
31
|
+
- If the user says “应用包”, “系统”, “包含多个表单”, “多个模块”, or asks for several named forms that relate to each other, treat it as a `package` with multiple `apps`
|
|
32
|
+
- Do not compress a multi-app system into one app with several text fields
|
|
33
|
+
- Names like “项目/需求/任务/缺陷/团队” or “费用申请/预算管理/报销审批” are usually separate apps, not text fields
|
|
34
|
+
- Build the apps first, then add `relation` fields to connect them
|
|
35
|
+
|
|
36
|
+
Default modeling rules:
|
|
37
|
+
|
|
38
|
+
- One business object -> one app
|
|
39
|
+
- Attributes of that object -> fields inside that app
|
|
40
|
+
- Another business object -> a separate app, not a text field
|
|
41
|
+
- Cross-object links -> relation fields, not text fields
|
|
42
|
+
|
|
20
43
|
- Authentication and workspace: `auth_*`, `workspace_*`
|
|
21
44
|
- File upload: `file_upload_local`
|
|
22
45
|
- Resource resolve/read: `package_list`, `package_resolve`, `package_create`, `builder_tool_contract`, `member_search`, `role_search`, `app_resolve`, `app_read_summary`, `app_read_fields`, `app_read_layout_summary`, `app_read_views_summary`, `app_read_flow_summary`
|
|
@@ -45,6 +68,7 @@ Default policy:
|
|
|
45
68
|
|
|
46
69
|
- Creating or updating one app inside an existing package: resolve the package/app, read compact state, plan patches on the server, then apply schema/layout/flow/views patches explicitly.
|
|
47
70
|
- If package creation looks necessary or beneficial, ask the user to confirm before calling `package_create`.
|
|
71
|
+
- If the user describes a system/package with multiple forms or modules, do not start with `app_schema_apply` on the package name. Resolve or create the package first, then create each app separately.
|
|
48
72
|
|
|
49
73
|
## Standard Operating Order
|
|
50
74
|
|
|
@@ -57,16 +81,19 @@ Before any business tool:
|
|
|
57
81
|
For builder work:
|
|
58
82
|
|
|
59
83
|
1. Resolve the target package with `package_resolve`; if resolution is ambiguous or you need a read-only fallback, use `package_list`. If you believe a new package should be created, ask the user to confirm before calling `package_create`.
|
|
60
|
-
2.
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
84
|
+
2. Decide whether the target is one app or a multi-app package:
|
|
85
|
+
- one app: continue with `app_resolve`
|
|
86
|
+
- multi-app package/system: create or resolve the package, then create each app separately before adding relations
|
|
87
|
+
3. Resolve the target app with `app_resolve` if the request is an update.
|
|
88
|
+
4. Read only the smallest summary you need: `app_read_summary`, `app_read_fields`, `app_read_layout_summary`, `app_read_views_summary`, `app_read_flow_summary`.
|
|
89
|
+
5. Use `app_schema_plan`, `app_layout_plan`, `app_flow_plan`, or `app_views_plan` before any non-trivial write.
|
|
90
|
+
6. Use `app_schema_apply` for create/upsert/remove field work. It publishes by default after the patch lands.
|
|
91
|
+
7. If the app must belong to a package, use `package_attach_app` explicitly after schema work unless readback already shows the target `tag_id`.
|
|
92
|
+
8. Use `app_layout_apply` only when the user is explicitly changing layout. Prefer the default `mode=merge`; use `mode=replace` only when you intend to place every field explicitly. It publishes by default.
|
|
93
|
+
9. Use `app_flow_apply` after schema exists. It publishes by default.
|
|
94
|
+
10. Use `app_views_apply` when the user wants explicit table/card/board/gantt views. It publishes by default.
|
|
95
|
+
11. Use `app_publish_verify` only when the user explicitly wants final publish/live verification or you need an explicit verification pass.
|
|
96
|
+
12. If a write fails with `APP_EDIT_LOCKED`, stop normal writes. Only use `app_release_edit_lock_if_mine` when the failed result shows the lock owner is the current logged-in user.
|
|
70
97
|
|
|
71
98
|
For view work, keep the order strict:
|
|
72
99
|
|
|
@@ -86,13 +113,17 @@ For flow work, keep the order strict:
|
|
|
86
113
|
5. `role_create` if the user wants a reusable role and no suitable role exists yet
|
|
87
114
|
6. Start from a canonical preset when possible
|
|
88
115
|
7. Use patch-style edits to that skeleton instead of freehand full-graph generation
|
|
89
|
-
8.
|
|
116
|
+
8. When patching a preset skeleton, reuse the preset node ids:
|
|
117
|
+
- `basic_approval` -> patch `approve_1`
|
|
118
|
+
- `basic_fill_then_approve` -> patch `fill_1` and `approve_1`
|
|
119
|
+
Do not invent a second approval/fill node id unless you are intentionally replacing the skeleton and removing the preset node.
|
|
120
|
+
9. Declare approver/fill/copy assignees explicitly:
|
|
90
121
|
- prefer `assignees.role_names`
|
|
91
122
|
- support `assignees.member_names` / `assignees.member_emails` / `assignees.member_uids`
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
123
|
+
10. When a node must edit specific fields, declare `permissions.editable_fields`
|
|
124
|
+
11. `app_flow_plan`
|
|
125
|
+
12. `app_flow_apply`
|
|
126
|
+
13. `app_read_flow_summary` after apply whenever the user asked for verification or apply returns `partial_success`
|
|
96
127
|
|
|
97
128
|
In `prod`, keep `plan` and `apply` as separate phases unless the user explicitly asks for a direct live execution.
|
|
98
129
|
|
|
@@ -109,9 +140,11 @@ For additive work on existing systems:
|
|
|
109
140
|
- `app_schema_apply` is the only public patch tool allowed to create an app shell; `app_layout_apply`, `app_flow_apply`, and `app_views_apply` require an existing app.
|
|
110
141
|
- Prefer `*_plan` before `*_apply`; the plan tools do server-side normalization and return the next executable call skeleton.
|
|
111
142
|
- For abstract requests like “默认视图”, “基础审批流”, “灵活流程”, or “美观布局”, first translate the intent into a stable preset or explicit patch. Do not send those phrases to MCP unchanged.
|
|
143
|
+
- If the user asks for a business system or package that contains several forms, do not use the system/package name as `app_name` and do not try to store the child app names as text fields.
|
|
112
144
|
- For flexible workflow requests, split the work into two steps:
|
|
113
145
|
1. create a base skeleton with a preset
|
|
114
146
|
2. apply explicit business-specific changes as patchable nodes/transitions
|
|
147
|
+
- For preset-based flows, treat preset node ids as part of the public contract. Patch the skeleton nodes by the same ids instead of creating a parallel node with a new id and leaving the preset node unassigned.
|
|
115
148
|
- Approval, fill, and copy nodes must declare at least one assignee. Treat this as a hard requirement, not an optional detail.
|
|
116
149
|
- For workflow nodes, use the canonical public shape:
|
|
117
150
|
- `assignees.role_names`
|
|
@@ -123,6 +156,7 @@ For additive work on existing systems:
|
|
|
123
156
|
- `app_schema_apply` does not treat package attachment as success criteria; if package ownership matters, verify `tag_ids_after` and call `package_attach_app` explicitly.
|
|
124
157
|
- `package_attach_app` is the source of truth for package ownership; do not assume app creation or publish implicitly attaches the app.
|
|
125
158
|
- `relation` and `subtable` must be explicit; do not infer them from vague natural language.
|
|
159
|
+
- Another app is not a field. If two business objects should both have their own records, build two apps and connect them with relation fields.
|
|
126
160
|
- In `prod`, prefer explicit patch tools and avoid any speculative create flow.
|
|
127
161
|
- Never try to bypass collaborative edit locks. `app_release_edit_lock_if_mine` is only for the case where the lock owner is the current authenticated user.
|
|
128
162
|
|
|
@@ -2,6 +2,18 @@
|
|
|
2
2
|
|
|
3
3
|
Use this when the user wants one new app inside an existing package.
|
|
4
4
|
|
|
5
|
+
Do not use this playbook when the user is really asking for a system/package with multiple forms or modules. In that case:
|
|
6
|
+
|
|
7
|
+
1. resolve or create the package
|
|
8
|
+
2. create each app separately
|
|
9
|
+
3. attach each app to the package
|
|
10
|
+
4. add relation fields between apps
|
|
11
|
+
|
|
12
|
+
Hierarchy reminder:
|
|
13
|
+
|
|
14
|
+
- package -> app -> field -> relation
|
|
15
|
+
- another business object is another app, not a text field
|
|
16
|
+
|
|
5
17
|
If creating a brand new package would help, ask the user to confirm package creation first. After confirmation, call `package_create` before this sequence.
|
|
6
18
|
|
|
7
19
|
## Minimal sequence
|
|
@@ -13,6 +25,22 @@ If creating a brand new package would help, ask the user to confirm package crea
|
|
|
13
25
|
5. `package_attach_app`
|
|
14
26
|
6. `app_publish_verify` only when the user asks for explicit final verification
|
|
15
27
|
|
|
28
|
+
## Multi-app systems are different
|
|
29
|
+
|
|
30
|
+
If the user says things like:
|
|
31
|
+
|
|
32
|
+
- “创建一个完整应用包”
|
|
33
|
+
- “包含项目、需求、任务、缺陷、团队这几个表单”
|
|
34
|
+
- “这些表单之间建立关联关系”
|
|
35
|
+
|
|
36
|
+
then do not treat that as one app.
|
|
37
|
+
|
|
38
|
+
Use this pattern instead:
|
|
39
|
+
|
|
40
|
+
1. `package_resolve` or `package_create`
|
|
41
|
+
2. for each app name, run `app_schema_plan -> app_schema_apply -> package_attach_app`
|
|
42
|
+
3. once the apps exist, add `relation` fields between them
|
|
43
|
+
|
|
16
44
|
## Example
|
|
17
45
|
|
|
18
46
|
Create a new package only after the user confirms package creation:
|
|
@@ -110,3 +138,11 @@ The create route did not resolve in the current backend route context. Re-run `w
|
|
|
110
138
|
### `PACKAGE_ATTACH_FAILED`
|
|
111
139
|
|
|
112
140
|
Do not retry schema creation. Re-run only `package_attach_app`, then verify with `app_read_summary`.
|
|
141
|
+
|
|
142
|
+
### Hierarchy modeling mistake
|
|
143
|
+
|
|
144
|
+
If the user asked for several named forms/apps but the draft patch turns them into text fields inside one app, stop and remodel the task as:
|
|
145
|
+
|
|
146
|
+
- one package
|
|
147
|
+
- many apps
|
|
148
|
+
- relation fields between those apps
|
|
@@ -11,6 +11,13 @@
|
|
|
11
11
|
- Treat `package_attach_app` as the source of truth for package ownership
|
|
12
12
|
- Check `tag_ids_after` after schema writes
|
|
13
13
|
|
|
14
|
+
## Package vs app vs field
|
|
15
|
+
|
|
16
|
+
- A package/system is not an app
|
|
17
|
+
- Another app is not a field
|
|
18
|
+
- If the user names multiple forms/modules that relate to each other, create multiple apps and connect them with relation fields
|
|
19
|
+
- Do not use child app names like “项目/需求/任务/缺陷/团队” as plain text fields inside one app
|
|
20
|
+
|
|
14
21
|
## Auto publish
|
|
15
22
|
|
|
16
23
|
- `app_schema_apply`, `app_layout_apply`, `app_flow_apply`, and `app_views_apply` publish by default
|
|
@@ -32,6 +39,7 @@
|
|
|
32
39
|
- Prefer roles over explicit members unless the user explicitly names people
|
|
33
40
|
- Resolve assignees with `role_search` / `member_search` before flow apply
|
|
34
41
|
- Use `permissions.editable_fields` for node-level editable field permissions; do not guess field ids
|
|
42
|
+
- Preset node ids matter. When patching `basic_approval` or `basic_fill_then_approve`, reuse `approve_1` and `fill_1` unless you are explicitly replacing the skeleton.
|
|
35
43
|
- If `app_flow_plan` reports `FLOW_DEPENDENCY_MISSING`, fix schema first
|
|
36
44
|
- Do not switch to hidden `solution_*` tools from public builder flows
|
|
37
45
|
|
|
@@ -8,6 +8,17 @@ Use the smallest v2 builder tool chain that can finish the task.
|
|
|
8
8
|
|
|
9
9
|
Use `plan` before `apply` unless the patch is trivial and already normalized.
|
|
10
10
|
|
|
11
|
+
## Hierarchy first
|
|
12
|
+
|
|
13
|
+
Before picking tools, decide which layer the request targets:
|
|
14
|
+
|
|
15
|
+
- `package`: a solution/app bundle like “研发项目管理” or “费控管理系统”
|
|
16
|
+
- `app`: one form/app inside that package
|
|
17
|
+
- `field`: one field inside one app
|
|
18
|
+
- `relation`: a field that links two apps
|
|
19
|
+
|
|
20
|
+
If the user asks for multiple forms/modules that relate to each other, this is a package-level multi-app task, not a single-app create.
|
|
21
|
+
|
|
11
22
|
## Resolve
|
|
12
23
|
|
|
13
24
|
- `package_create`: create a new package only after the user confirms package creation; exact-name duplicates return `noop=true`
|
|
@@ -57,6 +68,8 @@ These execute normalized patches and publish by default unless `publish=false`.
|
|
|
57
68
|
`package_resolve -> app_resolve -> app_schema_plan -> app_schema_apply -> package_attach_app`
|
|
58
69
|
- Create a brand new package, then create one app in it:
|
|
59
70
|
`package_create -> package_resolve -> app_schema_plan -> app_schema_apply -> package_attach_app`
|
|
71
|
+
- Create a brand new multi-app system/package:
|
|
72
|
+
`package_create/resolve -> per-app app_schema_plan/apply -> package_attach_app per app -> relation field patches`
|
|
60
73
|
- Update fields on an existing app:
|
|
61
74
|
`app_resolve -> app_read_fields -> app_schema_plan -> app_schema_apply`
|
|
62
75
|
- Tidy layout:
|
|
@@ -71,9 +84,12 @@ These execute normalized patches and publish by default unless `publish=false`.
|
|
|
71
84
|
- Do not handcraft raw Qingflow schema payloads
|
|
72
85
|
- Do not rely on internal `solution_*` tools in public builder flows
|
|
73
86
|
- Do not create a new package without first asking the user to confirm package creation
|
|
87
|
+
- Do not treat a package/system name as `app_name` when the user clearly wants multiple apps inside it
|
|
88
|
+
- Do not compress multiple business objects into one app with several text fields
|
|
74
89
|
- Do not skip summary reads before flow or view work
|
|
75
90
|
- Do not emit `column_names`; always use `columns`
|
|
76
91
|
- Do not reuse internal flow keys such as `role_entries` or `editable_que_ids` in public builder calls
|
|
77
92
|
- Do not pass natural-language preset guesses such as `default_approval`; map them to canonical preset values first
|
|
78
93
|
- Do not omit assignees on approval/fill/copy nodes
|
|
94
|
+
- Do not patch preset flows with brand new approval/fill node ids unless you are intentionally replacing the skeleton; reuse preset ids like `approve_1` and `fill_1`
|
|
79
95
|
- Do not guess role ids, member ids, or editable field ids; resolve names first
|
|
@@ -11,9 +11,14 @@ Use this when the app already exists and the task is only about workflow.
|
|
|
11
11
|
5. `role_create` if the user wants a reusable directory role and no good role exists
|
|
12
12
|
6. start from a canonical preset when possible
|
|
13
13
|
7. patch the skeleton instead of freehanding a full graph
|
|
14
|
-
8.
|
|
15
|
-
|
|
16
|
-
|
|
14
|
+
8. reuse preset node ids when patching:
|
|
15
|
+
- `basic_approval` -> patch `approve_1`
|
|
16
|
+
- `basic_fill_then_approve` -> patch `fill_1` and `approve_1`
|
|
17
|
+
Do not add a second approval/fill node with a new id unless you are intentionally replacing the skeleton.
|
|
18
|
+
The MCP now auto-aligns the simplest single-node preset overrides, but still prefer explicit preset ids so the merged graph stays predictable.
|
|
19
|
+
9. `app_flow_plan`
|
|
20
|
+
10. `app_flow_apply`
|
|
21
|
+
11. `app_read_flow_summary` when apply returns `partial_success` or the user asked for verification
|
|
17
22
|
|
|
18
23
|
If you are unsure about presets or node shapes, call `builder_tool_contract(tool_name="app_flow_apply")` before guessing.
|
|
19
24
|
|
|
@@ -145,6 +150,7 @@ Only after that should you use explicit nodes:
|
|
|
145
150
|
```
|
|
146
151
|
|
|
147
152
|
After `app_flow_plan` succeeds, prefer reusing its `suggested_next_call.arguments` directly. Do not rewrite the result into internal fields such as `role_entries` or `editable_que_ids`.
|
|
153
|
+
When you patch a preset, patch the preset node itself. Do not leave the preset approval node unassigned while adding a second custom approval node.
|
|
148
154
|
|
|
149
155
|
## Common failures
|
|
150
156
|
|
|
@@ -152,12 +158,19 @@ After `app_flow_plan` succeeds, prefer reusing its `suggested_next_call.argument
|
|
|
152
158
|
|
|
153
159
|
Approval, fill, and copy nodes must declare at least one assignee.
|
|
154
160
|
|
|
161
|
+
If this happens after using a preset, check for this specific mistake first:
|
|
162
|
+
|
|
163
|
+
- the preset created `approve_1` or `fill_1`
|
|
164
|
+
- your patch created a different node id instead of patching that preset node
|
|
165
|
+
- the original preset node remained in the graph without assignees
|
|
166
|
+
|
|
155
167
|
Preferred fix order:
|
|
156
168
|
|
|
157
169
|
1. `role_search`
|
|
158
170
|
2. `member_search` only if the user explicitly named members
|
|
159
171
|
3. `role_create` if the business needs a reusable role
|
|
160
|
-
4.
|
|
172
|
+
4. patch the preset node id itself with canonical `assignees.*`
|
|
173
|
+
5. retry `app_flow_plan` or `app_flow_apply`
|
|
161
174
|
|
|
162
175
|
### `FLOW_DEPENDENCY_MISSING`
|
|
163
176
|
|
|
@@ -3624,6 +3624,11 @@ def _merge_flow_graph(
|
|
|
3624
3624
|
) -> tuple[list[dict[str, Any]], list[dict[str, Any]]]:
|
|
3625
3625
|
if not override_nodes and not override_transitions:
|
|
3626
3626
|
return deepcopy(base_nodes), deepcopy(base_transitions)
|
|
3627
|
+
override_nodes, override_transitions = _align_flow_preset_override_ids(
|
|
3628
|
+
base_nodes=base_nodes,
|
|
3629
|
+
override_nodes=override_nodes,
|
|
3630
|
+
override_transitions=override_transitions,
|
|
3631
|
+
)
|
|
3627
3632
|
merged_nodes: list[dict[str, Any]] = []
|
|
3628
3633
|
override_map = {
|
|
3629
3634
|
str(node.get("id") or ""): deepcopy(node)
|
|
@@ -3654,6 +3659,72 @@ def _merge_flow_graph(
|
|
|
3654
3659
|
return merged_nodes, merged_transitions
|
|
3655
3660
|
|
|
3656
3661
|
|
|
3662
|
+
def _align_flow_preset_override_ids(
|
|
3663
|
+
*,
|
|
3664
|
+
base_nodes: list[dict[str, Any]],
|
|
3665
|
+
override_nodes: list[dict[str, Any]],
|
|
3666
|
+
override_transitions: list[dict[str, Any]],
|
|
3667
|
+
) -> tuple[list[dict[str, Any]], list[dict[str, Any]]]:
|
|
3668
|
+
"""Map simple preset skeleton overrides back onto canonical preset node ids.
|
|
3669
|
+
|
|
3670
|
+
This keeps preset-based plans stable when the caller customizes the single
|
|
3671
|
+
approval/fill/copy step semantically but forgets to reuse ids such as
|
|
3672
|
+
`approve_1` or `fill_1`.
|
|
3673
|
+
"""
|
|
3674
|
+
patchable_types = {"approve", "fill", "copy"}
|
|
3675
|
+
base_by_type: dict[str, list[str]] = {}
|
|
3676
|
+
override_by_type: dict[str, list[str]] = {}
|
|
3677
|
+
for node in base_nodes:
|
|
3678
|
+
if not isinstance(node, dict):
|
|
3679
|
+
continue
|
|
3680
|
+
node_type = str(node.get("type") or "")
|
|
3681
|
+
node_id = str(node.get("id") or "")
|
|
3682
|
+
if node_type in patchable_types and node_id:
|
|
3683
|
+
base_by_type.setdefault(node_type, []).append(node_id)
|
|
3684
|
+
for node in override_nodes:
|
|
3685
|
+
if not isinstance(node, dict):
|
|
3686
|
+
continue
|
|
3687
|
+
node_type = str(node.get("type") or "")
|
|
3688
|
+
node_id = str(node.get("id") or "")
|
|
3689
|
+
if node_type in patchable_types and node_id:
|
|
3690
|
+
override_by_type.setdefault(node_type, []).append(node_id)
|
|
3691
|
+
replacement_map: dict[str, str] = {}
|
|
3692
|
+
for node_type in patchable_types:
|
|
3693
|
+
base_ids = base_by_type.get(node_type) or []
|
|
3694
|
+
override_ids = override_by_type.get(node_type) or []
|
|
3695
|
+
if len(base_ids) != 1 or len(override_ids) != 1:
|
|
3696
|
+
continue
|
|
3697
|
+
base_id = base_ids[0]
|
|
3698
|
+
override_id = override_ids[0]
|
|
3699
|
+
if override_id == base_id:
|
|
3700
|
+
continue
|
|
3701
|
+
replacement_map[override_id] = base_id
|
|
3702
|
+
if not replacement_map:
|
|
3703
|
+
return deepcopy(override_nodes), deepcopy(override_transitions)
|
|
3704
|
+
aligned_nodes: list[dict[str, Any]] = []
|
|
3705
|
+
for node in override_nodes:
|
|
3706
|
+
if not isinstance(node, dict):
|
|
3707
|
+
continue
|
|
3708
|
+
aligned = deepcopy(node)
|
|
3709
|
+
node_id = str(aligned.get("id") or "")
|
|
3710
|
+
if node_id in replacement_map:
|
|
3711
|
+
aligned["id"] = replacement_map[node_id]
|
|
3712
|
+
aligned_nodes.append(aligned)
|
|
3713
|
+
aligned_transitions: list[dict[str, Any]] = []
|
|
3714
|
+
for transition in override_transitions:
|
|
3715
|
+
if not isinstance(transition, dict):
|
|
3716
|
+
continue
|
|
3717
|
+
aligned = deepcopy(transition)
|
|
3718
|
+
source = str(aligned.get("from") or "")
|
|
3719
|
+
target = str(aligned.get("to") or "")
|
|
3720
|
+
if source in replacement_map:
|
|
3721
|
+
aligned["from"] = replacement_map[source]
|
|
3722
|
+
if target in replacement_map:
|
|
3723
|
+
aligned["to"] = replacement_map[target]
|
|
3724
|
+
aligned_transitions.append(aligned)
|
|
3725
|
+
return aligned_nodes, aligned_transitions
|
|
3726
|
+
|
|
3727
|
+
|
|
3657
3728
|
def _extract_directory_items(listed: JSONObject) -> list[dict[str, Any]]:
|
|
3658
3729
|
if isinstance(listed.get("items"), list):
|
|
3659
3730
|
return [item for item in listed["items"] if isinstance(item, dict)]
|
|
@@ -16,7 +16,10 @@ from .base import ToolBase
|
|
|
16
16
|
|
|
17
17
|
|
|
18
18
|
DEFAULT_QUERY_PAGE_SIZE = 50
|
|
19
|
+
DEFAULT_ANALYSIS_PAGE_SIZE = 100
|
|
19
20
|
DEFAULT_SCAN_MAX_PAGES = 10
|
|
21
|
+
DEFAULT_ANALYSIS_SCAN_MAX_PAGES = 20
|
|
22
|
+
DEFAULT_ANALYSIS_AUTO_EXPAND_PAGE_CAP = 50
|
|
20
23
|
DEFAULT_ROW_LIMIT = 200
|
|
21
24
|
DEFAULT_OUTPUT_PROFILE = "compact"
|
|
22
25
|
MAX_LIST_COLUMN_LIMIT = 20
|
|
@@ -208,6 +211,7 @@ class RecordTools(ToolBase):
|
|
|
208
211
|
page_size: int = DEFAULT_QUERY_PAGE_SIZE,
|
|
209
212
|
requested_pages: int = 1,
|
|
210
213
|
scan_max_pages: int = DEFAULT_SCAN_MAX_PAGES,
|
|
214
|
+
auto_expand_pages: bool = False,
|
|
211
215
|
query_key: str | None = None,
|
|
212
216
|
filters: list[JSONObject] | None = None,
|
|
213
217
|
sorts: list[JSONObject] | None = None,
|
|
@@ -232,6 +236,7 @@ class RecordTools(ToolBase):
|
|
|
232
236
|
page_size=page_size,
|
|
233
237
|
requested_pages=requested_pages,
|
|
234
238
|
scan_max_pages=scan_max_pages,
|
|
239
|
+
auto_expand_pages=auto_expand_pages,
|
|
235
240
|
query_key=query_key,
|
|
236
241
|
filters=filters or [],
|
|
237
242
|
sorts=sorts or [],
|
|
@@ -261,9 +266,10 @@ class RecordTools(ToolBase):
|
|
|
261
266
|
amount_column: str | int | None = None,
|
|
262
267
|
metrics: list[str] | None = None,
|
|
263
268
|
page_num: int = 1,
|
|
264
|
-
page_size: int =
|
|
269
|
+
page_size: int = DEFAULT_ANALYSIS_PAGE_SIZE,
|
|
265
270
|
requested_pages: int = 1,
|
|
266
|
-
scan_max_pages: int =
|
|
271
|
+
scan_max_pages: int = DEFAULT_ANALYSIS_SCAN_MAX_PAGES,
|
|
272
|
+
auto_expand_pages: bool = False,
|
|
267
273
|
query_key: str | None = None,
|
|
268
274
|
filters: list[JSONObject] | None = None,
|
|
269
275
|
sorts: list[JSONObject] | None = None,
|
|
@@ -286,6 +292,7 @@ class RecordTools(ToolBase):
|
|
|
286
292
|
page_size=page_size,
|
|
287
293
|
requested_pages=requested_pages,
|
|
288
294
|
scan_max_pages=scan_max_pages,
|
|
295
|
+
auto_expand_pages=auto_expand_pages,
|
|
289
296
|
query_key=query_key,
|
|
290
297
|
filters=filters or [],
|
|
291
298
|
sorts=sorts or [],
|
|
@@ -422,6 +429,7 @@ class RecordTools(ToolBase):
|
|
|
422
429
|
def runner(session_profile, context):
|
|
423
430
|
field_mapping: list[JSONObject] = []
|
|
424
431
|
view_resolution: JSONObject | None = None
|
|
432
|
+
resolved_view_selection: ViewSelection | None = None
|
|
425
433
|
if resolve_fields and isinstance(normalized.get("app_key"), str) and normalized.get("app_key"):
|
|
426
434
|
index = self._get_field_index(profile, context, cast(str, normalized["app_key"]), force_refresh=False)
|
|
427
435
|
for candidate in _collect_plan_field_candidates(tool, normalized):
|
|
@@ -434,6 +442,7 @@ class RecordTools(ToolBase):
|
|
|
434
442
|
view_name=_normalize_optional_text(normalized.get("view_name")),
|
|
435
443
|
)
|
|
436
444
|
if view_selection is not None:
|
|
445
|
+
resolved_view_selection = view_selection
|
|
437
446
|
view_resolution = {
|
|
438
447
|
"resolved": True,
|
|
439
448
|
"view_key": view_selection.view_key,
|
|
@@ -441,7 +450,16 @@ class RecordTools(ToolBase):
|
|
|
441
450
|
"local_filtering": bool(view_selection.conditions),
|
|
442
451
|
"condition_group_count": len(view_selection.conditions),
|
|
443
452
|
}
|
|
444
|
-
|
|
453
|
+
probe = None
|
|
454
|
+
if tool in {"record_query", "record_aggregate"} and isinstance(normalized.get("app_key"), str) and normalized.get("app_key"):
|
|
455
|
+
probe = self._build_analysis_probe(
|
|
456
|
+
profile,
|
|
457
|
+
context,
|
|
458
|
+
app_key=cast(str, normalized["app_key"]),
|
|
459
|
+
arguments=normalized,
|
|
460
|
+
view_selection=resolved_view_selection,
|
|
461
|
+
)
|
|
462
|
+
estimate = _build_plan_estimate(tool, normalized, probe=probe)
|
|
445
463
|
readiness = _assess_plan_readiness(tool, normalized, validation, field_mapping, estimate)
|
|
446
464
|
return {
|
|
447
465
|
"profile": profile,
|
|
@@ -458,6 +476,13 @@ class RecordTools(ToolBase):
|
|
|
458
476
|
"ready_for_final_conclusion": readiness["ready_for_final_conclusion"],
|
|
459
477
|
"final_conclusion_blockers": readiness["final_conclusion_blockers"],
|
|
460
478
|
"recommended_next_actions": readiness["recommended_next_actions"],
|
|
479
|
+
"suggested_next_call": {
|
|
480
|
+
"tool_name": tool,
|
|
481
|
+
"arguments": {
|
|
482
|
+
**normalized,
|
|
483
|
+
**_callable_analysis_arguments(cast(JSONObject, estimate.get("recommended_arguments") if isinstance(estimate.get("recommended_arguments"), dict) else {})),
|
|
484
|
+
},
|
|
485
|
+
},
|
|
461
486
|
},
|
|
462
487
|
}
|
|
463
488
|
|
|
@@ -606,6 +631,7 @@ class RecordTools(ToolBase):
|
|
|
606
631
|
page_size: int,
|
|
607
632
|
requested_pages: int,
|
|
608
633
|
scan_max_pages: int,
|
|
634
|
+
auto_expand_pages: bool = False,
|
|
609
635
|
query_key: str | None,
|
|
610
636
|
filters: list[JSONObject],
|
|
611
637
|
sorts: list[JSONObject],
|
|
@@ -640,6 +666,7 @@ class RecordTools(ToolBase):
|
|
|
640
666
|
page_size=page_size,
|
|
641
667
|
requested_pages=requested_pages,
|
|
642
668
|
scan_max_pages=scan_max_pages,
|
|
669
|
+
auto_expand_pages=auto_expand_pages,
|
|
643
670
|
query_key=query_key,
|
|
644
671
|
filters=filters,
|
|
645
672
|
sorts=sorts,
|
|
@@ -687,6 +714,7 @@ class RecordTools(ToolBase):
|
|
|
687
714
|
page_size: int,
|
|
688
715
|
requested_pages: int,
|
|
689
716
|
scan_max_pages: int,
|
|
717
|
+
auto_expand_pages: bool = False,
|
|
690
718
|
query_key: str | None,
|
|
691
719
|
filters: list[JSONObject],
|
|
692
720
|
sorts: list[JSONObject],
|
|
@@ -725,6 +753,14 @@ class RecordTools(ToolBase):
|
|
|
725
753
|
has_more = False
|
|
726
754
|
group_stats: dict[str, JSONObject] = {}
|
|
727
755
|
total_amount = 0.0
|
|
756
|
+
scan_control: JSONObject = {
|
|
757
|
+
"requested_pages": max(requested_pages, 1),
|
|
758
|
+
"scan_max_pages": max(scan_max_pages, 1),
|
|
759
|
+
"auto_expand_pages": auto_expand_pages,
|
|
760
|
+
"auto_expand_applied": False,
|
|
761
|
+
"auto_expand_target_pages": pages_to_scan,
|
|
762
|
+
"auto_expand_page_cap": DEFAULT_ANALYSIS_AUTO_EXPAND_PAGE_CAP,
|
|
763
|
+
}
|
|
728
764
|
while scanned_pages < pages_to_scan:
|
|
729
765
|
page = self._search_page(
|
|
730
766
|
context,
|
|
@@ -743,6 +779,13 @@ class RecordTools(ToolBase):
|
|
|
743
779
|
items = rows if isinstance(rows, list) else []
|
|
744
780
|
if result_amount is None:
|
|
745
781
|
result_amount = _effective_total(page, page_size)
|
|
782
|
+
pages_to_scan, scan_control = _compute_scan_limit(
|
|
783
|
+
requested_pages=requested_pages,
|
|
784
|
+
scan_max_pages=scan_max_pages,
|
|
785
|
+
auto_expand_pages=auto_expand_pages,
|
|
786
|
+
page=page,
|
|
787
|
+
page_size=page_size,
|
|
788
|
+
)
|
|
746
789
|
has_more = _page_has_more(page, current_page, page_size, len(items))
|
|
747
790
|
for item in items:
|
|
748
791
|
if not isinstance(item, dict):
|
|
@@ -828,6 +871,13 @@ class RecordTools(ToolBase):
|
|
|
828
871
|
amount_total = _coerce_amount(item.get("amount_total"))
|
|
829
872
|
item["amount_ratio"] = (amount_total / total_amount) if amount_total is not None else None
|
|
830
873
|
effective_result_amount = scanned_records if view_selection is not None else (result_amount or scanned_records)
|
|
874
|
+
grouped_count = sum(int(item.get("count") or 0) for item in groups)
|
|
875
|
+
analysis_counts = _build_analysis_counts(
|
|
876
|
+
backend_total_count=result_amount,
|
|
877
|
+
scanned_count=scanned_records,
|
|
878
|
+
grouped_count=grouped_count,
|
|
879
|
+
local_filtering=view_selection is not None and bool(view_selection.conditions),
|
|
880
|
+
)
|
|
831
881
|
completeness = _build_completeness(
|
|
832
882
|
result_amount=effective_result_amount,
|
|
833
883
|
returned_items=len(groups),
|
|
@@ -846,6 +896,7 @@ class RecordTools(ToolBase):
|
|
|
846
896
|
"raw_next_page_token": None,
|
|
847
897
|
"output_next_page_token": None,
|
|
848
898
|
"stop_reason": "source_exhausted" if not has_more else "scan_limit",
|
|
899
|
+
**scan_control,
|
|
849
900
|
},
|
|
850
901
|
)
|
|
851
902
|
evidence = {
|
|
@@ -859,20 +910,65 @@ class RecordTools(ToolBase):
|
|
|
859
910
|
}
|
|
860
911
|
if strict_full and not bool(completeness.get("raw_scan_complete")):
|
|
861
912
|
self._raise_need_more_data(completeness, evidence, "Aggregate result is incomplete; increase requested_pages or scan_max_pages.")
|
|
913
|
+
analysis_status, raw_scan_complete = _analysis_status_from_completeness(completeness)
|
|
914
|
+
suggested_next_call = None
|
|
915
|
+
if analysis_status != "success":
|
|
916
|
+
suggested_next_call = {
|
|
917
|
+
"tool_name": "record_aggregate",
|
|
918
|
+
"arguments": {
|
|
919
|
+
"app_key": app_key,
|
|
920
|
+
"group_by": [field.que_title for field in group_fields],
|
|
921
|
+
"page_num": max(page_num, 1),
|
|
922
|
+
"page_size": page_size,
|
|
923
|
+
"requested_pages": max(requested_pages, 1),
|
|
924
|
+
"scan_max_pages": max(scan_max_pages, 1),
|
|
925
|
+
"auto_expand_pages": auto_expand_pages,
|
|
926
|
+
"query_key": query_key,
|
|
927
|
+
"filters": filters,
|
|
928
|
+
"sorts": sorts,
|
|
929
|
+
"time_range": time_range or {},
|
|
930
|
+
"amount_column": amount_field.que_title if amount_field is not None else amount_column,
|
|
931
|
+
"metrics": metric_names,
|
|
932
|
+
"strict_full": strict_full,
|
|
933
|
+
"list_type": list_type,
|
|
934
|
+
"view_key": view_selection.view_key if view_selection is not None else view_key,
|
|
935
|
+
"view_name": view_selection.view_name if view_selection is not None else view_name,
|
|
936
|
+
**_callable_analysis_arguments(
|
|
937
|
+
_recommended_analysis_arguments(
|
|
938
|
+
tool="record_aggregate",
|
|
939
|
+
arguments={
|
|
940
|
+
"page_size": page_size,
|
|
941
|
+
"requested_pages": requested_pages,
|
|
942
|
+
"scan_max_pages": scan_max_pages,
|
|
943
|
+
},
|
|
944
|
+
probe={"backend_total_count": result_amount},
|
|
945
|
+
routed_mode="summary",
|
|
946
|
+
)
|
|
947
|
+
),
|
|
948
|
+
},
|
|
949
|
+
}
|
|
862
950
|
response: JSONObject = {
|
|
863
951
|
"profile": profile,
|
|
864
952
|
"ws_id": session_profile.selected_ws_id,
|
|
865
953
|
"ok": True,
|
|
954
|
+
"status": analysis_status,
|
|
866
955
|
"request_route": self._request_route_payload(context),
|
|
867
956
|
"data": {
|
|
868
957
|
"app_key": app_key,
|
|
958
|
+
"analysis_status": analysis_status,
|
|
959
|
+
"safe_for_final_conclusion": raw_scan_complete and bool(strict_full),
|
|
869
960
|
"view": _view_selection_payload(view_selection),
|
|
870
961
|
"summary": {
|
|
871
|
-
"total_count":
|
|
962
|
+
"total_count": effective_result_amount,
|
|
963
|
+
"backend_total_count": result_amount,
|
|
964
|
+
"scanned_count": scanned_records,
|
|
965
|
+
"grouped_count": grouped_count,
|
|
872
966
|
"total_amount": total_amount if amount_field is not None else None,
|
|
873
967
|
},
|
|
968
|
+
"analysis_counts": analysis_counts,
|
|
874
969
|
"groups": groups,
|
|
875
970
|
"completeness": completeness,
|
|
971
|
+
"suggested_next_call": suggested_next_call,
|
|
876
972
|
},
|
|
877
973
|
}
|
|
878
974
|
if output_profile == "verbose":
|
|
@@ -1430,6 +1526,7 @@ class RecordTools(ToolBase):
|
|
|
1430
1526
|
page_size: int,
|
|
1431
1527
|
requested_pages: int,
|
|
1432
1528
|
scan_max_pages: int,
|
|
1529
|
+
auto_expand_pages: bool,
|
|
1433
1530
|
query_key: str | None,
|
|
1434
1531
|
filters: list[JSONObject],
|
|
1435
1532
|
sorts: list[JSONObject],
|
|
@@ -1476,6 +1573,14 @@ class RecordTools(ToolBase):
|
|
|
1476
1573
|
total_amount = 0.0
|
|
1477
1574
|
missing_count = 0
|
|
1478
1575
|
by_day: dict[str, JSONObject] = {}
|
|
1576
|
+
scan_control: JSONObject = {
|
|
1577
|
+
"requested_pages": max(requested_pages, 1),
|
|
1578
|
+
"scan_max_pages": max(scan_max_pages, 1),
|
|
1579
|
+
"auto_expand_pages": auto_expand_pages,
|
|
1580
|
+
"auto_expand_applied": False,
|
|
1581
|
+
"auto_expand_target_pages": min(max(requested_pages, 1), max(scan_max_pages, 1)),
|
|
1582
|
+
"auto_expand_page_cap": DEFAULT_ANALYSIS_AUTO_EXPAND_PAGE_CAP,
|
|
1583
|
+
}
|
|
1479
1584
|
while scanned_pages < scan_limit:
|
|
1480
1585
|
page = self._search_page(
|
|
1481
1586
|
context,
|
|
@@ -1494,6 +1599,13 @@ class RecordTools(ToolBase):
|
|
|
1494
1599
|
items = page_rows if isinstance(page_rows, list) else []
|
|
1495
1600
|
if result_amount is None:
|
|
1496
1601
|
result_amount = _effective_total(page, page_size)
|
|
1602
|
+
scan_limit, scan_control = _compute_scan_limit(
|
|
1603
|
+
requested_pages=requested_pages,
|
|
1604
|
+
scan_max_pages=scan_max_pages,
|
|
1605
|
+
auto_expand_pages=auto_expand_pages,
|
|
1606
|
+
page=page,
|
|
1607
|
+
page_size=page_size,
|
|
1608
|
+
)
|
|
1497
1609
|
has_more = _page_has_more(page, current_page, page_size, len(items))
|
|
1498
1610
|
for item in items:
|
|
1499
1611
|
if not isinstance(item, dict):
|
|
@@ -1531,6 +1643,12 @@ class RecordTools(ToolBase):
|
|
|
1531
1643
|
current_page += 1
|
|
1532
1644
|
raw_scan_complete = not has_more
|
|
1533
1645
|
effective_result_amount = scanned_records if view_selection is not None else max(result_amount or 0, scanned_records)
|
|
1646
|
+
analysis_counts = _build_analysis_counts(
|
|
1647
|
+
backend_total_count=result_amount,
|
|
1648
|
+
scanned_count=scanned_records,
|
|
1649
|
+
grouped_count=len(preview_rows),
|
|
1650
|
+
local_filtering=view_selection is not None and bool(view_selection.conditions),
|
|
1651
|
+
)
|
|
1534
1652
|
completeness = _build_completeness(
|
|
1535
1653
|
result_amount=effective_result_amount,
|
|
1536
1654
|
returned_items=len(preview_rows),
|
|
@@ -1549,6 +1667,7 @@ class RecordTools(ToolBase):
|
|
|
1549
1667
|
"raw_next_page_token": None,
|
|
1550
1668
|
"output_next_page_token": None,
|
|
1551
1669
|
"stop_reason": "source_exhausted" if raw_scan_complete else "scan_limit",
|
|
1670
|
+
**scan_control,
|
|
1552
1671
|
},
|
|
1553
1672
|
)
|
|
1554
1673
|
evidence = {
|
|
@@ -1562,24 +1681,73 @@ class RecordTools(ToolBase):
|
|
|
1562
1681
|
}
|
|
1563
1682
|
if strict_full and not raw_scan_complete:
|
|
1564
1683
|
self._raise_need_more_data(completeness, evidence, "Summary is incomplete; increase requested_pages or scan_max_pages.")
|
|
1684
|
+
analysis_status, raw_scan_complete = _analysis_status_from_completeness(completeness)
|
|
1685
|
+
suggested_next_call = None
|
|
1686
|
+
if analysis_status != "success":
|
|
1687
|
+
suggested_next_call = {
|
|
1688
|
+
"tool_name": "record_query",
|
|
1689
|
+
"arguments": {
|
|
1690
|
+
"query_mode": "summary",
|
|
1691
|
+
"app_key": app_key,
|
|
1692
|
+
"page_num": max(page_num, 1),
|
|
1693
|
+
"page_size": page_size,
|
|
1694
|
+
"requested_pages": max(requested_pages, 1),
|
|
1695
|
+
"scan_max_pages": max(scan_max_pages, 1),
|
|
1696
|
+
"auto_expand_pages": auto_expand_pages,
|
|
1697
|
+
"query_key": query_key,
|
|
1698
|
+
"filters": filters,
|
|
1699
|
+
"sorts": sorts,
|
|
1700
|
+
"max_rows": max_rows,
|
|
1701
|
+
"max_columns": max_columns,
|
|
1702
|
+
"select_columns": [field.que_title for field in preview_fields],
|
|
1703
|
+
"amount_column": amount_field.que_title if amount_field is not None else amount_column,
|
|
1704
|
+
"time_range": time_range or {},
|
|
1705
|
+
"stat_policy": stat_policy or {},
|
|
1706
|
+
"strict_full": strict_full,
|
|
1707
|
+
"output_profile": output_profile,
|
|
1708
|
+
"list_type": list_type,
|
|
1709
|
+
"view_key": view_selection.view_key if view_selection is not None else view_key,
|
|
1710
|
+
"view_name": view_selection.view_name if view_selection is not None else view_name,
|
|
1711
|
+
**_callable_analysis_arguments(
|
|
1712
|
+
_recommended_analysis_arguments(
|
|
1713
|
+
tool="record_query",
|
|
1714
|
+
arguments={
|
|
1715
|
+
"page_size": page_size,
|
|
1716
|
+
"requested_pages": requested_pages,
|
|
1717
|
+
"scan_max_pages": scan_max_pages,
|
|
1718
|
+
},
|
|
1719
|
+
probe={"backend_total_count": result_amount},
|
|
1720
|
+
routed_mode="summary",
|
|
1721
|
+
)
|
|
1722
|
+
),
|
|
1723
|
+
},
|
|
1724
|
+
}
|
|
1565
1725
|
response: JSONObject = {
|
|
1566
1726
|
"profile": profile,
|
|
1567
1727
|
"ws_id": session_profile.selected_ws_id,
|
|
1568
1728
|
"ok": True,
|
|
1729
|
+
"status": analysis_status,
|
|
1569
1730
|
"request_route": self._request_route_payload(context),
|
|
1570
1731
|
"data": {
|
|
1571
1732
|
"mode": "summary",
|
|
1733
|
+
"analysis_status": analysis_status,
|
|
1734
|
+
"safe_for_final_conclusion": raw_scan_complete and bool(strict_full),
|
|
1572
1735
|
"source_tool": "record_search",
|
|
1573
1736
|
"view": _view_selection_payload(view_selection),
|
|
1574
1737
|
"summary": {
|
|
1575
1738
|
"summary": {
|
|
1576
|
-
"total_count":
|
|
1739
|
+
"total_count": effective_result_amount,
|
|
1740
|
+
"backend_total_count": result_amount,
|
|
1741
|
+
"scanned_count": scanned_records,
|
|
1742
|
+
"preview_row_count": len(preview_rows),
|
|
1577
1743
|
"total_amount": total_amount if amount_field is not None else None,
|
|
1578
1744
|
"by_day": sorted(by_day.values(), key=lambda item: str(item.get("day"))),
|
|
1579
1745
|
"missing_count": missing_count,
|
|
1580
1746
|
},
|
|
1747
|
+
"analysis_counts": analysis_counts,
|
|
1581
1748
|
"rows": preview_rows,
|
|
1582
1749
|
"completeness": completeness,
|
|
1750
|
+
"suggested_next_call": suggested_next_call,
|
|
1583
1751
|
"applied_limits": {
|
|
1584
1752
|
"row_cap": max_rows,
|
|
1585
1753
|
"column_cap": resolved_column_cap,
|
|
@@ -1721,6 +1889,55 @@ class RecordTools(ToolBase):
|
|
|
1721
1889
|
return True
|
|
1722
1890
|
return False
|
|
1723
1891
|
|
|
1892
|
+
def _build_analysis_probe(
|
|
1893
|
+
self,
|
|
1894
|
+
profile: str,
|
|
1895
|
+
context, # type: ignore[no-untyped-def]
|
|
1896
|
+
*,
|
|
1897
|
+
app_key: str,
|
|
1898
|
+
arguments: JSONObject,
|
|
1899
|
+
view_selection: ViewSelection | None,
|
|
1900
|
+
) -> JSONObject:
|
|
1901
|
+
page_size = max(_coerce_count(arguments.get("page_size")) or DEFAULT_QUERY_PAGE_SIZE, 1)
|
|
1902
|
+
query_key = _normalize_optional_text(arguments.get("query_key"))
|
|
1903
|
+
filters = _as_object_list(arguments.get("filters"))
|
|
1904
|
+
sorts = _as_object_list(arguments.get("sorts"))
|
|
1905
|
+
app_form = self._get_field_index(profile, context, app_key, force_refresh=False)
|
|
1906
|
+
time_range = cast(JSONObject, arguments.get("time_range") if isinstance(arguments.get("time_range"), dict) else {})
|
|
1907
|
+
match_rules = self._resolve_match_rules(context, filters, app_form)
|
|
1908
|
+
sort_rules = self._resolve_sorts(sorts, app_form)
|
|
1909
|
+
time_field = self._resolve_time_range_column(time_range, app_form)
|
|
1910
|
+
match_rules = self._append_time_range_filter(match_rules, time_range, time_field)
|
|
1911
|
+
probe_page = self._search_page(
|
|
1912
|
+
context,
|
|
1913
|
+
app_key=app_key,
|
|
1914
|
+
page_num=max(_coerce_count(arguments.get("page_num")) or 1, 1),
|
|
1915
|
+
page_size=page_size,
|
|
1916
|
+
query_key=query_key,
|
|
1917
|
+
match_rules=match_rules,
|
|
1918
|
+
sorts=sort_rules,
|
|
1919
|
+
search_que_ids=None,
|
|
1920
|
+
list_type=_coerce_count(arguments.get("list_type")) or DEFAULT_RECORD_LIST_TYPE,
|
|
1921
|
+
)
|
|
1922
|
+
backend_total_count = _effective_total(probe_page, page_size)
|
|
1923
|
+
page_amount = _coerce_count(probe_page.get("pageAmount"))
|
|
1924
|
+
estimated_full_scan_pages = page_amount
|
|
1925
|
+
if estimated_full_scan_pages is None and backend_total_count > 0:
|
|
1926
|
+
estimated_full_scan_pages = (backend_total_count + page_size - 1) // page_size
|
|
1927
|
+
current_budget = min(
|
|
1928
|
+
max(_coerce_count(arguments.get("requested_pages")) or 1, 1),
|
|
1929
|
+
max(_coerce_count(arguments.get("scan_max_pages")) or 1, 1),
|
|
1930
|
+
)
|
|
1931
|
+
return {
|
|
1932
|
+
"page_size": page_size,
|
|
1933
|
+
"backend_total_count": backend_total_count,
|
|
1934
|
+
"backend_page_amount": page_amount,
|
|
1935
|
+
"estimated_full_scan_pages": estimated_full_scan_pages,
|
|
1936
|
+
"current_scan_budget": current_budget,
|
|
1937
|
+
"would_exceed_current_budget": bool(estimated_full_scan_pages is not None and estimated_full_scan_pages > current_budget),
|
|
1938
|
+
"local_filtering": bool(view_selection.conditions) if view_selection is not None else False,
|
|
1939
|
+
}
|
|
1940
|
+
|
|
1724
1941
|
def _search_page(
|
|
1725
1942
|
self,
|
|
1726
1943
|
context, # type: ignore[no-untyped-def]
|
|
@@ -2920,6 +3137,7 @@ def _normalize_plan_arguments(tool: str, arguments: JSONObject) -> JSONObject:
|
|
|
2920
3137
|
"pageSize": "page_size",
|
|
2921
3138
|
"requestedPages": "requested_pages",
|
|
2922
3139
|
"scanMaxPages": "scan_max_pages",
|
|
3140
|
+
"autoExpandPages": "auto_expand_pages",
|
|
2923
3141
|
"queryKey": "query_key",
|
|
2924
3142
|
"maxRows": "max_rows",
|
|
2925
3143
|
"maxColumns": "max_columns",
|
|
@@ -2976,7 +3194,7 @@ def _collect_plan_field_candidates(tool: str, arguments: JSONObject) -> list[JSO
|
|
|
2976
3194
|
return candidates
|
|
2977
3195
|
|
|
2978
3196
|
|
|
2979
|
-
def _build_plan_estimate(tool: str, arguments: JSONObject) -> JSONObject:
|
|
3197
|
+
def _build_plan_estimate(tool: str, arguments: JSONObject, *, probe: JSONObject | None = None) -> JSONObject:
|
|
2980
3198
|
page_size = _coerce_count(arguments.get("page_size")) or DEFAULT_QUERY_PAGE_SIZE
|
|
2981
3199
|
requested_pages = _coerce_count(arguments.get("requested_pages")) or 1
|
|
2982
3200
|
scan_max_pages = _coerce_count(arguments.get("scan_max_pages")) or requested_pages
|
|
@@ -2995,6 +3213,17 @@ def _build_plan_estimate(tool: str, arguments: JSONObject) -> JSONObject:
|
|
|
2995
3213
|
)
|
|
2996
3214
|
if routed_mode == "list":
|
|
2997
3215
|
reasons.append("list mode is not a safe final-analysis endpoint")
|
|
3216
|
+
recommended_arguments = _recommended_analysis_arguments(
|
|
3217
|
+
tool=tool,
|
|
3218
|
+
arguments=arguments,
|
|
3219
|
+
probe=probe,
|
|
3220
|
+
routed_mode=routed_mode if tool == "record_query" else "summary",
|
|
3221
|
+
)
|
|
3222
|
+
if probe and probe.get("backend_total_count") is not None and probe.get("estimated_full_scan_pages") is not None:
|
|
3223
|
+
estimated_scan_pages = max(estimated_scan_pages, min(int(probe["estimated_full_scan_pages"]), _coerce_count(recommended_arguments.get("scan_max_pages")) or estimated_scan_pages))
|
|
3224
|
+
if bool(probe.get("would_exceed_current_budget")):
|
|
3225
|
+
may_hit_limits = True
|
|
3226
|
+
reasons.append("matching rows appear to exceed the current scan budget")
|
|
2998
3227
|
return {
|
|
2999
3228
|
"page_size": page_size,
|
|
3000
3229
|
"requested_pages": requested_pages,
|
|
@@ -3003,10 +3232,51 @@ def _build_plan_estimate(tool: str, arguments: JSONObject) -> JSONObject:
|
|
|
3003
3232
|
"estimated_items_upper_bound": page_size * estimated_scan_pages,
|
|
3004
3233
|
"may_hit_limits": may_hit_limits,
|
|
3005
3234
|
"reasons": reasons,
|
|
3006
|
-
"probe":
|
|
3235
|
+
"probe": probe,
|
|
3236
|
+
"recommended_arguments": recommended_arguments,
|
|
3007
3237
|
}
|
|
3008
3238
|
|
|
3009
3239
|
|
|
3240
|
+
def _recommended_analysis_arguments(
|
|
3241
|
+
*,
|
|
3242
|
+
tool: str,
|
|
3243
|
+
arguments: JSONObject,
|
|
3244
|
+
probe: JSONObject | None,
|
|
3245
|
+
routed_mode: str,
|
|
3246
|
+
) -> JSONObject:
|
|
3247
|
+
base_page_size = _coerce_count(arguments.get("page_size")) or DEFAULT_QUERY_PAGE_SIZE
|
|
3248
|
+
base_requested_pages = _coerce_count(arguments.get("requested_pages")) or 1
|
|
3249
|
+
base_scan_max_pages = _coerce_count(arguments.get("scan_max_pages")) or base_requested_pages
|
|
3250
|
+
if tool == "record_aggregate" or routed_mode == "summary":
|
|
3251
|
+
recommended_page_size = max(base_page_size, DEFAULT_ANALYSIS_PAGE_SIZE)
|
|
3252
|
+
recommended_scan_max_pages = max(base_scan_max_pages, DEFAULT_ANALYSIS_SCAN_MAX_PAGES)
|
|
3253
|
+
else:
|
|
3254
|
+
recommended_page_size = base_page_size
|
|
3255
|
+
recommended_scan_max_pages = base_scan_max_pages
|
|
3256
|
+
backend_total_count = _coerce_count(probe.get("backend_total_count") if isinstance(probe, dict) else None)
|
|
3257
|
+
if backend_total_count is not None and backend_total_count > 0:
|
|
3258
|
+
estimated_full_scan_pages = (backend_total_count + recommended_page_size - 1) // recommended_page_size
|
|
3259
|
+
recommended_requested_pages = max(base_requested_pages, estimated_full_scan_pages)
|
|
3260
|
+
recommended_scan_max_pages = max(recommended_scan_max_pages, estimated_full_scan_pages)
|
|
3261
|
+
else:
|
|
3262
|
+
recommended_requested_pages = base_requested_pages
|
|
3263
|
+
estimated_full_scan_pages = None
|
|
3264
|
+
return {
|
|
3265
|
+
"page_size": recommended_page_size,
|
|
3266
|
+
"requested_pages": recommended_requested_pages,
|
|
3267
|
+
"scan_max_pages": recommended_scan_max_pages,
|
|
3268
|
+
"strict_full": True,
|
|
3269
|
+
"auto_expand_pages": True,
|
|
3270
|
+
"estimated_full_scan_pages": estimated_full_scan_pages,
|
|
3271
|
+
}
|
|
3272
|
+
|
|
3273
|
+
|
|
3274
|
+
def _callable_analysis_arguments(arguments: JSONObject) -> JSONObject:
|
|
3275
|
+
if not isinstance(arguments, dict):
|
|
3276
|
+
return {}
|
|
3277
|
+
return {key: value for key, value in arguments.items() if key not in {"estimated_full_scan_pages"}}
|
|
3278
|
+
|
|
3279
|
+
|
|
3010
3280
|
def _assess_plan_readiness(
|
|
3011
3281
|
tool: str,
|
|
3012
3282
|
arguments: JSONObject,
|
|
@@ -3039,6 +3309,10 @@ def _assess_plan_readiness(
|
|
|
3039
3309
|
if tool in {"record_query", "record_aggregate"} and not bool(arguments.get("strict_full")):
|
|
3040
3310
|
blockers.append("strict_full should be true for final conclusions")
|
|
3041
3311
|
actions.append("Set strict_full=true so incomplete scans block final conclusions.")
|
|
3312
|
+
probe = estimate.get("probe") if isinstance(estimate.get("probe"), dict) else None
|
|
3313
|
+
if isinstance(probe, dict) and bool(probe.get("would_exceed_current_budget")):
|
|
3314
|
+
blockers.append("current scan budget is smaller than the estimated matching dataset")
|
|
3315
|
+
actions.append("Use estimate.recommended_arguments or enable auto_expand_pages before trusting aggregate or summary results.")
|
|
3042
3316
|
actions.append("After execution, verify completeness before using the result as a final conclusion.")
|
|
3043
3317
|
return {
|
|
3044
3318
|
"ready_for_final_conclusion": not blockers,
|
|
@@ -3154,6 +3428,69 @@ def _view_selection_payload(view_selection: ViewSelection | None) -> JSONObject
|
|
|
3154
3428
|
}
|
|
3155
3429
|
|
|
3156
3430
|
|
|
3431
|
+
def _compute_scan_limit(
|
|
3432
|
+
*,
|
|
3433
|
+
requested_pages: int,
|
|
3434
|
+
scan_max_pages: int,
|
|
3435
|
+
auto_expand_pages: bool,
|
|
3436
|
+
page: JSONObject | None = None,
|
|
3437
|
+
page_size: int | None = None,
|
|
3438
|
+
) -> tuple[int, JSONObject]:
|
|
3439
|
+
base_requested = max(requested_pages, 1)
|
|
3440
|
+
base_scan_max = max(scan_max_pages, 1)
|
|
3441
|
+
initial_limit = min(base_requested, base_scan_max)
|
|
3442
|
+
meta: JSONObject = {
|
|
3443
|
+
"requested_pages": base_requested,
|
|
3444
|
+
"scan_max_pages": base_scan_max,
|
|
3445
|
+
"auto_expand_pages": auto_expand_pages,
|
|
3446
|
+
"auto_expand_applied": False,
|
|
3447
|
+
"auto_expand_target_pages": initial_limit,
|
|
3448
|
+
"auto_expand_page_cap": DEFAULT_ANALYSIS_AUTO_EXPAND_PAGE_CAP,
|
|
3449
|
+
}
|
|
3450
|
+
if not auto_expand_pages or not isinstance(page, dict):
|
|
3451
|
+
return initial_limit, meta
|
|
3452
|
+
effective_page_size = max(page_size or 0, 1)
|
|
3453
|
+
backend_total = _effective_total(page, effective_page_size)
|
|
3454
|
+
page_amount = _coerce_count(page.get("pageAmount"))
|
|
3455
|
+
target_pages = page_amount
|
|
3456
|
+
if target_pages is None and backend_total > 0:
|
|
3457
|
+
target_pages = (backend_total + effective_page_size - 1) // effective_page_size
|
|
3458
|
+
if target_pages is None:
|
|
3459
|
+
return initial_limit, meta
|
|
3460
|
+
expanded_limit = min(max(initial_limit, target_pages), DEFAULT_ANALYSIS_AUTO_EXPAND_PAGE_CAP)
|
|
3461
|
+
if expanded_limit > initial_limit:
|
|
3462
|
+
meta["auto_expand_applied"] = True
|
|
3463
|
+
meta["auto_expand_target_pages"] = target_pages
|
|
3464
|
+
meta["backend_total_count"] = backend_total
|
|
3465
|
+
meta["backend_page_amount"] = page_amount
|
|
3466
|
+
return expanded_limit, meta
|
|
3467
|
+
|
|
3468
|
+
|
|
3469
|
+
def _build_analysis_counts(
|
|
3470
|
+
*,
|
|
3471
|
+
backend_total_count: int | None,
|
|
3472
|
+
scanned_count: int,
|
|
3473
|
+
grouped_count: int,
|
|
3474
|
+
local_filtering: bool,
|
|
3475
|
+
) -> JSONObject:
|
|
3476
|
+
unscanned_count: int | None = None
|
|
3477
|
+
if backend_total_count is not None and not local_filtering:
|
|
3478
|
+
unscanned_count = max(backend_total_count - scanned_count, 0)
|
|
3479
|
+
return {
|
|
3480
|
+
"backend_total_count": backend_total_count,
|
|
3481
|
+
"scanned_count": scanned_count,
|
|
3482
|
+
"grouped_count": grouped_count,
|
|
3483
|
+
"unscanned_count": unscanned_count,
|
|
3484
|
+
"local_filtering": local_filtering,
|
|
3485
|
+
}
|
|
3486
|
+
|
|
3487
|
+
|
|
3488
|
+
def _analysis_status_from_completeness(completeness: JSONObject) -> tuple[str, bool]:
|
|
3489
|
+
raw_scan_complete = bool(completeness.get("raw_scan_complete"))
|
|
3490
|
+
status = "success" if raw_scan_complete else "partial_success"
|
|
3491
|
+
return status, raw_scan_complete
|
|
3492
|
+
|
|
3493
|
+
|
|
3157
3494
|
def _build_completeness(
|
|
3158
3495
|
*,
|
|
3159
3496
|
result_amount: int,
|