@coloopx/colocad 0.1.0-alpha.7 → 0.1.0-alpha.8
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +20 -0
- package/README.md +7 -2
- package/README.zh-CN.md +6 -1
- package/ROADMAP.md +125 -0
- package/ROADMAP.zh-CN.md +91 -0
- package/docs/README.md +7 -1
- package/docs/README.zh-CN.md +5 -3
- package/docs/agent-modeling-handbook.md +11 -3
- package/docs/agent-modeling-handbook.zh-CN.md +19 -2
- package/docs/best-practices.md +80 -3
- package/docs/best-practices.zh-CN.md +63 -0
- package/docs/cadquery-comparison.md +105 -0
- package/docs/cadquery-comparison.zh-CN.md +53 -0
- package/docs/cadquery-coverage.json +762 -0
- package/docs/engineering-queries.md +2 -0
- package/docs/engineering-queries.zh-CN.md +3 -1
- package/docs/integrations/overview.md +3 -1
- package/docs/troubleshooting.md +13 -1
- package/kernel/native/darwin-arm64/agent-cad +2 -2
- package/kernel/native/darwin-arm64/manifest.json +2 -2
- package/kernel/native/linux-arm64/agent-cad +2 -2
- package/kernel/native/linux-arm64/manifest.json +2 -2
- package/kernel/native/linux-x64/agent-cad +2 -2
- package/kernel/native/linux-x64/manifest.json +2 -2
- package/kernel/release.json +4 -4
- package/package.json +5 -2
- package/skills/colocad/SKILL.md +1 -0
- package/skills/colocad/references/modeling-loop.md +15 -1
- package/skills/colocad/references/prompt-patterns.md +27 -1
- package/skills/colocad/references/quick-build.md +2 -0
- package/skills/colocad/references/tuning.md +5 -1
|
@@ -10,6 +10,8 @@ This layer is ready for the declared mechanical assembly workflows: exact topolo
|
|
|
10
10
|
|
|
11
11
|
G1–G3 acceptance covers the frozen 24-case corpus, lifecycle tests and two live agent tasks. [Release archive checks](evidence/release-alpha.7.json) verify the installed SDK/CLI/native together on macOS arm64 and Linux arm64/x64; x64 is emulated on an arm64 host. Unsupported recognition/contact domains are listed below and remain explicit partial/error results.
|
|
12
12
|
|
|
13
|
+
Alpha.8 fixes the alpha.7 grid snapshot defect: nested lazy UV lines on cylinder seams are materialized exactly on the serialization copy. No tessellation, STEP reimport or geometry healing is substituted. See [repair and installed-package evidence](evidence/release-alpha.8.json); [the original failure](evidence/alpha.7-query-guidance.json) remains preserved.
|
|
14
|
+
|
|
13
15
|
## Build once, query the accepted revision
|
|
14
16
|
|
|
15
17
|
```js
|
|
@@ -4,11 +4,13 @@
|
|
|
4
4
|
|
|
5
5
|
这层能力用于在装配前查询零件的安装几何。几何解释在本地 Rust 内核中完成,SDK、CLI 与 agent 工具共用实现;agent、harness 和可执行文件仍位于同一沙箱中。
|
|
6
6
|
|
|
7
|
+
alpha.8 修复 alpha.7 的孔阵列快照问题:在序列化副本上精确补齐圆柱接缝内部 UV 直线数据,不使用三角化、STEP 重导入或几何修补代替快照。见[修复与安装包证据](evidence/release-alpha.8.json);[原始失败](evidence/alpha.7-query-guidance.json)保持留档。
|
|
8
|
+
|
|
7
9
|
## 构建一次,查询已接受版本
|
|
8
10
|
|
|
9
11
|
`cad.build(model, {queryable:true})` 返回 `artifact_directory` 和 `query_revision_id`,并保存 `query-manifest.json` 与精确 `query-definition-*.smb`。将带预期版本号、目标、坐标系、容差和查询列表的请求交给 `cad.query(directory, request)`。完整 SDK 代码见英文页。
|
|
10
12
|
|
|
11
|
-
查询不重放建模特征、不读 STEP、不导出 STEP、不修补几何、不生成网格。整个工件目录迁移或换进程重开后,快照引用保持一致。普通模型按需启用;声明 `Model.datums` 的模型自动启用,显式关闭会报错。SDK 的 `queryRoots` 由宿主配置,默认只允许 `outputRoot` 下的工件。
|
|
13
|
+
查询不重放建模特征、不读 STEP、不导出 STEP、不修补几何、不生成网格。整个工件目录迁移或换进程重开后,快照引用保持一致。普通模型按需启用;声明 `Model.datums` 或 `Model.interfaces` 的模型自动启用,显式关闭会报错。SDK 的 `queryRoots` 由宿主配置,默认只允许 `outputRoot` 下的工件。
|
|
12
14
|
|
|
13
15
|
```sh
|
|
14
16
|
colocad record model.json session --queryable --label '可查询零件'
|
|
@@ -36,9 +36,11 @@ With typed tools, allocate one `CadClient` for each sequential modeling session.
|
|
|
36
36
|
|
|
37
37
|
With shell tools, use argument arrays in a process API rather than interpolating model text into shell commands. Store JSON in a file. Agent-controlled dimensions belong in model input; executable paths, output roots and credentials belong to harness configuration. Keep normal harness permissions in force.
|
|
38
38
|
|
|
39
|
+
For exact queries, enable `queryable` when needed (datum/interface declarations enable it automatically). Register `cad_query` only when native capabilities advertise it. Raw `createCadTools` takes `{artifact_directory, request}`; session-bound tools take `{request}` and bind the accepted artifact. The host owns `queryRoots`; the agent chooses a target/space/tolerance within that boundary. Preserve `query_revision_id` in compact build/edit feedback and re-query after edits. Stale refs, partial coverage and ambiguity remain visible. [Query/edit walkthrough](../best-practices.md#query-declare-intent-assemble-then-challenge-the-edit).
|
|
40
|
+
|
|
39
41
|
## Results and failures
|
|
40
42
|
|
|
41
|
-
A useful tool result includes the artifact directory, counts/bounds/volume, validation
|
|
43
|
+
A useful tool result includes the artifact directory, revision, relevant measurements, coverage/ambiguity, counts/bounds/volume, validation and timings. Keep detailed feature/query diagnostics in the full receipt and attach a PNG for visual inspection. A path in text is not itself an image input to a remote vision model. The [Pi adapter](../../examples/integrations/pi-tools.mjs) shows text/image content and full `details` together.
|
|
42
44
|
|
|
43
45
|
`cad_validate` success is only preflight. A nonzero child exit, a failed receipt or an explicit degraded feature must remain visible. Do not turn a failed build into a success containing an empty output path. Do not retry the same invalid geometry indefinitely; change the failing feature or report the unresolved constraint.
|
|
44
46
|
|
package/docs/troubleshooting.md
CHANGED
|
@@ -6,12 +6,24 @@
|
|
|
6
6
|
| Skill helper cannot import colocad | Install the pinned package in the current project and run the helper from there. Copy the complete skill folder. |
|
|
7
7
|
| `colocad build` rejects `version: 1` | That CLI expects `format: "colocad/1"`. Use `CadClient` or the skill helper for a kernel model. |
|
|
8
8
|
| Output directory already exists | Pick a new revision destination for the CLI/helper. Keep earlier accepted output. `CadClient.outputRoot` can contain multiple revisions. |
|
|
9
|
-
| Unsupported platform / missing executable | Alpha.
|
|
9
|
+
| Unsupported platform / missing executable | Alpha.8 bundles Linux x64/arm64 and macOS arm64. Check `process.platform`, `process.arch` and the installed `kernel/native` directory. Source builds require a Rust toolchain and the pinned dependency. |
|
|
10
10
|
| Native launch fails | Check `LOCAL_EXECUTION_FAILED`, executable permissions, mount policy and `binary`/`AGENT_CAD_BINARY` overrides. Run the local CLI to isolate harness wiring. |
|
|
11
11
|
| `TIMEOUT` | Inspect the failing feature and workload. Simplify or split unnecessary work; adjust `wallMs` within 1–120,000 only when appropriate. Sandbox memory failures need sandbox diagnostics too. |
|
|
12
12
|
| `RESOURCE_LIMIT` / `OUTPUT_LIMIT` | Inspect capabilities and reduce model/mesh payload. SDK stdin is limited to 32 MiB; the feature-tree CLI accepts 2 MiB. |
|
|
13
13
|
| Unknown/missing parameter | Use an existing finite numeric parameter. `edit()` does not create parameters; update the owned source to add one. |
|
|
14
14
|
| An edit undoes a prior change | You reused stale source. Load the last accepted `model.json` or persist the merged parameters after each successful edit. |
|
|
15
|
+
| `cad_query` is absent | Inspect installed native capabilities. Alpha.7 includes exact queries; an older package or `AGENT_CAD_BINARY` override can expose older capabilities. Keep SDK and native from the same release/build. |
|
|
16
|
+
| No exact query snapshot | Rebuild the accepted source with `{queryable:true}` or CLI `--queryable`. A preview mesh or STEP file alone is not the native snapshot. Datums/interfaces require snapshots and reject explicit disabling with `QUERY_SNAPSHOT_REQUIRED`. |
|
|
17
|
+
| Alpha.7 `face_hole_grid` + `queryable` reports `BSplineCurve::make_nurb` status 1001 | Upgrade to alpha.8 with its matching native. The nested seam UV-line snapshot defect is fixed; preview precision was not the cause. Check `AGENT_CAD_BINARY` if an old native is still used. [Repair evidence](evidence/release-alpha.8.json). |
|
|
18
|
+
| `QUERY_REVISION_MISMATCH` | Obtain the latest accepted `query_revision_id` and rerun the semantic selection. Do not rewrite an old topology ref's revision field to bypass the check. |
|
|
19
|
+
| `QUERY_ARTIFACT_OUTSIDE_ROOT` | The harness must configure the appropriate trusted `queryRoots` or relocate the complete artifact directory under its allowed output root. The agent's query JSON cannot grant itself new roots. |
|
|
20
|
+
| `SELECTION_CARDINALITY`, missing datum/interface or ambiguous candidates | Inspect actual candidates, support plane, coordinate space and orientation. Refine a geometric rule; do not choose the first face or suppress ambiguity. |
|
|
21
|
+
| `INTERFACE_INCOMPATIBLE` / `INTERFACE_NOT_FOUND` during replacement | Compare required pitch, bore size, count, plane relation and register geometry. Preserve accepted source; retain a genuinely incompatible replacement as a failed attempt. |
|
|
22
|
+
| Query succeeds with `partial` coverage | Read diagnostics and source refs. Unsupported or unobserved geometry is not evidence of absence; required fit/clearance claims need complete supported evidence. |
|
|
23
|
+
| `QUERY_LIMIT` | Reduce target scope, nested selection/search size or page size. Check advertised budgets. Do not truncate candidates and still claim unique/complete recognition. |
|
|
24
|
+
| Positive distance but clearance fails | Inspect `relationship` and `overlap_mm3`. Boundary distance is unsigned; containment is not separation. Curved/region-specific contact certification also has explicit support limits. |
|
|
25
|
+
| `NO_MATERIAL_REMOVED` | The required cut was disjoint or tangent. Check tool placement, axis, depth and intended stock; do not treat the unchanged body as a machined result. |
|
|
26
|
+
| Old checkpoint is not reused after an upgrade | A changed/missing `execution_policy` intentionally forces replay. Keep accepted source and let the new executor rebuild; do not forge cache metadata. |
|
|
15
27
|
| Preview succeeds but no STEP in its directory | A cached observation points to `source_artifact_directory`. Retrieve STEP from the source geometry revision. |
|
|
16
28
|
| Cached preview cannot be read | Retain its source directory and files; do not modify cache bytes. Rebuild from the saved model if the cache was pruned or is invalid. |
|
|
17
29
|
| Agent cannot see the rendered image | Attach PNG as image content or invoke the harness's image-reading tool. Printing the local filename alone is insufficient. |
|
|
@@ -1,4 +1,4 @@
|
|
|
1
1
|
[diffend] Oversized file quarantined before diffing.
|
|
2
2
|
name: package/kernel/native/darwin-arm64/agent-cad
|
|
3
|
-
size:
|
|
4
|
-
sha256:
|
|
3
|
+
size: 37838080 bytes
|
|
4
|
+
sha256: cf358a37f19ed8dcd9b2607584152ad3a627c7e175faa631e2086ed1c5cc255a
|
|
@@ -1,4 +1,4 @@
|
|
|
1
1
|
[diffend] Oversized file quarantined before diffing.
|
|
2
2
|
name: package/kernel/native/linux-arm64/agent-cad
|
|
3
|
-
size:
|
|
4
|
-
sha256:
|
|
3
|
+
size: 37294920 bytes
|
|
4
|
+
sha256: 7250739a28794f67d9a6b004e31f24a2f04fb6c4b39f7c29b84eea14a379a0d3
|
|
@@ -1,4 +1,4 @@
|
|
|
1
1
|
[diffend] Oversized file quarantined before diffing.
|
|
2
2
|
name: package/kernel/native/linux-x64/agent-cad
|
|
3
|
-
size:
|
|
4
|
-
sha256:
|
|
3
|
+
size: 48565856 bytes
|
|
4
|
+
sha256: 5f685e3a268aa43c1a9670902c6fb57eeee5bda56b85620f6af4edf9abe33d01
|
package/kernel/release.json
CHANGED
|
@@ -1,8 +1,8 @@
|
|
|
1
1
|
{
|
|
2
|
-
"version": "0.1.0-alpha.
|
|
2
|
+
"version": "0.1.0-alpha.8",
|
|
3
3
|
"sha256": {
|
|
4
|
-
"darwin-arm64": "
|
|
5
|
-
"linux-arm64": "
|
|
6
|
-
"linux-x64": "
|
|
4
|
+
"darwin-arm64": "cf358a37f19ed8dcd9b2607584152ad3a627c7e175faa631e2086ed1c5cc255a",
|
|
5
|
+
"linux-arm64": "7250739a28794f67d9a6b004e31f24a2f04fb6c4b39f7c29b84eea14a379a0d3",
|
|
6
|
+
"linux-x64": "5f685e3a268aa43c1a9670902c6fb57eeee5bda56b85620f6af4edf9abe33d01"
|
|
7
7
|
}
|
|
8
8
|
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@coloopx/colocad",
|
|
3
|
-
"version": "0.1.0-alpha.
|
|
3
|
+
"version": "0.1.0-alpha.8",
|
|
4
4
|
"description": "Parametric feature instructions, a local Rust B-rep kernel and an experimental SolidWorks author",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"type": "module",
|
|
@@ -67,7 +67,10 @@
|
|
|
67
67
|
"examples/integrations/guided-workflow.mjs",
|
|
68
68
|
"docs/*.md",
|
|
69
69
|
"docs/integrations/*.md",
|
|
70
|
-
"CHANGELOG.md"
|
|
70
|
+
"CHANGELOG.md",
|
|
71
|
+
"ROADMAP.md",
|
|
72
|
+
"ROADMAP.zh-CN.md",
|
|
73
|
+
"docs/cadquery-coverage.json"
|
|
71
74
|
],
|
|
72
75
|
"scripts": {
|
|
73
76
|
"schema": "node scripts/schema.mjs",
|
package/skills/colocad/SKILL.md
CHANGED
|
@@ -9,6 +9,7 @@ Choose the references needed for this task:
|
|
|
9
9
|
|
|
10
10
|
- **A dimensioned part or common structure:** [quick-build](references/quick-build.md) has editable stock, plate, sleeve, angle and housing examples. Start with the controlling section or stock and add only required features.
|
|
11
11
|
- **Drawings or a real product:** [materials](references/materials.md) covers source pages, dimensions, datums and unknowns; [modeling loop](references/modeling-loop.md) covers whole-product interfaces, features, edits and verification.
|
|
12
|
+
- **Measure or replace an assembly part:** [modeling loop](references/modeling-loop.md#geometry-driven-assembly-with-exact-queries) covers query revisions, semantic datums/interfaces, replacement and fit checks. Read the installed query schema; alpha.7 exposes this through `cad_query`.
|
|
12
13
|
- **A harness/tool adapter:** [tool contracts](references/tool-contracts.md) explains actual tool names, state, compact feedback and errors.
|
|
13
14
|
- **Task/prompt guidance:** [prompt patterns](references/prompt-patterns.md) gives short task, interpretation, repair and handoff examples. Adapt them to the user's scope.
|
|
14
15
|
- **Failures, quality or performance tuning:** [tuning](references/tuning.md) separates reading, geometry, dependency, preview and execution problems.
|
|
@@ -13,6 +13,20 @@ Read for assemblies, drawing reconstruction or edits whose dependencies matter.
|
|
|
13
13
|
|
|
14
14
|
This is an adaptive loop, not a mandatory approval gate between every feature. Use fewer milestones for a simple part. For a long product, keep checkpoints that explain both shape construction and assembly.
|
|
15
15
|
|
|
16
|
+
## Geometry-driven assembly with exact queries
|
|
17
|
+
|
|
18
|
+
Use this path when installation geometry must be measured or a part replaced. Check installed capabilities; alpha.7 provides `cad_query`.
|
|
19
|
+
|
|
20
|
+
Exact grid snapshots require alpha.8 or later: alpha.7 had a nested seam UV-line serialization defect. Upgrade the package and matching native if `BSplineCurve::make_nurb` status 1001 appears on this path; changing preview precision does not repair it. Use the normal grid/query workflow after upgrading.
|
|
21
|
+
|
|
22
|
+
- Build exact query artifacts with `queryable`, or declare `Model.datums`/`Model.interfaces`, which enable them. Keep the accepted `query_revision_id`, target definition/occurrence, coordinate space and tolerance with the observation.
|
|
23
|
+
- Narrow the mounting plane before its openings and group. Inspect complete/partial coverage, cardinality and symmetric alternatives. A normal-only selection can include a blind-hole bottom; a position-sensitive recipe can use an earlier named plane with `planar_coplanar_with`.
|
|
24
|
+
- Persist supported semantic recipes in datums/interfaces, not snapshot refs or face indices. A symmetric group needs design intent or a measured orientation datum for an oriented mate. Let named attachments drive the existing mate solver.
|
|
25
|
+
- Require all-hole compatibility and relevant exact pair checks. Boundary distance is unsigned; inspect relationship and overlap before calling it clearance. Unsupported/partial evidence cannot certify fit or absence.
|
|
26
|
+
- After an edit/replacement, re-query the new revision and rerun required checks. Missing, ambiguous or incompatible geometry is a retained failure. Never rewrite an old ref's revision to make it appear current. Resume from accepted source after failure/interruption.
|
|
27
|
+
|
|
28
|
+
For a replacement workflow, challenge the same recipe with compatible and incompatible sources. Do not add a new part template or hardcoded face mapping for each input. Use only the pose/pair checks required by the task; report their discrete scope.
|
|
29
|
+
|
|
16
30
|
## Match evidence to the claim
|
|
17
31
|
|
|
18
32
|
| Claim | A useful check |
|
|
@@ -28,6 +42,6 @@ Negative material checks matter: verify that a supposed opening is empty and adj
|
|
|
28
42
|
|
|
29
43
|
## State that survives a long session
|
|
30
44
|
|
|
31
|
-
Keep source IDs and coordinate conventions, accepted model path/hash, exposed parameters, interface
|
|
45
|
+
Keep source IDs and coordinate conventions, accepted model path/hash, exposed parameters, semantic datum/interface recipes, latest receipt and `query_revision_id`, target/space/tolerance, coverage/ambiguity, passed checks, unresolved assumptions, and the next targeted test. Reload the current accepted source after a failure or context reset. In `ModelingSession`, recorded `cad_edit` takes only `{parameters}` and uses that accepted source.
|
|
32
46
|
|
|
33
47
|
The KK60 example illustrates this process. Enlarging the manufacturer's section corrected the screw axis to 23 mm; the real helical B-rep exposed an internal STEP budget defect; independent material probes and rail/carriage checks established specific geometry claims. Its undisclosed nut recirculation remains unmodeled. Do not reuse its assumed internal dimensions as a universal actuator design.
|
|
@@ -58,12 +58,38 @@ read back STEP when available. Distinguish untested pairs and unknown internals
|
|
|
58
58
|
from passed checks; do not use appearance alone as evidence of fit.
|
|
59
59
|
```
|
|
60
60
|
|
|
61
|
+
## Replace a component using measured installation geometry
|
|
62
|
+
|
|
63
|
+
```text
|
|
64
|
+
Use the installed exact-query API to fit this replacement STEP to the accepted assembly.
|
|
65
|
+
The task supplies required mounting geometry, tolerances and pairs/poses to check.
|
|
66
|
+
Query the accepted revision in the stated coordinate space. Select the support plane
|
|
67
|
+
before its openings/groups; report partial coverage, extra holes and ambiguity.
|
|
68
|
+
Preserve geometric selection recipes as named datums/interfaces, then solve mates
|
|
69
|
+
and check all required hole correspondences plus exact pair relationships/clearance.
|
|
70
|
+
Rebuild after replacement and query the NEW revision. Retain incompatible attempts;
|
|
71
|
+
do not patch face numbers or invent orientation to force success. Return editable
|
|
72
|
+
source, measured deviations, accepted STEP and session, with CLI review artifacts.
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
## Repair a stale or incomplete observation
|
|
76
|
+
|
|
77
|
+
```text
|
|
78
|
+
The supplied receipt identifies a stale revision, ambiguous selection or partial query.
|
|
79
|
+
Reload the latest accepted source/artifact and read its current query_revision_id.
|
|
80
|
+
For stale observations, rerun the semantic query; never relabel an old topology ref.
|
|
81
|
+
For ambiguity, inspect candidates and refine the geometric criterion or orientation
|
|
82
|
+
using available design evidence. For partial coverage, identify unsupported geometry
|
|
83
|
+
and which required claim remains unverified. Keep errors and accepted state intact.
|
|
84
|
+
```
|
|
85
|
+
|
|
61
86
|
## Handoff after a long run
|
|
62
87
|
|
|
63
88
|
```text
|
|
64
89
|
Persist: product/configuration; source identifiers; units and datums; accepted
|
|
65
90
|
model path/hash; session path/latest step; parameters and dependency decisions;
|
|
66
|
-
|
|
91
|
+
semantic datum/interface recipes; current query revision and target/space/tolerance;
|
|
92
|
+
coverage/ambiguity; passed checks with values; failed attempts; unresolved assumptions; next targeted
|
|
67
93
|
check. Keep large meshes and receipts as file references rather than inline text.
|
|
68
94
|
```
|
|
69
95
|
|
|
@@ -10,6 +10,8 @@ node "$skill/scripts/build.mjs" "$skill/assets/plate-grid.json" ./plate-v1 \
|
|
|
10
10
|
|
|
11
11
|
Each invocation takes a new output directory. Edit the saved `model.json` to add features or pass it back to the helper with another `--set` to change an existing dimension. Overrides are saved in the built model. Only request a preview when you need to inspect it.
|
|
12
12
|
|
|
13
|
+
Exact grid snapshots require alpha.8 or later: alpha.7 had a nested seam UV-line serialization defect. Upgrade the package and matching native if `BSplineCurve::make_nurb` status 1001 appears on this path; changing preview precision does not repair it. Use the normal grid/query workflow after upgrading.
|
|
14
|
+
|
|
13
15
|
| Shape or structure | Shortest construction in this skill | Datum and acceptance check |
|
|
14
16
|
|---|---|---|
|
|
15
17
|
| Block, pin, ball or taper | Use one feature from [block](../assets/block.json), [pin](../assets/pin.json), [ball](../assets/ball.json) or [taper](../assets/taper.json). For example, pass `pin.json` with `--set diameter=8 --set length=40` for a Ø8 × 40 mm pin. | Box origin is the lower corner. A cylinder or cone starts at its base center and grows along +Z. Check the outer envelope. |
|
|
@@ -19,7 +19,7 @@ Change one cause at a time and retain the original acceptance target. These are
|
|
|
19
19
|
|
|
20
20
|
Keep source interpretation/inference, CAD startup, geometry, validation, STEP export, tessellation, view rendering and video encoding separate. Time-to-STEP and time-to-preview answer different questions. A cached view cannot be compared with another tool's fresh geometry build as if they were the same task.
|
|
21
21
|
|
|
22
|
-
For a fair build123d comparison, use the same dimensions, features, instances, precision and required artifacts. Verify both geometries independently, pin environments and define cold/warm behavior. Report repeat counts, median/tails, failures and the accounting boundary. For memory, state sampled RSS/process sum/cgroup, sampling interval and whether the plot is instantaneous or the peak-so-far; retaining a peak is a presentation choice, not the original trace.
|
|
22
|
+
For a fair CadQuery or build123d comparison, use the same dimensions, features, instances, precision and required artifacts. Verify both geometries independently, pin environments and define cold/warm behavior. Report repeat counts, median/tails, failures and the accounting boundary. For memory, state sampled RSS/process sum/cgroup, sampling interval and whether the plot is instantaneous or the peak-so-far; retaining a peak is a presentation choice, not the original trace.
|
|
23
23
|
|
|
24
24
|
A 1 vCPU / 512 MiB CAD run establishes only that scoped run. It does not establish the memory budget of the resident agent/harness, an in-process LLM, Chromium or FFmpeg. Do not extrapolate a plate's speedup to a threaded multi-part product.
|
|
25
25
|
|
|
@@ -28,3 +28,7 @@ A 1 vCPU / 512 MiB CAD run establishes only that scoped run. It does not establi
|
|
|
28
28
|
Hold product tasks, source packs, model/provider version, tools and budget fixed. Compare the existing skill with one changed instruction/tool response policy. Include a dimensioned part, a part needing section interpretation, an editable assembly and an ambiguous source case. Keep some tasks unseen during tuning. Measure required dimensions/features, edit preservation, fit checks, unsupported claims, correction count, inference cost and end-to-end duration.
|
|
29
29
|
|
|
30
30
|
Use numerical checks and inspectable artifacts for acceptance. Scripted tool tests establish transport and geometry contracts; they do not establish that a model understood an unseen drawing. Report failed tasks and unknown evidence, and do not describe an unrun prompt ablation as an improvement result.
|
|
31
|
+
|
|
32
|
+
## Preserve an improvement experiment
|
|
33
|
+
|
|
34
|
+
For an explicit optimization task, freeze the baseline, task split, required geometry, tolerances, verifier and resource budget before changing code or guidance. Retain the original failure and a small reproducer. Change one hypothesis, pass affected-family and frozen regressions, then challenge held-out product families. Include failures in the denominator and preserve rejected or inconclusive candidates. Do not change the evaluator/reference to make a candidate pass. A performance gain needs repeat measurements beyond noise; a skill gain needs live-agent evidence, not only scripted tests. Stop at the authorized iteration/cost budget and keep the last accepted state. Publishing is a separate authorized action, not a tool-score side effect.
|