@calcit/procs 0.12.9 → 0.12.10

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Binary file
@@ -0,0 +1,24 @@
1
+ # 2026-03-13 22:16 validate struct type arg arity
2
+
3
+ ## 背景
4
+
5
+ 发现类型注解里可以写出 `:: Pair :number :string` 这样的形式,即使 `Pair` 是非泛型 `defstruct`。旧逻辑会把后续类型参数直接附着到 `Struct(...)` 注解上,但并不会校验 `generics` 个数,最终在值匹配时只看 struct 名字,导致漏检。
6
+
7
+ ## 本次修改
8
+
9
+ - 在 `src/calcit/type_annotation.rs` 为 `CalcitTypeAnnotation` / `CalcitFnTypeAnnotation` 增加 `validate_applied_type_args()`。
10
+ - 对 `Struct` 注解增加规则:
11
+ - 非泛型 struct 不允许携带类型参数;
12
+ - 泛型 struct 必须严格匹配声明的类型参数个数。
13
+ - 对 `Enum` 注解增加保守规则:当前不接受额外类型参数。
14
+ - 在 `src/builtins/records.rs` 中,`defstruct` 字段类型解析后立即做校验。
15
+ - 在 `src/calcit/sum_type.rs` 中,`defenum` payload 类型解析后立即做校验。
16
+ - 在 `src/calcit/sum_type.rs` 与 `src/calcit/type_annotation.rs` 补充回归测试,覆盖:
17
+ - 非泛型 struct 带类型参数时报错;
18
+ - 泛型 struct 类型参数个数不匹配时报错;
19
+ - enum payload 中引用非法 struct 类型注解时报错。
20
+ - 同时修正 `calcit/test-generics.cirru` 中 `Wrapped` 的 payload 类型,把错误的 `:: Pair :number :string` 改为 `Pair`。
21
+
22
+ ## 结论
23
+
24
+ 现在像 `:: Pair :number :string` 这种写法不会再静默通过;定义期就会被校验出来,避免文档和测试继续误导后续修改。
@@ -0,0 +1,13 @@
1
+ # Profiling tools reorganization
2
+
3
+ ## Summary
4
+
5
+ - Moved profiling scripts from `scripts/` into dedicated `profiling/` directory.
6
+ - Added `profiling/README.md` with end-to-end usage for xctrace and samply workflows.
7
+ - Removed obsolete `scripts/profiling.sh` flamegraph helper script.
8
+
9
+ ## Knowledge notes
10
+
11
+ - Keep profiling tooling isolated under `profiling/` to reduce script namespace clutter.
12
+ - `samply` profiling should run against built `target/*/cr` binaries to avoid sampling `rustc` compile threads.
13
+ - Preserve `.tmp-profiles/` as transient output directory for trace artifacts.
@@ -0,0 +1,14 @@
1
+ # Runtime Boundary Cleanup
2
+
3
+ ## Summary
4
+
5
+ - removed the dead `CalcitImport.coord` compatibility path and the related runner parameter flow
6
+ - renamed remaining `evaled`-era runtime lookup terminology to `runtime-ready` / `runtime cache`
7
+ - synced the runtime boundary refactor draft with the current migration state and recent cleanup progress
8
+
9
+ ## Knowledge
10
+
11
+ - `CalcitImport.def_id` is now the stable import-side runtime lookup anchor; the old `coord -> EntryBook` path no longer needs to be preserved in import metadata
12
+ - type-annotation lookup registration should describe runtime-ready value lookup, not the removed evaled-store model
13
+ - incremental reload messaging should talk about clearing runtime caches rather than evaled states to match the current architecture
14
+ - this migration tail is now mostly about narrowing fallback bridges and improving reload coverage, not about preserving old runtime naming or lookup compatibility
@@ -0,0 +1,28 @@
1
+ # Runtime boundary lookup cleanup
2
+
3
+ ## Summary
4
+
5
+ - Consolidated runtime-vs-compiled lookup helpers inside `program` and removed extra `runner` bridging paths.
6
+ - Tightened snapshot fallback and codegen metadata lookup so source-backed defs no longer silently rely on runtime-derived compiled entries.
7
+ - Skipped runtime-only placeholder defs in JS/IR codegen and added program-level regression tests for reload invalidation and snapshot behavior.
8
+ - Reframed the runtime-boundary draft into a closure plan focused on stabilizing compiled/runtime boundaries, adding watch-reload regression coverage, and avoiding new architectural layers.
9
+ - Simplified `runner` and `preprocess` lookup flow by removing thin wrappers, deduplicating repeated symbol fallback order, and collapsing silent program-value reads onto shared helpers.
10
+ - Added regression coverage for reload package clearing, source-backed snapshot rebuild after changes, compiled fallback cache behavior, and strict-vs-lenient runtime resolution semantics.
11
+
12
+ ## Knowledge points
13
+
14
+ - Runtime execution helpers should live in `program` so `runner` only maps runtime state to evaluation flow and user-facing errors.
15
+ - Metadata queries such as codegen type hints should prefer compiled/source schema and only fall back to ready runtime values, never by executing compiled payloads.
16
+ - Reload invalidation needs direct transitive-dependency tests plus namespace-header coverage; otherwise runtime cache cleanup regresses quietly.
17
+ - If compiled execution is used as a fallback read path, tests must assert it does not implicitly backfill runtime cells; otherwise compiled/runtime boundaries drift back together.
18
+ - Keep cleanup work biased toward removing duplicate lookup order and unused parameters before introducing any new abstraction; the stable payoff is clearer boundaries with lower entity count.
19
+ - `runner`-side error mapping for strict runtime resolution is still a meaningful boundary, but repeated namespace lookup order and required-value program reads can be shared safely.
20
+
21
+ ## Validation
22
+
23
+ - `cargo fmt`
24
+ - release fibo profiling during optimization review
25
+ - `cargo test clear_runtime_caches_for_reload -- --nocapture`
26
+ - `cargo test snapshot_rebuilds_changed_source_backed_def_after_reload_changes -- --nocapture`
27
+ - `cargo test program::tests -- --nocapture`
28
+ - `cargo test -q`
@@ -0,0 +1,87 @@
1
+ # 2026-03-17 下午至晚间改动总结(1739-2031)
2
+
3
+ ## 总览
4
+
5
+ - 时间段:`2026-0317-1739` 至 `2026-0317-2031`
6
+ - 主要改动:测试减法、program/preprocess 架构减法、类型注解热路径优化、samply 验证、计划文档同步
7
+
8
+ ## 关键变更(按时间顺序)
9
+
10
+ ### 1) 测试减法(1739 / 1745)
11
+
12
+ - 清理 `src/program/tests.rs` 中 helper/重复语义测试:
13
+ - `runtime_snapshot_fallback_only_allows_runtime_only_defs`
14
+ - `preprocess_ns_def_accepts_compiled_only_value_without_source_lookup`
15
+ - 保留行为级回归用例,继续覆盖 snapshot/runtime-only 与 compiled-only 消费边界。
16
+
17
+ ### 2) preprocess 推断逻辑去重(1751)
18
+
19
+ - 文件:`src/runner/preprocess.rs`
20
+ - 抽取 `infer_return_type_from_compiled_callable(...)`,统一 `Import`/`Symbol` 分支的 compiled callable 返回类型推断。
21
+ - 保留 `Symbol` 分支对源码 tag 的回退解析行为。
22
+
23
+ ### 3) program snapshot helper 内联(1753)
24
+
25
+ - 文件:`src/program.rs`
26
+ - 删除并内联单次用途 helper:
27
+ - `collect_referenced_compiled_def_ids(...)`
28
+ - `should_use_runtime_snapshot_fallback(...)`
29
+ - 逻辑并入 `collect_snapshot_fill_tasks(...)` 与 `build_snapshot_fill_compiled_def(...)`。
30
+
31
+ ### 4) 文档刷新 + 首轮热路径减法(1812)
32
+
33
+ - 文件:`drafts/runtime-boundary-refactor-plan.md`
34
+ - 修正过时描述:`run_program_with_docs` 现状与 preprocess 返回值关系。
35
+ - 将阶段状态与 `samply` 观察、下一步优先级对齐。
36
+ - 文件:`src/runner/preprocess.rs`
37
+ - 去除 `drop_left` 中间列表分配。
38
+ - `resolve_generic_return_type` 改为接收迭代器,调用处直接 `iter().skip(1)`。
39
+
40
+ ### 5) materialize executable fast path(2012)
41
+
42
+ - 文件:`src/program.rs`
43
+ - `materialize_compiled_executable_payload(...)`:
44
+ - `Proc | Syntax` 直接返回 `preprocessed_code`。
45
+ - `Fn | Macro` 继续 `evaluate_expr` materialize。
46
+ - `LazyValue | Value` 继续保持不可执行语义。
47
+ - 删除无调用 helper:`with_compiled_executable_payload(...)`。
48
+
49
+ ### 6) type-annotation 单次扫描收敛(2023)
50
+
51
+ - 文件:`src/calcit/type_annotation.rs`
52
+ - 在 `parse_fn_annotation_from_schema_form` 中引入 `collect_fn_schema_fields`,由多次 key 扫描改为一次遍历收集。
53
+ - 删除无用 helper:`schema_has_any_field`。
54
+
55
+ ### 7) schema key 热路径进一步减法(2031)
56
+
57
+ - 文件:`src/calcit/type_annotation.rs`
58
+ - 新增单 key 快路径:
59
+ - `schema_key_matches(...)`
60
+ - `extract_schema_value_single(...)`
61
+ - 将常见单 key 查询点改为快路径:
62
+ - `extract_return_type_from_hint_form`
63
+ - `extract_generics_from_hint_form`
64
+ - `extract_arg_types_from_hint_form`
65
+ - 清理过渡遗留 helper:
66
+ - `schema_key_matches_any(...)`
67
+ - `extract_schema_value(...)`
68
+
69
+ ## 验证汇总
70
+
71
+ - 多轮定向测试均通过:
72
+ - `cargo test -q program::tests`
73
+ - `cargo test -q runner::preprocess::tests`
74
+ - `cargo test -q calcit::type_annotation::tests`
75
+ - 全量 Rust 测试持续通过:`cargo test -q`(会话末保持全绿)
76
+ - 语义门禁持续通过:`yarn check-all`
77
+
78
+ ## profiling 结论汇总
79
+
80
+ - 使用既有流程:`profiling/samply-once.sh` + `profiling/samply-summary.py`
81
+ - 在 materialize 目标链路过滤中,样本权重由 14 降至 7(单轮观测,方向符合预期)。
82
+ - 在 schema-key 相关过滤中,基线 `fibo-release-iter5-20260317.samply` 为 21,本轮新采样 `fibo-release-20260317-203129.samply` 为 5,方向上显著下降。
83
+
84
+ ## 本轮经验
85
+
86
+ - 以“删 helper/删重复分支/删中间分配”为主线做减法,优先保证行为级测试覆盖。
87
+ - 每次改动后固定执行“定向测试 → 全量 Rust → `yarn check-all`”可有效阻断语义回退。
@@ -0,0 +1,26 @@
1
+ ## Summary
2
+
3
+ This commit finalizes two CLI workflow improvements:
4
+
5
+ 1. **Tips policy refactor**
6
+ - Added `--tips-level` support (`minimal|full|none`) as a unified switch.
7
+ - Kept `--tips` as a shortcut for full tips output.
8
+ - Updated handlers so tips rendering is centralized and priority-aware.
9
+ - Updated docs to align with the new default behavior (minimal, high-priority-first hints).
10
+
11
+ 2. **`cr edit def --overwrite` metadata safety**
12
+ - Fixed overwrite behavior to preserve existing definition metadata (`doc`, `examples`, `schema`).
13
+ - Overwrite now updates only the `code` field when the definition already exists.
14
+ - This avoids accidental schema loss during whole-definition rewrites.
15
+
16
+ ## Files touched (high level)
17
+
18
+ - CLI args and command wiring for tips level handling.
19
+ - Query/tree/tips handler integration and output behavior updates.
20
+ - `edit` handler overwrite logic for metadata-preserving updates.
21
+ - Agent docs and advanced workflow docs reflecting the new tips model.
22
+
23
+ ## Validation notes
24
+
25
+ - Rust formatting/lint/tests were run during implementation iterations.
26
+ - Manual end-to-end verification confirmed schema is retained after `--overwrite`.
package/lib/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@calcit/procs",
3
- "version": "0.12.9",
3
+ "version": "0.12.10",
4
4
  "main": "./lib/calcit.procs.mjs",
5
5
  "devDependencies": {
6
6
  "@types/node": "^25.0.9",
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@calcit/procs",
3
- "version": "0.12.9",
3
+ "version": "0.12.10",
4
4
  "main": "./lib/calcit.procs.mjs",
5
5
  "devDependencies": {
6
6
  "@types/node": "^25.0.9",
@@ -0,0 +1,84 @@
1
+ # Profiling 工具说明
2
+
3
+ 本目录集中放置 calcit 的 profiling 脚本:
4
+
5
+ - `profile-once.sh`:基于 `xctrace Time Profiler` 一键录制并汇总。
6
+ - `profile-summary.py`:解析 `xctrace` 导出的 XML 热点。
7
+ - `samply-once.sh`:基于 `samply` 一键录制并汇总(函数级 self-time)。
8
+ - `samply-summary.py`:解析 `.samply` 并输出热点函数占比。
9
+
10
+ ## 1) xctrace 路线
11
+
12
+ ### 一键执行
13
+
14
+ ```bash
15
+ profiling/profile-once.sh calcit/fibo.cirru --top 20 --include 'calcit::'
16
+ ```
17
+
18
+ 输出:
19
+
20
+ - `.tmp-profiles/<entry>-<timestamp>.trace`
21
+ - 终端中的热点函数统计、前缀聚合和优化提示
22
+
23
+ ### 仅汇总已有 trace/xml
24
+
25
+ ```bash
26
+ python3 profiling/profile-summary.py --trace .tmp-profiles/fibo-xxxx.trace --top 30
27
+ python3 profiling/profile-summary.py --xml /path/to/time-profile.xml --top 30
28
+ ```
29
+
30
+ 常用参数:
31
+
32
+ - `--top N`:显示前 N 个热点
33
+ - `--include REGEX`:仅保留匹配符号(可重复)
34
+ - `--exclude REGEX`:排除匹配符号(可重复)
35
+ - `--keep-xml`:保留由 trace 导出的临时 XML
36
+
37
+ ## 2) samply 路线(更精准函数级)
38
+
39
+ ### 一键执行(debug)
40
+
41
+ ```bash
42
+ profiling/samply-once.sh calcit/fibo.cirru --top 20 --collapse-hash
43
+ ```
44
+
45
+ ### 一键执行(release)
46
+
47
+ ```bash
48
+ profiling/samply-once.sh calcit/fibo.cirru --release --top 20 --include 'calcit::|im_ternary_tree|alloc::'
49
+ ```
50
+
51
+ 输出:
52
+
53
+ - `.tmp-profiles/<entry>-<mode>-<timestamp>.samply`
54
+ - 终端中的函数级 self-time 热点(含占比)
55
+
56
+ ### 仅汇总已有 `.samply`
57
+
58
+ ```bash
59
+ python3 profiling/samply-summary.py --input .tmp-profiles/fibo-debug-xxxx.samply --binary target/debug/cr --top 30
60
+ ```
61
+
62
+ release 对应:
63
+
64
+ ```bash
65
+ python3 profiling/samply-summary.py --input .tmp-profiles/fibo-release-xxxx.samply --binary target/release/cr --top 30
66
+ ```
67
+
68
+ 常用参数:
69
+
70
+ - `--thread INDEX`:指定线程,默认自动选择样本最多线程
71
+ - `--collapse-hash`:折叠 Rust 哈希后缀,便于分组
72
+ - `--image-base 0x100000000`:`atos` 符号化基址(默认值)
73
+ - `--include / --exclude`:正则过滤
74
+
75
+ ## 依赖
76
+
77
+ - xctrace 路线:`xctrace`, `cargo`, `python3`
78
+ - samply 路线:`samply`, `cargo`, `python3`
79
+ - 可选(samply 地址回填):`atos`(macOS 自带)
80
+
81
+ ## 说明
82
+
83
+ - `samply-once.sh` 会先 `cargo build`,再直接对 `target/debug/cr` 或 `target/release/cr` 录制,避免把 `rustc` 编译过程混入热点。
84
+ - 临时产物位于 `.tmp-profiles/`,可按需清理。
@@ -0,0 +1,44 @@
1
+ #!/usr/bin/env bash
2
+
3
+ set -euo pipefail
4
+
5
+ if [[ $# -lt 1 ]]; then
6
+ echo "Usage: profiling/profile-once.sh <entry.cirru> [profile-summary args...]" >&2
7
+ echo "Example: profiling/profile-once.sh calcit/fibo.cirru --top 20 --include 'calcit::'" >&2
8
+ exit 1
9
+ fi
10
+
11
+ entry="$1"
12
+ shift
13
+
14
+ if [[ ! -f "$entry" ]]; then
15
+ echo "Entry file not found: $entry" >&2
16
+ exit 2
17
+ fi
18
+
19
+ for cmd in xctrace cargo python3; do
20
+ if ! command -v "$cmd" >/dev/null 2>&1; then
21
+ echo "Missing required command: $cmd" >&2
22
+ exit 2
23
+ fi
24
+ done
25
+
26
+ mkdir -p .tmp-profiles
27
+
28
+ entry_name="$(basename "$entry" .cirru)"
29
+ stamp="$(date +%Y%m%d-%H%M%S)"
30
+ trace_path=".tmp-profiles/${entry_name}-${stamp}.trace"
31
+
32
+ echo "Recording trace to: $trace_path"
33
+ xctrace record \
34
+ --template "Time Profiler" \
35
+ --output "$trace_path" \
36
+ --launch -- \
37
+ cargo run --release --bin cr -- "$entry"
38
+
39
+ echo
40
+ echo "Summarizing hotspots..."
41
+ python3 profiling/profile-summary.py --trace "$trace_path" "$@"
42
+
43
+ echo
44
+ echo "Done. Trace bundle: $trace_path"
@@ -0,0 +1,197 @@
1
+ #!/usr/bin/env python3
2
+
3
+ import argparse
4
+ import os
5
+ import re
6
+ import subprocess
7
+ import sys
8
+ import tempfile
9
+ import xml.etree.ElementTree as ET
10
+ from collections import Counter
11
+ from pathlib import Path
12
+
13
+
14
+ def export_time_profile_xml(trace_path: Path, xml_path: Path) -> None:
15
+ cmd = [
16
+ "xctrace",
17
+ "export",
18
+ "--input",
19
+ str(trace_path),
20
+ "--xpath",
21
+ '/trace-toc/run[@number="1"]/data/table[@schema="time-profile"]',
22
+ "--output",
23
+ str(xml_path),
24
+ ]
25
+ result = subprocess.run(cmd, capture_output=True, text=True)
26
+ if result.returncode != 0:
27
+ stderr = result.stderr.strip() or "(no stderr)"
28
+ raise RuntimeError(f"xctrace export failed: {stderr}")
29
+
30
+
31
+ def parse_frame_names(xml_path: Path) -> Counter[str]:
32
+ counts: Counter[str] = Counter()
33
+ for _, elem in ET.iterparse(xml_path, events=("end",)):
34
+ if elem.tag == "frame":
35
+ name = elem.attrib.get("name")
36
+ if name:
37
+ counts[name] += 1
38
+ elem.clear()
39
+ return counts
40
+
41
+
42
+ def compile_patterns(patterns: list[str]) -> list[re.Pattern[str]]:
43
+ compiled = []
44
+ for pattern in patterns:
45
+ compiled.append(re.compile(pattern))
46
+ return compiled
47
+
48
+
49
+ def filter_counts(
50
+ counts: Counter[str],
51
+ include_patterns: list[re.Pattern[str]],
52
+ exclude_patterns: list[re.Pattern[str]],
53
+ ) -> Counter[str]:
54
+ if not include_patterns and not exclude_patterns:
55
+ return counts
56
+
57
+ filtered: Counter[str] = Counter()
58
+ for name, count in counts.items():
59
+ if include_patterns and not any(regex.search(name) for regex in include_patterns):
60
+ continue
61
+ if exclude_patterns and any(regex.search(name) for regex in exclude_patterns):
62
+ continue
63
+ filtered[name] = count
64
+ return filtered
65
+
66
+
67
+ def summarize_by_prefix(counts: Counter[str]) -> list[tuple[str, int]]:
68
+ group = Counter()
69
+ for name, count in counts.items():
70
+ parts = name.split("::")
71
+ if len(parts) >= 2:
72
+ key = "::".join(parts[:2])
73
+ else:
74
+ key = parts[0]
75
+ group[key] += count
76
+ return group.most_common(10)
77
+
78
+
79
+ def derive_hints(counts: Counter[str]) -> list[str]:
80
+ joined = "\n".join(counts.keys())
81
+ hints: list[str] = []
82
+ if re.search(r"calcit::runner::(call_expr|evaluate_expr|run_fn_owned)", joined):
83
+ hints.append("解释器调度路径是主要热点,可优先检查 `call_expr`/`evaluate_expr` 的分支和数据转换。")
84
+ if re.search(r"CalcitProc::get_type_signature|check_proc_arity|type_annotation", joined):
85
+ hints.append("运行时类型签名/arity 检查占比不低,可考虑缓存签名结果或减少重复检查路径。")
86
+ if re.search(r"im_ternary_tree::.*to_vec|CalcitList::to_vec|drop_left", joined):
87
+ hints.append("持久结构与 Vec 转换频繁,建议减少 `to_vec` 往返或批量化访问。")
88
+ if re.search(r"alloc::|RawVec|drop::|triomphe::|rpds::", joined):
89
+ hints.append("分配与释放开销明显,建议优先减少短生命周期对象和临时容器创建。")
90
+ return hints
91
+
92
+
93
+ def parse_args() -> argparse.Namespace:
94
+ parser = argparse.ArgumentParser(
95
+ description="Summarize xctrace Time Profiler output into LLM-friendly hotspot text."
96
+ )
97
+ parser.add_argument("--trace", type=Path, help="Path to .trace bundle generated by xctrace")
98
+ parser.add_argument("--xml", type=Path, help="Path to exported time-profile XML")
99
+ parser.add_argument("--top", type=int, default=30, help="Number of top hotspot functions")
100
+ parser.add_argument(
101
+ "--include",
102
+ action="append",
103
+ default=[],
104
+ help="Regex include filter for symbol names (repeatable)",
105
+ )
106
+ parser.add_argument(
107
+ "--exclude",
108
+ action="append",
109
+ default=[],
110
+ help="Regex exclude filter for symbol names (repeatable)",
111
+ )
112
+ parser.add_argument(
113
+ "--keep-xml",
114
+ action="store_true",
115
+ help="Keep temporary XML when using --trace",
116
+ )
117
+ args = parser.parse_args()
118
+ if not args.trace and not args.xml:
119
+ parser.error("One of --trace or --xml is required")
120
+ if args.trace and args.xml:
121
+ parser.error("Use only one of --trace or --xml")
122
+ if args.top <= 0:
123
+ parser.error("--top must be > 0")
124
+ return args
125
+
126
+
127
+ def main() -> int:
128
+ args = parse_args()
129
+
130
+ xml_path: Path
131
+ temp_dir = None
132
+ if args.trace:
133
+ if not args.trace.exists():
134
+ print(f"Trace file not found: {args.trace}", file=sys.stderr)
135
+ return 2
136
+ temp_dir = tempfile.TemporaryDirectory(prefix="xctrace-export-")
137
+ xml_path = Path(temp_dir.name) / "time-profile.xml"
138
+ try:
139
+ export_time_profile_xml(args.trace, xml_path)
140
+ except RuntimeError as error:
141
+ print(str(error), file=sys.stderr)
142
+ return 3
143
+ if args.keep_xml:
144
+ kept = args.trace.parent / f"{args.trace.name}.time-profile.xml"
145
+ kept.write_bytes(xml_path.read_bytes())
146
+ print(f"Saved exported XML to: {kept}")
147
+ else:
148
+ xml_path = args.xml
149
+ if not xml_path.exists():
150
+ print(f"XML file not found: {xml_path}", file=sys.stderr)
151
+ return 2
152
+
153
+ counts = parse_frame_names(xml_path)
154
+ include_patterns = compile_patterns(args.include)
155
+ exclude_patterns = compile_patterns(args.exclude)
156
+ filtered = filter_counts(counts, include_patterns, exclude_patterns)
157
+
158
+ total_frames = sum(counts.values())
159
+ filtered_frames = sum(filtered.values())
160
+ print(f"Source XML: {xml_path}")
161
+ print(f"Total named frames: {total_frames}")
162
+ print(f"Frames after filter: {filtered_frames}")
163
+ print()
164
+
165
+ print(f"Top {args.top} Hotspots")
166
+ print("-" * 72)
167
+ if not filtered:
168
+ print("(no matched symbols)")
169
+ else:
170
+ for name, count in filtered.most_common(args.top):
171
+ ratio = (count / filtered_frames * 100.0) if filtered_frames else 0.0
172
+ print(f"{count:6d} {ratio:6.2f}% {name}")
173
+
174
+ print()
175
+ print("Top Prefix Groups")
176
+ print("-" * 72)
177
+ for group, count in summarize_by_prefix(filtered):
178
+ ratio = (count / filtered_frames * 100.0) if filtered_frames else 0.0
179
+ print(f"{count:6d} {ratio:6.2f}% {group}")
180
+
181
+ print()
182
+ print("Optimization Hints")
183
+ print("-" * 72)
184
+ hints = derive_hints(filtered)
185
+ if not hints:
186
+ print("- 无明显通用模式,请结合 Top Hotspots 逐个函数分析。")
187
+ else:
188
+ for hint in hints:
189
+ print(f"- {hint}")
190
+
191
+ if temp_dir is not None and not args.keep_xml:
192
+ temp_dir.cleanup()
193
+ return 0
194
+
195
+
196
+ if __name__ == "__main__":
197
+ raise SystemExit(main())
@@ -0,0 +1,65 @@
1
+ #!/usr/bin/env bash
2
+
3
+ set -euo pipefail
4
+
5
+ if [[ $# -lt 1 ]]; then
6
+ echo "Usage: profiling/samply-once.sh <entry.cirru> [--release] [samply-summary args...]" >&2
7
+ echo "Example: profiling/samply-once.sh calcit/fibo.cirru --top 20 --include 'calcit::'" >&2
8
+ echo "Example: profiling/samply-once.sh calcit/fibo.cirru --release --collapse-hash" >&2
9
+ exit 1
10
+ fi
11
+
12
+ entry="$1"
13
+ shift
14
+
15
+ if [[ ! -f "$entry" ]]; then
16
+ echo "Entry file not found: $entry" >&2
17
+ exit 2
18
+ fi
19
+
20
+ mode="debug"
21
+ if [[ "${1:-}" == "--release" ]]; then
22
+ mode="release"
23
+ shift
24
+ fi
25
+
26
+ for cmd in samply cargo python3; do
27
+ if ! command -v "$cmd" >/dev/null 2>&1; then
28
+ echo "Missing required command: $cmd" >&2
29
+ exit 2
30
+ fi
31
+ done
32
+
33
+ mkdir -p .tmp-profiles
34
+
35
+ entry_name="$(basename "$entry" .cirru)"
36
+ stamp="$(date +%Y%m%d-%H%M%S)"
37
+ samply_path=".tmp-profiles/${entry_name}-${mode}-${stamp}.samply"
38
+
39
+ if [[ "$mode" == "release" ]]; then
40
+ build_args=(build --release --bin cr)
41
+ binary="target/release/cr"
42
+ else
43
+ build_args=(build --bin cr)
44
+ binary="target/debug/cr"
45
+ fi
46
+
47
+ echo "Building binary: $binary"
48
+ cargo "${build_args[@]}"
49
+
50
+ if [[ ! -f "$binary" ]]; then
51
+ echo "Expected binary not found after build: $binary" >&2
52
+ exit 3
53
+ fi
54
+
55
+ echo "Recording samply profile to: $samply_path"
56
+ samply record --save-only -o "$samply_path" -- "$binary" "$entry"
57
+
58
+ binary_arg=(--binary "$binary")
59
+
60
+ echo
61
+ echo "Summarizing hotspots..."
62
+ python3 profiling/samply-summary.py --input "$samply_path" "${binary_arg[@]}" "$@"
63
+
64
+ echo
65
+ echo "Done. Samply file: $samply_path"
@@ -0,0 +1,255 @@
1
+ #!/usr/bin/env python3
2
+
3
+ import argparse
4
+ import json
5
+ import re
6
+ import subprocess
7
+ import sys
8
+ from collections import Counter
9
+ from pathlib import Path
10
+
11
+
12
+ ADDRESS_RE = re.compile(r"^0x[0-9a-fA-F]+$")
13
+
14
+
15
+ def parse_args() -> argparse.Namespace:
16
+ parser = argparse.ArgumentParser(
17
+ description="Summarize samply JSON into function-level self-time hotspots."
18
+ )
19
+ parser.add_argument("--input", type=Path, required=True, help="Path to .samply JSON")
20
+ parser.add_argument("--top", type=int, default=30, help="Number of top functions")
21
+ parser.add_argument("--thread", type=int, help="Thread index to analyze (default: auto)")
22
+ parser.add_argument(
23
+ "--include",
24
+ action="append",
25
+ default=[],
26
+ help="Regex include filter for symbol names (repeatable)",
27
+ )
28
+ parser.add_argument(
29
+ "--exclude",
30
+ action="append",
31
+ default=[],
32
+ help="Regex exclude filter for symbol names (repeatable)",
33
+ )
34
+ parser.add_argument(
35
+ "--binary",
36
+ type=Path,
37
+ help="Mach-O binary for atos symbolization fallback (e.g. target/debug/cr)",
38
+ )
39
+ parser.add_argument(
40
+ "--image-base",
41
+ type=lambda raw: int(raw, 0),
42
+ default=0x100000000,
43
+ help="Image base added to frame address before atos (default: 0x100000000)",
44
+ )
45
+ parser.add_argument(
46
+ "--collapse-hash",
47
+ action="store_true",
48
+ help="Collapse Rust symbol hash suffix (::h...) for easier grouping",
49
+ )
50
+ args = parser.parse_args()
51
+ if args.top <= 0:
52
+ parser.error("--top must be > 0")
53
+ return args
54
+
55
+
56
+ def compile_patterns(patterns: list[str]) -> list[re.Pattern[str]]:
57
+ return [re.compile(pattern) for pattern in patterns]
58
+
59
+
60
+ def choose_thread(data: dict, explicit: int | None) -> tuple[int, dict]:
61
+ threads = data.get("threads")
62
+ if not isinstance(threads, list) or not threads:
63
+ raise ValueError("No threads found in .samply file")
64
+
65
+ if explicit is not None:
66
+ if explicit < 0 or explicit >= len(threads):
67
+ raise ValueError(f"--thread out of range: {explicit} (total {len(threads)})")
68
+ return explicit, threads[explicit]
69
+
70
+ best_idx = 0
71
+ best_count = -1
72
+ for index, thread in enumerate(threads):
73
+ samples = thread.get("samples", {})
74
+ stacks = samples.get("stack", [])
75
+ count = sum(1 for entry in stacks if entry is not None)
76
+ if count > best_count:
77
+ best_count = count
78
+ best_idx = index
79
+ return best_idx, threads[best_idx]
80
+
81
+
82
+ def lookup_frame_symbol(thread: dict, frame_index: int) -> tuple[str, int | None]:
83
+ frame_table = thread.get("frameTable", {})
84
+ func_table = thread.get("funcTable", {})
85
+ native_symbols = thread.get("nativeSymbols", {})
86
+ strings = thread.get("stringArray", [])
87
+
88
+ func_index = frame_table.get("func", [None])[frame_index]
89
+ if func_index is not None:
90
+ name_index = func_table.get("name", [None])[func_index]
91
+ if name_index is not None:
92
+ symbol = strings[name_index]
93
+ if ADDRESS_RE.match(symbol):
94
+ return symbol, int(symbol, 16)
95
+ return symbol, None
96
+
97
+ native_index = frame_table.get("nativeSymbol", [None])[frame_index]
98
+ if native_index is not None:
99
+ name_index = native_symbols.get("name", [None])[native_index]
100
+ if name_index is not None:
101
+ symbol = strings[name_index]
102
+ if ADDRESS_RE.match(symbol):
103
+ return symbol, int(symbol, 16)
104
+ return symbol, None
105
+
106
+ address = frame_table.get("address", [None])[frame_index]
107
+ if address is not None:
108
+ return f"0x{address:x}", address
109
+
110
+ return "<unknown>", None
111
+
112
+
113
+ def atos_symbolize(binary: Path, image_base: int, addresses: list[int]) -> dict[int, str]:
114
+ if not addresses:
115
+ return {}
116
+
117
+ absolute_addresses = [hex(image_base + address) for address in addresses]
118
+ cmd = ["atos", "-o", str(binary), *absolute_addresses]
119
+ result = subprocess.run(cmd, capture_output=True, text=True)
120
+
121
+ mapping: dict[int, str] = {}
122
+ lines = result.stdout.splitlines()
123
+ for address, line in zip(addresses, lines):
124
+ symbol = line.strip()
125
+ if symbol and not ADDRESS_RE.match(symbol):
126
+ mapping[address] = symbol
127
+ return mapping
128
+
129
+
130
+ def normalize_symbol(raw: str, collapse_hash: bool) -> str:
131
+ text = raw.strip()
132
+ if collapse_hash:
133
+ text = re.sub(r"::h[0-9a-f]{16,}$", "", text)
134
+ return text
135
+
136
+
137
+ def summarize(
138
+ thread: dict,
139
+ include_patterns: list[re.Pattern[str]],
140
+ exclude_patterns: list[re.Pattern[str]],
141
+ collapse_hash: bool,
142
+ atos_map: dict[int, str],
143
+ ) -> tuple[Counter[str], float]:
144
+ stack_table = thread.get("stackTable", {})
145
+ samples = thread.get("samples", {})
146
+
147
+ sample_stacks = samples.get("stack", [])
148
+ weights = samples.get("weight")
149
+ if not isinstance(weights, list) or len(weights) != len(sample_stacks):
150
+ weights = [1.0] * len(sample_stacks)
151
+
152
+ counts: Counter[str] = Counter()
153
+ total_weight = 0.0
154
+ for index, stack_index in enumerate(sample_stacks):
155
+ if stack_index is None:
156
+ continue
157
+ frame_index = stack_table.get("frame", [None])[stack_index]
158
+ if frame_index is None:
159
+ continue
160
+ symbol, address = lookup_frame_symbol(thread, frame_index)
161
+ if address is not None and address in atos_map:
162
+ symbol = atos_map[address]
163
+ symbol = normalize_symbol(symbol, collapse_hash)
164
+
165
+ if include_patterns and not any(regex.search(symbol) for regex in include_patterns):
166
+ continue
167
+ if exclude_patterns and any(regex.search(symbol) for regex in exclude_patterns):
168
+ continue
169
+
170
+ weight = float(weights[index])
171
+ counts[symbol] += weight
172
+ total_weight += weight
173
+
174
+ return counts, total_weight
175
+
176
+
177
+ def collect_unresolved_addresses(thread: dict, sample_limit: int) -> list[int]:
178
+ stack_table = thread.get("stackTable", {})
179
+ sample_stacks = thread.get("samples", {}).get("stack", [])
180
+ addresses: Counter[int] = Counter()
181
+
182
+ for stack_index in sample_stacks:
183
+ if stack_index is None:
184
+ continue
185
+ frame_index = stack_table.get("frame", [None])[stack_index]
186
+ if frame_index is None:
187
+ continue
188
+ symbol, address = lookup_frame_symbol(thread, frame_index)
189
+ if address is None:
190
+ continue
191
+ if ADDRESS_RE.match(symbol):
192
+ addresses[address] += 1
193
+
194
+ return [address for address, _ in addresses.most_common(sample_limit)]
195
+
196
+
197
+ def main() -> int:
198
+ args = parse_args()
199
+ if not args.input.exists():
200
+ print(f"Input file not found: {args.input}", file=sys.stderr)
201
+ return 2
202
+
203
+ try:
204
+ data = json.loads(args.input.read_text())
205
+ except json.JSONDecodeError as error:
206
+ print(f"Invalid JSON in {args.input}: {error}", file=sys.stderr)
207
+ return 2
208
+
209
+ try:
210
+ thread_index, thread = choose_thread(data, args.thread)
211
+ except ValueError as error:
212
+ print(str(error), file=sys.stderr)
213
+ return 2
214
+
215
+ include_patterns = compile_patterns(args.include)
216
+ exclude_patterns = compile_patterns(args.exclude)
217
+
218
+ atos_map: dict[int, str] = {}
219
+ if args.binary is not None:
220
+ unresolved = collect_unresolved_addresses(thread, sample_limit=max(200, args.top * 10))
221
+ if unresolved:
222
+ atos_map = atos_symbolize(args.binary, args.image_base, unresolved)
223
+
224
+ counts, total_weight = summarize(
225
+ thread,
226
+ include_patterns,
227
+ exclude_patterns,
228
+ args.collapse_hash,
229
+ atos_map,
230
+ )
231
+
232
+ print(f"Input .samply: {args.input}")
233
+ print(f"Thread index: {thread_index}")
234
+ print(f"Thread name: {thread.get('name', '<unknown>')}")
235
+ print(f"Samples after filter (weight): {total_weight:.2f}")
236
+ if args.binary is not None:
237
+ print(f"Atos binary: {args.binary}")
238
+ print(f"Atos symbolized addresses: {len(atos_map)}")
239
+ print()
240
+
241
+ print(f"Top {args.top} Self-Time Hotspots")
242
+ print("-" * 72)
243
+ if not counts:
244
+ print("(no matched symbols)")
245
+ return 0
246
+
247
+ for symbol, weight in counts.most_common(args.top):
248
+ ratio = (weight / total_weight * 100.0) if total_weight else 0.0
249
+ print(f"{weight:10.2f} {ratio:6.2f}% {symbol}")
250
+
251
+ return 0
252
+
253
+
254
+ if __name__ == "__main__":
255
+ raise SystemExit(main())