oh-langfuse 0.1.41 → 0.1.43
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +62 -85
- package/bin/cli.js +21 -14
- package/codex_langfuse_notify.py +139 -59
- package/langfuse_hook.py +223 -57
- package/package.json +35 -35
- package/scripts/auto-update-runtime.mjs +4 -2
- package/scripts/codex-langfuse-setup.mjs +163 -11
- package/scripts/langfuse-check.mjs +11 -5
- package/scripts/langfuse-setup.mjs +155 -12
- package/scripts/metrics-utils.mjs +134 -10
- package/scripts/opencode-langfuse-setup.mjs +118 -61
- package/scripts/real-self-verify.mjs +13 -8
package/README.md
CHANGED
|
@@ -1,20 +1,12 @@
|
|
|
1
|
-
# oh-langfuse
|
|
1
|
+
# oh-langfuse
|
|
2
2
|
|
|
3
|
-
`oh-langfuse`
|
|
3
|
+
`oh-langfuse` 鏄敤浜庣粰 Claude Code銆丱penCode 鍜?Codex 閰嶇疆 Langfuse 杩借釜鐨勫懡浠よ宸ュ叿銆傚畠鎻愪緵浜や簰寮忓畨瑁呭悜瀵硷紝涔熸敮鎸?`setup` / `check` 鐩存帴鍛戒护锛屾柟渚垮湪鐢ㄦ埛鏈哄櫒涓婂畨瑁呫€佷慨澶嶅拰鏍¢獙閰嶇疆銆?
|
|
4
|
+
褰撳墠 npm 鐗堟湰锛歚0.1.41`
|
|
4
5
|
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
##
|
|
8
|
-
|
|
9
|
-
- 给 Claude Code 安装 `Stop` hook,把会话事件写入 Langfuse。
|
|
10
|
-
- 给 OpenCode 安装并 patch `opencode-plugin-langfuse`,开启 OpenTelemetry。
|
|
11
|
-
- 给 Codex 安装 `notify` hook,增量读取 session JSONL 并写入 Langfuse。
|
|
12
|
-
- 安装完成后自动执行对应 `check`,尽早发现配置缺失。
|
|
13
|
-
- 校验员工号格式,必须匹配 `^[a-z](?:\d{8}|wx\d{7})$`,例如 `h00613222` 或 `hwx1234567`。
|
|
14
|
-
|
|
15
|
-
## 快速使用
|
|
16
|
-
|
|
17
|
-
建议始终带 `@latest`,避免 npx 使用本地旧缓存:
|
|
6
|
+
## 鑳藉仛浠€涔?
|
|
7
|
+
- 缁?Claude Code 瀹夎 `Stop` hook锛屾妸浼氳瘽浜嬩欢鍐欏叆 Langfuse銆?- 缁?OpenCode 瀹夎骞?patch `opencode-plugin-langfuse`锛屽紑鍚?OpenTelemetry銆?- 缁?Codex 瀹夎 `notify` hook锛屽閲忚鍙?session JSONL 骞跺啓鍏?Langfuse銆?- 瀹夎瀹屾垚鍚庤嚜鍔ㄦ墽琛屽搴?`check`锛屽敖鏃╁彂鐜伴厤缃己澶便€?- 鏍¢獙鍛樺伐鍙锋牸寮忥紝蹇呴』鍖归厤 `^[a-z](?:\d{8}|wx\d{7})$`锛屼緥濡?`h00613222` 鎴?`hwx1234567`銆?
|
|
8
|
+
## 蹇€熶娇鐢?
|
|
9
|
+
寤鸿濮嬬粓甯?`@latest`锛岄伩鍏?npx 浣跨敤鏈湴鏃х紦瀛橈細
|
|
18
10
|
|
|
19
11
|
```bash
|
|
20
12
|
npx oh-langfuse@latest
|
|
@@ -22,7 +14,7 @@ npx oh-langfuse@latest setup
|
|
|
22
14
|
npx oh-langfuse@latest check
|
|
23
15
|
```
|
|
24
16
|
|
|
25
|
-
|
|
17
|
+
涔熷彲浠ユ寚瀹氱洰鏍囷細
|
|
26
18
|
|
|
27
19
|
```bash
|
|
28
20
|
npx oh-langfuse@latest setup claude
|
|
@@ -34,19 +26,15 @@ npx oh-langfuse@latest check opencode
|
|
|
34
26
|
npx oh-langfuse@latest check codex
|
|
35
27
|
```
|
|
36
28
|
|
|
37
|
-
##
|
|
38
|
-
|
|
39
|
-
安装或更新 runtime 后,工具会在 `~/.config/oh-langfuse/runtime.json` 记录 Claude Code / OpenCode / Codex 当前写入本机 runtime 的 `oh-langfuse` 版本。
|
|
40
|
-
|
|
41
|
-
OpenCode 生成的 `launch-opencode-langfuse.*` 会在启动 agent 前执行一次更新检测:如果 npm 上的 `oh-langfuse@latest` 高于本机 runtime 版本,会提示是否更新;用户确认后才会运行 `npx oh-langfuse@latest update opencode`。Claude Code 和 Codex 也会生成对应 launcher:
|
|
42
|
-
|
|
43
|
-
OpenCode setup 还会生成 `~/.config/opencode/bin/opencode` / `opencode.cmd` 命令包装器,并把该目录写入用户 PATH。新开终端后直接运行 `opencode`,会先用轻量脚本检测 `oh-langfuse` runtime 是否需要更新,再转发到真实 OpenCode CLI。为了避免 Windows 下在 agent 启动链路里嵌套运行 Node/npm installer 导致启动崩溃,`opencode` 命令只做检测和提示;需要更新时请关闭 agent 后运行提示中的 `npx oh-langfuse@latest update opencode`。
|
|
29
|
+
## 鑷姩鏇存柊
|
|
44
30
|
|
|
31
|
+
瀹夎鎴栨洿鏂?runtime 鍚庯紝宸ュ叿浼氬湪 `~/.config/oh-langfuse/runtime.json` 璁板綍 Claude Code / OpenCode / Codex 褰撳墠鍐欏叆鏈満 runtime 鐨?`oh-langfuse` 鐗堟湰銆?
|
|
32
|
+
OpenCode 鐢熸垚鐨?`launch-opencode-langfuse.*` 浼氬湪鍚姩 agent 鍓嶆墽琛屼竴娆℃洿鏂版娴嬶細濡傛灉 npm 涓婄殑 `oh-langfuse@latest` 楂樹簬鏈満 runtime 鐗堟湰锛屼細鎻愮ず鏄惁鏇存柊锛涚敤鎴风‘璁ゅ悗鎵嶄細杩愯 `npx oh-langfuse@latest update opencode`銆侰laude Code 鍜?Codex 涔熶細鐢熸垚瀵瑰簲 launcher锛?
|
|
33
|
+
OpenCode setup 杩樹細鐢熸垚 `~/.config/opencode/bin/opencode` / `opencode.cmd` 鍛戒护鍖呰鍣紝骞舵妸璇ョ洰褰曞啓鍏ョ敤鎴?PATH銆傛柊寮€缁堢鍚庣洿鎺ヨ繍琛?`opencode`锛屼細鍏堢敤杞婚噺鑴氭湰妫€娴?`oh-langfuse` runtime 鏄惁闇€瑕佹洿鏂帮紝鍐嶈浆鍙戝埌鐪熷疄 OpenCode CLI銆備负浜嗛伩鍏?Windows 涓嬪湪 agent 鍚姩閾捐矾閲屽祵濂楄繍琛?Node/npm installer 瀵艰嚧鍚姩宕╂簝锛宍opencode` 鍛戒护鍙仛妫€娴嬪拰鎻愮ず锛涢渶瑕佹洿鏂版椂璇峰叧闂?agent 鍚庤繍琛屾彁绀轰腑鐨?`npx oh-langfuse@latest update opencode`銆?
|
|
45
34
|
- `~/.claude/launch-claude-langfuse.cmd` / `~/.claude/launch-claude-langfuse.sh`
|
|
46
35
|
- `~/.codex/launch-codex-langfuse.cmd` / `~/.codex/launch-codex-langfuse.sh`
|
|
47
36
|
|
|
48
|
-
|
|
49
|
-
|
|
37
|
+
濡傛灉鐩存帴杩愯绯荤粺鍘熷鐨?`claude` / `codex` 鍛戒护锛屽伐鍏蜂笉浼氬己琛屾浛鎹㈣鍛戒护锛涢渶瑕佷娇鐢ㄤ笂杩?launcher 鎵嶈兘鍋氬埌鍚姩鍓嶆娴嬪苟鎻愮ず鏇存柊銆備篃鍙互鎵嬪姩杩愯锛?
|
|
50
38
|
```bash
|
|
51
39
|
npx oh-langfuse@latest auto-update opencode
|
|
52
40
|
npx oh-langfuse@latest auto-update claude
|
|
@@ -55,111 +43,100 @@ npx oh-langfuse@latest auto-update codex
|
|
|
55
43
|
|
|
56
44
|
## Skill 使用量统计
|
|
57
45
|
|
|
58
|
-
|
|
46
|
+
工具会把 skill 使用汇总写入每次交互的 `Agent Turn` observation,不再额外生成 `Skill Use` observation。
|
|
59
47
|
|
|
60
48
|
```text
|
|
61
|
-
Observation Name:
|
|
62
|
-
metadata.
|
|
63
|
-
metadata.
|
|
49
|
+
Observation Name: Agent Turn
|
|
50
|
+
metadata.skill_use_count: 本轮 skill 使用总次数
|
|
51
|
+
metadata.unique_skill_count: 本轮不同 skill 数量
|
|
52
|
+
metadata.repeated_skill_count: 本轮重复 skill 使用次数
|
|
53
|
+
metadata.skill_names: 去重后的 skill 名称列表
|
|
54
|
+
metadata.skill_names_all: 按使用顺序记录的 skill 名称列表
|
|
64
55
|
```
|
|
65
56
|
|
|
66
57
|
Dashboard 中统计全部 skill 使用量可配置:
|
|
67
58
|
|
|
68
59
|
```text
|
|
69
60
|
View: Observations
|
|
70
|
-
Metric:
|
|
71
|
-
Filter: Observation Name equals
|
|
61
|
+
Metric: Sum skill_use_count
|
|
62
|
+
Filter: Observation Name equals Agent Turn
|
|
72
63
|
```
|
|
73
64
|
|
|
74
|
-
如果当前 Langfuse Dashboard 支持 metadata 维度,可用 `metadata.
|
|
75
|
-
|
|
76
|
-
本地开发运行:
|
|
65
|
+
如果当前 Langfuse Dashboard 支持 metadata 维度,可用 `metadata.skill_names` 或导出数据中的 `metadata.skill_names_all` 按 skill 聚合。`skill_names_json`、`skill_names_all_json` 和 `skill_use_events_json` 不再写入,避免和数组字段重复。
|
|
66
|
+
鏈湴寮€鍙戣繍琛岋細
|
|
77
67
|
|
|
78
68
|
```bash
|
|
79
69
|
npm install
|
|
80
70
|
npm start
|
|
81
71
|
```
|
|
82
72
|
|
|
83
|
-
##
|
|
73
|
+
## 浜や簰寮忚彍鍗?
|
|
74
|
+
杩愯 `npx oh-langfuse@latest` 浼氭墦寮€浜や簰寮忚彍鍗曪細
|
|
84
75
|
|
|
85
|
-
|
|
76
|
+
- `Up` / `Down` 绉诲姩閫夐」銆?- 鏁板瓧閿揩閫熷畾浣嶃€?- `Space` 鍕鹃€夋垨鍙栨秷銆?- `Enter` 纭銆?- `q` 鎴?`Esc` 閫€鍑恒€?
|
|
77
|
+
澶氶€夎彍鍗曢粯璁や笉鍕鹃€変换浣曠洰鏍囷紝閬垮厤璇銆傜粓绔笉鏀寔鍘熷鎸夐敭杈撳叆鏃讹紝浼氳嚜鍔ㄩ檷绾т负鏁板瓧杈撳叆妯″紡銆?
|
|
78
|
+
## OpenCode 璇存槑
|
|
86
79
|
|
|
87
|
-
|
|
88
|
-
- 数字键快速定位。
|
|
89
|
-
- `Space` 勾选或取消。
|
|
90
|
-
- `Enter` 确认。
|
|
91
|
-
- `q` 或 `Esc` 退出。
|
|
92
|
-
|
|
93
|
-
多选菜单默认不勾选任何目标,避免误装。终端不支持原始按键输入时,会自动降级为数字输入模式。
|
|
94
|
-
|
|
95
|
-
## OpenCode 说明
|
|
96
|
-
|
|
97
|
-
OpenCode 安装会写入:
|
|
80
|
+
OpenCode 瀹夎浼氬啓鍏ワ細
|
|
98
81
|
|
|
99
82
|
- `~/.config/opencode/opencode.json`
|
|
100
83
|
- `~/.config/opencode/plugins/opencode-plugin-langfuse`
|
|
101
84
|
- `~/.config/opencode-plugin-langfuse/config.json`
|
|
102
|
-
- Windows
|
|
103
|
-
- Linux/macOS
|
|
104
|
-
|
|
105
|
-
Windows 下安装器会尝试写入用户级 `LANGFUSE_PUBLIC_KEY`、`LANGFUSE_SECRET_KEY`、`LANGFUSE_BASEURL`。这些变量对当前已经打开的终端不会立即生效,需要新开终端,或者使用生成的 launcher 启动 OpenCode。
|
|
106
|
-
|
|
107
|
-
`check opencode` 已按实际情况处理 Windows:如果当前终端还看不到 `LANGFUSE_*`,但用户级环境变量已写入,或 launcher 已生成,会显示 OpenCode 仍有可用的 Langfuse 环境变量来源,不再把这种情况误判为安装失败。
|
|
85
|
+
- Windows 涓嬬殑 `~/.config/opencode/launch-opencode-langfuse.cmd`
|
|
86
|
+
- Linux/macOS 涓嬬殑 `~/.config/opencode/launch-opencode-langfuse.sh`
|
|
108
87
|
|
|
109
|
-
|
|
88
|
+
Windows 涓嬪畨瑁呭櫒浼氬皾璇曞啓鍏ョ敤鎴风骇 `LANGFUSE_PUBLIC_KEY`銆乣LANGFUSE_SECRET_KEY`銆乣LANGFUSE_BASEURL`銆傝繖浜涘彉閲忓褰撳墠宸茬粡鎵撳紑鐨勭粓绔笉浼氱珛鍗崇敓鏁堬紝闇€瑕佹柊寮€缁堢锛屾垨鑰呬娇鐢ㄧ敓鎴愮殑 launcher 鍚姩 OpenCode銆?
|
|
89
|
+
`check opencode` 宸叉寜瀹為檯鎯呭喌澶勭悊 Windows锛氬鏋滃綋鍓嶇粓绔繕鐪嬩笉鍒?`LANGFUSE_*`锛屼絾鐢ㄦ埛绾х幆澧冨彉閲忓凡鍐欏叆锛屾垨 launcher 宸茬敓鎴愶紝浼氭樉绀?OpenCode 浠嶆湁鍙敤鐨?Langfuse 鐜鍙橀噺鏉ユ簮锛屼笉鍐嶆妸杩欑鎯呭喌璇垽涓哄畨瑁呭け璐ャ€?
|
|
90
|
+
WSL 鐢ㄦ埛璇风洿鎺ュ湪 WSL shell 涓畨瑁呭拰妫€鏌ワ細
|
|
110
91
|
|
|
111
92
|
```bash
|
|
112
93
|
npx oh-langfuse@latest setup opencode --userId=h00613222 --yes
|
|
113
94
|
npx oh-langfuse@latest check opencode
|
|
114
95
|
```
|
|
115
96
|
|
|
116
|
-
|
|
97
|
+
涓嶄細鍋?Windows 鍒?WSL 鐨勮浆鍙戯紱鍦?WSL 涓繍琛屾椂锛岄厤缃啓鍏?WSL 鐢ㄦ埛鑷繁鐨?`$HOME/.config/opencode`銆?
|
|
98
|
+
## Claude Code 璇存槑
|
|
117
99
|
|
|
118
|
-
|
|
100
|
+
Claude Code 瀹夎浼氾細
|
|
119
101
|
|
|
120
|
-
|
|
102
|
+
- 瀹夎 `langfuse_hook.py`
|
|
103
|
+
- 鍒涘缓 `~/.claude/langfuse-venv`
|
|
104
|
+
- 瀹夎 Python 鍖?`langfuse`
|
|
105
|
+
- 鍚堝苟鏇存柊 `~/.claude/settings.json`
|
|
106
|
+
- 閰嶇疆 `Stop` hook
|
|
121
107
|
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
- 安装 Python 包 `langfuse`
|
|
125
|
-
- 合并更新 `~/.claude/settings.json`
|
|
126
|
-
- 配置 `Stop` hook
|
|
108
|
+
Linux 濡傛灉缂哄皯 venv 鏀寔锛岃鏍规嵁鎻愮ず瀹夎 `python3-venv` 鎴栧搴?Python 鐗堟湰鐨?venv 鍖呫€?
|
|
109
|
+
## Codex 璇存槑
|
|
127
110
|
|
|
128
|
-
|
|
111
|
+
Codex 瀹夎浼氾細
|
|
129
112
|
|
|
130
|
-
|
|
113
|
+
- 瀹夎 `codex_langfuse_notify.py`
|
|
114
|
+
- 鍒涘缓 `~/.codex/langfuse-venv`
|
|
115
|
+
- 鍐欏叆 `~/.codex/langfuse/config.json`
|
|
116
|
+
- 鏇存柊 `~/.codex/config.toml` 椤跺眰 `notify`
|
|
131
117
|
|
|
132
|
-
Codex
|
|
118
|
+
閰嶇疆鍚庨渶瑕侀噸鍚?Codex銆俷otify hook 浼氬閲忚鍙?`~/.codex/sessions/**/*.jsonl`锛屾妸鏂扮殑 user銆乤ssistant銆乼ool銆乼oken 浜嬩欢杞崲涓?Langfuse observation锛屽苟鎶婅鍙栧亸绉昏褰曞埌 `~/.codex/langfuse/state.json`銆?
|
|
119
|
+
## 鐜鍙橀噺
|
|
133
120
|
|
|
134
|
-
|
|
135
|
-
- 创建 `~/.codex/langfuse-venv`
|
|
136
|
-
- 写入 `~/.codex/langfuse/config.json`
|
|
137
|
-
- 更新 `~/.codex/config.toml` 顶层 `notify`
|
|
138
|
-
|
|
139
|
-
配置后需要重启 Codex。notify hook 会增量读取 `~/.codex/sessions/**/*.jsonl`,把新的 user、assistant、tool、token 事件转换为 Langfuse observation,并把读取偏移记录到 `~/.codex/langfuse/state.json`。
|
|
140
|
-
|
|
141
|
-
## 环境变量
|
|
142
|
-
|
|
143
|
-
| 变量 | 作用 |
|
|
121
|
+
| 鍙橀噺 | 浣滅敤 |
|
|
144
122
|
| --- | --- |
|
|
145
|
-
| `LANGFUSE_BASEURL` / `LANGFUSE_HOST` | Langfuse
|
|
123
|
+
| `LANGFUSE_BASEURL` / `LANGFUSE_HOST` | Langfuse 鏈嶅姟鍦板潃 |
|
|
146
124
|
| `LANGFUSE_PUBLIC_KEY` | Langfuse public key |
|
|
147
125
|
| `LANGFUSE_SECRET_KEY` | Langfuse secret key |
|
|
148
|
-
| `LANGFUSE_USER_ID` / `CC_USER_ID` |
|
|
149
|
-
| `CODEX_HOME` |
|
|
150
|
-
| `OPENCODE_SKIP_PLUGIN_INSTALL` |
|
|
151
|
-
| `OPENCODE_NPM_REGISTRY` | OpenCode
|
|
152
|
-
|
|
153
|
-
Secret key 不会在交互式界面中明文展示。
|
|
126
|
+
| `LANGFUSE_USER_ID` / `CC_USER_ID` | 鐢ㄦ埛鏍囪瘑锛屽繀椤诲尮閰嶅憳宸ュ彿瑙勫垯 |
|
|
127
|
+
| `CODEX_HOME` | 鑷畾涔?Codex home 鐩綍 |
|
|
128
|
+
| `OPENCODE_SKIP_PLUGIN_INSTALL` | 璺宠繃 OpenCode npm 鎻掍欢瀹夎 |
|
|
129
|
+
| `OPENCODE_NPM_REGISTRY` | OpenCode 鎻掍欢 npm 瀹夎婧?|
|
|
154
130
|
|
|
155
|
-
|
|
131
|
+
Secret key 涓嶄細鍦ㄤ氦浜掑紡鐣岄潰涓槑鏂囧睍绀恒€?
|
|
132
|
+
## 甯哥敤缁存姢鍛戒护
|
|
156
133
|
|
|
157
134
|
```bash
|
|
158
135
|
npm run check
|
|
159
136
|
npm pack --dry-run
|
|
160
137
|
```
|
|
161
138
|
|
|
162
|
-
|
|
139
|
+
鍙戝竷鍖呮毚闇蹭袱涓懡浠わ細
|
|
163
140
|
|
|
164
141
|
- `oh-langfuse`
|
|
165
|
-
- `code-tool-langfuse
|
|
142
|
+
- `code-tool-langfuse`锛屽吋瀹规棫鍏ュ彛
|
package/bin/cli.js
CHANGED
|
@@ -244,7 +244,7 @@ async function ensureEnvironment(rl, target, options) {
|
|
|
244
244
|
"Action Needed",
|
|
245
245
|
missing.map(([name, hint]) => `${paint(name, t.red)} ${paint(hint, t.muted)}`)
|
|
246
246
|
);
|
|
247
|
-
console.log("");
|
|
247
|
+
console.log("");
|
|
248
248
|
await rl.question(`${paint("Press Enter after installing the missing dependency, or Ctrl+C to exit.", t.gold)} `);
|
|
249
249
|
return false;
|
|
250
250
|
}
|
|
@@ -280,7 +280,7 @@ function labelValue(label, value, valueStyle = t.reset) {
|
|
|
280
280
|
return `${paint(label.padEnd(22), t.muted)} ${paint(value, valueStyle)}`;
|
|
281
281
|
}
|
|
282
282
|
|
|
283
|
-
function runNodeScript(name, args = [], { dryRun = false } = {}) {
|
|
283
|
+
function runNodeScript(name, args = [], { dryRun = false, quiet = false } = {}) {
|
|
284
284
|
const target = scriptPath(name);
|
|
285
285
|
const cmd = `${process.execPath} ${target} ${args.join(" ")}`.trim();
|
|
286
286
|
if (dryRun) {
|
|
@@ -288,8 +288,8 @@ function runNodeScript(name, args = [], { dryRun = false } = {}) {
|
|
|
288
288
|
return 0;
|
|
289
289
|
}
|
|
290
290
|
console.log("");
|
|
291
|
-
console.log(paint("Running installer...", t.bold, t.teal));
|
|
292
|
-
console.log(paint("─".repeat(Math.min(terminalWidth(), 64)), t.panel));
|
|
291
|
+
if (!quiet) console.log(paint("Running installer...", t.bold, t.teal));
|
|
292
|
+
if (!quiet) console.log(paint("─".repeat(Math.min(terminalWidth(), 64)), t.panel));
|
|
293
293
|
const r = spawnSync(process.execPath, [target, ...args], { stdio: "inherit" });
|
|
294
294
|
return r.status != null ? r.status : r.error ? 1 : 0;
|
|
295
295
|
}
|
|
@@ -777,7 +777,7 @@ async function setupLangfuseMenu(rl, options) {
|
|
|
777
777
|
}
|
|
778
778
|
|
|
779
779
|
function printHelp() {
|
|
780
|
-
renderBrand({ dryRun: false });
|
|
780
|
+
renderBrand({ dryRun: false });
|
|
781
781
|
console.log("");
|
|
782
782
|
renderSection("Usage", [
|
|
783
783
|
"npx oh-langfuse@latest",
|
|
@@ -812,7 +812,14 @@ function printHelp() {
|
|
|
812
812
|
`${paint("--help", t.gold)} Show this help.`
|
|
813
813
|
]);
|
|
814
814
|
}
|
|
815
|
-
|
|
815
|
+
|
|
816
|
+
function commandNeedsPrompt(cmd, target) {
|
|
817
|
+
if (!cmd) return true;
|
|
818
|
+
if (cmd === "setup") return true;
|
|
819
|
+
if (cmd === "check" && !["claude", "opencode", "codex", "environment"].includes(target || "")) return true;
|
|
820
|
+
return false;
|
|
821
|
+
}
|
|
822
|
+
|
|
816
823
|
async function main() {
|
|
817
824
|
const args = parseArgs(process.argv.slice(2));
|
|
818
825
|
const options = {
|
|
@@ -840,10 +847,10 @@ async function main() {
|
|
|
840
847
|
|
|
841
848
|
if (!cmd) return await interactiveMain(options);
|
|
842
849
|
|
|
843
|
-
const rl = createPromptInterface({ input: process.stdin, output: process.stdout });
|
|
844
|
-
try {
|
|
845
|
-
if (cmd === "setup" && target === "claude") return await setupClaude(rl, options);
|
|
846
|
-
if (cmd === "setup" && target === "opencode") return await setupOpenCode(rl, options);
|
|
850
|
+
const rl = commandNeedsPrompt(cmd, target) ? createPromptInterface({ input: process.stdin, output: process.stdout }) : null;
|
|
851
|
+
try {
|
|
852
|
+
if (cmd === "setup" && target === "claude") return await setupClaude(rl, options);
|
|
853
|
+
if (cmd === "setup" && target === "opencode") return await setupOpenCode(rl, options);
|
|
847
854
|
if (cmd === "setup" && target === "codex") return await setupCodex(rl, options);
|
|
848
855
|
if (cmd === "setup") return await setupLangfuseMenu(rl, options);
|
|
849
856
|
if (cmd === "update") {
|
|
@@ -864,7 +871,7 @@ async function main() {
|
|
|
864
871
|
...(hasValue(options.pipIndexUrl) ? [`--pipIndexUrl=${options.pipIndexUrl}`] : []),
|
|
865
872
|
...(options.skipCheck ? ["--skip-check"] : []),
|
|
866
873
|
...(options.yes ? ["--yes"] : []),
|
|
867
|
-
], options);
|
|
874
|
+
], { ...options, quiet: true });
|
|
868
875
|
}
|
|
869
876
|
if (cmd === "check" && target === "claude") return checkClaude(options);
|
|
870
877
|
if (cmd === "check" && target === "opencode") return checkOpenCode(options);
|
|
@@ -876,9 +883,9 @@ async function main() {
|
|
|
876
883
|
return 0;
|
|
877
884
|
}
|
|
878
885
|
if (cmd === "check") return await checkMenu(rl, options);
|
|
879
|
-
} finally {
|
|
880
|
-
rl
|
|
881
|
-
}
|
|
886
|
+
} finally {
|
|
887
|
+
rl?.close();
|
|
888
|
+
}
|
|
882
889
|
|
|
883
890
|
printHelp();
|
|
884
891
|
return 1;
|
package/codex_langfuse_notify.py
CHANGED
|
@@ -13,12 +13,40 @@ import re
|
|
|
13
13
|
import sys
|
|
14
14
|
import time
|
|
15
15
|
import hashlib
|
|
16
|
-
from dataclasses import dataclass
|
|
17
|
-
from datetime import datetime, timezone
|
|
18
|
-
from pathlib import Path
|
|
19
|
-
from typing import Any, Dict, List, Optional, Tuple
|
|
20
|
-
|
|
21
|
-
|
|
16
|
+
from dataclasses import dataclass
|
|
17
|
+
from datetime import datetime, timezone
|
|
18
|
+
from pathlib import Path
|
|
19
|
+
from typing import Any, Dict, List, Optional, Tuple
|
|
20
|
+
from urllib.parse import urlparse
|
|
21
|
+
|
|
22
|
+
|
|
23
|
+
def configure_langfuse_no_proxy() -> None:
|
|
24
|
+
hosts = ["localhost", "127.0.0.1"]
|
|
25
|
+
for key in ("LANGFUSE_HOST", "LANGFUSE_BASEURL", "CODEX_LANGFUSE_BASE_URL"):
|
|
26
|
+
value = os.environ.get(key)
|
|
27
|
+
if not value:
|
|
28
|
+
continue
|
|
29
|
+
parsed = urlparse(value if "://" in value else f"http://{value}")
|
|
30
|
+
if parsed.hostname:
|
|
31
|
+
hosts.append(parsed.hostname)
|
|
32
|
+
if parsed.netloc:
|
|
33
|
+
hosts.append(parsed.netloc)
|
|
34
|
+
existing = []
|
|
35
|
+
for key in ("NO_PROXY", "no_proxy"):
|
|
36
|
+
existing.extend([item.strip() for item in os.environ.get(key, "").split(",") if item.strip()])
|
|
37
|
+
merged = []
|
|
38
|
+
for item in [*existing, *hosts]:
|
|
39
|
+
if item and item not in merged:
|
|
40
|
+
merged.append(item)
|
|
41
|
+
if merged:
|
|
42
|
+
value = ",".join(merged)
|
|
43
|
+
os.environ["NO_PROXY"] = value
|
|
44
|
+
os.environ["no_proxy"] = value
|
|
45
|
+
|
|
46
|
+
|
|
47
|
+
configure_langfuse_no_proxy()
|
|
48
|
+
|
|
49
|
+
try:
|
|
22
50
|
from langfuse import Langfuse, propagate_attributes
|
|
23
51
|
except Exception:
|
|
24
52
|
sys.exit(0)
|
|
@@ -33,7 +61,8 @@ LOG_FILE = STATE_DIR / "codex_langfuse_notify.log"
|
|
|
33
61
|
|
|
34
62
|
DEBUG = os.environ.get("CODEX_LANGFUSE_DEBUG", "").lower() == "true"
|
|
35
63
|
MAX_CHARS = int(os.environ.get("CODEX_LANGFUSE_MAX_CHARS", "20000"))
|
|
36
|
-
METRICS_SCHEMA_VERSION = "1.
|
|
64
|
+
METRICS_SCHEMA_VERSION = "1.1"
|
|
65
|
+
AGENT_NAME = "codex"
|
|
37
66
|
|
|
38
67
|
|
|
39
68
|
def log(level: str, message: str) -> None:
|
|
@@ -388,30 +417,43 @@ def build_interaction_metadata(
|
|
|
388
417
|
model: Optional[str],
|
|
389
418
|
user_message_count: int = 1,
|
|
390
419
|
assistant_message_count: int = 1,
|
|
420
|
+
skill_use_events: Optional[List[Dict[str, Any]]] = None,
|
|
391
421
|
) -> Dict[str, Any]:
|
|
392
422
|
tokens = normalize_token_metrics(token_metrics)
|
|
423
|
+
interaction_id = build_interaction_id(source, session_id, turn_number)
|
|
424
|
+
events = list(skill_use_events or [])
|
|
425
|
+
skill_names_all = [str(event.get("skill_name") or "") for event in events if event.get("skill_name")]
|
|
426
|
+
unique_skill_names = list(dict.fromkeys(skill_names_all))
|
|
427
|
+
effective_skill_count = len(events) if events else int(skill_use_count or 0)
|
|
393
428
|
return {
|
|
394
429
|
"source": source,
|
|
430
|
+
"agent": source,
|
|
395
431
|
"user_id": user_id or "",
|
|
396
432
|
"session_id": session_id,
|
|
397
|
-
"interaction_id":
|
|
433
|
+
"interaction_id": interaction_id,
|
|
398
434
|
"metrics_schema_version": METRICS_SCHEMA_VERSION,
|
|
399
435
|
"interaction_count": 1,
|
|
400
436
|
"user_message_count": user_message_count,
|
|
401
437
|
"assistant_message_count": assistant_message_count,
|
|
402
438
|
"tool_call_count": int(tool_call_count or 0),
|
|
403
439
|
"tool_result_count": int(tool_result_count or 0),
|
|
404
|
-
"skill_use_count":
|
|
440
|
+
"skill_use_count": effective_skill_count,
|
|
441
|
+
"unique_skill_count": len(unique_skill_names),
|
|
442
|
+
"repeated_skill_count": max(0, effective_skill_count - len(unique_skill_names)),
|
|
405
443
|
**tokens,
|
|
406
444
|
"model": model,
|
|
407
445
|
"turn_number": int(turn_number or 0),
|
|
408
446
|
"efficiency": {
|
|
409
447
|
"tokens_per_interaction": tokens.get("total_tokens"),
|
|
410
448
|
"tool_calls_per_interaction": int(tool_call_count or 0),
|
|
411
|
-
"skills_per_interaction":
|
|
449
|
+
"skills_per_interaction": effective_skill_count,
|
|
412
450
|
"output_input_token_ratio": _ratio(tokens.get("output_tokens"), tokens.get("input_tokens")),
|
|
413
451
|
"tokens_per_tool_call": _ratio(tokens.get("total_tokens"), int(tool_call_count or 0)),
|
|
414
452
|
},
|
|
453
|
+
**({
|
|
454
|
+
"skill_names": unique_skill_names,
|
|
455
|
+
"skill_names_all": skill_names_all,
|
|
456
|
+
} if events else {}),
|
|
415
457
|
}
|
|
416
458
|
|
|
417
459
|
|
|
@@ -439,16 +481,33 @@ def _skill_namespace(name: str) -> str:
|
|
|
439
481
|
return name.split(":", 1)[0] if ":" in name else ""
|
|
440
482
|
|
|
441
483
|
|
|
484
|
+
def _skill_event_type(detected_by: str) -> str:
|
|
485
|
+
return "invoked" if detected_by in ("tool_call", "plugin_event", "attribution_skill", "slash_command") else "detected"
|
|
486
|
+
|
|
487
|
+
|
|
488
|
+
def _skill_id_segment(name: str) -> str:
|
|
489
|
+
segment = re.sub(r"[^A-Za-z0-9_.:-]+", "-", str(name or "").strip()).strip("-")
|
|
490
|
+
return (segment or "unknown")[:96]
|
|
491
|
+
|
|
492
|
+
|
|
442
493
|
def detect_skill_usages(tool_calls: List[Dict[str, Any]], known_skills: set) -> List[Dict[str, str]]:
|
|
443
|
-
found: Dict[str, str] =
|
|
494
|
+
found: List[Dict[str, str]] = []
|
|
495
|
+
seen_call_ids: set = set()
|
|
444
496
|
for call in tool_calls or []:
|
|
445
497
|
tool_name = str(call.get("name") or "")
|
|
498
|
+
call_id = str(call.get("id") or call.get("call_id") or call.get("callId") or call.get("tool_call_id") or call.get("toolCallId") or "").strip()
|
|
446
499
|
input_obj = call.get("input") if isinstance(call.get("input"), (dict, list, str)) else {}
|
|
447
500
|
if tool_name.lower() == "skill" and isinstance(input_obj, dict):
|
|
448
501
|
for key in ("skill_name", "skill", "name"):
|
|
449
502
|
value = input_obj.get(key)
|
|
450
503
|
if isinstance(value, str) and value.strip():
|
|
451
|
-
|
|
504
|
+
name = value.strip()
|
|
505
|
+
if call_id:
|
|
506
|
+
dedupe_key = f"call:{call_id}"
|
|
507
|
+
if dedupe_key in seen_call_ids:
|
|
508
|
+
break
|
|
509
|
+
seen_call_ids.add(dedupe_key)
|
|
510
|
+
found.append({"name": name, "skill_namespace": _skill_namespace(name), "detected_by": "tool_call", "skill_call_id": call_id})
|
|
452
511
|
break
|
|
453
512
|
try:
|
|
454
513
|
text = json.dumps(input_obj, ensure_ascii=False)
|
|
@@ -457,11 +516,53 @@ def detect_skill_usages(tool_calls: List[Dict[str, Any]], known_skills: set) ->
|
|
|
457
516
|
for match in re.finditer(r"([A-Za-z]:)?[^\"'\n\r]*[\\/]+([^\\/\"'\n\r]+)[\\/]+SKILL\.md", text, re.IGNORECASE):
|
|
458
517
|
candidate = match.group(2)
|
|
459
518
|
if candidate and (candidate in known_skills or not known_skills):
|
|
460
|
-
found
|
|
461
|
-
return
|
|
462
|
-
|
|
463
|
-
|
|
464
|
-
|
|
519
|
+
found.append({"name": candidate, "skill_namespace": _skill_namespace(candidate), "detected_by": "skill_file_path"})
|
|
520
|
+
return found
|
|
521
|
+
|
|
522
|
+
|
|
523
|
+
def build_skill_use_events(interaction_id: str, skill_usages: List[Dict[str, str]]) -> List[Dict[str, Any]]:
|
|
524
|
+
events: List[Dict[str, Any]] = []
|
|
525
|
+
deduped: List[Dict[str, str]] = []
|
|
526
|
+
seen_call_ids: set = set()
|
|
527
|
+
for skill in skill_usages or []:
|
|
528
|
+
call_id = str(skill.get("skill_call_id") or "").strip()
|
|
529
|
+
if call_id:
|
|
530
|
+
dedupe_key = f"call:{call_id}"
|
|
531
|
+
if dedupe_key in seen_call_ids:
|
|
532
|
+
continue
|
|
533
|
+
seen_call_ids.add(dedupe_key)
|
|
534
|
+
deduped.append(skill)
|
|
535
|
+
total = len(deduped)
|
|
536
|
+
for index, skill in enumerate(deduped, start=1):
|
|
537
|
+
name = str(skill.get("name") or "").strip()
|
|
538
|
+
if not name:
|
|
539
|
+
continue
|
|
540
|
+
detected_by = str(skill.get("detected_by") or "metadata")
|
|
541
|
+
call_id = str(skill.get("skill_call_id") or "").strip()
|
|
542
|
+
events.append({
|
|
543
|
+
"skill_use_id": f"{interaction_id}:skill:{index}:{_skill_id_segment(name)}",
|
|
544
|
+
"skill_use_index": index,
|
|
545
|
+
"skill_use_count_in_interaction": total,
|
|
546
|
+
"skill_event_type": _skill_event_type(detected_by),
|
|
547
|
+
"skill_trigger": "unknown",
|
|
548
|
+
"skill_name": name,
|
|
549
|
+
"skill_use_count": 1,
|
|
550
|
+
"skill_namespace": skill.get("skill_namespace") or _skill_namespace(name),
|
|
551
|
+
"detected_by": detected_by,
|
|
552
|
+
**({"skill_call_id": call_id} if call_id else {}),
|
|
553
|
+
})
|
|
554
|
+
return events
|
|
555
|
+
|
|
556
|
+
|
|
557
|
+
def summarize_skill_usages(skill_usages: List[Dict[str, str]]) -> List[Dict[str, Any]]:
|
|
558
|
+
summary: Dict[str, Dict[str, Any]] = {}
|
|
559
|
+
for item in skill_usages or []:
|
|
560
|
+
name = item.get("name")
|
|
561
|
+
if not name:
|
|
562
|
+
continue
|
|
563
|
+
entry = summary.setdefault(name, {"name": name, "count": 0, "detected_by": item.get("detected_by")})
|
|
564
|
+
entry["count"] += 1
|
|
565
|
+
return list(summary.values())
|
|
465
566
|
|
|
466
567
|
|
|
467
568
|
def get_payload(row: Dict[str, Any]) -> Dict[str, Any]:
|
|
@@ -586,6 +687,8 @@ def emit_codex_turn(
|
|
|
586
687
|
tool_calls = material.get("tool_calls") or []
|
|
587
688
|
tool_results = material.get("tool_results") or []
|
|
588
689
|
skill_usages = detect_skill_usages(tool_calls, discover_known_skills())
|
|
690
|
+
interaction_id = build_interaction_id("codex", session_id, turn_num)
|
|
691
|
+
skill_use_events = build_skill_use_events(interaction_id, skill_usages)
|
|
589
692
|
interaction_meta = build_interaction_metadata(
|
|
590
693
|
"codex",
|
|
591
694
|
user_id,
|
|
@@ -594,28 +697,28 @@ def emit_codex_turn(
|
|
|
594
697
|
usage_details,
|
|
595
698
|
len(tool_calls),
|
|
596
699
|
len(tool_results),
|
|
597
|
-
len(
|
|
700
|
+
len(skill_use_events),
|
|
598
701
|
model,
|
|
599
702
|
user_message_count=1 if material.get("user_text") else 0,
|
|
600
703
|
assistant_message_count=1 if material.get("assistant_text") else 0,
|
|
704
|
+
skill_use_events=skill_use_events,
|
|
601
705
|
)
|
|
602
|
-
skill_summary =
|
|
603
|
-
{"name": item["name"], "count": 1, "detected_by": item["detected_by"]}
|
|
604
|
-
for item in skill_usages
|
|
605
|
-
]
|
|
706
|
+
skill_summary = summarize_skill_usages(skill_usages)
|
|
606
707
|
|
|
607
708
|
with propagate_attributes(
|
|
608
709
|
user_id=user_id,
|
|
609
710
|
session_id=session_id,
|
|
610
|
-
trace_name=
|
|
611
|
-
tags=[
|
|
711
|
+
trace_name="Agent Turn",
|
|
712
|
+
tags=[AGENT_NAME],
|
|
612
713
|
):
|
|
613
|
-
with langfuse.start_as_current_observation(
|
|
614
|
-
name=
|
|
714
|
+
with langfuse.start_as_current_observation(
|
|
715
|
+
name="Agent Turn",
|
|
615
716
|
input={"role": "user", "content": user_text},
|
|
717
|
+
output={"role": "assistant", "content": assistant_text},
|
|
616
718
|
metadata={
|
|
617
719
|
**interaction_meta,
|
|
618
|
-
"source":
|
|
720
|
+
"source": AGENT_NAME,
|
|
721
|
+
"agent": AGENT_NAME,
|
|
619
722
|
"session_id": session_id,
|
|
620
723
|
"turn_number": turn_num,
|
|
621
724
|
"session_path": str(session_path),
|
|
@@ -628,15 +731,7 @@ def emit_codex_turn(
|
|
|
628
731
|
},
|
|
629
732
|
) as trace_span:
|
|
630
733
|
with langfuse.start_as_current_observation(
|
|
631
|
-
name="
|
|
632
|
-
input={"role": "user", "content": user_text},
|
|
633
|
-
output={"role": "assistant", "content": assistant_text},
|
|
634
|
-
metadata=interaction_meta,
|
|
635
|
-
):
|
|
636
|
-
pass
|
|
637
|
-
|
|
638
|
-
with langfuse.start_as_current_observation(
|
|
639
|
-
name="Codex Response",
|
|
734
|
+
name="Agent Response",
|
|
640
735
|
as_type="generation",
|
|
641
736
|
model=model,
|
|
642
737
|
input={"role": "user", "content": user_text},
|
|
@@ -644,7 +739,8 @@ def emit_codex_turn(
|
|
|
644
739
|
usage_details=usage_details or None,
|
|
645
740
|
metadata={
|
|
646
741
|
"assistant_text": assistant_meta,
|
|
647
|
-
"source":
|
|
742
|
+
"source": AGENT_NAME,
|
|
743
|
+
"agent": AGENT_NAME,
|
|
648
744
|
"user_id": user_id or "",
|
|
649
745
|
"session_id": session_id,
|
|
650
746
|
"interaction_id": interaction_meta["interaction_id"],
|
|
@@ -653,32 +749,15 @@ def emit_codex_turn(
|
|
|
653
749
|
):
|
|
654
750
|
pass
|
|
655
751
|
|
|
656
|
-
for skill in skill_usages:
|
|
657
|
-
with langfuse.start_as_current_observation(
|
|
658
|
-
name="Skill Use",
|
|
659
|
-
metadata={
|
|
660
|
-
"source": "codex",
|
|
661
|
-
"user_id": user_id or "",
|
|
662
|
-
"session_id": session_id,
|
|
663
|
-
"interaction_id": interaction_meta["interaction_id"],
|
|
664
|
-
"skill_name": skill["name"],
|
|
665
|
-
"skill_use_count": 1,
|
|
666
|
-
"skill_namespace": skill["skill_namespace"],
|
|
667
|
-
"detected_by": skill["detected_by"],
|
|
668
|
-
"turn_number": turn_num,
|
|
669
|
-
"metrics_schema_version": METRICS_SCHEMA_VERSION,
|
|
670
|
-
},
|
|
671
|
-
):
|
|
672
|
-
pass
|
|
673
|
-
|
|
674
752
|
for call in tool_calls:
|
|
675
753
|
tool_input, input_meta = truncate(call.get("input"))
|
|
676
754
|
with langfuse.start_as_current_observation(
|
|
677
|
-
name=
|
|
755
|
+
name="Tool Call",
|
|
678
756
|
as_type="tool",
|
|
679
757
|
input=tool_input,
|
|
680
758
|
metadata={
|
|
681
|
-
"source":
|
|
759
|
+
"source": AGENT_NAME,
|
|
760
|
+
"agent": AGENT_NAME,
|
|
682
761
|
"user_id": user_id or "",
|
|
683
762
|
"session_id": session_id,
|
|
684
763
|
"interaction_id": interaction_meta["interaction_id"],
|
|
@@ -694,10 +773,11 @@ def emit_codex_turn(
|
|
|
694
773
|
for result in tool_results:
|
|
695
774
|
output, output_meta = truncate(result.get("output"))
|
|
696
775
|
with langfuse.start_as_current_observation(
|
|
697
|
-
name=
|
|
776
|
+
name="Tool Result",
|
|
698
777
|
as_type="tool",
|
|
699
778
|
metadata={
|
|
700
|
-
"source":
|
|
779
|
+
"source": AGENT_NAME,
|
|
780
|
+
"agent": AGENT_NAME,
|
|
701
781
|
"user_id": user_id or "",
|
|
702
782
|
"session_id": session_id,
|
|
703
783
|
"interaction_id": interaction_meta["interaction_id"],
|