pi-dsh-mimic 0.1.0 → 0.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +21 -18
- package/README.zh-CN.md +17 -15
- package/docs/advanced.md +9 -2
- package/docs/advanced.zh-CN.md +7 -2
- package/docs/project2-evidence.md +10 -4
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -33,6 +33,14 @@ the task, does not inject `We need`, and does not proxy the API.
|
|
|
33
33
|
|
|
34
34
|
## Project2 results
|
|
35
35
|
|
|
36
|
+
Project2 V4.1b is a personal, self-hosted long-horizon repository-maintenance
|
|
37
|
+
evaluation. The model repairs a multi-module Python backend and ESP32-S3
|
|
38
|
+
firmware project covering authentication and session privacy, database
|
|
39
|
+
migrations, cross-module features, backward compatibility,
|
|
40
|
+
Wi-Fi/MQTT/NVS/protocol and ESP-IDF contracts, and delivery evidence. It is not
|
|
41
|
+
a general benchmark; these scores compare engineering completion on one frozen
|
|
42
|
+
task.
|
|
43
|
+
|
|
36
44
|
Default Pi scored 92. Four final runs of the same one-shot request flow scored
|
|
37
45
|
**98, 96, 96, and 98**, all with F3 at 16/16:
|
|
38
46
|
|
|
@@ -42,7 +50,7 @@ Default Pi scored 92. Four final runs of the same one-shot request flow scored
|
|
|
42
50
|
| Early Minimal-mimic prototype | Official DeepSeek API | **98** | F3 16, F6 10, F8 7 |
|
|
43
51
|
| Same-flow replication | Official DeepSeek API | 96 | F3 16, F6 10, F8 5 |
|
|
44
52
|
| First packaged implementation | Official DeepSeek API | 96 | F3 16, F6 8, F8 7 |
|
|
45
|
-
| Current independent implementation
|
|
53
|
+
| Current independent implementation | OpenCode Go | **98** | F3 16, F6 10, F8 7 |
|
|
46
54
|
|
|
47
55
|
The results show that a DSH Minimal first request can establish a strong V4 Pro
|
|
48
56
|
trajectory inside Pi and that restoring Pi's full tool catalog does not break
|
|
@@ -57,35 +65,30 @@ Requirements: Node.js 22.19 or later, Pi 0.84.2 or later, and either official
|
|
|
57
65
|
DeepSeek API or OpenCode Go configuration. Pi 0.84.2 includes both provider
|
|
58
66
|
routes.
|
|
59
67
|
|
|
60
|
-
|
|
68
|
+
Install from npm for the current user:
|
|
61
69
|
|
|
62
70
|
```bash
|
|
63
|
-
pi
|
|
64
|
-
--provider deepseek \
|
|
65
|
-
--model deepseek-v4-pro
|
|
71
|
+
pi install npm:pi-dsh-mimic
|
|
66
72
|
```
|
|
67
73
|
|
|
68
|
-
|
|
69
|
-
environment variable:
|
|
74
|
+
Install at project scope:
|
|
70
75
|
|
|
71
76
|
```bash
|
|
72
|
-
pi -
|
|
73
|
-
--provider opencode-go \
|
|
74
|
-
--model deepseek-v4-pro
|
|
77
|
+
pi install -l npm:pi-dsh-mimic
|
|
75
78
|
```
|
|
76
79
|
|
|
77
|
-
|
|
80
|
+
Try the current checkout directly:
|
|
78
81
|
|
|
79
82
|
```bash
|
|
80
|
-
pi
|
|
83
|
+
pi -e ./packages/pi-dsh-mimic \
|
|
84
|
+
--provider deepseek \
|
|
85
|
+
--model deepseek-v4-pro
|
|
81
86
|
```
|
|
82
87
|
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
[Pi Package Catalog](https://pi.dev/packages)
|
|
87
|
-
after publication to npm. The published install command will be
|
|
88
|
-
`pi install npm:pi-dsh-mimic`.
|
|
88
|
+
For OpenCode Go, configure Pi's expected `OPENCODE_API_KEY` environment variable
|
|
89
|
+
and select the `opencode-go` provider. The package is published to npm with the
|
|
90
|
+
`pi-package` keyword required by the
|
|
91
|
+
[Pi Package Catalog](https://pi.dev/packages).
|
|
89
92
|
|
|
90
93
|
Select V4 Pro before starting a new session. Switching to V4 Pro inside an
|
|
91
94
|
existing conversation does not forge a new bootstrap.
|
package/README.zh-CN.md
CHANGED
|
@@ -28,6 +28,11 @@ DeepSeek Harness Minimal 环境，让 DeepSeek V4 Pro 从已验证的高能力
|
|
|
28
28
|
|
|
29
29
|
## Project2 结果
|
|
30
30
|
|
|
31
|
+
Project2 V4.1b 是一个个人、自托管的长程代码维护评测。模型需要修复一个多模块 Python
|
|
32
|
+
后端与 ESP32-S3 固件仓库，覆盖鉴权与 session 隐私、数据库迁移、跨模块功能、兼容性、
|
|
33
|
+
Wi-Fi/MQTT/NVS/协议与 ESP-IDF 契约，以及交付说明。它不是通用 benchmark；这里的分数
|
|
34
|
+
只用于比较同一冻结任务下的工程完成度。
|
|
35
|
+
|
|
31
36
|
默认 Pi 得到 92。相同 one-shot 请求流程的四次最终结果为 **98、96、96、98**，均保持
|
|
32
37
|
F3 16/16：
|
|
33
38
|
|
|
@@ -37,7 +42,7 @@ F3 16/16：
|
|
|
37
42
|
| 早期 Minimal 模拟原型 | DeepSeek 官方 API | **98** | F3 16，F6 10，F8 7 |
|
|
38
43
|
| 同流程复现实验 | DeepSeek 官方 API | 96 | F3 16，F6 10，F8 5 |
|
|
39
44
|
| 首次 package 化实现 | DeepSeek 官方 API | 96 | F3 16，F6 8，F8 7 |
|
|
40
|
-
| 当前独立实现
|
|
45
|
+
| 当前独立实现 | OpenCode Go | **98** | F3 16，F6 10，F8 7 |
|
|
41
46
|
|
|
42
47
|
这些结果表明：DSH Minimal 首请求可以在 Pi 中稳定建立高能力轨迹，而恢复 Pi 完整工具
|
|
43
48
|
目录不会破坏它。实验过程、被否决的设计和每项实现选择的依据见
|
|
@@ -49,32 +54,29 @@ F3 16/16：
|
|
|
49
54
|
要求：Node.js 22.19 或更新版本、Pi 0.84.2 或更新版本，并已配置 DeepSeek 官方 API 或
|
|
50
55
|
OpenCode Go。Pi 0.84.2 已内置两条 provider 路径。
|
|
51
56
|
|
|
52
|
-
|
|
57
|
+
从 npm 安装到当前用户：
|
|
53
58
|
|
|
54
59
|
```bash
|
|
55
|
-
pi
|
|
56
|
-
--provider deepseek \
|
|
57
|
-
--model deepseek-v4-pro
|
|
60
|
+
pi install npm:pi-dsh-mimic
|
|
58
61
|
```
|
|
59
62
|
|
|
60
|
-
|
|
63
|
+
项目级安装：
|
|
61
64
|
|
|
62
65
|
```bash
|
|
63
|
-
pi -
|
|
64
|
-
--provider opencode-go \
|
|
65
|
-
--model deepseek-v4-pro
|
|
66
|
+
pi install -l npm:pi-dsh-mimic
|
|
66
67
|
```
|
|
67
68
|
|
|
68
|
-
|
|
69
|
+
直接试用当前 checkout：
|
|
69
70
|
|
|
70
71
|
```bash
|
|
71
|
-
pi
|
|
72
|
+
pi -e ./packages/pi-dsh-mimic \
|
|
73
|
+
--provider deepseek \
|
|
74
|
+
--model deepseek-v4-pro
|
|
72
75
|
```
|
|
73
76
|
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
[Pi Package Catalog](https://pi.dev/packages)
|
|
77
|
-
`pi install npm:pi-dsh-mimic`。当前 checkout 不会被市场索引。
|
|
77
|
+
使用 OpenCode Go 时，把 key 配置在 Pi 期望的 `OPENCODE_API_KEY` 环境变量中，并把
|
|
78
|
+
provider 改为 `opencode-go`。Package 已发布到 npm，并带有 Pi gallery 要求的
|
|
79
|
+
`pi-package` keyword，可由 [Pi Package Catalog](https://pi.dev/packages) 索引。
|
|
78
80
|
|
|
79
81
|
先选择 V4 Pro，再新建 session。在已有对话中途切换到 V4 Pro 不会伪造新的 bootstrap。
|
|
80
82
|
|
package/docs/advanced.md
CHANGED
|
@@ -16,7 +16,14 @@ runtime.
|
|
|
16
16
|
|
|
17
17
|
## Experimental basis
|
|
18
18
|
|
|
19
|
-
|
|
19
|
+
Project2 V4.1b is a personal, self-hosted long-horizon repository-maintenance
|
|
20
|
+
evaluation. The model repairs a deliberately broken multi-module Python backend
|
|
21
|
+
and ESP32-S3 firmware project covering authentication and session privacy,
|
|
22
|
+
database migrations, cross-module features, backward compatibility,
|
|
23
|
+
Wi-Fi/MQTT/NVS/protocol and ESP-IDF contracts, and final delivery evidence. It
|
|
24
|
+
is not a general benchmark.
|
|
25
|
+
|
|
26
|
+
Local runs use this Project2 V4.1b task at frozen commit
|
|
20
27
|
`04255b55f16c4439e538239fb9783070c4165081`. When a run resumed after an
|
|
21
28
|
interruption, only its final submission score entered the comparison. A wrong
|
|
22
29
|
project root, a zero-token configuration failure, and duplicate evaluation do
|
|
@@ -78,7 +85,7 @@ implementations:
|
|
|
78
85
|
| Implementation | Provider | Score | F3 | F6 | F8 |
|
|
79
86
|
| --- | --- | ---: | ---: | ---: | ---: |
|
|
80
87
|
| First packaged implementation | Official DeepSeek API | 96 | 16 | 8 | 7 |
|
|
81
|
-
| Current independent implementation
|
|
88
|
+
| Current independent implementation | OpenCode Go | **98** | 16 | 10 | 7 |
|
|
82
89
|
|
|
83
90
|
The initial OpenCode Go request hit an account opt-in 403 before producing a
|
|
84
91
|
token. The same session resumed after opt-in, stopped naturally, and scored 98.
|
package/docs/advanced.zh-CN.md
CHANGED
|
@@ -13,7 +13,12 @@ session、工具执行、文件编辑和插件组合。用户既能获得 Projec
|
|
|
13
13
|
|
|
14
14
|
## 实验口径
|
|
15
15
|
|
|
16
|
-
|
|
16
|
+
Project2 V4.1b 是一个个人、自托管的长程代码维护评测。模型接手一个故意保留缺陷的
|
|
17
|
+
多模块 Python 后端与 ESP32-S3 固件仓库，完成鉴权与 session 隐私、数据库迁移、跨模块
|
|
18
|
+
功能、兼容性、Wi-Fi/MQTT/NVS/协议与 ESP-IDF 契约，以及最终交付证据。它不是通用
|
|
19
|
+
benchmark。
|
|
20
|
+
|
|
21
|
+
本地实验使用这套 Project2 V4.1b 任务，冻结于提交
|
|
17
22
|
`04255b55f16c4439e538239fb9783070c4165081`。同一运行中断后续跑时，只记录最终交卷分；
|
|
18
23
|
错误 project root、零 token 配置失败和重复评分不计作新样本。
|
|
19
24
|
|
|
@@ -65,7 +70,7 @@ context 必须显式携带 `session_id`，voice 路径先取得 current session
|
|
|
65
70
|
| 实现 | Provider | 分数 | F3 | F6 | F8 |
|
|
66
71
|
| --- | --- | ---: | ---: | ---: | ---: |
|
|
67
72
|
| 首次 package 化实现 | DeepSeek 官方 API | 96 | 16 | 8 | 7 |
|
|
68
|
-
| 当前独立实现
|
|
73
|
+
| 当前独立实现 | OpenCode Go | **98** | 16 | 10 | 7 |
|
|
69
74
|
|
|
70
75
|
OpenCode Go 的初始请求在生成 token 前遇到账户 opt-in 403。同一 session 完成 opt-in 后
|
|
71
76
|
续跑，模型最终自然 `stop` 并得到 98。这次运行验证了当前实现能够通过 OpenCode Go
|
|
@@ -15,6 +15,11 @@
|
|
|
15
15
|
本文记录支持当前实现的完整结果、纳入与排除规则、外部校准和来源关系。原始 run、
|
|
16
16
|
trajectory、provider capture 与 grader 输出由维护者实验归档保存，不随仓库分发。
|
|
17
17
|
|
|
18
|
+
Project2 V4.1b 是一个个人、自托管的长程代码维护评测，不是跨项目通用 benchmark。
|
|
19
|
+
候选模型需要修复一个多模块 Python 后端与 ESP32-S3 固件仓库，覆盖鉴权与 session 隐私、
|
|
20
|
+
数据库迁移、跨模块功能、兼容性、Wi-Fi/MQTT/NVS/协议与 ESP-IDF 契约，以及交付证据。
|
|
21
|
+
100 分 Ability 分数表示这套冻结任务中的工程完成度。
|
|
22
|
+
|
|
18
23
|
## 计分口径
|
|
19
24
|
|
|
20
25
|
- 同一运行中断后续跑，只保留最终交卷分；续跑前分数属于中间状态。
|
|
@@ -41,7 +46,7 @@ trajectory、provider capture 与 grader 输出由维护者实验归档保存，
|
|
|
41
46
|
| 默认 Pi baseline | 完整 Pi prompt + 5 工具，全程 native | `Let me` | **92** | F3 11，F6 10，F8 6 | 默认对照；ambient-session 少 5 分 |
|
|
42
47
|
| 早期 Minimal 模拟原型 | 原任务 + Minimal + DSH `bash/editor`；随后 Pi 原生 5 工具；不重放 context | `We need` | **98** | F3 16，F6 10，F8 7 | 首次证明 Pi 能进入高能力轨迹 |
|
|
43
48
|
| 首次 package 化实现 | 同一首包；随后 Pi 原生；同 session 续过一次 402 | `We need` | **96** | F3 16，F6 8，F8 7 | 干净 package 复现目标轨迹；迁移少 2 分 |
|
|
44
|
-
| 当前独立实现
|
|
49
|
+
| 当前独立实现 | 初始首包遇 403；opt-in 后同 session 续跑并恢复 Pi 原生 | `We need` | **98** | F3 16，F6 10，F8 7 | OpenCode Go 再次进入同一高分档 |
|
|
45
50
|
| 持续 DSH wire | 同一首包；后续请求继续 DSH 规范化 | `We need` | **94** | F3 11，F6 10，F8 8 | 指纹保持，关键安全判断没有保持 |
|
|
46
51
|
| Pi context 重放 | 同一首包；随后 Pi 原生，但把完整 system context 作为额外 user message | `We need` | **90** | F3 11，F6 8，F8 6 | 额外 context 增加干扰，迁移退化 |
|
|
47
52
|
| 同流程复现实验 | 同一首包；随后 Pi 原生；不重放 context | `We need` | **96** | F3 16，F6 10，F8 5 | 高分区间复现；ESP 静态完成度较低 |
|
|
@@ -103,7 +108,7 @@ blocks 只计 assistant content 中的 `thinking` block。`We need` 与 `Let me`
|
|
|
103
108
|
| 默认 Pi baseline | 92 | 113 | 78 | 0 | 115 | 2 / 156 |
|
|
104
109
|
| 早期 Minimal 模拟原型 | 98 | 149 | 135 | 8 | 1 | 10 / 1 |
|
|
105
110
|
| 首次 package 化实现 | 96 | 147 | 147 | 11 | 0 | 11 / 0 |
|
|
106
|
-
| 当前独立实现
|
|
111
|
+
| 当前独立实现 | 98 | 160 | 156 | 7 | 0 | 7 / 0 |
|
|
107
112
|
| 持续 DSH wire | 94 | 196 | 190 | 6 | 2 | 7 / 2 |
|
|
108
113
|
| Pi context 重放 | 90 | 189 | 184 | 6 | 0 | 8 / 0 |
|
|
109
114
|
| 同流程复现实验 | 96 | 160 | 155 | 10 | 0 | 12 / 0 |
|
|
@@ -120,7 +125,7 @@ blocks 只计 assistant content 中的 `thinking` block。`We need` 与 `Let me`
|
|
|
120
125
|
| 默认 Pi baseline | 157 | `bash` 79，`edit` 32，`read` 30，`write` 16 | `toolUse` 112，`stop` 1，`error` 1 |
|
|
121
126
|
| 早期 Minimal 模拟原型 | 148 | `bash` 83，`edit` 49，`write` 16 | `toolUse` 148，`stop` 1 |
|
|
122
127
|
| 首次 package 化实现 | 189 | `bash` 82，`read` 48，`edit` 42，`write` 17 | `toolUse` 146，`stop` 1，`error` 2 |
|
|
123
|
-
| 当前独立实现
|
|
128
|
+
| 当前独立实现 | 193 | `bash` 98，`read` 49，`edit` 31，`write` 15 | `toolUse` 159，`stop` 1，`error` 1 |
|
|
124
129
|
| 持续 DSH wire | 225 | `bash` 94，`edit` 65，`read` 53，`write` 13 | `toolUse` 195，`stop` 1 |
|
|
125
130
|
| Pi context 重放 | 220 | `bash` 76，`edit` 66，`read` 63，`write` 15 | `toolUse` 188，`stop` 1 |
|
|
126
131
|
| 同流程复现实验 | 176 | `bash` 176 | `toolUse` 159，`stop` 1，`error` 1 |
|
|
@@ -213,7 +218,8 @@ evaluator。
|
|
|
213
218
|
|
|
214
219
|
## 当前决定
|
|
215
220
|
|
|
216
|
-
- 产品名为 `pi-dsh-mimic
|
|
221
|
+
- 产品名为 `pi-dsh-mimic`。`0.1.0` 是首次公开实现；`0.1.1` 只补充 Project2 语境和
|
|
222
|
+
已发布安装说明，运行时代码不变。
|
|
217
223
|
- 默认流程固定为 task-bearing one-shot：请求 #1 模拟 DSH Minimal，请求 #2 恢复 Pi。
|
|
218
224
|
- 同时支持 Pi 的 `deepseek` 与 `opencode-go` provider。
|
|
219
225
|
- Minimal persona 保持；完整 Pi system context 不重放。
|