dsh-eval-guard 0.7.3 → 0.7.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/ISSUES.md +5 -1
- package/README.md +4 -2
- package/assets/neural-pendulum/SPEC.md +6 -3
- package/assets/neural-pendulum/TASK.md +7 -4
- package/assets/neural-pendulum/sim.cpp +38 -5
- package/assets/neural-pendulum/sim_agent +0 -0
- package/assets/neural-pendulum/sim_eval +0 -0
- package/assets/neural-pendulum/workspace.zip +0 -0
- package/lib/index.js +5 -1
- package/native/dsh-eval-supervisor +0 -0
- package/native/supervisor.c +42 -10
- package/neural-pendulum-eval/README.md +4 -2
- package/neural-pendulum-eval/run_eval.py +3 -1
- package/neural-pendulum-eval/sim_eval +0 -0
- package/package.json +2 -2
- package/eval/__pycache__/bomberman.cpython-310.pyc +0 -0
- package/eval/__pycache__/commit0.cpython-310.pyc +0 -0
- package/eval/__pycache__/common.cpython-310.pyc +0 -0
- package/eval/__pycache__/ctf.cpython-310.pyc +0 -0
- package/eval/__pycache__/featurebench.cpython-310.pyc +0 -0
- package/eval/__pycache__/land.cpython-310.pyc +0 -0
- package/eval/__pycache__/neural-cifar.cpython-310.pyc +0 -0
- package/eval/__pycache__/neural-cpp.cpython-310.pyc +0 -0
- package/eval/__pycache__/neural-fp32.cpython-310.pyc +0 -0
- package/eval/__pycache__/neural-pendulum.cpython-310.pyc +0 -0
- package/eval/__pycache__/swe-bench-pro.cpython-310.pyc +0 -0
- package/eval/__pycache__/tank.cpython-310.pyc +0 -0
- package/eval/__pycache__/warehouse.cpython-310.pyc +0 -0
- package/eval/legacy/__pycache__/bomberman.cpython-310.pyc +0 -0
- package/eval/legacy/__pycache__/commit0.cpython-310.pyc +0 -0
- package/eval/legacy/__pycache__/ctf.cpython-310.pyc +0 -0
- package/eval/legacy/__pycache__/featurebench.cpython-310.pyc +0 -0
- package/eval/legacy/__pycache__/land.cpython-310.pyc +0 -0
- package/eval/legacy/__pycache__/swe_bench_pro.cpython-310.pyc +0 -0
- package/eval/legacy/__pycache__/tank.cpython-310.pyc +0 -0
- package/eval/legacy/__pycache__/warehouse.cpython-310.pyc +0 -0
- package/neural-cifar-eval/__pycache__/cifar_cache.cpython-310.pyc +0 -0
- package/neural-cifar-eval/__pycache__/common.cpython-310.pyc +0 -0
- package/neural-cifar-eval/__pycache__/fetch_cifar.cpython-310.pyc +0 -0
- package/neural-cifar-eval/__pycache__/ops_eval.cpython-310.pyc +0 -0
- package/neural-cifar-eval/__pycache__/run_eval.cpython-310.pyc +0 -0
- package/neural-pendulum-eval/__pycache__/ops_eval.cpython-310.pyc +0 -0
- package/neural-pendulum-eval/__pycache__/run_eval.cpython-310.pyc +0 -0
package/ISSUES.md
CHANGED
|
@@ -1,10 +1,14 @@
|
|
|
1
1
|
# dsh-eval-guard 问题清单与修复记录
|
|
2
2
|
|
|
3
|
-
> 版本:0.7.
|
|
3
|
+
> 版本:0.7.5 · 审计基线:本仓库工作树(含 0.7.5 修复与 Neural-Pendulum 仿真器分离)
|
|
4
4
|
> 对照 DSH:0.2.1-alpha.1 源码(并核对 git tag 0.1.2 / 0.1.5 / 0.1.7 / 0.2.0-rc.1 / rc.2 / 0.2.1-alpha.1)
|
|
5
5
|
> 本文件先记录问题,再记录修复状态。状态:✅ 已修复 / 🟡 部分修复 / ⛔ 保留(有明确原因或需要架构级改动)
|
|
6
6
|
>
|
|
7
7
|
> **2026-10-22 更新**:本轮只修复“守护本体 + Neural-Pendulum”。已修复:fork/子代理会话识别与约束、子代理路径审计、sandbox 继承环境清洗、PTC 控制通道标记回填、allow 规则宿主绑定安全、写路径规则、设置页任务默认值原子保存、Job 异步失败清理与重启 PID 校验、Pendulum 按请求时刻计分与 done final_score。
|
|
8
|
+
>
|
|
9
|
+
> **2026-10-23 会话回归修复(未发版)**:修复 Neural-Pendulum 仿真器 Coriolis 项多乘 cos 导致高转速下能量注入并产生 NaN;修正关节 3 在绝对角坐标下的广义力矩映射,由 (tau1,0,tau3) 改为 (tau1,-tau3,tau3);TASK/SPEC/prompt 明确 theta 约定并说明工作区无 PyTorch/numpy;网络 syscall 改为 AF_INET/AF_INET6 的 EPERM 软拒绝,避免 pip/NSS 本地检查被误杀。
|
|
10
|
+
|
|
11
|
+
> **0.7.5**:把 0.7.4 误带的开发用 RL 可行性代码从插件中撤除,只保留修正后的 simulator;同时把 agent 训练仿真器与评测仿真器拆开:`assets/neural-pendulum/sim_agent` 只做随机近向下 reset,评测用 `neural-pendulum-eval/sim_eval` 只做固定初态 reset,两者不共享 reset 分支。工作区仍只交付二进制,不提供仿真源码。
|
|
8
12
|
> **下文仍有若干条目标记为 ✅ 但实际未完成,不要再按旧状态理解**:arena legacy 的 `std::thread` 静态规则未移除;`cifar_cache.py` 解压仍无 filter、缓存路径仍用 `Path.home()/.dsh`;CIFAR 隐藏标签仍以文件形式落盘;FP32“30 分钟整轮”仍是 op 间检查;无 baseline 时评测仍 fail-open。其余任务相关修复按用户要求暂不动。
|
|
9
13
|
|
|
10
14
|
## 0. 修复状态总览
|
package/README.md
CHANGED
|
@@ -3,7 +3,7 @@
|
|
|
3
3
|
在 DSH 中启用后,约束**所有用户主对话**:
|
|
4
4
|
|
|
5
5
|
- 文件类工具(read/write/edit/grep/glob/ops_*/drawio_* 等)做参数路径审计:只允许修改会话工作区;工作区之外允许读取“用户不可写但可读”的文件(系统文件、只读资料),用户可写的数据目录仍不可读写。进程沙箱内 `/tmp` 映射到工作区 `.tmp`;文件工具不会重写路径,读写 `/tmp` 请直接使用工作区内的 `.tmp` 路径。
|
|
6
|
-
- 进程类工具(bash / run_code / 后台任务)由 bwrap + seccomp supervisor 二次约束:只读系统视图、工作区可写、私有 /tmp、无网络;进入沙箱前清空继承环境,只回填 PATH/HOME/TMPDIR/XDG/LANG 等非敏感变量,避免宿主 API key /
|
|
6
|
+
- 进程类工具(bash / run_code / 后台任务)由 bwrap + seccomp supervisor 二次约束:只读系统视图、工作区可写、私有 /tmp、无网络;进入沙箱前清空继承环境,只回填 PATH/HOME/TMPDIR/XDG/LANG 等非敏感变量,避免宿主 API key / 代理泄漏。文件越界访问会记录并终止进程组;网络 `connect/bind/sendto` 对 AF_INET/AF_INET6 返回 EPERM 并记录,但不杀进程组(`socket()` 创建允许,避免 pip/NSS 等本地检查被误杀),真正的网络边界是 bwrap 的私有 network namespace。
|
|
7
7
|
- 禁止联网、子代理、Agent Team/跨 agent 通信,以及可绕过单次评测的持久化工具(goal / schedule / drawio / canvas 等)。用户 fork 出的会话仍按主会话约束;子代理会话本身也同样受工具审计和进程沙箱约束(黑名单只是额外的一层)。
|
|
8
8
|
- 工具黑名单可配置;未知工具默认放行。DSH 内置与 dsh-ops 的文件工具会做参数路径审计。
|
|
9
9
|
|
|
@@ -59,6 +59,8 @@ CIFAR-10 下载缓存固定在 DSH home 下(默认 `~/.dsh/eval-guard/cache/ci
|
|
|
59
59
|
|
|
60
60
|
Neural-Pendulum 要求从零实现纯 C++17、仅 CPU、fp32 的神经网络基础设施,并用它训练控制器驱动预编译三节摆从接近全向下起摆、保持全向上 10 秒。后端评测与 Neural-FP32/Neural-CIFAR 使用同一套算子/阈值(单次调用 60 秒)。算子评测全部通过后才驱动 build/control,在固定初始状态上统计连续全向上时间 t_can。评测脚本在独立目录 `dsh-eval-guard/neural-pendulum-eval/`;宿主侧提交与测试工具为 eval_submit / eval_test,工具的注册依赖会话级 run 绑定。
|
|
61
61
|
|
|
62
|
+
**仿真器分离**:工作区 `tools/sim` 是只接受随机近向下 reset 的**训练/自测**仿真器,不包含固定评测初态;评测使用独立 `neural-pendulum-eval/sim_eval`,它只做固定初态 reset。工作区不提供仿真器源码。
|
|
63
|
+
|
|
62
64
|
**分数**:eval_test 返回 stage / t_can / completed / error;时间积分 `final_score` 写入宿主侧事件与完成标记(`~/.dsh/eval-guard/neural-pendulum/<workspace>/done`),用于赛后排行,不返回给 agent。分数在**测试请求时刻**切换,当前测试结果也计入本次积分;宿主按事件重新计算最终值,而不是使用“评测完成时刻”的旧值。起算时间取首次提交写入的 meta.json。
|
|
63
65
|
|
|
64
66
|
评测时在临时目录独立构建,然后从 assets/neural-cpp/hidden/ 注入 C++ 测试源码;隐藏测试在候选构建**开始前**读入内存,构建完成后再落盘编译,测试数据由固定种子在进程内生成,不依赖或复制外部数据文件。
|
|
@@ -174,7 +176,7 @@ DSH_CHECKOUT=/path/to/deepseek-harness scripts/install.sh web
|
|
|
174
176
|
|
|
175
177
|
- 只支持 Linux;启用守护前会探测 bwrap 与 unprivileged user namespace,探测失败时设置页显示原因且不生效。
|
|
176
178
|
- seccomp 用户通知的路径检查与 open 之间存在极小的 TOCTOU 窗口;bwrap 只读/无网络挂载是第二层内核边界。
|
|
177
|
-
- io_uring 被拒绝(以 EPERM 软拒绝,不杀进程);正常 syscall 不记录,进程审计只覆盖被拦截的文件/网络 syscall。
|
|
179
|
+
- io_uring 被拒绝(以 EPERM 软拒绝,不杀进程);正常 syscall 不记录,进程审计只覆盖被拦截的文件/网络 syscall。AF_INET/AF_INET6 的连接类 syscall 也是 EPERM 软拒绝;文件策略违规仍会终止进程组。
|
|
178
180
|
- 工具路径审计只覆盖已知文件工具(DSH 内置、dsh-ops、drawio);未知工具默认放行,这是产品选择,不是完整内核边界。
|
|
179
181
|
- `forbiddenPaths` 的 `!` 例外只放宽审计;它不会把宿主 home、DSH 状态目录、共享 /tmp 或 workspace 的祖先 ro-bind 进沙箱。系统目录本身已只读可见,通常不需要 `!`。
|
|
180
182
|
- 只读文件放行是产品选择:工作区之外“用户不可写但可读”的文件允许读取,因此只读安装的隐藏资产/数据不在 OS 权限层面被隐藏;真正需要隔离的隐藏数据仍需独立用户/容器。
|
|
@@ -25,7 +25,7 @@ agent 必须:
|
|
|
25
25
|
4. 保证基础设施通过 `build/nn_task` 固定 CLI 的数值评测:评测侧用 PyTorch 在相同数据上计算参考值,逐操作比较加权平均相对误差并对照阈值。
|
|
26
26
|
5. 提交的源码中不得出现 torch;候选运行时不提供 Python/torch。
|
|
27
27
|
|
|
28
|
-
|
|
28
|
+
评测侧用 PyTorch 计算参考值;工作区/候选运行时不提供 PyTorch 或 numpy(也无网络),开发阶段请用 C++ 或独立 double 标量参考做自查。提交源码必须完全独立。
|
|
29
29
|
|
|
30
30
|
## 3. 被控对象
|
|
31
31
|
|
|
@@ -34,7 +34,7 @@ agent 必须:
|
|
|
34
34
|
- 3 根均匀杆
|
|
35
35
|
- 第一关节、第三关节受控;第二关节被动
|
|
36
36
|
- 控制量: 两个归一化力矩 [u1, u3],每个 in [-1, 1]
|
|
37
|
-
- 物理力矩: tau_1 = u1 * tau_max
|
|
37
|
+
- 物理力矩: 关节 1 电机力矩 tau_1 = u1 * tau_max;关节 3 电机力矩 tau_3 = u3 * tau_max(关节 3 是杆 2/杆 3 之间的内部电机,杆 2 受到 -tau_3 反作用);关节 2 电机力矩为 0。在绝对角度坐标下广义驱动力向量为 (tau_1, -tau_3, tau_3)。
|
|
38
38
|
- 控制频率: 100 Hz,控制周期 dt = 0.01 s
|
|
39
39
|
- 每个控制周期内部用 4 个 RK4 子步
|
|
40
40
|
- 最大仿真时长: 30.0 s
|
|
@@ -118,7 +118,8 @@ obs = [a1, a2, a3] (rad)
|
|
|
118
118
|
~~~
|
|
119
119
|
tau_1 = u1 * tau_max
|
|
120
120
|
tau_3 = u3 * tau_max
|
|
121
|
-
tau_2 = 0
|
|
121
|
+
tau_2(joint motor) = 0
|
|
122
|
+
absolute generalized force = (tau_1, -tau_3, tau_3)
|
|
122
123
|
~~~
|
|
123
124
|
|
|
124
125
|
非有限值、超范围或格式错误:
|
|
@@ -131,6 +132,8 @@ score = -1
|
|
|
131
132
|
## 7. 训练仿真接口
|
|
132
133
|
|
|
133
134
|
工作区提供预编译 tools/sim,不提供源码和内部参数。
|
|
135
|
+
`tools/sim` 是**训练/自测专用**:每次 reset 只生成随机近向下初始状态,不包含固定评测初态,也不提供切换评测初态的参数。
|
|
136
|
+
评测使用独立程序 sim_eval;它每次 reset 固定为评测初始状态,不提供随机训练 reset。工作区不包含 sim_eval,也不包含任何固定评测初态参数。
|
|
134
137
|
环境只提供状态转移,不提供 reward、loss、cost 或 done 信号。训练信号由 agent 自行构造。
|
|
135
138
|
|
|
136
139
|
优化目标:
|
|
@@ -10,7 +10,7 @@
|
|
|
10
10
|
- 保持三根杆全部向上至少 t_tgt = 10.0 秒
|
|
11
11
|
- 同时保证神经网络基础设施通过 build/nn_task 固定 CLI 的数值评测
|
|
12
12
|
|
|
13
|
-
|
|
13
|
+
评测侧会用 PyTorch 计算算子参考值,但**工作区/候选运行时不提供 PyTorch 或 numpy**,也没有网络,无法 pip 安装。开发阶段请用 C++,或 Python 标准库/独立 double 实现做数值对照;最终提交的基础设施和控制器源码必须是纯 C++,不得出现 torch。
|
|
14
14
|
|
|
15
15
|
## 环境
|
|
16
16
|
|
|
@@ -22,6 +22,8 @@ tools/sim
|
|
|
22
22
|
|
|
23
23
|
不提供仿真器源码和物理参数。
|
|
24
24
|
|
|
25
|
+
工具说明:工作区的 `tools/sim` 是训练/自测专用仿真器,每次 `reset` 只会生成随机近向下初始状态;它不是评测器,也没有切换到固定评测初态的参数。评测使用工作区之外的独立固定初态仿真程序。
|
|
26
|
+
|
|
25
27
|
### 初始状态
|
|
26
28
|
|
|
27
29
|
训练时每次 reset:
|
|
@@ -47,7 +49,7 @@ joint 2:被动
|
|
|
47
49
|
joint 3:驱动
|
|
48
50
|
~~~
|
|
49
51
|
|
|
50
|
-
|
|
52
|
+
第一关节和第三关节各有一个归一化力矩输入。第三关节是杆 2/杆 3 之间的内部电机:它的力矩同时作用在杆 3 和杆 2 上(反作用),不是只作用在杆 3 上的外部力矩;关节 2 没有主动电机力矩。
|
|
51
53
|
|
|
52
54
|
### 观测
|
|
53
55
|
|
|
@@ -61,6 +63,7 @@ obs = [a1, a2, a3] (rad)
|
|
|
61
63
|
即 3 维观测,单位 rad,不除以 pi。
|
|
62
64
|
|
|
63
65
|
- theta_i 是第 i 根杆的绝对角度
|
|
66
|
+
- theta_i = 0 表示竖直向下,theta_i = pi 表示竖直向上
|
|
64
67
|
- a_i 是相对竖直向上的角度误差
|
|
65
68
|
- 不提供角速度
|
|
66
69
|
- 不提供第三杆末端位置
|
|
@@ -133,7 +136,7 @@ tools/sim 使用 JSONL 协议。
|
|
|
133
136
|
此时不返回新观测,也不会继续推进仿真。
|
|
134
137
|
必须 reset 后才能开始新 episode;reset 前的 step 都返回该错误。
|
|
135
138
|
|
|
136
|
-
可以反复 reset / step
|
|
139
|
+
可以反复 reset / step 进行数据采集、模型训练和自测。该程序只做随机训练 reset,不要依赖它获得评测固定初态;评测器是独立程序。
|
|
137
140
|
|
|
138
141
|
环境只返回状态,不提供 reward、loss、cost 或 done 信号。
|
|
139
142
|
训练信号由你自行构造。
|
|
@@ -468,7 +471,7 @@ T_done 是第一次返回 completed 的测试请求时间。
|
|
|
468
471
|
|
|
469
472
|
1. 阅读 build/nn_task 的 CLI 协议与 tools/sim 协议
|
|
470
473
|
2. 先实现最小可用的 tensor/算子/反向/优化器/线程池,并让 build/nn_task 跑通
|
|
471
|
-
3.
|
|
474
|
+
3. 用独立 double 标量参考、有限差分梯度检查逐算子核对误差,边界形状/广播/groups/stride/dilation 都要覆盖(工作区没有 PyTorch/numpy)
|
|
472
475
|
4. 用 tools/sim 采集数据并训练控制器,导出权重供 build/control 使用
|
|
473
476
|
5. 反复提交和测试,优先把 t_can 提升到 10 秒
|
|
474
477
|
6. 完成后立即总结
|
|
@@ -65,7 +65,13 @@ static void deriv(const State& s, const std::array<double,N>& tau, State& out) {
|
|
|
65
65
|
std::array<double,N> C{}, G{}, Q{}, rhs{};
|
|
66
66
|
for(int i=0;i<N;i++){
|
|
67
67
|
double ci=0;
|
|
68
|
-
for(int j=0;j<N;j++) if(j!=i)
|
|
68
|
+
for(int j=0;j<N;j++) if(j!=i) {
|
|
69
|
+
// Coriolis coefficient: d(M_ij)/d(theta_j) = LEN[min] * GCOEF[max] * sin(...);
|
|
70
|
+
// M_ij itself already contains cos(...), so multiplying M_ij by sin would
|
|
71
|
+
// add an extra cos factor and inject energy at high speed (NaN after ~7 s).
|
|
72
|
+
const int a=std::min(i,j), b=std::max(i,j);
|
|
73
|
+
ci += LEN[a] * GCOEF[b] * std::sin(s.theta[i]-s.theta[j]) * s.omega[j]*s.omega[j];
|
|
74
|
+
}
|
|
69
75
|
C[i]=ci;
|
|
70
76
|
G[i]=GRAV*std::sin(s.theta[i])*GCOEF[i];
|
|
71
77
|
}
|
|
@@ -83,13 +89,35 @@ static void deriv(const State& s, const std::array<double,N>& tau, State& out) {
|
|
|
83
89
|
} // namespace
|
|
84
90
|
|
|
85
91
|
int main(int argc, char** argv) {
|
|
86
|
-
|
|
87
|
-
|
|
92
|
+
#ifdef DSH_EVAL_SIM
|
|
93
|
+
/* Evaluator-only binary: every reset MUST use the fixed evaluation state.
|
|
94
|
+
* This build deliberately contains no random training reset path. */
|
|
95
|
+
(void)argc;
|
|
96
|
+
(void)argv;
|
|
97
|
+
#else
|
|
98
|
+
/* Agent/training binary: every reset is a random near-down state. This
|
|
99
|
+
* build deliberately contains no fixed evaluation state and no --eval flag,
|
|
100
|
+
* so the agent cannot query the evaluation initial condition from it. */
|
|
101
|
+
int seed=0;
|
|
102
|
+
for(int i=1;i<argc;i++){
|
|
103
|
+
std::string a=argv[i];
|
|
104
|
+
if(a=="--seed" && i+1<argc) seed=std::atoi(argv[++i]);
|
|
105
|
+
}
|
|
88
106
|
std::mt19937 rng(seed);
|
|
89
107
|
std::uniform_real_distribution<double> u(-1.0,1.0);
|
|
108
|
+
#endif
|
|
90
109
|
State s;
|
|
91
110
|
int steps=0; bool ready=false;
|
|
92
|
-
auto reset=[&](){
|
|
111
|
+
auto reset=[&](){
|
|
112
|
+
steps=0; ready=true;
|
|
113
|
+
#ifdef DSH_EVAL_SIM
|
|
114
|
+
s.theta={0.013,-0.008,0.021};
|
|
115
|
+
s.omega={0,0,0};
|
|
116
|
+
#else
|
|
117
|
+
s.theta={u(rng)*0.05,u(rng)*0.05,u(rng)*0.05};
|
|
118
|
+
s.omega={u(rng)*0.02,u(rng)*0.02,u(rng)*0.02};
|
|
119
|
+
#endif
|
|
120
|
+
};
|
|
93
121
|
// zero state
|
|
94
122
|
s.theta={0,0,0}; s.omega={0,0,0};
|
|
95
123
|
std::string line;
|
|
@@ -101,7 +129,12 @@ int main(int argc, char** argv) {
|
|
|
101
129
|
// parse torque array
|
|
102
130
|
double u1=0,u3=0; auto p=line.find('['); if(p!=std::string::npos){ char* end=nullptr; u1=std::strtod(line.c_str()+p+1,&end); if(end){ const char* q=end; while(*q && (*q==','||*q==' ')) q++; u3=std::strtod(q,&end); } }
|
|
103
131
|
u1=std::max(-1.0,std::min(1.0,u1)); u3=std::max(-1.0,std::min(1.0,u3));
|
|
104
|
-
|
|
132
|
+
// Joint 3 is an internal motor between link 2 and link 3. In absolute
|
|
133
|
+
// coordinates its generalized force is (tau1, -tau3, tau3): the reaction
|
|
134
|
+
// on link 2 is part of the same joint torque. (tau1, 0, tau3) would be a
|
|
135
|
+
// reactionless external torque on link 3 and would contradict joint 2
|
|
136
|
+
// being passive.
|
|
137
|
+
std::array<double,N> tau{{u1*TAU_MAX,-u3*TAU_MAX,u3*TAU_MAX}};
|
|
105
138
|
// one control step with 4 RK4 substeps
|
|
106
139
|
double h=DT/SUB;
|
|
107
140
|
for(int sub=0;sub<SUB;sub++){
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
package/lib/index.js
CHANGED
|
@@ -887,6 +887,8 @@ function bwrapArgs(workspace, rules, runtime) {
|
|
|
887
887
|
args.push("--setenv", "LC_ALL", "C.UTF-8");
|
|
888
888
|
args.push("--setenv", "TERM", "dumb");
|
|
889
889
|
args.push("--setenv", "UV_USE_IO_URING", "0");
|
|
890
|
+
args.push("--setenv", "PIP_DISABLE_PIP_VERSION_CHECK", "1");
|
|
891
|
+
args.push("--setenv", "PIP_NO_INPUT", "1");
|
|
890
892
|
return args;
|
|
891
893
|
}
|
|
892
894
|
/**
|
|
@@ -2480,10 +2482,12 @@ const NEURAL_PENDULUM_DEFINITION = {
|
|
|
2480
2482
|
"",
|
|
2481
2483
|
"请先完整阅读 TASK.md;它是权威规则。硬约束:仅 C++17 标准库、仅 CPU、fp32;不使用 Python/OpenCL/CUDA/BLAS/Eigen/第三方库;必须从提交源码构建出 build/nn_task 与 build/control。",
|
|
2482
2484
|
"",
|
|
2483
|
-
"环境细节:joint1 与 joint3 驱动,joint2 被动;控制频率 100 Hz;每回合最多 3000 步(30 s),达到上限后继续 step 返回 {\"error\":\"episode_finished\"},必须 reset
|
|
2485
|
+
"环境细节:joint1 与 joint3 驱动,joint2 被动;控制频率 100 Hz;每回合最多 3000 步(30 s),达到上限后继续 step 返回 {\"error\":\"episode_finished\"},必须 reset。theta=0 表示竖直向下、theta=pi 表示竖直向上;观测 a_i=wrap(theta_i-pi),即三个关节角度(弧度,不除 pi)。环境不返回 reward/loss/cost/done;不提供杆长、质量、惯量、角速度或 tip。训练 reset 带小范围随机初始角度和角速度;评测使用固定初始状态,无随机扰动。",
|
|
2484
2486
|
"",
|
|
2485
2487
|
"优化目标:从接近全向下起摆,使三根杆全部进入 0.05 rad 的竖直向上容差,并尽可能长时间保持;评测指标为 t_can。损失函数、奖励、优化器、网络结构和训练方法由你自行设计,不规定任何训练方案。",
|
|
2486
2488
|
"",
|
|
2489
|
+
"注意:评测侧用 PyTorch 计算算子参考值,但工作区/候选运行时不提供 PyTorch 或 numpy,也没有网络,无法 pip 安装;开发阶段请用 C++ 或 Python 标准库/独立 double 实现做数值对照。",
|
|
2490
|
+
"",
|
|
2487
2491
|
"评测会从干净副本重新构建提交源码,先运行与 Neural-FP32/Neural-CIFAR 相同的算子数值评测:评测侧用 PyTorch 在相同数据上计算参考值,逐操作比较加权平均相对误差并对照阈值,任一算子不可执行、崩溃、超时、输出缺失或误差超阈值即判为基础设施失败;然后在固定初始状态上驱动 build/control,统计连续全向上保持时间 t_can。不要把任何未提交的二进制、临时构建产物或外部进程当作交付依赖。",
|
|
2488
2492
|
"使用宿主工具 eval_submit 提交当前源码快照,使用 eval_test 请求宿主评测最新提交。提交快照、事件、测试结果和最终分数全部保存在工作区之外的宿主目录;不要尝试读取或修改这些数据,也不要依赖任何工作区内的测试结果文件。最终评测使用最新提交快照,按宿主事件时间积分计算分数。",
|
|
2489
2493
|
"",
|
|
Binary file
|
package/native/supervisor.c
CHANGED
|
@@ -3,8 +3,9 @@
|
|
|
3
3
|
*
|
|
4
4
|
* Installs a seccomp user-notification filter before exec'ing bwrap and enforces
|
|
5
5
|
* a path policy on file-access and network syscalls. Normal syscalls are not
|
|
6
|
-
* logged
|
|
7
|
-
*
|
|
6
|
+
* logged. File-policy violations are audited and kill the process group;
|
|
7
|
+
* AF_INET/AF_INET6 connect/bind/sendto are audited and soft-denied with EPERM
|
|
8
|
+
* so local software checks (pip version probe, NSS fallback) can continue.
|
|
8
9
|
*
|
|
9
10
|
* Usage:
|
|
10
11
|
* dsh-eval-supervisor --policy <policy-file> --audit <audit-file> -- <program> [args...]
|
|
@@ -87,6 +88,7 @@ static int g_dev_count;
|
|
|
87
88
|
static Rule g_rules[MAX_RULES];
|
|
88
89
|
static int g_rule_count;
|
|
89
90
|
static int g_audit_fd = -1;
|
|
91
|
+
static int g_soft_deny; /* network-family denial: EPERM, do not kill the group */
|
|
90
92
|
static volatile sig_atomic_t g_stop;
|
|
91
93
|
/* The sandbox mounted a private tmpfs at /tmp (workspace under host /tmp). */
|
|
92
94
|
static int g_tmpfs_tmp;
|
|
@@ -555,6 +557,7 @@ static int check_notification(pid_t pid, const struct seccomp_notif *req, int *w
|
|
|
555
557
|
|
|
556
558
|
path_out[0] = '\0';
|
|
557
559
|
g_current_pid = pid;
|
|
560
|
+
g_soft_deny = 0;
|
|
558
561
|
if (nr == SYS_execve || nr == SYS_execveat) {
|
|
559
562
|
char executable[MAX_PATH];
|
|
560
563
|
executable[0] = '\0';
|
|
@@ -567,20 +570,43 @@ static int check_notification(pid_t pid, const struct seccomp_notif *req, int *w
|
|
|
567
570
|
return 1;
|
|
568
571
|
}
|
|
569
572
|
if (nr == SYS_socket) {
|
|
570
|
-
|
|
573
|
+
/* Creating a socket does not transmit anything. bwrap already put the
|
|
574
|
+
* command in a private network namespace; tools such as pip create an
|
|
575
|
+
* AF_INET socket for local/version checks without connecting. All the
|
|
576
|
+
* operations that can actually reach a peer (connect/bind/listen/accept/
|
|
577
|
+
* sendto/recvfrom) remain denied below. */
|
|
571
578
|
*write_access = 0;
|
|
572
579
|
snprintf(path_out, MAX_PATH, "<network:socket>");
|
|
573
|
-
|
|
574
|
-
* needed by NSS, Python and the dynamic loader. AF_INET/AF_INET6 and raw
|
|
575
|
-
* sockets are still blocked. */
|
|
576
|
-
if (domain == AF_UNIX || domain == AF_NETLINK) return 1;
|
|
577
|
-
return 0;
|
|
580
|
+
return 1;
|
|
578
581
|
}
|
|
579
582
|
if (is_network_syscall(nr)) {
|
|
580
583
|
*write_access = 0;
|
|
581
584
|
snprintf(path_out, MAX_PATH, "<network:%s>", syscall_name(nr));
|
|
582
|
-
/*
|
|
583
|
-
*
|
|
585
|
+
/* bwrap itself needs AF_UNIX bind/listen/connect while it sets up the
|
|
586
|
+
* namespace; only the target's traffic is checked. */
|
|
587
|
+
if (!g_command_started || (g_current_pid > 0 && is_bwrap(g_current_pid))) return 1;
|
|
588
|
+
if (nr == SYS_listen || nr == SYS_accept || nr == SYS_accept4 || nr == SYS_recvfrom) {
|
|
589
|
+
/* No peer can appear outside the private netns; these operate on an
|
|
590
|
+
* already-authorized local socket. */
|
|
591
|
+
return 1;
|
|
592
|
+
}
|
|
593
|
+
if (nr == SYS_connect || nr == SYS_bind || nr == SYS_sendto) {
|
|
594
|
+
unsigned long addr_ptr = nr == SYS_sendto ? (unsigned long)req->data.args[4] : a1;
|
|
595
|
+
unsigned short family = 0;
|
|
596
|
+
if (addr_ptr != 0 && read_remote(pid, addr_ptr, &family, sizeof(family)) == 0) {
|
|
597
|
+
if (family == AF_UNIX || family == AF_NETLINK) return 1;
|
|
598
|
+
if (family == AF_INET || family == AF_INET6) {
|
|
599
|
+
/* A version check or NSS probe may create the socket and attempt a
|
|
600
|
+
* connection; return EPERM instead of killing the whole group. The
|
|
601
|
+
* private netns is the hard boundary; this is the graceful denial. */
|
|
602
|
+
g_soft_deny = 1;
|
|
603
|
+
snprintf(path_out, MAX_PATH, "<network:%s:inet>", syscall_name(nr));
|
|
604
|
+
return 0;
|
|
605
|
+
}
|
|
606
|
+
}
|
|
607
|
+
g_soft_deny = 1;
|
|
608
|
+
return 0;
|
|
609
|
+
}
|
|
584
610
|
return 1;
|
|
585
611
|
}
|
|
586
612
|
if (nr == SYS_io_uring_setup || nr == SYS_ptrace || nr == SYS_open_by_handle_at) {
|
|
@@ -817,6 +843,12 @@ static void reply_loop(int fd) {
|
|
|
817
843
|
char syscall_buf[64];
|
|
818
844
|
snprintf(syscall_buf, sizeof(syscall_buf), "%s", syscall_name((long)req.data.nr));
|
|
819
845
|
audit_record((pid_t)req.pid, syscall_buf, path, write_access);
|
|
846
|
+
if (g_soft_deny) {
|
|
847
|
+
/* A software check (pip version probe, NSS lookup fallback) gets a
|
|
848
|
+
* normal EPERM and can continue; file-policy violations still kill. */
|
|
849
|
+
respond(fd, &req, 0);
|
|
850
|
+
continue;
|
|
851
|
+
}
|
|
820
852
|
/* io_uring is a soft denial: libuv probes it despite UV_USE_IO_URING=0
|
|
821
853
|
* and must receive EPERM and fall back to epoll instead of taking the
|
|
822
854
|
* whole command group down. No io_uring instance can exist, so the
|
|
@@ -13,8 +13,10 @@ plugin UI.
|
|
|
13
13
|
- `ops_eval.py` — operator oracle: PyTorch references computed in-process,
|
|
14
14
|
weighted mean relative error compared with per-operation thresholds. Same
|
|
15
15
|
operations, cases, seeds and thresholds as Neural-FP32 / Neural-CIFAR.
|
|
16
|
-
- `sim_eval` — trusted simulator binary
|
|
17
|
-
|
|
16
|
+
- `sim_eval` — trusted evaluator-only simulator binary. Every `reset` is the
|
|
17
|
+
fixed evaluation initial state; it has no random training reset and no
|
|
18
|
+
`--eval` switch. The workspace `tools/sim` is a separate agent/training
|
|
19
|
+
binary that only does random near-down resets and is not shipped here.
|
|
18
20
|
|
|
19
21
|
## Requirements
|
|
20
22
|
|
|
@@ -266,7 +266,9 @@ def main() -> int:
|
|
|
266
266
|
return emit(fail(ops_eval.infrastructure_error(ops), 'infra', workspace, events, ops=ops))
|
|
267
267
|
|
|
268
268
|
env = child_env()
|
|
269
|
-
|
|
269
|
+
# sim_eval is the evaluator-only binary: reset is always the fixed
|
|
270
|
+
# evaluation initial state; no flag or random mode exists there.
|
|
271
|
+
sim = subprocess.Popen([str(SIM)], stdin=subprocess.PIPE,
|
|
270
272
|
stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, env=env)
|
|
271
273
|
control = subprocess.Popen([str(control_path)], cwd=str(source),
|
|
272
274
|
stdin=subprocess.PIPE, stdout=subprocess.PIPE,
|
|
Binary file
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "dsh-eval-guard",
|
|
3
|
-
"version": "0.7.
|
|
3
|
+
"version": "0.7.5",
|
|
4
4
|
"type": "module",
|
|
5
5
|
"description": "Evaluation guard for DSH: workspace-only filesystem, no network/subagents, wall-clock limits, and task installation for meow-ai-arena and benchmark tasks.",
|
|
6
6
|
"license": "MIT",
|
|
@@ -48,7 +48,7 @@
|
|
|
48
48
|
"build": "tsdown && node scripts/build-native.mjs",
|
|
49
49
|
"build:native": "node scripts/build-native.mjs",
|
|
50
50
|
"prepare": "tsdown",
|
|
51
|
-
"test": "node test/simulate.ts && node test/scoring.ts && node test/jobs.ts",
|
|
51
|
+
"test": "node test/simulate.ts && node test/scoring.ts && node test/jobs.ts && node test/pendulum-sim.ts",
|
|
52
52
|
"typecheck": "tsc --noEmit -p tsconfig.json"
|
|
53
53
|
},
|
|
54
54
|
"dsh": {
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|