dsh-scnet 0.4.2 → 0.5.0-rc.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +20 -1
- package/README_CN.md +20 -1
- package/docs/installation.md +10 -0
- package/docs/installation.zh-CN.md +13 -1
- package/docs/testing.md +11 -0
- package/docs/testing.zh-CN.md +10 -0
- package/index.mjs +184 -0
- package/package.json +14 -4
- package/skills/scnet-hpc/SKILL.md +63 -105
- package/skills/scnet-hpc/VERSION +1 -0
- package/skills/scnet-hpc/clusters/_template.conf +10 -4
- package/skills/scnet-hpc/references/adding-cluster.md +6 -1
- package/skills/scnet-hpc/references/backends.md +177 -0
- package/skills/scnet-hpc/references/notebook.md +60 -0
- package/skills/scnet-hpc/references/openapi.md +140 -0
- package/skills/scnet-hpc/references/quickstart-en.md +22 -0
- package/skills/scnet-hpc/references/setup.md +4 -0
- package/skills/scnet-hpc/scripts/__pycache__/compute-probe.cpython-312.pyc +0 -0
- package/skills/scnet-hpc/scripts/__pycache__/scnet.cpython-312.pyc +0 -0
- package/skills/scnet-hpc/scripts/__pycache__/scnet_config.cpython-312.pyc +0 -0
- package/skills/scnet-hpc/scripts/__pycache__/scnet_credentials.cpython-312.pyc +0 -0
- package/skills/scnet-hpc/scripts/__pycache__/scnet_version.cpython-312.pyc +0 -0
- package/skills/scnet-hpc/scripts/scnet.py +1573 -0
- package/skills/scnet-hpc/scripts/scnet_backends/__init__.py +53 -0
- package/skills/scnet-hpc/scripts/scnet_backends/base.py +64 -0
- package/skills/scnet-hpc/scripts/scnet_backends/external.py +92 -0
- package/skills/scnet-hpc/scripts/scnet_backends/openapi.py +771 -0
- package/skills/scnet-hpc/scripts/scnet_backends/profile.py +78 -0
- package/skills/scnet-hpc/scripts/scnet_backends/ssh.py +258 -0
- package/skills/scnet-hpc/scripts/scnet_config.py +175 -0
- package/skills/scnet-hpc/scripts/scnet_credentials.py +160 -0
- package/skills/scnet-hpc/scripts/scnet_sdk/__init__.py +11 -0
- package/skills/scnet-hpc/scripts/scnet_sdk/cli.py +394 -0
- package/skills/scnet-hpc/scripts/scnet_sdk/client.py +240 -0
- package/skills/scnet-hpc/scripts/scnet_sdk/notebook.py +379 -0
- package/skills/scnet-hpc/scripts/scnet_version.py +20 -0
- package/skills/scnet-hpc/scripts/setup-ssh.sh +33 -1
- package/skills/scnet-hpc/scripts/setup.sh +471 -0
package/README.md
CHANGED
|
@@ -2,7 +2,9 @@
|
|
|
2
2
|
|
|
3
3
|
[简体中文](./README_CN.md)
|
|
4
4
|
|
|
5
|
-
DSH-SCNet (`dsh-scnet` on npm) is a community-maintained DSH bundle for operating Supercomputing Network (SCNet)
|
|
5
|
+
DSH-SCNet (`dsh-scnet` on npm) is a community-maintained DSH bundle for operating Supercomputing Network (SCNet). It packages the source [`scnet-hpc`](https://github.com/lql341/scnet-hpc) Skill, profile-aware utilities, and bounded tools for SSH, Slurm, OpenAPI files, regional resources, and Notebook discovery.
|
|
6
|
+
|
|
7
|
+
Current release: **0.5.0-rc.1-rc.1**
|
|
6
8
|
|
|
7
9
|
> This is an independent community project. It is compatible with DeepSeek Harness but is not an official DeepSeek product and does not imply endorsement, partnership, or authorization by DeepSeek.
|
|
8
10
|
|
|
@@ -40,6 +42,14 @@ See [Installation](./docs/installation.md) for npm, GitHub, and local-checkout w
|
|
|
40
42
|
| `scnet_probe_cluster` | Produce an initial profile from read-only login-node probes |
|
|
41
43
|
| `scnet_refresh_cluster` | Refresh time-sensitive profile fields; compute probing is opt-in |
|
|
42
44
|
| `scnet_run_compute_probe` | Submit a minimal compute-node capability probe |
|
|
45
|
+
| `scnet_status` | Read the saved backend, SSH profile, and OpenAPI configuration |
|
|
46
|
+
| `scnet_openapi_regions` | List authorized OpenAPI regions without exposing tokens |
|
|
47
|
+
| `scnet_job_queues` | Query regional Slurm queues and live resource availability |
|
|
48
|
+
| `scnet_file_list` | List shared-storage files through the common file API |
|
|
49
|
+
| `scnet_notebook_regions` | List Notebook-capable regions |
|
|
50
|
+
| `scnet_notebook_resources` | Query Notebook CPU/GPU/DCU resources |
|
|
51
|
+
| `scnet_notebook_list` | List Notebook instances with sensitive fields redacted |
|
|
52
|
+
| `scnet_notebook_show` | Inspect one redacted Notebook instance |
|
|
43
53
|
|
|
44
54
|
Packaged profiles currently cover Zhengzhou, Kunshan, Wuzhen, and Xi'an SCNet environments. Cluster specifications and scheduler policies remain profile-specific and should be verified against the target environment.
|
|
45
55
|
|
|
@@ -55,6 +65,8 @@ Packaged profiles currently cover Zhengzhou, Kunshan, Wuzhen, and Xi'an SCNet en
|
|
|
55
65
|
│ ├── clusters/
|
|
56
66
|
│ ├── references/
|
|
57
67
|
│ └── scripts/
|
|
68
|
+
│ ├── scnet_backends/
|
|
69
|
+
│ └── scnet_sdk/
|
|
58
70
|
├── scripts/validate-package.mjs
|
|
59
71
|
├── docs/
|
|
60
72
|
├── sync.sh
|
|
@@ -87,6 +99,8 @@ See [Testing](./docs/testing.md) for local package installation and risk-ordered
|
|
|
87
99
|
|
|
88
100
|
- The repository and npm package must not contain private keys, tokens, usernames, private endpoints, or local probe caches.
|
|
89
101
|
- SSH configuration, remote probes, and Slurm submission are state-changing operations and require an explicit target and user authorization.
|
|
102
|
+
- OpenAPI and Notebook DSH tools are read-only; lifecycle mutations remain behind the Skill CLI confirmation flow.
|
|
103
|
+
- Notebook passwords and credential-bearing URL queries are redacted by default.
|
|
90
104
|
- Generated Slurm files may include local usernames and are ignored by Git.
|
|
91
105
|
- Accelerator compatibility claims require evidence from the target compute node.
|
|
92
106
|
|
|
@@ -96,6 +110,11 @@ The project uses the abbreviated `DSH` ecosystem identifier in its name. Referen
|
|
|
96
110
|
|
|
97
111
|
The naming and attribution policy follows the [DeepSeek Harness brand guidelines](https://github.com/deepseek-ai/deepseek-harness/blob/dsh-v0.1.0-rc.8/BRAND_GUIDELINES.md).
|
|
98
112
|
|
|
113
|
+
## Versioning
|
|
114
|
+
|
|
115
|
+
The npm package version matches `skills/scnet-hpc/VERSION` and follows the same SemVer release
|
|
116
|
+
train as the source Skill.
|
|
117
|
+
|
|
99
118
|
## License
|
|
100
119
|
|
|
101
120
|
This project is released under the [MIT License](LICENSE). Subject to the license terms, the software may be used, copied, modified, merged, published, sublicensed, and distributed, including for commercial purposes.
|
package/README_CN.md
CHANGED
|
@@ -2,7 +2,9 @@
|
|
|
2
2
|
|
|
3
3
|
[English](./README.md) | 简体中文
|
|
4
4
|
|
|
5
|
-
<p align="center">在 DeepSeek Harness 里操作超算互联网(SCNet
|
|
5
|
+
<p align="center">在 DeepSeek Harness 里操作超算互联网(SCNet):配置 SSH、管理 Slurm 作业、查询 OpenAPI 区域和文件、检查 Notebook 资源与实例,并保留海光 DCU/DTK 诊断能力。</p>
|
|
6
|
+
|
|
7
|
+
当前版本:**0.5.0-rc.1-rc.1**
|
|
6
8
|
|
|
7
9
|
> 本项目是独立维护的社区项目,与 DeepSeek Harness 兼容,但不是 DeepSeek 官方产品,也不表示 DeepSeek 对本项目提供背书、合作或授权。
|
|
8
10
|
|
|
@@ -48,6 +50,14 @@ git 源、本地目录安装以及验证步骤见[安装说明](./docs/installat
|
|
|
48
50
|
| scnet_probe_cluster | 探测新集群并生成 profile |
|
|
49
51
|
| scnet_refresh_cluster | 动态刷新已有集群的规则缓存 |
|
|
50
52
|
| scnet_run_compute_probe | 在计算节点运行最小能力探针 |
|
|
53
|
+
| scnet_status | 只读查看 backend、SSH profile 和 OpenAPI 配置 |
|
|
54
|
+
| scnet_openapi_regions | 列出授权 OpenAPI 区域,不暴露 token |
|
|
55
|
+
| scnet_job_queues | 查询区域 Slurm 队列和实时空闲资源 |
|
|
56
|
+
| scnet_file_list | 通过通用文件 API 列出共享存储 |
|
|
57
|
+
| scnet_notebook_regions | 列出支持 Notebook 的区域 |
|
|
58
|
+
| scnet_notebook_resources | 查询 Notebook CPU/GPU/DCU 资源 |
|
|
59
|
+
| scnet_notebook_list | 列出 Notebook 实例并脱敏敏感字段 |
|
|
60
|
+
| scnet_notebook_show | 查询一个脱敏后的 Notebook 实例详情 |
|
|
51
61
|
|
|
52
62
|
## 文档
|
|
53
63
|
|
|
@@ -68,6 +78,8 @@ git 源、本地目录安装以及验证步骤见[安装说明](./docs/installat
|
|
|
68
78
|
│ │ ├── wuzhshell.conf
|
|
69
79
|
│ │ └── xianshell.conf
|
|
70
80
|
│ ├── scripts/
|
|
81
|
+
│ │ ├── scnet_backends/
|
|
82
|
+
│ │ └── scnet_sdk/
|
|
71
83
|
│ └── references/
|
|
72
84
|
├── sync.sh # 从 canonical 仓库同步
|
|
73
85
|
├── README.md # 英文默认说明
|
|
@@ -96,6 +108,8 @@ git 源、本地目录安装以及验证步骤见[安装说明](./docs/installat
|
|
|
96
108
|
- `scnet_generate_job` 生成的 `.slurm` 可能含本机用户名,`.gitignore` 已排除。
|
|
97
109
|
- SSH 接入端点仅使用平台面向用户公开提供的信息,由 canonical 仓库维护。
|
|
98
110
|
- `scnet_refresh_cluster` / `scnet_run_compute_probe` 会 SSH 到远端并可能提交作业;运行前确认集群和副作用。
|
|
111
|
+
- 新增 OpenAPI 和 Notebook DSH 工具均为只读;Notebook 生命周期变更仍通过 Skill CLI 的确认流程执行。
|
|
112
|
+
- Notebook 密码和带凭据的 URL 查询参数默认脱敏。
|
|
99
113
|
- `clusters/.cache/` 是本机动态探测缓存,`.gitignore` 已排除,不应提交。
|
|
100
114
|
|
|
101
115
|
## License
|
|
@@ -105,3 +119,8 @@ git 源、本地目录安装以及验证步骤见[安装说明](./docs/installat
|
|
|
105
119
|
再发布本项目或其重要组成部分时,应保留版权声明和 MIT 许可声明。本项目按“现状”提供,不对适销性、特定用途适用性或不侵权作任何明示或默示保证;使用者应自行评估 DSH bundle、skill 说明、集群配置、脚本和生成结果在其环境中的适用性与风险。
|
|
106
120
|
|
|
107
121
|
项目命名与归属声明遵循 [DeepSeek Harness 品牌素材使用规范](https://github.com/deepseek-ai/deepseek-harness/blob/dsh-v0.1.0-rc.8/BRAND_GUIDELINES.zh.md)。
|
|
122
|
+
|
|
123
|
+
## 版本管理
|
|
124
|
+
|
|
125
|
+
npm package 版本与 `skills/scnet-hpc/VERSION` 保持一致,并与源 Skill 使用同一套 SemVer
|
|
126
|
+
发布版本。
|
package/docs/installation.md
CHANGED
|
@@ -58,3 +58,13 @@ dsh web
|
|
|
58
58
|
## Credentials
|
|
59
59
|
|
|
60
60
|
Model credentials belong in the DSH credential store or its supported settings interface. SCNet private keys remain local and are supplied only when the user explicitly invokes SSH setup. Neither credential type belongs in this repository.
|
|
61
|
+
|
|
62
|
+
SCNet OpenAPI uses a platform username, AccessKey, and SecretKey. Configure them through the
|
|
63
|
+
bundled Skill:
|
|
64
|
+
|
|
65
|
+
```sh
|
|
66
|
+
python3 skills/scnet-hpc/scripts/scnet.py setup modify --mode openapi
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
On macOS credentials can be stored in Keychain. On Linux, install `libsecret-tools` for Secret
|
|
70
|
+
Service; otherwise inject the documented `SCNET_OPENAPI_*` environment variables.
|
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
# 安装 DSH-SCNet
|
|
2
2
|
|
|
3
|
-
DSH-SCNet(npm 包名:`dsh-scnet
|
|
3
|
+
DSH-SCNet(npm 包名:`dsh-scnet`)是一个 DeepSeek Harness(DSH)bundle 插件,把
|
|
4
|
+
SCNet 使用规范、SSH/Slurm 工具、OpenAPI 只读资源工具和 Notebook 资源工具打包到一起。
|
|
4
5
|
|
|
5
6
|
## 前置条件
|
|
6
7
|
|
|
@@ -71,3 +72,14 @@ DEEPSEEK_API_KEY: sk-你的key
|
|
|
71
72
|
```
|
|
72
73
|
|
|
73
74
|
保存后无需重启,下一次请求即生效。Key 只保存在本机 `~/.dsh` 下,不会写入仓库。
|
|
75
|
+
|
|
76
|
+
## 配置 SCNet OpenAPI
|
|
77
|
+
|
|
78
|
+
SCNet OpenAPI 使用平台用户名、AccessKey 和 SecretKey:
|
|
79
|
+
|
|
80
|
+
```sh
|
|
81
|
+
python3 skills/scnet-hpc/scripts/scnet.py setup modify --mode openapi
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
macOS 可保存到 Keychain。Linux 安装 `libsecret-tools` 后使用 Secret Service;否则通过
|
|
85
|
+
文档定义的 `SCNET_OPENAPI_*` 环境变量注入。OpenAPI 凭据不会写入插件仓库。
|
package/docs/testing.md
CHANGED
|
@@ -37,6 +37,14 @@ Confirm that the composed configuration includes:
|
|
|
37
37
|
| `scnet_list_clusters` | Returns the packaged cluster identifiers |
|
|
38
38
|
| `scnet_show_cluster` | Returns the selected profile contents |
|
|
39
39
|
| `scnet_generate_job` | Writes a profile-aware Slurm script in the current directory |
|
|
40
|
+
| `scnet_status` | Returns redacted local SCNet configuration |
|
|
41
|
+
| `scnet_openapi_regions` | Returns authorized regions without tokens |
|
|
42
|
+
| `scnet_job_queues` | Returns live regional Slurm queues |
|
|
43
|
+
| `scnet_file_list` | Lists shared-storage files without modification |
|
|
44
|
+
| `scnet_notebook_regions` | Lists Notebook-capable regions |
|
|
45
|
+
| `scnet_notebook_resources` | Returns Notebook resource groups |
|
|
46
|
+
| `scnet_notebook_list` | Lists redacted Notebook instances |
|
|
47
|
+
| `scnet_notebook_show` | Returns one redacted Notebook instance |
|
|
40
48
|
|
|
41
49
|
Check both accelerator and CPU-only job generation. Review the output for the selected partition, memory, and GRES rules.
|
|
42
50
|
|
|
@@ -52,6 +60,9 @@ Check both accelerator and CPU-only job generation. Review the output for the se
|
|
|
52
60
|
|
|
53
61
|
`scnet_run_compute_probe` and refresh operations with compute probing submit Slurm jobs. Confirm the target cluster, account policy, accelerator count, CPU count, and time limit before execution.
|
|
54
62
|
|
|
63
|
+
Notebook lifecycle mutations are intentionally not exposed as deterministic DSH tools in this
|
|
64
|
+
release. Test them only through the Skill CLI and start with `--dry-run`.
|
|
65
|
+
|
|
55
66
|
## Cleanup
|
|
56
67
|
|
|
57
68
|
Remove the isolated test profile after verification:
|
package/docs/testing.zh-CN.md
CHANGED
|
@@ -58,5 +58,15 @@ DEEPSEEK_API_KEY: sk-你的key
|
|
|
58
58
|
| 5 | 用 scnet_probe_cluster 探测集群 | 需要真实集群,会 SSH 到远端 | 中 |
|
|
59
59
|
| 6 | 用 scnet_refresh_cluster dry_run=true 刷新规则 | 输出即将写入的缓存,不落盘 | 中(会 SSH) |
|
|
60
60
|
| 7 | 用 scnet_run_compute_probe 跑计算节点探针 | 返回 PROBE_ 结果 | 高(提交作业并等待) |
|
|
61
|
+
| 8 | 用 scnet_status 查看当前配置 | 返回脱敏配置 | 只读 |
|
|
62
|
+
| 9 | 用 scnet_openapi_regions 查看授权区域 | 返回区域且不含 token | 只读 |
|
|
63
|
+
| 10 | 用 scnet_job_queues 查看昆山队列 | 返回实时队列和空闲资源 | 只读 |
|
|
64
|
+
| 11 | 用 scnet_file_list 查看默认目录 | 返回共享存储文件 | 只读 |
|
|
65
|
+
| 12 | 用 scnet_notebook_regions 查看区域 | 返回 Notebook 可用区域 | 只读 |
|
|
66
|
+
| 13 | 用 scnet_notebook_resources 查看昆山资源 | 返回 CPU/GPU/DCU 资源 | 只读 |
|
|
67
|
+
| 14 | 用 scnet_notebook_list 查看实例 | 返回脱敏实例列表 | 只读 |
|
|
61
68
|
|
|
62
69
|
第 4 至 7 步只有在你有私钥和目标集群时才测,测前确认副作用。
|
|
70
|
+
|
|
71
|
+
Notebook 创建、启动、停止和释放没有作为 DSH deterministic tool 暴露。此类操作通过
|
|
72
|
+
Skill CLI 执行,并先使用 `--dry-run`。
|
package/index.mjs
CHANGED
|
@@ -31,6 +31,35 @@ function runBash(script, args, timeoutMs = 120000, cwd = SKILL_DIR) {
|
|
|
31
31
|
})
|
|
32
32
|
}
|
|
33
33
|
|
|
34
|
+
function runScnet(args, timeoutMs = 120000) {
|
|
35
|
+
return new Promise((resolve) => {
|
|
36
|
+
execFile(
|
|
37
|
+
'python3',
|
|
38
|
+
[join(SCRIPTS_DIR, 'scnet.py'), '--json', ...args],
|
|
39
|
+
{ cwd: SKILL_DIR, timeout: timeoutMs, maxBuffer: 16 * 1024 * 1024 },
|
|
40
|
+
(error, stdout, stderr) => {
|
|
41
|
+
const code = error ? (typeof error.code === 'number' ? error.code : 1) : 0
|
|
42
|
+
resolve({
|
|
43
|
+
ok: code === 0,
|
|
44
|
+
code,
|
|
45
|
+
stdout: String(stdout || ''),
|
|
46
|
+
stderr: String(stderr || ''),
|
|
47
|
+
})
|
|
48
|
+
},
|
|
49
|
+
)
|
|
50
|
+
})
|
|
51
|
+
}
|
|
52
|
+
|
|
53
|
+
function scnetResult(res, action) {
|
|
54
|
+
if (res.ok) return res.stdout.trim() || '{}'
|
|
55
|
+
return `${action}失败(exit ${res.code}):\n${res.stderr.trim() || res.stdout.trim()}`
|
|
56
|
+
}
|
|
57
|
+
|
|
58
|
+
function regionArgs(region) {
|
|
59
|
+
const value = String(region || '').trim()
|
|
60
|
+
return value ? ['--region', value] : []
|
|
61
|
+
}
|
|
62
|
+
|
|
34
63
|
function textBlock(value) {
|
|
35
64
|
return [{ type: 'text', text: value }]
|
|
36
65
|
}
|
|
@@ -91,6 +120,161 @@ export function apply(ctx) {
|
|
|
91
120
|
}),
|
|
92
121
|
)
|
|
93
122
|
|
|
123
|
+
ctx.tools.register(
|
|
124
|
+
defineTool({
|
|
125
|
+
name: 'scnet_status',
|
|
126
|
+
description: '只读查看 scnet-hpc 当前默认 backend、SSH profile 和 OpenAPI 区域配置;不会连接远端或修改配置。',
|
|
127
|
+
parameters: {},
|
|
128
|
+
output: {
|
|
129
|
+
schema: { type: 'string' },
|
|
130
|
+
render: (_args, value) => textBlock(value),
|
|
131
|
+
},
|
|
132
|
+
async execute() {
|
|
133
|
+
return scnetResult(await runScnet(['config']), '读取 SCNet 配置')
|
|
134
|
+
},
|
|
135
|
+
}),
|
|
136
|
+
)
|
|
137
|
+
|
|
138
|
+
ctx.tools.register(
|
|
139
|
+
defineTool({
|
|
140
|
+
name: 'scnet_openapi_regions',
|
|
141
|
+
description: '通过已配置的 SCNet OpenAPI 凭据只读列出授权计算区域,不显示 token。',
|
|
142
|
+
parameters: {},
|
|
143
|
+
output: {
|
|
144
|
+
schema: { type: 'string' },
|
|
145
|
+
render: (_args, value) => textBlock(value),
|
|
146
|
+
},
|
|
147
|
+
async execute() {
|
|
148
|
+
return scnetResult(
|
|
149
|
+
await runScnet(['--backend', 'openapi', 'clusters']),
|
|
150
|
+
'查询 OpenAPI 区域',
|
|
151
|
+
)
|
|
152
|
+
},
|
|
153
|
+
}),
|
|
154
|
+
)
|
|
155
|
+
|
|
156
|
+
ctx.tools.register(
|
|
157
|
+
defineTool({
|
|
158
|
+
name: 'scnet_job_queues',
|
|
159
|
+
description: '通过 OpenAPI 只读查询目标区域可访问的 Slurm 队列、空闲资源和单作业限制。',
|
|
160
|
+
parameters: {
|
|
161
|
+
region: { type: 'string', description: '区域名称或 ID;省略时使用已保存的默认区域' },
|
|
162
|
+
},
|
|
163
|
+
output: {
|
|
164
|
+
schema: { type: 'string' },
|
|
165
|
+
render: (_args, value) => textBlock(value),
|
|
166
|
+
},
|
|
167
|
+
async execute(args) {
|
|
168
|
+
return scnetResult(
|
|
169
|
+
await runScnet(['--backend', 'openapi', ...regionArgs(args.region), 'queues']),
|
|
170
|
+
'查询作业队列',
|
|
171
|
+
)
|
|
172
|
+
},
|
|
173
|
+
}),
|
|
174
|
+
)
|
|
175
|
+
|
|
176
|
+
ctx.tools.register(
|
|
177
|
+
defineTool({
|
|
178
|
+
name: 'scnet_file_list',
|
|
179
|
+
description: '通过 OpenAPI 只读列出区域共享存储中的文件和目录。',
|
|
180
|
+
parameters: {
|
|
181
|
+
region: { type: 'string', description: '区域名称或 ID;省略时使用默认区域' },
|
|
182
|
+
path: { type: 'string', description: '绝对目录路径;省略时使用区域用户主目录' },
|
|
183
|
+
limit: { type: 'number', description: '最多返回条数,默认 100' },
|
|
184
|
+
},
|
|
185
|
+
output: {
|
|
186
|
+
schema: { type: 'string' },
|
|
187
|
+
render: (_args, value) => textBlock(value),
|
|
188
|
+
},
|
|
189
|
+
async execute(args) {
|
|
190
|
+
const argv = ['--backend', 'openapi', ...regionArgs(args.region), 'files']
|
|
191
|
+
if (args.path) argv.push('--path', String(args.path))
|
|
192
|
+
if (args.limit) argv.push('--limit', String(args.limit))
|
|
193
|
+
return scnetResult(await runScnet(argv), '查询文件')
|
|
194
|
+
},
|
|
195
|
+
}),
|
|
196
|
+
)
|
|
197
|
+
|
|
198
|
+
ctx.tools.register(
|
|
199
|
+
defineTool({
|
|
200
|
+
name: 'scnet_notebook_regions',
|
|
201
|
+
description: '只读列出支持 SCNet Notebook 服务的授权区域。',
|
|
202
|
+
parameters: {},
|
|
203
|
+
output: {
|
|
204
|
+
schema: { type: 'string' },
|
|
205
|
+
render: (_args, value) => textBlock(value),
|
|
206
|
+
},
|
|
207
|
+
async execute() {
|
|
208
|
+
return scnetResult(
|
|
209
|
+
await runScnet(['--backend', 'openapi', 'notebook', 'regions']),
|
|
210
|
+
'查询 Notebook 区域',
|
|
211
|
+
)
|
|
212
|
+
},
|
|
213
|
+
}),
|
|
214
|
+
)
|
|
215
|
+
|
|
216
|
+
ctx.tools.register(
|
|
217
|
+
defineTool({
|
|
218
|
+
name: 'scnet_notebook_resources',
|
|
219
|
+
description: '只读查询目标区域可用于 Notebook 的 CPU/GPU/DCU 资源及当前空闲卡数。',
|
|
220
|
+
parameters: {
|
|
221
|
+
region: { type: 'string', description: '区域名称或 ID;省略时使用默认区域' },
|
|
222
|
+
},
|
|
223
|
+
output: {
|
|
224
|
+
schema: { type: 'string' },
|
|
225
|
+
render: (_args, value) => textBlock(value),
|
|
226
|
+
},
|
|
227
|
+
async execute(args) {
|
|
228
|
+
const argv = ['--backend', 'openapi', 'notebook', 'resources']
|
|
229
|
+
if (args.region) argv.push('--region', String(args.region))
|
|
230
|
+
return scnetResult(await runScnet(argv), '查询 Notebook 资源')
|
|
231
|
+
},
|
|
232
|
+
}),
|
|
233
|
+
)
|
|
234
|
+
|
|
235
|
+
ctx.tools.register(
|
|
236
|
+
defineTool({
|
|
237
|
+
name: 'scnet_notebook_list',
|
|
238
|
+
description: '只读列出目标区域的 Notebook 实例;密码和带凭据的 URL 默认脱敏。',
|
|
239
|
+
parameters: {
|
|
240
|
+
region: { type: 'string', description: '区域名称或 ID;省略时使用默认区域' },
|
|
241
|
+
status: { type: 'string', description: '可选状态过滤,如 Running、Terminated、Failed' },
|
|
242
|
+
},
|
|
243
|
+
output: {
|
|
244
|
+
schema: { type: 'string' },
|
|
245
|
+
render: (_args, value) => textBlock(value),
|
|
246
|
+
},
|
|
247
|
+
async execute(args) {
|
|
248
|
+
const argv = ['--backend', 'openapi', 'notebook', 'list']
|
|
249
|
+
if (args.region) argv.push('--region', String(args.region))
|
|
250
|
+
if (args.status) argv.push('--status', String(args.status))
|
|
251
|
+
return scnetResult(await runScnet(argv), '查询 Notebook 实例')
|
|
252
|
+
},
|
|
253
|
+
}),
|
|
254
|
+
)
|
|
255
|
+
|
|
256
|
+
ctx.tools.register(
|
|
257
|
+
defineTool({
|
|
258
|
+
name: 'scnet_notebook_show',
|
|
259
|
+
description: '只读查询一个 Notebook 实例详情;敏感字段默认脱敏。',
|
|
260
|
+
parameters: {
|
|
261
|
+
notebook_id: { type: 'string', required: true, description: 'Notebook 实例 ID' },
|
|
262
|
+
region: { type: 'string', description: '区域名称或 ID;省略时使用默认区域' },
|
|
263
|
+
},
|
|
264
|
+
output: {
|
|
265
|
+
schema: { type: 'string' },
|
|
266
|
+
render: (_args, value) => textBlock(value),
|
|
267
|
+
},
|
|
268
|
+
async execute(args) {
|
|
269
|
+
const id = String(args.notebook_id || '').trim()
|
|
270
|
+
if (!id) return '缺少必填参数 notebook_id。'
|
|
271
|
+
const argv = ['--backend', 'openapi', 'notebook', 'show', id]
|
|
272
|
+
if (args.region) argv.push('--region', String(args.region))
|
|
273
|
+
return scnetResult(await runScnet(argv), '查询 Notebook 详情')
|
|
274
|
+
},
|
|
275
|
+
}),
|
|
276
|
+
)
|
|
277
|
+
|
|
94
278
|
ctx.tools.register(
|
|
95
279
|
defineTool({
|
|
96
280
|
name: 'scnet_show_cluster',
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "dsh-scnet",
|
|
3
|
-
"version": "0.
|
|
4
|
-
"description": "Community DeepSeek Harness bundle for SCNet
|
|
3
|
+
"version": "0.5.0-rc.1",
|
|
4
|
+
"description": "Community DeepSeek Harness bundle for SCNet SSH and OpenAPI jobs, files, Notebook resources, Slurm diagnostics, and Hygon DCU/DTK guidance.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "./index.mjs",
|
|
7
7
|
"exports": {
|
|
@@ -9,8 +9,16 @@
|
|
|
9
9
|
"./cordis.patch.yml": "./cordis.patch.yml",
|
|
10
10
|
"./package.json": "./package.json"
|
|
11
11
|
},
|
|
12
|
-
"
|
|
13
|
-
"@deepseek-ai/dsh-tools": "^0.1.0-rc.8"
|
|
12
|
+
"peerDependencies": {
|
|
13
|
+
"@deepseek-ai/dsh-tools": "^0.1.0-rc.8 || ^0.1.5-rc.3"
|
|
14
|
+
},
|
|
15
|
+
"peerDependenciesMeta": {
|
|
16
|
+
"@deepseek-ai/dsh-tools": {
|
|
17
|
+
"optional": true
|
|
18
|
+
}
|
|
19
|
+
},
|
|
20
|
+
"devDependencies": {
|
|
21
|
+
"@deepseek-ai/dsh-tools": "^0.1.5-rc.3"
|
|
14
22
|
},
|
|
15
23
|
"repository": {
|
|
16
24
|
"type": "git",
|
|
@@ -43,6 +51,8 @@
|
|
|
43
51
|
"dsh-plugin",
|
|
44
52
|
"dsh-bundle",
|
|
45
53
|
"slurm",
|
|
54
|
+
"openapi",
|
|
55
|
+
"notebook",
|
|
46
56
|
"hpc",
|
|
47
57
|
"hygon",
|
|
48
58
|
"dcu",
|
|
@@ -1,111 +1,69 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: scnet-hpc
|
|
3
|
-
description: Configure
|
|
3
|
+
description: Configure or operate SCNet HPC through SSH, OpenAPI, and pluggable future backends.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# SCNet HPC
|
|
7
7
|
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
-
|
|
61
|
-
|
|
62
|
-
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
requested memory within `cpus-per-task × DEF_MEM_PER_CPU`. Use `scripts/new-job.sh` to
|
|
71
|
-
calculate a conservative value.
|
|
72
|
-
3. **Framework validation belongs on compute nodes.** A login node may lack accelerator
|
|
73
|
-
libraries even after modules are loaded. Use `srun`, `sbatch`, or the compute probe for
|
|
74
|
-
imports, kernels, collectives, and accelerator capabilities.
|
|
75
|
-
4. **Offline nodes require staged dependencies.** When `COMPUTE_NODE_OFFLINE=yes`, download or
|
|
76
|
-
build dependencies on a networked node and disable runtime downloads in the job.
|
|
77
|
-
5. **Test files use shared storage.** Create per-job temporary directories under
|
|
78
|
-
`$HOME/.scnet-hpc/tmp/$SLURM_JOB_ID`; do not depend on `/tmp` being shared across nodes.
|
|
79
|
-
6. **Job exit status must propagate.** Capture the workload return code and terminate the batch
|
|
80
|
-
script with `exit "$rc"`. Evaluate both accounting state and job logs.
|
|
81
|
-
7. **Compatibility claims require target-node evidence.** Package resolution or import success
|
|
82
|
-
alone is insufficient for native extensions and accelerator software. Validate the relevant
|
|
83
|
-
ABI, import, kernel, functional, and end-to-end layers.
|
|
84
|
-
|
|
85
|
-
## Repository tools
|
|
86
|
-
|
|
87
|
-
| Tool | Purpose |
|
|
88
|
-
|---|---|
|
|
89
|
-
| `scripts/setup-ssh.sh` | Install a supplied key and add an SSH host entry |
|
|
90
|
-
| `scripts/new-job.sh` | Generate a profile-aware Slurm script |
|
|
91
|
-
| `scripts/probe-cluster.sh` | Produce an initial profile from read-only login-node probes |
|
|
92
|
-
| `scripts/refresh-cluster.sh` | Refresh dynamic profile fields; compute probing is opt-in |
|
|
93
|
-
| `scripts/run-compute-probe.sh` | Submit a minimal accelerator capability probe |
|
|
94
|
-
|
|
95
|
-
Use `scripts/new-job.sh --cluster <name> ...` for routine job generation. Validate the generated
|
|
96
|
-
script with `sbatch --test-only` before submission when the target cluster supports it.
|
|
97
|
-
|
|
98
|
-
## Evidence and reporting
|
|
99
|
-
|
|
100
|
-
Distinguish:
|
|
101
|
-
|
|
102
|
-
- profile values;
|
|
103
|
-
- live scheduler observations;
|
|
104
|
-
- login-node observations;
|
|
105
|
-
- compute-node observations;
|
|
106
|
-
- conclusions inferred from those observations.
|
|
107
|
-
|
|
108
|
-
For failures, report the failing command, relevant log excerpt, scheduler state and exit code,
|
|
109
|
-
target profile, and the next bounded diagnostic action. For compatibility results, state the
|
|
110
|
-
software and toolchain versions, accelerator architecture, validation layer reached, and
|
|
111
|
-
untested boundaries.
|
|
8
|
+
Use the selected backend, cluster profile, and target-cluster evidence. SSH is the default;
|
|
9
|
+
OpenAPI is the structured control plane. Never silently switch backends.
|
|
10
|
+
|
|
11
|
+
## First use and selection
|
|
12
|
+
|
|
13
|
+
- Run `scripts/setup.sh new|modify|status|reset` for explicit configuration lifecycle.
|
|
14
|
+
- Advanced OpenAPI discovery: `python3 scripts/scnet.py setup`.
|
|
15
|
+
- Read-only preflight: `python3 scripts/scnet.py doctor`.
|
|
16
|
+
- Backend precedence: `--backend`, `SCNET_HPC_BACKEND`, saved setup choice, profile
|
|
17
|
+
`DEFAULT_BACKEND`, then `ssh`.
|
|
18
|
+
- Read `clusters/<cluster>.conf` before reporting static facts. Treat
|
|
19
|
+
`clusters/.cache/<cluster>.auto.conf` as optional, time-sensitive SSH probe data.
|
|
20
|
+
- With multiple profiles, require an explicit cluster before access changes, installation,
|
|
21
|
+
job generation, or submission. Query the selected backend for live queues, limits, and state.
|
|
22
|
+
- SSH setup is profile-scoped; record usernames and key expiry metadata per profile. OpenAPI
|
|
23
|
+
credentials are platform-scoped and discover all authorized regions/schedulers automatically.
|
|
24
|
+
- OpenAPI may enable multiple regions locally, but every mutating operation must resolve to one
|
|
25
|
+
explicit/default target region.
|
|
26
|
+
|
|
27
|
+
Read [references/backends.md](references/backends.md) for backend selection and extension.
|
|
28
|
+
Read [references/openapi.md](references/openapi.md) for OpenAPI credentials and limits.
|
|
29
|
+
|
|
30
|
+
## Route only as needed
|
|
31
|
+
|
|
32
|
+
- SSH access: `references/setup.md`
|
|
33
|
+
- Python/modules/offline dependencies: `references/environment.md`
|
|
34
|
+
- Scheduler/runtime/accelerator diagnosis: `references/troubleshooting.md`
|
|
35
|
+
- Add or refresh profiles: `references/adding-cluster.md`
|
|
36
|
+
- Hygon DCU/DTK/HIP/profiling: `references/hygon-dcu-development.md`
|
|
37
|
+
- Compatibility reports: `references/software-compatibility.md`
|
|
38
|
+
- English instructions: `references/quickstart-en.md`
|
|
39
|
+
|
|
40
|
+
Read only the selected profile and references needed for the current operation. Profile values
|
|
41
|
+
override examples.
|
|
42
|
+
|
|
43
|
+
## Boundaries and evidence
|
|
44
|
+
|
|
45
|
+
- Read-only inspection and local generation do not authorize remote mutation.
|
|
46
|
+
- Resolve backend, target, paths, command, and expected effect before submit, cancel, upload,
|
|
47
|
+
download, overwrite, deletion, or SSH configuration. Never silently retry submission.
|
|
48
|
+
- Compute probes consume scheduler resources; run them only when requested or required for
|
|
49
|
+
target-node evidence.
|
|
50
|
+
- Keep credentials, tokens, personal paths, internal hosts, nodes, job IDs, and private mirrors
|
|
51
|
+
out of profiles and public reports.
|
|
52
|
+
- Preserve existing SSH configuration; use `scripts/setup-ssh.sh` for bounded changes.
|
|
53
|
+
- Respect profile-specific partition, GRES, memory, walltime, offline-network, and shared-storage
|
|
54
|
+
constraints. Propagate workload exit status.
|
|
55
|
+
- Distinguish profile values, live scheduler state, login-node evidence, compute-node evidence,
|
|
56
|
+
and inference. Treat Hygon guidance as architecture/toolchain-specific.
|
|
57
|
+
|
|
58
|
+
## Stable entrypoints
|
|
59
|
+
|
|
60
|
+
- `scripts/setup.sh`: first-use configuration panel.
|
|
61
|
+
- `python3 scripts/scnet.py ...`: backend-neutral operations, dry-run, raw output, and doctor.
|
|
62
|
+
- `python3 scripts/scnet.py notebook ...`: Notebook discovery and confirmation-driven
|
|
63
|
+
lifecycle operations. Use `--dry-run` before create/start/stop/release.
|
|
64
|
+
- `scripts/new-job.sh`: profile-aware Slurm generation.
|
|
65
|
+
- `scripts/refresh-cluster.sh`: SSH-derived profile refresh.
|
|
66
|
+
- `scripts/probe-cluster.sh`: initial profile creation.
|
|
67
|
+
- `scripts/run-compute-probe.sh`: bounded accelerator probe.
|
|
68
|
+
|
|
69
|
+
Use compact normalized output by default. Request raw backend data only for diagnosis.
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
0.5.0-rc.1
|
|
@@ -5,13 +5,14 @@
|
|
|
5
5
|
# 2. 填下面的值(不确定的用 scripts/probe-cluster.sh 探测)
|
|
6
6
|
# 3. scripts/setup-ssh.sh --cluster <集群短名> <私钥文件> [用户名]
|
|
7
7
|
#
|
|
8
|
-
# CLUSTER_ID
|
|
9
|
-
#
|
|
10
|
-
# _common.sh
|
|
8
|
+
# CLUSTER_ID 是 profile 标识。SSH backend 还要求 SSH_HOST / SSH_PORT;OpenAPI backend
|
|
9
|
+
# 要求 OPENAPI_REGION_ID,并在多调度器区域填写 OPENAPI_SCHEDULER_ID。其余可留空。
|
|
10
|
+
# 旧 SSH profile 漏填 SSH_HOST 时,_common.sh 会从本机 ssh config 尝试补全。
|
|
11
11
|
|
|
12
|
-
# ----
|
|
12
|
+
# ---- 通用与 SSH 连接 ----
|
|
13
13
|
CLUSTER_ID="" # 短名,同时用作 ssh 别名,如 "zzeshell"
|
|
14
14
|
CLUSTER_DESC="" # 人类可读描述,如 "海光 DCU(郑州)"
|
|
15
|
+
DEFAULT_BACKEND="ssh" # ssh / openapi / 已注册的外部 backend
|
|
15
16
|
SSH_HOST="" # 如 "xxx.scnet.cn"
|
|
16
17
|
SSH_PORT="22"
|
|
17
18
|
HOME_BASE="/public/home" # 家目录父路径
|
|
@@ -19,6 +20,11 @@ HOME_BASE="/public/home" # 家目录父路径
|
|
|
19
20
|
KEY_NAME_MARKER="" # 私钥文件名里的主机标识,如 "_xxx.scnet.cn_"
|
|
20
21
|
NEEDS_UPDATE_HOSTKEYS_NO="no" # 平台主机密钥签名有问题时设 yes
|
|
21
22
|
|
|
23
|
+
# ---- OpenAPI(可选,不得写入凭据)----
|
|
24
|
+
OPENAPI_REGION_ID="" # SCNet 授权区域 ID
|
|
25
|
+
OPENAPI_SCHEDULER_ID="" # 区域存在多个调度器时填写
|
|
26
|
+
OPENAPI_USERNAME="" # 通常由区域信息自动返回
|
|
27
|
+
|
|
22
28
|
# ---- 调度器 ----
|
|
23
29
|
SCHEDULER="slurm"
|
|
24
30
|
PARTITION=""
|