dsh-scnet 0.4.1 → 0.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (38) hide show
  1. package/README.md +20 -113
  2. package/README_CN.md +20 -1
  3. package/docs/installation.md +10 -0
  4. package/docs/installation.zh-CN.md +13 -1
  5. package/docs/testing.md +11 -0
  6. package/docs/testing.zh-CN.md +10 -0
  7. package/index.mjs +184 -0
  8. package/package.json +4 -2
  9. package/skills/scnet-hpc/SKILL.md +63 -105
  10. package/skills/scnet-hpc/VERSION +1 -0
  11. package/skills/scnet-hpc/clusters/_template.conf +10 -4
  12. package/skills/scnet-hpc/references/adding-cluster.md +6 -1
  13. package/skills/scnet-hpc/references/backends.md +177 -0
  14. package/skills/scnet-hpc/references/notebook.md +60 -0
  15. package/skills/scnet-hpc/references/openapi.md +140 -0
  16. package/skills/scnet-hpc/references/quickstart-en.md +22 -0
  17. package/skills/scnet-hpc/references/setup.md +4 -0
  18. package/skills/scnet-hpc/scripts/__pycache__/compute-probe.cpython-312.pyc +0 -0
  19. package/skills/scnet-hpc/scripts/__pycache__/scnet.cpython-312.pyc +0 -0
  20. package/skills/scnet-hpc/scripts/__pycache__/scnet_config.cpython-312.pyc +0 -0
  21. package/skills/scnet-hpc/scripts/__pycache__/scnet_credentials.cpython-312.pyc +0 -0
  22. package/skills/scnet-hpc/scripts/__pycache__/scnet_version.cpython-312.pyc +0 -0
  23. package/skills/scnet-hpc/scripts/scnet.py +1573 -0
  24. package/skills/scnet-hpc/scripts/scnet_backends/__init__.py +53 -0
  25. package/skills/scnet-hpc/scripts/scnet_backends/base.py +64 -0
  26. package/skills/scnet-hpc/scripts/scnet_backends/external.py +92 -0
  27. package/skills/scnet-hpc/scripts/scnet_backends/openapi.py +771 -0
  28. package/skills/scnet-hpc/scripts/scnet_backends/profile.py +78 -0
  29. package/skills/scnet-hpc/scripts/scnet_backends/ssh.py +258 -0
  30. package/skills/scnet-hpc/scripts/scnet_config.py +175 -0
  31. package/skills/scnet-hpc/scripts/scnet_credentials.py +160 -0
  32. package/skills/scnet-hpc/scripts/scnet_sdk/__init__.py +11 -0
  33. package/skills/scnet-hpc/scripts/scnet_sdk/cli.py +394 -0
  34. package/skills/scnet-hpc/scripts/scnet_sdk/client.py +240 -0
  35. package/skills/scnet-hpc/scripts/scnet_sdk/notebook.py +379 -0
  36. package/skills/scnet-hpc/scripts/scnet_version.py +20 -0
  37. package/skills/scnet-hpc/scripts/setup-ssh.sh +33 -1
  38. package/skills/scnet-hpc/scripts/setup.sh +471 -0
@@ -1,111 +1,69 @@
1
1
  ---
2
2
  name: scnet-hpc
3
- description: Configure and operate SCNet HPC clusters through SSH and Slurm, including profile-based resource requests, job submission and diagnosis, environment preparation, and accelerator compatibility checks. Use for SCNet access, Slurm CPU/DCU jobs, Hygon DCU/DTK, offline compute nodes, or adding and verifying cluster profiles. Treat other accelerator platforms as unverified until tested on their target compute nodes.
3
+ description: Configure or operate SCNet HPC through SSH, OpenAPI, and pluggable future backends.
4
4
  ---
5
5
 
6
6
  # SCNet HPC
7
7
 
8
- Operate SCNet clusters from repository profiles and target-cluster evidence. The included
9
- accelerator guidance is validated primarily for Hygon DCU/DTK and `gfx9xx`; do not transfer
10
- those capability conclusions to other accelerator stacks. The repository scripts support Linux
11
- and macOS natively. On Windows, use WSL2; native Windows execution is not supported.
12
-
13
- ## 中文说明
14
-
15
- 本技能用于通过 SSH 和 Slurm 操作 SCNet 超算集群,覆盖基于 profile 的资源申请、作业生成与诊断、环境准备,以及加速器兼容性验证。Linux 和 macOS 可直接运行仓库脚本;Windows 不提供原生脚本入口,建议使用 WSL2。官方 SCNet 客户端支持 Windows 10 及以上、macOS 12 Monterey 及以上(ARM/x86),下载地址见 [SCNet 客户端下载页](https://www.scnet.cn/ui/mall/client/download)。
16
-
17
- ## Source of truth
18
-
19
- - Read `clusters/<cluster>.conf` before reporting connection, partition, memory, hardware,
20
- module, network, or known-limit information.
21
- - Treat `clusters/.cache/<cluster>.auto.conf` as optional, time-sensitive probe data that
22
- overrides matching profile fields.
23
- - Verify dynamic scheduler state on the target cluster when the task depends on current
24
- availability, limits, or node count.
25
- - Do not infer one cluster's parameters from another cluster.
26
-
27
- List available profiles with:
28
-
29
- ```bash
30
- ls clusters/*.conf
31
- ```
32
-
33
- If more than one profile exists, require an explicit cluster selection before generating a
34
- job, changing SSH configuration, submitting work, or installing software.
35
-
36
- ## Route the task
37
-
38
- Read only the references required for the current operation:
39
-
40
- | Operation | Reference |
41
- |---|---|
42
- | Configure or repair SSH access | `references/setup.md` |
43
- | Prepare Python, modules, virtual environments, or offline dependencies | `references/environment.md` |
44
- | Diagnose Slurm, runtime, library, network, or accelerator failures | `references/troubleshooting.md` |
45
- | Add or refresh a cluster profile | `references/adding-cluster.md` |
46
- | Work with Hygon DCU, DTK, HIP libraries, profiling, migration, or containers | `references/hygon-dcu-development.md` |
47
- | Evaluate software compatibility or prepare a public compatibility report | `references/software-compatibility.md` |
48
- | Provide English operating instructions | `references/quickstart-en.md` |
49
-
50
- Cluster parameters always come from the selected profile, including when an example in a
51
- reference uses different values.
52
-
53
- ## Execution boundaries
54
-
55
- - Read-only inspection and local script generation do not authorize SSH configuration changes,
56
- dependency installation, job submission, job cancellation, or remote file deletion.
57
- - Before a remote mutation, resolve the target cluster, command, paths, and expected effect.
58
- - A compute-node probe consumes scheduler resources. Run it only when the user requests
59
- submission or verification that requires compute-node evidence.
60
- - Preserve existing SSH configuration. `scripts/setup-ssh.sh` adds a missing host entry and
61
- backs up a conflicting destination key; it does not rewrite an existing host block.
62
- - Do not place credentials, usernames, internal hosts, node names, job IDs, private mirrors, or
63
- absolute home paths in committed profiles or public reports.
64
-
65
- ## Operational invariants
66
-
67
- 1. **Scheduler constraints are profile-specific.** If `MIN_GRES` is non-empty, request at least
68
- that accelerator resource unless an independent CPU partition is selected.
69
- 2. **Memory requests follow the partition limit.** When `DEF_MEM_PER_CPU` is defined, keep
70
- requested memory within `cpus-per-task × DEF_MEM_PER_CPU`. Use `scripts/new-job.sh` to
71
- calculate a conservative value.
72
- 3. **Framework validation belongs on compute nodes.** A login node may lack accelerator
73
- libraries even after modules are loaded. Use `srun`, `sbatch`, or the compute probe for
74
- imports, kernels, collectives, and accelerator capabilities.
75
- 4. **Offline nodes require staged dependencies.** When `COMPUTE_NODE_OFFLINE=yes`, download or
76
- build dependencies on a networked node and disable runtime downloads in the job.
77
- 5. **Test files use shared storage.** Create per-job temporary directories under
78
- `$HOME/.scnet-hpc/tmp/$SLURM_JOB_ID`; do not depend on `/tmp` being shared across nodes.
79
- 6. **Job exit status must propagate.** Capture the workload return code and terminate the batch
80
- script with `exit "$rc"`. Evaluate both accounting state and job logs.
81
- 7. **Compatibility claims require target-node evidence.** Package resolution or import success
82
- alone is insufficient for native extensions and accelerator software. Validate the relevant
83
- ABI, import, kernel, functional, and end-to-end layers.
84
-
85
- ## Repository tools
86
-
87
- | Tool | Purpose |
88
- |---|---|
89
- | `scripts/setup-ssh.sh` | Install a supplied key and add an SSH host entry |
90
- | `scripts/new-job.sh` | Generate a profile-aware Slurm script |
91
- | `scripts/probe-cluster.sh` | Produce an initial profile from read-only login-node probes |
92
- | `scripts/refresh-cluster.sh` | Refresh dynamic profile fields; compute probing is opt-in |
93
- | `scripts/run-compute-probe.sh` | Submit a minimal accelerator capability probe |
94
-
95
- Use `scripts/new-job.sh --cluster <name> ...` for routine job generation. Validate the generated
96
- script with `sbatch --test-only` before submission when the target cluster supports it.
97
-
98
- ## Evidence and reporting
99
-
100
- Distinguish:
101
-
102
- - profile values;
103
- - live scheduler observations;
104
- - login-node observations;
105
- - compute-node observations;
106
- - conclusions inferred from those observations.
107
-
108
- For failures, report the failing command, relevant log excerpt, scheduler state and exit code,
109
- target profile, and the next bounded diagnostic action. For compatibility results, state the
110
- software and toolchain versions, accelerator architecture, validation layer reached, and
111
- untested boundaries.
8
+ Use the selected backend, cluster profile, and target-cluster evidence. SSH is the default;
9
+ OpenAPI is the structured control plane. Never silently switch backends.
10
+
11
+ ## First use and selection
12
+
13
+ - Run `scripts/setup.sh new|modify|status|reset` for explicit configuration lifecycle.
14
+ - Advanced OpenAPI discovery: `python3 scripts/scnet.py setup`.
15
+ - Read-only preflight: `python3 scripts/scnet.py doctor`.
16
+ - Backend precedence: `--backend`, `SCNET_HPC_BACKEND`, saved setup choice, profile
17
+ `DEFAULT_BACKEND`, then `ssh`.
18
+ - Read `clusters/<cluster>.conf` before reporting static facts. Treat
19
+ `clusters/.cache/<cluster>.auto.conf` as optional, time-sensitive SSH probe data.
20
+ - With multiple profiles, require an explicit cluster before access changes, installation,
21
+ job generation, or submission. Query the selected backend for live queues, limits, and state.
22
+ - SSH setup is profile-scoped; record usernames and key expiry metadata per profile. OpenAPI
23
+ credentials are platform-scoped and discover all authorized regions/schedulers automatically.
24
+ - OpenAPI may enable multiple regions locally, but every mutating operation must resolve to one
25
+ explicit/default target region.
26
+
27
+ Read [references/backends.md](references/backends.md) for backend selection and extension.
28
+ Read [references/openapi.md](references/openapi.md) for OpenAPI credentials and limits.
29
+
30
+ ## Route only as needed
31
+
32
+ - SSH access: `references/setup.md`
33
+ - Python/modules/offline dependencies: `references/environment.md`
34
+ - Scheduler/runtime/accelerator diagnosis: `references/troubleshooting.md`
35
+ - Add or refresh profiles: `references/adding-cluster.md`
36
+ - Hygon DCU/DTK/HIP/profiling: `references/hygon-dcu-development.md`
37
+ - Compatibility reports: `references/software-compatibility.md`
38
+ - English instructions: `references/quickstart-en.md`
39
+
40
+ Read only the selected profile and references needed for the current operation. Profile values
41
+ override examples.
42
+
43
+ ## Boundaries and evidence
44
+
45
+ - Read-only inspection and local generation do not authorize remote mutation.
46
+ - Resolve backend, target, paths, command, and expected effect before submit, cancel, upload,
47
+ download, overwrite, deletion, or SSH configuration. Never silently retry submission.
48
+ - Compute probes consume scheduler resources; run them only when requested or required for
49
+ target-node evidence.
50
+ - Keep credentials, tokens, personal paths, internal hosts, nodes, job IDs, and private mirrors
51
+ out of profiles and public reports.
52
+ - Preserve existing SSH configuration; use `scripts/setup-ssh.sh` for bounded changes.
53
+ - Respect profile-specific partition, GRES, memory, walltime, offline-network, and shared-storage
54
+ constraints. Propagate workload exit status.
55
+ - Distinguish profile values, live scheduler state, login-node evidence, compute-node evidence,
56
+ and inference. Treat Hygon guidance as architecture/toolchain-specific.
57
+
58
+ ## Stable entrypoints
59
+
60
+ - `scripts/setup.sh`: first-use configuration panel.
61
+ - `python3 scripts/scnet.py ...`: backend-neutral operations, dry-run, raw output, and doctor.
62
+ - `python3 scripts/scnet.py notebook ...`: Notebook discovery and confirmation-driven
63
+ lifecycle operations. Use `--dry-run` before create/start/stop/release.
64
+ - `scripts/new-job.sh`: profile-aware Slurm generation.
65
+ - `scripts/refresh-cluster.sh`: SSH-derived profile refresh.
66
+ - `scripts/probe-cluster.sh`: initial profile creation.
67
+ - `scripts/run-compute-probe.sh`: bounded accelerator probe.
68
+
69
+ Use compact normalized output by default. Request raw backend data only for diagnosis.
@@ -0,0 +1 @@
1
+ 0.5.0
@@ -5,13 +5,14 @@
5
5
  # 2. 填下面的值(不确定的用 scripts/probe-cluster.sh 探测)
6
6
  # 3. scripts/setup-ssh.sh --cluster <集群短名> <私钥文件> [用户名]
7
7
  #
8
- # CLUSTER_ID / SSH_HOST / SSH_PORT 是必填连接参数,应写死;其余可留空。
9
- # 留空的项脚本会跳过、文档里显示为「未探测」。旧 profile 漏填 SSH_HOST 时,
10
- # _common.sh 会临时从本机 ssh config 补全,但新集群仍建议在此显式填写。
8
+ # CLUSTER_ID 是 profile 标识。SSH backend 还要求 SSH_HOST / SSH_PORT;OpenAPI backend
9
+ # 要求 OPENAPI_REGION_ID,并在多调度器区域填写 OPENAPI_SCHEDULER_ID。其余可留空。
10
+ # 旧 SSH profile 漏填 SSH_HOST 时,_common.sh 会从本机 ssh config 尝试补全。
11
11
 
12
- # ---- 连接(必填)----
12
+ # ---- 通用与 SSH 连接 ----
13
13
  CLUSTER_ID="" # 短名,同时用作 ssh 别名,如 "zzeshell"
14
14
  CLUSTER_DESC="" # 人类可读描述,如 "海光 DCU(郑州)"
15
+ DEFAULT_BACKEND="ssh" # ssh / openapi / 已注册的外部 backend
15
16
  SSH_HOST="" # 如 "xxx.scnet.cn"
16
17
  SSH_PORT="22"
17
18
  HOME_BASE="/public/home" # 家目录父路径
@@ -19,6 +20,11 @@ HOME_BASE="/public/home" # 家目录父路径
19
20
  KEY_NAME_MARKER="" # 私钥文件名里的主机标识,如 "_xxx.scnet.cn_"
20
21
  NEEDS_UPDATE_HOSTKEYS_NO="no" # 平台主机密钥签名有问题时设 yes
21
22
 
23
+ # ---- OpenAPI(可选,不得写入凭据)----
24
+ OPENAPI_REGION_ID="" # SCNet 授权区域 ID
25
+ OPENAPI_SCHEDULER_ID="" # 区域存在多个调度器时填写
26
+ OPENAPI_USERNAME="" # 通常由区域信息自动返回
27
+
22
28
  # ---- 调度器 ----
23
29
  SCHEDULER="slurm"
24
30
  PARTITION=""
@@ -134,12 +134,17 @@ Triton 是多种推理框架的运行前提;不可用时,对应的 Triton
134
134
 
135
135
  ## profile 字段参考
136
136
 
137
- 必填三项:`CLUSTER_ID`、`SSH_HOST`、`SSH_PORT`。其余留空则脚本跳过、文档显示为未探测。
137
+ `CLUSTER_ID` 是 profile 标识。使用 SSH backend 时还需要 `SSH_HOST` 和 `SSH_PORT`;
138
+ 使用 OpenAPI backend 时需要 `OPENAPI_REGION_ID`,区域存在多个调度器时再填写
139
+ `OPENAPI_SCHEDULER_ID`。不得把 AK、SK 或 token 写进 profile。
138
140
 
139
141
  影响脚本行为的关键项:
140
142
 
141
143
  | 字段 | 谁用 | 留空的后果 |
142
144
  |---|---|---|
145
+ | `DEFAULT_BACKEND` | `scnet.py` | 默认使用 `ssh` |
146
+ | `OPENAPI_REGION_ID` | OpenAPI backend | 多区域账户需要每次显式传 `--region` |
147
+ | `OPENAPI_SCHEDULER_ID` | OpenAPI backend | 多调度器区域需要每次显式传 `--scheduler-id` |
143
148
  | `MIN_GRES` | 文档提示 | 不提示 QOS 强制申请加速器 |
144
149
  | `GRES_TYPE` | `new-job.sh` | 生成的脚本不含 `--gres` |
145
150
  | `DEF_MEM_PER_CPU` | `new-job.sh` | 生成的脚本不含 `--mem`,需手填 |
@@ -0,0 +1,177 @@
1
+ # SCNet backend selection and extension
2
+
3
+ The skill separates cluster intent from transport. SSH and OpenAPI are built in; future MCP
4
+ servers, platform connectors, and site-specific gateways can be added without changing the
5
+ task-level commands.
6
+
7
+ ## Selection
8
+
9
+ Backend precedence:
10
+
11
+ 1. `--backend <name>`;
12
+ 2. `SCNET_HPC_BACKEND`;
13
+ 3. saved setup-panel `default_backend`;
14
+ 4. selected profile `DEFAULT_BACKEND`;
15
+ 5. `ssh`.
16
+
17
+ Do not silently fall back between backends. A failed OpenAPI request must not become an SSH
18
+ submission, and a failed SSH command must not be replayed through OpenAPI. Replaying mutations
19
+ can duplicate jobs or apply them to a different region.
20
+
21
+ List registered implementations and their declared capabilities:
22
+
23
+ ```bash
24
+ python3 scripts/scnet.py backends
25
+ python3 scripts/scnet.py --backend <name> capabilities
26
+ ```
27
+
28
+ Built-in capability summary:
29
+
30
+ | Domain / operation | SSH | OpenAPI |
31
+ |---|---:|---:|
32
+ | Local profiles / authorized regions | yes | yes |
33
+ | Queues and limits | yes | yes |
34
+ | Job status and logs | yes | yes |
35
+ | Submit and cancel | yes | yes |
36
+ | Upload and download | yes | yes |
37
+ | Create remote directory | no | yes |
38
+ | Arbitrary remote command | yes | no |
39
+ | Modules, builds, interactive diagnosis | yes | no |
40
+ | Notebook read-only discovery | no | yes |
41
+
42
+ Use OpenAPI for structured control-plane work. Use SSH for environment setup, compilation,
43
+ interactive inspection, full scheduler tooling, and operations not represented by a structured
44
+ backend.
45
+
46
+ ## Common CLI
47
+
48
+ On a new computer, start with the dependency-light Bash panel:
49
+
50
+ ```bash
51
+ ./scripts/setup.sh
52
+ python3 scripts/scnet.py doctor
53
+ ```
54
+
55
+ Configuration lifecycle:
56
+
57
+ ```bash
58
+ ./scripts/setup.sh new
59
+ ./scripts/setup.sh modify
60
+ ./scripts/setup.sh status
61
+ ./scripts/setup.sh reset
62
+ ```
63
+
64
+ `new` adds a connection without replacing an existing profile/account. `modify` preserves
65
+ unrelated configuration. `status` is read-only. `reset` is scoped and does not remove real SSH
66
+ keys or `~/.ssh/config` unless that is performed separately and explicitly.
67
+
68
+ Use `./scripts/setup.sh --skip-connect` when only local selections should be saved. The Bash
69
+ panel can be rerun to update a profile's SSH user/key metadata. OpenAPI uses one platform
70
+ credential set and discovers regions, users, schedulers, and home paths automatically; no region
71
+ ID is required during setup.
72
+ Developers
73
+ can use `python3 scripts/scnet.py setup` for live OpenAPI region discovery. Both panels store no
74
+ secrets; provide OpenAPI credentials through a host credential manager or environment injection.
75
+
76
+ Region selection is multi-select for local enablement, followed by a single default region.
77
+ Mutating operations remain single-region by design.
78
+
79
+ Global options must appear before the operation:
80
+
81
+ ```bash
82
+ python3 scripts/scnet.py --backend ssh --cluster <profile> queues
83
+ python3 scripts/scnet.py --backend openapi --region <region-id> limits
84
+ python3 scripts/scnet.py --backend openapi --region <region-id> job <job-id>
85
+ ```
86
+
87
+ Use `--dry-run` before a mutation to inspect the normalized action without contacting the
88
+ backend:
89
+
90
+ ```bash
91
+ python3 scripts/scnet.py --backend openapi --dry-run \
92
+ submit --name smoke --queue debug --work-dir /absolute/work/dir \
93
+ --command 'echo hello'
94
+ ```
95
+
96
+ Mutating examples:
97
+
98
+ ```bash
99
+ python3 scripts/scnet.py --backend ssh --cluster <profile> \
100
+ submit --remote-path /absolute/path/job.slurm
101
+
102
+ python3 scripts/scnet.py --backend openapi --region <region-id> submit \
103
+ --name probe --queue debug --work-dir /absolute/work/dir \
104
+ --command 'python3 probe.py' --dcus 1 --cpus 8 --walltime 00:10:00
105
+ ```
106
+
107
+ The CLI does not interpret a command as authorization. Follow the skill's mutation boundaries
108
+ before invoking submit, cancel, upload, download, or `exec`.
109
+
110
+ ## External backend protocol
111
+
112
+ An external backend is an executable named:
113
+
114
+ ```text
115
+ scnet-hpc-backend-<name>
116
+ ```
117
+
118
+ Discover it either by:
119
+
120
+ - placing it in a directory listed by `SCNET_HPC_BACKEND_PATH`; or
121
+ - placing it on `PATH` and listing `<name>` in `SCNET_HPC_EXTERNAL_BACKENDS`.
122
+
123
+ The router writes one JSON request to standard input:
124
+
125
+ ```json
126
+ {
127
+ "protocol": "scnet-hpc.backend",
128
+ "protocol_version": 1,
129
+ "operation": "queues",
130
+ "profile_name": "example",
131
+ "profile": {},
132
+ "options": {}
133
+ }
134
+ ```
135
+
136
+ The executable returns one JSON object:
137
+
138
+ ```json
139
+ {
140
+ "ok": true,
141
+ "protocol_version": 1,
142
+ "data": []
143
+ }
144
+ ```
145
+
146
+ It must implement the `capabilities` operation and return:
147
+
148
+ ```json
149
+ {"ok": true, "protocol_version": 1, "data": {"capabilities": ["clusters", "queues"]}}
150
+ ```
151
+
152
+ Return `{"ok": false, "error": "..."}` for a bounded failure. Never print credentials or tokens
153
+ to standard output or standard error.
154
+
155
+ ## MCP and connector integration
156
+
157
+ If an official SCNet MCP server becomes available, prefer its native tools when the host exposes
158
+ them and the user has selected or authorized that connection. Keep the same normalized operation
159
+ semantics: clusters, queues, limits, job, logs, submit, cancel, files, upload, and download.
160
+
161
+ For hosts that cannot expose MCP tools directly, use a small
162
+ `scnet-hpc-backend-mcp` bridge implementing the protocol above. Platform connectors can use the
163
+ same adapter pattern. Do not hard-code speculative MCP tool names, schemas, or authentication
164
+ until the official interface is available and verified.
165
+
166
+ ## Profiles
167
+
168
+ Profiles may contain non-secret backend hints:
169
+
170
+ ```bash
171
+ DEFAULT_BACKEND="ssh"
172
+ OPENAPI_REGION_ID=""
173
+ OPENAPI_SCHEDULER_ID=""
174
+ OPENAPI_USERNAME=""
175
+ ```
176
+
177
+ Keep AK, SK, tokens, cookies, and passwords out of profiles.
@@ -0,0 +1,60 @@
1
+ # Notebook service
2
+
3
+ Notebook is a separate SCNet OpenAPI service domain. It reuses platform authentication and
4
+ region discovery, but it is not a Slurm job and does not use `hpcUrls`.
5
+
6
+ Read-only commands:
7
+
8
+ ```bash
9
+ python3 scripts/scnet.py --backend openapi notebook regions
10
+ python3 scripts/scnet.py --backend openapi --region <id> notebook resources
11
+ python3 scripts/scnet.py --backend openapi --region <id> notebook images
12
+ python3 scripts/scnet.py --backend openapi --region <id> notebook list
13
+ python3 scripts/scnet.py --backend openapi --region <id> notebook show <notebook-id>
14
+ python3 scripts/scnet.py --backend openapi --region <id> notebook url <notebook-id>
15
+ ```
16
+
17
+ The default output redacts SSH passwords and URL query credentials. Use `--reveal` only when
18
+ the user explicitly needs a URL to open, and do not place the resulting value in reports or
19
+ long-lived logs.
20
+
21
+ Lifecycle commands:
22
+
23
+ ```bash
24
+ python3 scripts/scnet.py --dry-run --backend openapi \
25
+ notebook create --region <id>
26
+ python3 scripts/scnet.py --backend openapi \
27
+ notebook create --region <id>
28
+ python3 scripts/scnet.py --backend openapi \
29
+ notebook start <id> --region <id>
30
+ python3 scripts/scnet.py --backend openapi \
31
+ notebook stop <id> --region <id>
32
+ python3 scripts/scnet.py --backend openapi \
33
+ notebook release <id> --region <id>
34
+ ```
35
+
36
+ `create` automatically chooses the smallest currently available resource and the smallest
37
+ trusted preset Jupyter image. Advanced users may override the resource group or image ID.
38
+
39
+ `smoke` performs a bounded lifecycle:
40
+
41
+ ```bash
42
+ python3 scripts/scnet.py --dry-run --backend openapi \
43
+ notebook smoke --region <id>
44
+ python3 scripts/scnet.py --backend openapi \
45
+ notebook smoke --region <id>
46
+ ```
47
+
48
+ It creates one minimal Notebook, waits for `Running`, validates the redacted Jupyter URL, then
49
+ stops and releases the instance in a cleanup block.
50
+
51
+ Mutations require interactive confirmation. Non-interactive callers must pass `--yes`. Release
52
+ requires the Notebook ID in interactive mode because it is irreversible.
53
+
54
+ Official API sections:
55
+
56
+ - Notebook resource discovery;
57
+ - image discovery;
58
+ - instance list/detail;
59
+ - Jupyter URL lookup;
60
+ - instance lifecycle.
@@ -0,0 +1,140 @@
1
+ # SCNet OpenAPI backend
2
+
3
+ This backend follows the official SCNet OpenAPI 2.0 documentation reviewed on
4
+ 2026-09-26:
5
+
6
+ - Overview: <https://www.scnet.cn/ac/openapi/doc/2.0/index.html>
7
+ - Recommended AK/SK authentication:
8
+ <https://www.scnet.cn/ac/openapi/doc/2.0/api/safecertification/get-user-tokens-aksk.html>
9
+ - Job submission:
10
+ <https://www.scnet.cn/ac/openapi/doc/2.0/api/jobmanager/job.html>
11
+ - Creating a directory:
12
+ <https://www.scnet.cn/ac/openapi/doc/2.0/api/efile/folder-create.html>
13
+ - Chunked upload and merge:
14
+ <https://www.scnet.cn/ac/openapi/doc/2.0/api/efile/chunk-upload.html>
15
+ and <https://www.scnet.cn/ac/openapi/doc/2.0/api/efile/merge-file.html>
16
+
17
+ The local implementation intentionally normalizes large responses. Use `--raw` only when the
18
+ compact result is insufficient.
19
+
20
+ ## Credentials
21
+
22
+ Obtain AK/SK from the SCNet website:
23
+
24
+ 1. Sign in to SCNet.
25
+ 2. Open **Personal Center / 个人中心**.
26
+ 3. Open **Access Control / 访问控制**.
27
+ 4. Generate and download the authorization file containing AccessKey and SecretKey.
28
+
29
+ Official instructions:
30
+ <https://www.scnet.cn/ac/openapi/doc/2.0/api/safecertification/get-user-tokens-aksk.html>
31
+
32
+ Preferred environment variables:
33
+
34
+ ```bash
35
+ export SCNET_OPENAPI_USER="<platform-user>"
36
+ export SCNET_OPENAPI_ACCESS_KEY="<access-key>"
37
+ export SCNET_OPENAPI_SECRET_KEY="<secret-key>"
38
+ ```
39
+
40
+ The client signs the canonical AK/timestamp/user JSON with HMAC-SHA256 and obtains a fresh
41
+ per-region token for each invocation. It does not persist tokens. The setup panel can store the
42
+ platform credential set in macOS Keychain or Linux Secret Service; otherwise use environment
43
+ variables. Region IDs, region users, schedulers, and home paths are discovered and cached as
44
+ non-secret metadata.
45
+
46
+ Interactive terminals support ↑/↓ plus Enter and direct numeric selection. Press Enter without
47
+ typing a number to accept the current/default item. SecretKey input is intentionally invisible.
48
+ This behavior is the same in macOS Terminal/iTerm2 and Ubuntu/Debian terminals.
49
+ Ubuntu/Debian users can install `secret-tool` with `sudo apt install libsecret-tools`.
50
+
51
+ For an externally managed token:
52
+
53
+ ```bash
54
+ export SCNET_OPENAPI_TOKEN="<region-token>"
55
+ export SCNET_OPENAPI_REGION_ID="<region-id>"
56
+ ```
57
+
58
+ Optional overrides:
59
+
60
+ ```bash
61
+ export SCNET_OPENAPI_AUTH_BASE="https://api.scnet.cn"
62
+ export SCNET_OPENAPI_CENTER_URL="https://www.scnet.cn/ac/openapi/v2/center"
63
+ export SCNET_OPENAPI_SCHEDULER_ID="<scheduler-id>"
64
+ export SCNET_OPENAPI_USERNAME="<region-user>"
65
+ ```
66
+
67
+ Never put these values in a prompt, profile, repository file, generated report, or shell trace.
68
+ Use the host's credential store or secret injection when available.
69
+
70
+ ## Read-only operations
71
+
72
+ ```bash
73
+ python3 scripts/scnet.py --backend openapi clusters
74
+ python3 scripts/scnet.py --backend openapi --region <id> queues
75
+ python3 scripts/scnet.py --backend openapi --region <id> limits
76
+ python3 scripts/scnet.py --backend openapi --region <id> job <job-id>
77
+ python3 scripts/scnet.py --backend openapi --region <id> \
78
+ logs --path /absolute/path/std.out.123 --lines 200
79
+ python3 scripts/scnet.py --backend openapi --region <id> \
80
+ files --path /absolute/path
81
+ python3 scripts/scnet.py --backend openapi --region <id> \
82
+ mkdir /absolute/work/dir --parents
83
+ ```
84
+
85
+ If a region exposes multiple schedulers, add `--scheduler-id`.
86
+
87
+ ## Mutations
88
+
89
+ OpenAPI submission uses the documented BASIC/BASE command mode. Extra scheduler directives can
90
+ be repeated:
91
+
92
+ ```bash
93
+ python3 scripts/scnet.py --backend openapi --region <id> submit \
94
+ --name probe --queue debug --work-dir /absolute/work/dir \
95
+ --command $'module load ...\npython3 probe.py' \
96
+ --nodes 1 --cpus 8 --dcus 1 --walltime 00:10:00 \
97
+ --scheduler-option '#SBATCH --account=example'
98
+ ```
99
+
100
+ Other mutations:
101
+
102
+ ```bash
103
+ python3 scripts/scnet.py --backend openapi --region <id> cancel <job-id>
104
+ python3 scripts/scnet.py --backend openapi --region <id> \
105
+ upload local.file /absolute/remote/directory
106
+ python3 scripts/scnet.py --backend openapi --region <id> \
107
+ download /absolute/remote/file local.file
108
+ ```
109
+
110
+ Downloads refuse to replace an existing local path unless `--cover` is supplied. Uploads send
111
+ `uncover` by default and require `--cover` for replacement.
112
+
113
+ Uploads larger than 8 MiB automatically use the documented burst/merge flow. Override the
114
+ threshold and chunk size with `--chunk-size <bytes>`; use `--chunk-size 0` to force ordinary
115
+ multipart upload. The current implementation keeps one chunk in memory at a time.
116
+
117
+ Preview a mutation without contacting the backend:
118
+
119
+ ```bash
120
+ python3 scripts/scnet.py --backend openapi --region <id> --dry-run \
121
+ submit --name probe --queue debug --work-dir /absolute/work/dir \
122
+ --command 'echo hello' --cpus 1 --dcus 1
123
+ ```
124
+
125
+ Do not automatically retry submission or cancellation after a timeout. Query job state first:
126
+ the public documentation does not establish an idempotency key for submission.
127
+
128
+ ## Known validation boundaries
129
+
130
+ - The implementation is validated locally for parsing, signing, routing, normalization, and the
131
+ external-backend contract. Live OpenAPI operations still require testing against each
132
+ authorized SCNet region.
133
+ - The official cancellation page describes one method value in prose and uses `5` in all request
134
+ examples. The client defaults to `5`; override it with `SCNET_OPENAPI_CANCEL_METHOD` if a live
135
+ region requires a different documented value.
136
+ - Small ordinary uploads buffer one file in memory; large uploads use chunked transfer.
137
+ - OpenAPI does not replace SSH for module discovery, environment construction, compilation,
138
+ interactive `srun`, performance profiling, or low-level scheduler diagnosis.
139
+ - API success is determined by both HTTP transport and the JSON `code`; HTTP 200 alone is not
140
+ sufficient.
@@ -19,6 +19,28 @@ Check at least `PARTITION`, `PARTITION_CPU`, `GRES_TYPE`, `MIN_GRES`,
19
19
  `DEF_MEM_PER_CPU`, `MODULE_LOADS`, `COMPUTE_NODE_OFFLINE`, and
20
20
  `KNOWN_LIMITATIONS`.
21
21
 
22
+ On a new computer, run the Bash setup panel and then its read-only doctor:
23
+
24
+ ```bash
25
+ ./scripts/setup.sh
26
+ python3 scripts/scnet.py doctor
27
+ ```
28
+
29
+ List available access backends:
30
+
31
+ ```bash
32
+ python3 scripts/scnet.py backends
33
+ ```
34
+
35
+ SSH is the default. Use OpenAPI for structured queue, limit, job, log, and file operations:
36
+
37
+ ```bash
38
+ python3 scripts/scnet.py --backend openapi clusters
39
+ python3 scripts/scnet.py --backend openapi --region <region-id> queues
40
+ ```
41
+
42
+ See `references/backends.md` and `references/openapi.md` before configuring another backend.
43
+
22
44
  ## 2. Install the skill
23
45
 
24
46
  ```bash
@@ -85,6 +85,10 @@ ssh -O exit <集群> # 手工断开(网络切换后卡住时用)
85
85
  `<用户名>_<主机>_RsaKeyExpireTime_2026-10-29_12-51-40.txt` 表示 2026-10-29 过期。
86
86
  到期前从控制台重新下载并重跑 setup 脚本。
87
87
 
88
+ 可以重复运行 `scripts/setup.sh` 更新某个 profile 的用户名和私钥。面板会从密钥文件名
89
+ 中的 `RsaKeyExpireTime_YYYY-MM-DD` 自动记录有效期,`python3 scripts/scnet.py doctor`
90
+ 会报告剩余天数和已过期状态。密钥内容不会进入配置文件。
91
+
88
92
  profile 里的 `KEY_NAME_MARKER` 就是用来从这种文件名里提取用户名的;公开
89
93
  `scnet-hpc` profile 使用 `_<CLUSTER_HOST>_` 占位符,所以安装时通常需要显式传
90
94
  用户名,或在本地私有 profile 中补齐真实标记。