dsh-scnet 0.6.5 → 0.6.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +9 -44
- package/README_CN.md +8 -38
- package/connector/contract.mjs +96 -0
- package/connector/index.mjs +21 -0
- package/connector/oauth2.mjs +263 -0
- package/index.mjs +34 -14
- package/package.json +5 -1
- package/skills/scnet-hpc/VERSION +1 -1
- package/skills/scnet-hpc/references/environment.md +14 -0
- package/skills/scnet-hpc/scripts/_common.sh +31 -0
- package/skills/scnet-hpc/scripts/new-job.sh +1 -0
- package/skills/scnet-hpc/scripts/setup.sh +5 -5
package/README.md
CHANGED
|
@@ -4,45 +4,19 @@
|
|
|
4
4
|
|
|
5
5
|
DSH-SCNet (`dsh-scnet` on npm) is a community-maintained DSH bundle for operating Supercomputing Network (SCNet). It packages the source [`scnet-hpc`](https://github.com/lql341/scnet-hpc) Skill, profile-aware utilities, and bounded tools for SSH, Slurm, OpenAPI files, regional resources, and Notebook discovery.
|
|
6
6
|
|
|
7
|
-
Current release: **0.6.
|
|
7
|
+
Current release: **0.6.7**
|
|
8
8
|
|
|
9
|
-
|
|
9
|
+
<!-- scnet-release:start -->
|
|
10
|
+
## 0.6.7 highlights
|
|
10
11
|
|
|
11
|
-
-
|
|
12
|
-
|
|
12
|
+
- Desktop hosts can inject a verified Python 3 interpreter through `DSH_SCNET_PYTHON` or
|
|
13
|
+
`SCNET_PYTHON`; local entrypoints no longer require the command name `python3`.
|
|
14
|
+
- Shared shell helpers reject Python 2 and invalid configured interpreters before running
|
|
15
|
+
OpenAPI or local job-generation code.
|
|
16
|
+
<!-- scnet-release:end -->
|
|
13
17
|
|
|
14
18
|
> This is an independent community project. It is compatible with DeepSeek Harness but is not an official DeepSeek product and does not imply endorsement, partnership, or authorization by DeepSeek.
|
|
15
19
|
|
|
16
|
-
## What's new in 0.6.3
|
|
17
|
-
|
|
18
|
-
- Added read-only account and resource summaries plus bounded `scnet_job_wait`.
|
|
19
|
-
- Submission queue preflight now fails closed and never silently selects a queue.
|
|
20
|
-
- Download paths can default to the remote filename; ordinary and chunked uploads return the same
|
|
21
|
-
directory/filename result shape.
|
|
22
|
-
|
|
23
|
-
## 0.6.2 highlights
|
|
24
|
-
|
|
25
|
-
- Completed jobs now resolve through the fast filtered history-list endpoint rather than timing
|
|
26
|
-
out on the empty history-detail response.
|
|
27
|
-
- Historical field aliases such as `workdir` are normalized for log-path discovery.
|
|
28
|
-
- OpenAPI operations continue without persistent caching when the cache directory is read-only.
|
|
29
|
-
|
|
30
|
-
## 0.6.1 highlights
|
|
31
|
-
|
|
32
|
-
- Job details now fall back to the history endpoint when realtime records expire, with complete
|
|
33
|
-
normalized terminal states.
|
|
34
|
-
- Added compact `scnet_job_list`, `scnet_limits`, and `scnet_file_transfer` tools.
|
|
35
|
-
- File uploads now make the remote-directory contract explicit: the local filename is preserved
|
|
36
|
-
and must not be passed as the remote directory.
|
|
37
|
-
|
|
38
|
-
## 0.6.0 highlights
|
|
39
|
-
|
|
40
|
-
- Added first-class DSH tools for submitting jobs, inspecting job status, reading logs, and
|
|
41
|
-
cancelling jobs through the existing SSH/OpenAPI backends.
|
|
42
|
-
- Added dry-run support and one-missing-decision-at-a-time validation for job lifecycle calls.
|
|
43
|
-
- Job submission reports the authoritative work directory and log paths; OpenAPI log lookup can
|
|
44
|
-
derive `std.out` and `std.err` paths from `job_id` and `work_dir`.
|
|
45
|
-
|
|
46
20
|
## Requirements
|
|
47
21
|
|
|
48
22
|
- Node.js 22.19 or later
|
|
@@ -130,16 +104,7 @@ The `skills/scnet-hpc/` directory is generated from the canonical `scnet-hpc` re
|
|
|
130
104
|
|
|
131
105
|
Do not maintain the generated directory independently. The sync excludes the canonical installer and local probe cache because neither belongs in the npm runtime package.
|
|
132
106
|
|
|
133
|
-
Release order is canonical-to-downstream: update `
|
|
134
|
-
synchronizes the skill and version into `dsh-scnet` and `codex-scnet-hpc`. Publish the matching
|
|
135
|
-
`dsh-scnet` npm package only after that downstream change is reviewed.
|
|
136
|
-
|
|
137
|
-
For a synchronized release, run the package validation on `main`, then trigger the repository's
|
|
138
|
-
`Publish DSH-SCNet` workflow with the exact current version from `package.json`. The
|
|
139
|
-
workflow verifies the package/Skill version match, runs validation, creates the tarball, publishes
|
|
140
|
-
stable releases with the `latest` tag, and creates the matching Git tag. Do not bump only
|
|
141
|
-
`package.json`: `skills/scnet-hpc/VERSION`, the package version, and both README release markers
|
|
142
|
-
must stay aligned.
|
|
107
|
+
Release order is canonical-to-downstream: update the canonical `VERSION` and release highlights once. CI synchronizes both distribution repositories. After each synchronized pull request passes validation, the workflow merges it automatically. The resulting `main` push validates and packs the bundle, publishes the npm package with the appropriate `latest` or `next` tag, and creates the matching Git tag and GitHub Release. Do not bump only `package.json`: the canonical `VERSION` remains the release source of truth.
|
|
143
108
|
|
|
144
109
|
The workflow uses npm Trusted Publishing through GitHub OIDC, so account 2FA/Passkey should remain
|
|
145
110
|
enabled and no long-lived `NPM_TOKEN` is required. Configure `lql341/dsh-scnet` and
|
package/README_CN.md
CHANGED
|
@@ -4,40 +4,18 @@
|
|
|
4
4
|
|
|
5
5
|
<p align="center">在 DeepSeek Harness 里操作超算互联网(SCNet):配置 SSH、管理 Slurm 作业、查询 OpenAPI 区域和文件、检查 Notebook 资源与实例,并保留海光 DCU/DTK 诊断能力。</p>
|
|
6
6
|
|
|
7
|
-
当前版本:**0.6.
|
|
7
|
+
当前版本:**0.6.7**
|
|
8
8
|
|
|
9
|
-
|
|
9
|
+
<!-- scnet-release:start -->
|
|
10
|
+
## 0.6.7 更新
|
|
10
11
|
|
|
11
|
-
-
|
|
12
|
-
|
|
12
|
+
- Desktop 主机可通过 `DSH_SCNET_PYTHON` 或 `SCNET_PYTHON` 注入经过校验的 Python 3
|
|
13
|
+
解释器,本地入口不再强制要求命令名为 `python3`。
|
|
14
|
+
- 共享 shell 辅助函数会在运行 OpenAPI 或本地作业生成前拒绝 Python 2 和无效解释器路径。
|
|
15
|
+
<!-- scnet-release:end -->
|
|
13
16
|
|
|
14
17
|
> 本项目是独立维护的社区项目,与 DeepSeek Harness 兼容,但不是 DeepSeek 官方产品,也不表示 DeepSeek 对本项目提供背书、合作或授权。
|
|
15
18
|
|
|
16
|
-
## 0.6.3 更新
|
|
17
|
-
|
|
18
|
-
- 新增只读账户摘要、资源摘要和有界等待作业工具 `scnet_job_wait`。
|
|
19
|
-
- 提交前队列预检失败会停止提交,不再静默选择队列。
|
|
20
|
-
- 下载时可省略本地路径自动使用远端文件名;普通上传和分片上传返回统一结果结构。
|
|
21
|
-
|
|
22
|
-
## 0.6.2 更新
|
|
23
|
-
|
|
24
|
-
- 已结束作业改用快速的历史列表过滤查询,不再卡在返回空数据的历史详情接口。
|
|
25
|
-
- 兼容历史记录中的 `workdir` 等字段变体,可继续推导日志路径。
|
|
26
|
-
- 缓存目录只读时自动降级为无持久缓存,OpenAPI 主功能不再被缓存故障阻断。
|
|
27
|
-
|
|
28
|
-
## 0.6.1 更新
|
|
29
|
-
|
|
30
|
-
- 作业实时记录过期时自动回退历史接口,并补全终态映射。
|
|
31
|
-
- 新增紧凑的 `scnet_job_list`、`scnet_limits`、`scnet_file_transfer` 工具。
|
|
32
|
-
- 明确 OpenAPI 上传参数是远端目录,文件名沿用本地文件名,不能把文件名当作远端目录传入。
|
|
33
|
-
|
|
34
|
-
## 0.6.0 更新
|
|
35
|
-
|
|
36
|
-
- 新增作业提交、状态查询、日志读取和取消作业的 DSH 工具,复用现有 SSH/OpenAPI backend。
|
|
37
|
-
- 作业生命周期调用支持 `dry_run`,并按 CLI 顺序一次只提示一个缺失决策。
|
|
38
|
-
- 提交结果返回权威的工作目录和日志路径;OpenAPI 日志查询支持由 `job_id` 和 `work_dir`
|
|
39
|
-
推导 `std.out`、`std.err` 路径。
|
|
40
|
-
|
|
41
19
|
## 这是什么
|
|
42
20
|
|
|
43
21
|
一个 DeepSeek Harness bundle 插件,把 `scnet-hpc` 的 Agent Skill 和 bash 脚本包装成 DSH 可安装包。装好后 DSH 模型在涉及超算集群、Slurm 作业、海光 DCU/DTK 时会加载对应 skill,并可调用工具完成确定性操作。
|
|
@@ -141,15 +119,7 @@ git 源、本地目录安装以及验证步骤见[安装说明](./docs/installat
|
|
|
141
119
|
只改 canonical,本仓库跑 sync 更新;不要把 `skills/scnet-hpc/` 当手工维护目录。
|
|
142
120
|
`sync.sh` 会自动排除 `scripts/install.sh` 和 `clusters/.cache/`,避免把安装脚本或本机动态缓存打进发布包。
|
|
143
121
|
|
|
144
|
-
|
|
145
|
-
再把 Skill 内容和版本同步到 `dsh-scnet` 与 `codex-scnet-hpc`。下游变更审核通过后,
|
|
146
|
-
才能发布对应版本的 `dsh-scnet` npm 包。
|
|
147
|
-
|
|
148
|
-
下游同步合并到 `main` 后,先运行本地校验,再手动触发仓库的 `Publish DSH-SCNet` workflow,
|
|
149
|
-
输入 `package.json` 中的当前版本。workflow 会校验 package 与 Skill
|
|
150
|
-
版本一致,执行验证、生成 tarball、以稳定版 `latest` 标签发布,并创建对应 Git tag。
|
|
151
|
-
不要只修改 `package.json`;`skills/scnet-hpc/VERSION`、package 版本和两份 README 的版本
|
|
152
|
-
标记必须保持一致。
|
|
122
|
+
发布只需在 canonical 仓库更新一次 `VERSION` 与版本亮点,CI 会同步两个分发仓库。下游同步 PR 校验通过后自动合并;合并产生的 `main` push 会校验并打包 bundle,以对应的 `latest` 或 `next` 标签发布 npm package,并创建匹配的 Git tag 与 GitHub Release。不要只修改 `package.json`;canonical `VERSION` 仍是唯一版本来源。
|
|
153
123
|
|
|
154
124
|
该 workflow 使用 GitHub OIDC 的 npm Trusted Publishing,因此应保留账号的 2FA/Passkey,
|
|
155
125
|
不需要长期保存 `NPM_TOKEN`。首次使用 OIDC 发布前,请在 npm package 设置中把
|
|
@@ -0,0 +1,96 @@
|
|
|
1
|
+
const OPERATION_DEFINITIONS = {
|
|
2
|
+
login: { scope: 'openid', mutatesRemote: false, supportsDryRun: false },
|
|
3
|
+
status: { scope: 'scnet.read', mutatesRemote: false, supportsDryRun: false },
|
|
4
|
+
refresh: { scope: 'openid', mutatesRemote: false, supportsDryRun: false },
|
|
5
|
+
logout: { scope: 'openid', mutatesRemote: true, supportsDryRun: false },
|
|
6
|
+
linkAccount: { scope: 'account.link', mutatesRemote: true, supportsDryRun: true },
|
|
7
|
+
unlinkAccount: { scope: 'account.link', mutatesRemote: true, supportsDryRun: true },
|
|
8
|
+
listRegions: { scope: 'scnet.read', mutatesRemote: false, supportsDryRun: false },
|
|
9
|
+
listSchedulers: { scope: 'scnet.read', mutatesRemote: false, supportsDryRun: false },
|
|
10
|
+
listQueues: { scope: 'scnet.read', mutatesRemote: false, supportsDryRun: false },
|
|
11
|
+
submitJob: { scope: 'jobs.submit', mutatesRemote: true, supportsDryRun: true },
|
|
12
|
+
getJob: { scope: 'jobs.read', mutatesRemote: false, supportsDryRun: false },
|
|
13
|
+
cancelJob: { scope: 'jobs.cancel', mutatesRemote: true, supportsDryRun: true },
|
|
14
|
+
listFiles: { scope: 'files.read', mutatesRemote: false, supportsDryRun: false },
|
|
15
|
+
transferFile: { scope: 'files.write', mutatesRemote: true, supportsDryRun: true },
|
|
16
|
+
}
|
|
17
|
+
|
|
18
|
+
export const SCNET_CONNECTOR_CONTRACT_VERSION = '0.1.0'
|
|
19
|
+
export const CONNECTOR_OPERATION_DEFINITIONS = Object.freeze(
|
|
20
|
+
Object.fromEntries(
|
|
21
|
+
Object.entries(OPERATION_DEFINITIONS).map(([name, definition]) => [name, Object.freeze({ ...definition })]),
|
|
22
|
+
),
|
|
23
|
+
)
|
|
24
|
+
export const CONNECTOR_CAPABILITY_STATES = Object.freeze([
|
|
25
|
+
'supported',
|
|
26
|
+
'unsupported',
|
|
27
|
+
'requires-remote-host',
|
|
28
|
+
])
|
|
29
|
+
|
|
30
|
+
function nonEmptyString(value, label) {
|
|
31
|
+
const normalized = String(value ?? '').trim()
|
|
32
|
+
if (!normalized) throw new TypeError(`${label} must be a non-empty string`)
|
|
33
|
+
return normalized
|
|
34
|
+
}
|
|
35
|
+
|
|
36
|
+
function normalizeState(value, operation) {
|
|
37
|
+
const state = nonEmptyString(value, `capability ${operation}`)
|
|
38
|
+
if (!CONNECTOR_CAPABILITY_STATES.includes(state)) {
|
|
39
|
+
throw new TypeError(
|
|
40
|
+
`capability ${operation} must be one of ${CONNECTOR_CAPABILITY_STATES.join(', ')}`,
|
|
41
|
+
)
|
|
42
|
+
}
|
|
43
|
+
return state
|
|
44
|
+
}
|
|
45
|
+
|
|
46
|
+
export function createCapabilityMatrix(entries = {}) {
|
|
47
|
+
if (entries === null || typeof entries !== 'object' || Array.isArray(entries)) {
|
|
48
|
+
throw new TypeError('capabilities must be an object')
|
|
49
|
+
}
|
|
50
|
+
return Object.freeze(
|
|
51
|
+
Object.fromEntries(
|
|
52
|
+
Object.keys(OPERATION_DEFINITIONS).map((operation) => [
|
|
53
|
+
operation,
|
|
54
|
+
normalizeState(entries[operation] ?? 'unsupported', operation),
|
|
55
|
+
]),
|
|
56
|
+
),
|
|
57
|
+
)
|
|
58
|
+
}
|
|
59
|
+
|
|
60
|
+
export function createConnectorDescriptor({
|
|
61
|
+
id = 'scnet',
|
|
62
|
+
version = SCNET_CONNECTOR_CONTRACT_VERSION,
|
|
63
|
+
issuer,
|
|
64
|
+
capabilities = {},
|
|
65
|
+
} = {}) {
|
|
66
|
+
const descriptor = {
|
|
67
|
+
contractVersion: SCNET_CONNECTOR_CONTRACT_VERSION,
|
|
68
|
+
id: nonEmptyString(id, 'connector id'),
|
|
69
|
+
version: nonEmptyString(version, 'connector version'),
|
|
70
|
+
issuer: nonEmptyString(issuer, 'issuer'),
|
|
71
|
+
capabilities: createCapabilityMatrix(capabilities),
|
|
72
|
+
}
|
|
73
|
+
return Object.freeze(descriptor)
|
|
74
|
+
}
|
|
75
|
+
|
|
76
|
+
export function getOperationDefinition(operation) {
|
|
77
|
+
const name = nonEmptyString(operation, 'operation')
|
|
78
|
+
const definition = CONNECTOR_OPERATION_DEFINITIONS[name]
|
|
79
|
+
if (!definition) throw new RangeError(`unsupported connector operation: ${name}`)
|
|
80
|
+
return definition
|
|
81
|
+
}
|
|
82
|
+
|
|
83
|
+
export function assertCapability(descriptor, operation, { allowRemoteHost = false } = {}) {
|
|
84
|
+
if (!descriptor || typeof descriptor !== 'object') {
|
|
85
|
+
throw new TypeError('connector descriptor is required')
|
|
86
|
+
}
|
|
87
|
+
const definition = getOperationDefinition(operation)
|
|
88
|
+
const state = descriptor.capabilities?.[operation]
|
|
89
|
+
if (state === 'supported' || (state === 'requires-remote-host' && allowRemoteHost)) {
|
|
90
|
+
return definition
|
|
91
|
+
}
|
|
92
|
+
if (state === 'requires-remote-host') {
|
|
93
|
+
throw new Error(`${operation} requires a remote host`)
|
|
94
|
+
}
|
|
95
|
+
throw new Error(`${operation} is not supported by connector ${descriptor.id || 'unknown'}`)
|
|
96
|
+
}
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
export {
|
|
2
|
+
CONNECTOR_CAPABILITY_STATES,
|
|
3
|
+
CONNECTOR_OPERATION_DEFINITIONS,
|
|
4
|
+
SCNET_CONNECTOR_CONTRACT_VERSION,
|
|
5
|
+
assertCapability,
|
|
6
|
+
createCapabilityMatrix,
|
|
7
|
+
createConnectorDescriptor,
|
|
8
|
+
getOperationDefinition,
|
|
9
|
+
} from './contract.mjs'
|
|
10
|
+
|
|
11
|
+
export {
|
|
12
|
+
MemoryCredentialStore,
|
|
13
|
+
TokenLifecycle,
|
|
14
|
+
createAuthorizationRequest,
|
|
15
|
+
createCodeVerifier,
|
|
16
|
+
createPkceChallenge,
|
|
17
|
+
createRefreshTokenRequest,
|
|
18
|
+
createTokenExchangeRequest,
|
|
19
|
+
normalizeTokenResponse,
|
|
20
|
+
validateAuthorizationCallback,
|
|
21
|
+
} from './oauth2.mjs'
|
|
@@ -0,0 +1,263 @@
|
|
|
1
|
+
import { createHash, randomBytes, randomUUID } from 'node:crypto'
|
|
2
|
+
|
|
3
|
+
const LOOPBACK_HOSTS = new Set(['127.0.0.1', '[::1]', 'localhost'])
|
|
4
|
+
|
|
5
|
+
function requiredString(value, label) {
|
|
6
|
+
const normalized = String(value ?? '').trim()
|
|
7
|
+
if (!normalized) throw new TypeError(`${label} must be a non-empty string`)
|
|
8
|
+
return normalized
|
|
9
|
+
}
|
|
10
|
+
|
|
11
|
+
function parseUrl(value, label) {
|
|
12
|
+
let parsed
|
|
13
|
+
try {
|
|
14
|
+
parsed = new URL(requiredString(value, label))
|
|
15
|
+
} catch {
|
|
16
|
+
throw new TypeError(`${label} must be a valid URL`)
|
|
17
|
+
}
|
|
18
|
+
return parsed
|
|
19
|
+
}
|
|
20
|
+
|
|
21
|
+
function assertHttpsEndpoint(value, label) {
|
|
22
|
+
const parsed = parseUrl(value, label)
|
|
23
|
+
if (parsed.protocol !== 'https:') {
|
|
24
|
+
throw new TypeError(`${label} must use https`)
|
|
25
|
+
}
|
|
26
|
+
return parsed
|
|
27
|
+
}
|
|
28
|
+
|
|
29
|
+
function assertRedirectUri(value) {
|
|
30
|
+
const parsed = parseUrl(value, 'redirectUri')
|
|
31
|
+
const isLoopback = parsed.protocol === 'http:' && LOOPBACK_HOSTS.has(parsed.hostname)
|
|
32
|
+
if (parsed.protocol !== 'https:' && !isLoopback) {
|
|
33
|
+
throw new TypeError('redirectUri must use https or a loopback http address')
|
|
34
|
+
}
|
|
35
|
+
return parsed
|
|
36
|
+
}
|
|
37
|
+
|
|
38
|
+
function base64url(value) {
|
|
39
|
+
return Buffer.from(value).toString('base64url')
|
|
40
|
+
}
|
|
41
|
+
|
|
42
|
+
export function createCodeVerifier() {
|
|
43
|
+
return base64url(randomBytes(32))
|
|
44
|
+
}
|
|
45
|
+
|
|
46
|
+
export function createPkceChallenge(codeVerifier) {
|
|
47
|
+
const verifier = requiredString(codeVerifier, 'codeVerifier')
|
|
48
|
+
return base64url(createHash('sha256').update(verifier).digest())
|
|
49
|
+
}
|
|
50
|
+
|
|
51
|
+
export function createAuthorizationRequest({
|
|
52
|
+
authorizationEndpoint,
|
|
53
|
+
clientId,
|
|
54
|
+
redirectUri,
|
|
55
|
+
issuer,
|
|
56
|
+
scope = [],
|
|
57
|
+
state = randomUUID(),
|
|
58
|
+
nonce = randomUUID(),
|
|
59
|
+
codeVerifier = createCodeVerifier(),
|
|
60
|
+
} = {}) {
|
|
61
|
+
const endpoint = assertHttpsEndpoint(authorizationEndpoint, 'authorizationEndpoint')
|
|
62
|
+
const redirect = assertRedirectUri(redirectUri)
|
|
63
|
+
const client = requiredString(clientId, 'clientId')
|
|
64
|
+
const normalizedIssuer = issuer === undefined ? undefined : assertHttpsEndpoint(issuer, 'issuer').origin
|
|
65
|
+
const scopes = Array.isArray(scope) ? scope : String(scope).split(/\s+/)
|
|
66
|
+
const normalizedScopes = [...new Set(scopes.map((item) => String(item).trim()).filter(Boolean))]
|
|
67
|
+
if (normalizedScopes.length === 0) throw new TypeError('scope must contain at least one value')
|
|
68
|
+
|
|
69
|
+
const params = new URLSearchParams({
|
|
70
|
+
client_id: client,
|
|
71
|
+
redirect_uri: redirect.href,
|
|
72
|
+
response_type: 'code',
|
|
73
|
+
scope: normalizedScopes.join(' '),
|
|
74
|
+
code_challenge: createPkceChallenge(codeVerifier),
|
|
75
|
+
code_challenge_method: 'S256',
|
|
76
|
+
state: requiredString(state, 'state'),
|
|
77
|
+
nonce: requiredString(nonce, 'nonce'),
|
|
78
|
+
})
|
|
79
|
+
return Object.freeze({
|
|
80
|
+
url: `${endpoint.href}${endpoint.search ? '&' : '?'}${params}`,
|
|
81
|
+
authorizationEndpoint: endpoint.href,
|
|
82
|
+
clientId: client,
|
|
83
|
+
redirectUri: redirect.href,
|
|
84
|
+
issuer: normalizedIssuer,
|
|
85
|
+
scope: Object.freeze(normalizedScopes),
|
|
86
|
+
state,
|
|
87
|
+
nonce,
|
|
88
|
+
codeVerifier,
|
|
89
|
+
})
|
|
90
|
+
}
|
|
91
|
+
|
|
92
|
+
function assertExactRedirect(callbackUrl, redirectUri) {
|
|
93
|
+
const actual = parseUrl(callbackUrl, 'callbackUrl')
|
|
94
|
+
const expected = assertRedirectUri(redirectUri)
|
|
95
|
+
if (actual.origin !== expected.origin || actual.pathname !== expected.pathname) {
|
|
96
|
+
throw new Error('OAuth callback redirect URI does not match the request')
|
|
97
|
+
}
|
|
98
|
+
if (actual.port !== expected.port || actual.hash !== expected.hash) {
|
|
99
|
+
throw new Error('OAuth callback redirect URI does not match the request')
|
|
100
|
+
}
|
|
101
|
+
for (const [key, value] of expected.searchParams) {
|
|
102
|
+
if (actual.searchParams.get(key) !== value) {
|
|
103
|
+
throw new Error('OAuth callback redirect URI does not match the request')
|
|
104
|
+
}
|
|
105
|
+
}
|
|
106
|
+
return actual
|
|
107
|
+
}
|
|
108
|
+
|
|
109
|
+
export function validateAuthorizationCallback(callbackUrl, request) {
|
|
110
|
+
const callback = assertExactRedirect(callbackUrl, request.redirectUri)
|
|
111
|
+
const returnedState = callback.searchParams.get('state')
|
|
112
|
+
if (returnedState !== request.state) throw new Error('OAuth state mismatch')
|
|
113
|
+
|
|
114
|
+
const error = callback.searchParams.get('error')
|
|
115
|
+
if (error) {
|
|
116
|
+
const description = callback.searchParams.get('error_description')
|
|
117
|
+
throw new Error(`OAuth authorization failed: ${error}${description ? ` (${description})` : ''}`)
|
|
118
|
+
}
|
|
119
|
+
|
|
120
|
+
const code = callback.searchParams.get('code')
|
|
121
|
+
if (!code) throw new Error('OAuth callback did not contain an authorization code')
|
|
122
|
+
if (request.issuer !== undefined) {
|
|
123
|
+
const returnedIssuer = callback.searchParams.get('iss')
|
|
124
|
+
if (returnedIssuer !== request.issuer) throw new Error('OAuth issuer mismatch')
|
|
125
|
+
}
|
|
126
|
+
return Object.freeze({
|
|
127
|
+
code,
|
|
128
|
+
state: returnedState,
|
|
129
|
+
issuer: callback.searchParams.get('iss') || undefined,
|
|
130
|
+
})
|
|
131
|
+
}
|
|
132
|
+
|
|
133
|
+
export function createTokenExchangeRequest({
|
|
134
|
+
tokenEndpoint,
|
|
135
|
+
clientId,
|
|
136
|
+
code,
|
|
137
|
+
redirectUri,
|
|
138
|
+
codeVerifier,
|
|
139
|
+
}) {
|
|
140
|
+
const endpoint = assertHttpsEndpoint(tokenEndpoint, 'tokenEndpoint')
|
|
141
|
+
const redirect = assertRedirectUri(redirectUri)
|
|
142
|
+
const params = new URLSearchParams({
|
|
143
|
+
grant_type: 'authorization_code',
|
|
144
|
+
client_id: requiredString(clientId, 'clientId'),
|
|
145
|
+
code: requiredString(code, 'code'),
|
|
146
|
+
redirect_uri: redirect.href,
|
|
147
|
+
code_verifier: requiredString(codeVerifier, 'codeVerifier'),
|
|
148
|
+
})
|
|
149
|
+
return Object.freeze({
|
|
150
|
+
method: 'POST',
|
|
151
|
+
url: endpoint.href,
|
|
152
|
+
headers: Object.freeze({ 'content-type': 'application/x-www-form-urlencoded' }),
|
|
153
|
+
body: params.toString(),
|
|
154
|
+
})
|
|
155
|
+
}
|
|
156
|
+
|
|
157
|
+
export function createRefreshTokenRequest({ tokenEndpoint, clientId, refreshToken, scope } = {}) {
|
|
158
|
+
const endpoint = assertHttpsEndpoint(tokenEndpoint, 'tokenEndpoint')
|
|
159
|
+
const params = new URLSearchParams({
|
|
160
|
+
grant_type: 'refresh_token',
|
|
161
|
+
client_id: requiredString(clientId, 'clientId'),
|
|
162
|
+
refresh_token: requiredString(refreshToken, 'refreshToken'),
|
|
163
|
+
})
|
|
164
|
+
if (scope !== undefined) params.set('scope', requiredString(scope, 'scope'))
|
|
165
|
+
return Object.freeze({
|
|
166
|
+
method: 'POST',
|
|
167
|
+
url: endpoint.href,
|
|
168
|
+
headers: Object.freeze({ 'content-type': 'application/x-www-form-urlencoded' }),
|
|
169
|
+
body: params.toString(),
|
|
170
|
+
})
|
|
171
|
+
}
|
|
172
|
+
|
|
173
|
+
export function normalizeTokenResponse(response, now = Date.now()) {
|
|
174
|
+
if (!response || typeof response !== 'object') throw new TypeError('token response must be an object')
|
|
175
|
+
const accessToken = requiredString(response.access_token, 'access_token')
|
|
176
|
+
const refreshToken = response.refresh_token === undefined ? undefined : requiredString(response.refresh_token, 'refresh_token')
|
|
177
|
+
const tokenType = String(response.token_type || 'Bearer')
|
|
178
|
+
const expiresIn = Number(response.expires_in)
|
|
179
|
+
if (!Number.isFinite(expiresIn) || expiresIn <= 0) throw new TypeError('expires_in must be a positive number')
|
|
180
|
+
return Object.freeze({
|
|
181
|
+
accessToken,
|
|
182
|
+
refreshToken,
|
|
183
|
+
tokenType,
|
|
184
|
+
scope: response.scope ? String(response.scope) : undefined,
|
|
185
|
+
expiresAt: now + expiresIn * 1000,
|
|
186
|
+
})
|
|
187
|
+
}
|
|
188
|
+
|
|
189
|
+
export class MemoryCredentialStore {
|
|
190
|
+
#values = new Map()
|
|
191
|
+
|
|
192
|
+
async get(key) {
|
|
193
|
+
return this.#values.get(requiredString(key, 'credential key'))
|
|
194
|
+
}
|
|
195
|
+
|
|
196
|
+
async set(key, value) {
|
|
197
|
+
const normalizedKey = requiredString(key, 'credential key')
|
|
198
|
+
if (value === undefined) throw new TypeError('credential value must not be undefined')
|
|
199
|
+
this.#values.set(normalizedKey, structuredClone(value))
|
|
200
|
+
}
|
|
201
|
+
|
|
202
|
+
async delete(key) {
|
|
203
|
+
this.#values.delete(requiredString(key, 'credential key'))
|
|
204
|
+
}
|
|
205
|
+
|
|
206
|
+
async clear() {
|
|
207
|
+
this.#values.clear()
|
|
208
|
+
}
|
|
209
|
+
}
|
|
210
|
+
|
|
211
|
+
export class TokenLifecycle {
|
|
212
|
+
#store
|
|
213
|
+
#key
|
|
214
|
+
#now
|
|
215
|
+
#refreshPromise
|
|
216
|
+
|
|
217
|
+
constructor({ store = new MemoryCredentialStore(), key = 'scnet.tokens', now = () => Date.now() } = {}) {
|
|
218
|
+
this.#store = store
|
|
219
|
+
this.#key = requiredString(key, 'token key')
|
|
220
|
+
this.#now = now
|
|
221
|
+
}
|
|
222
|
+
|
|
223
|
+
async save(tokenResponse) {
|
|
224
|
+
const token = normalizeTokenResponse(tokenResponse, this.#now())
|
|
225
|
+
await this.#store.set(this.#key, token)
|
|
226
|
+
return this.status()
|
|
227
|
+
}
|
|
228
|
+
|
|
229
|
+
async status() {
|
|
230
|
+
const token = await this.#store.get(this.#key)
|
|
231
|
+
if (!token) return { authenticated: false, expiresAt: undefined }
|
|
232
|
+
return {
|
|
233
|
+
authenticated: true,
|
|
234
|
+
expiresAt: token.expiresAt,
|
|
235
|
+
hasRefreshToken: Boolean(token.refreshToken),
|
|
236
|
+
scope: token.scope,
|
|
237
|
+
}
|
|
238
|
+
}
|
|
239
|
+
|
|
240
|
+
async accessToken({ refresh } = {}) {
|
|
241
|
+
const token = await this.#store.get(this.#key)
|
|
242
|
+
if (!token) throw new Error('SCNet authentication required')
|
|
243
|
+
if (token.expiresAt > this.#now() + 30000) return token.accessToken
|
|
244
|
+
if (typeof refresh !== 'function' || !token.refreshToken) {
|
|
245
|
+
throw new Error('SCNet access token expired; re-authentication required')
|
|
246
|
+
}
|
|
247
|
+
if (!this.#refreshPromise) {
|
|
248
|
+
this.#refreshPromise = Promise.resolve()
|
|
249
|
+
.then(() => refresh(token.refreshToken))
|
|
250
|
+
.then((response) => this.save(response))
|
|
251
|
+
.finally(() => {
|
|
252
|
+
this.#refreshPromise = undefined
|
|
253
|
+
})
|
|
254
|
+
}
|
|
255
|
+
await this.#refreshPromise
|
|
256
|
+
const refreshed = await this.#store.get(this.#key)
|
|
257
|
+
return refreshed.accessToken
|
|
258
|
+
}
|
|
259
|
+
|
|
260
|
+
async clear() {
|
|
261
|
+
await this.#store.delete(this.#key)
|
|
262
|
+
}
|
|
263
|
+
}
|
package/index.mjs
CHANGED
|
@@ -12,6 +12,11 @@ const SKILL_DIR = join(__dirname, 'skills', 'scnet-hpc')
|
|
|
12
12
|
const SCRIPTS_DIR = join(SKILL_DIR, 'scripts')
|
|
13
13
|
const CLUSTERS_DIR = join(SKILL_DIR, 'clusters')
|
|
14
14
|
|
|
15
|
+
export function resolvePythonCandidates(env = process.env) {
|
|
16
|
+
const configured = String(env.DSH_SCNET_PYTHON ?? env.SCNET_PYTHON ?? '').trim()
|
|
17
|
+
return configured ? [configured] : ['python3', 'python']
|
|
18
|
+
}
|
|
19
|
+
|
|
15
20
|
function runBash(script, args, timeoutMs = 120000, cwd = SKILL_DIR) {
|
|
16
21
|
return new Promise((resolve) => {
|
|
17
22
|
execFile(
|
|
@@ -32,21 +37,35 @@ function runBash(script, args, timeoutMs = 120000, cwd = SKILL_DIR) {
|
|
|
32
37
|
}
|
|
33
38
|
|
|
34
39
|
function runScnet(args, timeoutMs = 120000) {
|
|
40
|
+
const candidates = resolvePythonCandidates()
|
|
41
|
+
const scriptArgs = [join(SCRIPTS_DIR, 'scnet.py'), '--json', ...args]
|
|
42
|
+
|
|
35
43
|
return new Promise((resolve) => {
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
44
|
+
const attempt = (index) => {
|
|
45
|
+
const executable = candidates[index]
|
|
46
|
+
execFile(
|
|
47
|
+
executable,
|
|
48
|
+
scriptArgs,
|
|
49
|
+
{ cwd: SKILL_DIR, timeout: timeoutMs, maxBuffer: 16 * 1024 * 1024 },
|
|
50
|
+
(error, stdout, stderr) => {
|
|
51
|
+
if (error?.code === 'ENOENT' && index + 1 < candidates.length) {
|
|
52
|
+
attempt(index + 1)
|
|
53
|
+
return
|
|
54
|
+
}
|
|
55
|
+
const code = error ? (typeof error.code === 'number' ? error.code : 1) : 0
|
|
56
|
+
resolve({
|
|
57
|
+
ok: code === 0,
|
|
58
|
+
code,
|
|
59
|
+
executable,
|
|
60
|
+
stdout: String(stdout || ''),
|
|
61
|
+
stderr: error?.code === 'ENOENT'
|
|
62
|
+
? `无法启动 Python 解释器 ${executable};请设置 DSH_SCNET_PYTHON 或 SCNET_PYTHON 指向 Python 3。`
|
|
63
|
+
: String(stderr || ''),
|
|
64
|
+
})
|
|
65
|
+
},
|
|
66
|
+
)
|
|
67
|
+
}
|
|
68
|
+
attempt(0)
|
|
50
69
|
})
|
|
51
70
|
}
|
|
52
71
|
|
|
@@ -1173,5 +1192,6 @@ export const __testables = {
|
|
|
1173
1192
|
parseWalltimeSeconds,
|
|
1174
1193
|
pickBackend,
|
|
1175
1194
|
queueCandidates,
|
|
1195
|
+
resolvePythonCandidates,
|
|
1176
1196
|
translateScnetError,
|
|
1177
1197
|
}
|
package/package.json
CHANGED
|
@@ -1,11 +1,12 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "dsh-scnet",
|
|
3
|
-
"version": "0.6.
|
|
3
|
+
"version": "0.6.7",
|
|
4
4
|
"description": "Community DeepSeek Harness bundle for SCNet SSH and OpenAPI jobs, files, Notebook resources, Slurm diagnostics, and Hygon DCU/DTK guidance.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "./index.mjs",
|
|
7
7
|
"exports": {
|
|
8
8
|
".": "./index.mjs",
|
|
9
|
+
"./connector": "./connector/index.mjs",
|
|
9
10
|
"./cordis.patch.yml": "./cordis.patch.yml",
|
|
10
11
|
"./package.json": "./package.json"
|
|
11
12
|
},
|
|
@@ -26,8 +27,11 @@
|
|
|
26
27
|
},
|
|
27
28
|
"files": [
|
|
28
29
|
"index.mjs",
|
|
30
|
+
"connector",
|
|
29
31
|
"cordis.patch.yml",
|
|
30
32
|
"skills",
|
|
33
|
+
"!skills/**/__pycache__",
|
|
34
|
+
"!skills/**/*.pyc",
|
|
31
35
|
"docs",
|
|
32
36
|
"README_CN.md",
|
|
33
37
|
"README.md",
|
package/skills/scnet-hpc/VERSION
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
0.6.
|
|
1
|
+
0.6.7
|
|
@@ -50,6 +50,20 @@ ssh <集群> 'module avail 2>&1'
|
|
|
50
50
|
|
|
51
51
|
## Python 环境
|
|
52
52
|
|
|
53
|
+
### 本地脚本解释器
|
|
54
|
+
|
|
55
|
+
Linux Desktop 可将随包 Python 的绝对路径注入本地脚本:
|
|
56
|
+
|
|
57
|
+
```bash
|
|
58
|
+
export DSH_SCNET_PYTHON=/path/to/bundled/python
|
|
59
|
+
# 通用环境也可使用:
|
|
60
|
+
export SCNET_PYTHON=/path/to/python3
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
`DSH_SCNET_PYTHON` 优先于 `SCNET_PYTHON`。脚本会验证解释器确实是 Python 3;
|
|
64
|
+
未注入时才按 `python3`、`python` 顺序探测。该设置只影响本机的 OpenAPI CLI 和作业
|
|
65
|
+
脚本生成,不会把本机解释器路径传到远端计算节点。
|
|
66
|
+
|
|
53
67
|
超算通常没有系统级的 torch,用 venv 或 conda env。profile 的 `DEFAULT_VENV`
|
|
54
68
|
记录默认环境路径(相对家目录)。
|
|
55
69
|
|
|
@@ -12,6 +12,37 @@ CLUSTER_DIR="$REPO_ROOT/clusters"
|
|
|
12
12
|
die() { printf '错误: %s\n' "$1" >&2; exit 1; }
|
|
13
13
|
info() { printf ' %s\n' "$1"; }
|
|
14
14
|
|
|
15
|
+
# Resolve the local Python 3 interpreter without assuming that every desktop
|
|
16
|
+
# distribution exposes the command as `python3`. Desktop hosts may inject an
|
|
17
|
+
# absolute path through DSH_SCNET_PYTHON; SCNET_PYTHON is the portable alias.
|
|
18
|
+
resolve_python() {
|
|
19
|
+
local configured="${DSH_SCNET_PYTHON:-${SCNET_PYTHON:-}}"
|
|
20
|
+
local candidate resolved
|
|
21
|
+
|
|
22
|
+
if [ -n "$configured" ]; then
|
|
23
|
+
if [ -x "$configured" ]; then
|
|
24
|
+
resolved="$configured"
|
|
25
|
+
else
|
|
26
|
+
resolved="$(command -v "$configured" 2>/dev/null || true)"
|
|
27
|
+
fi
|
|
28
|
+
[ -n "$resolved" ] || return 1
|
|
29
|
+
"$resolved" -c 'import sys; raise SystemExit(0 if sys.version_info.major == 3 else 1)' \
|
|
30
|
+
>/dev/null 2>&1 || return 1
|
|
31
|
+
printf '%s\n' "$resolved"
|
|
32
|
+
return 0
|
|
33
|
+
fi
|
|
34
|
+
|
|
35
|
+
for candidate in python3 python; do
|
|
36
|
+
resolved="$(command -v "$candidate" 2>/dev/null || true)"
|
|
37
|
+
[ -n "$resolved" ] || continue
|
|
38
|
+
"$resolved" -c 'import sys; raise SystemExit(0 if sys.version_info.major == 3 else 1)' \
|
|
39
|
+
>/dev/null 2>&1 || continue
|
|
40
|
+
printf '%s\n' "$resolved"
|
|
41
|
+
return 0
|
|
42
|
+
done
|
|
43
|
+
return 1
|
|
44
|
+
}
|
|
45
|
+
|
|
15
46
|
list_clusters() {
|
|
16
47
|
find "$CLUSTER_DIR" -maxdepth 1 -name '*.conf' ! -name '_*' -exec basename {} .conf \; 2>/dev/null | sort
|
|
17
48
|
}
|
|
@@ -370,8 +370,8 @@ if [ -z "$ACTION" ]; then
|
|
|
370
370
|
fi
|
|
371
371
|
|
|
372
372
|
if [ "$ACTION" = status ] || [ "$ACTION" = reset ]; then
|
|
373
|
-
if
|
|
374
|
-
exec
|
|
373
|
+
if PYTHON_BIN="$(resolve_python)"; then
|
|
374
|
+
exec "$PYTHON_BIN" "$SCRIPT_DIR/scnet.py" setup "$ACTION"
|
|
375
375
|
fi
|
|
376
376
|
[ "$ACTION" = status ] || die "reset 需要 Python 3 以执行安全的分范围重置"
|
|
377
377
|
config_root="${XDG_CONFIG_HOME:-$HOME/.config}/scnet-hpc"
|
|
@@ -450,16 +450,16 @@ if [ "$BACKEND" = ssh ] || [ "$BACKEND" = both ]; then
|
|
|
450
450
|
fi
|
|
451
451
|
|
|
452
452
|
if [ "$BACKEND" = openapi ] || [ "$BACKEND" = both ]; then
|
|
453
|
-
if
|
|
453
|
+
if PYTHON_BIN="$(resolve_python)"; then
|
|
454
454
|
printf '%s\n' \
|
|
455
455
|
"OpenAPI 使用一组平台 AK/SK 自动发现全部授权区域。" >&2
|
|
456
456
|
openapi_args=(setup "$ACTION" --mode openapi)
|
|
457
457
|
[ "$SKIP_CONNECT" = yes ] && openapi_args+=(--skip-connect)
|
|
458
|
-
|
|
458
|
+
"$PYTHON_BIN" "$SCRIPT_DIR/scnet.py" "${openapi_args[@]}"
|
|
459
459
|
else
|
|
460
460
|
printf '%s\n' \
|
|
461
461
|
"OpenAPI backend 需要 Python 3;SSH 配置不受影响。" \
|
|
462
|
-
"
|
|
462
|
+
"请设置 DSH_SCNET_PYTHON 或 SCNET_PYTHON 指向 Python 3,或安装 python3。" >&2
|
|
463
463
|
fi
|
|
464
464
|
fi
|
|
465
465
|
|