@sx4im/skillcheck 0.1.1 → 0.2.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +19 -48
- package/dist/bin/skillcheck.js +2 -2
- package/dist/bin/skillcheck.js.map +1 -1
- package/dist/src/adapters/nvidia-nim.js +5 -2
- package/dist/src/adapters/nvidia-nim.js.map +1 -1
- package/dist/src/cli.d.ts +1 -3
- package/dist/src/cli.js +61 -67
- package/dist/src/cli.js.map +1 -1
- package/dist/src/config.d.ts +10 -0
- package/dist/src/config.js +64 -0
- package/dist/src/config.js.map +1 -0
- package/dist/src/env.js +30 -13
- package/dist/src/env.js.map +1 -1
- package/dist/src/eval.d.ts +1 -0
- package/dist/src/eval.js +7 -4
- package/dist/src/eval.js.map +1 -1
- package/dist/src/generate.js +3 -1
- package/dist/src/generate.js.map +1 -1
- package/dist/src/grade.js +7 -2
- package/dist/src/grade.js.map +1 -1
- package/dist/src/run.js +7 -2
- package/dist/src/run.js.map +1 -1
- package/dist/src/ui.d.ts +14 -0
- package/dist/src/ui.js +327 -0
- package/dist/src/ui.js.map +1 -0
- package/docs/skillcheck-cloud.md +111 -0
- package/examples/nvidia-proxy/README.md +19 -0
- package/examples/nvidia-proxy/server.mjs +96 -0
- package/package.json +4 -2
package/README.md
CHANGED
|
@@ -8,86 +8,57 @@ Measure whether an agent skill actually improves task performance.
|
|
|
8
8
|
|
|
9
9
|
- M0-M4 gates have passed and are recorded in `BUILD-LOG.md`.
|
|
10
10
|
- M5 completed a 20-skill launch corpus in `results/launch/20260605T110514Z-qwen-next`.
|
|
11
|
-
-
|
|
12
|
-
- Do not publish to npm until `RELEASE-CHECKLIST.md` is complete.
|
|
11
|
+
- Published on npm as `@sx4im/skillcheck`.
|
|
13
12
|
|
|
14
13
|
## One-Line Install
|
|
15
14
|
|
|
16
|
-
After npm publication, install the CLI globally with one command:
|
|
17
|
-
|
|
18
15
|
```bash
|
|
19
16
|
npm install -g @sx4im/skillcheck
|
|
20
17
|
```
|
|
21
18
|
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
```bash
|
|
25
|
-
skillcheck --help
|
|
26
|
-
```
|
|
19
|
+
Requires Node.js 20 or newer.
|
|
27
20
|
|
|
28
|
-
|
|
21
|
+
## Usage
|
|
29
22
|
|
|
30
23
|
```bash
|
|
31
|
-
skillcheck
|
|
24
|
+
skillcheck
|
|
32
25
|
```
|
|
33
26
|
|
|
34
|
-
|
|
27
|
+
On first run, Skillcheck asks for your Skillcheck API URL and saves it locally. Then it opens the interactive picker. Select `SKILL.md`, `AGENTS.md`, `CLAUDE.md`, `.cursorrules`, or a folder containing one.
|
|
35
28
|
|
|
36
|
-
|
|
29
|
+
Change the saved API URL later:
|
|
37
30
|
|
|
38
31
|
```bash
|
|
39
|
-
|
|
32
|
+
skillcheck setup
|
|
40
33
|
```
|
|
41
34
|
|
|
42
|
-
|
|
35
|
+
You can also pass a path directly:
|
|
43
36
|
|
|
44
37
|
```bash
|
|
45
|
-
|
|
38
|
+
skillcheck check path/to/SKILL.md
|
|
39
|
+
skillcheck path/to/skill-folder
|
|
46
40
|
```
|
|
47
41
|
|
|
48
|
-
|
|
42
|
+
The quick run shows a blue/white result card in the terminal. It does not write a results folder unless you explicitly pass `--output`.
|
|
49
43
|
|
|
50
|
-
|
|
44
|
+
## Cloud Setup
|
|
51
45
|
|
|
52
|
-
|
|
46
|
+
For shared/public usage, point the CLI at your Skillcheck Cloud endpoint:
|
|
53
47
|
|
|
54
48
|
```bash
|
|
55
|
-
|
|
56
|
-
npm run build
|
|
57
|
-
node dist/bin/skillcheck.js --help
|
|
49
|
+
export SKILLCHECK_API_URL=https://api.yourdomain.com/v1
|
|
58
50
|
```
|
|
59
51
|
|
|
60
|
-
|
|
52
|
+
See `docs/skillcheck-cloud.md` for the proxy and dashboard plan.
|
|
61
53
|
|
|
62
|
-
|
|
63
|
-
node dist/bin/skillcheck.js eval fixtures/m2/strong-skill/SKILL.md --task-suite fixtures/m2/deterministic-tasks.json --tasks 3 --trials 3 --output /tmp/skillcheck-result.json
|
|
64
|
-
```
|
|
65
|
-
|
|
66
|
-
## Environment
|
|
67
|
-
|
|
68
|
-
Copy `.env.example` to `.env` and set `NVIDIA_API_KEY`.
|
|
69
|
-
|
|
70
|
-
Required for live NVIDIA NIM calls:
|
|
71
|
-
|
|
72
|
-
```bash
|
|
73
|
-
NVIDIA_API_KEY=...
|
|
74
|
-
NVIDIA_BASE_URL=https://integrate.api.nvidia.com/v1
|
|
75
|
-
NVIDIA_GENERATOR_MODEL=qwen/qwen3-next-80b-a3b-instruct
|
|
76
|
-
NVIDIA_GRADER_MODEL=qwen/qwen3-next-80b-a3b-instruct
|
|
77
|
-
NVIDIA_RUNNER_MODEL=qwen/qwen3-next-80b-a3b-instruct
|
|
78
|
-
```
|
|
79
|
-
|
|
80
|
-
Recommended launch-run provider controls:
|
|
54
|
+
## Local Development
|
|
81
55
|
|
|
82
56
|
```bash
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
NVIDIA_MAX_RETRY_DELAY_MS=60000
|
|
57
|
+
npm ci
|
|
58
|
+
npm run build
|
|
59
|
+
node dist/bin/skillcheck.js --help
|
|
87
60
|
```
|
|
88
61
|
|
|
89
|
-
The final M5 run used `--concurrency 1` and a 5 second process-wide request delay. The completed run had 75 recovered NVIDIA connection retries and no `429` rate-limit lines.
|
|
90
|
-
|
|
91
62
|
## Commands
|
|
92
63
|
|
|
93
64
|
Friendly skill check:
|
package/dist/bin/skillcheck.js
CHANGED
|
@@ -1,8 +1,8 @@
|
|
|
1
1
|
#!/usr/bin/env node
|
|
2
2
|
import { main } from '../src/cli.js';
|
|
3
|
+
import { formatFatalError } from '../src/ui.js';
|
|
3
4
|
main(process.argv).catch((error) => {
|
|
4
|
-
|
|
5
|
-
console.error(message);
|
|
5
|
+
console.error(formatFatalError(error));
|
|
6
6
|
process.exitCode = 1;
|
|
7
7
|
});
|
|
8
8
|
//# sourceMappingURL=skillcheck.js.map
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"skillcheck.js","sourceRoot":"","sources":["../../packages/cli/bin/skillcheck.ts"],"names":[],"mappings":";AACA,OAAO,EAAE,IAAI,EAAE,MAAM,eAAe,CAAC;
|
|
1
|
+
{"version":3,"file":"skillcheck.js","sourceRoot":"","sources":["../../packages/cli/bin/skillcheck.ts"],"names":[],"mappings":";AACA,OAAO,EAAE,IAAI,EAAE,MAAM,eAAe,CAAC;AACrC,OAAO,EAAE,gBAAgB,EAAE,MAAM,cAAc,CAAC;AAEhD,IAAI,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC,KAAK,CAAC,CAAC,KAAc,EAAE,EAAE;IAC1C,OAAO,CAAC,KAAK,CAAC,gBAAgB,CAAC,KAAK,CAAC,CAAC,CAAC;IACvC,OAAO,CAAC,QAAQ,GAAG,CAAC,CAAC;AACvB,CAAC,CAAC,CAAC"}
|
|
@@ -64,9 +64,12 @@ function getRetryAfterMs(error) {
|
|
|
64
64
|
return undefined;
|
|
65
65
|
}
|
|
66
66
|
function describeRetry(error, attempt, waitMs, maxAttempts) {
|
|
67
|
+
if (process.env.SKILLCHECK_DEBUG_RETRIES !== '1') {
|
|
68
|
+
return;
|
|
69
|
+
}
|
|
67
70
|
const status = getStatus(error);
|
|
68
71
|
const label = status === undefined ? 'connection' : `status ${status}`;
|
|
69
|
-
console.error(`[
|
|
72
|
+
console.error(`[skillcheck] retry ${attempt + 1}/${maxAttempts} after ${label}; waiting ${waitMs} ms`);
|
|
70
73
|
}
|
|
71
74
|
export class NvidiaNimClient {
|
|
72
75
|
static requestQueue = Promise.resolve();
|
|
@@ -132,7 +135,7 @@ export class NvidiaNimClient {
|
|
|
132
135
|
const content = extractMessageText(message);
|
|
133
136
|
if (content === undefined) {
|
|
134
137
|
const keys = message ? Object.keys(message).sort().join(',') : 'none';
|
|
135
|
-
throw new Error(`
|
|
138
|
+
throw new Error(`Skillcheck Cloud response did not include text content; message keys: ${keys}`);
|
|
136
139
|
}
|
|
137
140
|
return {
|
|
138
141
|
content,
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"nvidia-nim.js","sourceRoot":"","sources":["../../../packages/cli/src/adapters/nvidia-nim.ts"],"names":[],"mappings":"AAAA,OAAO,MAAM,MAAM,QAAQ,CAAC;AAuB5B,MAAM,kBAAkB,GAAG,IAAI,GAAG,CAAC,CAAC,GAAG,EAAE,GAAG,EAAE,GAAG,EAAE,GAAG,EAAE,GAAG,EAAE,GAAG,EAAE,GAAG,CAAC,CAAC,CAAC;AAExE,SAAS,kBAAkB,CAAC,OAAgB;IAC1C,IAAI,OAAO,OAAO,KAAK,QAAQ,EAAE,CAAC;QAChC,OAAO,OAAO,CAAC;IACjB,CAAC;IAED,IAAI,KAAK,CAAC,OAAO,CAAC,OAAO,CAAC,EAAE,CAAC;QAC3B,MAAM,KAAK,GAAG,OAAO;aAClB,GAAG,CAAC,CAAC,IAAI,EAAE,EAAE;YACZ,IAAI,OAAO,IAAI,KAAK,QAAQ,EAAE,CAAC;gBAC7B,OAAO,IAAI,CAAC;YACd,CAAC;YACD,IAAI,OAAO,IAAI,KAAK,QAAQ,IAAI,IAAI,KAAK,IAAI,IAAI,MAAM,IAAI,IAAI,EAAE,CAAC;gBAChE,MAAM,IAAI,GAAI,IAA2B,CAAC,IAAI,CAAC;gBAC/C,OAAO,OAAO,IAAI,KAAK,QAAQ,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC;YAC9C,CAAC;YACD,OAAO,EAAE,CAAC;QACZ,CAAC,CAAC;aACD,IAAI,CAAC,EAAE,CAAC,CAAC;QACZ,OAAO,KAAK,IAAI,SAAS,CAAC;IAC5B,CAAC;IAED,OAAO,SAAS,CAAC;AACnB,CAAC;AAED,SAAS,kBAAkB,CAAC,OAAgB;IAC1C,IAAI,OAAO,OAAO,KAAK,QAAQ,IAAI,OAAO,KAAK,IAAI,EAAE,CAAC;QACpD,OAAO,SAAS,CAAC;IACnB,CAAC;IAED,MAAM,MAAM,GAAG,OAAgF,CAAC;IAChG,OAAO,CACL,kBAAkB,CAAC,MAAM,CAAC,OAAO,CAAC;QAClC,kBAAkB,CAAC,MAAM,CAAC,iBAAiB,CAAC;QAC5C,kBAAkB,CAAC,MAAM,CAAC,OAAO,CAAC,CACnC,CAAC;AACJ,CAAC;AAED,SAAS,KAAK,CAAC,EAAU;IACvB,OAAO,IAAI,OAAO,CAAC,CAAC,OAAO,EAAE,EAAE,CAAC,UAAU,CAAC,OAAO,EAAE,EAAE,CAAC,CAAC,CAAC;AAC3D,CAAC;AAED,SAAS,SAAS,CAAC,KAAc;IAC/B,IAAI,OAAO,KAAK,KAAK,QAAQ,IAAI,KAAK,KAAK,IAAI,IAAI,QAAQ,IAAI,KAAK,EAAE,CAAC;QACrE,MAAM,MAAM,GAAI,KAA8B,CAAC,MAAM,CAAC;QACtD,OAAO,OAAO,MAAM,KAAK,QAAQ,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,SAAS,CAAC;IACzD,CAAC;IACD,OAAO,SAAS,CAAC;AACnB,CAAC;AAED,SAAS,eAAe,CAAC,KAAc;IACrC,IAAI,OAAO,KAAK,KAAK,QAAQ,IAAI,KAAK,KAAK,IAAI,IAAI,CAAC,CAAC,SAAS,IAAI,KAAK,CAAC,EAAE,CAAC;QACzE,OAAO,SAAS,CAAC;IACnB,CAAC;IAED,MAAM,OAAO,GAAI,KAA+B,CAAC,OAAO,CAAC;IACzD,MAAM,UAAU,GACd,OAAO,YAAY,OAAO;QACxB,CAAC,CAAC,OAAO,CAAC,GAAG,CAAC,aAAa,CAAC;QAC5B,CAAC,CAAC,OAAO,OAAO,KAAK,QAAQ,IAAI,OAAO,KAAK,IAAI,IAAI,aAAa,IAAI,OAAO;YAC3E,CAAC,CAAE,OAAuC,CAAC,aAAa,CAAC;YACzD,CAAC,CAAC,SAAS,CAAC;IAElB,IAAI,OAAO,UAAU,KAAK,QAAQ,EAAE,CAAC;QACnC,OAAO,SAAS,CAAC;IACnB,CAAC;IAED,MAAM,OAAO,GAAG,MAAM,CAAC,UAAU,CAAC,CAAC;IACnC,IAAI,MAAM,CAAC,QAAQ,CAAC,OAAO,CAAC,IAAI,OAAO,GAAG,CAAC,EAAE,CAAC;QAC5C,OAAO,OAAO,GAAG,IAAI,CAAC;IACxB,CAAC;IAED,MAAM,MAAM,GAAG,IAAI,CAAC,KAAK,CAAC,UAAU,CAAC,CAAC;IACtC,IAAI,MAAM,CAAC,QAAQ,CAAC,MAAM,CAAC,EAAE,CAAC;QAC5B,OAAO,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC,GAAG,EAAE,CAAC,CAAC;IAC1C,CAAC;IAED,OAAO,SAAS,CAAC;AACnB,CAAC;AAED,SAAS,aAAa,CAAC,KAAc,EAAE,OAAe,EAAE,MAAc,EAAE,WAAmB;IACzF,MAAM,MAAM,GAAG,SAAS,CAAC,KAAK,CAAC,CAAC;IAChC,MAAM,KAAK,GAAG,MAAM,KAAK,SAAS,CAAC,CAAC,CAAC,YAAY,CAAC,CAAC,CAAC,UAAU,MAAM,EAAE,CAAC;IACvE,OAAO,CAAC,KAAK,CAAC,
|
|
1
|
+
{"version":3,"file":"nvidia-nim.js","sourceRoot":"","sources":["../../../packages/cli/src/adapters/nvidia-nim.ts"],"names":[],"mappings":"AAAA,OAAO,MAAM,MAAM,QAAQ,CAAC;AAuB5B,MAAM,kBAAkB,GAAG,IAAI,GAAG,CAAC,CAAC,GAAG,EAAE,GAAG,EAAE,GAAG,EAAE,GAAG,EAAE,GAAG,EAAE,GAAG,EAAE,GAAG,CAAC,CAAC,CAAC;AAExE,SAAS,kBAAkB,CAAC,OAAgB;IAC1C,IAAI,OAAO,OAAO,KAAK,QAAQ,EAAE,CAAC;QAChC,OAAO,OAAO,CAAC;IACjB,CAAC;IAED,IAAI,KAAK,CAAC,OAAO,CAAC,OAAO,CAAC,EAAE,CAAC;QAC3B,MAAM,KAAK,GAAG,OAAO;aAClB,GAAG,CAAC,CAAC,IAAI,EAAE,EAAE;YACZ,IAAI,OAAO,IAAI,KAAK,QAAQ,EAAE,CAAC;gBAC7B,OAAO,IAAI,CAAC;YACd,CAAC;YACD,IAAI,OAAO,IAAI,KAAK,QAAQ,IAAI,IAAI,KAAK,IAAI,IAAI,MAAM,IAAI,IAAI,EAAE,CAAC;gBAChE,MAAM,IAAI,GAAI,IAA2B,CAAC,IAAI,CAAC;gBAC/C,OAAO,OAAO,IAAI,KAAK,QAAQ,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC;YAC9C,CAAC;YACD,OAAO,EAAE,CAAC;QACZ,CAAC,CAAC;aACD,IAAI,CAAC,EAAE,CAAC,CAAC;QACZ,OAAO,KAAK,IAAI,SAAS,CAAC;IAC5B,CAAC;IAED,OAAO,SAAS,CAAC;AACnB,CAAC;AAED,SAAS,kBAAkB,CAAC,OAAgB;IAC1C,IAAI,OAAO,OAAO,KAAK,QAAQ,IAAI,OAAO,KAAK,IAAI,EAAE,CAAC;QACpD,OAAO,SAAS,CAAC;IACnB,CAAC;IAED,MAAM,MAAM,GAAG,OAAgF,CAAC;IAChG,OAAO,CACL,kBAAkB,CAAC,MAAM,CAAC,OAAO,CAAC;QAClC,kBAAkB,CAAC,MAAM,CAAC,iBAAiB,CAAC;QAC5C,kBAAkB,CAAC,MAAM,CAAC,OAAO,CAAC,CACnC,CAAC;AACJ,CAAC;AAED,SAAS,KAAK,CAAC,EAAU;IACvB,OAAO,IAAI,OAAO,CAAC,CAAC,OAAO,EAAE,EAAE,CAAC,UAAU,CAAC,OAAO,EAAE,EAAE,CAAC,CAAC,CAAC;AAC3D,CAAC;AAED,SAAS,SAAS,CAAC,KAAc;IAC/B,IAAI,OAAO,KAAK,KAAK,QAAQ,IAAI,KAAK,KAAK,IAAI,IAAI,QAAQ,IAAI,KAAK,EAAE,CAAC;QACrE,MAAM,MAAM,GAAI,KAA8B,CAAC,MAAM,CAAC;QACtD,OAAO,OAAO,MAAM,KAAK,QAAQ,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,SAAS,CAAC;IACzD,CAAC;IACD,OAAO,SAAS,CAAC;AACnB,CAAC;AAED,SAAS,eAAe,CAAC,KAAc;IACrC,IAAI,OAAO,KAAK,KAAK,QAAQ,IAAI,KAAK,KAAK,IAAI,IAAI,CAAC,CAAC,SAAS,IAAI,KAAK,CAAC,EAAE,CAAC;QACzE,OAAO,SAAS,CAAC;IACnB,CAAC;IAED,MAAM,OAAO,GAAI,KAA+B,CAAC,OAAO,CAAC;IACzD,MAAM,UAAU,GACd,OAAO,YAAY,OAAO;QACxB,CAAC,CAAC,OAAO,CAAC,GAAG,CAAC,aAAa,CAAC;QAC5B,CAAC,CAAC,OAAO,OAAO,KAAK,QAAQ,IAAI,OAAO,KAAK,IAAI,IAAI,aAAa,IAAI,OAAO;YAC3E,CAAC,CAAE,OAAuC,CAAC,aAAa,CAAC;YACzD,CAAC,CAAC,SAAS,CAAC;IAElB,IAAI,OAAO,UAAU,KAAK,QAAQ,EAAE,CAAC;QACnC,OAAO,SAAS,CAAC;IACnB,CAAC;IAED,MAAM,OAAO,GAAG,MAAM,CAAC,UAAU,CAAC,CAAC;IACnC,IAAI,MAAM,CAAC,QAAQ,CAAC,OAAO,CAAC,IAAI,OAAO,GAAG,CAAC,EAAE,CAAC;QAC5C,OAAO,OAAO,GAAG,IAAI,CAAC;IACxB,CAAC;IAED,MAAM,MAAM,GAAG,IAAI,CAAC,KAAK,CAAC,UAAU,CAAC,CAAC;IACtC,IAAI,MAAM,CAAC,QAAQ,CAAC,MAAM,CAAC,EAAE,CAAC;QAC5B,OAAO,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC,GAAG,EAAE,CAAC,CAAC;IAC1C,CAAC;IAED,OAAO,SAAS,CAAC;AACnB,CAAC;AAED,SAAS,aAAa,CAAC,KAAc,EAAE,OAAe,EAAE,MAAc,EAAE,WAAmB;IACzF,IAAI,OAAO,CAAC,GAAG,CAAC,wBAAwB,KAAK,GAAG,EAAE,CAAC;QACjD,OAAO;IACT,CAAC;IACD,MAAM,MAAM,GAAG,SAAS,CAAC,KAAK,CAAC,CAAC;IAChC,MAAM,KAAK,GAAG,MAAM,KAAK,SAAS,CAAC,CAAC,CAAC,YAAY,CAAC,CAAC,CAAC,UAAU,MAAM,EAAE,CAAC;IACvE,OAAO,CAAC,KAAK,CAAC,sBAAsB,OAAO,GAAG,CAAC,IAAI,WAAW,UAAU,KAAK,aAAa,MAAM,KAAK,CAAC,CAAC;AACzG,CAAC;AAED,MAAM,OAAO,eAAe;IAClB,MAAM,CAAC,YAAY,GAAkB,OAAO,CAAC,OAAO,EAAE,CAAC;IACvD,MAAM,CAAC,aAAa,GAAG,CAAC,CAAC;IAEhB,MAAM,CAAS;IACf,cAAc,CAAS;IACvB,WAAW,CAAS;IACpB,eAAe,CAAS;IAEzC,YAAY,MAAoB;QAC9B,IAAI,CAAC,cAAc,GAAG,MAAM,CAAC,cAAc,CAAC;QAC5C,IAAI,CAAC,WAAW,GAAG,MAAM,CAAC,WAAW,CAAC;QACtC,IAAI,CAAC,eAAe,GAAG,MAAM,CAAC,eAAe,CAAC;QAC9C,IAAI,CAAC,MAAM,GAAG,IAAI,MAAM,CAAC;YACvB,MAAM,EAAE,MAAM,CAAC,MAAM;YACrB,OAAO,EAAE,MAAM,CAAC,OAAO;YACvB,UAAU,EAAE,CAAC;YACb,OAAO,EAAE,MAAM,CAAC,SAAS;SAC1B,CAAC,CAAC;IACL,CAAC;IAEO,KAAK,CAAC,oBAAoB,CAAI,SAA2B;QAC/D,MAAM,eAAe,GAAG,eAAe,CAAC,YAAY,CAAC;QACrD,IAAI,qBAAqB,GAAe,GAAG,EAAE,CAAC,SAAS,CAAC;QACxD,eAAe,CAAC,YAAY,GAAG,IAAI,OAAO,CAAO,CAAC,OAAO,EAAE,EAAE;YAC3D,qBAAqB,GAAG,OAAO,CAAC;QAClC,CAAC,CAAC,CAAC;QAEH,MAAM,eAAe,CAAC;QACtB,IAAI,CAAC;YACH,IAAI,IAAI,CAAC,cAAc,GAAG,CAAC,EAAE,CAAC;gBAC5B,MAAM,SAAS,GAAG,IAAI,CAAC,GAAG,EAAE,GAAG,eAAe,CAAC,aAAa,CAAC;gBAC7D,MAAM,MAAM,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,IAAI,CAAC,cAAc,GAAG,SAAS,CAAC,CAAC;gBAC5D,IAAI,MAAM,GAAG,CAAC,EAAE,CAAC;oBACf,MAAM,KAAK,CAAC,MAAM,CAAC,CAAC;gBACtB,CAAC;YACH,CAAC;YACD,eAAe,CAAC,aAAa,GAAG,IAAI,CAAC,GAAG,EAAE,CAAC;YAC3C,OAAO,MAAM,SAAS,EAAE,CAAC;QAC3B,CAAC;gBAAS,CAAC;YACT,qBAAqB,EAAE,CAAC;QAC1B,CAAC;IACH,CAAC;IAED,KAAK,CAAC,QAAQ,CAAC,OAA0B;QACvC,IAAI,SAAkB,CAAC;QAEvB,KAAK,IAAI,OAAO,GAAG,CAAC,EAAE,OAAO,GAAG,IAAI,CAAC,WAAW,EAAE,OAAO,IAAI,CAAC,EAAE,CAAC;YAC/D,IAAI,CAAC;gBACH,MAAM,SAAS,GAAG,OAAO,CAAC,kBAAkB;oBAC1C,CAAC,CAAC;wBACE,oBAAoB,EAAE,OAAO,CAAC,kBAAkB;qBACjD;oBACH,CAAC,CAAC,SAAS,CAAC;gBACd,MAAM,WAAW,GAAG;oBAClB,KAAK,EAAE,OAAO,CAAC,KAAK;oBACpB,QAAQ,EAAE,OAAO,CAAC,QAAQ;oBAC1B,WAAW,EAAE,OAAO,CAAC,WAAW;oBAChC,UAAU,EAAE,OAAO,CAAC,SAAS;oBAC7B,eAAe,EAAE,OAAO,CAAC,cAAc,CAAC,CAAC,CAAC,EAAE,IAAI,EAAE,OAAO,CAAC,cAAc,EAAE,CAAC,CAAC,CAAC,SAAS;oBACtF,oBAAoB,EAAE,OAAO,CAAC,kBAAkB;oBAChD,UAAU,EAAE,SAAS;oBACrB,MAAM,EAAE,KAAK;iBACd,CAAC;gBACF,MAAM,QAAQ,GAAG,CAAC,MAAM,IAAI,CAAC,oBAAoB,CAAC,GAAG,EAAE,CACrD,IAAI,CAAC,MAAM,CAAC,IAAI,CAAC,WAAW,CAAC,MAAM,CAAC,WAAW,CAAC,CACjD,CAAmB,CAAC;gBAErB,MAAM,OAAO,GAAG,QAAQ,CAAC,OAAO,CAAC,CAAC,CAAC,EAAE,OAAO,CAAC;gBAC7C,MAAM,OAAO,GAAG,kBAAkB,CAAC,OAAO,CAAC,CAAC;gBAC5C,IAAI,OAAO,KAAK,SAAS,EAAE,CAAC;oBAC1B,MAAM,IAAI,GAAG,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,IAAI,CAAC,OAAO,CAAC,CAAC,IAAI,EAAE,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC;oBACtE,MAAM,IAAI,KAAK,CAAC,yEAAyE,IAAI,EAAE,CAAC,CAAC;gBACnG,CAAC;gBAED,OAAO;oBACL,OAAO;oBACP,KAAK,EAAE,QAAQ,CAAC,KAAK;oBACrB,KAAK,EAAE;wBACL,YAAY,EAAE,QAAQ,CAAC,KAAK,EAAE,aAAa;wBAC3C,gBAAgB,EAAE,QAAQ,CAAC,KAAK,EAAE,iBAAiB;wBACnD,WAAW,EAAE,QAAQ,CAAC,KAAK,EAAE,YAAY;qBAC1C;iBACF,CAAC;YACJ,CAAC;YAAC,OAAO,KAAK,EAAE,CAAC;gBACf,SAAS,GAAG,KAAK,CAAC;gBAClB,MAAM,MAAM,GAAG,SAAS,CAAC,KAAK,CAAC,CAAC;gBAChC,MAAM,SAAS,GAAG,MAAM,KAAK,SAAS,IAAI,kBAAkB,CAAC,GAAG,CAAC,MAAM,CAAC,CAAC;gBACzE,IAAI,CAAC,SAAS,IAAI,OAAO,KAAK,IAAI,CAAC,WAAW,GAAG,CAAC,EAAE,CAAC;oBACnD,MAAM;gBACR,CAAC;gBAED,MAAM,YAAY,GAAG,eAAe,CAAC,KAAK,CAAC,CAAC;gBAC5C,MAAM,WAAW,GAAG,MAAM,KAAK,GAAG,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,GAAG,CAAC;gBAChD,MAAM,MAAM,GAAG,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,MAAM,EAAE,GAAG,GAAG,CAAC,CAAC;gBAC/C,MAAM,MAAM,GAAG,YAAY,IAAI,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,eAAe,EAAE,WAAW,GAAG,CAAC,IAAI,OAAO,GAAG,MAAM,CAAC,CAAC;gBACnG,aAAa,CAAC,KAAK,EAAE,OAAO,EAAE,MAAM,EAAE,IAAI,CAAC,WAAW,CAAC,CAAC;gBACxD,MAAM,KAAK,CAAC,MAAM,CAAC,CAAC;YACtB,CAAC;QACH,CAAC;QAED,MAAM,SAAS,YAAY,KAAK,CAAC,CAAC,CAAC,SAAS,CAAC,CAAC,CAAC,IAAI,KAAK,CAAC,MAAM,CAAC,SAAS,CAAC,CAAC,CAAC;IAC9E,CAAC"}
|
package/dist/src/cli.d.ts
CHANGED
|
@@ -1,11 +1,9 @@
|
|
|
1
1
|
import { type EvalOptions } from './eval.js';
|
|
2
|
-
export declare function defaultCheckOutputPath(inputPath: string, now?: Date): string;
|
|
3
2
|
interface CheckOptions {
|
|
4
3
|
evalOptions: EvalOptions;
|
|
5
4
|
json: boolean;
|
|
6
|
-
output
|
|
5
|
+
output?: string;
|
|
7
6
|
}
|
|
8
7
|
export declare function parseCheckOptions(argv: string[], inputIndex?: number): CheckOptions;
|
|
9
|
-
export declare function formatCheckSummary(result: unknown, outputPath: string): string;
|
|
10
8
|
export declare function main(argv: string[]): Promise<void>;
|
|
11
9
|
export {};
|
package/dist/src/cli.js
CHANGED
|
@@ -1,31 +1,38 @@
|
|
|
1
1
|
import { existsSync } from 'node:fs';
|
|
2
|
-
import path from 'node:path';
|
|
3
2
|
import { NvidiaNimClient } from './adapters/nvidia-nim.js';
|
|
3
|
+
import { getConfiguredApiUrl, loadUserConfig, normalizeApiUrl, saveUserConfig } from './config.js';
|
|
4
4
|
import { runCorpus } from './corpus.js';
|
|
5
5
|
import { evalSkill } from './eval.js';
|
|
6
6
|
import { runM0Gate } from './m0/run.js';
|
|
7
7
|
import { runRot } from './rot.js';
|
|
8
|
+
import { formatResultCard, printHelpUi, printBanner, printSetupIntro, promptText, selectSkillPath, startProgress, validateSkillInput } from './ui.js';
|
|
8
9
|
import { verifyResult } from './verify.js';
|
|
9
10
|
function printHelp() {
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
11
|
+
printHelpUi();
|
|
12
|
+
}
|
|
13
|
+
async function ensureCloudConfigured(force = false) {
|
|
14
|
+
if (!force && getConfiguredApiUrl()) {
|
|
15
|
+
return;
|
|
16
|
+
}
|
|
17
|
+
if (!process.stdin.isTTY || !process.stdout.isTTY) {
|
|
18
|
+
throw new Error('Skillcheck setup is required. Run `skillcheck setup` in an interactive terminal.');
|
|
19
|
+
}
|
|
20
|
+
printBanner();
|
|
21
|
+
printSetupIntro();
|
|
22
|
+
for (;;) {
|
|
23
|
+
const value = await promptText('Skillcheck API URL: ');
|
|
24
|
+
try {
|
|
25
|
+
const apiUrl = normalizeApiUrl(value);
|
|
26
|
+
const current = loadUserConfig();
|
|
27
|
+
const filePath = saveUserConfig({ ...current, apiUrl });
|
|
28
|
+
console.log(`Saved Skillcheck API URL to ${filePath}\n`);
|
|
29
|
+
return;
|
|
30
|
+
}
|
|
31
|
+
catch (error) {
|
|
32
|
+
const message = error instanceof Error ? error.message : String(error);
|
|
33
|
+
console.log(`${message}\n`);
|
|
34
|
+
}
|
|
35
|
+
}
|
|
29
36
|
}
|
|
30
37
|
function readOption(argv, name) {
|
|
31
38
|
const index = argv.indexOf(name);
|
|
@@ -63,19 +70,6 @@ function readOptionalNumberOption(argv, name) {
|
|
|
63
70
|
function hasFlag(argv, name) {
|
|
64
71
|
return argv.includes(name);
|
|
65
72
|
}
|
|
66
|
-
function slugify(value) {
|
|
67
|
-
const slug = value
|
|
68
|
-
.toLowerCase()
|
|
69
|
-
.replace(/[^a-z0-9]+/g, '-')
|
|
70
|
-
.replace(/^-+|-+$/g, '');
|
|
71
|
-
return slug || 'skill';
|
|
72
|
-
}
|
|
73
|
-
export function defaultCheckOutputPath(inputPath, now = new Date()) {
|
|
74
|
-
const parsed = path.parse(path.resolve(inputPath));
|
|
75
|
-
const name = parsed.name || path.basename(parsed.dir) || 'skill';
|
|
76
|
-
const timestamp = now.toISOString().replace(/[:.]/g, '-');
|
|
77
|
-
return path.join('skillcheck-results', `${slugify(name)}-${timestamp}.json`);
|
|
78
|
-
}
|
|
79
73
|
function parseEvalOptions(argv, inputIndex = 3) {
|
|
80
74
|
const inputPath = argv[inputIndex];
|
|
81
75
|
if (!inputPath || inputPath.startsWith('--')) {
|
|
@@ -102,7 +96,7 @@ export function parseCheckOptions(argv, inputIndex = 3) {
|
|
|
102
96
|
if (!inputPath || inputPath.startsWith('--')) {
|
|
103
97
|
throw new Error('Usage: skillcheck check <path-to-skill-file-or-folder>\nExample: skillcheck check ./SKILL.md');
|
|
104
98
|
}
|
|
105
|
-
const output = readOption(argv, '--output')
|
|
99
|
+
const output = readOption(argv, '--output');
|
|
106
100
|
return {
|
|
107
101
|
evalOptions: {
|
|
108
102
|
inputPath,
|
|
@@ -113,7 +107,8 @@ export function parseCheckOptions(argv, inputIndex = 3) {
|
|
|
113
107
|
runner: readOption(argv, '--runner'),
|
|
114
108
|
grader: readOption(argv, '--grader'),
|
|
115
109
|
generator: readOption(argv, '--generator'),
|
|
116
|
-
taskSuite: readOption(argv, '--task-suite')
|
|
110
|
+
taskSuite: readOption(argv, '--task-suite'),
|
|
111
|
+
saveArtifacts: Boolean(output)
|
|
117
112
|
},
|
|
118
113
|
json: hasFlag(argv, '--json'),
|
|
119
114
|
output
|
|
@@ -144,48 +139,47 @@ function parseCorpusRunOptions(argv) {
|
|
|
144
139
|
limit: readOptionalNumberOption(argv, '--limit')
|
|
145
140
|
};
|
|
146
141
|
}
|
|
147
|
-
function
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
|
|
162
|
-
|
|
163
|
-
? `[${ci[0].toFixed(1)}, ${ci[1].toFixed(1)}] pp`
|
|
164
|
-
: 'n/a';
|
|
165
|
-
return `Skillcheck result
|
|
166
|
-
Skill: ${String(skill.name ?? 'unknown')}
|
|
167
|
-
Verdict: ${String(score.verdict ?? 'unknown')}
|
|
168
|
-
Effect: ${formatNumber(score.effect_pp)} pp, 95% CI ${ciText}
|
|
169
|
-
Pass rate: with skill ${formatPercent(score.with_skill_pass)} vs without skill ${formatPercent(score.no_skill_pass)}
|
|
170
|
-
Token overhead: ${String(score.token_overhead ?? 'n/a')} tokens
|
|
171
|
-
Tasks/trials: ${String(config.tasks ?? 'n/a')} tasks x ${String(config.trials ?? 'n/a')} trials
|
|
172
|
-
Saved JSON: ${outputPath}`;
|
|
173
|
-
}
|
|
174
|
-
async function runCheck(options) {
|
|
175
|
-
console.error(`Checking ${options.evalOptions.inputPath} (${options.evalOptions.tasks} tasks x ${options.evalOptions.trials} trials)...`);
|
|
176
|
-
const result = await evalSkill(options.evalOptions);
|
|
142
|
+
async function runCheck(options, showBanner = true) {
|
|
143
|
+
if (showBanner && !options.json) {
|
|
144
|
+
printBanner();
|
|
145
|
+
}
|
|
146
|
+
await validateSkillInput(options.evalOptions.inputPath);
|
|
147
|
+
await ensureCloudConfigured(false);
|
|
148
|
+
const progress = options.json ? undefined : startProgress();
|
|
149
|
+
let result;
|
|
150
|
+
try {
|
|
151
|
+
result = await evalSkill(options.evalOptions);
|
|
152
|
+
progress?.finish();
|
|
153
|
+
}
|
|
154
|
+
catch (error) {
|
|
155
|
+
progress?.fail();
|
|
156
|
+
throw error;
|
|
157
|
+
}
|
|
177
158
|
if (options.json) {
|
|
178
159
|
console.log(JSON.stringify(result, null, 2));
|
|
179
160
|
return;
|
|
180
161
|
}
|
|
181
|
-
console.log(
|
|
162
|
+
console.log(formatResultCard(result, options.output));
|
|
163
|
+
}
|
|
164
|
+
async function runInteractiveCheck() {
|
|
165
|
+
await ensureCloudConfigured(false);
|
|
166
|
+
const selectedPath = await selectSkillPath();
|
|
167
|
+
await runCheck(parseCheckOptions(['node', 'skillcheck', 'check', selectedPath]), false);
|
|
182
168
|
}
|
|
183
169
|
export async function main(argv) {
|
|
184
170
|
const command = argv[2];
|
|
185
|
-
if (
|
|
171
|
+
if (command === '--help' || command === '-h') {
|
|
186
172
|
printHelp();
|
|
187
173
|
return;
|
|
188
174
|
}
|
|
175
|
+
if (!command) {
|
|
176
|
+
await runInteractiveCheck();
|
|
177
|
+
return;
|
|
178
|
+
}
|
|
179
|
+
if (command === 'setup' || command === 'config') {
|
|
180
|
+
await ensureCloudConfigured(true);
|
|
181
|
+
return;
|
|
182
|
+
}
|
|
189
183
|
if (command === 'm0') {
|
|
190
184
|
const report = await runM0Gate((config) => new NvidiaNimClient(config));
|
|
191
185
|
console.log(JSON.stringify(report, null, 2));
|
package/dist/src/cli.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"cli.js","sourceRoot":"","sources":["../../packages/cli/src/cli.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,UAAU,EAAE,MAAM,SAAS,CAAC;AACrC,OAAO,
|
|
1
|
+
{"version":3,"file":"cli.js","sourceRoot":"","sources":["../../packages/cli/src/cli.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,UAAU,EAAE,MAAM,SAAS,CAAC;AACrC,OAAO,EAAE,eAAe,EAAE,MAAM,0BAA0B,CAAC;AAC3D,OAAO,EAAE,mBAAmB,EAAE,cAAc,EAAE,eAAe,EAAE,cAAc,EAAE,MAAM,aAAa,CAAC;AACnG,OAAO,EAAE,SAAS,EAAyB,MAAM,aAAa,CAAC;AAC/D,OAAO,EAAE,SAAS,EAAoB,MAAM,WAAW,CAAC;AACxD,OAAO,EAAE,SAAS,EAAE,MAAM,aAAa,CAAC;AACxC,OAAO,EAAE,MAAM,EAAmB,MAAM,UAAU,CAAC;AACnD,OAAO,EACL,gBAAgB,EAChB,WAAW,EACX,WAAW,EACX,eAAe,EACf,UAAU,EACV,eAAe,EACf,aAAa,EACb,kBAAkB,EACnB,MAAM,SAAS,CAAC;AACjB,OAAO,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;AAE3C,SAAS,SAAS;IAChB,WAAW,EAAE,CAAC;AAChB,CAAC;AAED,KAAK,UAAU,qBAAqB,CAAC,KAAK,GAAG,KAAK;IAChD,IAAI,CAAC,KAAK,IAAI,mBAAmB,EAAE,EAAE,CAAC;QACpC,OAAO;IACT,CAAC;IACD,IAAI,CAAC,OAAO,CAAC,KAAK,CAAC,KAAK,IAAI,CAAC,OAAO,CAAC,MAAM,CAAC,KAAK,EAAE,CAAC;QAClD,MAAM,IAAI,KAAK,CAAC,kFAAkF,CAAC,CAAC;IACtG,CAAC;IAED,WAAW,EAAE,CAAC;IACd,eAAe,EAAE,CAAC;IAClB,SAAS,CAAC;QACR,MAAM,KAAK,GAAG,MAAM,UAAU,CAAC,sBAAsB,CAAC,CAAC;QACvD,IAAI,CAAC;YACH,MAAM,MAAM,GAAG,eAAe,CAAC,KAAK,CAAC,CAAC;YACtC,MAAM,OAAO,GAAG,cAAc,EAAE,CAAC;YACjC,MAAM,QAAQ,GAAG,cAAc,CAAC,EAAE,GAAG,OAAO,EAAE,MAAM,EAAE,CAAC,CAAC;YACxD,OAAO,CAAC,GAAG,CAAC,+BAA+B,QAAQ,IAAI,CAAC,CAAC;YACzD,OAAO;QACT,CAAC;QAAC,OAAO,KAAK,EAAE,CAAC;YACf,MAAM,OAAO,GAAG,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC;YACvE,OAAO,CAAC,GAAG,CAAC,GAAG,OAAO,IAAI,CAAC,CAAC;QAC9B,CAAC;IACH,CAAC;AACH,CAAC;AAED,SAAS,UAAU,CAAC,IAAc,EAAE,IAAY;IAC9C,MAAM,KAAK,GAAG,IAAI,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC;IACjC,IAAI,KAAK,KAAK,CAAC,CAAC,EAAE,CAAC;QACjB,OAAO,SAAS,CAAC;IACnB,CAAC;IACD,MAAM,KAAK,GAAG,IAAI,CAAC,KAAK,GAAG,CAAC,CAAC,CAAC;IAC9B,IAAI,CAAC,KAAK,IAAI,KAAK,CAAC,UAAU,CAAC,IAAI,CAAC,EAAE,CAAC;QACrC,MAAM,IAAI,KAAK,CAAC,qBAAqB,IAAI,EAAE,CAAC,CAAC;IAC/C,CAAC;IACD,OAAO,KAAK,CAAC;AACf,CAAC;AAED,SAAS,gBAAgB,CAAC,IAAc,EAAE,IAAY,EAAE,QAAgB;IACtE,MAAM,KAAK,GAAG,UAAU,CAAC,IAAI,EAAE,IAAI,CAAC,CAAC;IACrC,IAAI,KAAK,KAAK,SAAS,EAAE,CAAC;QACxB,OAAO,QAAQ,CAAC;IAClB,CAAC;IACD,MAAM,MAAM,GAAG,MAAM,CAAC,KAAK,CAAC,CAAC;IAC7B,IAAI,CAAC,MAAM,CAAC,SAAS,CAAC,MAAM,CAAC,IAAI,MAAM,IAAI,CAAC,EAAE,CAAC;QAC7C,MAAM,IAAI,KAAK,CAAC,GAAG,IAAI,6BAA6B,CAAC,CAAC;IACxD,CAAC;IACD,OAAO,MAAM,CAAC;AAChB,CAAC;AAED,SAAS,wBAAwB,CAAC,IAAc,EAAE,IAAY;IAC5D,MAAM,KAAK,GAAG,UAAU,CAAC,IAAI,EAAE,IAAI,CAAC,CAAC;IACrC,IAAI,KAAK,KAAK,SAAS,EAAE,CAAC;QACxB,OAAO,SAAS,CAAC;IACnB,CAAC;IACD,MAAM,MAAM,GAAG,MAAM,CAAC,KAAK,CAAC,CAAC;IAC7B,IAAI,CAAC,MAAM,CAAC,SAAS,CAAC,MAAM,CAAC,IAAI,MAAM,IAAI,CAAC,EAAE,CAAC;QAC7C,MAAM,IAAI,KAAK,CAAC,GAAG,IAAI,6BAA6B,CAAC,CAAC;IACxD,CAAC;IACD,OAAO,MAAM,CAAC;AAChB,CAAC;AAED,SAAS,OAAO,CAAC,IAAc,EAAE,IAAY;IAC3C,OAAO,IAAI,CAAC,QAAQ,CAAC,IAAI,CAAC,CAAC;AAC7B,CAAC;AAED,SAAS,gBAAgB,CAAC,IAAc,EAAE,UAAU,GAAG,CAAC;IACtD,MAAM,SAAS,GAAG,IAAI,CAAC,UAAU,CAAC,CAAC;IACnC,IAAI,CAAC,SAAS,IAAI,SAAS,CAAC,UAAU,CAAC,IAAI,CAAC,EAAE,CAAC;QAC7C,MAAM,IAAI,KAAK,CAAC,6EAA6E,CAAC,CAAC;IACjG,CAAC;IAED,MAAM,IAAI,GAAG,UAAU,CAAC,IAAI,EAAE,QAAQ,CAAC,IAAI,QAAQ,CAAC;IACpD,IAAI,IAAI,KAAK,QAAQ,EAAE,CAAC;QACtB,MAAM,IAAI,KAAK,CAAC,uCAAuC,CAAC,CAAC;IAC3D,CAAC;IAED,OAAO;QACL,SAAS;QACT,MAAM,EAAE,UAAU,CAAC,IAAI,EAAE,UAAU,CAAC;QACpC,KAAK,EAAE,gBAAgB,CAAC,IAAI,EAAE,SAAS,EAAE,EAAE,CAAC;QAC5C,MAAM,EAAE,gBAAgB,CAAC,IAAI,EAAE,UAAU,EAAE,CAAC,CAAC;QAC7C,IAAI;QACJ,MAAM,EAAE,UAAU,CAAC,IAAI,EAAE,UAAU,CAAC;QACpC,MAAM,EAAE,UAAU,CAAC,IAAI,EAAE,UAAU,CAAC;QACpC,SAAS,EAAE,UAAU,CAAC,IAAI,EAAE,aAAa,CAAC;QAC1C,SAAS,EAAE,UAAU,CAAC,IAAI,EAAE,cAAc,CAAC;KAC5C,CAAC;AACJ,CAAC;AAQD,MAAM,UAAU,iBAAiB,CAAC,IAAc,EAAE,UAAU,GAAG,CAAC;IAC9D,MAAM,SAAS,GAAG,IAAI,CAAC,UAAU,CAAC,CAAC;IACnC,IAAI,CAAC,SAAS,IAAI,SAAS,CAAC,UAAU,CAAC,IAAI,CAAC,EAAE,CAAC;QAC7C,MAAM,IAAI,KAAK,CACb,8FAA8F,CAC/F,CAAC;IACJ,CAAC;IAED,MAAM,MAAM,GAAG,UAAU,CAAC,IAAI,EAAE,UAAU,CAAC,CAAC;IAC5C,OAAO;QACL,WAAW,EAAE;YACX,SAAS;YACT,MAAM;YACN,KAAK,EAAE,gBAAgB,CAAC,IAAI,EAAE,SAAS,EAAE,CAAC,CAAC;YAC3C,MAAM,EAAE,gBAAgB,CAAC,IAAI,EAAE,UAAU,EAAE,CAAC,CAAC;YAC7C,IAAI,EAAE,QAAQ;YACd,MAAM,EAAE,UAAU,CAAC,IAAI,EAAE,UAAU,CAAC;YACpC,MAAM,EAAE,UAAU,CAAC,IAAI,EAAE,UAAU,CAAC;YACpC,SAAS,EAAE,UAAU,CAAC,IAAI,EAAE,aAAa,CAAC;YAC1C,SAAS,EAAE,UAAU,CAAC,IAAI,EAAE,cAAc,CAAC;YAC3C,aAAa,EAAE,OAAO,CAAC,MAAM,CAAC;SAC/B;QACD,IAAI,EAAE,OAAO,CAAC,IAAI,EAAE,QAAQ,CAAC;QAC7B,MAAM;KACP,CAAC;AACJ,CAAC;AAED,SAAS,eAAe,CAAC,IAAc;IACrC,OAAO;QACL,UAAU,EAAE,UAAU,CAAC,IAAI,EAAE,WAAW,CAAC,IAAI,SAAS;QACtD,MAAM,EAAE,UAAU,CAAC,IAAI,EAAE,UAAU,CAAC,IAAI,yBAAyB;QACjE,KAAK,EAAE,UAAU,CAAC,IAAI,EAAE,SAAS,CAAC;QAClC,MAAM,EAAE,UAAU,CAAC,IAAI,EAAE,UAAU,CAAC;QACpC,KAAK,EAAE,gBAAgB,CAAC,IAAI,EAAE,SAAS,EAAE,EAAE,CAAC;QAC5C,MAAM,EAAE,gBAAgB,CAAC,IAAI,EAAE,UAAU,EAAE,CAAC,CAAC;KAC9C,CAAC;AACJ,CAAC;AAED,SAAS,qBAAqB,CAAC,IAAc;IAC3C,MAAM,MAAM,GAAG,UAAU,CAAC,IAAI,EAAE,UAAU,CAAC,CAAC;IAC5C,IAAI,CAAC,MAAM,EAAE,CAAC;QACZ,MAAM,IAAI,KAAK,CAAC,mEAAmE,CAAC,CAAC;IACvF,CAAC;IAED,OAAO;QACL,MAAM;QACN,SAAS,EAAE,UAAU,CAAC,IAAI,EAAE,WAAW,CAAC,IAAI,gBAAgB;QAC5D,KAAK,EAAE,gBAAgB,CAAC,IAAI,EAAE,SAAS,EAAE,EAAE,CAAC;QAC5C,MAAM,EAAE,gBAAgB,CAAC,IAAI,EAAE,UAAU,EAAE,CAAC,CAAC;QAC7C,WAAW,EAAE,gBAAgB,CAAC,IAAI,EAAE,eAAe,EAAE,CAAC,CAAC;QACvD,MAAM,EAAE,UAAU,CAAC,IAAI,EAAE,UAAU,CAAC;QACpC,KAAK,EAAE,wBAAwB,CAAC,IAAI,EAAE,SAAS,CAAC;KACjD,CAAC;AACJ,CAAC;AAED,KAAK,UAAU,QAAQ,CAAC,OAAqB,EAAE,UAAU,GAAG,IAAI;IAC9D,IAAI,UAAU,IAAI,CAAC,OAAO,CAAC,IAAI,EAAE,CAAC;QAChC,WAAW,EAAE,CAAC;IAChB,CAAC;IACD,MAAM,kBAAkB,CAAC,OAAO,CAAC,WAAW,CAAC,SAAS,CAAC,CAAC;IACxD,MAAM,qBAAqB,CAAC,KAAK,CAAC,CAAC;IACnC,MAAM,QAAQ,GAAG,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,SAAS,CAAC,CAAC,CAAC,aAAa,EAAE,CAAC;IAC5D,IAAI,MAAe,CAAC;IACpB,IAAI,CAAC;QACH,MAAM,GAAG,MAAM,SAAS,CAAC,OAAO,CAAC,WAAW,CAAC,CAAC;QAC9C,QAAQ,EAAE,MAAM,EAAE,CAAC;IACrB,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QACf,QAAQ,EAAE,IAAI,EAAE,CAAC;QACjB,MAAM,KAAK,CAAC;IACd,CAAC;IACD,IAAI,OAAO,CAAC,IAAI,EAAE,CAAC;QACjB,OAAO,CAAC,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,MAAM,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC,CAAC;QAC7C,OAAO;IACT,CAAC;IACD,OAAO,CAAC,GAAG,CAAC,gBAAgB,CAAC,MAAM,EAAE,OAAO,CAAC,MAAM,CAAC,CAAC,CAAC;AACxD,CAAC;AAED,KAAK,UAAU,mBAAmB;IAChC,MAAM,qBAAqB,CAAC,KAAK,CAAC,CAAC;IACnC,MAAM,YAAY,GAAG,MAAM,eAAe,EAAE,CAAC;IAC7C,MAAM,QAAQ,CAAC,iBAAiB,CAAC,CAAC,MAAM,EAAE,YAAY,EAAE,OAAO,EAAE,YAAY,CAAC,CAAC,EAAE,KAAK,CAAC,CAAC;AAC1F,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,IAAI,CAAC,IAAc;IACvC,MAAM,OAAO,GAAG,IAAI,CAAC,CAAC,CAAC,CAAC;IAExB,IAAI,OAAO,KAAK,QAAQ,IAAI,OAAO,KAAK,IAAI,EAAE,CAAC;QAC7C,SAAS,EAAE,CAAC;QACZ,OAAO;IACT,CAAC;IAED,IAAI,CAAC,OAAO,EAAE,CAAC;QACb,MAAM,mBAAmB,EAAE,CAAC;QAC5B,OAAO;IACT,CAAC;IAED,IAAI,OAAO,KAAK,OAAO,IAAI,OAAO,KAAK,QAAQ,EAAE,CAAC;QAChD,MAAM,qBAAqB,CAAC,IAAI,CAAC,CAAC;QAClC,OAAO;IACT,CAAC;IAED,IAAI,OAAO,KAAK,IAAI,EAAE,CAAC;QACrB,MAAM,MAAM,GAAG,MAAM,SAAS,CAAC,CAAC,MAAM,EAAE,EAAE,CAAC,IAAI,eAAe,CAAC,MAAM,CAAC,CAAC,CAAC;QACxE,OAAO,CAAC,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,MAAM,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC,CAAC;QAC7C,OAAO,CAAC,QAAQ,GAAG,MAAM,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;QACzC,OAAO;IACT,CAAC;IAED,IAAI,OAAO,KAAK,OAAO,EAAE,CAAC;QACxB,MAAM,QAAQ,CAAC,iBAAiB,CAAC,IAAI,CAAC,CAAC,CAAC;QACxC,OAAO;IACT,CAAC;IAED,IAAI,OAAO,KAAK,MAAM,EAAE,CAAC;QACvB,MAAM,MAAM,GAAG,MAAM,SAAS,CAAC,gBAAgB,CAAC,IAAI,CAAC,CAAC,CAAC;QACvD,OAAO,CAAC,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,MAAM,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC,CAAC;QAC7C,OAAO;IACT,CAAC;IAED,IAAI,OAAO,KAAK,QAAQ,EAAE,CAAC;QACzB,MAAM,UAAU,GAAG,IAAI,CAAC,CAAC,CAAC,CAAC;QAC3B,IAAI,CAAC,UAAU,IAAI,UAAU,CAAC,UAAU,CAAC,IAAI,CAAC,EAAE,CAAC;YAC/C,MAAM,IAAI,KAAK,CAAC,qDAAqD,CAAC,CAAC;QACzE,CAAC;QACD,MAAM,MAAM,GAAG,MAAM,YAAY,CAAC;YAChC,UAAU;YACV,MAAM,EAAE,gBAAgB,CAAC,IAAI,EAAE,UAAU,EAAE,CAAC,CAAC;SAC9C,CAAC,CAAC;QACH,OAAO,CAAC,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,MAAM,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC,CAAC;QAC7C,OAAO;IACT,CAAC;IAED,IAAI,OAAO,KAAK,QAAQ,EAAE,CAAC;QACzB,MAAM,UAAU,GAAG,IAAI,CAAC,CAAC,CAAC,CAAC;QAC3B,IAAI,UAAU,KAAK,KAAK,EAAE,CAAC;YACzB,MAAM,IAAI,KAAK,CAAC,mEAAmE,CAAC,CAAC;QACvF,CAAC;QACD,MAAM,MAAM,GAAG,MAAM,SAAS,CAAC,qBAAqB,CAAC,IAAI,CAAC,CAAC,CAAC;QAC5D,OAAO,CAAC,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,MAAM,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC,CAAC;QAC7C,OAAO;IACT,CAAC;IAED,IAAI,OAAO,KAAK,KAAK,EAAE,CAAC;QACtB,MAAM,MAAM,GAAG,MAAM,MAAM,CAAC,eAAe,CAAC,IAAI,CAAC,CAAC,CAAC;QACnD,OAAO,CAAC,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,MAAM,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC,CAAC;QAC7C,OAAO;IACT,CAAC;IAED,IAAI,CAAC,OAAO,CAAC,UAAU,CAAC,GAAG,CAAC,IAAI,UAAU,CAAC,OAAO,CAAC,EAAE,CAAC;QACpD,MAAM,QAAQ,CAAC,iBAAiB,CAAC,CAAC,MAAM,EAAE,YAAY,EAAE,OAAO,EAAE,GAAG,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;QACrF,OAAO;IACT,CAAC;IAED,MAAM,IAAI,KAAK,CAAC,oBAAoB,OAAO,EAAE,CAAC,CAAC;AACjD,CAAC"}
|
|
@@ -0,0 +1,10 @@
|
|
|
1
|
+
export interface SkillcheckUserConfig {
|
|
2
|
+
apiUrl?: string;
|
|
3
|
+
token?: string;
|
|
4
|
+
}
|
|
5
|
+
export declare function userConfigPath(): string;
|
|
6
|
+
export declare function normalizeApiUrl(value: string): string;
|
|
7
|
+
export declare function loadUserConfig(): SkillcheckUserConfig;
|
|
8
|
+
export declare function saveUserConfig(config: SkillcheckUserConfig): string;
|
|
9
|
+
export declare function getConfiguredApiUrl(): string | undefined;
|
|
10
|
+
export declare function getConfiguredToken(): string | undefined;
|
|
@@ -0,0 +1,64 @@
|
|
|
1
|
+
import { mkdirSync, readFileSync, writeFileSync } from 'node:fs';
|
|
2
|
+
import { homedir } from 'node:os';
|
|
3
|
+
import path from 'node:path';
|
|
4
|
+
function configDir() {
|
|
5
|
+
if (process.env.SKILLCHECK_CONFIG_DIR?.trim()) {
|
|
6
|
+
return process.env.SKILLCHECK_CONFIG_DIR.trim();
|
|
7
|
+
}
|
|
8
|
+
if (process.env.XDG_CONFIG_HOME?.trim()) {
|
|
9
|
+
return path.join(process.env.XDG_CONFIG_HOME.trim(), 'skillcheck');
|
|
10
|
+
}
|
|
11
|
+
return path.join(homedir(), '.config', 'skillcheck');
|
|
12
|
+
}
|
|
13
|
+
export function userConfigPath() {
|
|
14
|
+
if (process.env.SKILLCHECK_CONFIG?.trim()) {
|
|
15
|
+
return process.env.SKILLCHECK_CONFIG.trim();
|
|
16
|
+
}
|
|
17
|
+
return path.join(configDir(), 'config.json');
|
|
18
|
+
}
|
|
19
|
+
export function normalizeApiUrl(value) {
|
|
20
|
+
const trimmed = value.trim().replace(/\/+$/, '');
|
|
21
|
+
if (!trimmed) {
|
|
22
|
+
throw new Error('Skillcheck API URL cannot be empty.');
|
|
23
|
+
}
|
|
24
|
+
const url = new URL(trimmed);
|
|
25
|
+
if (url.protocol !== 'https:' && url.protocol !== 'http:') {
|
|
26
|
+
throw new Error('Skillcheck API URL must start with https:// or http://.');
|
|
27
|
+
}
|
|
28
|
+
if (url.pathname === '/' || url.pathname === '') {
|
|
29
|
+
url.pathname = '/v1';
|
|
30
|
+
}
|
|
31
|
+
return url.toString().replace(/\/+$/, '');
|
|
32
|
+
}
|
|
33
|
+
export function loadUserConfig() {
|
|
34
|
+
try {
|
|
35
|
+
const parsed = JSON.parse(readFileSync(userConfigPath(), 'utf8'));
|
|
36
|
+
if (typeof parsed !== 'object' || parsed === null) {
|
|
37
|
+
return {};
|
|
38
|
+
}
|
|
39
|
+
const record = parsed;
|
|
40
|
+
return {
|
|
41
|
+
apiUrl: typeof record.apiUrl === 'string' ? record.apiUrl : undefined,
|
|
42
|
+
token: typeof record.token === 'string' ? record.token : undefined
|
|
43
|
+
};
|
|
44
|
+
}
|
|
45
|
+
catch (error) {
|
|
46
|
+
if (error.code === 'ENOENT') {
|
|
47
|
+
return {};
|
|
48
|
+
}
|
|
49
|
+
return {};
|
|
50
|
+
}
|
|
51
|
+
}
|
|
52
|
+
export function saveUserConfig(config) {
|
|
53
|
+
const filePath = userConfigPath();
|
|
54
|
+
mkdirSync(path.dirname(filePath), { recursive: true, mode: 0o700 });
|
|
55
|
+
writeFileSync(filePath, `${JSON.stringify(config, null, 2)}\n`, { mode: 0o600 });
|
|
56
|
+
return filePath;
|
|
57
|
+
}
|
|
58
|
+
export function getConfiguredApiUrl() {
|
|
59
|
+
return process.env.SKILLCHECK_API_URL?.trim() || loadUserConfig().apiUrl;
|
|
60
|
+
}
|
|
61
|
+
export function getConfiguredToken() {
|
|
62
|
+
return process.env.SKILLCHECK_TOKEN?.trim() || process.env.SKILLCHECK_API_KEY?.trim() || loadUserConfig().token;
|
|
63
|
+
}
|
|
64
|
+
//# sourceMappingURL=config.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"config.js","sourceRoot":"","sources":["../../packages/cli/src/config.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,SAAS,EAAE,YAAY,EAAE,aAAa,EAAE,MAAM,SAAS,CAAC;AACjE,OAAO,EAAE,OAAO,EAAE,MAAM,SAAS,CAAC;AAClC,OAAO,IAAI,MAAM,WAAW,CAAC;AAO7B,SAAS,SAAS;IAChB,IAAI,OAAO,CAAC,GAAG,CAAC,qBAAqB,EAAE,IAAI,EAAE,EAAE,CAAC;QAC9C,OAAO,OAAO,CAAC,GAAG,CAAC,qBAAqB,CAAC,IAAI,EAAE,CAAC;IAClD,CAAC;IACD,IAAI,OAAO,CAAC,GAAG,CAAC,eAAe,EAAE,IAAI,EAAE,EAAE,CAAC;QACxC,OAAO,IAAI,CAAC,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,eAAe,CAAC,IAAI,EAAE,EAAE,YAAY,CAAC,CAAC;IACrE,CAAC;IACD,OAAO,IAAI,CAAC,IAAI,CAAC,OAAO,EAAE,EAAE,SAAS,EAAE,YAAY,CAAC,CAAC;AACvD,CAAC;AAED,MAAM,UAAU,cAAc;IAC5B,IAAI,OAAO,CAAC,GAAG,CAAC,iBAAiB,EAAE,IAAI,EAAE,EAAE,CAAC;QAC1C,OAAO,OAAO,CAAC,GAAG,CAAC,iBAAiB,CAAC,IAAI,EAAE,CAAC;IAC9C,CAAC;IACD,OAAO,IAAI,CAAC,IAAI,CAAC,SAAS,EAAE,EAAE,aAAa,CAAC,CAAC;AAC/C,CAAC;AAED,MAAM,UAAU,eAAe,CAAC,KAAa;IAC3C,MAAM,OAAO,GAAG,KAAK,CAAC,IAAI,EAAE,CAAC,OAAO,CAAC,MAAM,EAAE,EAAE,CAAC,CAAC;IACjD,IAAI,CAAC,OAAO,EAAE,CAAC;QACb,MAAM,IAAI,KAAK,CAAC,qCAAqC,CAAC,CAAC;IACzD,CAAC;IACD,MAAM,GAAG,GAAG,IAAI,GAAG,CAAC,OAAO,CAAC,CAAC;IAC7B,IAAI,GAAG,CAAC,QAAQ,KAAK,QAAQ,IAAI,GAAG,CAAC,QAAQ,KAAK,OAAO,EAAE,CAAC;QAC1D,MAAM,IAAI,KAAK,CAAC,yDAAyD,CAAC,CAAC;IAC7E,CAAC;IACD,IAAI,GAAG,CAAC,QAAQ,KAAK,GAAG,IAAI,GAAG,CAAC,QAAQ,KAAK,EAAE,EAAE,CAAC;QAChD,GAAG,CAAC,QAAQ,GAAG,KAAK,CAAC;IACvB,CAAC;IACD,OAAO,GAAG,CAAC,QAAQ,EAAE,CAAC,OAAO,CAAC,MAAM,EAAE,EAAE,CAAC,CAAC;AAC5C,CAAC;AAED,MAAM,UAAU,cAAc;IAC5B,IAAI,CAAC;QACH,MAAM,MAAM,GAAG,IAAI,CAAC,KAAK,CAAC,YAAY,CAAC,cAAc,EAAE,EAAE,MAAM,CAAC,CAAY,CAAC;QAC7E,IAAI,OAAO,MAAM,KAAK,QAAQ,IAAI,MAAM,KAAK,IAAI,EAAE,CAAC;YAClD,OAAO,EAAE,CAAC;QACZ,CAAC;QACD,MAAM,MAAM,GAAG,MAAiC,CAAC;QACjD,OAAO;YACL,MAAM,EAAE,OAAO,MAAM,CAAC,MAAM,KAAK,QAAQ,CAAC,CAAC,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC,CAAC,SAAS;YACrE,KAAK,EAAE,OAAO,MAAM,CAAC,KAAK,KAAK,QAAQ,CAAC,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC,CAAC,SAAS;SACnE,CAAC;IACJ,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QACf,IAAK,KAA+B,CAAC,IAAI,KAAK,QAAQ,EAAE,CAAC;YACvD,OAAO,EAAE,CAAC;QACZ,CAAC;QACD,OAAO,EAAE,CAAC;IACZ,CAAC;AACH,CAAC;AAED,MAAM,UAAU,cAAc,CAAC,MAA4B;IACzD,MAAM,QAAQ,GAAG,cAAc,EAAE,CAAC;IAClC,SAAS,CAAC,IAAI,CAAC,OAAO,CAAC,QAAQ,CAAC,EAAE,EAAE,SAAS,EAAE,IAAI,EAAE,IAAI,EAAE,KAAK,EAAE,CAAC,CAAC;IACpE,aAAa,CAAC,QAAQ,EAAE,GAAG,IAAI,CAAC,SAAS,CAAC,MAAM,EAAE,IAAI,EAAE,CAAC,CAAC,IAAI,EAAE,EAAE,IAAI,EAAE,KAAK,EAAE,CAAC,CAAC;IACjF,OAAO,QAAQ,CAAC;AAClB,CAAC;AAED,MAAM,UAAU,mBAAmB;IACjC,OAAO,OAAO,CAAC,GAAG,CAAC,kBAAkB,EAAE,IAAI,EAAE,IAAI,cAAc,EAAE,CAAC,MAAM,CAAC;AAC3E,CAAC;AAED,MAAM,UAAU,kBAAkB;IAChC,OAAO,OAAO,CAAC,GAAG,CAAC,gBAAgB,EAAE,IAAI,EAAE,IAAI,OAAO,CAAC,GAAG,CAAC,kBAAkB,EAAE,IAAI,EAAE,IAAI,cAAc,EAAE,CAAC,KAAK,CAAC;AAClH,CAAC"}
|
package/dist/src/env.js
CHANGED
|
@@ -1,39 +1,56 @@
|
|
|
1
1
|
import dotenv from 'dotenv';
|
|
2
|
+
import { getConfiguredApiUrl, getConfiguredToken } from './config.js';
|
|
2
3
|
dotenv.config();
|
|
3
|
-
|
|
4
|
+
const DEFAULT_MODEL = 'qwen/qwen3-next-80b-a3b-instruct';
|
|
5
|
+
function readEnv(name) {
|
|
4
6
|
const value = process.env[name]?.trim();
|
|
5
|
-
|
|
6
|
-
|
|
7
|
+
return value || undefined;
|
|
8
|
+
}
|
|
9
|
+
function resolveApiKey(proxyUrl) {
|
|
10
|
+
const nvidiaApiKey = readEnv('NVIDIA_API_KEY');
|
|
11
|
+
if (nvidiaApiKey) {
|
|
12
|
+
return nvidiaApiKey;
|
|
13
|
+
}
|
|
14
|
+
const proxyApiKey = getConfiguredToken();
|
|
15
|
+
if (proxyUrl) {
|
|
16
|
+
return proxyApiKey ?? 'skillcheck-cloud';
|
|
7
17
|
}
|
|
8
|
-
|
|
18
|
+
throw new Error('Skillcheck Cloud is not connected for this workspace. Set SKILLCHECK_API_URL or try again later.');
|
|
19
|
+
}
|
|
20
|
+
function resolveModel(role) {
|
|
21
|
+
return (readEnv(`NVIDIA_${role}_MODEL`) ??
|
|
22
|
+
readEnv(`SKILLCHECK_${role}_MODEL`) ??
|
|
23
|
+
readEnv('SKILLCHECK_MODEL') ??
|
|
24
|
+
DEFAULT_MODEL);
|
|
9
25
|
}
|
|
10
26
|
export function loadNvidiaConfig() {
|
|
11
27
|
const timeoutMs = Number(process.env.NVIDIA_TIMEOUT_MS?.trim() || 120000);
|
|
12
28
|
if (!Number.isFinite(timeoutMs) || timeoutMs <= 0) {
|
|
13
|
-
throw new Error('
|
|
29
|
+
throw new Error('Request timeout must be a positive number when set');
|
|
14
30
|
}
|
|
15
31
|
const requestDelayMs = Number(process.env.NVIDIA_REQUEST_DELAY_MS?.trim() || 5000);
|
|
16
32
|
if (!Number.isFinite(requestDelayMs) || requestDelayMs < 0) {
|
|
17
|
-
throw new Error('
|
|
33
|
+
throw new Error('Request delay must be a non-negative number when set');
|
|
18
34
|
}
|
|
19
35
|
const maxAttempts = Number(process.env.NVIDIA_MAX_ATTEMPTS?.trim() || 8);
|
|
20
36
|
if (!Number.isInteger(maxAttempts) || maxAttempts <= 0) {
|
|
21
|
-
throw new Error('
|
|
37
|
+
throw new Error('Retry attempt budget must be a positive integer when set');
|
|
22
38
|
}
|
|
23
39
|
const maxRetryDelayMs = Number(process.env.NVIDIA_MAX_RETRY_DELAY_MS?.trim() || 60000);
|
|
24
40
|
if (!Number.isFinite(maxRetryDelayMs) || maxRetryDelayMs <= 0) {
|
|
25
|
-
throw new Error('
|
|
41
|
+
throw new Error('Retry delay cap must be a positive number when set');
|
|
26
42
|
}
|
|
43
|
+
const proxyUrl = getConfiguredApiUrl();
|
|
27
44
|
return {
|
|
28
|
-
apiKey:
|
|
29
|
-
baseUrl:
|
|
45
|
+
apiKey: resolveApiKey(proxyUrl),
|
|
46
|
+
baseUrl: proxyUrl ?? readEnv('NVIDIA_BASE_URL') ?? 'https://integrate.api.nvidia.com/v1',
|
|
30
47
|
timeoutMs,
|
|
31
48
|
requestDelayMs,
|
|
32
49
|
maxAttempts,
|
|
33
50
|
maxRetryDelayMs,
|
|
34
|
-
generatorModel:
|
|
35
|
-
graderModel:
|
|
36
|
-
runnerModel:
|
|
51
|
+
generatorModel: resolveModel('GENERATOR'),
|
|
52
|
+
graderModel: resolveModel('GRADER'),
|
|
53
|
+
runnerModel: resolveModel('RUNNER')
|
|
37
54
|
};
|
|
38
55
|
}
|
|
39
56
|
//# sourceMappingURL=env.js.map
|
package/dist/src/env.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"env.js","sourceRoot":"","sources":["../../packages/cli/src/env.ts"],"names":[],"mappings":"AAAA,OAAO,MAAM,MAAM,QAAQ,CAAC;
|
|
1
|
+
{"version":3,"file":"env.js","sourceRoot":"","sources":["../../packages/cli/src/env.ts"],"names":[],"mappings":"AAAA,OAAO,MAAM,MAAM,QAAQ,CAAC;AAC5B,OAAO,EAAE,mBAAmB,EAAE,kBAAkB,EAAE,MAAM,aAAa,CAAC;AAEtE,MAAM,CAAC,MAAM,EAAE,CAAC;AAEhB,MAAM,aAAa,GAAG,kCAAkC,CAAC;AAczD,SAAS,OAAO,CAAC,IAAY;IAC3B,MAAM,KAAK,GAAG,OAAO,CAAC,GAAG,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,CAAC;IACxC,OAAO,KAAK,IAAI,SAAS,CAAC;AAC5B,CAAC;AAED,SAAS,aAAa,CAAC,QAA4B;IACjD,MAAM,YAAY,GAAG,OAAO,CAAC,gBAAgB,CAAC,CAAC;IAC/C,IAAI,YAAY,EAAE,CAAC;QACjB,OAAO,YAAY,CAAC;IACtB,CAAC;IAED,MAAM,WAAW,GAAG,kBAAkB,EAAE,CAAC;IACzC,IAAI,QAAQ,EAAE,CAAC;QACb,OAAO,WAAW,IAAI,kBAAkB,CAAC;IAC3C,CAAC;IAED,MAAM,IAAI,KAAK,CAAC,kGAAkG,CAAC,CAAC;AACtH,CAAC;AAED,SAAS,YAAY,CAAC,IAAuC;IAC3D,OAAO,CACL,OAAO,CAAC,UAAU,IAAI,QAAQ,CAAC;QAC/B,OAAO,CAAC,cAAc,IAAI,QAAQ,CAAC;QACnC,OAAO,CAAC,kBAAkB,CAAC;QAC3B,aAAa,CACd,CAAC;AACJ,CAAC;AAED,MAAM,UAAU,gBAAgB;IAC9B,MAAM,SAAS,GAAG,MAAM,CAAC,OAAO,CAAC,GAAG,CAAC,iBAAiB,EAAE,IAAI,EAAE,IAAI,MAAM,CAAC,CAAC;IAC1E,IAAI,CAAC,MAAM,CAAC,QAAQ,CAAC,SAAS,CAAC,IAAI,SAAS,IAAI,CAAC,EAAE,CAAC;QAClD,MAAM,IAAI,KAAK,CAAC,oDAAoD,CAAC,CAAC;IACxE,CAAC;IAED,MAAM,cAAc,GAAG,MAAM,CAAC,OAAO,CAAC,GAAG,CAAC,uBAAuB,EAAE,IAAI,EAAE,IAAI,IAAI,CAAC,CAAC;IACnF,IAAI,CAAC,MAAM,CAAC,QAAQ,CAAC,cAAc,CAAC,IAAI,cAAc,GAAG,CAAC,EAAE,CAAC;QAC3D,MAAM,IAAI,KAAK,CAAC,sDAAsD,CAAC,CAAC;IAC1E,CAAC;IAED,MAAM,WAAW,GAAG,MAAM,CAAC,OAAO,CAAC,GAAG,CAAC,mBAAmB,EAAE,IAAI,EAAE,IAAI,CAAC,CAAC,CAAC;IACzE,IAAI,CAAC,MAAM,CAAC,SAAS,CAAC,WAAW,CAAC,IAAI,WAAW,IAAI,CAAC,EAAE,CAAC;QACvD,MAAM,IAAI,KAAK,CAAC,0DAA0D,CAAC,CAAC;IAC9E,CAAC;IAED,MAAM,eAAe,GAAG,MAAM,CAAC,OAAO,CAAC,GAAG,CAAC,yBAAyB,EAAE,IAAI,EAAE,IAAI,KAAK,CAAC,CAAC;IACvF,IAAI,CAAC,MAAM,CAAC,QAAQ,CAAC,eAAe,CAAC,IAAI,eAAe,IAAI,CAAC,EAAE,CAAC;QAC9D,MAAM,IAAI,KAAK,CAAC,oDAAoD,CAAC,CAAC;IACxE,CAAC;IAED,MAAM,QAAQ,GAAG,mBAAmB,EAAE,CAAC;IAEvC,OAAO;QACL,MAAM,EAAE,aAAa,CAAC,QAAQ,CAAC;QAC/B,OAAO,EAAE,QAAQ,IAAI,OAAO,CAAC,iBAAiB,CAAC,IAAI,qCAAqC;QACxF,SAAS;QACT,cAAc;QACd,WAAW;QACX,eAAe;QACf,cAAc,EAAE,YAAY,CAAC,WAAW,CAAC;QACzC,WAAW,EAAE,YAAY,CAAC,QAAQ,CAAC;QACnC,WAAW,EAAE,YAAY,CAAC,QAAQ,CAAC;KACpC,CAAC;AACJ,CAAC"}
|