skilld-harness 3.6.3 → 3.6.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +4 -1
- package/dist/index.d.mts.map +1 -1
- package/dist/index.mjs +93 -48
- package/dist/index.mjs.map +1 -1
- package/dist/skills/generate-package-skill/SKILL.md +36 -11
- package/dist/skills/generate-package-skill/assets/harness-request.md +3 -1
- package/dist/skills/generate-package-skill/scripts/extract-blocks.mjs +94 -0
- package/dist/skills/generate-package-skill/scripts/serve-fixture.mjs +43 -19
- package/dist/skills/review-skill/SKILL.md +14 -3
- package/dist/skills/review-skill/assets/harness-request.md +2 -1
- package/dist/skills/skilld/SKILL.md +17 -6
- package/package.json +1 -1
|
@@ -13,15 +13,19 @@ The reader knows the language, the framework, and the domain. Write only what it
|
|
|
13
13
|
Take a package name, directory, or prepared source. Ask for the destination only when it is missing.
|
|
14
14
|
In a monorepo, put the Skill in the published package directory.
|
|
15
15
|
Write one Skill per installed package, unless a second package has distinct users.
|
|
16
|
+
Skip an internal package, whose users are the repository's own packages, such as a shared engine or types package. Report why, and write nothing.
|
|
16
17
|
`assets/` holds the Harness request. A direct run ignores it.
|
|
17
18
|
|
|
18
19
|
## 1. Research
|
|
19
20
|
|
|
20
|
-
1. Record the exact package version.
|
|
21
|
+
1. Record the exact version the destination branch publishes: its `package.json` version and the npm dist-tag that carries it.
|
|
22
|
+
If npm `latest` is an older major, test the branch's version. If the branch is ahead of its last release, read source at that release tag and report the unreleased changes.
|
|
21
23
|
2. Read the manifest, every exported entry point, and the public types.
|
|
24
|
+
For many sibling integrations, read the shared runtime and the most used few, and report which you sampled.
|
|
22
25
|
3. For a framework module, read what setup registers: auto-imports, components, the config key and defaults, hooks, server routes.
|
|
23
26
|
4. If the package wraps, re-exports, or peers on another package, read the types or tagged source of the version the consumer gets, never a default branch.
|
|
24
27
|
5. Read the current official docs and examples, and release notes only for breaking changes and removed APIs.
|
|
28
|
+
When docs cover several frameworks, read the consumer's framework pages first, then the core pages they link.
|
|
25
29
|
|
|
26
30
|
Test an existing Skill's claims; do not copy its layout.
|
|
27
31
|
|
|
@@ -32,18 +36,26 @@ Observed behaviour beats documentation.
|
|
|
32
36
|
1. Create a minimal consumer fixture outside the package source. Install the recorded version, or link the local build.
|
|
33
37
|
Before packing, build the package and its native code (NAPI, WASM); `prepack` can need them.
|
|
34
38
|
Pack with the repository's package manager: `npm pack` leaves pnpm `catalog:` versions.
|
|
35
|
-
|
|
39
|
+
Give the fixture its own `pnpm-workspace.yaml`, so a parent workspace cannot satisfy its imports.
|
|
40
|
+
Install only the package, documented peers, and check tools such as `typescript`. A resolution error is a finding.
|
|
41
|
+
Install each peer at npm `latest`. If `latest` is outside the peer range, test both versions.
|
|
36
42
|
2. Use consumer defaults. The package repository's config and fixtures can turn them off.
|
|
37
43
|
In Nuxt, keep test modules out of `modules/`; Nuxt registers every module there.
|
|
38
44
|
One fixture page can exercise many examples.
|
|
39
45
|
3. Run each example you include. Compare output with the claim: HTML, return values, type errors, build logs, exit codes, report files.
|
|
40
|
-
|
|
46
|
+
Each concrete claim in prose is an example too: a value, count, default, result shape, error text, exit code, or bundle effect. Run it, or report it read from source.
|
|
47
|
+
Run each example with one failure input as well, such as an unreachable URL, a bad token, or a stalled request.
|
|
48
|
+
Run each example under each global setting the Skill describes, such as a staging environment or a site-wide switch.
|
|
49
|
+
For each claim that a setting or dependency is required, run once without it and record what fails.
|
|
50
|
+
Test each mode the package declares: framework modes (SSR, SPA, streaming, dev, build), each condition in `exports`, the oldest runtime in `engines`, and each CLI binary with a config file and with flags. Report other modes untested.
|
|
41
51
|
Wrap every run in `timeout 120`.
|
|
42
52
|
Grep the package for agent and CI detection, such as `CLAUDECODE` or `CI`; unset each variable it reads with `env -u VAR`.
|
|
43
53
|
Read dev warnings in the dev log, prerender results in build output, and runtime results from the production server.
|
|
44
54
|
If a trap says nothing happens, run it and confirm the silence.
|
|
45
55
|
To fetch from a server, run [scripts/serve-fixture.mjs](scripts/serve-fixture.mjs) in the fixture: `node SKILL_DIR/scripts/serve-fixture.mjs --fetch / -- node .output/server/index.mjs`.
|
|
46
|
-
|
|
56
|
+
Quote `'{port}'` in a server argument, since some shells expand it. `--header 'User-Agent: Googlebot/2.1'` sends a header, `Host` included.
|
|
57
|
+
For a binary, use `--fetch-raw PATH --out DIR`. `DIR/responses.json` lists each status, content type, and response headers.
|
|
58
|
+
For an HTTP client package, start a fake target on port 0 inside the test, and assert on the requests it receives.
|
|
47
59
|
Without `--fetch`, it holds the server until SIGTERM. Background it with your tool's option; `&` and `nohup` die with the shell call.
|
|
48
60
|
Never kill by port or with `pkill -f`: another Agent can own that process.
|
|
49
61
|
In a browser, set a desktop user agent; a package can treat `HeadlessChrome` as a bot.
|
|
@@ -77,19 +89,23 @@ Shape:
|
|
|
77
89
|
- Name the package and tested version in the first paragraph, not the frontmatter.
|
|
78
90
|
- Order, dropping empty sections: setup, automatic behaviour, common tasks, integrations, traps, version limits, config, debug.
|
|
79
91
|
- A config example that is a common task goes there; the config section only lists options.
|
|
80
|
-
- Write each code block as a complete
|
|
92
|
+
- Write each code block as a complete file with the imports the framework accepts. Start it with a path comment, such as `// server/api/search.ts`, so it can be extracted.
|
|
93
|
+
- Write the tested version once, in the first paragraph. Other version literals, such as a User-Agent string, go stale.
|
|
81
94
|
- A trap detailed in a reference gets one linking line in traps.
|
|
82
95
|
- Aim for 150 lines and about 2,000 tokens in `SKILL.md`. Never exceed 500 lines.
|
|
96
|
+
Past the aim, keep silent failures before loud ones and common-path traps before rare ones. Merge traps that share a fix. Cut a common-task example before a trap.
|
|
83
97
|
- Keep one file. Move a topic to `references/<topic>.md` only past about 40 lines and when under a third of tasks need it.
|
|
84
98
|
- Link each reference from `SKILL.md`, one level deep. A reference over 100 lines starts with contents.
|
|
85
99
|
- Write at most eight reference files. Add `scripts/` only when running code beats reading it.
|
|
100
|
+
- Skillgen maintains only `SKILL.md` and Markdown under `references/`: at most 9 files and 64 KiB in total. It never updates `scripts/`.
|
|
86
101
|
- Never include credentials, caches, build output, or dependency directories.
|
|
87
102
|
|
|
88
103
|
The frontmatter contains only `name` and `description`.
|
|
89
104
|
The name uses lowercase letters, numbers, and single hyphens, at most 64 characters, and matches the directory.
|
|
90
105
|
For a scoped package, drop the `@` and replace `/` with a hyphen: `@nuxtjs/seo` becomes `nuxtjs-seo`.
|
|
91
106
|
The description, at most 1024 characters in third person, says what the Skill does, then when to use it, in the words a user types: package name, main exports, config key, error symptoms.
|
|
92
|
-
|
|
107
|
+
Write it as one plain line of 300 to 450 characters. Skill loaders parse frontmatter with different YAML parsers, so use no double quotes, backticks, or `%`. Name an error symptom in plain words, never as a quoted message.
|
|
108
|
+
Good: `Adds and debugs Schema.org JSON-LD in Nuxt with nuxt-schema-org. Use when a task mentions structured data, rich results, useSchemaOrg, defineArticle, or the schemaOrg config key.`
|
|
93
109
|
|
|
94
110
|
## 4. Check and report
|
|
95
111
|
|
|
@@ -105,24 +121,33 @@ Never invent evidence or silently skip a required check.
|
|
|
105
121
|
|
|
106
122
|
Check a matching task, an unrelated task, and a missing-input task.
|
|
107
123
|
When available, use a fresh session in each Agent the user asks to support.
|
|
124
|
+
If the user names no Agent, validate the format with `skilld run SKILL_DIR --json` and expect `_tag: Success`. Test the current Agent if it can start a fresh session.
|
|
108
125
|
Record its version, model, task, output, and required permissions.
|
|
109
126
|
Distinguish format validation, discovery, activation, and task completion.
|
|
110
127
|
Report unavailable Agent paths untested. Package example checks do not prove cross-Agent execution.
|
|
111
128
|
|
|
112
|
-
Before finishing,
|
|
129
|
+
Before finishing, extract every block with [scripts/extract-blocks.mjs](scripts/extract-blocks.mjs): `node SKILL_DIR/scripts/extract-blocks.mjs SKILL.md DIR`.
|
|
130
|
+
Copy each block into the fixture and run it unchanged. Never test a copy you edited by hand. Repeat after each edit.
|
|
131
|
+
`--replace https://example.com=http://localhost:PORT` swaps a placeholder. `DIR/blocks.json` maps each block to its line.
|
|
132
|
+
|
|
133
|
+
Confirm:
|
|
113
134
|
|
|
114
135
|
- Each example ran against the recorded version, or is reported untested.
|
|
115
136
|
- Each reference is linked. Delete stale files from an earlier Skill.
|
|
116
|
-
- The frontmatter follows the rules above.
|
|
137
|
+
- The frontmatter follows the rules above, parses with a strict YAML parser, and `skilld run SKILL_DIR --json` returns `_tag: Success`. Fix each failure before you finish.
|
|
117
138
|
|
|
118
139
|
Report to the user:
|
|
119
140
|
|
|
120
141
|
- The files and the tested version.
|
|
121
142
|
- Each untested example, and each mismatch: example, documented result, observed result. These are package bugs.
|
|
143
|
+
Draft them as one issue for the package Repository, ordered by impact: silent wrong results, crashes, doc mismatches, cleanups.
|
|
144
|
+
Give each item expected, observed, a minimal repro with its output, and a source link at the release tag. File it only if asked.
|
|
122
145
|
- The source path or documentation URL behind each version-specific rule.
|
|
123
|
-
- If the
|
|
124
|
-
-
|
|
125
|
-
Replace
|
|
146
|
+
- If the Skill sits inside the published package directory, add its directory to `files` in `package.json`. After one build, list packed files with `npm pack --dry-run --ignore-scripts`; `pnpm pack` rejects that flag.
|
|
147
|
+
- Edit the README beside the Skill's `package.json`. If it already links the skilld.dev page, keep that link. Else add the badge below after the others.
|
|
148
|
+
Replace a `skilld add` tip that names this package in place with the tip below. Else add the tip after the install command.
|
|
149
|
+
Replace `OWNER`, `REPOSITORY`, and `PACKAGE`. Count every `SKILL.md` in the Repository, hidden Agent folders included; the skilld.dev indexer skips test and fixture folders.
|
|
150
|
+
If it holds several Skills, append `/SKILL_NAME` to the page and badge paths. The README omits the run command; the page shows it.
|
|
126
151
|
|
|
127
152
|
```html
|
|
128
153
|
<a href="https://skilld.dev/gh/OWNER/REPOSITORY">
|
|
@@ -10,5 +10,7 @@ Use only the visible prepared source and cited official documentation.
|
|
|
10
10
|
Pin external documentation to the dependency versions in the prepared manifest.
|
|
11
11
|
If the session cannot run an example, list it as untested in your final message.
|
|
12
12
|
List each documentation and behaviour mismatch in your final message.
|
|
13
|
-
Write no files outside `{{OUTPUT_PATH}}`.
|
|
13
|
+
Write no files outside `{{OUTPUT_PATH}}`. Build fixtures in a scratch directory outside it.
|
|
14
|
+
Write only `SKILL.md` and Markdown files under `references/`: at most 9 files and 64 KiB in total. Skillgen rejects other output.
|
|
14
15
|
Finish only after checking every output rule in this Skill.
|
|
16
|
+
After you finish, the Harness checks the output. It returns each failed check to you; fix them in place.
|
|
@@ -0,0 +1,94 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
// Write each fenced code block of a Skill file to its own directory, so every example runs exactly as written.
|
|
3
|
+
//
|
|
4
|
+
// Usage: node extract-blocks.mjs FILE OUT_DIR [--replace FROM=TO]...
|
|
5
|
+
//
|
|
6
|
+
// Block N goes to OUT_DIR/N/. A first line that names a path, such as `// server/api/search.ts`,
|
|
7
|
+
// `# scripts/check.sh`, or `<!-- app/pages/index.vue -->`, sets the file name. Otherwise the file is `block.EXT`.
|
|
8
|
+
// `--replace` swaps a placeholder in every block, such as `https://example.com=http://localhost:3000`.
|
|
9
|
+
// OUT_DIR/blocks.json lists each block with its number, start line, language, and file.
|
|
10
|
+
// The script refuses a path that leaves OUT_DIR. It never runs a block.
|
|
11
|
+
|
|
12
|
+
import { mkdir, readFile, writeFile } from 'node:fs/promises'
|
|
13
|
+
import { dirname, isAbsolute, join, normalize, sep } from 'node:path'
|
|
14
|
+
import process from 'node:process'
|
|
15
|
+
|
|
16
|
+
function usage(message) {
|
|
17
|
+
process.stderr.write(`${message}\nUsage: node extract-blocks.mjs FILE OUT_DIR [--replace FROM=TO]...\n`)
|
|
18
|
+
process.exit(2)
|
|
19
|
+
}
|
|
20
|
+
|
|
21
|
+
const EXTENSIONS = { javascript: 'js', typescript: 'ts', shell: 'sh', bash: 'sh', zsh: 'sh', console: 'sh', yml: 'yaml', jsonc: 'json', markdown: 'md' }
|
|
22
|
+
// A path comment: `// a/b.ts`, `# a/b.sh`, `-- a.sql`, `/* a.css */`, or `<!-- a.vue -->`. The path needs an extension.
|
|
23
|
+
const PATH_COMMENT = /^\s*(?:\/\/|#|--|\/\*|<!--)\s*([\w@.\-/[\]]+\.\w+)\s*(?:(?:\*\/|-->)\s*)?$/
|
|
24
|
+
const FENCE_OPEN = /^ {0,3}(`{3,}|~{3,})\s*([\w+-]*)/
|
|
25
|
+
|
|
26
|
+
function parseArgs(argv) {
|
|
27
|
+
const [file, out, ...rest] = argv
|
|
28
|
+
if (!file || !out)
|
|
29
|
+
usage('Give the Skill file and an output directory.')
|
|
30
|
+
const replacements = []
|
|
31
|
+
for (let index = 0; index < rest.length; index++) {
|
|
32
|
+
if (rest[index] !== '--replace' || rest[index + 1] === undefined)
|
|
33
|
+
usage(`Unknown option: ${rest[index]}`)
|
|
34
|
+
const pair = rest[++index]
|
|
35
|
+
const split = pair.indexOf('=')
|
|
36
|
+
if (split <= 0)
|
|
37
|
+
usage('--replace needs FROM=TO.')
|
|
38
|
+
replacements.push([pair.slice(0, split), pair.slice(split + 1)])
|
|
39
|
+
}
|
|
40
|
+
return { file, out, replacements }
|
|
41
|
+
}
|
|
42
|
+
|
|
43
|
+
/** Fenced blocks with the line their content starts on. Nested shorter fences stay inside the block. */
|
|
44
|
+
function fencedBlocks(markdown) {
|
|
45
|
+
const lines = markdown.split('\n')
|
|
46
|
+
const blocks = []
|
|
47
|
+
for (let index = 0; index < lines.length; index++) {
|
|
48
|
+
const open = FENCE_OPEN.exec(lines[index])
|
|
49
|
+
if (!open)
|
|
50
|
+
continue
|
|
51
|
+
const [, fence, lang] = open
|
|
52
|
+
const start = index + 1
|
|
53
|
+
let end = start
|
|
54
|
+
while (end < lines.length && !new RegExp(`^ {0,3}${fence[0]}{${fence.length},}\\s*$`).test(lines[end]))
|
|
55
|
+
end++
|
|
56
|
+
blocks.push({ line: start + 1, lang: lang.toLowerCase(), body: lines.slice(start, end).join('\n') })
|
|
57
|
+
index = end
|
|
58
|
+
}
|
|
59
|
+
return blocks
|
|
60
|
+
}
|
|
61
|
+
|
|
62
|
+
function fileFor(block) {
|
|
63
|
+
const named = PATH_COMMENT.exec(block.body.split('\n')[0] ?? '')?.[1]
|
|
64
|
+
if (named) {
|
|
65
|
+
const path = normalize(named)
|
|
66
|
+
if (isAbsolute(path) || path === '..' || path.startsWith(`..${sep}`))
|
|
67
|
+
return { _tag: 'Escapes', path: named }
|
|
68
|
+
return { _tag: 'Named', path }
|
|
69
|
+
}
|
|
70
|
+
return { _tag: 'Default', path: `block.${EXTENSIONS[block.lang] ?? (block.lang || 'txt')}` }
|
|
71
|
+
}
|
|
72
|
+
|
|
73
|
+
const { file, out, replacements } = parseArgs(process.argv.slice(2))
|
|
74
|
+
const markdown = await readFile(file, 'utf8').catch(error => usage(`Cannot read ${file}: ${error.message}`))
|
|
75
|
+
const manifest = []
|
|
76
|
+
let status = 0
|
|
77
|
+
for (const [index, block] of fencedBlocks(markdown).entries()) {
|
|
78
|
+
const number = index + 1
|
|
79
|
+
const target = fileFor(block)
|
|
80
|
+
if (target._tag === 'Escapes') {
|
|
81
|
+
process.stderr.write(`block ${number} line ${block.line}: path ${target.path} leaves the output directory; skipped\n`)
|
|
82
|
+
status = 1
|
|
83
|
+
continue
|
|
84
|
+
}
|
|
85
|
+
const content = replacements.reduce((text, [from, to]) => text.replaceAll(from, to), block.body)
|
|
86
|
+
const destination = join(out, String(number), target.path)
|
|
87
|
+
await mkdir(dirname(destination), { recursive: true })
|
|
88
|
+
await writeFile(destination, content.endsWith('\n') ? content : `${content}\n`)
|
|
89
|
+
manifest.push({ block: number, line: block.line, lang: block.lang, file: join(String(number), target.path) })
|
|
90
|
+
process.stderr.write(`block ${number} line ${block.line} ${block.lang || '-'} -> ${join(String(number), target.path)}\n`)
|
|
91
|
+
}
|
|
92
|
+
await mkdir(out, { recursive: true })
|
|
93
|
+
await writeFile(join(out, 'blocks.json'), `${JSON.stringify(manifest, null, 2)}\n`)
|
|
94
|
+
process.exit(status)
|
|
@@ -1,25 +1,27 @@
|
|
|
1
1
|
#!/usr/bin/env node
|
|
2
2
|
// Start a fixture server on a free port, fetch paths, then stop its whole process group.
|
|
3
3
|
//
|
|
4
|
-
// Usage: node serve-fixture.mjs [--fetch PATH]... [--fetch-raw PATH]... [--out DIR] [--hold SECONDS] [--timeout SECONDS] -- COMMAND [ARG]...
|
|
4
|
+
// Usage: node serve-fixture.mjs [--fetch PATH]... [--fetch-raw PATH]... [--header 'NAME: VALUE']... [--out DIR] [--hold SECONDS] [--timeout SECONDS] -- COMMAND [ARG]...
|
|
5
5
|
//
|
|
6
6
|
// The script replaces `{port}` in each argument and sets PORT, NITRO_PORT, and NUXT_PORT.
|
|
7
7
|
// It prints `ready http://localhost:PORT` on stderr when the server answers.
|
|
8
8
|
// `--fetch` asks for HTML. `--fetch-raw` asks for any type, such as an image, and needs `--out`.
|
|
9
|
+
// `--header` sends a request header with every fetch, such as `Host` or `User-Agent`. Node `fetch` drops `Host`; this script does not.
|
|
9
10
|
// Each fetch prints `GET PATH STATUS CONTENT-TYPE BYTES` on stderr. The body goes to stdout, or to DIR when `--out` is set.
|
|
10
|
-
// With `--out`, DIR/responses.json lists each path with its file, status, content type, and
|
|
11
|
+
// With `--out`, DIR/responses.json lists each path with its file, status, content type, byte count, and response headers.
|
|
11
12
|
// Without `--fetch`, or with `--hold`, the server stays up until the hold time ends, the script gets SIGTERM, or its parent exits.
|
|
12
13
|
// The script never kills by port or by name. It stops only the process group it started. POSIX only.
|
|
13
14
|
|
|
14
15
|
import { spawn } from 'node:child_process'
|
|
15
16
|
import { mkdir, writeFile } from 'node:fs/promises'
|
|
17
|
+
import { request as httpRequest } from 'node:http'
|
|
16
18
|
import { createServer } from 'node:net'
|
|
17
19
|
import { join } from 'node:path'
|
|
18
20
|
import process from 'node:process'
|
|
19
21
|
import { setTimeout as delay } from 'node:timers/promises'
|
|
20
22
|
|
|
21
23
|
function usage(message) {
|
|
22
|
-
process.stderr.write(`${message}\nUsage: node serve-fixture.mjs [--fetch PATH]... [--fetch-raw PATH]... [--out DIR] [--hold SECONDS] [--timeout SECONDS] -- COMMAND [ARG]...\n`)
|
|
24
|
+
process.stderr.write(`${message}\nUsage: node serve-fixture.mjs [--fetch PATH]... [--fetch-raw PATH]... [--header 'NAME: VALUE']... [--out DIR] [--hold SECONDS] [--timeout SECONDS] -- COMMAND [ARG]...\n`)
|
|
23
25
|
process.exit(2)
|
|
24
26
|
}
|
|
25
27
|
|
|
@@ -31,7 +33,7 @@ function parseSeconds(flag, value) {
|
|
|
31
33
|
}
|
|
32
34
|
|
|
33
35
|
function parseArgs(argv) {
|
|
34
|
-
const options = { fetch: [], out: undefined, hold: undefined, timeout: 120, command: [] }
|
|
36
|
+
const options = { fetch: [], headers: {}, out: undefined, hold: undefined, timeout: 120, command: [] }
|
|
35
37
|
for (let index = 0; index < argv.length; index++) {
|
|
36
38
|
const flag = argv[index]
|
|
37
39
|
if (flag === '--') {
|
|
@@ -41,16 +43,28 @@ function parseArgs(argv) {
|
|
|
41
43
|
const value = argv[++index]
|
|
42
44
|
if (value === undefined)
|
|
43
45
|
usage(`${flag} needs a value.`)
|
|
44
|
-
if (flag === '--fetch' || flag === '--fetch-raw')
|
|
46
|
+
if (flag === '--fetch' || flag === '--fetch-raw') {
|
|
45
47
|
options.fetch.push({ path: value.startsWith('/') ? value : `/${value}`, raw: flag === '--fetch-raw' })
|
|
46
|
-
|
|
48
|
+
}
|
|
49
|
+
else if (flag === '--header') {
|
|
50
|
+
const split = value.indexOf(':')
|
|
51
|
+
const name = split > 0 ? value.slice(0, split).trim().toLowerCase() : ''
|
|
52
|
+
if (!name || !value.slice(split + 1).trim())
|
|
53
|
+
usage('--header needs NAME: VALUE, such as \'User-Agent: Googlebot/2.1\'.')
|
|
54
|
+
options.headers[name] = value.slice(split + 1).trim()
|
|
55
|
+
}
|
|
56
|
+
else if (flag === '--out') {
|
|
47
57
|
options.out = value
|
|
48
|
-
|
|
58
|
+
}
|
|
59
|
+
else if (flag === '--hold') {
|
|
49
60
|
options.hold = parseSeconds(flag, value)
|
|
50
|
-
|
|
61
|
+
}
|
|
62
|
+
else if (flag === '--timeout') {
|
|
51
63
|
options.timeout = parseSeconds(flag, value)
|
|
52
|
-
|
|
64
|
+
}
|
|
65
|
+
else {
|
|
53
66
|
usage(`Unknown option: ${flag}`)
|
|
67
|
+
}
|
|
54
68
|
}
|
|
55
69
|
if (options.command.length === 0)
|
|
56
70
|
usage('Give the server command after --.')
|
|
@@ -168,13 +182,10 @@ async function waitUntilReady({ path, raw }) {
|
|
|
168
182
|
while (Date.now() < deadline) {
|
|
169
183
|
if (exited)
|
|
170
184
|
return { _tag: 'Exited' }
|
|
171
|
-
const response = await
|
|
185
|
+
const response = await get(path, raw, Math.max(deadline - Date.now(), 1))
|
|
172
186
|
.catch(() => undefined) // Connection refused while the server starts. Retry until the deadline.
|
|
173
|
-
if (response && ![502, 503, 504].includes(response.status))
|
|
174
|
-
await response.body?.cancel()
|
|
187
|
+
if (response && ![502, 503, 504].includes(response.status))
|
|
175
188
|
return { _tag: 'Ready' }
|
|
176
|
-
}
|
|
177
|
-
await response?.body?.cancel()
|
|
178
189
|
await delay(500)
|
|
179
190
|
}
|
|
180
191
|
return { _tag: 'TimedOut' }
|
|
@@ -182,16 +193,29 @@ async function waitUntilReady({ path, raw }) {
|
|
|
182
193
|
|
|
183
194
|
const responses = []
|
|
184
195
|
|
|
196
|
+
// node:http, because `fetch` silently replaces a `Host` header with the origin's host.
|
|
197
|
+
function get(path, raw, timeoutMs) {
|
|
198
|
+
return new Promise((resolve, reject) => {
|
|
199
|
+
const req = httpRequest(`${origin}${path}`, { headers: { accept: raw ? '*/*' : 'text/html', ...options.headers }, signal: AbortSignal.timeout(timeoutMs) }, (response) => {
|
|
200
|
+
const chunks = []
|
|
201
|
+
response.on('data', chunk => chunks.push(chunk))
|
|
202
|
+
response.once('error', reject)
|
|
203
|
+
response.once('end', () => resolve({ status: response.statusCode, headers: response.headers, body: Buffer.concat(chunks) }))
|
|
204
|
+
})
|
|
205
|
+
req.once('error', reject)
|
|
206
|
+
req.end()
|
|
207
|
+
})
|
|
208
|
+
}
|
|
209
|
+
|
|
185
210
|
async function fetchPath({ path, raw }) {
|
|
186
|
-
const
|
|
187
|
-
const
|
|
188
|
-
|
|
189
|
-
process.stderr.write(`GET ${path} ${response.status} ${contentType ?? '-'} ${body.byteLength}\n`)
|
|
211
|
+
const { status, headers, body } = await get(path, raw, options.timeout * 1000)
|
|
212
|
+
const contentType = headers['content-type'] ?? null
|
|
213
|
+
process.stderr.write(`GET ${path} ${status} ${contentType ?? '-'} ${body.byteLength}\n`)
|
|
190
214
|
if (options.out) {
|
|
191
215
|
const file = fileName(path, contentType)
|
|
192
216
|
await mkdir(options.out, { recursive: true })
|
|
193
217
|
await writeFile(join(options.out, file), body)
|
|
194
|
-
responses.push({ path, file, status
|
|
218
|
+
responses.push({ path, file, status, contentType, bytes: body.byteLength, headers })
|
|
195
219
|
await writeFile(join(options.out, 'responses.json'), `${JSON.stringify(responses, null, 2)}\n`)
|
|
196
220
|
}
|
|
197
221
|
else {
|
|
@@ -18,12 +18,23 @@ Review the supplied Skill as an Agent would use it.
|
|
|
18
18
|
7. Check instructions for missing inputs, unclear outcomes, and silent failure paths.
|
|
19
19
|
8. Check commands for destructive scope, credential exposure, and unverified downloads.
|
|
20
20
|
9. Check examples against the cited API or project source. If a runtime is available, run them and report each result that differs from the claim.
|
|
21
|
+
An example is each code block and each checkable prose claim: a default, a list, a count, error text, an exit code, or "X happens when Y".
|
|
22
|
+
Test the exact version the Skill names, never the repository head. Use the toolchain its users run, in each mode the claim covers, such as SSR, dev, and build.
|
|
23
|
+
Install it in an empty scratch directory with its own `pnpm-workspace.yaml`. Prefer a local fixture to a live site. Never use personal credentials.
|
|
24
|
+
For cloud-bound code, typecheck it and run it against in-memory stand-ins. Confirm a failing claim with a second probe before you report it.
|
|
25
|
+
Mark each example executed, read from source, or unverified.
|
|
21
26
|
10. Find repeated prose and material that belongs in a reference.
|
|
22
27
|
11. Find text the reader already knows: domain or framework explanations, generic debug advice, changelog paraphrase, and internals the reader cannot act on.
|
|
23
|
-
12. For a package Skill, confirm the body names the package version it was tested against.
|
|
28
|
+
12. For a package Skill, confirm the body names the package version it was tested against, once, in the first paragraph. Flag other version literals.
|
|
29
|
+
13. Confirm the body covers each export, option, or symptom the description promises.
|
|
24
30
|
|
|
25
|
-
Rank each finding
|
|
26
|
-
|
|
31
|
+
Rank each finding:
|
|
32
|
+
|
|
33
|
+
- `error`: following the Skill gives a wrong result, a failure, or a security gap.
|
|
34
|
+
- `warning`: an Agent would act wrongly on the claim, a failure stays silent, or a needed input is missing.
|
|
35
|
+
- `note`: precision, repetition, structure, or token cost.
|
|
36
|
+
|
|
37
|
+
Give the exact path, the line, and a direct fix.
|
|
27
38
|
Do not rewrite the Skill unless the request asks for changes.
|
|
28
39
|
|
|
29
40
|
For a direct run, present the findings to the user.
|
|
@@ -4,7 +4,8 @@ Review the prepared Skill at `{{SOURCE_PATH}}`.
|
|
|
4
4
|
Write the result to `{{OUTPUT_PATH}}/review.json`.
|
|
5
5
|
|
|
6
6
|
Read this Skill fully before reviewing.
|
|
7
|
-
|
|
7
|
+
`{{OUTPUT_PATH}}` holds only `review.json`. Build fixtures in a scratch directory outside it.
|
|
8
|
+
Start each `message` with the line it concerns, such as `Line 42:`.
|
|
8
9
|
Use this JSON shape:
|
|
9
10
|
|
|
10
11
|
```json
|
|
@@ -111,6 +111,8 @@ An empty list proves nothing. Patterns miss obfuscated code.
|
|
|
111
111
|
|
|
112
112
|
A remote run stops with `BEHAVIOR_CONFIRMATION_REQUIRED` when the Skill has an `ask` behavior.
|
|
113
113
|
skilld loaded nothing. Show the user every behavior in `error.message`.
|
|
114
|
+
A match can carry a model reading, such as `(model reading: quoted example. REASON)`.
|
|
115
|
+
Show it as a language model's reading. It never replaces the user's approval.
|
|
114
116
|
If the user approves, run the command at the end of the message and add `--json`.
|
|
115
117
|
Never add `--allow` to any command without the user's approval in this session.
|
|
116
118
|
If the user declines, stop and load nothing.
|
|
@@ -265,19 +267,28 @@ skilld install skilld --global
|
|
|
265
267
|
Install every Skill a Repository, curator, or collection names:
|
|
266
268
|
|
|
267
269
|
```sh
|
|
268
|
-
skilld add OWNER/REPOSITORY
|
|
269
|
-
skilld add @LOGIN/SLUG --global
|
|
270
|
+
skilld add OWNER/REPOSITORY --all
|
|
271
|
+
skilld add @LOGIN/SLUG --all --global
|
|
270
272
|
```
|
|
271
273
|
|
|
272
274
|
`skilld add` accepts `--global`, `--agent`, and `--mode` like `skilld install`.
|
|
273
275
|
It prints one `Installed Skill` line per Skill.
|
|
274
|
-
|
|
275
|
-
A
|
|
276
|
-
|
|
276
|
+
If several Skills are listed, an Agent run requires `--all`.
|
|
277
|
+
A normal terminal asks which Skills to install. `--plain` also requires `--all` for several Skills.
|
|
278
|
+
One listed Skill installs without a picker. `add` does not support JSON output.
|
|
279
|
+
Run `skilld run` with the same ref first, then confirm the list with the user before using `--all`.
|
|
277
280
|
`skilld add` with one Skill selector installs that Skill like `skilld install`.
|
|
281
|
+
Listing never requests registry indexing.
|
|
282
|
+
Skills discovered through GitHub still use hosted delivery by default.
|
|
283
|
+
If delivery fails, skilld shows an explicit direct installation command where possible.
|
|
284
|
+
Only use direct installation when the user chooses it. Never use it to bypass a verification or policy failure.
|
|
285
|
+
`skilld add OWNER/REPOSITORY --all --direct` lists and installs public GitHub files without skilld.dev.
|
|
286
|
+
It records `unverified`. Curator and collection refs require hosted delivery.
|
|
287
|
+
Behavior approval still applies.
|
|
288
|
+
`skilld add ./PATH` installs one local Skill whose directory contains `SKILL.md`.
|
|
278
289
|
|
|
279
290
|
Always use the source selector shown by `skilld search`.
|
|
280
|
-
After installation, report the Skill name, scope,
|
|
291
|
+
After installation, report the Skill name, scope, installed agent paths, and source status.
|
|
281
292
|
|
|
282
293
|
## Fork a Skill
|
|
283
294
|
|