@icyouo/evt-cli 0.1.1 → 0.1.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +36 -5
- package/README.zh-CN.md +29 -4
- package/package.json +4 -3
- package/scripts/check-api-audit.js +260 -0
- package/skills/evt-api-scanner/SKILL.md +93 -6
- package/skills/evt-flow-generator/SKILL.md +139 -0
- package/skills/evt-profile-generator/SKILL.md +122 -0
- package/src/index.js +7 -0
- package/src/util/args.js +3 -0
- package/test/apiAudit.test.js +137 -0
package/README.md
CHANGED
|
@@ -10,7 +10,7 @@ evt-cli is a general-purpose HTTP CLI for running YAML-defined APIs and workflow
|
|
|
10
10
|
- Flows come only from `flows/*.yaml`.
|
|
11
11
|
- Environments, base URLs, headers, and test fixtures come only from `profiles/*.json`.
|
|
12
12
|
- Endpoint definitions are persisted as YAML. Scanner JSON is only an internal machine-readable handoff.
|
|
13
|
-
-
|
|
13
|
+
- Bundled skills are preferred for project data bootstrap. The built-in regex scanner remains as an endpoint-only fallback for users without an agent or skill workflow.
|
|
14
14
|
- The default data directory is bundled with the CLI package. You can also point to your own project data directory with `--config-root` or `EVT_CLI_ROOT`.
|
|
15
15
|
|
|
16
16
|
## Quick Start
|
|
@@ -78,6 +78,24 @@ evt-cli ships with `data/**/*.example.*` for validation and reference. Real proj
|
|
|
78
78
|
|
|
79
79
|
For compatibility with existing projects, evt also reads `apis/`, `flows/`, `profiles/`, and `scanner.config.json` directly under `--config-root` when those paths exist.
|
|
80
80
|
|
|
81
|
+
## Bootstrap Project Data With Skills
|
|
82
|
+
|
|
83
|
+
evt-cli ships generic example data and bundled skills. A skill-aware agent can
|
|
84
|
+
turn an empty project data directory into runnable project data:
|
|
85
|
+
|
|
86
|
+
1. Use `evt-profile-generator` to create `data/profiles/<env>.json` from source
|
|
87
|
+
config, docs, endpoint schemas, and user-provided test credentials.
|
|
88
|
+
2. Use `evt-api-scanner` to discover API source paths, write
|
|
89
|
+
`data/scanner.json`, and generate `data/apis/*.yaml`.
|
|
90
|
+
3. Use `evt-flow-generator` to create `data/flows/login.yaml`, smoke flows, and
|
|
91
|
+
feature flows from the generated endpoints and product workflow.
|
|
92
|
+
4. Run `evt validate --config-root ./cli`.
|
|
93
|
+
5. Run flows with `--dry-run` first, then against a safe test environment.
|
|
94
|
+
|
|
95
|
+
Profile and flow generation are AI-first tasks because base URLs, headers,
|
|
96
|
+
login state, test accounts, token response paths, and feature sequences are
|
|
97
|
+
project-specific. evt-cli keeps code fallback only for endpoint scanning.
|
|
98
|
+
|
|
81
99
|
## Local Check And Package
|
|
82
100
|
|
|
83
101
|
```bash
|
|
@@ -151,11 +169,17 @@ Each endpoint should include:
|
|
|
151
169
|
- `response`
|
|
152
170
|
- `dangerous: true` when the endpoint is destructive or sensitive
|
|
153
171
|
|
|
154
|
-
##
|
|
172
|
+
## Skills, Scan, Sync, And Coverage
|
|
173
|
+
|
|
174
|
+
evt-cli includes these bundled skills:
|
|
155
175
|
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
|
|
176
|
+
- `skills/evt-profile-generator/SKILL.md`
|
|
177
|
+
- `skills/evt-api-scanner/SKILL.md`
|
|
178
|
+
- `skills/evt-flow-generator/SKILL.md`
|
|
179
|
+
|
|
180
|
+
Agents can use these skills to create profile JSON, discover source
|
|
181
|
+
directories, write scanner config, generate YAML endpoint definitions, create
|
|
182
|
+
flow YAML, and audit coverage.
|
|
159
183
|
|
|
160
184
|
Endpoint definitions are written to YAML under `data/apis/*.yaml`. The scanner
|
|
161
185
|
may use JSON internally, but JSON is not the endpoint definition format.
|
|
@@ -172,6 +196,12 @@ Generate or update YAML endpoint definitions:
|
|
|
172
196
|
evt api sync --config-root ./cli
|
|
173
197
|
```
|
|
174
198
|
|
|
199
|
+
Audit YAML quality after sync:
|
|
200
|
+
|
|
201
|
+
```bash
|
|
202
|
+
evt api audit --config-root ./cli --strict
|
|
203
|
+
```
|
|
204
|
+
|
|
175
205
|
Check YAML coverage:
|
|
176
206
|
|
|
177
207
|
```bash
|
|
@@ -243,6 +273,7 @@ evt profile list
|
|
|
243
273
|
evt api list
|
|
244
274
|
evt api discover --config-root ./cli
|
|
245
275
|
evt api sync --config-root ./cli
|
|
276
|
+
evt api audit --config-root ./cli --strict
|
|
246
277
|
evt api coverage --config-root ./cli
|
|
247
278
|
evt api call todo.list --dry-run
|
|
248
279
|
evt flow run login --profile local
|
package/README.zh-CN.md
CHANGED
|
@@ -10,7 +10,7 @@ evt-cli 是一个通用 HTTP CLI,用 YAML 定义接口、用 YAML 定义 flow
|
|
|
10
10
|
- flow 只来自 `flows/*.yaml`。
|
|
11
11
|
- 环境、base URL、headers、测试 fixtures 只来自 `profiles/*.json`。
|
|
12
12
|
- endpoint 定义持久化格式始终是 YAML。scanner JSON 只作为内部机器中间结果。
|
|
13
|
-
- 内置
|
|
13
|
+
- 内置 skills 优先用于项目 data 初始化。现有正则 scanner 只保留给没有 agent 或 skill workflow 的用户做 endpoint 扫描兜底。
|
|
14
14
|
- 默认数据目录是当前 CLI 包内目录;也可以通过 `--config-root` 或 `EVT_CLI_ROOT` 指向项目自己的数据目录。
|
|
15
15
|
|
|
16
16
|
## 快速开始
|
|
@@ -78,6 +78,18 @@ evt-cli 自带 `data/**/*.example.*`,用于开箱验证和复制参考。真
|
|
|
78
78
|
|
|
79
79
|
为了兼容已有项目,`--config-root` 下直接存在 `apis/`、`flows/`、`profiles/`、`scanner.config.json` 时也会被读取。
|
|
80
80
|
|
|
81
|
+
## 用 Skills 初始化项目 Data
|
|
82
|
+
|
|
83
|
+
evt-cli 自带通用 example data 和内置 skills。支持 skill 的 agent 可以把一个空的项目 data 目录生成成可运行的项目 data:
|
|
84
|
+
|
|
85
|
+
1. 使用 `evt-profile-generator` 根据源码配置、文档、endpoint schema 和用户提供的测试账号生成 `data/profiles/<env>.json`。
|
|
86
|
+
2. 使用 `evt-api-scanner` 发现 API 源码路径,写入 `data/scanner.json`,并生成 `data/apis/*.yaml`。
|
|
87
|
+
3. 使用 `evt-flow-generator` 根据生成的 endpoint 和产品流程生成 `data/flows/login.yaml`、smoke flow 和 feature flow。
|
|
88
|
+
4. 执行 `evt validate --config-root ./cli`。
|
|
89
|
+
5. 先用 `--dry-run` 跑 flow,再在安全测试环境里跑真实请求。
|
|
90
|
+
|
|
91
|
+
profile 和 flow 生成是 AI-first 任务,因为 base URL、headers、登录态、测试账号、token 返回路径和 feature 调用顺序都依赖具体项目语义。evt-cli 只保留 endpoint 扫描的代码兜底。
|
|
92
|
+
|
|
81
93
|
## 本地校验和打包
|
|
82
94
|
|
|
83
95
|
```bash
|
|
@@ -150,10 +162,16 @@ endpoints:
|
|
|
150
162
|
- `response`
|
|
151
163
|
- 必要时标记 `dangerous: true`
|
|
152
164
|
|
|
153
|
-
##
|
|
165
|
+
## Skills、扫描、同步和覆盖率
|
|
166
|
+
|
|
167
|
+
evt-cli 内置这些 skills:
|
|
154
168
|
|
|
155
|
-
|
|
156
|
-
|
|
169
|
+
- `skills/evt-profile-generator/SKILL.md`
|
|
170
|
+
- `skills/evt-api-scanner/SKILL.md`
|
|
171
|
+
- `skills/evt-flow-generator/SKILL.md`
|
|
172
|
+
|
|
173
|
+
agent 可以使用这些 skills 生成 profile JSON、发现源码目录、写入 scanner 配置、
|
|
174
|
+
生成 YAML endpoint 定义、生成 flow YAML,并做覆盖率审计。
|
|
157
175
|
|
|
158
176
|
endpoint 定义会写到 `data/apis/*.yaml`。scanner 可以在内部使用 JSON,但 JSON
|
|
159
177
|
不是 endpoint 的最终定义格式。
|
|
@@ -170,6 +188,12 @@ evt api discover --config-root ./cli
|
|
|
170
188
|
evt api sync --config-root ./cli
|
|
171
189
|
```
|
|
172
190
|
|
|
191
|
+
检查 YAML 质量门禁:
|
|
192
|
+
|
|
193
|
+
```bash
|
|
194
|
+
evt api audit --config-root ./cli --strict
|
|
195
|
+
```
|
|
196
|
+
|
|
173
197
|
检查 YAML 覆盖率:
|
|
174
198
|
|
|
175
199
|
```bash
|
|
@@ -239,6 +263,7 @@ evt profile list
|
|
|
239
263
|
evt api list
|
|
240
264
|
evt api discover --config-root ./cli
|
|
241
265
|
evt api sync --config-root ./cli
|
|
266
|
+
evt api audit --config-root ./cli --strict
|
|
242
267
|
evt api coverage --config-root ./cli
|
|
243
268
|
evt api call todo.list --dry-run
|
|
244
269
|
evt flow run login --profile local
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@icyouo/evt-cli",
|
|
3
|
-
"version": "0.1.
|
|
3
|
+
"version": "0.1.2",
|
|
4
4
|
"description": "Run YAML-defined HTTP APIs and interactive workflows from your terminal.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"repository": {
|
|
@@ -27,10 +27,11 @@
|
|
|
27
27
|
"scripts": {
|
|
28
28
|
"ci:local": "node scripts/local-ci.js",
|
|
29
29
|
"test": "node --test",
|
|
30
|
-
"check": "node --check bin/evt.js && node --check src/index.js && node --check src/api/liveTester.js && node --check scripts/sync-api-coverage.js && node --check scripts/check-api-coverage.js && node --check scripts/local-ci.js && node --check scripts/build-package.js && node --check skills/evt-api-scanner/scripts/discover.js && node --check skills/evt-api-scanner/scripts/scan.js",
|
|
30
|
+
"check": "node --check bin/evt.js && node --check src/index.js && node --check src/api/liveTester.js && node --check scripts/sync-api-coverage.js && node --check scripts/check-api-coverage.js && node --check scripts/check-api-audit.js && node --check scripts/local-ci.js && node --check scripts/build-package.js && node --check skills/evt-api-scanner/scripts/discover.js && node --check skills/evt-api-scanner/scripts/scan.js",
|
|
31
31
|
"package:local": "node scripts/build-package.js",
|
|
32
32
|
"sync:api": "node scripts/sync-api-coverage.js",
|
|
33
|
-
"coverage:api": "node scripts/check-api-coverage.js"
|
|
33
|
+
"coverage:api": "node scripts/check-api-coverage.js",
|
|
34
|
+
"audit:api": "node scripts/check-api-audit.js"
|
|
34
35
|
},
|
|
35
36
|
"engines": {
|
|
36
37
|
"node": ">=20"
|
|
@@ -0,0 +1,260 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
|
|
3
|
+
const { listProfiles, loadApiRegistry, loadProfile } = require("../src/config/loaders");
|
|
4
|
+
const { compareScanToRegistry, scanServices } = require("../src/config/serviceScanner");
|
|
5
|
+
const { setConfigRoot } = require("../src/util/paths");
|
|
6
|
+
|
|
7
|
+
function toNumber(value, fallback) {
|
|
8
|
+
if (value === undefined || value === null || value === "") return fallback;
|
|
9
|
+
const parsed = Number(value);
|
|
10
|
+
if (!Number.isFinite(parsed)) throw new Error(`Expected number, got: ${value}`);
|
|
11
|
+
return parsed;
|
|
12
|
+
}
|
|
13
|
+
|
|
14
|
+
function camelOption(name) {
|
|
15
|
+
return name.replace(/-([a-z])/g, (_, letter) => letter.toUpperCase());
|
|
16
|
+
}
|
|
17
|
+
|
|
18
|
+
function parseAuditOptions(argv = []) {
|
|
19
|
+
const options = {};
|
|
20
|
+
const booleanOptions = new Set(["strict", "preferFallback", "strictSkill"]);
|
|
21
|
+
for (let index = 0; index < argv.length; index += 1) {
|
|
22
|
+
const token = argv[index];
|
|
23
|
+
if (!token.startsWith("--")) continue;
|
|
24
|
+
const raw = token.slice(2);
|
|
25
|
+
const equals = raw.indexOf("=");
|
|
26
|
+
const name = camelOption(equals >= 0 ? raw.slice(0, equals) : raw);
|
|
27
|
+
const inlineValue = equals >= 0 ? raw.slice(equals + 1) : undefined;
|
|
28
|
+
if (booleanOptions.has(name)) {
|
|
29
|
+
options[name] = inlineValue === undefined ? true : inlineValue !== "false";
|
|
30
|
+
} else if (inlineValue !== undefined) {
|
|
31
|
+
options[name] = inlineValue;
|
|
32
|
+
} else {
|
|
33
|
+
index += 1;
|
|
34
|
+
if (index >= argv.length) throw new Error(`--${raw} expects a value`);
|
|
35
|
+
options[name] = argv[index];
|
|
36
|
+
}
|
|
37
|
+
}
|
|
38
|
+
return options;
|
|
39
|
+
}
|
|
40
|
+
|
|
41
|
+
function hasObjectEntries(value) {
|
|
42
|
+
return Boolean(value) && typeof value === "object" && !Array.isArray(value) && Object.keys(value).length > 0;
|
|
43
|
+
}
|
|
44
|
+
|
|
45
|
+
function schemaGroupHasFields(group) {
|
|
46
|
+
return hasObjectEntries(group);
|
|
47
|
+
}
|
|
48
|
+
|
|
49
|
+
function hasMeaningfulSchema(endpoint) {
|
|
50
|
+
const schema = endpoint.schema;
|
|
51
|
+
if (!hasObjectEntries(schema)) return false;
|
|
52
|
+
return schemaGroupHasFields(schema.path) || schemaGroupHasFields(schema.query) || schemaGroupHasFields(schema.body);
|
|
53
|
+
}
|
|
54
|
+
|
|
55
|
+
function hasDetailedResponse(endpoint) {
|
|
56
|
+
const response = endpoint.response;
|
|
57
|
+
if (!hasObjectEntries(response)) return false;
|
|
58
|
+
const data = response.data;
|
|
59
|
+
if (!data || typeof data !== "object" || Array.isArray(data)) return true;
|
|
60
|
+
if (data.type !== "object") return true;
|
|
61
|
+
return Boolean(data.model || data.fields || data.items || response.envelope);
|
|
62
|
+
}
|
|
63
|
+
|
|
64
|
+
function isGenericResponse(endpoint) {
|
|
65
|
+
return hasObjectEntries(endpoint.response) && !hasDetailedResponse(endpoint);
|
|
66
|
+
}
|
|
67
|
+
|
|
68
|
+
function hasUnresolvedPath(endpoint) {
|
|
69
|
+
const value = String(endpoint.path || "");
|
|
70
|
+
return /\$\{|AppConfig|undefined|null/.test(value);
|
|
71
|
+
}
|
|
72
|
+
|
|
73
|
+
function hasUsablePath(endpoint) {
|
|
74
|
+
const value = String(endpoint.path || "");
|
|
75
|
+
return /^https?:\/\//i.test(value) || value.startsWith("/") || Boolean(endpoint.service);
|
|
76
|
+
}
|
|
77
|
+
|
|
78
|
+
function looksPrivate(endpoint) {
|
|
79
|
+
const haystack = `${endpoint.id || ""} ${endpoint.path || ""}`.toLowerCase();
|
|
80
|
+
return /(user|account|asset|balance|order|position|history|withdraw|deposit|transfer|address|invite|identity|auth|password|bind|google|phone|email)/.test(haystack) &&
|
|
81
|
+
!/(login|register|oauth|public|config|country|currency|currencies|banner|language|version|market|ticker|kline|depth|symbol|spotitems|contractitems)/.test(haystack);
|
|
82
|
+
}
|
|
83
|
+
|
|
84
|
+
function looksPublic(endpoint) {
|
|
85
|
+
const haystack = `${endpoint.id || ""} ${endpoint.path || ""}`.toLowerCase();
|
|
86
|
+
return /(login|register|oauth|public|config|country|currency|currencies|banner|language|version|market|ticker|kline|depth|symbol|spotitems|contractitems)/.test(haystack);
|
|
87
|
+
}
|
|
88
|
+
|
|
89
|
+
function looksDangerous(endpoint) {
|
|
90
|
+
const method = String(endpoint.method || "GET").toUpperCase();
|
|
91
|
+
if (method === "GET") return false;
|
|
92
|
+
const haystack = `${endpoint.id || ""} ${endpoint.path || ""}`.toLowerCase();
|
|
93
|
+
return /(create|change|adjust|cancel|close|open|transfer|withdraw|bind|unbind|modify|patch|add|update|delete|disable|identity|set|unlink|margin|leverage|order|password)/.test(haystack) &&
|
|
94
|
+
!/(login|check|send|verify|oauth)/.test(haystack);
|
|
95
|
+
}
|
|
96
|
+
|
|
97
|
+
function endpointList(registry) {
|
|
98
|
+
return Object.values(registry).sort((left, right) => left.id.localeCompare(right.id));
|
|
99
|
+
}
|
|
100
|
+
|
|
101
|
+
function ratio(count, total) {
|
|
102
|
+
return total === 0 ? 0 : count / total;
|
|
103
|
+
}
|
|
104
|
+
|
|
105
|
+
function collectProfileServices() {
|
|
106
|
+
const services = new Set();
|
|
107
|
+
const profiles = [];
|
|
108
|
+
for (const profileName of listProfiles()) {
|
|
109
|
+
const profile = loadProfile(profileName);
|
|
110
|
+
profiles.push(profileName);
|
|
111
|
+
for (const service of Object.keys(profile.baseUrls || {})) {
|
|
112
|
+
services.add(service);
|
|
113
|
+
}
|
|
114
|
+
if (profile.baseUrl) services.add("default");
|
|
115
|
+
}
|
|
116
|
+
return { profiles, services };
|
|
117
|
+
}
|
|
118
|
+
|
|
119
|
+
function summarize(registry, scanned) {
|
|
120
|
+
const endpoints = endpointList(registry);
|
|
121
|
+
const comparison = compareScanToRegistry(scanned, registry);
|
|
122
|
+
const { profiles, services } = collectProfileServices();
|
|
123
|
+
const byMethodPath = new Map();
|
|
124
|
+
for (const endpoint of endpoints) {
|
|
125
|
+
const key = `${String(endpoint.method || "GET").toUpperCase()}:${endpoint.path}`;
|
|
126
|
+
if (!byMethodPath.has(key)) byMethodPath.set(key, []);
|
|
127
|
+
byMethodPath.get(key).push(endpoint.id);
|
|
128
|
+
}
|
|
129
|
+
|
|
130
|
+
const emptySchema = endpoints.filter((endpoint) => !hasMeaningfulSchema(endpoint));
|
|
131
|
+
const missingResponse = endpoints.filter((endpoint) => !hasObjectEntries(endpoint.response));
|
|
132
|
+
const genericResponse = endpoints.filter(isGenericResponse);
|
|
133
|
+
const unresolvedPath = endpoints.filter(hasUnresolvedPath);
|
|
134
|
+
const unusablePath = endpoints.filter((endpoint) => !hasUsablePath(endpoint));
|
|
135
|
+
const unmatchedService = endpoints.filter((endpoint) => endpoint.service && profiles.length > 0 && !services.has(endpoint.service));
|
|
136
|
+
const duplicateMethodPath = Array.from(byMethodPath.entries())
|
|
137
|
+
.filter(([, ids]) => ids.length > 1)
|
|
138
|
+
.map(([key, ids]) => ({ key, ids }));
|
|
139
|
+
const authSuspect = endpoints.filter((endpoint) =>
|
|
140
|
+
(endpoint.auth === false && looksPrivate(endpoint)) ||
|
|
141
|
+
(endpoint.auth === true && looksPublic(endpoint))
|
|
142
|
+
);
|
|
143
|
+
const dangerousSuspect = endpoints.filter((endpoint) => !endpoint.dangerous && looksDangerous(endpoint));
|
|
144
|
+
|
|
145
|
+
return {
|
|
146
|
+
scanned: scanned.length,
|
|
147
|
+
endpoints: endpoints.length,
|
|
148
|
+
covered: comparison.covered.length,
|
|
149
|
+
missing: comparison.missing.map((endpoint) => endpoint.id),
|
|
150
|
+
profiles,
|
|
151
|
+
emptySchema: emptySchema.map((endpoint) => endpoint.id),
|
|
152
|
+
missingResponse: missingResponse.map((endpoint) => endpoint.id),
|
|
153
|
+
genericResponse: genericResponse.map((endpoint) => endpoint.id),
|
|
154
|
+
unresolvedPath: unresolvedPath.map((endpoint) => endpoint.id),
|
|
155
|
+
unusablePath: unusablePath.map((endpoint) => endpoint.id),
|
|
156
|
+
unmatchedService: unmatchedService.map((endpoint) => `${endpoint.id}:${endpoint.service}`),
|
|
157
|
+
duplicateMethodPath,
|
|
158
|
+
authSuspect: authSuspect.map((endpoint) => endpoint.id),
|
|
159
|
+
dangerousSuspect: dangerousSuspect.map((endpoint) => endpoint.id),
|
|
160
|
+
ratios: {
|
|
161
|
+
emptySchema: ratio(emptySchema.length, endpoints.length),
|
|
162
|
+
genericResponse: ratio(genericResponse.length, endpoints.length)
|
|
163
|
+
}
|
|
164
|
+
};
|
|
165
|
+
}
|
|
166
|
+
|
|
167
|
+
function evaluateStrict(summary, options) {
|
|
168
|
+
const maxMissing = toNumber(options.maxMissing, 0);
|
|
169
|
+
const maxEmptySchemaRatio = toNumber(options.maxEmptySchemaRatio, 0.25);
|
|
170
|
+
const maxGenericResponseRatio = toNumber(options.maxGenericResponseRatio, 0.05);
|
|
171
|
+
const maxUnresolvedPath = toNumber(options.maxUnresolvedPath, 0);
|
|
172
|
+
const maxUnmatchedService = toNumber(options.maxUnmatchedService, 0);
|
|
173
|
+
const maxUnusablePath = toNumber(options.maxUnusablePath, 0);
|
|
174
|
+
const maxDuplicateMethodPath = toNumber(options.maxDuplicateMethodPath, 0);
|
|
175
|
+
const failures = [];
|
|
176
|
+
|
|
177
|
+
if (summary.missing.length > maxMissing) failures.push(`missing scan coverage ${summary.missing.length} > ${maxMissing}`);
|
|
178
|
+
if (summary.ratios.emptySchema > maxEmptySchemaRatio) {
|
|
179
|
+
failures.push(`empty schema ratio ${summary.ratios.emptySchema.toFixed(3)} > ${maxEmptySchemaRatio}`);
|
|
180
|
+
}
|
|
181
|
+
if (summary.ratios.genericResponse > maxGenericResponseRatio) {
|
|
182
|
+
failures.push(`generic response ratio ${summary.ratios.genericResponse.toFixed(3)} > ${maxGenericResponseRatio}`);
|
|
183
|
+
}
|
|
184
|
+
if (summary.unresolvedPath.length > maxUnresolvedPath) failures.push(`unresolved paths ${summary.unresolvedPath.length} > ${maxUnresolvedPath}`);
|
|
185
|
+
if (summary.unmatchedService.length > maxUnmatchedService) failures.push(`unmatched services ${summary.unmatchedService.length} > ${maxUnmatchedService}`);
|
|
186
|
+
if (summary.unusablePath.length > maxUnusablePath) failures.push(`unusable paths ${summary.unusablePath.length} > ${maxUnusablePath}`);
|
|
187
|
+
if (summary.duplicateMethodPath.length > maxDuplicateMethodPath) {
|
|
188
|
+
failures.push(`duplicate method+path ${summary.duplicateMethodPath.length} > ${maxDuplicateMethodPath}`);
|
|
189
|
+
}
|
|
190
|
+
if (summary.missingResponse.length > 0) failures.push(`missing responses ${summary.missingResponse.length} > 0`);
|
|
191
|
+
|
|
192
|
+
return failures;
|
|
193
|
+
}
|
|
194
|
+
|
|
195
|
+
function compact(summary) {
|
|
196
|
+
return {
|
|
197
|
+
scanned: summary.scanned,
|
|
198
|
+
endpoints: summary.endpoints,
|
|
199
|
+
covered: summary.covered,
|
|
200
|
+
missing: summary.missing,
|
|
201
|
+
profiles: summary.profiles,
|
|
202
|
+
counts: {
|
|
203
|
+
emptySchema: summary.emptySchema.length,
|
|
204
|
+
missingResponse: summary.missingResponse.length,
|
|
205
|
+
genericResponse: summary.genericResponse.length,
|
|
206
|
+
unresolvedPath: summary.unresolvedPath.length,
|
|
207
|
+
unusablePath: summary.unusablePath.length,
|
|
208
|
+
unmatchedService: summary.unmatchedService.length,
|
|
209
|
+
duplicateMethodPath: summary.duplicateMethodPath.length,
|
|
210
|
+
authSuspect: summary.authSuspect.length,
|
|
211
|
+
dangerousSuspect: summary.dangerousSuspect.length
|
|
212
|
+
},
|
|
213
|
+
ratios: summary.ratios,
|
|
214
|
+
samples: {
|
|
215
|
+
emptySchema: summary.emptySchema.slice(0, 25),
|
|
216
|
+
genericResponse: summary.genericResponse.slice(0, 25),
|
|
217
|
+
unresolvedPath: summary.unresolvedPath.slice(0, 25),
|
|
218
|
+
unmatchedService: summary.unmatchedService.slice(0, 25),
|
|
219
|
+
authSuspect: summary.authSuspect.slice(0, 25),
|
|
220
|
+
dangerousSuspect: summary.dangerousSuspect.slice(0, 25)
|
|
221
|
+
},
|
|
222
|
+
duplicateMethodPath: summary.duplicateMethodPath.slice(0, 25)
|
|
223
|
+
};
|
|
224
|
+
}
|
|
225
|
+
|
|
226
|
+
function runApiAudit(argv = process.argv.slice(2), options = {}) {
|
|
227
|
+
const auditOptions = parseAuditOptions(argv);
|
|
228
|
+
const strict = Boolean(auditOptions.strict || options.strict);
|
|
229
|
+
setConfigRoot(auditOptions.configRoot);
|
|
230
|
+
const registry = loadApiRegistry();
|
|
231
|
+
const scanned = scanServices(auditOptions);
|
|
232
|
+
const summary = summarize(registry, scanned);
|
|
233
|
+
const failures = strict ? evaluateStrict(summary, auditOptions) : [];
|
|
234
|
+
const payload = {
|
|
235
|
+
ok: failures.length === 0,
|
|
236
|
+
strict: Boolean(strict),
|
|
237
|
+
...compact(summary),
|
|
238
|
+
failures
|
|
239
|
+
};
|
|
240
|
+
|
|
241
|
+
if (!options.silent) {
|
|
242
|
+
console.log(JSON.stringify(payload, null, 2));
|
|
243
|
+
}
|
|
244
|
+
if (failures.length > 0 && options.exitOnFailure) process.exit(1);
|
|
245
|
+
return { failed: failures.length > 0, payload, summary };
|
|
246
|
+
}
|
|
247
|
+
|
|
248
|
+
if (require.main === module) {
|
|
249
|
+
runApiAudit(process.argv.slice(2), { exitOnFailure: true });
|
|
250
|
+
}
|
|
251
|
+
|
|
252
|
+
module.exports = {
|
|
253
|
+
runApiAudit,
|
|
254
|
+
summarize,
|
|
255
|
+
evaluateStrict,
|
|
256
|
+
parseAuditOptions,
|
|
257
|
+
hasMeaningfulSchema,
|
|
258
|
+
hasDetailedResponse,
|
|
259
|
+
isGenericResponse
|
|
260
|
+
};
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: evt-api-scanner
|
|
3
|
-
description: Use when an agent needs to discover API source locations, generate evt-cli scanner config, scan a codebase for HTTP endpoints, sync YAML API definitions,
|
|
3
|
+
description: Use when an agent needs to discover API source locations, generate evt-cli scanner config, scan a codebase for HTTP endpoints, sync YAML API definitions, check endpoint coverage, or bootstrap endpoint data for evt-cli projects.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# evt-api-scanner
|
|
@@ -14,24 +14,63 @@ scan, sync, or audit YAML API definitions.
|
|
|
14
14
|
- Prefer the user's explicit `--config-root`.
|
|
15
15
|
- Otherwise use `EVT_CLI_ROOT`.
|
|
16
16
|
- Otherwise use `./cli` from the project root.
|
|
17
|
-
2.
|
|
17
|
+
2. Read any existing non-example API YAML first. Preserve existing endpoint IDs,
|
|
18
|
+
service names, auth flags, schema, response, and dangerous markers unless
|
|
19
|
+
source evidence proves they are wrong.
|
|
20
|
+
3. Discover source locations and write scanner config:
|
|
18
21
|
|
|
19
22
|
```bash
|
|
20
23
|
node skills/evt-api-scanner/scripts/discover.js --root . --config-root ./cli
|
|
21
24
|
```
|
|
22
25
|
|
|
23
|
-
|
|
26
|
+
4. Generate or update endpoint YAML:
|
|
24
27
|
|
|
25
28
|
```bash
|
|
26
29
|
evt api sync --config-root ./cli
|
|
27
30
|
```
|
|
28
31
|
|
|
29
|
-
|
|
32
|
+
5. Enrich the generated YAML from source. Do not stop at scanner skeletons.
|
|
33
|
+
6. Audit YAML quality:
|
|
34
|
+
|
|
35
|
+
```bash
|
|
36
|
+
evt api audit --config-root ./cli --strict
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
If strict audit fails because of empty schemas, generic responses, unresolved
|
|
40
|
+
paths, or unmatched services, use the audit samples as the next edit queue and
|
|
41
|
+
continue enrichment.
|
|
42
|
+
|
|
43
|
+
7. Check YAML coverage:
|
|
30
44
|
|
|
31
45
|
```bash
|
|
32
46
|
evt api coverage --config-root ./cli
|
|
33
47
|
```
|
|
34
48
|
|
|
49
|
+
8. When bootstrapping an empty project, continue with:
|
|
50
|
+
- `evt-profile-generator` to create `data/profiles/*.json`.
|
|
51
|
+
- `evt-flow-generator` to create `data/flows/*.yaml`.
|
|
52
|
+
|
|
53
|
+
## Source Evidence
|
|
54
|
+
|
|
55
|
+
Collect these signals before finalizing YAML:
|
|
56
|
+
|
|
57
|
+
- HTTP service declarations for method, path, body type, and function names.
|
|
58
|
+
- Repository or client call sites for auth requirements, default arguments,
|
|
59
|
+
endpoint aliases, and feature ownership.
|
|
60
|
+
- Request DTOs, method parameters, annotations, or typed request builders for
|
|
61
|
+
`schema.path`, `schema.query`, and `schema.body`.
|
|
62
|
+
- Response DTOs, generic wrappers, serializers, or model declarations for
|
|
63
|
+
`response.envelope`, `response.data.model`, `response.data.type`, and fields.
|
|
64
|
+
- App config, DI modules, HTTP clients, interceptors, or base URL providers for
|
|
65
|
+
`service` names and absolute URL handling.
|
|
66
|
+
- Existing tests, docs, and product flows for dangerous operations and required
|
|
67
|
+
ordering.
|
|
68
|
+
|
|
69
|
+
For Kotlin/KMP projects, inspect service interfaces, repositories, DTO/model
|
|
70
|
+
files, app config, DI modules, and header/interceptor builders. For Swift, JS,
|
|
71
|
+
and Dart projects, inspect equivalent API clients, request models, response
|
|
72
|
+
models, environment config, and auth/header middleware.
|
|
73
|
+
|
|
35
74
|
## Endpoint Format
|
|
36
75
|
|
|
37
76
|
Endpoint definitions are always persisted as YAML under `data/apis/*.yaml` or
|
|
@@ -48,10 +87,58 @@ endpoints:
|
|
|
48
87
|
method: "POST"
|
|
49
88
|
path: "/api/login"
|
|
50
89
|
auth: false
|
|
51
|
-
schema:
|
|
52
|
-
|
|
90
|
+
schema:
|
|
91
|
+
body:
|
|
92
|
+
email:
|
|
93
|
+
type: "string"
|
|
94
|
+
required: true
|
|
95
|
+
response:
|
|
96
|
+
envelope: "Resp"
|
|
97
|
+
data:
|
|
98
|
+
type: "object"
|
|
99
|
+
model: "LoginData"
|
|
100
|
+
fields:
|
|
101
|
+
token: "string"
|
|
102
|
+
user:
|
|
103
|
+
type: "object"
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
## Quality Rules
|
|
107
|
+
|
|
108
|
+
- Endpoint IDs should be stable. Preserve existing IDs. If no YAML exists, use
|
|
109
|
+
source function names exactly and do not add, remove, or normalize prefixes
|
|
110
|
+
such as `get` unless a project convention proves that mapping.
|
|
111
|
+
- Do not leave `schema: {}` when source parameters or request DTOs exist.
|
|
112
|
+
- Do not use generic `response.data.type: object` when a response type or DTO
|
|
113
|
+
can be resolved.
|
|
114
|
+
- For wrapped responses such as `Resp<T>`, record the envelope and the nested
|
|
115
|
+
data model.
|
|
116
|
+
- Preserve path parameters, query parameters embedded in path strings, and
|
|
117
|
+
absolute URLs. Map alternate hosts to `service` when profiles can provide a
|
|
118
|
+
named base URL.
|
|
119
|
+
- Infer `auth` from the HTTP client, repository call path, interceptor, or
|
|
120
|
+
feature state. Public market/config endpoints are usually unauthenticated;
|
|
121
|
+
user/account/order/history endpoints usually require auth. Mark uncertain
|
|
122
|
+
cases in the final report.
|
|
123
|
+
- Mark mutating or irreversible endpoints with `dangerous: true`; examples are
|
|
124
|
+
order creation/cancelation, withdrawal, transfer, account deletion, and
|
|
125
|
+
credential changes.
|
|
126
|
+
- Keep scanner JSON internal. Runtime endpoint data remains YAML.
|
|
127
|
+
|
|
128
|
+
## Completion Check
|
|
129
|
+
|
|
130
|
+
Run these before final answer:
|
|
131
|
+
|
|
132
|
+
```bash
|
|
133
|
+
evt validate --config-root ./cli
|
|
134
|
+
evt api audit --config-root ./cli --strict
|
|
135
|
+
evt api coverage --config-root ./cli
|
|
53
136
|
```
|
|
54
137
|
|
|
138
|
+
Report endpoint count, empty-schema count, generic-response count, auth
|
|
139
|
+
uncertain count, and any endpoints whose ID or path could not be resolved
|
|
140
|
+
confidently.
|
|
141
|
+
|
|
55
142
|
## Fallback
|
|
56
143
|
|
|
57
144
|
evt-cli should prefer this skill scanner when configured. If the skill scanner
|
|
@@ -0,0 +1,139 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: evt-flow-generator
|
|
3
|
+
description: Use when an agent needs to create or update evt-cli YAML flows from API YAML definitions, product workflows, authentication requirements, or test scenarios, including interactive inputs, waits, token cache saves, and feature smoke flows.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# evt-flow-generator
|
|
7
|
+
|
|
8
|
+
Use this skill to produce executable evt-cli flows under `data/flows/*.yaml` or
|
|
9
|
+
legacy `flows/*.yaml`.
|
|
10
|
+
|
|
11
|
+
## Workflow
|
|
12
|
+
|
|
13
|
+
1. Locate the evt config root.
|
|
14
|
+
- Prefer the user's explicit `--config-root`.
|
|
15
|
+
- Otherwise use `EVT_CLI_ROOT`.
|
|
16
|
+
- Otherwise use `./cli` from the project root.
|
|
17
|
+
2. Inspect available endpoints:
|
|
18
|
+
|
|
19
|
+
```bash
|
|
20
|
+
evt api list --config-root ./cli
|
|
21
|
+
evt api show <namespace.endpoint> --config-root ./cli
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
3. Read existing flow examples and product docs/source to identify:
|
|
25
|
+
- login or session refresh sequence
|
|
26
|
+
- token field path in the login response
|
|
27
|
+
- prerequisite calls for feature flows
|
|
28
|
+
- required waits between calls
|
|
29
|
+
- safe smoke-test values from profile fixtures
|
|
30
|
+
4. Read the evt runtime semantics before saving values:
|
|
31
|
+
- each step result contains `response`, the full parsed response
|
|
32
|
+
- each step result contains `data`, the unwrapped `response.data` when present
|
|
33
|
+
- for wrapped APIs, prefer `steps.<id>.response.data.<field>` for cache saves
|
|
34
|
+
5. Write YAML flows such as:
|
|
35
|
+
- `data/flows/login.yaml`
|
|
36
|
+
- `data/flows/smoke.yaml`
|
|
37
|
+
- `data/flows/<feature>.yaml`
|
|
38
|
+
6. Validate:
|
|
39
|
+
|
|
40
|
+
```bash
|
|
41
|
+
evt validate --config-root ./cli
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
## Flow Format
|
|
45
|
+
|
|
46
|
+
```yaml
|
|
47
|
+
name: login
|
|
48
|
+
description: Login and save session token.
|
|
49
|
+
|
|
50
|
+
inputs:
|
|
51
|
+
email:
|
|
52
|
+
prompt: Email
|
|
53
|
+
type: string
|
|
54
|
+
required: true
|
|
55
|
+
password:
|
|
56
|
+
prompt: Password
|
|
57
|
+
type: password
|
|
58
|
+
required: true
|
|
59
|
+
|
|
60
|
+
steps:
|
|
61
|
+
- id: login
|
|
62
|
+
call: auth.login
|
|
63
|
+
body:
|
|
64
|
+
email: "{{inputs.email}}"
|
|
65
|
+
password: "{{inputs.password}}"
|
|
66
|
+
expect:
|
|
67
|
+
path: response.data.token
|
|
68
|
+
exists: true
|
|
69
|
+
|
|
70
|
+
save:
|
|
71
|
+
cache:
|
|
72
|
+
token: "{{steps.login.response.data.token}}"
|
|
73
|
+
required:
|
|
74
|
+
- cache.token
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
Use waits when the backend needs spacing:
|
|
78
|
+
|
|
79
|
+
```yaml
|
|
80
|
+
steps:
|
|
81
|
+
- id: createOrder
|
|
82
|
+
call: trade.createOrder
|
|
83
|
+
body:
|
|
84
|
+
symbol: "{{inputs.symbol}}"
|
|
85
|
+
side: "BUY"
|
|
86
|
+
- wait: 2s
|
|
87
|
+
- id: queryOrder
|
|
88
|
+
call: trade.orderDetail
|
|
89
|
+
query:
|
|
90
|
+
orderId: "{{steps.createOrder.response.data.orderId}}"
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
Use `afterWait` when only one step needs a post-call interval.
|
|
94
|
+
|
|
95
|
+
## Rules
|
|
96
|
+
|
|
97
|
+
- Runtime flows are YAML. Do not persist flow definitions as JSON.
|
|
98
|
+
- Keep bundled `*.example.yaml` generic; write project-specific flows to
|
|
99
|
+
non-example YAML files.
|
|
100
|
+
- Use `inputs` for values a user may need to type dynamically.
|
|
101
|
+
- Use `profile.inputs` by naming flow inputs the same way as profile keys.
|
|
102
|
+
- Use profile-backed values for device/session fields when source does so, for
|
|
103
|
+
example `{{profile.device.fingerprintId}}` instead of a new placeholder input.
|
|
104
|
+
- Use `wait` or `afterWait` for backend intervals; valid values include `500ms`,
|
|
105
|
+
`2s`, and `1m`.
|
|
106
|
+
- Preserve waits found in source, tests, docs, or existing flows.
|
|
107
|
+
- Save login state through `save.cache.token` and `save.required`. For wrapped
|
|
108
|
+
responses, prefer `{{steps.login.response.data.token}}`.
|
|
109
|
+
- Reference previous step outputs with `{{steps.<id>.<path>}}`.
|
|
110
|
+
- Prefer `expect` checks for required response fields when a flow depends on
|
|
111
|
+
them.
|
|
112
|
+
- Put logout last in flows that include logout.
|
|
113
|
+
- For feature flows, include prerequisite reads, state-changing calls, follow-up
|
|
114
|
+
reads, waits, cleanup, and final assertions when the scenario needs them.
|
|
115
|
+
- Use endpoint schema and profile fixtures to build request bodies. Do not
|
|
116
|
+
invent field names that are absent from endpoint YAML or source models.
|
|
117
|
+
- Do not include destructive business actions unless the project or user states
|
|
118
|
+
that the environment is safe for testing.
|
|
119
|
+
|
|
120
|
+
## Completion Check
|
|
121
|
+
|
|
122
|
+
After writing flows, run:
|
|
123
|
+
|
|
124
|
+
```bash
|
|
125
|
+
evt validate --config-root ./cli
|
|
126
|
+
evt flow run login --config-root ./cli --profile <profile> --dry-run
|
|
127
|
+
```
|
|
128
|
+
|
|
129
|
+
For login flows, inspect the dry-run output and verify:
|
|
130
|
+
|
|
131
|
+
- the expected endpoint order is present
|
|
132
|
+
- required waits are present
|
|
133
|
+
- token save path matches the response model
|
|
134
|
+
- headers come from the selected profile
|
|
135
|
+
- no required input is missing
|
|
136
|
+
|
|
137
|
+
When real credentials and a safe test environment are available, run the login
|
|
138
|
+
flow without `--dry-run`, confirm the token is written to cache, then run the
|
|
139
|
+
authenticated smoke or feature flow. Report any step that was only inferred.
|
|
@@ -0,0 +1,122 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: evt-profile-generator
|
|
3
|
+
description: Use when an agent needs to create or update evt-cli profile JSON files from project source, environment config, API YAML schemas, documentation, or user-provided credentials so evt-cli can run real APIs and flows.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# evt-profile-generator
|
|
7
|
+
|
|
8
|
+
Use this skill to produce usable evt-cli profiles under `data/profiles/*.json`
|
|
9
|
+
or legacy `profiles/*.json`.
|
|
10
|
+
|
|
11
|
+
## Workflow
|
|
12
|
+
|
|
13
|
+
1. Locate the evt config root.
|
|
14
|
+
- Prefer the user's explicit `--config-root`.
|
|
15
|
+
- Otherwise use `EVT_CLI_ROOT`.
|
|
16
|
+
- Otherwise use `./cli` from the project root.
|
|
17
|
+
2. Read existing profile examples:
|
|
18
|
+
|
|
19
|
+
```bash
|
|
20
|
+
evt profile list --config-root ./cli
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
3. Inspect project source, docs, and environment files for:
|
|
24
|
+
- base URLs by environment
|
|
25
|
+
- static headers, app version, device/platform headers, tenant headers
|
|
26
|
+
- auth header name and token scheme
|
|
27
|
+
- login inputs and test fixtures required by endpoint schemas
|
|
28
|
+
4. Inspect the project's HTTP runtime before writing headers:
|
|
29
|
+
- environment config and app config
|
|
30
|
+
- DI modules or client factories
|
|
31
|
+
- header builders and interceptors
|
|
32
|
+
- auth/session/cache code
|
|
33
|
+
- platform/device providers
|
|
34
|
+
5. Write real profiles such as `data/profiles/dev.json`,
|
|
35
|
+
`data/profiles/android.dev.json`, or `data/profiles/ios.dev.json`.
|
|
36
|
+
6. Validate:
|
|
37
|
+
|
|
38
|
+
```bash
|
|
39
|
+
evt validate --config-root ./cli
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
## Profile Format
|
|
43
|
+
|
|
44
|
+
Use valid JSON only. Do not add comments.
|
|
45
|
+
|
|
46
|
+
```json
|
|
47
|
+
{
|
|
48
|
+
"name": "dev",
|
|
49
|
+
"env": "dev",
|
|
50
|
+
"platform": "ANDROID",
|
|
51
|
+
"baseUrls": {
|
|
52
|
+
"default": "https://api.dev.example.com"
|
|
53
|
+
},
|
|
54
|
+
"auth": {
|
|
55
|
+
"header": "Authorization",
|
|
56
|
+
"scheme": "bearer"
|
|
57
|
+
},
|
|
58
|
+
"headers": {
|
|
59
|
+
"Accept": "application/json",
|
|
60
|
+
"Content-Type": "application/json",
|
|
61
|
+
"Platform": "{{profile.platform}}"
|
|
62
|
+
},
|
|
63
|
+
"device": {
|
|
64
|
+
"fingerprintId": "dev-fingerprint"
|
|
65
|
+
},
|
|
66
|
+
"inputs": {
|
|
67
|
+
"email": "user@example.com",
|
|
68
|
+
"password": "Password123."
|
|
69
|
+
},
|
|
70
|
+
"fixtures": {
|
|
71
|
+
"defaults": {
|
|
72
|
+
"page": 1,
|
|
73
|
+
"pageSize": 20
|
|
74
|
+
}
|
|
75
|
+
}
|
|
76
|
+
}
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
## Rules
|
|
80
|
+
|
|
81
|
+
- Keep bundled `*.example.json` generic; write project-specific data to
|
|
82
|
+
non-example profile files.
|
|
83
|
+
- Do not invent production secrets. Use user-provided test credentials,
|
|
84
|
+
documented test values, or safe placeholders.
|
|
85
|
+
- Prefer separate profiles when platforms have different headers, device
|
|
86
|
+
fields, or base URLs.
|
|
87
|
+
- Use exact enum casing and header names from source. If source uses `ANDROID`,
|
|
88
|
+
do not write `android`.
|
|
89
|
+
- Include backing fields for every templated header value, such as
|
|
90
|
+
`{{profile.device.fingerprintId}}`.
|
|
91
|
+
- Only include headers sent by the same runtime client. Do not merge unrelated
|
|
92
|
+
web, server, or third-party headers just because they appear elsewhere in the
|
|
93
|
+
repository.
|
|
94
|
+
- Keep static app headers and device headers in the profile; keep dynamic
|
|
95
|
+
per-request headers in endpoint YAML or flow steps.
|
|
96
|
+
- Put values needed by interactive flows in `inputs`.
|
|
97
|
+
- Put reusable endpoint test values in `fixtures.defaults` or grouped fixture
|
|
98
|
+
objects named after the endpoint domain.
|
|
99
|
+
- Configure token injection with `auth.header` and `auth.scheme`; token values
|
|
100
|
+
themselves belong in evt cache after login, not in the profile.
|
|
101
|
+
- If multiple API hosts exist, add named entries in `baseUrls` and set matching
|
|
102
|
+
endpoint metadata only when the API YAML supports it.
|
|
103
|
+
- Make `baseUrls` names match endpoint `service` values exactly.
|
|
104
|
+
- If auth uses a raw token header, set `"scheme": "raw"`. If it uses an
|
|
105
|
+
Authorization bearer token, set `"scheme": "bearer"`.
|
|
106
|
+
- Prefer values from source or docs over guesses. If a required runtime value is
|
|
107
|
+
unknown, leave a realistic placeholder and make the related flow input
|
|
108
|
+
interactive.
|
|
109
|
+
|
|
110
|
+
## Completion Check
|
|
111
|
+
|
|
112
|
+
After writing the profile, run:
|
|
113
|
+
|
|
114
|
+
```bash
|
|
115
|
+
evt validate --config-root ./cli
|
|
116
|
+
evt profile list --config-root ./cli
|
|
117
|
+
```
|
|
118
|
+
|
|
119
|
+
If endpoint YAML and flows already exist, also dry-run login with the generated
|
|
120
|
+
profile and compare the produced request headers with the source header builder.
|
|
121
|
+
Report placeholder credentials, uncertain headers, and base URLs that were not
|
|
122
|
+
found in source.
|
package/src/index.js
CHANGED
|
@@ -20,6 +20,7 @@ const { compareScanToRegistry, scanServices } = require("./config/serviceScanner
|
|
|
20
20
|
const { runLiveApiTest } = require("./api/liveTester");
|
|
21
21
|
const { runSyncApiCoverage } = require("../scripts/sync-api-coverage");
|
|
22
22
|
const { runApiCoverage } = require("../scripts/check-api-coverage");
|
|
23
|
+
const { runApiAudit } = require("../scripts/check-api-audit");
|
|
23
24
|
|
|
24
25
|
function print(value, json = false) {
|
|
25
26
|
if (json || typeof value !== "string") {
|
|
@@ -40,6 +41,7 @@ function usage() {
|
|
|
40
41
|
" evt api scan [--missing] [--scan-config path/to/scanner.json]",
|
|
41
42
|
" evt api sync [--config-root ./cli]",
|
|
42
43
|
" evt api coverage [--config-root ./cli]",
|
|
44
|
+
" evt api audit [--config-root ./cli] [--strict]",
|
|
43
45
|
" evt api call <id> [--profile local] [--set k=v] [--body '{...}'] [--dry-run]",
|
|
44
46
|
" evt api test-all [--profile local] [--include-dangerous] [--only namespace]",
|
|
45
47
|
" evt validate",
|
|
@@ -109,6 +111,11 @@ async function handleApi(tokens) {
|
|
|
109
111
|
if (result.failed) throw new Error("API coverage failed");
|
|
110
112
|
return;
|
|
111
113
|
}
|
|
114
|
+
if (sub === "audit") {
|
|
115
|
+
const result = runApiAudit(tokens.slice(1));
|
|
116
|
+
if (result.failed) throw new Error("API audit failed");
|
|
117
|
+
return;
|
|
118
|
+
}
|
|
112
119
|
|
|
113
120
|
const registry = loadApiRegistry();
|
|
114
121
|
|
package/src/util/args.js
CHANGED
|
@@ -0,0 +1,137 @@
|
|
|
1
|
+
const test = require("node:test");
|
|
2
|
+
const assert = require("node:assert/strict");
|
|
3
|
+
const fs = require("node:fs");
|
|
4
|
+
const os = require("node:os");
|
|
5
|
+
const path = require("node:path");
|
|
6
|
+
const { spawnSync } = require("node:child_process");
|
|
7
|
+
const { runApiAudit } = require("../scripts/check-api-audit");
|
|
8
|
+
|
|
9
|
+
function writeFile(root, relative, content) {
|
|
10
|
+
const file = path.join(root, relative);
|
|
11
|
+
fs.mkdirSync(path.dirname(file), { recursive: true });
|
|
12
|
+
fs.writeFileSync(file, content);
|
|
13
|
+
return file;
|
|
14
|
+
}
|
|
15
|
+
|
|
16
|
+
function fixtureRoot() {
|
|
17
|
+
const root = fs.mkdtempSync(path.join(os.tmpdir(), "evt-api-audit-"));
|
|
18
|
+
writeFile(root, "src/AuthService.kt", `
|
|
19
|
+
class AuthService(val http: Http) {
|
|
20
|
+
suspend fun login(): Resp<LoginData> {
|
|
21
|
+
return http.post("/api/login")
|
|
22
|
+
}
|
|
23
|
+
}
|
|
24
|
+
`);
|
|
25
|
+
writeFile(root, "data/scanner.json", JSON.stringify({
|
|
26
|
+
root,
|
|
27
|
+
targets: [
|
|
28
|
+
{
|
|
29
|
+
language: "kotlin",
|
|
30
|
+
paths: ["src"],
|
|
31
|
+
namespaceStripSuffixes: ["Service"]
|
|
32
|
+
}
|
|
33
|
+
]
|
|
34
|
+
}, null, 2));
|
|
35
|
+
return root;
|
|
36
|
+
}
|
|
37
|
+
|
|
38
|
+
test("api audit flags scanner skeletons in strict mode", () => {
|
|
39
|
+
const root = fixtureRoot();
|
|
40
|
+
writeFile(root, "data/apis/auth.yaml", `
|
|
41
|
+
namespace: auth
|
|
42
|
+
|
|
43
|
+
endpoints:
|
|
44
|
+
login:
|
|
45
|
+
method: "POST"
|
|
46
|
+
path: "/api/login"
|
|
47
|
+
auth: false
|
|
48
|
+
schema: {}
|
|
49
|
+
response:
|
|
50
|
+
data:
|
|
51
|
+
type: "object"
|
|
52
|
+
`);
|
|
53
|
+
|
|
54
|
+
const result = runApiAudit(["--config-root", root, "--strict"], { silent: true });
|
|
55
|
+
|
|
56
|
+
assert.equal(result.failed, true);
|
|
57
|
+
assert.equal(result.payload.counts.emptySchema, 1);
|
|
58
|
+
assert.equal(result.payload.counts.genericResponse, 1);
|
|
59
|
+
assert.match(result.payload.failures.join("\\n"), /empty schema ratio/);
|
|
60
|
+
assert.match(result.payload.failures.join("\\n"), /generic response ratio/);
|
|
61
|
+
});
|
|
62
|
+
|
|
63
|
+
test("api audit passes enriched endpoint definitions", () => {
|
|
64
|
+
const root = fixtureRoot();
|
|
65
|
+
writeFile(root, "data/apis/auth.yaml", `
|
|
66
|
+
namespace: auth
|
|
67
|
+
|
|
68
|
+
endpoints:
|
|
69
|
+
login:
|
|
70
|
+
method: "POST"
|
|
71
|
+
path: "/api/login"
|
|
72
|
+
auth: false
|
|
73
|
+
schema:
|
|
74
|
+
body:
|
|
75
|
+
email:
|
|
76
|
+
type: "string"
|
|
77
|
+
required: true
|
|
78
|
+
password:
|
|
79
|
+
type: "string"
|
|
80
|
+
required: true
|
|
81
|
+
response:
|
|
82
|
+
envelope: "Resp"
|
|
83
|
+
data:
|
|
84
|
+
type: "object"
|
|
85
|
+
model: "LoginData"
|
|
86
|
+
fields:
|
|
87
|
+
token: "string"
|
|
88
|
+
`);
|
|
89
|
+
|
|
90
|
+
const result = runApiAudit(["--config-root", root, "--strict"], { silent: true });
|
|
91
|
+
|
|
92
|
+
assert.equal(result.failed, false);
|
|
93
|
+
assert.equal(result.payload.counts.emptySchema, 0);
|
|
94
|
+
assert.equal(result.payload.counts.genericResponse, 0);
|
|
95
|
+
assert.deepEqual(result.payload.failures, []);
|
|
96
|
+
});
|
|
97
|
+
|
|
98
|
+
test("cli api audit accepts strict as a boolean flag", () => {
|
|
99
|
+
const root = fixtureRoot();
|
|
100
|
+
writeFile(root, "data/apis/auth.yaml", `
|
|
101
|
+
namespace: auth
|
|
102
|
+
|
|
103
|
+
endpoints:
|
|
104
|
+
login:
|
|
105
|
+
method: "POST"
|
|
106
|
+
path: "/api/login"
|
|
107
|
+
auth: false
|
|
108
|
+
schema:
|
|
109
|
+
body:
|
|
110
|
+
email:
|
|
111
|
+
type: "string"
|
|
112
|
+
required: true
|
|
113
|
+
response:
|
|
114
|
+
envelope: "Resp"
|
|
115
|
+
data:
|
|
116
|
+
type: "object"
|
|
117
|
+
model: "LoginData"
|
|
118
|
+
fields:
|
|
119
|
+
token: "string"
|
|
120
|
+
`);
|
|
121
|
+
|
|
122
|
+
const result = spawnSync(process.execPath, [
|
|
123
|
+
path.join(__dirname, "..", "bin", "evt.js"),
|
|
124
|
+
"api",
|
|
125
|
+
"audit",
|
|
126
|
+
"--config-root",
|
|
127
|
+
root,
|
|
128
|
+
"--strict"
|
|
129
|
+
], {
|
|
130
|
+
encoding: "utf8"
|
|
131
|
+
});
|
|
132
|
+
|
|
133
|
+
assert.equal(result.status, 0, result.stderr || result.stdout);
|
|
134
|
+
const payload = JSON.parse(result.stdout);
|
|
135
|
+
assert.equal(payload.ok, true);
|
|
136
|
+
assert.equal(payload.strict, true);
|
|
137
|
+
});
|