okfy-ai 0.1.5 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +18 -3
- package/dist/{chunk-JA6B2QIM.js → chunk-7V2ZN6IS.js} +32 -14
- package/dist/cli.js +600 -23
- package/dist/index.d.ts +2 -1
- package/dist/index.js +3 -1
- package/docs/mcp-clients.md +31 -2
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -4,13 +4,14 @@ Turn docs into agent-readable Open Knowledge Format v0.1-conformant bundles, the
|
|
|
4
4
|
|
|
5
5
|
## Use With Agents
|
|
6
6
|
|
|
7
|
-
|
|
7
|
+
Create a registered source and print a client-ready setup preview:
|
|
8
8
|
|
|
9
9
|
```bash
|
|
10
|
-
npx -y okfy-ai
|
|
11
|
-
npx -y okfy-ai serve stripe --mcp --auto-refresh
|
|
10
|
+
npx -y okfy-ai init stripe https://docs.stripe.com/checkout --client generic --max-pages 100 --max-depth 4
|
|
12
11
|
```
|
|
13
12
|
|
|
13
|
+
`init` prints the MCP launch command, client config, and a first prompt. It does not write client config files by default. The generated launch command will look like `npx -y okfy-ai serve stripe --mcp --auto-refresh`.
|
|
14
|
+
|
|
14
15
|
The MCP server uses the cached local bundle immediately. When the source is stale, `--auto-refresh` refreshes it according to the source policy while exposing freshness metadata through `bundle_summary`.
|
|
15
16
|
|
|
16
17
|
Add the source-backed server to an MCP client:
|
|
@@ -37,6 +38,7 @@ Use the stripe-okf MCP server. Search for Checkout Sessions, read the most relev
|
|
|
37
38
|
Claude Code:
|
|
38
39
|
|
|
39
40
|
```bash
|
|
41
|
+
npx -y okfy-ai init stripe https://docs.stripe.com/checkout --client claude-code
|
|
40
42
|
claude mcp add --transport stdio stripe-okf -- npx -y okfy-ai serve stripe --mcp --auto-refresh
|
|
41
43
|
```
|
|
42
44
|
|
|
@@ -53,6 +55,14 @@ enabled = true
|
|
|
53
55
|
|
|
54
56
|
Claude Desktop, Cursor, and other `mcpServers` clients can use the JSON config above. More setup: https://github.com/0dust/OKFy/blob/main/docs/mcp-clients.md
|
|
55
57
|
|
|
58
|
+
If setup is not working, run:
|
|
59
|
+
|
|
60
|
+
```bash
|
|
61
|
+
npx -y okfy-ai doctor stripe --client codex
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
`doctor` checks the registered source, bundle validity, freshness, `npx` availability, generated command shape, MCP tool visibility, and JSON-RPC-clean stdout, then tells you the next repair command or config edit.
|
|
65
|
+
|
|
56
66
|
## Keep Sources Fresh
|
|
57
67
|
|
|
58
68
|
Registered sources are the local-first workflow for third-party docs sites that change over time:
|
|
@@ -61,11 +71,14 @@ Registered sources are the local-first workflow for third-party docs sites that
|
|
|
61
71
|
npx -y okfy-ai add stripe https://docs.stripe.com/checkout --max-pages 100 --max-depth 4
|
|
62
72
|
npx -y okfy-ai sources
|
|
63
73
|
npx -y okfy-ai check stripe
|
|
74
|
+
npx -y okfy-ai doctor stripe
|
|
64
75
|
npx -y okfy-ai update stripe
|
|
65
76
|
npx -y okfy-ai remove stripe
|
|
66
77
|
npx -y okfy-ai serve stripe --mcp --auto-refresh
|
|
67
78
|
```
|
|
68
79
|
|
|
80
|
+
If you want registration plus client-specific setup artifacts, use `npx -y okfy-ai init stripe https://docs.stripe.com/checkout --client generic --max-pages 100 --max-depth 4`.
|
|
81
|
+
|
|
69
82
|
By default, okfy stores registered sources under `~/.okfy`. Set `OKFY_HOME` to use a different local cache for CI, tests, or per-project isolation.
|
|
70
83
|
|
|
71
84
|
Freshness is age-based. A registered bundle is fresh when it exists, validates, and was successfully refreshed within its configured max age. The default mode is `stale-while-refresh`: if the bundle is stale, MCP search and read tools keep serving the current cached bundle while a background refresh runs.
|
|
@@ -155,6 +168,8 @@ npx -y okfy-ai demo
|
|
|
155
168
|
## CLI Commands
|
|
156
169
|
|
|
157
170
|
```bash
|
|
171
|
+
okfy init <name> <url>
|
|
172
|
+
okfy doctor <name>
|
|
158
173
|
okfy add <name> <url>
|
|
159
174
|
okfy sources
|
|
160
175
|
okfy check <name-or-bundle>
|
|
@@ -1122,6 +1122,23 @@ import { Server } from "@modelcontextprotocol/sdk/server/index.js";
|
|
|
1122
1122
|
import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
|
|
1123
1123
|
import { CallToolRequestSchema, ListToolsRequestSchema } from "@modelcontextprotocol/sdk/types.js";
|
|
1124
1124
|
import { z } from "zod";
|
|
1125
|
+
var MCP_TOOL_NAMES = [
|
|
1126
|
+
"search_concepts",
|
|
1127
|
+
"read_concept",
|
|
1128
|
+
"get_neighbors",
|
|
1129
|
+
"list_types",
|
|
1130
|
+
"list_tags",
|
|
1131
|
+
"bundle_summary"
|
|
1132
|
+
];
|
|
1133
|
+
var [
|
|
1134
|
+
SEARCH_CONCEPTS_TOOL,
|
|
1135
|
+
READ_CONCEPT_TOOL,
|
|
1136
|
+
GET_NEIGHBORS_TOOL,
|
|
1137
|
+
LIST_TYPES_TOOL,
|
|
1138
|
+
LIST_TAGS_TOOL,
|
|
1139
|
+
BUNDLE_SUMMARY_TOOL
|
|
1140
|
+
] = MCP_TOOL_NAMES;
|
|
1141
|
+
var REFRESHABLE_TOOL_NAMES = new Set(MCP_TOOL_NAMES.filter((tool) => tool !== BUNDLE_SUMMARY_TOOL));
|
|
1125
1142
|
function json(value, maxChars = 12e3) {
|
|
1126
1143
|
let text = JSON.stringify(value, null, 2);
|
|
1127
1144
|
if (text.length > maxChars) text = `${text.slice(0, maxChars)}
|
|
@@ -1166,7 +1183,7 @@ function shouldRefresh(status, hasSearch) {
|
|
|
1166
1183
|
return status === "stale" || status === "missing" || status === "failed";
|
|
1167
1184
|
}
|
|
1168
1185
|
function refreshableTool(name) {
|
|
1169
|
-
return name
|
|
1186
|
+
return REFRESHABLE_TOOL_NAMES.has(name);
|
|
1170
1187
|
}
|
|
1171
1188
|
async function createMcpServer(options) {
|
|
1172
1189
|
let activeBundleDir = options.bundleDir;
|
|
@@ -1265,7 +1282,7 @@ async function createMcpServer(options) {
|
|
|
1265
1282
|
server.setRequestHandler(ListToolsRequestSchema, async () => ({
|
|
1266
1283
|
tools: [
|
|
1267
1284
|
{
|
|
1268
|
-
name:
|
|
1285
|
+
name: SEARCH_CONCEPTS_TOOL,
|
|
1269
1286
|
description: "Search OKF concepts by query, type, and tags.",
|
|
1270
1287
|
inputSchema: {
|
|
1271
1288
|
type: "object",
|
|
@@ -1279,7 +1296,7 @@ async function createMcpServer(options) {
|
|
|
1279
1296
|
}
|
|
1280
1297
|
},
|
|
1281
1298
|
{
|
|
1282
|
-
name:
|
|
1299
|
+
name: READ_CONCEPT_TOOL,
|
|
1283
1300
|
description: "Read one OKF concept by id or path.",
|
|
1284
1301
|
inputSchema: {
|
|
1285
1302
|
type: "object",
|
|
@@ -1288,7 +1305,7 @@ async function createMcpServer(options) {
|
|
|
1288
1305
|
}
|
|
1289
1306
|
},
|
|
1290
1307
|
{
|
|
1291
|
-
name:
|
|
1308
|
+
name: GET_NEIGHBORS_TOOL,
|
|
1292
1309
|
description: "Return outbound links and backlinks for a concept.",
|
|
1293
1310
|
inputSchema: {
|
|
1294
1311
|
type: "object",
|
|
@@ -1296,22 +1313,22 @@ async function createMcpServer(options) {
|
|
|
1296
1313
|
required: ["id"]
|
|
1297
1314
|
}
|
|
1298
1315
|
},
|
|
1299
|
-
{ name:
|
|
1300
|
-
{ name:
|
|
1301
|
-
{ name:
|
|
1316
|
+
{ name: LIST_TYPES_TOOL, description: "List concept types and counts.", inputSchema: { type: "object", properties: {} } },
|
|
1317
|
+
{ name: LIST_TAGS_TOOL, description: "List concept tags and counts.", inputSchema: { type: "object", properties: {} } },
|
|
1318
|
+
{ name: BUNDLE_SUMMARY_TOOL, description: "Return bundle stats and validation status.", inputSchema: { type: "object", properties: {} } }
|
|
1302
1319
|
]
|
|
1303
1320
|
}));
|
|
1304
1321
|
server.setRequestHandler(CallToolRequestSchema, async (request) => {
|
|
1305
1322
|
const args = request.params.arguments ?? {};
|
|
1306
1323
|
try {
|
|
1307
|
-
if (request.params.name ===
|
|
1324
|
+
if (request.params.name === BUNDLE_SUMMARY_TOOL && options.source) await getFreshness();
|
|
1308
1325
|
await prepareBundleForTool(request.params.name);
|
|
1309
|
-
if (request.params.name ===
|
|
1326
|
+
if (request.params.name === SEARCH_CONCEPTS_TOOL) {
|
|
1310
1327
|
if (!search) return bundleUnavailable();
|
|
1311
1328
|
const parsed = searchSchema.parse(args);
|
|
1312
1329
|
return json(search.search(parsed.query, parsed), maxResultChars);
|
|
1313
1330
|
}
|
|
1314
|
-
if (request.params.name ===
|
|
1331
|
+
if (request.params.name === READ_CONCEPT_TOOL) {
|
|
1315
1332
|
if (!search) return bundleUnavailable();
|
|
1316
1333
|
const parsed = readSchema.parse(args);
|
|
1317
1334
|
const concept = search.getConcept(parsed.id);
|
|
@@ -1328,7 +1345,7 @@ async function createMcpServer(options) {
|
|
|
1328
1345
|
maxResultChars
|
|
1329
1346
|
);
|
|
1330
1347
|
}
|
|
1331
|
-
if (request.params.name ===
|
|
1348
|
+
if (request.params.name === GET_NEIGHBORS_TOOL) {
|
|
1332
1349
|
if (!search) return bundleUnavailable();
|
|
1333
1350
|
const currentSearch = search;
|
|
1334
1351
|
const parsed = neighborsSchema.parse(args);
|
|
@@ -1363,17 +1380,17 @@ async function createMcpServer(options) {
|
|
|
1363
1380
|
edges
|
|
1364
1381
|
});
|
|
1365
1382
|
}
|
|
1366
|
-
if (request.params.name ===
|
|
1383
|
+
if (request.params.name === LIST_TYPES_TOOL) {
|
|
1367
1384
|
if (!search) return bundleUnavailable();
|
|
1368
1385
|
const stats = await inspectBundle(activeBundleDir);
|
|
1369
1386
|
return json(stats.typeDistribution);
|
|
1370
1387
|
}
|
|
1371
|
-
if (request.params.name ===
|
|
1388
|
+
if (request.params.name === LIST_TAGS_TOOL) {
|
|
1372
1389
|
if (!search) return bundleUnavailable();
|
|
1373
1390
|
const stats = await inspectBundle(activeBundleDir);
|
|
1374
1391
|
return json(stats.tagDistribution);
|
|
1375
1392
|
}
|
|
1376
|
-
if (request.params.name ===
|
|
1393
|
+
if (request.params.name === BUNDLE_SUMMARY_TOOL) {
|
|
1377
1394
|
if (!search) return bundleUnavailable();
|
|
1378
1395
|
const [stats, validation] = await Promise.all([inspectBundle(activeBundleDir), validateBundle(activeBundleDir)]);
|
|
1379
1396
|
return json({
|
|
@@ -1883,6 +1900,7 @@ export {
|
|
|
1883
1900
|
BundleSearch,
|
|
1884
1901
|
validateBundle,
|
|
1885
1902
|
inspectBundle,
|
|
1903
|
+
MCP_TOOL_NAMES,
|
|
1886
1904
|
createMcpServer,
|
|
1887
1905
|
serveMcpStdio,
|
|
1888
1906
|
evaluateFreshness,
|
package/dist/cli.js
CHANGED
|
@@ -1,5 +1,6 @@
|
|
|
1
1
|
#!/usr/bin/env node
|
|
2
2
|
import {
|
|
3
|
+
MCP_TOOL_NAMES,
|
|
3
4
|
crawlWebsite,
|
|
4
5
|
evaluateFreshness,
|
|
5
6
|
hashBundleContents,
|
|
@@ -12,26 +13,334 @@ import {
|
|
|
12
13
|
refreshSource,
|
|
13
14
|
removeSource,
|
|
14
15
|
resolveBundleDir,
|
|
16
|
+
resolveOkfyHome,
|
|
15
17
|
resolveSourceDir,
|
|
16
18
|
serveMcpStdio,
|
|
17
19
|
validateBundle,
|
|
18
20
|
validateSourceName,
|
|
19
21
|
writeRefreshState,
|
|
20
22
|
writeSourceManifest
|
|
21
|
-
} from "./chunk-
|
|
23
|
+
} from "./chunk-7V2ZN6IS.js";
|
|
22
24
|
|
|
23
25
|
// src/cli.ts
|
|
24
|
-
import
|
|
25
|
-
import
|
|
26
|
+
import fs2 from "fs";
|
|
27
|
+
import path2 from "path";
|
|
28
|
+
import { execFile } from "child_process";
|
|
26
29
|
import { fileURLToPath } from "url";
|
|
27
30
|
import { Command } from "commander";
|
|
28
31
|
import pc from "picocolors";
|
|
32
|
+
|
|
33
|
+
// src/setup.ts
|
|
34
|
+
import fs from "fs/promises";
|
|
35
|
+
import path from "path";
|
|
36
|
+
import { spawn } from "child_process";
|
|
37
|
+
var EXPECTED_MCP_TOOLS = [...MCP_TOOL_NAMES];
|
|
38
|
+
var MAX_CAPTURE_CHARS = 64e3;
|
|
39
|
+
var MAX_DIAGNOSTIC_CHARS = 1e3;
|
|
40
|
+
var MAX_MESSAGES = 100;
|
|
41
|
+
function parseSetupClient(value) {
|
|
42
|
+
const normalized = value.trim().toLowerCase();
|
|
43
|
+
if (normalized === "claude-code" || normalized === "claude") return "claude-code";
|
|
44
|
+
if (normalized === "claude-desktop" || normalized === "cursor" || normalized === "mcp-json" || normalized === "desktop") {
|
|
45
|
+
return "mcp-json";
|
|
46
|
+
}
|
|
47
|
+
if (normalized === "codex") return "codex";
|
|
48
|
+
if (normalized === "generic" || normalized === "json") return "generic";
|
|
49
|
+
throw new Error(`Invalid setup client "${value}". Use claude-code, claude-desktop, cursor, codex, or generic.`);
|
|
50
|
+
}
|
|
51
|
+
function defaultOkfyHome() {
|
|
52
|
+
return resolveOkfyHome({ env: { OKFY_HOME: "" } });
|
|
53
|
+
}
|
|
54
|
+
function setupStatus(checks) {
|
|
55
|
+
if (checks.some((check) => check.severity === "fail")) return "failed";
|
|
56
|
+
if (checks.some((check) => check.severity === "warn")) return "warning";
|
|
57
|
+
return "ready";
|
|
58
|
+
}
|
|
59
|
+
function createSetupReport(input) {
|
|
60
|
+
const okfyHome = path.resolve(input.okfyHome ?? resolveOkfyHome());
|
|
61
|
+
const defaultHome = defaultOkfyHome();
|
|
62
|
+
const serverName = mcpServerName(input.sourceName);
|
|
63
|
+
const codexServerName = codexMcpServerName(input.sourceName);
|
|
64
|
+
const command = serveCommand(input.sourceName, okfyHome, defaultHome);
|
|
65
|
+
return {
|
|
66
|
+
sourceName: input.sourceName,
|
|
67
|
+
client: input.client,
|
|
68
|
+
serverName,
|
|
69
|
+
codexServerName,
|
|
70
|
+
okfyHome,
|
|
71
|
+
defaultOkfyHome: defaultHome,
|
|
72
|
+
command,
|
|
73
|
+
artifacts: renderClientArtifacts({ client: input.client, sourceName: input.sourceName, okfyHome, defaultOkfyHome: defaultHome }),
|
|
74
|
+
firstPrompt: firstAgentPrompt(input.client === "codex" ? codexServerName : serverName),
|
|
75
|
+
checks: input.checks,
|
|
76
|
+
status: setupStatus(input.checks)
|
|
77
|
+
};
|
|
78
|
+
}
|
|
79
|
+
function renderClientArtifacts(input) {
|
|
80
|
+
const okfyHome = path.resolve(input.okfyHome ?? resolveOkfyHome());
|
|
81
|
+
const defaultHome = input.defaultOkfyHome ?? defaultOkfyHome();
|
|
82
|
+
const serverName = mcpServerName(input.sourceName);
|
|
83
|
+
const codexName = codexMcpServerName(input.sourceName);
|
|
84
|
+
const command = serveCommand(input.sourceName, okfyHome, defaultHome);
|
|
85
|
+
const env = Object.keys(command.env).length ? command.env : void 0;
|
|
86
|
+
if (input.client === "claude-code") {
|
|
87
|
+
return [
|
|
88
|
+
{
|
|
89
|
+
client: input.client,
|
|
90
|
+
label: "Claude Code",
|
|
91
|
+
format: "shell",
|
|
92
|
+
body: `claude mcp add --transport stdio${shellEnvArgs(command.env, "-e")} ${serverName} -- ${command.display}`
|
|
93
|
+
}
|
|
94
|
+
];
|
|
95
|
+
}
|
|
96
|
+
if (input.client === "codex") {
|
|
97
|
+
return [
|
|
98
|
+
{
|
|
99
|
+
client: input.client,
|
|
100
|
+
label: "Codex config.toml",
|
|
101
|
+
format: "toml",
|
|
102
|
+
body: codexToml(codexName, command, env)
|
|
103
|
+
},
|
|
104
|
+
{
|
|
105
|
+
client: input.client,
|
|
106
|
+
label: "Codex CLI",
|
|
107
|
+
format: "shell",
|
|
108
|
+
body: `codex mcp add${shellEnvArgs(command.env, "--env")} ${codexName} -- ${command.display}`
|
|
109
|
+
}
|
|
110
|
+
];
|
|
111
|
+
}
|
|
112
|
+
const label = input.client === "mcp-json" ? "Claude Desktop / Cursor mcpServers JSON" : "Generic mcpServers JSON";
|
|
113
|
+
return [
|
|
114
|
+
{
|
|
115
|
+
client: input.client,
|
|
116
|
+
label,
|
|
117
|
+
format: "json",
|
|
118
|
+
body: JSON.stringify(
|
|
119
|
+
{
|
|
120
|
+
mcpServers: {
|
|
121
|
+
[serverName]: {
|
|
122
|
+
command: command.command,
|
|
123
|
+
args: command.args,
|
|
124
|
+
...env ? { env } : {}
|
|
125
|
+
}
|
|
126
|
+
}
|
|
127
|
+
},
|
|
128
|
+
null,
|
|
129
|
+
2
|
|
130
|
+
)
|
|
131
|
+
}
|
|
132
|
+
];
|
|
133
|
+
}
|
|
134
|
+
function firstAgentPrompt(serverName) {
|
|
135
|
+
return `Use the ${serverName} MCP server. Start with bundle_summary to understand the bundle and freshness. Search before reading concepts, read only the most relevant concepts, inspect neighbors when relationships matter, and cite source_resource URLs in the final answer.`;
|
|
136
|
+
}
|
|
137
|
+
function serveCommand(sourceName, okfyHome, defaultHome = defaultOkfyHome()) {
|
|
138
|
+
const args = sourceName.startsWith("-") ? ["-y", "okfy-ai", "serve", "--mcp", "--auto-refresh", "--", sourceName] : ["-y", "okfy-ai", "serve", sourceName, "--mcp", "--auto-refresh"];
|
|
139
|
+
const env = needsOkfyHomeEnv(okfyHome, defaultHome) ? { OKFY_HOME: path.resolve(okfyHome) } : {};
|
|
140
|
+
return {
|
|
141
|
+
command: "npx",
|
|
142
|
+
args,
|
|
143
|
+
env,
|
|
144
|
+
display: ["npx", ...args].join(" ")
|
|
145
|
+
};
|
|
146
|
+
}
|
|
147
|
+
function setupCheck(id, label, severity, message, fix) {
|
|
148
|
+
return { id, label, severity, message, ...fix ? { fix } : {} };
|
|
149
|
+
}
|
|
150
|
+
async function executableOnPath(command, env = process.env) {
|
|
151
|
+
const searchPath = env.PATH ?? "";
|
|
152
|
+
const extensions = process.platform === "win32" ? (env.PATHEXT ?? ".EXE;.CMD;.BAT;.COM").split(";") : [""];
|
|
153
|
+
for (const directory of searchPath.split(path.delimiter)) {
|
|
154
|
+
if (!directory) continue;
|
|
155
|
+
for (const extension of extensions) {
|
|
156
|
+
const candidate = path.join(directory, `${command}${extension}`);
|
|
157
|
+
try {
|
|
158
|
+
await fs.access(candidate, fs.constants.X_OK);
|
|
159
|
+
return true;
|
|
160
|
+
} catch {
|
|
161
|
+
}
|
|
162
|
+
}
|
|
163
|
+
}
|
|
164
|
+
return false;
|
|
165
|
+
}
|
|
166
|
+
function evaluateMcpProbeMessages(messages) {
|
|
167
|
+
const toolsResponse = messages.find((message) => message.id === 2);
|
|
168
|
+
const tools = toolsResponse?.result?.tools?.map((tool) => tool.name).filter((name) => Boolean(name)) ?? [];
|
|
169
|
+
const missingTools = EXPECTED_MCP_TOOLS.filter((tool) => !tools.includes(tool));
|
|
170
|
+
return { ok: missingTools.length === 0, tools, missingTools };
|
|
171
|
+
}
|
|
172
|
+
async function probeMcpStdio(options) {
|
|
173
|
+
const child = spawn(options.command, options.args, {
|
|
174
|
+
env: options.env,
|
|
175
|
+
stdio: ["pipe", "pipe", "pipe"]
|
|
176
|
+
});
|
|
177
|
+
return probeChildProcess(child, options.timeoutMs ?? 5e3);
|
|
178
|
+
}
|
|
179
|
+
async function probeChildProcess(child, timeoutMs) {
|
|
180
|
+
const messages = [];
|
|
181
|
+
let stdoutBuffer = "";
|
|
182
|
+
let stderr = "";
|
|
183
|
+
let contamination;
|
|
184
|
+
let spawnError;
|
|
185
|
+
let exit;
|
|
186
|
+
const closed = new Promise((resolve) => {
|
|
187
|
+
child.once("close", (code, signal) => {
|
|
188
|
+
exit = { code, signal };
|
|
189
|
+
resolve(exit);
|
|
190
|
+
});
|
|
191
|
+
});
|
|
192
|
+
child.on("error", (error) => {
|
|
193
|
+
spawnError = error;
|
|
194
|
+
});
|
|
195
|
+
child.stdin.on("error", (error) => {
|
|
196
|
+
spawnError ??= error;
|
|
197
|
+
});
|
|
198
|
+
child.stdout.on("data", (chunk) => {
|
|
199
|
+
stdoutBuffer = appendBounded(stdoutBuffer, chunk.toString("utf8"));
|
|
200
|
+
let newlineIndex = stdoutBuffer.indexOf("\n");
|
|
201
|
+
while (newlineIndex >= 0) {
|
|
202
|
+
const line = stdoutBuffer.slice(0, newlineIndex).trim();
|
|
203
|
+
stdoutBuffer = stdoutBuffer.slice(newlineIndex + 1);
|
|
204
|
+
if (line) {
|
|
205
|
+
try {
|
|
206
|
+
if (messages.length >= MAX_MESSAGES) contamination = `MCP stdout exceeded ${MAX_MESSAGES} JSON-RPC messages.`;
|
|
207
|
+
else messages.push(JSON.parse(line));
|
|
208
|
+
} catch {
|
|
209
|
+
contamination = line;
|
|
210
|
+
}
|
|
211
|
+
}
|
|
212
|
+
newlineIndex = stdoutBuffer.indexOf("\n");
|
|
213
|
+
}
|
|
214
|
+
if (stdoutBuffer.length >= MAX_CAPTURE_CHARS) contamination = `MCP stdout line exceeded ${MAX_CAPTURE_CHARS} characters.`;
|
|
215
|
+
});
|
|
216
|
+
child.stderr.on("data", (chunk) => {
|
|
217
|
+
stderr = appendBounded(stderr, chunk.toString("utf8"));
|
|
218
|
+
});
|
|
219
|
+
const send = (id, method, params = {}) => {
|
|
220
|
+
child.stdin.write(`${JSON.stringify({ jsonrpc: "2.0", id, method, params })}
|
|
221
|
+
`);
|
|
222
|
+
};
|
|
223
|
+
try {
|
|
224
|
+
send(1, "initialize", {
|
|
225
|
+
protocolVersion: "2025-06-18",
|
|
226
|
+
capabilities: {},
|
|
227
|
+
clientInfo: { name: "okfy-doctor", version: "0.1.0" }
|
|
228
|
+
});
|
|
229
|
+
await waitForMessage(1, messages, () => contamination, () => spawnError, () => exit, () => stderr, timeoutMs);
|
|
230
|
+
child.stdin.write(`${JSON.stringify({ jsonrpc: "2.0", method: "notifications/initialized", params: {} })}
|
|
231
|
+
`);
|
|
232
|
+
send(2, "tools/list");
|
|
233
|
+
await waitForMessage(2, messages, () => contamination, () => spawnError, () => exit, () => stderr, timeoutMs);
|
|
234
|
+
const result = evaluateMcpProbeMessages(messages);
|
|
235
|
+
if (!result.ok) {
|
|
236
|
+
return {
|
|
237
|
+
ok: false,
|
|
238
|
+
tools: result.tools,
|
|
239
|
+
stderr,
|
|
240
|
+
error: {
|
|
241
|
+
code: "missing_tools",
|
|
242
|
+
message: `MCP server did not expose expected tools: ${result.missingTools.join(", ")}.`
|
|
243
|
+
}
|
|
244
|
+
};
|
|
245
|
+
}
|
|
246
|
+
return { ok: true, tools: result.tools, stderr };
|
|
247
|
+
} catch (error) {
|
|
248
|
+
if (error instanceof ProbeFailure) {
|
|
249
|
+
return { ok: false, tools: [], stderr, error: { code: error.code, message: error.message } };
|
|
250
|
+
}
|
|
251
|
+
return { ok: false, tools: [], stderr, error: { code: "protocol_error", message: error instanceof Error ? error.message : String(error) } };
|
|
252
|
+
} finally {
|
|
253
|
+
await stopChild(child, closed, () => exit);
|
|
254
|
+
}
|
|
255
|
+
}
|
|
256
|
+
var ProbeFailure = class extends Error {
|
|
257
|
+
constructor(code, message) {
|
|
258
|
+
super(message);
|
|
259
|
+
this.code = code;
|
|
260
|
+
}
|
|
261
|
+
code;
|
|
262
|
+
};
|
|
263
|
+
async function waitForMessage(id, messages, contamination, spawnError, childExit, capturedStderr, timeoutMs) {
|
|
264
|
+
const deadline = Date.now() + timeoutMs;
|
|
265
|
+
while (Date.now() < deadline) {
|
|
266
|
+
const badLine = contamination();
|
|
267
|
+
if (badLine) throw new ProbeFailure("stdout_contamination", `MCP stdout contained non-JSON output: ${badLine}`);
|
|
268
|
+
const error = spawnError();
|
|
269
|
+
if (error) throw new ProbeFailure("startup_failed", error.message);
|
|
270
|
+
const message = messages.find((candidate) => candidate.id === id);
|
|
271
|
+
if (message) return message;
|
|
272
|
+
const exit = childExit();
|
|
273
|
+
if (exit) {
|
|
274
|
+
const details = capturedStderr() ? ` stderr: ${truncate(capturedStderr())}` : "";
|
|
275
|
+
throw new ProbeFailure("startup_failed", `MCP subprocess exited before response ${id} (${formatExit(exit)}).${details}`);
|
|
276
|
+
}
|
|
277
|
+
await new Promise((resolve) => setTimeout(resolve, 25));
|
|
278
|
+
}
|
|
279
|
+
throw new ProbeFailure("timeout", `Timed out waiting for MCP response ${id}.`);
|
|
280
|
+
}
|
|
281
|
+
async function stopChild(child, closed, childExit) {
|
|
282
|
+
try {
|
|
283
|
+
if (!child.stdin.destroyed) child.stdin.end();
|
|
284
|
+
} catch {
|
|
285
|
+
}
|
|
286
|
+
if (childExit()) return;
|
|
287
|
+
child.kill("SIGTERM");
|
|
288
|
+
const exited = await Promise.race([closed.then(() => true), new Promise((resolve) => setTimeout(() => resolve(false), 500))]);
|
|
289
|
+
if (!exited && !childExit()) child.kill("SIGKILL");
|
|
290
|
+
}
|
|
291
|
+
function appendBounded(current, addition) {
|
|
292
|
+
const next = current + addition;
|
|
293
|
+
if (next.length <= MAX_CAPTURE_CHARS) return next;
|
|
294
|
+
return next.slice(next.length - MAX_CAPTURE_CHARS);
|
|
295
|
+
}
|
|
296
|
+
function truncate(value) {
|
|
297
|
+
const normalized = value.trim();
|
|
298
|
+
if (normalized.length <= MAX_DIAGNOSTIC_CHARS) return normalized;
|
|
299
|
+
return `${normalized.slice(0, MAX_DIAGNOSTIC_CHARS)}...truncated`;
|
|
300
|
+
}
|
|
301
|
+
function formatExit(exit) {
|
|
302
|
+
if (exit.signal) return `signal ${exit.signal}`;
|
|
303
|
+
return `exit code ${exit.code ?? "unknown"}`;
|
|
304
|
+
}
|
|
305
|
+
function needsOkfyHomeEnv(okfyHome, defaultHome) {
|
|
306
|
+
return path.resolve(okfyHome) !== path.resolve(defaultHome);
|
|
307
|
+
}
|
|
308
|
+
function mcpServerName(sourceName) {
|
|
309
|
+
const safeName = sourceName.replace(/[._]+/g, "-").replace(/^-+/, "");
|
|
310
|
+
return `${safeName || "source"}-okf`;
|
|
311
|
+
}
|
|
312
|
+
function codexMcpServerName(sourceName) {
|
|
313
|
+
const safeName = sourceName.replace(/[^a-z0-9]+/g, "_").replace(/^_+/, "");
|
|
314
|
+
return `${safeName || "source"}_okf`;
|
|
315
|
+
}
|
|
316
|
+
function shellEnvArgs(env, flag) {
|
|
317
|
+
const entries = Object.entries(env);
|
|
318
|
+
if (!entries.length) return "";
|
|
319
|
+
return entries.map(([key, value]) => ` ${flag} ${shellQuote(`${key}=${value}`)}`).join("");
|
|
320
|
+
}
|
|
321
|
+
function shellQuote(value) {
|
|
322
|
+
if (/^[A-Za-z0-9_./:=@+-]+$/.test(value)) return value;
|
|
323
|
+
return `'${value.replace(/'/g, "'\\''")}'`;
|
|
324
|
+
}
|
|
325
|
+
function codexToml(serverName, command, env) {
|
|
326
|
+
const lines = [
|
|
327
|
+
`[mcp_servers.${serverName}]`,
|
|
328
|
+
`command = ${JSON.stringify(command.command)}`,
|
|
329
|
+
`args = [${command.args.map((arg) => JSON.stringify(arg)).join(", ")}]`
|
|
330
|
+
];
|
|
331
|
+
if (env?.OKFY_HOME) lines.push(`env = { OKFY_HOME = ${JSON.stringify(env.OKFY_HOME)} }`);
|
|
332
|
+
lines.push("startup_timeout_sec = 20", "tool_timeout_sec = 60", "enabled = true");
|
|
333
|
+
return lines.join("\n");
|
|
334
|
+
}
|
|
335
|
+
|
|
336
|
+
// src/cli.ts
|
|
29
337
|
var program = new Command();
|
|
30
|
-
var
|
|
338
|
+
var cliPath = fileURLToPath(import.meta.url);
|
|
339
|
+
var packageRoot = path2.resolve(path2.dirname(cliPath), "..");
|
|
31
340
|
var isTty = Boolean(process.stderr.isTTY);
|
|
32
341
|
function readPackageVersion() {
|
|
33
342
|
try {
|
|
34
|
-
const raw =
|
|
343
|
+
const raw = fs2.readFileSync(path2.join(packageRoot, "package.json"), "utf8");
|
|
35
344
|
const parsed = JSON.parse(raw);
|
|
36
345
|
return parsed.version ?? "0.0.0";
|
|
37
346
|
} catch {
|
|
@@ -55,7 +364,7 @@ function printJson(value) {
|
|
|
55
364
|
}
|
|
56
365
|
async function pathExists(target) {
|
|
57
366
|
try {
|
|
58
|
-
await
|
|
367
|
+
await fs2.promises.access(target);
|
|
59
368
|
return true;
|
|
60
369
|
} catch (error) {
|
|
61
370
|
if (error?.code === "ENOENT") return false;
|
|
@@ -104,12 +413,12 @@ async function summarizeState(record, maxAgeSeconds) {
|
|
|
104
413
|
nextRefreshAllowedAt: state?.nextRefreshAllowedAt ?? null,
|
|
105
414
|
refreshInProgress: decision.status === "refreshing",
|
|
106
415
|
lastError: state?.lastError ?? null,
|
|
107
|
-
bundle:
|
|
416
|
+
bundle: decision.validation ? {
|
|
108
417
|
conceptCount: decision.validation.conceptCount,
|
|
109
418
|
warningCount: decision.validation.warningCount,
|
|
110
419
|
valid: decision.validation.valid,
|
|
111
|
-
contentHash:
|
|
112
|
-
} : null
|
|
420
|
+
contentHash: await hashBundleContents(record.bundleDir)
|
|
421
|
+
} : decision.status === "missing" ? null : state?.bundle ?? null
|
|
113
422
|
};
|
|
114
423
|
}
|
|
115
424
|
function sourceRow(record, state) {
|
|
@@ -146,6 +455,9 @@ function refreshMode(value) {
|
|
|
146
455
|
if (value === "off" || value === "stale-while-refresh" || value === "blocking") return value;
|
|
147
456
|
throw new Error(`Invalid refresh mode "${value}". Use off, stale-while-refresh, or blocking.`);
|
|
148
457
|
}
|
|
458
|
+
function setupClient(value) {
|
|
459
|
+
return parseSetupClient(value);
|
|
460
|
+
}
|
|
149
461
|
function manifestFromOptions(name, seedUrl, options) {
|
|
150
462
|
const now = (/* @__PURE__ */ new Date()).toISOString();
|
|
151
463
|
return {
|
|
@@ -174,10 +486,45 @@ function manifestFromOptions(name, seedUrl, options) {
|
|
|
174
486
|
minIntervalSeconds: options.minRefreshInterval
|
|
175
487
|
},
|
|
176
488
|
bundle: {
|
|
177
|
-
dir: options.out ?
|
|
489
|
+
dir: options.out ? path2.resolve(options.out) : "bundle"
|
|
178
490
|
}
|
|
179
491
|
};
|
|
180
492
|
}
|
|
493
|
+
function addSourceRegistrationOptions(command) {
|
|
494
|
+
return command.option("--max-pages <n>", "Maximum pages", numberOption, 100).option("--max-depth <n>", "Maximum crawl depth", numberOption, 4).option("--include <pattern>", "Include glob or regex", collect, []).option("--exclude <pattern>", "Exclude glob or regex", collect, []).option("--same-origin", "Stay on same origin", true).option("--no-same-origin", "Allow cross-origin links").option("--respect-robots", "Respect robots.txt", true).option("--no-respect-robots", "Ignore robots.txt").option("--concurrency <n>", "Fetch concurrency", numberOption, 4).option("--allow-private-network", "Allow localhost/private IP crawl targets", false).option("--refresh-mode <mode>", "Refresh mode: off, stale-while-refresh, or blocking", refreshMode, "stale-while-refresh").option("--max-age <duration>", "Freshness max age", duration, 24 * 60 * 60).option("--min-refresh-interval <duration>", "Minimum interval between refresh attempts", duration, 15 * 60).option("--out <dir>", "Explicit active bundle directory").option("--force", "Overwrite an existing source registration", false);
|
|
495
|
+
}
|
|
496
|
+
async function registerWebsiteSource(name, url, options) {
|
|
497
|
+
const manifest = manifestFromOptions(name, url, options);
|
|
498
|
+
const sourceDir = resolveSourceDir(manifest.name);
|
|
499
|
+
if (await pathExists(sourceDir) && !options.force) {
|
|
500
|
+
throw new Error(`Source "${manifest.name}" already exists. Use --force to overwrite it.`);
|
|
501
|
+
}
|
|
502
|
+
let backupDir;
|
|
503
|
+
if (options.force && await pathExists(sourceDir)) {
|
|
504
|
+
backupDir = `${sourceDir}.backup-${process.pid}-${Date.now()}`;
|
|
505
|
+
await fs2.promises.rename(sourceDir, backupDir);
|
|
506
|
+
}
|
|
507
|
+
try {
|
|
508
|
+
await writeSourceManifest(manifest);
|
|
509
|
+
const result = await runSourceRefresh(manifest, { force: true });
|
|
510
|
+
if (result.status === "fresh") {
|
|
511
|
+
if (backupDir) await fs2.promises.rm(backupDir, { recursive: true, force: true });
|
|
512
|
+
return { manifest, result };
|
|
513
|
+
}
|
|
514
|
+
if (backupDir) {
|
|
515
|
+
await restoreSourceBackup(sourceDir, backupDir);
|
|
516
|
+
throw new Error(result.error?.message ?? `Refresh failed for source "${manifest.name}".`);
|
|
517
|
+
}
|
|
518
|
+
return { manifest, result };
|
|
519
|
+
} catch (error) {
|
|
520
|
+
if (backupDir) await restoreSourceBackup(sourceDir, backupDir);
|
|
521
|
+
throw error;
|
|
522
|
+
}
|
|
523
|
+
}
|
|
524
|
+
async function restoreSourceBackup(sourceDir, backupDir) {
|
|
525
|
+
await fs2.promises.rm(sourceDir, { recursive: true, force: true });
|
|
526
|
+
if (await pathExists(backupDir)) await fs2.promises.rename(backupDir, sourceDir);
|
|
527
|
+
}
|
|
181
528
|
async function runSourceRefresh(manifest, options = {}) {
|
|
182
529
|
const state = await readStateIfExists(manifest.name);
|
|
183
530
|
const sourceDir = resolveSourceDir(manifest.name);
|
|
@@ -237,6 +584,208 @@ function printStatus(message) {
|
|
|
237
584
|
process.stderr.write(`${message}
|
|
238
585
|
`);
|
|
239
586
|
}
|
|
587
|
+
function setupHomeCheck(okfyHome) {
|
|
588
|
+
const defaultHome = defaultOkfyHome();
|
|
589
|
+
if (path2.resolve(okfyHome) === path2.resolve(defaultHome)) {
|
|
590
|
+
return setupCheck("source_home", "Source store", "pass", `Using default OKFY_HOME ${okfyHome}.`);
|
|
591
|
+
}
|
|
592
|
+
return setupCheck(
|
|
593
|
+
"source_home",
|
|
594
|
+
"Source store",
|
|
595
|
+
"pass",
|
|
596
|
+
`Using non-default OKFY_HOME ${okfyHome}; generated configs include this environment override.`
|
|
597
|
+
);
|
|
598
|
+
}
|
|
599
|
+
function setupFreshnessCheck(record, state) {
|
|
600
|
+
if (state.status === "fresh" && state.bundle?.valid === true) {
|
|
601
|
+
return setupCheck(
|
|
602
|
+
"freshness",
|
|
603
|
+
"Freshness",
|
|
604
|
+
"pass",
|
|
605
|
+
`Source "${record.name}" is fresh with ${state.bundle.conceptCount} concepts.`
|
|
606
|
+
);
|
|
607
|
+
}
|
|
608
|
+
if (state.status === "stale") {
|
|
609
|
+
return setupCheck(
|
|
610
|
+
"freshness",
|
|
611
|
+
"Freshness",
|
|
612
|
+
"warn",
|
|
613
|
+
`Source "${record.name}" is stale.`,
|
|
614
|
+
`Run npx -y okfy-ai update ${record.name}, or keep --auto-refresh enabled in the MCP config.`
|
|
615
|
+
);
|
|
616
|
+
}
|
|
617
|
+
if (state.status === "refreshing") {
|
|
618
|
+
return setupCheck(
|
|
619
|
+
"freshness",
|
|
620
|
+
"Freshness",
|
|
621
|
+
"warn",
|
|
622
|
+
`Source "${record.name}" is already refreshing.`,
|
|
623
|
+
`Wait for the current refresh to finish, then run npx -y okfy-ai doctor ${record.name}.`
|
|
624
|
+
);
|
|
625
|
+
}
|
|
626
|
+
return setupCheck(
|
|
627
|
+
"freshness",
|
|
628
|
+
"Freshness",
|
|
629
|
+
"fail",
|
|
630
|
+
state.lastError?.message ?? `Source "${record.name}" is ${state.status}.`,
|
|
631
|
+
`Run npx -y okfy-ai update ${record.name}.`
|
|
632
|
+
);
|
|
633
|
+
}
|
|
634
|
+
async function setupBundleCheck(bundleDir) {
|
|
635
|
+
try {
|
|
636
|
+
const validation = await validateBundle(bundleDir);
|
|
637
|
+
if (validation.valid) {
|
|
638
|
+
return setupCheck("bundle", "Bundle validation", "pass", `Bundle is valid with ${validation.conceptCount} concepts.`);
|
|
639
|
+
}
|
|
640
|
+
const firstIssue = validation.issues[0];
|
|
641
|
+
return setupCheck(
|
|
642
|
+
"bundle",
|
|
643
|
+
"Bundle validation",
|
|
644
|
+
"fail",
|
|
645
|
+
firstIssue ? `${firstIssue.code}: ${firstIssue.message}` : "Bundle validation failed.",
|
|
646
|
+
"Run npx -y okfy-ai check <source> --json for validation details."
|
|
647
|
+
);
|
|
648
|
+
} catch (error) {
|
|
649
|
+
return setupCheck(
|
|
650
|
+
"bundle",
|
|
651
|
+
"Bundle validation",
|
|
652
|
+
"fail",
|
|
653
|
+
error?.message ?? "Bundle validation failed.",
|
|
654
|
+
"Run npx -y okfy-ai update <source> to rebuild the bundle."
|
|
655
|
+
);
|
|
656
|
+
}
|
|
657
|
+
}
|
|
658
|
+
async function setupNpxCheck() {
|
|
659
|
+
const fix = "Install Node.js >=20 with npm/npx, use an absolute npx path, or switch the config to an installed okfy command.";
|
|
660
|
+
if (!await executableOnPath("npx")) {
|
|
661
|
+
return setupCheck("npx", "npx availability", "fail", "`npx` was not found on PATH, but generated MCP configs use npx by default.", fix);
|
|
662
|
+
}
|
|
663
|
+
const health = await commandHealth("npx", ["--version"], process.env);
|
|
664
|
+
if (!health.ok) {
|
|
665
|
+
return setupCheck("npx", "npx availability", "fail", `\`npx\` was found but failed to run: ${health.message}`, fix);
|
|
666
|
+
}
|
|
667
|
+
return setupCheck("npx", "npx availability", "pass", `\`npx\` is available on PATH (${health.message}).`);
|
|
668
|
+
}
|
|
669
|
+
function setupMcpProbeCheck(probe) {
|
|
670
|
+
if (probe.ok) {
|
|
671
|
+
return setupCheck("mcp_probe", "MCP stdio probe", "pass", `MCP tools visible: ${probe.tools.join(", ")}.`);
|
|
672
|
+
}
|
|
673
|
+
const message = probe.error?.message ?? "MCP probe failed.";
|
|
674
|
+
const fix = probe.error?.code === "stdout_contamination" ? "Move human logs to stderr so stdout contains only MCP JSON-RPC messages." : "Run the generated serve command in your MCP client, then rerun doctor with the same OKFY_HOME.";
|
|
675
|
+
return setupCheck("mcp_probe", "MCP stdio probe", "fail", message, fix);
|
|
676
|
+
}
|
|
677
|
+
async function runSetupProbe(name, timeoutSeconds) {
|
|
678
|
+
const command = serveCommand(name, resolveOkfyHome());
|
|
679
|
+
return probeMcpStdio({
|
|
680
|
+
command: process.execPath,
|
|
681
|
+
args: [cliPath, "serve", name, "--mcp", "--auto-refresh"],
|
|
682
|
+
env: { ...process.env, ...command.env },
|
|
683
|
+
timeoutMs: timeoutSeconds * 1e3
|
|
684
|
+
});
|
|
685
|
+
}
|
|
686
|
+
async function commandHealth(command, args, env) {
|
|
687
|
+
return new Promise((resolve) => {
|
|
688
|
+
execFile(command, args, { env, timeout: 3e3 }, (error, stdout, stderr) => {
|
|
689
|
+
const message = (stderr || stdout || (error instanceof Error ? error.message : String(error ?? ""))).trim();
|
|
690
|
+
if (error) resolve({ ok: false, message: message || "command failed" });
|
|
691
|
+
else resolve({ ok: true, message: message || "ok" });
|
|
692
|
+
});
|
|
693
|
+
});
|
|
694
|
+
}
|
|
695
|
+
async function setupReportForRecord(options) {
|
|
696
|
+
const state = await summarizeState(options.record, options.maxAge);
|
|
697
|
+
await writeRefreshState(options.record.name, state);
|
|
698
|
+
const bundleCheck = await setupBundleCheck(options.record.bundleDir);
|
|
699
|
+
const npxCheck = await setupNpxCheck();
|
|
700
|
+
const checks = [
|
|
701
|
+
setupCheck("source", "Registered source", "pass", `Source "${options.record.name}" exists.`),
|
|
702
|
+
setupHomeCheck(resolveOkfyHome()),
|
|
703
|
+
bundleCheck,
|
|
704
|
+
setupFreshnessCheck(options.record, state),
|
|
705
|
+
npxCheck
|
|
706
|
+
];
|
|
707
|
+
if (bundleCheck.severity === "fail" || npxCheck.severity === "fail") {
|
|
708
|
+
checks.push(
|
|
709
|
+
setupCheck(
|
|
710
|
+
"mcp_probe",
|
|
711
|
+
"MCP stdio probe",
|
|
712
|
+
"warn",
|
|
713
|
+
"Skipped MCP probe because setup prerequisites failed.",
|
|
714
|
+
"Fix the failed checks above, then rerun doctor."
|
|
715
|
+
)
|
|
716
|
+
);
|
|
717
|
+
} else {
|
|
718
|
+
checks.push(setupMcpProbeCheck(await runSetupProbe(options.record.name, options.probeTimeoutSeconds)));
|
|
719
|
+
}
|
|
720
|
+
return createSetupReport({
|
|
721
|
+
sourceName: options.record.name,
|
|
722
|
+
client: options.client,
|
|
723
|
+
okfyHome: resolveOkfyHome(),
|
|
724
|
+
checks
|
|
725
|
+
});
|
|
726
|
+
}
|
|
727
|
+
function setupReportForMissingSource(name, client, error) {
|
|
728
|
+
const message = error instanceof Error ? error.message : `Source "${name}" was not found.`;
|
|
729
|
+
return createSetupReport({
|
|
730
|
+
sourceName: name,
|
|
731
|
+
client,
|
|
732
|
+
okfyHome: resolveOkfyHome(),
|
|
733
|
+
checks: [
|
|
734
|
+
setupCheck(
|
|
735
|
+
"source",
|
|
736
|
+
"Registered source",
|
|
737
|
+
"fail",
|
|
738
|
+
message,
|
|
739
|
+
`Run npx -y okfy-ai sources to list sources in this OKFY_HOME, or run npx -y okfy-ai init ${name} <docs-url> --client generic.`
|
|
740
|
+
),
|
|
741
|
+
setupHomeCheck(resolveOkfyHome())
|
|
742
|
+
]
|
|
743
|
+
});
|
|
744
|
+
}
|
|
745
|
+
function setupReportForInitFailure(name, client, error) {
|
|
746
|
+
const message = error instanceof Error ? error.message : "Init failed.";
|
|
747
|
+
return createSetupReport({
|
|
748
|
+
sourceName: name,
|
|
749
|
+
client,
|
|
750
|
+
okfyHome: resolveOkfyHome(),
|
|
751
|
+
checks: [
|
|
752
|
+
setupCheck(
|
|
753
|
+
"source",
|
|
754
|
+
"Registered source",
|
|
755
|
+
"fail",
|
|
756
|
+
message,
|
|
757
|
+
`Check the source name and URL, then rerun npx -y okfy-ai init ${name} <docs-url>.`
|
|
758
|
+
),
|
|
759
|
+
setupHomeCheck(resolveOkfyHome())
|
|
760
|
+
]
|
|
761
|
+
});
|
|
762
|
+
}
|
|
763
|
+
function printSetupReport(report, json) {
|
|
764
|
+
if (json) {
|
|
765
|
+
printJson(report);
|
|
766
|
+
return;
|
|
767
|
+
}
|
|
768
|
+
const color = report.status === "failed" ? pc.red : report.status === "warning" ? pc.yellow : pc.green;
|
|
769
|
+
console.log(color(`Setup status: ${report.status}`));
|
|
770
|
+
console.log(`Source: ${report.sourceName}`);
|
|
771
|
+
console.log(`OKFY_HOME: ${report.okfyHome}`);
|
|
772
|
+
console.log("\nChecks:");
|
|
773
|
+
for (const check of report.checks) {
|
|
774
|
+
const label = check.severity === "fail" ? pc.red("FAIL") : check.severity === "warn" ? pc.yellow("WARN") : pc.green("PASS");
|
|
775
|
+
console.log(` ${label} ${check.label}: ${check.message}`);
|
|
776
|
+
if (check.fix) console.log(` Fix: ${check.fix}`);
|
|
777
|
+
}
|
|
778
|
+
console.log("\nMCP launch command:");
|
|
779
|
+
console.log(` ${report.command.display}`);
|
|
780
|
+
if (Object.keys(report.command.env).length) console.log(` env: ${JSON.stringify(report.command.env)}`);
|
|
781
|
+
for (const artifact of report.artifacts) {
|
|
782
|
+
console.log(`
|
|
783
|
+
${artifact.label}:`);
|
|
784
|
+
console.log(artifact.body);
|
|
785
|
+
}
|
|
786
|
+
console.log("\nFirst prompt:");
|
|
787
|
+
console.log(report.firstPrompt);
|
|
788
|
+
}
|
|
240
789
|
function printCrawlProgress(event) {
|
|
241
790
|
const clear = isTty ? "\r\x1B[K" : "";
|
|
242
791
|
switch (event.type) {
|
|
@@ -269,16 +818,44 @@ function printCrawlProgress(event) {
|
|
|
269
818
|
}
|
|
270
819
|
}
|
|
271
820
|
program.name("okfy").description("Turn docs into agent memory with Open Knowledge Format and MCP.").version(readPackageVersion());
|
|
272
|
-
program.command("
|
|
821
|
+
var initCommand = program.command("init").argument("<name>", "Local source name").argument("<url>", "Docs URL to crawl").option("--client <client>", "Target client: claude-code, claude-desktop, cursor, codex, or generic", setupClient, parseSetupClient("generic"));
|
|
822
|
+
addSourceRegistrationOptions(initCommand).option("--probe-timeout <duration>", "MCP setup probe timeout", duration, 5).option("--json", "Print JSON output", false).action(async (name, url, options) => {
|
|
273
823
|
try {
|
|
274
|
-
const
|
|
275
|
-
|
|
276
|
-
|
|
277
|
-
|
|
278
|
-
|
|
279
|
-
|
|
280
|
-
|
|
281
|
-
|
|
824
|
+
const { manifest } = await registerWebsiteSource(name, url, options);
|
|
825
|
+
const report = await setupReportForRecord({
|
|
826
|
+
record: await registeredRecord(manifest.name),
|
|
827
|
+
client: options.client,
|
|
828
|
+
maxAge: options.maxAge,
|
|
829
|
+
probeTimeoutSeconds: options.probeTimeout
|
|
830
|
+
});
|
|
831
|
+
printSetupReport(report, options.json);
|
|
832
|
+
if (report.status === "failed") process.exitCode = 1;
|
|
833
|
+
} catch (error) {
|
|
834
|
+
if (options.json) printSetupReport(setupReportForInitFailure(name, options.client, error), true);
|
|
835
|
+
else console.error(pc.red(error?.message ?? "Init failed."));
|
|
836
|
+
process.exitCode = 1;
|
|
837
|
+
}
|
|
838
|
+
});
|
|
839
|
+
program.command("doctor").argument("<name>", "Registered source name").option("--client <client>", "Target client: claude-code, claude-desktop, cursor, codex, or generic", setupClient, parseSetupClient("generic")).option("--max-age <duration>", "Override freshness max age", duration).option("--probe-timeout <duration>", "MCP setup probe timeout", duration, 5).option("--json", "Print JSON output", false).action(async (name, options) => {
|
|
840
|
+
try {
|
|
841
|
+
const report = await setupReportForRecord({
|
|
842
|
+
record: await registeredRecord(name),
|
|
843
|
+
client: options.client,
|
|
844
|
+
maxAge: options.maxAge,
|
|
845
|
+
probeTimeoutSeconds: options.probeTimeout
|
|
846
|
+
});
|
|
847
|
+
printSetupReport(report, options.json);
|
|
848
|
+
if (report.status === "failed") process.exitCode = 1;
|
|
849
|
+
} catch (error) {
|
|
850
|
+
const report = setupReportForMissingSource(name, options.client, error);
|
|
851
|
+
printSetupReport(report, options.json);
|
|
852
|
+
process.exitCode = 1;
|
|
853
|
+
}
|
|
854
|
+
});
|
|
855
|
+
var addCommand = program.command("add").argument("<name>", "Local source name").argument("<url>", "Docs URL to crawl");
|
|
856
|
+
addSourceRegistrationOptions(addCommand).option("--json", "Print JSON output", false).action(async (name, url, options) => {
|
|
857
|
+
try {
|
|
858
|
+
const { manifest, result } = await registerWebsiteSource(name, url, options);
|
|
282
859
|
const bundlePath = resolveBundleDir(manifest);
|
|
283
860
|
const payload = {
|
|
284
861
|
name: manifest.name,
|
|
@@ -487,7 +1064,7 @@ program.command("serve").argument("<name-or-bundle>", "Registered source name or
|
|
|
487
1064
|
printStatus(`okfy serve: starting MCP stdio server "${options.name}"`);
|
|
488
1065
|
await serveMcpStdio({ bundleDir: target, name: options.name, maxResultChars: options.maxResultChars });
|
|
489
1066
|
printStatus("okfy serve: ready on stdio (stdout is reserved for MCP JSON-RPC)");
|
|
490
|
-
printStatus(
|
|
1067
|
+
printStatus(`okfy serve: tools ${MCP_TOOL_NAMES.join(", ")}`);
|
|
491
1068
|
return;
|
|
492
1069
|
}
|
|
493
1070
|
try {
|
|
@@ -524,7 +1101,7 @@ program.command("serve").argument("<name-or-bundle>", "Registered source name or
|
|
|
524
1101
|
}
|
|
525
1102
|
});
|
|
526
1103
|
printStatus("okfy serve: ready on stdio (stdout is reserved for MCP JSON-RPC)");
|
|
527
|
-
printStatus(
|
|
1104
|
+
printStatus(`okfy serve: tools ${MCP_TOOL_NAMES.join(", ")}`);
|
|
528
1105
|
} catch (error) {
|
|
529
1106
|
console.error(pc.red(error?.message ?? "Serve failed."));
|
|
530
1107
|
process.exitCode = 1;
|
|
@@ -532,8 +1109,8 @@ program.command("serve").argument("<name-or-bundle>", "Registered source name or
|
|
|
532
1109
|
});
|
|
533
1110
|
function resolveDemoBundle() {
|
|
534
1111
|
const relativeBundle = "examples/bundles/okfy-docs";
|
|
535
|
-
if (
|
|
536
|
-
return
|
|
1112
|
+
if (fs2.existsSync(relativeBundle)) return relativeBundle;
|
|
1113
|
+
return path2.join(packageRoot, relativeBundle);
|
|
537
1114
|
}
|
|
538
1115
|
program.command("demo").description("Run offline demo against committed example bundle").action(async () => {
|
|
539
1116
|
const bundle = resolveDemoBundle();
|
package/dist/index.d.ts
CHANGED
|
@@ -181,6 +181,7 @@ declare class BundleSearch {
|
|
|
181
181
|
|
|
182
182
|
type RefreshMode$2 = "off" | "stale-while-refresh" | "blocking";
|
|
183
183
|
type FreshnessStatus = "fresh" | "stale" | "missing" | "failed" | "refreshing";
|
|
184
|
+
declare const MCP_TOOL_NAMES: readonly ["search_concepts", "read_concept", "get_neighbors", "list_types", "list_tags", "bundle_summary"];
|
|
184
185
|
type SourceMetadata = {
|
|
185
186
|
name: string;
|
|
186
187
|
kind: string;
|
|
@@ -429,4 +430,4 @@ declare function readRefreshState(name: string, options?: SourceStoreOptions): P
|
|
|
429
430
|
declare function listSources(options?: SourceStoreOptions): Promise<SourceRecord[]>;
|
|
430
431
|
declare function removeSource(name: string, options?: SourceStoreOptions): Promise<void>;
|
|
431
432
|
|
|
432
|
-
export { BundleSearch, type BundleStats, type Concept, type ContentType, type CrawlOptions, type CrawlProgressEvent, type CrawlResult, type CrawlRunner, type FreshnessDecision, type FreshnessReason, type FreshnessState, type FreshnessStatus, type ImportOptions, type KnowledgeGraph, type NormalizedDocument, type RawDocument, type RefreshContext, type RefreshErrorDetails, type RefreshHooks, type RefreshMode$2 as RefreshMode, type RefreshResult$1 as RefreshResult, type RefreshSkipReason, type RefreshSourceManifest, type SearchResult, type ServeOptions, type SourceKind, type SourceManifest, type SourceMetadata, type SourceRecord, type RefreshMode as SourceRefreshMode, type RefreshResult as SourceRefreshResult, type RefreshState$1 as SourceRefreshState, type RefreshStatus as SourceRefreshStatus, type SourceStoreOptions, type RefreshState as StoredRefreshState, type ValidationIssue, type ValidationReport, type WriteBundleOptions, assertSafeForceOutDir, buildGraph, crawlWebsite, createMcpServer, descriptionFromMarkdown, evaluateFreshness, extractHeadings, extractInternalLinks, extractMarkdownLinks, hashBundleContents, importLocal, inferTags, inferType, inspectBundle, listSources, normalizeDocument, parseDurationSeconds, readBundle, readConceptFile, readRefreshState, readSourceManifest, refreshSource, removeSource, resolveBundleDir, resolveOkfyHome, resolveSourceDir, serveMcpStdio, validateBundle, validateSourceName, writeOkfBundle, writeRefreshState, writeSourceManifest };
|
|
433
|
+
export { BundleSearch, type BundleStats, type Concept, type ContentType, type CrawlOptions, type CrawlProgressEvent, type CrawlResult, type CrawlRunner, type FreshnessDecision, type FreshnessReason, type FreshnessState, type FreshnessStatus, type ImportOptions, type KnowledgeGraph, MCP_TOOL_NAMES, type NormalizedDocument, type RawDocument, type RefreshContext, type RefreshErrorDetails, type RefreshHooks, type RefreshMode$2 as RefreshMode, type RefreshResult$1 as RefreshResult, type RefreshSkipReason, type RefreshSourceManifest, type SearchResult, type ServeOptions, type SourceKind, type SourceManifest, type SourceMetadata, type SourceRecord, type RefreshMode as SourceRefreshMode, type RefreshResult as SourceRefreshResult, type RefreshState$1 as SourceRefreshState, type RefreshStatus as SourceRefreshStatus, type SourceStoreOptions, type RefreshState as StoredRefreshState, type ValidationIssue, type ValidationReport, type WriteBundleOptions, assertSafeForceOutDir, buildGraph, crawlWebsite, createMcpServer, descriptionFromMarkdown, evaluateFreshness, extractHeadings, extractInternalLinks, extractMarkdownLinks, hashBundleContents, importLocal, inferTags, inferType, inspectBundle, listSources, normalizeDocument, parseDurationSeconds, readBundle, readConceptFile, readRefreshState, readSourceManifest, refreshSource, removeSource, resolveBundleDir, resolveOkfyHome, resolveSourceDir, serveMcpStdio, validateBundle, validateSourceName, writeOkfBundle, writeRefreshState, writeSourceManifest };
|
package/dist/index.js
CHANGED
|
@@ -1,5 +1,6 @@
|
|
|
1
1
|
import {
|
|
2
2
|
BundleSearch,
|
|
3
|
+
MCP_TOOL_NAMES,
|
|
3
4
|
assertSafeForceOutDir,
|
|
4
5
|
buildGraph,
|
|
5
6
|
crawlWebsite,
|
|
@@ -32,9 +33,10 @@ import {
|
|
|
32
33
|
writeOkfBundle,
|
|
33
34
|
writeRefreshState,
|
|
34
35
|
writeSourceManifest
|
|
35
|
-
} from "./chunk-
|
|
36
|
+
} from "./chunk-7V2ZN6IS.js";
|
|
36
37
|
export {
|
|
37
38
|
BundleSearch,
|
|
39
|
+
MCP_TOOL_NAMES,
|
|
38
40
|
assertSafeForceOutDir,
|
|
39
41
|
buildGraph,
|
|
40
42
|
crawlWebsite,
|
package/docs/mcp-clients.md
CHANGED
|
@@ -3,10 +3,11 @@
|
|
|
3
3
|
okfy is meant to be launched by your agent as a local stdio MCP server. The default setup uses `npx -y okfy-ai`, so Claude, Codex, Cursor, or another MCP client can run okfy without a global install:
|
|
4
4
|
|
|
5
5
|
```bash
|
|
6
|
-
npx -y okfy-ai
|
|
7
|
-
npx -y okfy-ai serve stripe --mcp --auto-refresh
|
|
6
|
+
npx -y okfy-ai init stripe https://docs.stripe.com/checkout --client generic --max-pages 100 --max-depth 4
|
|
8
7
|
```
|
|
9
8
|
|
|
9
|
+
The generated launch command will use `npx -y okfy-ai serve stripe --mcp --auto-refresh`.
|
|
10
|
+
|
|
10
11
|
MCP stdio means the client starts okfy as a local subprocess, sends JSON-RPC on stdin, and reads JSON-RPC responses on stdout. okfy logs and refresh progress belong on stderr so the MCP protocol stays clean.
|
|
11
12
|
|
|
12
13
|
## Registered Source Workflow
|
|
@@ -17,11 +18,14 @@ Use registered sources for third-party docs sites that should stay fresh over ti
|
|
|
17
18
|
npx -y okfy-ai add stripe https://docs.stripe.com/checkout --max-pages 100 --max-depth 4
|
|
18
19
|
npx -y okfy-ai sources
|
|
19
20
|
npx -y okfy-ai check stripe
|
|
21
|
+
npx -y okfy-ai doctor stripe
|
|
20
22
|
npx -y okfy-ai update stripe
|
|
21
23
|
npx -y okfy-ai remove stripe
|
|
22
24
|
npx -y okfy-ai serve stripe --mcp --auto-refresh
|
|
23
25
|
```
|
|
24
26
|
|
|
27
|
+
If you want registration plus client-specific setup artifacts, use `npx -y okfy-ai init stripe https://docs.stripe.com/checkout --client generic --max-pages 100 --max-depth 4`.
|
|
28
|
+
|
|
25
29
|
By default, okfy stores sources in `~/.okfy`. Override that with `OKFY_HOME` when you want CI isolation, a project-local cache, or a disposable test home:
|
|
26
30
|
|
|
27
31
|
```text
|
|
@@ -39,6 +43,14 @@ $OKFY_HOME/
|
|
|
39
43
|
|
|
40
44
|
This is local-first. There is no OKFY cloud registry, account, central cache, hosted ranking, or cloud refresh worker. Refreshes run on your machine by rerunning the stored crawl configuration.
|
|
41
45
|
|
|
46
|
+
`init` is the setup shortcut over the registered-source workflow. It creates the source, validates the bundle, and prints client-specific config plus a first prompt without writing client config files by default.
|
|
47
|
+
|
|
48
|
+
`doctor` re-runs setup checks later. It verifies source existence, bundle validity, freshness, `npx` availability, generated command shape, MCP tool visibility, and JSON-RPC-clean stdout. Use it when the client cannot start okfy, the agent cannot see tools, or answers look stale:
|
|
49
|
+
|
|
50
|
+
```bash
|
|
51
|
+
npx -y okfy-ai doctor stripe --client codex
|
|
52
|
+
```
|
|
53
|
+
|
|
42
54
|
Default refresh mode is `stale-while-refresh`: if the cached bundle is stale, MCP tools keep serving the current bundle while okfy refreshes in the background. Use blocking mode when you want stale sources refreshed before search/read/list tool calls answer:
|
|
43
55
|
|
|
44
56
|
```bash
|
|
@@ -72,6 +84,7 @@ Direct bundle paths do not use source auto-refresh. Use `add` plus `serve <sourc
|
|
|
72
84
|
Add a registered source as a local stdio server:
|
|
73
85
|
|
|
74
86
|
```bash
|
|
87
|
+
npx -y okfy-ai init stripe https://docs.stripe.com/checkout --client claude-code
|
|
75
88
|
claude mcp add --transport stdio stripe-okf -- npx -y okfy-ai serve stripe --mcp --auto-refresh
|
|
76
89
|
claude mcp list
|
|
77
90
|
```
|
|
@@ -132,6 +145,7 @@ final answer with cited resource fields
|
|
|
132
145
|
|
|
133
146
|
Troubleshooting:
|
|
134
147
|
|
|
148
|
+
- Run `npx -y okfy-ai doctor stripe --client claude-code` for a setup report.
|
|
135
149
|
- `spawn npx ENOENT`: install Node.js >=20 and ensure `npx` is on `PATH`.
|
|
136
150
|
- Server pending: run `/mcp`; approve project-scoped `.mcp.json` if prompted.
|
|
137
151
|
- Unknown source name: run `npx -y okfy-ai sources` and confirm the source exists in the same `OKFY_HOME`.
|
|
@@ -143,6 +157,10 @@ Troubleshooting:
|
|
|
143
157
|
|
|
144
158
|
Claude Desktop and Cursor use MCP server JSON. Add this entry to `claude_desktop_config.json`, `.cursor/mcp.json`, or any client that accepts `mcpServers` JSON:
|
|
145
159
|
|
|
160
|
+
```bash
|
|
161
|
+
npx -y okfy-ai init stripe https://docs.stripe.com/checkout --client cursor
|
|
162
|
+
```
|
|
163
|
+
|
|
146
164
|
```json
|
|
147
165
|
{
|
|
148
166
|
"mcpServers": {
|
|
@@ -183,6 +201,7 @@ Use stripe-okf. Find concepts about MCP tools, read the relevant concept, then t
|
|
|
183
201
|
|
|
184
202
|
Troubleshooting:
|
|
185
203
|
|
|
204
|
+
- Run `npx -y okfy-ai doctor stripe --client cursor` for a setup report.
|
|
186
205
|
- Desktop cannot find `npx`: replace `"command": "npx"` with the full path from `which npx`.
|
|
187
206
|
- Server exits immediately: run the exact command in a terminal and fix source or bundle validation errors.
|
|
188
207
|
- No okfy tools visible: restart the client after config changes.
|
|
@@ -207,6 +226,10 @@ Trusted project config path:
|
|
|
207
226
|
|
|
208
227
|
Add:
|
|
209
228
|
|
|
229
|
+
```bash
|
|
230
|
+
npx -y okfy-ai init stripe https://docs.stripe.com/checkout --client codex
|
|
231
|
+
```
|
|
232
|
+
|
|
210
233
|
```toml
|
|
211
234
|
[mcp_servers.stripe_okf]
|
|
212
235
|
command = "npx"
|
|
@@ -266,6 +289,7 @@ final answer with citations
|
|
|
266
289
|
|
|
267
290
|
Troubleshooting:
|
|
268
291
|
|
|
292
|
+
- Run `npx -y okfy-ai doctor stripe --client codex` for a setup report.
|
|
269
293
|
- Config ignored: project `.codex/config.toml` loads only for trusted projects; use user config if unsure.
|
|
270
294
|
- Server startup timeout: increase `startup_timeout_sec` if first `npx` install or first source load is slow.
|
|
271
295
|
- Tool timeout: increase `tool_timeout_sec` for large bundles or blocking refresh mode.
|
|
@@ -277,6 +301,10 @@ Troubleshooting:
|
|
|
277
301
|
|
|
278
302
|
Use this JSON for clients that accept Claude-style `mcpServers` config:
|
|
279
303
|
|
|
304
|
+
```bash
|
|
305
|
+
npx -y okfy-ai init stripe https://docs.stripe.com/checkout --client generic
|
|
306
|
+
```
|
|
307
|
+
|
|
280
308
|
```json
|
|
281
309
|
{
|
|
282
310
|
"mcpServers": {
|
|
@@ -332,6 +360,7 @@ Use stripe-okf. Search for OKF bundle structure, read the most relevant concepts
|
|
|
332
360
|
|
|
333
361
|
Troubleshooting:
|
|
334
362
|
|
|
363
|
+
- Run `npx -y okfy-ai doctor stripe --client generic` for a setup report.
|
|
335
364
|
- stdout has logs: okfy must write only MCP JSON-RPC messages to stdout; logs belong on stderr.
|
|
336
365
|
- Client cannot start process: use absolute `command` path, and set `OKFY_HOME` when using a non-default source cache.
|
|
337
366
|
- `tools/list` empty: confirm `okfy serve` was started with `--mcp`.
|