@mfdaves/okf-mcp 0.5.0 → 0.5.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -4,7 +4,7 @@
4
4
 
5
5
  It consumes an OKF bundle directory of Markdown files with YAML frontmatter. An optional workspace mode can federate several bundles. Concepts are exposed through CLI commands and MCP resources and tools for validation, structured search, graph navigation, provenance inspection, and proposal-based authoring.
6
6
 
7
- The core intentionally has no database, embeddings, build step, or hosted-service dependency. It uses `js-yaml` for safe YAML, CommonMark for Markdown structure, and the official Model Context Protocol TypeScript SDK v2 for stdio MCP. Local root mode makes no network calls. Optional remote loading fetches public Markdown concepts and only their explicitly referenced inert assets from GitHub. Nothing in the v0.2 computation support executes code or attests a receipt.
7
+ The core intentionally has no database, embeddings, build step, or hosted-service dependency. It uses `js-yaml` for safe YAML, CommonMark for Markdown structure, MiniSearch for in-memory BM25+ text retrieval, and the official Model Context Protocol TypeScript SDK v2 for stdio MCP. Local root mode makes no network calls. Optional remote loading fetches public Markdown concepts and only their explicitly referenced inert assets from GitHub. Nothing in the v0.2 computation support executes code or attests a receipt.
8
8
 
9
9
  ## OKF v0.2 Support And Extensions
10
10
 
@@ -35,23 +35,23 @@ Node 22 or newer is required.
35
35
  Install from the GitHub release:
36
36
 
37
37
  ```bash
38
- git clone --branch v0.5.0 https://github.com/mfdaves/okf-mcp.git
38
+ git clone --branch v0.5.1 https://github.com/mfdaves/okf-mcp.git
39
39
  cd okf-mcp
40
40
  npm ci
41
41
  node bin/okf-mcp.js --root ./path/to/okf validate
42
42
  ```
43
43
 
44
- After version `0.5.0` is published on npm, pin it for reproducible use:
44
+ Pin the published version for reproducible use:
45
45
 
46
46
  ```bash
47
- npx -y @mfdaves/okf-mcp@0.5.0 --version
48
- npx -y @mfdaves/okf-mcp@0.5.0 --root ./path/to/okf validate
47
+ npx -y @mfdaves/okf-mcp@0.5.1 --version
48
+ npx -y @mfdaves/okf-mcp@0.5.1 --root ./path/to/okf validate
49
49
  ```
50
50
 
51
51
  For a persistent installation:
52
52
 
53
53
  ```bash
54
- npm install --global @mfdaves/okf-mcp@0.5.0
54
+ npm install --global @mfdaves/okf-mcp@0.5.1
55
55
 
56
56
  okf --version
57
57
  okf --root ./path/to/okf validate
@@ -95,7 +95,7 @@ okf --root okf/bundles/okf-mcp concept overview/okf-mcp
95
95
  Load the reference bundle directly from this release:
96
96
 
97
97
  ```bash
98
- okf --remote-bundle okf-mcp=https://github.com/mfdaves/okf-mcp/tree/v0.5.0/okf/bundles/okf-mcp --inspect
98
+ okf --remote-bundle okf-mcp=https://github.com/mfdaves/okf-mcp/tree/v0.5.1/okf/bundles/okf-mcp --inspect
99
99
  ```
100
100
 
101
101
  The `@mfdaves/okf-mcp` npm package includes both `okf.project.yaml` and the
@@ -171,7 +171,7 @@ Commands:
171
171
 
172
172
  ## MCP Client Config
173
173
 
174
- The npm-based examples below apply after version `0.5.0` is published there. A source checkout can invoke its executable `bin/okf-mcp.js` with the same arguments.
174
+ The npm-based examples below use the current published release. A source checkout can invoke its executable `bin/okf-mcp.js` with the same arguments.
175
175
 
176
176
  Example client configuration:
177
177
 
@@ -182,7 +182,7 @@ Example client configuration:
182
182
  "command": "npx",
183
183
  "args": [
184
184
  "-y",
185
- "@mfdaves/okf-mcp@0.5.0",
185
+ "@mfdaves/okf-mcp@0.5.1",
186
186
  "--root",
187
187
  "/absolute/path/to/okf",
188
188
  "mcp"
@@ -201,7 +201,7 @@ Project config mode, with read-only project helpers but without proposal mutatio
201
201
  "command": "npx",
202
202
  "args": [
203
203
  "-y",
204
- "@mfdaves/okf-mcp@0.5.0",
204
+ "@mfdaves/okf-mcp@0.5.1",
205
205
  "--project",
206
206
  "/absolute/path/to/repo/okf.project.yaml",
207
207
  "mcp"
@@ -502,7 +502,16 @@ Use `list_remote_bundles` to inspect what was loaded.
502
502
  - `offset`
503
503
 
504
504
  `list_concepts` also accepts a text `query` and applies it together with its
505
- listing filters. Tags and types are matched case-insensitively. Arbitrary
505
+ listing filters. Text search tokenizes case-insensitively and requires every
506
+ query term, regardless of order. BM25+ ranks title, type, tags, aliases,
507
+ description, path, and body matches; frontmatter remains available through
508
+ exact structured filters but is not copied into the text index. Scores are
509
+ relative within a result set and are not a stable cross-version scale.
510
+
511
+ Queries are bounded to 512 characters and 16 terms. Prefix expansion, fuzzy
512
+ matching, stemming, and stop-word removal are intentionally disabled so code
513
+ identifiers and domain terminology remain literal. Punctuation-only queries
514
+ return no matches. Tags and types are matched case-insensitively. Arbitrary
506
515
  frontmatter filters support exact scalar matching and array-contains
507
516
  matching. `relationType` selects concepts with an outgoing relation of that
508
517
  type.
@@ -518,6 +527,21 @@ Example:
518
527
  }
519
528
  ```
520
529
 
530
+ For local relevance and performance checks, run the non-packaged development
531
+ benchmark with a bundle root and an optional JSON array of `{ "query": "...",
532
+ "expected": "path/or/concept-id" }` judgments:
533
+
534
+ ```bash
535
+ node --expose-gc scripts/search-benchmark.js \
536
+ --root /path/to/okf \
537
+ --qrels /path/to/qrels.json
538
+ ```
539
+
540
+ It reports OKF and search-index build time, retained heap/RSS, p50/p95 query
541
+ latency, Recall@10, MRR@10, and representative rankings. Search indexes are
542
+ process-local and keyed to the parsed OKF index, so remote loads and accepted
543
+ proposals receive a fresh index automatically.
544
+
521
545
  ## Graph Behavior
522
546
 
523
547
  As an okf-mcp graph projection, Markdown links become `markdown_link` edges, extension `relations` become typed `relation` edges, and standard v0.2 path-valued fields become `resource`, `source`, `computation`, `executor`, and `attester` edges. Internal concept references resolve to canonical nodes; explicitly referenced non-Markdown files resolve to okf-mcp `okf-asset://` nodes; URLs and scope descriptors remain unfetched external or opaque leaves.
@@ -23,7 +23,7 @@ relations:
23
23
 
24
24
  The MCP interface groups tools by intent.
25
25
 
26
- Discovery tools list bundles, concepts, types, tags, custom relation types, standard edge kinds, and loaded remote bundles. `get_concept` accepts a portable Concept ID or compatibility locator and returns raw frontmatter and body plus normalized v0.2 signals, referenced-asset metadata, and declared Git-source availability. Search filters lifecycle, trust, freshness at a deterministic date, source presence, generator or verifier actors, runtime, and static attestation readiness.
26
+ Discovery tools list bundles, concepts, types, tags, custom relation types, standard edge kinds, and loaded remote bundles. `get_concept` accepts a portable Concept ID or compatibility locator and returns raw frontmatter and body plus normalized v0.2 signals, referenced-asset metadata, and declared Git-source availability. Search uses a process-local MiniSearch BM25+ index and filters lifecycle, trust, freshness at a deterministic date, source presence, generator or verifier actors, runtime, and static attestation readiness.
27
27
 
28
28
  Graph tools return bounded graphs, neighbors, subgraphs, paths, summaries, provenance traces, and rendered exports. They can filter semantic edge kinds and optionally include inert asset nodes or unfetched external leaves. Extensionless canonical, `.md`, and custom-id aliases are accepted for lookup.
29
29
 
@@ -39,4 +39,4 @@ Every tool supplies a purpose-specific description, descriptions for all input p
39
39
 
40
40
  The official MCP SDK validates tool arguments against the advertised schema without coercion. Unknown or disabled tools are rejected by SDK dispatch. Argument-schema failures and expected validation, storage, network, read-only, and proposal-conflict failures from enabled tools are returned as tool results with `isError: true`. A validation operation that successfully reports invalid OKF remains a successful tool result.
41
41
 
42
- `list_concepts` applies its optional text query together with its structured filters. Relation-type filtering selects concepts with an outgoing relation of the requested type.
42
+ `search_concepts` and the optional text query on `list_concepts` require every case-insensitive query term, regardless of order. They rank title, type, tags, aliases, description, path, and body while keeping arbitrary frontmatter as an exact structured filter. Queries are bounded to 512 characters and 16 terms; fuzzy, prefix, stemming, and stop-word expansion are not applied. Scores are relative retrieval scores rather than a stable public scale. Relation-type filtering selects concepts with an outgoing relation of the requested type.
@@ -21,7 +21,7 @@ relations:
21
21
 
22
22
  `okf-mcp` turns one OKF v0.2 root and its explicitly referenced inert assets into a searchable in-memory knowledge graph. It provides a CLI, SDK-backed stdio MCP server, pinned Git-source reads, optional workspace federation, generator plugins, remote GitHub bundles, and an HTTP authoring API.
23
23
 
24
- The runtime uses `js-yaml` and CommonMark and has no database, embedding service, computation executor, attester runtime, or build step. `--root` is the normal single-catalog interface and performs no network calls. The optional project manifest federates multiple roots and generator configuration; it is an okf-mcp extension rather than part of OKF v0.2.
24
+ The runtime uses `js-yaml`, CommonMark, and a process-local MiniSearch BM25+ text index. It has no database, embedding service, computation executor, attester runtime, or build step. `--root` is the normal single-catalog interface and performs no network calls. The optional project manifest federates multiple roots and generator configuration; it is an okf-mcp extension rather than part of OKF v0.2.
25
25
 
26
26
  The primary runtime entry is the [MCP stdio server](../runtime/mcp-server.md). Concept interpretation and graph construction belong to the [indexer](../runtime/indexer.md). Durable knowledge changes follow the [concept authoring workflow](../workflows/concept-authoring.md) and [authoring safety policy](../policies/authoring-safety.md).
27
27
 
@@ -33,6 +33,6 @@ Each indexed Markdown document is exposed as a `text/markdown` resource. The [MC
33
33
 
34
34
  Root mode is the normal single-catalog interface and can supply the local store used by the [authoring workflow](../workflows/concept-authoring.md). Project mode remains an optional federation/configuration extension. Proposal mutation tools remain disabled unless the server starts with `--authoring`; runtime remote loading remains disabled unless it starts with `--allow-remote-tool`.
35
35
 
36
- Remote GitHub bundles are fetched as Markdown and added to the in-memory index. They remain read only and never execute remote code. After an accepted local proposal, one reconstruction path rebuilds the index from configured local bundles, configured remote bundles, and runtime-loaded remote bundles, preserving remote concepts and relationships.
36
+ Remote GitHub bundles are fetched as Markdown and added to the in-memory index. They remain read only and never execute remote code. After an accepted local proposal, one reconstruction path rebuilds the index from configured local bundles, configured remote bundles, and runtime-loaded remote bundles, preserving remote concepts and relationships. The lexical search index is cached against that reconstructed OKF index, so the next query builds a fresh per-server BM25+ view without global or persisted state.
37
37
 
38
38
  This component is a stdio MCP transport. The separate [HTTP authoring API](../interfaces/http-authoring-api.md) does not implement MCP over HTTP.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@mfdaves/okf-mcp",
3
- "version": "0.5.0",
3
+ "version": "0.5.1",
4
4
  "description": "Local-first Open Knowledge Format validator, graph index, CLI, and MCP stdio server.",
5
5
  "homepage": "https://github.com/mfdaves/okf-mcp#readme",
6
6
  "repository": {
@@ -55,7 +55,8 @@
55
55
  "dependencies": {
56
56
  "@modelcontextprotocol/server": "2.0.0",
57
57
  "commonmark": "^0.31.2",
58
- "js-yaml": "^5.2.3"
58
+ "js-yaml": "^5.2.3",
59
+ "minisearch": "7.2.0"
59
60
  },
60
61
  "devDependencies": {
61
62
  "@modelcontextprotocol/client": "2.0.0"
package/server.json CHANGED
@@ -8,12 +8,12 @@
8
8
  "source": "github"
9
9
  },
10
10
  "websiteUrl": "https://github.com/mfdaves/okf-mcp#readme",
11
- "version": "0.5.0",
11
+ "version": "0.5.1",
12
12
  "packages": [
13
13
  {
14
14
  "registryType": "npm",
15
15
  "identifier": "@mfdaves/okf-mcp",
16
- "version": "0.5.0",
16
+ "version": "0.5.1",
17
17
  "transport": {
18
18
  "type": "stdio"
19
19
  },
package/src/cli.js CHANGED
@@ -448,6 +448,7 @@ function usage() {
448
448
  " --max-content-bytes <n> Bound asset reads to 1..1048576 bytes (default 65536).",
449
449
  " Search filters: --status, --trust-tier, --freshness, --as-of, --has-sources,",
450
450
  " --runtime, --attestation-ready, --frontmatter key=value.",
451
+ " Text search requires all terms and accepts at most 512 characters and 16 terms.",
451
452
  " --version, -v Print the package version.",
452
453
  " --help, -h Print this help.",
453
454
  " --debug Include stack traces in error output.",
package/src/mcp-server.js CHANGED
@@ -18,6 +18,7 @@ const {
18
18
  validateIndex,
19
19
  } = require("./indexer");
20
20
  const { searchConcepts } = require("./search");
21
+ const { MAX_QUERY_CHARACTERS, MAX_QUERY_TERMS } = require("./search-index");
21
22
  const { exportGraph, findPaths, getGraph, getNeighbors, getSubgraph, graphSummary } = require("./graph");
22
23
  const { fetchGitHubBundle, fetchRemoteBundles, sanitizeRemoteId } = require("./remote");
23
24
  const { ConceptAuthoringService } = require("./authoring");
@@ -138,7 +139,7 @@ const TOOL_DEFINITIONS = {
138
139
  "List compact OKF concept summaries with optional bundle, type, tag, and text filters.",
139
140
  READ_ONLY,
140
141
  {
141
- query: stringParameter("Optional text matched against concept metadata and content."),
142
+ query: stringParameter(`Optional all-term text query matched against concept metadata and content; maximum ${MAX_QUERY_TERMS} terms.`, { maxLength: MAX_QUERY_CHARACTERS }),
142
143
  bundle: nonEmptyStringParameter("Limit results to this bundle id."),
143
144
  type: nonEmptyStringParameter("Limit results to this exact concept type."),
144
145
  tag: nonEmptyStringParameter("Limit results to concepts containing this tag."),
@@ -168,7 +169,7 @@ const TOOL_DEFINITIONS = {
168
169
  "Search OKF concepts and return ranked summaries using text and structured filters.",
169
170
  READ_ONLY,
170
171
  {
171
- query: stringParameter("Text query matched against concept metadata and content."),
172
+ query: stringParameter(`All-term text query matched against concept metadata and content; maximum ${MAX_QUERY_TERMS} terms.`, { maxLength: MAX_QUERY_CHARACTERS }),
172
173
  bundle: nonEmptyStringParameter("Limit results to this bundle id."),
173
174
  types: stringArrayParameter("Limit results to any of these concept types.", { nonEmptyItems: true }),
174
175
  tagsAny: stringArrayParameter("Require at least one of these tags.", { nonEmptyItems: true }),
@@ -0,0 +1,83 @@
1
+ "use strict";
2
+
3
+ const MiniSearch = require("minisearch");
4
+
5
+ const MAX_QUERY_CHARACTERS = 512;
6
+ const MAX_QUERY_TERMS = 16;
7
+ const SEARCH_FIELDS = Object.freeze([
8
+ "title",
9
+ "type",
10
+ "tags",
11
+ "aliases",
12
+ "description",
13
+ "path",
14
+ "body",
15
+ ]);
16
+ const SEARCH_BOOSTS = Object.freeze({
17
+ title: 8,
18
+ type: 5,
19
+ tags: 4,
20
+ aliases: 4,
21
+ description: 3,
22
+ path: 2,
23
+ body: 1,
24
+ });
25
+ const SEARCH_INDEXES = new WeakMap();
26
+ const tokenize = MiniSearch.getDefault("tokenize");
27
+ const processTerm = MiniSearch.getDefault("processTerm");
28
+
29
+ function queryTerms(query) {
30
+ return tokenize(String(query || ""))
31
+ .map((term) => processTerm(term))
32
+ .filter((term) => typeof term === "string" && term.length > 0);
33
+ }
34
+
35
+ function normalizeSearchQuery(value) {
36
+ const query = String(value || "").trim();
37
+ if (Array.from(query).length > MAX_QUERY_CHARACTERS) {
38
+ throw new TypeError(`Search query must not exceed ${MAX_QUERY_CHARACTERS} characters.`);
39
+ }
40
+ const terms = queryTerms(query);
41
+ if (terms.length > MAX_QUERY_TERMS) {
42
+ throw new TypeError(`Search query must not exceed ${MAX_QUERY_TERMS} terms.`);
43
+ }
44
+ return { query, terms };
45
+ }
46
+
47
+ function searchDocument(doc) {
48
+ return {
49
+ id: doc.uri,
50
+ title: String(doc.title || ""),
51
+ type: String(doc.type || ""),
52
+ tags: (doc.tags || []).join(" "),
53
+ aliases: (doc.aliases || []).join(" "),
54
+ description: String(doc.description || ""),
55
+ path: String(doc.path || ""),
56
+ body: String(doc.body || ""),
57
+ };
58
+ }
59
+
60
+ function prepareSearchIndex(index) {
61
+ let searchIndex = SEARCH_INDEXES.get(index);
62
+ if (searchIndex) {
63
+ return searchIndex;
64
+ }
65
+ searchIndex = new MiniSearch({
66
+ fields: SEARCH_FIELDS,
67
+ idField: "id",
68
+ searchOptions: {
69
+ boost: SEARCH_BOOSTS,
70
+ combineWith: "AND",
71
+ },
72
+ });
73
+ searchIndex.addAll(index.concepts.map(searchDocument));
74
+ SEARCH_INDEXES.set(index, searchIndex);
75
+ return searchIndex;
76
+ }
77
+
78
+ module.exports = {
79
+ MAX_QUERY_CHARACTERS,
80
+ MAX_QUERY_TERMS,
81
+ normalizeSearchQuery,
82
+ prepareSearchIndex,
83
+ };
package/src/search.js CHANGED
@@ -3,6 +3,7 @@
3
3
  const { conceptSummary } = require("./indexer");
4
4
  const { normalizeV02Signals } = require("./v02");
5
5
  const { resolveConcept } = require("./indexer");
6
+ const { normalizeSearchQuery, prepareSearchIndex } = require("./search-index");
6
7
 
7
8
  function lower(value) {
8
9
  return String(value || "").toLowerCase();
@@ -34,22 +35,30 @@ function docText(doc) {
34
35
  doc.path,
35
36
  doc.tags.join(" "),
36
37
  doc.aliases.join(" "),
37
- JSON.stringify(doc.frontmatter),
38
38
  doc.body,
39
39
  ].join("\n");
40
40
  }
41
41
 
42
- function snippetFor(doc, query) {
43
- if (!query) {
42
+ function snippetFor(doc, terms) {
43
+ if (!terms.length) {
44
44
  return "";
45
45
  }
46
46
  const text = docText(doc).replace(/\s+/g, " ");
47
- const idx = lower(text).indexOf(lower(query));
47
+ const normalized = lower(text);
48
+ let idx = -1;
49
+ let matchedLength = 0;
50
+ terms.forEach((term) => {
51
+ const candidate = normalized.indexOf(lower(term));
52
+ if (candidate !== -1 && (idx === -1 || candidate < idx)) {
53
+ idx = candidate;
54
+ matchedLength = String(term).length;
55
+ }
56
+ });
48
57
  if (idx === -1) {
49
58
  return "";
50
59
  }
51
60
  const start = Math.max(0, idx - 50);
52
- return text.slice(start, idx + query.length + 80).trim();
61
+ return text.slice(start, idx + matchedLength + 80).trim();
53
62
  }
54
63
 
55
64
  function linkedSets(index) {
@@ -166,50 +175,34 @@ function applyFilters(index, options) {
166
175
  });
167
176
  }
168
177
 
169
- function scoreDoc(doc, query) {
170
- if (!query) {
171
- return 0;
172
- }
173
- const q = lower(query);
174
- let score = 0;
175
- if (lower(doc.title).includes(q)) {
176
- score += 8;
177
- }
178
- if (lower(doc.type).includes(q)) {
179
- score += 5;
180
- }
181
- if (doc.tags.some((tag) => lower(tag).includes(q))) {
182
- score += 4;
183
- }
184
- if (doc.aliases.some((alias) => lower(alias).includes(q))) {
185
- score += 4;
186
- }
187
- if (lower(doc.description).includes(q)) {
188
- score += 3;
189
- }
190
- if (lower(doc.path).includes(q)) {
191
- score += 2;
192
- }
193
- if (lower(doc.body).includes(q)) {
194
- score += 1;
195
- }
196
- return score;
178
+ function compareDocs(left, right) {
179
+ return left.path.localeCompare(right.path) || left.uri.localeCompare(right.uri);
197
180
  }
198
181
 
199
182
  function searchConcepts(index, options) {
200
183
  const config = options || {};
201
- const query = String(config.query || "").trim();
184
+ const { query, terms } = normalizeSearchQuery(config.query);
202
185
  const limit = Math.max(1, Math.min(Number(config.limit || 25), 250));
203
186
  const offset = Math.max(0, Number(config.offset || 0));
204
- let results = applyFilters(index, config).map((doc) => ({
205
- doc,
206
- score: scoreDoc(doc, query),
207
- }));
187
+ const filteredDocs = applyFilters(index, config);
188
+ let results;
208
189
  if (query) {
209
- results = results.filter((entry) => entry.score > 0);
210
- results.sort((a, b) => b.score - a.score || a.doc.path.localeCompare(b.doc.path));
190
+ if (!terms.length) {
191
+ results = [];
192
+ } else {
193
+ const docsByUri = new Map(filteredDocs.map((doc) => [doc.uri, doc]));
194
+ const allowedUris = new Set(docsByUri.keys());
195
+ results = prepareSearchIndex(index).search(query, {
196
+ filter: (result) => allowedUris.has(String(result.id)),
197
+ }).map((result) => ({
198
+ doc: docsByUri.get(String(result.id)),
199
+ score: result.score,
200
+ })).filter((entry) => Boolean(entry.doc));
201
+ results.sort((a, b) => b.score - a.score || compareDocs(a.doc, b.doc));
202
+ }
211
203
  } else {
212
- results.sort((a, b) => a.doc.path.localeCompare(b.doc.path));
204
+ results = filteredDocs.map((doc) => ({ doc, score: 0 }));
205
+ results.sort((a, b) => compareDocs(a.doc, b.doc));
213
206
  }
214
207
  const page = results.slice(offset, offset + limit).map((entry) => {
215
208
  const summary = conceptSummary(entry.doc);
@@ -225,7 +218,7 @@ function searchConcepts(index, options) {
225
218
  }
226
219
  return Object.assign(summary, {
227
220
  score: entry.score,
228
- snippet: snippetFor(entry.doc, query),
221
+ snippet: snippetFor(entry.doc, terms),
229
222
  });
230
223
  });
231
224
  return {